Linux 7.0-rc4 Required for the ds4422 series which is build upon; 5187e03b817c ("iio: dac: ds4424: reject -128 RAW value")
Unified · Split
| @@ -210,10 +210,16 @@ Daniel Borkmann <daniel@iogearbox.net> <daniel.borkmann@tik.ee.ethz.ch> · 209 unmodified lines | |||
| 210 | Daniel Borkmann <daniel@iogearbox.net> <dborkmann@redhat.com> | 210 | Daniel Borkmann <daniel@iogearbox.net> <dborkmann@redhat.com> |
| 211 | Daniel Borkmann <daniel@iogearbox.net> <dborkman@redhat.com> | 211 | Daniel Borkmann <daniel@iogearbox.net> <dborkman@redhat.com> |
| 212 | Daniel Borkmann <daniel@iogearbox.net> <dxchgb@gmail.com> | 212 | Daniel Borkmann <daniel@iogearbox.net> <dxchgb@gmail.com> |
| 213 | Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@linaro.org> | ||
| 214 | Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@free.fr> | ||
| 215 | Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@linexp.org> | ||
| 216 | Daniel Lezcano <daniel.lezcano@kernel.org> <dlezcano@fr.ibm.com> | ||
| 213 | Daniel Thompson <danielt@kernel.org> <daniel.thompson@linaro.org> | 217 | Daniel Thompson <danielt@kernel.org> <daniel.thompson@linaro.org> |
| 218 | Daniele Alessandrelli <daniele.alessandrelli@gmail.com> <daniele.alessandrelli@intel.com> | ||
| 214 | Danilo Krummrich <dakr@kernel.org> <dakr@redhat.com> | 219 | Danilo Krummrich <dakr@kernel.org> <dakr@redhat.com> |
| 215 | David Brownell <david-b@pacbell.net> | 220 | David Brownell <david-b@pacbell.net> |
| 216 | David Collins <quic_collinsd@quicinc.com> <collinsd@codeaurora.org> | 221 | David Collins <quic_collinsd@quicinc.com> <collinsd@codeaurora.org> |
| 222 | David Gow <david@davidgow.net> <davidgow@google.com> | ||
| 217 | David Heidelberg <david@ixit.cz> <d.okias@gmail.com> | 223 | David Heidelberg <david@ixit.cz> <d.okias@gmail.com> |
| 218 | David Hildenbrand <david@kernel.org> <david@redhat.com> | 224 | David Hildenbrand <david@kernel.org> <david@redhat.com> |
| 219 | David Rheinsberg <david@readahead.eu> <dh.herrmann@gmail.com> | 225 | David Rheinsberg <david@readahead.eu> <dh.herrmann@gmail.com> |
| @@ -348,6 +354,7 @@ Jarkko Sakkinen <jarkko@kernel.org> <jarkko.sakkinen@opinsys.com> · 128 unmodified lines | |||
| 348 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> | 354 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> |
| 349 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@nvidia.com> | 355 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@nvidia.com> |
| 350 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> | 356 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> |
| 357 | Jason Xing <kerneljasonxing@gmail.com> <kernelxing@tencent.com> | ||
| 351 | <javier@osg.samsung.com> <javier.martinez@collabora.co.uk> | 358 | <javier@osg.samsung.com> <javier.martinez@collabora.co.uk> |
| 352 | Javi Merino <javi.merino@kernel.org> <javi.merino@arm.com> | 359 | Javi Merino <javi.merino@kernel.org> <javi.merino@arm.com> |
| 353 | Jayachandran C <c.jayachandran@gmail.com> <jayachandranc@netlogicmicro.com> | 360 | Jayachandran C <c.jayachandran@gmail.com> <jayachandranc@netlogicmicro.com> |
| @@ -396,6 +403,7 @@ Jiri Slaby <jirislaby@kernel.org> <xslaby@fi.muni.cz> · 42 unmodified lines | |||
| 396 | Jisheng Zhang <jszhang@kernel.org> <jszhang@marvell.com> | 403 | Jisheng Zhang <jszhang@kernel.org> <jszhang@marvell.com> |
| 397 | Jisheng Zhang <jszhang@kernel.org> <Jisheng.Zhang@synaptics.com> | 404 | Jisheng Zhang <jszhang@kernel.org> <Jisheng.Zhang@synaptics.com> |
| 398 | Jishnu Prakash <quic_jprakash@quicinc.com> <jprakash@codeaurora.org> | 405 | Jishnu Prakash <quic_jprakash@quicinc.com> <jprakash@codeaurora.org> |
| 406 | Joe Damato <joe@dama.to> <jdamato@fastly.com> | ||
| 399 | Joel Granados <joel.granados@kernel.org> <j.granados@samsung.com> | 407 | Joel Granados <joel.granados@kernel.org> <j.granados@samsung.com> |
| 400 | Johan Hovold <johan@kernel.org> <jhovold@gmail.com> | 408 | Johan Hovold <johan@kernel.org> <jhovold@gmail.com> |
| 401 | Johan Hovold <johan@kernel.org> <johan@hovoldconsulting.com> | 409 | Johan Hovold <johan@kernel.org> <johan@hovoldconsulting.com> |
| @@ -490,7 +498,8 @@ Lior David <quic_liord@quicinc.com> <liord@codeaurora.org> · 88 unmodified lines | |||
| 490 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@linaro.org> | 498 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@linaro.org> |
| 491 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@intel.com> | 499 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@intel.com> |
| 492 | Lorenzo Pieralisi <lpieralisi@kernel.org> <lorenzo.pieralisi@arm.com> | 500 | Lorenzo Pieralisi <lpieralisi@kernel.org> <lorenzo.pieralisi@arm.com> |
| 493 | Lorenzo Stoakes <lorenzo.stoakes@oracle.com> <lstoakes@gmail.com> | 501 | Lorenzo Stoakes <ljs@kernel.org> <lstoakes@gmail.com> |
| 502 | Lorenzo Stoakes <ljs@kernel.org> <lorenzo.stoakes@oracle.com> | ||
| 494 | Luca Ceresoli <luca.ceresoli@bootlin.com> <luca@lucaceresoli.net> | 503 | Luca Ceresoli <luca.ceresoli@bootlin.com> <luca@lucaceresoli.net> |
| 495 | Luca Weiss <luca@lucaweiss.eu> <luca@z3ntu.xyz> | 504 | Luca Weiss <luca@lucaweiss.eu> <luca@z3ntu.xyz> |
| 496 | Lucas De Marchi <demarchi@kernel.org> <lucas.demarchi@intel.com> | 505 | Lucas De Marchi <demarchi@kernel.org> <lucas.demarchi@intel.com> |
| @@ -876,6 +885,7 @@ Vivien Didelot <vivien.didelot@gmail.com> <vivien.didelot@savoirfairelinux.com> · 379 unmodified lines | |||
| 876 | Vlad Dogaru <ddvlad@gmail.com> <vlad.dogaru@intel.com> | 885 | Vlad Dogaru <ddvlad@gmail.com> <vlad.dogaru@intel.com> |
| 877 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@parallels.com> | 886 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@parallels.com> |
| 878 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@virtuozzo.com> | 887 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@virtuozzo.com> |
| 888 | Vlastimil Babka <vbabka@kernel.org> <vbabka@suse.cz> | ||
| 879 | WangYuli <wangyuli@aosc.io> <wangyl5933@chinaunicom.cn> | 889 | WangYuli <wangyuli@aosc.io> <wangyl5933@chinaunicom.cn> |
| 880 | WangYuli <wangyuli@aosc.io> <wangyuli@deepin.org> | 890 | WangYuli <wangyuli@aosc.io> <wangyuli@deepin.org> |
| 881 | Weiwen Hu <huweiwen@linux.alibaba.com> <sehuww@mail.scut.edu.cn> | 891 | Weiwen Hu <huweiwen@linux.alibaba.com> <sehuww@mail.scut.edu.cn> |
| @@ -890,7 +900,8 @@ Yanteng Si <si.yanteng@linux.dev> <siyanteng@loongson.cn> · 8 unmodified lines | |||
| 890 | Ying Huang <huang.ying.caritas@gmail.com> <ying.huang@intel.com> | 900 | Ying Huang <huang.ying.caritas@gmail.com> <ying.huang@intel.com> |
| 891 | Yixun Lan <dlan@kernel.org> <dlan@gentoo.org> | 901 | Yixun Lan <dlan@kernel.org> <dlan@gentoo.org> |
| 892 | Yixun Lan <dlan@kernel.org> <yixun.lan@amlogic.com> | 902 | Yixun Lan <dlan@kernel.org> <yixun.lan@amlogic.com> |
| 893 | Yosry Ahmed <yosry.ahmed@linux.dev> <yosryahmed@google.com> | 903 | Yosry Ahmed <yosry@kernel.org> <yosryahmed@google.com> |
| 904 | Yosry Ahmed <yosry@kernel.org> <yosry.ahmed@linux.dev> | ||
| 894 | Yu-Chun Lin <eleanor.lin@realtek.com> <eleanor15x@gmail.com> | 905 | Yu-Chun Lin <eleanor.lin@realtek.com> <eleanor15x@gmail.com> |
| 895 | Yusuke Goda <goda.yusuke@renesas.com> | 906 | Yusuke Goda <goda.yusuke@renesas.com> |
| 896 | Zack Rusin <zack.rusin@broadcom.com> <zackr@vmware.com> | 907 | Zack Rusin <zack.rusin@broadcom.com> <zackr@vmware.com> |
| @@ -1242,6 +1242,10 @@ N: Veaceslav Falico · 1241 unmodified lines | |||
| 1242 | E: vfalico@gmail.com | 1242 | E: vfalico@gmail.com |
| 1243 | D: Co-maintainer and co-author of the network bonding driver. | 1243 | D: Co-maintainer and co-author of the network bonding driver. |
| 1244 | 1244 | ||
| 1245 | N: Thomas Falcon | ||
| 1246 | E: tlfalcon@linux.ibm.com | ||
| 1247 | D: Initial author of the IBM ibmvnic network driver | ||
| 1248 | |||
| 1245 | N: János Farkas | 1249 | N: János Farkas |
| 1246 | E: chexum@shadow.banki.hu | 1250 | E: chexum@shadow.banki.hu |
| 1247 | D: romfs, various (mostly networking) fixes | 1251 | D: romfs, various (mostly networking) fixes |
| @@ -2415,6 +2419,10 @@ S: Am Muehlenweg 38 · 1167 unmodified lines | |||
| 2415 | S: D53424 Remagen | 2419 | S: D53424 Remagen |
| 2416 | S: Germany | 2420 | S: Germany |
| 2417 | 2421 | ||
| 2422 | N: Jonathan Lemon | ||
| 2423 | E: jonathan.lemon@gmail.com | ||
| 2424 | D: OpenCompute PTP clock driver (ptp_ocp) | ||
| 2425 | |||
| 2418 | N: Colin Leroy | 2426 | N: Colin Leroy |
| 2419 | E: colin@colino.net | 2427 | E: colin@colino.net |
| 2420 | W: http://www.geekounet.org/ | 2428 | W: http://www.geekounet.org/ |
| @@ -151,11 +151,11 @@ Description: · 150 unmodified lines | |||
| 151 | The algorithm_params file is write-only and is used to setup | 151 | The algorithm_params file is write-only and is used to setup |
| 152 | compression algorithm parameters. | 152 | compression algorithm parameters. |
| 153 | 153 | ||
| 154 | What: /sys/block/zram<id>/writeback_compressed | 154 | What: /sys/block/zram<id>/compressed_writeback |
| 155 | Date: Decemeber 2025 | 155 | Date: Decemeber 2025 |
| 156 | Contact: Richard Chang <richardycc@google.com> | 156 | Contact: Richard Chang <richardycc@google.com> |
| 157 | Description: | 157 | Description: |
| 158 | The writeback_compressed device atrribute toggles compressed | 158 | The compressed_writeback device atrribute toggles compressed |
| 159 | writeback feature. | 159 | writeback feature. |
| 160 | 160 | ||
| 161 | What: /sys/block/zram<id>/writeback_batch_size | 161 | What: /sys/block/zram<id>/writeback_batch_size |
| @@ -1,4 +1,4 @@ | |||
| 1 | What: /sys/bus/platform/devices/INOU0000:XX/fn_lock_toggle_enable | 1 | What: /sys/bus/platform/devices/INOU0000:XX/fn_lock |
| 2 | Date: November 2025 | 2 | Date: November 2025 |
| 3 | KernelVersion: 6.19 | 3 | KernelVersion: 6.19 |
| 4 | Contact: Armin Wolf <W_Armin@gmx.de> | 4 | Contact: Armin Wolf <W_Armin@gmx.de> |
| @@ -8,15 +8,15 @@ Description: · 3 unmodified lines | |||
| 8 | 8 | ||
| 9 | Reading this file returns the current enable status of the FN lock functionality. | 9 | Reading this file returns the current enable status of the FN lock functionality. |
| 10 | 10 | ||
| 11 | What: /sys/bus/platform/devices/INOU0000:XX/super_key_toggle_enable | 11 | What: /sys/bus/platform/devices/INOU0000:XX/super_key_enable |
| 12 | Date: November 2025 | 12 | Date: November 2025 |
| 13 | KernelVersion: 6.19 | 13 | KernelVersion: 6.19 |
| 14 | Contact: Armin Wolf <W_Armin@gmx.de> | 14 | Contact: Armin Wolf <W_Armin@gmx.de> |
| 15 | Description: | 15 | Description: |
| 16 | Allows userspace applications to enable/disable the super key functionality | 16 | Allows userspace applications to enable/disable the super key of the integrated |
| 17 | of the integrated keyboard by writing "1"/"0" into this file. | 17 | keyboard by writing "1"/"0" into this file. |
| 18 | 18 | ||
| 19 | Reading this file returns the current enable status of the super key functionality. | 19 | Reading this file returns the current enable status of the super key. |
| 20 | 20 | ||
| 21 | What: /sys/bus/platform/devices/INOU0000:XX/touchpad_toggle_enable | 21 | What: /sys/bus/platform/devices/INOU0000:XX/touchpad_toggle_enable |
| 22 | Date: November 2025 | 22 | Date: November 2025 |
| @@ -216,7 +216,7 @@ writeback_limit WO specifies the maximum amount of write IO zram · 215 unmodified lines | |||
| 216 | writeback_limit_enable RW show and set writeback_limit feature | 216 | writeback_limit_enable RW show and set writeback_limit feature |
| 217 | writeback_batch_size RW show and set maximum number of in-flight | 217 | writeback_batch_size RW show and set maximum number of in-flight |
| 218 | writeback operations | 218 | writeback operations |
| 219 | writeback_compressed RW show and set compressed writeback feature | 219 | compressed_writeback RW show and set compressed writeback feature |
| 220 | comp_algorithm RW show and change the compression algorithm | 220 | comp_algorithm RW show and change the compression algorithm |
| 221 | algorithm_params WO setup compression algorithm parameters | 221 | algorithm_params WO setup compression algorithm parameters |
| 222 | compact WO trigger memory compaction | 222 | compact WO trigger memory compaction |
| @@ -439,11 +439,11 @@ budget in next setting is user's job. · 216 unmodified lines | |||
| 439 | By default zram stores written back pages in decompressed (raw) form, which | 439 | By default zram stores written back pages in decompressed (raw) form, which |
| 440 | means that writeback operation involves decompression of the page before | 440 | means that writeback operation involves decompression of the page before |
| 441 | writing it to the backing device. This behavior can be changed by enabling | 441 | writing it to the backing device. This behavior can be changed by enabling |
| 442 | `writeback_compressed` feature, which causes zram to write compressed pages | 442 | `compressed_writeback` feature, which causes zram to write compressed pages |
| 443 | to the backing device, thus avoiding decompression overhead. To enable | 443 | to the backing device, thus avoiding decompression overhead. To enable |
| 444 | this feature, execute:: | 444 | this feature, execute:: |
| 445 | 445 | ||
| 446 | $ echo yes > /sys/block/zramX/writeback_compressed | 446 | $ echo yes > /sys/block/zramX/compressed_writeback |
| 447 | 447 | ||
| 448 | Note that this feature should be configured before the `zramX` device is | 448 | Note that this feature should be configured before the `zramX` device is |
| 449 | initialized. | 449 | initialized. |
| @@ -74,6 +74,7 @@ · 73 unmodified lines | |||
| 74 | TPM TPM drivers are enabled. | 74 | TPM TPM drivers are enabled. |
| 75 | UMS USB Mass Storage support is enabled. | 75 | UMS USB Mass Storage support is enabled. |
| 76 | USB USB support is enabled. | 76 | USB USB support is enabled. |
| 77 | NVME NVMe support is enabled | ||
| 77 | USBHID USB Human Interface Device support is enabled. | 78 | USBHID USB Human Interface Device support is enabled. |
| 78 | V4L Video For Linux support is enabled. | 79 | V4L Video For Linux support is enabled. |
| 79 | VGA The VGA console has been enabled. | 80 | VGA The VGA console has been enabled. |
| @@ -4787,6 +4788,18 @@ Kernel parameters · 4707 unmodified lines | |||
| 4787 | This can be set from sysctl after boot. | 4788 | This can be set from sysctl after boot. |
| 4788 | See Documentation/admin-guide/sysctl/vm.rst for details. | 4789 | See Documentation/admin-guide/sysctl/vm.rst for details. |
| 4789 | 4790 | ||
| 4791 | nvme.quirks= [NVME] A list of quirk entries to augment the built-in | ||
| 4792 | nvme quirk list. List entries are separated by a | ||
| 4793 | '-' character. | ||
| 4794 | Each entry has the form VendorID:ProductID:quirk_names. | ||
| 4795 | The IDs are 4-digits hex numbers and quirk_names is a | ||
| 4796 | list of quirk names separated by commas. A quirk name | ||
| 4797 | can be prefixed by '^', meaning that the specified | ||
| 4798 | quirk must be disabled. | ||
| 4799 | |||
| 4800 | Example: | ||
| 4801 | nvme.quirks=7710:2267:bogus_nid,^identify_cns-9900:7711:broken_msi | ||
| 4802 | |||
| 4790 | ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver. | 4803 | ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver. |
| 4791 | See Documentation/core-api/debugging-via-ohci1394.rst for more | 4804 | See Documentation/core-api/debugging-via-ohci1394.rst for more |
| 4792 | info. | 4805 | info. |
| @@ -8183,6 +8196,9 @@ Kernel parameters · 3390 unmodified lines | |||
| 8183 | p = USB_QUIRK_SHORT_SET_ADDRESS_REQ_TIMEOUT | 8196 | p = USB_QUIRK_SHORT_SET_ADDRESS_REQ_TIMEOUT |
| 8184 | (Reduce timeout of the SET_ADDRESS | 8197 | (Reduce timeout of the SET_ADDRESS |
| 8185 | request from 5000 ms to 500 ms); | 8198 | request from 5000 ms to 500 ms); |
| 8199 | q = USB_QUIRK_FORCE_ONE_CONFIG (Device | ||
| 8200 | claims zero configurations, | ||
| 8201 | forcing to 1); | ||
| 8186 | Example: quirks=0781:5580:bk,0a5c:5834:gij | 8202 | Example: quirks=0781:5580:bk,0a5c:5834:gij |
| 8187 | 8203 | ||
| 8188 | usbhid.mousepoll= | 8204 | usbhid.mousepoll= |
| @@ -24,7 +24,7 @@ Keyboard settings · 23 unmodified lines | |||
| 24 | 24 | ||
| 25 | The ``uniwill-laptop`` driver allows the user to enable/disable: | 25 | The ``uniwill-laptop`` driver allows the user to enable/disable: |
| 26 | 26 | ||
| 27 | - the FN and super key lock functionality of the integrated keyboard | 27 | - the FN lock and super key of the integrated keyboard |
| 28 | - the touchpad toggle functionality of the integrated touchpad | 28 | - the touchpad toggle functionality of the integrated touchpad |
| 29 | 29 | ||
| 30 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. | 30 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. |
| @@ -594,6 +594,9 @@ Values: · 593 unmodified lines | |||
| 594 | their sockets will only be able to connect within their own | 594 | their sockets will only be able to connect within their own |
| 595 | namespace. | 595 | namespace. |
| 596 | 596 | ||
| 597 | The first write to ``child_ns_mode`` locks its value. Subsequent writes of the | ||
| 598 | same value succeed, but writing a different value returns ``-EBUSY``. | ||
| 599 | |||
| 597 | Changing ``child_ns_mode`` only affects namespaces created after the change; | 600 | Changing ``child_ns_mode`` only affects namespaces created after the change; |
| 598 | it does not modify the current namespace or any existing children. | 601 | it does not modify the current namespace or any existing children. |
| 599 | 602 | ||
| @@ -253,7 +253,6 @@ allOf: · 252 unmodified lines | |||
| 253 | enum: | 253 | enum: |
| 254 | # these platforms support 2 streams MST on some interfaces, | 254 | # these platforms support 2 streams MST on some interfaces, |
| 255 | # others are SST only | 255 | # others are SST only |
| 256 | - qcom,glymur-dp | ||
| 257 | - qcom,sc8280xp-dp | 256 | - qcom,sc8280xp-dp |
| 258 | - qcom,x1e80100-dp | 257 | - qcom,x1e80100-dp |
| 259 | then: | 258 | then: |
| @@ -310,6 +309,26 @@ allOf: · 50 unmodified lines | |||
| 310 | minItems: 6 | 309 | minItems: 6 |
| 311 | maxItems: 8 | 310 | maxItems: 8 |
| 312 | 311 | ||
| 312 | - if: | ||
| 313 | properties: | ||
| 314 | compatible: | ||
| 315 | contains: | ||
| 316 | enum: | ||
| 317 | # these platforms support 2 streams MST on some interfaces, | ||
| 318 | # others are SST only, but all controllers have 4 ports | ||
| 319 | - qcom,glymur-dp | ||
| 320 | then: | ||
| 321 | properties: | ||
| 322 | reg: | ||
| 323 | minItems: 9 | ||
| 324 | maxItems: 9 | ||
| 325 | clocks: | ||
| 326 | minItems: 5 | ||
| 327 | maxItems: 6 | ||
| 328 | clocks-names: | ||
| 329 | minItems: 5 | ||
| 330 | maxItems: 6 | ||
| 331 | |||
| 313 | unevaluatedProperties: false | 332 | unevaluatedProperties: false |
| 314 | 333 | ||
| 315 | examples: | 334 | examples: |
| @@ -176,13 +176,17 @@ examples: · 175 unmodified lines | |||
| 176 | }; | 176 | }; |
| 177 | }; | 177 | }; |
| 178 | 178 | ||
| 179 | displayport-controller@ae90000 { | 179 | displayport-controller@af54000 { |
| 180 | compatible = "qcom,glymur-dp"; | 180 | compatible = "qcom,glymur-dp"; |
| 181 | reg = <0xae90000 0x200>, | 181 | reg = <0xaf54000 0x200>, |
| 182 | <0xae90200 0x200>, | 182 | <0xaf54200 0x200>, |
| 183 | <0xae90400 0x600>, | 183 | <0xaf55000 0xc00>, |
| 184 | <0xae91000 0x400>, | 184 | <0xaf56000 0x400>, |
| 185 | <0xae91400 0x400>; | 185 | <0xaf57000 0x400>, |
| 186 | <0xaf58000 0x400>, | ||
| 187 | <0xaf59000 0x400>, | ||
| 188 | <0xaf5a000 0x600>, | ||
| 189 | <0xaf5b000 0x600>; | ||
| 186 | 190 | ||
| 187 | interrupt-parent = <&mdss>; | 191 | interrupt-parent = <&mdss>; |
| 188 | interrupts = <12>; | 192 | interrupts = <12>; |
| @@ -10,7 +10,7 @@ maintainers: · 9 unmodified lines | |||
| 10 | - Krzysztof Kozlowski <krzk@kernel.org> | 10 | - Krzysztof Kozlowski <krzk@kernel.org> |
| 11 | 11 | ||
| 12 | description: | 12 | description: |
| 13 | SM8650 MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like | 13 | SM8750 MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like |
| 14 | DPU display controller, DSI and DP interfaces etc. | 14 | DPU display controller, DSI and DP interfaces etc. |
| 15 | 15 | ||
| 16 | $ref: /schemas/display/msm/mdss-common.yaml# | 16 | $ref: /schemas/display/msm/mdss-common.yaml# |
| @@ -16,7 +16,6 @@ description: | · 15 unmodified lines | |||
| 16 | properties: | 16 | properties: |
| 17 | compatible: | 17 | compatible: |
| 18 | enum: | 18 | enum: |
| 19 | - kontron,sa67mcu-hwmon | ||
| 20 | - kontron,sl28cpld-fan | 19 | - kontron,sl28cpld-fan |
| 21 | 20 | ||
| 22 | reg: | 21 | reg: |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | |||
| 7 | title: Synopsys DesignWare APB I2C Controller | 7 | title: Synopsys DesignWare APB I2C Controller |
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Jarkko Nikula <jarkko.nikula@linux.intel.com> | 10 | - Mika Westerberg <mika.westerberg@linux.intel.com> |
| 11 | 11 | ||
| 12 | allOf: | 12 | allOf: |
| 13 | - $ref: /schemas/i2c/i2c-controller.yaml# | 13 | - $ref: /schemas/i2c/i2c-controller.yaml# |
| @@ -87,6 +87,7 @@ required: · 86 unmodified lines | |||
| 87 | 87 | ||
| 88 | allOf: | 88 | allOf: |
| 89 | - $ref: can-controller.yaml# | 89 | - $ref: can-controller.yaml# |
| 90 | - $ref: /schemas/memory-controllers/mc-peripheral-props.yaml | ||
| 90 | - if: | 91 | - if: |
| 91 | properties: | 92 | properties: |
| 92 | compatible: | 93 | compatible: |
| @@ -0,0 +1,93 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/powerpc/fsl/fsl,mpc83xx.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Freescale PowerQUICC II Pro (MPC83xx) platforms | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - J. Neuschäfer <j.ne@posteo.net> | ||
| 11 | |||
| 12 | properties: | ||
| 13 | $nodename: | ||
| 14 | const: '/' | ||
| 15 | compatible: | ||
| 16 | oneOf: | ||
| 17 | - description: MPC83xx Reference Design Boards | ||
| 18 | items: | ||
| 19 | - enum: | ||
| 20 | - fsl,mpc8308rdb | ||
| 21 | - fsl,mpc8315erdb | ||
| 22 | - fsl,mpc8360rdk | ||
| 23 | - fsl,mpc8377rdb | ||
| 24 | - fsl,mpc8377wlan | ||
| 25 | - fsl,mpc8378rdb | ||
| 26 | - fsl,mpc8379rdb | ||
| 27 | |||
| 28 | - description: MPC8313E Reference Design Board | ||
| 29 | items: | ||
| 30 | - const: MPC8313ERDB | ||
| 31 | - const: MPC831xRDB | ||
| 32 | - const: MPC83xxRDB | ||
| 33 | |||
| 34 | - description: MPC8323E Reference Design Board | ||
| 35 | items: | ||
| 36 | - const: MPC8323ERDB | ||
| 37 | - const: MPC832xRDB | ||
| 38 | - const: MPC83xxRDB | ||
| 39 | |||
| 40 | - description: MPC8349E-mITX(-GP) Reference Design Platform | ||
| 41 | items: | ||
| 42 | - enum: | ||
| 43 | - MPC8349EMITX | ||
| 44 | - MPC8349EMITXGP | ||
| 45 | - const: MPC834xMITX | ||
| 46 | - const: MPC83xxMITX | ||
| 47 | |||
| 48 | - description: Keymile KMETER1 board | ||
| 49 | const: keymile,KMETER1 | ||
| 50 | |||
| 51 | - description: MPC8308 P1M board | ||
| 52 | const: denx,mpc8308_p1m | ||
| 53 | |||
| 54 | patternProperties: | ||
| 55 | "^soc@.*$": | ||
| 56 | type: object | ||
| 57 | properties: | ||
| 58 | compatible: | ||
| 59 | oneOf: | ||
| 60 | - items: | ||
| 61 | - enum: | ||
| 62 | - fsl,mpc8315-immr | ||
| 63 | - fsl,mpc8308-immr | ||
| 64 | - const: simple-bus | ||
| 65 | - items: | ||
| 66 | - const: fsl,mpc8360-immr | ||
| 67 | - const: fsl,immr | ||
| 68 | - const: fsl,soc | ||
| 69 | - const: simple-bus | ||
| 70 | - const: simple-bus | ||
| 71 | |||
| 72 | additionalProperties: true | ||
| 73 | |||
| 74 | examples: | ||
| 75 | - | | ||
| 76 | / { | ||
| 77 | compatible = "fsl,mpc8315erdb"; | ||
| 78 | model = "MPC8315E-RDB"; | ||
| 79 | #address-cells = <1>; | ||
| 80 | #size-cells = <1>; | ||
| 81 | |||
| 82 | soc@e0000000 { | ||
| 83 | compatible = "fsl,mpc8315-immr", "simple-bus"; | ||
| 84 | reg = <0xe0000000 0x00000200>; | ||
| 85 | #address-cells = <1>; | ||
| 86 | #size-cells = <1>; | ||
| 87 | device_type = "soc"; | ||
| 88 | ranges = <0 0xe0000000 0x00100000>; | ||
| 89 | bus-frequency = <0>; | ||
| 90 | }; | ||
| 91 | }; | ||
| 92 | |||
| 93 | ... | ||
| @@ -287,7 +287,7 @@ examples: · 286 unmodified lines | |||
| 287 | regulator-max-microvolt = <1700000>; | 287 | regulator-max-microvolt = <1700000>; |
| 288 | }; | 288 | }; |
| 289 | mt6359_vrfck_1_ldo_reg: ldo_vrfck_1 { | 289 | mt6359_vrfck_1_ldo_reg: ldo_vrfck_1 { |
| 290 | regulator-name = "vrfck"; | 290 | regulator-name = "vrfck_1"; |
| 291 | regulator-min-microvolt = <1240000>; | 291 | regulator-min-microvolt = <1240000>; |
| 292 | regulator-max-microvolt = <1600000>; | 292 | regulator-max-microvolt = <1600000>; |
| 293 | }; | 293 | }; |
| @@ -309,7 +309,7 @@ examples: · 15 unmodified lines | |||
| 309 | regulator-max-microvolt = <3300000>; | 309 | regulator-max-microvolt = <3300000>; |
| 310 | }; | 310 | }; |
| 311 | mt6359_vemc_1_ldo_reg: ldo_vemc_1 { | 311 | mt6359_vemc_1_ldo_reg: ldo_vemc_1 { |
| 312 | regulator-name = "vemc"; | 312 | regulator-name = "vemc_1"; |
| 313 | regulator-min-microvolt = <2500000>; | 313 | regulator-min-microvolt = <2500000>; |
| 314 | regulator-max-microvolt = <3300000>; | 314 | regulator-max-microvolt = <3300000>; |
| 315 | }; | 315 | }; |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | |||
| 23 | enum: | 23 | enum: |
| 24 | - nvidia,tegra210-audio-graph-card | 24 | - nvidia,tegra210-audio-graph-card |
| 25 | - nvidia,tegra186-audio-graph-card | 25 | - nvidia,tegra186-audio-graph-card |
| 26 | - nvidia,tegra238-audio-graph-card | ||
| 26 | - nvidia,tegra264-audio-graph-card | 27 | - nvidia,tegra264-audio-graph-card |
| 27 | 28 | ||
| 28 | clocks: | 29 | clocks: |
| @@ -20,6 +20,7 @@ properties: · 19 unmodified lines | |||
| 20 | - renesas,r9a07g044-ssi # RZ/G2{L,LC} | 20 | - renesas,r9a07g044-ssi # RZ/G2{L,LC} |
| 21 | - renesas,r9a07g054-ssi # RZ/V2L | 21 | - renesas,r9a07g054-ssi # RZ/V2L |
| 22 | - renesas,r9a08g045-ssi # RZ/G3S | 22 | - renesas,r9a08g045-ssi # RZ/G3S |
| 23 | - renesas,r9a08g046-ssi # RZ/G3L | ||
| 23 | - const: renesas,rz-ssi | 24 | - const: renesas,rz-ssi |
| 24 | 25 | ||
| 25 | reg: | 26 | reg: |
| @@ -6,9 +6,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 5 unmodified lines | |||
| 6 | 6 | ||
| 7 | title: Allwinner A31 SPI Controller | 7 | title: Allwinner A31 SPI Controller |
| 8 | 8 | ||
| 9 | allOf: | ||
| 10 | - $ref: spi-controller.yaml | ||
| 11 | |||
| 12 | maintainers: | 9 | maintainers: |
| 13 | - Chen-Yu Tsai <wens@csie.org> | 10 | - Chen-Yu Tsai <wens@csie.org> |
| 14 | - Maxime Ripard <mripard@kernel.org> | 11 | - Maxime Ripard <mripard@kernel.org> |
| @@ -82,11 +79,11 @@ patternProperties: · 67 unmodified lines | |||
| 82 | 79 | ||
| 83 | spi-rx-bus-width: | 80 | spi-rx-bus-width: |
| 84 | items: | 81 | items: |
| 85 | - const: 1 | 82 | enum: [0, 1, 2, 4] |
| 86 | 83 | ||
| 87 | spi-tx-bus-width: | 84 | spi-tx-bus-width: |
| 88 | items: | 85 | items: |
| 89 | - const: 1 | 86 | enum: [0, 1, 2, 4] |
| 90 | 87 | ||
| 91 | required: | 88 | required: |
| 92 | - compatible | 89 | - compatible |
| @@ -95,6 +92,28 @@ required: · 2 unmodified lines | |||
| 95 | - clocks | 92 | - clocks |
| 96 | - clock-names | 93 | - clock-names |
| 97 | 94 | ||
| 95 | allOf: | ||
| 96 | - $ref: spi-controller.yaml | ||
| 97 | - if: | ||
| 98 | not: | ||
| 99 | properties: | ||
| 100 | compatible: | ||
| 101 | contains: | ||
| 102 | enum: | ||
| 103 | - allwinner,sun50i-r329-spi | ||
| 104 | - allwinner,sun55i-a523-spi | ||
| 105 | then: | ||
| 106 | patternProperties: | ||
| 107 | "^.*@[0-9a-f]+": | ||
| 108 | properties: | ||
| 109 | spi-rx-bus-width: | ||
| 110 | items: | ||
| 111 | enum: [0, 1] | ||
| 112 | |||
| 113 | spi-tx-bus-width: | ||
| 114 | items: | ||
| 115 | enum: [0, 1] | ||
| 116 | |||
| 98 | unevaluatedProperties: false | 117 | unevaluatedProperties: false |
| 99 | 118 | ||
| 100 | examples: | 119 | examples: |
| @@ -22,21 +22,6 @@ allOf: · 21 unmodified lines | |||
| 22 | properties: | 22 | properties: |
| 23 | reg: | 23 | reg: |
| 24 | minItems: 2 | 24 | minItems: 2 |
| 25 | - if: | ||
| 26 | properties: | ||
| 27 | compatible: | ||
| 28 | contains: | ||
| 29 | enum: | ||
| 30 | - baikal,bt1-sys-ssi | ||
| 31 | then: | ||
| 32 | properties: | ||
| 33 | mux-controls: | ||
| 34 | maxItems: 1 | ||
| 35 | required: | ||
| 36 | - mux-controls | ||
| 37 | else: | ||
| 38 | required: | ||
| 39 | - interrupts | ||
| 40 | - if: | 25 | - if: |
| 41 | properties: | 26 | properties: |
| 42 | compatible: | 27 | compatible: |
| @@ -75,10 +60,6 @@ properties: · 32 unmodified lines | |||
| 75 | const: intel,mountevans-imc-ssi | 60 | const: intel,mountevans-imc-ssi |
| 76 | - description: AMD Pensando Elba SoC SPI Controller | 61 | - description: AMD Pensando Elba SoC SPI Controller |
| 77 | const: amd,pensando-elba-spi | 62 | const: amd,pensando-elba-spi |
| 78 | - description: Baikal-T1 SPI Controller | ||
| 79 | const: baikal,bt1-ssi | ||
| 80 | - description: Baikal-T1 System Boot SPI Controller | ||
| 81 | const: baikal,bt1-sys-ssi | ||
| 82 | - description: Canaan Kendryte K210 SoS SPI Controller | 63 | - description: Canaan Kendryte K210 SoS SPI Controller |
| 83 | const: canaan,k210-spi | 64 | const: canaan,k210-spi |
| 84 | - description: Renesas RZ/N1 SPI Controller | 65 | - description: Renesas RZ/N1 SPI Controller |
| @@ -170,6 +151,7 @@ required: · 85 unmodified lines | |||
| 170 | - "#address-cells" | 151 | - "#address-cells" |
| 171 | - "#size-cells" | 152 | - "#size-cells" |
| 172 | - clocks | 153 | - clocks |
| 154 | - interrupts | ||
| 173 | 155 | ||
| 174 | examples: | 156 | examples: |
| 175 | - | | 157 | - | |
| @@ -190,15 +172,4 @@ examples: · 14 unmodified lines | |||
| 190 | rx-sample-delay-ns = <7>; | 172 | rx-sample-delay-ns = <7>; |
| 191 | }; | 173 | }; |
| 192 | }; | 174 | }; |
| 193 | - | | ||
| 194 | spi@1f040100 { | ||
| 195 | compatible = "baikal,bt1-sys-ssi"; | ||
| 196 | reg = <0x1f040100 0x900>, | ||
| 197 | <0x1c000000 0x1000000>; | ||
| 198 | #address-cells = <1>; | ||
| 199 | #size-cells = <0>; | ||
| 200 | mux-controls = <&boot_mux>; | ||
| 201 | clocks = <&ccu_sys>; | ||
| 202 | clock-names = "ssi_clk"; | ||
| 203 | }; | ||
| 204 | ... | 175 | ... |
| @@ -57,7 +57,7 @@ Supported chips: · 56 unmodified lines | |||
| 57 | - https://ww1.microchip.com/downloads/en/DeviceDoc/EMC1438%20DS%20Rev.%201.0%20(04-29-10).pdf | 57 | - https://ww1.microchip.com/downloads/en/DeviceDoc/EMC1438%20DS%20Rev.%201.0%20(04-29-10).pdf |
| 58 | 58 | ||
| 59 | Author: | 59 | Author: |
| 60 | Kalhan Trisal <kalhan.trisal@intel.com | 60 | Kalhan Trisal <kalhan.trisal@intel.com> |
| 61 | 61 | ||
| 62 | 62 | ||
| 63 | Description | 63 | Description |
| @@ -220,7 +220,6 @@ Hardware Monitoring Kernel Drivers · 219 unmodified lines | |||
| 220 | q54sj108a2 | 220 | q54sj108a2 |
| 221 | qnap-mcu-hwmon | 221 | qnap-mcu-hwmon |
| 222 | raspberrypi-hwmon | 222 | raspberrypi-hwmon |
| 223 | sa67 | ||
| 224 | sbrmi | 223 | sbrmi |
| 225 | sbtsi_temp | 224 | sbtsi_temp |
| 226 | sch5627 | 225 | sch5627 |
| @@ -1,41 +0,0 @@ | |||
| 1 | .. SPDX-License-Identifier: GPL-2.0-only | ||
| 2 | |||
| 3 | Kernel driver sa67mcu | ||
| 4 | ===================== | ||
| 5 | |||
| 6 | Supported chips: | ||
| 7 | |||
| 8 | * Kontron sa67mcu | ||
| 9 | |||
| 10 | Prefix: 'sa67mcu' | ||
| 11 | |||
| 12 | Datasheet: not available | ||
| 13 | |||
| 14 | Authors: Michael Walle <mwalle@kernel.org> | ||
| 15 | |||
| 16 | Description | ||
| 17 | ----------- | ||
| 18 | |||
| 19 | The sa67mcu is a board management controller which also exposes a hardware | ||
| 20 | monitoring controller. | ||
| 21 | |||
| 22 | The controller has two voltage and one temperature sensor. The values are | ||
| 23 | hold in two 8 bit registers to form one 16 bit value. Reading the lower byte | ||
| 24 | will also capture the high byte to make the access atomic. The unit of the | ||
| 25 | volatge sensors are 1mV and the unit of the temperature sensor is 0.1degC. | ||
| 26 | |||
| 27 | Sysfs entries | ||
| 28 | ------------- | ||
| 29 | |||
| 30 | The following attributes are supported. | ||
| 31 | |||
| 32 | ======================= ======================================================== | ||
| 33 | in0_label "VDDIN" | ||
| 34 | in0_input Measured VDDIN voltage. | ||
| 35 | |||
| 36 | in1_label "VDD_RTC" | ||
| 37 | in1_input Measured VDD_RTC voltage. | ||
| 38 | |||
| 39 | temp1_input MCU temperature. Roughly the board temperature. | ||
| 40 | ======================= ======================================================== | ||
| 41 | |||
| @@ -152,7 +152,7 @@ operations: · 151 unmodified lines | |||
| 152 | - compound-ops | 152 | - compound-ops |
| 153 | - | 153 | - |
| 154 | name: threads-set | 154 | name: threads-set |
| 155 | doc: set the number of running threads | 155 | doc: set the maximum number of running threads |
| 156 | attribute-set: server | 156 | attribute-set: server |
| 157 | flags: [admin-perm] | 157 | flags: [admin-perm] |
| 158 | do: | 158 | do: |
| @@ -165,7 +165,7 @@ operations: · 6 unmodified lines | |||
| 165 | - min-threads | 165 | - min-threads |
| 166 | - | 166 | - |
| 167 | name: threads-get | 167 | name: threads-get |
| 168 | doc: get the number of running threads | 168 | doc: get the maximum number of running threads |
| 169 | attribute-set: server | 169 | attribute-set: server |
| 170 | do: | 170 | do: |
| 171 | reply: | 171 | reply: |
| @@ -43,7 +43,6 @@ options should be enabled to use sched_ext: · 42 unmodified lines | |||
| 43 | CONFIG_DEBUG_INFO_BTF=y | 43 | CONFIG_DEBUG_INFO_BTF=y |
| 44 | CONFIG_BPF_JIT_ALWAYS_ON=y | 44 | CONFIG_BPF_JIT_ALWAYS_ON=y |
| 45 | CONFIG_BPF_JIT_DEFAULT_ON=y | 45 | CONFIG_BPF_JIT_DEFAULT_ON=y |
| 46 | CONFIG_PAHOLE_HAS_BTF_TAG=y | ||
| 47 | 46 | ||
| 48 | sched_ext is used only when the BPF scheduler is loaded and running. | 47 | sched_ext is used only when the BPF scheduler is loaded and running. |
| 49 | 48 | ||
| @@ -58,7 +57,8 @@ in ``ops->flags``, all ``SCHED_NORMAL``, ``SCHED_BATCH``, ``SCHED_IDLE``, and · 8 unmodified lines | |||
| 58 | However, when the BPF scheduler is loaded and ``SCX_OPS_SWITCH_PARTIAL`` is | 57 | However, when the BPF scheduler is loaded and ``SCX_OPS_SWITCH_PARTIAL`` is |
| 59 | set in ``ops->flags``, only tasks with the ``SCHED_EXT`` policy are scheduled | 58 | set in ``ops->flags``, only tasks with the ``SCHED_EXT`` policy are scheduled |
| 60 | by sched_ext, while tasks with ``SCHED_NORMAL``, ``SCHED_BATCH`` and | 59 | by sched_ext, while tasks with ``SCHED_NORMAL``, ``SCHED_BATCH`` and |
| 61 | ``SCHED_IDLE`` policies are scheduled by the fair-class scheduler. | 60 | ``SCHED_IDLE`` policies are scheduled by the fair-class scheduler which has |
| 61 | higher sched_class precedence than ``SCHED_EXT``. | ||
| 62 | 62 | ||
| 63 | Terminating the sched_ext scheduler program, triggering `SysRq-S`, or | 63 | Terminating the sched_ext scheduler program, triggering `SysRq-S`, or |
| 64 | detection of any internal error including stalled runnable tasks aborts the | 64 | detection of any internal error including stalled runnable tasks aborts the |
| @@ -345,6 +345,8 @@ Where to Look · 280 unmodified lines | |||
| 345 | The functions prefixed with ``scx_bpf_`` can be called from the BPF | 345 | The functions prefixed with ``scx_bpf_`` can be called from the BPF |
| 346 | scheduler. | 346 | scheduler. |
| 347 | 347 | ||
| 348 | * ``kernel/sched/ext_idle.c`` contains the built-in idle CPU selection policy. | ||
| 349 | |||
| 348 | * ``tools/sched_ext/`` hosts example BPF scheduler implementations. | 350 | * ``tools/sched_ext/`` hosts example BPF scheduler implementations. |
| 349 | 351 | ||
| 350 | * ``scx_simple[.bpf].c``: Minimal global FIFO scheduler example using a | 352 | * ``scx_simple[.bpf].c``: Minimal global FIFO scheduler example using a |
| @@ -353,13 +355,35 @@ Where to Look · 2 unmodified lines | |||
| 353 | * ``scx_qmap[.bpf].c``: A multi-level FIFO scheduler supporting five | 355 | * ``scx_qmap[.bpf].c``: A multi-level FIFO scheduler supporting five |
| 354 | levels of priority implemented with ``BPF_MAP_TYPE_QUEUE``. | 356 | levels of priority implemented with ``BPF_MAP_TYPE_QUEUE``. |
| 355 | 357 | ||
| 358 | * ``scx_central[.bpf].c``: A central FIFO scheduler where all scheduling | ||
| 359 | decisions are made on one CPU, demonstrating ``LOCAL_ON`` dispatching, | ||
| 360 | tickless operation, and kthread preemption. | ||
| 361 | |||
| 362 | * ``scx_cpu0[.bpf].c``: A scheduler that queues all tasks to a shared DSQ | ||
| 363 | and only dispatches them on CPU0 in FIFO order. Useful for testing bypass | ||
| 364 | behavior. | ||
| 365 | |||
| 366 | * ``scx_flatcg[.bpf].c``: A flattened cgroup hierarchy scheduler | ||
| 367 | implementing hierarchical weight-based cgroup CPU control by compounding | ||
| 368 | each cgroup's share at every level into a single flat scheduling layer. | ||
| 369 | |||
| 370 | * ``scx_pair[.bpf].c``: A core-scheduling example that always makes | ||
| 371 | sibling CPU pairs execute tasks from the same CPU cgroup. | ||
| 372 | |||
| 373 | * ``scx_sdt[.bpf].c``: A variation of ``scx_simple`` demonstrating BPF | ||
| 374 | arena memory management for per-task data. | ||
| 375 | |||
| 376 | * ``scx_userland[.bpf].c``: A minimal scheduler demonstrating user space | ||
| 377 | scheduling. Tasks with CPU affinity are direct-dispatched in FIFO order; | ||
| 378 | all others are scheduled in user space by a simple vruntime scheduler. | ||
| 379 | |||
| 356 | ABI Instability | 380 | ABI Instability |
| 357 | =============== | 381 | =============== |
| 358 | 382 | ||
| 359 | The APIs provided by sched_ext to BPF schedulers programs have no stability | 383 | The APIs provided by sched_ext to BPF schedulers programs have no stability |
| 360 | guarantees. This includes the ops table callbacks and constants defined in | 384 | guarantees. This includes the ops table callbacks and constants defined in |
| 361 | ``include/linux/sched/ext.h``, as well as the ``scx_bpf_`` kfuncs defined in | 385 | ``include/linux/sched/ext.h``, as well as the ``scx_bpf_`` kfuncs defined in |
| 362 | ``kernel/sched/ext.c``. | 386 | ``kernel/sched/ext.c`` and ``kernel/sched/ext_idle.c``. |
| 363 | 387 | ||
| 364 | While we will attempt to provide a relatively stable API surface when | 388 | While we will attempt to provide a relatively stable API surface when |
| 365 | possible, they are subject to change without warning between kernel | 389 | possible, they are subject to change without warning between kernel |
| @@ -2372,6 +2372,10 @@ quirk_flags · 2371 unmodified lines | |||
| 2372 | audible volume | 2372 | audible volume |
| 2373 | * bit 25: ``mixer_capture_min_mute`` | 2373 | * bit 25: ``mixer_capture_min_mute`` |
| 2374 | Similar to bit 24 but for capture streams | 2374 | Similar to bit 24 but for capture streams |
| 2375 | * bit 26: ``skip_iface_setup`` | ||
| 2376 | Skip the probe-time interface setup (usb_set_interface, | ||
| 2377 | init_pitch, init_sample_rate); redundant with | ||
| 2378 | snd_usb_endpoint_prepare() at stream-open time | ||
| 2375 | 2379 | ||
| 2376 | This module supports multiple devices, autoprobe and hotplugging. | 2380 | This module supports multiple devices, autoprobe and hotplugging. |
| 2377 | 2381 | ||
| @@ -1396,7 +1396,10 @@ or its flags may be modified, but it may not be resized. · 1395 unmodified lines | |||
| 1396 | Memory for the region is taken starting at the address denoted by the | 1396 | Memory for the region is taken starting at the address denoted by the |
| 1397 | field userspace_addr, which must point at user addressable memory for | 1397 | field userspace_addr, which must point at user addressable memory for |
| 1398 | the entire memory slot size. Any object may back this memory, including | 1398 | the entire memory slot size. Any object may back this memory, including |
| 1399 | anonymous memory, ordinary files, and hugetlbfs. | 1399 | anonymous memory, ordinary files, and hugetlbfs. Changes in the backing |
| 1400 | of the memory region are automatically reflected into the guest. | ||
| 1401 | For example, an mmap() that affects the region will be made visible | ||
| 1402 | immediately. Another example is madvise(MADV_DROP). | ||
| 1400 | 1403 | ||
| 1401 | On architectures that support a form of address tagging, userspace_addr must | 1404 | On architectures that support a form of address tagging, userspace_addr must |
| 1402 | be an untagged address. | 1405 | be an untagged address. |
| @@ -1412,11 +1415,6 @@ use it. The latter can be set, if KVM_CAP_READONLY_MEM capability allows it, · 9 unmodified lines | |||
| 1412 | to make a new slot read-only. In this case, writes to this memory will be | 1415 | to make a new slot read-only. In this case, writes to this memory will be |
| 1413 | posted to userspace as KVM_EXIT_MMIO exits. | 1416 | posted to userspace as KVM_EXIT_MMIO exits. |
| 1414 | 1417 | ||
| 1415 | When the KVM_CAP_SYNC_MMU capability is available, changes in the backing of | ||
| 1416 | the memory region are automatically reflected into the guest. For example, an | ||
| 1417 | mmap() that affects the region will be made visible immediately. Another | ||
| 1418 | example is madvise(MADV_DROP). | ||
| 1419 | |||
| 1420 | For TDX guest, deleting/moving memory region loses guest memory contents. | 1418 | For TDX guest, deleting/moving memory region loses guest memory contents. |
| 1421 | Read only region isn't supported. Only as-id 0 is supported. | 1419 | Read only region isn't supported. Only as-id 0 is supported. |
| 1422 | 1420 | ||
| @@ -8437,115 +8435,123 @@ KVM_CHECK_EXTENSION. · 7014 unmodified lines | |||
| 8437 | 8435 | ||
| 8438 | The valid bits in cap.args[0] are: | 8436 | The valid bits in cap.args[0] are: |
| 8439 | 8437 | ||
| 8440 | =================================== ============================================ | 8438 | ======================================== ================================================ |
| 8441 | KVM_X86_QUIRK_LINT0_REENABLED By default, the reset value for the LVT | 8439 | KVM_X86_QUIRK_LINT0_REENABLED By default, the reset value for the LVT |
| 8442 | LINT0 register is 0x700 (APIC_MODE_EXTINT). | 8440 | LINT0 register is 0x700 (APIC_MODE_EXTINT). |
| 8443 | When this quirk is disabled, the reset value | 8441 | When this quirk is disabled, the reset value |
| 8444 | is 0x10000 (APIC_LVT_MASKED). | 8442 | is 0x10000 (APIC_LVT_MASKED). |
| 8445 | 8443 | ||
| 8446 | KVM_X86_QUIRK_CD_NW_CLEARED By default, KVM clears CR0.CD and CR0.NW on | 8444 | KVM_X86_QUIRK_CD_NW_CLEARED By default, KVM clears CR0.CD and CR0.NW on |
| 8447 | AMD CPUs to workaround buggy guest firmware | 8445 | AMD CPUs to workaround buggy guest firmware |
| 8448 | that runs in perpetuity with CR0.CD, i.e. | 8446 | that runs in perpetuity with CR0.CD, i.e. |
| 8449 | with caches in "no fill" mode. | 8447 | with caches in "no fill" mode. |
| 8450 | 8448 | ||
| 8451 | When this quirk is disabled, KVM does not | 8449 | When this quirk is disabled, KVM does not |
| 8452 | change the value of CR0.CD and CR0.NW. | 8450 | change the value of CR0.CD and CR0.NW. |
| 8453 | 8451 | ||
| 8454 | KVM_X86_QUIRK_LAPIC_MMIO_HOLE By default, the MMIO LAPIC interface is | 8452 | KVM_X86_QUIRK_LAPIC_MMIO_HOLE By default, the MMIO LAPIC interface is |
| 8455 | available even when configured for x2APIC | 8453 | available even when configured for x2APIC |
| 8456 | mode. When this quirk is disabled, KVM | 8454 | mode. When this quirk is disabled, KVM |
| 8457 | disables the MMIO LAPIC interface if the | 8455 | disables the MMIO LAPIC interface if the |
| 8458 | LAPIC is in x2APIC mode. | 8456 | LAPIC is in x2APIC mode. |
| 8459 | 8457 | ||
| 8460 | KVM_X86_QUIRK_OUT_7E_INC_RIP By default, KVM pre-increments %rip before | 8458 | KVM_X86_QUIRK_OUT_7E_INC_RIP By default, KVM pre-increments %rip before |
| 8461 | exiting to userspace for an OUT instruction | 8459 | exiting to userspace for an OUT instruction |
| 8462 | to port 0x7e. When this quirk is disabled, | 8460 | to port 0x7e. When this quirk is disabled, |
| 8463 | KVM does not pre-increment %rip before | 8461 | KVM does not pre-increment %rip before |
| 8464 | exiting to userspace. | 8462 | exiting to userspace. |
| 8465 | 8463 | ||
| 8466 | KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT When this quirk is disabled, KVM sets | 8464 | KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT When this quirk is disabled, KVM sets |
| 8467 | CPUID.01H:ECX[bit 3] (MONITOR/MWAIT) if | 8465 | CPUID.01H:ECX[bit 3] (MONITOR/MWAIT) if |
| 8468 | IA32_MISC_ENABLE[bit 18] (MWAIT) is set. | 8466 | IA32_MISC_ENABLE[bit 18] (MWAIT) is set. |
| 8469 | Additionally, when this quirk is disabled, | 8467 | Additionally, when this quirk is disabled, |
| 8470 | KVM clears CPUID.01H:ECX[bit 3] if | 8468 | KVM clears CPUID.01H:ECX[bit 3] if |
| 8471 | IA32_MISC_ENABLE[bit 18] is cleared. | 8469 | IA32_MISC_ENABLE[bit 18] is cleared. |
| 8472 | 8470 | ||
| 8473 | KVM_X86_QUIRK_FIX_HYPERCALL_INSN By default, KVM rewrites guest | 8471 | KVM_X86_QUIRK_FIX_HYPERCALL_INSN By default, KVM rewrites guest |
| 8474 | VMMCALL/VMCALL instructions to match the | 8472 | VMMCALL/VMCALL instructions to match the |
| 8475 | vendor's hypercall instruction for the | 8473 | vendor's hypercall instruction for the |
| 8476 | system. When this quirk is disabled, KVM | 8474 | system. When this quirk is disabled, KVM |
| 8477 | will no longer rewrite invalid guest | 8475 | will no longer rewrite invalid guest |
| 8478 | hypercall instructions. Executing the | 8476 | hypercall instructions. Executing the |
| 8479 | incorrect hypercall instruction will | 8477 | incorrect hypercall instruction will |
| 8480 | generate a #UD within the guest. | 8478 | generate a #UD within the guest. |
| 8481 | 8479 | ||
| 8482 | KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS By default, KVM emulates MONITOR/MWAIT (if | 8480 | KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS By default, KVM emulates MONITOR/MWAIT (if |
| 8483 | they are intercepted) as NOPs regardless of | 8481 | they are intercepted) as NOPs regardless of |
| 8484 | whether or not MONITOR/MWAIT are supported | 8482 | whether or not MONITOR/MWAIT are supported |
| 8485 | according to guest CPUID. When this quirk | 8483 | according to guest CPUID. When this quirk |
| 8486 | is disabled and KVM_X86_DISABLE_EXITS_MWAIT | 8484 | is disabled and KVM_X86_DISABLE_EXITS_MWAIT |
| 8487 | is not set (MONITOR/MWAIT are intercepted), | 8485 | is not set (MONITOR/MWAIT are intercepted), |
| 8488 | KVM will inject a #UD on MONITOR/MWAIT if | 8486 | KVM will inject a #UD on MONITOR/MWAIT if |
| 8489 | they're unsupported per guest CPUID. Note, | 8487 | they're unsupported per guest CPUID. Note, |
| 8490 | KVM will modify MONITOR/MWAIT support in | 8488 | KVM will modify MONITOR/MWAIT support in |
| 8491 | guest CPUID on writes to MISC_ENABLE if | 8489 | guest CPUID on writes to MISC_ENABLE if |
| 8492 | KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT is | 8490 | KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT is |
| 8493 | disabled. | 8491 | disabled. |
| 8494 | 8492 | ||
| 8495 | KVM_X86_QUIRK_SLOT_ZAP_ALL By default, for KVM_X86_DEFAULT_VM VMs, KVM | 8493 | KVM_X86_QUIRK_SLOT_ZAP_ALL By default, for KVM_X86_DEFAULT_VM VMs, KVM |
| 8496 | invalidates all SPTEs in all memslots and | 8494 | invalidates all SPTEs in all memslots and |
| 8497 | address spaces when a memslot is deleted or | 8495 | address spaces when a memslot is deleted or |
| 8498 | moved. When this quirk is disabled (or the | 8496 | moved. When this quirk is disabled (or the |
| 8499 | VM type isn't KVM_X86_DEFAULT_VM), KVM only | 8497 | VM type isn't KVM_X86_DEFAULT_VM), KVM only |
| 8500 | ensures the backing memory of the deleted | 8498 | ensures the backing memory of the deleted |
| 8501 | or moved memslot isn't reachable, i.e KVM | 8499 | or moved memslot isn't reachable, i.e KVM |
| 8502 | _may_ invalidate only SPTEs related to the | 8500 | _may_ invalidate only SPTEs related to the |
| 8503 | memslot. | 8501 | memslot. |
| 8504 | 8502 | ||
| 8505 | KVM_X86_QUIRK_STUFF_FEATURE_MSRS By default, at vCPU creation, KVM sets the | 8503 | KVM_X86_QUIRK_STUFF_FEATURE_MSRS By default, at vCPU creation, KVM sets the |
| 8506 | vCPU's MSR_IA32_PERF_CAPABILITIES (0x345), | 8504 | vCPU's MSR_IA32_PERF_CAPABILITIES (0x345), |
| 8507 | MSR_IA32_ARCH_CAPABILITIES (0x10a), | 8505 | MSR_IA32_ARCH_CAPABILITIES (0x10a), |
| 8508 | MSR_PLATFORM_INFO (0xce), and all VMX MSRs | 8506 | MSR_PLATFORM_INFO (0xce), and all VMX MSRs |
| 8509 | (0x480..0x492) to the maximal capabilities | 8507 | (0x480..0x492) to the maximal capabilities |
| 8510 | supported by KVM. KVM also sets | 8508 | supported by KVM. KVM also sets |
| 8511 | MSR_IA32_UCODE_REV (0x8b) to an arbitrary | 8509 | MSR_IA32_UCODE_REV (0x8b) to an arbitrary |
| 8512 | value (which is different for Intel vs. | 8510 | value (which is different for Intel vs. |
| 8513 | AMD). Lastly, when guest CPUID is set (by | 8511 | AMD). Lastly, when guest CPUID is set (by |
| 8514 | userspace), KVM modifies select VMX MSR | 8512 | userspace), KVM modifies select VMX MSR |
| 8515 | fields to force consistency between guest | 8513 | fields to force consistency between guest |
| 8516 | CPUID and L2's effective ISA. When this | 8514 | CPUID and L2's effective ISA. When this |
| 8517 | quirk is disabled, KVM zeroes the vCPU's MSR | 8515 | quirk is disabled, KVM zeroes the vCPU's MSR |
| 8518 | values (with two exceptions, see below), | 8516 | values (with two exceptions, see below), |
| 8519 | i.e. treats the feature MSRs like CPUID | 8517 | i.e. treats the feature MSRs like CPUID |
| 8520 | leaves and gives userspace full control of | 8518 | leaves and gives userspace full control of |
| 8521 | the vCPU model definition. This quirk does | 8519 | the vCPU model definition. This quirk does |
| 8522 | not affect VMX MSRs CR0/CR4_FIXED1 (0x487 | 8520 | not affect VMX MSRs CR0/CR4_FIXED1 (0x487 |
| 8523 | and 0x489), as KVM does now allow them to | 8521 | and 0x489), as KVM does now allow them to |
| 8524 | be set by userspace (KVM sets them based on | 8522 | be set by userspace (KVM sets them based on |
| 8525 | guest CPUID, for safety purposes). | 8523 | guest CPUID, for safety purposes). |
| 8526 | 8524 | ||
| 8527 | KVM_X86_QUIRK_IGNORE_GUEST_PAT By default, on Intel platforms, KVM ignores | 8525 | KVM_X86_QUIRK_IGNORE_GUEST_PAT By default, on Intel platforms, KVM ignores |
| 8528 | guest PAT and forces the effective memory | 8526 | guest PAT and forces the effective memory |
| 8529 | type to WB in EPT. The quirk is not available | 8527 | type to WB in EPT. The quirk is not available |
| 8530 | on Intel platforms which are incapable of | 8528 | on Intel platforms which are incapable of |
| 8531 | safely honoring guest PAT (i.e., without CPU | 8529 | safely honoring guest PAT (i.e., without CPU |
| 8532 | self-snoop, KVM always ignores guest PAT and | 8530 | self-snoop, KVM always ignores guest PAT and |
| 8533 | forces effective memory type to WB). It is | 8531 | forces effective memory type to WB). It is |
| 8534 | also ignored on AMD platforms or, on Intel, | 8532 | also ignored on AMD platforms or, on Intel, |
| 8535 | when a VM has non-coherent DMA devices | 8533 | when a VM has non-coherent DMA devices |
| 8536 | assigned; KVM always honors guest PAT in | 8534 | assigned; KVM always honors guest PAT in |
| 8537 | such case. The quirk is needed to avoid | 8535 | such case. The quirk is needed to avoid |
| 8538 | slowdowns on certain Intel Xeon platforms | 8536 | slowdowns on certain Intel Xeon platforms |
| 8539 | (e.g. ICX, SPR) where self-snoop feature is | 8537 | (e.g. ICX, SPR) where self-snoop feature is |
| 8540 | supported but UC is slow enough to cause | 8538 | supported but UC is slow enough to cause |
| 8541 | issues with some older guests that use | 8539 | issues with some older guests that use |
| 8542 | UC instead of WC to map the video RAM. | 8540 | UC instead of WC to map the video RAM. |
| 8543 | Userspace can disable the quirk to honor | 8541 | Userspace can disable the quirk to honor |
| 8544 | guest PAT if it knows that there is no such | 8542 | guest PAT if it knows that there is no such |
| 8545 | guest software, for example if it does not | 8543 | guest software, for example if it does not |
| 8546 | expose a bochs graphics device (which is | 8544 | expose a bochs graphics device (which is |
| 8547 | known to have had a buggy driver). | 8545 | known to have had a buggy driver). |
| 8548 | =================================== ============================================ | 8546 | |
| 8547 | KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM By default, KVM relaxes the consistency | ||
| 8548 | check for GUEST_IA32_DEBUGCTL in vmcs12 | ||
| 8549 | to allow FREEZE_IN_SMM to be set. When | ||
| 8550 | this quirk is disabled, KVM requires this | ||
| 8551 | bit to be cleared. Note that the vmcs02 | ||
| 8552 | bit is still completely controlled by the | ||
| 8553 | host, regardless of the quirk setting. | ||
| 8554 | ======================================== ================================================ | ||
| 8549 | 8555 | ||
| 8550 | 7.32 KVM_CAP_MAX_VCPU_ID | 8556 | 7.32 KVM_CAP_MAX_VCPU_ID |
| 8551 | ------------------------ | 8557 | ------------------------ |
| @@ -17,6 +17,8 @@ The acquisition orders for mutexes are as follows: · 16 unmodified lines | |||
| 17 | 17 | ||
| 18 | - kvm->lock is taken outside kvm->slots_lock and kvm->irq_lock | 18 | - kvm->lock is taken outside kvm->slots_lock and kvm->irq_lock |
| 19 | 19 | ||
| 20 | - vcpu->mutex is taken outside kvm->slots_lock and kvm->slots_arch_lock | ||
| 21 | |||
| 20 | - kvm->slots_lock is taken outside kvm->irq_lock, though acquiring | 22 | - kvm->slots_lock is taken outside kvm->irq_lock, though acquiring |
| 21 | them together is quite rare. | 23 | them together is quite rare. |
| 22 | 24 | ||
| @@ -996,10 +996,8 @@ F: Documentation/devicetree/bindings/thermal/amazon,al-thermal.yaml · 995 unmodified lines | |||
| 996 | F: drivers/thermal/thermal_mmio.c | 996 | F: drivers/thermal/thermal_mmio.c |
| 997 | 997 | ||
| 998 | AMAZON ETHERNET DRIVERS | 998 | AMAZON ETHERNET DRIVERS |
| 999 | M: Shay Agroskin <shayagr@amazon.com> | ||
| 1000 | M: Arthur Kiyanovski <akiyano@amazon.com> | 999 | M: Arthur Kiyanovski <akiyano@amazon.com> |
| 1001 | R: David Arinzon <darinzon@amazon.com> | 1000 | M: David Arinzon <darinzon@amazon.com> |
| 1002 | R: Saeed Bishara <saeedb@amazon.com> | ||
| 1003 | L: netdev@vger.kernel.org | 1001 | L: netdev@vger.kernel.org |
| 1004 | S: Maintained | 1002 | S: Maintained |
| 1005 | F: Documentation/networking/device_drivers/ethernet/amazon/ena.rst | 1003 | F: Documentation/networking/device_drivers/ethernet/amazon/ena.rst |
| @@ -1295,7 +1293,6 @@ F: include/trace/events/amdxdna.h · 289 unmodified lines | |||
| 1295 | F: include/uapi/drm/amdxdna_accel.h | 1293 | F: include/uapi/drm/amdxdna_accel.h |
| 1296 | 1294 | ||
| 1297 | AMD XGBE DRIVER | 1295 | AMD XGBE DRIVER |
| 1298 | M: "Shyam Sundar S K" <Shyam-sundar.S-k@amd.com> | ||
| 1299 | M: Raju Rangoju <Raju.Rangoju@amd.com> | 1296 | M: Raju Rangoju <Raju.Rangoju@amd.com> |
| 1300 | L: netdev@vger.kernel.org | 1297 | L: netdev@vger.kernel.org |
| 1301 | S: Maintained | 1298 | S: Maintained |
| @@ -4638,7 +4635,6 @@ F: drivers/bluetooth/ · 3336 unmodified lines | |||
| 4638 | 4635 | ||
| 4639 | BLUETOOTH SUBSYSTEM | 4636 | BLUETOOTH SUBSYSTEM |
| 4640 | M: Marcel Holtmann <marcel@holtmann.org> | 4637 | M: Marcel Holtmann <marcel@holtmann.org> |
| 4641 | M: Johan Hedberg <johan.hedberg@gmail.com> | ||
| 4642 | M: Luiz Augusto von Dentz <luiz.dentz@gmail.com> | 4638 | M: Luiz Augusto von Dentz <luiz.dentz@gmail.com> |
| 4643 | L: linux-bluetooth@vger.kernel.org | 4639 | L: linux-bluetooth@vger.kernel.org |
| 4644 | S: Supported | 4640 | S: Supported |
| @@ -6233,20 +6229,20 @@ F: drivers/scsi/fnic/ · 1588 unmodified lines | |||
| 6233 | 6229 | ||
| 6234 | CISCO SCSI HBA DRIVER | 6230 | CISCO SCSI HBA DRIVER |
| 6235 | M: Karan Tilak Kumar <kartilak@cisco.com> | 6231 | M: Karan Tilak Kumar <kartilak@cisco.com> |
| 6232 | M: Narsimhulu Musini <nmusini@cisco.com> | ||
| 6236 | M: Sesidhar Baddela <sebaddel@cisco.com> | 6233 | M: Sesidhar Baddela <sebaddel@cisco.com> |
| 6237 | L: linux-scsi@vger.kernel.org | 6234 | L: linux-scsi@vger.kernel.org |
| 6238 | S: Supported | 6235 | S: Supported |
| 6239 | F: drivers/scsi/snic/ | 6236 | F: drivers/scsi/snic/ |
| 6240 | 6237 | ||
| 6241 | CISCO VIC ETHERNET NIC DRIVER | 6238 | CISCO VIC ETHERNET NIC DRIVER |
| 6242 | M: Christian Benvenuti <benve@cisco.com> | ||
| 6243 | M: Satish Kharat <satishkh@cisco.com> | 6239 | M: Satish Kharat <satishkh@cisco.com> |
| 6244 | S: Maintained | 6240 | S: Maintained |
| 6245 | F: drivers/net/ethernet/cisco/enic/ | 6241 | F: drivers/net/ethernet/cisco/enic/ |
| 6246 | 6242 | ||
| 6247 | CISCO VIC LOW LATENCY NIC DRIVER | 6243 | CISCO VIC LOW LATENCY NIC DRIVER |
| 6248 | M: Christian Benvenuti <benve@cisco.com> | ||
| 6249 | M: Nelson Escobar <neescoba@cisco.com> | 6244 | M: Nelson Escobar <neescoba@cisco.com> |
| 6245 | M: Satish Kharat <satishkh@cisco.com> | ||
| 6250 | S: Supported | 6246 | S: Supported |
| 6251 | F: drivers/infiniband/hw/usnic/ | 6247 | F: drivers/infiniband/hw/usnic/ |
| 6252 | 6248 | ||
| @@ -6300,7 +6296,7 @@ S: Maintained · 47 unmodified lines | |||
| 6300 | F: include/linux/clk.h | 6296 | F: include/linux/clk.h |
| 6301 | 6297 | ||
| 6302 | CLOCKSOURCE, CLOCKEVENT DRIVERS | 6298 | CLOCKSOURCE, CLOCKEVENT DRIVERS |
| 6303 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 6299 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 6304 | M: Thomas Gleixner <tglx@kernel.org> | 6300 | M: Thomas Gleixner <tglx@kernel.org> |
| 6305 | L: linux-kernel@vger.kernel.org | 6301 | L: linux-kernel@vger.kernel.org |
| 6306 | S: Supported | 6302 | S: Supported |
| @@ -6689,7 +6685,7 @@ F: rust/kernel/cpu.rs · 382 unmodified lines | |||
| 6689 | 6685 | ||
| 6690 | CPU IDLE TIME MANAGEMENT FRAMEWORK | 6686 | CPU IDLE TIME MANAGEMENT FRAMEWORK |
| 6691 | M: "Rafael J. Wysocki" <rafael@kernel.org> | 6687 | M: "Rafael J. Wysocki" <rafael@kernel.org> |
| 6692 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 6688 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 6693 | R: Christian Loehle <christian.loehle@arm.com> | 6689 | R: Christian Loehle <christian.loehle@arm.com> |
| 6694 | L: linux-pm@vger.kernel.org | 6690 | L: linux-pm@vger.kernel.org |
| 6695 | S: Maintained | 6691 | S: Maintained |
| @@ -6719,7 +6715,7 @@ F: arch/x86/kernel/msr.c · 23 unmodified lines | |||
| 6719 | 6715 | ||
| 6720 | CPUIDLE DRIVER - ARM BIG LITTLE | 6716 | CPUIDLE DRIVER - ARM BIG LITTLE |
| 6721 | M: Lorenzo Pieralisi <lpieralisi@kernel.org> | 6717 | M: Lorenzo Pieralisi <lpieralisi@kernel.org> |
| 6722 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 6718 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 6723 | L: linux-pm@vger.kernel.org | 6719 | L: linux-pm@vger.kernel.org |
| 6724 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 6720 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 6725 | S: Maintained | 6721 | S: Maintained |
| @@ -6727,7 +6723,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git · 1 unmodified line | |||
| 6727 | F: drivers/cpuidle/cpuidle-big_little.c | 6723 | F: drivers/cpuidle/cpuidle-big_little.c |
| 6728 | 6724 | ||
| 6729 | CPUIDLE DRIVER - ARM EXYNOS | 6725 | CPUIDLE DRIVER - ARM EXYNOS |
| 6730 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 6726 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 6731 | M: Kukjin Kim <kgene@kernel.org> | 6727 | M: Kukjin Kim <kgene@kernel.org> |
| 6732 | R: Krzysztof Kozlowski <krzk@kernel.org> | 6728 | R: Krzysztof Kozlowski <krzk@kernel.org> |
| 6733 | L: linux-pm@vger.kernel.org | 6729 | L: linux-pm@vger.kernel.org |
| @@ -8650,9 +8646,8 @@ F: drivers/gpu/drm/lima/ · 1916 unmodified lines | |||
| 8650 | F: include/uapi/drm/lima_drm.h | 8646 | F: include/uapi/drm/lima_drm.h |
| 8651 | 8647 | ||
| 8652 | DRM DRIVERS FOR LOONGSON | 8648 | DRM DRIVERS FOR LOONGSON |
| 8653 | M: Sui Jingfeng <suijingfeng@loongson.cn> | ||
| 8654 | L: dri-devel@lists.freedesktop.org | 8649 | L: dri-devel@lists.freedesktop.org |
| 8655 | S: Supported | 8650 | S: Orphan |
| 8656 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git | 8651 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git |
| 8657 | F: drivers/gpu/drm/loongson/ | 8652 | F: drivers/gpu/drm/loongson/ |
| 8658 | 8653 | ||
| @@ -10192,8 +10187,8 @@ F: drivers/i2c/busses/i2c-cpm.c · 1533 unmodified lines | |||
| 10192 | 10187 | ||
| 10193 | FREESCALE IMX / MXC FEC DRIVER | 10188 | FREESCALE IMX / MXC FEC DRIVER |
| 10194 | M: Wei Fang <wei.fang@nxp.com> | 10189 | M: Wei Fang <wei.fang@nxp.com> |
| 10190 | R: Frank Li <frank.li@nxp.com> | ||
| 10195 | R: Shenwei Wang <shenwei.wang@nxp.com> | 10191 | R: Shenwei Wang <shenwei.wang@nxp.com> |
| 10196 | R: Clark Wang <xiaoning.wang@nxp.com> | ||
| 10197 | L: imx@lists.linux.dev | 10192 | L: imx@lists.linux.dev |
| 10198 | L: netdev@vger.kernel.org | 10193 | L: netdev@vger.kernel.org |
| 10199 | S: Maintained | 10194 | S: Maintained |
| @@ -10505,7 +10500,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git · 305 unmodified lines | |||
| 10505 | F: Documentation/trace/ftrace* | 10500 | F: Documentation/trace/ftrace* |
| 10506 | F: arch/*/*/*/*ftrace* | 10501 | F: arch/*/*/*/*ftrace* |
| 10507 | F: arch/*/*/*ftrace* | 10502 | F: arch/*/*/*ftrace* |
| 10508 | F: include/*/ftrace.h | 10503 | F: include/*/*ftrace* |
| 10509 | F: kernel/trace/fgraph.c | 10504 | F: kernel/trace/fgraph.c |
| 10510 | F: kernel/trace/ftrace* | 10505 | F: kernel/trace/ftrace* |
| 10511 | F: samples/ftrace | 10506 | F: samples/ftrace |
| @@ -12237,7 +12232,6 @@ IBM Power SRIOV Virtual NIC Device Driver · 1725 unmodified lines | |||
| 12237 | M: Haren Myneni <haren@linux.ibm.com> | 12232 | M: Haren Myneni <haren@linux.ibm.com> |
| 12238 | M: Rick Lindsley <ricklind@linux.ibm.com> | 12233 | M: Rick Lindsley <ricklind@linux.ibm.com> |
| 12239 | R: Nick Child <nnac123@linux.ibm.com> | 12234 | R: Nick Child <nnac123@linux.ibm.com> |
| 12240 | R: Thomas Falcon <tlfalcon@linux.ibm.com> | ||
| 12241 | L: netdev@vger.kernel.org | 12235 | L: netdev@vger.kernel.org |
| 12242 | S: Maintained | 12236 | S: Maintained |
| 12243 | F: drivers/net/ethernet/ibm/ibmvnic.* | 12237 | F: drivers/net/ethernet/ibm/ibmvnic.* |
| @@ -13963,7 +13957,7 @@ F: fs/smb/server/ · 1719 unmodified lines | |||
| 13963 | 13957 | ||
| 13964 | KERNEL UNIT TESTING FRAMEWORK (KUnit) | 13958 | KERNEL UNIT TESTING FRAMEWORK (KUnit) |
| 13965 | M: Brendan Higgins <brendan.higgins@linux.dev> | 13959 | M: Brendan Higgins <brendan.higgins@linux.dev> |
| 13966 | M: David Gow <davidgow@google.com> | 13960 | M: David Gow <david@davidgow.net> |
| 13967 | R: Rae Moar <raemoar63@gmail.com> | 13961 | R: Rae Moar <raemoar63@gmail.com> |
| 13968 | L: linux-kselftest@vger.kernel.org | 13962 | L: linux-kselftest@vger.kernel.org |
| 13969 | L: kunit-dev@googlegroups.com | 13963 | L: kunit-dev@googlegroups.com |
| @@ -14432,9 +14426,9 @@ LANTIQ PEF2256 DRIVER · 462 unmodified lines | |||
| 14432 | M: Herve Codina <herve.codina@bootlin.com> | 14426 | M: Herve Codina <herve.codina@bootlin.com> |
| 14433 | S: Maintained | 14427 | S: Maintained |
| 14434 | F: Documentation/devicetree/bindings/net/lantiq,pef2256.yaml | 14428 | F: Documentation/devicetree/bindings/net/lantiq,pef2256.yaml |
| 14435 | F: drivers/net/wan/framer/pef2256/ | 14429 | F: drivers/net/wan/framer/ |
| 14436 | F: drivers/pinctrl/pinctrl-pef2256.c | 14430 | F: drivers/pinctrl/pinctrl-pef2256.c |
| 14437 | F: include/linux/framer/pef2256.h | 14431 | F: include/linux/framer/ |
| 14438 | 14432 | ||
| 14439 | LASI 53c700 driver for PARISC | 14433 | LASI 53c700 driver for PARISC |
| 14440 | M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> | 14434 | M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> |
| @@ -14783,7 +14777,7 @@ F: drivers/misc/lis3lv02d/ · 342 unmodified lines | |||
| 14783 | F: drivers/platform/x86/hp/hp_accel.c | 14777 | F: drivers/platform/x86/hp/hp_accel.c |
| 14784 | 14778 | ||
| 14785 | LIST KUNIT TEST | 14779 | LIST KUNIT TEST |
| 14786 | M: David Gow <davidgow@google.com> | 14780 | M: David Gow <david@davidgow.net> |
| 14787 | L: linux-kselftest@vger.kernel.org | 14781 | L: linux-kselftest@vger.kernel.org |
| 14788 | L: kunit-dev@googlegroups.com | 14782 | L: kunit-dev@googlegroups.com |
| 14789 | S: Maintained | 14783 | S: Maintained |
| @@ -15396,10 +15390,8 @@ F: drivers/crypto/marvell/ · 606 unmodified lines | |||
| 15396 | F: include/linux/soc/marvell/octeontx2/ | 15390 | F: include/linux/soc/marvell/octeontx2/ |
| 15397 | 15391 | ||
| 15398 | MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2) | 15392 | MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2) |
| 15399 | M: Mirko Lindner <mlindner@marvell.com> | ||
| 15400 | M: Stephen Hemminger <stephen@networkplumber.org> | ||
| 15401 | L: netdev@vger.kernel.org | 15393 | L: netdev@vger.kernel.org |
| 15402 | S: Odd fixes | 15394 | S: Orphan |
| 15403 | F: drivers/net/ethernet/marvell/sk* | 15395 | F: drivers/net/ethernet/marvell/sk* |
| 15404 | 15396 | ||
| 15405 | MARVELL LIBERTAS WIRELESS DRIVER | 15397 | MARVELL LIBERTAS WIRELESS DRIVER |
| @@ -15496,7 +15488,6 @@ MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER · 90 unmodified lines | |||
| 15496 | M: Sunil Goutham <sgoutham@marvell.com> | 15488 | M: Sunil Goutham <sgoutham@marvell.com> |
| 15497 | M: Linu Cherian <lcherian@marvell.com> | 15489 | M: Linu Cherian <lcherian@marvell.com> |
| 15498 | M: Geetha sowjanya <gakula@marvell.com> | 15490 | M: Geetha sowjanya <gakula@marvell.com> |
| 15499 | M: Jerin Jacob <jerinj@marvell.com> | ||
| 15500 | M: hariprasad <hkelam@marvell.com> | 15491 | M: hariprasad <hkelam@marvell.com> |
| 15501 | M: Subbaraya Sundeep <sbhatta@marvell.com> | 15492 | M: Subbaraya Sundeep <sbhatta@marvell.com> |
| 15502 | L: netdev@vger.kernel.org | 15493 | L: netdev@vger.kernel.org |
| @@ -15511,7 +15502,7 @@ S: Supported · 8 unmodified lines | |||
| 15511 | F: drivers/perf/marvell_pem_pmu.c | 15502 | F: drivers/perf/marvell_pem_pmu.c |
| 15512 | 15503 | ||
| 15513 | MARVELL PRESTERA ETHERNET SWITCH DRIVER | 15504 | MARVELL PRESTERA ETHERNET SWITCH DRIVER |
| 15514 | M: Taras Chornyi <taras.chornyi@plvision.eu> | 15505 | M: Elad Nachman <enachman@marvell.com> |
| 15515 | S: Supported | 15506 | S: Supported |
| 15516 | W: https://github.com/Marvell-switching/switchdev-prestera | 15507 | W: https://github.com/Marvell-switching/switchdev-prestera |
| 15517 | F: drivers/net/ethernet/marvell/prestera/ | 15508 | F: drivers/net/ethernet/marvell/prestera/ |
| @@ -16185,7 +16176,6 @@ F: drivers/dma/mediatek/ · 667 unmodified lines | |||
| 16185 | 16176 | ||
| 16186 | MEDIATEK ETHERNET DRIVER | 16177 | MEDIATEK ETHERNET DRIVER |
| 16187 | M: Felix Fietkau <nbd@nbd.name> | 16178 | M: Felix Fietkau <nbd@nbd.name> |
| 16188 | M: Sean Wang <sean.wang@mediatek.com> | ||
| 16189 | M: Lorenzo Bianconi <lorenzo@kernel.org> | 16179 | M: Lorenzo Bianconi <lorenzo@kernel.org> |
| 16190 | L: netdev@vger.kernel.org | 16180 | L: netdev@vger.kernel.org |
| 16191 | S: Maintained | 16181 | S: Maintained |
| @@ -16378,8 +16368,6 @@ F: include/soc/mediatek/smi.h · 186 unmodified lines | |||
| 16378 | MEDIATEK SWITCH DRIVER | 16368 | MEDIATEK SWITCH DRIVER |
| 16379 | M: Chester A. Unal <chester.a.unal@arinc9.com> | 16369 | M: Chester A. Unal <chester.a.unal@arinc9.com> |
| 16380 | M: Daniel Golle <daniel@makrotopia.org> | 16370 | M: Daniel Golle <daniel@makrotopia.org> |
| 16381 | M: DENG Qingfang <dqfext@gmail.com> | ||
| 16382 | M: Sean Wang <sean.wang@mediatek.com> | ||
| 16383 | L: netdev@vger.kernel.org | 16371 | L: netdev@vger.kernel.org |
| 16384 | S: Maintained | 16372 | S: Maintained |
| 16385 | F: drivers/net/dsa/mt7530-mdio.c | 16373 | F: drivers/net/dsa/mt7530-mdio.c |
| @@ -16389,7 +16377,6 @@ F: net/dsa/tag_mtk.c · 3 unmodified lines | |||
| 16389 | 16377 | ||
| 16390 | MEDIATEK T7XX 5G WWAN MODEM DRIVER | 16378 | MEDIATEK T7XX 5G WWAN MODEM DRIVER |
| 16391 | M: Chandrashekar Devegowda <chandrashekar.devegowda@intel.com> | 16379 | M: Chandrashekar Devegowda <chandrashekar.devegowda@intel.com> |
| 16392 | R: Chiranjeevi Rapolu <chiranjeevi.rapolu@linux.intel.com> | ||
| 16393 | R: Liu Haijun <haijun.liu@mediatek.com> | 16380 | R: Liu Haijun <haijun.liu@mediatek.com> |
| 16394 | R: Ricardo Martinez <ricardo.martinez@linux.intel.com> | 16381 | R: Ricardo Martinez <ricardo.martinez@linux.intel.com> |
| 16395 | L: netdev@vger.kernel.org | 16382 | L: netdev@vger.kernel.org |
| @@ -16674,9 +16661,9 @@ F: mm/balloon.c · 278 unmodified lines | |||
| 16674 | MEMORY MANAGEMENT - CORE | 16661 | MEMORY MANAGEMENT - CORE |
| 16675 | M: Andrew Morton <akpm@linux-foundation.org> | 16662 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16676 | M: David Hildenbrand <david@kernel.org> | 16663 | M: David Hildenbrand <david@kernel.org> |
| 16677 | R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16664 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16678 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16665 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16679 | R: Vlastimil Babka <vbabka@suse.cz> | 16666 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16680 | R: Mike Rapoport <rppt@kernel.org> | 16667 | R: Mike Rapoport <rppt@kernel.org> |
| 16681 | R: Suren Baghdasaryan <surenb@google.com> | 16668 | R: Suren Baghdasaryan <surenb@google.com> |
| 16682 | R: Michal Hocko <mhocko@suse.com> | 16669 | R: Michal Hocko <mhocko@suse.com> |
| @@ -16804,9 +16791,9 @@ F: mm/workingset.c · 121 unmodified lines | |||
| 16804 | MEMORY MANAGEMENT - MISC | 16791 | MEMORY MANAGEMENT - MISC |
| 16805 | M: Andrew Morton <akpm@linux-foundation.org> | 16792 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16806 | M: David Hildenbrand <david@kernel.org> | 16793 | M: David Hildenbrand <david@kernel.org> |
| 16807 | R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16794 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16808 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16795 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16809 | R: Vlastimil Babka <vbabka@suse.cz> | 16796 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16810 | R: Mike Rapoport <rppt@kernel.org> | 16797 | R: Mike Rapoport <rppt@kernel.org> |
| 16811 | R: Suren Baghdasaryan <surenb@google.com> | 16798 | R: Suren Baghdasaryan <surenb@google.com> |
| 16812 | R: Michal Hocko <mhocko@suse.com> | 16799 | R: Michal Hocko <mhocko@suse.com> |
| @@ -16861,7 +16848,7 @@ F: mm/oom_kill.c · 48 unmodified lines | |||
| 16861 | 16848 | ||
| 16862 | MEMORY MANAGEMENT - PAGE ALLOCATOR | 16849 | MEMORY MANAGEMENT - PAGE ALLOCATOR |
| 16863 | M: Andrew Morton <akpm@linux-foundation.org> | 16850 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16864 | M: Vlastimil Babka <vbabka@suse.cz> | 16851 | M: Vlastimil Babka <vbabka@kernel.org> |
| 16865 | R: Suren Baghdasaryan <surenb@google.com> | 16852 | R: Suren Baghdasaryan <surenb@google.com> |
| 16866 | R: Michal Hocko <mhocko@suse.com> | 16853 | R: Michal Hocko <mhocko@suse.com> |
| 16867 | R: Brendan Jackman <jackmanb@google.com> | 16854 | R: Brendan Jackman <jackmanb@google.com> |
| @@ -16895,7 +16882,7 @@ R: David Hildenbrand <david@kernel.org> · 27 unmodified lines | |||
| 16895 | R: Michal Hocko <mhocko@kernel.org> | 16882 | R: Michal Hocko <mhocko@kernel.org> |
| 16896 | R: Qi Zheng <zhengqi.arch@bytedance.com> | 16883 | R: Qi Zheng <zhengqi.arch@bytedance.com> |
| 16897 | R: Shakeel Butt <shakeel.butt@linux.dev> | 16884 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 16898 | R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16885 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16899 | L: linux-mm@kvack.org | 16886 | L: linux-mm@kvack.org |
| 16900 | S: Maintained | 16887 | S: Maintained |
| 16901 | F: mm/vmscan.c | 16888 | F: mm/vmscan.c |
| @@ -16904,10 +16891,10 @@ F: mm/workingset.c · 2 unmodified lines | |||
| 16904 | MEMORY MANAGEMENT - RMAP (REVERSE MAPPING) | 16891 | MEMORY MANAGEMENT - RMAP (REVERSE MAPPING) |
| 16905 | M: Andrew Morton <akpm@linux-foundation.org> | 16892 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16906 | M: David Hildenbrand <david@kernel.org> | 16893 | M: David Hildenbrand <david@kernel.org> |
| 16907 | M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16894 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 16908 | R: Rik van Riel <riel@surriel.com> | 16895 | R: Rik van Riel <riel@surriel.com> |
| 16909 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16896 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16910 | R: Vlastimil Babka <vbabka@suse.cz> | 16897 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16911 | R: Harry Yoo <harry.yoo@oracle.com> | 16898 | R: Harry Yoo <harry.yoo@oracle.com> |
| 16912 | R: Jann Horn <jannh@google.com> | 16899 | R: Jann Horn <jannh@google.com> |
| 16913 | L: linux-mm@kvack.org | 16900 | L: linux-mm@kvack.org |
| @@ -16949,7 +16936,7 @@ F: mm/swapfile.c · 35 unmodified lines | |||
| 16949 | MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE) | 16936 | MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE) |
| 16950 | M: Andrew Morton <akpm@linux-foundation.org> | 16937 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16951 | M: David Hildenbrand <david@kernel.org> | 16938 | M: David Hildenbrand <david@kernel.org> |
| 16952 | M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16939 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 16953 | R: Zi Yan <ziy@nvidia.com> | 16940 | R: Zi Yan <ziy@nvidia.com> |
| 16954 | R: Baolin Wang <baolin.wang@linux.alibaba.com> | 16941 | R: Baolin Wang <baolin.wang@linux.alibaba.com> |
| 16955 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16942 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| @@ -16989,7 +16976,7 @@ F: tools/testing/selftests/mm/uffd-*.[ch] · 33 unmodified lines | |||
| 16989 | 16976 | ||
| 16990 | MEMORY MANAGEMENT - RUST | 16977 | MEMORY MANAGEMENT - RUST |
| 16991 | M: Alice Ryhl <aliceryhl@google.com> | 16978 | M: Alice Ryhl <aliceryhl@google.com> |
| 16992 | R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16979 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16993 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16980 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16994 | L: linux-mm@kvack.org | 16981 | L: linux-mm@kvack.org |
| 16995 | L: rust-for-linux@vger.kernel.org | 16982 | L: rust-for-linux@vger.kernel.org |
| @@ -17005,8 +16992,8 @@ F: rust/kernel/page.rs · 9 unmodified lines | |||
| 17005 | MEMORY MAPPING | 16992 | MEMORY MAPPING |
| 17006 | M: Andrew Morton <akpm@linux-foundation.org> | 16993 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17007 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 16994 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17008 | M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 16995 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17009 | R: Vlastimil Babka <vbabka@suse.cz> | 16996 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17010 | R: Jann Horn <jannh@google.com> | 16997 | R: Jann Horn <jannh@google.com> |
| 17011 | R: Pedro Falcato <pfalcato@suse.de> | 16998 | R: Pedro Falcato <pfalcato@suse.de> |
| 17012 | L: linux-mm@kvack.org | 16999 | L: linux-mm@kvack.org |
| @@ -17035,8 +17022,8 @@ MEMORY MAPPING - LOCKING · 22 unmodified lines | |||
| 17035 | M: Andrew Morton <akpm@linux-foundation.org> | 17022 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17036 | M: Suren Baghdasaryan <surenb@google.com> | 17023 | M: Suren Baghdasaryan <surenb@google.com> |
| 17037 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 17024 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17038 | M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 17025 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17039 | R: Vlastimil Babka <vbabka@suse.cz> | 17026 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17040 | R: Shakeel Butt <shakeel.butt@linux.dev> | 17027 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 17041 | L: linux-mm@kvack.org | 17028 | L: linux-mm@kvack.org |
| 17042 | S: Maintained | 17029 | S: Maintained |
| @@ -17050,9 +17037,9 @@ F: mm/mmap_lock.c · 7 unmodified lines | |||
| 17050 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) | 17037 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) |
| 17051 | M: Andrew Morton <akpm@linux-foundation.org> | 17038 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17052 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 17039 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17053 | M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 17040 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17054 | M: David Hildenbrand <david@kernel.org> | 17041 | M: David Hildenbrand <david@kernel.org> |
| 17055 | R: Vlastimil Babka <vbabka@suse.cz> | 17042 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17056 | R: Jann Horn <jannh@google.com> | 17043 | R: Jann Horn <jannh@google.com> |
| 17057 | L: linux-mm@kvack.org | 17044 | L: linux-mm@kvack.org |
| 17058 | S: Maintained | 17045 | S: Maintained |
| @@ -19247,8 +19234,6 @@ F: tools/objtool/ · 2188 unmodified lines | |||
| 19247 | 19234 | ||
| 19248 | OCELOT ETHERNET SWITCH DRIVER | 19235 | OCELOT ETHERNET SWITCH DRIVER |
| 19249 | M: Vladimir Oltean <vladimir.oltean@nxp.com> | 19236 | M: Vladimir Oltean <vladimir.oltean@nxp.com> |
| 19250 | M: Claudiu Manoil <claudiu.manoil@nxp.com> | ||
| 19251 | M: Alexandre Belloni <alexandre.belloni@bootlin.com> | ||
| 19252 | M: UNGLinuxDriver@microchip.com | 19237 | M: UNGLinuxDriver@microchip.com |
| 19253 | L: netdev@vger.kernel.org | 19238 | L: netdev@vger.kernel.org |
| 19254 | S: Supported | 19239 | S: Supported |
| @@ -19834,7 +19819,6 @@ F: arch/*/boot/dts/ · 579 unmodified lines | |||
| 19834 | F: include/dt-bindings/ | 19819 | F: include/dt-bindings/ |
| 19835 | 19820 | ||
| 19836 | OPENCOMPUTE PTP CLOCK DRIVER | 19821 | OPENCOMPUTE PTP CLOCK DRIVER |
| 19837 | M: Jonathan Lemon <jonathan.lemon@gmail.com> | ||
| 19838 | M: Vadim Fedorenko <vadim.fedorenko@linux.dev> | 19822 | M: Vadim Fedorenko <vadim.fedorenko@linux.dev> |
| 19839 | L: netdev@vger.kernel.org | 19823 | L: netdev@vger.kernel.org |
| 19840 | S: Maintained | 19824 | S: Maintained |
| @@ -20142,9 +20126,8 @@ F: Documentation/devicetree/bindings/pci/marvell,armada-3700-pcie.yaml · 301 unmodified lines | |||
| 20142 | F: drivers/pci/controller/pci-aardvark.c | 20126 | F: drivers/pci/controller/pci-aardvark.c |
| 20143 | 20127 | ||
| 20144 | PCI DRIVER FOR ALTERA PCIE IP | 20128 | PCI DRIVER FOR ALTERA PCIE IP |
| 20145 | M: Joyce Ooi <joyce.ooi@intel.com> | ||
| 20146 | L: linux-pci@vger.kernel.org | 20129 | L: linux-pci@vger.kernel.org |
| 20147 | S: Supported | 20130 | S: Orphan |
| 20148 | F: Documentation/devicetree/bindings/pci/altr,pcie-root-port.yaml | 20131 | F: Documentation/devicetree/bindings/pci/altr,pcie-root-port.yaml |
| 20149 | F: drivers/pci/controller/pcie-altera.c | 20132 | F: drivers/pci/controller/pcie-altera.c |
| 20150 | 20133 | ||
| @@ -20389,9 +20372,8 @@ S: Supported · 238 unmodified lines | |||
| 20389 | F: Documentation/PCI/pci-error-recovery.rst | 20372 | F: Documentation/PCI/pci-error-recovery.rst |
| 20390 | 20373 | ||
| 20391 | PCI MSI DRIVER FOR ALTERA MSI IP | 20374 | PCI MSI DRIVER FOR ALTERA MSI IP |
| 20392 | M: Joyce Ooi <joyce.ooi@intel.com> | ||
| 20393 | L: linux-pci@vger.kernel.org | 20375 | L: linux-pci@vger.kernel.org |
| 20394 | S: Supported | 20376 | S: Orphan |
| 20395 | F: Documentation/devicetree/bindings/interrupt-controller/altr,msi-controller.yaml | 20377 | F: Documentation/devicetree/bindings/interrupt-controller/altr,msi-controller.yaml |
| 20396 | F: drivers/pci/controller/pcie-altera-msi.c | 20378 | F: drivers/pci/controller/pcie-altera-msi.c |
| 20397 | 20379 | ||
| @@ -20529,7 +20511,7 @@ F: Documentation/devicetree/bindings/pci/hisilicon,kirin-pcie.yaml · 131 unmodified lines | |||
| 20529 | F: drivers/pci/controller/dwc/pcie-kirin.c | 20511 | F: drivers/pci/controller/dwc/pcie-kirin.c |
| 20530 | 20512 | ||
| 20531 | PCIE DRIVER FOR HISILICON STB | 20513 | PCIE DRIVER FOR HISILICON STB |
| 20532 | M: Shawn Guo <shawn.guo@linaro.org> | 20514 | M: Shawn Guo <shawnguo@kernel.org> |
| 20533 | L: linux-pci@vger.kernel.org | 20515 | L: linux-pci@vger.kernel.org |
| 20534 | S: Maintained | 20516 | S: Maintained |
| 20535 | F: Documentation/devicetree/bindings/pci/hisilicon-histb-pcie.txt | 20517 | F: Documentation/devicetree/bindings/pci/hisilicon-histb-pcie.txt |
| @@ -21478,9 +21460,8 @@ S: Supported · 942 unmodified lines | |||
| 21478 | F: drivers/scsi/qedi/ | 21460 | F: drivers/scsi/qedi/ |
| 21479 | 21461 | ||
| 21480 | QLOGIC QL4xxx ETHERNET DRIVER | 21462 | QLOGIC QL4xxx ETHERNET DRIVER |
| 21481 | M: Manish Chopra <manishc@marvell.com> | ||
| 21482 | L: netdev@vger.kernel.org | 21463 | L: netdev@vger.kernel.org |
| 21483 | S: Maintained | 21464 | S: Orphan |
| 21484 | F: drivers/net/ethernet/qlogic/qed/ | 21465 | F: drivers/net/ethernet/qlogic/qed/ |
| 21485 | F: drivers/net/ethernet/qlogic/qede/ | 21466 | F: drivers/net/ethernet/qlogic/qede/ |
| 21486 | F: include/linux/qed/ | 21467 | F: include/linux/qed/ |
| @@ -21715,7 +21696,7 @@ S: Maintained · 228 unmodified lines | |||
| 21715 | F: drivers/net/ethernet/qualcomm/emac/ | 21696 | F: drivers/net/ethernet/qualcomm/emac/ |
| 21716 | 21697 | ||
| 21717 | QUALCOMM ETHQOS ETHERNET DRIVER | 21698 | QUALCOMM ETHQOS ETHERNET DRIVER |
| 21718 | M: Vinod Koul <vkoul@kernel.org> | 21699 | M: Mohd Ayaan Anwar <mohd.anwar@oss.qualcomm.com> |
| 21719 | L: netdev@vger.kernel.org | 21700 | L: netdev@vger.kernel.org |
| 21720 | L: linux-arm-msm@vger.kernel.org | 21701 | L: linux-arm-msm@vger.kernel.org |
| 21721 | S: Maintained | 21702 | S: Maintained |
| @@ -21975,7 +21956,7 @@ F: drivers/media/radio/radio-tea5777.c · 253 unmodified lines | |||
| 21975 | 21956 | ||
| 21976 | RADOS BLOCK DEVICE (RBD) | 21957 | RADOS BLOCK DEVICE (RBD) |
| 21977 | M: Ilya Dryomov <idryomov@gmail.com> | 21958 | M: Ilya Dryomov <idryomov@gmail.com> |
| 21978 | R: Dongsheng Yang <dongsheng.yang@easystack.cn> | 21959 | R: Dongsheng Yang <dongsheng.yang@linux.dev> |
| 21979 | L: ceph-devel@vger.kernel.org | 21960 | L: ceph-devel@vger.kernel.org |
| 21980 | S: Supported | 21961 | S: Supported |
| 21981 | W: http://ceph.com/ | 21962 | W: http://ceph.com/ |
| @@ -22304,6 +22285,16 @@ L: linux-wireless@vger.kernel.org · 322 unmodified lines | |||
| 22304 | S: Orphan | 22285 | S: Orphan |
| 22305 | F: drivers/net/wireless/rsi/ | 22286 | F: drivers/net/wireless/rsi/ |
| 22306 | 22287 | ||
| 22288 | RELAY | ||
| 22289 | M: Andrew Morton <akpm@linux-foundation.org> | ||
| 22290 | M: Jens Axboe <axboe@kernel.dk> | ||
| 22291 | M: Jason Xing <kernelxing@tencent.com> | ||
| 22292 | L: linux-kernel@vger.kernel.org | ||
| 22293 | S: Maintained | ||
| 22294 | F: Documentation/filesystems/relay.rst | ||
| 22295 | F: include/linux/relay.h | ||
| 22296 | F: kernel/relay.c | ||
| 22297 | |||
| 22307 | REGISTER MAP ABSTRACTION | 22298 | REGISTER MAP ABSTRACTION |
| 22308 | M: Mark Brown <broonie@kernel.org> | 22299 | M: Mark Brown <broonie@kernel.org> |
| 22309 | L: linux-kernel@vger.kernel.org | 22300 | L: linux-kernel@vger.kernel.org |
| @@ -23193,8 +23184,8 @@ K: \b(?i:rust)\b · 883 unmodified lines | |||
| 23193 | 23184 | ||
| 23194 | RUST [ALLOC] | 23185 | RUST [ALLOC] |
| 23195 | M: Danilo Krummrich <dakr@kernel.org> | 23186 | M: Danilo Krummrich <dakr@kernel.org> |
| 23196 | R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | 23187 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 23197 | R: Vlastimil Babka <vbabka@suse.cz> | 23188 | R: Vlastimil Babka <vbabka@kernel.org> |
| 23198 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 23189 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 23199 | R: Uladzislau Rezki <urezki@gmail.com> | 23190 | R: Uladzislau Rezki <urezki@gmail.com> |
| 23200 | L: rust-for-linux@vger.kernel.org | 23191 | L: rust-for-linux@vger.kernel.org |
| @@ -24357,7 +24348,6 @@ F: Documentation/devicetree/bindings/interrupt-controller/kontron,sl28cpld-intc. · 1156 unmodified lines | |||
| 24357 | F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml | 24348 | F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml |
| 24358 | F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml | 24349 | F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml |
| 24359 | F: drivers/gpio/gpio-sl28cpld.c | 24350 | F: drivers/gpio/gpio-sl28cpld.c |
| 24360 | F: drivers/hwmon/sa67mcu-hwmon.c | ||
| 24361 | F: drivers/hwmon/sl28cpld-hwmon.c | 24351 | F: drivers/hwmon/sl28cpld-hwmon.c |
| 24362 | F: drivers/irqchip/irq-sl28cpld.c | 24352 | F: drivers/irqchip/irq-sl28cpld.c |
| 24363 | F: drivers/pwm/pwm-sl28cpld.c | 24353 | F: drivers/pwm/pwm-sl28cpld.c |
| @@ -24370,12 +24360,13 @@ F: Documentation/devicetree/bindings/nvmem/layouts/kontron,sl28-vpd.yaml · 6 unmodified lines | |||
| 24370 | F: drivers/nvmem/layouts/sl28vpd.c | 24360 | F: drivers/nvmem/layouts/sl28vpd.c |
| 24371 | 24361 | ||
| 24372 | SLAB ALLOCATOR | 24362 | SLAB ALLOCATOR |
| 24373 | M: Vlastimil Babka <vbabka@suse.cz> | 24363 | M: Vlastimil Babka <vbabka@kernel.org> |
| 24364 | M: Harry Yoo <harry.yoo@oracle.com> | ||
| 24374 | M: Andrew Morton <akpm@linux-foundation.org> | 24365 | M: Andrew Morton <akpm@linux-foundation.org> |
| 24366 | R: Hao Li <hao.li@linux.dev> | ||
| 24375 | R: Christoph Lameter <cl@gentwo.org> | 24367 | R: Christoph Lameter <cl@gentwo.org> |
| 24376 | R: David Rientjes <rientjes@google.com> | 24368 | R: David Rientjes <rientjes@google.com> |
| 24377 | R: Roman Gushchin <roman.gushchin@linux.dev> | 24369 | R: Roman Gushchin <roman.gushchin@linux.dev> |
| 24378 | R: Harry Yoo <harry.yoo@oracle.com> | ||
| 24379 | L: linux-mm@kvack.org | 24370 | L: linux-mm@kvack.org |
| 24380 | S: Maintained | 24371 | S: Maintained |
| 24381 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git | 24372 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git |
| @@ -25786,6 +25777,7 @@ F: include/net/pkt_cls.h · 1404 unmodified lines | |||
| 25786 | F: include/net/pkt_sched.h | 25777 | F: include/net/pkt_sched.h |
| 25787 | F: include/net/sch_priv.h | 25778 | F: include/net/sch_priv.h |
| 25788 | F: include/net/tc_act/ | 25779 | F: include/net/tc_act/ |
| 25780 | F: include/net/tc_wrapper.h | ||
| 25789 | F: include/uapi/linux/pkt_cls.h | 25781 | F: include/uapi/linux/pkt_cls.h |
| 25790 | F: include/uapi/linux/pkt_sched.h | 25782 | F: include/uapi/linux/pkt_sched.h |
| 25791 | F: include/uapi/linux/tc_act/ | 25783 | F: include/uapi/linux/tc_act/ |
| @@ -26237,7 +26229,7 @@ F: drivers/media/radio/radio-raremono.c · 445 unmodified lines | |||
| 26237 | 26229 | ||
| 26238 | THERMAL | 26230 | THERMAL |
| 26239 | M: Rafael J. Wysocki <rafael@kernel.org> | 26231 | M: Rafael J. Wysocki <rafael@kernel.org> |
| 26240 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 26232 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 26241 | R: Zhang Rui <rui.zhang@intel.com> | 26233 | R: Zhang Rui <rui.zhang@intel.com> |
| 26242 | R: Lukasz Luba <lukasz.luba@arm.com> | 26234 | R: Lukasz Luba <lukasz.luba@arm.com> |
| 26243 | L: linux-pm@vger.kernel.org | 26235 | L: linux-pm@vger.kernel.org |
| @@ -26267,7 +26259,7 @@ F: drivers/thermal/amlogic_thermal.c · 23 unmodified lines | |||
| 26267 | 26259 | ||
| 26268 | THERMAL/CPU_COOLING | 26260 | THERMAL/CPU_COOLING |
| 26269 | M: Amit Daniel Kachhap <amit.kachhap@gmail.com> | 26261 | M: Amit Daniel Kachhap <amit.kachhap@gmail.com> |
| 26270 | M: Daniel Lezcano <daniel.lezcano@linaro.org> | 26262 | M: Daniel Lezcano <daniel.lezcano@kernel.org> |
| 26271 | M: Viresh Kumar <viresh.kumar@linaro.org> | 26263 | M: Viresh Kumar <viresh.kumar@linaro.org> |
| 26272 | R: Lukasz Luba <lukasz.luba@arm.com> | 26264 | R: Lukasz Luba <lukasz.luba@arm.com> |
| 26273 | L: linux-pm@vger.kernel.org | 26265 | L: linux-pm@vger.kernel.org |
| @@ -29206,7 +29198,7 @@ K: zstd · 2932 unmodified lines | |||
| 29206 | 29198 | ||
| 29207 | ZSWAP COMPRESSED SWAP CACHING | 29199 | ZSWAP COMPRESSED SWAP CACHING |
| 29208 | M: Johannes Weiner <hannes@cmpxchg.org> | 29200 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 29209 | M: Yosry Ahmed <yosry.ahmed@linux.dev> | 29201 | M: Yosry Ahmed <yosry@kernel.org> |
| 29210 | M: Nhat Pham <nphamcs@gmail.com> | 29202 | M: Nhat Pham <nphamcs@gmail.com> |
| 29211 | R: Chengming Zhou <chengming.zhou@linux.dev> | 29203 | R: Chengming Zhou <chengming.zhou@linux.dev> |
| 29212 | L: linux-mm@kvack.org | 29204 | L: linux-mm@kvack.org |
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | VERSION = 7 | 2 | VERSION = 7 |
| 3 | PATCHLEVEL = 0 | 3 | PATCHLEVEL = 0 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc1 | 5 | EXTRAVERSION = -rc4 |
| 6 | NAME = Baby Opossum Posse | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| @@ -476,6 +476,7 @@ KBUILD_USERLDFLAGS := $(USERLDFLAGS) · 467 unmodified lines | |||
| 476 | export rust_common_flags := --edition=2021 \ | 476 | export rust_common_flags := --edition=2021 \ |
| 477 | -Zbinary_dep_depinfo=y \ | 477 | -Zbinary_dep_depinfo=y \ |
| 478 | -Astable_features \ | 478 | -Astable_features \ |
| 479 | -Aunused_features \ | ||
| 479 | -Dnon_ascii_idents \ | 480 | -Dnon_ascii_idents \ |
| 480 | -Dunsafe_op_in_unsafe_fn \ | 481 | -Dunsafe_op_in_unsafe_fn \ |
| 481 | -Wmissing_docs \ | 482 | -Wmissing_docs \ |
| @@ -1113,6 +1114,9 @@ KBUILD_CFLAGS += -fno-builtin-wcslen · 631 unmodified lines | |||
| 1113 | # change __FILE__ to the relative path to the source directory | 1114 | # change __FILE__ to the relative path to the source directory |
| 1114 | ifdef building_out_of_srctree | 1115 | ifdef building_out_of_srctree |
| 1115 | KBUILD_CPPFLAGS += -fmacro-prefix-map=$(srcroot)/= | 1116 | KBUILD_CPPFLAGS += -fmacro-prefix-map=$(srcroot)/= |
| 1117 | ifeq ($(call rustc-option-yn, --remap-path-scope=macro),y) | ||
| 1118 | KBUILD_RUSTFLAGS += --remap-path-prefix=$(srcroot)/= --remap-path-scope=macro | ||
| 1119 | endif | ||
| 1116 | endif | 1120 | endif |
| 1117 | 1121 | ||
| 1118 | # include additional Makefiles when needed | 1122 | # include additional Makefiles when needed |
| @@ -1497,13 +1501,13 @@ ifneq ($(wildcard $(resolve_btfids_O)),) · 378 unmodified lines | |||
| 1497 | $(Q)$(MAKE) -sC $(srctree)/tools/bpf/resolve_btfids O=$(resolve_btfids_O) clean | 1501 | $(Q)$(MAKE) -sC $(srctree)/tools/bpf/resolve_btfids O=$(resolve_btfids_O) clean |
| 1498 | endif | 1502 | endif |
| 1499 | 1503 | ||
| 1500 | PHONY += objtool_clean | 1504 | PHONY += objtool_clean objtool_mrproper |
| 1501 | 1505 | ||
| 1502 | objtool_O = $(abspath $(objtree))/tools/objtool | 1506 | objtool_O = $(abspath $(objtree))/tools/objtool |
| 1503 | 1507 | ||
| 1504 | objtool_clean: | 1508 | objtool_clean objtool_mrproper: |
| 1505 | ifneq ($(wildcard $(objtool_O)),) | 1509 | ifneq ($(wildcard $(objtool_O)),) |
| 1506 | $(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) clean | 1510 | $(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) $(patsubst objtool_%,%,$@) |
| 1507 | endif | 1511 | endif |
| 1508 | 1512 | ||
| 1509 | tools/: FORCE | 1513 | tools/: FORCE |
| @@ -1686,7 +1690,7 @@ PHONY += $(mrproper-dirs) mrproper · 176 unmodified lines | |||
| 1686 | $(mrproper-dirs): | 1690 | $(mrproper-dirs): |
| 1687 | $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@) | 1691 | $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@) |
| 1688 | 1692 | ||
| 1689 | mrproper: clean $(mrproper-dirs) | 1693 | mrproper: clean objtool_mrproper $(mrproper-dirs) |
| 1690 | $(call cmd,rmfiles) | 1694 | $(call cmd,rmfiles) |
| 1691 | @find . $(RCS_FIND_IGNORE) \ | 1695 | @find . $(RCS_FIND_IGNORE) \ |
| 1692 | \( -name '*.rmeta' \) \ | 1696 | \( -name '*.rmeta' \) \ |
| @@ -71,6 +71,7 @@ SECTIONS · 70 unmodified lines | |||
| 71 | 71 | ||
| 72 | STABS_DEBUG | 72 | STABS_DEBUG |
| 73 | DWARF_DEBUG | 73 | DWARF_DEBUG |
| 74 | MODINFO | ||
| 74 | ELF_DETAILS | 75 | ELF_DETAILS |
| 75 | 76 | ||
| 76 | DISCARDS | 77 | DISCARDS |
| @@ -123,6 +123,7 @@ SECTIONS · 122 unmodified lines | |||
| 123 | _end = . ; | 123 | _end = . ; |
| 124 | 124 | ||
| 125 | STABS_DEBUG | 125 | STABS_DEBUG |
| 126 | MODINFO | ||
| 126 | ELF_DETAILS | 127 | ELF_DETAILS |
| 127 | DISCARDS | 128 | DISCARDS |
| 128 | 129 | ||
| @@ -21,6 +21,7 @@ SECTIONS · 20 unmodified lines | |||
| 21 | COMMON_DISCARDS | 21 | COMMON_DISCARDS |
| 22 | *(.ARM.exidx*) | 22 | *(.ARM.exidx*) |
| 23 | *(.ARM.extab*) | 23 | *(.ARM.extab*) |
| 24 | *(.modinfo) | ||
| 24 | *(.note.*) | 25 | *(.note.*) |
| 25 | *(.rel.*) | 26 | *(.rel.*) |
| 26 | *(.printk_index) | 27 | *(.printk_index) |
| @@ -154,6 +154,7 @@ SECTIONS · 153 unmodified lines | |||
| 154 | 154 | ||
| 155 | STABS_DEBUG | 155 | STABS_DEBUG |
| 156 | DWARF_DEBUG | 156 | DWARF_DEBUG |
| 157 | MODINFO | ||
| 157 | ARM_DETAILS | 158 | ARM_DETAILS |
| 158 | 159 | ||
| 159 | ARM_ASSERTS | 160 | ARM_ASSERTS |
| @@ -153,6 +153,7 @@ SECTIONS · 152 unmodified lines | |||
| 153 | 153 | ||
| 154 | STABS_DEBUG | 154 | STABS_DEBUG |
| 155 | DWARF_DEBUG | 155 | DWARF_DEBUG |
| 156 | MODINFO | ||
| 156 | ARM_DETAILS | 157 | ARM_DETAILS |
| 157 | 158 | ||
| 158 | ARM_ASSERTS | 159 | ARM_ASSERTS |
| @@ -91,8 +91,9 @@ __XCHG_GEN(_mb) · 90 unmodified lines | |||
| 91 | #define __xchg_wrapper(sfx, ptr, x) \ | 91 | #define __xchg_wrapper(sfx, ptr, x) \ |
| 92 | ({ \ | 92 | ({ \ |
| 93 | __typeof__(*(ptr)) __ret; \ | 93 | __typeof__(*(ptr)) __ret; \ |
| 94 | __ret = (__typeof__(*(ptr))) \ | 94 | __ret = (__force __typeof__(*(ptr))) \ |
| 95 | __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ | 95 | __arch_xchg##sfx((__force unsigned long)(x), (ptr), \ |
| 96 | sizeof(*(ptr))); \ | ||
| 96 | __ret; \ | 97 | __ret; \ |
| 97 | }) | 98 | }) |
| 98 | 99 | ||
| @@ -175,9 +176,10 @@ __CMPXCHG_GEN(_mb) · 76 unmodified lines | |||
| 175 | #define __cmpxchg_wrapper(sfx, ptr, o, n) \ | 176 | #define __cmpxchg_wrapper(sfx, ptr, o, n) \ |
| 176 | ({ \ | 177 | ({ \ |
| 177 | __typeof__(*(ptr)) __ret; \ | 178 | __typeof__(*(ptr)) __ret; \ |
| 178 | __ret = (__typeof__(*(ptr))) \ | 179 | __ret = (__force __typeof__(*(ptr))) \ |
| 179 | __cmpxchg##sfx((ptr), (unsigned long)(o), \ | 180 | __cmpxchg##sfx((ptr), (__force unsigned long)(o), \ |
| 180 | (unsigned long)(n), sizeof(*(ptr))); \ | 181 | (__force unsigned long)(n), \ |
| 182 | sizeof(*(ptr))); \ | ||
| 181 | __ret; \ | 183 | __ret; \ |
| 182 | }) | 184 | }) |
| 183 | 185 | ||
| @@ -264,19 +264,33 @@ __iowrite64_copy(void __iomem *to, const void *from, size_t count) · 263 unmodified lines | |||
| 264 | typedef int (*ioremap_prot_hook_t)(phys_addr_t phys_addr, size_t size, | 264 | typedef int (*ioremap_prot_hook_t)(phys_addr_t phys_addr, size_t size, |
| 265 | pgprot_t *prot); | 265 | pgprot_t *prot); |
| 266 | int arm64_ioremap_prot_hook_register(const ioremap_prot_hook_t hook); | 266 | int arm64_ioremap_prot_hook_register(const ioremap_prot_hook_t hook); |
| 267 | void __iomem *__ioremap_prot(phys_addr_t phys, size_t size, pgprot_t prot); | ||
| 267 | 268 | ||
| 268 | #define ioremap_prot ioremap_prot | 269 | static inline void __iomem *ioremap_prot(phys_addr_t phys, size_t size, |
| 270 | pgprot_t user_prot) | ||
| 271 | { | ||
| 272 | pgprot_t prot; | ||
| 273 | ptdesc_t user_prot_val = pgprot_val(user_prot); | ||
| 274 | |||
| 275 | if (WARN_ON_ONCE(!(user_prot_val & PTE_USER))) | ||
| 276 | return NULL; | ||
| 269 | 277 | ||
| 270 | #define _PAGE_IOREMAP PROT_DEVICE_nGnRE | 278 | prot = __pgprot_modify(PAGE_KERNEL, PTE_ATTRINDX_MASK, |
| 279 | user_prot_val & PTE_ATTRINDX_MASK); | ||
| 280 | return __ioremap_prot(phys, size, prot); | ||
| 281 | } | ||
| 282 | #define ioremap_prot ioremap_prot | ||
| 271 | 283 | ||
| 284 | #define ioremap(addr, size) \ | ||
| 285 | __ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) | ||
| 272 | #define ioremap_wc(addr, size) \ | 286 | #define ioremap_wc(addr, size) \ |
| 273 | ioremap_prot((addr), (size), __pgprot(PROT_NORMAL_NC)) | 287 | __ioremap_prot((addr), (size), __pgprot(PROT_NORMAL_NC)) |
| 274 | #define ioremap_np(addr, size) \ | 288 | #define ioremap_np(addr, size) \ |
| 275 | ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRnE)) | 289 | __ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRnE)) |
| 276 | 290 | ||
| 277 | 291 | ||
| 278 | #define ioremap_encrypted(addr, size) \ | 292 | #define ioremap_encrypted(addr, size) \ |
| 279 | ioremap_prot((addr), (size), PAGE_KERNEL) | 293 | __ioremap_prot((addr), (size), PAGE_KERNEL) |
| 280 | 294 | ||
| 281 | /* | 295 | /* |
| 282 | * io{read,write}{16,32,64}be() macros | 296 | * io{read,write}{16,32,64}be() macros |
| @@ -297,7 +311,7 @@ static inline void __iomem *ioremap_cache(phys_addr_t addr, size_t size) · 14 unmodified lines | |||
| 297 | if (pfn_is_map_memory(__phys_to_pfn(addr))) | 311 | if (pfn_is_map_memory(__phys_to_pfn(addr))) |
| 298 | return (void __iomem *)__phys_to_virt(addr); | 312 | return (void __iomem *)__phys_to_virt(addr); |
| 299 | 313 | ||
| 300 | return ioremap_prot(addr, size, __pgprot(PROT_NORMAL)); | 314 | return __ioremap_prot(addr, size, __pgprot(PROT_NORMAL)); |
| 301 | } | 315 | } |
| 302 | 316 | ||
| 303 | /* | 317 | /* |
| @@ -784,6 +784,9 @@ struct kvm_host_data { · 783 unmodified lines | |||
| 784 | /* Number of debug breakpoints/watchpoints for this CPU (minus 1) */ | 784 | /* Number of debug breakpoints/watchpoints for this CPU (minus 1) */ |
| 785 | unsigned int debug_brps; | 785 | unsigned int debug_brps; |
| 786 | unsigned int debug_wrps; | 786 | unsigned int debug_wrps; |
| 787 | |||
| 788 | /* Last vgic_irq part of the AP list recorded in an LR */ | ||
| 789 | struct vgic_irq *last_lr_irq; | ||
| 787 | }; | 790 | }; |
| 788 | 791 | ||
| 789 | struct kvm_host_psci_config { | 792 | struct kvm_host_psci_config { |
| @@ -1616,7 +1619,8 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); · 826 unmodified lines | |||
| 1616 | (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1PIE, IMP)) | 1619 | (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1PIE, IMP)) |
| 1617 | 1620 | ||
| 1618 | #define kvm_has_s1poe(k) \ | 1621 | #define kvm_has_s1poe(k) \ |
| 1619 | (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) | 1622 | (system_supports_poe() && \ |
| 1623 | kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) | ||
| 1620 | 1624 | ||
| 1621 | #define kvm_has_ras(k) \ | 1625 | #define kvm_has_ras(k) \ |
| 1622 | (kvm_has_feat((k), ID_AA64PFR0_EL1, RAS, IMP)) | 1626 | (kvm_has_feat((k), ID_AA64PFR0_EL1, RAS, IMP)) |
| @@ -397,6 +397,8 @@ int kvm_vcpu_allocate_vncr_tlb(struct kvm_vcpu *vcpu); · 396 unmodified lines | |||
| 397 | int kvm_handle_vncr_abort(struct kvm_vcpu *vcpu); | 397 | int kvm_handle_vncr_abort(struct kvm_vcpu *vcpu); |
| 398 | void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val); | 398 | void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val); |
| 399 | 399 | ||
| 400 | u16 get_asid_by_regime(struct kvm_vcpu *vcpu, enum trans_regime regime); | ||
| 401 | |||
| 400 | #define vncr_fixmap(c) \ | 402 | #define vncr_fixmap(c) \ |
| 401 | ({ \ | 403 | ({ \ |
| 402 | u32 __c = (c); \ | 404 | u32 __c = (c); \ |
| @@ -50,11 +50,11 @@ · 49 unmodified lines | |||
| 50 | 50 | ||
| 51 | #define _PAGE_DEFAULT (_PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) | 51 | #define _PAGE_DEFAULT (_PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) |
| 52 | 52 | ||
| 53 | #define _PAGE_KERNEL (PROT_NORMAL) | 53 | #define _PAGE_KERNEL (PROT_NORMAL | PTE_DIRTY) |
| 54 | #define _PAGE_KERNEL_RO ((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) | 54 | #define _PAGE_KERNEL_RO ((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY | PTE_DIRTY) |
| 55 | #define _PAGE_KERNEL_ROX ((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) | 55 | #define _PAGE_KERNEL_ROX ((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY | PTE_DIRTY) |
| 56 | #define _PAGE_KERNEL_EXEC (PROT_NORMAL & ~PTE_PXN) | 56 | #define _PAGE_KERNEL_EXEC ((PROT_NORMAL & ~PTE_PXN) | PTE_DIRTY) |
| 57 | #define _PAGE_KERNEL_EXEC_CONT ((PROT_NORMAL & ~PTE_PXN) | PTE_CONT) | 57 | #define _PAGE_KERNEL_EXEC_CONT ((PROT_NORMAL & ~PTE_PXN) | PTE_CONT | PTE_DIRTY) |
| 58 | 58 | ||
| 59 | #define _PAGE_SHARED (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) | 59 | #define _PAGE_SHARED (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) |
| 60 | #define _PAGE_SHARED_EXEC (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_WRITE) | 60 | #define _PAGE_SHARED_EXEC (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_WRITE) |
| @@ -164,9 +164,6 @@ static inline bool __pure lpa2_is_enabled(void) · 103 unmodified lines | |||
| 164 | #define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) | 164 | #define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) |
| 165 | #define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) | 165 | #define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) |
| 166 | 166 | ||
| 167 | #define PAGE_GCS __pgprot(_PAGE_GCS) | ||
| 168 | #define PAGE_GCS_RO __pgprot(_PAGE_GCS_RO) | ||
| 169 | |||
| 170 | #define PIE_E0 ( \ | 167 | #define PIE_E0 ( \ |
| 171 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ | 168 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ |
| 172 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ | 169 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ |
| @@ -2,6 +2,10 @@ · 1 unmodified line | |||
| 2 | #ifndef _ASM_RUNTIME_CONST_H | 2 | #ifndef _ASM_RUNTIME_CONST_H |
| 3 | #define _ASM_RUNTIME_CONST_H | 3 | #define _ASM_RUNTIME_CONST_H |
| 4 | 4 | ||
| 5 | #ifdef MODULE | ||
| 6 | #error "Cannot use runtime-const infrastructure from modules" | ||
| 7 | #endif | ||
| 8 | |||
| 5 | #include <asm/cacheflush.h> | 9 | #include <asm/cacheflush.h> |
| 6 | 10 | ||
| 7 | /* Sigh. You can still run arm64 in BE mode */ | 11 | /* Sigh. You can still run arm64 in BE mode */ |
| @@ -31,19 +31,11 @@ · 30 unmodified lines | |||
| 31 | */ | 31 | */ |
| 32 | #define __TLBI_0(op, arg) asm (ARM64_ASM_PREAMBLE \ | 32 | #define __TLBI_0(op, arg) asm (ARM64_ASM_PREAMBLE \ |
| 33 | "tlbi " #op "\n" \ | 33 | "tlbi " #op "\n" \ |
| 34 | ALTERNATIVE("nop\n nop", \ | ||
| 35 | "dsb ish\n tlbi " #op, \ | ||
| 36 | ARM64_WORKAROUND_REPEAT_TLBI, \ | ||
| 37 | CONFIG_ARM64_WORKAROUND_REPEAT_TLBI) \ | ||
| 38 | : : ) | 34 | : : ) |
| 39 | 35 | ||
| 40 | #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE \ | 36 | #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE \ |
| 41 | "tlbi " #op ", %0\n" \ | 37 | "tlbi " #op ", %x0\n" \ |
| 42 | ALTERNATIVE("nop\n nop", \ | 38 | : : "rZ" (arg)) |
| 43 | "dsb ish\n tlbi " #op ", %0", \ | ||
| 44 | ARM64_WORKAROUND_REPEAT_TLBI, \ | ||
| 45 | CONFIG_ARM64_WORKAROUND_REPEAT_TLBI) \ | ||
| 46 | : : "r" (arg)) | ||
| 47 | 39 | ||
| 48 | #define __TLBI_N(op, arg, n, ...) __TLBI_##n(op, arg) | 40 | #define __TLBI_N(op, arg, n, ...) __TLBI_##n(op, arg) |
| 49 | 41 | ||
| @@ -181,6 +173,34 @@ static inline unsigned long get_trans_granule(void) · 131 unmodified lines | |||
| 181 | (__pages >> (5 * (scale) + 1)) - 1; \ | 173 | (__pages >> (5 * (scale) + 1)) - 1; \ |
| 182 | }) | 174 | }) |
| 183 | 175 | ||
| 176 | #define __repeat_tlbi_sync(op, arg...) \ | ||
| 177 | do { \ | ||
| 178 | if (!alternative_has_cap_unlikely(ARM64_WORKAROUND_REPEAT_TLBI)) \ | ||
| 179 | break; \ | ||
| 180 | __tlbi(op, ##arg); \ | ||
| 181 | dsb(ish); \ | ||
| 182 | } while (0) | ||
| 183 | |||
| 184 | /* | ||
| 185 | * Complete broadcast TLB maintenance issued by the host which invalidates | ||
| 186 | * stage 1 information in the host's own translation regime. | ||
| 187 | */ | ||
| 188 | static inline void __tlbi_sync_s1ish(void) | ||
| 189 | { | ||
| 190 | dsb(ish); | ||
| 191 | __repeat_tlbi_sync(vale1is, 0); | ||
| 192 | } | ||
| 193 | |||
| 194 | /* | ||
| 195 | * Complete broadcast TLB maintenance issued by hyp code which invalidates | ||
| 196 | * stage 1 translation information in any translation regime. | ||
| 197 | */ | ||
| 198 | static inline void __tlbi_sync_s1ish_hyp(void) | ||
| 199 | { | ||
| 200 | dsb(ish); | ||
| 201 | __repeat_tlbi_sync(vale2is, 0); | ||
| 202 | } | ||
| 203 | |||
| 184 | /* | 204 | /* |
| 185 | * TLB Invalidation | 205 | * TLB Invalidation |
| 186 | * ================ | 206 | * ================ |
| @@ -279,7 +299,7 @@ static inline void flush_tlb_all(void) · 92 unmodified lines | |||
| 279 | { | 299 | { |
| 280 | dsb(ishst); | 300 | dsb(ishst); |
| 281 | __tlbi(vmalle1is); | 301 | __tlbi(vmalle1is); |
| 282 | dsb(ish); | 302 | __tlbi_sync_s1ish(); |
| 283 | isb(); | 303 | isb(); |
| 284 | } | 304 | } |
| 285 | 305 | ||
| @@ -291,7 +311,7 @@ static inline void flush_tlb_mm(struct mm_struct *mm) · 5 unmodified lines | |||
| 291 | asid = __TLBI_VADDR(0, ASID(mm)); | 311 | asid = __TLBI_VADDR(0, ASID(mm)); |
| 292 | __tlbi(aside1is, asid); | 312 | __tlbi(aside1is, asid); |
| 293 | __tlbi_user(aside1is, asid); | 313 | __tlbi_user(aside1is, asid); |
| 294 | dsb(ish); | 314 | __tlbi_sync_s1ish(); |
| 295 | mmu_notifier_arch_invalidate_secondary_tlbs(mm, 0, -1UL); | 315 | mmu_notifier_arch_invalidate_secondary_tlbs(mm, 0, -1UL); |
| 296 | } | 316 | } |
| 297 | 317 | ||
| @@ -345,20 +365,11 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, · 47 unmodified lines | |||
| 345 | unsigned long uaddr) | 365 | unsigned long uaddr) |
| 346 | { | 366 | { |
| 347 | flush_tlb_page_nosync(vma, uaddr); | 367 | flush_tlb_page_nosync(vma, uaddr); |
| 348 | dsb(ish); | 368 | __tlbi_sync_s1ish(); |
| 349 | } | 369 | } |
| 350 | 370 | ||
| 351 | static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) | 371 | static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) |
| 352 | { | 372 | { |
| 353 | /* | ||
| 354 | * TLB flush deferral is not required on systems which are affected by | ||
| 355 | * ARM64_WORKAROUND_REPEAT_TLBI, as __tlbi()/__tlbi_user() implementation | ||
| 356 | * will have two consecutive TLBI instructions with a dsb(ish) in between | ||
| 357 | * defeating the purpose (i.e save overall 'dsb ish' cost). | ||
| 358 | */ | ||
| 359 | if (alternative_has_cap_unlikely(ARM64_WORKAROUND_REPEAT_TLBI)) | ||
| 360 | return false; | ||
| 361 | |||
| 362 | return true; | 373 | return true; |
| 363 | } | 374 | } |
| 364 | 375 | ||
| @@ -374,7 +385,7 @@ static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) · 9 unmodified lines | |||
| 374 | */ | 385 | */ |
| 375 | static inline void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) | 386 | static inline void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) |
| 376 | { | 387 | { |
| 377 | dsb(ish); | 388 | __tlbi_sync_s1ish(); |
| 378 | } | 389 | } |
| 379 | 390 | ||
| 380 | /* | 391 | /* |
| @@ -509,7 +520,7 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, · 128 unmodified lines | |||
| 509 | { | 520 | { |
| 510 | __flush_tlb_range_nosync(vma->vm_mm, start, end, stride, | 521 | __flush_tlb_range_nosync(vma->vm_mm, start, end, stride, |
| 511 | last_level, tlb_level); | 522 | last_level, tlb_level); |
| 512 | dsb(ish); | 523 | __tlbi_sync_s1ish(); |
| 513 | } | 524 | } |
| 514 | 525 | ||
| 515 | static inline void local_flush_tlb_contpte(struct vm_area_struct *vma, | 526 | static inline void local_flush_tlb_contpte(struct vm_area_struct *vma, |
| @@ -557,7 +568,7 @@ static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end · 41 unmodified lines | |||
| 557 | dsb(ishst); | 568 | dsb(ishst); |
| 558 | __flush_tlb_range_op(vaale1is, start, pages, stride, 0, | 569 | __flush_tlb_range_op(vaale1is, start, pages, stride, 0, |
| 559 | TLBI_TTL_UNKNOWN, false, lpa2_is_enabled()); | 570 | TLBI_TTL_UNKNOWN, false, lpa2_is_enabled()); |
| 560 | dsb(ish); | 571 | __tlbi_sync_s1ish(); |
| 561 | isb(); | 572 | isb(); |
| 562 | } | 573 | } |
| 563 | 574 | ||
| @@ -571,7 +582,7 @@ static inline void __flush_tlb_kernel_pgtable(unsigned long kaddr) · 7 unmodified lines | |||
| 571 | 582 | ||
| 572 | dsb(ishst); | 583 | dsb(ishst); |
| 573 | __tlbi(vaae1is, addr); | 584 | __tlbi(vaae1is, addr); |
| 574 | dsb(ish); | 585 | __tlbi_sync_s1ish(); |
| 575 | isb(); | 586 | isb(); |
| 576 | } | 587 | } |
| 577 | 588 | ||
| @@ -377,7 +377,7 @@ void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size) · 376 unmodified lines | |||
| 377 | prot = __acpi_get_writethrough_mem_attribute(); | 377 | prot = __acpi_get_writethrough_mem_attribute(); |
| 378 | } | 378 | } |
| 379 | } | 379 | } |
| 380 | return ioremap_prot(phys, size, prot); | 380 | return __ioremap_prot(phys, size, prot); |
| 381 | } | 381 | } |
| 382 | 382 | ||
| 383 | /* | 383 | /* |
| @@ -2345,6 +2345,15 @@ static bool can_trap_icv_dir_el1(const struct arm64_cpu_capabilities *entry, · 2344 unmodified lines | |||
| 2345 | !is_midr_in_range_list(has_vgic_v3)) | 2345 | !is_midr_in_range_list(has_vgic_v3)) |
| 2346 | return false; | 2346 | return false; |
| 2347 | 2347 | ||
| 2348 | /* | ||
| 2349 | * pKVM prevents late onlining of CPUs. This means that whatever | ||
| 2350 | * state the capability is in after deprivilege cannot be affected | ||
| 2351 | * by a new CPU booting -- this is garanteed to be a CPU we have | ||
| 2352 | * already seen, and the cap is therefore unchanged. | ||
| 2353 | */ | ||
| 2354 | if (system_capabilities_finalized() && is_protected_kvm_enabled()) | ||
| 2355 | return cpus_have_final_cap(ARM64_HAS_ICH_HCR_EL2_TDIR); | ||
| 2356 | |||
| 2348 | if (is_kernel_in_hyp_mode()) | 2357 | if (is_kernel_in_hyp_mode()) |
| 2349 | res.a1 = read_sysreg_s(SYS_ICH_VTR_EL2); | 2358 | res.a1 = read_sysreg_s(SYS_ICH_VTR_EL2); |
| 2350 | else | 2359 | else |
| @@ -37,7 +37,7 @@ __do_compat_cache_op(unsigned long start, unsigned long end) · 36 unmodified lines | |||
| 37 | * We pick the reserved-ASID to minimise the impact. | 37 | * We pick the reserved-ASID to minimise the impact. |
| 38 | */ | 38 | */ |
| 39 | __tlbi(aside1is, __TLBI_VADDR(0, 0)); | 39 | __tlbi(aside1is, __TLBI_VADDR(0, 0)); |
| 40 | dsb(ish); | 40 | __tlbi_sync_s1ish(); |
| 41 | } | 41 | } |
| 42 | 42 | ||
| 43 | ret = caches_clean_inval_user_pou(start, start + chunk); | 43 | ret = caches_clean_inval_user_pou(start, start + chunk); |
| @@ -400,16 +400,25 @@ static inline · 399 unmodified lines | |||
| 400 | int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val) | 400 | int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val) |
| 401 | { | 401 | { |
| 402 | /* | 402 | /* |
| 403 | * Abort call on counterless CPU or when interrupts are | 403 | * Abort call on counterless CPU. |
| 404 | * disabled - can lead to deadlock in smp sync call. | ||
| 405 | */ | 404 | */ |
| 406 | if (!cpu_has_amu_feat(cpu)) | 405 | if (!cpu_has_amu_feat(cpu)) |
| 407 | return -EOPNOTSUPP; | 406 | return -EOPNOTSUPP; |
| 408 | 407 | ||
| 409 | if (WARN_ON_ONCE(irqs_disabled())) | 408 | if (irqs_disabled()) { |
| 410 | return -EPERM; | 409 | /* |
| 411 | 410 | * When IRQs are disabled (tick path: sched_tick -> | |
| 412 | smp_call_function_single(cpu, func, val, 1); | 411 | * topology_scale_freq_tick or cppc_scale_freq_tick), only local |
| 412 | * CPU counter reads are allowed. Remote CPU counter read would | ||
| 413 | * require smp_call_function_single() which is unsafe with IRQs | ||
| 414 | * disabled. | ||
| 415 | */ | ||
| 416 | if (WARN_ON_ONCE(cpu != smp_processor_id())) | ||
| 417 | return -EPERM; | ||
| 418 | func(val); | ||
| 419 | } else { | ||
| 420 | smp_call_function_single(cpu, func, val, 1); | ||
| 421 | } | ||
| 413 | 422 | ||
| 414 | return 0; | 423 | return 0; |
| 415 | } | 424 | } |
| @@ -349,6 +349,7 @@ SECTIONS · 348 unmodified lines | |||
| 349 | 349 | ||
| 350 | STABS_DEBUG | 350 | STABS_DEBUG |
| 351 | DWARF_DEBUG | 351 | DWARF_DEBUG |
| 352 | MODINFO | ||
| 352 | ELF_DETAILS | 353 | ELF_DETAILS |
| 353 | 354 | ||
| 354 | HEAD_SYMBOLS | 355 | HEAD_SYMBOLS |
| @@ -21,7 +21,6 @@ menuconfig KVM · 20 unmodified lines | |||
| 21 | bool "Kernel-based Virtual Machine (KVM) support" | 21 | bool "Kernel-based Virtual Machine (KVM) support" |
| 22 | select KVM_COMMON | 22 | select KVM_COMMON |
| 23 | select KVM_GENERIC_HARDWARE_ENABLING | 23 | select KVM_GENERIC_HARDWARE_ENABLING |
| 24 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 25 | select HAVE_KVM_CPU_RELAX_INTERCEPT | 24 | select HAVE_KVM_CPU_RELAX_INTERCEPT |
| 26 | select KVM_MMIO | 25 | select KVM_MMIO |
| 27 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT | 26 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| @@ -358,7 +358,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 357 unmodified lines | |||
| 358 | break; | 358 | break; |
| 359 | case KVM_CAP_IOEVENTFD: | 359 | case KVM_CAP_IOEVENTFD: |
| 360 | case KVM_CAP_USER_MEMORY: | 360 | case KVM_CAP_USER_MEMORY: |
| 361 | case KVM_CAP_SYNC_MMU: | ||
| 362 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: | 361 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: |
| 363 | case KVM_CAP_ONE_REG: | 362 | case KVM_CAP_ONE_REG: |
| 364 | case KVM_CAP_ARM_PSCI: | 363 | case KVM_CAP_ARM_PSCI: |
| @@ -540,31 +540,8 @@ static int walk_s1(struct kvm_vcpu *vcpu, struct s1_walk_info *wi, · 539 unmodified lines | |||
| 540 | wr->pa |= va & GENMASK_ULL(va_bottom - 1, 0); | 540 | wr->pa |= va & GENMASK_ULL(va_bottom - 1, 0); |
| 541 | 541 | ||
| 542 | wr->nG = (wi->regime != TR_EL2) && (desc & PTE_NG); | 542 | wr->nG = (wi->regime != TR_EL2) && (desc & PTE_NG); |
| 543 | if (wr->nG) { | 543 | if (wr->nG) |
| 544 | u64 asid_ttbr, tcr; | 544 | wr->asid = get_asid_by_regime(vcpu, wi->regime); |
| 545 | |||
| 546 | switch (wi->regime) { | ||
| 547 | case TR_EL10: | ||
| 548 | tcr = vcpu_read_sys_reg(vcpu, TCR_EL1); | ||
| 549 | asid_ttbr = ((tcr & TCR_A1) ? | ||
| 550 | vcpu_read_sys_reg(vcpu, TTBR1_EL1) : | ||
| 551 | vcpu_read_sys_reg(vcpu, TTBR0_EL1)); | ||
| 552 | break; | ||
| 553 | case TR_EL20: | ||
| 554 | tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | ||
| 555 | asid_ttbr = ((tcr & TCR_A1) ? | ||
| 556 | vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | ||
| 557 | vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | ||
| 558 | break; | ||
| 559 | default: | ||
| 560 | BUG(); | ||
| 561 | } | ||
| 562 | |||
| 563 | wr->asid = FIELD_GET(TTBR_ASID_MASK, asid_ttbr); | ||
| 564 | if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | ||
| 565 | !(tcr & TCR_ASID16)) | ||
| 566 | wr->asid &= GENMASK(7, 0); | ||
| 567 | } | ||
| 568 | 545 | ||
| 569 | return 0; | 546 | return 0; |
| 570 | 547 | ||
| @@ -1527,8 +1504,6 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) · 956 unmodified lines | |||
| 1527 | fail = true; | 1504 | fail = true; |
| 1528 | } | 1505 | } |
| 1529 | 1506 | ||
| 1530 | isb(); | ||
| 1531 | |||
| 1532 | if (!fail) | 1507 | if (!fail) |
| 1533 | par = read_sysreg_par(); | 1508 | par = read_sysreg_par(); |
| 1534 | 1509 | ||
| @@ -29,7 +29,7 @@ · 28 unmodified lines | |||
| 29 | 29 | ||
| 30 | #include "trace.h" | 30 | #include "trace.h" |
| 31 | 31 | ||
| 32 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 32 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 33 | KVM_GENERIC_VM_STATS() | 33 | KVM_GENERIC_VM_STATS() |
| 34 | }; | 34 | }; |
| 35 | 35 | ||
| @@ -42,7 +42,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 6 unmodified lines | |||
| 42 | sizeof(kvm_vm_stats_desc), | 42 | sizeof(kvm_vm_stats_desc), |
| 43 | }; | 43 | }; |
| 44 | 44 | ||
| 45 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 45 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 46 | KVM_GENERIC_VCPU_STATS(), | 46 | KVM_GENERIC_VCPU_STATS(), |
| 47 | STATS_DESC_COUNTER(VCPU, hvc_exit_stat), | 47 | STATS_DESC_COUNTER(VCPU, hvc_exit_stat), |
| 48 | STATS_DESC_COUNTER(VCPU, wfe_exit_stat), | 48 | STATS_DESC_COUNTER(VCPU, wfe_exit_stat), |
| @@ -518,7 +518,7 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) · 517 unmodified lines | |||
| 518 | granule = kvm_granule_size(level); | 518 | granule = kvm_granule_size(level); |
| 519 | cur.start = ALIGN_DOWN(addr, granule); | 519 | cur.start = ALIGN_DOWN(addr, granule); |
| 520 | cur.end = cur.start + granule; | 520 | cur.end = cur.start + granule; |
| 521 | if (!range_included(&cur, range)) | 521 | if (!range_included(&cur, range) && level < KVM_PGTABLE_LAST_LEVEL) |
| 522 | continue; | 522 | continue; |
| 523 | *range = cur; | 523 | *range = cur; |
| 524 | return 0; | 524 | return 0; |
| @@ -271,7 +271,7 @@ static void fixmap_clear_slot(struct hyp_fixmap_slot *slot) · 270 unmodified lines | |||
| 271 | */ | 271 | */ |
| 272 | dsb(ishst); | 272 | dsb(ishst); |
| 273 | __tlbi_level(vale2is, __TLBI_VADDR(addr, 0), level); | 273 | __tlbi_level(vale2is, __TLBI_VADDR(addr, 0), level); |
| 274 | dsb(ish); | 274 | __tlbi_sync_s1ish_hyp(); |
| 275 | isb(); | 275 | isb(); |
| 276 | } | 276 | } |
| 277 | 277 | ||
| @@ -342,6 +342,7 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc · 341 unmodified lines | |||
| 342 | /* No restrictions for non-protected VMs. */ | 342 | /* No restrictions for non-protected VMs. */ |
| 343 | if (!kvm_vm_is_protected(kvm)) { | 343 | if (!kvm_vm_is_protected(kvm)) { |
| 344 | hyp_vm->kvm.arch.flags = host_arch_flags; | 344 | hyp_vm->kvm.arch.flags = host_arch_flags; |
| 345 | hyp_vm->kvm.arch.flags &= ~BIT_ULL(KVM_ARCH_FLAG_ID_REGS_INITIALIZED); | ||
| 345 | 346 | ||
| 346 | bitmap_copy(kvm->arch.vcpu_features, | 347 | bitmap_copy(kvm->arch.vcpu_features, |
| 347 | host_kvm->arch.vcpu_features, | 348 | host_kvm->arch.vcpu_features, |
| @@ -391,7 +392,7 @@ static void unpin_host_sve_state(struct pkvm_hyp_vcpu *hyp_vcpu) · 43 unmodified lines | |||
| 391 | if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE)) | 392 | if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE)) |
| 392 | return; | 393 | return; |
| 393 | 394 | ||
| 394 | sve_state = kern_hyp_va(hyp_vcpu->vcpu.arch.sve_state); | 395 | sve_state = hyp_vcpu->vcpu.arch.sve_state; |
| 395 | hyp_unpin_shared_mem(sve_state, | 396 | hyp_unpin_shared_mem(sve_state, |
| 396 | sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); | 397 | sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); |
| 397 | } | 398 | } |
| @@ -471,6 +472,35 @@ err: · 73 unmodified lines | |||
| 471 | return ret; | 472 | return ret; |
| 472 | } | 473 | } |
| 473 | 474 | ||
| 475 | static int vm_copy_id_regs(struct pkvm_hyp_vcpu *hyp_vcpu) | ||
| 476 | { | ||
| 477 | struct pkvm_hyp_vm *hyp_vm = pkvm_hyp_vcpu_to_hyp_vm(hyp_vcpu); | ||
| 478 | const struct kvm *host_kvm = hyp_vm->host_kvm; | ||
| 479 | struct kvm *kvm = &hyp_vm->kvm; | ||
| 480 | |||
| 481 | if (!test_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &host_kvm->arch.flags)) | ||
| 482 | return -EINVAL; | ||
| 483 | |||
| 484 | if (test_and_set_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &kvm->arch.flags)) | ||
| 485 | return 0; | ||
| 486 | |||
| 487 | memcpy(kvm->arch.id_regs, host_kvm->arch.id_regs, sizeof(kvm->arch.id_regs)); | ||
| 488 | |||
| 489 | return 0; | ||
| 490 | } | ||
| 491 | |||
| 492 | static int pkvm_vcpu_init_sysregs(struct pkvm_hyp_vcpu *hyp_vcpu) | ||
| 493 | { | ||
| 494 | int ret = 0; | ||
| 495 | |||
| 496 | if (pkvm_hyp_vcpu_is_protected(hyp_vcpu)) | ||
| 497 | kvm_init_pvm_id_regs(&hyp_vcpu->vcpu); | ||
| 498 | else | ||
| 499 | ret = vm_copy_id_regs(hyp_vcpu); | ||
| 500 | |||
| 501 | return ret; | ||
| 502 | } | ||
| 503 | |||
| 474 | static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, | 504 | static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, |
| 475 | struct pkvm_hyp_vm *hyp_vm, | 505 | struct pkvm_hyp_vm *hyp_vm, |
| 476 | struct kvm_vcpu *host_vcpu) | 506 | struct kvm_vcpu *host_vcpu) |
| @@ -490,8 +520,9 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, · 13 unmodified lines | |||
| 490 | hyp_vcpu->vcpu.arch.cflags = READ_ONCE(host_vcpu->arch.cflags); | 520 | hyp_vcpu->vcpu.arch.cflags = READ_ONCE(host_vcpu->arch.cflags); |
| 491 | hyp_vcpu->vcpu.arch.mp_state.mp_state = KVM_MP_STATE_STOPPED; | 521 | hyp_vcpu->vcpu.arch.mp_state.mp_state = KVM_MP_STATE_STOPPED; |
| 492 | 522 | ||
| 493 | if (pkvm_hyp_vcpu_is_protected(hyp_vcpu)) | 523 | ret = pkvm_vcpu_init_sysregs(hyp_vcpu); |
| 494 | kvm_init_pvm_id_regs(&hyp_vcpu->vcpu); | 524 | if (ret) |
| 525 | goto done; | ||
| 495 | 526 | ||
| 496 | ret = pkvm_vcpu_init_traps(hyp_vcpu); | 527 | ret = pkvm_vcpu_init_traps(hyp_vcpu); |
| 497 | if (ret) | 528 | if (ret) |
| @@ -169,7 +169,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, · 168 unmodified lines | |||
| 169 | */ | 169 | */ |
| 170 | dsb(ish); | 170 | dsb(ish); |
| 171 | __tlbi(vmalle1is); | 171 | __tlbi(vmalle1is); |
| 172 | dsb(ish); | 172 | __tlbi_sync_s1ish_hyp(); |
| 173 | isb(); | 173 | isb(); |
| 174 | 174 | ||
| 175 | exit_vmid_context(&cxt); | 175 | exit_vmid_context(&cxt); |
| @@ -226,7 +226,7 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · 50 unmodified lines | |||
| 226 | 226 | ||
| 227 | dsb(ish); | 227 | dsb(ish); |
| 228 | __tlbi(vmalle1is); | 228 | __tlbi(vmalle1is); |
| 229 | dsb(ish); | 229 | __tlbi_sync_s1ish_hyp(); |
| 230 | isb(); | 230 | isb(); |
| 231 | 231 | ||
| 232 | exit_vmid_context(&cxt); | 232 | exit_vmid_context(&cxt); |
| @@ -240,7 +240,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 7 unmodified lines | |||
| 240 | enter_vmid_context(mmu, &cxt, false); | 240 | enter_vmid_context(mmu, &cxt, false); |
| 241 | 241 | ||
| 242 | __tlbi(vmalls12e1is); | 242 | __tlbi(vmalls12e1is); |
| 243 | dsb(ish); | 243 | __tlbi_sync_s1ish_hyp(); |
| 244 | isb(); | 244 | isb(); |
| 245 | 245 | ||
| 246 | exit_vmid_context(&cxt); | 246 | exit_vmid_context(&cxt); |
| @@ -266,5 +266,5 @@ void __kvm_flush_vm_context(void) · 19 unmodified lines | |||
| 266 | /* Same remark as in enter_vmid_context() */ | 266 | /* Same remark as in enter_vmid_context() */ |
| 267 | dsb(ish); | 267 | dsb(ish); |
| 268 | __tlbi(alle1is); | 268 | __tlbi(alle1is); |
| 269 | dsb(ish); | 269 | __tlbi_sync_s1ish_hyp(); |
| 270 | } | 270 | } |
| @@ -501,7 +501,7 @@ static int hyp_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx, · 500 unmodified lines | |||
| 501 | *unmapped += granule; | 501 | *unmapped += granule; |
| 502 | } | 502 | } |
| 503 | 503 | ||
| 504 | dsb(ish); | 504 | __tlbi_sync_s1ish_hyp(); |
| 505 | isb(); | 505 | isb(); |
| 506 | mm_ops->put_page(ctx->ptep); | 506 | mm_ops->put_page(ctx->ptep); |
| 507 | 507 | ||
| @@ -115,7 +115,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, · 114 unmodified lines | |||
| 115 | */ | 115 | */ |
| 116 | dsb(ish); | 116 | dsb(ish); |
| 117 | __tlbi(vmalle1is); | 117 | __tlbi(vmalle1is); |
| 118 | dsb(ish); | 118 | __tlbi_sync_s1ish_hyp(); |
| 119 | isb(); | 119 | isb(); |
| 120 | 120 | ||
| 121 | exit_vmid_context(&cxt); | 121 | exit_vmid_context(&cxt); |
| @@ -176,7 +176,7 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · 54 unmodified lines | |||
| 176 | 176 | ||
| 177 | dsb(ish); | 177 | dsb(ish); |
| 178 | __tlbi(vmalle1is); | 178 | __tlbi(vmalle1is); |
| 179 | dsb(ish); | 179 | __tlbi_sync_s1ish_hyp(); |
| 180 | isb(); | 180 | isb(); |
| 181 | 181 | ||
| 182 | exit_vmid_context(&cxt); | 182 | exit_vmid_context(&cxt); |
| @@ -192,7 +192,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 9 unmodified lines | |||
| 192 | enter_vmid_context(mmu, &cxt); | 192 | enter_vmid_context(mmu, &cxt); |
| 193 | 193 | ||
| 194 | __tlbi(vmalls12e1is); | 194 | __tlbi(vmalls12e1is); |
| 195 | dsb(ish); | 195 | __tlbi_sync_s1ish_hyp(); |
| 196 | isb(); | 196 | isb(); |
| 197 | 197 | ||
| 198 | exit_vmid_context(&cxt); | 198 | exit_vmid_context(&cxt); |
| @@ -217,7 +217,7 @@ void __kvm_flush_vm_context(void) · 18 unmodified lines | |||
| 217 | { | 217 | { |
| 218 | dsb(ishst); | 218 | dsb(ishst); |
| 219 | __tlbi(alle1is); | 219 | __tlbi(alle1is); |
| 220 | dsb(ish); | 220 | __tlbi_sync_s1ish_hyp(); |
| 221 | } | 221 | } |
| 222 | 222 | ||
| 223 | /* | 223 | /* |
| @@ -358,7 +358,7 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding) · 134 unmodified lines | |||
| 358 | default: | 358 | default: |
| 359 | ret = -EINVAL; | 359 | ret = -EINVAL; |
| 360 | } | 360 | } |
| 361 | dsb(ish); | 361 | __tlbi_sync_s1ish_hyp(); |
| 362 | isb(); | 362 | isb(); |
| 363 | 363 | ||
| 364 | if (mmu) | 364 | if (mmu) |
| @@ -1751,17 +1751,16 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 1750 unmodified lines | |||
| 1751 | 1751 | ||
| 1752 | force_pte = (max_map_size == PAGE_SIZE); | 1752 | force_pte = (max_map_size == PAGE_SIZE); |
| 1753 | vma_pagesize = min_t(long, vma_pagesize, max_map_size); | 1753 | vma_pagesize = min_t(long, vma_pagesize, max_map_size); |
| 1754 | vma_shift = __ffs(vma_pagesize); | ||
| 1754 | } | 1755 | } |
| 1755 | 1756 | ||
| 1756 | /* | 1757 | /* |
| 1757 | * Both the canonical IPA and fault IPA must be hugepage-aligned to | 1758 | * Both the canonical IPA and fault IPA must be aligned to the |
| 1758 | * ensure we find the right PFN and lay down the mapping in the right | 1759 | * mapping size to ensure we find the right PFN and lay down the |
| 1759 | * place. | 1760 | * mapping in the right place. |
| 1760 | */ | 1761 | */ |
| 1761 | if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE) { | 1762 | fault_ipa = ALIGN_DOWN(fault_ipa, vma_pagesize); |
| 1762 | fault_ipa &= ~(vma_pagesize - 1); | 1763 | ipa = ALIGN_DOWN(ipa, vma_pagesize); |
| 1763 | ipa &= ~(vma_pagesize - 1); | ||
| 1764 | } | ||
| 1765 | 1764 | ||
| 1766 | gfn = ipa >> PAGE_SHIFT; | 1765 | gfn = ipa >> PAGE_SHIFT; |
| 1767 | mte_allowed = kvm_vma_mte_allowed(vma); | 1766 | mte_allowed = kvm_vma_mte_allowed(vma); |
| @@ -1839,10 +1838,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 71 unmodified lines | |||
| 1839 | if (exec_fault && s2_force_noncacheable) | 1838 | if (exec_fault && s2_force_noncacheable) |
| 1840 | ret = -ENOEXEC; | 1839 | ret = -ENOEXEC; |
| 1841 | 1840 | ||
| 1842 | if (ret) { | 1841 | if (ret) |
| 1843 | kvm_release_page_unused(page); | 1842 | goto out_put_page; |
| 1844 | return ret; | ||
| 1845 | } | ||
| 1846 | 1843 | ||
| 1847 | /* | 1844 | /* |
| 1848 | * Guest performs atomic/exclusive operations on memory with unsupported | 1845 | * Guest performs atomic/exclusive operations on memory with unsupported |
| @@ -1852,7 +1849,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 3 unmodified lines | |||
| 1852 | */ | 1849 | */ |
| 1853 | if (esr_fsc_is_excl_atomic_fault(kvm_vcpu_get_esr(vcpu))) { | 1850 | if (esr_fsc_is_excl_atomic_fault(kvm_vcpu_get_esr(vcpu))) { |
| 1854 | kvm_inject_dabt_excl_atomic(vcpu, kvm_vcpu_get_hfar(vcpu)); | 1851 | kvm_inject_dabt_excl_atomic(vcpu, kvm_vcpu_get_hfar(vcpu)); |
| 1855 | return 1; | 1852 | ret = 1; |
| 1853 | goto out_put_page; | ||
| 1856 | } | 1854 | } |
| 1857 | 1855 | ||
| 1858 | if (nested) | 1856 | if (nested) |
| @@ -1938,6 +1936,10 @@ out_unlock: · 79 unmodified lines | |||
| 1938 | mark_page_dirty_in_slot(kvm, memslot, gfn); | 1936 | mark_page_dirty_in_slot(kvm, memslot, gfn); |
| 1939 | 1937 | ||
| 1940 | return ret != -EAGAIN ? ret : 0; | 1938 | return ret != -EAGAIN ? ret : 0; |
| 1939 | |||
| 1940 | out_put_page: | ||
| 1941 | kvm_release_page_unused(page); | ||
| 1942 | return ret; | ||
| 1941 | } | 1943 | } |
| 1942 | 1944 | ||
| 1943 | /* Resolve the access fault by making the page young again. */ | 1945 | /* Resolve the access fault by making the page young again. */ |
| @@ -152,31 +152,31 @@ static int get_ia_size(struct s2_walk_info *wi) · 151 unmodified lines | |||
| 152 | return 64 - wi->t0sz; | 152 | return 64 - wi->t0sz; |
| 153 | } | 153 | } |
| 154 | 154 | ||
| 155 | static int check_base_s2_limits(struct s2_walk_info *wi, | 155 | static int check_base_s2_limits(struct kvm_vcpu *vcpu, struct s2_walk_info *wi, |
| 156 | int level, int input_size, int stride) | 156 | int level, int input_size, int stride) |
| 157 | { | 157 | { |
| 158 | int start_size, ia_size; | 158 | int start_size, pa_max; |
| 159 | 159 | ||
| 160 | ia_size = get_ia_size(wi); | 160 | pa_max = kvm_get_pa_bits(vcpu->kvm); |
| 161 | 161 | ||
| 162 | /* Check translation limits */ | 162 | /* Check translation limits */ |
| 163 | switch (BIT(wi->pgshift)) { | 163 | switch (BIT(wi->pgshift)) { |
| 164 | case SZ_64K: | 164 | case SZ_64K: |
| 165 | if (level == 0 || (level == 1 && ia_size <= 42)) | 165 | if (level == 0 || (level == 1 && pa_max <= 42)) |
| 166 | return -EFAULT; | 166 | return -EFAULT; |
| 167 | break; | 167 | break; |
| 168 | case SZ_16K: | 168 | case SZ_16K: |
| 169 | if (level == 0 || (level == 1 && ia_size <= 40)) | 169 | if (level == 0 || (level == 1 && pa_max <= 40)) |
| 170 | return -EFAULT; | 170 | return -EFAULT; |
| 171 | break; | 171 | break; |
| 172 | case SZ_4K: | 172 | case SZ_4K: |
| 173 | if (level < 0 || (level == 0 && ia_size <= 42)) | 173 | if (level < 0 || (level == 0 && pa_max <= 42)) |
| 174 | return -EFAULT; | 174 | return -EFAULT; |
| 175 | break; | 175 | break; |
| 176 | } | 176 | } |
| 177 | 177 | ||
| 178 | /* Check input size limits */ | 178 | /* Check input size limits */ |
| 179 | if (input_size > ia_size) | 179 | if (input_size > pa_max) |
| 180 | return -EFAULT; | 180 | return -EFAULT; |
| 181 | 181 | ||
| 182 | /* Check number of entries in starting level table */ | 182 | /* Check number of entries in starting level table */ |
| @@ -269,16 +269,19 @@ static int walk_nested_s2_pgd(struct kvm_vcpu *vcpu, phys_addr_t ipa, · 86 unmodified lines | |||
| 269 | if (input_size > 48 || input_size < 25) | 269 | if (input_size > 48 || input_size < 25) |
| 270 | return -EFAULT; | 270 | return -EFAULT; |
| 271 | 271 | ||
| 272 | ret = check_base_s2_limits(wi, level, input_size, stride); | 272 | ret = check_base_s2_limits(vcpu, wi, level, input_size, stride); |
| 273 | if (WARN_ON(ret)) | 273 | if (WARN_ON(ret)) { |
| 274 | out->esr = compute_fsc(0, ESR_ELx_FSC_FAULT); | ||
| 274 | return ret; | 275 | return ret; |
| 276 | } | ||
| 275 | 277 | ||
| 276 | base_lower_bound = 3 + input_size - ((3 - level) * stride + | 278 | base_lower_bound = 3 + input_size - ((3 - level) * stride + |
| 277 | wi->pgshift); | 279 | wi->pgshift); |
| 278 | base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound); | 280 | base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound); |
| 279 | 281 | ||
| 280 | if (check_output_size(wi, base_addr)) { | 282 | if (check_output_size(wi, base_addr)) { |
| 281 | out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ); | 283 | /* R_BFHQH */ |
| 284 | out->esr = compute_fsc(0, ESR_ELx_FSC_ADDRSZ); | ||
| 282 | return 1; | 285 | return 1; |
| 283 | } | 286 | } |
| 284 | 287 | ||
| @@ -293,8 +296,10 @@ static int walk_nested_s2_pgd(struct kvm_vcpu *vcpu, phys_addr_t ipa, · 8 unmodified lines | |||
| 293 | 296 | ||
| 294 | paddr = base_addr | index; | 297 | paddr = base_addr | index; |
| 295 | ret = read_guest_s2_desc(vcpu, paddr, &desc, wi); | 298 | ret = read_guest_s2_desc(vcpu, paddr, &desc, wi); |
| 296 | if (ret < 0) | 299 | if (ret < 0) { |
| 300 | out->esr = ESR_ELx_FSC_SEA_TTW(level); | ||
| 297 | return ret; | 301 | return ret; |
| 302 | } | ||
| 298 | 303 | ||
| 299 | new_desc = desc; | 304 | new_desc = desc; |
| 300 | 305 | ||
| @@ -854,6 +859,33 @@ int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2) · 553 unmodified lines | |||
| 854 | return kvm_inject_nested_sync(vcpu, esr_el2); | 859 | return kvm_inject_nested_sync(vcpu, esr_el2); |
| 855 | } | 860 | } |
| 856 | 861 | ||
| 862 | u16 get_asid_by_regime(struct kvm_vcpu *vcpu, enum trans_regime regime) | ||
| 863 | { | ||
| 864 | enum vcpu_sysreg ttbr_elx; | ||
| 865 | u64 tcr; | ||
| 866 | u16 asid; | ||
| 867 | |||
| 868 | switch (regime) { | ||
| 869 | case TR_EL10: | ||
| 870 | tcr = vcpu_read_sys_reg(vcpu, TCR_EL1); | ||
| 871 | ttbr_elx = (tcr & TCR_A1) ? TTBR1_EL1 : TTBR0_EL1; | ||
| 872 | break; | ||
| 873 | case TR_EL20: | ||
| 874 | tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | ||
| 875 | ttbr_elx = (tcr & TCR_A1) ? TTBR1_EL2 : TTBR0_EL2; | ||
| 876 | break; | ||
| 877 | default: | ||
| 878 | BUG(); | ||
| 879 | } | ||
| 880 | |||
| 881 | asid = FIELD_GET(TTBRx_EL1_ASID, vcpu_read_sys_reg(vcpu, ttbr_elx)); | ||
| 882 | if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | ||
| 883 | !(tcr & TCR_ASID16)) | ||
| 884 | asid &= GENMASK(7, 0); | ||
| 885 | |||
| 886 | return asid; | ||
| 887 | } | ||
| 888 | |||
| 857 | static void invalidate_vncr(struct vncr_tlb *vt) | 889 | static void invalidate_vncr(struct vncr_tlb *vt) |
| 858 | { | 890 | { |
| 859 | vt->valid = false; | 891 | vt->valid = false; |
| @@ -1154,9 +1186,6 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm) · 294 unmodified lines | |||
| 1154 | { | 1186 | { |
| 1155 | int i; | 1187 | int i; |
| 1156 | 1188 | ||
| 1157 | if (!kvm->arch.nested_mmus_size) | ||
| 1158 | return; | ||
| 1159 | |||
| 1160 | for (i = 0; i < kvm->arch.nested_mmus_size; i++) { | 1189 | for (i = 0; i < kvm->arch.nested_mmus_size; i++) { |
| 1161 | struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i]; | 1190 | struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i]; |
| 1162 | 1191 | ||
| @@ -1336,20 +1365,8 @@ static bool kvm_vncr_tlb_lookup(struct kvm_vcpu *vcpu) · 173 unmodified lines | |||
| 1336 | if (read_vncr_el2(vcpu) != vt->gva) | 1365 | if (read_vncr_el2(vcpu) != vt->gva) |
| 1337 | return false; | 1366 | return false; |
| 1338 | 1367 | ||
| 1339 | if (vt->wr.nG) { | 1368 | if (vt->wr.nG) |
| 1340 | u64 tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | 1369 | return get_asid_by_regime(vcpu, TR_EL20) == vt->wr.asid; |
| 1341 | u64 ttbr = ((tcr & TCR_A1) ? | ||
| 1342 | vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | ||
| 1343 | vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | ||
| 1344 | u16 asid; | ||
| 1345 | |||
| 1346 | asid = FIELD_GET(TTBR_ASID_MASK, ttbr); | ||
| 1347 | if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | ||
| 1348 | !(tcr & TCR_ASID16)) | ||
| 1349 | asid &= GENMASK(7, 0); | ||
| 1350 | |||
| 1351 | return asid == vt->wr.asid; | ||
| 1352 | } | ||
| 1353 | 1370 | ||
| 1354 | return true; | 1371 | return true; |
| 1355 | } | 1372 | } |
| @@ -1452,21 +1469,8 @@ static void kvm_map_l1_vncr(struct kvm_vcpu *vcpu) · 96 unmodified lines | |||
| 1452 | if (read_vncr_el2(vcpu) != vt->gva) | 1469 | if (read_vncr_el2(vcpu) != vt->gva) |
| 1453 | return; | 1470 | return; |
| 1454 | 1471 | ||
| 1455 | if (vt->wr.nG) { | 1472 | if (vt->wr.nG && get_asid_by_regime(vcpu, TR_EL20) != vt->wr.asid) |
| 1456 | u64 tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | 1473 | return; |
| 1457 | u64 ttbr = ((tcr & TCR_A1) ? | ||
| 1458 | vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | ||
| 1459 | vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | ||
| 1460 | u16 asid; | ||
| 1461 | |||
| 1462 | asid = FIELD_GET(TTBR_ASID_MASK, ttbr); | ||
| 1463 | if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | ||
| 1464 | !(tcr & TCR_ASID16)) | ||
| 1465 | asid &= GENMASK(7, 0); | ||
| 1466 | |||
| 1467 | if (asid != vt->wr.asid) | ||
| 1468 | return; | ||
| 1469 | } | ||
| 1470 | 1474 | ||
| 1471 | vt->cpu = smp_processor_id(); | 1475 | vt->cpu = smp_processor_id(); |
| 1472 | 1476 | ||
| @@ -1816,6 +1816,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, · 1815 unmodified lines | |||
| 1816 | ID_AA64MMFR3_EL1_SCTLRX | | 1816 | ID_AA64MMFR3_EL1_SCTLRX | |
| 1817 | ID_AA64MMFR3_EL1_S1POE | | 1817 | ID_AA64MMFR3_EL1_S1POE | |
| 1818 | ID_AA64MMFR3_EL1_S1PIE; | 1818 | ID_AA64MMFR3_EL1_S1PIE; |
| 1819 | |||
| 1820 | if (!system_supports_poe()) | ||
| 1821 | val &= ~ID_AA64MMFR3_EL1_S1POE; | ||
| 1819 | break; | 1822 | break; |
| 1820 | case SYS_ID_MMFR4_EL1: | 1823 | case SYS_ID_MMFR4_EL1: |
| 1821 | val &= ~ID_MMFR4_EL1_CCIDX; | 1824 | val &= ~ID_MMFR4_EL1_CCIDX; |
| @@ -143,6 +143,21 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) · 142 unmodified lines | |||
| 143 | kvm->arch.vgic.in_kernel = true; | 143 | kvm->arch.vgic.in_kernel = true; |
| 144 | kvm->arch.vgic.vgic_model = type; | 144 | kvm->arch.vgic.vgic_model = type; |
| 145 | kvm->arch.vgic.implementation_rev = KVM_VGIC_IMP_REV_LATEST; | 145 | kvm->arch.vgic.implementation_rev = KVM_VGIC_IMP_REV_LATEST; |
| 146 | kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF; | ||
| 147 | |||
| 148 | aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC; | ||
| 149 | pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC; | ||
| 150 | |||
| 151 | if (type == KVM_DEV_TYPE_ARM_VGIC_V2) { | ||
| 152 | kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF; | ||
| 153 | } else { | ||
| 154 | INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions); | ||
| 155 | aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP); | ||
| 156 | pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3); | ||
| 157 | } | ||
| 158 | |||
| 159 | kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0); | ||
| 160 | kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1); | ||
| 146 | 161 | ||
| 147 | kvm_for_each_vcpu(i, vcpu, kvm) { | 162 | kvm_for_each_vcpu(i, vcpu, kvm) { |
| 148 | ret = vgic_allocate_private_irqs_locked(vcpu, type); | 163 | ret = vgic_allocate_private_irqs_locked(vcpu, type); |
| @@ -157,25 +172,10 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) · 8 unmodified lines | |||
| 157 | vgic_cpu->private_irqs = NULL; | 172 | vgic_cpu->private_irqs = NULL; |
| 158 | } | 173 | } |
| 159 | 174 | ||
| 175 | kvm->arch.vgic.vgic_model = 0; | ||
| 160 | goto out_unlock; | 176 | goto out_unlock; |
| 161 | } | 177 | } |
| 162 | 178 | ||
| 163 | kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF; | ||
| 164 | |||
| 165 | aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC; | ||
| 166 | pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC; | ||
| 167 | |||
| 168 | if (type == KVM_DEV_TYPE_ARM_VGIC_V2) { | ||
| 169 | kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF; | ||
| 170 | } else { | ||
| 171 | INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions); | ||
| 172 | aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP); | ||
| 173 | pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3); | ||
| 174 | } | ||
| 175 | |||
| 176 | kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0); | ||
| 177 | kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1); | ||
| 178 | |||
| 179 | if (type == KVM_DEV_TYPE_ARM_VGIC_V3) | 179 | if (type == KVM_DEV_TYPE_ARM_VGIC_V3) |
| 180 | kvm->arch.vgic.nassgicap = system_supports_direct_sgis(); | 180 | kvm->arch.vgic.nassgicap = system_supports_direct_sgis(); |
| 181 | 181 | ||
| @@ -115,7 +115,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu) · 114 unmodified lines | |||
| 115 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; | 115 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; |
| 116 | struct vgic_v2_cpu_if *cpuif = &vgic_cpu->vgic_v2; | 116 | struct vgic_v2_cpu_if *cpuif = &vgic_cpu->vgic_v2; |
| 117 | u32 eoicount = FIELD_GET(GICH_HCR_EOICOUNT, cpuif->vgic_hcr); | 117 | u32 eoicount = FIELD_GET(GICH_HCR_EOICOUNT, cpuif->vgic_hcr); |
| 118 | struct vgic_irq *irq; | 118 | struct vgic_irq *irq = *host_data_ptr(last_lr_irq); |
| 119 | 119 | ||
| 120 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); | 120 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); |
| 121 | 121 | ||
| @@ -123,7 +123,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu) · 1 unmodified line | |||
| 123 | vgic_v2_fold_lr(vcpu, cpuif->vgic_lr[lr]); | 123 | vgic_v2_fold_lr(vcpu, cpuif->vgic_lr[lr]); |
| 124 | 124 | ||
| 125 | /* See the GICv3 equivalent for the EOIcount handling rationale */ | 125 | /* See the GICv3 equivalent for the EOIcount handling rationale */ |
| 126 | list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { | 126 | list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) { |
| 127 | u32 lr; | 127 | u32 lr; |
| 128 | 128 | ||
| 129 | if (!eoicount) { | 129 | if (!eoicount) { |
| @@ -148,7 +148,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) · 147 unmodified lines | |||
| 148 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; | 148 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; |
| 149 | struct vgic_v3_cpu_if *cpuif = &vgic_cpu->vgic_v3; | 149 | struct vgic_v3_cpu_if *cpuif = &vgic_cpu->vgic_v3; |
| 150 | u32 eoicount = FIELD_GET(ICH_HCR_EL2_EOIcount, cpuif->vgic_hcr); | 150 | u32 eoicount = FIELD_GET(ICH_HCR_EL2_EOIcount, cpuif->vgic_hcr); |
| 151 | struct vgic_irq *irq; | 151 | struct vgic_irq *irq = *host_data_ptr(last_lr_irq); |
| 152 | 152 | ||
| 153 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); | 153 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); |
| 154 | 154 | ||
| @@ -158,12 +158,12 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) · 3 unmodified lines | |||
| 158 | /* | 158 | /* |
| 159 | * EOIMode=0: use EOIcount to emulate deactivation. We are | 159 | * EOIMode=0: use EOIcount to emulate deactivation. We are |
| 160 | * guaranteed to deactivate in reverse order of the activation, so | 160 | * guaranteed to deactivate in reverse order of the activation, so |
| 161 | * just pick one active interrupt after the other in the ap_list, | 161 | * just pick one active interrupt after the other in the tail part |
| 162 | * and replay the deactivation as if the CPU was doing it. We also | 162 | * of the ap_list, past the LRs, and replay the deactivation as if |
| 163 | * rely on priority drop to have taken place, and the list to be | 163 | * the CPU was doing it. We also rely on priority drop to have taken |
| 164 | * sorted by priority. | 164 | * place, and the list to be sorted by priority. |
| 165 | */ | 165 | */ |
| 166 | list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { | 166 | list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) { |
| 167 | u64 lr; | 167 | u64 lr; |
| 168 | 168 | ||
| 169 | /* | 169 | /* |
| @@ -814,6 +814,9 @@ retry: · 813 unmodified lines | |||
| 814 | 814 | ||
| 815 | static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu) | 815 | static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu) |
| 816 | { | 816 | { |
| 817 | if (!*host_data_ptr(last_lr_irq)) | ||
| 818 | return; | ||
| 819 | |||
| 817 | if (kvm_vgic_global_state.type == VGIC_V2) | 820 | if (kvm_vgic_global_state.type == VGIC_V2) |
| 818 | vgic_v2_fold_lr_state(vcpu); | 821 | vgic_v2_fold_lr_state(vcpu); |
| 819 | else | 822 | else |
| @@ -960,10 +963,13 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu) · 140 unmodified lines | |||
| 960 | if (irqs_outside_lrs(&als)) | 963 | if (irqs_outside_lrs(&als)) |
| 961 | vgic_sort_ap_list(vcpu); | 964 | vgic_sort_ap_list(vcpu); |
| 962 | 965 | ||
| 966 | *host_data_ptr(last_lr_irq) = NULL; | ||
| 967 | |||
| 963 | list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { | 968 | list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { |
| 964 | scoped_guard(raw_spinlock, &irq->irq_lock) { | 969 | scoped_guard(raw_spinlock, &irq->irq_lock) { |
| 965 | if (likely(vgic_target_oracle(irq) == vcpu)) { | 970 | if (likely(vgic_target_oracle(irq) == vcpu)) { |
| 966 | vgic_populate_lr(vcpu, irq, count++); | 971 | vgic_populate_lr(vcpu, irq, count++); |
| 972 | *host_data_ptr(last_lr_irq) = irq; | ||
| 967 | } | 973 | } |
| 968 | } | 974 | } |
| 969 | 975 | ||
| @@ -32,7 +32,11 @@ static inline unsigned long xloops_to_cycles(unsigned long xloops) · 31 unmodified lines | |||
| 32 | * Note that userspace cannot change the offset behind our back either, | 32 | * Note that userspace cannot change the offset behind our back either, |
| 33 | * as the vcpu mutex is held as long as KVM_RUN is in progress. | 33 | * as the vcpu mutex is held as long as KVM_RUN is in progress. |
| 34 | */ | 34 | */ |
| 35 | #define __delay_cycles() __arch_counter_get_cntvct_stable() | 35 | static cycles_t notrace __delay_cycles(void) |
| 36 | { | ||
| 37 | guard(preempt_notrace)(); | ||
| 38 | return __arch_counter_get_cntvct_stable(); | ||
| 39 | } | ||
| 36 | 40 | ||
| 37 | void __delay(unsigned long cycles) | 41 | void __delay(unsigned long cycles) |
| 38 | { | 42 | { |
| @@ -599,6 +599,27 @@ void contpte_clear_young_dirty_ptes(struct vm_area_struct *vma, · 598 unmodified lines | |||
| 599 | } | 599 | } |
| 600 | EXPORT_SYMBOL_GPL(contpte_clear_young_dirty_ptes); | 600 | EXPORT_SYMBOL_GPL(contpte_clear_young_dirty_ptes); |
| 601 | 601 | ||
| 602 | static bool contpte_all_subptes_match_access_flags(pte_t *ptep, pte_t entry) | ||
| 603 | { | ||
| 604 | pte_t *cont_ptep = contpte_align_down(ptep); | ||
| 605 | /* | ||
| 606 | * PFNs differ per sub-PTE. Match only bits consumed by | ||
| 607 | * __ptep_set_access_flags(): AF, DIRTY and write permission. | ||
| 608 | */ | ||
| 609 | const pteval_t cmp_mask = PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; | ||
| 610 | pteval_t entry_cmp = pte_val(entry) & cmp_mask; | ||
| 611 | int i; | ||
| 612 | |||
| 613 | for (i = 0; i < CONT_PTES; i++) { | ||
| 614 | pteval_t pte_cmp = pte_val(__ptep_get(cont_ptep + i)) & cmp_mask; | ||
| 615 | |||
| 616 | if (pte_cmp != entry_cmp) | ||
| 617 | return false; | ||
| 618 | } | ||
| 619 | |||
| 620 | return true; | ||
| 621 | } | ||
| 622 | |||
| 602 | int contpte_ptep_set_access_flags(struct vm_area_struct *vma, | 623 | int contpte_ptep_set_access_flags(struct vm_area_struct *vma, |
| 603 | unsigned long addr, pte_t *ptep, | 624 | unsigned long addr, pte_t *ptep, |
| 604 | pte_t entry, int dirty) | 625 | pte_t entry, int dirty) |
| @@ -608,13 +629,37 @@ int contpte_ptep_set_access_flags(struct vm_area_struct *vma, · 3 unmodified lines | |||
| 608 | int i; | 629 | int i; |
| 609 | 630 | ||
| 610 | /* | 631 | /* |
| 611 | * Gather the access/dirty bits for the contiguous range. If nothing has | 632 | * Check whether all sub-PTEs in the CONT block already match the |
| 612 | * changed, its a noop. | 633 | * requested access flags/write permission, using raw per-PTE values |
| 634 | * rather than the gathered ptep_get() view. | ||
| 635 | * | ||
| 636 | * __ptep_set_access_flags() can update AF, dirty and write | ||
| 637 | * permission, but only to make the mapping more permissive. | ||
| 638 | * | ||
| 639 | * ptep_get() gathers AF/dirty state across the whole CONT block, | ||
| 640 | * which is correct for a CPU with FEAT_HAFDBS. But page-table | ||
| 641 | * walkers that evaluate each descriptor individually (e.g. a CPU | ||
| 642 | * without DBM support, or an SMMU without HTTU, or with HA/HD | ||
| 643 | * disabled in CD.TCR) can keep faulting on the target sub-PTE if | ||
| 644 | * only a sibling has been updated. Gathering can therefore cause | ||
| 645 | * false no-ops when only a sibling has been updated: | ||
| 646 | * - write faults: target still has PTE_RDONLY (needs PTE_RDONLY cleared) | ||
| 647 | * - read faults: target still lacks PTE_AF | ||
| 648 | * | ||
| 649 | * Per Arm ARM (DDI 0487) D8.7.1, any sub-PTE in a CONT range may | ||
| 650 | * become the effective cached translation, so all entries must have | ||
| 651 | * consistent attributes. Check the full CONT block before returning | ||
| 652 | * no-op, and when any sub-PTE mismatches, proceed to update the whole | ||
| 653 | * range. | ||
| 613 | */ | 654 | */ |
| 614 | orig_pte = pte_mknoncont(ptep_get(ptep)); | 655 | if (contpte_all_subptes_match_access_flags(ptep, entry)) |
| 615 | if (pte_val(orig_pte) == pte_val(entry)) | ||
| 616 | return 0; | 656 | return 0; |
| 617 | 657 | ||
| 658 | /* | ||
| 659 | * Use raw target pte (not gathered) for write-bit unfold decision. | ||
| 660 | */ | ||
| 661 | orig_pte = pte_mknoncont(__ptep_get(ptep)); | ||
| 662 | |||
| 618 | /* | 663 | /* |
| 619 | * We can fix up access/dirty bits without having to unfold the contig | 664 | * We can fix up access/dirty bits without having to unfold the contig |
| 620 | * range. But if the write bit is changing, we must unfold. | 665 | * range. But if the write bit is changing, we must unfold. |
| @@ -14,8 +14,8 @@ int arm64_ioremap_prot_hook_register(ioremap_prot_hook_t hook) · 13 unmodified lines | |||
| 14 | return 0; | 14 | return 0; |
| 15 | } | 15 | } |
| 16 | 16 | ||
| 17 | void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, | 17 | void __iomem *__ioremap_prot(phys_addr_t phys_addr, size_t size, |
| 18 | pgprot_t pgprot) | 18 | pgprot_t pgprot) |
| 19 | { | 19 | { |
| 20 | unsigned long last_addr = phys_addr + size - 1; | 20 | unsigned long last_addr = phys_addr + size - 1; |
| 21 | 21 | ||
| @@ -39,7 +39,7 @@ void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, · 17 unmodified lines | |||
| 39 | 39 | ||
| 40 | return generic_ioremap_prot(phys_addr, size, pgprot); | 40 | return generic_ioremap_prot(phys_addr, size, pgprot); |
| 41 | } | 41 | } |
| 42 | EXPORT_SYMBOL(ioremap_prot); | 42 | EXPORT_SYMBOL(__ioremap_prot); |
| 43 | 43 | ||
| 44 | /* | 44 | /* |
| 45 | * Must be called after early_fixmap_init | 45 | * Must be called after early_fixmap_init |
| @@ -34,6 +34,8 @@ static pgprot_t protection_map[16] __ro_after_init = { · 33 unmodified lines | |||
| 34 | [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_SHARED_EXEC | 34 | [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_SHARED_EXEC |
| 35 | }; | 35 | }; |
| 36 | 36 | ||
| 37 | static ptdesc_t gcs_page_prot __ro_after_init = _PAGE_GCS_RO; | ||
| 38 | |||
| 37 | /* | 39 | /* |
| 38 | * You really shouldn't be using read() or write() on /dev/mem. This might go | 40 | * You really shouldn't be using read() or write() on /dev/mem. This might go |
| 39 | * away in the future. | 41 | * away in the future. |
| @@ -73,9 +75,11 @@ static int __init adjust_protection_map(void) · 33 unmodified lines | |||
| 73 | protection_map[VM_EXEC | VM_SHARED] = PAGE_EXECONLY; | 75 | protection_map[VM_EXEC | VM_SHARED] = PAGE_EXECONLY; |
| 74 | } | 76 | } |
| 75 | 77 | ||
| 76 | if (lpa2_is_enabled()) | 78 | if (lpa2_is_enabled()) { |
| 77 | for (int i = 0; i < ARRAY_SIZE(protection_map); i++) | 79 | for (int i = 0; i < ARRAY_SIZE(protection_map); i++) |
| 78 | pgprot_val(protection_map[i]) &= ~PTE_SHARED; | 80 | pgprot_val(protection_map[i]) &= ~PTE_SHARED; |
| 81 | gcs_page_prot &= ~PTE_SHARED; | ||
| 82 | } | ||
| 79 | 83 | ||
| 80 | return 0; | 84 | return 0; |
| 81 | } | 85 | } |
| @@ -87,7 +91,11 @@ pgprot_t vm_get_page_prot(vm_flags_t vm_flags) · 5 unmodified lines | |||
| 87 | 91 | ||
| 88 | /* Short circuit GCS to avoid bloating the table. */ | 92 | /* Short circuit GCS to avoid bloating the table. */ |
| 89 | if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { | 93 | if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { |
| 90 | prot = _PAGE_GCS_RO; | 94 | /* Honour mprotect(PROT_NONE) on shadow stack mappings */ |
| 95 | if (vm_flags & VM_ACCESS_FLAGS) | ||
| 96 | prot = gcs_page_prot; | ||
| 97 | else | ||
| 98 | prot = pgprot_val(protection_map[VM_NONE]); | ||
| 91 | } else { | 99 | } else { |
| 92 | prot = pgprot_val(protection_map[vm_flags & | 100 | prot = pgprot_val(protection_map[vm_flags & |
| 93 | (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); | 101 | (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); |
| @@ -2119,7 +2119,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · 2118 unmodified lines | |||
| 2119 | extable_offset = round_up(prog_size + PLT_TARGET_SIZE, extable_align); | 2119 | extable_offset = round_up(prog_size + PLT_TARGET_SIZE, extable_align); |
| 2120 | image_size = extable_offset + extable_size; | 2120 | image_size = extable_offset + extable_size; |
| 2121 | ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, | 2121 | ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, |
| 2122 | sizeof(u32), &header, &image_ptr, | 2122 | sizeof(u64), &header, &image_ptr, |
| 2123 | jit_fill_hole); | 2123 | jit_fill_hole); |
| 2124 | if (!ro_header) { | 2124 | if (!ro_header) { |
| 2125 | prog = orig_prog; | 2125 | prog = orig_prog; |
| @@ -109,6 +109,7 @@ SECTIONS · 108 unmodified lines | |||
| 109 | 109 | ||
| 110 | STABS_DEBUG | 110 | STABS_DEBUG |
| 111 | DWARF_DEBUG | 111 | DWARF_DEBUG |
| 112 | MODINFO | ||
| 112 | ELF_DETAILS | 113 | ELF_DETAILS |
| 113 | 114 | ||
| 114 | DISCARDS | 115 | DISCARDS |
| @@ -62,6 +62,7 @@ SECTIONS · 61 unmodified lines | |||
| 62 | 62 | ||
| 63 | STABS_DEBUG | 63 | STABS_DEBUG |
| 64 | DWARF_DEBUG | 64 | DWARF_DEBUG |
| 65 | MODINFO | ||
| 65 | ELF_DETAILS | 66 | ELF_DETAILS |
| 66 | .hexagon.attributes 0 : { *(.hexagon.attributes) } | 67 | .hexagon.attributes 0 : { *(.hexagon.attributes) } |
| 67 | 68 | ||
| @@ -147,6 +147,7 @@ SECTIONS · 146 unmodified lines | |||
| 147 | 147 | ||
| 148 | STABS_DEBUG | 148 | STABS_DEBUG |
| 149 | DWARF_DEBUG | 149 | DWARF_DEBUG |
| 150 | MODINFO | ||
| 150 | ELF_DETAILS | 151 | ELF_DETAILS |
| 151 | 152 | ||
| 152 | #ifdef CONFIG_EFI_STUB | 153 | #ifdef CONFIG_EFI_STUB |
| @@ -28,7 +28,6 @@ config KVM · 27 unmodified lines | |||
| 28 | select KVM_COMMON | 28 | select KVM_COMMON |
| 29 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT | 29 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| 30 | select KVM_GENERIC_HARDWARE_ENABLING | 30 | select KVM_GENERIC_HARDWARE_ENABLING |
| 31 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 32 | select KVM_MMIO | 31 | select KVM_MMIO |
| 33 | select VIRT_XFER_TO_GUEST_WORK | 32 | select VIRT_XFER_TO_GUEST_WORK |
| 34 | select SCHED_INFO | 33 | select SCHED_INFO |
| @@ -14,7 +14,7 @@ · 13 unmodified lines | |||
| 14 | #define CREATE_TRACE_POINTS | 14 | #define CREATE_TRACE_POINTS |
| 15 | #include "trace.h" | 15 | #include "trace.h" |
| 16 | 16 | ||
| 17 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 17 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 18 | KVM_GENERIC_VCPU_STATS(), | 18 | KVM_GENERIC_VCPU_STATS(), |
| 19 | STATS_DESC_COUNTER(VCPU, int_exits), | 19 | STATS_DESC_COUNTER(VCPU, int_exits), |
| 20 | STATS_DESC_COUNTER(VCPU, idle_exits), | 20 | STATS_DESC_COUNTER(VCPU, idle_exits), |
| @@ -10,7 +10,7 @@ · 9 unmodified lines | |||
| 10 | #include <asm/kvm_eiointc.h> | 10 | #include <asm/kvm_eiointc.h> |
| 11 | #include <asm/kvm_pch_pic.h> | 11 | #include <asm/kvm_pch_pic.h> |
| 12 | 12 | ||
| 13 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 13 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 14 | KVM_GENERIC_VM_STATS(), | 14 | KVM_GENERIC_VM_STATS(), |
| 15 | STATS_DESC_ICOUNTER(VM, pages), | 15 | STATS_DESC_ICOUNTER(VM, pages), |
| 16 | STATS_DESC_ICOUNTER(VM, hugepages), | 16 | STATS_DESC_ICOUNTER(VM, hugepages), |
| @@ -118,7 +118,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 101 unmodified lines | |||
| 118 | case KVM_CAP_ONE_REG: | 118 | case KVM_CAP_ONE_REG: |
| 119 | case KVM_CAP_ENABLE_CAP: | 119 | case KVM_CAP_ENABLE_CAP: |
| 120 | case KVM_CAP_READONLY_MEM: | 120 | case KVM_CAP_READONLY_MEM: |
| 121 | case KVM_CAP_SYNC_MMU: | ||
| 122 | case KVM_CAP_IMMEDIATE_EXIT: | 121 | case KVM_CAP_IMMEDIATE_EXIT: |
| 123 | case KVM_CAP_IOEVENTFD: | 122 | case KVM_CAP_IOEVENTFD: |
| 124 | case KVM_CAP_MP_STATE: | 123 | case KVM_CAP_MP_STATE: |
| @@ -85,6 +85,7 @@ SECTIONS { · 84 unmodified lines | |||
| 85 | _end = .; | 85 | _end = .; |
| 86 | 86 | ||
| 87 | STABS_DEBUG | 87 | STABS_DEBUG |
| 88 | MODINFO | ||
| 88 | ELF_DETAILS | 89 | ELF_DETAILS |
| 89 | 90 | ||
| 90 | /* Sections to be discarded */ | 91 | /* Sections to be discarded */ |
| @@ -58,6 +58,7 @@ SECTIONS · 57 unmodified lines | |||
| 58 | _end = . ; | 58 | _end = . ; |
| 59 | 59 | ||
| 60 | STABS_DEBUG | 60 | STABS_DEBUG |
| 61 | MODINFO | ||
| 61 | ELF_DETAILS | 62 | ELF_DETAILS |
| 62 | 63 | ||
| 63 | /* Sections to be discarded */ | 64 | /* Sections to be discarded */ |
| @@ -51,6 +51,7 @@ __init_begin = .; · 50 unmodified lines | |||
| 51 | _end = . ; | 51 | _end = . ; |
| 52 | 52 | ||
| 53 | STABS_DEBUG | 53 | STABS_DEBUG |
| 54 | MODINFO | ||
| 54 | ELF_DETAILS | 55 | ELF_DETAILS |
| 55 | 56 | ||
| 56 | /* Sections to be discarded */ | 57 | /* Sections to be discarded */ |
| @@ -217,6 +217,7 @@ SECTIONS · 216 unmodified lines | |||
| 217 | 217 | ||
| 218 | STABS_DEBUG | 218 | STABS_DEBUG |
| 219 | DWARF_DEBUG | 219 | DWARF_DEBUG |
| 220 | MODINFO | ||
| 220 | ELF_DETAILS | 221 | ELF_DETAILS |
| 221 | 222 | ||
| 222 | /* These must appear regardless of . */ | 223 | /* These must appear regardless of . */ |
| @@ -23,7 +23,6 @@ config KVM · 22 unmodified lines | |||
| 23 | select KVM_COMMON | 23 | select KVM_COMMON |
| 24 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT | 24 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| 25 | select KVM_MMIO | 25 | select KVM_MMIO |
| 26 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 27 | select KVM_GENERIC_HARDWARE_ENABLING | 26 | select KVM_GENERIC_HARDWARE_ENABLING |
| 28 | select HAVE_KVM_READONLY_MEM | 27 | select HAVE_KVM_READONLY_MEM |
| 29 | help | 28 | help |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | |||
| 38 | #define VECTORSPACING 0x100 /* for EI/VI mode */ | 38 | #define VECTORSPACING 0x100 /* for EI/VI mode */ |
| 39 | #endif | 39 | #endif |
| 40 | 40 | ||
| 41 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 41 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 42 | KVM_GENERIC_VM_STATS() | 42 | KVM_GENERIC_VM_STATS() |
| 43 | }; | 43 | }; |
| 44 | 44 | ||
| @@ -51,7 +51,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 6 unmodified lines | |||
| 51 | sizeof(kvm_vm_stats_desc), | 51 | sizeof(kvm_vm_stats_desc), |
| 52 | }; | 52 | }; |
| 53 | 53 | ||
| 54 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 54 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 55 | KVM_GENERIC_VCPU_STATS(), | 55 | KVM_GENERIC_VCPU_STATS(), |
| 56 | STATS_DESC_COUNTER(VCPU, wait_exits), | 56 | STATS_DESC_COUNTER(VCPU, wait_exits), |
| 57 | STATS_DESC_COUNTER(VCPU, cache_exits), | 57 | STATS_DESC_COUNTER(VCPU, cache_exits), |
| @@ -1035,7 +1035,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 977 unmodified lines | |||
| 1035 | case KVM_CAP_ONE_REG: | 1035 | case KVM_CAP_ONE_REG: |
| 1036 | case KVM_CAP_ENABLE_CAP: | 1036 | case KVM_CAP_ENABLE_CAP: |
| 1037 | case KVM_CAP_READONLY_MEM: | 1037 | case KVM_CAP_READONLY_MEM: |
| 1038 | case KVM_CAP_SYNC_MMU: | ||
| 1039 | case KVM_CAP_IMMEDIATE_EXIT: | 1038 | case KVM_CAP_IMMEDIATE_EXIT: |
| 1040 | r = 1; | 1039 | r = 1; |
| 1041 | break; | 1040 | break; |
| @@ -57,6 +57,7 @@ SECTIONS · 56 unmodified lines | |||
| 57 | 57 | ||
| 58 | STABS_DEBUG | 58 | STABS_DEBUG |
| 59 | DWARF_DEBUG | 59 | DWARF_DEBUG |
| 60 | MODINFO | ||
| 60 | ELF_DETAILS | 61 | ELF_DETAILS |
| 61 | 62 | ||
| 62 | DISCARDS | 63 | DISCARDS |
| @@ -101,6 +101,7 @@ SECTIONS · 100 unmodified lines | |||
| 101 | /* Throw in the debugging sections */ | 101 | /* Throw in the debugging sections */ |
| 102 | STABS_DEBUG | 102 | STABS_DEBUG |
| 103 | DWARF_DEBUG | 103 | DWARF_DEBUG |
| 104 | MODINFO | ||
| 104 | ELF_DETAILS | 105 | ELF_DETAILS |
| 105 | 106 | ||
| 106 | /* Sections to be discarded -- must be last */ | 107 | /* Sections to be discarded -- must be last */ |
| @@ -90,6 +90,7 @@ SECTIONS · 89 unmodified lines | |||
| 90 | /* Sections to be discarded */ | 90 | /* Sections to be discarded */ |
| 91 | DISCARDS | 91 | DISCARDS |
| 92 | /DISCARD/ : { | 92 | /DISCARD/ : { |
| 93 | *(.modinfo) | ||
| 93 | #ifdef CONFIG_64BIT | 94 | #ifdef CONFIG_64BIT |
| 94 | /* temporary hack until binutils is fixed to not emit these | 95 | /* temporary hack until binutils is fixed to not emit these |
| 95 | * for static binaries | 96 | * for static binaries |
| @@ -85,7 +85,7 @@ extern void __update_cache(pte_t pte); · 84 unmodified lines | |||
| 85 | printk("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, (unsigned long)pgd_val(e)) | 85 | printk("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, (unsigned long)pgd_val(e)) |
| 86 | 86 | ||
| 87 | /* This is the size of the initially mapped kernel memory */ | 87 | /* This is the size of the initially mapped kernel memory */ |
| 88 | #if defined(CONFIG_64BIT) | 88 | #if defined(CONFIG_64BIT) || defined(CONFIG_KALLSYMS) |
| 89 | #define KERNEL_INITIAL_ORDER 26 /* 1<<26 = 64MB */ | 89 | #define KERNEL_INITIAL_ORDER 26 /* 1<<26 = 64MB */ |
| 90 | #else | 90 | #else |
| 91 | #define KERNEL_INITIAL_ORDER 25 /* 1<<25 = 32MB */ | 91 | #define KERNEL_INITIAL_ORDER 25 /* 1<<25 = 32MB */ |
| @@ -56,6 +56,7 @@ ENTRY(parisc_kernel_start) · 55 unmodified lines | |||
| 56 | 56 | ||
| 57 | .import __bss_start,data | 57 | .import __bss_start,data |
| 58 | .import __bss_stop,data | 58 | .import __bss_stop,data |
| 59 | .import __end,data | ||
| 59 | 60 | ||
| 60 | load32 PA(__bss_start),%r3 | 61 | load32 PA(__bss_start),%r3 |
| 61 | load32 PA(__bss_stop),%r4 | 62 | load32 PA(__bss_stop),%r4 |
| @@ -149,7 +150,11 @@ $cpu_ok: · 87 unmodified lines | |||
| 149 | * everything ... it will get remapped correctly later */ | 150 | * everything ... it will get remapped correctly later */ |
| 150 | ldo 0+_PAGE_KERNEL_RWX(%r0),%r3 /* Hardwired 0 phys addr start */ | 151 | ldo 0+_PAGE_KERNEL_RWX(%r0),%r3 /* Hardwired 0 phys addr start */ |
| 151 | load32 (1<<(KERNEL_INITIAL_ORDER-PAGE_SHIFT)),%r11 /* PFN count */ | 152 | load32 (1<<(KERNEL_INITIAL_ORDER-PAGE_SHIFT)),%r11 /* PFN count */ |
| 152 | load32 PA(pg0),%r1 | 153 | load32 PA(_end),%r1 |
| 154 | SHRREG %r1,PAGE_SHIFT,%r1 /* %r1 is PFN count for _end symbol */ | ||
| 155 | cmpb,<<,n %r11,%r1,1f | ||
| 156 | copy %r1,%r11 /* %r1 PFN count smaller than %r11 */ | ||
| 157 | 1: load32 PA(pg0),%r1 | ||
| 153 | 158 | ||
| 154 | $pgt_fill_loop: | 159 | $pgt_fill_loop: |
| 155 | STREGM %r3,ASM_PTE_ENTRY_SIZE(%r1) | 160 | STREGM %r3,ASM_PTE_ENTRY_SIZE(%r1) |
| @@ -120,14 +120,6 @@ void __init setup_arch(char **cmdline_p) · 119 unmodified lines | |||
| 120 | #endif | 120 | #endif |
| 121 | printk(KERN_CONT ".\n"); | 121 | printk(KERN_CONT ".\n"); |
| 122 | 122 | ||
| 123 | /* | ||
| 124 | * Check if initial kernel page mappings are sufficient. | ||
| 125 | * panic early if not, else we may access kernel functions | ||
| 126 | * and variables which can't be reached. | ||
| 127 | */ | ||
| 128 | if (__pa((unsigned long) &_end) >= KERNEL_INITIAL_SIZE) | ||
| 129 | panic("KERNEL_INITIAL_ORDER too small!"); | ||
| 130 | |||
| 131 | #ifdef CONFIG_64BIT | 123 | #ifdef CONFIG_64BIT |
| 132 | if(parisc_narrow_firmware) { | 124 | if(parisc_narrow_firmware) { |
| 133 | printk(KERN_INFO "Kernel is using PDC in 32-bit mode.\n"); | 125 | printk(KERN_INFO "Kernel is using PDC in 32-bit mode.\n"); |
| @@ -279,6 +271,18 @@ void __init start_parisc(void) · 145 unmodified lines | |||
| 279 | int ret, cpunum; | 271 | int ret, cpunum; |
| 280 | struct pdc_coproc_cfg coproc_cfg; | 272 | struct pdc_coproc_cfg coproc_cfg; |
| 281 | 273 | ||
| 274 | /* | ||
| 275 | * Check if initial kernel page mapping is sufficient. | ||
| 276 | * Print warning if not, because we may access kernel functions and | ||
| 277 | * variables which can't be reached yet through the initial mappings. | ||
| 278 | * Note that the panic() and printk() functions are not functional | ||
| 279 | * yet, so we need to use direct iodc() firmware calls instead. | ||
| 280 | */ | ||
| 281 | const char warn1[] = "CRITICAL: Kernel may crash because " | ||
| 282 | "KERNEL_INITIAL_ORDER is too small.\n"; | ||
| 283 | if (__pa((unsigned long) &_end) >= KERNEL_INITIAL_SIZE) | ||
| 284 | pdc_iodc_print(warn1, sizeof(warn1) - 1); | ||
| 285 | |||
| 282 | /* check QEMU/SeaBIOS marker in PAGE0 */ | 286 | /* check QEMU/SeaBIOS marker in PAGE0 */ |
| 283 | running_on_qemu = (memcmp(&PAGE0->pad0, "SeaBIOS", 8) == 0); | 287 | running_on_qemu = (memcmp(&PAGE0->pad0, "SeaBIOS", 8) == 0); |
| 284 | 288 | ||
| @@ -165,6 +165,7 @@ SECTIONS · 164 unmodified lines | |||
| 165 | _end = . ; | 165 | _end = . ; |
| 166 | 166 | ||
| 167 | STABS_DEBUG | 167 | STABS_DEBUG |
| 168 | MODINFO | ||
| 168 | ELF_DETAILS | 169 | ELF_DETAILS |
| 169 | .note 0 : { *(.note) } | 170 | .note 0 : { *(.note) } |
| 170 | 171 | ||
| @@ -573,8 +573,8 @@ config ARCH_USING_PATCHABLE_FUNCTION_ENTRY · 572 unmodified lines | |||
| 573 | depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2) | 573 | depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2) |
| 574 | depends on $(cc-option,-fpatchable-function-entry=2) | 574 | depends on $(cc-option,-fpatchable-function-entry=2) |
| 575 | def_bool y if PPC32 | 575 | def_bool y if PPC32 |
| 576 | def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN | 576 | def_bool $(success,$(srctree)/arch/powerpc/tools/check-fpatchable-function-entry.sh $(CC) $(CLANG_FLAGS) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN |
| 577 | def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN | 577 | def_bool $(success,$(srctree)/arch/powerpc/tools/check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN |
| 578 | 578 | ||
| 579 | config PPC_FTRACE_OUT_OF_LINE | 579 | config PPC_FTRACE_OUT_OF_LINE |
| 580 | def_bool PPC64 && ARCH_USING_PATCHABLE_FUNCTION_ENTRY | 580 | def_bool PPC64 && ARCH_USING_PATCHABLE_FUNCTION_ENTRY |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | }; | 37 | }; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | memory { | 40 | memory@0 { |
| 41 | device_type = "memory"; | 41 | device_type = "memory"; |
| 42 | reg = <0x00000000 0x8000000>; // 128MB at 0 | 42 | reg = <0x00000000 0x8000000>; // 128MB at 0 |
| 43 | }; | 43 | }; |
| @@ -1,156 +0,0 @@ | |||
| 1 | /* T4240 Interlaken LAC Portal device tree stub with 24 portals. | ||
| 2 | * | ||
| 3 | * Copyright 2012 Freescale Semiconductor Inc. | ||
| 4 | * | ||
| 5 | * Redistribution and use in source and binary forms, with or without | ||
| 6 | * modification, are permitted provided that the following conditions are met: | ||
| 7 | * * Redistributions of source code must retain the above copyright | ||
| 8 | * notice, this list of conditions and the following disclaimer. | ||
| 9 | * * Redistributions in binary form must reproduce the above copyright | ||
| 10 | * notice, this list of conditions and the following disclaimer in the | ||
| 11 | * documentation and/or other materials provided with the distribution. | ||
| 12 | * * Neither the name of Freescale Semiconductor nor the | ||
| 13 | * names of its contributors may be used to endorse or promote products | ||
| 14 | * derived from this software without specific prior written permission. | ||
| 15 | * | ||
| 16 | * | ||
| 17 | * ALTERNATIVELY, this software may be distributed under the terms of the | ||
| 18 | * GNU General Public License ("GPL") as published by the Free Software | ||
| 19 | * Foundation, either version 2 of that License or (at your option) any | ||
| 20 | * later version. | ||
| 21 | * | ||
| 22 | * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY | ||
| 23 | * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| 24 | * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| 25 | * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | ||
| 26 | * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | ||
| 27 | * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
| 28 | * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | ||
| 29 | * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| 30 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | ||
| 31 | * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| 32 | */ | ||
| 33 | |||
| 34 | #address-cells = <0x1>; | ||
| 35 | #size-cells = <0x1>; | ||
| 36 | compatible = "fsl,interlaken-lac-portals"; | ||
| 37 | |||
| 38 | lportal0: lac-portal@0 { | ||
| 39 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 40 | reg = <0x0 0x1000>; | ||
| 41 | }; | ||
| 42 | |||
| 43 | lportal1: lac-portal@1000 { | ||
| 44 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 45 | reg = <0x1000 0x1000>; | ||
| 46 | }; | ||
| 47 | |||
| 48 | lportal2: lac-portal@2000 { | ||
| 49 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 50 | reg = <0x2000 0x1000>; | ||
| 51 | }; | ||
| 52 | |||
| 53 | lportal3: lac-portal@3000 { | ||
| 54 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 55 | reg = <0x3000 0x1000>; | ||
| 56 | }; | ||
| 57 | |||
| 58 | lportal4: lac-portal@4000 { | ||
| 59 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 60 | reg = <0x4000 0x1000>; | ||
| 61 | }; | ||
| 62 | |||
| 63 | lportal5: lac-portal@5000 { | ||
| 64 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 65 | reg = <0x5000 0x1000>; | ||
| 66 | }; | ||
| 67 | |||
| 68 | lportal6: lac-portal@6000 { | ||
| 69 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 70 | reg = <0x6000 0x1000>; | ||
| 71 | }; | ||
| 72 | |||
| 73 | lportal7: lac-portal@7000 { | ||
| 74 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 75 | reg = <0x7000 0x1000>; | ||
| 76 | }; | ||
| 77 | |||
| 78 | lportal8: lac-portal@8000 { | ||
| 79 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 80 | reg = <0x8000 0x1000>; | ||
| 81 | }; | ||
| 82 | |||
| 83 | lportal9: lac-portal@9000 { | ||
| 84 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 85 | reg = <0x9000 0x1000>; | ||
| 86 | }; | ||
| 87 | |||
| 88 | lportal10: lac-portal@A000 { | ||
| 89 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 90 | reg = <0xA000 0x1000>; | ||
| 91 | }; | ||
| 92 | |||
| 93 | lportal11: lac-portal@B000 { | ||
| 94 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 95 | reg = <0xB000 0x1000>; | ||
| 96 | }; | ||
| 97 | |||
| 98 | lportal12: lac-portal@C000 { | ||
| 99 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 100 | reg = <0xC000 0x1000>; | ||
| 101 | }; | ||
| 102 | |||
| 103 | lportal13: lac-portal@D000 { | ||
| 104 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 105 | reg = <0xD000 0x1000>; | ||
| 106 | }; | ||
| 107 | |||
| 108 | lportal14: lac-portal@E000 { | ||
| 109 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 110 | reg = <0xE000 0x1000>; | ||
| 111 | }; | ||
| 112 | |||
| 113 | lportal15: lac-portal@F000 { | ||
| 114 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 115 | reg = <0xF000 0x1000>; | ||
| 116 | }; | ||
| 117 | |||
| 118 | lportal16: lac-portal@10000 { | ||
| 119 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 120 | reg = <0x10000 0x1000>; | ||
| 121 | }; | ||
| 122 | |||
| 123 | lportal17: lac-portal@11000 { | ||
| 124 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 125 | reg = <0x11000 0x1000>; | ||
| 126 | }; | ||
| 127 | |||
| 128 | lportal18: lac-portal@1200 { | ||
| 129 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 130 | reg = <0x12000 0x1000>; | ||
| 131 | }; | ||
| 132 | |||
| 133 | lportal19: lac-portal@13000 { | ||
| 134 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 135 | reg = <0x13000 0x1000>; | ||
| 136 | }; | ||
| 137 | |||
| 138 | lportal20: lac-portal@14000 { | ||
| 139 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 140 | reg = <0x14000 0x1000>; | ||
| 141 | }; | ||
| 142 | |||
| 143 | lportal21: lac-portal@15000 { | ||
| 144 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 145 | reg = <0x15000 0x1000>; | ||
| 146 | }; | ||
| 147 | |||
| 148 | lportal22: lac-portal@16000 { | ||
| 149 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 150 | reg = <0x16000 0x1000>; | ||
| 151 | }; | ||
| 152 | |||
| 153 | lportal23: lac-portal@17000 { | ||
| 154 | compatible = "fsl,interlaken-lac-portal-v1.0"; | ||
| 155 | reg = <0x17000 0x1000>; | ||
| 156 | }; | ||
| @@ -1,45 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * T4 Interlaken Look-aside Controller (LAC) device tree stub | ||
| 3 | * | ||
| 4 | * Copyright 2012 Freescale Semiconductor Inc. | ||
| 5 | * | ||
| 6 | * Redistribution and use in source and binary forms, with or without | ||
| 7 | * modification, are permitted provided that the following conditions are met: | ||
| 8 | * * Redistributions of source code must retain the above copyright | ||
| 9 | * notice, this list of conditions and the following disclaimer. | ||
| 10 | * * Redistributions in binary form must reproduce the above copyright | ||
| 11 | * notice, this list of conditions and the following disclaimer in the | ||
| 12 | * documentation and/or other materials provided with the distribution. | ||
| 13 | * * Neither the name of Freescale Semiconductor nor the | ||
| 14 | * names of its contributors may be used to endorse or promote products | ||
| 15 | * derived from this software without specific prior written permission. | ||
| 16 | * | ||
| 17 | * | ||
| 18 | * ALTERNATIVELY, this software may be distributed under the terms of the | ||
| 19 | * GNU General Public License ("GPL") as published by the Free Software | ||
| 20 | * Foundation, either version 2 of that License or (at your option) any | ||
| 21 | * later version. | ||
| 22 | * | ||
| 23 | * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY | ||
| 24 | * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| 25 | * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| 26 | * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | ||
| 27 | * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | ||
| 28 | * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
| 29 | * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | ||
| 30 | * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | ||
| 32 | * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| 33 | */ | ||
| 34 | |||
| 35 | lac: lac@229000 { | ||
| 36 | compatible = "fsl,interlaken-lac"; | ||
| 37 | reg = <0x229000 0x1000>; | ||
| 38 | interrupts = <16 2 1 18>; | ||
| 39 | }; | ||
| 40 | |||
| 41 | lac-hv@228000 { | ||
| 42 | compatible = "fsl,interlaken-lac-hv"; | ||
| 43 | reg = <0x228000 0x1000>; | ||
| 44 | fsl,non-hv-node = <&lac>; | ||
| 45 | }; | ||
| @@ -1,43 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * PQ3 MPIC Message (Group B) device tree stub [ controller @ offset 0x42400 ] | ||
| 3 | * | ||
| 4 | * Copyright 2012 Freescale Semiconductor Inc. | ||
| 5 | * | ||
| 6 | * Redistribution and use in source and binary forms, with or without | ||
| 7 | * modification, are permitted provided that the following conditions are met: | ||
| 8 | * * Redistributions of source code must retain the above copyright | ||
| 9 | * notice, this list of conditions and the following disclaimer. | ||
| 10 | * * Redistributions in binary form must reproduce the above copyright | ||
| 11 | * notice, this list of conditions and the following disclaimer in the | ||
| 12 | * documentation and/or other materials provided with the distribution. | ||
| 13 | * * Neither the name of Freescale Semiconductor nor the | ||
| 14 | * names of its contributors may be used to endorse or promote products | ||
| 15 | * derived from this software without specific prior written permission. | ||
| 16 | * | ||
| 17 | * | ||
| 18 | * ALTERNATIVELY, this software may be distributed under the terms of the | ||
| 19 | * GNU General Public License ("GPL") as published by the Free Software | ||
| 20 | * Foundation, either version 2 of that License or (at your option) any | ||
| 21 | * later version. | ||
| 22 | * | ||
| 23 | * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY | ||
| 24 | * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| 25 | * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| 26 | * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | ||
| 27 | * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | ||
| 28 | * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
| 29 | * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | ||
| 30 | * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | ||
| 32 | * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| 33 | */ | ||
| 34 | |||
| 35 | message@42400 { | ||
| 36 | compatible = "fsl,mpic-v3.1-msgr"; | ||
| 37 | reg = <0x42400 0x200>; | ||
| 38 | interrupts = < | ||
| 39 | 0xb4 2 0 0 | ||
| 40 | 0xb5 2 0 0 | ||
| 41 | 0xb6 2 0 0 | ||
| 42 | 0xb7 2 0 0>; | ||
| 43 | }; | ||
| @@ -1,80 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * QorIQ FMan v3 1g port #1 device tree stub [ controller @ offset 0x400000 ] | ||
| 3 | * | ||
| 4 | * Copyright 2012 - 2015 Freescale Semiconductor Inc. | ||
| 5 | * | ||
| 6 | * Redistribution and use in source and binary forms, with or without | ||
| 7 | * modification, are permitted provided that the following conditions are met: | ||
| 8 | * * Redistributions of source code must retain the above copyright | ||
| 9 | * notice, this list of conditions and the following disclaimer. | ||
| 10 | * * Redistributions in binary form must reproduce the above copyright | ||
| 11 | * notice, this list of conditions and the following disclaimer in the | ||
| 12 | * documentation and/or other materials provided with the distribution. | ||
| 13 | * * Neither the name of Freescale Semiconductor nor the | ||
| 14 | * names of its contributors may be used to endorse or promote products | ||
| 15 | * derived from this software without specific prior written permission. | ||
| 16 | * | ||
| 17 | * | ||
| 18 | * ALTERNATIVELY, this software may be distributed under the terms of the | ||
| 19 | * GNU General Public License ("GPL") as published by the Free Software | ||
| 20 | * Foundation, either version 2 of that License or (at your option) any | ||
| 21 | * later version. | ||
| 22 | * | ||
| 23 | * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY | ||
| 24 | * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| 25 | * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| 26 | * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | ||
| 27 | * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | ||
| 28 | * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
| 29 | * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | ||
| 30 | * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | ||
| 32 | * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| 33 | */ | ||
| 34 | |||
| 35 | fman@400000 { | ||
| 36 | fman0_rx_0x09: port@89000 { | ||
| 37 | cell-index = <0x9>; | ||
| 38 | compatible = "fsl,fman-v3-port-rx"; | ||
| 39 | reg = <0x89000 0x1000>; | ||
| 40 | fsl,fman-10g-port; | ||
| 41 | fsl,fman-best-effort-port; | ||
| 42 | }; | ||
| 43 | |||
| 44 | fman0_tx_0x29: port@a9000 { | ||
| 45 | cell-index = <0x29>; | ||
| 46 | compatible = "fsl,fman-v3-port-tx"; | ||
| 47 | reg = <0xa9000 0x1000>; | ||
| 48 | fsl,fman-10g-port; | ||
| 49 | fsl,fman-best-effort-port; | ||
| 50 | }; | ||
| 51 | |||
| 52 | ethernet@e2000 { | ||
| 53 | cell-index = <1>; | ||
| 54 | compatible = "fsl,fman-memac"; | ||
| 55 | reg = <0xe2000 0x1000>; | ||
| 56 | fsl,fman-ports = <&fman0_rx_0x09 &fman0_tx_0x29>; | ||
| 57 | ptp-timer = <&ptp_timer0>; | ||
| 58 | pcsphy-handle = <&pcsphy1>, <&qsgmiia_pcs1>; | ||
| 59 | pcs-handle-names = "sgmii", "qsgmii"; | ||
| 60 | }; | ||
| 61 | |||
| 62 | mdio@e1000 { | ||
| 63 | qsgmiia_pcs1: ethernet-pcs@1 { | ||
| 64 | compatible = "fsl,lynx-pcs"; | ||
| 65 | reg = <1>; | ||
| 66 | }; | ||
| 67 | }; | ||
| 68 | |||
| 69 | mdio@e3000 { | ||
| 70 | #address-cells = <1>; | ||
| 71 | #size-cells = <0>; | ||
| 72 | compatible = "fsl,fman-memac-mdio", "fsl,fman-xmdio"; | ||
| 73 | reg = <0xe3000 0x1000>; | ||
| 74 | fsl,erratum-a011043; /* must ignore read errors */ | ||
| 75 | |||
| 76 | pcsphy1: ethernet-phy@0 { | ||
| 77 | reg = <0x0>; | ||
| 78 | }; | ||
| 79 | }; | ||
| 80 | }; | ||
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | }; | 37 | }; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | memory { | 40 | memory@0 { |
| 41 | device_type = "memory"; | 41 | device_type = "memory"; |
| 42 | reg = <0x00000000 0x08000000>; // 128MB at 0 | 42 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 43 | }; | 43 | }; |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | |||
| 38 | }; | 38 | }; |
| 39 | }; | 39 | }; |
| 40 | 40 | ||
| 41 | memory { | 41 | memory@0 { |
| 42 | device_type = "memory"; | 42 | device_type = "memory"; |
| 43 | reg = <0x00000000 0x08000000>; // 128MB at 0 | 43 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 44 | }; | 44 | }; |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | |||
| 6 | */ | 6 | */ |
| 7 | 7 | ||
| 8 | /dts-v1/; | 8 | /dts-v1/; |
| 9 | #include <dt-bindings/interrupt-controller/irq.h> | ||
| 9 | 10 | ||
| 10 | / { | 11 | / { |
| 11 | model = "MPC8313ERDB"; | 12 | model = "MPC8313ERDB"; |
| @@ -38,7 +39,7 @@ · 26 unmodified lines | |||
| 38 | }; | 39 | }; |
| 39 | }; | 40 | }; |
| 40 | 41 | ||
| 41 | memory { | 42 | memory@0 { |
| 42 | device_type = "memory"; | 43 | device_type = "memory"; |
| 43 | reg = <0x00000000 0x08000000>; // 128MB at 0 | 44 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 44 | }; | 45 | }; |
| @@ -48,7 +49,7 @@ · 3 unmodified lines | |||
| 48 | #size-cells = <1>; | 49 | #size-cells = <1>; |
| 49 | compatible = "fsl,mpc8313-elbc", "fsl,elbc", "simple-bus"; | 50 | compatible = "fsl,mpc8313-elbc", "fsl,elbc", "simple-bus"; |
| 50 | reg = <0xe0005000 0x1000>; | 51 | reg = <0xe0005000 0x1000>; |
| 51 | interrupts = <77 0x8>; | 52 | interrupts = <77 IRQ_TYPE_LEVEL_LOW>; |
| 52 | interrupt-parent = <&ipic>; | 53 | interrupt-parent = <&ipic>; |
| 53 | 54 | ||
| 54 | // CS0 and CS1 are swapped when | 55 | // CS0 and CS1 are swapped when |
| @@ -118,7 +119,7 @@ · 63 unmodified lines | |||
| 118 | cell-index = <0>; | 119 | cell-index = <0>; |
| 119 | compatible = "fsl-i2c"; | 120 | compatible = "fsl-i2c"; |
| 120 | reg = <0x3000 0x100>; | 121 | reg = <0x3000 0x100>; |
| 121 | interrupts = <14 0x8>; | 122 | interrupts = <14 IRQ_TYPE_LEVEL_LOW>; |
| 122 | interrupt-parent = <&ipic>; | 123 | interrupt-parent = <&ipic>; |
| 123 | dfsrr; | 124 | dfsrr; |
| 124 | rtc@68 { | 125 | rtc@68 { |
| @@ -131,7 +132,7 @@ · 6 unmodified lines | |||
| 131 | compatible = "fsl,sec2.2", "fsl,sec2.1", | 132 | compatible = "fsl,sec2.2", "fsl,sec2.1", |
| 132 | "fsl,sec2.0"; | 133 | "fsl,sec2.0"; |
| 133 | reg = <0x30000 0x10000>; | 134 | reg = <0x30000 0x10000>; |
| 134 | interrupts = <11 0x8>; | 135 | interrupts = <11 IRQ_TYPE_LEVEL_LOW>; |
| 135 | interrupt-parent = <&ipic>; | 136 | interrupt-parent = <&ipic>; |
| 136 | fsl,num-channels = <1>; | 137 | fsl,num-channels = <1>; |
| 137 | fsl,channel-fifo-len = <24>; | 138 | fsl,channel-fifo-len = <24>; |
| @@ -146,7 +147,7 @@ · 8 unmodified lines | |||
| 146 | cell-index = <1>; | 147 | cell-index = <1>; |
| 147 | compatible = "fsl-i2c"; | 148 | compatible = "fsl-i2c"; |
| 148 | reg = <0x3100 0x100>; | 149 | reg = <0x3100 0x100>; |
| 149 | interrupts = <15 0x8>; | 150 | interrupts = <15 IRQ_TYPE_LEVEL_LOW>; |
| 150 | interrupt-parent = <&ipic>; | 151 | interrupt-parent = <&ipic>; |
| 151 | dfsrr; | 152 | dfsrr; |
| 152 | }; | 153 | }; |
| @@ -155,7 +156,7 @@ · 2 unmodified lines | |||
| 155 | cell-index = <0>; | 156 | cell-index = <0>; |
| 156 | compatible = "fsl,spi"; | 157 | compatible = "fsl,spi"; |
| 157 | reg = <0x7000 0x1000>; | 158 | reg = <0x7000 0x1000>; |
| 158 | interrupts = <16 0x8>; | 159 | interrupts = <16 IRQ_TYPE_LEVEL_LOW>; |
| 159 | interrupt-parent = <&ipic>; | 160 | interrupt-parent = <&ipic>; |
| 160 | mode = "cpu"; | 161 | mode = "cpu"; |
| 161 | }; | 162 | }; |
| @@ -167,7 +168,7 @@ · 5 unmodified lines | |||
| 167 | #address-cells = <1>; | 168 | #address-cells = <1>; |
| 168 | #size-cells = <0>; | 169 | #size-cells = <0>; |
| 169 | interrupt-parent = <&ipic>; | 170 | interrupt-parent = <&ipic>; |
| 170 | interrupts = <38 0x8>; | 171 | interrupts = <38 IRQ_TYPE_LEVEL_LOW>; |
| 171 | phy_type = "utmi_wide"; | 172 | phy_type = "utmi_wide"; |
| 172 | sleep = <&pmc 0x00300000>; | 173 | sleep = <&pmc 0x00300000>; |
| 173 | }; | 174 | }; |
| @@ -175,7 +176,8 @@ · 1 unmodified line | |||
| 175 | ptp_clock@24E00 { | 176 | ptp_clock@24E00 { |
| 176 | compatible = "fsl,etsec-ptp"; | 177 | compatible = "fsl,etsec-ptp"; |
| 177 | reg = <0x24E00 0xB0>; | 178 | reg = <0x24E00 0xB0>; |
| 178 | interrupts = <12 0x8 13 0x8>; | 179 | interrupts = <12 IRQ_TYPE_LEVEL_LOW>, |
| 180 | <13 IRQ_TYPE_LEVEL_LOW>; | ||
| 179 | interrupt-parent = < &ipic >; | 181 | interrupt-parent = < &ipic >; |
| 180 | fsl,tclk-period = <10>; | 182 | fsl,tclk-period = <10>; |
| 181 | fsl,tmr-prsc = <100>; | 183 | fsl,tmr-prsc = <100>; |
| @@ -197,7 +199,9 @@ · 15 unmodified lines | |||
| 197 | compatible = "gianfar"; | 199 | compatible = "gianfar"; |
| 198 | reg = <0x24000 0x1000>; | 200 | reg = <0x24000 0x1000>; |
| 199 | local-mac-address = [ 00 00 00 00 00 00 ]; | 201 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 200 | interrupts = <37 0x8 36 0x8 35 0x8>; | 202 | interrupts = <37 IRQ_TYPE_LEVEL_LOW>, |
| 203 | <36 IRQ_TYPE_LEVEL_LOW>, | ||
| 204 | <35 IRQ_TYPE_LEVEL_LOW>; | ||
| 201 | interrupt-parent = <&ipic>; | 205 | interrupt-parent = <&ipic>; |
| 202 | tbi-handle = < &tbi0 >; | 206 | tbi-handle = < &tbi0 >; |
| 203 | /* Vitesse 7385 isn't on the MDIO bus */ | 207 | /* Vitesse 7385 isn't on the MDIO bus */ |
| @@ -211,7 +215,7 @@ · 7 unmodified lines | |||
| 211 | reg = <0x520 0x20>; | 215 | reg = <0x520 0x20>; |
| 212 | phy4: ethernet-phy@4 { | 216 | phy4: ethernet-phy@4 { |
| 213 | interrupt-parent = <&ipic>; | 217 | interrupt-parent = <&ipic>; |
| 214 | interrupts = <20 0x8>; | 218 | interrupts = <20 IRQ_TYPE_LEVEL_LOW>; |
| 215 | reg = <0x4>; | 219 | reg = <0x4>; |
| 216 | }; | 220 | }; |
| 217 | tbi0: tbi-phy@11 { | 221 | tbi0: tbi-phy@11 { |
| @@ -231,7 +235,9 @@ · 13 unmodified lines | |||
| 231 | reg = <0x25000 0x1000>; | 235 | reg = <0x25000 0x1000>; |
| 232 | ranges = <0x0 0x25000 0x1000>; | 236 | ranges = <0x0 0x25000 0x1000>; |
| 233 | local-mac-address = [ 00 00 00 00 00 00 ]; | 237 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 234 | interrupts = <34 0x8 33 0x8 32 0x8>; | 238 | interrupts = <34 IRQ_TYPE_LEVEL_LOW>, |
| 239 | <33 IRQ_TYPE_LEVEL_LOW>, | ||
| 240 | <32 IRQ_TYPE_LEVEL_LOW>; | ||
| 235 | interrupt-parent = <&ipic>; | 241 | interrupt-parent = <&ipic>; |
| 236 | tbi-handle = < &tbi1 >; | 242 | tbi-handle = < &tbi1 >; |
| 237 | phy-handle = < &phy4 >; | 243 | phy-handle = < &phy4 >; |
| @@ -259,7 +265,7 @@ · 21 unmodified lines | |||
| 259 | compatible = "fsl,ns16550", "ns16550"; | 265 | compatible = "fsl,ns16550", "ns16550"; |
| 260 | reg = <0x4500 0x100>; | 266 | reg = <0x4500 0x100>; |
| 261 | clock-frequency = <0>; | 267 | clock-frequency = <0>; |
| 262 | interrupts = <9 0x8>; | 268 | interrupts = <9 IRQ_TYPE_LEVEL_LOW>; |
| 263 | interrupt-parent = <&ipic>; | 269 | interrupt-parent = <&ipic>; |
| 264 | }; | 270 | }; |
| 265 | 271 | ||
| @@ -269,15 +275,12 @@ · 3 unmodified lines | |||
| 269 | compatible = "fsl,ns16550", "ns16550"; | 275 | compatible = "fsl,ns16550", "ns16550"; |
| 270 | reg = <0x4600 0x100>; | 276 | reg = <0x4600 0x100>; |
| 271 | clock-frequency = <0>; | 277 | clock-frequency = <0>; |
| 272 | interrupts = <10 0x8>; | 278 | interrupts = <10 IRQ_TYPE_LEVEL_LOW>; |
| 273 | interrupt-parent = <&ipic>; | 279 | interrupt-parent = <&ipic>; |
| 274 | }; | 280 | }; |
| 275 | 281 | ||
| 276 | /* IPIC | 282 | /* IPIC |
| 277 | * interrupts cell = <intr #, sense> | 283 | * interrupts cell = <intr #, type> |
| 278 | * sense values match linux IORESOURCE_IRQ_* defines: | ||
| 279 | * sense == 8: Level, low assertion | ||
| 280 | * sense == 2: Edge, high-to-low change | ||
| 281 | */ | 284 | */ |
| 282 | ipic: pic@700 { | 285 | ipic: pic@700 { |
| 283 | interrupt-controller; | 286 | interrupt-controller; |
| @@ -290,7 +293,7 @@ · 6 unmodified lines | |||
| 290 | pmc: power@b00 { | 293 | pmc: power@b00 { |
| 291 | compatible = "fsl,mpc8313-pmc", "fsl,mpc8349-pmc"; | 294 | compatible = "fsl,mpc8313-pmc", "fsl,mpc8349-pmc"; |
| 292 | reg = <0xb00 0x100 0xa00 0x100>; | 295 | reg = <0xb00 0x100 0xa00 0x100>; |
| 293 | interrupts = <80 8>; | 296 | interrupts = <80 IRQ_TYPE_LEVEL_LOW>; |
| 294 | interrupt-parent = <&ipic>; | 297 | interrupt-parent = <&ipic>; |
| 295 | fsl,mpc8313-wakeup-timer = <>m1>; | 298 | fsl,mpc8313-wakeup-timer = <>m1>; |
| 296 | 299 | ||
| @@ -306,14 +309,20 @@ · 9 unmodified lines | |||
| 306 | gtm1: timer@500 { | 309 | gtm1: timer@500 { |
| 307 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; | 310 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; |
| 308 | reg = <0x500 0x100>; | 311 | reg = <0x500 0x100>; |
| 309 | interrupts = <90 8 78 8 84 8 72 8>; | 312 | interrupts = <90 IRQ_TYPE_LEVEL_LOW>, |
| 313 | <78 IRQ_TYPE_LEVEL_LOW>, | ||
| 314 | <84 IRQ_TYPE_LEVEL_LOW>, | ||
| 315 | <72 IRQ_TYPE_LEVEL_LOW>; | ||
| 310 | interrupt-parent = <&ipic>; | 316 | interrupt-parent = <&ipic>; |
| 311 | }; | 317 | }; |
| 312 | 318 | ||
| 313 | timer@600 { | 319 | timer@600 { |
| 314 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; | 320 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; |
| 315 | reg = <0x600 0x100>; | 321 | reg = <0x600 0x100>; |
| 316 | interrupts = <91 8 79 8 85 8 73 8>; | 322 | interrupts = <91 IRQ_TYPE_LEVEL_LOW>, |
| 323 | <79 IRQ_TYPE_LEVEL_LOW>, | ||
| 324 | <85 IRQ_TYPE_LEVEL_LOW>, | ||
| 325 | <73 IRQ_TYPE_LEVEL_LOW>; | ||
| 317 | interrupt-parent = <&ipic>; | 326 | interrupt-parent = <&ipic>; |
| 318 | }; | 327 | }; |
| 319 | }; | 328 | }; |
| @@ -341,7 +350,7 @@ · 21 unmodified lines | |||
| 341 | 0x7800 0x0 0x0 0x3 &ipic 17 0x8 | 350 | 0x7800 0x0 0x0 0x3 &ipic 17 0x8 |
| 342 | 0x7800 0x0 0x0 0x4 &ipic 18 0x8>; | 351 | 0x7800 0x0 0x0 0x4 &ipic 18 0x8>; |
| 343 | interrupt-parent = <&ipic>; | 352 | interrupt-parent = <&ipic>; |
| 344 | interrupts = <66 0x8>; | 353 | interrupts = <66 IRQ_TYPE_LEVEL_LOW>; |
| 345 | bus-range = <0x0 0x0>; | 354 | bus-range = <0x0 0x0>; |
| 346 | ranges = <0x02000000 0x0 0x90000000 0x90000000 0x0 0x10000000 | 355 | ranges = <0x02000000 0x0 0x90000000 0x90000000 0x0 0x10000000 |
| 347 | 0x42000000 0x0 0x80000000 0x80000000 0x0 0x10000000 | 356 | 0x42000000 0x0 0x80000000 0x80000000 0x0 0x10000000 |
| @@ -363,14 +372,14 @@ · 15 unmodified lines | |||
| 363 | reg = <0xe00082a8 4>; | 372 | reg = <0xe00082a8 4>; |
| 364 | ranges = <0 0xe0008100 0x1a8>; | 373 | ranges = <0 0xe0008100 0x1a8>; |
| 365 | interrupt-parent = <&ipic>; | 374 | interrupt-parent = <&ipic>; |
| 366 | interrupts = <71 8>; | 375 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 367 | 376 | ||
| 368 | dma-channel@0 { | 377 | dma-channel@0 { |
| 369 | compatible = "fsl,mpc8313-dma-channel", | 378 | compatible = "fsl,mpc8313-dma-channel", |
| 370 | "fsl,elo-dma-channel"; | 379 | "fsl,elo-dma-channel"; |
| 371 | reg = <0 0x28>; | 380 | reg = <0 0x28>; |
| 372 | interrupt-parent = <&ipic>; | 381 | interrupt-parent = <&ipic>; |
| 373 | interrupts = <71 8>; | 382 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 374 | cell-index = <0>; | 383 | cell-index = <0>; |
| 375 | }; | 384 | }; |
| 376 | 385 | ||
| @@ -379,7 +388,7 @@ · 2 unmodified lines | |||
| 379 | "fsl,elo-dma-channel"; | 388 | "fsl,elo-dma-channel"; |
| 380 | reg = <0x80 0x28>; | 389 | reg = <0x80 0x28>; |
| 381 | interrupt-parent = <&ipic>; | 390 | interrupt-parent = <&ipic>; |
| 382 | interrupts = <71 8>; | 391 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 383 | cell-index = <1>; | 392 | cell-index = <1>; |
| 384 | }; | 393 | }; |
| 385 | 394 | ||
| @@ -388,7 +397,7 @@ · 2 unmodified lines | |||
| 388 | "fsl,elo-dma-channel"; | 397 | "fsl,elo-dma-channel"; |
| 389 | reg = <0x100 0x28>; | 398 | reg = <0x100 0x28>; |
| 390 | interrupt-parent = <&ipic>; | 399 | interrupt-parent = <&ipic>; |
| 391 | interrupts = <71 8>; | 400 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 392 | cell-index = <2>; | 401 | cell-index = <2>; |
| 393 | }; | 402 | }; |
| 394 | 403 | ||
| @@ -397,7 +406,7 @@ · 2 unmodified lines | |||
| 397 | "fsl,elo-dma-channel"; | 406 | "fsl,elo-dma-channel"; |
| 398 | reg = <0x180 0x28>; | 407 | reg = <0x180 0x28>; |
| 399 | interrupt-parent = <&ipic>; | 408 | interrupt-parent = <&ipic>; |
| 400 | interrupts = <71 8>; | 409 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 401 | cell-index = <3>; | 410 | cell-index = <3>; |
| 402 | }; | 411 | }; |
| 403 | }; | 412 | }; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | |||
| 40 | }; | 40 | }; |
| 41 | }; | 41 | }; |
| 42 | 42 | ||
| 43 | memory { | 43 | memory@0 { |
| 44 | device_type = "memory"; | 44 | device_type = "memory"; |
| 45 | reg = <0x00000000 0x08000000>; // 128MB at 0 | 45 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 46 | }; | 46 | }; |
| @@ -50,7 +50,7 @@ · 3 unmodified lines | |||
| 50 | #size-cells = <1>; | 50 | #size-cells = <1>; |
| 51 | compatible = "fsl,mpc8315-elbc", "fsl,elbc", "simple-bus"; | 51 | compatible = "fsl,mpc8315-elbc", "fsl,elbc", "simple-bus"; |
| 52 | reg = <0xe0005000 0x1000>; | 52 | reg = <0xe0005000 0x1000>; |
| 53 | interrupts = <77 0x8>; | 53 | interrupts = <77 IRQ_TYPE_LEVEL_LOW>; |
| 54 | interrupt-parent = <&ipic>; | 54 | interrupt-parent = <&ipic>; |
| 55 | 55 | ||
| 56 | // CS0 and CS1 are swapped when | 56 | // CS0 and CS1 are swapped when |
| @@ -112,7 +112,7 @@ · 55 unmodified lines | |||
| 112 | cell-index = <0>; | 112 | cell-index = <0>; |
| 113 | compatible = "fsl-i2c"; | 113 | compatible = "fsl-i2c"; |
| 114 | reg = <0x3000 0x100>; | 114 | reg = <0x3000 0x100>; |
| 115 | interrupts = <14 0x8>; | 115 | interrupts = <14 IRQ_TYPE_LEVEL_LOW>; |
| 116 | interrupt-parent = <&ipic>; | 116 | interrupt-parent = <&ipic>; |
| 117 | dfsrr; | 117 | dfsrr; |
| 118 | rtc@68 { | 118 | rtc@68 { |
| @@ -133,8 +133,10 @@ · 14 unmodified lines | |||
| 133 | cell-index = <0>; | 133 | cell-index = <0>; |
| 134 | compatible = "fsl,spi"; | 134 | compatible = "fsl,spi"; |
| 135 | reg = <0x7000 0x1000>; | 135 | reg = <0x7000 0x1000>; |
| 136 | interrupts = <16 0x8>; | 136 | interrupts = <16 IRQ_TYPE_LEVEL_LOW>; |
| 137 | interrupt-parent = <&ipic>; | 137 | interrupt-parent = <&ipic>; |
| 138 | #address-cells = <1>; | ||
| 139 | #size-cells = <0>; | ||
| 138 | mode = "cpu"; | 140 | mode = "cpu"; |
| 139 | }; | 141 | }; |
| 140 | 142 | ||
| @@ -145,35 +147,35 @@ · 4 unmodified lines | |||
| 145 | reg = <0x82a8 4>; | 147 | reg = <0x82a8 4>; |
| 146 | ranges = <0 0x8100 0x1a8>; | 148 | ranges = <0 0x8100 0x1a8>; |
| 147 | interrupt-parent = <&ipic>; | 149 | interrupt-parent = <&ipic>; |
| 148 | interrupts = <71 8>; | 150 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 149 | cell-index = <0>; | 151 | cell-index = <0>; |
| 150 | dma-channel@0 { | 152 | dma-channel@0 { |
| 151 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; | 153 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 152 | reg = <0 0x80>; | 154 | reg = <0 0x80>; |
| 153 | cell-index = <0>; | 155 | cell-index = <0>; |
| 154 | interrupt-parent = <&ipic>; | 156 | interrupt-parent = <&ipic>; |
| 155 | interrupts = <71 8>; | 157 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 156 | }; | 158 | }; |
| 157 | dma-channel@80 { | 159 | dma-channel@80 { |
| 158 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; | 160 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 159 | reg = <0x80 0x80>; | 161 | reg = <0x80 0x80>; |
| 160 | cell-index = <1>; | 162 | cell-index = <1>; |
| 161 | interrupt-parent = <&ipic>; | 163 | interrupt-parent = <&ipic>; |
| 162 | interrupts = <71 8>; | 164 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 163 | }; | 165 | }; |
| 164 | dma-channel@100 { | 166 | dma-channel@100 { |
| 165 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; | 167 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 166 | reg = <0x100 0x80>; | 168 | reg = <0x100 0x80>; |
| 167 | cell-index = <2>; | 169 | cell-index = <2>; |
| 168 | interrupt-parent = <&ipic>; | 170 | interrupt-parent = <&ipic>; |
| 169 | interrupts = <71 8>; | 171 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 170 | }; | 172 | }; |
| 171 | dma-channel@180 { | 173 | dma-channel@180 { |
| 172 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; | 174 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 173 | reg = <0x180 0x28>; | 175 | reg = <0x180 0x28>; |
| 174 | cell-index = <3>; | 176 | cell-index = <3>; |
| 175 | interrupt-parent = <&ipic>; | 177 | interrupt-parent = <&ipic>; |
| 176 | interrupts = <71 8>; | 178 | interrupts = <71 IRQ_TYPE_LEVEL_LOW>; |
| 177 | }; | 179 | }; |
| 178 | }; | 180 | }; |
| 179 | 181 | ||
| @@ -183,7 +185,7 @@ · 3 unmodified lines | |||
| 183 | #address-cells = <1>; | 185 | #address-cells = <1>; |
| 184 | #size-cells = <0>; | 186 | #size-cells = <0>; |
| 185 | interrupt-parent = <&ipic>; | 187 | interrupt-parent = <&ipic>; |
| 186 | interrupts = <38 0x8>; | 188 | interrupts = <38 IRQ_TYPE_LEVEL_LOW>; |
| 187 | phy_type = "utmi"; | 189 | phy_type = "utmi"; |
| 188 | }; | 190 | }; |
| 189 | 191 | ||
| @@ -197,7 +199,9 @@ · 7 unmodified lines | |||
| 197 | reg = <0x24000 0x1000>; | 199 | reg = <0x24000 0x1000>; |
| 198 | ranges = <0x0 0x24000 0x1000>; | 200 | ranges = <0x0 0x24000 0x1000>; |
| 199 | local-mac-address = [ 00 00 00 00 00 00 ]; | 201 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 200 | interrupts = <32 0x8 33 0x8 34 0x8>; | 202 | interrupts = <32 IRQ_TYPE_LEVEL_LOW>, |
| 203 | <33 IRQ_TYPE_LEVEL_LOW>, | ||
| 204 | <34 IRQ_TYPE_LEVEL_LOW>; | ||
| 201 | interrupt-parent = <&ipic>; | 205 | interrupt-parent = <&ipic>; |
| 202 | tbi-handle = <&tbi0>; | 206 | tbi-handle = <&tbi0>; |
| 203 | phy-handle = < &phy0 >; | 207 | phy-handle = < &phy0 >; |
| @@ -238,7 +242,9 @@ · 34 unmodified lines | |||
| 238 | reg = <0x25000 0x1000>; | 242 | reg = <0x25000 0x1000>; |
| 239 | ranges = <0x0 0x25000 0x1000>; | 243 | ranges = <0x0 0x25000 0x1000>; |
| 240 | local-mac-address = [ 00 00 00 00 00 00 ]; | 244 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 241 | interrupts = <35 0x8 36 0x8 37 0x8>; | 245 | interrupts = <35 IRQ_TYPE_LEVEL_LOW>, |
| 246 | <36 IRQ_TYPE_LEVEL_LOW>, | ||
| 247 | <37 IRQ_TYPE_LEVEL_LOW>; | ||
| 242 | interrupt-parent = <&ipic>; | 248 | interrupt-parent = <&ipic>; |
| 243 | tbi-handle = <&tbi1>; | 249 | tbi-handle = <&tbi1>; |
| 244 | phy-handle = < &phy1 >; | 250 | phy-handle = < &phy1 >; |
| @@ -263,7 +269,7 @@ · 18 unmodified lines | |||
| 263 | compatible = "fsl,ns16550", "ns16550"; | 269 | compatible = "fsl,ns16550", "ns16550"; |
| 264 | reg = <0x4500 0x100>; | 270 | reg = <0x4500 0x100>; |
| 265 | clock-frequency = <133333333>; | 271 | clock-frequency = <133333333>; |
| 266 | interrupts = <9 0x8>; | 272 | interrupts = <9 IRQ_TYPE_LEVEL_LOW>; |
| 267 | interrupt-parent = <&ipic>; | 273 | interrupt-parent = <&ipic>; |
| 268 | }; | 274 | }; |
| 269 | 275 | ||
| @@ -273,7 +279,7 @@ · 3 unmodified lines | |||
| 273 | compatible = "fsl,ns16550", "ns16550"; | 279 | compatible = "fsl,ns16550", "ns16550"; |
| 274 | reg = <0x4600 0x100>; | 280 | reg = <0x4600 0x100>; |
| 275 | clock-frequency = <133333333>; | 281 | clock-frequency = <133333333>; |
| 276 | interrupts = <10 0x8>; | 282 | interrupts = <10 IRQ_TYPE_LEVEL_LOW>; |
| 277 | interrupt-parent = <&ipic>; | 283 | interrupt-parent = <&ipic>; |
| 278 | }; | 284 | }; |
| 279 | 285 | ||
| @@ -282,7 +288,7 @@ · 2 unmodified lines | |||
| 282 | "fsl,sec2.4", "fsl,sec2.2", "fsl,sec2.1", | 288 | "fsl,sec2.4", "fsl,sec2.2", "fsl,sec2.1", |
| 283 | "fsl,sec2.0"; | 289 | "fsl,sec2.0"; |
| 284 | reg = <0x30000 0x10000>; | 290 | reg = <0x30000 0x10000>; |
| 285 | interrupts = <11 0x8>; | 291 | interrupts = <11 IRQ_TYPE_LEVEL_LOW>; |
| 286 | interrupt-parent = <&ipic>; | 292 | interrupt-parent = <&ipic>; |
| 287 | fsl,num-channels = <4>; | 293 | fsl,num-channels = <4>; |
| 288 | fsl,channel-fifo-len = <24>; | 294 | fsl,channel-fifo-len = <24>; |
| @@ -294,7 +300,7 @@ · 5 unmodified lines | |||
| 294 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; | 300 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; |
| 295 | reg = <0x18000 0x1000>; | 301 | reg = <0x18000 0x1000>; |
| 296 | cell-index = <1>; | 302 | cell-index = <1>; |
| 297 | interrupts = <44 0x8>; | 303 | interrupts = <44 IRQ_TYPE_LEVEL_LOW>; |
| 298 | interrupt-parent = <&ipic>; | 304 | interrupt-parent = <&ipic>; |
| 299 | }; | 305 | }; |
| 300 | 306 | ||
| @@ -302,14 +308,17 @@ · 1 unmodified line | |||
| 302 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; | 308 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; |
| 303 | reg = <0x19000 0x1000>; | 309 | reg = <0x19000 0x1000>; |
| 304 | cell-index = <2>; | 310 | cell-index = <2>; |
| 305 | interrupts = <45 0x8>; | 311 | interrupts = <45 IRQ_TYPE_LEVEL_LOW>; |
| 306 | interrupt-parent = <&ipic>; | 312 | interrupt-parent = <&ipic>; |
| 307 | }; | 313 | }; |
| 308 | 314 | ||
| 309 | gtm1: timer@500 { | 315 | gtm1: timer@500 { |
| 310 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; | 316 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; |
| 311 | reg = <0x500 0x100>; | 317 | reg = <0x500 0x100>; |
| 312 | interrupts = <90 8 78 8 84 8 72 8>; | 318 | interrupts = <90 IRQ_TYPE_LEVEL_LOW>, |
| 319 | <78 IRQ_TYPE_LEVEL_LOW>, | ||
| 320 | <84 IRQ_TYPE_LEVEL_LOW>, | ||
| 321 | <72 IRQ_TYPE_LEVEL_LOW>; | ||
| 313 | interrupt-parent = <&ipic>; | 322 | interrupt-parent = <&ipic>; |
| 314 | clock-frequency = <133333333>; | 323 | clock-frequency = <133333333>; |
| 315 | }; | 324 | }; |
| @@ -317,16 +326,16 @@ · 1 unmodified line | |||
| 317 | timer@600 { | 326 | timer@600 { |
| 318 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; | 327 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; |
| 319 | reg = <0x600 0x100>; | 328 | reg = <0x600 0x100>; |
| 320 | interrupts = <91 8 79 8 85 8 73 8>; | 329 | interrupts = <91 IRQ_TYPE_LEVEL_LOW>, |
| 330 | <79 IRQ_TYPE_LEVEL_LOW>, | ||
| 331 | <85 IRQ_TYPE_LEVEL_LOW>, | ||
| 332 | <73 IRQ_TYPE_LEVEL_LOW>; | ||
| 321 | interrupt-parent = <&ipic>; | 333 | interrupt-parent = <&ipic>; |
| 322 | clock-frequency = <133333333>; | 334 | clock-frequency = <133333333>; |
| 323 | }; | 335 | }; |
| 324 | 336 | ||
| 325 | /* IPIC | 337 | /* IPIC |
| 326 | * interrupts cell = <intr #, sense> | 338 | * interrupts cell = <intr #, type> |
| 327 | * sense values match linux IORESOURCE_IRQ_* defines: | ||
| 328 | * sense == 8: Level, low assertion | ||
| 329 | * sense == 2: Edge, high-to-low change | ||
| 330 | */ | 339 | */ |
| 331 | ipic: interrupt-controller@700 { | 340 | ipic: interrupt-controller@700 { |
| 332 | interrupt-controller; | 341 | interrupt-controller; |
| @@ -340,14 +349,14 @@ · 7 unmodified lines | |||
| 340 | compatible = "fsl,ipic-msi"; | 349 | compatible = "fsl,ipic-msi"; |
| 341 | reg = <0x7c0 0x40>; | 350 | reg = <0x7c0 0x40>; |
| 342 | msi-available-ranges = <0 0x100>; | 351 | msi-available-ranges = <0 0x100>; |
| 343 | interrupts = <0x43 0x8 | 352 | interrupts = <0x43 IRQ_TYPE_LEVEL_LOW |
| 344 | 0x4 0x8 | 353 | 0x4 IRQ_TYPE_LEVEL_LOW |
| 345 | 0x51 0x8 | 354 | 0x51 IRQ_TYPE_LEVEL_LOW |
| 346 | 0x52 0x8 | 355 | 0x52 IRQ_TYPE_LEVEL_LOW |
| 347 | 0x56 0x8 | 356 | 0x56 IRQ_TYPE_LEVEL_LOW |
| 348 | 0x57 0x8 | 357 | 0x57 IRQ_TYPE_LEVEL_LOW |
| 349 | 0x58 0x8 | 358 | 0x58 IRQ_TYPE_LEVEL_LOW |
| 350 | 0x59 0x8>; | 359 | 0x59 IRQ_TYPE_LEVEL_LOW>; |
| 351 | interrupt-parent = < &ipic >; | 360 | interrupt-parent = < &ipic >; |
| 352 | }; | 361 | }; |
| 353 | 362 | ||
| @@ -355,7 +364,7 @@ · 1 unmodified line | |||
| 355 | compatible = "fsl,mpc8315-pmc", "fsl,mpc8313-pmc", | 364 | compatible = "fsl,mpc8315-pmc", "fsl,mpc8313-pmc", |
| 356 | "fsl,mpc8349-pmc"; | 365 | "fsl,mpc8349-pmc"; |
| 357 | reg = <0xb00 0x100 0xa00 0x100>; | 366 | reg = <0xb00 0x100 0xa00 0x100>; |
| 358 | interrupts = <80 8>; | 367 | interrupts = <80 IRQ_TYPE_LEVEL_LOW>; |
| 359 | interrupt-parent = <&ipic>; | 368 | interrupt-parent = <&ipic>; |
| 360 | fsl,mpc8313-wakeup-timer = <>m1>; | 369 | fsl,mpc8313-wakeup-timer = <>m1>; |
| 361 | }; | 370 | }; |
| @@ -374,24 +383,24 @@ · 12 unmodified lines | |||
| 374 | interrupt-map-mask = <0xf800 0x0 0x0 0x7>; | 383 | interrupt-map-mask = <0xf800 0x0 0x0 0x7>; |
| 375 | interrupt-map = < | 384 | interrupt-map = < |
| 376 | /* IDSEL 0x0E -mini PCI */ | 385 | /* IDSEL 0x0E -mini PCI */ |
| 377 | 0x7000 0x0 0x0 0x1 &ipic 18 0x8 | 386 | 0x7000 0x0 0x0 0x1 &ipic 18 IRQ_TYPE_LEVEL_LOW |
| 378 | 0x7000 0x0 0x0 0x2 &ipic 18 0x8 | 387 | 0x7000 0x0 0x0 0x2 &ipic 18 IRQ_TYPE_LEVEL_LOW |
| 379 | 0x7000 0x0 0x0 0x3 &ipic 18 0x8 | 388 | 0x7000 0x0 0x0 0x3 &ipic 18 IRQ_TYPE_LEVEL_LOW |
| 380 | 0x7000 0x0 0x0 0x4 &ipic 18 0x8 | 389 | 0x7000 0x0 0x0 0x4 &ipic 18 IRQ_TYPE_LEVEL_LOW |
| 381 | 390 | ||
| 382 | /* IDSEL 0x0F -mini PCI */ | 391 | /* IDSEL 0x0F -mini PCI */ |
| 383 | 0x7800 0x0 0x0 0x1 &ipic 17 0x8 | 392 | 0x7800 0x0 0x0 0x1 &ipic 17 IRQ_TYPE_LEVEL_LOW |
| 384 | 0x7800 0x0 0x0 0x2 &ipic 17 0x8 | 393 | 0x7800 0x0 0x0 0x2 &ipic 17 IRQ_TYPE_LEVEL_LOW |
| 385 | 0x7800 0x0 0x0 0x3 &ipic 17 0x8 | 394 | 0x7800 0x0 0x0 0x3 &ipic 17 IRQ_TYPE_LEVEL_LOW |
| 386 | 0x7800 0x0 0x0 0x4 &ipic 17 0x8 | 395 | 0x7800 0x0 0x0 0x4 &ipic 17 IRQ_TYPE_LEVEL_LOW |
| 387 | 396 | ||
| 388 | /* IDSEL 0x10 - PCI slot */ | 397 | /* IDSEL 0x10 - PCI slot */ |
| 389 | 0x8000 0x0 0x0 0x1 &ipic 48 0x8 | 398 | 0x8000 0x0 0x0 0x1 &ipic 48 IRQ_TYPE_LEVEL_LOW |
| 390 | 0x8000 0x0 0x0 0x2 &ipic 17 0x8 | 399 | 0x8000 0x0 0x0 0x2 &ipic 17 IRQ_TYPE_LEVEL_LOW |
| 391 | 0x8000 0x0 0x0 0x3 &ipic 48 0x8 | 400 | 0x8000 0x0 0x0 0x3 &ipic 48 IRQ_TYPE_LEVEL_LOW |
| 392 | 0x8000 0x0 0x0 0x4 &ipic 17 0x8>; | 401 | 0x8000 0x0 0x0 0x4 &ipic 17 IRQ_TYPE_LEVEL_LOW>; |
| 393 | interrupt-parent = <&ipic>; | 402 | interrupt-parent = <&ipic>; |
| 394 | interrupts = <66 0x8>; | 403 | interrupts = <66 IRQ_TYPE_LEVEL_LOW>; |
| 395 | bus-range = <0x0 0x0>; | 404 | bus-range = <0x0 0x0>; |
| 396 | ranges = <0x02000000 0 0x90000000 0x90000000 0 0x10000000 | 405 | ranges = <0x02000000 0 0x90000000 0x90000000 0 0x10000000 |
| 397 | 0x42000000 0 0x80000000 0x80000000 0 0x10000000 | 406 | 0x42000000 0 0x80000000 0x80000000 0 0x10000000 |
| @@ -417,10 +426,10 @@ · 19 unmodified lines | |||
| 417 | 0x01000000 0 0x00000000 0xb1000000 0 0x00800000>; | 426 | 0x01000000 0 0x00000000 0xb1000000 0 0x00800000>; |
| 418 | bus-range = <0 255>; | 427 | bus-range = <0 255>; |
| 419 | interrupt-map-mask = <0xf800 0 0 7>; | 428 | interrupt-map-mask = <0xf800 0 0 7>; |
| 420 | interrupt-map = <0 0 0 1 &ipic 1 8 | 429 | interrupt-map = <0 0 0 1 &ipic 1 IRQ_TYPE_LEVEL_LOW |
| 421 | 0 0 0 2 &ipic 1 8 | 430 | 0 0 0 2 &ipic 1 IRQ_TYPE_LEVEL_LOW |
| 422 | 0 0 0 3 &ipic 1 8 | 431 | 0 0 0 3 &ipic 1 IRQ_TYPE_LEVEL_LOW |
| 423 | 0 0 0 4 &ipic 1 8>; | 432 | 0 0 0 4 &ipic 1 IRQ_TYPE_LEVEL_LOW>; |
| 424 | clock-frequency = <0>; | 433 | clock-frequency = <0>; |
| 425 | 434 | ||
| 426 | pcie@0 { | 435 | pcie@0 { |
| @@ -448,10 +457,10 @@ · 21 unmodified lines | |||
| 448 | 0x01000000 0 0x00000000 0xd1000000 0 0x00800000>; | 457 | 0x01000000 0 0x00000000 0xd1000000 0 0x00800000>; |
| 449 | bus-range = <0 255>; | 458 | bus-range = <0 255>; |
| 450 | interrupt-map-mask = <0xf800 0 0 7>; | 459 | interrupt-map-mask = <0xf800 0 0 7>; |
| 451 | interrupt-map = <0 0 0 1 &ipic 2 8 | 460 | interrupt-map = <0 0 0 1 &ipic 2 IRQ_TYPE_LEVEL_LOW |
| 452 | 0 0 0 2 &ipic 2 8 | 461 | 0 0 0 2 &ipic 2 IRQ_TYPE_LEVEL_LOW |
| 453 | 0 0 0 3 &ipic 2 8 | 462 | 0 0 0 3 &ipic 2 IRQ_TYPE_LEVEL_LOW |
| 454 | 0 0 0 4 &ipic 2 8>; | 463 | 0 0 0 4 &ipic 2 IRQ_TYPE_LEVEL_LOW>; |
| 455 | clock-frequency = <0>; | 464 | clock-frequency = <0>; |
| 456 | 465 | ||
| 457 | pcie@0 { | 466 | pcie@0 { |
| @@ -471,12 +480,12 @@ · 13 unmodified lines | |||
| 471 | leds { | 480 | leds { |
| 472 | compatible = "gpio-leds"; | 481 | compatible = "gpio-leds"; |
| 473 | 482 | ||
| 474 | pwr { | 483 | led-pwr { |
| 475 | gpios = <&mcu_pio 0 0>; | 484 | gpios = <&mcu_pio 0 0>; |
| 476 | default-state = "on"; | 485 | default-state = "on"; |
| 477 | }; | 486 | }; |
| 478 | 487 | ||
| 479 | hdd { | 488 | led-hdd { |
| 480 | gpios = <&mcu_pio 1 0>; | 489 | gpios = <&mcu_pio 1 0>; |
| 481 | linux,default-trigger = "disk-activity"; | 490 | linux,default-trigger = "disk-activity"; |
| 482 | }; | 491 | }; |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | |||
| 38 | }; | 38 | }; |
| 39 | }; | 39 | }; |
| 40 | 40 | ||
| 41 | memory { | 41 | memory@0 { |
| 42 | device_type = "memory"; | 42 | device_type = "memory"; |
| 43 | reg = <0x00000000 0x04000000>; | 43 | reg = <0x00000000 0x04000000>; |
| 44 | }; | 44 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | |||
| 39 | }; | 39 | }; |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | memory { | 42 | memory@0 { |
| 43 | device_type = "memory"; | 43 | device_type = "memory"; |
| 44 | reg = <0x00000000 0x10000000>; | 44 | reg = <0x00000000 0x10000000>; |
| 45 | }; | 45 | }; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | }; | 37 | }; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | memory { | 40 | memory@0 { |
| 41 | device_type = "memory"; | 41 | device_type = "memory"; |
| 42 | reg = <0x00000000 0x10000000>; | 42 | reg = <0x00000000 0x10000000>; |
| 43 | }; | 43 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | |||
| 39 | }; | 39 | }; |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | memory { | 42 | memory@0 { |
| 43 | device_type = "memory"; | 43 | device_type = "memory"; |
| 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 | 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 45 | }; | 45 | }; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | |||
| 40 | }; | 40 | }; |
| 41 | }; | 41 | }; |
| 42 | 42 | ||
| 43 | memory { | 43 | memory@0 { |
| 44 | device_type = "memory"; | 44 | device_type = "memory"; |
| 45 | reg = <0x00000000 0x20000000>; // 512MB at 0 | 45 | reg = <0x00000000 0x20000000>; // 512MB at 0 |
| 46 | }; | 46 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | |||
| 39 | }; | 39 | }; |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | memory { | 42 | memory@0 { |
| 43 | device_type = "memory"; | 43 | device_type = "memory"; |
| 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 | 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 45 | }; | 45 | }; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | }; | 37 | }; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | memory { | 40 | memory@0 { |
| 41 | device_type = "memory"; | 41 | device_type = "memory"; |
| 42 | reg = <0x00000000 0x10000000>; // 256MB at 0 | 42 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 43 | }; | 43 | }; |
| @@ -120,10 +120,8 @@ · 119 unmodified lines | |||
| 120 | 120 | ||
| 121 | #if defined(CONFIG_44x) | 121 | #if defined(CONFIG_44x) |
| 122 | #include <asm/nohash/32/pte-44x.h> | 122 | #include <asm/nohash/32/pte-44x.h> |
| 123 | #elif defined(CONFIG_PPC_85xx) && defined(CONFIG_PTE_64BIT) | ||
| 124 | #include <asm/nohash/pte-e500.h> | ||
| 125 | #elif defined(CONFIG_PPC_85xx) | 123 | #elif defined(CONFIG_PPC_85xx) |
| 126 | #include <asm/nohash/32/pte-85xx.h> | 124 | #include <asm/nohash/pte-e500.h> |
| 127 | #elif defined(CONFIG_PPC_8xx) | 125 | #elif defined(CONFIG_PPC_8xx) |
| 128 | #include <asm/nohash/32/pte-8xx.h> | 126 | #include <asm/nohash/32/pte-8xx.h> |
| 129 | #endif | 127 | #endif |
| @@ -1,59 +0,0 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | ||
| 2 | #ifndef _ASM_POWERPC_NOHASH_32_PTE_85xx_H | ||
| 3 | #define _ASM_POWERPC_NOHASH_32_PTE_85xx_H | ||
| 4 | #ifdef __KERNEL__ | ||
| 5 | |||
| 6 | /* PTE bit definitions for Freescale BookE SW loaded TLB MMU based | ||
| 7 | * processors | ||
| 8 | * | ||
| 9 | MMU Assist Register 3: | ||
| 10 | |||
| 11 | 32 33 34 35 36 ... 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ||
| 12 | RPN...................... 0 0 U0 U1 U2 U3 UX SX UW SW UR SR | ||
| 13 | |||
| 14 | - PRESENT *must* be in the bottom two bits because swap PTEs use | ||
| 15 | the top 30 bits. | ||
| 16 | |||
| 17 | */ | ||
| 18 | |||
| 19 | /* Definitions for FSL Book-E Cores */ | ||
| 20 | #define _PAGE_READ 0x00001 /* H: Read permission (SR) */ | ||
| 21 | #define _PAGE_PRESENT 0x00002 /* S: PTE contains a translation */ | ||
| 22 | #define _PAGE_WRITE 0x00004 /* S: Write permission (SW) */ | ||
| 23 | #define _PAGE_DIRTY 0x00008 /* S: Page dirty */ | ||
| 24 | #define _PAGE_EXEC 0x00010 /* H: SX permission */ | ||
| 25 | #define _PAGE_ACCESSED 0x00020 /* S: Page referenced */ | ||
| 26 | |||
| 27 | #define _PAGE_ENDIAN 0x00040 /* H: E bit */ | ||
| 28 | #define _PAGE_GUARDED 0x00080 /* H: G bit */ | ||
| 29 | #define _PAGE_COHERENT 0x00100 /* H: M bit */ | ||
| 30 | #define _PAGE_NO_CACHE 0x00200 /* H: I bit */ | ||
| 31 | #define _PAGE_WRITETHRU 0x00400 /* H: W bit */ | ||
| 32 | #define _PAGE_SPECIAL 0x00800 /* S: Special page */ | ||
| 33 | |||
| 34 | #define _PMD_PRESENT 0 | ||
| 35 | #define _PMD_PRESENT_MASK (PAGE_MASK) | ||
| 36 | #define _PMD_BAD (~PAGE_MASK) | ||
| 37 | #define _PMD_USER 0 | ||
| 38 | |||
| 39 | #define _PTE_NONE_MASK 0 | ||
| 40 | |||
| 41 | #define PTE_WIMGE_SHIFT (6) | ||
| 42 | |||
| 43 | /* | ||
| 44 | * We define 2 sets of base prot bits, one for basic pages (ie, | ||
| 45 | * cacheable kernel and user pages) and one for non cacheable | ||
| 46 | * pages. We always set _PAGE_COHERENT when SMP is enabled or | ||
| 47 | * the processor might need it for DMA coherency. | ||
| 48 | */ | ||
| 49 | #define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED) | ||
| 50 | #if defined(CONFIG_SMP) || defined(CONFIG_PPC_E500MC) | ||
| 51 | #define _PAGE_BASE (_PAGE_BASE_NC | _PAGE_COHERENT) | ||
| 52 | #else | ||
| 53 | #define _PAGE_BASE (_PAGE_BASE_NC) | ||
| 54 | #endif | ||
| 55 | |||
| 56 | #include <asm/pgtable-masks.h> | ||
| 57 | |||
| 58 | #endif /* __KERNEL__ */ | ||
| 59 | #endif /* _ASM_POWERPC_NOHASH_32_PTE_FSL_85xx_H */ | ||
| @@ -49,7 +49,7 @@ static inline unsigned long pud_val(pud_t x) · 48 unmodified lines | |||
| 49 | #endif /* CONFIG_PPC64 */ | 49 | #endif /* CONFIG_PPC64 */ |
| 50 | 50 | ||
| 51 | /* PGD level */ | 51 | /* PGD level */ |
| 52 | #if defined(CONFIG_PPC_85xx) && defined(CONFIG_PTE_64BIT) | 52 | #if defined(CONFIG_PPC_85xx) |
| 53 | typedef struct { unsigned long long pgd; } pgd_t; | 53 | typedef struct { unsigned long long pgd; } pgd_t; |
| 54 | 54 | ||
| 55 | static inline unsigned long long pgd_val(pgd_t x) | 55 | static inline unsigned long long pgd_val(pgd_t x) |
| @@ -15,6 +15,9 @@ · 14 unmodified lines | |||
| 15 | #define TASK_SIZE_MAX TASK_SIZE_USER64 | 15 | #define TASK_SIZE_MAX TASK_SIZE_USER64 |
| 16 | #endif | 16 | #endif |
| 17 | 17 | ||
| 18 | /* Threshold above which VMX copy path is used */ | ||
| 19 | #define VMX_COPY_THRESHOLD 3328 | ||
| 20 | |||
| 18 | #include <asm-generic/access_ok.h> | 21 | #include <asm-generic/access_ok.h> |
| 19 | 22 | ||
| 20 | /* | 23 | /* |
| @@ -255,7 +258,7 @@ __gus_failed: \ · 234 unmodified lines | |||
| 255 | ".section .fixup,\"ax\"\n" \ | 258 | ".section .fixup,\"ax\"\n" \ |
| 256 | "4: li %0,%3\n" \ | 259 | "4: li %0,%3\n" \ |
| 257 | " li %1,0\n" \ | 260 | " li %1,0\n" \ |
| 258 | " li %1+1,0\n" \ | 261 | " li %L1,0\n" \ |
| 259 | " b 3b\n" \ | 262 | " b 3b\n" \ |
| 260 | ".previous\n" \ | 263 | ".previous\n" \ |
| 261 | EX_TABLE(1b, 4b) \ | 264 | EX_TABLE(1b, 4b) \ |
| @@ -326,40 +329,62 @@ do { \ · 64 unmodified lines | |||
| 326 | extern unsigned long __copy_tofrom_user(void __user *to, | 329 | extern unsigned long __copy_tofrom_user(void __user *to, |
| 327 | const void __user *from, unsigned long size); | 330 | const void __user *from, unsigned long size); |
| 328 | 331 | ||
| 329 | #ifdef __powerpc64__ | 332 | unsigned long __copy_tofrom_user_base(void __user *to, |
| 330 | static inline unsigned long | 333 | const void __user *from, unsigned long size); |
| 331 | raw_copy_in_user(void __user *to, const void __user *from, unsigned long n) | 334 | |
| 335 | unsigned long __copy_tofrom_user_power7_vmx(void __user *to, | ||
| 336 | const void __user *from, unsigned long size); | ||
| 337 | |||
| 338 | static __always_inline bool will_use_vmx(unsigned long n) | ||
| 339 | { | ||
| 340 | return IS_ENABLED(CONFIG_ALTIVEC) && cpu_has_feature(CPU_FTR_VMX_COPY) && | ||
| 341 | n > VMX_COPY_THRESHOLD; | ||
| 342 | } | ||
| 343 | |||
| 344 | static __always_inline unsigned long | ||
| 345 | raw_copy_tofrom_user(void __user *to, const void __user *from, | ||
| 346 | unsigned long n, unsigned long dir) | ||
| 332 | { | 347 | { |
| 333 | unsigned long ret; | 348 | unsigned long ret; |
| 334 | 349 | ||
| 335 | barrier_nospec(); | 350 | if (will_use_vmx(n) && enter_vmx_usercopy()) { |
| 336 | allow_user_access(to, KUAP_READ_WRITE); | 351 | allow_user_access(to, dir); |
| 352 | ret = __copy_tofrom_user_power7_vmx(to, from, n); | ||
| 353 | prevent_user_access(dir); | ||
| 354 | exit_vmx_usercopy(); | ||
| 355 | |||
| 356 | if (unlikely(ret)) { | ||
| 357 | allow_user_access(to, dir); | ||
| 358 | ret = __copy_tofrom_user_base(to, from, n); | ||
| 359 | prevent_user_access(dir); | ||
| 360 | } | ||
| 361 | return ret; | ||
| 362 | } | ||
| 363 | |||
| 364 | allow_user_access(to, dir); | ||
| 337 | ret = __copy_tofrom_user(to, from, n); | 365 | ret = __copy_tofrom_user(to, from, n); |
| 338 | prevent_user_access(KUAP_READ_WRITE); | 366 | prevent_user_access(dir); |
| 339 | return ret; | 367 | return ret; |
| 340 | } | 368 | } |
| 341 | #endif /* __powerpc64__ */ | ||
| 342 | 369 | ||
| 343 | static inline unsigned long raw_copy_from_user(void *to, | 370 | #ifdef CONFIG_PPC64 |
| 344 | const void __user *from, unsigned long n) | 371 | static inline unsigned long |
| 372 | raw_copy_in_user(void __user *to, const void __user *from, unsigned long n) | ||
| 345 | { | 373 | { |
| 346 | unsigned long ret; | 374 | barrier_nospec(); |
| 375 | return raw_copy_tofrom_user(to, from, n, KUAP_READ_WRITE); | ||
| 376 | } | ||
| 377 | #endif /* CONFIG_PPC64 */ | ||
| 347 | 378 | ||
| 348 | allow_user_access(NULL, KUAP_READ); | 379 | static inline unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n) |
| 349 | ret = __copy_tofrom_user((__force void __user *)to, from, n); | 380 | { |
| 350 | prevent_user_access(KUAP_READ); | 381 | return raw_copy_tofrom_user((__force void __user *)to, from, n, KUAP_READ); |
| 351 | return ret; | ||
| 352 | } | 382 | } |
| 353 | 383 | ||
| 354 | static inline unsigned long | 384 | static inline unsigned long |
| 355 | raw_copy_to_user(void __user *to, const void *from, unsigned long n) | 385 | raw_copy_to_user(void __user *to, const void *from, unsigned long n) |
| 356 | { | 386 | { |
| 357 | unsigned long ret; | 387 | return raw_copy_tofrom_user(to, (__force const void __user *)from, n, KUAP_WRITE); |
| 358 | |||
| 359 | allow_user_access(to, KUAP_WRITE); | ||
| 360 | ret = __copy_tofrom_user(to, (__force const void __user *)from, n); | ||
| 361 | prevent_user_access(KUAP_WRITE); | ||
| 362 | return ret; | ||
| 363 | } | 388 | } |
| 364 | 389 | ||
| 365 | unsigned long __arch_clear_user(void __user *addr, unsigned long size); | 390 | unsigned long __arch_clear_user(void __user *addr, unsigned long size); |
| @@ -305,7 +305,6 @@ set_ivor: · 304 unmodified lines | |||
| 305 | * r12 is pointer to the pte | 305 | * r12 is pointer to the pte |
| 306 | * r10 is the pshift from the PGD, if we're a hugepage | 306 | * r10 is the pshift from the PGD, if we're a hugepage |
| 307 | */ | 307 | */ |
| 308 | #ifdef CONFIG_PTE_64BIT | ||
| 309 | #ifdef CONFIG_HUGETLB_PAGE | 308 | #ifdef CONFIG_HUGETLB_PAGE |
| 310 | #define FIND_PTE \ | 309 | #define FIND_PTE \ |
| 311 | rlwinm r12, r13, 14, 18, 28; /* Compute pgdir/pmd offset */ \ | 310 | rlwinm r12, r13, 14, 18, 28; /* Compute pgdir/pmd offset */ \ |
| @@ -329,15 +328,6 @@ set_ivor: · 17 unmodified lines | |||
| 329 | rlwimi r12, r13, 23, 20, 28; /* Compute pte address */ \ | 328 | rlwimi r12, r13, 23, 20, 28; /* Compute pte address */ \ |
| 330 | lwz r11, 4(r12); /* Get pte entry */ | 329 | lwz r11, 4(r12); /* Get pte entry */ |
| 331 | #endif /* HUGEPAGE */ | 330 | #endif /* HUGEPAGE */ |
| 332 | #else /* !PTE_64BIT */ | ||
| 333 | #define FIND_PTE \ | ||
| 334 | rlwimi r11, r13, 12, 20, 29; /* Create L1 (pgdir/pmd) address */ \ | ||
| 335 | lwz r11, 0(r11); /* Get L1 entry */ \ | ||
| 336 | rlwinm. r12, r11, 0, 0, 19; /* Extract L2 (pte) base address */ \ | ||
| 337 | beq 2f; /* Bail if no table */ \ | ||
| 338 | rlwimi r12, r13, 22, 20, 29; /* Compute PTE address */ \ | ||
| 339 | lwz r11, 0(r12); /* Get Linux PTE */ | ||
| 340 | #endif | ||
| 341 | 331 | ||
| 342 | /* | 332 | /* |
| 343 | * Interrupt vector entry code | 333 | * Interrupt vector entry code |
| @@ -473,21 +463,15 @@ END_BTB_FLUSH_SECTION · 129 unmodified lines | |||
| 473 | 4: | 463 | 4: |
| 474 | FIND_PTE | 464 | FIND_PTE |
| 475 | 465 | ||
| 476 | #ifdef CONFIG_PTE_64BIT | ||
| 477 | li r13,_PAGE_PRESENT|_PAGE_BAP_SR | 466 | li r13,_PAGE_PRESENT|_PAGE_BAP_SR |
| 478 | oris r13,r13,_PAGE_ACCESSED@h | 467 | oris r13,r13,_PAGE_ACCESSED@h |
| 479 | #else | ||
| 480 | li r13,_PAGE_PRESENT|_PAGE_READ|_PAGE_ACCESSED | ||
| 481 | #endif | ||
| 482 | andc. r13,r13,r11 /* Check permission */ | 468 | andc. r13,r13,r11 /* Check permission */ |
| 483 | 469 | ||
| 484 | #ifdef CONFIG_PTE_64BIT | ||
| 485 | #ifdef CONFIG_SMP | 470 | #ifdef CONFIG_SMP |
| 486 | subf r13,r11,r12 /* create false data dep */ | 471 | subf r13,r11,r12 /* create false data dep */ |
| 487 | lwzx r13,r11,r13 /* Get upper pte bits */ | 472 | lwzx r13,r11,r13 /* Get upper pte bits */ |
| 488 | #else | 473 | #else |
| 489 | lwz r13,0(r12) /* Get upper pte bits */ | 474 | lwz r13,0(r12) /* Get upper pte bits */ |
| 490 | #endif | ||
| 491 | #endif | 475 | #endif |
| 492 | 476 | ||
| 493 | bne 2f /* Bail if permission/valid mismatch */ | 477 | bne 2f /* Bail if permission/valid mismatch */ |
| @@ -552,12 +536,8 @@ END_BTB_FLUSH_SECTION · 58 unmodified lines | |||
| 552 | 536 | ||
| 553 | FIND_PTE | 537 | FIND_PTE |
| 554 | /* Make up the required permissions for kernel code */ | 538 | /* Make up the required permissions for kernel code */ |
| 555 | #ifdef CONFIG_PTE_64BIT | ||
| 556 | li r13,_PAGE_PRESENT | _PAGE_BAP_SX | 539 | li r13,_PAGE_PRESENT | _PAGE_BAP_SX |
| 557 | oris r13,r13,_PAGE_ACCESSED@h | 540 | oris r13,r13,_PAGE_ACCESSED@h |
| 558 | #else | ||
| 559 | li r13,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | ||
| 560 | #endif | ||
| 561 | b 4f | 541 | b 4f |
| 562 | 542 | ||
| 563 | /* Get the PGD for the current thread */ | 543 | /* Get the PGD for the current thread */ |
| @@ -573,23 +553,17 @@ END_BTB_FLUSH_SECTION · 9 unmodified lines | |||
| 573 | 553 | ||
| 574 | FIND_PTE | 554 | FIND_PTE |
| 575 | /* Make up the required permissions for user code */ | 555 | /* Make up the required permissions for user code */ |
| 576 | #ifdef CONFIG_PTE_64BIT | ||
| 577 | li r13,_PAGE_PRESENT | _PAGE_BAP_UX | 556 | li r13,_PAGE_PRESENT | _PAGE_BAP_UX |
| 578 | oris r13,r13,_PAGE_ACCESSED@h | 557 | oris r13,r13,_PAGE_ACCESSED@h |
| 579 | #else | ||
| 580 | li r13,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | ||
| 581 | #endif | ||
| 582 | 558 | ||
| 583 | 4: | 559 | 4: |
| 584 | andc. r13,r13,r11 /* Check permission */ | 560 | andc. r13,r13,r11 /* Check permission */ |
| 585 | 561 | ||
| 586 | #ifdef CONFIG_PTE_64BIT | ||
| 587 | #ifdef CONFIG_SMP | 562 | #ifdef CONFIG_SMP |
| 588 | subf r13,r11,r12 /* create false data dep */ | 563 | subf r13,r11,r12 /* create false data dep */ |
| 589 | lwzx r13,r11,r13 /* Get upper pte bits */ | 564 | lwzx r13,r11,r13 /* Get upper pte bits */ |
| 590 | #else | 565 | #else |
| 591 | lwz r13,0(r12) /* Get upper pte bits */ | 566 | lwz r13,0(r12) /* Get upper pte bits */ |
| 592 | #endif | ||
| 593 | #endif | 567 | #endif |
| 594 | 568 | ||
| 595 | bne 2f /* Bail if permission mismatch */ | 569 | bne 2f /* Bail if permission mismatch */ |
| @@ -683,7 +657,7 @@ interrupt_end: · 87 unmodified lines | |||
| 683 | * r10 - tsize encoding (if HUGETLB_PAGE) or available to use | 657 | * r10 - tsize encoding (if HUGETLB_PAGE) or available to use |
| 684 | * r11 - TLB (info from Linux PTE) | 658 | * r11 - TLB (info from Linux PTE) |
| 685 | * r12 - available to use | 659 | * r12 - available to use |
| 686 | * r13 - upper bits of PTE (if PTE_64BIT) or available to use | 660 | * r13 - upper bits of PTE |
| 687 | * CR5 - results of addr >= PAGE_OFFSET | 661 | * CR5 - results of addr >= PAGE_OFFSET |
| 688 | * MAS0, MAS1 - loaded with proper value when we get here | 662 | * MAS0, MAS1 - loaded with proper value when we get here |
| 689 | * MAS2, MAS3 - will need additional info from Linux PTE | 663 | * MAS2, MAS3 - will need additional info from Linux PTE |
| @@ -751,7 +725,6 @@ finish_tlb_load: · 61 unmodified lines | |||
| 751 | * here we (properly should) assume have the appropriate value. | 725 | * here we (properly should) assume have the appropriate value. |
| 752 | */ | 726 | */ |
| 753 | finish_tlb_load_cont: | 727 | finish_tlb_load_cont: |
| 754 | #ifdef CONFIG_PTE_64BIT | ||
| 755 | rlwinm r12, r11, 32-2, 26, 31 /* Move in perm bits */ | 728 | rlwinm r12, r11, 32-2, 26, 31 /* Move in perm bits */ |
| 756 | andi. r10, r11, _PAGE_DIRTY | 729 | andi. r10, r11, _PAGE_DIRTY |
| 757 | bne 1f | 730 | bne 1f |
| @@ -764,26 +737,9 @@ BEGIN_MMU_FTR_SECTION · 6 unmodified lines | |||
| 764 | srwi r10, r13, 12 /* grab RPN[12:31] */ | 737 | srwi r10, r13, 12 /* grab RPN[12:31] */ |
| 765 | mtspr SPRN_MAS7, r10 | 738 | mtspr SPRN_MAS7, r10 |
| 766 | END_MMU_FTR_SECTION_IFSET(MMU_FTR_BIG_PHYS) | 739 | END_MMU_FTR_SECTION_IFSET(MMU_FTR_BIG_PHYS) |
| 767 | #else | ||
| 768 | li r10, (_PAGE_EXEC | _PAGE_READ) | ||
| 769 | mr r13, r11 | ||
| 770 | rlwimi r10, r11, 31, 29, 29 /* extract _PAGE_DIRTY into SW */ | ||
| 771 | and r12, r11, r10 | ||
| 772 | mcrf cr0, cr5 /* Test for user page */ | ||
| 773 | slwi r10, r12, 1 | ||
| 774 | or r10, r10, r12 | ||
| 775 | rlwinm r10, r10, 0, ~_PAGE_EXEC /* Clear SX on user pages */ | ||
| 776 | isellt r12, r10, r12 | ||
| 777 | rlwimi r13, r12, 0, 20, 31 /* Get RPN from PTE, merge w/ perms */ | ||
| 778 | mtspr SPRN_MAS3, r13 | ||
| 779 | #endif | ||
| 780 | 740 | ||
| 781 | mfspr r12, SPRN_MAS2 | 741 | mfspr r12, SPRN_MAS2 |
| 782 | #ifdef CONFIG_PTE_64BIT | ||
| 783 | rlwimi r12, r11, 32-19, 27, 31 /* extract WIMGE from pte */ | 742 | rlwimi r12, r11, 32-19, 27, 31 /* extract WIMGE from pte */ |
| 784 | #else | ||
| 785 | rlwimi r12, r11, 26, 27, 31 /* extract WIMGE from pte */ | ||
| 786 | #endif | ||
| 787 | #ifdef CONFIG_HUGETLB_PAGE | 743 | #ifdef CONFIG_HUGETLB_PAGE |
| 788 | beq 6, 3f /* don't mask if page isn't huge */ | 744 | beq 6, 3f /* don't mask if page isn't huge */ |
| 789 | li r13, 1 | 745 | li r13, 1 |
| @@ -1159,7 +1159,7 @@ spapr_tce_platform_iommu_attach_dev(struct iommu_domain *platform_domain, · 1158 unmodified lines | |||
| 1159 | struct device *dev, | 1159 | struct device *dev, |
| 1160 | struct iommu_domain *old) | 1160 | struct iommu_domain *old) |
| 1161 | { | 1161 | { |
| 1162 | struct iommu_domain *domain = iommu_get_domain_for_dev(dev); | 1162 | struct iommu_domain *domain = iommu_driver_get_domain_for_dev(dev); |
| 1163 | struct iommu_table_group *table_group; | 1163 | struct iommu_table_group *table_group; |
| 1164 | struct iommu_group *grp; | 1164 | struct iommu_group *grp; |
| 1165 | 1165 | ||
| @@ -212,6 +212,13 @@ struct pci_dev *of_create_pci_dev(struct device_node *node, · 211 unmodified lines | |||
| 212 | dev->error_state = pci_channel_io_normal; | 212 | dev->error_state = pci_channel_io_normal; |
| 213 | dev->dma_mask = 0xffffffff; | 213 | dev->dma_mask = 0xffffffff; |
| 214 | 214 | ||
| 215 | /* | ||
| 216 | * Assume 64-bit addresses for MSI initially. Will be changed to 32-bit | ||
| 217 | * if MSI (rather than MSI-X) capability does not have | ||
| 218 | * PCI_MSI_FLAGS_64BIT. Can also be overridden by driver. | ||
| 219 | */ | ||
| 220 | dev->msi_addr_mask = DMA_BIT_MASK(64); | ||
| 221 | |||
| 215 | /* Early fixups, before probing the BARs */ | 222 | /* Early fixups, before probing the BARs */ |
| 216 | pci_fixup_device(pci_fixup_early, dev); | 223 | pci_fixup_device(pci_fixup_early, dev); |
| 217 | 224 | ||
| @@ -2893,7 +2893,8 @@ static void __init fixup_device_tree_pmac(void) · 2892 unmodified lines | |||
| 2893 | for (node = 0; prom_next_node(&node); ) { | 2893 | for (node = 0; prom_next_node(&node); ) { |
| 2894 | type[0] = '\0'; | 2894 | type[0] = '\0'; |
| 2895 | prom_getprop(node, "device_type", type, sizeof(type)); | 2895 | prom_getprop(node, "device_type", type, sizeof(type)); |
| 2896 | if (prom_strcmp(type, "escc") && prom_strcmp(type, "i2s")) | 2896 | if (prom_strcmp(type, "escc") && prom_strcmp(type, "i2s") && |
| 2897 | prom_strcmp(type, "media-bay")) | ||
| 2897 | continue; | 2898 | continue; |
| 2898 | 2899 | ||
| 2899 | if (prom_getproplen(node, "#size-cells") != PROM_ERROR) | 2900 | if (prom_getproplen(node, "#size-cells") != PROM_ERROR) |
| @@ -35,7 +35,6 @@ · 34 unmodified lines | |||
| 35 | #include <linux/of_irq.h> | 35 | #include <linux/of_irq.h> |
| 36 | #include <linux/hugetlb.h> | 36 | #include <linux/hugetlb.h> |
| 37 | #include <linux/pgtable.h> | 37 | #include <linux/pgtable.h> |
| 38 | #include <asm/kexec.h> | ||
| 39 | #include <asm/io.h> | 38 | #include <asm/io.h> |
| 40 | #include <asm/paca.h> | 39 | #include <asm/paca.h> |
| 41 | #include <asm/processor.h> | 40 | #include <asm/processor.h> |
| @@ -995,15 +994,6 @@ void __init setup_arch(char **cmdline_p) · 953 unmodified lines | |||
| 995 | 994 | ||
| 996 | initmem_init(); | 995 | initmem_init(); |
| 997 | 996 | ||
| 998 | /* | ||
| 999 | * Reserve large chunks of memory for use by CMA for kdump, fadump, KVM and | ||
| 1000 | * hugetlb. These must be called after initmem_init(), so that | ||
| 1001 | * pageblock_order is initialised. | ||
| 1002 | */ | ||
| 1003 | fadump_cma_init(); | ||
| 1004 | kdump_cma_reserve(); | ||
| 1005 | kvm_cma_reserve(); | ||
| 1006 | |||
| 1007 | early_memtest(min_low_pfn << PAGE_SHIFT, max_low_pfn << PAGE_SHIFT); | 997 | early_memtest(min_low_pfn << PAGE_SHIFT, max_low_pfn << PAGE_SHIFT); |
| 1008 | 998 | ||
| 1009 | if (ppc_md.setup_arch) | 999 | if (ppc_md.setup_arch) |
| @@ -37,11 +37,29 @@ unsigned long ftrace_call_adjust(unsigned long addr) · 36 unmodified lines | |||
| 37 | if (addr >= (unsigned long)__exittext_begin && addr < (unsigned long)__exittext_end) | 37 | if (addr >= (unsigned long)__exittext_begin && addr < (unsigned long)__exittext_end) |
| 38 | return 0; | 38 | return 0; |
| 39 | 39 | ||
| 40 | if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY) && | 40 | if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) { |
| 41 | !IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | 41 | if (!IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 42 | addr += MCOUNT_INSN_SIZE; | ||
| 43 | if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) | ||
| 44 | addr += MCOUNT_INSN_SIZE; | 42 | addr += MCOUNT_INSN_SIZE; |
| 43 | if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) | ||
| 44 | addr += MCOUNT_INSN_SIZE; | ||
| 45 | } else if (IS_ENABLED(CONFIG_CC_IS_CLANG) && IS_ENABLED(CONFIG_PPC64)) { | ||
| 46 | /* | ||
| 47 | * addr points to global entry point though the NOP was emitted at local | ||
| 48 | * entry point due to https://github.com/llvm/llvm-project/issues/163706 | ||
| 49 | * Handle that here with ppc_function_entry() for kernel symbols while | ||
| 50 | * adjusting module addresses in the else case, by looking for the below | ||
| 51 | * module global entry point sequence: | ||
| 52 | * ld r2, -8(r12) | ||
| 53 | * add r2, r2, r12 | ||
| 54 | */ | ||
| 55 | if (is_kernel_text(addr) || is_kernel_inittext(addr)) | ||
| 56 | addr = ppc_function_entry((void *)addr); | ||
| 57 | else if ((ppc_inst_val(ppc_inst_read((u32 *)addr)) == | ||
| 58 | PPC_RAW_LD(_R2, _R12, -8)) && | ||
| 59 | (ppc_inst_val(ppc_inst_read((u32 *)(addr+4))) == | ||
| 60 | PPC_RAW_ADD(_R2, _R2, _R12))) | ||
| 61 | addr += 8; | ||
| 62 | } | ||
| 45 | } | 63 | } |
| 46 | 64 | ||
| 47 | return addr; | 65 | return addr; |
| @@ -397,6 +397,7 @@ SECTIONS · 396 unmodified lines | |||
| 397 | _end = . ; | 397 | _end = . ; |
| 398 | 398 | ||
| 399 | DWARF_DEBUG | 399 | DWARF_DEBUG |
| 400 | MODINFO | ||
| 400 | ELF_DETAILS | 401 | ELF_DETAILS |
| 401 | 402 | ||
| 402 | DISCARDS | 403 | DISCARDS |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | |||
| 23 | #include <asm/firmware.h> | 23 | #include <asm/firmware.h> |
| 24 | 24 | ||
| 25 | #define cpu_to_be_ulong __PASTE(cpu_to_be, BITS_PER_LONG) | 25 | #define cpu_to_be_ulong __PASTE(cpu_to_be, BITS_PER_LONG) |
| 26 | #define __be_word __PASTE(__be, BITS_PER_LONG) | ||
| 26 | 27 | ||
| 27 | #ifdef CONFIG_CRASH_DUMP | 28 | #ifdef CONFIG_CRASH_DUMP |
| 28 | void machine_crash_shutdown(struct pt_regs *regs) | 29 | void machine_crash_shutdown(struct pt_regs *regs) |
| @@ -146,25 +147,25 @@ int __init overlaps_crashkernel(unsigned long start, unsigned long size) · 117 unmodified lines | |||
| 146 | } | 147 | } |
| 147 | 148 | ||
| 148 | /* Values we need to export to the second kernel via the device tree. */ | 149 | /* Values we need to export to the second kernel via the device tree. */ |
| 149 | static phys_addr_t crashk_base; | 150 | static __be_word crashk_base; |
| 150 | static phys_addr_t crashk_size; | 151 | static __be_word crashk_size; |
| 151 | static unsigned long long mem_limit; | 152 | static __be_word mem_limit; |
| 152 | 153 | ||
| 153 | static struct property crashk_base_prop = { | 154 | static struct property crashk_base_prop = { |
| 154 | .name = "linux,crashkernel-base", | 155 | .name = "linux,crashkernel-base", |
| 155 | .length = sizeof(phys_addr_t), | 156 | .length = sizeof(__be_word), |
| 156 | .value = &crashk_base | 157 | .value = &crashk_base |
| 157 | }; | 158 | }; |
| 158 | 159 | ||
| 159 | static struct property crashk_size_prop = { | 160 | static struct property crashk_size_prop = { |
| 160 | .name = "linux,crashkernel-size", | 161 | .name = "linux,crashkernel-size", |
| 161 | .length = sizeof(phys_addr_t), | 162 | .length = sizeof(__be_word), |
| 162 | .value = &crashk_size, | 163 | .value = &crashk_size, |
| 163 | }; | 164 | }; |
| 164 | 165 | ||
| 165 | static struct property memory_limit_prop = { | 166 | static struct property memory_limit_prop = { |
| 166 | .name = "linux,memory-limit", | 167 | .name = "linux,memory-limit", |
| 167 | .length = sizeof(unsigned long long), | 168 | .length = sizeof(__be_word), |
| 168 | .value = &mem_limit, | 169 | .value = &mem_limit, |
| 169 | }; | 170 | }; |
| 170 | 171 | ||
| @@ -193,11 +194,11 @@ static void __init export_crashk_values(struct device_node *node) · 22 unmodified lines | |||
| 193 | } | 194 | } |
| 194 | #endif /* CONFIG_CRASH_RESERVE */ | 195 | #endif /* CONFIG_CRASH_RESERVE */ |
| 195 | 196 | ||
| 196 | static phys_addr_t kernel_end; | 197 | static __be_word kernel_end; |
| 197 | 198 | ||
| 198 | static struct property kernel_end_prop = { | 199 | static struct property kernel_end_prop = { |
| 199 | .name = "linux,kernel-end", | 200 | .name = "linux,kernel-end", |
| 200 | .length = sizeof(phys_addr_t), | 201 | .length = sizeof(__be_word), |
| 201 | .value = &kernel_end, | 202 | .value = &kernel_end, |
| 202 | }; | 203 | }; |
| 203 | 204 | ||
| @@ -450,6 +450,11 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf) · 449 unmodified lines | |||
| 450 | kbuf->buffer = headers; | 450 | kbuf->buffer = headers; |
| 451 | kbuf->mem = KEXEC_BUF_MEM_UNKNOWN; | 451 | kbuf->mem = KEXEC_BUF_MEM_UNKNOWN; |
| 452 | kbuf->bufsz = headers_sz; | 452 | kbuf->bufsz = headers_sz; |
| 453 | |||
| 454 | /* | ||
| 455 | * Account for extra space required to accommodate additional memory | ||
| 456 | * ranges in elfcorehdr due to memory hotplug events. | ||
| 457 | */ | ||
| 453 | kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem); | 458 | kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem); |
| 454 | kbuf->top_down = false; | 459 | kbuf->top_down = false; |
| 455 | 460 | ||
| @@ -460,7 +465,14 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf) · 4 unmodified lines | |||
| 460 | } | 465 | } |
| 461 | 466 | ||
| 462 | image->elf_load_addr = kbuf->mem; | 467 | image->elf_load_addr = kbuf->mem; |
| 463 | image->elf_headers_sz = headers_sz; | 468 | |
| 469 | /* | ||
| 470 | * If CONFIG_CRASH_HOTPLUG is enabled, the elfcorehdr kexec segment | ||
| 471 | * memsz can be larger than bufsz. Always initialize elf_headers_sz | ||
| 472 | * with memsz. This ensures the correct size is reserved for elfcorehdr | ||
| 473 | * memory in the FDT prepared for kdump. | ||
| 474 | */ | ||
| 475 | image->elf_headers_sz = kbuf->memsz; | ||
| 464 | image->elf_headers = headers; | 476 | image->elf_headers = headers; |
| 465 | out: | 477 | out: |
| 466 | kfree(cmem); | 478 | kfree(cmem); |
| @@ -38,7 +38,6 @@ config KVM_BOOK3S_64_HANDLER · 37 unmodified lines | |||
| 38 | config KVM_BOOK3S_PR_POSSIBLE | 38 | config KVM_BOOK3S_PR_POSSIBLE |
| 39 | bool | 39 | bool |
| 40 | select KVM_MMIO | 40 | select KVM_MMIO |
| 41 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 42 | 41 | ||
| 43 | config KVM_BOOK3S_HV_POSSIBLE | 42 | config KVM_BOOK3S_HV_POSSIBLE |
| 44 | bool | 43 | bool |
| @@ -81,7 +80,6 @@ config KVM_BOOK3S_64_HV · 36 unmodified lines | |||
| 81 | tristate "KVM for POWER7 and later using hypervisor mode in host" | 80 | tristate "KVM for POWER7 and later using hypervisor mode in host" |
| 82 | depends on KVM_BOOK3S_64 && PPC_POWERNV | 81 | depends on KVM_BOOK3S_64 && PPC_POWERNV |
| 83 | select KVM_BOOK3S_HV_POSSIBLE | 82 | select KVM_BOOK3S_HV_POSSIBLE |
| 84 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 85 | select KVM_BOOK3S_HV_PMU | 83 | select KVM_BOOK3S_HV_PMU |
| 86 | select CMA | 84 | select CMA |
| 87 | help | 85 | help |
| @@ -203,7 +201,6 @@ config KVM_E500V2 · 115 unmodified lines | |||
| 203 | depends on !CONTEXT_TRACKING_USER | 201 | depends on !CONTEXT_TRACKING_USER |
| 204 | select KVM | 202 | select KVM |
| 205 | select KVM_MMIO | 203 | select KVM_MMIO |
| 206 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 207 | help | 204 | help |
| 208 | Support running unmodified E500 guest kernels in virtual machines on | 205 | Support running unmodified E500 guest kernels in virtual machines on |
| 209 | E500v2 host processors. | 206 | E500v2 host processors. |
| @@ -220,7 +217,6 @@ config KVM_E500MC · 10 unmodified lines | |||
| 220 | select KVM | 217 | select KVM |
| 221 | select KVM_MMIO | 218 | select KVM_MMIO |
| 222 | select KVM_BOOKE_HV | 219 | select KVM_BOOKE_HV |
| 223 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 224 | help | 220 | help |
| 225 | Support running unmodified E500MC/E5500/E6500 guest kernels in | 221 | Support running unmodified E500MC/E5500/E6500 guest kernels in |
| 226 | virtual machines on E500MC/E5500/E6500 host processors. | 222 | virtual machines on E500MC/E5500/E6500 host processors. |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | |||
| 38 | 38 | ||
| 39 | /* #define EXIT_DEBUG */ | 39 | /* #define EXIT_DEBUG */ |
| 40 | 40 | ||
| 41 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 41 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 42 | KVM_GENERIC_VM_STATS(), | 42 | KVM_GENERIC_VM_STATS(), |
| 43 | STATS_DESC_ICOUNTER(VM, num_2M_pages), | 43 | STATS_DESC_ICOUNTER(VM, num_2M_pages), |
| 44 | STATS_DESC_ICOUNTER(VM, num_1G_pages) | 44 | STATS_DESC_ICOUNTER(VM, num_1G_pages) |
| @@ -53,7 +53,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 8 unmodified lines | |||
| 53 | sizeof(kvm_vm_stats_desc), | 53 | sizeof(kvm_vm_stats_desc), |
| 54 | }; | 54 | }; |
| 55 | 55 | ||
| 56 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 56 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 57 | KVM_GENERIC_VCPU_STATS(), | 57 | KVM_GENERIC_VCPU_STATS(), |
| 58 | STATS_DESC_COUNTER(VCPU, sum_exits), | 58 | STATS_DESC_COUNTER(VCPU, sum_exits), |
| 59 | STATS_DESC_COUNTER(VCPU, mmio_exits), | 59 | STATS_DESC_COUNTER(VCPU, mmio_exits), |
| @@ -36,7 +36,7 @@ · 35 unmodified lines | |||
| 36 | 36 | ||
| 37 | unsigned long kvmppc_booke_handlers; | 37 | unsigned long kvmppc_booke_handlers; |
| 38 | 38 | ||
| 39 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 39 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 40 | KVM_GENERIC_VM_STATS(), | 40 | KVM_GENERIC_VM_STATS(), |
| 41 | STATS_DESC_ICOUNTER(VM, num_2M_pages), | 41 | STATS_DESC_ICOUNTER(VM, num_2M_pages), |
| 42 | STATS_DESC_ICOUNTER(VM, num_1G_pages) | 42 | STATS_DESC_ICOUNTER(VM, num_1G_pages) |
| @@ -51,7 +51,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 8 unmodified lines | |||
| 51 | sizeof(kvm_vm_stats_desc), | 51 | sizeof(kvm_vm_stats_desc), |
| 52 | }; | 52 | }; |
| 53 | 53 | ||
| 54 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 54 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 55 | KVM_GENERIC_VCPU_STATS(), | 55 | KVM_GENERIC_VCPU_STATS(), |
| 56 | STATS_DESC_COUNTER(VCPU, sum_exits), | 56 | STATS_DESC_COUNTER(VCPU, sum_exits), |
| 57 | STATS_DESC_COUNTER(VCPU, mmio_exits), | 57 | STATS_DESC_COUNTER(VCPU, mmio_exits), |
| @@ -39,15 +39,11 @@ enum vcpu_ftr { · 38 unmodified lines | |||
| 39 | /* bits [6-5] MAS2_X1 and MAS2_X0 and [4-0] bits for WIMGE */ | 39 | /* bits [6-5] MAS2_X1 and MAS2_X0 and [4-0] bits for WIMGE */ |
| 40 | #define E500_TLB_MAS2_ATTR (0x7f) | 40 | #define E500_TLB_MAS2_ATTR (0x7f) |
| 41 | 41 | ||
| 42 | struct tlbe_ref { | 42 | struct tlbe_priv { |
| 43 | kvm_pfn_t pfn; /* valid only for TLB0, except briefly */ | 43 | kvm_pfn_t pfn; /* valid only for TLB0, except briefly */ |
| 44 | unsigned int flags; /* E500_TLB_* */ | 44 | unsigned int flags; /* E500_TLB_* */ |
| 45 | }; | 45 | }; |
| 46 | 46 | ||
| 47 | struct tlbe_priv { | ||
| 48 | struct tlbe_ref ref; | ||
| 49 | }; | ||
| 50 | |||
| 51 | #ifdef CONFIG_KVM_E500V2 | 47 | #ifdef CONFIG_KVM_E500V2 |
| 52 | struct vcpu_id_table; | 48 | struct vcpu_id_table; |
| 53 | #endif | 49 | #endif |
| @@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500 *vcpu_e500) · 919 unmodified lines | |||
| 920 | vcpu_e500->gtlb_offset[0] = 0; | 920 | vcpu_e500->gtlb_offset[0] = 0; |
| 921 | vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE; | 921 | vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE; |
| 922 | 922 | ||
| 923 | vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref, | 923 | vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv, |
| 924 | vcpu_e500->gtlb_params[0].entries); | 924 | vcpu_e500->gtlb_params[0].entries); |
| 925 | if (!vcpu_e500->gtlb_priv[0]) | 925 | if (!vcpu_e500->gtlb_priv[0]) |
| 926 | goto free_vcpu; | 926 | goto free_vcpu; |
| 927 | 927 | ||
| 928 | vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref, | 928 | vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv, |
| 929 | vcpu_e500->gtlb_params[1].entries); | 929 | vcpu_e500->gtlb_params[1].entries); |
| 930 | if (!vcpu_e500->gtlb_priv[1]) | 930 | if (!vcpu_e500->gtlb_priv[1]) |
| 931 | goto free_vcpu; | 931 | goto free_vcpu; |
| @@ -189,16 +189,16 @@ void inval_gtlbe_on_host(struct kvmppc_vcpu_e500 *vcpu_e500, int tlbsel, · 188 unmodified lines | |||
| 189 | { | 189 | { |
| 190 | struct kvm_book3e_206_tlb_entry *gtlbe = | 190 | struct kvm_book3e_206_tlb_entry *gtlbe = |
| 191 | get_entry(vcpu_e500, tlbsel, esel); | 191 | get_entry(vcpu_e500, tlbsel, esel); |
| 192 | struct tlbe_ref *ref = &vcpu_e500->gtlb_priv[tlbsel][esel].ref; | 192 | struct tlbe_priv *tlbe = &vcpu_e500->gtlb_priv[tlbsel][esel]; |
| 193 | 193 | ||
| 194 | /* Don't bother with unmapped entries */ | 194 | /* Don't bother with unmapped entries */ |
| 195 | if (!(ref->flags & E500_TLB_VALID)) { | 195 | if (!(tlbe->flags & E500_TLB_VALID)) { |
| 196 | WARN(ref->flags & (E500_TLB_BITMAP | E500_TLB_TLB0), | 196 | WARN(tlbe->flags & (E500_TLB_BITMAP | E500_TLB_TLB0), |
| 197 | "%s: flags %x\n", __func__, ref->flags); | 197 | "%s: flags %x\n", __func__, tlbe->flags); |
| 198 | WARN_ON(tlbsel == 1 && vcpu_e500->g2h_tlb1_map[esel]); | 198 | WARN_ON(tlbsel == 1 && vcpu_e500->g2h_tlb1_map[esel]); |
| 199 | } | 199 | } |
| 200 | 200 | ||
| 201 | if (tlbsel == 1 && ref->flags & E500_TLB_BITMAP) { | 201 | if (tlbsel == 1 && tlbe->flags & E500_TLB_BITMAP) { |
| 202 | u64 tmp = vcpu_e500->g2h_tlb1_map[esel]; | 202 | u64 tmp = vcpu_e500->g2h_tlb1_map[esel]; |
| 203 | int hw_tlb_indx; | 203 | int hw_tlb_indx; |
| 204 | unsigned long flags; | 204 | unsigned long flags; |
| @@ -216,28 +216,28 @@ void inval_gtlbe_on_host(struct kvmppc_vcpu_e500 *vcpu_e500, int tlbsel, · 11 unmodified lines | |||
| 216 | } | 216 | } |
| 217 | mb(); | 217 | mb(); |
| 218 | vcpu_e500->g2h_tlb1_map[esel] = 0; | 218 | vcpu_e500->g2h_tlb1_map[esel] = 0; |
| 219 | ref->flags &= ~(E500_TLB_BITMAP | E500_TLB_VALID); | 219 | tlbe->flags &= ~(E500_TLB_BITMAP | E500_TLB_VALID); |
| 220 | local_irq_restore(flags); | 220 | local_irq_restore(flags); |
| 221 | } | 221 | } |
| 222 | 222 | ||
| 223 | if (tlbsel == 1 && ref->flags & E500_TLB_TLB0) { | 223 | if (tlbsel == 1 && tlbe->flags & E500_TLB_TLB0) { |
| 224 | /* | 224 | /* |
| 225 | * TLB1 entry is backed by 4k pages. This should happen | 225 | * TLB1 entry is backed by 4k pages. This should happen |
| 226 | * rarely and is not worth optimizing. Invalidate everything. | 226 | * rarely and is not worth optimizing. Invalidate everything. |
| 227 | */ | 227 | */ |
| 228 | kvmppc_e500_tlbil_all(vcpu_e500); | 228 | kvmppc_e500_tlbil_all(vcpu_e500); |
| 229 | ref->flags &= ~(E500_TLB_TLB0 | E500_TLB_VALID); | 229 | tlbe->flags &= ~(E500_TLB_TLB0 | E500_TLB_VALID); |
| 230 | } | 230 | } |
| 231 | 231 | ||
| 232 | /* | 232 | /* |
| 233 | * If TLB entry is still valid then it's a TLB0 entry, and thus | 233 | * If TLB entry is still valid then it's a TLB0 entry, and thus |
| 234 | * backed by at most one host tlbe per shadow pid | 234 | * backed by at most one host tlbe per shadow pid |
| 235 | */ | 235 | */ |
| 236 | if (ref->flags & E500_TLB_VALID) | 236 | if (tlbe->flags & E500_TLB_VALID) |
| 237 | kvmppc_e500_tlbil_one(vcpu_e500, gtlbe); | 237 | kvmppc_e500_tlbil_one(vcpu_e500, gtlbe); |
| 238 | 238 | ||
| 239 | /* Mark the TLB as not backed by the host anymore */ | 239 | /* Mark the TLB as not backed by the host anymore */ |
| 240 | ref->flags = 0; | 240 | tlbe->flags = 0; |
| 241 | } | 241 | } |
| 242 | 242 | ||
| 243 | static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) | 243 | static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) |
| @@ -245,26 +245,26 @@ static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) · 1 unmodified line | |||
| 245 | return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); | 245 | return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); |
| 246 | } | 246 | } |
| 247 | 247 | ||
| 248 | static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, | 248 | static inline void kvmppc_e500_tlbe_setup(struct tlbe_priv *tlbe, |
| 249 | struct kvm_book3e_206_tlb_entry *gtlbe, | 249 | struct kvm_book3e_206_tlb_entry *gtlbe, |
| 250 | kvm_pfn_t pfn, unsigned int wimg, | 250 | kvm_pfn_t pfn, unsigned int wimg, |
| 251 | bool writable) | 251 | bool writable) |
| 252 | { | 252 | { |
| 253 | ref->pfn = pfn; | 253 | tlbe->pfn = pfn; |
| 254 | ref->flags = E500_TLB_VALID; | 254 | tlbe->flags = E500_TLB_VALID; |
| 255 | if (writable) | 255 | if (writable) |
| 256 | ref->flags |= E500_TLB_WRITABLE; | 256 | tlbe->flags |= E500_TLB_WRITABLE; |
| 257 | 257 | ||
| 258 | /* Use guest supplied MAS2_G and MAS2_E */ | 258 | /* Use guest supplied MAS2_G and MAS2_E */ |
| 259 | ref->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; | 259 | tlbe->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; |
| 260 | } | 260 | } |
| 261 | 261 | ||
| 262 | static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) | 262 | static inline void kvmppc_e500_tlbe_release(struct tlbe_priv *tlbe) |
| 263 | { | 263 | { |
| 264 | if (ref->flags & E500_TLB_VALID) { | 264 | if (tlbe->flags & E500_TLB_VALID) { |
| 265 | /* FIXME: don't log bogus pfn for TLB1 */ | 265 | /* FIXME: don't log bogus pfn for TLB1 */ |
| 266 | trace_kvm_booke206_ref_release(ref->pfn, ref->flags); | 266 | trace_kvm_booke206_ref_release(tlbe->pfn, tlbe->flags); |
| 267 | ref->flags = 0; | 267 | tlbe->flags = 0; |
| 268 | } | 268 | } |
| 269 | } | 269 | } |
| 270 | 270 | ||
| @@ -284,11 +284,8 @@ static void clear_tlb_privs(struct kvmppc_vcpu_e500 *vcpu_e500) · 13 unmodified lines | |||
| 284 | int i; | 284 | int i; |
| 285 | 285 | ||
| 286 | for (tlbsel = 0; tlbsel <= 1; tlbsel++) { | 286 | for (tlbsel = 0; tlbsel <= 1; tlbsel++) { |
| 287 | for (i = 0; i < vcpu_e500->gtlb_params[tlbsel].entries; i++) { | 287 | for (i = 0; i < vcpu_e500->gtlb_params[tlbsel].entries; i++) |
| 288 | struct tlbe_ref *ref = | 288 | kvmppc_e500_tlbe_release(&vcpu_e500->gtlb_priv[tlbsel][i]); |
| 289 | &vcpu_e500->gtlb_priv[tlbsel][i].ref; | ||
| 290 | kvmppc_e500_ref_release(ref); | ||
| 291 | } | ||
| 292 | } | 289 | } |
| 293 | } | 290 | } |
| 294 | 291 | ||
| @@ -304,18 +301,18 @@ void kvmppc_core_flush_tlb(struct kvm_vcpu *vcpu) · 9 unmodified lines | |||
| 304 | static void kvmppc_e500_setup_stlbe( | 301 | static void kvmppc_e500_setup_stlbe( |
| 305 | struct kvm_vcpu *vcpu, | 302 | struct kvm_vcpu *vcpu, |
| 306 | struct kvm_book3e_206_tlb_entry *gtlbe, | 303 | struct kvm_book3e_206_tlb_entry *gtlbe, |
| 307 | int tsize, struct tlbe_ref *ref, u64 gvaddr, | 304 | int tsize, struct tlbe_priv *tlbe, u64 gvaddr, |
| 308 | struct kvm_book3e_206_tlb_entry *stlbe) | 305 | struct kvm_book3e_206_tlb_entry *stlbe) |
| 309 | { | 306 | { |
| 310 | kvm_pfn_t pfn = ref->pfn; | 307 | kvm_pfn_t pfn = tlbe->pfn; |
| 311 | u32 pr = vcpu->arch.shared->msr & MSR_PR; | 308 | u32 pr = vcpu->arch.shared->msr & MSR_PR; |
| 312 | bool writable = !!(ref->flags & E500_TLB_WRITABLE); | 309 | bool writable = !!(tlbe->flags & E500_TLB_WRITABLE); |
| 313 | 310 | ||
| 314 | BUG_ON(!(ref->flags & E500_TLB_VALID)); | 311 | BUG_ON(!(tlbe->flags & E500_TLB_VALID)); |
| 315 | 312 | ||
| 316 | /* Force IPROT=0 for all guest mappings. */ | 313 | /* Force IPROT=0 for all guest mappings. */ |
| 317 | stlbe->mas1 = MAS1_TSIZE(tsize) | get_tlb_sts(gtlbe) | MAS1_VALID; | 314 | stlbe->mas1 = MAS1_TSIZE(tsize) | get_tlb_sts(gtlbe) | MAS1_VALID; |
| 318 | stlbe->mas2 = (gvaddr & MAS2_EPN) | (ref->flags & E500_TLB_MAS2_ATTR); | 315 | stlbe->mas2 = (gvaddr & MAS2_EPN) | (tlbe->flags & E500_TLB_MAS2_ATTR); |
| 319 | stlbe->mas7_3 = ((u64)pfn << PAGE_SHIFT) | | 316 | stlbe->mas7_3 = ((u64)pfn << PAGE_SHIFT) | |
| 320 | e500_shadow_mas3_attrib(gtlbe->mas7_3, writable, pr); | 317 | e500_shadow_mas3_attrib(gtlbe->mas7_3, writable, pr); |
| 321 | } | 318 | } |
| @@ -323,7 +320,7 @@ static void kvmppc_e500_setup_stlbe( · 1 unmodified line | |||
| 323 | static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, | 320 | static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, |
| 324 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, | 321 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, |
| 325 | int tlbsel, struct kvm_book3e_206_tlb_entry *stlbe, | 322 | int tlbsel, struct kvm_book3e_206_tlb_entry *stlbe, |
| 326 | struct tlbe_ref *ref) | 323 | struct tlbe_priv *tlbe) |
| 327 | { | 324 | { |
| 328 | struct kvm_memory_slot *slot; | 325 | struct kvm_memory_slot *slot; |
| 329 | unsigned int psize; | 326 | unsigned int psize; |
| @@ -455,9 +452,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, · 125 unmodified lines | |||
| 455 | } | 452 | } |
| 456 | } | 453 | } |
| 457 | 454 | ||
| 458 | kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg, writable); | 455 | kvmppc_e500_tlbe_setup(tlbe, gtlbe, pfn, wimg, writable); |
| 459 | kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, | 456 | kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, |
| 460 | ref, gvaddr, stlbe); | 457 | tlbe, gvaddr, stlbe); |
| 461 | writable = tlbe_is_writable(stlbe); | 458 | writable = tlbe_is_writable(stlbe); |
| 462 | 459 | ||
| 463 | /* Clear i-cache for new pages */ | 460 | /* Clear i-cache for new pages */ |
| @@ -474,17 +471,17 @@ static int kvmppc_e500_tlb0_map(struct kvmppc_vcpu_e500 *vcpu_e500, int esel, · 10 unmodified lines | |||
| 474 | struct kvm_book3e_206_tlb_entry *stlbe) | 471 | struct kvm_book3e_206_tlb_entry *stlbe) |
| 475 | { | 472 | { |
| 476 | struct kvm_book3e_206_tlb_entry *gtlbe; | 473 | struct kvm_book3e_206_tlb_entry *gtlbe; |
| 477 | struct tlbe_ref *ref; | 474 | struct tlbe_priv *tlbe; |
| 478 | int stlbsel = 0; | 475 | int stlbsel = 0; |
| 479 | int sesel = 0; | 476 | int sesel = 0; |
| 480 | int r; | 477 | int r; |
| 481 | 478 | ||
| 482 | gtlbe = get_entry(vcpu_e500, 0, esel); | 479 | gtlbe = get_entry(vcpu_e500, 0, esel); |
| 483 | ref = &vcpu_e500->gtlb_priv[0][esel].ref; | 480 | tlbe = &vcpu_e500->gtlb_priv[0][esel]; |
| 484 | 481 | ||
| 485 | r = kvmppc_e500_shadow_map(vcpu_e500, get_tlb_eaddr(gtlbe), | 482 | r = kvmppc_e500_shadow_map(vcpu_e500, get_tlb_eaddr(gtlbe), |
| 486 | get_tlb_raddr(gtlbe) >> PAGE_SHIFT, | 483 | get_tlb_raddr(gtlbe) >> PAGE_SHIFT, |
| 487 | gtlbe, 0, stlbe, ref); | 484 | gtlbe, 0, stlbe, tlbe); |
| 488 | if (r) | 485 | if (r) |
| 489 | return r; | 486 | return r; |
| 490 | 487 | ||
| @@ -494,7 +491,7 @@ static int kvmppc_e500_tlb0_map(struct kvmppc_vcpu_e500 *vcpu_e500, int esel, · 3 unmodified lines | |||
| 494 | } | 491 | } |
| 495 | 492 | ||
| 496 | static int kvmppc_e500_tlb1_map_tlb1(struct kvmppc_vcpu_e500 *vcpu_e500, | 493 | static int kvmppc_e500_tlb1_map_tlb1(struct kvmppc_vcpu_e500 *vcpu_e500, |
| 497 | struct tlbe_ref *ref, | 494 | struct tlbe_priv *tlbe, |
| 498 | int esel) | 495 | int esel) |
| 499 | { | 496 | { |
| 500 | unsigned int sesel = vcpu_e500->host_tlb1_nv++; | 497 | unsigned int sesel = vcpu_e500->host_tlb1_nv++; |
| @@ -507,10 +504,10 @@ static int kvmppc_e500_tlb1_map_tlb1(struct kvmppc_vcpu_e500 *vcpu_e500, · 6 unmodified lines | |||
| 507 | vcpu_e500->g2h_tlb1_map[idx] &= ~(1ULL << sesel); | 504 | vcpu_e500->g2h_tlb1_map[idx] &= ~(1ULL << sesel); |
| 508 | } | 505 | } |
| 509 | 506 | ||
| 510 | vcpu_e500->gtlb_priv[1][esel].ref.flags |= E500_TLB_BITMAP; | 507 | vcpu_e500->gtlb_priv[1][esel].flags |= E500_TLB_BITMAP; |
| 511 | vcpu_e500->g2h_tlb1_map[esel] |= (u64)1 << sesel; | 508 | vcpu_e500->g2h_tlb1_map[esel] |= (u64)1 << sesel; |
| 512 | vcpu_e500->h2g_tlb1_rmap[sesel] = esel + 1; | 509 | vcpu_e500->h2g_tlb1_rmap[sesel] = esel + 1; |
| 513 | WARN_ON(!(ref->flags & E500_TLB_VALID)); | 510 | WARN_ON(!(tlbe->flags & E500_TLB_VALID)); |
| 514 | 511 | ||
| 515 | return sesel; | 512 | return sesel; |
| 516 | } | 513 | } |
| @@ -522,24 +519,24 @@ static int kvmppc_e500_tlb1_map(struct kvmppc_vcpu_e500 *vcpu_e500, · 5 unmodified lines | |||
| 522 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, | 519 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, |
| 523 | struct kvm_book3e_206_tlb_entry *stlbe, int esel) | 520 | struct kvm_book3e_206_tlb_entry *stlbe, int esel) |
| 524 | { | 521 | { |
| 525 | struct tlbe_ref *ref = &vcpu_e500->gtlb_priv[1][esel].ref; | 522 | struct tlbe_priv *tlbe = &vcpu_e500->gtlb_priv[1][esel]; |
| 526 | int sesel; | 523 | int sesel; |
| 527 | int r; | 524 | int r; |
| 528 | 525 | ||
| 529 | r = kvmppc_e500_shadow_map(vcpu_e500, gvaddr, gfn, gtlbe, 1, stlbe, | 526 | r = kvmppc_e500_shadow_map(vcpu_e500, gvaddr, gfn, gtlbe, 1, stlbe, |
| 530 | ref); | 527 | tlbe); |
| 531 | if (r) | 528 | if (r) |
| 532 | return r; | 529 | return r; |
| 533 | 530 | ||
| 534 | /* Use TLB0 when we can only map a page with 4k */ | 531 | /* Use TLB0 when we can only map a page with 4k */ |
| 535 | if (get_tlb_tsize(stlbe) == BOOK3E_PAGESZ_4K) { | 532 | if (get_tlb_tsize(stlbe) == BOOK3E_PAGESZ_4K) { |
| 536 | vcpu_e500->gtlb_priv[1][esel].ref.flags |= E500_TLB_TLB0; | 533 | vcpu_e500->gtlb_priv[1][esel].flags |= E500_TLB_TLB0; |
| 537 | write_stlbe(vcpu_e500, gtlbe, stlbe, 0, 0); | 534 | write_stlbe(vcpu_e500, gtlbe, stlbe, 0, 0); |
| 538 | return 0; | 535 | return 0; |
| 539 | } | 536 | } |
| 540 | 537 | ||
| 541 | /* Otherwise map into TLB1 */ | 538 | /* Otherwise map into TLB1 */ |
| 542 | sesel = kvmppc_e500_tlb1_map_tlb1(vcpu_e500, ref, esel); | 539 | sesel = kvmppc_e500_tlb1_map_tlb1(vcpu_e500, tlbe, esel); |
| 543 | write_stlbe(vcpu_e500, gtlbe, stlbe, 1, sesel); | 540 | write_stlbe(vcpu_e500, gtlbe, stlbe, 1, sesel); |
| 544 | 541 | ||
| 545 | return 0; | 542 | return 0; |
| @@ -561,11 +558,11 @@ void kvmppc_mmu_map(struct kvm_vcpu *vcpu, u64 eaddr, gpa_t gpaddr, · 15 unmodified lines | |||
| 561 | priv = &vcpu_e500->gtlb_priv[tlbsel][esel]; | 558 | priv = &vcpu_e500->gtlb_priv[tlbsel][esel]; |
| 562 | 559 | ||
| 563 | /* Triggers after clear_tlb_privs or on initial mapping */ | 560 | /* Triggers after clear_tlb_privs or on initial mapping */ |
| 564 | if (!(priv->ref.flags & E500_TLB_VALID)) { | 561 | if (!(priv->flags & E500_TLB_VALID)) { |
| 565 | kvmppc_e500_tlb0_map(vcpu_e500, esel, &stlbe); | 562 | kvmppc_e500_tlb0_map(vcpu_e500, esel, &stlbe); |
| 566 | } else { | 563 | } else { |
| 567 | kvmppc_e500_setup_stlbe(vcpu, gtlbe, BOOK3E_PAGESZ_4K, | 564 | kvmppc_e500_setup_stlbe(vcpu, gtlbe, BOOK3E_PAGESZ_4K, |
| 568 | &priv->ref, eaddr, &stlbe); | 565 | priv, eaddr, &stlbe); |
| 569 | write_stlbe(vcpu_e500, gtlbe, &stlbe, 0, 0); | 566 | write_stlbe(vcpu_e500, gtlbe, &stlbe, 0, 0); |
| 570 | } | 567 | } |
| 571 | break; | 568 | break; |
| @@ -623,12 +623,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 622 unmodified lines | |||
| 623 | r = !!(hv_enabled && kvmppc_hv_ops->enable_nested && | 623 | r = !!(hv_enabled && kvmppc_hv_ops->enable_nested && |
| 624 | !kvmppc_hv_ops->enable_nested(NULL)); | 624 | !kvmppc_hv_ops->enable_nested(NULL)); |
| 625 | break; | 625 | break; |
| 626 | #endif | ||
| 627 | case KVM_CAP_SYNC_MMU: | ||
| 628 | BUILD_BUG_ON(!IS_ENABLED(CONFIG_KVM_GENERIC_MMU_NOTIFIER)); | ||
| 629 | r = 1; | ||
| 630 | break; | ||
| 631 | #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE | ||
| 632 | case KVM_CAP_PPC_HTAB_FD: | 626 | case KVM_CAP_PPC_HTAB_FD: |
| 633 | r = hv_enabled; | 627 | r = hv_enabled; |
| 634 | break; | 628 | break; |
| @@ -562,3 +562,4 @@ exc; std r10,32(3) · 561 unmodified lines | |||
| 562 | li r5,4096 | 562 | li r5,4096 |
| 563 | b .Ldst_aligned | 563 | b .Ldst_aligned |
| 564 | EXPORT_SYMBOL(__copy_tofrom_user) | 564 | EXPORT_SYMBOL(__copy_tofrom_user) |
| 565 | EXPORT_SYMBOL(__copy_tofrom_user_base) | ||
| @@ -5,13 +5,9 @@ · 4 unmodified lines | |||
| 5 | * | 5 | * |
| 6 | * Author: Anton Blanchard <anton@au.ibm.com> | 6 | * Author: Anton Blanchard <anton@au.ibm.com> |
| 7 | */ | 7 | */ |
| 8 | #include <linux/export.h> | ||
| 8 | #include <asm/ppc_asm.h> | 9 | #include <asm/ppc_asm.h> |
| 9 | 10 | ||
| 10 | #ifndef SELFTEST_CASE | ||
| 11 | /* 0 == don't use VMX, 1 == use VMX */ | ||
| 12 | #define SELFTEST_CASE 0 | ||
| 13 | #endif | ||
| 14 | |||
| 15 | #ifdef __BIG_ENDIAN__ | 11 | #ifdef __BIG_ENDIAN__ |
| 16 | #define LVS(VRT,RA,RB) lvsl VRT,RA,RB | 12 | #define LVS(VRT,RA,RB) lvsl VRT,RA,RB |
| 17 | #define VPERM(VRT,VRA,VRB,VRC) vperm VRT,VRA,VRB,VRC | 13 | #define VPERM(VRT,VRA,VRB,VRC) vperm VRT,VRA,VRB,VRC |
| @@ -47,10 +43,14 @@ · 29 unmodified lines | |||
| 47 | ld r15,STK_REG(R15)(r1) | 43 | ld r15,STK_REG(R15)(r1) |
| 48 | ld r14,STK_REG(R14)(r1) | 44 | ld r14,STK_REG(R14)(r1) |
| 49 | .Ldo_err3: | 45 | .Ldo_err3: |
| 50 | bl CFUNC(exit_vmx_usercopy) | 46 | ld r6,STK_REG(R31)(r1) /* original destination pointer */ |
| 47 | ld r5,STK_REG(R29)(r1) /* original number of bytes */ | ||
| 48 | subf r7,r6,r3 /* #bytes copied */ | ||
| 49 | subf r3,r7,r5 /* #bytes not copied in r3 */ | ||
| 51 | ld r0,STACKFRAMESIZE+16(r1) | 50 | ld r0,STACKFRAMESIZE+16(r1) |
| 52 | mtlr r0 | 51 | mtlr r0 |
| 53 | b .Lexit | 52 | addi r1,r1,STACKFRAMESIZE |
| 53 | blr | ||
| 54 | #endif /* CONFIG_ALTIVEC */ | 54 | #endif /* CONFIG_ALTIVEC */ |
| 55 | 55 | ||
| 56 | .Ldo_err2: | 56 | .Ldo_err2: |
| @@ -74,7 +74,6 @@ · 17 unmodified lines | |||
| 74 | 74 | ||
| 75 | _GLOBAL(__copy_tofrom_user_power7) | 75 | _GLOBAL(__copy_tofrom_user_power7) |
| 76 | cmpldi r5,16 | 76 | cmpldi r5,16 |
| 77 | cmpldi cr1,r5,3328 | ||
| 78 | 77 | ||
| 79 | std r3,-STACKFRAMESIZE+STK_REG(R31)(r1) | 78 | std r3,-STACKFRAMESIZE+STK_REG(R31)(r1) |
| 80 | std r4,-STACKFRAMESIZE+STK_REG(R30)(r1) | 79 | std r4,-STACKFRAMESIZE+STK_REG(R30)(r1) |
| @@ -82,12 +81,6 @@ _GLOBAL(__copy_tofrom_user_power7) · 1 unmodified line | |||
| 82 | 81 | ||
| 83 | blt .Lshort_copy | 82 | blt .Lshort_copy |
| 84 | 83 | ||
| 85 | #ifdef CONFIG_ALTIVEC | ||
| 86 | test_feature = SELFTEST_CASE | ||
| 87 | BEGIN_FTR_SECTION | ||
| 88 | bgt cr1,.Lvmx_copy | ||
| 89 | END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC) | ||
| 90 | #endif | ||
| 91 | 84 | ||
| 92 | .Lnonvmx_copy: | 85 | .Lnonvmx_copy: |
| 93 | /* Get the source 8B aligned */ | 86 | /* Get the source 8B aligned */ |
| @@ -263,23 +256,14 @@ err1; stb r0,0(r3) · 169 unmodified lines | |||
| 263 | 15: li r3,0 | 256 | 15: li r3,0 |
| 264 | blr | 257 | blr |
| 265 | 258 | ||
| 266 | .Lunwind_stack_nonvmx_copy: | ||
| 267 | addi r1,r1,STACKFRAMESIZE | ||
| 268 | b .Lnonvmx_copy | ||
| 269 | |||
| 270 | .Lvmx_copy: | ||
| 271 | #ifdef CONFIG_ALTIVEC | 259 | #ifdef CONFIG_ALTIVEC |
| 260 | _GLOBAL(__copy_tofrom_user_power7_vmx) | ||
| 272 | mflr r0 | 261 | mflr r0 |
| 273 | std r0,16(r1) | 262 | std r0,16(r1) |
| 274 | stdu r1,-STACKFRAMESIZE(r1) | 263 | stdu r1,-STACKFRAMESIZE(r1) |
| 275 | bl CFUNC(enter_vmx_usercopy) | ||
| 276 | cmpwi cr1,r3,0 | ||
| 277 | ld r0,STACKFRAMESIZE+16(r1) | ||
| 278 | ld r3,STK_REG(R31)(r1) | ||
| 279 | ld r4,STK_REG(R30)(r1) | ||
| 280 | ld r5,STK_REG(R29)(r1) | ||
| 281 | mtlr r0 | ||
| 282 | 264 | ||
| 265 | std r3,STK_REG(R31)(r1) | ||
| 266 | std r5,STK_REG(R29)(r1) | ||
| 283 | /* | 267 | /* |
| 284 | * We prefetch both the source and destination using enhanced touch | 268 | * We prefetch both the source and destination using enhanced touch |
| 285 | * instructions. We use a stream ID of 0 for the load side and | 269 | * instructions. We use a stream ID of 0 for the load side and |
| @@ -300,8 +284,6 @@ err1; stb r0,0(r3) · 14 unmodified lines | |||
| 300 | 284 | ||
| 301 | DCBT_SETUP_STREAMS(r6, r7, r9, r10, r8) | 285 | DCBT_SETUP_STREAMS(r6, r7, r9, r10, r8) |
| 302 | 286 | ||
| 303 | beq cr1,.Lunwind_stack_nonvmx_copy | ||
| 304 | |||
| 305 | /* | 287 | /* |
| 306 | * If source and destination are not relatively aligned we use a | 288 | * If source and destination are not relatively aligned we use a |
| 307 | * slower permute loop. | 289 | * slower permute loop. |
| @@ -478,7 +460,8 @@ err3; lbz r0,0(r4) · 170 unmodified lines | |||
| 478 | err3; stb r0,0(r3) | 460 | err3; stb r0,0(r3) |
| 479 | 461 | ||
| 480 | 15: addi r1,r1,STACKFRAMESIZE | 462 | 15: addi r1,r1,STACKFRAMESIZE |
| 481 | b CFUNC(exit_vmx_usercopy) /* tail call optimise */ | 463 | li r3,0 |
| 464 | blr | ||
| 482 | 465 | ||
| 483 | .Lvmx_unaligned_copy: | 466 | .Lvmx_unaligned_copy: |
| 484 | /* Get the destination 16B aligned */ | 467 | /* Get the destination 16B aligned */ |
| @@ -681,5 +664,7 @@ err3; lbz r0,0(r4) · 196 unmodified lines | |||
| 681 | err3; stb r0,0(r3) | 664 | err3; stb r0,0(r3) |
| 682 | 665 | ||
| 683 | 15: addi r1,r1,STACKFRAMESIZE | 666 | 15: addi r1,r1,STACKFRAMESIZE |
| 684 | b CFUNC(exit_vmx_usercopy) /* tail call optimise */ | 667 | li r3,0 |
| 668 | blr | ||
| 669 | EXPORT_SYMBOL(__copy_tofrom_user_power7_vmx) | ||
| 685 | #endif /* CONFIG_ALTIVEC */ | 670 | #endif /* CONFIG_ALTIVEC */ |
| @@ -27,6 +27,7 @@ int enter_vmx_usercopy(void) · 26 unmodified lines | |||
| 27 | 27 | ||
| 28 | return 1; | 28 | return 1; |
| 29 | } | 29 | } |
| 30 | EXPORT_SYMBOL(enter_vmx_usercopy); | ||
| 30 | 31 | ||
| 31 | /* | 32 | /* |
| 32 | * This function must return 0 because we tail call optimise when calling | 33 | * This function must return 0 because we tail call optimise when calling |
| @@ -49,6 +50,7 @@ int exit_vmx_usercopy(void) · 16 unmodified lines | |||
| 49 | set_dec(1); | 50 | set_dec(1); |
| 50 | return 0; | 51 | return 0; |
| 51 | } | 52 | } |
| 53 | EXPORT_SYMBOL(exit_vmx_usercopy); | ||
| 52 | 54 | ||
| 53 | int enter_vmx_ops(void) | 55 | int enter_vmx_ops(void) |
| 54 | { | 56 | { |
| @@ -30,6 +30,10 @@ · 29 unmodified lines | |||
| 30 | #include <asm/setup.h> | 30 | #include <asm/setup.h> |
| 31 | #include <asm/fixmap.h> | 31 | #include <asm/fixmap.h> |
| 32 | 32 | ||
| 33 | #include <asm/fadump.h> | ||
| 34 | #include <asm/kexec.h> | ||
| 35 | #include <asm/kvm_ppc.h> | ||
| 36 | |||
| 33 | #include <mm/mmu_decl.h> | 37 | #include <mm/mmu_decl.h> |
| 34 | 38 | ||
| 35 | unsigned long long memory_limit __initdata; | 39 | unsigned long long memory_limit __initdata; |
| @@ -268,6 +272,16 @@ void __init paging_init(void) · 232 unmodified lines | |||
| 268 | 272 | ||
| 269 | void __init arch_mm_preinit(void) | 273 | void __init arch_mm_preinit(void) |
| 270 | { | 274 | { |
| 275 | |||
| 276 | /* | ||
| 277 | * Reserve large chunks of memory for use by CMA for kdump, fadump, KVM | ||
| 278 | * and hugetlb. These must be called after pageblock_order is | ||
| 279 | * initialised. | ||
| 280 | */ | ||
| 281 | fadump_cma_init(); | ||
| 282 | kdump_cma_reserve(); | ||
| 283 | kvm_cma_reserve(); | ||
| 284 | |||
| 271 | /* | 285 | /* |
| 272 | * book3s is limited to 16 page sizes due to encoding this in | 286 | * book3s is limited to 16 page sizes due to encoding this in |
| 273 | * a 4-bit field for slices. | 287 | * a 4-bit field for slices. |
| @@ -81,9 +81,6 @@ · 80 unmodified lines | |||
| 81 | 81 | ||
| 82 | #ifdef CONFIG_PPC64 | 82 | #ifdef CONFIG_PPC64 |
| 83 | 83 | ||
| 84 | /* for gpr non volatile registers BPG_REG_6 to 10 */ | ||
| 85 | #define BPF_PPC_STACK_SAVE (6 * 8) | ||
| 86 | |||
| 87 | /* If dummy pass (!image), account for maximum possible instructions */ | 84 | /* If dummy pass (!image), account for maximum possible instructions */ |
| 88 | #define PPC_LI64(d, i) do { \ | 85 | #define PPC_LI64(d, i) do { \ |
| 89 | if (!image) \ | 86 | if (!image) \ |
| @@ -219,8 +216,6 @@ int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, · 129 unmodified lines | |||
| 219 | int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass, | 216 | int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass, |
| 220 | struct codegen_context *ctx, int insn_idx, | 217 | struct codegen_context *ctx, int insn_idx, |
| 221 | int jmp_off, int dst_reg, u32 code); | 218 | int jmp_off, int dst_reg, u32 code); |
| 222 | |||
| 223 | int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); | ||
| 224 | #endif | 219 | #endif |
| 225 | 220 | ||
| 226 | #endif | 221 | #endif |
| @@ -450,7 +450,7 @@ bool bpf_jit_supports_subprog_tailcalls(void) · 449 unmodified lines | |||
| 450 | 450 | ||
| 451 | bool bpf_jit_supports_kfunc_call(void) | 451 | bool bpf_jit_supports_kfunc_call(void) |
| 452 | { | 452 | { |
| 453 | return true; | 453 | return IS_ENABLED(CONFIG_PPC64); |
| 454 | } | 454 | } |
| 455 | 455 | ||
| 456 | bool bpf_jit_supports_arena(void) | 456 | bool bpf_jit_supports_arena(void) |
| @@ -638,19 +638,12 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context · 181 unmodified lines | |||
| 638 | * for the traced function (BPF subprog/callee) to fetch it. | 638 | * for the traced function (BPF subprog/callee) to fetch it. |
| 639 | */ | 639 | */ |
| 640 | static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_context *ctx, | 640 | static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_context *ctx, |
| 641 | int func_frame_offset, | 641 | int bpf_frame_size, int r4_off) |
| 642 | int bpf_dummy_frame_size, int r4_off) | ||
| 643 | { | 642 | { |
| 644 | if (IS_ENABLED(CONFIG_PPC64)) { | 643 | if (IS_ENABLED(CONFIG_PPC64)) { |
| 645 | /* See Generated stack layout */ | 644 | EMIT(PPC_RAW_LD(_R4, _R1, bpf_frame_size)); |
| 646 | int tailcallinfo_offset = BPF_PPC_TAILCALL; | 645 | /* Refer to trampoline's Generated stack layout */ |
| 647 | 646 | EMIT(PPC_RAW_LD(_R3, _R4, -BPF_PPC_TAILCALL)); | |
| 648 | /* | ||
| 649 | * func_frame_offset = ...(1) | ||
| 650 | * bpf_dummy_frame_size + trampoline_frame_size | ||
| 651 | */ | ||
| 652 | EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); | ||
| 653 | EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); | ||
| 654 | 647 | ||
| 655 | /* | 648 | /* |
| 656 | * Setting the tail_call_info in trampoline's frame | 649 | * Setting the tail_call_info in trampoline's frame |
| @@ -658,22 +651,14 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte · 1 unmodified line | |||
| 658 | */ | 651 | */ |
| 659 | EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); | 652 | EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); |
| 660 | PPC_BCC_CONST_SHORT(COND_GT, 8); | 653 | PPC_BCC_CONST_SHORT(COND_GT, 8); |
| 661 | EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); | 654 | EMIT(PPC_RAW_ADDI(_R3, _R4, -BPF_PPC_TAILCALL)); |
| 655 | |||
| 662 | /* | 656 | /* |
| 663 | * From ...(1) above: | 657 | * Trampoline's tail_call_info is at the same offset, as that of |
| 664 | * trampoline_frame_bottom = ...(2) | 658 | * any bpf program, with reference to previous frame. Update the |
| 665 | * func_frame_offset - bpf_dummy_frame_size | 659 | * address of main's tail_call_info in trampoline frame. |
| 666 | * | ||
| 667 | * Using ...(2) derived above: | ||
| 668 | * trampoline_tail_call_info_offset = ...(3) | ||
| 669 | * trampoline_frame_bottom - tailcallinfo_offset | ||
| 670 | * | ||
| 671 | * From ...(3): | ||
| 672 | * Use trampoline_tail_call_info_offset to write reference of main's | ||
| 673 | * tail_call_info in trampoline frame. | ||
| 674 | */ | 660 | */ |
| 675 | EMIT(PPC_RAW_STL(_R3, _R1, (func_frame_offset - bpf_dummy_frame_size) | 661 | EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size - BPF_PPC_TAILCALL)); |
| 676 | - tailcallinfo_offset)); | ||
| 677 | } else { | 662 | } else { |
| 678 | /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ | 663 | /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ |
| 679 | EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); | 664 | EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); |
| @@ -681,7 +666,7 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte · 1 unmodified line | |||
| 681 | } | 666 | } |
| 682 | 667 | ||
| 683 | static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_context *ctx, | 668 | static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_context *ctx, |
| 684 | int func_frame_offset, int r4_off) | 669 | int bpf_frame_size, int r4_off) |
| 685 | { | 670 | { |
| 686 | if (IS_ENABLED(CONFIG_PPC32)) { | 671 | if (IS_ENABLED(CONFIG_PPC32)) { |
| 687 | /* | 672 | /* |
| @@ -692,12 +677,12 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_cont · 4 unmodified lines | |||
| 692 | } | 677 | } |
| 693 | } | 678 | } |
| 694 | 679 | ||
| 695 | static void bpf_trampoline_save_args(u32 *image, struct codegen_context *ctx, int func_frame_offset, | 680 | static void bpf_trampoline_save_args(u32 *image, struct codegen_context *ctx, |
| 696 | int nr_regs, int regs_off) | 681 | int bpf_frame_size, int nr_regs, int regs_off) |
| 697 | { | 682 | { |
| 698 | int param_save_area_offset; | 683 | int param_save_area_offset; |
| 699 | 684 | ||
| 700 | param_save_area_offset = func_frame_offset; /* the two frames we alloted */ | 685 | param_save_area_offset = bpf_frame_size; |
| 701 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ | 686 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ |
| 702 | 687 | ||
| 703 | for (int i = 0; i < nr_regs; i++) { | 688 | for (int i = 0; i < nr_regs; i++) { |
| @@ -720,11 +705,11 @@ static void bpf_trampoline_restore_args_regs(u32 *image, struct codegen_context · 16 unmodified lines | |||
| 720 | 705 | ||
| 721 | /* Used when we call into the traced function. Replicate parameter save area */ | 706 | /* Used when we call into the traced function. Replicate parameter save area */ |
| 722 | static void bpf_trampoline_restore_args_stack(u32 *image, struct codegen_context *ctx, | 707 | static void bpf_trampoline_restore_args_stack(u32 *image, struct codegen_context *ctx, |
| 723 | int func_frame_offset, int nr_regs, int regs_off) | 708 | int bpf_frame_size, int nr_regs, int regs_off) |
| 724 | { | 709 | { |
| 725 | int param_save_area_offset; | 710 | int param_save_area_offset; |
| 726 | 711 | ||
| 727 | param_save_area_offset = func_frame_offset; /* the two frames we alloted */ | 712 | param_save_area_offset = bpf_frame_size; |
| 728 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ | 713 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ |
| 729 | 714 | ||
| 730 | for (int i = 8; i < nr_regs; i++) { | 715 | for (int i = 8; i < nr_regs; i++) { |
| @@ -741,10 +726,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 10 unmodified lines | |||
| 741 | void *func_addr) | 726 | void *func_addr) |
| 742 | { | 727 | { |
| 743 | int regs_off, nregs_off, ip_off, run_ctx_off, retval_off, nvr_off, alt_lr_off, r4_off = 0; | 728 | int regs_off, nregs_off, ip_off, run_ctx_off, retval_off, nvr_off, alt_lr_off, r4_off = 0; |
| 744 | int i, ret, nr_regs, bpf_frame_size = 0, bpf_dummy_frame_size = 0, func_frame_offset; | ||
| 745 | struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; | 729 | struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; |
| 746 | struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; | 730 | struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; |
| 747 | struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; | 731 | struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; |
| 732 | int i, ret, nr_regs, retaddr_off, bpf_frame_size = 0; | ||
| 748 | struct codegen_context codegen_ctx, *ctx; | 733 | struct codegen_context codegen_ctx, *ctx; |
| 749 | u32 *image = (u32 *)rw_image; | 734 | u32 *image = (u32 *)rw_image; |
| 750 | ppc_inst_t branch_insn; | 735 | ppc_inst_t branch_insn; |
| @@ -770,24 +755,19 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 19 unmodified lines | |||
| 770 | * Generated stack layout: | 755 | * Generated stack layout: |
| 771 | * | 756 | * |
| 772 | * func prev back chain [ back chain ] | 757 | * func prev back chain [ back chain ] |
| 773 | * [ ] | ||
| 774 | * bpf prog redzone/tailcallcnt [ ... ] 64 bytes (64-bit powerpc) | ||
| 775 | * [ ] -- | ||
| 776 | * LR save area [ r0 save (64-bit) ] | header | ||
| 777 | * [ r0 save (32-bit) ] | | ||
| 778 | * dummy frame for unwind [ back chain 1 ] -- | ||
| 779 | * [ tail_call_info ] optional - 64-bit powerpc | 758 | * [ tail_call_info ] optional - 64-bit powerpc |
| 780 | * [ padding ] align stack frame | 759 | * [ padding ] align stack frame |
| 781 | * r4_off [ r4 (tailcallcnt) ] optional - 32-bit powerpc | 760 | * r4_off [ r4 (tailcallcnt) ] optional - 32-bit powerpc |
| 782 | * alt_lr_off [ real lr (ool stub)] optional - actual lr | 761 | * alt_lr_off [ real lr (ool stub)] optional - actual lr |
| 762 | * retaddr_off [ return address ] | ||
| 783 | * [ r26 ] | 763 | * [ r26 ] |
| 784 | * nvr_off [ r25 ] nvr save area | 764 | * nvr_off [ r25 ] nvr save area |
| 785 | * retval_off [ return value ] | 765 | * retval_off [ return value ] |
| 786 | * [ reg argN ] | 766 | * [ reg argN ] |
| 787 | * [ ... ] | 767 | * [ ... ] |
| 788 | * regs_off [ reg_arg1 ] prog ctx context | 768 | * regs_off [ reg_arg1 ] prog_ctx |
| 789 | * nregs_off [ args count ] | 769 | * nregs_off [ args count ] ((u64 *)prog_ctx)[-1] |
| 790 | * ip_off [ traced function ] | 770 | * ip_off [ traced function ] ((u64 *)prog_ctx)[-2] |
| 791 | * [ ... ] | 771 | * [ ... ] |
| 792 | * run_ctx_off [ bpf_tramp_run_ctx ] | 772 | * run_ctx_off [ bpf_tramp_run_ctx ] |
| 793 | * [ reg argN ] | 773 | * [ reg argN ] |
| @@ -843,6 +823,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 49 unmodified lines | |||
| 843 | nvr_off = bpf_frame_size; | 823 | nvr_off = bpf_frame_size; |
| 844 | bpf_frame_size += 2 * SZL; | 824 | bpf_frame_size += 2 * SZL; |
| 845 | 825 | ||
| 826 | /* Save area for return address */ | ||
| 827 | retaddr_off = bpf_frame_size; | ||
| 828 | bpf_frame_size += SZL; | ||
| 829 | |||
| 846 | /* Optional save area for actual LR in case of ool ftrace */ | 830 | /* Optional save area for actual LR in case of ool ftrace */ |
| 847 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | 831 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 848 | alt_lr_off = bpf_frame_size; | 832 | alt_lr_off = bpf_frame_size; |
| @@ -869,16 +853,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 20 unmodified lines | |||
| 869 | /* Padding to align stack frame, if any */ | 853 | /* Padding to align stack frame, if any */ |
| 870 | bpf_frame_size = round_up(bpf_frame_size, SZL * 2); | 854 | bpf_frame_size = round_up(bpf_frame_size, SZL * 2); |
| 871 | 855 | ||
| 872 | /* Dummy frame size for proper unwind - includes 64-bytes red zone for 64-bit powerpc */ | 856 | /* Store original return value */ |
| 873 | bpf_dummy_frame_size = STACK_FRAME_MIN_SIZE + 64; | ||
| 874 | |||
| 875 | /* Offset to the traced function's stack frame */ | ||
| 876 | func_frame_offset = bpf_dummy_frame_size + bpf_frame_size; | ||
| 877 | |||
| 878 | /* Create dummy frame for unwind, store original return value */ | ||
| 879 | EMIT(PPC_RAW_STL(_R0, _R1, PPC_LR_STKOFF)); | 857 | EMIT(PPC_RAW_STL(_R0, _R1, PPC_LR_STKOFF)); |
| 880 | /* Protect red zone where tail call count goes */ | ||
| 881 | EMIT(PPC_RAW_STLU(_R1, _R1, -bpf_dummy_frame_size)); | ||
| 882 | 858 | ||
| 883 | /* Create our stack frame */ | 859 | /* Create our stack frame */ |
| 884 | EMIT(PPC_RAW_STLU(_R1, _R1, -bpf_frame_size)); | 860 | EMIT(PPC_RAW_STLU(_R1, _R1, -bpf_frame_size)); |
| @@ -893,34 +869,44 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 8 unmodified lines | |||
| 893 | if (IS_ENABLED(CONFIG_PPC32) && nr_regs < 2) | 869 | if (IS_ENABLED(CONFIG_PPC32) && nr_regs < 2) |
| 894 | EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); | 870 | EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); |
| 895 | 871 | ||
| 896 | bpf_trampoline_save_args(image, ctx, func_frame_offset, nr_regs, regs_off); | 872 | bpf_trampoline_save_args(image, ctx, bpf_frame_size, nr_regs, regs_off); |
| 897 | 873 | ||
| 898 | /* Save our return address */ | 874 | /* Save our LR/return address */ |
| 899 | EMIT(PPC_RAW_MFLR(_R3)); | 875 | EMIT(PPC_RAW_MFLR(_R3)); |
| 900 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) | 876 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) |
| 901 | EMIT(PPC_RAW_STL(_R3, _R1, alt_lr_off)); | 877 | EMIT(PPC_RAW_STL(_R3, _R1, alt_lr_off)); |
| 902 | else | 878 | else |
| 903 | EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | 879 | EMIT(PPC_RAW_STL(_R3, _R1, retaddr_off)); |
| 904 | 880 | ||
| 905 | /* | 881 | /* |
| 906 | * Save ip address of the traced function. | 882 | * Derive IP address of the traced function. |
| 907 | * We could recover this from LR, but we will need to address for OOL trampoline, | 883 | * In case of CONFIG_PPC_FTRACE_OUT_OF_LINE or BPF program, LR points to the instruction |
| 908 | * and optional GEP area. | 884 | * after the 'bl' instruction in the OOL stub. Refer to ftrace_init_ool_stub() and |
| 885 | * bpf_arch_text_poke() for OOL stub of kernel functions and bpf programs respectively. | ||
| 886 | * Relevant stub sequence: | ||
| 887 | * | ||
| 888 | * bl <tramp> | ||
| 889 | * LR (R3) => mtlr r0 | ||
| 890 | * b <func_addr+4> | ||
| 891 | * | ||
| 892 | * Recover kernel function/bpf program address from the unconditional | ||
| 893 | * branch instruction at the end of OOL stub. | ||
| 909 | */ | 894 | */ |
| 910 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE) || flags & BPF_TRAMP_F_IP_ARG) { | 895 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE) || flags & BPF_TRAMP_F_IP_ARG) { |
| 911 | EMIT(PPC_RAW_LWZ(_R4, _R3, 4)); | 896 | EMIT(PPC_RAW_LWZ(_R4, _R3, 4)); |
| 912 | EMIT(PPC_RAW_SLWI(_R4, _R4, 6)); | 897 | EMIT(PPC_RAW_SLWI(_R4, _R4, 6)); |
| 913 | EMIT(PPC_RAW_SRAWI(_R4, _R4, 6)); | 898 | EMIT(PPC_RAW_SRAWI(_R4, _R4, 6)); |
| 914 | EMIT(PPC_RAW_ADD(_R3, _R3, _R4)); | 899 | EMIT(PPC_RAW_ADD(_R3, _R3, _R4)); |
| 915 | EMIT(PPC_RAW_ADDI(_R3, _R3, 4)); | ||
| 916 | } | 900 | } |
| 917 | 901 | ||
| 918 | if (flags & BPF_TRAMP_F_IP_ARG) | 902 | if (flags & BPF_TRAMP_F_IP_ARG) |
| 919 | EMIT(PPC_RAW_STL(_R3, _R1, ip_off)); | 903 | EMIT(PPC_RAW_STL(_R3, _R1, ip_off)); |
| 920 | 904 | ||
| 921 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) | 905 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 922 | /* Fake our LR for unwind */ | 906 | /* Fake our LR for BPF_TRAMP_F_CALL_ORIG case */ |
| 923 | EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | 907 | EMIT(PPC_RAW_ADDI(_R3, _R3, 4)); |
| 908 | EMIT(PPC_RAW_STL(_R3, _R1, retaddr_off)); | ||
| 909 | } | ||
| 924 | 910 | ||
| 925 | /* Save function arg count -- see bpf_get_func_arg_cnt() */ | 911 | /* Save function arg count -- see bpf_get_func_arg_cnt() */ |
| 926 | EMIT(PPC_RAW_LI(_R3, nr_regs)); | 912 | EMIT(PPC_RAW_LI(_R3, nr_regs)); |
| @@ -958,20 +944,19 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 31 unmodified lines | |||
| 958 | /* Call the traced function */ | 944 | /* Call the traced function */ |
| 959 | if (flags & BPF_TRAMP_F_CALL_ORIG) { | 945 | if (flags & BPF_TRAMP_F_CALL_ORIG) { |
| 960 | /* | 946 | /* |
| 961 | * The address in LR save area points to the correct point in the original function | 947 | * retaddr on trampoline stack points to the correct point in the original function |
| 962 | * with both PPC_FTRACE_OUT_OF_LINE as well as with traditional ftrace instruction | 948 | * with both PPC_FTRACE_OUT_OF_LINE as well as with traditional ftrace instruction |
| 963 | * sequence | 949 | * sequence |
| 964 | */ | 950 | */ |
| 965 | EMIT(PPC_RAW_LL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | 951 | EMIT(PPC_RAW_LL(_R3, _R1, retaddr_off)); |
| 966 | EMIT(PPC_RAW_MTCTR(_R3)); | 952 | EMIT(PPC_RAW_MTCTR(_R3)); |
| 967 | 953 | ||
| 968 | /* Replicate tail_call_cnt before calling the original BPF prog */ | 954 | /* Replicate tail_call_cnt before calling the original BPF prog */ |
| 969 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) | 955 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) |
| 970 | bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, | 956 | bpf_trampoline_setup_tail_call_info(image, ctx, bpf_frame_size, r4_off); |
| 971 | bpf_dummy_frame_size, r4_off); | ||
| 972 | 957 | ||
| 973 | /* Restore args */ | 958 | /* Restore args */ |
| 974 | bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs, regs_off); | 959 | bpf_trampoline_restore_args_stack(image, ctx, bpf_frame_size, nr_regs, regs_off); |
| 975 | 960 | ||
| 976 | /* Restore TOC for 64-bit */ | 961 | /* Restore TOC for 64-bit */ |
| 977 | if (IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2) && !IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) | 962 | if (IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2) && !IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) |
| @@ -985,7 +970,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 7 unmodified lines | |||
| 985 | 970 | ||
| 986 | /* Restore updated tail_call_cnt */ | 971 | /* Restore updated tail_call_cnt */ |
| 987 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) | 972 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) |
| 988 | bpf_trampoline_restore_tail_call_cnt(image, ctx, func_frame_offset, r4_off); | 973 | bpf_trampoline_restore_tail_call_cnt(image, ctx, bpf_frame_size, r4_off); |
| 989 | 974 | ||
| 990 | /* Reserve space to patch branch instruction to skip fexit progs */ | 975 | /* Reserve space to patch branch instruction to skip fexit progs */ |
| 991 | if (ro_image) /* image is NULL for dummy pass */ | 976 | if (ro_image) /* image is NULL for dummy pass */ |
| @@ -1037,7 +1022,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 45 unmodified lines | |||
| 1037 | EMIT(PPC_RAW_LD(_R2, _R1, 24)); | 1022 | EMIT(PPC_RAW_LD(_R2, _R1, 24)); |
| 1038 | if (flags & BPF_TRAMP_F_SKIP_FRAME) { | 1023 | if (flags & BPF_TRAMP_F_SKIP_FRAME) { |
| 1039 | /* Skip the traced function and return to parent */ | 1024 | /* Skip the traced function and return to parent */ |
| 1040 | EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | 1025 | EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); |
| 1041 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); | 1026 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1042 | EMIT(PPC_RAW_MTLR(_R0)); | 1027 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1043 | EMIT(PPC_RAW_BLR()); | 1028 | EMIT(PPC_RAW_BLR()); |
| @@ -1045,13 +1030,13 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 1 unmodified line | |||
| 1045 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | 1030 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 1046 | EMIT(PPC_RAW_LL(_R0, _R1, alt_lr_off)); | 1031 | EMIT(PPC_RAW_LL(_R0, _R1, alt_lr_off)); |
| 1047 | EMIT(PPC_RAW_MTLR(_R0)); | 1032 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1048 | EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | 1033 | EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); |
| 1049 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); | 1034 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1050 | EMIT(PPC_RAW_BLR()); | 1035 | EMIT(PPC_RAW_BLR()); |
| 1051 | } else { | 1036 | } else { |
| 1052 | EMIT(PPC_RAW_LL(_R0, _R1, bpf_frame_size + PPC_LR_STKOFF)); | 1037 | EMIT(PPC_RAW_LL(_R0, _R1, retaddr_off)); |
| 1053 | EMIT(PPC_RAW_MTCTR(_R0)); | 1038 | EMIT(PPC_RAW_MTCTR(_R0)); |
| 1054 | EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | 1039 | EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); |
| 1055 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); | 1040 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1056 | EMIT(PPC_RAW_MTLR(_R0)); | 1041 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1057 | EMIT(PPC_RAW_BCTR()); | 1042 | EMIT(PPC_RAW_BCTR()); |
| @@ -32,23 +32,27 @@ · 31 unmodified lines | |||
| 32 | * | 32 | * |
| 33 | * [ prev sp ] <------------- | 33 | * [ prev sp ] <------------- |
| 34 | * [ tail_call_info ] 8 | | 34 | * [ tail_call_info ] 8 | |
| 35 | * [ nv gpr save area ] 6*8 + (12*8) | | 35 | * [ nv gpr save area ] (6 * 8) | |
| 36 | * [ addl. nv gpr save area] (12 * 8) | <--- exception boundary/callback program | ||
| 36 | * [ local_tmp_var ] 24 | | 37 | * [ local_tmp_var ] 24 | |
| 37 | * fp (r31) --> [ ebpf stack space ] upto 512 | | 38 | * fp (r31) --> [ ebpf stack space ] upto 512 | |
| 38 | * [ frame header ] 32/112 | | 39 | * [ frame header ] 32/112 | |
| 39 | * sp (r1) ---> [ stack pointer ] -------------- | 40 | * sp (r1) ---> [ stack pointer ] -------------- |
| 40 | * | 41 | * |
| 41 | * Additional (12*8) in 'nv gpr save area' only in case of | 42 | * Additional (12 * 8) in 'nv gpr save area' only in case of |
| 42 | * exception boundary. | 43 | * exception boundary/callback. |
| 43 | */ | 44 | */ |
| 44 | 45 | ||
| 46 | /* BPF non-volatile registers save area size */ | ||
| 47 | #define BPF_PPC_STACK_SAVE (6 * 8) | ||
| 48 | |||
| 45 | /* for bpf JIT code internal usage */ | 49 | /* for bpf JIT code internal usage */ |
| 46 | #define BPF_PPC_STACK_LOCALS 24 | 50 | #define BPF_PPC_STACK_LOCALS 24 |
| 47 | /* | 51 | /* |
| 48 | * for additional non volatile registers(r14-r25) to be saved | 52 | * for additional non volatile registers(r14-r25) to be saved |
| 49 | * at exception boundary | 53 | * at exception boundary |
| 50 | */ | 54 | */ |
| 51 | #define BPF_PPC_EXC_STACK_SAVE (12*8) | 55 | #define BPF_PPC_EXC_STACK_SAVE (12 * 8) |
| 52 | 56 | ||
| 53 | /* stack frame excluding BPF stack, ensure this is quadword aligned */ | 57 | /* stack frame excluding BPF stack, ensure this is quadword aligned */ |
| 54 | #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ | 58 | #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ |
| @@ -125,12 +129,13 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx) · 70 unmodified lines | |||
| 125 | * [ ... ] | | 129 | * [ ... ] | |
| 126 | * sp (r1) ---> [ stack pointer ] -------------- | 130 | * sp (r1) ---> [ stack pointer ] -------------- |
| 127 | * [ tail_call_info ] 8 | 131 | * [ tail_call_info ] 8 |
| 128 | * [ nv gpr save area ] 6*8 + (12*8) | 132 | * [ nv gpr save area ] (6 * 8) |
| 133 | * [ addl. nv gpr save area] (12 * 8) <--- exception boundary/callback program | ||
| 129 | * [ local_tmp_var ] 24 | 134 | * [ local_tmp_var ] 24 |
| 130 | * [ unused red zone ] 224 | 135 | * [ unused red zone ] 224 |
| 131 | * | 136 | * |
| 132 | * Additional (12*8) in 'nv gpr save area' only in case of | 137 | * Additional (12 * 8) in 'nv gpr save area' only in case of |
| 133 | * exception boundary. | 138 | * exception boundary/callback. |
| 134 | */ | 139 | */ |
| 135 | static int bpf_jit_stack_local(struct codegen_context *ctx) | 140 | static int bpf_jit_stack_local(struct codegen_context *ctx) |
| 136 | { | 141 | { |
| @@ -148,7 +153,7 @@ static int bpf_jit_stack_local(struct codegen_context *ctx) · 11 unmodified lines | |||
| 148 | } | 153 | } |
| 149 | } | 154 | } |
| 150 | 155 | ||
| 151 | int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) | 156 | static int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) |
| 152 | { | 157 | { |
| 153 | return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE; | 158 | return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE; |
| 154 | } | 159 | } |
| @@ -237,10 +242,6 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) · 82 unmodified lines | |||
| 237 | 242 | ||
| 238 | if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { | 243 | if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { |
| 239 | /* | 244 | /* |
| 240 | * exception_cb uses boundary frame after stack walk. | ||
| 241 | * It can simply use redzone, this optimization reduces | ||
| 242 | * stack walk loop by one level. | ||
| 243 | * | ||
| 244 | * We need a stack frame, but we don't necessarily need to | 245 | * We need a stack frame, but we don't necessarily need to |
| 245 | * save/restore LR unless we call other functions | 246 | * save/restore LR unless we call other functions |
| 246 | */ | 247 | */ |
| @@ -284,6 +285,22 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) · 37 unmodified lines | |||
| 284 | * program(main prog) as third arg | 285 | * program(main prog) as third arg |
| 285 | */ | 286 | */ |
| 286 | EMIT(PPC_RAW_MR(_R1, _R5)); | 287 | EMIT(PPC_RAW_MR(_R1, _R5)); |
| 288 | /* | ||
| 289 | * Exception callback reuses the stack frame of exception boundary. | ||
| 290 | * But BPF stack depth of exception callback and exception boundary | ||
| 291 | * don't have to be same. If BPF stack depth is different, adjust the | ||
| 292 | * stack frame size considering BPF stack depth of exception callback. | ||
| 293 | * The non-volatile register save area remains unchanged. These non- | ||
| 294 | * volatile registers are restored in exception callback's epilogue. | ||
| 295 | */ | ||
| 296 | EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R5, 0)); | ||
| 297 | EMIT(PPC_RAW_SUB(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_1), _R1)); | ||
| 298 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | ||
| 299 | -BPF_PPC_EXC_STACKFRAME)); | ||
| 300 | EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), ctx->stack_size)); | ||
| 301 | PPC_BCC_CONST_SHORT(COND_EQ, 12); | ||
| 302 | EMIT(PPC_RAW_MR(_R1, bpf_to_ppc(TMP_REG_1))); | ||
| 303 | EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_EXC_STACKFRAME + ctx->stack_size))); | ||
| 287 | } | 304 | } |
| 288 | 305 | ||
| 289 | /* | 306 | /* |
| @@ -482,6 +499,83 @@ int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context * · 192 unmodified lines | |||
| 482 | return 0; | 499 | return 0; |
| 483 | } | 500 | } |
| 484 | 501 | ||
| 502 | static int zero_extend(u32 *image, struct codegen_context *ctx, u32 src_reg, u32 dst_reg, u32 size) | ||
| 503 | { | ||
| 504 | switch (size) { | ||
| 505 | case 1: | ||
| 506 | /* zero-extend 8 bits into 64 bits */ | ||
| 507 | EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 56)); | ||
| 508 | return 0; | ||
| 509 | case 2: | ||
| 510 | /* zero-extend 16 bits into 64 bits */ | ||
| 511 | EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 48)); | ||
| 512 | return 0; | ||
| 513 | case 4: | ||
| 514 | /* zero-extend 32 bits into 64 bits */ | ||
| 515 | EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 32)); | ||
| 516 | fallthrough; | ||
| 517 | case 8: | ||
| 518 | /* Nothing to do */ | ||
| 519 | return 0; | ||
| 520 | default: | ||
| 521 | return -1; | ||
| 522 | } | ||
| 523 | } | ||
| 524 | |||
| 525 | static int sign_extend(u32 *image, struct codegen_context *ctx, u32 src_reg, u32 dst_reg, u32 size) | ||
| 526 | { | ||
| 527 | switch (size) { | ||
| 528 | case 1: | ||
| 529 | /* sign-extend 8 bits into 64 bits */ | ||
| 530 | EMIT(PPC_RAW_EXTSB(dst_reg, src_reg)); | ||
| 531 | return 0; | ||
| 532 | case 2: | ||
| 533 | /* sign-extend 16 bits into 64 bits */ | ||
| 534 | EMIT(PPC_RAW_EXTSH(dst_reg, src_reg)); | ||
| 535 | return 0; | ||
| 536 | case 4: | ||
| 537 | /* sign-extend 32 bits into 64 bits */ | ||
| 538 | EMIT(PPC_RAW_EXTSW(dst_reg, src_reg)); | ||
| 539 | fallthrough; | ||
| 540 | case 8: | ||
| 541 | /* Nothing to do */ | ||
| 542 | return 0; | ||
| 543 | default: | ||
| 544 | return -1; | ||
| 545 | } | ||
| 546 | } | ||
| 547 | |||
| 548 | /* | ||
| 549 | * Handle powerpc ABI expectations from caller: | ||
| 550 | * - Unsigned arguments are zero-extended. | ||
| 551 | * - Signed arguments are sign-extended. | ||
| 552 | */ | ||
| 553 | static int prepare_for_kfunc_call(const struct bpf_prog *fp, u32 *image, | ||
| 554 | struct codegen_context *ctx, | ||
| 555 | const struct bpf_insn *insn) | ||
| 556 | { | ||
| 557 | const struct btf_func_model *m = bpf_jit_find_kfunc_model(fp, insn); | ||
| 558 | int i; | ||
| 559 | |||
| 560 | if (!m) | ||
| 561 | return -1; | ||
| 562 | |||
| 563 | for (i = 0; i < m->nr_args; i++) { | ||
| 564 | /* Note that BPF ABI only allows up to 5 args for kfuncs */ | ||
| 565 | u32 reg = bpf_to_ppc(BPF_REG_1 + i), size = m->arg_size[i]; | ||
| 566 | |||
| 567 | if (!(m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG)) { | ||
| 568 | if (zero_extend(image, ctx, reg, reg, size)) | ||
| 569 | return -1; | ||
| 570 | } else { | ||
| 571 | if (sign_extend(image, ctx, reg, reg, size)) | ||
| 572 | return -1; | ||
| 573 | } | ||
| 574 | } | ||
| 575 | |||
| 576 | return 0; | ||
| 577 | } | ||
| 578 | |||
| 485 | static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 out) | 579 | static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 out) |
| 486 | { | 580 | { |
| 487 | /* | 581 | /* |
| @@ -522,9 +616,30 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o · 34 unmodified lines | |||
| 522 | 616 | ||
| 523 | /* | 617 | /* |
| 524 | * tail_call_info++; <- Actual value of tcc here | 618 | * tail_call_info++; <- Actual value of tcc here |
| 619 | * Writeback this updated value only if tailcall succeeds. | ||
| 525 | */ | 620 | */ |
| 526 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); | 621 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); |
| 527 | 622 | ||
| 623 | /* prog = array->ptrs[index]; */ | ||
| 624 | EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_2), b2p_index, 8)); | ||
| 625 | EMIT(PPC_RAW_ADD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), b2p_bpf_array)); | ||
| 626 | EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | ||
| 627 | offsetof(struct bpf_array, ptrs))); | ||
| 628 | |||
| 629 | /* | ||
| 630 | * if (prog == NULL) | ||
| 631 | * goto out; | ||
| 632 | */ | ||
| 633 | EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), 0)); | ||
| 634 | PPC_BCC_SHORT(COND_EQ, out); | ||
| 635 | |||
| 636 | /* goto *(prog->bpf_func + prologue_size); */ | ||
| 637 | EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | ||
| 638 | offsetof(struct bpf_prog, bpf_func))); | ||
| 639 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | ||
| 640 | FUNCTION_DESCR_SIZE + bpf_tailcall_prologue_size)); | ||
| 641 | EMIT(PPC_RAW_MTCTR(bpf_to_ppc(TMP_REG_2))); | ||
| 642 | |||
| 528 | /* | 643 | /* |
| 529 | * Before writing updated tail_call_info, distinguish if current frame | 644 | * Before writing updated tail_call_info, distinguish if current frame |
| 530 | * is storing a reference to tail_call_info or actual tcc value in | 645 | * is storing a reference to tail_call_info or actual tcc value in |
| @@ -539,24 +654,6 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o · 8 unmodified lines | |||
| 539 | /* Writeback updated value to tail_call_info */ | 654 | /* Writeback updated value to tail_call_info */ |
| 540 | EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); | 655 | EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); |
| 541 | 656 | ||
| 542 | /* prog = array->ptrs[index]; */ | ||
| 543 | EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); | ||
| 544 | EMIT(PPC_RAW_ADD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), b2p_bpf_array)); | ||
| 545 | EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), offsetof(struct bpf_array, ptrs))); | ||
| 546 | |||
| 547 | /* | ||
| 548 | * if (prog == NULL) | ||
| 549 | * goto out; | ||
| 550 | */ | ||
| 551 | EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_1), 0)); | ||
| 552 | PPC_BCC_SHORT(COND_EQ, out); | ||
| 553 | |||
| 554 | /* goto *(prog->bpf_func + prologue_size); */ | ||
| 555 | EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), offsetof(struct bpf_prog, bpf_func))); | ||
| 556 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), | ||
| 557 | FUNCTION_DESCR_SIZE + bpf_tailcall_prologue_size)); | ||
| 558 | EMIT(PPC_RAW_MTCTR(bpf_to_ppc(TMP_REG_1))); | ||
| 559 | |||
| 560 | /* tear down stack, restore NVRs, ... */ | 657 | /* tear down stack, restore NVRs, ... */ |
| 561 | bpf_jit_emit_common_epilogue(image, ctx); | 658 | bpf_jit_emit_common_epilogue(image, ctx); |
| 562 | 659 | ||
| @@ -1123,14 +1220,16 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code · 560 unmodified lines | |||
| 1123 | /* special mov32 for zext */ | 1220 | /* special mov32 for zext */ |
| 1124 | EMIT(PPC_RAW_RLWINM(dst_reg, dst_reg, 0, 0, 31)); | 1221 | EMIT(PPC_RAW_RLWINM(dst_reg, dst_reg, 0, 0, 31)); |
| 1125 | break; | 1222 | break; |
| 1126 | } else if (off == 8) { | 1223 | } |
| 1127 | EMIT(PPC_RAW_EXTSB(dst_reg, src_reg)); | 1224 | if (off == 0) { |
| 1128 | } else if (off == 16) { | 1225 | /* MOV */ |
| 1129 | EMIT(PPC_RAW_EXTSH(dst_reg, src_reg)); | 1226 | if (dst_reg != src_reg) |
| 1130 | } else if (off == 32) { | 1227 | EMIT(PPC_RAW_MR(dst_reg, src_reg)); |
| 1131 | EMIT(PPC_RAW_EXTSW(dst_reg, src_reg)); | 1228 | } else { |
| 1132 | } else if (dst_reg != src_reg) | 1229 | /* MOVSX: dst = (s8,s16,s32)src (off = 8,16,32) */ |
| 1133 | EMIT(PPC_RAW_MR(dst_reg, src_reg)); | 1230 | if (sign_extend(image, ctx, src_reg, dst_reg, off / 8)) |
| 1231 | return -1; | ||
| 1232 | } | ||
| 1134 | goto bpf_alu32_trunc; | 1233 | goto bpf_alu32_trunc; |
| 1135 | case BPF_ALU | BPF_MOV | BPF_K: /* (u32) dst = imm */ | 1234 | case BPF_ALU | BPF_MOV | BPF_K: /* (u32) dst = imm */ |
| 1136 | case BPF_ALU64 | BPF_MOV | BPF_K: /* dst = (s64) imm */ | 1235 | case BPF_ALU64 | BPF_MOV | BPF_K: /* dst = (s64) imm */ |
| @@ -1598,6 +1697,12 @@ emit_clear: · 461 unmodified lines | |||
| 1598 | if (ret < 0) | 1697 | if (ret < 0) |
| 1599 | return ret; | 1698 | return ret; |
| 1600 | 1699 | ||
| 1700 | /* Take care of powerpc ABI requirements before kfunc call */ | ||
| 1701 | if (insn[i].src_reg == BPF_PSEUDO_KFUNC_CALL) { | ||
| 1702 | if (prepare_for_kfunc_call(fp, image, ctx, &insn[i])) | ||
| 1703 | return -1; | ||
| 1704 | } | ||
| 1705 | |||
| 1601 | ret = bpf_jit_emit_func_call_rel(image, fimage, ctx, func_addr); | 1706 | ret = bpf_jit_emit_func_call_rel(image, fimage, ctx, func_addr); |
| 1602 | if (ret) | 1707 | if (ret) |
| 1603 | return ret; | 1708 | return ret; |
| @@ -103,6 +103,11 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re · 102 unmodified lines | |||
| 103 | void | 103 | void |
| 104 | perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) | 104 | perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) |
| 105 | { | 105 | { |
| 106 | perf_callchain_store(entry, perf_arch_instruction_pointer(regs)); | ||
| 107 | |||
| 108 | if (!current->mm) | ||
| 109 | return; | ||
| 110 | |||
| 106 | if (!is_32bit_task()) | 111 | if (!is_32bit_task()) |
| 107 | perf_callchain_user_64(entry, regs); | 112 | perf_callchain_user_64(entry, regs); |
| 108 | else | 113 | else |
| @@ -142,7 +142,6 @@ void perf_callchain_user_32(struct perf_callchain_entry_ctx *entry, · 141 unmodified lines | |||
| 142 | next_ip = perf_arch_instruction_pointer(regs); | 142 | next_ip = perf_arch_instruction_pointer(regs); |
| 143 | lr = regs->link; | 143 | lr = regs->link; |
| 144 | sp = regs->gpr[1]; | 144 | sp = regs->gpr[1]; |
| 145 | perf_callchain_store(entry, next_ip); | ||
| 146 | 145 | ||
| 147 | while (entry->nr < entry->max_stack) { | 146 | while (entry->nr < entry->max_stack) { |
| 148 | fp = (unsigned int __user *) (unsigned long) sp; | 147 | fp = (unsigned int __user *) (unsigned long) sp; |
| @@ -77,7 +77,6 @@ void perf_callchain_user_64(struct perf_callchain_entry_ctx *entry, · 76 unmodified lines | |||
| 77 | next_ip = perf_arch_instruction_pointer(regs); | 77 | next_ip = perf_arch_instruction_pointer(regs); |
| 78 | lr = regs->link; | 78 | lr = regs->link; |
| 79 | sp = regs->gpr[1]; | 79 | sp = regs->gpr[1]; |
| 80 | perf_callchain_store(entry, next_ip); | ||
| 81 | 80 | ||
| 82 | while (entry->nr < entry->max_stack) { | 81 | while (entry->nr < entry->max_stack) { |
| 83 | fp = (unsigned long __user *) sp; | 82 | fp = (unsigned long __user *) sp; |
| @@ -155,8 +155,8 @@ machine_device_initcall(mpc83xx_km, mpc83xx_declare_of_platform_devices); · 154 unmodified lines | |||
| 155 | 155 | ||
| 156 | /* list of the supported boards */ | 156 | /* list of the supported boards */ |
| 157 | static char *board[] __initdata = { | 157 | static char *board[] __initdata = { |
| 158 | "Keymile,KMETER1", | 158 | "keymile,KMETER1", |
| 159 | "Keymile,kmpbec8321", | 159 | "keymile,kmpbec8321", |
| 160 | NULL | 160 | NULL |
| 161 | }; | 161 | }; |
| 162 | 162 | ||
| @@ -276,7 +276,7 @@ config PPC_BOOK3S · 275 unmodified lines | |||
| 276 | config PPC_E500 | 276 | config PPC_E500 |
| 277 | select FSL_EMB_PERFMON | 277 | select FSL_EMB_PERFMON |
| 278 | bool | 278 | bool |
| 279 | select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 | 279 | select ARCH_SUPPORTS_HUGETLBFS |
| 280 | select PPC_SMP_MUXED_IPI | 280 | select PPC_SMP_MUXED_IPI |
| 281 | select PPC_DOORBELL | 281 | select PPC_DOORBELL |
| 282 | select PPC_KUEP | 282 | select PPC_KUEP |
| @@ -337,7 +337,7 @@ config BOOKE · 54 unmodified lines | |||
| 337 | config PTE_64BIT | 337 | config PTE_64BIT |
| 338 | bool | 338 | bool |
| 339 | depends on 44x || PPC_E500 || PPC_86xx | 339 | depends on 44x || PPC_E500 || PPC_86xx |
| 340 | default y if PHYS_64BIT | 340 | default y if PPC_E500 || PHYS_64BIT |
| 341 | 341 | ||
| 342 | config PHYS_64BIT | 342 | config PHYS_64BIT |
| 343 | bool 'Large physical address support' if PPC_E500 || PPC_86xx | 343 | bool 'Large physical address support' if PPC_E500 || PPC_86xx |
| @@ -605,7 +605,7 @@ static int pseries_irq_domain_alloc(struct irq_domain *domain, unsigned int virq · 604 unmodified lines | |||
| 605 | &pseries_msi_irq_chip, pseries_dev); | 605 | &pseries_msi_irq_chip, pseries_dev); |
| 606 | } | 606 | } |
| 607 | 607 | ||
| 608 | pseries_dev->msi_used++; | 608 | pseries_dev->msi_used += nr_irqs; |
| 609 | return 0; | 609 | return 0; |
| 610 | 610 | ||
| 611 | out: | 611 | out: |
File renamed without changes.
| @@ -15,9 +15,9 @@ if [ -z "$is_64bit" ]; then · 14 unmodified lines | |||
| 15 | RELOCATION=R_PPC_ADDR32 | 15 | RELOCATION=R_PPC_ADDR32 |
| 16 | fi | 16 | fi |
| 17 | 17 | ||
| 18 | num_ool_stubs_total=$($objdump -r -j __patchable_function_entries "$vmlinux_o" | | 18 | num_ool_stubs_total=$($objdump -r -j __patchable_function_entries -d "$vmlinux_o" | |
| 19 | grep -c "$RELOCATION") | 19 | grep -c "$RELOCATION") |
| 20 | num_ool_stubs_inittext=$($objdump -r -j __patchable_function_entries "$vmlinux_o" | | 20 | num_ool_stubs_inittext=$($objdump -r -j __patchable_function_entries -d "$vmlinux_o" | |
| 21 | grep -e ".init.text" -e ".text.startup" | grep -c "$RELOCATION") | 21 | grep -e ".init.text" -e ".text.startup" | grep -c "$RELOCATION") |
| 22 | num_ool_stubs_text=$((num_ool_stubs_total - num_ool_stubs_inittext)) | 22 | num_ool_stubs_text=$((num_ool_stubs_total - num_ool_stubs_inittext)) |
| 23 | 23 | ||
| @@ -170,6 +170,7 @@ SECTIONS · 169 unmodified lines | |||
| 170 | 170 | ||
| 171 | STABS_DEBUG | 171 | STABS_DEBUG |
| 172 | DWARF_DEBUG | 172 | DWARF_DEBUG |
| 173 | MODINFO | ||
| 173 | ELF_DETAILS | 174 | ELF_DETAILS |
| 174 | .riscv.attributes 0 : { *(.riscv.attributes) } | 175 | .riscv.attributes 0 : { *(.riscv.attributes) } |
| 175 | 176 | ||
| @@ -30,7 +30,6 @@ config KVM · 29 unmodified lines | |||
| 30 | select KVM_GENERIC_HARDWARE_ENABLING | 30 | select KVM_GENERIC_HARDWARE_ENABLING |
| 31 | select KVM_MMIO | 31 | select KVM_MMIO |
| 32 | select VIRT_XFER_TO_GUEST_WORK | 32 | select VIRT_XFER_TO_GUEST_WORK |
| 33 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 34 | select SCHED_INFO | 33 | select SCHED_INFO |
| 35 | select GUEST_PERF_EVENTS if PERF_EVENTS | 34 | select GUEST_PERF_EVENTS if PERF_EVENTS |
| 36 | help | 35 | help |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | |||
| 13 | #include <linux/irqchip/riscv-imsic.h> | 13 | #include <linux/irqchip/riscv-imsic.h> |
| 14 | #include <linux/irqdomain.h> | 14 | #include <linux/irqdomain.h> |
| 15 | #include <linux/kvm_host.h> | 15 | #include <linux/kvm_host.h> |
| 16 | #include <linux/nospec.h> | ||
| 16 | #include <linux/percpu.h> | 17 | #include <linux/percpu.h> |
| 17 | #include <linux/spinlock.h> | 18 | #include <linux/spinlock.h> |
| 18 | #include <asm/cpufeature.h> | 19 | #include <asm/cpufeature.h> |
| @@ -182,9 +183,14 @@ int kvm_riscv_vcpu_aia_get_csr(struct kvm_vcpu *vcpu, · 163 unmodified lines | |||
| 182 | unsigned long *out_val) | 183 | unsigned long *out_val) |
| 183 | { | 184 | { |
| 184 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; | 185 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; |
| 186 | unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long); | ||
| 185 | 187 | ||
| 186 | if (reg_num >= sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long)) | 188 | if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA)) |
| 187 | return -ENOENT; | 189 | return -ENOENT; |
| 190 | if (reg_num >= regs_max) | ||
| 191 | return -ENOENT; | ||
| 192 | |||
| 193 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 188 | 194 | ||
| 189 | *out_val = 0; | 195 | *out_val = 0; |
| 190 | if (kvm_riscv_aia_available()) | 196 | if (kvm_riscv_aia_available()) |
| @@ -198,9 +204,14 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu, · 7 unmodified lines | |||
| 198 | unsigned long val) | 204 | unsigned long val) |
| 199 | { | 205 | { |
| 200 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; | 206 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; |
| 207 | unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long); | ||
| 201 | 208 | ||
| 202 | if (reg_num >= sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long)) | 209 | if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA)) |
| 203 | return -ENOENT; | 210 | return -ENOENT; |
| 211 | if (reg_num >= regs_max) | ||
| 212 | return -ENOENT; | ||
| 213 | |||
| 214 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 204 | 215 | ||
| 205 | if (kvm_riscv_aia_available()) { | 216 | if (kvm_riscv_aia_available()) { |
| 206 | ((unsigned long *)csr)[reg_num] = val; | 217 | ((unsigned long *)csr)[reg_num] = val; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | |||
| 10 | #include <linux/irqchip/riscv-aplic.h> | 10 | #include <linux/irqchip/riscv-aplic.h> |
| 11 | #include <linux/kvm_host.h> | 11 | #include <linux/kvm_host.h> |
| 12 | #include <linux/math.h> | 12 | #include <linux/math.h> |
| 13 | #include <linux/nospec.h> | ||
| 13 | #include <linux/spinlock.h> | 14 | #include <linux/spinlock.h> |
| 14 | #include <linux/swab.h> | 15 | #include <linux/swab.h> |
| 15 | #include <kvm/iodev.h> | 16 | #include <kvm/iodev.h> |
| @@ -45,7 +46,7 @@ static u32 aplic_read_sourcecfg(struct aplic *aplic, u32 irq) · 29 unmodified lines | |||
| 45 | 46 | ||
| 46 | if (!irq || aplic->nr_irqs <= irq) | 47 | if (!irq || aplic->nr_irqs <= irq) |
| 47 | return 0; | 48 | return 0; |
| 48 | irqd = &aplic->irqs[irq]; | 49 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 49 | 50 | ||
| 50 | raw_spin_lock_irqsave(&irqd->lock, flags); | 51 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 51 | ret = irqd->sourcecfg; | 52 | ret = irqd->sourcecfg; |
| @@ -61,7 +62,7 @@ static void aplic_write_sourcecfg(struct aplic *aplic, u32 irq, u32 val) · 9 unmodified lines | |||
| 61 | 62 | ||
| 62 | if (!irq || aplic->nr_irqs <= irq) | 63 | if (!irq || aplic->nr_irqs <= irq) |
| 63 | return; | 64 | return; |
| 64 | irqd = &aplic->irqs[irq]; | 65 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 65 | 66 | ||
| 66 | if (val & APLIC_SOURCECFG_D) | 67 | if (val & APLIC_SOURCECFG_D) |
| 67 | val = 0; | 68 | val = 0; |
| @@ -81,7 +82,7 @@ static u32 aplic_read_target(struct aplic *aplic, u32 irq) · 13 unmodified lines | |||
| 81 | 82 | ||
| 82 | if (!irq || aplic->nr_irqs <= irq) | 83 | if (!irq || aplic->nr_irqs <= irq) |
| 83 | return 0; | 84 | return 0; |
| 84 | irqd = &aplic->irqs[irq]; | 85 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 85 | 86 | ||
| 86 | raw_spin_lock_irqsave(&irqd->lock, flags); | 87 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 87 | ret = irqd->target; | 88 | ret = irqd->target; |
| @@ -97,7 +98,7 @@ static void aplic_write_target(struct aplic *aplic, u32 irq, u32 val) · 9 unmodified lines | |||
| 97 | 98 | ||
| 98 | if (!irq || aplic->nr_irqs <= irq) | 99 | if (!irq || aplic->nr_irqs <= irq) |
| 99 | return; | 100 | return; |
| 100 | irqd = &aplic->irqs[irq]; | 101 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 101 | 102 | ||
| 102 | val &= APLIC_TARGET_EIID_MASK | | 103 | val &= APLIC_TARGET_EIID_MASK | |
| 103 | (APLIC_TARGET_HART_IDX_MASK << APLIC_TARGET_HART_IDX_SHIFT) | | 104 | (APLIC_TARGET_HART_IDX_MASK << APLIC_TARGET_HART_IDX_SHIFT) | |
| @@ -116,7 +117,7 @@ static bool aplic_read_pending(struct aplic *aplic, u32 irq) · 12 unmodified lines | |||
| 116 | 117 | ||
| 117 | if (!irq || aplic->nr_irqs <= irq) | 118 | if (!irq || aplic->nr_irqs <= irq) |
| 118 | return false; | 119 | return false; |
| 119 | irqd = &aplic->irqs[irq]; | 120 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 120 | 121 | ||
| 121 | raw_spin_lock_irqsave(&irqd->lock, flags); | 122 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 122 | ret = (irqd->state & APLIC_IRQ_STATE_PENDING) ? true : false; | 123 | ret = (irqd->state & APLIC_IRQ_STATE_PENDING) ? true : false; |
| @@ -132,7 +133,7 @@ static void aplic_write_pending(struct aplic *aplic, u32 irq, bool pending) · 9 unmodified lines | |||
| 132 | 133 | ||
| 133 | if (!irq || aplic->nr_irqs <= irq) | 134 | if (!irq || aplic->nr_irqs <= irq) |
| 134 | return; | 135 | return; |
| 135 | irqd = &aplic->irqs[irq]; | 136 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 136 | 137 | ||
| 137 | raw_spin_lock_irqsave(&irqd->lock, flags); | 138 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 138 | 139 | ||
| @@ -170,7 +171,7 @@ static bool aplic_read_enabled(struct aplic *aplic, u32 irq) · 31 unmodified lines | |||
| 170 | 171 | ||
| 171 | if (!irq || aplic->nr_irqs <= irq) | 172 | if (!irq || aplic->nr_irqs <= irq) |
| 172 | return false; | 173 | return false; |
| 173 | irqd = &aplic->irqs[irq]; | 174 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 174 | 175 | ||
| 175 | raw_spin_lock_irqsave(&irqd->lock, flags); | 176 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 176 | ret = (irqd->state & APLIC_IRQ_STATE_ENABLED) ? true : false; | 177 | ret = (irqd->state & APLIC_IRQ_STATE_ENABLED) ? true : false; |
| @@ -186,7 +187,7 @@ static void aplic_write_enabled(struct aplic *aplic, u32 irq, bool enabled) · 9 unmodified lines | |||
| 186 | 187 | ||
| 187 | if (!irq || aplic->nr_irqs <= irq) | 188 | if (!irq || aplic->nr_irqs <= irq) |
| 188 | return; | 189 | return; |
| 189 | irqd = &aplic->irqs[irq]; | 190 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 190 | 191 | ||
| 191 | raw_spin_lock_irqsave(&irqd->lock, flags); | 192 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 192 | if (enabled) | 193 | if (enabled) |
| @@ -205,7 +206,7 @@ static bool aplic_read_input(struct aplic *aplic, u32 irq) · 12 unmodified lines | |||
| 205 | 206 | ||
| 206 | if (!irq || aplic->nr_irqs <= irq) | 207 | if (!irq || aplic->nr_irqs <= irq) |
| 207 | return false; | 208 | return false; |
| 208 | irqd = &aplic->irqs[irq]; | 209 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 209 | 210 | ||
| 210 | raw_spin_lock_irqsave(&irqd->lock, flags); | 211 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 211 | 212 | ||
| @@ -254,7 +255,7 @@ static void aplic_update_irq_range(struct kvm *kvm, u32 first, u32 last) · 42 unmodified lines | |||
| 254 | for (irq = first; irq <= last; irq++) { | 255 | for (irq = first; irq <= last; irq++) { |
| 255 | if (!irq || aplic->nr_irqs <= irq) | 256 | if (!irq || aplic->nr_irqs <= irq) |
| 256 | continue; | 257 | continue; |
| 257 | irqd = &aplic->irqs[irq]; | 258 | irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; |
| 258 | 259 | ||
| 259 | raw_spin_lock_irqsave(&irqd->lock, flags); | 260 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 260 | 261 | ||
| @@ -283,7 +284,7 @@ int kvm_riscv_aia_aplic_inject(struct kvm *kvm, u32 source, bool level) · 22 unmodified lines | |||
| 283 | 284 | ||
| 284 | if (!aplic || !source || (aplic->nr_irqs <= source)) | 285 | if (!aplic || !source || (aplic->nr_irqs <= source)) |
| 285 | return -ENODEV; | 286 | return -ENODEV; |
| 286 | irqd = &aplic->irqs[source]; | 287 | irqd = &aplic->irqs[array_index_nospec(source, aplic->nr_irqs)]; |
| 287 | ie = (aplic->domaincfg & APLIC_DOMAINCFG_IE) ? true : false; | 288 | ie = (aplic->domaincfg & APLIC_DOMAINCFG_IE) ? true : false; |
| 288 | 289 | ||
| 289 | raw_spin_lock_irqsave(&irqd->lock, flags); | 290 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | |||
| 11 | #include <linux/irqchip/riscv-imsic.h> | 11 | #include <linux/irqchip/riscv-imsic.h> |
| 12 | #include <linux/kvm_host.h> | 12 | #include <linux/kvm_host.h> |
| 13 | #include <linux/uaccess.h> | 13 | #include <linux/uaccess.h> |
| 14 | #include <linux/cpufeature.h> | ||
| 14 | 15 | ||
| 15 | static int aia_create(struct kvm_device *dev, u32 type) | 16 | static int aia_create(struct kvm_device *dev, u32 type) |
| 16 | { | 17 | { |
| @@ -22,6 +23,9 @@ static int aia_create(struct kvm_device *dev, u32 type) · 5 unmodified lines | |||
| 22 | if (irqchip_in_kernel(kvm)) | 23 | if (irqchip_in_kernel(kvm)) |
| 23 | return -EEXIST; | 24 | return -EEXIST; |
| 24 | 25 | ||
| 26 | if (!riscv_isa_extension_available(NULL, SSAIA)) | ||
| 27 | return -ENODEV; | ||
| 28 | |||
| 25 | ret = -EBUSY; | 29 | ret = -EBUSY; |
| 26 | if (kvm_trylock_all_vcpus(kvm)) | 30 | if (kvm_trylock_all_vcpus(kvm)) |
| 27 | return ret; | 31 | return ret; |
| @@ -437,7 +441,7 @@ static int aia_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr) · 409 unmodified lines | |||
| 437 | 441 | ||
| 438 | static int aia_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr) | 442 | static int aia_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr) |
| 439 | { | 443 | { |
| 440 | int nr_vcpus; | 444 | int nr_vcpus, r = -ENXIO; |
| 441 | 445 | ||
| 442 | switch (attr->group) { | 446 | switch (attr->group) { |
| 443 | case KVM_DEV_RISCV_AIA_GRP_CONFIG: | 447 | case KVM_DEV_RISCV_AIA_GRP_CONFIG: |
| @@ -466,12 +470,18 @@ static int aia_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr) · 22 unmodified lines | |||
| 466 | } | 470 | } |
| 467 | break; | 471 | break; |
| 468 | case KVM_DEV_RISCV_AIA_GRP_APLIC: | 472 | case KVM_DEV_RISCV_AIA_GRP_APLIC: |
| 469 | return kvm_riscv_aia_aplic_has_attr(dev->kvm, attr->attr); | 473 | mutex_lock(&dev->kvm->lock); |
| 474 | r = kvm_riscv_aia_aplic_has_attr(dev->kvm, attr->attr); | ||
| 475 | mutex_unlock(&dev->kvm->lock); | ||
| 476 | break; | ||
| 470 | case KVM_DEV_RISCV_AIA_GRP_IMSIC: | 477 | case KVM_DEV_RISCV_AIA_GRP_IMSIC: |
| 471 | return kvm_riscv_aia_imsic_has_attr(dev->kvm, attr->attr); | 478 | mutex_lock(&dev->kvm->lock); |
| 479 | r = kvm_riscv_aia_imsic_has_attr(dev->kvm, attr->attr); | ||
| 480 | mutex_unlock(&dev->kvm->lock); | ||
| 481 | break; | ||
| 472 | } | 482 | } |
| 473 | 483 | ||
| 474 | return -ENXIO; | 484 | return r; |
| 475 | } | 485 | } |
| 476 | 486 | ||
| 477 | struct kvm_device_ops kvm_riscv_aia_device_ops = { | 487 | struct kvm_device_ops kvm_riscv_aia_device_ops = { |
| @@ -908,6 +908,10 @@ int kvm_riscv_vcpu_aia_imsic_rmw(struct kvm_vcpu *vcpu, unsigned long isel, · 907 unmodified lines | |||
| 908 | int r, rc = KVM_INSN_CONTINUE_NEXT_SEPC; | 908 | int r, rc = KVM_INSN_CONTINUE_NEXT_SEPC; |
| 909 | struct imsic *imsic = vcpu->arch.aia_context.imsic_state; | 909 | struct imsic *imsic = vcpu->arch.aia_context.imsic_state; |
| 910 | 910 | ||
| 911 | /* If IMSIC vCPU state not initialized then forward to user space */ | ||
| 912 | if (!imsic) | ||
| 913 | return KVM_INSN_EXIT_TO_USER_SPACE; | ||
| 914 | |||
| 911 | if (isel == KVM_RISCV_AIA_IMSIC_TOPEI) { | 915 | if (isel == KVM_RISCV_AIA_IMSIC_TOPEI) { |
| 912 | /* Read pending and enabled interrupt with highest priority */ | 916 | /* Read pending and enabled interrupt with highest priority */ |
| 913 | topei = imsic_mrif_topei(imsic->swfile, imsic->nr_eix, | 917 | topei = imsic_mrif_topei(imsic->swfile, imsic->nr_eix, |
| @@ -245,6 +245,7 @@ out: · 244 unmodified lines | |||
| 245 | bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) | 245 | bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) |
| 246 | { | 246 | { |
| 247 | struct kvm_gstage gstage; | 247 | struct kvm_gstage gstage; |
| 248 | bool mmu_locked; | ||
| 248 | 249 | ||
| 249 | if (!kvm->arch.pgd) | 250 | if (!kvm->arch.pgd) |
| 250 | return false; | 251 | return false; |
| @@ -253,9 +254,12 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) · 2 unmodified lines | |||
| 253 | gstage.flags = 0; | 254 | gstage.flags = 0; |
| 254 | gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid); | 255 | gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid); |
| 255 | gstage.pgd = kvm->arch.pgd; | 256 | gstage.pgd = kvm->arch.pgd; |
| 257 | mmu_locked = spin_trylock(&kvm->mmu_lock); | ||
| 256 | kvm_riscv_gstage_unmap_range(&gstage, range->start << PAGE_SHIFT, | 258 | kvm_riscv_gstage_unmap_range(&gstage, range->start << PAGE_SHIFT, |
| 257 | (range->end - range->start) << PAGE_SHIFT, | 259 | (range->end - range->start) << PAGE_SHIFT, |
| 258 | range->may_block); | 260 | range->may_block); |
| 261 | if (mmu_locked) | ||
| 262 | spin_unlock(&kvm->mmu_lock); | ||
| 259 | return false; | 263 | return false; |
| 260 | } | 264 | } |
| 261 | 265 | ||
| @@ -535,7 +539,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot, · 273 unmodified lines | |||
| 535 | goto out_unlock; | 539 | goto out_unlock; |
| 536 | 540 | ||
| 537 | /* Check if we are backed by a THP and thus use block mapping if possible */ | 541 | /* Check if we are backed by a THP and thus use block mapping if possible */ |
| 538 | if (vma_pagesize == PAGE_SIZE) | 542 | if (!logging && (vma_pagesize == PAGE_SIZE)) |
| 539 | vma_pagesize = transparent_hugepage_adjust(kvm, memslot, hva, &hfn, &gpa); | 543 | vma_pagesize = transparent_hugepage_adjust(kvm, memslot, hva, &hfn, &gpa); |
| 540 | 544 | ||
| 541 | if (writable) { | 545 | if (writable) { |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | |||
| 24 | #define CREATE_TRACE_POINTS | 24 | #define CREATE_TRACE_POINTS |
| 25 | #include "trace.h" | 25 | #include "trace.h" |
| 26 | 26 | ||
| 27 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 27 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 28 | KVM_GENERIC_VCPU_STATS(), | 28 | KVM_GENERIC_VCPU_STATS(), |
| 29 | STATS_DESC_COUNTER(VCPU, ecall_exit_stat), | 29 | STATS_DESC_COUNTER(VCPU, ecall_exit_stat), |
| 30 | STATS_DESC_COUNTER(VCPU, wfi_exit_stat), | 30 | STATS_DESC_COUNTER(VCPU, wfi_exit_stat), |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | |||
| 10 | #include <linux/errno.h> | 10 | #include <linux/errno.h> |
| 11 | #include <linux/err.h> | 11 | #include <linux/err.h> |
| 12 | #include <linux/kvm_host.h> | 12 | #include <linux/kvm_host.h> |
| 13 | #include <linux/nospec.h> | ||
| 13 | #include <linux/uaccess.h> | 14 | #include <linux/uaccess.h> |
| 14 | #include <asm/cpufeature.h> | 15 | #include <asm/cpufeature.h> |
| 15 | 16 | ||
| @@ -93,9 +94,11 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu, · 77 unmodified lines | |||
| 93 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) | 94 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) |
| 94 | reg_val = &cntx->fp.f.fcsr; | 95 | reg_val = &cntx->fp.f.fcsr; |
| 95 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && | 96 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && |
| 96 | reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) | 97 | reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) { |
| 98 | reg_num = array_index_nospec(reg_num, | ||
| 99 | ARRAY_SIZE(cntx->fp.f.f)); | ||
| 97 | reg_val = &cntx->fp.f.f[reg_num]; | 100 | reg_val = &cntx->fp.f.f[reg_num]; |
| 98 | else | 101 | } else |
| 99 | return -ENOENT; | 102 | return -ENOENT; |
| 100 | } else if ((rtype == KVM_REG_RISCV_FP_D) && | 103 | } else if ((rtype == KVM_REG_RISCV_FP_D) && |
| 101 | riscv_isa_extension_available(vcpu->arch.isa, d)) { | 104 | riscv_isa_extension_available(vcpu->arch.isa, d)) { |
| @@ -107,6 +110,8 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu, · 5 unmodified lines | |||
| 107 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { | 110 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { |
| 108 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) | 111 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) |
| 109 | return -EINVAL; | 112 | return -EINVAL; |
| 113 | reg_num = array_index_nospec(reg_num, | ||
| 114 | ARRAY_SIZE(cntx->fp.d.f)); | ||
| 110 | reg_val = &cntx->fp.d.f[reg_num]; | 115 | reg_val = &cntx->fp.d.f[reg_num]; |
| 111 | } else | 116 | } else |
| 112 | return -ENOENT; | 117 | return -ENOENT; |
| @@ -138,9 +143,11 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu, · 25 unmodified lines | |||
| 138 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) | 143 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) |
| 139 | reg_val = &cntx->fp.f.fcsr; | 144 | reg_val = &cntx->fp.f.fcsr; |
| 140 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && | 145 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && |
| 141 | reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) | 146 | reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) { |
| 147 | reg_num = array_index_nospec(reg_num, | ||
| 148 | ARRAY_SIZE(cntx->fp.f.f)); | ||
| 142 | reg_val = &cntx->fp.f.f[reg_num]; | 149 | reg_val = &cntx->fp.f.f[reg_num]; |
| 143 | else | 150 | } else |
| 144 | return -ENOENT; | 151 | return -ENOENT; |
| 145 | } else if ((rtype == KVM_REG_RISCV_FP_D) && | 152 | } else if ((rtype == KVM_REG_RISCV_FP_D) && |
| 146 | riscv_isa_extension_available(vcpu->arch.isa, d)) { | 153 | riscv_isa_extension_available(vcpu->arch.isa, d)) { |
| @@ -152,6 +159,8 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu, · 5 unmodified lines | |||
| 152 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { | 159 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { |
| 153 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) | 160 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) |
| 154 | return -EINVAL; | 161 | return -EINVAL; |
| 162 | reg_num = array_index_nospec(reg_num, | ||
| 163 | ARRAY_SIZE(cntx->fp.d.f)); | ||
| 155 | reg_val = &cntx->fp.d.f[reg_num]; | 164 | reg_val = &cntx->fp.d.f[reg_num]; |
| 156 | } else | 165 | } else |
| 157 | return -ENOENT; | 166 | return -ENOENT; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | |||
| 10 | #include <linux/bitops.h> | 10 | #include <linux/bitops.h> |
| 11 | #include <linux/errno.h> | 11 | #include <linux/errno.h> |
| 12 | #include <linux/err.h> | 12 | #include <linux/err.h> |
| 13 | #include <linux/nospec.h> | ||
| 13 | #include <linux/uaccess.h> | 14 | #include <linux/uaccess.h> |
| 14 | #include <linux/kvm_host.h> | 15 | #include <linux/kvm_host.h> |
| 15 | #include <asm/cacheflush.h> | 16 | #include <asm/cacheflush.h> |
| @@ -127,6 +128,7 @@ static int kvm_riscv_vcpu_isa_check_host(unsigned long kvm_ext, unsigned long *g · 111 unmodified lines | |||
| 127 | kvm_ext >= ARRAY_SIZE(kvm_isa_ext_arr)) | 128 | kvm_ext >= ARRAY_SIZE(kvm_isa_ext_arr)) |
| 128 | return -ENOENT; | 129 | return -ENOENT; |
| 129 | 130 | ||
| 131 | kvm_ext = array_index_nospec(kvm_ext, ARRAY_SIZE(kvm_isa_ext_arr)); | ||
| 130 | *guest_ext = kvm_isa_ext_arr[kvm_ext]; | 132 | *guest_ext = kvm_isa_ext_arr[kvm_ext]; |
| 131 | switch (*guest_ext) { | 133 | switch (*guest_ext) { |
| 132 | case RISCV_ISA_EXT_SMNPM: | 134 | case RISCV_ISA_EXT_SMNPM: |
| @@ -443,13 +445,16 @@ static int kvm_riscv_vcpu_get_reg_core(struct kvm_vcpu *vcpu, · 310 unmodified lines | |||
| 443 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | | 445 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | |
| 444 | KVM_REG_SIZE_MASK | | 446 | KVM_REG_SIZE_MASK | |
| 445 | KVM_REG_RISCV_CORE); | 447 | KVM_REG_RISCV_CORE); |
| 448 | unsigned long regs_max = sizeof(struct kvm_riscv_core) / sizeof(unsigned long); | ||
| 446 | unsigned long reg_val; | 449 | unsigned long reg_val; |
| 447 | 450 | ||
| 448 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) | 451 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) |
| 449 | return -EINVAL; | 452 | return -EINVAL; |
| 450 | if (reg_num >= sizeof(struct kvm_riscv_core) / sizeof(unsigned long)) | 453 | if (reg_num >= regs_max) |
| 451 | return -ENOENT; | 454 | return -ENOENT; |
| 452 | 455 | ||
| 456 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 457 | |||
| 453 | if (reg_num == KVM_REG_RISCV_CORE_REG(regs.pc)) | 458 | if (reg_num == KVM_REG_RISCV_CORE_REG(regs.pc)) |
| 454 | reg_val = cntx->sepc; | 459 | reg_val = cntx->sepc; |
| 455 | else if (KVM_REG_RISCV_CORE_REG(regs.pc) < reg_num && | 460 | else if (KVM_REG_RISCV_CORE_REG(regs.pc) < reg_num && |
| @@ -476,13 +481,16 @@ static int kvm_riscv_vcpu_set_reg_core(struct kvm_vcpu *vcpu, · 20 unmodified lines | |||
| 476 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | | 481 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | |
| 477 | KVM_REG_SIZE_MASK | | 482 | KVM_REG_SIZE_MASK | |
| 478 | KVM_REG_RISCV_CORE); | 483 | KVM_REG_RISCV_CORE); |
| 484 | unsigned long regs_max = sizeof(struct kvm_riscv_core) / sizeof(unsigned long); | ||
| 479 | unsigned long reg_val; | 485 | unsigned long reg_val; |
| 480 | 486 | ||
| 481 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) | 487 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) |
| 482 | return -EINVAL; | 488 | return -EINVAL; |
| 483 | if (reg_num >= sizeof(struct kvm_riscv_core) / sizeof(unsigned long)) | 489 | if (reg_num >= regs_max) |
| 484 | return -ENOENT; | 490 | return -ENOENT; |
| 485 | 491 | ||
| 492 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 493 | |||
| 486 | if (copy_from_user(®_val, uaddr, KVM_REG_SIZE(reg->id))) | 494 | if (copy_from_user(®_val, uaddr, KVM_REG_SIZE(reg->id))) |
| 487 | return -EFAULT; | 495 | return -EFAULT; |
| 488 | 496 | ||
| @@ -507,10 +515,13 @@ static int kvm_riscv_vcpu_general_get_csr(struct kvm_vcpu *vcpu, · 18 unmodified lines | |||
| 507 | unsigned long *out_val) | 515 | unsigned long *out_val) |
| 508 | { | 516 | { |
| 509 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; | 517 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; |
| 518 | unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long); | ||
| 510 | 519 | ||
| 511 | if (reg_num >= sizeof(struct kvm_riscv_csr) / sizeof(unsigned long)) | 520 | if (reg_num >= regs_max) |
| 512 | return -ENOENT; | 521 | return -ENOENT; |
| 513 | 522 | ||
| 523 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 524 | |||
| 514 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { | 525 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { |
| 515 | kvm_riscv_vcpu_flush_interrupts(vcpu); | 526 | kvm_riscv_vcpu_flush_interrupts(vcpu); |
| 516 | *out_val = (csr->hvip >> VSIP_TO_HVIP_SHIFT) & VSIP_VALID_MASK; | 527 | *out_val = (csr->hvip >> VSIP_TO_HVIP_SHIFT) & VSIP_VALID_MASK; |
| @@ -526,10 +537,13 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu, · 9 unmodified lines | |||
| 526 | unsigned long reg_val) | 537 | unsigned long reg_val) |
| 527 | { | 538 | { |
| 528 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; | 539 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; |
| 540 | unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long); | ||
| 529 | 541 | ||
| 530 | if (reg_num >= sizeof(struct kvm_riscv_csr) / sizeof(unsigned long)) | 542 | if (reg_num >= regs_max) |
| 531 | return -ENOENT; | 543 | return -ENOENT; |
| 532 | 544 | ||
| 545 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 546 | |||
| 533 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { | 547 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { |
| 534 | reg_val &= VSIP_VALID_MASK; | 548 | reg_val &= VSIP_VALID_MASK; |
| 535 | reg_val <<= VSIP_TO_HVIP_SHIFT; | 549 | reg_val <<= VSIP_TO_HVIP_SHIFT; |
| @@ -548,10 +562,15 @@ static inline int kvm_riscv_vcpu_smstateen_set_csr(struct kvm_vcpu *vcpu, · 12 unmodified lines | |||
| 548 | unsigned long reg_val) | 562 | unsigned long reg_val) |
| 549 | { | 563 | { |
| 550 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; | 564 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; |
| 565 | unsigned long regs_max = sizeof(struct kvm_riscv_smstateen_csr) / | ||
| 566 | sizeof(unsigned long); | ||
| 551 | 567 | ||
| 552 | if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) / | 568 | if (!riscv_isa_extension_available(vcpu->arch.isa, SMSTATEEN)) |
| 553 | sizeof(unsigned long)) | 569 | return -ENOENT; |
| 554 | return -EINVAL; | 570 | if (reg_num >= regs_max) |
| 571 | return -ENOENT; | ||
| 572 | |||
| 573 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 555 | 574 | ||
| 556 | ((unsigned long *)csr)[reg_num] = reg_val; | 575 | ((unsigned long *)csr)[reg_num] = reg_val; |
| 557 | return 0; | 576 | return 0; |
| @@ -562,10 +581,15 @@ static int kvm_riscv_vcpu_smstateen_get_csr(struct kvm_vcpu *vcpu, · 4 unmodified lines | |||
| 562 | unsigned long *out_val) | 581 | unsigned long *out_val) |
| 563 | { | 582 | { |
| 564 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; | 583 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; |
| 584 | unsigned long regs_max = sizeof(struct kvm_riscv_smstateen_csr) / | ||
| 585 | sizeof(unsigned long); | ||
| 565 | 586 | ||
| 566 | if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) / | 587 | if (!riscv_isa_extension_available(vcpu->arch.isa, SMSTATEEN)) |
| 567 | sizeof(unsigned long)) | 588 | return -ENOENT; |
| 568 | return -EINVAL; | 589 | if (reg_num >= regs_max) |
| 590 | return -ENOENT; | ||
| 591 | |||
| 592 | reg_num = array_index_nospec(reg_num, regs_max); | ||
| 569 | 593 | ||
| 570 | *out_val = ((unsigned long *)csr)[reg_num]; | 594 | *out_val = ((unsigned long *)csr)[reg_num]; |
| 571 | return 0; | 595 | return 0; |
| @@ -595,10 +619,7 @@ static int kvm_riscv_vcpu_get_reg_csr(struct kvm_vcpu *vcpu, · 23 unmodified lines | |||
| 595 | rc = kvm_riscv_vcpu_aia_get_csr(vcpu, reg_num, ®_val); | 619 | rc = kvm_riscv_vcpu_aia_get_csr(vcpu, reg_num, ®_val); |
| 596 | break; | 620 | break; |
| 597 | case KVM_REG_RISCV_CSR_SMSTATEEN: | 621 | case KVM_REG_RISCV_CSR_SMSTATEEN: |
| 598 | rc = -EINVAL; | 622 | rc = kvm_riscv_vcpu_smstateen_get_csr(vcpu, reg_num, ®_val); |
| 599 | if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) | ||
| 600 | rc = kvm_riscv_vcpu_smstateen_get_csr(vcpu, reg_num, | ||
| 601 | ®_val); | ||
| 602 | break; | 623 | break; |
| 603 | default: | 624 | default: |
| 604 | rc = -ENOENT; | 625 | rc = -ENOENT; |
| @@ -640,10 +661,7 @@ static int kvm_riscv_vcpu_set_reg_csr(struct kvm_vcpu *vcpu, · 35 unmodified lines | |||
| 640 | rc = kvm_riscv_vcpu_aia_set_csr(vcpu, reg_num, reg_val); | 661 | rc = kvm_riscv_vcpu_aia_set_csr(vcpu, reg_num, reg_val); |
| 641 | break; | 662 | break; |
| 642 | case KVM_REG_RISCV_CSR_SMSTATEEN: | 663 | case KVM_REG_RISCV_CSR_SMSTATEEN: |
| 643 | rc = -EINVAL; | 664 | rc = kvm_riscv_vcpu_smstateen_set_csr(vcpu, reg_num, reg_val); |
| 644 | if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) | ||
| 645 | rc = kvm_riscv_vcpu_smstateen_set_csr(vcpu, reg_num, | ||
| 646 | reg_val); | ||
| 647 | break; | 665 | break; |
| 648 | default: | 666 | default: |
| 649 | rc = -ENOENT; | 667 | rc = -ENOENT; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | |||
| 10 | #include <linux/errno.h> | 10 | #include <linux/errno.h> |
| 11 | #include <linux/err.h> | 11 | #include <linux/err.h> |
| 12 | #include <linux/kvm_host.h> | 12 | #include <linux/kvm_host.h> |
| 13 | #include <linux/nospec.h> | ||
| 13 | #include <linux/perf/riscv_pmu.h> | 14 | #include <linux/perf/riscv_pmu.h> |
| 14 | #include <asm/csr.h> | 15 | #include <asm/csr.h> |
| 15 | #include <asm/kvm_vcpu_sbi.h> | 16 | #include <asm/kvm_vcpu_sbi.h> |
| @@ -87,7 +88,8 @@ static void kvm_pmu_release_perf_event(struct kvm_pmc *pmc) · 71 unmodified lines | |||
| 87 | 88 | ||
| 88 | static u64 kvm_pmu_get_perf_event_hw_config(u32 sbi_event_code) | 89 | static u64 kvm_pmu_get_perf_event_hw_config(u32 sbi_event_code) |
| 89 | { | 90 | { |
| 90 | return hw_event_perf_map[sbi_event_code]; | 91 | return hw_event_perf_map[array_index_nospec(sbi_event_code, |
| 92 | SBI_PMU_HW_GENERAL_MAX)]; | ||
| 91 | } | 93 | } |
| 92 | 94 | ||
| 93 | static u64 kvm_pmu_get_perf_event_cache_config(u32 sbi_event_code) | 95 | static u64 kvm_pmu_get_perf_event_cache_config(u32 sbi_event_code) |
| @@ -218,6 +220,7 @@ static int pmu_fw_ctr_read_hi(struct kvm_vcpu *vcpu, unsigned long cidx, · 124 unmodified lines | |||
| 218 | return -EINVAL; | 220 | return -EINVAL; |
| 219 | } | 221 | } |
| 220 | 222 | ||
| 223 | cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | ||
| 221 | pmc = &kvpmu->pmc[cidx]; | 224 | pmc = &kvpmu->pmc[cidx]; |
| 222 | 225 | ||
| 223 | if (pmc->cinfo.type != SBI_PMU_CTR_TYPE_FW) | 226 | if (pmc->cinfo.type != SBI_PMU_CTR_TYPE_FW) |
| @@ -244,6 +247,7 @@ static int pmu_ctr_read(struct kvm_vcpu *vcpu, unsigned long cidx, · 20 unmodified lines | |||
| 244 | return -EINVAL; | 247 | return -EINVAL; |
| 245 | } | 248 | } |
| 246 | 249 | ||
| 250 | cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | ||
| 247 | pmc = &kvpmu->pmc[cidx]; | 251 | pmc = &kvpmu->pmc[cidx]; |
| 248 | 252 | ||
| 249 | if (pmc->cinfo.type == SBI_PMU_CTR_TYPE_FW) { | 253 | if (pmc->cinfo.type == SBI_PMU_CTR_TYPE_FW) { |
| @@ -520,11 +524,12 @@ int kvm_riscv_vcpu_pmu_ctr_info(struct kvm_vcpu *vcpu, unsigned long cidx, · 270 unmodified lines | |||
| 520 | { | 524 | { |
| 521 | struct kvm_pmu *kvpmu = vcpu_to_pmu(vcpu); | 525 | struct kvm_pmu *kvpmu = vcpu_to_pmu(vcpu); |
| 522 | 526 | ||
| 523 | if (cidx > RISCV_KVM_MAX_COUNTERS || cidx == 1) { | 527 | if (cidx >= RISCV_KVM_MAX_COUNTERS || cidx == 1) { |
| 524 | retdata->err_val = SBI_ERR_INVALID_PARAM; | 528 | retdata->err_val = SBI_ERR_INVALID_PARAM; |
| 525 | return 0; | 529 | return 0; |
| 526 | } | 530 | } |
| 527 | 531 | ||
| 532 | cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | ||
| 528 | retdata->out_val = kvpmu->pmc[cidx].cinfo.value; | 533 | retdata->out_val = kvpmu->pmc[cidx].cinfo.value; |
| 529 | 534 | ||
| 530 | return 0; | 535 | return 0; |
| @@ -559,7 +564,8 @@ int kvm_riscv_vcpu_pmu_ctr_start(struct kvm_vcpu *vcpu, unsigned long ctr_base, · 28 unmodified lines | |||
| 559 | } | 564 | } |
| 560 | /* Start the counters that have been configured and requested by the guest */ | 565 | /* Start the counters that have been configured and requested by the guest */ |
| 561 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { | 566 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { |
| 562 | pmc_index = i + ctr_base; | 567 | pmc_index = array_index_nospec(i + ctr_base, |
| 568 | RISCV_KVM_MAX_COUNTERS); | ||
| 563 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) | 569 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) |
| 564 | continue; | 570 | continue; |
| 565 | /* The guest started the counter again. Reset the overflow status */ | 571 | /* The guest started the counter again. Reset the overflow status */ |
| @@ -630,7 +636,8 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base, · 64 unmodified lines | |||
| 630 | 636 | ||
| 631 | /* Stop the counters that have been configured and requested by the guest */ | 637 | /* Stop the counters that have been configured and requested by the guest */ |
| 632 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { | 638 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { |
| 633 | pmc_index = i + ctr_base; | 639 | pmc_index = array_index_nospec(i + ctr_base, |
| 640 | RISCV_KVM_MAX_COUNTERS); | ||
| 634 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) | 641 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) |
| 635 | continue; | 642 | continue; |
| 636 | pmc = &kvpmu->pmc[pmc_index]; | 643 | pmc = &kvpmu->pmc[pmc_index]; |
| @@ -761,6 +768,7 @@ int kvm_riscv_vcpu_pmu_ctr_cfg_match(struct kvm_vcpu *vcpu, unsigned long ctr_ba · 124 unmodified lines | |||
| 761 | } | 768 | } |
| 762 | } | 769 | } |
| 763 | 770 | ||
| 771 | ctr_idx = array_index_nospec(ctr_idx, RISCV_KVM_MAX_COUNTERS); | ||
| 764 | pmc = &kvpmu->pmc[ctr_idx]; | 772 | pmc = &kvpmu->pmc[ctr_idx]; |
| 765 | pmc->idx = ctr_idx; | 773 | pmc->idx = ctr_idx; |
| 766 | 774 | ||
| @@ -13,7 +13,7 @@ · 12 unmodified lines | |||
| 13 | #include <linux/kvm_host.h> | 13 | #include <linux/kvm_host.h> |
| 14 | #include <asm/kvm_mmu.h> | 14 | #include <asm/kvm_mmu.h> |
| 15 | 15 | ||
| 16 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 16 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 17 | KVM_GENERIC_VM_STATS() | 17 | KVM_GENERIC_VM_STATS() |
| 18 | }; | 18 | }; |
| 19 | static_assert(ARRAY_SIZE(kvm_vm_stats_desc) == | 19 | static_assert(ARRAY_SIZE(kvm_vm_stats_desc) == |
| @@ -181,7 +181,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 161 unmodified lines | |||
| 181 | break; | 181 | break; |
| 182 | case KVM_CAP_IOEVENTFD: | 182 | case KVM_CAP_IOEVENTFD: |
| 183 | case KVM_CAP_USER_MEMORY: | 183 | case KVM_CAP_USER_MEMORY: |
| 184 | case KVM_CAP_SYNC_MMU: | ||
| 185 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: | 184 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: |
| 186 | case KVM_CAP_ONE_REG: | 185 | case KVM_CAP_ONE_REG: |
| 187 | case KVM_CAP_READONLY_MEM: | 186 | case KVM_CAP_READONLY_MEM: |
| @@ -19,9 +19,9 @@ struct s390_idle_data { · 18 unmodified lines | |||
| 19 | unsigned long mt_cycles_enter[8]; | 19 | unsigned long mt_cycles_enter[8]; |
| 20 | }; | 20 | }; |
| 21 | 21 | ||
| 22 | DECLARE_PER_CPU(struct s390_idle_data, s390_idle); | ||
| 23 | |||
| 22 | extern struct device_attribute dev_attr_idle_count; | 24 | extern struct device_attribute dev_attr_idle_count; |
| 23 | extern struct device_attribute dev_attr_idle_time_us; | 25 | extern struct device_attribute dev_attr_idle_time_us; |
| 24 | 26 | ||
| 25 | void psw_idle(struct s390_idle_data *data, unsigned long psw_mask); | ||
| 26 | |||
| 27 | #endif /* _S390_IDLE_H */ | 27 | #endif /* _S390_IDLE_H */ |
| @@ -159,7 +159,7 @@ static __always_inline void __stackleak_poison(unsigned long erase_low, · 158 unmodified lines | |||
| 159 | " j 4f\n" | 159 | " j 4f\n" |
| 160 | "3: mvc 8(1,%[addr]),0(%[addr])\n" | 160 | "3: mvc 8(1,%[addr]),0(%[addr])\n" |
| 161 | "4:" | 161 | "4:" |
| 162 | : [addr] "+&a" (erase_low), [count] "+&d" (count), [tmp] "=&a" (tmp) | 162 | : [addr] "+&a" (erase_low), [count] "+&a" (count), [tmp] "=&a" (tmp) |
| 163 | : [poison] "d" (poison) | 163 | : [poison] "d" (poison) |
| 164 | : "memory", "cc" | 164 | : "memory", "cc" |
| 165 | ); | 165 | ); |
| @@ -2,6 +2,12 @@ · 1 unmodified line | |||
| 2 | #ifndef _S390_VTIME_H | 2 | #ifndef _S390_VTIME_H |
| 3 | #define _S390_VTIME_H | 3 | #define _S390_VTIME_H |
| 4 | 4 | ||
| 5 | #include <asm/lowcore.h> | ||
| 6 | #include <asm/cpu_mf.h> | ||
| 7 | #include <asm/idle.h> | ||
| 8 | |||
| 9 | DECLARE_PER_CPU(u64, mt_cycles[8]); | ||
| 10 | |||
| 5 | static inline void update_timer_sys(void) | 11 | static inline void update_timer_sys(void) |
| 6 | { | 12 | { |
| 7 | struct lowcore *lc = get_lowcore(); | 13 | struct lowcore *lc = get_lowcore(); |
| @@ -20,4 +26,32 @@ static inline void update_timer_mcck(void) · 12 unmodified lines | |||
| 20 | lc->last_update_timer = lc->mcck_enter_timer; | 26 | lc->last_update_timer = lc->mcck_enter_timer; |
| 21 | } | 27 | } |
| 22 | 28 | ||
| 29 | static inline void update_timer_idle(void) | ||
| 30 | { | ||
| 31 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); | ||
| 32 | struct lowcore *lc = get_lowcore(); | ||
| 33 | u64 cycles_new[8]; | ||
| 34 | int i, mtid; | ||
| 35 | |||
| 36 | mtid = smp_cpu_mtid; | ||
| 37 | if (mtid) { | ||
| 38 | stcctm(MT_DIAG, mtid, cycles_new); | ||
| 39 | for (i = 0; i < mtid; i++) | ||
| 40 | __this_cpu_add(mt_cycles[i], cycles_new[i] - idle->mt_cycles_enter[i]); | ||
| 41 | } | ||
| 42 | /* | ||
| 43 | * This is a bit subtle: Forward last_update_clock so it excludes idle | ||
| 44 | * time. For correct steal time calculation in do_account_vtime() add | ||
| 45 | * passed wall time before idle_enter to steal_timer: | ||
| 46 | * During the passed wall time before idle_enter CPU time may have | ||
| 47 | * been accounted to system, hardirq, softirq, etc. lowcore fields. | ||
| 48 | * The accounted CPU times will be subtracted again from steal_timer | ||
| 49 | * when accumulated steal time is calculated in do_account_vtime(). | ||
| 50 | */ | ||
| 51 | lc->steal_timer += idle->clock_idle_enter - lc->last_update_clock; | ||
| 52 | lc->last_update_clock = lc->int_clock; | ||
| 53 | lc->system_timer += lc->last_update_timer - idle->timer_idle_enter; | ||
| 54 | lc->last_update_timer = lc->sys_enter_timer; | ||
| 55 | } | ||
| 56 | |||
| 23 | #endif /* _S390_VTIME_H */ | 57 | #endif /* _S390_VTIME_H */ |
| @@ -56,8 +56,6 @@ long sys_s390_pci_mmio_write(unsigned long, const void __user *, size_t); · 55 unmodified lines | |||
| 56 | long sys_s390_pci_mmio_read(unsigned long, void __user *, size_t); | 56 | long sys_s390_pci_mmio_read(unsigned long, void __user *, size_t); |
| 57 | long sys_s390_sthyi(unsigned long function_code, void __user *buffer, u64 __user *return_code, unsigned long flags); | 57 | long sys_s390_sthyi(unsigned long function_code, void __user *buffer, u64 __user *return_code, unsigned long flags); |
| 58 | 58 | ||
| 59 | DECLARE_PER_CPU(u64, mt_cycles[8]); | ||
| 60 | |||
| 61 | unsigned long stack_alloc(void); | 59 | unsigned long stack_alloc(void); |
| 62 | void stack_free(unsigned long stack); | 60 | void stack_free(unsigned long stack); |
| 63 | 61 | ||
| @@ -15,37 +15,22 @@ · 14 unmodified lines | |||
| 15 | #include <trace/events/power.h> | 15 | #include <trace/events/power.h> |
| 16 | #include <asm/cpu_mf.h> | 16 | #include <asm/cpu_mf.h> |
| 17 | #include <asm/cputime.h> | 17 | #include <asm/cputime.h> |
| 18 | #include <asm/idle.h> | ||
| 18 | #include <asm/nmi.h> | 19 | #include <asm/nmi.h> |
| 19 | #include <asm/smp.h> | 20 | #include <asm/smp.h> |
| 20 | #include "entry.h" | ||
| 21 | 21 | ||
| 22 | static DEFINE_PER_CPU(struct s390_idle_data, s390_idle); | 22 | DEFINE_PER_CPU(struct s390_idle_data, s390_idle); |
| 23 | 23 | ||
| 24 | void account_idle_time_irq(void) | 24 | void account_idle_time_irq(void) |
| 25 | { | 25 | { |
| 26 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); | 26 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); |
| 27 | struct lowcore *lc = get_lowcore(); | ||
| 28 | unsigned long idle_time; | 27 | unsigned long idle_time; |
| 29 | u64 cycles_new[8]; | ||
| 30 | int i; | ||
| 31 | 28 | ||
| 32 | if (smp_cpu_mtid) { | 29 | idle_time = get_lowcore()->int_clock - idle->clock_idle_enter; |
| 33 | stcctm(MT_DIAG, smp_cpu_mtid, cycles_new); | ||
| 34 | for (i = 0; i < smp_cpu_mtid; i++) | ||
| 35 | this_cpu_add(mt_cycles[i], cycles_new[i] - idle->mt_cycles_enter[i]); | ||
| 36 | } | ||
| 37 | |||
| 38 | idle_time = lc->int_clock - idle->clock_idle_enter; | ||
| 39 | |||
| 40 | lc->steal_timer += idle->clock_idle_enter - lc->last_update_clock; | ||
| 41 | lc->last_update_clock = lc->int_clock; | ||
| 42 | |||
| 43 | lc->system_timer += lc->last_update_timer - idle->timer_idle_enter; | ||
| 44 | lc->last_update_timer = lc->sys_enter_timer; | ||
| 45 | 30 | ||
| 46 | /* Account time spent with enabled wait psw loaded as idle time. */ | 31 | /* Account time spent with enabled wait psw loaded as idle time. */ |
| 47 | WRITE_ONCE(idle->idle_time, READ_ONCE(idle->idle_time) + idle_time); | 32 | __atomic64_add(idle_time, &idle->idle_time); |
| 48 | WRITE_ONCE(idle->idle_count, READ_ONCE(idle->idle_count) + 1); | 33 | __atomic64_add_const(1, &idle->idle_count); |
| 49 | account_idle_time(cputime_to_nsecs(idle_time)); | 34 | account_idle_time(cputime_to_nsecs(idle_time)); |
| 50 | } | 35 | } |
| 51 | 36 | ||
| @@ -2377,7 +2377,7 @@ void __init setup_ipl(void) · 2376 unmodified lines | |||
| 2377 | atomic_notifier_chain_register(&panic_notifier_list, &on_panic_nb); | 2377 | atomic_notifier_chain_register(&panic_notifier_list, &on_panic_nb); |
| 2378 | } | 2378 | } |
| 2379 | 2379 | ||
| 2380 | void s390_reset_system(void) | 2380 | void __no_stack_protector s390_reset_system(void) |
| 2381 | { | 2381 | { |
| 2382 | /* Disable prefixing */ | 2382 | /* Disable prefixing */ |
| 2383 | set_prefix(0); | 2383 | set_prefix(0); |
| @@ -146,6 +146,10 @@ void noinstr do_io_irq(struct pt_regs *regs) · 145 unmodified lines | |||
| 146 | struct pt_regs *old_regs = set_irq_regs(regs); | 146 | struct pt_regs *old_regs = set_irq_regs(regs); |
| 147 | bool from_idle; | 147 | bool from_idle; |
| 148 | 148 | ||
| 149 | from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | ||
| 150 | if (from_idle) | ||
| 151 | update_timer_idle(); | ||
| 152 | |||
| 149 | irq_enter_rcu(); | 153 | irq_enter_rcu(); |
| 150 | 154 | ||
| 151 | if (user_mode(regs)) { | 155 | if (user_mode(regs)) { |
| @@ -154,7 +158,6 @@ void noinstr do_io_irq(struct pt_regs *regs) · 2 unmodified lines | |||
| 154 | current->thread.last_break = regs->last_break; | 158 | current->thread.last_break = regs->last_break; |
| 155 | } | 159 | } |
| 156 | 160 | ||
| 157 | from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | ||
| 158 | if (from_idle) | 161 | if (from_idle) |
| 159 | account_idle_time_irq(); | 162 | account_idle_time_irq(); |
| 160 | 163 | ||
| @@ -182,6 +185,10 @@ void noinstr do_ext_irq(struct pt_regs *regs) · 21 unmodified lines | |||
| 182 | struct pt_regs *old_regs = set_irq_regs(regs); | 185 | struct pt_regs *old_regs = set_irq_regs(regs); |
| 183 | bool from_idle; | 186 | bool from_idle; |
| 184 | 187 | ||
| 188 | from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | ||
| 189 | if (from_idle) | ||
| 190 | update_timer_idle(); | ||
| 191 | |||
| 185 | irq_enter_rcu(); | 192 | irq_enter_rcu(); |
| 186 | 193 | ||
| 187 | if (user_mode(regs)) { | 194 | if (user_mode(regs)) { |
| @@ -194,7 +201,6 @@ void noinstr do_ext_irq(struct pt_regs *regs) · 6 unmodified lines | |||
| 194 | regs->int_parm = get_lowcore()->ext_params; | 201 | regs->int_parm = get_lowcore()->ext_params; |
| 195 | regs->int_parm_long = get_lowcore()->ext_params2; | 202 | regs->int_parm_long = get_lowcore()->ext_params2; |
| 196 | 203 | ||
| 197 | from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | ||
| 198 | if (from_idle) | 204 | if (from_idle) |
| 199 | account_idle_time_irq(); | 205 | account_idle_time_irq(); |
| 200 | 206 | ||
| @@ -221,6 +221,7 @@ SECTIONS · 220 unmodified lines | |||
| 221 | /* Debugging sections. */ | 221 | /* Debugging sections. */ |
| 222 | STABS_DEBUG | 222 | STABS_DEBUG |
| 223 | DWARF_DEBUG | 223 | DWARF_DEBUG |
| 224 | MODINFO | ||
| 224 | ELF_DETAILS | 225 | ELF_DETAILS |
| 225 | 226 | ||
| 226 | /* | 227 | /* |
| @@ -48,8 +48,7 @@ static inline void set_vtimer(u64 expires) · 47 unmodified lines | |||
| 48 | 48 | ||
| 49 | static inline int virt_timer_forward(u64 elapsed) | 49 | static inline int virt_timer_forward(u64 elapsed) |
| 50 | { | 50 | { |
| 51 | BUG_ON(!irqs_disabled()); | 51 | lockdep_assert_irqs_disabled(); |
| 52 | |||
| 53 | if (list_empty(&virt_timer_list)) | 52 | if (list_empty(&virt_timer_list)) |
| 54 | return 0; | 53 | return 0; |
| 55 | elapsed = atomic64_add_return(elapsed, &virt_timer_elapsed); | 54 | elapsed = atomic64_add_return(elapsed, &virt_timer_elapsed); |
| @@ -137,23 +136,16 @@ static int do_account_vtime(struct task_struct *tsk) · 81 unmodified lines | |||
| 137 | lc->system_timer += timer; | 136 | lc->system_timer += timer; |
| 138 | 137 | ||
| 139 | /* Update MT utilization calculation */ | 138 | /* Update MT utilization calculation */ |
| 140 | if (smp_cpu_mtid && | 139 | if (smp_cpu_mtid && time_after64(jiffies_64, __this_cpu_read(mt_scaling_jiffies))) |
| 141 | time_after64(jiffies_64, this_cpu_read(mt_scaling_jiffies))) | ||
| 142 | update_mt_scaling(); | 140 | update_mt_scaling(); |
| 143 | 141 | ||
| 144 | /* Calculate cputime delta */ | 142 | /* Calculate cputime delta */ |
| 145 | user = update_tsk_timer(&tsk->thread.user_timer, | 143 | user = update_tsk_timer(&tsk->thread.user_timer, lc->user_timer); |
| 146 | READ_ONCE(lc->user_timer)); | 144 | guest = update_tsk_timer(&tsk->thread.guest_timer, lc->guest_timer); |
| 147 | guest = update_tsk_timer(&tsk->thread.guest_timer, | 145 | system = update_tsk_timer(&tsk->thread.system_timer, lc->system_timer); |
| 148 | READ_ONCE(lc->guest_timer)); | 146 | hardirq = update_tsk_timer(&tsk->thread.hardirq_timer, lc->hardirq_timer); |
| 149 | system = update_tsk_timer(&tsk->thread.system_timer, | 147 | softirq = update_tsk_timer(&tsk->thread.softirq_timer, lc->softirq_timer); |
| 150 | READ_ONCE(lc->system_timer)); | 148 | lc->steal_timer += clock - user - guest - system - hardirq - softirq; |
| 151 | hardirq = update_tsk_timer(&tsk->thread.hardirq_timer, | ||
| 152 | READ_ONCE(lc->hardirq_timer)); | ||
| 153 | softirq = update_tsk_timer(&tsk->thread.softirq_timer, | ||
| 154 | READ_ONCE(lc->softirq_timer)); | ||
| 155 | lc->steal_timer += | ||
| 156 | clock - user - guest - system - hardirq - softirq; | ||
| 157 | 149 | ||
| 158 | /* Push account value */ | 150 | /* Push account value */ |
| 159 | if (user) { | 151 | if (user) { |
| @@ -225,10 +217,6 @@ static u64 vtime_delta(void) · 65 unmodified lines | |||
| 225 | return timer - lc->last_update_timer; | 217 | return timer - lc->last_update_timer; |
| 226 | } | 218 | } |
| 227 | 219 | ||
| 228 | /* | ||
| 229 | * Update process times based on virtual cpu times stored by entry.S | ||
| 230 | * to the lowcore fields user_timer, system_timer & steal_clock. | ||
| 231 | */ | ||
| 232 | void vtime_account_kernel(struct task_struct *tsk) | 220 | void vtime_account_kernel(struct task_struct *tsk) |
| 233 | { | 221 | { |
| 234 | struct lowcore *lc = get_lowcore(); | 222 | struct lowcore *lc = get_lowcore(); |
| @@ -238,27 +226,17 @@ void vtime_account_kernel(struct task_struct *tsk) · 3 unmodified lines | |||
| 238 | lc->guest_timer += delta; | 226 | lc->guest_timer += delta; |
| 239 | else | 227 | else |
| 240 | lc->system_timer += delta; | 228 | lc->system_timer += delta; |
| 241 | |||
| 242 | virt_timer_forward(delta); | ||
| 243 | } | 229 | } |
| 244 | EXPORT_SYMBOL_GPL(vtime_account_kernel); | 230 | EXPORT_SYMBOL_GPL(vtime_account_kernel); |
| 245 | 231 | ||
| 246 | void vtime_account_softirq(struct task_struct *tsk) | 232 | void vtime_account_softirq(struct task_struct *tsk) |
| 247 | { | 233 | { |
| 248 | u64 delta = vtime_delta(); | 234 | get_lowcore()->softirq_timer += vtime_delta(); |
| 249 | |||
| 250 | get_lowcore()->softirq_timer += delta; | ||
| 251 | |||
| 252 | virt_timer_forward(delta); | ||
| 253 | } | 235 | } |
| 254 | 236 | ||
| 255 | void vtime_account_hardirq(struct task_struct *tsk) | 237 | void vtime_account_hardirq(struct task_struct *tsk) |
| 256 | { | 238 | { |
| 257 | u64 delta = vtime_delta(); | 239 | get_lowcore()->hardirq_timer += vtime_delta(); |
| 258 | |||
| 259 | get_lowcore()->hardirq_timer += delta; | ||
| 260 | |||
| 261 | virt_timer_forward(delta); | ||
| 262 | } | 240 | } |
| 263 | 241 | ||
| 264 | /* | 242 | /* |
| @@ -28,9 +28,7 @@ config KVM · 27 unmodified lines | |||
| 28 | select HAVE_KVM_INVALID_WAKEUPS | 28 | select HAVE_KVM_INVALID_WAKEUPS |
| 29 | select HAVE_KVM_NO_POLL | 29 | select HAVE_KVM_NO_POLL |
| 30 | select KVM_VFIO | 30 | select KVM_VFIO |
| 31 | select MMU_NOTIFIER | ||
| 32 | select VIRT_XFER_TO_GUEST_WORK | 31 | select VIRT_XFER_TO_GUEST_WORK |
| 33 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 34 | select KVM_MMU_LOCKLESS_AGING | 32 | select KVM_MMU_LOCKLESS_AGING |
| 35 | help | 33 | help |
| 36 | Support hosting paravirtualized guest machines using the SIE | 34 | Support hosting paravirtualized guest machines using the SIE |
| @@ -65,7 +65,7 @@ · 64 unmodified lines | |||
| 65 | #define VCPU_IRQS_MAX_BUF (sizeof(struct kvm_s390_irq) * \ | 65 | #define VCPU_IRQS_MAX_BUF (sizeof(struct kvm_s390_irq) * \ |
| 66 | (KVM_MAX_VCPUS + LOCAL_IRQS)) | 66 | (KVM_MAX_VCPUS + LOCAL_IRQS)) |
| 67 | 67 | ||
| 68 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 68 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 69 | KVM_GENERIC_VM_STATS(), | 69 | KVM_GENERIC_VM_STATS(), |
| 70 | STATS_DESC_COUNTER(VM, inject_io), | 70 | STATS_DESC_COUNTER(VM, inject_io), |
| 71 | STATS_DESC_COUNTER(VM, inject_float_mchk), | 71 | STATS_DESC_COUNTER(VM, inject_float_mchk), |
| @@ -91,7 +91,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 19 unmodified lines | |||
| 91 | sizeof(kvm_vm_stats_desc), | 91 | sizeof(kvm_vm_stats_desc), |
| 92 | }; | 92 | }; |
| 93 | 93 | ||
| 94 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 94 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 95 | KVM_GENERIC_VCPU_STATS(), | 95 | KVM_GENERIC_VCPU_STATS(), |
| 96 | STATS_DESC_COUNTER(VCPU, exit_userspace), | 96 | STATS_DESC_COUNTER(VCPU, exit_userspace), |
| 97 | STATS_DESC_COUNTER(VCPU, exit_null), | 97 | STATS_DESC_COUNTER(VCPU, exit_null), |
| @@ -601,7 +601,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 503 unmodified lines | |||
| 601 | switch (ext) { | 601 | switch (ext) { |
| 602 | case KVM_CAP_S390_PSW: | 602 | case KVM_CAP_S390_PSW: |
| 603 | case KVM_CAP_S390_GMAP: | 603 | case KVM_CAP_S390_GMAP: |
| 604 | case KVM_CAP_SYNC_MMU: | ||
| 605 | #ifdef CONFIG_KVM_S390_UCONTROL | 604 | #ifdef CONFIG_KVM_S390_UCONTROL |
| 606 | case KVM_CAP_S390_UCONTROL: | 605 | case KVM_CAP_S390_UCONTROL: |
| 607 | #endif | 606 | #endif |
| @@ -28,8 +28,8 @@ static void xor_xc_2(unsigned long bytes, unsigned long * __restrict p1, · 27 unmodified lines | |||
| 28 | " j 3f\n" | 28 | " j 3f\n" |
| 29 | "2: xc 0(1,%1),0(%2)\n" | 29 | "2: xc 0(1,%1),0(%2)\n" |
| 30 | "3:" | 30 | "3:" |
| 31 | : : "d" (bytes), "a" (p1), "a" (p2) | 31 | : "+a" (bytes), "+a" (p1), "+a" (p2) |
| 32 | : "0", "cc", "memory"); | 32 | : : "0", "cc", "memory"); |
| 33 | } | 33 | } |
| 34 | 34 | ||
| 35 | static void xor_xc_3(unsigned long bytes, unsigned long * __restrict p1, | 35 | static void xor_xc_3(unsigned long bytes, unsigned long * __restrict p1, |
| @@ -54,7 +54,7 @@ static void xor_xc_3(unsigned long bytes, unsigned long * __restrict p1, · 18 unmodified lines | |||
| 54 | "2: xc 0(1,%1),0(%2)\n" | 54 | "2: xc 0(1,%1),0(%2)\n" |
| 55 | "3: xc 0(1,%1),0(%3)\n" | 55 | "3: xc 0(1,%1),0(%3)\n" |
| 56 | "4:" | 56 | "4:" |
| 57 | : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3) | 57 | : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3) |
| 58 | : : "0", "cc", "memory"); | 58 | : : "0", "cc", "memory"); |
| 59 | } | 59 | } |
| 60 | 60 | ||
| @@ -85,7 +85,7 @@ static void xor_xc_4(unsigned long bytes, unsigned long * __restrict p1, · 24 unmodified lines | |||
| 85 | "3: xc 0(1,%1),0(%3)\n" | 85 | "3: xc 0(1,%1),0(%3)\n" |
| 86 | "4: xc 0(1,%1),0(%4)\n" | 86 | "4: xc 0(1,%1),0(%4)\n" |
| 87 | "5:" | 87 | "5:" |
| 88 | : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4) | 88 | : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4) |
| 89 | : : "0", "cc", "memory"); | 89 | : : "0", "cc", "memory"); |
| 90 | } | 90 | } |
| 91 | 91 | ||
| @@ -96,7 +96,6 @@ static void xor_xc_5(unsigned long bytes, unsigned long * __restrict p1, · 4 unmodified lines | |||
| 96 | const unsigned long * __restrict p5) | 96 | const unsigned long * __restrict p5) |
| 97 | { | 97 | { |
| 98 | asm volatile( | 98 | asm volatile( |
| 99 | " larl 1,2f\n" | ||
| 100 | " aghi %0,-1\n" | 99 | " aghi %0,-1\n" |
| 101 | " jm 6f\n" | 100 | " jm 6f\n" |
| 102 | " srlg 0,%0,8\n" | 101 | " srlg 0,%0,8\n" |
| @@ -122,7 +121,7 @@ static void xor_xc_5(unsigned long bytes, unsigned long * __restrict p1, · 19 unmodified lines | |||
| 122 | "4: xc 0(1,%1),0(%4)\n" | 121 | "4: xc 0(1,%1),0(%4)\n" |
| 123 | "5: xc 0(1,%1),0(%5)\n" | 122 | "5: xc 0(1,%1),0(%5)\n" |
| 124 | "6:" | 123 | "6:" |
| 125 | : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4), | 124 | : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4), |
| 126 | "+a" (p5) | 125 | "+a" (p5) |
| 127 | : : "0", "cc", "memory"); | 126 | : : "0", "cc", "memory"); |
| 128 | } | 127 | } |
| @@ -62,7 +62,7 @@ int __pfault_init(void) · 61 unmodified lines | |||
| 62 | "0: nopr %%r7\n" | 62 | "0: nopr %%r7\n" |
| 63 | EX_TABLE(0b, 0b) | 63 | EX_TABLE(0b, 0b) |
| 64 | : [rc] "+d" (rc) | 64 | : [rc] "+d" (rc) |
| 65 | : [refbk] "a" (&pfault_init_refbk), "m" (pfault_init_refbk) | 65 | : [refbk] "a" (virt_to_phys(&pfault_init_refbk)), "m" (pfault_init_refbk) |
| 66 | : "cc"); | 66 | : "cc"); |
| 67 | return rc; | 67 | return rc; |
| 68 | } | 68 | } |
| @@ -84,7 +84,7 @@ void __pfault_fini(void) · 15 unmodified lines | |||
| 84 | "0: nopr %%r7\n" | 84 | "0: nopr %%r7\n" |
| 85 | EX_TABLE(0b, 0b) | 85 | EX_TABLE(0b, 0b) |
| 86 | : | 86 | : |
| 87 | : [refbk] "a" (&pfault_fini_refbk), "m" (pfault_fini_refbk) | 87 | : [refbk] "a" (virt_to_phys(&pfault_fini_refbk)), "m" (pfault_fini_refbk) |
| 88 | : "cc"); | 88 | : "cc"); |
| 89 | } | 89 | } |
| 90 | 90 | ||
| @@ -89,6 +89,7 @@ SECTIONS · 88 unmodified lines | |||
| 89 | 89 | ||
| 90 | STABS_DEBUG | 90 | STABS_DEBUG |
| 91 | DWARF_DEBUG | 91 | DWARF_DEBUG |
| 92 | MODINFO | ||
| 92 | ELF_DETAILS | 93 | ELF_DETAILS |
| 93 | 94 | ||
| 94 | DISCARDS | 95 | DISCARDS |
| @@ -312,6 +312,8 @@ static dma_addr_t dma_4u_map_phys(struct device *dev, phys_addr_t phys, · 311 unmodified lines | |||
| 312 | if (direction != DMA_TO_DEVICE) | 312 | if (direction != DMA_TO_DEVICE) |
| 313 | iopte_protection |= IOPTE_WRITE; | 313 | iopte_protection |= IOPTE_WRITE; |
| 314 | 314 | ||
| 315 | phys &= IO_PAGE_MASK; | ||
| 316 | |||
| 315 | for (i = 0; i < npages; i++, base++, phys += IO_PAGE_SIZE) | 317 | for (i = 0; i < npages; i++, base++, phys += IO_PAGE_SIZE) |
| 316 | iopte_val(*base) = iopte_protection | phys; | 318 | iopte_val(*base) = iopte_protection | phys; |
| 317 | 319 | ||
| @@ -355,6 +355,13 @@ static struct pci_dev *of_create_pci_dev(struct pci_pbm_info *pbm, · 354 unmodified lines | |||
| 355 | dev->error_state = pci_channel_io_normal; | 355 | dev->error_state = pci_channel_io_normal; |
| 356 | dev->dma_mask = 0xffffffff; | 356 | dev->dma_mask = 0xffffffff; |
| 357 | 357 | ||
| 358 | /* | ||
| 359 | * Assume 64-bit addresses for MSI initially. Will be changed to 32-bit | ||
| 360 | * if MSI (rather than MSI-X) capability does not have | ||
| 361 | * PCI_MSI_FLAGS_64BIT. Can also be overridden by driver. | ||
| 362 | */ | ||
| 363 | dev->msi_addr_mask = DMA_BIT_MASK(64); | ||
| 364 | |||
| 358 | if (of_node_name_eq(node, "pci")) { | 365 | if (of_node_name_eq(node, "pci")) { |
| 359 | /* a PCI-PCI bridge */ | 366 | /* a PCI-PCI bridge */ |
| 360 | dev->hdr_type = PCI_HEADER_TYPE_BRIDGE; | 367 | dev->hdr_type = PCI_HEADER_TYPE_BRIDGE; |
| @@ -410,6 +410,8 @@ static dma_addr_t dma_4v_map_phys(struct device *dev, phys_addr_t phys, · 409 unmodified lines | |||
| 410 | 410 | ||
| 411 | iommu_batch_start(dev, prot, entry); | 411 | iommu_batch_start(dev, prot, entry); |
| 412 | 412 | ||
| 413 | phys &= IO_PAGE_MASK; | ||
| 414 | |||
| 413 | for (i = 0; i < npages; i++, phys += IO_PAGE_SIZE) { | 415 | for (i = 0; i < npages; i++, phys += IO_PAGE_SIZE) { |
| 414 | long err = iommu_batch_add(phys, mask); | 416 | long err = iommu_batch_add(phys, mask); |
| 415 | if (unlikely(err < 0L)) | 417 | if (unlikely(err < 0L)) |
| @@ -191,6 +191,7 @@ SECTIONS · 190 unmodified lines | |||
| 191 | 191 | ||
| 192 | STABS_DEBUG | 192 | STABS_DEBUG |
| 193 | DWARF_DEBUG | 193 | DWARF_DEBUG |
| 194 | MODINFO | ||
| 194 | ELF_DETAILS | 195 | ELF_DETAILS |
| 195 | 196 | ||
| 196 | DISCARDS | 197 | DISCARDS |
| @@ -69,11 +69,11 @@ struct io_thread_req { · 68 unmodified lines | |||
| 69 | }; | 69 | }; |
| 70 | 70 | ||
| 71 | 71 | ||
| 72 | static struct io_thread_req * (*irq_req_buffer)[]; | 72 | static struct io_thread_req **irq_req_buffer; |
| 73 | static struct io_thread_req *irq_remainder; | 73 | static struct io_thread_req *irq_remainder; |
| 74 | static int irq_remainder_size; | 74 | static int irq_remainder_size; |
| 75 | 75 | ||
| 76 | static struct io_thread_req * (*io_req_buffer)[]; | 76 | static struct io_thread_req **io_req_buffer; |
| 77 | static struct io_thread_req *io_remainder; | 77 | static struct io_thread_req *io_remainder; |
| 78 | static int io_remainder_size; | 78 | static int io_remainder_size; |
| 79 | 79 | ||
| @@ -398,7 +398,7 @@ static int thread_fd = -1; · 318 unmodified lines | |||
| 398 | 398 | ||
| 399 | static int bulk_req_safe_read( | 399 | static int bulk_req_safe_read( |
| 400 | int fd, | 400 | int fd, |
| 401 | struct io_thread_req * (*request_buffer)[], | 401 | struct io_thread_req **request_buffer, |
| 402 | struct io_thread_req **remainder, | 402 | struct io_thread_req **remainder, |
| 403 | int *remainder_size, | 403 | int *remainder_size, |
| 404 | int max_recs | 404 | int max_recs |
| @@ -465,7 +465,7 @@ static irqreturn_t ubd_intr(int irq, void *dev) · 60 unmodified lines | |||
| 465 | &irq_remainder, &irq_remainder_size, | 465 | &irq_remainder, &irq_remainder_size, |
| 466 | UBD_REQ_BUFFER_SIZE)) >= 0) { | 466 | UBD_REQ_BUFFER_SIZE)) >= 0) { |
| 467 | for (i = 0; i < len / sizeof(struct io_thread_req *); i++) | 467 | for (i = 0; i < len / sizeof(struct io_thread_req *); i++) |
| 468 | ubd_end_request((*irq_req_buffer)[i]); | 468 | ubd_end_request(irq_req_buffer[i]); |
| 469 | } | 469 | } |
| 470 | 470 | ||
| 471 | if (len < 0 && len != -EAGAIN) | 471 | if (len < 0 && len != -EAGAIN) |
| @@ -1512,7 +1512,7 @@ void *io_thread(void *arg) · 1040 unmodified lines | |||
| 1512 | } | 1512 | } |
| 1513 | 1513 | ||
| 1514 | for (count = 0; count < n/sizeof(struct io_thread_req *); count++) { | 1514 | for (count = 0; count < n/sizeof(struct io_thread_req *); count++) { |
| 1515 | struct io_thread_req *req = (*io_req_buffer)[count]; | 1515 | struct io_thread_req *req = io_req_buffer[count]; |
| 1516 | int i; | 1516 | int i; |
| 1517 | 1517 | ||
| 1518 | io_count++; | 1518 | io_count++; |
| @@ -172,6 +172,7 @@ SECTIONS · 171 unmodified lines | |||
| 172 | 172 | ||
| 173 | STABS_DEBUG | 173 | STABS_DEBUG |
| 174 | DWARF_DEBUG | 174 | DWARF_DEBUG |
| 175 | MODINFO | ||
| 175 | ELF_DETAILS | 176 | ELF_DETAILS |
| 176 | 177 | ||
| 177 | DISCARDS | 178 | DISCARDS |
| @@ -113,6 +113,7 @@ SECTIONS · 112 unmodified lines | |||
| 113 | 113 | ||
| 114 | STABS_DEBUG | 114 | STABS_DEBUG |
| 115 | DWARF_DEBUG | 115 | DWARF_DEBUG |
| 116 | MODINFO | ||
| 116 | ELF_DETAILS | 117 | ELF_DETAILS |
| 117 | 118 | ||
| 118 | DISCARDS | 119 | DISCARDS |
| @@ -113,6 +113,7 @@ vmlinux-objs-$(CONFIG_EFI_SBAT) += $(obj)/sbat.o · 112 unmodified lines | |||
| 113 | 113 | ||
| 114 | ifdef CONFIG_EFI_SBAT | 114 | ifdef CONFIG_EFI_SBAT |
| 115 | $(obj)/sbat.o: $(CONFIG_EFI_SBAT_FILE) | 115 | $(obj)/sbat.o: $(CONFIG_EFI_SBAT_FILE) |
| 116 | AFLAGS_sbat.o += -I $(srctree) | ||
| 116 | endif | 117 | endif |
| 117 | 118 | ||
| 118 | $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE | 119 | $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE |
| @@ -28,17 +28,17 @@ · 27 unmodified lines | |||
| 28 | #include "sev.h" | 28 | #include "sev.h" |
| 29 | 29 | ||
| 30 | static struct ghcb boot_ghcb_page __aligned(PAGE_SIZE); | 30 | static struct ghcb boot_ghcb_page __aligned(PAGE_SIZE); |
| 31 | struct ghcb *boot_ghcb; | 31 | struct ghcb *boot_ghcb __section(".data"); |
| 32 | 32 | ||
| 33 | #undef __init | 33 | #undef __init |
| 34 | #define __init | 34 | #define __init |
| 35 | 35 | ||
| 36 | #define __BOOT_COMPRESSED | 36 | #define __BOOT_COMPRESSED |
| 37 | 37 | ||
| 38 | u8 snp_vmpl; | 38 | u8 snp_vmpl __section(".data"); |
| 39 | u16 ghcb_version; | 39 | u16 ghcb_version __section(".data"); |
| 40 | 40 | ||
| 41 | u64 boot_svsm_caa_pa; | 41 | u64 boot_svsm_caa_pa __section(".data"); |
| 42 | 42 | ||
| 43 | /* Include code for early handlers */ | 43 | /* Include code for early handlers */ |
| 44 | #include "../../boot/startup/sev-shared.c" | 44 | #include "../../boot/startup/sev-shared.c" |
| @@ -188,6 +188,7 @@ bool sev_es_check_ghcb_fault(unsigned long address) · 143 unmodified lines | |||
| 188 | MSR_AMD64_SNP_RESERVED_BIT13 | \ | 188 | MSR_AMD64_SNP_RESERVED_BIT13 | \ |
| 189 | MSR_AMD64_SNP_RESERVED_BIT15 | \ | 189 | MSR_AMD64_SNP_RESERVED_BIT15 | \ |
| 190 | MSR_AMD64_SNP_SECURE_AVIC | \ | 190 | MSR_AMD64_SNP_SECURE_AVIC | \ |
| 191 | MSR_AMD64_SNP_RESERVED_BITS19_22 | \ | ||
| 191 | MSR_AMD64_SNP_RESERVED_MASK) | 192 | MSR_AMD64_SNP_RESERVED_MASK) |
| 192 | 193 | ||
| 193 | #ifdef CONFIG_AMD_SECURE_AVIC | 194 | #ifdef CONFIG_AMD_SECURE_AVIC |
| @@ -88,7 +88,7 @@ SECTIONS · 87 unmodified lines | |||
| 88 | /DISCARD/ : { | 88 | /DISCARD/ : { |
| 89 | *(.dynamic) *(.dynsym) *(.dynstr) *(.dynbss) | 89 | *(.dynamic) *(.dynsym) *(.dynstr) *(.dynbss) |
| 90 | *(.hash) *(.gnu.hash) | 90 | *(.hash) *(.gnu.hash) |
| 91 | *(.note.*) | 91 | *(.note.*) *(.modinfo) |
| 92 | } | 92 | } |
| 93 | 93 | ||
| 94 | .got.plt (INFO) : { | 94 | .got.plt (INFO) : { |
| @@ -31,7 +31,7 @@ static u32 cpuid_std_range_max __ro_after_init; · 30 unmodified lines | |||
| 31 | static u32 cpuid_hyp_range_max __ro_after_init; | 31 | static u32 cpuid_hyp_range_max __ro_after_init; |
| 32 | static u32 cpuid_ext_range_max __ro_after_init; | 32 | static u32 cpuid_ext_range_max __ro_after_init; |
| 33 | 33 | ||
| 34 | bool sev_snp_needs_sfw; | 34 | bool sev_snp_needs_sfw __section(".data"); |
| 35 | 35 | ||
| 36 | void __noreturn | 36 | void __noreturn |
| 37 | sev_es_terminate(unsigned int set, unsigned int reason) | 37 | sev_es_terminate(unsigned int set, unsigned int reason) |
| @@ -89,6 +89,7 @@ static const char * const sev_status_feat_names[] = { · 88 unmodified lines | |||
| 89 | [MSR_AMD64_SNP_VMSA_REG_PROT_BIT] = "VMSARegProt", | 89 | [MSR_AMD64_SNP_VMSA_REG_PROT_BIT] = "VMSARegProt", |
| 90 | [MSR_AMD64_SNP_SMT_PROT_BIT] = "SMTProt", | 90 | [MSR_AMD64_SNP_SMT_PROT_BIT] = "SMTProt", |
| 91 | [MSR_AMD64_SNP_SECURE_AVIC_BIT] = "SecureAVIC", | 91 | [MSR_AMD64_SNP_SECURE_AVIC_BIT] = "SecureAVIC", |
| 92 | [MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT] = "IBPBOnEntry", | ||
| 92 | }; | 93 | }; |
| 93 | 94 | ||
| 94 | /* | 95 | /* |
| @@ -160,8 +160,6 @@ void __init fred_complete_exception_setup(void) · 159 unmodified lines | |||
| 160 | static noinstr void fred_extint(struct pt_regs *regs) | 160 | static noinstr void fred_extint(struct pt_regs *regs) |
| 161 | { | 161 | { |
| 162 | unsigned int vector = regs->fred_ss.vector; | 162 | unsigned int vector = regs->fred_ss.vector; |
| 163 | unsigned int index = array_index_nospec(vector - FIRST_SYSTEM_VECTOR, | ||
| 164 | NR_SYSTEM_VECTORS); | ||
| 165 | 163 | ||
| 166 | if (WARN_ON_ONCE(vector < FIRST_EXTERNAL_VECTOR)) | 164 | if (WARN_ON_ONCE(vector < FIRST_EXTERNAL_VECTOR)) |
| 167 | return; | 165 | return; |
| @@ -170,7 +168,8 @@ static noinstr void fred_extint(struct pt_regs *regs) · 2 unmodified lines | |||
| 170 | irqentry_state_t state = irqentry_enter(regs); | 168 | irqentry_state_t state = irqentry_enter(regs); |
| 171 | 169 | ||
| 172 | instrumentation_begin(); | 170 | instrumentation_begin(); |
| 173 | sysvec_table[index](regs); | 171 | sysvec_table[array_index_nospec(vector - FIRST_SYSTEM_VECTOR, |
| 172 | NR_SYSTEM_VECTORS)](regs); | ||
| 174 | instrumentation_end(); | 173 | instrumentation_end(); |
| 175 | irqentry_exit(regs, state); | 174 | irqentry_exit(regs, state); |
| 176 | } else { | 175 | } else { |
| @@ -35,9 +35,38 @@ · 34 unmodified lines | |||
| 35 | #endif | 35 | #endif |
| 36 | .endm | 36 | .endm |
| 37 | 37 | ||
| 38 | /* | ||
| 39 | * WARNING: | ||
| 40 | * | ||
| 41 | * A bug in the libgcc unwinder as of at least gcc 15.2 (2026) means that | ||
| 42 | * the unwinder fails to recognize the signal frame flag. | ||
| 43 | * | ||
| 44 | * There is a hacky legacy fallback path in libgcc which ends up | ||
| 45 | * getting invoked instead. It happens to work as long as BOTH of the | ||
| 46 | * following conditions are true: | ||
| 47 | * | ||
| 48 | * 1. There is at least one byte before the each of the sigreturn | ||
| 49 | * functions which falls outside any function. This is enforced by | ||
| 50 | * an explicit nop instruction before the ALIGN. | ||
| 51 | * 2. The code sequences between the entry point up to and including | ||
| 52 | * the int $0x80 below need to match EXACTLY. Do not change them | ||
| 53 | * in any way. The exact byte sequences are: | ||
| 54 | * | ||
| 55 | * __kernel_sigreturn: | ||
| 56 | * 0: 58 pop %eax | ||
| 57 | * 1: b8 77 00 00 00 mov $0x77,%eax | ||
| 58 | * 6: cd 80 int $0x80 | ||
| 59 | * | ||
| 60 | * __kernel_rt_sigreturn: | ||
| 61 | * 0: b8 ad 00 00 00 mov $0xad,%eax | ||
| 62 | * 5: cd 80 int $0x80 | ||
| 63 | * | ||
| 64 | * For details, see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=124050 | ||
| 65 | */ | ||
| 38 | .text | 66 | .text |
| 39 | .globl __kernel_sigreturn | 67 | .globl __kernel_sigreturn |
| 40 | .type __kernel_sigreturn,@function | 68 | .type __kernel_sigreturn,@function |
| 69 | nop /* libgcc hack: see comment above */ | ||
| 41 | ALIGN | 70 | ALIGN |
| 42 | __kernel_sigreturn: | 71 | __kernel_sigreturn: |
| 43 | STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext | 72 | STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext |
| @@ -52,6 +81,7 @@ SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) · 8 unmodified lines | |||
| 52 | 81 | ||
| 53 | .globl __kernel_rt_sigreturn | 82 | .globl __kernel_rt_sigreturn |
| 54 | .type __kernel_rt_sigreturn,@function | 83 | .type __kernel_rt_sigreturn,@function |
| 84 | nop /* libgcc hack: see comment above */ | ||
| 55 | ALIGN | 85 | ALIGN |
| 56 | __kernel_rt_sigreturn: | 86 | __kernel_rt_sigreturn: |
| 57 | STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext | 87 | STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext |
| @@ -6497,6 +6497,32 @@ static struct intel_uncore_type gnr_uncore_ubox = { · 6496 unmodified lines | |||
| 6497 | .attr_update = uncore_alias_groups, | 6497 | .attr_update = uncore_alias_groups, |
| 6498 | }; | 6498 | }; |
| 6499 | 6499 | ||
| 6500 | static struct uncore_event_desc gnr_uncore_imc_events[] = { | ||
| 6501 | INTEL_UNCORE_EVENT_DESC(clockticks, "event=0x01,umask=0x00"), | ||
| 6502 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0, "event=0x05,umask=0xcf"), | ||
| 6503 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0.scale, "6.103515625e-5"), | ||
| 6504 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0.unit, "MiB"), | ||
| 6505 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1, "event=0x06,umask=0xcf"), | ||
| 6506 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1.scale, "6.103515625e-5"), | ||
| 6507 | INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1.unit, "MiB"), | ||
| 6508 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0, "event=0x05,umask=0xf0"), | ||
| 6509 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0.scale, "6.103515625e-5"), | ||
| 6510 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0.unit, "MiB"), | ||
| 6511 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1, "event=0x06,umask=0xf0"), | ||
| 6512 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1.scale, "6.103515625e-5"), | ||
| 6513 | INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1.unit, "MiB"), | ||
| 6514 | { /* end: all zeroes */ }, | ||
| 6515 | }; | ||
| 6516 | |||
| 6517 | static struct intel_uncore_type gnr_uncore_imc = { | ||
| 6518 | SPR_UNCORE_MMIO_COMMON_FORMAT(), | ||
| 6519 | .name = "imc", | ||
| 6520 | .fixed_ctr_bits = 48, | ||
| 6521 | .fixed_ctr = SNR_IMC_MMIO_PMON_FIXED_CTR, | ||
| 6522 | .fixed_ctl = SNR_IMC_MMIO_PMON_FIXED_CTL, | ||
| 6523 | .event_descs = gnr_uncore_imc_events, | ||
| 6524 | }; | ||
| 6525 | |||
| 6500 | static struct intel_uncore_type gnr_uncore_pciex8 = { | 6526 | static struct intel_uncore_type gnr_uncore_pciex8 = { |
| 6501 | SPR_UNCORE_PCI_COMMON_FORMAT(), | 6527 | SPR_UNCORE_PCI_COMMON_FORMAT(), |
| 6502 | .name = "pciex8", | 6528 | .name = "pciex8", |
| @@ -6544,7 +6570,7 @@ static struct intel_uncore_type *gnr_uncores[UNCORE_GNR_NUM_UNCORE_TYPES] = { · 41 unmodified lines | |||
| 6544 | NULL, | 6570 | NULL, |
| 6545 | &spr_uncore_pcu, | 6571 | &spr_uncore_pcu, |
| 6546 | &gnr_uncore_ubox, | 6572 | &gnr_uncore_ubox, |
| 6547 | &spr_uncore_imc, | 6573 | &gnr_uncore_imc, |
| 6548 | NULL, | 6574 | NULL, |
| 6549 | &gnr_uncore_upi, | 6575 | &gnr_uncore_upi, |
| 6550 | NULL, | 6576 | NULL, |
| @@ -7,7 +7,7 @@ · 6 unmodified lines | |||
| 7 | #include <linux/objtool.h> | 7 | #include <linux/objtool.h> |
| 8 | #include <asm/asm.h> | 8 | #include <asm/asm.h> |
| 9 | 9 | ||
| 10 | #ifndef __ASSEMBLY__ | 10 | #ifndef __ASSEMBLER__ |
| 11 | struct bug_entry; | 11 | struct bug_entry; |
| 12 | extern void __WARN_trap(struct bug_entry *bug, ...); | 12 | extern void __WARN_trap(struct bug_entry *bug, ...); |
| 13 | #endif | 13 | #endif |
| @@ -137,7 +137,7 @@ do { \ · 123 unmodified lines | |||
| 137 | 137 | ||
| 138 | #ifdef HAVE_ARCH_BUG_FORMAT_ARGS | 138 | #ifdef HAVE_ARCH_BUG_FORMAT_ARGS |
| 139 | 139 | ||
| 140 | #ifndef __ASSEMBLY__ | 140 | #ifndef __ASSEMBLER__ |
| 141 | #include <linux/static_call_types.h> | 141 | #include <linux/static_call_types.h> |
| 142 | DECLARE_STATIC_CALL(WARN_trap, __WARN_trap); | 142 | DECLARE_STATIC_CALL(WARN_trap, __WARN_trap); |
| 143 | 143 | ||
| @@ -153,7 +153,7 @@ struct arch_va_list { · 9 unmodified lines | |||
| 153 | struct sysv_va_list args; | 153 | struct sysv_va_list args; |
| 154 | }; | 154 | }; |
| 155 | extern void *__warn_args(struct arch_va_list *args, struct pt_regs *regs); | 155 | extern void *__warn_args(struct arch_va_list *args, struct pt_regs *regs); |
| 156 | #endif /* __ASSEMBLY__ */ | 156 | #endif /* __ASSEMBLER__ */ |
| 157 | 157 | ||
| 158 | #define __WARN_bug_entry(flags, format) ({ \ | 158 | #define __WARN_bug_entry(flags, format) ({ \ |
| 159 | struct bug_entry *bug; \ | 159 | struct bug_entry *bug; \ |
| @@ -111,6 +111,12 @@ extern bhi_thunk __bhi_args_end[]; · 110 unmodified lines | |||
| 111 | 111 | ||
| 112 | struct pt_regs; | 112 | struct pt_regs; |
| 113 | 113 | ||
| 114 | #ifdef CONFIG_CALL_PADDING | ||
| 115 | #define CFI_OFFSET (CONFIG_FUNCTION_PADDING_CFI+5) | ||
| 116 | #else | ||
| 117 | #define CFI_OFFSET 5 | ||
| 118 | #endif | ||
| 119 | |||
| 114 | #ifdef CONFIG_CFI | 120 | #ifdef CONFIG_CFI |
| 115 | enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); | 121 | enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); |
| 116 | #define __bpfcall | 122 | #define __bpfcall |
| @@ -119,11 +125,9 @@ static inline int cfi_get_offset(void) · 2 unmodified lines | |||
| 119 | { | 125 | { |
| 120 | switch (cfi_mode) { | 126 | switch (cfi_mode) { |
| 121 | case CFI_FINEIBT: | 127 | case CFI_FINEIBT: |
| 122 | return 16; | 128 | return /* fineibt_prefix_size */ 16; |
| 123 | case CFI_KCFI: | 129 | case CFI_KCFI: |
| 124 | if (IS_ENABLED(CONFIG_CALL_PADDING)) | 130 | return CFI_OFFSET; |
| 125 | return 16; | ||
| 126 | return 5; | ||
| 127 | default: | 131 | default: |
| 128 | return 0; | 132 | return 0; |
| 129 | } | 133 | } |
| @@ -138,7 +138,7 @@ extern void __init efi_apply_memmap_quirks(void); · 137 unmodified lines | |||
| 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); | 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); |
| 139 | extern void efi_delete_dummy_variable(void); | 139 | extern void efi_delete_dummy_variable(void); |
| 140 | extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); | 140 | extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); |
| 141 | extern void efi_free_boot_services(void); | 141 | extern void efi_unmap_boot_services(void); |
| 142 | 142 | ||
| 143 | void arch_efi_call_virt_setup(void); | 143 | void arch_efi_call_virt_setup(void); |
| 144 | void arch_efi_call_virt_teardown(void); | 144 | void arch_efi_call_virt_teardown(void); |
| @@ -77,7 +77,7 @@ static __always_inline void native_local_irq_restore(unsigned long flags) · 76 unmodified lines | |||
| 77 | #endif | 77 | #endif |
| 78 | 78 | ||
| 79 | #ifndef CONFIG_PARAVIRT | 79 | #ifndef CONFIG_PARAVIRT |
| 80 | #ifndef __ASSEMBLY__ | 80 | #ifndef __ASSEMBLER__ |
| 81 | /* | 81 | /* |
| 82 | * Used in the idle loop; sti takes one instruction cycle | 82 | * Used in the idle loop; sti takes one instruction cycle |
| 83 | * to complete: | 83 | * to complete: |
| @@ -95,7 +95,7 @@ static __always_inline void halt(void) · 11 unmodified lines | |||
| 95 | { | 95 | { |
| 96 | native_halt(); | 96 | native_halt(); |
| 97 | } | 97 | } |
| 98 | #endif /* __ASSEMBLY__ */ | 98 | #endif /* __ASSEMBLER__ */ |
| 99 | #endif /* CONFIG_PARAVIRT */ | 99 | #endif /* CONFIG_PARAVIRT */ |
| 100 | 100 | ||
| 101 | #ifdef CONFIG_PARAVIRT_XXL | 101 | #ifdef CONFIG_PARAVIRT_XXL |
| @@ -2485,7 +2485,8 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages); · 2484 unmodified lines | |||
| 2485 | KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS | \ | 2485 | KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS | \ |
| 2486 | KVM_X86_QUIRK_SLOT_ZAP_ALL | \ | 2486 | KVM_X86_QUIRK_SLOT_ZAP_ALL | \ |
| 2487 | KVM_X86_QUIRK_STUFF_FEATURE_MSRS | \ | 2487 | KVM_X86_QUIRK_STUFF_FEATURE_MSRS | \ |
| 2488 | KVM_X86_QUIRK_IGNORE_GUEST_PAT) | 2488 | KVM_X86_QUIRK_IGNORE_GUEST_PAT | \ |
| 2489 | KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM) | ||
| 2489 | 2490 | ||
| 2490 | #define KVM_X86_CONDITIONAL_QUIRKS \ | 2491 | #define KVM_X86_CONDITIONAL_QUIRKS \ |
| 2491 | (KVM_X86_QUIRK_CD_NW_CLEARED | \ | 2492 | (KVM_X86_QUIRK_CD_NW_CLEARED | \ |
| @@ -68,7 +68,7 @@ · 67 unmodified lines | |||
| 68 | * Depending on -fpatchable-function-entry=N,N usage (CONFIG_CALL_PADDING) the | 68 | * Depending on -fpatchable-function-entry=N,N usage (CONFIG_CALL_PADDING) the |
| 69 | * CFI symbol layout changes. | 69 | * CFI symbol layout changes. |
| 70 | * | 70 | * |
| 71 | * Without CALL_THUNKS: | 71 | * Without CALL_PADDING: |
| 72 | * | 72 | * |
| 73 | * .align FUNCTION_ALIGNMENT | 73 | * .align FUNCTION_ALIGNMENT |
| 74 | * __cfi_##name: | 74 | * __cfi_##name: |
| @@ -77,7 +77,7 @@ · 2 unmodified lines | |||
| 77 | * .long __kcfi_typeid_##name | 77 | * .long __kcfi_typeid_##name |
| 78 | * name: | 78 | * name: |
| 79 | * | 79 | * |
| 80 | * With CALL_THUNKS: | 80 | * With CALL_PADDING: |
| 81 | * | 81 | * |
| 82 | * .align FUNCTION_ALIGNMENT | 82 | * .align FUNCTION_ALIGNMENT |
| 83 | * __cfi_##name: | 83 | * __cfi_##name: |
| @@ -740,7 +740,10 @@ · 739 unmodified lines | |||
| 740 | #define MSR_AMD64_SNP_SMT_PROT BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT) | 740 | #define MSR_AMD64_SNP_SMT_PROT BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT) |
| 741 | #define MSR_AMD64_SNP_SECURE_AVIC_BIT 18 | 741 | #define MSR_AMD64_SNP_SECURE_AVIC_BIT 18 |
| 742 | #define MSR_AMD64_SNP_SECURE_AVIC BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT) | 742 | #define MSR_AMD64_SNP_SECURE_AVIC BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT) |
| 743 | #define MSR_AMD64_SNP_RESV_BIT 19 | 743 | #define MSR_AMD64_SNP_RESERVED_BITS19_22 GENMASK_ULL(22, 19) |
| 744 | #define MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT 23 | ||
| 745 | #define MSR_AMD64_SNP_IBPB_ON_ENTRY BIT_ULL(MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT) | ||
| 746 | #define MSR_AMD64_SNP_RESV_BIT 24 | ||
| 744 | #define MSR_AMD64_SNP_RESERVED_MASK GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT) | 747 | #define MSR_AMD64_SNP_RESERVED_MASK GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT) |
| 745 | #define MSR_AMD64_SAVIC_CONTROL 0xc0010138 | 748 | #define MSR_AMD64_SAVIC_CONTROL 0xc0010138 |
| 746 | #define MSR_AMD64_SAVIC_EN_BIT 0 | 749 | #define MSR_AMD64_SAVIC_EN_BIT 0 |
| @@ -22,6 +22,7 @@ extern int numa_off; · 21 unmodified lines | |||
| 22 | */ | 22 | */ |
| 23 | extern s16 __apicid_to_node[MAX_LOCAL_APIC]; | 23 | extern s16 __apicid_to_node[MAX_LOCAL_APIC]; |
| 24 | extern nodemask_t numa_nodes_parsed __initdata; | 24 | extern nodemask_t numa_nodes_parsed __initdata; |
| 25 | extern nodemask_t numa_phys_nodes_parsed __initdata; | ||
| 25 | 26 | ||
| 26 | static inline void set_apicid_to_node(int apicid, s16 node) | 27 | static inline void set_apicid_to_node(int apicid, s16 node) |
| 27 | { | 28 | { |
| @@ -48,6 +49,7 @@ extern void __init init_cpu_to_node(void); · 20 unmodified lines | |||
| 48 | extern void numa_add_cpu(unsigned int cpu); | 49 | extern void numa_add_cpu(unsigned int cpu); |
| 49 | extern void numa_remove_cpu(unsigned int cpu); | 50 | extern void numa_remove_cpu(unsigned int cpu); |
| 50 | extern void init_gi_nodes(void); | 51 | extern void init_gi_nodes(void); |
| 52 | extern int num_phys_nodes(void); | ||
| 51 | #else /* CONFIG_NUMA */ | 53 | #else /* CONFIG_NUMA */ |
| 52 | static inline void numa_set_node(int cpu, int node) { } | 54 | static inline void numa_set_node(int cpu, int node) { } |
| 53 | static inline void numa_clear_node(int cpu) { } | 55 | static inline void numa_clear_node(int cpu) { } |
| @@ -55,6 +57,10 @@ static inline void init_cpu_to_node(void) { } · 1 unmodified line | |||
| 55 | static inline void numa_add_cpu(unsigned int cpu) { } | 57 | static inline void numa_add_cpu(unsigned int cpu) { } |
| 56 | static inline void numa_remove_cpu(unsigned int cpu) { } | 58 | static inline void numa_remove_cpu(unsigned int cpu) { } |
| 57 | static inline void init_gi_nodes(void) { } | 59 | static inline void init_gi_nodes(void) { } |
| 60 | static inline int num_phys_nodes(void) | ||
| 61 | { | ||
| 62 | return 1; | ||
| 63 | } | ||
| 58 | #endif /* CONFIG_NUMA */ | 64 | #endif /* CONFIG_NUMA */ |
| 59 | 65 | ||
| 60 | #ifdef CONFIG_DEBUG_PER_CPU_MAPS | 66 | #ifdef CONFIG_DEBUG_PER_CPU_MAPS |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | |||
| 20 | 20 | ||
| 21 | #define PER_CPU_VAR(var) __percpu(var)__percpu_rel | 21 | #define PER_CPU_VAR(var) __percpu(var)__percpu_rel |
| 22 | 22 | ||
| 23 | #else /* !__ASSEMBLY__: */ | 23 | #else /* !__ASSEMBLER__: */ |
| 24 | 24 | ||
| 25 | #include <linux/args.h> | 25 | #include <linux/args.h> |
| 26 | #include <linux/bits.h> | 26 | #include <linux/bits.h> |
| @@ -19,10 +19,8 @@ · 18 unmodified lines | |||
| 19 | extern p4d_t level4_kernel_pgt[512]; | 19 | extern p4d_t level4_kernel_pgt[512]; |
| 20 | extern p4d_t level4_ident_pgt[512]; | 20 | extern p4d_t level4_ident_pgt[512]; |
| 21 | extern pud_t level3_kernel_pgt[512]; | 21 | extern pud_t level3_kernel_pgt[512]; |
| 22 | extern pud_t level3_ident_pgt[512]; | ||
| 23 | extern pmd_t level2_kernel_pgt[512]; | 22 | extern pmd_t level2_kernel_pgt[512]; |
| 24 | extern pmd_t level2_fixmap_pgt[512]; | 23 | extern pmd_t level2_fixmap_pgt[512]; |
| 25 | extern pmd_t level2_ident_pgt[512]; | ||
| 26 | extern pte_t level1_fixmap_pgt[512 * FIXMAP_PMD_NUM]; | 24 | extern pte_t level1_fixmap_pgt[512 * FIXMAP_PMD_NUM]; |
| 27 | extern pgd_t init_top_pgt[]; | 25 | extern pgd_t init_top_pgt[]; |
| 28 | 26 | ||
| @@ -6,7 +6,7 @@ · 5 unmodified lines | |||
| 6 | #error "Cannot use runtime-const infrastructure from modules" | 6 | #error "Cannot use runtime-const infrastructure from modules" |
| 7 | #endif | 7 | #endif |
| 8 | 8 | ||
| 9 | #ifdef __ASSEMBLY__ | 9 | #ifdef __ASSEMBLER__ |
| 10 | 10 | ||
| 11 | .macro RUNTIME_CONST_PTR sym reg | 11 | .macro RUNTIME_CONST_PTR sym reg |
| 12 | movq $0x0123456789abcdef, %\reg | 12 | movq $0x0123456789abcdef, %\reg |
| @@ -16,7 +16,7 @@ · 3 unmodified lines | |||
| 16 | .popsection | 16 | .popsection |
| 17 | .endm | 17 | .endm |
| 18 | 18 | ||
| 19 | #else /* __ASSEMBLY__ */ | 19 | #else /* __ASSEMBLER__ */ |
| 20 | 20 | ||
| 21 | #define runtime_const_ptr(sym) ({ \ | 21 | #define runtime_const_ptr(sym) ({ \ |
| 22 | typeof(sym) __ret; \ | 22 | typeof(sym) __ret; \ |
| @@ -74,5 +74,5 @@ static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), · 51 unmodified lines | |||
| 74 | } | 74 | } |
| 75 | } | 75 | } |
| 76 | 76 | ||
| 77 | #endif /* __ASSEMBLY__ */ | 77 | #endif /* __ASSEMBLER__ */ |
| 78 | #endif | 78 | #endif |
| @@ -155,6 +155,7 @@ extern unsigned int __max_logical_packages; · 154 unmodified lines | |||
| 155 | extern unsigned int __max_threads_per_core; | 155 | extern unsigned int __max_threads_per_core; |
| 156 | extern unsigned int __num_threads_per_package; | 156 | extern unsigned int __num_threads_per_package; |
| 157 | extern unsigned int __num_cores_per_package; | 157 | extern unsigned int __num_cores_per_package; |
| 158 | extern unsigned int __num_nodes_per_package; | ||
| 158 | 159 | ||
| 159 | const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c); | 160 | const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c); |
| 160 | enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c); | 161 | enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c); |
| @@ -179,6 +180,11 @@ static inline unsigned int topology_num_threads_per_package(void) · 18 unmodified lines | |||
| 179 | return __num_threads_per_package; | 180 | return __num_threads_per_package; |
| 180 | } | 181 | } |
| 181 | 182 | ||
| 183 | static inline unsigned int topology_num_nodes_per_package(void) | ||
| 184 | { | ||
| 185 | return __num_nodes_per_package; | ||
| 186 | } | ||
| 187 | |||
| 182 | #ifdef CONFIG_X86_LOCAL_APIC | 188 | #ifdef CONFIG_X86_LOCAL_APIC |
| 183 | int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level); | 189 | int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level); |
| 184 | #else | 190 | #else |
| @@ -25,6 +25,8 @@ extern int ibt_selftest_noendbr(void); · 24 unmodified lines | |||
| 25 | void handle_invalid_op(struct pt_regs *regs); | 25 | void handle_invalid_op(struct pt_regs *regs); |
| 26 | #endif | 26 | #endif |
| 27 | 27 | ||
| 28 | noinstr bool handle_bug(struct pt_regs *regs); | ||
| 29 | |||
| 28 | static inline int get_si_code(unsigned long condition) | 30 | static inline int get_si_code(unsigned long condition) |
| 29 | { | 31 | { |
| 30 | if (condition & DR_STEP) | 32 | if (condition & DR_STEP) |
| @@ -476,6 +476,7 @@ struct kvm_sync_regs { · 475 unmodified lines | |||
| 476 | #define KVM_X86_QUIRK_SLOT_ZAP_ALL (1 << 7) | 476 | #define KVM_X86_QUIRK_SLOT_ZAP_ALL (1 << 7) |
| 477 | #define KVM_X86_QUIRK_STUFF_FEATURE_MSRS (1 << 8) | 477 | #define KVM_X86_QUIRK_STUFF_FEATURE_MSRS (1 << 8) |
| 478 | #define KVM_X86_QUIRK_IGNORE_GUEST_PAT (1 << 9) | 478 | #define KVM_X86_QUIRK_IGNORE_GUEST_PAT (1 << 9) |
| 479 | #define KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM (1 << 10) | ||
| 479 | 480 | ||
| 480 | #define KVM_STATE_NESTED_FORMAT_VMX 0 | 481 | #define KVM_STATE_NESTED_FORMAT_VMX 0 |
| 481 | #define KVM_STATE_NESTED_FORMAT_SVM 1 | 482 | #define KVM_STATE_NESTED_FORMAT_SVM 1 |
| @@ -1182,7 +1182,7 @@ void __init_or_module noinline apply_seal_endbr(s32 *start, s32 *end) · 1181 unmodified lines | |||
| 1182 | 1182 | ||
| 1183 | poison_endbr(addr); | 1183 | poison_endbr(addr); |
| 1184 | if (IS_ENABLED(CONFIG_FINEIBT)) | 1184 | if (IS_ENABLED(CONFIG_FINEIBT)) |
| 1185 | poison_cfi(addr - 16); | 1185 | poison_cfi(addr - CFI_OFFSET); |
| 1186 | } | 1186 | } |
| 1187 | } | 1187 | } |
| 1188 | 1188 | ||
| @@ -1389,6 +1389,8 @@ extern u8 fineibt_preamble_end[]; · 200 unmodified lines | |||
| 1389 | #define fineibt_preamble_ud 0x13 | 1389 | #define fineibt_preamble_ud 0x13 |
| 1390 | #define fineibt_preamble_hash 5 | 1390 | #define fineibt_preamble_hash 5 |
| 1391 | 1391 | ||
| 1392 | #define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE) | ||
| 1393 | |||
| 1392 | /* | 1394 | /* |
| 1393 | * <fineibt_caller_start>: | 1395 | * <fineibt_caller_start>: |
| 1394 | * 0: b8 78 56 34 12 mov $0x12345678, %eax | 1396 | * 0: b8 78 56 34 12 mov $0x12345678, %eax |
| @@ -1634,7 +1636,7 @@ static int cfi_rewrite_preamble(s32 *start, s32 *end) · 239 unmodified lines | |||
| 1634 | * have determined there are no indirect calls to it and we | 1636 | * have determined there are no indirect calls to it and we |
| 1635 | * don't need no CFI either. | 1637 | * don't need no CFI either. |
| 1636 | */ | 1638 | */ |
| 1637 | if (!is_endbr(addr + 16)) | 1639 | if (!is_endbr(addr + CFI_OFFSET)) |
| 1638 | continue; | 1640 | continue; |
| 1639 | 1641 | ||
| 1640 | hash = decode_preamble_hash(addr, &arity); | 1642 | hash = decode_preamble_hash(addr, &arity); |
| @@ -1642,6 +1644,15 @@ static int cfi_rewrite_preamble(s32 *start, s32 *end) · 1 unmodified line | |||
| 1642 | addr, addr, 5, addr)) | 1644 | addr, addr, 5, addr)) |
| 1643 | return -EINVAL; | 1645 | return -EINVAL; |
| 1644 | 1646 | ||
| 1647 | /* | ||
| 1648 | * FineIBT relies on being at func-16, so if the preamble is | ||
| 1649 | * actually larger than that, place it the tail end. | ||
| 1650 | * | ||
| 1651 | * NOTE: this is possible with things like DEBUG_CALL_THUNKS | ||
| 1652 | * and DEBUG_FORCE_FUNCTION_ALIGN_64B. | ||
| 1653 | */ | ||
| 1654 | addr += CFI_OFFSET - fineibt_prefix_size; | ||
| 1655 | |||
| 1645 | text_poke_early(addr, fineibt_preamble_start, fineibt_preamble_size); | 1656 | text_poke_early(addr, fineibt_preamble_start, fineibt_preamble_size); |
| 1646 | WARN_ON(*(u32 *)(addr + fineibt_preamble_hash) != 0x12345678); | 1657 | WARN_ON(*(u32 *)(addr + fineibt_preamble_hash) != 0x12345678); |
| 1647 | text_poke_early(addr + fineibt_preamble_hash, &hash, 4); | 1658 | text_poke_early(addr + fineibt_preamble_hash, &hash, 4); |
| @@ -1664,10 +1675,10 @@ static void cfi_rewrite_endbr(s32 *start, s32 *end) · 16 unmodified lines | |||
| 1664 | for (s = start; s < end; s++) { | 1675 | for (s = start; s < end; s++) { |
| 1665 | void *addr = (void *)s + *s; | 1676 | void *addr = (void *)s + *s; |
| 1666 | 1677 | ||
| 1667 | if (!exact_endbr(addr + 16)) | 1678 | if (!exact_endbr(addr + CFI_OFFSET)) |
| 1668 | continue; | 1679 | continue; |
| 1669 | 1680 | ||
| 1670 | poison_endbr(addr + 16); | 1681 | poison_endbr(addr + CFI_OFFSET); |
| 1671 | } | 1682 | } |
| 1672 | } | 1683 | } |
| 1673 | 1684 | ||
| @@ -1772,7 +1783,8 @@ static void __apply_fineibt(s32 *start_retpoline, s32 *end_retpoline, · 98 unmodified lines | |||
| 1772 | if (FINEIBT_WARN(fineibt_preamble_size, 20) || | 1783 | if (FINEIBT_WARN(fineibt_preamble_size, 20) || |
| 1773 | FINEIBT_WARN(fineibt_preamble_bhi + fineibt_bhi1_size, 20) || | 1784 | FINEIBT_WARN(fineibt_preamble_bhi + fineibt_bhi1_size, 20) || |
| 1774 | FINEIBT_WARN(fineibt_caller_size, 14) || | 1785 | FINEIBT_WARN(fineibt_caller_size, 14) || |
| 1775 | FINEIBT_WARN(fineibt_paranoid_size, 20)) | 1786 | FINEIBT_WARN(fineibt_paranoid_size, 20) || |
| 1787 | WARN_ON_ONCE(CFI_OFFSET < fineibt_prefix_size)) | ||
| 1776 | return; | 1788 | return; |
| 1777 | 1789 | ||
| 1778 | if (cfi_mode == CFI_AUTO) { | 1790 | if (cfi_mode == CFI_AUTO) { |
| @@ -1885,6 +1897,11 @@ static void poison_cfi(void *addr) · 106 unmodified lines | |||
| 1885 | */ | 1897 | */ |
| 1886 | switch (cfi_mode) { | 1898 | switch (cfi_mode) { |
| 1887 | case CFI_FINEIBT: | 1899 | case CFI_FINEIBT: |
| 1900 | /* | ||
| 1901 | * FineIBT preamble is at func-16. | ||
| 1902 | */ | ||
| 1903 | addr += CFI_OFFSET - fineibt_prefix_size; | ||
| 1904 | |||
| 1888 | /* | 1905 | /* |
| 1889 | * FineIBT prefix should start with an ENDBR. | 1906 | * FineIBT prefix should start with an ENDBR. |
| 1890 | */ | 1907 | */ |
| @@ -1923,8 +1940,6 @@ static void poison_cfi(void *addr) · 32 unmodified lines | |||
| 1923 | } | 1940 | } |
| 1924 | } | 1941 | } |
| 1925 | 1942 | ||
| 1926 | #define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE) | ||
| 1927 | |||
| 1928 | /* | 1943 | /* |
| 1929 | * When regs->ip points to a 0xD6 byte in the FineIBT preamble, | 1944 | * When regs->ip points to a 0xD6 byte in the FineIBT preamble, |
| 1930 | * return true and fill out target and type. | 1945 | * return true and fill out target and type. |
| @@ -1894,6 +1894,7 @@ void __init check_x2apic(void) · 1893 unmodified lines | |||
| 1894 | 1894 | ||
| 1895 | static inline void try_to_enable_x2apic(int remap_mode) { } | 1895 | static inline void try_to_enable_x2apic(int remap_mode) { } |
| 1896 | static inline void __x2apic_enable(void) { } | 1896 | static inline void __x2apic_enable(void) { } |
| 1897 | static inline void __x2apic_disable(void) { } | ||
| 1897 | #endif /* !CONFIG_X86_X2APIC */ | 1898 | #endif /* !CONFIG_X86_X2APIC */ |
| 1898 | 1899 | ||
| 1899 | void __init enable_IR_x2apic(void) | 1900 | void __init enable_IR_x2apic(void) |
| @@ -2456,6 +2457,11 @@ static void lapic_resume(void *data) · 556 unmodified lines | |||
| 2456 | if (x2apic_mode) { | 2457 | if (x2apic_mode) { |
| 2457 | __x2apic_enable(); | 2458 | __x2apic_enable(); |
| 2458 | } else { | 2459 | } else { |
| 2460 | if (x2apic_enabled()) { | ||
| 2461 | pr_warn_once("x2apic: re-enabled by firmware during resume. Disabling\n"); | ||
| 2462 | __x2apic_disable(); | ||
| 2463 | } | ||
| 2464 | |||
| 2459 | /* | 2465 | /* |
| 2460 | * Make sure the APICBASE points to the right address | 2466 | * Make sure the APICBASE points to the right address |
| 2461 | * | 2467 | * |
| @@ -95,6 +95,9 @@ EXPORT_SYMBOL(__max_dies_per_package); · 94 unmodified lines | |||
| 95 | unsigned int __max_logical_packages __ro_after_init = 1; | 95 | unsigned int __max_logical_packages __ro_after_init = 1; |
| 96 | EXPORT_SYMBOL(__max_logical_packages); | 96 | EXPORT_SYMBOL(__max_logical_packages); |
| 97 | 97 | ||
| 98 | unsigned int __num_nodes_per_package __ro_after_init = 1; | ||
| 99 | EXPORT_SYMBOL(__num_nodes_per_package); | ||
| 100 | |||
| 98 | unsigned int __num_cores_per_package __ro_after_init = 1; | 101 | unsigned int __num_cores_per_package __ro_after_init = 1; |
| 99 | EXPORT_SYMBOL(__num_cores_per_package); | 102 | EXPORT_SYMBOL(__num_cores_per_package); |
| 100 | 103 | ||
| @@ -364,7 +364,7 @@ void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d) · 363 unmodified lines | |||
| 364 | msr_clear_bit(MSR_RMID_SNC_CONFIG, 0); | 364 | msr_clear_bit(MSR_RMID_SNC_CONFIG, 0); |
| 365 | } | 365 | } |
| 366 | 366 | ||
| 367 | /* CPU models that support MSR_RMID_SNC_CONFIG */ | 367 | /* CPU models that support SNC and MSR_RMID_SNC_CONFIG */ |
| 368 | static const struct x86_cpu_id snc_cpu_ids[] __initconst = { | 368 | static const struct x86_cpu_id snc_cpu_ids[] __initconst = { |
| 369 | X86_MATCH_VFM(INTEL_ICELAKE_X, 0), | 369 | X86_MATCH_VFM(INTEL_ICELAKE_X, 0), |
| 370 | X86_MATCH_VFM(INTEL_SAPPHIRERAPIDS_X, 0), | 370 | X86_MATCH_VFM(INTEL_SAPPHIRERAPIDS_X, 0), |
| @@ -375,40 +375,14 @@ static const struct x86_cpu_id snc_cpu_ids[] __initconst = { · 4 unmodified lines | |||
| 375 | {} | 375 | {} |
| 376 | }; | 376 | }; |
| 377 | 377 | ||
| 378 | /* | ||
| 379 | * There isn't a simple hardware bit that indicates whether a CPU is running | ||
| 380 | * in Sub-NUMA Cluster (SNC) mode. Infer the state by comparing the | ||
| 381 | * number of CPUs sharing the L3 cache with CPU0 to the number of CPUs in | ||
| 382 | * the same NUMA node as CPU0. | ||
| 383 | * It is not possible to accurately determine SNC state if the system is | ||
| 384 | * booted with a maxcpus=N parameter. That distorts the ratio of SNC nodes | ||
| 385 | * to L3 caches. It will be OK if system is booted with hyperthreading | ||
| 386 | * disabled (since this doesn't affect the ratio). | ||
| 387 | */ | ||
| 388 | static __init int snc_get_config(void) | 378 | static __init int snc_get_config(void) |
| 389 | { | 379 | { |
| 390 | struct cacheinfo *ci = get_cpu_cacheinfo_level(0, RESCTRL_L3_CACHE); | 380 | int ret = topology_num_nodes_per_package(); |
| 391 | const cpumask_t *node0_cpumask; | ||
| 392 | int cpus_per_node, cpus_per_l3; | ||
| 393 | int ret; | ||
| 394 | |||
| 395 | if (!x86_match_cpu(snc_cpu_ids) || !ci) | ||
| 396 | return 1; | ||
| 397 | 381 | ||
| 398 | cpus_read_lock(); | 382 | if (ret > 1 && !x86_match_cpu(snc_cpu_ids)) { |
| 399 | if (num_online_cpus() != num_present_cpus()) | 383 | pr_warn("CoD enabled system? Resctrl not supported\n"); |
| 400 | pr_warn("Some CPUs offline, SNC detection may be incorrect\n"); | ||
| 401 | cpus_read_unlock(); | ||
| 402 | |||
| 403 | node0_cpumask = cpumask_of_node(cpu_to_node(0)); | ||
| 404 | |||
| 405 | cpus_per_node = cpumask_weight(node0_cpumask); | ||
| 406 | cpus_per_l3 = cpumask_weight(&ci->shared_cpu_map); | ||
| 407 | |||
| 408 | if (!cpus_per_node || !cpus_per_l3) | ||
| 409 | return 1; | 384 | return 1; |
| 410 | 385 | } | |
| 411 | ret = cpus_per_l3 / cpus_per_node; | ||
| 412 | 386 | ||
| 413 | /* sanity check: Only valid results are 1, 2, 3, 4, 6 */ | 387 | /* sanity check: Only valid results are 1, 2, 3, 4, 6 */ |
| 414 | switch (ret) { | 388 | switch (ret) { |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | |||
| 31 | #include <asm/mpspec.h> | 31 | #include <asm/mpspec.h> |
| 32 | #include <asm/msr.h> | 32 | #include <asm/msr.h> |
| 33 | #include <asm/smp.h> | 33 | #include <asm/smp.h> |
| 34 | #include <asm/numa.h> | ||
| 34 | 35 | ||
| 35 | #include "cpu.h" | 36 | #include "cpu.h" |
| 36 | 37 | ||
| @@ -492,11 +493,19 @@ void __init topology_init_possible_cpus(void) · 455 unmodified lines | |||
| 492 | set_nr_cpu_ids(allowed); | 493 | set_nr_cpu_ids(allowed); |
| 493 | 494 | ||
| 494 | cnta = domain_weight(TOPO_PKG_DOMAIN); | 495 | cnta = domain_weight(TOPO_PKG_DOMAIN); |
| 495 | cntb = domain_weight(TOPO_DIE_DOMAIN); | ||
| 496 | __max_logical_packages = cnta; | 496 | __max_logical_packages = cnta; |
| 497 | |||
| 498 | pr_info("Max. logical packages: %3u\n", __max_logical_packages); | ||
| 499 | |||
| 500 | cntb = num_phys_nodes(); | ||
| 501 | __num_nodes_per_package = DIV_ROUND_UP(cntb, cnta); | ||
| 502 | |||
| 503 | pr_info("Max. logical nodes: %3u\n", cntb); | ||
| 504 | pr_info("Num. nodes per package:%3u\n", __num_nodes_per_package); | ||
| 505 | |||
| 506 | cntb = domain_weight(TOPO_DIE_DOMAIN); | ||
| 497 | __max_dies_per_package = 1U << (get_count_order(cntb) - get_count_order(cnta)); | 507 | __max_dies_per_package = 1U << (get_count_order(cntb) - get_count_order(cnta)); |
| 498 | 508 | ||
| 499 | pr_info("Max. logical packages: %3u\n", cnta); | ||
| 500 | pr_info("Max. logical dies: %3u\n", cntb); | 509 | pr_info("Max. logical dies: %3u\n", cntb); |
| 501 | pr_info("Max. dies per package: %3u\n", __max_dies_per_package); | 510 | pr_info("Max. dies per package: %3u\n", __max_dies_per_package); |
| 502 | 511 | ||
| @@ -616,38 +616,10 @@ SYM_DATA(early_recursion_flag, .long 0) · 615 unmodified lines | |||
| 616 | 616 | ||
| 617 | .data | 617 | .data |
| 618 | 618 | ||
| 619 | #if defined(CONFIG_XEN_PV) || defined(CONFIG_PVH) | ||
| 620 | SYM_DATA_START_PTI_ALIGNED(init_top_pgt) | ||
| 621 | .quad level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | ||
| 622 | .org init_top_pgt + L4_PAGE_OFFSET*8, 0 | ||
| 623 | .quad level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | ||
| 624 | .org init_top_pgt + L4_START_KERNEL*8, 0 | ||
| 625 | /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ | ||
| 626 | .quad level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC | ||
| 627 | .fill PTI_USER_PGD_FILL,8,0 | ||
| 628 | SYM_DATA_END(init_top_pgt) | ||
| 629 | |||
| 630 | SYM_DATA_START_PAGE_ALIGNED(level3_ident_pgt) | ||
| 631 | .quad level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | ||
| 632 | .fill 511, 8, 0 | ||
| 633 | SYM_DATA_END(level3_ident_pgt) | ||
| 634 | SYM_DATA_START_PAGE_ALIGNED(level2_ident_pgt) | ||
| 635 | /* | ||
| 636 | * Since I easily can, map the first 1G. | ||
| 637 | * Don't set NX because code runs from these pages. | ||
| 638 | * | ||
| 639 | * Note: This sets _PAGE_GLOBAL despite whether | ||
| 640 | * the CPU supports it or it is enabled. But, | ||
| 641 | * the CPU should ignore the bit. | ||
| 642 | */ | ||
| 643 | PMDS(0, __PAGE_KERNEL_IDENT_LARGE_EXEC, PTRS_PER_PMD) | ||
| 644 | SYM_DATA_END(level2_ident_pgt) | ||
| 645 | #else | ||
| 646 | SYM_DATA_START_PTI_ALIGNED(init_top_pgt) | 619 | SYM_DATA_START_PTI_ALIGNED(init_top_pgt) |
| 647 | .fill 512,8,0 | 620 | .fill 512,8,0 |
| 648 | .fill PTI_USER_PGD_FILL,8,0 | 621 | .fill PTI_USER_PGD_FILL,8,0 |
| 649 | SYM_DATA_END(init_top_pgt) | 622 | SYM_DATA_END(init_top_pgt) |
| 650 | #endif | ||
| 651 | 623 | ||
| 652 | SYM_DATA_START_PAGE_ALIGNED(level4_kernel_pgt) | 624 | SYM_DATA_START_PAGE_ALIGNED(level4_kernel_pgt) |
| 653 | .fill 511,8,0 | 625 | .fill 511,8,0 |
| @@ -468,13 +468,6 @@ static int x86_cluster_flags(void) · 467 unmodified lines | |||
| 468 | } | 468 | } |
| 469 | #endif | 469 | #endif |
| 470 | 470 | ||
| 471 | /* | ||
| 472 | * Set if a package/die has multiple NUMA nodes inside. | ||
| 473 | * AMD Magny-Cours, Intel Cluster-on-Die, and Intel | ||
| 474 | * Sub-NUMA Clustering have this. | ||
| 475 | */ | ||
| 476 | static bool x86_has_numa_in_package; | ||
| 477 | |||
| 478 | static struct sched_domain_topology_level x86_topology[] = { | 471 | static struct sched_domain_topology_level x86_topology[] = { |
| 479 | SDTL_INIT(tl_smt_mask, cpu_smt_flags, SMT), | 472 | SDTL_INIT(tl_smt_mask, cpu_smt_flags, SMT), |
| 480 | #ifdef CONFIG_SCHED_CLUSTER | 473 | #ifdef CONFIG_SCHED_CLUSTER |
| @@ -496,7 +489,7 @@ static void __init build_sched_topology(void) · 15 unmodified lines | |||
| 496 | * PKG domain since the NUMA domains will auto-magically create the | 489 | * PKG domain since the NUMA domains will auto-magically create the |
| 497 | * right spanning domains based on the SLIT. | 490 | * right spanning domains based on the SLIT. |
| 498 | */ | 491 | */ |
| 499 | if (x86_has_numa_in_package) { | 492 | if (topology_num_nodes_per_package() > 1) { |
| 500 | unsigned int pkgdom = ARRAY_SIZE(x86_topology) - 2; | 493 | unsigned int pkgdom = ARRAY_SIZE(x86_topology) - 2; |
| 501 | 494 | ||
| 502 | memset(&x86_topology[pkgdom], 0, sizeof(x86_topology[pkgdom])); | 495 | memset(&x86_topology[pkgdom], 0, sizeof(x86_topology[pkgdom])); |
| @@ -513,33 +506,149 @@ static void __init build_sched_topology(void) · 10 unmodified lines | |||
| 513 | } | 506 | } |
| 514 | 507 | ||
| 515 | #ifdef CONFIG_NUMA | 508 | #ifdef CONFIG_NUMA |
| 516 | static int sched_avg_remote_distance; | 509 | /* |
| 517 | static int avg_remote_numa_distance(void) | 510 | * Test if the on-trace cluster at (N,N) is symmetric. |
| 511 | * Uses upper triangle iteration to avoid obvious duplicates. | ||
| 512 | */ | ||
| 513 | static bool slit_cluster_symmetric(int N) | ||
| 518 | { | 514 | { |
| 519 | int i, j; | 515 | int u = topology_num_nodes_per_package(); |
| 520 | int distance, nr_remote, total_distance; | 516 | |
| 521 | 517 | for (int k = 0; k < u; k++) { | |
| 522 | if (sched_avg_remote_distance > 0) | 518 | for (int l = k; l < u; l++) { |
| 523 | return sched_avg_remote_distance; | 519 | if (node_distance(N + k, N + l) != |
| 524 | 520 | node_distance(N + l, N + k)) | |
| 525 | nr_remote = 0; | 521 | return false; |
| 526 | total_distance = 0; | ||
| 527 | for_each_node_state(i, N_CPU) { | ||
| 528 | for_each_node_state(j, N_CPU) { | ||
| 529 | distance = node_distance(i, j); | ||
| 530 | |||
| 531 | if (distance >= REMOTE_DISTANCE) { | ||
| 532 | nr_remote++; | ||
| 533 | total_distance += distance; | ||
| 534 | } | ||
| 535 | } | 522 | } |
| 536 | } | 523 | } |
| 537 | if (nr_remote) | ||
| 538 | sched_avg_remote_distance = total_distance / nr_remote; | ||
| 539 | else | ||
| 540 | sched_avg_remote_distance = REMOTE_DISTANCE; | ||
| 541 | 524 | ||
| 542 | return sched_avg_remote_distance; | 525 | return true; |
| 526 | } | ||
| 527 | |||
| 528 | /* | ||
| 529 | * Return the package-id of the cluster, or ~0 if indeterminate. | ||
| 530 | * Each node in the on-trace cluster should have the same package-id. | ||
| 531 | */ | ||
| 532 | static u32 slit_cluster_package(int N) | ||
| 533 | { | ||
| 534 | int u = topology_num_nodes_per_package(); | ||
| 535 | u32 pkg_id = ~0; | ||
| 536 | |||
| 537 | for (int n = 0; n < u; n++) { | ||
| 538 | const struct cpumask *cpus = cpumask_of_node(N + n); | ||
| 539 | int cpu; | ||
| 540 | |||
| 541 | for_each_cpu(cpu, cpus) { | ||
| 542 | u32 id = topology_logical_package_id(cpu); | ||
| 543 | |||
| 544 | if (pkg_id == ~0) | ||
| 545 | pkg_id = id; | ||
| 546 | if (pkg_id != id) | ||
| 547 | return ~0; | ||
| 548 | } | ||
| 549 | } | ||
| 550 | |||
| 551 | return pkg_id; | ||
| 552 | } | ||
| 553 | |||
| 554 | /* | ||
| 555 | * Validate the SLIT table is of the form expected for SNC, specifically: | ||
| 556 | * | ||
| 557 | * - each on-trace cluster should be symmetric, | ||
| 558 | * - each on-trace cluster should have a unique package-id. | ||
| 559 | * | ||
| 560 | * If you NUMA_EMU on top of SNC, you get to keep the pieces. | ||
| 561 | */ | ||
| 562 | static bool slit_validate(void) | ||
| 563 | { | ||
| 564 | int u = topology_num_nodes_per_package(); | ||
| 565 | u32 pkg_id, prev_pkg_id = ~0; | ||
| 566 | |||
| 567 | for (int pkg = 0; pkg < topology_max_packages(); pkg++) { | ||
| 568 | int n = pkg * u; | ||
| 569 | |||
| 570 | /* | ||
| 571 | * Ensure the on-trace cluster is symmetric and each cluster | ||
| 572 | * has a different package id. | ||
| 573 | */ | ||
| 574 | if (!slit_cluster_symmetric(n)) | ||
| 575 | return false; | ||
| 576 | pkg_id = slit_cluster_package(n); | ||
| 577 | if (pkg_id == ~0) | ||
| 578 | return false; | ||
| 579 | if (pkg && pkg_id == prev_pkg_id) | ||
| 580 | return false; | ||
| 581 | |||
| 582 | prev_pkg_id = pkg_id; | ||
| 583 | } | ||
| 584 | |||
| 585 | return true; | ||
| 586 | } | ||
| 587 | |||
| 588 | /* | ||
| 589 | * Compute a sanitized SLIT table for SNC; notably SNC-3 can end up with | ||
| 590 | * asymmetric off-trace clusters, reflecting physical assymmetries. However | ||
| 591 | * this leads to 'unfortunate' sched_domain configurations. | ||
| 592 | * | ||
| 593 | * For example dual socket GNR with SNC-3: | ||
| 594 | * | ||
| 595 | * node distances: | ||
| 596 | * node 0 1 2 3 4 5 | ||
| 597 | * 0: 10 15 17 21 28 26 | ||
| 598 | * 1: 15 10 15 23 26 23 | ||
| 599 | * 2: 17 15 10 26 23 21 | ||
| 600 | * 3: 21 28 26 10 15 17 | ||
| 601 | * 4: 23 26 23 15 10 15 | ||
| 602 | * 5: 26 23 21 17 15 10 | ||
| 603 | * | ||
| 604 | * Fix things up by averaging out the off-trace clusters; resulting in: | ||
| 605 | * | ||
| 606 | * node 0 1 2 3 4 5 | ||
| 607 | * 0: 10 15 17 24 24 24 | ||
| 608 | * 1: 15 10 15 24 24 24 | ||
| 609 | * 2: 17 15 10 24 24 24 | ||
| 610 | * 3: 24 24 24 10 15 17 | ||
| 611 | * 4: 24 24 24 15 10 15 | ||
| 612 | * 5: 24 24 24 17 15 10 | ||
| 613 | */ | ||
| 614 | static int slit_cluster_distance(int i, int j) | ||
| 615 | { | ||
| 616 | static int slit_valid = -1; | ||
| 617 | int u = topology_num_nodes_per_package(); | ||
| 618 | long d = 0; | ||
| 619 | int x, y; | ||
| 620 | |||
| 621 | if (slit_valid < 0) { | ||
| 622 | slit_valid = slit_validate(); | ||
| 623 | if (!slit_valid) | ||
| 624 | pr_err(FW_BUG "SLIT table doesn't have the expected form for SNC -- fixup disabled!\n"); | ||
| 625 | else | ||
| 626 | pr_info("Fixing up SNC SLIT table.\n"); | ||
| 627 | } | ||
| 628 | |||
| 629 | /* | ||
| 630 | * Is this a unit cluster on the trace? | ||
| 631 | */ | ||
| 632 | if ((i / u) == (j / u) || !slit_valid) | ||
| 633 | return node_distance(i, j); | ||
| 634 | |||
| 635 | /* | ||
| 636 | * Off-trace cluster. | ||
| 637 | * | ||
| 638 | * Notably average out the symmetric pair of off-trace clusters to | ||
| 639 | * ensure the resulting SLIT table is symmetric. | ||
| 640 | */ | ||
| 641 | x = i - (i % u); | ||
| 642 | y = j - (j % u); | ||
| 643 | |||
| 644 | for (i = x; i < x + u; i++) { | ||
| 645 | for (j = y; j < y + u; j++) { | ||
| 646 | d += node_distance(i, j); | ||
| 647 | d += node_distance(j, i); | ||
| 648 | } | ||
| 649 | } | ||
| 650 | |||
| 651 | return d / (2*u*u); | ||
| 543 | } | 652 | } |
| 544 | 653 | ||
| 545 | int arch_sched_node_distance(int from, int to) | 654 | int arch_sched_node_distance(int from, int to) |
| @@ -549,34 +658,14 @@ int arch_sched_node_distance(int from, int to) · 3 unmodified lines | |||
| 549 | switch (boot_cpu_data.x86_vfm) { | 658 | switch (boot_cpu_data.x86_vfm) { |
| 550 | case INTEL_GRANITERAPIDS_X: | 659 | case INTEL_GRANITERAPIDS_X: |
| 551 | case INTEL_ATOM_DARKMONT_X: | 660 | case INTEL_ATOM_DARKMONT_X: |
| 552 | 661 | if (topology_max_packages() == 1 || | |
| 553 | if (!x86_has_numa_in_package || topology_max_packages() == 1 || | 662 | topology_num_nodes_per_package() < 3) |
| 554 | d < REMOTE_DISTANCE) | ||
| 555 | return d; | 663 | return d; |
| 556 | 664 | ||
| 557 | /* | 665 | /* |
| 558 | * With SNC enabled, there could be too many levels of remote | 666 | * Handle SNC-3 asymmetries. |
| 559 | * NUMA node distances, creating NUMA domain levels | ||
| 560 | * including local nodes and partial remote nodes. | ||
| 561 | * | ||
| 562 | * Trim finer distance tuning for NUMA nodes in remote package | ||
| 563 | * for the purpose of building sched domains. Group NUMA nodes | ||
| 564 | * in the remote package in the same sched group. | ||
| 565 | * Simplify NUMA domains and avoid extra NUMA levels including | ||
| 566 | * different remote NUMA nodes and local nodes. | ||
| 567 | * | ||
| 568 | * GNR and CWF don't expect systems with more than 2 packages | ||
| 569 | * and more than 2 hops between packages. Single average remote | ||
| 570 | * distance won't be appropriate if there are more than 2 | ||
| 571 | * packages as average distance to different remote packages | ||
| 572 | * could be different. | ||
| 573 | */ | 667 | */ |
| 574 | WARN_ONCE(topology_max_packages() > 2, | 668 | return slit_cluster_distance(from, to); |
| 575 | "sched: Expect only up to 2 packages for GNR or CWF, " | ||
| 576 | "but saw %d packages when building sched domains.", | ||
| 577 | topology_max_packages()); | ||
| 578 | |||
| 579 | d = avg_remote_numa_distance(); | ||
| 580 | } | 669 | } |
| 581 | return d; | 670 | return d; |
| 582 | } | 671 | } |
| @@ -606,7 +695,7 @@ void set_cpu_sibling_map(int cpu) · 23 unmodified lines | |||
| 606 | o = &cpu_data(i); | 695 | o = &cpu_data(i); |
| 607 | 696 | ||
| 608 | if (match_pkg(c, o) && !topology_same_node(c, o)) | 697 | if (match_pkg(c, o) && !topology_same_node(c, o)) |
| 609 | x86_has_numa_in_package = true; | 698 | WARN_ON_ONCE(topology_num_nodes_per_package() == 1); |
| 610 | 699 | ||
| 611 | if ((i == cpu) || (has_smt && match_smt(c, o))) | 700 | if ((i == cpu) || (has_smt && match_smt(c, o))) |
| 612 | link_mask(topology_sibling_cpumask, cpu, i); | 701 | link_mask(topology_sibling_cpumask, cpu, i); |
| @@ -397,7 +397,7 @@ static inline void handle_invalid_op(struct pt_regs *regs) · 396 unmodified lines | |||
| 397 | ILL_ILLOPN, error_get_trap_addr(regs)); | 397 | ILL_ILLOPN, error_get_trap_addr(regs)); |
| 398 | } | 398 | } |
| 399 | 399 | ||
| 400 | static noinstr bool handle_bug(struct pt_regs *regs) | 400 | noinstr bool handle_bug(struct pt_regs *regs) |
| 401 | { | 401 | { |
| 402 | unsigned long addr = regs->ip; | 402 | unsigned long addr = regs->ip; |
| 403 | bool handled = false; | 403 | bool handled = false; |
| @@ -427,6 +427,7 @@ SECTIONS · 426 unmodified lines | |||
| 427 | .llvm_bb_addr_map : { *(.llvm_bb_addr_map) } | 427 | .llvm_bb_addr_map : { *(.llvm_bb_addr_map) } |
| 428 | #endif | 428 | #endif |
| 429 | 429 | ||
| 430 | MODINFO | ||
| 430 | ELF_DETAILS | 431 | ELF_DETAILS |
| 431 | 432 | ||
| 432 | DISCARDS | 433 | DISCARDS |
| @@ -20,7 +20,6 @@ if VIRTUALIZATION · 19 unmodified lines | |||
| 20 | config KVM_X86 | 20 | config KVM_X86 |
| 21 | def_tristate KVM if (KVM_INTEL != n || KVM_AMD != n) | 21 | def_tristate KVM if (KVM_INTEL != n || KVM_AMD != n) |
| 22 | select KVM_COMMON | 22 | select KVM_COMMON |
| 23 | select KVM_GENERIC_MMU_NOTIFIER | ||
| 24 | select KVM_ELIDE_TLB_FLUSH_IF_YOUNG | 23 | select KVM_ELIDE_TLB_FLUSH_IF_YOUNG |
| 25 | select KVM_MMU_LOCKLESS_AGING | 24 | select KVM_MMU_LOCKLESS_AGING |
| 26 | select HAVE_KVM_IRQCHIP | 25 | select HAVE_KVM_IRQCHIP |
| @@ -776,7 +776,10 @@ do { \ · 775 unmodified lines | |||
| 776 | #define SYNTHESIZED_F(name) \ | 776 | #define SYNTHESIZED_F(name) \ |
| 777 | ({ \ | 777 | ({ \ |
| 778 | kvm_cpu_cap_synthesized |= feature_bit(name); \ | 778 | kvm_cpu_cap_synthesized |= feature_bit(name); \ |
| 779 | F(name); \ | 779 | \ |
| 780 | BUILD_BUG_ON(X86_FEATURE_##name >= MAX_CPU_FEATURES); \ | ||
| 781 | if (boot_cpu_has(X86_FEATURE_##name)) \ | ||
| 782 | F(name); \ | ||
| 780 | }) | 783 | }) |
| 781 | 784 | ||
| 782 | /* | 785 | /* |
| @@ -1981,16 +1981,17 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 1980 unmodified lines | |||
| 1981 | if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) | 1981 | if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) |
| 1982 | goto out_flush_all; | 1982 | goto out_flush_all; |
| 1983 | 1983 | ||
| 1984 | if (is_noncanonical_invlpg_address(entries[i], vcpu)) | ||
| 1985 | continue; | ||
| 1986 | |||
| 1987 | /* | 1984 | /* |
| 1988 | * Lower 12 bits of 'address' encode the number of additional | 1985 | * Lower 12 bits of 'address' encode the number of additional |
| 1989 | * pages to flush. | 1986 | * pages to flush. |
| 1990 | */ | 1987 | */ |
| 1991 | gva = entries[i] & PAGE_MASK; | 1988 | gva = entries[i] & PAGE_MASK; |
| 1992 | for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) | 1989 | for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) { |
| 1990 | if (is_noncanonical_invlpg_address(gva + j * PAGE_SIZE, vcpu)) | ||
| 1991 | continue; | ||
| 1992 | |||
| 1993 | kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE); | 1993 | kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE); |
| 1994 | } | ||
| 1994 | 1995 | ||
| 1995 | ++vcpu->stat.tlb_flush; | 1996 | ++vcpu->stat.tlb_flush; |
| 1996 | } | 1997 | } |
| @@ -321,7 +321,8 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin, · 320 unmodified lines | |||
| 321 | idx = srcu_read_lock(&kvm->irq_srcu); | 321 | idx = srcu_read_lock(&kvm->irq_srcu); |
| 322 | gsi = kvm_irq_map_chip_pin(kvm, irqchip, pin); | 322 | gsi = kvm_irq_map_chip_pin(kvm, irqchip, pin); |
| 323 | if (gsi != -1) | 323 | if (gsi != -1) |
| 324 | hlist_for_each_entry_rcu(kimn, &ioapic->mask_notifier_list, link) | 324 | hlist_for_each_entry_srcu(kimn, &ioapic->mask_notifier_list, link, |
| 325 | srcu_read_lock_held(&kvm->irq_srcu)) | ||
| 325 | if (kimn->irq == gsi) | 326 | if (kimn->irq == gsi) |
| 326 | kimn->func(kimn, mask); | 327 | kimn->func(kimn, mask); |
| 327 | srcu_read_unlock(&kvm->irq_srcu, idx); | 328 | srcu_read_unlock(&kvm->irq_srcu, idx); |
| @@ -189,12 +189,12 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 188 unmodified lines | |||
| 189 | struct kvm_vcpu *vcpu = &svm->vcpu; | 189 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 190 | 190 | ||
| 191 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); | 191 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); |
| 192 | |||
| 193 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; | 192 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; |
| 194 | vmcb->control.avic_physical_id |= avic_get_max_physical_id(vcpu); | 193 | vmcb->control.avic_physical_id |= avic_get_max_physical_id(vcpu); |
| 195 | |||
| 196 | vmcb->control.int_ctl |= AVIC_ENABLE_MASK; | 194 | vmcb->control.int_ctl |= AVIC_ENABLE_MASK; |
| 197 | 195 | ||
| 196 | svm_clr_intercept(svm, INTERCEPT_CR8_WRITE); | ||
| 197 | |||
| 198 | /* | 198 | /* |
| 199 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR | 199 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR |
| 200 | * accesses, while interrupt injection to a running vCPU can be | 200 | * accesses, while interrupt injection to a running vCPU can be |
| @@ -226,6 +226,9 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) · 25 unmodified lines | |||
| 226 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); | 226 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); |
| 227 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; | 227 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; |
| 228 | 228 | ||
| 229 | if (!sev_es_guest(svm->vcpu.kvm)) | ||
| 230 | svm_set_intercept(svm, INTERCEPT_CR8_WRITE); | ||
| 231 | |||
| 229 | /* | 232 | /* |
| 230 | * If running nested and the guest uses its own MSR bitmap, there | 233 | * If running nested and the guest uses its own MSR bitmap, there |
| 231 | * is no need to update L0's msr bitmap | 234 | * is no need to update L0's msr bitmap |
| @@ -368,7 +371,7 @@ void avic_init_vmcb(struct vcpu_svm *svm, struct vmcb *vmcb) · 136 unmodified lines | |||
| 368 | vmcb->control.avic_physical_id = __sme_set(__pa(kvm_svm->avic_physical_id_table)); | 371 | vmcb->control.avic_physical_id = __sme_set(__pa(kvm_svm->avic_physical_id_table)); |
| 369 | vmcb->control.avic_vapic_bar = APIC_DEFAULT_PHYS_BASE; | 372 | vmcb->control.avic_vapic_bar = APIC_DEFAULT_PHYS_BASE; |
| 370 | 373 | ||
| 371 | if (kvm_apicv_activated(svm->vcpu.kvm)) | 374 | if (kvm_vcpu_apicv_active(&svm->vcpu)) |
| 372 | avic_activate_vmcb(svm); | 375 | avic_activate_vmcb(svm); |
| 373 | else | 376 | else |
| 374 | avic_deactivate_vmcb(svm); | 377 | avic_deactivate_vmcb(svm); |
| @@ -418,6 +418,15 @@ static bool nested_vmcb_check_controls(struct kvm_vcpu *vcpu) · 417 unmodified lines | |||
| 418 | return __nested_vmcb_check_controls(vcpu, ctl); | 418 | return __nested_vmcb_check_controls(vcpu, ctl); |
| 419 | } | 419 | } |
| 420 | 420 | ||
| 421 | int nested_svm_check_cached_vmcb12(struct kvm_vcpu *vcpu) | ||
| 422 | { | ||
| 423 | if (!nested_vmcb_check_save(vcpu) || | ||
| 424 | !nested_vmcb_check_controls(vcpu)) | ||
| 425 | return -EINVAL; | ||
| 426 | |||
| 427 | return 0; | ||
| 428 | } | ||
| 429 | |||
| 421 | /* | 430 | /* |
| 422 | * If a feature is not advertised to L1, clear the corresponding vmcb12 | 431 | * If a feature is not advertised to L1, clear the corresponding vmcb12 |
| 423 | * intercept. | 432 | * intercept. |
| @@ -1028,8 +1037,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) · 604 unmodified lines | |||
| 1028 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); | 1037 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 1029 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); | 1038 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 1030 | 1039 | ||
| 1031 | if (!nested_vmcb_check_save(vcpu) || | 1040 | if (nested_svm_check_cached_vmcb12(vcpu) < 0) { |
| 1032 | !nested_vmcb_check_controls(vcpu)) { | ||
| 1033 | vmcb12->control.exit_code = SVM_EXIT_ERR; | 1041 | vmcb12->control.exit_code = SVM_EXIT_ERR; |
| 1034 | vmcb12->control.exit_info_1 = 0; | 1042 | vmcb12->control.exit_info_1 = 0; |
| 1035 | vmcb12->control.exit_info_2 = 0; | 1043 | vmcb12->control.exit_info_2 = 0; |
| @@ -1077,8 +1077,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · 1076 unmodified lines | |||
| 1077 | svm_set_intercept(svm, INTERCEPT_CR0_WRITE); | 1077 | svm_set_intercept(svm, INTERCEPT_CR0_WRITE); |
| 1078 | svm_set_intercept(svm, INTERCEPT_CR3_WRITE); | 1078 | svm_set_intercept(svm, INTERCEPT_CR3_WRITE); |
| 1079 | svm_set_intercept(svm, INTERCEPT_CR4_WRITE); | 1079 | svm_set_intercept(svm, INTERCEPT_CR4_WRITE); |
| 1080 | if (!kvm_vcpu_apicv_active(vcpu)) | 1080 | svm_set_intercept(svm, INTERCEPT_CR8_WRITE); |
| 1081 | svm_set_intercept(svm, INTERCEPT_CR8_WRITE); | ||
| 1082 | 1081 | ||
| 1083 | set_dr_intercepts(svm); | 1082 | set_dr_intercepts(svm); |
| 1084 | 1083 | ||
| @@ -1189,7 +1188,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · 104 unmodified lines | |||
| 1189 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) | 1188 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) |
| 1190 | svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP; | 1189 | svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP; |
| 1191 | 1190 | ||
| 1192 | if (kvm_vcpu_apicv_active(vcpu)) | 1191 | if (enable_apicv && irqchip_in_kernel(vcpu->kvm)) |
| 1193 | avic_init_vmcb(svm, vmcb); | 1192 | avic_init_vmcb(svm, vmcb); |
| 1194 | 1193 | ||
| 1195 | if (vnmi) | 1194 | if (vnmi) |
| @@ -2674,9 +2673,11 @@ static int dr_interception(struct kvm_vcpu *vcpu) · 1478 unmodified lines | |||
| 2674 | 2673 | ||
| 2675 | static int cr8_write_interception(struct kvm_vcpu *vcpu) | 2674 | static int cr8_write_interception(struct kvm_vcpu *vcpu) |
| 2676 | { | 2675 | { |
| 2676 | u8 cr8_prev = kvm_get_cr8(vcpu); | ||
| 2677 | int r; | 2677 | int r; |
| 2678 | 2678 | ||
| 2679 | u8 cr8_prev = kvm_get_cr8(vcpu); | 2679 | WARN_ON_ONCE(kvm_vcpu_apicv_active(vcpu)); |
| 2680 | |||
| 2680 | /* instruction emulation calls kvm_set_cr8() */ | 2681 | /* instruction emulation calls kvm_set_cr8() */ |
| 2681 | r = cr_interception(vcpu); | 2682 | r = cr_interception(vcpu); |
| 2682 | if (lapic_in_kernel(vcpu)) | 2683 | if (lapic_in_kernel(vcpu)) |
| @@ -4879,11 +4880,15 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) · 2196 unmodified lines | |||
| 4879 | vmcb12 = map.hva; | 4880 | vmcb12 = map.hva; |
| 4880 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); | 4881 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 4881 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); | 4882 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 4882 | ret = enter_svm_guest_mode(vcpu, smram64->svm_guest_vmcb_gpa, vmcb12, false); | ||
| 4883 | 4883 | ||
| 4884 | if (ret) | 4884 | if (nested_svm_check_cached_vmcb12(vcpu) < 0) |
| 4885 | goto unmap_save; | ||
| 4886 | |||
| 4887 | if (enter_svm_guest_mode(vcpu, smram64->svm_guest_vmcb_gpa, | ||
| 4888 | vmcb12, false) != 0) | ||
| 4885 | goto unmap_save; | 4889 | goto unmap_save; |
| 4886 | 4890 | ||
| 4891 | ret = 0; | ||
| 4887 | svm->nested.nested_run_pending = 1; | 4892 | svm->nested.nested_run_pending = 1; |
| 4888 | 4893 | ||
| 4889 | unmap_save: | 4894 | unmap_save: |
| @@ -797,6 +797,7 @@ static inline int nested_svm_simple_vmexit(struct vcpu_svm *svm, u32 exit_code) · 796 unmodified lines | |||
| 797 | 797 | ||
| 798 | int nested_svm_exit_handled(struct vcpu_svm *svm); | 798 | int nested_svm_exit_handled(struct vcpu_svm *svm); |
| 799 | int nested_svm_check_permissions(struct kvm_vcpu *vcpu); | 799 | int nested_svm_check_permissions(struct kvm_vcpu *vcpu); |
| 800 | int nested_svm_check_cached_vmcb12(struct kvm_vcpu *vcpu); | ||
| 800 | int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr, | 801 | int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr, |
| 801 | bool has_error_code, u32 error_code); | 802 | bool has_error_code, u32 error_code); |
| 802 | int nested_svm_exit_special(struct vcpu_svm *svm); | 803 | int nested_svm_exit_special(struct vcpu_svm *svm); |
| @@ -3300,10 +3300,24 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu, · 3299 unmodified lines | |||
| 3300 | if (CC(vmcs12->guest_cr4 & X86_CR4_CET && !(vmcs12->guest_cr0 & X86_CR0_WP))) | 3300 | if (CC(vmcs12->guest_cr4 & X86_CR4_CET && !(vmcs12->guest_cr0 & X86_CR0_WP))) |
| 3301 | return -EINVAL; | 3301 | return -EINVAL; |
| 3302 | 3302 | ||
| 3303 | if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) && | 3303 | if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) { |
| 3304 | (CC(!kvm_dr7_valid(vmcs12->guest_dr7)) || | 3304 | u64 debugctl = vmcs12->guest_ia32_debugctl; |
| 3305 | CC(!vmx_is_valid_debugctl(vcpu, vmcs12->guest_ia32_debugctl, false)))) | 3305 | |
| 3306 | return -EINVAL; | 3306 | /* |
| 3307 | * FREEZE_IN_SMM is not virtualized, but allow L1 to set it in | ||
| 3308 | * vmcs12's DEBUGCTL under a quirk for backwards compatibility. | ||
| 3309 | * Note that the quirk only relaxes the consistency check. The | ||
| 3310 | * vmcc02 bit is still under the control of the host. In | ||
| 3311 | * particular, if a host administrator decides to clear the bit, | ||
| 3312 | * then L1 has no say in the matter. | ||
| 3313 | */ | ||
| 3314 | if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM)) | ||
| 3315 | debugctl &= ~DEBUGCTLMSR_FREEZE_IN_SMM; | ||
| 3316 | |||
| 3317 | if (CC(!kvm_dr7_valid(vmcs12->guest_dr7)) || | ||
| 3318 | CC(!vmx_is_valid_debugctl(vcpu, debugctl, false))) | ||
| 3319 | return -EINVAL; | ||
| 3320 | } | ||
| 3307 | 3321 | ||
| 3308 | if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PAT) && | 3322 | if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PAT) && |
| 3309 | CC(!kvm_pat_valid(vmcs12->guest_ia32_pat))) | 3323 | CC(!kvm_pat_valid(vmcs12->guest_ia32_pat))) |
| @@ -6842,13 +6856,34 @@ void vmx_leave_nested(struct kvm_vcpu *vcpu) · 3532 unmodified lines | |||
| 6842 | free_nested(vcpu); | 6856 | free_nested(vcpu); |
| 6843 | } | 6857 | } |
| 6844 | 6858 | ||
| 6859 | int nested_vmx_check_restored_vmcs12(struct kvm_vcpu *vcpu) | ||
| 6860 | { | ||
| 6861 | enum vm_entry_failure_code ignored; | ||
| 6862 | struct vmcs12 *vmcs12 = get_vmcs12(vcpu); | ||
| 6863 | |||
| 6864 | if (nested_cpu_has_shadow_vmcs(vmcs12) && | ||
| 6865 | vmcs12->vmcs_link_pointer != INVALID_GPA) { | ||
| 6866 | struct vmcs12 *shadow_vmcs12 = get_shadow_vmcs12(vcpu); | ||
| 6867 | |||
| 6868 | if (shadow_vmcs12->hdr.revision_id != VMCS12_REVISION || | ||
| 6869 | !shadow_vmcs12->hdr.shadow_vmcs) | ||
| 6870 | return -EINVAL; | ||
| 6871 | } | ||
| 6872 | |||
| 6873 | if (nested_vmx_check_controls(vcpu, vmcs12) || | ||
| 6874 | nested_vmx_check_host_state(vcpu, vmcs12) || | ||
| 6875 | nested_vmx_check_guest_state(vcpu, vmcs12, &ignored)) | ||
| 6876 | return -EINVAL; | ||
| 6877 | |||
| 6878 | return 0; | ||
| 6879 | } | ||
| 6880 | |||
| 6845 | static int vmx_set_nested_state(struct kvm_vcpu *vcpu, | 6881 | static int vmx_set_nested_state(struct kvm_vcpu *vcpu, |
| 6846 | struct kvm_nested_state __user *user_kvm_nested_state, | 6882 | struct kvm_nested_state __user *user_kvm_nested_state, |
| 6847 | struct kvm_nested_state *kvm_state) | 6883 | struct kvm_nested_state *kvm_state) |
| 6848 | { | 6884 | { |
| 6849 | struct vcpu_vmx *vmx = to_vmx(vcpu); | 6885 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| 6850 | struct vmcs12 *vmcs12; | 6886 | struct vmcs12 *vmcs12; |
| 6851 | enum vm_entry_failure_code ignored; | ||
| 6852 | struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = | 6887 | struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = |
| 6853 | &user_kvm_nested_state->data.vmx[0]; | 6888 | &user_kvm_nested_state->data.vmx[0]; |
| 6854 | int ret; | 6889 | int ret; |
| @@ -6979,25 +7014,20 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · 124 unmodified lines | |||
| 6979 | vmx->nested.mtf_pending = | 7014 | vmx->nested.mtf_pending = |
| 6980 | !!(kvm_state->flags & KVM_STATE_NESTED_MTF_PENDING); | 7015 | !!(kvm_state->flags & KVM_STATE_NESTED_MTF_PENDING); |
| 6981 | 7016 | ||
| 6982 | ret = -EINVAL; | ||
| 6983 | if (nested_cpu_has_shadow_vmcs(vmcs12) && | 7017 | if (nested_cpu_has_shadow_vmcs(vmcs12) && |
| 6984 | vmcs12->vmcs_link_pointer != INVALID_GPA) { | 7018 | vmcs12->vmcs_link_pointer != INVALID_GPA) { |
| 6985 | struct vmcs12 *shadow_vmcs12 = get_shadow_vmcs12(vcpu); | 7019 | struct vmcs12 *shadow_vmcs12 = get_shadow_vmcs12(vcpu); |
| 6986 | 7020 | ||
| 7021 | ret = -EINVAL; | ||
| 6987 | if (kvm_state->size < | 7022 | if (kvm_state->size < |
| 6988 | sizeof(*kvm_state) + | 7023 | sizeof(*kvm_state) + |
| 6989 | sizeof(user_vmx_nested_state->vmcs12) + sizeof(*shadow_vmcs12)) | 7024 | sizeof(user_vmx_nested_state->vmcs12) + sizeof(*shadow_vmcs12)) |
| 6990 | goto error_guest_mode; | 7025 | goto error_guest_mode; |
| 6991 | 7026 | ||
| 7027 | ret = -EFAULT; | ||
| 6992 | if (copy_from_user(shadow_vmcs12, | 7028 | if (copy_from_user(shadow_vmcs12, |
| 6993 | user_vmx_nested_state->shadow_vmcs12, | 7029 | user_vmx_nested_state->shadow_vmcs12, |
| 6994 | sizeof(*shadow_vmcs12))) { | 7030 | sizeof(*shadow_vmcs12))) |
| 6995 | ret = -EFAULT; | ||
| 6996 | goto error_guest_mode; | ||
| 6997 | } | ||
| 6998 | |||
| 6999 | if (shadow_vmcs12->hdr.revision_id != VMCS12_REVISION || | ||
| 7000 | !shadow_vmcs12->hdr.shadow_vmcs) | ||
| 7001 | goto error_guest_mode; | 7031 | goto error_guest_mode; |
| 7002 | } | 7032 | } |
| 7003 | 7033 | ||
| @@ -7008,9 +7038,8 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · 4 unmodified lines | |||
| 7008 | kvm_state->hdr.vmx.preemption_timer_deadline; | 7038 | kvm_state->hdr.vmx.preemption_timer_deadline; |
| 7009 | } | 7039 | } |
| 7010 | 7040 | ||
| 7011 | if (nested_vmx_check_controls(vcpu, vmcs12) || | 7041 | ret = nested_vmx_check_restored_vmcs12(vcpu); |
| 7012 | nested_vmx_check_host_state(vcpu, vmcs12) || | 7042 | if (ret < 0) |
| 7013 | nested_vmx_check_guest_state(vcpu, vmcs12, &ignored)) | ||
| 7014 | goto error_guest_mode; | 7043 | goto error_guest_mode; |
| 7015 | 7044 | ||
| 7016 | vmx->nested.dirty_vmcs12 = true; | 7045 | vmx->nested.dirty_vmcs12 = true; |
| @@ -22,6 +22,7 @@ void nested_vmx_setup_ctls_msrs(struct vmcs_config *vmcs_conf, u32 ept_caps); · 21 unmodified lines | |||
| 22 | void nested_vmx_hardware_unsetup(void); | 22 | void nested_vmx_hardware_unsetup(void); |
| 23 | __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *)); | 23 | __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *)); |
| 24 | void nested_vmx_set_vmcs_shadowing_bitmap(void); | 24 | void nested_vmx_set_vmcs_shadowing_bitmap(void); |
| 25 | int nested_vmx_check_restored_vmcs12(struct kvm_vcpu *vcpu); | ||
| 25 | void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu); | 26 | void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu); |
| 26 | enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, | 27 | enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, |
| 27 | bool from_vmentry); | 28 | bool from_vmentry); |
| @@ -1149,7 +1149,7 @@ static void add_atomic_switch_msr(struct vcpu_vmx *vmx, unsigned msr, · 1148 unmodified lines | |||
| 1149 | } | 1149 | } |
| 1150 | 1150 | ||
| 1151 | vmx_add_auto_msr(&m->guest, msr, guest_val, VM_ENTRY_MSR_LOAD_COUNT, kvm); | 1151 | vmx_add_auto_msr(&m->guest, msr, guest_val, VM_ENTRY_MSR_LOAD_COUNT, kvm); |
| 1152 | vmx_add_auto_msr(&m->guest, msr, host_val, VM_EXIT_MSR_LOAD_COUNT, kvm); | 1152 | vmx_add_auto_msr(&m->host, msr, host_val, VM_EXIT_MSR_LOAD_COUNT, kvm); |
| 1153 | } | 1153 | } |
| 1154 | 1154 | ||
| 1155 | static bool update_transition_efer(struct vcpu_vmx *vmx) | 1155 | static bool update_transition_efer(struct vcpu_vmx *vmx) |
| @@ -8528,9 +8528,13 @@ int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) · 7372 unmodified lines | |||
| 8528 | } | 8528 | } |
| 8529 | 8529 | ||
| 8530 | if (vmx->nested.smm.guest_mode) { | 8530 | if (vmx->nested.smm.guest_mode) { |
| 8531 | /* Triple fault if the state is invalid. */ | ||
| 8532 | if (nested_vmx_check_restored_vmcs12(vcpu) < 0) | ||
| 8533 | return 1; | ||
| 8534 | |||
| 8531 | ret = nested_vmx_enter_non_root_mode(vcpu, false); | 8535 | ret = nested_vmx_enter_non_root_mode(vcpu, false); |
| 8532 | if (ret) | 8536 | if (ret != NVMX_VMENTRY_SUCCESS) |
| 8533 | return ret; | 8537 | return 1; |
| 8534 | 8538 | ||
| 8535 | vmx->nested.nested_run_pending = 1; | 8539 | vmx->nested.nested_run_pending = 1; |
| 8536 | vmx->nested.smm.guest_mode = false; | 8540 | vmx->nested.smm.guest_mode = false; |
| @@ -243,7 +243,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_ipiv); · 242 unmodified lines | |||
| 243 | bool __read_mostly enable_device_posted_irqs = true; | 243 | bool __read_mostly enable_device_posted_irqs = true; |
| 244 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_device_posted_irqs); | 244 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_device_posted_irqs); |
| 245 | 245 | ||
| 246 | const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | 246 | const struct kvm_stats_desc kvm_vm_stats_desc[] = { |
| 247 | KVM_GENERIC_VM_STATS(), | 247 | KVM_GENERIC_VM_STATS(), |
| 248 | STATS_DESC_COUNTER(VM, mmu_shadow_zapped), | 248 | STATS_DESC_COUNTER(VM, mmu_shadow_zapped), |
| 249 | STATS_DESC_COUNTER(VM, mmu_pte_write), | 249 | STATS_DESC_COUNTER(VM, mmu_pte_write), |
| @@ -269,7 +269,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 19 unmodified lines | |||
| 269 | sizeof(kvm_vm_stats_desc), | 269 | sizeof(kvm_vm_stats_desc), |
| 270 | }; | 270 | }; |
| 271 | 271 | ||
| 272 | const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | 272 | const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { |
| 273 | KVM_GENERIC_VCPU_STATS(), | 273 | KVM_GENERIC_VCPU_STATS(), |
| 274 | STATS_DESC_COUNTER(VCPU, pf_taken), | 274 | STATS_DESC_COUNTER(VCPU, pf_taken), |
| 275 | STATS_DESC_COUNTER(VCPU, pf_fixed), | 275 | STATS_DESC_COUNTER(VCPU, pf_fixed), |
| @@ -4805,7 +4805,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 4529 unmodified lines | |||
| 4805 | #endif | 4805 | #endif |
| 4806 | case KVM_CAP_NOP_IO_DELAY: | 4806 | case KVM_CAP_NOP_IO_DELAY: |
| 4807 | case KVM_CAP_MP_STATE: | 4807 | case KVM_CAP_MP_STATE: |
| 4808 | case KVM_CAP_SYNC_MMU: | ||
| 4809 | case KVM_CAP_USER_NMI: | 4808 | case KVM_CAP_USER_NMI: |
| 4810 | case KVM_CAP_IRQ_INJECT_STATUS: | 4809 | case KVM_CAP_IRQ_INJECT_STATUS: |
| 4811 | case KVM_CAP_IOEVENTFD: | 4810 | case KVM_CAP_IOEVENTFD: |
| @@ -411,14 +411,11 @@ void __init early_fixup_exception(struct pt_regs *regs, int trapnr) · 410 unmodified lines | |||
| 411 | return; | 411 | return; |
| 412 | 412 | ||
| 413 | if (trapnr == X86_TRAP_UD) { | 413 | if (trapnr == X86_TRAP_UD) { |
| 414 | if (report_bug(regs->ip, regs) == BUG_TRAP_TYPE_WARN) { | 414 | if (handle_bug(regs)) |
| 415 | /* Skip the ud2. */ | ||
| 416 | regs->ip += LEN_UD2; | ||
| 417 | return; | 415 | return; |
| 418 | } | ||
| 419 | 416 | ||
| 420 | /* | 417 | /* |
| 421 | * If this was a BUG and report_bug returns or if this | 418 | * If this was a BUG and handle_bug returns or if this |
| 422 | * was just a normal #UD, we want to continue onward and | 419 | * was just a normal #UD, we want to continue onward and |
| 423 | * crash. | 420 | * crash. |
| 424 | */ | 421 | */ |
| @@ -48,6 +48,8 @@ s16 __apicid_to_node[MAX_LOCAL_APIC] = { · 47 unmodified lines | |||
| 48 | [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE | 48 | [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE |
| 49 | }; | 49 | }; |
| 50 | 50 | ||
| 51 | nodemask_t numa_phys_nodes_parsed __initdata; | ||
| 52 | |||
| 51 | int numa_cpu_node(int cpu) | 53 | int numa_cpu_node(int cpu) |
| 52 | { | 54 | { |
| 53 | u32 apicid = early_per_cpu(x86_cpu_to_apicid, cpu); | 55 | u32 apicid = early_per_cpu(x86_cpu_to_apicid, cpu); |
| @@ -57,6 +59,11 @@ int numa_cpu_node(int cpu) · 3 unmodified lines | |||
| 57 | return NUMA_NO_NODE; | 59 | return NUMA_NO_NODE; |
| 58 | } | 60 | } |
| 59 | 61 | ||
| 62 | int __init num_phys_nodes(void) | ||
| 63 | { | ||
| 64 | return bitmap_weight(numa_phys_nodes_parsed.bits, MAX_NUMNODES); | ||
| 65 | } | ||
| 66 | |||
| 60 | cpumask_var_t node_to_cpumask_map[MAX_NUMNODES]; | 67 | cpumask_var_t node_to_cpumask_map[MAX_NUMNODES]; |
| 61 | EXPORT_SYMBOL(node_to_cpumask_map); | 68 | EXPORT_SYMBOL(node_to_cpumask_map); |
| 62 | 69 | ||
| @@ -210,6 +217,7 @@ static int __init dummy_numa_init(void) · 147 unmodified lines | |||
| 210 | 0LLU, PFN_PHYS(max_pfn) - 1); | 217 | 0LLU, PFN_PHYS(max_pfn) - 1); |
| 211 | 218 | ||
| 212 | node_set(0, numa_nodes_parsed); | 219 | node_set(0, numa_nodes_parsed); |
| 220 | node_set(0, numa_phys_nodes_parsed); | ||
| 213 | numa_add_memblk(0, 0, PFN_PHYS(max_pfn)); | 221 | numa_add_memblk(0, 0, PFN_PHYS(max_pfn)); |
| 214 | 222 | ||
| 215 | return 0; | 223 | return 0; |
| @@ -57,6 +57,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa) · 56 unmodified lines | |||
| 57 | } | 57 | } |
| 58 | set_apicid_to_node(apic_id, node); | 58 | set_apicid_to_node(apic_id, node); |
| 59 | node_set(node, numa_nodes_parsed); | 59 | node_set(node, numa_nodes_parsed); |
| 60 | node_set(node, numa_phys_nodes_parsed); | ||
| 60 | pr_debug("SRAT: PXM %u -> APIC 0x%04x -> Node %u\n", pxm, apic_id, node); | 61 | pr_debug("SRAT: PXM %u -> APIC 0x%04x -> Node %u\n", pxm, apic_id, node); |
| 61 | } | 62 | } |
| 62 | 63 | ||
| @@ -97,6 +98,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) · 34 unmodified lines | |||
| 97 | 98 | ||
| 98 | set_apicid_to_node(apic_id, node); | 99 | set_apicid_to_node(apic_id, node); |
| 99 | node_set(node, numa_nodes_parsed); | 100 | node_set(node, numa_nodes_parsed); |
| 101 | node_set(node, numa_phys_nodes_parsed); | ||
| 100 | pr_debug("SRAT: PXM %u -> APIC 0x%02x -> Node %u\n", pxm, apic_id, node); | 102 | pr_debug("SRAT: PXM %u -> APIC 0x%02x -> Node %u\n", pxm, apic_id, node); |
| 101 | } | 103 | } |
| 102 | 104 | ||
| @@ -438,17 +438,8 @@ static void emit_kcfi(u8 **pprog, u32 hash) · 437 unmodified lines | |||
| 438 | 438 | ||
| 439 | EMIT1_off32(0xb8, hash); /* movl $hash, %eax */ | 439 | EMIT1_off32(0xb8, hash); /* movl $hash, %eax */ |
| 440 | #ifdef CONFIG_CALL_PADDING | 440 | #ifdef CONFIG_CALL_PADDING |
| 441 | EMIT1(0x90); | 441 | for (int i = 0; i < CONFIG_FUNCTION_PADDING_CFI; i++) |
| 442 | EMIT1(0x90); | 442 | EMIT1(0x90); |
| 443 | EMIT1(0x90); | ||
| 444 | EMIT1(0x90); | ||
| 445 | EMIT1(0x90); | ||
| 446 | EMIT1(0x90); | ||
| 447 | EMIT1(0x90); | ||
| 448 | EMIT1(0x90); | ||
| 449 | EMIT1(0x90); | ||
| 450 | EMIT1(0x90); | ||
| 451 | EMIT1(0x90); | ||
| 452 | #endif | 443 | #endif |
| 453 | EMIT_ENDBR(); | 444 | EMIT_ENDBR(); |
| 454 | 445 | ||
| @@ -836,7 +836,7 @@ static void __init __efi_enter_virtual_mode(void) · 835 unmodified lines | |||
| 836 | } | 836 | } |
| 837 | 837 | ||
| 838 | efi_check_for_embedded_firmwares(); | 838 | efi_check_for_embedded_firmwares(); |
| 839 | efi_free_boot_services(); | 839 | efi_unmap_boot_services(); |
| 840 | 840 | ||
| 841 | if (!efi_is_mixed()) | 841 | if (!efi_is_mixed()) |
| 842 | efi_native_runtime_setup(); | 842 | efi_native_runtime_setup(); |
| @@ -341,7 +341,7 @@ void __init efi_reserve_boot_services(void) · 340 unmodified lines | |||
| 341 | 341 | ||
| 342 | /* | 342 | /* |
| 343 | * Because the following memblock_reserve() is paired | 343 | * Because the following memblock_reserve() is paired |
| 344 | * with memblock_free_late() for this region in | 344 | * with free_reserved_area() for this region in |
| 345 | * efi_free_boot_services(), we must be extremely | 345 | * efi_free_boot_services(), we must be extremely |
| 346 | * careful not to reserve, and subsequently free, | 346 | * careful not to reserve, and subsequently free, |
| 347 | * critical regions of memory (like the kernel image) or | 347 | * critical regions of memory (like the kernel image) or |
| @@ -404,17 +404,33 @@ static void __init efi_unmap_pages(efi_memory_desc_t *md) · 56 unmodified lines | |||
| 404 | pr_err("Failed to unmap VA mapping for 0x%llx\n", va); | 404 | pr_err("Failed to unmap VA mapping for 0x%llx\n", va); |
| 405 | } | 405 | } |
| 406 | 406 | ||
| 407 | void __init efi_free_boot_services(void) | 407 | struct efi_freeable_range { |
| 408 | u64 start; | ||
| 409 | u64 end; | ||
| 410 | }; | ||
| 411 | |||
| 412 | static struct efi_freeable_range *ranges_to_free; | ||
| 413 | |||
| 414 | void __init efi_unmap_boot_services(void) | ||
| 408 | { | 415 | { |
| 409 | struct efi_memory_map_data data = { 0 }; | 416 | struct efi_memory_map_data data = { 0 }; |
| 410 | efi_memory_desc_t *md; | 417 | efi_memory_desc_t *md; |
| 411 | int num_entries = 0; | 418 | int num_entries = 0; |
| 419 | int idx = 0; | ||
| 420 | size_t sz; | ||
| 412 | void *new, *new_md; | 421 | void *new, *new_md; |
| 413 | 422 | ||
| 414 | /* Keep all regions for /sys/kernel/debug/efi */ | 423 | /* Keep all regions for /sys/kernel/debug/efi */ |
| 415 | if (efi_enabled(EFI_DBG)) | 424 | if (efi_enabled(EFI_DBG)) |
| 416 | return; | 425 | return; |
| 417 | 426 | ||
| 427 | sz = sizeof(*ranges_to_free) * efi.memmap.nr_map + 1; | ||
| 428 | ranges_to_free = kzalloc(sz, GFP_KERNEL); | ||
| 429 | if (!ranges_to_free) { | ||
| 430 | pr_err("Failed to allocate storage for freeable EFI regions\n"); | ||
| 431 | return; | ||
| 432 | } | ||
| 433 | |||
| 418 | for_each_efi_memory_desc(md) { | 434 | for_each_efi_memory_desc(md) { |
| 419 | unsigned long long start = md->phys_addr; | 435 | unsigned long long start = md->phys_addr; |
| 420 | unsigned long long size = md->num_pages << EFI_PAGE_SHIFT; | 436 | unsigned long long size = md->num_pages << EFI_PAGE_SHIFT; |
| @@ -471,7 +487,15 @@ void __init efi_free_boot_services(void) · 50 unmodified lines | |||
| 471 | start = SZ_1M; | 487 | start = SZ_1M; |
| 472 | } | 488 | } |
| 473 | 489 | ||
| 474 | memblock_free_late(start, size); | 490 | /* |
| 491 | * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory | ||
| 492 | * map are still not initialized and we can't reliably free | ||
| 493 | * memory here. | ||
| 494 | * Queue the ranges to free at a later point. | ||
| 495 | */ | ||
| 496 | ranges_to_free[idx].start = start; | ||
| 497 | ranges_to_free[idx].end = start + size; | ||
| 498 | idx++; | ||
| 475 | } | 499 | } |
| 476 | 500 | ||
| 477 | if (!num_entries) | 501 | if (!num_entries) |
| @@ -512,6 +536,31 @@ void __init efi_free_boot_services(void) · 34 unmodified lines | |||
| 512 | } | 536 | } |
| 513 | } | 537 | } |
| 514 | 538 | ||
| 539 | static int __init efi_free_boot_services(void) | ||
| 540 | { | ||
| 541 | struct efi_freeable_range *range = ranges_to_free; | ||
| 542 | unsigned long freed = 0; | ||
| 543 | |||
| 544 | if (!ranges_to_free) | ||
| 545 | return 0; | ||
| 546 | |||
| 547 | while (range->start) { | ||
| 548 | void *start = phys_to_virt(range->start); | ||
| 549 | void *end = phys_to_virt(range->end); | ||
| 550 | |||
| 551 | free_reserved_area(start, end, -1, NULL); | ||
| 552 | freed += (end - start); | ||
| 553 | range++; | ||
| 554 | } | ||
| 555 | kfree(ranges_to_free); | ||
| 556 | |||
| 557 | if (freed) | ||
| 558 | pr_info("Freeing EFI boot services memory: %ldK\n", freed / SZ_1K); | ||
| 559 | |||
| 560 | return 0; | ||
| 561 | } | ||
| 562 | arch_initcall(efi_free_boot_services); | ||
| 563 | |||
| 515 | /* | 564 | /* |
| 516 | * A number of config table entries get remapped to virtual addresses | 565 | * A number of config table entries get remapped to virtual addresses |
| 517 | * after entering EFI virtual mode. However, the kexec kernel requires | 566 | * after entering EFI virtual mode. However, the kexec kernel requires |
| @@ -25,11 +25,6 @@ struct hvm_start_info __initdata pvh_start_info; · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | const unsigned int __initconst pvh_start_info_sz = sizeof(pvh_start_info); | 26 | const unsigned int __initconst pvh_start_info_sz = sizeof(pvh_start_info); |
| 27 | 27 | ||
| 28 | static u64 __init pvh_get_root_pointer(void) | ||
| 29 | { | ||
| 30 | return pvh_start_info.rsdp_paddr; | ||
| 31 | } | ||
| 32 | |||
| 33 | /* | 28 | /* |
| 34 | * Xen guests are able to obtain the memory map from the hypervisor via the | 29 | * Xen guests are able to obtain the memory map from the hypervisor via the |
| 35 | * HYPERVISOR_memory_op hypercall. | 30 | * HYPERVISOR_memory_op hypercall. |
| @@ -95,7 +90,7 @@ static void __init init_pvh_bootparams(bool xen_guest) · 59 unmodified lines | |||
| 95 | pvh_bootparams.hdr.version = (2 << 8) | 12; | 90 | pvh_bootparams.hdr.version = (2 << 8) | 12; |
| 96 | pvh_bootparams.hdr.type_of_loader = ((xen_guest ? 0x9 : 0xb) << 4) | 0; | 91 | pvh_bootparams.hdr.type_of_loader = ((xen_guest ? 0x9 : 0xb) << 4) | 0; |
| 97 | 92 | ||
| 98 | x86_init.acpi.get_root_pointer = pvh_get_root_pointer; | 93 | pvh_bootparams.acpi_rsdp_addr = pvh_start_info.rsdp_paddr; |
| 99 | } | 94 | } |
| 100 | 95 | ||
| 101 | /* | 96 | /* |
| @@ -392,7 +392,7 @@ static void __init xen_init_capabilities(void) · 391 unmodified lines | |||
| 392 | 392 | ||
| 393 | /* | 393 | /* |
| 394 | * Xen PV would need some work to support PCID: CR3 handling as well | 394 | * Xen PV would need some work to support PCID: CR3 handling as well |
| 395 | * as xen_flush_tlb_others() would need updating. | 395 | * as xen_flush_tlb_multi() would need updating. |
| 396 | */ | 396 | */ |
| 397 | setup_clear_cpu_cap(X86_FEATURE_PCID); | 397 | setup_clear_cpu_cap(X86_FEATURE_PCID); |
| 398 | 398 | ||
| @@ -105,6 +105,9 @@ pte_t xen_make_pte_init(pteval_t pte); · 104 unmodified lines | |||
| 105 | static pud_t level3_user_vsyscall[PTRS_PER_PUD] __page_aligned_bss; | 105 | static pud_t level3_user_vsyscall[PTRS_PER_PUD] __page_aligned_bss; |
| 106 | #endif | 106 | #endif |
| 107 | 107 | ||
| 108 | static pud_t level3_ident_pgt[PTRS_PER_PUD] __page_aligned_bss; | ||
| 109 | static pmd_t level2_ident_pgt[PTRS_PER_PMD] __page_aligned_bss; | ||
| 110 | |||
| 108 | /* | 111 | /* |
| 109 | * Protects atomic reservation decrease/increase against concurrent increases. | 112 | * Protects atomic reservation decrease/increase against concurrent increases. |
| 110 | * Also protects non-atomic updates of current_pages and balloon lists. | 113 | * Also protects non-atomic updates of current_pages and balloon lists. |
| @@ -1777,6 +1780,12 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) · 1666 unmodified lines | |||
| 1777 | /* Zap identity mapping */ | 1780 | /* Zap identity mapping */ |
| 1778 | init_top_pgt[0] = __pgd(0); | 1781 | init_top_pgt[0] = __pgd(0); |
| 1779 | 1782 | ||
| 1783 | init_top_pgt[pgd_index(__PAGE_OFFSET_BASE_L4)].pgd = | ||
| 1784 | __pa_symbol(level3_ident_pgt) + _KERNPG_TABLE_NOENC; | ||
| 1785 | init_top_pgt[pgd_index(__START_KERNEL_map)].pgd = | ||
| 1786 | __pa_symbol(level3_kernel_pgt) + _PAGE_TABLE_NOENC; | ||
| 1787 | level3_ident_pgt[0].pud = __pa_symbol(level2_ident_pgt) + _KERNPG_TABLE_NOENC; | ||
| 1788 | |||
| 1780 | /* Pre-constructed entries are in pfn, so convert to mfn */ | 1789 | /* Pre-constructed entries are in pfn, so convert to mfn */ |
| 1781 | /* L4[273] -> level3_ident_pgt */ | 1790 | /* L4[273] -> level3_ident_pgt */ |
| 1782 | /* L4[511] -> level3_kernel_pgt */ | 1791 | /* L4[511] -> level3_kernel_pgt */ |
| @@ -398,8 +398,7 @@ static struct bio *bio_copy_kern(struct request *rq, void *data, unsigned int le · 397 unmodified lines | |||
| 398 | if (op_is_write(op)) | 398 | if (op_is_write(op)) |
| 399 | memcpy(page_address(page), p, bytes); | 399 | memcpy(page_address(page), p, bytes); |
| 400 | 400 | ||
| 401 | if (bio_add_page(bio, page, bytes, 0) < bytes) | 401 | __bio_add_page(bio, page, bytes, 0); |
| 402 | break; | ||
| 403 | 402 | ||
| 404 | len -= bytes; | 403 | len -= bytes; |
| 405 | p += bytes; | 404 | p += bytes; |
| @@ -4793,38 +4793,45 @@ static void blk_mq_update_queue_map(struct blk_mq_tag_set *set) · 4792 unmodified lines | |||
| 4793 | } | 4793 | } |
| 4794 | } | 4794 | } |
| 4795 | 4795 | ||
| 4796 | static int blk_mq_realloc_tag_set_tags(struct blk_mq_tag_set *set, | 4796 | static struct blk_mq_tags **blk_mq_prealloc_tag_set_tags( |
| 4797 | int new_nr_hw_queues) | 4797 | struct blk_mq_tag_set *set, |
| 4798 | int new_nr_hw_queues) | ||
| 4798 | { | 4799 | { |
| 4799 | struct blk_mq_tags **new_tags; | 4800 | struct blk_mq_tags **new_tags; |
| 4800 | int i; | 4801 | int i; |
| 4801 | 4802 | ||
| 4802 | if (set->nr_hw_queues >= new_nr_hw_queues) | 4803 | if (set->nr_hw_queues >= new_nr_hw_queues) |
| 4803 | goto done; | 4804 | return NULL; |
| 4804 | 4805 | ||
| 4805 | new_tags = kcalloc_node(new_nr_hw_queues, sizeof(struct blk_mq_tags *), | 4806 | new_tags = kcalloc_node(new_nr_hw_queues, sizeof(struct blk_mq_tags *), |
| 4806 | GFP_KERNEL, set->numa_node); | 4807 | GFP_KERNEL, set->numa_node); |
| 4807 | if (!new_tags) | 4808 | if (!new_tags) |
| 4808 | return -ENOMEM; | 4809 | return ERR_PTR(-ENOMEM); |
| 4809 | 4810 | ||
| 4810 | if (set->tags) | 4811 | if (set->tags) |
| 4811 | memcpy(new_tags, set->tags, set->nr_hw_queues * | 4812 | memcpy(new_tags, set->tags, set->nr_hw_queues * |
| 4812 | sizeof(*set->tags)); | 4813 | sizeof(*set->tags)); |
| 4813 | kfree(set->tags); | ||
| 4814 | set->tags = new_tags; | ||
| 4815 | 4814 | ||
| 4816 | for (i = set->nr_hw_queues; i < new_nr_hw_queues; i++) { | 4815 | for (i = set->nr_hw_queues; i < new_nr_hw_queues; i++) { |
| 4817 | if (!__blk_mq_alloc_map_and_rqs(set, i)) { | 4816 | if (blk_mq_is_shared_tags(set->flags)) { |
| 4818 | while (--i >= set->nr_hw_queues) | 4817 | new_tags[i] = set->shared_tags; |
| 4819 | __blk_mq_free_map_and_rqs(set, i); | 4818 | } else { |
| 4820 | return -ENOMEM; | 4819 | new_tags[i] = blk_mq_alloc_map_and_rqs(set, i, |
| 4820 | set->queue_depth); | ||
| 4821 | if (!new_tags[i]) | ||
| 4822 | goto out_unwind; | ||
| 4821 | } | 4823 | } |
| 4822 | cond_resched(); | 4824 | cond_resched(); |
| 4823 | } | 4825 | } |
| 4824 | 4826 | ||
| 4825 | done: | 4827 | return new_tags; |
| 4826 | set->nr_hw_queues = new_nr_hw_queues; | 4828 | out_unwind: |
| 4827 | return 0; | 4829 | while (--i >= set->nr_hw_queues) { |
| 4830 | if (!blk_mq_is_shared_tags(set->flags)) | ||
| 4831 | blk_mq_free_map_and_rqs(set, new_tags[i], i); | ||
| 4832 | } | ||
| 4833 | kfree(new_tags); | ||
| 4834 | return ERR_PTR(-ENOMEM); | ||
| 4828 | } | 4835 | } |
| 4829 | 4836 | ||
| 4830 | /* | 4837 | /* |
| @@ -5113,6 +5120,7 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, · 282 unmodified lines | |||
| 5113 | unsigned int memflags; | 5120 | unsigned int memflags; |
| 5114 | int i; | 5121 | int i; |
| 5115 | struct xarray elv_tbl; | 5122 | struct xarray elv_tbl; |
| 5123 | struct blk_mq_tags **new_tags; | ||
| 5116 | bool queues_frozen = false; | 5124 | bool queues_frozen = false; |
| 5117 | 5125 | ||
| 5118 | lockdep_assert_held(&set->tag_list_lock); | 5126 | lockdep_assert_held(&set->tag_list_lock); |
| @@ -5147,11 +5155,18 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, · 28 unmodified lines | |||
| 5147 | if (blk_mq_elv_switch_none(q, &elv_tbl)) | 5155 | if (blk_mq_elv_switch_none(q, &elv_tbl)) |
| 5148 | goto switch_back; | 5156 | goto switch_back; |
| 5149 | 5157 | ||
| 5158 | new_tags = blk_mq_prealloc_tag_set_tags(set, nr_hw_queues); | ||
| 5159 | if (IS_ERR(new_tags)) | ||
| 5160 | goto switch_back; | ||
| 5161 | |||
| 5150 | list_for_each_entry(q, &set->tag_list, tag_set_list) | 5162 | list_for_each_entry(q, &set->tag_list, tag_set_list) |
| 5151 | blk_mq_freeze_queue_nomemsave(q); | 5163 | blk_mq_freeze_queue_nomemsave(q); |
| 5152 | queues_frozen = true; | 5164 | queues_frozen = true; |
| 5153 | if (blk_mq_realloc_tag_set_tags(set, nr_hw_queues) < 0) | 5165 | if (new_tags) { |
| 5154 | goto switch_back; | 5166 | kfree(set->tags); |
| 5167 | set->tags = new_tags; | ||
| 5168 | } | ||
| 5169 | set->nr_hw_queues = nr_hw_queues; | ||
| 5155 | 5170 | ||
| 5156 | fallback: | 5171 | fallback: |
| 5157 | blk_mq_update_queue_map(set); | 5172 | blk_mq_update_queue_map(set); |
| @@ -78,8 +78,14 @@ queue_requests_store(struct gendisk *disk, const char *page, size_t count) · 77 unmodified lines | |||
| 78 | /* | 78 | /* |
| 79 | * Serialize updating nr_requests with concurrent queue_requests_store() | 79 | * Serialize updating nr_requests with concurrent queue_requests_store() |
| 80 | * and switching elevator. | 80 | * and switching elevator. |
| 81 | * | ||
| 82 | * Use trylock to avoid circular lock dependency with kernfs active | ||
| 83 | * reference during concurrent disk deletion: | ||
| 84 | * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del) | ||
| 85 | * kn->active -> update_nr_hwq_lock (via this sysfs write path) | ||
| 81 | */ | 86 | */ |
| 82 | down_write(&set->update_nr_hwq_lock); | 87 | if (!down_write_trylock(&set->update_nr_hwq_lock)) |
| 88 | return -EBUSY; | ||
| 83 | 89 | ||
| 84 | if (nr == q->nr_requests) | 90 | if (nr == q->nr_requests) |
| 85 | goto unlock; | 91 | goto unlock; |
| @@ -807,7 +807,16 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, · 806 unmodified lines | |||
| 807 | elv_iosched_load_module(ctx.name); | 807 | elv_iosched_load_module(ctx.name); |
| 808 | ctx.type = elevator_find_get(ctx.name); | 808 | ctx.type = elevator_find_get(ctx.name); |
| 809 | 809 | ||
| 810 | down_read(&set->update_nr_hwq_lock); | 810 | /* |
| 811 | * Use trylock to avoid circular lock dependency with kernfs active | ||
| 812 | * reference during concurrent disk deletion: | ||
| 813 | * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del) | ||
| 814 | * kn->active -> update_nr_hwq_lock (via this sysfs write path) | ||
| 815 | */ | ||
| 816 | if (!down_read_trylock(&set->update_nr_hwq_lock)) { | ||
| 817 | ret = -EBUSY; | ||
| 818 | goto out; | ||
| 819 | } | ||
| 811 | if (!blk_queue_no_elv_switch(q)) { | 820 | if (!blk_queue_no_elv_switch(q)) { |
| 812 | ret = elevator_change(q, &ctx); | 821 | ret = elevator_change(q, &ctx); |
| 813 | if (!ret) | 822 | if (!ret) |
| @@ -817,6 +826,7 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, · 3 unmodified lines | |||
| 817 | } | 826 | } |
| 818 | up_read(&set->update_nr_hwq_lock); | 827 | up_read(&set->update_nr_hwq_lock); |
| 819 | 828 | ||
| 829 | out: | ||
| 820 | if (ctx.type) | 830 | if (ctx.type) |
| 821 | elevator_put(ctx.type); | 831 | elevator_put(ctx.type); |
| 822 | return ret; | 832 | return ret; |
| @@ -876,8 +876,6 @@ config CRYPTO_BLAKE2B · 875 unmodified lines | |||
| 876 | - blake2b-384 | 876 | - blake2b-384 |
| 877 | - blake2b-512 | 877 | - blake2b-512 |
| 878 | 878 | ||
| 879 | Used by the btrfs filesystem. | ||
| 880 | |||
| 881 | See https://blake2.net for further information. | 879 | See https://blake2.net for further information. |
| 882 | 880 | ||
| 883 | config CRYPTO_CMAC | 881 | config CRYPTO_CMAC |
| @@ -965,7 +963,6 @@ config CRYPTO_SHA256 · 81 unmodified lines | |||
| 965 | 10118-3), including HMAC support. | 963 | 10118-3), including HMAC support. |
| 966 | 964 | ||
| 967 | This is required for IPsec AH (XFRM_AH) and IPsec ESP (XFRM_ESP). | 965 | This is required for IPsec AH (XFRM_AH) and IPsec ESP (XFRM_ESP). |
| 968 | Used by the btrfs filesystem, Ceph, NFS, and SMB. | ||
| 969 | 966 | ||
| 970 | config CRYPTO_SHA512 | 967 | config CRYPTO_SHA512 |
| 971 | tristate "SHA-384 and SHA-512" | 968 | tristate "SHA-384 and SHA-512" |
| @@ -1039,8 +1036,6 @@ config CRYPTO_XXHASH · 67 unmodified lines | |||
| 1039 | 1036 | ||
| 1040 | Extremely fast, working at speeds close to RAM limits. | 1037 | Extremely fast, working at speeds close to RAM limits. |
| 1041 | 1038 | ||
| 1042 | Used by the btrfs filesystem. | ||
| 1043 | |||
| 1044 | endmenu | 1039 | endmenu |
| 1045 | 1040 | ||
| 1046 | menu "CRCs (cyclic redundancy checks)" | 1041 | menu "CRCs (cyclic redundancy checks)" |
| @@ -1058,8 +1053,6 @@ config CRYPTO_CRC32C · 11 unmodified lines | |||
| 1058 | on Communications, Vol. 41, No. 6, June 1993, selected for use with | 1053 | on Communications, Vol. 41, No. 6, June 1993, selected for use with |
| 1059 | iSCSI. | 1054 | iSCSI. |
| 1060 | 1055 | ||
| 1061 | Used by btrfs, ext4, jbd2, NVMeoF/TCP, and iSCSI. | ||
| 1062 | |||
| 1063 | config CRYPTO_CRC32 | 1056 | config CRYPTO_CRC32 |
| 1064 | tristate "CRC32" | 1057 | tristate "CRC32" |
| 1065 | select CRYPTO_HASH | 1058 | select CRYPTO_HASH |
| @@ -1067,8 +1060,6 @@ config CRYPTO_CRC32 · 1 unmodified line | |||
| 1067 | help | 1060 | help |
| 1068 | CRC32 CRC algorithm (IEEE 802.3) | 1061 | CRC32 CRC algorithm (IEEE 802.3) |
| 1069 | 1062 | ||
| 1070 | Used by RoCEv2 and f2fs. | ||
| 1071 | |||
| 1072 | endmenu | 1063 | endmenu |
| 1073 | 1064 | ||
| 1074 | menu "Compression" | 1065 | menu "Compression" |
| @@ -4132,7 +4132,7 @@ static const struct alg_test_desc alg_test_descs[] = { · 4131 unmodified lines | |||
| 4132 | .fips_allowed = 1, | 4132 | .fips_allowed = 1, |
| 4133 | }, { | 4133 | }, { |
| 4134 | .alg = "authenc(hmac(sha224),cbc(aes))", | 4134 | .alg = "authenc(hmac(sha224),cbc(aes))", |
| 4135 | .generic_driver = "authenc(hmac-sha224-lib,cbc(aes-generic))", | 4135 | .generic_driver = "authenc(hmac-sha224-lib,cbc(aes-lib))", |
| 4136 | .test = alg_test_aead, | 4136 | .test = alg_test_aead, |
| 4137 | .suite = { | 4137 | .suite = { |
| 4138 | .aead = __VECS(hmac_sha224_aes_cbc_tv_temp) | 4138 | .aead = __VECS(hmac_sha224_aes_cbc_tv_temp) |
| @@ -4194,7 +4194,7 @@ static const struct alg_test_desc alg_test_descs[] = { · 55 unmodified lines | |||
| 4194 | .fips_allowed = 1, | 4194 | .fips_allowed = 1, |
| 4195 | }, { | 4195 | }, { |
| 4196 | .alg = "authenc(hmac(sha384),cbc(aes))", | 4196 | .alg = "authenc(hmac(sha384),cbc(aes))", |
| 4197 | .generic_driver = "authenc(hmac-sha384-lib,cbc(aes-generic))", | 4197 | .generic_driver = "authenc(hmac-sha384-lib,cbc(aes-lib))", |
| 4198 | .test = alg_test_aead, | 4198 | .test = alg_test_aead, |
| 4199 | .suite = { | 4199 | .suite = { |
| 4200 | .aead = __VECS(hmac_sha384_aes_cbc_tv_temp) | 4200 | .aead = __VECS(hmac_sha384_aes_cbc_tv_temp) |
| @@ -23,9 +23,9 @@ · 22 unmodified lines | |||
| 23 | #include "amdxdna_pci_drv.h" | 23 | #include "amdxdna_pci_drv.h" |
| 24 | #include "amdxdna_pm.h" | 24 | #include "amdxdna_pm.h" |
| 25 | 25 | ||
| 26 | static bool force_cmdlist; | 26 | static bool force_cmdlist = true; |
| 27 | module_param(force_cmdlist, bool, 0600); | 27 | module_param(force_cmdlist, bool, 0600); |
| 28 | MODULE_PARM_DESC(force_cmdlist, "Force use command list (Default false)"); | 28 | MODULE_PARM_DESC(force_cmdlist, "Force use command list (Default true)"); |
| 29 | 29 | ||
| 30 | #define HWCTX_MAX_TIMEOUT 60000 /* milliseconds */ | 30 | #define HWCTX_MAX_TIMEOUT 60000 /* milliseconds */ |
| 31 | 31 | ||
| @@ -53,6 +53,7 @@ static void aie2_hwctx_stop(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hwct · 21 unmodified lines | |||
| 53 | { | 53 | { |
| 54 | drm_sched_stop(&hwctx->priv->sched, bad_job); | 54 | drm_sched_stop(&hwctx->priv->sched, bad_job); |
| 55 | aie2_destroy_context(xdna->dev_handle, hwctx); | 55 | aie2_destroy_context(xdna->dev_handle, hwctx); |
| 56 | drm_sched_start(&hwctx->priv->sched, 0); | ||
| 56 | } | 57 | } |
| 57 | 58 | ||
| 58 | static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hwctx) | 59 | static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hwctx) |
| @@ -80,7 +81,6 @@ static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hw · 21 unmodified lines | |||
| 80 | } | 81 | } |
| 81 | 82 | ||
| 82 | out: | 83 | out: |
| 83 | drm_sched_start(&hwctx->priv->sched, 0); | ||
| 84 | XDNA_DBG(xdna, "%s restarted, ret %d", hwctx->name, ret); | 84 | XDNA_DBG(xdna, "%s restarted, ret %d", hwctx->name, ret); |
| 85 | return ret; | 85 | return ret; |
| 86 | } | 86 | } |
| @@ -165,7 +165,6 @@ aie2_sched_notify(struct amdxdna_sched_job *job) · 78 unmodified lines | |||
| 165 | 165 | ||
| 166 | trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq); | 166 | trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq); |
| 167 | 167 | ||
| 168 | amdxdna_pm_suspend_put(job->hwctx->client->xdna); | ||
| 169 | job->hwctx->priv->completed++; | 168 | job->hwctx->priv->completed++; |
| 170 | dma_fence_signal(fence); | 169 | dma_fence_signal(fence); |
| 171 | 170 | ||
| @@ -186,13 +185,13 @@ aie2_sched_resp_handler(void *handle, void __iomem *data, size_t size) · 14 unmodified lines | |||
| 186 | cmd_abo = job->cmd_bo; | 185 | cmd_abo = job->cmd_bo; |
| 187 | 186 | ||
| 188 | if (unlikely(job->job_timeout)) { | 187 | if (unlikely(job->job_timeout)) { |
| 189 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT); | 188 | amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_TIMEOUT); |
| 190 | ret = -EINVAL; | 189 | ret = -EINVAL; |
| 191 | goto out; | 190 | goto out; |
| 192 | } | 191 | } |
| 193 | 192 | ||
| 194 | if (unlikely(!data) || unlikely(size != sizeof(u32))) { | 193 | if (unlikely(!data) || unlikely(size != sizeof(u32))) { |
| 195 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | 194 | amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ABORT); |
| 196 | ret = -EINVAL; | 195 | ret = -EINVAL; |
| 197 | goto out; | 196 | goto out; |
| 198 | } | 197 | } |
| @@ -202,7 +201,7 @@ aie2_sched_resp_handler(void *handle, void __iomem *data, size_t size) · 3 unmodified lines | |||
| 202 | if (status == AIE2_STATUS_SUCCESS) | 201 | if (status == AIE2_STATUS_SUCCESS) |
| 203 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_COMPLETED); | 202 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_COMPLETED); |
| 204 | else | 203 | else |
| 205 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ERROR); | 204 | amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ERROR); |
| 206 | 205 | ||
| 207 | out: | 206 | out: |
| 208 | aie2_sched_notify(job); | 207 | aie2_sched_notify(job); |
| @@ -244,13 +243,13 @@ aie2_sched_cmdlist_resp_handler(void *handle, void __iomem *data, size_t size) · 35 unmodified lines | |||
| 244 | cmd_abo = job->cmd_bo; | 243 | cmd_abo = job->cmd_bo; |
| 245 | 244 | ||
| 246 | if (unlikely(job->job_timeout)) { | 245 | if (unlikely(job->job_timeout)) { |
| 247 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT); | 246 | amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_TIMEOUT); |
| 248 | ret = -EINVAL; | 247 | ret = -EINVAL; |
| 249 | goto out; | 248 | goto out; |
| 250 | } | 249 | } |
| 251 | 250 | ||
| 252 | if (unlikely(!data) || unlikely(size != sizeof(u32) * 3)) { | 251 | if (unlikely(!data) || unlikely(size != sizeof(u32) * 3)) { |
| 253 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | 252 | amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ABORT); |
| 254 | ret = -EINVAL; | 253 | ret = -EINVAL; |
| 255 | goto out; | 254 | goto out; |
| 256 | } | 255 | } |
| @@ -270,19 +269,12 @@ aie2_sched_cmdlist_resp_handler(void *handle, void __iomem *data, size_t size) · 13 unmodified lines | |||
| 270 | fail_cmd_idx, fail_cmd_status); | 269 | fail_cmd_idx, fail_cmd_status); |
| 271 | 270 | ||
| 272 | if (fail_cmd_status == AIE2_STATUS_SUCCESS) { | 271 | if (fail_cmd_status == AIE2_STATUS_SUCCESS) { |
| 273 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | 272 | amdxdna_cmd_set_error(cmd_abo, job, fail_cmd_idx, ERT_CMD_STATE_ABORT); |
| 274 | ret = -EINVAL; | 273 | ret = -EINVAL; |
| 275 | goto out; | 274 | } else { |
| 275 | amdxdna_cmd_set_error(cmd_abo, job, fail_cmd_idx, ERT_CMD_STATE_ERROR); | ||
| 276 | } | 276 | } |
| 277 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ERROR); | ||
| 278 | |||
| 279 | if (amdxdna_cmd_get_op(cmd_abo) == ERT_CMD_CHAIN) { | ||
| 280 | struct amdxdna_cmd_chain *cc = amdxdna_cmd_get_payload(cmd_abo, NULL); | ||
| 281 | 277 | ||
| 282 | cc->error_index = fail_cmd_idx; | ||
| 283 | if (cc->error_index >= cc->command_count) | ||
| 284 | cc->error_index = 0; | ||
| 285 | } | ||
| 286 | out: | 278 | out: |
| 287 | aie2_sched_notify(job); | 279 | aie2_sched_notify(job); |
| 288 | return ret; | 280 | return ret; |
| @@ -306,10 +298,6 @@ aie2_sched_job_run(struct drm_sched_job *sched_job) · 17 unmodified lines | |||
| 306 | kref_get(&job->refcnt); | 298 | kref_get(&job->refcnt); |
| 307 | fence = dma_fence_get(job->fence); | 299 | fence = dma_fence_get(job->fence); |
| 308 | 300 | ||
| 309 | ret = amdxdna_pm_resume_get(hwctx->client->xdna); | ||
| 310 | if (ret) | ||
| 311 | goto out; | ||
| 312 | |||
| 313 | if (job->drv_cmd) { | 301 | if (job->drv_cmd) { |
| 314 | switch (job->drv_cmd->opcode) { | 302 | switch (job->drv_cmd->opcode) { |
| 315 | case SYNC_DEBUG_BO: | 303 | case SYNC_DEBUG_BO: |
| @@ -336,7 +324,6 @@ aie2_sched_job_run(struct drm_sched_job *sched_job) · 20 unmodified lines | |||
| 336 | 324 | ||
| 337 | out: | 325 | out: |
| 338 | if (ret) { | 326 | if (ret) { |
| 339 | amdxdna_pm_suspend_put(hwctx->client->xdna); | ||
| 340 | dma_fence_put(job->fence); | 327 | dma_fence_put(job->fence); |
| 341 | aie2_job_put(job); | 328 | aie2_job_put(job); |
| 342 | mmput(job->mm); | 329 | mmput(job->mm); |
| @@ -497,7 +484,7 @@ static void aie2_release_resource(struct amdxdna_hwctx *hwctx) · 154 unmodified lines | |||
| 497 | 484 | ||
| 498 | if (AIE2_FEATURE_ON(xdna->dev_handle, AIE2_TEMPORAL_ONLY)) { | 485 | if (AIE2_FEATURE_ON(xdna->dev_handle, AIE2_TEMPORAL_ONLY)) { |
| 499 | ret = aie2_destroy_context(xdna->dev_handle, hwctx); | 486 | ret = aie2_destroy_context(xdna->dev_handle, hwctx); |
| 500 | if (ret) | 487 | if (ret && ret != -ENODEV) |
| 501 | XDNA_ERR(xdna, "Destroy temporal only context failed, ret %d", ret); | 488 | XDNA_ERR(xdna, "Destroy temporal only context failed, ret %d", ret); |
| 502 | } else { | 489 | } else { |
| 503 | ret = xrs_release_resource(xdna->xrs_hdl, (uintptr_t)hwctx); | 490 | ret = xrs_release_resource(xdna->xrs_hdl, (uintptr_t)hwctx); |
| @@ -629,7 +616,7 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 125 unmodified lines | |||
| 629 | goto free_entity; | 616 | goto free_entity; |
| 630 | } | 617 | } |
| 631 | 618 | ||
| 632 | ret = amdxdna_pm_resume_get(xdna); | 619 | ret = amdxdna_pm_resume_get_locked(xdna); |
| 633 | if (ret) | 620 | if (ret) |
| 634 | goto free_col_list; | 621 | goto free_col_list; |
| 635 | 622 | ||
| @@ -760,7 +747,7 @@ static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size · 124 unmodified lines | |||
| 760 | if (!hwctx->cus) | 747 | if (!hwctx->cus) |
| 761 | return -ENOMEM; | 748 | return -ENOMEM; |
| 762 | 749 | ||
| 763 | ret = amdxdna_pm_resume_get(xdna); | 750 | ret = amdxdna_pm_resume_get_locked(xdna); |
| 764 | if (ret) | 751 | if (ret) |
| 765 | goto free_cus; | 752 | goto free_cus; |
| 766 | 753 | ||
| @@ -1070,6 +1057,8 @@ void aie2_hmm_invalidate(struct amdxdna_gem_obj *abo, · 303 unmodified lines | |||
| 1070 | 1057 | ||
| 1071 | ret = dma_resv_wait_timeout(gobj->resv, DMA_RESV_USAGE_BOOKKEEP, | 1058 | ret = dma_resv_wait_timeout(gobj->resv, DMA_RESV_USAGE_BOOKKEEP, |
| 1072 | true, MAX_SCHEDULE_TIMEOUT); | 1059 | true, MAX_SCHEDULE_TIMEOUT); |
| 1073 | if (!ret || ret == -ERESTARTSYS) | 1060 | if (!ret) |
| 1074 | XDNA_ERR(xdna, "Failed to wait for bo, ret %ld", ret); | 1061 | XDNA_ERR(xdna, "Failed to wait for bo, ret %ld", ret); |
| 1062 | else if (ret == -ERESTARTSYS) | ||
| 1063 | XDNA_DBG(xdna, "Wait for bo interrupted by signal"); | ||
| 1075 | } | 1064 | } |
| @@ -40,11 +40,8 @@ static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev, · 39 unmodified lines | |||
| 40 | return -ENODEV; | 40 | return -ENODEV; |
| 41 | 41 | ||
| 42 | ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg); | 42 | ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg); |
| 43 | if (ret == -ETIME) { | 43 | if (ret == -ETIME) |
| 44 | xdna_mailbox_stop_channel(ndev->mgmt_chann); | 44 | aie2_destroy_mgmt_chann(ndev); |
| 45 | xdna_mailbox_destroy_channel(ndev->mgmt_chann); | ||
| 46 | ndev->mgmt_chann = NULL; | ||
| 47 | } | ||
| 48 | 45 | ||
| 49 | if (!ret && *hdl->status != AIE2_STATUS_SUCCESS) { | 46 | if (!ret && *hdl->status != AIE2_STATUS_SUCCESS) { |
| 50 | XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x", | 47 | XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x", |
| @@ -216,8 +213,10 @@ static int aie2_destroy_context_req(struct amdxdna_dev_hdl *ndev, u32 id) · 165 unmodified lines | |||
| 216 | 213 | ||
| 217 | req.context_id = id; | 214 | req.context_id = id; |
| 218 | ret = aie2_send_mgmt_msg_wait(ndev, &msg); | 215 | ret = aie2_send_mgmt_msg_wait(ndev, &msg); |
| 219 | if (ret) | 216 | if (ret && ret != -ENODEV) |
| 220 | XDNA_WARN(xdna, "Destroy context failed, ret %d", ret); | 217 | XDNA_WARN(xdna, "Destroy context failed, ret %d", ret); |
| 218 | else if (ret == -ENODEV) | ||
| 219 | XDNA_DBG(xdna, "Destroy context: device already stopped"); | ||
| 221 | 220 | ||
| 222 | return ret; | 221 | return ret; |
| 223 | } | 222 | } |
| @@ -294,13 +293,20 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 70 unmodified lines | |||
| 294 | } | 293 | } |
| 295 | 294 | ||
| 296 | intr_reg = i2x.mb_head_ptr_reg + 4; | 295 | intr_reg = i2x.mb_head_ptr_reg + 4; |
| 297 | hwctx->priv->mbox_chann = xdna_mailbox_create_channel(ndev->mbox, &x2i, &i2x, | 296 | hwctx->priv->mbox_chann = xdna_mailbox_alloc_channel(ndev->mbox); |
| 298 | intr_reg, ret); | ||
| 299 | if (!hwctx->priv->mbox_chann) { | 297 | if (!hwctx->priv->mbox_chann) { |
| 300 | XDNA_ERR(xdna, "Not able to create channel"); | 298 | XDNA_ERR(xdna, "Not able to create channel"); |
| 301 | ret = -EINVAL; | 299 | ret = -EINVAL; |
| 302 | goto del_ctx_req; | 300 | goto del_ctx_req; |
| 303 | } | 301 | } |
| 302 | |||
| 303 | ret = xdna_mailbox_start_channel(hwctx->priv->mbox_chann, &x2i, &i2x, | ||
| 304 | intr_reg, ret); | ||
| 305 | if (ret) { | ||
| 306 | XDNA_ERR(xdna, "Not able to create channel"); | ||
| 307 | ret = -EINVAL; | ||
| 308 | goto free_channel; | ||
| 309 | } | ||
| 304 | ndev->hwctx_num++; | 310 | ndev->hwctx_num++; |
| 305 | 311 | ||
| 306 | XDNA_DBG(xdna, "Mailbox channel irq: %d, msix_id: %d", ret, resp.msix_id); | 312 | XDNA_DBG(xdna, "Mailbox channel irq: %d, msix_id: %d", ret, resp.msix_id); |
| @@ -308,6 +314,8 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 1 unmodified line | |||
| 308 | 314 | ||
| 309 | return 0; | 315 | return 0; |
| 310 | 316 | ||
| 317 | free_channel: | ||
| 318 | xdna_mailbox_free_channel(hwctx->priv->mbox_chann); | ||
| 311 | del_ctx_req: | 319 | del_ctx_req: |
| 312 | aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); | 320 | aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); |
| 313 | return ret; | 321 | return ret; |
| @@ -318,9 +326,12 @@ int aie2_destroy_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwc · 4 unmodified lines | |||
| 318 | struct amdxdna_dev *xdna = ndev->xdna; | 326 | struct amdxdna_dev *xdna = ndev->xdna; |
| 319 | int ret; | 327 | int ret; |
| 320 | 328 | ||
| 329 | if (!hwctx->priv->mbox_chann) | ||
| 330 | return 0; | ||
| 331 | |||
| 321 | xdna_mailbox_stop_channel(hwctx->priv->mbox_chann); | 332 | xdna_mailbox_stop_channel(hwctx->priv->mbox_chann); |
| 322 | ret = aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); | 333 | ret = aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); |
| 323 | xdna_mailbox_destroy_channel(hwctx->priv->mbox_chann); | 334 | xdna_mailbox_free_channel(hwctx->priv->mbox_chann); |
| 324 | XDNA_DBG(xdna, "Destroyed fw ctx %d", hwctx->fw_ctx_id); | 335 | XDNA_DBG(xdna, "Destroyed fw ctx %d", hwctx->fw_ctx_id); |
| 325 | hwctx->priv->mbox_chann = NULL; | 336 | hwctx->priv->mbox_chann = NULL; |
| 326 | hwctx->fw_ctx_id = -1; | 337 | hwctx->fw_ctx_id = -1; |
| @@ -694,11 +705,11 @@ aie2_cmdlist_fill_npu_cf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *siz · 367 unmodified lines | |||
| 694 | u32 cmd_len; | 705 | u32 cmd_len; |
| 695 | void *cmd; | 706 | void *cmd; |
| 696 | 707 | ||
| 697 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 698 | cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); | 708 | cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 699 | if (*size < sizeof(*npu_slot) + cmd_len) | 709 | if (*size < sizeof(*npu_slot) + cmd_len) |
| 700 | return -EINVAL; | 710 | return -EINVAL; |
| 701 | 711 | ||
| 712 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 702 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); | 713 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 703 | if (npu_slot->cu_idx == INVALID_CU_IDX) | 714 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 704 | return -EINVAL; | 715 | return -EINVAL; |
| @@ -719,7 +730,6 @@ aie2_cmdlist_fill_npu_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 14 unmodified lines | |||
| 719 | u32 cmd_len; | 730 | u32 cmd_len; |
| 720 | u32 arg_sz; | 731 | u32 arg_sz; |
| 721 | 732 | ||
| 722 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 723 | sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); | 733 | sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 724 | arg_sz = cmd_len - sizeof(*sn); | 734 | arg_sz = cmd_len - sizeof(*sn); |
| 725 | if (cmd_len < sizeof(*sn) || arg_sz > MAX_NPU_ARGS_SIZE) | 735 | if (cmd_len < sizeof(*sn) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -728,6 +738,7 @@ aie2_cmdlist_fill_npu_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 2 unmodified lines | |||
| 728 | if (*size < sizeof(*npu_slot) + arg_sz) | 738 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 729 | return -EINVAL; | 739 | return -EINVAL; |
| 730 | 740 | ||
| 741 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 731 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); | 742 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 732 | if (npu_slot->cu_idx == INVALID_CU_IDX) | 743 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 733 | return -EINVAL; | 744 | return -EINVAL; |
| @@ -751,7 +762,6 @@ aie2_cmdlist_fill_npu_preempt(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t · 17 unmodified lines | |||
| 751 | u32 cmd_len; | 762 | u32 cmd_len; |
| 752 | u32 arg_sz; | 763 | u32 arg_sz; |
| 753 | 764 | ||
| 754 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 755 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); | 765 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 756 | arg_sz = cmd_len - sizeof(*pd); | 766 | arg_sz = cmd_len - sizeof(*pd); |
| 757 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) | 767 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -760,6 +770,7 @@ aie2_cmdlist_fill_npu_preempt(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t · 2 unmodified lines | |||
| 760 | if (*size < sizeof(*npu_slot) + arg_sz) | 770 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 761 | return -EINVAL; | 771 | return -EINVAL; |
| 762 | 772 | ||
| 773 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 763 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); | 774 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 764 | if (npu_slot->cu_idx == INVALID_CU_IDX) | 775 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 765 | return -EINVAL; | 776 | return -EINVAL; |
| @@ -787,7 +798,6 @@ aie2_cmdlist_fill_npu_elf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 21 unmodified lines | |||
| 787 | u32 cmd_len; | 798 | u32 cmd_len; |
| 788 | u32 arg_sz; | 799 | u32 arg_sz; |
| 789 | 800 | ||
| 790 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 791 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); | 801 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 792 | arg_sz = cmd_len - sizeof(*pd); | 802 | arg_sz = cmd_len - sizeof(*pd); |
| 793 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) | 803 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -796,6 +806,7 @@ aie2_cmdlist_fill_npu_elf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 2 unmodified lines | |||
| 796 | if (*size < sizeof(*npu_slot) + arg_sz) | 806 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 797 | return -EINVAL; | 807 | return -EINVAL; |
| 798 | 808 | ||
| 809 | memset(npu_slot, 0, sizeof(*npu_slot)); | ||
| 799 | npu_slot->type = EXEC_NPU_TYPE_ELF; | 810 | npu_slot->type = EXEC_NPU_TYPE_ELF; |
| 800 | npu_slot->inst_buf_addr = pd->inst_buf; | 811 | npu_slot->inst_buf_addr = pd->inst_buf; |
| 801 | npu_slot->save_buf_addr = pd->save_buf; | 812 | npu_slot->save_buf_addr = pd->save_buf; |
| @@ -909,6 +920,20 @@ void aie2_msg_init(struct amdxdna_dev_hdl *ndev) · 107 unmodified lines | |||
| 909 | ndev->exec_msg_ops = &legacy_exec_message_ops; | 920 | ndev->exec_msg_ops = &legacy_exec_message_ops; |
| 910 | } | 921 | } |
| 911 | 922 | ||
| 923 | void aie2_destroy_mgmt_chann(struct amdxdna_dev_hdl *ndev) | ||
| 924 | { | ||
| 925 | struct amdxdna_dev *xdna = ndev->xdna; | ||
| 926 | |||
| 927 | drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock)); | ||
| 928 | |||
| 929 | if (!ndev->mgmt_chann) | ||
| 930 | return; | ||
| 931 | |||
| 932 | xdna_mailbox_stop_channel(ndev->mgmt_chann); | ||
| 933 | xdna_mailbox_free_channel(ndev->mgmt_chann); | ||
| 934 | ndev->mgmt_chann = NULL; | ||
| 935 | } | ||
| 936 | |||
| 912 | static inline struct amdxdna_gem_obj * | 937 | static inline struct amdxdna_gem_obj * |
| 913 | aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job) | 938 | aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job) |
| 914 | { | 939 | { |
| @@ -32,6 +32,11 @@ static int aie2_max_col = XRS_MAX_COL; · 31 unmodified lines | |||
| 32 | module_param(aie2_max_col, uint, 0600); | 32 | module_param(aie2_max_col, uint, 0600); |
| 33 | MODULE_PARM_DESC(aie2_max_col, "Maximum column could be used"); | 33 | MODULE_PARM_DESC(aie2_max_col, "Maximum column could be used"); |
| 34 | 34 | ||
| 35 | static char *npu_fw[] = { | ||
| 36 | "npu_7.sbin", | ||
| 37 | "npu.sbin" | ||
| 38 | }; | ||
| 39 | |||
| 35 | /* | 40 | /* |
| 36 | * The management mailbox channel is allocated by firmware. | 41 | * The management mailbox channel is allocated by firmware. |
| 37 | * The related register and ring buffer information is on SRAM BAR. | 42 | * The related register and ring buffer information is on SRAM BAR. |
| @@ -323,10 +328,9 @@ static void aie2_hw_stop(struct amdxdna_dev *xdna) · 285 unmodified lines | |||
| 323 | return; | 328 | return; |
| 324 | } | 329 | } |
| 325 | 330 | ||
| 331 | aie2_runtime_cfg(ndev, AIE2_RT_CFG_CLK_GATING, NULL); | ||
| 326 | aie2_mgmt_fw_fini(ndev); | 332 | aie2_mgmt_fw_fini(ndev); |
| 327 | xdna_mailbox_stop_channel(ndev->mgmt_chann); | 333 | aie2_destroy_mgmt_chann(ndev); |
| 328 | xdna_mailbox_destroy_channel(ndev->mgmt_chann); | ||
| 329 | ndev->mgmt_chann = NULL; | ||
| 330 | drmm_kfree(&xdna->ddev, ndev->mbox); | 334 | drmm_kfree(&xdna->ddev, ndev->mbox); |
| 331 | ndev->mbox = NULL; | 335 | ndev->mbox = NULL; |
| 332 | aie2_psp_stop(ndev->psp_hdl); | 336 | aie2_psp_stop(ndev->psp_hdl); |
| @@ -357,10 +361,29 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 24 unmodified lines | |||
| 357 | } | 361 | } |
| 358 | pci_set_master(pdev); | 362 | pci_set_master(pdev); |
| 359 | 363 | ||
| 364 | mbox_res.ringbuf_base = ndev->sram_base; | ||
| 365 | mbox_res.ringbuf_size = pci_resource_len(pdev, xdna->dev_info->sram_bar); | ||
| 366 | mbox_res.mbox_base = ndev->mbox_base; | ||
| 367 | mbox_res.mbox_size = MBOX_SIZE(ndev); | ||
| 368 | mbox_res.name = "xdna_mailbox"; | ||
| 369 | ndev->mbox = xdnam_mailbox_create(&xdna->ddev, &mbox_res); | ||
| 370 | if (!ndev->mbox) { | ||
| 371 | XDNA_ERR(xdna, "failed to create mailbox device"); | ||
| 372 | ret = -ENODEV; | ||
| 373 | goto disable_dev; | ||
| 374 | } | ||
| 375 | |||
| 376 | ndev->mgmt_chann = xdna_mailbox_alloc_channel(ndev->mbox); | ||
| 377 | if (!ndev->mgmt_chann) { | ||
| 378 | XDNA_ERR(xdna, "failed to alloc channel"); | ||
| 379 | ret = -ENODEV; | ||
| 380 | goto disable_dev; | ||
| 381 | } | ||
| 382 | |||
| 360 | ret = aie2_smu_init(ndev); | 383 | ret = aie2_smu_init(ndev); |
| 361 | if (ret) { | 384 | if (ret) { |
| 362 | XDNA_ERR(xdna, "failed to init smu, ret %d", ret); | 385 | XDNA_ERR(xdna, "failed to init smu, ret %d", ret); |
| 363 | goto disable_dev; | 386 | goto free_channel; |
| 364 | } | 387 | } |
| 365 | 388 | ||
| 366 | ret = aie2_psp_start(ndev->psp_hdl); | 389 | ret = aie2_psp_start(ndev->psp_hdl); |
| @@ -375,18 +398,6 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 8 unmodified lines | |||
| 375 | goto stop_psp; | 398 | goto stop_psp; |
| 376 | } | 399 | } |
| 377 | 400 | ||
| 378 | mbox_res.ringbuf_base = ndev->sram_base; | ||
| 379 | mbox_res.ringbuf_size = pci_resource_len(pdev, xdna->dev_info->sram_bar); | ||
| 380 | mbox_res.mbox_base = ndev->mbox_base; | ||
| 381 | mbox_res.mbox_size = MBOX_SIZE(ndev); | ||
| 382 | mbox_res.name = "xdna_mailbox"; | ||
| 383 | ndev->mbox = xdnam_mailbox_create(&xdna->ddev, &mbox_res); | ||
| 384 | if (!ndev->mbox) { | ||
| 385 | XDNA_ERR(xdna, "failed to create mailbox device"); | ||
| 386 | ret = -ENODEV; | ||
| 387 | goto stop_psp; | ||
| 388 | } | ||
| 389 | |||
| 390 | mgmt_mb_irq = pci_irq_vector(pdev, ndev->mgmt_chan_idx); | 401 | mgmt_mb_irq = pci_irq_vector(pdev, ndev->mgmt_chan_idx); |
| 391 | if (mgmt_mb_irq < 0) { | 402 | if (mgmt_mb_irq < 0) { |
| 392 | ret = mgmt_mb_irq; | 403 | ret = mgmt_mb_irq; |
| @@ -395,52 +406,55 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 2 unmodified lines | |||
| 395 | } | 406 | } |
| 396 | 407 | ||
| 397 | xdna_mailbox_intr_reg = ndev->mgmt_i2x.mb_head_ptr_reg + 4; | 408 | xdna_mailbox_intr_reg = ndev->mgmt_i2x.mb_head_ptr_reg + 4; |
| 398 | ndev->mgmt_chann = xdna_mailbox_create_channel(ndev->mbox, | 409 | ret = xdna_mailbox_start_channel(ndev->mgmt_chann, |
| 399 | &ndev->mgmt_x2i, | 410 | &ndev->mgmt_x2i, |
| 400 | &ndev->mgmt_i2x, | 411 | &ndev->mgmt_i2x, |
| 401 | xdna_mailbox_intr_reg, | 412 | xdna_mailbox_intr_reg, |
| 402 | mgmt_mb_irq); | 413 | mgmt_mb_irq); |
| 403 | if (!ndev->mgmt_chann) { | 414 | if (ret) { |
| 404 | XDNA_ERR(xdna, "failed to create management mailbox channel"); | 415 | XDNA_ERR(xdna, "failed to start management mailbox channel"); |
| 405 | ret = -EINVAL; | 416 | ret = -EINVAL; |
| 406 | goto stop_psp; | 417 | goto stop_psp; |
| 407 | } | 418 | } |
| 408 | 419 | ||
| 409 | ret = aie2_pm_init(ndev); | 420 | ret = aie2_mgmt_fw_init(ndev); |
| 410 | if (ret) { | 421 | if (ret) { |
| 411 | XDNA_ERR(xdna, "failed to init pm, ret %d", ret); | 422 | XDNA_ERR(xdna, "initial mgmt firmware failed, ret %d", ret); |
| 412 | goto destroy_mgmt_chann; | 423 | goto stop_fw; |
| 413 | } | 424 | } |
| 414 | 425 | ||
| 415 | ret = aie2_mgmt_fw_init(ndev); | 426 | ret = aie2_pm_init(ndev); |
| 416 | if (ret) { | 427 | if (ret) { |
| 417 | XDNA_ERR(xdna, "initial mgmt firmware failed, ret %d", ret); | 428 | XDNA_ERR(xdna, "failed to init pm, ret %d", ret); |
| 418 | goto destroy_mgmt_chann; | 429 | goto stop_fw; |
| 419 | } | 430 | } |
| 420 | 431 | ||
| 421 | ret = aie2_mgmt_fw_query(ndev); | 432 | ret = aie2_mgmt_fw_query(ndev); |
| 422 | if (ret) { | 433 | if (ret) { |
| 423 | XDNA_ERR(xdna, "failed to query fw, ret %d", ret); | 434 | XDNA_ERR(xdna, "failed to query fw, ret %d", ret); |
| 424 | goto destroy_mgmt_chann; | 435 | goto stop_fw; |
| 425 | } | 436 | } |
| 426 | 437 | ||
| 427 | ret = aie2_error_async_events_alloc(ndev); | 438 | ret = aie2_error_async_events_alloc(ndev); |
| 428 | if (ret) { | 439 | if (ret) { |
| 429 | XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret); | 440 | XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret); |
| 430 | goto destroy_mgmt_chann; | 441 | goto stop_fw; |
| 431 | } | 442 | } |
| 432 | 443 | ||
| 433 | ndev->dev_status = AIE2_DEV_START; | 444 | ndev->dev_status = AIE2_DEV_START; |
| 434 | 445 | ||
| 435 | return 0; | 446 | return 0; |
| 436 | 447 | ||
| 437 | destroy_mgmt_chann: | 448 | stop_fw: |
| 449 | aie2_suspend_fw(ndev); | ||
| 438 | xdna_mailbox_stop_channel(ndev->mgmt_chann); | 450 | xdna_mailbox_stop_channel(ndev->mgmt_chann); |
| 439 | xdna_mailbox_destroy_channel(ndev->mgmt_chann); | ||
| 440 | stop_psp: | 451 | stop_psp: |
| 441 | aie2_psp_stop(ndev->psp_hdl); | 452 | aie2_psp_stop(ndev->psp_hdl); |
| 442 | fini_smu: | 453 | fini_smu: |
| 443 | aie2_smu_fini(ndev); | 454 | aie2_smu_fini(ndev); |
| 455 | free_channel: | ||
| 456 | xdna_mailbox_free_channel(ndev->mgmt_chann); | ||
| 457 | ndev->mgmt_chann = NULL; | ||
| 444 | disable_dev: | 458 | disable_dev: |
| 445 | pci_disable_device(pdev); | 459 | pci_disable_device(pdev); |
| 446 | 460 | ||
| @@ -451,7 +465,6 @@ static int aie2_hw_suspend(struct amdxdna_dev *xdna) · 4 unmodified lines | |||
| 451 | { | 465 | { |
| 452 | struct amdxdna_client *client; | 466 | struct amdxdna_client *client; |
| 453 | 467 | ||
| 454 | guard(mutex)(&xdna->dev_lock); | ||
| 455 | list_for_each_entry(client, &xdna->client_list, node) | 468 | list_for_each_entry(client, &xdna->client_list, node) |
| 456 | aie2_hwctx_suspend(client); | 469 | aie2_hwctx_suspend(client); |
| 457 | 470 | ||
| @@ -489,6 +502,7 @@ static int aie2_init(struct amdxdna_dev *xdna) · 31 unmodified lines | |||
| 489 | struct psp_config psp_conf; | 502 | struct psp_config psp_conf; |
| 490 | const struct firmware *fw; | 503 | const struct firmware *fw; |
| 491 | unsigned long bars = 0; | 504 | unsigned long bars = 0; |
| 505 | char *fw_full_path; | ||
| 492 | int i, nvec, ret; | 506 | int i, nvec, ret; |
| 493 | 507 | ||
| 494 | if (!hypervisor_is_type(X86_HYPER_NATIVE)) { | 508 | if (!hypervisor_is_type(X86_HYPER_NATIVE)) { |
| @@ -503,7 +517,19 @@ static int aie2_init(struct amdxdna_dev *xdna) · 8 unmodified lines | |||
| 503 | ndev->priv = xdna->dev_info->dev_priv; | 517 | ndev->priv = xdna->dev_info->dev_priv; |
| 504 | ndev->xdna = xdna; | 518 | ndev->xdna = xdna; |
| 505 | 519 | ||
| 506 | ret = request_firmware(&fw, ndev->priv->fw_path, &pdev->dev); | 520 | for (i = 0; i < ARRAY_SIZE(npu_fw); i++) { |
| 521 | fw_full_path = kasprintf(GFP_KERNEL, "%s%s", ndev->priv->fw_path, npu_fw[i]); | ||
| 522 | if (!fw_full_path) | ||
| 523 | return -ENOMEM; | ||
| 524 | |||
| 525 | ret = firmware_request_nowarn(&fw, fw_full_path, &pdev->dev); | ||
| 526 | kfree(fw_full_path); | ||
| 527 | if (!ret) { | ||
| 528 | XDNA_INFO(xdna, "Load firmware %s%s", ndev->priv->fw_path, npu_fw[i]); | ||
| 529 | break; | ||
| 530 | } | ||
| 531 | } | ||
| 532 | |||
| 507 | if (ret) { | 533 | if (ret) { |
| 508 | XDNA_ERR(xdna, "failed to request_firmware %s, ret %d", | 534 | XDNA_ERR(xdna, "failed to request_firmware %s, ret %d", |
| 509 | ndev->priv->fw_path, ret); | 535 | ndev->priv->fw_path, ret); |
| @@ -951,7 +977,7 @@ static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_i · 441 unmodified lines | |||
| 951 | if (!drm_dev_enter(&xdna->ddev, &idx)) | 977 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 952 | return -ENODEV; | 978 | return -ENODEV; |
| 953 | 979 | ||
| 954 | ret = amdxdna_pm_resume_get(xdna); | 980 | ret = amdxdna_pm_resume_get_locked(xdna); |
| 955 | if (ret) | 981 | if (ret) |
| 956 | goto dev_exit; | 982 | goto dev_exit; |
| 957 | 983 | ||
| @@ -1044,7 +1070,7 @@ static int aie2_get_array(struct amdxdna_client *client, · 86 unmodified lines | |||
| 1044 | if (!drm_dev_enter(&xdna->ddev, &idx)) | 1070 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 1045 | return -ENODEV; | 1071 | return -ENODEV; |
| 1046 | 1072 | ||
| 1047 | ret = amdxdna_pm_resume_get(xdna); | 1073 | ret = amdxdna_pm_resume_get_locked(xdna); |
| 1048 | if (ret) | 1074 | if (ret) |
| 1049 | goto dev_exit; | 1075 | goto dev_exit; |
| 1050 | 1076 | ||
| @@ -1134,7 +1160,7 @@ static int aie2_set_state(struct amdxdna_client *client, · 83 unmodified lines | |||
| 1134 | if (!drm_dev_enter(&xdna->ddev, &idx)) | 1160 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 1135 | return -ENODEV; | 1161 | return -ENODEV; |
| 1136 | 1162 | ||
| 1137 | ret = amdxdna_pm_resume_get(xdna); | 1163 | ret = amdxdna_pm_resume_get_locked(xdna); |
| 1138 | if (ret) | 1164 | if (ret) |
| 1139 | goto dev_exit; | 1165 | goto dev_exit; |
| 1140 | 1166 | ||
| @@ -303,6 +303,7 @@ int aie2_get_array_async_error(struct amdxdna_dev_hdl *ndev, · 302 unmodified lines | |||
| 303 | 303 | ||
| 304 | /* aie2_message.c */ | 304 | /* aie2_message.c */ |
| 305 | void aie2_msg_init(struct amdxdna_dev_hdl *ndev); | 305 | void aie2_msg_init(struct amdxdna_dev_hdl *ndev); |
| 306 | void aie2_destroy_mgmt_chann(struct amdxdna_dev_hdl *ndev); | ||
| 306 | int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev); | 307 | int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev); |
| 307 | int aie2_resume_fw(struct amdxdna_dev_hdl *ndev); | 308 | int aie2_resume_fw(struct amdxdna_dev_hdl *ndev); |
| 308 | int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value); | 309 | int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value); |
| @@ -31,7 +31,7 @@ int aie2_pm_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level) · 30 unmodified lines | |||
| 31 | { | 31 | { |
| 32 | int ret; | 32 | int ret; |
| 33 | 33 | ||
| 34 | ret = amdxdna_pm_resume_get(ndev->xdna); | 34 | ret = amdxdna_pm_resume_get_locked(ndev->xdna); |
| 35 | if (ret) | 35 | if (ret) |
| 36 | return ret; | 36 | return ret; |
| 37 | 37 | ||
| @@ -17,6 +17,7 @@ · 16 unmodified lines | |||
| 17 | #include "amdxdna_ctx.h" | 17 | #include "amdxdna_ctx.h" |
| 18 | #include "amdxdna_gem.h" | 18 | #include "amdxdna_gem.h" |
| 19 | #include "amdxdna_pci_drv.h" | 19 | #include "amdxdna_pci_drv.h" |
| 20 | #include "amdxdna_pm.h" | ||
| 20 | 21 | ||
| 21 | #define MAX_HWCTX_ID 255 | 22 | #define MAX_HWCTX_ID 255 |
| 22 | #define MAX_ARG_COUNT 4095 | 23 | #define MAX_ARG_COUNT 4095 |
| @@ -104,7 +105,10 @@ void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size) · 81 unmodified lines | |||
| 104 | 105 | ||
| 105 | if (size) { | 106 | if (size) { |
| 106 | count = FIELD_GET(AMDXDNA_CMD_COUNT, cmd->header); | 107 | count = FIELD_GET(AMDXDNA_CMD_COUNT, cmd->header); |
| 107 | if (unlikely(count <= num_masks)) { | 108 | if (unlikely(count <= num_masks || |
| 109 | count * sizeof(u32) + | ||
| 110 | offsetof(struct amdxdna_cmd, data[0]) > | ||
| 111 | abo->mem.size)) { | ||
| 108 | *size = 0; | 112 | *size = 0; |
| 109 | return NULL; | 113 | return NULL; |
| 110 | } | 114 | } |
| @@ -132,6 +136,33 @@ u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo) · 21 unmodified lines | |||
| 132 | return INVALID_CU_IDX; | 136 | return INVALID_CU_IDX; |
| 133 | } | 137 | } |
| 134 | 138 | ||
| 139 | int amdxdna_cmd_set_error(struct amdxdna_gem_obj *abo, | ||
| 140 | struct amdxdna_sched_job *job, u32 cmd_idx, | ||
| 141 | enum ert_cmd_state error_state) | ||
| 142 | { | ||
| 143 | struct amdxdna_client *client = job->hwctx->client; | ||
| 144 | struct amdxdna_cmd *cmd = abo->mem.kva; | ||
| 145 | struct amdxdna_cmd_chain *cc = NULL; | ||
| 146 | |||
| 147 | cmd->header &= ~AMDXDNA_CMD_STATE; | ||
| 148 | cmd->header |= FIELD_PREP(AMDXDNA_CMD_STATE, error_state); | ||
| 149 | |||
| 150 | if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN) { | ||
| 151 | cc = amdxdna_cmd_get_payload(abo, NULL); | ||
| 152 | cc->error_index = (cmd_idx < cc->command_count) ? cmd_idx : 0; | ||
| 153 | abo = amdxdna_gem_get_obj(client, cc->data[0], AMDXDNA_BO_CMD); | ||
| 154 | if (!abo) | ||
| 155 | return -EINVAL; | ||
| 156 | cmd = abo->mem.kva; | ||
| 157 | } | ||
| 158 | |||
| 159 | memset(cmd->data, 0xff, abo->mem.size - sizeof(*cmd)); | ||
| 160 | if (cc) | ||
| 161 | amdxdna_gem_put_obj(abo); | ||
| 162 | |||
| 163 | return 0; | ||
| 164 | } | ||
| 165 | |||
| 135 | /* | 166 | /* |
| 136 | * This should be called in close() and remove(). DO NOT call in other syscalls. | 167 | * This should be called in close() and remove(). DO NOT call in other syscalls. |
| 137 | * This guarantee that when hwctx and resources will be released, if user | 168 | * This guarantee that when hwctx and resources will be released, if user |
| @@ -266,9 +297,9 @@ int amdxdna_drm_config_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 128 unmodified lines | |||
| 266 | struct amdxdna_drm_config_hwctx *args = data; | 297 | struct amdxdna_drm_config_hwctx *args = data; |
| 267 | struct amdxdna_dev *xdna = to_xdna_dev(dev); | 298 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 268 | struct amdxdna_hwctx *hwctx; | 299 | struct amdxdna_hwctx *hwctx; |
| 269 | int ret, idx; | ||
| 270 | u32 buf_size; | 300 | u32 buf_size; |
| 271 | void *buf; | 301 | void *buf; |
| 302 | int ret; | ||
| 272 | u64 val; | 303 | u64 val; |
| 273 | 304 | ||
| 274 | if (XDNA_MBZ_DBG(xdna, &args->pad, sizeof(args->pad))) | 305 | if (XDNA_MBZ_DBG(xdna, &args->pad, sizeof(args->pad))) |
| @@ -310,20 +341,17 @@ int amdxdna_drm_config_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 35 unmodified lines | |||
| 310 | return -EINVAL; | 341 | return -EINVAL; |
| 311 | } | 342 | } |
| 312 | 343 | ||
| 313 | mutex_lock(&xdna->dev_lock); | 344 | guard(mutex)(&xdna->dev_lock); |
| 314 | idx = srcu_read_lock(&client->hwctx_srcu); | ||
| 315 | hwctx = xa_load(&client->hwctx_xa, args->handle); | 345 | hwctx = xa_load(&client->hwctx_xa, args->handle); |
| 316 | if (!hwctx) { | 346 | if (!hwctx) { |
| 317 | XDNA_DBG(xdna, "PID %d failed to get hwctx %d", client->pid, args->handle); | 347 | XDNA_DBG(xdna, "PID %d failed to get hwctx %d", client->pid, args->handle); |
| 318 | ret = -EINVAL; | 348 | ret = -EINVAL; |
| 319 | goto unlock_srcu; | 349 | goto free_buf; |
| 320 | } | 350 | } |
| 321 | 351 | ||
| 322 | ret = xdna->dev_info->ops->hwctx_config(hwctx, args->param_type, val, buf, buf_size); | 352 | ret = xdna->dev_info->ops->hwctx_config(hwctx, args->param_type, val, buf, buf_size); |
| 323 | 353 | ||
| 324 | unlock_srcu: | 354 | free_buf: |
| 325 | srcu_read_unlock(&client->hwctx_srcu, idx); | ||
| 326 | mutex_unlock(&xdna->dev_lock); | ||
| 327 | kfree(buf); | 355 | kfree(buf); |
| 328 | return ret; | 356 | return ret; |
| 329 | } | 357 | } |
| @@ -334,7 +362,7 @@ int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl) · 4 unmodified lines | |||
| 334 | struct amdxdna_hwctx *hwctx; | 362 | struct amdxdna_hwctx *hwctx; |
| 335 | struct amdxdna_gem_obj *abo; | 363 | struct amdxdna_gem_obj *abo; |
| 336 | struct drm_gem_object *gobj; | 364 | struct drm_gem_object *gobj; |
| 337 | int ret, idx; | 365 | int ret; |
| 338 | 366 | ||
| 339 | if (!xdna->dev_info->ops->hwctx_sync_debug_bo) | 367 | if (!xdna->dev_info->ops->hwctx_sync_debug_bo) |
| 340 | return -EOPNOTSUPP; | 368 | return -EOPNOTSUPP; |
| @@ -345,17 +373,15 @@ int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl) · 4 unmodified lines | |||
| 345 | 373 | ||
| 346 | abo = to_xdna_obj(gobj); | 374 | abo = to_xdna_obj(gobj); |
| 347 | guard(mutex)(&xdna->dev_lock); | 375 | guard(mutex)(&xdna->dev_lock); |
| 348 | idx = srcu_read_lock(&client->hwctx_srcu); | ||
| 349 | hwctx = xa_load(&client->hwctx_xa, abo->assigned_hwctx); | 376 | hwctx = xa_load(&client->hwctx_xa, abo->assigned_hwctx); |
| 350 | if (!hwctx) { | 377 | if (!hwctx) { |
| 351 | ret = -EINVAL; | 378 | ret = -EINVAL; |
| 352 | goto unlock_srcu; | 379 | goto put_obj; |
| 353 | } | 380 | } |
| 354 | 381 | ||
| 355 | ret = xdna->dev_info->ops->hwctx_sync_debug_bo(hwctx, debug_bo_hdl); | 382 | ret = xdna->dev_info->ops->hwctx_sync_debug_bo(hwctx, debug_bo_hdl); |
| 356 | 383 | ||
| 357 | unlock_srcu: | 384 | put_obj: |
| 358 | srcu_read_unlock(&client->hwctx_srcu, idx); | ||
| 359 | drm_gem_object_put(gobj); | 385 | drm_gem_object_put(gobj); |
| 360 | return ret; | 386 | return ret; |
| 361 | } | 387 | } |
| @@ -420,6 +446,7 @@ put_shmem_bo: · 58 unmodified lines | |||
| 420 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job) | 446 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job) |
| 421 | { | 447 | { |
| 422 | trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release"); | 448 | trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release"); |
| 449 | amdxdna_pm_suspend_put(job->hwctx->client->xdna); | ||
| 423 | amdxdna_arg_bos_put(job); | 450 | amdxdna_arg_bos_put(job); |
| 424 | amdxdna_gem_put_obj(job->cmd_bo); | 451 | amdxdna_gem_put_obj(job->cmd_bo); |
| 425 | dma_fence_put(job->fence); | 452 | dma_fence_put(job->fence); |
| @@ -457,6 +484,12 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, · 31 unmodified lines | |||
| 457 | goto cmd_put; | 484 | goto cmd_put; |
| 458 | } | 485 | } |
| 459 | 486 | ||
| 487 | ret = amdxdna_pm_resume_get(xdna); | ||
| 488 | if (ret) { | ||
| 489 | XDNA_ERR(xdna, "Resume failed, ret %d", ret); | ||
| 490 | goto put_bos; | ||
| 491 | } | ||
| 492 | |||
| 460 | idx = srcu_read_lock(&client->hwctx_srcu); | 493 | idx = srcu_read_lock(&client->hwctx_srcu); |
| 461 | hwctx = xa_load(&client->hwctx_xa, hwctx_hdl); | 494 | hwctx = xa_load(&client->hwctx_xa, hwctx_hdl); |
| 462 | if (!hwctx) { | 495 | if (!hwctx) { |
| @@ -497,6 +530,8 @@ put_fence: · 34 unmodified lines | |||
| 497 | dma_fence_put(job->fence); | 530 | dma_fence_put(job->fence); |
| 498 | unlock_srcu: | 531 | unlock_srcu: |
| 499 | srcu_read_unlock(&client->hwctx_srcu, idx); | 532 | srcu_read_unlock(&client->hwctx_srcu, idx); |
| 533 | amdxdna_pm_suspend_put(xdna); | ||
| 534 | put_bos: | ||
| 500 | amdxdna_arg_bos_put(job); | 535 | amdxdna_arg_bos_put(job); |
| 501 | cmd_put: | 536 | cmd_put: |
| 502 | amdxdna_gem_put_obj(job->cmd_bo); | 537 | amdxdna_gem_put_obj(job->cmd_bo); |
| @@ -167,6 +167,9 @@ amdxdna_cmd_get_state(struct amdxdna_gem_obj *abo) · 166 unmodified lines | |||
| 167 | 167 | ||
| 168 | void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size); | 168 | void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size); |
| 169 | u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo); | 169 | u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo); |
| 170 | int amdxdna_cmd_set_error(struct amdxdna_gem_obj *abo, | ||
| 171 | struct amdxdna_sched_job *job, u32 cmd_idx, | ||
| 172 | enum ert_cmd_state error_state); | ||
| 170 | 173 | ||
| 171 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job); | 174 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job); |
| 172 | void amdxdna_hwctx_remove_all(struct amdxdna_client *client); | 175 | void amdxdna_hwctx_remove_all(struct amdxdna_client *client); |
| @@ -21,8 +21,6 @@ · 20 unmodified lines | |||
| 21 | #include "amdxdna_pci_drv.h" | 21 | #include "amdxdna_pci_drv.h" |
| 22 | #include "amdxdna_ubuf.h" | 22 | #include "amdxdna_ubuf.h" |
| 23 | 23 | ||
| 24 | #define XDNA_MAX_CMD_BO_SIZE SZ_32K | ||
| 25 | |||
| 26 | MODULE_IMPORT_NS("DMA_BUF"); | 24 | MODULE_IMPORT_NS("DMA_BUF"); |
| 27 | 25 | ||
| 28 | static int | 26 | static int |
| @@ -745,12 +743,6 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 716 unmodified lines | |||
| 745 | { | 743 | { |
| 746 | struct amdxdna_dev *xdna = to_xdna_dev(dev); | 744 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 747 | struct amdxdna_gem_obj *abo; | 745 | struct amdxdna_gem_obj *abo; |
| 748 | int ret; | ||
| 749 | |||
| 750 | if (args->size > XDNA_MAX_CMD_BO_SIZE) { | ||
| 751 | XDNA_ERR(xdna, "Command bo size 0x%llx too large", args->size); | ||
| 752 | return ERR_PTR(-EINVAL); | ||
| 753 | } | ||
| 754 | 746 | ||
| 755 | if (args->size < sizeof(struct amdxdna_cmd)) { | 747 | if (args->size < sizeof(struct amdxdna_cmd)) { |
| 756 | XDNA_DBG(xdna, "Command BO size 0x%llx too small", args->size); | 748 | XDNA_DBG(xdna, "Command BO size 0x%llx too small", args->size); |
| @@ -764,17 +756,7 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 7 unmodified lines | |||
| 764 | abo->type = AMDXDNA_BO_CMD; | 756 | abo->type = AMDXDNA_BO_CMD; |
| 765 | abo->client = filp->driver_priv; | 757 | abo->client = filp->driver_priv; |
| 766 | 758 | ||
| 767 | ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva); | ||
| 768 | if (ret) { | ||
| 769 | XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret); | ||
| 770 | goto release_obj; | ||
| 771 | } | ||
| 772 | |||
| 773 | return abo; | 759 | return abo; |
| 774 | |||
| 775 | release_obj: | ||
| 776 | drm_gem_object_put(to_gobj(abo)); | ||
| 777 | return ERR_PTR(ret); | ||
| 778 | } | 760 | } |
| 779 | 761 | ||
| 780 | int amdxdna_drm_create_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) | 762 | int amdxdna_drm_create_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) |
| @@ -871,6 +853,7 @@ struct amdxdna_gem_obj *amdxdna_gem_get_obj(struct amdxdna_client *client, · 90 unmodified lines | |||
| 871 | struct amdxdna_dev *xdna = client->xdna; | 853 | struct amdxdna_dev *xdna = client->xdna; |
| 872 | struct amdxdna_gem_obj *abo; | 854 | struct amdxdna_gem_obj *abo; |
| 873 | struct drm_gem_object *gobj; | 855 | struct drm_gem_object *gobj; |
| 856 | int ret; | ||
| 874 | 857 | ||
| 875 | gobj = drm_gem_object_lookup(client->filp, bo_hdl); | 858 | gobj = drm_gem_object_lookup(client->filp, bo_hdl); |
| 876 | if (!gobj) { | 859 | if (!gobj) { |
| @@ -879,9 +862,26 @@ struct amdxdna_gem_obj *amdxdna_gem_get_obj(struct amdxdna_client *client, · 2 unmodified lines | |||
| 879 | } | 862 | } |
| 880 | 863 | ||
| 881 | abo = to_xdna_obj(gobj); | 864 | abo = to_xdna_obj(gobj); |
| 882 | if (bo_type == AMDXDNA_BO_INVALID || abo->type == bo_type) | 865 | if (bo_type != AMDXDNA_BO_INVALID && abo->type != bo_type) |
| 866 | goto put_obj; | ||
| 867 | |||
| 868 | if (bo_type != AMDXDNA_BO_CMD || abo->mem.kva) | ||
| 883 | return abo; | 869 | return abo; |
| 884 | 870 | ||
| 871 | if (abo->mem.size > SZ_32K) { | ||
| 872 | XDNA_ERR(xdna, "Cmd bo is too big %ld", abo->mem.size); | ||
| 873 | goto put_obj; | ||
| 874 | } | ||
| 875 | |||
| 876 | ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva); | ||
| 877 | if (ret) { | ||
| 878 | XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret); | ||
| 879 | goto put_obj; | ||
| 880 | } | ||
| 881 | |||
| 882 | return abo; | ||
| 883 | |||
| 884 | put_obj: | ||
| 885 | drm_gem_object_put(gobj); | 885 | drm_gem_object_put(gobj); |
| 886 | return NULL; | 886 | return NULL; |
| 887 | } | 887 | } |
| @@ -460,26 +460,49 @@ msg_id_failed: · 459 unmodified lines | |||
| 460 | return ret; | 460 | return ret; |
| 461 | } | 461 | } |
| 462 | 462 | ||
| 463 | struct mailbox_channel * | 463 | struct mailbox_channel *xdna_mailbox_alloc_channel(struct mailbox *mb) |
| 464 | xdna_mailbox_create_channel(struct mailbox *mb, | ||
| 465 | const struct xdna_mailbox_chann_res *x2i, | ||
| 466 | const struct xdna_mailbox_chann_res *i2x, | ||
| 467 | u32 iohub_int_addr, | ||
| 468 | int mb_irq) | ||
| 469 | { | 464 | { |
| 470 | struct mailbox_channel *mb_chann; | 465 | struct mailbox_channel *mb_chann; |
| 471 | int ret; | ||
| 472 | |||
| 473 | if (!is_power_of_2(x2i->rb_size) || !is_power_of_2(i2x->rb_size)) { | ||
| 474 | pr_err("Ring buf size must be power of 2"); | ||
| 475 | return NULL; | ||
| 476 | } | ||
| 477 | 466 | ||
| 478 | mb_chann = kzalloc_obj(*mb_chann); | 467 | mb_chann = kzalloc_obj(*mb_chann); |
| 479 | if (!mb_chann) | 468 | if (!mb_chann) |
| 480 | return NULL; | 469 | return NULL; |
| 481 | 470 | ||
| 471 | INIT_WORK(&mb_chann->rx_work, mailbox_rx_worker); | ||
| 472 | mb_chann->work_q = create_singlethread_workqueue(MAILBOX_NAME); | ||
| 473 | if (!mb_chann->work_q) { | ||
| 474 | MB_ERR(mb_chann, "Create workqueue failed"); | ||
| 475 | goto free_chann; | ||
| 476 | } | ||
| 482 | mb_chann->mb = mb; | 477 | mb_chann->mb = mb; |
| 478 | |||
| 479 | return mb_chann; | ||
| 480 | |||
| 481 | free_chann: | ||
| 482 | kfree(mb_chann); | ||
| 483 | return NULL; | ||
| 484 | } | ||
| 485 | |||
| 486 | void xdna_mailbox_free_channel(struct mailbox_channel *mb_chann) | ||
| 487 | { | ||
| 488 | destroy_workqueue(mb_chann->work_q); | ||
| 489 | kfree(mb_chann); | ||
| 490 | } | ||
| 491 | |||
| 492 | int | ||
| 493 | xdna_mailbox_start_channel(struct mailbox_channel *mb_chann, | ||
| 494 | const struct xdna_mailbox_chann_res *x2i, | ||
| 495 | const struct xdna_mailbox_chann_res *i2x, | ||
| 496 | u32 iohub_int_addr, | ||
| 497 | int mb_irq) | ||
| 498 | { | ||
| 499 | int ret; | ||
| 500 | |||
| 501 | if (!is_power_of_2(x2i->rb_size) || !is_power_of_2(i2x->rb_size)) { | ||
| 502 | pr_err("Ring buf size must be power of 2"); | ||
| 503 | return -EINVAL; | ||
| 504 | } | ||
| 505 | |||
| 483 | mb_chann->msix_irq = mb_irq; | 506 | mb_chann->msix_irq = mb_irq; |
| 484 | mb_chann->iohub_int_addr = iohub_int_addr; | 507 | mb_chann->iohub_int_addr = iohub_int_addr; |
| 485 | memcpy(&mb_chann->res[CHAN_RES_X2I], x2i, sizeof(*x2i)); | 508 | memcpy(&mb_chann->res[CHAN_RES_X2I], x2i, sizeof(*x2i)); |
| @@ -489,61 +512,37 @@ xdna_mailbox_create_channel(struct mailbox *mb, · 3 unmodified lines | |||
| 489 | mb_chann->x2i_tail = mailbox_get_tailptr(mb_chann, CHAN_RES_X2I); | 512 | mb_chann->x2i_tail = mailbox_get_tailptr(mb_chann, CHAN_RES_X2I); |
| 490 | mb_chann->i2x_head = mailbox_get_headptr(mb_chann, CHAN_RES_I2X); | 513 | mb_chann->i2x_head = mailbox_get_headptr(mb_chann, CHAN_RES_I2X); |
| 491 | 514 | ||
| 492 | INIT_WORK(&mb_chann->rx_work, mailbox_rx_worker); | ||
| 493 | mb_chann->work_q = create_singlethread_workqueue(MAILBOX_NAME); | ||
| 494 | if (!mb_chann->work_q) { | ||
| 495 | MB_ERR(mb_chann, "Create workqueue failed"); | ||
| 496 | goto free_and_out; | ||
| 497 | } | ||
| 498 | |||
| 499 | /* Everything look good. Time to enable irq handler */ | 515 | /* Everything look good. Time to enable irq handler */ |
| 500 | ret = request_irq(mb_irq, mailbox_irq_handler, 0, MAILBOX_NAME, mb_chann); | 516 | ret = request_irq(mb_irq, mailbox_irq_handler, 0, MAILBOX_NAME, mb_chann); |
| 501 | if (ret) { | 517 | if (ret) { |
| 502 | MB_ERR(mb_chann, "Failed to request irq %d ret %d", mb_irq, ret); | 518 | MB_ERR(mb_chann, "Failed to request irq %d ret %d", mb_irq, ret); |
| 503 | goto destroy_wq; | 519 | return ret; |
| 504 | } | 520 | } |
| 505 | 521 | ||
| 506 | mb_chann->bad_state = false; | 522 | mb_chann->bad_state = false; |
| 507 | mailbox_reg_write(mb_chann, mb_chann->iohub_int_addr, 0); | 523 | mailbox_reg_write(mb_chann, mb_chann->iohub_int_addr, 0); |
| 508 | 524 | ||
| 509 | MB_DBG(mb_chann, "Mailbox channel created (irq: %d)", mb_chann->msix_irq); | 525 | MB_DBG(mb_chann, "Mailbox channel started (irq: %d)", mb_chann->msix_irq); |
| 510 | return mb_chann; | 526 | return 0; |
| 511 | |||
| 512 | destroy_wq: | ||
| 513 | destroy_workqueue(mb_chann->work_q); | ||
| 514 | free_and_out: | ||
| 515 | kfree(mb_chann); | ||
| 516 | return NULL; | ||
| 517 | } | 527 | } |
| 518 | 528 | ||
| 519 | int xdna_mailbox_destroy_channel(struct mailbox_channel *mb_chann) | 529 | void xdna_mailbox_stop_channel(struct mailbox_channel *mb_chann) |
| 520 | { | 530 | { |
| 521 | struct mailbox_msg *mb_msg; | 531 | struct mailbox_msg *mb_msg; |
| 522 | unsigned long msg_id; | 532 | unsigned long msg_id; |
| 523 | 533 | ||
| 524 | MB_DBG(mb_chann, "IRQ disabled and RX work cancelled"); | 534 | /* Disable an irq and wait. This might sleep. */ |
| 525 | free_irq(mb_chann->msix_irq, mb_chann); | 535 | free_irq(mb_chann->msix_irq, mb_chann); |
| 526 | destroy_workqueue(mb_chann->work_q); | ||
| 527 | /* We can clean up and release resources */ | ||
| 528 | 536 | ||
| 537 | /* Cancel RX work and wait for it to finish */ | ||
| 538 | drain_workqueue(mb_chann->work_q); | ||
| 539 | |||
| 540 | /* We can clean up and release resources */ | ||
| 529 | xa_for_each(&mb_chann->chan_xa, msg_id, mb_msg) | 541 | xa_for_each(&mb_chann->chan_xa, msg_id, mb_msg) |
| 530 | mailbox_release_msg(mb_chann, mb_msg); | 542 | mailbox_release_msg(mb_chann, mb_msg); |
| 531 | |||
| 532 | xa_destroy(&mb_chann->chan_xa); | 543 | xa_destroy(&mb_chann->chan_xa); |
| 533 | 544 | ||
| 534 | MB_DBG(mb_chann, "Mailbox channel destroyed, irq: %d", mb_chann->msix_irq); | 545 | MB_DBG(mb_chann, "Mailbox channel stopped, irq: %d", mb_chann->msix_irq); |
| 535 | kfree(mb_chann); | ||
| 536 | return 0; | ||
| 537 | } | ||
| 538 | |||
| 539 | void xdna_mailbox_stop_channel(struct mailbox_channel *mb_chann) | ||
| 540 | { | ||
| 541 | /* Disable an irq and wait. This might sleep. */ | ||
| 542 | disable_irq(mb_chann->msix_irq); | ||
| 543 | |||
| 544 | /* Cancel RX work and wait for it to finish */ | ||
| 545 | cancel_work_sync(&mb_chann->rx_work); | ||
| 546 | MB_DBG(mb_chann, "IRQ disabled and RX work cancelled"); | ||
| 547 | } | 546 | } |
| 548 | 547 | ||
| 549 | struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, | 548 | struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, |
| @@ -74,9 +74,16 @@ struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, · 73 unmodified lines | |||
| 74 | const struct xdna_mailbox_res *res); | 74 | const struct xdna_mailbox_res *res); |
| 75 | 75 | ||
| 76 | /* | 76 | /* |
| 77 | * xdna_mailbox_create_channel() -- Create a mailbox channel instance | 77 | * xdna_mailbox_alloc_channel() -- alloc a mailbox channel |
| 78 | * | 78 | * |
| 79 | * @mailbox: the handle return from xdna_mailbox_create() | 79 | * @mb: mailbox handle |
| 80 | */ | ||
| 81 | struct mailbox_channel *xdna_mailbox_alloc_channel(struct mailbox *mb); | ||
| 82 | |||
| 83 | /* | ||
| 84 | * xdna_mailbox_start_channel() -- start a mailbox channel instance | ||
| 85 | * | ||
| 86 | * @mb_chann: the handle return from xdna_mailbox_alloc_channel() | ||
| 80 | * @x2i: host to firmware mailbox resources | 87 | * @x2i: host to firmware mailbox resources |
| 81 | * @i2x: firmware to host mailbox resources | 88 | * @i2x: firmware to host mailbox resources |
| 82 | * @xdna_mailbox_intr_reg: register addr of MSI-X interrupt | 89 | * @xdna_mailbox_intr_reg: register addr of MSI-X interrupt |
| @@ -84,28 +91,24 @@ struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, · 1 unmodified line | |||
| 84 | * | 91 | * |
| 85 | * Return: If success, return a handle of mailbox channel. Otherwise, return NULL. | 92 | * Return: If success, return a handle of mailbox channel. Otherwise, return NULL. |
| 86 | */ | 93 | */ |
| 87 | struct mailbox_channel * | 94 | int |
| 88 | xdna_mailbox_create_channel(struct mailbox *mailbox, | 95 | xdna_mailbox_start_channel(struct mailbox_channel *mb_chann, |
| 89 | const struct xdna_mailbox_chann_res *x2i, | 96 | const struct xdna_mailbox_chann_res *x2i, |
| 90 | const struct xdna_mailbox_chann_res *i2x, | 97 | const struct xdna_mailbox_chann_res *i2x, |
| 91 | u32 xdna_mailbox_intr_reg, | 98 | u32 xdna_mailbox_intr_reg, |
| 92 | int mb_irq); | 99 | int mb_irq); |
| 93 | 100 | ||
| 94 | /* | 101 | /* |
| 95 | * xdna_mailbox_destroy_channel() -- destroy mailbox channel | 102 | * xdna_mailbox_free_channel() -- free mailbox channel |
| 96 | * | 103 | * |
| 97 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() | 104 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() |
| 98 | * | ||
| 99 | * Return: if success, return 0. otherwise return error code | ||
| 100 | */ | 105 | */ |
| 101 | int xdna_mailbox_destroy_channel(struct mailbox_channel *mailbox_chann); | 106 | void xdna_mailbox_free_channel(struct mailbox_channel *mailbox_chann); |
| 102 | 107 | ||
| 103 | /* | 108 | /* |
| 104 | * xdna_mailbox_stop_channel() -- stop mailbox channel | 109 | * xdna_mailbox_stop_channel() -- stop mailbox channel |
| 105 | * | 110 | * |
| 106 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() | 111 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() |
| 107 | * | ||
| 108 | * Return: if success, return 0. otherwise return error code | ||
| 109 | */ | 112 | */ |
| 110 | void xdna_mailbox_stop_channel(struct mailbox_channel *mailbox_chann); | 113 | void xdna_mailbox_stop_channel(struct mailbox_channel *mailbox_chann); |
| 111 | 114 | ||
| @@ -23,6 +23,9 @@ MODULE_FIRMWARE("amdnpu/1502_00/npu.sbin"); · 22 unmodified lines | |||
| 23 | MODULE_FIRMWARE("amdnpu/17f0_10/npu.sbin"); | 23 | MODULE_FIRMWARE("amdnpu/17f0_10/npu.sbin"); |
| 24 | MODULE_FIRMWARE("amdnpu/17f0_11/npu.sbin"); | 24 | MODULE_FIRMWARE("amdnpu/17f0_11/npu.sbin"); |
| 25 | MODULE_FIRMWARE("amdnpu/17f0_20/npu.sbin"); | 25 | MODULE_FIRMWARE("amdnpu/17f0_20/npu.sbin"); |
| 26 | MODULE_FIRMWARE("amdnpu/1502_00/npu_7.sbin"); | ||
| 27 | MODULE_FIRMWARE("amdnpu/17f0_10/npu_7.sbin"); | ||
| 28 | MODULE_FIRMWARE("amdnpu/17f0_11/npu_7.sbin"); | ||
| 26 | 29 | ||
| 27 | /* | 30 | /* |
| 28 | * 0.0: Initial version | 31 | * 0.0: Initial version |
| @@ -16,6 +16,7 @@ int amdxdna_pm_suspend(struct device *dev) · 15 unmodified lines | |||
| 16 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); | 16 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); |
| 17 | int ret = -EOPNOTSUPP; | 17 | int ret = -EOPNOTSUPP; |
| 18 | 18 | ||
| 19 | guard(mutex)(&xdna->dev_lock); | ||
| 19 | if (xdna->dev_info->ops->suspend) | 20 | if (xdna->dev_info->ops->suspend) |
| 20 | ret = xdna->dev_info->ops->suspend(xdna); | 21 | ret = xdna->dev_info->ops->suspend(xdna); |
| 21 | 22 | ||
| @@ -28,6 +29,7 @@ int amdxdna_pm_resume(struct device *dev) · 6 unmodified lines | |||
| 28 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); | 29 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); |
| 29 | int ret = -EOPNOTSUPP; | 30 | int ret = -EOPNOTSUPP; |
| 30 | 31 | ||
| 32 | guard(mutex)(&xdna->dev_lock); | ||
| 31 | if (xdna->dev_info->ops->resume) | 33 | if (xdna->dev_info->ops->resume) |
| 32 | ret = xdna->dev_info->ops->resume(xdna); | 34 | ret = xdna->dev_info->ops->resume(xdna); |
| 33 | 35 | ||
| @@ -15,4 +15,15 @@ void amdxdna_pm_suspend_put(struct amdxdna_dev *xdna); · 14 unmodified lines | |||
| 15 | void amdxdna_pm_init(struct amdxdna_dev *xdna); | 15 | void amdxdna_pm_init(struct amdxdna_dev *xdna); |
| 16 | void amdxdna_pm_fini(struct amdxdna_dev *xdna); | 16 | void amdxdna_pm_fini(struct amdxdna_dev *xdna); |
| 17 | 17 | ||
| 18 | static inline int amdxdna_pm_resume_get_locked(struct amdxdna_dev *xdna) | ||
| 19 | { | ||
| 20 | int ret; | ||
| 21 | |||
| 22 | mutex_unlock(&xdna->dev_lock); | ||
| 23 | ret = amdxdna_pm_resume_get(xdna); | ||
| 24 | mutex_lock(&xdna->dev_lock); | ||
| 25 | |||
| 26 | return ret; | ||
| 27 | } | ||
| 28 | |||
| 18 | #endif /* _AMDXDNA_PM_H_ */ | 29 | #endif /* _AMDXDNA_PM_H_ */ |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | |||
| 7 | #include <drm/drm_device.h> | 7 | #include <drm/drm_device.h> |
| 8 | #include <drm/drm_print.h> | 8 | #include <drm/drm_print.h> |
| 9 | #include <linux/dma-buf.h> | 9 | #include <linux/dma-buf.h> |
| 10 | #include <linux/overflow.h> | ||
| 10 | #include <linux/pagemap.h> | 11 | #include <linux/pagemap.h> |
| 11 | #include <linux/vmalloc.h> | 12 | #include <linux/vmalloc.h> |
| 12 | 13 | ||
| @@ -176,7 +177,10 @@ struct dma_buf *amdxdna_get_ubuf(struct drm_device *dev, · 163 unmodified lines | |||
| 176 | goto free_ent; | 177 | goto free_ent; |
| 177 | } | 178 | } |
| 178 | 179 | ||
| 179 | exp_info.size += va_ent[i].len; | 180 | if (check_add_overflow(exp_info.size, va_ent[i].len, &exp_info.size)) { |
| 181 | ret = -EINVAL; | ||
| 182 | goto free_ent; | ||
| 183 | } | ||
| 180 | } | 184 | } |
| 181 | 185 | ||
| 182 | ubuf->nr_pages = exp_info.size >> PAGE_SHIFT; | 186 | ubuf->nr_pages = exp_info.size >> PAGE_SHIFT; |
| @@ -67,12 +67,12 @@ const struct dpm_clk_freq npu1_dpm_clk_table[] = { · 66 unmodified lines | |||
| 67 | 67 | ||
| 68 | static const struct aie2_fw_feature_tbl npu1_fw_feature_table[] = { | 68 | static const struct aie2_fw_feature_tbl npu1_fw_feature_table[] = { |
| 69 | { .major = 5, .min_minor = 7 }, | 69 | { .major = 5, .min_minor = 7 }, |
| 70 | { .features = BIT_U64(AIE2_NPU_COMMAND), .min_minor = 8 }, | 70 | { .features = BIT_U64(AIE2_NPU_COMMAND), .major = 5, .min_minor = 8 }, |
| 71 | { 0 } | 71 | { 0 } |
| 72 | }; | 72 | }; |
| 73 | 73 | ||
| 74 | static const struct amdxdna_dev_priv npu1_dev_priv = { | 74 | static const struct amdxdna_dev_priv npu1_dev_priv = { |
| 75 | .fw_path = "amdnpu/1502_00/npu.sbin", | 75 | .fw_path = "amdnpu/1502_00/", |
| 76 | .rt_config = npu1_default_rt_cfg, | 76 | .rt_config = npu1_default_rt_cfg, |
| 77 | .dpm_clk_tbl = npu1_dpm_clk_table, | 77 | .dpm_clk_tbl = npu1_dpm_clk_table, |
| 78 | .fw_feature_tbl = npu1_fw_feature_table, | 78 | .fw_feature_tbl = npu1_fw_feature_table, |
| @@ -98,7 +98,7 @@ const struct aie2_fw_feature_tbl npu4_fw_feature_table[] = { · 97 unmodified lines | |||
| 98 | }; | 98 | }; |
| 99 | 99 | ||
| 100 | static const struct amdxdna_dev_priv npu4_dev_priv = { | 100 | static const struct amdxdna_dev_priv npu4_dev_priv = { |
| 101 | .fw_path = "amdnpu/17f0_10/npu.sbin", | 101 | .fw_path = "amdnpu/17f0_10/", |
| 102 | .rt_config = npu4_default_rt_cfg, | 102 | .rt_config = npu4_default_rt_cfg, |
| 103 | .dpm_clk_tbl = npu4_dpm_clk_table, | 103 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 104 | .fw_feature_tbl = npu4_fw_feature_table, | 104 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -63,7 +63,7 @@ · 62 unmodified lines | |||
| 63 | #define NPU5_SRAM_BAR_BASE MMNPU_APERTURE1_BASE | 63 | #define NPU5_SRAM_BAR_BASE MMNPU_APERTURE1_BASE |
| 64 | 64 | ||
| 65 | static const struct amdxdna_dev_priv npu5_dev_priv = { | 65 | static const struct amdxdna_dev_priv npu5_dev_priv = { |
| 66 | .fw_path = "amdnpu/17f0_11/npu.sbin", | 66 | .fw_path = "amdnpu/17f0_11/", |
| 67 | .rt_config = npu4_default_rt_cfg, | 67 | .rt_config = npu4_default_rt_cfg, |
| 68 | .dpm_clk_tbl = npu4_dpm_clk_table, | 68 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 69 | .fw_feature_tbl = npu4_fw_feature_table, | 69 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -63,7 +63,7 @@ · 62 unmodified lines | |||
| 63 | #define NPU6_SRAM_BAR_BASE MMNPU_APERTURE1_BASE | 63 | #define NPU6_SRAM_BAR_BASE MMNPU_APERTURE1_BASE |
| 64 | 64 | ||
| 65 | static const struct amdxdna_dev_priv npu6_dev_priv = { | 65 | static const struct amdxdna_dev_priv npu6_dev_priv = { |
| 66 | .fw_path = "amdnpu/17f0_10/npu.sbin", | 66 | .fw_path = "amdnpu/17f0_10/", |
| 67 | .rt_config = npu4_default_rt_cfg, | 67 | .rt_config = npu4_default_rt_cfg, |
| 68 | .dpm_clk_tbl = npu4_dpm_clk_table, | 68 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 69 | .fw_feature_tbl = npu4_fw_feature_table, | 69 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -154,7 +154,7 @@ static void cmd_state_init(struct cmd_state *st) · 153 unmodified lines | |||
| 154 | 154 | ||
| 155 | static u64 cmd_to_addr(u32 *cmd) | 155 | static u64 cmd_to_addr(u32 *cmd) |
| 156 | { | 156 | { |
| 157 | return ((u64)((cmd[0] & 0xff0000) << 16)) | cmd[1]; | 157 | return (((u64)cmd[0] & 0xff0000) << 16) | cmd[1]; |
| 158 | } | 158 | } |
| 159 | 159 | ||
| 160 | static u64 dma_length(struct ethosu_validated_cmdstream_info *info, | 160 | static u64 dma_length(struct ethosu_validated_cmdstream_info *info, |
| @@ -245,11 +245,14 @@ static int calc_sizes(struct drm_device *ddev, · 84 unmodified lines | |||
| 245 | ((st->ifm.stride_kernel >> 1) & 0x1) + 1; | 245 | ((st->ifm.stride_kernel >> 1) & 0x1) + 1; |
| 246 | u32 stride_x = ((st->ifm.stride_kernel >> 5) & 0x2) + | 246 | u32 stride_x = ((st->ifm.stride_kernel >> 5) & 0x2) + |
| 247 | (st->ifm.stride_kernel & 0x1) + 1; | 247 | (st->ifm.stride_kernel & 0x1) + 1; |
| 248 | u32 ifm_height = st->ofm.height[2] * stride_y + | 248 | s32 ifm_height = st->ofm.height[2] * stride_y + |
| 249 | st->ifm.height[2] - (st->ifm.pad_top + st->ifm.pad_bottom); | 249 | st->ifm.height[2] - (st->ifm.pad_top + st->ifm.pad_bottom); |
| 250 | u32 ifm_width = st->ofm.width * stride_x + | 250 | s32 ifm_width = st->ofm.width * stride_x + |
| 251 | st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right); | 251 | st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right); |
| 252 | 252 | ||
| 253 | if (ifm_height < 0 || ifm_width < 0) | ||
| 254 | return -EINVAL; | ||
| 255 | |||
| 253 | len = feat_matrix_length(info, &st->ifm, ifm_width, | 256 | len = feat_matrix_length(info, &st->ifm, ifm_width, |
| 254 | ifm_height, st->ifm.depth); | 257 | ifm_height, st->ifm.depth); |
| 255 | dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n", | 258 | dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n", |
| @@ -417,7 +420,10 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev, · 161 unmodified lines | |||
| 417 | return ret; | 420 | return ret; |
| 418 | break; | 421 | break; |
| 419 | case NPU_OP_ELEMENTWISE: | 422 | case NPU_OP_ELEMENTWISE: |
| 420 | use_ifm2 = !((st.ifm2.broadcast == 8) || (param == 5) || | 423 | use_scale = ethosu_is_u65(edev) ? |
| 424 | (st.ifm2.broadcast & 0x80) : | ||
| 425 | (st.ifm2.broadcast == 8); | ||
| 426 | use_ifm2 = !(use_scale || (param == 5) || | ||
| 421 | (param == 6) || (param == 7) || (param == 0x24)); | 427 | (param == 6) || (param == 7) || (param == 0x24)); |
| 422 | use_ifm = st.ifm.broadcast != 8; | 428 | use_ifm = st.ifm.broadcast != 8; |
| 423 | ret = calc_sizes_elemwise(ddev, info, cmd, &st, use_ifm, use_ifm2); | 429 | ret = calc_sizes_elemwise(ddev, info, cmd, &st, use_ifm, use_ifm2); |
| @@ -143,23 +143,29 @@ out: · 142 unmodified lines | |||
| 143 | return ret; | 143 | return ret; |
| 144 | } | 144 | } |
| 145 | 145 | ||
| 146 | static void ethosu_job_err_cleanup(struct ethosu_job *job) | ||
| 147 | { | ||
| 148 | unsigned int i; | ||
| 149 | |||
| 150 | for (i = 0; i < job->region_cnt; i++) | ||
| 151 | drm_gem_object_put(job->region_bo[i]); | ||
| 152 | |||
| 153 | drm_gem_object_put(job->cmd_bo); | ||
| 154 | |||
| 155 | kfree(job); | ||
| 156 | } | ||
| 157 | |||
| 146 | static void ethosu_job_cleanup(struct kref *ref) | 158 | static void ethosu_job_cleanup(struct kref *ref) |
| 147 | { | 159 | { |
| 148 | struct ethosu_job *job = container_of(ref, struct ethosu_job, | 160 | struct ethosu_job *job = container_of(ref, struct ethosu_job, |
| 149 | refcount); | 161 | refcount); |
| 150 | unsigned int i; | ||
| 151 | 162 | ||
| 152 | pm_runtime_put_autosuspend(job->dev->base.dev); | 163 | pm_runtime_put_autosuspend(job->dev->base.dev); |
| 153 | 164 | ||
| 154 | dma_fence_put(job->done_fence); | 165 | dma_fence_put(job->done_fence); |
| 155 | dma_fence_put(job->inference_done_fence); | 166 | dma_fence_put(job->inference_done_fence); |
| 156 | 167 | ||
| 157 | for (i = 0; i < job->region_cnt; i++) | 168 | ethosu_job_err_cleanup(job); |
| 158 | drm_gem_object_put(job->region_bo[i]); | ||
| 159 | |||
| 160 | drm_gem_object_put(job->cmd_bo); | ||
| 161 | |||
| 162 | kfree(job); | ||
| 163 | } | 169 | } |
| 164 | 170 | ||
| 165 | static void ethosu_job_put(struct ethosu_job *job) | 171 | static void ethosu_job_put(struct ethosu_job *job) |
| @@ -454,12 +460,16 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file · 288 unmodified lines | |||
| 454 | } | 460 | } |
| 455 | } | 461 | } |
| 456 | ret = ethosu_job_push(ejob); | 462 | ret = ethosu_job_push(ejob); |
| 463 | if (!ret) { | ||
| 464 | ethosu_job_put(ejob); | ||
| 465 | return 0; | ||
| 466 | } | ||
| 457 | 467 | ||
| 458 | out_cleanup_job: | 468 | out_cleanup_job: |
| 459 | if (ret) | 469 | if (ret) |
| 460 | drm_sched_job_cleanup(&ejob->base); | 470 | drm_sched_job_cleanup(&ejob->base); |
| 461 | out_put_job: | 471 | out_put_job: |
| 462 | ethosu_job_put(ejob); | 472 | ethosu_job_err_cleanup(ejob); |
| 463 | 473 | ||
| 464 | return ret; | 474 | return ret; |
| 465 | } | 475 | } |
| @@ -121,12 +121,6 @@ · 120 unmodified lines | |||
| 121 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY 0x0003006cu | 121 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY 0x0003006cu |
| 122 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY_STATUS_DLY_MASK GENMASK(7, 0) | 122 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY_STATUS_DLY_MASK GENMASK(7, 0) |
| 123 | 123 | ||
| 124 | #define VPU_40XX_HOST_SS_AON_RETENTION0 0x0003000cu | ||
| 125 | #define VPU_40XX_HOST_SS_AON_RETENTION1 0x00030010u | ||
| 126 | #define VPU_40XX_HOST_SS_AON_RETENTION2 0x00030014u | ||
| 127 | #define VPU_40XX_HOST_SS_AON_RETENTION3 0x00030018u | ||
| 128 | #define VPU_40XX_HOST_SS_AON_RETENTION4 0x0003001cu | ||
| 129 | |||
| 130 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN 0x00030200u | 124 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN 0x00030200u |
| 131 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_EN_MASK BIT_MASK(0) | 125 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_EN_MASK BIT_MASK(0) |
| 132 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_HW_PG_EN_MASK BIT_MASK(1) | 126 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_HW_PG_EN_MASK BIT_MASK(1) |
| @@ -931,7 +931,6 @@ static int soc_cpu_boot_40xx(struct ivpu_device *vdev) · 930 unmodified lines | |||
| 931 | 931 | ||
| 932 | static int soc_cpu_boot_60xx(struct ivpu_device *vdev) | 932 | static int soc_cpu_boot_60xx(struct ivpu_device *vdev) |
| 933 | { | 933 | { |
| 934 | REGV_WR64(VPU_40XX_HOST_SS_AON_RETENTION1, vdev->fw->mem_bp->vpu_addr); | ||
| 935 | soc_cpu_set_entry_point_40xx(vdev, vdev->fw->cold_boot_entry_point); | 934 | soc_cpu_set_entry_point_40xx(vdev, vdev->fw->cold_boot_entry_point); |
| 936 | 935 | ||
| 937 | return 0; | 936 | return 0; |
| @@ -9,6 +9,7 @@ config ARCH_SUPPORTS_ACPI · 8 unmodified lines | |||
| 9 | menuconfig ACPI | 9 | menuconfig ACPI |
| 10 | bool "ACPI (Advanced Configuration and Power Interface) Support" | 10 | bool "ACPI (Advanced Configuration and Power Interface) Support" |
| 11 | depends on ARCH_SUPPORTS_ACPI | 11 | depends on ARCH_SUPPORTS_ACPI |
| 12 | select AUXILIARY_BUS | ||
| 12 | select PNP | 13 | select PNP |
| 13 | select NLS | 14 | select NLS |
| 14 | select CRC32 | 15 | select CRC32 |
| @@ -135,7 +135,7 @@ struct platform_device *acpi_create_platform_device(struct acpi_device *adev, · 134 unmodified lines | |||
| 135 | } | 135 | } |
| 136 | } | 136 | } |
| 137 | 137 | ||
| 138 | if (adev->device_type == ACPI_BUS_TYPE_DEVICE && !adev->pnp.type.backlight) { | 138 | if (adev->device_type == ACPI_BUS_TYPE_DEVICE) { |
| 139 | LIST_HEAD(resource_list); | 139 | LIST_HEAD(resource_list); |
| 140 | 140 | ||
| 141 | count = acpi_dev_get_resources(adev, &resource_list, NULL, NULL); | 141 | count = acpi_dev_get_resources(adev, &resource_list, NULL, NULL); |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | |||
| 9 | 9 | ||
| 10 | #define pr_fmt(fmt) "ACPI: video: " fmt | 10 | #define pr_fmt(fmt) "ACPI: video: " fmt |
| 11 | 11 | ||
| 12 | #include <linux/auxiliary_bus.h> | ||
| 12 | #include <linux/kernel.h> | 13 | #include <linux/kernel.h> |
| 13 | #include <linux/module.h> | 14 | #include <linux/module.h> |
| 14 | #include <linux/init.h> | 15 | #include <linux/init.h> |
| @@ -21,7 +22,6 @@ · 6 unmodified lines | |||
| 21 | #include <linux/sort.h> | 22 | #include <linux/sort.h> |
| 22 | #include <linux/pci.h> | 23 | #include <linux/pci.h> |
| 23 | #include <linux/pci_ids.h> | 24 | #include <linux/pci_ids.h> |
| 24 | #include <linux/platform_device.h> | ||
| 25 | #include <linux/slab.h> | 25 | #include <linux/slab.h> |
| 26 | #include <linux/dmi.h> | 26 | #include <linux/dmi.h> |
| 27 | #include <linux/suspend.h> | 27 | #include <linux/suspend.h> |
| @@ -77,8 +77,9 @@ static int register_count; · 49 unmodified lines | |||
| 77 | static DEFINE_MUTEX(register_count_mutex); | 77 | static DEFINE_MUTEX(register_count_mutex); |
| 78 | static DEFINE_MUTEX(video_list_lock); | 78 | static DEFINE_MUTEX(video_list_lock); |
| 79 | static LIST_HEAD(video_bus_head); | 79 | static LIST_HEAD(video_bus_head); |
| 80 | static int acpi_video_bus_probe(struct platform_device *pdev); | 80 | static int acpi_video_bus_probe(struct auxiliary_device *aux_dev, |
| 81 | static void acpi_video_bus_remove(struct platform_device *pdev); | 81 | const struct auxiliary_device_id *id); |
| 82 | static void acpi_video_bus_remove(struct auxiliary_device *aux); | ||
| 82 | static void acpi_video_bus_notify(acpi_handle handle, u32 event, void *data); | 83 | static void acpi_video_bus_notify(acpi_handle handle, u32 event, void *data); |
| 83 | 84 | ||
| 84 | /* | 85 | /* |
| @@ -93,19 +94,16 @@ enum acpi_video_level_idx { · 8 unmodified lines | |||
| 93 | ACPI_VIDEO_FIRST_LEVEL, /* actual supported levels begin here */ | 94 | ACPI_VIDEO_FIRST_LEVEL, /* actual supported levels begin here */ |
| 94 | }; | 95 | }; |
| 95 | 96 | ||
| 96 | static const struct acpi_device_id video_device_ids[] = { | 97 | static const struct auxiliary_device_id video_bus_auxiliary_id_table[] = { |
| 97 | {ACPI_VIDEO_HID, 0}, | 98 | { .name = "acpi.video_bus" }, |
| 98 | {"", 0}, | 99 | {}, |
| 99 | }; | 100 | }; |
| 100 | MODULE_DEVICE_TABLE(acpi, video_device_ids); | 101 | MODULE_DEVICE_TABLE(auxiliary, video_bus_auxiliary_id_table); |
| 101 | 102 | ||
| 102 | static struct platform_driver acpi_video_bus = { | 103 | static struct auxiliary_driver acpi_video_bus = { |
| 103 | .probe = acpi_video_bus_probe, | 104 | .probe = acpi_video_bus_probe, |
| 104 | .remove = acpi_video_bus_remove, | 105 | .remove = acpi_video_bus_remove, |
| 105 | .driver = { | 106 | .id_table = video_bus_auxiliary_id_table, |
| 106 | .name = "acpi-video", | ||
| 107 | .acpi_match_table = video_device_ids, | ||
| 108 | }, | ||
| 109 | }; | 107 | }; |
| 110 | 108 | ||
| 111 | struct acpi_video_bus_flags { | 109 | struct acpi_video_bus_flags { |
| @@ -1885,7 +1883,7 @@ static void acpi_video_dev_add_notify_handler(struct acpi_video_device *device) · 1773 unmodified lines | |||
| 1885 | } | 1883 | } |
| 1886 | 1884 | ||
| 1887 | static int acpi_video_bus_add_notify_handler(struct acpi_video_bus *video, | 1885 | static int acpi_video_bus_add_notify_handler(struct acpi_video_bus *video, |
| 1888 | struct platform_device *pdev) | 1886 | struct device *parent) |
| 1889 | { | 1887 | { |
| 1890 | struct input_dev *input; | 1888 | struct input_dev *input; |
| 1891 | struct acpi_video_device *dev; | 1889 | struct acpi_video_device *dev; |
| @@ -1908,7 +1906,7 @@ static int acpi_video_bus_add_notify_handler(struct acpi_video_bus *video, · 16 unmodified lines | |||
| 1908 | input->phys = video->phys; | 1906 | input->phys = video->phys; |
| 1909 | input->id.bustype = BUS_HOST; | 1907 | input->id.bustype = BUS_HOST; |
| 1910 | input->id.product = 0x06; | 1908 | input->id.product = 0x06; |
| 1911 | input->dev.parent = &pdev->dev; | 1909 | input->dev.parent = parent; |
| 1912 | input->evbit[0] = BIT(EV_KEY); | 1910 | input->evbit[0] = BIT(EV_KEY); |
| 1913 | set_bit(KEY_SWITCHVIDEOMODE, input->keybit); | 1911 | set_bit(KEY_SWITCHVIDEOMODE, input->keybit); |
| 1914 | set_bit(KEY_VIDEO_NEXT, input->keybit); | 1912 | set_bit(KEY_VIDEO_NEXT, input->keybit); |
| @@ -1980,9 +1978,10 @@ static int acpi_video_bus_put_devices(struct acpi_video_bus *video) · 65 unmodified lines | |||
| 1980 | 1978 | ||
| 1981 | static int instance; | 1979 | static int instance; |
| 1982 | 1980 | ||
| 1983 | static int acpi_video_bus_probe(struct platform_device *pdev) | 1981 | static int acpi_video_bus_probe(struct auxiliary_device *aux_dev, |
| 1982 | const struct auxiliary_device_id *id_unused) | ||
| 1984 | { | 1983 | { |
| 1985 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 1984 | struct acpi_device *device = ACPI_COMPANION(&aux_dev->dev); |
| 1986 | struct acpi_video_bus *video; | 1985 | struct acpi_video_bus *video; |
| 1987 | bool auto_detect; | 1986 | bool auto_detect; |
| 1988 | int error; | 1987 | int error; |
| @@ -2019,7 +2018,7 @@ static int acpi_video_bus_probe(struct platform_device *pdev) · 30 unmodified lines | |||
| 2019 | instance++; | 2018 | instance++; |
| 2020 | } | 2019 | } |
| 2021 | 2020 | ||
| 2022 | platform_set_drvdata(pdev, video); | 2021 | auxiliary_set_drvdata(aux_dev, video); |
| 2023 | 2022 | ||
| 2024 | video->device = device; | 2023 | video->device = device; |
| 2025 | strscpy(acpi_device_name(device), ACPI_VIDEO_BUS_NAME); | 2024 | strscpy(acpi_device_name(device), ACPI_VIDEO_BUS_NAME); |
| @@ -2068,7 +2067,7 @@ static int acpi_video_bus_probe(struct platform_device *pdev) · 42 unmodified lines | |||
| 2068 | !auto_detect) | 2067 | !auto_detect) |
| 2069 | acpi_video_bus_register_backlight(video); | 2068 | acpi_video_bus_register_backlight(video); |
| 2070 | 2069 | ||
| 2071 | error = acpi_video_bus_add_notify_handler(video, pdev); | 2070 | error = acpi_video_bus_add_notify_handler(video, &aux_dev->dev); |
| 2072 | if (error) | 2071 | if (error) |
| 2073 | goto err_del; | 2072 | goto err_del; |
| 2074 | 2073 | ||
| @@ -2096,10 +2095,10 @@ err_free_video: · 21 unmodified lines | |||
| 2096 | return error; | 2095 | return error; |
| 2097 | } | 2096 | } |
| 2098 | 2097 | ||
| 2099 | static void acpi_video_bus_remove(struct platform_device *pdev) | 2098 | static void acpi_video_bus_remove(struct auxiliary_device *aux_dev) |
| 2100 | { | 2099 | { |
| 2101 | struct acpi_video_bus *video = platform_get_drvdata(pdev); | 2100 | struct acpi_video_bus *video = auxiliary_get_drvdata(aux_dev); |
| 2102 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 2101 | struct acpi_device *device = ACPI_COMPANION(&aux_dev->dev); |
| 2103 | 2102 | ||
| 2104 | acpi_dev_remove_notify_handler(device, ACPI_DEVICE_NOTIFY, | 2103 | acpi_dev_remove_notify_handler(device, ACPI_DEVICE_NOTIFY, |
| 2105 | acpi_video_bus_notify); | 2104 | acpi_video_bus_notify); |
| @@ -2163,7 +2162,7 @@ int acpi_video_register(void) · 57 unmodified lines | |||
| 2163 | 2162 | ||
| 2164 | dmi_check_system(video_dmi_table); | 2163 | dmi_check_system(video_dmi_table); |
| 2165 | 2164 | ||
| 2166 | ret = platform_driver_register(&acpi_video_bus); | 2165 | ret = auxiliary_driver_register(&acpi_video_bus); |
| 2167 | if (ret) | 2166 | if (ret) |
| 2168 | goto leave; | 2167 | goto leave; |
| 2169 | 2168 | ||
| @@ -2183,7 +2182,7 @@ void acpi_video_unregister(void) · 13 unmodified lines | |||
| 2183 | { | 2182 | { |
| 2184 | mutex_lock(®ister_count_mutex); | 2183 | mutex_lock(®ister_count_mutex); |
| 2185 | if (register_count) { | 2184 | if (register_count) { |
| 2186 | platform_driver_unregister(&acpi_video_bus); | 2185 | auxiliary_driver_unregister(&acpi_video_bus); |
| 2187 | register_count = 0; | 2186 | register_count = 0; |
| 2188 | may_report_brightness_keys = false; | 2187 | may_report_brightness_keys = false; |
| 2189 | } | 2188 | } |
| @@ -379,8 +379,9 @@ const union acpi_predefined_info acpi_gbl_predefined_methods[] = { · 378 unmodified lines | |||
| 379 | 379 | ||
| 380 | {{"_CPC", METHOD_0ARGS, | 380 | {{"_CPC", METHOD_0ARGS, |
| 381 | METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Variable-length (Ints/Bufs) */ | 381 | METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Variable-length (Ints/Bufs) */ |
| 382 | PACKAGE_INFO(ACPI_PTYPE1_VAR, ACPI_RTYPE_INTEGER | ACPI_RTYPE_BUFFER, 0, | 382 | PACKAGE_INFO(ACPI_PTYPE1_VAR, |
| 383 | 0, 0, 0), | 383 | ACPI_RTYPE_INTEGER | ACPI_RTYPE_BUFFER | |
| 384 | ACPI_RTYPE_PACKAGE, 0, 0, 0, 0), | ||
| 384 | 385 | ||
| 385 | {{"_CR3", METHOD_0ARGS, /* ACPI 6.0 */ | 386 | {{"_CR3", METHOD_0ARGS, /* ACPI 6.0 */ |
| 386 | METHOD_RETURNS(ACPI_RTYPE_INTEGER)}}, | 387 | METHOD_RETURNS(ACPI_RTYPE_INTEGER)}}, |
| @@ -1456,15 +1456,6 @@ int acpi_dev_pm_attach(struct device *dev, bool power_on) · 1455 unmodified lines | |||
| 1456 | if (!adev || !acpi_match_device_ids(adev, special_pm_ids)) | 1456 | if (!adev || !acpi_match_device_ids(adev, special_pm_ids)) |
| 1457 | return 0; | 1457 | return 0; |
| 1458 | 1458 | ||
| 1459 | /* | ||
| 1460 | * Skip devices whose ACPI companions don't support power management and | ||
| 1461 | * don't have a wakeup GPE. | ||
| 1462 | */ | ||
| 1463 | if (!acpi_device_power_manageable(adev) && !acpi_device_can_wakeup(adev)) { | ||
| 1464 | dev_dbg(dev, "No ACPI power management or wakeup GPE\n"); | ||
| 1465 | return 0; | ||
| 1466 | } | ||
| 1467 | |||
| 1468 | /* | 1459 | /* |
| 1469 | * Only attach the power domain to the first device if the | 1460 | * Only attach the power domain to the first device if the |
| 1470 | * companion is shared by multiple. This is to prevent doing power | 1461 | * companion is shared by multiple. This is to prevent doing power |
| @@ -389,6 +389,19 @@ static const struct dmi_system_id acpi_osi_dmi_table[] __initconst = { · 388 unmodified lines | |||
| 389 | }, | 389 | }, |
| 390 | }, | 390 | }, |
| 391 | 391 | ||
| 392 | /* | ||
| 393 | * The screen backlight turns off during udev device creation | ||
| 394 | * when returning true for _OSI("Windows 2009") | ||
| 395 | */ | ||
| 396 | { | ||
| 397 | .callback = dmi_disable_osi_win7, | ||
| 398 | .ident = "Acer Aspire One D255", | ||
| 399 | .matches = { | ||
| 400 | DMI_MATCH(DMI_SYS_VENDOR, "Acer"), | ||
| 401 | DMI_MATCH(DMI_PRODUCT_NAME, "AOD255"), | ||
| 402 | }, | ||
| 403 | }, | ||
| 404 | |||
| 392 | /* | 405 | /* |
| 393 | * The wireless hotkey does not work on those machines when | 406 | * The wireless hotkey does not work on those machines when |
| 394 | * returning true for _OSI("Windows 2012") | 407 | * returning true for _OSI("Windows 2012") |
| @@ -1681,7 +1681,7 @@ acpi_status __init acpi_os_initialize(void) · 1680 unmodified lines | |||
| 1681 | * Use acpi_os_map_generic_address to pre-map the reset | 1681 | * Use acpi_os_map_generic_address to pre-map the reset |
| 1682 | * register if it's in system memory. | 1682 | * register if it's in system memory. |
| 1683 | */ | 1683 | */ |
| 1684 | void *rv; | 1684 | void __iomem *rv; |
| 1685 | 1685 | ||
| 1686 | rv = acpi_os_map_generic_address(&acpi_gbl_FADT.reset_register); | 1686 | rv = acpi_os_map_generic_address(&acpi_gbl_FADT.reset_register); |
| 1687 | pr_debug("%s: Reset register mapping %s\n", __func__, | 1687 | pr_debug("%s: Reset register mapping %s\n", __func__, |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | |||
| 6 | #define pr_fmt(fmt) "ACPI: " fmt | 6 | #define pr_fmt(fmt) "ACPI: " fmt |
| 7 | 7 | ||
| 8 | #include <linux/async.h> | 8 | #include <linux/async.h> |
| 9 | #include <linux/auxiliary_bus.h> | ||
| 9 | #include <linux/module.h> | 10 | #include <linux/module.h> |
| 10 | #include <linux/init.h> | 11 | #include <linux/init.h> |
| 11 | #include <linux/slab.h> | 12 | #include <linux/slab.h> |
| @@ -2192,6 +2193,44 @@ static acpi_status acpi_bus_check_add_2(acpi_handle handle, u32 lvl_not_used, · 2180 unmodified lines | |||
| 2192 | return acpi_bus_check_add(handle, false, (struct acpi_device **)ret_p); | 2193 | return acpi_bus_check_add(handle, false, (struct acpi_device **)ret_p); |
| 2193 | } | 2194 | } |
| 2194 | 2195 | ||
| 2196 | static void acpi_video_bus_device_release(struct device *dev) | ||
| 2197 | { | ||
| 2198 | struct auxiliary_device *aux_dev = to_auxiliary_dev(dev); | ||
| 2199 | |||
| 2200 | kfree(aux_dev); | ||
| 2201 | } | ||
| 2202 | |||
| 2203 | static void acpi_create_video_bus_device(struct acpi_device *adev, | ||
| 2204 | struct acpi_device *parent) | ||
| 2205 | { | ||
| 2206 | struct auxiliary_device *aux_dev; | ||
| 2207 | static unsigned int aux_dev_id; | ||
| 2208 | |||
| 2209 | aux_dev = kzalloc_obj(*aux_dev); | ||
| 2210 | if (!aux_dev) | ||
| 2211 | return; | ||
| 2212 | |||
| 2213 | aux_dev->id = aux_dev_id++; | ||
| 2214 | aux_dev->name = "video_bus"; | ||
| 2215 | aux_dev->dev.parent = acpi_get_first_physical_node(parent); | ||
| 2216 | if (!aux_dev->dev.parent) | ||
| 2217 | goto err; | ||
| 2218 | |||
| 2219 | aux_dev->dev.release = acpi_video_bus_device_release; | ||
| 2220 | |||
| 2221 | if (auxiliary_device_init(aux_dev)) | ||
| 2222 | goto err; | ||
| 2223 | |||
| 2224 | ACPI_COMPANION_SET(&aux_dev->dev, adev); | ||
| 2225 | if (__auxiliary_device_add(aux_dev, "acpi")) | ||
| 2226 | auxiliary_device_uninit(aux_dev); | ||
| 2227 | |||
| 2228 | return; | ||
| 2229 | |||
| 2230 | err: | ||
| 2231 | kfree(aux_dev); | ||
| 2232 | } | ||
| 2233 | |||
| 2195 | struct acpi_scan_system_dev { | 2234 | struct acpi_scan_system_dev { |
| 2196 | struct list_head node; | 2235 | struct list_head node; |
| 2197 | struct acpi_device *adev; | 2236 | struct acpi_device *adev; |
| @@ -2229,6 +2268,12 @@ static void acpi_default_enumeration(struct acpi_device *device) · 31 unmodified lines | |||
| 2229 | sd->adev = device; | 2268 | sd->adev = device; |
| 2230 | list_add_tail(&sd->node, &acpi_scan_system_dev_list); | 2269 | list_add_tail(&sd->node, &acpi_scan_system_dev_list); |
| 2231 | } | 2270 | } |
| 2271 | } else if (device->pnp.type.backlight) { | ||
| 2272 | struct acpi_device *parent; | ||
| 2273 | |||
| 2274 | parent = acpi_dev_parent(device); | ||
| 2275 | if (parent) | ||
| 2276 | acpi_create_video_bus_device(device, parent); | ||
| 2232 | } else { | 2277 | } else { |
| 2233 | /* For a regular device object, create a platform device. */ | 2278 | /* For a regular device object, create a platform device. */ |
| 2234 | acpi_create_platform_device(device, NULL); | 2279 | acpi_create_platform_device(device, NULL); |
| @@ -386,6 +386,14 @@ static const struct dmi_system_id acpisleep_dmi_table[] __initconst = { · 385 unmodified lines | |||
| 386 | DMI_MATCH(DMI_PRODUCT_NAME, "80E1"), | 386 | DMI_MATCH(DMI_PRODUCT_NAME, "80E1"), |
| 387 | }, | 387 | }, |
| 388 | }, | 388 | }, |
| 389 | { | ||
| 390 | .callback = init_nvs_save_s3, | ||
| 391 | .ident = "Lenovo G70-35", | ||
| 392 | .matches = { | ||
| 393 | DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"), | ||
| 394 | DMI_MATCH(DMI_PRODUCT_NAME, "80Q5"), | ||
| 395 | }, | ||
| 396 | }, | ||
| 389 | /* | 397 | /* |
| 390 | * ThinkPad X1 Tablet(2016) cannot do suspend-to-idle using | 398 | * ThinkPad X1 Tablet(2016) cannot do suspend-to-idle using |
| 391 | * the Low Power S0 Idle firmware interface (see | 399 | * the Low Power S0 Idle firmware interface (see |
| @@ -142,6 +142,30 @@ pub(crate) struct ShrinkablePageRange { · 141 unmodified lines | |||
| 142 | _pin: PhantomPinned, | 142 | _pin: PhantomPinned, |
| 143 | } | 143 | } |
| 144 | 144 | ||
| 145 | // We do not define any ops. For now, used only to check identity of vmas. | ||
| 146 | static BINDER_VM_OPS: bindings::vm_operations_struct = pin_init::zeroed(); | ||
| 147 | |||
| 148 | // To ensure that we do not accidentally install pages into or zap pages from the wrong vma, we | ||
| 149 | // check its vm_ops and private data before using it. | ||
| 150 | fn check_vma(vma: &virt::VmaRef, owner: *const ShrinkablePageRange) -> Option<&virt::VmaMixedMap> { | ||
| 151 | // SAFETY: Just reading the vm_ops pointer of any active vma is safe. | ||
| 152 | let vm_ops = unsafe { (*vma.as_ptr()).vm_ops }; | ||
| 153 | if !ptr::eq(vm_ops, &BINDER_VM_OPS) { | ||
| 154 | return None; | ||
| 155 | } | ||
| 156 | |||
| 157 | // SAFETY: Reading the vm_private_data pointer of a binder-owned vma is safe. | ||
| 158 | let vm_private_data = unsafe { (*vma.as_ptr()).vm_private_data }; | ||
| 159 | // The ShrinkablePageRange is only dropped when the Process is dropped, which only happens once | ||
| 160 | // the file's ->release handler is invoked, which means the ShrinkablePageRange outlives any | ||
| 161 | // VMA associated with it, so there can't be any false positives due to pointer reuse here. | ||
| 162 | if !ptr::eq(vm_private_data, owner.cast()) { | ||
| 163 | return None; | ||
| 164 | } | ||
| 165 | |||
| 166 | vma.as_mixedmap_vma() | ||
| 167 | } | ||
| 168 | |||
| 145 | struct Inner { | 169 | struct Inner { |
| 146 | /// Array of pages. | 170 | /// Array of pages. |
| 147 | /// | 171 | /// |
| @@ -308,6 +332,18 @@ impl ShrinkablePageRange { · 160 unmodified lines | |||
| 308 | inner.size = num_pages; | 332 | inner.size = num_pages; |
| 309 | inner.vma_addr = vma.start(); | 333 | inner.vma_addr = vma.start(); |
| 310 | 334 | ||
| 335 | // This pointer is only used for comparison - it's not dereferenced. | ||
| 336 | // | ||
| 337 | // SAFETY: We own the vma, and we don't use any methods on VmaNew that rely on | ||
| 338 | // `vm_private_data`. | ||
| 339 | unsafe { | ||
| 340 | (*vma.as_ptr()).vm_private_data = ptr::from_ref(self).cast_mut().cast::<c_void>() | ||
| 341 | }; | ||
| 342 | |||
| 343 | // SAFETY: We own the vma, and we don't use any methods on VmaNew that rely on | ||
| 344 | // `vm_ops`. | ||
| 345 | unsafe { (*vma.as_ptr()).vm_ops = &BINDER_VM_OPS }; | ||
| 346 | |||
| 311 | Ok(num_pages) | 347 | Ok(num_pages) |
| 312 | } | 348 | } |
| 313 | 349 | ||
| @@ -399,22 +435,25 @@ impl ShrinkablePageRange { · 85 unmodified lines | |||
| 399 | // | 435 | // |
| 400 | // Using `mmput_async` avoids this, because then the `mm` cleanup is instead queued to a | 436 | // Using `mmput_async` avoids this, because then the `mm` cleanup is instead queued to a |
| 401 | // workqueue. | 437 | // workqueue. |
| 402 | MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?) | 438 | let mm = MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?); |
| 403 | .mmap_read_lock() | 439 | { |
| 404 | .vma_lookup(vma_addr) | 440 | let vma_read; |
| 405 | .ok_or(ESRCH)? | 441 | let mmap_read; |
| 406 | .as_mixedmap_vma() | 442 | let vma = if let Some(ret) = mm.lock_vma_under_rcu(vma_addr) { |
| 407 | .ok_or(ESRCH)? | 443 | vma_read = ret; |
| 408 | .vm_insert_page(user_page_addr, &new_page) | 444 | check_vma(&vma_read, self) |
| 409 | .inspect_err(|err| { | 445 | } else { |
| 410 | pr_warn!( | 446 | mmap_read = mm.mmap_read_lock(); |
| 411 | "Failed to vm_insert_page({}): vma_addr:{} i:{} err:{:?}", | 447 | mmap_read |
| 412 | user_page_addr, | 448 | .vma_lookup(vma_addr) |
| 413 | vma_addr, | 449 | .and_then(|vma| check_vma(vma, self)) |
| 414 | i, | 450 | }; |
| 415 | err | 451 | |
| 416 | ) | 452 | match vma { |
| 417 | })?; | 453 | Some(vma) => vma.vm_insert_page(user_page_addr, &new_page)?, |
| 454 | None => return Err(ESRCH), | ||
| 455 | } | ||
| 456 | } | ||
| 418 | 457 | ||
| 419 | let inner = self.lock.lock(); | 458 | let inner = self.lock.lock(); |
| 420 | 459 | ||
| @@ -667,12 +706,15 @@ unsafe extern "C" fn rust_shrink_free_page( · 246 unmodified lines | |||
| 667 | let mmap_read; | 706 | let mmap_read; |
| 668 | let mm_mutex; | 707 | let mm_mutex; |
| 669 | let vma_addr; | 708 | let vma_addr; |
| 709 | let range_ptr; | ||
| 670 | 710 | ||
| 671 | { | 711 | { |
| 672 | // CAST: The `list_head` field is first in `PageInfo`. | 712 | // CAST: The `list_head` field is first in `PageInfo`. |
| 673 | let info = item as *mut PageInfo; | 713 | let info = item as *mut PageInfo; |
| 674 | // SAFETY: The `range` field of `PageInfo` is immutable. | 714 | // SAFETY: The `range` field of `PageInfo` is immutable. |
| 675 | let range = unsafe { &*((*info).range) }; | 715 | range_ptr = unsafe { (*info).range }; |
| 716 | // SAFETY: The `range` outlives its `PageInfo` values. | ||
| 717 | let range = unsafe { &*range_ptr }; | ||
| 676 | 718 | ||
| 677 | mm = match range.mm.mmget_not_zero() { | 719 | mm = match range.mm.mmget_not_zero() { |
| 678 | Some(mm) => MmWithUser::into_mmput_async(mm), | 720 | Some(mm) => MmWithUser::into_mmput_async(mm), |
| @@ -717,9 +759,11 @@ unsafe extern "C" fn rust_shrink_free_page( · 38 unmodified lines | |||
| 717 | // SAFETY: The lru lock is locked when this method is called. | 759 | // SAFETY: The lru lock is locked when this method is called. |
| 718 | unsafe { bindings::spin_unlock(&raw mut (*lru).lock) }; | 760 | unsafe { bindings::spin_unlock(&raw mut (*lru).lock) }; |
| 719 | 761 | ||
| 720 | if let Some(vma) = mmap_read.vma_lookup(vma_addr) { | 762 | if let Some(unchecked_vma) = mmap_read.vma_lookup(vma_addr) { |
| 721 | let user_page_addr = vma_addr + (page_index << PAGE_SHIFT); | 763 | if let Some(vma) = check_vma(unchecked_vma, range_ptr) { |
| 722 | vma.zap_page_range_single(user_page_addr, PAGE_SIZE); | 764 | let user_page_addr = vma_addr + (page_index << PAGE_SHIFT); |
| 765 | vma.zap_page_range_single(user_page_addr, PAGE_SIZE); | ||
| 766 | } | ||
| 723 | } | 767 | } |
| 724 | 768 | ||
| 725 | drop(mmap_read); | 769 | drop(mmap_read); |
| @@ -1295,7 +1295,8 @@ impl Process { · 1294 unmodified lines | |||
| 1295 | } | 1295 | } |
| 1296 | 1296 | ||
| 1297 | pub(crate) fn dead_binder_done(&self, cookie: u64, thread: &Thread) { | 1297 | pub(crate) fn dead_binder_done(&self, cookie: u64, thread: &Thread) { |
| 1298 | if let Some(death) = self.inner.lock().pull_delivered_death(cookie) { | 1298 | let death = self.inner.lock().pull_delivered_death(cookie); |
| 1299 | if let Some(death) = death { | ||
| 1299 | death.set_notification_done(thread); | 1300 | death.set_notification_done(thread); |
| 1300 | } | 1301 | } |
| 1301 | } | 1302 | } |
| @@ -118,7 +118,7 @@ impl<T> ArrayRangeAllocator<T> { · 117 unmodified lines | |||
| 118 | size: usize, | 118 | size: usize, |
| 119 | is_oneway: bool, | 119 | is_oneway: bool, |
| 120 | pid: Pid, | 120 | pid: Pid, |
| 121 | ) -> Result<usize> { | 121 | ) -> Result<(usize, bool)> { |
| 122 | // Compute new value of free_oneway_space, which is set only on success. | 122 | // Compute new value of free_oneway_space, which is set only on success. |
| 123 | let new_oneway_space = if is_oneway { | 123 | let new_oneway_space = if is_oneway { |
| 124 | match self.free_oneway_space.checked_sub(size) { | 124 | match self.free_oneway_space.checked_sub(size) { |
| @@ -146,7 +146,38 @@ impl<T> ArrayRangeAllocator<T> { · 21 unmodified lines | |||
| 146 | .ok() | 146 | .ok() |
| 147 | .unwrap(); | 147 | .unwrap(); |
| 148 | 148 | ||
| 149 | Ok(insert_at_offset) | 149 | // Start detecting spammers once we have less than 20% |
| 150 | // of async space left (which is less than 10% of total | ||
| 151 | // buffer size). | ||
| 152 | // | ||
| 153 | // (This will short-circuit, so `low_oneway_space` is | ||
| 154 | // only called when necessary.) | ||
| 155 | let oneway_spam_detected = | ||
| 156 | is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | ||
| 157 | |||
| 158 | Ok((insert_at_offset, oneway_spam_detected)) | ||
| 159 | } | ||
| 160 | |||
| 161 | /// Find the amount and size of buffers allocated by the current caller. | ||
| 162 | /// | ||
| 163 | /// The idea is that once we cross the threshold, whoever is responsible | ||
| 164 | /// for the low async space is likely to try to send another async transaction, | ||
| 165 | /// and at some point we'll catch them in the act. This is more efficient | ||
| 166 | /// than keeping a map per pid. | ||
| 167 | fn low_oneway_space(&self, calling_pid: Pid) -> bool { | ||
| 168 | let mut total_alloc_size = 0; | ||
| 169 | let mut num_buffers = 0; | ||
| 170 | |||
| 171 | // Warn if this pid has more than 50 transactions, or more than 50% of | ||
| 172 | // async space (which is 25% of total buffer size). Oneway spam is only | ||
| 173 | // detected when the threshold is exceeded. | ||
| 174 | for range in &self.ranges { | ||
| 175 | if range.state.is_oneway() && range.state.pid() == calling_pid { | ||
| 176 | total_alloc_size += range.size; | ||
| 177 | num_buffers += 1; | ||
| 178 | } | ||
| 179 | } | ||
| 180 | num_buffers > 50 || total_alloc_size > self.size / 4 | ||
| 150 | } | 181 | } |
| 151 | 182 | ||
| 152 | pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> { | 183 | pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> { |
| @@ -188,11 +188,11 @@ impl<T> RangeAllocator<T> { · 187 unmodified lines | |||
| 188 | self.reserve_new(args) | 188 | self.reserve_new(args) |
| 189 | } | 189 | } |
| 190 | Impl::Array(array) => { | 190 | Impl::Array(array) => { |
| 191 | let offset = | 191 | let (offset, oneway_spam_detected) = |
| 192 | array.reserve_new(args.debug_id, args.size, args.is_oneway, args.pid)?; | 192 | array.reserve_new(args.debug_id, args.size, args.is_oneway, args.pid)?; |
| 193 | Ok(ReserveNew::Success(ReserveNewSuccess { | 193 | Ok(ReserveNew::Success(ReserveNewSuccess { |
| 194 | offset, | 194 | offset, |
| 195 | oneway_spam_detected: false, | 195 | oneway_spam_detected, |
| 196 | _empty_array_alloc: args.empty_array_alloc, | 196 | _empty_array_alloc: args.empty_array_alloc, |
| 197 | _new_tree_alloc: args.new_tree_alloc, | 197 | _new_tree_alloc: args.new_tree_alloc, |
| 198 | _tree_alloc: args.tree_alloc, | 198 | _tree_alloc: args.tree_alloc, |
| @@ -164,15 +164,6 @@ impl<T> TreeRangeAllocator<T> { · 163 unmodified lines | |||
| 164 | self.free_oneway_space | 164 | self.free_oneway_space |
| 165 | }; | 165 | }; |
| 166 | 166 | ||
| 167 | // Start detecting spammers once we have less than 20% | ||
| 168 | // of async space left (which is less than 10% of total | ||
| 169 | // buffer size). | ||
| 170 | // | ||
| 171 | // (This will short-circut, so `low_oneway_space` is | ||
| 172 | // only called when necessary.) | ||
| 173 | let oneway_spam_detected = | ||
| 174 | is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | ||
| 175 | |||
| 176 | let (found_size, found_off, tree_node, free_tree_node) = match self.find_best_match(size) { | 167 | let (found_size, found_off, tree_node, free_tree_node) = match self.find_best_match(size) { |
| 177 | None => { | 168 | None => { |
| 178 | pr_warn!("ENOSPC from range_alloc.reserve_new - size: {}", size); | 169 | pr_warn!("ENOSPC from range_alloc.reserve_new - size: {}", size); |
| @@ -203,6 +194,15 @@ impl<T> TreeRangeAllocator<T> { · 24 unmodified lines | |||
| 203 | self.free_tree.insert(free_tree_node); | 194 | self.free_tree.insert(free_tree_node); |
| 204 | } | 195 | } |
| 205 | 196 | ||
| 197 | // Start detecting spammers once we have less than 20% | ||
| 198 | // of async space left (which is less than 10% of total | ||
| 199 | // buffer size). | ||
| 200 | // | ||
| 201 | // (This will short-circuit, so `low_oneway_space` is | ||
| 202 | // only called when necessary.) | ||
| 203 | let oneway_spam_detected = | ||
| 204 | is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | ||
| 205 | |||
| 206 | Ok((found_off, oneway_spam_detected)) | 206 | Ok((found_off, oneway_spam_detected)) |
| 207 | } | 207 | } |
| 208 | 208 | ||
| @@ -1015,12 +1015,9 @@ impl Thread { · 1014 unmodified lines | |||
| 1015 | 1015 | ||
| 1016 | // Copy offsets if there are any. | 1016 | // Copy offsets if there are any. |
| 1017 | if offsets_size > 0 { | 1017 | if offsets_size > 0 { |
| 1018 | { | 1018 | let mut offsets_reader = |
| 1019 | let mut reader = | 1019 | UserSlice::new(UserPtr::from_addr(trd_data_ptr.offsets as _), offsets_size) |
| 1020 | UserSlice::new(UserPtr::from_addr(trd_data_ptr.offsets as _), offsets_size) | 1020 | .reader(); |
| 1021 | .reader(); | ||
| 1022 | alloc.copy_into(&mut reader, aligned_data_size, offsets_size)?; | ||
| 1023 | } | ||
| 1024 | 1021 | ||
| 1025 | let offsets_start = aligned_data_size; | 1022 | let offsets_start = aligned_data_size; |
| 1026 | let offsets_end = aligned_data_size + offsets_size; | 1023 | let offsets_end = aligned_data_size + offsets_size; |
| @@ -1041,11 +1038,9 @@ impl Thread { · 14 unmodified lines | |||
| 1041 | .step_by(size_of::<u64>()) | 1038 | .step_by(size_of::<u64>()) |
| 1042 | .enumerate() | 1039 | .enumerate() |
| 1043 | { | 1040 | { |
| 1044 | let offset: usize = view | 1041 | let offset = offsets_reader.read::<u64>()?; |
| 1045 | .alloc | 1042 | view.alloc.write(index_offset, &offset)?; |
| 1046 | .read::<u64>(index_offset)? | 1043 | let offset: usize = offset.try_into().map_err(|_| EINVAL)?; |
| 1047 | .try_into() | ||
| 1048 | .map_err(|_| EINVAL)?; | ||
| 1049 | 1044 | ||
| 1050 | if offset < end_of_previous_object || !is_aligned(offset, size_of::<u32>()) { | 1045 | if offset < end_of_previous_object || !is_aligned(offset, size_of::<u32>()) { |
| 1051 | pr_warn!("Got transaction with invalid offset."); | 1046 | pr_warn!("Got transaction with invalid offset."); |
| @@ -4189,6 +4189,7 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { · 4188 unmodified lines | |||
| 4189 | ATA_QUIRK_FIRMWARE_WARN }, | 4189 | ATA_QUIRK_FIRMWARE_WARN }, |
| 4190 | 4190 | ||
| 4191 | /* Seagate disks with LPM issues */ | 4191 | /* Seagate disks with LPM issues */ |
| 4192 | { "ST1000DM010-2EP102", NULL, ATA_QUIRK_NOLPM }, | ||
| 4192 | { "ST2000DM008-2FR102", NULL, ATA_QUIRK_NOLPM }, | 4193 | { "ST2000DM008-2FR102", NULL, ATA_QUIRK_NOLPM }, |
| 4193 | 4194 | ||
| 4194 | /* drives which fail FPDMA_AA activation (some may freeze afterwards) | 4195 | /* drives which fail FPDMA_AA activation (some may freeze afterwards) |
| @@ -4231,6 +4232,7 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { · 36 unmodified lines | |||
| 4231 | /* Devices that do not need bridging limits applied */ | 4232 | /* Devices that do not need bridging limits applied */ |
| 4232 | { "MTRON MSP-SATA*", NULL, ATA_QUIRK_BRIDGE_OK }, | 4233 | { "MTRON MSP-SATA*", NULL, ATA_QUIRK_BRIDGE_OK }, |
| 4233 | { "BUFFALO HD-QSU2/R5", NULL, ATA_QUIRK_BRIDGE_OK }, | 4234 | { "BUFFALO HD-QSU2/R5", NULL, ATA_QUIRK_BRIDGE_OK }, |
| 4235 | { "QEMU HARDDISK", "2.5+", ATA_QUIRK_BRIDGE_OK }, | ||
| 4234 | 4236 | ||
| 4235 | /* Devices which aren't very happy with higher link speeds */ | 4237 | /* Devices which aren't very happy with higher link speeds */ |
| 4236 | { "WD My Book", NULL, ATA_QUIRK_1_5_GBPS }, | 4238 | { "WD My Book", NULL, ATA_QUIRK_1_5_GBPS }, |
| @@ -6269,10 +6271,6 @@ static void ata_port_detach(struct ata_port *ap) · 2032 unmodified lines | |||
| 6269 | } | 6271 | } |
| 6270 | } | 6272 | } |
| 6271 | 6273 | ||
| 6272 | /* Make sure the deferred qc work finished. */ | ||
| 6273 | cancel_work_sync(&ap->deferred_qc_work); | ||
| 6274 | WARN_ON(ap->deferred_qc); | ||
| 6275 | |||
| 6276 | /* Tell EH to disable all devices */ | 6274 | /* Tell EH to disable all devices */ |
| 6277 | ap->pflags |= ATA_PFLAG_UNLOADING; | 6275 | ap->pflags |= ATA_PFLAG_UNLOADING; |
| 6278 | ata_port_schedule_eh(ap); | 6276 | ata_port_schedule_eh(ap); |
| @@ -6283,9 +6281,11 @@ static void ata_port_detach(struct ata_port *ap) · 4 unmodified lines | |||
| 6283 | /* wait till EH commits suicide */ | 6281 | /* wait till EH commits suicide */ |
| 6284 | ata_port_wait_eh(ap); | 6282 | ata_port_wait_eh(ap); |
| 6285 | 6283 | ||
| 6286 | /* it better be dead now */ | 6284 | /* It better be dead now and not have any remaining deferred qc. */ |
| 6287 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); | 6285 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); |
| 6286 | WARN_ON(ap->deferred_qc); | ||
| 6288 | 6287 | ||
| 6288 | cancel_work_sync(&ap->deferred_qc_work); | ||
| 6289 | cancel_delayed_work_sync(&ap->hotplug_task); | 6289 | cancel_delayed_work_sync(&ap->hotplug_task); |
| 6290 | cancel_delayed_work_sync(&ap->scsi_rescan_task); | 6290 | cancel_delayed_work_sync(&ap->scsi_rescan_task); |
| 6291 | 6291 | ||
| @@ -640,12 +640,29 @@ void ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 639 unmodified lines | |||
| 640 | set_host_byte(scmd, DID_OK); | 640 | set_host_byte(scmd, DID_OK); |
| 641 | 641 | ||
| 642 | ata_qc_for_each_raw(ap, qc, i) { | 642 | ata_qc_for_each_raw(ap, qc, i) { |
| 643 | if (qc->flags & ATA_QCFLAG_ACTIVE && | 643 | if (qc->scsicmd != scmd) |
| 644 | qc->scsicmd == scmd) | 644 | continue; |
| 645 | if ((qc->flags & ATA_QCFLAG_ACTIVE) || | ||
| 646 | qc == ap->deferred_qc) | ||
| 645 | break; | 647 | break; |
| 646 | } | 648 | } |
| 647 | 649 | ||
| 648 | if (i < ATA_MAX_QUEUE) { | 650 | if (i < ATA_MAX_QUEUE && qc == ap->deferred_qc) { |
| 651 | /* | ||
| 652 | * This is a deferred command that timed out while | ||
| 653 | * waiting for the command queue to drain. Since the qc | ||
| 654 | * is not active yet (deferred_qc is still set, so the | ||
| 655 | * deferred qc work has not issued the command yet), | ||
| 656 | * simply signal the timeout by finishing the SCSI | ||
| 657 | * command and clear the deferred qc to prevent the | ||
| 658 | * deferred qc work from issuing this qc. | ||
| 659 | */ | ||
| 660 | WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); | ||
| 661 | ap->deferred_qc = NULL; | ||
| 662 | cancel_work(&ap->deferred_qc_work); | ||
| 663 | set_host_byte(scmd, DID_TIME_OUT); | ||
| 664 | scsi_eh_finish_cmd(scmd, &ap->eh_done_q); | ||
| 665 | } else if (i < ATA_MAX_QUEUE) { | ||
| 649 | /* the scmd has an associated qc */ | 666 | /* the scmd has an associated qc */ |
| 650 | if (!(qc->flags & ATA_QCFLAG_EH)) { | 667 | if (!(qc->flags & ATA_QCFLAG_EH)) { |
| 651 | /* which hasn't failed yet, timeout */ | 668 | /* which hasn't failed yet, timeout */ |
| @@ -1699,6 +1699,7 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap) · 1698 unmodified lines | |||
| 1699 | 1699 | ||
| 1700 | scmd = qc->scsicmd; | 1700 | scmd = qc->scsicmd; |
| 1701 | ap->deferred_qc = NULL; | 1701 | ap->deferred_qc = NULL; |
| 1702 | cancel_work(&ap->deferred_qc_work); | ||
| 1702 | ata_qc_free(qc); | 1703 | ata_qc_free(qc); |
| 1703 | scmd->result = (DID_SOFT_ERROR << 16); | 1704 | scmd->result = (DID_SOFT_ERROR << 16); |
| 1704 | scsi_done(scmd); | 1705 | scsi_done(scmd); |
| @@ -179,19 +179,10 @@ void device_release_driver_internal(struct device *dev, const struct device_driv · 178 unmodified lines | |||
| 179 | void driver_detach(const struct device_driver *drv); | 179 | void driver_detach(const struct device_driver *drv); |
| 180 | void driver_deferred_probe_del(struct device *dev); | 180 | void driver_deferred_probe_del(struct device *dev); |
| 181 | void device_set_deferred_probe_reason(const struct device *dev, struct va_format *vaf); | 181 | void device_set_deferred_probe_reason(const struct device *dev, struct va_format *vaf); |
| 182 | static inline int driver_match_device_locked(const struct device_driver *drv, | ||
| 183 | struct device *dev) | ||
| 184 | { | ||
| 185 | device_lock_assert(dev); | ||
| 186 | |||
| 187 | return drv->bus->match ? drv->bus->match(dev, drv) : 1; | ||
| 188 | } | ||
| 189 | |||
| 190 | static inline int driver_match_device(const struct device_driver *drv, | 182 | static inline int driver_match_device(const struct device_driver *drv, |
| 191 | struct device *dev) | 183 | struct device *dev) |
| 192 | { | 184 | { |
| 193 | guard(device)(dev); | 185 | return drv->bus->match ? drv->bus->match(dev, drv) : 1; |
| 194 | return driver_match_device_locked(drv, dev); | ||
| 195 | } | 186 | } |
| 196 | 187 | ||
| 197 | static inline void dev_sync_state(struct device *dev) | 188 | static inline void dev_sync_state(struct device *dev) |
| @@ -928,7 +928,7 @@ static int __device_attach_driver(struct device_driver *drv, void *_data) · 927 unmodified lines | |||
| 928 | bool async_allowed; | 928 | bool async_allowed; |
| 929 | int ret; | 929 | int ret; |
| 930 | 930 | ||
| 931 | ret = driver_match_device_locked(drv, dev); | 931 | ret = driver_match_device(drv, dev); |
| 932 | if (ret == 0) { | 932 | if (ret == 0) { |
| 933 | /* no match */ | 933 | /* no match */ |
| 934 | return 0; | 934 | return 0; |
| @@ -797,7 +797,18 @@ struct fwnode_handle * · 796 unmodified lines | |||
| 797 | fwnode_get_next_child_node(const struct fwnode_handle *fwnode, | 797 | fwnode_get_next_child_node(const struct fwnode_handle *fwnode, |
| 798 | struct fwnode_handle *child) | 798 | struct fwnode_handle *child) |
| 799 | { | 799 | { |
| 800 | return fwnode_call_ptr_op(fwnode, get_next_child_node, child); | 800 | struct fwnode_handle *next; |
| 801 | |||
| 802 | if (IS_ERR_OR_NULL(fwnode)) | ||
| 803 | return NULL; | ||
| 804 | |||
| 805 | /* Try to find a child in primary fwnode */ | ||
| 806 | next = fwnode_call_ptr_op(fwnode, get_next_child_node, child); | ||
| 807 | if (next) | ||
| 808 | return next; | ||
| 809 | |||
| 810 | /* When no more children in primary, continue with secondary */ | ||
| 811 | return fwnode_call_ptr_op(fwnode->secondary, get_next_child_node, child); | ||
| 801 | } | 812 | } |
| 802 | EXPORT_SYMBOL_GPL(fwnode_get_next_child_node); | 813 | EXPORT_SYMBOL_GPL(fwnode_get_next_child_node); |
| 803 | 814 | ||
| @@ -841,19 +852,7 @@ EXPORT_SYMBOL_GPL(fwnode_get_next_available_child_node); · 37 unmodified lines | |||
| 841 | struct fwnode_handle *device_get_next_child_node(const struct device *dev, | 852 | struct fwnode_handle *device_get_next_child_node(const struct device *dev, |
| 842 | struct fwnode_handle *child) | 853 | struct fwnode_handle *child) |
| 843 | { | 854 | { |
| 844 | const struct fwnode_handle *fwnode = dev_fwnode(dev); | 855 | return fwnode_get_next_child_node(dev_fwnode(dev), child); |
| 845 | struct fwnode_handle *next; | ||
| 846 | |||
| 847 | if (IS_ERR_OR_NULL(fwnode)) | ||
| 848 | return NULL; | ||
| 849 | |||
| 850 | /* Try to find a child in primary fwnode */ | ||
| 851 | next = fwnode_get_next_child_node(fwnode, child); | ||
| 852 | if (next) | ||
| 853 | return next; | ||
| 854 | |||
| 855 | /* When no more children in primary, continue with secondary */ | ||
| 856 | return fwnode_get_next_child_node(fwnode->secondary, child); | ||
| 857 | } | 856 | } |
| 858 | EXPORT_SYMBOL_GPL(device_get_next_child_node); | 857 | EXPORT_SYMBOL_GPL(device_get_next_child_node); |
| 859 | 858 | ||
| @@ -483,38 +483,20 @@ void drbd_al_begin_io(struct drbd_device *device, struct drbd_interval *i) · 482 unmodified lines | |||
| 483 | 483 | ||
| 484 | int drbd_al_begin_io_nonblock(struct drbd_device *device, struct drbd_interval *i) | 484 | int drbd_al_begin_io_nonblock(struct drbd_device *device, struct drbd_interval *i) |
| 485 | { | 485 | { |
| 486 | struct lru_cache *al = device->act_log; | ||
| 487 | /* for bios crossing activity log extent boundaries, | 486 | /* for bios crossing activity log extent boundaries, |
| 488 | * we may need to activate two extents in one go */ | 487 | * we may need to activate two extents in one go */ |
| 489 | unsigned first = i->sector >> (AL_EXTENT_SHIFT-9); | 488 | unsigned first = i->sector >> (AL_EXTENT_SHIFT-9); |
| 490 | unsigned last = i->size == 0 ? first : (i->sector + (i->size >> 9) - 1) >> (AL_EXTENT_SHIFT-9); | 489 | unsigned last = i->size == 0 ? first : (i->sector + (i->size >> 9) - 1) >> (AL_EXTENT_SHIFT-9); |
| 491 | unsigned nr_al_extents; | ||
| 492 | unsigned available_update_slots; | ||
| 493 | unsigned enr; | 490 | unsigned enr; |
| 494 | 491 | ||
| 495 | D_ASSERT(device, first <= last); | 492 | if (i->partially_in_al_next_enr) { |
| 496 | 493 | D_ASSERT(device, first < i->partially_in_al_next_enr); | |
| 497 | nr_al_extents = 1 + last - first; /* worst case: all touched extends are cold. */ | 494 | D_ASSERT(device, last >= i->partially_in_al_next_enr); |
| 498 | available_update_slots = min(al->nr_elements - al->used, | 495 | first = i->partially_in_al_next_enr; |
| 499 | al->max_pending_changes - al->pending_changes); | ||
| 500 | |||
| 501 | /* We want all necessary updates for a given request within the same transaction | ||
| 502 | * We could first check how many updates are *actually* needed, | ||
| 503 | * and use that instead of the worst-case nr_al_extents */ | ||
| 504 | if (available_update_slots < nr_al_extents) { | ||
| 505 | /* Too many activity log extents are currently "hot". | ||
| 506 | * | ||
| 507 | * If we have accumulated pending changes already, | ||
| 508 | * we made progress. | ||
| 509 | * | ||
| 510 | * If we cannot get even a single pending change through, | ||
| 511 | * stop the fast path until we made some progress, | ||
| 512 | * or requests to "cold" extents could be starved. */ | ||
| 513 | if (!al->pending_changes) | ||
| 514 | __set_bit(__LC_STARVING, &device->act_log->flags); | ||
| 515 | return -ENOBUFS; | ||
| 516 | } | 496 | } |
| 517 | 497 | ||
| 498 | D_ASSERT(device, first <= last); | ||
| 499 | |||
| 518 | /* Is resync active in this area? */ | 500 | /* Is resync active in this area? */ |
| 519 | for (enr = first; enr <= last; enr++) { | 501 | for (enr = first; enr <= last; enr++) { |
| 520 | struct lc_element *tmp; | 502 | struct lc_element *tmp; |
| @@ -529,14 +511,21 @@ int drbd_al_begin_io_nonblock(struct drbd_device *device, struct drbd_interval * · 8 unmodified lines | |||
| 529 | } | 511 | } |
| 530 | } | 512 | } |
| 531 | 513 | ||
| 532 | /* Checkout the refcounts. | 514 | /* Try to checkout the refcounts. */ |
| 533 | * Given that we checked for available elements and update slots above, | ||
| 534 | * this has to be successful. */ | ||
| 535 | for (enr = first; enr <= last; enr++) { | 515 | for (enr = first; enr <= last; enr++) { |
| 536 | struct lc_element *al_ext; | 516 | struct lc_element *al_ext; |
| 537 | al_ext = lc_get_cumulative(device->act_log, enr); | 517 | al_ext = lc_get_cumulative(device->act_log, enr); |
| 538 | if (!al_ext) | 518 | |
| 539 | drbd_info(device, "LOGIC BUG for enr=%u\n", enr); | 519 | if (!al_ext) { |
| 520 | /* Did not work. We may have exhausted the possible | ||
| 521 | * changes per transaction. Or raced with someone | ||
| 522 | * "locking" it against changes. | ||
| 523 | * Remember where to continue from. | ||
| 524 | */ | ||
| 525 | if (enr > first) | ||
| 526 | i->partially_in_al_next_enr = enr; | ||
| 527 | return -ENOBUFS; | ||
| 528 | } | ||
| 540 | } | 529 | } |
| 541 | return 0; | 530 | return 0; |
| 542 | } | 531 | } |
| @@ -556,7 +545,11 @@ void drbd_al_complete_io(struct drbd_device *device, struct drbd_interval *i) · 13 unmodified lines | |||
| 556 | 545 | ||
| 557 | for (enr = first; enr <= last; enr++) { | 546 | for (enr = first; enr <= last; enr++) { |
| 558 | extent = lc_find(device->act_log, enr); | 547 | extent = lc_find(device->act_log, enr); |
| 559 | if (!extent) { | 548 | /* Yes, this masks a bug elsewhere. However, during normal |
| 549 | * operation this is harmless, so no need to crash the kernel | ||
| 550 | * by the BUG_ON(refcount == 0) in lc_put(). | ||
| 551 | */ | ||
| 552 | if (!extent || extent->refcnt == 0) { | ||
| 560 | drbd_err(device, "al_complete_io() called on inactive extent %u\n", enr); | 553 | drbd_err(device, "al_complete_io() called on inactive extent %u\n", enr); |
| 561 | continue; | 554 | continue; |
| 562 | } | 555 | } |
| @@ -8,12 +8,15 @@ · 7 unmodified lines | |||
| 8 | struct drbd_interval { | 8 | struct drbd_interval { |
| 9 | struct rb_node rb; | 9 | struct rb_node rb; |
| 10 | sector_t sector; /* start sector of the interval */ | 10 | sector_t sector; /* start sector of the interval */ |
| 11 | unsigned int size; /* size in bytes */ | ||
| 12 | sector_t end; /* highest interval end in subtree */ | 11 | sector_t end; /* highest interval end in subtree */ |
| 12 | unsigned int size; /* size in bytes */ | ||
| 13 | unsigned int local:1 /* local or remote request? */; | 13 | unsigned int local:1 /* local or remote request? */; |
| 14 | unsigned int waiting:1; /* someone is waiting for completion */ | 14 | unsigned int waiting:1; /* someone is waiting for completion */ |
| 15 | unsigned int completed:1; /* this has been completed already; | 15 | unsigned int completed:1; /* this has been completed already; |
| 16 | * ignore for conflict detection */ | 16 | * ignore for conflict detection */ |
| 17 | |||
| 18 | /* to resume a partially successful drbd_al_begin_io_nonblock(); */ | ||
| 19 | unsigned int partially_in_al_next_enr; | ||
| 17 | }; | 20 | }; |
| 18 | 21 | ||
| 19 | static inline void drbd_clear_interval(struct drbd_interval *i) | 22 | static inline void drbd_clear_interval(struct drbd_interval *i) |
| @@ -32,6 +32,7 @@ · 31 unmodified lines | |||
| 32 | #include <linux/memcontrol.h> | 32 | #include <linux/memcontrol.h> |
| 33 | #include <linux/mm_inline.h> | 33 | #include <linux/mm_inline.h> |
| 34 | #include <linux/slab.h> | 34 | #include <linux/slab.h> |
| 35 | #include <linux/string.h> | ||
| 35 | #include <linux/random.h> | 36 | #include <linux/random.h> |
| 36 | #include <linux/reboot.h> | 37 | #include <linux/reboot.h> |
| 37 | #include <linux/notifier.h> | 38 | #include <linux/notifier.h> |
| @@ -732,9 +733,9 @@ int drbd_send_sync_param(struct drbd_peer_device *peer_device) · 694 unmodified lines | |||
| 732 | } | 733 | } |
| 733 | 734 | ||
| 734 | if (apv >= 88) | 735 | if (apv >= 88) |
| 735 | strcpy(p->verify_alg, nc->verify_alg); | 736 | strscpy(p->verify_alg, nc->verify_alg); |
| 736 | if (apv >= 89) | 737 | if (apv >= 89) |
| 737 | strcpy(p->csums_alg, nc->csums_alg); | 738 | strscpy(p->csums_alg, nc->csums_alg); |
| 738 | rcu_read_unlock(); | 739 | rcu_read_unlock(); |
| 739 | 740 | ||
| 740 | return drbd_send_command(peer_device, sock, cmd, size, NULL, 0); | 741 | return drbd_send_command(peer_device, sock, cmd, size, NULL, 0); |
| @@ -745,6 +746,7 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 4 unmodified lines | |||
| 745 | struct drbd_socket *sock; | 746 | struct drbd_socket *sock; |
| 746 | struct p_protocol *p; | 747 | struct p_protocol *p; |
| 747 | struct net_conf *nc; | 748 | struct net_conf *nc; |
| 749 | size_t integrity_alg_len; | ||
| 748 | int size, cf; | 750 | int size, cf; |
| 749 | 751 | ||
| 750 | sock = &connection->data; | 752 | sock = &connection->data; |
| @@ -762,8 +764,10 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 11 unmodified lines | |||
| 762 | } | 764 | } |
| 763 | 765 | ||
| 764 | size = sizeof(*p); | 766 | size = sizeof(*p); |
| 765 | if (connection->agreed_pro_version >= 87) | 767 | if (connection->agreed_pro_version >= 87) { |
| 766 | size += strlen(nc->integrity_alg) + 1; | 768 | integrity_alg_len = strlen(nc->integrity_alg) + 1; |
| 769 | size += integrity_alg_len; | ||
| 770 | } | ||
| 767 | 771 | ||
| 768 | p->protocol = cpu_to_be32(nc->wire_protocol); | 772 | p->protocol = cpu_to_be32(nc->wire_protocol); |
| 769 | p->after_sb_0p = cpu_to_be32(nc->after_sb_0p); | 773 | p->after_sb_0p = cpu_to_be32(nc->after_sb_0p); |
| @@ -778,7 +782,7 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 8 unmodified lines | |||
| 778 | p->conn_flags = cpu_to_be32(cf); | 782 | p->conn_flags = cpu_to_be32(cf); |
| 779 | 783 | ||
| 780 | if (connection->agreed_pro_version >= 87) | 784 | if (connection->agreed_pro_version >= 87) |
| 781 | strcpy(p->integrity_alg, nc->integrity_alg); | 785 | strscpy(p->integrity_alg, nc->integrity_alg, integrity_alg_len); |
| 782 | rcu_read_unlock(); | 786 | rcu_read_unlock(); |
| 783 | 787 | ||
| 784 | return __conn_send_command(connection, sock, cmd, size, NULL, 0); | 788 | return __conn_send_command(connection, sock, cmd, size, NULL, 0); |
| @@ -3801,14 +3801,14 @@ static int receive_SyncParam(struct drbd_connection *connection, struct packet_i · 3800 unmodified lines | |||
| 3801 | *new_net_conf = *old_net_conf; | 3801 | *new_net_conf = *old_net_conf; |
| 3802 | 3802 | ||
| 3803 | if (verify_tfm) { | 3803 | if (verify_tfm) { |
| 3804 | strcpy(new_net_conf->verify_alg, p->verify_alg); | 3804 | strscpy(new_net_conf->verify_alg, p->verify_alg); |
| 3805 | new_net_conf->verify_alg_len = strlen(p->verify_alg) + 1; | 3805 | new_net_conf->verify_alg_len = strlen(p->verify_alg) + 1; |
| 3806 | crypto_free_shash(peer_device->connection->verify_tfm); | 3806 | crypto_free_shash(peer_device->connection->verify_tfm); |
| 3807 | peer_device->connection->verify_tfm = verify_tfm; | 3807 | peer_device->connection->verify_tfm = verify_tfm; |
| 3808 | drbd_info(device, "using verify-alg: \"%s\"\n", p->verify_alg); | 3808 | drbd_info(device, "using verify-alg: \"%s\"\n", p->verify_alg); |
| 3809 | } | 3809 | } |
| 3810 | if (csums_tfm) { | 3810 | if (csums_tfm) { |
| 3811 | strcpy(new_net_conf->csums_alg, p->csums_alg); | 3811 | strscpy(new_net_conf->csums_alg, p->csums_alg); |
| 3812 | new_net_conf->csums_alg_len = strlen(p->csums_alg) + 1; | 3812 | new_net_conf->csums_alg_len = strlen(p->csums_alg) + 1; |
| 3813 | crypto_free_shash(peer_device->connection->csums_tfm); | 3813 | crypto_free_shash(peer_device->connection->csums_tfm); |
| 3814 | peer_device->connection->csums_tfm = csums_tfm; | 3814 | peer_device->connection->csums_tfm = csums_tfm; |
| @@ -621,7 +621,8 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what, · 620 unmodified lines | |||
| 621 | break; | 621 | break; |
| 622 | 622 | ||
| 623 | case READ_COMPLETED_WITH_ERROR: | 623 | case READ_COMPLETED_WITH_ERROR: |
| 624 | drbd_set_out_of_sync(peer_device, req->i.sector, req->i.size); | 624 | drbd_set_out_of_sync(first_peer_device(device), |
| 625 | req->i.sector, req->i.size); | ||
| 625 | drbd_report_io_error(device, req); | 626 | drbd_report_io_error(device, req); |
| 626 | __drbd_chk_io_error(device, DRBD_READ_ERROR); | 627 | __drbd_chk_io_error(device, DRBD_READ_ERROR); |
| 627 | fallthrough; | 628 | fallthrough; |
| @@ -4443,7 +4443,9 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, · 4442 unmodified lines | |||
| 4443 | 4443 | ||
| 4444 | /* Skip partition scan if disabled by user */ | 4444 | /* Skip partition scan if disabled by user */ |
| 4445 | if (ub->dev_info.flags & UBLK_F_NO_AUTO_PART_SCAN) { | 4445 | if (ub->dev_info.flags & UBLK_F_NO_AUTO_PART_SCAN) { |
| 4446 | clear_bit(GD_SUPPRESS_PART_SCAN, &disk->state); | 4446 | /* Not clear for unprivileged daemons, see comment above */ |
| 4447 | if (!ub->unprivileged_daemons) | ||
| 4448 | clear_bit(GD_SUPPRESS_PART_SCAN, &disk->state); | ||
| 4447 | } else { | 4449 | } else { |
| 4448 | /* Schedule async partition scan for trusted daemons */ | 4450 | /* Schedule async partition scan for trusted daemons */ |
| 4449 | if (!ub->unprivileged_daemons) | 4451 | if (!ub->unprivileged_daemons) |
| @@ -5006,15 +5008,22 @@ static int ublk_ctrl_get_features(const struct ublksrv_ctrl_cmd *header) · 556 unmodified lines | |||
| 5006 | return 0; | 5008 | return 0; |
| 5007 | } | 5009 | } |
| 5008 | 5010 | ||
| 5009 | static void ublk_ctrl_set_size(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) | 5011 | static int ublk_ctrl_set_size(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) |
| 5010 | { | 5012 | { |
| 5011 | struct ublk_param_basic *p = &ub->params.basic; | 5013 | struct ublk_param_basic *p = &ub->params.basic; |
| 5012 | u64 new_size = header->data[0]; | 5014 | u64 new_size = header->data[0]; |
| 5015 | int ret = 0; | ||
| 5013 | 5016 | ||
| 5014 | mutex_lock(&ub->mutex); | 5017 | mutex_lock(&ub->mutex); |
| 5018 | if (!ub->ub_disk) { | ||
| 5019 | ret = -ENODEV; | ||
| 5020 | goto out; | ||
| 5021 | } | ||
| 5015 | p->dev_sectors = new_size; | 5022 | p->dev_sectors = new_size; |
| 5016 | set_capacity_and_notify(ub->ub_disk, p->dev_sectors); | 5023 | set_capacity_and_notify(ub->ub_disk, p->dev_sectors); |
| 5024 | out: | ||
| 5017 | mutex_unlock(&ub->mutex); | 5025 | mutex_unlock(&ub->mutex); |
| 5026 | return ret; | ||
| 5018 | } | 5027 | } |
| 5019 | 5028 | ||
| 5020 | struct count_busy { | 5029 | struct count_busy { |
| @@ -5335,8 +5344,7 @@ static int ublk_ctrl_uring_cmd(struct io_uring_cmd *cmd, · 314 unmodified lines | |||
| 5335 | ret = ublk_ctrl_end_recovery(ub, &header); | 5344 | ret = ublk_ctrl_end_recovery(ub, &header); |
| 5336 | break; | 5345 | break; |
| 5337 | case UBLK_CMD_UPDATE_SIZE: | 5346 | case UBLK_CMD_UPDATE_SIZE: |
| 5338 | ublk_ctrl_set_size(ub, &header); | 5347 | ret = ublk_ctrl_set_size(ub, &header); |
| 5339 | ret = 0; | ||
| 5340 | break; | 5348 | break; |
| 5341 | case UBLK_CMD_QUIESCE_DEV: | 5349 | case UBLK_CMD_QUIESCE_DEV: |
| 5342 | ret = ublk_ctrl_quiesce_dev(ub, &header); | 5350 | ret = ublk_ctrl_quiesce_dev(ub, &header); |
| @@ -542,6 +542,21 @@ out: · 541 unmodified lines | |||
| 542 | zloop_put_cmd(cmd); | 542 | zloop_put_cmd(cmd); |
| 543 | } | 543 | } |
| 544 | 544 | ||
| 545 | /* | ||
| 546 | * Sync the entire FS containing the zone files instead of walking all files. | ||
| 547 | */ | ||
| 548 | static int zloop_flush(struct zloop_device *zlo) | ||
| 549 | { | ||
| 550 | struct super_block *sb = file_inode(zlo->data_dir)->i_sb; | ||
| 551 | int ret; | ||
| 552 | |||
| 553 | down_read(&sb->s_umount); | ||
| 554 | ret = sync_filesystem(sb); | ||
| 555 | up_read(&sb->s_umount); | ||
| 556 | |||
| 557 | return ret; | ||
| 558 | } | ||
| 559 | |||
| 545 | static void zloop_handle_cmd(struct zloop_cmd *cmd) | 560 | static void zloop_handle_cmd(struct zloop_cmd *cmd) |
| 546 | { | 561 | { |
| 547 | struct request *rq = blk_mq_rq_from_pdu(cmd); | 562 | struct request *rq = blk_mq_rq_from_pdu(cmd); |
| @@ -562,11 +577,7 @@ static void zloop_handle_cmd(struct zloop_cmd *cmd) · 14 unmodified lines | |||
| 562 | zloop_rw(cmd); | 577 | zloop_rw(cmd); |
| 563 | return; | 578 | return; |
| 564 | case REQ_OP_FLUSH: | 579 | case REQ_OP_FLUSH: |
| 565 | /* | 580 | cmd->ret = zloop_flush(zlo); |
| 566 | * Sync the entire FS containing the zone files instead of | ||
| 567 | * walking all files | ||
| 568 | */ | ||
| 569 | cmd->ret = sync_filesystem(file_inode(zlo->data_dir)->i_sb); | ||
| 570 | break; | 581 | break; |
| 571 | case REQ_OP_ZONE_RESET: | 582 | case REQ_OP_ZONE_RESET: |
| 572 | cmd->ret = zloop_reset_zone(zlo, rq_zone_no(rq)); | 583 | cmd->ret = zloop_reset_zone(zlo, rq_zone_no(rq)); |
| @@ -981,7 +992,8 @@ static int zloop_ctl_add(struct zloop_options *opts) · 408 unmodified lines | |||
| 981 | struct queue_limits lim = { | 992 | struct queue_limits lim = { |
| 982 | .max_hw_sectors = SZ_1M >> SECTOR_SHIFT, | 993 | .max_hw_sectors = SZ_1M >> SECTOR_SHIFT, |
| 983 | .chunk_sectors = opts->zone_size, | 994 | .chunk_sectors = opts->zone_size, |
| 984 | .features = BLK_FEAT_ZONED, | 995 | .features = BLK_FEAT_ZONED | BLK_FEAT_WRITE_CACHE, |
| 996 | |||
| 985 | }; | 997 | }; |
| 986 | unsigned int nr_zones, i, j; | 998 | unsigned int nr_zones, i, j; |
| 987 | struct zloop_device *zlo; | 999 | struct zloop_device *zlo; |
| @@ -1162,7 +1174,12 @@ static int zloop_ctl_remove(struct zloop_options *opts) · 174 unmodified lines | |||
| 1162 | int ret; | 1174 | int ret; |
| 1163 | 1175 | ||
| 1164 | if (!(opts->mask & ZLOOP_OPT_ID)) { | 1176 | if (!(opts->mask & ZLOOP_OPT_ID)) { |
| 1165 | pr_err("No ID specified\n"); | 1177 | pr_err("No ID specified for remove\n"); |
| 1178 | return -EINVAL; | ||
| 1179 | } | ||
| 1180 | |||
| 1181 | if (opts->mask & ~ZLOOP_OPT_ID) { | ||
| 1182 | pr_err("Invalid option specified for remove\n"); | ||
| 1166 | return -EINVAL; | 1183 | return -EINVAL; |
| 1167 | } | 1184 | } |
| 1168 | 1185 | ||
| @@ -549,7 +549,7 @@ static ssize_t bd_stat_show(struct device *dev, struct device_attribute *attr, · 548 unmodified lines | |||
| 549 | return ret; | 549 | return ret; |
| 550 | } | 550 | } |
| 551 | 551 | ||
| 552 | static ssize_t writeback_compressed_store(struct device *dev, | 552 | static ssize_t compressed_writeback_store(struct device *dev, |
| 553 | struct device_attribute *attr, | 553 | struct device_attribute *attr, |
| 554 | const char *buf, size_t len) | 554 | const char *buf, size_t len) |
| 555 | { | 555 | { |
| @@ -564,12 +564,12 @@ static ssize_t writeback_compressed_store(struct device *dev, · 8 unmodified lines | |||
| 564 | return -EBUSY; | 564 | return -EBUSY; |
| 565 | } | 565 | } |
| 566 | 566 | ||
| 567 | zram->wb_compressed = val; | 567 | zram->compressed_wb = val; |
| 568 | 568 | ||
| 569 | return len; | 569 | return len; |
| 570 | } | 570 | } |
| 571 | 571 | ||
| 572 | static ssize_t writeback_compressed_show(struct device *dev, | 572 | static ssize_t compressed_writeback_show(struct device *dev, |
| 573 | struct device_attribute *attr, | 573 | struct device_attribute *attr, |
| 574 | char *buf) | 574 | char *buf) |
| 575 | { | 575 | { |
| @@ -577,7 +577,7 @@ static ssize_t writeback_compressed_show(struct device *dev, · 1 unmodified line | |||
| 577 | struct zram *zram = dev_to_zram(dev); | 577 | struct zram *zram = dev_to_zram(dev); |
| 578 | 578 | ||
| 579 | guard(rwsem_read)(&zram->dev_lock); | 579 | guard(rwsem_read)(&zram->dev_lock); |
| 580 | val = zram->wb_compressed; | 580 | val = zram->compressed_wb; |
| 581 | 581 | ||
| 582 | return sysfs_emit(buf, "%d\n", val); | 582 | return sysfs_emit(buf, "%d\n", val); |
| 583 | } | 583 | } |
| @@ -946,7 +946,7 @@ static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) · 362 unmodified lines | |||
| 946 | goto out; | 946 | goto out; |
| 947 | } | 947 | } |
| 948 | 948 | ||
| 949 | if (zram->wb_compressed) { | 949 | if (zram->compressed_wb) { |
| 950 | /* | 950 | /* |
| 951 | * ZRAM_WB slots get freed, we need to preserve data required | 951 | * ZRAM_WB slots get freed, we need to preserve data required |
| 952 | * for read decompression. | 952 | * for read decompression. |
| @@ -960,7 +960,7 @@ static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) · 7 unmodified lines | |||
| 960 | set_slot_flag(zram, index, ZRAM_WB); | 960 | set_slot_flag(zram, index, ZRAM_WB); |
| 961 | set_slot_handle(zram, index, req->blk_idx); | 961 | set_slot_handle(zram, index, req->blk_idx); |
| 962 | 962 | ||
| 963 | if (zram->wb_compressed) { | 963 | if (zram->compressed_wb) { |
| 964 | if (huge) | 964 | if (huge) |
| 965 | set_slot_flag(zram, index, ZRAM_HUGE); | 965 | set_slot_flag(zram, index, ZRAM_HUGE); |
| 966 | set_slot_size(zram, index, size); | 966 | set_slot_size(zram, index, size); |
| @@ -1100,7 +1100,7 @@ static int zram_writeback_slots(struct zram *zram, · 133 unmodified lines | |||
| 1100 | */ | 1100 | */ |
| 1101 | if (!test_slot_flag(zram, index, ZRAM_PP_SLOT)) | 1101 | if (!test_slot_flag(zram, index, ZRAM_PP_SLOT)) |
| 1102 | goto next; | 1102 | goto next; |
| 1103 | if (zram->wb_compressed) | 1103 | if (zram->compressed_wb) |
| 1104 | err = read_from_zspool_raw(zram, req->page, index); | 1104 | err = read_from_zspool_raw(zram, req->page, index); |
| 1105 | else | 1105 | else |
| 1106 | err = read_from_zspool(zram, req->page, index); | 1106 | err = read_from_zspool(zram, req->page, index); |
| @@ -1429,7 +1429,7 @@ static void zram_async_read_endio(struct bio *bio) · 322 unmodified lines | |||
| 1429 | * | 1429 | * |
| 1430 | * Keep the existing behavior for now. | 1430 | * Keep the existing behavior for now. |
| 1431 | */ | 1431 | */ |
| 1432 | if (zram->wb_compressed == false) { | 1432 | if (zram->compressed_wb == false) { |
| 1433 | /* No decompression needed, complete the parent IO */ | 1433 | /* No decompression needed, complete the parent IO */ |
| 1434 | bio_endio(req->parent); | 1434 | bio_endio(req->parent); |
| 1435 | bio_put(bio); | 1435 | bio_put(bio); |
| @@ -1508,7 +1508,7 @@ static int read_from_bdev_sync(struct zram *zram, struct page *page, u32 index, · 72 unmodified lines | |||
| 1508 | flush_work(&req.work); | 1508 | flush_work(&req.work); |
| 1509 | destroy_work_on_stack(&req.work); | 1509 | destroy_work_on_stack(&req.work); |
| 1510 | 1510 | ||
| 1511 | if (req.error || zram->wb_compressed == false) | 1511 | if (req.error || zram->compressed_wb == false) |
| 1512 | return req.error; | 1512 | return req.error; |
| 1513 | 1513 | ||
| 1514 | return decompress_bdev_page(zram, page, index); | 1514 | return decompress_bdev_page(zram, page, index); |
| @@ -3007,7 +3007,7 @@ static DEVICE_ATTR_WO(writeback); · 1492 unmodified lines | |||
| 3007 | static DEVICE_ATTR_RW(writeback_limit); | 3007 | static DEVICE_ATTR_RW(writeback_limit); |
| 3008 | static DEVICE_ATTR_RW(writeback_limit_enable); | 3008 | static DEVICE_ATTR_RW(writeback_limit_enable); |
| 3009 | static DEVICE_ATTR_RW(writeback_batch_size); | 3009 | static DEVICE_ATTR_RW(writeback_batch_size); |
| 3010 | static DEVICE_ATTR_RW(writeback_compressed); | 3010 | static DEVICE_ATTR_RW(compressed_writeback); |
| 3011 | #endif | 3011 | #endif |
| 3012 | #ifdef CONFIG_ZRAM_MULTI_COMP | 3012 | #ifdef CONFIG_ZRAM_MULTI_COMP |
| 3013 | static DEVICE_ATTR_RW(recomp_algorithm); | 3013 | static DEVICE_ATTR_RW(recomp_algorithm); |
| @@ -3031,7 +3031,7 @@ static struct attribute *zram_disk_attrs[] = { · 17 unmodified lines | |||
| 3031 | &dev_attr_writeback_limit.attr, | 3031 | &dev_attr_writeback_limit.attr, |
| 3032 | &dev_attr_writeback_limit_enable.attr, | 3032 | &dev_attr_writeback_limit_enable.attr, |
| 3033 | &dev_attr_writeback_batch_size.attr, | 3033 | &dev_attr_writeback_batch_size.attr, |
| 3034 | &dev_attr_writeback_compressed.attr, | 3034 | &dev_attr_compressed_writeback.attr, |
| 3035 | #endif | 3035 | #endif |
| 3036 | &dev_attr_io_stat.attr, | 3036 | &dev_attr_io_stat.attr, |
| 3037 | &dev_attr_mm_stat.attr, | 3037 | &dev_attr_mm_stat.attr, |
| @@ -3091,7 +3091,7 @@ static int zram_add(void) · 53 unmodified lines | |||
| 3091 | init_rwsem(&zram->dev_lock); | 3091 | init_rwsem(&zram->dev_lock); |
| 3092 | #ifdef CONFIG_ZRAM_WRITEBACK | 3092 | #ifdef CONFIG_ZRAM_WRITEBACK |
| 3093 | zram->wb_batch_size = 32; | 3093 | zram->wb_batch_size = 32; |
| 3094 | zram->wb_compressed = false; | 3094 | zram->compressed_wb = false; |
| 3095 | #endif | 3095 | #endif |
| 3096 | 3096 | ||
| 3097 | /* gendisk structure */ | 3097 | /* gendisk structure */ |
| @@ -133,7 +133,7 @@ struct zram { · 132 unmodified lines | |||
| 133 | #ifdef CONFIG_ZRAM_WRITEBACK | 133 | #ifdef CONFIG_ZRAM_WRITEBACK |
| 134 | struct file *backing_dev; | 134 | struct file *backing_dev; |
| 135 | bool wb_limit_enable; | 135 | bool wb_limit_enable; |
| 136 | bool wb_compressed; | 136 | bool compressed_wb; |
| 137 | u32 wb_batch_size; | 137 | u32 wb_batch_size; |
| 138 | u64 bd_wb_limit; | 138 | u64 bd_wb_limit; |
| 139 | struct block_device *bdev; | 139 | struct block_device *bdev; |
| @@ -2046,19 +2046,23 @@ retry: · 2045 unmodified lines | |||
| 2046 | } | 2046 | } |
| 2047 | 2047 | ||
| 2048 | out: | 2048 | out: |
| 2049 | if (ret && retries < MAX_INIT_RETRIES) { | 2049 | if (ret) { |
| 2050 | bt_dev_warn(hdev, "Retry BT power ON:%d", retries); | ||
| 2051 | qca_power_shutdown(hu); | 2050 | qca_power_shutdown(hu); |
| 2052 | if (hu->serdev) { | 2051 | |
| 2053 | serdev_device_close(hu->serdev); | 2052 | if (retries < MAX_INIT_RETRIES) { |
| 2054 | ret = serdev_device_open(hu->serdev); | 2053 | bt_dev_warn(hdev, "Retry BT power ON:%d", retries); |
| 2055 | if (ret) { | 2054 | if (hu->serdev) { |
| 2056 | bt_dev_err(hdev, "failed to open port"); | 2055 | serdev_device_close(hu->serdev); |
| 2057 | return ret; | 2056 | ret = serdev_device_open(hu->serdev); |
| 2057 | if (ret) { | ||
| 2058 | bt_dev_err(hdev, "failed to open port"); | ||
| 2059 | return ret; | ||
| 2060 | } | ||
| 2058 | } | 2061 | } |
| 2062 | retries++; | ||
| 2063 | goto retry; | ||
| 2059 | } | 2064 | } |
| 2060 | retries++; | 2065 | return ret; |
| 2061 | goto retry; | ||
| 2062 | } | 2066 | } |
| 2063 | 2067 | ||
| 2064 | /* Setup bdaddr */ | 2068 | /* Setup bdaddr */ |
| @@ -202,11 +202,16 @@ static int ipmi_ipmb_slave_cb(struct i2c_client *client, · 201 unmodified lines | |||
| 202 | break; | 202 | break; |
| 203 | 203 | ||
| 204 | case I2C_SLAVE_READ_REQUESTED: | 204 | case I2C_SLAVE_READ_REQUESTED: |
| 205 | *val = 0xff; | ||
| 206 | ipmi_ipmb_check_msg_done(iidev); | ||
| 207 | break; | ||
| 208 | |||
| 205 | case I2C_SLAVE_STOP: | 209 | case I2C_SLAVE_STOP: |
| 206 | ipmi_ipmb_check_msg_done(iidev); | 210 | ipmi_ipmb_check_msg_done(iidev); |
| 207 | break; | 211 | break; |
| 208 | 212 | ||
| 209 | case I2C_SLAVE_READ_PROCESSED: | 213 | case I2C_SLAVE_READ_PROCESSED: |
| 214 | *val = 0xff; | ||
| 210 | break; | 215 | break; |
| 211 | } | 216 | } |
| 212 | 217 | ||
| @@ -602,6 +602,22 @@ static int __ipmi_bmc_register(struct ipmi_smi *intf, · 601 unmodified lines | |||
| 602 | static int __scan_channels(struct ipmi_smi *intf, | 602 | static int __scan_channels(struct ipmi_smi *intf, |
| 603 | struct ipmi_device_id *id, bool rescan); | 603 | struct ipmi_device_id *id, bool rescan); |
| 604 | 604 | ||
| 605 | static void ipmi_lock_xmit_msgs(struct ipmi_smi *intf, int run_to_completion, | ||
| 606 | unsigned long *flags) | ||
| 607 | { | ||
| 608 | if (run_to_completion) | ||
| 609 | return; | ||
| 610 | spin_lock_irqsave(&intf->xmit_msgs_lock, *flags); | ||
| 611 | } | ||
| 612 | |||
| 613 | static void ipmi_unlock_xmit_msgs(struct ipmi_smi *intf, int run_to_completion, | ||
| 614 | unsigned long *flags) | ||
| 615 | { | ||
| 616 | if (run_to_completion) | ||
| 617 | return; | ||
| 618 | spin_unlock_irqrestore(&intf->xmit_msgs_lock, *flags); | ||
| 619 | } | ||
| 620 | |||
| 605 | static void free_ipmi_user(struct kref *ref) | 621 | static void free_ipmi_user(struct kref *ref) |
| 606 | { | 622 | { |
| 607 | struct ipmi_user *user = container_of(ref, struct ipmi_user, refcount); | 623 | struct ipmi_user *user = container_of(ref, struct ipmi_user, refcount); |
| @@ -1869,21 +1885,32 @@ static struct ipmi_smi_msg *smi_add_send_msg(struct ipmi_smi *intf, · 1261 unmodified lines | |||
| 1869 | return smi_msg; | 1885 | return smi_msg; |
| 1870 | } | 1886 | } |
| 1871 | 1887 | ||
| 1872 | static void smi_send(struct ipmi_smi *intf, | 1888 | static int smi_send(struct ipmi_smi *intf, |
| 1873 | const struct ipmi_smi_handlers *handlers, | 1889 | const struct ipmi_smi_handlers *handlers, |
| 1874 | struct ipmi_smi_msg *smi_msg, int priority) | 1890 | struct ipmi_smi_msg *smi_msg, int priority) |
| 1875 | { | 1891 | { |
| 1876 | int run_to_completion = READ_ONCE(intf->run_to_completion); | 1892 | int run_to_completion = READ_ONCE(intf->run_to_completion); |
| 1877 | unsigned long flags = 0; | 1893 | unsigned long flags = 0; |
| 1894 | int rv = 0; | ||
| 1878 | 1895 | ||
| 1879 | if (!run_to_completion) | 1896 | ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); |
| 1880 | spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | ||
| 1881 | smi_msg = smi_add_send_msg(intf, smi_msg, priority); | 1897 | smi_msg = smi_add_send_msg(intf, smi_msg, priority); |
| 1882 | if (!run_to_completion) | 1898 | ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); |
| 1883 | spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | ||
| 1884 | 1899 | ||
| 1885 | if (smi_msg) | 1900 | if (smi_msg) { |
| 1886 | handlers->sender(intf->send_info, smi_msg); | 1901 | rv = handlers->sender(intf->send_info, smi_msg); |
| 1902 | if (rv) { | ||
| 1903 | ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | ||
| 1904 | intf->curr_msg = NULL; | ||
| 1905 | ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | ||
| 1906 | /* | ||
| 1907 | * Something may have been added to the transmit | ||
| 1908 | * queue, so schedule a check for that. | ||
| 1909 | */ | ||
| 1910 | queue_work(system_wq, &intf->smi_work); | ||
| 1911 | } | ||
| 1912 | } | ||
| 1913 | return rv; | ||
| 1887 | } | 1914 | } |
| 1888 | 1915 | ||
| 1889 | static bool is_maintenance_mode_cmd(struct kernel_ipmi_msg *msg) | 1916 | static bool is_maintenance_mode_cmd(struct kernel_ipmi_msg *msg) |
| @@ -2296,6 +2323,7 @@ static int i_ipmi_request(struct ipmi_user *user, · 406 unmodified lines | |||
| 2296 | struct ipmi_recv_msg *recv_msg; | 2323 | struct ipmi_recv_msg *recv_msg; |
| 2297 | int run_to_completion = READ_ONCE(intf->run_to_completion); | 2324 | int run_to_completion = READ_ONCE(intf->run_to_completion); |
| 2298 | int rv = 0; | 2325 | int rv = 0; |
| 2326 | bool in_seq_table = false; | ||
| 2299 | 2327 | ||
| 2300 | if (supplied_recv) { | 2328 | if (supplied_recv) { |
| 2301 | recv_msg = supplied_recv; | 2329 | recv_msg = supplied_recv; |
| @@ -2349,33 +2377,50 @@ static int i_ipmi_request(struct ipmi_user *user, · 47 unmodified lines | |||
| 2349 | rv = i_ipmi_req_ipmb(intf, addr, msgid, msg, smi_msg, recv_msg, | 2377 | rv = i_ipmi_req_ipmb(intf, addr, msgid, msg, smi_msg, recv_msg, |
| 2350 | source_address, source_lun, | 2378 | source_address, source_lun, |
| 2351 | retries, retry_time_ms); | 2379 | retries, retry_time_ms); |
| 2380 | in_seq_table = true; | ||
| 2352 | } else if (is_ipmb_direct_addr(addr)) { | 2381 | } else if (is_ipmb_direct_addr(addr)) { |
| 2353 | rv = i_ipmi_req_ipmb_direct(intf, addr, msgid, msg, smi_msg, | 2382 | rv = i_ipmi_req_ipmb_direct(intf, addr, msgid, msg, smi_msg, |
| 2354 | recv_msg, source_lun); | 2383 | recv_msg, source_lun); |
| 2355 | } else if (is_lan_addr(addr)) { | 2384 | } else if (is_lan_addr(addr)) { |
| 2356 | rv = i_ipmi_req_lan(intf, addr, msgid, msg, smi_msg, recv_msg, | 2385 | rv = i_ipmi_req_lan(intf, addr, msgid, msg, smi_msg, recv_msg, |
| 2357 | source_lun, retries, retry_time_ms); | 2386 | source_lun, retries, retry_time_ms); |
| 2387 | in_seq_table = true; | ||
| 2358 | } else { | 2388 | } else { |
| 2359 | /* Unknown address type. */ | 2389 | /* Unknown address type. */ |
| 2360 | ipmi_inc_stat(intf, sent_invalid_commands); | 2390 | ipmi_inc_stat(intf, sent_invalid_commands); |
| 2361 | rv = -EINVAL; | 2391 | rv = -EINVAL; |
| 2362 | } | 2392 | } |
| 2363 | 2393 | ||
| 2364 | if (rv) { | 2394 | if (!rv) { |
| 2365 | out_err: | ||
| 2366 | if (!supplied_smi) | ||
| 2367 | ipmi_free_smi_msg(smi_msg); | ||
| 2368 | if (!supplied_recv) | ||
| 2369 | ipmi_free_recv_msg(recv_msg); | ||
| 2370 | } else { | ||
| 2371 | dev_dbg(intf->si_dev, "Send: %*ph\n", | 2395 | dev_dbg(intf->si_dev, "Send: %*ph\n", |
| 2372 | smi_msg->data_size, smi_msg->data); | 2396 | smi_msg->data_size, smi_msg->data); |
| 2373 | 2397 | ||
| 2374 | smi_send(intf, intf->handlers, smi_msg, priority); | 2398 | rv = smi_send(intf, intf->handlers, smi_msg, priority); |
| 2399 | if (rv != IPMI_CC_NO_ERROR) | ||
| 2400 | /* smi_send() returns an IPMI err, return a Linux one. */ | ||
| 2401 | rv = -EIO; | ||
| 2402 | if (rv && in_seq_table) { | ||
| 2403 | /* | ||
| 2404 | * If it's in the sequence table, it will be | ||
| 2405 | * retried later, so ignore errors. | ||
| 2406 | */ | ||
| 2407 | rv = 0; | ||
| 2408 | /* But we need to fix the timeout. */ | ||
| 2409 | intf_start_seq_timer(intf, smi_msg->msgid); | ||
| 2410 | ipmi_free_smi_msg(smi_msg); | ||
| 2411 | smi_msg = NULL; | ||
| 2412 | } | ||
| 2375 | } | 2413 | } |
| 2414 | out_err: | ||
| 2376 | if (!run_to_completion) | 2415 | if (!run_to_completion) |
| 2377 | mutex_unlock(&intf->users_mutex); | 2416 | mutex_unlock(&intf->users_mutex); |
| 2378 | 2417 | ||
| 2418 | if (rv) { | ||
| 2419 | if (!supplied_smi) | ||
| 2420 | ipmi_free_smi_msg(smi_msg); | ||
| 2421 | if (!supplied_recv) | ||
| 2422 | ipmi_free_recv_msg(recv_msg); | ||
| 2423 | } | ||
| 2379 | return rv; | 2424 | return rv; |
| 2380 | } | 2425 | } |
| 2381 | 2426 | ||
| @@ -3949,12 +3994,12 @@ static int handle_ipmb_get_msg_cmd(struct ipmi_smi *intf, · 1567 unmodified lines | |||
| 3949 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", | 3994 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", |
| 3950 | msg->data_size, msg->data); | 3995 | msg->data_size, msg->data); |
| 3951 | 3996 | ||
| 3952 | smi_send(intf, intf->handlers, msg, 0); | 3997 | if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) |
| 3953 | /* | 3998 | /* |
| 3954 | * We used the message, so return the value that | 3999 | * We used the message, so return the value that |
| 3955 | * causes it to not be freed or queued. | 4000 | * causes it to not be freed or queued. |
| 3956 | */ | 4001 | */ |
| 3957 | rv = -1; | 4002 | rv = -1; |
| 3958 | } else if (!IS_ERR(recv_msg)) { | 4003 | } else if (!IS_ERR(recv_msg)) { |
| 3959 | /* Extract the source address from the data. */ | 4004 | /* Extract the source address from the data. */ |
| 3960 | ipmb_addr = (struct ipmi_ipmb_addr *) &recv_msg->addr; | 4005 | ipmb_addr = (struct ipmi_ipmb_addr *) &recv_msg->addr; |
| @@ -4028,12 +4073,12 @@ static int handle_ipmb_direct_rcv_cmd(struct ipmi_smi *intf, · 67 unmodified lines | |||
| 4028 | msg->data[4] = IPMI_INVALID_CMD_COMPLETION_CODE; | 4073 | msg->data[4] = IPMI_INVALID_CMD_COMPLETION_CODE; |
| 4029 | msg->data_size = 5; | 4074 | msg->data_size = 5; |
| 4030 | 4075 | ||
| 4031 | smi_send(intf, intf->handlers, msg, 0); | 4076 | if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) |
| 4032 | /* | 4077 | /* |
| 4033 | * We used the message, so return the value that | 4078 | * We used the message, so return the value that |
| 4034 | * causes it to not be freed or queued. | 4079 | * causes it to not be freed or queued. |
| 4035 | */ | 4080 | */ |
| 4036 | rv = -1; | 4081 | rv = -1; |
| 4037 | } else if (!IS_ERR(recv_msg)) { | 4082 | } else if (!IS_ERR(recv_msg)) { |
| 4038 | /* Extract the source address from the data. */ | 4083 | /* Extract the source address from the data. */ |
| 4039 | daddr = (struct ipmi_ipmb_direct_addr *)&recv_msg->addr; | 4084 | daddr = (struct ipmi_ipmb_direct_addr *)&recv_msg->addr; |
| @@ -4173,7 +4218,7 @@ static int handle_lan_get_msg_cmd(struct ipmi_smi *intf, · 133 unmodified lines | |||
| 4173 | struct ipmi_smi_msg *msg) | 4218 | struct ipmi_smi_msg *msg) |
| 4174 | { | 4219 | { |
| 4175 | struct cmd_rcvr *rcvr; | 4220 | struct cmd_rcvr *rcvr; |
| 4176 | int rv = 0; | 4221 | int rv = 0; /* Free by default */ |
| 4177 | unsigned char netfn; | 4222 | unsigned char netfn; |
| 4178 | unsigned char cmd; | 4223 | unsigned char cmd; |
| 4179 | unsigned char chan; | 4224 | unsigned char chan; |
| @@ -4226,12 +4271,12 @@ static int handle_lan_get_msg_cmd(struct ipmi_smi *intf, · 46 unmodified lines | |||
| 4226 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", | 4271 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", |
| 4227 | msg->data_size, msg->data); | 4272 | msg->data_size, msg->data); |
| 4228 | 4273 | ||
| 4229 | smi_send(intf, intf->handlers, msg, 0); | 4274 | if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) |
| 4230 | /* | 4275 | /* |
| 4231 | * We used the message, so return the value that | 4276 | * We used the message, so return the value that |
| 4232 | * causes it to not be freed or queued. | 4277 | * causes it to not be freed or queued. |
| 4233 | */ | 4278 | */ |
| 4234 | rv = -1; | 4279 | rv = -1; |
| 4235 | } else if (!IS_ERR(recv_msg)) { | 4280 | } else if (!IS_ERR(recv_msg)) { |
| 4236 | /* Extract the source address from the data. */ | 4281 | /* Extract the source address from the data. */ |
| 4237 | lan_addr = (struct ipmi_lan_addr *) &recv_msg->addr; | 4282 | lan_addr = (struct ipmi_lan_addr *) &recv_msg->addr; |
| @@ -4824,8 +4869,7 @@ static void smi_work(struct work_struct *t) · 586 unmodified lines | |||
| 4824 | * message delivery. | 4869 | * message delivery. |
| 4825 | */ | 4870 | */ |
| 4826 | restart: | 4871 | restart: |
| 4827 | if (!run_to_completion) | 4872 | ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); |
| 4828 | spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | ||
| 4829 | if (intf->curr_msg == NULL && !intf->in_shutdown) { | 4873 | if (intf->curr_msg == NULL && !intf->in_shutdown) { |
| 4830 | struct list_head *entry = NULL; | 4874 | struct list_head *entry = NULL; |
| 4831 | 4875 | ||
| @@ -4841,8 +4885,7 @@ restart: · 9 unmodified lines | |||
| 4841 | intf->curr_msg = newmsg; | 4885 | intf->curr_msg = newmsg; |
| 4842 | } | 4886 | } |
| 4843 | } | 4887 | } |
| 4844 | if (!run_to_completion) | 4888 | ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); |
| 4845 | spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | ||
| 4846 | 4889 | ||
| 4847 | if (newmsg) { | 4890 | if (newmsg) { |
| 4848 | cc = intf->handlers->sender(intf->send_info, newmsg); | 4891 | cc = intf->handlers->sender(intf->send_info, newmsg); |
| @@ -4850,8 +4893,11 @@ restart: · 1 unmodified line | |||
| 4850 | if (newmsg->recv_msg) | 4893 | if (newmsg->recv_msg) |
| 4851 | deliver_err_response(intf, | 4894 | deliver_err_response(intf, |
| 4852 | newmsg->recv_msg, cc); | 4895 | newmsg->recv_msg, cc); |
| 4853 | else | 4896 | ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); |
| 4854 | ipmi_free_smi_msg(newmsg); | 4897 | intf->curr_msg = NULL; |
| 4898 | ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | ||
| 4899 | ipmi_free_smi_msg(newmsg); | ||
| 4900 | newmsg = NULL; | ||
| 4855 | goto restart; | 4901 | goto restart; |
| 4856 | } | 4902 | } |
| 4857 | } | 4903 | } |
| @@ -4919,16 +4965,14 @@ void ipmi_smi_msg_received(struct ipmi_smi *intf, · 61 unmodified lines | |||
| 4919 | spin_unlock_irqrestore(&intf->waiting_rcv_msgs_lock, | 4965 | spin_unlock_irqrestore(&intf->waiting_rcv_msgs_lock, |
| 4920 | flags); | 4966 | flags); |
| 4921 | 4967 | ||
| 4922 | if (!run_to_completion) | 4968 | ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); |
| 4923 | spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | ||
| 4924 | /* | 4969 | /* |
| 4925 | * We can get an asynchronous event or receive message in addition | 4970 | * We can get an asynchronous event or receive message in addition |
| 4926 | * to commands we send. | 4971 | * to commands we send. |
| 4927 | */ | 4972 | */ |
| 4928 | if (msg == intf->curr_msg) | 4973 | if (msg == intf->curr_msg) |
| 4929 | intf->curr_msg = NULL; | 4974 | intf->curr_msg = NULL; |
| 4930 | if (!run_to_completion) | 4975 | ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); |
| 4931 | spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | ||
| 4932 | 4976 | ||
| 4933 | if (run_to_completion) | 4977 | if (run_to_completion) |
| 4934 | smi_work(&intf->smi_work); | 4978 | smi_work(&intf->smi_work); |
| @@ -5041,7 +5085,12 @@ static void check_msg_timeout(struct ipmi_smi *intf, struct seq_table *ent, · 106 unmodified lines | |||
| 5041 | ipmi_inc_stat(intf, | 5085 | ipmi_inc_stat(intf, |
| 5042 | retransmitted_ipmb_commands); | 5086 | retransmitted_ipmb_commands); |
| 5043 | 5087 | ||
| 5044 | smi_send(intf, intf->handlers, smi_msg, 0); | 5088 | /* If this fails we'll retry later or timeout. */ |
| 5089 | if (smi_send(intf, intf->handlers, smi_msg, 0) != IPMI_CC_NO_ERROR) { | ||
| 5090 | /* But fix the timeout. */ | ||
| 5091 | intf_start_seq_timer(intf, smi_msg->msgid); | ||
| 5092 | ipmi_free_smi_msg(smi_msg); | ||
| 5093 | } | ||
| 5045 | } else | 5094 | } else |
| 5046 | ipmi_free_smi_msg(smi_msg); | 5095 | ipmi_free_smi_msg(smi_msg); |
| 5047 | 5096 | ||
| @@ -809,6 +809,12 @@ restart: · 808 unmodified lines | |||
| 809 | */ | 809 | */ |
| 810 | return_hosed_msg(smi_info, IPMI_BUS_ERR); | 810 | return_hosed_msg(smi_info, IPMI_BUS_ERR); |
| 811 | } | 811 | } |
| 812 | if (smi_info->waiting_msg != NULL) { | ||
| 813 | /* Also handle if there was a message waiting. */ | ||
| 814 | smi_info->curr_msg = smi_info->waiting_msg; | ||
| 815 | smi_info->waiting_msg = NULL; | ||
| 816 | return_hosed_msg(smi_info, IPMI_BUS_ERR); | ||
| 817 | } | ||
| 812 | smi_mod_timer(smi_info, jiffies + SI_TIMEOUT_HOSED); | 818 | smi_mod_timer(smi_info, jiffies + SI_TIMEOUT_HOSED); |
| 813 | goto out; | 819 | goto out; |
| 814 | } | 820 | } |
| @@ -918,9 +924,14 @@ static int sender(void *send_info, struct ipmi_smi_msg *msg) · 103 unmodified lines | |||
| 918 | { | 924 | { |
| 919 | struct smi_info *smi_info = send_info; | 925 | struct smi_info *smi_info = send_info; |
| 920 | unsigned long flags; | 926 | unsigned long flags; |
| 927 | int rv = IPMI_CC_NO_ERROR; | ||
| 921 | 928 | ||
| 922 | debug_timestamp(smi_info, "Enqueue"); | 929 | debug_timestamp(smi_info, "Enqueue"); |
| 923 | 930 | ||
| 931 | /* | ||
| 932 | * Check here for run to completion mode. A check under lock is | ||
| 933 | * later. | ||
| 934 | */ | ||
| 924 | if (smi_info->si_state == SI_HOSED) | 935 | if (smi_info->si_state == SI_HOSED) |
| 925 | return IPMI_BUS_ERR; | 936 | return IPMI_BUS_ERR; |
| 926 | 937 | ||
| @@ -934,18 +945,15 @@ static int sender(void *send_info, struct ipmi_smi_msg *msg) · 7 unmodified lines | |||
| 934 | } | 945 | } |
| 935 | 946 | ||
| 936 | spin_lock_irqsave(&smi_info->si_lock, flags); | 947 | spin_lock_irqsave(&smi_info->si_lock, flags); |
| 937 | /* | 948 | if (smi_info->si_state == SI_HOSED) { |
| 938 | * The following two lines don't need to be under the lock for | 949 | rv = IPMI_BUS_ERR; |
| 939 | * the lock's sake, but they do need SMP memory barriers to | 950 | } else { |
| 940 | * avoid getting things out of order. We are already claiming | 951 | BUG_ON(smi_info->waiting_msg); |
| 941 | * the lock, anyway, so just do it under the lock to avoid the | 952 | smi_info->waiting_msg = msg; |
| 942 | * ordering problem. | 953 | check_start_timer_thread(smi_info); |
| 943 | */ | 954 | } |
| 944 | BUG_ON(smi_info->waiting_msg); | ||
| 945 | smi_info->waiting_msg = msg; | ||
| 946 | check_start_timer_thread(smi_info); | ||
| 947 | spin_unlock_irqrestore(&smi_info->si_lock, flags); | 955 | spin_unlock_irqrestore(&smi_info->si_lock, flags); |
| 948 | return IPMI_CC_NO_ERROR; | 956 | return rv; |
| 949 | } | 957 | } |
| 950 | 958 | ||
| 951 | static void set_run_to_completion(void *send_info, bool i_run_to_completion) | 959 | static void set_run_to_completion(void *send_info, bool i_run_to_completion) |
| @@ -1113,7 +1121,9 @@ static void smi_timeout(struct timer_list *t) · 161 unmodified lines | |||
| 1113 | * SI_USEC_PER_JIFFY); | 1121 | * SI_USEC_PER_JIFFY); |
| 1114 | smi_result = smi_event_handler(smi_info, time_diff); | 1122 | smi_result = smi_event_handler(smi_info, time_diff); |
| 1115 | 1123 | ||
| 1116 | if ((smi_info->io.irq) && (!smi_info->interrupt_disabled)) { | 1124 | if (smi_info->si_state == SI_HOSED) { |
| 1125 | timeout = jiffies + SI_TIMEOUT_HOSED; | ||
| 1126 | } else if ((smi_info->io.irq) && (!smi_info->interrupt_disabled)) { | ||
| 1117 | /* Running with interrupts, only do long timeouts. */ | 1127 | /* Running with interrupts, only do long timeouts. */ |
| 1118 | timeout = jiffies + SI_TIMEOUT_JIFFIES; | 1128 | timeout = jiffies + SI_TIMEOUT_JIFFIES; |
| 1119 | smi_inc_stat(smi_info, long_timeouts); | 1129 | smi_inc_stat(smi_info, long_timeouts); |
| @@ -2226,7 +2236,8 @@ static void wait_msg_processed(struct smi_info *smi_info) · 1106 unmodified lines | |||
| 2226 | unsigned long jiffies_now; | 2236 | unsigned long jiffies_now; |
| 2227 | long time_diff; | 2237 | long time_diff; |
| 2228 | 2238 | ||
| 2229 | while (smi_info->curr_msg || (smi_info->si_state != SI_NORMAL)) { | 2239 | while (smi_info->si_state != SI_HOSED && |
| 2240 | (smi_info->curr_msg || (smi_info->si_state != SI_NORMAL))) { | ||
| 2230 | jiffies_now = jiffies; | 2241 | jiffies_now = jiffies; |
| 2231 | time_diff = (((long)jiffies_now - (long)smi_info->last_timeout_jiffies) | 2242 | time_diff = (((long)jiffies_now - (long)smi_info->last_timeout_jiffies) |
| 2232 | * SI_USEC_PER_JIFFY); | 2243 | * SI_USEC_PER_JIFFY); |
| @@ -168,7 +168,7 @@ static void ipmi_ls2k_remove(struct platform_device *pdev) · 167 unmodified lines | |||
| 168 | ipmi_si_remove_by_dev(&pdev->dev); | 168 | ipmi_si_remove_by_dev(&pdev->dev); |
| 169 | } | 169 | } |
| 170 | 170 | ||
| 171 | struct platform_driver ipmi_ls2k_platform_driver = { | 171 | static struct platform_driver ipmi_ls2k_platform_driver = { |
| 172 | .driver = { | 172 | .driver = { |
| 173 | .name = "ls2k-ipmi-si", | 173 | .name = "ls2k-ipmi-si", |
| 174 | }, | 174 | }, |
| @@ -96,8 +96,7 @@ static ATOMIC_NOTIFIER_HEAD(random_ready_notifier); · 95 unmodified lines | |||
| 96 | /* Control how we warn userspace. */ | 96 | /* Control how we warn userspace. */ |
| 97 | static struct ratelimit_state urandom_warning = | 97 | static struct ratelimit_state urandom_warning = |
| 98 | RATELIMIT_STATE_INIT_FLAGS("urandom_warning", HZ, 3, RATELIMIT_MSG_ON_RELEASE); | 98 | RATELIMIT_STATE_INIT_FLAGS("urandom_warning", HZ, 3, RATELIMIT_MSG_ON_RELEASE); |
| 99 | static int ratelimit_disable __read_mostly = | 99 | static int ratelimit_disable __read_mostly = 0; |
| 100 | IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM); | ||
| 101 | module_param_named(ratelimit_disable, ratelimit_disable, int, 0644); | 100 | module_param_named(ratelimit_disable, ratelimit_disable, int, 0644); |
| 102 | MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression"); | 101 | MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression"); |
| 103 | 102 | ||
| @@ -168,12 +167,6 @@ int __cold execute_with_initialized_rng(struct notifier_block *nb) · 64 unmodified lines | |||
| 168 | return ret; | 167 | return ret; |
| 169 | } | 168 | } |
| 170 | 169 | ||
| 171 | #define warn_unseeded_randomness() \ | ||
| 172 | if (IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM) && !crng_ready()) \ | ||
| 173 | printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n", \ | ||
| 174 | __func__, (void *)_RET_IP_, crng_init) | ||
| 175 | |||
| 176 | |||
| 177 | /********************************************************************* | 170 | /********************************************************************* |
| 178 | * | 171 | * |
| 179 | * Fast key erasure RNG, the "crng". | 172 | * Fast key erasure RNG, the "crng". |
| @@ -434,7 +427,6 @@ static void _get_random_bytes(void *buf, size_t len) · 254 unmodified lines | |||
| 434 | */ | 427 | */ |
| 435 | void get_random_bytes(void *buf, size_t len) | 428 | void get_random_bytes(void *buf, size_t len) |
| 436 | { | 429 | { |
| 437 | warn_unseeded_randomness(); | ||
| 438 | _get_random_bytes(buf, len); | 430 | _get_random_bytes(buf, len); |
| 439 | } | 431 | } |
| 440 | EXPORT_SYMBOL(get_random_bytes); | 432 | EXPORT_SYMBOL(get_random_bytes); |
| @@ -523,8 +515,6 @@ type get_random_ ##type(void) \ · 82 unmodified lines | |||
| 523 | struct batch_ ##type *batch; \ | 515 | struct batch_ ##type *batch; \ |
| 524 | unsigned long next_gen; \ | 516 | unsigned long next_gen; \ |
| 525 | \ | 517 | \ |
| 526 | warn_unseeded_randomness(); \ | ||
| 527 | \ | ||
| 528 | if (!crng_ready()) { \ | 518 | if (!crng_ready()) { \ |
| 529 | _get_random_bytes(&ret, sizeof(ret)); \ | 519 | _get_random_bytes(&ret, sizeof(ret)); \ |
| 530 | return ret; \ | 520 | return ret; \ |
| @@ -346,7 +346,29 @@ static struct platform_driver imx8qxp_clk_driver = { · 345 unmodified lines | |||
| 346 | }, | 346 | }, |
| 347 | .probe = imx8qxp_clk_probe, | 347 | .probe = imx8qxp_clk_probe, |
| 348 | }; | 348 | }; |
| 349 | module_platform_driver(imx8qxp_clk_driver); | 349 | |
| 350 | static int __init imx8qxp_clk_init(void) | ||
| 351 | { | ||
| 352 | int ret; | ||
| 353 | |||
| 354 | ret = platform_driver_register(&imx8qxp_clk_driver); | ||
| 355 | if (ret) | ||
| 356 | return ret; | ||
| 357 | |||
| 358 | ret = imx_clk_scu_module_init(); | ||
| 359 | if (ret) | ||
| 360 | platform_driver_unregister(&imx8qxp_clk_driver); | ||
| 361 | |||
| 362 | return ret; | ||
| 363 | } | ||
| 364 | module_init(imx8qxp_clk_init); | ||
| 365 | |||
| 366 | static void __exit imx8qxp_clk_exit(void) | ||
| 367 | { | ||
| 368 | imx_clk_scu_module_exit(); | ||
| 369 | platform_driver_unregister(&imx8qxp_clk_driver); | ||
| 370 | } | ||
| 371 | module_exit(imx8qxp_clk_exit); | ||
| 350 | 372 | ||
| 351 | MODULE_AUTHOR("Aisheng Dong <aisheng.dong@nxp.com>"); | 373 | MODULE_AUTHOR("Aisheng Dong <aisheng.dong@nxp.com>"); |
| 352 | MODULE_DESCRIPTION("NXP i.MX8QXP clock driver"); | 374 | MODULE_DESCRIPTION("NXP i.MX8QXP clock driver"); |
| @@ -191,6 +191,16 @@ static bool imx_scu_clk_is_valid(u32 rsrc_id) · 190 unmodified lines | |||
| 191 | return p != NULL; | 191 | return p != NULL; |
| 192 | } | 192 | } |
| 193 | 193 | ||
| 194 | int __init imx_clk_scu_module_init(void) | ||
| 195 | { | ||
| 196 | return platform_driver_register(&imx_clk_scu_driver); | ||
| 197 | } | ||
| 198 | |||
| 199 | void __exit imx_clk_scu_module_exit(void) | ||
| 200 | { | ||
| 201 | return platform_driver_unregister(&imx_clk_scu_driver); | ||
| 202 | } | ||
| 203 | |||
| 194 | int imx_clk_scu_init(struct device_node *np, | 204 | int imx_clk_scu_init(struct device_node *np, |
| 195 | const struct imx_clk_scu_rsrc_table *data) | 205 | const struct imx_clk_scu_rsrc_table *data) |
| 196 | { | 206 | { |
| @@ -215,7 +225,7 @@ int imx_clk_scu_init(struct device_node *np, · 18 unmodified lines | |||
| 215 | rsrc_table = data; | 225 | rsrc_table = data; |
| 216 | } | 226 | } |
| 217 | 227 | ||
| 218 | return platform_driver_register(&imx_clk_scu_driver); | 228 | return 0; |
| 219 | } | 229 | } |
| 220 | 230 | ||
| 221 | /* | 231 | /* |
| @@ -25,6 +25,8 @@ extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8dxl; · 24 unmodified lines | |||
| 25 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qxp; | 25 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qxp; |
| 26 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qm; | 26 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qm; |
| 27 | 27 | ||
| 28 | int __init imx_clk_scu_module_init(void); | ||
| 29 | void __exit imx_clk_scu_module_exit(void); | ||
| 28 | int imx_clk_scu_init(struct device_node *np, | 30 | int imx_clk_scu_init(struct device_node *np, |
| 29 | const struct imx_clk_scu_rsrc_table *data); | 31 | const struct imx_clk_scu_rsrc_table *data); |
| 30 | struct clk_hw *imx_scu_of_clk_src_get(struct of_phandle_args *clkspec, | 32 | struct clk_hw *imx_scu_of_clk_src_get(struct of_phandle_args *clkspec, |
| @@ -1476,13 +1476,13 @@ static void __intel_pstate_update_max_freq(struct cpufreq_policy *policy, · 1475 unmodified lines | |||
| 1476 | refresh_frequency_limits(policy); | 1476 | refresh_frequency_limits(policy); |
| 1477 | } | 1477 | } |
| 1478 | 1478 | ||
| 1479 | static bool intel_pstate_update_max_freq(struct cpudata *cpudata) | 1479 | static bool intel_pstate_update_max_freq(int cpu) |
| 1480 | { | 1480 | { |
| 1481 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpudata->cpu); | 1481 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpu); |
| 1482 | if (!policy) | 1482 | if (!policy) |
| 1483 | return false; | 1483 | return false; |
| 1484 | 1484 | ||
| 1485 | __intel_pstate_update_max_freq(policy, cpudata); | 1485 | __intel_pstate_update_max_freq(policy, all_cpu_data[cpu]); |
| 1486 | 1486 | ||
| 1487 | return true; | 1487 | return true; |
| 1488 | } | 1488 | } |
| @@ -1501,7 +1501,7 @@ static void intel_pstate_update_limits_for_all(void) · 12 unmodified lines | |||
| 1501 | int cpu; | 1501 | int cpu; |
| 1502 | 1502 | ||
| 1503 | for_each_possible_cpu(cpu) | 1503 | for_each_possible_cpu(cpu) |
| 1504 | intel_pstate_update_max_freq(all_cpu_data[cpu]); | 1504 | intel_pstate_update_max_freq(cpu); |
| 1505 | 1505 | ||
| 1506 | mutex_lock(&hybrid_capacity_lock); | 1506 | mutex_lock(&hybrid_capacity_lock); |
| 1507 | 1507 | ||
| @@ -1647,8 +1647,8 @@ static ssize_t store_no_turbo(struct kobject *a, struct kobj_attribute *b, · 139 unmodified lines | |||
| 1647 | static void update_cpu_qos_request(int cpu, enum freq_qos_req_type type) | 1647 | static void update_cpu_qos_request(int cpu, enum freq_qos_req_type type) |
| 1648 | { | 1648 | { |
| 1649 | struct cpudata *cpudata = all_cpu_data[cpu]; | 1649 | struct cpudata *cpudata = all_cpu_data[cpu]; |
| 1650 | unsigned int freq = cpudata->pstate.turbo_freq; | ||
| 1651 | struct freq_qos_request *req; | 1650 | struct freq_qos_request *req; |
| 1651 | unsigned int freq; | ||
| 1652 | 1652 | ||
| 1653 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpu); | 1653 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpu); |
| 1654 | if (!policy) | 1654 | if (!policy) |
| @@ -1661,6 +1661,8 @@ static void update_cpu_qos_request(int cpu, enum freq_qos_req_type type) · 6 unmodified lines | |||
| 1661 | if (hwp_active) | 1661 | if (hwp_active) |
| 1662 | intel_pstate_get_hwp_cap(cpudata); | 1662 | intel_pstate_get_hwp_cap(cpudata); |
| 1663 | 1663 | ||
| 1664 | freq = cpudata->pstate.turbo_freq; | ||
| 1665 | |||
| 1664 | if (type == FREQ_QOS_MIN) { | 1666 | if (type == FREQ_QOS_MIN) { |
| 1665 | freq = DIV_ROUND_UP(freq * global.min_perf_pct, 100); | 1667 | freq = DIV_ROUND_UP(freq * global.min_perf_pct, 100); |
| 1666 | } else { | 1668 | } else { |
| @@ -1908,7 +1910,7 @@ static void intel_pstate_notify_work(struct work_struct *work) · 241 unmodified lines | |||
| 1908 | struct cpudata *cpudata = | 1910 | struct cpudata *cpudata = |
| 1909 | container_of(to_delayed_work(work), struct cpudata, hwp_notify_work); | 1911 | container_of(to_delayed_work(work), struct cpudata, hwp_notify_work); |
| 1910 | 1912 | ||
| 1911 | if (intel_pstate_update_max_freq(cpudata)) { | 1913 | if (intel_pstate_update_max_freq(cpudata->cpu)) { |
| 1912 | /* | 1914 | /* |
| 1913 | * The driver will not be unregistered while this function is | 1915 | * The driver will not be unregistered while this function is |
| 1914 | * running, so update the capacity without acquiring the driver | 1916 | * running, so update the capacity without acquiring the driver |
| @@ -359,16 +359,6 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev, · 358 unmodified lines | |||
| 359 | int cpuidle_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, | 359 | int cpuidle_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, |
| 360 | bool *stop_tick) | 360 | bool *stop_tick) |
| 361 | { | 361 | { |
| 362 | /* | ||
| 363 | * If there is only a single idle state (or none), there is nothing | ||
| 364 | * meaningful for the governor to choose. Skip the governor and | ||
| 365 | * always use state 0 with the tick running. | ||
| 366 | */ | ||
| 367 | if (drv->state_count <= 1) { | ||
| 368 | *stop_tick = false; | ||
| 369 | return 0; | ||
| 370 | } | ||
| 371 | |||
| 372 | return cpuidle_curr_governor->select(drv, dev, stop_tick); | 362 | return cpuidle_curr_governor->select(drv, dev, stop_tick); |
| 373 | } | 363 | } |
| 374 | 364 | ||
| @@ -52,9 +52,10 @@ static int atmel_sha204a_rng_read_nonblocking(struct hwrng *rng, void *data, · 51 unmodified lines | |||
| 52 | rng->priv = 0; | 52 | rng->priv = 0; |
| 53 | } else { | 53 | } else { |
| 54 | work_data = kmalloc_obj(*work_data, GFP_ATOMIC); | 54 | work_data = kmalloc_obj(*work_data, GFP_ATOMIC); |
| 55 | if (!work_data) | 55 | if (!work_data) { |
| 56 | atomic_dec(&i2c_priv->tfm_count); | ||
| 56 | return -ENOMEM; | 57 | return -ENOMEM; |
| 57 | 58 | } | |
| 58 | work_data->ctx = i2c_priv; | 59 | work_data->ctx = i2c_priv; |
| 59 | work_data->client = i2c_priv->client; | 60 | work_data->client = i2c_priv->client; |
| 60 | 61 | ||
| @@ -378,9 +378,9 @@ void sev_tsm_init_locked(struct sev_device *sev, void *tio_status_page) · 377 unmodified lines | |||
| 378 | return; | 378 | return; |
| 379 | 379 | ||
| 380 | error_exit: | 380 | error_exit: |
| 381 | kfree(t); | ||
| 382 | pr_err("Failed to enable SEV-TIO: ret=%d en=%d initdone=%d SEV=%d\n", | 381 | pr_err("Failed to enable SEV-TIO: ret=%d en=%d initdone=%d SEV=%d\n", |
| 383 | ret, t->tio_en, t->tio_init_done, boot_cpu_has(X86_FEATURE_SEV)); | 382 | ret, t->tio_en, t->tio_init_done, boot_cpu_has(X86_FEATURE_SEV)); |
| 383 | kfree(t); | ||
| 384 | } | 384 | } |
| 385 | 385 | ||
| 386 | void sev_tsm_uninit(struct sev_device *sev) | 386 | void sev_tsm_uninit(struct sev_device *sev) |
| @@ -1105,15 +1105,12 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages) · 1104 unmodified lines | |||
| 1105 | { | 1105 | { |
| 1106 | struct psp_device *psp_master = psp_get_master_device(); | 1106 | struct psp_device *psp_master = psp_get_master_device(); |
| 1107 | struct snp_hv_fixed_pages_entry *entry; | 1107 | struct snp_hv_fixed_pages_entry *entry; |
| 1108 | struct sev_device *sev; | ||
| 1109 | unsigned int order; | 1108 | unsigned int order; |
| 1110 | struct page *page; | 1109 | struct page *page; |
| 1111 | 1110 | ||
| 1112 | if (!psp_master || !psp_master->sev_data) | 1111 | if (!psp_master) |
| 1113 | return NULL; | 1112 | return NULL; |
| 1114 | 1113 | ||
| 1115 | sev = psp_master->sev_data; | ||
| 1116 | |||
| 1117 | order = get_order(PMD_SIZE * num_2mb_pages); | 1114 | order = get_order(PMD_SIZE * num_2mb_pages); |
| 1118 | 1115 | ||
| 1119 | /* | 1116 | /* |
| @@ -1126,7 +1123,8 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages) · 6 unmodified lines | |||
| 1126 | * This API uses SNP_INIT_EX to transition allocated pages to HV_Fixed | 1123 | * This API uses SNP_INIT_EX to transition allocated pages to HV_Fixed |
| 1127 | * page state, fail if SNP is already initialized. | 1124 | * page state, fail if SNP is already initialized. |
| 1128 | */ | 1125 | */ |
| 1129 | if (sev->snp_initialized) | 1126 | if (psp_master->sev_data && |
| 1127 | ((struct sev_device *)psp_master->sev_data)->snp_initialized) | ||
| 1130 | return NULL; | 1128 | return NULL; |
| 1131 | 1129 | ||
| 1132 | /* Re-use freed pages that match the request */ | 1130 | /* Re-use freed pages that match the request */ |
| @@ -1162,7 +1160,7 @@ void snp_free_hv_fixed_pages(struct page *page) · 29 unmodified lines | |||
| 1162 | struct psp_device *psp_master = psp_get_master_device(); | 1160 | struct psp_device *psp_master = psp_get_master_device(); |
| 1163 | struct snp_hv_fixed_pages_entry *entry, *nentry; | 1161 | struct snp_hv_fixed_pages_entry *entry, *nentry; |
| 1164 | 1162 | ||
| 1165 | if (!psp_master || !psp_master->sev_data) | 1163 | if (!psp_master) |
| 1166 | return; | 1164 | return; |
| 1167 | 1165 | ||
| 1168 | /* | 1166 | /* |
| @@ -152,6 +152,24 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c); · 151 unmodified lines | |||
| 152 | int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port, | 152 | int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port, |
| 153 | struct access_coordinate *c); | 153 | struct access_coordinate *c); |
| 154 | 154 | ||
| 155 | static inline struct device *port_to_host(struct cxl_port *port) | ||
| 156 | { | ||
| 157 | struct cxl_port *parent = is_cxl_root(port) ? NULL : | ||
| 158 | to_cxl_port(port->dev.parent); | ||
| 159 | |||
| 160 | /* | ||
| 161 | * The host of CXL root port and the first level of ports is | ||
| 162 | * the platform firmware device, the host of all other ports | ||
| 163 | * is their parent port. | ||
| 164 | */ | ||
| 165 | if (!parent) | ||
| 166 | return port->uport_dev; | ||
| 167 | else if (is_cxl_root(parent)) | ||
| 168 | return parent->uport_dev; | ||
| 169 | else | ||
| 170 | return &parent->dev; | ||
| 171 | } | ||
| 172 | |||
| 155 | static inline struct device *dport_to_host(struct cxl_dport *dport) | 173 | static inline struct device *dport_to_host(struct cxl_dport *dport) |
| 156 | { | 174 | { |
| 157 | struct cxl_port *port = dport->port; | 175 | struct cxl_port *port = dport->port; |
| @@ -904,7 +904,7 @@ static void cxl_decoder_reset(struct cxl_decoder *cxld) · 903 unmodified lines | |||
| 904 | if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) | 904 | if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) |
| 905 | return; | 905 | return; |
| 906 | 906 | ||
| 907 | if (test_bit(CXL_DECODER_F_LOCK, &cxld->flags)) | 907 | if (cxld->flags & CXL_DECODER_F_LOCK) |
| 908 | return; | 908 | return; |
| 909 | 909 | ||
| 910 | if (port->commit_end == id) | 910 | if (port->commit_end == id) |
| @@ -311,6 +311,7 @@ static bool cxl_mem_raw_command_allowed(u16 opcode) · 310 unmodified lines | |||
| 311 | * cxl_payload_from_user_allowed() - Check contents of in_payload. | 311 | * cxl_payload_from_user_allowed() - Check contents of in_payload. |
| 312 | * @opcode: The mailbox command opcode. | 312 | * @opcode: The mailbox command opcode. |
| 313 | * @payload_in: Pointer to the input payload passed in from user space. | 313 | * @payload_in: Pointer to the input payload passed in from user space. |
| 314 | * @in_size: Size of @payload_in in bytes. | ||
| 314 | * | 315 | * |
| 315 | * Return: | 316 | * Return: |
| 316 | * * true - payload_in passes check for @opcode. | 317 | * * true - payload_in passes check for @opcode. |
| @@ -325,12 +326,15 @@ static bool cxl_mem_raw_command_allowed(u16 opcode) · 8 unmodified lines | |||
| 325 | * | 326 | * |
| 326 | * The specific checks are determined by the opcode. | 327 | * The specific checks are determined by the opcode. |
| 327 | */ | 328 | */ |
| 328 | static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) | 329 | static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in, |
| 330 | size_t in_size) | ||
| 329 | { | 331 | { |
| 330 | switch (opcode) { | 332 | switch (opcode) { |
| 331 | case CXL_MBOX_OP_SET_PARTITION_INFO: { | 333 | case CXL_MBOX_OP_SET_PARTITION_INFO: { |
| 332 | struct cxl_mbox_set_partition_info *pi = payload_in; | 334 | struct cxl_mbox_set_partition_info *pi = payload_in; |
| 333 | 335 | ||
| 336 | if (in_size < sizeof(*pi)) | ||
| 337 | return false; | ||
| 334 | if (pi->flags & CXL_SET_PARTITION_IMMEDIATE_FLAG) | 338 | if (pi->flags & CXL_SET_PARTITION_IMMEDIATE_FLAG) |
| 335 | return false; | 339 | return false; |
| 336 | break; | 340 | break; |
| @@ -338,6 +342,8 @@ static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) · 1 unmodified line | |||
| 338 | case CXL_MBOX_OP_CLEAR_LOG: { | 342 | case CXL_MBOX_OP_CLEAR_LOG: { |
| 339 | const uuid_t *uuid = (uuid_t *)payload_in; | 343 | const uuid_t *uuid = (uuid_t *)payload_in; |
| 340 | 344 | ||
| 345 | if (in_size < sizeof(uuid_t)) | ||
| 346 | return false; | ||
| 341 | /* | 347 | /* |
| 342 | * Restrict the ‘Clear log’ action to only apply to | 348 | * Restrict the ‘Clear log’ action to only apply to |
| 343 | * Vendor debug logs. | 349 | * Vendor debug logs. |
| @@ -365,7 +371,8 @@ static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd, · 21 unmodified lines | |||
| 365 | if (IS_ERR(mbox_cmd->payload_in)) | 371 | if (IS_ERR(mbox_cmd->payload_in)) |
| 366 | return PTR_ERR(mbox_cmd->payload_in); | 372 | return PTR_ERR(mbox_cmd->payload_in); |
| 367 | 373 | ||
| 368 | if (!cxl_payload_from_user_allowed(opcode, mbox_cmd->payload_in)) { | 374 | if (!cxl_payload_from_user_allowed(opcode, mbox_cmd->payload_in, |
| 375 | in_size)) { | ||
| 369 | dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n", | 376 | dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n", |
| 370 | cxl_mem_opcode_to_name(opcode)); | 377 | cxl_mem_opcode_to_name(opcode)); |
| 371 | kvfree(mbox_cmd->payload_in); | 378 | kvfree(mbox_cmd->payload_in); |
| @@ -1089,10 +1089,8 @@ static int cxlmd_add(struct cxl_memdev *cxlmd, struct cxl_dev_state *cxlds) · 1088 unmodified lines | |||
| 1089 | DEFINE_FREE(put_cxlmd, struct cxl_memdev *, | 1089 | DEFINE_FREE(put_cxlmd, struct cxl_memdev *, |
| 1090 | if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) | 1090 | if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) |
| 1091 | 1091 | ||
| 1092 | static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) | 1092 | static bool cxl_memdev_attach_failed(struct cxl_memdev *cxlmd) |
| 1093 | { | 1093 | { |
| 1094 | int rc; | ||
| 1095 | |||
| 1096 | /* | 1094 | /* |
| 1097 | * If @attach is provided fail if the driver is not attached upon | 1095 | * If @attach is provided fail if the driver is not attached upon |
| 1098 | * return. Note that failure here could be the result of a race to | 1096 | * return. Note that failure here could be the result of a race to |
| @@ -1100,7 +1098,14 @@ static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) · 1 unmodified line | |||
| 1100 | * succeeded and then cxl_mem unbound before the lock is acquired. | 1098 | * succeeded and then cxl_mem unbound before the lock is acquired. |
| 1101 | */ | 1099 | */ |
| 1102 | guard(device)(&cxlmd->dev); | 1100 | guard(device)(&cxlmd->dev); |
| 1103 | if (cxlmd->attach && !cxlmd->dev.driver) { | 1101 | return (cxlmd->attach && !cxlmd->dev.driver); |
| 1102 | } | ||
| 1103 | |||
| 1104 | static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) | ||
| 1105 | { | ||
| 1106 | int rc; | ||
| 1107 | |||
| 1108 | if (cxl_memdev_attach_failed(cxlmd)) { | ||
| 1104 | cxl_memdev_unregister(cxlmd); | 1109 | cxl_memdev_unregister(cxlmd); |
| 1105 | return ERR_PTR(-ENXIO); | 1110 | return ERR_PTR(-ENXIO); |
| 1106 | } | 1111 | } |
| @@ -115,15 +115,17 @@ static void unregister_nvb(void *_cxl_nvb) · 114 unmodified lines | |||
| 115 | device_unregister(&cxl_nvb->dev); | 115 | device_unregister(&cxl_nvb->dev); |
| 116 | } | 116 | } |
| 117 | 117 | ||
| 118 | /** | 118 | static bool cxl_nvdimm_bridge_failed_attach(struct cxl_nvdimm_bridge *cxl_nvb) |
| 119 | * devm_cxl_add_nvdimm_bridge() - add the root of a LIBNVDIMM topology | 119 | { |
| 120 | * @host: platform firmware root device | 120 | struct device *dev = &cxl_nvb->dev; |
| 121 | * @port: CXL port at the root of a CXL topology | 121 | |
| 122 | * | 122 | guard(device)(dev); |
| 123 | * Return: bridge device that can host cxl_nvdimm objects | 123 | /* If the device has no driver, then it failed to attach. */ |
| 124 | */ | 124 | return dev->driver == NULL; |
| 125 | struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, | 125 | } |
| 126 | struct cxl_port *port) | 126 | |
| 127 | struct cxl_nvdimm_bridge *__devm_cxl_add_nvdimm_bridge(struct device *host, | ||
| 128 | struct cxl_port *port) | ||
| 127 | { | 129 | { |
| 128 | struct cxl_nvdimm_bridge *cxl_nvb; | 130 | struct cxl_nvdimm_bridge *cxl_nvb; |
| 129 | struct device *dev; | 131 | struct device *dev; |
| @@ -145,6 +147,11 @@ struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, · 15 unmodified lines | |||
| 145 | if (rc) | 147 | if (rc) |
| 146 | goto err; | 148 | goto err; |
| 147 | 149 | ||
| 150 | if (cxl_nvdimm_bridge_failed_attach(cxl_nvb)) { | ||
| 151 | unregister_nvb(cxl_nvb); | ||
| 152 | return ERR_PTR(-ENODEV); | ||
| 153 | } | ||
| 154 | |||
| 148 | rc = devm_add_action_or_reset(host, unregister_nvb, cxl_nvb); | 155 | rc = devm_add_action_or_reset(host, unregister_nvb, cxl_nvb); |
| 149 | if (rc) | 156 | if (rc) |
| 150 | return ERR_PTR(rc); | 157 | return ERR_PTR(rc); |
| @@ -155,7 +162,7 @@ err: · 4 unmodified lines | |||
| 155 | put_device(dev); | 162 | put_device(dev); |
| 156 | return ERR_PTR(rc); | 163 | return ERR_PTR(rc); |
| 157 | } | 164 | } |
| 158 | EXPORT_SYMBOL_NS_GPL(devm_cxl_add_nvdimm_bridge, "CXL"); | 165 | EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_nvdimm_bridge, "cxl_pmem"); |
| 159 | 166 | ||
| 160 | static void cxl_nvdimm_release(struct device *dev) | 167 | static void cxl_nvdimm_release(struct device *dev) |
| 161 | { | 168 | { |
| @@ -255,6 +262,21 @@ int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port, · 93 unmodified lines | |||
| 255 | if (!cxl_nvb) | 262 | if (!cxl_nvb) |
| 256 | return -ENODEV; | 263 | return -ENODEV; |
| 257 | 264 | ||
| 265 | /* | ||
| 266 | * Take the uport_dev lock to guard against race of nvdimm_bus object. | ||
| 267 | * cxl_acpi_probe() registers the nvdimm_bus and is done under the | ||
| 268 | * root port uport_dev lock. | ||
| 269 | * | ||
| 270 | * Take the cxl_nvb device lock to ensure that cxl_nvb driver is in a | ||
| 271 | * consistent state. And the driver registers nvdimm_bus. | ||
| 272 | */ | ||
| 273 | guard(device)(cxl_nvb->port->uport_dev); | ||
| 274 | guard(device)(&cxl_nvb->dev); | ||
| 275 | if (!cxl_nvb->nvdimm_bus) { | ||
| 276 | rc = -ENODEV; | ||
| 277 | goto err_alloc; | ||
| 278 | } | ||
| 279 | |||
| 258 | cxl_nvd = cxl_nvdimm_alloc(cxl_nvb, cxlmd); | 280 | cxl_nvd = cxl_nvdimm_alloc(cxl_nvb, cxlmd); |
| 259 | if (IS_ERR(cxl_nvd)) { | 281 | if (IS_ERR(cxl_nvd)) { |
| 260 | rc = PTR_ERR(cxl_nvd); | 282 | rc = PTR_ERR(cxl_nvd); |
| @@ -615,22 +615,8 @@ struct cxl_port *parent_port_of(struct cxl_port *port) · 614 unmodified lines | |||
| 615 | static void unregister_port(void *_port) | 615 | static void unregister_port(void *_port) |
| 616 | { | 616 | { |
| 617 | struct cxl_port *port = _port; | 617 | struct cxl_port *port = _port; |
| 618 | struct cxl_port *parent = parent_port_of(port); | ||
| 619 | struct device *lock_dev; | ||
| 620 | 618 | ||
| 621 | /* | 619 | device_lock_assert(port_to_host(port)); |
| 622 | * CXL root port's and the first level of ports are unregistered | ||
| 623 | * under the platform firmware device lock, all other ports are | ||
| 624 | * unregistered while holding their parent port lock. | ||
| 625 | */ | ||
| 626 | if (!parent) | ||
| 627 | lock_dev = port->uport_dev; | ||
| 628 | else if (is_cxl_root(parent)) | ||
| 629 | lock_dev = parent->uport_dev; | ||
| 630 | else | ||
| 631 | lock_dev = &parent->dev; | ||
| 632 | |||
| 633 | device_lock_assert(lock_dev); | ||
| 634 | port->dead = true; | 620 | port->dead = true; |
| 635 | device_unregister(&port->dev); | 621 | device_unregister(&port->dev); |
| 636 | } | 622 | } |
| @@ -1427,20 +1413,11 @@ static struct device *grandparent(struct device *dev) · 790 unmodified lines | |||
| 1427 | return NULL; | 1413 | return NULL; |
| 1428 | } | 1414 | } |
| 1429 | 1415 | ||
| 1430 | static struct device *endpoint_host(struct cxl_port *endpoint) | ||
| 1431 | { | ||
| 1432 | struct cxl_port *port = to_cxl_port(endpoint->dev.parent); | ||
| 1433 | |||
| 1434 | if (is_cxl_root(port)) | ||
| 1435 | return port->uport_dev; | ||
| 1436 | return &port->dev; | ||
| 1437 | } | ||
| 1438 | |||
| 1439 | static void delete_endpoint(void *data) | 1416 | static void delete_endpoint(void *data) |
| 1440 | { | 1417 | { |
| 1441 | struct cxl_memdev *cxlmd = data; | 1418 | struct cxl_memdev *cxlmd = data; |
| 1442 | struct cxl_port *endpoint = cxlmd->endpoint; | 1419 | struct cxl_port *endpoint = cxlmd->endpoint; |
| 1443 | struct device *host = endpoint_host(endpoint); | 1420 | struct device *host = port_to_host(endpoint); |
| 1444 | 1421 | ||
| 1445 | scoped_guard(device, host) { | 1422 | scoped_guard(device, host) { |
| 1446 | if (host->driver && !endpoint->dead) { | 1423 | if (host->driver && !endpoint->dead) { |
| @@ -1456,7 +1433,7 @@ static void delete_endpoint(void *data) · 9 unmodified lines | |||
| 1456 | 1433 | ||
| 1457 | int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint) | 1434 | int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint) |
| 1458 | { | 1435 | { |
| 1459 | struct device *host = endpoint_host(endpoint); | 1436 | struct device *host = port_to_host(endpoint); |
| 1460 | struct device *dev = &cxlmd->dev; | 1437 | struct device *dev = &cxlmd->dev; |
| 1461 | 1438 | ||
| 1462 | get_device(host); | 1439 | get_device(host); |
| @@ -1790,7 +1767,16 @@ static struct cxl_dport *find_or_add_dport(struct cxl_port *port, · 327 unmodified lines | |||
| 1790 | { | 1767 | { |
| 1791 | struct cxl_dport *dport; | 1768 | struct cxl_dport *dport; |
| 1792 | 1769 | ||
| 1793 | device_lock_assert(&port->dev); | 1770 | /* |
| 1771 | * The port is already visible in CXL hierarchy, but it may still | ||
| 1772 | * be in the process of binding to the CXL port driver at this point. | ||
| 1773 | * | ||
| 1774 | * port creation and driver binding are protected by the port's host | ||
| 1775 | * lock, so acquire the host lock here to ensure the port has completed | ||
| 1776 | * driver binding before proceeding with dport addition. | ||
| 1777 | */ | ||
| 1778 | guard(device)(port_to_host(port)); | ||
| 1779 | guard(device)(&port->dev); | ||
| 1794 | dport = cxl_find_dport_by_dev(port, dport_dev); | 1780 | dport = cxl_find_dport_by_dev(port, dport_dev); |
| 1795 | if (!dport) { | 1781 | if (!dport) { |
| 1796 | dport = probe_dport(port, dport_dev); | 1782 | dport = probe_dport(port, dport_dev); |
| @@ -1857,13 +1843,11 @@ retry: · 60 unmodified lines | |||
| 1857 | * RP port enumerated by cxl_acpi without dport will | 1843 | * RP port enumerated by cxl_acpi without dport will |
| 1858 | * have the dport added here. | 1844 | * have the dport added here. |
| 1859 | */ | 1845 | */ |
| 1860 | scoped_guard(device, &port->dev) { | 1846 | dport = find_or_add_dport(port, dport_dev); |
| 1861 | dport = find_or_add_dport(port, dport_dev); | 1847 | if (IS_ERR(dport)) { |
| 1862 | if (IS_ERR(dport)) { | 1848 | if (PTR_ERR(dport) == -EAGAIN) |
| 1863 | if (PTR_ERR(dport) == -EAGAIN) | 1849 | goto retry; |
| 1864 | goto retry; | 1850 | return PTR_ERR(dport); |
| 1865 | return PTR_ERR(dport); | ||
| 1866 | } | ||
| 1867 | } | 1851 | } |
| 1868 | 1852 | ||
| 1869 | rc = cxl_add_ep(dport, &cxlmd->dev); | 1853 | rc = cxl_add_ep(dport, &cxlmd->dev); |
| @@ -1100,12 +1100,12 @@ static int cxl_rr_assign_decoder(struct cxl_port *port, struct cxl_region *cxlr, · 1099 unmodified lines | |||
| 1100 | static void cxl_region_setup_flags(struct cxl_region *cxlr, | 1100 | static void cxl_region_setup_flags(struct cxl_region *cxlr, |
| 1101 | struct cxl_decoder *cxld) | 1101 | struct cxl_decoder *cxld) |
| 1102 | { | 1102 | { |
| 1103 | if (test_bit(CXL_DECODER_F_LOCK, &cxld->flags)) { | 1103 | if (cxld->flags & CXL_DECODER_F_LOCK) { |
| 1104 | set_bit(CXL_REGION_F_LOCK, &cxlr->flags); | 1104 | set_bit(CXL_REGION_F_LOCK, &cxlr->flags); |
| 1105 | clear_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); | 1105 | clear_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); |
| 1106 | } | 1106 | } |
| 1107 | 1107 | ||
| 1108 | if (test_bit(CXL_DECODER_F_NORMALIZED_ADDRESSING, &cxld->flags)) | 1108 | if (cxld->flags & CXL_DECODER_F_NORMALIZED_ADDRESSING) |
| 1109 | set_bit(CXL_REGION_F_NORMALIZED_ADDRESSING, &cxlr->flags); | 1109 | set_bit(CXL_REGION_F_NORMALIZED_ADDRESSING, &cxlr->flags); |
| 1110 | } | 1110 | } |
| 1111 | 1111 | ||
| @@ -574,11 +574,16 @@ struct cxl_nvdimm_bridge { · 573 unmodified lines | |||
| 574 | 574 | ||
| 575 | #define CXL_DEV_ID_LEN 19 | 575 | #define CXL_DEV_ID_LEN 19 |
| 576 | 576 | ||
| 577 | enum { | ||
| 578 | CXL_NVD_F_INVALIDATED = 0, | ||
| 579 | }; | ||
| 580 | |||
| 577 | struct cxl_nvdimm { | 581 | struct cxl_nvdimm { |
| 578 | struct device dev; | 582 | struct device dev; |
| 579 | struct cxl_memdev *cxlmd; | 583 | struct cxl_memdev *cxlmd; |
| 580 | u8 dev_id[CXL_DEV_ID_LEN]; /* for nvdimm, string of 'serial' */ | 584 | u8 dev_id[CXL_DEV_ID_LEN]; /* for nvdimm, string of 'serial' */ |
| 581 | u64 dirty_shutdowns; | 585 | u64 dirty_shutdowns; |
| 586 | unsigned long flags; | ||
| 582 | }; | 587 | }; |
| 583 | 588 | ||
| 584 | struct cxl_pmem_region_mapping { | 589 | struct cxl_pmem_region_mapping { |
| @@ -920,6 +925,8 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); · 335 unmodified lines | |||
| 920 | struct cxl_nvdimm_bridge *to_cxl_nvdimm_bridge(struct device *dev); | 925 | struct cxl_nvdimm_bridge *to_cxl_nvdimm_bridge(struct device *dev); |
| 921 | struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, | 926 | struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, |
| 922 | struct cxl_port *port); | 927 | struct cxl_port *port); |
| 928 | struct cxl_nvdimm_bridge *__devm_cxl_add_nvdimm_bridge(struct device *host, | ||
| 929 | struct cxl_port *port); | ||
| 923 | struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev); | 930 | struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev); |
| 924 | bool is_cxl_nvdimm(struct device *dev); | 931 | bool is_cxl_nvdimm(struct device *dev); |
| 925 | int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port, | 932 | int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port, |
| @@ -13,6 +13,20 @@ · 12 unmodified lines | |||
| 13 | 13 | ||
| 14 | static __read_mostly DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); | 14 | static __read_mostly DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); |
| 15 | 15 | ||
| 16 | /** | ||
| 17 | * devm_cxl_add_nvdimm_bridge() - add the root of a LIBNVDIMM topology | ||
| 18 | * @host: platform firmware root device | ||
| 19 | * @port: CXL port at the root of a CXL topology | ||
| 20 | * | ||
| 21 | * Return: bridge device that can host cxl_nvdimm objects | ||
| 22 | */ | ||
| 23 | struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, | ||
| 24 | struct cxl_port *port) | ||
| 25 | { | ||
| 26 | return __devm_cxl_add_nvdimm_bridge(host, port); | ||
| 27 | } | ||
| 28 | EXPORT_SYMBOL_NS_GPL(devm_cxl_add_nvdimm_bridge, "CXL"); | ||
| 29 | |||
| 16 | static void clear_exclusive(void *mds) | 30 | static void clear_exclusive(void *mds) |
| 17 | { | 31 | { |
| 18 | clear_exclusive_cxl_commands(mds, exclusive_cmds); | 32 | clear_exclusive_cxl_commands(mds, exclusive_cmds); |
| @@ -129,6 +143,9 @@ static int cxl_nvdimm_probe(struct device *dev) · 110 unmodified lines | |||
| 129 | struct nvdimm *nvdimm; | 143 | struct nvdimm *nvdimm; |
| 130 | int rc; | 144 | int rc; |
| 131 | 145 | ||
| 146 | if (test_bit(CXL_NVD_F_INVALIDATED, &cxl_nvd->flags)) | ||
| 147 | return -EBUSY; | ||
| 148 | |||
| 132 | set_exclusive_cxl_commands(mds, exclusive_cmds); | 149 | set_exclusive_cxl_commands(mds, exclusive_cmds); |
| 133 | rc = devm_add_action_or_reset(dev, clear_exclusive, mds); | 150 | rc = devm_add_action_or_reset(dev, clear_exclusive, mds); |
| 134 | if (rc) | 151 | if (rc) |
| @@ -309,8 +326,10 @@ static int detach_nvdimm(struct device *dev, void *data) · 174 unmodified lines | |||
| 309 | scoped_guard(device, dev) { | 326 | scoped_guard(device, dev) { |
| 310 | if (dev->driver) { | 327 | if (dev->driver) { |
| 311 | cxl_nvd = to_cxl_nvdimm(dev); | 328 | cxl_nvd = to_cxl_nvdimm(dev); |
| 312 | if (cxl_nvd->cxlmd && cxl_nvd->cxlmd->cxl_nvb == data) | 329 | if (cxl_nvd->cxlmd && cxl_nvd->cxlmd->cxl_nvb == data) { |
| 313 | release = true; | 330 | release = true; |
| 331 | set_bit(CXL_NVD_F_INVALIDATED, &cxl_nvd->flags); | ||
| 332 | } | ||
| 314 | } | 333 | } |
| 315 | } | 334 | } |
| 316 | if (release) | 335 | if (release) |
| @@ -353,6 +372,7 @@ static struct cxl_driver cxl_nvdimm_bridge_driver = { · 36 unmodified lines | |||
| 353 | .probe = cxl_nvdimm_bridge_probe, | 372 | .probe = cxl_nvdimm_bridge_probe, |
| 354 | .id = CXL_DEVICE_NVDIMM_BRIDGE, | 373 | .id = CXL_DEVICE_NVDIMM_BRIDGE, |
| 355 | .drv = { | 374 | .drv = { |
| 375 | .probe_type = PROBE_FORCE_SYNCHRONOUS, | ||
| 356 | .suppress_bind_attrs = true, | 376 | .suppress_bind_attrs = true, |
| 357 | }, | 377 | }, |
| 358 | }; | 378 | }; |
| @@ -981,11 +981,7 @@ zl3073x_devm_dpll_init(struct zl3073x_dev *zldev, u8 num_dplls) · 980 unmodified lines | |||
| 981 | } | 981 | } |
| 982 | 982 | ||
| 983 | /* Add devres action to release DPLL related resources */ | 983 | /* Add devres action to release DPLL related resources */ |
| 984 | rc = devm_add_action_or_reset(zldev->dev, zl3073x_dev_dpll_fini, zldev); | 984 | return devm_add_action_or_reset(zldev->dev, zl3073x_dev_dpll_fini, zldev); |
| 985 | if (rc) | ||
| 986 | goto error; | ||
| 987 | |||
| 988 | return 0; | ||
| 989 | 985 | ||
| 990 | error: | 986 | error: |
| 991 | zl3073x_dev_dpll_fini(zldev); | 987 | zl3073x_dev_dpll_fini(zldev); |
| @@ -1026,6 +1022,7 @@ int zl3073x_dev_probe(struct zl3073x_dev *zldev, · 34 unmodified lines | |||
| 1026 | "Unknown or non-match chip ID: 0x%0x\n", | 1022 | "Unknown or non-match chip ID: 0x%0x\n", |
| 1027 | id); | 1023 | id); |
| 1028 | } | 1024 | } |
| 1025 | zldev->chip_id = id; | ||
| 1029 | 1026 | ||
| 1030 | /* Read revision, firmware version and custom config version */ | 1027 | /* Read revision, firmware version and custom config version */ |
| 1031 | rc = zl3073x_read_u16(zldev, ZL_REG_REVISION, &revision); | 1028 | rc = zl3073x_read_u16(zldev, ZL_REG_REVISION, &revision); |
| @@ -35,6 +35,7 @@ struct zl3073x_dpll; · 34 unmodified lines | |||
| 35 | * @dev: pointer to device | 35 | * @dev: pointer to device |
| 36 | * @regmap: regmap to access device registers | 36 | * @regmap: regmap to access device registers |
| 37 | * @multiop_lock: to serialize multiple register operations | 37 | * @multiop_lock: to serialize multiple register operations |
| 38 | * @chip_id: chip ID read from hardware | ||
| 38 | * @ref: array of input references' invariants | 39 | * @ref: array of input references' invariants |
| 39 | * @out: array of outs' invariants | 40 | * @out: array of outs' invariants |
| 40 | * @synth: array of synths' invariants | 41 | * @synth: array of synths' invariants |
| @@ -48,6 +49,7 @@ struct zl3073x_dev { · 7 unmodified lines | |||
| 48 | struct device *dev; | 49 | struct device *dev; |
| 49 | struct regmap *regmap; | 50 | struct regmap *regmap; |
| 50 | struct mutex multiop_lock; | 51 | struct mutex multiop_lock; |
| 52 | u16 chip_id; | ||
| 51 | 53 | ||
| 52 | /* Invariants */ | 54 | /* Invariants */ |
| 53 | struct zl3073x_ref ref[ZL3073X_NUM_REFS]; | 55 | struct zl3073x_ref ref[ZL3073X_NUM_REFS]; |
| @@ -144,6 +146,32 @@ int zl3073x_write_hwreg_seq(struct zl3073x_dev *zldev, · 90 unmodified lines | |||
| 144 | 146 | ||
| 145 | int zl3073x_ref_phase_offsets_update(struct zl3073x_dev *zldev, int channel); | 147 | int zl3073x_ref_phase_offsets_update(struct zl3073x_dev *zldev, int channel); |
| 146 | 148 | ||
| 149 | /** | ||
| 150 | * zl3073x_dev_is_ref_phase_comp_32bit - check ref phase comp register size | ||
| 151 | * @zldev: pointer to zl3073x device | ||
| 152 | * | ||
| 153 | * Some chip IDs have a 32-bit wide ref_phase_offset_comp register instead | ||
| 154 | * of the default 48-bit. | ||
| 155 | * | ||
| 156 | * Return: true if the register is 32-bit, false if 48-bit | ||
| 157 | */ | ||
| 158 | static inline bool | ||
| 159 | zl3073x_dev_is_ref_phase_comp_32bit(struct zl3073x_dev *zldev) | ||
| 160 | { | ||
| 161 | switch (zldev->chip_id) { | ||
| 162 | case 0x0E30: | ||
| 163 | case 0x0E93: | ||
| 164 | case 0x0E94: | ||
| 165 | case 0x0E95: | ||
| 166 | case 0x0E96: | ||
| 167 | case 0x0E97: | ||
| 168 | case 0x1F60: | ||
| 169 | return true; | ||
| 170 | default: | ||
| 171 | return false; | ||
| 172 | } | ||
| 173 | } | ||
| 174 | |||
| 147 | static inline bool | 175 | static inline bool |
| 148 | zl3073x_is_n_pin(u8 id) | 176 | zl3073x_is_n_pin(u8 id) |
| 149 | { | 177 | { |
| @@ -475,8 +475,11 @@ zl3073x_dpll_input_pin_phase_adjust_get(const struct dpll_pin *dpll_pin, · 474 unmodified lines | |||
| 475 | ref_id = zl3073x_input_pin_ref_get(pin->id); | 475 | ref_id = zl3073x_input_pin_ref_get(pin->id); |
| 476 | ref = zl3073x_ref_state_get(zldev, ref_id); | 476 | ref = zl3073x_ref_state_get(zldev, ref_id); |
| 477 | 477 | ||
| 478 | /* Perform sign extension for 48bit signed value */ | 478 | /* Perform sign extension based on register width */ |
| 479 | phase_comp = sign_extend64(ref->phase_comp, 47); | 479 | if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) |
| 480 | phase_comp = sign_extend64(ref->phase_comp, 31); | ||
| 481 | else | ||
| 482 | phase_comp = sign_extend64(ref->phase_comp, 47); | ||
| 480 | 483 | ||
| 481 | /* Reverse two's complement negation applied during set and convert | 484 | /* Reverse two's complement negation applied during set and convert |
| 482 | * to 32bit signed int | 485 | * to 32bit signed int |
| @@ -121,8 +121,16 @@ int zl3073x_ref_state_fetch(struct zl3073x_dev *zldev, u8 index) · 120 unmodified lines | |||
| 121 | return rc; | 121 | return rc; |
| 122 | 122 | ||
| 123 | /* Read phase compensation register */ | 123 | /* Read phase compensation register */ |
| 124 | rc = zl3073x_read_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | 124 | if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) { |
| 125 | &ref->phase_comp); | 125 | u32 val; |
| 126 | |||
| 127 | rc = zl3073x_read_u32(zldev, ZL_REG_REF_PHASE_OFFSET_COMP_32, | ||
| 128 | &val); | ||
| 129 | ref->phase_comp = val; | ||
| 130 | } else { | ||
| 131 | rc = zl3073x_read_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | ||
| 132 | &ref->phase_comp); | ||
| 133 | } | ||
| 126 | if (rc) | 134 | if (rc) |
| 127 | return rc; | 135 | return rc; |
| 128 | 136 | ||
| @@ -179,9 +187,16 @@ int zl3073x_ref_state_set(struct zl3073x_dev *zldev, u8 index, · 50 unmodified lines | |||
| 179 | if (!rc && dref->sync_ctrl != ref->sync_ctrl) | 187 | if (!rc && dref->sync_ctrl != ref->sync_ctrl) |
| 180 | rc = zl3073x_write_u8(zldev, ZL_REG_REF_SYNC_CTRL, | 188 | rc = zl3073x_write_u8(zldev, ZL_REG_REF_SYNC_CTRL, |
| 181 | ref->sync_ctrl); | 189 | ref->sync_ctrl); |
| 182 | if (!rc && dref->phase_comp != ref->phase_comp) | 190 | if (!rc && dref->phase_comp != ref->phase_comp) { |
| 183 | rc = zl3073x_write_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | 191 | if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) |
| 184 | ref->phase_comp); | 192 | rc = zl3073x_write_u32(zldev, |
| 193 | ZL_REG_REF_PHASE_OFFSET_COMP_32, | ||
| 194 | ref->phase_comp); | ||
| 195 | else | ||
| 196 | rc = zl3073x_write_u48(zldev, | ||
| 197 | ZL_REG_REF_PHASE_OFFSET_COMP, | ||
| 198 | ref->phase_comp); | ||
| 199 | } | ||
| 185 | if (rc) | 200 | if (rc) |
| 186 | return rc; | 201 | return rc; |
| 187 | 202 | ||
| @@ -194,6 +194,7 @@ · 193 unmodified lines | |||
| 194 | #define ZL_REF_CONFIG_DIFF_EN BIT(2) | 194 | #define ZL_REF_CONFIG_DIFF_EN BIT(2) |
| 195 | 195 | ||
| 196 | #define ZL_REG_REF_PHASE_OFFSET_COMP ZL_REG(10, 0x28, 6) | 196 | #define ZL_REG_REF_PHASE_OFFSET_COMP ZL_REG(10, 0x28, 6) |
| 197 | #define ZL_REG_REF_PHASE_OFFSET_COMP_32 ZL_REG(10, 0x28, 4) | ||
| 197 | 198 | ||
| 198 | #define ZL_REG_REF_SYNC_CTRL ZL_REG(10, 0x2e, 1) | 199 | #define ZL_REG_REF_SYNC_CTRL ZL_REG(10, 0x2e, 1) |
| 199 | #define ZL_REF_SYNC_CTRL_MODE GENMASK(2, 0) | 200 | #define ZL_REF_SYNC_CTRL_MODE GENMASK(2, 0) |
| @@ -848,7 +848,7 @@ static int ar_context_init(struct ar_context *ctx, struct fw_ohci *ohci, · 847 unmodified lines | |||
| 848 | { | 848 | { |
| 849 | struct device *dev = ohci->card.device; | 849 | struct device *dev = ohci->card.device; |
| 850 | unsigned int i; | 850 | unsigned int i; |
| 851 | struct page *pages[AR_BUFFERS + AR_WRAPAROUND_PAGES]; | 851 | struct page *pages[AR_BUFFERS + AR_WRAPAROUND_PAGES] = { NULL }; |
| 852 | dma_addr_t dma_addrs[AR_BUFFERS]; | 852 | dma_addr_t dma_addrs[AR_BUFFERS]; |
| 853 | void *vaddr; | 853 | void *vaddr; |
| 854 | struct descriptor *d; | 854 | struct descriptor *d; |
| @@ -1610,11 +1610,17 @@ static int cs_dsp_load(struct cs_dsp *dsp, const struct firmware *firmware, · 1609 unmodified lines | |||
| 1610 | region_name); | 1610 | region_name); |
| 1611 | 1611 | ||
| 1612 | if (reg) { | 1612 | if (reg) { |
| 1613 | /* | ||
| 1614 | * Although we expect the underlying bus does not require | ||
| 1615 | * physically-contiguous buffers, we pessimistically use | ||
| 1616 | * a temporary buffer instead of trusting that the | ||
| 1617 | * alignment of region->data is ok. | ||
| 1618 | */ | ||
| 1613 | region_len = le32_to_cpu(region->len); | 1619 | region_len = le32_to_cpu(region->len); |
| 1614 | if (region_len > buf_len) { | 1620 | if (region_len > buf_len) { |
| 1615 | buf_len = round_up(region_len, PAGE_SIZE); | 1621 | buf_len = round_up(region_len, PAGE_SIZE); |
| 1616 | kfree(buf); | 1622 | vfree(buf); |
| 1617 | buf = kmalloc(buf_len, GFP_KERNEL | GFP_DMA); | 1623 | buf = vmalloc(buf_len); |
| 1618 | if (!buf) { | 1624 | if (!buf) { |
| 1619 | ret = -ENOMEM; | 1625 | ret = -ENOMEM; |
| 1620 | goto out_fw; | 1626 | goto out_fw; |
| @@ -1643,7 +1649,7 @@ static int cs_dsp_load(struct cs_dsp *dsp, const struct firmware *firmware, · 22 unmodified lines | |||
| 1643 | 1649 | ||
| 1644 | ret = 0; | 1650 | ret = 0; |
| 1645 | out_fw: | 1651 | out_fw: |
| 1646 | kfree(buf); | 1652 | vfree(buf); |
| 1647 | 1653 | ||
| 1648 | if (ret == -EOVERFLOW) | 1654 | if (ret == -EOVERFLOW) |
| 1649 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); | 1655 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); |
| @@ -2331,11 +2337,17 @@ static int cs_dsp_load_coeff(struct cs_dsp *dsp, const struct firmware *firmware · 681 unmodified lines | |||
| 2331 | } | 2337 | } |
| 2332 | 2338 | ||
| 2333 | if (reg) { | 2339 | if (reg) { |
| 2340 | /* | ||
| 2341 | * Although we expect the underlying bus does not require | ||
| 2342 | * physically-contiguous buffers, we pessimistically use | ||
| 2343 | * a temporary buffer instead of trusting that the | ||
| 2344 | * alignment of blk->data is ok. | ||
| 2345 | */ | ||
| 2334 | region_len = le32_to_cpu(blk->len); | 2346 | region_len = le32_to_cpu(blk->len); |
| 2335 | if (region_len > buf_len) { | 2347 | if (region_len > buf_len) { |
| 2336 | buf_len = round_up(region_len, PAGE_SIZE); | 2348 | buf_len = round_up(region_len, PAGE_SIZE); |
| 2337 | kfree(buf); | 2349 | vfree(buf); |
| 2338 | buf = kmalloc(buf_len, GFP_KERNEL | GFP_DMA); | 2350 | buf = vmalloc(buf_len); |
| 2339 | if (!buf) { | 2351 | if (!buf) { |
| 2340 | ret = -ENOMEM; | 2352 | ret = -ENOMEM; |
| 2341 | goto out_fw; | 2353 | goto out_fw; |
| @@ -2366,7 +2378,7 @@ static int cs_dsp_load_coeff(struct cs_dsp *dsp, const struct firmware *firmware · 24 unmodified lines | |||
| 2366 | 2378 | ||
| 2367 | ret = 0; | 2379 | ret = 0; |
| 2368 | out_fw: | 2380 | out_fw: |
| 2369 | kfree(buf); | 2381 | vfree(buf); |
| 2370 | 2382 | ||
| 2371 | if (ret == -EOVERFLOW) | 2383 | if (ret == -EOVERFLOW) |
| 2372 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); | 2384 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); |
| @@ -85,7 +85,7 @@ static struct kobject *mokvar_kobj; · 84 unmodified lines | |||
| 85 | * as an alternative to ordinary EFI variables, due to platform-dependent | 85 | * as an alternative to ordinary EFI variables, due to platform-dependent |
| 86 | * limitations. The memory occupied by this table is marked as reserved. | 86 | * limitations. The memory occupied by this table is marked as reserved. |
| 87 | * | 87 | * |
| 88 | * This routine must be called before efi_free_boot_services() in order | 88 | * This routine must be called before efi_unmap_boot_services() in order |
| 89 | * to guarantee that it can mark the table as reserved. | 89 | * to guarantee that it can mark the table as reserved. |
| 90 | * | 90 | * |
| 91 | * Implicit inputs: | 91 | * Implicit inputs: |
| @@ -768,7 +768,9 @@ static int stratix10_rsu_probe(struct platform_device *pdev) · 767 unmodified lines | |||
| 768 | rsu_async_status_callback); | 768 | rsu_async_status_callback); |
| 769 | if (ret) { | 769 | if (ret) { |
| 770 | dev_err(dev, "Error, getting RSU status %i\n", ret); | 770 | dev_err(dev, "Error, getting RSU status %i\n", ret); |
| 771 | stratix10_svc_remove_async_client(priv->chan); | ||
| 771 | stratix10_svc_free_channel(priv->chan); | 772 | stratix10_svc_free_channel(priv->chan); |
| 773 | return ret; | ||
| 772 | } | 774 | } |
| 773 | 775 | ||
| 774 | /* get DCMF version from firmware */ | 776 | /* get DCMF version from firmware */ |
| @@ -37,15 +37,14 @@ · 36 unmodified lines | |||
| 37 | * service layer will return error to FPGA manager when timeout occurs, | 37 | * service layer will return error to FPGA manager when timeout occurs, |
| 38 | * timeout is set to 30 seconds (30 * 1000) at Intel Stratix10 SoC. | 38 | * timeout is set to 30 seconds (30 * 1000) at Intel Stratix10 SoC. |
| 39 | */ | 39 | */ |
| 40 | #define SVC_NUM_DATA_IN_FIFO 32 | 40 | #define SVC_NUM_DATA_IN_FIFO 8 |
| 41 | #define SVC_NUM_CHANNEL 4 | 41 | #define SVC_NUM_CHANNEL 4 |
| 42 | #define FPGA_CONFIG_DATA_CLAIM_TIMEOUT_MS 200 | 42 | #define FPGA_CONFIG_DATA_CLAIM_TIMEOUT_MS 2000 |
| 43 | #define FPGA_CONFIG_STATUS_TIMEOUT_SEC 30 | 43 | #define FPGA_CONFIG_STATUS_TIMEOUT_SEC 30 |
| 44 | #define BYTE_TO_WORD_SIZE 4 | 44 | #define BYTE_TO_WORD_SIZE 4 |
| 45 | 45 | ||
| 46 | /* stratix10 service layer clients */ | 46 | /* stratix10 service layer clients */ |
| 47 | #define STRATIX10_RSU "stratix10-rsu" | 47 | #define STRATIX10_RSU "stratix10-rsu" |
| 48 | #define INTEL_FCS "intel-fcs" | ||
| 49 | 48 | ||
| 50 | /* Maximum number of SDM client IDs. */ | 49 | /* Maximum number of SDM client IDs. */ |
| 51 | #define MAX_SDM_CLIENT_IDS 16 | 50 | #define MAX_SDM_CLIENT_IDS 16 |
| @@ -105,11 +104,9 @@ struct stratix10_svc_chan; · 53 unmodified lines | |||
| 105 | /** | 104 | /** |
| 106 | * struct stratix10_svc - svc private data | 105 | * struct stratix10_svc - svc private data |
| 107 | * @stratix10_svc_rsu: pointer to stratix10 RSU device | 106 | * @stratix10_svc_rsu: pointer to stratix10 RSU device |
| 108 | * @intel_svc_fcs: pointer to the FCS device | ||
| 109 | */ | 107 | */ |
| 110 | struct stratix10_svc { | 108 | struct stratix10_svc { |
| 111 | struct platform_device *stratix10_svc_rsu; | 109 | struct platform_device *stratix10_svc_rsu; |
| 112 | struct platform_device *intel_svc_fcs; | ||
| 113 | }; | 110 | }; |
| 114 | 111 | ||
| 115 | /** | 112 | /** |
| @@ -251,12 +248,10 @@ struct stratix10_async_ctrl { · 135 unmodified lines | |||
| 251 | * @num_active_client: number of active service client | 248 | * @num_active_client: number of active service client |
| 252 | * @node: list management | 249 | * @node: list management |
| 253 | * @genpool: memory pool pointing to the memory region | 250 | * @genpool: memory pool pointing to the memory region |
| 254 | * @task: pointer to the thread task which handles SMC or HVC call | ||
| 255 | * @svc_fifo: a queue for storing service message data | ||
| 256 | * @complete_status: state for completion | 251 | * @complete_status: state for completion |
| 257 | * @svc_fifo_lock: protect access to service message data queue | ||
| 258 | * @invoke_fn: function to issue secure monitor call or hypervisor call | 252 | * @invoke_fn: function to issue secure monitor call or hypervisor call |
| 259 | * @svc: manages the list of client svc drivers | 253 | * @svc: manages the list of client svc drivers |
| 254 | * @sdm_lock: only allows a single command single response to SDM | ||
| 260 | * @actrl: async control structure | 255 | * @actrl: async control structure |
| 261 | * | 256 | * |
| 262 | * This struct is used to create communication channels for service clients, to | 257 | * This struct is used to create communication channels for service clients, to |
| @@ -269,12 +264,10 @@ struct stratix10_svc_controller { · 6 unmodified lines | |||
| 269 | int num_active_client; | 264 | int num_active_client; |
| 270 | struct list_head node; | 265 | struct list_head node; |
| 271 | struct gen_pool *genpool; | 266 | struct gen_pool *genpool; |
| 272 | struct task_struct *task; | ||
| 273 | struct kfifo svc_fifo; | ||
| 274 | struct completion complete_status; | 267 | struct completion complete_status; |
| 275 | spinlock_t svc_fifo_lock; | ||
| 276 | svc_invoke_fn *invoke_fn; | 268 | svc_invoke_fn *invoke_fn; |
| 277 | struct stratix10_svc *svc; | 269 | struct stratix10_svc *svc; |
| 270 | struct mutex sdm_lock; | ||
| 278 | struct stratix10_async_ctrl actrl; | 271 | struct stratix10_async_ctrl actrl; |
| 279 | }; | 272 | }; |
| 280 | 273 | ||
| @@ -283,6 +276,9 @@ struct stratix10_svc_controller { · 2 unmodified lines | |||
| 283 | * @ctrl: pointer to service controller which is the provider of this channel | 276 | * @ctrl: pointer to service controller which is the provider of this channel |
| 284 | * @scl: pointer to service client which owns the channel | 277 | * @scl: pointer to service client which owns the channel |
| 285 | * @name: service client name associated with the channel | 278 | * @name: service client name associated with the channel |
| 279 | * @task: pointer to the thread task which handles SMC or HVC call | ||
| 280 | * @svc_fifo: a queue for storing service message data (separate fifo for every channel) | ||
| 281 | * @svc_fifo_lock: protect access to service message data queue (locking pending fifo) | ||
| 286 | * @lock: protect access to the channel | 282 | * @lock: protect access to the channel |
| 287 | * @async_chan: reference to asynchronous channel object for this channel | 283 | * @async_chan: reference to asynchronous channel object for this channel |
| 288 | * | 284 | * |
| @@ -293,6 +289,9 @@ struct stratix10_svc_chan { · 4 unmodified lines | |||
| 293 | struct stratix10_svc_controller *ctrl; | 289 | struct stratix10_svc_controller *ctrl; |
| 294 | struct stratix10_svc_client *scl; | 290 | struct stratix10_svc_client *scl; |
| 295 | char *name; | 291 | char *name; |
| 292 | struct task_struct *task; | ||
| 293 | struct kfifo svc_fifo; | ||
| 294 | spinlock_t svc_fifo_lock; | ||
| 296 | spinlock_t lock; | 295 | spinlock_t lock; |
| 297 | struct stratix10_async_chan *async_chan; | 296 | struct stratix10_async_chan *async_chan; |
| 298 | }; | 297 | }; |
| @@ -527,10 +526,10 @@ static void svc_thread_recv_status_ok(struct stratix10_svc_data *p_data, · 228 unmodified lines | |||
| 527 | */ | 526 | */ |
| 528 | static int svc_normal_to_secure_thread(void *data) | 527 | static int svc_normal_to_secure_thread(void *data) |
| 529 | { | 528 | { |
| 530 | struct stratix10_svc_controller | 529 | struct stratix10_svc_chan *chan = (struct stratix10_svc_chan *)data; |
| 531 | *ctrl = (struct stratix10_svc_controller *)data; | 530 | struct stratix10_svc_controller *ctrl = chan->ctrl; |
| 532 | struct stratix10_svc_data *pdata; | 531 | struct stratix10_svc_data *pdata = NULL; |
| 533 | struct stratix10_svc_cb_data *cbdata; | 532 | struct stratix10_svc_cb_data *cbdata = NULL; |
| 534 | struct arm_smccc_res res; | 533 | struct arm_smccc_res res; |
| 535 | unsigned long a0, a1, a2, a3, a4, a5, a6, a7; | 534 | unsigned long a0, a1, a2, a3, a4, a5, a6, a7; |
| 536 | int ret_fifo = 0; | 535 | int ret_fifo = 0; |
| @@ -555,12 +554,12 @@ static int svc_normal_to_secure_thread(void *data) · 18 unmodified lines | |||
| 555 | a6 = 0; | 554 | a6 = 0; |
| 556 | a7 = 0; | 555 | a7 = 0; |
| 557 | 556 | ||
| 558 | pr_debug("smc_hvc_shm_thread is running\n"); | 557 | pr_debug("%s: %s: Thread is running!\n", __func__, chan->name); |
| 559 | 558 | ||
| 560 | while (!kthread_should_stop()) { | 559 | while (!kthread_should_stop()) { |
| 561 | ret_fifo = kfifo_out_spinlocked(&ctrl->svc_fifo, | 560 | ret_fifo = kfifo_out_spinlocked(&chan->svc_fifo, |
| 562 | pdata, sizeof(*pdata), | 561 | pdata, sizeof(*pdata), |
| 563 | &ctrl->svc_fifo_lock); | 562 | &chan->svc_fifo_lock); |
| 564 | 563 | ||
| 565 | if (!ret_fifo) | 564 | if (!ret_fifo) |
| 566 | continue; | 565 | continue; |
| @@ -569,9 +568,25 @@ static int svc_normal_to_secure_thread(void *data) · 2 unmodified lines | |||
| 569 | (unsigned int)pdata->paddr, pdata->command, | 568 | (unsigned int)pdata->paddr, pdata->command, |
| 570 | (unsigned int)pdata->size); | 569 | (unsigned int)pdata->size); |
| 571 | 570 | ||
| 571 | /* SDM can only process one command at a time */ | ||
| 572 | pr_debug("%s: %s: Thread is waiting for mutex!\n", | ||
| 573 | __func__, chan->name); | ||
| 574 | if (mutex_lock_interruptible(&ctrl->sdm_lock)) { | ||
| 575 | /* item already dequeued; notify client to unblock it */ | ||
| 576 | cbdata->status = BIT(SVC_STATUS_ERROR); | ||
| 577 | cbdata->kaddr1 = NULL; | ||
| 578 | cbdata->kaddr2 = NULL; | ||
| 579 | cbdata->kaddr3 = NULL; | ||
| 580 | if (pdata->chan->scl) | ||
| 581 | pdata->chan->scl->receive_cb(pdata->chan->scl, | ||
| 582 | cbdata); | ||
| 583 | break; | ||
| 584 | } | ||
| 585 | |||
| 572 | switch (pdata->command) { | 586 | switch (pdata->command) { |
| 573 | case COMMAND_RECONFIG_DATA_CLAIM: | 587 | case COMMAND_RECONFIG_DATA_CLAIM: |
| 574 | svc_thread_cmd_data_claim(ctrl, pdata, cbdata); | 588 | svc_thread_cmd_data_claim(ctrl, pdata, cbdata); |
| 589 | mutex_unlock(&ctrl->sdm_lock); | ||
| 575 | continue; | 590 | continue; |
| 576 | case COMMAND_RECONFIG: | 591 | case COMMAND_RECONFIG: |
| 577 | a0 = INTEL_SIP_SMC_FPGA_CONFIG_START; | 592 | a0 = INTEL_SIP_SMC_FPGA_CONFIG_START; |
| @@ -700,10 +715,11 @@ static int svc_normal_to_secure_thread(void *data) · 122 unmodified lines | |||
| 700 | break; | 715 | break; |
| 701 | default: | 716 | default: |
| 702 | pr_warn("it shouldn't happen\n"); | 717 | pr_warn("it shouldn't happen\n"); |
| 703 | break; | 718 | mutex_unlock(&ctrl->sdm_lock); |
| 719 | continue; | ||
| 704 | } | 720 | } |
| 705 | pr_debug("%s: before SMC call -- a0=0x%016x a1=0x%016x", | 721 | pr_debug("%s: %s: before SMC call -- a0=0x%016x a1=0x%016x", |
| 706 | __func__, | 722 | __func__, chan->name, |
| 707 | (unsigned int)a0, | 723 | (unsigned int)a0, |
| 708 | (unsigned int)a1); | 724 | (unsigned int)a1); |
| 709 | pr_debug(" a2=0x%016x\n", (unsigned int)a2); | 725 | pr_debug(" a2=0x%016x\n", (unsigned int)a2); |
| @@ -712,8 +728,8 @@ static int svc_normal_to_secure_thread(void *data) · 2 unmodified lines | |||
| 712 | pr_debug(" a5=0x%016x\n", (unsigned int)a5); | 728 | pr_debug(" a5=0x%016x\n", (unsigned int)a5); |
| 713 | ctrl->invoke_fn(a0, a1, a2, a3, a4, a5, a6, a7, &res); | 729 | ctrl->invoke_fn(a0, a1, a2, a3, a4, a5, a6, a7, &res); |
| 714 | 730 | ||
| 715 | pr_debug("%s: after SMC call -- res.a0=0x%016x", | 731 | pr_debug("%s: %s: after SMC call -- res.a0=0x%016x", |
| 716 | __func__, (unsigned int)res.a0); | 732 | __func__, chan->name, (unsigned int)res.a0); |
| 717 | pr_debug(" res.a1=0x%016x, res.a2=0x%016x", | 733 | pr_debug(" res.a1=0x%016x, res.a2=0x%016x", |
| 718 | (unsigned int)res.a1, (unsigned int)res.a2); | 734 | (unsigned int)res.a1, (unsigned int)res.a2); |
| 719 | pr_debug(" res.a3=0x%016x\n", (unsigned int)res.a3); | 735 | pr_debug(" res.a3=0x%016x\n", (unsigned int)res.a3); |
| @@ -728,6 +744,7 @@ static int svc_normal_to_secure_thread(void *data) · 8 unmodified lines | |||
| 728 | cbdata->kaddr2 = NULL; | 744 | cbdata->kaddr2 = NULL; |
| 729 | cbdata->kaddr3 = NULL; | 745 | cbdata->kaddr3 = NULL; |
| 730 | pdata->chan->scl->receive_cb(pdata->chan->scl, cbdata); | 746 | pdata->chan->scl->receive_cb(pdata->chan->scl, cbdata); |
| 747 | mutex_unlock(&ctrl->sdm_lock); | ||
| 731 | continue; | 748 | continue; |
| 732 | } | 749 | } |
| 733 | 750 | ||
| @@ -801,6 +818,8 @@ static int svc_normal_to_secure_thread(void *data) · 67 unmodified lines | |||
| 801 | break; | 818 | break; |
| 802 | 819 | ||
| 803 | } | 820 | } |
| 821 | |||
| 822 | mutex_unlock(&ctrl->sdm_lock); | ||
| 804 | } | 823 | } |
| 805 | 824 | ||
| 806 | kfree(cbdata); | 825 | kfree(cbdata); |
| @@ -1696,22 +1715,33 @@ int stratix10_svc_send(struct stratix10_svc_chan *chan, void *msg) · 889 unmodified lines | |||
| 1696 | if (!p_data) | 1715 | if (!p_data) |
| 1697 | return -ENOMEM; | 1716 | return -ENOMEM; |
| 1698 | 1717 | ||
| 1699 | /* first client will create kernel thread */ | 1718 | /* first caller creates the per-channel kthread */ |
| 1700 | if (!chan->ctrl->task) { | 1719 | if (!chan->task) { |
| 1701 | chan->ctrl->task = | 1720 | struct task_struct *task; |
| 1702 | kthread_run_on_cpu(svc_normal_to_secure_thread, | 1721 | |
| 1703 | (void *)chan->ctrl, | 1722 | task = kthread_run_on_cpu(svc_normal_to_secure_thread, |
| 1704 | cpu, "svc_smc_hvc_thread"); | 1723 | (void *)chan, |
| 1705 | if (IS_ERR(chan->ctrl->task)) { | 1724 | cpu, "svc_smc_hvc_thread"); |
| 1725 | if (IS_ERR(task)) { | ||
| 1706 | dev_err(chan->ctrl->dev, | 1726 | dev_err(chan->ctrl->dev, |
| 1707 | "failed to create svc_smc_hvc_thread\n"); | 1727 | "failed to create svc_smc_hvc_thread\n"); |
| 1708 | kfree(p_data); | 1728 | kfree(p_data); |
| 1709 | return -EINVAL; | 1729 | return -EINVAL; |
| 1710 | } | 1730 | } |
| 1731 | |||
| 1732 | spin_lock(&chan->lock); | ||
| 1733 | if (chan->task) { | ||
| 1734 | /* another caller won the race; discard our thread */ | ||
| 1735 | spin_unlock(&chan->lock); | ||
| 1736 | kthread_stop(task); | ||
| 1737 | } else { | ||
| 1738 | chan->task = task; | ||
| 1739 | spin_unlock(&chan->lock); | ||
| 1740 | } | ||
| 1711 | } | 1741 | } |
| 1712 | 1742 | ||
| 1713 | pr_debug("%s: sent P-va=%p, P-com=%x, P-size=%u\n", __func__, | 1743 | pr_debug("%s: %s: sent P-va=%p, P-com=%x, P-size=%u\n", __func__, |
| 1714 | p_msg->payload, p_msg->command, | 1744 | chan->name, p_msg->payload, p_msg->command, |
| 1715 | (unsigned int)p_msg->payload_length); | 1745 | (unsigned int)p_msg->payload_length); |
| 1716 | 1746 | ||
| 1717 | if (list_empty(&svc_data_mem)) { | 1747 | if (list_empty(&svc_data_mem)) { |
| @@ -1747,12 +1777,16 @@ int stratix10_svc_send(struct stratix10_svc_chan *chan, void *msg) · 29 unmodified lines | |||
| 1747 | p_data->arg[2] = p_msg->arg[2]; | 1777 | p_data->arg[2] = p_msg->arg[2]; |
| 1748 | p_data->size = p_msg->payload_length; | 1778 | p_data->size = p_msg->payload_length; |
| 1749 | p_data->chan = chan; | 1779 | p_data->chan = chan; |
| 1750 | pr_debug("%s: put to FIFO pa=0x%016x, cmd=%x, size=%u\n", __func__, | 1780 | pr_debug("%s: %s: put to FIFO pa=0x%016x, cmd=%x, size=%u\n", |
| 1751 | (unsigned int)p_data->paddr, p_data->command, | 1781 | __func__, |
| 1752 | (unsigned int)p_data->size); | 1782 | chan->name, |
| 1753 | ret = kfifo_in_spinlocked(&chan->ctrl->svc_fifo, p_data, | 1783 | (unsigned int)p_data->paddr, |
| 1784 | p_data->command, | ||
| 1785 | (unsigned int)p_data->size); | ||
| 1786 | |||
| 1787 | ret = kfifo_in_spinlocked(&chan->svc_fifo, p_data, | ||
| 1754 | sizeof(*p_data), | 1788 | sizeof(*p_data), |
| 1755 | &chan->ctrl->svc_fifo_lock); | 1789 | &chan->svc_fifo_lock); |
| 1756 | 1790 | ||
| 1757 | kfree(p_data); | 1791 | kfree(p_data); |
| 1758 | 1792 | ||
| @@ -1773,11 +1807,12 @@ EXPORT_SYMBOL_GPL(stratix10_svc_send); · 14 unmodified lines | |||
| 1773 | */ | 1807 | */ |
| 1774 | void stratix10_svc_done(struct stratix10_svc_chan *chan) | 1808 | void stratix10_svc_done(struct stratix10_svc_chan *chan) |
| 1775 | { | 1809 | { |
| 1776 | /* stop thread when thread is running AND only one active client */ | 1810 | /* stop thread when thread is running */ |
| 1777 | if (chan->ctrl->task && chan->ctrl->num_active_client <= 1) { | 1811 | if (chan->task) { |
| 1778 | pr_debug("svc_smc_hvc_shm_thread is stopped\n"); | 1812 | pr_debug("%s: %s: svc_smc_hvc_shm_thread is stopping\n", |
| 1779 | kthread_stop(chan->ctrl->task); | 1813 | __func__, chan->name); |
| 1780 | chan->ctrl->task = NULL; | 1814 | kthread_stop(chan->task); |
| 1815 | chan->task = NULL; | ||
| 1781 | } | 1816 | } |
| 1782 | } | 1817 | } |
| 1783 | EXPORT_SYMBOL_GPL(stratix10_svc_done); | 1818 | EXPORT_SYMBOL_GPL(stratix10_svc_done); |
| @@ -1817,8 +1852,8 @@ void *stratix10_svc_allocate_memory(struct stratix10_svc_chan *chan, · 33 unmodified lines | |||
| 1817 | pmem->paddr = pa; | 1852 | pmem->paddr = pa; |
| 1818 | pmem->size = s; | 1853 | pmem->size = s; |
| 1819 | list_add_tail(&pmem->node, &svc_data_mem); | 1854 | list_add_tail(&pmem->node, &svc_data_mem); |
| 1820 | pr_debug("%s: va=%p, pa=0x%016x\n", __func__, | 1855 | pr_debug("%s: %s: va=%p, pa=0x%016x\n", __func__, |
| 1821 | pmem->vaddr, (unsigned int)pmem->paddr); | 1856 | chan->name, pmem->vaddr, (unsigned int)pmem->paddr); |
| 1822 | 1857 | ||
| 1823 | return (void *)va; | 1858 | return (void *)va; |
| 1824 | } | 1859 | } |
| @@ -1855,6 +1890,13 @@ static const struct of_device_id stratix10_svc_drv_match[] = { · 30 unmodified lines | |||
| 1855 | {}, | 1890 | {}, |
| 1856 | }; | 1891 | }; |
| 1857 | 1892 | ||
| 1893 | static const char * const chan_names[SVC_NUM_CHANNEL] = { | ||
| 1894 | SVC_CLIENT_FPGA, | ||
| 1895 | SVC_CLIENT_RSU, | ||
| 1896 | SVC_CLIENT_FCS, | ||
| 1897 | SVC_CLIENT_HWMON | ||
| 1898 | }; | ||
| 1899 | |||
| 1858 | static int stratix10_svc_drv_probe(struct platform_device *pdev) | 1900 | static int stratix10_svc_drv_probe(struct platform_device *pdev) |
| 1859 | { | 1901 | { |
| 1860 | struct device *dev = &pdev->dev; | 1902 | struct device *dev = &pdev->dev; |
| @@ -1862,11 +1904,11 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 1 unmodified line | |||
| 1862 | struct stratix10_svc_chan *chans; | 1904 | struct stratix10_svc_chan *chans; |
| 1863 | struct gen_pool *genpool; | 1905 | struct gen_pool *genpool; |
| 1864 | struct stratix10_svc_sh_memory *sh_memory; | 1906 | struct stratix10_svc_sh_memory *sh_memory; |
| 1865 | struct stratix10_svc *svc; | 1907 | struct stratix10_svc *svc = NULL; |
| 1866 | 1908 | ||
| 1867 | svc_invoke_fn *invoke_fn; | 1909 | svc_invoke_fn *invoke_fn; |
| 1868 | size_t fifo_size; | 1910 | size_t fifo_size; |
| 1869 | int ret; | 1911 | int ret, i = 0; |
| 1870 | 1912 | ||
| 1871 | /* get SMC or HVC function */ | 1913 | /* get SMC or HVC function */ |
| 1872 | invoke_fn = get_invoke_func(dev); | 1914 | invoke_fn = get_invoke_func(dev); |
| @@ -1905,8 +1947,8 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 32 unmodified lines | |||
| 1905 | controller->num_active_client = 0; | 1947 | controller->num_active_client = 0; |
| 1906 | controller->chans = chans; | 1948 | controller->chans = chans; |
| 1907 | controller->genpool = genpool; | 1949 | controller->genpool = genpool; |
| 1908 | controller->task = NULL; | ||
| 1909 | controller->invoke_fn = invoke_fn; | 1950 | controller->invoke_fn = invoke_fn; |
| 1951 | INIT_LIST_HEAD(&controller->node); | ||
| 1910 | init_completion(&controller->complete_status); | 1952 | init_completion(&controller->complete_status); |
| 1911 | 1953 | ||
| 1912 | ret = stratix10_svc_async_init(controller); | 1954 | ret = stratix10_svc_async_init(controller); |
| @@ -1917,32 +1959,20 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 4 unmodified lines | |||
| 1917 | } | 1959 | } |
| 1918 | 1960 | ||
| 1919 | fifo_size = sizeof(struct stratix10_svc_data) * SVC_NUM_DATA_IN_FIFO; | 1961 | fifo_size = sizeof(struct stratix10_svc_data) * SVC_NUM_DATA_IN_FIFO; |
| 1920 | ret = kfifo_alloc(&controller->svc_fifo, fifo_size, GFP_KERNEL); | 1962 | mutex_init(&controller->sdm_lock); |
| 1921 | if (ret) { | ||
| 1922 | dev_err(dev, "failed to allocate FIFO\n"); | ||
| 1923 | goto err_async_exit; | ||
| 1924 | } | ||
| 1925 | spin_lock_init(&controller->svc_fifo_lock); | ||
| 1926 | |||
| 1927 | chans[0].scl = NULL; | ||
| 1928 | chans[0].ctrl = controller; | ||
| 1929 | chans[0].name = SVC_CLIENT_FPGA; | ||
| 1930 | spin_lock_init(&chans[0].lock); | ||
| 1931 | 1963 | ||
| 1932 | chans[1].scl = NULL; | 1964 | for (i = 0; i < SVC_NUM_CHANNEL; i++) { |
| 1933 | chans[1].ctrl = controller; | 1965 | chans[i].scl = NULL; |
| 1934 | chans[1].name = SVC_CLIENT_RSU; | 1966 | chans[i].ctrl = controller; |
| 1935 | spin_lock_init(&chans[1].lock); | 1967 | chans[i].name = (char *)chan_names[i]; |
| 1936 | 1968 | spin_lock_init(&chans[i].lock); | |
| 1937 | chans[2].scl = NULL; | 1969 | ret = kfifo_alloc(&chans[i].svc_fifo, fifo_size, GFP_KERNEL); |
| 1938 | chans[2].ctrl = controller; | 1970 | if (ret) { |
| 1939 | chans[2].name = SVC_CLIENT_FCS; | 1971 | dev_err(dev, "failed to allocate FIFO %d\n", i); |
| 1940 | spin_lock_init(&chans[2].lock); | 1972 | goto err_free_fifos; |
| 1941 | 1973 | } | |
| 1942 | chans[3].scl = NULL; | 1974 | spin_lock_init(&chans[i].svc_fifo_lock); |
| 1943 | chans[3].ctrl = controller; | 1975 | } |
| 1944 | chans[3].name = SVC_CLIENT_HWMON; | ||
| 1945 | spin_lock_init(&chans[3].lock); | ||
| 1946 | 1976 | ||
| 1947 | list_add_tail(&controller->node, &svc_ctrl); | 1977 | list_add_tail(&controller->node, &svc_ctrl); |
| 1948 | platform_set_drvdata(pdev, controller); | 1978 | platform_set_drvdata(pdev, controller); |
| @@ -1951,7 +1981,7 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 2 unmodified lines | |||
| 1951 | svc = devm_kzalloc(dev, sizeof(*svc), GFP_KERNEL); | 1981 | svc = devm_kzalloc(dev, sizeof(*svc), GFP_KERNEL); |
| 1952 | if (!svc) { | 1982 | if (!svc) { |
| 1953 | ret = -ENOMEM; | 1983 | ret = -ENOMEM; |
| 1954 | goto err_free_kfifo; | 1984 | goto err_free_fifos; |
| 1955 | } | 1985 | } |
| 1956 | controller->svc = svc; | 1986 | controller->svc = svc; |
| 1957 | 1987 | ||
| @@ -1959,51 +1989,43 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 1 unmodified line | |||
| 1959 | if (!svc->stratix10_svc_rsu) { | 1989 | if (!svc->stratix10_svc_rsu) { |
| 1960 | dev_err(dev, "failed to allocate %s device\n", STRATIX10_RSU); | 1990 | dev_err(dev, "failed to allocate %s device\n", STRATIX10_RSU); |
| 1961 | ret = -ENOMEM; | 1991 | ret = -ENOMEM; |
| 1962 | goto err_free_kfifo; | 1992 | goto err_free_fifos; |
| 1963 | } | 1993 | } |
| 1964 | 1994 | ||
| 1965 | ret = platform_device_add(svc->stratix10_svc_rsu); | 1995 | ret = platform_device_add(svc->stratix10_svc_rsu); |
| 1966 | if (ret) { | 1996 | if (ret) |
| 1967 | platform_device_put(svc->stratix10_svc_rsu); | 1997 | goto err_put_device; |
| 1968 | goto err_free_kfifo; | ||
| 1969 | } | ||
| 1970 | |||
| 1971 | svc->intel_svc_fcs = platform_device_alloc(INTEL_FCS, 1); | ||
| 1972 | if (!svc->intel_svc_fcs) { | ||
| 1973 | dev_err(dev, "failed to allocate %s device\n", INTEL_FCS); | ||
| 1974 | ret = -ENOMEM; | ||
| 1975 | goto err_unregister_rsu_dev; | ||
| 1976 | } | ||
| 1977 | |||
| 1978 | ret = platform_device_add(svc->intel_svc_fcs); | ||
| 1979 | if (ret) { | ||
| 1980 | platform_device_put(svc->intel_svc_fcs); | ||
| 1981 | goto err_unregister_rsu_dev; | ||
| 1982 | } | ||
| 1983 | 1998 | ||
| 1984 | ret = of_platform_default_populate(dev_of_node(dev), NULL, dev); | 1999 | ret = of_platform_default_populate(dev_of_node(dev), NULL, dev); |
| 1985 | if (ret) | 2000 | if (ret) |
| 1986 | goto err_unregister_fcs_dev; | 2001 | goto err_unregister_rsu_dev; |
| 1987 | 2002 | ||
| 1988 | pr_info("Intel Service Layer Driver Initialized\n"); | 2003 | pr_info("Intel Service Layer Driver Initialized\n"); |
| 1989 | 2004 | ||
| 1990 | return 0; | 2005 | return 0; |
| 1991 | 2006 | ||
| 1992 | err_unregister_fcs_dev: | ||
| 1993 | platform_device_unregister(svc->intel_svc_fcs); | ||
| 1994 | err_unregister_rsu_dev: | 2007 | err_unregister_rsu_dev: |
| 1995 | platform_device_unregister(svc->stratix10_svc_rsu); | 2008 | platform_device_unregister(svc->stratix10_svc_rsu); |
| 1996 | err_free_kfifo: | 2009 | goto err_free_fifos; |
| 1997 | kfifo_free(&controller->svc_fifo); | 2010 | err_put_device: |
| 1998 | err_async_exit: | 2011 | platform_device_put(svc->stratix10_svc_rsu); |
| 2012 | err_free_fifos: | ||
| 2013 | /* only remove from list if list_add_tail() was reached */ | ||
| 2014 | if (!list_empty(&controller->node)) | ||
| 2015 | list_del(&controller->node); | ||
| 2016 | /* free only the FIFOs that were successfully allocated */ | ||
| 2017 | while (i--) | ||
| 2018 | kfifo_free(&chans[i].svc_fifo); | ||
| 1999 | stratix10_svc_async_exit(controller); | 2019 | stratix10_svc_async_exit(controller); |
| 2000 | err_destroy_pool: | 2020 | err_destroy_pool: |
| 2001 | gen_pool_destroy(genpool); | 2021 | gen_pool_destroy(genpool); |
| 2022 | |||
| 2002 | return ret; | 2023 | return ret; |
| 2003 | } | 2024 | } |
| 2004 | 2025 | ||
| 2005 | static void stratix10_svc_drv_remove(struct platform_device *pdev) | 2026 | static void stratix10_svc_drv_remove(struct platform_device *pdev) |
| 2006 | { | 2027 | { |
| 2028 | int i; | ||
| 2007 | struct stratix10_svc_controller *ctrl = platform_get_drvdata(pdev); | 2029 | struct stratix10_svc_controller *ctrl = platform_get_drvdata(pdev); |
| 2008 | struct stratix10_svc *svc = ctrl->svc; | 2030 | struct stratix10_svc *svc = ctrl->svc; |
| 2009 | 2031 | ||
| @@ -2011,14 +2033,16 @@ static void stratix10_svc_drv_remove(struct platform_device *pdev) · 1 unmodified line | |||
| 2011 | 2033 | ||
| 2012 | of_platform_depopulate(ctrl->dev); | 2034 | of_platform_depopulate(ctrl->dev); |
| 2013 | 2035 | ||
| 2014 | platform_device_unregister(svc->intel_svc_fcs); | ||
| 2015 | platform_device_unregister(svc->stratix10_svc_rsu); | 2036 | platform_device_unregister(svc->stratix10_svc_rsu); |
| 2016 | 2037 | ||
| 2017 | kfifo_free(&ctrl->svc_fifo); | 2038 | for (i = 0; i < SVC_NUM_CHANNEL; i++) { |
| 2018 | if (ctrl->task) { | 2039 | if (ctrl->chans[i].task) { |
| 2019 | kthread_stop(ctrl->task); | 2040 | kthread_stop(ctrl->chans[i].task); |
| 2020 | ctrl->task = NULL; | 2041 | ctrl->chans[i].task = NULL; |
| 2042 | } | ||
| 2043 | kfifo_free(&ctrl->chans[i].svc_fifo); | ||
| 2021 | } | 2044 | } |
| 2045 | |||
| 2022 | if (ctrl->genpool) | 2046 | if (ctrl->genpool) |
| 2023 | gen_pool_destroy(ctrl->genpool); | 2047 | gen_pool_destroy(ctrl->genpool); |
| 2024 | list_del(&ctrl->node); | 2048 | list_del(&ctrl->node); |
| @@ -38,8 +38,10 @@ MODULE_DESCRIPTION("GPIB driver for LPVO usb devices"); · 37 unmodified lines | |||
| 38 | /* | 38 | /* |
| 39 | * Table of devices that work with this driver. | 39 | * Table of devices that work with this driver. |
| 40 | * | 40 | * |
| 41 | * Currently, only one device is known to be used in the | 41 | * Currently, only one device is known to be used in the lpvo_usb_gpib |
| 42 | * lpvo_usb_gpib adapter (FTDI 0403:6001). | 42 | * adapter (FTDI 0403:6001) but as this device id is already handled by the |
| 43 | * ftdi_sio USB serial driver the LPVO driver must not bind to it by default. | ||
| 44 | * | ||
| 43 | * If your adapter uses a different chip, insert a line | 45 | * If your adapter uses a different chip, insert a line |
| 44 | * in the following table with proper <Vendor-id>, <Product-id>. | 46 | * in the following table with proper <Vendor-id>, <Product-id>. |
| 45 | * | 47 | * |
| @@ -50,7 +52,6 @@ MODULE_DESCRIPTION("GPIB driver for LPVO usb devices"); · 4 unmodified lines | |||
| 50 | */ | 52 | */ |
| 51 | 53 | ||
| 52 | static const struct usb_device_id skel_table[] = { | 54 | static const struct usb_device_id skel_table[] = { |
| 53 | { USB_DEVICE(0x0403, 0x6001) }, | ||
| 54 | { } /* Terminating entry */ | 55 | { } /* Terminating entry */ |
| 55 | }; | 56 | }; |
| 56 | MODULE_DEVICE_TABLE(usb, skel_table); | 57 | MODULE_DEVICE_TABLE(usb, skel_table); |
| @@ -748,14 +748,14 @@ static bool gpio_shared_entry_is_really_shared(struct gpio_shared_entry *entry) · 747 unmodified lines | |||
| 748 | static void gpio_shared_free_exclusive(void) | 748 | static void gpio_shared_free_exclusive(void) |
| 749 | { | 749 | { |
| 750 | struct gpio_shared_entry *entry, *epos; | 750 | struct gpio_shared_entry *entry, *epos; |
| 751 | struct gpio_shared_ref *ref, *rpos; | ||
| 751 | 752 | ||
| 752 | list_for_each_entry_safe(entry, epos, &gpio_shared_list, list) { | 753 | list_for_each_entry_safe(entry, epos, &gpio_shared_list, list) { |
| 753 | if (gpio_shared_entry_is_really_shared(entry)) | 754 | if (gpio_shared_entry_is_really_shared(entry)) |
| 754 | continue; | 755 | continue; |
| 755 | 756 | ||
| 756 | gpio_shared_drop_ref(list_first_entry(&entry->refs, | 757 | list_for_each_entry_safe(ref, rpos, &entry->refs, list) |
| 757 | struct gpio_shared_ref, | 758 | gpio_shared_drop_ref(ref); |
| 758 | list)); | ||
| 759 | gpio_shared_drop_entry(entry); | 759 | gpio_shared_drop_entry(entry); |
| 760 | } | 760 | } |
| 761 | } | 761 | } |
| @@ -3267,8 +3267,12 @@ static int gpiochip_get(struct gpio_chip *gc, unsigned int offset) · 3266 unmodified lines | |||
| 3267 | 3267 | ||
| 3268 | /* Make sure this is called after checking for gc->get(). */ | 3268 | /* Make sure this is called after checking for gc->get(). */ |
| 3269 | ret = gc->get(gc, offset); | 3269 | ret = gc->get(gc, offset); |
| 3270 | if (ret > 1) | 3270 | if (ret > 1) { |
| 3271 | ret = -EBADE; | 3271 | gpiochip_warn(gc, |
| 3272 | "invalid return value from gc->get(): %d, consider fixing the driver\n", | ||
| 3273 | ret); | ||
| 3274 | ret = !!ret; | ||
| 3275 | } | ||
| 3272 | 3276 | ||
| 3273 | return ret; | 3277 | return ret; |
| 3274 | } | 3278 | } |
| @@ -641,6 +641,7 @@ static void aca_error_fini(struct aca_error *aerr) · 640 unmodified lines | |||
| 641 | aca_bank_error_remove(aerr, bank_error); | 641 | aca_bank_error_remove(aerr, bank_error); |
| 642 | 642 | ||
| 643 | out_unlock: | 643 | out_unlock: |
| 644 | mutex_unlock(&aerr->lock); | ||
| 644 | mutex_destroy(&aerr->lock); | 645 | mutex_destroy(&aerr->lock); |
| 645 | } | 646 | } |
| 646 | 647 | ||
| @@ -1439,7 +1439,10 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, · 1438 unmodified lines | |||
| 1439 | *process_info = info; | 1439 | *process_info = info; |
| 1440 | } | 1440 | } |
| 1441 | 1441 | ||
| 1442 | vm->process_info = *process_info; | 1442 | if (cmpxchg(&vm->process_info, NULL, *process_info) != NULL) { |
| 1443 | ret = -EINVAL; | ||
| 1444 | goto already_acquired; | ||
| 1445 | } | ||
| 1443 | 1446 | ||
| 1444 | /* Validate page directory and attach eviction fence */ | 1447 | /* Validate page directory and attach eviction fence */ |
| 1445 | ret = amdgpu_bo_reserve(vm->root.bo, true); | 1448 | ret = amdgpu_bo_reserve(vm->root.bo, true); |
| @@ -1479,6 +1482,7 @@ validate_pd_fail: · 33 unmodified lines | |||
| 1479 | amdgpu_bo_unreserve(vm->root.bo); | 1482 | amdgpu_bo_unreserve(vm->root.bo); |
| 1480 | reserve_pd_fail: | 1483 | reserve_pd_fail: |
| 1481 | vm->process_info = NULL; | 1484 | vm->process_info = NULL; |
| 1485 | already_acquired: | ||
| 1482 | if (info) { | 1486 | if (info) { |
| 1483 | dma_fence_put(&info->eviction_fence->base); | 1487 | dma_fence_put(&info->eviction_fence->base); |
| 1484 | *process_info = NULL; | 1488 | *process_info = NULL; |
| @@ -2690,8 +2690,10 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 2689 unmodified lines | |||
| 2690 | break; | 2690 | break; |
| 2691 | default: | 2691 | default: |
| 2692 | r = amdgpu_discovery_set_ip_blocks(adev); | 2692 | r = amdgpu_discovery_set_ip_blocks(adev); |
| 2693 | if (r) | 2693 | if (r) { |
| 2694 | adev->num_ip_blocks = 0; | ||
| 2694 | return r; | 2695 | return r; |
| 2696 | } | ||
| 2695 | break; | 2697 | break; |
| 2696 | } | 2698 | } |
| 2697 | 2699 | ||
| @@ -3247,6 +3249,8 @@ int amdgpu_device_set_cg_state(struct amdgpu_device *adev, · 549 unmodified lines | |||
| 3247 | i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; | 3249 | i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; |
| 3248 | if (!adev->ip_blocks[i].status.late_initialized) | 3250 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3249 | continue; | 3251 | continue; |
| 3252 | if (!adev->ip_blocks[i].version) | ||
| 3253 | continue; | ||
| 3250 | /* skip CG for GFX, SDMA on S0ix */ | 3254 | /* skip CG for GFX, SDMA on S0ix */ |
| 3251 | if (adev->in_s0ix && | 3255 | if (adev->in_s0ix && |
| 3252 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || | 3256 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || |
| @@ -3286,6 +3290,8 @@ int amdgpu_device_set_pg_state(struct amdgpu_device *adev, · 33 unmodified lines | |||
| 3286 | i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; | 3290 | i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; |
| 3287 | if (!adev->ip_blocks[i].status.late_initialized) | 3291 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3288 | continue; | 3292 | continue; |
| 3293 | if (!adev->ip_blocks[i].version) | ||
| 3294 | continue; | ||
| 3289 | /* skip PG for GFX, SDMA on S0ix */ | 3295 | /* skip PG for GFX, SDMA on S0ix */ |
| 3290 | if (adev->in_s0ix && | 3296 | if (adev->in_s0ix && |
| 3291 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || | 3297 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || |
| @@ -3493,6 +3499,8 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 201 unmodified lines | |||
| 3493 | int i, r; | 3499 | int i, r; |
| 3494 | 3500 | ||
| 3495 | for (i = 0; i < adev->num_ip_blocks; i++) { | 3501 | for (i = 0; i < adev->num_ip_blocks; i++) { |
| 3502 | if (!adev->ip_blocks[i].version) | ||
| 3503 | continue; | ||
| 3496 | if (!adev->ip_blocks[i].version->funcs->early_fini) | 3504 | if (!adev->ip_blocks[i].version->funcs->early_fini) |
| 3497 | continue; | 3505 | continue; |
| 3498 | 3506 | ||
| @@ -3570,6 +3578,8 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) · 71 unmodified lines | |||
| 3570 | if (!adev->ip_blocks[i].status.sw) | 3578 | if (!adev->ip_blocks[i].status.sw) |
| 3571 | continue; | 3579 | continue; |
| 3572 | 3580 | ||
| 3581 | if (!adev->ip_blocks[i].version) | ||
| 3582 | continue; | ||
| 3573 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { | 3583 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { |
| 3574 | amdgpu_ucode_free_bo(adev); | 3584 | amdgpu_ucode_free_bo(adev); |
| 3575 | amdgpu_free_static_csa(&adev->virt.csa_obj); | 3585 | amdgpu_free_static_csa(&adev->virt.csa_obj); |
| @@ -3596,6 +3606,8 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) · 20 unmodified lines | |||
| 3596 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { | 3606 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { |
| 3597 | if (!adev->ip_blocks[i].status.late_initialized) | 3607 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3598 | continue; | 3608 | continue; |
| 3609 | if (!adev->ip_blocks[i].version) | ||
| 3610 | continue; | ||
| 3599 | if (adev->ip_blocks[i].version->funcs->late_fini) | 3611 | if (adev->ip_blocks[i].version->funcs->late_fini) |
| 3600 | adev->ip_blocks[i].version->funcs->late_fini(&adev->ip_blocks[i]); | 3612 | adev->ip_blocks[i].version->funcs->late_fini(&adev->ip_blocks[i]); |
| 3601 | adev->ip_blocks[i].status.late_initialized = false; | 3613 | adev->ip_blocks[i].status.late_initialized = false; |
| @@ -7059,6 +7071,15 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev) · 3457 unmodified lines | |||
| 7059 | dev_info(adev->dev, "PCI error: slot reset callback!!\n"); | 7071 | dev_info(adev->dev, "PCI error: slot reset callback!!\n"); |
| 7060 | 7072 | ||
| 7061 | memset(&reset_context, 0, sizeof(reset_context)); | 7073 | memset(&reset_context, 0, sizeof(reset_context)); |
| 7074 | INIT_LIST_HEAD(&device_list); | ||
| 7075 | hive = amdgpu_get_xgmi_hive(adev); | ||
| 7076 | if (hive) { | ||
| 7077 | mutex_lock(&hive->hive_lock); | ||
| 7078 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) | ||
| 7079 | list_add_tail(&tmp_adev->reset_list, &device_list); | ||
| 7080 | } else { | ||
| 7081 | list_add_tail(&adev->reset_list, &device_list); | ||
| 7082 | } | ||
| 7062 | 7083 | ||
| 7063 | if (adev->pcie_reset_ctx.swus) | 7084 | if (adev->pcie_reset_ctx.swus) |
| 7064 | link_dev = adev->pcie_reset_ctx.swus; | 7085 | link_dev = adev->pcie_reset_ctx.swus; |
| @@ -7099,19 +7120,13 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev) · 34 unmodified lines | |||
| 7099 | reset_context.reset_req_dev = adev; | 7120 | reset_context.reset_req_dev = adev; |
| 7100 | set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | 7121 | set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); |
| 7101 | set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags); | 7122 | set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags); |
| 7102 | INIT_LIST_HEAD(&device_list); | ||
| 7103 | 7123 | ||
| 7104 | hive = amdgpu_get_xgmi_hive(adev); | ||
| 7105 | if (hive) { | 7124 | if (hive) { |
| 7106 | mutex_lock(&hive->hive_lock); | ||
| 7107 | reset_context.hive = hive; | 7125 | reset_context.hive = hive; |
| 7108 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) { | 7126 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) |
| 7109 | tmp_adev->pcie_reset_ctx.in_link_reset = true; | 7127 | tmp_adev->pcie_reset_ctx.in_link_reset = true; |
| 7110 | list_add_tail(&tmp_adev->reset_list, &device_list); | ||
| 7111 | } | ||
| 7112 | } else { | 7128 | } else { |
| 7113 | set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags); | 7129 | set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags); |
| 7114 | list_add_tail(&adev->reset_list, &device_list); | ||
| 7115 | } | 7130 | } |
| 7116 | 7131 | ||
| 7117 | r = amdgpu_device_asic_reset(adev, &device_list, &reset_context); | 7132 | r = amdgpu_device_asic_reset(adev, &device_list, &reset_context); |
| @@ -83,7 +83,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) · 82 unmodified lines | |||
| 83 | { | 83 | { |
| 84 | struct amdgpu_device *adev = drm_to_adev(dev); | 84 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 85 | 85 | ||
| 86 | if (adev == NULL) | 86 | if (adev == NULL || !adev->num_ip_blocks) |
| 87 | return; | 87 | return; |
| 88 | 88 | ||
| 89 | amdgpu_unregister_gpu_instance(adev); | 89 | amdgpu_unregister_gpu_instance(adev); |
| @@ -368,15 +368,15 @@ struct amdgpu_mode_info { · 367 unmodified lines | |||
| 368 | 368 | ||
| 369 | struct drm_property *plane_ctm_property; | 369 | struct drm_property *plane_ctm_property; |
| 370 | /** | 370 | /** |
| 371 | * @shaper_lut_property: Plane property to set pre-blending shaper LUT | 371 | * @plane_shaper_lut_property: Plane property to set pre-blending |
| 372 | * that converts color content before 3D LUT. If | 372 | * shaper LUT that converts color content before 3D LUT. |
| 373 | * plane_shaper_tf_property != Identity TF, AMD color module will | 373 | * If plane_shaper_tf_property != Identity TF, AMD color module will |
| 374 | * combine the user LUT values with pre-defined TF into the LUT | 374 | * combine the user LUT values with pre-defined TF into the LUT |
| 375 | * parameters to be programmed. | 375 | * parameters to be programmed. |
| 376 | */ | 376 | */ |
| 377 | struct drm_property *plane_shaper_lut_property; | 377 | struct drm_property *plane_shaper_lut_property; |
| 378 | /** | 378 | /** |
| 379 | * @shaper_lut_size_property: Plane property for the size of | 379 | * @plane_shaper_lut_size_property: Plane property for the size of |
| 380 | * pre-blending shaper LUT as supported by the driver (read-only). | 380 | * pre-blending shaper LUT as supported by the driver (read-only). |
| 381 | */ | 381 | */ |
| 382 | struct drm_property *plane_shaper_lut_size_property; | 382 | struct drm_property *plane_shaper_lut_size_property; |
| @@ -400,10 +400,10 @@ struct amdgpu_mode_info { · 17 unmodified lines | |||
| 400 | */ | 400 | */ |
| 401 | struct drm_property *plane_lut3d_property; | 401 | struct drm_property *plane_lut3d_property; |
| 402 | /** | 402 | /** |
| 403 | * @plane_degamma_lut_size_property: Plane property to define the max | 403 | * @plane_lut3d_size_property: Plane property to define the max size |
| 404 | * size of 3D LUT as supported by the driver (read-only). The max size | 404 | * of 3D LUT as supported by the driver (read-only). The max size is |
| 405 | * is the max size of one dimension and, therefore, the max number of | 405 | * the max size of one dimension and, therefore, the max number of |
| 406 | * entries for 3D LUT array is the 3D LUT size cubed; | 406 | * entries for 3D LUT array is the 3D LUT size cubed. |
| 407 | */ | 407 | */ |
| 408 | struct drm_property *plane_lut3d_size_property; | 408 | struct drm_property *plane_lut3d_size_property; |
| 409 | /** | 409 | /** |
| @@ -332,13 +332,13 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 331 unmodified lines | |||
| 332 | if (!context || !context->initialized) { | 332 | if (!context || !context->initialized) { |
| 333 | dev_err(adev->dev, "TA is not initialized\n"); | 333 | dev_err(adev->dev, "TA is not initialized\n"); |
| 334 | ret = -EINVAL; | 334 | ret = -EINVAL; |
| 335 | goto err_free_shared_buf; | 335 | goto free_shared_buf; |
| 336 | } | 336 | } |
| 337 | 337 | ||
| 338 | if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_invoke) { | 338 | if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_invoke) { |
| 339 | dev_err(adev->dev, "Unsupported function to invoke TA\n"); | 339 | dev_err(adev->dev, "Unsupported function to invoke TA\n"); |
| 340 | ret = -EOPNOTSUPP; | 340 | ret = -EOPNOTSUPP; |
| 341 | goto err_free_shared_buf; | 341 | goto free_shared_buf; |
| 342 | } | 342 | } |
| 343 | 343 | ||
| 344 | context->session_id = ta_id; | 344 | context->session_id = ta_id; |
| @@ -346,7 +346,7 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 1 unmodified line | |||
| 346 | mutex_lock(&psp->ras_context.mutex); | 346 | mutex_lock(&psp->ras_context.mutex); |
| 347 | ret = prep_ta_mem_context(&context->mem_context, shared_buf, shared_buf_len); | 347 | ret = prep_ta_mem_context(&context->mem_context, shared_buf, shared_buf_len); |
| 348 | if (ret) | 348 | if (ret) |
| 349 | goto err_free_shared_buf; | 349 | goto unlock; |
| 350 | 350 | ||
| 351 | ret = psp_fn_ta_invoke(psp, cmd_id); | 351 | ret = psp_fn_ta_invoke(psp, cmd_id); |
| 352 | if (ret || context->resp_status) { | 352 | if (ret || context->resp_status) { |
| @@ -354,15 +354,17 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 1 unmodified line | |||
| 354 | ret, context->resp_status); | 354 | ret, context->resp_status); |
| 355 | if (!ret) { | 355 | if (!ret) { |
| 356 | ret = -EINVAL; | 356 | ret = -EINVAL; |
| 357 | goto err_free_shared_buf; | 357 | goto unlock; |
| 358 | } | 358 | } |
| 359 | } | 359 | } |
| 360 | 360 | ||
| 361 | if (copy_to_user((char *)&buf[copy_pos], context->mem_context.shared_buf, shared_buf_len)) | 361 | if (copy_to_user((char *)&buf[copy_pos], context->mem_context.shared_buf, shared_buf_len)) |
| 362 | ret = -EFAULT; | 362 | ret = -EFAULT; |
| 363 | 363 | ||
| 364 | err_free_shared_buf: | 364 | unlock: |
| 365 | mutex_unlock(&psp->ras_context.mutex); | 365 | mutex_unlock(&psp->ras_context.mutex); |
| 366 | |||
| 367 | free_shared_buf: | ||
| 366 | kfree(shared_buf); | 368 | kfree(shared_buf); |
| 367 | 369 | ||
| 368 | return ret; | 370 | return ret; |
| @@ -446,8 +446,7 @@ static int amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue) · 445 unmodified lines | |||
| 446 | return ret; | 446 | return ret; |
| 447 | } | 447 | } |
| 448 | 448 | ||
| 449 | static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, | 449 | static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue) |
| 450 | int queue_id) | ||
| 451 | { | 450 | { |
| 452 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | 451 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; |
| 453 | struct amdgpu_device *adev = uq_mgr->adev; | 452 | struct amdgpu_device *adev = uq_mgr->adev; |
| @@ -461,7 +460,6 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, · 7 unmodified lines | |||
| 461 | uq_funcs->mqd_destroy(queue); | 460 | uq_funcs->mqd_destroy(queue); |
| 462 | amdgpu_userq_fence_driver_free(queue); | 461 | amdgpu_userq_fence_driver_free(queue); |
| 463 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ | 462 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ |
| 464 | xa_erase_irq(&uq_mgr->userq_xa, (unsigned long)queue_id); | ||
| 465 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); | 463 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); |
| 466 | queue->userq_mgr = NULL; | 464 | queue->userq_mgr = NULL; |
| 467 | list_del(&queue->userq_va_list); | 465 | list_del(&queue->userq_va_list); |
| @@ -470,12 +468,6 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, · 2 unmodified lines | |||
| 470 | up_read(&adev->reset_domain->sem); | 468 | up_read(&adev->reset_domain->sem); |
| 471 | } | 469 | } |
| 472 | 470 | ||
| 473 | static struct amdgpu_usermode_queue * | ||
| 474 | amdgpu_userq_find(struct amdgpu_userq_mgr *uq_mgr, int qid) | ||
| 475 | { | ||
| 476 | return xa_load(&uq_mgr->userq_xa, qid); | ||
| 477 | } | ||
| 478 | |||
| 479 | void | 471 | void |
| 480 | amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *uq_mgr, | 472 | amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *uq_mgr, |
| 481 | struct amdgpu_eviction_fence_mgr *evf_mgr) | 473 | struct amdgpu_eviction_fence_mgr *evf_mgr) |
| @@ -625,22 +617,13 @@ unref_bo: · 143 unmodified lines | |||
| 625 | } | 617 | } |
| 626 | 618 | ||
| 627 | static int | 619 | static int |
| 628 | amdgpu_userq_destroy(struct drm_file *filp, int queue_id) | 620 | amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_queue *queue) |
| 629 | { | 621 | { |
| 630 | struct amdgpu_fpriv *fpriv = filp->driver_priv; | ||
| 631 | struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr; | ||
| 632 | struct amdgpu_device *adev = uq_mgr->adev; | 622 | struct amdgpu_device *adev = uq_mgr->adev; |
| 633 | struct amdgpu_usermode_queue *queue; | ||
| 634 | int r = 0; | 623 | int r = 0; |
| 635 | 624 | ||
| 636 | cancel_delayed_work_sync(&uq_mgr->resume_work); | 625 | cancel_delayed_work_sync(&uq_mgr->resume_work); |
| 637 | mutex_lock(&uq_mgr->userq_mutex); | 626 | mutex_lock(&uq_mgr->userq_mutex); |
| 638 | queue = amdgpu_userq_find(uq_mgr, queue_id); | ||
| 639 | if (!queue) { | ||
| 640 | drm_dbg_driver(adev_to_drm(uq_mgr->adev), "Invalid queue id to destroy\n"); | ||
| 641 | mutex_unlock(&uq_mgr->userq_mutex); | ||
| 642 | return -EINVAL; | ||
| 643 | } | ||
| 644 | amdgpu_userq_wait_for_last_fence(queue); | 627 | amdgpu_userq_wait_for_last_fence(queue); |
| 645 | /* Cancel any pending hang detection work and cleanup */ | 628 | /* Cancel any pending hang detection work and cleanup */ |
| 646 | if (queue->hang_detect_fence) { | 629 | if (queue->hang_detect_fence) { |
| @@ -672,7 +655,7 @@ amdgpu_userq_destroy(struct drm_file *filp, int queue_id) · 25 unmodified lines | |||
| 672 | drm_warn(adev_to_drm(uq_mgr->adev), "trying to destroy a HW mapping userq\n"); | 655 | drm_warn(adev_to_drm(uq_mgr->adev), "trying to destroy a HW mapping userq\n"); |
| 673 | queue->state = AMDGPU_USERQ_STATE_HUNG; | 656 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 674 | } | 657 | } |
| 675 | amdgpu_userq_cleanup(queue, queue_id); | 658 | amdgpu_userq_cleanup(queue); |
| 676 | mutex_unlock(&uq_mgr->userq_mutex); | 659 | mutex_unlock(&uq_mgr->userq_mutex); |
| 677 | 660 | ||
| 678 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); | 661 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); |
| @@ -680,6 +663,37 @@ amdgpu_userq_destroy(struct drm_file *filp, int queue_id) · 1 unmodified line | |||
| 680 | return r; | 663 | return r; |
| 681 | } | 664 | } |
| 682 | 665 | ||
| 666 | static void amdgpu_userq_kref_destroy(struct kref *kref) | ||
| 667 | { | ||
| 668 | int r; | ||
| 669 | struct amdgpu_usermode_queue *queue = | ||
| 670 | container_of(kref, struct amdgpu_usermode_queue, refcount); | ||
| 671 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | ||
| 672 | |||
| 673 | r = amdgpu_userq_destroy(uq_mgr, queue); | ||
| 674 | if (r) | ||
| 675 | drm_file_err(uq_mgr->file, "Failed to destroy usermode queue %d\n", r); | ||
| 676 | } | ||
| 677 | |||
| 678 | struct amdgpu_usermode_queue *amdgpu_userq_get(struct amdgpu_userq_mgr *uq_mgr, u32 qid) | ||
| 679 | { | ||
| 680 | struct amdgpu_usermode_queue *queue; | ||
| 681 | |||
| 682 | xa_lock(&uq_mgr->userq_xa); | ||
| 683 | queue = xa_load(&uq_mgr->userq_xa, qid); | ||
| 684 | if (queue) | ||
| 685 | kref_get(&queue->refcount); | ||
| 686 | xa_unlock(&uq_mgr->userq_xa); | ||
| 687 | |||
| 688 | return queue; | ||
| 689 | } | ||
| 690 | |||
| 691 | void amdgpu_userq_put(struct amdgpu_usermode_queue *queue) | ||
| 692 | { | ||
| 693 | if (queue) | ||
| 694 | kref_put(&queue->refcount, amdgpu_userq_kref_destroy); | ||
| 695 | } | ||
| 696 | |||
| 683 | static int amdgpu_userq_priority_permit(struct drm_file *filp, | 697 | static int amdgpu_userq_priority_permit(struct drm_file *filp, |
| 684 | int priority) | 698 | int priority) |
| 685 | { | 699 | { |
| @@ -834,6 +848,9 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 148 unmodified lines | |||
| 834 | goto unlock; | 848 | goto unlock; |
| 835 | } | 849 | } |
| 836 | 850 | ||
| 851 | /* drop this refcount during queue destroy */ | ||
| 852 | kref_init(&queue->refcount); | ||
| 853 | |||
| 837 | /* Wait for mode-1 reset to complete */ | 854 | /* Wait for mode-1 reset to complete */ |
| 838 | down_read(&adev->reset_domain->sem); | 855 | down_read(&adev->reset_domain->sem); |
| 839 | r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, GFP_KERNEL)); | 856 | r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, GFP_KERNEL)); |
| @@ -985,7 +1002,9 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, · 145 unmodified lines | |||
| 985 | struct drm_file *filp) | 1002 | struct drm_file *filp) |
| 986 | { | 1003 | { |
| 987 | union drm_amdgpu_userq *args = data; | 1004 | union drm_amdgpu_userq *args = data; |
| 988 | int r; | 1005 | struct amdgpu_fpriv *fpriv = filp->driver_priv; |
| 1006 | struct amdgpu_usermode_queue *queue; | ||
| 1007 | int r = 0; | ||
| 989 | 1008 | ||
| 990 | if (!amdgpu_userq_enabled(dev)) | 1009 | if (!amdgpu_userq_enabled(dev)) |
| 991 | return -ENOTSUPP; | 1010 | return -ENOTSUPP; |
| @@ -1000,11 +1019,16 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, · 8 unmodified lines | |||
| 1000 | drm_file_err(filp, "Failed to create usermode queue\n"); | 1019 | drm_file_err(filp, "Failed to create usermode queue\n"); |
| 1001 | break; | 1020 | break; |
| 1002 | 1021 | ||
| 1003 | case AMDGPU_USERQ_OP_FREE: | 1022 | case AMDGPU_USERQ_OP_FREE: { |
| 1004 | r = amdgpu_userq_destroy(filp, args->in.queue_id); | 1023 | xa_lock(&fpriv->userq_mgr.userq_xa); |
| 1005 | if (r) | 1024 | queue = __xa_erase(&fpriv->userq_mgr.userq_xa, args->in.queue_id); |
| 1006 | drm_file_err(filp, "Failed to destroy usermode queue\n"); | 1025 | xa_unlock(&fpriv->userq_mgr.userq_xa); |
| 1026 | if (!queue) | ||
| 1027 | return -ENOENT; | ||
| 1028 | |||
| 1029 | amdgpu_userq_put(queue); | ||
| 1007 | break; | 1030 | break; |
| 1031 | } | ||
| 1008 | 1032 | ||
| 1009 | default: | 1033 | default: |
| 1010 | drm_dbg_driver(dev, "Invalid user queue op specified: %d\n", args->in.op); | 1034 | drm_dbg_driver(dev, "Invalid user queue op specified: %d\n", args->in.op); |
| @@ -1023,16 +1047,23 @@ amdgpu_userq_restore_all(struct amdgpu_userq_mgr *uq_mgr) · 12 unmodified lines | |||
| 1023 | 1047 | ||
| 1024 | /* Resume all the queues for this process */ | 1048 | /* Resume all the queues for this process */ |
| 1025 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { | 1049 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1050 | queue = amdgpu_userq_get(uq_mgr, queue_id); | ||
| 1051 | if (!queue) | ||
| 1052 | continue; | ||
| 1053 | |||
| 1026 | if (!amdgpu_userq_buffer_vas_mapped(queue)) { | 1054 | if (!amdgpu_userq_buffer_vas_mapped(queue)) { |
| 1027 | drm_file_err(uq_mgr->file, | 1055 | drm_file_err(uq_mgr->file, |
| 1028 | "trying restore queue without va mapping\n"); | 1056 | "trying restore queue without va mapping\n"); |
| 1029 | queue->state = AMDGPU_USERQ_STATE_INVALID_VA; | 1057 | queue->state = AMDGPU_USERQ_STATE_INVALID_VA; |
| 1058 | amdgpu_userq_put(queue); | ||
| 1030 | continue; | 1059 | continue; |
| 1031 | } | 1060 | } |
| 1032 | 1061 | ||
| 1033 | r = amdgpu_userq_restore_helper(queue); | 1062 | r = amdgpu_userq_restore_helper(queue); |
| 1034 | if (r) | 1063 | if (r) |
| 1035 | ret = r; | 1064 | ret = r; |
| 1065 | |||
| 1066 | amdgpu_userq_put(queue); | ||
| 1036 | } | 1067 | } |
| 1037 | 1068 | ||
| 1038 | if (ret) | 1069 | if (ret) |
| @@ -1266,9 +1297,13 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 227 unmodified lines | |||
| 1266 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | 1297 | amdgpu_userq_detect_and_reset_queues(uq_mgr); |
| 1267 | /* Try to unmap all the queues in this process ctx */ | 1298 | /* Try to unmap all the queues in this process ctx */ |
| 1268 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { | 1299 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1300 | queue = amdgpu_userq_get(uq_mgr, queue_id); | ||
| 1301 | if (!queue) | ||
| 1302 | continue; | ||
| 1269 | r = amdgpu_userq_preempt_helper(queue); | 1303 | r = amdgpu_userq_preempt_helper(queue); |
| 1270 | if (r) | 1304 | if (r) |
| 1271 | ret = r; | 1305 | ret = r; |
| 1306 | amdgpu_userq_put(queue); | ||
| 1272 | } | 1307 | } |
| 1273 | 1308 | ||
| 1274 | if (ret) | 1309 | if (ret) |
| @@ -1301,16 +1336,24 @@ amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr) · 26 unmodified lines | |||
| 1301 | int ret; | 1336 | int ret; |
| 1302 | 1337 | ||
| 1303 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { | 1338 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1339 | queue = amdgpu_userq_get(uq_mgr, queue_id); | ||
| 1340 | if (!queue) | ||
| 1341 | continue; | ||
| 1342 | |||
| 1304 | struct dma_fence *f = queue->last_fence; | 1343 | struct dma_fence *f = queue->last_fence; |
| 1305 | 1344 | ||
| 1306 | if (!f || dma_fence_is_signaled(f)) | 1345 | if (!f || dma_fence_is_signaled(f)) { |
| 1346 | amdgpu_userq_put(queue); | ||
| 1307 | continue; | 1347 | continue; |
| 1348 | } | ||
| 1308 | ret = dma_fence_wait_timeout(f, true, msecs_to_jiffies(100)); | 1349 | ret = dma_fence_wait_timeout(f, true, msecs_to_jiffies(100)); |
| 1309 | if (ret <= 0) { | 1350 | if (ret <= 0) { |
| 1310 | drm_file_err(uq_mgr->file, "Timed out waiting for fence=%llu:%llu\n", | 1351 | drm_file_err(uq_mgr->file, "Timed out waiting for fence=%llu:%llu\n", |
| 1311 | f->context, f->seqno); | 1352 | f->context, f->seqno); |
| 1353 | amdgpu_userq_put(queue); | ||
| 1312 | return -ETIMEDOUT; | 1354 | return -ETIMEDOUT; |
| 1313 | } | 1355 | } |
| 1356 | amdgpu_userq_put(queue); | ||
| 1314 | } | 1357 | } |
| 1315 | 1358 | ||
| 1316 | return 0; | 1359 | return 0; |
| @@ -1361,20 +1404,23 @@ int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *f · 44 unmodified lines | |||
| 1361 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr) | 1404 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr) |
| 1362 | { | 1405 | { |
| 1363 | struct amdgpu_usermode_queue *queue; | 1406 | struct amdgpu_usermode_queue *queue; |
| 1364 | unsigned long queue_id; | 1407 | unsigned long queue_id = 0; |
| 1408 | |||
| 1409 | for (;;) { | ||
| 1410 | xa_lock(&userq_mgr->userq_xa); | ||
| 1411 | queue = xa_find(&userq_mgr->userq_xa, &queue_id, ULONG_MAX, | ||
| 1412 | XA_PRESENT); | ||
| 1413 | if (queue) | ||
| 1414 | __xa_erase(&userq_mgr->userq_xa, queue_id); | ||
| 1415 | xa_unlock(&userq_mgr->userq_xa); | ||
| 1365 | 1416 | ||
| 1366 | cancel_delayed_work_sync(&userq_mgr->resume_work); | 1417 | if (!queue) |
| 1418 | break; | ||
| 1367 | 1419 | ||
| 1368 | mutex_lock(&userq_mgr->userq_mutex); | 1420 | amdgpu_userq_put(queue); |
| 1369 | amdgpu_userq_detect_and_reset_queues(userq_mgr); | ||
| 1370 | xa_for_each(&userq_mgr->userq_xa, queue_id, queue) { | ||
| 1371 | amdgpu_userq_wait_for_last_fence(queue); | ||
| 1372 | amdgpu_userq_unmap_helper(queue); | ||
| 1373 | amdgpu_userq_cleanup(queue, queue_id); | ||
| 1374 | } | 1421 | } |
| 1375 | 1422 | ||
| 1376 | xa_destroy(&userq_mgr->userq_xa); | 1423 | xa_destroy(&userq_mgr->userq_xa); |
| 1377 | mutex_unlock(&userq_mgr->userq_mutex); | ||
| 1378 | mutex_destroy(&userq_mgr->userq_mutex); | 1424 | mutex_destroy(&userq_mgr->userq_mutex); |
| 1379 | } | 1425 | } |
| 1380 | 1426 | ||
| @@ -74,6 +74,7 @@ struct amdgpu_usermode_queue { · 73 unmodified lines | |||
| 74 | struct dentry *debugfs_queue; | 74 | struct dentry *debugfs_queue; |
| 75 | struct delayed_work hang_detect_work; | 75 | struct delayed_work hang_detect_work; |
| 76 | struct dma_fence *hang_detect_fence; | 76 | struct dma_fence *hang_detect_fence; |
| 77 | struct kref refcount; | ||
| 77 | 78 | ||
| 78 | struct list_head userq_va_list; | 79 | struct list_head userq_va_list; |
| 79 | }; | 80 | }; |
| @@ -112,6 +113,9 @@ struct amdgpu_db_info { · 32 unmodified lines | |||
| 112 | struct amdgpu_userq_obj *db_obj; | 113 | struct amdgpu_userq_obj *db_obj; |
| 113 | }; | 114 | }; |
| 114 | 115 | ||
| 116 | struct amdgpu_usermode_queue *amdgpu_userq_get(struct amdgpu_userq_mgr *uq_mgr, u32 qid); | ||
| 117 | void amdgpu_userq_put(struct amdgpu_usermode_queue *queue); | ||
| 118 | |||
| 115 | int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp); | 119 | int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp); |
| 116 | 120 | ||
| 117 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, | 121 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, |
| @@ -35,6 +35,8 @@ · 34 unmodified lines | |||
| 35 | static const struct dma_fence_ops amdgpu_userq_fence_ops; | 35 | static const struct dma_fence_ops amdgpu_userq_fence_ops; |
| 36 | static struct kmem_cache *amdgpu_userq_fence_slab; | 36 | static struct kmem_cache *amdgpu_userq_fence_slab; |
| 37 | 37 | ||
| 38 | #define AMDGPU_USERQ_MAX_HANDLES (1U << 16) | ||
| 39 | |||
| 38 | int amdgpu_userq_fence_slab_init(void) | 40 | int amdgpu_userq_fence_slab_init(void) |
| 39 | { | 41 | { |
| 40 | amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence", | 42 | amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence", |
| @@ -464,7 +466,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 423 unmodified lines | |||
| 464 | struct drm_amdgpu_userq_signal *args = data; | 466 | struct drm_amdgpu_userq_signal *args = data; |
| 465 | struct drm_gem_object **gobj_write = NULL; | 467 | struct drm_gem_object **gobj_write = NULL; |
| 466 | struct drm_gem_object **gobj_read = NULL; | 468 | struct drm_gem_object **gobj_read = NULL; |
| 467 | struct amdgpu_usermode_queue *queue; | 469 | struct amdgpu_usermode_queue *queue = NULL; |
| 468 | struct amdgpu_userq_fence *userq_fence; | 470 | struct amdgpu_userq_fence *userq_fence; |
| 469 | struct drm_syncobj **syncobj = NULL; | 471 | struct drm_syncobj **syncobj = NULL; |
| 470 | u32 *bo_handles_write, num_write_bo_handles; | 472 | u32 *bo_handles_write, num_write_bo_handles; |
| @@ -478,6 +480,11 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 7 unmodified lines | |||
| 478 | if (!amdgpu_userq_enabled(dev)) | 480 | if (!amdgpu_userq_enabled(dev)) |
| 479 | return -ENOTSUPP; | 481 | return -ENOTSUPP; |
| 480 | 482 | ||
| 483 | if (args->num_syncobj_handles > AMDGPU_USERQ_MAX_HANDLES || | ||
| 484 | args->num_bo_write_handles > AMDGPU_USERQ_MAX_HANDLES || | ||
| 485 | args->num_bo_read_handles > AMDGPU_USERQ_MAX_HANDLES) | ||
| 486 | return -EINVAL; | ||
| 487 | |||
| 481 | num_syncobj_handles = args->num_syncobj_handles; | 488 | num_syncobj_handles = args->num_syncobj_handles; |
| 482 | syncobj_handles = memdup_user(u64_to_user_ptr(args->syncobj_handles), | 489 | syncobj_handles = memdup_user(u64_to_user_ptr(args->syncobj_handles), |
| 483 | size_mul(sizeof(u32), num_syncobj_handles)); | 490 | size_mul(sizeof(u32), num_syncobj_handles)); |
| @@ -546,7 +553,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 62 unmodified lines | |||
| 546 | } | 553 | } |
| 547 | 554 | ||
| 548 | /* Retrieve the user queue */ | 555 | /* Retrieve the user queue */ |
| 549 | queue = xa_load(&userq_mgr->userq_xa, args->queue_id); | 556 | queue = amdgpu_userq_get(userq_mgr, args->queue_id); |
| 550 | if (!queue) { | 557 | if (!queue) { |
| 551 | r = -ENOENT; | 558 | r = -ENOENT; |
| 552 | goto put_gobj_write; | 559 | goto put_gobj_write; |
| @@ -641,6 +648,9 @@ free_syncobj: · 88 unmodified lines | |||
| 641 | free_syncobj_handles: | 648 | free_syncobj_handles: |
| 642 | kfree(syncobj_handles); | 649 | kfree(syncobj_handles); |
| 643 | 650 | ||
| 651 | if (queue) | ||
| 652 | amdgpu_userq_put(queue); | ||
| 653 | |||
| 644 | return r; | 654 | return r; |
| 645 | } | 655 | } |
| 646 | 656 | ||
| @@ -653,7 +663,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 6 unmodified lines | |||
| 653 | struct drm_amdgpu_userq_wait *wait_info = data; | 663 | struct drm_amdgpu_userq_wait *wait_info = data; |
| 654 | struct amdgpu_fpriv *fpriv = filp->driver_priv; | 664 | struct amdgpu_fpriv *fpriv = filp->driver_priv; |
| 655 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; | 665 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; |
| 656 | struct amdgpu_usermode_queue *waitq; | 666 | struct amdgpu_usermode_queue *waitq = NULL; |
| 657 | struct drm_gem_object **gobj_write; | 667 | struct drm_gem_object **gobj_write; |
| 658 | struct drm_gem_object **gobj_read; | 668 | struct drm_gem_object **gobj_read; |
| 659 | struct dma_fence **fences = NULL; | 669 | struct dma_fence **fences = NULL; |
| @@ -664,6 +674,11 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 4 unmodified lines | |||
| 664 | if (!amdgpu_userq_enabled(dev)) | 674 | if (!amdgpu_userq_enabled(dev)) |
| 665 | return -ENOTSUPP; | 675 | return -ENOTSUPP; |
| 666 | 676 | ||
| 677 | if (wait_info->num_syncobj_handles > AMDGPU_USERQ_MAX_HANDLES || | ||
| 678 | wait_info->num_bo_write_handles > AMDGPU_USERQ_MAX_HANDLES || | ||
| 679 | wait_info->num_bo_read_handles > AMDGPU_USERQ_MAX_HANDLES) | ||
| 680 | return -EINVAL; | ||
| 681 | |||
| 667 | num_read_bo_handles = wait_info->num_bo_read_handles; | 682 | num_read_bo_handles = wait_info->num_bo_read_handles; |
| 668 | bo_handles_read = memdup_user(u64_to_user_ptr(wait_info->bo_read_handles), | 683 | bo_handles_read = memdup_user(u64_to_user_ptr(wait_info->bo_read_handles), |
| 669 | size_mul(sizeof(u32), num_read_bo_handles)); | 684 | size_mul(sizeof(u32), num_read_bo_handles)); |
| @@ -833,7 +848,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 163 unmodified lines | |||
| 833 | 848 | ||
| 834 | dma_resv_for_each_fence(&resv_cursor, gobj_read[i]->resv, | 849 | dma_resv_for_each_fence(&resv_cursor, gobj_read[i]->resv, |
| 835 | DMA_RESV_USAGE_READ, fence) { | 850 | DMA_RESV_USAGE_READ, fence) { |
| 836 | if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | 851 | if (num_fences >= wait_info->num_fences) { |
| 837 | r = -EINVAL; | 852 | r = -EINVAL; |
| 838 | goto free_fences; | 853 | goto free_fences; |
| 839 | } | 854 | } |
| @@ -850,7 +865,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 10 unmodified lines | |||
| 850 | 865 | ||
| 851 | dma_resv_for_each_fence(&resv_cursor, gobj_write[i]->resv, | 866 | dma_resv_for_each_fence(&resv_cursor, gobj_write[i]->resv, |
| 852 | DMA_RESV_USAGE_WRITE, fence) { | 867 | DMA_RESV_USAGE_WRITE, fence) { |
| 853 | if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | 868 | if (num_fences >= wait_info->num_fences) { |
| 854 | r = -EINVAL; | 869 | r = -EINVAL; |
| 855 | goto free_fences; | 870 | goto free_fences; |
| 856 | } | 871 | } |
| @@ -874,8 +889,9 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 17 unmodified lines | |||
| 874 | goto free_fences; | 889 | goto free_fences; |
| 875 | 890 | ||
| 876 | dma_fence_unwrap_for_each(f, &iter, fence) { | 891 | dma_fence_unwrap_for_each(f, &iter, fence) { |
| 877 | if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | 892 | if (num_fences >= wait_info->num_fences) { |
| 878 | r = -EINVAL; | 893 | r = -EINVAL; |
| 894 | dma_fence_put(fence); | ||
| 879 | goto free_fences; | 895 | goto free_fences; |
| 880 | } | 896 | } |
| 881 | 897 | ||
| @@ -898,8 +914,9 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 16 unmodified lines | |||
| 898 | if (r) | 914 | if (r) |
| 899 | goto free_fences; | 915 | goto free_fences; |
| 900 | 916 | ||
| 901 | if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | 917 | if (num_fences >= wait_info->num_fences) { |
| 902 | r = -EINVAL; | 918 | r = -EINVAL; |
| 919 | dma_fence_put(fence); | ||
| 903 | goto free_fences; | 920 | goto free_fences; |
| 904 | } | 921 | } |
| 905 | 922 | ||
| @@ -912,7 +929,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 6 unmodified lines | |||
| 912 | */ | 929 | */ |
| 913 | num_fences = dma_fence_dedup_array(fences, num_fences); | 930 | num_fences = dma_fence_dedup_array(fences, num_fences); |
| 914 | 931 | ||
| 915 | waitq = xa_load(&userq_mgr->userq_xa, wait_info->waitq_id); | 932 | waitq = amdgpu_userq_get(userq_mgr, wait_info->waitq_id); |
| 916 | if (!waitq) { | 933 | if (!waitq) { |
| 917 | r = -EINVAL; | 934 | r = -EINVAL; |
| 918 | goto free_fences; | 935 | goto free_fences; |
| @@ -969,32 +986,14 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 50 unmodified lines | |||
| 969 | r = -EFAULT; | 986 | r = -EFAULT; |
| 970 | goto free_fences; | 987 | goto free_fences; |
| 971 | } | 988 | } |
| 972 | |||
| 973 | kfree(fences); | ||
| 974 | kfree(fence_info); | ||
| 975 | } | 989 | } |
| 976 | 990 | ||
| 977 | drm_exec_fini(&exec); | ||
| 978 | for (i = 0; i < num_read_bo_handles; i++) | ||
| 979 | drm_gem_object_put(gobj_read[i]); | ||
| 980 | kfree(gobj_read); | ||
| 981 | |||
| 982 | for (i = 0; i < num_write_bo_handles; i++) | ||
| 983 | drm_gem_object_put(gobj_write[i]); | ||
| 984 | kfree(gobj_write); | ||
| 985 | |||
| 986 | kfree(timeline_points); | ||
| 987 | kfree(timeline_handles); | ||
| 988 | kfree(syncobj_handles); | ||
| 989 | kfree(bo_handles_write); | ||
| 990 | kfree(bo_handles_read); | ||
| 991 | |||
| 992 | return 0; | ||
| 993 | |||
| 994 | free_fences: | 991 | free_fences: |
| 995 | while (num_fences-- > 0) | 992 | if (fences) { |
| 996 | dma_fence_put(fences[num_fences]); | 993 | while (num_fences-- > 0) |
| 997 | kfree(fences); | 994 | dma_fence_put(fences[num_fences]); |
| 995 | kfree(fences); | ||
| 996 | } | ||
| 998 | free_fence_info: | 997 | free_fence_info: |
| 999 | kfree(fence_info); | 998 | kfree(fence_info); |
| 1000 | exec_fini: | 999 | exec_fini: |
| @@ -1018,5 +1017,8 @@ free_bo_handles_write: · 17 unmodified lines | |||
| 1018 | free_bo_handles_read: | 1017 | free_bo_handles_read: |
| 1019 | kfree(bo_handles_read); | 1018 | kfree(bo_handles_read); |
| 1020 | 1019 | ||
| 1020 | if (waitq) | ||
| 1021 | amdgpu_userq_put(waitq); | ||
| 1022 | |||
| 1021 | return r; | 1023 | return r; |
| 1022 | } | 1024 | } |
| @@ -720,11 +720,6 @@ static int mes_v11_0_set_hw_resources(struct amdgpu_mes *mes) · 719 unmodified lines | |||
| 720 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; | 720 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; |
| 721 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; | 721 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; |
| 722 | mes_set_hw_res_pkt.oversubscription_timer = 50; | 722 | mes_set_hw_res_pkt.oversubscription_timer = 50; |
| 723 | if ((mes->adev->mes.sched_version & AMDGPU_MES_VERSION_MASK) >= 0x7f) | ||
| 724 | mes_set_hw_res_pkt.enable_lr_compute_wa = 1; | ||
| 725 | else | ||
| 726 | dev_info_once(mes->adev->dev, | ||
| 727 | "MES FW version must be >= 0x7f to enable LR compute workaround.\n"); | ||
| 728 | 723 | ||
| 729 | if (amdgpu_mes_log_enable) { | 724 | if (amdgpu_mes_log_enable) { |
| 730 | mes_set_hw_res_pkt.enable_mes_event_int_logging = 1; | 725 | mes_set_hw_res_pkt.enable_mes_event_int_logging = 1; |
| @@ -731,6 +731,9 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe) · 730 unmodified lines | |||
| 731 | int i; | 731 | int i; |
| 732 | struct amdgpu_device *adev = mes->adev; | 732 | struct amdgpu_device *adev = mes->adev; |
| 733 | union MESAPI_SET_HW_RESOURCES mes_set_hw_res_pkt; | 733 | union MESAPI_SET_HW_RESOURCES mes_set_hw_res_pkt; |
| 734 | uint32_t mes_rev = (pipe == AMDGPU_MES_SCHED_PIPE) ? | ||
| 735 | (mes->sched_version & AMDGPU_MES_VERSION_MASK) : | ||
| 736 | (mes->kiq_version & AMDGPU_MES_VERSION_MASK); | ||
| 734 | 737 | ||
| 735 | memset(&mes_set_hw_res_pkt, 0, sizeof(mes_set_hw_res_pkt)); | 738 | memset(&mes_set_hw_res_pkt, 0, sizeof(mes_set_hw_res_pkt)); |
| 736 | 739 | ||
| @@ -779,18 +782,13 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe) · 42 unmodified lines | |||
| 779 | mes_set_hw_res_pkt.use_different_vmid_compute = 1; | 782 | mes_set_hw_res_pkt.use_different_vmid_compute = 1; |
| 780 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; | 783 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; |
| 781 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; | 784 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; |
| 782 | if ((mes->adev->mes.sched_version & AMDGPU_MES_VERSION_MASK) >= 0x82) | ||
| 783 | mes_set_hw_res_pkt.enable_lr_compute_wa = 1; | ||
| 784 | else | ||
| 785 | dev_info_once(adev->dev, | ||
| 786 | "MES FW version must be >= 0x82 to enable LR compute workaround.\n"); | ||
| 787 | 785 | ||
| 788 | /* | 786 | /* |
| 789 | * Keep oversubscribe timer for sdma . When we have unmapped doorbell | 787 | * Keep oversubscribe timer for sdma . When we have unmapped doorbell |
| 790 | * handling support, other queue will not use the oversubscribe timer. | 788 | * handling support, other queue will not use the oversubscribe timer. |
| 791 | * handling mode - 0: disabled; 1: basic version; 2: basic+ version | 789 | * handling mode - 0: disabled; 1: basic version; 2: basic+ version |
| 792 | */ | 790 | */ |
| 793 | mes_set_hw_res_pkt.oversubscription_timer = 50; | 791 | mes_set_hw_res_pkt.oversubscription_timer = mes_rev < 0x8b ? 0 : 50; |
| 794 | mes_set_hw_res_pkt.unmapped_doorbell_handling = 1; | 792 | mes_set_hw_res_pkt.unmapped_doorbell_handling = 1; |
| 795 | 793 | ||
| 796 | if (amdgpu_mes_log_enable) { | 794 | if (amdgpu_mes_log_enable) { |
| @@ -69,12 +69,12 @@ static int psp_v15_0_0_ring_stop(struct psp_context *psp, · 68 unmodified lines | |||
| 69 | 0x80000000, 0x80000000, false); | 69 | 0x80000000, 0x80000000, false); |
| 70 | } else { | 70 | } else { |
| 71 | /* Write the ring destroy command*/ | 71 | /* Write the ring destroy command*/ |
| 72 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_64, | 72 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, |
| 73 | GFX_CTRL_CMD_ID_DESTROY_RINGS); | 73 | GFX_CTRL_CMD_ID_DESTROY_RINGS); |
| 74 | /* there might be handshake issue with hardware which needs delay */ | 74 | /* there might be handshake issue with hardware which needs delay */ |
| 75 | mdelay(20); | 75 | mdelay(20); |
| 76 | /* Wait for response flag (bit 31) */ | 76 | /* Wait for response flag (bit 31) */ |
| 77 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | 77 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), |
| 78 | 0x80000000, 0x80000000, false); | 78 | 0x80000000, 0x80000000, false); |
| 79 | } | 79 | } |
| 80 | 80 | ||
| @@ -116,7 +116,7 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 35 unmodified lines | |||
| 116 | 116 | ||
| 117 | } else { | 117 | } else { |
| 118 | /* Wait for sOS ready for ring creation */ | 118 | /* Wait for sOS ready for ring creation */ |
| 119 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | 119 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), |
| 120 | 0x80000000, 0x80000000, false); | 120 | 0x80000000, 0x80000000, false); |
| 121 | if (ret) { | 121 | if (ret) { |
| 122 | DRM_ERROR("Failed to wait for trust OS ready for ring creation\n"); | 122 | DRM_ERROR("Failed to wait for trust OS ready for ring creation\n"); |
| @@ -125,23 +125,23 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 2 unmodified lines | |||
| 125 | 125 | ||
| 126 | /* Write low address of the ring to C2PMSG_69 */ | 126 | /* Write low address of the ring to C2PMSG_69 */ |
| 127 | psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr); | 127 | psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr); |
| 128 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_69, psp_ring_reg); | 128 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_69, psp_ring_reg); |
| 129 | /* Write high address of the ring to C2PMSG_70 */ | 129 | /* Write high address of the ring to C2PMSG_70 */ |
| 130 | psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr); | 130 | psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr); |
| 131 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_70, psp_ring_reg); | 131 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_70, psp_ring_reg); |
| 132 | /* Write size of ring to C2PMSG_71 */ | 132 | /* Write size of ring to C2PMSG_71 */ |
| 133 | psp_ring_reg = ring->ring_size; | 133 | psp_ring_reg = ring->ring_size; |
| 134 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_71, psp_ring_reg); | 134 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_71, psp_ring_reg); |
| 135 | /* Write the ring initialization command to C2PMSG_64 */ | 135 | /* Write the ring initialization command to C2PMSG_64 */ |
| 136 | psp_ring_reg = ring_type; | 136 | psp_ring_reg = ring_type; |
| 137 | psp_ring_reg = psp_ring_reg << 16; | 137 | psp_ring_reg = psp_ring_reg << 16; |
| 138 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_64, psp_ring_reg); | 138 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, psp_ring_reg); |
| 139 | 139 | ||
| 140 | /* there might be handshake issue with hardware which needs delay */ | 140 | /* there might be handshake issue with hardware which needs delay */ |
| 141 | mdelay(20); | 141 | mdelay(20); |
| 142 | 142 | ||
| 143 | /* Wait for response flag (bit 31) in C2PMSG_64 */ | 143 | /* Wait for response flag (bit 31) in C2PMSG_64 */ |
| 144 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | 144 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), |
| 145 | 0x80000000, 0x8000FFFF, false); | 145 | 0x80000000, 0x8000FFFF, false); |
| 146 | } | 146 | } |
| 147 | 147 | ||
| @@ -174,7 +174,7 @@ static uint32_t psp_v15_0_0_ring_get_wptr(struct psp_context *psp) · 26 unmodified lines | |||
| 174 | if (amdgpu_sriov_vf(adev)) | 174 | if (amdgpu_sriov_vf(adev)) |
| 175 | data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_102); | 175 | data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_102); |
| 176 | else | 176 | else |
| 177 | data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_67); | 177 | data = RREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67); |
| 178 | 178 | ||
| 179 | return data; | 179 | return data; |
| 180 | } | 180 | } |
| @@ -188,7 +188,7 @@ static void psp_v15_0_0_ring_set_wptr(struct psp_context *psp, uint32_t value) · 7 unmodified lines | |||
| 188 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_101, | 188 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_101, |
| 189 | GFX_CTRL_CMD_ID_CONSUME_CMD); | 189 | GFX_CTRL_CMD_ID_CONSUME_CMD); |
| 190 | } else | 190 | } else |
| 191 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_67, value); | 191 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67, value); |
| 192 | } | 192 | } |
| 193 | 193 | ||
| 194 | static const struct psp_funcs psp_v15_0_0_funcs = { | 194 | static const struct psp_funcs psp_v15_0_0_funcs = { |
| @@ -858,7 +858,9 @@ static int soc21_common_early_init(struct amdgpu_ip_block *ip_block) · 857 unmodified lines | |||
| 858 | AMD_CG_SUPPORT_IH_CG | | 858 | AMD_CG_SUPPORT_IH_CG | |
| 859 | AMD_CG_SUPPORT_BIF_MGCG | | 859 | AMD_CG_SUPPORT_BIF_MGCG | |
| 860 | AMD_CG_SUPPORT_BIF_LS; | 860 | AMD_CG_SUPPORT_BIF_LS; |
| 861 | adev->pg_flags = AMD_PG_SUPPORT_VCN | | 861 | adev->pg_flags = AMD_PG_SUPPORT_VCN_DPG | |
| 862 | AMD_PG_SUPPORT_VCN | | ||
| 863 | AMD_PG_SUPPORT_JPEG_DPG | | ||
| 862 | AMD_PG_SUPPORT_JPEG | | 864 | AMD_PG_SUPPORT_JPEG | |
| 863 | AMD_PG_SUPPORT_GFX_PG; | 865 | AMD_PG_SUPPORT_GFX_PG; |
| 864 | adev->external_rev_id = adev->rev_id + 0x1; | 866 | adev->external_rev_id = adev->rev_id + 0x1; |
| @@ -174,6 +174,10 @@ static int vcn_v5_0_0_sw_init(struct amdgpu_ip_block *ip_block) · 173 unmodified lines | |||
| 174 | fw_shared->present_flag_0 = cpu_to_le32(AMDGPU_FW_SHARED_FLAG_0_UNIFIED_QUEUE); | 174 | fw_shared->present_flag_0 = cpu_to_le32(AMDGPU_FW_SHARED_FLAG_0_UNIFIED_QUEUE); |
| 175 | fw_shared->sq.is_enabled = 1; | 175 | fw_shared->sq.is_enabled = 1; |
| 176 | 176 | ||
| 177 | fw_shared->present_flag_0 |= cpu_to_le32(AMDGPU_VCN_SMU_DPM_INTERFACE_FLAG); | ||
| 178 | fw_shared->smu_dpm_interface.smu_interface_type = (adev->flags & AMD_IS_APU) ? | ||
| 179 | AMDGPU_VCN_SMU_DPM_INTERFACE_APU : AMDGPU_VCN_SMU_DPM_INTERFACE_DGPU; | ||
| 180 | |||
| 177 | if (amdgpu_vcnfw_log) | 181 | if (amdgpu_vcnfw_log) |
| 178 | amdgpu_vcn_fwlog_init(&adev->vcn.inst[i]); | 182 | amdgpu_vcn_fwlog_init(&adev->vcn.inst[i]); |
| 179 | 183 | ||
| @@ -593,6 +593,7 @@ int pqm_update_queue_properties(struct process_queue_manager *pqm, · 592 unmodified lines | |||
| 593 | p->queue_size)) { | 593 | p->queue_size)) { |
| 594 | pr_debug("ring buf 0x%llx size 0x%llx not mapped on GPU\n", | 594 | pr_debug("ring buf 0x%llx size 0x%llx not mapped on GPU\n", |
| 595 | p->queue_address, p->queue_size); | 595 | p->queue_address, p->queue_size); |
| 596 | amdgpu_bo_unreserve(vm->root.bo); | ||
| 596 | return -EFAULT; | 597 | return -EFAULT; |
| 597 | } | 598 | } |
| 598 | 599 | ||
| @@ -1706,6 +1706,7 @@ __set_dm_plane_colorop_3dlut(struct drm_plane_state *plane_state, · 1705 unmodified lines | |||
| 1706 | struct dc_transfer_func *tf = &dc_plane_state->in_shaper_func; | 1706 | struct dc_transfer_func *tf = &dc_plane_state->in_shaper_func; |
| 1707 | struct drm_atomic_state *state = plane_state->state; | 1707 | struct drm_atomic_state *state = plane_state->state; |
| 1708 | const struct amdgpu_device *adev = drm_to_adev(colorop->dev); | 1708 | const struct amdgpu_device *adev = drm_to_adev(colorop->dev); |
| 1709 | bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | ||
| 1709 | const struct drm_device *dev = colorop->dev; | 1710 | const struct drm_device *dev = colorop->dev; |
| 1710 | const struct drm_color_lut32 *lut3d; | 1711 | const struct drm_color_lut32 *lut3d; |
| 1711 | uint32_t lut3d_size; | 1712 | uint32_t lut3d_size; |
| @@ -1722,7 +1723,7 @@ __set_dm_plane_colorop_3dlut(struct drm_plane_state *plane_state, · 10 unmodified lines | |||
| 1722 | } | 1723 | } |
| 1723 | 1724 | ||
| 1724 | if (colorop_state && !colorop_state->bypass && colorop->type == DRM_COLOROP_3D_LUT) { | 1725 | if (colorop_state && !colorop_state->bypass && colorop->type == DRM_COLOROP_3D_LUT) { |
| 1725 | if (!adev->dm.dc->caps.color.dpp.hw_3d_lut) { | 1726 | if (!has_3dlut) { |
| 1726 | drm_dbg(dev, "3D LUT is not supported by hardware\n"); | 1727 | drm_dbg(dev, "3D LUT is not supported by hardware\n"); |
| 1727 | return -EINVAL; | 1728 | return -EINVAL; |
| 1728 | } | 1729 | } |
| @@ -1875,6 +1876,7 @@ amdgpu_dm_plane_set_colorop_properties(struct drm_plane_state *plane_state, · 146 unmodified lines | |||
| 1875 | struct drm_colorop *colorop = plane_state->color_pipeline; | 1876 | struct drm_colorop *colorop = plane_state->color_pipeline; |
| 1876 | struct drm_device *dev = plane_state->plane->dev; | 1877 | struct drm_device *dev = plane_state->plane->dev; |
| 1877 | struct amdgpu_device *adev = drm_to_adev(dev); | 1878 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 1879 | bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | ||
| 1878 | int ret; | 1880 | int ret; |
| 1879 | 1881 | ||
| 1880 | /* 1D Curve - DEGAM TF */ | 1882 | /* 1D Curve - DEGAM TF */ |
| @@ -1907,7 +1909,7 @@ amdgpu_dm_plane_set_colorop_properties(struct drm_plane_state *plane_state, · 26 unmodified lines | |||
| 1907 | if (ret) | 1909 | if (ret) |
| 1908 | return ret; | 1910 | return ret; |
| 1909 | 1911 | ||
| 1910 | if (adev->dm.dc->caps.color.dpp.hw_3d_lut) { | 1912 | if (has_3dlut) { |
| 1911 | /* 1D Curve & LUT - SHAPER TF & LUT */ | 1913 | /* 1D Curve & LUT - SHAPER TF & LUT */ |
| 1912 | colorop = colorop->next; | 1914 | colorop = colorop->next; |
| 1913 | if (!colorop) { | 1915 | if (!colorop) { |
| @@ -60,6 +60,7 @@ int amdgpu_dm_initialize_default_pipeline(struct drm_plane *plane, struct drm_pr · 59 unmodified lines | |||
| 60 | struct drm_colorop *ops[MAX_COLOR_PIPELINE_OPS]; | 60 | struct drm_colorop *ops[MAX_COLOR_PIPELINE_OPS]; |
| 61 | struct drm_device *dev = plane->dev; | 61 | struct drm_device *dev = plane->dev; |
| 62 | struct amdgpu_device *adev = drm_to_adev(dev); | 62 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 63 | bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | ||
| 63 | int ret; | 64 | int ret; |
| 64 | int i = 0; | 65 | int i = 0; |
| 65 | 66 | ||
| @@ -112,7 +113,7 @@ int amdgpu_dm_initialize_default_pipeline(struct drm_plane *plane, struct drm_pr · 46 unmodified lines | |||
| 112 | 113 | ||
| 113 | i++; | 114 | i++; |
| 114 | 115 | ||
| 115 | if (adev->dm.dc->caps.color.dpp.hw_3d_lut) { | 116 | if (has_3dlut) { |
| 116 | /* 1D curve - SHAPER TF */ | 117 | /* 1D curve - SHAPER TF */ |
| 117 | ops[i] = kzalloc_obj(*ops[0]); | 118 | ops[i] = kzalloc_obj(*ops[0]); |
| 118 | if (!ops[i]) { | 119 | if (!ops[i]) { |
| @@ -765,15 +765,15 @@ int amdgpu_dm_crtc_init(struct amdgpu_display_manager *dm, · 764 unmodified lines | |||
| 765 | dm->adev->mode_info.crtcs[crtc_index] = acrtc; | 765 | dm->adev->mode_info.crtcs[crtc_index] = acrtc; |
| 766 | 766 | ||
| 767 | /* Don't enable DRM CRTC degamma property for | 767 | /* Don't enable DRM CRTC degamma property for |
| 768 | * 1. Degamma is replaced by color pipeline. | 768 | * 1. DCE since it doesn't support programmable degamma anywhere. |
| 769 | * 2. DCE since it doesn't support programmable degamma anywhere. | 769 | * 2. DCN401 since pre-blending degamma LUT doesn't apply to cursor. |
| 770 | * 3. DCN401 since pre-blending degamma LUT doesn't apply to cursor. | 770 | * Note: DEGAMMA properties are created even if the primary plane has the |
| 771 | * COLOR_PIPELINE property. User space can use either the DEGAMMA properties | ||
| 772 | * or the COLOR_PIPELINE property. An atomic commit which attempts to enable | ||
| 773 | * both is rejected. | ||
| 771 | */ | 774 | */ |
| 772 | if (plane->color_pipeline_property) | 775 | has_degamma = dm->adev->dm.dc->caps.color.dpp.dcn_arch && |
| 773 | has_degamma = false; | 776 | dm->adev->dm.dc->ctx->dce_version != DCN_VERSION_4_01; |
| 774 | else | ||
| 775 | has_degamma = dm->adev->dm.dc->caps.color.dpp.dcn_arch && | ||
| 776 | dm->adev->dm.dc->ctx->dce_version != DCN_VERSION_4_01; | ||
| 777 | 777 | ||
| 778 | drm_crtc_enable_color_mgmt(&acrtc->base, has_degamma ? MAX_COLOR_LUT_ENTRIES : 0, | 778 | drm_crtc_enable_color_mgmt(&acrtc->base, has_degamma ? MAX_COLOR_LUT_ENTRIES : 0, |
| 779 | true, MAX_COLOR_LUT_ENTRIES); | 779 | true, MAX_COLOR_LUT_ENTRIES); |
| @@ -1256,6 +1256,14 @@ static int amdgpu_dm_plane_atomic_check(struct drm_plane *plane, · 1255 unmodified lines | |||
| 1256 | if (ret) | 1256 | if (ret) |
| 1257 | return ret; | 1257 | return ret; |
| 1258 | 1258 | ||
| 1259 | /* Reject commits that attempt to use both COLOR_PIPELINE and CRTC DEGAMMA_LUT */ | ||
| 1260 | if (new_plane_state->color_pipeline && new_crtc_state->degamma_lut) { | ||
| 1261 | drm_dbg_atomic(plane->dev, | ||
| 1262 | "[PLANE:%d:%s] COLOR_PIPELINE and CRTC DEGAMMA_LUT cannot be enabled simultaneously\n", | ||
| 1263 | plane->base.id, plane->name); | ||
| 1264 | return -EINVAL; | ||
| 1265 | } | ||
| 1266 | |||
| 1259 | ret = amdgpu_dm_plane_fill_dc_scaling_info(adev, new_plane_state, &scaling_info); | 1267 | ret = amdgpu_dm_plane_fill_dc_scaling_info(adev, new_plane_state, &scaling_info); |
| 1260 | if (ret) | 1268 | if (ret) |
| 1261 | return ret; | 1269 | return ret; |
| @@ -170,11 +170,11 @@ struct dc_stream_state *dc_create_stream_for_sink( · 169 unmodified lines | |||
| 170 | if (sink == NULL) | 170 | if (sink == NULL) |
| 171 | goto fail; | 171 | goto fail; |
| 172 | 172 | ||
| 173 | stream = kzalloc_obj(struct dc_stream_state); | 173 | stream = kzalloc_obj(struct dc_stream_state, GFP_ATOMIC); |
| 174 | if (stream == NULL) | 174 | if (stream == NULL) |
| 175 | goto fail; | 175 | goto fail; |
| 176 | 176 | ||
| 177 | stream->update_scratch = kzalloc((int32_t) dc_update_scratch_space_size(), GFP_KERNEL); | 177 | stream->update_scratch = kzalloc((int32_t) dc_update_scratch_space_size(), GFP_ATOMIC); |
| 178 | if (stream->update_scratch == NULL) | 178 | if (stream->update_scratch == NULL) |
| 179 | goto fail; | 179 | goto fail; |
| 180 | 180 | ||
| @@ -38,7 +38,11 @@ · 37 unmodified lines | |||
| 38 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 0),\ | 38 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 0),\ |
| 39 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 1),\ | 39 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 1),\ |
| 40 | SR(DISPCLK_FREQ_CHANGE_CNTL),\ | 40 | SR(DISPCLK_FREQ_CHANGE_CNTL),\ |
| 41 | SR(DC_MEM_GLOBAL_PWR_REQ_CNTL) | 41 | SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ |
| 42 | SR(MICROSECOND_TIME_BASE_DIV),\ | ||
| 43 | SR(MILLISECOND_TIME_BASE_DIV),\ | ||
| 44 | SR(DCCG_GATE_DISABLE_CNTL),\ | ||
| 45 | SR(DCCG_GATE_DISABLE_CNTL2) | ||
| 42 | 46 | ||
| 43 | #define DCCG_REG_LIST_DCN2() \ | 47 | #define DCCG_REG_LIST_DCN2() \ |
| 44 | DCCG_COMMON_REG_LIST_DCN_BASE(),\ | 48 | DCCG_COMMON_REG_LIST_DCN_BASE(),\ |
| @@ -96,6 +96,25 @@ static void dccg21_update_dpp_dto(struct dccg *dccg, int dpp_inst, int req_dppcl · 95 unmodified lines | |||
| 96 | dccg->pipe_dppclk_khz[dpp_inst] = req_dppclk; | 96 | dccg->pipe_dppclk_khz[dpp_inst] = req_dppclk; |
| 97 | } | 97 | } |
| 98 | 98 | ||
| 99 | /* | ||
| 100 | * On DCN21 S0i3 resume, BIOS programs MICROSECOND_TIME_BASE_DIV to | ||
| 101 | * 0x00120464 as a marker that golden init has already been done. | ||
| 102 | * dcn21_s0i3_golden_init_wa() reads this marker later in bios_golden_init() | ||
| 103 | * to decide whether to skip golden init. | ||
| 104 | * | ||
| 105 | * dccg2_init() unconditionally overwrites MICROSECOND_TIME_BASE_DIV to | ||
| 106 | * 0x00120264, destroying the marker before it can be read. | ||
| 107 | * | ||
| 108 | * Guard the call: if the S0i3 marker is present, skip dccg2_init() so the | ||
| 109 | * WA can function correctly. bios_golden_init() will handle init in that case. | ||
| 110 | */ | ||
| 111 | static void dccg21_init(struct dccg *dccg) | ||
| 112 | { | ||
| 113 | if (dccg2_is_s0i3_golden_init_wa_done(dccg)) | ||
| 114 | return; | ||
| 115 | |||
| 116 | dccg2_init(dccg); | ||
| 117 | } | ||
| 99 | 118 | ||
| 100 | static const struct dccg_funcs dccg21_funcs = { | 119 | static const struct dccg_funcs dccg21_funcs = { |
| 101 | .update_dpp_dto = dccg21_update_dpp_dto, | 120 | .update_dpp_dto = dccg21_update_dpp_dto, |
| @@ -103,7 +122,7 @@ static const struct dccg_funcs dccg21_funcs = { · 1 unmodified line | |||
| 103 | .set_fifo_errdet_ovr_en = dccg2_set_fifo_errdet_ovr_en, | 122 | .set_fifo_errdet_ovr_en = dccg2_set_fifo_errdet_ovr_en, |
| 104 | .otg_add_pixel = dccg2_otg_add_pixel, | 123 | .otg_add_pixel = dccg2_otg_add_pixel, |
| 105 | .otg_drop_pixel = dccg2_otg_drop_pixel, | 124 | .otg_drop_pixel = dccg2_otg_drop_pixel, |
| 106 | .dccg_init = dccg2_init, | 125 | .dccg_init = dccg21_init, |
| 107 | .refclk_setup = dccg2_refclk_setup, /* Deprecated - for backward compatibility only */ | 126 | .refclk_setup = dccg2_refclk_setup, /* Deprecated - for backward compatibility only */ |
| 108 | .allow_clock_gating = dccg2_allow_clock_gating, | 127 | .allow_clock_gating = dccg2_allow_clock_gating, |
| 109 | .enable_memory_low_power = dccg2_enable_memory_low_power, | 128 | .enable_memory_low_power = dccg2_enable_memory_low_power, |
| @@ -34,7 +34,13 @@ · 33 unmodified lines | |||
| 34 | DCCG_SRII(DTO_PARAM, DPPCLK, 1),\ | 34 | DCCG_SRII(DTO_PARAM, DPPCLK, 1),\ |
| 35 | DCCG_SRII(DTO_PARAM, DPPCLK, 2),\ | 35 | DCCG_SRII(DTO_PARAM, DPPCLK, 2),\ |
| 36 | DCCG_SRII(DTO_PARAM, DPPCLK, 3),\ | 36 | DCCG_SRII(DTO_PARAM, DPPCLK, 3),\ |
| 37 | SR(REFCLK_CNTL) | 37 | SR(REFCLK_CNTL),\ |
| 38 | SR(DISPCLK_FREQ_CHANGE_CNTL),\ | ||
| 39 | SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | ||
| 40 | SR(MICROSECOND_TIME_BASE_DIV),\ | ||
| 41 | SR(MILLISECOND_TIME_BASE_DIV),\ | ||
| 42 | SR(DCCG_GATE_DISABLE_CNTL),\ | ||
| 43 | SR(DCCG_GATE_DISABLE_CNTL2) | ||
| 38 | 44 | ||
| 39 | #define DCCG_MASK_SH_LIST_DCN301(mask_sh) \ | 45 | #define DCCG_MASK_SH_LIST_DCN301(mask_sh) \ |
| 40 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_ENABLE, DPPCLK, 0, mask_sh),\ | 46 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_ENABLE, DPPCLK, 0, mask_sh),\ |
| @@ -64,9 +64,12 @@ · 63 unmodified lines | |||
| 64 | SR(DSCCLK1_DTO_PARAM),\ | 64 | SR(DSCCLK1_DTO_PARAM),\ |
| 65 | SR(DSCCLK2_DTO_PARAM),\ | 65 | SR(DSCCLK2_DTO_PARAM),\ |
| 66 | SR(DSCCLK_DTO_CTRL),\ | 66 | SR(DSCCLK_DTO_CTRL),\ |
| 67 | SR(DCCG_GATE_DISABLE_CNTL),\ | ||
| 67 | SR(DCCG_GATE_DISABLE_CNTL2),\ | 68 | SR(DCCG_GATE_DISABLE_CNTL2),\ |
| 68 | SR(DCCG_GATE_DISABLE_CNTL3),\ | 69 | SR(DCCG_GATE_DISABLE_CNTL3),\ |
| 69 | SR(HDMISTREAMCLK0_DTO_PARAM) | 70 | SR(HDMISTREAMCLK0_DTO_PARAM),\ |
| 71 | SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | ||
| 72 | SR(MICROSECOND_TIME_BASE_DIV) | ||
| 70 | 73 | ||
| 71 | 74 | ||
| 72 | #define DCCG_MASK_SH_LIST_DCN31(mask_sh) \ | 75 | #define DCCG_MASK_SH_LIST_DCN31(mask_sh) \ |
| @@ -70,11 +70,14 @@ · 69 unmodified lines | |||
| 70 | SR(DSCCLK2_DTO_PARAM),\ | 70 | SR(DSCCLK2_DTO_PARAM),\ |
| 71 | SR(DSCCLK3_DTO_PARAM),\ | 71 | SR(DSCCLK3_DTO_PARAM),\ |
| 72 | SR(DSCCLK_DTO_CTRL),\ | 72 | SR(DSCCLK_DTO_CTRL),\ |
| 73 | SR(DCCG_GATE_DISABLE_CNTL),\ | ||
| 73 | SR(DCCG_GATE_DISABLE_CNTL2),\ | 74 | SR(DCCG_GATE_DISABLE_CNTL2),\ |
| 74 | SR(DCCG_GATE_DISABLE_CNTL3),\ | 75 | SR(DCCG_GATE_DISABLE_CNTL3),\ |
| 75 | SR(HDMISTREAMCLK0_DTO_PARAM),\ | 76 | SR(HDMISTREAMCLK0_DTO_PARAM),\ |
| 76 | SR(OTG_PIXEL_RATE_DIV),\ | 77 | SR(OTG_PIXEL_RATE_DIV),\ |
| 77 | SR(DTBCLK_P_CNTL) | 78 | SR(DTBCLK_P_CNTL),\ |
| 79 | SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | ||
| 80 | SR(MICROSECOND_TIME_BASE_DIV) | ||
| 78 | 81 | ||
| 79 | #define DCCG_MASK_SH_LIST_DCN314_COMMON(mask_sh) \ | 82 | #define DCCG_MASK_SH_LIST_DCN314_COMMON(mask_sh) \ |
| 80 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_DB_EN, DPPCLK, 0, mask_sh),\ | 83 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_DB_EN, DPPCLK, 0, mask_sh),\ |
| @@ -72,7 +72,11 @@ void dcn401_initialize_min_clocks(struct dc *dc) · 71 unmodified lines | |||
| 72 | * audio corruption. Read current DISPCLK from DENTIST and request the same | 72 | * audio corruption. Read current DISPCLK from DENTIST and request the same |
| 73 | * freq to ensure that the timing is valid and unchanged. | 73 | * freq to ensure that the timing is valid and unchanged. |
| 74 | */ | 74 | */ |
| 75 | clocks->dispclk_khz = dc->clk_mgr->funcs->get_dispclk_from_dentist(dc->clk_mgr); | 75 | if (dc->clk_mgr->funcs->get_dispclk_from_dentist) { |
| 76 | clocks->dispclk_khz = dc->clk_mgr->funcs->get_dispclk_from_dentist(dc->clk_mgr); | ||
| 77 | } else { | ||
| 78 | clocks->dispclk_khz = dc->clk_mgr->boot_snapshot.dispclk * 1000; | ||
| 79 | } | ||
| 76 | } | 80 | } |
| 77 | clocks->ref_dtbclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].dtbclk_mhz * 1000; | 81 | clocks->ref_dtbclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].dtbclk_mhz * 1000; |
| 78 | clocks->fclk_p_state_change_support = true; | 82 | clocks->fclk_p_state_change_support = true; |
| @@ -82,6 +82,24 @@ · 81 unmodified lines | |||
| 82 | #define regMPASP_SMN_IH_SW_INT_CTRL 0x0142 | 82 | #define regMPASP_SMN_IH_SW_INT_CTRL 0x0142 |
| 83 | #define regMPASP_SMN_IH_SW_INT_CTRL_BASE_IDX 0 | 83 | #define regMPASP_SMN_IH_SW_INT_CTRL_BASE_IDX 0 |
| 84 | 84 | ||
| 85 | // addressBlock: mp_SmuMpASPPub_PcruDec | ||
| 86 | // base address: 0x3800000 | ||
| 87 | #define regMPASP_PCRU1_MPASP_C2PMSG_64 0x4280 | ||
| 88 | #define regMPASP_PCRU1_MPASP_C2PMSG_64_BASE_IDX 3 | ||
| 89 | #define regMPASP_PCRU1_MPASP_C2PMSG_65 0x4281 | ||
| 90 | #define regMPASP_PCRU1_MPASP_C2PMSG_65_BASE_IDX 3 | ||
| 91 | #define regMPASP_PCRU1_MPASP_C2PMSG_66 0x4282 | ||
| 92 | #define regMPASP_PCRU1_MPASP_C2PMSG_66_BASE_IDX 3 | ||
| 93 | #define regMPASP_PCRU1_MPASP_C2PMSG_67 0x4283 | ||
| 94 | #define regMPASP_PCRU1_MPASP_C2PMSG_67_BASE_IDX 3 | ||
| 95 | #define regMPASP_PCRU1_MPASP_C2PMSG_68 0x4284 | ||
| 96 | #define regMPASP_PCRU1_MPASP_C2PMSG_68_BASE_IDX 3 | ||
| 97 | #define regMPASP_PCRU1_MPASP_C2PMSG_69 0x4285 | ||
| 98 | #define regMPASP_PCRU1_MPASP_C2PMSG_69_BASE_IDX 3 | ||
| 99 | #define regMPASP_PCRU1_MPASP_C2PMSG_70 0x4286 | ||
| 100 | #define regMPASP_PCRU1_MPASP_C2PMSG_70_BASE_IDX 3 | ||
| 101 | #define regMPASP_PCRU1_MPASP_C2PMSG_71 0x4287 | ||
| 102 | #define regMPASP_PCRU1_MPASP_C2PMSG_71_BASE_IDX 3 | ||
| 85 | 103 | ||
| 86 | // addressBlock: mp_SmuMp1_SmnDec | 104 | // addressBlock: mp_SmuMp1_SmnDec |
| 87 | // base address: 0x0 | 105 | // base address: 0x0 |
| @@ -2034,6 +2034,7 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu, · 2033 unmodified lines | |||
| 2034 | smu, SMU_DRIVER_TABLE_GPU_METRICS); | 2034 | smu, SMU_DRIVER_TABLE_GPU_METRICS); |
| 2035 | SmuMetricsExternal_t metrics_ext; | 2035 | SmuMetricsExternal_t metrics_ext; |
| 2036 | SmuMetrics_t *metrics = &metrics_ext.SmuMetrics; | 2036 | SmuMetrics_t *metrics = &metrics_ext.SmuMetrics; |
| 2037 | uint32_t mp1_ver = amdgpu_ip_version(smu->adev, MP1_HWIP, 0); | ||
| 2037 | int ret = 0; | 2038 | int ret = 0; |
| 2038 | 2039 | ||
| 2039 | ret = smu_cmn_get_metrics_table(smu, | 2040 | ret = smu_cmn_get_metrics_table(smu, |
| @@ -2058,7 +2059,12 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu, · 18 unmodified lines | |||
| 2058 | metrics->Vcn1ActivityPercentage); | 2059 | metrics->Vcn1ActivityPercentage); |
| 2059 | 2060 | ||
| 2060 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; | 2061 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; |
| 2061 | gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | 2062 | |
| 2063 | if ((mp1_ver == IP_VERSION(13, 0, 0) && smu->smc_fw_version <= 0x004e1e00) || | ||
| 2064 | (mp1_ver == IP_VERSION(13, 0, 10) && smu->smc_fw_version <= 0x00500800)) | ||
| 2065 | gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | ||
| 2066 | else | ||
| 2067 | gpu_metrics->energy_accumulator = UINT_MAX; | ||
| 2062 | 2068 | ||
| 2063 | if (metrics->AverageGfxActivity <= SMU_13_0_0_BUSY_THRESHOLD) | 2069 | if (metrics->AverageGfxActivity <= SMU_13_0_0_BUSY_THRESHOLD) |
| 2064 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; | 2070 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; |
| @@ -2216,7 +2222,8 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu) · 151 unmodified lines | |||
| 2216 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | | 2222 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2217 | BIT(PP_OD_FEATURE_UCLK_BIT) | | 2223 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2218 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | | 2224 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2219 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | 2225 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | |
| 2226 | BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | ||
| 2220 | res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table); | 2227 | res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table); |
| 2221 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; | 2228 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2222 | if (res == 0) | 2229 | if (res == 0) |
| @@ -2065,7 +2065,8 @@ static ssize_t smu_v13_0_7_get_gpu_metrics(struct smu_context *smu, · 2064 unmodified lines | |||
| 2065 | metrics->Vcn1ActivityPercentage); | 2065 | metrics->Vcn1ActivityPercentage); |
| 2066 | 2066 | ||
| 2067 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; | 2067 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; |
| 2068 | gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | 2068 | gpu_metrics->energy_accumulator = smu->smc_fw_version <= 0x00521400 ? |
| 2069 | metrics->EnergyAccumulator : UINT_MAX; | ||
| 2069 | 2070 | ||
| 2070 | if (metrics->AverageGfxActivity <= SMU_13_0_7_BUSY_THRESHOLD) | 2071 | if (metrics->AverageGfxActivity <= SMU_13_0_7_BUSY_THRESHOLD) |
| 2071 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; | 2072 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; |
| @@ -2223,7 +2224,8 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu) · 151 unmodified lines | |||
| 2223 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | | 2224 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2224 | BIT(PP_OD_FEATURE_UCLK_BIT) | | 2225 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2225 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | | 2226 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2226 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | 2227 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | |
| 2228 | BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | ||
| 2227 | res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table); | 2229 | res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table); |
| 2228 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; | 2230 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2229 | if (res == 0) | 2231 | if (res == 0) |
| @@ -2311,7 +2311,8 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu) · 2310 unmodified lines | |||
| 2311 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | | 2311 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2312 | BIT(PP_OD_FEATURE_UCLK_BIT) | | 2312 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2313 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | | 2313 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2314 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | 2314 | BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | |
| 2315 | BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | ||
| 2315 | res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table); | 2316 | res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table); |
| 2316 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; | 2317 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2317 | if (res == 0) | 2318 | if (res == 0) |
| @@ -1881,6 +1881,14 @@ static int samsung_dsim_register_te_irq(struct samsung_dsim *dsi, struct device · 1880 unmodified lines | |||
| 1881 | return 0; | 1881 | return 0; |
| 1882 | } | 1882 | } |
| 1883 | 1883 | ||
| 1884 | static void samsung_dsim_unregister_te_irq(struct samsung_dsim *dsi) | ||
| 1885 | { | ||
| 1886 | if (dsi->te_gpio) { | ||
| 1887 | free_irq(gpiod_to_irq(dsi->te_gpio), dsi); | ||
| 1888 | gpiod_put(dsi->te_gpio); | ||
| 1889 | } | ||
| 1890 | } | ||
| 1891 | |||
| 1884 | static int samsung_dsim_host_attach(struct mipi_dsi_host *host, | 1892 | static int samsung_dsim_host_attach(struct mipi_dsi_host *host, |
| 1885 | struct mipi_dsi_device *device) | 1893 | struct mipi_dsi_device *device) |
| 1886 | { | 1894 | { |
| @@ -1961,7 +1969,7 @@ of_find_panel_or_bridge: · 74 unmodified lines | |||
| 1961 | if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO)) { | 1969 | if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO)) { |
| 1962 | ret = samsung_dsim_register_te_irq(dsi, &device->dev); | 1970 | ret = samsung_dsim_register_te_irq(dsi, &device->dev); |
| 1963 | if (ret) | 1971 | if (ret) |
| 1964 | return ret; | 1972 | goto err_remove_bridge; |
| 1965 | } | 1973 | } |
| 1966 | 1974 | ||
| 1967 | // The next bridge can be used by host_ops->attach | 1975 | // The next bridge can be used by host_ops->attach |
| @@ -1982,15 +1990,12 @@ of_find_panel_or_bridge: · 14 unmodified lines | |||
| 1982 | err_release_next_bridge: | 1990 | err_release_next_bridge: |
| 1983 | drm_bridge_put(dsi->bridge.next_bridge); | 1991 | drm_bridge_put(dsi->bridge.next_bridge); |
| 1984 | dsi->bridge.next_bridge = NULL; | 1992 | dsi->bridge.next_bridge = NULL; |
| 1985 | return ret; | ||
| 1986 | } | ||
| 1987 | 1993 | ||
| 1988 | static void samsung_dsim_unregister_te_irq(struct samsung_dsim *dsi) | 1994 | if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO)) |
| 1989 | { | 1995 | samsung_dsim_unregister_te_irq(dsi); |
| 1990 | if (dsi->te_gpio) { | 1996 | err_remove_bridge: |
| 1991 | free_irq(gpiod_to_irq(dsi->te_gpio), dsi); | 1997 | drm_bridge_remove(&dsi->bridge); |
| 1992 | gpiod_put(dsi->te_gpio); | 1998 | return ret; |
| 1993 | } | ||
| 1994 | } | 1999 | } |
| 1995 | 2000 | ||
| 1996 | static int samsung_dsim_host_detach(struct mipi_dsi_host *host, | 2001 | static int samsung_dsim_host_detach(struct mipi_dsi_host *host, |
| @@ -2049,7 +2049,9 @@ struct dw_dp *dw_dp_bind(struct device *dev, struct drm_encoder *encoder, · 2048 unmodified lines | |||
| 2049 | bridge->type = DRM_MODE_CONNECTOR_DisplayPort; | 2049 | bridge->type = DRM_MODE_CONNECTOR_DisplayPort; |
| 2050 | bridge->ycbcr_420_allowed = true; | 2050 | bridge->ycbcr_420_allowed = true; |
| 2051 | 2051 | ||
| 2052 | devm_drm_bridge_add(dev, bridge); | 2052 | ret = devm_drm_bridge_add(dev, bridge); |
| 2053 | if (ret) | ||
| 2054 | return ERR_PTR(ret); | ||
| 2053 | 2055 | ||
| 2054 | dp->aux.dev = dev; | 2056 | dp->aux.dev = dev; |
| 2055 | dp->aux.drm_dev = encoder->dev; | 2057 | dp->aux.drm_dev = encoder->dev; |
| @@ -351,9 +351,9 @@ static u8 sn65dsi83_get_dsi_range(struct sn65dsi83 *ctx, · 350 unmodified lines | |||
| 351 | * DSI_CLK = mode clock * bpp / dsi_data_lanes / 2 | 351 | * DSI_CLK = mode clock * bpp / dsi_data_lanes / 2 |
| 352 | * the 2 is there because the bus is DDR. | 352 | * the 2 is there because the bus is DDR. |
| 353 | */ | 353 | */ |
| 354 | return DIV_ROUND_UP(clamp((unsigned int)mode->clock * | 354 | return clamp((unsigned int)mode->clock * |
| 355 | mipi_dsi_pixel_format_to_bpp(ctx->dsi->format) / | 355 | mipi_dsi_pixel_format_to_bpp(ctx->dsi->format) / |
| 356 | ctx->dsi->lanes / 2, 40000U, 500000U), 5000U); | 356 | ctx->dsi->lanes / 2, 40000U, 500000U) / 5000U; |
| 357 | } | 357 | } |
| 358 | 358 | ||
| 359 | static u8 sn65dsi83_get_dsi_div(struct sn65dsi83 *ctx) | 359 | static u8 sn65dsi83_get_dsi_div(struct sn65dsi83 *ctx) |
| @@ -517,6 +517,7 @@ static void sn65dsi83_atomic_pre_enable(struct drm_bridge *bridge, · 157 unmodified lines | |||
| 517 | struct drm_atomic_state *state) | 517 | struct drm_atomic_state *state) |
| 518 | { | 518 | { |
| 519 | struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge); | 519 | struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge); |
| 520 | const unsigned int dual_factor = ctx->lvds_dual_link ? 2 : 1; | ||
| 520 | const struct drm_bridge_state *bridge_state; | 521 | const struct drm_bridge_state *bridge_state; |
| 521 | const struct drm_crtc_state *crtc_state; | 522 | const struct drm_crtc_state *crtc_state; |
| 522 | const struct drm_display_mode *mode; | 523 | const struct drm_display_mode *mode; |
| @@ -653,18 +654,18 @@ static void sn65dsi83_atomic_pre_enable(struct drm_bridge *bridge, · 130 unmodified lines | |||
| 653 | /* 32 + 1 pixel clock to ensure proper operation */ | 654 | /* 32 + 1 pixel clock to ensure proper operation */ |
| 654 | le16val = cpu_to_le16(32 + 1); | 655 | le16val = cpu_to_le16(32 + 1); |
| 655 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_SYNC_DELAY_LOW, &le16val, 2); | 656 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_SYNC_DELAY_LOW, &le16val, 2); |
| 656 | le16val = cpu_to_le16(mode->hsync_end - mode->hsync_start); | 657 | le16val = cpu_to_le16((mode->hsync_end - mode->hsync_start) / dual_factor); |
| 657 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_HSYNC_PULSE_WIDTH_LOW, | 658 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_HSYNC_PULSE_WIDTH_LOW, |
| 658 | &le16val, 2); | 659 | &le16val, 2); |
| 659 | le16val = cpu_to_le16(mode->vsync_end - mode->vsync_start); | 660 | le16val = cpu_to_le16(mode->vsync_end - mode->vsync_start); |
| 660 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_VSYNC_PULSE_WIDTH_LOW, | 661 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_VSYNC_PULSE_WIDTH_LOW, |
| 661 | &le16val, 2); | 662 | &le16val, 2); |
| 662 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_BACK_PORCH, | 663 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_BACK_PORCH, |
| 663 | mode->htotal - mode->hsync_end); | 664 | (mode->htotal - mode->hsync_end) / dual_factor); |
| 664 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_BACK_PORCH, | 665 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_BACK_PORCH, |
| 665 | mode->vtotal - mode->vsync_end); | 666 | mode->vtotal - mode->vsync_end); |
| 666 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_FRONT_PORCH, | 667 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_FRONT_PORCH, |
| 667 | mode->hsync_start - mode->hdisplay); | 668 | (mode->hsync_start - mode->hdisplay) / dual_factor); |
| 668 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_FRONT_PORCH, | 669 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_FRONT_PORCH, |
| 669 | mode->vsync_start - mode->vdisplay); | 670 | mode->vsync_start - mode->vdisplay); |
| 670 | regmap_write(ctx->regmap, REG_VID_CHA_TEST_PATTERN, 0x00); | 671 | regmap_write(ctx->regmap, REG_VID_CHA_TEST_PATTERN, 0x00); |
| @@ -1415,6 +1415,7 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, · 1414 unmodified lines | |||
| 1415 | { | 1415 | { |
| 1416 | struct ti_sn65dsi86 *pdata = dev_get_drvdata(adev->dev.parent); | 1416 | struct ti_sn65dsi86 *pdata = dev_get_drvdata(adev->dev.parent); |
| 1417 | struct device_node *np = pdata->dev->of_node; | 1417 | struct device_node *np = pdata->dev->of_node; |
| 1418 | const struct i2c_client *client = to_i2c_client(pdata->dev); | ||
| 1418 | int ret; | 1419 | int ret; |
| 1419 | 1420 | ||
| 1420 | pdata->next_bridge = devm_drm_of_get_bridge(&adev->dev, np, 1, 0); | 1421 | pdata->next_bridge = devm_drm_of_get_bridge(&adev->dev, np, 1, 0); |
| @@ -1433,8 +1434,9 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, · 12 unmodified lines | |||
| 1433 | ? DRM_MODE_CONNECTOR_DisplayPort : DRM_MODE_CONNECTOR_eDP; | 1434 | ? DRM_MODE_CONNECTOR_DisplayPort : DRM_MODE_CONNECTOR_eDP; |
| 1434 | 1435 | ||
| 1435 | if (pdata->bridge.type == DRM_MODE_CONNECTOR_DisplayPort) { | 1436 | if (pdata->bridge.type == DRM_MODE_CONNECTOR_DisplayPort) { |
| 1436 | pdata->bridge.ops = DRM_BRIDGE_OP_EDID | DRM_BRIDGE_OP_DETECT | | 1437 | pdata->bridge.ops = DRM_BRIDGE_OP_EDID | DRM_BRIDGE_OP_DETECT; |
| 1437 | DRM_BRIDGE_OP_HPD; | 1438 | if (client->irq) |
| 1439 | pdata->bridge.ops |= DRM_BRIDGE_OP_HPD; | ||
| 1438 | /* | 1440 | /* |
| 1439 | * If comms were already enabled they would have been enabled | 1441 | * If comms were already enabled they would have been enabled |
| 1440 | * with the wrong value of HPD_DISABLE. Update it now. Comms | 1442 | * with the wrong value of HPD_DISABLE. Update it now. Comms |
| @@ -930,7 +930,8 @@ int drm_client_modeset_probe(struct drm_client_dev *client, unsigned int width, · 929 unmodified lines | |||
| 930 | mutex_unlock(&client->modeset_mutex); | 930 | mutex_unlock(&client->modeset_mutex); |
| 931 | out: | 931 | out: |
| 932 | kfree(crtcs); | 932 | kfree(crtcs); |
| 933 | modes_destroy(dev, modes, connector_count); | 933 | if (modes) |
| 934 | modes_destroy(dev, modes, connector_count); | ||
| 934 | kfree(modes); | 935 | kfree(modes); |
| 935 | kfree(offsets); | 936 | kfree(offsets); |
| 936 | kfree(enabled); | 937 | kfree(enabled); |
| @@ -1338,14 +1338,14 @@ bool drm_gpusvm_range_pages_valid(struct drm_gpusvm *gpusvm, · 1337 unmodified lines | |||
| 1338 | EXPORT_SYMBOL_GPL(drm_gpusvm_range_pages_valid); | 1338 | EXPORT_SYMBOL_GPL(drm_gpusvm_range_pages_valid); |
| 1339 | 1339 | ||
| 1340 | /** | 1340 | /** |
| 1341 | * drm_gpusvm_range_pages_valid_unlocked() - GPU SVM range pages valid unlocked | 1341 | * drm_gpusvm_pages_valid_unlocked() - GPU SVM pages valid unlocked |
| 1342 | * @gpusvm: Pointer to the GPU SVM structure | 1342 | * @gpusvm: Pointer to the GPU SVM structure |
| 1343 | * @range: Pointer to the GPU SVM range structure | 1343 | * @svm_pages: Pointer to the GPU SVM pages structure |
| 1344 | * | 1344 | * |
| 1345 | * This function determines if a GPU SVM range pages are valid. Expected be | 1345 | * This function determines if a GPU SVM pages are valid. Expected be called |
| 1346 | * called without holding gpusvm->notifier_lock. | 1346 | * without holding gpusvm->notifier_lock. |
| 1347 | * | 1347 | * |
| 1348 | * Return: True if GPU SVM range has valid pages, False otherwise | 1348 | * Return: True if GPU SVM pages are valid, False otherwise |
| 1349 | */ | 1349 | */ |
| 1350 | static bool drm_gpusvm_pages_valid_unlocked(struct drm_gpusvm *gpusvm, | 1350 | static bool drm_gpusvm_pages_valid_unlocked(struct drm_gpusvm *gpusvm, |
| 1351 | struct drm_gpusvm_pages *svm_pages) | 1351 | struct drm_gpusvm_pages *svm_pages) |
| @@ -480,18 +480,8 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, · 479 unmodified lines | |||
| 480 | .start = start, | 480 | .start = start, |
| 481 | .end = end, | 481 | .end = end, |
| 482 | .pgmap_owner = pagemap->owner, | 482 | .pgmap_owner = pagemap->owner, |
| 483 | /* | 483 | .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT | |
| 484 | * FIXME: MIGRATE_VMA_SELECT_DEVICE_PRIVATE intermittently | 484 | MIGRATE_VMA_SELECT_DEVICE_PRIVATE, |
| 485 | * causes 'xe_exec_system_allocator --r *race*no*' to trigger aa | ||
| 486 | * engine reset and a hard hang due to getting stuck on a folio | ||
| 487 | * lock. This should work and needs to be root-caused. The only | ||
| 488 | * downside of not selecting MIGRATE_VMA_SELECT_DEVICE_PRIVATE | ||
| 489 | * is that device-to-device migrations won’t work; instead, | ||
| 490 | * memory will bounce through system memory. This path should be | ||
| 491 | * rare and only occur when the madvise attributes of memory are | ||
| 492 | * changed or atomics are being used. | ||
| 493 | */ | ||
| 494 | .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT, | ||
| 495 | }; | 485 | }; |
| 496 | unsigned long i, npages = npages_in_range(start, end); | 486 | unsigned long i, npages = npages_in_range(start, end); |
| 497 | unsigned long own_pages = 0, migrated_pages = 0; | 487 | unsigned long own_pages = 0, migrated_pages = 0; |
| @@ -339,7 +339,9 @@ static int gud_stats_debugfs(struct seq_file *m, void *data) · 338 unmodified lines | |||
| 339 | } | 339 | } |
| 340 | 340 | ||
| 341 | static const struct drm_crtc_helper_funcs gud_crtc_helper_funcs = { | 341 | static const struct drm_crtc_helper_funcs gud_crtc_helper_funcs = { |
| 342 | .atomic_check = drm_crtc_helper_atomic_check | 342 | .atomic_check = drm_crtc_helper_atomic_check, |
| 343 | .atomic_enable = gud_crtc_atomic_enable, | ||
| 344 | .atomic_disable = gud_crtc_atomic_disable, | ||
| 343 | }; | 345 | }; |
| 344 | 346 | ||
| 345 | static const struct drm_crtc_funcs gud_crtc_funcs = { | 347 | static const struct drm_crtc_funcs gud_crtc_funcs = { |
| @@ -364,6 +366,10 @@ static const struct drm_plane_funcs gud_plane_funcs = { · 18 unmodified lines | |||
| 364 | DRM_GEM_SHADOW_PLANE_FUNCS, | 366 | DRM_GEM_SHADOW_PLANE_FUNCS, |
| 365 | }; | 367 | }; |
| 366 | 368 | ||
| 369 | static const struct drm_mode_config_helper_funcs gud_mode_config_helpers = { | ||
| 370 | .atomic_commit_tail = drm_atomic_helper_commit_tail_rpm, | ||
| 371 | }; | ||
| 372 | |||
| 367 | static const struct drm_mode_config_funcs gud_mode_config_funcs = { | 373 | static const struct drm_mode_config_funcs gud_mode_config_funcs = { |
| 368 | .fb_create = drm_gem_fb_create_with_dirty, | 374 | .fb_create = drm_gem_fb_create_with_dirty, |
| 369 | .atomic_check = drm_atomic_helper_check, | 375 | .atomic_check = drm_atomic_helper_check, |
| @@ -499,6 +505,7 @@ static int gud_probe(struct usb_interface *intf, const struct usb_device_id *id) · 129 unmodified lines | |||
| 499 | drm->mode_config.min_height = le32_to_cpu(desc.min_height); | 505 | drm->mode_config.min_height = le32_to_cpu(desc.min_height); |
| 500 | drm->mode_config.max_height = le32_to_cpu(desc.max_height); | 506 | drm->mode_config.max_height = le32_to_cpu(desc.max_height); |
| 501 | drm->mode_config.funcs = &gud_mode_config_funcs; | 507 | drm->mode_config.funcs = &gud_mode_config_funcs; |
| 508 | drm->mode_config.helper_private = &gud_mode_config_helpers; | ||
| 502 | 509 | ||
| 503 | /* Format init */ | 510 | /* Format init */ |
| 504 | formats_dev = devm_kmalloc(dev, GUD_FORMATS_MAX_NUM, GFP_KERNEL); | 511 | formats_dev = devm_kmalloc(dev, GUD_FORMATS_MAX_NUM, GFP_KERNEL); |
| @@ -62,6 +62,10 @@ int gud_usb_set_u8(struct gud_device *gdrm, u8 request, u8 val); · 61 unmodified lines | |||
| 62 | 62 | ||
| 63 | void gud_clear_damage(struct gud_device *gdrm); | 63 | void gud_clear_damage(struct gud_device *gdrm); |
| 64 | void gud_flush_work(struct work_struct *work); | 64 | void gud_flush_work(struct work_struct *work); |
| 65 | void gud_crtc_atomic_enable(struct drm_crtc *crtc, | ||
| 66 | struct drm_atomic_state *state); | ||
| 67 | void gud_crtc_atomic_disable(struct drm_crtc *crtc, | ||
| 68 | struct drm_atomic_state *state); | ||
| 65 | int gud_plane_atomic_check(struct drm_plane *plane, | 69 | int gud_plane_atomic_check(struct drm_plane *plane, |
| 66 | struct drm_atomic_state *state); | 70 | struct drm_atomic_state *state); |
| 67 | void gud_plane_atomic_update(struct drm_plane *plane, | 71 | void gud_plane_atomic_update(struct drm_plane *plane, |
| @@ -580,6 +580,39 @@ out: · 579 unmodified lines | |||
| 580 | return ret; | 580 | return ret; |
| 581 | } | 581 | } |
| 582 | 582 | ||
| 583 | void gud_crtc_atomic_enable(struct drm_crtc *crtc, | ||
| 584 | struct drm_atomic_state *state) | ||
| 585 | { | ||
| 586 | struct drm_device *drm = crtc->dev; | ||
| 587 | struct gud_device *gdrm = to_gud_device(drm); | ||
| 588 | int idx; | ||
| 589 | |||
| 590 | if (!drm_dev_enter(drm, &idx)) | ||
| 591 | return; | ||
| 592 | |||
| 593 | gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 1); | ||
| 594 | gud_usb_set(gdrm, GUD_REQ_SET_STATE_COMMIT, 0, NULL, 0); | ||
| 595 | gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, 1); | ||
| 596 | |||
| 597 | drm_dev_exit(idx); | ||
| 598 | } | ||
| 599 | |||
| 600 | void gud_crtc_atomic_disable(struct drm_crtc *crtc, | ||
| 601 | struct drm_atomic_state *state) | ||
| 602 | { | ||
| 603 | struct drm_device *drm = crtc->dev; | ||
| 604 | struct gud_device *gdrm = to_gud_device(drm); | ||
| 605 | int idx; | ||
| 606 | |||
| 607 | if (!drm_dev_enter(drm, &idx)) | ||
| 608 | return; | ||
| 609 | |||
| 610 | gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, 0); | ||
| 611 | gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 0); | ||
| 612 | |||
| 613 | drm_dev_exit(idx); | ||
| 614 | } | ||
| 615 | |||
| 583 | void gud_plane_atomic_update(struct drm_plane *plane, | 616 | void gud_plane_atomic_update(struct drm_plane *plane, |
| 584 | struct drm_atomic_state *atomic_state) | 617 | struct drm_atomic_state *atomic_state) |
| 585 | { | 618 | { |
| @@ -607,24 +640,12 @@ void gud_plane_atomic_update(struct drm_plane *plane, · 21 unmodified lines | |||
| 607 | mutex_unlock(&gdrm->damage_lock); | 640 | mutex_unlock(&gdrm->damage_lock); |
| 608 | } | 641 | } |
| 609 | 642 | ||
| 610 | if (!drm_dev_enter(drm, &idx)) | 643 | if (!crtc || !drm_dev_enter(drm, &idx)) |
| 611 | return; | 644 | return; |
| 612 | 645 | ||
| 613 | if (!old_state->fb) | ||
| 614 | gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 1); | ||
| 615 | |||
| 616 | if (fb && (crtc->state->mode_changed || crtc->state->connectors_changed)) | ||
| 617 | gud_usb_set(gdrm, GUD_REQ_SET_STATE_COMMIT, 0, NULL, 0); | ||
| 618 | |||
| 619 | if (crtc->state->active_changed) | ||
| 620 | gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, crtc->state->active); | ||
| 621 | |||
| 622 | if (!fb) | ||
| 623 | goto ctrl_disable; | ||
| 624 | |||
| 625 | ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE); | 646 | ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE); |
| 626 | if (ret) | 647 | if (ret) |
| 627 | goto ctrl_disable; | 648 | goto out; |
| 628 | 649 | ||
| 629 | drm_atomic_helper_damage_iter_init(&iter, old_state, new_state); | 650 | drm_atomic_helper_damage_iter_init(&iter, old_state, new_state); |
| 630 | drm_atomic_for_each_plane_damage(&iter, &damage) | 651 | drm_atomic_for_each_plane_damage(&iter, &damage) |
| @@ -632,9 +653,6 @@ void gud_plane_atomic_update(struct drm_plane *plane, · 1 unmodified line | |||
| 632 | 653 | ||
| 633 | drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE); | 654 | drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE); |
| 634 | 655 | ||
| 635 | ctrl_disable: | 656 | out: |
| 636 | if (!crtc->state->enable) | ||
| 637 | gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 0); | ||
| 638 | |||
| 639 | drm_dev_exit(idx); | 657 | drm_dev_exit(idx); |
| 640 | } | 658 | } |
| @@ -43,12 +43,6 @@ bool intel_alpm_is_alpm_aux_less(struct intel_dp *intel_dp, · 42 unmodified lines | |||
| 43 | 43 | ||
| 44 | void intel_alpm_init(struct intel_dp *intel_dp) | 44 | void intel_alpm_init(struct intel_dp *intel_dp) |
| 45 | { | 45 | { |
| 46 | u8 dpcd; | ||
| 47 | |||
| 48 | if (drm_dp_dpcd_readb(&intel_dp->aux, DP_RECEIVER_ALPM_CAP, &dpcd) < 0) | ||
| 49 | return; | ||
| 50 | |||
| 51 | intel_dp->alpm_dpcd = dpcd; | ||
| 52 | mutex_init(&intel_dp->alpm.lock); | 46 | mutex_init(&intel_dp->alpm.lock); |
| 53 | } | 47 | } |
| 54 | 48 | ||
| @@ -562,12 +556,7 @@ void intel_alpm_disable(struct intel_dp *intel_dp) · 507 unmodified lines | |||
| 562 | mutex_lock(&intel_dp->alpm.lock); | 556 | mutex_lock(&intel_dp->alpm.lock); |
| 563 | 557 | ||
| 564 | intel_de_rmw(display, ALPM_CTL(display, cpu_transcoder), | 558 | intel_de_rmw(display, ALPM_CTL(display, cpu_transcoder), |
| 565 | ALPM_CTL_ALPM_ENABLE | ALPM_CTL_LOBF_ENABLE | | 559 | ALPM_CTL_ALPM_ENABLE | ALPM_CTL_LOBF_ENABLE, 0); |
| 566 | ALPM_CTL_ALPM_AUX_LESS_ENABLE, 0); | ||
| 567 | |||
| 568 | intel_de_rmw(display, | ||
| 569 | PORT_ALPM_CTL(cpu_transcoder), | ||
| 570 | PORT_ALPM_CTL_ALPM_AUX_LESS_ENABLE, 0); | ||
| 571 | 560 | ||
| 572 | drm_dbg_kms(display->drm, "Disabling ALPM\n"); | 561 | drm_dbg_kms(display->drm, "Disabling ALPM\n"); |
| 573 | mutex_unlock(&intel_dp->alpm.lock); | 562 | mutex_unlock(&intel_dp->alpm.lock); |
| @@ -1614,7 +1614,6 @@ static void hsw_configure_cpu_transcoder(const struct intel_crtc_state *crtc_sta · 1613 unmodified lines | |||
| 1614 | } | 1614 | } |
| 1615 | 1615 | ||
| 1616 | intel_set_transcoder_timings(crtc_state); | 1616 | intel_set_transcoder_timings(crtc_state); |
| 1617 | intel_vrr_set_transcoder_timings(crtc_state); | ||
| 1618 | 1617 | ||
| 1619 | if (cpu_transcoder != TRANSCODER_EDP) | 1618 | if (cpu_transcoder != TRANSCODER_EDP) |
| 1620 | intel_de_write(display, TRANS_MULT(display, cpu_transcoder), | 1619 | intel_de_write(display, TRANS_MULT(display, cpu_transcoder), |
| @@ -4577,6 +4577,7 @@ static bool · 4576 unmodified lines | |||
| 4577 | intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) | 4577 | intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) |
| 4578 | { | 4578 | { |
| 4579 | struct intel_display *display = to_intel_display(intel_dp); | 4579 | struct intel_display *display = to_intel_display(intel_dp); |
| 4580 | int ret; | ||
| 4580 | 4581 | ||
| 4581 | /* this function is meant to be called only once */ | 4582 | /* this function is meant to be called only once */ |
| 4582 | drm_WARN_ON(display->drm, intel_dp->dpcd[DP_DPCD_REV] != 0); | 4583 | drm_WARN_ON(display->drm, intel_dp->dpcd[DP_DPCD_REV] != 0); |
| @@ -4616,6 +4617,12 @@ intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector · 33 unmodified lines | |||
| 4616 | */ | 4617 | */ |
| 4617 | intel_dp_init_source_oui(intel_dp); | 4618 | intel_dp_init_source_oui(intel_dp); |
| 4618 | 4619 | ||
| 4620 | /* Read the ALPM DPCD caps */ | ||
| 4621 | ret = drm_dp_dpcd_read_byte(&intel_dp->aux, DP_RECEIVER_ALPM_CAP, | ||
| 4622 | &intel_dp->alpm_dpcd); | ||
| 4623 | if (ret < 0) | ||
| 4624 | return false; | ||
| 4625 | |||
| 4619 | /* | 4626 | /* |
| 4620 | * This has to be called after intel_dp->edp_dpcd is filled, PSR checks | 4627 | * This has to be called after intel_dp->edp_dpcd is filled, PSR checks |
| 4621 | * for SET_POWER_CAPABLE bit in intel_dp->edp_dpcd[1] | 4628 | * for SET_POWER_CAPABLE bit in intel_dp->edp_dpcd[1] |
| @@ -1307,9 +1307,14 @@ static bool psr2_granularity_check(struct intel_crtc_state *crtc_state, · 1306 unmodified lines | |||
| 1307 | u16 sink_y_granularity = crtc_state->has_panel_replay ? | 1307 | u16 sink_y_granularity = crtc_state->has_panel_replay ? |
| 1308 | connector->dp.panel_replay_caps.su_y_granularity : | 1308 | connector->dp.panel_replay_caps.su_y_granularity : |
| 1309 | connector->dp.psr_caps.su_y_granularity; | 1309 | connector->dp.psr_caps.su_y_granularity; |
| 1310 | u16 sink_w_granularity = crtc_state->has_panel_replay ? | 1310 | u16 sink_w_granularity; |
| 1311 | connector->dp.panel_replay_caps.su_w_granularity : | 1311 | |
| 1312 | connector->dp.psr_caps.su_w_granularity; | 1312 | if (crtc_state->has_panel_replay) |
| 1313 | sink_w_granularity = connector->dp.panel_replay_caps.su_w_granularity == | ||
| 1314 | DP_PANEL_REPLAY_FULL_LINE_GRANULARITY ? | ||
| 1315 | crtc_hdisplay : connector->dp.panel_replay_caps.su_w_granularity; | ||
| 1316 | else | ||
| 1317 | sink_w_granularity = connector->dp.psr_caps.su_w_granularity; | ||
| 1313 | 1318 | ||
| 1314 | /* PSR2 HW only send full lines so we only need to validate the width */ | 1319 | /* PSR2 HW only send full lines so we only need to validate the width */ |
| 1315 | if (crtc_hdisplay % sink_w_granularity) | 1320 | if (crtc_hdisplay % sink_w_granularity) |
| @@ -2614,6 +2619,12 @@ void intel_psr2_program_trans_man_trk_ctl(struct intel_dsb *dsb, · 1298 unmodified lines | |||
| 2614 | 2619 | ||
| 2615 | intel_de_write_dsb(display, dsb, PIPE_SRCSZ_ERLY_TPT(crtc->pipe), | 2620 | intel_de_write_dsb(display, dsb, PIPE_SRCSZ_ERLY_TPT(crtc->pipe), |
| 2616 | crtc_state->pipe_srcsz_early_tpt); | 2621 | crtc_state->pipe_srcsz_early_tpt); |
| 2622 | |||
| 2623 | if (!crtc_state->dsc.compression_enable) | ||
| 2624 | return; | ||
| 2625 | |||
| 2626 | intel_dsc_su_et_parameters_configure(dsb, encoder, crtc_state, | ||
| 2627 | drm_rect_height(&crtc_state->psr2_su_area)); | ||
| 2617 | } | 2628 | } |
| 2618 | 2629 | ||
| 2619 | static void psr2_man_trk_ctl_calc(struct intel_crtc_state *crtc_state, | 2630 | static void psr2_man_trk_ctl_calc(struct intel_crtc_state *crtc_state, |
| @@ -2684,11 +2695,12 @@ static void clip_area_update(struct drm_rect *overlap_damage_area, · 64 unmodified lines | |||
| 2684 | overlap_damage_area->y2 = damage_area->y2; | 2695 | overlap_damage_area->y2 = damage_area->y2; |
| 2685 | } | 2696 | } |
| 2686 | 2697 | ||
| 2687 | static void intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) | 2698 | static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) |
| 2688 | { | 2699 | { |
| 2689 | struct intel_display *display = to_intel_display(crtc_state); | 2700 | struct intel_display *display = to_intel_display(crtc_state); |
| 2690 | const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config; | 2701 | const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config; |
| 2691 | u16 y_alignment; | 2702 | u16 y_alignment; |
| 2703 | bool su_area_changed = false; | ||
| 2692 | 2704 | ||
| 2693 | /* ADLP aligns the SU region to vdsc slice height in case dsc is enabled */ | 2705 | /* ADLP aligns the SU region to vdsc slice height in case dsc is enabled */ |
| 2694 | if (crtc_state->dsc.compression_enable && | 2706 | if (crtc_state->dsc.compression_enable && |
| @@ -2697,10 +2709,18 @@ static void intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_st · 2 unmodified lines | |||
| 2697 | else | 2709 | else |
| 2698 | y_alignment = crtc_state->su_y_granularity; | 2710 | y_alignment = crtc_state->su_y_granularity; |
| 2699 | 2711 | ||
| 2700 | crtc_state->psr2_su_area.y1 -= crtc_state->psr2_su_area.y1 % y_alignment; | 2712 | if (crtc_state->psr2_su_area.y1 % y_alignment) { |
| 2701 | if (crtc_state->psr2_su_area.y2 % y_alignment) | 2713 | crtc_state->psr2_su_area.y1 -= crtc_state->psr2_su_area.y1 % y_alignment; |
| 2714 | su_area_changed = true; | ||
| 2715 | } | ||
| 2716 | |||
| 2717 | if (crtc_state->psr2_su_area.y2 % y_alignment) { | ||
| 2702 | crtc_state->psr2_su_area.y2 = ((crtc_state->psr2_su_area.y2 / | 2718 | crtc_state->psr2_su_area.y2 = ((crtc_state->psr2_su_area.y2 / |
| 2703 | y_alignment) + 1) * y_alignment; | 2719 | y_alignment) + 1) * y_alignment; |
| 2720 | su_area_changed = true; | ||
| 2721 | } | ||
| 2722 | |||
| 2723 | return su_area_changed; | ||
| 2704 | } | 2724 | } |
| 2705 | 2725 | ||
| 2706 | /* | 2726 | /* |
| @@ -2834,7 +2854,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 127 unmodified lines | |||
| 2834 | struct intel_crtc_state *crtc_state = intel_atomic_get_new_crtc_state(state, crtc); | 2854 | struct intel_crtc_state *crtc_state = intel_atomic_get_new_crtc_state(state, crtc); |
| 2835 | struct intel_plane_state *new_plane_state, *old_plane_state; | 2855 | struct intel_plane_state *new_plane_state, *old_plane_state; |
| 2836 | struct intel_plane *plane; | 2856 | struct intel_plane *plane; |
| 2837 | bool full_update = false, cursor_in_su_area = false; | 2857 | bool full_update = false, su_area_changed; |
| 2838 | int i, ret; | 2858 | int i, ret; |
| 2839 | 2859 | ||
| 2840 | if (!crtc_state->enable_psr2_sel_fetch) | 2860 | if (!crtc_state->enable_psr2_sel_fetch) |
| @@ -2941,15 +2961,32 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 100 unmodified lines | |||
| 2941 | if (ret) | 2961 | if (ret) |
| 2942 | return ret; | 2962 | return ret; |
| 2943 | 2963 | ||
| 2944 | /* | 2964 | do { |
| 2945 | * Adjust su area to cover cursor fully as necessary (early | 2965 | bool cursor_in_su_area; |
| 2946 | * transport). This needs to be done after | 2966 | |
| 2947 | * drm_atomic_add_affected_planes to ensure visible cursor is added into | 2967 | /* |
| 2948 | * affected planes even when cursor is not updated by itself. | 2968 | * Adjust su area to cover cursor fully as necessary |
| 2949 | */ | 2969 | * (early transport). This needs to be done after |
| 2950 | intel_psr2_sel_fetch_et_alignment(state, crtc, &cursor_in_su_area); | 2970 | * drm_atomic_add_affected_planes to ensure visible |
| 2971 | * cursor is added into affected planes even when | ||
| 2972 | * cursor is not updated by itself. | ||
| 2973 | */ | ||
| 2974 | intel_psr2_sel_fetch_et_alignment(state, crtc, &cursor_in_su_area); | ||
| 2951 | 2975 | ||
| 2952 | intel_psr2_sel_fetch_pipe_alignment(crtc_state); | 2976 | su_area_changed = intel_psr2_sel_fetch_pipe_alignment(crtc_state); |
| 2977 | |||
| 2978 | /* | ||
| 2979 | * If the cursor was outside the SU area before | ||
| 2980 | * alignment, the alignment step (which only expands | ||
| 2981 | * SU) may pull the cursor partially inside, so we | ||
| 2982 | * must run ET alignment again to fully cover it. But | ||
| 2983 | * if the cursor was already fully inside before | ||
| 2984 | * alignment, expanding the SU area won't change that, | ||
| 2985 | * so no further work is needed. | ||
| 2986 | */ | ||
| 2987 | if (cursor_in_su_area) | ||
| 2988 | break; | ||
| 2989 | } while (su_area_changed); | ||
| 2953 | 2990 | ||
| 2954 | /* | 2991 | /* |
| 2955 | * Now that we have the pipe damaged area check if it intersect with | 2992 | * Now that we have the pipe damaged area check if it intersect with |
| @@ -3009,6 +3046,10 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 53 unmodified lines | |||
| 3009 | } | 3046 | } |
| 3010 | 3047 | ||
| 3011 | skip_sel_fetch_set_loop: | 3048 | skip_sel_fetch_set_loop: |
| 3049 | if (full_update) | ||
| 3050 | clip_area_update(&crtc_state->psr2_su_area, &crtc_state->pipe_src, | ||
| 3051 | &crtc_state->pipe_src); | ||
| 3052 | |||
| 3012 | psr2_man_trk_ctl_calc(crtc_state, full_update); | 3053 | psr2_man_trk_ctl_calc(crtc_state, full_update); |
| 3013 | crtc_state->pipe_srcsz_early_tpt = | 3054 | crtc_state->pipe_srcsz_early_tpt = |
| 3014 | psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update); | 3055 | psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update); |
| @@ -767,6 +767,29 @@ void intel_dsc_dp_pps_write(struct intel_encoder *encoder, · 766 unmodified lines | |||
| 767 | sizeof(dp_dsc_pps_sdp)); | 767 | sizeof(dp_dsc_pps_sdp)); |
| 768 | } | 768 | } |
| 769 | 769 | ||
| 770 | void intel_dsc_su_et_parameters_configure(struct intel_dsb *dsb, struct intel_encoder *encoder, | ||
| 771 | const struct intel_crtc_state *crtc_state, int su_lines) | ||
| 772 | { | ||
| 773 | struct intel_display *display = to_intel_display(crtc_state); | ||
| 774 | struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); | ||
| 775 | const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config; | ||
| 776 | enum pipe pipe = crtc->pipe; | ||
| 777 | int vdsc_instances_per_pipe = intel_dsc_get_vdsc_per_pipe(crtc_state); | ||
| 778 | int slice_row_per_frame = su_lines / vdsc_cfg->slice_height; | ||
| 779 | u32 val; | ||
| 780 | |||
| 781 | drm_WARN_ON_ONCE(display->drm, su_lines % vdsc_cfg->slice_height); | ||
| 782 | drm_WARN_ON_ONCE(display->drm, vdsc_instances_per_pipe > 2); | ||
| 783 | |||
| 784 | val = DSC_SUPS0_SU_SLICE_ROW_PER_FRAME(slice_row_per_frame); | ||
| 785 | val |= DSC_SUPS0_SU_PIC_HEIGHT(su_lines); | ||
| 786 | |||
| 787 | intel_de_write_dsb(display, dsb, LNL_DSC0_SU_PARAMETER_SET_0(pipe), val); | ||
| 788 | |||
| 789 | if (vdsc_instances_per_pipe == 2) | ||
| 790 | intel_de_write_dsb(display, dsb, LNL_DSC1_SU_PARAMETER_SET_0(pipe), val); | ||
| 791 | } | ||
| 792 | |||
| 770 | static i915_reg_t dss_ctl1_reg(struct intel_crtc *crtc, enum transcoder cpu_transcoder) | 793 | static i915_reg_t dss_ctl1_reg(struct intel_crtc *crtc, enum transcoder cpu_transcoder) |
| 771 | { | 794 | { |
| 772 | return is_pipe_dsc(crtc, cpu_transcoder) ? | 795 | return is_pipe_dsc(crtc, cpu_transcoder) ? |
| @@ -13,6 +13,7 @@ struct drm_printer; · 12 unmodified lines | |||
| 13 | enum transcoder; | 13 | enum transcoder; |
| 14 | struct intel_crtc; | 14 | struct intel_crtc; |
| 15 | struct intel_crtc_state; | 15 | struct intel_crtc_state; |
| 16 | struct intel_dsb; | ||
| 16 | struct intel_encoder; | 17 | struct intel_encoder; |
| 17 | 18 | ||
| 18 | bool intel_dsc_source_support(const struct intel_crtc_state *crtc_state); | 19 | bool intel_dsc_source_support(const struct intel_crtc_state *crtc_state); |
| @@ -31,6 +32,8 @@ void intel_dsc_dsi_pps_write(struct intel_encoder *encoder, · 12 unmodified lines | |||
| 31 | const struct intel_crtc_state *crtc_state); | 32 | const struct intel_crtc_state *crtc_state); |
| 32 | void intel_dsc_dp_pps_write(struct intel_encoder *encoder, | 33 | void intel_dsc_dp_pps_write(struct intel_encoder *encoder, |
| 33 | const struct intel_crtc_state *crtc_state); | 34 | const struct intel_crtc_state *crtc_state); |
| 35 | void intel_dsc_su_et_parameters_configure(struct intel_dsb *dsb, struct intel_encoder *encoder, | ||
| 36 | const struct intel_crtc_state *crtc_state, int su_lines); | ||
| 34 | void intel_vdsc_state_dump(struct drm_printer *p, int indent, | 37 | void intel_vdsc_state_dump(struct drm_printer *p, int indent, |
| 35 | const struct intel_crtc_state *crtc_state); | 38 | const struct intel_crtc_state *crtc_state); |
| 36 | int intel_vdsc_min_cdclk(const struct intel_crtc_state *crtc_state); | 39 | int intel_vdsc_min_cdclk(const struct intel_crtc_state *crtc_state); |
| @@ -196,6 +196,18 @@ · 195 unmodified lines | |||
| 196 | #define DSC_PPS18_NSL_BPG_OFFSET(offset) REG_FIELD_PREP(DSC_PPS18_NSL_BPG_OFFSET_MASK, offset) | 196 | #define DSC_PPS18_NSL_BPG_OFFSET(offset) REG_FIELD_PREP(DSC_PPS18_NSL_BPG_OFFSET_MASK, offset) |
| 197 | #define DSC_PPS18_SL_OFFSET_ADJ(offset) REG_FIELD_PREP(DSC_PPS18_SL_OFFSET_ADJ_MASK, offset) | 197 | #define DSC_PPS18_SL_OFFSET_ADJ(offset) REG_FIELD_PREP(DSC_PPS18_SL_OFFSET_ADJ_MASK, offset) |
| 198 | 198 | ||
| 199 | #define _LNL_DSC0_SU_PARAMETER_SET_0_PA 0x78064 | ||
| 200 | #define _LNL_DSC1_SU_PARAMETER_SET_0_PA 0x78164 | ||
| 201 | #define _LNL_DSC0_SU_PARAMETER_SET_0_PB 0x78264 | ||
| 202 | #define _LNL_DSC1_SU_PARAMETER_SET_0_PB 0x78364 | ||
| 203 | #define LNL_DSC0_SU_PARAMETER_SET_0(pipe) _MMIO_PIPE((pipe), _LNL_DSC0_SU_PARAMETER_SET_0_PA, _LNL_DSC0_SU_PARAMETER_SET_0_PB) | ||
| 204 | #define LNL_DSC1_SU_PARAMETER_SET_0(pipe) _MMIO_PIPE((pipe), _LNL_DSC1_SU_PARAMETER_SET_0_PA, _LNL_DSC1_SU_PARAMETER_SET_0_PB) | ||
| 205 | |||
| 206 | #define DSC_SUPS0_SU_SLICE_ROW_PER_FRAME_MASK REG_GENMASK(31, 20) | ||
| 207 | #define DSC_SUPS0_SU_SLICE_ROW_PER_FRAME(rows) REG_FIELD_PREP(DSC_SUPS0_SU_SLICE_ROW_PER_FRAME_MASK, (rows)) | ||
| 208 | #define DSC_SUPS0_SU_PIC_HEIGHT_MASK REG_GENMASK(15, 0) | ||
| 209 | #define DSC_SUPS0_SU_PIC_HEIGHT(h) REG_FIELD_PREP(DSC_SUPS0_SU_PIC_HEIGHT_MASK, (h)) | ||
| 210 | |||
| 199 | /* Icelake Rate Control Buffer Threshold Registers */ | 211 | /* Icelake Rate Control Buffer Threshold Registers */ |
| 200 | #define DSCA_RC_BUF_THRESH_0 _MMIO(0x6B230) | 212 | #define DSCA_RC_BUF_THRESH_0 _MMIO(0x6B230) |
| 201 | #define DSCA_RC_BUF_THRESH_0_UDW _MMIO(0x6B230 + 4) | 213 | #define DSCA_RC_BUF_THRESH_0_UDW _MMIO(0x6B230 + 4) |
| @@ -597,6 +597,18 @@ void intel_vrr_set_transcoder_timings(const struct intel_crtc_state *crtc_state) · 596 unmodified lines | |||
| 597 | if (!HAS_VRR(display)) | 597 | if (!HAS_VRR(display)) |
| 598 | return; | 598 | return; |
| 599 | 599 | ||
| 600 | /* | ||
| 601 | * Bspec says: | ||
| 602 | * "(note: VRR needs to be programmed after | ||
| 603 | * TRANS_DDI_FUNC_CTL and before TRANS_CONF)." | ||
| 604 | * | ||
| 605 | * In practice it turns out that ICL can hang if | ||
| 606 | * TRANS_VRR_VMAX/FLIPLINE are written before | ||
| 607 | * enabling TRANS_DDI_FUNC_CTL. | ||
| 608 | */ | ||
| 609 | drm_WARN_ON(display->drm, | ||
| 610 | !(intel_de_read(display, TRANS_DDI_FUNC_CTL(display, cpu_transcoder)) & TRANS_DDI_FUNC_ENABLE)); | ||
| 611 | |||
| 600 | /* | 612 | /* |
| 601 | * This bit seems to have two meanings depending on the platform: | 613 | * This bit seems to have two meanings depending on the platform: |
| 602 | * TGL: generate VRR "safe window" for DSB vblank waits | 614 | * TGL: generate VRR "safe window" for DSB vblank waits |
| @@ -939,6 +951,8 @@ void intel_vrr_transcoder_enable(const struct intel_crtc_state *crtc_state) · 336 unmodified lines | |||
| 939 | { | 951 | { |
| 940 | struct intel_display *display = to_intel_display(crtc_state); | 952 | struct intel_display *display = to_intel_display(crtc_state); |
| 941 | 953 | ||
| 954 | intel_vrr_set_transcoder_timings(crtc_state); | ||
| 955 | |||
| 942 | if (!intel_vrr_possible(crtc_state)) | 956 | if (!intel_vrr_possible(crtc_state)) |
| 943 | return; | 957 | return; |
| 944 | 958 | ||
| @@ -153,8 +153,12 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, · 152 unmodified lines | |||
| 153 | } | 153 | } |
| 154 | } while (1); | 154 | } while (1); |
| 155 | 155 | ||
| 156 | nr_pages = min_t(unsigned long, | 156 | nr_pages = min_array(((unsigned long[]) { |
| 157 | folio_nr_pages(folio), page_count - i); | 157 | folio_nr_pages(folio), |
| 158 | page_count - i, | ||
| 159 | max_segment / PAGE_SIZE, | ||
| 160 | }), 3); | ||
| 161 | |||
| 158 | if (!i || | 162 | if (!i || |
| 159 | sg->length >= max_segment || | 163 | sg->length >= max_segment || |
| 160 | folio_pfn(folio) != next_pfn) { | 164 | folio_pfn(folio) != next_pfn) { |
| @@ -164,7 +168,9 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, · 3 unmodified lines | |||
| 164 | st->nents++; | 168 | st->nents++; |
| 165 | sg_set_folio(sg, folio, nr_pages * PAGE_SIZE, 0); | 169 | sg_set_folio(sg, folio, nr_pages * PAGE_SIZE, 0); |
| 166 | } else { | 170 | } else { |
| 167 | /* XXX: could overflow? */ | 171 | nr_pages = min_t(unsigned long, nr_pages, |
| 172 | (max_segment - sg->length) / PAGE_SIZE); | ||
| 173 | |||
| 168 | sg->length += nr_pages * PAGE_SIZE; | 174 | sg->length += nr_pages * PAGE_SIZE; |
| 169 | } | 175 | } |
| 170 | next_pfn = folio_pfn(folio) + nr_pages; | 176 | next_pfn = folio_pfn(folio) + nr_pages; |
| @@ -256,7 +256,9 @@ static int imx_pd_probe(struct platform_device *pdev) · 255 unmodified lines | |||
| 256 | 256 | ||
| 257 | platform_set_drvdata(pdev, imxpd); | 257 | platform_set_drvdata(pdev, imxpd); |
| 258 | 258 | ||
| 259 | devm_drm_bridge_add(dev, &imxpd->bridge); | 259 | ret = devm_drm_bridge_add(dev, &imxpd->bridge); |
| 260 | if (ret) | ||
| 261 | return ret; | ||
| 260 | 262 | ||
| 261 | return component_add(dev, &imx_pd_ops); | 263 | return component_add(dev, &imx_pd_ops); |
| 262 | } | 264 | } |
| @@ -92,7 +92,6 @@ static int logicvc_drm_config_parse(struct logicvc_drm *logicvc) · 91 unmodified lines | |||
| 92 | struct device *dev = drm_dev->dev; | 92 | struct device *dev = drm_dev->dev; |
| 93 | struct device_node *of_node = dev->of_node; | 93 | struct device_node *of_node = dev->of_node; |
| 94 | struct logicvc_drm_config *config = &logicvc->config; | 94 | struct logicvc_drm_config *config = &logicvc->config; |
| 95 | struct device_node *layers_node; | ||
| 96 | int ret; | 95 | int ret; |
| 97 | 96 | ||
| 98 | logicvc_of_property_parse_bool(of_node, LOGICVC_OF_PROPERTY_DITHERING, | 97 | logicvc_of_property_parse_bool(of_node, LOGICVC_OF_PROPERTY_DITHERING, |
| @@ -128,7 +127,8 @@ static int logicvc_drm_config_parse(struct logicvc_drm *logicvc) · 29 unmodified lines | |||
| 128 | if (ret) | 127 | if (ret) |
| 129 | return ret; | 128 | return ret; |
| 130 | 129 | ||
| 131 | layers_node = of_get_child_by_name(of_node, "layers"); | 130 | struct device_node *layers_node __free(device_node) = |
| 131 | of_get_child_by_name(of_node, "layers"); | ||
| 132 | if (!layers_node) { | 132 | if (!layers_node) { |
| 133 | drm_err(drm_dev, "Missing non-optional layers node\n"); | 133 | drm_err(drm_dev, "Missing non-optional layers node\n"); |
| 134 | return -EINVAL; | 134 | return -EINVAL; |
| @@ -78,7 +78,7 @@ static void a2xx_gpummu_destroy(struct msm_mmu *mmu) · 77 unmodified lines | |||
| 78 | { | 78 | { |
| 79 | struct a2xx_gpummu *gpummu = to_a2xx_gpummu(mmu); | 79 | struct a2xx_gpummu *gpummu = to_a2xx_gpummu(mmu); |
| 80 | 80 | ||
| 81 | dma_free_attrs(mmu->dev, TABLE_SIZE, gpummu->table, gpummu->pt_base, | 81 | dma_free_attrs(mmu->dev, TABLE_SIZE + 32, gpummu->table, gpummu->pt_base, |
| 82 | DMA_ATTR_FORCE_CONTIGUOUS); | 82 | DMA_ATTR_FORCE_CONTIGUOUS); |
| 83 | 83 | ||
| 84 | kfree(gpummu); | 84 | kfree(gpummu); |
| @@ -1759,7 +1759,7 @@ static const u32 x285_protect_regs[] = { · 1758 unmodified lines | |||
| 1759 | A6XX_PROTECT_NORDWR(0x27c06, 0x0000), | 1759 | A6XX_PROTECT_NORDWR(0x27c06, 0x0000), |
| 1760 | }; | 1760 | }; |
| 1761 | 1761 | ||
| 1762 | DECLARE_ADRENO_PROTECT(x285_protect, 64); | 1762 | DECLARE_ADRENO_PROTECT(x285_protect, 15); |
| 1763 | 1763 | ||
| 1764 | static const struct adreno_reglist_pipe a840_nonctxt_regs[] = { | 1764 | static const struct adreno_reglist_pipe a840_nonctxt_regs[] = { |
| 1765 | { REG_A8XX_CP_SMMU_STREAM_ID_LPAC, 0x00000101, BIT(PIPE_NONE) }, | 1765 | { REG_A8XX_CP_SMMU_STREAM_ID_LPAC, 0x00000101, BIT(PIPE_NONE) }, |
| @@ -1966,5 +1966,4 @@ static inline __always_unused void __build_asserts(void) · 200 unmodified lines | |||
| 1966 | BUILD_BUG_ON(a660_protect.count > a660_protect.count_max); | 1966 | BUILD_BUG_ON(a660_protect.count > a660_protect.count_max); |
| 1967 | BUILD_BUG_ON(a690_protect.count > a690_protect.count_max); | 1967 | BUILD_BUG_ON(a690_protect.count > a690_protect.count_max); |
| 1968 | BUILD_BUG_ON(a730_protect.count > a730_protect.count_max); | 1968 | BUILD_BUG_ON(a730_protect.count > a730_protect.count_max); |
| 1969 | BUILD_BUG_ON(a840_protect.count > a840_protect.count_max); | ||
| 1970 | } | 1969 | } |
| @@ -310,11 +310,21 @@ static void a8xx_set_ubwc_config(struct msm_gpu *gpu) · 309 unmodified lines | |||
| 310 | hbb = cfg->highest_bank_bit - 13; | 310 | hbb = cfg->highest_bank_bit - 13; |
| 311 | hbb_hi = hbb >> 2; | 311 | hbb_hi = hbb >> 2; |
| 312 | hbb_lo = hbb & 3; | 312 | hbb_lo = hbb & 3; |
| 313 | a8xx_write_pipe(gpu, PIPE_BV, REG_A8XX_GRAS_NC_MODE_CNTL, hbb << 5); | 313 | |
| 314 | a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_GRAS_NC_MODE_CNTL, hbb << 5); | 314 | a8xx_write_pipe(gpu, PIPE_BV, REG_A8XX_GRAS_NC_MODE_CNTL, |
| 315 | hbb << 5 | | ||
| 316 | level3_swizzling_dis << 4 | | ||
| 317 | level2_swizzling_dis << 3); | ||
| 318 | |||
| 319 | a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_GRAS_NC_MODE_CNTL, | ||
| 320 | hbb << 5 | | ||
| 321 | level3_swizzling_dis << 4 | | ||
| 322 | level2_swizzling_dis << 3); | ||
| 315 | 323 | ||
| 316 | a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_RB_CCU_NC_MODE_CNTL, | 324 | a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_RB_CCU_NC_MODE_CNTL, |
| 317 | yuvnotcomptofc << 6 | | 325 | yuvnotcomptofc << 6 | |
| 326 | level3_swizzling_dis << 5 | | ||
| 327 | level2_swizzling_dis << 4 | | ||
| 318 | hbb_hi << 3 | | 328 | hbb_hi << 3 | |
| 319 | hbb_lo << 1); | 329 | hbb_lo << 1); |
| 320 | 330 | ||
| @@ -302,6 +302,7 @@ static const struct of_device_id dt_match[] = { · 301 unmodified lines | |||
| 302 | { .compatible = "qcom,kgsl-3d0" }, | 302 | { .compatible = "qcom,kgsl-3d0" }, |
| 303 | {} | 303 | {} |
| 304 | }; | 304 | }; |
| 305 | MODULE_DEVICE_TABLE(of, dt_match); | ||
| 305 | 306 | ||
| 306 | static int adreno_runtime_resume(struct device *dev) | 307 | static int adreno_runtime_resume(struct device *dev) |
| 307 | { | 308 | { |
| @@ -133,7 +133,7 @@ static const struct dpu_sspp_cfg sc8280xp_sspp[] = { · 132 unmodified lines | |||
| 133 | static const struct dpu_lm_cfg sc8280xp_lm[] = { | 133 | static const struct dpu_lm_cfg sc8280xp_lm[] = { |
| 134 | { | 134 | { |
| 135 | .name = "lm_0", .id = LM_0, | 135 | .name = "lm_0", .id = LM_0, |
| 136 | .base = 0x44000, .len = 0x320, | 136 | .base = 0x44000, .len = 0x400, |
| 137 | .features = MIXER_MSM8998_MASK, | 137 | .features = MIXER_MSM8998_MASK, |
| 138 | .sblk = &sdm845_lm_sblk, | 138 | .sblk = &sdm845_lm_sblk, |
| 139 | .lm_pair = LM_1, | 139 | .lm_pair = LM_1, |
| @@ -141,7 +141,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | |||
| 141 | .dspp = DSPP_0, | 141 | .dspp = DSPP_0, |
| 142 | }, { | 142 | }, { |
| 143 | .name = "lm_1", .id = LM_1, | 143 | .name = "lm_1", .id = LM_1, |
| 144 | .base = 0x45000, .len = 0x320, | 144 | .base = 0x45000, .len = 0x400, |
| 145 | .features = MIXER_MSM8998_MASK, | 145 | .features = MIXER_MSM8998_MASK, |
| 146 | .sblk = &sdm845_lm_sblk, | 146 | .sblk = &sdm845_lm_sblk, |
| 147 | .lm_pair = LM_0, | 147 | .lm_pair = LM_0, |
| @@ -149,7 +149,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | |||
| 149 | .dspp = DSPP_1, | 149 | .dspp = DSPP_1, |
| 150 | }, { | 150 | }, { |
| 151 | .name = "lm_2", .id = LM_2, | 151 | .name = "lm_2", .id = LM_2, |
| 152 | .base = 0x46000, .len = 0x320, | 152 | .base = 0x46000, .len = 0x400, |
| 153 | .features = MIXER_MSM8998_MASK, | 153 | .features = MIXER_MSM8998_MASK, |
| 154 | .sblk = &sdm845_lm_sblk, | 154 | .sblk = &sdm845_lm_sblk, |
| 155 | .lm_pair = LM_3, | 155 | .lm_pair = LM_3, |
| @@ -157,7 +157,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | |||
| 157 | .dspp = DSPP_2, | 157 | .dspp = DSPP_2, |
| 158 | }, { | 158 | }, { |
| 159 | .name = "lm_3", .id = LM_3, | 159 | .name = "lm_3", .id = LM_3, |
| 160 | .base = 0x47000, .len = 0x320, | 160 | .base = 0x47000, .len = 0x400, |
| 161 | .features = MIXER_MSM8998_MASK, | 161 | .features = MIXER_MSM8998_MASK, |
| 162 | .sblk = &sdm845_lm_sblk, | 162 | .sblk = &sdm845_lm_sblk, |
| 163 | .lm_pair = LM_2, | 163 | .lm_pair = LM_2, |
| @@ -165,14 +165,14 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | |||
| 165 | .dspp = DSPP_3, | 165 | .dspp = DSPP_3, |
| 166 | }, { | 166 | }, { |
| 167 | .name = "lm_4", .id = LM_4, | 167 | .name = "lm_4", .id = LM_4, |
| 168 | .base = 0x48000, .len = 0x320, | 168 | .base = 0x48000, .len = 0x400, |
| 169 | .features = MIXER_MSM8998_MASK, | 169 | .features = MIXER_MSM8998_MASK, |
| 170 | .sblk = &sdm845_lm_sblk, | 170 | .sblk = &sdm845_lm_sblk, |
| 171 | .lm_pair = LM_5, | 171 | .lm_pair = LM_5, |
| 172 | .pingpong = PINGPONG_4, | 172 | .pingpong = PINGPONG_4, |
| 173 | }, { | 173 | }, { |
| 174 | .name = "lm_5", .id = LM_5, | 174 | .name = "lm_5", .id = LM_5, |
| 175 | .base = 0x49000, .len = 0x320, | 175 | .base = 0x49000, .len = 0x400, |
| 176 | .features = MIXER_MSM8998_MASK, | 176 | .features = MIXER_MSM8998_MASK, |
| 177 | .sblk = &sdm845_lm_sblk, | 177 | .sblk = &sdm845_lm_sblk, |
| 178 | .lm_pair = LM_4, | 178 | .lm_pair = LM_4, |
| @@ -134,7 +134,7 @@ static const struct dpu_sspp_cfg sm8450_sspp[] = { · 133 unmodified lines | |||
| 134 | static const struct dpu_lm_cfg sm8450_lm[] = { | 134 | static const struct dpu_lm_cfg sm8450_lm[] = { |
| 135 | { | 135 | { |
| 136 | .name = "lm_0", .id = LM_0, | 136 | .name = "lm_0", .id = LM_0, |
| 137 | .base = 0x44000, .len = 0x320, | 137 | .base = 0x44000, .len = 0x400, |
| 138 | .features = MIXER_MSM8998_MASK, | 138 | .features = MIXER_MSM8998_MASK, |
| 139 | .sblk = &sdm845_lm_sblk, | 139 | .sblk = &sdm845_lm_sblk, |
| 140 | .lm_pair = LM_1, | 140 | .lm_pair = LM_1, |
| @@ -142,7 +142,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | |||
| 142 | .dspp = DSPP_0, | 142 | .dspp = DSPP_0, |
| 143 | }, { | 143 | }, { |
| 144 | .name = "lm_1", .id = LM_1, | 144 | .name = "lm_1", .id = LM_1, |
| 145 | .base = 0x45000, .len = 0x320, | 145 | .base = 0x45000, .len = 0x400, |
| 146 | .features = MIXER_MSM8998_MASK, | 146 | .features = MIXER_MSM8998_MASK, |
| 147 | .sblk = &sdm845_lm_sblk, | 147 | .sblk = &sdm845_lm_sblk, |
| 148 | .lm_pair = LM_0, | 148 | .lm_pair = LM_0, |
| @@ -150,7 +150,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | |||
| 150 | .dspp = DSPP_1, | 150 | .dspp = DSPP_1, |
| 151 | }, { | 151 | }, { |
| 152 | .name = "lm_2", .id = LM_2, | 152 | .name = "lm_2", .id = LM_2, |
| 153 | .base = 0x46000, .len = 0x320, | 153 | .base = 0x46000, .len = 0x400, |
| 154 | .features = MIXER_MSM8998_MASK, | 154 | .features = MIXER_MSM8998_MASK, |
| 155 | .sblk = &sdm845_lm_sblk, | 155 | .sblk = &sdm845_lm_sblk, |
| 156 | .lm_pair = LM_3, | 156 | .lm_pair = LM_3, |
| @@ -158,7 +158,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | |||
| 158 | .dspp = DSPP_2, | 158 | .dspp = DSPP_2, |
| 159 | }, { | 159 | }, { |
| 160 | .name = "lm_3", .id = LM_3, | 160 | .name = "lm_3", .id = LM_3, |
| 161 | .base = 0x47000, .len = 0x320, | 161 | .base = 0x47000, .len = 0x400, |
| 162 | .features = MIXER_MSM8998_MASK, | 162 | .features = MIXER_MSM8998_MASK, |
| 163 | .sblk = &sdm845_lm_sblk, | 163 | .sblk = &sdm845_lm_sblk, |
| 164 | .lm_pair = LM_2, | 164 | .lm_pair = LM_2, |
| @@ -166,14 +166,14 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | |||
| 166 | .dspp = DSPP_3, | 166 | .dspp = DSPP_3, |
| 167 | }, { | 167 | }, { |
| 168 | .name = "lm_4", .id = LM_4, | 168 | .name = "lm_4", .id = LM_4, |
| 169 | .base = 0x48000, .len = 0x320, | 169 | .base = 0x48000, .len = 0x400, |
| 170 | .features = MIXER_MSM8998_MASK, | 170 | .features = MIXER_MSM8998_MASK, |
| 171 | .sblk = &sdm845_lm_sblk, | 171 | .sblk = &sdm845_lm_sblk, |
| 172 | .lm_pair = LM_5, | 172 | .lm_pair = LM_5, |
| 173 | .pingpong = PINGPONG_4, | 173 | .pingpong = PINGPONG_4, |
| 174 | }, { | 174 | }, { |
| 175 | .name = "lm_5", .id = LM_5, | 175 | .name = "lm_5", .id = LM_5, |
| 176 | .base = 0x49000, .len = 0x320, | 176 | .base = 0x49000, .len = 0x400, |
| 177 | .features = MIXER_MSM8998_MASK, | 177 | .features = MIXER_MSM8998_MASK, |
| 178 | .sblk = &sdm845_lm_sblk, | 178 | .sblk = &sdm845_lm_sblk, |
| 179 | .lm_pair = LM_4, | 179 | .lm_pair = LM_4, |
| @@ -366,8 +366,8 @@ static const struct dpu_intf_cfg sa8775p_intf[] = { · 365 unmodified lines | |||
| 366 | .type = INTF_NONE, | 366 | .type = INTF_NONE, |
| 367 | .controller_id = MSM_DP_CONTROLLER_0, /* pair with intf_0 for DP MST */ | 367 | .controller_id = MSM_DP_CONTROLLER_0, /* pair with intf_0 for DP MST */ |
| 368 | .prog_fetch_lines_worst_case = 24, | 368 | .prog_fetch_lines_worst_case = 24, |
| 369 | .intr_underrun = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 17), | 369 | .intr_underrun = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 16), |
| 370 | .intr_vsync = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 16), | 370 | .intr_vsync = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 17), |
| 371 | }, { | 371 | }, { |
| 372 | .name = "intf_7", .id = INTF_7, | 372 | .name = "intf_7", .id = INTF_7, |
| 373 | .base = 0x3b000, .len = 0x280, | 373 | .base = 0x3b000, .len = 0x280, |
| @@ -131,7 +131,7 @@ static const struct dpu_sspp_cfg sm8550_sspp[] = { · 130 unmodified lines | |||
| 131 | static const struct dpu_lm_cfg sm8550_lm[] = { | 131 | static const struct dpu_lm_cfg sm8550_lm[] = { |
| 132 | { | 132 | { |
| 133 | .name = "lm_0", .id = LM_0, | 133 | .name = "lm_0", .id = LM_0, |
| 134 | .base = 0x44000, .len = 0x320, | 134 | .base = 0x44000, .len = 0x400, |
| 135 | .features = MIXER_MSM8998_MASK, | 135 | .features = MIXER_MSM8998_MASK, |
| 136 | .sblk = &sdm845_lm_sblk, | 136 | .sblk = &sdm845_lm_sblk, |
| 137 | .lm_pair = LM_1, | 137 | .lm_pair = LM_1, |
| @@ -139,7 +139,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | |||
| 139 | .dspp = DSPP_0, | 139 | .dspp = DSPP_0, |
| 140 | }, { | 140 | }, { |
| 141 | .name = "lm_1", .id = LM_1, | 141 | .name = "lm_1", .id = LM_1, |
| 142 | .base = 0x45000, .len = 0x320, | 142 | .base = 0x45000, .len = 0x400, |
| 143 | .features = MIXER_MSM8998_MASK, | 143 | .features = MIXER_MSM8998_MASK, |
| 144 | .sblk = &sdm845_lm_sblk, | 144 | .sblk = &sdm845_lm_sblk, |
| 145 | .lm_pair = LM_0, | 145 | .lm_pair = LM_0, |
| @@ -147,7 +147,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | |||
| 147 | .dspp = DSPP_1, | 147 | .dspp = DSPP_1, |
| 148 | }, { | 148 | }, { |
| 149 | .name = "lm_2", .id = LM_2, | 149 | .name = "lm_2", .id = LM_2, |
| 150 | .base = 0x46000, .len = 0x320, | 150 | .base = 0x46000, .len = 0x400, |
| 151 | .features = MIXER_MSM8998_MASK, | 151 | .features = MIXER_MSM8998_MASK, |
| 152 | .sblk = &sdm845_lm_sblk, | 152 | .sblk = &sdm845_lm_sblk, |
| 153 | .lm_pair = LM_3, | 153 | .lm_pair = LM_3, |
| @@ -155,7 +155,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | |||
| 155 | .dspp = DSPP_2, | 155 | .dspp = DSPP_2, |
| 156 | }, { | 156 | }, { |
| 157 | .name = "lm_3", .id = LM_3, | 157 | .name = "lm_3", .id = LM_3, |
| 158 | .base = 0x47000, .len = 0x320, | 158 | .base = 0x47000, .len = 0x400, |
| 159 | .features = MIXER_MSM8998_MASK, | 159 | .features = MIXER_MSM8998_MASK, |
| 160 | .sblk = &sdm845_lm_sblk, | 160 | .sblk = &sdm845_lm_sblk, |
| 161 | .lm_pair = LM_2, | 161 | .lm_pair = LM_2, |
| @@ -163,14 +163,14 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | |||
| 163 | .dspp = DSPP_3, | 163 | .dspp = DSPP_3, |
| 164 | }, { | 164 | }, { |
| 165 | .name = "lm_4", .id = LM_4, | 165 | .name = "lm_4", .id = LM_4, |
| 166 | .base = 0x48000, .len = 0x320, | 166 | .base = 0x48000, .len = 0x400, |
| 167 | .features = MIXER_MSM8998_MASK, | 167 | .features = MIXER_MSM8998_MASK, |
| 168 | .sblk = &sdm845_lm_sblk, | 168 | .sblk = &sdm845_lm_sblk, |
| 169 | .lm_pair = LM_5, | 169 | .lm_pair = LM_5, |
| 170 | .pingpong = PINGPONG_4, | 170 | .pingpong = PINGPONG_4, |
| 171 | }, { | 171 | }, { |
| 172 | .name = "lm_5", .id = LM_5, | 172 | .name = "lm_5", .id = LM_5, |
| 173 | .base = 0x49000, .len = 0x320, | 173 | .base = 0x49000, .len = 0x400, |
| 174 | .features = MIXER_MSM8998_MASK, | 174 | .features = MIXER_MSM8998_MASK, |
| 175 | .sblk = &sdm845_lm_sblk, | 175 | .sblk = &sdm845_lm_sblk, |
| 176 | .lm_pair = LM_4, | 176 | .lm_pair = LM_4, |
| @@ -131,7 +131,7 @@ static const struct dpu_sspp_cfg sar2130p_sspp[] = { · 130 unmodified lines | |||
| 131 | static const struct dpu_lm_cfg sar2130p_lm[] = { | 131 | static const struct dpu_lm_cfg sar2130p_lm[] = { |
| 132 | { | 132 | { |
| 133 | .name = "lm_0", .id = LM_0, | 133 | .name = "lm_0", .id = LM_0, |
| 134 | .base = 0x44000, .len = 0x320, | 134 | .base = 0x44000, .len = 0x400, |
| 135 | .features = MIXER_MSM8998_MASK, | 135 | .features = MIXER_MSM8998_MASK, |
| 136 | .sblk = &sdm845_lm_sblk, | 136 | .sblk = &sdm845_lm_sblk, |
| 137 | .lm_pair = LM_1, | 137 | .lm_pair = LM_1, |
| @@ -139,7 +139,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | |||
| 139 | .dspp = DSPP_0, | 139 | .dspp = DSPP_0, |
| 140 | }, { | 140 | }, { |
| 141 | .name = "lm_1", .id = LM_1, | 141 | .name = "lm_1", .id = LM_1, |
| 142 | .base = 0x45000, .len = 0x320, | 142 | .base = 0x45000, .len = 0x400, |
| 143 | .features = MIXER_MSM8998_MASK, | 143 | .features = MIXER_MSM8998_MASK, |
| 144 | .sblk = &sdm845_lm_sblk, | 144 | .sblk = &sdm845_lm_sblk, |
| 145 | .lm_pair = LM_0, | 145 | .lm_pair = LM_0, |
| @@ -147,7 +147,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | |||
| 147 | .dspp = DSPP_1, | 147 | .dspp = DSPP_1, |
| 148 | }, { | 148 | }, { |
| 149 | .name = "lm_2", .id = LM_2, | 149 | .name = "lm_2", .id = LM_2, |
| 150 | .base = 0x46000, .len = 0x320, | 150 | .base = 0x46000, .len = 0x400, |
| 151 | .features = MIXER_MSM8998_MASK, | 151 | .features = MIXER_MSM8998_MASK, |
| 152 | .sblk = &sdm845_lm_sblk, | 152 | .sblk = &sdm845_lm_sblk, |
| 153 | .lm_pair = LM_3, | 153 | .lm_pair = LM_3, |
| @@ -155,7 +155,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | |||
| 155 | .dspp = DSPP_2, | 155 | .dspp = DSPP_2, |
| 156 | }, { | 156 | }, { |
| 157 | .name = "lm_3", .id = LM_3, | 157 | .name = "lm_3", .id = LM_3, |
| 158 | .base = 0x47000, .len = 0x320, | 158 | .base = 0x47000, .len = 0x400, |
| 159 | .features = MIXER_MSM8998_MASK, | 159 | .features = MIXER_MSM8998_MASK, |
| 160 | .sblk = &sdm845_lm_sblk, | 160 | .sblk = &sdm845_lm_sblk, |
| 161 | .lm_pair = LM_2, | 161 | .lm_pair = LM_2, |
| @@ -163,14 +163,14 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | |||
| 163 | .dspp = DSPP_3, | 163 | .dspp = DSPP_3, |
| 164 | }, { | 164 | }, { |
| 165 | .name = "lm_4", .id = LM_4, | 165 | .name = "lm_4", .id = LM_4, |
| 166 | .base = 0x48000, .len = 0x320, | 166 | .base = 0x48000, .len = 0x400, |
| 167 | .features = MIXER_MSM8998_MASK, | 167 | .features = MIXER_MSM8998_MASK, |
| 168 | .sblk = &sdm845_lm_sblk, | 168 | .sblk = &sdm845_lm_sblk, |
| 169 | .lm_pair = LM_5, | 169 | .lm_pair = LM_5, |
| 170 | .pingpong = PINGPONG_4, | 170 | .pingpong = PINGPONG_4, |
| 171 | }, { | 171 | }, { |
| 172 | .name = "lm_5", .id = LM_5, | 172 | .name = "lm_5", .id = LM_5, |
| 173 | .base = 0x49000, .len = 0x320, | 173 | .base = 0x49000, .len = 0x400, |
| 174 | .features = MIXER_MSM8998_MASK, | 174 | .features = MIXER_MSM8998_MASK, |
| 175 | .sblk = &sdm845_lm_sblk, | 175 | .sblk = &sdm845_lm_sblk, |
| 176 | .lm_pair = LM_4, | 176 | .lm_pair = LM_4, |
| @@ -130,7 +130,7 @@ static const struct dpu_sspp_cfg x1e80100_sspp[] = { · 129 unmodified lines | |||
| 130 | static const struct dpu_lm_cfg x1e80100_lm[] = { | 130 | static const struct dpu_lm_cfg x1e80100_lm[] = { |
| 131 | { | 131 | { |
| 132 | .name = "lm_0", .id = LM_0, | 132 | .name = "lm_0", .id = LM_0, |
| 133 | .base = 0x44000, .len = 0x320, | 133 | .base = 0x44000, .len = 0x400, |
| 134 | .features = MIXER_MSM8998_MASK, | 134 | .features = MIXER_MSM8998_MASK, |
| 135 | .sblk = &sdm845_lm_sblk, | 135 | .sblk = &sdm845_lm_sblk, |
| 136 | .lm_pair = LM_1, | 136 | .lm_pair = LM_1, |
| @@ -138,7 +138,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | |||
| 138 | .dspp = DSPP_0, | 138 | .dspp = DSPP_0, |
| 139 | }, { | 139 | }, { |
| 140 | .name = "lm_1", .id = LM_1, | 140 | .name = "lm_1", .id = LM_1, |
| 141 | .base = 0x45000, .len = 0x320, | 141 | .base = 0x45000, .len = 0x400, |
| 142 | .features = MIXER_MSM8998_MASK, | 142 | .features = MIXER_MSM8998_MASK, |
| 143 | .sblk = &sdm845_lm_sblk, | 143 | .sblk = &sdm845_lm_sblk, |
| 144 | .lm_pair = LM_0, | 144 | .lm_pair = LM_0, |
| @@ -146,7 +146,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | |||
| 146 | .dspp = DSPP_1, | 146 | .dspp = DSPP_1, |
| 147 | }, { | 147 | }, { |
| 148 | .name = "lm_2", .id = LM_2, | 148 | .name = "lm_2", .id = LM_2, |
| 149 | .base = 0x46000, .len = 0x320, | 149 | .base = 0x46000, .len = 0x400, |
| 150 | .features = MIXER_MSM8998_MASK, | 150 | .features = MIXER_MSM8998_MASK, |
| 151 | .sblk = &sdm845_lm_sblk, | 151 | .sblk = &sdm845_lm_sblk, |
| 152 | .lm_pair = LM_3, | 152 | .lm_pair = LM_3, |
| @@ -154,7 +154,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | |||
| 154 | .dspp = DSPP_2, | 154 | .dspp = DSPP_2, |
| 155 | }, { | 155 | }, { |
| 156 | .name = "lm_3", .id = LM_3, | 156 | .name = "lm_3", .id = LM_3, |
| 157 | .base = 0x47000, .len = 0x320, | 157 | .base = 0x47000, .len = 0x400, |
| 158 | .features = MIXER_MSM8998_MASK, | 158 | .features = MIXER_MSM8998_MASK, |
| 159 | .sblk = &sdm845_lm_sblk, | 159 | .sblk = &sdm845_lm_sblk, |
| 160 | .lm_pair = LM_2, | 160 | .lm_pair = LM_2, |
| @@ -162,14 +162,14 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | |||
| 162 | .dspp = DSPP_3, | 162 | .dspp = DSPP_3, |
| 163 | }, { | 163 | }, { |
| 164 | .name = "lm_4", .id = LM_4, | 164 | .name = "lm_4", .id = LM_4, |
| 165 | .base = 0x48000, .len = 0x320, | 165 | .base = 0x48000, .len = 0x400, |
| 166 | .features = MIXER_MSM8998_MASK, | 166 | .features = MIXER_MSM8998_MASK, |
| 167 | .sblk = &sdm845_lm_sblk, | 167 | .sblk = &sdm845_lm_sblk, |
| 168 | .lm_pair = LM_5, | 168 | .lm_pair = LM_5, |
| 169 | .pingpong = PINGPONG_4, | 169 | .pingpong = PINGPONG_4, |
| 170 | }, { | 170 | }, { |
| 171 | .name = "lm_5", .id = LM_5, | 171 | .name = "lm_5", .id = LM_5, |
| 172 | .base = 0x49000, .len = 0x320, | 172 | .base = 0x49000, .len = 0x400, |
| 173 | .features = MIXER_MSM8998_MASK, | 173 | .features = MIXER_MSM8998_MASK, |
| 174 | .sblk = &sdm845_lm_sblk, | 174 | .sblk = &sdm845_lm_sblk, |
| 175 | .lm_pair = LM_4, | 175 | .lm_pair = LM_4, |
| @@ -89,7 +89,7 @@ static void dpu_setup_dspp_gc(struct dpu_hw_dspp *ctx, · 88 unmodified lines | |||
| 89 | base = ctx->cap->sblk->gc.base; | 89 | base = ctx->cap->sblk->gc.base; |
| 90 | 90 | ||
| 91 | if (!base) { | 91 | if (!base) { |
| 92 | DRM_ERROR("invalid ctx %pK gc base\n", ctx); | 92 | DRM_ERROR("invalid ctx %p gc base\n", ctx); |
| 93 | return; | 93 | return; |
| 94 | } | 94 | } |
| 95 | 95 | ||
| @@ -156,11 +156,13 @@ static void dpu_hw_sspp_setup_pe_config_v13(struct dpu_hw_sspp *ctx, · 155 unmodified lines | |||
| 156 | u8 color; | 156 | u8 color; |
| 157 | u32 lr_pe[4], tb_pe[4]; | 157 | u32 lr_pe[4], tb_pe[4]; |
| 158 | const u32 bytemask = 0xff; | 158 | const u32 bytemask = 0xff; |
| 159 | u32 offset = ctx->cap->sblk->sspp_rec0_blk.base; | 159 | u32 offset; |
| 160 | 160 | ||
| 161 | if (!ctx || !pe_ext) | 161 | if (!ctx || !pe_ext) |
| 162 | return; | 162 | return; |
| 163 | 163 | ||
| 164 | offset = ctx->cap->sblk->sspp_rec0_blk.base; | ||
| 165 | |||
| 164 | c = &ctx->hw; | 166 | c = &ctx->hw; |
| 165 | /* program SW pixel extension override for all pipes*/ | 167 | /* program SW pixel extension override for all pipes*/ |
| 166 | for (color = 0; color < DPU_MAX_PLANES; color++) { | 168 | for (color = 0; color < DPU_MAX_PLANES; color++) { |
| @@ -350,26 +350,28 @@ static bool _dpu_rm_check_lm_and_get_connected_blks(struct dpu_rm *rm, · 349 unmodified lines | |||
| 350 | return true; | 350 | return true; |
| 351 | } | 351 | } |
| 352 | 352 | ||
| 353 | static bool dpu_rm_find_lms(struct dpu_rm *rm, | 353 | static int _dpu_rm_reserve_lms(struct dpu_rm *rm, |
| 354 | struct dpu_global_state *global_state, | 354 | struct dpu_global_state *global_state, |
| 355 | uint32_t crtc_id, bool skip_dspp, | 355 | uint32_t crtc_id, |
| 356 | struct msm_display_topology *topology, | 356 | struct msm_display_topology *topology) |
| 357 | int *lm_idx, int *pp_idx, int *dspp_idx) | ||
| 358 | 357 | ||
| 359 | { | 358 | { |
| 359 | int lm_idx[MAX_BLOCKS]; | ||
| 360 | int pp_idx[MAX_BLOCKS]; | ||
| 361 | int dspp_idx[MAX_BLOCKS] = {0}; | ||
| 360 | int i, lm_count = 0; | 362 | int i, lm_count = 0; |
| 361 | 363 | ||
| 364 | if (!topology->num_lm) { | ||
| 365 | DPU_ERROR("zero LMs in topology\n"); | ||
| 366 | return -EINVAL; | ||
| 367 | } | ||
| 368 | |||
| 362 | /* Find a primary mixer */ | 369 | /* Find a primary mixer */ |
| 363 | for (i = 0; i < ARRAY_SIZE(rm->mixer_blks) && | 370 | for (i = 0; i < ARRAY_SIZE(rm->mixer_blks) && |
| 364 | lm_count < topology->num_lm; i++) { | 371 | lm_count < topology->num_lm; i++) { |
| 365 | if (!rm->mixer_blks[i]) | 372 | if (!rm->mixer_blks[i]) |
| 366 | continue; | 373 | continue; |
| 367 | 374 | ||
| 368 | if (skip_dspp && to_dpu_hw_mixer(rm->mixer_blks[i])->cap->dspp) { | ||
| 369 | DPU_DEBUG("Skipping LM_%d, skipping LMs with DSPPs\n", i); | ||
| 370 | continue; | ||
| 371 | } | ||
| 372 | |||
| 373 | /* | 375 | /* |
| 374 | * Reset lm_count to an even index. This will drop the previous | 376 | * Reset lm_count to an even index. This will drop the previous |
| 375 | * primary mixer if failed to find its peer. | 377 | * primary mixer if failed to find its peer. |
| @@ -408,38 +410,12 @@ static bool dpu_rm_find_lms(struct dpu_rm *rm, · 32 unmodified lines | |||
| 408 | } | 410 | } |
| 409 | } | 411 | } |
| 410 | 412 | ||
| 411 | return lm_count == topology->num_lm; | 413 | if (lm_count != topology->num_lm) { |
| 412 | } | ||
| 413 | |||
| 414 | static int _dpu_rm_reserve_lms(struct dpu_rm *rm, | ||
| 415 | struct dpu_global_state *global_state, | ||
| 416 | uint32_t crtc_id, | ||
| 417 | struct msm_display_topology *topology) | ||
| 418 | |||
| 419 | { | ||
| 420 | int lm_idx[MAX_BLOCKS]; | ||
| 421 | int pp_idx[MAX_BLOCKS]; | ||
| 422 | int dspp_idx[MAX_BLOCKS] = {0}; | ||
| 423 | int i; | ||
| 424 | bool found; | ||
| 425 | |||
| 426 | if (!topology->num_lm) { | ||
| 427 | DPU_ERROR("zero LMs in topology\n"); | ||
| 428 | return -EINVAL; | ||
| 429 | } | ||
| 430 | |||
| 431 | /* Try using non-DSPP LM blocks first */ | ||
| 432 | found = dpu_rm_find_lms(rm, global_state, crtc_id, !topology->num_dspp, | ||
| 433 | topology, lm_idx, pp_idx, dspp_idx); | ||
| 434 | if (!found && !topology->num_dspp) | ||
| 435 | found = dpu_rm_find_lms(rm, global_state, crtc_id, false, | ||
| 436 | topology, lm_idx, pp_idx, dspp_idx); | ||
| 437 | if (!found) { | ||
| 438 | DPU_DEBUG("unable to find appropriate mixers\n"); | 414 | DPU_DEBUG("unable to find appropriate mixers\n"); |
| 439 | return -ENAVAIL; | 415 | return -ENAVAIL; |
| 440 | } | 416 | } |
| 441 | 417 | ||
| 442 | for (i = 0; i < topology->num_lm; i++) { | 418 | for (i = 0; i < lm_count; i++) { |
| 443 | global_state->mixer_to_crtc_id[lm_idx[i]] = crtc_id; | 419 | global_state->mixer_to_crtc_id[lm_idx[i]] = crtc_id; |
| 444 | global_state->pingpong_to_crtc_id[pp_idx[i]] = crtc_id; | 420 | global_state->pingpong_to_crtc_id[pp_idx[i]] = crtc_id; |
| 445 | global_state->dspp_to_crtc_id[dspp_idx[i]] = | 421 | global_state->dspp_to_crtc_id[dspp_idx[i]] = |
| @@ -584,13 +584,30 @@ void dsi_link_clk_disable_v2(struct msm_dsi_host *msm_host) · 583 unmodified lines | |||
| 584 | * FIXME: Reconsider this if/when CMD mode handling is rewritten to use | 584 | * FIXME: Reconsider this if/when CMD mode handling is rewritten to use |
| 585 | * transfer time and data overhead as a starting point of the calculations. | 585 | * transfer time and data overhead as a starting point of the calculations. |
| 586 | */ | 586 | */ |
| 587 | static unsigned long dsi_adjust_pclk_for_compression(const struct drm_display_mode *mode, | 587 | static unsigned long |
| 588 | const struct drm_dsc_config *dsc) | 588 | dsi_adjust_pclk_for_compression(const struct drm_display_mode *mode, |
| 589 | const struct drm_dsc_config *dsc, | ||
| 590 | bool is_bonded_dsi) | ||
| 589 | { | 591 | { |
| 590 | int new_hdisplay = DIV_ROUND_UP(mode->hdisplay * drm_dsc_get_bpp_int(dsc), | 592 | int hdisplay, new_hdisplay, new_htotal; |
| 591 | dsc->bits_per_component * 3); | ||
| 592 | 593 | ||
| 593 | int new_htotal = mode->htotal - mode->hdisplay + new_hdisplay; | 594 | /* |
| 595 | * For bonded DSI, split hdisplay across two links and round up each | ||
| 596 | * half separately, passing the full hdisplay would only round up once. | ||
| 597 | * This also aligns with the hdisplay we program later in | ||
| 598 | * dsi_timing_setup() | ||
| 599 | */ | ||
| 600 | hdisplay = mode->hdisplay; | ||
| 601 | if (is_bonded_dsi) | ||
| 602 | hdisplay /= 2; | ||
| 603 | |||
| 604 | new_hdisplay = DIV_ROUND_UP(hdisplay * drm_dsc_get_bpp_int(dsc), | ||
| 605 | dsc->bits_per_component * 3); | ||
| 606 | |||
| 607 | if (is_bonded_dsi) | ||
| 608 | new_hdisplay *= 2; | ||
| 609 | |||
| 610 | new_htotal = mode->htotal - mode->hdisplay + new_hdisplay; | ||
| 594 | 611 | ||
| 595 | return mult_frac(mode->clock * 1000u, new_htotal, mode->htotal); | 612 | return mult_frac(mode->clock * 1000u, new_htotal, mode->htotal); |
| 596 | } | 613 | } |
| @@ -603,7 +620,7 @@ static unsigned long dsi_get_pclk_rate(const struct drm_display_mode *mode, · 6 unmodified lines | |||
| 603 | pclk_rate = mode->clock * 1000u; | 620 | pclk_rate = mode->clock * 1000u; |
| 604 | 621 | ||
| 605 | if (dsc) | 622 | if (dsc) |
| 606 | pclk_rate = dsi_adjust_pclk_for_compression(mode, dsc); | 623 | pclk_rate = dsi_adjust_pclk_for_compression(mode, dsc, is_bonded_dsi); |
| 607 | 624 | ||
| 608 | /* | 625 | /* |
| 609 | * For bonded DSI mode, the current DRM mode has the complete width of the | 626 | * For bonded DSI mode, the current DRM mode has the complete width of the |
| @@ -993,7 +1010,7 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 383 unmodified lines | |||
| 993 | 1010 | ||
| 994 | if (msm_host->dsc) { | 1011 | if (msm_host->dsc) { |
| 995 | struct drm_dsc_config *dsc = msm_host->dsc; | 1012 | struct drm_dsc_config *dsc = msm_host->dsc; |
| 996 | u32 bytes_per_pclk; | 1013 | u32 bits_per_pclk; |
| 997 | 1014 | ||
| 998 | /* update dsc params with timing params */ | 1015 | /* update dsc params with timing params */ |
| 999 | if (!dsc || !mode->hdisplay || !mode->vdisplay) { | 1016 | if (!dsc || !mode->hdisplay || !mode->vdisplay) { |
| @@ -1015,7 +1032,9 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 15 unmodified lines | |||
| 1015 | 1032 | ||
| 1016 | /* | 1033 | /* |
| 1017 | * DPU sends 3 bytes per pclk cycle to DSI. If widebus is | 1034 | * DPU sends 3 bytes per pclk cycle to DSI. If widebus is |
| 1018 | * enabled, bus width is extended to 6 bytes. | 1035 | * enabled, MDP always sends out 48-bit compressed data per |
| 1036 | * pclk and on average, DSI consumes an amount of compressed | ||
| 1037 | * data equivalent to the uncompressed pixel depth per pclk. | ||
| 1019 | * | 1038 | * |
| 1020 | * Calculate the number of pclks needed to transmit one line of | 1039 | * Calculate the number of pclks needed to transmit one line of |
| 1021 | * the compressed data. | 1040 | * the compressed data. |
| @@ -1027,12 +1046,12 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 5 unmodified lines | |||
| 1027 | * unused anyway. | 1046 | * unused anyway. |
| 1028 | */ | 1047 | */ |
| 1029 | h_total -= hdisplay; | 1048 | h_total -= hdisplay; |
| 1030 | if (wide_bus_enabled && !(msm_host->mode_flags & MIPI_DSI_MODE_VIDEO)) | 1049 | if (wide_bus_enabled) |
| 1031 | bytes_per_pclk = 6; | 1050 | bits_per_pclk = mipi_dsi_pixel_format_to_bpp(msm_host->format); |
| 1032 | else | 1051 | else |
| 1033 | bytes_per_pclk = 3; | 1052 | bits_per_pclk = 24; |
| 1034 | 1053 | ||
| 1035 | hdisplay = DIV_ROUND_UP(msm_dsc_get_bytes_per_line(msm_host->dsc), bytes_per_pclk); | 1054 | hdisplay = DIV_ROUND_UP(msm_dsc_get_bytes_per_line(msm_host->dsc) * 8, bits_per_pclk); |
| 1036 | 1055 | ||
| 1037 | h_total += hdisplay; | 1056 | h_total += hdisplay; |
| 1038 | ha_end = ha_start + hdisplay; | 1057 | ha_end = ha_start + hdisplay; |
| @@ -51,8 +51,8 @@ · 50 unmodified lines | |||
| 51 | #define DSI_PHY_7NM_QUIRK_V4_3 BIT(3) | 51 | #define DSI_PHY_7NM_QUIRK_V4_3 BIT(3) |
| 52 | /* Hardware is V5.2 */ | 52 | /* Hardware is V5.2 */ |
| 53 | #define DSI_PHY_7NM_QUIRK_V5_2 BIT(4) | 53 | #define DSI_PHY_7NM_QUIRK_V5_2 BIT(4) |
| 54 | /* Hardware is V7.0 */ | 54 | /* Hardware is V7.2 */ |
| 55 | #define DSI_PHY_7NM_QUIRK_V7_0 BIT(5) | 55 | #define DSI_PHY_7NM_QUIRK_V7_2 BIT(5) |
| 56 | 56 | ||
| 57 | struct dsi_pll_config { | 57 | struct dsi_pll_config { |
| 58 | bool enable_ssc; | 58 | bool enable_ssc; |
| @@ -143,7 +143,7 @@ static void dsi_pll_calc_dec_frac(struct dsi_pll_7nm *pll, struct dsi_pll_config · 84 unmodified lines | |||
| 143 | 143 | ||
| 144 | if (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_PRE_V4_1) { | 144 | if (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_PRE_V4_1) { |
| 145 | config->pll_clock_inverters = 0x28; | 145 | config->pll_clock_inverters = 0x28; |
| 146 | } else if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | 146 | } else if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { |
| 147 | if (pll_freq < 163000000ULL) | 147 | if (pll_freq < 163000000ULL) |
| 148 | config->pll_clock_inverters = 0xa0; | 148 | config->pll_clock_inverters = 0xa0; |
| 149 | else if (pll_freq < 175000000ULL) | 149 | else if (pll_freq < 175000000ULL) |
| @@ -284,7 +284,7 @@ static void dsi_pll_config_hzindep_reg(struct dsi_pll_7nm *pll) · 134 unmodified lines | |||
| 284 | } | 284 | } |
| 285 | 285 | ||
| 286 | if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || | 286 | if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 287 | (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | 287 | (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { |
| 288 | if (pll->vco_current_rate < 1557000000ULL) | 288 | if (pll->vco_current_rate < 1557000000ULL) |
| 289 | vco_config_1 = 0x08; | 289 | vco_config_1 = 0x08; |
| 290 | else | 290 | else |
| @@ -699,7 +699,7 @@ static int dsi_7nm_set_usecase(struct msm_dsi_phy *phy) · 408 unmodified lines | |||
| 699 | case MSM_DSI_PHY_MASTER: | 699 | case MSM_DSI_PHY_MASTER: |
| 700 | pll_7nm->slave = pll_7nm_list[(pll_7nm->phy->id + 1) % DSI_MAX]; | 700 | pll_7nm->slave = pll_7nm_list[(pll_7nm->phy->id + 1) % DSI_MAX]; |
| 701 | /* v7.0: Enable ATB_EN0 and alternate clock output to external phy */ | 701 | /* v7.0: Enable ATB_EN0 and alternate clock output to external phy */ |
| 702 | if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0) | 702 | if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2) |
| 703 | writel(0x07, base + REG_DSI_7nm_PHY_CMN_CTRL_5); | 703 | writel(0x07, base + REG_DSI_7nm_PHY_CMN_CTRL_5); |
| 704 | break; | 704 | break; |
| 705 | case MSM_DSI_PHY_SLAVE: | 705 | case MSM_DSI_PHY_SLAVE: |
| @@ -987,7 +987,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 281 unmodified lines | |||
| 987 | /* Request for REFGEN READY */ | 987 | /* Request for REFGEN READY */ |
| 988 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || | 988 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || |
| 989 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || | 989 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 990 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | 990 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { |
| 991 | writel(0x1, phy->base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); | 991 | writel(0x1, phy->base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); |
| 992 | udelay(500); | 992 | udelay(500); |
| 993 | } | 993 | } |
| @@ -1021,7 +1021,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 27 unmodified lines | |||
| 1021 | lane_ctrl0 = 0x1f; | 1021 | lane_ctrl0 = 0x1f; |
| 1022 | } | 1022 | } |
| 1023 | 1023 | ||
| 1024 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | 1024 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { |
| 1025 | if (phy->cphy_mode) { | 1025 | if (phy->cphy_mode) { |
| 1026 | /* TODO: different for second phy */ | 1026 | /* TODO: different for second phy */ |
| 1027 | vreg_ctrl_0 = 0x57; | 1027 | vreg_ctrl_0 = 0x57; |
| @@ -1097,7 +1097,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 69 unmodified lines | |||
| 1097 | 1097 | ||
| 1098 | /* program CMN_CTRL_4 for minor_ver 2 chipsets*/ | 1098 | /* program CMN_CTRL_4 for minor_ver 2 chipsets*/ |
| 1099 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || | 1099 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 1100 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0) || | 1100 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2) || |
| 1101 | (readl(base + REG_DSI_7nm_PHY_CMN_REVISION_ID0) & (0xf0)) == 0x20) | 1101 | (readl(base + REG_DSI_7nm_PHY_CMN_REVISION_ID0) & (0xf0)) == 0x20) |
| 1102 | writel(0x04, base + REG_DSI_7nm_PHY_CMN_CTRL_4); | 1102 | writel(0x04, base + REG_DSI_7nm_PHY_CMN_CTRL_4); |
| 1103 | 1103 | ||
| @@ -1213,7 +1213,7 @@ static void dsi_7nm_phy_disable(struct msm_dsi_phy *phy) · 109 unmodified lines | |||
| 1213 | /* Turn off REFGEN Vote */ | 1213 | /* Turn off REFGEN Vote */ |
| 1214 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || | 1214 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || |
| 1215 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || | 1215 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 1216 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | 1216 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { |
| 1217 | writel(0x0, base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); | 1217 | writel(0x0, base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); |
| 1218 | wmb(); | 1218 | wmb(); |
| 1219 | /* Delay to ensure HW removes vote before PHY shut down */ | 1219 | /* Delay to ensure HW removes vote before PHY shut down */ |
| @@ -1502,7 +1502,7 @@ const struct msm_dsi_phy_cfg dsi_phy_3nm_8750_cfgs = { · 282 unmodified lines | |||
| 1502 | #endif | 1502 | #endif |
| 1503 | .io_start = { 0xae95000, 0xae97000 }, | 1503 | .io_start = { 0xae95000, 0xae97000 }, |
| 1504 | .num_dsi_phy = 2, | 1504 | .num_dsi_phy = 2, |
| 1505 | .quirks = DSI_PHY_7NM_QUIRK_V7_0, | 1505 | .quirks = DSI_PHY_7NM_QUIRK_V7_2, |
| 1506 | }; | 1506 | }; |
| 1507 | 1507 | ||
| 1508 | const struct msm_dsi_phy_cfg dsi_phy_3nm_kaanapali_cfgs = { | 1508 | const struct msm_dsi_phy_cfg dsi_phy_3nm_kaanapali_cfgs = { |
| @@ -1525,5 +1525,5 @@ const struct msm_dsi_phy_cfg dsi_phy_3nm_kaanapali_cfgs = { · 16 unmodified lines | |||
| 1525 | #endif | 1525 | #endif |
| 1526 | .io_start = { 0x9ac1000, 0x9ac4000 }, | 1526 | .io_start = { 0x9ac1000, 0x9ac4000 }, |
| 1527 | .num_dsi_phy = 2, | 1527 | .num_dsi_phy = 2, |
| 1528 | .quirks = DSI_PHY_7NM_QUIRK_V7_0, | 1528 | .quirks = DSI_PHY_7NM_QUIRK_V7_2, |
| 1529 | }; | 1529 | }; |
| @@ -1230,6 +1230,9 @@ nouveau_connector_aux_xfer(struct drm_dp_aux *obj, struct drm_dp_aux_msg *msg) · 1229 unmodified lines | |||
| 1230 | u8 size = msg->size; | 1230 | u8 size = msg->size; |
| 1231 | int ret; | 1231 | int ret; |
| 1232 | 1232 | ||
| 1233 | if (pm_runtime_suspended(nv_connector->base.dev->dev)) | ||
| 1234 | return -EBUSY; | ||
| 1235 | |||
| 1233 | nv_encoder = find_encoder(&nv_connector->base, DCB_OUTPUT_DP); | 1236 | nv_encoder = find_encoder(&nv_connector->base, DCB_OUTPUT_DP); |
| 1234 | if (!nv_encoder) | 1237 | if (!nv_encoder) |
| 1235 | return -ENODEV; | 1238 | return -ENODEV; |
| @@ -737,8 +737,8 @@ r535_gsp_acpi_caps(acpi_handle handle, CAPS_METHOD_DATA *caps) · 736 unmodified lines | |||
| 737 | if (!obj) | 737 | if (!obj) |
| 738 | goto done; | 738 | goto done; |
| 739 | 739 | ||
| 740 | if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || | 740 | if (obj->type != ACPI_TYPE_BUFFER || |
| 741 | WARN_ON(obj->buffer.length != 4)) | 741 | obj->buffer.length != 4) |
| 742 | goto done; | 742 | goto done; |
| 743 | 743 | ||
| 744 | caps->status = 0; | 744 | caps->status = 0; |
| @@ -773,8 +773,8 @@ r535_gsp_acpi_jt(acpi_handle handle, JT_METHOD_DATA *jt) · 28 unmodified lines | |||
| 773 | if (!obj) | 773 | if (!obj) |
| 774 | goto done; | 774 | goto done; |
| 775 | 775 | ||
| 776 | if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || | 776 | if (obj->type != ACPI_TYPE_BUFFER || |
| 777 | WARN_ON(obj->buffer.length != 4)) | 777 | obj->buffer.length != 4) |
| 778 | goto done; | 778 | goto done; |
| 779 | 779 | ||
| 780 | jt->status = 0; | 780 | jt->status = 0; |
| @@ -861,8 +861,8 @@ r535_gsp_acpi_dod(acpi_handle handle, DOD_METHOD_DATA *dod) · 80 unmodified lines | |||
| 861 | 861 | ||
| 862 | _DOD = output.pointer; | 862 | _DOD = output.pointer; |
| 863 | 863 | ||
| 864 | if (WARN_ON(_DOD->type != ACPI_TYPE_PACKAGE) || | 864 | if (_DOD->type != ACPI_TYPE_PACKAGE || |
| 865 | WARN_ON(_DOD->package.count > ARRAY_SIZE(dod->acpiIdList))) | 865 | _DOD->package.count > ARRAY_SIZE(dod->acpiIdList)) |
| 866 | return; | 866 | return; |
| 867 | 867 | ||
| 868 | for (int i = 0; i < _DOD->package.count; i++) { | 868 | for (int i = 0; i < _DOD->package.count; i++) { |
| @@ -893,14 +893,15 @@ panthor_queue_get_syncwait_obj(struct panthor_group *group, struct panthor_queue · 892 unmodified lines | |||
| 893 | 893 | ||
| 894 | out_sync: | 894 | out_sync: |
| 895 | /* Make sure the CPU caches are invalidated before the seqno is read. | 895 | /* Make sure the CPU caches are invalidated before the seqno is read. |
| 896 | * drm_gem_shmem_sync() is a NOP if map_wc=true, so no need to check | 896 | * panthor_gem_sync() is a NOP if map_wc=true, so no need to check |
| 897 | * it here. | 897 | * it here. |
| 898 | */ | 898 | */ |
| 899 | panthor_gem_sync(&bo->base.base, queue->syncwait.offset, | 899 | panthor_gem_sync(&bo->base.base, |
| 900 | DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE, | ||
| 901 | queue->syncwait.offset, | ||
| 900 | queue->syncwait.sync64 ? | 902 | queue->syncwait.sync64 ? |
| 901 | sizeof(struct panthor_syncobj_64b) : | 903 | sizeof(struct panthor_syncobj_64b) : |
| 902 | sizeof(struct panthor_syncobj_32b), | 904 | sizeof(struct panthor_syncobj_32b)); |
| 903 | DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE); | ||
| 904 | 905 | ||
| 905 | return queue->syncwait.kmap + queue->syncwait.offset; | 906 | return queue->syncwait.kmap + queue->syncwait.offset; |
| 906 | 907 | ||
| @@ -1122,6 +1122,7 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 1121 unmodified lines | |||
| 1122 | struct mipi_dsi_device *device) | 1122 | struct mipi_dsi_device *device) |
| 1123 | { | 1123 | { |
| 1124 | struct rzg2l_mipi_dsi *dsi = host_to_rzg2l_mipi_dsi(host); | 1124 | struct rzg2l_mipi_dsi *dsi = host_to_rzg2l_mipi_dsi(host); |
| 1125 | int bpp; | ||
| 1125 | int ret; | 1126 | int ret; |
| 1126 | 1127 | ||
| 1127 | if (device->lanes > dsi->num_data_lanes) { | 1128 | if (device->lanes > dsi->num_data_lanes) { |
| @@ -1131,7 +1132,8 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 3 unmodified lines | |||
| 1131 | return -EINVAL; | 1132 | return -EINVAL; |
| 1132 | } | 1133 | } |
| 1133 | 1134 | ||
| 1134 | switch (mipi_dsi_pixel_format_to_bpp(device->format)) { | 1135 | bpp = mipi_dsi_pixel_format_to_bpp(device->format); |
| 1136 | switch (bpp) { | ||
| 1135 | case 24: | 1137 | case 24: |
| 1136 | break; | 1138 | break; |
| 1137 | case 18: | 1139 | case 18: |
| @@ -1162,6 +1164,18 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 24 unmodified lines | |||
| 1162 | 1164 | ||
| 1163 | drm_bridge_add(&dsi->bridge); | 1165 | drm_bridge_add(&dsi->bridge); |
| 1164 | 1166 | ||
| 1167 | /* | ||
| 1168 | * Report the required division ratio setting for the MIPI clock dividers. | ||
| 1169 | * | ||
| 1170 | * vclk * bpp = hsclk * 8 * num_lanes | ||
| 1171 | * | ||
| 1172 | * vclk * DSI_AB_divider = hsclk * 16 | ||
| 1173 | * | ||
| 1174 | * which simplifies to... | ||
| 1175 | * DSI_AB_divider = bpp * 2 / num_lanes | ||
| 1176 | */ | ||
| 1177 | rzg2l_cpg_dsi_div_set_divider(bpp * 2 / dsi->lanes, PLL5_TARGET_DSI); | ||
| 1178 | |||
| 1165 | return 0; | 1179 | return 0; |
| 1166 | } | 1180 | } |
| 1167 | 1181 | ||
| @@ -361,6 +361,7 @@ static void drm_sched_run_free_queue(struct drm_gpu_scheduler *sched) · 360 unmodified lines | |||
| 361 | /** | 361 | /** |
| 362 | * drm_sched_job_done - complete a job | 362 | * drm_sched_job_done - complete a job |
| 363 | * @s_job: pointer to the job which is done | 363 | * @s_job: pointer to the job which is done |
| 364 | * @result: 0 on success, -ERRNO on error | ||
| 364 | * | 365 | * |
| 365 | * Finish the job's fence and resubmit the work items. | 366 | * Finish the job's fence and resubmit the work items. |
| 366 | */ | 367 | */ |
| @@ -347,6 +347,12 @@ static int st7586_probe(struct spi_device *spi) · 346 unmodified lines | |||
| 347 | if (ret) | 347 | if (ret) |
| 348 | return ret; | 348 | return ret; |
| 349 | 349 | ||
| 350 | /* | ||
| 351 | * Override value set by mipi_dbi_spi_init(). This driver is a bit | ||
| 352 | * non-standard, so best to set it explicitly here. | ||
| 353 | */ | ||
| 354 | dbi->write_memory_bpw = 8; | ||
| 355 | |||
| 350 | /* Cannot read from this controller via SPI */ | 356 | /* Cannot read from this controller via SPI */ |
| 351 | dbi->read_commands = NULL; | 357 | dbi->read_commands = NULL; |
| 352 | 358 | ||
| @@ -356,15 +362,6 @@ static int st7586_probe(struct spi_device *spi) · 3 unmodified lines | |||
| 356 | if (ret) | 362 | if (ret) |
| 357 | return ret; | 363 | return ret; |
| 358 | 364 | ||
| 359 | /* | ||
| 360 | * we are using 8-bit data, so we are not actually swapping anything, | ||
| 361 | * but setting mipi->swap_bytes makes mipi_dbi_typec3_command() do the | ||
| 362 | * right thing and not use 16-bit transfers (which results in swapped | ||
| 363 | * bytes on little-endian systems and causes out of order data to be | ||
| 364 | * sent to the display). | ||
| 365 | */ | ||
| 366 | dbi->swap_bytes = true; | ||
| 367 | |||
| 368 | drm_mode_config_reset(drm); | 365 | drm_mode_config_reset(drm); |
| 369 | 366 | ||
| 370 | ret = drm_dev_register(drm, 0); | 367 | ret = drm_dev_register(drm, 0); |
| @@ -737,6 +737,7 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 736 unmodified lines | |||
| 737 | unsigned int height = drm_rect_height(rect); | 737 | unsigned int height = drm_rect_height(rect); |
| 738 | unsigned int line_length = DIV_ROUND_UP(width, 8); | 738 | unsigned int line_length = DIV_ROUND_UP(width, 8); |
| 739 | unsigned int page_height = SSD130X_PAGE_HEIGHT; | 739 | unsigned int page_height = SSD130X_PAGE_HEIGHT; |
| 740 | u8 page_start = ssd130x->page_offset + y / page_height; | ||
| 740 | unsigned int pages = DIV_ROUND_UP(height, page_height); | 741 | unsigned int pages = DIV_ROUND_UP(height, page_height); |
| 741 | struct drm_device *drm = &ssd130x->drm; | 742 | struct drm_device *drm = &ssd130x->drm; |
| 742 | u32 array_idx = 0; | 743 | u32 array_idx = 0; |
| @@ -774,14 +775,11 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 31 unmodified lines | |||
| 774 | */ | 775 | */ |
| 775 | 776 | ||
| 776 | if (!ssd130x->page_address_mode) { | 777 | if (!ssd130x->page_address_mode) { |
| 777 | u8 page_start; | ||
| 778 | |||
| 779 | /* Set address range for horizontal addressing mode */ | 778 | /* Set address range for horizontal addressing mode */ |
| 780 | ret = ssd130x_set_col_range(ssd130x, ssd130x->col_offset + x, width); | 779 | ret = ssd130x_set_col_range(ssd130x, ssd130x->col_offset + x, width); |
| 781 | if (ret < 0) | 780 | if (ret < 0) |
| 782 | return ret; | 781 | return ret; |
| 783 | 782 | ||
| 784 | page_start = ssd130x->page_offset + y / page_height; | ||
| 785 | ret = ssd130x_set_page_range(ssd130x, page_start, pages); | 783 | ret = ssd130x_set_page_range(ssd130x, page_start, pages); |
| 786 | if (ret < 0) | 784 | if (ret < 0) |
| 787 | return ret; | 785 | return ret; |
| @@ -813,7 +811,7 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 25 unmodified lines | |||
| 813 | */ | 811 | */ |
| 814 | if (ssd130x->page_address_mode) { | 812 | if (ssd130x->page_address_mode) { |
| 815 | ret = ssd130x_set_page_pos(ssd130x, | 813 | ret = ssd130x_set_page_pos(ssd130x, |
| 816 | ssd130x->page_offset + i, | 814 | page_start + i, |
| 817 | ssd130x->col_offset + x); | 815 | ssd130x->col_offset + x); |
| 818 | if (ret < 0) | 816 | if (ret < 0) |
| 819 | return ret; | 817 | return ret; |
| @@ -541,8 +541,8 @@ static int sharp_memory_probe(struct spi_device *spi) · 540 unmodified lines | |||
| 541 | 541 | ||
| 542 | smd = devm_drm_dev_alloc(dev, &sharp_memory_drm_driver, | 542 | smd = devm_drm_dev_alloc(dev, &sharp_memory_drm_driver, |
| 543 | struct sharp_memory_device, drm); | 543 | struct sharp_memory_device, drm); |
| 544 | if (!smd) | 544 | if (IS_ERR(smd)) |
| 545 | return -ENOMEM; | 545 | return PTR_ERR(smd); |
| 546 | 546 | ||
| 547 | spi_set_drvdata(spi, smd); | 547 | spi_set_drvdata(spi, smd); |
| 548 | 548 | ||
| @@ -222,13 +222,13 @@ static void ttm_bo_reserve_interrupted(struct kunit *test) · 221 unmodified lines | |||
| 222 | KUNIT_FAIL(test, "Couldn't create ttm bo reserve task\n"); | 222 | KUNIT_FAIL(test, "Couldn't create ttm bo reserve task\n"); |
| 223 | 223 | ||
| 224 | /* Take a lock so the threaded reserve has to wait */ | 224 | /* Take a lock so the threaded reserve has to wait */ |
| 225 | mutex_lock(&bo->base.resv->lock.base); | 225 | dma_resv_lock(bo->base.resv, NULL); |
| 226 | 226 | ||
| 227 | wake_up_process(task); | 227 | wake_up_process(task); |
| 228 | msleep(20); | 228 | msleep(20); |
| 229 | err = kthread_stop(task); | 229 | err = kthread_stop(task); |
| 230 | 230 | ||
| 231 | mutex_unlock(&bo->base.resv->lock.base); | 231 | dma_resv_unlock(bo->base.resv); |
| 232 | 232 | ||
| 233 | KUNIT_ASSERT_EQ(test, err, -ERESTARTSYS); | 233 | KUNIT_ASSERT_EQ(test, err, -ERESTARTSYS); |
| 234 | } | 234 | } |
| @@ -1107,8 +1107,7 @@ struct ttm_bo_swapout_walk { · 1106 unmodified lines | |||
| 1107 | static s64 | 1107 | static s64 |
| 1108 | ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) | 1108 | ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) |
| 1109 | { | 1109 | { |
| 1110 | struct ttm_resource *res = bo->resource; | 1110 | struct ttm_place place = { .mem_type = bo->resource->mem_type }; |
| 1111 | struct ttm_place place = { .mem_type = res->mem_type }; | ||
| 1112 | struct ttm_bo_swapout_walk *swapout_walk = | 1111 | struct ttm_bo_swapout_walk *swapout_walk = |
| 1113 | container_of(walk, typeof(*swapout_walk), walk); | 1112 | container_of(walk, typeof(*swapout_walk), walk); |
| 1114 | struct ttm_operation_ctx *ctx = walk->arg.ctx; | 1113 | struct ttm_operation_ctx *ctx = walk->arg.ctx; |
| @@ -1148,7 +1147,7 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 33 unmodified lines | |||
| 1148 | /* | 1147 | /* |
| 1149 | * Move to system cached | 1148 | * Move to system cached |
| 1150 | */ | 1149 | */ |
| 1151 | if (res->mem_type != TTM_PL_SYSTEM) { | 1150 | if (bo->resource->mem_type != TTM_PL_SYSTEM) { |
| 1152 | struct ttm_resource *evict_mem; | 1151 | struct ttm_resource *evict_mem; |
| 1153 | struct ttm_place hop; | 1152 | struct ttm_place hop; |
| 1154 | 1153 | ||
| @@ -1180,15 +1179,15 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 25 unmodified lines | |||
| 1180 | 1179 | ||
| 1181 | if (ttm_tt_is_populated(tt)) { | 1180 | if (ttm_tt_is_populated(tt)) { |
| 1182 | spin_lock(&bdev->lru_lock); | 1181 | spin_lock(&bdev->lru_lock); |
| 1183 | ttm_resource_del_bulk_move(res, bo); | 1182 | ttm_resource_del_bulk_move(bo->resource, bo); |
| 1184 | spin_unlock(&bdev->lru_lock); | 1183 | spin_unlock(&bdev->lru_lock); |
| 1185 | 1184 | ||
| 1186 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); | 1185 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); |
| 1187 | 1186 | ||
| 1188 | spin_lock(&bdev->lru_lock); | 1187 | spin_lock(&bdev->lru_lock); |
| 1189 | if (ret) | 1188 | if (ret) |
| 1190 | ttm_resource_add_bulk_move(res, bo); | 1189 | ttm_resource_add_bulk_move(bo->resource, bo); |
| 1191 | ttm_resource_move_to_lru_tail(res); | 1190 | ttm_resource_move_to_lru_tail(bo->resource); |
| 1192 | spin_unlock(&bdev->lru_lock); | 1191 | spin_unlock(&bdev->lru_lock); |
| 1193 | } | 1192 | } |
| 1194 | 1193 | ||
| @@ -17,7 +17,7 @@ static inline bool ttm_pool_uses_dma32(struct ttm_pool *pool) · 16 unmodified lines | |||
| 17 | return pool->alloc_flags & TTM_ALLOCATION_POOL_USE_DMA32; | 17 | return pool->alloc_flags & TTM_ALLOCATION_POOL_USE_DMA32; |
| 18 | } | 18 | } |
| 19 | 19 | ||
| 20 | static inline bool ttm_pool_beneficial_order(struct ttm_pool *pool) | 20 | static inline unsigned int ttm_pool_beneficial_order(struct ttm_pool *pool) |
| 21 | { | 21 | { |
| 22 | return pool->alloc_flags & 0xff; | 22 | return pool->alloc_flags & 0xff; |
| 23 | } | 23 | } |
| @@ -105,6 +105,7 @@ struct vmw_cmdbuf_context { · 104 unmodified lines | |||
| 105 | * @handle: DMA address handle for the command buffer space if @using_mob is | 105 | * @handle: DMA address handle for the command buffer space if @using_mob is |
| 106 | * false. Immutable. | 106 | * false. Immutable. |
| 107 | * @size: The size of the command buffer space. Immutable. | 107 | * @size: The size of the command buffer space. Immutable. |
| 108 | * @id: Monotonically increasing ID of the last cmdbuf submitted. | ||
| 108 | * @num_contexts: Number of contexts actually enabled. | 109 | * @num_contexts: Number of contexts actually enabled. |
| 109 | */ | 110 | */ |
| 110 | struct vmw_cmdbuf_man { | 111 | struct vmw_cmdbuf_man { |
| @@ -132,6 +133,7 @@ struct vmw_cmdbuf_man { · 21 unmodified lines | |||
| 132 | bool has_pool; | 133 | bool has_pool; |
| 133 | dma_addr_t handle; | 134 | dma_addr_t handle; |
| 134 | size_t size; | 135 | size_t size; |
| 136 | u64 id; | ||
| 135 | u32 num_contexts; | 137 | u32 num_contexts; |
| 136 | }; | 138 | }; |
| 137 | 139 | ||
| @@ -303,6 +305,8 @@ static int vmw_cmdbuf_header_submit(struct vmw_cmdbuf_header *header) · 165 unmodified lines | |||
| 303 | struct vmw_cmdbuf_man *man = header->man; | 305 | struct vmw_cmdbuf_man *man = header->man; |
| 304 | u32 val; | 306 | u32 val; |
| 305 | 307 | ||
| 308 | header->cb_header->id = man->id++; | ||
| 309 | |||
| 306 | val = upper_32_bits(header->handle); | 310 | val = upper_32_bits(header->handle); |
| 307 | vmw_write(man->dev_priv, SVGA_REG_COMMAND_HIGH, val); | 311 | vmw_write(man->dev_priv, SVGA_REG_COMMAND_HIGH, val); |
| 308 | 312 | ||
| @@ -1143,7 +1143,7 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, · 1142 unmodified lines | |||
| 1143 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); | 1143 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); |
| 1144 | if (ret != 0) { | 1144 | if (ret != 0) { |
| 1145 | drm_dbg(&dev_priv->drm, "Could not find or use MOB buffer.\n"); | 1145 | drm_dbg(&dev_priv->drm, "Could not find or use MOB buffer.\n"); |
| 1146 | return PTR_ERR(vmw_bo); | 1146 | return ret; |
| 1147 | } | 1147 | } |
| 1148 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_MOB, VMW_BO_DOMAIN_MOB); | 1148 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_MOB, VMW_BO_DOMAIN_MOB); |
| 1149 | ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo); | 1149 | ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo); |
| @@ -1199,7 +1199,7 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv, · 49 unmodified lines | |||
| 1199 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); | 1199 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); |
| 1200 | if (ret != 0) { | 1200 | if (ret != 0) { |
| 1201 | drm_dbg(&dev_priv->drm, "Could not find or use GMR region.\n"); | 1201 | drm_dbg(&dev_priv->drm, "Could not find or use GMR region.\n"); |
| 1202 | return PTR_ERR(vmw_bo); | 1202 | return ret; |
| 1203 | } | 1203 | } |
| 1204 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM, | 1204 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM, |
| 1205 | VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM); | 1205 | VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM); |
| @@ -260,6 +260,13 @@ out_no_dirty: · 259 unmodified lines | |||
| 260 | return ret; | 260 | return ret; |
| 261 | } | 261 | } |
| 262 | 262 | ||
| 263 | static void vmw_bo_dirty_free(struct kref *kref) | ||
| 264 | { | ||
| 265 | struct vmw_bo_dirty *dirty = container_of(kref, struct vmw_bo_dirty, ref_count); | ||
| 266 | |||
| 267 | kvfree(dirty); | ||
| 268 | } | ||
| 269 | |||
| 263 | /** | 270 | /** |
| 264 | * vmw_bo_dirty_release - Release a dirty-tracking user from a buffer object | 271 | * vmw_bo_dirty_release - Release a dirty-tracking user from a buffer object |
| 265 | * @vbo: The buffer object | 272 | * @vbo: The buffer object |
| @@ -274,7 +281,7 @@ void vmw_bo_dirty_release(struct vmw_bo *vbo) · 8 unmodified lines | |||
| 274 | { | 281 | { |
| 275 | struct vmw_bo_dirty *dirty = vbo->dirty; | 282 | struct vmw_bo_dirty *dirty = vbo->dirty; |
| 276 | 283 | ||
| 277 | if (dirty && kref_put(&dirty->ref_count, (void *)kvfree)) | 284 | if (dirty && kref_put(&dirty->ref_count, vmw_bo_dirty_free)) |
| 278 | vbo->dirty = NULL; | 285 | vbo->dirty = NULL; |
| 279 | } | 286 | } |
| 280 | 287 | ||
| @@ -96,6 +96,12 @@ · 95 unmodified lines | |||
| 96 | #define ENABLE_SEMAPHORE_POLL_BIT REG_BIT(13) | 96 | #define ENABLE_SEMAPHORE_POLL_BIT REG_BIT(13) |
| 97 | 97 | ||
| 98 | #define RING_CMD_CCTL(base) XE_REG((base) + 0xc4, XE_REG_OPTION_MASKED) | 98 | #define RING_CMD_CCTL(base) XE_REG((base) + 0xc4, XE_REG_OPTION_MASKED) |
| 99 | |||
| 100 | #define CS_MMIO_GROUP_INSTANCE_SELECT(base) XE_REG((base) + 0xcc) | ||
| 101 | #define SELECTIVE_READ_ADDRESSING REG_BIT(30) | ||
| 102 | #define SELECTIVE_READ_GROUP REG_GENMASK(29, 23) | ||
| 103 | #define SELECTIVE_READ_INSTANCE REG_GENMASK(22, 16) | ||
| 104 | |||
| 99 | /* | 105 | /* |
| 100 | * CMD_CCTL read/write fields take a MOCS value and _not_ a table index. | 106 | * CMD_CCTL read/write fields take a MOCS value and _not_ a table index. |
| 101 | * The lsb of each can be considered a separate enabling bit for encryption. | 107 | * The lsb of each can be considered a separate enabling bit for encryption. |
| @@ -830,6 +830,7 @@ static void xe_config_device_release(struct config_item *item) · 829 unmodified lines | |||
| 830 | 830 | ||
| 831 | mutex_destroy(&dev->lock); | 831 | mutex_destroy(&dev->lock); |
| 832 | 832 | ||
| 833 | kfree(dev->config.ctx_restore_mid_bb[0].cs); | ||
| 833 | kfree(dev->config.ctx_restore_post_bb[0].cs); | 834 | kfree(dev->config.ctx_restore_post_bb[0].cs); |
| 834 | kfree(dev); | 835 | kfree(dev); |
| 835 | } | 836 | } |
| @@ -266,6 +266,16 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe, · 265 unmodified lines | |||
| 266 | return q; | 266 | return q; |
| 267 | } | 267 | } |
| 268 | 268 | ||
| 269 | static void __xe_exec_queue_fini(struct xe_exec_queue *q) | ||
| 270 | { | ||
| 271 | int i; | ||
| 272 | |||
| 273 | q->ops->fini(q); | ||
| 274 | |||
| 275 | for (i = 0; i < q->width; ++i) | ||
| 276 | xe_lrc_put(q->lrc[i]); | ||
| 277 | } | ||
| 278 | |||
| 269 | static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) | 279 | static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) |
| 270 | { | 280 | { |
| 271 | int i, err; | 281 | int i, err; |
| @@ -320,21 +330,10 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) · 48 unmodified lines | |||
| 320 | return 0; | 330 | return 0; |
| 321 | 331 | ||
| 322 | err_lrc: | 332 | err_lrc: |
| 323 | for (i = i - 1; i >= 0; --i) | 333 | __xe_exec_queue_fini(q); |
| 324 | xe_lrc_put(q->lrc[i]); | ||
| 325 | return err; | 334 | return err; |
| 326 | } | 335 | } |
| 327 | 336 | ||
| 328 | static void __xe_exec_queue_fini(struct xe_exec_queue *q) | ||
| 329 | { | ||
| 330 | int i; | ||
| 331 | |||
| 332 | q->ops->fini(q); | ||
| 333 | |||
| 334 | for (i = 0; i < q->width; ++i) | ||
| 335 | xe_lrc_put(q->lrc[i]); | ||
| 336 | } | ||
| 337 | |||
| 338 | struct xe_exec_queue *xe_exec_queue_create(struct xe_device *xe, struct xe_vm *vm, | 337 | struct xe_exec_queue *xe_exec_queue_create(struct xe_device *xe, struct xe_vm *vm, |
| 339 | u32 logical_mask, u16 width, | 338 | u32 logical_mask, u16 width, |
| 340 | struct xe_hw_engine *hwe, u32 flags, | 339 | struct xe_hw_engine *hwe, u32 flags, |
| @@ -435,15 +435,11 @@ static int proxy_channel_alloc(struct xe_gsc *gsc) · 434 unmodified lines | |||
| 435 | return 0; | 435 | return 0; |
| 436 | } | 436 | } |
| 437 | 437 | ||
| 438 | static void xe_gsc_proxy_remove(void *arg) | 438 | static void xe_gsc_proxy_stop(struct xe_gsc *gsc) |
| 439 | { | 439 | { |
| 440 | struct xe_gsc *gsc = arg; | ||
| 441 | struct xe_gt *gt = gsc_to_gt(gsc); | 440 | struct xe_gt *gt = gsc_to_gt(gsc); |
| 442 | struct xe_device *xe = gt_to_xe(gt); | 441 | struct xe_device *xe = gt_to_xe(gt); |
| 443 | 442 | ||
| 444 | if (!gsc->proxy.component_added) | ||
| 445 | return; | ||
| 446 | |||
| 447 | /* disable HECI2 IRQs */ | 443 | /* disable HECI2 IRQs */ |
| 448 | scoped_guard(xe_pm_runtime, xe) { | 444 | scoped_guard(xe_pm_runtime, xe) { |
| 449 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FW_GSC); | 445 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FW_GSC); |
| @@ -455,6 +451,30 @@ static void xe_gsc_proxy_remove(void *arg) · 5 unmodified lines | |||
| 455 | } | 451 | } |
| 456 | 452 | ||
| 457 | xe_gsc_wait_for_worker_completion(gsc); | 453 | xe_gsc_wait_for_worker_completion(gsc); |
| 454 | gsc->proxy.started = false; | ||
| 455 | } | ||
| 456 | |||
| 457 | static void xe_gsc_proxy_remove(void *arg) | ||
| 458 | { | ||
| 459 | struct xe_gsc *gsc = arg; | ||
| 460 | struct xe_gt *gt = gsc_to_gt(gsc); | ||
| 461 | struct xe_device *xe = gt_to_xe(gt); | ||
| 462 | |||
| 463 | if (!gsc->proxy.component_added) | ||
| 464 | return; | ||
| 465 | |||
| 466 | /* | ||
| 467 | * GSC proxy start is an async process that can be ongoing during | ||
| 468 | * Xe module load/unload. Using devm managed action to register | ||
| 469 | * xe_gsc_proxy_stop could cause issues if Xe module unload has | ||
| 470 | * already started when the action is registered, potentially leading | ||
| 471 | * to the cleanup being called at the wrong time. Therefore, instead | ||
| 472 | * of registering a separate devm action to undo what is done in | ||
| 473 | * proxy start, we call it from here, but only if the start has | ||
| 474 | * completed successfully (tracked with the 'started' flag). | ||
| 475 | */ | ||
| 476 | if (gsc->proxy.started) | ||
| 477 | xe_gsc_proxy_stop(gsc); | ||
| 458 | 478 | ||
| 459 | component_del(xe->drm.dev, &xe_gsc_proxy_component_ops); | 479 | component_del(xe->drm.dev, &xe_gsc_proxy_component_ops); |
| 460 | gsc->proxy.component_added = false; | 480 | gsc->proxy.component_added = false; |
| @@ -510,6 +530,7 @@ int xe_gsc_proxy_init(struct xe_gsc *gsc) · 49 unmodified lines | |||
| 510 | */ | 530 | */ |
| 511 | int xe_gsc_proxy_start(struct xe_gsc *gsc) | 531 | int xe_gsc_proxy_start(struct xe_gsc *gsc) |
| 512 | { | 532 | { |
| 533 | struct xe_gt *gt = gsc_to_gt(gsc); | ||
| 513 | int err; | 534 | int err; |
| 514 | 535 | ||
| 515 | /* enable the proxy interrupt in the GSC shim layer */ | 536 | /* enable the proxy interrupt in the GSC shim layer */ |
| @@ -521,12 +542,18 @@ int xe_gsc_proxy_start(struct xe_gsc *gsc) · 5 unmodified lines | |||
| 521 | */ | 542 | */ |
| 522 | err = xe_gsc_proxy_request_handler(gsc); | 543 | err = xe_gsc_proxy_request_handler(gsc); |
| 523 | if (err) | 544 | if (err) |
| 524 | return err; | 545 | goto err_irq_disable; |
| 525 | 546 | ||
| 526 | if (!xe_gsc_proxy_init_done(gsc)) { | 547 | if (!xe_gsc_proxy_init_done(gsc)) { |
| 527 | xe_gt_err(gsc_to_gt(gsc), "GSC FW reports proxy init not completed\n"); | 548 | xe_gt_err(gt, "GSC FW reports proxy init not completed\n"); |
| 528 | return -EIO; | 549 | err = -EIO; |
| 550 | goto err_irq_disable; | ||
| 529 | } | 551 | } |
| 530 | 552 | ||
| 553 | gsc->proxy.started = true; | ||
| 531 | return 0; | 554 | return 0; |
| 555 | |||
| 556 | err_irq_disable: | ||
| 557 | gsc_proxy_irq_toggle(gsc, false); | ||
| 558 | return err; | ||
| 532 | } | 559 | } |
| @@ -58,6 +58,8 @@ struct xe_gsc { · 57 unmodified lines | |||
| 58 | struct mutex mutex; | 58 | struct mutex mutex; |
| 59 | /** @proxy.component_added: whether the component has been added */ | 59 | /** @proxy.component_added: whether the component has been added */ |
| 60 | bool component_added; | 60 | bool component_added; |
| 61 | /** @proxy.started: whether the proxy has been started */ | ||
| 62 | bool started; | ||
| 61 | /** @proxy.bo: object to store message to and from the GSC */ | 63 | /** @proxy.bo: object to store message to and from the GSC */ |
| 62 | struct xe_bo *bo; | 64 | struct xe_bo *bo; |
| 63 | /** @proxy.to_gsc: map of the memory used to send messages to the GSC */ | 65 | /** @proxy.to_gsc: map of the memory used to send messages to the GSC */ |
| @@ -210,11 +210,15 @@ static int emit_nop_job(struct xe_gt *gt, struct xe_exec_queue *q) · 209 unmodified lines | |||
| 210 | return ret; | 210 | return ret; |
| 211 | } | 211 | } |
| 212 | 212 | ||
| 213 | /* Dwords required to emit a RMW of a register */ | ||
| 214 | #define EMIT_RMW_DW 20 | ||
| 215 | |||
| 213 | static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) | 216 | static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) |
| 214 | { | 217 | { |
| 215 | struct xe_reg_sr *sr = &q->hwe->reg_lrc; | 218 | struct xe_hw_engine *hwe = q->hwe; |
| 219 | struct xe_reg_sr *sr = &hwe->reg_lrc; | ||
| 216 | struct xe_reg_sr_entry *entry; | 220 | struct xe_reg_sr_entry *entry; |
| 217 | int count_rmw = 0, count = 0, ret; | 221 | int count_rmw = 0, count_rmw_mcr = 0, count = 0, ret; |
| 218 | unsigned long idx; | 222 | unsigned long idx; |
| 219 | struct xe_bb *bb; | 223 | struct xe_bb *bb; |
| 220 | size_t bb_len = 0; | 224 | size_t bb_len = 0; |
| @@ -224,6 +228,8 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 3 unmodified lines | |||
| 224 | xa_for_each(&sr->xa, idx, entry) { | 228 | xa_for_each(&sr->xa, idx, entry) { |
| 225 | if (entry->reg.masked || entry->clr_bits == ~0) | 229 | if (entry->reg.masked || entry->clr_bits == ~0) |
| 226 | ++count; | 230 | ++count; |
| 231 | else if (entry->reg.mcr) | ||
| 232 | ++count_rmw_mcr; | ||
| 227 | else | 233 | else |
| 228 | ++count_rmw; | 234 | ++count_rmw; |
| 229 | } | 235 | } |
| @@ -231,17 +237,35 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 1 unmodified line | |||
| 231 | if (count) | 237 | if (count) |
| 232 | bb_len += count * 2 + 1; | 238 | bb_len += count * 2 + 1; |
| 233 | 239 | ||
| 234 | if (count_rmw) | 240 | /* |
| 235 | bb_len += count_rmw * 20 + 7; | 241 | * RMW of MCR registers is the same as a normal RMW, except an |
| 242 | * additional LRI (3 dwords) is required per register to steer the read | ||
| 243 | * to a nom-terminated instance. | ||
| 244 | * | ||
| 245 | * We could probably shorten the batch slightly by eliding the | ||
| 246 | * steering for consecutive MCR registers that have the same | ||
| 247 | * group/instance target, but it's not worth the extra complexity to do | ||
| 248 | * so. | ||
| 249 | */ | ||
| 250 | bb_len += count_rmw * EMIT_RMW_DW; | ||
| 251 | bb_len += count_rmw_mcr * (EMIT_RMW_DW + 3); | ||
| 252 | |||
| 253 | /* | ||
| 254 | * After doing all RMW, we need 7 trailing dwords to clean up, | ||
| 255 | * plus an additional 3 dwords to reset steering if any of the | ||
| 256 | * registers were MCR. | ||
| 257 | */ | ||
| 258 | if (count_rmw || count_rmw_mcr) | ||
| 259 | bb_len += 7 + (count_rmw_mcr ? 3 : 0); | ||
| 236 | 260 | ||
| 237 | if (q->hwe->class == XE_ENGINE_CLASS_RENDER) | 261 | if (hwe->class == XE_ENGINE_CLASS_RENDER) |
| 238 | /* | 262 | /* |
| 239 | * Big enough to emit all of the context's 3DSTATE via | 263 | * Big enough to emit all of the context's 3DSTATE via |
| 240 | * xe_lrc_emit_hwe_state_instructions() | 264 | * xe_lrc_emit_hwe_state_instructions() |
| 241 | */ | 265 | */ |
| 242 | bb_len += xe_gt_lrc_size(gt, q->hwe->class) / sizeof(u32); | 266 | bb_len += xe_gt_lrc_size(gt, hwe->class) / sizeof(u32); |
| 243 | 267 | ||
| 244 | xe_gt_dbg(gt, "LRC %s WA job: %zu dwords\n", q->hwe->name, bb_len); | 268 | xe_gt_dbg(gt, "LRC %s WA job: %zu dwords\n", hwe->name, bb_len); |
| 245 | 269 | ||
| 246 | bb = xe_bb_new(gt, bb_len, false); | 270 | bb = xe_bb_new(gt, bb_len, false); |
| 247 | if (IS_ERR(bb)) | 271 | if (IS_ERR(bb)) |
| @@ -276,13 +300,23 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 28 unmodified lines | |||
| 276 | } | 300 | } |
| 277 | } | 301 | } |
| 278 | 302 | ||
| 279 | if (count_rmw) { | 303 | if (count_rmw || count_rmw_mcr) { |
| 280 | /* Emit MI_MATH for each RMW reg: 20dw per reg + 7 trailing dw */ | ||
| 281 | |||
| 282 | xa_for_each(&sr->xa, idx, entry) { | 304 | xa_for_each(&sr->xa, idx, entry) { |
| 283 | if (entry->reg.masked || entry->clr_bits == ~0) | 305 | if (entry->reg.masked || entry->clr_bits == ~0) |
| 284 | continue; | 306 | continue; |
| 285 | 307 | ||
| 308 | if (entry->reg.mcr) { | ||
| 309 | struct xe_reg_mcr reg = { .__reg.raw = entry->reg.raw }; | ||
| 310 | u8 group, instance; | ||
| 311 | |||
| 312 | xe_gt_mcr_get_nonterminated_steering(gt, reg, &group, &instance); | ||
| 313 | *cs++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | ||
| 314 | *cs++ = CS_MMIO_GROUP_INSTANCE_SELECT(hwe->mmio_base).addr; | ||
| 315 | *cs++ = SELECTIVE_READ_ADDRESSING | | ||
| 316 | REG_FIELD_PREP(SELECTIVE_READ_GROUP, group) | | ||
| 317 | REG_FIELD_PREP(SELECTIVE_READ_INSTANCE, instance); | ||
| 318 | } | ||
| 319 | |||
| 286 | *cs++ = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO; | 320 | *cs++ = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO; |
| 287 | *cs++ = entry->reg.addr; | 321 | *cs++ = entry->reg.addr; |
| 288 | *cs++ = CS_GPR_REG(0, 0).addr; | 322 | *cs++ = CS_GPR_REG(0, 0).addr; |
| @@ -308,8 +342,9 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 19 unmodified lines | |||
| 308 | *cs++ = CS_GPR_REG(0, 0).addr; | 342 | *cs++ = CS_GPR_REG(0, 0).addr; |
| 309 | *cs++ = entry->reg.addr; | 343 | *cs++ = entry->reg.addr; |
| 310 | 344 | ||
| 311 | xe_gt_dbg(gt, "REG[%#x] = ~%#x|%#x\n", | 345 | xe_gt_dbg(gt, "REG[%#x] = ~%#x|%#x%s\n", |
| 312 | entry->reg.addr, entry->clr_bits, entry->set_bits); | 346 | entry->reg.addr, entry->clr_bits, entry->set_bits, |
| 347 | entry->reg.mcr ? " (MCR)" : ""); | ||
| 313 | } | 348 | } |
| 314 | 349 | ||
| 315 | /* reset used GPR */ | 350 | /* reset used GPR */ |
| @@ -321,6 +356,13 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 5 unmodified lines | |||
| 321 | *cs++ = 0; | 356 | *cs++ = 0; |
| 322 | *cs++ = CS_GPR_REG(0, 2).addr; | 357 | *cs++ = CS_GPR_REG(0, 2).addr; |
| 323 | *cs++ = 0; | 358 | *cs++ = 0; |
| 359 | |||
| 360 | /* reset steering */ | ||
| 361 | if (count_rmw_mcr) { | ||
| 362 | *cs++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | ||
| 363 | *cs++ = CS_MMIO_GROUP_INSTANCE_SELECT(q->hwe->mmio_base).addr; | ||
| 364 | *cs++ = 0; | ||
| 365 | } | ||
| 324 | } | 366 | } |
| 325 | 367 | ||
| 326 | cs = xe_lrc_emit_hwe_state_instructions(q, cs); | 368 | cs = xe_lrc_emit_hwe_state_instructions(q, cs); |
| @@ -75,7 +75,8 @@ static inline struct xe_lrc *xe_lrc_get(struct xe_lrc *lrc) · 74 unmodified lines | |||
| 75 | */ | 75 | */ |
| 76 | static inline void xe_lrc_put(struct xe_lrc *lrc) | 76 | static inline void xe_lrc_put(struct xe_lrc *lrc) |
| 77 | { | 77 | { |
| 78 | kref_put(&lrc->refcount, xe_lrc_destroy); | 78 | if (lrc) |
| 79 | kref_put(&lrc->refcount, xe_lrc_destroy); | ||
| 79 | } | 80 | } |
| 80 | 81 | ||
| 81 | /** | 82 | /** |
| @@ -98,10 +98,12 @@ int xe_reg_sr_add(struct xe_reg_sr *sr, · 97 unmodified lines | |||
| 98 | *pentry = *e; | 98 | *pentry = *e; |
| 99 | ret = xa_err(xa_store(&sr->xa, idx, pentry, GFP_KERNEL)); | 99 | ret = xa_err(xa_store(&sr->xa, idx, pentry, GFP_KERNEL)); |
| 100 | if (ret) | 100 | if (ret) |
| 101 | goto fail; | 101 | goto fail_free; |
| 102 | 102 | ||
| 103 | return 0; | 103 | return 0; |
| 104 | 104 | ||
| 105 | fail_free: | ||
| 106 | kfree(pentry); | ||
| 105 | fail: | 107 | fail: |
| 106 | xe_gt_err(gt, | 108 | xe_gt_err(gt, |
| 107 | "discarding save-restore reg %04lx (clear: %08x, set: %08x, masked: %s, mcr: %s): ret=%d\n", | 109 | "discarding save-restore reg %04lx (clear: %08x, set: %08x, masked: %s, mcr: %s): ret=%d\n", |
| @@ -280,6 +280,9 @@ static void __emit_job_gen12_simple(struct xe_sched_job *job, struct xe_lrc *lrc · 279 unmodified lines | |||
| 280 | 280 | ||
| 281 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); | 281 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 282 | 282 | ||
| 283 | /* Don't preempt fence signaling */ | ||
| 284 | dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | ||
| 285 | |||
| 283 | if (job->user_fence.used) { | 286 | if (job->user_fence.used) { |
| 284 | i = emit_flush_dw(dw, i); | 287 | i = emit_flush_dw(dw, i); |
| 285 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, | 288 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, |
| @@ -345,6 +348,9 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc, · 59 unmodified lines | |||
| 345 | 348 | ||
| 346 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); | 349 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 347 | 350 | ||
| 351 | /* Don't preempt fence signaling */ | ||
| 352 | dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | ||
| 353 | |||
| 348 | if (job->user_fence.used) { | 354 | if (job->user_fence.used) { |
| 349 | i = emit_flush_dw(dw, i); | 355 | i = emit_flush_dw(dw, i); |
| 350 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, | 356 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, |
| @@ -397,6 +403,9 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job, · 46 unmodified lines | |||
| 397 | 403 | ||
| 398 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); | 404 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 399 | 405 | ||
| 406 | /* Don't preempt fence signaling */ | ||
| 407 | dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | ||
| 408 | |||
| 400 | i = emit_render_cache_flush(job, dw, i); | 409 | i = emit_render_cache_flush(job, dw, i); |
| 401 | 410 | ||
| 402 | if (job->user_fence.used) | 411 | if (job->user_fence.used) |
| @@ -146,8 +146,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 145 unmodified lines | |||
| 146 | 146 | ||
| 147 | if (!signal) { | 147 | if (!signal) { |
| 148 | sync->fence = drm_syncobj_fence_get(sync->syncobj); | 148 | sync->fence = drm_syncobj_fence_get(sync->syncobj); |
| 149 | if (XE_IOCTL_DBG(xe, !sync->fence)) | 149 | if (XE_IOCTL_DBG(xe, !sync->fence)) { |
| 150 | return -EINVAL; | 150 | err = -EINVAL; |
| 151 | goto free_sync; | ||
| 152 | } | ||
| 151 | } | 153 | } |
| 152 | break; | 154 | break; |
| 153 | 155 | ||
| @@ -167,17 +169,21 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 13 unmodified lines | |||
| 167 | 169 | ||
| 168 | if (signal) { | 170 | if (signal) { |
| 169 | sync->chain_fence = dma_fence_chain_alloc(); | 171 | sync->chain_fence = dma_fence_chain_alloc(); |
| 170 | if (!sync->chain_fence) | 172 | if (!sync->chain_fence) { |
| 171 | return -ENOMEM; | 173 | err = -ENOMEM; |
| 174 | goto free_sync; | ||
| 175 | } | ||
| 172 | } else { | 176 | } else { |
| 173 | sync->fence = drm_syncobj_fence_get(sync->syncobj); | 177 | sync->fence = drm_syncobj_fence_get(sync->syncobj); |
| 174 | if (XE_IOCTL_DBG(xe, !sync->fence)) | 178 | if (XE_IOCTL_DBG(xe, !sync->fence)) { |
| 175 | return -EINVAL; | 179 | err = -EINVAL; |
| 180 | goto free_sync; | ||
| 181 | } | ||
| 176 | 182 | ||
| 177 | err = dma_fence_chain_find_seqno(&sync->fence, | 183 | err = dma_fence_chain_find_seqno(&sync->fence, |
| 178 | sync_in.timeline_value); | 184 | sync_in.timeline_value); |
| 179 | if (err) | 185 | if (err) |
| 180 | return err; | 186 | goto free_sync; |
| 181 | } | 187 | } |
| 182 | break; | 188 | break; |
| 183 | 189 | ||
| @@ -200,8 +206,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 16 unmodified lines | |||
| 200 | if (XE_IOCTL_DBG(xe, IS_ERR(sync->ufence))) | 206 | if (XE_IOCTL_DBG(xe, IS_ERR(sync->ufence))) |
| 201 | return PTR_ERR(sync->ufence); | 207 | return PTR_ERR(sync->ufence); |
| 202 | sync->ufence_chain_fence = dma_fence_chain_alloc(); | 208 | sync->ufence_chain_fence = dma_fence_chain_alloc(); |
| 203 | if (!sync->ufence_chain_fence) | 209 | if (!sync->ufence_chain_fence) { |
| 204 | return -ENOMEM; | 210 | err = -ENOMEM; |
| 211 | goto free_sync; | ||
| 212 | } | ||
| 205 | sync->ufence_syncobj = ufence_syncobj; | 213 | sync->ufence_syncobj = ufence_syncobj; |
| 206 | } | 214 | } |
| 207 | 215 | ||
| @@ -216,6 +224,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 8 unmodified lines | |||
| 216 | sync->timeline_value = sync_in.timeline_value; | 224 | sync->timeline_value = sync_in.timeline_value; |
| 217 | 225 | ||
| 218 | return 0; | 226 | return 0; |
| 227 | |||
| 228 | free_sync: | ||
| 229 | xe_sync_entry_cleanup(sync); | ||
| 230 | return err; | ||
| 219 | } | 231 | } |
| 220 | ALLOW_ERROR_INJECTION(xe_sync_entry_parse, ERRNO); | 232 | ALLOW_ERROR_INJECTION(xe_sync_entry_parse, ERRNO); |
| 221 | 233 | ||
| @@ -453,7 +453,7 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 452 unmodified lines | |||
| 453 | madvise_range.num_vmas, | 453 | madvise_range.num_vmas, |
| 454 | args->atomic.val)) { | 454 | args->atomic.val)) { |
| 455 | err = -EINVAL; | 455 | err = -EINVAL; |
| 456 | goto madv_fini; | 456 | goto free_vmas; |
| 457 | } | 457 | } |
| 458 | } | 458 | } |
| 459 | 459 | ||
| @@ -490,6 +490,7 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 30 unmodified lines | |||
| 490 | err_fini: | 490 | err_fini: |
| 491 | if (madvise_range.has_bo_vmas) | 491 | if (madvise_range.has_bo_vmas) |
| 492 | drm_exec_fini(&exec); | 492 | drm_exec_fini(&exec); |
| 493 | free_vmas: | ||
| 493 | kfree(madvise_range.vmas); | 494 | kfree(madvise_range.vmas); |
| 494 | madvise_range.vmas = NULL; | 495 | madvise_range.vmas = NULL; |
| 495 | madv_fini: | 496 | madv_fini: |
| @@ -241,12 +241,13 @@ static const struct xe_rtp_entry_sr gt_was[] = { · 240 unmodified lines | |||
| 241 | 241 | ||
| 242 | { XE_RTP_NAME("16025250150"), | 242 | { XE_RTP_NAME("16025250150"), |
| 243 | XE_RTP_RULES(GRAPHICS_VERSION(2001)), | 243 | XE_RTP_RULES(GRAPHICS_VERSION(2001)), |
| 244 | XE_RTP_ACTIONS(SET(LSN_VC_REG2, | 244 | XE_RTP_ACTIONS(FIELD_SET(LSN_VC_REG2, |
| 245 | LSN_LNI_WGT(1) | | 245 | LSN_LNI_WGT_MASK | LSN_LNE_WGT_MASK | |
| 246 | LSN_LNE_WGT(1) | | 246 | LSN_DIM_X_WGT_MASK | LSN_DIM_Y_WGT_MASK | |
| 247 | LSN_DIM_X_WGT(1) | | 247 | LSN_DIM_Z_WGT_MASK, |
| 248 | LSN_DIM_Y_WGT(1) | | 248 | LSN_LNI_WGT(1) | LSN_LNE_WGT(1) | |
| 249 | LSN_DIM_Z_WGT(1))) | 249 | LSN_DIM_X_WGT(1) | LSN_DIM_Y_WGT(1) | |
| 250 | LSN_DIM_Z_WGT(1))) | ||
| 250 | }, | 251 | }, |
| 251 | 252 | ||
| 252 | /* Xe2_HPM */ | 253 | /* Xe2_HPM */ |
| @@ -47,16 +47,12 @@ struct PteArray<const NUM_ENTRIES: usize>([u64; NUM_ENTRIES]); · 46 unmodified lines | |||
| 47 | unsafe impl<const NUM_ENTRIES: usize> AsBytes for PteArray<NUM_ENTRIES> {} | 47 | unsafe impl<const NUM_ENTRIES: usize> AsBytes for PteArray<NUM_ENTRIES> {} |
| 48 | 48 | ||
| 49 | impl<const NUM_PAGES: usize> PteArray<NUM_PAGES> { | 49 | impl<const NUM_PAGES: usize> PteArray<NUM_PAGES> { |
| 50 | /// Creates a new page table array mapping `NUM_PAGES` GSP pages starting at address `start`. | 50 | /// Returns the page table entry for `index`, for a mapping starting at `start`. |
| 51 | fn new(start: DmaAddress) -> Result<Self> { | 51 | // TODO: Replace with `IoView` projection once available. |
| 52 | let mut ptes = [0u64; NUM_PAGES]; | 52 | fn entry(start: DmaAddress, index: usize) -> Result<u64> { |
| 53 | for (i, pte) in ptes.iter_mut().enumerate() { | 53 | start |
| 54 | *pte = start | 54 | .checked_add(num::usize_as_u64(index) << GSP_PAGE_SHIFT) |
| 55 | .checked_add(num::usize_as_u64(i) << GSP_PAGE_SHIFT) | 55 | .ok_or(EOVERFLOW) |
| 56 | .ok_or(EOVERFLOW)?; | ||
| 57 | } | ||
| 58 | |||
| 59 | Ok(Self(ptes)) | ||
| 60 | } | 56 | } |
| 61 | } | 57 | } |
| 62 | 58 | ||
| @@ -86,16 +82,22 @@ impl LogBuffer { · 23 unmodified lines | |||
| 86 | NUM_PAGES * GSP_PAGE_SIZE, | 82 | NUM_PAGES * GSP_PAGE_SIZE, |
| 87 | GFP_KERNEL | __GFP_ZERO, | 83 | GFP_KERNEL | __GFP_ZERO, |
| 88 | )?); | 84 | )?); |
| 89 | let ptes = PteArray::<NUM_PAGES>::new(obj.0.dma_handle())?; | 85 | |
| 86 | let start_addr = obj.0.dma_handle(); | ||
| 90 | 87 | ||
| 91 | // SAFETY: `obj` has just been created and we are its sole user. | 88 | // SAFETY: `obj` has just been created and we are its sole user. |
| 92 | unsafe { | 89 | let pte_region = unsafe { |
| 93 | // Copy the self-mapping PTE at the expected location. | ||
| 94 | obj.0 | 90 | obj.0 |
| 95 | .as_slice_mut(size_of::<u64>(), size_of_val(&ptes))? | 91 | .as_slice_mut(size_of::<u64>(), NUM_PAGES * size_of::<u64>())? |
| 96 | .copy_from_slice(ptes.as_bytes()) | ||
| 97 | }; | 92 | }; |
| 98 | 93 | ||
| 94 | // Write values one by one to avoid an on-stack instance of `PteArray`. | ||
| 95 | for (i, chunk) in pte_region.chunks_exact_mut(size_of::<u64>()).enumerate() { | ||
| 96 | let pte_value = PteArray::<0>::entry(start_addr, i)?; | ||
| 97 | |||
| 98 | chunk.copy_from_slice(&pte_value.to_ne_bytes()); | ||
| 99 | } | ||
| 100 | |||
| 99 | Ok(obj) | 101 | Ok(obj) |
| 100 | } | 102 | } |
| 101 | } | 103 | } |
| @@ -143,14 +145,14 @@ impl Gsp { · 41 unmodified lines | |||
| 143 | // _kgspInitLibosLoggingStructures (allocates memory for buffers) | 145 | // _kgspInitLibosLoggingStructures (allocates memory for buffers) |
| 144 | // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array) | 146 | // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array) |
| 145 | dma_write!( | 147 | dma_write!( |
| 146 | libos[0] = LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0) | 148 | libos, [0]?, LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0) |
| 147 | )?; | 149 | ); |
| 148 | dma_write!( | 150 | dma_write!( |
| 149 | libos[1] = LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0) | 151 | libos, [1]?, LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0) |
| 150 | )?; | 152 | ); |
| 151 | dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0))?; | 153 | dma_write!(libos, [2]?, LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)); |
| 152 | dma_write!(rmargs[0].inner = fw::GspArgumentsCached::new(cmdq))?; | 154 | dma_write!(rmargs, [0]?.inner, fw::GspArgumentsCached::new(cmdq)); |
| 153 | dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", rmargs))?; | 155 | dma_write!(libos, [3]?, LibosMemoryRegionInitArgument::new("RMARGS", rmargs)); |
| 154 | }, | 156 | }, |
| 155 | })) | 157 | })) |
| 156 | }) | 158 | }) |
| @@ -157,7 +157,7 @@ impl super::Gsp { · 156 unmodified lines | |||
| 157 | 157 | ||
| 158 | let wpr_meta = | 158 | let wpr_meta = |
| 159 | CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; | 159 | CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; |
| 160 | dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?; | 160 | dma_write!(wpr_meta, [0]?, GspFwWprMeta::new(&gsp_fw, &fb_layout)); |
| 161 | 161 | ||
| 162 | self.cmdq | 162 | self.cmdq |
| 163 | .send_command(bar, commands::SetSystemInfo::new(pdev))?; | 163 | .send_command(bar, commands::SetSystemInfo::new(pdev))?; |
| @@ -2,11 +2,7 @@ · 1 unmodified line | |||
| 2 | 2 | ||
| 3 | use core::{ | 3 | use core::{ |
| 4 | cmp, | 4 | cmp, |
| 5 | mem, | 5 | mem, // |
| 6 | sync::atomic::{ | ||
| 7 | fence, | ||
| 8 | Ordering, // | ||
| 9 | }, // | ||
| 10 | }; | 6 | }; |
| 11 | 7 | ||
| 12 | use kernel::{ | 8 | use kernel::{ |
| @@ -146,30 +142,36 @@ static_assert!(align_of::<MsgqData>() == GSP_PAGE_SIZE); · 133 unmodified lines | |||
| 146 | #[repr(C)] | 142 | #[repr(C)] |
| 147 | // There is no struct defined for this in the open-gpu-kernel-source headers. | 143 | // There is no struct defined for this in the open-gpu-kernel-source headers. |
| 148 | // Instead it is defined by code in `GspMsgQueuesInit()`. | 144 | // Instead it is defined by code in `GspMsgQueuesInit()`. |
| 149 | struct Msgq { | 145 | // TODO: Revert to private once `IoView` projections replace the `gsp_mem` module. |
| 146 | pub(super) struct Msgq { | ||
| 150 | /// Header for sending messages, including the write pointer. | 147 | /// Header for sending messages, including the write pointer. |
| 151 | tx: MsgqTxHeader, | 148 | pub(super) tx: MsgqTxHeader, |
| 152 | /// Header for receiving messages, including the read pointer. | 149 | /// Header for receiving messages, including the read pointer. |
| 153 | rx: MsgqRxHeader, | 150 | pub(super) rx: MsgqRxHeader, |
| 154 | /// The message queue proper. | 151 | /// The message queue proper. |
| 155 | msgq: MsgqData, | 152 | msgq: MsgqData, |
| 156 | } | 153 | } |
| 157 | 154 | ||
| 158 | /// Structure shared between the driver and the GSP and containing the command and message queues. | 155 | /// Structure shared between the driver and the GSP and containing the command and message queues. |
| 159 | #[repr(C)] | 156 | #[repr(C)] |
| 160 | struct GspMem { | 157 | // TODO: Revert to private once `IoView` projections replace the `gsp_mem` module. |
| 158 | pub(super) struct GspMem { | ||
| 161 | /// Self-mapping page table entries. | 159 | /// Self-mapping page table entries. |
| 162 | ptes: PteArray<{ GSP_PAGE_SIZE / size_of::<u64>() }>, | 160 | ptes: PteArray<{ Self::PTE_ARRAY_SIZE }>, |
| 163 | /// CPU queue: the driver writes commands here, and the GSP reads them. It also contains the | 161 | /// CPU queue: the driver writes commands here, and the GSP reads them. It also contains the |
| 164 | /// write and read pointers that the CPU updates. | 162 | /// write and read pointers that the CPU updates. |
| 165 | /// | 163 | /// |
| 166 | /// This member is read-only for the GSP. | 164 | /// This member is read-only for the GSP. |
| 167 | cpuq: Msgq, | 165 | pub(super) cpuq: Msgq, |
| 168 | /// GSP queue: the GSP writes messages here, and the driver reads them. It also contains the | 166 | /// GSP queue: the GSP writes messages here, and the driver reads them. It also contains the |
| 169 | /// write and read pointers that the GSP updates. | 167 | /// write and read pointers that the GSP updates. |
| 170 | /// | 168 | /// |
| 171 | /// This member is read-only for the driver. | 169 | /// This member is read-only for the driver. |
| 172 | gspq: Msgq, | 170 | pub(super) gspq: Msgq, |
| 171 | } | ||
| 172 | |||
| 173 | impl GspMem { | ||
| 174 | const PTE_ARRAY_SIZE: usize = GSP_PAGE_SIZE / size_of::<u64>(); | ||
| 173 | } | 175 | } |
| 174 | 176 | ||
| 175 | // SAFETY: These structs don't meet the no-padding requirements of AsBytes but | 177 | // SAFETY: These structs don't meet the no-padding requirements of AsBytes but |
| @@ -201,9 +203,19 @@ impl DmaGspMem { · 25 unmodified lines | |||
| 201 | 203 | ||
| 202 | let gsp_mem = | 204 | let gsp_mem = |
| 203 | CoherentAllocation::<GspMem>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; | 205 | CoherentAllocation::<GspMem>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; |
| 204 | dma_write!(gsp_mem[0].ptes = PteArray::new(gsp_mem.dma_handle())?)?; | 206 | |
| 205 | dma_write!(gsp_mem[0].cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES))?; | 207 | let start = gsp_mem.dma_handle(); |
| 206 | dma_write!(gsp_mem[0].cpuq.rx = MsgqRxHeader::new())?; | 208 | // Write values one by one to avoid an on-stack instance of `PteArray`. |
| 209 | for i in 0..GspMem::PTE_ARRAY_SIZE { | ||
| 210 | dma_write!(gsp_mem, [0]?.ptes.0[i], PteArray::<0>::entry(start, i)?); | ||
| 211 | } | ||
| 212 | |||
| 213 | dma_write!( | ||
| 214 | gsp_mem, | ||
| 215 | [0]?.cpuq.tx, | ||
| 216 | MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES) | ||
| 217 | ); | ||
| 218 | dma_write!(gsp_mem, [0]?.cpuq.rx, MsgqRxHeader::new()); | ||
| 207 | 219 | ||
| 208 | Ok(Self(gsp_mem)) | 220 | Ok(Self(gsp_mem)) |
| 209 | } | 221 | } |
| @@ -317,12 +329,7 @@ impl DmaGspMem { · 107 unmodified lines | |||
| 317 | // | 329 | // |
| 318 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. | 330 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 319 | fn gsp_write_ptr(&self) -> u32 { | 331 | fn gsp_write_ptr(&self) -> u32 { |
| 320 | let gsp_mem = self.0.start_ptr(); | 332 | super::fw::gsp_mem::gsp_write_ptr(&self.0) |
| 321 | |||
| 322 | // SAFETY: | ||
| 323 | // - The 'CoherentAllocation' contains at least one object. | ||
| 324 | // - By the invariants of `CoherentAllocation` the pointer is valid. | ||
| 325 | (unsafe { (*gsp_mem).gspq.tx.write_ptr() } % MSGQ_NUM_PAGES) | ||
| 326 | } | 333 | } |
| 327 | 334 | ||
| 328 | // Returns the index of the memory page the GSP will read the next command from. | 335 | // Returns the index of the memory page the GSP will read the next command from. |
| @@ -331,12 +338,7 @@ impl DmaGspMem { · 2 unmodified lines | |||
| 331 | // | 338 | // |
| 332 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. | 339 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 333 | fn gsp_read_ptr(&self) -> u32 { | 340 | fn gsp_read_ptr(&self) -> u32 { |
| 334 | let gsp_mem = self.0.start_ptr(); | 341 | super::fw::gsp_mem::gsp_read_ptr(&self.0) |
| 335 | |||
| 336 | // SAFETY: | ||
| 337 | // - The 'CoherentAllocation' contains at least one object. | ||
| 338 | // - By the invariants of `CoherentAllocation` the pointer is valid. | ||
| 339 | (unsafe { (*gsp_mem).gspq.rx.read_ptr() } % MSGQ_NUM_PAGES) | ||
| 340 | } | 342 | } |
| 341 | 343 | ||
| 342 | // Returns the index of the memory page the CPU can read the next message from. | 344 | // Returns the index of the memory page the CPU can read the next message from. |
| @@ -345,27 +347,12 @@ impl DmaGspMem { · 2 unmodified lines | |||
| 345 | // | 347 | // |
| 346 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. | 348 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 347 | fn cpu_read_ptr(&self) -> u32 { | 349 | fn cpu_read_ptr(&self) -> u32 { |
| 348 | let gsp_mem = self.0.start_ptr(); | 350 | super::fw::gsp_mem::cpu_read_ptr(&self.0) |
| 349 | |||
| 350 | // SAFETY: | ||
| 351 | // - The ['CoherentAllocation'] contains at least one object. | ||
| 352 | // - By the invariants of CoherentAllocation the pointer is valid. | ||
| 353 | (unsafe { (*gsp_mem).cpuq.rx.read_ptr() } % MSGQ_NUM_PAGES) | ||
| 354 | } | 351 | } |
| 355 | 352 | ||
| 356 | // Informs the GSP that it can send `elem_count` new pages into the message queue. | 353 | // Informs the GSP that it can send `elem_count` new pages into the message queue. |
| 357 | fn advance_cpu_read_ptr(&mut self, elem_count: u32) { | 354 | fn advance_cpu_read_ptr(&mut self, elem_count: u32) { |
| 358 | let rptr = self.cpu_read_ptr().wrapping_add(elem_count) % MSGQ_NUM_PAGES; | 355 | super::fw::gsp_mem::advance_cpu_read_ptr(&self.0, elem_count) |
| 359 | |||
| 360 | // Ensure read pointer is properly ordered. | ||
| 361 | fence(Ordering::SeqCst); | ||
| 362 | |||
| 363 | let gsp_mem = self.0.start_ptr_mut(); | ||
| 364 | |||
| 365 | // SAFETY: | ||
| 366 | // - The 'CoherentAllocation' contains at least one object. | ||
| 367 | // - By the invariants of `CoherentAllocation` the pointer is valid. | ||
| 368 | unsafe { (*gsp_mem).cpuq.rx.set_read_ptr(rptr) }; | ||
| 369 | } | 356 | } |
| 370 | 357 | ||
| 371 | // Returns the index of the memory page the CPU can write the next command to. | 358 | // Returns the index of the memory page the CPU can write the next command to. |
| @@ -374,26 +361,12 @@ impl DmaGspMem { · 2 unmodified lines | |||
| 374 | // | 361 | // |
| 375 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. | 362 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 376 | fn cpu_write_ptr(&self) -> u32 { | 363 | fn cpu_write_ptr(&self) -> u32 { |
| 377 | let gsp_mem = self.0.start_ptr(); | 364 | super::fw::gsp_mem::cpu_write_ptr(&self.0) |
| 378 | |||
| 379 | // SAFETY: | ||
| 380 | // - The 'CoherentAllocation' contains at least one object. | ||
| 381 | // - By the invariants of `CoherentAllocation` the pointer is valid. | ||
| 382 | (unsafe { (*gsp_mem).cpuq.tx.write_ptr() } % MSGQ_NUM_PAGES) | ||
| 383 | } | 365 | } |
| 384 | 366 | ||
| 385 | // Informs the GSP that it can process `elem_count` new pages from the command queue. | 367 | // Informs the GSP that it can process `elem_count` new pages from the command queue. |
| 386 | fn advance_cpu_write_ptr(&mut self, elem_count: u32) { | 368 | fn advance_cpu_write_ptr(&mut self, elem_count: u32) { |
| 387 | let wptr = self.cpu_write_ptr().wrapping_add(elem_count) & MSGQ_NUM_PAGES; | 369 | super::fw::gsp_mem::advance_cpu_write_ptr(&self.0, elem_count) |
| 388 | let gsp_mem = self.0.start_ptr_mut(); | ||
| 389 | |||
| 390 | // SAFETY: | ||
| 391 | // - The 'CoherentAllocation' contains at least one object. | ||
| 392 | // - By the invariants of `CoherentAllocation` the pointer is valid. | ||
| 393 | unsafe { (*gsp_mem).cpuq.tx.set_write_ptr(wptr) }; | ||
| 394 | |||
| 395 | // Ensure all command data is visible before triggering the GSP read. | ||
| 396 | fence(Ordering::SeqCst); | ||
| 397 | } | 370 | } |
| 398 | } | 371 | } |
| 399 | 372 | ||
| @@ -40,6 +40,75 @@ use crate::{ · 39 unmodified lines | |||
| 40 | }, | 40 | }, |
| 41 | }; | 41 | }; |
| 42 | 42 | ||
| 43 | // TODO: Replace with `IoView` projections once available; the `unwrap()` calls go away once we | ||
| 44 | // switch to the new `dma::Coherent` API. | ||
| 45 | pub(super) mod gsp_mem { | ||
| 46 | use core::sync::atomic::{ | ||
| 47 | fence, | ||
| 48 | Ordering, // | ||
| 49 | }; | ||
| 50 | |||
| 51 | use kernel::{ | ||
| 52 | dma::CoherentAllocation, | ||
| 53 | dma_read, | ||
| 54 | dma_write, | ||
| 55 | prelude::*, // | ||
| 56 | }; | ||
| 57 | |||
| 58 | use crate::gsp::cmdq::{ | ||
| 59 | GspMem, | ||
| 60 | MSGQ_NUM_PAGES, // | ||
| 61 | }; | ||
| 62 | |||
| 63 | pub(in crate::gsp) fn gsp_write_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | ||
| 64 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 65 | || -> Result<u32> { Ok(dma_read!(qs, [0]?.gspq.tx.0.writePtr) % MSGQ_NUM_PAGES) }().unwrap() | ||
| 66 | } | ||
| 67 | |||
| 68 | pub(in crate::gsp) fn gsp_read_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | ||
| 69 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 70 | || -> Result<u32> { Ok(dma_read!(qs, [0]?.gspq.rx.0.readPtr) % MSGQ_NUM_PAGES) }().unwrap() | ||
| 71 | } | ||
| 72 | |||
| 73 | pub(in crate::gsp) fn cpu_read_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | ||
| 74 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 75 | || -> Result<u32> { Ok(dma_read!(qs, [0]?.cpuq.rx.0.readPtr) % MSGQ_NUM_PAGES) }().unwrap() | ||
| 76 | } | ||
| 77 | |||
| 78 | pub(in crate::gsp) fn advance_cpu_read_ptr(qs: &CoherentAllocation<GspMem>, count: u32) { | ||
| 79 | let rptr = cpu_read_ptr(qs).wrapping_add(count) % MSGQ_NUM_PAGES; | ||
| 80 | |||
| 81 | // Ensure read pointer is properly ordered. | ||
| 82 | fence(Ordering::SeqCst); | ||
| 83 | |||
| 84 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 85 | || -> Result { | ||
| 86 | dma_write!(qs, [0]?.cpuq.rx.0.readPtr, rptr); | ||
| 87 | Ok(()) | ||
| 88 | }() | ||
| 89 | .unwrap() | ||
| 90 | } | ||
| 91 | |||
| 92 | pub(in crate::gsp) fn cpu_write_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | ||
| 93 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 94 | || -> Result<u32> { Ok(dma_read!(qs, [0]?.cpuq.tx.0.writePtr) % MSGQ_NUM_PAGES) }().unwrap() | ||
| 95 | } | ||
| 96 | |||
| 97 | pub(in crate::gsp) fn advance_cpu_write_ptr(qs: &CoherentAllocation<GspMem>, count: u32) { | ||
| 98 | let wptr = cpu_write_ptr(qs).wrapping_add(count) % MSGQ_NUM_PAGES; | ||
| 99 | |||
| 100 | // PANIC: A `dma::CoherentAllocation` always contains at least one element. | ||
| 101 | || -> Result { | ||
| 102 | dma_write!(qs, [0]?.cpuq.tx.0.writePtr, wptr); | ||
| 103 | Ok(()) | ||
| 104 | }() | ||
| 105 | .unwrap(); | ||
| 106 | |||
| 107 | // Ensure all command data is visible before triggering the GSP read. | ||
| 108 | fence(Ordering::SeqCst); | ||
| 109 | } | ||
| 110 | } | ||
| 111 | |||
| 43 | /// Empty type to group methods related to heap parameters for running the GSP firmware. | 112 | /// Empty type to group methods related to heap parameters for running the GSP firmware. |
| 44 | enum GspFwHeapParams {} | 113 | enum GspFwHeapParams {} |
| 45 | 114 | ||
| @@ -708,22 +777,6 @@ impl MsgqTxHeader { · 662 unmodified lines | |||
| 708 | entryOff: num::usize_into_u32::<GSP_PAGE_SIZE>(), | 777 | entryOff: num::usize_into_u32::<GSP_PAGE_SIZE>(), |
| 709 | }) | 778 | }) |
| 710 | } | 779 | } |
| 711 | |||
| 712 | /// Returns the value of the write pointer for this queue. | ||
| 713 | pub(crate) fn write_ptr(&self) -> u32 { | ||
| 714 | let ptr = core::ptr::from_ref(&self.0.writePtr); | ||
| 715 | |||
| 716 | // SAFETY: `ptr` is a valid pointer to a `u32`. | ||
| 717 | unsafe { ptr.read_volatile() } | ||
| 718 | } | ||
| 719 | |||
| 720 | /// Sets the value of the write pointer for this queue. | ||
| 721 | pub(crate) fn set_write_ptr(&mut self, val: u32) { | ||
| 722 | let ptr = core::ptr::from_mut(&mut self.0.writePtr); | ||
| 723 | |||
| 724 | // SAFETY: `ptr` is a valid pointer to a `u32`. | ||
| 725 | unsafe { ptr.write_volatile(val) } | ||
| 726 | } | ||
| 727 | } | 780 | } |
| 728 | 781 | ||
| 729 | // SAFETY: Padding is explicit and does not contain uninitialized data. | 782 | // SAFETY: Padding is explicit and does not contain uninitialized data. |
| @@ -739,22 +792,6 @@ impl MsgqRxHeader { · 9 unmodified lines | |||
| 739 | pub(crate) fn new() -> Self { | 792 | pub(crate) fn new() -> Self { |
| 740 | Self(Default::default()) | 793 | Self(Default::default()) |
| 741 | } | 794 | } |
| 742 | |||
| 743 | /// Returns the value of the read pointer for this queue. | ||
| 744 | pub(crate) fn read_ptr(&self) -> u32 { | ||
| 745 | let ptr = core::ptr::from_ref(&self.0.readPtr); | ||
| 746 | |||
| 747 | // SAFETY: `ptr` is a valid pointer to a `u32`. | ||
| 748 | unsafe { ptr.read_volatile() } | ||
| 749 | } | ||
| 750 | |||
| 751 | /// Sets the value of the read pointer for this queue. | ||
| 752 | pub(crate) fn set_read_ptr(&mut self, val: u32) { | ||
| 753 | let ptr = core::ptr::from_mut(&mut self.0.readPtr); | ||
| 754 | |||
| 755 | // SAFETY: `ptr` is a valid pointer to a `u32`. | ||
| 756 | unsafe { ptr.write_volatile(val) } | ||
| 757 | } | ||
| 758 | } | 795 | } |
| 759 | 796 | ||
| 760 | // SAFETY: Padding is explicit and does not contain uninitialized data. | 797 | // SAFETY: Padding is explicit and does not contain uninitialized data. |
| @@ -365,6 +365,9 @@ static const struct apple_non_apple_keyboard non_apple_keyboards[] = { · 364 unmodified lines | |||
| 365 | { "A3R" }, | 365 | { "A3R" }, |
| 366 | { "hfd.cn" }, | 366 | { "hfd.cn" }, |
| 367 | { "WKB603" }, | 367 | { "WKB603" }, |
| 368 | { "TH87" }, /* EPOMAKER TH87 BT mode */ | ||
| 369 | { "HFD Epomaker TH87" }, /* EPOMAKER TH87 USB mode */ | ||
| 370 | { "2.4G Wireless Receiver" }, /* EPOMAKER TH87 dongle */ | ||
| 368 | }; | 371 | }; |
| 369 | 372 | ||
| 370 | static bool apple_is_non_apple_keyboard(struct hid_device *hdev) | 373 | static bool apple_is_non_apple_keyboard(struct hid_device *hdev) |
| @@ -686,9 +689,7 @@ static const __u8 *apple_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 315 unmodified lines | |||
| 686 | hid_info(hdev, | 689 | hid_info(hdev, |
| 687 | "fixing up Magic Keyboard battery report descriptor\n"); | 690 | "fixing up Magic Keyboard battery report descriptor\n"); |
| 688 | *rsize = *rsize - 1; | 691 | *rsize = *rsize - 1; |
| 689 | rdesc = kmemdup(rdesc + 1, *rsize, GFP_KERNEL); | 692 | rdesc = rdesc + 1; |
| 690 | if (!rdesc) | ||
| 691 | return NULL; | ||
| 692 | 693 | ||
| 693 | rdesc[0] = 0x05; | 694 | rdesc[0] = 0x05; |
| 694 | rdesc[1] = 0x01; | 695 | rdesc[1] = 0x01; |
| @@ -1399,14 +1399,21 @@ static const __u8 *asus_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 1398 unmodified lines | |||
| 1399 | */ | 1399 | */ |
| 1400 | if (*rsize == rsize_orig && | 1400 | if (*rsize == rsize_orig && |
| 1401 | rdesc[offs] == 0x09 && rdesc[offs + 1] == 0x76) { | 1401 | rdesc[offs] == 0x09 && rdesc[offs + 1] == 0x76) { |
| 1402 | *rsize = rsize_orig + 1; | 1402 | __u8 *new_rdesc; |
| 1403 | rdesc = kmemdup(rdesc, *rsize, GFP_KERNEL); | 1403 | |
| 1404 | if (!rdesc) | 1404 | new_rdesc = devm_kzalloc(&hdev->dev, rsize_orig + 1, |
| 1405 | return NULL; | 1405 | GFP_KERNEL); |
| 1406 | if (!new_rdesc) | ||
| 1407 | return rdesc; | ||
| 1406 | 1408 | ||
| 1407 | hid_info(hdev, "Fixing up %s keyb report descriptor\n", | 1409 | hid_info(hdev, "Fixing up %s keyb report descriptor\n", |
| 1408 | drvdata->quirks & QUIRK_T100CHI ? | 1410 | drvdata->quirks & QUIRK_T100CHI ? |
| 1409 | "T100CHI" : "T90CHI"); | 1411 | "T100CHI" : "T90CHI"); |
| 1412 | |||
| 1413 | memcpy(new_rdesc, rdesc, rsize_orig); | ||
| 1414 | *rsize = rsize_orig + 1; | ||
| 1415 | rdesc = new_rdesc; | ||
| 1416 | |||
| 1410 | memmove(rdesc + offs + 4, rdesc + offs + 2, 12); | 1417 | memmove(rdesc + offs + 4, rdesc + offs + 2, 12); |
| 1411 | rdesc[offs] = 0x19; | 1418 | rdesc[offs] = 0x19; |
| 1412 | rdesc[offs + 1] = 0x00; | 1419 | rdesc[offs + 1] = 0x00; |
| @@ -1490,6 +1497,9 @@ static const struct hid_device_id asus_devices[] = { · 77 unmodified lines | |||
| 1490 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, | 1497 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1491 | USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X), | 1498 | USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X), |
| 1492 | QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD | QUIRK_ROG_ALLY_XPAD }, | 1499 | QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD | QUIRK_ROG_ALLY_XPAD }, |
| 1500 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, | ||
| 1501 | USB_DEVICE_ID_ASUSTEK_XGM_2023), | ||
| 1502 | }, | ||
| 1493 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, | 1503 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1494 | USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD), | 1504 | USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD), |
| 1495 | QUIRK_ROG_CLAYMORE_II_KEYBOARD }, | 1505 | QUIRK_ROG_CLAYMORE_II_KEYBOARD }, |
| @@ -99,7 +99,7 @@ static int cmhid_raw_event(struct hid_device *hid, struct hid_report *report, · 98 unmodified lines | |||
| 99 | { | 99 | { |
| 100 | struct cmhid *cm = hid_get_drvdata(hid); | 100 | struct cmhid *cm = hid_get_drvdata(hid); |
| 101 | 101 | ||
| 102 | if (len != CM6533_JD_RAWEV_LEN) | 102 | if (len != CM6533_JD_RAWEV_LEN || !(hid->claimed & HID_CLAIMED_INPUT)) |
| 103 | goto out; | 103 | goto out; |
| 104 | if (memcmp(data+CM6533_JD_SFX_OFFSET, ji_sfx, sizeof(ji_sfx))) | 104 | if (memcmp(data+CM6533_JD_SFX_OFFSET, ji_sfx, sizeof(ji_sfx))) |
| 105 | goto out; | 105 | goto out; |
| @@ -153,7 +153,7 @@ static int creative_sb0540_raw_event(struct hid_device *hid, · 152 unmodified lines | |||
| 153 | u64 code, main_code; | 153 | u64 code, main_code; |
| 154 | int key; | 154 | int key; |
| 155 | 155 | ||
| 156 | if (len != 6) | 156 | if (len != 6 || !(hid->claimed & HID_CLAIMED_INPUT)) |
| 157 | return 0; | 157 | return 0; |
| 158 | 158 | ||
| 159 | /* From daemons/hw_hiddev.c sb0540_rec() in lirc */ | 159 | /* From daemons/hw_hiddev.c sb0540_rec() in lirc */ |
| @@ -229,6 +229,7 @@ · 228 unmodified lines | |||
| 229 | #define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X 0x1b4c | 229 | #define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X 0x1b4c |
| 230 | #define USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD 0x196b | 230 | #define USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD 0x196b |
| 231 | #define USB_DEVICE_ID_ASUSTEK_FX503VD_KEYBOARD 0x1869 | 231 | #define USB_DEVICE_ID_ASUSTEK_FX503VD_KEYBOARD 0x1869 |
| 232 | #define USB_DEVICE_ID_ASUSTEK_XGM_2023 0x1a9a | ||
| 232 | 233 | ||
| 233 | #define USB_VENDOR_ID_ATEN 0x0557 | 234 | #define USB_VENDOR_ID_ATEN 0x0557 |
| 234 | #define USB_DEVICE_ID_ATEN_UC100KM 0x2004 | 235 | #define USB_DEVICE_ID_ATEN_UC100KM 0x2004 |
| @@ -990,13 +990,11 @@ static const __u8 *magicmouse_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 989 unmodified lines | |||
| 990 | */ | 990 | */ |
| 991 | if ((is_usb_magicmouse2(hdev->vendor, hdev->product) || | 991 | if ((is_usb_magicmouse2(hdev->vendor, hdev->product) || |
| 992 | is_usb_magictrackpad2(hdev->vendor, hdev->product)) && | 992 | is_usb_magictrackpad2(hdev->vendor, hdev->product)) && |
| 993 | *rsize == 83 && rdesc[46] == 0x84 && rdesc[58] == 0x85) { | 993 | *rsize >= 83 && rdesc[46] == 0x84 && rdesc[58] == 0x85) { |
| 994 | hid_info(hdev, | 994 | hid_info(hdev, |
| 995 | "fixing up magicmouse battery report descriptor\n"); | 995 | "fixing up magicmouse battery report descriptor\n"); |
| 996 | *rsize = *rsize - 1; | 996 | *rsize = *rsize - 1; |
| 997 | rdesc = kmemdup(rdesc + 1, *rsize, GFP_KERNEL); | 997 | rdesc = rdesc + 1; |
| 998 | if (!rdesc) | ||
| 999 | return NULL; | ||
| 1000 | 998 | ||
| 1001 | rdesc[0] = 0x05; | 999 | rdesc[0] = 0x05; |
| 1002 | rdesc[1] = 0x01; | 1000 | rdesc[1] = 0x01; |
| @@ -353,6 +353,8 @@ static int mcp_i2c_smbus_read(struct mcp2221 *mcp, · 352 unmodified lines | |||
| 353 | usleep_range(90, 100); | 353 | usleep_range(90, 100); |
| 354 | retries++; | 354 | retries++; |
| 355 | } else { | 355 | } else { |
| 356 | usleep_range(980, 1000); | ||
| 357 | mcp_cancel_last_cmd(mcp); | ||
| 356 | return ret; | 358 | return ret; |
| 357 | } | 359 | } |
| 358 | } else { | 360 | } else { |
| @@ -77,6 +77,7 @@ MODULE_LICENSE("GPL"); · 76 unmodified lines | |||
| 77 | #define MT_QUIRK_ORIENTATION_INVERT BIT(22) | 77 | #define MT_QUIRK_ORIENTATION_INVERT BIT(22) |
| 78 | #define MT_QUIRK_APPLE_TOUCHBAR BIT(23) | 78 | #define MT_QUIRK_APPLE_TOUCHBAR BIT(23) |
| 79 | #define MT_QUIRK_YOGABOOK9I BIT(24) | 79 | #define MT_QUIRK_YOGABOOK9I BIT(24) |
| 80 | #define MT_QUIRK_KEEP_LATENCY_ON_CLOSE BIT(25) | ||
| 80 | 81 | ||
| 81 | #define MT_INPUTMODE_TOUCHSCREEN 0x02 | 82 | #define MT_INPUTMODE_TOUCHSCREEN 0x02 |
| 82 | #define MT_INPUTMODE_TOUCHPAD 0x03 | 83 | #define MT_INPUTMODE_TOUCHPAD 0x03 |
| @@ -214,6 +215,7 @@ static void mt_post_parse(struct mt_device *td, struct mt_application *app); · 131 unmodified lines | |||
| 214 | #define MT_CLS_WIN_8_DISABLE_WAKEUP 0x0016 | 215 | #define MT_CLS_WIN_8_DISABLE_WAKEUP 0x0016 |
| 215 | #define MT_CLS_WIN_8_NO_STICKY_FINGERS 0x0017 | 216 | #define MT_CLS_WIN_8_NO_STICKY_FINGERS 0x0017 |
| 216 | #define MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU 0x0018 | 217 | #define MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU 0x0018 |
| 218 | #define MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE 0x0019 | ||
| 217 | 219 | ||
| 218 | /* vendor specific classes */ | 220 | /* vendor specific classes */ |
| 219 | #define MT_CLS_3M 0x0101 | 221 | #define MT_CLS_3M 0x0101 |
| @@ -233,6 +235,7 @@ static void mt_post_parse(struct mt_device *td, struct mt_application *app); · 13 unmodified lines | |||
| 233 | #define MT_CLS_SMART_TECH 0x0113 | 235 | #define MT_CLS_SMART_TECH 0x0113 |
| 234 | #define MT_CLS_APPLE_TOUCHBAR 0x0114 | 236 | #define MT_CLS_APPLE_TOUCHBAR 0x0114 |
| 235 | #define MT_CLS_YOGABOOK9I 0x0115 | 237 | #define MT_CLS_YOGABOOK9I 0x0115 |
| 238 | #define MT_CLS_EGALAX_P80H84 0x0116 | ||
| 236 | #define MT_CLS_SIS 0x0457 | 239 | #define MT_CLS_SIS 0x0457 |
| 237 | 240 | ||
| 238 | #define MT_DEFAULT_MAXCONTACT 10 | 241 | #define MT_DEFAULT_MAXCONTACT 10 |
| @@ -334,6 +337,15 @@ static const struct mt_class mt_classes[] = { · 95 unmodified lines | |||
| 334 | MT_QUIRK_CONTACT_CNT_ACCURATE | | 337 | MT_QUIRK_CONTACT_CNT_ACCURATE | |
| 335 | MT_QUIRK_WIN8_PTP_BUTTONS, | 338 | MT_QUIRK_WIN8_PTP_BUTTONS, |
| 336 | .export_all_inputs = true }, | 339 | .export_all_inputs = true }, |
| 340 | { .name = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | ||
| 341 | .quirks = MT_QUIRK_ALWAYS_VALID | | ||
| 342 | MT_QUIRK_IGNORE_DUPLICATES | | ||
| 343 | MT_QUIRK_HOVERING | | ||
| 344 | MT_QUIRK_CONTACT_CNT_ACCURATE | | ||
| 345 | MT_QUIRK_STICKY_FINGERS | | ||
| 346 | MT_QUIRK_WIN8_PTP_BUTTONS | | ||
| 347 | MT_QUIRK_KEEP_LATENCY_ON_CLOSE, | ||
| 348 | .export_all_inputs = true }, | ||
| 337 | 349 | ||
| 338 | /* | 350 | /* |
| 339 | * vendor specific classes | 351 | * vendor specific classes |
| @@ -438,6 +450,11 @@ static const struct mt_class mt_classes[] = { · 98 unmodified lines | |||
| 438 | MT_QUIRK_YOGABOOK9I, | 450 | MT_QUIRK_YOGABOOK9I, |
| 439 | .export_all_inputs = true | 451 | .export_all_inputs = true |
| 440 | }, | 452 | }, |
| 453 | { .name = MT_CLS_EGALAX_P80H84, | ||
| 454 | .quirks = MT_QUIRK_ALWAYS_VALID | | ||
| 455 | MT_QUIRK_IGNORE_DUPLICATES | | ||
| 456 | MT_QUIRK_CONTACT_CNT_ACCURATE, | ||
| 457 | }, | ||
| 441 | { } | 458 | { } |
| 442 | }; | 459 | }; |
| 443 | 460 | ||
| @@ -849,7 +866,8 @@ static int mt_touch_input_mapping(struct hid_device *hdev, struct hid_input *hi, · 405 unmodified lines | |||
| 849 | if ((cls->name == MT_CLS_WIN_8 || | 866 | if ((cls->name == MT_CLS_WIN_8 || |
| 850 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT || | 867 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT || |
| 851 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU || | 868 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU || |
| 852 | cls->name == MT_CLS_WIN_8_DISABLE_WAKEUP) && | 869 | cls->name == MT_CLS_WIN_8_DISABLE_WAKEUP || |
| 870 | cls->name == MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE) && | ||
| 853 | (field->application == HID_DG_TOUCHPAD || | 871 | (field->application == HID_DG_TOUCHPAD || |
| 854 | field->application == HID_DG_TOUCHSCREEN)) | 872 | field->application == HID_DG_TOUCHSCREEN)) |
| 855 | app->quirks |= MT_QUIRK_CONFIDENCE; | 873 | app->quirks |= MT_QUIRK_CONFIDENCE; |
| @@ -1762,7 +1780,8 @@ static int mt_input_configured(struct hid_device *hdev, struct hid_input *hi) · 906 unmodified lines | |||
| 1762 | int ret; | 1780 | int ret; |
| 1763 | 1781 | ||
| 1764 | if (td->is_haptic_touchpad && (td->mtclass.name == MT_CLS_WIN_8 || | 1782 | if (td->is_haptic_touchpad && (td->mtclass.name == MT_CLS_WIN_8 || |
| 1765 | td->mtclass.name == MT_CLS_WIN_8_FORCE_MULTI_INPUT)) { | 1783 | td->mtclass.name == MT_CLS_WIN_8_FORCE_MULTI_INPUT || |
| 1784 | td->mtclass.name == MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE)) { | ||
| 1766 | if (hid_haptic_input_configured(hdev, td->haptic, hi) == 0) | 1785 | if (hid_haptic_input_configured(hdev, td->haptic, hi) == 0) |
| 1767 | td->is_haptic_touchpad = false; | 1786 | td->is_haptic_touchpad = false; |
| 1768 | } else { | 1787 | } else { |
| @@ -2075,7 +2094,12 @@ static void mt_on_hid_hw_open(struct hid_device *hdev) · 306 unmodified lines | |||
| 2075 | 2094 | ||
| 2076 | static void mt_on_hid_hw_close(struct hid_device *hdev) | 2095 | static void mt_on_hid_hw_close(struct hid_device *hdev) |
| 2077 | { | 2096 | { |
| 2078 | mt_set_modes(hdev, HID_LATENCY_HIGH, TOUCHPAD_REPORT_NONE); | 2097 | struct mt_device *td = hid_get_drvdata(hdev); |
| 2098 | |||
| 2099 | if (td->mtclass.quirks & MT_QUIRK_KEEP_LATENCY_ON_CLOSE) | ||
| 2100 | mt_set_modes(hdev, HID_LATENCY_NORMAL, TOUCHPAD_REPORT_NONE); | ||
| 2101 | else | ||
| 2102 | mt_set_modes(hdev, HID_LATENCY_HIGH, TOUCHPAD_REPORT_NONE); | ||
| 2079 | } | 2103 | } |
| 2080 | 2104 | ||
| 2081 | /* | 2105 | /* |
| @@ -2215,8 +2239,9 @@ static const struct hid_device_id mt_devices[] = { · 133 unmodified lines | |||
| 2215 | { .driver_data = MT_CLS_EGALAX_SERIAL, | 2239 | { .driver_data = MT_CLS_EGALAX_SERIAL, |
| 2216 | MT_USB_DEVICE(USB_VENDOR_ID_DWAV, | 2240 | MT_USB_DEVICE(USB_VENDOR_ID_DWAV, |
| 2217 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C000) }, | 2241 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C000) }, |
| 2218 | { .driver_data = MT_CLS_EGALAX, | 2242 | { .driver_data = MT_CLS_EGALAX_P80H84, |
| 2219 | MT_USB_DEVICE(USB_VENDOR_ID_DWAV, | 2243 | HID_DEVICE(HID_BUS_ANY, HID_GROUP_MULTITOUCH_WIN_8, |
| 2244 | USB_VENDOR_ID_DWAV, | ||
| 2220 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C002) }, | 2245 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C002) }, |
| 2221 | 2246 | ||
| 2222 | /* Elan devices */ | 2247 | /* Elan devices */ |
| @@ -2461,6 +2486,14 @@ static const struct hid_device_id mt_devices[] = { · 238 unmodified lines | |||
| 2461 | MT_USB_DEVICE(USB_VENDOR_ID_UNITEC, | 2486 | MT_USB_DEVICE(USB_VENDOR_ID_UNITEC, |
| 2462 | USB_DEVICE_ID_UNITEC_USB_TOUCH_0A19) }, | 2487 | USB_DEVICE_ID_UNITEC_USB_TOUCH_0A19) }, |
| 2463 | 2488 | ||
| 2489 | /* Uniwill touchpads */ | ||
| 2490 | { .driver_data = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | ||
| 2491 | HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, | ||
| 2492 | USB_VENDOR_ID_PIXART, 0x0255) }, | ||
| 2493 | { .driver_data = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | ||
| 2494 | HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, | ||
| 2495 | USB_VENDOR_ID_PIXART, 0x0274) }, | ||
| 2496 | |||
| 2464 | /* VTL panels */ | 2497 | /* VTL panels */ |
| 2465 | { .driver_data = MT_CLS_VTL, | 2498 | { .driver_data = MT_CLS_VTL, |
| 2466 | MT_USB_DEVICE(USB_VENDOR_ID_VTL, | 2499 | MT_USB_DEVICE(USB_VENDOR_ID_VTL, |
| @@ -114,7 +114,7 @@ static int zc_raw_event(struct hid_device *hdev, struct hid_report *report, · 113 unmodified lines | |||
| 114 | unsigned key; | 114 | unsigned key; |
| 115 | unsigned short index; | 115 | unsigned short index; |
| 116 | 116 | ||
| 117 | if (report->id == data[0]) { | 117 | if (report->id == data[0] && (hdev->claimed & HID_CLAIMED_INPUT)) { |
| 118 | 118 | ||
| 119 | /* break keys */ | 119 | /* break keys */ |
| 120 | for (index = 0; index < 4; index++) { | 120 | for (index = 0; index < 4; index++) { |
| @@ -39,6 +39,8 @@ · 38 unmodified lines | |||
| 39 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_H 0xE345 | 39 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_H 0xE345 |
| 40 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_P 0xE445 | 40 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_P 0xE445 |
| 41 | #define PCI_DEVICE_ID_INTEL_ISH_WCL 0x4D45 | 41 | #define PCI_DEVICE_ID_INTEL_ISH_WCL 0x4D45 |
| 42 | #define PCI_DEVICE_ID_INTEL_ISH_NVL_H 0xD354 | ||
| 43 | #define PCI_DEVICE_ID_INTEL_ISH_NVL_S 0x6E78 | ||
| 42 | 44 | ||
| 43 | #define REVISION_ID_CHT_A0 0x6 | 45 | #define REVISION_ID_CHT_A0 0x6 |
| 44 | #define REVISION_ID_CHT_Ax_SI 0x0 | 46 | #define REVISION_ID_CHT_Ax_SI 0x0 |
| @@ -28,11 +28,15 @@ enum ishtp_driver_data_index { · 27 unmodified lines | |||
| 28 | ISHTP_DRIVER_DATA_LNL_M, | 28 | ISHTP_DRIVER_DATA_LNL_M, |
| 29 | ISHTP_DRIVER_DATA_PTL, | 29 | ISHTP_DRIVER_DATA_PTL, |
| 30 | ISHTP_DRIVER_DATA_WCL, | 30 | ISHTP_DRIVER_DATA_WCL, |
| 31 | ISHTP_DRIVER_DATA_NVL_H, | ||
| 32 | ISHTP_DRIVER_DATA_NVL_S, | ||
| 31 | }; | 33 | }; |
| 32 | 34 | ||
| 33 | #define ISH_FW_GEN_LNL_M "lnlm" | 35 | #define ISH_FW_GEN_LNL_M "lnlm" |
| 34 | #define ISH_FW_GEN_PTL "ptl" | 36 | #define ISH_FW_GEN_PTL "ptl" |
| 35 | #define ISH_FW_GEN_WCL "wcl" | 37 | #define ISH_FW_GEN_WCL "wcl" |
| 38 | #define ISH_FW_GEN_NVL_H "nvlh" | ||
| 39 | #define ISH_FW_GEN_NVL_S "nvls" | ||
| 36 | 40 | ||
| 37 | #define ISH_FIRMWARE_PATH(gen) "intel/ish/ish_" gen ".bin" | 41 | #define ISH_FIRMWARE_PATH(gen) "intel/ish/ish_" gen ".bin" |
| 38 | #define ISH_FIRMWARE_PATH_ALL "intel/ish/ish_*.bin" | 42 | #define ISH_FIRMWARE_PATH_ALL "intel/ish/ish_*.bin" |
| @@ -47,6 +51,12 @@ static struct ishtp_driver_data ishtp_driver_data[] = { · 8 unmodified lines | |||
| 47 | [ISHTP_DRIVER_DATA_WCL] = { | 51 | [ISHTP_DRIVER_DATA_WCL] = { |
| 48 | .fw_generation = ISH_FW_GEN_WCL, | 52 | .fw_generation = ISH_FW_GEN_WCL, |
| 49 | }, | 53 | }, |
| 54 | [ISHTP_DRIVER_DATA_NVL_H] = { | ||
| 55 | .fw_generation = ISH_FW_GEN_NVL_H, | ||
| 56 | }, | ||
| 57 | [ISHTP_DRIVER_DATA_NVL_S] = { | ||
| 58 | .fw_generation = ISH_FW_GEN_NVL_S, | ||
| 59 | }, | ||
| 50 | }; | 60 | }; |
| 51 | 61 | ||
| 52 | static const struct pci_device_id ish_pci_tbl[] = { | 62 | static const struct pci_device_id ish_pci_tbl[] = { |
| @@ -76,6 +86,8 @@ static const struct pci_device_id ish_pci_tbl[] = { · 23 unmodified lines | |||
| 76 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_H, ISHTP_DRIVER_DATA_PTL)}, | 86 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_H, ISHTP_DRIVER_DATA_PTL)}, |
| 77 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_P, ISHTP_DRIVER_DATA_PTL)}, | 87 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_P, ISHTP_DRIVER_DATA_PTL)}, |
| 78 | {PCI_DEVICE_DATA(INTEL, ISH_WCL, ISHTP_DRIVER_DATA_WCL)}, | 88 | {PCI_DEVICE_DATA(INTEL, ISH_WCL, ISHTP_DRIVER_DATA_WCL)}, |
| 89 | {PCI_DEVICE_DATA(INTEL, ISH_NVL_H, ISHTP_DRIVER_DATA_NVL_H)}, | ||
| 90 | {PCI_DEVICE_DATA(INTEL, ISH_NVL_S, ISHTP_DRIVER_DATA_NVL_S)}, | ||
| 79 | {} | 91 | {} |
| 80 | }; | 92 | }; |
| 81 | MODULE_DEVICE_TABLE(pci, ish_pci_tbl); | 93 | MODULE_DEVICE_TABLE(pci, ish_pci_tbl); |
| @@ -1452,10 +1452,13 @@ static int pidff_init_fields(struct pidff_device *pidff, struct input_dev *dev) · 1451 unmodified lines | |||
| 1452 | hid_warn(pidff->hid, "unknown ramp effect layout\n"); | 1452 | hid_warn(pidff->hid, "unknown ramp effect layout\n"); |
| 1453 | 1453 | ||
| 1454 | if (PIDFF_FIND_FIELDS(set_condition, PID_SET_CONDITION, 1)) { | 1454 | if (PIDFF_FIND_FIELDS(set_condition, PID_SET_CONDITION, 1)) { |
| 1455 | if (test_and_clear_bit(FF_SPRING, dev->ffbit) || | 1455 | bool test = false; |
| 1456 | test_and_clear_bit(FF_DAMPER, dev->ffbit) || | 1456 | |
| 1457 | test_and_clear_bit(FF_FRICTION, dev->ffbit) || | 1457 | test |= test_and_clear_bit(FF_SPRING, dev->ffbit); |
| 1458 | test_and_clear_bit(FF_INERTIA, dev->ffbit)) | 1458 | test |= test_and_clear_bit(FF_DAMPER, dev->ffbit); |
| 1459 | test |= test_and_clear_bit(FF_FRICTION, dev->ffbit); | ||
| 1460 | test |= test_and_clear_bit(FF_INERTIA, dev->ffbit); | ||
| 1461 | if (test) | ||
| 1459 | hid_warn(pidff->hid, "unknown condition effect layout\n"); | 1462 | hid_warn(pidff->hid, "unknown condition effect layout\n"); |
| 1460 | } | 1463 | } |
| 1461 | 1464 | ||
| @@ -1493,8 +1493,7 @@ config SENSORS_LM73 · 1492 unmodified lines | |||
| 1493 | 1493 | ||
| 1494 | config SENSORS_LM75 | 1494 | config SENSORS_LM75 |
| 1495 | tristate "National Semiconductor LM75 and compatibles" | 1495 | tristate "National Semiconductor LM75 and compatibles" |
| 1496 | depends on I2C | 1496 | depends on I3C_OR_I2C |
| 1497 | depends on I3C || !I3C | ||
| 1498 | select REGMAP_I2C | 1497 | select REGMAP_I2C |
| 1499 | select REGMAP_I3C if I3C | 1498 | select REGMAP_I3C if I3C |
| 1500 | help | 1499 | help |
| @@ -1927,16 +1926,6 @@ config SENSORS_RASPBERRYPI_HWMON · 426 unmodified lines | |||
| 1927 | This driver can also be built as a module. If so, the module | 1926 | This driver can also be built as a module. If so, the module |
| 1928 | will be called raspberrypi-hwmon. | 1927 | will be called raspberrypi-hwmon. |
| 1929 | 1928 | ||
| 1930 | config SENSORS_SA67MCU | ||
| 1931 | tristate "Kontron sa67mcu hardware monitoring driver" | ||
| 1932 | depends on MFD_SL28CPLD || COMPILE_TEST | ||
| 1933 | help | ||
| 1934 | If you say yes here you get support for the voltage and temperature | ||
| 1935 | monitor of the sa67 board management controller. | ||
| 1936 | |||
| 1937 | This driver can also be built as a module. If so, the module | ||
| 1938 | will be called sa67mcu-hwmon. | ||
| 1939 | |||
| 1940 | config SENSORS_SL28CPLD | 1929 | config SENSORS_SL28CPLD |
| 1941 | tristate "Kontron sl28cpld hardware monitoring driver" | 1930 | tristate "Kontron sl28cpld hardware monitoring driver" |
| 1942 | depends on MFD_SL28CPLD || COMPILE_TEST | 1931 | depends on MFD_SL28CPLD || COMPILE_TEST |
| @@ -2392,8 +2381,7 @@ config SENSORS_TMP103 · 449 unmodified lines | |||
| 2392 | 2381 | ||
| 2393 | config SENSORS_TMP108 | 2382 | config SENSORS_TMP108 |
| 2394 | tristate "Texas Instruments TMP108" | 2383 | tristate "Texas Instruments TMP108" |
| 2395 | depends on I2C | 2384 | depends on I3C_OR_I2C |
| 2396 | depends on I3C || !I3C | ||
| 2397 | select REGMAP_I2C | 2385 | select REGMAP_I2C |
| 2398 | select REGMAP_I3C if I3C | 2386 | select REGMAP_I3C if I3C |
| 2399 | help | 2387 | help |
| @@ -199,7 +199,6 @@ obj-$(CONFIG_SENSORS_PT5161L) += pt5161l.o · 198 unmodified lines | |||
| 199 | obj-$(CONFIG_SENSORS_PWM_FAN) += pwm-fan.o | 199 | obj-$(CONFIG_SENSORS_PWM_FAN) += pwm-fan.o |
| 200 | obj-$(CONFIG_SENSORS_QNAP_MCU_HWMON) += qnap-mcu-hwmon.o | 200 | obj-$(CONFIG_SENSORS_QNAP_MCU_HWMON) += qnap-mcu-hwmon.o |
| 201 | obj-$(CONFIG_SENSORS_RASPBERRYPI_HWMON) += raspberrypi-hwmon.o | 201 | obj-$(CONFIG_SENSORS_RASPBERRYPI_HWMON) += raspberrypi-hwmon.o |
| 202 | obj-$(CONFIG_SENSORS_SA67MCU) += sa67mcu-hwmon.o | ||
| 203 | obj-$(CONFIG_SENSORS_SBTSI) += sbtsi_temp.o | 202 | obj-$(CONFIG_SENSORS_SBTSI) += sbtsi_temp.o |
| 204 | obj-$(CONFIG_SENSORS_SBRMI) += sbrmi.o | 203 | obj-$(CONFIG_SENSORS_SBRMI) += sbrmi.o |
| 205 | obj-$(CONFIG_SENSORS_SCH56XX_COMMON)+= sch56xx-common.o | 204 | obj-$(CONFIG_SENSORS_SCH56XX_COMMON)+= sch56xx-common.o |
| @@ -37,7 +37,9 @@ · 36 unmodified lines | |||
| 37 | #define AHT10_CMD_MEAS 0b10101100 | 37 | #define AHT10_CMD_MEAS 0b10101100 |
| 38 | #define AHT10_CMD_RST 0b10111010 | 38 | #define AHT10_CMD_RST 0b10111010 |
| 39 | 39 | ||
| 40 | #define DHT20_CMD_INIT 0x71 | 40 | #define AHT20_CMD_INIT 0b10111110 |
| 41 | |||
| 42 | #define DHT20_CMD_INIT 0b01110001 | ||
| 41 | 43 | ||
| 42 | /* | 44 | /* |
| 43 | * Flags in the answer byte/command | 45 | * Flags in the answer byte/command |
| @@ -341,7 +343,7 @@ static int aht10_probe(struct i2c_client *client) · 297 unmodified lines | |||
| 341 | data->meas_size = AHT20_MEAS_SIZE; | 343 | data->meas_size = AHT20_MEAS_SIZE; |
| 342 | data->crc8 = true; | 344 | data->crc8 = true; |
| 343 | crc8_populate_msb(crc8_table, AHT20_CRC8_POLY); | 345 | crc8_populate_msb(crc8_table, AHT20_CRC8_POLY); |
| 344 | data->init_cmd = AHT10_CMD_INIT; | 346 | data->init_cmd = AHT20_CMD_INIT; |
| 345 | break; | 347 | break; |
| 346 | case dht20: | 348 | case dht20: |
| 347 | data->meas_size = AHT20_MEAS_SIZE; | 349 | data->meas_size = AHT20_MEAS_SIZE; |
| @@ -3590,10 +3590,13 @@ static int it87_resume(struct device *dev) · 3589 unmodified lines | |||
| 3590 | { | 3590 | { |
| 3591 | struct platform_device *pdev = to_platform_device(dev); | 3591 | struct platform_device *pdev = to_platform_device(dev); |
| 3592 | struct it87_data *data = dev_get_drvdata(dev); | 3592 | struct it87_data *data = dev_get_drvdata(dev); |
| 3593 | int err; | ||
| 3593 | 3594 | ||
| 3594 | it87_resume_sio(pdev); | 3595 | it87_resume_sio(pdev); |
| 3595 | 3596 | ||
| 3596 | it87_lock(data); | 3597 | err = it87_lock(data); |
| 3598 | if (err) | ||
| 3599 | return err; | ||
| 3597 | 3600 | ||
| 3598 | it87_check_pwm(dev); | 3601 | it87_check_pwm(dev); |
| 3599 | it87_check_limit_regs(data); | 3602 | it87_check_limit_regs(data); |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | |||
| 22 | 22 | ||
| 23 | #include <linux/bitfield.h> | 23 | #include <linux/bitfield.h> |
| 24 | #include <linux/hwmon.h> | 24 | #include <linux/hwmon.h> |
| 25 | #include <linux/math64.h> | ||
| 25 | #include <linux/mfd/macsmc.h> | 26 | #include <linux/mfd/macsmc.h> |
| 26 | #include <linux/module.h> | 27 | #include <linux/module.h> |
| 27 | #include <linux/of.h> | 28 | #include <linux/of.h> |
| @@ -130,7 +131,7 @@ static int macsmc_hwmon_read_ioft_scaled(struct apple_smc *smc, smc_key key, · 102 unmodified lines | |||
| 130 | if (ret < 0) | 131 | if (ret < 0) |
| 131 | return ret; | 132 | return ret; |
| 132 | 133 | ||
| 133 | *p = mult_frac(val, scale, 65536); | 134 | *p = mul_u64_u32_div(val, scale, 65536); |
| 134 | 135 | ||
| 135 | return 0; | 136 | return 0; |
| 136 | } | 137 | } |
| @@ -140,7 +141,7 @@ static int macsmc_hwmon_read_ioft_scaled(struct apple_smc *smc, smc_key key, · 3 unmodified lines | |||
| 140 | * them. | 141 | * them. |
| 141 | */ | 142 | */ |
| 142 | static int macsmc_hwmon_read_f32_scaled(struct apple_smc *smc, smc_key key, | 143 | static int macsmc_hwmon_read_f32_scaled(struct apple_smc *smc, smc_key key, |
| 143 | int *p, int scale) | 144 | long *p, int scale) |
| 144 | { | 145 | { |
| 145 | u32 fval; | 146 | u32 fval; |
| 146 | u64 val; | 147 | u64 val; |
| @@ -162,21 +163,21 @@ static int macsmc_hwmon_read_f32_scaled(struct apple_smc *smc, smc_key key, · 15 unmodified lines | |||
| 162 | val = 0; | 163 | val = 0; |
| 163 | else if (exp < 0) | 164 | else if (exp < 0) |
| 164 | val >>= -exp; | 165 | val >>= -exp; |
| 165 | else if (exp != 0 && (val & ~((1UL << (64 - exp)) - 1))) /* overflow */ | 166 | else if (exp != 0 && (val & ~((1ULL << (64 - exp)) - 1))) /* overflow */ |
| 166 | val = U64_MAX; | 167 | val = U64_MAX; |
| 167 | else | 168 | else |
| 168 | val <<= exp; | 169 | val <<= exp; |
| 169 | 170 | ||
| 170 | if (fval & FLT_SIGN_MASK) { | 171 | if (fval & FLT_SIGN_MASK) { |
| 171 | if (val > (-(s64)INT_MIN)) | 172 | if (val > (u64)LONG_MAX + 1) |
| 172 | *p = INT_MIN; | 173 | *p = LONG_MIN; |
| 173 | else | 174 | else |
| 174 | *p = -val; | 175 | *p = -(long)val; |
| 175 | } else { | 176 | } else { |
| 176 | if (val > INT_MAX) | 177 | if (val > (u64)LONG_MAX) |
| 177 | *p = INT_MAX; | 178 | *p = LONG_MAX; |
| 178 | else | 179 | else |
| 179 | *p = val; | 180 | *p = (long)val; |
| 180 | } | 181 | } |
| 181 | 182 | ||
| 182 | return 0; | 183 | return 0; |
| @@ -195,7 +196,7 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 12 unmodified lines | |||
| 195 | switch (sensor->info.type_code) { | 196 | switch (sensor->info.type_code) { |
| 196 | /* 32-bit IEEE 754 float */ | 197 | /* 32-bit IEEE 754 float */ |
| 197 | case __SMC_KEY('f', 'l', 't', ' '): { | 198 | case __SMC_KEY('f', 'l', 't', ' '): { |
| 198 | u32 flt_ = 0; | 199 | long flt_ = 0; |
| 199 | 200 | ||
| 200 | ret = macsmc_hwmon_read_f32_scaled(smc, sensor->macsmc_key, | 201 | ret = macsmc_hwmon_read_f32_scaled(smc, sensor->macsmc_key, |
| 201 | &flt_, scale); | 202 | &flt_, scale); |
| @@ -214,7 +215,10 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 12 unmodified lines | |||
| 214 | if (ret) | 215 | if (ret) |
| 215 | return ret; | 216 | return ret; |
| 216 | 217 | ||
| 217 | *val = (long)ioft; | 218 | if (ioft > LONG_MAX) |
| 219 | *val = LONG_MAX; | ||
| 220 | else | ||
| 221 | *val = (long)ioft; | ||
| 218 | break; | 222 | break; |
| 219 | } | 223 | } |
| 220 | default: | 224 | default: |
| @@ -224,29 +228,26 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 3 unmodified lines | |||
| 224 | return 0; | 228 | return 0; |
| 225 | } | 229 | } |
| 226 | 230 | ||
| 227 | static int macsmc_hwmon_write_f32(struct apple_smc *smc, smc_key key, int value) | 231 | static int macsmc_hwmon_write_f32(struct apple_smc *smc, smc_key key, long value) |
| 228 | { | 232 | { |
| 229 | u64 val; | 233 | u64 val; |
| 230 | u32 fval = 0; | 234 | u32 fval = 0; |
| 231 | int exp = 0, neg; | 235 | int exp, neg; |
| 232 | 236 | ||
| 237 | neg = value < 0; | ||
| 233 | val = abs(value); | 238 | val = abs(value); |
| 234 | neg = val != value; | ||
| 235 | 239 | ||
| 236 | if (val) { | 240 | if (val) { |
| 237 | int msb = __fls(val) - exp; | 241 | exp = __fls(val); |
| 238 | 242 | ||
| 239 | if (msb > 23) { | 243 | if (exp > 23) |
| 240 | val >>= msb - FLT_MANT_BIAS; | 244 | val >>= exp - 23; |
| 241 | exp -= msb - FLT_MANT_BIAS; | 245 | else |
| 242 | } else if (msb < 23) { | 246 | val <<= 23 - exp; |
| 243 | val <<= FLT_MANT_BIAS - msb; | ||
| 244 | exp += msb; | ||
| 245 | } | ||
| 246 | 247 | ||
| 247 | fval = FIELD_PREP(FLT_SIGN_MASK, neg) | | 248 | fval = FIELD_PREP(FLT_SIGN_MASK, neg) | |
| 248 | FIELD_PREP(FLT_EXP_MASK, exp + FLT_EXP_BIAS) | | 249 | FIELD_PREP(FLT_EXP_MASK, exp + FLT_EXP_BIAS) | |
| 249 | FIELD_PREP(FLT_MANT_MASK, val); | 250 | FIELD_PREP(FLT_MANT_MASK, val & FLT_MANT_MASK); |
| 250 | } | 251 | } |
| 251 | 252 | ||
| 252 | return apple_smc_write_u32(smc, key, fval); | 253 | return apple_smc_write_u32(smc, key, fval); |
| @@ -663,8 +664,8 @@ static int macsmc_hwmon_populate_sensors(struct macsmc_hwmon *hwmon, · 410 unmodified lines | |||
| 663 | if (!hwmon->volt.sensors) | 664 | if (!hwmon->volt.sensors) |
| 664 | return -ENOMEM; | 665 | return -ENOMEM; |
| 665 | 666 | ||
| 666 | for_each_child_of_node_with_prefix(hwmon_node, key_node, "volt-") { | 667 | for_each_child_of_node_with_prefix(hwmon_node, key_node, "voltage-") { |
| 667 | sensor = &hwmon->temp.sensors[hwmon->temp.count]; | 668 | sensor = &hwmon->volt.sensors[hwmon->volt.count]; |
| 668 | if (!macsmc_hwmon_create_sensor(hwmon->dev, hwmon->smc, key_node, sensor)) { | 669 | if (!macsmc_hwmon_create_sensor(hwmon->dev, hwmon->smc, key_node, sensor)) { |
| 669 | sensor->attrs = HWMON_I_INPUT; | 670 | sensor->attrs = HWMON_I_INPUT; |
| 670 | 671 | ||
| @@ -607,7 +607,7 @@ static int max6639_init_client(struct i2c_client *client, · 606 unmodified lines | |||
| 607 | return err; | 607 | return err; |
| 608 | 608 | ||
| 609 | /* Fans PWM polarity high by default */ | 609 | /* Fans PWM polarity high by default */ |
| 610 | err = regmap_write(data->regmap, MAX6639_REG_FAN_CONFIG2a(i), 0x00); | 610 | err = regmap_write(data->regmap, MAX6639_REG_FAN_CONFIG2a(i), 0x02); |
| 611 | if (err) | 611 | if (err) |
| 612 | return err; | 612 | return err; |
| 613 | 613 | ||
| @@ -79,7 +79,8 @@ static ssize_t q54sj108a2_debugfs_read(struct file *file, char __user *buf, · 78 unmodified lines | |||
| 79 | int idx = *idxp; | 79 | int idx = *idxp; |
| 80 | struct q54sj108a2_data *psu = to_psu(idxp, idx); | 80 | struct q54sj108a2_data *psu = to_psu(idxp, idx); |
| 81 | char data[I2C_SMBUS_BLOCK_MAX + 2] = { 0 }; | 81 | char data[I2C_SMBUS_BLOCK_MAX + 2] = { 0 }; |
| 82 | char data_char[I2C_SMBUS_BLOCK_MAX + 2] = { 0 }; | 82 | char data_char[I2C_SMBUS_BLOCK_MAX * 2 + 2] = { 0 }; |
| 83 | char *out = data; | ||
| 83 | char *res; | 84 | char *res; |
| 84 | 85 | ||
| 85 | switch (idx) { | 86 | switch (idx) { |
| @@ -150,27 +151,27 @@ static ssize_t q54sj108a2_debugfs_read(struct file *file, char __user *buf, · 64 unmodified lines | |||
| 150 | if (rc < 0) | 151 | if (rc < 0) |
| 151 | return rc; | 152 | return rc; |
| 152 | 153 | ||
| 153 | res = bin2hex(data, data_char, 32); | 154 | res = bin2hex(data_char, data, rc); |
| 154 | rc = res - data; | 155 | rc = res - data_char; |
| 155 | 156 | out = data_char; | |
| 156 | break; | 157 | break; |
| 157 | case Q54SJ108A2_DEBUGFS_FLASH_KEY: | 158 | case Q54SJ108A2_DEBUGFS_FLASH_KEY: |
| 158 | rc = i2c_smbus_read_block_data(psu->client, PMBUS_FLASH_KEY_WRITE, data); | 159 | rc = i2c_smbus_read_block_data(psu->client, PMBUS_FLASH_KEY_WRITE, data); |
| 159 | if (rc < 0) | 160 | if (rc < 0) |
| 160 | return rc; | 161 | return rc; |
| 161 | 162 | ||
| 162 | res = bin2hex(data, data_char, 4); | 163 | res = bin2hex(data_char, data, rc); |
| 163 | rc = res - data; | 164 | rc = res - data_char; |
| 164 | 165 | out = data_char; | |
| 165 | break; | 166 | break; |
| 166 | default: | 167 | default: |
| 167 | return -EINVAL; | 168 | return -EINVAL; |
| 168 | } | 169 | } |
| 169 | 170 | ||
| 170 | data[rc] = '\n'; | 171 | out[rc] = '\n'; |
| 171 | rc += 2; | 172 | rc += 2; |
| 172 | 173 | ||
| 173 | return simple_read_from_buffer(buf, count, ppos, data, rc); | 174 | return simple_read_from_buffer(buf, count, ppos, out, rc); |
| 174 | } | 175 | } |
| 175 | 176 | ||
| 176 | static ssize_t q54sj108a2_debugfs_write(struct file *file, const char __user *buf, | 177 | static ssize_t q54sj108a2_debugfs_write(struct file *file, const char __user *buf, |
| @@ -1,161 +0,0 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0-only | ||
| 2 | /* | ||
| 3 | * sl67mcu hardware monitoring driver | ||
| 4 | * | ||
| 5 | * Copyright 2025 Kontron Europe GmbH | ||
| 6 | */ | ||
| 7 | |||
| 8 | #include <linux/bitfield.h> | ||
| 9 | #include <linux/hwmon.h> | ||
| 10 | #include <linux/kernel.h> | ||
| 11 | #include <linux/mod_devicetable.h> | ||
| 12 | #include <linux/module.h> | ||
| 13 | #include <linux/platform_device.h> | ||
| 14 | #include <linux/property.h> | ||
| 15 | #include <linux/regmap.h> | ||
| 16 | |||
| 17 | #define SA67MCU_VOLTAGE(n) (0x00 + ((n) * 2)) | ||
| 18 | #define SA67MCU_TEMP(n) (0x04 + ((n) * 2)) | ||
| 19 | |||
| 20 | struct sa67mcu_hwmon { | ||
| 21 | struct regmap *regmap; | ||
| 22 | u32 offset; | ||
| 23 | }; | ||
| 24 | |||
| 25 | static int sa67mcu_hwmon_read(struct device *dev, | ||
| 26 | enum hwmon_sensor_types type, u32 attr, | ||
| 27 | int channel, long *input) | ||
| 28 | { | ||
| 29 | struct sa67mcu_hwmon *hwmon = dev_get_drvdata(dev); | ||
| 30 | unsigned int offset; | ||
| 31 | u8 reg[2]; | ||
| 32 | int ret; | ||
| 33 | |||
| 34 | switch (type) { | ||
| 35 | case hwmon_in: | ||
| 36 | switch (attr) { | ||
| 37 | case hwmon_in_input: | ||
| 38 | offset = hwmon->offset + SA67MCU_VOLTAGE(channel); | ||
| 39 | break; | ||
| 40 | default: | ||
| 41 | return -EOPNOTSUPP; | ||
| 42 | } | ||
| 43 | break; | ||
| 44 | case hwmon_temp: | ||
| 45 | switch (attr) { | ||
| 46 | case hwmon_temp_input: | ||
| 47 | offset = hwmon->offset + SA67MCU_TEMP(channel); | ||
| 48 | break; | ||
| 49 | default: | ||
| 50 | return -EOPNOTSUPP; | ||
| 51 | } | ||
| 52 | break; | ||
| 53 | default: | ||
| 54 | return -EOPNOTSUPP; | ||
| 55 | } | ||
| 56 | |||
| 57 | /* Reading the low byte will capture the value */ | ||
| 58 | ret = regmap_bulk_read(hwmon->regmap, offset, reg, ARRAY_SIZE(reg)); | ||
| 59 | if (ret) | ||
| 60 | return ret; | ||
| 61 | |||
| 62 | *input = reg[1] << 8 | reg[0]; | ||
| 63 | |||
| 64 | /* Temperatures are s16 and in 0.1degC steps. */ | ||
| 65 | if (type == hwmon_temp) | ||
| 66 | *input = sign_extend32(*input, 15) * 100; | ||
| 67 | |||
| 68 | return 0; | ||
| 69 | } | ||
| 70 | |||
| 71 | static const struct hwmon_channel_info * const sa67mcu_hwmon_info[] = { | ||
| 72 | HWMON_CHANNEL_INFO(in, | ||
| 73 | HWMON_I_INPUT | HWMON_I_LABEL, | ||
| 74 | HWMON_I_INPUT | HWMON_I_LABEL), | ||
| 75 | HWMON_CHANNEL_INFO(temp, HWMON_T_INPUT), | ||
| 76 | NULL | ||
| 77 | }; | ||
| 78 | |||
| 79 | static const char *const sa67mcu_hwmon_in_labels[] = { | ||
| 80 | "VDDIN", | ||
| 81 | "VDD_RTC", | ||
| 82 | }; | ||
| 83 | |||
| 84 | static int sa67mcu_hwmon_read_string(struct device *dev, | ||
| 85 | enum hwmon_sensor_types type, u32 attr, | ||
| 86 | int channel, const char **str) | ||
| 87 | { | ||
| 88 | switch (type) { | ||
| 89 | case hwmon_in: | ||
| 90 | switch (attr) { | ||
| 91 | case hwmon_in_label: | ||
| 92 | *str = sa67mcu_hwmon_in_labels[channel]; | ||
| 93 | return 0; | ||
| 94 | default: | ||
| 95 | return -EOPNOTSUPP; | ||
| 96 | } | ||
| 97 | default: | ||
| 98 | return -EOPNOTSUPP; | ||
| 99 | } | ||
| 100 | } | ||
| 101 | |||
| 102 | static const struct hwmon_ops sa67mcu_hwmon_ops = { | ||
| 103 | .visible = 0444, | ||
| 104 | .read = sa67mcu_hwmon_read, | ||
| 105 | .read_string = sa67mcu_hwmon_read_string, | ||
| 106 | }; | ||
| 107 | |||
| 108 | static const struct hwmon_chip_info sa67mcu_hwmon_chip_info = { | ||
| 109 | .ops = &sa67mcu_hwmon_ops, | ||
| 110 | .info = sa67mcu_hwmon_info, | ||
| 111 | }; | ||
| 112 | |||
| 113 | static int sa67mcu_hwmon_probe(struct platform_device *pdev) | ||
| 114 | { | ||
| 115 | struct sa67mcu_hwmon *hwmon; | ||
| 116 | struct device *hwmon_dev; | ||
| 117 | int ret; | ||
| 118 | |||
| 119 | if (!pdev->dev.parent) | ||
| 120 | return -ENODEV; | ||
| 121 | |||
| 122 | hwmon = devm_kzalloc(&pdev->dev, sizeof(*hwmon), GFP_KERNEL); | ||
| 123 | if (!hwmon) | ||
| 124 | return -ENOMEM; | ||
| 125 | |||
| 126 | hwmon->regmap = dev_get_regmap(pdev->dev.parent, NULL); | ||
| 127 | if (!hwmon->regmap) | ||
| 128 | return -ENODEV; | ||
| 129 | |||
| 130 | ret = device_property_read_u32(&pdev->dev, "reg", &hwmon->offset); | ||
| 131 | if (ret) | ||
| 132 | return -EINVAL; | ||
| 133 | |||
| 134 | hwmon_dev = devm_hwmon_device_register_with_info(&pdev->dev, | ||
| 135 | "sa67mcu_hwmon", hwmon, | ||
| 136 | &sa67mcu_hwmon_chip_info, | ||
| 137 | NULL); | ||
| 138 | if (IS_ERR(hwmon_dev)) | ||
| 139 | dev_err(&pdev->dev, "failed to register as hwmon device"); | ||
| 140 | |||
| 141 | return PTR_ERR_OR_ZERO(hwmon_dev); | ||
| 142 | } | ||
| 143 | |||
| 144 | static const struct of_device_id sa67mcu_hwmon_of_match[] = { | ||
| 145 | { .compatible = "kontron,sa67mcu-hwmon", }, | ||
| 146 | {} | ||
| 147 | }; | ||
| 148 | MODULE_DEVICE_TABLE(of, sa67mcu_hwmon_of_match); | ||
| 149 | |||
| 150 | static struct platform_driver sa67mcu_hwmon_driver = { | ||
| 151 | .probe = sa67mcu_hwmon_probe, | ||
| 152 | .driver = { | ||
| 153 | .name = "sa67mcu-hwmon", | ||
| 154 | .of_match_table = sa67mcu_hwmon_of_match, | ||
| 155 | }, | ||
| 156 | }; | ||
| 157 | module_platform_driver(sa67mcu_hwmon_driver); | ||
| 158 | |||
| 159 | MODULE_DESCRIPTION("sa67mcu Hardware Monitoring Driver"); | ||
| 160 | MODULE_AUTHOR("Michael Walle <mwalle@kernel.org>"); | ||
| 161 | MODULE_LICENSE("GPL"); | ||
| @@ -310,9 +310,10 @@ struct i801_priv { · 309 unmodified lines | |||
| 310 | 310 | ||
| 311 | /* | 311 | /* |
| 312 | * If set to true the host controller registers are reserved for | 312 | * If set to true the host controller registers are reserved for |
| 313 | * ACPI AML use. | 313 | * ACPI AML use. Needs extra protection by acpi_lock. |
| 314 | */ | 314 | */ |
| 315 | bool acpi_reserved; | 315 | bool acpi_reserved; |
| 316 | struct mutex acpi_lock; | ||
| 316 | }; | 317 | }; |
| 317 | 318 | ||
| 318 | #define FEATURE_SMBUS_PEC BIT(0) | 319 | #define FEATURE_SMBUS_PEC BIT(0) |
| @@ -894,8 +895,11 @@ static s32 i801_access(struct i2c_adapter *adap, u16 addr, · 575 unmodified lines | |||
| 894 | int hwpec, ret; | 895 | int hwpec, ret; |
| 895 | struct i801_priv *priv = i2c_get_adapdata(adap); | 896 | struct i801_priv *priv = i2c_get_adapdata(adap); |
| 896 | 897 | ||
| 897 | if (priv->acpi_reserved) | 898 | mutex_lock(&priv->acpi_lock); |
| 899 | if (priv->acpi_reserved) { | ||
| 900 | mutex_unlock(&priv->acpi_lock); | ||
| 898 | return -EBUSY; | 901 | return -EBUSY; |
| 902 | } | ||
| 899 | 903 | ||
| 900 | pm_runtime_get_sync(&priv->pci_dev->dev); | 904 | pm_runtime_get_sync(&priv->pci_dev->dev); |
| 901 | 905 | ||
| @@ -935,6 +939,7 @@ out: · 33 unmodified lines | |||
| 935 | iowrite8(SMBHSTSTS_INUSE_STS | STATUS_FLAGS, SMBHSTSTS(priv)); | 939 | iowrite8(SMBHSTSTS_INUSE_STS | STATUS_FLAGS, SMBHSTSTS(priv)); |
| 936 | 940 | ||
| 937 | pm_runtime_put_autosuspend(&priv->pci_dev->dev); | 941 | pm_runtime_put_autosuspend(&priv->pci_dev->dev); |
| 942 | mutex_unlock(&priv->acpi_lock); | ||
| 938 | return ret; | 943 | return ret; |
| 939 | } | 944 | } |
| 940 | 945 | ||
| @@ -1465,7 +1470,7 @@ i801_acpi_io_handler(u32 function, acpi_physical_address address, u32 bits, · 524 unmodified lines | |||
| 1465 | * further access from the driver itself. This device is now owned | 1470 | * further access from the driver itself. This device is now owned |
| 1466 | * by the system firmware. | 1471 | * by the system firmware. |
| 1467 | */ | 1472 | */ |
| 1468 | i2c_lock_bus(&priv->adapter, I2C_LOCK_SEGMENT); | 1473 | mutex_lock(&priv->acpi_lock); |
| 1469 | 1474 | ||
| 1470 | if (!priv->acpi_reserved && i801_acpi_is_smbus_ioport(priv, address)) { | 1475 | if (!priv->acpi_reserved && i801_acpi_is_smbus_ioport(priv, address)) { |
| 1471 | priv->acpi_reserved = true; | 1476 | priv->acpi_reserved = true; |
| @@ -1485,7 +1490,7 @@ i801_acpi_io_handler(u32 function, acpi_physical_address address, u32 bits, · 13 unmodified lines | |||
| 1485 | else | 1490 | else |
| 1486 | status = acpi_os_write_port(address, (u32)*value, bits); | 1491 | status = acpi_os_write_port(address, (u32)*value, bits); |
| 1487 | 1492 | ||
| 1488 | i2c_unlock_bus(&priv->adapter, I2C_LOCK_SEGMENT); | 1493 | mutex_unlock(&priv->acpi_lock); |
| 1489 | 1494 | ||
| 1490 | return status; | 1495 | return status; |
| 1491 | } | 1496 | } |
| @@ -1545,6 +1550,7 @@ static int i801_probe(struct pci_dev *dev, const struct pci_device_id *id) · 53 unmodified lines | |||
| 1545 | priv->adapter.dev.parent = &dev->dev; | 1550 | priv->adapter.dev.parent = &dev->dev; |
| 1546 | acpi_use_parent_companion(&priv->adapter.dev); | 1551 | acpi_use_parent_companion(&priv->adapter.dev); |
| 1547 | priv->adapter.retries = 3; | 1552 | priv->adapter.retries = 3; |
| 1553 | mutex_init(&priv->acpi_lock); | ||
| 1548 | 1554 | ||
| 1549 | priv->pci_dev = dev; | 1555 | priv->pci_dev = dev; |
| 1550 | priv->features = id->driver_data; | 1556 | priv->features = id->driver_data; |
| @@ -22,3 +22,15 @@ menuconfig I3C · 21 unmodified lines | |||
| 22 | if I3C | 22 | if I3C |
| 23 | source "drivers/i3c/master/Kconfig" | 23 | source "drivers/i3c/master/Kconfig" |
| 24 | endif # I3C | 24 | endif # I3C |
| 25 | |||
| 26 | config I3C_OR_I2C | ||
| 27 | tristate | ||
| 28 | default m if I3C=m | ||
| 29 | default I2C | ||
| 30 | help | ||
| 31 | Device drivers using module_i3c_i2c_driver() can use either | ||
| 32 | i2c or i3c hosts, but cannot be built-in for the kernel when | ||
| 33 | CONFIG_I3C=m. | ||
| 34 | |||
| 35 | Add 'depends on I2C_OR_I3C' in Kconfig for those drivers to | ||
| 36 | get the correct dependencies. | ||
| @@ -1024,7 +1024,7 @@ static int dw_i3c_master_reattach_i3c_dev(struct i3c_dev_desc *dev, · 1023 unmodified lines | |||
| 1024 | master->free_pos &= ~BIT(pos); | 1024 | master->free_pos &= ~BIT(pos); |
| 1025 | } | 1025 | } |
| 1026 | 1026 | ||
| 1027 | writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(dev->info.dyn_addr), | 1027 | writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(dev->info.dyn_addr) | DEV_ADDR_TABLE_SIR_REJECT, |
| 1028 | master->regs + | 1028 | master->regs + |
| 1029 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); | 1029 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); |
| 1030 | 1030 | ||
| @@ -1053,7 +1053,7 @@ static int dw_i3c_master_attach_i3c_dev(struct i3c_dev_desc *dev) · 22 unmodified lines | |||
| 1053 | master->free_pos &= ~BIT(pos); | 1053 | master->free_pos &= ~BIT(pos); |
| 1054 | i3c_dev_set_master_data(dev, data); | 1054 | i3c_dev_set_master_data(dev, data); |
| 1055 | 1055 | ||
| 1056 | writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(master->devs[pos].addr), | 1056 | writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(master->devs[pos].addr) | DEV_ADDR_TABLE_SIR_REJECT, |
| 1057 | master->regs + | 1057 | master->regs + |
| 1058 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); | 1058 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); |
| 1059 | 1059 | ||
| @@ -1659,6 +1659,8 @@ int dw_i3c_common_probe(struct dw_i3c_master *master, · 599 unmodified lines | |||
| 1659 | pm_runtime_get_noresume(&pdev->dev); | 1659 | pm_runtime_get_noresume(&pdev->dev); |
| 1660 | 1660 | ||
| 1661 | INIT_WORK(&master->hj_work, dw_i3c_hj_work); | 1661 | INIT_WORK(&master->hj_work, dw_i3c_hj_work); |
| 1662 | |||
| 1663 | device_set_of_node_from_dev(&master->base.i2c.dev, &pdev->dev); | ||
| 1662 | ret = i3c_master_register(&master->base, &pdev->dev, | 1664 | ret = i3c_master_register(&master->base, &pdev->dev, |
| 1663 | &dw_mipi_i3c_ops, false); | 1665 | &dw_mipi_i3c_ops, false); |
| 1664 | if (ret) | 1666 | if (ret) |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | |||
| 17 | #define CMD_0_TOC W0_BIT_(31) | 17 | #define CMD_0_TOC W0_BIT_(31) |
| 18 | #define CMD_0_ROC W0_BIT_(30) | 18 | #define CMD_0_ROC W0_BIT_(30) |
| 19 | #define CMD_0_ATTR W0_MASK(2, 0) | 19 | #define CMD_0_ATTR W0_MASK(2, 0) |
| 20 | #define CMD_0_TID W0_MASK(6, 3) | ||
| 20 | 21 | ||
| 21 | /* | 22 | /* |
| 22 | * Response Descriptor Structure | 23 | * Response Descriptor Structure |
| @@ -331,12 +331,10 @@ static int hci_cmd_v1_daa(struct i3c_hci *hci) · 330 unmodified lines | |||
| 331 | CMD_A0_ROC | CMD_A0_TOC; | 331 | CMD_A0_ROC | CMD_A0_TOC; |
| 332 | xfer->cmd_desc[1] = 0; | 332 | xfer->cmd_desc[1] = 0; |
| 333 | xfer->completion = &done; | 333 | xfer->completion = &done; |
| 334 | hci->io->queue_xfer(hci, xfer, 1); | 334 | xfer->timeout = HZ; |
| 335 | if (!wait_for_completion_timeout(&done, HZ) && | 335 | ret = i3c_hci_process_xfer(hci, xfer, 1); |
| 336 | hci->io->dequeue_xfer(hci, xfer, 1)) { | 336 | if (ret) |
| 337 | ret = -ETIME; | ||
| 338 | break; | 337 | break; |
| 339 | } | ||
| 340 | if ((RESP_STATUS(xfer->response) == RESP_ERR_ADDR_HEADER || | 338 | if ((RESP_STATUS(xfer->response) == RESP_ERR_ADDR_HEADER || |
| 341 | RESP_STATUS(xfer->response) == RESP_ERR_NACK) && | 339 | RESP_STATUS(xfer->response) == RESP_ERR_NACK) && |
| 342 | RESP_DATA_LENGTH(xfer->response) == 1) { | 340 | RESP_DATA_LENGTH(xfer->response) == 1) { |
| @@ -253,6 +253,7 @@ static int hci_cmd_v2_daa(struct i3c_hci *hci) · 252 unmodified lines | |||
| 253 | xfer[0].rnw = true; | 253 | xfer[0].rnw = true; |
| 254 | xfer[0].cmd_desc[1] = CMD_A1_DATA_LENGTH(8); | 254 | xfer[0].cmd_desc[1] = CMD_A1_DATA_LENGTH(8); |
| 255 | xfer[1].completion = &done; | 255 | xfer[1].completion = &done; |
| 256 | xfer[1].timeout = HZ; | ||
| 256 | 257 | ||
| 257 | for (;;) { | 258 | for (;;) { |
| 258 | ret = i3c_master_get_free_addr(&hci->master, next_addr); | 259 | ret = i3c_master_get_free_addr(&hci->master, next_addr); |
| @@ -272,12 +273,9 @@ static int hci_cmd_v2_daa(struct i3c_hci *hci) · 13 unmodified lines | |||
| 272 | CMD_A0_ASSIGN_ADDRESS(next_addr) | | 273 | CMD_A0_ASSIGN_ADDRESS(next_addr) | |
| 273 | CMD_A0_ROC | | 274 | CMD_A0_ROC | |
| 274 | CMD_A0_TOC; | 275 | CMD_A0_TOC; |
| 275 | hci->io->queue_xfer(hci, xfer, 2); | 276 | ret = i3c_hci_process_xfer(hci, xfer, 2); |
| 276 | if (!wait_for_completion_timeout(&done, HZ) && | 277 | if (ret) |
| 277 | hci->io->dequeue_xfer(hci, xfer, 2)) { | ||
| 278 | ret = -ETIME; | ||
| 279 | break; | 278 | break; |
| 280 | } | ||
| 281 | if (RESP_STATUS(xfer[0].response) != RESP_SUCCESS) { | 279 | if (RESP_STATUS(xfer[0].response) != RESP_SUCCESS) { |
| 282 | ret = 0; /* no more devices to be assigned */ | 280 | ret = 0; /* no more devices to be assigned */ |
| 283 | break; | 281 | break; |
| @@ -152,7 +152,11 @@ static int i3c_hci_bus_init(struct i3c_master_controller *m) · 151 unmodified lines | |||
| 152 | if (hci->quirks & HCI_QUIRK_RESP_BUF_THLD) | 152 | if (hci->quirks & HCI_QUIRK_RESP_BUF_THLD) |
| 153 | amd_set_resp_buf_thld(hci); | 153 | amd_set_resp_buf_thld(hci); |
| 154 | 154 | ||
| 155 | reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE); | 155 | scoped_guard(spinlock_irqsave, &hci->lock) |
| 156 | hci->irq_inactive = false; | ||
| 157 | |||
| 158 | /* Enable bus with Hot-Join disabled */ | ||
| 159 | reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE | HC_CONTROL_HOT_JOIN_CTRL); | ||
| 156 | dev_dbg(&hci->master.dev, "HC_CONTROL = %#x", reg_read(HC_CONTROL)); | 160 | dev_dbg(&hci->master.dev, "HC_CONTROL = %#x", reg_read(HC_CONTROL)); |
| 157 | 161 | ||
| 158 | return 0; | 162 | return 0; |
| @@ -177,21 +181,51 @@ static int i3c_hci_bus_disable(struct i3c_hci *hci) · 18 unmodified lines | |||
| 177 | return ret; | 181 | return ret; |
| 178 | } | 182 | } |
| 179 | 183 | ||
| 184 | static int i3c_hci_software_reset(struct i3c_hci *hci) | ||
| 185 | { | ||
| 186 | u32 regval; | ||
| 187 | int ret; | ||
| 188 | |||
| 189 | /* | ||
| 190 | * SOFT_RST must be clear before we write to it. | ||
| 191 | * Then we must wait until it clears again. | ||
| 192 | */ | ||
| 193 | ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | ||
| 194 | !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | ||
| 195 | if (ret) { | ||
| 196 | dev_err(&hci->master.dev, "%s: Software reset stuck\n", __func__); | ||
| 197 | return ret; | ||
| 198 | } | ||
| 199 | |||
| 200 | reg_write(RESET_CONTROL, SOFT_RST); | ||
| 201 | |||
| 202 | ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | ||
| 203 | !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | ||
| 204 | if (ret) { | ||
| 205 | dev_err(&hci->master.dev, "%s: Software reset failed\n", __func__); | ||
| 206 | return ret; | ||
| 207 | } | ||
| 208 | |||
| 209 | return 0; | ||
| 210 | } | ||
| 211 | |||
| 180 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci) | 212 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci) |
| 181 | { | 213 | { |
| 182 | struct platform_device *pdev = to_platform_device(hci->master.dev.parent); | 214 | struct platform_device *pdev = to_platform_device(hci->master.dev.parent); |
| 183 | int irq = platform_get_irq(pdev, 0); | 215 | int irq = platform_get_irq(pdev, 0); |
| 184 | 216 | ||
| 185 | reg_write(INTR_SIGNAL_ENABLE, 0x0); | 217 | reg_write(INTR_SIGNAL_ENABLE, 0x0); |
| 186 | hci->irq_inactive = true; | ||
| 187 | synchronize_irq(irq); | 218 | synchronize_irq(irq); |
| 219 | scoped_guard(spinlock_irqsave, &hci->lock) | ||
| 220 | hci->irq_inactive = true; | ||
| 188 | } | 221 | } |
| 189 | 222 | ||
| 190 | static void i3c_hci_bus_cleanup(struct i3c_master_controller *m) | 223 | static void i3c_hci_bus_cleanup(struct i3c_master_controller *m) |
| 191 | { | 224 | { |
| 192 | struct i3c_hci *hci = to_i3c_hci(m); | 225 | struct i3c_hci *hci = to_i3c_hci(m); |
| 193 | 226 | ||
| 194 | i3c_hci_bus_disable(hci); | 227 | if (i3c_hci_bus_disable(hci)) |
| 228 | i3c_hci_software_reset(hci); | ||
| 195 | hci->io->cleanup(hci); | 229 | hci->io->cleanup(hci); |
| 196 | } | 230 | } |
| 197 | 231 | ||
| @@ -212,6 +246,36 @@ void mipi_i3c_hci_dct_index_reset(struct i3c_hci *hci) · 14 unmodified lines | |||
| 212 | reg_write(DCT_SECTION, FIELD_PREP(DCT_TABLE_INDEX, 0)); | 246 | reg_write(DCT_SECTION, FIELD_PREP(DCT_TABLE_INDEX, 0)); |
| 213 | } | 247 | } |
| 214 | 248 | ||
| 249 | int i3c_hci_process_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) | ||
| 250 | { | ||
| 251 | struct completion *done = xfer[n - 1].completion; | ||
| 252 | unsigned long timeout = xfer[n - 1].timeout; | ||
| 253 | int ret; | ||
| 254 | |||
| 255 | ret = hci->io->queue_xfer(hci, xfer, n); | ||
| 256 | if (ret) | ||
| 257 | return ret; | ||
| 258 | |||
| 259 | if (!wait_for_completion_timeout(done, timeout)) { | ||
| 260 | if (hci->io->dequeue_xfer(hci, xfer, n)) { | ||
| 261 | dev_err(&hci->master.dev, "%s: timeout error\n", __func__); | ||
| 262 | return -ETIMEDOUT; | ||
| 263 | } | ||
| 264 | return 0; | ||
| 265 | } | ||
| 266 | |||
| 267 | if (hci->io->handle_error) { | ||
| 268 | bool error = false; | ||
| 269 | |||
| 270 | for (int i = 0; i < n && !error; i++) | ||
| 271 | error = RESP_STATUS(xfer[i].response); | ||
| 272 | if (error) | ||
| 273 | return hci->io->handle_error(hci, xfer, n); | ||
| 274 | } | ||
| 275 | |||
| 276 | return 0; | ||
| 277 | } | ||
| 278 | |||
| 215 | static int i3c_hci_send_ccc_cmd(struct i3c_master_controller *m, | 279 | static int i3c_hci_send_ccc_cmd(struct i3c_master_controller *m, |
| 216 | struct i3c_ccc_cmd *ccc) | 280 | struct i3c_ccc_cmd *ccc) |
| 217 | { | 281 | { |
| @@ -252,18 +316,14 @@ static int i3c_hci_send_ccc_cmd(struct i3c_master_controller *m, · 34 unmodified lines | |||
| 252 | last = i - 1; | 316 | last = i - 1; |
| 253 | xfer[last].cmd_desc[0] |= CMD_0_TOC; | 317 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 254 | xfer[last].completion = &done; | 318 | xfer[last].completion = &done; |
| 319 | xfer[last].timeout = HZ; | ||
| 255 | 320 | ||
| 256 | if (prefixed) | 321 | if (prefixed) |
| 257 | xfer--; | 322 | xfer--; |
| 258 | 323 | ||
| 259 | ret = hci->io->queue_xfer(hci, xfer, nxfers); | 324 | ret = i3c_hci_process_xfer(hci, xfer, nxfers); |
| 260 | if (ret) | 325 | if (ret) |
| 261 | goto out; | 326 | goto out; |
| 262 | if (!wait_for_completion_timeout(&done, HZ) && | ||
| 263 | hci->io->dequeue_xfer(hci, xfer, nxfers)) { | ||
| 264 | ret = -ETIME; | ||
| 265 | goto out; | ||
| 266 | } | ||
| 267 | for (i = prefixed; i < nxfers; i++) { | 327 | for (i = prefixed; i < nxfers; i++) { |
| 268 | if (ccc->rnw) | 328 | if (ccc->rnw) |
| 269 | ccc->dests[i - prefixed].payload.len = | 329 | ccc->dests[i - prefixed].payload.len = |
| @@ -334,15 +394,11 @@ static int i3c_hci_i3c_xfers(struct i3c_dev_desc *dev, · 64 unmodified lines | |||
| 334 | last = i - 1; | 394 | last = i - 1; |
| 335 | xfer[last].cmd_desc[0] |= CMD_0_TOC; | 395 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 336 | xfer[last].completion = &done; | 396 | xfer[last].completion = &done; |
| 397 | xfer[last].timeout = HZ; | ||
| 337 | 398 | ||
| 338 | ret = hci->io->queue_xfer(hci, xfer, nxfers); | 399 | ret = i3c_hci_process_xfer(hci, xfer, nxfers); |
| 339 | if (ret) | 400 | if (ret) |
| 340 | goto out; | 401 | goto out; |
| 341 | if (!wait_for_completion_timeout(&done, HZ) && | ||
| 342 | hci->io->dequeue_xfer(hci, xfer, nxfers)) { | ||
| 343 | ret = -ETIME; | ||
| 344 | goto out; | ||
| 345 | } | ||
| 346 | for (i = 0; i < nxfers; i++) { | 402 | for (i = 0; i < nxfers; i++) { |
| 347 | if (i3c_xfers[i].rnw) | 403 | if (i3c_xfers[i].rnw) |
| 348 | i3c_xfers[i].len = RESP_DATA_LENGTH(xfer[i].response); | 404 | i3c_xfers[i].len = RESP_DATA_LENGTH(xfer[i].response); |
| @@ -382,15 +438,11 @@ static int i3c_hci_i2c_xfers(struct i2c_dev_desc *dev, · 33 unmodified lines | |||
| 382 | last = i - 1; | 438 | last = i - 1; |
| 383 | xfer[last].cmd_desc[0] |= CMD_0_TOC; | 439 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 384 | xfer[last].completion = &done; | 440 | xfer[last].completion = &done; |
| 441 | xfer[last].timeout = m->i2c.timeout; | ||
| 385 | 442 | ||
| 386 | ret = hci->io->queue_xfer(hci, xfer, nxfers); | 443 | ret = i3c_hci_process_xfer(hci, xfer, nxfers); |
| 387 | if (ret) | 444 | if (ret) |
| 388 | goto out; | 445 | goto out; |
| 389 | if (!wait_for_completion_timeout(&done, m->i2c.timeout) && | ||
| 390 | hci->io->dequeue_xfer(hci, xfer, nxfers)) { | ||
| 391 | ret = -ETIME; | ||
| 392 | goto out; | ||
| 393 | } | ||
| 394 | for (i = 0; i < nxfers; i++) { | 446 | for (i = 0; i < nxfers; i++) { |
| 395 | if (RESP_STATUS(xfer[i].response) != RESP_SUCCESS) { | 447 | if (RESP_STATUS(xfer[i].response) != RESP_SUCCESS) { |
| 396 | ret = -EIO; | 448 | ret = -EIO; |
| @@ -566,6 +618,8 @@ static irqreturn_t i3c_hci_irq_handler(int irq, void *dev_id) · 169 unmodified lines | |||
| 566 | irqreturn_t result = IRQ_NONE; | 618 | irqreturn_t result = IRQ_NONE; |
| 567 | u32 val; | 619 | u32 val; |
| 568 | 620 | ||
| 621 | guard(spinlock)(&hci->lock); | ||
| 622 | |||
| 569 | /* | 623 | /* |
| 570 | * The IRQ can be shared, so the handler may be called when the IRQ is | 624 | * The IRQ can be shared, so the handler may be called when the IRQ is |
| 571 | * due to a different device. That could happen when runtime suspended, | 625 | * due to a different device. That could happen when runtime suspended, |
| @@ -601,34 +655,6 @@ static irqreturn_t i3c_hci_irq_handler(int irq, void *dev_id) · 29 unmodified lines | |||
| 601 | return result; | 655 | return result; |
| 602 | } | 656 | } |
| 603 | 657 | ||
| 604 | static int i3c_hci_software_reset(struct i3c_hci *hci) | ||
| 605 | { | ||
| 606 | u32 regval; | ||
| 607 | int ret; | ||
| 608 | |||
| 609 | /* | ||
| 610 | * SOFT_RST must be clear before we write to it. | ||
| 611 | * Then we must wait until it clears again. | ||
| 612 | */ | ||
| 613 | ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | ||
| 614 | !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | ||
| 615 | if (ret) { | ||
| 616 | dev_err(&hci->master.dev, "%s: Software reset stuck\n", __func__); | ||
| 617 | return ret; | ||
| 618 | } | ||
| 619 | |||
| 620 | reg_write(RESET_CONTROL, SOFT_RST); | ||
| 621 | |||
| 622 | ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | ||
| 623 | !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | ||
| 624 | if (ret) { | ||
| 625 | dev_err(&hci->master.dev, "%s: Software reset failed\n", __func__); | ||
| 626 | return ret; | ||
| 627 | } | ||
| 628 | |||
| 629 | return 0; | ||
| 630 | } | ||
| 631 | |||
| 632 | static inline bool is_version_1_1_or_newer(struct i3c_hci *hci) | 658 | static inline bool is_version_1_1_or_newer(struct i3c_hci *hci) |
| 633 | { | 659 | { |
| 634 | return hci->version_major > 1 || (hci->version_major == 1 && hci->version_minor > 0); | 660 | return hci->version_major > 1 || (hci->version_major == 1 && hci->version_minor > 0); |
| @@ -739,8 +765,12 @@ static int i3c_hci_runtime_suspend(struct device *dev) · 104 unmodified lines | |||
| 739 | int ret; | 765 | int ret; |
| 740 | 766 | ||
| 741 | ret = i3c_hci_bus_disable(hci); | 767 | ret = i3c_hci_bus_disable(hci); |
| 742 | if (ret) | 768 | if (ret) { |
| 769 | /* Fall back to software reset to disable the bus */ | ||
| 770 | ret = i3c_hci_software_reset(hci); | ||
| 771 | i3c_hci_sync_irq_inactive(hci); | ||
| 743 | return ret; | 772 | return ret; |
| 773 | } | ||
| 744 | 774 | ||
| 745 | hci->io->suspend(hci); | 775 | hci->io->suspend(hci); |
| 746 | 776 | ||
| @@ -760,11 +790,13 @@ static int i3c_hci_runtime_resume(struct device *dev) · 13 unmodified lines | |||
| 760 | 790 | ||
| 761 | mipi_i3c_hci_dat_v1.restore(hci); | 791 | mipi_i3c_hci_dat_v1.restore(hci); |
| 762 | 792 | ||
| 763 | hci->irq_inactive = false; | ||
| 764 | |||
| 765 | hci->io->resume(hci); | 793 | hci->io->resume(hci); |
| 766 | 794 | ||
| 767 | reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE); | 795 | scoped_guard(spinlock_irqsave, &hci->lock) |
| 796 | hci->irq_inactive = false; | ||
| 797 | |||
| 798 | /* Enable bus with Hot-Join disabled */ | ||
| 799 | reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE | HC_CONTROL_HOT_JOIN_CTRL); | ||
| 768 | 800 | ||
| 769 | return 0; | 801 | return 0; |
| 770 | } | 802 | } |
| @@ -924,6 +956,9 @@ static int i3c_hci_probe(struct platform_device *pdev) · 153 unmodified lines | |||
| 924 | if (!hci) | 956 | if (!hci) |
| 925 | return -ENOMEM; | 957 | return -ENOMEM; |
| 926 | 958 | ||
| 959 | spin_lock_init(&hci->lock); | ||
| 960 | mutex_init(&hci->control_mutex); | ||
| 961 | |||
| 927 | /* | 962 | /* |
| 928 | * Multi-bus instances share the same MMIO address range, but not | 963 | * Multi-bus instances share the same MMIO address range, but not |
| 929 | * necessarily in separate contiguous sub-ranges. To avoid overlapping | 964 | * necessarily in separate contiguous sub-ranges. To avoid overlapping |
| @@ -950,6 +985,8 @@ static int i3c_hci_probe(struct platform_device *pdev) · 20 unmodified lines | |||
| 950 | if (ret) | 985 | if (ret) |
| 951 | return ret; | 986 | return ret; |
| 952 | 987 | ||
| 988 | hci->irq_inactive = true; | ||
| 989 | |||
| 953 | irq = platform_get_irq(pdev, 0); | 990 | irq = platform_get_irq(pdev, 0); |
| 954 | ret = devm_request_irq(&pdev->dev, irq, i3c_hci_irq_handler, | 991 | ret = devm_request_irq(&pdev->dev, irq, i3c_hci_irq_handler, |
| 955 | IRQF_SHARED, NULL, hci); | 992 | IRQF_SHARED, NULL, hci); |
| @@ -129,9 +129,8 @@ struct hci_rh_data { · 128 unmodified lines | |||
| 129 | dma_addr_t xfer_dma, resp_dma, ibi_status_dma, ibi_data_dma; | 129 | dma_addr_t xfer_dma, resp_dma, ibi_status_dma, ibi_data_dma; |
| 130 | unsigned int xfer_entries, ibi_status_entries, ibi_chunks_total; | 130 | unsigned int xfer_entries, ibi_status_entries, ibi_chunks_total; |
| 131 | unsigned int xfer_struct_sz, resp_struct_sz, ibi_status_sz, ibi_chunk_sz; | 131 | unsigned int xfer_struct_sz, resp_struct_sz, ibi_status_sz, ibi_chunk_sz; |
| 132 | unsigned int done_ptr, ibi_chunk_ptr; | 132 | unsigned int done_ptr, ibi_chunk_ptr, xfer_space; |
| 133 | struct hci_xfer **src_xfers; | 133 | struct hci_xfer **src_xfers; |
| 134 | spinlock_t lock; | ||
| 135 | struct completion op_done; | 134 | struct completion op_done; |
| 136 | }; | 135 | }; |
| 137 | 136 | ||
| @@ -261,6 +260,7 @@ ring_ready: · 123 unmodified lines | |||
| 261 | 260 | ||
| 262 | rh->done_ptr = 0; | 261 | rh->done_ptr = 0; |
| 263 | rh->ibi_chunk_ptr = 0; | 262 | rh->ibi_chunk_ptr = 0; |
| 263 | rh->xfer_space = rh->xfer_entries; | ||
| 264 | } | 264 | } |
| 265 | 265 | ||
| 266 | static void hci_dma_init_rings(struct i3c_hci *hci) | 266 | static void hci_dma_init_rings(struct i3c_hci *hci) |
| @@ -344,7 +344,6 @@ static int hci_dma_init(struct i3c_hci *hci) · 77 unmodified lines | |||
| 344 | goto err_out; | 344 | goto err_out; |
| 345 | rh = &rings->headers[i]; | 345 | rh = &rings->headers[i]; |
| 346 | rh->regs = hci->base_regs + offset; | 346 | rh->regs = hci->base_regs + offset; |
| 347 | spin_lock_init(&rh->lock); | ||
| 348 | init_completion(&rh->op_done); | 347 | init_completion(&rh->op_done); |
| 349 | 348 | ||
| 350 | rh->xfer_entries = XFER_RING_ENTRIES; | 349 | rh->xfer_entries = XFER_RING_ENTRIES; |
| @@ -439,26 +438,63 @@ static void hci_dma_unmap_xfer(struct i3c_hci *hci, · 88 unmodified lines | |||
| 439 | } | 438 | } |
| 440 | } | 439 | } |
| 441 | 440 | ||
| 441 | static struct i3c_dma *hci_dma_map_xfer(struct device *dev, struct hci_xfer *xfer) | ||
| 442 | { | ||
| 443 | enum dma_data_direction dir = xfer->rnw ? DMA_FROM_DEVICE : DMA_TO_DEVICE; | ||
| 444 | bool need_bounce = device_iommu_mapped(dev) && xfer->rnw && (xfer->data_len & 3); | ||
| 445 | |||
| 446 | return i3c_master_dma_map_single(dev, xfer->data, xfer->data_len, need_bounce, dir); | ||
| 447 | } | ||
| 448 | |||
| 449 | static int hci_dma_map_xfer_list(struct i3c_hci *hci, struct device *dev, | ||
| 450 | struct hci_xfer *xfer_list, int n) | ||
| 451 | { | ||
| 452 | for (int i = 0; i < n; i++) { | ||
| 453 | struct hci_xfer *xfer = xfer_list + i; | ||
| 454 | |||
| 455 | if (!xfer->data) | ||
| 456 | continue; | ||
| 457 | |||
| 458 | xfer->dma = hci_dma_map_xfer(dev, xfer); | ||
| 459 | if (!xfer->dma) { | ||
| 460 | hci_dma_unmap_xfer(hci, xfer_list, i); | ||
| 461 | return -ENOMEM; | ||
| 462 | } | ||
| 463 | } | ||
| 464 | |||
| 465 | return 0; | ||
| 466 | } | ||
| 467 | |||
| 442 | static int hci_dma_queue_xfer(struct i3c_hci *hci, | 468 | static int hci_dma_queue_xfer(struct i3c_hci *hci, |
| 443 | struct hci_xfer *xfer_list, int n) | 469 | struct hci_xfer *xfer_list, int n) |
| 444 | { | 470 | { |
| 445 | struct hci_rings_data *rings = hci->io_data; | 471 | struct hci_rings_data *rings = hci->io_data; |
| 446 | struct hci_rh_data *rh; | 472 | struct hci_rh_data *rh; |
| 447 | unsigned int i, ring, enqueue_ptr; | 473 | unsigned int i, ring, enqueue_ptr; |
| 448 | u32 op1_val, op2_val; | 474 | u32 op1_val; |
| 475 | int ret; | ||
| 476 | |||
| 477 | ret = hci_dma_map_xfer_list(hci, rings->sysdev, xfer_list, n); | ||
| 478 | if (ret) | ||
| 479 | return ret; | ||
| 449 | 480 | ||
| 450 | /* For now we only use ring 0 */ | 481 | /* For now we only use ring 0 */ |
| 451 | ring = 0; | 482 | ring = 0; |
| 452 | rh = &rings->headers[ring]; | 483 | rh = &rings->headers[ring]; |
| 453 | 484 | ||
| 485 | spin_lock_irq(&hci->lock); | ||
| 486 | |||
| 487 | if (n > rh->xfer_space) { | ||
| 488 | spin_unlock_irq(&hci->lock); | ||
| 489 | hci_dma_unmap_xfer(hci, xfer_list, n); | ||
| 490 | return -EBUSY; | ||
| 491 | } | ||
| 492 | |||
| 454 | op1_val = rh_reg_read(RING_OPERATION1); | 493 | op1_val = rh_reg_read(RING_OPERATION1); |
| 455 | enqueue_ptr = FIELD_GET(RING_OP1_CR_ENQ_PTR, op1_val); | 494 | enqueue_ptr = FIELD_GET(RING_OP1_CR_ENQ_PTR, op1_val); |
| 456 | for (i = 0; i < n; i++) { | 495 | for (i = 0; i < n; i++) { |
| 457 | struct hci_xfer *xfer = xfer_list + i; | 496 | struct hci_xfer *xfer = xfer_list + i; |
| 458 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * enqueue_ptr; | 497 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * enqueue_ptr; |
| 459 | enum dma_data_direction dir = xfer->rnw ? DMA_FROM_DEVICE : | ||
| 460 | DMA_TO_DEVICE; | ||
| 461 | bool need_bounce; | ||
| 462 | 498 | ||
| 463 | /* store cmd descriptor */ | 499 | /* store cmd descriptor */ |
| 464 | *ring_data++ = xfer->cmd_desc[0]; | 500 | *ring_data++ = xfer->cmd_desc[0]; |
| @@ -477,18 +513,6 @@ static int hci_dma_queue_xfer(struct i3c_hci *hci, · 12 unmodified lines | |||
| 477 | 513 | ||
| 478 | /* 2nd and 3rd words of Data Buffer Descriptor Structure */ | 514 | /* 2nd and 3rd words of Data Buffer Descriptor Structure */ |
| 479 | if (xfer->data) { | 515 | if (xfer->data) { |
| 480 | need_bounce = device_iommu_mapped(rings->sysdev) && | ||
| 481 | xfer->rnw && | ||
| 482 | xfer->data_len != ALIGN(xfer->data_len, 4); | ||
| 483 | xfer->dma = i3c_master_dma_map_single(rings->sysdev, | ||
| 484 | xfer->data, | ||
| 485 | xfer->data_len, | ||
| 486 | need_bounce, | ||
| 487 | dir); | ||
| 488 | if (!xfer->dma) { | ||
| 489 | hci_dma_unmap_xfer(hci, xfer_list, i); | ||
| 490 | return -ENOMEM; | ||
| 491 | } | ||
| 492 | *ring_data++ = lower_32_bits(xfer->dma->addr); | 516 | *ring_data++ = lower_32_bits(xfer->dma->addr); |
| 493 | *ring_data++ = upper_32_bits(xfer->dma->addr); | 517 | *ring_data++ = upper_32_bits(xfer->dma->addr); |
| 494 | } else { | 518 | } else { |
| @@ -503,26 +527,14 @@ static int hci_dma_queue_xfer(struct i3c_hci *hci, · 8 unmodified lines | |||
| 503 | xfer->ring_entry = enqueue_ptr; | 527 | xfer->ring_entry = enqueue_ptr; |
| 504 | 528 | ||
| 505 | enqueue_ptr = (enqueue_ptr + 1) % rh->xfer_entries; | 529 | enqueue_ptr = (enqueue_ptr + 1) % rh->xfer_entries; |
| 506 | |||
| 507 | /* | ||
| 508 | * We may update the hardware view of the enqueue pointer | ||
| 509 | * only if we didn't reach its dequeue pointer. | ||
| 510 | */ | ||
| 511 | op2_val = rh_reg_read(RING_OPERATION2); | ||
| 512 | if (enqueue_ptr == FIELD_GET(RING_OP2_CR_DEQ_PTR, op2_val)) { | ||
| 513 | /* the ring is full */ | ||
| 514 | hci_dma_unmap_xfer(hci, xfer_list, i + 1); | ||
| 515 | return -EBUSY; | ||
| 516 | } | ||
| 517 | } | 530 | } |
| 518 | 531 | ||
| 519 | /* take care to update the hardware enqueue pointer atomically */ | 532 | rh->xfer_space -= n; |
| 520 | spin_lock_irq(&rh->lock); | 533 | |
| 521 | op1_val = rh_reg_read(RING_OPERATION1); | ||
| 522 | op1_val &= ~RING_OP1_CR_ENQ_PTR; | 534 | op1_val &= ~RING_OP1_CR_ENQ_PTR; |
| 523 | op1_val |= FIELD_PREP(RING_OP1_CR_ENQ_PTR, enqueue_ptr); | 535 | op1_val |= FIELD_PREP(RING_OP1_CR_ENQ_PTR, enqueue_ptr); |
| 524 | rh_reg_write(RING_OPERATION1, op1_val); | 536 | rh_reg_write(RING_OPERATION1, op1_val); |
| 525 | spin_unlock_irq(&rh->lock); | 537 | spin_unlock_irq(&hci->lock); |
| 526 | 538 | ||
| 527 | return 0; | 539 | return 0; |
| 528 | } | 540 | } |
| @@ -534,18 +546,29 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 5 unmodified lines | |||
| 534 | struct hci_rh_data *rh = &rings->headers[xfer_list[0].ring_number]; | 546 | struct hci_rh_data *rh = &rings->headers[xfer_list[0].ring_number]; |
| 535 | unsigned int i; | 547 | unsigned int i; |
| 536 | bool did_unqueue = false; | 548 | bool did_unqueue = false; |
| 537 | 549 | u32 ring_status; | |
| 538 | /* stop the ring */ | 550 | |
| 539 | rh_reg_write(RING_CONTROL, RING_CTRL_ABORT); | 551 | guard(mutex)(&hci->control_mutex); |
| 540 | if (wait_for_completion_timeout(&rh->op_done, HZ) == 0) { | 552 | |
| 541 | /* | 553 | ring_status = rh_reg_read(RING_STATUS); |
| 542 | * We're deep in it if ever this condition is ever met. | 554 | if (ring_status & RING_STATUS_RUNNING) { |
| 543 | * Hardware might still be writing to memory, etc. | 555 | /* stop the ring */ |
| 544 | */ | 556 | reinit_completion(&rh->op_done); |
| 545 | dev_crit(&hci->master.dev, "unable to abort the ring\n"); | 557 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | RING_CTRL_ABORT); |
| 546 | WARN_ON(1); | 558 | wait_for_completion_timeout(&rh->op_done, HZ); |
| 559 | ring_status = rh_reg_read(RING_STATUS); | ||
| 560 | if (ring_status & RING_STATUS_RUNNING) { | ||
| 561 | /* | ||
| 562 | * We're deep in it if ever this condition is ever met. | ||
| 563 | * Hardware might still be writing to memory, etc. | ||
| 564 | */ | ||
| 565 | dev_crit(&hci->master.dev, "unable to abort the ring\n"); | ||
| 566 | WARN_ON(1); | ||
| 567 | } | ||
| 547 | } | 568 | } |
| 548 | 569 | ||
| 570 | spin_lock_irq(&hci->lock); | ||
| 571 | |||
| 549 | for (i = 0; i < n; i++) { | 572 | for (i = 0; i < n; i++) { |
| 550 | struct hci_xfer *xfer = xfer_list + i; | 573 | struct hci_xfer *xfer = xfer_list + i; |
| 551 | int idx = xfer->ring_entry; | 574 | int idx = xfer->ring_entry; |
| @@ -559,7 +582,7 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 7 unmodified lines | |||
| 559 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * idx; | 582 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * idx; |
| 560 | 583 | ||
| 561 | /* store no-op cmd descriptor */ | 584 | /* store no-op cmd descriptor */ |
| 562 | *ring_data++ = FIELD_PREP(CMD_0_ATTR, 0x7); | 585 | *ring_data++ = FIELD_PREP(CMD_0_ATTR, 0x7) | FIELD_PREP(CMD_0_TID, xfer->cmd_tid); |
| 563 | *ring_data++ = 0; | 586 | *ring_data++ = 0; |
| 564 | if (hci->cmd == &mipi_i3c_hci_cmd_v2) { | 587 | if (hci->cmd == &mipi_i3c_hci_cmd_v2) { |
| 565 | *ring_data++ = 0; | 588 | *ring_data++ = 0; |
| @@ -577,15 +600,25 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 11 unmodified lines | |||
| 577 | } | 600 | } |
| 578 | 601 | ||
| 579 | /* restart the ring */ | 602 | /* restart the ring */ |
| 603 | mipi_i3c_hci_resume(hci); | ||
| 580 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE); | 604 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE); |
| 605 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | RING_CTRL_RUN_STOP); | ||
| 606 | |||
| 607 | spin_unlock_irq(&hci->lock); | ||
| 581 | 608 | ||
| 582 | return did_unqueue; | 609 | return did_unqueue; |
| 583 | } | 610 | } |
| 584 | 611 | ||
| 612 | static int hci_dma_handle_error(struct i3c_hci *hci, struct hci_xfer *xfer_list, int n) | ||
| 613 | { | ||
| 614 | return hci_dma_dequeue_xfer(hci, xfer_list, n) ? -EIO : 0; | ||
| 615 | } | ||
| 616 | |||
| 585 | static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) | 617 | static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) |
| 586 | { | 618 | { |
| 587 | u32 op1_val, op2_val, resp, *ring_resp; | 619 | u32 op1_val, op2_val, resp, *ring_resp; |
| 588 | unsigned int tid, done_ptr = rh->done_ptr; | 620 | unsigned int tid, done_ptr = rh->done_ptr; |
| 621 | unsigned int done_cnt = 0; | ||
| 589 | struct hci_xfer *xfer; | 622 | struct hci_xfer *xfer; |
| 590 | 623 | ||
| 591 | for (;;) { | 624 | for (;;) { |
| @@ -603,6 +636,7 @@ static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) · 11 unmodified lines | |||
| 603 | dev_dbg(&hci->master.dev, "orphaned ring entry"); | 636 | dev_dbg(&hci->master.dev, "orphaned ring entry"); |
| 604 | } else { | 637 | } else { |
| 605 | hci_dma_unmap_xfer(hci, xfer, 1); | 638 | hci_dma_unmap_xfer(hci, xfer, 1); |
| 639 | rh->src_xfers[done_ptr] = NULL; | ||
| 606 | xfer->ring_entry = -1; | 640 | xfer->ring_entry = -1; |
| 607 | xfer->response = resp; | 641 | xfer->response = resp; |
| 608 | if (tid != xfer->cmd_tid) { | 642 | if (tid != xfer->cmd_tid) { |
| @@ -617,15 +651,14 @@ static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) · 8 unmodified lines | |||
| 617 | 651 | ||
| 618 | done_ptr = (done_ptr + 1) % rh->xfer_entries; | 652 | done_ptr = (done_ptr + 1) % rh->xfer_entries; |
| 619 | rh->done_ptr = done_ptr; | 653 | rh->done_ptr = done_ptr; |
| 654 | done_cnt += 1; | ||
| 620 | } | 655 | } |
| 621 | 656 | ||
| 622 | /* take care to update the software dequeue pointer atomically */ | 657 | rh->xfer_space += done_cnt; |
| 623 | spin_lock(&rh->lock); | ||
| 624 | op1_val = rh_reg_read(RING_OPERATION1); | 658 | op1_val = rh_reg_read(RING_OPERATION1); |
| 625 | op1_val &= ~RING_OP1_CR_SW_DEQ_PTR; | 659 | op1_val &= ~RING_OP1_CR_SW_DEQ_PTR; |
| 626 | op1_val |= FIELD_PREP(RING_OP1_CR_SW_DEQ_PTR, done_ptr); | 660 | op1_val |= FIELD_PREP(RING_OP1_CR_SW_DEQ_PTR, done_ptr); |
| 627 | rh_reg_write(RING_OPERATION1, op1_val); | 661 | rh_reg_write(RING_OPERATION1, op1_val); |
| 628 | spin_unlock(&rh->lock); | ||
| 629 | } | 662 | } |
| 630 | 663 | ||
| 631 | static int hci_dma_request_ibi(struct i3c_hci *hci, struct i3c_dev_desc *dev, | 664 | static int hci_dma_request_ibi(struct i3c_hci *hci, struct i3c_dev_desc *dev, |
| @@ -805,13 +838,10 @@ static void hci_dma_process_ibi(struct i3c_hci *hci, struct hci_rh_data *rh) · 173 unmodified lines | |||
| 805 | i3c_master_queue_ibi(dev, slot); | 838 | i3c_master_queue_ibi(dev, slot); |
| 806 | 839 | ||
| 807 | done: | 840 | done: |
| 808 | /* take care to update the ibi dequeue pointer atomically */ | ||
| 809 | spin_lock(&rh->lock); | ||
| 810 | op1_val = rh_reg_read(RING_OPERATION1); | 841 | op1_val = rh_reg_read(RING_OPERATION1); |
| 811 | op1_val &= ~RING_OP1_IBI_DEQ_PTR; | 842 | op1_val &= ~RING_OP1_IBI_DEQ_PTR; |
| 812 | op1_val |= FIELD_PREP(RING_OP1_IBI_DEQ_PTR, deq_ptr); | 843 | op1_val |= FIELD_PREP(RING_OP1_IBI_DEQ_PTR, deq_ptr); |
| 813 | rh_reg_write(RING_OPERATION1, op1_val); | 844 | rh_reg_write(RING_OPERATION1, op1_val); |
| 814 | spin_unlock(&rh->lock); | ||
| 815 | 845 | ||
| 816 | /* update the chunk pointer */ | 846 | /* update the chunk pointer */ |
| 817 | rh->ibi_chunk_ptr += ibi_chunks; | 847 | rh->ibi_chunk_ptr += ibi_chunks; |
| @@ -845,29 +875,8 @@ static bool hci_dma_irq_handler(struct i3c_hci *hci) · 27 unmodified lines | |||
| 845 | hci_dma_xfer_done(hci, rh); | 875 | hci_dma_xfer_done(hci, rh); |
| 846 | if (status & INTR_RING_OP) | 876 | if (status & INTR_RING_OP) |
| 847 | complete(&rh->op_done); | 877 | complete(&rh->op_done); |
| 848 | 878 | if (status & INTR_TRANSFER_ABORT) | |
| 849 | if (status & INTR_TRANSFER_ABORT) { | 879 | dev_dbg(&hci->master.dev, "Ring %d: Transfer Aborted\n", i); |
| 850 | u32 ring_status; | ||
| 851 | |||
| 852 | dev_notice_ratelimited(&hci->master.dev, | ||
| 853 | "Ring %d: Transfer Aborted\n", i); | ||
| 854 | mipi_i3c_hci_resume(hci); | ||
| 855 | ring_status = rh_reg_read(RING_STATUS); | ||
| 856 | if (!(ring_status & RING_STATUS_RUNNING) && | ||
| 857 | status & INTR_TRANSFER_COMPLETION && | ||
| 858 | status & INTR_TRANSFER_ERR) { | ||
| 859 | /* | ||
| 860 | * Ring stop followed by run is an Intel | ||
| 861 | * specific required quirk after resuming the | ||
| 862 | * halted controller. Do it only when the ring | ||
| 863 | * is not in running state after a transfer | ||
| 864 | * error. | ||
| 865 | */ | ||
| 866 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE); | ||
| 867 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | | ||
| 868 | RING_CTRL_RUN_STOP); | ||
| 869 | } | ||
| 870 | } | ||
| 871 | if (status & INTR_IBI_RING_FULL) | 880 | if (status & INTR_IBI_RING_FULL) |
| 872 | dev_err_ratelimited(&hci->master.dev, | 881 | dev_err_ratelimited(&hci->master.dev, |
| 873 | "Ring %d: IBI Ring Full Condition\n", i); | 882 | "Ring %d: IBI Ring Full Condition\n", i); |
| @@ -883,6 +892,7 @@ const struct hci_io_ops mipi_i3c_hci_dma = { · 9 unmodified lines | |||
| 883 | .cleanup = hci_dma_cleanup, | 892 | .cleanup = hci_dma_cleanup, |
| 884 | .queue_xfer = hci_dma_queue_xfer, | 893 | .queue_xfer = hci_dma_queue_xfer, |
| 885 | .dequeue_xfer = hci_dma_dequeue_xfer, | 894 | .dequeue_xfer = hci_dma_dequeue_xfer, |
| 895 | .handle_error = hci_dma_handle_error, | ||
| 886 | .irq_handler = hci_dma_irq_handler, | 896 | .irq_handler = hci_dma_irq_handler, |
| 887 | .request_ibi = hci_dma_request_ibi, | 897 | .request_ibi = hci_dma_request_ibi, |
| 888 | .free_ibi = hci_dma_free_ibi, | 898 | .free_ibi = hci_dma_free_ibi, |
| @@ -50,6 +50,8 @@ struct i3c_hci { · 49 unmodified lines | |||
| 50 | const struct hci_io_ops *io; | 50 | const struct hci_io_ops *io; |
| 51 | void *io_data; | 51 | void *io_data; |
| 52 | const struct hci_cmd_ops *cmd; | 52 | const struct hci_cmd_ops *cmd; |
| 53 | spinlock_t lock; | ||
| 54 | struct mutex control_mutex; | ||
| 53 | atomic_t next_cmd_tid; | 55 | atomic_t next_cmd_tid; |
| 54 | bool irq_inactive; | 56 | bool irq_inactive; |
| 55 | u32 caps; | 57 | u32 caps; |
| @@ -87,6 +89,7 @@ struct hci_xfer { · 31 unmodified lines | |||
| 87 | unsigned int data_len; | 89 | unsigned int data_len; |
| 88 | unsigned int cmd_tid; | 90 | unsigned int cmd_tid; |
| 89 | struct completion *completion; | 91 | struct completion *completion; |
| 92 | unsigned long timeout; | ||
| 90 | union { | 93 | union { |
| 91 | struct { | 94 | struct { |
| 92 | /* PIO specific */ | 95 | /* PIO specific */ |
| @@ -120,6 +123,7 @@ struct hci_io_ops { · 27 unmodified lines | |||
| 120 | bool (*irq_handler)(struct i3c_hci *hci); | 123 | bool (*irq_handler)(struct i3c_hci *hci); |
| 121 | int (*queue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | 124 | int (*queue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); |
| 122 | bool (*dequeue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | 125 | bool (*dequeue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); |
| 126 | int (*handle_error)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | ||
| 123 | int (*request_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev, | 127 | int (*request_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev, |
| 124 | const struct i3c_ibi_setup *req); | 128 | const struct i3c_ibi_setup *req); |
| 125 | void (*free_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev); | 129 | void (*free_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev); |
| @@ -154,5 +158,6 @@ void mipi_i3c_hci_dct_index_reset(struct i3c_hci *hci); · 28 unmodified lines | |||
| 154 | void amd_set_od_pp_timing(struct i3c_hci *hci); | 158 | void amd_set_od_pp_timing(struct i3c_hci *hci); |
| 155 | void amd_set_resp_buf_thld(struct i3c_hci *hci); | 159 | void amd_set_resp_buf_thld(struct i3c_hci *hci); |
| 156 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci); | 160 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci); |
| 161 | int i3c_hci_process_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | ||
| 157 | 162 | ||
| 158 | #endif | 163 | #endif |
| @@ -123,7 +123,6 @@ struct hci_pio_ibi_data { · 122 unmodified lines | |||
| 123 | }; | 123 | }; |
| 124 | 124 | ||
| 125 | struct hci_pio_data { | 125 | struct hci_pio_data { |
| 126 | spinlock_t lock; | ||
| 127 | struct hci_xfer *curr_xfer, *xfer_queue; | 126 | struct hci_xfer *curr_xfer, *xfer_queue; |
| 128 | struct hci_xfer *curr_rx, *rx_queue; | 127 | struct hci_xfer *curr_rx, *rx_queue; |
| 129 | struct hci_xfer *curr_tx, *tx_queue; | 128 | struct hci_xfer *curr_tx, *tx_queue; |
| @@ -212,7 +211,6 @@ static int hci_pio_init(struct i3c_hci *hci) · 82 unmodified lines | |||
| 212 | return -ENOMEM; | 211 | return -ENOMEM; |
| 213 | 212 | ||
| 214 | hci->io_data = pio; | 213 | hci->io_data = pio; |
| 215 | spin_lock_init(&pio->lock); | ||
| 216 | 214 | ||
| 217 | __hci_pio_init(hci, &size_val); | 215 | __hci_pio_init(hci, &size_val); |
| 218 | 216 | ||
| @@ -631,7 +629,7 @@ static int hci_pio_queue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) · 412 unmodified lines | |||
| 631 | xfer[i].data_left = xfer[i].data_len; | 629 | xfer[i].data_left = xfer[i].data_len; |
| 632 | } | 630 | } |
| 633 | 631 | ||
| 634 | spin_lock_irq(&pio->lock); | 632 | spin_lock_irq(&hci->lock); |
| 635 | prev_queue_tail = pio->xfer_queue; | 633 | prev_queue_tail = pio->xfer_queue; |
| 636 | pio->xfer_queue = &xfer[n - 1]; | 634 | pio->xfer_queue = &xfer[n - 1]; |
| 637 | if (pio->curr_xfer) { | 635 | if (pio->curr_xfer) { |
| @@ -645,7 +643,7 @@ static int hci_pio_queue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) · 7 unmodified lines | |||
| 645 | pio_reg_read(INTR_STATUS), | 643 | pio_reg_read(INTR_STATUS), |
| 646 | pio_reg_read(INTR_SIGNAL_ENABLE)); | 644 | pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 647 | } | 645 | } |
| 648 | spin_unlock_irq(&pio->lock); | 646 | spin_unlock_irq(&hci->lock); |
| 649 | return 0; | 647 | return 0; |
| 650 | } | 648 | } |
| 651 | 649 | ||
| @@ -716,14 +714,14 @@ static bool hci_pio_dequeue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int · 64 unmodified lines | |||
| 716 | struct hci_pio_data *pio = hci->io_data; | 714 | struct hci_pio_data *pio = hci->io_data; |
| 717 | int ret; | 715 | int ret; |
| 718 | 716 | ||
| 719 | spin_lock_irq(&pio->lock); | 717 | spin_lock_irq(&hci->lock); |
| 720 | dev_dbg(&hci->master.dev, "n=%d status=%#x/%#x", n, | 718 | dev_dbg(&hci->master.dev, "n=%d status=%#x/%#x", n, |
| 721 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); | 719 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 722 | dev_dbg(&hci->master.dev, "main_status = %#x/%#x", | 720 | dev_dbg(&hci->master.dev, "main_status = %#x/%#x", |
| 723 | readl(hci->base_regs + 0x20), readl(hci->base_regs + 0x28)); | 721 | readl(hci->base_regs + 0x20), readl(hci->base_regs + 0x28)); |
| 724 | 722 | ||
| 725 | ret = hci_pio_dequeue_xfer_common(hci, pio, xfer, n); | 723 | ret = hci_pio_dequeue_xfer_common(hci, pio, xfer, n); |
| 726 | spin_unlock_irq(&pio->lock); | 724 | spin_unlock_irq(&hci->lock); |
| 727 | return ret; | 725 | return ret; |
| 728 | } | 726 | } |
| 729 | 727 | ||
| @@ -1016,15 +1014,12 @@ static bool hci_pio_irq_handler(struct i3c_hci *hci) · 286 unmodified lines | |||
| 1016 | struct hci_pio_data *pio = hci->io_data; | 1014 | struct hci_pio_data *pio = hci->io_data; |
| 1017 | u32 status; | 1015 | u32 status; |
| 1018 | 1016 | ||
| 1019 | spin_lock(&pio->lock); | ||
| 1020 | status = pio_reg_read(INTR_STATUS); | 1017 | status = pio_reg_read(INTR_STATUS); |
| 1021 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", | 1018 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", |
| 1022 | status, pio->enabled_irqs); | 1019 | status, pio->enabled_irqs); |
| 1023 | status &= pio->enabled_irqs | STAT_LATENCY_WARNINGS; | 1020 | status &= pio->enabled_irqs | STAT_LATENCY_WARNINGS; |
| 1024 | if (!status) { | 1021 | if (!status) |
| 1025 | spin_unlock(&pio->lock); | ||
| 1026 | return false; | 1022 | return false; |
| 1027 | } | ||
| 1028 | 1023 | ||
| 1029 | if (status & STAT_IBI_STATUS_THLD) | 1024 | if (status & STAT_IBI_STATUS_THLD) |
| 1030 | hci_pio_process_ibi(hci, pio); | 1025 | hci_pio_process_ibi(hci, pio); |
| @@ -1058,7 +1053,6 @@ static bool hci_pio_irq_handler(struct i3c_hci *hci) · 27 unmodified lines | |||
| 1058 | pio_reg_write(INTR_SIGNAL_ENABLE, pio->enabled_irqs); | 1053 | pio_reg_write(INTR_SIGNAL_ENABLE, pio->enabled_irqs); |
| 1059 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", | 1054 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", |
| 1060 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); | 1055 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 1061 | spin_unlock(&pio->lock); | ||
| 1062 | return true; | 1056 | return true; |
| 1063 | } | 1057 | } |
| 1064 | 1058 | ||
| @@ -541,7 +541,7 @@ static int ad7768_reg_access(struct iio_dev *indio_dev, · 540 unmodified lines | |||
| 541 | return ret; | 541 | return ret; |
| 542 | } | 542 | } |
| 543 | 543 | ||
| 544 | static void ad7768_fill_scale_tbl(struct iio_dev *dev) | 544 | static int ad7768_fill_scale_tbl(struct iio_dev *dev) |
| 545 | { | 545 | { |
| 546 | struct ad7768_state *st = iio_priv(dev); | 546 | struct ad7768_state *st = iio_priv(dev); |
| 547 | const struct iio_scan_type *scan_type; | 547 | const struct iio_scan_type *scan_type; |
| @@ -551,6 +551,11 @@ static void ad7768_fill_scale_tbl(struct iio_dev *dev) · 3 unmodified lines | |||
| 551 | u64 tmp2; | 551 | u64 tmp2; |
| 552 | 552 | ||
| 553 | scan_type = iio_get_current_scan_type(dev, &dev->channels[0]); | 553 | scan_type = iio_get_current_scan_type(dev, &dev->channels[0]); |
| 554 | if (IS_ERR(scan_type)) { | ||
| 555 | dev_err(&st->spi->dev, "Failed to get scan type.\n"); | ||
| 556 | return PTR_ERR(scan_type); | ||
| 557 | } | ||
| 558 | |||
| 554 | if (scan_type->sign == 's') | 559 | if (scan_type->sign == 's') |
| 555 | val2 = scan_type->realbits - 1; | 560 | val2 = scan_type->realbits - 1; |
| 556 | else | 561 | else |
| @@ -575,6 +580,8 @@ static void ad7768_fill_scale_tbl(struct iio_dev *dev) · 18 unmodified lines | |||
| 575 | st->scale_tbl[i][0] = tmp0; /* Integer part */ | 580 | st->scale_tbl[i][0] = tmp0; /* Integer part */ |
| 576 | st->scale_tbl[i][1] = abs(tmp1); /* Fractional part */ | 581 | st->scale_tbl[i][1] = abs(tmp1); /* Fractional part */ |
| 577 | } | 582 | } |
| 583 | |||
| 584 | return 0; | ||
| 578 | } | 585 | } |
| 579 | 586 | ||
| 580 | static int ad7768_set_sinc3_dec_rate(struct ad7768_state *st, | 587 | static int ad7768_set_sinc3_dec_rate(struct ad7768_state *st, |
| @@ -679,7 +686,9 @@ static int ad7768_configure_dig_fil(struct iio_dev *dev, · 98 unmodified lines | |||
| 679 | } | 686 | } |
| 680 | 687 | ||
| 681 | /* Update scale table: scale values vary according to the precision */ | 688 | /* Update scale table: scale values vary according to the precision */ |
| 682 | ad7768_fill_scale_tbl(dev); | 689 | ret = ad7768_fill_scale_tbl(dev); |
| 690 | if (ret) | ||
| 691 | return ret; | ||
| 683 | 692 | ||
| 684 | ad7768_fill_samp_freq_tbl(st); | 693 | ad7768_fill_samp_freq_tbl(st); |
| 685 | 694 | ||
| @@ -613,7 +613,7 @@ static int bme680_wait_for_eoc(struct bme680_data *data) · 612 unmodified lines | |||
| 613 | * + heater duration | 613 | * + heater duration |
| 614 | */ | 614 | */ |
| 615 | int wait_eoc_us = ((data->oversampling_temp + data->oversampling_press + | 615 | int wait_eoc_us = ((data->oversampling_temp + data->oversampling_press + |
| 616 | data->oversampling_humid) * 1936) + (477 * 4) + | 616 | data->oversampling_humid) * 1963) + (477 * 4) + |
| 617 | (477 * 5) + 1000 + (data->heater_dur * 1000); | 617 | (477 * 5) + 1000 + (data->heater_dur * 1000); |
| 618 | 618 | ||
| 619 | fsleep(wait_eoc_us); | 619 | fsleep(wait_eoc_us); |
| @@ -171,7 +171,7 @@ static int sps30_i2c_read_meas(struct sps30_state *state, __be32 *meas, size_t n · 170 unmodified lines | |||
| 171 | if (!sps30_i2c_meas_ready(state)) | 171 | if (!sps30_i2c_meas_ready(state)) |
| 172 | return -ETIMEDOUT; | 172 | return -ETIMEDOUT; |
| 173 | 173 | ||
| 174 | return sps30_i2c_command(state, SPS30_I2C_READ_MEAS, NULL, 0, meas, sizeof(num) * num); | 174 | return sps30_i2c_command(state, SPS30_I2C_READ_MEAS, NULL, 0, meas, sizeof(*meas) * num); |
| 175 | } | 175 | } |
| 176 | 176 | ||
| 177 | static int sps30_i2c_clean_fan(struct sps30_state *state) | 177 | static int sps30_i2c_clean_fan(struct sps30_state *state) |
| @@ -303,7 +303,7 @@ static int sps30_serial_read_meas(struct sps30_state *state, __be32 *meas, size_ · 302 unmodified lines | |||
| 303 | if (msleep_interruptible(1000)) | 303 | if (msleep_interruptible(1000)) |
| 304 | return -EINTR; | 304 | return -EINTR; |
| 305 | 305 | ||
| 306 | ret = sps30_serial_command(state, SPS30_SERIAL_READ_MEAS, NULL, 0, meas, num * sizeof(num)); | 306 | ret = sps30_serial_command(state, SPS30_SERIAL_READ_MEAS, NULL, 0, meas, num * sizeof(*meas)); |
| 307 | if (ret < 0) | 307 | if (ret < 0) |
| 308 | return ret; | 308 | return ret; |
| 309 | /* if measurements aren't ready sensor returns empty frame */ | 309 | /* if measurements aren't ready sensor returns empty frame */ |
| @@ -140,7 +140,7 @@ static int ds4424_write_raw(struct iio_dev *indio_dev, · 139 unmodified lines | |||
| 140 | 140 | ||
| 141 | switch (mask) { | 141 | switch (mask) { |
| 142 | case IIO_CHAN_INFO_RAW: | 142 | case IIO_CHAN_INFO_RAW: |
| 143 | if (val < S8_MIN || val > S8_MAX) | 143 | if (val <= S8_MIN || val > S8_MAX) |
| 144 | return -EINVAL; | 144 | return -EINVAL; |
| 145 | 145 | ||
| 146 | if (val > 0) { | 146 | if (val > 0) { |
| @@ -508,7 +508,7 @@ static int adf4377_soft_reset(struct adf4377_state *st) · 507 unmodified lines | |||
| 508 | return ret; | 508 | return ret; |
| 509 | 509 | ||
| 510 | return regmap_read_poll_timeout(st->regmap, 0x0, read_val, | 510 | return regmap_read_poll_timeout(st->regmap, 0x0, read_val, |
| 511 | !(read_val & (ADF4377_0000_SOFT_RESET_R_MSK | | 511 | !(read_val & (ADF4377_0000_SOFT_RESET_MSK | |
| 512 | ADF4377_0000_SOFT_RESET_R_MSK)), 200, 200 * 100); | 512 | ADF4377_0000_SOFT_RESET_R_MSK)), 200, 200 * 100); |
| 513 | } | 513 | } |
| 514 | 514 | ||
| @@ -322,7 +322,9 @@ static int mpu3050_read_raw(struct iio_dev *indio_dev, · 321 unmodified lines | |||
| 322 | } | 322 | } |
| 323 | case IIO_CHAN_INFO_RAW: | 323 | case IIO_CHAN_INFO_RAW: |
| 324 | /* Resume device */ | 324 | /* Resume device */ |
| 325 | pm_runtime_get_sync(mpu3050->dev); | 325 | ret = pm_runtime_resume_and_get(mpu3050->dev); |
| 326 | if (ret) | ||
| 327 | return ret; | ||
| 326 | mutex_lock(&mpu3050->lock); | 328 | mutex_lock(&mpu3050->lock); |
| 327 | 329 | ||
| 328 | ret = mpu3050_set_8khz_samplerate(mpu3050); | 330 | ret = mpu3050_set_8khz_samplerate(mpu3050); |
| @@ -647,14 +649,20 @@ out_trigger_unlock: · 318 unmodified lines | |||
| 647 | static int mpu3050_buffer_preenable(struct iio_dev *indio_dev) | 649 | static int mpu3050_buffer_preenable(struct iio_dev *indio_dev) |
| 648 | { | 650 | { |
| 649 | struct mpu3050 *mpu3050 = iio_priv(indio_dev); | 651 | struct mpu3050 *mpu3050 = iio_priv(indio_dev); |
| 652 | int ret; | ||
| 650 | 653 | ||
| 651 | pm_runtime_get_sync(mpu3050->dev); | 654 | ret = pm_runtime_resume_and_get(mpu3050->dev); |
| 655 | if (ret) | ||
| 656 | return ret; | ||
| 652 | 657 | ||
| 653 | /* Unless we have OUR trigger active, run at full speed */ | 658 | /* Unless we have OUR trigger active, run at full speed */ |
| 654 | if (!mpu3050->hw_irq_trigger) | 659 | if (!mpu3050->hw_irq_trigger) { |
| 655 | return mpu3050_set_8khz_samplerate(mpu3050); | 660 | ret = mpu3050_set_8khz_samplerate(mpu3050); |
| 661 | if (ret) | ||
| 662 | pm_runtime_put_autosuspend(mpu3050->dev); | ||
| 663 | } | ||
| 656 | 664 | ||
| 657 | return 0; | 665 | return ret; |
| 658 | } | 666 | } |
| 659 | 667 | ||
| 660 | static int mpu3050_buffer_postdisable(struct iio_dev *indio_dev) | 668 | static int mpu3050_buffer_postdisable(struct iio_dev *indio_dev) |
| @@ -19,8 +19,7 @@ static int mpu3050_i2c_bypass_select(struct i2c_mux_core *mux, u32 chan_id) · 18 unmodified lines | |||
| 19 | struct mpu3050 *mpu3050 = i2c_mux_priv(mux); | 19 | struct mpu3050 *mpu3050 = i2c_mux_priv(mux); |
| 20 | 20 | ||
| 21 | /* Just power up the device, that is all that is needed */ | 21 | /* Just power up the device, that is all that is needed */ |
| 22 | pm_runtime_get_sync(mpu3050->dev); | 22 | return pm_runtime_resume_and_get(mpu3050->dev); |
| 23 | return 0; | ||
| 24 | } | 23 | } |
| 25 | 24 | ||
| 26 | static int mpu3050_i2c_bypass_deselect(struct i2c_mux_core *mux, u32 chan_id) | 25 | static int mpu3050_i2c_bypass_deselect(struct i2c_mux_core *mux, u32 chan_id) |
| @@ -526,7 +526,7 @@ int adis_init(struct adis *adis, struct iio_dev *indio_dev, · 525 unmodified lines | |||
| 526 | 526 | ||
| 527 | adis->spi = spi; | 527 | adis->spi = spi; |
| 528 | adis->data = data; | 528 | adis->data = data; |
| 529 | if (!adis->ops->write && !adis->ops->read && !adis->ops->reset) | 529 | if (!adis->ops) |
| 530 | adis->ops = &adis_default_ops; | 530 | adis->ops = &adis_default_ops; |
| 531 | else if (!adis->ops->write || !adis->ops->read || !adis->ops->reset) | 531 | else if (!adis->ops->write || !adis->ops->read || !adis->ops->reset) |
| 532 | return -EINVAL; | 532 | return -EINVAL; |
| @@ -651,6 +651,8 @@ static int inv_icm42600_accel_write_odr(struct iio_dev *indio_dev, · 650 unmodified lines | |||
| 651 | return -EINVAL; | 651 | return -EINVAL; |
| 652 | 652 | ||
| 653 | conf.odr = inv_icm42600_accel_odr_conv[idx / 2]; | 653 | conf.odr = inv_icm42600_accel_odr_conv[idx / 2]; |
| 654 | if (conf.odr == st->conf.accel.odr) | ||
| 655 | return 0; | ||
| 654 | 656 | ||
| 655 | pm_runtime_get_sync(dev); | 657 | pm_runtime_get_sync(dev); |
| 656 | mutex_lock(&st->lock); | 658 | mutex_lock(&st->lock); |
| @@ -371,6 +371,8 @@ static int inv_icm42600_buffer_predisable(struct iio_dev *indio_dev) · 370 unmodified lines | |||
| 371 | static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev) | 371 | static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev) |
| 372 | { | 372 | { |
| 373 | struct inv_icm42600_state *st = iio_device_get_drvdata(indio_dev); | 373 | struct inv_icm42600_state *st = iio_device_get_drvdata(indio_dev); |
| 374 | struct inv_icm42600_sensor_state *sensor_st = iio_priv(indio_dev); | ||
| 375 | struct inv_sensors_timestamp *ts = &sensor_st->ts; | ||
| 374 | struct device *dev = regmap_get_device(st->map); | 376 | struct device *dev = regmap_get_device(st->map); |
| 375 | unsigned int sensor; | 377 | unsigned int sensor; |
| 376 | unsigned int *watermark; | 378 | unsigned int *watermark; |
| @@ -392,6 +394,8 @@ static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev) · 15 unmodified lines | |||
| 392 | 394 | ||
| 393 | mutex_lock(&st->lock); | 395 | mutex_lock(&st->lock); |
| 394 | 396 | ||
| 397 | inv_sensors_timestamp_apply_odr(ts, 0, 0, 0); | ||
| 398 | |||
| 395 | ret = inv_icm42600_buffer_set_fifo_en(st, st->fifo.en & ~sensor); | 399 | ret = inv_icm42600_buffer_set_fifo_en(st, st->fifo.en & ~sensor); |
| 396 | if (ret) | 400 | if (ret) |
| 397 | goto out_unlock; | 401 | goto out_unlock; |
| @@ -358,6 +358,8 @@ static int inv_icm42600_gyro_write_odr(struct iio_dev *indio_dev, · 357 unmodified lines | |||
| 358 | return -EINVAL; | 358 | return -EINVAL; |
| 359 | 359 | ||
| 360 | conf.odr = inv_icm42600_gyro_odr_conv[idx / 2]; | 360 | conf.odr = inv_icm42600_gyro_odr_conv[idx / 2]; |
| 361 | if (conf.odr == st->conf.gyro.odr) | ||
| 362 | return 0; | ||
| 361 | 363 | ||
| 362 | pm_runtime_get_sync(dev); | 364 | pm_runtime_get_sync(dev); |
| 363 | mutex_lock(&st->lock); | 365 | mutex_lock(&st->lock); |
| @@ -205,7 +205,7 @@ struct inv_icm45600_sensor_state { · 204 unmodified lines | |||
| 205 | #define INV_ICM45600_SPI_SLEW_RATE_38NS 0 | 205 | #define INV_ICM45600_SPI_SLEW_RATE_38NS 0 |
| 206 | 206 | ||
| 207 | #define INV_ICM45600_REG_INT1_CONFIG2 0x0018 | 207 | #define INV_ICM45600_REG_INT1_CONFIG2 0x0018 |
| 208 | #define INV_ICM45600_INT1_CONFIG2_PUSH_PULL BIT(2) | 208 | #define INV_ICM45600_INT1_CONFIG2_OPEN_DRAIN BIT(2) |
| 209 | #define INV_ICM45600_INT1_CONFIG2_LATCHED BIT(1) | 209 | #define INV_ICM45600_INT1_CONFIG2_LATCHED BIT(1) |
| 210 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_HIGH BIT(0) | 210 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_HIGH BIT(0) |
| 211 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_LOW 0x00 | 211 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_LOW 0x00 |
| @@ -637,8 +637,8 @@ static int inv_icm45600_irq_init(struct inv_icm45600_state *st, int irq, · 636 unmodified lines | |||
| 637 | break; | 637 | break; |
| 638 | } | 638 | } |
| 639 | 639 | ||
| 640 | if (!open_drain) | 640 | if (open_drain) |
| 641 | val |= INV_ICM45600_INT1_CONFIG2_PUSH_PULL; | 641 | val |= INV_ICM45600_INT1_CONFIG2_OPEN_DRAIN; |
| 642 | 642 | ||
| 643 | ret = regmap_write(st->map, INV_ICM45600_REG_INT1_CONFIG2, val); | 643 | ret = regmap_write(st->map, INV_ICM45600_REG_INT1_CONFIG2, val); |
| 644 | if (ret) | 644 | if (ret) |
| @@ -744,6 +744,11 @@ int inv_icm45600_core_probe(struct regmap *regmap, const struct inv_icm45600_chi · 99 unmodified lines | |||
| 744 | */ | 744 | */ |
| 745 | fsleep(5 * USEC_PER_MSEC); | 745 | fsleep(5 * USEC_PER_MSEC); |
| 746 | 746 | ||
| 747 | /* set pm_runtime active early for disable vddio resource cleanup */ | ||
| 748 | ret = pm_runtime_set_active(dev); | ||
| 749 | if (ret) | ||
| 750 | return ret; | ||
| 751 | |||
| 747 | ret = inv_icm45600_enable_regulator_vddio(st); | 752 | ret = inv_icm45600_enable_regulator_vddio(st); |
| 748 | if (ret) | 753 | if (ret) |
| 749 | return ret; | 754 | return ret; |
| @@ -776,7 +781,7 @@ int inv_icm45600_core_probe(struct regmap *regmap, const struct inv_icm45600_chi · 26 unmodified lines | |||
| 776 | if (ret) | 781 | if (ret) |
| 777 | return ret; | 782 | return ret; |
| 778 | 783 | ||
| 779 | ret = devm_pm_runtime_set_active_enabled(dev); | 784 | ret = devm_pm_runtime_enable(dev); |
| 780 | if (ret) | 785 | if (ret) |
| 781 | return ret; | 786 | return ret; |
| 782 | 787 | ||
| @@ -1943,6 +1943,14 @@ int inv_mpu_core_probe(struct regmap *regmap, int irq, const char *name, · 1942 unmodified lines | |||
| 1943 | irq_type); | 1943 | irq_type); |
| 1944 | return -EINVAL; | 1944 | return -EINVAL; |
| 1945 | } | 1945 | } |
| 1946 | |||
| 1947 | /* | ||
| 1948 | * Acking interrupts by status register does not work reliably | ||
| 1949 | * but seem to work when this bit is set. | ||
| 1950 | */ | ||
| 1951 | if (st->chip_type == INV_MPU9150) | ||
| 1952 | st->irq_mask |= INV_MPU6050_INT_RD_CLEAR; | ||
| 1953 | |||
| 1946 | device_set_wakeup_capable(dev, true); | 1954 | device_set_wakeup_capable(dev, true); |
| 1947 | 1955 | ||
| 1948 | st->vdd_supply = devm_regulator_get(dev, "vdd"); | 1956 | st->vdd_supply = devm_regulator_get(dev, "vdd"); |
| @@ -390,6 +390,8 @@ struct inv_mpu6050_state { · 389 unmodified lines | |||
| 390 | /* enable level triggering */ | 390 | /* enable level triggering */ |
| 391 | #define INV_MPU6050_LATCH_INT_EN 0x20 | 391 | #define INV_MPU6050_LATCH_INT_EN 0x20 |
| 392 | #define INV_MPU6050_BIT_BYPASS_EN 0x2 | 392 | #define INV_MPU6050_BIT_BYPASS_EN 0x2 |
| 393 | /* allow acking interrupts by any register read */ | ||
| 394 | #define INV_MPU6050_INT_RD_CLEAR 0x10 | ||
| 393 | 395 | ||
| 394 | /* Allowed timestamp period jitter in percent */ | 396 | /* Allowed timestamp period jitter in percent */ |
| 395 | #define INV_MPU6050_TS_PERIOD_JITTER 4 | 397 | #define INV_MPU6050_TS_PERIOD_JITTER 4 |
| @@ -248,7 +248,6 @@ static irqreturn_t inv_mpu6050_interrupt_handle(int irq, void *p) · 247 unmodified lines | |||
| 248 | switch (st->chip_type) { | 248 | switch (st->chip_type) { |
| 249 | case INV_MPU6000: | 249 | case INV_MPU6000: |
| 250 | case INV_MPU6050: | 250 | case INV_MPU6050: |
| 251 | case INV_MPU9150: | ||
| 252 | /* | 251 | /* |
| 253 | * WoM is not supported and interrupt status read seems to be broken for | 252 | * WoM is not supported and interrupt status read seems to be broken for |
| 254 | * some chips. Since data ready is the only interrupt, bypass interrupt | 253 | * some chips. Since data ready is the only interrupt, bypass interrupt |
| @@ -257,6 +256,10 @@ static irqreturn_t inv_mpu6050_interrupt_handle(int irq, void *p) · 2 unmodified lines | |||
| 257 | wom_bits = 0; | 256 | wom_bits = 0; |
| 258 | int_status = INV_MPU6050_BIT_RAW_DATA_RDY_INT; | 257 | int_status = INV_MPU6050_BIT_RAW_DATA_RDY_INT; |
| 259 | goto data_ready_interrupt; | 258 | goto data_ready_interrupt; |
| 259 | case INV_MPU9150: | ||
| 260 | /* IRQ needs to be acked */ | ||
| 261 | wom_bits = 0; | ||
| 262 | break; | ||
| 260 | case INV_MPU6500: | 263 | case INV_MPU6500: |
| 261 | case INV_MPU6515: | 264 | case INV_MPU6515: |
| 262 | case INV_MPU6880: | 265 | case INV_MPU6880: |
| @@ -228,8 +228,10 @@ static ssize_t iio_buffer_write(struct file *filp, const char __user *buf, · 227 unmodified lines | |||
| 228 | written = 0; | 228 | written = 0; |
| 229 | add_wait_queue(&rb->pollq, &wait); | 229 | add_wait_queue(&rb->pollq, &wait); |
| 230 | do { | 230 | do { |
| 231 | if (!indio_dev->info) | 231 | if (!indio_dev->info) { |
| 232 | return -ENODEV; | 232 | ret = -ENODEV; |
| 233 | break; | ||
| 234 | } | ||
| 233 | 235 | ||
| 234 | if (!iio_buffer_space_available(rb)) { | 236 | if (!iio_buffer_space_available(rb)) { |
| 235 | if (signal_pending(current)) { | 237 | if (signal_pending(current)) { |
| @@ -109,9 +109,9 @@ static int bh1780_read_raw(struct iio_dev *indio_dev, · 108 unmodified lines | |||
| 109 | case IIO_LIGHT: | 109 | case IIO_LIGHT: |
| 110 | pm_runtime_get_sync(&bh1780->client->dev); | 110 | pm_runtime_get_sync(&bh1780->client->dev); |
| 111 | value = bh1780_read_word(bh1780, BH1780_REG_DLOW); | 111 | value = bh1780_read_word(bh1780, BH1780_REG_DLOW); |
| 112 | pm_runtime_put_autosuspend(&bh1780->client->dev); | ||
| 112 | if (value < 0) | 113 | if (value < 0) |
| 113 | return value; | 114 | return value; |
| 114 | pm_runtime_put_autosuspend(&bh1780->client->dev); | ||
| 115 | *val = value; | 115 | *val = value; |
| 116 | 116 | ||
| 117 | return IIO_VAL_INT; | 117 | return IIO_VAL_INT; |
| @@ -143,8 +143,7 @@ config MMC5633 · 142 unmodified lines | |||
| 143 | tristate "MEMSIC MMC5633 3-axis magnetic sensor" | 143 | tristate "MEMSIC MMC5633 3-axis magnetic sensor" |
| 144 | select REGMAP_I2C | 144 | select REGMAP_I2C |
| 145 | select REGMAP_I3C if I3C | 145 | select REGMAP_I3C if I3C |
| 146 | depends on I2C | 146 | depends on I3C_OR_I2C |
| 147 | depends on I3C || !I3C | ||
| 148 | help | 147 | help |
| 149 | Say yes here to build support for the MEMSIC MMC5633 3-axis | 148 | Say yes here to build support for the MEMSIC MMC5633 3-axis |
| 150 | magnetic sensor. | 149 | magnetic sensor. |
| @@ -171,7 +171,7 @@ static s16 tlv493d_get_channel_data(u8 *b, enum tlv493d_channels ch) · 170 unmodified lines | |||
| 171 | switch (ch) { | 171 | switch (ch) { |
| 172 | case TLV493D_AXIS_X: | 172 | case TLV493D_AXIS_X: |
| 173 | val = FIELD_GET(TLV493D_BX_MAG_X_AXIS_MSB, b[TLV493D_RD_REG_BX]) << 4 | | 173 | val = FIELD_GET(TLV493D_BX_MAG_X_AXIS_MSB, b[TLV493D_RD_REG_BX]) << 4 | |
| 174 | FIELD_GET(TLV493D_BX2_MAG_X_AXIS_LSB, b[TLV493D_RD_REG_BX2]) >> 4; | 174 | FIELD_GET(TLV493D_BX2_MAG_X_AXIS_LSB, b[TLV493D_RD_REG_BX2]); |
| 175 | break; | 175 | break; |
| 176 | case TLV493D_AXIS_Y: | 176 | case TLV493D_AXIS_Y: |
| 177 | val = FIELD_GET(TLV493D_BY_MAG_Y_AXIS_MSB, b[TLV493D_RD_REG_BY]) << 4 | | 177 | val = FIELD_GET(TLV493D_BY_MAG_Y_AXIS_MSB, b[TLV493D_RD_REG_BY]) << 4 | |
| @@ -221,7 +221,7 @@ static int mcp4131_write_raw(struct iio_dev *indio_dev, · 220 unmodified lines | |||
| 221 | 221 | ||
| 222 | mutex_lock(&data->lock); | 222 | mutex_lock(&data->lock); |
| 223 | 223 | ||
| 224 | data->buf[0] = address << MCP4131_WIPER_SHIFT; | 224 | data->buf[0] = address; |
| 225 | data->buf[0] |= MCP4131_WRITE | (val >> 8); | 225 | data->buf[0] |= MCP4131_WRITE | (val >> 8); |
| 226 | data->buf[1] = val & 0xFF; /* 8 bits here */ | 226 | data->buf[1] = val & 0xFF; /* 8 bits here */ |
| 227 | 227 | ||
| @@ -719,6 +719,9 @@ static int hx9023s_set_samp_freq(struct hx9023s_data *data, int val, int val2) · 718 unmodified lines | |||
| 719 | struct device *dev = regmap_get_device(data->regmap); | 719 | struct device *dev = regmap_get_device(data->regmap); |
| 720 | unsigned int i, period_ms; | 720 | unsigned int i, period_ms; |
| 721 | 721 | ||
| 722 | if (!val && !val2) | ||
| 723 | return -EINVAL; | ||
| 724 | |||
| 722 | period_ms = div_u64(NANO, (val * MEGA + val2)); | 725 | period_ms = div_u64(NANO, (val * MEGA + val2)); |
| 723 | 726 | ||
| 724 | for (i = 0; i < ARRAY_SIZE(hx9023s_samp_freq_table); i++) { | 727 | for (i = 0; i < ARRAY_SIZE(hx9023s_samp_freq_table); i++) { |
| @@ -1034,9 +1037,8 @@ static int hx9023s_send_cfg(const struct firmware *fw, struct hx9023s_data *data · 309 unmodified lines | |||
| 1034 | if (!bin) | 1037 | if (!bin) |
| 1035 | return -ENOMEM; | 1038 | return -ENOMEM; |
| 1036 | 1039 | ||
| 1037 | memcpy(bin->data, fw->data, fw->size); | ||
| 1038 | |||
| 1039 | bin->fw_size = fw->size; | 1040 | bin->fw_size = fw->size; |
| 1041 | memcpy(bin->data, fw->data, bin->fw_size); | ||
| 1040 | bin->fw_ver = bin->data[FW_VER_OFFSET]; | 1042 | bin->fw_ver = bin->data[FW_VER_OFFSET]; |
| 1041 | bin->reg_count = get_unaligned_le16(bin->data + FW_REG_CNT_OFFSET); | 1043 | bin->reg_count = get_unaligned_le16(bin->data + FW_REG_CNT_OFFSET); |
| 1042 | 1044 | ||
| @@ -6,6 +6,7 @@ menuconfig INFINIBAND · 5 unmodified lines | |||
| 6 | depends on INET | 6 | depends on INET |
| 7 | depends on m || IPV6 != m | 7 | depends on m || IPV6 != m |
| 8 | depends on !ALPHA | 8 | depends on !ALPHA |
| 9 | select DMA_SHARED_BUFFER | ||
| 9 | select IRQ_POLL | 10 | select IRQ_POLL |
| 10 | select DIMLIB | 11 | select DIMLIB |
| 11 | help | 12 | help |
| @@ -926,6 +926,13 @@ static int gid_table_setup_one(struct ib_device *ib_dev) · 925 unmodified lines | |||
| 926 | if (err) | 926 | if (err) |
| 927 | return err; | 927 | return err; |
| 928 | 928 | ||
| 929 | /* | ||
| 930 | * Mark the device as ready for GID cache updates. This allows netdev | ||
| 931 | * event handlers to update the GID cache even before the device is | ||
| 932 | * fully registered. | ||
| 933 | */ | ||
| 934 | ib_device_enable_gid_updates(ib_dev); | ||
| 935 | |||
| 929 | rdma_roce_rescan_device(ib_dev); | 936 | rdma_roce_rescan_device(ib_dev); |
| 930 | 937 | ||
| 931 | return err; | 938 | return err; |
| @@ -1637,6 +1644,12 @@ void ib_cache_release_one(struct ib_device *device) · 705 unmodified lines | |||
| 1637 | 1644 | ||
| 1638 | void ib_cache_cleanup_one(struct ib_device *device) | 1645 | void ib_cache_cleanup_one(struct ib_device *device) |
| 1639 | { | 1646 | { |
| 1647 | /* | ||
| 1648 | * Clear the GID updates mark first to prevent event handlers from | ||
| 1649 | * accessing the device while it's being torn down. | ||
| 1650 | */ | ||
| 1651 | ib_device_disable_gid_updates(device); | ||
| 1652 | |||
| 1640 | /* The cleanup function waits for all in-progress workqueue | 1653 | /* The cleanup function waits for all in-progress workqueue |
| 1641 | * elements and cleans up the GID cache. This function should be | 1654 | * elements and cleans up the GID cache. This function should be |
| 1642 | * called after the device was removed from the devices list and | 1655 | * called after the device was removed from the devices list and |
| @@ -2729,6 +2729,9 @@ static int cma_listen_on_dev(struct rdma_id_private *id_priv, · 2728 unmodified lines | |||
| 2729 | *to_destroy = NULL; | 2729 | *to_destroy = NULL; |
| 2730 | if (cma_family(id_priv) == AF_IB && !rdma_cap_ib_cm(cma_dev->device, 1)) | 2730 | if (cma_family(id_priv) == AF_IB && !rdma_cap_ib_cm(cma_dev->device, 1)) |
| 2731 | return 0; | 2731 | return 0; |
| 2732 | if (id_priv->restricted_node_type != RDMA_NODE_UNSPECIFIED && | ||
| 2733 | id_priv->restricted_node_type != cma_dev->device->node_type) | ||
| 2734 | return 0; | ||
| 2732 | 2735 | ||
| 2733 | dev_id_priv = | 2736 | dev_id_priv = |
| 2734 | __rdma_create_id(net, cma_listen_handler, id_priv, | 2737 | __rdma_create_id(net, cma_listen_handler, id_priv, |
| @@ -2736,6 +2739,7 @@ static int cma_listen_on_dev(struct rdma_id_private *id_priv, · 1 unmodified line | |||
| 2736 | if (IS_ERR(dev_id_priv)) | 2739 | if (IS_ERR(dev_id_priv)) |
| 2737 | return PTR_ERR(dev_id_priv); | 2740 | return PTR_ERR(dev_id_priv); |
| 2738 | 2741 | ||
| 2742 | dev_id_priv->restricted_node_type = id_priv->restricted_node_type; | ||
| 2739 | dev_id_priv->state = RDMA_CM_ADDR_BOUND; | 2743 | dev_id_priv->state = RDMA_CM_ADDR_BOUND; |
| 2740 | memcpy(cma_src_addr(dev_id_priv), cma_src_addr(id_priv), | 2744 | memcpy(cma_src_addr(dev_id_priv), cma_src_addr(id_priv), |
| 2741 | rdma_addr_size(cma_src_addr(id_priv))); | 2745 | rdma_addr_size(cma_src_addr(id_priv))); |
| @@ -4194,7 +4198,7 @@ int rdma_restrict_node_type(struct rdma_cm_id *id, u8 node_type) · 1452 unmodified lines | |||
| 4194 | } | 4198 | } |
| 4195 | 4199 | ||
| 4196 | mutex_lock(&lock); | 4200 | mutex_lock(&lock); |
| 4197 | if (id_priv->cma_dev) | 4201 | if (READ_ONCE(id_priv->state) != RDMA_CM_IDLE) |
| 4198 | ret = -EALREADY; | 4202 | ret = -EALREADY; |
| 4199 | else | 4203 | else |
| 4200 | id_priv->restricted_node_type = node_type; | 4204 | id_priv->restricted_node_type = node_type; |
| @@ -100,6 +100,9 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, · 99 unmodified lines | |||
| 100 | roce_netdev_callback cb, | 100 | roce_netdev_callback cb, |
| 101 | void *cookie); | 101 | void *cookie); |
| 102 | 102 | ||
| 103 | void ib_device_enable_gid_updates(struct ib_device *device); | ||
| 104 | void ib_device_disable_gid_updates(struct ib_device *device); | ||
| 105 | |||
| 103 | typedef int (*nldev_callback)(struct ib_device *device, | 106 | typedef int (*nldev_callback)(struct ib_device *device, |
| 104 | struct sk_buff *skb, | 107 | struct sk_buff *skb, |
| 105 | struct netlink_callback *cb, | 108 | struct netlink_callback *cb, |
| @@ -93,6 +93,7 @@ static struct workqueue_struct *ib_unreg_wq; · 92 unmodified lines | |||
| 93 | static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); | 93 | static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); |
| 94 | static DECLARE_RWSEM(devices_rwsem); | 94 | static DECLARE_RWSEM(devices_rwsem); |
| 95 | #define DEVICE_REGISTERED XA_MARK_1 | 95 | #define DEVICE_REGISTERED XA_MARK_1 |
| 96 | #define DEVICE_GID_UPDATES XA_MARK_2 | ||
| 96 | 97 | ||
| 97 | static u32 highest_client_id; | 98 | static u32 highest_client_id; |
| 98 | #define CLIENT_REGISTERED XA_MARK_1 | 99 | #define CLIENT_REGISTERED XA_MARK_1 |
| @@ -2412,11 +2413,42 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, · 2313 unmodified lines | |||
| 2412 | unsigned long index; | 2413 | unsigned long index; |
| 2413 | 2414 | ||
| 2414 | down_read(&devices_rwsem); | 2415 | down_read(&devices_rwsem); |
| 2415 | xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) | 2416 | xa_for_each_marked(&devices, index, dev, DEVICE_GID_UPDATES) |
| 2416 | ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); | 2417 | ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); |
| 2417 | up_read(&devices_rwsem); | 2418 | up_read(&devices_rwsem); |
| 2418 | } | 2419 | } |
| 2419 | 2420 | ||
| 2421 | /** | ||
| 2422 | * ib_device_enable_gid_updates - Mark device as ready for GID cache updates | ||
| 2423 | * @device: Device to mark | ||
| 2424 | * | ||
| 2425 | * Called after GID table is allocated and initialized. After this mark is set, | ||
| 2426 | * netdevice event handlers can update the device's GID cache. This allows | ||
| 2427 | * events that arrive during device registration to be processed, avoiding | ||
| 2428 | * stale GID entries when netdev properties change during the device | ||
| 2429 | * registration process. | ||
| 2430 | */ | ||
| 2431 | void ib_device_enable_gid_updates(struct ib_device *device) | ||
| 2432 | { | ||
| 2433 | down_write(&devices_rwsem); | ||
| 2434 | xa_set_mark(&devices, device->index, DEVICE_GID_UPDATES); | ||
| 2435 | up_write(&devices_rwsem); | ||
| 2436 | } | ||
| 2437 | |||
| 2438 | /** | ||
| 2439 | * ib_device_disable_gid_updates - Clear the GID updates mark | ||
| 2440 | * @device: Device to unmark | ||
| 2441 | * | ||
| 2442 | * Called before GID table cleanup to prevent event handlers from accessing | ||
| 2443 | * the device while it's being torn down. | ||
| 2444 | */ | ||
| 2445 | void ib_device_disable_gid_updates(struct ib_device *device) | ||
| 2446 | { | ||
| 2447 | down_write(&devices_rwsem); | ||
| 2448 | xa_clear_mark(&devices, device->index, DEVICE_GID_UPDATES); | ||
| 2449 | up_write(&devices_rwsem); | ||
| 2450 | } | ||
| 2451 | |||
| 2420 | /* | 2452 | /* |
| 2421 | * ib_enum_all_devs - enumerate all ib_devices | 2453 | * ib_enum_all_devs - enumerate all ib_devices |
| 2422 | * @cb: Callback to call for each found ib_device | 2454 | * @cb: Callback to call for each found ib_device |
| @@ -218,13 +218,11 @@ ib_umem_dmabuf_get_pinned_with_dma_device(struct ib_device *device, · 217 unmodified lines | |||
| 218 | 218 | ||
| 219 | err = ib_umem_dmabuf_map_pages(umem_dmabuf); | 219 | err = ib_umem_dmabuf_map_pages(umem_dmabuf); |
| 220 | if (err) | 220 | if (err) |
| 221 | goto err_unpin; | 221 | goto err_release; |
| 222 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); | 222 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); |
| 223 | 223 | ||
| 224 | return umem_dmabuf; | 224 | return umem_dmabuf; |
| 225 | 225 | ||
| 226 | err_unpin: | ||
| 227 | dma_buf_unpin(umem_dmabuf->attach); | ||
| 228 | err_release: | 226 | err_release: |
| 229 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); | 227 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); |
| 230 | ib_umem_release(&umem_dmabuf->umem); | 228 | ib_umem_release(&umem_dmabuf->umem); |
| @@ -10,6 +10,8 @@ · 9 unmodified lines | |||
| 10 | #include "rdma_core.h" | 10 | #include "rdma_core.h" |
| 11 | #include "uverbs.h" | 11 | #include "uverbs.h" |
| 12 | 12 | ||
| 13 | MODULE_IMPORT_NS("DMA_BUF"); | ||
| 14 | |||
| 13 | static int uverbs_dmabuf_attach(struct dma_buf *dmabuf, | 15 | static int uverbs_dmabuf_attach(struct dma_buf *dmabuf, |
| 14 | struct dma_buf_attachment *attachment) | 16 | struct dma_buf_attachment *attachment) |
| 15 | { | 17 | { |
| @@ -54,9 +54,6 @@ static void bng_re_destroy_chip_ctx(struct bng_re_dev *rdev) · 53 unmodified lines | |||
| 54 | { | 54 | { |
| 55 | struct bng_re_chip_ctx *chip_ctx; | 55 | struct bng_re_chip_ctx *chip_ctx; |
| 56 | 56 | ||
| 57 | if (!rdev->chip_ctx) | ||
| 58 | return; | ||
| 59 | |||
| 60 | kfree(rdev->dev_attr); | 57 | kfree(rdev->dev_attr); |
| 61 | rdev->dev_attr = NULL; | 58 | rdev->dev_attr = NULL; |
| 62 | 59 | ||
| @@ -124,12 +121,6 @@ static int bng_re_net_ring_free(struct bng_re_dev *rdev, · 61 unmodified lines | |||
| 124 | struct bnge_fw_msg fw_msg = {}; | 121 | struct bnge_fw_msg fw_msg = {}; |
| 125 | int rc = -EINVAL; | 122 | int rc = -EINVAL; |
| 126 | 123 | ||
| 127 | if (!rdev) | ||
| 128 | return rc; | ||
| 129 | |||
| 130 | if (!aux_dev) | ||
| 131 | return rc; | ||
| 132 | |||
| 133 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_FREE); | 124 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_FREE); |
| 134 | req.ring_type = type; | 125 | req.ring_type = type; |
| 135 | req.ring_id = cpu_to_le16(fw_ring_id); | 126 | req.ring_id = cpu_to_le16(fw_ring_id); |
| @@ -150,10 +141,7 @@ static int bng_re_net_ring_alloc(struct bng_re_dev *rdev, · 14 unmodified lines | |||
| 150 | struct hwrm_ring_alloc_input req = {}; | 141 | struct hwrm_ring_alloc_input req = {}; |
| 151 | struct hwrm_ring_alloc_output resp; | 142 | struct hwrm_ring_alloc_output resp; |
| 152 | struct bnge_fw_msg fw_msg = {}; | 143 | struct bnge_fw_msg fw_msg = {}; |
| 153 | int rc = -EINVAL; | 144 | int rc; |
| 154 | |||
| 155 | if (!aux_dev) | ||
| 156 | return rc; | ||
| 157 | 145 | ||
| 158 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_ALLOC); | 146 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_ALLOC); |
| 159 | req.enables = 0; | 147 | req.enables = 0; |
| @@ -184,10 +172,7 @@ static int bng_re_stats_ctx_free(struct bng_re_dev *rdev) · 24 unmodified lines | |||
| 184 | struct hwrm_stat_ctx_free_input req = {}; | 172 | struct hwrm_stat_ctx_free_input req = {}; |
| 185 | struct hwrm_stat_ctx_free_output resp = {}; | 173 | struct hwrm_stat_ctx_free_output resp = {}; |
| 186 | struct bnge_fw_msg fw_msg = {}; | 174 | struct bnge_fw_msg fw_msg = {}; |
| 187 | int rc = -EINVAL; | 175 | int rc; |
| 188 | |||
| 189 | if (!aux_dev) | ||
| 190 | return rc; | ||
| 191 | 176 | ||
| 192 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_FREE); | 177 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_FREE); |
| 193 | req.stat_ctx_id = cpu_to_le32(rdev->stats_ctx.fw_id); | 178 | req.stat_ctx_id = cpu_to_le32(rdev->stats_ctx.fw_id); |
| @@ -208,13 +193,10 @@ static int bng_re_stats_ctx_alloc(struct bng_re_dev *rdev) · 14 unmodified lines | |||
| 208 | struct hwrm_stat_ctx_alloc_output resp = {}; | 193 | struct hwrm_stat_ctx_alloc_output resp = {}; |
| 209 | struct hwrm_stat_ctx_alloc_input req = {}; | 194 | struct hwrm_stat_ctx_alloc_input req = {}; |
| 210 | struct bnge_fw_msg fw_msg = {}; | 195 | struct bnge_fw_msg fw_msg = {}; |
| 211 | int rc = -EINVAL; | 196 | int rc; |
| 212 | 197 | ||
| 213 | stats->fw_id = BNGE_INVALID_STATS_CTX_ID; | 198 | stats->fw_id = BNGE_INVALID_STATS_CTX_ID; |
| 214 | 199 | ||
| 215 | if (!aux_dev) | ||
| 216 | return rc; | ||
| 217 | |||
| 218 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_ALLOC); | 200 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_ALLOC); |
| 219 | req.update_period_ms = cpu_to_le32(1000); | 201 | req.update_period_ms = cpu_to_le32(1000); |
| 220 | req.stats_dma_addr = cpu_to_le64(stats->dma_map); | 202 | req.stats_dma_addr = cpu_to_le64(stats->dma_map); |
| @@ -303,7 +285,7 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 82 unmodified lines | |||
| 303 | if (rc) { | 285 | if (rc) { |
| 304 | ibdev_err(&rdev->ibdev, | 286 | ibdev_err(&rdev->ibdev, |
| 305 | "Failed to register with netedev: %#x\n", rc); | 287 | "Failed to register with netedev: %#x\n", rc); |
| 306 | return -EINVAL; | 288 | goto reg_netdev_fail; |
| 307 | } | 289 | } |
| 308 | 290 | ||
| 309 | set_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | 291 | set_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); |
| @@ -312,19 +294,16 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 2 unmodified lines | |||
| 312 | ibdev_err(&rdev->ibdev, | 294 | ibdev_err(&rdev->ibdev, |
| 313 | "RoCE requires minimum 2 MSI-X vectors, but only %d reserved\n", | 295 | "RoCE requires minimum 2 MSI-X vectors, but only %d reserved\n", |
| 314 | rdev->aux_dev->auxr_info->msix_requested); | 296 | rdev->aux_dev->auxr_info->msix_requested); |
| 315 | bnge_unregister_dev(rdev->aux_dev); | 297 | rc = -EINVAL; |
| 316 | clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | 298 | goto msix_ctx_fail; |
| 317 | return -EINVAL; | ||
| 318 | } | 299 | } |
| 319 | ibdev_dbg(&rdev->ibdev, "Got %d MSI-X vectors\n", | 300 | ibdev_dbg(&rdev->ibdev, "Got %d MSI-X vectors\n", |
| 320 | rdev->aux_dev->auxr_info->msix_requested); | 301 | rdev->aux_dev->auxr_info->msix_requested); |
| 321 | 302 | ||
| 322 | rc = bng_re_setup_chip_ctx(rdev); | 303 | rc = bng_re_setup_chip_ctx(rdev); |
| 323 | if (rc) { | 304 | if (rc) { |
| 324 | bnge_unregister_dev(rdev->aux_dev); | ||
| 325 | clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | ||
| 326 | ibdev_err(&rdev->ibdev, "Failed to get chip context\n"); | 305 | ibdev_err(&rdev->ibdev, "Failed to get chip context\n"); |
| 327 | return -EINVAL; | 306 | goto msix_ctx_fail; |
| 328 | } | 307 | } |
| 329 | 308 | ||
| 330 | bng_re_query_hwrm_version(rdev); | 309 | bng_re_query_hwrm_version(rdev); |
| @@ -333,16 +312,14 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 2 unmodified lines | |||
| 333 | if (rc) { | 312 | if (rc) { |
| 334 | ibdev_err(&rdev->ibdev, | 313 | ibdev_err(&rdev->ibdev, |
| 335 | "Failed to allocate RCFW Channel: %#x\n", rc); | 314 | "Failed to allocate RCFW Channel: %#x\n", rc); |
| 336 | goto fail; | 315 | goto alloc_fw_chl_fail; |
| 337 | } | 316 | } |
| 338 | 317 | ||
| 339 | /* Allocate nq record memory */ | 318 | /* Allocate nq record memory */ |
| 340 | rdev->nqr = kzalloc_obj(*rdev->nqr); | 319 | rdev->nqr = kzalloc_obj(*rdev->nqr); |
| 341 | if (!rdev->nqr) { | 320 | if (!rdev->nqr) { |
| 342 | bng_re_destroy_chip_ctx(rdev); | 321 | rc = -ENOMEM; |
| 343 | bnge_unregister_dev(rdev->aux_dev); | 322 | goto nq_alloc_fail; |
| 344 | clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | ||
| 345 | return -ENOMEM; | ||
| 346 | } | 323 | } |
| 347 | 324 | ||
| 348 | rdev->nqr->num_msix = rdev->aux_dev->auxr_info->msix_requested; | 325 | rdev->nqr->num_msix = rdev->aux_dev->auxr_info->msix_requested; |
| @@ -411,9 +388,15 @@ disable_rcfw: · 62 unmodified lines | |||
| 411 | free_ring: | 388 | free_ring: |
| 412 | bng_re_net_ring_free(rdev, rdev->rcfw.creq.ring_id, type); | 389 | bng_re_net_ring_free(rdev, rdev->rcfw.creq.ring_id, type); |
| 413 | free_rcfw: | 390 | free_rcfw: |
| 391 | kfree(rdev->nqr); | ||
| 392 | nq_alloc_fail: | ||
| 414 | bng_re_free_rcfw_channel(&rdev->rcfw); | 393 | bng_re_free_rcfw_channel(&rdev->rcfw); |
| 415 | fail: | 394 | alloc_fw_chl_fail: |
| 416 | bng_re_dev_uninit(rdev); | 395 | bng_re_destroy_chip_ctx(rdev); |
| 396 | msix_ctx_fail: | ||
| 397 | bnge_unregister_dev(rdev->aux_dev); | ||
| 398 | clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | ||
| 399 | reg_netdev_fail: | ||
| 417 | return rc; | 400 | return rc; |
| 418 | } | 401 | } |
| 419 | 402 | ||
| @@ -486,8 +469,7 @@ static void bng_re_remove(struct auxiliary_device *adev) · 66 unmodified lines | |||
| 486 | 469 | ||
| 487 | rdev = dev_info->rdev; | 470 | rdev = dev_info->rdev; |
| 488 | 471 | ||
| 489 | if (rdev) | 472 | bng_re_remove_device(rdev, adev); |
| 490 | bng_re_remove_device(rdev, adev); | ||
| 491 | kfree(dev_info); | 473 | kfree(dev_info); |
| 492 | } | 474 | } |
| 493 | 475 | ||
| @@ -1661,7 +1661,7 @@ static struct efa_mr *efa_alloc_mr(struct ib_pd *ibpd, int access_flags, · 1660 unmodified lines | |||
| 1661 | struct efa_mr *mr; | 1661 | struct efa_mr *mr; |
| 1662 | 1662 | ||
| 1663 | if (udata && udata->inlen && | 1663 | if (udata && udata->inlen && |
| 1664 | !ib_is_udata_cleared(udata, 0, sizeof(udata->inlen))) { | 1664 | !ib_is_udata_cleared(udata, 0, udata->inlen)) { |
| 1665 | ibdev_dbg(&dev->ibdev, | 1665 | ibdev_dbg(&dev->ibdev, |
| 1666 | "Incompatible ABI params, udata not cleared\n"); | 1666 | "Incompatible ABI params, udata not cleared\n"); |
| 1667 | return ERR_PTR(-EINVAL); | 1667 | return ERR_PTR(-EINVAL); |
| @@ -1218,7 +1218,7 @@ int ionic_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 1217 unmodified lines | |||
| 1218 | rdma_udata_to_drv_context(udata, struct ionic_ctx, ibctx); | 1218 | rdma_udata_to_drv_context(udata, struct ionic_ctx, ibctx); |
| 1219 | struct ionic_vcq *vcq = to_ionic_vcq(ibcq); | 1219 | struct ionic_vcq *vcq = to_ionic_vcq(ibcq); |
| 1220 | struct ionic_tbl_buf buf = {}; | 1220 | struct ionic_tbl_buf buf = {}; |
| 1221 | struct ionic_cq_resp resp; | 1221 | struct ionic_cq_resp resp = {}; |
| 1222 | struct ionic_cq_req req; | 1222 | struct ionic_cq_req req; |
| 1223 | int udma_idx = 0, rc; | 1223 | int udma_idx = 0, rc; |
| 1224 | 1224 | ||
| @@ -81,6 +81,8 @@ static int ionic_query_port(struct ib_device *ibdev, u32 port, · 80 unmodified lines | |||
| 81 | return -EINVAL; | 81 | return -EINVAL; |
| 82 | 82 | ||
| 83 | ndev = ib_device_get_netdev(ibdev, port); | 83 | ndev = ib_device_get_netdev(ibdev, port); |
| 84 | if (!ndev) | ||
| 85 | return -ENODEV; | ||
| 84 | 86 | ||
| 85 | if (netif_running(ndev) && netif_carrier_ok(ndev)) { | 87 | if (netif_running(ndev) && netif_carrier_ok(ndev)) { |
| 86 | attr->state = IB_PORT_ACTIVE; | 88 | attr->state = IB_PORT_ACTIVE; |
| @@ -5212,7 +5212,7 @@ static int irdma_create_user_ah(struct ib_ah *ibah, · 5211 unmodified lines | |||
| 5212 | #define IRDMA_CREATE_AH_MIN_RESP_LEN offsetofend(struct irdma_create_ah_resp, rsvd) | 5212 | #define IRDMA_CREATE_AH_MIN_RESP_LEN offsetofend(struct irdma_create_ah_resp, rsvd) |
| 5213 | struct irdma_ah *ah = container_of(ibah, struct irdma_ah, ibah); | 5213 | struct irdma_ah *ah = container_of(ibah, struct irdma_ah, ibah); |
| 5214 | struct irdma_device *iwdev = to_iwdev(ibah->pd->device); | 5214 | struct irdma_device *iwdev = to_iwdev(ibah->pd->device); |
| 5215 | struct irdma_create_ah_resp uresp; | 5215 | struct irdma_create_ah_resp uresp = {}; |
| 5216 | struct irdma_ah *parent_ah; | 5216 | struct irdma_ah *parent_ah; |
| 5217 | int err; | 5217 | int err; |
| 5218 | 5218 | ||
| @@ -428,6 +428,8 @@ static int mthca_create_srq(struct ib_srq *ibsrq, · 427 unmodified lines | |||
| 428 | 428 | ||
| 429 | if (context && ib_copy_to_udata(udata, &srq->srqn, sizeof(__u32))) { | 429 | if (context && ib_copy_to_udata(udata, &srq->srqn, sizeof(__u32))) { |
| 430 | mthca_free_srq(to_mdev(ibsrq->device), srq); | 430 | mthca_free_srq(to_mdev(ibsrq->device), srq); |
| 431 | mthca_unmap_user_db(to_mdev(ibsrq->device), &context->uar, | ||
| 432 | context->db_tab, ucmd.db_index); | ||
| 431 | return -EFAULT; | 433 | return -EFAULT; |
| 432 | } | 434 | } |
| 433 | 435 | ||
| @@ -436,6 +438,7 @@ static int mthca_create_srq(struct ib_srq *ibsrq, · 2 unmodified lines | |||
| 436 | 438 | ||
| 437 | static int mthca_destroy_srq(struct ib_srq *srq, struct ib_udata *udata) | 439 | static int mthca_destroy_srq(struct ib_srq *srq, struct ib_udata *udata) |
| 438 | { | 440 | { |
| 441 | mthca_free_srq(to_mdev(srq->device), to_msrq(srq)); | ||
| 439 | if (udata) { | 442 | if (udata) { |
| 440 | struct mthca_ucontext *context = | 443 | struct mthca_ucontext *context = |
| 441 | rdma_udata_to_drv_context( | 444 | rdma_udata_to_drv_context( |
| @@ -446,8 +449,6 @@ static int mthca_destroy_srq(struct ib_srq *srq, struct ib_udata *udata) · 4 unmodified lines | |||
| 446 | mthca_unmap_user_db(to_mdev(srq->device), &context->uar, | 449 | mthca_unmap_user_db(to_mdev(srq->device), &context->uar, |
| 447 | context->db_tab, to_msrq(srq)->db_index); | 450 | context->db_tab, to_msrq(srq)->db_index); |
| 448 | } | 451 | } |
| 449 | |||
| 450 | mthca_free_srq(to_mdev(srq->device), to_msrq(srq)); | ||
| 451 | return 0; | 452 | return 0; |
| 452 | } | 453 | } |
| 453 | 454 | ||
| @@ -3474,6 +3474,7 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, · 3473 unmodified lines | |||
| 3474 | int lpi_base; | 3474 | int lpi_base; |
| 3475 | int nr_lpis; | 3475 | int nr_lpis; |
| 3476 | int nr_ites; | 3476 | int nr_ites; |
| 3477 | int id_bits; | ||
| 3477 | int sz; | 3478 | int sz; |
| 3478 | 3479 | ||
| 3479 | if (!its_alloc_device_table(its, dev_id)) | 3480 | if (!its_alloc_device_table(its, dev_id)) |
| @@ -3485,7 +3486,10 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, · 5 unmodified lines | |||
| 3485 | /* | 3486 | /* |
| 3486 | * Even if the device wants a single LPI, the ITT must be | 3487 | * Even if the device wants a single LPI, the ITT must be |
| 3487 | * sized as a power of two (and you need at least one bit...). | 3488 | * sized as a power of two (and you need at least one bit...). |
| 3489 | * Also honor the ITS's own EID limit. | ||
| 3488 | */ | 3490 | */ |
| 3491 | id_bits = FIELD_GET(GITS_TYPER_IDBITS, its->typer) + 1; | ||
| 3492 | nvecs = min_t(unsigned int, nvecs, BIT(id_bits)); | ||
| 3489 | nr_ites = max(2, nvecs); | 3493 | nr_ites = max(2, nvecs); |
| 3490 | sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); | 3494 | sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); |
| 3491 | sz = max(sz, ITS_ITT_ALIGN); | 3495 | sz = max(sz, ITS_ITT_ALIGN); |
| @@ -699,7 +699,7 @@ static int __init gicv5_irs_init(struct gicv5_irs_chip_data *irs_data) · 698 unmodified lines | |||
| 699 | */ | 699 | */ |
| 700 | if (list_empty(&irs_nodes)) { | 700 | if (list_empty(&irs_nodes)) { |
| 701 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR0); | 701 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR0); |
| 702 | gicv5_global_data.virt_capable = !FIELD_GET(GICV5_IRS_IDR0_VIRT, idr); | 702 | gicv5_global_data.virt_capable = !!FIELD_GET(GICV5_IRS_IDR0_VIRT, idr); |
| 703 | 703 | ||
| 704 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR1); | 704 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR1); |
| 705 | irs_setup_pri_bits(idr); | 705 | irs_setup_pri_bits(idr); |
| @@ -125,32 +125,45 @@ static const struct irq_domain_ops extirq_domain_ops = { · 124 unmodified lines | |||
| 125 | static int | 125 | static int |
| 126 | ls_extirq_parse_map(struct ls_extirq_data *priv, struct device_node *node) | 126 | ls_extirq_parse_map(struct ls_extirq_data *priv, struct device_node *node) |
| 127 | { | 127 | { |
| 128 | struct of_imap_parser imap_parser; | 128 | const __be32 *map; |
| 129 | struct of_imap_item imap_item; | 129 | u32 mapsize; |
| 130 | int ret; | 130 | int ret; |
| 131 | 131 | ||
| 132 | ret = of_imap_parser_init(&imap_parser, node, &imap_item); | 132 | map = of_get_property(node, "interrupt-map", &mapsize); |
| 133 | if (ret) | 133 | if (!map) |
| 134 | return ret; | 134 | return -ENOENT; |
| 135 | if (mapsize % sizeof(*map)) | ||
| 136 | return -EINVAL; | ||
| 137 | mapsize /= sizeof(*map); | ||
| 135 | 138 | ||
| 136 | for_each_of_imap_item(&imap_parser, &imap_item) { | 139 | while (mapsize) { |
| 137 | struct device_node *ipar; | 140 | struct device_node *ipar; |
| 138 | u32 hwirq; | 141 | u32 hwirq, intsize, j; |
| 139 | int i; | ||
| 140 | 142 | ||
| 141 | hwirq = imap_item.child_imap[0]; | 143 | if (mapsize < 3) |
| 142 | if (hwirq >= MAXIRQ) { | 144 | return -EINVAL; |
| 143 | of_node_put(imap_item.parent_args.np); | 145 | hwirq = be32_to_cpup(map); |
| 146 | if (hwirq >= MAXIRQ) | ||
| 144 | return -EINVAL; | 147 | return -EINVAL; |
| 145 | } | ||
| 146 | priv->nirq = max(priv->nirq, hwirq + 1); | 148 | priv->nirq = max(priv->nirq, hwirq + 1); |
| 147 | 149 | ||
| 148 | ipar = of_node_get(imap_item.parent_args.np); | 150 | ipar = of_find_node_by_phandle(be32_to_cpup(map + 2)); |
| 149 | priv->map[hwirq].fwnode = of_fwnode_handle(ipar); | 151 | map += 3; |
| 152 | mapsize -= 3; | ||
| 153 | if (!ipar) | ||
| 154 | return -EINVAL; | ||
| 155 | priv->map[hwirq].fwnode = &ipar->fwnode; | ||
| 156 | ret = of_property_read_u32(ipar, "#interrupt-cells", &intsize); | ||
| 157 | if (ret) | ||
| 158 | return ret; | ||
| 150 | 159 | ||
| 151 | priv->map[hwirq].param_count = imap_item.parent_args.args_count; | 160 | if (intsize > mapsize) |
| 152 | for (i = 0; i < priv->map[hwirq].param_count; i++) | 161 | return -EINVAL; |
| 153 | priv->map[hwirq].param[i] = imap_item.parent_args.args[i]; | 162 | |
| 163 | priv->map[hwirq].param_count = intsize; | ||
| 164 | for (j = 0; j < intsize; ++j) | ||
| 165 | priv->map[hwirq].param[j] = be32_to_cpup(map++); | ||
| 166 | mapsize -= intsize; | ||
| 154 | } | 167 | } |
| 155 | return 0; | 168 | return 0; |
| 156 | } | 169 | } |
| @@ -177,8 +190,10 @@ static int ls_extirq_probe(struct platform_device *pdev) · 20 unmodified lines | |||
| 177 | return dev_err_probe(dev, -ENOMEM, "Failed to allocate memory\n"); | 190 | return dev_err_probe(dev, -ENOMEM, "Failed to allocate memory\n"); |
| 178 | 191 | ||
| 179 | priv->intpcr = devm_of_iomap(dev, node, 0, NULL); | 192 | priv->intpcr = devm_of_iomap(dev, node, 0, NULL); |
| 180 | if (!priv->intpcr) | 193 | if (IS_ERR(priv->intpcr)) { |
| 181 | return dev_err_probe(dev, -ENOMEM, "Cannot ioremap OF node %pOF\n", node); | 194 | return dev_err_probe(dev, PTR_ERR(priv->intpcr), |
| 195 | "Cannot ioremap OF node %pOF\n", node); | ||
| 196 | } | ||
| 182 | 197 | ||
| 183 | ret = ls_extirq_parse_map(priv, node); | 198 | ret = ls_extirq_parse_map(priv, node); |
| 184 | if (ret) | 199 | if (ret) |
| @@ -136,7 +136,7 @@ static void icu_unmask_irq(struct irq_data *d) · 135 unmodified lines | |||
| 136 | } | 136 | } |
| 137 | } | 137 | } |
| 138 | 138 | ||
| 139 | struct irq_chip icu_irq_chip = { | 139 | static const struct irq_chip icu_irq_chip = { |
| 140 | .name = "icu_irq", | 140 | .name = "icu_irq", |
| 141 | .irq_mask = icu_mask_irq, | 141 | .irq_mask = icu_mask_irq, |
| 142 | .irq_mask_ack = icu_mask_ack_irq, | 142 | .irq_mask_ack = icu_mask_ack_irq, |
| @@ -116,6 +116,16 @@ static struct syscore aplic_syscore = { · 115 unmodified lines | |||
| 116 | .ops = &aplic_syscore_ops, | 116 | .ops = &aplic_syscore_ops, |
| 117 | }; | 117 | }; |
| 118 | 118 | ||
| 119 | static bool aplic_syscore_registered __ro_after_init; | ||
| 120 | |||
| 121 | static void aplic_syscore_init(void) | ||
| 122 | { | ||
| 123 | if (!aplic_syscore_registered) { | ||
| 124 | register_syscore(&aplic_syscore); | ||
| 125 | aplic_syscore_registered = true; | ||
| 126 | } | ||
| 127 | } | ||
| 128 | |||
| 119 | static int aplic_pm_notifier(struct notifier_block *nb, unsigned long action, void *data) | 129 | static int aplic_pm_notifier(struct notifier_block *nb, unsigned long action, void *data) |
| 120 | { | 130 | { |
| 121 | struct aplic_priv *priv = container_of(nb, struct aplic_priv, genpd_nb); | 131 | struct aplic_priv *priv = container_of(nb, struct aplic_priv, genpd_nb); |
| @@ -372,18 +382,21 @@ static int aplic_probe(struct platform_device *pdev) · 250 unmodified lines | |||
| 372 | rc = aplic_msi_setup(dev, regs); | 382 | rc = aplic_msi_setup(dev, regs); |
| 373 | else | 383 | else |
| 374 | rc = aplic_direct_setup(dev, regs); | 384 | rc = aplic_direct_setup(dev, regs); |
| 375 | if (rc) | 385 | |
| 386 | if (rc) { | ||
| 376 | dev_err_probe(dev, rc, "failed to setup APLIC in %s mode\n", | 387 | dev_err_probe(dev, rc, "failed to setup APLIC in %s mode\n", |
| 377 | msi_mode ? "MSI" : "direct"); | 388 | msi_mode ? "MSI" : "direct"); |
| 378 | else | 389 | return rc; |
| 379 | register_syscore(&aplic_syscore); | 390 | } |
| 391 | |||
| 392 | aplic_syscore_init(); | ||
| 380 | 393 | ||
| 381 | #ifdef CONFIG_ACPI | 394 | #ifdef CONFIG_ACPI |
| 382 | if (!acpi_disabled) | 395 | if (!acpi_disabled) |
| 383 | acpi_dev_clear_dependencies(ACPI_COMPANION(dev)); | 396 | acpi_dev_clear_dependencies(ACPI_COMPANION(dev)); |
| 384 | #endif | 397 | #endif |
| 385 | 398 | ||
| 386 | return rc; | 399 | return 0; |
| 387 | } | 400 | } |
| 388 | 401 | ||
| 389 | static const struct of_device_id aplic_match[] = { | 402 | static const struct of_device_id aplic_match[] = { |
| @@ -172,8 +172,13 @@ static void plic_irq_disable(struct irq_data *d) · 171 unmodified lines | |||
| 172 | static void plic_irq_eoi(struct irq_data *d) | 172 | static void plic_irq_eoi(struct irq_data *d) |
| 173 | { | 173 | { |
| 174 | struct plic_handler *handler = this_cpu_ptr(&plic_handlers); | 174 | struct plic_handler *handler = this_cpu_ptr(&plic_handlers); |
| 175 | u32 __iomem *reg; | ||
| 176 | bool enabled; | ||
| 177 | |||
| 178 | reg = handler->enable_base + (d->hwirq / 32) * sizeof(u32); | ||
| 179 | enabled = readl(reg) & BIT(d->hwirq % 32); | ||
| 175 | 180 | ||
| 176 | if (unlikely(irqd_irq_disabled(d))) { | 181 | if (unlikely(!enabled)) { |
| 177 | plic_toggle(handler, d->hwirq, 1); | 182 | plic_toggle(handler, d->hwirq, 1); |
| 178 | writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM); | 183 | writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM); |
| 179 | plic_toggle(handler, d->hwirq, 0); | 184 | plic_toggle(handler, d->hwirq, 0); |
| @@ -168,7 +168,9 @@ static int dvb_dvr_open(struct inode *inode, struct file *file) · 167 unmodified lines | |||
| 168 | mutex_unlock(&dmxdev->mutex); | 168 | mutex_unlock(&dmxdev->mutex); |
| 169 | return -ENOMEM; | 169 | return -ENOMEM; |
| 170 | } | 170 | } |
| 171 | dvb_ringbuffer_init(&dmxdev->dvr_buffer, mem, DVR_BUFFER_SIZE); | 171 | dmxdev->dvr_buffer.data = mem; |
| 172 | dmxdev->dvr_buffer.size = DVR_BUFFER_SIZE; | ||
| 173 | dvb_ringbuffer_reset(&dmxdev->dvr_buffer); | ||
| 172 | if (dmxdev->may_do_mmap) | 174 | if (dmxdev->may_do_mmap) |
| 173 | dvb_vb2_init(&dmxdev->dvr_vb2_ctx, "dvr", | 175 | dvb_vb2_init(&dmxdev->dvr_vb2_ctx, "dvr", |
| 174 | &dmxdev->mutex, | 176 | &dmxdev->mutex, |
| @@ -228,6 +228,9 @@ static int handle_one_ule_extension( struct dvb_net_priv *p ) · 227 unmodified lines | |||
| 228 | unsigned char hlen = (p->ule_sndu_type & 0x0700) >> 8; | 228 | unsigned char hlen = (p->ule_sndu_type & 0x0700) >> 8; |
| 229 | unsigned char htype = p->ule_sndu_type & 0x00FF; | 229 | unsigned char htype = p->ule_sndu_type & 0x00FF; |
| 230 | 230 | ||
| 231 | if (htype >= ARRAY_SIZE(ule_mandatory_ext_handlers)) | ||
| 232 | return -1; | ||
| 233 | |||
| 231 | /* Discriminate mandatory and optional extension headers. */ | 234 | /* Discriminate mandatory and optional extension headers. */ |
| 232 | if (hlen == 0) { | 235 | if (hlen == 0) { |
| 233 | /* Mandatory extension header */ | 236 | /* Mandatory extension header */ |
| @@ -1,10 +1,9 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0-only | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | config AMD_SBRMI_I2C | 2 | config AMD_SBRMI_I2C |
| 3 | tristate "AMD side band RMI support" | 3 | tristate "AMD side band RMI support" |
| 4 | depends on I2C | 4 | depends on I3C_OR_I2C |
| 5 | depends on ARM || ARM64 || COMPILE_TEST | 5 | depends on ARM || ARM64 || COMPILE_TEST |
| 6 | select REGMAP_I2C | 6 | select REGMAP_I2C |
| 7 | depends on I3C || !I3C | ||
| 8 | select REGMAP_I3C if I3C | 7 | select REGMAP_I3C if I3C |
| 9 | help | 8 | help |
| 10 | Side band RMI over I2C/I3C support for AMD out of band management. | 9 | Side band RMI over I2C/I3C support for AMD out of band management. |
| @@ -36,6 +36,8 @@ struct dw_mci_rockchip_priv_data { · 35 unmodified lines | |||
| 36 | int default_sample_phase; | 36 | int default_sample_phase; |
| 37 | int num_phases; | 37 | int num_phases; |
| 38 | bool internal_phase; | 38 | bool internal_phase; |
| 39 | int sample_phase; | ||
| 40 | int drv_phase; | ||
| 39 | }; | 41 | }; |
| 40 | 42 | ||
| 41 | /* | 43 | /* |
| @@ -573,9 +575,43 @@ static void dw_mci_rockchip_remove(struct platform_device *pdev) · 531 unmodified lines | |||
| 573 | dw_mci_pltfm_remove(pdev); | 575 | dw_mci_pltfm_remove(pdev); |
| 574 | } | 576 | } |
| 575 | 577 | ||
| 578 | static int dw_mci_rockchip_runtime_suspend(struct device *dev) | ||
| 579 | { | ||
| 580 | struct platform_device *pdev = to_platform_device(dev); | ||
| 581 | struct dw_mci *host = platform_get_drvdata(pdev); | ||
| 582 | struct dw_mci_rockchip_priv_data *priv = host->priv; | ||
| 583 | |||
| 584 | if (priv->internal_phase) { | ||
| 585 | priv->sample_phase = rockchip_mmc_get_phase(host, true); | ||
| 586 | priv->drv_phase = rockchip_mmc_get_phase(host, false); | ||
| 587 | } | ||
| 588 | |||
| 589 | return dw_mci_runtime_suspend(dev); | ||
| 590 | } | ||
| 591 | |||
| 592 | static int dw_mci_rockchip_runtime_resume(struct device *dev) | ||
| 593 | { | ||
| 594 | struct platform_device *pdev = to_platform_device(dev); | ||
| 595 | struct dw_mci *host = platform_get_drvdata(pdev); | ||
| 596 | struct dw_mci_rockchip_priv_data *priv = host->priv; | ||
| 597 | int ret; | ||
| 598 | |||
| 599 | ret = dw_mci_runtime_resume(dev); | ||
| 600 | if (ret) | ||
| 601 | return ret; | ||
| 602 | |||
| 603 | if (priv->internal_phase) { | ||
| 604 | rockchip_mmc_set_phase(host, true, priv->sample_phase); | ||
| 605 | rockchip_mmc_set_phase(host, false, priv->drv_phase); | ||
| 606 | mci_writel(host, MISC_CON, MEM_CLK_AUTOGATE_ENABLE); | ||
| 607 | } | ||
| 608 | |||
| 609 | return ret; | ||
| 610 | } | ||
| 611 | |||
| 576 | static const struct dev_pm_ops dw_mci_rockchip_dev_pm_ops = { | 612 | static const struct dev_pm_ops dw_mci_rockchip_dev_pm_ops = { |
| 577 | SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) | 613 | SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) |
| 578 | RUNTIME_PM_OPS(dw_mci_runtime_suspend, dw_mci_runtime_resume, NULL) | 614 | RUNTIME_PM_OPS(dw_mci_rockchip_runtime_suspend, dw_mci_rockchip_runtime_resume, NULL) |
| 579 | }; | 615 | }; |
| 580 | 616 | ||
| 581 | static struct platform_driver dw_mci_rockchip_pltfm_driver = { | 617 | static struct platform_driver dw_mci_rockchip_pltfm_driver = { |
| @@ -109,6 +109,7 @@ static int of_get_dml_pipe_index(struct device_node *np, const char *name) · 108 unmodified lines | |||
| 109 | &dma_spec)) | 109 | &dma_spec)) |
| 110 | return -ENODEV; | 110 | return -ENODEV; |
| 111 | 111 | ||
| 112 | of_node_put(dma_spec.np); | ||
| 112 | if (dma_spec.args_count) | 113 | if (dma_spec.args_count) |
| 113 | return dma_spec.args[0]; | 114 | return dma_spec.args[0]; |
| 114 | 115 | ||
| @@ -116,7 +116,7 @@ static void sdhci_brcmstb_restore_regs(struct mmc_host *mmc, enum cfg_core_ver v · 115 unmodified lines | |||
| 116 | writel(sr->boot_main_ctl, priv->boot_regs + SDIO_BOOT_MAIN_CTL); | 116 | writel(sr->boot_main_ctl, priv->boot_regs + SDIO_BOOT_MAIN_CTL); |
| 117 | 117 | ||
| 118 | if (ver == SDIO_CFG_CORE_V1) { | 118 | if (ver == SDIO_CFG_CORE_V1) { |
| 119 | writel(sr->sd_pin_sel, cr + SDIO_CFG_SD_PIN_SEL); | 119 | writel(sr->sd_pin_sel, cr + SDIO_CFG_V1_SD_PIN_SEL); |
| 120 | return; | 120 | return; |
| 121 | } | 121 | } |
| 122 | 122 | ||
| @@ -324,7 +324,7 @@ static bool bond_sk_check(struct bonding *bond) · 323 unmodified lines | |||
| 324 | } | 324 | } |
| 325 | } | 325 | } |
| 326 | 326 | ||
| 327 | bool bond_xdp_check(struct bonding *bond, int mode) | 327 | bool __bond_xdp_check(int mode, int xmit_policy) |
| 328 | { | 328 | { |
| 329 | switch (mode) { | 329 | switch (mode) { |
| 330 | case BOND_MODE_ROUNDROBIN: | 330 | case BOND_MODE_ROUNDROBIN: |
| @@ -335,7 +335,7 @@ bool bond_xdp_check(struct bonding *bond, int mode) · 4 unmodified lines | |||
| 335 | /* vlan+srcmac is not supported with XDP as in most cases the 802.1q | 335 | /* vlan+srcmac is not supported with XDP as in most cases the 802.1q |
| 336 | * payload is not in the packet due to hardware offload. | 336 | * payload is not in the packet due to hardware offload. |
| 337 | */ | 337 | */ |
| 338 | if (bond->params.xmit_policy != BOND_XMIT_POLICY_VLAN_SRCMAC) | 338 | if (xmit_policy != BOND_XMIT_POLICY_VLAN_SRCMAC) |
| 339 | return true; | 339 | return true; |
| 340 | fallthrough; | 340 | fallthrough; |
| 341 | default: | 341 | default: |
| @@ -343,6 +343,11 @@ bool bond_xdp_check(struct bonding *bond, int mode) · 1 unmodified line | |||
| 343 | } | 343 | } |
| 344 | } | 344 | } |
| 345 | 345 | ||
| 346 | bool bond_xdp_check(struct bonding *bond, int mode) | ||
| 347 | { | ||
| 348 | return __bond_xdp_check(mode, bond->params.xmit_policy); | ||
| 349 | } | ||
| 350 | |||
| 346 | /*---------------------------------- VLAN -----------------------------------*/ | 351 | /*---------------------------------- VLAN -----------------------------------*/ |
| 347 | 352 | ||
| 348 | /* In the following 2 functions, bond_vlan_rx_add_vid and bond_vlan_rx_kill_vid, | 353 | /* In the following 2 functions, bond_vlan_rx_add_vid and bond_vlan_rx_kill_vid, |
| @@ -1504,6 +1509,50 @@ static netdev_features_t bond_fix_features(struct net_device *dev, · 1155 unmodified lines | |||
| 1504 | return features; | 1509 | return features; |
| 1505 | } | 1510 | } |
| 1506 | 1511 | ||
| 1512 | static int bond_header_create(struct sk_buff *skb, struct net_device *bond_dev, | ||
| 1513 | unsigned short type, const void *daddr, | ||
| 1514 | const void *saddr, unsigned int len) | ||
| 1515 | { | ||
| 1516 | struct bonding *bond = netdev_priv(bond_dev); | ||
| 1517 | const struct header_ops *slave_ops; | ||
| 1518 | struct slave *slave; | ||
| 1519 | int ret = 0; | ||
| 1520 | |||
| 1521 | rcu_read_lock(); | ||
| 1522 | slave = rcu_dereference(bond->curr_active_slave); | ||
| 1523 | if (slave) { | ||
| 1524 | slave_ops = READ_ONCE(slave->dev->header_ops); | ||
| 1525 | if (slave_ops && slave_ops->create) | ||
| 1526 | ret = slave_ops->create(skb, slave->dev, | ||
| 1527 | type, daddr, saddr, len); | ||
| 1528 | } | ||
| 1529 | rcu_read_unlock(); | ||
| 1530 | return ret; | ||
| 1531 | } | ||
| 1532 | |||
| 1533 | static int bond_header_parse(const struct sk_buff *skb, unsigned char *haddr) | ||
| 1534 | { | ||
| 1535 | struct bonding *bond = netdev_priv(skb->dev); | ||
| 1536 | const struct header_ops *slave_ops; | ||
| 1537 | struct slave *slave; | ||
| 1538 | int ret = 0; | ||
| 1539 | |||
| 1540 | rcu_read_lock(); | ||
| 1541 | slave = rcu_dereference(bond->curr_active_slave); | ||
| 1542 | if (slave) { | ||
| 1543 | slave_ops = READ_ONCE(slave->dev->header_ops); | ||
| 1544 | if (slave_ops && slave_ops->parse) | ||
| 1545 | ret = slave_ops->parse(skb, haddr); | ||
| 1546 | } | ||
| 1547 | rcu_read_unlock(); | ||
| 1548 | return ret; | ||
| 1549 | } | ||
| 1550 | |||
| 1551 | static const struct header_ops bond_header_ops = { | ||
| 1552 | .create = bond_header_create, | ||
| 1553 | .parse = bond_header_parse, | ||
| 1554 | }; | ||
| 1555 | |||
| 1507 | static void bond_setup_by_slave(struct net_device *bond_dev, | 1556 | static void bond_setup_by_slave(struct net_device *bond_dev, |
| 1508 | struct net_device *slave_dev) | 1557 | struct net_device *slave_dev) |
| 1509 | { | 1558 | { |
| @@ -1511,7 +1560,8 @@ static void bond_setup_by_slave(struct net_device *bond_dev, · 1 unmodified line | |||
| 1511 | 1560 | ||
| 1512 | dev_close(bond_dev); | 1561 | dev_close(bond_dev); |
| 1513 | 1562 | ||
| 1514 | bond_dev->header_ops = slave_dev->header_ops; | 1563 | bond_dev->header_ops = slave_dev->header_ops ? |
| 1564 | &bond_header_ops : NULL; | ||
| 1515 | 1565 | ||
| 1516 | bond_dev->type = slave_dev->type; | 1566 | bond_dev->type = slave_dev->type; |
| 1517 | bond_dev->hard_header_len = slave_dev->hard_header_len; | 1567 | bond_dev->hard_header_len = slave_dev->hard_header_len; |
| @@ -2796,8 +2846,14 @@ static void bond_miimon_commit(struct bonding *bond) · 1278 unmodified lines | |||
| 2796 | 2846 | ||
| 2797 | continue; | 2847 | continue; |
| 2798 | 2848 | ||
| 2849 | case BOND_LINK_FAIL: | ||
| 2850 | case BOND_LINK_BACK: | ||
| 2851 | slave_dbg(bond->dev, slave->dev, "link_new_state %d on slave\n", | ||
| 2852 | slave->link_new_state); | ||
| 2853 | continue; | ||
| 2854 | |||
| 2799 | default: | 2855 | default: |
| 2800 | slave_err(bond->dev, slave->dev, "invalid new link %d on slave\n", | 2856 | slave_err(bond->dev, slave->dev, "invalid link_new_state %d on slave\n", |
| 2801 | slave->link_new_state); | 2857 | slave->link_new_state); |
| 2802 | bond_propose_link_state(slave, BOND_LINK_NOCHANGE); | 2858 | bond_propose_link_state(slave, BOND_LINK_NOCHANGE); |
| 2803 | 2859 | ||
| @@ -3372,7 +3428,7 @@ int bond_rcv_validate(const struct sk_buff *skb, struct bonding *bond, · 568 unmodified lines | |||
| 3372 | } else if (is_arp) { | 3428 | } else if (is_arp) { |
| 3373 | return bond_arp_rcv(skb, bond, slave); | 3429 | return bond_arp_rcv(skb, bond, slave); |
| 3374 | #if IS_ENABLED(CONFIG_IPV6) | 3430 | #if IS_ENABLED(CONFIG_IPV6) |
| 3375 | } else if (is_ipv6) { | 3431 | } else if (is_ipv6 && likely(ipv6_mod_enabled())) { |
| 3376 | return bond_na_rcv(skb, bond, slave); | 3432 | return bond_na_rcv(skb, bond, slave); |
| 3377 | #endif | 3433 | #endif |
| 3378 | } else { | 3434 | } else { |
| @@ -5064,13 +5120,18 @@ static void bond_set_slave_arr(struct bonding *bond, · 1685 unmodified lines | |||
| 5064 | { | 5120 | { |
| 5065 | struct bond_up_slave *usable, *all; | 5121 | struct bond_up_slave *usable, *all; |
| 5066 | 5122 | ||
| 5067 | usable = rtnl_dereference(bond->usable_slaves); | ||
| 5068 | rcu_assign_pointer(bond->usable_slaves, usable_slaves); | ||
| 5069 | kfree_rcu(usable, rcu); | ||
| 5070 | |||
| 5071 | all = rtnl_dereference(bond->all_slaves); | 5123 | all = rtnl_dereference(bond->all_slaves); |
| 5072 | rcu_assign_pointer(bond->all_slaves, all_slaves); | 5124 | rcu_assign_pointer(bond->all_slaves, all_slaves); |
| 5073 | kfree_rcu(all, rcu); | 5125 | kfree_rcu(all, rcu); |
| 5126 | |||
| 5127 | if (BOND_MODE(bond) == BOND_MODE_BROADCAST) { | ||
| 5128 | kfree_rcu(usable_slaves, rcu); | ||
| 5129 | return; | ||
| 5130 | } | ||
| 5131 | |||
| 5132 | usable = rtnl_dereference(bond->usable_slaves); | ||
| 5133 | rcu_assign_pointer(bond->usable_slaves, usable_slaves); | ||
| 5134 | kfree_rcu(usable, rcu); | ||
| 5074 | } | 5135 | } |
| 5075 | 5136 | ||
| 5076 | static void bond_reset_slave_arr(struct bonding *bond) | 5137 | static void bond_reset_slave_arr(struct bonding *bond) |
| @@ -1575,6 +1575,8 @@ static int bond_option_fail_over_mac_set(struct bonding *bond, · 1574 unmodified lines | |||
| 1575 | static int bond_option_xmit_hash_policy_set(struct bonding *bond, | 1575 | static int bond_option_xmit_hash_policy_set(struct bonding *bond, |
| 1576 | const struct bond_opt_value *newval) | 1576 | const struct bond_opt_value *newval) |
| 1577 | { | 1577 | { |
| 1578 | if (bond->xdp_prog && !__bond_xdp_check(BOND_MODE(bond), newval->value)) | ||
| 1579 | return -EOPNOTSUPP; | ||
| 1578 | netdev_dbg(bond->dev, "Setting xmit hash policy to %s (%llu)\n", | 1580 | netdev_dbg(bond->dev, "Setting xmit hash policy to %s (%llu)\n", |
| 1579 | newval->string, newval->value); | 1581 | newval->string, newval->value); |
| 1580 | bond->params.xmit_policy = newval->value; | 1582 | bond->params.xmit_policy = newval->value; |
| @@ -297,6 +297,7 @@ static void ser_release(struct work_struct *work) · 296 unmodified lines | |||
| 297 | dev_close(ser->dev); | 297 | dev_close(ser->dev); |
| 298 | unregister_netdevice(ser->dev); | 298 | unregister_netdevice(ser->dev); |
| 299 | debugfs_deinit(ser); | 299 | debugfs_deinit(ser); |
| 300 | tty_kref_put(tty->link); | ||
| 300 | tty_kref_put(tty); | 301 | tty_kref_put(tty); |
| 301 | } | 302 | } |
| 302 | rtnl_unlock(); | 303 | rtnl_unlock(); |
| @@ -331,6 +332,7 @@ static int ldisc_open(struct tty_struct *tty) · 28 unmodified lines | |||
| 331 | 332 | ||
| 332 | ser = netdev_priv(dev); | 333 | ser = netdev_priv(dev); |
| 333 | ser->tty = tty_kref_get(tty); | 334 | ser->tty = tty_kref_get(tty); |
| 335 | tty_kref_get(tty->link); | ||
| 334 | ser->dev = dev; | 336 | ser->dev = dev; |
| 335 | debugfs_init(ser, tty); | 337 | debugfs_init(ser, tty); |
| 336 | tty->receive_room = 4096; | 338 | tty->receive_room = 4096; |
| @@ -339,6 +341,7 @@ static int ldisc_open(struct tty_struct *tty) · 2 unmodified lines | |||
| 339 | rtnl_lock(); | 341 | rtnl_lock(); |
| 340 | result = register_netdevice(dev); | 342 | result = register_netdevice(dev); |
| 341 | if (result) { | 343 | if (result) { |
| 344 | tty_kref_put(tty->link); | ||
| 342 | tty_kref_put(tty); | 345 | tty_kref_put(tty); |
| 343 | rtnl_unlock(); | 346 | rtnl_unlock(); |
| 344 | free_netdev(dev); | 347 | free_netdev(dev); |
| @@ -8,7 +8,7 @@ · 7 unmodified lines | |||
| 8 | #include <linux/units.h> | 8 | #include <linux/units.h> |
| 9 | #include <linux/can/dev.h> | 9 | #include <linux/can/dev.h> |
| 10 | 10 | ||
| 11 | #define CAN_CALC_MAX_ERROR 50 /* in one-tenth of a percent */ | 11 | #define CAN_CALC_MAX_ERROR 500 /* max error 5% */ |
| 12 | 12 | ||
| 13 | /* CiA recommended sample points for Non Return to Zero encoding. */ | 13 | /* CiA recommended sample points for Non Return to Zero encoding. */ |
| 14 | static int can_calc_sample_point_nrz(const struct can_bittiming *bt) | 14 | static int can_calc_sample_point_nrz(const struct can_bittiming *bt) |
| @@ -241,6 +241,7 @@ static int __init dummy_can_init(void) · 240 unmodified lines | |||
| 241 | 241 | ||
| 242 | dev->netdev_ops = &dummy_can_netdev_ops; | 242 | dev->netdev_ops = &dummy_can_netdev_ops; |
| 243 | dev->ethtool_ops = &dummy_can_ethtool_ops; | 243 | dev->ethtool_ops = &dummy_can_ethtool_ops; |
| 244 | dev->flags |= IFF_ECHO; /* enable echo handling */ | ||
| 244 | priv = netdev_priv(dev); | 245 | priv = netdev_priv(dev); |
| 245 | priv->can.bittiming_const = &dummy_can_bittiming_const; | 246 | priv->can.bittiming_const = &dummy_can_bittiming_const; |
| 246 | priv->can.bitrate_max = 20 * MEGA /* BPS */; | 247 | priv->can.bitrate_max = 20 * MEGA /* BPS */; |
| @@ -755,7 +755,9 @@ static int hi3110_open(struct net_device *net) · 754 unmodified lines | |||
| 755 | return ret; | 755 | return ret; |
| 756 | 756 | ||
| 757 | mutex_lock(&priv->hi3110_lock); | 757 | mutex_lock(&priv->hi3110_lock); |
| 758 | hi3110_power_enable(priv->transceiver, 1); | 758 | ret = hi3110_power_enable(priv->transceiver, 1); |
| 759 | if (ret) | ||
| 760 | goto out_close_candev; | ||
| 759 | 761 | ||
| 760 | priv->force_quit = 0; | 762 | priv->force_quit = 0; |
| 761 | priv->tx_skb = NULL; | 763 | priv->tx_skb = NULL; |
| @@ -790,6 +792,7 @@ static int hi3110_open(struct net_device *net) · 28 unmodified lines | |||
| 790 | hi3110_hw_sleep(spi); | 792 | hi3110_hw_sleep(spi); |
| 791 | out_close: | 793 | out_close: |
| 792 | hi3110_power_enable(priv->transceiver, 0); | 794 | hi3110_power_enable(priv->transceiver, 0); |
| 795 | out_close_candev: | ||
| 793 | close_candev(net); | 796 | close_candev(net); |
| 794 | mutex_unlock(&priv->hi3110_lock); | 797 | mutex_unlock(&priv->hi3110_lock); |
| 795 | return ret; | 798 | return ret; |
| @@ -1214,6 +1214,7 @@ static int mcp251x_open(struct net_device *net) · 1213 unmodified lines | |||
| 1214 | { | 1214 | { |
| 1215 | struct mcp251x_priv *priv = netdev_priv(net); | 1215 | struct mcp251x_priv *priv = netdev_priv(net); |
| 1216 | struct spi_device *spi = priv->spi; | 1216 | struct spi_device *spi = priv->spi; |
| 1217 | bool release_irq = false; | ||
| 1217 | unsigned long flags = 0; | 1218 | unsigned long flags = 0; |
| 1218 | int ret; | 1219 | int ret; |
| 1219 | 1220 | ||
| @@ -1257,12 +1258,24 @@ static int mcp251x_open(struct net_device *net) · 37 unmodified lines | |||
| 1257 | return 0; | 1258 | return 0; |
| 1258 | 1259 | ||
| 1259 | out_free_irq: | 1260 | out_free_irq: |
| 1260 | free_irq(spi->irq, priv); | 1261 | /* The IRQ handler might be running, and if so it will be waiting |
| 1262 | * for the lock. But free_irq() must wait for the handler to finish | ||
| 1263 | * so calling it here would deadlock. | ||
| 1264 | * | ||
| 1265 | * Setting priv->force_quit will let the handler exit right away | ||
| 1266 | * without any access to the hardware. This make it safe to call | ||
| 1267 | * free_irq() after the lock is released. | ||
| 1268 | */ | ||
| 1269 | priv->force_quit = 1; | ||
| 1270 | release_irq = true; | ||
| 1271 | |||
| 1261 | mcp251x_hw_sleep(spi); | 1272 | mcp251x_hw_sleep(spi); |
| 1262 | out_close: | 1273 | out_close: |
| 1263 | mcp251x_power_enable(priv->transceiver, 0); | 1274 | mcp251x_power_enable(priv->transceiver, 0); |
| 1264 | close_candev(net); | 1275 | close_candev(net); |
| 1265 | mutex_unlock(&priv->mcp_lock); | 1276 | mutex_unlock(&priv->mcp_lock); |
| 1277 | if (release_irq) | ||
| 1278 | free_irq(spi->irq, priv); | ||
| 1266 | return ret; | 1279 | return ret; |
| 1267 | } | 1280 | } |
| 1268 | 1281 | ||
| @@ -445,6 +445,11 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 444 unmodified lines | |||
| 445 | start = CPC_HEADER_SIZE; | 445 | start = CPC_HEADER_SIZE; |
| 446 | 446 | ||
| 447 | while (msg_count) { | 447 | while (msg_count) { |
| 448 | if (start + CPC_MSG_HEADER_LEN > urb->actual_length) { | ||
| 449 | netdev_err(netdev, "format error\n"); | ||
| 450 | break; | ||
| 451 | } | ||
| 452 | |||
| 448 | msg = (struct ems_cpc_msg *)&ibuf[start]; | 453 | msg = (struct ems_cpc_msg *)&ibuf[start]; |
| 449 | 454 | ||
| 450 | switch (msg->type) { | 455 | switch (msg->type) { |
| @@ -474,7 +479,7 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 23 unmodified lines | |||
| 474 | start += CPC_MSG_HEADER_LEN + msg->length; | 479 | start += CPC_MSG_HEADER_LEN + msg->length; |
| 475 | msg_count--; | 480 | msg_count--; |
| 476 | 481 | ||
| 477 | if (start > urb->transfer_buffer_length) { | 482 | if (start > urb->actual_length) { |
| 478 | netdev_err(netdev, "format error\n"); | 483 | netdev_err(netdev, "format error\n"); |
| 479 | break; | 484 | break; |
| 480 | } | 485 | } |
| @@ -272,6 +272,9 @@ struct esd_usb { · 271 unmodified lines | |||
| 272 | 272 | ||
| 273 | struct usb_anchor rx_submitted; | 273 | struct usb_anchor rx_submitted; |
| 274 | 274 | ||
| 275 | unsigned int rx_pipe; | ||
| 276 | unsigned int tx_pipe; | ||
| 277 | |||
| 275 | int net_count; | 278 | int net_count; |
| 276 | u32 version; | 279 | u32 version; |
| 277 | int rxinitdone; | 280 | int rxinitdone; |
| @@ -537,7 +540,7 @@ static void esd_usb_read_bulk_callback(struct urb *urb) · 259 unmodified lines | |||
| 537 | } | 540 | } |
| 538 | 541 | ||
| 539 | resubmit_urb: | 542 | resubmit_urb: |
| 540 | usb_fill_bulk_urb(urb, dev->udev, usb_rcvbulkpipe(dev->udev, 1), | 543 | usb_fill_bulk_urb(urb, dev->udev, dev->rx_pipe, |
| 541 | urb->transfer_buffer, ESD_USB_RX_BUFFER_SIZE, | 544 | urb->transfer_buffer, ESD_USB_RX_BUFFER_SIZE, |
| 542 | esd_usb_read_bulk_callback, dev); | 545 | esd_usb_read_bulk_callback, dev); |
| 543 | 546 | ||
| @@ -626,9 +629,7 @@ static int esd_usb_send_msg(struct esd_usb *dev, union esd_usb_msg *msg) · 82 unmodified lines | |||
| 626 | { | 629 | { |
| 627 | int actual_length; | 630 | int actual_length; |
| 628 | 631 | ||
| 629 | return usb_bulk_msg(dev->udev, | 632 | return usb_bulk_msg(dev->udev, dev->tx_pipe, msg, |
| 630 | usb_sndbulkpipe(dev->udev, 2), | ||
| 631 | msg, | ||
| 632 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ | 633 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ |
| 633 | &actual_length, | 634 | &actual_length, |
| 634 | 1000); | 635 | 1000); |
| @@ -639,12 +640,8 @@ static int esd_usb_wait_msg(struct esd_usb *dev, · 4 unmodified lines | |||
| 639 | { | 640 | { |
| 640 | int actual_length; | 641 | int actual_length; |
| 641 | 642 | ||
| 642 | return usb_bulk_msg(dev->udev, | 643 | return usb_bulk_msg(dev->udev, dev->rx_pipe, msg, |
| 643 | usb_rcvbulkpipe(dev->udev, 1), | 644 | sizeof(*msg), &actual_length, 1000); |
| 644 | msg, | ||
| 645 | sizeof(*msg), | ||
| 646 | &actual_length, | ||
| 647 | 1000); | ||
| 648 | } | 645 | } |
| 649 | 646 | ||
| 650 | static int esd_usb_setup_rx_urbs(struct esd_usb *dev) | 647 | static int esd_usb_setup_rx_urbs(struct esd_usb *dev) |
| @@ -677,8 +674,7 @@ static int esd_usb_setup_rx_urbs(struct esd_usb *dev) · 26 unmodified lines | |||
| 677 | 674 | ||
| 678 | urb->transfer_dma = buf_dma; | 675 | urb->transfer_dma = buf_dma; |
| 679 | 676 | ||
| 680 | usb_fill_bulk_urb(urb, dev->udev, | 677 | usb_fill_bulk_urb(urb, dev->udev, dev->rx_pipe, |
| 681 | usb_rcvbulkpipe(dev->udev, 1), | ||
| 682 | buf, ESD_USB_RX_BUFFER_SIZE, | 678 | buf, ESD_USB_RX_BUFFER_SIZE, |
| 683 | esd_usb_read_bulk_callback, dev); | 679 | esd_usb_read_bulk_callback, dev); |
| 684 | urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP; | 680 | urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP; |
| @@ -903,7 +899,7 @@ static netdev_tx_t esd_usb_start_xmit(struct sk_buff *skb, · 218 unmodified lines | |||
| 903 | /* hnd must not be 0 - MSB is stripped in txdone handling */ | 899 | /* hnd must not be 0 - MSB is stripped in txdone handling */ |
| 904 | msg->tx.hnd = BIT(31) | i; /* returned in TX done message */ | 900 | msg->tx.hnd = BIT(31) | i; /* returned in TX done message */ |
| 905 | 901 | ||
| 906 | usb_fill_bulk_urb(urb, dev->udev, usb_sndbulkpipe(dev->udev, 2), buf, | 902 | usb_fill_bulk_urb(urb, dev->udev, dev->tx_pipe, buf, |
| 907 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ | 903 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ |
| 908 | esd_usb_write_bulk_callback, context); | 904 | esd_usb_write_bulk_callback, context); |
| 909 | 905 | ||
| @@ -1298,10 +1294,16 @@ done: · 388 unmodified lines | |||
| 1298 | static int esd_usb_probe(struct usb_interface *intf, | 1294 | static int esd_usb_probe(struct usb_interface *intf, |
| 1299 | const struct usb_device_id *id) | 1295 | const struct usb_device_id *id) |
| 1300 | { | 1296 | { |
| 1297 | struct usb_endpoint_descriptor *ep_in, *ep_out; | ||
| 1301 | struct esd_usb *dev; | 1298 | struct esd_usb *dev; |
| 1302 | union esd_usb_msg *msg; | 1299 | union esd_usb_msg *msg; |
| 1303 | int i, err; | 1300 | int i, err; |
| 1304 | 1301 | ||
| 1302 | err = usb_find_common_endpoints(intf->cur_altsetting, &ep_in, &ep_out, | ||
| 1303 | NULL, NULL); | ||
| 1304 | if (err) | ||
| 1305 | return err; | ||
| 1306 | |||
| 1305 | dev = kzalloc_obj(*dev); | 1307 | dev = kzalloc_obj(*dev); |
| 1306 | if (!dev) { | 1308 | if (!dev) { |
| 1307 | err = -ENOMEM; | 1309 | err = -ENOMEM; |
| @@ -1309,6 +1311,8 @@ static int esd_usb_probe(struct usb_interface *intf, · 1 unmodified line | |||
| 1309 | } | 1311 | } |
| 1310 | 1312 | ||
| 1311 | dev->udev = interface_to_usbdev(intf); | 1313 | dev->udev = interface_to_usbdev(intf); |
| 1314 | dev->rx_pipe = usb_rcvbulkpipe(dev->udev, ep_in->bEndpointAddress); | ||
| 1315 | dev->tx_pipe = usb_sndbulkpipe(dev->udev, ep_out->bEndpointAddress); | ||
| 1312 | 1316 | ||
| 1313 | init_usb_anchor(&dev->rx_submitted); | 1317 | init_usb_anchor(&dev->rx_submitted); |
| 1314 | 1318 | ||
| @@ -1461,12 +1461,18 @@ static void es58x_read_bulk_callback(struct urb *urb) · 1460 unmodified lines | |||
| 1461 | } | 1461 | } |
| 1462 | 1462 | ||
| 1463 | resubmit_urb: | 1463 | resubmit_urb: |
| 1464 | usb_anchor_urb(urb, &es58x_dev->rx_urbs); | ||
| 1464 | ret = usb_submit_urb(urb, GFP_ATOMIC); | 1465 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 1466 | if (!ret) | ||
| 1467 | return; | ||
| 1468 | |||
| 1469 | usb_unanchor_urb(urb); | ||
| 1470 | |||
| 1465 | if (ret == -ENODEV) { | 1471 | if (ret == -ENODEV) { |
| 1466 | for (i = 0; i < es58x_dev->num_can_ch; i++) | 1472 | for (i = 0; i < es58x_dev->num_can_ch; i++) |
| 1467 | if (es58x_dev->netdev[i]) | 1473 | if (es58x_dev->netdev[i]) |
| 1468 | netif_device_detach(es58x_dev->netdev[i]); | 1474 | netif_device_detach(es58x_dev->netdev[i]); |
| 1469 | } else if (ret) | 1475 | } else |
| 1470 | dev_err_ratelimited(dev, | 1476 | dev_err_ratelimited(dev, |
| 1471 | "Failed resubmitting read bulk urb: %pe\n", | 1477 | "Failed resubmitting read bulk urb: %pe\n", |
| 1472 | ERR_PTR(ret)); | 1478 | ERR_PTR(ret)); |
| @@ -413,6 +413,7 @@ static void f81604_read_bulk_callback(struct urb *urb) · 412 unmodified lines | |||
| 413 | { | 413 | { |
| 414 | struct f81604_can_frame *frame = urb->transfer_buffer; | 414 | struct f81604_can_frame *frame = urb->transfer_buffer; |
| 415 | struct net_device *netdev = urb->context; | 415 | struct net_device *netdev = urb->context; |
| 416 | struct f81604_port_priv *priv = netdev_priv(netdev); | ||
| 416 | int ret; | 417 | int ret; |
| 417 | 418 | ||
| 418 | if (!netif_device_present(netdev)) | 419 | if (!netif_device_present(netdev)) |
| @@ -445,10 +446,15 @@ static void f81604_read_bulk_callback(struct urb *urb) · 26 unmodified lines | |||
| 445 | f81604_process_rx_packet(netdev, frame); | 446 | f81604_process_rx_packet(netdev, frame); |
| 446 | 447 | ||
| 447 | resubmit_urb: | 448 | resubmit_urb: |
| 449 | usb_anchor_urb(urb, &priv->urbs_anchor); | ||
| 448 | ret = usb_submit_urb(urb, GFP_ATOMIC); | 450 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 451 | if (!ret) | ||
| 452 | return; | ||
| 453 | usb_unanchor_urb(urb); | ||
| 454 | |||
| 449 | if (ret == -ENODEV) | 455 | if (ret == -ENODEV) |
| 450 | netif_device_detach(netdev); | 456 | netif_device_detach(netdev); |
| 451 | else if (ret) | 457 | else |
| 452 | netdev_err(netdev, | 458 | netdev_err(netdev, |
| 453 | "%s: failed to resubmit read bulk urb: %pe\n", | 459 | "%s: failed to resubmit read bulk urb: %pe\n", |
| 454 | __func__, ERR_PTR(ret)); | 460 | __func__, ERR_PTR(ret)); |
| @@ -620,6 +626,12 @@ static void f81604_read_int_callback(struct urb *urb) · 165 unmodified lines | |||
| 620 | netdev_info(netdev, "%s: Int URB aborted: %pe\n", __func__, | 626 | netdev_info(netdev, "%s: Int URB aborted: %pe\n", __func__, |
| 621 | ERR_PTR(urb->status)); | 627 | ERR_PTR(urb->status)); |
| 622 | 628 | ||
| 629 | if (urb->actual_length < sizeof(*data)) { | ||
| 630 | netdev_warn(netdev, "%s: short int URB: %u < %zu\n", | ||
| 631 | __func__, urb->actual_length, sizeof(*data)); | ||
| 632 | goto resubmit_urb; | ||
| 633 | } | ||
| 634 | |||
| 623 | switch (urb->status) { | 635 | switch (urb->status) { |
| 624 | case 0: /* success */ | 636 | case 0: /* success */ |
| 625 | break; | 637 | break; |
| @@ -646,10 +658,15 @@ static void f81604_read_int_callback(struct urb *urb) · 20 unmodified lines | |||
| 646 | f81604_handle_tx(priv, data); | 658 | f81604_handle_tx(priv, data); |
| 647 | 659 | ||
| 648 | resubmit_urb: | 660 | resubmit_urb: |
| 661 | usb_anchor_urb(urb, &priv->urbs_anchor); | ||
| 649 | ret = usb_submit_urb(urb, GFP_ATOMIC); | 662 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 663 | if (!ret) | ||
| 664 | return; | ||
| 665 | usb_unanchor_urb(urb); | ||
| 666 | |||
| 650 | if (ret == -ENODEV) | 667 | if (ret == -ENODEV) |
| 651 | netif_device_detach(netdev); | 668 | netif_device_detach(netdev); |
| 652 | else if (ret) | 669 | else |
| 653 | netdev_err(netdev, "%s: failed to resubmit int urb: %pe\n", | 670 | netdev_err(netdev, "%s: failed to resubmit int urb: %pe\n", |
| 654 | __func__, ERR_PTR(ret)); | 671 | __func__, ERR_PTR(ret)); |
| 655 | } | 672 | } |
| @@ -874,9 +891,27 @@ static void f81604_write_bulk_callback(struct urb *urb) · 218 unmodified lines | |||
| 874 | if (!netif_device_present(netdev)) | 891 | if (!netif_device_present(netdev)) |
| 875 | return; | 892 | return; |
| 876 | 893 | ||
| 877 | if (urb->status) | 894 | if (!urb->status) |
| 878 | netdev_info(netdev, "%s: Tx URB error: %pe\n", __func__, | 895 | return; |
| 879 | ERR_PTR(urb->status)); | 896 | |
| 897 | switch (urb->status) { | ||
| 898 | case -ENOENT: | ||
| 899 | case -ECONNRESET: | ||
| 900 | case -ESHUTDOWN: | ||
| 901 | return; | ||
| 902 | default: | ||
| 903 | break; | ||
| 904 | } | ||
| 905 | |||
| 906 | if (net_ratelimit()) | ||
| 907 | netdev_err(netdev, "%s: Tx URB error: %pe\n", __func__, | ||
| 908 | ERR_PTR(urb->status)); | ||
| 909 | |||
| 910 | can_free_echo_skb(netdev, 0, NULL); | ||
| 911 | netdev->stats.tx_dropped++; | ||
| 912 | netdev->stats.tx_errors++; | ||
| 913 | |||
| 914 | netif_wake_queue(netdev); | ||
| 880 | } | 915 | } |
| 881 | 916 | ||
| 882 | static void f81604_clear_reg_work(struct work_struct *work) | 917 | static void f81604_clear_reg_work(struct work_struct *work) |
| @@ -772,9 +772,8 @@ device_detach: · 771 unmodified lines | |||
| 772 | } | 772 | } |
| 773 | } | 773 | } |
| 774 | 774 | ||
| 775 | static int gs_usb_set_bittiming(struct net_device *netdev) | 775 | static int gs_usb_set_bittiming(struct gs_can *dev) |
| 776 | { | 776 | { |
| 777 | struct gs_can *dev = netdev_priv(netdev); | ||
| 778 | struct can_bittiming *bt = &dev->can.bittiming; | 777 | struct can_bittiming *bt = &dev->can.bittiming; |
| 779 | struct gs_device_bittiming dbt = { | 778 | struct gs_device_bittiming dbt = { |
| 780 | .prop_seg = cpu_to_le32(bt->prop_seg), | 779 | .prop_seg = cpu_to_le32(bt->prop_seg), |
| @@ -791,9 +790,8 @@ static int gs_usb_set_bittiming(struct net_device *netdev) · 10 unmodified lines | |||
| 791 | GFP_KERNEL); | 790 | GFP_KERNEL); |
| 792 | } | 791 | } |
| 793 | 792 | ||
| 794 | static int gs_usb_set_data_bittiming(struct net_device *netdev) | 793 | static int gs_usb_set_data_bittiming(struct gs_can *dev) |
| 795 | { | 794 | { |
| 796 | struct gs_can *dev = netdev_priv(netdev); | ||
| 797 | struct can_bittiming *bt = &dev->can.fd.data_bittiming; | 795 | struct can_bittiming *bt = &dev->can.fd.data_bittiming; |
| 798 | struct gs_device_bittiming dbt = { | 796 | struct gs_device_bittiming dbt = { |
| 799 | .prop_seg = cpu_to_le32(bt->prop_seg), | 797 | .prop_seg = cpu_to_le32(bt->prop_seg), |
| @@ -1057,6 +1055,20 @@ static int gs_can_open(struct net_device *netdev) · 257 unmodified lines | |||
| 1057 | if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) | 1055 | if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) |
| 1058 | flags |= GS_CAN_MODE_HW_TIMESTAMP; | 1056 | flags |= GS_CAN_MODE_HW_TIMESTAMP; |
| 1059 | 1057 | ||
| 1058 | rc = gs_usb_set_bittiming(dev); | ||
| 1059 | if (rc) { | ||
| 1060 | netdev_err(netdev, "failed to set bittiming: %pe\n", ERR_PTR(rc)); | ||
| 1061 | goto out_usb_kill_anchored_urbs; | ||
| 1062 | } | ||
| 1063 | |||
| 1064 | if (ctrlmode & CAN_CTRLMODE_FD) { | ||
| 1065 | rc = gs_usb_set_data_bittiming(dev); | ||
| 1066 | if (rc) { | ||
| 1067 | netdev_err(netdev, "failed to set data bittiming: %pe\n", ERR_PTR(rc)); | ||
| 1068 | goto out_usb_kill_anchored_urbs; | ||
| 1069 | } | ||
| 1070 | } | ||
| 1071 | |||
| 1060 | /* finally start device */ | 1072 | /* finally start device */ |
| 1061 | dev->can.state = CAN_STATE_ERROR_ACTIVE; | 1073 | dev->can.state = CAN_STATE_ERROR_ACTIVE; |
| 1062 | dm.flags = cpu_to_le32(flags); | 1074 | dm.flags = cpu_to_le32(flags); |
| @@ -1370,7 +1382,6 @@ static struct gs_can *gs_make_candev(unsigned int channel, · 307 unmodified lines | |||
| 1370 | dev->can.state = CAN_STATE_STOPPED; | 1382 | dev->can.state = CAN_STATE_STOPPED; |
| 1371 | dev->can.clock.freq = le32_to_cpu(bt_const.fclk_can); | 1383 | dev->can.clock.freq = le32_to_cpu(bt_const.fclk_can); |
| 1372 | dev->can.bittiming_const = &dev->bt_const; | 1384 | dev->can.bittiming_const = &dev->bt_const; |
| 1373 | dev->can.do_set_bittiming = gs_usb_set_bittiming; | ||
| 1374 | 1385 | ||
| 1375 | dev->can.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC; | 1386 | dev->can.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC; |
| 1376 | 1387 | ||
| @@ -1394,7 +1405,6 @@ static struct gs_can *gs_make_candev(unsigned int channel, · 17 unmodified lines | |||
| 1394 | * GS_CAN_FEATURE_BT_CONST_EXT is set. | 1405 | * GS_CAN_FEATURE_BT_CONST_EXT is set. |
| 1395 | */ | 1406 | */ |
| 1396 | dev->can.fd.data_bittiming_const = &dev->bt_const; | 1407 | dev->can.fd.data_bittiming_const = &dev->bt_const; |
| 1397 | dev->can.fd.do_set_data_bittiming = gs_usb_set_data_bittiming; | ||
| 1398 | } | 1408 | } |
| 1399 | 1409 | ||
| 1400 | if (feature & GS_CAN_FEATURE_TERMINATION) { | 1410 | if (feature & GS_CAN_FEATURE_TERMINATION) { |
| @@ -748,7 +748,7 @@ static void ucan_read_bulk_callback(struct urb *urb) · 747 unmodified lines | |||
| 748 | len = le16_to_cpu(m->len); | 748 | len = le16_to_cpu(m->len); |
| 749 | 749 | ||
| 750 | /* check sanity (length of content) */ | 750 | /* check sanity (length of content) */ |
| 751 | if (urb->actual_length - pos < len) { | 751 | if ((len == 0) || (urb->actual_length - pos < len)) { |
| 752 | netdev_warn(up->netdev, | 752 | netdev_warn(up->netdev, |
| 753 | "invalid message (short; no data; l:%d)\n", | 753 | "invalid message (short; no data; l:%d)\n", |
| 754 | urb->actual_length); | 754 | urb->actual_length); |
| @@ -1108,6 +1108,7 @@ static int ksz_ptp_msg_irq_setup(struct ksz_port *port, u8 n) · 1107 unmodified lines | |||
| 1108 | const struct ksz_dev_ops *ops = port->ksz_dev->dev_ops; | 1108 | const struct ksz_dev_ops *ops = port->ksz_dev->dev_ops; |
| 1109 | struct ksz_irq *ptpirq = &port->ptpirq; | 1109 | struct ksz_irq *ptpirq = &port->ptpirq; |
| 1110 | struct ksz_ptp_irq *ptpmsg_irq; | 1110 | struct ksz_ptp_irq *ptpmsg_irq; |
| 1111 | int ret; | ||
| 1111 | 1112 | ||
| 1112 | ptpmsg_irq = &port->ptpmsg_irq[n]; | 1113 | ptpmsg_irq = &port->ptpmsg_irq[n]; |
| 1113 | ptpmsg_irq->num = irq_create_mapping(ptpirq->domain, n); | 1114 | ptpmsg_irq->num = irq_create_mapping(ptpirq->domain, n); |
| @@ -1119,9 +1120,13 @@ static int ksz_ptp_msg_irq_setup(struct ksz_port *port, u8 n) · 5 unmodified lines | |||
| 1119 | 1120 | ||
| 1120 | strscpy(ptpmsg_irq->name, name[n]); | 1121 | strscpy(ptpmsg_irq->name, name[n]); |
| 1121 | 1122 | ||
| 1122 | return request_threaded_irq(ptpmsg_irq->num, NULL, | 1123 | ret = request_threaded_irq(ptpmsg_irq->num, NULL, |
| 1123 | ksz_ptp_msg_thread_fn, IRQF_ONESHOT, | 1124 | ksz_ptp_msg_thread_fn, IRQF_ONESHOT, |
| 1124 | ptpmsg_irq->name, ptpmsg_irq); | 1125 | ptpmsg_irq->name, ptpmsg_irq); |
| 1126 | if (ret) | ||
| 1127 | irq_dispose_mapping(ptpmsg_irq->num); | ||
| 1128 | |||
| 1129 | return ret; | ||
| 1125 | } | 1130 | } |
| 1126 | 1131 | ||
| 1127 | int ksz_ptp_irq_setup(struct dsa_switch *ds, u8 p) | 1132 | int ksz_ptp_irq_setup(struct dsa_switch *ds, u8 p) |
| @@ -149,7 +149,6 @@ static int mxl862xx_setup_mdio(struct dsa_switch *ds) · 148 unmodified lines | |||
| 149 | return -ENOMEM; | 149 | return -ENOMEM; |
| 150 | 150 | ||
| 151 | bus->priv = priv; | 151 | bus->priv = priv; |
| 152 | ds->user_mii_bus = bus; | ||
| 153 | bus->name = KBUILD_MODNAME "-mii"; | 152 | bus->name = KBUILD_MODNAME "-mii"; |
| 154 | snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(dev)); | 153 | snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(dev)); |
| 155 | bus->read_c45 = mxl862xx_phy_read_c45_mii_bus; | 154 | bus->read_c45 = mxl862xx_phy_read_c45_mii_bus; |
| @@ -769,7 +769,7 @@ static int rtl8365mb_phy_ocp_write(struct realtek_priv *priv, int phy, · 768 unmodified lines | |||
| 769 | out: | 769 | out: |
| 770 | rtl83xx_unlock(priv); | 770 | rtl83xx_unlock(priv); |
| 771 | 771 | ||
| 772 | return 0; | 772 | return ret; |
| 773 | } | 773 | } |
| 774 | 774 | ||
| 775 | static int rtl8365mb_phy_read(struct realtek_priv *priv, int phy, int regnum) | 775 | static int rtl8365mb_phy_read(struct realtek_priv *priv, int phy, int regnum) |
| @@ -1480,8 +1480,7 @@ static void rtl8365mb_stats_update(struct realtek_priv *priv, int port) · 704 unmodified lines | |||
| 1480 | 1480 | ||
| 1481 | stats->rx_packets = cnt[RTL8365MB_MIB_ifInUcastPkts] + | 1481 | stats->rx_packets = cnt[RTL8365MB_MIB_ifInUcastPkts] + |
| 1482 | cnt[RTL8365MB_MIB_ifInMulticastPkts] + | 1482 | cnt[RTL8365MB_MIB_ifInMulticastPkts] + |
| 1483 | cnt[RTL8365MB_MIB_ifInBroadcastPkts] - | 1483 | cnt[RTL8365MB_MIB_ifInBroadcastPkts]; |
| 1484 | cnt[RTL8365MB_MIB_ifOutDiscards]; | ||
| 1485 | 1484 | ||
| 1486 | stats->tx_packets = cnt[RTL8365MB_MIB_ifOutUcastPkts] + | 1485 | stats->tx_packets = cnt[RTL8365MB_MIB_ifOutUcastPkts] + |
| 1487 | cnt[RTL8365MB_MIB_ifOutMulticastPkts] + | 1486 | cnt[RTL8365MB_MIB_ifOutMulticastPkts] + |
| @@ -12,11 +12,11 @@ static inline u32 rtl8366rb_led_group_port_mask(u8 led_group, u8 port) · 11 unmodified lines | |||
| 12 | case 0: | 12 | case 0: |
| 13 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | 13 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); |
| 14 | case 1: | 14 | case 1: |
| 15 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | 15 | return FIELD_PREP(RTL8366RB_LED_X_1_CTRL_MASK, BIT(port)); |
| 16 | case 2: | 16 | case 2: |
| 17 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | 17 | return FIELD_PREP(RTL8366RB_LED_2_X_CTRL_MASK, BIT(port)); |
| 18 | case 3: | 18 | case 3: |
| 19 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | 19 | return FIELD_PREP(RTL8366RB_LED_X_3_CTRL_MASK, BIT(port)); |
| 20 | default: | 20 | default: |
| 21 | return 0; | 21 | return 0; |
| 22 | } | 22 | } |
| @@ -2278,6 +2278,12 @@ int sja1105_static_config_reload(struct sja1105_private *priv, · 2277 unmodified lines | |||
| 2278 | * change it through the dynamic interface later. | 2278 | * change it through the dynamic interface later. |
| 2279 | */ | 2279 | */ |
| 2280 | dsa_switch_for_each_available_port(dp, ds) { | 2280 | dsa_switch_for_each_available_port(dp, ds) { |
| 2281 | /* May be called during unbind when we unoffload a VLAN-aware | ||
| 2282 | * bridge from port 1 while port 0 was already torn down | ||
| 2283 | */ | ||
| 2284 | if (!dp->pl) | ||
| 2285 | continue; | ||
| 2286 | |||
| 2281 | phylink_replay_link_begin(dp->pl); | 2287 | phylink_replay_link_begin(dp->pl); |
| 2282 | mac[dp->index].speed = priv->info->port_speed[SJA1105_SPEED_AUTO]; | 2288 | mac[dp->index].speed = priv->info->port_speed[SJA1105_SPEED_AUTO]; |
| 2283 | } | 2289 | } |
| @@ -2333,13 +2339,13 @@ int sja1105_static_config_reload(struct sja1105_private *priv, · 49 unmodified lines | |||
| 2333 | goto out; | 2339 | goto out; |
| 2334 | } | 2340 | } |
| 2335 | 2341 | ||
| 2336 | dsa_switch_for_each_available_port(dp, ds) | ||
| 2337 | phylink_replay_link_end(dp->pl); | ||
| 2338 | |||
| 2339 | rc = sja1105_reload_cbs(priv); | 2342 | rc = sja1105_reload_cbs(priv); |
| 2340 | if (rc < 0) | 2343 | |
| 2341 | goto out; | ||
| 2342 | out: | 2344 | out: |
| 2345 | dsa_switch_for_each_available_port(dp, ds) | ||
| 2346 | if (dp->pl) | ||
| 2347 | phylink_replay_link_end(dp->pl); | ||
| 2348 | |||
| 2343 | mutex_unlock(&priv->mgmt_lock); | 2349 | mutex_unlock(&priv->mgmt_lock); |
| 2344 | mutex_unlock(&priv->fdb_lock); | 2350 | mutex_unlock(&priv->fdb_lock); |
| 2345 | 2351 | ||
| @@ -431,7 +431,7 @@ · 430 unmodified lines | |||
| 431 | #define MAC_SSIR_SSINC_INDEX 16 | 431 | #define MAC_SSIR_SSINC_INDEX 16 |
| 432 | #define MAC_SSIR_SSINC_WIDTH 8 | 432 | #define MAC_SSIR_SSINC_WIDTH 8 |
| 433 | #define MAC_TCR_SS_INDEX 29 | 433 | #define MAC_TCR_SS_INDEX 29 |
| 434 | #define MAC_TCR_SS_WIDTH 2 | 434 | #define MAC_TCR_SS_WIDTH 3 |
| 435 | #define MAC_TCR_TE_INDEX 0 | 435 | #define MAC_TCR_TE_INDEX 0 |
| 436 | #define MAC_TCR_TE_WIDTH 1 | 436 | #define MAC_TCR_TE_WIDTH 1 |
| 437 | #define MAC_TCR_VNE_INDEX 24 | 437 | #define MAC_TCR_VNE_INDEX 24 |
| @@ -1120,7 +1120,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 1119 unmodified lines | |||
| 1120 | { | 1120 | { |
| 1121 | struct xgbe_prv_data *pdata = netdev_priv(netdev); | 1121 | struct xgbe_prv_data *pdata = netdev_priv(netdev); |
| 1122 | struct xgbe_hw_if *hw_if = &pdata->hw_if; | 1122 | struct xgbe_hw_if *hw_if = &pdata->hw_if; |
| 1123 | unsigned long flags; | ||
| 1124 | 1123 | ||
| 1125 | DBGPR("-->xgbe_powerdown\n"); | 1124 | DBGPR("-->xgbe_powerdown\n"); |
| 1126 | 1125 | ||
| @@ -1131,8 +1130,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 4 unmodified lines | |||
| 1131 | return -EINVAL; | 1130 | return -EINVAL; |
| 1132 | } | 1131 | } |
| 1133 | 1132 | ||
| 1134 | spin_lock_irqsave(&pdata->lock, flags); | ||
| 1135 | |||
| 1136 | if (caller == XGMAC_DRIVER_CONTEXT) | 1133 | if (caller == XGMAC_DRIVER_CONTEXT) |
| 1137 | netif_device_detach(netdev); | 1134 | netif_device_detach(netdev); |
| 1138 | 1135 | ||
| @@ -1148,8 +1145,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 9 unmodified lines | |||
| 1148 | 1145 | ||
| 1149 | pdata->power_down = 1; | 1146 | pdata->power_down = 1; |
| 1150 | 1147 | ||
| 1151 | spin_unlock_irqrestore(&pdata->lock, flags); | ||
| 1152 | |||
| 1153 | DBGPR("<--xgbe_powerdown\n"); | 1148 | DBGPR("<--xgbe_powerdown\n"); |
| 1154 | 1149 | ||
| 1155 | return 0; | 1150 | return 0; |
| @@ -1159,7 +1154,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 3 unmodified lines | |||
| 1159 | { | 1154 | { |
| 1160 | struct xgbe_prv_data *pdata = netdev_priv(netdev); | 1155 | struct xgbe_prv_data *pdata = netdev_priv(netdev); |
| 1161 | struct xgbe_hw_if *hw_if = &pdata->hw_if; | 1156 | struct xgbe_hw_if *hw_if = &pdata->hw_if; |
| 1162 | unsigned long flags; | ||
| 1163 | 1157 | ||
| 1164 | DBGPR("-->xgbe_powerup\n"); | 1158 | DBGPR("-->xgbe_powerup\n"); |
| 1165 | 1159 | ||
| @@ -1170,8 +1164,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 4 unmodified lines | |||
| 1170 | return -EINVAL; | 1164 | return -EINVAL; |
| 1171 | } | 1165 | } |
| 1172 | 1166 | ||
| 1173 | spin_lock_irqsave(&pdata->lock, flags); | ||
| 1174 | |||
| 1175 | pdata->power_down = 0; | 1167 | pdata->power_down = 0; |
| 1176 | 1168 | ||
| 1177 | xgbe_napi_enable(pdata, 0); | 1169 | xgbe_napi_enable(pdata, 0); |
| @@ -1186,8 +1178,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 8 unmodified lines | |||
| 1186 | 1178 | ||
| 1187 | xgbe_start_timers(pdata); | 1179 | xgbe_start_timers(pdata); |
| 1188 | 1180 | ||
| 1189 | spin_unlock_irqrestore(&pdata->lock, flags); | ||
| 1190 | |||
| 1191 | DBGPR("<--xgbe_powerup\n"); | 1181 | DBGPR("<--xgbe_powerup\n"); |
| 1192 | 1182 | ||
| 1193 | return 0; | 1183 | return 0; |
| @@ -1281,20 +1271,25 @@ static int xgbe_start(struct xgbe_prv_data *pdata) · 87 unmodified lines | |||
| 1281 | if (ret) | 1271 | if (ret) |
| 1282 | goto err_napi; | 1272 | goto err_napi; |
| 1283 | 1273 | ||
| 1274 | /* Reset the phy settings */ | ||
| 1275 | ret = xgbe_phy_reset(pdata); | ||
| 1276 | if (ret) | ||
| 1277 | goto err_irqs; | ||
| 1278 | |||
| 1279 | /* Start the phy */ | ||
| 1284 | ret = phy_if->phy_start(pdata); | 1280 | ret = phy_if->phy_start(pdata); |
| 1285 | if (ret) | 1281 | if (ret) |
| 1286 | goto err_irqs; | 1282 | goto err_irqs; |
| 1287 | 1283 | ||
| 1288 | hw_if->enable_tx(pdata); | 1284 | hw_if->enable_tx(pdata); |
| 1289 | hw_if->enable_rx(pdata); | 1285 | hw_if->enable_rx(pdata); |
| 1286 | /* Synchronize flag with hardware state after enabling TX/RX. | ||
| 1287 | * This prevents stale state after device restart cycles. | ||
| 1288 | */ | ||
| 1289 | pdata->data_path_stopped = false; | ||
| 1290 | 1290 | ||
| 1291 | udp_tunnel_nic_reset_ntf(netdev); | 1291 | udp_tunnel_nic_reset_ntf(netdev); |
| 1292 | 1292 | ||
| 1293 | /* Reset the phy settings */ | ||
| 1294 | ret = xgbe_phy_reset(pdata); | ||
| 1295 | if (ret) | ||
| 1296 | goto err_txrx; | ||
| 1297 | |||
| 1298 | netif_tx_start_all_queues(netdev); | 1293 | netif_tx_start_all_queues(netdev); |
| 1299 | 1294 | ||
| 1300 | xgbe_start_timers(pdata); | 1295 | xgbe_start_timers(pdata); |
| @@ -1304,10 +1299,6 @@ static int xgbe_start(struct xgbe_prv_data *pdata) · 3 unmodified lines | |||
| 1304 | 1299 | ||
| 1305 | return 0; | 1300 | return 0; |
| 1306 | 1301 | ||
| 1307 | err_txrx: | ||
| 1308 | hw_if->disable_rx(pdata); | ||
| 1309 | hw_if->disable_tx(pdata); | ||
| 1310 | |||
| 1311 | err_irqs: | 1302 | err_irqs: |
| 1312 | xgbe_free_irqs(pdata); | 1303 | xgbe_free_irqs(pdata); |
| 1313 | 1304 | ||
| @@ -76,7 +76,6 @@ struct xgbe_prv_data *xgbe_alloc_pdata(struct device *dev) · 75 unmodified lines | |||
| 76 | pdata->netdev = netdev; | 76 | pdata->netdev = netdev; |
| 77 | pdata->dev = dev; | 77 | pdata->dev = dev; |
| 78 | 78 | ||
| 79 | spin_lock_init(&pdata->lock); | ||
| 80 | spin_lock_init(&pdata->xpcs_lock); | 79 | spin_lock_init(&pdata->xpcs_lock); |
| 81 | mutex_init(&pdata->rss_mutex); | 80 | mutex_init(&pdata->rss_mutex); |
| 82 | spin_lock_init(&pdata->tstamp_lock); | 81 | spin_lock_init(&pdata->tstamp_lock); |
| @@ -1942,7 +1942,7 @@ static void xgbe_set_rx_adap_mode(struct xgbe_prv_data *pdata, · 1941 unmodified lines | |||
| 1942 | static void xgbe_rx_adaptation(struct xgbe_prv_data *pdata) | 1942 | static void xgbe_rx_adaptation(struct xgbe_prv_data *pdata) |
| 1943 | { | 1943 | { |
| 1944 | struct xgbe_phy_data *phy_data = pdata->phy_data; | 1944 | struct xgbe_phy_data *phy_data = pdata->phy_data; |
| 1945 | unsigned int reg; | 1945 | int reg; |
| 1946 | 1946 | ||
| 1947 | /* step 2: force PCS to send RX_ADAPT Req to PHY */ | 1947 | /* step 2: force PCS to send RX_ADAPT Req to PHY */ |
| 1948 | XMDIO_WRITE_BITS(pdata, MDIO_MMD_PMAPMD, MDIO_PMA_RX_EQ_CTRL4, | 1948 | XMDIO_WRITE_BITS(pdata, MDIO_MMD_PMAPMD, MDIO_PMA_RX_EQ_CTRL4, |
| @@ -1964,11 +1964,20 @@ static void xgbe_rx_adaptation(struct xgbe_prv_data *pdata) · 15 unmodified lines | |||
| 1964 | 1964 | ||
| 1965 | /* Step 4: Check for Block lock */ | 1965 | /* Step 4: Check for Block lock */ |
| 1966 | 1966 | ||
| 1967 | /* Link status is latched low, so read once to clear | ||
| 1968 | * and then read again to get current state | ||
| 1969 | */ | ||
| 1970 | reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); | ||
| 1971 | reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); | 1967 | reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); |
| 1968 | if (reg < 0) | ||
| 1969 | goto set_mode; | ||
| 1970 | |||
| 1971 | /* Link status is latched low so that momentary link drops | ||
| 1972 | * can be detected. If link was already down read again | ||
| 1973 | * to get the latest state. | ||
| 1974 | */ | ||
| 1975 | if (!pdata->phy.link && !(reg & MDIO_STAT1_LSTATUS)) { | ||
| 1976 | reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); | ||
| 1977 | if (reg < 0) | ||
| 1978 | goto set_mode; | ||
| 1979 | } | ||
| 1980 | |||
| 1972 | if (reg & MDIO_STAT1_LSTATUS) { | 1981 | if (reg & MDIO_STAT1_LSTATUS) { |
| 1973 | /* If the block lock is found, update the helpers | 1982 | /* If the block lock is found, update the helpers |
| 1974 | * and declare the link up | 1983 | * and declare the link up |
| @@ -2008,6 +2017,48 @@ rx_adapt_reinit: · 33 unmodified lines | |||
| 2008 | xgbe_rx_adaptation(pdata); | 2017 | xgbe_rx_adaptation(pdata); |
| 2009 | } | 2018 | } |
| 2010 | 2019 | ||
| 2020 | /* | ||
| 2021 | * xgbe_phy_stop_data_path - Stop TX/RX to prevent packet corruption | ||
| 2022 | * @pdata: driver private data | ||
| 2023 | * | ||
| 2024 | * This function stops the data path (TX and RX) to prevent packet | ||
| 2025 | * corruption during critical PHY operations like RX adaptation. | ||
| 2026 | * Must be called before initiating RX adaptation when link goes down. | ||
| 2027 | */ | ||
| 2028 | static void xgbe_phy_stop_data_path(struct xgbe_prv_data *pdata) | ||
| 2029 | { | ||
| 2030 | if (pdata->data_path_stopped) | ||
| 2031 | return; | ||
| 2032 | |||
| 2033 | /* Stop TX/RX to prevent packet corruption during RX adaptation */ | ||
| 2034 | pdata->hw_if.disable_tx(pdata); | ||
| 2035 | pdata->hw_if.disable_rx(pdata); | ||
| 2036 | pdata->data_path_stopped = true; | ||
| 2037 | |||
| 2038 | netif_dbg(pdata, link, pdata->netdev, | ||
| 2039 | "stopping data path for RX adaptation\n"); | ||
| 2040 | } | ||
| 2041 | |||
| 2042 | /* | ||
| 2043 | * xgbe_phy_start_data_path - Re-enable TX/RX after RX adaptation | ||
| 2044 | * @pdata: driver private data | ||
| 2045 | * | ||
| 2046 | * This function re-enables the data path (TX and RX) after RX adaptation | ||
| 2047 | * has completed successfully. Only called when link is confirmed up. | ||
| 2048 | */ | ||
| 2049 | static void xgbe_phy_start_data_path(struct xgbe_prv_data *pdata) | ||
| 2050 | { | ||
| 2051 | if (!pdata->data_path_stopped) | ||
| 2052 | return; | ||
| 2053 | |||
| 2054 | pdata->hw_if.enable_rx(pdata); | ||
| 2055 | pdata->hw_if.enable_tx(pdata); | ||
| 2056 | pdata->data_path_stopped = false; | ||
| 2057 | |||
| 2058 | netif_dbg(pdata, link, pdata->netdev, | ||
| 2059 | "restarting data path after RX adaptation\n"); | ||
| 2060 | } | ||
| 2061 | |||
| 2011 | static void xgbe_phy_rx_reset(struct xgbe_prv_data *pdata) | 2062 | static void xgbe_phy_rx_reset(struct xgbe_prv_data *pdata) |
| 2012 | { | 2063 | { |
| 2013 | int reg; | 2064 | int reg; |
| @@ -2801,13 +2852,27 @@ static int xgbe_phy_link_status(struct xgbe_prv_data *pdata, int *an_restart) · 787 unmodified lines | |||
| 2801 | if (pdata->en_rx_adap) { | 2852 | if (pdata->en_rx_adap) { |
| 2802 | /* if the link is available and adaptation is done, | 2853 | /* if the link is available and adaptation is done, |
| 2803 | * declare link up | 2854 | * declare link up |
| 2855 | * | ||
| 2856 | * Note: When link is up and adaptation is done, we can | ||
| 2857 | * safely re-enable the data path if it was stopped | ||
| 2858 | * for adaptation. | ||
| 2804 | */ | 2859 | */ |
| 2805 | if ((reg & MDIO_STAT1_LSTATUS) && pdata->rx_adapt_done) | 2860 | if ((reg & MDIO_STAT1_LSTATUS) && pdata->rx_adapt_done) { |
| 2861 | xgbe_phy_start_data_path(pdata); | ||
| 2806 | return 1; | 2862 | return 1; |
| 2863 | } | ||
| 2807 | /* If either link is not available or adaptation is not done, | 2864 | /* If either link is not available or adaptation is not done, |
| 2808 | * retrigger the adaptation logic. (if the mode is not set, | 2865 | * retrigger the adaptation logic. (if the mode is not set, |
| 2809 | * then issue mailbox command first) | 2866 | * then issue mailbox command first) |
| 2810 | */ | 2867 | */ |
| 2868 | |||
| 2869 | /* CRITICAL: Stop data path BEFORE triggering RX adaptation | ||
| 2870 | * to prevent CRC errors from packets corrupted during | ||
| 2871 | * the adaptation process. This is especially important | ||
| 2872 | * when AN is OFF in 10G KR mode. | ||
| 2873 | */ | ||
| 2874 | xgbe_phy_stop_data_path(pdata); | ||
| 2875 | |||
| 2811 | if (pdata->mode_set) { | 2876 | if (pdata->mode_set) { |
| 2812 | xgbe_phy_rx_adaptation(pdata); | 2877 | xgbe_phy_rx_adaptation(pdata); |
| 2813 | } else { | 2878 | } else { |
| @@ -2815,8 +2880,11 @@ static int xgbe_phy_link_status(struct xgbe_prv_data *pdata, int *an_restart) · 1 unmodified line | |||
| 2815 | xgbe_phy_set_mode(pdata, phy_data->cur_mode); | 2880 | xgbe_phy_set_mode(pdata, phy_data->cur_mode); |
| 2816 | } | 2881 | } |
| 2817 | 2882 | ||
| 2818 | if (pdata->rx_adapt_done) | 2883 | if (pdata->rx_adapt_done) { |
| 2884 | /* Adaptation complete, safe to re-enable data path */ | ||
| 2885 | xgbe_phy_start_data_path(pdata); | ||
| 2819 | return 1; | 2886 | return 1; |
| 2887 | } | ||
| 2820 | } else if (reg & MDIO_STAT1_LSTATUS) | 2888 | } else if (reg & MDIO_STAT1_LSTATUS) |
| 2821 | return 1; | 2889 | return 1; |
| 2822 | 2890 | ||
| @@ -1004,9 +1004,6 @@ struct xgbe_prv_data { · 1003 unmodified lines | |||
| 1004 | unsigned int pp3; | 1004 | unsigned int pp3; |
| 1005 | unsigned int pp4; | 1005 | unsigned int pp4; |
| 1006 | 1006 | ||
| 1007 | /* Overall device lock */ | ||
| 1008 | spinlock_t lock; | ||
| 1009 | |||
| 1010 | /* XPCS indirect addressing lock */ | 1007 | /* XPCS indirect addressing lock */ |
| 1011 | spinlock_t xpcs_lock; | 1008 | spinlock_t xpcs_lock; |
| 1012 | unsigned int xpcs_window_def_reg; | 1009 | unsigned int xpcs_window_def_reg; |
| @@ -1246,6 +1243,10 @@ struct xgbe_prv_data { · 233 unmodified lines | |||
| 1246 | bool en_rx_adap; | 1243 | bool en_rx_adap; |
| 1247 | int rx_adapt_retries; | 1244 | int rx_adapt_retries; |
| 1248 | bool rx_adapt_done; | 1245 | bool rx_adapt_done; |
| 1246 | /* Flag to track if data path (TX/RX) was stopped for RX adaptation. | ||
| 1247 | * This prevents packet corruption during the adaptation window. | ||
| 1248 | */ | ||
| 1249 | bool data_path_stopped; | ||
| 1249 | bool mode_set; | 1250 | bool mode_set; |
| 1250 | bool sph; | 1251 | bool sph; |
| 1251 | }; | 1252 | }; |
| @@ -934,6 +934,17 @@ int arc_emac_probe(struct net_device *ndev, int interface) · 933 unmodified lines | |||
| 934 | /* Set poll rate so that it polls every 1 ms */ | 934 | /* Set poll rate so that it polls every 1 ms */ |
| 935 | arc_reg_set(priv, R_POLLRATE, clock_frequency / 1000000); | 935 | arc_reg_set(priv, R_POLLRATE, clock_frequency / 1000000); |
| 936 | 936 | ||
| 937 | /* | ||
| 938 | * Put the device into a known quiescent state before requesting | ||
| 939 | * the IRQ. Clear only EMAC interrupt status bits here; leave the | ||
| 940 | * MDIO completion bit alone and avoid writing TXPL_MASK, which is | ||
| 941 | * used to force TX polling rather than acknowledge interrupts. | ||
| 942 | */ | ||
| 943 | arc_reg_set(priv, R_ENABLE, 0); | ||
| 944 | arc_reg_set(priv, R_STATUS, RXINT_MASK | TXINT_MASK | ERR_MASK | | ||
| 945 | TXCH_MASK | MSER_MASK | RXCR_MASK | | ||
| 946 | RXFR_MASK | RXFL_MASK); | ||
| 947 | |||
| 937 | ndev->irq = irq; | 948 | ndev->irq = irq; |
| 938 | dev_info(dev, "IRQ is %d\n", ndev->irq); | 949 | dev_info(dev, "IRQ is %d\n", ndev->irq); |
| 939 | 950 | ||
| @@ -6232,6 +6232,9 @@ int bnxt_hwrm_cfa_ntuple_filter_free(struct bnxt *bp, · 6231 unmodified lines | |||
| 6232 | int rc; | 6232 | int rc; |
| 6233 | 6233 | ||
| 6234 | set_bit(BNXT_FLTR_FW_DELETED, &fltr->base.state); | 6234 | set_bit(BNXT_FLTR_FW_DELETED, &fltr->base.state); |
| 6235 | if (!test_bit(BNXT_STATE_OPEN, &bp->state)) | ||
| 6236 | return 0; | ||
| 6237 | |||
| 6235 | rc = hwrm_req_init(bp, req, HWRM_CFA_NTUPLE_FILTER_FREE); | 6238 | rc = hwrm_req_init(bp, req, HWRM_CFA_NTUPLE_FILTER_FREE); |
| 6236 | if (rc) | 6239 | if (rc) |
| 6237 | return rc; | 6240 | return rc; |
| @@ -10879,12 +10882,10 @@ void bnxt_del_one_rss_ctx(struct bnxt *bp, struct bnxt_rss_ctx *rss_ctx, · 4641 unmodified lines | |||
| 10879 | struct bnxt_ntuple_filter *ntp_fltr; | 10882 | struct bnxt_ntuple_filter *ntp_fltr; |
| 10880 | int i; | 10883 | int i; |
| 10881 | 10884 | ||
| 10882 | if (netif_running(bp->dev)) { | 10885 | bnxt_hwrm_vnic_free_one(bp, &rss_ctx->vnic); |
| 10883 | bnxt_hwrm_vnic_free_one(bp, &rss_ctx->vnic); | 10886 | for (i = 0; i < BNXT_MAX_CTX_PER_VNIC; i++) { |
| 10884 | for (i = 0; i < BNXT_MAX_CTX_PER_VNIC; i++) { | 10887 | if (vnic->fw_rss_cos_lb_ctx[i] != INVALID_HW_RING_ID) |
| 10885 | if (vnic->fw_rss_cos_lb_ctx[i] != INVALID_HW_RING_ID) | 10888 | bnxt_hwrm_vnic_ctx_free_one(bp, vnic, i); |
| 10886 | bnxt_hwrm_vnic_ctx_free_one(bp, vnic, i); | ||
| 10887 | } | ||
| 10888 | } | 10889 | } |
| 10889 | if (!all) | 10890 | if (!all) |
| 10890 | return; | 10891 | return; |
| @@ -979,8 +979,8 @@ static int bnxt_set_channels(struct net_device *dev, · 978 unmodified lines | |||
| 979 | 979 | ||
| 980 | if (bnxt_get_nr_rss_ctxs(bp, req_rx_rings) != | 980 | if (bnxt_get_nr_rss_ctxs(bp, req_rx_rings) != |
| 981 | bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings) && | 981 | bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings) && |
| 982 | netif_is_rxfh_configured(dev)) { | 982 | (netif_is_rxfh_configured(dev) || bp->num_rss_ctx)) { |
| 983 | netdev_warn(dev, "RSS table size change required, RSS table entries must be default to proceed\n"); | 983 | netdev_warn(dev, "RSS table size change required, RSS table entries must be default (with no additional RSS contexts present) to proceed\n"); |
| 984 | return -EINVAL; | 984 | return -EINVAL; |
| 985 | } | 985 | } |
| 986 | 986 | ||
| @@ -1342,8 +1342,7 @@ static void bcmgenet_get_ethtool_stats(struct net_device *dev, · 1341 unmodified lines | |||
| 1342 | } | 1342 | } |
| 1343 | } | 1343 | } |
| 1344 | 1344 | ||
| 1345 | void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, | 1345 | void bcmgenet_eee_enable_set(struct net_device *dev, bool enable) |
| 1346 | bool tx_lpi_enabled) | ||
| 1347 | { | 1346 | { |
| 1348 | struct bcmgenet_priv *priv = netdev_priv(dev); | 1347 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1349 | u32 off = priv->hw_params->tbuf_offset + TBUF_ENERGY_CTRL; | 1348 | u32 off = priv->hw_params->tbuf_offset + TBUF_ENERGY_CTRL; |
| @@ -1363,7 +1362,7 @@ void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, · 13 unmodified lines | |||
| 1363 | 1362 | ||
| 1364 | /* Enable EEE and switch to a 27Mhz clock automatically */ | 1363 | /* Enable EEE and switch to a 27Mhz clock automatically */ |
| 1365 | reg = bcmgenet_readl(priv->base + off); | 1364 | reg = bcmgenet_readl(priv->base + off); |
| 1366 | if (tx_lpi_enabled) | 1365 | if (enable) |
| 1367 | reg |= TBUF_EEE_EN | TBUF_PM_EN; | 1366 | reg |= TBUF_EEE_EN | TBUF_PM_EN; |
| 1368 | else | 1367 | else |
| 1369 | reg &= ~(TBUF_EEE_EN | TBUF_PM_EN); | 1368 | reg &= ~(TBUF_EEE_EN | TBUF_PM_EN); |
| @@ -1382,14 +1381,12 @@ void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, · 12 unmodified lines | |||
| 1382 | priv->clk_eee_enabled = false; | 1381 | priv->clk_eee_enabled = false; |
| 1383 | } | 1382 | } |
| 1384 | 1383 | ||
| 1385 | priv->eee.eee_enabled = enable; | ||
| 1386 | priv->eee.tx_lpi_enabled = tx_lpi_enabled; | ||
| 1387 | } | 1384 | } |
| 1388 | 1385 | ||
| 1389 | static int bcmgenet_get_eee(struct net_device *dev, struct ethtool_keee *e) | 1386 | static int bcmgenet_get_eee(struct net_device *dev, struct ethtool_keee *e) |
| 1390 | { | 1387 | { |
| 1391 | struct bcmgenet_priv *priv = netdev_priv(dev); | 1388 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1392 | struct ethtool_keee *p = &priv->eee; | 1389 | int ret; |
| 1393 | 1390 | ||
| 1394 | if (GENET_IS_V1(priv)) | 1391 | if (GENET_IS_V1(priv)) |
| 1395 | return -EOPNOTSUPP; | 1392 | return -EOPNOTSUPP; |
| @@ -1397,17 +1394,21 @@ static int bcmgenet_get_eee(struct net_device *dev, struct ethtool_keee *e) · 1 unmodified line | |||
| 1397 | if (!dev->phydev) | 1394 | if (!dev->phydev) |
| 1398 | return -ENODEV; | 1395 | return -ENODEV; |
| 1399 | 1396 | ||
| 1400 | e->tx_lpi_enabled = p->tx_lpi_enabled; | 1397 | ret = phy_ethtool_get_eee(dev->phydev, e); |
| 1398 | if (ret) | ||
| 1399 | return ret; | ||
| 1400 | |||
| 1401 | /* tx_lpi_timer is maintained by the MAC hardware register; the | ||
| 1402 | * PHY-level eee_cfg timer is not set for GENET. | ||
| 1403 | */ | ||
| 1401 | e->tx_lpi_timer = bcmgenet_umac_readl(priv, UMAC_EEE_LPI_TIMER); | 1404 | e->tx_lpi_timer = bcmgenet_umac_readl(priv, UMAC_EEE_LPI_TIMER); |
| 1402 | 1405 | ||
| 1403 | return phy_ethtool_get_eee(dev->phydev, e); | 1406 | return 0; |
| 1404 | } | 1407 | } |
| 1405 | 1408 | ||
| 1406 | static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_keee *e) | 1409 | static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_keee *e) |
| 1407 | { | 1410 | { |
| 1408 | struct bcmgenet_priv *priv = netdev_priv(dev); | 1411 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1409 | struct ethtool_keee *p = &priv->eee; | ||
| 1410 | bool active; | ||
| 1411 | 1412 | ||
| 1412 | if (GENET_IS_V1(priv)) | 1413 | if (GENET_IS_V1(priv)) |
| 1413 | return -EOPNOTSUPP; | 1414 | return -EOPNOTSUPP; |
| @@ -1415,15 +1416,7 @@ static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_keee *e) · 1 unmodified line | |||
| 1415 | if (!dev->phydev) | 1416 | if (!dev->phydev) |
| 1416 | return -ENODEV; | 1417 | return -ENODEV; |
| 1417 | 1418 | ||
| 1418 | p->eee_enabled = e->eee_enabled; | 1419 | bcmgenet_umac_writel(priv, e->tx_lpi_timer, UMAC_EEE_LPI_TIMER); |
| 1419 | |||
| 1420 | if (!p->eee_enabled) { | ||
| 1421 | bcmgenet_eee_enable_set(dev, false, false); | ||
| 1422 | } else { | ||
| 1423 | active = phy_init_eee(dev->phydev, false) >= 0; | ||
| 1424 | bcmgenet_umac_writel(priv, e->tx_lpi_timer, UMAC_EEE_LPI_TIMER); | ||
| 1425 | bcmgenet_eee_enable_set(dev, active, e->tx_lpi_enabled); | ||
| 1426 | } | ||
| 1427 | 1420 | ||
| 1428 | return phy_ethtool_set_eee(dev->phydev, e); | 1421 | return phy_ethtool_set_eee(dev->phydev, e); |
| 1429 | } | 1422 | } |
| @@ -665,8 +665,6 @@ struct bcmgenet_priv { · 664 unmodified lines | |||
| 665 | u8 sopass[SOPASS_MAX]; | 665 | u8 sopass[SOPASS_MAX]; |
| 666 | 666 | ||
| 667 | struct bcmgenet_mib_counters mib; | 667 | struct bcmgenet_mib_counters mib; |
| 668 | |||
| 669 | struct ethtool_keee eee; | ||
| 670 | }; | 668 | }; |
| 671 | 669 | ||
| 672 | static inline bool bcmgenet_has_40bits(struct bcmgenet_priv *priv) | 670 | static inline bool bcmgenet_has_40bits(struct bcmgenet_priv *priv) |
| @@ -749,7 +747,6 @@ int bcmgenet_wol_power_down_cfg(struct bcmgenet_priv *priv, · 76 unmodified lines | |||
| 749 | int bcmgenet_wol_power_up_cfg(struct bcmgenet_priv *priv, | 747 | int bcmgenet_wol_power_up_cfg(struct bcmgenet_priv *priv, |
| 750 | enum bcmgenet_power_mode mode); | 748 | enum bcmgenet_power_mode mode); |
| 751 | 749 | ||
| 752 | void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, | 750 | void bcmgenet_eee_enable_set(struct net_device *dev, bool enable); |
| 753 | bool tx_lpi_enabled); | ||
| 754 | 751 | ||
| 755 | #endif /* __BCMGENET_H__ */ | 752 | #endif /* __BCMGENET_H__ */ |
| @@ -29,7 +29,6 @@ static void bcmgenet_mac_config(struct net_device *dev) · 28 unmodified lines | |||
| 29 | struct bcmgenet_priv *priv = netdev_priv(dev); | 29 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 30 | struct phy_device *phydev = dev->phydev; | 30 | struct phy_device *phydev = dev->phydev; |
| 31 | u32 reg, cmd_bits = 0; | 31 | u32 reg, cmd_bits = 0; |
| 32 | bool active; | ||
| 33 | 32 | ||
| 34 | /* speed */ | 33 | /* speed */ |
| 35 | if (phydev->speed == SPEED_1000) | 34 | if (phydev->speed == SPEED_1000) |
| @@ -90,10 +89,6 @@ static void bcmgenet_mac_config(struct net_device *dev) · 54 unmodified lines | |||
| 90 | bcmgenet_umac_writel(priv, reg, UMAC_CMD); | 89 | bcmgenet_umac_writel(priv, reg, UMAC_CMD); |
| 91 | spin_unlock_bh(&priv->reg_lock); | 90 | spin_unlock_bh(&priv->reg_lock); |
| 92 | 91 | ||
| 93 | active = phy_init_eee(phydev, 0) >= 0; | ||
| 94 | bcmgenet_eee_enable_set(dev, | ||
| 95 | priv->eee.eee_enabled && active, | ||
| 96 | priv->eee.tx_lpi_enabled); | ||
| 97 | } | 92 | } |
| 98 | 93 | ||
| 99 | /* setup netdev link state when PHY link status change and | 94 | /* setup netdev link state when PHY link status change and |
| @@ -113,6 +108,8 @@ void bcmgenet_mii_setup(struct net_device *dev) · 13 unmodified lines | |||
| 113 | bcmgenet_ext_writel(priv, reg, EXT_RGMII_OOB_CTRL); | 108 | bcmgenet_ext_writel(priv, reg, EXT_RGMII_OOB_CTRL); |
| 114 | } | 109 | } |
| 115 | 110 | ||
| 111 | bcmgenet_eee_enable_set(dev, phydev->enable_tx_lpi); | ||
| 112 | |||
| 116 | phy_print_status(phydev); | 113 | phy_print_status(phydev); |
| 117 | } | 114 | } |
| 118 | 115 | ||
| @@ -412,6 +409,9 @@ int bcmgenet_mii_probe(struct net_device *dev) · 293 unmodified lines | |||
| 412 | /* Indicate that the MAC is responsible for PHY PM */ | 409 | /* Indicate that the MAC is responsible for PHY PM */ |
| 413 | dev->phydev->mac_managed_pm = true; | 410 | dev->phydev->mac_managed_pm = true; |
| 414 | 411 | ||
| 412 | if (!GENET_IS_V1(priv)) | ||
| 413 | phy_support_eee(dev->phydev); | ||
| 414 | |||
| 415 | return 0; | 415 | return 0; |
| 416 | } | 416 | } |
| 417 | 417 | ||
| @@ -36,6 +36,7 @@ · 35 unmodified lines | |||
| 36 | #include <linux/tcp.h> | 36 | #include <linux/tcp.h> |
| 37 | #include <linux/types.h> | 37 | #include <linux/types.h> |
| 38 | #include <linux/udp.h> | 38 | #include <linux/udp.h> |
| 39 | #include <linux/gcd.h> | ||
| 39 | #include <net/pkt_sched.h> | 40 | #include <net/pkt_sched.h> |
| 40 | #include "macb.h" | 41 | #include "macb.h" |
| 41 | 42 | ||
| @@ -668,6 +669,97 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode, · 626 unmodified lines | |||
| 668 | netif_tx_stop_all_queues(ndev); | 669 | netif_tx_stop_all_queues(ndev); |
| 669 | } | 670 | } |
| 670 | 671 | ||
| 672 | /* Use juggling algorithm to left rotate tx ring and tx skb array */ | ||
| 673 | static void gem_shuffle_tx_one_ring(struct macb_queue *queue) | ||
| 674 | { | ||
| 675 | unsigned int head, tail, count, ring_size, desc_size; | ||
| 676 | struct macb_tx_skb tx_skb, *skb_curr, *skb_next; | ||
| 677 | struct macb_dma_desc *desc_curr, *desc_next; | ||
| 678 | unsigned int i, cycles, shift, curr, next; | ||
| 679 | struct macb *bp = queue->bp; | ||
| 680 | unsigned char desc[24]; | ||
| 681 | unsigned long flags; | ||
| 682 | |||
| 683 | desc_size = macb_dma_desc_get_size(bp); | ||
| 684 | |||
| 685 | if (WARN_ON_ONCE(desc_size > ARRAY_SIZE(desc))) | ||
| 686 | return; | ||
| 687 | |||
| 688 | spin_lock_irqsave(&queue->tx_ptr_lock, flags); | ||
| 689 | head = queue->tx_head; | ||
| 690 | tail = queue->tx_tail; | ||
| 691 | ring_size = bp->tx_ring_size; | ||
| 692 | count = CIRC_CNT(head, tail, ring_size); | ||
| 693 | |||
| 694 | if (!(tail % ring_size)) | ||
| 695 | goto unlock; | ||
| 696 | |||
| 697 | if (!count) { | ||
| 698 | queue->tx_head = 0; | ||
| 699 | queue->tx_tail = 0; | ||
| 700 | goto unlock; | ||
| 701 | } | ||
| 702 | |||
| 703 | shift = tail % ring_size; | ||
| 704 | cycles = gcd(ring_size, shift); | ||
| 705 | |||
| 706 | for (i = 0; i < cycles; i++) { | ||
| 707 | memcpy(&desc, macb_tx_desc(queue, i), desc_size); | ||
| 708 | memcpy(&tx_skb, macb_tx_skb(queue, i), | ||
| 709 | sizeof(struct macb_tx_skb)); | ||
| 710 | |||
| 711 | curr = i; | ||
| 712 | next = (curr + shift) % ring_size; | ||
| 713 | |||
| 714 | while (next != i) { | ||
| 715 | desc_curr = macb_tx_desc(queue, curr); | ||
| 716 | desc_next = macb_tx_desc(queue, next); | ||
| 717 | |||
| 718 | memcpy(desc_curr, desc_next, desc_size); | ||
| 719 | |||
| 720 | if (next == ring_size - 1) | ||
| 721 | desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); | ||
| 722 | if (curr == ring_size - 1) | ||
| 723 | desc_curr->ctrl |= MACB_BIT(TX_WRAP); | ||
| 724 | |||
| 725 | skb_curr = macb_tx_skb(queue, curr); | ||
| 726 | skb_next = macb_tx_skb(queue, next); | ||
| 727 | memcpy(skb_curr, skb_next, sizeof(struct macb_tx_skb)); | ||
| 728 | |||
| 729 | curr = next; | ||
| 730 | next = (curr + shift) % ring_size; | ||
| 731 | } | ||
| 732 | |||
| 733 | desc_curr = macb_tx_desc(queue, curr); | ||
| 734 | memcpy(desc_curr, &desc, desc_size); | ||
| 735 | if (i == ring_size - 1) | ||
| 736 | desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); | ||
| 737 | if (curr == ring_size - 1) | ||
| 738 | desc_curr->ctrl |= MACB_BIT(TX_WRAP); | ||
| 739 | memcpy(macb_tx_skb(queue, curr), &tx_skb, | ||
| 740 | sizeof(struct macb_tx_skb)); | ||
| 741 | } | ||
| 742 | |||
| 743 | queue->tx_head = count; | ||
| 744 | queue->tx_tail = 0; | ||
| 745 | |||
| 746 | /* Make descriptor updates visible to hardware */ | ||
| 747 | wmb(); | ||
| 748 | |||
| 749 | unlock: | ||
| 750 | spin_unlock_irqrestore(&queue->tx_ptr_lock, flags); | ||
| 751 | } | ||
| 752 | |||
| 753 | /* Rotate the queue so that the tail is at index 0 */ | ||
| 754 | static void gem_shuffle_tx_rings(struct macb *bp) | ||
| 755 | { | ||
| 756 | struct macb_queue *queue; | ||
| 757 | int q; | ||
| 758 | |||
| 759 | for (q = 0, queue = bp->queues; q < bp->num_queues; q++, queue++) | ||
| 760 | gem_shuffle_tx_one_ring(queue); | ||
| 761 | } | ||
| 762 | |||
| 671 | static void macb_mac_link_up(struct phylink_config *config, | 763 | static void macb_mac_link_up(struct phylink_config *config, |
| 672 | struct phy_device *phy, | 764 | struct phy_device *phy, |
| 673 | unsigned int mode, phy_interface_t interface, | 765 | unsigned int mode, phy_interface_t interface, |
| @@ -706,8 +798,6 @@ static void macb_mac_link_up(struct phylink_config *config, · 32 unmodified lines | |||
| 706 | ctrl |= MACB_BIT(PAE); | 798 | ctrl |= MACB_BIT(PAE); |
| 707 | 799 | ||
| 708 | for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { | 800 | for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { |
| 709 | queue->tx_head = 0; | ||
| 710 | queue->tx_tail = 0; | ||
| 711 | queue_writel(queue, IER, | 801 | queue_writel(queue, IER, |
| 712 | bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); | 802 | bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); |
| 713 | } | 803 | } |
| @@ -721,8 +811,10 @@ static void macb_mac_link_up(struct phylink_config *config, · 7 unmodified lines | |||
| 721 | 811 | ||
| 722 | spin_unlock_irqrestore(&bp->lock, flags); | 812 | spin_unlock_irqrestore(&bp->lock, flags); |
| 723 | 813 | ||
| 724 | if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) | 814 | if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) { |
| 725 | macb_set_tx_clk(bp, speed); | 815 | macb_set_tx_clk(bp, speed); |
| 816 | gem_shuffle_tx_rings(bp); | ||
| 817 | } | ||
| 726 | 818 | ||
| 727 | /* Enable Rx and Tx; Enable PTP unicast */ | 819 | /* Enable Rx and Tx; Enable PTP unicast */ |
| 728 | ctrl = macb_readl(bp, NCR); | 820 | ctrl = macb_readl(bp, NCR); |
| @@ -1533,7 +1533,7 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) · 1532 unmodified lines | |||
| 1533 | if_id = (status & 0xFFFF0000) >> 16; | 1533 | if_id = (status & 0xFFFF0000) >> 16; |
| 1534 | if (if_id >= ethsw->sw_attr.num_ifs) { | 1534 | if (if_id >= ethsw->sw_attr.num_ifs) { |
| 1535 | dev_err(dev, "Invalid if_id %d in IRQ status\n", if_id); | 1535 | dev_err(dev, "Invalid if_id %d in IRQ status\n", if_id); |
| 1536 | goto out; | 1536 | goto out_clear; |
| 1537 | } | 1537 | } |
| 1538 | port_priv = ethsw->ports[if_id]; | 1538 | port_priv = ethsw->ports[if_id]; |
| 1539 | 1539 | ||
| @@ -1553,6 +1553,7 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) · 13 unmodified lines | |||
| 1553 | dpaa2_switch_port_connect_mac(port_priv); | 1553 | dpaa2_switch_port_connect_mac(port_priv); |
| 1554 | } | 1554 | } |
| 1555 | 1555 | ||
| 1556 | out_clear: | ||
| 1556 | err = dpsw_clear_irq_status(ethsw->mc_io, 0, ethsw->dpsw_handle, | 1557 | err = dpsw_clear_irq_status(ethsw->mc_io, 0, ethsw->dpsw_handle, |
| 1557 | DPSW_IRQ_INDEX_IF, status); | 1558 | DPSW_IRQ_INDEX_IF, status); |
| 1558 | if (err) | 1559 | if (err) |
| @@ -3034,6 +3035,13 @@ static int dpaa2_switch_init(struct fsl_mc_device *sw_dev) · 1475 unmodified lines | |||
| 3034 | goto err_close; | 3035 | goto err_close; |
| 3035 | } | 3036 | } |
| 3036 | 3037 | ||
| 3038 | if (ethsw->sw_attr.num_ifs >= DPSW_MAX_IF) { | ||
| 3039 | dev_err(dev, "DPSW num_ifs %u exceeds max %u\n", | ||
| 3040 | ethsw->sw_attr.num_ifs, DPSW_MAX_IF); | ||
| 3041 | err = -EINVAL; | ||
| 3042 | goto err_close; | ||
| 3043 | } | ||
| 3044 | |||
| 3037 | err = dpsw_get_api_version(ethsw->mc_io, 0, | 3045 | err = dpsw_get_api_version(ethsw->mc_io, 0, |
| 3038 | ðsw->major, | 3046 | ðsw->major, |
| 3039 | ðsw->minor); | 3047 | ðsw->minor); |
| @@ -3467,7 +3467,7 @@ static int enetc_int_vector_init(struct enetc_ndev_priv *priv, int i, · 3466 unmodified lines | |||
| 3467 | priv->rx_ring[i] = bdr; | 3467 | priv->rx_ring[i] = bdr; |
| 3468 | 3468 | ||
| 3469 | err = __xdp_rxq_info_reg(&bdr->xdp.rxq, priv->ndev, i, 0, | 3469 | err = __xdp_rxq_info_reg(&bdr->xdp.rxq, priv->ndev, i, 0, |
| 3470 | ENETC_RXB_DMA_SIZE_XDP); | 3470 | ENETC_RXB_TRUESIZE); |
| 3471 | if (err) | 3471 | if (err) |
| 3472 | goto free_vector; | 3472 | goto free_vector; |
| 3473 | 3473 | ||
| @@ -333,11 +333,13 @@ static int netc_get_phy_addr(struct device_node *np) · 332 unmodified lines | |||
| 333 | 333 | ||
| 334 | mdio_node = of_get_child_by_name(np, "mdio"); | 334 | mdio_node = of_get_child_by_name(np, "mdio"); |
| 335 | if (!mdio_node) | 335 | if (!mdio_node) |
| 336 | return 0; | 336 | return -ENODEV; |
| 337 | 337 | ||
| 338 | phy_node = of_get_next_child(mdio_node, NULL); | 338 | phy_node = of_get_next_child(mdio_node, NULL); |
| 339 | if (!phy_node) | 339 | if (!phy_node) { |
| 340 | err = -ENODEV; | ||
| 340 | goto of_put_mdio_node; | 341 | goto of_put_mdio_node; |
| 342 | } | ||
| 341 | 343 | ||
| 342 | err = of_property_read_u32(phy_node, "reg", &addr); | 344 | err = of_property_read_u32(phy_node, "reg", &addr); |
| 343 | if (err) | 345 | if (err) |
| @@ -423,6 +425,9 @@ static int imx95_enetc_mdio_phyaddr_config(struct platform_device *pdev) · 79 unmodified lines | |||
| 423 | 425 | ||
| 424 | addr = netc_get_phy_addr(gchild); | 426 | addr = netc_get_phy_addr(gchild); |
| 425 | if (addr < 0) { | 427 | if (addr < 0) { |
| 428 | if (addr == -ENODEV) | ||
| 429 | continue; | ||
| 430 | |||
| 426 | dev_err(dev, "Failed to get PHY address\n"); | 431 | dev_err(dev, "Failed to get PHY address\n"); |
| 427 | return addr; | 432 | return addr; |
| 428 | } | 433 | } |
| @@ -433,12 +438,6 @@ static int imx95_enetc_mdio_phyaddr_config(struct platform_device *pdev) · 4 unmodified lines | |||
| 433 | return -EINVAL; | 438 | return -EINVAL; |
| 434 | } | 439 | } |
| 435 | 440 | ||
| 436 | /* The default value of LaBCR[MDIO_PHYAD_PRTAD ] is | ||
| 437 | * 0, so no need to set the register. | ||
| 438 | */ | ||
| 439 | if (!addr) | ||
| 440 | continue; | ||
| 441 | |||
| 442 | switch (bus_devfn) { | 441 | switch (bus_devfn) { |
| 443 | case IMX95_ENETC0_BUS_DEVFN: | 442 | case IMX95_ENETC0_BUS_DEVFN: |
| 444 | netc_reg_write(priv->ierb, IERB_LBCR(0), | 443 | netc_reg_write(priv->ierb, IERB_LBCR(0), |
| @@ -578,16 +577,13 @@ static int imx94_enetc_mdio_phyaddr_config(struct netc_blk_ctrl *priv, · 133 unmodified lines | |||
| 578 | 577 | ||
| 579 | addr = netc_get_phy_addr(np); | 578 | addr = netc_get_phy_addr(np); |
| 580 | if (addr < 0) { | 579 | if (addr < 0) { |
| 580 | if (addr == -ENODEV) | ||
| 581 | return 0; | ||
| 582 | |||
| 581 | dev_err(dev, "Failed to get PHY address\n"); | 583 | dev_err(dev, "Failed to get PHY address\n"); |
| 582 | return addr; | 584 | return addr; |
| 583 | } | 585 | } |
| 584 | 586 | ||
| 585 | /* The default value of LaBCR[MDIO_PHYAD_PRTAD] is 0, | ||
| 586 | * so no need to set the register. | ||
| 587 | */ | ||
| 588 | if (!addr) | ||
| 589 | return 0; | ||
| 590 | |||
| 591 | if (phy_mask & BIT(addr)) { | 587 | if (phy_mask & BIT(addr)) { |
| 592 | dev_err(dev, | 588 | dev_err(dev, |
| 593 | "Find same PHY address in EMDIO and ENETC node\n"); | 589 | "Find same PHY address in EMDIO and ENETC node\n"); |
| @@ -167,6 +167,25 @@ gve_free_pending_packet(struct gve_tx_ring *tx, · 166 unmodified lines | |||
| 167 | } | 167 | } |
| 168 | } | 168 | } |
| 169 | 169 | ||
| 170 | static void gve_unmap_packet(struct device *dev, | ||
| 171 | struct gve_tx_pending_packet_dqo *pkt) | ||
| 172 | { | ||
| 173 | int i; | ||
| 174 | |||
| 175 | if (!pkt->num_bufs) | ||
| 176 | return; | ||
| 177 | |||
| 178 | /* SKB linear portion is guaranteed to be mapped */ | ||
| 179 | dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), | ||
| 180 | dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); | ||
| 181 | for (i = 1; i < pkt->num_bufs; i++) { | ||
| 182 | netmem_dma_unmap_page_attrs(dev, dma_unmap_addr(pkt, dma[i]), | ||
| 183 | dma_unmap_len(pkt, len[i]), | ||
| 184 | DMA_TO_DEVICE, 0); | ||
| 185 | } | ||
| 186 | pkt->num_bufs = 0; | ||
| 187 | } | ||
| 188 | |||
| 170 | /* gve_tx_free_desc - Cleans up all pending tx requests and buffers. | 189 | /* gve_tx_free_desc - Cleans up all pending tx requests and buffers. |
| 171 | */ | 190 | */ |
| 172 | static void gve_tx_clean_pending_packets(struct gve_tx_ring *tx) | 191 | static void gve_tx_clean_pending_packets(struct gve_tx_ring *tx) |
| @@ -176,21 +195,12 @@ static void gve_tx_clean_pending_packets(struct gve_tx_ring *tx) · 3 unmodified lines | |||
| 176 | for (i = 0; i < tx->dqo.num_pending_packets; i++) { | 195 | for (i = 0; i < tx->dqo.num_pending_packets; i++) { |
| 177 | struct gve_tx_pending_packet_dqo *cur_state = | 196 | struct gve_tx_pending_packet_dqo *cur_state = |
| 178 | &tx->dqo.pending_packets[i]; | 197 | &tx->dqo.pending_packets[i]; |
| 179 | int j; | 198 | |
| 180 | 199 | if (tx->dqo.qpl) | |
| 181 | for (j = 0; j < cur_state->num_bufs; j++) { | 200 | gve_free_tx_qpl_bufs(tx, cur_state); |
| 182 | if (j == 0) { | 201 | else |
| 183 | dma_unmap_single(tx->dev, | 202 | gve_unmap_packet(tx->dev, cur_state); |
| 184 | dma_unmap_addr(cur_state, dma[j]), | 203 | |
| 185 | dma_unmap_len(cur_state, len[j]), | ||
| 186 | DMA_TO_DEVICE); | ||
| 187 | } else { | ||
| 188 | dma_unmap_page(tx->dev, | ||
| 189 | dma_unmap_addr(cur_state, dma[j]), | ||
| 190 | dma_unmap_len(cur_state, len[j]), | ||
| 191 | DMA_TO_DEVICE); | ||
| 192 | } | ||
| 193 | } | ||
| 194 | if (cur_state->skb) { | 204 | if (cur_state->skb) { |
| 195 | dev_consume_skb_any(cur_state->skb); | 205 | dev_consume_skb_any(cur_state->skb); |
| 196 | cur_state->skb = NULL; | 206 | cur_state->skb = NULL; |
| @@ -1154,22 +1164,6 @@ static void remove_from_list(struct gve_tx_ring *tx, · 957 unmodified lines | |||
| 1154 | } | 1164 | } |
| 1155 | } | 1165 | } |
| 1156 | 1166 | ||
| 1157 | static void gve_unmap_packet(struct device *dev, | ||
| 1158 | struct gve_tx_pending_packet_dqo *pkt) | ||
| 1159 | { | ||
| 1160 | int i; | ||
| 1161 | |||
| 1162 | /* SKB linear portion is guaranteed to be mapped */ | ||
| 1163 | dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), | ||
| 1164 | dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); | ||
| 1165 | for (i = 1; i < pkt->num_bufs; i++) { | ||
| 1166 | netmem_dma_unmap_page_attrs(dev, dma_unmap_addr(pkt, dma[i]), | ||
| 1167 | dma_unmap_len(pkt, len[i]), | ||
| 1168 | DMA_TO_DEVICE, 0); | ||
| 1169 | } | ||
| 1170 | pkt->num_bufs = 0; | ||
| 1171 | } | ||
| 1172 | |||
| 1173 | /* Completion types and expected behavior: | 1167 | /* Completion types and expected behavior: |
| 1174 | * No Miss compl + Packet compl = Packet completed normally. | 1168 | * No Miss compl + Packet compl = Packet completed normally. |
| 1175 | * Miss compl + Re-inject compl = Packet completed normally. | 1169 | * Miss compl + Re-inject compl = Packet completed normally. |
| @@ -2952,8 +2952,6 @@ static int e1000_tx_map(struct e1000_adapter *adapter, · 2951 unmodified lines | |||
| 2952 | dma_error: | 2952 | dma_error: |
| 2953 | dev_err(&pdev->dev, "TX DMA map failed\n"); | 2953 | dev_err(&pdev->dev, "TX DMA map failed\n"); |
| 2954 | buffer_info->dma = 0; | 2954 | buffer_info->dma = 0; |
| 2955 | if (count) | ||
| 2956 | count--; | ||
| 2957 | 2955 | ||
| 2958 | while (count--) { | 2956 | while (count--) { |
| 2959 | if (i == 0) | 2957 | if (i == 0) |
| @@ -33,6 +33,7 @@ · 32 unmodified lines | |||
| 33 | 33 | ||
| 34 | /* Extended Device Control */ | 34 | /* Extended Device Control */ |
| 35 | #define E1000_CTRL_EXT_LPCD 0x00000004 /* LCD Power Cycle Done */ | 35 | #define E1000_CTRL_EXT_LPCD 0x00000004 /* LCD Power Cycle Done */ |
| 36 | #define E1000_CTRL_EXT_DPG_EN 0x00000008 /* Dynamic Power Gating Enable */ | ||
| 36 | #define E1000_CTRL_EXT_SDP3_DATA 0x00000080 /* Value of SW Definable Pin 3 */ | 37 | #define E1000_CTRL_EXT_SDP3_DATA 0x00000080 /* Value of SW Definable Pin 3 */ |
| 37 | #define E1000_CTRL_EXT_FORCE_SMBUS 0x00000800 /* Force SMBus mode */ | 38 | #define E1000_CTRL_EXT_FORCE_SMBUS 0x00000800 /* Force SMBus mode */ |
| 38 | #define E1000_CTRL_EXT_EE_RST 0x00002000 /* Reinitialize from EEPROM */ | 39 | #define E1000_CTRL_EXT_EE_RST 0x00002000 /* Reinitialize from EEPROM */ |
| @@ -117,7 +117,8 @@ enum e1000_boards { · 116 unmodified lines | |||
| 117 | board_pch_cnp, | 117 | board_pch_cnp, |
| 118 | board_pch_tgp, | 118 | board_pch_tgp, |
| 119 | board_pch_adp, | 119 | board_pch_adp, |
| 120 | board_pch_mtp | 120 | board_pch_mtp, |
| 121 | board_pch_ptp | ||
| 121 | }; | 122 | }; |
| 122 | 123 | ||
| 123 | struct e1000_ps_page { | 124 | struct e1000_ps_page { |
| @@ -527,6 +528,7 @@ extern const struct e1000_info e1000_pch_cnp_info; · 403 unmodified lines | |||
| 527 | extern const struct e1000_info e1000_pch_tgp_info; | 528 | extern const struct e1000_info e1000_pch_tgp_info; |
| 528 | extern const struct e1000_info e1000_pch_adp_info; | 529 | extern const struct e1000_info e1000_pch_adp_info; |
| 529 | extern const struct e1000_info e1000_pch_mtp_info; | 530 | extern const struct e1000_info e1000_pch_mtp_info; |
| 531 | extern const struct e1000_info e1000_pch_ptp_info; | ||
| 530 | extern const struct e1000_info e1000_es2_info; | 532 | extern const struct e1000_info e1000_es2_info; |
| 531 | 533 | ||
| 532 | void e1000e_ptp_init(struct e1000_adapter *adapter); | 534 | void e1000e_ptp_init(struct e1000_adapter *adapter); |
| @@ -118,8 +118,6 @@ struct e1000_hw; · 117 unmodified lines | |||
| 118 | #define E1000_DEV_ID_PCH_ARL_I219_V24 0x57A1 | 118 | #define E1000_DEV_ID_PCH_ARL_I219_V24 0x57A1 |
| 119 | #define E1000_DEV_ID_PCH_PTP_I219_LM25 0x57B3 | 119 | #define E1000_DEV_ID_PCH_PTP_I219_LM25 0x57B3 |
| 120 | #define E1000_DEV_ID_PCH_PTP_I219_V25 0x57B4 | 120 | #define E1000_DEV_ID_PCH_PTP_I219_V25 0x57B4 |
| 121 | #define E1000_DEV_ID_PCH_PTP_I219_LM26 0x57B5 | ||
| 122 | #define E1000_DEV_ID_PCH_PTP_I219_V26 0x57B6 | ||
| 123 | #define E1000_DEV_ID_PCH_PTP_I219_LM27 0x57B7 | 121 | #define E1000_DEV_ID_PCH_PTP_I219_LM27 0x57B7 |
| 124 | #define E1000_DEV_ID_PCH_PTP_I219_V27 0x57B8 | 122 | #define E1000_DEV_ID_PCH_PTP_I219_V27 0x57B8 |
| 125 | #define E1000_DEV_ID_PCH_NVL_I219_LM29 0x57B9 | 123 | #define E1000_DEV_ID_PCH_NVL_I219_LM29 0x57B9 |
| @@ -528,7 +528,7 @@ static s32 e1000_init_phy_params_pchlan(struct e1000_hw *hw) · 527 unmodified lines | |||
| 528 | 528 | ||
| 529 | phy->id = e1000_phy_unknown; | 529 | phy->id = e1000_phy_unknown; |
| 530 | 530 | ||
| 531 | if (hw->mac.type == e1000_pch_mtp) { | 531 | if (hw->mac.type == e1000_pch_mtp || hw->mac.type == e1000_pch_ptp) { |
| 532 | phy->retry_count = 2; | 532 | phy->retry_count = 2; |
| 533 | e1000e_enable_phy_retry(hw); | 533 | e1000e_enable_phy_retry(hw); |
| 534 | } | 534 | } |
| @@ -4932,6 +4932,15 @@ static s32 e1000_reset_hw_ich8lan(struct e1000_hw *hw) · 4397 unmodified lines | |||
| 4932 | reg |= E1000_KABGTXD_BGSQLBIAS; | 4932 | reg |= E1000_KABGTXD_BGSQLBIAS; |
| 4933 | ew32(KABGTXD, reg); | 4933 | ew32(KABGTXD, reg); |
| 4934 | 4934 | ||
| 4935 | /* The hardware reset value of the DPG_EN bit is 1. | ||
| 4936 | * Clear DPG_EN to prevent unexpected autonomous power gating. | ||
| 4937 | */ | ||
| 4938 | if (hw->mac.type >= e1000_pch_ptp) { | ||
| 4939 | reg = er32(CTRL_EXT); | ||
| 4940 | reg &= ~E1000_CTRL_EXT_DPG_EN; | ||
| 4941 | ew32(CTRL_EXT, reg); | ||
| 4942 | } | ||
| 4943 | |||
| 4935 | return 0; | 4944 | return 0; |
| 4936 | } | 4945 | } |
| 4937 | 4946 | ||
| @@ -6208,3 +6217,23 @@ const struct e1000_info e1000_pch_mtp_info = { · 1270 unmodified lines | |||
| 6208 | .phy_ops = &ich8_phy_ops, | 6217 | .phy_ops = &ich8_phy_ops, |
| 6209 | .nvm_ops = &spt_nvm_ops, | 6218 | .nvm_ops = &spt_nvm_ops, |
| 6210 | }; | 6219 | }; |
| 6220 | |||
| 6221 | const struct e1000_info e1000_pch_ptp_info = { | ||
| 6222 | .mac = e1000_pch_ptp, | ||
| 6223 | .flags = FLAG_IS_ICH | ||
| 6224 | | FLAG_HAS_WOL | ||
| 6225 | | FLAG_HAS_HW_TIMESTAMP | ||
| 6226 | | FLAG_HAS_CTRLEXT_ON_LOAD | ||
| 6227 | | FLAG_HAS_AMT | ||
| 6228 | | FLAG_HAS_FLASH | ||
| 6229 | | FLAG_HAS_JUMBO_FRAMES | ||
| 6230 | | FLAG_APME_IN_WUC, | ||
| 6231 | .flags2 = FLAG2_HAS_PHY_STATS | ||
| 6232 | | FLAG2_HAS_EEE, | ||
| 6233 | .pba = 26, | ||
| 6234 | .max_hw_frame_size = 9022, | ||
| 6235 | .get_variants = e1000_get_variants_ich8lan, | ||
| 6236 | .mac_ops = &ich8_mac_ops, | ||
| 6237 | .phy_ops = &ich8_phy_ops, | ||
| 6238 | .nvm_ops = &spt_nvm_ops, | ||
| 6239 | }; | ||
| @@ -55,6 +55,7 @@ static const struct e1000_info *e1000_info_tbl[] = { · 54 unmodified lines | |||
| 55 | [board_pch_tgp] = &e1000_pch_tgp_info, | 55 | [board_pch_tgp] = &e1000_pch_tgp_info, |
| 56 | [board_pch_adp] = &e1000_pch_adp_info, | 56 | [board_pch_adp] = &e1000_pch_adp_info, |
| 57 | [board_pch_mtp] = &e1000_pch_mtp_info, | 57 | [board_pch_mtp] = &e1000_pch_mtp_info, |
| 58 | [board_pch_ptp] = &e1000_pch_ptp_info, | ||
| 58 | }; | 59 | }; |
| 59 | 60 | ||
| 60 | struct e1000_reg_info { | 61 | struct e1000_reg_info { |
| @@ -5651,8 +5652,6 @@ static int e1000_tx_map(struct e1000_ring *tx_ring, struct sk_buff *skb, · 5590 unmodified lines | |||
| 5651 | dma_error: | 5652 | dma_error: |
| 5652 | dev_err(&pdev->dev, "Tx DMA map failed\n"); | 5653 | dev_err(&pdev->dev, "Tx DMA map failed\n"); |
| 5653 | buffer_info->dma = 0; | 5654 | buffer_info->dma = 0; |
| 5654 | if (count) | ||
| 5655 | count--; | ||
| 5656 | 5655 | ||
| 5657 | while (count--) { | 5656 | while (count--) { |
| 5658 | if (i == 0) | 5657 | if (i == 0) |
| @@ -7922,14 +7921,12 @@ static const struct pci_device_id e1000_pci_tbl[] = { · 2263 unmodified lines | |||
| 7922 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_LNP_I219_V21), board_pch_mtp }, | 7921 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_LNP_I219_V21), board_pch_mtp }, |
| 7923 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_LM24), board_pch_mtp }, | 7922 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_LM24), board_pch_mtp }, |
| 7924 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_V24), board_pch_mtp }, | 7923 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_V24), board_pch_mtp }, |
| 7925 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM25), board_pch_mtp }, | 7924 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM25), board_pch_ptp }, |
| 7926 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V25), board_pch_mtp }, | 7925 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V25), board_pch_ptp }, |
| 7927 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM26), board_pch_mtp }, | 7926 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM27), board_pch_ptp }, |
| 7928 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V26), board_pch_mtp }, | 7927 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V27), board_pch_ptp }, |
| 7929 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM27), board_pch_mtp }, | 7928 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_LM29), board_pch_ptp }, |
| 7930 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V27), board_pch_mtp }, | 7929 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_V29), board_pch_ptp }, |
| 7931 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_LM29), board_pch_mtp }, | ||
| 7932 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_V29), board_pch_mtp }, | ||
| 7933 | 7930 | ||
| 7934 | { 0, 0, 0, 0, 0, 0, 0 } /* terminate list */ | 7931 | { 0, 0, 0, 0, 0, 0, 0 } /* terminate list */ |
| 7935 | }; | 7932 | }; |
| @@ -3569,6 +3569,7 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3568 unmodified lines | |||
| 3569 | u16 pf_q = vsi->base_queue + ring->queue_index; | 3569 | u16 pf_q = vsi->base_queue + ring->queue_index; |
| 3570 | struct i40e_hw *hw = &vsi->back->hw; | 3570 | struct i40e_hw *hw = &vsi->back->hw; |
| 3571 | struct i40e_hmc_obj_rxq rx_ctx; | 3571 | struct i40e_hmc_obj_rxq rx_ctx; |
| 3572 | u32 xdp_frame_sz; | ||
| 3572 | int err = 0; | 3573 | int err = 0; |
| 3573 | bool ok; | 3574 | bool ok; |
| 3574 | 3575 | ||
| @@ -3578,49 +3579,47 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3 unmodified lines | |||
| 3578 | memset(&rx_ctx, 0, sizeof(rx_ctx)); | 3579 | memset(&rx_ctx, 0, sizeof(rx_ctx)); |
| 3579 | 3580 | ||
| 3580 | ring->rx_buf_len = vsi->rx_buf_len; | 3581 | ring->rx_buf_len = vsi->rx_buf_len; |
| 3582 | xdp_frame_sz = i40e_rx_pg_size(ring) / 2; | ||
| 3581 | 3583 | ||
| 3582 | /* XDP RX-queue info only needed for RX rings exposed to XDP */ | 3584 | /* XDP RX-queue info only needed for RX rings exposed to XDP */ |
| 3583 | if (ring->vsi->type != I40E_VSI_MAIN) | 3585 | if (ring->vsi->type != I40E_VSI_MAIN) |
| 3584 | goto skip; | 3586 | goto skip; |
| 3585 | 3587 | ||
| 3586 | if (!xdp_rxq_info_is_reg(&ring->xdp_rxq)) { | ||
| 3587 | err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, | ||
| 3588 | ring->queue_index, | ||
| 3589 | ring->q_vector->napi.napi_id, | ||
| 3590 | ring->rx_buf_len); | ||
| 3591 | if (err) | ||
| 3592 | return err; | ||
| 3593 | } | ||
| 3594 | |||
| 3595 | ring->xsk_pool = i40e_xsk_pool(ring); | 3588 | ring->xsk_pool = i40e_xsk_pool(ring); |
| 3596 | if (ring->xsk_pool) { | 3589 | if (ring->xsk_pool) { |
| 3597 | xdp_rxq_info_unreg(&ring->xdp_rxq); | 3590 | xdp_frame_sz = xsk_pool_get_rx_frag_step(ring->xsk_pool); |
| 3598 | ring->rx_buf_len = xsk_pool_get_rx_frame_size(ring->xsk_pool); | 3591 | ring->rx_buf_len = xsk_pool_get_rx_frame_size(ring->xsk_pool); |
| 3599 | err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, | 3592 | err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, |
| 3600 | ring->queue_index, | 3593 | ring->queue_index, |
| 3601 | ring->q_vector->napi.napi_id, | 3594 | ring->q_vector->napi.napi_id, |
| 3602 | ring->rx_buf_len); | 3595 | xdp_frame_sz); |
| 3603 | if (err) | 3596 | if (err) |
| 3604 | return err; | 3597 | return err; |
| 3605 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, | 3598 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, |
| 3606 | MEM_TYPE_XSK_BUFF_POOL, | 3599 | MEM_TYPE_XSK_BUFF_POOL, |
| 3607 | NULL); | 3600 | NULL); |
| 3608 | if (err) | 3601 | if (err) |
| 3609 | return err; | 3602 | goto unreg_xdp; |
| 3610 | dev_info(&vsi->back->pdev->dev, | 3603 | dev_info(&vsi->back->pdev->dev, |
| 3611 | "Registered XDP mem model MEM_TYPE_XSK_BUFF_POOL on Rx ring %d\n", | 3604 | "Registered XDP mem model MEM_TYPE_XSK_BUFF_POOL on Rx ring %d\n", |
| 3612 | ring->queue_index); | 3605 | ring->queue_index); |
| 3613 | 3606 | ||
| 3614 | } else { | 3607 | } else { |
| 3608 | err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, | ||
| 3609 | ring->queue_index, | ||
| 3610 | ring->q_vector->napi.napi_id, | ||
| 3611 | xdp_frame_sz); | ||
| 3612 | if (err) | ||
| 3613 | return err; | ||
| 3615 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, | 3614 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, |
| 3616 | MEM_TYPE_PAGE_SHARED, | 3615 | MEM_TYPE_PAGE_SHARED, |
| 3617 | NULL); | 3616 | NULL); |
| 3618 | if (err) | 3617 | if (err) |
| 3619 | return err; | 3618 | goto unreg_xdp; |
| 3620 | } | 3619 | } |
| 3621 | 3620 | ||
| 3622 | skip: | 3621 | skip: |
| 3623 | xdp_init_buff(&ring->xdp, i40e_rx_pg_size(ring) / 2, &ring->xdp_rxq); | 3622 | xdp_init_buff(&ring->xdp, xdp_frame_sz, &ring->xdp_rxq); |
| 3624 | 3623 | ||
| 3625 | rx_ctx.dbuff = DIV_ROUND_UP(ring->rx_buf_len, | 3624 | rx_ctx.dbuff = DIV_ROUND_UP(ring->rx_buf_len, |
| 3626 | BIT_ULL(I40E_RXQ_CTX_DBUFF_SHIFT)); | 3625 | BIT_ULL(I40E_RXQ_CTX_DBUFF_SHIFT)); |
| @@ -3654,7 +3653,8 @@ skip: · 27 unmodified lines | |||
| 3654 | dev_info(&vsi->back->pdev->dev, | 3653 | dev_info(&vsi->back->pdev->dev, |
| 3655 | "Failed to clear LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", | 3654 | "Failed to clear LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", |
| 3656 | ring->queue_index, pf_q, err); | 3655 | ring->queue_index, pf_q, err); |
| 3657 | return -ENOMEM; | 3656 | err = -ENOMEM; |
| 3657 | goto unreg_xdp; | ||
| 3658 | } | 3658 | } |
| 3659 | 3659 | ||
| 3660 | /* set the context in the HMC */ | 3660 | /* set the context in the HMC */ |
| @@ -3663,7 +3663,8 @@ skip: · 2 unmodified lines | |||
| 3663 | dev_info(&vsi->back->pdev->dev, | 3663 | dev_info(&vsi->back->pdev->dev, |
| 3664 | "Failed to set LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", | 3664 | "Failed to set LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", |
| 3665 | ring->queue_index, pf_q, err); | 3665 | ring->queue_index, pf_q, err); |
| 3666 | return -ENOMEM; | 3666 | err = -ENOMEM; |
| 3667 | goto unreg_xdp; | ||
| 3667 | } | 3668 | } |
| 3668 | 3669 | ||
| 3669 | /* configure Rx buffer alignment */ | 3670 | /* configure Rx buffer alignment */ |
| @@ -3671,7 +3672,8 @@ skip: · 1 unmodified line | |||
| 3671 | if (I40E_2K_TOO_SMALL_WITH_PADDING) { | 3672 | if (I40E_2K_TOO_SMALL_WITH_PADDING) { |
| 3672 | dev_info(&vsi->back->pdev->dev, | 3673 | dev_info(&vsi->back->pdev->dev, |
| 3673 | "2k Rx buffer is too small to fit standard MTU and skb_shared_info\n"); | 3674 | "2k Rx buffer is too small to fit standard MTU and skb_shared_info\n"); |
| 3674 | return -EOPNOTSUPP; | 3675 | err = -EOPNOTSUPP; |
| 3676 | goto unreg_xdp; | ||
| 3675 | } | 3677 | } |
| 3676 | clear_ring_build_skb_enabled(ring); | 3678 | clear_ring_build_skb_enabled(ring); |
| 3677 | } else { | 3679 | } else { |
| @@ -3701,6 +3703,11 @@ skip: · 23 unmodified lines | |||
| 3701 | } | 3703 | } |
| 3702 | 3704 | ||
| 3703 | return 0; | 3705 | return 0; |
| 3706 | unreg_xdp: | ||
| 3707 | if (ring->vsi->type == I40E_VSI_MAIN) | ||
| 3708 | xdp_rxq_info_unreg(&ring->xdp_rxq); | ||
| 3709 | |||
| 3710 | return err; | ||
| 3704 | } | 3711 | } |
| 3705 | 3712 | ||
| 3706 | /** | 3713 | /** |
| @@ -88,7 +88,7 @@ TRACE_EVENT(i40e_napi_poll, · 87 unmodified lines | |||
| 88 | __entry->rx_clean_complete = rx_clean_complete; | 88 | __entry->rx_clean_complete = rx_clean_complete; |
| 89 | __entry->tx_clean_complete = tx_clean_complete; | 89 | __entry->tx_clean_complete = tx_clean_complete; |
| 90 | __entry->irq_num = q->irq_num; | 90 | __entry->irq_num = q->irq_num; |
| 91 | __entry->curr_cpu = get_cpu(); | 91 | __entry->curr_cpu = smp_processor_id(); |
| 92 | __assign_str(qname); | 92 | __assign_str(qname); |
| 93 | __assign_str(dev_name); | 93 | __assign_str(dev_name); |
| 94 | __assign_bitmask(irq_affinity, cpumask_bits(&q->affinity_mask), | 94 | __assign_bitmask(irq_affinity, cpumask_bits(&q->affinity_mask), |
| @@ -1470,6 +1470,9 @@ void i40e_clean_rx_ring(struct i40e_ring *rx_ring) · 1469 unmodified lines | |||
| 1470 | if (!rx_ring->rx_bi) | 1470 | if (!rx_ring->rx_bi) |
| 1471 | return; | 1471 | return; |
| 1472 | 1472 | ||
| 1473 | if (xdp_rxq_info_is_reg(&rx_ring->xdp_rxq)) | ||
| 1474 | xdp_rxq_info_unreg(&rx_ring->xdp_rxq); | ||
| 1475 | |||
| 1473 | if (rx_ring->xsk_pool) { | 1476 | if (rx_ring->xsk_pool) { |
| 1474 | i40e_xsk_clean_rx_ring(rx_ring); | 1477 | i40e_xsk_clean_rx_ring(rx_ring); |
| 1475 | goto skip_free; | 1478 | goto skip_free; |
| @@ -1527,8 +1530,6 @@ skip_free: · 51 unmodified lines | |||
| 1527 | void i40e_free_rx_resources(struct i40e_ring *rx_ring) | 1530 | void i40e_free_rx_resources(struct i40e_ring *rx_ring) |
| 1528 | { | 1531 | { |
| 1529 | i40e_clean_rx_ring(rx_ring); | 1532 | i40e_clean_rx_ring(rx_ring); |
| 1530 | if (rx_ring->vsi->type == I40E_VSI_MAIN) | ||
| 1531 | xdp_rxq_info_unreg(&rx_ring->xdp_rxq); | ||
| 1532 | rx_ring->xdp_prog = NULL; | 1533 | rx_ring->xdp_prog = NULL; |
| 1533 | kfree(rx_ring->rx_bi); | 1534 | kfree(rx_ring->rx_bi); |
| 1534 | rx_ring->rx_bi = NULL; | 1535 | rx_ring->rx_bi = NULL; |
| @@ -3833,10 +3833,10 @@ static int i40e_vc_del_cloud_filter(struct i40e_vf *vf, u8 *msg) · 3832 unmodified lines | |||
| 3833 | cfilter.n_proto = ETH_P_IP; | 3833 | cfilter.n_proto = ETH_P_IP; |
| 3834 | if (mask.dst_ip[0] & tcf.dst_ip[0]) | 3834 | if (mask.dst_ip[0] & tcf.dst_ip[0]) |
| 3835 | memcpy(&cfilter.ip.v4.dst_ip, tcf.dst_ip, | 3835 | memcpy(&cfilter.ip.v4.dst_ip, tcf.dst_ip, |
| 3836 | ARRAY_SIZE(tcf.dst_ip)); | 3836 | sizeof(cfilter.ip.v4.dst_ip)); |
| 3837 | else if (mask.src_ip[0] & tcf.dst_ip[0]) | 3837 | else if (mask.src_ip[0] & tcf.src_ip[0]) |
| 3838 | memcpy(&cfilter.ip.v4.src_ip, tcf.src_ip, | 3838 | memcpy(&cfilter.ip.v4.src_ip, tcf.src_ip, |
| 3839 | ARRAY_SIZE(tcf.dst_ip)); | 3839 | sizeof(cfilter.ip.v4.src_ip)); |
| 3840 | break; | 3840 | break; |
| 3841 | case VIRTCHNL_TCP_V6_FLOW: | 3841 | case VIRTCHNL_TCP_V6_FLOW: |
| 3842 | cfilter.n_proto = ETH_P_IPV6; | 3842 | cfilter.n_proto = ETH_P_IPV6; |
| @@ -3891,7 +3891,7 @@ static int i40e_vc_del_cloud_filter(struct i40e_vf *vf, u8 *msg) · 48 unmodified lines | |||
| 3891 | /* for ipv6, mask is set for all sixteen bytes (4 words) */ | 3891 | /* for ipv6, mask is set for all sixteen bytes (4 words) */ |
| 3892 | if (cfilter.n_proto == ETH_P_IPV6 && mask.dst_ip[3]) | 3892 | if (cfilter.n_proto == ETH_P_IPV6 && mask.dst_ip[3]) |
| 3893 | if (memcmp(&cfilter.ip.v6.dst_ip6, &cf->ip.v6.dst_ip6, | 3893 | if (memcmp(&cfilter.ip.v6.dst_ip6, &cf->ip.v6.dst_ip6, |
| 3894 | sizeof(cfilter.ip.v6.src_ip6))) | 3894 | sizeof(cfilter.ip.v6.dst_ip6))) |
| 3895 | continue; | 3895 | continue; |
| 3896 | if (mask.vlan_id) | 3896 | if (mask.vlan_id) |
| 3897 | if (cfilter.vlan_id != cf->vlan_id) | 3897 | if (cfilter.vlan_id != cf->vlan_id) |
| @@ -3979,10 +3979,10 @@ static int i40e_vc_add_cloud_filter(struct i40e_vf *vf, u8 *msg) · 81 unmodified lines | |||
| 3979 | cfilter->n_proto = ETH_P_IP; | 3979 | cfilter->n_proto = ETH_P_IP; |
| 3980 | if (mask.dst_ip[0] & tcf.dst_ip[0]) | 3980 | if (mask.dst_ip[0] & tcf.dst_ip[0]) |
| 3981 | memcpy(&cfilter->ip.v4.dst_ip, tcf.dst_ip, | 3981 | memcpy(&cfilter->ip.v4.dst_ip, tcf.dst_ip, |
| 3982 | ARRAY_SIZE(tcf.dst_ip)); | 3982 | sizeof(cfilter->ip.v4.dst_ip)); |
| 3983 | else if (mask.src_ip[0] & tcf.dst_ip[0]) | 3983 | else if (mask.src_ip[0] & tcf.src_ip[0]) |
| 3984 | memcpy(&cfilter->ip.v4.src_ip, tcf.src_ip, | 3984 | memcpy(&cfilter->ip.v4.src_ip, tcf.src_ip, |
| 3985 | ARRAY_SIZE(tcf.dst_ip)); | 3985 | sizeof(cfilter->ip.v4.src_ip)); |
| 3986 | break; | 3986 | break; |
| 3987 | case VIRTCHNL_TCP_V6_FLOW: | 3987 | case VIRTCHNL_TCP_V6_FLOW: |
| 3988 | cfilter->n_proto = ETH_P_IPV6; | 3988 | cfilter->n_proto = ETH_P_IPV6; |
| @@ -260,7 +260,6 @@ struct iavf_adapter { · 259 unmodified lines | |||
| 260 | struct work_struct adminq_task; | 260 | struct work_struct adminq_task; |
| 261 | struct work_struct finish_config; | 261 | struct work_struct finish_config; |
| 262 | wait_queue_head_t down_waitqueue; | 262 | wait_queue_head_t down_waitqueue; |
| 263 | wait_queue_head_t reset_waitqueue; | ||
| 264 | wait_queue_head_t vc_waitqueue; | 263 | wait_queue_head_t vc_waitqueue; |
| 265 | struct iavf_q_vector *q_vectors; | 264 | struct iavf_q_vector *q_vectors; |
| 266 | struct list_head vlan_filter_list; | 265 | struct list_head vlan_filter_list; |
| @@ -626,5 +625,5 @@ void iavf_add_adv_rss_cfg(struct iavf_adapter *adapter); · 359 unmodified lines | |||
| 626 | void iavf_del_adv_rss_cfg(struct iavf_adapter *adapter); | 625 | void iavf_del_adv_rss_cfg(struct iavf_adapter *adapter); |
| 627 | struct iavf_mac_filter *iavf_add_filter(struct iavf_adapter *adapter, | 626 | struct iavf_mac_filter *iavf_add_filter(struct iavf_adapter *adapter, |
| 628 | const u8 *macaddr); | 627 | const u8 *macaddr); |
| 629 | int iavf_wait_for_reset(struct iavf_adapter *adapter); | 628 | void iavf_reset_step(struct iavf_adapter *adapter); |
| 630 | #endif /* _IAVF_H_ */ | 629 | #endif /* _IAVF_H_ */ |
| @@ -492,7 +492,6 @@ static int iavf_set_ringparam(struct net_device *netdev, · 491 unmodified lines | |||
| 492 | { | 492 | { |
| 493 | struct iavf_adapter *adapter = netdev_priv(netdev); | 493 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 494 | u32 new_rx_count, new_tx_count; | 494 | u32 new_rx_count, new_tx_count; |
| 495 | int ret = 0; | ||
| 496 | 495 | ||
| 497 | if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending)) | 496 | if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending)) |
| 498 | return -EINVAL; | 497 | return -EINVAL; |
| @@ -537,13 +536,11 @@ static int iavf_set_ringparam(struct net_device *netdev, · 38 unmodified lines | |||
| 537 | } | 536 | } |
| 538 | 537 | ||
| 539 | if (netif_running(netdev)) { | 538 | if (netif_running(netdev)) { |
| 540 | iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | 539 | adapter->flags |= IAVF_FLAG_RESET_NEEDED; |
| 541 | ret = iavf_wait_for_reset(adapter); | 540 | iavf_reset_step(adapter); |
| 542 | if (ret) | ||
| 543 | netdev_warn(netdev, "Changing ring parameters timeout or interrupted waiting for reset"); | ||
| 544 | } | 541 | } |
| 545 | 542 | ||
| 546 | return ret; | 543 | return 0; |
| 547 | } | 544 | } |
| 548 | 545 | ||
| 549 | /** | 546 | /** |
| @@ -1723,7 +1720,6 @@ static int iavf_set_channels(struct net_device *netdev, · 1173 unmodified lines | |||
| 1723 | { | 1720 | { |
| 1724 | struct iavf_adapter *adapter = netdev_priv(netdev); | 1721 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 1725 | u32 num_req = ch->combined_count; | 1722 | u32 num_req = ch->combined_count; |
| 1726 | int ret = 0; | ||
| 1727 | 1723 | ||
| 1728 | if ((adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_ADQ) && | 1724 | if ((adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_ADQ) && |
| 1729 | adapter->num_tc) { | 1725 | adapter->num_tc) { |
| @@ -1745,13 +1741,10 @@ static int iavf_set_channels(struct net_device *netdev, · 15 unmodified lines | |||
| 1745 | 1741 | ||
| 1746 | adapter->num_req_queues = num_req; | 1742 | adapter->num_req_queues = num_req; |
| 1747 | adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED; | 1743 | adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED; |
| 1748 | iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | 1744 | adapter->flags |= IAVF_FLAG_RESET_NEEDED; |
| 1745 | iavf_reset_step(adapter); | ||
| 1749 | 1746 | ||
| 1750 | ret = iavf_wait_for_reset(adapter); | 1747 | return 0; |
| 1751 | if (ret) | ||
| 1752 | netdev_warn(netdev, "Changing channel count timeout or interrupted waiting for reset"); | ||
| 1753 | |||
| 1754 | return ret; | ||
| 1755 | } | 1748 | } |
| 1756 | 1749 | ||
| 1757 | /** | 1750 | /** |
| @@ -185,31 +185,6 @@ static bool iavf_is_reset_in_progress(struct iavf_adapter *adapter) · 184 unmodified lines | |||
| 185 | return false; | 185 | return false; |
| 186 | } | 186 | } |
| 187 | 187 | ||
| 188 | /** | ||
| 189 | * iavf_wait_for_reset - Wait for reset to finish. | ||
| 190 | * @adapter: board private structure | ||
| 191 | * | ||
| 192 | * Returns 0 if reset finished successfully, negative on timeout or interrupt. | ||
| 193 | */ | ||
| 194 | int iavf_wait_for_reset(struct iavf_adapter *adapter) | ||
| 195 | { | ||
| 196 | int ret = wait_event_interruptible_timeout(adapter->reset_waitqueue, | ||
| 197 | !iavf_is_reset_in_progress(adapter), | ||
| 198 | msecs_to_jiffies(5000)); | ||
| 199 | |||
| 200 | /* If ret < 0 then it means wait was interrupted. | ||
| 201 | * If ret == 0 then it means we got a timeout while waiting | ||
| 202 | * for reset to finish. | ||
| 203 | * If ret > 0 it means reset has finished. | ||
| 204 | */ | ||
| 205 | if (ret > 0) | ||
| 206 | return 0; | ||
| 207 | else if (ret < 0) | ||
| 208 | return -EINTR; | ||
| 209 | else | ||
| 210 | return -EBUSY; | ||
| 211 | } | ||
| 212 | |||
| 213 | /** | 188 | /** |
| 214 | * iavf_allocate_dma_mem_d - OS specific memory alloc for shared code | 189 | * iavf_allocate_dma_mem_d - OS specific memory alloc for shared code |
| 215 | * @hw: pointer to the HW structure | 190 | * @hw: pointer to the HW structure |
| @@ -2793,7 +2768,22 @@ static void iavf_init_config_adapter(struct iavf_adapter *adapter) · 2577 unmodified lines | |||
| 2793 | netdev->watchdog_timeo = 5 * HZ; | 2768 | netdev->watchdog_timeo = 5 * HZ; |
| 2794 | 2769 | ||
| 2795 | netdev->min_mtu = ETH_MIN_MTU; | 2770 | netdev->min_mtu = ETH_MIN_MTU; |
| 2796 | netdev->max_mtu = LIBIE_MAX_MTU; | 2771 | |
| 2772 | /* PF/VF API: vf_res->max_mtu is max frame size (not MTU). | ||
| 2773 | * Convert to MTU. | ||
| 2774 | */ | ||
| 2775 | if (!adapter->vf_res->max_mtu) { | ||
| 2776 | netdev->max_mtu = LIBIE_MAX_MTU; | ||
| 2777 | } else if (adapter->vf_res->max_mtu < LIBETH_RX_LL_LEN + ETH_MIN_MTU || | ||
| 2778 | adapter->vf_res->max_mtu > | ||
| 2779 | LIBETH_RX_LL_LEN + LIBIE_MAX_MTU) { | ||
| 2780 | netdev_warn_once(adapter->netdev, | ||
| 2781 | "invalid max frame size %d from PF, using default MTU %d", | ||
| 2782 | adapter->vf_res->max_mtu, LIBIE_MAX_MTU); | ||
| 2783 | netdev->max_mtu = LIBIE_MAX_MTU; | ||
| 2784 | } else { | ||
| 2785 | netdev->max_mtu = adapter->vf_res->max_mtu - LIBETH_RX_LL_LEN; | ||
| 2786 | } | ||
| 2797 | 2787 | ||
| 2798 | if (!is_valid_ether_addr(adapter->hw.mac.addr)) { | 2788 | if (!is_valid_ether_addr(adapter->hw.mac.addr)) { |
| 2799 | dev_info(&pdev->dev, "Invalid MAC address %pM, using random\n", | 2789 | dev_info(&pdev->dev, "Invalid MAC address %pM, using random\n", |
| @@ -3021,6 +3011,8 @@ static void iavf_disable_vf(struct iavf_adapter *adapter) · 221 unmodified lines | |||
| 3021 | 3011 | ||
| 3022 | adapter->flags |= IAVF_FLAG_PF_COMMS_FAILED; | 3012 | adapter->flags |= IAVF_FLAG_PF_COMMS_FAILED; |
| 3023 | 3013 | ||
| 3014 | iavf_ptp_release(adapter); | ||
| 3015 | |||
| 3024 | /* We don't use netif_running() because it may be true prior to | 3016 | /* We don't use netif_running() because it may be true prior to |
| 3025 | * ndo_open() returning, so we can't assume it means all our open | 3017 | * ndo_open() returning, so we can't assume it means all our open |
| 3026 | * tasks have finished, since we're not holding the rtnl_lock here. | 3018 | * tasks have finished, since we're not holding the rtnl_lock here. |
| @@ -3096,18 +3088,16 @@ static void iavf_reconfig_qs_bw(struct iavf_adapter *adapter) · 69 unmodified lines | |||
| 3096 | } | 3088 | } |
| 3097 | 3089 | ||
| 3098 | /** | 3090 | /** |
| 3099 | * iavf_reset_task - Call-back task to handle hardware reset | 3091 | * iavf_reset_step - Perform the VF reset sequence |
| 3100 | * @work: pointer to work_struct | 3092 | * @adapter: board private structure |
| 3101 | * | 3093 | * |
| 3102 | * During reset we need to shut down and reinitialize the admin queue | 3094 | * Requests a reset from PF, polls for completion, and reconfigures |
| 3103 | * before we can use it to communicate with the PF again. We also clear | 3095 | * the driver. Caller must hold the netdev instance lock. |
| 3104 | * and reinit the rings because that context is lost as well. | 3096 | * |
| 3105 | **/ | 3097 | * This can sleep for several seconds while polling HW registers. |
| 3106 | static void iavf_reset_task(struct work_struct *work) | 3098 | */ |
| 3099 | void iavf_reset_step(struct iavf_adapter *adapter) | ||
| 3107 | { | 3100 | { |
| 3108 | struct iavf_adapter *adapter = container_of(work, | ||
| 3109 | struct iavf_adapter, | ||
| 3110 | reset_task); | ||
| 3111 | struct virtchnl_vf_resource *vfres = adapter->vf_res; | 3101 | struct virtchnl_vf_resource *vfres = adapter->vf_res; |
| 3112 | struct net_device *netdev = adapter->netdev; | 3102 | struct net_device *netdev = adapter->netdev; |
| 3113 | struct iavf_hw *hw = &adapter->hw; | 3103 | struct iavf_hw *hw = &adapter->hw; |
| @@ -3118,7 +3108,7 @@ static void iavf_reset_task(struct work_struct *work) · 4 unmodified lines | |||
| 3118 | int i = 0, err; | 3108 | int i = 0, err; |
| 3119 | bool running; | 3109 | bool running; |
| 3120 | 3110 | ||
| 3121 | netdev_lock(netdev); | 3111 | netdev_assert_locked(netdev); |
| 3122 | 3112 | ||
| 3123 | iavf_misc_irq_disable(adapter); | 3113 | iavf_misc_irq_disable(adapter); |
| 3124 | if (adapter->flags & IAVF_FLAG_RESET_NEEDED) { | 3114 | if (adapter->flags & IAVF_FLAG_RESET_NEEDED) { |
| @@ -3163,7 +3153,6 @@ static void iavf_reset_task(struct work_struct *work) · 38 unmodified lines | |||
| 3163 | dev_err(&adapter->pdev->dev, "Reset never finished (%x)\n", | 3153 | dev_err(&adapter->pdev->dev, "Reset never finished (%x)\n", |
| 3164 | reg_val); | 3154 | reg_val); |
| 3165 | iavf_disable_vf(adapter); | 3155 | iavf_disable_vf(adapter); |
| 3166 | netdev_unlock(netdev); | ||
| 3167 | return; /* Do not attempt to reinit. It's dead, Jim. */ | 3156 | return; /* Do not attempt to reinit. It's dead, Jim. */ |
| 3168 | } | 3157 | } |
| 3169 | 3158 | ||
| @@ -3175,7 +3164,6 @@ continue_reset: · 5 unmodified lines | |||
| 3175 | iavf_startup(adapter); | 3164 | iavf_startup(adapter); |
| 3176 | queue_delayed_work(adapter->wq, &adapter->watchdog_task, | 3165 | queue_delayed_work(adapter->wq, &adapter->watchdog_task, |
| 3177 | msecs_to_jiffies(30)); | 3166 | msecs_to_jiffies(30)); |
| 3178 | netdev_unlock(netdev); | ||
| 3179 | return; | 3167 | return; |
| 3180 | } | 3168 | } |
| 3181 | 3169 | ||
| @@ -3196,6 +3184,8 @@ continue_reset: · 14 unmodified lines | |||
| 3196 | iavf_change_state(adapter, __IAVF_RESETTING); | 3184 | iavf_change_state(adapter, __IAVF_RESETTING); |
| 3197 | adapter->flags &= ~IAVF_FLAG_RESET_PENDING; | 3185 | adapter->flags &= ~IAVF_FLAG_RESET_PENDING; |
| 3198 | 3186 | ||
| 3187 | iavf_ptp_release(adapter); | ||
| 3188 | |||
| 3199 | /* free the Tx/Rx rings and descriptors, might be better to just | 3189 | /* free the Tx/Rx rings and descriptors, might be better to just |
| 3200 | * re-use them sometime in the future | 3190 | * re-use them sometime in the future |
| 3201 | */ | 3191 | */ |
| @@ -3316,9 +3306,6 @@ continue_reset: · 114 unmodified lines | |||
| 3316 | 3306 | ||
| 3317 | adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED; | 3307 | adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED; |
| 3318 | 3308 | ||
| 3319 | wake_up(&adapter->reset_waitqueue); | ||
| 3320 | netdev_unlock(netdev); | ||
| 3321 | |||
| 3322 | return; | 3309 | return; |
| 3323 | reset_err: | 3310 | reset_err: |
| 3324 | if (running) { | 3311 | if (running) { |
| @@ -3327,10 +3314,21 @@ reset_err: · 2 unmodified lines | |||
| 3327 | } | 3314 | } |
| 3328 | iavf_disable_vf(adapter); | 3315 | iavf_disable_vf(adapter); |
| 3329 | 3316 | ||
| 3330 | netdev_unlock(netdev); | ||
| 3331 | dev_err(&adapter->pdev->dev, "failed to allocate resources during reinit\n"); | 3317 | dev_err(&adapter->pdev->dev, "failed to allocate resources during reinit\n"); |
| 3332 | } | 3318 | } |
| 3333 | 3319 | ||
| 3320 | static void iavf_reset_task(struct work_struct *work) | ||
| 3321 | { | ||
| 3322 | struct iavf_adapter *adapter = container_of(work, | ||
| 3323 | struct iavf_adapter, | ||
| 3324 | reset_task); | ||
| 3325 | struct net_device *netdev = adapter->netdev; | ||
| 3326 | |||
| 3327 | netdev_lock(netdev); | ||
| 3328 | iavf_reset_step(adapter); | ||
| 3329 | netdev_unlock(netdev); | ||
| 3330 | } | ||
| 3331 | |||
| 3334 | /** | 3332 | /** |
| 3335 | * iavf_adminq_task - worker thread to clean the admin queue | 3333 | * iavf_adminq_task - worker thread to clean the admin queue |
| 3336 | * @work: pointer to work_struct containing our data | 3334 | * @work: pointer to work_struct containing our data |
| @@ -4596,22 +4594,17 @@ static int iavf_close(struct net_device *netdev) · 1259 unmodified lines | |||
| 4596 | static int iavf_change_mtu(struct net_device *netdev, int new_mtu) | 4594 | static int iavf_change_mtu(struct net_device *netdev, int new_mtu) |
| 4597 | { | 4595 | { |
| 4598 | struct iavf_adapter *adapter = netdev_priv(netdev); | 4596 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 4599 | int ret = 0; | ||
| 4600 | 4597 | ||
| 4601 | netdev_dbg(netdev, "changing MTU from %d to %d\n", | 4598 | netdev_dbg(netdev, "changing MTU from %d to %d\n", |
| 4602 | netdev->mtu, new_mtu); | 4599 | netdev->mtu, new_mtu); |
| 4603 | WRITE_ONCE(netdev->mtu, new_mtu); | 4600 | WRITE_ONCE(netdev->mtu, new_mtu); |
| 4604 | 4601 | ||
| 4605 | if (netif_running(netdev)) { | 4602 | if (netif_running(netdev)) { |
| 4606 | iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | 4603 | adapter->flags |= IAVF_FLAG_RESET_NEEDED; |
| 4607 | ret = iavf_wait_for_reset(adapter); | 4604 | iavf_reset_step(adapter); |
| 4608 | if (ret < 0) | ||
| 4609 | netdev_warn(netdev, "MTU change interrupted waiting for reset"); | ||
| 4610 | else if (ret) | ||
| 4611 | netdev_warn(netdev, "MTU change timed out waiting for reset"); | ||
| 4612 | } | 4605 | } |
| 4613 | 4606 | ||
| 4614 | return ret; | 4607 | return 0; |
| 4615 | } | 4608 | } |
| 4616 | 4609 | ||
| 4617 | /** | 4610 | /** |
| @@ -5416,9 +5409,6 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 798 unmodified lines | |||
| 5416 | /* Setup the wait queue for indicating transition to down status */ | 5409 | /* Setup the wait queue for indicating transition to down status */ |
| 5417 | init_waitqueue_head(&adapter->down_waitqueue); | 5410 | init_waitqueue_head(&adapter->down_waitqueue); |
| 5418 | 5411 | ||
| 5419 | /* Setup the wait queue for indicating transition to running state */ | ||
| 5420 | init_waitqueue_head(&adapter->reset_waitqueue); | ||
| 5421 | |||
| 5422 | /* Setup the wait queue for indicating virtchannel events */ | 5412 | /* Setup the wait queue for indicating virtchannel events */ |
| 5423 | init_waitqueue_head(&adapter->vc_waitqueue); | 5413 | init_waitqueue_head(&adapter->vc_waitqueue); |
| 5424 | 5414 | ||
| @@ -2736,7 +2736,6 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter, · 2735 unmodified lines | |||
| 2736 | case VIRTCHNL_OP_ENABLE_QUEUES: | 2736 | case VIRTCHNL_OP_ENABLE_QUEUES: |
| 2737 | /* enable transmits */ | 2737 | /* enable transmits */ |
| 2738 | iavf_irq_enable(adapter, true); | 2738 | iavf_irq_enable(adapter, true); |
| 2739 | wake_up(&adapter->reset_waitqueue); | ||
| 2740 | adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED; | 2739 | adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED; |
| 2741 | break; | 2740 | break; |
| 2742 | case VIRTCHNL_OP_DISABLE_QUEUES: | 2741 | case VIRTCHNL_OP_DISABLE_QUEUES: |
| @@ -1360,7 +1360,7 @@ ice_devlink_enable_roce_get(struct devlink *devlink, u32 id, · 1359 unmodified lines | |||
| 1360 | 1360 | ||
| 1361 | cdev = pf->cdev_info; | 1361 | cdev = pf->cdev_info; |
| 1362 | if (!cdev) | 1362 | if (!cdev) |
| 1363 | return -ENODEV; | ||
13 more lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 269 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 301 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 237 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 236 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 560 lines aren’t shown here.
This diff is large: its 245 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 476 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 420 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 275 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 331 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 295 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 309 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 193 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 398 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 217 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 306 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 2 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 548 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 162 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 434 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 2 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 866 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 4 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 166 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 427 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.