Linux 7.1-rc6
Unified · Split
| @@ -19,6 +19,7 @@ Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org> · 18 unmodified lines | |||
| 19 | Ahmad Masri <quic_amasri@quicinc.com> <amasri@codeaurora.org> | 19 | Ahmad Masri <quic_amasri@quicinc.com> <amasri@codeaurora.org> |
| 20 | Adam Oldham <oldhamca@gmail.com> | 20 | Adam Oldham <oldhamca@gmail.com> |
| 21 | Adam Radford <aradford@gmail.com> | 21 | Adam Radford <aradford@gmail.com> |
| 22 | Aditya Garg <gargaditya08@proton.me> <gargaditya08@live.com> | ||
| 22 | Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com> | 23 | Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com> |
| 23 | Adrian Bunk <bunk@stusta.de> | 24 | Adrian Bunk <bunk@stusta.de> |
| 24 | Ajay Kaher <ajay.kaher@broadcom.com> <akaher@vmware.com> | 25 | Ajay Kaher <ajay.kaher@broadcom.com> <akaher@vmware.com> |
| @@ -207,6 +208,7 @@ Claudiu Beznea <claudiu.beznea@tuxon.dev> <claudiu.beznea@microchip.com> · 182 unmodified lines | |||
| 207 | Colin Ian King <colin.i.king@gmail.com> <colin.king@canonical.com> | 208 | Colin Ian King <colin.i.king@gmail.com> <colin.king@canonical.com> |
| 208 | Corey Minyard <minyard@acm.org> | 209 | Corey Minyard <minyard@acm.org> |
| 209 | Damian Hobson-Garcia <dhobsong@igel.co.jp> | 210 | Damian Hobson-Garcia <dhobsong@igel.co.jp> |
| 211 | Dan Carpenter <error27@gmail.com> <dan.carpenter@linaro.org> | ||
| 210 | Dan Carpenter <error27@gmail.com> <dan.carpenter@oracle.com> | 212 | Dan Carpenter <error27@gmail.com> <dan.carpenter@oracle.com> |
| 211 | Dan Williams <djbw@kernel.org> <dan.j.williams@intel.com> | 213 | Dan Williams <djbw@kernel.org> <dan.j.williams@intel.com> |
| 212 | Daniel Borkmann <daniel@iogearbox.net> <danborkmann@googlemail.com> | 214 | Daniel Borkmann <daniel@iogearbox.net> <danborkmann@googlemail.com> |
| @@ -261,8 +263,9 @@ Enric Balletbo i Serra <eballetbo@kernel.org> <enric.balletbo@collabora.com> · 48 unmodified lines | |||
| 261 | Enric Balletbo i Serra <eballetbo@kernel.org> <eballetbo@iseebcn.com> | 263 | Enric Balletbo i Serra <eballetbo@kernel.org> <eballetbo@iseebcn.com> |
| 262 | Erik Kaneda <erik.kaneda@intel.com> <erik.schmauss@intel.com> | 264 | Erik Kaneda <erik.kaneda@intel.com> <erik.schmauss@intel.com> |
| 263 | Ethan Carter Edwards <ethan@ethancedwards.com> Ethan Edwards <ethancarteredwards@gmail.com> | 265 | Ethan Carter Edwards <ethan@ethancedwards.com> Ethan Edwards <ethancarteredwards@gmail.com> |
| 264 | Eugen Hristev <eugen.hristev@linaro.org> <eugen.hristev@microchip.com> | 266 | Eugen Hristev <ehristev@kernel.org> <eugen.hristev@microchip.com> |
| 265 | Eugen Hristev <eugen.hristev@linaro.org> <eugen.hristev@collabora.com> | 267 | Eugen Hristev <ehristev@kernel.org> <eugen.hristev@linaro.org> |
| 268 | Eugen Hristev <ehristev@kernel.org> <eugen.hristev@collabora.com> | ||
| 266 | Evgeniy Polyakov <johnpol@2ka.mipt.ru> | 269 | Evgeniy Polyakov <johnpol@2ka.mipt.ru> |
| 267 | Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> <ezequiel@collabora.com> | 270 | Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> <ezequiel@collabora.com> |
| 268 | Faith Ekstrand <faith.ekstrand@collabora.com> <jason@jlekstrand.net> | 271 | Faith Ekstrand <faith.ekstrand@collabora.com> <jason@jlekstrand.net> |
| @@ -337,6 +340,7 @@ Henrik Rydberg <rydberg@bitmath.org> · 68 unmodified lines | |||
| 337 | Herbert Xu <herbert@gondor.apana.org.au> | 340 | Herbert Xu <herbert@gondor.apana.org.au> |
| 338 | Huacai Chen <chenhuacai@kernel.org> <chenhc@lemote.com> | 341 | Huacai Chen <chenhuacai@kernel.org> <chenhc@lemote.com> |
| 339 | Huacai Chen <chenhuacai@kernel.org> <chenhuacai@loongson.cn> | 342 | Huacai Chen <chenhuacai@kernel.org> <chenhuacai@loongson.cn> |
| 343 | Ian Ray <ian.ray@gehealthcare.com> <ian.ray@ge.com> | ||
| 340 | Ignat Korchagin <ignat@linux.win> <ignat@cloudflare.com> | 344 | Ignat Korchagin <ignat@linux.win> <ignat@cloudflare.com> |
| 341 | Igor Korotin <igor.korotin@linux.dev> <igor.korotin.linux@gmail.com> | 345 | Igor Korotin <igor.korotin@linux.dev> <igor.korotin.linux@gmail.com> |
| 342 | Ike Panhc <ikepanhc@gmail.com> <ike.pan@canonical.com> | 346 | Ike Panhc <ikepanhc@gmail.com> <ike.pan@canonical.com> |
| @@ -495,6 +499,7 @@ Leon Romanovsky <leon@kernel.org> <leon@leon.nu> · 152 unmodified lines | |||
| 495 | Leon Romanovsky <leon@kernel.org> <leonro@mellanox.com> | 499 | Leon Romanovsky <leon@kernel.org> <leonro@mellanox.com> |
| 496 | Leon Romanovsky <leon@kernel.org> <leonro@nvidia.com> | 500 | Leon Romanovsky <leon@kernel.org> <leonro@nvidia.com> |
| 497 | Leo Yan <leo.yan@linux.dev> <leo.yan@linaro.org> | 501 | Leo Yan <leo.yan@linux.dev> <leo.yan@linaro.org> |
| 502 | Liam R. Howlett <liam@infradead.org> <Liam.Howlett@oracle.com> | ||
| 498 | Liam Mark <quic_lmark@quicinc.com> <lmark@codeaurora.org> | 503 | Liam Mark <quic_lmark@quicinc.com> <lmark@codeaurora.org> |
| 499 | Linas Vepstas <linas@austin.ibm.com> | 504 | Linas Vepstas <linas@austin.ibm.com> |
| 500 | Linus Lüssing <linus.luessing@c0d3.blue> <linus.luessing@ascom.ch> | 505 | Linus Lüssing <linus.luessing@c0d3.blue> <linus.luessing@ascom.ch> |
| @@ -505,6 +510,8 @@ Linus Walleij <linusw@kernel.org> <linus.walleij@stericsson.com> · 4 unmodified lines | |||
| 505 | Linus Walleij <linusw@kernel.org> <linus.walleij@linaro.org> | 510 | Linus Walleij <linusw@kernel.org> <linus.walleij@linaro.org> |
| 506 | Linus Walleij <linusw@kernel.org> <triad@df.lth.se> | 511 | Linus Walleij <linusw@kernel.org> <triad@df.lth.se> |
| 507 | <linux-hardening@vger.kernel.org> <kernel-hardening@lists.openwall.com> | 512 | <linux-hardening@vger.kernel.org> <kernel-hardening@lists.openwall.com> |
| 513 | Li Wang <li.wang@linux.dev> <liwang@redhat.com> | ||
| 514 | Li Wang <li.wang@linux.dev> <wangli.ahau@gmail.com> | ||
| 508 | Li Yang <leoyang.li@nxp.com> <leoli@freescale.com> | 515 | Li Yang <leoyang.li@nxp.com> <leoli@freescale.com> |
| 509 | Li Yang <leoyang.li@nxp.com> <leo@zh-kernel.org> | 516 | Li Yang <leoyang.li@nxp.com> <leo@zh-kernel.org> |
| 510 | Lior David <quic_liord@quicinc.com> <liord@codeaurora.org> | 517 | Lior David <quic_liord@quicinc.com> <liord@codeaurora.org> |
| @@ -579,6 +586,8 @@ Mayuresh Janorkar <mayur@ti.com> · 68 unmodified lines | |||
| 579 | Md Sadre Alam <quic_mdalam@quicinc.com> <mdalam@codeaurora.org> | 586 | Md Sadre Alam <quic_mdalam@quicinc.com> <mdalam@codeaurora.org> |
| 580 | Miaoqing Pan <quic_miaoqing@quicinc.com> <miaoqing@codeaurora.org> | 587 | Miaoqing Pan <quic_miaoqing@quicinc.com> <miaoqing@codeaurora.org> |
| 581 | Michael Buesch <m@bues.ch> | 588 | Michael Buesch <m@bues.ch> |
| 589 | Michal Grzeschik <mgr@kernel.org> <m.grzeschik@pengutronix.de> | ||
| 590 | Michal Grzeschik <mgr@kernel.org> <mgr@pengutronix.de> | ||
| 582 | Michael Riesch <michael.riesch@collabora.com> <michael.riesch@wolfvision.net> | 591 | Michael Riesch <michael.riesch@collabora.com> <michael.riesch@wolfvision.net> |
| 583 | Michal Simek <michal.simek@amd.com> <michal.simek@xilinx.com> | 592 | Michal Simek <michal.simek@amd.com> <michal.simek@xilinx.com> |
| 584 | Michel Dänzer <michel@tungstengraphics.com> | 593 | Michel Dänzer <michel@tungstengraphics.com> |
| @@ -677,6 +686,7 @@ Peter A Jonsson <pj@ludd.ltu.se> · 92 unmodified lines | |||
| 677 | Peter Hilber <peter.hilber@oss.qualcomm.com> <quic_philber@quicinc.com> | 686 | Peter Hilber <peter.hilber@oss.qualcomm.com> <quic_philber@quicinc.com> |
| 678 | Peter Oruba <peter.oruba@amd.com> | 687 | Peter Oruba <peter.oruba@amd.com> |
| 679 | Peter Oruba <peter@oruba.de> | 688 | Peter Oruba <peter@oruba.de> |
| 689 | Peter Rosin <peda@lysator.liu.se> <peda@axentia.se> | ||
| 680 | Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> <pierre-louis.bossart@linux.intel.com> | 690 | Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> <pierre-louis.bossart@linux.intel.com> |
| 681 | Pratyush Anand <pratyush.anand@gmail.com> <pratyush.anand@st.com> | 691 | Pratyush Anand <pratyush.anand@gmail.com> <pratyush.anand@st.com> |
| 682 | Pratyush Yadav <pratyush@kernel.org> <ptyadav@amazon.de> | 692 | Pratyush Yadav <pratyush@kernel.org> <ptyadav@amazon.de> |
| @@ -687,6 +697,7 @@ Punit Agrawal <punitagrawal@gmail.com> <punit.agrawal@arm.com> · 4 unmodified lines | |||
| 687 | Puranjay Mohan <puranjay@kernel.org> <puranjay12@gmail.com> | 697 | Puranjay Mohan <puranjay@kernel.org> <puranjay12@gmail.com> |
| 688 | Qais Yousef <qyousef@layalina.io> <qais.yousef@imgtec.com> | 698 | Qais Yousef <qyousef@layalina.io> <qais.yousef@imgtec.com> |
| 689 | Qais Yousef <qyousef@layalina.io> <qais.yousef@arm.com> | 699 | Qais Yousef <qyousef@layalina.io> <qais.yousef@arm.com> |
| 700 | Qi Zheng <qi.zheng@linux.dev> <zhengqi.arch@bytedance.com> | ||
| 690 | Quentin Monnet <qmo@kernel.org> <quentin.monnet@netronome.com> | 701 | Quentin Monnet <qmo@kernel.org> <quentin.monnet@netronome.com> |
| 691 | Quentin Monnet <qmo@kernel.org> <quentin@isovalent.com> | 702 | Quentin Monnet <qmo@kernel.org> <quentin@isovalent.com> |
| 692 | Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com> | 703 | Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com> |
| @@ -850,6 +861,7 @@ Tobias Klauser <tklauser@distanz.ch> <klto@zhaw.ch> · 157 unmodified lines | |||
| 850 | Tobias Klauser <tklauser@distanz.ch> <tklauser@nuerscht.ch> | 861 | Tobias Klauser <tklauser@distanz.ch> <tklauser@nuerscht.ch> |
| 851 | Tobias Klauser <tklauser@distanz.ch> <tklauser@xenon.tklauser.home> | 862 | Tobias Klauser <tklauser@distanz.ch> <tklauser@xenon.tklauser.home> |
| 852 | Todor Tomov <todor.too@gmail.com> <todor.tomov@linaro.org> | 863 | Todor Tomov <todor.too@gmail.com> <todor.tomov@linaro.org> |
| 864 | Tomasz Jeznach <tomasz.jeznach@linux.dev> <tjeznach@rivosinc.com> | ||
| 853 | Tony Luck <tony.luck@intel.com> | 865 | Tony Luck <tony.luck@intel.com> |
| 854 | Trilok Soni <quic_tsoni@quicinc.com> <tsoni@codeaurora.org> | 866 | Trilok Soni <quic_tsoni@quicinc.com> <tsoni@codeaurora.org> |
| 855 | TripleX Chung <xxx.phy@gmail.com> <triplex@zh-kernel.org> | 867 | TripleX Chung <xxx.phy@gmail.com> <triplex@zh-kernel.org> |
| @@ -786,6 +786,7 @@ networking/altera_tse networking/device_drivers/ethernet/altera/altera_tse · 785 unmodified lines | |||
| 786 | networking/bpf_flow_dissector bpf/prog_flow_dissector | 786 | networking/bpf_flow_dissector bpf/prog_flow_dissector |
| 787 | networking/cxacru networking/device_drivers/atm/cxacru | 787 | networking/cxacru networking/device_drivers/atm/cxacru |
| 788 | networking/defza networking/device_drivers/fddi/defza | 788 | networking/defza networking/device_drivers/fddi/defza |
| 789 | networking/device_drivers/3com/3c509 networking/device_drivers/ethernet/3com/3c509 | ||
| 789 | networking/device_drivers/3com/vortex networking/device_drivers/ethernet/3com/vortex | 790 | networking/device_drivers/3com/vortex networking/device_drivers/ethernet/3com/vortex |
| 790 | networking/device_drivers/amazon/ena networking/device_drivers/ethernet/amazon/ena | 791 | networking/device_drivers/amazon/ena networking/device_drivers/ethernet/amazon/ena |
| 791 | networking/device_drivers/aquantia/atlantic networking/device_drivers/ethernet/aquantia/atlantic | 792 | networking/device_drivers/aquantia/atlantic networking/device_drivers/ethernet/aquantia/atlantic |
File renamed without changes.
| @@ -47,21 +47,19 @@ Please note that implementation details can be changed. · 46 unmodified lines | |||
| 47 | Called when swp_entry's refcnt goes down to 0. A charge against swap | 47 | Called when swp_entry's refcnt goes down to 0. A charge against swap |
| 48 | disappears. | 48 | disappears. |
| 49 | 49 | ||
| 50 | 3. charge-commit-cancel | 50 | 3. charge-commit |
| 51 | ======================= | 51 | ======================= |
| 52 | 52 | ||
| 53 | Memcg pages are charged in two steps: | 53 | Memcg pages are charged in two steps: |
| 54 | 54 | ||
| 55 | - mem_cgroup_try_charge() | 55 | - mem_cgroup_try_charge() |
| 56 | - mem_cgroup_commit_charge() or mem_cgroup_cancel_charge() | 56 | - commit_charge() |
| 57 | 57 | ||
| 58 | At try_charge(), there are no flags to say "this page is charged". | 58 | At try_charge(), there are no flags to say "this page is charged". |
| 59 | at this point, usage += PAGE_SIZE. | 59 | at this point, usage += PAGE_SIZE. |
| 60 | 60 | ||
| 61 | At commit(), the page is associated with the memcg. | 61 | At commit(), the page is associated with the memcg. |
| 62 | 62 | ||
| 63 | At cancel(), simply usage -= PAGE_SIZE. | ||
| 64 | |||
| 65 | Under below explanation, we assume CONFIG_SWAP=y. | 63 | Under below explanation, we assume CONFIG_SWAP=y. |
| 66 | 64 | ||
| 67 | 4. Anonymous | 65 | 4. Anonymous |
| @@ -220,7 +220,7 @@ cgroup v2 currently supports the following mount options. · 219 unmodified lines | |||
| 220 | memory_hugetlb_accounting | 220 | memory_hugetlb_accounting |
| 221 | Count HugeTLB memory usage towards the cgroup's overall | 221 | Count HugeTLB memory usage towards the cgroup's overall |
| 222 | memory usage for the memory controller (for the purpose of | 222 | memory usage for the memory controller (for the purpose of |
| 223 | statistics reporting and memory protetion). This is a new | 223 | statistics reporting and memory protection). This is a new |
| 224 | behavior that could regress existing setups, so it must be | 224 | behavior that could regress existing setups, so it must be |
| 225 | explicitly opted in with this mount option. | 225 | explicitly opted in with this mount option. |
| 226 | 226 | ||
| @@ -789,24 +789,6 @@ Kernel parameters · 788 unmodified lines | |||
| 789 | cio_ignore= [S390] | 789 | cio_ignore= [S390] |
| 790 | See Documentation/arch/s390/common_io.rst for details. | 790 | See Documentation/arch/s390/common_io.rst for details. |
| 791 | 791 | ||
| 792 | clearcpuid=X[,X...] [X86] | ||
| 793 | Disable CPUID feature X for the kernel. See | ||
| 794 | arch/x86/include/asm/cpufeatures.h for the valid bit | ||
| 795 | numbers X. Note the Linux-specific bits are not necessarily | ||
| 796 | stable over kernel options, but the vendor-specific | ||
| 797 | ones should be. | ||
| 798 | X can also be a string as appearing in the flags: line | ||
| 799 | in /proc/cpuinfo which does not have the above | ||
| 800 | instability issue. However, not all features have names | ||
| 801 | in /proc/cpuinfo. | ||
| 802 | Note that using this option will taint your kernel. | ||
| 803 | Also note that user programs calling CPUID directly | ||
| 804 | or using the feature without checking anything | ||
| 805 | will still see it. This just prevents it from | ||
| 806 | being used by the kernel or shown in /proc/cpuinfo. | ||
| 807 | Also note the kernel might malfunction if you disable | ||
| 808 | some critical bits. | ||
| 809 | |||
| 810 | clk_ignore_unused | 792 | clk_ignore_unused |
| 811 | [CLK] | 793 | [CLK] |
| 812 | Prevents the clock framework from automatically gating | 794 | Prevents the clock framework from automatically gating |
| @@ -43,6 +43,11 @@ Support for changing the platform performance mode is currently not implemented. · 42 unmodified lines | |||
| 43 | Battery Charging Control | 43 | Battery Charging Control |
| 44 | ------------------------ | 44 | ------------------------ |
| 45 | 45 | ||
| 46 | .. warning:: Some devices do not properly implement the charging threshold interface. Forcing | ||
| 47 | the driver to enable access to said interface on such devices might damage the | ||
| 48 | battery [1]_. Because of this the driver will not enable said feature even when | ||
| 49 | using the ``force`` module parameter. | ||
| 50 | |||
| 46 | The ``uniwill-laptop`` driver supports controlling the battery charge limit. This happens over | 51 | The ``uniwill-laptop`` driver supports controlling the battery charge limit. This happens over |
| 47 | the standard ``charge_control_end_threshold`` power supply sysfs attribute. All values | 52 | the standard ``charge_control_end_threshold`` power supply sysfs attribute. All values |
| 48 | between 1 and 100 percent are supported. | 53 | between 1 and 100 percent are supported. |
| @@ -70,3 +75,8 @@ The ``uniwill-laptop`` driver allows to set the configurable TGP for devices wit · 21 unmodified lines | |||
| 70 | allow it. | 75 | allow it. |
| 71 | 76 | ||
| 72 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. | 77 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. |
| 78 | |||
| 79 | References | ||
| 80 | ========== | ||
| 81 | |||
| 82 | .. [1] https://www.reddit.com/r/XMG_gg/comments/ld9yyf/battery_limit_hidden_function_discovered_on/ | ||
| @@ -358,9 +358,9 @@ Dynamic energy performance profile · 357 unmodified lines | |||
| 358 | The amd-pstate driver supports dynamically selecting the energy performance | 358 | The amd-pstate driver supports dynamically selecting the energy performance |
| 359 | profile based on whether the machine is running on AC or DC power. | 359 | profile based on whether the machine is running on AC or DC power. |
| 360 | 360 | ||
| 361 | Whether this behavior is enabled by default depends on the kernel | 361 | Whether this behavior is enabled by default depends on the kernel command line option |
| 362 | config option `CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP`. This behavior can also be overridden | 362 | ``amd_dynamic_epp`` is set. This behavior can also be overridden |
| 363 | at runtime by the sysfs file ``/sys/devices/system/cpu/cpufreq/policyX/dynamic_epp``. | 363 | at runtime by the sysfs file ``/sys/devices/system/cpu/amd_pstate/dynamic_epp``. |
| 364 | 364 | ||
| 365 | When set to enabled, the driver will select a different energy performance | 365 | When set to enabled, the driver will select a different energy performance |
| 366 | profile when the machine is running on battery or AC power. The driver will | 366 | profile when the machine is running on battery or AC power. The driver will |
| @@ -485,9 +485,8 @@ kernel parameter ``amd_prefcore=disable``. · 118 unmodified lines | |||
| 485 | ``amd_dynamic_epp`` | 485 | ``amd_dynamic_epp`` |
| 486 | 486 | ||
| 487 | When AMD pstate is in auto mode, dynamic EPP will control whether the kernel | 487 | When AMD pstate is in auto mode, dynamic EPP will control whether the kernel |
| 488 | autonomously changes the EPP mode. The default is configured by | 488 | autonomously changes the EPP mode. The default is disabled. It can be enabled |
| 489 | ``CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP`` but can be explicitly enabled with | 489 | with the kernel parameter ``amd_dynamic_epp=enable``. |
| 490 | ``amd_dynamic_epp=enable`` or disabled with ``amd_dynamic_epp=disable``. | ||
| 491 | 490 | ||
| 492 | User Space Interface in ``sysfs`` - General | 491 | User Space Interface in ``sysfs`` - General |
| 493 | =========================================== | 492 | =========================================== |
| @@ -355,11 +355,12 @@ HyperThreading (HT) in the context of Intel processors, is enabled on at least · 354 unmodified lines | |||
| 355 | one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely, | 355 | one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely, |
| 356 | the priority of a given CPU reflects its highest HWP performance level which | 356 | the priority of a given CPU reflects its highest HWP performance level which |
| 357 | causes the CPU scheduler to generally prefer more performant CPUs, so the less | 357 | causes the CPU scheduler to generally prefer more performant CPUs, so the less |
| 358 | performant CPUs are used when the other ones are fully loaded. However, SMT | 358 | performant CPUs are used when the other ones are fully loaded. SMT siblings |
| 359 | siblings (that is, logical CPUs sharing one physical core) are treated in a | 359 | (that is, logical CPUs sharing one physical core) are given the same priority. |
| 360 | special way such that if one of them is in use, the effective priority of the | 360 | The scheduler can pull tasks from lower-priority cores and place them on any |
| 361 | other ones is lowered below the priorities of the CPUs located in the other | 361 | sibling. Since the scheduler spreads tasks among physical cores, tasks will be |
| 362 | physical cores. | 362 | placed on the SMT siblings of physical cores only after all physical cores are |
| 363 | busy. | ||
| 363 | 364 | ||
| 364 | This approach maximizes performance in the majority of cases, but unfortunately | 365 | This approach maximizes performance in the majority of cases, but unfortunately |
| 365 | it also leads to excessive energy usage in some important scenarios, like video | 366 | it also leads to excessive energy usage in some important scenarios, like video |
| @@ -21,13 +21,13 @@ call at each patchable function entry, and patches it dynamically at runtime to · 20 unmodified lines | |||
| 21 | enable or disable the redirection. In the case of RISC-V, 2 instructions, | 21 | enable or disable the redirection. In the case of RISC-V, 2 instructions, |
| 22 | AUIPC + JALR, are required to compose a function call. However, it is impossible | 22 | AUIPC + JALR, are required to compose a function call. However, it is impossible |
| 23 | to patch 2 instructions and expect that a concurrent read-side executes them | 23 | to patch 2 instructions and expect that a concurrent read-side executes them |
| 24 | without a race condition. This series makes atmoic code patching possible in | 24 | without a race condition. This series makes atomic code patching possible in |
| 25 | RISC-V ftrace. Kernel preemption makes things even worse as it allows the old | 25 | RISC-V ftrace. Kernel preemption makes things even worse as it allows the old |
| 26 | state to persist across the patching process with stop_machine(). | 26 | state to persist across the patching process with stop_machine(). |
| 27 | 27 | ||
| 28 | In order to get rid of stop_machine() and run dynamic ftrace with full kernel | 28 | In order to get rid of stop_machine() and run dynamic ftrace with full kernel |
| 29 | preemption, we partially initialize each patchable function entry at boot-time, | 29 | preemption, we partially initialize each patchable function entry at boot-time, |
| 30 | setting the first instruction to AUIPC, and the second to NOP. Now, atmoic | 30 | setting the first instruction to AUIPC, and the second to NOP. Now, atomic |
| 31 | patching is possible because the kernel only has to update one instruction. | 31 | patching is possible because the kernel only has to update one instruction. |
| 32 | According to Ziccif, as long as an instruction is naturally aligned, the ISA | 32 | According to Ziccif, as long as an instruction is naturally aligned, the ISA |
| 33 | guarantee an atomic update. | 33 | guarantee an atomic update. |
| @@ -36,8 +36,8 @@ By fixing down the first instruction, AUIPC, the range of the ftrace trampoline · 2 unmodified lines | |||
| 36 | is limited to +-2K from the predetermined target, ftrace_caller, due to the lack | 36 | is limited to +-2K from the predetermined target, ftrace_caller, due to the lack |
| 37 | of immediate encoding space in RISC-V. To address the issue, we introduce | 37 | of immediate encoding space in RISC-V. To address the issue, we introduce |
| 38 | CALL_OPS, where an 8B naturally align metadata is added in front of each | 38 | CALL_OPS, where an 8B naturally align metadata is added in front of each |
| 39 | pacthable function. The metadata is resolved at the first trampoline, then the | 39 | patchable function. The metadata is resolved at the first trampoline, then the |
| 40 | execution can be derect to another custom trampoline. | 40 | execution can be directed to another custom trampoline. |
| 41 | 41 | ||
| 42 | CMODX in the User Space | 42 | CMODX in the User Space |
| 43 | ----------------------- | 43 | ----------------------- |
| @@ -78,7 +78,7 @@ the program. · 77 unmodified lines | |||
| 78 | 78 | ||
| 79 | Per-task indirect branch tracking state can be monitored and | 79 | Per-task indirect branch tracking state can be monitored and |
| 80 | controlled via the :c:macro:`PR_GET_CFI` and :c:macro:`PR_SET_CFI` | 80 | controlled via the :c:macro:`PR_GET_CFI` and :c:macro:`PR_SET_CFI` |
| 81 | ``prctl()` arguments (respectively), by supplying | 81 | ``prctl()`` arguments (respectively), by supplying |
| 82 | :c:macro:`PR_CFI_BRANCH_LANDING_PADS` as the second argument. These | 82 | :c:macro:`PR_CFI_BRANCH_LANDING_PADS` as the second argument. These |
| 83 | are architecture-agnostic, and will return -EINVAL if the underlying | 83 | are architecture-agnostic, and will return -EINVAL if the underlying |
| 84 | functionality is not supported. | 84 | functionality is not supported. |
| @@ -187,6 +187,10 @@ to disable features using the feature number as defined in · 186 unmodified lines | |||
| 187 | Protection can be disabled using clearcpuid=514. The number 514 is calculated | 187 | Protection can be disabled using clearcpuid=514. The number 514 is calculated |
| 188 | from #define X86_FEATURE_UMIP (16*32 + 2). | 188 | from #define X86_FEATURE_UMIP (16*32 + 2). |
| 189 | 189 | ||
| 190 | DO NOT USE this cmdline option in production - it is meant to be used only as | ||
| 191 | a quick'n'dirty debugging aid to rule out a feature-enabling code is the | ||
| 192 | culprit. If you use it, it'll taint the kernel. | ||
| 193 | |||
| 190 | In addition, there exists a variety of custom command-line parameters that | 194 | In addition, there exists a variety of custom command-line parameters that |
| 191 | disable specific features. The list of parameters includes, but is not limited | 195 | disable specific features. The list of parameters includes, but is not limited |
| 192 | to, nofsgsbase, nosgx, noxsave, etc. 5-level paging can also be disabled using | 196 | to, nofsgsbase, nosgx, noxsave, etc. 5-level paging can also be disabled using |
| @@ -158,13 +158,22 @@ returned. · 157 unmodified lines | |||
| 158 | When a message has been received, the location and size of the data with the | 158 | When a message has been received, the location and size of the data with the |
| 159 | message can be determined by calling:: | 159 | message can be determined by calling:: |
| 160 | 160 | ||
| 161 | void crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | 161 | int crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, |
| 162 | enum krb5_crypto_mode mode, | 162 | enum krb5_crypto_mode mode, |
| 163 | size_t *_offset, size_t *_len); | 163 | size_t *_offset, size_t *_len); |
| 164 | 164 | ||
| 165 | The caller provides the offset and length of the message to the function, which | 165 | The caller provides the offset and length of the message to the function, which |
| 166 | then alters those values to indicate the region containing the data (plus any | 166 | then alters those values to indicate the region containing the data (plus any |
| 167 | padding). It is up to the caller to determine how much padding there is. | 167 | padding). It is up to the caller to determine how much padding there is. The |
| 168 | function returns an error if the length is too small or if the mode is | ||
| 169 | unsupported. An additional function:: | ||
| 170 | |||
| 171 | int crypto_krb5_check_data_len(const struct krb5_enctype *krb5, | ||
| 172 | enum krb5_crypto_mode mode, | ||
| 173 | size_t len, size_t min_content); | ||
| 174 | |||
| 175 | is provided to just do a basic check that the decrypted/verified message would | ||
| 176 | have a sufficient minimum payload. | ||
| 168 | 177 | ||
| 169 | Preparation Functions | 178 | Preparation Functions |
| 170 | --------------------- | 179 | --------------------- |
| @@ -219,6 +219,7 @@ allOf: · 218 unmodified lines | |||
| 219 | - required: | 219 | - required: |
| 220 | - "#sound-dai-cells" | 220 | - "#sound-dai-cells" |
| 221 | else: | 221 | else: |
| 222 | $ref: /schemas/sound/dai-common.yaml# | ||
| 222 | properties: | 223 | properties: |
| 223 | aux-bus: false | 224 | aux-bus: false |
| 224 | required: | 225 | required: |
| @@ -243,7 +244,7 @@ allOf: · 18 unmodified lines | |||
| 243 | clocks: | 244 | clocks: |
| 244 | minItems: 5 | 245 | minItems: 5 |
| 245 | maxItems: 5 | 246 | maxItems: 5 |
| 246 | clocks-names: | 247 | clock-names: |
| 247 | minItems: 5 | 248 | minItems: 5 |
| 248 | maxItems: 5 | 249 | maxItems: 5 |
| 249 | 250 | ||
| @@ -264,7 +265,7 @@ allOf: · 14 unmodified lines | |||
| 264 | clocks: | 265 | clocks: |
| 265 | minItems: 5 | 266 | minItems: 5 |
| 266 | maxItems: 6 | 267 | maxItems: 6 |
| 267 | clocks-names: | 268 | clock-names: |
| 268 | minItems: 5 | 269 | minItems: 5 |
| 269 | maxItems: 6 | 270 | maxItems: 6 |
| 270 | 271 | ||
| @@ -277,7 +278,6 @@ allOf: · 6 unmodified lines | |||
| 277 | - qcom,sc8180x-dp | 278 | - qcom,sc8180x-dp |
| 278 | - qcom,sdm845-dp | 279 | - qcom,sdm845-dp |
| 279 | - qcom,sm8350-dp | 280 | - qcom,sm8350-dp |
| 280 | - qcom,sm8650-dp | ||
| 281 | then: | 281 | then: |
| 282 | properties: | 282 | properties: |
| 283 | reg: | 283 | reg: |
| @@ -286,6 +286,24 @@ allOf: · 2 unmodified lines | |||
| 286 | clocks: | 286 | clocks: |
| 287 | minItems: 6 | 287 | minItems: 6 |
| 288 | maxItems: 6 | 288 | maxItems: 6 |
| 289 | clock-names: | ||
| 290 | minItems: 6 | ||
| 291 | maxItems: 6 | ||
| 292 | |||
| 293 | - if: | ||
| 294 | properties: | ||
| 295 | compatible: | ||
| 296 | contains: | ||
| 297 | enum: | ||
| 298 | - qcom,sm8650-dp | ||
| 299 | then: | ||
| 300 | properties: | ||
| 301 | reg: | ||
| 302 | minItems: 5 | ||
| 303 | maxItems: 9 | ||
| 304 | clocks: | ||
| 305 | minItems: 6 | ||
| 306 | maxItems: 6 | ||
| 289 | clocks-names: | 307 | clocks-names: |
| 290 | minItems: 6 | 308 | minItems: 6 |
| 291 | maxItems: 6 | 309 | maxItems: 6 |
| @@ -306,7 +324,7 @@ allOf: · 14 unmodified lines | |||
| 306 | clocks: | 324 | clocks: |
| 307 | minItems: 6 | 325 | minItems: 6 |
| 308 | maxItems: 8 | 326 | maxItems: 8 |
| 309 | clocks-names: | 327 | clock-names: |
| 310 | minItems: 6 | 328 | minItems: 6 |
| 311 | maxItems: 8 | 329 | maxItems: 8 |
| 312 | 330 | ||
| @@ -326,7 +344,7 @@ allOf: · 13 unmodified lines | |||
| 326 | clocks: | 344 | clocks: |
| 327 | minItems: 5 | 345 | minItems: 5 |
| 328 | maxItems: 6 | 346 | maxItems: 6 |
| 329 | clocks-names: | 347 | clock-names: |
| 330 | minItems: 5 | 348 | minItems: 5 |
| 331 | maxItems: 6 | 349 | maxItems: 6 |
| 332 | 350 | ||
| @@ -119,7 +119,7 @@ examples: · 118 unmodified lines | |||
| 119 | mdss_mdp: display-controller@ae01000 { | 119 | mdss_mdp: display-controller@ae01000 { |
| 120 | compatible = "qcom,eliza-dpu"; | 120 | compatible = "qcom,eliza-dpu"; |
| 121 | reg = <0x0ae01000 0x93000>, | 121 | reg = <0x0ae01000 0x93000>, |
| 122 | <0x0aeb0000 0x2008>; | 122 | <0x0aeb0000 0x3000>; |
| 123 | reg-names = "mdp", | 123 | reg-names = "mdp", |
| 124 | "vbif"; | 124 | "vbif"; |
| 125 | 125 | ||
| @@ -304,7 +304,7 @@ examples: · 178 unmodified lines | |||
| 304 | mdss_dsi0_phy: phy@ae95000 { | 304 | mdss_dsi0_phy: phy@ae95000 { |
| 305 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; | 305 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; |
| 306 | reg = <0x0ae95000 0x200>, | 306 | reg = <0x0ae95000 0x200>, |
| 307 | <0x0ae95200 0x280>, | 307 | <0x0ae95200 0x300>, |
| 308 | <0x0ae95500 0x400>; | 308 | <0x0ae95500 0x400>; |
| 309 | reg-names = "dsi_phy", | 309 | reg-names = "dsi_phy", |
| 310 | "dsi_phy_lane", | 310 | "dsi_phy_lane", |
| @@ -388,7 +388,7 @@ examples: · 77 unmodified lines | |||
| 388 | mdss_dsi1_phy: phy@ae97000 { | 388 | mdss_dsi1_phy: phy@ae97000 { |
| 389 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; | 389 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; |
| 390 | reg = <0x0ae97000 0x200>, | 390 | reg = <0x0ae97000 0x200>, |
| 391 | <0x0ae97200 0x280>, | 391 | <0x0ae97200 0x300>, |
| 392 | <0x0ae97500 0x400>; | 392 | <0x0ae97500 0x400>; |
| 393 | reg-names = "dsi_phy", | 393 | reg-names = "dsi_phy", |
| 394 | "dsi_phy_lane", | 394 | "dsi_phy_lane", |
| @@ -407,11 +407,15 @@ examples: · 12 unmodified lines | |||
| 407 | 407 | ||
| 408 | displayport-controller@af54000 { | 408 | displayport-controller@af54000 { |
| 409 | compatible = "qcom,eliza-dp", "qcom,sm8650-dp"; | 409 | compatible = "qcom,eliza-dp", "qcom,sm8650-dp"; |
| 410 | reg = <0xaf54000 0x104>, | 410 | reg = <0x0af54000 0x200>, |
| 411 | <0xaf54200 0xc0>, | 411 | <0x0af54200 0x200>, |
| 412 | <0xaf55000 0x770>, | 412 | <0x0af55000 0xc00>, |
| 413 | <0xaf56000 0x9c>, | 413 | <0x0af56000 0x400>, |
| 414 | <0xaf57000 0x9c>; | 414 | <0x0af57000 0x400>, |
| 415 | <0x0af58000 0x400>, | ||
| 416 | <0x0af59000 0x400>, | ||
| 417 | <0x0af5a000 0x600>, | ||
| 418 | <0x0af5b000 0x600>; | ||
| 415 | 419 | ||
| 416 | interrupts-extended = <&mdss 12>; | 420 | interrupts-extended = <&mdss 12>; |
| 417 | 421 | ||
| @@ -70,7 +70,7 @@ examples: · 69 unmodified lines | |||
| 70 | display-controller@ae01000 { | 70 | display-controller@ae01000 { |
| 71 | compatible = "qcom,sm8650-dpu"; | 71 | compatible = "qcom,sm8650-dpu"; |
| 72 | reg = <0x0ae01000 0x8f000>, | 72 | reg = <0x0ae01000 0x8f000>, |
| 73 | <0x0aeb0000 0x2008>; | 73 | <0x0aeb0000 0x3000>; |
| 74 | reg-names = "mdp", "vbif"; | 74 | reg-names = "mdp", "vbif"; |
| 75 | 75 | ||
| 76 | clocks = <&gcc_axi_clk>, | 76 | clocks = <&gcc_axi_clk>, |
| @@ -112,7 +112,7 @@ examples: · 111 unmodified lines | |||
| 112 | display-controller@ae01000 { | 112 | display-controller@ae01000 { |
| 113 | compatible = "qcom,sm8650-dpu"; | 113 | compatible = "qcom,sm8650-dpu"; |
| 114 | reg = <0x0ae01000 0x8f000>, | 114 | reg = <0x0ae01000 0x8f000>, |
| 115 | <0x0aeb0000 0x2008>; | 115 | <0x0aeb0000 0x3000>; |
| 116 | reg-names = "mdp", "vbif"; | 116 | reg-names = "mdp", "vbif"; |
| 117 | 117 | ||
| 118 | clocks = <&gcc_axi_clk>, | 118 | clocks = <&gcc_axi_clk>, |
| @@ -117,7 +117,7 @@ examples: · 116 unmodified lines | |||
| 117 | display-controller@ae01000 { | 117 | display-controller@ae01000 { |
| 118 | compatible = "qcom,sm8750-dpu"; | 118 | compatible = "qcom,sm8750-dpu"; |
| 119 | reg = <0x0ae01000 0x93000>, | 119 | reg = <0x0ae01000 0x93000>, |
| 120 | <0x0aeb0000 0x2008>; | 120 | <0x0aeb0000 0x3000>; |
| 121 | reg-names = "mdp", | 121 | reg-names = "mdp", |
| 122 | "vbif"; | 122 | "vbif"; |
| 123 | 123 | ||
| @@ -389,11 +389,15 @@ examples: · 265 unmodified lines | |||
| 389 | 389 | ||
| 390 | displayport-controller@af54000 { | 390 | displayport-controller@af54000 { |
| 391 | compatible = "qcom,sm8750-dp", "qcom,sm8650-dp"; | 391 | compatible = "qcom,sm8750-dp", "qcom,sm8650-dp"; |
| 392 | reg = <0xaf54000 0x104>, | 392 | reg = <0x0af54000 0x200>, |
| 393 | <0xaf54200 0xc0>, | 393 | <0x0af54200 0x200>, |
| 394 | <0xaf55000 0x770>, | 394 | <0x0af55000 0xc00>, |
| 395 | <0xaf56000 0x9c>, | 395 | <0x0af56000 0x400>, |
| 396 | <0xaf57000 0x9c>; | 396 | <0x0af57000 0x400>, |
| 397 | <0x0af58000 0x400>, | ||
| 398 | <0x0af59000 0x400>, | ||
| 399 | <0x0af5a000 0x600>, | ||
| 400 | <0x0af5b000 0x600>; | ||
| 397 | 401 | ||
| 398 | interrupts-extended = <&mdss 12>; | 402 | interrupts-extended = <&mdss 12>; |
| 399 | 403 | ||
| @@ -16,10 +16,15 @@ allOf: · 15 unmodified lines | |||
| 16 | 16 | ||
| 17 | properties: | 17 | properties: |
| 18 | compatible: | 18 | compatible: |
| 19 | enum: | 19 | oneOf: |
| 20 | - amlogic,meson6-i2c # Meson6, Meson8 and compatible SoCs | 20 | - items: |
| 21 | - amlogic,meson-gxbb-i2c # GXBB and compatible SoCs | 21 | - enum: |
| 22 | - amlogic,meson-axg-i2c # AXG and compatible SoCs | 22 | - amlogic,t7-i2c |
| 23 | - const: amlogic,meson-axg-i2c | ||
| 24 | - enum: | ||
| 25 | - amlogic,meson6-i2c # Meson6, Meson8 and compatible SoCs | ||
| 26 | - amlogic,meson-gxbb-i2c # GXBB and compatible SoCs | ||
| 27 | - amlogic,meson-axg-i2c # AXG and compatible SoCs | ||
| 23 | 28 | ||
| 24 | reg: | 29 | reg: |
| 25 | maxItems: 1 | 30 | maxItems: 1 |
| @@ -22,7 +22,9 @@ properties: · 21 unmodified lines | |||
| 22 | compatible: | 22 | compatible: |
| 23 | oneOf: | 23 | oneOf: |
| 24 | - items: | 24 | - items: |
| 25 | - const: apple,t6020-i2c | 25 | - enum: |
| 26 | - apple,t6020-i2c | ||
| 27 | - apple,t8122-i2c | ||
| 26 | - const: apple,t8103-i2c | 28 | - const: apple,t8103-i2c |
| 27 | - items: | 29 | - items: |
| 28 | - enum: | 30 | - enum: |
| @@ -24,6 +24,7 @@ properties: · 23 unmodified lines | |||
| 24 | compatible: | 24 | compatible: |
| 25 | items: | 25 | items: |
| 26 | - enum: | 26 | - enum: |
| 27 | - qcom,eliza-ipcc | ||
| 27 | - qcom,glymur-ipcc | 28 | - qcom,glymur-ipcc |
| 28 | - qcom,kaanapali-ipcc | 29 | - qcom,kaanapali-ipcc |
| 29 | - qcom,milos-ipcc | 30 | - qcom,milos-ipcc |
| @@ -73,6 +73,15 @@ properties: · 72 unmodified lines | |||
| 73 | HSP CSR is to control and get status of different high-speed peripherals | 73 | HSP CSR is to control and get status of different high-speed peripherals |
| 74 | (such as Ethernet, USB, SATA, etc.) via register, which can tune | 74 | (such as Ethernet, USB, SATA, etc.) via register, which can tune |
| 75 | board-level's parameters of PHY, etc. | 75 | board-level's parameters of PHY, etc. |
| 76 | |||
| 77 | Additional background information about the High-Speed Subsystem | ||
| 78 | and the HSP CSR block is available in Chapter 10 ("High-Speed Interface") | ||
| 79 | of the EIC7700X SoC Technical Reference Manual, Part 4 | ||
| 80 | (EIC7700X_SoC_Technical_Reference_Manual_Part4.pdf). The manual is | ||
| 81 | publicly available at | ||
| 82 | https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual/releases | ||
| 83 | |||
| 84 | This reference is provided for background information only. | ||
| 76 | $ref: /schemas/types.yaml#/definitions/phandle-array | 85 | $ref: /schemas/types.yaml#/definitions/phandle-array |
| 77 | items: | 86 | items: |
| 78 | - items: | 87 | - items: |
| @@ -82,6 +91,8 @@ properties: · 3 unmodified lines | |||
| 82 | - description: Offset of AXI clock controller Low-Power request | 91 | - description: Offset of AXI clock controller Low-Power request |
| 83 | register | 92 | register |
| 84 | - description: Offset of register controlling TX/RX clock delay | 93 | - description: Offset of register controlling TX/RX clock delay |
| 94 | - description: Optional offset of register controlling TXD delay | ||
| 95 | - description: Optional offset of register controlling RXD delay | ||
| 85 | 96 | ||
| 86 | required: | 97 | required: |
| 87 | - compatible | 98 | - compatible |
| @@ -116,7 +127,7 @@ examples: · 28 unmodified lines | |||
| 116 | reset-names = "stmmaceth"; | 127 | reset-names = "stmmaceth"; |
| 117 | rx-internal-delay-ps = <200>; | 128 | rx-internal-delay-ps = <200>; |
| 118 | tx-internal-delay-ps = <200>; | 129 | tx-internal-delay-ps = <200>; |
| 119 | eswin,hsp-sp-csr = <&hsp_sp_csr 0x100 0x108 0x118>; | 130 | eswin,hsp-sp-csr = <&hsp_sp_csr 0x100 0x108 0x118 0x114 0x11c>; |
| 120 | snps,axi-config = <&stmmac_axi_setup>; | 131 | snps,axi-config = <&stmmac_axi_setup>; |
| 121 | snps,aal; | 132 | snps,aal; |
| 122 | snps,fixed-burst; | 133 | snps,fixed-burst; |
| @@ -18,7 +18,9 @@ properties: · 17 unmodified lines | |||
| 18 | description: Phandles of rt5650 and rt5514 codecs | 18 | description: Phandles of rt5650 and rt5514 codecs |
| 19 | items: | 19 | items: |
| 20 | - description: phandle of rt5650 codec | 20 | - description: phandle of rt5650 codec |
| 21 | maxItems: 1 | ||
| 21 | - description: phandle of rt5514 codec | 22 | - description: phandle of rt5514 codec |
| 23 | maxItems: 1 | ||
| 22 | 24 | ||
| 23 | mediatek,platform: | 25 | mediatek,platform: |
| 24 | $ref: /schemas/types.yaml#/definitions/phandle | 26 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -20,6 +20,9 @@ properties: · 19 unmodified lines | |||
| 20 | - fsl,ls1021a-qspi | 20 | - fsl,ls1021a-qspi |
| 21 | - fsl,ls2080a-qspi | 21 | - fsl,ls2080a-qspi |
| 22 | - spacemit,k1-qspi | 22 | - spacemit,k1-qspi |
| 23 | - items: | ||
| 24 | - const: spacemit,k3-qspi | ||
| 25 | - const: spacemit,k1-qspi | ||
| 23 | - items: | 26 | - items: |
| 24 | - enum: | 27 | - enum: |
| 25 | - fsl,ls1043a-qspi | 28 | - fsl,ls1043a-qspi |
| @@ -25,6 +25,7 @@ properties: · 24 unmodified lines | |||
| 25 | - items: | 25 | - items: |
| 26 | - enum: | 26 | - enum: |
| 27 | - qcom,ipq5018-snand | 27 | - qcom,ipq5018-snand |
| 28 | - qcom,ipq5210-snand | ||
| 28 | - qcom,ipq5332-snand | 29 | - qcom,ipq5332-snand |
| 29 | - qcom,ipq5424-snand | 30 | - qcom,ipq5424-snand |
| 30 | - const: qcom,ipq9574-snand | 31 | - const: qcom,ipq9574-snand |
| @@ -41,12 +41,13 @@ properties: · 40 unmodified lines | |||
| 41 | - const: usb_en | 41 | - const: usb_en |
| 42 | 42 | ||
| 43 | resets: | 43 | resets: |
| 44 | maxItems: 2 | 44 | maxItems: 3 |
| 45 | 45 | ||
| 46 | reset-names: | 46 | reset-names: |
| 47 | items: | 47 | items: |
| 48 | - const: vaux | 48 | - const: vaux |
| 49 | - const: usb_rst | 49 | - const: usb_rst |
| 50 | - const: usb_phy | ||
| 50 | 51 | ||
| 51 | eswin,hsp-sp-csr: | 52 | eswin,hsp-sp-csr: |
| 52 | description: | 53 | description: |
| @@ -85,8 +86,8 @@ examples: · 32 unmodified lines | |||
| 85 | interrupt-parent = <&plic>; | 86 | interrupt-parent = <&plic>; |
| 86 | interrupts = <85>; | 87 | interrupts = <85>; |
| 87 | interrupt-names = "peripheral"; | 88 | interrupt-names = "peripheral"; |
| 88 | resets = <&reset 84>, <&hspcrg 2>; | 89 | resets = <&reset 84>, <&hspcrg 2>, <&hspcrg 4>; |
| 89 | reset-names = "vaux", "usb_rst"; | 90 | reset-names = "vaux", "usb_rst", "usb_phy"; |
| 90 | dr_mode = "peripheral"; | 91 | dr_mode = "peripheral"; |
| 91 | maximum-speed = "high-speed"; | 92 | maximum-speed = "high-speed"; |
| 92 | phy_type = "utmi"; | 93 | phy_type = "utmi"; |
| @@ -81,9 +81,7 @@ properties: · 80 unmodified lines | |||
| 81 | const: usb2-phy | 81 | const: usb2-phy |
| 82 | 82 | ||
| 83 | usb-phy: | 83 | usb-phy: |
| 84 | $ref: /schemas/types.yaml#/definitions/phandle-array | 84 | maxItems: 1 |
| 85 | description: Phandle for the PHY device. | ||
| 86 | deprecated: true | ||
| 87 | 85 | ||
| 88 | ctrl-module: | 86 | ctrl-module: |
| 89 | $ref: /schemas/types.yaml#/definitions/phandle | 87 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -96,6 +94,9 @@ required: · 6 unmodified lines | |||
| 96 | - interrupts | 94 | - interrupts |
| 97 | - interrupt-names | 95 | - interrupt-names |
| 98 | 96 | ||
| 97 | allOf: | ||
| 98 | - $ref: usb-hcd.yaml# | ||
| 99 | |||
| 99 | unevaluatedProperties: false | 100 | unevaluatedProperties: false |
| 100 | 101 | ||
| 101 | examples: | 102 | examples: |
| @@ -57,7 +57,7 @@ Mount options unique to the isofs filesystem. · 56 unmodified lines | |||
| 57 | Recommended documents about ISO 9660 standard are located at: | 57 | Recommended documents about ISO 9660 standard are located at: |
| 58 | 58 | ||
| 59 | - http://www.y-adagio.com/ | 59 | - http://www.y-adagio.com/ |
| 60 | - ftp://ftp.ecma.ch/ecma-st/Ecma-119.pdf | 60 | - https://ecma-international.org/wp-content/uploads/ECMA-119_2nd_edition_december_1987.pdf |
| 61 | 61 | ||
| 62 | Quoting from the PDF "This 2nd Edition of Standard ECMA-119 is technically | 62 | Quoting from the PDF "This 2nd Edition of Standard ECMA-119 is technically |
| 63 | identical with ISO 9660.", so it is a valid and gratis substitute of the | 63 | identical with ISO 9660.", so it is a valid and gratis substitute of the |
| @@ -22,5 +22,5 @@ The following sensors are supported · 21 unmodified lines | |||
| 22 | sysfs-Interface | 22 | sysfs-Interface |
| 23 | --------------- | 23 | --------------- |
| 24 | 24 | ||
| 25 | temp0_input | 25 | temp1_input |
| 26 | - Temperature of external NTC (milli-degree C) | 26 | - Temperature of external NTC (milli-degree C) |
| @@ -135,4 +135,4 @@ References · 134 unmodified lines | |||
| 135 | 135 | ||
| 136 | 4. **Lenovo IdeaPad Laptop Driver:** Reference for DMI-based hardware | 136 | 4. **Lenovo IdeaPad Laptop Driver:** Reference for DMI-based hardware |
| 137 | feature gating in Lenovo laptops. | 137 | feature gating in Lenovo laptops. |
| 138 | https://github.com/torvalds/linux/blob/master/drivers/platform/x86/ideapad-laptop.c | 138 | https://github.com/torvalds/linux/blob/master/drivers/platform/x86/lenovo/ideapad-laptop.c |
| @@ -69,6 +69,15 @@ properties: · 68 unmodified lines | |||
| 69 | header: | 69 | header: |
| 70 | description: For C-compatible languages, header which already defines this value. | 70 | description: For C-compatible languages, header which already defines this value. |
| 71 | type: string | 71 | type: string |
| 72 | scope: | ||
| 73 | description: | | ||
| 74 | Visibility of this definition. "uapi" (default) renders into | ||
| 75 | the uAPI header, "kernel" renders into the kernel-side | ||
| 76 | generated header, "user" renders into the user-side | ||
| 77 | generated header. When combined with `header:`, the | ||
| 78 | definition is not rendered, and the named header is | ||
| 79 | included only by code matching the scope. | ||
| 80 | enum: [ uapi, kernel, user ] | ||
| 72 | type: | 81 | type: |
| 73 | enum: [ const, enum, flags ] | 82 | enum: [ const, enum, flags ] |
| 74 | doc: | 83 | doc: |
| @@ -83,6 +83,15 @@ properties: · 82 unmodified lines | |||
| 83 | header: | 83 | header: |
| 84 | description: For C-compatible languages, header which already defines this value. | 84 | description: For C-compatible languages, header which already defines this value. |
| 85 | type: string | 85 | type: string |
| 86 | scope: | ||
| 87 | description: | | ||
| 88 | Visibility of this definition. "uapi" (default) renders into | ||
| 89 | the uAPI header, "kernel" renders into the kernel-side | ||
| 90 | generated header, "user" renders into the user-side | ||
| 91 | generated header. When combined with `header:`, the | ||
| 92 | definition is not rendered, and the named header is | ||
| 93 | included only by code matching the scope. | ||
| 94 | enum: [ uapi, kernel, user ] | ||
| 86 | type: | 95 | type: |
| 87 | enum: [ const, enum, flags, struct ] # Trim | 96 | enum: [ const, enum, flags, struct ] # Trim |
| 88 | doc: | 97 | doc: |
| @@ -55,6 +55,15 @@ properties: · 54 unmodified lines | |||
| 55 | header: | 55 | header: |
| 56 | description: For C-compatible languages, header which already defines this value. | 56 | description: For C-compatible languages, header which already defines this value. |
| 57 | type: string | 57 | type: string |
| 58 | scope: | ||
| 59 | description: | | ||
| 60 | Visibility of this definition. "uapi" (default) renders into | ||
| 61 | the uAPI header, "kernel" renders into the kernel-side | ||
| 62 | generated header, "user" renders into the user-side | ||
| 63 | generated header. When combined with `header:`, the | ||
| 64 | definition is not rendered, and the named header is | ||
| 65 | included only by code matching the scope. | ||
| 66 | enum: [ uapi, kernel, user ] | ||
| 58 | type: | 67 | type: |
| 59 | enum: [ const, enum, flags ] | 68 | enum: [ const, enum, flags ] |
| 60 | doc: | 69 | doc: |
| @@ -87,6 +87,15 @@ properties: · 86 unmodified lines | |||
| 87 | header: | 87 | header: |
| 88 | description: For C-compatible languages, header which already defines this value. | 88 | description: For C-compatible languages, header which already defines this value. |
| 89 | type: string | 89 | type: string |
| 90 | scope: | ||
| 91 | description: | | ||
| 92 | Visibility of this definition. "uapi" (default) renders into | ||
| 93 | the uAPI header, "kernel" renders into the kernel-side | ||
| 94 | generated header, "user" renders into the user-side | ||
| 95 | generated header. When combined with `header:`, the | ||
| 96 | definition is not rendered, and the named header is | ||
| 97 | included only by code matching the scope. | ||
| 98 | enum: [ uapi, kernel, user ] | ||
| 90 | type: | 99 | type: |
| 91 | enum: [ const, enum, flags, struct ] # Trim | 100 | enum: [ const, enum, flags, struct ] # Trim |
| 92 | doc: | 101 | doc: |
| @@ -12,6 +12,12 @@ protocol: genetlink · 11 unmodified lines | |||
| 12 | doc: Netlink protocol to request a transport layer security handshake. | 12 | doc: Netlink protocol to request a transport layer security handshake. |
| 13 | 13 | ||
| 14 | definitions: | 14 | definitions: |
| 15 | - | ||
| 16 | type: const | ||
| 17 | name: max-errno | ||
| 18 | value: 4095 | ||
| 19 | header: linux/err.h | ||
| 20 | scope: kernel | ||
| 15 | - | 21 | - |
| 16 | type: enum | 22 | type: enum |
| 17 | name: handler-class | 23 | name: handler-class |
| @@ -80,6 +86,8 @@ attribute-sets: · 62 unmodified lines | |||
| 80 | - | 86 | - |
| 81 | name: status | 87 | name: status |
| 82 | type: u32 | 88 | type: u32 |
| 89 | checks: | ||
| 90 | max: max-errno | ||
| 83 | - | 91 | - |
| 84 | name: sockfd | 92 | name: sockfd |
| 85 | type: s32 | 93 | type: s32 |
| @@ -33,6 +33,11 @@ doc: | · 32 unmodified lines | |||
| 33 | @cap-get operation. | 33 | @cap-get operation. |
| 34 | 34 | ||
| 35 | definitions: | 35 | definitions: |
| 36 | - | ||
| 37 | type: const | ||
| 38 | name: max-handle-id | ||
| 39 | value: 0x3fffffe | ||
| 40 | scope: kernel | ||
| 36 | - | 41 | - |
| 37 | type: enum | 42 | type: enum |
| 38 | name: scope | 43 | name: scope |
| @@ -140,6 +145,8 @@ attribute-sets: · 101 unmodified lines | |||
| 140 | - | 145 | - |
| 141 | name: id | 146 | name: id |
| 142 | type: u32 | 147 | type: u32 |
| 148 | checks: | ||
| 149 | max: max-handle-id | ||
| 143 | doc: | | 150 | doc: | |
| 144 | Numeric identifier of a shaper. The id semantic depends on | 151 | Numeric identifier of a shaper. The id semantic depends on |
| 145 | the scope. For @queue scope it's the queue id and for @node | 152 | the scope. For @queue scope it's the queue id and for @node |
| @@ -188,6 +188,7 @@ operations: · 187 unmodified lines | |||
| 188 | name: dev-set | 188 | name: dev-set |
| 189 | doc: Set the configuration of a PSP device. | 189 | doc: Set the configuration of a PSP device. |
| 190 | attribute-set: dev | 190 | attribute-set: dev |
| 191 | flags: [admin-perm] | ||
| 191 | do: | 192 | do: |
| 192 | request: | 193 | request: |
| 193 | attributes: | 194 | attributes: |
| @@ -207,6 +208,7 @@ operations: · 13 unmodified lines | |||
| 207 | name: key-rotate | 208 | name: key-rotate |
| 208 | doc: Rotate the device key. | 209 | doc: Rotate the device key. |
| 209 | attribute-set: dev | 210 | attribute-set: dev |
| 211 | flags: [admin-perm] | ||
| 210 | do: | 212 | do: |
| 211 | request: | 213 | request: |
| 212 | attributes: | 214 | attributes: |
| @@ -0,0 +1,249 @@ | |||
| 1 | .. SPDX-License-Identifier: GPL-2.0 | ||
| 2 | |||
| 3 | ============================================================================= | ||
| 4 | Linux and the 3Com EtherLink III Series Ethercards (driver v1.18c and higher) | ||
| 5 | ============================================================================= | ||
| 6 | |||
| 7 | This file contains the instructions and caveats for v1.18c and higher versions | ||
| 8 | of the 3c509 driver. You should not use the driver without reading this file. | ||
| 9 | |||
| 10 | release 1.0 | ||
| 11 | |||
| 12 | 28 February 2002 | ||
| 13 | |||
| 14 | Current maintainer (corrections to): | ||
| 15 | Maciej W. Rozycki <macro@orcam.me.uk> | ||
| 16 | |||
| 17 | Introduction | ||
| 18 | ============ | ||
| 19 | |||
| 20 | The following are notes and information on using the 3Com EtherLink III series | ||
| 21 | ethercards in Linux. These cards are commonly known by the most widely-used | ||
| 22 | card's 3Com model number, 3c509. They are all 10mb/s ISA-bus cards and shouldn't | ||
| 23 | be (but sometimes are) confused with the similarly-numbered PCI-bus "3c905" | ||
| 24 | (aka "Vortex" or "Boomerang") series. Kernel support for the 3c509 family is | ||
| 25 | provided by the module 3c509.c, which has code to support all of the following | ||
| 26 | models: | ||
| 27 | |||
| 28 | - 3c509 (original ISA card) | ||
| 29 | - 3c509B (later revision of the ISA card; supports full-duplex) | ||
| 30 | - 3c589 (PCMCIA) | ||
| 31 | - 3c589B (later revision of the 3c589; supports full-duplex) | ||
| 32 | - 3c579 (EISA) | ||
| 33 | |||
| 34 | Large portions of this documentation were heavily borrowed from the guide | ||
| 35 | written the original author of the 3c509 driver, Donald Becker. The master | ||
| 36 | copy of that document, which contains notes on older versions of the driver, | ||
| 37 | currently resides on Scyld web server: http://www.scyld.com/. | ||
| 38 | |||
| 39 | |||
| 40 | Special Driver Features | ||
| 41 | ======================= | ||
| 42 | |||
| 43 | Overriding card settings | ||
| 44 | |||
| 45 | The driver allows boot- or load-time overriding of the card's detected IOADDR, | ||
| 46 | IRQ, and transceiver settings, although this capability shouldn't generally be | ||
| 47 | needed except to enable full-duplex mode (see below). An example of the syntax | ||
| 48 | for LILO parameters for doing this:: | ||
| 49 | |||
| 50 | ether=10,0x310,3,0x3c509,eth0 | ||
| 51 | |||
| 52 | This configures the first found 3c509 card for IRQ 10, base I/O 0x310, and | ||
| 53 | transceiver type 3 (10base2). The flag "0x3c509" must be set to avoid conflicts | ||
| 54 | with other card types when overriding the I/O address. When the driver is | ||
| 55 | loaded as a module, only the IRQ may be overridden. For example, | ||
| 56 | setting two cards to IRQ10 and IRQ11 is done by using the irq module | ||
| 57 | option:: | ||
| 58 | |||
| 59 | options 3c509 irq=10,11 | ||
| 60 | |||
| 61 | |||
| 62 | Full-duplex mode | ||
| 63 | ================ | ||
| 64 | |||
| 65 | The v1.18c driver added support for the 3c509B's full-duplex capabilities. | ||
| 66 | In order to enable and successfully use full-duplex mode, three conditions | ||
| 67 | must be met: | ||
| 68 | |||
| 69 | (a) You must have a Etherlink III card model whose hardware supports full- | ||
| 70 | duplex operations. Currently, the only members of the 3c509 family that are | ||
| 71 | positively known to support full-duplex are the 3c509B (ISA bus) and 3c589B | ||
| 72 | (PCMCIA) cards. Cards without the "B" model designation do *not* support | ||
| 73 | full-duplex mode; these include the original 3c509 (no "B"), the original | ||
| 74 | 3c589, the 3c529 (MCA bus), and the 3c579 (EISA bus). | ||
| 75 | |||
| 76 | (b) You must be using your card's 10baseT transceiver (i.e., the RJ-45 | ||
| 77 | connector), not its AUI (thick-net) or 10base2 (thin-net/coax) interfaces. | ||
| 78 | AUI and 10base2 network cabling is physically incapable of full-duplex | ||
| 79 | operation. | ||
| 80 | |||
| 81 | (c) Most importantly, your 3c509B must be connected to a link partner that is | ||
| 82 | itself full-duplex capable. This is almost certainly one of two things: a full- | ||
| 83 | duplex-capable Ethernet switch (*not* a hub), or a full-duplex-capable NIC on | ||
| 84 | another system that's connected directly to the 3c509B via a crossover cable. | ||
| 85 | |||
| 86 | Full-duplex mode can be enabled using 'ethtool'. | ||
| 87 | |||
| 88 | .. warning:: | ||
| 89 | |||
| 90 | Extremely important caution concerning full-duplex mode | ||
| 91 | |||
| 92 | Understand that the 3c509B's hardware's full-duplex support is much more | ||
| 93 | limited than that provide by more modern network interface cards. Although | ||
| 94 | at the physical layer of the network it fully supports full-duplex operation, | ||
| 95 | the card was designed before the current Ethernet auto-negotiation (N-way) | ||
| 96 | spec was written. This means that the 3c509B family ***cannot and will not | ||
| 97 | auto-negotiate a full-duplex connection with its link partner under any | ||
| 98 | circumstances, no matter how it is initialized***. If the full-duplex mode | ||
| 99 | of the 3c509B is enabled, its link partner will very likely need to be | ||
| 100 | independently _forced_ into full-duplex mode as well; otherwise various nasty | ||
| 101 | failures will occur - at the very least, you'll see massive numbers of packet | ||
| 102 | collisions. This is one of very rare circumstances where disabling auto- | ||
| 103 | negotiation and forcing the duplex mode of a network interface card or switch | ||
| 104 | would ever be necessary or desirable. | ||
| 105 | |||
| 106 | |||
| 107 | Available Transceiver Types | ||
| 108 | =========================== | ||
| 109 | |||
| 110 | For versions of the driver v1.18c and above, the available transceiver types are: | ||
| 111 | |||
| 112 | == ========================================================================= | ||
| 113 | 0 transceiver type from EEPROM config (normally 10baseT); force half-duplex | ||
| 114 | 1 AUI (thick-net / DB15 connector) | ||
| 115 | 2 (undefined) | ||
| 116 | 3 10base2 (thin-net == coax / BNC connector) | ||
| 117 | 4 10baseT (RJ-45 connector); force half-duplex mode | ||
| 118 | 8 transceiver type and duplex mode taken from card's EEPROM config settings | ||
| 119 | 12 10baseT (RJ-45 connector); force full-duplex mode | ||
| 120 | == ========================================================================= | ||
| 121 | |||
| 122 | Prior to driver version 1.18c, only transceiver codes 0-4 were supported. Note | ||
| 123 | that the new transceiver codes 8 and 12 are the *only* ones that will enable | ||
| 124 | full-duplex mode, no matter what the card's detected EEPROM settings might be. | ||
| 125 | This insured that merely upgrading the driver from an earlier version would | ||
| 126 | never automatically enable full-duplex mode in an existing installation; | ||
| 127 | it must always be explicitly enabled via one of these code in order to be | ||
| 128 | activated. | ||
| 129 | |||
| 130 | The transceiver type can be changed using 'ethtool'. | ||
| 131 | |||
| 132 | |||
| 133 | Interpretation of error messages and common problems | ||
| 134 | ---------------------------------------------------- | ||
| 135 | |||
| 136 | Error Messages | ||
| 137 | ^^^^^^^^^^^^^^ | ||
| 138 | |||
| 139 | eth0: Infinite loop in interrupt, status 2011. | ||
| 140 | These are "mostly harmless" message indicating that the driver had too much | ||
| 141 | work during that interrupt cycle. With a status of 0x2011 you are receiving | ||
| 142 | packets faster than they can be removed from the card. This should be rare | ||
| 143 | or impossible in normal operation. Possible causes of this error report are: | ||
| 144 | |||
| 145 | - a "green" mode enabled that slows the processor down when there is no | ||
| 146 | keyboard activity. | ||
| 147 | |||
| 148 | - some other device or device driver hogging the bus or disabling interrupts. | ||
| 149 | Check /proc/interrupts for excessive interrupt counts. The timer tick | ||
| 150 | interrupt should always be incrementing faster than the others. | ||
| 151 | |||
| 152 | No received packets | ||
| 153 | ^^^^^^^^^^^^^^^^^^^ | ||
| 154 | |||
| 155 | If a 3c509, 3c562 or 3c589 can successfully transmit packets, but never | ||
| 156 | receives packets (as reported by /proc/net/dev or 'ifconfig') you likely | ||
| 157 | have an interrupt line problem. Check /proc/interrupts to verify that the | ||
| 158 | card is actually generating interrupts. If the interrupt count is not | ||
| 159 | increasing you likely have a physical conflict with two devices trying to | ||
| 160 | use the same ISA IRQ line. The common conflict is with a sound card on IRQ10 | ||
| 161 | or IRQ5, and the easiest solution is to move the 3c509 to a different | ||
| 162 | interrupt line. If the device is receiving packets but 'ping' doesn't work, | ||
| 163 | you have a routing problem. | ||
| 164 | |||
| 165 | Tx Carrier Errors Reported in /proc/net/dev | ||
| 166 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 167 | |||
| 168 | |||
| 169 | If an EtherLink III appears to transmit packets, but the "Tx carrier errors" | ||
| 170 | field in /proc/net/dev increments as quickly as the Tx packet count, you | ||
| 171 | likely have an unterminated network or the incorrect media transceiver selected. | ||
| 172 | |||
| 173 | 3c509B card is not detected on machines with an ISA PnP BIOS. | ||
| 174 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 175 | |||
| 176 | While the updated driver works with most PnP BIOS programs, it does not work | ||
| 177 | with all. This can be fixed by disabling PnP support using the 3Com-supplied | ||
| 178 | setup program. | ||
| 179 | |||
| 180 | 3c509 card is not detected on overclocked machines | ||
| 181 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 182 | |||
| 183 | Increase the delay time in id_read_eeprom() from the current value, 500, | ||
| 184 | to an absurdly high value, such as 5000. | ||
| 185 | |||
| 186 | |||
| 187 | Decoding Status and Error Messages | ||
| 188 | ---------------------------------- | ||
| 189 | |||
| 190 | |||
| 191 | The bits in the main status register are: | ||
| 192 | |||
| 193 | ===== ====================================== | ||
| 194 | value description | ||
| 195 | ===== ====================================== | ||
| 196 | 0x01 Interrupt latch | ||
| 197 | 0x02 Tx overrun, or Rx underrun | ||
| 198 | 0x04 Tx complete | ||
| 199 | 0x08 Tx FIFO room available | ||
| 200 | 0x10 A complete Rx packet has arrived | ||
| 201 | 0x20 A Rx packet has started to arrive | ||
| 202 | 0x40 The driver has requested an interrupt | ||
| 203 | 0x80 Statistics counter nearly full | ||
| 204 | ===== ====================================== | ||
| 205 | |||
| 206 | The bits in the transmit (Tx) status word are: | ||
| 207 | |||
| 208 | ===== ============================================ | ||
| 209 | value description | ||
| 210 | ===== ============================================ | ||
| 211 | 0x02 Out-of-window collision. | ||
| 212 | 0x04 Status stack overflow (normally impossible). | ||
| 213 | 0x08 16 collisions. | ||
| 214 | 0x10 Tx underrun (not enough PCI bus bandwidth). | ||
| 215 | 0x20 Tx jabber. | ||
| 216 | 0x40 Tx interrupt requested. | ||
| 217 | 0x80 Status is valid (this should always be set). | ||
| 218 | ===== ============================================ | ||
| 219 | |||
| 220 | |||
| 221 | When a transmit error occurs the driver produces a status message such as:: | ||
| 222 | |||
| 223 | eth0: Transmit error, Tx status register 82 | ||
| 224 | |||
| 225 | The two values typically seen here are: | ||
| 226 | |||
| 227 | 0x82 | ||
| 228 | ^^^^ | ||
| 229 | |||
| 230 | Out of window collision. This typically occurs when some other Ethernet | ||
| 231 | host is incorrectly set to full duplex on a half duplex network. | ||
| 232 | |||
| 233 | 0x88 | ||
| 234 | ^^^^ | ||
| 235 | |||
| 236 | 16 collisions. This typically occurs when the network is exceptionally busy | ||
| 237 | or when another host doesn't correctly back off after a collision. If this | ||
| 238 | error is mixed with 0x82 errors it is the result of a host incorrectly set | ||
| 239 | to full duplex (see above). | ||
| 240 | |||
| 241 | Both of these errors are the result of network problems that should be | ||
| 242 | corrected. They do not represent driver malfunction. | ||
| 243 | |||
| 244 | |||
| 245 | Revision history (this file) | ||
| 246 | ============================ | ||
| 247 | |||
| 248 | 28Feb02 v1.0 DR New; major portions based on Becker original 3c509 docs | ||
| 249 | |||
| @@ -10,6 +10,7 @@ Contents: · 9 unmodified lines | |||
| 10 | .. toctree:: | 10 | .. toctree:: |
| 11 | :maxdepth: 2 | 11 | :maxdepth: 2 |
| 12 | 12 | ||
| 13 | 3com/3c509 | ||
| 13 | 3com/vortex | 14 | 3com/vortex |
| 14 | amazon/ena | 15 | amazon/ena |
| 15 | altera/altera_tse | 16 | altera/altera_tse |
| @@ -86,6 +86,7 @@ regressions and security problems. · 85 unmodified lines | |||
| 86 | debugging/index | 86 | debugging/index |
| 87 | handling-regressions | 87 | handling-regressions |
| 88 | security-bugs | 88 | security-bugs |
| 89 | threat-model | ||
| 89 | cve | 90 | cve |
| 90 | embargoed-hardware-issues | 91 | embargoed-hardware-issues |
| 91 | 92 | ||
| @@ -66,6 +66,42 @@ In addition, the following information are highly desirable: · 65 unmodified lines | |||
| 66 | the issue appear. It is useful to share them, as they can be helpful to | 66 | the issue appear. It is useful to share them, as they can be helpful to |
| 67 | keep end users protected during the time it takes them to apply the fix. | 67 | keep end users protected during the time it takes them to apply the fix. |
| 68 | 68 | ||
| 69 | What qualifies as a security bug | ||
| 70 | -------------------------------- | ||
| 71 | |||
| 72 | It is important that most bugs are handled publicly so as to involve the widest | ||
| 73 | possible audience and find the best solution. By nature, bugs that are handled | ||
| 74 | in closed discussions between a small set of participants are less likely to | ||
| 75 | produce the best possible fix (e.g., risk of missing valid use cases, limited | ||
| 76 | testing abilities). | ||
| 77 | |||
| 78 | It turns out that the majority of the bugs reported via the security team are | ||
| 79 | just regular bugs that have been improperly qualified as security bugs due to | ||
| 80 | a lack of awareness of the Linux kernel's threat model, as described in | ||
| 81 | Documentation/process/threat-model.rst, and ought to have been sent through | ||
| 82 | the normal channels described in Documentation/admin-guide/reporting-issues.rst | ||
| 83 | instead. | ||
| 84 | |||
| 85 | The security list exists for urgent bugs that grant an attacker a capability | ||
| 86 | they are not supposed to have on a correctly configured production system, and | ||
| 87 | can be easily exploited, representing an imminent threat to many users. Before | ||
| 88 | reporting, consider whether the issue actually crosses a trust boundary on such | ||
| 89 | a system. | ||
| 90 | |||
| 91 | **If you resorted to AI assistance to identify a bug, you must treat it as | ||
| 92 | public**. While you may have valid reasons to believe it is not, the security | ||
| 93 | team's experience shows that bugs discovered this way systematically surface | ||
| 94 | simultaneously across multiple researchers, often on the same day. In this | ||
| 95 | case, do not publicly share a reproducer, as this could cause unintended harm; | ||
| 96 | just mention that one is available and maintainers might ask for it privately | ||
| 97 | if they need it. | ||
| 98 | |||
| 99 | If you are unsure whether an issue qualifies, err on the side of reporting | ||
| 100 | privately: the security team would rather triage a borderline report than miss | ||
| 101 | a real vulnerability. Reporting ordinary bugs to the security list, however, | ||
| 102 | does not make them move faster and consumes triage capacity that other reports | ||
| 103 | need. | ||
| 104 | |||
| 69 | Identifying contacts | 105 | Identifying contacts |
| 70 | -------------------- | 106 | -------------------- |
| 71 | 107 | ||
| @@ -74,7 +110,7 @@ affected subsystem's maintainers and Cc: the Linux kernel security team. Do · 2 unmodified lines | |||
| 74 | not send it to a public list at this stage, unless you have good reasons to | 110 | not send it to a public list at this stage, unless you have good reasons to |
| 75 | consider the issue as being public or trivial to discover (e.g. result of a | 111 | consider the issue as being public or trivial to discover (e.g. result of a |
| 76 | widely available automated vulnerability scanning tool that can be repeated by | 112 | widely available automated vulnerability scanning tool that can be repeated by |
| 77 | anyone). | 113 | anyone, or use of AI-based tools). |
| 78 | 114 | ||
| 79 | If you're sending a report for issues affecting multiple parts in the kernel, | 115 | If you're sending a report for issues affecting multiple parts in the kernel, |
| 80 | even if they're fairly similar issues, please send individual messages (think | 116 | even if they're fairly similar issues, please send individual messages (think |
| @@ -131,6 +167,64 @@ the Linux kernel security team only. Your message will be triaged, and you · 50 unmodified lines | |||
| 131 | will receive instructions about whom to contact, if needed. Your message may | 167 | will receive instructions about whom to contact, if needed. Your message may |
| 132 | equally be forwarded as-is to the relevant maintainers. | 168 | equally be forwarded as-is to the relevant maintainers. |
| 133 | 169 | ||
| 170 | Responsible use of AI to find bugs | ||
| 171 | ---------------------------------- | ||
| 172 | |||
| 173 | A significant fraction of bug reports submitted to the security team are | ||
| 174 | actually the result of code reviews assisted by AI tools. While this can be an | ||
| 175 | efficient means to find bugs in rarely explored areas, it causes an overload on | ||
| 176 | maintainers, who are sometimes forced to ignore such reports due to their poor | ||
| 177 | quality or accuracy. As such, reporters must be particularly cautious about a | ||
| 178 | number of points which tend to make these reports needlessly difficult to | ||
| 179 | handle: | ||
| 180 | |||
| 181 | * **Length**: AI-generated reports tend to be excessively long, containing | ||
| 182 | multiple sections and excessive detail. This makes it difficult to spot | ||
| 183 | important information such as affected files, versions, and impact. Please | ||
| 184 | ensure that a clear summary of the problem and all critical details are | ||
| 185 | presented first. Do not require triage engineers to scan multiple pages of | ||
| 186 | text. Configure your tools to produce concise, human-style reports. | ||
| 187 | |||
| 188 | * **Formatting**: Most AI-generated reports are littered with Markdown tags. | ||
| 189 | These decorations complicate the search for important information and do | ||
| 190 | not survive the quoting processes involved in forwarding or replying. | ||
| 191 | Please **always convert your report to plain text** without any formatting | ||
| 192 | decorations before sending it. | ||
| 193 | |||
| 194 | * **Impact Evaluation**: Many AI-generated reports lack an understanding | ||
| 195 | of the kernel's threat model (see Documentation/process/threat-model.rst) | ||
| 196 | and go to great lengths inventing theoretical consequences. This adds | ||
| 197 | noise and complicates triage. Please stick to verifiable facts (e.g., | ||
| 198 | "this bug permits any user to gain CAP_NET_ADMIN") without enumerating | ||
| 199 | speculative implications. Have your tool read this documentation as | ||
| 200 | part of the evaluation process. | ||
| 201 | |||
| 202 | * **Reproducer**: AI-based tools are often capable of generating reproducers. | ||
| 203 | Please always ensure your tool provides one and **test it thoroughly**. If | ||
| 204 | the reproducer does not work, or if the tool cannot produce one, the | ||
| 205 | validity of the report should be seriously questioned. Note that since the | ||
| 206 | report will be posted to a public list, the reproducer should only be | ||
| 207 | shared upon maintainers' request. | ||
| 208 | |||
| 209 | * **Propose a Fix**: Many AI tools are actually better at writing code than | ||
| 210 | evaluating it. Please ask your tool to propose a fix and **test it** before | ||
| 211 | reporting the problem. If the fix cannot be tested because it relies on | ||
| 212 | rare hardware or almost extinct network protocols, the issue is likely not | ||
| 213 | a security bug. In any case, if a fix is proposed, it must adhere to | ||
| 214 | Documentation/process/submitting-patches.rst and include a 'Fixes:' tag | ||
| 215 | designating the commit that introduced the bug. | ||
| 216 | |||
| 217 | Failure to consider these points exposes your report to the risk of being | ||
| 218 | ignored. | ||
| 219 | |||
| 220 | Use common sense when evaluating the report. If the affected file has not been | ||
| 221 | touched for more than one year and is maintained by a single individual, it is | ||
| 222 | likely that usage has declined and exposed users are virtually non-existent | ||
| 223 | (e.g., drivers for very old hardware, obsolete filesystems). In such cases, | ||
| 224 | there is no need to consume a maintainer's time with an unimportant report. If | ||
| 225 | the issue is clearly trivial and publicly discoverable, you should report it | ||
| 226 | directly to the public mailing lists. | ||
| 227 | |||
| 134 | Sending the report | 228 | Sending the report |
| 135 | ------------------ | 229 | ------------------ |
| 136 | 230 | ||
| @@ -148,7 +242,15 @@ run additional tests. Reports where the reporter does not respond promptly · 11 unmodified lines | |||
| 148 | or cannot effectively discuss their findings may be abandoned if the | 242 | or cannot effectively discuss their findings may be abandoned if the |
| 149 | communication does not quickly improve. | 243 | communication does not quickly improve. |
| 150 | 244 | ||
| 151 | The report must be sent to maintainers, with the security team in ``Cc:``. | 245 | The report must be sent to maintainers. If there are two or fewer |
| 246 | recipients in your message, you must also always Cc: the Linux kernel | ||
| 247 | security team who will ensure the message is delivered to the proper | ||
| 248 | people, and will be able to assist small maintainer teams with processes | ||
| 249 | they may not be familiar with. For larger teams, Cc: the Linux kernel | ||
| 250 | security team for your first few reports or when seeking specific help, | ||
| 251 | such as when resending a message which got no response within a week. | ||
| 252 | Once you have become comfortable with the process for a few reports, it is | ||
| 253 | no longer necessary to Cc: the security list when sending to large teams. | ||
| 152 | The Linux kernel security team can be contacted by email at | 254 | The Linux kernel security team can be contacted by email at |
| 153 | <security@kernel.org>. This is a private list of security officers | 255 | <security@kernel.org>. This is a private list of security officers |
| 154 | who will help verify the bug report and assist developers working on a fix. | 256 | who will help verify the bug report and assist developers working on a fix. |
| @@ -0,0 +1,235 @@ | |||
| 1 | The Linux Kernel threat model | ||
| 2 | ============================= | ||
| 3 | |||
| 4 | There are a lot of assumptions regarding what the kernel does and does not | ||
| 5 | protect against. These assumptions tend to cause confusion for bug reports | ||
| 6 | (:doc:`security-related ones <security-bugs>` vs :doc:`non-security ones | ||
| 7 | <../admin-guide/reporting-issues>`), and can complicate security enforcement | ||
| 8 | when the responsibilities for some boundaries is not clear between the kernel, | ||
| 9 | distros, administrators and users. | ||
| 10 | |||
| 11 | This document tries to clarify the responsibilities of the kernel in this | ||
| 12 | domain. | ||
| 13 | |||
| 14 | The kernel's responsibilities | ||
| 15 | ----------------------------- | ||
| 16 | |||
| 17 | The kernel abstracts access to local hardware resources and to remote systems | ||
| 18 | in a way that allows multiple local users to get a fair share of the available | ||
| 19 | resources granted to them, and, when the underlying hardware permits, to assign | ||
| 20 | a level of confidentiality to their communications and to the data they are | ||
| 21 | processing or storing. | ||
| 22 | |||
| 23 | The kernel assumes that the underlying hardware behaves according to its | ||
| 24 | specifications. This includes the integrity of the CPU's instruction set, the | ||
| 25 | transparency of the branch prediction unit and the cache units, the consistency | ||
| 26 | of the Memory Management Unit (MMU), the isolation of DMA-capable peripherals | ||
| 27 | (e.g., via IOMMU), state transitions in controllers, ranges of values read from | ||
| 28 | registers, the respect of documented hardware limitations, etc. | ||
| 29 | |||
| 30 | When hardware fails to maintain its specified isolation (e.g., CPU bugs, | ||
| 31 | side-channels, hardware response to unexpected inputs), the kernel will usually | ||
| 32 | attempt to implement reasonable mitigations. These are best-effort measures | ||
| 33 | intended to reduce the attack surface or elevate the cost of an attack within | ||
| 34 | the limits of the hardware's facilities; they do not constitute a | ||
| 35 | kernel-provided safety guarantee. | ||
| 36 | |||
| 37 | Users always perform their activities under the authority of an administrator | ||
| 38 | who is able to grant or deny various types of permissions that may affect how | ||
| 39 | users benefit from available resources, or the level of confidentiality of | ||
| 40 | their activities. Administrators may also delegate all or part of their own | ||
| 41 | permissions to some users, particularly via capabilities but not only. All this | ||
| 42 | is performed via configuration (sysctl, file-system permissions etc). | ||
| 43 | |||
| 44 | The Linux Kernel applies a certain collection of default settings that match | ||
| 45 | its threat model. Distros have their own threat model and will come with their | ||
| 46 | own configuration presets, that the administrator may have to adjust to better | ||
| 47 | suit their expectations (relax or restrict). | ||
| 48 | |||
| 49 | By default, the Linux Kernel guarantees the following protections when running | ||
| 50 | on common processors featuring privilege levels and memory management units: | ||
| 51 | |||
| 52 | * **User-based isolation**: an unprivileged user may restrict access to their | ||
| 53 | own data from other unprivileged users running on the same system. This | ||
| 54 | includes: | ||
| 55 | |||
| 56 | * stored data, via file system permissions | ||
| 57 | * in-memory data (pages are not accessible by default to other users) | ||
| 58 | * process activity (ptrace is not permitted to other users) | ||
| 59 | * inter-process communication (other users may not observe data exchanged via | ||
| 60 | UNIX domain sockets or other IPC mechanisms). | ||
| 61 | * network communications within the same or with other systems | ||
| 62 | |||
| 63 | * **Capability-based protection**: | ||
| 64 | |||
| 65 | * users not having elevated capabilities (including but not limited to | ||
| 66 | CAP_SYS_ADMIN) may not alter the | ||
| 67 | kernel's configuration, memory nor state, change other users' view of the | ||
| 68 | file system layout, grant any user capabilities they do not have, nor | ||
| 69 | affect the system's availability (shutdown, reboot, panic, hang, or making | ||
| 70 | the system unresponsive via unbounded resource exhaustion). | ||
| 71 | * users not having the ``CAP_NET_ADMIN`` capability may not alter the network | ||
| 72 | configuration, intercept nor spoof network communications from other users | ||
| 73 | nor systems. | ||
| 74 | * users not having ``CAP_SYS_PTRACE`` may not observe other users' processes | ||
| 75 | activities. | ||
| 76 | |||
| 77 | When ``CONFIG_USER_NS`` is set, the kernel also permits unprivileged users to | ||
| 78 | create their own user namespace in which they have all capabilities, but with a | ||
| 79 | number of restrictions (they may not perform actions that have impacts on the | ||
| 80 | initial user namespace, such as changing time, loading modules or mounting | ||
| 81 | block devices). Please refer to ``user_namespaces(7)`` for more details, the | ||
| 82 | possibilities of user namespaces are not covered in this document. | ||
| 83 | |||
| 84 | The kernel also offers a lot of troubleshooting and debugging facilities, which | ||
| 85 | can constitute attack vectors when placed in wrong hands. While some of them | ||
| 86 | are designed to be accessible to regular local users with a low risk (e.g. | ||
| 87 | kernel logs via ``/proc/kmsg``), some would expose enough information to | ||
| 88 | represent a risk in most places and the decision to expose them is under the | ||
| 89 | administrator's responsibility (perf events, traces), and others are not | ||
| 90 | designed to be accessed by non-privileged users (e.g. debugfs). Access to these | ||
| 91 | facilities by a user who has been explicitly granted permission by an | ||
| 92 | administrator does not constitute a security breach. | ||
| 93 | |||
| 94 | Bugs that permit to violate the principles above constitute security breaches. | ||
| 95 | However, bugs that permit one violation only once another one was already | ||
| 96 | achieved are only weaknesses. The kernel applies a number of self-protection | ||
| 97 | measures whose purpose is to avoid crossing a security boundary when certain | ||
| 98 | classes of bugs are found, but a failure of these extra protections do not | ||
| 99 | constitute a vulnerability alone. | ||
| 100 | |||
| 101 | What does not constitute a security bug | ||
| 102 | --------------------------------------- | ||
| 103 | |||
| 104 | In the Linux kernel's threat model, the following classes of problems are | ||
| 105 | **NOT** considered as Linux Kernel security bugs. However, when it is believed | ||
| 106 | that the kernel could do better, they should be reported, so that they can be | ||
| 107 | reviewed and fixed where reasonably possible, but they will be handled as any | ||
| 108 | regular bug: | ||
| 109 | |||
| 110 | * **Configuration**: | ||
| 111 | |||
| 112 | * outdated kernels and particularly end-of-life branches are out of the scope | ||
| 113 | of the kernel's threat model: administrators are responsible for keeping | ||
| 114 | their system up to date. For a bug to qualify as a security bug, it must be | ||
| 115 | demonstrated that it affects actively maintained versions. | ||
| 116 | |||
| 117 | * build-level: changes to the kernel configuration that are explicitly | ||
| 118 | documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or | ||
| 119 | targeted at developers only. | ||
| 120 | |||
| 121 | * OS-level: changes to command line parameters, sysctls, filesystem | ||
| 122 | permissions, user capabilities, exposure of privileged interfaces, that | ||
| 123 | explicitly increase exposure by either offering non-default access to | ||
| 124 | unprivileged users, or reduce the kernel's ability to enforce some | ||
| 125 | protections or mitigations. Example: write access to procfs or debugfs. | ||
| 126 | |||
| 127 | * issues triggered only when using features intended for development or | ||
| 128 | debugging (e.g., LOCKDEP, KASAN, FAULT_INJECTION): these features are known | ||
| 129 | to introduce overhead and potential instability and are not intended for | ||
| 130 | production use. | ||
| 131 | |||
| 132 | * issues affecting drivers exposed under CONFIG_STAGING, as well as features | ||
| 133 | marked EXPERIMENTAL in the configuration. | ||
| 134 | |||
| 135 | * loading of explicitly insecure/broken/staging modules, and generally any | ||
| 136 | using any subsystem marked as experimental or not intended for production | ||
| 137 | use. | ||
| 138 | |||
| 139 | * running out-of-tree modules or unofficial kernel forks; these should be | ||
| 140 | reported to the relevant vendor. | ||
| 141 | |||
| 142 | * **Excess of initial privileges**: | ||
| 143 | |||
| 144 | * actions performed by a user already possessing the privileges required to | ||
| 145 | perform that action or modify that state (e.g. ``CAP_SYS_ADMIN``, | ||
| 146 | ``CAP_NET_ADMIN``, ``CAP_SYS_RAWIO``, ``CAP_SYS_MODULE`` with no further | ||
| 147 | boundary being crossed). | ||
| 148 | |||
| 149 | * actions performed in user namespace that do not bypass the restrictions | ||
| 150 | imposed to the initial user (e.g. ptrace usage, signal delivery, resource | ||
| 151 | usage, access to FS/device/sysctl/memory, network binding, system/network | ||
| 152 | configuration etc). | ||
| 153 | |||
| 154 | * anything performed by the root user in the initial namespace (e.g. kernel | ||
| 155 | oops when writing to a privileged device). | ||
| 156 | |||
| 157 | * **Out of production use**: | ||
| 158 | |||
| 159 | This covers theoretical/probabilistic attacks that rely on laboratory | ||
| 160 | conditions with zero system noise, or those requiring an unrealistic number | ||
| 161 | of attempts (e.g., billions of trials) that would be detected by standard | ||
| 162 | system monitoring long before success, such as: | ||
| 163 | |||
| 164 | * prediction of random numbers that only works in a totally silent | ||
| 165 | environment (such as IP ID, TCP ports or sequence numbers that can only be | ||
| 166 | guessed in a lab). | ||
| 167 | |||
| 168 | * activity observation and information leaks based on probabilistic | ||
| 169 | approaches that are prone to measurement noise and not realistically | ||
| 170 | reproducible on a production system. | ||
| 171 | |||
| 172 | * issues that can only be triggered by heavy attacks (e.g. brute force) whose | ||
| 173 | impact on the system makes it unlikely or impossible to remain undetected | ||
| 174 | before they succeed (e.g. consuming all memory before succeeding). | ||
| 175 | |||
| 176 | * problems seen only under development simulators, emulators, or combinations | ||
| 177 | that do not exist on real systems at the time of reporting (issues | ||
| 178 | involving tens of millions of threads, tens of thousands of CPUs, | ||
| 179 | unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds. | ||
| 180 | |||
| 181 | * issues whose reproduction requires hardware modification or emulation, | ||
| 182 | including fake USB devices that pretend to be another one. | ||
| 183 | |||
| 184 | * as well as issues that can be triggered at a cost that is orders of | ||
| 185 | magnitude higher than the expected benefits (e.g. fully functional keyboard | ||
| 186 | emulator only to retrieve 7 uninitialized bytes in a structure, or | ||
| 187 | brute-force method involving millions of connection attempts to guess a | ||
| 188 | port number). | ||
| 189 | |||
| 190 | * **Hardening failures**: | ||
| 191 | |||
| 192 | * ability to bypass some of the kernel's hardening measures with no | ||
| 193 | demonstrable exploit path (e.g. ASLR bypass, events timing or probing with | ||
| 194 | no demonstrable consequence). These are just weaknesses, not | ||
| 195 | vulnerabilities. | ||
| 196 | |||
| 197 | * missing argument checks and failure to report certain errors with no | ||
| 198 | immediate consequence. | ||
| 199 | |||
| 200 | * **Random information leaks**: | ||
| 201 | |||
| 202 | This concerns information leaks of small data parts that happen to be there | ||
| 203 | and that cannot be chosen by the attacker, or face access restrictions: | ||
| 204 | |||
| 205 | * structure padding reported by syscalls or other interfaces. | ||
| 206 | |||
| 207 | * identifiers, partial data, non-terminated strings reported in error | ||
| 208 | messages. | ||
| 209 | |||
| 210 | * Leaks of kernel memory addresses/pointers do not constitute an immediately | ||
| 211 | exploitable vector and are not security bugs, though they must be reported | ||
| 212 | and fixed. | ||
| 213 | |||
| 214 | * **Crafted file system images**: | ||
| 215 | |||
| 216 | * bugs triggered by mounting a corrupted or maliciously crafted file system | ||
| 217 | image are generally not security bugs, as the kernel assumes the underlying | ||
| 218 | storage media is under the administrator's control, unless the filesystem | ||
| 219 | driver is specifically documented as being hardened against untrusted media. | ||
| 220 | |||
| 221 | * issues that are resolved, mitigated, or detected by running a filesystem | ||
| 222 | consistency check (fsck) on the image prior to mounting. | ||
| 223 | |||
| 224 | * **Physical access**: | ||
| 225 | |||
| 226 | Issues that require physical access to the machine, hardware modification, or | ||
| 227 | the use of specialized hardware (e.g., logic analyzers, DMA-attack tools over | ||
| 228 | PCI-E/Thunderbolt) are out of scope unless the system is explicitly | ||
| 229 | configured with technologies meant to defend against such attacks | ||
| 230 | (e.g. IOMMU). | ||
| 231 | |||
| 232 | * **Functional and performance regressions**: | ||
| 233 | |||
| 234 | Any issue that can be mitigated by setting proper permissions and limits | ||
| 235 | doesn't qualify as a security bug. | ||
| @@ -40,7 +40,7 @@ There are two drivers in the kernel · 39 unmodified lines | |||
| 40 | 40 | ||
| 41 | *For systems using SoundWire*: sound/soc/codecs/cs35l56.c and associated files | 41 | *For systems using SoundWire*: sound/soc/codecs/cs35l56.c and associated files |
| 42 | 42 | ||
| 43 | *For systems using HDA*: sound/pci/hda/cs35l56_hda.c | 43 | *For systems using HDA*: sound/hda/codecs/side-codecs/cs35l56_hda.c |
| 44 | 44 | ||
| 45 | Firmware | 45 | Firmware |
| 46 | ======== | 46 | ======== |
| @@ -24,6 +24,97 @@ Quick access to CPU number, node ID · 23 unmodified lines | |||
| 24 | Allows to implement per CPU data efficiently. Documentation is in code and | 24 | Allows to implement per CPU data efficiently. Documentation is in code and |
| 25 | selftests. :( | 25 | selftests. :( |
| 26 | 26 | ||
| 27 | Optimized RSEQ V2 | ||
| 28 | ----------------- | ||
| 29 | |||
| 30 | On architectures which utilize the generic entry code and generic TIF bits | ||
| 31 | the kernel supports runtime optimizations for RSEQ, which also enable | ||
| 32 | enhanced features like scheduler time slice extensions. | ||
| 33 | |||
| 34 | To enable them a task has to register the RSEQ region with at least the | ||
| 35 | length advertised by getauxval(AT_RSEQ_FEATURE_SIZE). | ||
| 36 | |||
| 37 | If existing binaries register with RSEQ_ORIG_SIZE (32 bytes), the kernel | ||
| 38 | keeps the legacy low performance mode enabled to fulfil the expectations | ||
| 39 | of existing users regarding the original RSEQ implementation behaviour. | ||
| 40 | |||
| 41 | The following table documents the ABI and behavioral guarantees of the | ||
| 42 | legacy and the optimized V2 mode. | ||
| 43 | |||
| 44 | .. list-table:: RSEQ modes | ||
| 45 | :header-rows: 1 | ||
| 46 | |||
| 47 | * - Nr | ||
| 48 | - What | ||
| 49 | |||
| 50 | - Legacy | ||
| 51 | - Optimized V2 | ||
| 52 | |||
| 53 | * - 1 | ||
| 54 | - The cpu_id_start, cpu_id, node_id and mm_cid fields (User mode read | ||
| 55 | only) | ||
| 56 | .. Legacy | ||
| 57 | - Updated by the kernel unconditionally after each context switch and | ||
| 58 | before signal delivery | ||
| 59 | .. Optimized V2 | ||
| 60 | - Updated by the kernel if and only if they change, i.e. if the task | ||
| 61 | is migrated or mm_cid changes | ||
| 62 | |||
| 63 | * - 2 | ||
| 64 | - The rseq_cs critical section field | ||
| 65 | .. Legacy | ||
| 66 | - Evaluated and handled unconditionally after each context switch and | ||
| 67 | before signal delivery | ||
| 68 | .. Optimized V2 | ||
| 69 | - Evaluated and handled conditionally only when user space was | ||
| 70 | interrupted and was scheduled out or before delivering a signal in | ||
| 71 | the interrupted context. | ||
| 72 | |||
| 73 | * - 3 | ||
| 74 | - Read only fields | ||
| 75 | .. Legacy | ||
| 76 | - No strict enforcement except in debug mode | ||
| 77 | .. Optimized V2 | ||
| 78 | - Strict enforcement | ||
| 79 | |||
| 80 | * - 4 | ||
| 81 | - membarrier(...RSEQ) | ||
| 82 | .. Legacy | ||
| 83 | - All running threads of the process are interrupted and the ID fields | ||
| 84 | are rewritten and eventually active critical sections are aborted | ||
| 85 | before they return to user space. All threads which are scheduled | ||
| 86 | out whether voluntary or not are covered by #1/#2 above. | ||
| 87 | .. Optimized V2 | ||
| 88 | - All running threads of the process are interrupted and eventually | ||
| 89 | active critical sections are aborted before these threads return to | ||
| 90 | user space. The ID fields are only updated if changed as a | ||
| 91 | consequence of the interrupt. All threads which are scheduled out | ||
| 92 | whether voluntary or not are covered by #1/#2 above. | ||
| 93 | |||
| 94 | * - 5 | ||
| 95 | - Time slice extensions | ||
| 96 | .. Legacy | ||
| 97 | - Not supported | ||
| 98 | .. Optimized V2 | ||
| 99 | - Supported | ||
| 100 | |||
| 101 | The legacy mode is obviously less performant as it does unconditional | ||
| 102 | updates and critical section checks even if not strictly required by the | ||
| 103 | ABI contract. That can't be changed anymore as some users depend on that | ||
| 104 | observed behavior, which in turn enables them to violate the ABI and | ||
| 105 | overwrite the cpu_id_start field for their own purposes. This is obviously | ||
| 106 | discouraged as it renders RSEQ incompatible with the intended usage and | ||
| 107 | breaks the expectation of other libraries in the same application. | ||
| 108 | |||
| 109 | The ABI compliant optimized v2 mode, which respects the read only fields, | ||
| 110 | does not require unconditional updates and therefore is way more | ||
| 111 | performant. The kernel validates the read only fields for compliance. If | ||
| 112 | user space modifies them, the process is killed. Compliant usage allows | ||
| 113 | multiple libraries in the same application to benefit from the RSEQ | ||
| 114 | functionality without disturbing each other. The ABI compliant optimized v2 | ||
| 115 | mode also enables extended RSEQ features like time slice extensions. | ||
| 116 | |||
| 117 | |||
| 27 | Scheduler time slice extensions | 118 | Scheduler time slice extensions |
| 28 | ------------------------------- | 119 | ------------------------------- |
| 29 | 120 | ||
| @@ -37,7 +128,8 @@ The prerequisites for this functionality are: · 7 unmodified lines | |||
| 37 | 128 | ||
| 38 | * Enabled at boot time (default is enabled) | 129 | * Enabled at boot time (default is enabled) |
| 39 | 130 | ||
| 40 | * A rseq userspace pointer has been registered for the thread | 131 | * A rseq userspace pointer has been registered for the thread in |
| 132 | optimized V2 mode | ||
| 41 | 133 | ||
| 42 | The thread has to enable the functionality via prctl(2):: | 134 | The thread has to enable the functionality via prctl(2):: |
| 43 | 135 | ||
| @@ -656,8 +656,8 @@ References · 655 unmodified lines | |||
| 656 | See [white-paper]_, [api-spec]_, [amd-apm]_, [kvm-forum]_, and [snp-fw-abi]_ | 656 | See [white-paper]_, [api-spec]_, [amd-apm]_, [kvm-forum]_, and [snp-fw-abi]_ |
| 657 | for more info. | 657 | for more info. |
| 658 | 658 | ||
| 659 | .. [white-paper] https://developer.amd.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf | 659 | .. [white-paper] https://docs.amd.com/v/u/en-US/memory-encryption-white-paper |
| 660 | .. [api-spec] https://support.amd.com/TechDocs/55766_SEV-KM_API_Specification.pdf | 660 | .. [api-spec] https://docs.amd.com/v/u/en-US/55766_PUB_3.24_SEV_API |
| 661 | .. [amd-apm] https://support.amd.com/TechDocs/24593.pdf (section 15.34) | 661 | .. [amd-apm] https://docs.amd.com/v/u/en-US/24593_3.44_APM_Vol2 (section 15.34) |
| 662 | .. [kvm-forum] https://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf | 662 | .. [kvm-forum] https://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf |
| 663 | .. [snp-fw-abi] https://www.amd.com/system/files/TechDocs/56860.pdf | 663 | .. [snp-fw-abi] https://www.amd.com/content/dam/amd/en/documents/developer/56860.pdf |
| @@ -68,6 +68,12 @@ Maintainers List · 67 unmodified lines | |||
| 68 | first. When adding to this list, please keep the entries in | 68 | first. When adding to this list, please keep the entries in |
| 69 | alphabetical order. | 69 | alphabetical order. |
| 70 | 70 | ||
| 71 | 3C509 NETWORK DRIVER | ||
| 72 | M: "Maciej W. Rozycki" <macro@orcam.me.uk> | ||
| 73 | L: netdev@vger.kernel.org | ||
| 74 | S: Maintained | ||
| 75 | F: drivers/net/ethernet/3com/3c509.c | ||
| 76 | |||
| 71 | 3C59X NETWORK DRIVER | 77 | 3C59X NETWORK DRIVER |
| 72 | M: Steffen Klassert <klassert@kernel.org> | 78 | M: Steffen Klassert <klassert@kernel.org> |
| 73 | L: netdev@vger.kernel.org | 79 | L: netdev@vger.kernel.org |
| @@ -1144,8 +1150,9 @@ F: Documentation/arch/x86/amd-hfi.rst · 1070 unmodified lines | |||
| 1144 | F: drivers/platform/x86/amd/hfi/ | 1150 | F: drivers/platform/x86/amd/hfi/ |
| 1145 | 1151 | ||
| 1146 | AMD IOMMU (AMD-VI) | 1152 | AMD IOMMU (AMD-VI) |
| 1147 | M: Joerg Roedel <joro@8bytes.org> | 1153 | M: Joerg Roedel (AMD) <joro@8bytes.org> |
| 1148 | R: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> | 1154 | R: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> |
| 1155 | R: Vasant Hegde <vasant.hegde@amd.com> | ||
| 1149 | L: iommu@lists.linux.dev | 1156 | L: iommu@lists.linux.dev |
| 1150 | S: Maintained | 1157 | S: Maintained |
| 1151 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git | 1158 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git |
| @@ -2033,7 +2040,7 @@ F: Documentation/hwmon/aquacomputer_d5next.rst · 881 unmodified lines | |||
| 2033 | F: drivers/hwmon/aquacomputer_d5next.c | 2040 | F: drivers/hwmon/aquacomputer_d5next.c |
| 2034 | 2041 | ||
| 2035 | AQUANTIA ETHERNET DRIVER (atlantic) | 2042 | AQUANTIA ETHERNET DRIVER (atlantic) |
| 2036 | M: Igor Russkikh <irusskikh@marvell.com> | 2043 | M: Sukhdeep Singh <sukhdeeps@marvell.com> |
| 2037 | L: netdev@vger.kernel.org | 2044 | L: netdev@vger.kernel.org |
| 2038 | S: Maintained | 2045 | S: Maintained |
| 2039 | W: https://www.marvell.com/ | 2046 | W: https://www.marvell.com/ |
| @@ -2042,7 +2049,7 @@ F: Documentation/networking/device_drivers/ethernet/aquantia/atlantic.rst · 2 unmodified lines | |||
| 2042 | F: drivers/net/ethernet/aquantia/atlantic/ | 2049 | F: drivers/net/ethernet/aquantia/atlantic/ |
| 2043 | 2050 | ||
| 2044 | AQUANTIA ETHERNET DRIVER PTP SUBSYSTEM | 2051 | AQUANTIA ETHERNET DRIVER PTP SUBSYSTEM |
| 2045 | M: Egor Pomozov <epomozov@marvell.com> | 2052 | M: Sukhdeep Singh <sukhdeeps@marvell.com> |
| 2046 | L: netdev@vger.kernel.org | 2053 | L: netdev@vger.kernel.org |
| 2047 | S: Maintained | 2054 | S: Maintained |
| 2048 | W: http://www.aquantia.com | 2055 | W: http://www.aquantia.com |
| @@ -2076,7 +2083,7 @@ F: Documentation/devicetree/bindings/display/snps,arcpgu.txt · 27 unmodified lines | |||
| 2076 | F: drivers/gpu/drm/tiny/arcpgu.c | 2083 | F: drivers/gpu/drm/tiny/arcpgu.c |
| 2077 | 2084 | ||
| 2078 | ARCNET NETWORK LAYER | 2085 | ARCNET NETWORK LAYER |
| 2079 | M: Michael Grzeschik <m.grzeschik@pengutronix.de> | 2086 | M: Michael Grzeschik <mgr@kernel.org> |
| 2080 | L: netdev@vger.kernel.org | 2087 | L: netdev@vger.kernel.org |
| 2081 | S: Maintained | 2088 | S: Maintained |
| 2082 | F: drivers/net/arcnet/ | 2089 | F: drivers/net/arcnet/ |
| @@ -3379,7 +3386,9 @@ F: drivers/irqchip/irq-rda-intc.c · 1296 unmodified lines | |||
| 3379 | F: drivers/tty/serial/rda-uart.c | 3386 | F: drivers/tty/serial/rda-uart.c |
| 3380 | 3387 | ||
| 3381 | ARM/REALTEK ARCHITECTURE | 3388 | ARM/REALTEK ARCHITECTURE |
| 3382 | M: Andreas Färber <afaerber@suse.de> | 3389 | M: James Tai <james.tai@realtek.com> |
| 3390 | M: Yu-Chun Lin <eleanor.lin@realtek.com> | ||
| 3391 | R: Andreas Färber <afaerber@suse.com> | ||
| 3383 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 3392 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 3384 | L: linux-realtek-soc@lists.infradead.org (moderated for non-subscribers) | 3393 | L: linux-realtek-soc@lists.infradead.org (moderated for non-subscribers) |
| 3385 | S: Maintained | 3394 | S: Maintained |
| @@ -3387,6 +3396,7 @@ F: Documentation/devicetree/bindings/arm/realtek.yaml · 1 unmodified line | |||
| 3387 | F: arch/arm/boot/dts/realtek/ | 3396 | F: arch/arm/boot/dts/realtek/ |
| 3388 | F: arch/arm/mach-realtek/ | 3397 | F: arch/arm/mach-realtek/ |
| 3389 | F: arch/arm64/boot/dts/realtek/ | 3398 | F: arch/arm64/boot/dts/realtek/ |
| 3399 | F: drivers/pinctrl/realtek/ | ||
| 3390 | 3400 | ||
| 3391 | ARM/RISC-V/RENESAS ARCHITECTURE | 3401 | ARM/RISC-V/RENESAS ARCHITECTURE |
| 3392 | M: Geert Uytterhoeven <geert+renesas@glider.be> | 3402 | M: Geert Uytterhoeven <geert+renesas@glider.be> |
| @@ -4199,8 +4209,8 @@ F: include/uapi/linux/sonet.h · 806 unmodified lines | |||
| 4199 | F: net/atm/ | 4209 | F: net/atm/ |
| 4200 | 4210 | ||
| 4201 | ATMEL MACB ETHERNET DRIVER | 4211 | ATMEL MACB ETHERNET DRIVER |
| 4202 | M: Nicolas Ferre <nicolas.ferre@microchip.com> | 4212 | M: Théo Lebrun <theo.lebrun@bootlin.com> |
| 4203 | M: Claudiu Beznea <claudiu.beznea@tuxon.dev> | 4213 | R: Conor Dooley <conor.dooley@microchip.com> |
| 4204 | S: Maintained | 4214 | S: Maintained |
| 4205 | F: drivers/net/ethernet/cadence/ | 4215 | F: drivers/net/ethernet/cadence/ |
| 4206 | 4216 | ||
| @@ -4317,18 +4327,16 @@ F: Documentation/devicetree/bindings/leds/backlight/awinic,aw99706.yaml · 110 unmodified lines | |||
| 4317 | F: drivers/video/backlight/aw99706.c | 4327 | F: drivers/video/backlight/aw99706.c |
| 4318 | 4328 | ||
| 4319 | AXENTIA ARM DEVICES | 4329 | AXENTIA ARM DEVICES |
| 4320 | M: Peter Rosin <peda@axentia.se> | ||
| 4321 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 4330 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 4322 | S: Maintained | 4331 | S: Orphan |
| 4323 | F: arch/arm/boot/dts/microchip/at91-linea.dtsi | 4332 | F: arch/arm/boot/dts/microchip/at91-linea.dtsi |
| 4324 | F: arch/arm/boot/dts/microchip/at91-natte.dtsi | 4333 | F: arch/arm/boot/dts/microchip/at91-natte.dtsi |
| 4325 | F: arch/arm/boot/dts/microchip/at91-nattis-2-natte-2.dts | 4334 | F: arch/arm/boot/dts/microchip/at91-nattis-2-natte-2.dts |
| 4326 | F: arch/arm/boot/dts/microchip/at91-tse850-3.dts | 4335 | F: arch/arm/boot/dts/microchip/at91-tse850-3.dts |
| 4327 | 4336 | ||
| 4328 | AXENTIA ASOC DRIVERS | 4337 | AXENTIA ASOC DRIVERS |
| 4329 | M: Peter Rosin <peda@axentia.se> | ||
| 4330 | L: linux-sound@vger.kernel.org | 4338 | L: linux-sound@vger.kernel.org |
| 4331 | S: Maintained | 4339 | S: Orphan |
| 4332 | F: Documentation/devicetree/bindings/sound/axentia,* | 4340 | F: Documentation/devicetree/bindings/sound/axentia,* |
| 4333 | F: sound/soc/atmel/tse850-pcm5142.c | 4341 | F: sound/soc/atmel/tse850-pcm5142.c |
| 4334 | 4342 | ||
| @@ -6383,6 +6391,7 @@ F: include/uapi/linux/comedi.h · 2048 unmodified lines | |||
| 6383 | COMMON CLK FRAMEWORK | 6391 | COMMON CLK FRAMEWORK |
| 6384 | M: Michael Turquette <mturquette@baylibre.com> | 6392 | M: Michael Turquette <mturquette@baylibre.com> |
| 6385 | M: Stephen Boyd <sboyd@kernel.org> | 6393 | M: Stephen Boyd <sboyd@kernel.org> |
| 6394 | R: Brian Masney <bmasney@redhat.com> | ||
| 6386 | L: linux-clk@vger.kernel.org | 6395 | L: linux-clk@vger.kernel.org |
| 6387 | S: Maintained | 6396 | S: Maintained |
| 6388 | Q: http://patchwork.kernel.org/project/linux-clk/list/ | 6397 | Q: http://patchwork.kernel.org/project/linux-clk/list/ |
| @@ -7102,6 +7111,12 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core/debugobjec · 713 unmodified lines | |||
| 7102 | F: include/linux/debugobjects.h | 7111 | F: include/linux/debugobjects.h |
| 7103 | F: lib/debugobjects.c | 7112 | F: lib/debugobjects.c |
| 7104 | 7113 | ||
| 7114 | DEC LANCE NETWORK DRIVER | ||
| 7115 | M: "Maciej W. Rozycki" <macro@orcam.me.uk> | ||
| 7116 | L: netdev@vger.kernel.org | ||
| 7117 | S: Maintained | ||
| 7118 | F: drivers/net/ethernet/amd/declance.c | ||
| 7119 | |||
| 7105 | DECSTATION PLATFORM SUPPORT | 7120 | DECSTATION PLATFORM SUPPORT |
| 7106 | M: "Maciej W. Rozycki" <macro@orcam.me.uk> | 7121 | M: "Maciej W. Rozycki" <macro@orcam.me.uk> |
| 7107 | L: linux-mips@vger.kernel.org | 7122 | L: linux-mips@vger.kernel.org |
| @@ -7898,7 +7913,7 @@ F: drivers/gpu/drm/sun4i/sun8i* · 790 unmodified lines | |||
| 7898 | 7913 | ||
| 7899 | DRM DRIVER FOR APPLE TOUCH BARS | 7914 | DRM DRIVER FOR APPLE TOUCH BARS |
| 7900 | M: Aun-Ali Zaidi <admin@kodeit.net> | 7915 | M: Aun-Ali Zaidi <admin@kodeit.net> |
| 7901 | M: Aditya Garg <gargaditya08@live.com> | 7916 | M: Aditya Garg <gargaditya08@proton.me> |
| 7902 | L: dri-devel@lists.freedesktop.org | 7917 | L: dri-devel@lists.freedesktop.org |
| 7903 | S: Maintained | 7918 | S: Maintained |
| 7904 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git | 7919 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git |
| @@ -8218,10 +8233,9 @@ F: include/uapi/drm/nouveau_drm.h · 313 unmodified lines | |||
| 8218 | CORE DRIVER FOR NVIDIA GPUS [RUST] | 8233 | CORE DRIVER FOR NVIDIA GPUS [RUST] |
| 8219 | M: Danilo Krummrich <dakr@kernel.org> | 8234 | M: Danilo Krummrich <dakr@kernel.org> |
| 8220 | M: Alexandre Courbot <acourbot@nvidia.com> | 8235 | M: Alexandre Courbot <acourbot@nvidia.com> |
| 8221 | L: nouveau@lists.freedesktop.org | 8236 | L: nova-gpu@lists.linux.dev |
| 8222 | S: Supported | 8237 | S: Supported |
| 8223 | W: https://rust-for-linux.com/nova-gpu-driver | 8238 | W: https://rust-for-linux.com/nova-gpu-driver |
| 8224 | Q: https://patchwork.freedesktop.org/project/nouveau/ | ||
| 8225 | B: https://gitlab.freedesktop.org/drm/nova/-/issues | 8239 | B: https://gitlab.freedesktop.org/drm/nova/-/issues |
| 8226 | C: irc://irc.oftc.net/nouveau | 8240 | C: irc://irc.oftc.net/nouveau |
| 8227 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next | 8241 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next |
| @@ -8230,10 +8244,9 @@ F: drivers/gpu/nova-core/ · 2 unmodified lines | |||
| 8230 | 8244 | ||
| 8231 | DRM DRIVER FOR NVIDIA GPUS [RUST] | 8245 | DRM DRIVER FOR NVIDIA GPUS [RUST] |
| 8232 | M: Danilo Krummrich <dakr@kernel.org> | 8246 | M: Danilo Krummrich <dakr@kernel.org> |
| 8233 | L: nouveau@lists.freedesktop.org | 8247 | L: nova-gpu@lists.linux.dev |
| 8234 | S: Supported | 8248 | S: Supported |
| 8235 | W: https://rust-for-linux.com/nova-gpu-driver | 8249 | W: https://rust-for-linux.com/nova-gpu-driver |
| 8236 | Q: https://patchwork.freedesktop.org/project/nouveau/ | ||
| 8237 | B: https://gitlab.freedesktop.org/drm/nova/-/issues | 8250 | B: https://gitlab.freedesktop.org/drm/nova/-/issues |
| 8238 | C: irc://irc.oftc.net/nouveau | 8251 | C: irc://irc.oftc.net/nouveau |
| 8239 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next | 8252 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next |
| @@ -10848,7 +10861,7 @@ F: include/linux/generic-radix-tree.h · 2608 unmodified lines | |||
| 10848 | F: lib/generic-radix-tree.c | 10861 | F: lib/generic-radix-tree.c |
| 10849 | 10862 | ||
| 10850 | GENERIC RESISTIVE TOUCHSCREEN ADC DRIVER | 10863 | GENERIC RESISTIVE TOUCHSCREEN ADC DRIVER |
| 10851 | M: Eugen Hristev <eugen.hristev@microchip.com> | 10864 | M: Eugen Hristev <ehristev@kernel.org> |
| 10852 | L: linux-input@vger.kernel.org | 10865 | L: linux-input@vger.kernel.org |
| 10853 | S: Maintained | 10866 | S: Maintained |
| 10854 | F: drivers/input/touchscreen/resistive-adc-touch.c | 10867 | F: drivers/input/touchscreen/resistive-adc-touch.c |
| @@ -12071,7 +12084,7 @@ F: Documentation/i2c/busses/i2c-nvidia-gpu.rst · 1216 unmodified lines | |||
| 12071 | F: drivers/i2c/busses/i2c-nvidia-gpu.c | 12084 | F: drivers/i2c/busses/i2c-nvidia-gpu.c |
| 12072 | 12085 | ||
| 12073 | I2C MUXES | 12086 | I2C MUXES |
| 12074 | M: Peter Rosin <peda@axentia.se> | 12087 | M: Peter Rosin <peda@lysator.liu.se> |
| 12075 | L: linux-i2c@vger.kernel.org | 12088 | L: linux-i2c@vger.kernel.org |
| 12076 | S: Maintained | 12089 | S: Maintained |
| 12077 | F: Documentation/devicetree/bindings/i2c/i2c-arb* | 12090 | F: Documentation/devicetree/bindings/i2c/i2c-arb* |
| @@ -12472,7 +12485,7 @@ F: drivers/iio/industrialio-backend.c · 394 unmodified lines | |||
| 12472 | F: include/linux/iio/backend.h | 12485 | F: include/linux/iio/backend.h |
| 12473 | 12486 | ||
| 12474 | IIO DIGITAL POTENTIOMETER DAC | 12487 | IIO DIGITAL POTENTIOMETER DAC |
| 12475 | M: Peter Rosin <peda@axentia.se> | 12488 | M: Peter Rosin <peda@lysator.liu.se> |
| 12476 | L: linux-iio@vger.kernel.org | 12489 | L: linux-iio@vger.kernel.org |
| 12477 | S: Maintained | 12490 | S: Maintained |
| 12478 | F: Documentation/ABI/testing/sysfs-bus-iio-dac-dpot-dac | 12491 | F: Documentation/ABI/testing/sysfs-bus-iio-dac-dpot-dac |
| @@ -12480,7 +12493,7 @@ F: Documentation/devicetree/bindings/iio/dac/dpot-dac.yaml · 1 unmodified line | |||
| 12480 | F: drivers/iio/dac/dpot-dac.c | 12493 | F: drivers/iio/dac/dpot-dac.c |
| 12481 | 12494 | ||
| 12482 | IIO ENVELOPE DETECTOR | 12495 | IIO ENVELOPE DETECTOR |
| 12483 | M: Peter Rosin <peda@axentia.se> | 12496 | M: Peter Rosin <peda@lysator.liu.se> |
| 12484 | L: linux-iio@vger.kernel.org | 12497 | L: linux-iio@vger.kernel.org |
| 12485 | S: Maintained | 12498 | S: Maintained |
| 12486 | F: Documentation/ABI/testing/sysfs-bus-iio-adc-envelope-detector | 12499 | F: Documentation/ABI/testing/sysfs-bus-iio-adc-envelope-detector |
| @@ -12496,7 +12509,7 @@ F: include/linux/iio/iio-gts-helper.h · 9 unmodified lines | |||
| 12496 | F: drivers/iio/test/iio-test-gts.c | 12509 | F: drivers/iio/test/iio-test-gts.c |
| 12497 | 12510 | ||
| 12498 | IIO MULTIPLEXER | 12511 | IIO MULTIPLEXER |
| 12499 | M: Peter Rosin <peda@axentia.se> | 12512 | M: Peter Rosin <peda@lysator.liu.se> |
| 12500 | L: linux-iio@vger.kernel.org | 12513 | L: linux-iio@vger.kernel.org |
| 12501 | S: Maintained | 12514 | S: Maintained |
| 12502 | F: Documentation/devicetree/bindings/iio/multiplexer/io-channel-mux.yaml | 12515 | F: Documentation/devicetree/bindings/iio/multiplexer/io-channel-mux.yaml |
| @@ -12528,7 +12541,7 @@ F: include/linux/iio/ · 25 unmodified lines | |||
| 12528 | F: tools/iio/ | 12541 | F: tools/iio/ |
| 12529 | 12542 | ||
| 12530 | IIO UNIT CONVERTER | 12543 | IIO UNIT CONVERTER |
| 12531 | M: Peter Rosin <peda@axentia.se> | 12544 | M: Peter Rosin <peda@lysator.liu.se> |
| 12532 | L: linux-iio@vger.kernel.org | 12545 | L: linux-iio@vger.kernel.org |
| 12533 | S: Maintained | 12546 | S: Maintained |
| 12534 | F: Documentation/devicetree/bindings/iio/afe/current-sense-amplifier.yaml | 12547 | F: Documentation/devicetree/bindings/iio/afe/current-sense-amplifier.yaml |
| @@ -12805,7 +12818,6 @@ M: Cezary Rojewski <cezary.rojewski@intel.com> · 270 unmodified lines | |||
| 12805 | M: Liam Girdwood <liam.r.girdwood@linux.intel.com> | 12818 | M: Liam Girdwood <liam.r.girdwood@linux.intel.com> |
| 12806 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> | 12819 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> |
| 12807 | M: Bard Liao <yung-chuan.liao@linux.intel.com> | 12820 | M: Bard Liao <yung-chuan.liao@linux.intel.com> |
| 12808 | M: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> | ||
| 12809 | M: Kai Vehmanen <kai.vehmanen@linux.intel.com> | 12821 | M: Kai Vehmanen <kai.vehmanen@linux.intel.com> |
| 12810 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> | 12822 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> |
| 12811 | L: linux-sound@vger.kernel.org | 12823 | L: linux-sound@vger.kernel.org |
| @@ -13498,7 +13510,7 @@ F: include/linux/iommu-dma.h · 686 unmodified lines | |||
| 13498 | F: include/linux/iova.h | 13510 | F: include/linux/iova.h |
| 13499 | 13511 | ||
| 13500 | IOMMU SUBSYSTEM | 13512 | IOMMU SUBSYSTEM |
| 13501 | M: Joerg Roedel <joro@8bytes.org> | 13513 | M: Joerg Roedel (AMD) <joro@8bytes.org> |
| 13502 | M: Will Deacon <will@kernel.org> | 13514 | M: Will Deacon <will@kernel.org> |
| 13503 | R: Robin Murphy <robin.murphy@arm.com> | 13515 | R: Robin Murphy <robin.murphy@arm.com> |
| 13504 | L: iommu@lists.linux.dev | 13516 | L: iommu@lists.linux.dev |
| @@ -13886,7 +13898,7 @@ M: Pratyush Yadav <pratyush@kernel.org> · 381 unmodified lines | |||
| 13886 | R: Dave Young <ruirui.yang@linux.dev> | 13898 | R: Dave Young <ruirui.yang@linux.dev> |
| 13887 | L: kexec@lists.infradead.org | 13899 | L: kexec@lists.infradead.org |
| 13888 | S: Maintained | 13900 | S: Maintained |
| 13889 | W: http://lse.sourceforge.net/kdump/ | 13901 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git |
| 13890 | F: Documentation/admin-guide/kdump/ | 13902 | F: Documentation/admin-guide/kdump/ |
| 13891 | F: fs/proc/vmcore.c | 13903 | F: fs/proc/vmcore.c |
| 13892 | F: include/linux/crash_core.h | 13904 | F: include/linux/crash_core.h |
| @@ -14079,6 +14091,7 @@ KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) · 186 unmodified lines | |||
| 14079 | M: Marc Zyngier <maz@kernel.org> | 14091 | M: Marc Zyngier <maz@kernel.org> |
| 14080 | M: Oliver Upton <oupton@kernel.org> | 14092 | M: Oliver Upton <oupton@kernel.org> |
| 14081 | R: Joey Gouly <joey.gouly@arm.com> | 14093 | R: Joey Gouly <joey.gouly@arm.com> |
| 14094 | R: Steffen Eiden <seiden@linux.ibm.com> | ||
| 14082 | R: Suzuki K Poulose <suzuki.poulose@arm.com> | 14095 | R: Suzuki K Poulose <suzuki.poulose@arm.com> |
| 14083 | R: Zenghui Yu <yuzenghui@huawei.com> | 14096 | R: Zenghui Yu <yuzenghui@huawei.com> |
| 14084 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 14097 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| @@ -14203,6 +14216,7 @@ M: Pasha Tatashin <pasha.tatashin@soleen.com> · 118 unmodified lines | |||
| 14203 | M: Pratyush Yadav <pratyush@kernel.org> | 14216 | M: Pratyush Yadav <pratyush@kernel.org> |
| 14204 | L: kexec@lists.infradead.org | 14217 | L: kexec@lists.infradead.org |
| 14205 | W: http://kernel.org/pub/linux/utils/kernel/kexec/ | 14218 | W: http://kernel.org/pub/linux/utils/kernel/kexec/ |
| 14219 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git | ||
| 14206 | F: include/linux/kexec.h | 14220 | F: include/linux/kexec.h |
| 14207 | F: include/uapi/linux/kexec.h | 14221 | F: include/uapi/linux/kexec.h |
| 14208 | F: kernel/kexec* | 14222 | F: kernel/kexec* |
| @@ -14919,6 +14933,7 @@ LIVE UPDATE · 710 unmodified lines | |||
| 14919 | M: Pasha Tatashin <pasha.tatashin@soleen.com> | 14933 | M: Pasha Tatashin <pasha.tatashin@soleen.com> |
| 14920 | M: Mike Rapoport <rppt@kernel.org> | 14934 | M: Mike Rapoport <rppt@kernel.org> |
| 14921 | M: Pratyush Yadav <pratyush@kernel.org> | 14935 | M: Pratyush Yadav <pratyush@kernel.org> |
| 14936 | L: kexec@lists.infradead.org | ||
| 14922 | L: linux-kernel@vger.kernel.org | 14937 | L: linux-kernel@vger.kernel.org |
| 14923 | S: Maintained | 14938 | S: Maintained |
| 14924 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git | 14939 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git |
| @@ -15278,7 +15293,7 @@ M: Andrea Cervesato <andrea.cervesato@suse.com> · 353 unmodified lines | |||
| 15278 | M: Cyril Hrubis <chrubis@suse.cz> | 15293 | M: Cyril Hrubis <chrubis@suse.cz> |
| 15279 | M: Jan Stancek <jstancek@redhat.com> | 15294 | M: Jan Stancek <jstancek@redhat.com> |
| 15280 | M: Petr Vorel <pvorel@suse.cz> | 15295 | M: Petr Vorel <pvorel@suse.cz> |
| 15281 | M: Li Wang <liwang@redhat.com> | 15296 | M: Li Wang <li.wang@linux.dev> |
| 15282 | M: Yang Xu <xuyang2018.jy@fujitsu.com> | 15297 | M: Yang Xu <xuyang2018.jy@fujitsu.com> |
| 15283 | M: Xiao Yang <yangx.jy@fujitsu.com> | 15298 | M: Xiao Yang <yangx.jy@fujitsu.com> |
| 15284 | L: ltp@lists.linux.it (subscribers-only) | 15299 | L: ltp@lists.linux.it (subscribers-only) |
| @@ -15425,7 +15440,7 @@ F: include/net/netns/mctp.h · 140 unmodified lines | |||
| 15425 | F: net/mctp/ | 15440 | F: net/mctp/ |
| 15426 | 15441 | ||
| 15427 | MAPLE TREE | 15442 | MAPLE TREE |
| 15428 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 15443 | M: Liam R. Howlett <liam@infradead.org> |
| 15429 | R: Alice Ryhl <aliceryhl@google.com> | 15444 | R: Alice Ryhl <aliceryhl@google.com> |
| 15430 | R: Andrew Ballance <andrewjballance@gmail.com> | 15445 | R: Andrew Ballance <andrewjballance@gmail.com> |
| 15431 | L: maple-tree@lists.infradead.org | 15446 | L: maple-tree@lists.infradead.org |
| @@ -15745,7 +15760,7 @@ F: Documentation/devicetree/bindings/media/i2c/maxim,max96717.yaml · 313 unmodified lines | |||
| 15745 | F: drivers/media/i2c/max96717.c | 15760 | F: drivers/media/i2c/max96717.c |
| 15746 | 15761 | ||
| 15747 | MAX9860 MONO AUDIO VOICE CODEC DRIVER | 15762 | MAX9860 MONO AUDIO VOICE CODEC DRIVER |
| 15748 | M: Peter Rosin <peda@axentia.se> | 15763 | M: Peter Rosin <peda@lysator.liu.se> |
| 15749 | L: linux-sound@vger.kernel.org | 15764 | L: linux-sound@vger.kernel.org |
| 15750 | S: Maintained | 15765 | S: Maintained |
| 15751 | F: Documentation/devicetree/bindings/sound/max9860.txt | 15766 | F: Documentation/devicetree/bindings/sound/max9860.txt |
| @@ -15960,7 +15975,7 @@ F: Documentation/devicetree/bindings/net/can/microchip,mcp251xfd.yaml · 208 unmodified lines | |||
| 15960 | F: drivers/net/can/spi/mcp251xfd/ | 15975 | F: drivers/net/can/spi/mcp251xfd/ |
| 15961 | 15976 | ||
| 15962 | MCP4018 AND MCP4531 MICROCHIP DIGITAL POTENTIOMETER DRIVERS | 15977 | MCP4018 AND MCP4531 MICROCHIP DIGITAL POTENTIOMETER DRIVERS |
| 15963 | M: Peter Rosin <peda@axentia.se> | 15978 | M: Peter Rosin <peda@lysator.liu.se> |
| 15964 | L: linux-iio@vger.kernel.org | 15979 | L: linux-iio@vger.kernel.org |
| 15965 | S: Maintained | 15980 | S: Maintained |
| 15966 | F: Documentation/ABI/testing/sysfs-bus-iio-potentiometer-mcp4531 | 15981 | F: Documentation/ABI/testing/sysfs-bus-iio-potentiometer-mcp4531 |
| @@ -16518,7 +16533,7 @@ F: drivers/usb/mtu3/ · 551 unmodified lines | |||
| 16518 | 16533 | ||
| 16519 | MEGACHIPS STDPXXXX-GE-B850V3-FW LVDS/DP++ BRIDGES | 16534 | MEGACHIPS STDPXXXX-GE-B850V3-FW LVDS/DP++ BRIDGES |
| 16520 | M: Peter Senna Tschudin <peter.senna@gmail.com> | 16535 | M: Peter Senna Tschudin <peter.senna@gmail.com> |
| 16521 | M: Ian Ray <ian.ray@ge.com> | 16536 | M: Ian Ray <ian.ray@gehealthcare.com> |
| 16522 | M: Martyn Welch <martyn.welch@collabora.co.uk> | 16537 | M: Martyn Welch <martyn.welch@collabora.co.uk> |
| 16523 | S: Maintained | 16538 | S: Maintained |
| 16524 | F: Documentation/devicetree/bindings/display/bridge/megachips-stdpxxxx-ge-b850v3-fw.txt | 16539 | F: Documentation/devicetree/bindings/display/bridge/megachips-stdpxxxx-ge-b850v3-fw.txt |
| @@ -16785,7 +16800,7 @@ MEMORY MANAGEMENT - CORE · 260 unmodified lines | |||
| 16785 | M: Andrew Morton <akpm@linux-foundation.org> | 16800 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16786 | M: David Hildenbrand <david@kernel.org> | 16801 | M: David Hildenbrand <david@kernel.org> |
| 16787 | R: Lorenzo Stoakes <ljs@kernel.org> | 16802 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16788 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16803 | R: Liam R. Howlett <liam@infradead.org> |
| 16789 | R: Vlastimil Babka <vbabka@kernel.org> | 16804 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16790 | R: Mike Rapoport <rppt@kernel.org> | 16805 | R: Mike Rapoport <rppt@kernel.org> |
| 16791 | R: Suren Baghdasaryan <surenb@google.com> | 16806 | R: Suren Baghdasaryan <surenb@google.com> |
| @@ -16831,7 +16846,7 @@ F: mm/sparse.c · 39 unmodified lines | |||
| 16831 | F: mm/util.c | 16846 | F: mm/util.c |
| 16832 | F: mm/vmpressure.c | 16847 | F: mm/vmpressure.c |
| 16833 | F: mm/vmstat.c | 16848 | F: mm/vmstat.c |
| 16834 | N: include/linux/page[-_]* | 16849 | N: include\/linux\/page[-_][a-zA-Z]* |
| 16835 | 16850 | ||
| 16836 | MEMORY MANAGEMENT - EXECMEM | 16851 | MEMORY MANAGEMENT - EXECMEM |
| 16837 | M: Andrew Morton <akpm@linux-foundation.org> | 16852 | M: Andrew Morton <akpm@linux-foundation.org> |
| @@ -16921,7 +16936,7 @@ MEMORY MANAGEMENT - MISC · 83 unmodified lines | |||
| 16921 | M: Andrew Morton <akpm@linux-foundation.org> | 16936 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16922 | M: David Hildenbrand <david@kernel.org> | 16937 | M: David Hildenbrand <david@kernel.org> |
| 16923 | R: Lorenzo Stoakes <ljs@kernel.org> | 16938 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16924 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 16939 | R: Liam R. Howlett <liam@infradead.org> |
| 16925 | R: Vlastimil Babka <vbabka@kernel.org> | 16940 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16926 | R: Mike Rapoport <rppt@kernel.org> | 16941 | R: Mike Rapoport <rppt@kernel.org> |
| 16927 | R: Suren Baghdasaryan <surenb@google.com> | 16942 | R: Suren Baghdasaryan <surenb@google.com> |
| @@ -16988,6 +17003,7 @@ S: Maintained · 60 unmodified lines | |||
| 16988 | F: include/linux/compaction.h | 17003 | F: include/linux/compaction.h |
| 16989 | F: include/linux/gfp.h | 17004 | F: include/linux/gfp.h |
| 16990 | F: include/linux/page-isolation.h | 17005 | F: include/linux/page-isolation.h |
| 17006 | F: include/linux/pageblock-flags.h | ||
| 16991 | F: mm/compaction.c | 17007 | F: mm/compaction.c |
| 16992 | F: mm/debug_page_alloc.c | 17008 | F: mm/debug_page_alloc.c |
| 16993 | F: mm/debug_page_ref.c | 17009 | F: mm/debug_page_ref.c |
| @@ -17009,7 +17025,7 @@ M: Andrew Morton <akpm@linux-foundation.org> · 15 unmodified lines | |||
| 17009 | M: Johannes Weiner <hannes@cmpxchg.org> | 17025 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 17010 | R: David Hildenbrand <david@kernel.org> | 17026 | R: David Hildenbrand <david@kernel.org> |
| 17011 | R: Michal Hocko <mhocko@kernel.org> | 17027 | R: Michal Hocko <mhocko@kernel.org> |
| 17012 | R: Qi Zheng <zhengqi.arch@bytedance.com> | 17028 | R: Qi Zheng <qi.zheng@linux.dev> |
| 17013 | R: Shakeel Butt <shakeel.butt@linux.dev> | 17029 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 17014 | R: Lorenzo Stoakes <ljs@kernel.org> | 17030 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 17015 | L: linux-mm@kvack.org | 17031 | L: linux-mm@kvack.org |
| @@ -17022,7 +17038,7 @@ M: Andrew Morton <akpm@linux-foundation.org> · 6 unmodified lines | |||
| 17022 | M: David Hildenbrand <david@kernel.org> | 17038 | M: David Hildenbrand <david@kernel.org> |
| 17023 | M: Lorenzo Stoakes <ljs@kernel.org> | 17039 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17024 | R: Rik van Riel <riel@surriel.com> | 17040 | R: Rik van Riel <riel@surriel.com> |
| 17025 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 17041 | R: Liam R. Howlett <liam@infradead.org> |
| 17026 | R: Vlastimil Babka <vbabka@kernel.org> | 17042 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17027 | R: Harry Yoo <harry@kernel.org> | 17043 | R: Harry Yoo <harry@kernel.org> |
| 17028 | R: Jann Horn <jannh@google.com> | 17044 | R: Jann Horn <jannh@google.com> |
| @@ -17069,7 +17085,7 @@ M: David Hildenbrand <david@kernel.org> · 40 unmodified lines | |||
| 17069 | M: Lorenzo Stoakes <ljs@kernel.org> | 17085 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17070 | R: Zi Yan <ziy@nvidia.com> | 17086 | R: Zi Yan <ziy@nvidia.com> |
| 17071 | R: Baolin Wang <baolin.wang@linux.alibaba.com> | 17087 | R: Baolin Wang <baolin.wang@linux.alibaba.com> |
| 17072 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 17088 | R: Liam R. Howlett <liam@infradead.org> |
| 17073 | R: Nico Pache <npache@redhat.com> | 17089 | R: Nico Pache <npache@redhat.com> |
| 17074 | R: Ryan Roberts <ryan.roberts@arm.com> | 17090 | R: Ryan Roberts <ryan.roberts@arm.com> |
| 17075 | R: Dev Jain <dev.jain@arm.com> | 17091 | R: Dev Jain <dev.jain@arm.com> |
| @@ -17107,7 +17123,7 @@ F: tools/testing/selftests/mm/uffd-*.[ch] · 31 unmodified lines | |||
| 17107 | MEMORY MANAGEMENT - RUST | 17123 | MEMORY MANAGEMENT - RUST |
| 17108 | M: Alice Ryhl <aliceryhl@google.com> | 17124 | M: Alice Ryhl <aliceryhl@google.com> |
| 17109 | R: Lorenzo Stoakes <ljs@kernel.org> | 17125 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 17110 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 17126 | R: Liam R. Howlett <liam@infradead.org> |
| 17111 | L: linux-mm@kvack.org | 17127 | L: linux-mm@kvack.org |
| 17112 | L: rust-for-linux@vger.kernel.org | 17128 | L: rust-for-linux@vger.kernel.org |
| 17113 | S: Maintained | 17129 | S: Maintained |
| @@ -17121,7 +17137,7 @@ F: rust/kernel/page.rs · 7 unmodified lines | |||
| 17121 | 17137 | ||
| 17122 | MEMORY MAPPING | 17138 | MEMORY MAPPING |
| 17123 | M: Andrew Morton <akpm@linux-foundation.org> | 17139 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17124 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 17140 | M: Liam R. Howlett <liam@infradead.org> |
| 17125 | M: Lorenzo Stoakes <ljs@kernel.org> | 17141 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17126 | R: Vlastimil Babka <vbabka@kernel.org> | 17142 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17127 | R: Jann Horn <jannh@google.com> | 17143 | R: Jann Horn <jannh@google.com> |
| @@ -17153,7 +17169,7 @@ F: tools/testing/vma/ · 25 unmodified lines | |||
| 17153 | MEMORY MAPPING - LOCKING | 17169 | MEMORY MAPPING - LOCKING |
| 17154 | M: Andrew Morton <akpm@linux-foundation.org> | 17170 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17155 | M: Suren Baghdasaryan <surenb@google.com> | 17171 | M: Suren Baghdasaryan <surenb@google.com> |
| 17156 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 17172 | M: Liam R. Howlett <liam@infradead.org> |
| 17157 | M: Lorenzo Stoakes <ljs@kernel.org> | 17173 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17158 | R: Vlastimil Babka <vbabka@kernel.org> | 17174 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17159 | R: Shakeel Butt <shakeel.butt@linux.dev> | 17175 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| @@ -17168,7 +17184,7 @@ F: mm/mmap_lock.c · 8 unmodified lines | |||
| 17168 | 17184 | ||
| 17169 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) | 17185 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) |
| 17170 | M: Andrew Morton <akpm@linux-foundation.org> | 17186 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17171 | M: Liam R. Howlett <Liam.Howlett@oracle.com> | 17187 | M: Liam R. Howlett <liam@infradead.org> |
| 17172 | M: Lorenzo Stoakes <ljs@kernel.org> | 17188 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17173 | M: David Hildenbrand <david@kernel.org> | 17189 | M: David Hildenbrand <david@kernel.org> |
| 17174 | R: Vlastimil Babka <vbabka@kernel.org> | 17190 | R: Vlastimil Babka <vbabka@kernel.org> |
| @@ -17363,7 +17379,7 @@ F: Documentation/devicetree/bindings/sound/mikroe,mikroe-proto.txt · 188 unmodified lines | |||
| 17363 | F: sound/soc/atmel | 17379 | F: sound/soc/atmel |
| 17364 | 17380 | ||
| 17365 | MICROCHIP CSI2DC DRIVER | 17381 | MICROCHIP CSI2DC DRIVER |
| 17366 | M: Eugen Hristev <eugen.hristev@microchip.com> | 17382 | M: Eugen Hristev <ehristev@kernel.org> |
| 17367 | L: linux-media@vger.kernel.org | 17383 | L: linux-media@vger.kernel.org |
| 17368 | S: Supported | 17384 | S: Supported |
| 17369 | F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml | 17385 | F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml |
| @@ -17390,7 +17406,7 @@ F: drivers/i2c/busses/i2c-at91-*.c · 20 unmodified lines | |||
| 17390 | F: drivers/i2c/busses/i2c-at91.h | 17406 | F: drivers/i2c/busses/i2c-at91.h |
| 17391 | 17407 | ||
| 17392 | MICROCHIP ISC DRIVER | 17408 | MICROCHIP ISC DRIVER |
| 17393 | M: Eugen Hristev <eugen.hristev@microchip.com> | 17409 | M: Eugen Hristev <ehristev@kernel.org> |
| 17394 | L: linux-media@vger.kernel.org | 17410 | L: linux-media@vger.kernel.org |
| 17395 | S: Supported | 17411 | S: Supported |
| 17396 | F: Documentation/devicetree/bindings/media/atmel,isc.yaml | 17412 | F: Documentation/devicetree/bindings/media/atmel,isc.yaml |
| @@ -17402,7 +17418,7 @@ F: drivers/staging/media/deprecated/atmel/atmel-sama*-isc* · 5 unmodified lines | |||
| 17402 | F: include/linux/atmel-isc-media.h | 17418 | F: include/linux/atmel-isc-media.h |
| 17403 | 17419 | ||
| 17404 | MICROCHIP ISI DRIVER | 17420 | MICROCHIP ISI DRIVER |
| 17405 | M: Eugen Hristev <eugen.hristev@microchip.com> | 17421 | M: Eugen Hristev <ehristev@kernel.org> |
| 17406 | L: linux-media@vger.kernel.org | 17422 | L: linux-media@vger.kernel.org |
| 17407 | S: Supported | 17423 | S: Supported |
| 17408 | F: drivers/media/platform/atmel/atmel-isi.c | 17424 | F: drivers/media/platform/atmel/atmel-isi.c |
| @@ -17592,7 +17608,7 @@ F: Documentation/devicetree/bindings/display/bridge/microchip,sam9x75-lvds.yaml · 183 unmodified lines | |||
| 17592 | F: drivers/gpu/drm/bridge/microchip-lvds.c | 17608 | F: drivers/gpu/drm/bridge/microchip-lvds.c |
| 17593 | 17609 | ||
| 17594 | MICROCHIP SAMA5D2-COMPATIBLE ADC DRIVER | 17610 | MICROCHIP SAMA5D2-COMPATIBLE ADC DRIVER |
| 17595 | M: Eugen Hristev <eugen.hristev@microchip.com> | 17611 | M: Eugen Hristev <ehristev@kernel.org> |
| 17596 | L: linux-iio@vger.kernel.org | 17612 | L: linux-iio@vger.kernel.org |
| 17597 | S: Supported | 17613 | S: Supported |
| 17598 | F: Documentation/devicetree/bindings/iio/adc/atmel,sama5d2-adc.yaml | 17614 | F: Documentation/devicetree/bindings/iio/adc/atmel,sama5d2-adc.yaml |
| @@ -18271,7 +18287,7 @@ F: include/linux/mmc/ · 672 unmodified lines | |||
| 18271 | F: include/uapi/linux/mmc/ | 18287 | F: include/uapi/linux/mmc/ |
| 18272 | 18288 | ||
| 18273 | MULTIPLEXER SUBSYSTEM | 18289 | MULTIPLEXER SUBSYSTEM |
| 18274 | M: Peter Rosin <peda@axentia.se> | 18290 | M: Peter Rosin <peda@lysator.liu.se> |
| 18275 | S: Odd Fixes | 18291 | S: Odd Fixes |
| 18276 | F: Documentation/ABI/testing/sysfs-class-mux* | 18292 | F: Documentation/ABI/testing/sysfs-class-mux* |
| 18277 | F: Documentation/devicetree/bindings/mux/ | 18293 | F: Documentation/devicetree/bindings/mux/ |
| @@ -18655,6 +18671,7 @@ F: tools/testing/selftests/net/ · 377 unmodified lines | |||
| 18655 | X: Documentation/networking/mac80211-injection.rst | 18671 | X: Documentation/networking/mac80211-injection.rst |
| 18656 | X: Documentation/networking/mac80211_hwsim/ | 18672 | X: Documentation/networking/mac80211_hwsim/ |
| 18657 | X: Documentation/networking/regulatory.rst | 18673 | X: Documentation/networking/regulatory.rst |
| 18674 | X: include/net/bluetooth/ | ||
| 18658 | X: include/net/cfg80211.h | 18675 | X: include/net/cfg80211.h |
| 18659 | X: include/net/ieee80211_radiotap.h | 18676 | X: include/net/ieee80211_radiotap.h |
| 18660 | X: include/net/iw_handler.h | 18677 | X: include/net/iw_handler.h |
| @@ -18705,19 +18722,59 @@ F: net/xfrm/ · 44 unmodified lines | |||
| 18705 | F: tools/testing/selftests/net/ipsec.c | 18722 | F: tools/testing/selftests/net/ipsec.c |
| 18706 | 18723 | ||
| 18707 | NETWORKING [IPv4/IPv6] | 18724 | NETWORKING [IPv4/IPv6] |
| 18708 | M: "David S. Miller" <davem@davemloft.net> | ||
| 18709 | M: David Ahern <dsahern@kernel.org> | 18725 | M: David Ahern <dsahern@kernel.org> |
| 18726 | M: Ido Schimmel <idosch@nvidia.com> | ||
| 18710 | L: netdev@vger.kernel.org | 18727 | L: netdev@vger.kernel.org |
| 18711 | S: Maintained | 18728 | S: Maintained |
| 18712 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git | 18729 | F: Documentation/netlink/specs/rt-addr.yaml |
| 18713 | F: arch/x86/net/* | 18730 | F: Documentation/netlink/specs/rt-neigh.yaml |
| 18714 | F: include/linux/ip.h | 18731 | F: Documentation/netlink/specs/rt-route.yaml |
| 18715 | F: include/linux/ipv6* | 18732 | F: Documentation/netlink/specs/rt-rule.yaml |
| 18733 | F: include/linux/inetdevice.h | ||
| 18734 | F: include/linux/mroute* | ||
| 18735 | F: include/net/addrconf.h | ||
| 18736 | F: include/net/arp.h | ||
| 18716 | F: include/net/fib* | 18737 | F: include/net/fib* |
| 18738 | F: include/net/if_inet6.h | ||
| 18739 | F: include/net/inetpeer.h | ||
| 18717 | F: include/net/ip* | 18740 | F: include/net/ip* |
| 18741 | F: include/net/lwtunnel.h | ||
| 18742 | F: include/net/ndisc.h | ||
| 18743 | F: include/net/netns/nexthop.h | ||
| 18744 | F: include/net/nexthop.h | ||
| 18718 | F: include/net/route.h | 18745 | F: include/net/route.h |
| 18719 | F: net/ipv4/ | 18746 | F: include/uapi/linux/fib_rules.h |
| 18720 | F: net/ipv6/ | 18747 | F: include/uapi/linux/in_route.h |
| 18748 | F: include/uapi/linux/mroute* | ||
| 18749 | F: include/uapi/linux/nexthop.h | ||
| 18750 | F: net/core/fib* | ||
| 18751 | F: net/core/lwtunnel.c | ||
| 18752 | F: net/ipv4/arp.c | ||
| 18753 | F: net/ipv4/devinet.c | ||
| 18754 | F: net/ipv4/fib* | ||
| 18755 | F: net/ipv4/icmp.c | ||
| 18756 | F: net/ipv4/igmp.c | ||
| 18757 | F: net/ipv4/inet_fragment.c | ||
| 18758 | F: net/ipv4/inetpeer.c | ||
| 18759 | F: net/ipv4/ip* | ||
| 18760 | F: net/ipv4/metrics.c | ||
| 18761 | F: net/ipv4/netlink.c | ||
| 18762 | F: net/ipv4/nexthop.c | ||
| 18763 | F: net/ipv4/route.c | ||
| 18764 | F: net/ipv6/addr* | ||
| 18765 | F: net/ipv6/anycast.c | ||
| 18766 | F: net/ipv6/exthdrs.c | ||
| 18767 | F: net/ipv6/exthdrs_core.c | ||
| 18768 | F: net/ipv6/fib* | ||
| 18769 | F: net/ipv6/icmp.c | ||
| 18770 | F: net/ipv6/ip* | ||
| 18771 | F: net/ipv6/mcast* | ||
| 18772 | F: net/ipv6/ndisc.c | ||
| 18773 | F: net/ipv6/output_core.c | ||
| 18774 | F: net/ipv6/reassembly.c | ||
| 18775 | F: net/ipv6/route.c | ||
| 18776 | F: tools/testing/selftests/net/fib* | ||
| 18777 | F: tools/testing/selftests/net/forwarding/ | ||
| 18721 | 18778 | ||
| 18722 | NETWORKING [LABELED] (NetLabel, Labeled IPsec, SECMARK) | 18779 | NETWORKING [LABELED] (NetLabel, Labeled IPsec, SECMARK) |
| 18723 | M: Paul Moore <paul@paul-moore.com> | 18780 | M: Paul Moore <paul@paul-moore.com> |
| @@ -18852,18 +18909,11 @@ F: Documentation/networking/net_failover.rst · 128 unmodified lines | |||
| 18852 | F: drivers/net/net_failover.c | 18909 | F: drivers/net/net_failover.c |
| 18853 | F: include/net/net_failover.h | 18910 | F: include/net/net_failover.h |
| 18854 | 18911 | ||
| 18855 | NEXTHOP | ||
| 18856 | M: David Ahern <dsahern@kernel.org> | ||
| 18857 | L: netdev@vger.kernel.org | ||
| 18858 | S: Maintained | ||
| 18859 | F: include/net/netns/nexthop.h | ||
| 18860 | F: include/net/nexthop.h | ||
| 18861 | F: include/uapi/linux/nexthop.h | ||
| 18862 | F: net/ipv4/nexthop.c | ||
| 18863 | |||
| 18864 | NFC SUBSYSTEM | 18912 | NFC SUBSYSTEM |
| 18865 | L: netdev@vger.kernel.org | 18913 | M: David Heidelberg <david+nfc@ixit.cz> |
| 18866 | S: Orphan | 18914 | L: oe-linux-nfc@lists.linux.dev |
| 18915 | S: Maintained | ||
| 18916 | T: git https://codeberg.org/linux-nfc/linux.git | ||
| 18867 | F: Documentation/devicetree/bindings/net/nfc/ | 18917 | F: Documentation/devicetree/bindings/net/nfc/ |
| 18868 | F: drivers/nfc/ | 18918 | F: drivers/nfc/ |
| 18869 | F: include/net/nfc/ | 18919 | F: include/net/nfc/ |
| @@ -18931,7 +18981,8 @@ F: drivers/hid/hid-nintendo* · 61 unmodified lines | |||
| 18931 | 18981 | ||
| 18932 | NIOS2 ARCHITECTURE | 18982 | NIOS2 ARCHITECTURE |
| 18933 | M: Dinh Nguyen <dinguyen@kernel.org> | 18983 | M: Dinh Nguyen <dinguyen@kernel.org> |
| 18934 | S: Maintained | 18984 | M: Simon Schuster <schuster.simon@siemens-energy.com> |
| 18985 | S: Supported | ||
| 18935 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/dinguyen/linux.git | 18986 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/dinguyen/linux.git |
| 18936 | F: arch/nios2/ | 18987 | F: arch/nios2/ |
| 18937 | 18988 | ||
| @@ -19347,7 +19398,7 @@ F: include/dt-bindings/display/tda998x.h · 409 unmodified lines | |||
| 19347 | K: "nxp,tda998x" | 19398 | K: "nxp,tda998x" |
| 19348 | 19399 | ||
| 19349 | NXP TFA9879 DRIVER | 19400 | NXP TFA9879 DRIVER |
| 19350 | M: Peter Rosin <peda@axentia.se> | 19401 | M: Peter Rosin <peda@lysator.liu.se> |
| 19351 | L: linux-sound@vger.kernel.org | 19402 | L: linux-sound@vger.kernel.org |
| 19352 | S: Maintained | 19403 | S: Maintained |
| 19353 | F: Documentation/devicetree/bindings/sound/trivial-codec.yaml | 19404 | F: Documentation/devicetree/bindings/sound/trivial-codec.yaml |
| @@ -19445,7 +19496,6 @@ F: include/misc/ocxl* · 91 unmodified lines | |||
| 19445 | F: include/uapi/misc/ocxl.h | 19496 | F: include/uapi/misc/ocxl.h |
| 19446 | 19497 | ||
| 19447 | OMAP AUDIO SUPPORT | 19498 | OMAP AUDIO SUPPORT |
| 19448 | M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | ||
| 19449 | M: Jarkko Nikula <jarkko.nikula@bitmer.com> | 19499 | M: Jarkko Nikula <jarkko.nikula@bitmer.com> |
| 19450 | L: linux-sound@vger.kernel.org | 19500 | L: linux-sound@vger.kernel.org |
| 19451 | L: linux-omap@vger.kernel.org | 19501 | L: linux-omap@vger.kernel.org |
| @@ -20348,13 +20398,14 @@ F: Documentation/devicetree/bindings/pci/marvell,armada8k-pcie.yaml · 896 unmodified lines | |||
| 20348 | F: drivers/pci/controller/dwc/pcie-armada8k.c | 20398 | F: drivers/pci/controller/dwc/pcie-armada8k.c |
| 20349 | 20399 | ||
| 20350 | PCI DRIVER FOR CADENCE PCIE IP | 20400 | PCI DRIVER FOR CADENCE PCIE IP |
| 20401 | R: Aksh Garg <a-garg7@ti.com> | ||
| 20351 | L: linux-pci@vger.kernel.org | 20402 | L: linux-pci@vger.kernel.org |
| 20352 | S: Orphan | 20403 | S: Orphan |
| 20353 | F: Documentation/devicetree/bindings/pci/cdns,* | 20404 | F: Documentation/devicetree/bindings/pci/cdns,* |
| 20354 | F: drivers/pci/controller/cadence/*cadence* | 20405 | F: drivers/pci/controller/cadence/ |
| 20355 | 20406 | ||
| 20356 | PCI DRIVER FOR CIX Sky1 | 20407 | PCI DRIVER FOR CIX Sky1 |
| 20357 | M: Hans Zhang <hans.zhang@cixtech.com> | 20408 | M: Hans Zhang <18255117159@163.com> |
| 20358 | L: linux-pci@vger.kernel.org | 20409 | L: linux-pci@vger.kernel.org |
| 20359 | S: Maintained | 20410 | S: Maintained |
| 20360 | F: Documentation/devicetree/bindings/pci/cix,sky1-pcie-*.yaml | 20411 | F: Documentation/devicetree/bindings/pci/cix,sky1-pcie-*.yaml |
| @@ -20466,7 +20517,7 @@ F: drivers/pci/controller/plda/pcie-plda-host.c · 105 unmodified lines | |||
| 20466 | F: drivers/pci/controller/plda/pcie-plda.h | 20517 | F: drivers/pci/controller/plda/pcie-plda.h |
| 20467 | 20518 | ||
| 20468 | PCI DRIVER FOR RENESAS R-CAR | 20519 | PCI DRIVER FOR RENESAS R-CAR |
| 20469 | M: Marek Vasut <marek.vasut+renesas@gmail.com> | 20520 | M: Marek Vasut <marek.vasut+renesas@mailbox.org> |
| 20470 | M: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com> | 20521 | M: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com> |
| 20471 | L: linux-pci@vger.kernel.org | 20522 | L: linux-pci@vger.kernel.org |
| 20472 | L: linux-renesas-soc@vger.kernel.org | 20523 | L: linux-renesas-soc@vger.kernel.org |
| @@ -20726,15 +20777,13 @@ F: Documentation/devicetree/bindings/pci/intel,keembay-pcie* · 253 unmodified lines | |||
| 20726 | F: drivers/pci/controller/dwc/pcie-keembay.c | 20777 | F: drivers/pci/controller/dwc/pcie-keembay.c |
| 20727 | 20778 | ||
| 20728 | PCIE DRIVER FOR INTEL LGM GW SOC | 20779 | PCIE DRIVER FOR INTEL LGM GW SOC |
| 20729 | M: Chuanhua Lei <lchuanhua@maxlinear.com> | ||
| 20730 | L: linux-pci@vger.kernel.org | 20780 | L: linux-pci@vger.kernel.org |
| 20731 | S: Maintained | 20781 | S: Orphan |
| 20732 | F: Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml | 20782 | F: Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml |
| 20733 | F: drivers/pci/controller/dwc/pcie-intel-gw.c | 20783 | F: drivers/pci/controller/dwc/pcie-intel-gw.c |
| 20734 | 20784 | ||
| 20735 | PCIE DRIVER FOR MEDIATEK | 20785 | PCIE DRIVER FOR MEDIATEK |
| 20736 | M: Ryder Lee <ryder.lee@mediatek.com> | 20786 | M: Ryder Lee <ryder.lee@mediatek.com> |
| 20737 | M: Jianjun Wang <jianjun.wang@mediatek.com> | ||
| 20738 | L: linux-pci@vger.kernel.org | 20787 | L: linux-pci@vger.kernel.org |
| 20739 | L: linux-mediatek@lists.infradead.org (moderated for non-subscribers) | 20788 | L: linux-mediatek@lists.infradead.org (moderated for non-subscribers) |
| 20740 | S: Supported | 20789 | S: Supported |
| @@ -20807,6 +20856,7 @@ M: Dominik Brodowski <linux@dominikbrodowski.net> · 66 unmodified lines | |||
| 20807 | S: Odd Fixes | 20856 | S: Odd Fixes |
| 20808 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux.git | 20857 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux.git |
| 20809 | F: Documentation/pcmcia/ | 20858 | F: Documentation/pcmcia/ |
| 20859 | F: drivers/net/ethernet/8390/pcnet_cs.c | ||
| 20810 | F: drivers/pcmcia/ | 20860 | F: drivers/pcmcia/ |
| 20811 | F: include/pcmcia/ | 20861 | F: include/pcmcia/ |
| 20812 | F: tools/pcmcia/ | 20862 | F: tools/pcmcia/ |
| @@ -22939,7 +22989,7 @@ N: riscv · 2126 unmodified lines | |||
| 22939 | K: riscv | 22989 | K: riscv |
| 22940 | 22990 | ||
| 22941 | RISC-V IOMMU | 22991 | RISC-V IOMMU |
| 22942 | M: Tomasz Jeznach <tjeznach@rivosinc.com> | 22992 | M: Tomasz Jeznach <tomasz.jeznach@linux.dev> |
| 22943 | L: iommu@lists.linux.dev | 22993 | L: iommu@lists.linux.dev |
| 22944 | L: linux-riscv@lists.infradead.org | 22994 | L: linux-riscv@lists.infradead.org |
| 22945 | S: Maintained | 22995 | S: Maintained |
| @@ -23402,7 +23452,7 @@ RUST [ALLOC] · 456 unmodified lines | |||
| 23402 | M: Danilo Krummrich <dakr@kernel.org> | 23452 | M: Danilo Krummrich <dakr@kernel.org> |
| 23403 | R: Lorenzo Stoakes <ljs@kernel.org> | 23453 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 23404 | R: Vlastimil Babka <vbabka@kernel.org> | 23454 | R: Vlastimil Babka <vbabka@kernel.org> |
| 23405 | R: Liam R. Howlett <Liam.Howlett@oracle.com> | 23455 | R: Liam R. Howlett <liam@infradead.org> |
| 23406 | R: Uladzislau Rezki <urezki@gmail.com> | 23456 | R: Uladzislau Rezki <urezki@gmail.com> |
| 23407 | L: rust-for-linux@vger.kernel.org | 23457 | L: rust-for-linux@vger.kernel.org |
| 23408 | S: Maintained | 23458 | S: Maintained |
| @@ -23554,7 +23604,7 @@ F: drivers/s390/net/ · 145 unmodified lines | |||
| 23554 | 23604 | ||
| 23555 | S390 PCI SUBSYSTEM | 23605 | S390 PCI SUBSYSTEM |
| 23556 | M: Niklas Schnelle <schnelle@linux.ibm.com> | 23606 | M: Niklas Schnelle <schnelle@linux.ibm.com> |
| 23557 | M: Gerald Schaefer <gerald.schaefer@linux.ibm.com> | 23607 | M: Gerd Bayer <gbayer@linux.ibm.com> |
| 23558 | L: linux-s390@vger.kernel.org | 23608 | L: linux-s390@vger.kernel.org |
| 23559 | S: Supported | 23609 | S: Supported |
| 23560 | F: Documentation/arch/s390/pci.rst | 23610 | F: Documentation/arch/s390/pci.rst |
| @@ -24108,7 +24158,7 @@ F: drivers/mmc/host/sdhci* · 547 unmodified lines | |||
| 24108 | 24158 | ||
| 24109 | SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) MICROCHIP DRIVER | 24159 | SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) MICROCHIP DRIVER |
| 24110 | M: Aubin Constans <aubin.constans@microchip.com> | 24160 | M: Aubin Constans <aubin.constans@microchip.com> |
| 24111 | R: Eugen Hristev <eugen.hristev@collabora.com> | 24161 | R: Eugen Hristev <ehristev@kernel.org> |
| 24112 | L: linux-mmc@vger.kernel.org | 24162 | L: linux-mmc@vger.kernel.org |
| 24113 | S: Supported | 24163 | S: Supported |
| 24114 | F: drivers/mmc/host/sdhci-of-at91.c | 24164 | F: drivers/mmc/host/sdhci-of-at91.c |
| @@ -24347,7 +24397,7 @@ F: include/media/i2c/rj54n1cb0c.h · 232 unmodified lines | |||
| 24347 | SHRINKER | 24397 | SHRINKER |
| 24348 | M: Andrew Morton <akpm@linux-foundation.org> | 24398 | M: Andrew Morton <akpm@linux-foundation.org> |
| 24349 | M: Dave Chinner <david@fromorbit.com> | 24399 | M: Dave Chinner <david@fromorbit.com> |
| 24350 | R: Qi Zheng <zhengqi.arch@bytedance.com> | 24400 | R: Qi Zheng <qi.zheng@linux.dev> |
| 24351 | R: Roman Gushchin <roman.gushchin@linux.dev> | 24401 | R: Roman Gushchin <roman.gushchin@linux.dev> |
| 24352 | R: Muchun Song <muchun.song@linux.dev> | 24402 | R: Muchun Song <muchun.song@linux.dev> |
| 24353 | L: linux-mm@kvack.org | 24403 | L: linux-mm@kvack.org |
| @@ -24649,6 +24699,7 @@ S: Maintained · 295 unmodified lines | |||
| 24649 | F: fs/smb/client/smbdirect.* | 24699 | F: fs/smb/client/smbdirect.* |
| 24650 | F: fs/smb/smbdirect/ | 24700 | F: fs/smb/smbdirect/ |
| 24651 | F: fs/smb/server/transport_rdma.* | 24701 | F: fs/smb/server/transport_rdma.* |
| 24702 | F: include/linux/smbdirect.h | ||
| 24652 | 24703 | ||
| 24653 | SMC91x ETHERNET DRIVER | 24704 | SMC91x ETHERNET DRIVER |
| 24654 | M: Nicolas Pitre <nico@fluxnic.net> | 24705 | M: Nicolas Pitre <nico@fluxnic.net> |
| @@ -24797,6 +24848,7 @@ SOFTWARE RAID (Multiple Disks) SUPPORT · 142 unmodified lines | |||
| 24797 | M: Song Liu <song@kernel.org> | 24848 | M: Song Liu <song@kernel.org> |
| 24798 | M: Yu Kuai <yukuai@fnnas.com> | 24849 | M: Yu Kuai <yukuai@fnnas.com> |
| 24799 | R: Li Nan <linan122@huawei.com> | 24850 | R: Li Nan <linan122@huawei.com> |
| 24851 | R: Xiao Ni <xiao@kernel.org> | ||
| 24800 | L: linux-raid@vger.kernel.org | 24852 | L: linux-raid@vger.kernel.org |
| 24801 | S: Supported | 24853 | S: Supported |
| 24802 | Q: https://patchwork.kernel.org/project/linux-raid/list/ | 24854 | Q: https://patchwork.kernel.org/project/linux-raid/list/ |
| @@ -25051,7 +25103,6 @@ SOUND - SOUND OPEN FIRMWARE (SOF) DRIVERS · 248 unmodified lines | |||
| 25051 | M: Liam Girdwood <lgirdwood@gmail.com> | 25103 | M: Liam Girdwood <lgirdwood@gmail.com> |
| 25052 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> | 25104 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> |
| 25053 | M: Bard Liao <yung-chuan.liao@linux.intel.com> | 25105 | M: Bard Liao <yung-chuan.liao@linux.intel.com> |
| 25054 | M: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> | ||
| 25055 | M: Daniel Baluta <daniel.baluta@nxp.com> | 25106 | M: Daniel Baluta <daniel.baluta@nxp.com> |
| 25056 | R: Kai Vehmanen <kai.vehmanen@linux.intel.com> | 25107 | R: Kai Vehmanen <kai.vehmanen@linux.intel.com> |
| 25057 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> | 25108 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> |
| @@ -26344,7 +26395,7 @@ F: arch/xtensa/ · 1286 unmodified lines | |||
| 26344 | F: drivers/irqchip/irq-xtensa-* | 26395 | F: drivers/irqchip/irq-xtensa-* |
| 26345 | 26396 | ||
| 26346 | TEXAS INSTRUMENTS ASoC DRIVERS | 26397 | TEXAS INSTRUMENTS ASoC DRIVERS |
| 26347 | M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | 26398 | M: Sen Wang <sen@ti.com> |
| 26348 | L: linux-sound@vger.kernel.org | 26399 | L: linux-sound@vger.kernel.org |
| 26349 | S: Maintained | 26400 | S: Maintained |
| 26350 | F: Documentation/devicetree/bindings/sound/davinci-mcasp-audio.yaml | 26401 | F: Documentation/devicetree/bindings/sound/davinci-mcasp-audio.yaml |
| @@ -26846,12 +26897,6 @@ S: Maintained · 495 unmodified lines | |||
| 26846 | F: Documentation/devicetree/bindings/iio/adc/ti,tsc2046.yaml | 26897 | F: Documentation/devicetree/bindings/iio/adc/ti,tsc2046.yaml |
| 26847 | F: drivers/iio/adc/ti-tsc2046.c | 26898 | F: drivers/iio/adc/ti-tsc2046.c |
| 26848 | 26899 | ||
| 26849 | TI TWL4030 SERIES SOC CODEC DRIVER | ||
| 26850 | M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | ||
| 26851 | L: linux-sound@vger.kernel.org | ||
| 26852 | S: Maintained | ||
| 26853 | F: sound/soc/codecs/twl4030* | ||
| 26854 | |||
| 26855 | TI VPE/CAL DRIVERS | 26900 | TI VPE/CAL DRIVERS |
| 26856 | M: Yemike Abhilash Chandra <y-abhilashchandra@ti.com> | 26901 | M: Yemike Abhilash Chandra <y-abhilashchandra@ti.com> |
| 26857 | L: linux-media@vger.kernel.org | 26902 | L: linux-media@vger.kernel.org |
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | VERSION = 7 | 2 | VERSION = 7 |
| 3 | PATCHLEVEL = 1 | 3 | PATCHLEVEL = 1 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc1 | 5 | EXTRAVERSION = -rc6 |
| 6 | NAME = Baby Opossum Posse | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| @@ -486,6 +486,8 @@ export rust_common_flags := --edition=2021 \ · 477 unmodified lines | |||
| 486 | -Wclippy::as_ptr_cast_mut \ | 486 | -Wclippy::as_ptr_cast_mut \ |
| 487 | -Wclippy::as_underscore \ | 487 | -Wclippy::as_underscore \ |
| 488 | -Wclippy::cast_lossless \ | 488 | -Wclippy::cast_lossless \ |
| 489 | -Aclippy::collapsible_if \ | ||
| 490 | -Aclippy::collapsible_match \ | ||
| 489 | -Wclippy::ignored_unit_patterns \ | 491 | -Wclippy::ignored_unit_patterns \ |
| 490 | -Aclippy::incompatible_msrv \ | 492 | -Aclippy::incompatible_msrv \ |
| 491 | -Wclippy::mut_mut \ | 493 | -Wclippy::mut_mut \ |
| @@ -5,4 +5,5 @@ generic-y += agp.h · 4 unmodified lines | |||
| 5 | generic-y += asm-offsets.h | 5 | generic-y += asm-offsets.h |
| 6 | generic-y += kvm_para.h | 6 | generic-y += kvm_para.h |
| 7 | generic-y += mcs_spinlock.h | 7 | generic-y += mcs_spinlock.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += text-patching.h | 9 | generic-y += text-patching.h |
| @@ -5,5 +5,6 @@ generic-y += extable.h · 4 unmodified lines | |||
| 5 | generic-y += kvm_para.h | 5 | generic-y += kvm_para.h |
| 6 | generic-y += mcs_spinlock.h | 6 | generic-y += mcs_spinlock.h |
| 7 | generic-y += parport.h | 7 | generic-y += parport.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += user.h | 9 | generic-y += user.h |
| 9 | generic-y += text-patching.h | 10 | generic-y += text-patching.h |
| @@ -34,9 +34,6 @@ · 33 unmodified lines | |||
| 34 | clocks = <&mstp9_clks R7S72100_CLK_SPIBSC0>; | 34 | clocks = <&mstp9_clks R7S72100_CLK_SPIBSC0>; |
| 35 | power-domains = <&cpg_clocks>; | 35 | power-domains = <&cpg_clocks>; |
| 36 | 36 | ||
| 37 | #address-cells = <1>; | ||
| 38 | #size-cells = <1>; | ||
| 39 | |||
| 40 | partitions { | 37 | partitions { |
| 41 | compatible = "fixed-partitions"; | 38 | compatible = "fixed-partitions"; |
| 42 | #address-cells = <1>; | 39 | #address-cells = <1>; |
| @@ -36,8 +36,6 @@ · 35 unmodified lines | |||
| 36 | power-domains = <&cpg_clocks>; | 36 | power-domains = <&cpg_clocks>; |
| 37 | bank-width = <4>; | 37 | bank-width = <4>; |
| 38 | device-width = <1>; | 38 | device-width = <1>; |
| 39 | #address-cells = <1>; | ||
| 40 | #size-cells = <1>; | ||
| 41 | 39 | ||
| 42 | partitions { | 40 | partitions { |
| 43 | compatible = "fixed-partitions"; | 41 | compatible = "fixed-partitions"; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | clock-div = <3>; | 37 | clock-div = <3>; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | bsc: bus { | 40 | bsc: bus@0 { |
| 41 | compatible = "simple-bus"; | 41 | compatible = "simple-bus"; |
| 42 | #address-cells = <1>; | 42 | #address-cells = <1>; |
| 43 | #size-cells = <1>; | 43 | #size-cells = <1>; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | |||
| 40 | spi2 = &hspi2; | 40 | spi2 = &hspi2; |
| 41 | }; | 41 | }; |
| 42 | 42 | ||
| 43 | lbsc: bus { | 43 | lbsc: bus@0 { |
| 44 | compatible = "simple-bus"; | 44 | compatible = "simple-bus"; |
| 45 | #address-cells = <1>; | 45 | #address-cells = <1>; |
| 46 | #size-cells = <1>; | 46 | #size-cells = <1>; |
| @@ -704,7 +704,7 @@ · 703 unmodified lines | |||
| 704 | }; | 704 | }; |
| 705 | }; | 705 | }; |
| 706 | 706 | ||
| 707 | lbsc: bus { | 707 | lbsc: bus@0 { |
| 708 | compatible = "simple-bus"; | 708 | compatible = "simple-bus"; |
| 709 | #address-cells = <1>; | 709 | #address-cells = <1>; |
| 710 | #size-cells = <1>; | 710 | #size-cells = <1>; |
| @@ -86,7 +86,7 @@ · 85 unmodified lines | |||
| 86 | bootph-all; | 86 | bootph-all; |
| 87 | }; | 87 | }; |
| 88 | 88 | ||
| 89 | lbsc: bus { | 89 | lbsc: bus@0 { |
| 90 | compatible = "simple-bus"; | 90 | compatible = "simple-bus"; |
| 91 | #address-cells = <1>; | 91 | #address-cells = <1>; |
| 92 | #size-cells = <1>; | 92 | #size-cells = <1>; |
| @@ -3,6 +3,7 @@ generic-y += early_ioremap.h · 2 unmodified lines | |||
| 3 | generic-y += extable.h | 3 | generic-y += extable.h |
| 4 | generic-y += flat.h | 4 | generic-y += flat.h |
| 5 | generic-y += parport.h | 5 | generic-y += parport.h |
| 6 | generic-y += ring_buffer.h | ||
| 6 | 7 | ||
| 7 | generated-y += mach-types.h | 8 | generated-y += mach-types.h |
| 8 | generated-y += unistd-nr.h | 9 | generated-y += unistd-nr.h |
| @@ -86,14 +86,6 @@ static u64 notrace intcp_read_sched_clock(void) · 85 unmodified lines | |||
| 86 | return val; | 86 | return val; |
| 87 | } | 87 | } |
| 88 | 88 | ||
| 89 | static void __init intcp_init_early(void) | ||
| 90 | { | ||
| 91 | cm_map = syscon_regmap_lookup_by_compatible("arm,core-module-integrator"); | ||
| 92 | if (IS_ERR(cm_map)) | ||
| 93 | return; | ||
| 94 | sched_clock_register(intcp_read_sched_clock, 32, 24000000); | ||
| 95 | } | ||
| 96 | |||
| 97 | static void __init intcp_init_irq_of(void) | 89 | static void __init intcp_init_irq_of(void) |
| 98 | { | 90 | { |
| 99 | cm_init(); | 91 | cm_init(); |
| @@ -119,6 +111,10 @@ static void __init intcp_init_of(void) · 19 unmodified lines | |||
| 119 | { | 111 | { |
| 120 | struct device_node *cpcon; | 112 | struct device_node *cpcon; |
| 121 | 113 | ||
| 114 | cm_map = syscon_regmap_lookup_by_compatible("arm,core-module-integrator"); | ||
| 115 | if (!IS_ERR(cm_map)) | ||
| 116 | sched_clock_register(intcp_read_sched_clock, 32, 24000000); | ||
| 117 | |||
| 122 | cpcon = of_find_matching_node(NULL, intcp_syscon_match); | 118 | cpcon = of_find_matching_node(NULL, intcp_syscon_match); |
| 123 | if (!cpcon) | 119 | if (!cpcon) |
| 124 | return; | 120 | return; |
| @@ -138,7 +134,6 @@ static const char * intcp_dt_board_compat[] = { · 13 unmodified lines | |||
| 138 | DT_MACHINE_START(INTEGRATOR_CP_DT, "ARM Integrator/CP (Device Tree)") | 134 | DT_MACHINE_START(INTEGRATOR_CP_DT, "ARM Integrator/CP (Device Tree)") |
| 139 | .reserve = integrator_reserve, | 135 | .reserve = integrator_reserve, |
| 140 | .map_io = intcp_map_io, | 136 | .map_io = intcp_map_io, |
| 141 | .init_early = intcp_init_early, | ||
| 142 | .init_irq = intcp_init_irq_of, | 137 | .init_irq = intcp_init_irq_of, |
| 143 | .init_machine = intcp_init_of, | 138 | .init_machine = intcp_init_of, |
| 144 | .dt_compat = intcp_dt_board_compat, | 139 | .dt_compat = intcp_dt_board_compat, |
| @@ -27,7 +27,12 @@ · 26 unmodified lines | |||
| 27 | status = "okay"; | 27 | status = "okay"; |
| 28 | 28 | ||
| 29 | ports { | 29 | ports { |
| 30 | #address-cells = <1>; | ||
| 31 | #size-cells = <0>; | ||
| 32 | |||
| 30 | port@1 { | 33 | port@1 { |
| 34 | reg = <1>; | ||
| 35 | |||
| 31 | lvds1_out: endpoint { | 36 | lvds1_out: endpoint { |
| 32 | remote-endpoint = <&panel_in>; | 37 | remote-endpoint = <&panel_in>; |
| 33 | }; | 38 | }; |
| @@ -699,7 +699,7 @@ · 698 unmodified lines | |||
| 699 | "renesas,rcar-gen5-scif", "renesas,scif"; | 699 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 700 | reg = <0 0xc0700000 0 0x40>; | 700 | reg = <0 0xc0700000 0 0x40>; |
| 701 | interrupts = <GIC_ESPI 10 IRQ_TYPE_LEVEL_HIGH>; | 701 | interrupts = <GIC_ESPI 10 IRQ_TYPE_LEVEL_HIGH>; |
| 702 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | 702 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; |
| 703 | clock-names = "fck", "brg_int", "scif_clk"; | 703 | clock-names = "fck", "brg_int", "scif_clk"; |
| 704 | status = "disabled"; | 704 | status = "disabled"; |
| 705 | }; | 705 | }; |
| @@ -709,7 +709,7 @@ · 3 unmodified lines | |||
| 709 | "renesas,rcar-gen5-scif", "renesas,scif"; | 709 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 710 | reg = <0 0xc0704000 0 0x40>; | 710 | reg = <0 0xc0704000 0 0x40>; |
| 711 | interrupts = <GIC_ESPI 11 IRQ_TYPE_LEVEL_HIGH>; | 711 | interrupts = <GIC_ESPI 11 IRQ_TYPE_LEVEL_HIGH>; |
| 712 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | 712 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; |
| 713 | clock-names = "fck", "brg_int", "scif_clk"; | 713 | clock-names = "fck", "brg_int", "scif_clk"; |
| 714 | status = "disabled"; | 714 | status = "disabled"; |
| 715 | }; | 715 | }; |
| @@ -719,7 +719,7 @@ · 3 unmodified lines | |||
| 719 | "renesas,rcar-gen5-scif", "renesas,scif"; | 719 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 720 | reg = <0 0xc0708000 0 0x40>; | 720 | reg = <0 0xc0708000 0 0x40>; |
| 721 | interrupts = <GIC_ESPI 12 IRQ_TYPE_LEVEL_HIGH>; | 721 | interrupts = <GIC_ESPI 12 IRQ_TYPE_LEVEL_HIGH>; |
| 722 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | 722 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; |
| 723 | clock-names = "fck", "brg_int", "scif_clk"; | 723 | clock-names = "fck", "brg_int", "scif_clk"; |
| 724 | status = "disabled"; | 724 | status = "disabled"; |
| 725 | }; | 725 | }; |
| @@ -729,7 +729,7 @@ · 3 unmodified lines | |||
| 729 | "renesas,rcar-gen5-scif", "renesas,scif"; | 729 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 730 | reg = <0 0xc070c000 0 0x40>; | 730 | reg = <0 0xc070c000 0 0x40>; |
| 731 | interrupts = <GIC_ESPI 13 IRQ_TYPE_LEVEL_HIGH>; | 731 | interrupts = <GIC_ESPI 13 IRQ_TYPE_LEVEL_HIGH>; |
| 732 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | 732 | clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; |
| 733 | clock-names = "fck", "brg_int", "scif_clk"; | 733 | clock-names = "fck", "brg_int", "scif_clk"; |
| 734 | status = "disabled"; | 734 | status = "disabled"; |
| 735 | }; | 735 | }; |
| @@ -1327,6 +1327,7 @@ · 1326 unmodified lines | |||
| 1327 | resets = <&cpg 0xaf>; | 1327 | resets = <&cpg 0xaf>; |
| 1328 | power-domains = <&cpg>; | 1328 | power-domains = <&cpg>; |
| 1329 | #reset-cells = <0>; | 1329 | #reset-cells = <0>; |
| 1330 | #mux-state-cells = <1>; | ||
| 1330 | status = "disabled"; | 1331 | status = "disabled"; |
| 1331 | }; | 1332 | }; |
| 1332 | 1333 | ||
| @@ -1345,6 +1345,7 @@ · 1344 unmodified lines | |||
| 1345 | resets = <&cpg 0xaf>; | 1345 | resets = <&cpg 0xaf>; |
| 1346 | power-domains = <&cpg>; | 1346 | power-domains = <&cpg>; |
| 1347 | #reset-cells = <0>; | 1347 | #reset-cells = <0>; |
| 1348 | #mux-state-cells = <1>; | ||
| 1348 | status = "disabled"; | 1349 | status = "disabled"; |
| 1349 | }; | 1350 | }; |
| 1350 | 1351 | ||
| @@ -1355,6 +1356,7 @@ · 4 unmodified lines | |||
| 1355 | resets = <&cpg 0xaf>; | 1356 | resets = <&cpg 0xaf>; |
| 1356 | power-domains = <&cpg>; | 1357 | power-domains = <&cpg>; |
| 1357 | #reset-cells = <0>; | 1358 | #reset-cells = <0>; |
| 1359 | #mux-state-cells = <1>; | ||
| 1358 | status = "disabled"; | 1360 | status = "disabled"; |
| 1359 | }; | 1361 | }; |
| 1360 | 1362 | ||
| @@ -46,7 +46,12 @@ · 45 unmodified lines | |||
| 46 | status = "okay"; | 46 | status = "okay"; |
| 47 | 47 | ||
| 48 | ports { | 48 | ports { |
| 49 | #address-cells = <1>; | ||
| 50 | #size-cells = <0>; | ||
| 51 | |||
| 49 | port@0 { | 52 | port@0 { |
| 53 | reg = <0>; | ||
| 54 | |||
| 50 | csi2_in: endpoint { | 55 | csi2_in: endpoint { |
| 51 | clock-lanes = <0>; | 56 | clock-lanes = <0>; |
| 52 | data-lanes = <1 2>; | 57 | data-lanes = <1 2>; |
| @@ -26,7 +26,12 @@ · 25 unmodified lines | |||
| 26 | status = "okay"; | 26 | status = "okay"; |
| 27 | 27 | ||
| 28 | ports { | 28 | ports { |
| 29 | #address-cells = <1>; | ||
| 30 | #size-cells = <0>; | ||
| 31 | |||
| 29 | port@0 { | 32 | port@0 { |
| 33 | reg = <0>; | ||
| 34 | |||
| 30 | du_out_rgb: endpoint { | 35 | du_out_rgb: endpoint { |
| 31 | remote-endpoint = <&adv7513_in>; | 36 | remote-endpoint = <&adv7513_in>; |
| 32 | }; | 37 | }; |
| @@ -27,7 +27,12 @@ · 26 unmodified lines | |||
| 27 | status = "okay"; | 27 | status = "okay"; |
| 28 | 28 | ||
| 29 | ports { | 29 | ports { |
| 30 | #address-cells = <1>; | ||
| 31 | #size-cells = <0>; | ||
| 32 | |||
| 30 | port@1 { | 33 | port@1 { |
| 34 | reg = <1>; | ||
| 35 | |||
| 31 | lvds0_out: endpoint { | 36 | lvds0_out: endpoint { |
| 32 | remote-endpoint = <&panel_in>; | 37 | remote-endpoint = <&panel_in>; |
| 33 | }; | 38 | }; |
| @@ -409,7 +409,7 @@ __AARCH64_INSN_FUNCS(cbz, 0x7F000000, 0x34000000) · 408 unmodified lines | |||
| 409 | __AARCH64_INSN_FUNCS(cbnz, 0x7F000000, 0x35000000) | 409 | __AARCH64_INSN_FUNCS(cbnz, 0x7F000000, 0x35000000) |
| 410 | __AARCH64_INSN_FUNCS(tbz, 0x7F000000, 0x36000000) | 410 | __AARCH64_INSN_FUNCS(tbz, 0x7F000000, 0x36000000) |
| 411 | __AARCH64_INSN_FUNCS(tbnz, 0x7F000000, 0x37000000) | 411 | __AARCH64_INSN_FUNCS(tbnz, 0x7F000000, 0x37000000) |
| 412 | __AARCH64_INSN_FUNCS(bcond, 0xFF000010, 0x54000000) | 412 | __AARCH64_INSN_FUNCS(bcond, 0xFF000000, 0x54000000) |
| 413 | __AARCH64_INSN_FUNCS(svc, 0xFFE0001F, 0xD4000001) | 413 | __AARCH64_INSN_FUNCS(svc, 0xFFE0001F, 0xD4000001) |
| 414 | __AARCH64_INSN_FUNCS(hvc, 0xFFE0001F, 0xD4000002) | 414 | __AARCH64_INSN_FUNCS(hvc, 0xFFE0001F, 0xD4000002) |
| 415 | __AARCH64_INSN_FUNCS(smc, 0xFFE0001F, 0xD4000003) | 415 | __AARCH64_INSN_FUNCS(smc, 0xFFE0001F, 0xD4000003) |
| @@ -40,7 +40,7 @@ static __always_inline void __pmr_local_irq_enable(void) · 39 unmodified lines | |||
| 40 | barrier(); | 40 | barrier(); |
| 41 | } | 41 | } |
| 42 | 42 | ||
| 43 | static inline void arch_local_irq_enable(void) | 43 | static __always_inline void arch_local_irq_enable(void) |
| 44 | { | 44 | { |
| 45 | if (system_uses_irq_prio_masking()) { | 45 | if (system_uses_irq_prio_masking()) { |
| 46 | __pmr_local_irq_enable(); | 46 | __pmr_local_irq_enable(); |
| @@ -68,7 +68,7 @@ static __always_inline void __pmr_local_irq_disable(void) · 21 unmodified lines | |||
| 68 | barrier(); | 68 | barrier(); |
| 69 | } | 69 | } |
| 70 | 70 | ||
| 71 | static inline void arch_local_irq_disable(void) | 71 | static __always_inline void arch_local_irq_disable(void) |
| 72 | { | 72 | { |
| 73 | if (system_uses_irq_prio_masking()) { | 73 | if (system_uses_irq_prio_masking()) { |
| 74 | __pmr_local_irq_disable(); | 74 | __pmr_local_irq_disable(); |
| @@ -90,7 +90,7 @@ static __always_inline unsigned long __pmr_local_save_flags(void) · 15 unmodified lines | |||
| 90 | /* | 90 | /* |
| 91 | * Save the current interrupt enable state. | 91 | * Save the current interrupt enable state. |
| 92 | */ | 92 | */ |
| 93 | static inline unsigned long arch_local_save_flags(void) | 93 | static __always_inline unsigned long arch_local_save_flags(void) |
| 94 | { | 94 | { |
| 95 | if (system_uses_irq_prio_masking()) { | 95 | if (system_uses_irq_prio_masking()) { |
| 96 | return __pmr_local_save_flags(); | 96 | return __pmr_local_save_flags(); |
| @@ -109,7 +109,7 @@ static __always_inline bool __pmr_irqs_disabled_flags(unsigned long flags) · 12 unmodified lines | |||
| 109 | return flags != GIC_PRIO_IRQON; | 109 | return flags != GIC_PRIO_IRQON; |
| 110 | } | 110 | } |
| 111 | 111 | ||
| 112 | static inline bool arch_irqs_disabled_flags(unsigned long flags) | 112 | static __always_inline bool arch_irqs_disabled_flags(unsigned long flags) |
| 113 | { | 113 | { |
| 114 | if (system_uses_irq_prio_masking()) { | 114 | if (system_uses_irq_prio_masking()) { |
| 115 | return __pmr_irqs_disabled_flags(flags); | 115 | return __pmr_irqs_disabled_flags(flags); |
| @@ -128,7 +128,7 @@ static __always_inline bool __pmr_irqs_disabled(void) · 12 unmodified lines | |||
| 128 | return __pmr_irqs_disabled_flags(__pmr_local_save_flags()); | 128 | return __pmr_irqs_disabled_flags(__pmr_local_save_flags()); |
| 129 | } | 129 | } |
| 130 | 130 | ||
| 131 | static inline bool arch_irqs_disabled(void) | 131 | static __always_inline bool arch_irqs_disabled(void) |
| 132 | { | 132 | { |
| 133 | if (system_uses_irq_prio_masking()) { | 133 | if (system_uses_irq_prio_masking()) { |
| 134 | return __pmr_irqs_disabled(); | 134 | return __pmr_irqs_disabled(); |
| @@ -160,7 +160,7 @@ static __always_inline unsigned long __pmr_local_irq_save(void) · 25 unmodified lines | |||
| 160 | return flags; | 160 | return flags; |
| 161 | } | 161 | } |
| 162 | 162 | ||
| 163 | static inline unsigned long arch_local_irq_save(void) | 163 | static __always_inline unsigned long arch_local_irq_save(void) |
| 164 | { | 164 | { |
| 165 | if (system_uses_irq_prio_masking()) { | 165 | if (system_uses_irq_prio_masking()) { |
| 166 | return __pmr_local_irq_save(); | 166 | return __pmr_local_irq_save(); |
| @@ -187,7 +187,7 @@ static __always_inline void __pmr_local_irq_restore(unsigned long flags) · 20 unmodified lines | |||
| 187 | /* | 187 | /* |
| 188 | * restore saved IRQ state | 188 | * restore saved IRQ state |
| 189 | */ | 189 | */ |
| 190 | static inline void arch_local_irq_restore(unsigned long flags) | 190 | static __always_inline void arch_local_irq_restore(unsigned long flags) |
| 191 | { | 191 | { |
| 192 | if (system_uses_irq_prio_masking()) { | 192 | if (system_uses_irq_prio_masking()) { |
| 193 | __pmr_local_irq_restore(flags); | 193 | __pmr_local_irq_restore(flags); |
| @@ -68,7 +68,12 @@ · 67 unmodified lines | |||
| 68 | #define KERNEL_SEGMENT_COUNT 5 | 68 | #define KERNEL_SEGMENT_COUNT 5 |
| 69 | 69 | ||
| 70 | #if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN | 70 | #if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN |
| 71 | #define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 1) | 71 | /* |
| 72 | * KERNEL_SEGMENT_COUNT counts the permanent kernel VMAs. The early mapping | ||
| 73 | * has one additional split, [_text, _stext). Reserve one more page for the | ||
| 74 | * SWAPPER_BLOCK_SIZE-unaligned boundaries. | ||
| 75 | */ | ||
| 76 | #define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 2) | ||
| 72 | /* | 77 | /* |
| 73 | * The initial ID map consists of the kernel image, mapped as two separate | 78 | * The initial ID map consists of the kernel image, mapped as two separate |
| 74 | * segments, and may appear misaligned wrt the swapper block size. This means | 79 | * segments, and may appear misaligned wrt the swapper block size. This means |
| @@ -50,6 +50,9 @@ · 49 unmodified lines | |||
| 50 | 50 | ||
| 51 | #include <linux/mm.h> | 51 | #include <linux/mm.h> |
| 52 | 52 | ||
| 53 | #define MARKER(m) \ | ||
| 54 | m, __after_##m = m - 1 | ||
| 55 | |||
| 53 | enum __kvm_host_smccc_func { | 56 | enum __kvm_host_smccc_func { |
| 54 | /* Hypercalls that are unavailable once pKVM has finalised. */ | 57 | /* Hypercalls that are unavailable once pKVM has finalised. */ |
| 55 | /* __KVM_HOST_SMCCC_FUNC___kvm_hyp_init */ | 58 | /* __KVM_HOST_SMCCC_FUNC___kvm_hyp_init */ |
| @@ -59,8 +62,10 @@ enum __kvm_host_smccc_func { · 3 unmodified lines | |||
| 59 | __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, | 62 | __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, |
| 60 | __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, | 63 | __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, |
| 61 | __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, | 64 | __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, |
| 65 | |||
| 66 | MARKER(__KVM_HOST_SMCCC_FUNC_MIN_PKVM), | ||
| 67 | |||
| 62 | __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, | 68 | __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, |
| 63 | __KVM_HOST_SMCCC_FUNC_MIN_PKVM = __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, | ||
| 64 | 69 | ||
| 65 | /* Hypercalls that are always available and common to [nh]VHE/pKVM. */ | 70 | /* Hypercalls that are always available and common to [nh]VHE/pKVM. */ |
| 66 | __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, | 71 | __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, |
| @@ -72,11 +77,20 @@ enum __kvm_host_smccc_func { · 5 unmodified lines | |||
| 72 | __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_range, | 77 | __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_range, |
| 73 | __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context, | 78 | __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context, |
| 74 | __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff, | 79 | __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff, |
| 80 | __KVM_HOST_SMCCC_FUNC___tracing_load, | ||
| 81 | __KVM_HOST_SMCCC_FUNC___tracing_unload, | ||
| 82 | __KVM_HOST_SMCCC_FUNC___tracing_enable, | ||
| 83 | __KVM_HOST_SMCCC_FUNC___tracing_swap_reader, | ||
| 84 | __KVM_HOST_SMCCC_FUNC___tracing_update_clock, | ||
| 85 | __KVM_HOST_SMCCC_FUNC___tracing_reset, | ||
| 86 | __KVM_HOST_SMCCC_FUNC___tracing_enable_event, | ||
| 87 | __KVM_HOST_SMCCC_FUNC___tracing_write_event, | ||
| 75 | __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, | 88 | __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, |
| 76 | __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs, | 89 | __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs, |
| 77 | __KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr, | 90 | __KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr, |
| 78 | __KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr, | 91 | __KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr, |
| 79 | __KVM_HOST_SMCCC_FUNC_MAX_NO_PKVM = __KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr, | 92 | |
| 93 | MARKER(__KVM_HOST_SMCCC_FUNC_PKVM_ONLY), | ||
| 80 | 94 | ||
| 81 | /* Hypercalls that are available only when pKVM has finalised. */ | 95 | /* Hypercalls that are available only when pKVM has finalised. */ |
| 82 | __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, | 96 | __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, |
| @@ -100,14 +114,8 @@ enum __kvm_host_smccc_func { · 17 unmodified lines | |||
| 100 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_load, | 114 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_load, |
| 101 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_put, | 115 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_put, |
| 102 | __KVM_HOST_SMCCC_FUNC___pkvm_tlb_flush_vmid, | 116 | __KVM_HOST_SMCCC_FUNC___pkvm_tlb_flush_vmid, |
| 103 | __KVM_HOST_SMCCC_FUNC___tracing_load, | 117 | |
| 104 | __KVM_HOST_SMCCC_FUNC___tracing_unload, | 118 | MARKER(__KVM_HOST_SMCCC_FUNC_MAX) |
| 105 | __KVM_HOST_SMCCC_FUNC___tracing_enable, | ||
| 106 | __KVM_HOST_SMCCC_FUNC___tracing_swap_reader, | ||
| 107 | __KVM_HOST_SMCCC_FUNC___tracing_update_clock, | ||
| 108 | __KVM_HOST_SMCCC_FUNC___tracing_reset, | ||
| 109 | __KVM_HOST_SMCCC_FUNC___tracing_enable_event, | ||
| 110 | __KVM_HOST_SMCCC_FUNC___tracing_write_event, | ||
| 111 | }; | 119 | }; |
| 112 | 120 | ||
| 113 | #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] | 121 | #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] |
| @@ -450,9 +450,6 @@ struct kvm_vcpu_fault_info { · 449 unmodified lines | |||
| 450 | r = __VNCR_START__ + ((VNCR_ ## r) / 8), \ | 450 | r = __VNCR_START__ + ((VNCR_ ## r) / 8), \ |
| 451 | __after_##r = __MAX__(__before_##r - 1, r) | 451 | __after_##r = __MAX__(__before_##r - 1, r) |
| 452 | 452 | ||
| 453 | #define MARKER(m) \ | ||
| 454 | m, __after_##m = m - 1 | ||
| 455 | |||
| 456 | enum vcpu_sysreg { | 453 | enum vcpu_sysreg { |
| 457 | __INVALID_SYSREG__, /* 0 is reserved as an invalid value */ | 454 | __INVALID_SYSREG__, /* 0 is reserved as an invalid value */ |
| 458 | MPIDR_EL1, /* MultiProcessor Affinity Register */ | 455 | MPIDR_EL1, /* MultiProcessor Affinity Register */ |
| @@ -514,7 +511,6 @@ enum vcpu_sysreg { · 55 unmodified lines | |||
| 514 | ACTLR_EL2, /* Auxiliary Control Register (EL2) */ | 511 | ACTLR_EL2, /* Auxiliary Control Register (EL2) */ |
| 515 | CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ | 512 | CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ |
| 516 | HACR_EL2, /* Hypervisor Auxiliary Control Register */ | 513 | HACR_EL2, /* Hypervisor Auxiliary Control Register */ |
| 517 | ZCR_EL2, /* SVE Control Register (EL2) */ | ||
| 518 | TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ | 514 | TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ |
| 519 | TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ | 515 | TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ |
| 520 | TCR_EL2, /* Translation Control Register (EL2) */ | 516 | TCR_EL2, /* Translation Control Register (EL2) */ |
| @@ -546,6 +542,7 @@ enum vcpu_sysreg { · 25 unmodified lines | |||
| 546 | SCTLR2_EL2, /* System Control Register 2 (EL2) */ | 542 | SCTLR2_EL2, /* System Control Register 2 (EL2) */ |
| 547 | MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */ | 543 | MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */ |
| 548 | CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */ | 544 | CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */ |
| 545 | ZCR_EL2, /* SVE Control Register (EL2) */ | ||
| 549 | 546 | ||
| 550 | /* Any VNCR-capable reg goes after this point */ | 547 | /* Any VNCR-capable reg goes after this point */ |
| 551 | MARKER(__VNCR_START__), | 548 | MARKER(__VNCR_START__), |
| @@ -1548,7 +1545,7 @@ static inline bool __vcpu_has_feature(const struct kvm_arch *ka, int feature) · 996 unmodified lines | |||
| 1548 | #define kvm_vcpu_has_feature(k, f) __vcpu_has_feature(&(k)->arch, (f)) | 1545 | #define kvm_vcpu_has_feature(k, f) __vcpu_has_feature(&(k)->arch, (f)) |
| 1549 | #define vcpu_has_feature(v, f) __vcpu_has_feature(&(v)->kvm->arch, (f)) | 1546 | #define vcpu_has_feature(v, f) __vcpu_has_feature(&(v)->kvm->arch, (f)) |
| 1550 | 1547 | ||
| 1551 | #define kvm_vcpu_initialized(v) vcpu_get_flag(vcpu, VCPU_INITIALIZED) | 1548 | #define kvm_vcpu_initialized(v) vcpu_get_flag(v, VCPU_INITIALIZED) |
| 1552 | 1549 | ||
| 1553 | int kvm_trng_call(struct kvm_vcpu *vcpu); | 1550 | int kvm_trng_call(struct kvm_vcpu *vcpu); |
| 1554 | #ifdef CONFIG_KVM | 1551 | #ifdef CONFIG_KVM |
| @@ -23,6 +23,7 @@ static inline u64 tcr_el2_ps_to_tcr_el1_ips(u64 tcr_el2) · 22 unmodified lines | |||
| 23 | static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr) | 23 | static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr) |
| 24 | { | 24 | { |
| 25 | return TCR_EPD1_MASK | /* disable TTBR1_EL1 */ | 25 | return TCR_EPD1_MASK | /* disable TTBR1_EL1 */ |
| 26 | ((tcr & TCR_EL2_DS) ? TCR_DS : 0) | | ||
| 26 | ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) | | 27 | ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) | |
| 27 | tcr_el2_ps_to_tcr_el1_ips(tcr) | | 28 | tcr_el2_ps_to_tcr_el1_ips(tcr) | |
| 28 | (tcr & TCR_EL2_TG0_MASK) | | 29 | (tcr & TCR_EL2_TG0_MASK) | |
| @@ -33,7 +33,7 @@ struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, · 32 unmodified lines | |||
| 33 | unsigned long vaddr); | 33 | unsigned long vaddr); |
| 34 | #define vma_alloc_zeroed_movable_folio vma_alloc_zeroed_movable_folio | 34 | #define vma_alloc_zeroed_movable_folio vma_alloc_zeroed_movable_folio |
| 35 | 35 | ||
| 36 | bool tag_clear_highpages(struct page *to, int numpages); | 36 | bool tag_clear_highpages(struct page *to, int numpages, bool clear_pages); |
| 37 | #define __HAVE_ARCH_TAG_CLEAR_HIGHPAGES | 37 | #define __HAVE_ARCH_TAG_CLEAR_HIGHPAGES |
| 38 | 38 | ||
| 39 | #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) | 39 | #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) |
| @@ -0,0 +1,10 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0-only */ | ||
| 2 | #ifndef _ASM_ARM64_RING_BUFFER_H | ||
| 3 | #define _ASM_ARM64_RING_BUFFER_H | ||
| 4 | |||
| 5 | #include <asm/cacheflush.h> | ||
| 6 | |||
| 7 | /* Flush D-cache on persistent ring buffer */ | ||
| 8 | #define arch_ring_buffer_flush_range(start, end) dcache_clean_pop(start, end) | ||
| 9 | |||
| 10 | #endif /* _ASM_ARM64_RING_BUFFER_H */ | ||
| @@ -844,7 +844,7 @@ · 843 unmodified lines | |||
| 844 | #define INIT_SCTLR_EL2_MMU_ON \ | 844 | #define INIT_SCTLR_EL2_MMU_ON \ |
| 845 | (SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA | SCTLR_ELx_I | \ | 845 | (SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA | SCTLR_ELx_I | \ |
| 846 | SCTLR_ELx_IESB | SCTLR_ELx_WXN | ENDIAN_SET_EL2 | \ | 846 | SCTLR_ELx_IESB | SCTLR_ELx_WXN | ENDIAN_SET_EL2 | \ |
| 847 | SCTLR_ELx_ITFSB | SCTLR_EL2_RES1) | 847 | SCTLR_ELx_ITFSB | SCTLR_ELx_EIS | SCTLR_ELx_EOS | SCTLR_EL2_RES1) |
| 848 | 848 | ||
| 849 | #define INIT_SCTLR_EL2_MMU_OFF \ | 849 | #define INIT_SCTLR_EL2_MMU_OFF \ |
| 850 | (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) | 850 | (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) |
| @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb) · 52 unmodified lines | |||
| 53 | static inline void tlb_flush(struct mmu_gather *tlb) | 53 | static inline void tlb_flush(struct mmu_gather *tlb) |
| 54 | { | 54 | { |
| 55 | struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0); | 55 | struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0); |
| 56 | tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE; | 56 | tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ? |
| 57 | TLBF_NONE : TLBF_NOWALKCACHE; | ||
| 57 | unsigned long stride = tlb_get_unmap_size(tlb); | 58 | unsigned long stride = tlb_get_unmap_size(tlb); |
| 58 | int tlb_level = tlb_get_level(tlb); | 59 | int tlb_level = tlb_get_level(tlb); |
| 59 | 60 | ||
| @@ -62,6 +62,13 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs, · 61 unmodified lines | |||
| 62 | irqentry_exit_to_kernel_mode_after_preempt(regs, state); | 62 | irqentry_exit_to_kernel_mode_after_preempt(regs, state); |
| 63 | } | 63 | } |
| 64 | 64 | ||
| 65 | static __always_inline void arm64_syscall_enter_from_user_mode(struct pt_regs *regs) | ||
| 66 | { | ||
| 67 | enter_from_user_mode(regs); | ||
| 68 | mte_disable_tco_entry(current); | ||
| 69 | sme_enter_from_user_mode(); | ||
| 70 | } | ||
| 71 | |||
| 65 | /* | 72 | /* |
| 66 | * Handle IRQ/context state management when entering from user mode. | 73 | * Handle IRQ/context state management when entering from user mode. |
| 67 | * Before this function is called it is not safe to call regular kernel code, | 74 | * Before this function is called it is not safe to call regular kernel code, |
| @@ -70,20 +77,30 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs, · 2 unmodified lines | |||
| 70 | static __always_inline void arm64_enter_from_user_mode(struct pt_regs *regs) | 77 | static __always_inline void arm64_enter_from_user_mode(struct pt_regs *regs) |
| 71 | { | 78 | { |
| 72 | enter_from_user_mode(regs); | 79 | enter_from_user_mode(regs); |
| 80 | rseq_note_user_irq_entry(); | ||
| 73 | mte_disable_tco_entry(current); | 81 | mte_disable_tco_entry(current); |
| 74 | sme_enter_from_user_mode(); | 82 | sme_enter_from_user_mode(); |
| 75 | } | 83 | } |
| 76 | 84 | ||
| 85 | static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs) | ||
| 86 | { | ||
| 87 | local_irq_disable(); | ||
| 88 | syscall_exit_to_user_mode_prepare(regs); | ||
| 89 | local_daif_mask(); | ||
| 90 | sme_exit_to_user_mode(); | ||
| 91 | mte_check_tfsr_exit(); | ||
| 92 | exit_to_user_mode(); | ||
| 93 | } | ||
| 94 | |||
| 77 | /* | 95 | /* |
| 78 | * Handle IRQ/context state management when exiting to user mode. | 96 | * Handle IRQ/context state management when exiting to user mode. |
| 79 | * After this function returns it is not safe to call regular kernel code, | 97 | * After this function returns it is not safe to call regular kernel code, |
| 80 | * instrumentable code, or any code which may trigger an exception. | 98 | * instrumentable code, or any code which may trigger an exception. |
| 81 | */ | 99 | */ |
| 82 | |||
| 83 | static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs) | 100 | static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs) |
| 84 | { | 101 | { |
| 85 | local_irq_disable(); | 102 | local_irq_disable(); |
| 86 | exit_to_user_mode_prepare_legacy(regs); | 103 | irqentry_exit_to_user_mode_prepare(regs); |
| 87 | local_daif_mask(); | 104 | local_daif_mask(); |
| 88 | sme_exit_to_user_mode(); | 105 | sme_exit_to_user_mode(); |
| 89 | mte_check_tfsr_exit(); | 106 | mte_check_tfsr_exit(); |
| @@ -92,7 +109,7 @@ static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs) · 2 unmodified lines | |||
| 92 | 109 | ||
| 93 | asmlinkage void noinstr asm_exit_to_user_mode(struct pt_regs *regs) | 110 | asmlinkage void noinstr asm_exit_to_user_mode(struct pt_regs *regs) |
| 94 | { | 111 | { |
| 95 | arm64_exit_to_user_mode(regs); | 112 | arm64_syscall_exit_to_user_mode(regs); |
| 96 | } | 113 | } |
| 97 | 114 | ||
| 98 | /* | 115 | /* |
| @@ -716,12 +733,12 @@ static void noinstr el0_brk64(struct pt_regs *regs, unsigned long esr) · 617 unmodified lines | |||
| 716 | 733 | ||
| 717 | static void noinstr el0_svc(struct pt_regs *regs) | 734 | static void noinstr el0_svc(struct pt_regs *regs) |
| 718 | { | 735 | { |
| 719 | arm64_enter_from_user_mode(regs); | 736 | arm64_syscall_enter_from_user_mode(regs); |
| 720 | cortex_a76_erratum_1463225_svc_handler(); | 737 | cortex_a76_erratum_1463225_svc_handler(); |
| 721 | fpsimd_syscall_enter(); | 738 | fpsimd_syscall_enter(); |
| 722 | local_daif_restore(DAIF_PROCCTX); | 739 | local_daif_restore(DAIF_PROCCTX); |
| 723 | do_el0_svc(regs); | 740 | do_el0_svc(regs); |
| 724 | arm64_exit_to_user_mode(regs); | 741 | arm64_syscall_exit_to_user_mode(regs); |
| 725 | fpsimd_syscall_exit(); | 742 | fpsimd_syscall_exit(); |
| 726 | } | 743 | } |
| 727 | 744 | ||
| @@ -868,11 +885,11 @@ static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr) · 140 unmodified lines | |||
| 868 | 885 | ||
| 869 | static void noinstr el0_svc_compat(struct pt_regs *regs) | 886 | static void noinstr el0_svc_compat(struct pt_regs *regs) |
| 870 | { | 887 | { |
| 871 | arm64_enter_from_user_mode(regs); | 888 | arm64_syscall_enter_from_user_mode(regs); |
| 872 | cortex_a76_erratum_1463225_svc_handler(); | 889 | cortex_a76_erratum_1463225_svc_handler(); |
| 873 | local_daif_restore(DAIF_PROCCTX); | 890 | local_daif_restore(DAIF_PROCCTX); |
| 874 | do_el0_svc_compat(regs); | 891 | do_el0_svc_compat(regs); |
| 875 | arm64_exit_to_user_mode(regs); | 892 | arm64_syscall_exit_to_user_mode(regs); |
| 876 | } | 893 | } |
| 877 | 894 | ||
| 878 | static void noinstr el0_bkpt32(struct pt_regs *regs, unsigned long esr) | 895 | static void noinstr el0_bkpt32(struct pt_regs *regs, unsigned long esr) |
| @@ -196,9 +196,9 @@ static int scs_handle_fde_frame(const struct eh_frame *frame, · 195 unmodified lines | |||
| 196 | loc += *opcode++ * code_alignment_factor; | 196 | loc += *opcode++ * code_alignment_factor; |
| 197 | loc += (*opcode++ << 8) * code_alignment_factor; | 197 | loc += (*opcode++ << 8) * code_alignment_factor; |
| 198 | loc += (*opcode++ << 16) * code_alignment_factor; | 198 | loc += (*opcode++ << 16) * code_alignment_factor; |
| 199 | loc += (*opcode++ << 24) * code_alignment_factor; | 199 | loc += ((u64)*opcode++ << 24) * code_alignment_factor; |
| 200 | size -= 4; | 200 | size -= 4; |
| 201 | break; | 201 | break; |
| 202 | 202 | ||
| 203 | case DW_CFA_def_cfa: | 203 | case DW_CFA_def_cfa: |
| 204 | case DW_CFA_offset_extended: | 204 | case DW_CFA_offset_extended: |
| @@ -983,8 +983,8 @@ static int sve_set_common(struct task_struct *target, · 982 unmodified lines | |||
| 983 | } | 983 | } |
| 984 | 984 | ||
| 985 | /* Always zero V regs, FPSR, and FPCR */ | 985 | /* Always zero V regs, FPSR, and FPCR */ |
| 986 | memset(¤t->thread.uw.fpsimd_state, 0, | 986 | memset(&target->thread.uw.fpsimd_state, 0, |
| 987 | sizeof(current->thread.uw.fpsimd_state)); | 987 | sizeof(target->thread.uw.fpsimd_state)); |
| 988 | 988 | ||
| 989 | /* Registers: FPSIMD-only case */ | 989 | /* Registers: FPSIMD-only case */ |
| 990 | 990 | ||
| @@ -67,6 +67,9 @@ struct rt_sigframe_user_layout { · 66 unmodified lines | |||
| 67 | unsigned long end_offset; | 67 | unsigned long end_offset; |
| 68 | }; | 68 | }; |
| 69 | 69 | ||
| 70 | #define TERMINATOR_SIZE round_up(sizeof(struct _aarch64_ctx), 16) | ||
| 71 | #define EXTRA_CONTEXT_SIZE round_up(sizeof(struct extra_context), 16) | ||
| 72 | |||
| 70 | /* | 73 | /* |
| 71 | * Holds any EL0-controlled state that influences unprivileged memory accesses. | 74 | * Holds any EL0-controlled state that influences unprivileged memory accesses. |
| 72 | * This includes both accesses done in userspace and uaccess done in the kernel. | 75 | * This includes both accesses done in userspace and uaccess done in the kernel. |
| @@ -74,13 +77,35 @@ struct rt_sigframe_user_layout { · 1 unmodified line | |||
| 74 | * This state needs to be carefully managed to ensure that it doesn't cause | 77 | * This state needs to be carefully managed to ensure that it doesn't cause |
| 75 | * uaccess to fail when setting up the signal frame, and the signal handler | 78 | * uaccess to fail when setting up the signal frame, and the signal handler |
| 76 | * itself also expects a well-defined state when entered. | 79 | * itself also expects a well-defined state when entered. |
| 80 | * | ||
| 81 | * The struct should be zero-initialised. Its members should only be accessed | ||
| 82 | * via the accessors below. __valid_fields tracks which of the fields are valid | ||
| 83 | * (have been set to some value). | ||
| 77 | */ | 84 | */ |
| 78 | struct user_access_state { | 85 | struct user_access_state { |
| 79 | u64 por_el0; | 86 | unsigned int __valid_fields; |
| 87 | u64 __por_el0; | ||
| 80 | }; | 88 | }; |
| 81 | 89 | ||
| 82 | #define TERMINATOR_SIZE round_up(sizeof(struct _aarch64_ctx), 16) | 90 | #define UA_STATE_HAS_POR_EL0 BIT(0) |
| 83 | #define EXTRA_CONTEXT_SIZE round_up(sizeof(struct extra_context), 16) | 91 | |
| 92 | static void set_ua_state_por_el0(struct user_access_state *ua_state, | ||
| 93 | u64 por_el0) | ||
| 94 | { | ||
| 95 | ua_state->__por_el0 = por_el0; | ||
| 96 | ua_state->__valid_fields |= UA_STATE_HAS_POR_EL0; | ||
| 97 | } | ||
| 98 | |||
| 99 | static int get_ua_state_por_el0(const struct user_access_state *ua_state, | ||
| 100 | u64 *por_el0) | ||
| 101 | { | ||
| 102 | if (ua_state->__valid_fields & UA_STATE_HAS_POR_EL0) { | ||
| 103 | *por_el0 = ua_state->__por_el0; | ||
| 104 | return 0; | ||
| 105 | } | ||
| 106 | |||
| 107 | return -ENOENT; | ||
| 108 | } | ||
| 84 | 109 | ||
| 85 | /* | 110 | /* |
| 86 | * Save the user access state into ua_state and reset it to disable any | 111 | * Save the user access state into ua_state and reset it to disable any |
| @@ -94,7 +119,7 @@ static void save_reset_user_access_state(struct user_access_state *ua_state) · 7 unmodified lines | |||
| 94 | for (int pkey = 0; pkey < arch_max_pkey(); pkey++) | 119 | for (int pkey = 0; pkey < arch_max_pkey(); pkey++) |
| 95 | por_enable_all |= POR_ELx_PERM_PREP(pkey, POE_RWX); | 120 | por_enable_all |= POR_ELx_PERM_PREP(pkey, POE_RWX); |
| 96 | 121 | ||
| 97 | ua_state->por_el0 = read_sysreg_s(SYS_POR_EL0); | 122 | set_ua_state_por_el0(ua_state, read_sysreg_s(SYS_POR_EL0)); |
| 98 | write_sysreg_s(por_enable_all, SYS_POR_EL0); | 123 | write_sysreg_s(por_enable_all, SYS_POR_EL0); |
| 99 | /* | 124 | /* |
| 100 | * No ISB required as we can tolerate spurious Overlay faults - | 125 | * No ISB required as we can tolerate spurious Overlay faults - |
| @@ -122,8 +147,10 @@ static void set_handler_user_access_state(void) · 21 unmodified lines | |||
| 122 | */ | 147 | */ |
| 123 | static void restore_user_access_state(const struct user_access_state *ua_state) | 148 | static void restore_user_access_state(const struct user_access_state *ua_state) |
| 124 | { | 149 | { |
| 125 | if (system_supports_poe()) | 150 | u64 por_el0; |
| 126 | write_sysreg_s(ua_state->por_el0, SYS_POR_EL0); | 151 | |
| 152 | if (get_ua_state_por_el0(ua_state, &por_el0) == 0) | ||
| 153 | write_sysreg_s(por_el0, SYS_POR_EL0); | ||
| 127 | } | 154 | } |
| 128 | 155 | ||
| 129 | static void init_user_layout(struct rt_sigframe_user_layout *user) | 156 | static void init_user_layout(struct rt_sigframe_user_layout *user) |
| @@ -333,11 +360,16 @@ static int restore_fpmr_context(struct user_ctxs *user) · 203 unmodified lines | |||
| 333 | static int preserve_poe_context(struct poe_context __user *ctx, | 360 | static int preserve_poe_context(struct poe_context __user *ctx, |
| 334 | const struct user_access_state *ua_state) | 361 | const struct user_access_state *ua_state) |
| 335 | { | 362 | { |
| 336 | int err = 0; | 363 | int err; |
| 364 | u64 por_el0; | ||
| 365 | |||
| 366 | err = get_ua_state_por_el0(ua_state, &por_el0); | ||
| 367 | if (WARN_ON_ONCE(err)) | ||
| 368 | return err; | ||
| 337 | 369 | ||
| 338 | __put_user_error(POE_MAGIC, &ctx->head.magic, err); | 370 | __put_user_error(POE_MAGIC, &ctx->head.magic, err); |
| 339 | __put_user_error(sizeof(*ctx), &ctx->head.size, err); | 371 | __put_user_error(sizeof(*ctx), &ctx->head.size, err); |
| 340 | __put_user_error(ua_state->por_el0, &ctx->por_el0, err); | 372 | __put_user_error(por_el0, &ctx->por_el0, err); |
| 341 | 373 | ||
| 342 | return err; | 374 | return err; |
| 343 | } | 375 | } |
| @@ -353,7 +385,7 @@ static int restore_poe_context(struct user_ctxs *user, · 9 unmodified lines | |||
| 353 | 385 | ||
| 354 | __get_user_error(por_el0, &(user->poe->por_el0), err); | 386 | __get_user_error(por_el0, &(user->poe->por_el0), err); |
| 355 | if (!err) | 387 | if (!err) |
| 356 | ua_state->por_el0 = por_el0; | 388 | set_ua_state_por_el0(ua_state, por_el0); |
| 357 | 389 | ||
| 358 | return err; | 390 | return err; |
| 359 | } | 391 | } |
| @@ -1095,7 +1127,7 @@ SYSCALL_DEFINE0(rt_sigreturn) · 735 unmodified lines | |||
| 1095 | { | 1127 | { |
| 1096 | struct pt_regs *regs = current_pt_regs(); | 1128 | struct pt_regs *regs = current_pt_regs(); |
| 1097 | struct rt_sigframe __user *frame; | 1129 | struct rt_sigframe __user *frame; |
| 1098 | struct user_access_state ua_state; | 1130 | struct user_access_state ua_state = {}; |
| 1099 | 1131 | ||
| 1100 | /* Always make any pending restarted system calls return -EINTR */ | 1132 | /* Always make any pending restarted system calls return -EINTR */ |
| 1101 | current->restart_block.fn = do_no_restart_syscall; | 1133 | current->restart_block.fn = do_no_restart_syscall; |
| @@ -1507,7 +1539,7 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set, · 405 unmodified lines | |||
| 1507 | { | 1539 | { |
| 1508 | struct rt_sigframe_user_layout user; | 1540 | struct rt_sigframe_user_layout user; |
| 1509 | struct rt_sigframe __user *frame; | 1541 | struct rt_sigframe __user *frame; |
| 1510 | struct user_access_state ua_state; | 1542 | struct user_access_state ua_state = {}; |
| 1511 | int err = 0; | 1543 | int err = 0; |
| 1512 | 1544 | ||
| 1513 | fpsimd_save_and_flush_current_state(); | 1545 | fpsimd_save_and_flush_current_state(); |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | |||
| 4 | * Author: Christoffer Dall <c.dall@virtualopensystems.com> | 4 | * Author: Christoffer Dall <c.dall@virtualopensystems.com> |
| 5 | */ | 5 | */ |
| 6 | 6 | ||
| 7 | #include <linux/arm-smccc.h> | ||
| 7 | #include <linux/bug.h> | 8 | #include <linux/bug.h> |
| 8 | #include <linux/cpu_pm.h> | 9 | #include <linux/cpu_pm.h> |
| 9 | #include <linux/errno.h> | 10 | #include <linux/errno.h> |
| @@ -554,8 +555,10 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 544 unmodified lines | |||
| 554 | kvm_destroy_mpidr_data(vcpu->kvm); | 555 | kvm_destroy_mpidr_data(vcpu->kvm); |
| 555 | 556 | ||
| 556 | err = kvm_vgic_vcpu_init(vcpu); | 557 | err = kvm_vgic_vcpu_init(vcpu); |
| 557 | if (err) | 558 | if (err) { |
| 559 | kvm_vgic_vcpu_destroy(vcpu); | ||
| 558 | return err; | 560 | return err; |
| 561 | } | ||
| 559 | 562 | ||
| 560 | err = kvm_share_hyp(vcpu, vcpu + 1); | 563 | err = kvm_share_hyp(vcpu, vcpu + 1); |
| 561 | if (err) | 564 | if (err) |
| @@ -824,6 +827,10 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) · 262 unmodified lines | |||
| 824 | { | 827 | { |
| 825 | bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF | HCR_VSE); | 828 | bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF | HCR_VSE); |
| 826 | 829 | ||
| 830 | irq_lines |= (!irqchip_in_kernel(v->kvm) && | ||
| 831 | (kvm_timer_should_notify_user(v) || | ||
| 832 | kvm_pmu_should_notify_user(v))); | ||
| 833 | |||
| 827 | return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) | 834 | return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) |
| 828 | && !kvm_arm_vcpu_stopped(v) && !v->arch.pause); | 835 | && !kvm_arm_vcpu_stopped(v) && !v->arch.pause); |
| 829 | } | 836 | } |
| @@ -2634,6 +2641,22 @@ static int init_pkvm_host_sve_state(void) · 1804 unmodified lines | |||
| 2634 | return 0; | 2641 | return 0; |
| 2635 | } | 2642 | } |
| 2636 | 2643 | ||
| 2644 | static int pkvm_check_sme_dvmsync_fw_call(void) | ||
| 2645 | { | ||
| 2646 | struct arm_smccc_res res; | ||
| 2647 | |||
| 2648 | if (!cpus_have_final_cap(ARM64_WORKAROUND_4193714)) | ||
| 2649 | return 0; | ||
| 2650 | |||
| 2651 | arm_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res); | ||
| 2652 | if (res.a0) { | ||
| 2653 | kvm_err("pKVM requires firmware support for C1-Pro erratum 4193714\n"); | ||
| 2654 | return -ENODEV; | ||
| 2655 | } | ||
| 2656 | |||
| 2657 | return 0; | ||
| 2658 | } | ||
| 2659 | |||
| 2637 | /* | 2660 | /* |
| 2638 | * Finalizes the initialization of hyp mode, once everything else is initialized | 2661 | * Finalizes the initialization of hyp mode, once everything else is initialized |
| 2639 | * and the initialziation process cannot fail. | 2662 | * and the initialziation process cannot fail. |
| @@ -2834,6 +2857,10 @@ static int __init init_hyp_mode(void) · 194 unmodified lines | |||
| 2834 | if (err) | 2857 | if (err) |
| 2835 | goto out_err; | 2858 | goto out_err; |
| 2836 | 2859 | ||
| 2860 | err = pkvm_check_sme_dvmsync_fw_call(); | ||
| 2861 | if (err) | ||
| 2862 | goto out_err; | ||
| 2863 | |||
| 2837 | err = kvm_hyp_init_protection(hyp_va_bits); | 2864 | err = kvm_hyp_init_protection(hyp_va_bits); |
| 2838 | if (err) { | 2865 | if (err) { |
| 2839 | kvm_err("Failed to init hyp memory protection\n"); | 2866 | kvm_err("Failed to init hyp memory protection\n"); |
| @@ -131,7 +131,6 @@ struct reg_feat_map_desc { · 130 unmodified lines | |||
| 131 | } | 131 | } |
| 132 | 132 | ||
| 133 | #define FEAT_SPE ID_AA64DFR0_EL1, PMSVer, IMP | 133 | #define FEAT_SPE ID_AA64DFR0_EL1, PMSVer, IMP |
| 134 | #define FEAT_SPE_FnE ID_AA64DFR0_EL1, PMSVer, V1P2 | ||
| 135 | #define FEAT_BRBE ID_AA64DFR0_EL1, BRBE, IMP | 134 | #define FEAT_BRBE ID_AA64DFR0_EL1, BRBE, IMP |
| 136 | #define FEAT_TRC_SR ID_AA64DFR0_EL1, TraceVer, IMP | 135 | #define FEAT_TRC_SR ID_AA64DFR0_EL1, TraceVer, IMP |
| 137 | #define FEAT_PMUv3 ID_AA64DFR0_EL1, PMUVer, IMP | 136 | #define FEAT_PMUv3 ID_AA64DFR0_EL1, PMUVer, IMP |
| @@ -192,7 +191,7 @@ struct reg_feat_map_desc { · 54 unmodified lines | |||
| 192 | #define FEAT_SRMASK ID_AA64MMFR4_EL1, SRMASK, IMP | 191 | #define FEAT_SRMASK ID_AA64MMFR4_EL1, SRMASK, IMP |
| 193 | #define FEAT_PoPS ID_AA64MMFR4_EL1, PoPS, IMP | 192 | #define FEAT_PoPS ID_AA64MMFR4_EL1, PoPS, IMP |
| 194 | #define FEAT_PFAR ID_AA64PFR1_EL1, PFAR, IMP | 193 | #define FEAT_PFAR ID_AA64PFR1_EL1, PFAR, IMP |
| 195 | #define FEAT_Debugv8p9 ID_AA64DFR0_EL1, PMUVer, V3P9 | 194 | #define FEAT_Debugv8p9 ID_AA64DFR0_EL1, DebugVer, V8P9 |
| 196 | #define FEAT_PMUv3_SS ID_AA64DFR0_EL1, PMSS, IMP | 195 | #define FEAT_PMUv3_SS ID_AA64DFR0_EL1, PMSS, IMP |
| 197 | #define FEAT_SEBEP ID_AA64DFR0_EL1, SEBEP, IMP | 196 | #define FEAT_SEBEP ID_AA64DFR0_EL1, SEBEP, IMP |
| 198 | #define FEAT_EBEP ID_AA64DFR1_EL1, EBEP, IMP | 197 | #define FEAT_EBEP ID_AA64DFR1_EL1, EBEP, IMP |
| @@ -283,7 +282,7 @@ static bool feat_anerr(struct kvm *kvm) · 84 unmodified lines | |||
| 283 | static bool feat_sme_smps(struct kvm *kvm) | 282 | static bool feat_sme_smps(struct kvm *kvm) |
| 284 | { | 283 | { |
| 285 | /* | 284 | /* |
| 286 | * Revists this if KVM ever supports SME -- this really should | 285 | * Revisit this if KVM ever supports SME -- this really should |
| 287 | * look at the guest's view of SMIDR_EL1. Funnily enough, this | 286 | * look at the guest's view of SMIDR_EL1. Funnily enough, this |
| 288 | * is not captured in the JSON file, but only as a note in the | 287 | * is not captured in the JSON file, but only as a note in the |
| 289 | * ARM ARM. | 288 | * ARM ARM. |
| @@ -295,17 +294,27 @@ static bool feat_sme_smps(struct kvm *kvm) · 5 unmodified lines | |||
| 295 | static bool feat_spe_fds(struct kvm *kvm) | 294 | static bool feat_spe_fds(struct kvm *kvm) |
| 296 | { | 295 | { |
| 297 | /* | 296 | /* |
| 298 | * Revists this if KVM ever supports SPE -- this really should | 297 | * Revisit this if KVM ever supports SPE -- this really should |
| 299 | * look at the guest's view of PMSIDR_EL1. | 298 | * look at the guest's view of PMSIDR_EL1. |
| 300 | */ | 299 | */ |
| 301 | return (kvm_has_feat(kvm, FEAT_SPEv1p4) && | 300 | return (kvm_has_feat(kvm, FEAT_SPEv1p4) && |
| 302 | (read_sysreg_s(SYS_PMSIDR_EL1) & PMSIDR_EL1_FDS)); | 301 | (read_sysreg_s(SYS_PMSIDR_EL1) & PMSIDR_EL1_FDS)); |
| 303 | } | 302 | } |
| 304 | 303 | ||
| 304 | static bool feat_spe_fne(struct kvm *kvm) | ||
| 305 | { | ||
| 306 | /* | ||
| 307 | * Revisit this if KVM ever supports SPE -- this really should | ||
| 308 | * look at the guest's view of PMSIDR_EL1. | ||
| 309 | */ | ||
| 310 | return (kvm_has_feat(kvm, FEAT_SPEv1p2) && | ||
| 311 | (read_sysreg_s(SYS_PMSIDR_EL1) & PMSIDR_EL1_FnE)); | ||
| 312 | } | ||
| 313 | |||
| 305 | static bool feat_trbe_mpam(struct kvm *kvm) | 314 | static bool feat_trbe_mpam(struct kvm *kvm) |
| 306 | { | 315 | { |
| 307 | /* | 316 | /* |
| 308 | * Revists this if KVM ever supports both MPAM and TRBE -- | 317 | * Revisit this if KVM ever supports both MPAM and TRBE -- |
| 309 | * this really should look at the guest's view of TRBIDR_EL1. | 318 | * this really should look at the guest's view of TRBIDR_EL1. |
| 310 | */ | 319 | */ |
| 311 | return (kvm_has_feat(kvm, FEAT_TRBE) && | 320 | return (kvm_has_feat(kvm, FEAT_TRBE) && |
| @@ -537,7 +546,7 @@ static const struct reg_bits_to_feat_map hdfgrtr_feat_map[] = { · 225 unmodified lines | |||
| 537 | HDFGRTR_EL2_PMBPTR_EL1 | | 546 | HDFGRTR_EL2_PMBPTR_EL1 | |
| 538 | HDFGRTR_EL2_PMBLIMITR_EL1, | 547 | HDFGRTR_EL2_PMBLIMITR_EL1, |
| 539 | FEAT_SPE), | 548 | FEAT_SPE), |
| 540 | NEEDS_FEAT(HDFGRTR_EL2_nPMSNEVFR_EL1, FEAT_SPE_FnE), | 549 | NEEDS_FEAT(HDFGRTR_EL2_nPMSNEVFR_EL1, feat_spe_fne), |
| 541 | NEEDS_FEAT(HDFGRTR_EL2_nBRBDATA | | 550 | NEEDS_FEAT(HDFGRTR_EL2_nBRBDATA | |
| 542 | HDFGRTR_EL2_nBRBCTL | | 551 | HDFGRTR_EL2_nBRBCTL | |
| 543 | HDFGRTR_EL2_nBRBIDR, | 552 | HDFGRTR_EL2_nBRBIDR, |
| @@ -605,7 +614,7 @@ static const struct reg_bits_to_feat_map hdfgwtr_feat_map[] = { · 61 unmodified lines | |||
| 605 | HDFGWTR_EL2_PMBPTR_EL1 | | 614 | HDFGWTR_EL2_PMBPTR_EL1 | |
| 606 | HDFGWTR_EL2_PMBLIMITR_EL1, | 615 | HDFGWTR_EL2_PMBLIMITR_EL1, |
| 607 | FEAT_SPE), | 616 | FEAT_SPE), |
| 608 | NEEDS_FEAT(HDFGWTR_EL2_nPMSNEVFR_EL1, FEAT_SPE_FnE), | 617 | NEEDS_FEAT(HDFGWTR_EL2_nPMSNEVFR_EL1, feat_spe_fne), |
| 609 | NEEDS_FEAT(HDFGWTR_EL2_nBRBDATA | | 618 | NEEDS_FEAT(HDFGWTR_EL2_nBRBDATA | |
| 610 | HDFGWTR_EL2_nBRBCTL, | 619 | HDFGWTR_EL2_nBRBCTL, |
| 611 | FEAT_BRBE), | 620 | FEAT_BRBE), |
| @@ -245,7 +245,7 @@ static inline void __activate_traps_ich_hfgxtr(struct kvm_vcpu *vcpu) · 244 unmodified lines | |||
| 245 | __activate_fgt(hctxt, vcpu, ICH_HFGITR_EL2); | 245 | __activate_fgt(hctxt, vcpu, ICH_HFGITR_EL2); |
| 246 | } | 246 | } |
| 247 | 247 | ||
| 248 | #define __deactivate_fgt(htcxt, vcpu, reg) \ | 248 | #define __deactivate_fgt(hctxt, vcpu, reg) \ |
| 249 | do { \ | 249 | do { \ |
| 250 | write_sysreg_s(ctxt_sys_reg(hctxt, reg), \ | 250 | write_sysreg_s(ctxt_sys_reg(hctxt, reg), \ |
| 251 | SYS_ ## reg); \ | 251 | SYS_ ## reg); \ |
| @@ -462,11 +462,13 @@ static inline bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code) · 210 unmodified lines | |||
| 462 | 462 | ||
| 463 | static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) | 463 | static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) |
| 464 | { | 464 | { |
| 465 | u64 zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | ||
| 466 | |||
| 465 | /* | 467 | /* |
| 466 | * The vCPU's saved SVE state layout always matches the max VL of the | 468 | * The vCPU's saved SVE state layout always matches the max VL of the |
| 467 | * vCPU. Start off with the max VL so we can load the SVE state. | 469 | * vCPU. Start off with the max VL so we can load the SVE state. |
| 468 | */ | 470 | */ |
| 469 | sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); | 471 | sve_cond_update_zcr_vq(zcr_el2, SYS_ZCR_EL2); |
| 470 | __sve_restore_state(vcpu_sve_pffr(vcpu), | 472 | __sve_restore_state(vcpu_sve_pffr(vcpu), |
| 471 | &vcpu->arch.ctxt.fp_regs.fpsr, | 473 | &vcpu->arch.ctxt.fp_regs.fpsr, |
| 472 | true); | 474 | true); |
| @@ -476,8 +478,10 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) · 3 unmodified lines | |||
| 476 | * nested guest, as the guest hypervisor could select a smaller VL. Slap | 478 | * nested guest, as the guest hypervisor could select a smaller VL. Slap |
| 477 | * that into hardware before wrapping up. | 479 | * that into hardware before wrapping up. |
| 478 | */ | 480 | */ |
| 479 | if (is_nested_ctxt(vcpu)) | 481 | if (is_nested_ctxt(vcpu)) { |
| 480 | sve_cond_update_zcr_vq(__vcpu_sys_reg(vcpu, ZCR_EL2), SYS_ZCR_EL2); | 482 | zcr_el2 = min(zcr_el2, __vcpu_sys_reg(vcpu, ZCR_EL2)); |
| 483 | sve_cond_update_zcr_vq(zcr_el2, SYS_ZCR_EL2); | ||
| 484 | } | ||
| 481 | 485 | ||
| 482 | write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); | 486 | write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); |
| 483 | } | 487 | } |
| @@ -501,11 +505,11 @@ static inline void fpsimd_lazy_switch_to_guest(struct kvm_vcpu *vcpu) · 17 unmodified lines | |||
| 501 | return; | 505 | return; |
| 502 | 506 | ||
| 503 | if (vcpu_has_sve(vcpu)) { | 507 | if (vcpu_has_sve(vcpu)) { |
| 508 | zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | ||
| 509 | |||
| 504 | /* A guest hypervisor may restrict the effective max VL. */ | 510 | /* A guest hypervisor may restrict the effective max VL. */ |
| 505 | if (is_nested_ctxt(vcpu)) | 511 | if (is_nested_ctxt(vcpu)) |
| 506 | zcr_el2 = __vcpu_sys_reg(vcpu, ZCR_EL2); | 512 | zcr_el2 = min(zcr_el2, __vcpu_sys_reg(vcpu, ZCR_EL2)); |
| 507 | else | ||
| 508 | zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | ||
| 509 | 513 | ||
| 510 | write_sysreg_el2(zcr_el2, SYS_ZCR); | 514 | write_sysreg_el2(zcr_el2, SYS_ZCR); |
| 511 | 515 | ||
| @@ -35,6 +35,9 @@ void trace_clock_update(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cyc) · 34 unmodified lines | |||
| 35 | struct clock_data *clock = &trace_clock_data; | 35 | struct clock_data *clock = &trace_clock_data; |
| 36 | u64 bank = clock->cur ^ 1; | 36 | u64 bank = clock->cur ^ 1; |
| 37 | 37 | ||
| 38 | if (!mult || shift >= 64) | ||
| 39 | return; | ||
| 40 | |||
| 38 | clock->data[bank].mult = mult; | 41 | clock->data[bank].mult = mult; |
| 39 | clock->data[bank].shift = shift; | 42 | clock->data[bank].shift = shift; |
| 40 | clock->data[bank].epoch_ns = epoch_ns; | 43 | clock->data[bank].epoch_ns = epoch_ns; |
| @@ -120,7 +120,7 @@ SYM_FUNC_START(__hyp_do_panic) · 119 unmodified lines | |||
| 120 | 120 | ||
| 121 | mov x29, x0 | 121 | mov x29, x0 |
| 122 | 122 | ||
| 123 | #ifdef PKVM_DISABLE_STAGE2_ON_PANIC | 123 | #ifdef CONFIG_PKVM_DISABLE_STAGE2_ON_PANIC |
| 124 | /* Ensure host stage-2 is disabled */ | 124 | /* Ensure host stage-2 is disabled */ |
| 125 | mrs x0, hcr_el2 | 125 | mrs x0, hcr_el2 |
| 126 | bic x0, x0, #HCR_VM | 126 | bic x0, x0, #HCR_VM |
| @@ -709,6 +709,14 @@ static const hcall_t host_hcall[] = { · 708 unmodified lines | |||
| 709 | HANDLE_FUNC(__kvm_tlb_flush_vmid_range), | 709 | HANDLE_FUNC(__kvm_tlb_flush_vmid_range), |
| 710 | HANDLE_FUNC(__kvm_flush_cpu_context), | 710 | HANDLE_FUNC(__kvm_flush_cpu_context), |
| 711 | HANDLE_FUNC(__kvm_timer_set_cntvoff), | 711 | HANDLE_FUNC(__kvm_timer_set_cntvoff), |
| 712 | HANDLE_FUNC(__tracing_load), | ||
| 713 | HANDLE_FUNC(__tracing_unload), | ||
| 714 | HANDLE_FUNC(__tracing_enable), | ||
| 715 | HANDLE_FUNC(__tracing_swap_reader), | ||
| 716 | HANDLE_FUNC(__tracing_update_clock), | ||
| 717 | HANDLE_FUNC(__tracing_reset), | ||
| 718 | HANDLE_FUNC(__tracing_enable_event), | ||
| 719 | HANDLE_FUNC(__tracing_write_event), | ||
| 712 | HANDLE_FUNC(__vgic_v3_save_aprs), | 720 | HANDLE_FUNC(__vgic_v3_save_aprs), |
| 713 | HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs), | 721 | HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs), |
| 714 | HANDLE_FUNC(__vgic_v5_save_apr), | 722 | HANDLE_FUNC(__vgic_v5_save_apr), |
| @@ -735,22 +743,16 @@ static const hcall_t host_hcall[] = { · 20 unmodified lines | |||
| 735 | HANDLE_FUNC(__pkvm_vcpu_load), | 743 | HANDLE_FUNC(__pkvm_vcpu_load), |
| 736 | HANDLE_FUNC(__pkvm_vcpu_put), | 744 | HANDLE_FUNC(__pkvm_vcpu_put), |
| 737 | HANDLE_FUNC(__pkvm_tlb_flush_vmid), | 745 | HANDLE_FUNC(__pkvm_tlb_flush_vmid), |
| 738 | HANDLE_FUNC(__tracing_load), | ||
| 739 | HANDLE_FUNC(__tracing_unload), | ||
| 740 | HANDLE_FUNC(__tracing_enable), | ||
| 741 | HANDLE_FUNC(__tracing_swap_reader), | ||
| 742 | HANDLE_FUNC(__tracing_update_clock), | ||
| 743 | HANDLE_FUNC(__tracing_reset), | ||
| 744 | HANDLE_FUNC(__tracing_enable_event), | ||
| 745 | HANDLE_FUNC(__tracing_write_event), | ||
| 746 | }; | 746 | }; |
| 747 | 747 | ||
| 748 | static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) | 748 | static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) |
| 749 | { | 749 | { |
| 750 | DECLARE_REG(unsigned long, id, host_ctxt, 0); | 750 | DECLARE_REG(unsigned long, id, host_ctxt, 0); |
| 751 | unsigned long hcall_min = 0, hcall_max = -1; | 751 | unsigned long hcall_min = 0, hcall_max = __KVM_HOST_SMCCC_FUNC_MAX; |
| 752 | hcall_t hfn; | 752 | hcall_t hfn; |
| 753 | 753 | ||
| 754 | BUILD_BUG_ON(ARRAY_SIZE(host_hcall) != __KVM_HOST_SMCCC_FUNC_MAX); | ||
| 755 | |||
| 754 | /* | 756 | /* |
| 755 | * If pKVM has been initialised then reject any calls to the | 757 | * If pKVM has been initialised then reject any calls to the |
| 756 | * early "privileged" hypercalls. Note that we cannot reject | 758 | * early "privileged" hypercalls. Note that we cannot reject |
| @@ -763,16 +765,14 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) · 6 unmodified lines | |||
| 763 | if (static_branch_unlikely(&kvm_protected_mode_initialized)) { | 765 | if (static_branch_unlikely(&kvm_protected_mode_initialized)) { |
| 764 | hcall_min = __KVM_HOST_SMCCC_FUNC_MIN_PKVM; | 766 | hcall_min = __KVM_HOST_SMCCC_FUNC_MIN_PKVM; |
| 765 | } else { | 767 | } else { |
| 766 | hcall_max = __KVM_HOST_SMCCC_FUNC_MAX_NO_PKVM; | 768 | hcall_max = __KVM_HOST_SMCCC_FUNC_PKVM_ONLY; |
| 767 | } | 769 | } |
| 768 | 770 | ||
| 769 | id &= ~ARM_SMCCC_CALL_HINTS; | 771 | id &= ~ARM_SMCCC_CALL_HINTS; |
| 770 | id -= KVM_HOST_SMCCC_ID(0); | 772 | id -= KVM_HOST_SMCCC_ID(0); |
| 771 | 773 | ||
| 772 | if (unlikely(id < hcall_min || id > hcall_max || | 774 | if (unlikely(id < hcall_min || id >= hcall_max)) |
| 773 | id >= ARRAY_SIZE(host_hcall))) { | ||
| 774 | goto inval; | 775 | goto inval; |
| 775 | } | ||
| 776 | 776 | ||
| 777 | hfn = host_hcall[id]; | 777 | hfn = host_hcall[id]; |
| 778 | if (unlikely(!hfn)) | 778 | if (unlikely(!hfn)) |
| @@ -805,6 +805,10 @@ static void handle_host_smc(struct kvm_cpu_context *host_ctxt) · 26 unmodified lines | |||
| 805 | } | 805 | } |
| 806 | 806 | ||
| 807 | func_id &= ~ARM_SMCCC_CALL_HINTS; | 807 | func_id &= ~ARM_SMCCC_CALL_HINTS; |
| 808 | if (upper_32_bits(func_id)) { | ||
| 809 | cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED; | ||
| 810 | goto exit_skip_instr; | ||
| 811 | } | ||
| 808 | 812 | ||
| 809 | handled = kvm_host_psci_handler(host_ctxt, func_id); | 813 | handled = kvm_host_psci_handler(host_ctxt, func_id); |
| 810 | if (!handled) | 814 | if (!handled) |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | |||
| 5 | */ | 5 | */ |
| 6 | 6 | ||
| 7 | #include <linux/kvm_host.h> | 7 | #include <linux/kvm_host.h> |
| 8 | |||
| 8 | #include <asm/kvm_emulate.h> | 9 | #include <asm/kvm_emulate.h> |
| 9 | #include <asm/kvm_hyp.h> | 10 | #include <asm/kvm_hyp.h> |
| 10 | #include <asm/kvm_mmu.h> | 11 | #include <asm/kvm_mmu.h> |
| @@ -14,6 +15,7 @@ · 3 unmodified lines | |||
| 14 | 15 | ||
| 15 | #include <hyp/fault.h> | 16 | #include <hyp/fault.h> |
| 16 | 17 | ||
| 18 | #include <nvhe/arm-smccc.h> | ||
| 17 | #include <nvhe/gfp.h> | 19 | #include <nvhe/gfp.h> |
| 18 | #include <nvhe/memory.h> | 20 | #include <nvhe/memory.h> |
| 19 | #include <nvhe/mem_protect.h> | 21 | #include <nvhe/mem_protect.h> |
| @@ -29,6 +31,19 @@ static struct hyp_pool host_s2_pool; · 9 unmodified lines | |||
| 29 | static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm); | 31 | static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm); |
| 30 | #define current_vm (*this_cpu_ptr(&__current_vm)) | 32 | #define current_vm (*this_cpu_ptr(&__current_vm)) |
| 31 | 33 | ||
| 34 | static void pkvm_sme_dvmsync_fw_call(void) | ||
| 35 | { | ||
| 36 | if (alternative_has_cap_unlikely(ARM64_WORKAROUND_4193714)) { | ||
| 37 | struct arm_smccc_res res; | ||
| 38 | |||
| 39 | /* | ||
| 40 | * Ignore the return value. Probing for the workaround | ||
| 41 | * availability took place in init_hyp_mode(). | ||
| 42 | */ | ||
| 43 | hyp_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res); | ||
| 44 | } | ||
| 45 | } | ||
| 46 | |||
| 32 | static void guest_lock_component(struct pkvm_hyp_vm *vm) | 47 | static void guest_lock_component(struct pkvm_hyp_vm *vm) |
| 33 | { | 48 | { |
| 34 | hyp_spin_lock(&vm->lock); | 49 | hyp_spin_lock(&vm->lock); |
| @@ -574,8 +589,14 @@ static int host_stage2_set_owner_metadata_locked(phys_addr_t addr, u64 size, · 539 unmodified lines | |||
| 574 | ret = host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, | 589 | ret = host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, |
| 575 | addr, size, &host_s2_pool, | 590 | addr, size, &host_s2_pool, |
| 576 | KVM_HOST_INVALID_PTE_TYPE_DONATION, annotation); | 591 | KVM_HOST_INVALID_PTE_TYPE_DONATION, annotation); |
| 577 | if (!ret) | 592 | if (!ret) { |
| 593 | /* | ||
| 594 | * After stage2 maintenance has happened, but before the page | ||
| 595 | * owner has changed. | ||
| 596 | */ | ||
| 597 | pkvm_sme_dvmsync_fw_call(); | ||
| 578 | __host_update_page_state(addr, size, PKVM_NOPAGE); | 598 | __host_update_page_state(addr, size, PKVM_NOPAGE); |
| 599 | } | ||
| 579 | 600 | ||
| 580 | return ret; | 601 | return ret; |
| 581 | } | 602 | } |
| @@ -1369,6 +1390,22 @@ unlock: · 787 unmodified lines | |||
| 1369 | return ret && ret != -EHWPOISON ? ret : 0; | 1390 | return ret && ret != -EHWPOISON ? ret : 0; |
| 1370 | } | 1391 | } |
| 1371 | 1392 | ||
| 1393 | /* | ||
| 1394 | * share/donate install at most one stage-2 leaf (PAGE_SIZE, or one | ||
| 1395 | * KVM_PGTABLE_LAST_LEVEL - 1 block for share). kvm_mmu_cache_min_pages() | ||
| 1396 | * bounds the worst-case allocation: exact for the PAGE_SIZE leaf, | ||
| 1397 | * conservative by one for the block. | ||
| 1398 | */ | ||
| 1399 | static int __guest_check_pgtable_memcache(struct pkvm_hyp_vcpu *vcpu) | ||
| 1400 | { | ||
| 1401 | struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu); | ||
| 1402 | |||
| 1403 | if (vcpu->vcpu.arch.pkvm_memcache.nr_pages < kvm_mmu_cache_min_pages(vm->pgt.mmu)) | ||
| 1404 | return -ENOMEM; | ||
| 1405 | |||
| 1406 | return 0; | ||
| 1407 | } | ||
| 1408 | |||
| 1372 | int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu) | 1409 | int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu) |
| 1373 | { | 1410 | { |
| 1374 | struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu); | 1411 | struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu); |
| @@ -1388,6 +1425,10 @@ int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu) · 13 unmodified lines | |||
| 1388 | if (ret) | 1425 | if (ret) |
| 1389 | goto unlock; | 1426 | goto unlock; |
| 1390 | 1427 | ||
| 1428 | ret = __guest_check_pgtable_memcache(vcpu); | ||
| 1429 | if (ret) | ||
| 1430 | goto unlock; | ||
| 1431 | |||
| 1391 | meta = host_stage2_encode_gfn_meta(vm, gfn); | 1432 | meta = host_stage2_encode_gfn_meta(vm, gfn); |
| 1392 | WARN_ON(host_stage2_set_owner_metadata_locked(phys, PAGE_SIZE, | 1433 | WARN_ON(host_stage2_set_owner_metadata_locked(phys, PAGE_SIZE, |
| 1393 | PKVM_ID_GUEST, meta)); | 1434 | PKVM_ID_GUEST, meta)); |
| @@ -1453,6 +1494,10 @@ int __pkvm_host_share_guest(u64 pfn, u64 gfn, u64 nr_pages, struct pkvm_hyp_vcpu · 59 unmodified lines | |||
| 1453 | } | 1494 | } |
| 1454 | } | 1495 | } |
| 1455 | 1496 | ||
| 1497 | ret = __guest_check_pgtable_memcache(vcpu); | ||
| 1498 | if (ret) | ||
| 1499 | goto unlock; | ||
| 1500 | |||
| 1456 | for_each_hyp_page(page, phys, size) { | 1501 | for_each_hyp_page(page, phys, size) { |
| 1457 | set_host_state(page, PKVM_PAGE_SHARED_OWNED); | 1502 | set_host_state(page, PKVM_PAGE_SHARED_OWNED); |
| 1458 | page->host_share_guest_count++; | 1503 | page->host_share_guest_count++; |
| @@ -266,7 +266,8 @@ struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle, · 265 unmodified lines | |||
| 266 | if (hyp_vm->kvm.created_vcpus <= vcpu_idx) | 266 | if (hyp_vm->kvm.created_vcpus <= vcpu_idx) |
| 267 | goto unlock; | 267 | goto unlock; |
| 268 | 268 | ||
| 269 | hyp_vcpu = hyp_vm->vcpus[vcpu_idx]; | 269 | /* Pairs with smp_store_release() in register_hyp_vcpu(). */ |
| 270 | hyp_vcpu = smp_load_acquire(&hyp_vm->vcpus[vcpu_idx]); | ||
| 270 | if (!hyp_vcpu) | 271 | if (!hyp_vcpu) |
| 271 | goto unlock; | 272 | goto unlock; |
| 272 | 273 | ||
| @@ -751,16 +752,30 @@ static struct pkvm_hyp_vcpu selftest_vcpu = { · 478 unmodified lines | |||
| 751 | struct pkvm_hyp_vcpu *init_selftest_vm(void *virt) | 752 | struct pkvm_hyp_vcpu *init_selftest_vm(void *virt) |
| 752 | { | 753 | { |
| 753 | struct hyp_page *p = hyp_virt_to_page(virt); | 754 | struct hyp_page *p = hyp_virt_to_page(virt); |
| 755 | unsigned long min_pages, seeded = 0; | ||
| 754 | int i; | 756 | int i; |
| 755 | 757 | ||
| 756 | selftest_vm.kvm.arch.mmu.vtcr = host_mmu.arch.mmu.vtcr; | 758 | selftest_vm.kvm.arch.mmu.vtcr = host_mmu.arch.mmu.vtcr; |
| 757 | WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, virt)); | 759 | WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, virt)); |
| 758 | 760 | ||
| 761 | /* | ||
| 762 | * Mirror pkvm_refill_memcache() for the share/donate pre-checks; | ||
| 763 | * the selftest invokes those functions directly and would | ||
| 764 | * otherwise see an empty memcache. | ||
| 765 | */ | ||
| 766 | min_pages = kvm_mmu_cache_min_pages(&selftest_vm.kvm.arch.mmu); | ||
| 767 | |||
| 759 | for (i = 0; i < pkvm_selftest_pages(); i++) { | 768 | for (i = 0; i < pkvm_selftest_pages(); i++) { |
| 760 | if (p[i].refcount) | 769 | if (p[i].refcount) |
| 761 | continue; | 770 | continue; |
| 762 | p[i].refcount = 1; | 771 | p[i].refcount = 1; |
| 763 | hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i])); | 772 | if (seeded < min_pages) { |
| 773 | push_hyp_memcache(&selftest_vcpu.vcpu.arch.pkvm_memcache, | ||
| 774 | hyp_page_to_virt(&p[i]), hyp_virt_to_phys); | ||
| 775 | seeded++; | ||
| 776 | } else { | ||
| 777 | hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i])); | ||
| 778 | } | ||
| 764 | } | 779 | } |
| 765 | 780 | ||
| 766 | selftest_vm.kvm.arch.pkvm.handle = __pkvm_reserve_vm(); | 781 | selftest_vm.kvm.arch.pkvm.handle = __pkvm_reserve_vm(); |
| @@ -860,12 +875,30 @@ err_unpin_kvm: · 93 unmodified lines | |||
| 860 | * the page-aligned size of 'struct pkvm_hyp_vcpu'. | 875 | * the page-aligned size of 'struct pkvm_hyp_vcpu'. |
| 861 | * Return 0 on success, negative error code on failure. | 876 | * Return 0 on success, negative error code on failure. |
| 862 | */ | 877 | */ |
| 878 | static int register_hyp_vcpu(struct pkvm_hyp_vm *hyp_vm, | ||
| 879 | struct pkvm_hyp_vcpu *hyp_vcpu) | ||
| 880 | { | ||
| 881 | unsigned int idx = hyp_vcpu->vcpu.vcpu_idx; | ||
| 882 | |||
| 883 | if (idx >= hyp_vm->kvm.created_vcpus) | ||
| 884 | return -EINVAL; | ||
| 885 | |||
| 886 | if (hyp_vm->vcpus[idx]) | ||
| 887 | return -EINVAL; | ||
| 888 | |||
| 889 | /* | ||
| 890 | * Ensure the hyp_vcpu is initialised before publishing it to | ||
| 891 | * the vCPU-load path via 'hyp_vm->vcpus[]'. | ||
| 892 | */ | ||
| 893 | smp_store_release(&hyp_vm->vcpus[idx], hyp_vcpu); | ||
| 894 | return 0; | ||
| 895 | } | ||
| 896 | |||
| 863 | int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu, | 897 | int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu, |
| 864 | unsigned long vcpu_hva) | 898 | unsigned long vcpu_hva) |
| 865 | { | 899 | { |
| 866 | struct pkvm_hyp_vcpu *hyp_vcpu; | 900 | struct pkvm_hyp_vcpu *hyp_vcpu; |
| 867 | struct pkvm_hyp_vm *hyp_vm; | 901 | struct pkvm_hyp_vm *hyp_vm; |
| 868 | unsigned int idx; | ||
| 869 | int ret; | 902 | int ret; |
| 870 | 903 | ||
| 871 | hyp_vcpu = map_donated_memory(vcpu_hva, sizeof(*hyp_vcpu)); | 904 | hyp_vcpu = map_donated_memory(vcpu_hva, sizeof(*hyp_vcpu)); |
| @@ -884,18 +917,11 @@ int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu, · 12 unmodified lines | |||
| 884 | if (ret) | 917 | if (ret) |
| 885 | goto unlock; | 918 | goto unlock; |
| 886 | 919 | ||
| 887 | idx = hyp_vcpu->vcpu.vcpu_idx; | 920 | ret = register_hyp_vcpu(hyp_vm, hyp_vcpu); |
| 888 | if (idx >= hyp_vm->kvm.created_vcpus) { | 921 | if (ret) { |
| 889 | ret = -EINVAL; | 922 | unpin_host_vcpu(host_vcpu); |
| 890 | goto unlock; | 923 | unpin_host_sve_state(hyp_vcpu); |
| 891 | } | ||
| 892 | |||
| 893 | if (hyp_vm->vcpus[idx]) { | ||
| 894 | ret = -EINVAL; | ||
| 895 | goto unlock; | ||
| 896 | } | 924 | } |
| 897 | |||
| 898 | hyp_vm->vcpus[idx] = hyp_vcpu; | ||
| 899 | unlock: | 925 | unlock: |
| 900 | hyp_spin_unlock(&vm_table_lock); | 926 | hyp_spin_unlock(&vm_table_lock); |
| 901 | 927 | ||
| @@ -312,15 +312,15 @@ void __noreturn __pkvm_init_finalise(void) · 311 unmodified lines | |||
| 312 | }; | 312 | }; |
| 313 | pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; | 313 | pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; |
| 314 | 314 | ||
| 315 | ret = fix_host_ownership(); | 315 | ret = fix_hyp_pgtable_refcnt(); |
| 316 | if (ret) | 316 | if (ret) |
| 317 | goto out; | 317 | goto out; |
| 318 | 318 | ||
| 319 | ret = fix_hyp_pgtable_refcnt(); | 319 | ret = hyp_create_fixmap(); |
| 320 | if (ret) | 320 | if (ret) |
| 321 | goto out; | 321 | goto out; |
| 322 | 322 | ||
| 323 | ret = hyp_create_fixmap(); | 323 | ret = fix_host_ownership(); |
| 324 | if (ret) | 324 | if (ret) |
| 325 | goto out; | 325 | goto out; |
| 326 | 326 | ||
| @@ -164,13 +164,16 @@ static int hyp_trace_buffer_load(struct hyp_trace_buffer *trace_buffer, · 163 unmodified lines | |||
| 164 | return ret; | 164 | return ret; |
| 165 | } | 165 | } |
| 166 | 166 | ||
| 167 | static bool hyp_trace_desc_validate(struct hyp_trace_desc *desc, size_t desc_size) | 167 | static bool hyp_trace_desc_is_valid(struct hyp_trace_desc *desc, size_t desc_size) |
| 168 | { | 168 | { |
| 169 | struct ring_buffer_desc *rb_desc; | 169 | struct ring_buffer_desc *rb_desc; |
| 170 | unsigned int cpu; | 170 | unsigned int cpu; |
| 171 | size_t nr_bpages; | 171 | size_t nr_bpages; |
| 172 | void *desc_end; | 172 | void *desc_end; |
| 173 | 173 | ||
| 174 | if (!is_protected_kvm_enabled()) | ||
| 175 | return true; | ||
| 176 | |||
| 174 | /* | 177 | /* |
| 175 | * Both desc_size and bpages_backing_size are untrusted host-provided | 178 | * Both desc_size and bpages_backing_size are untrusted host-provided |
| 176 | * values. We rely on __pkvm_host_donate_hyp() to enforce their validity. | 179 | * values. We rely on __pkvm_host_donate_hyp() to enforce their validity. |
| @@ -212,8 +215,10 @@ int __tracing_load(unsigned long desc_hva, size_t desc_size) · 35 unmodified lines | |||
| 212 | if (ret) | 215 | if (ret) |
| 213 | return ret; | 216 | return ret; |
| 214 | 217 | ||
| 215 | if (!hyp_trace_desc_validate(desc, desc_size)) | 218 | if (!hyp_trace_desc_is_valid(desc, desc_size)) { |
| 219 | ret = -EINVAL; | ||
| 216 | goto err_release_desc; | 220 | goto err_release_desc; |
| 221 | } | ||
| 217 | 222 | ||
| 218 | hyp_spin_lock(&trace_buffer.lock); | 223 | hyp_spin_lock(&trace_buffer.lock); |
| 219 | 224 | ||
| @@ -663,7 +663,8 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) · 662 unmodified lines | |||
| 663 | host_ctxt = host_data_ptr(host_ctxt); | 663 | host_ctxt = host_data_ptr(host_ctxt); |
| 664 | vcpu = host_ctxt->__hyp_running_vcpu; | 664 | vcpu = host_ctxt->__hyp_running_vcpu; |
| 665 | 665 | ||
| 666 | __deactivate_traps(vcpu); | 666 | if (vcpu) |
| 667 | __deactivate_traps(vcpu); | ||
| 667 | sysreg_restore_host_state_vhe(host_ctxt); | 668 | sysreg_restore_host_state_vhe(host_ctxt); |
| 668 | 669 | ||
| 669 | panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", | 670 | panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", |
| @@ -189,7 +189,7 @@ static void hyp_trace_buffer_unshare_hyp(struct hyp_trace_buffer *trace_buffer, · 188 unmodified lines | |||
| 189 | if (cpu > last_cpu) | 189 | if (cpu > last_cpu) |
| 190 | break; | 190 | break; |
| 191 | 191 | ||
| 192 | __share_page(rb_desc->meta_va); | 192 | __unshare_page(rb_desc->meta_va); |
| 193 | for (p = 0; p < rb_desc->nr_page_va; p++) | 193 | for (p = 0; p < rb_desc->nr_page_va; p++) |
| 194 | __unshare_page(rb_desc->page_va[p]); | 194 | __unshare_page(rb_desc->page_va[p]); |
| 195 | } | 195 | } |
| @@ -212,14 +212,15 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) · 16 unmodified lines | |||
| 212 | } | 212 | } |
| 213 | 213 | ||
| 214 | if (ret) { | 214 | if (ret) { |
| 215 | for (p--; p >= 0; p--) | 215 | while (--p >= 0) |
| 216 | __unshare_page(rb_desc->page_va[p]); | 216 | __unshare_page(rb_desc->page_va[p]); |
| 217 | __unshare_page(rb_desc->meta_va); | ||
| 217 | break; | 218 | break; |
| 218 | } | 219 | } |
| 219 | } | 220 | } |
| 220 | 221 | ||
| 221 | if (ret) | 222 | if (ret) |
| 222 | hyp_trace_buffer_unshare_hyp(trace_buffer, cpu--); | 223 | hyp_trace_buffer_unshare_hyp(trace_buffer, --cpu); |
| 223 | 224 | ||
| 224 | return ret; | 225 | return ret; |
| 225 | } | 226 | } |
| @@ -248,6 +249,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) · 22 unmodified lines | |||
| 248 | goto err_free_desc; | 249 | goto err_free_desc; |
| 249 | 250 | ||
| 250 | trace_buffer->desc = desc; | 251 | trace_buffer->desc = desc; |
| 252 | trace_buffer->desc_size = desc_size; | ||
| 251 | 253 | ||
| 252 | ret = hyp_trace_buffer_alloc_bpages_backing(trace_buffer, size); | 254 | ret = hyp_trace_buffer_alloc_bpages_backing(trace_buffer, size); |
| 253 | if (ret) | 255 | if (ret) |
| @@ -297,6 +299,7 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) · 43 unmodified lines | |||
| 297 | hyp_trace_buffer_free_bpages_backing(trace_buffer); | 299 | hyp_trace_buffer_free_bpages_backing(trace_buffer); |
| 298 | free_pages_exact(trace_buffer->desc, trace_buffer->desc_size); | 300 | free_pages_exact(trace_buffer->desc, trace_buffer->desc_size); |
| 299 | trace_buffer->desc = NULL; | 301 | trace_buffer->desc = NULL; |
| 302 | trace_buffer->desc_size = 0; | ||
| 300 | } | 303 | } |
| 301 | 304 | ||
| 302 | static int hyp_trace_enable_tracing(bool enable, void *priv) | 305 | static int hyp_trace_enable_tracing(bool enable, void *priv) |
| @@ -1576,21 +1576,24 @@ struct kvm_s2_fault_desc { · 1575 unmodified lines | |||
| 1576 | static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) | 1576 | static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) |
| 1577 | { | 1577 | { |
| 1578 | bool write_fault, exec_fault; | 1578 | bool write_fault, exec_fault; |
| 1579 | bool perm_fault = kvm_vcpu_trap_is_permission_fault(s2fd->vcpu); | ||
| 1579 | enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED; | 1580 | enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED; |
| 1580 | enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; | 1581 | enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; |
| 1581 | struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt; | 1582 | struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt; |
| 1582 | unsigned long mmu_seq; | 1583 | unsigned long mmu_seq; |
| 1583 | struct page *page; | 1584 | struct page *page; |
| 1584 | struct kvm *kvm = s2fd->vcpu->kvm; | 1585 | struct kvm *kvm = s2fd->vcpu->kvm; |
| 1585 | void *memcache; | 1586 | void *memcache = NULL; |
| 1586 | kvm_pfn_t pfn; | 1587 | kvm_pfn_t pfn; |
| 1587 | gfn_t gfn; | 1588 | gfn_t gfn; |
| 1588 | int ret; | 1589 | int ret; |
| 1589 | 1590 | ||
| 1590 | memcache = get_mmu_memcache(s2fd->vcpu); | 1591 | if (!perm_fault) { |
| 1591 | ret = topup_mmu_memcache(s2fd->vcpu, memcache); | 1592 | memcache = get_mmu_memcache(s2fd->vcpu); |
| 1592 | if (ret) | 1593 | ret = topup_mmu_memcache(s2fd->vcpu, memcache); |
| 1593 | return ret; | 1594 | if (ret) |
| 1595 | return ret; | ||
| 1596 | } | ||
| 1594 | 1597 | ||
| 1595 | if (s2fd->nested) | 1598 | if (s2fd->nested) |
| 1596 | gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT; | 1599 | gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT; |
| @@ -1631,9 +1634,19 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) · 34 unmodified lines | |||
| 1631 | goto out_unlock; | 1634 | goto out_unlock; |
| 1632 | } | 1635 | } |
| 1633 | 1636 | ||
| 1634 | ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE, | 1637 | if (perm_fault) { |
| 1635 | __pfn_to_phys(pfn), prot, | 1638 | /* |
| 1636 | memcache, flags); | 1639 | * Drop the SW bits in favour of those stored in the |
| 1640 | * PTE, which will be preserved. | ||
| 1641 | */ | ||
| 1642 | prot &= ~KVM_NV_GUEST_MAP_SZ; | ||
| 1643 | ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa, | ||
| 1644 | prot, flags); | ||
| 1645 | } else { | ||
| 1646 | ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE, | ||
| 1647 | __pfn_to_phys(pfn), prot, | ||
| 1648 | memcache, flags); | ||
| 1649 | } | ||
| 1637 | 1650 | ||
| 1638 | out_unlock: | 1651 | out_unlock: |
| 1639 | kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W); | 1652 | kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W); |
| @@ -1834,6 +1834,11 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) · 1833 unmodified lines | |||
| 1834 | resx.res1 = VNCR_EL2_RES1; | 1834 | resx.res1 = VNCR_EL2_RES1; |
| 1835 | set_sysreg_masks(kvm, VNCR_EL2, resx); | 1835 | set_sysreg_masks(kvm, VNCR_EL2, resx); |
| 1836 | 1836 | ||
| 1837 | /* ZCR_EL2 - bits 8:4 are RAZ/WI so treat them as RES0 */ | ||
| 1838 | resx.res0 = ZCR_ELx_RES0 | GENMASK_ULL(8, 4); | ||
| 1839 | resx.res1 = ZCR_ELx_RES1; | ||
| 1840 | set_sysreg_masks(kvm, ZCR_EL2, resx); | ||
| 1841 | |||
| 1837 | out: | 1842 | out: |
| 1838 | for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) | 1843 | for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) |
| 1839 | __vcpu_rmw_sys_reg(vcpu, sr, |=, 0); | 1844 | __vcpu_rmw_sys_reg(vcpu, sr, |=, 0); |
| @@ -174,8 +174,8 @@ static void kvm_pmu_set_pmc_value(struct kvm_pmc *pmc, u64 val, bool force) · 173 unmodified lines | |||
| 174 | * action is to use PMCR.P, which will reset them to | 174 | * action is to use PMCR.P, which will reset them to |
| 175 | * 0 (the only use of the 'force' parameter). | 175 | * 0 (the only use of the 'force' parameter). |
| 176 | */ | 176 | */ |
| 177 | val = __vcpu_sys_reg(vcpu, reg) & GENMASK(63, 32); | 177 | val = (__vcpu_sys_reg(vcpu, reg) & GENMASK(63, 32)) | |
| 178 | val |= lower_32_bits(val); | 178 | lower_32_bits(val); |
| 179 | } | 179 | } |
| 180 | 180 | ||
| 181 | __vcpu_assign_sys_reg(vcpu, reg, val); | 181 | __vcpu_assign_sys_reg(vcpu, reg, val); |
| @@ -2862,21 +2862,16 @@ static bool access_zcr_el2(struct kvm_vcpu *vcpu, · 2861 unmodified lines | |||
| 2862 | struct sys_reg_params *p, | 2862 | struct sys_reg_params *p, |
| 2863 | const struct sys_reg_desc *r) | 2863 | const struct sys_reg_desc *r) |
| 2864 | { | 2864 | { |
| 2865 | unsigned int vq; | ||
| 2866 | |||
| 2867 | if (guest_hyp_sve_traps_enabled(vcpu)) { | 2865 | if (guest_hyp_sve_traps_enabled(vcpu)) { |
| 2868 | kvm_inject_nested_sve_trap(vcpu); | 2866 | kvm_inject_nested_sve_trap(vcpu); |
| 2869 | return false; | 2867 | return false; |
| 2870 | } | 2868 | } |
| 2871 | 2869 | ||
| 2872 | if (!p->is_write) { | 2870 | if (!p->is_write) |
| 2873 | p->regval = __vcpu_sys_reg(vcpu, ZCR_EL2); | 2871 | p->regval = __vcpu_sys_reg(vcpu, ZCR_EL2); |
| 2874 | return true; | 2872 | else |
| 2875 | } | 2873 | __vcpu_assign_sys_reg(vcpu, ZCR_EL2, p->regval); |
| 2876 | 2874 | ||
| 2877 | vq = SYS_FIELD_GET(ZCR_ELx, LEN, p->regval) + 1; | ||
| 2878 | vq = min(vq, vcpu_sve_max_vq(vcpu)); | ||
| 2879 | __vcpu_assign_sys_reg(vcpu, ZCR_EL2, vq - 1); | ||
| 2880 | return true; | 2875 | return true; |
| 2881 | } | 2876 | } |
| 2882 | 2877 | ||
| @@ -2307,6 +2307,10 @@ static int vgic_its_restore_dte(struct vgic_its *its, u32 id, · 2306 unmodified lines | |||
| 2307 | /* dte entry is valid */ | 2307 | /* dte entry is valid */ |
| 2308 | offset = (entry & KVM_ITS_DTE_NEXT_MASK) >> KVM_ITS_DTE_NEXT_SHIFT; | 2308 | offset = (entry & KVM_ITS_DTE_NEXT_MASK) >> KVM_ITS_DTE_NEXT_SHIFT; |
| 2309 | 2309 | ||
| 2310 | /* Mimic the MAPD behaviour and reject invalid EID bits. */ | ||
| 2311 | if (num_eventid_bits > VITS_TYPER_IDBITS) | ||
| 2312 | return -EINVAL; | ||
| 2313 | |||
| 2310 | if (!vgic_its_check_id(its, baser, id, NULL)) | 2314 | if (!vgic_its_check_id(its, baser, id, NULL)) |
| 2311 | return -EINVAL; | 2315 | return -EINVAL; |
| 2312 | 2316 | ||
| @@ -91,7 +91,7 @@ static int vgic_mmio_uaccess_write_v2_misc(struct kvm_vcpu *vcpu, · 90 unmodified lines | |||
| 91 | * migration from old kernels to new kernels with legacy | 91 | * migration from old kernels to new kernels with legacy |
| 92 | * userspace. | 92 | * userspace. |
| 93 | */ | 93 | */ |
| 94 | reg = FIELD_GET(GICD_IIDR_REVISION_MASK, reg); | 94 | reg = FIELD_GET(GICD_IIDR_REVISION_MASK, val); |
| 95 | switch (reg) { | 95 | switch (reg) { |
| 96 | case KVM_VGIC_IMP_REV_2: | 96 | case KVM_VGIC_IMP_REV_2: |
| 97 | case KVM_VGIC_IMP_REV_3: | 97 | case KVM_VGIC_IMP_REV_3: |
| @@ -194,7 +194,7 @@ static int vgic_mmio_uaccess_write_v3_misc(struct kvm_vcpu *vcpu, · 193 unmodified lines | |||
| 194 | if ((reg ^ val) & ~GICD_IIDR_REVISION_MASK) | 194 | if ((reg ^ val) & ~GICD_IIDR_REVISION_MASK) |
| 195 | return -EINVAL; | 195 | return -EINVAL; |
| 196 | 196 | ||
| 197 | reg = FIELD_GET(GICD_IIDR_REVISION_MASK, reg); | 197 | reg = FIELD_GET(GICD_IIDR_REVISION_MASK, val); |
| 198 | switch (reg) { | 198 | switch (reg) { |
| 199 | case KVM_VGIC_IMP_REV_2: | 199 | case KVM_VGIC_IMP_REV_2: |
| 200 | case KVM_VGIC_IMP_REV_3: | 200 | case KVM_VGIC_IMP_REV_3: |
| @@ -1018,7 +1018,7 @@ struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, · 1017 unmodified lines | |||
| 1018 | return vma_alloc_folio(flags, 0, vma, vaddr); | 1018 | return vma_alloc_folio(flags, 0, vma, vaddr); |
| 1019 | } | 1019 | } |
| 1020 | 1020 | ||
| 1021 | bool tag_clear_highpages(struct page *page, int numpages) | 1021 | bool tag_clear_highpages(struct page *page, int numpages, bool clear_pages) |
| 1022 | { | 1022 | { |
| 1023 | /* | 1023 | /* |
| 1024 | * Check if MTE is supported and fall back to clear_highpage(). | 1024 | * Check if MTE is supported and fall back to clear_highpage(). |
| @@ -1026,13 +1026,16 @@ bool tag_clear_highpages(struct page *page, int numpages) · 1 unmodified line | |||
| 1026 | * post_alloc_hook() will invoke tag_clear_highpages(). | 1026 | * post_alloc_hook() will invoke tag_clear_highpages(). |
| 1027 | */ | 1027 | */ |
| 1028 | if (!system_supports_mte()) | 1028 | if (!system_supports_mte()) |
| 1029 | return false; | 1029 | return clear_pages; |
| 1030 | 1030 | ||
| 1031 | /* Newly allocated pages, shouldn't have been tagged yet */ | 1031 | /* Newly allocated pages, shouldn't have been tagged yet */ |
| 1032 | for (int i = 0; i < numpages; i++, page++) { | 1032 | for (int i = 0; i < numpages; i++, page++) { |
| 1033 | WARN_ON_ONCE(!try_page_mte_tagging(page)); | 1033 | WARN_ON_ONCE(!try_page_mte_tagging(page)); |
| 1034 | mte_zero_clear_page_tags(page_address(page)); | 1034 | if (clear_pages) |
| 1035 | mte_zero_clear_page_tags(page_address(page)); | ||
| 1036 | else | ||
| 1037 | mte_clear_page_tags(page_address(page)); | ||
| 1035 | set_page_mte_tagged(page); | 1038 | set_page_mte_tagged(page); |
| 1036 | } | 1039 | } |
| 1037 | return true; | 1040 | return false; |
| 1038 | } | 1041 | } |
| @@ -9,6 +9,7 @@ generic-y += qrwlock.h · 8 unmodified lines | |||
| 9 | generic-y += qrwlock_types.h | 9 | generic-y += qrwlock_types.h |
| 10 | generic-y += qspinlock.h | 10 | generic-y += qspinlock.h |
| 11 | generic-y += parport.h | 11 | generic-y += parport.h |
| 12 | generic-y += ring_buffer.h | ||
| 12 | generic-y += user.h | 13 | generic-y += user.h |
| 13 | generic-y += vmlinux.lds.h | 14 | generic-y += vmlinux.lds.h |
| 14 | generic-y += text-patching.h | 15 | generic-y += text-patching.h |
| @@ -5,4 +5,5 @@ generic-y += extable.h · 4 unmodified lines | |||
| 5 | generic-y += iomap.h | 5 | generic-y += iomap.h |
| 6 | generic-y += kvm_para.h | 6 | generic-y += kvm_para.h |
| 7 | generic-y += mcs_spinlock.h | 7 | generic-y += mcs_spinlock.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += text-patching.h | 9 | generic-y += text-patching.h |
| @@ -3,7 +3,7 @@ obj-y += mm/ · 2 unmodified lines | |||
| 3 | obj-y += net/ | 3 | obj-y += net/ |
| 4 | obj-y += vdso/ | 4 | obj-y += vdso/ |
| 5 | 5 | ||
| 6 | obj-$(CONFIG_KVM) += kvm/ | 6 | obj-$(subst m,y,$(CONFIG_KVM)) += kvm/ |
| 7 | 7 | ||
| 8 | # for cleaning | 8 | # for cleaning |
| 9 | subdir- += boot | 9 | subdir- += boot |
| @@ -220,6 +220,7 @@ menu "Kernel type and options" · 219 unmodified lines | |||
| 220 | 220 | ||
| 221 | choice | 221 | choice |
| 222 | prompt "Kernel type" | 222 | prompt "Kernel type" |
| 223 | default 64BIT # Keep existing behavior | ||
| 223 | 224 | ||
| 224 | config 32BIT | 225 | config 32BIT |
| 225 | bool "32-bit kernel" | 226 | bool "32-bit kernel" |
| @@ -55,9 +55,11 @@ endif · 54 unmodified lines | |||
| 55 | ifdef CONFIG_32BIT | 55 | ifdef CONFIG_32BIT |
| 56 | tool-archpref = $(32bit-tool-archpref) | 56 | tool-archpref = $(32bit-tool-archpref) |
| 57 | UTS_MACHINE := loongarch32 | 57 | UTS_MACHINE := loongarch32 |
| 58 | cflags-y += $(call cc-option,-m32) | ||
| 58 | else | 59 | else |
| 59 | tool-archpref = $(64bit-tool-archpref) | 60 | tool-archpref = $(64bit-tool-archpref) |
| 60 | UTS_MACHINE := loongarch64 | 61 | UTS_MACHINE := loongarch64 |
| 62 | cflags-y += $(call cc-option,-m64) | ||
| 61 | endif | 63 | endif |
| 62 | 64 | ||
| 63 | ifneq ($(SUBARCH),$(ARCH)) | 65 | ifneq ($(SUBARCH),$(ARCH)) |
| @@ -10,5 +10,6 @@ generic-y += qrwlock.h · 9 unmodified lines | |||
| 10 | generic-y += user.h | 10 | generic-y += user.h |
| 11 | generic-y += ioctl.h | 11 | generic-y += ioctl.h |
| 12 | generic-y += mmzone.h | 12 | generic-y += mmzone.h |
| 13 | generic-y += ring_buffer.h | ||
| 13 | generic-y += statfs.h | 14 | generic-y += statfs.h |
| 14 | generic-y += text-patching.h | 15 | generic-y += text-patching.h |
| @@ -20,3 +20,23 @@ asmlinkage void noinstr __no_stack_protector ret_from_kernel_thread(struct task_ · 19 unmodified lines | |||
| 20 | struct pt_regs *regs, | 20 | struct pt_regs *regs, |
| 21 | int (*fn)(void *), | 21 | int (*fn)(void *), |
| 22 | void *fn_arg); | 22 | void *fn_arg); |
| 23 | |||
| 24 | struct kvm_run; | ||
| 25 | struct kvm_vcpu; | ||
| 26 | struct loongarch_fpu; | ||
| 27 | |||
| 28 | void kvm_exc_entry(void); | ||
| 29 | int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu); | ||
| 30 | |||
| 31 | void kvm_save_fpu(struct loongarch_fpu *fpu); | ||
| 32 | void kvm_restore_fpu(struct loongarch_fpu *fpu); | ||
| 33 | |||
| 34 | #ifdef CONFIG_CPU_HAS_LSX | ||
| 35 | void kvm_save_lsx(struct loongarch_fpu *fpu); | ||
| 36 | void kvm_restore_lsx(struct loongarch_fpu *fpu); | ||
| 37 | #endif | ||
| 38 | |||
| 39 | #ifdef CONFIG_CPU_HAS_LASX | ||
| 40 | void kvm_save_lasx(struct loongarch_fpu *fpu); | ||
| 41 | void kvm_restore_lasx(struct loongarch_fpu *fpu); | ||
| 42 | #endif | ||
| @@ -30,6 +30,8 @@ static inline unsigned long efi_get_kimg_min_align(void) · 29 unmodified lines | |||
| 30 | return SZ_2M; | 30 | return SZ_2M; |
| 31 | } | 31 | } |
| 32 | 32 | ||
| 33 | #define EFI_KIMG_PREFERRED_ADDRESS PHYSADDR(VMLINUX_LOAD_ADDRESS) | 33 | unsigned long efi_get_kimg_kaslr_address(void); |
| 34 | |||
| 35 | #define EFI_KIMG_PREFERRED_ADDRESS efi_get_kimg_kaslr_address() | ||
| 34 | 36 | ||
| 35 | #endif /* _ASM_LOONGARCH_EFI_H */ | 37 | #endif /* _ASM_LOONGARCH_EFI_H */ |
| @@ -87,7 +87,6 @@ struct kvm_context { · 86 unmodified lines | |||
| 87 | struct kvm_world_switch { | 87 | struct kvm_world_switch { |
| 88 | int (*exc_entry)(void); | 88 | int (*exc_entry)(void); |
| 89 | int (*enter_guest)(struct kvm_run *run, struct kvm_vcpu *vcpu); | 89 | int (*enter_guest)(struct kvm_run *run, struct kvm_vcpu *vcpu); |
| 90 | unsigned long page_order; | ||
| 91 | }; | 90 | }; |
| 92 | 91 | ||
| 93 | #define MAX_PGTABLE_LEVELS 4 | 92 | #define MAX_PGTABLE_LEVELS 4 |
| @@ -359,8 +358,6 @@ void kvm_exc_entry(void); · 265 unmodified lines | |||
| 359 | int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu); | 358 | int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu); |
| 360 | 359 | ||
| 361 | extern unsigned long vpid_mask; | 360 | extern unsigned long vpid_mask; |
| 362 | extern const unsigned long kvm_exception_size; | ||
| 363 | extern const unsigned long kvm_enter_guest_size; | ||
| 364 | extern struct kvm_world_switch *kvm_loongarch_ops; | 361 | extern struct kvm_world_switch *kvm_loongarch_ops; |
| 365 | 362 | ||
| 366 | #define SW_GCSR (1 << 0) | 363 | #define SW_GCSR (1 << 0) |
| @@ -69,7 +69,7 @@ · 68 unmodified lines | |||
| 69 | 9, 10, 11, 12, 13, 14, 15, 16, \ | 69 | 9, 10, 11, 12, 13, 14, 15, 16, \ |
| 70 | 17, 18, 19, 20, 21, 22, 23, 24, \ | 70 | 17, 18, 19, 20, 21, 22, 23, 24, \ |
| 71 | 25, 26, 27, 28, 29, 30, 31; \ | 71 | 25, 26, 27, 28, 29, 30, 31; \ |
| 72 | .cfi_offset \num, SC_REGS + \num * SZREG; \ | 72 | .cfi_offset \num, SC_REGS + \num * 8; \ |
| 73 | .endr; \ | 73 | .endr; \ |
| 74 | \ | 74 | \ |
| 75 | nop; \ | 75 | nop; \ |
| @@ -4,6 +4,12 @@ · 3 unmodified lines | |||
| 4 | 4 | ||
| 5 | #ifdef CONFIG_PARAVIRT | 5 | #ifdef CONFIG_PARAVIRT |
| 6 | 6 | ||
| 7 | #include <linux/jump_label.h> | ||
| 8 | |||
| 9 | DECLARE_STATIC_KEY_FALSE(virt_preempt_key); | ||
| 10 | DECLARE_STATIC_KEY_FALSE(virt_spin_lock_key); | ||
| 11 | DECLARE_PER_CPU(struct kvm_steal_time, steal_time); | ||
| 12 | |||
| 7 | int __init pv_ipi_init(void); | 13 | int __init pv_ipi_init(void); |
| 8 | int __init pv_time_init(void); | 14 | int __init pv_time_init(void); |
| 9 | int __init pv_spinlock_init(void); | 15 | int __init pv_spinlock_init(void); |
| @@ -3,12 +3,9 @@ · 2 unmodified lines | |||
| 3 | #define _ASM_LOONGARCH_QSPINLOCK_H | 3 | #define _ASM_LOONGARCH_QSPINLOCK_H |
| 4 | 4 | ||
| 5 | #include <asm/kvm_para.h> | 5 | #include <asm/kvm_para.h> |
| 6 | #include <linux/jump_label.h> | 6 | #include <asm/paravirt.h> |
| 7 | 7 | ||
| 8 | #ifdef CONFIG_PARAVIRT | 8 | #ifdef CONFIG_PARAVIRT |
| 9 | DECLARE_STATIC_KEY_FALSE(virt_preempt_key); | ||
| 10 | DECLARE_STATIC_KEY_FALSE(virt_spin_lock_key); | ||
| 11 | DECLARE_PER_CPU(struct kvm_steal_time, steal_time); | ||
| 12 | 9 | ||
| 13 | #define virt_spin_lock virt_spin_lock | 10 | #define virt_spin_lock virt_spin_lock |
| 14 | 11 | ||
| @@ -85,12 +85,6 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode, · 84 unmodified lines | |||
| 85 | return count; | 85 | return count; |
| 86 | } | 86 | } |
| 87 | 87 | ||
| 88 | static inline bool loongarch_vdso_hres_capable(void) | ||
| 89 | { | ||
| 90 | return true; | ||
| 91 | } | ||
| 92 | #define __arch_vdso_hres_capable loongarch_vdso_hres_capable | ||
| 93 | |||
| 94 | #endif /* CONFIG_GENERIC_GETTIMEOFDAY */ | 88 | #endif /* CONFIG_GENERIC_GETTIMEOFDAY */ |
| 95 | 89 | ||
| 96 | #endif /* !__ASSEMBLER__ */ | 90 | #endif /* !__ASSEMBLER__ */ |
| @@ -60,16 +60,18 @@ NOKPROBE_SYMBOL(arch_prepare_kprobe); · 59 unmodified lines | |||
| 60 | /* Install breakpoint in text */ | 60 | /* Install breakpoint in text */ |
| 61 | void arch_arm_kprobe(struct kprobe *p) | 61 | void arch_arm_kprobe(struct kprobe *p) |
| 62 | { | 62 | { |
| 63 | *p->addr = KPROBE_BP_INSN; | 63 | u32 insn = KPROBE_BP_INSN; |
| 64 | flush_insn_slot(p); | 64 | |
| 65 | larch_insn_text_copy(p->addr, &insn, LOONGARCH_INSN_SIZE); | ||
| 65 | } | 66 | } |
| 66 | NOKPROBE_SYMBOL(arch_arm_kprobe); | 67 | NOKPROBE_SYMBOL(arch_arm_kprobe); |
| 67 | 68 | ||
| 68 | /* Remove breakpoint from text */ | 69 | /* Remove breakpoint from text */ |
| 69 | void arch_disarm_kprobe(struct kprobe *p) | 70 | void arch_disarm_kprobe(struct kprobe *p) |
| 70 | { | 71 | { |
| 71 | *p->addr = p->opcode; | 72 | u32 insn = p->opcode; |
| 72 | flush_insn_slot(p); | 73 | |
| 74 | larch_insn_text_copy(p->addr, &insn, LOONGARCH_INSN_SIZE); | ||
| 73 | } | 75 | } |
| 74 | NOKPROBE_SYMBOL(arch_disarm_kprobe); | 76 | NOKPROBE_SYMBOL(arch_disarm_kprobe); |
| 75 | 77 | ||
| @@ -184,16 +186,16 @@ static bool reenter_kprobe(struct kprobe *p, struct pt_regs *regs, · 108 unmodified lines | |||
| 184 | struct kprobe_ctlblk *kcb) | 186 | struct kprobe_ctlblk *kcb) |
| 185 | { | 187 | { |
| 186 | switch (kcb->kprobe_status) { | 188 | switch (kcb->kprobe_status) { |
| 187 | case KPROBE_HIT_SS: | ||
| 188 | case KPROBE_HIT_SSDONE: | 189 | case KPROBE_HIT_SSDONE: |
| 189 | case KPROBE_HIT_ACTIVE: | 190 | case KPROBE_HIT_ACTIVE: |
| 190 | kprobes_inc_nmissed_count(p); | 191 | kprobes_inc_nmissed_count(p); |
| 191 | setup_singlestep(p, regs, kcb, 1); | 192 | setup_singlestep(p, regs, kcb, 1); |
| 192 | break; | 193 | break; |
| 194 | case KPROBE_HIT_SS: | ||
| 193 | case KPROBE_REENTER: | 195 | case KPROBE_REENTER: |
| 194 | pr_warn("Failed to recover from reentered kprobes.\n"); | 196 | pr_warn("Failed to recover from reentered kprobes.\n"); |
| 195 | dump_kprobe(p); | 197 | dump_kprobe(p); |
| 196 | WARN_ON_ONCE(1); | 198 | BUG(); |
| 197 | break; | 199 | break; |
| 198 | default: | 200 | default: |
| 199 | WARN_ON(1); | 201 | WARN_ON(1); |
| @@ -134,11 +134,23 @@ early_param("nokaslr", nokaslr); · 133 unmodified lines | |||
| 134 | 134 | ||
| 135 | #define KASLR_DISABLED_MESSAGE "KASLR is disabled by %s in %s cmdline.\n" | 135 | #define KASLR_DISABLED_MESSAGE "KASLR is disabled by %s in %s cmdline.\n" |
| 136 | 136 | ||
| 137 | /* | ||
| 138 | * Note: strictly-defined KASLR means the kernel's final runtime address | ||
| 139 | * has a random offset from the kernel's load address, which is implemented | ||
| 140 | * in relocate.c; broadly-defined KALSR means the kernel's final runtime | ||
| 141 | * address has a random offset from the kernel's link address (a.k.a. | ||
| 142 | * VMLINUX_LOAD_ADDRESS), which also include the efistlub implementation, | ||
| 143 | * kexec_file implementation and QEMU direct kernel boot. kaslr_disabled() | ||
| 144 | * return true only means strictly-defined KASLR is disabled. | ||
| 145 | */ | ||
| 137 | static inline __init bool kaslr_disabled(void) | 146 | static inline __init bool kaslr_disabled(void) |
| 138 | { | 147 | { |
| 139 | char *str; | 148 | char *str; |
| 140 | const char *builtin_cmdline = CONFIG_CMDLINE; | 149 | const char *builtin_cmdline = CONFIG_CMDLINE; |
| 141 | 150 | ||
| 151 | if (kaslr_offset()) | ||
| 152 | return true; /* KASLR is performed during early boot. */ | ||
| 153 | |||
| 142 | str = strstr(builtin_cmdline, "nokaslr"); | 154 | str = strstr(builtin_cmdline, "nokaslr"); |
| 143 | if (str == builtin_cmdline || (str > builtin_cmdline && *(str - 1) == ' ')) { | 155 | if (str == builtin_cmdline || (str > builtin_cmdline && *(str - 1) == ' ')) { |
| 144 | pr_info(KASLR_DISABLED_MESSAGE, "\'nokaslr\'", "built-in"); | 156 | pr_info(KASLR_DISABLED_MESSAGE, "\'nokaslr\'", "built-in"); |
| @@ -210,14 +222,52 @@ static inline void __init *determine_relocation_address(void) · 65 unmodified lines | |||
| 210 | return RELOCATED_KASLR(destination); | 222 | return RELOCATED_KASLR(destination); |
| 211 | } | 223 | } |
| 212 | 224 | ||
| 225 | static unsigned long __init determine_initrd_address(unsigned long *size) | ||
| 226 | { | ||
| 227 | unsigned long start = 0; | ||
| 228 | unsigned long key_length; | ||
| 229 | char *p, *endp, *key = "initrd="; | ||
| 230 | |||
| 231 | key_length = strlen(key); | ||
| 232 | p = strstr(boot_command_line, key); | ||
| 233 | |||
| 234 | if (!p) { | ||
| 235 | key = "initrdmem="; | ||
| 236 | key_length = strlen(key); | ||
| 237 | p = strstr(boot_command_line, key); | ||
| 238 | } | ||
| 239 | |||
| 240 | if (p == boot_command_line || (p > boot_command_line && *(p - 1) == ' ')) { | ||
| 241 | p += key_length; | ||
| 242 | start = memparse(p, &endp); | ||
| 243 | if (*endp == ',') | ||
| 244 | *size = memparse(endp + 1, NULL); | ||
| 245 | } | ||
| 246 | |||
| 247 | return start; | ||
| 248 | } | ||
| 249 | |||
| 213 | static inline int __init relocation_addr_valid(void *location_new) | 250 | static inline int __init relocation_addr_valid(void *location_new) |
| 214 | { | 251 | { |
| 252 | unsigned long kernel_start, kernel_size; | ||
| 253 | unsigned long initrd_start, initrd_size = 0; | ||
| 254 | |||
| 215 | if ((unsigned long)location_new & 0x00000ffff) | 255 | if ((unsigned long)location_new & 0x00000ffff) |
| 216 | return 0; /* Inappropriately aligned new location */ | 256 | return 0; /* Inappropriately aligned new location */ |
| 217 | 257 | ||
| 218 | if ((unsigned long)location_new < (unsigned long)_end) | 258 | if ((unsigned long)location_new < (unsigned long)_end) |
| 219 | return 0; /* New location overlaps original kernel */ | 259 | return 0; /* New location overlaps original kernel */ |
| 220 | 260 | ||
| 261 | initrd_start = determine_initrd_address(&initrd_size); | ||
| 262 | if (initrd_start && initrd_size) { | ||
| 263 | kernel_start = PHYSADDR(location_new); | ||
| 264 | kernel_size = (unsigned long)_end - (unsigned long)_text; | ||
| 265 | |||
| 266 | if (kernel_start < (initrd_start + initrd_size) && | ||
| 267 | initrd_start < (kernel_start + kernel_size)) | ||
| 268 | return 0; /* initrd/initramfs overlaps kernel */ | ||
| 269 | } | ||
| 270 | |||
| 221 | return 1; | 271 | return 1; |
| 222 | } | 272 | } |
| 223 | #endif | 273 | #endif |
| @@ -7,11 +7,12 @@ include $(srctree)/virt/kvm/Makefile.kvm · 6 unmodified lines | |||
| 7 | 7 | ||
| 8 | obj-$(CONFIG_KVM) += kvm.o | 8 | obj-$(CONFIG_KVM) += kvm.o |
| 9 | 9 | ||
| 10 | obj-y += switch.o | ||
| 11 | |||
| 10 | kvm-y += exit.o | 12 | kvm-y += exit.o |
| 11 | kvm-y += interrupt.o | 13 | kvm-y += interrupt.o |
| 12 | kvm-y += main.o | 14 | kvm-y += main.o |
| 13 | kvm-y += mmu.o | 15 | kvm-y += mmu.o |
| 14 | kvm-y += switch.o | ||
| 15 | kvm-y += timer.o | 16 | kvm-y += timer.o |
| 16 | kvm-y += tlb.o | 17 | kvm-y += tlb.o |
| 17 | kvm-y += vcpu.o | 18 | kvm-y += vcpu.o |
| @@ -390,6 +390,7 @@ int kvm_emu_mmio_read(struct kvm_vcpu *vcpu, larch_inst inst) · 389 unmodified lines | |||
| 390 | run->mmio.len = 8; | 390 | run->mmio.len = 8; |
| 391 | break; | 391 | break; |
| 392 | default: | 392 | default: |
| 393 | ret = EMULATE_FAIL; | ||
| 393 | break; | 394 | break; |
| 394 | } | 395 | } |
| 395 | break; | 396 | break; |
| @@ -28,23 +28,29 @@ static unsigned int priority_to_irq[EXCCODE_INT_NUM] = { · 27 unmodified lines | |||
| 28 | static int kvm_irq_deliver(struct kvm_vcpu *vcpu, unsigned int priority) | 28 | static int kvm_irq_deliver(struct kvm_vcpu *vcpu, unsigned int priority) |
| 29 | { | 29 | { |
| 30 | unsigned int irq = 0; | 30 | unsigned int irq = 0; |
| 31 | unsigned long old, new; | ||
| 31 | 32 | ||
| 32 | clear_bit(priority, &vcpu->arch.irq_pending); | 33 | clear_bit(priority, &vcpu->arch.irq_pending); |
| 33 | if (priority < EXCCODE_INT_NUM) | 34 | if (priority < EXCCODE_INT_NUM) |
| 34 | irq = priority_to_irq[priority]; | 35 | irq = priority_to_irq[priority]; |
| 35 | 36 | ||
| 36 | if (kvm_guest_has_msgint(&vcpu->arch) && (priority == INT_AVEC)) { | ||
| 37 | dmsintc_inject_irq(vcpu); | ||
| 38 | set_gcsr_estat(irq); | ||
| 39 | return 1; | ||
| 40 | } | ||
| 41 | |||
| 42 | switch (priority) { | 37 | switch (priority) { |
| 38 | case INT_AVEC: | ||
| 39 | if (!kvm_guest_has_msgint(&vcpu->arch)) | ||
| 40 | break; | ||
| 41 | dmsintc_inject_irq(vcpu); | ||
| 42 | fallthrough; | ||
| 43 | case INT_TI: | 43 | case INT_TI: |
| 44 | case INT_IPI: | 44 | case INT_IPI: |
| 45 | case INT_SWI0: | 45 | case INT_SWI0: |
| 46 | case INT_SWI1: | 46 | case INT_SWI1: |
| 47 | old = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | ||
| 47 | set_gcsr_estat(irq); | 48 | set_gcsr_estat(irq); |
| 49 | new = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | ||
| 50 | |||
| 51 | /* Inject TI if TVAL inverted */ | ||
| 52 | if (new > old) | ||
| 53 | set_gcsr_estat(CPU_TIMER); | ||
| 48 | break; | 54 | break; |
| 49 | 55 | ||
| 50 | case INT_HWI0 ... INT_HWI7: | 56 | case INT_HWI0 ... INT_HWI7: |
| @@ -61,22 +67,28 @@ static int kvm_irq_deliver(struct kvm_vcpu *vcpu, unsigned int priority) · 10 unmodified lines | |||
| 61 | static int kvm_irq_clear(struct kvm_vcpu *vcpu, unsigned int priority) | 67 | static int kvm_irq_clear(struct kvm_vcpu *vcpu, unsigned int priority) |
| 62 | { | 68 | { |
| 63 | unsigned int irq = 0; | 69 | unsigned int irq = 0; |
| 70 | unsigned long old, new; | ||
| 64 | 71 | ||
| 65 | clear_bit(priority, &vcpu->arch.irq_clear); | 72 | clear_bit(priority, &vcpu->arch.irq_clear); |
| 66 | if (priority < EXCCODE_INT_NUM) | 73 | if (priority < EXCCODE_INT_NUM) |
| 67 | irq = priority_to_irq[priority]; | 74 | irq = priority_to_irq[priority]; |
| 68 | 75 | ||
| 69 | if (kvm_guest_has_msgint(&vcpu->arch) && (priority == INT_AVEC)) { | ||
| 70 | clear_gcsr_estat(irq); | ||
| 71 | return 1; | ||
| 72 | } | ||
| 73 | |||
| 74 | switch (priority) { | 76 | switch (priority) { |
| 77 | case INT_AVEC: | ||
| 78 | if (!kvm_guest_has_msgint(&vcpu->arch)) | ||
| 79 | break; | ||
| 80 | fallthrough; | ||
| 75 | case INT_TI: | 81 | case INT_TI: |
| 76 | case INT_IPI: | 82 | case INT_IPI: |
| 77 | case INT_SWI0: | 83 | case INT_SWI0: |
| 78 | case INT_SWI1: | 84 | case INT_SWI1: |
| 85 | old = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | ||
| 79 | clear_gcsr_estat(irq); | 86 | clear_gcsr_estat(irq); |
| 87 | new = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | ||
| 88 | |||
| 89 | /* Inject TI if TVAL inverted */ | ||
| 90 | if (new > old) | ||
| 91 | set_gcsr_estat(CPU_TIMER); | ||
| 80 | break; | 92 | break; |
| 81 | 93 | ||
| 82 | case INT_HWI0 ... INT_HWI7: | 94 | case INT_HWI0 ... INT_HWI7: |
| @@ -348,8 +348,7 @@ void kvm_arch_disable_virtualization_cpu(void) · 347 unmodified lines | |||
| 348 | 348 | ||
| 349 | static int kvm_loongarch_env_init(void) | 349 | static int kvm_loongarch_env_init(void) |
| 350 | { | 350 | { |
| 351 | int cpu, order, ret; | 351 | int cpu, ret; |
| 352 | void *addr; | ||
| 353 | struct kvm_context *context; | 352 | struct kvm_context *context; |
| 354 | 353 | ||
| 355 | vmcs = alloc_percpu(struct kvm_context); | 354 | vmcs = alloc_percpu(struct kvm_context); |
| @@ -365,30 +364,8 @@ static int kvm_loongarch_env_init(void) · 9 unmodified lines | |||
| 365 | return -ENOMEM; | 364 | return -ENOMEM; |
| 366 | } | 365 | } |
| 367 | 366 | ||
| 368 | /* | 367 | kvm_loongarch_ops->exc_entry = (void *)kvm_exc_entry; |
| 369 | * PGD register is shared between root kernel and kvm hypervisor. | 368 | kvm_loongarch_ops->enter_guest = (void *)kvm_enter_guest; |
| 370 | * So world switch entry should be in DMW area rather than TLB area | ||
| 371 | * to avoid page fault reenter. | ||
| 372 | * | ||
| 373 | * In future if hardware pagetable walking is supported, we won't | ||
| 374 | * need to copy world switch code to DMW area. | ||
| 375 | */ | ||
| 376 | order = get_order(kvm_exception_size + kvm_enter_guest_size); | ||
| 377 | addr = (void *)__get_free_pages(GFP_KERNEL, order); | ||
| 378 | if (!addr) { | ||
| 379 | free_percpu(vmcs); | ||
| 380 | vmcs = NULL; | ||
| 381 | kfree(kvm_loongarch_ops); | ||
| 382 | kvm_loongarch_ops = NULL; | ||
| 383 | return -ENOMEM; | ||
| 384 | } | ||
| 385 | |||
| 386 | memcpy(addr, kvm_exc_entry, kvm_exception_size); | ||
| 387 | memcpy(addr + kvm_exception_size, kvm_enter_guest, kvm_enter_guest_size); | ||
| 388 | flush_icache_range((unsigned long)addr, (unsigned long)addr + kvm_exception_size + kvm_enter_guest_size); | ||
| 389 | kvm_loongarch_ops->exc_entry = addr; | ||
| 390 | kvm_loongarch_ops->enter_guest = addr + kvm_exception_size; | ||
| 391 | kvm_loongarch_ops->page_order = order; | ||
| 392 | 369 | ||
| 393 | vpid_mask = read_csr_gstat(); | 370 | vpid_mask = read_csr_gstat(); |
| 394 | vpid_mask = (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_GSTAT_GIDBIT_SHIFT; | 371 | vpid_mask = (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_GSTAT_GIDBIT_SHIFT; |
| @@ -428,16 +405,10 @@ static int kvm_loongarch_env_init(void) · 33 unmodified lines | |||
| 428 | 405 | ||
| 429 | static void kvm_loongarch_env_exit(void) | 406 | static void kvm_loongarch_env_exit(void) |
| 430 | { | 407 | { |
| 431 | unsigned long addr; | ||
| 432 | |||
| 433 | if (vmcs) | 408 | if (vmcs) |
| 434 | free_percpu(vmcs); | 409 | free_percpu(vmcs); |
| 435 | 410 | ||
| 436 | if (kvm_loongarch_ops) { | 411 | if (kvm_loongarch_ops) { |
| 437 | if (kvm_loongarch_ops->exc_entry) { | ||
| 438 | addr = (unsigned long)kvm_loongarch_ops->exc_entry; | ||
| 439 | free_pages(addr, kvm_loongarch_ops->page_order); | ||
| 440 | } | ||
| 441 | kfree(kvm_loongarch_ops); | 412 | kfree(kvm_loongarch_ops); |
| 442 | } | 413 | } |
| 443 | 414 | ||
| @@ -95,7 +95,7 @@ static int kvm_flush_pte(kvm_pte_t *pte, phys_addr_t addr, kvm_ptw_ctx *ctx) · 94 unmodified lines | |||
| 95 | else | 95 | else |
| 96 | kvm->stat.pages--; | 96 | kvm->stat.pages--; |
| 97 | 97 | ||
| 98 | *pte = ctx->invalid_entry; | 98 | kvm_set_pte(pte, ctx->invalid_entry); |
| 99 | 99 | ||
| 100 | return 1; | 100 | return 1; |
| 101 | } | 101 | } |
| @@ -4,9 +4,11 @@ · 3 unmodified lines | |||
| 4 | */ | 4 | */ |
| 5 | 5 | ||
| 6 | #include <linux/linkage.h> | 6 | #include <linux/linkage.h> |
| 7 | #include <linux/kvm_types.h> | ||
| 7 | #include <asm/asm.h> | 8 | #include <asm/asm.h> |
| 8 | #include <asm/asmmacro.h> | 9 | #include <asm/asmmacro.h> |
| 9 | #include <asm/loongarch.h> | 10 | #include <asm/loongarch.h> |
| 11 | #include <asm/page.h> | ||
| 10 | #include <asm/regdef.h> | 12 | #include <asm/regdef.h> |
| 11 | #include <asm/unwind_hints.h> | 13 | #include <asm/unwind_hints.h> |
| 12 | 14 | ||
| @@ -100,11 +102,16 @@ · 87 unmodified lines | |||
| 100 | * - is still in guest mode, such as pgd table/vmid registers etc, | 102 | * - is still in guest mode, such as pgd table/vmid registers etc, |
| 101 | * - will fix with hw page walk enabled in future | 103 | * - will fix with hw page walk enabled in future |
| 102 | * load kvm_vcpu from reserved CSR KVM_VCPU_KS, and save a2 to KVM_TEMP_KS | 104 | * load kvm_vcpu from reserved CSR KVM_VCPU_KS, and save a2 to KVM_TEMP_KS |
| 105 | * | ||
| 106 | * PGD register is shared between root kernel and kvm hypervisor. | ||
| 107 | * So world switch entry should be in DMW area rather than TLB area | ||
| 108 | * to avoid page fault re-enter. | ||
| 103 | */ | 109 | */ |
| 104 | .text | 110 | .text |
| 111 | .p2align PAGE_SHIFT | ||
| 105 | .cfi_sections .debug_frame | 112 | .cfi_sections .debug_frame |
| 106 | SYM_CODE_START(kvm_exc_entry) | 113 | SYM_CODE_START(kvm_exc_entry) |
| 107 | UNWIND_HINT_UNDEFINED | 114 | UNWIND_HINT_END_OF_STACK |
| 108 | csrwr a2, KVM_TEMP_KS | 115 | csrwr a2, KVM_TEMP_KS |
| 109 | csrrd a2, KVM_VCPU_KS | 116 | csrrd a2, KVM_VCPU_KS |
| 110 | addi.d a2, a2, KVM_VCPU_ARCH | 117 | addi.d a2, a2, KVM_VCPU_ARCH |
| @@ -190,8 +197,8 @@ ret_to_host: · 79 unmodified lines | |||
| 190 | kvm_restore_host_gpr a2 | 197 | kvm_restore_host_gpr a2 |
| 191 | jr ra | 198 | jr ra |
| 192 | 199 | ||
| 193 | SYM_INNER_LABEL(kvm_exc_entry_end, SYM_L_LOCAL) | ||
| 194 | SYM_CODE_END(kvm_exc_entry) | 200 | SYM_CODE_END(kvm_exc_entry) |
| 201 | EXPORT_SYMBOL_FOR_KVM(kvm_exc_entry) | ||
| 195 | 202 | ||
| 196 | /* | 203 | /* |
| 197 | * int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu) | 204 | * int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu) |
| @@ -215,8 +222,8 @@ SYM_FUNC_START(kvm_enter_guest) · 17 unmodified lines | |||
| 215 | /* Save kvm_vcpu to kscratch */ | 222 | /* Save kvm_vcpu to kscratch */ |
| 216 | csrwr a1, KVM_VCPU_KS | 223 | csrwr a1, KVM_VCPU_KS |
| 217 | kvm_switch_to_guest | 224 | kvm_switch_to_guest |
| 218 | SYM_INNER_LABEL(kvm_enter_guest_end, SYM_L_LOCAL) | ||
| 219 | SYM_FUNC_END(kvm_enter_guest) | 225 | SYM_FUNC_END(kvm_enter_guest) |
| 226 | EXPORT_SYMBOL_FOR_KVM(kvm_enter_guest) | ||
| 220 | 227 | ||
| 221 | SYM_FUNC_START(kvm_save_fpu) | 228 | SYM_FUNC_START(kvm_save_fpu) |
| 222 | fpu_save_csr a0 t1 | 229 | fpu_save_csr a0 t1 |
| @@ -224,6 +231,7 @@ SYM_FUNC_START(kvm_save_fpu) · 1 unmodified line | |||
| 224 | fpu_save_cc a0 t1 t2 | 231 | fpu_save_cc a0 t1 t2 |
| 225 | jr ra | 232 | jr ra |
| 226 | SYM_FUNC_END(kvm_save_fpu) | 233 | SYM_FUNC_END(kvm_save_fpu) |
| 234 | EXPORT_SYMBOL_FOR_KVM(kvm_save_fpu) | ||
| 227 | 235 | ||
| 228 | SYM_FUNC_START(kvm_restore_fpu) | 236 | SYM_FUNC_START(kvm_restore_fpu) |
| 229 | fpu_restore_double a0 t1 | 237 | fpu_restore_double a0 t1 |
| @@ -231,6 +239,7 @@ SYM_FUNC_START(kvm_restore_fpu) · 1 unmodified line | |||
| 231 | fpu_restore_cc a0 t1 t2 | 239 | fpu_restore_cc a0 t1 t2 |
| 232 | jr ra | 240 | jr ra |
| 233 | SYM_FUNC_END(kvm_restore_fpu) | 241 | SYM_FUNC_END(kvm_restore_fpu) |
| 242 | EXPORT_SYMBOL_FOR_KVM(kvm_restore_fpu) | ||
| 234 | 243 | ||
| 235 | #ifdef CONFIG_CPU_HAS_LSX | 244 | #ifdef CONFIG_CPU_HAS_LSX |
| 236 | SYM_FUNC_START(kvm_save_lsx) | 245 | SYM_FUNC_START(kvm_save_lsx) |
| @@ -239,6 +248,7 @@ SYM_FUNC_START(kvm_save_lsx) · 2 unmodified lines | |||
| 239 | lsx_save_data a0 t1 | 248 | lsx_save_data a0 t1 |
| 240 | jr ra | 249 | jr ra |
| 241 | SYM_FUNC_END(kvm_save_lsx) | 250 | SYM_FUNC_END(kvm_save_lsx) |
| 251 | EXPORT_SYMBOL_FOR_KVM(kvm_save_lsx) | ||
| 242 | 252 | ||
| 243 | SYM_FUNC_START(kvm_restore_lsx) | 253 | SYM_FUNC_START(kvm_restore_lsx) |
| 244 | lsx_restore_data a0 t1 | 254 | lsx_restore_data a0 t1 |
| @@ -246,6 +256,7 @@ SYM_FUNC_START(kvm_restore_lsx) · 1 unmodified line | |||
| 246 | fpu_restore_csr a0 t1 t2 | 256 | fpu_restore_csr a0 t1 t2 |
| 247 | jr ra | 257 | jr ra |
| 248 | SYM_FUNC_END(kvm_restore_lsx) | 258 | SYM_FUNC_END(kvm_restore_lsx) |
| 259 | EXPORT_SYMBOL_FOR_KVM(kvm_restore_lsx) | ||
| 249 | #endif | 260 | #endif |
| 250 | 261 | ||
| 251 | #ifdef CONFIG_CPU_HAS_LASX | 262 | #ifdef CONFIG_CPU_HAS_LASX |
| @@ -255,6 +266,7 @@ SYM_FUNC_START(kvm_save_lasx) · 3 unmodified lines | |||
| 255 | lasx_save_data a0 t1 | 266 | lasx_save_data a0 t1 |
| 256 | jr ra | 267 | jr ra |
| 257 | SYM_FUNC_END(kvm_save_lasx) | 268 | SYM_FUNC_END(kvm_save_lasx) |
| 269 | EXPORT_SYMBOL_FOR_KVM(kvm_save_lasx) | ||
| 258 | 270 | ||
| 259 | SYM_FUNC_START(kvm_restore_lasx) | 271 | SYM_FUNC_START(kvm_restore_lasx) |
| 260 | lasx_restore_data a0 t1 | 272 | lasx_restore_data a0 t1 |
| @@ -262,10 +274,8 @@ SYM_FUNC_START(kvm_restore_lasx) · 1 unmodified line | |||
| 262 | fpu_restore_csr a0 t1 t2 | 274 | fpu_restore_csr a0 t1 t2 |
| 263 | jr ra | 275 | jr ra |
| 264 | SYM_FUNC_END(kvm_restore_lasx) | 276 | SYM_FUNC_END(kvm_restore_lasx) |
| 277 | EXPORT_SYMBOL_FOR_KVM(kvm_restore_lasx) | ||
| 265 | #endif | 278 | #endif |
| 266 | .section ".rodata" | ||
| 267 | SYM_DATA(kvm_exception_size, .quad kvm_exc_entry_end - kvm_exc_entry) | ||
| 268 | SYM_DATA(kvm_enter_guest_size, .quad kvm_enter_guest_end - kvm_enter_guest) | ||
| 269 | 279 | ||
| 270 | #ifdef CONFIG_CPU_HAS_LBT | 280 | #ifdef CONFIG_CPU_HAS_LBT |
| 271 | STACK_FRAME_NON_STANDARD kvm_restore_fpu | 281 | STACK_FRAME_NON_STANDARD kvm_restore_fpu |
| @@ -96,15 +96,21 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu) · 95 unmodified lines | |||
| 96 | * and set CSR TVAL with -1 | 96 | * and set CSR TVAL with -1 |
| 97 | */ | 97 | */ |
| 98 | write_gcsr_timertick(0); | 98 | write_gcsr_timertick(0); |
| 99 | __delay(2); /* Wait cycles until timer interrupt injected */ | ||
| 100 | 99 | ||
| 101 | /* | 100 | /* |
| 102 | * Writing CSR_TINTCLR_TI to LOONGARCH_CSR_TINTCLR will clear | 101 | * Writing CSR_TINTCLR_TI to LOONGARCH_CSR_TINTCLR will clear |
| 103 | * timer interrupt, and CSR TVAL keeps unchanged with -1, it | 102 | * timer interrupt, and CSR TVAL keeps unchanged with -1, it |
| 104 | * avoids spurious timer interrupt | 103 | * avoids spurious timer interrupt |
| 105 | */ | 104 | */ |
| 106 | if (!(estat & CPU_TIMER)) | 105 | if (!(estat & CPU_TIMER)) { |
| 106 | __delay(2); /* Wait cycles until timer interrupt injected */ | ||
| 107 | |||
| 108 | /* Write TVAL with max value if no TI shot */ | ||
| 109 | estat = kvm_read_hw_gcsr(LOONGARCH_CSR_ESTAT); | ||
| 110 | if (!(estat & CPU_TIMER)) | ||
| 111 | write_gcsr_timertick(CSR_TCFG_VAL); | ||
| 107 | gcsr_write(CSR_TINTCLR_TI, LOONGARCH_CSR_TINTCLR); | 112 | gcsr_write(CSR_TINTCLR_TI, LOONGARCH_CSR_TINTCLR); |
| 113 | } | ||
| 108 | return; | 114 | return; |
| 109 | } | 115 | } |
| 110 | 116 | ||
| @@ -125,7 +125,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 124 unmodified lines | |||
| 125 | r = 1; | 125 | r = 1; |
| 126 | break; | 126 | break; |
| 127 | case KVM_CAP_NR_VCPUS: | 127 | case KVM_CAP_NR_VCPUS: |
| 128 | r = num_online_cpus(); | 128 | r = min_t(unsigned int, num_online_cpus(), KVM_MAX_VCPUS); |
| 129 | break; | 129 | break; |
| 130 | case KVM_CAP_MAX_VCPUS: | 130 | case KVM_CAP_MAX_VCPUS: |
| 131 | r = KVM_MAX_VCPUS; | 131 | r = KVM_MAX_VCPUS; |
| @@ -123,11 +123,7 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) · 122 unmodified lines | |||
| 123 | { | 123 | { |
| 124 | unsigned long start_pfn = start >> PAGE_SHIFT; | 124 | unsigned long start_pfn = start >> PAGE_SHIFT; |
| 125 | unsigned long nr_pages = size >> PAGE_SHIFT; | 125 | unsigned long nr_pages = size >> PAGE_SHIFT; |
| 126 | struct page *page = pfn_to_page(start_pfn); | ||
| 127 | 126 | ||
| 128 | /* With altmap the first mapped page is offset from @start */ | ||
| 129 | if (altmap) | ||
| 130 | page += vmem_altmap_offset(altmap); | ||
| 131 | __remove_pages(start_pfn, nr_pages, altmap); | 127 | __remove_pages(start_pfn, nr_pages, altmap); |
| 132 | } | 128 | } |
| 133 | #endif | 129 | #endif |
| @@ -61,11 +61,16 @@ static void acpi_release_root_info(struct acpi_pci_root_info *ci) · 60 unmodified lines | |||
| 61 | static int acpi_prepare_root_resources(struct acpi_pci_root_info *ci) | 61 | static int acpi_prepare_root_resources(struct acpi_pci_root_info *ci) |
| 62 | { | 62 | { |
| 63 | int status; | 63 | int status; |
| 64 | unsigned long long pci_h = 0; | ||
| 64 | struct resource_entry *entry, *tmp; | 65 | struct resource_entry *entry, *tmp; |
| 65 | struct acpi_device *device = ci->bridge; | 66 | struct acpi_device *device = ci->bridge; |
| 66 | 67 | ||
| 67 | status = acpi_pci_probe_root_resources(ci); | 68 | status = acpi_pci_probe_root_resources(ci); |
| 68 | if (status > 0) { | 69 | if (status > 0) { |
| 70 | acpi_evaluate_integer(device->handle, "PCIH", NULL, &pci_h); | ||
| 71 | if (pci_h) | ||
| 72 | return status; | ||
| 73 | |||
| 69 | resource_list_for_each_entry_safe(entry, tmp, &ci->resources) { | 74 | resource_list_for_each_entry_safe(entry, tmp, &ci->resources) { |
| 70 | if (entry->res->flags & IORESOURCE_MEM) { | 75 | if (entry->res->flags & IORESOURCE_MEM) { |
| 71 | entry->offset = ci->root->mcfg_addr & GENMASK_ULL(63, 40); | 76 | entry->offset = ci->root->mcfg_addr & GENMASK_ULL(63, 40); |
| @@ -132,6 +132,9 @@ static void loongson_gpu_fixup_dma_hang(struct pci_dev *pdev, bool on) · 131 unmodified lines | |||
| 132 | crtc_reg = regbase; | 132 | crtc_reg = regbase; |
| 133 | crtc_offset = 0x400; | 133 | crtc_offset = 0x400; |
| 134 | break; | 134 | break; |
| 135 | default: | ||
| 136 | iounmap(regbase); | ||
| 137 | return; | ||
| 135 | } | 138 | } |
| 136 | 139 | ||
| 137 | for (i = 0; i < CRTC_NUM_MAX; i++, crtc_reg += crtc_offset) { | 140 | for (i = 0; i < CRTC_NUM_MAX; i++, crtc_reg += crtc_offset) { |
| @@ -12,6 +12,8 @@ obj-vdso-$(CONFIG_GENERIC_GETTIMEOFDAY) += vgettimeofday.o · 11 unmodified lines | |||
| 12 | ccflags-vdso := \ | 12 | ccflags-vdso := \ |
| 13 | $(filter -I%,$(KBUILD_CFLAGS)) \ | 13 | $(filter -I%,$(KBUILD_CFLAGS)) \ |
| 14 | $(filter -E%,$(KBUILD_CFLAGS)) \ | 14 | $(filter -E%,$(KBUILD_CFLAGS)) \ |
| 15 | $(filter -m32,$(KBUILD_CFLAGS)) \ | ||
| 16 | $(filter -m64,$(KBUILD_CFLAGS)) \ | ||
| 15 | $(filter -march=%,$(KBUILD_CFLAGS)) \ | 17 | $(filter -march=%,$(KBUILD_CFLAGS)) \ |
| 16 | $(filter -m%-float,$(KBUILD_CFLAGS)) \ | 18 | $(filter -m%-float,$(KBUILD_CFLAGS)) \ |
| 17 | $(CLANG_FLAGS) \ | 19 | $(CLANG_FLAGS) \ |
| @@ -3,5 +3,6 @@ generated-y += syscall_table.h · 2 unmodified lines | |||
| 3 | generic-y += extable.h | 3 | generic-y += extable.h |
| 4 | generic-y += kvm_para.h | 4 | generic-y += kvm_para.h |
| 5 | generic-y += mcs_spinlock.h | 5 | generic-y += mcs_spinlock.h |
| 6 | generic-y += ring_buffer.h | ||
| 6 | generic-y += spinlock.h | 7 | generic-y += spinlock.h |
| 7 | generic-y += text-patching.h | 8 | generic-y += text-patching.h |
| @@ -5,6 +5,7 @@ generic-y += extable.h · 4 unmodified lines | |||
| 5 | generic-y += kvm_para.h | 5 | generic-y += kvm_para.h |
| 6 | generic-y += mcs_spinlock.h | 6 | generic-y += mcs_spinlock.h |
| 7 | generic-y += parport.h | 7 | generic-y += parport.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += syscalls.h | 9 | generic-y += syscalls.h |
| 9 | generic-y += tlb.h | 10 | generic-y += tlb.h |
| 10 | generic-y += user.h | 11 | generic-y += user.h |
| @@ -10,6 +10,14 @@ · 9 unmodified lines | |||
| 10 | #include <linux/mc146818rtc.h> | 10 | #include <linux/mc146818rtc.h> |
| 11 | #include <linux/platform_device.h> | 11 | #include <linux/platform_device.h> |
| 12 | 12 | ||
| 13 | #include <asm/bootinfo.h> | ||
| 14 | |||
| 15 | #include <asm/dec/interrupts.h> | ||
| 16 | #include <asm/dec/ioasic_addrs.h> | ||
| 17 | #include <asm/dec/kn01.h> | ||
| 18 | #include <asm/dec/kn02.h> | ||
| 19 | #include <asm/dec/system.h> | ||
| 20 | |||
| 13 | static struct resource dec_rtc_resources[] = { | 21 | static struct resource dec_rtc_resources[] = { |
| 14 | { | 22 | { |
| 15 | .name = "rtc", | 23 | .name = "rtc", |
| @@ -30,11 +38,110 @@ static struct platform_device dec_rtc_device = { · 14 unmodified lines | |||
| 30 | .num_resources = ARRAY_SIZE(dec_rtc_resources), | 38 | .num_resources = ARRAY_SIZE(dec_rtc_resources), |
| 31 | }; | 39 | }; |
| 32 | 40 | ||
| 41 | static struct resource dec_dz_resources[] = { | ||
| 42 | { .name = "dz", .flags = IORESOURCE_MEM, }, | ||
| 43 | { .name = "dz", .flags = IORESOURCE_IRQ, }, | ||
| 44 | }; | ||
| 45 | |||
| 46 | static struct platform_device dec_dz_device = { | ||
| 47 | .name = "dz", | ||
| 48 | .id = PLATFORM_DEVID_NONE, | ||
| 49 | .resource = dec_dz_resources, | ||
| 50 | .num_resources = ARRAY_SIZE(dec_dz_resources), | ||
| 51 | }; | ||
| 52 | |||
| 53 | static struct platform_device *dec_dz_devices[] __initdata = { | ||
| 54 | &dec_dz_device, | ||
| 55 | }; | ||
| 56 | |||
| 57 | static struct resource dec_zs_resources[][2] = { | ||
| 58 | { | ||
| 59 | { .name = "scc0", .flags = IORESOURCE_MEM, }, | ||
| 60 | { .name = "scc0", .flags = IORESOURCE_IRQ, }, | ||
| 61 | }, | ||
| 62 | { | ||
| 63 | { .name = "scc1", .flags = IORESOURCE_MEM, }, | ||
| 64 | { .name = "scc1", .flags = IORESOURCE_IRQ, }, | ||
| 65 | }, | ||
| 66 | }; | ||
| 67 | |||
| 68 | static struct platform_device dec_zs_device[] = { | ||
| 69 | { | ||
| 70 | .name = "zs", | ||
| 71 | .id = 0, | ||
| 72 | .resource = dec_zs_resources[0], | ||
| 73 | .num_resources = ARRAY_SIZE(dec_zs_resources[0]), | ||
| 74 | }, | ||
| 75 | { | ||
| 76 | .name = "zs", | ||
| 77 | .id = 1, | ||
| 78 | .resource = dec_zs_resources[1], | ||
| 79 | .num_resources = ARRAY_SIZE(dec_zs_resources[1]), | ||
| 80 | }, | ||
| 81 | }; | ||
| 82 | |||
| 33 | static int __init dec_add_devices(void) | 83 | static int __init dec_add_devices(void) |
| 34 | { | 84 | { |
| 85 | struct platform_device *dec_zs_devices[ARRAY_SIZE(dec_zs_device)]; | ||
| 86 | int ret1, ret2, ret3; | ||
| 87 | int num_dz, num_zs; | ||
| 88 | int irq, i; | ||
| 89 | |||
| 35 | dec_rtc_resources[0].start = RTC_PORT(0); | 90 | dec_rtc_resources[0].start = RTC_PORT(0); |
| 36 | dec_rtc_resources[0].end = RTC_PORT(0) + dec_kn_slot_size - 1; | 91 | dec_rtc_resources[0].end = RTC_PORT(0) + dec_kn_slot_size - 1; |
| 37 | return platform_device_register(&dec_rtc_device); | 92 | |
| 93 | i = 0; | ||
| 94 | irq = dec_interrupt[DEC_IRQ_DZ11]; | ||
| 95 | if (IS_ENABLED(CONFIG_32BIT) && irq >= 0) { | ||
| 96 | resource_size_t base; | ||
| 97 | |||
| 98 | switch (mips_machtype) { | ||
| 99 | case MACH_DS23100: | ||
| 100 | case MACH_DS5100: | ||
| 101 | base = dec_kn_slot_base + KN01_DZ11; | ||
| 102 | break; | ||
| 103 | default: | ||
| 104 | base = dec_kn_slot_base + KN02_DZ11; | ||
| 105 | break; | ||
| 106 | } | ||
| 107 | dec_dz_device.resource[0].start = base; | ||
| 108 | dec_dz_device.resource[0].end = base + dec_kn_slot_size - 1; | ||
| 109 | dec_dz_device.resource[1].start = irq; | ||
| 110 | dec_dz_device.resource[1].end = irq; | ||
| 111 | i++; | ||
| 112 | } | ||
| 113 | num_dz = i; | ||
| 114 | |||
| 115 | i = 0; | ||
| 116 | irq = dec_interrupt[DEC_IRQ_SCC0]; | ||
| 117 | if (irq >= 0) { | ||
| 118 | resource_size_t base = dec_kn_slot_base + IOASIC_SCC0; | ||
| 119 | |||
| 120 | dec_zs_device[i].resource[0].start = base; | ||
| 121 | dec_zs_device[i].resource[0].end = base + dec_kn_slot_size - 1; | ||
| 122 | dec_zs_device[i].resource[1].start = irq; | ||
| 123 | dec_zs_device[i].resource[1].end = irq; | ||
| 124 | dec_zs_devices[i] = &dec_zs_device[i]; | ||
| 125 | i++; | ||
| 126 | } | ||
| 127 | irq = dec_interrupt[DEC_IRQ_SCC1]; | ||
| 128 | if (irq >= 0) { | ||
| 129 | resource_size_t base = dec_kn_slot_base + IOASIC_SCC1; | ||
| 130 | |||
| 131 | dec_zs_device[i].resource[0].start = base; | ||
| 132 | dec_zs_device[i].resource[0].end = base + dec_kn_slot_size - 1; | ||
| 133 | dec_zs_device[i].resource[1].start = irq; | ||
| 134 | dec_zs_device[i].resource[1].end = irq; | ||
| 135 | dec_zs_devices[i] = &dec_zs_device[i]; | ||
| 136 | i++; | ||
| 137 | } | ||
| 138 | num_zs = i; | ||
| 139 | |||
| 140 | ret1 = platform_device_register(&dec_rtc_device); | ||
| 141 | ret2 = IS_ENABLED(CONFIG_32BIT) ? | ||
| 142 | platform_add_devices(dec_dz_devices, num_dz) : 0; | ||
| 143 | ret3 = platform_add_devices(dec_zs_devices, num_zs); | ||
| 144 | return ret1 ? ret1 : ret2 ? ret2 : ret3; | ||
| 38 | } | 145 | } |
| 39 | 146 | ||
| 40 | device_initcall(dec_add_devices); | 147 | device_initcall(dec_add_devices); |
| @@ -12,5 +12,6 @@ generic-y += mcs_spinlock.h · 11 unmodified lines | |||
| 12 | generic-y += parport.h | 12 | generic-y += parport.h |
| 13 | generic-y += qrwlock.h | 13 | generic-y += qrwlock.h |
| 14 | generic-y += qspinlock.h | 14 | generic-y += qspinlock.h |
| 15 | generic-y += ring_buffer.h | ||
| 15 | generic-y += user.h | 16 | generic-y += user.h |
| 16 | generic-y += text-patching.h | 17 | generic-y += text-patching.h |
| @@ -5,6 +5,7 @@ generic-y += cmpxchg.h · 4 unmodified lines | |||
| 5 | generic-y += extable.h | 5 | generic-y += extable.h |
| 6 | generic-y += kvm_para.h | 6 | generic-y += kvm_para.h |
| 7 | generic-y += mcs_spinlock.h | 7 | generic-y += mcs_spinlock.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += spinlock.h | 9 | generic-y += spinlock.h |
| 9 | generic-y += user.h | 10 | generic-y += user.h |
| 10 | generic-y += text-patching.h | 11 | generic-y += text-patching.h |
| @@ -12,4 +12,6 @@ · 11 unmodified lines | |||
| 12 | #define __ALIGN .align 4 | 12 | #define __ALIGN .align 4 |
| 13 | #define __ALIGN_STR ".align 4" | 13 | #define __ALIGN_STR ".align 4" |
| 14 | 14 | ||
| 15 | #define _THIS_IP_ ({ unsigned long __ip; asm volatile("nextpc %0" : "=r" (__ip)); __ip; }) | ||
| 16 | |||
| 15 | #endif | 17 | #endif |
| @@ -8,4 +8,5 @@ generic-y += spinlock_types.h · 7 unmodified lines | |||
| 8 | generic-y += spinlock.h | 8 | generic-y += spinlock.h |
| 9 | generic-y += qrwlock_types.h | 9 | generic-y += qrwlock_types.h |
| 10 | generic-y += qrwlock.h | 10 | generic-y += qrwlock.h |
| 11 | generic-y += ring_buffer.h | ||
| 11 | generic-y += user.h | 12 | generic-y += user.h |
| @@ -174,15 +174,21 @@ ifeq ($(KBUILD_EXTMOD),) · 173 unmodified lines | |||
| 174 | # this hack. | 174 | # this hack. |
| 175 | prepare: vdso_prepare | 175 | prepare: vdso_prepare |
| 176 | vdso_prepare: prepare0 | 176 | vdso_prepare: prepare0 |
| 177 | $(if $(CONFIG_64BIT),$(Q)$(MAKE) \ | 177 | ifdef CONFIG_64BIT |
| 178 | $(build)=arch/parisc/kernel/vdso64 include/generated/vdso64-offsets.h) | 178 | $(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso64 include/generated/vdso64-offsets.h |
| 179 | $(if $(CONFIG_PA11)$(CONFIG_COMPAT),$(Q)$(MAKE) \ | 179 | $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \ |
| 180 | $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h) | 180 | $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h) |
| 181 | else | ||
| 182 | $(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h | ||
| 183 | endif | ||
| 181 | endif | 184 | endif |
| 182 | 185 | ||
| 183 | vdso-install-$(CONFIG_PA11) += arch/parisc/kernel/vdso32/vdso32.so | 186 | ifdef CONFIG_64BIT |
| 187 | vdso-install-y += arch/parisc/kernel/vdso64/vdso64.so | ||
| 184 | vdso-install-$(CONFIG_COMPAT) += arch/parisc/kernel/vdso32/vdso32.so | 188 | vdso-install-$(CONFIG_COMPAT) += arch/parisc/kernel/vdso32/vdso32.so |
| 185 | vdso-install-$(CONFIG_64BIT) += arch/parisc/kernel/vdso64/vdso64.so | 189 | else |
| 190 | vdso-install-y += arch/parisc/kernel/vdso32/vdso32.so | ||
| 191 | endif | ||
| 186 | 192 | ||
| 187 | install: KBUILD_IMAGE := vmlinux | 193 | install: KBUILD_IMAGE := vmlinux |
| 188 | zinstall: KBUILD_IMAGE := vmlinuz | 194 | zinstall: KBUILD_IMAGE := vmlinuz |
| @@ -4,4 +4,5 @@ generated-y += syscall_table_64.h · 3 unmodified lines | |||
| 4 | generic-y += agp.h | 4 | generic-y += agp.h |
| 5 | generic-y += kvm_para.h | 5 | generic-y += kvm_para.h |
| 6 | generic-y += mcs_spinlock.h | 6 | generic-y += mcs_spinlock.h |
| 7 | generic-y += ring_buffer.h | ||
| 7 | generic-y += user.h | 8 | generic-y += user.h |
| @@ -6,13 +6,14 @@ · 5 unmodified lines | |||
| 6 | 6 | ||
| 7 | #ifdef CONFIG_64BIT | 7 | #ifdef CONFIG_64BIT |
| 8 | #include <generated/vdso64-offsets.h> | 8 | #include <generated/vdso64-offsets.h> |
| 9 | #define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) | ||
| 9 | #endif | 10 | #endif |
| 10 | #if !defined(CONFIG_64BIT) || defined(CONFIG_COMPAT) | 11 | #if !defined(CONFIG_64BIT) || defined(CONFIG_COMPAT) |
| 11 | #include <generated/vdso32-offsets.h> | 12 | #include <generated/vdso32-offsets.h> |
| 12 | #endif | ||
| 13 | |||
| 14 | #define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) | ||
| 15 | #define VDSO32_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso32_offset_##name)) | 13 | #define VDSO32_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso32_offset_##name)) |
| 14 | #else | ||
| 15 | #define VDSO32_SYMBOL(tsk, name) 0UL | ||
| 16 | #endif | ||
| 16 | 17 | ||
| 17 | #endif /* __ASSEMBLER__ */ | 18 | #endif /* __ASSEMBLER__ */ |
| 18 | 19 | ||
| @@ -46,6 +46,9 @@ obj-$(CONFIG_KEXEC_FILE) += kexec_file.o · 45 unmodified lines | |||
| 46 | 46 | ||
| 47 | # vdso | 47 | # vdso |
| 48 | obj-y += vdso.o | 48 | obj-y += vdso.o |
| 49 | obj-$(CONFIG_64BIT) += vdso64/ | 49 | ifdef CONFIG_64BIT |
| 50 | obj-$(CONFIG_PA11) += vdso32/ | 50 | obj-y += vdso64/ |
| 51 | obj-$(CONFIG_COMPAT) += vdso32/ | 51 | obj-$(CONFIG_COMPAT) += vdso32/ |
| 52 | else | ||
| 53 | obj-y += vdso32/ | ||
| 54 | endif | ||
| @@ -41,9 +41,7 @@ · 40 unmodified lines | |||
| 41 | const struct dma_map_ops *hppa_dma_ops __ro_after_init; | 41 | const struct dma_map_ops *hppa_dma_ops __ro_after_init; |
| 42 | EXPORT_SYMBOL(hppa_dma_ops); | 42 | EXPORT_SYMBOL(hppa_dma_ops); |
| 43 | 43 | ||
| 44 | static struct device root = { | 44 | static struct device *root; |
| 45 | .init_name = "parisc", | ||
| 46 | }; | ||
| 47 | 45 | ||
| 48 | static inline int check_dev(struct device *dev) | 46 | static inline int check_dev(struct device *dev) |
| 49 | { | 47 | { |
| @@ -89,7 +87,7 @@ static int for_each_padev(int (*fn)(struct device *, void *), void * data) · 39 unmodified lines | |||
| 89 | .obj = data, | 87 | .obj = data, |
| 90 | .fn = fn, | 88 | .fn = fn, |
| 91 | }; | 89 | }; |
| 92 | return device_for_each_child(&root, &recurse_data, descend_children); | 90 | return device_for_each_child(root, &recurse_data, descend_children); |
| 93 | } | 91 | } |
| 94 | 92 | ||
| 95 | /** | 93 | /** |
| @@ -290,7 +288,7 @@ const struct parisc_device * · 194 unmodified lines | |||
| 290 | find_pa_parent_type(const struct parisc_device *padev, int type) | 288 | find_pa_parent_type(const struct parisc_device *padev, int type) |
| 291 | { | 289 | { |
| 292 | const struct device *dev = &padev->dev; | 290 | const struct device *dev = &padev->dev; |
| 293 | while (dev != &root) { | 291 | while (dev != root) { |
| 294 | struct parisc_device *candidate = to_parisc_device(dev); | 292 | struct parisc_device *candidate = to_parisc_device(dev); |
| 295 | if (candidate->id.hw_type == type) | 293 | if (candidate->id.hw_type == type) |
| 296 | return candidate; | 294 | return candidate; |
| @@ -319,7 +317,7 @@ static void get_node_path(struct device *dev, struct hardware_path *path) · 22 unmodified lines | |||
| 319 | dev = dev->parent; | 317 | dev = dev->parent; |
| 320 | } | 318 | } |
| 321 | 319 | ||
| 322 | while (dev != &root) { | 320 | while (dev != root) { |
| 323 | if (dev_is_pci(dev)) { | 321 | if (dev_is_pci(dev)) { |
| 324 | unsigned int devfn = to_pci_dev(dev)->devfn; | 322 | unsigned int devfn = to_pci_dev(dev)->devfn; |
| 325 | path->bc[i--] = PCI_SLOT(devfn) | (PCI_FUNC(devfn)<< 5); | 323 | path->bc[i--] = PCI_SLOT(devfn) | (PCI_FUNC(devfn)<< 5); |
| @@ -482,7 +480,7 @@ static struct parisc_device * __init alloc_tree_node( · 156 unmodified lines | |||
| 482 | static struct parisc_device *create_parisc_device(struct hardware_path *modpath) | 480 | static struct parisc_device *create_parisc_device(struct hardware_path *modpath) |
| 483 | { | 481 | { |
| 484 | int i; | 482 | int i; |
| 485 | struct device *parent = &root; | 483 | struct device *parent = root; |
| 486 | for (i = 0; i < 6; i++) { | 484 | for (i = 0; i < 6; i++) { |
| 487 | if (modpath->bc[i] == -1) | 485 | if (modpath->bc[i] == -1) |
| 488 | continue; | 486 | continue; |
| @@ -755,7 +753,7 @@ parse_tree_node(struct device *parent, int index, struct hardware_path *modpath) · 266 unmodified lines | |||
| 755 | struct device *hwpath_to_device(struct hardware_path *modpath) | 753 | struct device *hwpath_to_device(struct hardware_path *modpath) |
| 756 | { | 754 | { |
| 757 | int i; | 755 | int i; |
| 758 | struct device *parent = &root; | 756 | struct device *parent = root; |
| 759 | for (i = 0; i < 6; i++) { | 757 | for (i = 0; i < 6; i++) { |
| 760 | if (modpath->bc[i] == -1) | 758 | if (modpath->bc[i] == -1) |
| 761 | continue; | 759 | continue; |
| @@ -880,7 +878,7 @@ void __init walk_central_bus(void) · 118 unmodified lines | |||
| 880 | { | 878 | { |
| 881 | walk_native_bus(CENTRAL_BUS_ADDR, | 879 | walk_native_bus(CENTRAL_BUS_ADDR, |
| 882 | CENTRAL_BUS_ADDR + (MAX_NATIVE_DEVICES * NATIVE_DEVICE_OFFSET), | 880 | CENTRAL_BUS_ADDR + (MAX_NATIVE_DEVICES * NATIVE_DEVICE_OFFSET), |
| 883 | &root); | 881 | root); |
| 884 | } | 882 | } |
| 885 | 883 | ||
| 886 | static __init void print_parisc_device(struct parisc_device *dev) | 884 | static __init void print_parisc_device(struct parisc_device *dev) |
| @@ -907,9 +905,10 @@ void __init init_parisc_bus(void) · 20 unmodified lines | |||
| 907 | { | 905 | { |
| 908 | if (bus_register(&parisc_bus_type)) | 906 | if (bus_register(&parisc_bus_type)) |
| 909 | panic("Could not register PA-RISC bus type\n"); | 907 | panic("Could not register PA-RISC bus type\n"); |
| 910 | if (device_register(&root)) | 908 | |
| 909 | root = root_device_register("parisc"); | ||
| 910 | if (IS_ERR(root)) | ||
| 911 | panic("Could not register PA-RISC root device\n"); | 911 | panic("Could not register PA-RISC root device\n"); |
| 912 | get_device(&root); | ||
| 913 | } | 912 | } |
| 914 | 913 | ||
| 915 | static __init void qemu_header(void) | 914 | static __init void qemu_header(void) |
| @@ -83,11 +83,10 @@ config MSI_BITMAP_SELFTEST · 82 unmodified lines | |||
| 83 | depends on DEBUG_KERNEL | 83 | depends on DEBUG_KERNEL |
| 84 | 84 | ||
| 85 | config GUEST_STATE_BUFFER_TEST | 85 | config GUEST_STATE_BUFFER_TEST |
| 86 | def_tristate n | 86 | def_tristate KUNIT_ALL_TESTS |
| 87 | prompt "Enable Guest State Buffer unit tests" | 87 | prompt "Enable Guest State Buffer unit tests" |
| 88 | depends on KUNIT | 88 | depends on KUNIT |
| 89 | depends on KVM_BOOK3S_HV_POSSIBLE | 89 | depends on KVM_BOOK3S_HV_POSSIBLE |
| 90 | default KUNIT_ALL_TESTS | ||
| 91 | help | 90 | help |
| 92 | The Guest State Buffer is a data format specified in the PAPR. | 91 | The Guest State Buffer is a data format specified in the PAPR. |
| 93 | It is by hcalls to communicate the state of L2 guests between | 92 | It is by hcalls to communicate the state of L2 guests between |
| @@ -76,7 +76,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 75 unmodified lines | |||
| 76 | # CONFIG_HW_RANDOM is not set | 76 | # CONFIG_HW_RANDOM is not set |
| 77 | # CONFIG_HWMON is not set | 77 | # CONFIG_HWMON is not set |
| 78 | CONFIG_FB=y | 78 | CONFIG_FB=y |
| 79 | CONFIG_FIRMWARE_EDID=y | ||
| 80 | CONFIG_FB_TILEBLITTING=y | 79 | CONFIG_FB_TILEBLITTING=y |
| 81 | CONFIG_FB_RADEON=y | 80 | CONFIG_FB_RADEON=y |
| 82 | CONFIG_FB_3DFX=y | 81 | CONFIG_FB_3DFX=y |
| @@ -76,7 +76,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 75 unmodified lines | |||
| 76 | CONFIG_NVRAM=y | 76 | CONFIG_NVRAM=y |
| 77 | # CONFIG_HWMON is not set | 77 | # CONFIG_HWMON is not set |
| 78 | CONFIG_FB=y | 78 | CONFIG_FB=y |
| 79 | CONFIG_FIRMWARE_EDID=y | ||
| 80 | CONFIG_FB_OF=y | 79 | CONFIG_FB_OF=y |
| 81 | CONFIG_FB_MATROX=y | 80 | CONFIG_FB_MATROX=y |
| 82 | CONFIG_FB_MATROX_MILLENIUM=y | 81 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -85,6 +85,8 @@ CONFIG_PMAC_SMU=y · 84 unmodified lines | |||
| 85 | CONFIG_MAC_EMUMOUSEBTN=y | 85 | CONFIG_MAC_EMUMOUSEBTN=y |
| 86 | CONFIG_WINDFARM=y | 86 | CONFIG_WINDFARM=y |
| 87 | CONFIG_WINDFARM_PM81=y | 87 | CONFIG_WINDFARM_PM81=y |
| 88 | CONFIG_WINDFARM_PM72=y | ||
| 89 | CONFIG_WINDFARM_RM31=y | ||
| 88 | CONFIG_WINDFARM_PM91=y | 90 | CONFIG_WINDFARM_PM91=y |
| 89 | CONFIG_WINDFARM_PM112=y | 91 | CONFIG_WINDFARM_PM112=y |
| 90 | CONFIG_WINDFARM_PM121=y | 92 | CONFIG_WINDFARM_PM121=y |
| @@ -121,7 +123,6 @@ CONFIG_I2C_CHARDEV=y · 30 unmodified lines | |||
| 121 | CONFIG_AGP=m | 123 | CONFIG_AGP=m |
| 122 | CONFIG_AGP_UNINORTH=m | 124 | CONFIG_AGP_UNINORTH=m |
| 123 | CONFIG_FB=y | 125 | CONFIG_FB=y |
| 124 | CONFIG_FIRMWARE_EDID=y | ||
| 125 | CONFIG_FB_TILEBLITTING=y | 126 | CONFIG_FB_TILEBLITTING=y |
| 126 | CONFIG_FB_OF=y | 127 | CONFIG_FB_OF=y |
| 127 | CONFIG_FB_NVIDIA=y | 128 | CONFIG_FB_NVIDIA=y |
| @@ -98,7 +98,6 @@ CONFIG_SENSORS_LM85=y · 97 unmodified lines | |||
| 98 | CONFIG_SENSORS_LM90=y | 98 | CONFIG_SENSORS_LM90=y |
| 99 | CONFIG_DRM=y | 99 | CONFIG_DRM=y |
| 100 | CONFIG_DRM_RADEON=y | 100 | CONFIG_DRM_RADEON=y |
| 101 | CONFIG_FIRMWARE_EDID=y | ||
| 102 | CONFIG_FB_TILEBLITTING=y | 101 | CONFIG_FB_TILEBLITTING=y |
| 103 | CONFIG_FB_VGA16=y | 102 | CONFIG_FB_VGA16=y |
| 104 | CONFIG_FB_NVIDIA=y | 103 | CONFIG_FB_NVIDIA=y |
| @@ -196,7 +196,6 @@ CONFIG_I2C_CHARDEV=y · 195 unmodified lines | |||
| 196 | # CONFIG_PTP_1588_CLOCK is not set | 196 | # CONFIG_PTP_1588_CLOCK is not set |
| 197 | CONFIG_DRM=y | 197 | CONFIG_DRM=y |
| 198 | CONFIG_DRM_AST=y | 198 | CONFIG_DRM_AST=y |
| 199 | CONFIG_FIRMWARE_EDID=y | ||
| 200 | CONFIG_FB_OF=y | 199 | CONFIG_FB_OF=y |
| 201 | CONFIG_FB_MATROX=m | 200 | CONFIG_FB_MATROX=m |
| 202 | CONFIG_FB_MATROX_MILLENIUM=y | 201 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -249,7 +249,6 @@ CONFIG_I2C_CHARDEV=y · 248 unmodified lines | |||
| 249 | CONFIG_I2C_AMD8111=y | 249 | CONFIG_I2C_AMD8111=y |
| 250 | CONFIG_I2C_PASEMI=y | 250 | CONFIG_I2C_PASEMI=y |
| 251 | CONFIG_FB=y | 251 | CONFIG_FB=y |
| 252 | CONFIG_FIRMWARE_EDID=y | ||
| 253 | CONFIG_FB_OF=y | 252 | CONFIG_FB_OF=y |
| 254 | CONFIG_FB_MATROX=y | 253 | CONFIG_FB_MATROX=y |
| 255 | CONFIG_FB_MATROX_MILLENIUM=y | 254 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -118,7 +118,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 117 unmodified lines | |||
| 118 | CONFIG_I2C_CHARDEV=y | 118 | CONFIG_I2C_CHARDEV=y |
| 119 | CONFIG_I2C_AMD8111=y | 119 | CONFIG_I2C_AMD8111=y |
| 120 | CONFIG_FB=y | 120 | CONFIG_FB=y |
| 121 | CONFIG_FIRMWARE_EDID=y | ||
| 122 | CONFIG_FB_OF=y | 121 | CONFIG_FB_OF=y |
| 123 | CONFIG_FB_MATROX=y | 122 | CONFIG_FB_MATROX=y |
| 124 | CONFIG_FB_MATROX_MILLENIUM=y | 123 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -393,6 +393,7 @@ CONFIG_NETCONSOLE=m · 392 unmodified lines | |||
| 393 | CONFIG_TUN=m | 393 | CONFIG_TUN=m |
| 394 | CONFIG_VETH=m | 394 | CONFIG_VETH=m |
| 395 | CONFIG_VIRTIO_NET=m | 395 | CONFIG_VIRTIO_NET=m |
| 396 | CONFIG_EL3=m | ||
| 396 | CONFIG_VORTEX=m | 397 | CONFIG_VORTEX=m |
| 397 | CONFIG_TYPHOON=m | 398 | CONFIG_TYPHOON=m |
| 398 | CONFIG_ADAPTEC_STARFIRE=m | 399 | CONFIG_ADAPTEC_STARFIRE=m |
| @@ -214,7 +214,6 @@ CONFIG_SENSORS_IBMPOWERNV=m · 213 unmodified lines | |||
| 214 | CONFIG_DRM=m | 214 | CONFIG_DRM=m |
| 215 | CONFIG_DRM_AST=m | 215 | CONFIG_DRM_AST=m |
| 216 | CONFIG_FB=y | 216 | CONFIG_FB=y |
| 217 | CONFIG_FIRMWARE_EDID=y | ||
| 218 | # CONFIG_VGA_CONSOLE is not set | 217 | # CONFIG_VGA_CONSOLE is not set |
| 219 | CONFIG_FRAMEBUFFER_CONSOLE=y | 218 | CONFIG_FRAMEBUFFER_CONSOLE=y |
| 220 | CONFIG_LOGO=y | 219 | CONFIG_LOGO=y |
| @@ -5,4 +5,5 @@ generated-y += syscall_table_spu.h · 4 unmodified lines | |||
| 5 | generic-y += agp.h | 5 | generic-y += agp.h |
| 6 | generic-y += mcs_spinlock.h | 6 | generic-y += mcs_spinlock.h |
| 7 | generic-y += qrwlock.h | 7 | generic-y += qrwlock.h |
| 8 | generic-y += ring_buffer.h | ||
| 8 | generic-y += early_ioremap.h | 9 | generic-y += early_ioremap.h |
| @@ -79,10 +79,6 @@ extern int pmac_i2c_match_adapter(struct device_node *dev, · 78 unmodified lines | |||
| 79 | struct i2c_adapter *adapter); | 79 | struct i2c_adapter *adapter); |
| 80 | 80 | ||
| 81 | 81 | ||
| 82 | /* (legacy) Locking functions exposed to i2c-keywest */ | ||
| 83 | extern int pmac_low_i2c_lock(struct device_node *np); | ||
| 84 | extern int pmac_low_i2c_unlock(struct device_node *np); | ||
| 85 | |||
| 86 | /* Access functions for platform code */ | 82 | /* Access functions for platform code */ |
| 87 | extern int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled); | 83 | extern int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled); |
| 88 | extern void pmac_i2c_close(struct pmac_i2c_bus *bus); | 84 | extern void pmac_i2c_close(struct pmac_i2c_bus *bus); |
| @@ -458,6 +458,10 @@ DEFINE_PER_CPU(u8, irq_work_pending); · 457 unmodified lines | |||
| 458 | 458 | ||
| 459 | #endif /* 32 vs 64 bit */ | 459 | #endif /* 32 vs 64 bit */ |
| 460 | 460 | ||
| 461 | /* | ||
| 462 | * Must be called with preemption disabled since it updates | ||
| 463 | * per-CPU irq_work state and programs the local CPU decrementer. | ||
| 464 | */ | ||
| 461 | void arch_irq_work_raise(void) | 465 | void arch_irq_work_raise(void) |
| 462 | { | 466 | { |
| 463 | /* | 467 | /* |
| @@ -471,10 +475,8 @@ void arch_irq_work_raise(void) · 7 unmodified lines | |||
| 471 | * which could get tangled up if we're messing with the same state | 475 | * which could get tangled up if we're messing with the same state |
| 472 | * here. | 476 | * here. |
| 473 | */ | 477 | */ |
| 474 | preempt_disable(); | ||
| 475 | set_irq_work_pending_flag(); | 478 | set_irq_work_pending_flag(); |
| 476 | set_dec(1); | 479 | set_dec(1); |
| 477 | preempt_enable(); | ||
| 478 | } | 480 | } |
| 479 | 481 | ||
| 480 | static void set_dec_or_work(u64 val) | 482 | static void set_dec_or_work(u64 val) |
| @@ -62,6 +62,12 @@ CC32FLAGSREMOVE += -fno-stack-clash-protection · 61 unmodified lines | |||
| 62 | # 32-bit one. clang validates the values passed to these arguments during | 62 | # 32-bit one. clang validates the values passed to these arguments during |
| 63 | # parsing, even when -fno-stack-protector is passed afterwards. | 63 | # parsing, even when -fno-stack-protector is passed afterwards. |
| 64 | CC32FLAGSREMOVE += -mstack-protector-guard% | 64 | CC32FLAGSREMOVE += -mstack-protector-guard% |
| 65 | # ftrace is disabled for the vdso but arch/powerpc/Makefile adds this define to | ||
| 66 | # KBUILD_CPPFLAGS, which enables use of the 'patchable_function_entry' | ||
| 67 | # attribute in the 'inline' define via 'notrace'. This attribute is not | ||
| 68 | # supported for the powerpcle target, resulting in many instances of | ||
| 69 | # -Wunknown-attributes. | ||
| 70 | CC32FLAGSREMOVE += -DCC_USING_PATCHABLE_FUNCTION_ENTRY | ||
| 65 | endif | 71 | endif |
| 66 | LD32FLAGS := -Wl,-soname=linux-vdso32.so.1 | 72 | LD32FLAGS := -Wl,-soname=linux-vdso32.so.1 |
| 67 | AS32FLAGS := -D__VDSO32__ | 73 | AS32FLAGS := -D__VDSO32__ |
| @@ -16,4 +16,4 @@ GCOV_PROFILE_core_$(BITS).o := n · 15 unmodified lines | |||
| 16 | KCOV_INSTRUMENT_core_$(BITS).o := n | 16 | KCOV_INSTRUMENT_core_$(BITS).o := n |
| 17 | UBSAN_SANITIZE_core_$(BITS).o := n | 17 | UBSAN_SANITIZE_core_$(BITS).o := n |
| 18 | KASAN_SANITIZE_core.o := n | 18 | KASAN_SANITIZE_core.o := n |
| 19 | KASAN_SANITIZE_core_$(BITS) := n | 19 | KASAN_SANITIZE_core_$(BITS).o := n |
| @@ -52,7 +52,14 @@ int exit_vmx_usercopy(void) · 51 unmodified lines | |||
| 52 | } | 52 | } |
| 53 | EXPORT_SYMBOL(exit_vmx_usercopy); | 53 | EXPORT_SYMBOL(exit_vmx_usercopy); |
| 54 | 54 | ||
| 55 | int enter_vmx_ops(void) | 55 | /* |
| 56 | * Can be called from kexec copy_page() path with MMU off. The kexec | ||
| 57 | * code sets preempt_count to HARDIRQ_OFFSET so we return early here. | ||
| 58 | * Since in_interrupt() is always inline, __no_sanitize_address on this | ||
| 59 | * function is sufficient to avoid KASAN shadow memory accesses in real | ||
| 60 | * mode. | ||
| 61 | */ | ||
| 62 | int __no_sanitize_address enter_vmx_ops(void) | ||
| 56 | { | 63 | { |
| 57 | if (in_interrupt()) | 64 | if (in_interrupt()) |
| 58 | return 0; | 65 | return 0; |
| @@ -2242,6 +2242,7 @@ static void record_and_restart(struct perf_event *event, unsigned long val, · 2241 unmodified lines | |||
| 2242 | const u64 last_period = event->hw.last_period; | 2242 | const u64 last_period = event->hw.last_period; |
| 2243 | s64 prev, delta, left; | 2243 | s64 prev, delta, left; |
| 2244 | int record = 0; | 2244 | int record = 0; |
| 2245 | int mark_event = regs->dsisr & MMCRA_SAMPLE_ENABLE; | ||
| 2245 | 2246 | ||
| 2246 | if (event->hw.state & PERF_HES_STOPPED) { | 2247 | if (event->hw.state & PERF_HES_STOPPED) { |
| 2247 | write_pmc(event->hw.idx, 0); | 2248 | write_pmc(event->hw.idx, 0); |
| @@ -2304,9 +2305,9 @@ static void record_and_restart(struct perf_event *event, unsigned long val, · 56 unmodified lines | |||
| 2304 | * In ISA v3.0 and before values "0" and "7" are considered reserved. | 2305 | * In ISA v3.0 and before values "0" and "7" are considered reserved. |
| 2305 | * In ISA v3.1, value "7" has been used to indicate "larx/stcx". | 2306 | * In ISA v3.1, value "7" has been used to indicate "larx/stcx". |
| 2306 | * Drop the sample if "type" has reserved values for this field with a | 2307 | * Drop the sample if "type" has reserved values for this field with a |
| 2307 | * ISA version check. | 2308 | * ISA version check for marked events. |
| 2308 | */ | 2309 | */ |
| 2309 | if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC && | 2310 | if (mark_event && event->attr.sample_type & PERF_SAMPLE_DATA_SRC && |
| 2310 | ppmu->get_mem_data_src) { | 2311 | ppmu->get_mem_data_src) { |
| 2311 | val = (regs->dar & SIER_TYPE_MASK) >> SIER_TYPE_SHIFT; | 2312 | val = (regs->dar & SIER_TYPE_MASK) >> SIER_TYPE_SHIFT; |
| 2312 | if (val == 0 || (val == 7 && !cpu_has_feature(CPU_FTR_ARCH_31))) { | 2313 | if (val == 0 || (val == 7 && !cpu_has_feature(CPU_FTR_ARCH_31))) { |
| @@ -210,7 +210,7 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att · 209 unmodified lines | |||
| 210 | 0, 0, buf, &n, arg); | 210 | 0, 0, buf, &n, arg); |
| 211 | 211 | ||
| 212 | if (!ret) | 212 | if (!ret) |
| 213 | return n; | 213 | goto out_success; |
| 214 | 214 | ||
| 215 | if (ret != H_PARAMETER) | 215 | if (ret != H_PARAMETER) |
| 216 | goto out; | 216 | goto out; |
| @@ -244,12 +244,14 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att · 27 unmodified lines | |||
| 244 | starting_index, 0, buf, &n, arg); | 244 | starting_index, 0, buf, &n, arg); |
| 245 | 245 | ||
| 246 | if (!ret) | 246 | if (!ret) |
| 247 | return n; | 247 | goto out_success; |
| 248 | 248 | ||
| 249 | if (ret != H_PARAMETER) | 249 | if (ret != H_PARAMETER) |
| 250 | goto out; | 250 | goto out; |
| 251 | } | 251 | } |
| 252 | 252 | ||
| 253 | out_success: | ||
| 254 | put_cpu_var(hv_gpci_reqb); | ||
| 253 | return n; | 255 | return n; |
| 254 | 256 | ||
| 255 | out: | 257 | out: |
| @@ -278,7 +280,7 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute · 22 unmodified lines | |||
| 278 | 0, 0, buf, &n, arg); | 280 | 0, 0, buf, &n, arg); |
| 279 | 281 | ||
| 280 | if (!ret) | 282 | if (!ret) |
| 281 | return n; | 283 | goto out_success; |
| 282 | 284 | ||
| 283 | if (ret != H_PARAMETER) | 285 | if (ret != H_PARAMETER) |
| 284 | goto out; | 286 | goto out; |
| @@ -312,12 +314,14 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute · 27 unmodified lines | |||
| 312 | starting_index, 0, buf, &n, arg); | 314 | starting_index, 0, buf, &n, arg); |
| 313 | 315 | ||
| 314 | if (!ret) | 316 | if (!ret) |
| 315 | return n; | 317 | goto out_success; |
| 316 | 318 | ||
| 317 | if (ret != H_PARAMETER) | 319 | if (ret != H_PARAMETER) |
| 318 | goto out; | 320 | goto out; |
| 319 | } | 321 | } |
| 320 | 322 | ||
| 323 | out_success: | ||
| 324 | put_cpu_var(hv_gpci_reqb); | ||
| 321 | return n; | 325 | return n; |
| 322 | 326 | ||
| 323 | out: | 327 | out: |
| @@ -346,7 +350,7 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev, · 22 unmodified lines | |||
| 346 | 0, 0, buf, &n, arg); | 350 | 0, 0, buf, &n, arg); |
| 347 | 351 | ||
| 348 | if (!ret) | 352 | if (!ret) |
| 349 | return n; | 353 | goto out_success; |
| 350 | 354 | ||
| 351 | if (ret != H_PARAMETER) | 355 | if (ret != H_PARAMETER) |
| 352 | goto out; | 356 | goto out; |
| @@ -382,12 +386,14 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev, · 29 unmodified lines | |||
| 382 | starting_index, secondary_index, buf, &n, arg); | 386 | starting_index, secondary_index, buf, &n, arg); |
| 383 | 387 | ||
| 384 | if (!ret) | 388 | if (!ret) |
| 385 | return n; | 389 | goto out_success; |
| 386 | 390 | ||
| 387 | if (ret != H_PARAMETER) | 391 | if (ret != H_PARAMETER) |
| 388 | goto out; | 392 | goto out; |
| 389 | } | 393 | } |
| 390 | 394 | ||
| 395 | out_success: | ||
| 396 | put_cpu_var(hv_gpci_reqb); | ||
| 391 | return n; | 397 | return n; |
| 392 | 398 | ||
| 393 | out: | 399 | out: |
| @@ -416,7 +422,7 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device · 22 unmodified lines | |||
| 416 | 0, 0, buf, &n, arg); | 422 | 0, 0, buf, &n, arg); |
| 417 | 423 | ||
| 418 | if (!ret) | 424 | if (!ret) |
| 419 | return n; | 425 | goto out_success; |
| 420 | 426 | ||
| 421 | if (ret != H_PARAMETER) | 427 | if (ret != H_PARAMETER) |
| 422 | goto out; | 428 | goto out; |
| @@ -448,12 +454,14 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device · 25 unmodified lines | |||
| 448 | starting_index, 0, buf, &n, arg); | 454 | starting_index, 0, buf, &n, arg); |
| 449 | 455 | ||
| 450 | if (!ret) | 456 | if (!ret) |
| 451 | return n; | 457 | goto out_success; |
| 452 | 458 | ||
| 453 | if (ret != H_PARAMETER) | 459 | if (ret != H_PARAMETER) |
| 454 | goto out; | 460 | goto out; |
| 455 | } | 461 | } |
| 456 | 462 | ||
| 463 | out_success: | ||
| 464 | put_cpu_var(hv_gpci_reqb); | ||
| 457 | return n; | 465 | return n; |
| 458 | 466 | ||
| 459 | out: | 467 | out: |
| @@ -293,6 +293,8 @@ static int pika_dtm_thread(void __iomem *fpga) · 292 unmodified lines | |||
| 293 | schedule_timeout(HZ); | 293 | schedule_timeout(HZ); |
| 294 | } | 294 | } |
| 295 | 295 | ||
| 296 | put_device(&client->dev); | ||
| 297 | |||
| 296 | return 0; | 298 | return 0; |
| 297 | } | 299 | } |
| 298 | 300 | ||
| @@ -27,8 +27,8 @@ · 26 unmodified lines | |||
| 27 | 27 | ||
| 28 | static void __init km82xx_pic_init(void) | 28 | static void __init km82xx_pic_init(void) |
| 29 | { | 29 | { |
| 30 | struct device_node *np __free(device_node); | 30 | struct device_node *np __free(device_node) = of_find_compatible_node(NULL, |
| 31 | np = of_find_compatible_node(NULL, NULL, "fsl,pq2-pic"); | 31 | NULL, "fsl,pq2-pic"); |
| 32 | 32 | ||
| 33 | if (!np) { | 33 | if (!np) { |
| 34 | pr_err("PIC init: can not find cpm-pic node\n"); | 34 | pr_err("PIC init: can not find cpm-pic node\n"); |
| @@ -477,7 +477,7 @@ int cpm1_gpiochip_add16(struct device *dev) · 476 unmodified lines | |||
| 477 | struct device_node *np = dev->of_node; | 477 | struct device_node *np = dev->of_node; |
| 478 | struct cpm1_gpio16_chip *cpm1_gc; | 478 | struct cpm1_gpio16_chip *cpm1_gc; |
| 479 | struct gpio_chip *gc; | 479 | struct gpio_chip *gc; |
| 480 | u16 mask; | 480 | u32 mask; |
| 481 | 481 | ||
| 482 | cpm1_gc = devm_kzalloc(dev, sizeof(*cpm1_gc), GFP_KERNEL); | 482 | cpm1_gc = devm_kzalloc(dev, sizeof(*cpm1_gc), GFP_KERNEL); |
| 483 | if (!cpm1_gc) | 483 | if (!cpm1_gc) |
| @@ -485,7 +485,7 @@ int cpm1_gpiochip_add16(struct device *dev) · 1 unmodified line | |||
| 485 | 485 | ||
| 486 | spin_lock_init(&cpm1_gc->lock); | 486 | spin_lock_init(&cpm1_gc->lock); |
| 487 | 487 | ||
| 488 | if (!of_property_read_u16(np, "fsl,cpm1-gpio-irq-mask", &mask)) { | 488 | if (!of_property_read_u32(np, "fsl,cpm1-gpio-irq-mask", &mask)) { |
| 489 | int i, j; | 489 | int i, j; |
| 490 | 490 | ||
| 491 | for (i = 0, j = 0; i < 16; i++) | 491 | for (i = 0, j = 0; i < 16; i++) |
| @@ -272,13 +272,12 @@ void __init pas_pci_init(void) · 271 unmodified lines | |||
| 272 | { | 272 | { |
| 273 | struct device_node *root = of_find_node_by_path("/"); | 273 | struct device_node *root = of_find_node_by_path("/"); |
| 274 | struct device_node *np; | 274 | struct device_node *np; |
| 275 | int res; | ||
| 276 | 275 | ||
| 277 | pci_set_flags(PCI_SCAN_ALL_PCIE_DEVS); | 276 | pci_set_flags(PCI_SCAN_ALL_PCIE_DEVS); |
| 278 | 277 | ||
| 279 | np = of_find_compatible_node(root, NULL, "pasemi,rootbus"); | 278 | np = of_find_compatible_node(root, NULL, "pasemi,rootbus"); |
| 280 | if (np) { | 279 | if (np) { |
| 281 | res = pas_add_bridge(np); | 280 | pas_add_bridge(np); |
| 282 | of_node_put(np); | 281 | of_node_put(np); |
| 283 | } | 282 | } |
| 284 | of_node_put(root); | 283 | of_node_put(root); |
| @@ -1058,40 +1058,6 @@ int pmac_i2c_match_adapter(struct device_node *dev, struct i2c_adapter *adapter) · 1057 unmodified lines | |||
| 1058 | } | 1058 | } |
| 1059 | EXPORT_SYMBOL_GPL(pmac_i2c_match_adapter); | 1059 | EXPORT_SYMBOL_GPL(pmac_i2c_match_adapter); |
| 1060 | 1060 | ||
| 1061 | int pmac_low_i2c_lock(struct device_node *np) | ||
| 1062 | { | ||
| 1063 | struct pmac_i2c_bus *bus, *found = NULL; | ||
| 1064 | |||
| 1065 | list_for_each_entry(bus, &pmac_i2c_busses, link) { | ||
| 1066 | if (np == bus->controller) { | ||
| 1067 | found = bus; | ||
| 1068 | break; | ||
| 1069 | } | ||
| 1070 | } | ||
| 1071 | if (!found) | ||
| 1072 | return -ENODEV; | ||
| 1073 | return pmac_i2c_open(bus, 0); | ||
| 1074 | } | ||
| 1075 | EXPORT_SYMBOL_GPL(pmac_low_i2c_lock); | ||
| 1076 | |||
| 1077 | int pmac_low_i2c_unlock(struct device_node *np) | ||
| 1078 | { | ||
| 1079 | struct pmac_i2c_bus *bus, *found = NULL; | ||
| 1080 | |||
| 1081 | list_for_each_entry(bus, &pmac_i2c_busses, link) { | ||
| 1082 | if (np == bus->controller) { | ||
| 1083 | found = bus; | ||
| 1084 | break; | ||
| 1085 | } | ||
| 1086 | } | ||
| 1087 | if (!found) | ||
| 1088 | return -ENODEV; | ||
| 1089 | pmac_i2c_close(bus); | ||
| 1090 | return 0; | ||
| 1091 | } | ||
| 1092 | EXPORT_SYMBOL_GPL(pmac_low_i2c_unlock); | ||
| 1093 | |||
| 1094 | |||
| 1095 | int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled) | 1061 | int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled) |
| 1096 | { | 1062 | { |
| 1097 | int rc; | 1063 | int rc; |
| @@ -950,8 +950,6 @@ static int __init ps3_start_probe_thread(enum ps3_bus_type bus_type) · 949 unmodified lines | |||
| 950 | 950 | ||
| 951 | static int __init ps3_register_devices(void) | 951 | static int __init ps3_register_devices(void) |
| 952 | { | 952 | { |
| 953 | int result; | ||
| 954 | |||
| 955 | if (!firmware_has_feature(FW_FEATURE_PS3_LV1)) | 953 | if (!firmware_has_feature(FW_FEATURE_PS3_LV1)) |
| 956 | return -ENODEV; | 954 | return -ENODEV; |
| 957 | 955 | ||
| @@ -959,7 +957,7 @@ static int __init ps3_register_devices(void) · 1 unmodified line | |||
| 959 | 957 | ||
| 960 | /* ps3_repository_dump_bus_info(); */ | 958 | /* ps3_repository_dump_bus_info(); */ |
| 961 | 959 | ||
| 962 | result = ps3_start_probe_thread(PS3_BUS_TYPE_STORAGE); | 960 | ps3_start_probe_thread(PS3_BUS_TYPE_STORAGE); |
| 963 | 961 | ||
| 964 | ps3_register_vuart_devices(); | 962 | ps3_register_vuart_devices(); |
| 965 | 963 | ||
| @@ -16,6 +16,7 @@ static void *htm_buf; · 15 unmodified lines | |||
| 16 | static void *htm_status_buf; | 16 | static void *htm_status_buf; |
| 17 | static void *htm_info_buf; | 17 | static void *htm_info_buf; |
| 18 | static void *htm_caps_buf; | 18 | static void *htm_caps_buf; |
| 19 | static void *htm_mem_buf; | ||
| 19 | static u32 nodeindex; | 20 | static u32 nodeindex; |
| 20 | static u32 nodalchipindex; | 21 | static u32 nodalchipindex; |
| 21 | static u32 coreindexonchip; | 22 | static u32 coreindexonchip; |
| @@ -86,7 +87,7 @@ static ssize_t htm_return_check(long rc) · 64 unmodified lines | |||
| 86 | static ssize_t htmdump_read(struct file *filp, char __user *ubuf, | 87 | static ssize_t htmdump_read(struct file *filp, char __user *ubuf, |
| 87 | size_t count, loff_t *ppos) | 88 | size_t count, loff_t *ppos) |
| 88 | { | 89 | { |
| 89 | void *htm_buf = filp->private_data; | 90 | void *htm_buf_data = filp->private_data; |
| 90 | unsigned long page, read_size, available; | 91 | unsigned long page, read_size, available; |
| 91 | loff_t offset; | 92 | loff_t offset; |
| 92 | long rc, ret; | 93 | long rc, ret; |
| @@ -100,7 +101,7 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf, · 7 unmodified lines | |||
| 100 | * - last three values are address, size and offset | 101 | * - last three values are address, size and offset |
| 101 | */ | 102 | */ |
| 102 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | 103 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 103 | htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf), | 104 | htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf_data), |
| 104 | PAGE_SIZE, page); | 105 | PAGE_SIZE, page); |
| 105 | 106 | ||
| 106 | ret = htm_return_check(rc); | 107 | ret = htm_return_check(rc); |
| @@ -112,7 +113,61 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf, · 5 unmodified lines | |||
| 112 | available = PAGE_SIZE; | 113 | available = PAGE_SIZE; |
| 113 | read_size = min(count, available); | 114 | read_size = min(count, available); |
| 114 | *ppos += read_size; | 115 | *ppos += read_size; |
| 115 | return simple_read_from_buffer(ubuf, count, &offset, htm_buf, available); | 116 | return simple_read_from_buffer(ubuf, count, &offset, htm_buf_data, available); |
| 117 | } | ||
| 118 | |||
| 119 | static ssize_t htmsystem_mem_read(struct file *filp, char __user *ubuf, | ||
| 120 | size_t count, loff_t *ppos) | ||
| 121 | { | ||
| 122 | void *htm_mem_data = filp->private_data; | ||
| 123 | long rc, ret; | ||
| 124 | u64 *num_entries; | ||
| 125 | u64 to_copy = 0; | ||
| 126 | loff_t offset = 0; | ||
| 127 | u64 mem_offset = 0; | ||
| 128 | |||
| 129 | /* | ||
| 130 | * Invoke H_HTM call with: | ||
| 131 | * - operation as htm status (H_HTM_OP_STATUS) | ||
| 132 | * - last three values as addr, size and offset. "offset" | ||
| 133 | * is value from output buffer header that points to next | ||
| 134 | * entry to dump. 0 is the first entry to dump. next entry | ||
| 135 | * is read from the output bufferbyte offset 0x8. | ||
| 136 | * | ||
| 137 | * When first time hcall is invoked, mem_offset should be | ||
| 138 | * zero because zero is the first entry. | ||
| 139 | * In the next hcall, offset of next entry to read from is | ||
| 140 | * picked from output buffer header itself. So don't fill | ||
| 141 | * mem_offset for first read. | ||
| 142 | * | ||
| 143 | * If there is no further data to read in next iteration, | ||
| 144 | * offset value from output buffer header will point to -1. | ||
| 145 | */ | ||
| 146 | if (*ppos) { | ||
| 147 | mem_offset = *(u64 *)(htm_mem_data + 0x8); | ||
| 148 | if (mem_offset == -1) | ||
| 149 | return 0; | ||
| 150 | } | ||
| 151 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | ||
| 152 | htmtype, H_HTM_OP_DUMP_SYSMEM_CONF, virt_to_phys(htm_mem_data), | ||
| 153 | PAGE_SIZE, be64_to_cpu(mem_offset)); | ||
| 154 | ret = htm_return_check(rc); | ||
| 155 | if (ret <= 0) { | ||
| 156 | pr_debug("H_HTM hcall returned for op: H_HTM_OP_DUMP_SYSMEM_CONF with hcall returning %ld\n", ret); | ||
| 157 | return ret; | ||
| 158 | } | ||
| 159 | |||
| 160 | /* | ||
| 161 | * HTM system mem buffer, start of buffer + 0x10 gives the | ||
| 162 | * number of HTM entries in the buffer. | ||
| 163 | * So total count to copy is: | ||
| 164 | * 32 bytes (for first 5 fields) + (number of HTM entries * entry size) | ||
| 165 | */ | ||
| 166 | num_entries = htm_mem_data + 0x10; | ||
| 167 | to_copy = 32 + (be64_to_cpu(*num_entries) * 32); | ||
| 168 | |||
| 169 | *ppos += to_copy; | ||
| 170 | return simple_read_from_buffer(ubuf, count, &offset, htm_mem_data, to_copy); | ||
| 116 | } | 171 | } |
| 117 | 172 | ||
| 118 | static const struct file_operations htmdump_fops = { | 173 | static const struct file_operations htmdump_fops = { |
| @@ -121,6 +176,12 @@ static const struct file_operations htmdump_fops = { · 2 unmodified lines | |||
| 121 | .open = simple_open, | 176 | .open = simple_open, |
| 122 | }; | 177 | }; |
| 123 | 178 | ||
| 179 | static const struct file_operations htmsystem_mem_fops = { | ||
| 180 | .llseek = NULL, | ||
| 181 | .read = htmsystem_mem_read, | ||
| 182 | .open = simple_open, | ||
| 183 | }; | ||
| 184 | |||
| 124 | static int htmconfigure_set(void *data, u64 val) | 185 | static int htmconfigure_set(void *data, u64 val) |
| 125 | { | 186 | { |
| 126 | long rc, ret; | 187 | long rc, ret; |
| @@ -226,20 +287,31 @@ static int htmstart_get(void *data, u64 *val) · 99 unmodified lines | |||
| 226 | static ssize_t htmstatus_read(struct file *filp, char __user *ubuf, | 287 | static ssize_t htmstatus_read(struct file *filp, char __user *ubuf, |
| 227 | size_t count, loff_t *ppos) | 288 | size_t count, loff_t *ppos) |
| 228 | { | 289 | { |
| 229 | void *htm_status_buf = filp->private_data; | 290 | void *htm_status_data = filp->private_data; |
| 230 | long rc, ret; | 291 | long rc, ret; |
| 231 | u64 *num_entries; | 292 | u64 *num_entries; |
| 232 | u64 to_copy; | 293 | u64 to_copy; |
| 233 | int htmstatus_flag; | 294 | int htmstatus_flag; |
| 295 | loff_t offset = 0; | ||
| 296 | u64 status_offset = 0; | ||
| 234 | 297 | ||
| 235 | /* | 298 | /* |
| 236 | * Invoke H_HTM call with: | 299 | * Invoke H_HTM call with: |
| 237 | * - operation as htm status (H_HTM_OP_STATUS) | 300 | * - operation as htm status (H_HTM_OP_STATUS) |
| 238 | * - last three values as addr, size and offset | 301 | * - last three values as addr, size and offset. |
| 302 | * "offset" is value from output buffer header | ||
| 303 | * that points to next entry to dump. 0 is the first | ||
| 304 | * entry to dump. next entry is read from the output | ||
| 305 | * bufferbyte offset 0x8. | ||
| 239 | */ | 306 | */ |
| 307 | if (*ppos) { | ||
| 308 | status_offset = *(u64 *)(htm_status_data + 0x8); | ||
| 309 | if (status_offset == -1) | ||
| 310 | return 0; | ||
| 311 | } | ||
| 240 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | 312 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 241 | htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_buf), | 313 | htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_data), |
| 242 | PAGE_SIZE, 0); | 314 | PAGE_SIZE, be64_to_cpu(status_offset)); |
| 243 | 315 | ||
| 244 | ret = htm_return_check(rc); | 316 | ret = htm_return_check(rc); |
| 245 | if (ret <= 0) { | 317 | if (ret <= 0) { |
| @@ -255,13 +327,15 @@ static ssize_t htmstatus_read(struct file *filp, char __user *ubuf, · 9 unmodified lines | |||
| 255 | * So total count to copy is: | 327 | * So total count to copy is: |
| 256 | * 32 bytes (for first 7 fields) + (number of HTM entries * entry size) | 328 | * 32 bytes (for first 7 fields) + (number of HTM entries * entry size) |
| 257 | */ | 329 | */ |
| 258 | num_entries = htm_status_buf + 0x10; | 330 | num_entries = htm_status_data + 0x10; |
| 259 | if (htmtype == 0x2) | 331 | if (htmtype == 0x2) |
| 260 | htmstatus_flag = 0x8; | 332 | htmstatus_flag = 0x8; |
| 261 | else | 333 | else |
| 262 | htmstatus_flag = 0x6; | 334 | htmstatus_flag = 0x6; |
| 263 | to_copy = 32 + (be64_to_cpu(*num_entries) * htmstatus_flag); | 335 | to_copy = 32 + (be64_to_cpu(*num_entries) * htmstatus_flag); |
| 264 | return simple_read_from_buffer(ubuf, count, ppos, htm_status_buf, to_copy); | 336 | *ppos += to_copy; |
| 337 | |||
| 338 | return simple_read_from_buffer(ubuf, count, &offset, htm_status_data, to_copy); | ||
| 265 | } | 339 | } |
| 266 | 340 | ||
| 267 | static const struct file_operations htmstatus_fops = { | 341 | static const struct file_operations htmstatus_fops = { |
| @@ -273,19 +347,30 @@ static const struct file_operations htmstatus_fops = { · 5 unmodified lines | |||
| 273 | static ssize_t htminfo_read(struct file *filp, char __user *ubuf, | 347 | static ssize_t htminfo_read(struct file *filp, char __user *ubuf, |
| 274 | size_t count, loff_t *ppos) | 348 | size_t count, loff_t *ppos) |
| 275 | { | 349 | { |
| 276 | void *htm_info_buf = filp->private_data; | 350 | void *htm_info_data = filp->private_data; |
| 277 | long rc, ret; | 351 | long rc, ret; |
| 278 | u64 *num_entries; | 352 | u64 *num_entries; |
| 279 | u64 to_copy; | 353 | u64 to_copy; |
| 354 | loff_t offset = 0; | ||
| 355 | u64 info_offset = 0; | ||
| 280 | 356 | ||
| 281 | /* | 357 | /* |
| 282 | * Invoke H_HTM call with: | 358 | * Invoke H_HTM call with: |
| 283 | * - operation as htm status (H_HTM_OP_STATUS) | 359 | * - operation as htm status (H_HTM_OP_STATUS) |
| 284 | * - last three values as addr, size and offset | 360 | * - last three values as addr, size and offset |
| 361 | * "offset" is value from output buffer header | ||
| 362 | * that points to next entry to dump. 0 is the first | ||
| 363 | * entry to dump. next entry is read from the output | ||
| 364 | * bufferbyte offset 0x8. | ||
| 285 | */ | 365 | */ |
| 366 | if (*ppos) { | ||
| 367 | info_offset = *(u64 *)(htm_info_data + 0x8); | ||
| 368 | if (info_offset == -1) | ||
| 369 | return 0; | ||
| 370 | } | ||
| 286 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | 371 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 287 | htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_buf), | 372 | htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_data), |
| 288 | PAGE_SIZE, 0); | 373 | PAGE_SIZE, be64_to_cpu(info_offset)); |
| 289 | 374 | ||
| 290 | ret = htm_return_check(rc); | 375 | ret = htm_return_check(rc); |
| 291 | if (ret <= 0) { | 376 | if (ret <= 0) { |
| @@ -301,15 +386,17 @@ static ssize_t htminfo_read(struct file *filp, char __user *ubuf, · 9 unmodified lines | |||
| 301 | * So total count to copy is: | 386 | * So total count to copy is: |
| 302 | * 32 bytes (for first 5 fields) + (number of HTM entries * entry size) | 387 | * 32 bytes (for first 5 fields) + (number of HTM entries * entry size) |
| 303 | */ | 388 | */ |
| 304 | num_entries = htm_info_buf + 0x10; | 389 | num_entries = htm_info_data + 0x10; |
| 305 | to_copy = 32 + (be64_to_cpu(*num_entries) * 16); | 390 | to_copy = 32 + (be64_to_cpu(*num_entries) * 16); |
| 306 | return simple_read_from_buffer(ubuf, count, ppos, htm_info_buf, to_copy); | 391 | |
| 392 | *ppos += to_copy; | ||
| 393 | return simple_read_from_buffer(ubuf, count, &offset, htm_info_data, to_copy); | ||
| 307 | } | 394 | } |
| 308 | 395 | ||
| 309 | static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, | 396 | static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, |
| 310 | size_t count, loff_t *ppos) | 397 | size_t count, loff_t *ppos) |
| 311 | { | 398 | { |
| 312 | void *htm_caps_buf = filp->private_data; | 399 | void *htm_caps_data = filp->private_data; |
| 313 | long rc, ret; | 400 | long rc, ret; |
| 314 | 401 | ||
| 315 | /* | 402 | /* |
| @@ -319,7 +406,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, · 3 unmodified lines | |||
| 319 | * and zero | 406 | * and zero |
| 320 | */ | 407 | */ |
| 321 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | 408 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 322 | htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_buf), | 409 | htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_data), |
| 323 | 0x80, 0); | 410 | 0x80, 0); |
| 324 | 411 | ||
| 325 | ret = htm_return_check(rc); | 412 | ret = htm_return_check(rc); |
| @@ -328,7 +415,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, · 2 unmodified lines | |||
| 328 | return ret; | 415 | return ret; |
| 329 | } | 416 | } |
| 330 | 417 | ||
| 331 | return simple_read_from_buffer(ubuf, count, ppos, htm_caps_buf, 0x80); | 418 | return simple_read_from_buffer(ubuf, count, ppos, htm_caps_data, 0x80); |
| 332 | } | 419 | } |
| 333 | 420 | ||
| 334 | static const struct file_operations htminfo_fops = { | 421 | static const struct file_operations htminfo_fops = { |
| @@ -457,9 +544,17 @@ static int htmdump_init_debugfs(void) · 122 unmodified lines | |||
| 457 | return -ENOMEM; | 544 | return -ENOMEM; |
| 458 | } | 545 | } |
| 459 | 546 | ||
| 547 | /* Memory to present HTM system memory configuration */ | ||
| 548 | htm_mem_buf = kmalloc(PAGE_SIZE, GFP_KERNEL); | ||
| 549 | if (!htm_mem_buf) { | ||
| 550 | pr_err("Failed to allocate htm mem buf\n"); | ||
| 551 | return -ENOMEM; | ||
| 552 | } | ||
| 553 | |||
| 460 | debugfs_create_file("htmstatus", 0400, htmdump_debugfs_dir, htm_status_buf, &htmstatus_fops); | 554 | debugfs_create_file("htmstatus", 0400, htmdump_debugfs_dir, htm_status_buf, &htmstatus_fops); |
| 461 | debugfs_create_file("htminfo", 0400, htmdump_debugfs_dir, htm_info_buf, &htminfo_fops); | 555 | debugfs_create_file("htminfo", 0400, htmdump_debugfs_dir, htm_info_buf, &htminfo_fops); |
| 462 | debugfs_create_file("htmcaps", 0400, htmdump_debugfs_dir, htm_caps_buf, &htmcaps_fops); | 556 | debugfs_create_file("htmcaps", 0400, htmdump_debugfs_dir, htm_caps_buf, &htmcaps_fops); |
| 557 | debugfs_create_file("htmsystem_mem", 0400, htmdump_debugfs_dir, htm_mem_buf, &htmsystem_mem_fops); | ||
| 463 | 558 | ||
| 464 | return 0; | 559 | return 0; |
| 465 | } | 560 | } |
| @@ -482,6 +577,10 @@ static void __exit htmdump_exit(void) · 16 unmodified lines | |||
| 482 | { | 577 | { |
| 483 | debugfs_remove_recursive(htmdump_debugfs_dir); | 578 | debugfs_remove_recursive(htmdump_debugfs_dir); |
| 484 | kfree(htm_buf); | 579 | kfree(htm_buf); |
| 580 | kfree(htm_status_buf); | ||
| 581 | kfree(htm_info_buf); | ||
| 582 | kfree(htm_caps_buf); | ||
| 583 | kfree(htm_mem_buf); | ||
| 485 | } | 584 | } |
| 486 | 585 | ||
| 487 | module_init(htmdump_init); | 586 | module_init(htmdump_init); |
| @@ -190,33 +190,34 @@ static int hvpipe_rtas_recv_msg(char __user *buf, int size) · 189 unmodified lines | |||
| 190 | return -ENOMEM; | 190 | return -ENOMEM; |
| 191 | } | 191 | } |
| 192 | 192 | ||
| 193 | ret = rtas_ibm_receive_hvpipe_msg(work_area, &srcID, | 193 | /* |
| 194 | &bytes_written); | 194 | * Recv HVPIPE RTAS is successful. |
| 195 | if (!ret) { | 195 | * When releasing FD or no one is waiting on the |
| 196 | /* | 196 | * specific source, issue recv HVPIPE RTAS call |
| 197 | * Recv HVPIPE RTAS is successful. | 197 | * so that pipe is not blocked - this func is called |
| 198 | * When releasing FD or no one is waiting on the | 198 | * with NULL buf. |
| 199 | * specific source, issue recv HVPIPE RTAS call | 199 | */ |
| 200 | * so that pipe is not blocked - this func is called | 200 | ret = rtas_ibm_receive_hvpipe_msg(work_area, &srcID, &bytes_written); |
| 201 | * with NULL buf. | 201 | if (ret) { |
| 202 | */ | 202 | pr_err("ibm,receive-hvpipe-msg failed with %d\n", ret); |
| 203 | if (buf) { | 203 | goto out; |
| 204 | if (size < bytes_written) { | ||
| 205 | pr_err("Received the payload size = %d, but the buffer size = %d\n", | ||
| 206 | bytes_written, size); | ||
| 207 | bytes_written = size; | ||
| 208 | } | ||
| 209 | ret = copy_to_user(buf, | ||
| 210 | rtas_work_area_raw_buf(work_area), | ||
| 211 | bytes_written); | ||
| 212 | if (!ret) | ||
| 213 | ret = bytes_written; | ||
| 214 | } | ||
| 215 | } else { | ||
| 216 | pr_err("ibm,receive-hvpipe-msg failed with %d\n", | ||
| 217 | ret); | ||
| 218 | } | 204 | } |
| 219 | 205 | ||
| 206 | if (!buf) | ||
| 207 | goto out; | ||
| 208 | |||
| 209 | if (size < bytes_written) { | ||
| 210 | pr_err("Received the payload size = %d, but the buffer size = %d\n", | ||
| 211 | bytes_written, size); | ||
| 212 | bytes_written = size; | ||
| 213 | } | ||
| 214 | |||
| 215 | if (copy_to_user(buf, rtas_work_area_raw_buf(work_area), bytes_written)) | ||
| 216 | ret = -EFAULT; | ||
| 217 | else | ||
| 218 | ret = bytes_written; | ||
| 219 | |||
| 220 | out: | ||
| 220 | rtas_work_area_free(work_area); | 221 | rtas_work_area_free(work_area); |
| 221 | return ret; | 222 | return ret; |
| 222 | } | 223 | } |
| @@ -327,8 +328,8 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 104 unmodified lines | |||
| 327 | { | 328 | { |
| 328 | 329 | ||
| 329 | struct hvpipe_source_info *src_info = file->private_data; | 330 | struct hvpipe_source_info *src_info = file->private_data; |
| 330 | struct papr_hvpipe_hdr hdr; | 331 | struct papr_hvpipe_hdr hdr = {}; |
| 331 | long ret; | 332 | ssize_t ret = 0; |
| 332 | 333 | ||
| 333 | /* | 334 | /* |
| 334 | * Return -ENXIO during migration | 335 | * Return -ENXIO during migration |
| @@ -376,7 +377,7 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 41 unmodified lines | |||
| 376 | 377 | ||
| 377 | ret = copy_to_user(buf, &hdr, HVPIPE_HDR_LEN); | 378 | ret = copy_to_user(buf, &hdr, HVPIPE_HDR_LEN); |
| 378 | if (ret) | 379 | if (ret) |
| 379 | return ret; | 380 | return -EFAULT; |
| 380 | 381 | ||
| 381 | /* | 382 | /* |
| 382 | * Message event has payload, so get the payload with | 383 | * Message event has payload, so get the payload with |
| @@ -385,19 +386,23 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 2 unmodified lines | |||
| 385 | if (hdr.flags & HVPIPE_MSG_AVAILABLE) { | 386 | if (hdr.flags & HVPIPE_MSG_AVAILABLE) { |
| 386 | ret = hvpipe_rtas_recv_msg(buf + HVPIPE_HDR_LEN, | 387 | ret = hvpipe_rtas_recv_msg(buf + HVPIPE_HDR_LEN, |
| 387 | size - HVPIPE_HDR_LEN); | 388 | size - HVPIPE_HDR_LEN); |
| 388 | if (ret > 0) { | 389 | /* |
| 390 | * Always clear MSG_AVAILABLE once the RTAS call has drained | ||
| 391 | * the message, regardless of whether copy_to_user succeeded. | ||
| 392 | */ | ||
| 393 | if (ret >= 0 || ret == -EFAULT) | ||
| 389 | src_info->hvpipe_status &= ~HVPIPE_MSG_AVAILABLE; | 394 | src_info->hvpipe_status &= ~HVPIPE_MSG_AVAILABLE; |
| 390 | ret += HVPIPE_HDR_LEN; | ||
| 391 | } | ||
| 392 | } else if (hdr.flags & HVPIPE_LOST_CONNECTION) { | 395 | } else if (hdr.flags & HVPIPE_LOST_CONNECTION) { |
| 393 | /* | 396 | /* |
| 394 | * Hypervisor is closing the pipe for the specific | 397 | * Hypervisor is closing the pipe for the specific |
| 395 | * source. So notify user space. | 398 | * source. So notify user space. |
| 396 | */ | 399 | */ |
| 397 | src_info->hvpipe_status &= ~HVPIPE_LOST_CONNECTION; | 400 | src_info->hvpipe_status &= ~HVPIPE_LOST_CONNECTION; |
| 398 | ret = HVPIPE_HDR_LEN; | ||
| 399 | } | 401 | } |
| 400 | 402 | ||
| 403 | if (ret >= 0) | ||
| 404 | ret += HVPIPE_HDR_LEN; | ||
| 405 | |||
| 401 | return ret; | 406 | return ret; |
| 402 | } | 407 | } |
| 403 | 408 | ||
| @@ -444,16 +449,18 @@ static int papr_hvpipe_handle_release(struct inode *inode, · 40 unmodified lines | |||
| 444 | struct file *file) | 449 | struct file *file) |
| 445 | { | 450 | { |
| 446 | struct hvpipe_source_info *src_info; | 451 | struct hvpipe_source_info *src_info; |
| 452 | unsigned long flags; | ||
| 447 | 453 | ||
| 448 | /* | 454 | /* |
| 449 | * Hold the lock, remove source from src_list, reset the | 455 | * Hold the lock, remove source from src_list, reset the |
| 450 | * hvpipe status and release the lock to prevent any race | 456 | * hvpipe status and release the lock to prevent any race |
| 451 | * with message event IRQ. | 457 | * with message event IRQ. |
| 452 | */ | 458 | */ |
| 453 | spin_lock(&hvpipe_src_list_lock); | 459 | spin_lock_irqsave(&hvpipe_src_list_lock, flags); |
| 454 | src_info = file->private_data; | 460 | src_info = file->private_data; |
| 455 | list_del(&src_info->list); | 461 | list_del(&src_info->list); |
| 456 | file->private_data = NULL; | 462 | file->private_data = NULL; |
| 463 | spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | ||
| 457 | /* | 464 | /* |
| 458 | * If the pipe for this specific source has any pending | 465 | * If the pipe for this specific source has any pending |
| 459 | * payload, issue recv HVPIPE RTAS so that pipe will not | 466 | * payload, issue recv HVPIPE RTAS so that pipe will not |
| @@ -461,10 +468,8 @@ static int papr_hvpipe_handle_release(struct inode *inode, · 1 unmodified line | |||
| 461 | */ | 468 | */ |
| 462 | if (src_info->hvpipe_status & HVPIPE_MSG_AVAILABLE) { | 469 | if (src_info->hvpipe_status & HVPIPE_MSG_AVAILABLE) { |
| 463 | src_info->hvpipe_status = 0; | 470 | src_info->hvpipe_status = 0; |
| 464 | spin_unlock(&hvpipe_src_list_lock); | ||
| 465 | hvpipe_rtas_recv_msg(NULL, 0); | 471 | hvpipe_rtas_recv_msg(NULL, 0); |
| 466 | } else | 472 | } |
| 467 | spin_unlock(&hvpipe_src_list_lock); | ||
| 468 | 473 | ||
| 469 | kfree(src_info); | 474 | kfree(src_info); |
| 470 | return 0; | 475 | return 0; |
| @@ -479,50 +484,53 @@ static const struct file_operations papr_hvpipe_handle_ops = { · 8 unmodified lines | |||
| 479 | 484 | ||
| 480 | static int papr_hvpipe_dev_create_handle(u32 srcID) | 485 | static int papr_hvpipe_dev_create_handle(u32 srcID) |
| 481 | { | 486 | { |
| 482 | struct hvpipe_source_info *src_info __free(kfree) = NULL; | 487 | struct hvpipe_source_info *src_info; |
| 483 | 488 | int fd; | |
| 484 | spin_lock(&hvpipe_src_list_lock); | 489 | unsigned long flags; |
| 485 | /* | ||
| 486 | * Do not allow more than one process communicates with | ||
| 487 | * each source. | ||
| 488 | */ | ||
| 489 | src_info = hvpipe_find_source(srcID); | ||
| 490 | if (src_info) { | ||
| 491 | spin_unlock(&hvpipe_src_list_lock); | ||
| 492 | pr_err("pid(%d) is already using the source(%d)\n", | ||
| 493 | src_info->tsk->pid, srcID); | ||
| 494 | return -EALREADY; | ||
| 495 | } | ||
| 496 | spin_unlock(&hvpipe_src_list_lock); | ||
| 497 | 490 | ||
| 498 | src_info = kzalloc_obj(*src_info, GFP_KERNEL_ACCOUNT); | 491 | src_info = kzalloc_obj(*src_info, GFP_KERNEL_ACCOUNT); |
| 499 | if (!src_info) | 492 | if (!src_info) |
| 500 | return -ENOMEM; | 493 | return -ENOMEM; |
| 501 | 494 | ||
| 502 | src_info->srcID = srcID; | 495 | src_info->srcID = srcID; |
| 503 | src_info->tsk = current; | ||
| 504 | init_waitqueue_head(&src_info->recv_wqh); | 496 | init_waitqueue_head(&src_info->recv_wqh); |
| 505 | 497 | ||
| 506 | FD_PREPARE(fdf, O_RDONLY | O_CLOEXEC, | ||
| 507 | anon_inode_getfile("[papr-hvpipe]", &papr_hvpipe_handle_ops, | ||
| 508 | (void *)src_info, O_RDWR)); | ||
| 509 | if (fdf.err) | ||
| 510 | return fdf.err; | ||
| 511 | |||
| 512 | retain_and_null_ptr(src_info); | ||
| 513 | spin_lock(&hvpipe_src_list_lock); | ||
| 514 | /* | 498 | /* |
| 515 | * If two processes are executing ioctl() for the same | 499 | * Do not allow more than one process communicates with |
| 516 | * source ID concurrently, prevent the second process to | 500 | * each source. |
| 517 | * acquire FD. | ||
| 518 | */ | 501 | */ |
| 502 | spin_lock_irqsave(&hvpipe_src_list_lock, flags); | ||
| 519 | if (hvpipe_find_source(srcID)) { | 503 | if (hvpipe_find_source(srcID)) { |
| 520 | spin_unlock(&hvpipe_src_list_lock); | 504 | spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); |
| 505 | pr_err("pid(%s:%d) could not get the source(%d)\n", | ||
| 506 | current->comm, task_pid_nr(current), srcID); | ||
| 507 | kfree(src_info); | ||
| 521 | return -EALREADY; | 508 | return -EALREADY; |
| 522 | } | 509 | } |
| 523 | list_add(&src_info->list, &hvpipe_src_list); | 510 | list_add(&src_info->list, &hvpipe_src_list); |
| 524 | spin_unlock(&hvpipe_src_list_lock); | 511 | spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); |
| 525 | return fd_publish(fdf); | 512 | |
| 513 | fd = FD_ADD(O_RDONLY | O_CLOEXEC, | ||
| 514 | anon_inode_getfile("[papr-hvpipe]", &papr_hvpipe_handle_ops, | ||
| 515 | (void *)src_info, O_RDWR)); | ||
| 516 | if (fd < 0) { | ||
| 517 | spin_lock_irqsave(&hvpipe_src_list_lock, flags); | ||
| 518 | list_del(&src_info->list); | ||
| 519 | spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | ||
| 520 | /* | ||
| 521 | * if we fail to add FD, that means no userspace program is | ||
| 522 | * polling. In that case if there is a msg pending because the | ||
| 523 | * interrupt was fired after the src_info was added to the | ||
| 524 | * global list, then let's consume it here, to unblock the | ||
| 525 | * hvpipe | ||
| 526 | */ | ||
| 527 | if (src_info->hvpipe_status & HVPIPE_MSG_AVAILABLE) | ||
| 528 | hvpipe_rtas_recv_msg(NULL, 0); | ||
| 529 | kfree(src_info); | ||
| 530 | return fd; | ||
| 531 | } | ||
| 532 | |||
| 533 | return fd; | ||
| 526 | } | 534 | } |
| 527 | 535 | ||
| 528 | /* | 536 | /* |
| @@ -685,20 +693,19 @@ static int __init enable_hvpipe_IRQ(void) · 156 unmodified lines | |||
| 685 | struct device_node *np; | 693 | struct device_node *np; |
| 686 | 694 | ||
| 687 | hvpipe_check_exception_token = rtas_function_token(RTAS_FN_CHECK_EXCEPTION); | 695 | hvpipe_check_exception_token = rtas_function_token(RTAS_FN_CHECK_EXCEPTION); |
| 688 | if (hvpipe_check_exception_token == RTAS_UNKNOWN_SERVICE) | 696 | if (hvpipe_check_exception_token == RTAS_UNKNOWN_SERVICE) |
| 689 | return -ENODEV; | 697 | return -ENODEV; |
| 690 | 698 | ||
| 691 | /* hvpipe events */ | 699 | /* hvpipe events */ |
| 692 | np = of_find_node_by_path("/event-sources/ibm,hvpipe-msg-events"); | 700 | np = of_find_node_by_path("/event-sources/ibm,hvpipe-msg-events"); |
| 693 | if (np != NULL) { | 701 | if (!np) { |
| 694 | request_event_sources_irqs(np, hvpipe_event_interrupt, | 702 | pr_err("No device node found, could not enable hvpipe event IRQ\n"); |
| 695 | "HPIPE_EVENT"); | ||
| 696 | of_node_put(np); | ||
| 697 | } else { | ||
| 698 | pr_err("Can not enable hvpipe event IRQ\n"); | ||
| 699 | return -ENODEV; | 703 | return -ENODEV; |
| 700 | } | 704 | } |
| 701 | 705 | ||
| 706 | request_event_sources_irqs(np, hvpipe_event_interrupt, "HPIPE_EVENT"); | ||
| 707 | of_node_put(np); | ||
| 708 | |||
| 702 | return 0; | 709 | return 0; |
| 703 | } | 710 | } |
| 704 | 711 | ||
| @@ -775,23 +782,29 @@ static int __init papr_hvpipe_init(void) · 70 unmodified lines | |||
| 775 | } | 782 | } |
| 776 | 783 | ||
| 777 | ret = enable_hvpipe_IRQ(); | 784 | ret = enable_hvpipe_IRQ(); |
| 778 | if (!ret) { | 785 | if (ret) |
| 779 | ret = set_hvpipe_sys_param(1); | 786 | goto out_wq; |
| 780 | if (!ret) | ||
| 781 | ret = misc_register(&papr_hvpipe_dev); | ||
| 782 | } | ||
| 783 | 787 | ||
| 784 | if (!ret) { | 788 | ret = misc_register(&papr_hvpipe_dev); |
| 785 | pr_info("hvpipe feature is enabled\n"); | 789 | if (ret) |
| 786 | hvpipe_feature = true; | 790 | goto out_wq; |
| 787 | return 0; | ||
| 788 | } | ||
| 789 | 791 | ||
| 790 | pr_err("hvpipe feature is not enabled %d\n", ret); | 792 | ret = set_hvpipe_sys_param(1); |
| 793 | if (ret) | ||
| 794 | goto out_misc; | ||
| 795 | |||
| 796 | pr_info("hvpipe feature is enabled\n"); | ||
| 797 | hvpipe_feature = true; | ||
| 798 | return 0; | ||
| 799 | |||
| 800 | out_misc: | ||
| 801 | misc_deregister(&papr_hvpipe_dev); | ||
| 802 | out_wq: | ||
| 791 | destroy_workqueue(papr_hvpipe_wq); | 803 | destroy_workqueue(papr_hvpipe_wq); |
| 792 | out: | 804 | out: |
| 793 | kfree(papr_hvpipe_work); | 805 | kfree(papr_hvpipe_work); |
| 794 | papr_hvpipe_work = NULL; | 806 | papr_hvpipe_work = NULL; |
| 807 | pr_err("hvpipe feature is not enabled %d\n", ret); | ||
| 795 | return ret; | 808 | return ret; |
| 796 | } | 809 | } |
| 797 | machine_device_initcall(pseries, papr_hvpipe_init); | 810 | machine_device_initcall(pseries, papr_hvpipe_init); |
| @@ -21,7 +21,6 @@ struct hvpipe_source_info { · 20 unmodified lines | |||
| 21 | u32 srcID; | 21 | u32 srcID; |
| 22 | u32 hvpipe_status; | 22 | u32 hvpipe_status; |
| 23 | wait_queue_head_t recv_wqh; /* wake up poll() waitq */ | 23 | wait_queue_head_t recv_wqh; /* wake up poll() waitq */ |
| 24 | struct task_struct *tsk; | ||
| 25 | }; | 24 | }; |
| 26 | 25 | ||
| 27 | /* | 26 | /* |
| @@ -937,6 +937,28 @@ config RISCV_VECTOR_MISALIGNED · 936 unmodified lines | |||
| 937 | help | 937 | help |
| 938 | Enable detecting support for vector misaligned loads and stores. | 938 | Enable detecting support for vector misaligned loads and stores. |
| 939 | 939 | ||
| 940 | config RISCV_SBI_FWFT_DELEGATE_MISALIGNED | ||
| 941 | bool "Request firmware delegation of unaligned access exceptions" | ||
| 942 | depends on RISCV_SBI | ||
| 943 | depends on NONPORTABLE | ||
| 944 | help | ||
| 945 | Use SBI FWFT to request delegation of load address misaligned and | ||
| 946 | store address misaligned exceptions, if possible, and prefer Linux | ||
| 947 | kernel emulation of these accesses to firmware emulation. | ||
| 948 | |||
| 949 | Unfortunately, Linux's emulation is still incomplete. Namely, it | ||
| 950 | currently does not handle vector instructions and KVM guest accesses. | ||
| 951 | On platforms where these accesses would have been handled by firmware, | ||
| 952 | enabling this causes unexpected kernel oopses, userspaces crashes and | ||
| 953 | KVM guest crashes. If you are sure that these are not a problem for | ||
| 954 | your platform, you can say Y here, which may improve performance. | ||
| 955 | |||
| 956 | Saying N here will not worsen emulation support for unaligned accesses | ||
| 957 | even in the case where the firmware also has incomplete support. It | ||
| 958 | simply keeps the firmware's emulation enabled. | ||
| 959 | |||
| 960 | If you don't know what to do here, say N. | ||
| 961 | |||
| 940 | choice | 962 | choice |
| 941 | prompt "Unaligned Accesses Support" | 963 | prompt "Unaligned Accesses Support" |
| 942 | default RISCV_PROBE_UNALIGNED_ACCESS | 964 | default RISCV_PROBE_UNALIGNED_ACCESS |
| @@ -101,16 +101,6 @@ · 100 unmodified lines | |||
| 101 | status = "okay"; | 101 | status = "okay"; |
| 102 | }; | 102 | }; |
| 103 | 103 | ||
| 104 | &i2c0 { | ||
| 105 | pinctrl-names = "default"; | ||
| 106 | pinctrl-0 = <&i2c0_fabric>; | ||
| 107 | }; | ||
| 108 | |||
| 109 | &i2c1 { | ||
| 110 | pinctrl-names = "default"; | ||
| 111 | pinctrl-0 = <&i2c1_mssio>; | ||
| 112 | }; | ||
| 113 | |||
| 114 | &mmuart1 { | 104 | &mmuart1 { |
| 115 | pinctrl-names = "default"; | 105 | pinctrl-names = "default"; |
| 116 | pinctrl-0 = <&uart1_fabric>; | 106 | pinctrl-0 = <&uart1_fabric>; |
| @@ -14,6 +14,16 @@ · 13 unmodified lines | |||
| 14 | "microchip,mpfs"; | 14 | "microchip,mpfs"; |
| 15 | }; | 15 | }; |
| 16 | 16 | ||
| 17 | &i2c0 { | ||
| 18 | pinctrl-names = "default"; | ||
| 19 | pinctrl-0 = <&i2c0_fabric>; | ||
| 20 | }; | ||
| 21 | |||
| 22 | &i2c1 { | ||
| 23 | pinctrl-names = "default"; | ||
| 24 | pinctrl-0 = <&i2c1_mssio>; | ||
| 25 | }; | ||
| 26 | |||
| 17 | &syscontroller { | 27 | &syscontroller { |
| 18 | microchip,bitstream-flash = <&sys_ctrl_flash>; | 28 | microchip,bitstream-flash = <&sys_ctrl_flash>; |
| 19 | }; | 29 | }; |
| @@ -11,3 +11,22 @@ · 10 unmodified lines | |||
| 11 | "microchip,mpfs-icicle-kit", | 11 | "microchip,mpfs-icicle-kit", |
| 12 | "microchip,mpfs"; | 12 | "microchip,mpfs"; |
| 13 | }; | 13 | }; |
| 14 | |||
| 15 | &i2c0 { | ||
| 16 | pinctrl-names = "default"; | ||
| 17 | pinctrl-0 = <&i2c0_fabric>; | ||
| 18 | }; | ||
| 19 | |||
| 20 | /* | ||
| 21 | * Due to silicon errata, routing via MSS IOs doesn't work on ES devices. | ||
| 22 | * Instead, i2c1, appearing on B1/C1, which are normally MSS IOs, is routed | ||
| 23 | * via the fabric and back to B1/C1 via "fabric-test" functionality. | ||
| 24 | * This is done silently by Libero, so the iomux0 setting for i2c1 has to | ||
| 25 | * be fabric IO, despite tooling etc saying that MSS IOs are used. | ||
| 26 | * | ||
| 27 | * See Section 3.3 of https://ww1.microchip.com/downloads/aemDocuments/documents/FPGA/ProductDocuments/Errata/polarfiresoc/microsemi_polarfire_soc_fpga_egineering_samples_errata_er0219_v1.pdf | ||
| 28 | */ | ||
| 29 | &i2c1 { | ||
| 30 | pinctrl-names = "default"; | ||
| 31 | pinctrl-0 = <&i2c1_fabric>; | ||
| 32 | }; | ||
| @@ -135,29 +135,6 @@ · 134 unmodified lines | |||
| 135 | clock-frequency = <49152000>; | 135 | clock-frequency = <49152000>; |
| 136 | }; | 136 | }; |
| 137 | 137 | ||
| 138 | &camss { | ||
| 139 | assigned-clocks = <&ispcrg JH7110_ISPCLK_DOM4_APB_FUNC>, | ||
| 140 | <&ispcrg JH7110_ISPCLK_MIPI_RX0_PXL>; | ||
| 141 | assigned-clock-rates = <49500000>, <198000000>; | ||
| 142 | |||
| 143 | ports { | ||
| 144 | #address-cells = <1>; | ||
| 145 | #size-cells = <0>; | ||
| 146 | |||
| 147 | port@0 { | ||
| 148 | reg = <0>; | ||
| 149 | }; | ||
| 150 | |||
| 151 | port@1 { | ||
| 152 | reg = <1>; | ||
| 153 | |||
| 154 | camss_from_csi2rx: endpoint { | ||
| 155 | remote-endpoint = <&csi2rx_to_camss>; | ||
| 156 | }; | ||
| 157 | }; | ||
| 158 | }; | ||
| 159 | }; | ||
| 160 | |||
| 161 | &csi2rx { | 138 | &csi2rx { |
| 162 | assigned-clocks = <&ispcrg JH7110_ISPCLK_VIN_SYS>; | 139 | assigned-clocks = <&ispcrg JH7110_ISPCLK_VIN_SYS>; |
| 163 | assigned-clock-rates = <297000000>; | 140 | assigned-clock-rates = <297000000>; |
| @@ -175,9 +152,7 @@ · 11 unmodified lines | |||
| 175 | port@1 { | 152 | port@1 { |
| 176 | reg = <1>; | 153 | reg = <1>; |
| 177 | 154 | ||
| 178 | csi2rx_to_camss: endpoint { | 155 | /* remote CAMSS endpoint */ |
| 179 | remote-endpoint = <&camss_from_csi2rx>; | ||
| 180 | }; | ||
| 181 | }; | 156 | }; |
| 182 | }; | 157 | }; |
| 183 | }; | 158 | }; |
| @@ -1199,34 +1199,6 @@ · 1198 unmodified lines | |||
| 1199 | #phy-cells = <0>; | 1199 | #phy-cells = <0>; |
| 1200 | }; | 1200 | }; |
| 1201 | 1201 | ||
| 1202 | camss: isp@19840000 { | ||
| 1203 | compatible = "starfive,jh7110-camss"; | ||
| 1204 | reg = <0x0 0x19840000 0x0 0x10000>, | ||
| 1205 | <0x0 0x19870000 0x0 0x30000>; | ||
| 1206 | reg-names = "syscon", "isp"; | ||
| 1207 | clocks = <&ispcrg JH7110_ISPCLK_DOM4_APB_FUNC>, | ||
| 1208 | <&ispcrg JH7110_ISPCLK_ISPV2_TOP_WRAPPER_C>, | ||
| 1209 | <&ispcrg JH7110_ISPCLK_DVP_INV>, | ||
| 1210 | <&ispcrg JH7110_ISPCLK_VIN_P_AXI_WR>, | ||
| 1211 | <&ispcrg JH7110_ISPCLK_MIPI_RX0_PXL>, | ||
| 1212 | <&syscrg JH7110_SYSCLK_ISP_TOP_CORE>, | ||
| 1213 | <&syscrg JH7110_SYSCLK_ISP_TOP_AXI>; | ||
| 1214 | clock-names = "apb_func", "wrapper_clk_c", "dvp_inv", | ||
| 1215 | "axiwr", "mipi_rx0_pxl", "ispcore_2x", | ||
| 1216 | "isp_axi"; | ||
| 1217 | resets = <&ispcrg JH7110_ISPRST_ISPV2_TOP_WRAPPER_P>, | ||
| 1218 | <&ispcrg JH7110_ISPRST_ISPV2_TOP_WRAPPER_C>, | ||
| 1219 | <&ispcrg JH7110_ISPRST_VIN_P_AXI_RD>, | ||
| 1220 | <&ispcrg JH7110_ISPRST_VIN_P_AXI_WR>, | ||
| 1221 | <&syscrg JH7110_SYSRST_ISP_TOP>, | ||
| 1222 | <&syscrg JH7110_SYSRST_ISP_TOP_AXI>; | ||
| 1223 | reset-names = "wrapper_p", "wrapper_c", "axird", | ||
| 1224 | "axiwr", "isp_top_n", "isp_top_axi"; | ||
| 1225 | power-domains = <&pwrc JH7110_PD_ISP>; | ||
| 1226 | interrupts = <92>, <87>, <90>, <88>; | ||
| 1227 | status = "disabled"; | ||
| 1228 | }; | ||
| 1229 | |||
| 1230 | voutcrg: clock-controller@295c0000 { | 1202 | voutcrg: clock-controller@295c0000 { |
| 1231 | compatible = "starfive,jh7110-voutcrg"; | 1203 | compatible = "starfive,jh7110-voutcrg"; |
| 1232 | reg = <0x0 0x295c0000 0x0 0x10000>; | 1204 | reg = <0x0 0x295c0000 0x0 0x10000>; |
| @@ -57,7 +57,7 @@ void mips_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, · 56 unmodified lines | |||
| 57 | } | 57 | } |
| 58 | 58 | ||
| 59 | tmp = (1U << alt->patch_id); | 59 | tmp = (1U << alt->patch_id); |
| 60 | if (cpu_req_errata && tmp) { | 60 | if (cpu_req_errata & tmp) { |
| 61 | mutex_lock(&text_mutex); | 61 | mutex_lock(&text_mutex); |
| 62 | patch_text_nosync(ALT_OLD_PTR(alt), ALT_ALT_PTR(alt), | 62 | patch_text_nosync(ALT_OLD_PTR(alt), ALT_ALT_PTR(alt), |
| 63 | alt->alt_len); | 63 | alt->alt_len); |
| @@ -14,5 +14,6 @@ generic-y += ticket_spinlock.h · 13 unmodified lines | |||
| 14 | generic-y += qrwlock.h | 14 | generic-y += qrwlock.h |
| 15 | generic-y += qrwlock_types.h | 15 | generic-y += qrwlock_types.h |
| 16 | generic-y += qspinlock.h | 16 | generic-y += qspinlock.h |
| 17 | generic-y += ring_buffer.h | ||
| 17 | generic-y += user.h | 18 | generic-y += user.h |
| 18 | generic-y += vmlinux.lds.h | 19 | generic-y += vmlinux.lds.h |
| @@ -32,6 +32,10 @@ asmlinkage long __riscv_sys_ni_syscall(const struct pt_regs *); · 31 unmodified lines | |||
| 32 | __diag_push(); \ | 32 | __diag_push(); \ |
| 33 | __diag_ignore(GCC, 8, "-Wattribute-alias", \ | 33 | __diag_ignore(GCC, 8, "-Wattribute-alias", \ |
| 34 | "Type aliasing is used to sanitize syscall arguments"); \ | 34 | "Type aliasing is used to sanitize syscall arguments"); \ |
| 35 | __diag_ignore(clang, 23, "-Wunknown-warning-option", \ | ||
| 36 | "Avoid breaking versions without -Wattribute-alias"); \ | ||
| 37 | __diag_ignore(clang, 23, "-Wattribute-alias", \ | ||
| 38 | "Type aliasing is used to sanitize syscall arguments"); \ | ||
| 35 | static long __se_##prefix##name(ulong, ulong, ulong, ulong, ulong, ulong, \ | 39 | static long __se_##prefix##name(ulong, ulong, ulong, ulong, ulong, ulong, \ |
| 36 | ulong) \ | 40 | ulong) \ |
| 37 | __attribute__((alias(__stringify(___se_##prefix##name)))); \ | 41 | __attribute__((alias(__stringify(___se_##prefix##name)))); \ |
| @@ -107,6 +107,8 @@ static long compat_restore_sigcontext(struct pt_regs *regs, · 106 unmodified lines | |||
| 107 | 107 | ||
| 108 | /* sc_regs is structured the same as the start of pt_regs */ | 108 | /* sc_regs is structured the same as the start of pt_regs */ |
| 109 | err = __copy_from_user(&cregs, &sc->sc_regs, sizeof(sc->sc_regs)); | 109 | err = __copy_from_user(&cregs, &sc->sc_regs, sizeof(sc->sc_regs)); |
| 110 | if (unlikely(err)) | ||
| 111 | return err; | ||
| 110 | 112 | ||
| 111 | cregs_to_regs(&cregs, regs); | 113 | cregs_to_regs(&cregs, regs); |
| 112 | 114 | ||
| @@ -1,6 +1,7 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | 1 | /* SPDX-License-Identifier: GPL-2.0 */ |
| 2 | /* Copyright (C) 2023 Rivos Inc. */ | 2 | /* Copyright (C) 2023 Rivos Inc. */ |
| 3 | 3 | ||
| 4 | #include <linux/cfi_types.h> | ||
| 4 | #include <linux/linkage.h> | 5 | #include <linux/linkage.h> |
| 5 | #include <asm/asm.h> | 6 | #include <asm/asm.h> |
| 6 | 7 | ||
| @@ -9,7 +10,7 @@ · 2 unmodified lines | |||
| 9 | /* void __riscv_copy_words_unaligned(void *, const void *, size_t) */ | 10 | /* void __riscv_copy_words_unaligned(void *, const void *, size_t) */ |
| 10 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ | 11 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ |
| 11 | /* Note: The size is truncated to a multiple of 8 * SZREG */ | 12 | /* Note: The size is truncated to a multiple of 8 * SZREG */ |
| 12 | SYM_FUNC_START(__riscv_copy_words_unaligned) | 13 | SYM_TYPED_FUNC_START(__riscv_copy_words_unaligned) |
| 13 | andi a4, a2, ~((8*SZREG)-1) | 14 | andi a4, a2, ~((8*SZREG)-1) |
| 14 | beqz a4, 2f | 15 | beqz a4, 2f |
| 15 | add a3, a1, a4 | 16 | add a3, a1, a4 |
| @@ -41,7 +42,7 @@ SYM_FUNC_END(__riscv_copy_words_unaligned) · 25 unmodified lines | |||
| 41 | /* void __riscv_copy_bytes_unaligned(void *, const void *, size_t) */ | 42 | /* void __riscv_copy_bytes_unaligned(void *, const void *, size_t) */ |
| 42 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ | 43 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ |
| 43 | /* Note: The size is truncated to a multiple of 8 */ | 44 | /* Note: The size is truncated to a multiple of 8 */ |
| 44 | SYM_FUNC_START(__riscv_copy_bytes_unaligned) | 45 | SYM_TYPED_FUNC_START(__riscv_copy_bytes_unaligned) |
| 45 | andi a4, a2, ~(8-1) | 46 | andi a4, a2, ~(8-1) |
| 46 | beqz a4, 2f | 47 | beqz a4, 2f |
| 47 | add a3, a1, a4 | 48 | add a3, a1, a4 |
| @@ -896,10 +896,8 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) · 895 unmodified lines | |||
| 896 | * CPU cores with the ratified spec will contain non-zero | 896 | * CPU cores with the ratified spec will contain non-zero |
| 897 | * marchid. | 897 | * marchid. |
| 898 | */ | 898 | */ |
| 899 | if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) { | 899 | if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) |
| 900 | this_hwcap &= ~isa2hwcap[RISCV_ISA_EXT_v]; | ||
| 901 | clear_bit(RISCV_ISA_EXT_v, source_isa); | 900 | clear_bit(RISCV_ISA_EXT_v, source_isa); |
| 902 | } | ||
| 903 | 901 | ||
| 904 | riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap); | 902 | riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap); |
| 905 | 903 | ||
| @@ -1104,16 +1102,16 @@ early_param("riscv_isa_fallback", riscv_isa_fallback_setup); · 198 unmodified lines | |||
| 1104 | void __init riscv_fill_hwcap(void) | 1102 | void __init riscv_fill_hwcap(void) |
| 1105 | { | 1103 | { |
| 1106 | char print_str[NUM_ALPHA_EXTS + 1]; | 1104 | char print_str[NUM_ALPHA_EXTS + 1]; |
| 1107 | unsigned long isa2hwcap[26] = {0}; | 1105 | unsigned long isa2hwcap[RISCV_ISA_EXT_BASE] = {0}; |
| 1108 | int i, j; | 1106 | int i, j; |
| 1109 | 1107 | ||
| 1110 | isa2hwcap['i' - 'a'] = COMPAT_HWCAP_ISA_I; | 1108 | isa2hwcap[RISCV_ISA_EXT_i] = COMPAT_HWCAP_ISA_I; |
| 1111 | isa2hwcap['m' - 'a'] = COMPAT_HWCAP_ISA_M; | 1109 | isa2hwcap[RISCV_ISA_EXT_m] = COMPAT_HWCAP_ISA_M; |
| 1112 | isa2hwcap['a' - 'a'] = COMPAT_HWCAP_ISA_A; | 1110 | isa2hwcap[RISCV_ISA_EXT_a] = COMPAT_HWCAP_ISA_A; |
| 1113 | isa2hwcap['f' - 'a'] = COMPAT_HWCAP_ISA_F; | 1111 | isa2hwcap[RISCV_ISA_EXT_f] = COMPAT_HWCAP_ISA_F; |
| 1114 | isa2hwcap['d' - 'a'] = COMPAT_HWCAP_ISA_D; | 1112 | isa2hwcap[RISCV_ISA_EXT_d] = COMPAT_HWCAP_ISA_D; |
| 1115 | isa2hwcap['c' - 'a'] = COMPAT_HWCAP_ISA_C; | 1113 | isa2hwcap[RISCV_ISA_EXT_c] = COMPAT_HWCAP_ISA_C; |
| 1116 | isa2hwcap['v' - 'a'] = COMPAT_HWCAP_ISA_V; | 1114 | isa2hwcap[RISCV_ISA_EXT_v] = COMPAT_HWCAP_ISA_V; |
| 1117 | 1115 | ||
| 1118 | if (!acpi_disabled) { | 1116 | if (!acpi_disabled) { |
| 1119 | riscv_fill_hwcap_from_isa_string(isa2hwcap); | 1117 | riscv_fill_hwcap_from_isa_string(isa2hwcap); |
| @@ -577,8 +577,8 @@ static int compat_riscv_gpr_set(struct task_struct *target, · 576 unmodified lines | |||
| 577 | struct compat_user_regs_struct cregs; | 577 | struct compat_user_regs_struct cregs; |
| 578 | 578 | ||
| 579 | ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &cregs, 0, -1); | 579 | ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &cregs, 0, -1); |
| 580 | 580 | if (!ret) | |
| 581 | cregs_to_regs(&cregs, task_pt_regs(target)); | 581 | cregs_to_regs(&cregs, task_pt_regs(target)); |
| 582 | 582 | ||
| 583 | return ret; | 583 | return ret; |
| 584 | } | 584 | } |
| @@ -584,7 +584,7 @@ static int cpu_online_check_unaligned_access_emulated(unsigned int cpu) · 583 unmodified lines | |||
| 584 | 584 | ||
| 585 | static bool misaligned_traps_delegated; | 585 | static bool misaligned_traps_delegated; |
| 586 | 586 | ||
| 587 | #ifdef CONFIG_RISCV_SBI | 587 | #if defined(CONFIG_RISCV_SBI_FWFT_DELEGATE_MISALIGNED) |
| 588 | 588 | ||
| 589 | static int cpu_online_sbi_unaligned_setup(unsigned int cpu) | 589 | static int cpu_online_sbi_unaligned_setup(unsigned int cpu) |
| 590 | { | 590 | { |
| @@ -109,15 +109,16 @@ void set_indir_lp_lock(struct task_struct *task, bool lock) · 108 unmodified lines | |||
| 109 | task->thread_info.user_cfi_state.ufcfi_locked = lock; | 109 | task->thread_info.user_cfi_state.ufcfi_locked = lock; |
| 110 | } | 110 | } |
| 111 | /* | 111 | /* |
| 112 | * If size is 0, then to be compatible with regular stack we want it to be as big as | 112 | * The shadow stack only stores the return address and not any variables |
| 113 | * regular stack. Else PAGE_ALIGN it and return back | 113 | * this should be more than sufficient for most applications. |
| 114 | * Else PAGE_ALIGN it and return back | ||
| 114 | */ | 115 | */ |
| 115 | static unsigned long calc_shstk_size(unsigned long size) | 116 | static unsigned long calc_shstk_size(unsigned long size) |
| 116 | { | 117 | { |
| 117 | if (size) | 118 | if (size) |
| 118 | return PAGE_ALIGN(size); | 119 | return PAGE_ALIGN(size); |
| 119 | 120 | ||
| 120 | return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); | 121 | return PAGE_ALIGN(min(rlimit(RLIMIT_STACK) / 2, SZ_2G)); |
| 121 | } | 122 | } |
| 122 | 123 | ||
| 123 | /* | 124 | /* |
| @@ -2,6 +2,7 @@ · 1 unmodified line | |||
| 2 | /* Copyright (C) 2024 Rivos Inc. */ | 2 | /* Copyright (C) 2024 Rivos Inc. */ |
| 3 | 3 | ||
| 4 | #include <linux/args.h> | 4 | #include <linux/args.h> |
| 5 | #include <linux/cfi_types.h> | ||
| 5 | #include <linux/linkage.h> | 6 | #include <linux/linkage.h> |
| 6 | #include <asm/asm.h> | 7 | #include <asm/asm.h> |
| 7 | 8 | ||
| @@ -16,7 +17,7 @@ · 8 unmodified lines | |||
| 16 | /* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ | 17 | /* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ |
| 17 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ | 18 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ |
| 18 | /* Note: The size is truncated to a multiple of WORD_EEW */ | 19 | /* Note: The size is truncated to a multiple of WORD_EEW */ |
| 19 | SYM_FUNC_START(__riscv_copy_vec_words_unaligned) | 20 | SYM_TYPED_FUNC_START(__riscv_copy_vec_words_unaligned) |
| 20 | andi a4, a2, ~(WORD_EEW-1) | 21 | andi a4, a2, ~(WORD_EEW-1) |
| 21 | beqz a4, 2f | 22 | beqz a4, 2f |
| 22 | add a3, a1, a4 | 23 | add a3, a1, a4 |
| @@ -38,7 +39,7 @@ SYM_FUNC_END(__riscv_copy_vec_words_unaligned) · 15 unmodified lines | |||
| 38 | /* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ | 39 | /* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ |
| 39 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ | 40 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ |
| 40 | /* Note: The size is truncated to a multiple of 8 */ | 41 | /* Note: The size is truncated to a multiple of 8 */ |
| 41 | SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) | 42 | SYM_TYPED_FUNC_START(__riscv_copy_vec_bytes_unaligned) |
| 42 | andi a4, a2, ~(8-1) | 43 | andi a4, a2, ~(8-1) |
| 43 | beqz a4, 2f | 44 | beqz a4, 2f |
| 44 | add a3, a1, a4 | 45 | add a3, a1, a4 |
| @@ -415,7 +415,6 @@ int kvm_riscv_vcpu_mmio_load(struct kvm_vcpu *vcpu, struct kvm_run *run, · 414 unmodified lines | |||
| 415 | shift = 8 * (sizeof(ulong) - len); | 415 | shift = 8 * (sizeof(ulong) - len); |
| 416 | } else if ((insn & INSN_MASK_LBU) == INSN_MATCH_LBU) { | 416 | } else if ((insn & INSN_MASK_LBU) == INSN_MATCH_LBU) { |
| 417 | len = 1; | 417 | len = 1; |
| 418 | shift = 8 * (sizeof(ulong) - len); | ||
| 419 | #ifdef CONFIG_64BIT | 418 | #ifdef CONFIG_64BIT |
| 420 | } else if ((insn & INSN_MASK_LD) == INSN_MATCH_LD) { | 419 | } else if ((insn & INSN_MASK_LD) == INSN_MATCH_LD) { |
| 421 | len = 8; | 420 | len = 8; |
| @@ -649,22 +648,22 @@ int kvm_riscv_vcpu_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · 227 unmodified lines | |||
| 649 | case 1: | 648 | case 1: |
| 650 | data8 = *((u8 *)run->mmio.data); | 649 | data8 = *((u8 *)run->mmio.data); |
| 651 | SET_RD(insn, &vcpu->arch.guest_context, | 650 | SET_RD(insn, &vcpu->arch.guest_context, |
| 652 | (ulong)data8 << shift >> shift); | 651 | (long)((ulong)data8 << shift) >> shift); |
| 653 | break; | 652 | break; |
| 654 | case 2: | 653 | case 2: |
| 655 | data16 = *((u16 *)run->mmio.data); | 654 | data16 = *((u16 *)run->mmio.data); |
| 656 | SET_RD(insn, &vcpu->arch.guest_context, | 655 | SET_RD(insn, &vcpu->arch.guest_context, |
| 657 | (ulong)data16 << shift >> shift); | 656 | (long)((ulong)data16 << shift) >> shift); |
| 658 | break; | 657 | break; |
| 659 | case 4: | 658 | case 4: |
| 660 | data32 = *((u32 *)run->mmio.data); | 659 | data32 = *((u32 *)run->mmio.data); |
| 661 | SET_RD(insn, &vcpu->arch.guest_context, | 660 | SET_RD(insn, &vcpu->arch.guest_context, |
| 662 | (ulong)data32 << shift >> shift); | 661 | (long)((ulong)data32 << shift) >> shift); |
| 663 | break; | 662 | break; |
| 664 | case 8: | 663 | case 8: |
| 665 | data64 = *((u64 *)run->mmio.data); | 664 | data64 = *((u64 *)run->mmio.data); |
| 666 | SET_RD(insn, &vcpu->arch.guest_context, | 665 | SET_RD(insn, &vcpu->arch.guest_context, |
| 667 | (ulong)data64 << shift >> shift); | 666 | (long)((ulong)data64 << shift) >> shift); |
| 668 | break; | 667 | break; |
| 669 | default: | 668 | default: |
| 670 | return -EOPNOTSUPP; | 669 | return -EOPNOTSUPP; |
| @@ -453,8 +453,10 @@ int kvm_riscv_vcpu_pmu_snapshot_set_shmem(struct kvm_vcpu *vcpu, unsigned long s · 452 unmodified lines | |||
| 453 | } | 453 | } |
| 454 | 454 | ||
| 455 | kvpmu->sdata = kzalloc(snapshot_area_size, GFP_ATOMIC); | 455 | kvpmu->sdata = kzalloc(snapshot_area_size, GFP_ATOMIC); |
| 456 | if (!kvpmu->sdata) | 456 | if (!kvpmu->sdata) { |
| 457 | return -ENOMEM; | 457 | sbiret = SBI_ERR_FAILURE; |
| 458 | goto out; | ||
| 459 | } | ||
| 458 | 460 | ||
| 459 | /* No need to check writable slot explicitly as kvm_vcpu_write_guest does it internally */ | 461 | /* No need to check writable slot explicitly as kvm_vcpu_write_guest does it internally */ |
| 460 | if (kvm_vcpu_write_guest(vcpu, saddr, kvpmu->sdata, snapshot_area_size)) { | 462 | if (kvm_vcpu_write_guest(vcpu, saddr, kvpmu->sdata, snapshot_area_size)) { |
| @@ -499,8 +501,10 @@ int kvm_riscv_vcpu_pmu_event_info(struct kvm_vcpu *vcpu, unsigned long saddr_low · 38 unmodified lines | |||
| 499 | } | 501 | } |
| 500 | 502 | ||
| 501 | einfo = kzalloc(shmem_size, GFP_KERNEL); | 503 | einfo = kzalloc(shmem_size, GFP_KERNEL); |
| 502 | if (!einfo) | 504 | if (!einfo) { |
| 503 | return -ENOMEM; | 505 | ret = SBI_ERR_FAILURE; |
| 506 | goto out; | ||
| 507 | } | ||
| 504 | 508 | ||
| 505 | ret = kvm_vcpu_read_guest(vcpu, shmem, einfo, shmem_size); | 509 | ret = kvm_vcpu_read_guest(vcpu, shmem, einfo, shmem_size); |
| 506 | if (ret) { | 510 | if (ret) { |
| @@ -46,7 +46,7 @@ void kvm_riscv_vcpu_record_steal_time(struct kvm_vcpu *vcpu) · 45 unmodified lines | |||
| 46 | gfn = shmem >> PAGE_SHIFT; | 46 | gfn = shmem >> PAGE_SHIFT; |
| 47 | hva = kvm_vcpu_gfn_to_hva(vcpu, gfn); | 47 | hva = kvm_vcpu_gfn_to_hva(vcpu, gfn); |
| 48 | 48 | ||
| 49 | if (WARN_ON(kvm_is_error_hva(hva))) { | 49 | if (kvm_is_error_hva(hva)) { |
| 50 | vcpu->arch.sta.shmem = INVALID_GPA; | 50 | vcpu->arch.sta.shmem = INVALID_GPA; |
| 51 | return; | 51 | return; |
| 52 | } | 52 | } |
| @@ -55,6 +55,8 @@ static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run, · 54 unmodified lines | |||
| 55 | 55 | ||
| 56 | for_each_set_bit(i, &hmask, BITS_PER_LONG) { | 56 | for_each_set_bit(i, &hmask, BITS_PER_LONG) { |
| 57 | rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i); | 57 | rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i); |
| 58 | if (!rvcpu) | ||
| 59 | continue; | ||
| 58 | ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT); | 60 | ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT); |
| 59 | if (ret < 0) | 61 | if (ret < 0) |
| 60 | break; | 62 | break; |
| @@ -792,6 +792,27 @@ static void __init set_mmap_rnd_bits_max(void) · 791 unmodified lines | |||
| 792 | mmap_rnd_bits_max = MMAP_VA_BITS - PAGE_SHIFT - 3; | 792 | mmap_rnd_bits_max = MMAP_VA_BITS - PAGE_SHIFT - 3; |
| 793 | } | 793 | } |
| 794 | 794 | ||
| 795 | static bool __init is_vaddr_valid(unsigned long va) | ||
| 796 | { | ||
| 797 | unsigned long up = 0; | ||
| 798 | |||
| 799 | switch (satp_mode) { | ||
| 800 | case SATP_MODE_39: | ||
| 801 | up = 1UL << 38; | ||
| 802 | break; | ||
| 803 | case SATP_MODE_48: | ||
| 804 | up = 1UL << 47; | ||
| 805 | break; | ||
| 806 | case SATP_MODE_57: | ||
| 807 | up = 1UL << 56; | ||
| 808 | break; | ||
| 809 | default: | ||
| 810 | return false; | ||
| 811 | } | ||
| 812 | |||
| 813 | return (va < up) || (va >= (ULONG_MAX - up + 1)); | ||
| 814 | } | ||
| 815 | |||
| 795 | /* | 816 | /* |
| 796 | * There is a simple way to determine if 4-level is supported by the | 817 | * There is a simple way to determine if 4-level is supported by the |
| 797 | * underlying hardware: establish 1:1 mapping in 4-level page table mode | 818 | * underlying hardware: establish 1:1 mapping in 4-level page table mode |
| @@ -833,6 +854,9 @@ static __init void set_satp_mode(uintptr_t dtb_pa) · 35 unmodified lines | |||
| 833 | set_satp_mode_pmd + PMD_SIZE, | 854 | set_satp_mode_pmd + PMD_SIZE, |
| 834 | PMD_SIZE, PAGE_KERNEL_EXEC); | 855 | PMD_SIZE, PAGE_KERNEL_EXEC); |
| 835 | retry: | 856 | retry: |
| 857 | if (!is_vaddr_valid(set_satp_mode_pmd)) | ||
| 858 | goto out; | ||
| 859 | |||
| 836 | create_pgd_mapping(early_pg_dir, | 860 | create_pgd_mapping(early_pg_dir, |
| 837 | set_satp_mode_pmd, | 861 | set_satp_mode_pmd, |
| 838 | pgtable_l5_enabled ? | 862 | pgtable_l5_enabled ? |
| @@ -855,6 +879,7 @@ retry: · 16 unmodified lines | |||
| 855 | disable_pgtable_l4(); | 879 | disable_pgtable_l4(); |
| 856 | } | 880 | } |
| 857 | 881 | ||
| 882 | out: | ||
| 858 | memset(early_pg_dir, 0, PAGE_SIZE); | 883 | memset(early_pg_dir, 0, PAGE_SIZE); |
| 859 | memset(early_p4d, 0, PAGE_SIZE); | 884 | memset(early_p4d, 0, PAGE_SIZE); |
| 860 | memset(early_pud, 0, PAGE_SIZE); | 885 | memset(early_pud, 0, PAGE_SIZE); |
| @@ -7,3 +7,4 @@ generated-y += unistd_nr.h · 6 unmodified lines | |||
| 7 | generic-y += asm-offsets.h | 7 | generic-y += asm-offsets.h |
| 8 | generic-y += mcs_spinlock.h | 8 | generic-y += mcs_spinlock.h |
| 9 | generic-y += mmzone.h | 9 | generic-y += mmzone.h |
| 10 | generic-y += ring_buffer.h | ||
| @@ -1414,6 +1414,9 @@ static inline char *debug_get_user_string(const char __user *user_buf, · 1413 unmodified lines | |||
| 1414 | { | 1414 | { |
| 1415 | char *buffer; | 1415 | char *buffer; |
| 1416 | 1416 | ||
| 1417 | if (!user_len) | ||
| 1418 | return ERR_PTR(-EINVAL); | ||
| 1419 | |||
| 1417 | buffer = memdup_user_nul(user_buf, user_len); | 1420 | buffer = memdup_user_nul(user_buf, user_len); |
| 1418 | if (IS_ERR(buffer)) | 1421 | if (IS_ERR(buffer)) |
| 1419 | return buffer; | 1422 | return buffer; |
| @@ -1584,6 +1587,11 @@ static int debug_input_flush_fn(debug_info_t *id, struct debug_view *view, · 164 unmodified lines | |||
| 1584 | char input_buf[1]; | 1587 | char input_buf[1]; |
| 1585 | int rc = user_len; | 1588 | int rc = user_len; |
| 1586 | 1589 | ||
| 1590 | if (!user_len) { | ||
| 1591 | rc = -EINVAL; | ||
| 1592 | goto out; | ||
| 1593 | } | ||
| 1594 | |||
| 1587 | if (user_len > 0x10000) | 1595 | if (user_len > 0x10000) |
| 1588 | user_len = 0x10000; | 1596 | user_len = 0x10000; |
| 1589 | if (*offset != 0) { | 1597 | if (*offset != 0) { |
| @@ -186,6 +186,13 @@ static u64 pai_getctr(unsigned long *page, int nr, unsigned long offset) · 185 unmodified lines | |||
| 186 | return page[nr]; | 186 | return page[nr]; |
| 187 | } | 187 | } |
| 188 | 188 | ||
| 189 | static void pai_setctr(unsigned long *page, int nr, unsigned long offset, u64 v) | ||
| 190 | { | ||
| 191 | if (offset) | ||
| 192 | nr += offset / sizeof(*page); | ||
| 193 | page[nr] = v; | ||
| 194 | } | ||
| 195 | |||
| 189 | /* Read the counter values. Return value from location in CMP. For base | 196 | /* Read the counter values. Return value from location in CMP. For base |
| 190 | * event xxx_ALL sum up all events. Returns counter value. | 197 | * event xxx_ALL sum up all events. Returns counter value. |
| 191 | */ | 198 | */ |
| @@ -551,6 +558,8 @@ static void paicrypt_del(struct perf_event *event, int flags) · 359 unmodified lines | |||
| 551 | /* Create raw data and save it in buffer. Calculate the delta for each | 558 | /* Create raw data and save it in buffer. Calculate the delta for each |
| 552 | * counter between this invocation and the last invocation. | 559 | * counter between this invocation and the last invocation. |
| 553 | * Returns number of bytes copied. | 560 | * Returns number of bytes copied. |
| 561 | * After reading from PAI counter page, save the read value to the old | ||
| 562 | * page to calculate PAI counter deltas. | ||
| 554 | * Saves only entries with positive counter difference of the form | 563 | * Saves only entries with positive counter difference of the form |
| 555 | * 2 bytes: Number of counter | 564 | * 2 bytes: Number of counter |
| 556 | * 8 bytes: Value of counter | 565 | * 8 bytes: Value of counter |
| @@ -562,16 +571,22 @@ static size_t pai_copy(struct pai_userdata *userdata, unsigned long *page, · 5 unmodified lines | |||
| 562 | int i, outidx = 0; | 571 | int i, outidx = 0; |
| 563 | 572 | ||
| 564 | for (i = 1; i <= pp->num_avail; i++) { | 573 | for (i = 1; i <= pp->num_avail; i++) { |
| 565 | u64 val = 0, val_old = 0; | 574 | u64 val = 0, val_old = 0, val_k = 0, val_old_k = 0; |
| 566 | 575 | ||
| 567 | if (!exclude_kernel) { | 576 | if (!exclude_kernel) { |
| 568 | val += pai_getctr(page, i, pp->kernel_offset); | 577 | val_k = pai_getctr(page, i, pp->kernel_offset); |
| 569 | val_old += pai_getctr(page_old, i, pp->kernel_offset); | 578 | val_old_k = pai_getctr(page_old, i, pp->kernel_offset); |
| 579 | if (val_k != val_old_k) | ||
| 580 | pai_setctr(page_old, i, pp->kernel_offset, val_k); | ||
| 570 | } | 581 | } |
| 571 | if (!exclude_user) { | 582 | if (!exclude_user) { |
| 572 | val += pai_getctr(page, i, 0); | 583 | val = pai_getctr(page, i, 0); |
| 573 | val_old += pai_getctr(page_old, i, 0); | 584 | val_old = pai_getctr(page_old, i, 0); |
| 585 | if (val != val_old) | ||
| 586 | pai_setctr(page_old, i, 0, val); | ||
| 574 | } | 587 | } |
| 588 | val += val_k; | ||
| 589 | val_old += val_old_k; | ||
| 575 | if (val >= val_old) | 590 | if (val >= val_old) |
| 576 | val -= val_old; | 591 | val -= val_old; |
| 577 | else | 592 | else |
| @@ -602,8 +617,6 @@ static size_t pai_copy(struct pai_userdata *userdata, unsigned long *page, · 24 unmodified lines | |||
| 602 | static int pai_push_sample(size_t rawsize, struct pai_map *cpump, | 617 | static int pai_push_sample(size_t rawsize, struct pai_map *cpump, |
| 603 | struct perf_event *event) | 618 | struct perf_event *event) |
| 604 | { | 619 | { |
| 605 | int idx = PAI_PMU_IDX(event); | ||
| 606 | struct pai_pmu *pp = &pai_pmu[idx]; | ||
| 607 | struct perf_sample_data data; | 620 | struct perf_sample_data data; |
| 608 | struct perf_raw_record raw; | 621 | struct perf_raw_record raw; |
| 609 | struct pt_regs regs; | 622 | struct pt_regs regs; |
| @@ -634,8 +647,6 @@ static int pai_push_sample(size_t rawsize, struct pai_map *cpump, · 24 unmodified lines | |||
| 634 | 647 | ||
| 635 | overflow = perf_event_overflow(event, &data, ®s); | 648 | overflow = perf_event_overflow(event, &data, ®s); |
| 636 | perf_event_update_userpage(event); | 649 | perf_event_update_userpage(event); |
| 637 | /* Save crypto counter lowcore page after reading event data. */ | ||
| 638 | memcpy((void *)PAI_SAVE_AREA(event), cpump->area, pp->area_size); | ||
| 639 | return overflow; | 650 | return overflow; |
| 640 | } | 651 | } |
| 641 | 652 | ||
| @@ -651,7 +662,7 @@ static void pai_have_sample(struct perf_event *event, struct pai_map *cpump) · 9 unmodified lines | |||
| 651 | rawsize = pai_copy(cpump->save, cpump->area, pp, | 662 | rawsize = pai_copy(cpump->save, cpump->area, pp, |
| 652 | (unsigned long *)PAI_SAVE_AREA(event), | 663 | (unsigned long *)PAI_SAVE_AREA(event), |
| 653 | event->attr.exclude_user, | 664 | event->attr.exclude_user, |
| 654 | event->attr.exclude_kernel); | 665 | !pp->kernel_offset ? true : event->attr.exclude_kernel); |
| 655 | if (rawsize) /* No incremented counters */ | 666 | if (rawsize) /* No incremented counters */ |
| 656 | pai_push_sample(rawsize, cpump, event); | 667 | pai_push_sample(rawsize, cpump, event); |
| 657 | } | 668 | } |
| @@ -192,17 +192,21 @@ static void tl_to_masks(struct sysinfo_15_1_x *info) · 191 unmodified lines | |||
| 192 | end = (union topology_entry *)((unsigned long)info + info->length); | 192 | end = (union topology_entry *)((unsigned long)info + info->length); |
| 193 | while (tle < end) { | 193 | while (tle < end) { |
| 194 | switch (tle->nl) { | 194 | switch (tle->nl) { |
| 195 | /* | ||
| 196 | * Adjust drawer_id, book_id, and socked_id so they match the | ||
| 197 | * numbering scheme of e.g. the hardware management console. | ||
| 198 | */ | ||
| 195 | case 3: | 199 | case 3: |
| 196 | drawer = drawer->next; | 200 | drawer = drawer->next; |
| 197 | drawer->id = tle->container.id; | 201 | drawer->id = tle->container.id - 1; |
| 198 | break; | 202 | break; |
| 199 | case 2: | 203 | case 2: |
| 200 | book = book->next; | 204 | book = book->next; |
| 201 | book->id = tle->container.id; | 205 | book->id = tle->container.id - 1; |
| 202 | break; | 206 | break; |
| 203 | case 1: | 207 | case 1: |
| 204 | socket = socket->next; | 208 | socket = socket->next; |
| 205 | socket->id = tle->container.id; | 209 | socket->id = tle->container.id - 1; |
| 206 | break; | 210 | break; |
| 207 | case 0: | 211 | case 0: |
| 208 | add_cpus_to_mask(&tle->cpu, drawer, book, socket); | 212 | add_cpus_to_mask(&tle->cpu, drawer, book, socket); |
| @@ -267,6 +267,7 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g · 266 unmodified lines | |||
| 267 | /* No need to take locks as the page table is not installed yet. */ | 267 | /* No need to take locks as the page table is not installed yet. */ |
| 268 | pgste_init.prefix_notif = old.s.fc1.prefix_notif; | 268 | pgste_init.prefix_notif = old.s.fc1.prefix_notif; |
| 269 | pgste_init.vsie_notif = old.s.fc1.vsie_notif; | 269 | pgste_init.vsie_notif = old.s.fc1.vsie_notif; |
| 270 | pgste_init.vsie_gmem = old.s.fc1.vsie_notif; | ||
| 270 | pgste_init.pcl = uses_skeys && init.h.i; | 271 | pgste_init.pcl = uses_skeys && init.h.i; |
| 271 | dat_init_pgstes(pt, pgste_init.val); | 272 | dat_init_pgstes(pt, pgste_init.val); |
| 272 | } else { | 273 | } else { |
| @@ -145,7 +145,8 @@ union pgste { · 144 unmodified lines | |||
| 145 | unsigned long cmma_d : 1; /* Dirty flag for CMMA bits */ | 145 | unsigned long cmma_d : 1; /* Dirty flag for CMMA bits */ |
| 146 | unsigned long prefix_notif : 1; /* Guest prefix invalidation notification */ | 146 | unsigned long prefix_notif : 1; /* Guest prefix invalidation notification */ |
| 147 | unsigned long vsie_notif : 1; /* Referenced in a shadow table */ | 147 | unsigned long vsie_notif : 1; /* Referenced in a shadow table */ |
| 148 | unsigned long : 5; | 148 | unsigned long vsie_gmem : 1; /* Contains nested guest memory */ |
| 149 | unsigned long : 4; | ||
| 149 | unsigned long : 8; | 150 | unsigned long : 8; |
| 150 | }; | 151 | }; |
| 151 | struct { | 152 | struct { |
| @@ -1445,6 +1445,7 @@ static int _do_shadow_pte(struct gmap *sg, gpa_t raddr, union pte *ptep_h, union · 1444 unmodified lines | |||
| 1445 | } else { | 1445 | } else { |
| 1446 | pgste = _gmap_ptep_xchg(sg->parent, ptep_h, newpte, pgste, f->gfn, false); | 1446 | pgste = _gmap_ptep_xchg(sg->parent, ptep_h, newpte, pgste, f->gfn, false); |
| 1447 | pgste.vsie_notif = 1; | 1447 | pgste.vsie_notif = 1; |
| 1448 | pgste.vsie_gmem = 1; | ||
| 1448 | } | 1449 | } |
| 1449 | pgste_set_unlock(ptep_h, pgste); | 1450 | pgste_set_unlock(ptep_h, pgste); |
| 1450 | if (rc) | 1451 | if (rc) |
| @@ -125,7 +125,7 @@ struct gmap *gmap_new_child(struct gmap *parent, gfn_t limit) · 124 unmodified lines | |||
| 125 | 125 | ||
| 126 | int gmap_set_limit(struct gmap *gmap, gfn_t limit) | 126 | int gmap_set_limit(struct gmap *gmap, gfn_t limit) |
| 127 | { | 127 | { |
| 128 | struct kvm_s390_mmu_cache *mc; | 128 | struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; |
| 129 | int rc, type; | 129 | int rc, type; |
| 130 | 130 | ||
| 131 | type = gmap_limit_to_type(limit); | 131 | type = gmap_limit_to_type(limit); |
| @@ -142,7 +142,6 @@ int gmap_set_limit(struct gmap *gmap, gfn_t limit) · 10 unmodified lines | |||
| 142 | rc = dat_set_asce_limit(mc, &gmap->asce, type); | 142 | rc = dat_set_asce_limit(mc, &gmap->asce, type); |
| 143 | } while (rc == -ENOMEM); | 143 | } while (rc == -ENOMEM); |
| 144 | 144 | ||
| 145 | kvm_s390_free_mmu_cache(mc); | ||
| 146 | return 0; | 145 | return 0; |
| 147 | } | 146 | } |
| 148 | 147 | ||
| @@ -822,8 +821,8 @@ int gmap_ucas_translate(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, gpa_t · 673 unmodified lines | |||
| 822 | 821 | ||
| 823 | int gmap_ucas_map(struct gmap *gmap, gfn_t p_gfn, gfn_t c_gfn, unsigned long count) | 822 | int gmap_ucas_map(struct gmap *gmap, gfn_t p_gfn, gfn_t c_gfn, unsigned long count) |
| 824 | { | 823 | { |
| 825 | struct kvm_s390_mmu_cache *mc; | 824 | struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; |
| 826 | int rc; | 825 | int rc = 0; |
| 827 | 826 | ||
| 828 | mc = kvm_s390_new_mmu_cache(); | 827 | mc = kvm_s390_new_mmu_cache(); |
| 829 | if (!mc) | 828 | if (!mc) |
| @@ -1026,13 +1025,15 @@ int gmap_insert_rmap(struct gmap *sg, gfn_t p_gfn, gfn_t r_gfn, int level) · 196 unmodified lines | |||
| 1026 | int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gfn, gfn_t r_gfn, | 1025 | int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gfn, gfn_t r_gfn, |
| 1027 | kvm_pfn_t pfn, int level, bool wr) | 1026 | kvm_pfn_t pfn, int level, bool wr) |
| 1028 | { | 1027 | { |
| 1028 | unsigned long bitmask; | ||
| 1029 | union crste *crstep; | 1029 | union crste *crstep; |
| 1030 | union pgste pgste; | 1030 | union pgste pgste; |
| 1031 | union pte *ptep; | 1031 | union pte *ptep; |
| 1032 | union pte pte; | 1032 | union pte pte; |
| 1033 | int flags, rc; | 1033 | int flags, rc; |
| 1034 | 1034 | ||
| 1035 | KVM_BUG_ON(!is_shadow(sg), sg->kvm); | 1035 | if (KVM_BUG_ON(!is_shadow(sg) || level <= TABLE_TYPE_PAGE_TABLE, sg->kvm)) |
| 1036 | return -EINVAL; | ||
| 1036 | lockdep_assert_held(&sg->parent->children_lock); | 1037 | lockdep_assert_held(&sg->parent->children_lock); |
| 1037 | 1038 | ||
| 1038 | flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0); | 1039 | flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0); |
| @@ -1041,8 +1042,9 @@ int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gf · 2 unmodified lines | |||
| 1041 | if (rc) | 1042 | if (rc) |
| 1042 | return rc; | 1043 | return rc; |
| 1043 | if (level <= TABLE_TYPE_REGION1) { | 1044 | if (level <= TABLE_TYPE_REGION1) { |
| 1045 | bitmask = -1UL << (8 + 11 * level); | ||
| 1044 | scoped_guard(spinlock, &sg->host_to_rmap_lock) | 1046 | scoped_guard(spinlock, &sg->host_to_rmap_lock) |
| 1045 | rc = gmap_insert_rmap(sg, p_gfn, r_gfn, level); | 1047 | rc = gmap_insert_rmap(sg, p_gfn, r_gfn & bitmask, level); |
| 1046 | } | 1048 | } |
| 1047 | if (rc) | 1049 | if (rc) |
| 1048 | return rc; | 1050 | return rc; |
| @@ -1143,8 +1145,10 @@ void _gmap_handle_vsie_unshadow_event(struct gmap *parent, gfn_t gfn) · 94 unmodified lines | |||
| 1143 | } | 1145 | } |
| 1144 | scoped_guard(spinlock, &sg->host_to_rmap_lock) | 1146 | scoped_guard(spinlock, &sg->host_to_rmap_lock) |
| 1145 | head = radix_tree_delete(&sg->host_to_rmap, gfn); | 1147 | head = radix_tree_delete(&sg->host_to_rmap, gfn); |
| 1146 | gmap_for_each_rmap_safe(rmap, rnext, head) | 1148 | gmap_for_each_rmap_safe(rmap, rnext, head) { |
| 1147 | gmap_unshadow_level(sg, rmap->r_gfn, rmap->level); | 1149 | gmap_unshadow_level(sg, rmap->r_gfn, rmap->level); |
| 1150 | kfree(rmap); | ||
| 1151 | } | ||
| 1148 | } | 1152 | } |
| 1149 | } | 1153 | } |
| 1150 | 1154 | ||
| @@ -167,6 +167,36 @@ static inline bool gmap_unmap_prefix(struct gmap *gmap, gfn_t gfn, gfn_t end) · 166 unmodified lines | |||
| 167 | return _gmap_unmap_prefix(gmap, gfn, end, false); | 167 | return _gmap_unmap_prefix(gmap, gfn, end, false); |
| 168 | } | 168 | } |
| 169 | 169 | ||
| 170 | /** | ||
| 171 | * pte_needs_unshadow() -- Check if the pte operations triggers unshadowing. | ||
| 172 | * @oldpte: the previous value for the guest pte. | ||
| 173 | * @newpte: the new pte being set. | ||
| 174 | * @pgste: the pgste for the pte entry. | ||
| 175 | * | ||
| 176 | * If the pgste.vsie_notif bit is not set, return false: the page is not | ||
| 177 | * involved in vsie and thus should not trigger an unshadow operation. | ||
| 178 | * | ||
| 179 | * If the pgste.vsie_gmem bit is set, this pte represents shadowed guest | ||
| 180 | * memory. The access rights on g3's memory should be synchronized with g1's | ||
| 181 | * and g2's. Therefore unshadowing is triggered if the new and old pte | ||
| 182 | * differ in protection, or if the new pte is invalid. | ||
| 183 | * | ||
| 184 | * If the pgste.vsie_gmem bit is not set, this pte maps the g2 dat tables | ||
| 185 | * for g3. If the entry becomes writable or absent, it becomes impossible to | ||
| 186 | * guarantee that the shadow mapping will match g2's mapping. In that case, | ||
| 187 | * trigger an unshadow event. | ||
| 188 | * | ||
| 189 | * Return: true if an unshadow event should be triggered, otherwise false. | ||
| 190 | */ | ||
| 191 | static inline bool pte_needs_unshadow(union pte oldpte, union pte newpte, union pgste pgste) | ||
| 192 | { | ||
| 193 | if (!pgste.vsie_notif) | ||
| 194 | return false; | ||
| 195 | if (pgste.vsie_gmem) | ||
| 196 | return (oldpte.h.p != newpte.h.p) || newpte.h.i; | ||
| 197 | return !newpte.h.p || !newpte.s.pr; | ||
| 198 | } | ||
| 199 | |||
| 170 | static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, union pte newpte, | 200 | static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, union pte newpte, |
| 171 | union pgste pgste, gfn_t gfn, bool needs_lock) | 201 | union pgste pgste, gfn_t gfn, bool needs_lock) |
| 172 | { | 202 | { |
| @@ -180,8 +210,9 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un · 7 unmodified lines | |||
| 180 | pgste.prefix_notif = 0; | 210 | pgste.prefix_notif = 0; |
| 181 | gmap_unmap_prefix(gmap, gfn, gfn + 1); | 211 | gmap_unmap_prefix(gmap, gfn, gfn + 1); |
| 182 | } | 212 | } |
| 183 | if (pgste.vsie_notif && (ptep->h.p != newpte.h.p || newpte.h.i)) { | 213 | if (pte_needs_unshadow(*ptep, newpte, pgste)) { |
| 184 | pgste.vsie_notif = 0; | 214 | pgste.vsie_notif = 0; |
| 215 | pgste.vsie_gmem = 0; | ||
| 185 | if (needs_lock) | 216 | if (needs_lock) |
| 186 | gmap_handle_vsie_unshadow_event(gmap, gfn); | 217 | gmap_handle_vsie_unshadow_event(gmap, gfn); |
| 187 | else | 218 | else |
| @@ -189,6 +220,7 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un · 1 unmodified line | |||
| 189 | } | 220 | } |
| 190 | if (!ptep->s.d && newpte.s.d && !newpte.s.s) | 221 | if (!ptep->s.d && newpte.s.d && !newpte.s.s) |
| 191 | SetPageDirty(pfn_to_page(newpte.h.pfra)); | 222 | SetPageDirty(pfn_to_page(newpte.h.pfra)); |
| 223 | pgste.zero = 0; | ||
| 192 | return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->asce, uses_skeys(gmap)); | 224 | return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->asce, uses_skeys(gmap)); |
| 193 | } | 225 | } |
| 194 | 226 | ||
| @@ -198,6 +230,30 @@ static inline union pgste gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, uni · 3 unmodified lines | |||
| 198 | return _gmap_ptep_xchg(gmap, ptep, newpte, pgste, gfn, true); | 230 | return _gmap_ptep_xchg(gmap, ptep, newpte, pgste, gfn, true); |
| 199 | } | 231 | } |
| 200 | 232 | ||
| 233 | /** | ||
| 234 | * crste_needs_unshadow() -- Check if the crste operations triggers unshadowing. | ||
| 235 | * @oldcrste: the previous value for the crste. | ||
| 236 | * @newcrste: the new value for the crste. | ||
| 237 | * | ||
| 238 | * If the old crste did not have the vsie_notif bit set, return false: the | ||
| 239 | * page is not involved in vsie and thus should not trigger an unshadow | ||
| 240 | * operation. Conversely, if the bit is set, it can only be g3 memory, since | ||
| 241 | * dat tables are never mapped using large pages. | ||
| 242 | * | ||
| 243 | * Similar to the pgste.vsie_gmem case of pte_needs_unshadow(), if the | ||
| 244 | * protection bit is changing or the new page is invalid, trigger an | ||
| 245 | * unshadow event. Also trigger an unshadow event if the new crste does not | ||
| 246 | * have the vsie_notif bit set. | ||
| 247 | * | ||
| 248 | * Return: true if an unshadow event should be triggered, otherwise false. | ||
| 249 | */ | ||
| 250 | static inline bool crste_needs_unshadow(union crste oldcrste, union crste newcrste) | ||
| 251 | { | ||
| 252 | if (!oldcrste.s.fc1.vsie_notif) | ||
| 253 | return false; | ||
| 254 | return (newcrste.h.p != oldcrste.h.p) || newcrste.h.i || !newcrste.s.fc1.vsie_notif; | ||
| 255 | } | ||
| 256 | |||
| 201 | static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, union crste *crstep, | 257 | static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, union crste *crstep, |
| 202 | union crste oldcrste, union crste newcrste, | 258 | union crste oldcrste, union crste newcrste, |
| 203 | gfn_t gfn, bool needs_lock) | 259 | gfn_t gfn, bool needs_lock) |
| @@ -216,8 +272,7 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, unio · 12 unmodified lines | |||
| 216 | newcrste.s.fc1.prefix_notif = 0; | 272 | newcrste.s.fc1.prefix_notif = 0; |
| 217 | gmap_unmap_prefix(gmap, gfn, gfn + align); | 273 | gmap_unmap_prefix(gmap, gfn, gfn + align); |
| 218 | } | 274 | } |
| 219 | if (crste_leaf(oldcrste) && oldcrste.s.fc1.vsie_notif && | 275 | if (crste_leaf(oldcrste) && crste_needs_unshadow(oldcrste, newcrste)) { |
| 220 | (newcrste.h.p || newcrste.h.i || !newcrste.s.fc1.vsie_notif)) { | ||
| 221 | newcrste.s.fc1.vsie_notif = 0; | 276 | newcrste.s.fc1.vsie_notif = 0; |
| 222 | if (needs_lock) | 277 | if (needs_lock) |
| 223 | gmap_handle_vsie_unshadow_event(gmap, gfn); | 278 | gmap_handle_vsie_unshadow_event(gmap, gfn); |
| @@ -3310,8 +3310,7 @@ static void aen_host_forward(unsigned long si) · 3309 unmodified lines | |||
| 3310 | struct zpci_gaite *gaite; | 3310 | struct zpci_gaite *gaite; |
| 3311 | struct kvm *kvm; | 3311 | struct kvm *kvm; |
| 3312 | 3312 | ||
| 3313 | gaite = (struct zpci_gaite *)aift->gait + | 3313 | gaite = aift->gait + si; |
| 3314 | (si * sizeof(struct zpci_gaite)); | ||
| 3315 | if (gaite->count == 0) | 3314 | if (gaite->count == 0) |
| 3316 | return; | 3315 | return; |
| 3317 | if (gaite->aisb != 0) | 3316 | if (gaite->aisb != 0) |
| @@ -166,7 +166,7 @@ static int kvm_zpci_set_airq(struct zpci_dev *zdev) · 165 unmodified lines | |||
| 166 | fib.fmt0.noi = airq_iv_end(zdev->aibv); | 166 | fib.fmt0.noi = airq_iv_end(zdev->aibv); |
| 167 | fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector); | 167 | fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector); |
| 168 | fib.fmt0.aibvo = 0; | 168 | fib.fmt0.aibvo = 0; |
| 169 | fib.fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); | 169 | fib.fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; |
| 170 | fib.fmt0.aisbo = zdev->aisb & 63; | 170 | fib.fmt0.aisbo = zdev->aisb & 63; |
| 171 | fib.gd = zdev->gisa; | 171 | fib.gd = zdev->gisa; |
| 172 | 172 | ||
| @@ -290,8 +290,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 117 unmodified lines | |||
| 290 | phys_to_virt(fib->fmt0.aibv)); | 290 | phys_to_virt(fib->fmt0.aibv)); |
| 291 | 291 | ||
| 292 | spin_lock_irq(&aift->gait_lock); | 292 | spin_lock_irq(&aift->gait_lock); |
| 293 | gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb * | 293 | gaite = aift->gait + zdev->aisb; |
| 294 | sizeof(struct zpci_gaite)); | ||
| 295 | 294 | ||
| 296 | /* If assist not requested, host will get all alerts */ | 295 | /* If assist not requested, host will get all alerts */ |
| 297 | if (assist) | 296 | if (assist) |
| @@ -309,7 +308,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 11 unmodified lines | |||
| 309 | 308 | ||
| 310 | /* Update guest FIB for re-issue */ | 309 | /* Update guest FIB for re-issue */ |
| 311 | fib->fmt0.aisbo = zdev->aisb & 63; | 310 | fib->fmt0.aisbo = zdev->aisb & 63; |
| 312 | fib->fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); | 311 | fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; |
| 313 | fib->fmt0.isc = gisc; | 312 | fib->fmt0.isc = gisc; |
| 314 | 313 | ||
| 315 | /* Save some guest fib values in the host for later use */ | 314 | /* Save some guest fib values in the host for later use */ |
| @@ -357,8 +356,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) · 41 unmodified lines | |||
| 357 | if (zdev->kzdev->fib.fmt0.aibv == 0) | 356 | if (zdev->kzdev->fib.fmt0.aibv == 0) |
| 358 | goto out; | 357 | goto out; |
| 359 | spin_lock_irq(&aift->gait_lock); | 358 | spin_lock_irq(&aift->gait_lock); |
| 360 | gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb * | 359 | gaite = aift->gait + zdev->aisb; |
| 361 | sizeof(struct zpci_gaite)); | ||
| 362 | isc = gaite->gisc; | 360 | isc = gaite->gisc; |
| 363 | gaite->count--; | 361 | gaite->count--; |
| 364 | if (gaite->count == 0) { | 362 | if (gaite->count == 0) { |
| @@ -438,7 +438,7 @@ void do_secure_storage_access(struct pt_regs *regs) · 437 unmodified lines | |||
| 438 | panic("Unexpected PGM 0x3d with TEID bit 61=0"); | 438 | panic("Unexpected PGM 0x3d with TEID bit 61=0"); |
| 439 | } | 439 | } |
| 440 | if (is_kernel_fault(regs)) { | 440 | if (is_kernel_fault(regs)) { |
| 441 | folio = phys_to_folio(addr); | 441 | folio = virt_to_folio((void *)addr); |
| 442 | if (unlikely(!folio_try_get(folio))) | 442 | if (unlikely(!folio_try_get(folio))) |
| 443 | return; | 443 | return; |
| 444 | rc = uv_convert_from_secure(folio_to_phys(folio)); | 444 | rc = uv_convert_from_secure(folio_to_phys(folio)); |
| @@ -3,4 +3,5 @@ generated-y += syscall_table.h · 2 unmodified lines | |||
| 3 | generic-y += kvm_para.h | 3 | generic-y += kvm_para.h |
| 4 | generic-y += mcs_spinlock.h | 4 | generic-y += mcs_spinlock.h |
| 5 | generic-y += parport.h | 5 | generic-y += parport.h |
| 6 | generic-y += ring_buffer.h | ||
| 6 | generic-y += text-patching.h | 7 | generic-y += text-patching.h |
| @@ -7,7 +7,7 @@ · 6 unmodified lines | |||
| 7 | /* | 7 | /* |
| 8 | * This is set up by the setup-routine at boot-time | 8 | * This is set up by the setup-routine at boot-time |
| 9 | */ | 9 | */ |
| 10 | extern unsigned char *boot_params_page; | 10 | extern unsigned char boot_params_page[]; |
| 11 | #define PARAM boot_params_page | 11 | #define PARAM boot_params_page |
| 12 | 12 | ||
| 13 | #define MOUNT_ROOT_RDONLY (*(unsigned long *) (PARAM+0x000)) | 13 | #define MOUNT_ROOT_RDONLY (*(unsigned long *) (PARAM+0x000)) |
| @@ -4,4 +4,5 @@ generated-y += syscall_table_64.h · 3 unmodified lines | |||
| 4 | generic-y += agp.h | 4 | generic-y += agp.h |
| 5 | generic-y += kvm_para.h | 5 | generic-y += kvm_para.h |
| 6 | generic-y += mcs_spinlock.h | 6 | generic-y += mcs_spinlock.h |
| 7 | generic-y += ring_buffer.h | ||
| 7 | generic-y += text-patching.h | 8 | generic-y += text-patching.h |
| @@ -17,6 +17,7 @@ generic-y += module.lds.h · 16 unmodified lines | |||
| 17 | generic-y += parport.h | 17 | generic-y += parport.h |
| 18 | generic-y += percpu.h | 18 | generic-y += percpu.h |
| 19 | generic-y += preempt.h | 19 | generic-y += preempt.h |
| 20 | generic-y += ring_buffer.h | ||
| 20 | generic-y += runtime-const.h | 21 | generic-y += runtime-const.h |
| 21 | generic-y += softirq_stack.h | 22 | generic-y += softirq_stack.h |
| 22 | generic-y += switch_to.h | 23 | generic-y += switch_to.h |
| @@ -13,7 +13,7 @@ CFLAGS_REMOVE_syscall_64.o = $(CC_FLAGS_FTRACE) · 12 unmodified lines | |||
| 13 | CFLAGS_syscall_32.o += -fno-stack-protector | 13 | CFLAGS_syscall_32.o += -fno-stack-protector |
| 14 | CFLAGS_syscall_64.o += -fno-stack-protector | 14 | CFLAGS_syscall_64.o += -fno-stack-protector |
| 15 | 15 | ||
| 16 | obj-y := entry.o entry_$(BITS).o syscall_$(BITS).o | 16 | obj-y := entry.o entry_$(BITS).o syscall_$(BITS).o common.o |
| 17 | 17 | ||
| 18 | obj-y += vdso/ | 18 | obj-y += vdso/ |
| 19 | obj-y += vsyscall/ | 19 | obj-y += vsyscall/ |
| @@ -0,0 +1,61 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | ||
| 2 | |||
| 3 | #include <linux/entry-common.h> | ||
| 4 | #include <linux/kvm_types.h> | ||
| 5 | #include <linux/hrtimer_rearm.h> | ||
| 6 | #include <asm/fred.h> | ||
| 7 | #include <asm/desc.h> | ||
| 8 | |||
| 9 | #if IS_ENABLED(CONFIG_KVM_INTEL) | ||
| 10 | /* | ||
| 11 | * On VMX, NMIs and IRQs (as configured by KVM) are acknowledged by hardware as | ||
| 12 | * part of the VM-Exit, i.e. the event itself is consumed as part the VM-Exit. | ||
| 13 | * x86_entry_from_kvm() is invoked by KVM to effectively forward NMIs and IRQs | ||
| 14 | * to the kernel for servicing. On SVM, a.k.a. AMD, the NMI/IRQ VM-Exit is | ||
| 15 | * purely a signal that an NMI/IRQ is pending, i.e. the event that triggered | ||
| 16 | * the VM-Exit is held pending until it's unblocked in the host. | ||
| 17 | */ | ||
| 18 | noinstr void x86_entry_from_kvm(unsigned int event_type, unsigned int vector) | ||
| 19 | { | ||
| 20 | if (event_type == EVENT_TYPE_EXTINT) { | ||
| 21 | #ifdef CONFIG_X86_64 | ||
| 22 | /* | ||
| 23 | * Use FRED dispatch, even when running IDT. The dispatch | ||
| 24 | * tables are kept in sync between FRED and IDT, and the FRED | ||
| 25 | * dispatch works well with CFI. | ||
| 26 | */ | ||
| 27 | fred_entry_from_kvm(event_type, vector); | ||
| 28 | #else | ||
| 29 | idt_entry_from_kvm(vector); | ||
| 30 | #endif | ||
| 31 | /* | ||
| 32 | * Strictly speaking, only the NMI path requires noinstr. | ||
| 33 | */ | ||
| 34 | instrumentation_begin(); | ||
| 35 | /* | ||
| 36 | * KVM/VMX will dispatch from IRQ-disabled but for a context | ||
| 37 | * that will have IRQs-enabled. This confuses the entry code | ||
| 38 | * and it will not have reprogrammed the timer. Do so now. | ||
| 39 | */ | ||
| 40 | hrtimer_rearm_deferred(); | ||
| 41 | instrumentation_end(); | ||
| 42 | |||
| 43 | return; | ||
| 44 | } | ||
| 45 | |||
| 46 | WARN_ON_ONCE(event_type != EVENT_TYPE_NMI); | ||
| 47 | |||
| 48 | #ifdef CONFIG_X86_64 | ||
| 49 | if (cpu_feature_enabled(X86_FEATURE_FRED)) | ||
| 50 | return fred_entry_from_kvm(event_type, vector); | ||
| 51 | #endif | ||
| 52 | |||
| 53 | /* | ||
| 54 | * Notably, we must use IDT dispatch for NMI when running in IDT mode. | ||
| 55 | * The FRED NMI context is significantly different and will not work | ||
| 56 | * right (specifically FRED fixed the NMI recursion issue). | ||
| 57 | */ | ||
| 58 | idt_do_nmi_irqoff(); | ||
| 59 | } | ||
| 60 | EXPORT_SYMBOL_FOR_KVM(x86_entry_from_kvm); | ||
| 61 | #endif | ||
| @@ -75,3 +75,51 @@ THUNK warn_thunk_thunk, __warn_thunk · 74 unmodified lines | |||
| 75 | #if defined(CONFIG_STACKPROTECTOR) && defined(CONFIG_SMP) | 75 | #if defined(CONFIG_STACKPROTECTOR) && defined(CONFIG_SMP) |
| 76 | EXPORT_SYMBOL(__ref_stack_chk_guard); | 76 | EXPORT_SYMBOL(__ref_stack_chk_guard); |
| 77 | #endif | 77 | #endif |
| 78 | |||
| 79 | #if IS_ENABLED(CONFIG_KVM_INTEL) | ||
| 80 | .macro IDT_DO_EVENT_IRQOFF call_insn call_target | ||
| 81 | /* | ||
| 82 | * Unconditionally create a stack frame, getting the correct RSP on the | ||
| 83 | * stack (for x86-64) would take two instructions anyways, and RBP can | ||
| 84 | * be used to restore RSP to make objtool happy (see below). | ||
| 85 | */ | ||
| 86 | push %_ASM_BP | ||
| 87 | mov %_ASM_SP, %_ASM_BP | ||
| 88 | |||
| 89 | #ifdef CONFIG_X86_64 | ||
| 90 | /* | ||
| 91 | * Align RSP to a 16-byte boundary (to emulate CPU behavior) before | ||
| 92 | * creating the synthetic interrupt stack frame for the IRQ/NMI. | ||
| 93 | */ | ||
| 94 | and $-16, %rsp | ||
| 95 | push $__KERNEL_DS | ||
| 96 | push %rbp | ||
| 97 | #endif | ||
| 98 | pushf | ||
| 99 | push $__KERNEL_CS | ||
| 100 | \call_insn \call_target | ||
| 101 | |||
| 102 | /* | ||
| 103 | * "Restore" RSP from RBP, even though IRET has already unwound RSP to | ||
| 104 | * the correct value. objtool doesn't know the callee will IRET and, | ||
| 105 | * without the explicit restore, thinks the stack is getting walloped. | ||
| 106 | * Using an unwind hint is problematic due to x86-64's dynamic alignment. | ||
| 107 | */ | ||
| 108 | leave | ||
| 109 | RET | ||
| 110 | .endm | ||
| 111 | |||
| 112 | #ifndef CONFIG_X86_64 | ||
| 113 | .pushsection .text, "ax" | ||
| 114 | SYM_FUNC_START(idt_do_interrupt_irqoff) | ||
| 115 | IDT_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 | ||
| 116 | SYM_FUNC_END(idt_do_interrupt_irqoff) | ||
| 117 | .popsection | ||
| 118 | #endif | ||
| 119 | |||
| 120 | .pushsection .noinstr.text, "ax" | ||
| 121 | SYM_FUNC_START(idt_do_nmi_irqoff) | ||
| 122 | IDT_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx | ||
| 123 | SYM_FUNC_END(idt_do_nmi_irqoff) | ||
| 124 | .popsection | ||
| 125 | #endif | ||
| @@ -147,5 +147,4 @@ SYM_FUNC_START(asm_fred_entry_from_kvm) · 146 unmodified lines | |||
| 147 | RET | 147 | RET |
| 148 | 148 | ||
| 149 | SYM_FUNC_END(asm_fred_entry_from_kvm) | 149 | SYM_FUNC_END(asm_fred_entry_from_kvm) |
| 150 | EXPORT_SYMBOL_FOR_KVM(asm_fred_entry_from_kvm); | ||
| 151 | #endif | 150 | #endif |
| @@ -178,7 +178,7 @@ static int map_vdso(const struct vdso_image *image, unsigned long addr) · 177 unmodified lines | |||
| 178 | if (IS_ERR(vma)) { | 178 | if (IS_ERR(vma)) { |
| 179 | ret = PTR_ERR(vma); | 179 | ret = PTR_ERR(vma); |
| 180 | do_munmap(mm, text_start, image->size, NULL); | 180 | do_munmap(mm, text_start, image->size, NULL); |
| 181 | do_munmap(mm, addr, image->size, NULL); | 181 | do_munmap(mm, addr, VDSO_NR_PAGES * PAGE_SIZE, NULL); |
| 182 | goto up_fail; | 182 | goto up_fail; |
| 183 | } | 183 | } |
| 184 | 184 | ||
| @@ -1294,13 +1294,16 @@ int x86_perf_rdpmc_index(struct perf_event *event) · 1293 unmodified lines | |||
| 1294 | return event->hw.event_base_rdpmc; | 1294 | return event->hw.event_base_rdpmc; |
| 1295 | } | 1295 | } |
| 1296 | 1296 | ||
| 1297 | static inline int match_prev_assignment(struct hw_perf_event *hwc, | 1297 | static inline int match_prev_assignment(struct perf_event *event, |
| 1298 | struct cpu_hw_events *cpuc, | 1298 | struct cpu_hw_events *cpuc, |
| 1299 | int i) | 1299 | int i) |
| 1300 | { | 1300 | { |
| 1301 | struct hw_perf_event *hwc = &event->hw; | ||
| 1302 | |||
| 1301 | return hwc->idx == cpuc->assign[i] && | 1303 | return hwc->idx == cpuc->assign[i] && |
| 1302 | hwc->last_cpu == smp_processor_id() && | 1304 | hwc->last_cpu == smp_processor_id() && |
| 1303 | hwc->last_tag == cpuc->tags[i]; | 1305 | hwc->last_tag == cpuc->tags[i] && |
| 1306 | !is_acr_event_group(event); | ||
| 1304 | } | 1307 | } |
| 1305 | 1308 | ||
| 1306 | static void x86_pmu_start(struct perf_event *event, int flags); | 1309 | static void x86_pmu_start(struct perf_event *event, int flags); |
| @@ -1346,7 +1349,7 @@ static void x86_pmu_enable(struct pmu *pmu) · 39 unmodified lines | |||
| 1346 | * - no other event has used the counter since | 1349 | * - no other event has used the counter since |
| 1347 | */ | 1350 | */ |
| 1348 | if (hwc->idx == -1 || | 1351 | if (hwc->idx == -1 || |
| 1349 | match_prev_assignment(hwc, cpuc, i)) | 1352 | match_prev_assignment(event, cpuc, i)) |
| 1350 | continue; | 1353 | continue; |
| 1351 | 1354 | ||
| 1352 | /* | 1355 | /* |
| @@ -1367,7 +1370,7 @@ static void x86_pmu_enable(struct pmu *pmu) · 14 unmodified lines | |||
| 1367 | event = cpuc->event_list[i]; | 1370 | event = cpuc->event_list[i]; |
| 1368 | hwc = &event->hw; | 1371 | hwc = &event->hw; |
| 1369 | 1372 | ||
| 1370 | if (!match_prev_assignment(hwc, cpuc, i)) | 1373 | if (!match_prev_assignment(event, cpuc, i)) |
| 1371 | x86_assign_hw_event(event, cpuc, i); | 1374 | x86_assign_hw_event(event, cpuc, i); |
| 1372 | else if (i < n_running) | 1375 | else if (i < n_running) |
| 1373 | continue; | 1376 | continue; |
| @@ -3118,11 +3118,11 @@ static void intel_pmu_enable_fixed(struct perf_event *event) · 3117 unmodified lines | |||
| 3118 | intel_set_masks(event, idx); | 3118 | intel_set_masks(event, idx); |
| 3119 | 3119 | ||
| 3120 | /* | 3120 | /* |
| 3121 | * Enable IRQ generation (0x8), if not PEBS, | 3121 | * Enable IRQ generation (0x8), if not PEBS or self-reloaded |
| 3122 | * and enable ring-3 counting (0x2) and ring-0 counting (0x1) | 3122 | * ACR event, and enable ring-3 counting (0x2) and ring-0 |
| 3123 | * if requested: | 3123 | * counting (0x1) if requested: |
| 3124 | */ | 3124 | */ |
| 3125 | if (!event->attr.precise_ip) | 3125 | if (!event->attr.precise_ip && !is_acr_self_reload_event(event)) |
| 3126 | bits |= INTEL_FIXED_0_ENABLE_PMI; | 3126 | bits |= INTEL_FIXED_0_ENABLE_PMI; |
| 3127 | if (hwc->config & ARCH_PERFMON_EVENTSEL_USR) | 3127 | if (hwc->config & ARCH_PERFMON_EVENTSEL_USR) |
| 3128 | bits |= INTEL_FIXED_0_USER; | 3128 | bits |= INTEL_FIXED_0_USER; |
| @@ -3306,6 +3306,15 @@ static void intel_pmu_enable_event(struct perf_event *event) · 177 unmodified lines | |||
| 3306 | intel_set_masks(event, idx); | 3306 | intel_set_masks(event, idx); |
| 3307 | static_call_cond(intel_pmu_enable_acr_event)(event); | 3307 | static_call_cond(intel_pmu_enable_acr_event)(event); |
| 3308 | static_call_cond(intel_pmu_enable_event_ext)(event); | 3308 | static_call_cond(intel_pmu_enable_event_ext)(event); |
| 3309 | /* | ||
| 3310 | * For self-reloaded ACR event, don't enable PMI since | ||
| 3311 | * HW won't set overflow bit in GLOBAL_STATUS. Otherwise, | ||
| 3312 | * the PMI would be recognized as a suspicious NMI. | ||
| 3313 | */ | ||
| 3314 | if (is_acr_self_reload_event(event)) | ||
| 3315 | hwc->config &= ~ARCH_PERFMON_EVENTSEL_INT; | ||
| 3316 | else if (!event->attr.precise_ip) | ||
| 3317 | hwc->config |= ARCH_PERFMON_EVENTSEL_INT; | ||
| 3309 | __x86_pmu_enable_event(hwc, enable_mask); | 3318 | __x86_pmu_enable_event(hwc, enable_mask); |
| 3310 | break; | 3319 | break; |
| 3311 | case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS - 1: | 3320 | case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS - 1: |
| @@ -3332,23 +3341,41 @@ static void intel_pmu_enable_event(struct perf_event *event) · 20 unmodified lines | |||
| 3332 | static void intel_pmu_acr_late_setup(struct cpu_hw_events *cpuc) | 3341 | static void intel_pmu_acr_late_setup(struct cpu_hw_events *cpuc) |
| 3333 | { | 3342 | { |
| 3334 | struct perf_event *event, *leader; | 3343 | struct perf_event *event, *leader; |
| 3335 | int i, j, idx; | 3344 | int i, j, k, bit, idx; |
| 3336 | 3345 | ||
| 3346 | /* | ||
| 3347 | * FIXME: ACR mask parsing relies on cpuc->event_list[] (active events only). | ||
| 3348 | * Disabling an ACR event causes bit-shifting errors in the acr_mask of | ||
| 3349 | * remaining group members. As ACR sampling requires all events to be active, | ||
| 3350 | * this limitation is acceptable for now. Revisit if independent event toggling | ||
| 3351 | * is required. | ||
| 3352 | */ | ||
| 3337 | for (i = 0; i < cpuc->n_events; i++) { | 3353 | for (i = 0; i < cpuc->n_events; i++) { |
| 3338 | leader = cpuc->event_list[i]; | 3354 | leader = cpuc->event_list[i]; |
| 3339 | if (!is_acr_event_group(leader)) | 3355 | if (!is_acr_event_group(leader)) |
| 3340 | continue; | 3356 | continue; |
| 3341 | 3357 | ||
| 3342 | /* The ACR events must be contiguous. */ | 3358 | /* Find the last event of the ACR group. */ |
| 3343 | for (j = i; j < cpuc->n_events; j++) { | 3359 | for (j = i; j < cpuc->n_events; j++) { |
| 3344 | event = cpuc->event_list[j]; | 3360 | event = cpuc->event_list[j]; |
| 3345 | if (event->group_leader != leader->group_leader) | 3361 | if (event->group_leader != leader->group_leader) |
| 3346 | break; | 3362 | break; |
| 3347 | for_each_set_bit(idx, (unsigned long *)&event->attr.config2, X86_PMC_IDX_MAX) { | 3363 | } |
| 3348 | if (i + idx >= cpuc->n_events || | 3364 | |
| 3349 | !is_acr_event_group(cpuc->event_list[i + idx])) | 3365 | /* |
| 3350 | return; | 3366 | * Translate the user-space ACR mask (attr.config2) into the physical |
| 3351 | __set_bit(cpuc->assign[i + idx], (unsigned long *)&event->hw.config1); | 3367 | * counter bitmask (hw.config1) for each ACR event in the group. |
| 3368 | * NOTE: ACR event contiguity is guaranteed by intel_pmu_hw_config(). | ||
| 3369 | */ | ||
| 3370 | for (k = i; k < j; k++) { | ||
| 3371 | event = cpuc->event_list[k]; | ||
| 3372 | event->hw.config1 = 0; | ||
| 3373 | for_each_set_bit(bit, (unsigned long *)&event->attr.config2, X86_PMC_IDX_MAX) { | ||
| 3374 | idx = i + bit; | ||
| 3375 | /* Event index of ACR group must locate in [i, j). */ | ||
| 3376 | if (idx >= j || !is_acr_event_group(cpuc->event_list[idx])) | ||
| 3377 | continue; | ||
| 3378 | __set_bit(cpuc->assign[idx], (unsigned long *)&event->hw.config1); | ||
| 3352 | } | 3379 | } |
| 3353 | } | 3380 | } |
| 3354 | i = j - 1; | 3381 | i = j - 1; |
| @@ -7504,6 +7531,7 @@ static __always_inline void intel_pmu_init_pnc(struct pmu *pmu) · 4149 unmodified lines | |||
| 7504 | hybrid(pmu, event_constraints) = intel_pnc_event_constraints; | 7531 | hybrid(pmu, event_constraints) = intel_pnc_event_constraints; |
| 7505 | hybrid(pmu, pebs_constraints) = intel_pnc_pebs_event_constraints; | 7532 | hybrid(pmu, pebs_constraints) = intel_pnc_pebs_event_constraints; |
| 7506 | hybrid(pmu, extra_regs) = intel_pnc_extra_regs; | 7533 | hybrid(pmu, extra_regs) = intel_pnc_extra_regs; |
| 7534 | static_call_update(intel_pmu_enable_acr_event, intel_pmu_enable_acr); | ||
| 7507 | } | 7535 | } |
| 7508 | 7536 | ||
| 7509 | static __always_inline void intel_pmu_init_skt(struct pmu *pmu) | 7537 | static __always_inline void intel_pmu_init_skt(struct pmu *pmu) |
| @@ -137,6 +137,16 @@ static inline bool is_acr_event_group(struct perf_event *event) · 136 unmodified lines | |||
| 137 | return check_leader_group(event->group_leader, PERF_X86_EVENT_ACR); | 137 | return check_leader_group(event->group_leader, PERF_X86_EVENT_ACR); |
| 138 | } | 138 | } |
| 139 | 139 | ||
| 140 | static inline bool is_acr_self_reload_event(struct perf_event *event) | ||
| 141 | { | ||
| 142 | struct hw_perf_event *hwc = &event->hw; | ||
| 143 | |||
| 144 | if (hwc->idx < 0) | ||
| 145 | return false; | ||
| 146 | |||
| 147 | return test_bit(hwc->idx, (unsigned long *)&hwc->config1); | ||
| 148 | } | ||
| 149 | |||
| 140 | struct amd_nb { | 150 | struct amd_nb { |
| 141 | int nb_id; /* NorthBridge id */ | 151 | int nb_id; /* NorthBridge id */ |
| 142 | int refcnt; /* reference count */ | 152 | int refcnt; /* reference count */ |
| @@ -14,3 +14,4 @@ generic-y += early_ioremap.h · 13 unmodified lines | |||
| 14 | generic-y += fprobe.h | 14 | generic-y += fprobe.h |
| 15 | generic-y += mcs_spinlock.h | 15 | generic-y += mcs_spinlock.h |
| 16 | generic-y += mmzone.h | 16 | generic-y += mmzone.h |
| 17 | generic-y += ring_buffer.h | ||
| @@ -438,6 +438,10 @@ extern void idt_setup_traps(void); · 437 unmodified lines | |||
| 438 | extern void idt_setup_apic_and_irq_gates(void); | 438 | extern void idt_setup_apic_and_irq_gates(void); |
| 439 | extern bool idt_is_f00f_address(unsigned long address); | 439 | extern bool idt_is_f00f_address(unsigned long address); |
| 440 | 440 | ||
| 441 | extern void idt_do_interrupt_irqoff(unsigned long address); | ||
| 442 | extern void idt_do_nmi_irqoff(void); | ||
| 443 | extern void idt_entry_from_kvm(unsigned int vector); | ||
| 444 | |||
| 441 | #ifdef CONFIG_X86_64 | 445 | #ifdef CONFIG_X86_64 |
| 442 | extern void idt_setup_early_pf(void); | 446 | extern void idt_setup_early_pf(void); |
| 443 | #else | 447 | #else |
| @@ -145,7 +145,7 @@ struct gate_struct { · 144 unmodified lines | |||
| 145 | typedef struct gate_struct gate_desc; | 145 | typedef struct gate_struct gate_desc; |
| 146 | 146 | ||
| 147 | #ifndef _SETUP | 147 | #ifndef _SETUP |
| 148 | static inline unsigned long gate_offset(const gate_desc *g) | 148 | static __always_inline unsigned long gate_offset(const gate_desc *g) |
| 149 | { | 149 | { |
| 150 | #ifdef CONFIG_X86_64 | 150 | #ifdef CONFIG_X86_64 |
| 151 | return g->offset_low | ((unsigned long)g->offset_middle << 16) | | 151 | return g->offset_low | ((unsigned long)g->offset_middle << 16) | |
| @@ -137,7 +137,8 @@ extern void __init efi_dump_pagetable(void); · 136 unmodified lines | |||
| 137 | extern void __init efi_apply_memmap_quirks(void); | 137 | extern void __init efi_apply_memmap_quirks(void); |
| 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); | 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); |
| 139 | extern void efi_delete_dummy_variable(void); | 139 | extern void efi_delete_dummy_variable(void); |
| 140 | extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); | 140 | extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr, |
| 141 | const struct pt_regs *regs); | ||
| 141 | extern void efi_unmap_boot_services(void); | 142 | extern void efi_unmap_boot_services(void); |
| 142 | 143 | ||
| 143 | void arch_efi_call_virt_setup(void); | 144 | void arch_efi_call_virt_setup(void); |
| @@ -97,4 +97,6 @@ static __always_inline void arch_exit_to_user_mode(void) · 96 unmodified lines | |||
| 97 | } | 97 | } |
| 98 | #define arch_exit_to_user_mode arch_exit_to_user_mode | 98 | #define arch_exit_to_user_mode arch_exit_to_user_mode |
| 99 | 99 | ||
| 100 | extern void x86_entry_from_kvm(unsigned int entry_type, unsigned int vector); | ||
| 101 | |||
| 100 | #endif | 102 | #endif |
| @@ -110,7 +110,6 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { ret · 109 unmodified lines | |||
| 110 | static inline void cpu_init_fred_exceptions(void) { } | 110 | static inline void cpu_init_fred_exceptions(void) { } |
| 111 | static inline void cpu_init_fred_rsps(void) { } | 111 | static inline void cpu_init_fred_rsps(void) { } |
| 112 | static inline void fred_complete_exception_setup(void) { } | 112 | static inline void fred_complete_exception_setup(void) { } |
| 113 | static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { } | ||
| 114 | static inline void fred_sync_rsp0(unsigned long rsp0) { } | 113 | static inline void fred_sync_rsp0(unsigned long rsp0) { } |
| 115 | static inline void fred_update_rsp0(void) { } | 114 | static inline void fred_update_rsp0(void) { } |
| 116 | #endif /* CONFIG_X86_FRED */ | 115 | #endif /* CONFIG_X86_FRED */ |
| @@ -1504,6 +1504,7 @@ struct kvm_arch { · 1503 unmodified lines | |||
| 1504 | bool use_master_clock; | 1504 | bool use_master_clock; |
| 1505 | u64 master_kernel_ns; | 1505 | u64 master_kernel_ns; |
| 1506 | u64 master_cycle_now; | 1506 | u64 master_cycle_now; |
| 1507 | struct ratelimit_state kvmclock_update_rs; | ||
| 1507 | 1508 | ||
| 1508 | #ifdef CONFIG_KVM_HYPERV | 1509 | #ifdef CONFIG_KVM_HYPERV |
| 1509 | struct kvm_hv hyperv; | 1510 | struct kvm_hv hyperv; |
| @@ -803,9 +803,10 @@ · 802 unmodified lines | |||
| 803 | #define MSR_AMD64_LBR_SELECT 0xc000010e | 803 | #define MSR_AMD64_LBR_SELECT 0xc000010e |
| 804 | 804 | ||
| 805 | /* Zen4 */ | 805 | /* Zen4 */ |
| 806 | #define MSR_ZEN4_BP_CFG 0xc001102e | 806 | #define MSR_ZEN4_BP_CFG 0xc001102e |
| 807 | #define MSR_ZEN4_BP_CFG_BP_SPEC_REDUCE_BIT 4 | 807 | #define MSR_ZEN4_BP_CFG_BP_SPEC_REDUCE_BIT 4 |
| 808 | #define MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT 5 | 808 | #define MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT 5 |
| 809 | #define MSR_ZEN2_BP_CFG_BUG_FIX_BIT 33 | ||
| 809 | 810 | ||
| 810 | /* Fam 19h MSRs */ | 811 | /* Fam 19h MSRs */ |
| 811 | #define MSR_F19H_UMC_PERF_CTL 0xc0010800 | 812 | #define MSR_F19H_UMC_PERF_CTL 0xc0010800 |
| @@ -733,6 +733,7 @@ bool xen_set_default_idle(void); · 732 unmodified lines | |||
| 733 | #endif | 733 | #endif |
| 734 | 734 | ||
| 735 | void __noreturn stop_this_cpu(void *dummy); | 735 | void __noreturn stop_this_cpu(void *dummy); |
| 736 | extern bool x86_hypervisor_present; | ||
| 736 | void microcode_check(struct cpuinfo_x86 *prev_info); | 737 | void microcode_check(struct cpuinfo_x86 *prev_info); |
| 737 | void store_cpu_caps(struct cpuinfo_x86 *info); | 738 | void store_cpu_caps(struct cpuinfo_x86 *info); |
| 738 | 739 | ||
| @@ -88,19 +88,19 @@ static void amd_set_max_freq_ratio(void) · 87 unmodified lines | |||
| 88 | 88 | ||
| 89 | rc = cppc_get_perf_caps(0, &perf_caps); | 89 | rc = cppc_get_perf_caps(0, &perf_caps); |
| 90 | if (rc) { | 90 | if (rc) { |
| 91 | pr_warn("Could not retrieve perf counters (%d)\n", rc); | 91 | pr_debug("Could not retrieve perf counters (%d)\n", rc); |
| 92 | return; | 92 | return; |
| 93 | } | 93 | } |
| 94 | 94 | ||
| 95 | rc = amd_get_boost_ratio_numerator(0, &numerator); | 95 | rc = amd_get_boost_ratio_numerator(0, &numerator); |
| 96 | if (rc) { | 96 | if (rc) { |
| 97 | pr_warn("Could not retrieve highest performance (%d)\n", rc); | 97 | pr_debug("Could not retrieve highest performance (%d)\n", rc); |
| 98 | return; | 98 | return; |
| 99 | } | 99 | } |
| 100 | nominal_perf = perf_caps.nominal_perf; | 100 | nominal_perf = perf_caps.nominal_perf; |
| 101 | 101 | ||
| 102 | if (!nominal_perf) { | 102 | if (!nominal_perf) { |
| 103 | pr_warn("Could not retrieve nominal performance\n"); | 103 | pr_debug("Could not retrieve nominal performance\n"); |
| 104 | return; | 104 | return; |
| 105 | } | 105 | } |
| 106 | 106 | ||
| @@ -989,6 +989,9 @@ static void init_amd_zen2(struct cpuinfo_x86 *c) · 988 unmodified lines | |||
| 989 | 989 | ||
| 990 | /* Correct misconfigured CPUID on some clients. */ | 990 | /* Correct misconfigured CPUID on some clients. */ |
| 991 | clear_cpu_cap(c, X86_FEATURE_INVLPGB); | 991 | clear_cpu_cap(c, X86_FEATURE_INVLPGB); |
| 992 | |||
| 993 | if (!cpu_has(c, X86_FEATURE_HYPERVISOR)) | ||
| 994 | msr_set_bit(MSR_ZEN4_BP_CFG, MSR_ZEN2_BP_CFG_BUG_FIX_BIT); | ||
| 992 | } | 995 | } |
| 993 | 996 | ||
| 994 | static void init_amd_zen3(struct cpuinfo_x86 *c) | 997 | static void init_amd_zen3(struct cpuinfo_x86 *c) |
| @@ -92,6 +92,7 @@ static const struct cpuid_dep cpuid_deps[] = { · 91 unmodified lines | |||
| 92 | { X86_FEATURE_FRED, X86_FEATURE_LKGS }, | 92 | { X86_FEATURE_FRED, X86_FEATURE_LKGS }, |
| 93 | { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, | 93 | { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, |
| 94 | { X86_FEATURE_LASS, X86_FEATURE_SMAP }, | 94 | { X86_FEATURE_LASS, X86_FEATURE_SMAP }, |
| 95 | { X86_FEATURE_INVLPGB, X86_FEATURE_PCID }, | ||
| 95 | {} | 96 | {} |
| 96 | }; | 97 | }; |
| 97 | 98 | ||
| @@ -90,7 +90,6 @@ struct mca_config mca_cfg __read_mostly = { · 89 unmodified lines | |||
| 90 | }; | 90 | }; |
| 91 | 91 | ||
| 92 | static DEFINE_PER_CPU(struct mce_hw_err, hw_errs_seen); | 92 | static DEFINE_PER_CPU(struct mce_hw_err, hw_errs_seen); |
| 93 | static unsigned long mce_need_notify; | ||
| 94 | 93 | ||
| 95 | /* | 94 | /* |
| 96 | * MCA banks polled by the period polling timer for corrected events. | 95 | * MCA banks polled by the period polling timer for corrected events. |
| @@ -152,8 +151,10 @@ EXPORT_PER_CPU_SYMBOL_GPL(injectm); · 55 unmodified lines | |||
| 152 | 151 | ||
| 153 | void mce_log(struct mce_hw_err *err) | 152 | void mce_log(struct mce_hw_err *err) |
| 154 | { | 153 | { |
| 155 | if (mce_gen_pool_add(err)) | 154 | if (mce_gen_pool_add(err)) { |
| 155 | pr_info(HW_ERR "Machine check events logged\n"); | ||
| 156 | irq_work_queue(&mce_irq_work); | 156 | irq_work_queue(&mce_irq_work); |
| 157 | } | ||
| 157 | } | 158 | } |
| 158 | EXPORT_SYMBOL_GPL(mce_log); | 159 | EXPORT_SYMBOL_GPL(mce_log); |
| 159 | 160 | ||
| @@ -585,28 +586,6 @@ bool mce_is_correctable(struct mce *m) · 425 unmodified lines | |||
| 585 | } | 586 | } |
| 586 | EXPORT_SYMBOL_GPL(mce_is_correctable); | 587 | EXPORT_SYMBOL_GPL(mce_is_correctable); |
| 587 | 588 | ||
| 588 | /* | ||
| 589 | * Notify the user(s) about new machine check events. | ||
| 590 | * Can be called from interrupt context, but not from machine check/NMI | ||
| 591 | * context. | ||
| 592 | */ | ||
| 593 | static bool mce_notify_irq(void) | ||
| 594 | { | ||
| 595 | /* Not more than two messages every minute */ | ||
| 596 | static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2); | ||
| 597 | |||
| 598 | if (test_and_clear_bit(0, &mce_need_notify)) { | ||
| 599 | mce_work_trigger(); | ||
| 600 | |||
| 601 | if (__ratelimit(&ratelimit)) | ||
| 602 | pr_info(HW_ERR "Machine check events logged\n"); | ||
| 603 | |||
| 604 | return true; | ||
| 605 | } | ||
| 606 | |||
| 607 | return false; | ||
| 608 | } | ||
| 609 | |||
| 610 | static int mce_early_notifier(struct notifier_block *nb, unsigned long val, | 589 | static int mce_early_notifier(struct notifier_block *nb, unsigned long val, |
| 611 | void *data) | 590 | void *data) |
| 612 | { | 591 | { |
| @@ -618,9 +597,7 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val, · 5 unmodified lines | |||
| 618 | /* Emit the trace record: */ | 597 | /* Emit the trace record: */ |
| 619 | trace_mce_record(err); | 598 | trace_mce_record(err); |
| 620 | 599 | ||
| 621 | set_bit(0, &mce_need_notify); | 600 | mce_work_trigger(); |
| 622 | |||
| 623 | mce_notify_irq(); | ||
| 624 | 601 | ||
| 625 | return NOTIFY_DONE; | 602 | return NOTIFY_DONE; |
| 626 | } | 603 | } |
| @@ -1804,7 +1781,7 @@ static void mce_timer_fn(struct timer_list *t) · 1177 unmodified lines | |||
| 1804 | * Alert userspace if needed. If we logged an MCE, reduce the polling | 1781 | * Alert userspace if needed. If we logged an MCE, reduce the polling |
| 1805 | * interval, otherwise increase the polling interval. | 1782 | * interval, otherwise increase the polling interval. |
| 1806 | */ | 1783 | */ |
| 1807 | if (mce_notify_irq()) | 1784 | if (!mce_gen_pool_empty()) |
| 1808 | iv = max(iv / 2, (unsigned long) HZ/100); | 1785 | iv = max(iv / 2, (unsigned long) HZ/100); |
| 1809 | else | 1786 | else |
| 1810 | iv = min(iv * 2, round_jiffies_relative(check_interval * HZ)); | 1787 | iv = min(iv * 2, round_jiffies_relative(check_interval * HZ)); |
| @@ -322,7 +322,7 @@ static u32 get_patch_level(void) · 321 unmodified lines | |||
| 322 | { | 322 | { |
| 323 | u32 rev, dummy __always_unused; | 323 | u32 rev, dummy __always_unused; |
| 324 | 324 | ||
| 325 | if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) { | 325 | if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) { |
| 326 | int cpu = smp_processor_id(); | 326 | int cpu = smp_processor_id(); |
| 327 | 327 | ||
| 328 | if (!microcode_rev[cpu]) { | 328 | if (!microcode_rev[cpu]) { |
| @@ -714,7 +714,7 @@ static bool __apply_microcode_amd(struct microcode_amd *mc, u32 *cur_rev, · 385 unmodified lines | |||
| 714 | invlpg(p_addr_end); | 714 | invlpg(p_addr_end); |
| 715 | } | 715 | } |
| 716 | 716 | ||
| 717 | if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) | 717 | if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) |
| 718 | microcode_rev[smp_processor_id()] = mc->hdr.patch_id; | 718 | microcode_rev[smp_processor_id()] = mc->hdr.patch_id; |
| 719 | 719 | ||
| 720 | /* verify patch application was successful */ | 720 | /* verify patch application was successful */ |
| @@ -57,7 +57,7 @@ bool force_minrev = IS_ENABLED(CONFIG_MICROCODE_LATE_FORCE_MINREV); · 56 unmodified lines | |||
| 57 | u32 base_rev; | 57 | u32 base_rev; |
| 58 | u32 microcode_rev[NR_CPUS] = {}; | 58 | u32 microcode_rev[NR_CPUS] = {}; |
| 59 | 59 | ||
| 60 | bool hypervisor_present; | 60 | bool __ro_after_init x86_hypervisor_present; |
| 61 | 61 | ||
| 62 | /* | 62 | /* |
| 63 | * Synchronization. | 63 | * Synchronization. |
| @@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void) · 54 unmodified lines | |||
| 118 | /* | 118 | /* |
| 119 | * Disable when: | 119 | * Disable when: |
| 120 | * | 120 | * |
| 121 | * 1) The CPU does not support CPUID. | 121 | * 1) The CPU does not support CPUID, detected below in |
| 122 | */ | 122 | * load_ucode_bsp(). |
| 123 | if (!cpuid_feature()) { | 123 | * |
| 124 | dis_ucode_ldr = true; | ||
| 125 | return dis_ucode_ldr; | ||
| 126 | } | ||
| 127 | |||
| 128 | /* | ||
| 129 | * 2) Bit 31 in CPUID[1]:ECX is clear | 124 | * 2) Bit 31 in CPUID[1]:ECX is clear |
| 130 | * The bit is reserved for hypervisor use. This is still not | 125 | * The bit is reserved for hypervisor use. This is still not |
| 131 | * completely accurate as XEN PV guests don't see that CPUID bit | 126 | * completely accurate as XEN PV guests don't see that CPUID bit |
| @@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void) · 3 unmodified lines | |||
| 135 | * 3) Certain AMD patch levels are not allowed to be | 130 | * 3) Certain AMD patch levels are not allowed to be |
| 136 | * overwritten. | 131 | * overwritten. |
| 137 | */ | 132 | */ |
| 138 | hypervisor_present = native_cpuid_ecx(1) & BIT(31); | 133 | if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) || |
| 139 | |||
| 140 | if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) || | ||
| 141 | amd_check_current_patch_level()) | 134 | amd_check_current_patch_level()) |
| 142 | dis_ucode_ldr = true; | 135 | dis_ucode_ldr = true; |
| 143 | 136 | ||
| @@ -179,6 +172,11 @@ void __init load_ucode_bsp(void) · 35 unmodified lines | |||
| 179 | 172 | ||
| 180 | early_parse_cmdline(); | 173 | early_parse_cmdline(); |
| 181 | 174 | ||
| 175 | if (!cpuid_feature()) | ||
| 176 | dis_ucode_ldr = true; | ||
| 177 | else | ||
| 178 | x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31); | ||
| 179 | |||
| 182 | if (microcode_loader_disabled()) | 180 | if (microcode_loader_disabled()) |
| 183 | return; | 181 | return; |
| 184 | 182 | ||
| @@ -138,6 +138,9 @@ u32 intel_get_platform_id(void) · 137 unmodified lines | |||
| 138 | { | 138 | { |
| 139 | unsigned int val[2]; | 139 | unsigned int val[2]; |
| 140 | 140 | ||
| 141 | if (x86_hypervisor_present) | ||
| 142 | return 0; | ||
| 143 | |||
| 141 | /* | 144 | /* |
| 142 | * This can be called early. Use CPUID directly instead of | 145 | * This can be called early. Use CPUID directly instead of |
| 143 | * relying on cpuinfo_x86 which may not be fully initialized. | 146 | * relying on cpuinfo_x86 which may not be fully initialized. |
| @@ -48,7 +48,6 @@ extern struct early_load_data early_data; · 47 unmodified lines | |||
| 48 | extern struct ucode_cpu_info ucode_cpu_info[]; | 48 | extern struct ucode_cpu_info ucode_cpu_info[]; |
| 49 | extern u32 microcode_rev[NR_CPUS]; | 49 | extern u32 microcode_rev[NR_CPUS]; |
| 50 | extern u32 base_rev; | 50 | extern u32 base_rev; |
| 51 | extern bool hypervisor_present; | ||
| 52 | 51 | ||
| 53 | struct cpio_data find_microcode_in_initrd(const char *path); | 52 | struct cpio_data find_microcode_in_initrd(const char *path); |
| 54 | 53 | ||
| @@ -450,6 +450,10 @@ __init static int append_e820_table(struct boot_e820_entry *entries, u32 nr_entr · 449 unmodified lines | |||
| 450 | { | 450 | { |
| 451 | struct boot_e820_entry *entry = entries; | 451 | struct boot_e820_entry *entry = entries; |
| 452 | 452 | ||
| 453 | /* If there aren't any entries, we'll want to fall back to another source: */ | ||
| 454 | if (!nr_entries) | ||
| 455 | return -ENOENT; | ||
| 456 | |||
| 453 | while (nr_entries) { | 457 | while (nr_entries) { |
| 454 | u64 start = entry->addr; | 458 | u64 start = entry->addr; |
| 455 | u64 size = entry->size; | 459 | u64 size = entry->size; |
| @@ -458,7 +462,7 @@ __init static int append_e820_table(struct boot_e820_entry *entries, u32 nr_entr · 2 unmodified lines | |||
| 458 | 462 | ||
| 459 | /* Ignore the remaining entries on 64-bit overflow: */ | 463 | /* Ignore the remaining entries on 64-bit overflow: */ |
| 460 | if (start > end && likely(size)) | 464 | if (start > end && likely(size)) |
| 461 | return -1; | 465 | return -EINVAL; |
| 462 | 466 | ||
| 463 | e820__range_add(start, size, type); | 467 | e820__range_add(start, size, type); |
| 464 | 468 | ||
| @@ -27,14 +27,19 @@ · 26 unmodified lines | |||
| 27 | static inline bool check_xstate_in_sigframe(struct fxregs_state __user *fxbuf, | 27 | static inline bool check_xstate_in_sigframe(struct fxregs_state __user *fxbuf, |
| 28 | struct _fpx_sw_bytes *fx_sw) | 28 | struct _fpx_sw_bytes *fx_sw) |
| 29 | { | 29 | { |
| 30 | int min_xstate_size = sizeof(struct fxregs_state) + | ||
| 31 | sizeof(struct xstate_header); | ||
| 30 | void __user *fpstate = fxbuf; | 32 | void __user *fpstate = fxbuf; |
| 31 | unsigned int magic2; | 33 | unsigned int magic2; |
| 32 | 34 | ||
| 33 | if (__copy_from_user(fx_sw, &fxbuf->sw_reserved[0], sizeof(*fx_sw))) | 35 | if (__copy_from_user(fx_sw, &fxbuf->sw_reserved[0], sizeof(*fx_sw))) |
| 34 | return false; | 36 | return false; |
| 35 | 37 | ||
| 36 | /* Check for the first magic field */ | 38 | /* Check for the first magic field and other error scenarios. */ |
| 37 | if (fx_sw->magic1 != FP_XSTATE_MAGIC1) | 39 | if (fx_sw->magic1 != FP_XSTATE_MAGIC1 || |
| 40 | fx_sw->xstate_size < min_xstate_size || | ||
| 41 | fx_sw->xstate_size > x86_task_fpu(current)->fpstate->user_size || | ||
| 42 | fx_sw->xstate_size > fx_sw->extended_size) | ||
| 38 | goto setfx; | 43 | goto setfx; |
| 39 | 44 | ||
| 40 | /* | 45 | /* |
| @@ -43,7 +48,7 @@ static inline bool check_xstate_in_sigframe(struct fxregs_state __user *fxbuf, · 2 unmodified lines | |||
| 43 | * fpstate layout with out copying the extended state information | 48 | * fpstate layout with out copying the extended state information |
| 44 | * in the memory layout. | 49 | * in the memory layout. |
| 45 | */ | 50 | */ |
| 46 | if (__get_user(magic2, (__u32 __user *)(fpstate + x86_task_fpu(current)->fpstate->user_size))) | 51 | if (__get_user(magic2, (__u32 __user *)(fpstate + fx_sw->xstate_size))) |
| 47 | return false; | 52 | return false; |
| 48 | 53 | ||
| 49 | if (likely(magic2 == FP_XSTATE_MAGIC2)) | 54 | if (likely(magic2 == FP_XSTATE_MAGIC2)) |
| @@ -375,6 +375,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) · 374 unmodified lines | |||
| 375 | goto fail; | 375 | goto fail; |
| 376 | } | 376 | } |
| 377 | 377 | ||
| 378 | /* | ||
| 379 | * Generated trampoline may contain rIP-relative addressing which | ||
| 380 | * displacement needs to be fixed. | ||
| 381 | */ | ||
| 382 | text_poke_apply_relocation(trampoline, trampoline, size, | ||
| 383 | (void *)start_offset, size); | ||
| 384 | |||
| 378 | /* | 385 | /* |
| 379 | * The address of the ftrace_ops that is used for this trampoline | 386 | * The address of the ftrace_ops that is used for this trampoline |
| 380 | * is stored at the end of the trampoline. This will be used to | 387 | * is stored at the end of the trampoline. This will be used to |
| @@ -268,6 +268,13 @@ void __init idt_setup_early_pf(void) · 267 unmodified lines | |||
| 268 | } | 268 | } |
| 269 | #endif | 269 | #endif |
| 270 | 270 | ||
| 271 | #if IS_ENABLED(CONFIG_KVM_INTEL) && !defined(CONFIG_X86_64) | ||
| 272 | void idt_entry_from_kvm(unsigned int vector) | ||
| 273 | { | ||
| 274 | idt_do_interrupt_irqoff(gate_offset(idt_table + vector)); | ||
| 275 | } | ||
| 276 | #endif | ||
| 277 | |||
| 271 | static void __init idt_map_in_cea(void) | 278 | static void __init idt_map_in_cea(void) |
| 272 | { | 279 | { |
| 273 | /* | 280 | /* |
| @@ -614,7 +614,6 @@ DEFINE_IDTENTRY_RAW(exc_nmi_kvm_vmx) · 613 unmodified lines | |||
| 614 | { | 614 | { |
| 615 | exc_nmi(regs); | 615 | exc_nmi(regs); |
| 616 | } | 616 | } |
| 617 | EXPORT_SYMBOL_FOR_KVM(asm_exc_nmi_kvm_vmx); | ||
| 618 | #endif | 617 | #endif |
| 619 | 618 | ||
| 620 | #ifdef CONFIG_NMI_CHECK_CPU | 619 | #ifdef CONFIG_NMI_CHECK_CPU |
| @@ -136,6 +136,14 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) · 135 unmodified lines | |||
| 136 | * %r13 original CR4 when relocate_kernel() was invoked | 136 | * %r13 original CR4 when relocate_kernel() was invoked |
| 137 | */ | 137 | */ |
| 138 | 138 | ||
| 139 | /* | ||
| 140 | * Set return address to 0 if not preserving context. The purgatory | ||
| 141 | * shipped in kexec-tools will unconditionally look for the return | ||
| 142 | * address on the stack and set a kexec_jump_back_entry= command | ||
| 143 | * line option if it's non-zero. There's no other way that it can | ||
| 144 | * tell a preserve-context (kjump) kexec from a normal one. | ||
| 145 | */ | ||
| 146 | pushq $0 | ||
| 139 | /* store the start address on the stack */ | 147 | /* store the start address on the stack */ |
| 140 | pushq %rdx | 148 | pushq %rdx |
| 141 | 149 | ||
| @@ -4481,7 +4481,7 @@ static const struct opcode opcode_map_0f_38[256] = { · 4480 unmodified lines | |||
| 4481 | X16(N), X16(N), | 4481 | X16(N), X16(N), |
| 4482 | /* 0x20 - 0x2f */ | 4482 | /* 0x20 - 0x2f */ |
| 4483 | X8(N), | 4483 | X8(N), |
| 4484 | X2(N), GP(SrcReg | DstMem | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N, | 4484 | X2(N), GP(SrcMem | DstReg | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N, |
| 4485 | /* 0x30 - 0x7f */ | 4485 | /* 0x30 - 0x7f */ |
| 4486 | X16(N), X16(N), X16(N), X16(N), X16(N), | 4486 | X16(N), X16(N), X16(N), X16(N), X16(N), |
| 4487 | /* 0x80 - 0xef */ | 4487 | /* 0x80 - 0xef */ |
| @@ -2040,7 +2040,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · 2039 unmodified lines | |||
| 2040 | * flush). Translate the address here so the memory can be uniformly | 2040 | * flush). Translate the address here so the memory can be uniformly |
| 2041 | * read with kvm_read_guest(). | 2041 | * read with kvm_read_guest(). |
| 2042 | */ | 2042 | */ |
| 2043 | if (!hc->fast && is_guest_mode(vcpu)) { | 2043 | if (!hc->fast && mmu_is_nested(vcpu)) { |
| 2044 | hc->ingpa = translate_nested_gpa(vcpu, hc->ingpa, 0, NULL); | 2044 | hc->ingpa = translate_nested_gpa(vcpu, hc->ingpa, 0, NULL); |
| 2045 | if (unlikely(hc->ingpa == INVALID_GPA)) | 2045 | if (unlikely(hc->ingpa == INVALID_GPA)) |
| 2046 | return HV_STATUS_INVALID_HYPERCALL_INPUT; | 2046 | return HV_STATUS_INVALID_HYPERCALL_INPUT; |
| @@ -667,13 +667,15 @@ bool __kvm_apic_update_irr(unsigned long *pir, void *regs, int *max_irr) · 666 unmodified lines | |||
| 667 | u32 *__pir = (void *)pir_vals; | 667 | u32 *__pir = (void *)pir_vals; |
| 668 | u32 i, vec; | 668 | u32 i, vec; |
| 669 | u32 irr_val, prev_irr_val; | 669 | u32 irr_val, prev_irr_val; |
| 670 | int max_updated_irr; | 670 | int max_new_irr; |
| 671 | 671 | ||
| 672 | max_updated_irr = -1; | 672 | if (!pi_harvest_pir(pir, pir_vals)) { |
| 673 | *max_irr = -1; | 673 | *max_irr = apic_find_highest_vector(regs + APIC_IRR); |
| 674 | |||
| 675 | if (!pi_harvest_pir(pir, pir_vals)) | ||
| 676 | return false; | 674 | return false; |
| 675 | } | ||
| 676 | |||
| 677 | max_new_irr = -1; | ||
| 678 | *max_irr = -1; | ||
| 677 | 679 | ||
| 678 | for (i = vec = 0; i <= 7; i++, vec += 32) { | 680 | for (i = vec = 0; i <= 7; i++, vec += 32) { |
| 679 | u32 *p_irr = (u32 *)(regs + APIC_IRR + i * 0x10); | 681 | u32 *p_irr = (u32 *)(regs + APIC_IRR + i * 0x10); |
| @@ -688,25 +690,25 @@ bool __kvm_apic_update_irr(unsigned long *pir, void *regs, int *max_irr) · 8 unmodified lines | |||
| 688 | !try_cmpxchg(p_irr, &prev_irr_val, irr_val)); | 690 | !try_cmpxchg(p_irr, &prev_irr_val, irr_val)); |
| 689 | 691 | ||
| 690 | if (prev_irr_val != irr_val) | 692 | if (prev_irr_val != irr_val) |
| 691 | max_updated_irr = __fls(irr_val ^ prev_irr_val) + vec; | 693 | max_new_irr = __fls(irr_val ^ prev_irr_val) + vec; |
| 692 | } | 694 | } |
| 693 | if (irr_val) | 695 | if (irr_val) |
| 694 | *max_irr = __fls(irr_val) + vec; | 696 | *max_irr = __fls(irr_val) + vec; |
| 695 | } | 697 | } |
| 696 | 698 | ||
| 697 | return ((max_updated_irr != -1) && | 699 | return max_new_irr != -1 && max_new_irr == *max_irr; |
| 698 | (max_updated_irr == *max_irr)); | ||
| 699 | } | 700 | } |
| 700 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(__kvm_apic_update_irr); | 701 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(__kvm_apic_update_irr); |
| 701 | 702 | ||
| 702 | bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, unsigned long *pir, int *max_irr) | 703 | bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, unsigned long *pir, int *max_irr) |
| 703 | { | 704 | { |
| 704 | struct kvm_lapic *apic = vcpu->arch.apic; | 705 | struct kvm_lapic *apic = vcpu->arch.apic; |
| 705 | bool irr_updated = __kvm_apic_update_irr(pir, apic->regs, max_irr); | 706 | bool max_irr_is_from_pir; |
| 706 | 707 | ||
| 707 | if (unlikely(!apic->apicv_active && irr_updated)) | 708 | max_irr_is_from_pir = __kvm_apic_update_irr(pir, apic->regs, max_irr); |
| 709 | if (unlikely(!apic->apicv_active && max_irr_is_from_pir)) | ||
| 708 | apic->irr_pending = true; | 710 | apic->irr_pending = true; |
| 709 | return irr_updated; | 711 | return max_irr_is_from_pir; |
| 710 | } | 712 | } |
| 711 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_apic_update_irr); | 713 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_apic_update_irr); |
| 712 | 714 | ||
| @@ -182,6 +182,8 @@ static struct kmem_cache *pte_list_desc_cache; · 181 unmodified lines | |||
| 182 | struct kmem_cache *mmu_page_header_cache; | 182 | struct kmem_cache *mmu_page_header_cache; |
| 183 | 183 | ||
| 184 | static void mmu_spte_set(u64 *sptep, u64 spte); | 184 | static void mmu_spte_set(u64 *sptep, u64 spte); |
| 185 | static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp, | ||
| 186 | u64 *spte, struct list_head *invalid_list); | ||
| 185 | 187 | ||
| 186 | struct kvm_mmu_role_regs { | 188 | struct kvm_mmu_role_regs { |
| 187 | const unsigned long cr0; | 189 | const unsigned long cr0; |
| @@ -1287,19 +1289,6 @@ static void drop_spte(struct kvm *kvm, u64 *sptep) · 1099 unmodified lines | |||
| 1287 | rmap_remove(kvm, sptep); | 1289 | rmap_remove(kvm, sptep); |
| 1288 | } | 1290 | } |
| 1289 | 1291 | ||
| 1290 | static void drop_large_spte(struct kvm *kvm, u64 *sptep, bool flush) | ||
| 1291 | { | ||
| 1292 | struct kvm_mmu_page *sp; | ||
| 1293 | |||
| 1294 | sp = sptep_to_sp(sptep); | ||
| 1295 | WARN_ON_ONCE(sp->role.level == PG_LEVEL_4K); | ||
| 1296 | |||
| 1297 | drop_spte(kvm, sptep); | ||
| 1298 | |||
| 1299 | if (flush) | ||
| 1300 | kvm_flush_remote_tlbs_sptep(kvm, sptep); | ||
| 1301 | } | ||
| 1302 | |||
| 1303 | /* | 1292 | /* |
| 1304 | * Write-protect on the specified @sptep, @pt_protect indicates whether | 1293 | * Write-protect on the specified @sptep, @pt_protect indicates whether |
| 1305 | * spte write-protection is caused by protecting shadow page table. | 1294 | * spte write-protection is caused by protecting shadow page table. |
| @@ -2466,7 +2455,8 @@ static struct kvm_mmu_page *kvm_mmu_get_child_sp(struct kvm_vcpu *vcpu, · 1160 unmodified lines | |||
| 2466 | { | 2455 | { |
| 2467 | union kvm_mmu_page_role role; | 2456 | union kvm_mmu_page_role role; |
| 2468 | 2457 | ||
| 2469 | if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep)) | 2458 | if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep) && |
| 2459 | spte_to_child_sp(*sptep) && spte_to_child_sp(*sptep)->gfn == gfn) | ||
| 2470 | return ERR_PTR(-EEXIST); | 2460 | return ERR_PTR(-EEXIST); |
| 2471 | 2461 | ||
| 2472 | role = kvm_mmu_child_role(sptep, direct, access); | 2462 | role = kvm_mmu_child_role(sptep, direct, access); |
| @@ -2536,6 +2526,23 @@ static void shadow_walk_next(struct kvm_shadow_walk_iterator *iterator) · 63 unmodified lines | |||
| 2536 | __shadow_walk_next(iterator, *iterator->sptep); | 2526 | __shadow_walk_next(iterator, *iterator->sptep); |
| 2537 | } | 2527 | } |
| 2538 | 2528 | ||
| 2529 | /* | ||
| 2530 | * Note: while normally KVM uses a "bool flush" return value to let | ||
| 2531 | * the caller batch flushes, __link_shadow_page() flushes immediately | ||
| 2532 | * before populating the parent PTE with the new shadow page. The | ||
| 2533 | * typical callers, direct_map() and FNAME(fetch)(), are not going | ||
| 2534 | * to zap more than one huge SPTE anyway. | ||
| 2535 | * | ||
| 2536 | * The only exception, where @flush can be false, is when a huge SPTE | ||
| 2537 | * is replaced with a shadow page SPTE with a fully populated page table, | ||
| 2538 | * which can happen from shadow_mmu_split_huge_page(). In this case, | ||
| 2539 | * no memory is unmapped across the change to the page tables and no | ||
| 2540 | * immediate flush is needed for correctness. | ||
| 2541 | * | ||
| 2542 | * Even in that case, calls to kvm_mmu_commit_zap_page() are not | ||
| 2543 | * batched. Doing so would require adding an invalid_list argument | ||
| 2544 | * all the way down to __walk_slot_rmaps(). | ||
| 2545 | */ | ||
| 2539 | static void __link_shadow_page(struct kvm *kvm, | 2546 | static void __link_shadow_page(struct kvm *kvm, |
| 2540 | struct kvm_mmu_memory_cache *cache, u64 *sptep, | 2547 | struct kvm_mmu_memory_cache *cache, u64 *sptep, |
| 2541 | struct kvm_mmu_page *sp, bool flush) | 2548 | struct kvm_mmu_page *sp, bool flush) |
| @@ -2544,13 +2551,18 @@ static void __link_shadow_page(struct kvm *kvm, · 2 unmodified lines | |||
| 2544 | 2551 | ||
| 2545 | BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK); | 2552 | BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK); |
| 2546 | 2553 | ||
| 2547 | /* | 2554 | if (is_shadow_present_pte(*sptep)) { |
| 2548 | * If an SPTE is present already, it must be a leaf and therefore | 2555 | struct kvm_mmu_page *parent_sp; |
| 2549 | * a large one. Drop it, and flush the TLB if needed, before | 2556 | LIST_HEAD(invalid_list); |
| 2550 | * installing sp. | 2557 | |
| 2551 | */ | 2558 | parent_sp = sptep_to_sp(sptep); |
| 2552 | if (is_shadow_present_pte(*sptep)) | 2559 | WARN_ON_ONCE(parent_sp->role.level == PG_LEVEL_4K); |
| 2553 | drop_large_spte(kvm, sptep, flush); | 2560 | |
| 2561 | if (mmu_page_zap_pte(kvm, parent_sp, sptep, &invalid_list)) | ||
| 2562 | kvm_mmu_commit_zap_page(kvm, &invalid_list); | ||
| 2563 | else if (flush) | ||
| 2564 | kvm_flush_remote_tlbs_sptep(kvm, sptep); | ||
| 2565 | } | ||
| 2554 | 2566 | ||
| 2555 | spte = make_nonleaf_spte(sp->spt, sp_ad_disabled(sp)); | 2567 | spte = make_nonleaf_spte(sp->spt, sp_ad_disabled(sp)); |
| 2556 | 2568 | ||
| @@ -206,6 +206,35 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 205 unmodified lines | |||
| 206 | 206 | ||
| 207 | svm_clr_intercept(svm, INTERCEPT_CR8_WRITE); | 207 | svm_clr_intercept(svm, INTERCEPT_CR8_WRITE); |
| 208 | 208 | ||
| 209 | /* | ||
| 210 | * Flush the TLB when enabling (x2)AVIC and when transitioning between | ||
| 211 | * xAVIC and x2AVIC, as the CPU may have inserted a TLB entry for the | ||
| 212 | * "wrong" mapping. | ||
| 213 | * | ||
| 214 | * KVM uses a per-VM "scratch" page to back the APIC memslot, because | ||
| 215 | * KVM also uses per-VM page tables *and* maintains the page table (NPT | ||
| 216 | * or shadow page) mappings for said memslot even if one or more vCPUs | ||
| 217 | * have their local APIC hardware-disabled or are in x2APIC mode, i.e. | ||
| 218 | * even if one or more vCPUs' APIC MMIO BAR is effectively disabled. | ||
| 219 | * | ||
| 220 | * If xAVIC is fully enabled, hardware ignores the physical address in | ||
| 221 | * KVM's page tables, i.e. in the leaf SPTE for the APIC memslot, and | ||
| 222 | * instead redirects the access to the AVIC backing page, i.e. to the | ||
| 223 | * vCPU's virtual APIC page. If xAVIC is not enabled (APIC is either | ||
| 224 | * hardware-disabled or in x2APIC mode), then guest accesses will use | ||
| 225 | * the page table mapping verbatim, i.e. will access the per-VM scratch | ||
| 226 | * page, as normal memory. | ||
| 227 | * | ||
| 228 | * In both cases, the CPU is allowed to cache TLB entries for the APIC | ||
| 229 | * base GPA. So, KVM needs to flush the TLB when enabling xAVIC, as | ||
| 230 | * accesses need to be redirected to the virtual APIC page, but the TLB | ||
| 231 | * may contain entries pointing at the scratch page. KVM also needs to | ||
| 232 | * flush the TLB when enabling x2AVIC, as accesses need to go to the | ||
| 233 | * scratch page, but the TLB may contain entries tagged as xAVIC, i.e. | ||
| 234 | * entries pointing to the vCPU's virtual APIC page. | ||
| 235 | */ | ||
| 236 | kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu); | ||
| 237 | |||
| 209 | /* | 238 | /* |
| 210 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR | 239 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR |
| 211 | * accesses, while interrupt injection to a running vCPU can be | 240 | * accesses, while interrupt injection to a running vCPU can be |
| @@ -219,12 +248,6 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 7 unmodified lines | |||
| 219 | /* Disabling MSR intercept for x2APIC registers */ | 248 | /* Disabling MSR intercept for x2APIC registers */ |
| 220 | avic_set_x2apic_msr_interception(svm, false); | 249 | avic_set_x2apic_msr_interception(svm, false); |
| 221 | } else { | 250 | } else { |
| 222 | /* | ||
| 223 | * Flush the TLB, the guest may have inserted a non-APIC | ||
| 224 | * mapping into the TLB while AVIC was disabled. | ||
| 225 | */ | ||
| 226 | kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu); | ||
| 227 | |||
| 228 | /* Enabling MSR intercept for x2APIC registers */ | 251 | /* Enabling MSR intercept for x2APIC registers */ |
| 229 | avic_set_x2apic_msr_interception(svm, true); | 252 | avic_set_x2apic_msr_interception(svm, true); |
| 230 | } | 253 | } |
| @@ -1300,12 +1323,14 @@ bool __init avic_hardware_setup(void) · 1069 unmodified lines | |||
| 1300 | } | 1323 | } |
| 1301 | 1324 | ||
| 1302 | /* | 1325 | /* |
| 1303 | * Disable IPI virtualization for AMD Family 17h CPUs (Zen1 and Zen2) | 1326 | * Disable IPI virtualization for AMD Family 17h (Zen1 and Zen2) and |
| 1304 | * due to erratum 1235, which results in missed VM-Exits on the sender | 1327 | * Hygon Family 18h (derived from AMD Zen1) CPUs due to erratum 1235, |
| 1305 | * and thus missed wake events for blocking vCPUs due to the CPU | 1328 | * which results in missed VM-Exits on the sender and thus missed wake |
| 1306 | * failing to see a software update to clear IsRunning. | 1329 | * events for blocking vCPUs due to the CPU failing to see a software |
| 1330 | * update to clear IsRunning. | ||
| 1307 | */ | 1331 | */ |
| 1308 | enable_ipiv = enable_ipiv && boot_cpu_data.x86 != 0x17; | 1332 | if (boot_cpu_data.x86 == 0x17 || boot_cpu_data.x86 == 0x18) |
| 1333 | enable_ipiv = false; | ||
| 1309 | 1334 | ||
| 1310 | amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier); | 1335 | amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier); |
| 1311 | 1336 | ||
| @@ -160,6 +160,16 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm) · 159 unmodified lines | |||
| 160 | if (!intercept_smi) | 160 | if (!intercept_smi) |
| 161 | vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI); | 161 | vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI); |
| 162 | 162 | ||
| 163 | /* | ||
| 164 | * Intercept PAUSE if and only if L1 wants to. KVM intercepts PAUSE so | ||
| 165 | * that a vCPU that may be spinning waiting for a lock can be scheduled | ||
| 166 | * out in favor of the vCPU that holds said lock. KVM doesn't support | ||
| 167 | * yielding across L2 vCPUs, as KVM has limited visilibity into which | ||
| 168 | * L2 vCPUs are in the same L2 VM, i.e. may be contending for locks. | ||
| 169 | */ | ||
| 170 | if (!vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_PAUSE)) | ||
| 171 | vmcb_clr_intercept(&vmcb02->control, INTERCEPT_PAUSE); | ||
| 172 | |||
| 163 | if (nested_vmcb_needs_vls_intercept(svm)) { | 173 | if (nested_vmcb_needs_vls_intercept(svm)) { |
| 164 | /* | 174 | /* |
| 165 | * If the virtual VMLOAD/VMSAVE is not enabled for the L2, | 175 | * If the virtual VMLOAD/VMSAVE is not enabled for the L2, |
| @@ -819,7 +829,6 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) · 653 unmodified lines | |||
| 819 | struct vmcb *vmcb02 = svm->nested.vmcb02.ptr; | 829 | struct vmcb *vmcb02 = svm->nested.vmcb02.ptr; |
| 820 | struct vmcb *vmcb01 = svm->vmcb01.ptr; | 830 | struct vmcb *vmcb01 = svm->vmcb01.ptr; |
| 821 | struct kvm_vcpu *vcpu = &svm->vcpu; | 831 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 822 | u32 pause_count12, pause_thresh12; | ||
| 823 | 832 | ||
| 824 | nested_svm_transition_tlb_flush(vcpu); | 833 | nested_svm_transition_tlb_flush(vcpu); |
| 825 | 834 | ||
| @@ -947,31 +956,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) · 121 unmodified lines | |||
| 947 | vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE; | 956 | vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE; |
| 948 | 957 | ||
| 949 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER)) | 958 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER)) |
| 950 | pause_count12 = vmcb12_ctrl->pause_filter_count; | 959 | vmcb02->control.pause_filter_count = vmcb12_ctrl->pause_filter_count; |
| 951 | else | 960 | else |
| 952 | pause_count12 = 0; | 961 | vmcb02->control.pause_filter_count = 0; |
| 953 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD)) | 962 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD)) |
| 954 | pause_thresh12 = vmcb12_ctrl->pause_filter_thresh; | 963 | vmcb02->control.pause_filter_thresh = vmcb12_ctrl->pause_filter_thresh; |
| 955 | else | 964 | else |
| 956 | pause_thresh12 = 0; | 965 | vmcb02->control.pause_filter_thresh = 0; |
| 957 | if (kvm_pause_in_guest(svm->vcpu.kvm)) { | ||
| 958 | /* use guest values since host doesn't intercept PAUSE */ | ||
| 959 | vmcb02->control.pause_filter_count = pause_count12; | ||
| 960 | vmcb02->control.pause_filter_thresh = pause_thresh12; | ||
| 961 | |||
| 962 | } else { | ||
| 963 | /* start from host values otherwise */ | ||
| 964 | vmcb02->control.pause_filter_count = vmcb01->control.pause_filter_count; | ||
| 965 | vmcb02->control.pause_filter_thresh = vmcb01->control.pause_filter_thresh; | ||
| 966 | |||
| 967 | /* ... but ensure filtering is disabled if so requested. */ | ||
| 968 | if (vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_PAUSE)) { | ||
| 969 | if (!pause_count12) | ||
| 970 | vmcb02->control.pause_filter_count = 0; | ||
| 971 | if (!pause_thresh12) | ||
| 972 | vmcb02->control.pause_filter_thresh = 0; | ||
| 973 | } | ||
| 974 | } | ||
| 975 | 966 | ||
| 976 | /* | 967 | /* |
| 977 | * Take ALLOW_LARGER_RAP from vmcb12 even though it should be safe to | 968 | * Take ALLOW_LARGER_RAP from vmcb12 even though it should be safe to |
| @@ -1298,12 +1289,6 @@ void nested_svm_vmexit(struct vcpu_svm *svm) · 320 unmodified lines | |||
| 1298 | /* in case we halted in L2 */ | 1289 | /* in case we halted in L2 */ |
| 1299 | kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE); | 1290 | kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE); |
| 1300 | 1291 | ||
| 1301 | if (!kvm_pause_in_guest(vcpu->kvm)) { | ||
| 1302 | vmcb01->control.pause_filter_count = vmcb02->control.pause_filter_count; | ||
| 1303 | vmcb_mark_dirty(vmcb01, VMCB_INTERCEPTS); | ||
| 1304 | |||
| 1305 | } | ||
| 1306 | |||
| 1307 | /* | 1292 | /* |
| 1308 | * Invalidate last_bus_lock_rip unless KVM is still waiting for the | 1293 | * Invalidate last_bus_lock_rip unless KVM is still waiting for the |
| 1309 | * guest to make forward progress before re-enabling bus lock detection. | 1294 | * guest to make forward progress before re-enabling bus lock detection. |
| @@ -3662,23 +3662,26 @@ int pre_sev_run(struct vcpu_svm *svm, int cpu) · 3661 unmodified lines | |||
| 3662 | } | 3662 | } |
| 3663 | 3663 | ||
| 3664 | #define GHCB_SCRATCH_AREA_LIMIT (16ULL * PAGE_SIZE) | 3664 | #define GHCB_SCRATCH_AREA_LIMIT (16ULL * PAGE_SIZE) |
| 3665 | static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) | 3665 | static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 min_len) |
| 3666 | { | 3666 | { |
| 3667 | struct vmcb_control_area *control = &svm->vmcb->control; | 3667 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 3668 | u64 ghcb_scratch_beg, ghcb_scratch_end; | 3668 | u64 ghcb_scratch_beg, ghcb_scratch_end; |
| 3669 | u64 scratch_gpa_beg, scratch_gpa_end; | 3669 | u64 scratch_gpa_beg, scratch_gpa_end; |
| 3670 | void *scratch_va; | 3670 | void *scratch_va; |
| 3671 | 3671 | ||
| 3672 | if (WARN_ON_ONCE(!min_len)) | ||
| 3673 | goto e_scratch; | ||
| 3674 | |||
| 3672 | scratch_gpa_beg = svm->sev_es.sw_scratch; | 3675 | scratch_gpa_beg = svm->sev_es.sw_scratch; |
| 3673 | if (!scratch_gpa_beg) { | 3676 | if (!scratch_gpa_beg) { |
| 3674 | pr_err("vmgexit: scratch gpa not provided\n"); | 3677 | pr_err("vmgexit: scratch gpa not provided\n"); |
| 3675 | goto e_scratch; | 3678 | goto e_scratch; |
| 3676 | } | 3679 | } |
| 3677 | 3680 | ||
| 3678 | scratch_gpa_end = scratch_gpa_beg + len; | 3681 | scratch_gpa_end = scratch_gpa_beg + min_len; |
| 3679 | if (scratch_gpa_end < scratch_gpa_beg) { | 3682 | if (scratch_gpa_end < scratch_gpa_beg) { |
| 3680 | pr_err("vmgexit: scratch length (%#llx) not valid for scratch address (%#llx)\n", | 3683 | pr_err("vmgexit: scratch length (%#llx) not valid for scratch address (%#llx)\n", |
| 3681 | len, scratch_gpa_beg); | 3684 | min_len, scratch_gpa_beg); |
| 3682 | goto e_scratch; | 3685 | goto e_scratch; |
| 3683 | } | 3686 | } |
| 3684 | 3687 | ||
| @@ -3702,21 +3705,27 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) · 17 unmodified lines | |||
| 3702 | 3705 | ||
| 3703 | scratch_va = (void *)svm->sev_es.ghcb; | 3706 | scratch_va = (void *)svm->sev_es.ghcb; |
| 3704 | scratch_va += (scratch_gpa_beg - control->ghcb_gpa); | 3707 | scratch_va += (scratch_gpa_beg - control->ghcb_gpa); |
| 3708 | |||
| 3709 | svm->sev_es.ghcb_sa_len = ghcb_scratch_end - scratch_gpa_beg; | ||
| 3705 | } else { | 3710 | } else { |
| 3711 | /* GHCB v2 requires the scratch area to be within the GHCB. */ | ||
| 3712 | if (to_kvm_sev_info(svm->vcpu.kvm)->ghcb_version >= 2) | ||
| 3713 | goto e_scratch; | ||
| 3714 | |||
| 3706 | /* | 3715 | /* |
| 3707 | * The guest memory must be read into a kernel buffer, so | 3716 | * The guest memory must be read into a kernel buffer, so |
| 3708 | * limit the size | 3717 | * limit the size |
| 3709 | */ | 3718 | */ |
| 3710 | if (len > GHCB_SCRATCH_AREA_LIMIT) { | 3719 | if (min_len > GHCB_SCRATCH_AREA_LIMIT) { |
| 3711 | pr_err("vmgexit: scratch area exceeds KVM limits (%#llx requested, %#llx limit)\n", | 3720 | pr_err("vmgexit: scratch area exceeds KVM limits (%#llx requested, %#llx limit)\n", |
| 3712 | len, GHCB_SCRATCH_AREA_LIMIT); | 3721 | min_len, GHCB_SCRATCH_AREA_LIMIT); |
| 3713 | goto e_scratch; | 3722 | goto e_scratch; |
| 3714 | } | 3723 | } |
| 3715 | scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT); | 3724 | scratch_va = kvzalloc(min_len, GFP_KERNEL_ACCOUNT); |
| 3716 | if (!scratch_va) | 3725 | if (!scratch_va) |
| 3717 | return -ENOMEM; | 3726 | return -ENOMEM; |
| 3718 | 3727 | ||
| 3719 | if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, len)) { | 3728 | if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, min_len)) { |
| 3720 | /* Unable to copy scratch area from guest */ | 3729 | /* Unable to copy scratch area from guest */ |
| 3721 | pr_err("vmgexit: kvm_read_guest for scratch area failed\n"); | 3730 | pr_err("vmgexit: kvm_read_guest for scratch area failed\n"); |
| 3722 | 3731 | ||
| @@ -3732,11 +3741,10 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) · 9 unmodified lines | |||
| 3732 | */ | 3741 | */ |
| 3733 | svm->sev_es.ghcb_sa_sync = sync; | 3742 | svm->sev_es.ghcb_sa_sync = sync; |
| 3734 | svm->sev_es.ghcb_sa_free = true; | 3743 | svm->sev_es.ghcb_sa_free = true; |
| 3744 | svm->sev_es.ghcb_sa_len = min_len; | ||
| 3735 | } | 3745 | } |
| 3736 | 3746 | ||
| 3737 | svm->sev_es.ghcb_sa = scratch_va; | 3747 | svm->sev_es.ghcb_sa = scratch_va; |
| 3738 | svm->sev_es.ghcb_sa_len = len; | ||
| 3739 | |||
| 3740 | return 0; | 3748 | return 0; |
| 3741 | 3749 | ||
| 3742 | e_scratch: | 3750 | e_scratch: |
| @@ -3833,7 +3841,7 @@ struct psc_buffer { · 90 unmodified lines | |||
| 3833 | struct psc_entry entries[]; | 3841 | struct psc_entry entries[]; |
| 3834 | } __packed; | 3842 | } __packed; |
| 3835 | 3843 | ||
| 3836 | static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc); | 3844 | static int snp_begin_psc(struct vcpu_svm *svm); |
| 3837 | 3845 | ||
| 3838 | static void snp_complete_psc(struct vcpu_svm *svm, u64 psc_ret) | 3846 | static void snp_complete_psc(struct vcpu_svm *svm, u64 psc_ret) |
| 3839 | { | 3847 | { |
| @@ -3864,9 +3872,9 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm) · 24 unmodified lines | |||
| 3864 | */ | 3872 | */ |
| 3865 | for (idx = svm->sev_es.psc_idx; svm->sev_es.psc_inflight; | 3873 | for (idx = svm->sev_es.psc_idx; svm->sev_es.psc_inflight; |
| 3866 | svm->sev_es.psc_inflight--, idx++) { | 3874 | svm->sev_es.psc_inflight--, idx++) { |
| 3867 | struct psc_entry *entry = &entries[idx]; | 3875 | struct psc_entry entry = READ_ONCE(entries[idx]); |
| 3868 | 3876 | ||
| 3869 | entry->cur_page = entry->pagesize ? 512 : 1; | 3877 | entries[idx].cur_page = entry.pagesize ? 512 : 1; |
| 3870 | } | 3878 | } |
| 3871 | 3879 | ||
| 3872 | hdr->cur_entry = idx; | 3880 | hdr->cur_entry = idx; |
| @@ -3875,7 +3883,6 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm) · 2 unmodified lines | |||
| 3875 | static int snp_complete_one_psc(struct kvm_vcpu *vcpu) | 3883 | static int snp_complete_one_psc(struct kvm_vcpu *vcpu) |
| 3876 | { | 3884 | { |
| 3877 | struct vcpu_svm *svm = to_svm(vcpu); | 3885 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3878 | struct psc_buffer *psc = svm->sev_es.ghcb_sa; | ||
| 3879 | 3886 | ||
| 3880 | if (vcpu->run->hypercall.ret) { | 3887 | if (vcpu->run->hypercall.ret) { |
| 3881 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); | 3888 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); |
| @@ -3885,16 +3892,18 @@ static int snp_complete_one_psc(struct kvm_vcpu *vcpu) · 3 unmodified lines | |||
| 3885 | __snp_complete_one_psc(svm); | 3892 | __snp_complete_one_psc(svm); |
| 3886 | 3893 | ||
| 3887 | /* Handle the next range (if any). */ | 3894 | /* Handle the next range (if any). */ |
| 3888 | return snp_begin_psc(svm, psc); | 3895 | return snp_begin_psc(svm); |
| 3889 | } | 3896 | } |
| 3890 | 3897 | ||
| 3891 | static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc) | 3898 | static int snp_begin_psc(struct vcpu_svm *svm) |
| 3892 | { | 3899 | { |
| 3900 | struct vcpu_sev_es_state *sev_es = &svm->sev_es; | ||
| 3901 | struct psc_buffer *psc = sev_es->ghcb_sa; | ||
| 3893 | struct psc_entry *entries = psc->entries; | 3902 | struct psc_entry *entries = psc->entries; |
| 3894 | struct kvm_vcpu *vcpu = &svm->vcpu; | 3903 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 3895 | struct psc_hdr *hdr = &psc->hdr; | 3904 | struct psc_hdr *hdr = &psc->hdr; |
| 3896 | struct psc_entry entry_start; | 3905 | struct psc_entry entry_start; |
| 3897 | u16 idx, idx_start, idx_end; | 3906 | u16 idx, idx_start, idx_end, max_nr_entries; |
| 3898 | int npages; | 3907 | int npages; |
| 3899 | bool huge; | 3908 | bool huge; |
| 3900 | u64 gfn; | 3909 | u64 gfn; |
| @@ -3904,6 +3913,19 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc) · 3 unmodified lines | |||
| 3904 | return 1; | 3913 | return 1; |
| 3905 | } | 3914 | } |
| 3906 | 3915 | ||
| 3916 | /* | ||
| 3917 | * GHCB v2 requires the scratch area to reside within the GHCB itself, | ||
| 3918 | * and PSC requests are only supported for GHCB v2+. Thus it should be | ||
| 3919 | * impossible to exceed the max PSC entry count (which is derived from | ||
| 3920 | * the size of the shared GHCB buffer). | ||
| 3921 | */ | ||
| 3922 | max_nr_entries = (sev_es->ghcb_sa_len - sizeof(struct psc_hdr)) / | ||
| 3923 | sizeof(struct psc_entry); | ||
| 3924 | if (WARN_ON_ONCE(max_nr_entries > VMGEXIT_PSC_MAX_COUNT)) { | ||
| 3925 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); | ||
| 3926 | return 1; | ||
| 3927 | } | ||
| 3928 | |||
| 3907 | next_range: | 3929 | next_range: |
| 3908 | /* There should be no other PSCs in-flight at this point. */ | 3930 | /* There should be no other PSCs in-flight at this point. */ |
| 3909 | if (WARN_ON_ONCE(svm->sev_es.psc_inflight)) { | 3931 | if (WARN_ON_ONCE(svm->sev_es.psc_inflight)) { |
| @@ -3916,17 +3938,17 @@ next_range: · 6 unmodified lines | |||
| 3916 | * validation, so take care to only use validated copies of values used | 3938 | * validation, so take care to only use validated copies of values used |
| 3917 | * for things like array indexing. | 3939 | * for things like array indexing. |
| 3918 | */ | 3940 | */ |
| 3919 | idx_start = hdr->cur_entry; | 3941 | idx_start = READ_ONCE(hdr->cur_entry); |
| 3920 | idx_end = hdr->end_entry; | 3942 | idx_end = READ_ONCE(hdr->end_entry); |
| 3921 | 3943 | ||
| 3922 | if (idx_end >= VMGEXIT_PSC_MAX_COUNT) { | 3944 | if (idx_end >= max_nr_entries) { |
| 3923 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_INVALID_HDR); | 3945 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_INVALID_HDR); |
| 3924 | return 1; | 3946 | return 1; |
| 3925 | } | 3947 | } |
| 3926 | 3948 | ||
| 3927 | /* Find the start of the next range which needs processing. */ | 3949 | /* Find the start of the next range which needs processing. */ |
| 3928 | for (idx = idx_start; idx <= idx_end; idx++, hdr->cur_entry++) { | 3950 | for (idx = idx_start; idx <= idx_end; idx++, hdr->cur_entry++) { |
| 3929 | entry_start = entries[idx]; | 3951 | entry_start = READ_ONCE(entries[idx]); |
| 3930 | 3952 | ||
| 3931 | gfn = entry_start.gfn; | 3953 | gfn = entry_start.gfn; |
| 3932 | huge = entry_start.pagesize; | 3954 | huge = entry_start.pagesize; |
| @@ -3970,7 +3992,7 @@ next_range: · 37 unmodified lines | |||
| 3970 | * KVM_HC_MAP_GPA_RANGE exit. | 3992 | * KVM_HC_MAP_GPA_RANGE exit. |
| 3971 | */ | 3993 | */ |
| 3972 | while (++idx <= idx_end) { | 3994 | while (++idx <= idx_end) { |
| 3973 | struct psc_entry entry = entries[idx]; | 3995 | struct psc_entry entry = READ_ONCE(entries[idx]); |
| 3974 | 3996 | ||
| 3975 | if (entry.operation != entry_start.operation || | 3997 | if (entry.operation != entry_start.operation || |
| 3976 | entry.gfn != entry_start.gfn + npages || | 3998 | entry.gfn != entry_start.gfn + npages || |
| @@ -4493,13 +4515,22 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 516 unmodified lines | |||
| 4493 | case SVM_VMGEXIT_MMIO_READ: | 4515 | case SVM_VMGEXIT_MMIO_READ: |
| 4494 | case SVM_VMGEXIT_MMIO_WRITE: { | 4516 | case SVM_VMGEXIT_MMIO_WRITE: { |
| 4495 | bool is_write = control->exit_code == SVM_VMGEXIT_MMIO_WRITE; | 4517 | bool is_write = control->exit_code == SVM_VMGEXIT_MMIO_WRITE; |
| 4518 | u64 len = control->exit_info_2; | ||
| 4496 | 4519 | ||
| 4497 | ret = setup_vmgexit_scratch(svm, !is_write, control->exit_info_2); | 4520 | if (!len) |
| 4521 | return 1; | ||
| 4522 | |||
| 4523 | if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 && len > 8) { | ||
| 4524 | svm_vmgexit_bad_input(svm, GHCB_ERR_INVALID_INPUT); | ||
| 4525 | return 1; | ||
| 4526 | } | ||
| 4527 | |||
| 4528 | ret = setup_vmgexit_scratch(svm, !is_write, len); | ||
| 4498 | if (ret) | 4529 | if (ret) |
| 4499 | break; | 4530 | break; |
| 4500 | 4531 | ||
| 4501 | ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1, | 4532 | ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1, len, |
| 4502 | control->exit_info_2, svm->sev_es.ghcb_sa); | 4533 | svm->sev_es.ghcb_sa); |
| 4503 | break; | 4534 | break; |
| 4504 | } | 4535 | } |
| 4505 | case SVM_VMGEXIT_NMI_COMPLETE: | 4536 | case SVM_VMGEXIT_NMI_COMPLETE: |
| @@ -4546,11 +4577,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 40 unmodified lines | |||
| 4546 | vcpu->run->system_event.data[0] = control->ghcb_gpa; | 4577 | vcpu->run->system_event.data[0] = control->ghcb_gpa; |
| 4547 | break; | 4578 | break; |
| 4548 | case SVM_VMGEXIT_PSC: | 4579 | case SVM_VMGEXIT_PSC: |
| 4549 | ret = setup_vmgexit_scratch(svm, true, control->exit_info_2); | 4580 | ret = setup_vmgexit_scratch(svm, true, sizeof(struct psc_hdr)); |
| 4550 | if (ret) | 4581 | if (ret) |
| 4551 | break; | 4582 | break; |
| 4552 | 4583 | ||
| 4553 | ret = snp_begin_psc(svm, svm->sev_es.ghcb_sa); | 4584 | ret = snp_begin_psc(svm); |
| 4554 | break; | 4585 | break; |
| 4555 | case SVM_VMGEXIT_AP_CREATION: | 4586 | case SVM_VMGEXIT_AP_CREATION: |
| 4556 | ret = sev_snp_ap_creation(svm); | 4587 | ret = sev_snp_ap_creation(svm); |
| @@ -4572,6 +4603,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 15 unmodified lines | |||
| 4572 | control->exit_info_1, control->exit_info_2); | 4603 | control->exit_info_1, control->exit_info_2); |
| 4573 | ret = -EINVAL; | 4604 | ret = -EINVAL; |
| 4574 | break; | 4605 | break; |
| 4606 | case SVM_EXIT_IOIO: | ||
| 4607 | if (!((control->exit_info_1 & SVM_IOIO_SIZE_MASK) >> SVM_IOIO_SIZE_SHIFT)) | ||
| 4608 | return 1; | ||
| 4609 | |||
| 4610 | fallthrough; | ||
| 4575 | default: | 4611 | default: |
| 4576 | ret = svm_invoke_exit_handler(vcpu, control->exit_code); | 4612 | ret = svm_invoke_exit_handler(vcpu, control->exit_code); |
| 4577 | } | 4613 | } |
| @@ -4592,6 +4628,9 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in) · 14 unmodified lines | |||
| 4592 | if (unlikely(check_mul_overflow(count, size, &bytes))) | 4628 | if (unlikely(check_mul_overflow(count, size, &bytes))) |
| 4593 | return -EINVAL; | 4629 | return -EINVAL; |
| 4594 | 4630 | ||
| 4631 | if (!bytes) | ||
| 4632 | return 1; | ||
| 4633 | |||
| 4595 | r = setup_vmgexit_scratch(svm, in, bytes); | 4634 | r = setup_vmgexit_scratch(svm, in, bytes); |
| 4596 | if (r) | 4635 | if (r) |
| 4597 | return r; | 4636 | return r; |
| @@ -913,7 +913,15 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) · 912 unmodified lines | |||
| 913 | struct vmcb_control_area *control = &svm->vmcb->control; | 913 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 914 | int old = control->pause_filter_count; | 914 | int old = control->pause_filter_count; |
| 915 | 915 | ||
| 916 | if (kvm_pause_in_guest(vcpu->kvm)) | 916 | /* Adjusting pause_filter_count makes no sense if PLE is disabled. */ |
| 917 | WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm)); | ||
| 918 | |||
| 919 | /* | ||
| 920 | * While running L2, KVM should intercept PAUSE if and only if L1 wants | ||
| 921 | * to intercept PAUSE, and L1's intercept should take priority, i.e. | ||
| 922 | * KVM should never handle a PAUSE intercept from L2. | ||
| 923 | */ | ||
| 924 | if (WARN_ON_ONCE(is_guest_mode(vcpu))) | ||
| 917 | return; | 925 | return; |
| 918 | 926 | ||
| 919 | control->pause_filter_count = __grow_ple_window(old, | 927 | control->pause_filter_count = __grow_ple_window(old, |
| @@ -934,7 +942,10 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) · 14 unmodified lines | |||
| 934 | struct vmcb_control_area *control = &svm->vmcb->control; | 942 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 935 | int old = control->pause_filter_count; | 943 | int old = control->pause_filter_count; |
| 936 | 944 | ||
| 937 | if (kvm_pause_in_guest(vcpu->kvm)) | 945 | /* Adjusting pause_filter_count makes no sense if PLE is disabled. */ |
| 946 | WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm)); | ||
| 947 | |||
| 948 | if (is_guest_mode(vcpu)) | ||
| 938 | return; | 949 | return; |
| 939 | 950 | ||
| 940 | control->pause_filter_count = | 951 | control->pause_filter_count = |
| @@ -154,7 +154,7 @@ TRACE_EVENT(kvm_xen_hypercall, · 153 unmodified lines | |||
| 154 | __entry->a2 = a2; | 154 | __entry->a2 = a2; |
| 155 | __entry->a3 = a3; | 155 | __entry->a3 = a3; |
| 156 | __entry->a4 = a4; | 156 | __entry->a4 = a4; |
| 157 | __entry->a4 = a5; | 157 | __entry->a5 = a5; |
| 158 | ), | 158 | ), |
| 159 | 159 | ||
| 160 | TP_printk("cpl %d nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx a5 %lx", | 160 | TP_printk("cpl %d nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx a5 %lx", |
| @@ -14,6 +14,7 @@ extern bool __read_mostly flexpriority_enabled; · 13 unmodified lines | |||
| 14 | extern bool __read_mostly enable_ept; | 14 | extern bool __read_mostly enable_ept; |
| 15 | extern bool __read_mostly enable_unrestricted_guest; | 15 | extern bool __read_mostly enable_unrestricted_guest; |
| 16 | extern bool __read_mostly enable_ept_ad_bits; | 16 | extern bool __read_mostly enable_ept_ad_bits; |
| 17 | extern bool __read_mostly enable_cet; | ||
| 17 | extern bool __read_mostly enable_pml; | 18 | extern bool __read_mostly enable_pml; |
| 18 | extern int __read_mostly pt_mode; | 19 | extern int __read_mostly pt_mode; |
| 19 | 20 | ||
| @@ -31,38 +31,6 @@ · 30 unmodified lines | |||
| 31 | #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE | 31 | #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE |
| 32 | #endif | 32 | #endif |
| 33 | 33 | ||
| 34 | .macro VMX_DO_EVENT_IRQOFF call_insn call_target | ||
| 35 | /* | ||
| 36 | * Unconditionally create a stack frame, getting the correct RSP on the | ||
| 37 | * stack (for x86-64) would take two instructions anyways, and RBP can | ||
| 38 | * be used to restore RSP to make objtool happy (see below). | ||
| 39 | */ | ||
| 40 | push %_ASM_BP | ||
| 41 | mov %_ASM_SP, %_ASM_BP | ||
| 42 | |||
| 43 | #ifdef CONFIG_X86_64 | ||
| 44 | /* | ||
| 45 | * Align RSP to a 16-byte boundary (to emulate CPU behavior) before | ||
| 46 | * creating the synthetic interrupt stack frame for the IRQ/NMI. | ||
| 47 | */ | ||
| 48 | and $-16, %rsp | ||
| 49 | push $__KERNEL_DS | ||
| 50 | push %rbp | ||
| 51 | #endif | ||
| 52 | pushf | ||
| 53 | push $__KERNEL_CS | ||
| 54 | \call_insn \call_target | ||
| 55 | |||
| 56 | /* | ||
| 57 | * "Restore" RSP from RBP, even though IRET has already unwound RSP to | ||
| 58 | * the correct value. objtool doesn't know the callee will IRET and, | ||
| 59 | * without the explicit restore, thinks the stack is getting walloped. | ||
| 60 | * Using an unwind hint is problematic due to x86-64's dynamic alignment. | ||
| 61 | */ | ||
| 62 | leave | ||
| 63 | RET | ||
| 64 | .endm | ||
| 65 | |||
| 66 | .section .noinstr.text, "ax" | 34 | .section .noinstr.text, "ax" |
| 67 | 35 | ||
| 68 | /** | 36 | /** |
| @@ -320,10 +288,6 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) · 251 unmodified lines | |||
| 320 | 288 | ||
| 321 | SYM_FUNC_END(__vmx_vcpu_run) | 289 | SYM_FUNC_END(__vmx_vcpu_run) |
| 322 | 290 | ||
| 323 | SYM_FUNC_START(vmx_do_nmi_irqoff) | ||
| 324 | VMX_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx | ||
| 325 | SYM_FUNC_END(vmx_do_nmi_irqoff) | ||
| 326 | |||
| 327 | #ifndef CONFIG_CC_HAS_ASM_GOTO_OUTPUT | 291 | #ifndef CONFIG_CC_HAS_ASM_GOTO_OUTPUT |
| 328 | 292 | ||
| 329 | /** | 293 | /** |
| @@ -375,13 +339,3 @@ SYM_FUNC_START(vmread_error_trampoline) · 45 unmodified lines | |||
| 375 | RET | 339 | RET |
| 376 | SYM_FUNC_END(vmread_error_trampoline) | 340 | SYM_FUNC_END(vmread_error_trampoline) |
| 377 | #endif | 341 | #endif |
| 378 | |||
| 379 | .section .text, "ax" | ||
| 380 | |||
| 381 | #ifndef CONFIG_X86_FRED | ||
| 382 | |||
| 383 | SYM_FUNC_START(vmx_do_interrupt_irqoff) | ||
| 384 | VMX_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 | ||
| 385 | SYM_FUNC_END(vmx_do_interrupt_irqoff) | ||
| 386 | |||
| 387 | #endif | ||
| @@ -108,6 +108,9 @@ module_param_named(unrestricted_guest, · 107 unmodified lines | |||
| 108 | bool __read_mostly enable_ept_ad_bits = 1; | 108 | bool __read_mostly enable_ept_ad_bits = 1; |
| 109 | module_param_named(eptad, enable_ept_ad_bits, bool, 0444); | 109 | module_param_named(eptad, enable_ept_ad_bits, bool, 0444); |
| 110 | 110 | ||
| 111 | bool __read_mostly enable_cet = 1; | ||
| 112 | module_param_named(cet, enable_cet, bool, 0444); | ||
| 113 | |||
| 111 | static bool __read_mostly emulate_invalid_guest_state = true; | 114 | static bool __read_mostly emulate_invalid_guest_state = true; |
| 112 | module_param(emulate_invalid_guest_state, bool, 0444); | 115 | module_param(emulate_invalid_guest_state, bool, 0444); |
| 113 | 116 | ||
| @@ -4476,7 +4479,7 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) · 4362 unmodified lines | |||
| 4476 | * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter | 4479 | * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter |
| 4477 | * 3 and 4 for details. | 4480 | * 3 and 4 for details. |
| 4478 | */ | 4481 | */ |
| 4479 | if (cpu_has_load_cet_ctrl()) { | 4482 | if (enable_cet) { |
| 4480 | vmcs_writel(HOST_S_CET, kvm_host.s_cet); | 4483 | vmcs_writel(HOST_S_CET, kvm_host.s_cet); |
| 4481 | vmcs_writel(HOST_SSP, 0); | 4484 | vmcs_writel(HOST_SSP, 0); |
| 4482 | vmcs_writel(HOST_INTR_SSP_TABLE, 0); | 4485 | vmcs_writel(HOST_INTR_SSP_TABLE, 0); |
| @@ -4532,6 +4535,10 @@ static u32 vmx_get_initial_vmentry_ctrl(void) · 49 unmodified lines | |||
| 4532 | if (vmx_pt_mode_is_system()) | 4535 | if (vmx_pt_mode_is_system()) |
| 4533 | vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP | | 4536 | vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP | |
| 4534 | VM_ENTRY_LOAD_IA32_RTIT_CTL); | 4537 | VM_ENTRY_LOAD_IA32_RTIT_CTL); |
| 4538 | |||
| 4539 | if (!enable_cet) | ||
| 4540 | vmentry_ctrl &= ~VM_ENTRY_LOAD_CET_STATE; | ||
| 4541 | |||
| 4535 | /* | 4542 | /* |
| 4536 | * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically. | 4543 | * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically. |
| 4537 | */ | 4544 | */ |
| @@ -4546,6 +4553,9 @@ static u32 vmx_get_initial_vmexit_ctrl(void) · 8 unmodified lines | |||
| 4546 | { | 4553 | { |
| 4547 | u32 vmexit_ctrl = vmcs_config.vmexit_ctrl; | 4554 | u32 vmexit_ctrl = vmcs_config.vmexit_ctrl; |
| 4548 | 4555 | ||
| 4556 | if (!enable_cet) | ||
| 4557 | vmexit_ctrl &= ~VM_EXIT_LOAD_CET_STATE; | ||
| 4558 | |||
| 4549 | /* | 4559 | /* |
| 4550 | * Not used by KVM and never set in vmcs01 or vmcs02, but emulated for | 4560 | * Not used by KVM and never set in vmcs01 or vmcs02, but emulated for |
| 4551 | * nested virtualization and thus allowed to be set in vmcs12. | 4561 | * nested virtualization and thus allowed to be set in vmcs12. |
| @@ -7029,8 +7039,8 @@ static void vmx_set_rvi(int vector) · 2477 unmodified lines | |||
| 7029 | int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) | 7039 | int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) |
| 7030 | { | 7040 | { |
| 7031 | struct vcpu_vt *vt = to_vt(vcpu); | 7041 | struct vcpu_vt *vt = to_vt(vcpu); |
| 7042 | bool max_irr_is_from_pir; | ||
| 7032 | int max_irr; | 7043 | int max_irr; |
| 7033 | bool got_posted_interrupt; | ||
| 7034 | 7044 | ||
| 7035 | if (KVM_BUG_ON(!enable_apicv, vcpu->kvm)) | 7045 | if (KVM_BUG_ON(!enable_apicv, vcpu->kvm)) |
| 7036 | return -EIO; | 7046 | return -EIO; |
| @@ -7042,17 +7052,22 @@ int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) · 5 unmodified lines | |||
| 7042 | * But on x86 this is just a compiler barrier anyway. | 7052 | * But on x86 this is just a compiler barrier anyway. |
| 7043 | */ | 7053 | */ |
| 7044 | smp_mb__after_atomic(); | 7054 | smp_mb__after_atomic(); |
| 7045 | got_posted_interrupt = | 7055 | max_irr_is_from_pir = kvm_apic_update_irr(vcpu, vt->pi_desc.pir, |
| 7046 | kvm_apic_update_irr(vcpu, vt->pi_desc.pir, &max_irr); | 7056 | &max_irr); |
| 7047 | } else { | 7057 | } else { |
| 7048 | max_irr = kvm_lapic_find_highest_irr(vcpu); | 7058 | max_irr = kvm_lapic_find_highest_irr(vcpu); |
| 7049 | got_posted_interrupt = false; | 7059 | max_irr_is_from_pir = false; |
| 7050 | } | 7060 | } |
| 7051 | 7061 | ||
| 7052 | /* | 7062 | /* |
| 7053 | * Newly recognized interrupts are injected via either virtual interrupt | 7063 | * If APICv is enabled and L2 is not active, then update the Requesting |
| 7054 | * delivery (RVI) or KVM_REQ_EVENT. Virtual interrupt delivery is | 7064 | * Virtual Interrupt (RVI) portion of vmcs01.GUEST_INTR_STATUS with the |
| 7055 | * disabled in two cases: | 7065 | * highest priority IRR to deliver the IRQ via Virtual Interrupt |
| 7066 | * Delivery. Note, this is required even if the highest priority IRQ | ||
| 7067 | * was already pending in the IRR, as RVI isn't updated in lockstep with | ||
| 7068 | * the IRR (unlike apic->irr_pending). | ||
| 7069 | * | ||
| 7070 | * For the cases where Virtual Interrupt Delivery can't be used: | ||
| 7056 | * | 7071 | * |
| 7057 | * 1) If L2 is running and the vCPU has a new pending interrupt. If L1 | 7072 | * 1) If L2 is running and the vCPU has a new pending interrupt. If L1 |
| 7058 | * wants to exit on interrupts, KVM_REQ_EVENT is needed to synthesize a | 7073 | * wants to exit on interrupts, KVM_REQ_EVENT is needed to synthesize a |
| @@ -7063,10 +7078,29 @@ int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) · 4 unmodified lines | |||
| 7063 | * 2) If APICv is disabled for this vCPU, assigned devices may still | 7078 | * 2) If APICv is disabled for this vCPU, assigned devices may still |
| 7064 | * attempt to post interrupts. The posted interrupt vector will cause | 7079 | * attempt to post interrupts. The posted interrupt vector will cause |
| 7065 | * a VM-Exit and the subsequent entry will call sync_pir_to_irr. | 7080 | * a VM-Exit and the subsequent entry will call sync_pir_to_irr. |
| 7081 | * | ||
| 7082 | * In both cases, set KVM_REQ_EVENT if and only if the highest priority | ||
| 7083 | * pending IRQ came from the PIR, as setting KVM_REQ_EVENT if any IRQ | ||
| 7084 | * is pending may put the vCPU into an infinite loop, e.g. if the IRQ | ||
| 7085 | * is blocked, then it will stay pending until an IRQ window is opened. | ||
| 7086 | * | ||
| 7087 | * Note! It's possible that one or more IRQs were moved from the PIR | ||
| 7088 | * to the IRR _without_ max_irr_is_from_pir being true! I.e. if there | ||
| 7089 | * was a higher priority IRQ already pending in the IRR. Not setting | ||
| 7090 | * KVM_REQ_EVENT in this case is intentional and safe. If APICv is | ||
| 7091 | * inactive, or L2 is running with exit-on-interrupt off (in vmcs12), | ||
| 7092 | * i.e. without nested virtual interrupt delivery, then there's no need | ||
| 7093 | * to request an IRQ window as the lower priority IRQ only needs to be | ||
| 7094 | * delivered when the higher priority IRQ is dismissed from the ISR, | ||
| 7095 | * i.e. on the next EOI, and EOIs are always intercepted if APICv is | ||
| 7096 | * disabled or if L2 is running without nested VID. If L2 is running | ||
| 7097 | * exit-on-interrupt on (in vmcs12), then the higher priority IRQ will | ||
| 7098 | * trigger a nested VM-Exit, at which point KVM will re-evaluate L1's | ||
| 7099 | * pending IRQs. | ||
| 7066 | */ | 7100 | */ |
| 7067 | if (!is_guest_mode(vcpu) && kvm_vcpu_apicv_active(vcpu)) | 7101 | if (!is_guest_mode(vcpu) && kvm_vcpu_apicv_active(vcpu)) |
| 7068 | vmx_set_rvi(max_irr); | 7102 | vmx_set_rvi(max_irr); |
| 7069 | else if (got_posted_interrupt) | 7103 | else if (max_irr_is_from_pir) |
| 7070 | kvm_make_request(KVM_REQ_EVENT, vcpu); | 7104 | kvm_make_request(KVM_REQ_EVENT, vcpu); |
| 7071 | 7105 | ||
| 7072 | return max_irr; | 7106 | return max_irr; |
| @@ -7083,9 +7117,6 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) · 10 unmodified lines | |||
| 7083 | vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); | 7117 | vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); |
| 7084 | } | 7118 | } |
| 7085 | 7119 | ||
| 7086 | void vmx_do_interrupt_irqoff(unsigned long entry); | ||
| 7087 | void vmx_do_nmi_irqoff(void); | ||
| 7088 | |||
| 7089 | static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) | 7120 | static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) |
| 7090 | { | 7121 | { |
| 7091 | /* | 7122 | /* |
| @@ -7127,17 +7158,8 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, · 35 unmodified lines | |||
| 7127 | "unexpected VM-Exit interrupt info: 0x%x", intr_info)) | 7158 | "unexpected VM-Exit interrupt info: 0x%x", intr_info)) |
| 7128 | return; | 7159 | return; |
| 7129 | 7160 | ||
| 7130 | /* | ||
| 7131 | * Invoke the kernel's IRQ handler for the vector. Use the FRED path | ||
| 7132 | * when it's available even if FRED isn't fully enabled, e.g. even if | ||
| 7133 | * FRED isn't supported in hardware, in order to avoid the indirect | ||
| 7134 | * CALL in the non-FRED path. | ||
| 7135 | */ | ||
| 7136 | kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); | 7161 | kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); |
| 7137 | if (IS_ENABLED(CONFIG_X86_FRED)) | 7162 | x86_entry_from_kvm(EVENT_TYPE_EXTINT, vector); |
| 7138 | fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector); | ||
| 7139 | else | ||
| 7140 | vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector)); | ||
| 7141 | kvm_after_interrupt(vcpu); | 7163 | kvm_after_interrupt(vcpu); |
| 7142 | 7164 | ||
| 7143 | vcpu->arch.at_instruction_boundary = true; | 7165 | vcpu->arch.at_instruction_boundary = true; |
| @@ -7447,10 +7469,7 @@ noinstr void vmx_handle_nmi(struct kvm_vcpu *vcpu) · 303 unmodified lines | |||
| 7447 | return; | 7469 | return; |
| 7448 | 7470 | ||
| 7449 | kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); | 7471 | kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); |
| 7450 | if (cpu_feature_enabled(X86_FEATURE_FRED)) | 7472 | x86_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); |
| 7451 | fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); | ||
| 7452 | else | ||
| 7453 | vmx_do_nmi_irqoff(); | ||
| 7454 | kvm_after_interrupt(vcpu); | 7473 | kvm_after_interrupt(vcpu); |
| 7455 | } | 7474 | } |
| 7456 | 7475 | ||
| @@ -8131,7 +8150,7 @@ static __init void vmx_set_cpu_caps(void) · 674 unmodified lines | |||
| 8131 | * VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code | 8150 | * VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code |
| 8132 | * fails, so disable CET in this case too. | 8151 | * fails, so disable CET in this case too. |
| 8133 | */ | 8152 | */ |
| 8134 | if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest || | 8153 | if (!enable_cet || !enable_unrestricted_guest || |
| 8135 | !cpu_has_vmx_basic_no_hw_errcode_cc()) { | 8154 | !cpu_has_vmx_basic_no_hw_errcode_cc()) { |
| 8136 | kvm_cpu_cap_clear(X86_FEATURE_SHSTK); | 8155 | kvm_cpu_cap_clear(X86_FEATURE_SHSTK); |
| 8137 | kvm_cpu_cap_clear(X86_FEATURE_IBT); | 8156 | kvm_cpu_cap_clear(X86_FEATURE_IBT); |
| @@ -8606,6 +8625,9 @@ __init int vmx_hardware_setup(void) · 468 unmodified lines | |||
| 8606 | !cpu_has_vmx_invept_global()) | 8625 | !cpu_has_vmx_invept_global()) |
| 8607 | enable_ept = 0; | 8626 | enable_ept = 0; |
| 8608 | 8627 | ||
| 8628 | if (!cpu_has_load_cet_ctrl()) | ||
| 8629 | enable_cet = 0; | ||
| 8630 | |||
| 8609 | /* NX support is required for shadow paging. */ | 8631 | /* NX support is required for shadow paging. */ |
| 8610 | if (!enable_ept && !boot_cpu_has(X86_FEATURE_NX)) { | 8632 | if (!enable_ept && !boot_cpu_has(X86_FEATURE_NX)) { |
| 8611 | pr_err_ratelimited("NX (Execute Disable) not supported\n"); | 8633 | pr_err_ratelimited("NX (Execute Disable) not supported\n"); |
| @@ -4876,7 +4876,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 4875 unmodified lines | |||
| 4876 | r = tdp_enabled; | 4876 | r = tdp_enabled; |
| 4877 | break; | 4877 | break; |
| 4878 | case KVM_CAP_X86_APIC_BUS_CYCLES_NS: | 4878 | case KVM_CAP_X86_APIC_BUS_CYCLES_NS: |
| 4879 | r = APIC_BUS_CYCLE_NS_DEFAULT; | 4879 | r = kvm ? kvm->arch.apic_bus_cycle_ns : APIC_BUS_CYCLE_NS_DEFAULT; |
| 4880 | break; | 4880 | break; |
| 4881 | case KVM_CAP_EXIT_HYPERCALL: | 4881 | case KVM_CAP_EXIT_HYPERCALL: |
| 4882 | r = KVM_EXIT_HYPERCALL_VALID_MASK; | 4882 | r = KVM_EXIT_HYPERCALL_VALID_MASK; |
| @@ -5227,8 +5227,13 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · 344 unmodified lines | |||
| 5227 | * On a host with synchronized TSC, there is no need to update | 5227 | * On a host with synchronized TSC, there is no need to update |
| 5228 | * kvmclock on vcpu->cpu migration | 5228 | * kvmclock on vcpu->cpu migration |
| 5229 | */ | 5229 | */ |
| 5230 | if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1) | 5230 | if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1) { |
| 5231 | kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu); | 5231 | if (__ratelimit(&vcpu->kvm->arch.kvmclock_update_rs)) |
| 5232 | kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu); | ||
| 5233 | else | ||
| 5234 | kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu); | ||
| 5235 | } | ||
| 5236 | |||
| 5232 | if (vcpu->cpu != cpu) | 5237 | if (vcpu->cpu != cpu) |
| 5233 | kvm_make_request(KVM_REQ_MIGRATE_TIMER, vcpu); | 5238 | kvm_make_request(KVM_REQ_MIGRATE_TIMER, vcpu); |
| 5234 | vcpu->cpu = cpu; | 5239 | vcpu->cpu = cpu; |
| @@ -13366,6 +13371,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) · 8131 unmodified lines | |||
| 13366 | raw_spin_lock_init(&kvm->arch.tsc_write_lock); | 13371 | raw_spin_lock_init(&kvm->arch.tsc_write_lock); |
| 13367 | mutex_init(&kvm->arch.apic_map_lock); | 13372 | mutex_init(&kvm->arch.apic_map_lock); |
| 13368 | seqcount_raw_spinlock_init(&kvm->arch.pvclock_sc, &kvm->arch.tsc_write_lock); | 13373 | seqcount_raw_spinlock_init(&kvm->arch.pvclock_sc, &kvm->arch.tsc_write_lock); |
| 13374 | ratelimit_state_init(&kvm->arch.kvmclock_update_rs, HZ, 10); | ||
| 13375 | ratelimit_set_flags(&kvm->arch.kvmclock_update_rs, RATELIMIT_MSG_ON_RELEASE); | ||
| 13369 | kvm->arch.kvmclock_offset = -get_kvmclock_base_ns(); | 13376 | kvm->arch.kvmclock_offset = -get_kvmclock_base_ns(); |
| 13370 | 13377 | ||
| 13371 | raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); | 13378 | raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); |
| @@ -14323,7 +14330,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) · 951 unmodified lines | |||
| 14323 | * the RAP (Return Address Predicator). | 14330 | * the RAP (Return Address Predicator). |
| 14324 | */ | 14331 | */ |
| 14325 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) | 14332 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) |
| 14326 | kvm_register_is_dirty(vcpu, VCPU_EXREG_ERAPS); | 14333 | kvm_register_mark_dirty(vcpu, VCPU_EXREG_ERAPS); |
| 14327 | 14334 | ||
| 14328 | kvm_invalidate_pcid(vcpu, operand.pcid); | 14335 | kvm_invalidate_pcid(vcpu, operand.pcid); |
| 14329 | return kvm_skip_emulated_instruction(vcpu); | 14336 | return kvm_skip_emulated_instruction(vcpu); |
| @@ -686,7 +686,7 @@ page_fault_oops(struct pt_regs *regs, unsigned long error_code, · 685 unmodified lines | |||
| 686 | * avoid hanging the system. | 686 | * avoid hanging the system. |
| 687 | */ | 687 | */ |
| 688 | if (IS_ENABLED(CONFIG_EFI)) | 688 | if (IS_ENABLED(CONFIG_EFI)) |
| 689 | efi_crash_gracefully_on_page_fault(address); | 689 | efi_crash_gracefully_on_page_fault(address, regs); |
| 690 | 690 | ||
| 691 | /* Only not-present faults should be handled by KFENCE. */ | 691 | /* Only not-present faults should be handled by KFENCE. */ |
| 692 | if (!(error_code & X86_PF_PROT) && | 692 | if (!(error_code & X86_PF_PROT) && |
| @@ -761,7 +761,8 @@ int efi_capsule_setup_info(struct capsule_info *cap_info, void *kbuff, · 760 unmodified lines | |||
| 761 | * @return: Returns, if the page fault is not handled. This function | 761 | * @return: Returns, if the page fault is not handled. This function |
| 762 | * will never return if the page fault is handled successfully. | 762 | * will never return if the page fault is handled successfully. |
| 763 | */ | 763 | */ |
| 764 | void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) | 764 | void efi_crash_gracefully_on_page_fault(unsigned long phys_addr, |
| 765 | const struct pt_regs *regs) | ||
| 765 | { | 766 | { |
| 766 | if (!IS_ENABLED(CONFIG_X86_64)) | 767 | if (!IS_ENABLED(CONFIG_X86_64)) |
| 767 | return; | 768 | return; |
| @@ -770,7 +771,7 @@ void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) · 2 unmodified lines | |||
| 770 | * If we get an interrupt/NMI while processing an EFI runtime service | 771 | * If we get an interrupt/NMI while processing an EFI runtime service |
| 771 | * then this is a regular OOPS, not an EFI failure. | 772 | * then this is a regular OOPS, not an EFI failure. |
| 772 | */ | 773 | */ |
| 773 | if (in_interrupt()) | 774 | if (!in_task()) |
| 774 | return; | 775 | return; |
| 775 | 776 | ||
| 776 | /* | 777 | /* |
| @@ -810,6 +811,14 @@ void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) · 33 unmodified lines | |||
| 810 | return; | 811 | return; |
| 811 | } | 812 | } |
| 812 | 813 | ||
| 814 | /* | ||
| 815 | * The API does not permit entering a kernel mode FPU section with | ||
| 816 | * interrupts enabled and leaving it with interrupts disabled. So | ||
| 817 | * re-enable interrupts now if they were enabled when the page fault | ||
| 818 | * occurred. | ||
| 819 | */ | ||
| 820 | local_irq_restore(regs->flags); | ||
| 821 | |||
| 813 | /* | 822 | /* |
| 814 | * Before calling EFI Runtime Service, the kernel has switched the | 823 | * Before calling EFI Runtime Service, the kernel has switched the |
| 815 | * calling process to efi_mm. Hence, switch back to task_mm. | 824 | * calling process to efi_mm. Hence, switch back to task_mm. |
| @@ -49,7 +49,20 @@ static void x86_virt_invoke_kvm_emergency_callback(void) · 48 unmodified lines | |||
| 49 | { | 49 | { |
| 50 | cpu_emergency_virt_cb *kvm_callback; | 50 | cpu_emergency_virt_cb *kvm_callback; |
| 51 | 51 | ||
| 52 | kvm_callback = rcu_dereference(kvm_emergency_callback); | 52 | /* |
| 53 | * RCU may not be watching the crashing CPU here, so rcu_dereference() | ||
| 54 | * triggers a suspicious-RCU-usage splat. In principle, a concurrent | ||
| 55 | * KVM module unload could race with this read; see commit 2baa33a8ddd6 | ||
| 56 | * ("KVM: x86: Leave user-return notifier registered on reboot/shutdown") | ||
| 57 | * which notes that nothing prevents module unload during panic/reboot. | ||
| 58 | * | ||
| 59 | * However, taking a lock here would be riskier than the current race: | ||
| 60 | * the system is going down via NMI shootdown, and any lock could be | ||
| 61 | * held by an already-stopped CPU. Use rcu_dereference_raw() to silence | ||
| 62 | * the lockdep splat and accept the comically small remaining race; | ||
| 63 | * panic context inherently cannot guarantee complete correctness. | ||
| 64 | */ | ||
| 65 | kvm_callback = rcu_dereference_raw(kvm_emergency_callback); | ||
| 53 | if (kvm_callback) | 66 | if (kvm_callback) |
| 54 | kvm_callback(); | 67 | kvm_callback(); |
| 55 | } | 68 | } |
| @@ -2145,7 +2145,10 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) · 2144 unmodified lines | |||
| 2145 | 2145 | ||
| 2146 | static void xen_enter_lazy_mmu(void) | 2146 | static void xen_enter_lazy_mmu(void) |
| 2147 | { | 2147 | { |
| 2148 | enter_lazy(XEN_LAZY_MMU); | 2148 | preempt_disable(); |
| 2149 | if (xen_get_lazy_mode() != XEN_LAZY_MMU) | ||
| 2150 | enter_lazy(XEN_LAZY_MMU); | ||
| 2151 | preempt_enable(); | ||
| 2149 | } | 2152 | } |
| 2150 | 2153 | ||
| 2151 | static void xen_flush_lazy_mmu(void) | 2154 | static void xen_flush_lazy_mmu(void) |
| @@ -2182,7 +2185,8 @@ static void xen_leave_lazy_mmu(void) · 30 unmodified lines | |||
| 2182 | { | 2185 | { |
| 2183 | preempt_disable(); | 2186 | preempt_disable(); |
| 2184 | xen_mc_flush(); | 2187 | xen_mc_flush(); |
| 2185 | leave_lazy(XEN_LAZY_MMU); | 2188 | if (xen_get_lazy_mode() != XEN_LAZY_NONE) |
| 2189 | leave_lazy(XEN_LAZY_MMU); | ||
| 2186 | preempt_enable(); | 2190 | preempt_enable(); |
| 2187 | } | 2191 | } |
| 2188 | 2192 | ||
| @@ -655,7 +655,7 @@ static void __init xen_e820_swap_entry_with_ram(struct e820_entry *swap_entry) · 654 unmodified lines | |||
| 655 | /* Fill new entry (keep size and page offset). */ | 655 | /* Fill new entry (keep size and page offset). */ |
| 656 | entry->type = swap_entry->type; | 656 | entry->type = swap_entry->type; |
| 657 | entry->addr = entry_end - swap_size + | 657 | entry->addr = entry_end - swap_size + |
| 658 | swap_addr - swap_entry->addr; | 658 | swap_entry->addr - swap_addr; |
| 659 | entry->size = swap_entry->size; | 659 | entry->size = swap_entry->size; |
| 660 | 660 | ||
| 661 | /* Convert old entry to RAM, align to pages. */ | 661 | /* Convert old entry to RAM, align to pages. */ |
| @@ -695,17 +695,22 @@ static void __init xen_e820_resolve_conflicts(phys_addr_t start, · 33 unmodified lines | |||
| 695 | return; | 695 | return; |
| 696 | 696 | ||
| 697 | end = start + size; | 697 | end = start + size; |
| 698 | entry = xen_e820_table.entries; | 698 | mapcnt = 0; |
| 699 | 699 | ||
| 700 | for (mapcnt = 0; mapcnt < xen_e820_table.nr_entries; mapcnt++) { | 700 | while (mapcnt < xen_e820_table.nr_entries) { |
| 701 | entry = xen_e820_table.entries + mapcnt; | ||
| 701 | if (entry->addr >= end) | 702 | if (entry->addr >= end) |
| 702 | return; | 703 | return; |
| 703 | 704 | ||
| 704 | if (entry->addr + entry->size > start && | 705 | if (entry->addr + entry->size > start && |
| 705 | entry->type == E820_TYPE_NVS) | 706 | entry->type == E820_TYPE_NVS) { |
| 706 | xen_e820_swap_entry_with_ram(entry); | 707 | xen_e820_swap_entry_with_ram(entry); |
| 708 | /* E820 map has been changed, restart loop! */ | ||
| 709 | mapcnt = 0; | ||
| 710 | continue; | ||
| 711 | } | ||
| 707 | 712 | ||
| 708 | entry++; | 713 | mapcnt++; |
| 709 | } | 714 | } |
| 710 | } | 715 | } |
| 711 | 716 | ||
| @@ -6,5 +6,6 @@ generic-y += mcs_spinlock.h · 5 unmodified lines | |||
| 6 | generic-y += parport.h | 6 | generic-y += parport.h |
| 7 | generic-y += qrwlock.h | 7 | generic-y += qrwlock.h |
| 8 | generic-y += qspinlock.h | 8 | generic-y += qspinlock.h |
| 9 | generic-y += ring_buffer.h | ||
| 9 | generic-y += user.h | 10 | generic-y += user.h |
| 10 | generic-y += text-patching.h | 11 | generic-y += text-patching.h |
| @@ -55,6 +55,10 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size) · 54 unmodified lines | |||
| 55 | { | 55 | { |
| 56 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); | 56 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); |
| 57 | struct bio_integrity_payload *bip = bio_integrity(bio); | 57 | struct bio_integrity_payload *bip = bio_integrity(bio); |
| 58 | struct bvec_iter data_iter = { | ||
| 59 | .bi_sector = sector, | ||
| 60 | .bi_size = size, | ||
| 61 | }; | ||
| 58 | 62 | ||
| 59 | /* | 63 | /* |
| 60 | * Reinitialize bip->bip_iter. | 64 | * Reinitialize bip->bip_iter. |
| @@ -65,7 +69,7 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size) · 4 unmodified lines | |||
| 65 | memset(&bip->bip_iter, 0, sizeof(bip->bip_iter)); | 69 | memset(&bip->bip_iter, 0, sizeof(bip->bip_iter)); |
| 66 | bip->bip_iter.bi_sector = sector; | 70 | bip->bip_iter.bi_sector = sector; |
| 67 | bip->bip_iter.bi_size = bio_integrity_bytes(bi, size >> SECTOR_SHIFT); | 71 | bip->bip_iter.bi_size = bio_integrity_bytes(bi, size >> SECTOR_SHIFT); |
| 68 | return blk_status_to_errno(bio_integrity_verify(bio, &bip->bip_iter)); | 72 | return blk_status_to_errno(bio_integrity_verify(bio, &data_iter)); |
| 69 | } | 73 | } |
| 70 | 74 | ||
| 71 | static int __init fs_bio_integrity_init(void) | 75 | static int __init fs_bio_integrity_init(void) |
| @@ -308,7 +308,6 @@ static int bio_integrity_copy_user(struct bio *bio, struct bio_vec *bvec, · 307 unmodified lines | |||
| 308 | } | 308 | } |
| 309 | 309 | ||
| 310 | bip->bip_flags |= BIP_COPY_USER; | 310 | bip->bip_flags |= BIP_COPY_USER; |
| 311 | bip->bip_vcnt = nr_vecs; | ||
| 312 | return 0; | 311 | return 0; |
| 313 | free_bip: | 312 | free_bip: |
| 314 | bio_integrity_free(bio); | 313 | bio_integrity_free(bio); |
| @@ -403,6 +402,24 @@ int bio_integrity_map_user(struct bio *bio, struct iov_iter *iter) · 88 unmodified lines | |||
| 403 | if (unlikely(ret < 0)) | 402 | if (unlikely(ret < 0)) |
| 404 | goto free_bvec; | 403 | goto free_bvec; |
| 405 | 404 | ||
| 405 | /* | ||
| 406 | * Handle partial pinning. This can happen when pin_user_pages_fast() | ||
| 407 | * returns fewer pages than requested. | ||
| 408 | */ | ||
| 409 | if (user_backed_iter(iter) && unlikely(ret != bytes)) { | ||
| 410 | if (ret > 0) { | ||
| 411 | int npinned = DIV_ROUND_UP(offset + ret, PAGE_SIZE); | ||
| 412 | int i; | ||
| 413 | |||
| 414 | for (i = 0; i < npinned; i++) | ||
| 415 | unpin_user_page(pages[i]); | ||
| 416 | } | ||
| 417 | if (pages != stack_pages) | ||
| 418 | kvfree(pages); | ||
| 419 | ret = -EFAULT; | ||
| 420 | goto free_bvec; | ||
| 421 | } | ||
| 422 | |||
| 406 | nr_bvecs = bvec_from_pages(bvec, pages, nr_vecs, bytes, offset, | 423 | nr_bvecs = bvec_from_pages(bvec, pages, nr_vecs, bytes, offset, |
| 407 | &is_p2p); | 424 | &is_p2p); |
| 408 | if (pages != stack_pages) | 425 | if (pages != stack_pages) |
| @@ -1279,11 +1279,12 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, · 1278 unmodified lines | |||
| 1279 | return bio_iov_iter_align_down(bio, iter, len_align_mask); | 1279 | return bio_iov_iter_align_down(bio, iter, len_align_mask); |
| 1280 | } | 1280 | } |
| 1281 | 1281 | ||
| 1282 | static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size) | 1282 | static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size, |
| 1283 | size_t minsize) | ||
| 1283 | { | 1284 | { |
| 1284 | struct folio *folio; | 1285 | struct folio *folio; |
| 1285 | 1286 | ||
| 1286 | while (*size > PAGE_SIZE) { | 1287 | while (*size > minsize) { |
| 1287 | folio = folio_alloc(gfp | __GFP_NORETRY, get_order(*size)); | 1288 | folio = folio_alloc(gfp | __GFP_NORETRY, get_order(*size)); |
| 1288 | if (folio) | 1289 | if (folio) |
| 1289 | return folio; | 1290 | return folio; |
| @@ -1307,7 +1308,7 @@ static void bio_free_folios(struct bio *bio) · 17 unmodified lines | |||
| 1307 | } | 1308 | } |
| 1308 | 1309 | ||
| 1309 | static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, | 1310 | static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, |
| 1310 | size_t maxlen) | 1311 | size_t maxlen, size_t minsize) |
| 1311 | { | 1312 | { |
| 1312 | size_t total_len = min(maxlen, iov_iter_count(iter)); | 1313 | size_t total_len = min(maxlen, iov_iter_count(iter)); |
| 1313 | 1314 | ||
| @@ -1322,13 +1323,13 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, · 8 unmodified lines | |||
| 1322 | size_t this_len = min(total_len, SZ_1M); | 1323 | size_t this_len = min(total_len, SZ_1M); |
| 1323 | struct folio *folio; | 1324 | struct folio *folio; |
| 1324 | 1325 | ||
| 1325 | if (this_len > PAGE_SIZE * 2) | 1326 | if (this_len > minsize * 2) |
| 1326 | this_len = rounddown_pow_of_two(this_len); | 1327 | this_len = rounddown_pow_of_two(this_len); |
| 1327 | 1328 | ||
| 1328 | if (bio->bi_iter.bi_size > BIO_MAX_SIZE - this_len) | 1329 | if (bio->bi_iter.bi_size > BIO_MAX_SIZE - this_len) |
| 1329 | break; | 1330 | break; |
| 1330 | 1331 | ||
| 1331 | folio = folio_alloc_greedy(GFP_KERNEL, &this_len); | 1332 | folio = folio_alloc_greedy(GFP_KERNEL, &this_len, minsize); |
| 1332 | if (!folio) | 1333 | if (!folio) |
| 1333 | break; | 1334 | break; |
| 1334 | bio_add_folio_nofail(bio, folio, this_len, 0); | 1335 | bio_add_folio_nofail(bio, folio, this_len, 0); |
| @@ -1344,16 +1345,16 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, · 9 unmodified lines | |||
| 1344 | 1345 | ||
| 1345 | if (!bio->bi_iter.bi_size) | 1346 | if (!bio->bi_iter.bi_size) |
| 1346 | return -ENOMEM; | 1347 | return -ENOMEM; |
| 1347 | return 0; | 1348 | return bio_iov_iter_align_down(bio, iter, minsize - 1); |
| 1348 | } | 1349 | } |
| 1349 | 1350 | ||
| 1350 | static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, | 1351 | static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, |
| 1351 | size_t maxlen) | 1352 | size_t maxlen, size_t minsize) |
| 1352 | { | 1353 | { |
| 1353 | size_t len = min3(iov_iter_count(iter), maxlen, SZ_1M); | 1354 | size_t len = min3(iov_iter_count(iter), maxlen, SZ_1M); |
| 1354 | struct folio *folio; | 1355 | struct folio *folio; |
| 1355 | 1356 | ||
| 1356 | folio = folio_alloc_greedy(GFP_KERNEL, &len); | 1357 | folio = folio_alloc_greedy(GFP_KERNEL, &len, minsize); |
| 1357 | if (!folio) | 1358 | if (!folio) |
| 1358 | return -ENOMEM; | 1359 | return -ENOMEM; |
| 1359 | 1360 | ||
| @@ -1382,7 +1383,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 22 unmodified lines | |||
| 1382 | bvec_set_folio(&bio->bi_io_vec[0], folio, bio->bi_iter.bi_size, 0); | 1383 | bvec_set_folio(&bio->bi_io_vec[0], folio, bio->bi_iter.bi_size, 0); |
| 1383 | if (iov_iter_extract_will_pin(iter)) | 1384 | if (iov_iter_extract_will_pin(iter)) |
| 1384 | bio_set_flag(bio, BIO_PAGE_PINNED); | 1385 | bio_set_flag(bio, BIO_PAGE_PINNED); |
| 1385 | return 0; | 1386 | return bio_iov_iter_align_down(bio, iter, minsize - 1); |
| 1386 | } | 1387 | } |
| 1387 | 1388 | ||
| 1388 | /** | 1389 | /** |
| @@ -1390,6 +1391,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 1 unmodified line | |||
| 1390 | * @bio: bio to send | 1391 | * @bio: bio to send |
| 1391 | * @iter: iter to read from / write into | 1392 | * @iter: iter to read from / write into |
| 1392 | * @maxlen: maximum size to bounce | 1393 | * @maxlen: maximum size to bounce |
| 1394 | * @minsize: minimum folio allocation size | ||
| 1393 | * | 1395 | * |
| 1394 | * Helper for direct I/O implementations that need to bounce buffer because | 1396 | * Helper for direct I/O implementations that need to bounce buffer because |
| 1395 | * we need to checksum the data or perform other operations that require | 1397 | * we need to checksum the data or perform other operations that require |
| @@ -1397,11 +1399,12 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 1 unmodified line | |||
| 1397 | * copies the data into it. Needs to be paired with bio_iov_iter_unbounce() | 1399 | * copies the data into it. Needs to be paired with bio_iov_iter_unbounce() |
| 1398 | * called on completion. | 1400 | * called on completion. |
| 1399 | */ | 1401 | */ |
| 1400 | int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen) | 1402 | int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen, |
| 1403 | size_t minsize) | ||
| 1401 | { | 1404 | { |
| 1402 | if (op_is_write(bio_op(bio))) | 1405 | if (op_is_write(bio_op(bio))) |
| 1403 | return bio_iov_iter_bounce_write(bio, iter, maxlen); | 1406 | return bio_iov_iter_bounce_write(bio, iter, maxlen, minsize); |
| 1404 | return bio_iov_iter_bounce_read(bio, iter, maxlen); | 1407 | return bio_iov_iter_bounce_read(bio, iter, maxlen, minsize); |
| 1405 | } | 1408 | } |
| 1406 | 1409 | ||
| 1407 | static void bvec_unpin(struct bio_vec *bv, bool mark_dirty) | 1410 | static void bvec_unpin(struct bio_vec *bv, bool mark_dirty) |
| @@ -2241,7 +2241,7 @@ void blk_cgroup_bio_start(struct bio *bio) · 2240 unmodified lines | |||
| 2241 | } | 2241 | } |
| 2242 | 2242 | ||
| 2243 | u64_stats_update_end_irqrestore(&bis->sync, flags); | 2243 | u64_stats_update_end_irqrestore(&bis->sync, flags); |
| 2244 | css_rstat_updated(&blkcg->css, cpu); | 2244 | __css_rstat_updated(&blkcg->css, cpu); |
| 2245 | put_cpu(); | 2245 | put_cpu(); |
| 2246 | } | 2246 | } |
| 2247 | 2247 | ||
| @@ -3077,7 +3077,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 3076 unmodified lines | |||
| 3077 | /* | 3077 | /* |
| 3078 | * Check if there is a suitable cached request and return it. | 3078 | * Check if there is a suitable cached request and return it. |
| 3079 | */ | 3079 | */ |
| 3080 | static struct request *blk_mq_peek_cached_request(struct blk_plug *plug, | 3080 | static struct request *blk_mq_get_cached_request(struct blk_plug *plug, |
| 3081 | struct request_queue *q, blk_opf_t opf) | 3081 | struct request_queue *q, blk_opf_t opf) |
| 3082 | { | 3082 | { |
| 3083 | enum hctx_type type = blk_mq_get_hctx_type(opf); | 3083 | enum hctx_type type = blk_mq_get_hctx_type(opf); |
| @@ -3093,27 +3093,10 @@ static struct request *blk_mq_peek_cached_request(struct blk_plug *plug, · 9 unmodified lines | |||
| 3093 | return NULL; | 3093 | return NULL; |
| 3094 | if (op_is_flush(rq->cmd_flags) != op_is_flush(opf)) | 3094 | if (op_is_flush(rq->cmd_flags) != op_is_flush(opf)) |
| 3095 | return NULL; | 3095 | return NULL; |
| 3096 | rq_list_pop(&plug->cached_rqs); | ||
| 3096 | return rq; | 3097 | return rq; |
| 3097 | } | 3098 | } |
| 3098 | 3099 | ||
| 3099 | static void blk_mq_use_cached_rq(struct request *rq, struct blk_plug *plug, | ||
| 3100 | struct bio *bio) | ||
| 3101 | { | ||
| 3102 | if (rq_list_pop(&plug->cached_rqs) != rq) | ||
| 3103 | WARN_ON_ONCE(1); | ||
| 3104 | |||
| 3105 | /* | ||
| 3106 | * If any qos ->throttle() end up blocking, we will have flushed the | ||
| 3107 | * plug and hence killed the cached_rq list as well. Pop this entry | ||
| 3108 | * before we throttle. | ||
| 3109 | */ | ||
| 3110 | rq_qos_throttle(rq->q, bio); | ||
| 3111 | |||
| 3112 | blk_mq_rq_time_init(rq, blk_time_get_ns()); | ||
| 3113 | rq->cmd_flags = bio->bi_opf; | ||
| 3114 | INIT_LIST_HEAD(&rq->queuelist); | ||
| 3115 | } | ||
| 3116 | |||
| 3117 | static bool bio_unaligned(const struct bio *bio, struct request_queue *q) | 3100 | static bool bio_unaligned(const struct bio *bio, struct request_queue *q) |
| 3118 | { | 3101 | { |
| 3119 | unsigned int bs_mask = queue_logical_block_size(q) - 1; | 3102 | unsigned int bs_mask = queue_logical_block_size(q) - 1; |
| @@ -3152,7 +3135,7 @@ void blk_mq_submit_bio(struct bio *bio) · 32 unmodified lines | |||
| 3152 | /* | 3135 | /* |
| 3153 | * If the plug has a cached request for this queue, try to use it. | 3136 | * If the plug has a cached request for this queue, try to use it. |
| 3154 | */ | 3137 | */ |
| 3155 | rq = blk_mq_peek_cached_request(plug, q, bio->bi_opf); | 3138 | rq = blk_mq_get_cached_request(plug, q, bio->bi_opf); |
| 3156 | 3139 | ||
| 3157 | /* | 3140 | /* |
| 3158 | * A BIO that was released from a zone write plug has already been | 3141 | * A BIO that was released from a zone write plug has already been |
| @@ -3211,7 +3194,10 @@ void blk_mq_submit_bio(struct bio *bio) · 52 unmodified lines | |||
| 3211 | 3194 | ||
| 3212 | new_request: | 3195 | new_request: |
| 3213 | if (rq) { | 3196 | if (rq) { |
| 3214 | blk_mq_use_cached_rq(rq, plug, bio); | 3197 | rq_qos_throttle(rq->q, bio); |
| 3198 | blk_mq_rq_time_init(rq, blk_time_get_ns()); | ||
| 3199 | rq->cmd_flags = bio->bi_opf; | ||
| 3200 | INIT_LIST_HEAD(&rq->queuelist); | ||
| 3215 | } else { | 3201 | } else { |
| 3216 | rq = blk_mq_get_new_requests(q, plug, bio); | 3202 | rq = blk_mq_get_new_requests(q, plug, bio); |
| 3217 | if (unlikely(!rq)) { | 3203 | if (unlikely(!rq)) { |
| @@ -3257,12 +3243,10 @@ new_request: · 39 unmodified lines | |||
| 3257 | return; | 3243 | return; |
| 3258 | 3244 | ||
| 3259 | queue_exit: | 3245 | queue_exit: |
| 3260 | /* | ||
| 3261 | * Don't drop the queue reference if we were trying to use a cached | ||
| 3262 | * request and thus didn't acquire one. | ||
| 3263 | */ | ||
| 3264 | if (!rq) | 3246 | if (!rq) |
| 3265 | blk_queue_exit(q); | 3247 | blk_queue_exit(q); |
| 3248 | else | ||
| 3249 | rq_list_add_head(&plug->cached_rqs, rq); | ||
| 3266 | } | 3250 | } |
| 3267 | 3251 | ||
| 3268 | #ifdef CONFIG_BLK_MQ_STACKING | 3252 | #ifdef CONFIG_BLK_MQ_STACKING |
| @@ -3307,6 +3291,25 @@ blk_status_t blk_insert_cloned_request(struct request *rq) · 38 unmodified lines | |||
| 3307 | return BLK_STS_IOERR; | 3291 | return BLK_STS_IOERR; |
| 3308 | } | 3292 | } |
| 3309 | 3293 | ||
| 3294 | /* | ||
| 3295 | * Integrity segment counting depends on the same queue limits | ||
| 3296 | * (virt_boundary_mask, seg_boundary_mask, max_segment_size) that | ||
| 3297 | * vary across stacked queues, so recompute against the bottom | ||
| 3298 | * queue just like nr_phys_segments above. | ||
| 3299 | */ | ||
| 3300 | if (blk_integrity_rq(rq) && rq->bio) { | ||
| 3301 | unsigned short max_int_segs = queue_max_integrity_segments(q); | ||
| 3302 | |||
| 3303 | rq->nr_integrity_segments = | ||
| 3304 | blk_rq_count_integrity_sg(rq->q, rq->bio); | ||
| 3305 | if (rq->nr_integrity_segments > max_int_segs) { | ||
| 3306 | printk(KERN_ERR "%s: over max integrity segments limit. (%u > %u)\n", | ||
| 3307 | __func__, rq->nr_integrity_segments, | ||
| 3308 | max_int_segs); | ||
| 3309 | return BLK_STS_IOERR; | ||
| 3310 | } | ||
| 3311 | } | ||
| 3312 | |||
| 3310 | if (q->disk && should_fail_request(q->disk->part0, blk_rq_bytes(rq))) | 3313 | if (q->disk && should_fail_request(q->disk->part0, blk_rq_bytes(rq))) |
| 3311 | return BLK_STS_IOERR; | 3314 | return BLK_STS_IOERR; |
| 3312 | 3315 | ||
| @@ -623,6 +623,28 @@ static void disk_mark_zone_wplug_dead(struct blk_zone_wplug *zwplug) · 622 unmodified lines | |||
| 623 | } | 623 | } |
| 624 | } | 624 | } |
| 625 | 625 | ||
| 626 | static inline bool disk_check_zone_wplug_dead(struct blk_zone_wplug *zwplug) | ||
| 627 | { | ||
| 628 | if (!(zwplug->flags & BLK_ZONE_WPLUG_DEAD)) | ||
| 629 | return false; | ||
| 630 | |||
| 631 | /* | ||
| 632 | * If a new write is received right after a zone reset completes and | ||
| 633 | * while the disk_zone_wplugs_worker() thread has not yet released the | ||
| 634 | * reference on the zone write plug after processing the last write to | ||
| 635 | * the zone, then the new write BIO will see the zone write plug marked | ||
| 636 | * as dead. This case is however a false positive and a perfectly valid | ||
| 637 | * pattern. In such case, restore the zone write plug to a live one. | ||
| 638 | */ | ||
| 639 | if (!zwplug->wp_offset && bio_list_empty(&zwplug->bio_list)) { | ||
| 640 | zwplug->flags &= ~BLK_ZONE_WPLUG_DEAD; | ||
| 641 | refcount_inc(&zwplug->ref); | ||
| 642 | return false; | ||
| 643 | } | ||
| 644 | |||
| 645 | return true; | ||
| 646 | } | ||
| 647 | |||
| 626 | static bool disk_zone_wplug_submit_bio(struct gendisk *disk, | 648 | static bool disk_zone_wplug_submit_bio(struct gendisk *disk, |
| 627 | struct blk_zone_wplug *zwplug); | 649 | struct blk_zone_wplug *zwplug); |
| 628 | 650 | ||
| @@ -1444,12 +1466,12 @@ static bool blk_zone_wplug_handle_write(struct bio *bio, unsigned int nr_segs) · 815 unmodified lines | |||
| 1444 | spin_lock_irqsave(&zwplug->lock, flags); | 1466 | spin_lock_irqsave(&zwplug->lock, flags); |
| 1445 | 1467 | ||
| 1446 | /* | 1468 | /* |
| 1447 | * If we got a zone write plug marked as dead, then the user is issuing | 1469 | * Check if we got a zone write plug marked as dead. If yes, then the |
| 1448 | * writes to a full zone, or without synchronizing with zone reset or | 1470 | * user is likely issuing writes to a full zone, or without |
| 1449 | * zone finish operations. In such case, fail the BIO to signal this | 1471 | * synchronizing with zone reset or zone finish operations. In such |
| 1450 | * invalid usage. | 1472 | * case, fail the BIO to signal this invalid usage. |
| 1451 | */ | 1473 | */ |
| 1452 | if (zwplug->flags & BLK_ZONE_WPLUG_DEAD) { | 1474 | if (disk_check_zone_wplug_dead(zwplug)) { |
| 1453 | spin_unlock_irqrestore(&zwplug->lock, flags); | 1475 | spin_unlock_irqrestore(&zwplug->lock, flags); |
| 1454 | disk_put_zone_wplug(zwplug); | 1476 | disk_put_zone_wplug(zwplug); |
| 1455 | bio_io_error(bio); | 1477 | bio_io_error(bio); |
| @@ -1979,8 +2001,10 @@ static void disk_set_zones_cond_array(struct gendisk *disk, u8 *zones_cond) · 523 unmodified lines | |||
| 1979 | 2001 | ||
| 1980 | void disk_free_zone_resources(struct gendisk *disk) | 2002 | void disk_free_zone_resources(struct gendisk *disk) |
| 1981 | { | 2003 | { |
| 1982 | if (disk->zone_wplugs_worker) | 2004 | if (disk->zone_wplugs_worker) { |
| 1983 | kthread_stop(disk->zone_wplugs_worker); | 2005 | kthread_stop(disk->zone_wplugs_worker); |
| 2006 | disk->zone_wplugs_worker = NULL; | ||
| 2007 | } | ||
| 1984 | WARN_ON_ONCE(!list_empty(&disk->zone_wplugs_list)); | 2008 | WARN_ON_ONCE(!list_empty(&disk->zone_wplugs_list)); |
| 1985 | 2009 | ||
| 1986 | if (disk->zone_wplugs_wq) { | 2010 | if (disk->zone_wplugs_wq) { |
| @@ -2113,9 +2137,6 @@ commit: · 126 unmodified lines | |||
| 2113 | ret = queue_limits_commit_update(q, &lim); | 2137 | ret = queue_limits_commit_update(q, &lim); |
| 2114 | 2138 | ||
| 2115 | unfreeze: | 2139 | unfreeze: |
| 2116 | if (ret) | ||
| 2117 | disk_free_zone_resources(disk); | ||
| 2118 | |||
| 2119 | blk_mq_unfreeze_queue(q, memflags); | 2140 | blk_mq_unfreeze_queue(q, memflags); |
| 2120 | 2141 | ||
| 2121 | return ret; | 2142 | return ret; |
| @@ -857,6 +857,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) · 856 unmodified lines | |||
| 857 | #endif | 857 | #endif |
| 858 | 858 | ||
| 859 | struct blk_iou_cmd { | 859 | struct blk_iou_cmd { |
| 860 | u64 start; | ||
| 861 | u64 len; | ||
| 860 | int res; | 862 | int res; |
| 861 | bool nowait; | 863 | bool nowait; |
| 862 | }; | 864 | }; |
| @@ -946,23 +948,27 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) · 83 unmodified lines | |||
| 946 | { | 948 | { |
| 947 | struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); | 949 | struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); |
| 948 | struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); | 950 | struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); |
| 949 | const struct io_uring_sqe *sqe = cmd->sqe; | ||
| 950 | u32 cmd_op = cmd->cmd_op; | 951 | u32 cmd_op = cmd->cmd_op; |
| 951 | uint64_t start, len; | ||
| 952 | 952 | ||
| 953 | if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || | 953 | /* Read what we need from the SQE on the first issue */ |
| 954 | sqe->rw_flags || sqe->file_index)) | 954 | if (!(issue_flags & IORING_URING_CMD_REISSUE)) { |
| 955 | return -EINVAL; | 955 | const struct io_uring_sqe *sqe = cmd->sqe; |
| 956 | |||
| 957 | if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || | ||
| 958 | sqe->rw_flags || sqe->file_index)) | ||
| 959 | return -EINVAL; | ||
| 960 | |||
| 961 | bic->start = READ_ONCE(sqe->addr); | ||
| 962 | bic->len = READ_ONCE(sqe->addr3); | ||
| 963 | } | ||
| 956 | 964 | ||
| 957 | bic->res = 0; | 965 | bic->res = 0; |
| 958 | bic->nowait = issue_flags & IO_URING_F_NONBLOCK; | 966 | bic->nowait = issue_flags & IO_URING_F_NONBLOCK; |
| 959 | 967 | ||
| 960 | start = READ_ONCE(sqe->addr); | ||
| 961 | len = READ_ONCE(sqe->addr3); | ||
| 962 | |||
| 963 | switch (cmd_op) { | 968 | switch (cmd_op) { |
| 964 | case BLOCK_URING_CMD_DISCARD: | 969 | case BLOCK_URING_CMD_DISCARD: |
| 965 | return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); | 970 | return blkdev_cmd_discard(cmd, bdev, bic->start, bic->len, |
| 971 | bic->nowait); | ||
| 966 | } | 972 | } |
| 967 | return -EINVAL; | 973 | return -EINVAL; |
| 968 | } | 974 | } |
| @@ -390,6 +390,11 @@ static int crypto_authenc_esn_create(struct crypto_template *tmpl, · 389 unmodified lines | |||
| 390 | auth = crypto_spawn_ahash_alg(&ctx->auth); | 390 | auth = crypto_spawn_ahash_alg(&ctx->auth); |
| 391 | auth_base = &auth->base; | 391 | auth_base = &auth->base; |
| 392 | 392 | ||
| 393 | if (auth->digestsize > 0 && auth->digestsize < 4) { | ||
| 394 | err = -EINVAL; | ||
| 395 | goto err_free_inst; | ||
| 396 | } | ||
| 397 | |||
| 393 | err = crypto_grab_skcipher(&ctx->enc, aead_crypto_instance(inst), | 398 | err = crypto_grab_skcipher(&ctx->enc, aead_crypto_instance(inst), |
| 394 | crypto_attr_alg_name(tb[2]), 0, mask); | 399 | crypto_attr_alg_name(tb[2]), 0, mask); |
| 395 | if (err) | 400 | if (err) |
| @@ -134,27 +134,69 @@ EXPORT_SYMBOL(crypto_krb5_how_much_data); · 133 unmodified lines | |||
| 134 | * Find the offset and size of the data in a secure message so that this | 134 | * Find the offset and size of the data in a secure message so that this |
| 135 | * information can be used in the metadata buffer which will get added to the | 135 | * information can be used in the metadata buffer which will get added to the |
| 136 | * digest by crypto_krb5_verify_mic(). | 136 | * digest by crypto_krb5_verify_mic(). |
| 137 | * | ||
| 138 | * Return: 0 if successful, -EBADMSG if the message is too short or -EINVAL if | ||
| 139 | * the mode is unsupported. | ||
| 137 | */ | 140 | */ |
| 138 | void crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | 141 | int crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, |
| 139 | enum krb5_crypto_mode mode, | 142 | enum krb5_crypto_mode mode, |
| 140 | size_t *_offset, size_t *_len) | 143 | size_t *_offset, size_t *_len) |
| 141 | { | 144 | { |
| 142 | switch (mode) { | 145 | switch (mode) { |
| 143 | case KRB5_CHECKSUM_MODE: | 146 | case KRB5_CHECKSUM_MODE: |
| 147 | if (*_len < krb5->cksum_len) | ||
| 148 | return -EBADMSG; | ||
| 144 | *_offset += krb5->cksum_len; | 149 | *_offset += krb5->cksum_len; |
| 145 | *_len -= krb5->cksum_len; | 150 | *_len -= krb5->cksum_len; |
| 146 | return; | 151 | return 0; |
| 147 | case KRB5_ENCRYPT_MODE: | 152 | case KRB5_ENCRYPT_MODE: |
| 153 | if (*_len < krb5->conf_len + krb5->cksum_len) | ||
| 154 | return -EBADMSG; | ||
| 148 | *_offset += krb5->conf_len; | 155 | *_offset += krb5->conf_len; |
| 149 | *_len -= krb5->conf_len + krb5->cksum_len; | 156 | *_len -= krb5->conf_len + krb5->cksum_len; |
| 150 | return; | 157 | return 0; |
| 151 | default: | 158 | default: |
| 152 | WARN_ON_ONCE(1); | 159 | WARN_ON_ONCE(1); |
| 153 | return; | 160 | return -EINVAL; |
| 154 | } | 161 | } |
| 155 | } | 162 | } |
| 156 | EXPORT_SYMBOL(crypto_krb5_where_is_the_data); | 163 | EXPORT_SYMBOL(crypto_krb5_where_is_the_data); |
| 157 | 164 | ||
| 165 | /** | ||
| 166 | * crypto_krb5_check_data_len - Check a message is big enough | ||
| 167 | * @krb5: The encoding to use. | ||
| 168 | * @mode: Mode of operation. | ||
| 169 | * @len: The length of the secure blob. | ||
| 170 | * @min_content: Minimum length of the content inside the blob. | ||
| 171 | * | ||
| 172 | * Check that a message is large enough to hold whatever bits the encryption | ||
| 173 | * type wants to glue on (nonce, checksum) plus a minimum amount of content. | ||
| 174 | * | ||
| 175 | * Return: 0 if successful, -EBADMSG if the message is too short or -EINVAL if | ||
| 176 | * the mode is unsupported. | ||
| 177 | */ | ||
| 178 | int crypto_krb5_check_data_len(const struct krb5_enctype *krb5, | ||
| 179 | enum krb5_crypto_mode mode, | ||
| 180 | size_t len, size_t min_content) | ||
| 181 | { | ||
| 182 | switch (mode) { | ||
| 183 | case KRB5_CHECKSUM_MODE: | ||
| 184 | if (len < krb5->cksum_len || | ||
| 185 | len - krb5->cksum_len < min_content) | ||
| 186 | return -EBADMSG; | ||
| 187 | return 0; | ||
| 188 | case KRB5_ENCRYPT_MODE: | ||
| 189 | if (len < krb5->conf_len + krb5->cksum_len || | ||
| 190 | len - (krb5->conf_len + krb5->cksum_len) < min_content) | ||
| 191 | return -EBADMSG; | ||
| 192 | return 0; | ||
| 193 | default: | ||
| 194 | WARN_ON_ONCE(1); | ||
| 195 | return -EINVAL; | ||
| 196 | } | ||
| 197 | } | ||
| 198 | EXPORT_SYMBOL(crypto_krb5_check_data_len); | ||
| 199 | |||
| 158 | /* | 200 | /* |
| 159 | * Prepare the encryption with derived key data. | 201 | * Prepare the encryption with derived key data. |
| 160 | */ | 202 | */ |
| @@ -511,6 +511,11 @@ static int aie2_init(struct amdxdna_dev *xdna) · 510 unmodified lines | |||
| 511 | return -EINVAL; | 511 | return -EINVAL; |
| 512 | } | 512 | } |
| 513 | 513 | ||
| 514 | if (!xdna->group) { | ||
| 515 | XDNA_ERR(xdna, "Running without IOMMU not supported"); | ||
| 516 | return -EINVAL; | ||
| 517 | } | ||
| 518 | |||
| 514 | ndev = drmm_kzalloc(&xdna->ddev, sizeof(*ndev), GFP_KERNEL); | 519 | ndev = drmm_kzalloc(&xdna->ddev, sizeof(*ndev), GFP_KERNEL); |
| 515 | if (!ndev) | 520 | if (!ndev) |
| 516 | return -ENOMEM; | 521 | return -ENOMEM; |
| @@ -490,6 +490,9 @@ static struct dma_buf *amdxdna_gem_prime_export(struct drm_gem_object *gobj, int · 489 unmodified lines | |||
| 490 | struct amdxdna_gem_obj *abo = to_xdna_obj(gobj); | 490 | struct amdxdna_gem_obj *abo = to_xdna_obj(gobj); |
| 491 | DEFINE_DMA_BUF_EXPORT_INFO(exp_info); | 491 | DEFINE_DMA_BUF_EXPORT_INFO(exp_info); |
| 492 | 492 | ||
| 493 | if (abo->private_buffer) | ||
| 494 | return ERR_PTR(-EOPNOTSUPP); | ||
| 495 | |||
| 493 | if (abo->dma_buf) { | 496 | if (abo->dma_buf) { |
| 494 | get_dma_buf(abo->dma_buf); | 497 | get_dma_buf(abo->dma_buf); |
| 495 | return abo->dma_buf; | 498 | return abo->dma_buf; |
| @@ -685,6 +688,7 @@ amdxdna_gem_create_ubuf_object(struct drm_device *dev, struct amdxdna_drm_create · 189 unmodified lines | |||
| 685 | { | 688 | { |
| 686 | struct amdxdna_dev *xdna = to_xdna_dev(dev); | 689 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 687 | struct amdxdna_drm_va_tbl va_tbl; | 690 | struct amdxdna_drm_va_tbl va_tbl; |
| 691 | struct amdxdna_gem_obj *abo; | ||
| 688 | struct drm_gem_object *gobj; | 692 | struct drm_gem_object *gobj; |
| 689 | struct dma_buf *dma_buf; | 693 | struct dma_buf *dma_buf; |
| 690 | 694 | ||
| @@ -711,7 +715,10 @@ amdxdna_gem_create_ubuf_object(struct drm_device *dev, struct amdxdna_drm_create · 20 unmodified lines | |||
| 711 | 715 | ||
| 712 | dma_buf_put(dma_buf); | 716 | dma_buf_put(dma_buf); |
| 713 | 717 | ||
| 714 | return to_xdna_obj(gobj); | 718 | abo = to_xdna_obj(gobj); |
| 719 | abo->private_buffer = true; | ||
| 720 | |||
| 721 | return abo; | ||
| 715 | } | 722 | } |
| 716 | 723 | ||
| 717 | struct drm_gem_object * | 724 | struct drm_gem_object * |
| @@ -54,6 +54,8 @@ struct amdxdna_gem_obj { · 53 unmodified lines | |||
| 54 | 54 | ||
| 55 | /* True, if BO is managed by XRT, not application */ | 55 | /* True, if BO is managed by XRT, not application */ |
| 56 | bool internal; | 56 | bool internal; |
| 57 | /* True, if BO is not exportable */ | ||
| 58 | bool private_buffer; | ||
| 57 | }; | 59 | }; |
| 58 | 60 | ||
| 59 | #define to_gobj(obj) (&(obj)->base.base) | 61 | #define to_gobj(obj) (&(obj)->base.base) |
| @@ -69,60 +69,10 @@ static void amdxdna_ubuf_release(struct dma_buf *dbuf) · 68 unmodified lines | |||
| 69 | kfree(ubuf); | 69 | kfree(ubuf); |
| 70 | } | 70 | } |
| 71 | 71 | ||
| 72 | static vm_fault_t amdxdna_ubuf_vm_fault(struct vm_fault *vmf) | ||
| 73 | { | ||
| 74 | struct vm_area_struct *vma = vmf->vma; | ||
| 75 | struct amdxdna_ubuf_priv *ubuf; | ||
| 76 | unsigned long pfn; | ||
| 77 | pgoff_t pgoff; | ||
| 78 | |||
| 79 | ubuf = vma->vm_private_data; | ||
| 80 | pgoff = (vmf->address - vma->vm_start) >> PAGE_SHIFT; | ||
| 81 | |||
| 82 | pfn = page_to_pfn(ubuf->pages[pgoff]); | ||
| 83 | return vmf_insert_pfn(vma, vmf->address, pfn); | ||
| 84 | } | ||
| 85 | |||
| 86 | static const struct vm_operations_struct amdxdna_ubuf_vm_ops = { | ||
| 87 | .fault = amdxdna_ubuf_vm_fault, | ||
| 88 | }; | ||
| 89 | |||
| 90 | static int amdxdna_ubuf_mmap(struct dma_buf *dbuf, struct vm_area_struct *vma) | ||
| 91 | { | ||
| 92 | struct amdxdna_ubuf_priv *ubuf = dbuf->priv; | ||
| 93 | |||
| 94 | vma->vm_ops = &amdxdna_ubuf_vm_ops; | ||
| 95 | vma->vm_private_data = ubuf; | ||
| 96 | vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); | ||
| 97 | |||
| 98 | return 0; | ||
| 99 | } | ||
| 100 | |||
| 101 | static int amdxdna_ubuf_vmap(struct dma_buf *dbuf, struct iosys_map *map) | ||
| 102 | { | ||
| 103 | struct amdxdna_ubuf_priv *ubuf = dbuf->priv; | ||
| 104 | void *kva; | ||
| 105 | |||
| 106 | kva = vmap(ubuf->pages, ubuf->nr_pages, VM_MAP, PAGE_KERNEL); | ||
| 107 | if (!kva) | ||
| 108 | return -EINVAL; | ||
| 109 | |||
| 110 | iosys_map_set_vaddr(map, kva); | ||
| 111 | return 0; | ||
| 112 | } | ||
| 113 | |||
| 114 | static void amdxdna_ubuf_vunmap(struct dma_buf *dbuf, struct iosys_map *map) | ||
| 115 | { | ||
| 116 | vunmap(map->vaddr); | ||
| 117 | } | ||
| 118 | |||
| 119 | static const struct dma_buf_ops amdxdna_ubuf_dmabuf_ops = { | 72 | static const struct dma_buf_ops amdxdna_ubuf_dmabuf_ops = { |
| 120 | .map_dma_buf = amdxdna_ubuf_map, | 73 | .map_dma_buf = amdxdna_ubuf_map, |
| 121 | .unmap_dma_buf = amdxdna_ubuf_unmap, | 74 | .unmap_dma_buf = amdxdna_ubuf_unmap, |
| 122 | .release = amdxdna_ubuf_release, | 75 | .release = amdxdna_ubuf_release, |
| 123 | .mmap = amdxdna_ubuf_mmap, | ||
| 124 | .vmap = amdxdna_ubuf_vmap, | ||
| 125 | .vunmap = amdxdna_ubuf_vunmap, | ||
| 126 | }; | 76 | }; |
| 127 | 77 | ||
| 128 | struct dma_buf *amdxdna_get_ubuf(struct drm_device *dev, | 78 | struct dma_buf *amdxdna_get_ubuf(struct drm_device *dev, |
| @@ -450,7 +450,7 @@ priority_bands_fops_write(struct file *file, const char __user *user_buf, size_t · 449 unmodified lines | |||
| 450 | u32 band; | 450 | u32 band; |
| 451 | int ret; | 451 | int ret; |
| 452 | 452 | ||
| 453 | if (size >= sizeof(buf)) | 453 | if (*pos != 0 || size >= sizeof(buf)) |
| 454 | return -EINVAL; | 454 | return -EINVAL; |
| 455 | 455 | ||
| 456 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, pos, user_buf, size); | 456 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, pos, user_buf, size); |
| @@ -537,6 +537,26 @@ static const struct file_operations ivpu_fops = { · 536 unmodified lines | |||
| 537 | #endif | 537 | #endif |
| 538 | }; | 538 | }; |
| 539 | 539 | ||
| 540 | static int ivpu_gem_prime_handle_to_fd(struct drm_device *dev, struct drm_file *file_priv, | ||
| 541 | u32 handle, u32 flags, int *prime_fd) | ||
| 542 | { | ||
| 543 | struct drm_gem_object *obj; | ||
| 544 | |||
| 545 | obj = drm_gem_object_lookup(file_priv, handle); | ||
| 546 | if (!obj) | ||
| 547 | return -ENOENT; | ||
| 548 | |||
| 549 | if (drm_gem_is_imported(obj)) { | ||
| 550 | /* Do not allow re-exporting */ | ||
| 551 | drm_gem_object_put(obj); | ||
| 552 | return -EOPNOTSUPP; | ||
| 553 | } | ||
| 554 | |||
| 555 | drm_gem_object_put(obj); | ||
| 556 | |||
| 557 | return drm_gem_prime_handle_to_fd(dev, file_priv, handle, flags, prime_fd); | ||
| 558 | } | ||
| 559 | |||
| 540 | static const struct drm_driver driver = { | 560 | static const struct drm_driver driver = { |
| 541 | .driver_features = DRIVER_GEM | DRIVER_COMPUTE_ACCEL, | 561 | .driver_features = DRIVER_GEM | DRIVER_COMPUTE_ACCEL, |
| 542 | 562 | ||
| @@ -545,6 +565,7 @@ static const struct drm_driver driver = { · 2 unmodified lines | |||
| 545 | 565 | ||
| 546 | .gem_create_object = ivpu_gem_create_object, | 566 | .gem_create_object = ivpu_gem_create_object, |
| 547 | .gem_prime_import = ivpu_gem_prime_import, | 567 | .gem_prime_import = ivpu_gem_prime_import, |
| 568 | .prime_handle_to_fd = ivpu_gem_prime_handle_to_fd, | ||
| 548 | 569 | ||
| 549 | .ioctls = ivpu_drm_ioctls, | 570 | .ioctls = ivpu_drm_ioctls, |
| 550 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), | 571 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), |
| @@ -606,8 +606,11 @@ static const struct vm_operations_struct drm_vm_ops = { · 605 unmodified lines | |||
| 606 | static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) | 606 | static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) |
| 607 | { | 607 | { |
| 608 | struct qaic_bo *bo = to_qaic_bo(obj); | 608 | struct qaic_bo *bo = to_qaic_bo(obj); |
| 609 | unsigned long remap_start; | ||
| 609 | unsigned long offset = 0; | 610 | unsigned long offset = 0; |
| 611 | unsigned long remap_end; | ||
| 610 | struct scatterlist *sg; | 612 | struct scatterlist *sg; |
| 613 | unsigned long length; | ||
| 611 | int ret = 0; | 614 | int ret = 0; |
| 612 | 615 | ||
| 613 | if (drm_gem_is_imported(obj)) | 616 | if (drm_gem_is_imported(obj)) |
| @@ -615,11 +618,27 @@ static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struc · 1 unmodified line | |||
| 615 | 618 | ||
| 616 | for (sg = bo->sgt->sgl; sg; sg = sg_next(sg)) { | 619 | for (sg = bo->sgt->sgl; sg; sg = sg_next(sg)) { |
| 617 | if (sg_page(sg)) { | 620 | if (sg_page(sg)) { |
| 621 | /* if sg is too large for the VMA, so truncate it to fit */ | ||
| 622 | if (check_add_overflow(vma->vm_start, offset, &remap_start)) | ||
| 623 | return -EINVAL; | ||
| 624 | if (check_add_overflow(remap_start, sg->length, &remap_end)) | ||
| 625 | return -EINVAL; | ||
| 626 | |||
| 627 | if (remap_end > vma->vm_end) { | ||
| 628 | if (check_sub_overflow(vma->vm_end, remap_start, &length)) | ||
| 629 | return -EINVAL; | ||
| 630 | } else { | ||
| 631 | length = sg->length; | ||
| 632 | } | ||
| 633 | |||
| 634 | if (length == 0) | ||
| 635 | goto out; | ||
| 636 | |||
| 618 | ret = remap_pfn_range(vma, vma->vm_start + offset, page_to_pfn(sg_page(sg)), | 637 | ret = remap_pfn_range(vma, vma->vm_start + offset, page_to_pfn(sg_page(sg)), |
| 619 | sg->length, vma->vm_page_prot); | 638 | length, vma->vm_page_prot); |
| 620 | if (ret) | 639 | if (ret) |
| 621 | goto out; | 640 | goto out; |
| 622 | offset += sg->length; | 641 | offset += length; |
| 623 | } | 642 | } |
| 624 | } | 643 | } |
| 625 | 644 | ||
| @@ -497,11 +497,11 @@ static void decode_ras_msg(struct qaic_device *qdev, struct ras_data *msg) · 496 unmodified lines | |||
| 497 | qdev->ce_count++; | 497 | qdev->ce_count++; |
| 498 | break; | 498 | break; |
| 499 | case UE: | 499 | case UE: |
| 500 | if (qdev->ce_count != UINT_MAX) | 500 | if (qdev->ue_count != UINT_MAX) |
| 501 | qdev->ue_count++; | 501 | qdev->ue_count++; |
| 502 | break; | 502 | break; |
| 503 | case UE_NF: | 503 | case UE_NF: |
| 504 | if (qdev->ce_count != UINT_MAX) | 504 | if (qdev->ue_nf_count != UINT_MAX) |
| 505 | qdev->ue_nf_count++; | 505 | qdev->ue_nf_count++; |
| 506 | break; | 506 | break; |
| 507 | default: | 507 | default: |
| @@ -79,11 +79,6 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 78 unmodified lines | |||
| 79 | rkt_obj->size = args->size; | 79 | rkt_obj->size = args->size; |
| 80 | rkt_obj->offset = 0; | 80 | rkt_obj->offset = 0; |
| 81 | 81 | ||
| 82 | ret = drm_gem_handle_create(file, gem_obj, &args->handle); | ||
| 83 | drm_gem_object_put(gem_obj); | ||
| 84 | if (ret) | ||
| 85 | goto err; | ||
| 86 | |||
| 87 | sgt = drm_gem_shmem_get_pages_sgt(shmem_obj); | 82 | sgt = drm_gem_shmem_get_pages_sgt(shmem_obj); |
| 88 | if (IS_ERR(sgt)) { | 83 | if (IS_ERR(sgt)) { |
| 89 | ret = PTR_ERR(sgt); | 84 | ret = PTR_ERR(sgt); |
| @@ -95,6 +90,8 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 5 unmodified lines | |||
| 95 | rkt_obj->size, PAGE_SIZE, | 90 | rkt_obj->size, PAGE_SIZE, |
| 96 | 0, 0); | 91 | 0, 0); |
| 97 | mutex_unlock(&rocket_priv->mm_lock); | 92 | mutex_unlock(&rocket_priv->mm_lock); |
| 93 | if (ret) | ||
| 94 | goto err; | ||
| 98 | 95 | ||
| 99 | ret = iommu_map_sgtable(rocket_priv->domain->domain, | 96 | ret = iommu_map_sgtable(rocket_priv->domain->domain, |
| 100 | rkt_obj->mm.start, | 97 | rkt_obj->mm.start, |
| @@ -112,8 +109,18 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 11 unmodified lines | |||
| 112 | args->offset = drm_vma_node_offset_addr(&gem_obj->vma_node); | 109 | args->offset = drm_vma_node_offset_addr(&gem_obj->vma_node); |
| 113 | args->dma_address = rkt_obj->mm.start; | 110 | args->dma_address = rkt_obj->mm.start; |
| 114 | 111 | ||
| 112 | ret = drm_gem_handle_create(file, gem_obj, &args->handle); | ||
| 113 | if (ret) | ||
| 114 | goto err_unmap; | ||
| 115 | |||
| 116 | drm_gem_object_put(gem_obj); | ||
| 117 | |||
| 115 | return 0; | 118 | return 0; |
| 116 | 119 | ||
| 120 | err_unmap: | ||
| 121 | iommu_unmap(rocket_priv->domain->domain, | ||
| 122 | rkt_obj->mm.start, rkt_obj->size); | ||
| 123 | |||
| 117 | err_remove_node: | 124 | err_remove_node: |
| 118 | mutex_lock(&rocket_priv->mm_lock); | 125 | mutex_lock(&rocket_priv->mm_lock); |
| 119 | drm_mm_remove_node(&rkt_obj->mm); | 126 | drm_mm_remove_node(&rkt_obj->mm); |
| @@ -145,6 +152,8 @@ int rocket_ioctl_prep_bo(struct drm_device *dev, void *data, struct drm_file *fi · 25 unmodified lines | |||
| 145 | ret = dma_resv_wait_timeout(gem_obj->resv, DMA_RESV_USAGE_WRITE, true, timeout); | 152 | ret = dma_resv_wait_timeout(gem_obj->resv, DMA_RESV_USAGE_WRITE, true, timeout); |
| 146 | if (!ret) | 153 | if (!ret) |
| 147 | ret = timeout ? -ETIMEDOUT : -EBUSY; | 154 | ret = timeout ? -ETIMEDOUT : -EBUSY; |
| 155 | else if (ret > 0) | ||
| 156 | ret = 0; | ||
| 148 | 157 | ||
| 149 | shmem_obj = &to_rocket_bo(gem_obj)->base; | 158 | shmem_obj = &to_rocket_bo(gem_obj)->base; |
| 150 | 159 | ||
| @@ -192,11 +192,15 @@ static const struct dmi_system_id ac_dmi_table[] __initconst = { · 191 unmodified lines | |||
| 192 | 192 | ||
| 193 | static int acpi_ac_probe(struct platform_device *pdev) | 193 | static int acpi_ac_probe(struct platform_device *pdev) |
| 194 | { | 194 | { |
| 195 | struct acpi_device *adev = ACPI_COMPANION(&pdev->dev); | ||
| 196 | struct power_supply_config psy_cfg = {}; | 195 | struct power_supply_config psy_cfg = {}; |
| 196 | struct acpi_device *adev; | ||
| 197 | struct acpi_ac *ac; | 197 | struct acpi_ac *ac; |
| 198 | int result; | 198 | int result; |
| 199 | 199 | ||
| 200 | adev = ACPI_COMPANION(&pdev->dev); | ||
| 201 | if (!adev) | ||
| 202 | return -ENODEV; | ||
| 203 | |||
| 200 | ac = kzalloc_obj(struct acpi_ac); | 204 | ac = kzalloc_obj(struct acpi_ac); |
| 201 | if (!ac) | 205 | if (!ac) |
| 202 | return -ENOMEM; | 206 | return -ENOMEM; |
| @@ -423,7 +423,11 @@ static void acpi_pad_notify(acpi_handle handle, u32 event, void *data) · 422 unmodified lines | |||
| 423 | 423 | ||
| 424 | static int acpi_pad_probe(struct platform_device *pdev) | 424 | static int acpi_pad_probe(struct platform_device *pdev) |
| 425 | { | 425 | { |
| 426 | struct acpi_device *adev = ACPI_COMPANION(&pdev->dev); | 426 | struct acpi_device *adev; |
| 427 | |||
| 428 | adev = ACPI_COMPANION(&pdev->dev); | ||
| 429 | if (!adev) | ||
| 430 | return -ENODEV; | ||
| 427 | 431 | ||
| 428 | return acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, | 432 | return acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, |
| 429 | acpi_pad_notify, adev); | 433 | acpi_pad_notify, adev); |
| @@ -605,15 +605,12 @@ static umode_t acpi_tad_attr_is_visible(struct kobject *kobj, · 604 unmodified lines | |||
| 605 | return 0; | 605 | return 0; |
| 606 | } | 606 | } |
| 607 | 607 | ||
| 608 | static const struct attribute_group acpi_tad_attr_group = { | 608 | static const struct attribute_group acpi_tad_group = { |
| 609 | .attrs = acpi_tad_attrs, | 609 | .attrs = acpi_tad_attrs, |
| 610 | .is_visible = acpi_tad_attr_is_visible, | 610 | .is_visible = acpi_tad_attr_is_visible, |
| 611 | }; | 611 | }; |
| 612 | 612 | ||
| 613 | static const struct attribute_group *acpi_tad_attr_groups[] = { | 613 | __ATTRIBUTE_GROUPS(acpi_tad); |
| 614 | &acpi_tad_attr_group, | ||
| 615 | NULL, | ||
| 616 | }; | ||
| 617 | 614 | ||
| 618 | #ifdef CONFIG_RTC_CLASS | 615 | #ifdef CONFIG_RTC_CLASS |
| 619 | /* RTC class device interface */ | 616 | /* RTC class device interface */ |
| @@ -683,9 +680,8 @@ static int acpi_tad_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *t) · 63 unmodified lines | |||
| 683 | 680 | ||
| 684 | acpi_tad_rt_to_tm(&rt, &tm_now); | 681 | acpi_tad_rt_to_tm(&rt, &tm_now); |
| 685 | 682 | ||
| 686 | value = ktime_divns(ktime_sub(rtc_tm_to_ktime(t->time), | 683 | value = rtc_tm_to_time64(&t->time) - rtc_tm_to_time64(&tm_now); |
| 687 | rtc_tm_to_ktime(tm_now)), NSEC_PER_SEC); | 684 | if (value <= 0 || value >= U32_MAX) |
| 688 | if (value <= 0 || value > U32_MAX) | ||
| 689 | return -EINVAL; | 685 | return -EINVAL; |
| 690 | } | 686 | } |
| 691 | 687 | ||
| @@ -748,8 +744,7 @@ static int acpi_tad_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *t) · 56 unmodified lines | |||
| 748 | 744 | ||
| 749 | if (retval != ACPI_TAD_WAKE_DISABLED) { | 745 | if (retval != ACPI_TAD_WAKE_DISABLED) { |
| 750 | t->enabled = 1; | 746 | t->enabled = 1; |
| 751 | t->time = rtc_ktime_to_tm(ktime_add_ns(rtc_tm_to_ktime(tm_now), | 747 | rtc_time64_to_tm(rtc_tm_to_time64(&tm_now) + retval, &t->time); |
| 752 | (u64)retval * NSEC_PER_SEC)); | ||
| 753 | } else { | 748 | } else { |
| 754 | t->enabled = 0; | 749 | t->enabled = 0; |
| 755 | t->time = tm_now; | 750 | t->time = tm_now; |
| @@ -795,9 +790,9 @@ static int acpi_tad_disable_timer(struct device *dev, u32 timer_id) · 39 unmodified lines | |||
| 795 | return acpi_tad_wake_set(dev, "_STV", timer_id, ACPI_TAD_WAKE_DISABLED); | 790 | return acpi_tad_wake_set(dev, "_STV", timer_id, ACPI_TAD_WAKE_DISABLED); |
| 796 | } | 791 | } |
| 797 | 792 | ||
| 798 | static void acpi_tad_remove(struct platform_device *pdev) | 793 | static void acpi_tad_remove(void *data) |
| 799 | { | 794 | { |
| 800 | struct device *dev = &pdev->dev; | 795 | struct device *dev = data; |
| 801 | struct acpi_tad_driver_data *dd = dev_get_drvdata(dev); | 796 | struct acpi_tad_driver_data *dd = dev_get_drvdata(dev); |
| 802 | 797 | ||
| 803 | device_init_wakeup(dev, false); | 798 | device_init_wakeup(dev, false); |
| @@ -820,10 +815,15 @@ static void acpi_tad_remove(struct platform_device *pdev) · 16 unmodified lines | |||
| 820 | static int acpi_tad_probe(struct platform_device *pdev) | 815 | static int acpi_tad_probe(struct platform_device *pdev) |
| 821 | { | 816 | { |
| 822 | struct device *dev = &pdev->dev; | 817 | struct device *dev = &pdev->dev; |
| 823 | acpi_handle handle = ACPI_HANDLE(dev); | ||
| 824 | struct acpi_tad_driver_data *dd; | 818 | struct acpi_tad_driver_data *dd; |
| 819 | acpi_handle handle; | ||
| 825 | acpi_status status; | 820 | acpi_status status; |
| 826 | unsigned long long caps; | 821 | unsigned long long caps; |
| 822 | int ret; | ||
| 823 | |||
| 824 | handle = ACPI_HANDLE(dev); | ||
| 825 | if (!handle) | ||
| 826 | return -ENODEV; | ||
| 827 | 827 | ||
| 828 | /* | 828 | /* |
| 829 | * Initialization failure messages are mostly about firmware issues, so | 829 | * Initialization failure messages are mostly about firmware issues, so |
| @@ -863,13 +863,21 @@ static int acpi_tad_probe(struct platform_device *pdev) · 33 unmodified lines | |||
| 863 | } | 863 | } |
| 864 | 864 | ||
| 865 | /* | 865 | /* |
| 866 | * The platform bus type layer tells the ACPI PM domain powers up the | 866 | * The platform bus type probe callback tells the ACPI PM domain to |
| 867 | * device, so set the runtime PM status of it to "active". | 867 | * power up the device, so set the runtime PM status of it to "active". |
| 868 | */ | 868 | */ |
| 869 | pm_runtime_set_active(dev); | 869 | pm_runtime_set_active(dev); |
| 870 | pm_runtime_enable(dev); | 870 | pm_runtime_enable(dev); |
| 871 | pm_runtime_suspend(dev); | 871 | pm_runtime_suspend(dev); |
| 872 | 872 | ||
| 873 | /* | ||
| 874 | * acpi_tad_remove() needs to run after unregistering the RTC class | ||
| 875 | * device to avoid racing with the latter's callbacks. | ||
| 876 | */ | ||
| 877 | ret = devm_add_action_or_reset(&pdev->dev, acpi_tad_remove, &pdev->dev); | ||
| 878 | if (ret) | ||
| 879 | return ret; | ||
| 880 | |||
| 873 | if (caps & ACPI_TAD_RT) | 881 | if (caps & ACPI_TAD_RT) |
| 874 | acpi_tad_register_rtc(dev, caps); | 882 | acpi_tad_register_rtc(dev, caps); |
| 875 | 883 | ||
| @@ -885,10 +893,9 @@ static struct platform_driver acpi_tad_driver = { · 9 unmodified lines | |||
| 885 | .driver = { | 893 | .driver = { |
| 886 | .name = "acpi-tad", | 894 | .name = "acpi-tad", |
| 887 | .acpi_match_table = acpi_tad_ids, | 895 | .acpi_match_table = acpi_tad_ids, |
| 888 | .dev_groups = acpi_tad_attr_groups, | 896 | .dev_groups = acpi_tad_groups, |
| 889 | }, | 897 | }, |
| 890 | .probe = acpi_tad_probe, | 898 | .probe = acpi_tad_probe, |
| 891 | .remove = acpi_tad_remove, | ||
| 892 | }; | 899 | }; |
| 893 | MODULE_DEVICE_TABLE(acpi, acpi_tad_ids); | 900 | MODULE_DEVICE_TABLE(acpi, acpi_tad_ids); |
| 894 | 901 | ||
| @@ -78,18 +78,22 @@ ACPI_EXPORT_SYMBOL(acpi_update_all_gpes) · 77 unmodified lines | |||
| 78 | 78 | ||
| 79 | /******************************************************************************* | 79 | /******************************************************************************* |
| 80 | * | 80 | * |
| 81 | * FUNCTION: acpi_enable_gpe | 81 | * FUNCTION: acpi_enable_gpe_cond |
| 82 | * | 82 | * |
| 83 | * PARAMETERS: gpe_device - Parent GPE Device. NULL for GPE0/GPE1 | 83 | * PARAMETERS: gpe_device - Parent GPE Device. NULL for GPE0/GPE1 |
| 84 | * gpe_number - GPE level within the GPE block | 84 | * gpe_number - GPE level within the GPE block |
| 85 | * dispatch_type - GPE dispatch type to match | ||
| 85 | * | 86 | * |
| 86 | * RETURN: Status | 87 | * RETURN: Status |
| 87 | * | 88 | * |
| 88 | * DESCRIPTION: Add a reference to a GPE. On the first reference, the GPE is | 89 | * DESCRIPTION: Add a reference to a GPE so long as its dispatch type matches |
| 89 | * hardware-enabled. | 90 | * the supplied one, or it is different from ACPI_GPE_DISPATCH_NONE |
| 91 | * if the supplied one is ACPI_GPE_DISPATCH_MASK. On the first | ||
| 92 | * reference, the GPE is hardware-enabled. | ||
| 90 | * | 93 | * |
| 91 | ******************************************************************************/ | 94 | ******************************************************************************/ |
| 92 | acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) | 95 | acpi_status acpi_enable_gpe_cond(acpi_handle gpe_device, u32 gpe_number, |
| 96 | u8 dispatch_type) | ||
| 93 | { | 97 | { |
| 94 | acpi_status status = AE_BAD_PARAMETER; | 98 | acpi_status status = AE_BAD_PARAMETER; |
| 95 | struct acpi_gpe_event_info *gpe_event_info; | 99 | struct acpi_gpe_event_info *gpe_event_info; |
| @@ -100,14 +104,18 @@ acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) · 4 unmodified lines | |||
| 100 | flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock); | 104 | flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock); |
| 101 | 105 | ||
| 102 | /* | 106 | /* |
| 103 | * Ensure that we have a valid GPE number and that there is some way | 107 | * Ensure that we have a valid GPE number and that the dispatch type of |
| 104 | * of handling the GPE (handler or a GPE method). In other words, we | 108 | * the GPE matches the supplied one (or it is not ACPI_GPE_DISPATCH_NONE |
| 105 | * won't allow a valid GPE to be enabled if there is no way to handle it. | 109 | * if the supplied one is ACPI_GPE_DISPATCH_MASK). |
| 106 | */ | 110 | */ |
| 107 | gpe_event_info = acpi_ev_get_gpe_event_info(gpe_device, gpe_number); | 111 | gpe_event_info = acpi_ev_get_gpe_event_info(gpe_device, gpe_number); |
| 108 | if (gpe_event_info) { | 112 | if (gpe_event_info) { |
| 109 | if (ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags) != | 113 | if (dispatch_type == ACPI_GPE_DISPATCH_MASK) |
| 110 | ACPI_GPE_DISPATCH_NONE) { | 114 | dispatch_type = ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags); |
| 115 | else if (dispatch_type != ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags)) | ||
| 116 | dispatch_type = ACPI_GPE_DISPATCH_NONE; | ||
| 117 | |||
| 118 | if (dispatch_type != ACPI_GPE_DISPATCH_NONE) { | ||
| 111 | status = acpi_ev_add_gpe_reference(gpe_event_info, TRUE); | 119 | status = acpi_ev_add_gpe_reference(gpe_event_info, TRUE); |
| 112 | if (ACPI_SUCCESS(status) && | 120 | if (ACPI_SUCCESS(status) && |
| 113 | ACPI_GPE_IS_POLLING_NEEDED(gpe_event_info)) { | 121 | ACPI_GPE_IS_POLLING_NEEDED(gpe_event_info)) { |
| @@ -128,6 +136,30 @@ acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) · 14 unmodified lines | |||
| 128 | acpi_os_release_lock(acpi_gbl_gpe_lock, flags); | 136 | acpi_os_release_lock(acpi_gbl_gpe_lock, flags); |
| 129 | return_ACPI_STATUS(status); | 137 | return_ACPI_STATUS(status); |
| 130 | } | 138 | } |
| 139 | ACPI_EXPORT_SYMBOL(acpi_enable_gpe_cond) | ||
| 140 | |||
| 141 | /******************************************************************************* | ||
| 142 | * | ||
| 143 | * FUNCTION: acpi_enable_gpe | ||
| 144 | * | ||
| 145 | * PARAMETERS: gpe_device - Parent GPE Device. NULL for GPE0/GPE1 | ||
| 146 | * gpe_number - GPE level within the GPE block | ||
| 147 | * | ||
| 148 | * RETURN: Status | ||
| 149 | * | ||
| 150 | * DESCRIPTION: Add a reference to a GPE. On the first reference, the GPE is | ||
| 151 | * hardware-enabled. | ||
| 152 | * | ||
| 153 | ******************************************************************************/ | ||
| 154 | acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) | ||
| 155 | { | ||
| 156 | /* | ||
| 157 | * Ensure that there is some way of handling the GPE (handler or a GPE | ||
| 158 | * method). In other words, we won't allow a valid GPE to be enabled if | ||
| 159 | * there is no way to handle it. | ||
| 160 | */ | ||
| 161 | return acpi_enable_gpe_cond(gpe_device, gpe_number, ACPI_GPE_DISPATCH_MASK); | ||
| 162 | } | ||
| 131 | ACPI_EXPORT_SYMBOL(acpi_enable_gpe) | 163 | ACPI_EXPORT_SYMBOL(acpi_enable_gpe) |
| 132 | 164 | ||
| 133 | /******************************************************************************* | 165 | /******************************************************************************* |
| @@ -401,8 +401,18 @@ static struct acpi_generic_address *einj_get_trigger_parameter_region( · 400 unmodified lines | |||
| 401 | 401 | ||
| 402 | return NULL; | 402 | return NULL; |
| 403 | } | 403 | } |
| 404 | |||
| 405 | static bool is_memory_injection(u32 type, u32 flags) | ||
| 406 | { | ||
| 407 | if (flags & SETWA_FLAGS_EINJV2) | ||
| 408 | return !!(type & ACPI_EINJV2_MEMORY); | ||
| 409 | if (type & ACPI5_VENDOR_BIT) | ||
| 410 | return !!(vendor_flags & SETWA_FLAGS_MEM); | ||
| 411 | return !!(type & MEM_ERROR_MASK) || !!(flags & SETWA_FLAGS_MEM); | ||
| 412 | } | ||
| 413 | |||
| 404 | /* Execute instructions in trigger error action table */ | 414 | /* Execute instructions in trigger error action table */ |
| 405 | static int __einj_error_trigger(u64 trigger_paddr, u32 type, | 415 | static int __einj_error_trigger(u64 trigger_paddr, u32 type, u32 flags, |
| 406 | u64 param1, u64 param2) | 416 | u64 param1, u64 param2) |
| 407 | { | 417 | { |
| 408 | struct acpi_einj_trigger trigger_tab; | 418 | struct acpi_einj_trigger trigger_tab; |
| @@ -480,7 +490,7 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, · 71 unmodified lines | |||
| 480 | * This will cause resource conflict with regular memory. So | 490 | * This will cause resource conflict with regular memory. So |
| 481 | * remove it from trigger table resources. | 491 | * remove it from trigger table resources. |
| 482 | */ | 492 | */ |
| 483 | if ((param_extension || acpi5) && (type & MEM_ERROR_MASK) && param2) { | 493 | if ((param_extension || acpi5) && is_memory_injection(type, flags)) { |
| 484 | struct apei_resources addr_resources; | 494 | struct apei_resources addr_resources; |
| 485 | 495 | ||
| 486 | apei_resources_init(&addr_resources); | 496 | apei_resources_init(&addr_resources); |
| @@ -660,7 +670,7 @@ static int __einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2, · 173 unmodified lines | |||
| 660 | return rc; | 670 | return rc; |
| 661 | trigger_paddr = apei_exec_ctx_get_output(&ctx); | 671 | trigger_paddr = apei_exec_ctx_get_output(&ctx); |
| 662 | if (notrigger == 0) { | 672 | if (notrigger == 0) { |
| 663 | rc = __einj_error_trigger(trigger_paddr, type, param1, param2); | 673 | rc = __einj_error_trigger(trigger_paddr, type, flags, param1, param2); |
| 664 | if (rc) | 674 | if (rc) |
| 665 | return rc; | 675 | return rc; |
| 666 | } | 676 | } |
| @@ -718,35 +728,30 @@ int einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2, u64 param3, · 51 unmodified lines | |||
| 718 | SETWA_FLAGS_PCIE_SBDF | SETWA_FLAGS_EINJV2))) | 728 | SETWA_FLAGS_PCIE_SBDF | SETWA_FLAGS_EINJV2))) |
| 719 | return -EINVAL; | 729 | return -EINVAL; |
| 720 | 730 | ||
| 731 | /* | ||
| 732 | * Injections targeting a CXL 1.0/1.1 port have to be injected | ||
| 733 | * via the einj_cxl_rch_error_inject() path as that does the proper | ||
| 734 | * validation of the given RCRB base (MMIO) address. | ||
| 735 | */ | ||
| 736 | if (einj_is_cxl_error_type(type) && (flags & SETWA_FLAGS_MEM)) | ||
| 737 | return -EINVAL; | ||
| 738 | |||
| 721 | /* check if type is a valid EINJv2 error type */ | 739 | /* check if type is a valid EINJv2 error type */ |
| 722 | if (is_v2) { | 740 | if (is_v2) { |
| 723 | if (!(type & available_error_type_v2)) | 741 | if (!(type & available_error_type_v2)) |
| 724 | return -EINVAL; | 742 | return -EINVAL; |
| 725 | } | 743 | } |
| 726 | /* | ||
| 727 | * We need extra sanity checks for memory errors. | ||
| 728 | * Other types leap directly to injection. | ||
| 729 | */ | ||
| 730 | 744 | ||
| 731 | /* ensure param1/param2 existed */ | 745 | /* ensure param1/param2 existed */ |
| 732 | if (!(param_extension || acpi5)) | 746 | if (!(param_extension || acpi5)) |
| 733 | goto inject; | 747 | goto inject; |
| 734 | 748 | ||
| 735 | /* ensure injection is memory related */ | ||
| 736 | if (type & ACPI5_VENDOR_BIT) { | ||
| 737 | if (vendor_flags != SETWA_FLAGS_MEM) | ||
| 738 | goto inject; | ||
| 739 | } else if (!(type & MEM_ERROR_MASK) && !(flags & SETWA_FLAGS_MEM)) { | ||
| 740 | goto inject; | ||
| 741 | } | ||
| 742 | |||
| 743 | /* | 749 | /* |
| 744 | * Injections targeting a CXL 1.0/1.1 port have to be injected | 750 | * We need extra sanity checks for memory errors. |
| 745 | * via the einj_cxl_rch_error_inject() path as that does the proper | 751 | * Other types leap directly to injection. |
| 746 | * validation of the given RCRB base (MMIO) address. | ||
| 747 | */ | 752 | */ |
| 748 | if (einj_is_cxl_error_type(type) && (flags & SETWA_FLAGS_MEM)) | 753 | if (!is_memory_injection(type, flags)) |
| 749 | return -EINVAL; | 754 | goto inject; |
| 750 | 755 | ||
| 751 | /* | 756 | /* |
| 752 | * Disallow crazy address masks that give BIOS leeway to pick | 757 | * Disallow crazy address masks that give BIOS leeway to pick |
| @@ -16,7 +16,7 @@ · 15 unmodified lines | |||
| 16 | 16 | ||
| 17 | static int psci_acpi_cpu_init_idle(unsigned int cpu) | 17 | static int psci_acpi_cpu_init_idle(unsigned int cpu) |
| 18 | { | 18 | { |
| 19 | int i, count; | 19 | int i; |
| 20 | struct acpi_lpi_state *lpi; | 20 | struct acpi_lpi_state *lpi; |
| 21 | struct acpi_processor *pr = per_cpu(processors, cpu); | 21 | struct acpi_processor *pr = per_cpu(processors, cpu); |
| 22 | 22 | ||
| @@ -30,14 +30,10 @@ static int psci_acpi_cpu_init_idle(unsigned int cpu) · 7 unmodified lines | |||
| 30 | if (!psci_ops.cpu_suspend) | 30 | if (!psci_ops.cpu_suspend) |
| 31 | return -EOPNOTSUPP; | 31 | return -EOPNOTSUPP; |
| 32 | 32 | ||
| 33 | count = pr->power.count - 1; | 33 | for (i = 1; i < pr->power.count; i++) { |
| 34 | if (count <= 0) | ||
| 35 | return -ENODEV; | ||
| 36 | |||
| 37 | for (i = 0; i < count; i++) { | ||
| 38 | u32 state; | 34 | u32 state; |
| 39 | 35 | ||
| 40 | lpi = &pr->power.lpi_states[i + 1]; | 36 | lpi = &pr->power.lpi_states[i]; |
| 41 | /* | 37 | /* |
| 42 | * Only bits[31:0] represent a PSCI power_state while | 38 | * Only bits[31:0] represent a PSCI power_state while |
| 43 | * bits[63:32] must be 0x0 as per ARM ACPI FFH Specification | 39 | * bits[63:32] must be 0x0 as per ARM ACPI FFH Specification |
| @@ -94,6 +94,7 @@ struct acpi_battery { · 93 unmodified lines | |||
| 94 | struct power_supply *bat; | 94 | struct power_supply *bat; |
| 95 | struct power_supply_desc bat_desc; | 95 | struct power_supply_desc bat_desc; |
| 96 | struct acpi_device *device; | 96 | struct acpi_device *device; |
| 97 | struct device *phys_dev; | ||
| 97 | struct notifier_block pm_nb; | 98 | struct notifier_block pm_nb; |
| 98 | struct list_head list; | 99 | struct list_head list; |
| 99 | unsigned long update_time; | 100 | unsigned long update_time; |
| @@ -1033,7 +1034,7 @@ static int acpi_battery_update(struct acpi_battery *battery, bool resume) · 933 unmodified lines | |||
| 1033 | if ((battery->state & ACPI_BATTERY_STATE_CRITICAL) || | 1034 | if ((battery->state & ACPI_BATTERY_STATE_CRITICAL) || |
| 1034 | (test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags) && | 1035 | (test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags) && |
| 1035 | (battery->capacity_now <= battery->alarm))) | 1036 | (battery->capacity_now <= battery->alarm))) |
| 1036 | acpi_pm_wakeup_event(&battery->device->dev); | 1037 | acpi_pm_wakeup_event(battery->phys_dev); |
| 1037 | 1038 | ||
| 1038 | return result; | 1039 | return result; |
| 1039 | } | 1040 | } |
| @@ -1214,10 +1215,14 @@ static void sysfs_battery_cleanup(struct acpi_battery *battery) · 174 unmodified lines | |||
| 1214 | 1215 | ||
| 1215 | static int acpi_battery_probe(struct platform_device *pdev) | 1216 | static int acpi_battery_probe(struct platform_device *pdev) |
| 1216 | { | 1217 | { |
| 1217 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | ||
| 1218 | struct acpi_battery *battery; | 1218 | struct acpi_battery *battery; |
| 1219 | struct acpi_device *device; | ||
| 1219 | int result; | 1220 | int result; |
| 1220 | 1221 | ||
| 1222 | device = ACPI_COMPANION(&pdev->dev); | ||
| 1223 | if (!device) | ||
| 1224 | return -ENODEV; | ||
| 1225 | |||
| 1221 | if (device->dep_unmet) | 1226 | if (device->dep_unmet) |
| 1222 | return -EPROBE_DEFER; | 1227 | return -EPROBE_DEFER; |
| 1223 | 1228 | ||
| @@ -1227,6 +1232,7 @@ static int acpi_battery_probe(struct platform_device *pdev) · 3 unmodified lines | |||
| 1227 | 1232 | ||
| 1228 | platform_set_drvdata(pdev, battery); | 1233 | platform_set_drvdata(pdev, battery); |
| 1229 | 1234 | ||
| 1235 | battery->phys_dev = &pdev->dev; | ||
| 1230 | battery->device = device; | 1236 | battery->device = device; |
| 1231 | 1237 | ||
| 1232 | result = devm_mutex_init(&pdev->dev, &battery->update_lock); | 1238 | result = devm_mutex_init(&pdev->dev, &battery->update_lock); |
| @@ -179,6 +179,7 @@ struct acpi_button { · 178 unmodified lines | |||
| 179 | ktime_t last_time; | 179 | ktime_t last_time; |
| 180 | bool suspended; | 180 | bool suspended; |
| 181 | bool lid_state_initialized; | 181 | bool lid_state_initialized; |
| 182 | bool gpe_enabled; | ||
| 182 | }; | 183 | }; |
| 183 | 184 | ||
| 184 | static struct acpi_device *lid_device; | 185 | static struct acpi_device *lid_device; |
| @@ -531,15 +532,20 @@ static int acpi_lid_input_open(struct input_dev *input) · 346 unmodified lines | |||
| 531 | 532 | ||
| 532 | static int acpi_button_probe(struct platform_device *pdev) | 533 | static int acpi_button_probe(struct platform_device *pdev) |
| 533 | { | 534 | { |
| 534 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | ||
| 535 | acpi_notify_handler handler; | 535 | acpi_notify_handler handler; |
| 536 | struct acpi_device *device; | ||
| 536 | struct acpi_button *button; | 537 | struct acpi_button *button; |
| 537 | struct input_dev *input; | 538 | struct input_dev *input; |
| 538 | const char *hid = acpi_device_hid(device); | ||
| 539 | acpi_status status; | 539 | acpi_status status; |
| 540 | char *name, *class; | 540 | char *name, *class; |
| 541 | const char *hid; | ||
| 541 | int error = 0; | 542 | int error = 0; |
| 542 | 543 | ||
| 544 | device = ACPI_COMPANION(&pdev->dev); | ||
| 545 | if (!device) | ||
| 546 | return -ENODEV; | ||
| 547 | |||
| 548 | hid = acpi_device_hid(device); | ||
| 543 | if (!strcmp(hid, ACPI_BUTTON_HID_LID) && | 549 | if (!strcmp(hid, ACPI_BUTTON_HID_LID) && |
| 544 | lid_init_state == ACPI_BUTTON_LID_INIT_DISABLED) | 550 | lid_init_state == ACPI_BUTTON_LID_INIT_DISABLED) |
| 545 | return -ENODEV; | 551 | return -ENODEV; |
| @@ -641,6 +647,21 @@ static int acpi_button_probe(struct platform_device *pdev) · 95 unmodified lines | |||
| 641 | status = acpi_install_notify_handler(device->handle, | 647 | status = acpi_install_notify_handler(device->handle, |
| 642 | ACPI_ALL_NOTIFY, handler, | 648 | ACPI_ALL_NOTIFY, handler, |
| 643 | button); | 649 | button); |
| 650 | if (ACPI_SUCCESS(status) && device->wakeup.flags.valid) { | ||
| 651 | acpi_status st; | ||
| 652 | |||
| 653 | /* | ||
| 654 | * If the wakeup GPE has a handler method, enable it in | ||
| 655 | * case it is also used for signaling runtime events. | ||
| 656 | */ | ||
| 657 | st = acpi_enable_gpe_cond(device->wakeup.gpe_device, | ||
| 658 | device->wakeup.gpe_number, | ||
| 659 | ACPI_GPE_DISPATCH_METHOD); | ||
| 660 | button->gpe_enabled = ACPI_SUCCESS(st); | ||
| 661 | if (button->gpe_enabled) | ||
| 662 | dev_dbg(button->dev, "Enabled ACPI GPE%02llx\n", | ||
| 663 | device->wakeup.gpe_number); | ||
| 664 | } | ||
| 644 | break; | 665 | break; |
| 645 | } | 666 | } |
| 646 | if (ACPI_FAILURE(status)) { | 667 | if (ACPI_FAILURE(status)) { |
| @@ -666,6 +687,7 @@ err_remove_fs: · 19 unmodified lines | |||
| 666 | acpi_button_remove_fs(button); | 687 | acpi_button_remove_fs(button); |
| 667 | err_free_button: | 688 | err_free_button: |
| 668 | kfree(button); | 689 | kfree(button); |
| 690 | memset(acpi_device_class(device), 0, sizeof(acpi_device_class)); | ||
| 669 | return error; | 691 | return error; |
| 670 | } | 692 | } |
| 671 | 693 | ||
| @@ -684,7 +706,13 @@ static void acpi_button_remove(struct platform_device *pdev) · 12 unmodified lines | |||
| 684 | acpi_button_event); | 706 | acpi_button_event); |
| 685 | break; | 707 | break; |
| 686 | default: | 708 | default: |
| 687 | acpi_remove_notify_handler(adev->handle, ACPI_DEVICE_NOTIFY, | 709 | if (button->gpe_enabled) { |
| 710 | dev_dbg(button->dev, "Disabling ACPI GPE%02llx\n", | ||
| 711 | adev->wakeup.gpe_number); | ||
| 712 | acpi_disable_gpe(adev->wakeup.gpe_device, | ||
| 713 | adev->wakeup.gpe_number); | ||
| 714 | } | ||
| 715 | acpi_remove_notify_handler(adev->handle, ACPI_ALL_NOTIFY, | ||
| 688 | button->type == ACPI_BUTTON_TYPE_LID ? | 716 | button->type == ACPI_BUTTON_TYPE_LID ? |
| 689 | acpi_lid_notify : | 717 | acpi_lid_notify : |
| 690 | acpi_button_notify); | 718 | acpi_button_notify); |
| @@ -362,7 +362,7 @@ static int send_pcc_cmd(int pcc_ss_id, u16 cmd) · 361 unmodified lines | |||
| 362 | end: | 362 | end: |
| 363 | if (cmd == CMD_WRITE) { | 363 | if (cmd == CMD_WRITE) { |
| 364 | if (unlikely(ret)) { | 364 | if (unlikely(ret)) { |
| 365 | for_each_online_cpu(i) { | 365 | for_each_possible_cpu(i) { |
| 366 | struct cpc_desc *desc = per_cpu(cpc_desc_ptr, i); | 366 | struct cpc_desc *desc = per_cpu(cpc_desc_ptr, i); |
| 367 | 367 | ||
| 368 | if (!desc) | 368 | if (!desc) |
| @@ -524,13 +524,13 @@ int acpi_get_psd_map(unsigned int cpu, struct cppc_cpudata *cpu_data) · 155 unmodified lines | |||
| 524 | else if (pdomain->coord_type == DOMAIN_COORD_TYPE_SW_ANY) | 524 | else if (pdomain->coord_type == DOMAIN_COORD_TYPE_SW_ANY) |
| 525 | cpu_data->shared_type = CPUFREQ_SHARED_TYPE_ANY; | 525 | cpu_data->shared_type = CPUFREQ_SHARED_TYPE_ANY; |
| 526 | 526 | ||
| 527 | for_each_online_cpu(i) { | 527 | for_each_possible_cpu(i) { |
| 528 | if (i == cpu) | 528 | if (i == cpu) |
| 529 | continue; | 529 | continue; |
| 530 | 530 | ||
| 531 | match_cpc_ptr = per_cpu(cpc_desc_ptr, i); | 531 | match_cpc_ptr = per_cpu(cpc_desc_ptr, i); |
| 532 | if (!match_cpc_ptr) | 532 | if (!match_cpc_ptr) |
| 533 | goto err_fault; | 533 | continue; |
| 534 | 534 | ||
| 535 | match_pdomain = &(match_cpc_ptr->domain_info); | 535 | match_pdomain = &(match_cpc_ptr->domain_info); |
| 536 | if (match_pdomain->domain != pdomain->domain) | 536 | if (match_pdomain->domain != pdomain->domain) |
| @@ -1676,10 +1676,14 @@ static int acpi_ec_setup(struct acpi_ec *ec, struct acpi_device *device, bool ca · 1675 unmodified lines | |||
| 1676 | 1676 | ||
| 1677 | static int acpi_ec_probe(struct platform_device *pdev) | 1677 | static int acpi_ec_probe(struct platform_device *pdev) |
| 1678 | { | 1678 | { |
| 1679 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 1679 | struct acpi_device *device; |
| 1680 | struct acpi_ec *ec; | 1680 | struct acpi_ec *ec; |
| 1681 | int ret; | 1681 | int ret; |
| 1682 | 1682 | ||
| 1683 | device = ACPI_COMPANION(&pdev->dev); | ||
| 1684 | if (!device) | ||
| 1685 | return -ENODEV; | ||
| 1686 | |||
| 1683 | if (boot_ec && (boot_ec->handle == device->handle || | 1687 | if (boot_ec && (boot_ec->handle == device->handle || |
| 1684 | !strcmp(acpi_device_hid(device), ACPI_ECDT_HID))) { | 1688 | !strcmp(acpi_device_hid(device), ACPI_ECDT_HID))) { |
| 1685 | /* Fast path: this device corresponds to the boot EC. */ | 1689 | /* Fast path: this device corresponds to the boot EC. */ |
| @@ -50,9 +50,13 @@ static void acpi_hed_notify(acpi_handle handle, u32 event, void *data) · 49 unmodified lines | |||
| 50 | 50 | ||
| 51 | static int acpi_hed_probe(struct platform_device *pdev) | 51 | static int acpi_hed_probe(struct platform_device *pdev) |
| 52 | { | 52 | { |
| 53 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 53 | struct acpi_device *device; |
| 54 | int err; | 54 | int err; |
| 55 | 55 | ||
| 56 | device = ACPI_COMPANION(&pdev->dev); | ||
| 57 | if (!device) | ||
| 58 | return -ENODEV; | ||
| 59 | |||
| 56 | /* Only one hardware error device */ | 60 | /* Only one hardware error device */ |
| 57 | if (hed_handle) | 61 | if (hed_handle) |
| 58 | return -EINVAL; | 62 | return -EINVAL; |
| @@ -3341,12 +3341,16 @@ static int acpi_nfit_probe(struct platform_device *pdev) · 3340 unmodified lines | |||
| 3341 | struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL }; | 3341 | struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL }; |
| 3342 | struct acpi_nfit_desc *acpi_desc; | 3342 | struct acpi_nfit_desc *acpi_desc; |
| 3343 | struct device *dev = &pdev->dev; | 3343 | struct device *dev = &pdev->dev; |
| 3344 | struct acpi_device *adev = ACPI_COMPANION(dev); | ||
| 3345 | struct acpi_table_header *tbl; | 3344 | struct acpi_table_header *tbl; |
| 3345 | struct acpi_device *adev; | ||
| 3346 | acpi_status status = AE_OK; | 3346 | acpi_status status = AE_OK; |
| 3347 | acpi_size sz; | 3347 | acpi_size sz; |
| 3348 | int rc = 0; | 3348 | int rc = 0; |
| 3349 | 3349 | ||
| 3350 | adev = ACPI_COMPANION(&pdev->dev); | ||
| 3351 | if (!adev) | ||
| 3352 | return -ENODEV; | ||
| 3353 | |||
| 3350 | rc = acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, | 3354 | rc = acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, |
| 3351 | acpi_nfit_notify, dev); | 3355 | acpi_nfit_notify, dev); |
| 3352 | if (rc) | 3356 | if (rc) |
| @@ -360,10 +360,14 @@ static void pfrt_log_put_idx(void *data) · 359 unmodified lines | |||
| 360 | 360 | ||
| 361 | static int acpi_pfrt_log_probe(struct platform_device *pdev) | 361 | static int acpi_pfrt_log_probe(struct platform_device *pdev) |
| 362 | { | 362 | { |
| 363 | acpi_handle handle = ACPI_HANDLE(&pdev->dev); | ||
| 364 | struct pfrt_log_device *pfrt_log_dev; | 363 | struct pfrt_log_device *pfrt_log_dev; |
| 364 | acpi_handle handle; | ||
| 365 | int ret; | 365 | int ret; |
| 366 | 366 | ||
| 367 | handle = ACPI_HANDLE(&pdev->dev); | ||
| 368 | if (!handle) | ||
| 369 | return -ENODEV; | ||
| 370 | |||
| 367 | if (!acpi_has_method(handle, "_DSM")) { | 371 | if (!acpi_has_method(handle, "_DSM")) { |
| 368 | dev_dbg(&pdev->dev, "Missing _DSM\n"); | 372 | dev_dbg(&pdev->dev, "Missing _DSM\n"); |
| 369 | return -ENODEV; | 373 | return -ENODEV; |
| @@ -538,10 +538,14 @@ static void pfru_put_idx(void *data) · 537 unmodified lines | |||
| 538 | 538 | ||
| 539 | static int acpi_pfru_probe(struct platform_device *pdev) | 539 | static int acpi_pfru_probe(struct platform_device *pdev) |
| 540 | { | 540 | { |
| 541 | acpi_handle handle = ACPI_HANDLE(&pdev->dev); | ||
| 542 | struct pfru_device *pfru_dev; | 541 | struct pfru_device *pfru_dev; |
| 542 | acpi_handle handle; | ||
| 543 | int ret; | 543 | int ret; |
| 544 | 544 | ||
| 545 | handle = ACPI_HANDLE(&pdev->dev); | ||
| 546 | if (!handle) | ||
| 547 | return -ENODEV; | ||
| 548 | |||
| 545 | if (!acpi_has_method(handle, "_DSM")) { | 549 | if (!acpi_has_method(handle, "_DSM")) { |
| 546 | dev_dbg(&pdev->dev, "Missing _DSM\n"); | 550 | dev_dbg(&pdev->dev, "Missing _DSM\n"); |
| 547 | return -ENODEV; | 551 | return -ENODEV; |
| @@ -629,11 +629,15 @@ static void acpi_sbs_callback(void *context) · 628 unmodified lines | |||
| 629 | 629 | ||
| 630 | static int acpi_sbs_probe(struct platform_device *pdev) | 630 | static int acpi_sbs_probe(struct platform_device *pdev) |
| 631 | { | 631 | { |
| 632 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 632 | struct acpi_device *device; |
| 633 | struct acpi_sbs *sbs; | 633 | struct acpi_sbs *sbs; |
| 634 | int result = 0; | 634 | int result = 0; |
| 635 | int id; | 635 | int id; |
| 636 | 636 | ||
| 637 | device = ACPI_COMPANION(&pdev->dev); | ||
| 638 | if (!device) | ||
| 639 | return -ENODEV; | ||
| 640 | |||
| 637 | sbs = kzalloc_obj(struct acpi_sbs); | 641 | sbs = kzalloc_obj(struct acpi_sbs); |
| 638 | if (!sbs) { | 642 | if (!sbs) { |
| 639 | result = -ENOMEM; | 643 | result = -ENOMEM; |
| @@ -237,11 +237,15 @@ static int smbus_alarm(void *context) · 236 unmodified lines | |||
| 237 | 237 | ||
| 238 | static int acpi_smbus_hc_probe(struct platform_device *pdev) | 238 | static int acpi_smbus_hc_probe(struct platform_device *pdev) |
| 239 | { | 239 | { |
| 240 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 240 | struct acpi_device *device; |
| 241 | int status; | 241 | int status; |
| 242 | unsigned long long val; | 242 | unsigned long long val; |
| 243 | struct acpi_smb_hc *hc; | 243 | struct acpi_smb_hc *hc; |
| 244 | 244 | ||
| 245 | device = ACPI_COMPANION(&pdev->dev); | ||
| 246 | if (!device) | ||
| 247 | return -ENODEV; | ||
| 248 | |||
| 245 | status = acpi_evaluate_integer(device->handle, "_EC", NULL, &val); | 249 | status = acpi_evaluate_integer(device->handle, "_EC", NULL, &val); |
| 246 | if (ACPI_FAILURE(status)) { | 250 | if (ACPI_FAILURE(status)) { |
| 247 | pr_err("error obtaining _EC.\n"); | 251 | pr_err("error obtaining _EC.\n"); |
| @@ -789,7 +789,7 @@ static int acpi_thermal_probe(struct platform_device *pdev) · 788 unmodified lines | |||
| 789 | int i; | 789 | int i; |
| 790 | 790 | ||
| 791 | if (!device) | 791 | if (!device) |
| 792 | return -EINVAL; | 792 | return -ENODEV; |
| 793 | 793 | ||
| 794 | tz = kzalloc_obj(struct acpi_thermal); | 794 | tz = kzalloc_obj(struct acpi_thermal); |
| 795 | if (!tz) | 795 | if (!tz) |
| @@ -38,9 +38,13 @@ static u32 acpi_tiny_power_button_event(void *not_used) · 37 unmodified lines | |||
| 38 | 38 | ||
| 39 | static int acpi_tiny_power_button_probe(struct platform_device *pdev) | 39 | static int acpi_tiny_power_button_probe(struct platform_device *pdev) |
| 40 | { | 40 | { |
| 41 | struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | 41 | struct acpi_device *device; |
| 42 | acpi_status status; | 42 | acpi_status status; |
| 43 | 43 | ||
| 44 | device = ACPI_COMPANION(&pdev->dev); | ||
| 45 | if (!device) | ||
| 46 | return -ENODEV; | ||
| 47 | |||
| 44 | if (device->device_type == ACPI_BUS_TYPE_POWER_BUTTON) { | 48 | if (device->device_type == ACPI_BUS_TYPE_POWER_BUTTON) { |
| 45 | status = acpi_install_fixed_event_handler(ACPI_EVENT_POWER_BUTTON, | 49 | status = acpi_install_fixed_event_handler(ACPI_EVENT_POWER_BUTTON, |
| 46 | acpi_tiny_power_button_event, | 50 | acpi_tiny_power_button_event, |
| @@ -916,6 +916,14 @@ static const struct dmi_system_id video_detect_dmi_table[] = { · 915 unmodified lines | |||
| 916 | DMI_MATCH(DMI_PRODUCT_NAME, "82K8"), | 916 | DMI_MATCH(DMI_PRODUCT_NAME, "82K8"), |
| 917 | }, | 917 | }, |
| 918 | }, | 918 | }, |
| 919 | { | ||
| 920 | .callback = video_detect_force_native, | ||
| 921 | /* HP OMEN Gaming Laptop 16-n0xxx */ | ||
| 922 | .matches = { | ||
| 923 | DMI_MATCH(DMI_SYS_VENDOR, "HP"), | ||
| 924 | DMI_MATCH(DMI_PRODUCT_NAME, "OMEN by HP Gaming Laptop 16-n0xxx"), | ||
| 925 | }, | ||
| 926 | }, | ||
| 919 | 927 | ||
| 920 | /* | 928 | /* |
| 921 | * x86 android tablets which directly control the backlight through | 929 | * x86 android tablets which directly control the backlight through |
| @@ -157,6 +157,14 @@ impl Allocation { · 156 unmodified lines | |||
| 157 | self.get_or_init_info().target_node = Some(target_node); | 157 | self.get_or_init_info().target_node = Some(target_node); |
| 158 | } | 158 | } |
| 159 | 159 | ||
| 160 | pub(crate) fn take_oneway_node(&mut self) -> Option<DArc<Node>> { | ||
| 161 | if let Some(info) = self.allocation_info.as_mut() { | ||
| 162 | info.oneway_node.take() | ||
| 163 | } else { | ||
| 164 | None | ||
| 165 | } | ||
| 166 | } | ||
| 167 | |||
| 160 | /// Reserve enough space to push at least `num_fds` fds. | 168 | /// Reserve enough space to push at least `num_fds` fds. |
| 161 | pub(crate) fn info_add_fd_reserve(&mut self, num_fds: usize) -> Result { | 169 | pub(crate) fn info_add_fd_reserve(&mut self, num_fds: usize) -> Result { |
| 162 | self.get_or_init_info() | 170 | self.get_or_init_info() |
| @@ -1402,7 +1402,12 @@ impl Process { · 1401 unmodified lines | |||
| 1402 | // Clear delivered_deaths list. | 1402 | // Clear delivered_deaths list. |
| 1403 | // | 1403 | // |
| 1404 | // Scope ensures that MutexGuard is dropped while executing the body. | 1404 | // Scope ensures that MutexGuard is dropped while executing the body. |
| 1405 | while let Some(delivered_death) = { self.inner.lock().delivered_deaths.pop_front() } { | 1405 | while let Some(delivered_death) = { |
| 1406 | // Explicitly bind to avoid tail expression lifetime extension of the lockguard | ||
| 1407 | // Can be removed when the kernel moves to edition 2024 | ||
| 1408 | let maybe_death = self.inner.lock().delivered_deaths.pop_front(); | ||
| 1409 | maybe_death | ||
| 1410 | } { | ||
| 1406 | drop(delivered_death); | 1411 | drop(delivered_death); |
| 1407 | } | 1412 | } |
| 1408 | 1413 | ||
| @@ -204,7 +204,6 @@ impl<T> ArrayRangeAllocator<T> { · 203 unmodified lines | |||
| 204 | // caller will mark them as unused, which means that they can be freed if the system comes | 204 | // caller will mark them as unused, which means that they can be freed if the system comes |
| 205 | // under memory pressure. | 205 | // under memory pressure. |
| 206 | let mut freed_range = FreedRange::interior_pages(offset, size); | 206 | let mut freed_range = FreedRange::interior_pages(offset, size); |
| 207 | #[expect(clippy::collapsible_if)] // reads better like this | ||
| 208 | if offset % PAGE_SIZE != 0 { | 207 | if offset % PAGE_SIZE != 0 { |
| 209 | if i == 0 || self.ranges[i - 1].endpoint() <= (offset & PAGE_MASK) { | 208 | if i == 0 || self.ranges[i - 1].endpoint() <= (offset & PAGE_MASK) { |
| 210 | freed_range.start_page_idx -= 1; | 209 | freed_range.start_page_idx -= 1; |
| @@ -270,7 +270,8 @@ impl Transaction { · 269 unmodified lines | |||
| 270 | /// Not used for replies. | 270 | /// Not used for replies. |
| 271 | pub(crate) fn submit(self: DLArc<Self>, info: &mut TransactionInfo) -> BinderResult { | 271 | pub(crate) fn submit(self: DLArc<Self>, info: &mut TransactionInfo) -> BinderResult { |
| 272 | // Defined before `process_inner` so that the destructor runs after releasing the lock. | 272 | // Defined before `process_inner` so that the destructor runs after releasing the lock. |
| 273 | let mut _t_outdated; | 273 | let _t_outdated; |
| 274 | let _oneway_node; | ||
| 274 | 275 | ||
| 275 | let oneway = self.flags & TF_ONE_WAY != 0; | 276 | let oneway = self.flags & TF_ONE_WAY != 0; |
| 276 | let process = self.to.clone(); | 277 | let process = self.to.clone(); |
| @@ -287,6 +288,14 @@ impl Transaction { · 10 unmodified lines | |||
| 287 | if let Some(t_outdated) = | 288 | if let Some(t_outdated) = |
| 288 | target_node.take_outdated_transaction(&self, &mut process_inner) | 289 | target_node.take_outdated_transaction(&self, &mut process_inner) |
| 289 | { | 290 | { |
| 291 | let mut alloc_guard = t_outdated.allocation.lock(); | ||
| 292 | if let Some(alloc) = (*alloc_guard).as_mut() { | ||
| 293 | // Take the oneway node to prevent `Allocation::drop` from calling | ||
| 294 | // `pending_oneway_finished()`, which would be incorrect as this | ||
| 295 | // transaction is not being submitted. | ||
| 296 | _oneway_node = alloc.take_oneway_node(); | ||
| 297 | } | ||
| 298 | drop(alloc_guard); | ||
| 290 | // Save the transaction to be dropped after locks are released. | 299 | // Save the transaction to be dropped after locks are released. |
| 291 | _t_outdated = t_outdated; | 300 | _t_outdated = t_outdated; |
| 292 | } | 301 | } |
| @@ -5584,6 +5584,7 @@ void ata_link_init(struct ata_port *ap, struct ata_link *link, int pmp) · 5583 unmodified lines | |||
| 5584 | link->pmp = pmp; | 5584 | link->pmp = pmp; |
| 5585 | link->active_tag = ATA_TAG_POISON; | 5585 | link->active_tag = ATA_TAG_POISON; |
| 5586 | link->hw_sata_spd_limit = UINT_MAX; | 5586 | link->hw_sata_spd_limit = UINT_MAX; |
| 5587 | INIT_WORK(&link->deferred_qc_work, ata_scsi_deferred_qc_work); | ||
| 5587 | 5588 | ||
| 5588 | /* can't use iterator, ap isn't initialized yet */ | 5589 | /* can't use iterator, ap isn't initialized yet */ |
| 5589 | for (i = 0; i < ATA_MAX_DEVICES; i++) { | 5590 | for (i = 0; i < ATA_MAX_DEVICES; i++) { |
| @@ -5666,7 +5667,6 @@ struct ata_port *ata_port_alloc(struct ata_host *host) · 76 unmodified lines | |||
| 5666 | mutex_init(&ap->scsi_scan_mutex); | 5667 | mutex_init(&ap->scsi_scan_mutex); |
| 5667 | INIT_DELAYED_WORK(&ap->hotplug_task, ata_scsi_hotplug); | 5668 | INIT_DELAYED_WORK(&ap->hotplug_task, ata_scsi_hotplug); |
| 5668 | INIT_DELAYED_WORK(&ap->scsi_rescan_task, ata_scsi_dev_rescan); | 5669 | INIT_DELAYED_WORK(&ap->scsi_rescan_task, ata_scsi_dev_rescan); |
| 5669 | INIT_WORK(&ap->deferred_qc_work, ata_scsi_deferred_qc_work); | ||
| 5670 | INIT_LIST_HEAD(&ap->eh_done_q); | 5670 | INIT_LIST_HEAD(&ap->eh_done_q); |
| 5671 | init_waitqueue_head(&ap->eh_wait_q); | 5671 | init_waitqueue_head(&ap->eh_wait_q); |
| 5672 | init_completion(&ap->park_req_pending); | 5672 | init_completion(&ap->park_req_pending); |
| @@ -6291,12 +6291,15 @@ static void ata_port_detach(struct ata_port *ap) · 618 unmodified lines | |||
| 6291 | 6291 | ||
| 6292 | /* It better be dead now and not have any remaining deferred qc. */ | 6292 | /* It better be dead now and not have any remaining deferred qc. */ |
| 6293 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); | 6293 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); |
| 6294 | WARN_ON(ap->deferred_qc); | ||
| 6295 | 6294 | ||
| 6296 | cancel_work_sync(&ap->deferred_qc_work); | ||
| 6297 | cancel_delayed_work_sync(&ap->hotplug_task); | 6295 | cancel_delayed_work_sync(&ap->hotplug_task); |
| 6298 | cancel_delayed_work_sync(&ap->scsi_rescan_task); | 6296 | cancel_delayed_work_sync(&ap->scsi_rescan_task); |
| 6299 | 6297 | ||
| 6298 | ata_for_each_link(link, ap, PMP_FIRST) { | ||
| 6299 | WARN_ON(link->deferred_qc); | ||
| 6300 | cancel_work_sync(&link->deferred_qc_work); | ||
| 6301 | } | ||
| 6302 | |||
| 6300 | /* Delete port multiplier link transport devices */ | 6303 | /* Delete port multiplier link transport devices */ |
| 6301 | if (ap->pmp_link) { | 6304 | if (ap->pmp_link) { |
| 6302 | int i; | 6305 | int i; |
| @@ -651,11 +651,11 @@ int ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 650 unmodified lines | |||
| 651 | if (qc->scsicmd != scmd) | 651 | if (qc->scsicmd != scmd) |
| 652 | continue; | 652 | continue; |
| 653 | if ((qc->flags & ATA_QCFLAG_ACTIVE) || | 653 | if ((qc->flags & ATA_QCFLAG_ACTIVE) || |
| 654 | qc == ap->deferred_qc) | 654 | qc == qc->dev->link->deferred_qc) |
| 655 | break; | 655 | break; |
| 656 | } | 656 | } |
| 657 | 657 | ||
| 658 | if (i < ATA_MAX_QUEUE && qc == ap->deferred_qc) { | 658 | if (i < ATA_MAX_QUEUE && qc == qc->dev->link->deferred_qc) { |
| 659 | /* | 659 | /* |
| 660 | * This is a deferred command that timed out while | 660 | * This is a deferred command that timed out while |
| 661 | * waiting for the command queue to drain. Since the qc | 661 | * waiting for the command queue to drain. Since the qc |
| @@ -666,8 +666,8 @@ int ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 4 unmodified lines | |||
| 666 | * deferred qc work from issuing this qc. | 666 | * deferred qc work from issuing this qc. |
| 667 | */ | 667 | */ |
| 668 | WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); | 668 | WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); |
| 669 | ap->deferred_qc = NULL; | 669 | qc->dev->link->deferred_qc = NULL; |
| 670 | cancel_work(&ap->deferred_qc_work); | 670 | cancel_work(&qc->dev->link->deferred_qc_work); |
| 671 | set_host_byte(scmd, DID_TIME_OUT); | 671 | set_host_byte(scmd, DID_TIME_OUT); |
| 672 | scsi_eh_finish_cmd(scmd, &ap->eh_done_q); | 672 | scsi_eh_finish_cmd(scmd, &ap->eh_done_q); |
| 673 | } else if (i < ATA_MAX_QUEUE) { | 673 | } else if (i < ATA_MAX_QUEUE) { |
| @@ -110,13 +110,24 @@ int sata_pmp_qc_defer_cmd_switch(struct ata_queued_cmd *qc) · 109 unmodified lines | |||
| 110 | { | 110 | { |
| 111 | struct ata_link *link = qc->dev->link; | 111 | struct ata_link *link = qc->dev->link; |
| 112 | struct ata_port *ap = link->ap; | 112 | struct ata_port *ap = link->ap; |
| 113 | int ret; | ||
| 113 | 114 | ||
| 114 | if (ap->excl_link == NULL || ap->excl_link == link) { | 115 | if (ap->excl_link == NULL || ap->excl_link == link) { |
| 115 | if (ap->nr_active_links == 0 || ata_link_active(link)) { | 116 | if (ap->nr_active_links == 0 || ata_link_active(link)) { |
| 116 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; | 117 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; |
| 117 | return ata_std_qc_defer(qc); | 118 | ret = ata_std_qc_defer(qc); |
| 119 | if (ret == ATA_DEFER_LINK) | ||
| 120 | return ATA_DEFER_LINK_EXCL; | ||
| 121 | return ret; | ||
| 118 | } | 122 | } |
| 119 | 123 | ||
| 124 | /* | ||
| 125 | * Note: ap->excl_link contains the link that is next in line, | ||
| 126 | * i.e. implicit round robin. If there is only one link | ||
| 127 | * dispatching, ap->excl_link will be left unclaimed, allowing | ||
| 128 | * other links to set ap->excl_link, ensuring that the currently | ||
| 129 | * active link cannot queue any more. | ||
| 130 | */ | ||
| 120 | ap->excl_link = link; | 131 | ap->excl_link = link; |
| 121 | } | 132 | } |
| 122 | 133 | ||
| @@ -571,8 +582,11 @@ static void sata_pmp_detach(struct ata_device *dev) · 448 unmodified lines | |||
| 571 | if (ap->ops->pmp_detach) | 582 | if (ap->ops->pmp_detach) |
| 572 | ap->ops->pmp_detach(ap); | 583 | ap->ops->pmp_detach(ap); |
| 573 | 584 | ||
| 574 | ata_for_each_link(tlink, ap, EDGE) | 585 | ata_for_each_link(tlink, ap, EDGE) { |
| 586 | WARN_ON(tlink->deferred_qc); | ||
| 587 | cancel_work_sync(&tlink->deferred_qc_work); | ||
| 575 | ata_eh_detach_dev(tlink->device); | 588 | ata_eh_detach_dev(tlink->device); |
| 589 | } | ||
| 576 | 590 | ||
| 577 | spin_lock_irqsave(ap->lock, flags); | 591 | spin_lock_irqsave(ap->lock, flags); |
| 578 | ap->nr_pmp_links = 0; | 592 | ap->nr_pmp_links = 0; |
| @@ -1664,8 +1664,9 @@ static void ata_scsi_qc_done(struct ata_queued_cmd *qc, bool set_result, · 1663 unmodified lines | |||
| 1664 | 1664 | ||
| 1665 | void ata_scsi_deferred_qc_work(struct work_struct *work) | 1665 | void ata_scsi_deferred_qc_work(struct work_struct *work) |
| 1666 | { | 1666 | { |
| 1667 | struct ata_port *ap = | 1667 | struct ata_link *link = |
| 1668 | container_of(work, struct ata_port, deferred_qc_work); | 1668 | container_of(work, struct ata_link, deferred_qc_work); |
| 1669 | struct ata_port *ap = link->ap; | ||
| 1669 | struct ata_queued_cmd *qc; | 1670 | struct ata_queued_cmd *qc; |
| 1670 | unsigned long flags; | 1671 | unsigned long flags; |
| 1671 | 1672 | ||
| @@ -1676,10 +1677,10 @@ void ata_scsi_deferred_qc_work(struct work_struct *work) · 4 unmodified lines | |||
| 1676 | * such case, we should not need any more deferring the qc, so warn if | 1677 | * such case, we should not need any more deferring the qc, so warn if |
| 1677 | * qc_defer() says otherwise. | 1678 | * qc_defer() says otherwise. |
| 1678 | */ | 1679 | */ |
| 1679 | qc = ap->deferred_qc; | 1680 | qc = link->deferred_qc; |
| 1680 | if (qc && !ata_port_eh_scheduled(ap)) { | 1681 | if (qc && !ata_port_eh_scheduled(ap)) { |
| 1681 | WARN_ON_ONCE(ap->ops->qc_defer(qc)); | 1682 | WARN_ON_ONCE(ap->ops->qc_defer(qc)); |
| 1682 | ap->deferred_qc = NULL; | 1683 | link->deferred_qc = NULL; |
| 1683 | ata_qc_issue(qc); | 1684 | ata_qc_issue(qc); |
| 1684 | } | 1685 | } |
| 1685 | 1686 | ||
| @@ -1688,7 +1689,7 @@ void ata_scsi_deferred_qc_work(struct work_struct *work) · 2 unmodified lines | |||
| 1688 | 1689 | ||
| 1689 | void ata_scsi_requeue_deferred_qc(struct ata_port *ap) | 1690 | void ata_scsi_requeue_deferred_qc(struct ata_port *ap) |
| 1690 | { | 1691 | { |
| 1691 | struct ata_queued_cmd *qc = ap->deferred_qc; | 1692 | struct ata_link *link; |
| 1692 | 1693 | ||
| 1693 | lockdep_assert_held(ap->lock); | 1694 | lockdep_assert_held(ap->lock); |
| 1694 | 1695 | ||
| @@ -1697,16 +1698,21 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap) · 2 unmodified lines | |||
| 1697 | * do not try to be smart about what to do with this deferred command | 1698 | * do not try to be smart about what to do with this deferred command |
| 1698 | * and simply requeue it by completing it with DID_REQUEUE. | 1699 | * and simply requeue it by completing it with DID_REQUEUE. |
| 1699 | */ | 1700 | */ |
| 1700 | if (qc) { | 1701 | ata_for_each_link(link, ap, PMP_FIRST) { |
| 1701 | ap->deferred_qc = NULL; | 1702 | struct ata_queued_cmd *qc = link->deferred_qc; |
| 1702 | cancel_work(&ap->deferred_qc_work); | 1703 | |
| 1703 | ata_scsi_qc_done(qc, true, DID_REQUEUE << 16); | 1704 | if (qc) { |
| 1705 | link->deferred_qc = NULL; | ||
| 1706 | cancel_work(&link->deferred_qc_work); | ||
| 1707 | ata_scsi_qc_done(qc, true, DID_REQUEUE << 16); | ||
| 1708 | } | ||
| 1704 | } | 1709 | } |
| 1705 | } | 1710 | } |
| 1706 | 1711 | ||
| 1707 | static void ata_scsi_schedule_deferred_qc(struct ata_port *ap) | 1712 | static void ata_scsi_schedule_deferred_qc(struct ata_link *link) |
| 1708 | { | 1713 | { |
| 1709 | struct ata_queued_cmd *qc = ap->deferred_qc; | 1714 | struct ata_queued_cmd *qc = link->deferred_qc; |
| 1715 | struct ata_port *ap = link->ap; | ||
| 1710 | 1716 | ||
| 1711 | lockdep_assert_held(ap->lock); | 1717 | lockdep_assert_held(ap->lock); |
| 1712 | 1718 | ||
| @@ -1723,12 +1729,12 @@ static void ata_scsi_schedule_deferred_qc(struct ata_port *ap) · 10 unmodified lines | |||
| 1723 | return; | 1729 | return; |
| 1724 | } | 1730 | } |
| 1725 | if (!ap->ops->qc_defer(qc)) | 1731 | if (!ap->ops->qc_defer(qc)) |
| 1726 | queue_work(system_highpri_wq, &ap->deferred_qc_work); | 1732 | queue_work(system_highpri_wq, &link->deferred_qc_work); |
| 1727 | } | 1733 | } |
| 1728 | 1734 | ||
| 1729 | static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) | 1735 | static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) |
| 1730 | { | 1736 | { |
| 1731 | struct ata_port *ap = qc->ap; | 1737 | struct ata_link *link = qc->dev->link; |
| 1732 | struct scsi_cmnd *cmd = qc->scsicmd; | 1738 | struct scsi_cmnd *cmd = qc->scsicmd; |
| 1733 | u8 *cdb = cmd->cmnd; | 1739 | u8 *cdb = cmd->cmnd; |
| 1734 | bool have_sense = qc->flags & ATA_QCFLAG_SENSE_VALID; | 1740 | bool have_sense = qc->flags & ATA_QCFLAG_SENSE_VALID; |
| @@ -1759,22 +1765,23 @@ static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) · 24 unmodified lines | |||
| 1759 | 1765 | ||
| 1760 | ata_scsi_qc_done(qc, false, 0); | 1766 | ata_scsi_qc_done(qc, false, 0); |
| 1761 | 1767 | ||
| 1762 | ata_scsi_schedule_deferred_qc(ap); | 1768 | ata_scsi_schedule_deferred_qc(link); |
| 1763 | } | 1769 | } |
| 1764 | 1770 | ||
| 1765 | static int ata_scsi_qc_issue(struct ata_port *ap, struct ata_queued_cmd *qc) | 1771 | static int ata_scsi_qc_issue(struct ata_port *ap, struct ata_queued_cmd *qc) |
| 1766 | { | 1772 | { |
| 1773 | struct ata_link *link = qc->dev->link; | ||
| 1767 | int ret; | 1774 | int ret; |
| 1768 | 1775 | ||
| 1769 | if (!ap->ops->qc_defer) | 1776 | if (!ap->ops->qc_defer) |
| 1770 | goto issue; | 1777 | goto issue_qc; |
| 1771 | 1778 | ||
| 1772 | /* | 1779 | /* |
| 1773 | * If we already have a deferred qc, then rely on the SCSI layer to | 1780 | * If we already have a deferred qc, then rely on the SCSI layer to |
| 1774 | * requeue and defer all incoming commands until the deferred qc is | 1781 | * requeue and defer all incoming commands until the deferred qc is |
| 1775 | * processed, once all on-going commands complete. | 1782 | * processed, once all on-going commands complete. |
| 1776 | */ | 1783 | */ |
| 1777 | if (ap->deferred_qc) { | 1784 | if (link->deferred_qc) { |
| 1778 | ata_qc_free(qc); | 1785 | ata_qc_free(qc); |
| 1779 | return SCSI_MLQUEUE_DEVICE_BUSY; | 1786 | return SCSI_MLQUEUE_DEVICE_BUSY; |
| 1780 | } | 1787 | } |
| @@ -1786,38 +1793,46 @@ static int ata_scsi_qc_issue(struct ata_port *ap, struct ata_queued_cmd *qc) · 5 unmodified lines | |||
| 1786 | break; | 1793 | break; |
| 1787 | case ATA_DEFER_LINK: | 1794 | case ATA_DEFER_LINK: |
| 1788 | ret = SCSI_MLQUEUE_DEVICE_BUSY; | 1795 | ret = SCSI_MLQUEUE_DEVICE_BUSY; |
| 1789 | break; | 1796 | goto defer_qc; |
| 1797 | case ATA_DEFER_LINK_EXCL: | ||
| 1798 | /* | ||
| 1799 | * Drivers making use of ap->excl_link cannot store the QC in | ||
| 1800 | * link->deferred_qc, because the ap->excl_link handling is | ||
| 1801 | * incompatible with the link->deferred_qc workqueue handling. | ||
| 1802 | */ | ||
| 1803 | ret = SCSI_MLQUEUE_DEVICE_BUSY; | ||
| 1804 | goto free_qc; | ||
| 1790 | case ATA_DEFER_PORT: | 1805 | case ATA_DEFER_PORT: |
| 1791 | ret = SCSI_MLQUEUE_HOST_BUSY; | 1806 | ret = SCSI_MLQUEUE_HOST_BUSY; |
| 1792 | break; | 1807 | goto free_qc; |
| 1793 | default: | 1808 | default: |
| 1794 | WARN_ON_ONCE(1); | 1809 | WARN_ON_ONCE(1); |
| 1795 | ret = SCSI_MLQUEUE_HOST_BUSY; | 1810 | ret = SCSI_MLQUEUE_HOST_BUSY; |
| 1796 | break; | 1811 | goto free_qc; |
| 1797 | } | 1812 | } |
| 1798 | 1813 | ||
| 1799 | if (ret) { | 1814 | issue_qc: |
| 1800 | /* | 1815 | ata_qc_issue(qc); |
| 1801 | * We must defer this qc: if this is not an NCQ command, keep | 1816 | return 0; |
| 1802 | * this qc as a deferred one and report to the SCSI layer that | ||
| 1803 | * we issued it so that it is not requeued. The deferred qc will | ||
| 1804 | * be issued with the port deferred_qc_work once all on-going | ||
| 1805 | * commands complete. | ||
| 1806 | */ | ||
| 1807 | if (!ata_is_ncq(qc->tf.protocol)) { | ||
| 1808 | ap->deferred_qc = qc; | ||
| 1809 | return 0; | ||
| 1810 | } | ||
| 1811 | 1817 | ||
| 1812 | /* Force a requeue of the command to defer its execution. */ | 1818 | defer_qc: |
| 1813 | ata_qc_free(qc); | 1819 | /* |
| 1814 | return ret; | 1820 | * We must defer this qc: if this is not an NCQ command, keep |
| 1821 | * this qc as a deferred one and report to the SCSI layer that | ||
| 1822 | * we issued it so that it is not requeued. The deferred qc will | ||
| 1823 | * be issued with the port deferred_qc_work once all on-going | ||
| 1824 | * commands complete. | ||
| 1825 | */ | ||
| 1826 | if (!ata_is_ncq(qc->tf.protocol)) { | ||
| 1827 | link->deferred_qc = qc; | ||
| 1828 | return 0; | ||
| 1815 | } | 1829 | } |
| 1816 | 1830 | ||
| 1817 | issue: | 1831 | free_qc: |
| 1818 | ata_qc_issue(qc); | 1832 | /* Force a requeue of the command to defer its execution. */ |
| 1833 | ata_qc_free(qc); | ||
| 1819 | 1834 | ||
| 1820 | return 0; | 1835 | return ret; |
| 1821 | } | 1836 | } |
| 1822 | 1837 | ||
| 1823 | /** | 1838 | /** |
| @@ -459,19 +459,11 @@ static void pata_parport_dev_release(struct device *dev) · 458 unmodified lines | |||
| 459 | kfree(pi); | 459 | kfree(pi); |
| 460 | } | 460 | } |
| 461 | 461 | ||
| 462 | static void pata_parport_bus_release(struct device *dev) | ||
| 463 | { | ||
| 464 | /* nothing to do here but required to avoid warning on device removal */ | ||
| 465 | } | ||
| 466 | |||
| 467 | static const struct bus_type pata_parport_bus_type = { | 462 | static const struct bus_type pata_parport_bus_type = { |
| 468 | .name = DRV_NAME, | 463 | .name = DRV_NAME, |
| 469 | }; | 464 | }; |
| 470 | 465 | ||
| 471 | static struct device pata_parport_bus = { | 466 | static struct device *pata_parport_bus; |
| 472 | .init_name = DRV_NAME, | ||
| 473 | .release = pata_parport_bus_release, | ||
| 474 | }; | ||
| 475 | 467 | ||
| 476 | static const struct scsi_host_template pata_parport_sht = { | 468 | static const struct scsi_host_template pata_parport_sht = { |
| 477 | PATA_PARPORT_SHT("pata_parport") | 469 | PATA_PARPORT_SHT("pata_parport") |
| @@ -518,7 +510,7 @@ static struct pi_adapter *pi_init_one(struct parport *parport, · 40 unmodified lines | |||
| 518 | } | 510 | } |
| 519 | 511 | ||
| 520 | /* set up pi->dev before pi_probe_unit() so it can use dev_printk() */ | 512 | /* set up pi->dev before pi_probe_unit() so it can use dev_printk() */ |
| 521 | pi->dev.parent = &pata_parport_bus; | 513 | pi->dev.parent = pata_parport_bus; |
| 522 | pi->dev.bus = &pata_parport_bus_type; | 514 | pi->dev.bus = &pata_parport_bus_type; |
| 523 | pi->dev.driver = &pr->driver; | 515 | pi->dev.driver = &pr->driver; |
| 524 | pi->dev.release = pata_parport_dev_release; | 516 | pi->dev.release = pata_parport_dev_release; |
| @@ -780,8 +772,9 @@ static __init int pata_parport_init(void) · 255 unmodified lines | |||
| 780 | return error; | 772 | return error; |
| 781 | } | 773 | } |
| 782 | 774 | ||
| 783 | error = device_register(&pata_parport_bus); | 775 | pata_parport_bus = root_device_register(DRV_NAME); |
| 784 | if (error) { | 776 | if (IS_ERR(pata_parport_bus)) { |
| 777 | error = PTR_ERR(pata_parport_bus); | ||
| 785 | pr_err("failed to register pata_parport bus, error: %d\n", error); | 778 | pr_err("failed to register pata_parport bus, error: %d\n", error); |
| 786 | goto out_unregister_bus; | 779 | goto out_unregister_bus; |
| 787 | } | 780 | } |
| @@ -811,7 +804,7 @@ out_remove_del: · 23 unmodified lines | |||
| 811 | out_remove_new: | 804 | out_remove_new: |
| 812 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); | 805 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); |
| 813 | out_unregister_dev: | 806 | out_unregister_dev: |
| 814 | device_unregister(&pata_parport_bus); | 807 | root_device_unregister(pata_parport_bus); |
| 815 | out_unregister_bus: | 808 | out_unregister_bus: |
| 816 | bus_unregister(&pata_parport_bus_type); | 809 | bus_unregister(&pata_parport_bus_type); |
| 817 | return error; | 810 | return error; |
| @@ -822,7 +815,7 @@ static __exit void pata_parport_exit(void) · 4 unmodified lines | |||
| 822 | parport_unregister_driver(&pata_parport_driver); | 815 | parport_unregister_driver(&pata_parport_driver); |
| 823 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); | 816 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); |
| 824 | bus_remove_file(&pata_parport_bus_type, &bus_attr_delete_device); | 817 | bus_remove_file(&pata_parport_bus_type, &bus_attr_delete_device); |
| 825 | device_unregister(&pata_parport_bus); | 818 | root_device_unregister(pata_parport_bus); |
| 826 | bus_unregister(&pata_parport_bus_type); | 819 | bus_unregister(&pata_parport_bus_type); |
| 827 | } | 820 | } |
| 828 | 821 | ||
| @@ -789,6 +789,7 @@ static int sil24_qc_defer(struct ata_queued_cmd *qc) · 788 unmodified lines | |||
| 789 | struct ata_link *link = qc->dev->link; | 789 | struct ata_link *link = qc->dev->link; |
| 790 | struct ata_port *ap = link->ap; | 790 | struct ata_port *ap = link->ap; |
| 791 | u8 prot = qc->tf.protocol; | 791 | u8 prot = qc->tf.protocol; |
| 792 | int ret; | ||
| 792 | 793 | ||
| 793 | /* | 794 | /* |
| 794 | * There is a bug in the chip: | 795 | * There is a bug in the chip: |
| @@ -826,7 +827,10 @@ static int sil24_qc_defer(struct ata_queued_cmd *qc) · 31 unmodified lines | |||
| 826 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; | 827 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; |
| 827 | } | 828 | } |
| 828 | 829 | ||
| 829 | return ata_std_qc_defer(qc); | 830 | ret = ata_std_qc_defer(qc); |
| 831 | if (ret == ATA_DEFER_LINK) | ||
| 832 | return ATA_DEFER_LINK_EXCL; | ||
| 833 | return ret; | ||
| 830 | } | 834 | } |
| 831 | 835 | ||
| 832 | static enum ata_completion_errors sil24_qc_prep(struct ata_queued_cmd *qc) | 836 | static enum ata_completion_errors sil24_qc_prep(struct ata_queued_cmd *qc) |
| @@ -1230,8 +1230,10 @@ void memblk_nr_poison_inc(unsigned long pfn) · 1229 unmodified lines | |||
| 1230 | const unsigned long block_id = pfn_to_block_id(pfn); | 1230 | const unsigned long block_id = pfn_to_block_id(pfn); |
| 1231 | struct memory_block *mem = find_memory_block_by_id(block_id); | 1231 | struct memory_block *mem = find_memory_block_by_id(block_id); |
| 1232 | 1232 | ||
| 1233 | if (mem) | 1233 | if (mem) { |
| 1234 | atomic_long_inc(&mem->nr_hwpoison); | 1234 | atomic_long_inc(&mem->nr_hwpoison); |
| 1235 | put_device(&mem->dev); | ||
| 1236 | } | ||
| 1235 | } | 1237 | } |
| 1236 | 1238 | ||
| 1237 | void memblk_nr_poison_sub(unsigned long pfn, long i) | 1239 | void memblk_nr_poison_sub(unsigned long pfn, long i) |
| @@ -1239,8 +1241,10 @@ void memblk_nr_poison_sub(unsigned long pfn, long i) · 1 unmodified line | |||
| 1239 | const unsigned long block_id = pfn_to_block_id(pfn); | 1241 | const unsigned long block_id = pfn_to_block_id(pfn); |
| 1240 | struct memory_block *mem = find_memory_block_by_id(block_id); | 1242 | struct memory_block *mem = find_memory_block_by_id(block_id); |
| 1241 | 1243 | ||
| 1242 | if (mem) | 1244 | if (mem) { |
| 1243 | atomic_long_sub(i, &mem->nr_hwpoison); | 1245 | atomic_long_sub(i, &mem->nr_hwpoison); |
| 1246 | put_device(&mem->dev); | ||
| 1247 | } | ||
| 1244 | } | 1248 | } |
| 1245 | 1249 | ||
| 1246 | static unsigned long memblk_nr_poison(struct memory_block *mem) | 1250 | static unsigned long memblk_nr_poison(struct memory_block *mem) |
| @@ -606,6 +606,12 @@ static void platform_device_release(struct device *dev) · 605 unmodified lines | |||
| 606 | kfree(pa); | 606 | kfree(pa); |
| 607 | } | 607 | } |
| 608 | 608 | ||
| 609 | static void platform_device_release_full(struct device *dev) | ||
| 610 | { | ||
| 611 | device_remove_software_node(dev); | ||
| 612 | platform_device_release(dev); | ||
| 613 | } | ||
| 614 | |||
| 609 | /** | 615 | /** |
| 610 | * platform_device_alloc - create a platform device | 616 | * platform_device_alloc - create a platform device |
| 611 | * @name: base name of the device we're adding | 617 | * @name: base name of the device we're adding |
| @@ -848,7 +854,13 @@ struct platform_device *platform_device_register_full(const struct platform_devi · 236 unmodified lines | |||
| 848 | int ret; | 854 | int ret; |
| 849 | struct platform_device *pdev; | 855 | struct platform_device *pdev; |
| 850 | 856 | ||
| 851 | if (pdevinfo->swnode && pdevinfo->properties) | 857 | /* |
| 858 | * Only one software node per device is allowed. Make sure we don't | ||
| 859 | * accept or create two. | ||
| 860 | */ | ||
| 861 | if ((pdevinfo->swnode && pdevinfo->properties) || | ||
| 862 | (pdevinfo->swnode && is_software_node(pdevinfo->fwnode)) || | ||
| 863 | (pdevinfo->properties && is_software_node(pdevinfo->fwnode))) | ||
| 852 | return ERR_PTR(-EINVAL); | 864 | return ERR_PTR(-EINVAL); |
| 853 | 865 | ||
| 854 | pdev = platform_device_alloc(pdevinfo->name, pdevinfo->id); | 866 | pdev = platform_device_alloc(pdevinfo->name, pdevinfo->id); |
| @@ -878,6 +890,8 @@ struct platform_device *platform_device_register_full(const struct platform_devi · 23 unmodified lines | |||
| 878 | ret = device_add_software_node(&pdev->dev, pdevinfo->swnode); | 890 | ret = device_add_software_node(&pdev->dev, pdevinfo->swnode); |
| 879 | if (ret) | 891 | if (ret) |
| 880 | goto err; | 892 | goto err; |
| 893 | |||
| 894 | pdev->dev.release = platform_device_release_full; | ||
| 881 | } else if (pdevinfo->properties) { | 895 | } else if (pdevinfo->properties) { |
| 882 | ret = device_create_managed_software_node(&pdev->dev, | 896 | ret = device_create_managed_software_node(&pdev->dev, |
| 883 | pdevinfo->properties, NULL); | 897 | pdevinfo->properties, NULL); |
| @@ -172,7 +172,7 @@ static int regmap_sdw_mbq_read(void *context, unsigned int reg, unsigned int *va · 171 unmodified lines | |||
| 172 | ret = regmap_sdw_mbq_read_impl(slave, reg, val, mbq_size); | 172 | ret = regmap_sdw_mbq_read_impl(slave, reg, val, mbq_size); |
| 173 | if (ret == -ENODATA) { | 173 | if (ret == -ENODATA) { |
| 174 | if (!deferrable) | 174 | if (!deferrable) |
| 175 | dev_warn(dev, "Defer on undeferable control: %x\n", reg); | 175 | dev_warn(dev, "Defer on undeferrable control: %x\n", reg); |
| 176 | 176 | ||
| 177 | ret = regmap_sdw_mbq_poll_busy(slave, reg, ctx); | 177 | ret = regmap_sdw_mbq_poll_busy(slave, reg, ctx); |
| 178 | if (ret) | 178 | if (ret) |
| @@ -3257,6 +3257,9 @@ static int _regmap_update_bits(struct regmap *map, unsigned int reg, · 3256 unmodified lines | |||
| 3257 | *change = false; | 3257 | *change = false; |
| 3258 | 3258 | ||
| 3259 | if (regmap_volatile(map, reg) && map->reg_update_bits) { | 3259 | if (regmap_volatile(map, reg) && map->reg_update_bits) { |
| 3260 | if (map->cache_only) | ||
| 3261 | return -EBUSY; | ||
| 3262 | |||
| 3260 | reg = regmap_reg_addr(map, reg); | 3263 | reg = regmap_reg_addr(map, reg); |
| 3261 | ret = map->reg_update_bits(map->bus_context, reg, mask, val); | 3264 | ret = map->reg_update_bits(map->bus_context, reg, mask, val); |
| 3262 | if (ret == 0 && change) | 3265 | if (ret == 0 && change) |
| @@ -4565,24 +4565,12 @@ out: · 4564 unmodified lines | |||
| 4565 | return ret; | 4565 | return ret; |
| 4566 | } | 4566 | } |
| 4567 | 4567 | ||
| 4568 | static void cancel_tasks_sync(struct rbd_device *rbd_dev) | ||
| 4569 | { | ||
| 4570 | dout("%s rbd_dev %p\n", __func__, rbd_dev); | ||
| 4571 | |||
| 4572 | cancel_work_sync(&rbd_dev->acquired_lock_work); | ||
| 4573 | cancel_work_sync(&rbd_dev->released_lock_work); | ||
| 4574 | cancel_delayed_work_sync(&rbd_dev->lock_dwork); | ||
| 4575 | cancel_work_sync(&rbd_dev->unlock_work); | ||
| 4576 | } | ||
| 4577 | |||
| 4578 | /* | 4568 | /* |
| 4579 | * header_rwsem must not be held to avoid a deadlock with | 4569 | * header_rwsem must not be held to avoid a deadlock with |
| 4580 | * rbd_dev_refresh() when flushing notifies. | 4570 | * rbd_dev_refresh() when flushing notifies. |
| 4581 | */ | 4571 | */ |
| 4582 | static void rbd_unregister_watch(struct rbd_device *rbd_dev) | 4572 | static void rbd_unregister_watch(struct rbd_device *rbd_dev) |
| 4583 | { | 4573 | { |
| 4584 | cancel_tasks_sync(rbd_dev); | ||
| 4585 | |||
| 4586 | mutex_lock(&rbd_dev->watch_mutex); | 4574 | mutex_lock(&rbd_dev->watch_mutex); |
| 4587 | if (rbd_dev->watch_state == RBD_WATCH_STATE_REGISTERED) | 4575 | if (rbd_dev->watch_state == RBD_WATCH_STATE_REGISTERED) |
| 4588 | __rbd_unregister_watch(rbd_dev); | 4576 | __rbd_unregister_watch(rbd_dev); |
| @@ -6548,10 +6536,18 @@ out_err: · 1959 unmodified lines | |||
| 6548 | 6536 | ||
| 6549 | static void rbd_dev_image_unlock(struct rbd_device *rbd_dev) | 6537 | static void rbd_dev_image_unlock(struct rbd_device *rbd_dev) |
| 6550 | { | 6538 | { |
| 6539 | dout("%s rbd_dev %p\n", __func__, rbd_dev); | ||
| 6540 | |||
| 6541 | disable_delayed_work_sync(&rbd_dev->lock_dwork); | ||
| 6542 | disable_work_sync(&rbd_dev->unlock_work); | ||
| 6543 | |||
| 6551 | down_write(&rbd_dev->lock_rwsem); | 6544 | down_write(&rbd_dev->lock_rwsem); |
| 6552 | if (__rbd_is_lock_owner(rbd_dev)) | 6545 | if (__rbd_is_lock_owner(rbd_dev)) |
| 6553 | __rbd_release_lock(rbd_dev); | 6546 | __rbd_release_lock(rbd_dev); |
| 6554 | up_write(&rbd_dev->lock_rwsem); | 6547 | up_write(&rbd_dev->lock_rwsem); |
| 6548 | |||
| 6549 | flush_work(&rbd_dev->acquired_lock_work); | ||
| 6550 | flush_work(&rbd_dev->released_lock_work); | ||
| 6555 | } | 6551 | } |
| 6556 | 6552 | ||
| 6557 | /* | 6553 | /* |
| @@ -900,12 +900,29 @@ static int ublk_validate_params(const struct ublk_device *ub) · 899 unmodified lines | |||
| 900 | if (p->logical_bs_shift > PAGE_SHIFT || p->logical_bs_shift < 9) | 900 | if (p->logical_bs_shift > PAGE_SHIFT || p->logical_bs_shift < 9) |
| 901 | return -EINVAL; | 901 | return -EINVAL; |
| 902 | 902 | ||
| 903 | /* | ||
| 904 | * 256M is a reasonable upper bound for physical block size, | ||
| 905 | * io_min and io_opt; it aligns with the maximum physical | ||
| 906 | * block size possible in NVMe. | ||
| 907 | */ | ||
| 908 | if (p->physical_bs_shift > ilog2(SZ_256M)) | ||
| 909 | return -EINVAL; | ||
| 910 | |||
| 911 | if (p->io_min_shift > ilog2(SZ_256M)) | ||
| 912 | return -EINVAL; | ||
| 913 | |||
| 914 | if (p->io_opt_shift > ilog2(SZ_256M)) | ||
| 915 | return -EINVAL; | ||
| 916 | |||
| 903 | if (p->logical_bs_shift > p->physical_bs_shift) | 917 | if (p->logical_bs_shift > p->physical_bs_shift) |
| 904 | return -EINVAL; | 918 | return -EINVAL; |
| 905 | 919 | ||
| 906 | if (p->max_sectors > (ub->dev_info.max_io_buf_bytes >> 9)) | 920 | if (p->max_sectors > (ub->dev_info.max_io_buf_bytes >> 9)) |
| 907 | return -EINVAL; | 921 | return -EINVAL; |
| 908 | 922 | ||
| 923 | if (p->max_sectors < PAGE_SECTORS) | ||
| 924 | return -EINVAL; | ||
| 925 | |||
| 909 | if (ublk_dev_is_zoned(ub) && !p->chunk_sectors) | 926 | if (ublk_dev_is_zoned(ub) && !p->chunk_sectors) |
| 910 | return -EINVAL; | 927 | return -EINVAL; |
| 911 | } else | 928 | } else |
| @@ -2397,8 +2414,14 @@ static void ublk_reset_ch_dev(struct ublk_device *ub) · 1485 unmodified lines | |||
| 2397 | { | 2414 | { |
| 2398 | int i; | 2415 | int i; |
| 2399 | 2416 | ||
| 2400 | for (i = 0; i < ub->dev_info.nr_hw_queues; i++) | 2417 | for (i = 0; i < ub->dev_info.nr_hw_queues; i++) { |
| 2401 | ublk_queue_reinit(ub, ublk_get_queue(ub, i)); | 2418 | struct ublk_queue *ubq = ublk_get_queue(ub, i); |
| 2419 | |||
| 2420 | /* Sync with ublk_cancel_cmd() */ | ||
| 2421 | spin_lock(&ubq->cancel_lock); | ||
| 2422 | ublk_queue_reinit(ub, ubq); | ||
| 2423 | spin_unlock(&ubq->cancel_lock); | ||
| 2424 | } | ||
| 2402 | 2425 | ||
| 2403 | /* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */ | 2426 | /* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */ |
| 2404 | ub->mm = NULL; | 2427 | ub->mm = NULL; |
| @@ -2739,6 +2762,7 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag, · 334 unmodified lines | |||
| 2739 | { | 2762 | { |
| 2740 | struct ublk_io *io = &ubq->ios[tag]; | 2763 | struct ublk_io *io = &ubq->ios[tag]; |
| 2741 | struct ublk_device *ub = ubq->dev; | 2764 | struct ublk_device *ub = ubq->dev; |
| 2765 | struct io_uring_cmd *cmd = NULL; | ||
| 2742 | struct request *req; | 2766 | struct request *req; |
| 2743 | bool done; | 2767 | bool done; |
| 2744 | 2768 | ||
| @@ -2761,12 +2785,15 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag, · 16 unmodified lines | |||
| 2761 | 2785 | ||
| 2762 | spin_lock(&ubq->cancel_lock); | 2786 | spin_lock(&ubq->cancel_lock); |
| 2763 | done = !!(io->flags & UBLK_IO_FLAG_CANCELED); | 2787 | done = !!(io->flags & UBLK_IO_FLAG_CANCELED); |
| 2764 | if (!done) | 2788 | if (!done) { |
| 2765 | io->flags |= UBLK_IO_FLAG_CANCELED; | 2789 | io->flags |= UBLK_IO_FLAG_CANCELED; |
| 2790 | cmd = io->cmd; | ||
| 2791 | io->cmd = NULL; | ||
| 2792 | } | ||
| 2766 | spin_unlock(&ubq->cancel_lock); | 2793 | spin_unlock(&ubq->cancel_lock); |
| 2767 | 2794 | ||
| 2768 | if (!done) | 2795 | if (!done && cmd) |
| 2769 | io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, issue_flags); | 2796 | io_uring_cmd_done(cmd, UBLK_IO_RES_ABORT, issue_flags); |
| 2770 | } | 2797 | } |
| 2771 | 2798 | ||
| 2772 | /* | 2799 | /* |
| @@ -3496,8 +3523,10 @@ static void ublk_ch_uring_cmd_cb(struct io_tw_req tw_req, io_tw_token_t tw) · 723 unmodified lines | |||
| 3496 | { | 3523 | { |
| 3497 | unsigned int issue_flags = IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; | 3524 | unsigned int issue_flags = IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; |
| 3498 | struct io_uring_cmd *cmd = io_uring_cmd_from_tw(tw_req); | 3525 | struct io_uring_cmd *cmd = io_uring_cmd_from_tw(tw_req); |
| 3499 | int ret = ublk_ch_uring_cmd_local(cmd, issue_flags); | 3526 | int ret = -ECANCELED; |
| 3500 | 3527 | ||
| 3528 | if (!tw.cancel) | ||
| 3529 | ret = ublk_ch_uring_cmd_local(cmd, issue_flags); | ||
| 3501 | if (ret != -EIOCBQUEUED) | 3530 | if (ret != -EIOCBQUEUED) |
| 3502 | io_uring_cmd_done(cmd, ret, issue_flags); | 3531 | io_uring_cmd_done(cmd, ret, issue_flags); |
| 3503 | } | 3532 | } |
| @@ -4990,13 +5019,15 @@ static int ublk_ctrl_set_params(struct ublk_device *ub, · 1486 unmodified lines | |||
| 4990 | */ | 5019 | */ |
| 4991 | ret = -EACCES; | 5020 | ret = -EACCES; |
| 4992 | } else if (copy_from_user(&ub->params, argp, ph.len)) { | 5021 | } else if (copy_from_user(&ub->params, argp, ph.len)) { |
| 5022 | /* zero out partial copy so no stale params survive */ | ||
| 5023 | memset(&ub->params, 0, sizeof(ub->params)); | ||
| 4993 | ret = -EFAULT; | 5024 | ret = -EFAULT; |
| 4994 | } else { | 5025 | } else { |
| 4995 | /* clear all we don't support yet */ | 5026 | /* clear all we don't support yet */ |
| 4996 | ub->params.types &= UBLK_PARAM_TYPE_ALL; | 5027 | ub->params.types &= UBLK_PARAM_TYPE_ALL; |
| 4997 | ret = ublk_validate_params(ub); | 5028 | ret = ublk_validate_params(ub); |
| 4998 | if (ret) | 5029 | if (ret) |
| 4999 | ub->params.types = 0; | 5030 | memset(&ub->params, 0, sizeof(ub->params)); |
| 5000 | } | 5031 | } |
| 5001 | mutex_unlock(&ub->mutex); | 5032 | mutex_unlock(&ub->mutex); |
| 5002 | 5033 | ||
| @@ -33,6 +33,7 @@ · 32 unmodified lines | |||
| 33 | #include <linux/cpuhotplug.h> | 33 | #include <linux/cpuhotplug.h> |
| 34 | #include <linux/part_stat.h> | 34 | #include <linux/part_stat.h> |
| 35 | #include <linux/kernel_read_file.h> | 35 | #include <linux/kernel_read_file.h> |
| 36 | #include <linux/rcupdate.h> | ||
| 36 | 37 | ||
| 37 | #include "zram_drv.h" | 38 | #include "zram_drv.h" |
| 38 | 39 | ||
| @@ -504,6 +505,7 @@ struct zram_wb_ctl { · 465 unmodified lines | |||
| 504 | wait_queue_head_t done_wait; | 505 | wait_queue_head_t done_wait; |
| 505 | spinlock_t done_lock; | 506 | spinlock_t done_lock; |
| 506 | atomic_t num_inflight; | 507 | atomic_t num_inflight; |
| 508 | struct rcu_head rcu; | ||
| 507 | }; | 509 | }; |
| 508 | 510 | ||
| 509 | struct zram_wb_req { | 511 | struct zram_wb_req { |
| @@ -847,7 +849,7 @@ static void release_wb_ctl(struct zram_wb_ctl *wb_ctl) · 337 unmodified lines | |||
| 847 | release_wb_req(req); | 849 | release_wb_req(req); |
| 848 | } | 850 | } |
| 849 | 851 | ||
| 850 | kfree(wb_ctl); | 852 | kfree_rcu(wb_ctl, rcu); |
| 851 | } | 853 | } |
| 852 | 854 | ||
| 853 | static struct zram_wb_ctl *init_wb_ctl(struct zram *zram) | 855 | static struct zram_wb_ctl *init_wb_ctl(struct zram *zram) |
| @@ -964,11 +966,13 @@ static void zram_writeback_endio(struct bio *bio) · 110 unmodified lines | |||
| 964 | struct zram_wb_ctl *wb_ctl = bio->bi_private; | 966 | struct zram_wb_ctl *wb_ctl = bio->bi_private; |
| 965 | unsigned long flags; | 967 | unsigned long flags; |
| 966 | 968 | ||
| 969 | rcu_read_lock(); | ||
| 967 | spin_lock_irqsave(&wb_ctl->done_lock, flags); | 970 | spin_lock_irqsave(&wb_ctl->done_lock, flags); |
| 968 | list_add(&req->entry, &wb_ctl->done_reqs); | 971 | list_add(&req->entry, &wb_ctl->done_reqs); |
| 969 | spin_unlock_irqrestore(&wb_ctl->done_lock, flags); | 972 | spin_unlock_irqrestore(&wb_ctl->done_lock, flags); |
| 970 | 973 | ||
| 971 | wake_up(&wb_ctl->done_wait); | 974 | wake_up(&wb_ctl->done_wait); |
| 975 | rcu_read_unlock(); | ||
| 972 | } | 976 | } |
| 973 | 977 | ||
| 974 | static void zram_submit_wb_request(struct zram *zram, | 978 | static void zram_submit_wb_request(struct zram *zram, |
| @@ -289,6 +289,9 @@ static inline void btintel_pcie_dump_debug_registers(struct hci_dev *hdev) · 288 unmodified lines | |||
| 289 | skb_put_data(skb, buf, strlen(buf)); | 289 | skb_put_data(skb, buf, strlen(buf)); |
| 290 | data->boot_stage_cache = reg; | 290 | data->boot_stage_cache = reg; |
| 291 | 291 | ||
| 292 | if (reg & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING) | ||
| 293 | bt_dev_warn(hdev, "Controller device warning (boot_stage: 0x%8.8x)", reg); | ||
| 294 | |||
| 292 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_IPC_STATUS_REG); | 295 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_IPC_STATUS_REG); |
| 293 | snprintf(buf, sizeof(buf), "ipc status: 0x%8.8x", reg); | 296 | snprintf(buf, sizeof(buf), "ipc status: 0x%8.8x", reg); |
| 294 | skb_put_data(skb, buf, strlen(buf)); | 297 | skb_put_data(skb, buf, strlen(buf)); |
| @@ -579,12 +582,10 @@ static int btintel_pcie_get_mac_access(struct btintel_pcie_data *data) · 284 unmodified lines | |||
| 579 | 582 | ||
| 580 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); | 583 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| 581 | 584 | ||
| 582 | reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS; | 585 | if (!(reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ)) { |
| 583 | reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ; | ||
| 584 | if ((reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_STS) == 0) | ||
| 585 | reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; | 586 | reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; |
| 586 | 587 | btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 587 | btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | 588 | } |
| 588 | 589 | ||
| 589 | do { | 590 | do { |
| 590 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); | 591 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| @@ -604,16 +605,10 @@ static void btintel_pcie_release_mac_access(struct btintel_pcie_data *data) · 13 unmodified lines | |||
| 604 | 605 | ||
| 605 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); | 606 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| 606 | 607 | ||
| 607 | if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ) | 608 | if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ) { |
| 608 | reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; | 609 | reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; |
| 609 | 610 | btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 610 | if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS) | 611 | } |
| 611 | reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS; | ||
| 612 | |||
| 613 | if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ) | ||
| 614 | reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ; | ||
| 615 | |||
| 616 | btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | ||
| 617 | } | 612 | } |
| 618 | 613 | ||
| 619 | static void *btintel_pcie_copy_tlv(void *dest, enum btintel_pcie_tlv_type type, | 614 | static void *btintel_pcie_copy_tlv(void *dest, enum btintel_pcie_tlv_type type, |
| @@ -880,8 +875,11 @@ static inline bool btintel_pcie_in_lockdown(struct btintel_pcie_data *data) · 260 unmodified lines | |||
| 880 | 875 | ||
| 881 | static inline bool btintel_pcie_in_error(struct btintel_pcie_data *data) | 876 | static inline bool btintel_pcie_in_error(struct btintel_pcie_data *data) |
| 882 | { | 877 | { |
| 883 | return (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_ERR) || | 878 | if (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING) |
| 884 | (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER); | 879 | bt_dev_warn(data->hdev, "Controller device warning (boot_stage: 0x%8.8x)", |
| 880 | data->boot_stage_cache); | ||
| 881 | |||
| 882 | return data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER; | ||
| 885 | } | 883 | } |
| 886 | 884 | ||
| 887 | static void btintel_pcie_msix_gp1_handler(struct btintel_pcie_data *data) | 885 | static void btintel_pcie_msix_gp1_handler(struct btintel_pcie_data *data) |
| @@ -914,7 +912,8 @@ static void btintel_pcie_msix_gp0_handler(struct btintel_pcie_data *data) · 26 unmodified lines | |||
| 914 | data->img_resp_cache = reg; | 912 | data->img_resp_cache = reg; |
| 915 | 913 | ||
| 916 | if (btintel_pcie_in_error(data)) { | 914 | if (btintel_pcie_in_error(data)) { |
| 917 | bt_dev_err(data->hdev, "Controller in error state"); | 915 | bt_dev_err(data->hdev, "Controller in error state (boot_stage: 0x%8.8x)", |
| 916 | data->boot_stage_cache); | ||
| 918 | btintel_pcie_dump_debug_registers(data->hdev); | 917 | btintel_pcie_dump_debug_registers(data->hdev); |
| 919 | return; | 918 | return; |
| 920 | } | 919 | } |
| @@ -34,9 +34,6 @@ · 33 unmodified lines | |||
| 34 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_STS (BIT(20)) | 34 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_STS (BIT(20)) |
| 35 | 35 | ||
| 36 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ (BIT(21)) | 36 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ (BIT(21)) |
| 37 | /* Stop MAC Access disconnection request */ | ||
| 38 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS (BIT(22)) | ||
| 39 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ (BIT(23)) | ||
| 40 | 37 | ||
| 41 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_STS (BIT(28)) | 38 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_STS (BIT(28)) |
| 42 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_DISCON (BIT(29)) | 39 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_DISCON (BIT(29)) |
| @@ -48,7 +45,7 @@ · 5 unmodified lines | |||
| 48 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_OPFW (BIT(2)) | 45 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_OPFW (BIT(2)) |
| 49 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ROM_LOCKDOWN (BIT(10)) | 46 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ROM_LOCKDOWN (BIT(10)) |
| 50 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_IML_LOCKDOWN (BIT(11)) | 47 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_IML_LOCKDOWN (BIT(11)) |
| 51 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_ERR (BIT(12)) | 48 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING (BIT(12)) |
| 52 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER (BIT(13)) | 49 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER (BIT(13)) |
| 53 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_HALTED (BIT(14)) | 50 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_HALTED (BIT(14)) |
| 54 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_MAC_ACCESS_ON (BIT(16)) | 51 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_MAC_ACCESS_ON (BIT(16)) |
| @@ -537,6 +537,7 @@ static void btmtk_usb_wmt_recv(struct urb *urb) · 536 unmodified lines | |||
| 537 | return; | 537 | return; |
| 538 | } else if (urb->status == -ENOENT) { | 538 | } else if (urb->status == -ENOENT) { |
| 539 | /* Avoid suspend failed when usb_kill_urb */ | 539 | /* Avoid suspend failed when usb_kill_urb */ |
| 540 | kfree(urb->setup_packet); | ||
| 540 | return; | 541 | return; |
| 541 | } | 542 | } |
| 542 | 543 | ||
| @@ -610,6 +611,7 @@ static int btmtk_usb_submit_wmt_recv_urb(struct hci_dev *hdev) · 67 unmodified lines | |||
| 610 | if (err != -EPERM && err != -ENODEV) | 611 | if (err != -EPERM && err != -ENODEV) |
| 611 | bt_dev_err(hdev, "urb %p submission failed (%d)", | 612 | bt_dev_err(hdev, "urb %p submission failed (%d)", |
| 612 | urb, -err); | 613 | urb, -err); |
| 614 | kfree(dr); | ||
| 613 | usb_unanchor_urb(urb); | 615 | usb_unanchor_urb(urb); |
| 614 | } | 616 | } |
| 615 | 617 | ||
| @@ -695,8 +697,13 @@ static int btmtk_usb_hci_wmt_sync(struct hci_dev *hdev, · 79 unmodified lines | |||
| 695 | if (data->evt_skb == NULL) | 697 | if (data->evt_skb == NULL) |
| 696 | goto err_free_wc; | 698 | goto err_free_wc; |
| 697 | 699 | ||
| 698 | /* Parse and handle the return WMT event */ | 700 | wmt_evt = skb_pull_data(data->evt_skb, sizeof(*wmt_evt)); |
| 699 | wmt_evt = (struct btmtk_hci_wmt_evt *)data->evt_skb->data; | 701 | if (!wmt_evt) { |
| 702 | bt_dev_err(hdev, "WMT event too short (%u bytes)", | ||
| 703 | data->evt_skb->len); | ||
| 704 | err = -EINVAL; | ||
| 705 | goto err_free_skb; | ||
| 706 | } | ||
| 700 | if (wmt_evt->whdr.op != hdr->op) { | 707 | if (wmt_evt->whdr.op != hdr->op) { |
| 701 | bt_dev_err(hdev, "Wrong op received %d expected %d", | 708 | bt_dev_err(hdev, "Wrong op received %d expected %d", |
| 702 | wmt_evt->whdr.op, hdr->op); | 709 | wmt_evt->whdr.op, hdr->op); |
| @@ -712,6 +719,12 @@ static int btmtk_usb_hci_wmt_sync(struct hci_dev *hdev, · 9 unmodified lines | |||
| 712 | status = BTMTK_WMT_PATCH_DONE; | 719 | status = BTMTK_WMT_PATCH_DONE; |
| 713 | break; | 720 | break; |
| 714 | case BTMTK_WMT_FUNC_CTRL: | 721 | case BTMTK_WMT_FUNC_CTRL: |
| 722 | if (!skb_pull_data(data->evt_skb, | ||
| 723 | sizeof(wmt_evt_funcc->status))) { | ||
| 724 | status = BTMTK_WMT_ON_UNDONE; | ||
| 725 | break; | ||
| 726 | } | ||
| 727 | |||
| 715 | wmt_evt_funcc = (struct btmtk_hci_wmt_evt_funcc *)wmt_evt; | 728 | wmt_evt_funcc = (struct btmtk_hci_wmt_evt_funcc *)wmt_evt; |
| 716 | if (be16_to_cpu(wmt_evt_funcc->status) == 0x404) | 729 | if (be16_to_cpu(wmt_evt_funcc->status) == 0x404) |
| 717 | status = BTMTK_WMT_ON_DONE; | 730 | status = BTMTK_WMT_ON_DONE; |
| @@ -3540,7 +3540,13 @@ static int btusb_setup_qca_load_rampatch(struct hci_dev *hdev, · 3539 unmodified lines | |||
| 3540 | "firmware rome 0x%x build 0x%x", | 3540 | "firmware rome 0x%x build 0x%x", |
| 3541 | rver_rom, rver_patch, ver_rom, ver_patch); | 3541 | rver_rom, rver_patch, ver_rom, ver_patch); |
| 3542 | 3542 | ||
| 3543 | if (rver_rom != ver_rom || rver_patch <= ver_patch) { | 3543 | /* Allow rampatch when the patch version equals the firmware version. |
| 3544 | * A firmware download may be aborted by a transient USB error (e.g. | ||
| 3545 | * disconnect) after the controller updates version info but before | ||
| 3546 | * completion. | ||
| 3547 | * Allowing equal versions enables re-flashing during recovery. | ||
| 3548 | */ | ||
| 3549 | if (rver_rom != ver_rom || rver_patch < ver_patch) { | ||
| 3544 | bt_dev_err(hdev, "rampatch file version did not match with firmware"); | 3550 | bt_dev_err(hdev, "rampatch file version did not match with firmware"); |
| 3545 | err = -EINVAL; | 3551 | err = -EINVAL; |
| 3546 | goto done; | 3552 | goto done; |
| @@ -191,6 +191,9 @@ static int ath_recv(struct hci_uart *hu, const void *data, int count) · 190 unmodified lines | |||
| 191 | { | 191 | { |
| 192 | struct ath_struct *ath = hu->priv; | 192 | struct ath_struct *ath = hu->priv; |
| 193 | 193 | ||
| 194 | if (!ath) | ||
| 195 | return -ENODEV; | ||
| 196 | |||
| 194 | ath->rx_skb = h4_recv_buf(hu, ath->rx_skb, data, count, | 197 | ath->rx_skb = h4_recv_buf(hu, ath->rx_skb, data, count, |
| 195 | ath_recv_pkts, ARRAY_SIZE(ath_recv_pkts)); | 198 | ath_recv_pkts, ARRAY_SIZE(ath_recv_pkts)); |
| 196 | if (IS_ERR(ath->rx_skb)) { | 199 | if (IS_ERR(ath->rx_skb)) { |
| @@ -585,6 +585,9 @@ static int bcsp_recv(struct hci_uart *hu, const void *data, int count) · 584 unmodified lines | |||
| 585 | if (!test_bit(HCI_UART_REGISTERED, &hu->flags)) | 585 | if (!test_bit(HCI_UART_REGISTERED, &hu->flags)) |
| 586 | return -EUNATCH; | 586 | return -EUNATCH; |
| 587 | 587 | ||
| 588 | if (!bcsp) | ||
| 589 | return -ENODEV; | ||
| 590 | |||
| 588 | BT_DBG("hu %p count %d rx_state %d rx_count %ld", | 591 | BT_DBG("hu %p count %d rx_state %d rx_count %ld", |
| 589 | hu, count, bcsp->rx_state, bcsp->rx_count); | 592 | hu, count, bcsp->rx_state, bcsp->rx_count); |
| 590 | 593 | ||
| @@ -109,6 +109,9 @@ static int h4_recv(struct hci_uart *hu, const void *data, int count) · 108 unmodified lines | |||
| 109 | { | 109 | { |
| 110 | struct h4_struct *h4 = hu->priv; | 110 | struct h4_struct *h4 = hu->priv; |
| 111 | 111 | ||
| 112 | if (!h4) | ||
| 113 | return -ENODEV; | ||
| 114 | |||
| 112 | h4->rx_skb = h4_recv_buf(hu, h4->rx_skb, data, count, | 115 | h4->rx_skb = h4_recv_buf(hu, h4->rx_skb, data, count, |
| 113 | h4_recv_pkts, ARRAY_SIZE(h4_recv_pkts)); | 116 | h4_recv_pkts, ARRAY_SIZE(h4_recv_pkts)); |
| 114 | if (IS_ERR(h4->rx_skb)) { | 117 | if (IS_ERR(h4->rx_skb)) { |
| @@ -587,6 +587,9 @@ static int h5_recv(struct hci_uart *hu, const void *data, int count) · 586 unmodified lines | |||
| 587 | struct h5 *h5 = hu->priv; | 587 | struct h5 *h5 = hu->priv; |
| 588 | const unsigned char *ptr = data; | 588 | const unsigned char *ptr = data; |
| 589 | 589 | ||
| 590 | if (!h5) | ||
| 591 | return -ENODEV; | ||
| 592 | |||
| 590 | BT_DBG("%s pending %zu count %d", hu->hdev->name, h5->rx_pending, | 593 | BT_DBG("%s pending %zu count %d", hu->hdev->name, h5->rx_pending, |
| 591 | count); | 594 | count); |
| 592 | 595 | ||
| @@ -194,7 +194,15 @@ void hci_uart_init_work(struct work_struct *work) · 193 unmodified lines | |||
| 194 | err = hci_register_dev(hu->hdev); | 194 | err = hci_register_dev(hu->hdev); |
| 195 | if (err < 0) { | 195 | if (err < 0) { |
| 196 | BT_ERR("Can't register HCI device"); | 196 | BT_ERR("Can't register HCI device"); |
| 197 | |||
| 198 | percpu_down_write(&hu->proto_lock); | ||
| 197 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); | 199 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); |
| 200 | percpu_up_write(&hu->proto_lock); | ||
| 201 | |||
| 202 | /* Safely cancel work after clearing flags */ | ||
| 203 | cancel_work_sync(&hu->write_work); | ||
| 204 | |||
| 205 | /* Close protocol before freeing hdev */ | ||
| 198 | hu->proto->close(hu); | 206 | hu->proto->close(hu); |
| 199 | hdev = hu->hdev; | 207 | hdev = hu->hdev; |
| 200 | hu->hdev = NULL; | 208 | hu->hdev = NULL; |
| @@ -263,8 +271,12 @@ static int hci_uart_open(struct hci_dev *hdev) · 62 unmodified lines | |||
| 263 | /* Close device */ | 271 | /* Close device */ |
| 264 | static int hci_uart_close(struct hci_dev *hdev) | 272 | static int hci_uart_close(struct hci_dev *hdev) |
| 265 | { | 273 | { |
| 274 | struct hci_uart *hu = hci_get_drvdata(hdev); | ||
| 275 | |||
| 266 | BT_DBG("hdev %p", hdev); | 276 | BT_DBG("hdev %p", hdev); |
| 267 | 277 | ||
| 278 | cancel_work_sync(&hu->write_work); | ||
| 279 | |||
| 268 | hci_uart_flush(hdev); | 280 | hci_uart_flush(hdev); |
| 269 | hdev->flush = NULL; | 281 | hdev->flush = NULL; |
| 270 | return 0; | 282 | return 0; |
| @@ -531,6 +543,7 @@ static void hci_uart_tty_close(struct tty_struct *tty) · 260 unmodified lines | |||
| 531 | { | 543 | { |
| 532 | struct hci_uart *hu = tty->disc_data; | 544 | struct hci_uart *hu = tty->disc_data; |
| 533 | struct hci_dev *hdev; | 545 | struct hci_dev *hdev; |
| 546 | bool proto_ready; | ||
| 534 | 547 | ||
| 535 | BT_DBG("tty %p", tty); | 548 | BT_DBG("tty %p", tty); |
| 536 | 549 | ||
| @@ -540,24 +553,38 @@ static void hci_uart_tty_close(struct tty_struct *tty) · 3 unmodified lines | |||
| 540 | if (!hu) | 553 | if (!hu) |
| 541 | return; | 554 | return; |
| 542 | 555 | ||
| 543 | hdev = hu->hdev; | 556 | /* Wait for init_ready to finish to prevent registration races */ |
| 544 | if (hdev) | 557 | cancel_work_sync(&hu->init_ready); |
| 545 | hci_uart_close(hdev); | ||
| 546 | 558 | ||
| 547 | if (test_bit(HCI_UART_PROTO_READY, &hu->flags)) { | 559 | proto_ready = test_bit(HCI_UART_PROTO_READY, &hu->flags); |
| 560 | if (proto_ready) { | ||
| 548 | percpu_down_write(&hu->proto_lock); | 561 | percpu_down_write(&hu->proto_lock); |
| 549 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); | 562 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); |
| 550 | percpu_up_write(&hu->proto_lock); | 563 | percpu_up_write(&hu->proto_lock); |
| 564 | } | ||
| 551 | 565 | ||
| 552 | cancel_work_sync(&hu->init_ready); | 566 | /* |
| 553 | cancel_work_sync(&hu->write_work); | 567 | * Unconditionally cancel write_work AFTER clearing PROTO_READY. |
| 568 | * This ensures that concurrent protocol timers cannot requeue | ||
| 569 | * write_work via hci_uart_tx_wakeup(), permanently preventing | ||
| 570 | * double-free races and UAFs. | ||
| 571 | */ | ||
| 572 | cancel_work_sync(&hu->write_work); | ||
| 573 | |||
| 574 | hdev = hu->hdev; | ||
| 575 | if (hdev) | ||
| 576 | hci_uart_close(hdev); /* proto->flush is safely skipped */ | ||
| 554 | 577 | ||
| 578 | if (proto_ready) { | ||
| 555 | if (hdev) { | 579 | if (hdev) { |
| 556 | if (test_bit(HCI_UART_REGISTERED, &hu->flags)) | 580 | if (test_bit(HCI_UART_REGISTERED, &hu->flags)) |
| 557 | hci_unregister_dev(hdev); | 581 | hci_unregister_dev(hdev); |
| 558 | hci_free_dev(hdev); | ||
| 559 | } | 582 | } |
| 583 | /* Close protocol before freeing hdev (intrinsically purges queues) */ | ||
| 560 | hu->proto->close(hu); | 584 | hu->proto->close(hu); |
| 585 | |||
| 586 | if (hdev) | ||
| 587 | hci_free_dev(hdev); | ||
| 561 | } | 588 | } |
| 562 | clear_bit(HCI_UART_PROTO_SET, &hu->flags); | 589 | clear_bit(HCI_UART_PROTO_SET, &hu->flags); |
| 563 | 590 | ||
| @@ -625,11 +652,12 @@ static void hci_uart_tty_receive(struct tty_struct *tty, const u8 *data, · 61 unmodified lines | |||
| 625 | * tty caller | 652 | * tty caller |
| 626 | */ | 653 | */ |
| 627 | hu->proto->recv(hu, data, count); | 654 | hu->proto->recv(hu, data, count); |
| 628 | percpu_up_read(&hu->proto_lock); | ||
| 629 | 655 | ||
| 630 | if (hu->hdev) | 656 | if (hu->hdev) |
| 631 | hu->hdev->stat.byte_rx += count; | 657 | hu->hdev->stat.byte_rx += count; |
| 632 | 658 | ||
| 659 | percpu_up_read(&hu->proto_lock); | ||
| 660 | |||
| 633 | tty_unthrottle(tty); | 661 | tty_unthrottle(tty); |
| 634 | } | 662 | } |
| 635 | 663 | ||
| @@ -695,6 +723,10 @@ static int hci_uart_register_dev(struct hci_uart *hu) · 59 unmodified lines | |||
| 695 | percpu_down_write(&hu->proto_lock); | 723 | percpu_down_write(&hu->proto_lock); |
| 696 | clear_bit(HCI_UART_PROTO_INIT, &hu->flags); | 724 | clear_bit(HCI_UART_PROTO_INIT, &hu->flags); |
| 697 | percpu_up_write(&hu->proto_lock); | 725 | percpu_up_write(&hu->proto_lock); |
| 726 | /* Cancel work after clearing flags */ | ||
| 727 | cancel_work_sync(&hu->write_work); | ||
| 728 | |||
| 729 | /* Close protocol before freeing hdev */ | ||
| 698 | hu->proto->close(hu); | 730 | hu->proto->close(hu); |
| 699 | hu->hdev = NULL; | 731 | hu->hdev = NULL; |
| 700 | hci_free_dev(hdev); | 732 | hci_free_dev(hdev); |
| @@ -48,13 +48,12 @@ · 47 unmodified lines | |||
| 48 | #define HCI_MAX_IBS_SIZE 10 | 48 | #define HCI_MAX_IBS_SIZE 10 |
| 49 | 49 | ||
| 50 | #define IBS_WAKE_RETRANS_TIMEOUT_MS 100 | 50 | #define IBS_WAKE_RETRANS_TIMEOUT_MS 100 |
| 51 | #define IBS_BTSOC_TX_IDLE_TIMEOUT_MS 200 | 51 | #define IBS_BTSOC_TX_IDLE_TIMEOUT msecs_to_jiffies(200) |
| 52 | #define IBS_HOST_TX_IDLE_TIMEOUT_MS 2000 | 52 | #define IBS_HOST_TX_IDLE_TIMEOUT_MS 2000 |
| 53 | #define CMD_TRANS_TIMEOUT_MS 100 | 53 | #define CMD_TRANS_TIMEOUT msecs_to_jiffies(100) |
| 54 | #define MEMDUMP_TIMEOUT_MS 8000 | 54 | #define MEMDUMP_TIMEOUT msecs_to_jiffies(8000) |
| 55 | #define IBS_DISABLE_SSR_TIMEOUT_MS \ | 55 | #define FW_DOWNLOAD_TIMEOUT msecs_to_jiffies(3000) |
| 56 | (MEMDUMP_TIMEOUT_MS + FW_DOWNLOAD_TIMEOUT_MS) | 56 | #define IBS_DISABLE_SSR_TIMEOUT (MEMDUMP_TIMEOUT + FW_DOWNLOAD_TIMEOUT) |
| 57 | #define FW_DOWNLOAD_TIMEOUT_MS 3000 | ||
| 58 | 57 | ||
| 59 | /* susclk rate */ | 58 | /* susclk rate */ |
| 60 | #define SUSCLK_RATE_32KHZ 32768 | 59 | #define SUSCLK_RATE_32KHZ 32768 |
| @@ -1096,7 +1095,7 @@ static void qca_controller_memdump(struct work_struct *work) · 1035 unmodified lines | |||
| 1096 | 1095 | ||
| 1097 | queue_delayed_work(qca->workqueue, | 1096 | queue_delayed_work(qca->workqueue, |
| 1098 | &qca->ctrl_memdump_timeout, | 1097 | &qca->ctrl_memdump_timeout, |
| 1099 | msecs_to_jiffies(MEMDUMP_TIMEOUT_MS)); | 1098 | MEMDUMP_TIMEOUT); |
| 1100 | skb_pull(skb, sizeof(qca_memdump->ram_dump_size)); | 1099 | skb_pull(skb, sizeof(qca_memdump->ram_dump_size)); |
| 1101 | qca_memdump->current_seq_no = 0; | 1100 | qca_memdump->current_seq_no = 0; |
| 1102 | qca_memdump->received_dump = 0; | 1101 | qca_memdump->received_dump = 0; |
| @@ -1369,7 +1368,7 @@ static int qca_set_baudrate(struct hci_dev *hdev, uint8_t baudrate) · 266 unmodified lines | |||
| 1369 | 1368 | ||
| 1370 | if (hu->serdev) | 1369 | if (hu->serdev) |
| 1371 | serdev_device_wait_until_sent(hu->serdev, | 1370 | serdev_device_wait_until_sent(hu->serdev, |
| 1372 | msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS)); | 1371 | CMD_TRANS_TIMEOUT); |
| 1373 | 1372 | ||
| 1374 | /* Give the controller time to process the request */ | 1373 | /* Give the controller time to process the request */ |
| 1375 | switch (qca_soc_type(hu)) { | 1374 | switch (qca_soc_type(hu)) { |
| @@ -1401,8 +1400,8 @@ static inline void host_set_baudrate(struct hci_uart *hu, unsigned int speed) · 25 unmodified lines | |||
| 1401 | 1400 | ||
| 1402 | static int qca_send_power_pulse(struct hci_uart *hu, bool on) | 1401 | static int qca_send_power_pulse(struct hci_uart *hu, bool on) |
| 1403 | { | 1402 | { |
| 1403 | int timeout = CMD_TRANS_TIMEOUT; | ||
| 1404 | int ret; | 1404 | int ret; |
| 1405 | int timeout = msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS); | ||
| 1406 | u8 cmd = on ? QCA_WCN3990_POWERON_PULSE : QCA_WCN3990_POWEROFF_PULSE; | 1405 | u8 cmd = on ? QCA_WCN3990_POWERON_PULSE : QCA_WCN3990_POWEROFF_PULSE; |
| 1407 | 1406 | ||
| 1408 | /* These power pulses are single byte command which are sent | 1407 | /* These power pulses are single byte command which are sent |
| @@ -1607,7 +1606,7 @@ static void qca_wait_for_dump_collection(struct hci_dev *hdev) · 198 unmodified lines | |||
| 1607 | struct qca_data *qca = hu->priv; | 1606 | struct qca_data *qca = hu->priv; |
| 1608 | 1607 | ||
| 1609 | wait_on_bit_timeout(&qca->flags, QCA_MEMDUMP_COLLECTION, | 1608 | wait_on_bit_timeout(&qca->flags, QCA_MEMDUMP_COLLECTION, |
| 1610 | TASK_UNINTERRUPTIBLE, MEMDUMP_TIMEOUT_MS); | 1609 | TASK_UNINTERRUPTIBLE, MEMDUMP_TIMEOUT); |
| 1611 | 1610 | ||
| 1612 | clear_bit(QCA_MEMDUMP_COLLECTION, &qca->flags); | 1611 | clear_bit(QCA_MEMDUMP_COLLECTION, &qca->flags); |
| 1613 | } | 1612 | } |
| @@ -1681,8 +1680,8 @@ static void qca_hw_error(struct hci_dev *hdev, u8 code) · 67 unmodified lines | |||
| 1681 | mod_timer(&qca->tx_idle_timer, jiffies + | 1680 | mod_timer(&qca->tx_idle_timer, jiffies + |
| 1682 | msecs_to_jiffies(qca->tx_idle_delay)); | 1681 | msecs_to_jiffies(qca->tx_idle_delay)); |
| 1683 | 1682 | ||
| 1684 | /* Controller reset completion time is 50ms */ | 1683 | /* Wait for the controller to load the rampatch and NVM. */ |
| 1685 | msleep(50); | 1684 | msleep(100); |
| 1686 | 1685 | ||
| 1687 | clear_bit(QCA_SSR_TRIGGERED, &qca->flags); | 1686 | clear_bit(QCA_SSR_TRIGGERED, &qca->flags); |
| 1688 | clear_bit(QCA_IBS_DISABLED, &qca->flags); | 1687 | clear_bit(QCA_IBS_DISABLED, &qca->flags); |
| @@ -2591,7 +2590,7 @@ static void qca_serdev_remove(struct serdev_device *serdev) · 902 unmodified lines | |||
| 2591 | static void qca_serdev_shutdown(struct serdev_device *serdev) | 2590 | static void qca_serdev_shutdown(struct serdev_device *serdev) |
| 2592 | { | 2591 | { |
| 2593 | int ret; | 2592 | int ret; |
| 2594 | int timeout = msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS); | 2593 | int timeout = CMD_TRANS_TIMEOUT; |
| 2595 | struct qca_serdev *qcadev = serdev_device_get_drvdata(serdev); | 2594 | struct qca_serdev *qcadev = serdev_device_get_drvdata(serdev); |
| 2596 | struct hci_uart *hu = &qcadev->serdev_hu; | 2595 | struct hci_uart *hu = &qcadev->serdev_hu; |
| 2597 | struct hci_dev *hdev = hu->hdev; | 2596 | struct hci_dev *hdev = hu->hdev; |
| @@ -2648,7 +2647,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 50 unmodified lines | |||
| 2648 | bool tx_pending = false; | 2647 | bool tx_pending = false; |
| 2649 | int ret = 0; | 2648 | int ret = 0; |
| 2650 | u8 cmd; | 2649 | u8 cmd; |
| 2651 | u32 wait_timeout = 0; | 2650 | unsigned long wait_timeout = 0; |
| 2652 | 2651 | ||
| 2653 | set_bit(QCA_SUSPENDING, &qca->flags); | 2652 | set_bit(QCA_SUSPENDING, &qca->flags); |
| 2654 | 2653 | ||
| @@ -2669,15 +2668,15 @@ static int __maybe_unused qca_suspend(struct device *dev) · 14 unmodified lines | |||
| 2669 | if (test_bit(QCA_IBS_DISABLED, &qca->flags) || | 2668 | if (test_bit(QCA_IBS_DISABLED, &qca->flags) || |
| 2670 | test_bit(QCA_SSR_TRIGGERED, &qca->flags)) { | 2669 | test_bit(QCA_SSR_TRIGGERED, &qca->flags)) { |
| 2671 | wait_timeout = test_bit(QCA_SSR_TRIGGERED, &qca->flags) ? | 2670 | wait_timeout = test_bit(QCA_SSR_TRIGGERED, &qca->flags) ? |
| 2672 | IBS_DISABLE_SSR_TIMEOUT_MS : | 2671 | IBS_DISABLE_SSR_TIMEOUT : |
| 2673 | FW_DOWNLOAD_TIMEOUT_MS; | 2672 | FW_DOWNLOAD_TIMEOUT; |
| 2674 | 2673 | ||
| 2675 | /* QCA_IBS_DISABLED flag is set to true, During FW download | 2674 | /* QCA_IBS_DISABLED flag is set to true, During FW download |
| 2676 | * and during memory dump collection. It is reset to false, | 2675 | * and during memory dump collection. It is reset to false, |
| 2677 | * After FW download complete. | 2676 | * After FW download complete. |
| 2678 | */ | 2677 | */ |
| 2679 | wait_on_bit_timeout(&qca->flags, QCA_IBS_DISABLED, | 2678 | wait_on_bit_timeout(&qca->flags, QCA_IBS_DISABLED, |
| 2680 | TASK_UNINTERRUPTIBLE, msecs_to_jiffies(wait_timeout)); | 2679 | TASK_UNINTERRUPTIBLE, wait_timeout); |
| 2681 | 2680 | ||
| 2682 | if (test_bit(QCA_IBS_DISABLED, &qca->flags)) { | 2681 | if (test_bit(QCA_IBS_DISABLED, &qca->flags)) { |
| 2683 | bt_dev_err(hu->hdev, "SSR or FW download time out"); | 2682 | bt_dev_err(hu->hdev, "SSR or FW download time out"); |
| @@ -2729,7 +2728,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 45 unmodified lines | |||
| 2729 | 2728 | ||
| 2730 | if (tx_pending) { | 2729 | if (tx_pending) { |
| 2731 | serdev_device_wait_until_sent(hu->serdev, | 2730 | serdev_device_wait_until_sent(hu->serdev, |
| 2732 | msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS)); | 2731 | CMD_TRANS_TIMEOUT); |
| 2733 | serial_clock_vote(HCI_IBS_TX_VOTE_CLOCK_OFF, hu); | 2732 | serial_clock_vote(HCI_IBS_TX_VOTE_CLOCK_OFF, hu); |
| 2734 | } | 2733 | } |
| 2735 | 2734 | ||
| @@ -2738,7 +2737,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 2 unmodified lines | |||
| 2738 | */ | 2737 | */ |
| 2739 | ret = wait_event_interruptible_timeout(qca->suspend_wait_q, | 2738 | ret = wait_event_interruptible_timeout(qca->suspend_wait_q, |
| 2740 | qca->rx_ibs_state == HCI_IBS_RX_ASLEEP, | 2739 | qca->rx_ibs_state == HCI_IBS_RX_ASLEEP, |
| 2741 | msecs_to_jiffies(IBS_BTSOC_TX_IDLE_TIMEOUT_MS)); | 2740 | IBS_BTSOC_TX_IDLE_TIMEOUT); |
| 2742 | if (ret == 0) { | 2741 | if (ret == 0) { |
| 2743 | ret = -ETIMEDOUT; | 2742 | ret = -ETIMEDOUT; |
| 2744 | goto error; | 2743 | goto error; |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | |||
| 12 | #include <net/bluetooth/hci_core.h> | 12 | #include <net/bluetooth/hci_core.h> |
| 13 | 13 | ||
| 14 | #define VERSION "0.1" | 14 | #define VERSION "0.1" |
| 15 | #define VIRTBT_RX_BUF_SIZE 1000 | ||
| 15 | 16 | ||
| 16 | enum { | 17 | enum { |
| 17 | VIRTBT_VQ_TX, | 18 | VIRTBT_VQ_TX, |
| @@ -33,11 +34,11 @@ static int virtbt_add_inbuf(struct virtio_bluetooth *vbt) · 15 unmodified lines | |||
| 33 | struct sk_buff *skb; | 34 | struct sk_buff *skb; |
| 34 | int err; | 35 | int err; |
| 35 | 36 | ||
| 36 | skb = alloc_skb(1000, GFP_KERNEL); | 37 | skb = alloc_skb(VIRTBT_RX_BUF_SIZE, GFP_KERNEL); |
| 37 | if (!skb) | 38 | if (!skb) |
| 38 | return -ENOMEM; | 39 | return -ENOMEM; |
| 39 | 40 | ||
| 40 | sg_init_one(sg, skb->data, 1000); | 41 | sg_init_one(sg, skb->data, VIRTBT_RX_BUF_SIZE); |
| 41 | 42 | ||
| 42 | err = virtqueue_add_inbuf(vq, sg, 1, skb, GFP_KERNEL); | 43 | err = virtqueue_add_inbuf(vq, sg, 1, skb, GFP_KERNEL); |
| 43 | if (err < 0) { | 44 | if (err < 0) { |
| @@ -197,6 +198,7 @@ static int virtbt_shutdown_generic(struct hci_dev *hdev) · 153 unmodified lines | |||
| 197 | 198 | ||
| 198 | static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb) | 199 | static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb) |
| 199 | { | 200 | { |
| 201 | size_t min_hdr; | ||
| 200 | __u8 pkt_type; | 202 | __u8 pkt_type; |
| 201 | 203 | ||
| 202 | pkt_type = *((__u8 *) skb->data); | 204 | pkt_type = *((__u8 *) skb->data); |
| @@ -204,16 +206,32 @@ static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb) · 1 unmodified line | |||
| 204 | 206 | ||
| 205 | switch (pkt_type) { | 207 | switch (pkt_type) { |
| 206 | case HCI_EVENT_PKT: | 208 | case HCI_EVENT_PKT: |
| 209 | min_hdr = sizeof(struct hci_event_hdr); | ||
| 210 | break; | ||
| 207 | case HCI_ACLDATA_PKT: | 211 | case HCI_ACLDATA_PKT: |
| 212 | min_hdr = sizeof(struct hci_acl_hdr); | ||
| 213 | break; | ||
| 208 | case HCI_SCODATA_PKT: | 214 | case HCI_SCODATA_PKT: |
| 215 | min_hdr = sizeof(struct hci_sco_hdr); | ||
| 216 | break; | ||
| 209 | case HCI_ISODATA_PKT: | 217 | case HCI_ISODATA_PKT: |
| 210 | hci_skb_pkt_type(skb) = pkt_type; | 218 | min_hdr = sizeof(struct hci_iso_hdr); |
| 211 | hci_recv_frame(vbt->hdev, skb); | ||
| 212 | break; | 219 | break; |
| 213 | default: | 220 | default: |
| 214 | kfree_skb(skb); | 221 | kfree_skb(skb); |
| 215 | break; | 222 | return; |
| 223 | } | ||
| 224 | |||
| 225 | if (skb->len < min_hdr) { | ||
| 226 | bt_dev_err_ratelimited(vbt->hdev, | ||
| 227 | "rx pkt_type 0x%02x payload %u < hdr %zu\n", | ||
| 228 | pkt_type, skb->len, min_hdr); | ||
| 229 | kfree_skb(skb); | ||
| 230 | return; | ||
| 216 | } | 231 | } |
| 232 | |||
| 233 | hci_skb_pkt_type(skb) = pkt_type; | ||
| 234 | hci_recv_frame(vbt->hdev, skb); | ||
| 217 | } | 235 | } |
| 218 | 236 | ||
| 219 | static void virtbt_rx_work(struct work_struct *work) | 237 | static void virtbt_rx_work(struct work_struct *work) |
| @@ -227,8 +245,15 @@ static void virtbt_rx_work(struct work_struct *work) · 7 unmodified lines | |||
| 227 | if (!skb) | 245 | if (!skb) |
| 228 | return; | 246 | return; |
| 229 | 247 | ||
| 230 | skb_put(skb, len); | 248 | if (!len || len > VIRTBT_RX_BUF_SIZE) { |
| 231 | virtbt_rx_handle(vbt, skb); | 249 | bt_dev_err_ratelimited(vbt->hdev, |
| 250 | "rx reply len %u outside [1, %u]\n", | ||
| 251 | len, VIRTBT_RX_BUF_SIZE); | ||
| 252 | kfree_skb(skb); | ||
| 253 | } else { | ||
| 254 | skb_put(skb, len); | ||
| 255 | virtbt_rx_handle(vbt, skb); | ||
| 256 | } | ||
| 232 | 257 | ||
| 233 | if (virtbt_add_inbuf(vbt) < 0) | 258 | if (virtbt_add_inbuf(vbt) < 0) |
| 234 | return; | 259 | return; |
| @@ -631,6 +631,16 @@ int register_cdrom(struct gendisk *disk, struct cdrom_device_info *cdi) · 630 unmodified lines | |||
| 631 | 631 | ||
| 632 | WARN_ON(!cdo->generic_packet); | 632 | WARN_ON(!cdo->generic_packet); |
| 633 | 633 | ||
| 634 | /* | ||
| 635 | * Propagate the drive's write support to the block layer so BLKROGET | ||
| 636 | * reflects actual write capability. Drivers that use GET CONFIGURATION | ||
| 637 | * features (CDC_MRW_W, CDC_RAM) must have called | ||
| 638 | * cdrom_probe_write_features() before register_cdrom() so the mask is | ||
| 639 | * complete here. | ||
| 640 | */ | ||
| 641 | set_disk_ro(disk, !CDROM_CAN(CDC_DVD_RAM | CDC_MRW_W | CDC_RAM | | ||
| 642 | CDC_CD_RW)); | ||
| 643 | |||
| 634 | cd_dbg(CD_REG_UNREG, "drive \"/dev/%s\" registered\n", cdi->name); | 644 | cd_dbg(CD_REG_UNREG, "drive \"/dev/%s\" registered\n", cdi->name); |
| 635 | mutex_lock(&cdrom_mutex); | 645 | mutex_lock(&cdrom_mutex); |
| 636 | list_add(&cdi->list, &cdrom_list); | 646 | list_add(&cdi->list, &cdrom_list); |
| @@ -742,6 +752,44 @@ static int cdrom_is_random_writable(struct cdrom_device_info *cdi, int *write) · 105 unmodified lines | |||
| 742 | return 0; | 752 | return 0; |
| 743 | } | 753 | } |
| 744 | 754 | ||
| 755 | /* | ||
| 756 | * Probe write-related MMC features via GET CONFIGURATION and update | ||
| 757 | * cdi->mask accordingly. Drivers that populate cdi->mask from the MODE SENSE | ||
| 758 | * capabilities page (e.g. sr) should call this after those MODE SENSE bits | ||
| 759 | * have been set but before register_cdrom(), so that the full set of | ||
| 760 | * write-capability bits is known by the time register_cdrom() decides on the | ||
| 761 | * initial read-only state of the disk. | ||
| 762 | */ | ||
| 763 | void cdrom_probe_write_features(struct cdrom_device_info *cdi) | ||
| 764 | { | ||
| 765 | int mrw, mrw_write, ram_write; | ||
| 766 | |||
| 767 | mrw = 0; | ||
| 768 | if (!cdrom_is_mrw(cdi, &mrw_write)) | ||
| 769 | mrw = 1; | ||
| 770 | |||
| 771 | if (CDROM_CAN(CDC_MO_DRIVE)) | ||
| 772 | ram_write = 1; | ||
| 773 | else | ||
| 774 | (void) cdrom_is_random_writable(cdi, &ram_write); | ||
| 775 | |||
| 776 | if (mrw) | ||
| 777 | cdi->mask &= ~CDC_MRW; | ||
| 778 | else | ||
| 779 | cdi->mask |= CDC_MRW; | ||
| 780 | |||
| 781 | if (mrw_write) | ||
| 782 | cdi->mask &= ~CDC_MRW_W; | ||
| 783 | else | ||
| 784 | cdi->mask |= CDC_MRW_W; | ||
| 785 | |||
| 786 | if (ram_write) | ||
| 787 | cdi->mask &= ~CDC_RAM; | ||
| 788 | else | ||
| 789 | cdi->mask |= CDC_RAM; | ||
| 790 | } | ||
| 791 | EXPORT_SYMBOL(cdrom_probe_write_features); | ||
| 792 | |||
| 745 | static int cdrom_media_erasable(struct cdrom_device_info *cdi) | 793 | static int cdrom_media_erasable(struct cdrom_device_info *cdi) |
| 746 | { | 794 | { |
| 747 | disc_information di; | 795 | disc_information di; |
| @@ -894,33 +942,8 @@ static int cdrom_is_dvd_rw(struct cdrom_device_info *cdi) · 146 unmodified lines | |||
| 894 | */ | 942 | */ |
| 895 | static int cdrom_open_write(struct cdrom_device_info *cdi) | 943 | static int cdrom_open_write(struct cdrom_device_info *cdi) |
| 896 | { | 944 | { |
| 897 | int mrw, mrw_write, ram_write; | ||
| 898 | int ret = 1; | 945 | int ret = 1; |
| 899 | 946 | ||
| 900 | mrw = 0; | ||
| 901 | if (!cdrom_is_mrw(cdi, &mrw_write)) | ||
| 902 | mrw = 1; | ||
| 903 | |||
| 904 | if (CDROM_CAN(CDC_MO_DRIVE)) | ||
| 905 | ram_write = 1; | ||
| 906 | else | ||
| 907 | (void) cdrom_is_random_writable(cdi, &ram_write); | ||
| 908 | |||
| 909 | if (mrw) | ||
| 910 | cdi->mask &= ~CDC_MRW; | ||
| 911 | else | ||
| 912 | cdi->mask |= CDC_MRW; | ||
| 913 | |||
| 914 | if (mrw_write) | ||
| 915 | cdi->mask &= ~CDC_MRW_W; | ||
| 916 | else | ||
| 917 | cdi->mask |= CDC_MRW_W; | ||
| 918 | |||
| 919 | if (ram_write) | ||
| 920 | cdi->mask &= ~CDC_RAM; | ||
| 921 | else | ||
| 922 | cdi->mask |= CDC_RAM; | ||
| 923 | |||
| 924 | if (CDROM_CAN(CDC_MRW_W)) | 947 | if (CDROM_CAN(CDC_MRW_W)) |
| 925 | ret = cdrom_mrw_open_write(cdi); | 948 | ret = cdrom_mrw_open_write(cdi); |
| 926 | else if (CDROM_CAN(CDC_DVD_RAM)) | 949 | else if (CDROM_CAN(CDC_DVD_RAM)) |
| @@ -168,6 +168,10 @@ struct smi_info { · 167 unmodified lines | |||
| 168 | OEM2_DATA_AVAIL) | 168 | OEM2_DATA_AVAIL) |
| 169 | unsigned char msg_flags; | 169 | unsigned char msg_flags; |
| 170 | 170 | ||
| 171 | /* When requesting events and messages, don't do it forever. */ | ||
| 172 | unsigned int num_requests_in_a_row; | ||
| 173 | bool last_was_flag_fetch; | ||
| 174 | |||
| 171 | /* Does the BMC have an event buffer? */ | 175 | /* Does the BMC have an event buffer? */ |
| 172 | bool has_event_buffer; | 176 | bool has_event_buffer; |
| 173 | 177 | ||
| @@ -410,7 +414,10 @@ static void start_getting_msg_queue(struct smi_info *smi_info) · 236 unmodified lines | |||
| 410 | 414 | ||
| 411 | start_new_msg(smi_info, smi_info->curr_msg->data, | 415 | start_new_msg(smi_info, smi_info->curr_msg->data, |
| 412 | smi_info->curr_msg->data_size); | 416 | smi_info->curr_msg->data_size); |
| 413 | smi_info->si_state = SI_GETTING_MESSAGES; | 417 | if (smi_info->si_state != SI_GETTING_MESSAGES) { |
| 418 | smi_info->num_requests_in_a_row = 0; | ||
| 419 | smi_info->si_state = SI_GETTING_MESSAGES; | ||
| 420 | } | ||
| 414 | } | 421 | } |
| 415 | 422 | ||
| 416 | static void start_getting_events(struct smi_info *smi_info) | 423 | static void start_getting_events(struct smi_info *smi_info) |
| @@ -421,7 +428,10 @@ static void start_getting_events(struct smi_info *smi_info) · 4 unmodified lines | |||
| 421 | 428 | ||
| 422 | start_new_msg(smi_info, smi_info->curr_msg->data, | 429 | start_new_msg(smi_info, smi_info->curr_msg->data, |
| 423 | smi_info->curr_msg->data_size); | 430 | smi_info->curr_msg->data_size); |
| 424 | smi_info->si_state = SI_GETTING_EVENTS; | 431 | if (smi_info->si_state != SI_GETTING_EVENTS) { |
| 432 | smi_info->num_requests_in_a_row = 0; | ||
| 433 | smi_info->si_state = SI_GETTING_EVENTS; | ||
| 434 | } | ||
| 425 | } | 435 | } |
| 426 | 436 | ||
| 427 | /* | 437 | /* |
| @@ -487,15 +497,19 @@ retry: · 59 unmodified lines | |||
| 487 | } else if (smi_info->msg_flags & RECEIVE_MSG_AVAIL) { | 497 | } else if (smi_info->msg_flags & RECEIVE_MSG_AVAIL) { |
| 488 | /* Messages available. */ | 498 | /* Messages available. */ |
| 489 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); | 499 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); |
| 490 | if (!smi_info->curr_msg) | 500 | if (!smi_info->curr_msg) { |
| 501 | smi_info->si_state = SI_NORMAL; | ||
| 491 | return; | 502 | return; |
| 503 | } | ||
| 492 | 504 | ||
| 493 | start_getting_msg_queue(smi_info); | 505 | start_getting_msg_queue(smi_info); |
| 494 | } else if (smi_info->msg_flags & EVENT_MSG_BUFFER_FULL) { | 506 | } else if (smi_info->msg_flags & EVENT_MSG_BUFFER_FULL) { |
| 495 | /* Events available. */ | 507 | /* Events available. */ |
| 496 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); | 508 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); |
| 497 | if (!smi_info->curr_msg) | 509 | if (!smi_info->curr_msg) { |
| 510 | smi_info->si_state = SI_NORMAL; | ||
| 498 | return; | 511 | return; |
| 512 | } | ||
| 499 | 513 | ||
| 500 | start_getting_events(smi_info); | 514 | start_getting_events(smi_info); |
| 501 | } else if (smi_info->msg_flags & OEM_DATA_AVAIL && | 515 | } else if (smi_info->msg_flags & OEM_DATA_AVAIL && |
| @@ -595,6 +609,7 @@ static void handle_transaction_done(struct smi_info *smi_info) · 93 unmodified lines | |||
| 595 | smi_info->si_state = SI_NORMAL; | 609 | smi_info->si_state = SI_NORMAL; |
| 596 | } else { | 610 | } else { |
| 597 | smi_info->msg_flags = msg[3]; | 611 | smi_info->msg_flags = msg[3]; |
| 612 | smi_info->last_was_flag_fetch = true; | ||
| 598 | handle_flags(smi_info); | 613 | handle_flags(smi_info); |
| 599 | } | 614 | } |
| 600 | break; | 615 | break; |
| @@ -630,7 +645,13 @@ static void handle_transaction_done(struct smi_info *smi_info) · 29 unmodified lines | |||
| 630 | */ | 645 | */ |
| 631 | msg = smi_info->curr_msg; | 646 | msg = smi_info->curr_msg; |
| 632 | smi_info->curr_msg = NULL; | 647 | smi_info->curr_msg = NULL; |
| 633 | if (msg->rsp[2] != 0) { | 648 | /* |
| 649 | * It appears some BMCs, with no event data, return no | ||
| 650 | * data in the message and not a 0x80 error as the | ||
| 651 | * spec says they should. Shut down processing if | ||
| 652 | * the data is not the right length. | ||
| 653 | */ | ||
| 654 | if (msg->rsp[2] != 0 || msg->rsp_size != 19) { | ||
| 634 | /* Error getting event, probably done. */ | 655 | /* Error getting event, probably done. */ |
| 635 | msg->done(msg); | 656 | msg->done(msg); |
| 636 | 657 | ||
| @@ -640,6 +661,11 @@ static void handle_transaction_done(struct smi_info *smi_info) · 3 unmodified lines | |||
| 640 | } else { | 661 | } else { |
| 641 | smi_inc_stat(smi_info, events); | 662 | smi_inc_stat(smi_info, events); |
| 642 | 663 | ||
| 664 | smi_info->num_requests_in_a_row++; | ||
| 665 | if (smi_info->num_requests_in_a_row > 10) | ||
| 666 | /* Stop if we do this too many times. */ | ||
| 667 | smi_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; | ||
| 668 | |||
| 643 | /* | 669 | /* |
| 644 | * Do this before we deliver the message | 670 | * Do this before we deliver the message |
| 645 | * because delivering the message releases the | 671 | * because delivering the message releases the |
| @@ -678,6 +704,11 @@ static void handle_transaction_done(struct smi_info *smi_info) · 32 unmodified lines | |||
| 678 | } else { | 704 | } else { |
| 679 | smi_inc_stat(smi_info, incoming_messages); | 705 | smi_inc_stat(smi_info, incoming_messages); |
| 680 | 706 | ||
| 707 | smi_info->num_requests_in_a_row++; | ||
| 708 | if (smi_info->num_requests_in_a_row > 10) | ||
| 709 | /* Stop if we do this too many times. */ | ||
| 710 | smi_info->msg_flags &= ~RECEIVE_MSG_AVAIL; | ||
| 711 | |||
| 681 | /* | 712 | /* |
| 682 | * Do this before we deliver the message | 713 | * Do this before we deliver the message |
| 683 | * because delivering the message releases the | 714 | * because delivering the message releases the |
| @@ -819,6 +850,26 @@ restart: · 135 unmodified lines | |||
| 819 | goto out; | 850 | goto out; |
| 820 | } | 851 | } |
| 821 | 852 | ||
| 853 | /* | ||
| 854 | * If we are currently idle, or if the last thing that was | ||
| 855 | * done was a flag fetch and there is a message pending, try | ||
| 856 | * to start the next message. | ||
| 857 | * | ||
| 858 | * We do the waiting message check to avoid a stuck flag | ||
| 859 | * completely wedging the driver. Let a message through | ||
| 860 | * in between flag operations if that happens. | ||
| 861 | */ | ||
| 862 | if (si_sm_result == SI_SM_IDLE || | ||
| 863 | (si_sm_result == SI_SM_ATTN && smi_info->waiting_msg && | ||
| 864 | smi_info->last_was_flag_fetch)) { | ||
| 865 | smi_info->last_was_flag_fetch = false; | ||
| 866 | smi_inc_stat(smi_info, idles); | ||
| 867 | |||
| 868 | si_sm_result = start_next_msg(smi_info); | ||
| 869 | if (si_sm_result != SI_SM_IDLE) | ||
| 870 | goto restart; | ||
| 871 | } | ||
| 872 | |||
| 822 | /* | 873 | /* |
| 823 | * We prefer handling attn over new messages. But don't do | 874 | * We prefer handling attn over new messages. But don't do |
| 824 | * this if there is not yet an upper layer to handle anything. | 875 | * this if there is not yet an upper layer to handle anything. |
| @@ -846,15 +897,6 @@ restart: · 21 unmodified lines | |||
| 846 | } | 897 | } |
| 847 | } | 898 | } |
| 848 | 899 | ||
| 849 | /* If we are currently idle, try to start the next message. */ | ||
| 850 | if (si_sm_result == SI_SM_IDLE) { | ||
| 851 | smi_inc_stat(smi_info, idles); | ||
| 852 | |||
| 853 | si_sm_result = start_next_msg(smi_info); | ||
| 854 | if (si_sm_result != SI_SM_IDLE) | ||
| 855 | goto restart; | ||
| 856 | } | ||
| 857 | |||
| 858 | if ((si_sm_result == SI_SM_IDLE) | 900 | if ((si_sm_result == SI_SM_IDLE) |
| 859 | && (atomic_read(&smi_info->req_events))) { | 901 | && (atomic_read(&smi_info->req_events))) { |
| 860 | /* | 902 | /* |
| @@ -225,6 +225,9 @@ struct ssif_info { · 224 unmodified lines | |||
| 225 | bool has_event_buffer; | 225 | bool has_event_buffer; |
| 226 | bool supports_alert; | 226 | bool supports_alert; |
| 227 | 227 | ||
| 228 | /* When requesting events and messages, don't do it forever. */ | ||
| 229 | unsigned int num_requests_in_a_row; | ||
| 230 | |||
| 228 | /* | 231 | /* |
| 229 | * Used to tell what we should do with alerts. If we are | 232 | * Used to tell what we should do with alerts. If we are |
| 230 | * waiting on a response, read the data immediately. | 233 | * waiting on a response, read the data immediately. |
| @@ -413,7 +416,10 @@ static void start_event_fetch(struct ssif_info *ssif_info, unsigned long *flags) · 182 unmodified lines | |||
| 413 | } | 416 | } |
| 414 | 417 | ||
| 415 | ssif_info->curr_msg = msg; | 418 | ssif_info->curr_msg = msg; |
| 416 | ssif_info->ssif_state = SSIF_GETTING_EVENTS; | 419 | if (ssif_info->ssif_state != SSIF_GETTING_EVENTS) { |
| 420 | ssif_info->num_requests_in_a_row = 0; | ||
| 421 | ssif_info->ssif_state = SSIF_GETTING_EVENTS; | ||
| 422 | } | ||
| 417 | ipmi_ssif_unlock_cond(ssif_info, flags); | 423 | ipmi_ssif_unlock_cond(ssif_info, flags); |
| 418 | 424 | ||
| 419 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); | 425 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); |
| @@ -436,7 +442,10 @@ static void start_recv_msg_fetch(struct ssif_info *ssif_info, · 16 unmodified lines | |||
| 436 | } | 442 | } |
| 437 | 443 | ||
| 438 | ssif_info->curr_msg = msg; | 444 | ssif_info->curr_msg = msg; |
| 439 | ssif_info->ssif_state = SSIF_GETTING_MESSAGES; | 445 | if (ssif_info->ssif_state != SSIF_GETTING_MESSAGES) { |
| 446 | ssif_info->num_requests_in_a_row = 0; | ||
| 447 | ssif_info->ssif_state = SSIF_GETTING_MESSAGES; | ||
| 448 | } | ||
| 440 | ipmi_ssif_unlock_cond(ssif_info, flags); | 449 | ipmi_ssif_unlock_cond(ssif_info, flags); |
| 441 | 450 | ||
| 442 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); | 451 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); |
| @@ -843,6 +852,11 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result, · 400 unmodified lines | |||
| 843 | ssif_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; | 852 | ssif_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; |
| 844 | handle_flags(ssif_info, flags); | 853 | handle_flags(ssif_info, flags); |
| 845 | } else { | 854 | } else { |
| 855 | ssif_info->num_requests_in_a_row++; | ||
| 856 | if (ssif_info->num_requests_in_a_row > 10) | ||
| 857 | /* Stop if we do this too many times. */ | ||
| 858 | ssif_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; | ||
| 859 | |||
| 846 | handle_flags(ssif_info, flags); | 860 | handle_flags(ssif_info, flags); |
| 847 | ssif_inc_stat(ssif_info, events); | 861 | ssif_inc_stat(ssif_info, events); |
| 848 | deliver_recv_msg(ssif_info, msg); | 862 | deliver_recv_msg(ssif_info, msg); |
| @@ -876,6 +890,11 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result, · 27 unmodified lines | |||
| 876 | ssif_info->msg_flags &= ~RECEIVE_MSG_AVAIL; | 890 | ssif_info->msg_flags &= ~RECEIVE_MSG_AVAIL; |
| 877 | handle_flags(ssif_info, flags); | 891 | handle_flags(ssif_info, flags); |
| 878 | } else { | 892 | } else { |
| 893 | ssif_info->num_requests_in_a_row++; | ||
| 894 | if (ssif_info->num_requests_in_a_row > 10) | ||
| 895 | /* Stop if we do this too many times. */ | ||
| 896 | ssif_info->msg_flags &= ~RECEIVE_MSG_AVAIL; | ||
| 897 | |||
| 879 | ssif_inc_stat(ssif_info, incoming_messages); | 898 | ssif_inc_stat(ssif_info, incoming_messages); |
| 880 | handle_flags(ssif_info, flags); | 899 | handle_flags(ssif_info, flags); |
| 881 | deliver_recv_msg(ssif_info, msg); | 900 | deliver_recv_msg(ssif_info, msg); |
| @@ -1886,6 +1905,7 @@ static int ssif_probe(struct i2c_client *client) · 1004 unmodified lines | |||
| 1886 | "kssif%4.4x", thread_num); | 1905 | "kssif%4.4x", thread_num); |
| 1887 | if (IS_ERR(ssif_info->thread)) { | 1906 | if (IS_ERR(ssif_info->thread)) { |
| 1888 | rv = PTR_ERR(ssif_info->thread); | 1907 | rv = PTR_ERR(ssif_info->thread); |
| 1908 | ssif_info->thread = NULL; | ||
| 1889 | dev_notice(&ssif_info->client->dev, | 1909 | dev_notice(&ssif_info->client->dev, |
| 1890 | "Could not start kernel thread: error %d\n", | 1910 | "Could not start kernel thread: error %d\n", |
| 1891 | rv); | 1911 | rv); |
| @@ -110,6 +110,7 @@ struct eqc_match_data { · 109 unmodified lines | |||
| 110 | 110 | ||
| 111 | const char *reset_auxdev_name; | 111 | const char *reset_auxdev_name; |
| 112 | const char *pinctrl_auxdev_name; | 112 | const char *pinctrl_auxdev_name; |
| 113 | const char *eth_phy_auxdev_name; | ||
| 113 | 114 | ||
| 114 | unsigned int early_clk_count; | 115 | unsigned int early_clk_count; |
| 115 | }; | 116 | }; |
| @@ -321,38 +322,18 @@ static void eqc_probe_init_fixed_factors(struct device *dev, · 205 unmodified lines | |||
| 321 | } | 322 | } |
| 322 | } | 323 | } |
| 323 | 324 | ||
| 324 | static void eqc_auxdev_release(struct device *dev) | 325 | static void eqc_auxdev_create_optional(struct device *dev, void __iomem *base, |
| 325 | { | 326 | const char *name) |
| 326 | struct auxiliary_device *adev = to_auxiliary_dev(dev); | ||
| 327 | |||
| 328 | kfree(adev); | ||
| 329 | } | ||
| 330 | |||
| 331 | static int eqc_auxdev_create(struct device *dev, void __iomem *base, | ||
| 332 | const char *name, u32 id) | ||
| 333 | { | 327 | { |
| 334 | struct auxiliary_device *adev; | 328 | struct auxiliary_device *adev; |
| 335 | int ret; | ||
| 336 | |||
| 337 | adev = kzalloc_obj(*adev); | ||
| 338 | if (!adev) | ||
| 339 | return -ENOMEM; | ||
| 340 | |||
| 341 | adev->name = name; | ||
| 342 | adev->dev.parent = dev; | ||
| 343 | adev->dev.platform_data = (void __force *)base; | ||
| 344 | adev->dev.release = eqc_auxdev_release; | ||
| 345 | adev->id = id; | ||
| 346 | 329 | ||
| 347 | ret = auxiliary_device_init(adev); | 330 | if (name) { |
| 348 | if (ret) | 331 | adev = devm_auxiliary_device_create(dev, name, |
| 349 | return ret; | 332 | (void __force *)base); |
| 350 | 333 | if (!adev) | |
| 351 | ret = auxiliary_device_add(adev); | 334 | dev_warn(dev, "failed creating auxiliary device %s.%s\n", |
| 352 | if (ret) | 335 | KBUILD_MODNAME, name); |
| 353 | auxiliary_device_uninit(adev); | 336 | } |
| 354 | |||
| 355 | return ret; | ||
| 356 | } | 337 | } |
| 357 | 338 | ||
| 358 | static int eqc_probe(struct platform_device *pdev) | 339 | static int eqc_probe(struct platform_device *pdev) |
| @@ -364,7 +345,6 @@ static int eqc_probe(struct platform_device *pdev) · 5 unmodified lines | |||
| 364 | unsigned int i, clk_count; | 345 | unsigned int i, clk_count; |
| 365 | struct resource *res; | 346 | struct resource *res; |
| 366 | void __iomem *base; | 347 | void __iomem *base; |
| 367 | int ret; | ||
| 368 | 348 | ||
| 369 | data = device_get_match_data(dev); | 349 | data = device_get_match_data(dev); |
| 370 | if (!data) | 350 | if (!data) |
| @@ -378,21 +358,10 @@ static int eqc_probe(struct platform_device *pdev) · 7 unmodified lines | |||
| 378 | if (!base) | 358 | if (!base) |
| 379 | return -ENOMEM; | 359 | return -ENOMEM; |
| 380 | 360 | ||
| 381 | /* Init optional reset auxiliary device. */ | 361 | /* Init optional auxiliary devices. */ |
| 382 | if (data->reset_auxdev_name) { | 362 | eqc_auxdev_create_optional(dev, base, data->reset_auxdev_name); |
| 383 | ret = eqc_auxdev_create(dev, base, data->reset_auxdev_name, 0); | 363 | eqc_auxdev_create_optional(dev, base, data->pinctrl_auxdev_name); |
| 384 | if (ret) | 364 | eqc_auxdev_create_optional(dev, base, data->eth_phy_auxdev_name); |
| 385 | dev_warn(dev, "failed creating auxiliary device %s.%s: %d\n", | ||
| 386 | KBUILD_MODNAME, data->reset_auxdev_name, ret); | ||
| 387 | } | ||
| 388 | |||
| 389 | /* Init optional pinctrl auxiliary device. */ | ||
| 390 | if (data->pinctrl_auxdev_name) { | ||
| 391 | ret = eqc_auxdev_create(dev, base, data->pinctrl_auxdev_name, 0); | ||
| 392 | if (ret) | ||
| 393 | dev_warn(dev, "failed creating auxiliary device %s.%s: %d\n", | ||
| 394 | KBUILD_MODNAME, data->pinctrl_auxdev_name, ret); | ||
| 395 | } | ||
| 396 | 365 | ||
| 397 | if (data->pll_count + data->div_count + data->fixed_factor_count == 0) | 366 | if (data->pll_count + data->div_count + data->fixed_factor_count == 0) |
| 398 | return 0; /* Zero clocks, we are done. */ | 367 | return 0; /* Zero clocks, we are done. */ |
| @@ -553,6 +522,7 @@ static const struct eqc_match_data eqc_eyeq5_match_data = { · 154 unmodified lines | |||
| 553 | 522 | ||
| 554 | .reset_auxdev_name = "reset", | 523 | .reset_auxdev_name = "reset", |
| 555 | .pinctrl_auxdev_name = "pinctrl", | 524 | .pinctrl_auxdev_name = "pinctrl", |
| 525 | .eth_phy_auxdev_name = "phy", | ||
| 556 | 526 | ||
| 557 | .early_clk_count = ARRAY_SIZE(eqc_eyeq5_early_plls) + | 527 | .early_clk_count = ARRAY_SIZE(eqc_eyeq5_early_plls) + |
| 558 | ARRAY_SIZE(eqc_eyeq5_early_fixed_factors), | 528 | ARRAY_SIZE(eqc_eyeq5_early_fixed_factors), |
| @@ -153,7 +153,7 @@ static int rk808_clkout_probe(struct platform_device *pdev) · 152 unmodified lines | |||
| 153 | struct rk808_clkout *rk808_clkout; | 153 | struct rk808_clkout *rk808_clkout; |
| 154 | int ret; | 154 | int ret; |
| 155 | 155 | ||
| 156 | dev->of_node = pdev->dev.parent->of_node; | 156 | device_set_of_node_from_dev(dev, dev->parent); |
| 157 | 157 | ||
| 158 | rk808_clkout = devm_kzalloc(dev, | 158 | rk808_clkout = devm_kzalloc(dev, |
| 159 | sizeof(*rk808_clkout), GFP_KERNEL); | 159 | sizeof(*rk808_clkout), GFP_KERNEL); |
| @@ -846,7 +846,7 @@ static const struct clk_parent_data top_parents[] = { · 845 unmodified lines | |||
| 846 | CCU_PARENT_HW(pll6_d3), | 846 | CCU_PARENT_HW(pll6_d3), |
| 847 | }; | 847 | }; |
| 848 | CCU_MUX_DIV_GATE_FC_DEFINE(top_dclk, top_parents, APMU_TOP_DCLK_CTRL, 5, 3, | 848 | CCU_MUX_DIV_GATE_FC_DEFINE(top_dclk, top_parents, APMU_TOP_DCLK_CTRL, 5, 3, |
| 849 | BIT(8), 2, 3, BIT(1), 0); | 849 | BIT(8), 2, 3, BIT(1), CLK_IS_CRITICAL); |
| 850 | 850 | ||
| 851 | static const struct clk_parent_data ucie_parents[] = { | 851 | static const struct clk_parent_data ucie_parents[] = { |
| 852 | CCU_PARENT_HW(pll1_d8_307p2), | 852 | CCU_PARENT_HW(pll1_d8_307p2), |
| @@ -274,6 +274,7 @@ static int waveform_ai_cmdtest(struct comedi_device *dev, · 273 unmodified lines | |||
| 274 | /* Step 2a : make sure trigger sources are unique */ | 274 | /* Step 2a : make sure trigger sources are unique */ |
| 275 | 275 | ||
| 276 | err |= comedi_check_trigger_is_unique(cmd->convert_src); | 276 | err |= comedi_check_trigger_is_unique(cmd->convert_src); |
| 277 | err |= comedi_check_trigger_is_unique(cmd->scan_begin_src); | ||
| 277 | err |= comedi_check_trigger_is_unique(cmd->stop_src); | 278 | err |= comedi_check_trigger_is_unique(cmd->stop_src); |
| 278 | 279 | ||
| 279 | /* Step 2b : and mutually compatible */ | 280 | /* Step 2b : and mutually compatible */ |
| @@ -324,10 +325,10 @@ static int waveform_ai_cmdtest(struct comedi_device *dev, · 44 unmodified lines | |||
| 324 | arg = min(arg, | 325 | arg = min(arg, |
| 325 | rounddown(UINT_MAX, (unsigned int)NSEC_PER_USEC)); | 326 | rounddown(UINT_MAX, (unsigned int)NSEC_PER_USEC)); |
| 326 | arg = NSEC_PER_USEC * DIV_ROUND_CLOSEST(arg, NSEC_PER_USEC); | 327 | arg = NSEC_PER_USEC * DIV_ROUND_CLOSEST(arg, NSEC_PER_USEC); |
| 327 | if (cmd->scan_begin_arg == TRIG_TIMER) { | 328 | if (cmd->scan_begin_src == TRIG_TIMER) { |
| 328 | /* limit convert_arg to keep scan_begin_arg in range */ | 329 | /* limit convert_arg to keep scan_begin_arg in range */ |
| 329 | limit = UINT_MAX / cmd->scan_end_arg; | 330 | limit = UINT_MAX / cmd->scan_end_arg; |
| 330 | limit = rounddown(limit, (unsigned int)NSEC_PER_SEC); | 331 | limit = rounddown(limit, (unsigned int)NSEC_PER_USEC); |
| 331 | arg = min(arg, limit); | 332 | arg = min(arg, limit); |
| 332 | } | 333 | } |
| 333 | err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg); | 334 | err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg); |
| @@ -124,7 +124,8 @@ struct counter_device *counter_alloc(size_t sizeof_priv) · 123 unmodified lines | |||
| 124 | 124 | ||
| 125 | err_dev_set_name: | 125 | err_dev_set_name: |
| 126 | 126 | ||
| 127 | counter_chrdev_remove(counter); | 127 | put_device(dev); |
| 128 | return NULL; | ||
| 128 | err_chrdev_add: | 129 | err_chrdev_add: |
| 129 | 130 | ||
| 130 | ida_free(&counter_ida, dev->id); | 131 | ida_free(&counter_ida, dev->id); |
| @@ -70,18 +70,6 @@ config X86_AMD_PSTATE_DEFAULT_MODE · 69 unmodified lines | |||
| 70 | For details, take a look at: | 70 | For details, take a look at: |
| 71 | <file:Documentation/admin-guide/pm/amd-pstate.rst>. | 71 | <file:Documentation/admin-guide/pm/amd-pstate.rst>. |
| 72 | 72 | ||
| 73 | config X86_AMD_PSTATE_DYNAMIC_EPP | ||
| 74 | bool "AMD Processor P-State dynamic EPP support" | ||
| 75 | depends on X86_AMD_PSTATE | ||
| 76 | default n | ||
| 77 | help | ||
| 78 | Allow the kernel to dynamically change the energy performance | ||
| 79 | value from events like ACPI platform profile and AC adapter plug | ||
| 80 | events. | ||
| 81 | |||
| 82 | This feature can also be changed at runtime, this configuration | ||
| 83 | option only sets the kernel default value behavior. | ||
| 84 | |||
| 85 | config X86_AMD_PSTATE_UT | 73 | config X86_AMD_PSTATE_UT |
| 86 | tristate "selftest for AMD Processor P-State driver" | 74 | tristate "selftest for AMD Processor P-State driver" |
| 87 | depends on X86 && ACPI_PROCESSOR | 75 | depends on X86 && ACPI_PROCESSOR |
| @@ -274,20 +274,21 @@ static int amd_pstate_set_mode(enum amd_pstate_mode mode) · 273 unmodified lines | |||
| 274 | 274 | ||
| 275 | static int amd_pstate_ut_epp(u32 index) | 275 | static int amd_pstate_ut_epp(u32 index) |
| 276 | { | 276 | { |
| 277 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = NULL; | ||
| 278 | char *buf __free(cleanup_page) = NULL; | ||
| 279 | static const char * const epp_strings[] = { | 277 | static const char * const epp_strings[] = { |
| 280 | "performance", | ||
| 281 | "balance_performance", | ||
| 282 | "balance_power", | ||
| 283 | "power", | 278 | "power", |
| 279 | "balance_power", | ||
| 280 | "balance_performance", | ||
| 281 | "performance", | ||
| 284 | }; | 282 | }; |
| 285 | struct amd_cpudata *cpudata; | 283 | char *buf __free(cleanup_page) = NULL; |
| 284 | struct cpufreq_policy *policy = NULL; | ||
| 286 | enum amd_pstate_mode orig_mode; | 285 | enum amd_pstate_mode orig_mode; |
| 286 | struct amd_cpudata *cpudata; | ||
| 287 | unsigned long orig_policy; | ||
| 287 | bool orig_dynamic_epp; | 288 | bool orig_dynamic_epp; |
| 288 | int ret, cpu = 0; | 289 | int ret, cpu = 0; |
| 289 | int i; | ||
| 290 | u16 epp; | 290 | u16 epp; |
| 291 | int i; | ||
| 291 | 292 | ||
| 292 | policy = cpufreq_cpu_get(cpu); | 293 | policy = cpufreq_cpu_get(cpu); |
| 293 | if (!policy) | 294 | if (!policy) |
| @@ -297,11 +298,9 @@ static int amd_pstate_ut_epp(u32 index) · 3 unmodified lines | |||
| 297 | orig_mode = amd_pstate_get_status(); | 298 | orig_mode = amd_pstate_get_status(); |
| 298 | orig_dynamic_epp = cpudata->dynamic_epp; | 299 | orig_dynamic_epp = cpudata->dynamic_epp; |
| 299 | 300 | ||
| 300 | /* disable dynamic EPP before running test */ | 301 | /* Drop reference before potential driver change. */ |
| 301 | if (cpudata->dynamic_epp) { | 302 | cpufreq_cpu_put(policy); |
| 302 | pr_debug("Dynamic EPP is enabled, disabling it\n"); | 303 | policy = NULL; |
| 303 | amd_pstate_clear_dynamic_epp(policy); | ||
| 304 | } | ||
| 305 | 304 | ||
| 306 | buf = (char *)__get_free_page(GFP_KERNEL); | 305 | buf = (char *)__get_free_page(GFP_KERNEL); |
| 307 | if (!buf) | 306 | if (!buf) |
| @@ -311,6 +310,27 @@ static int amd_pstate_ut_epp(u32 index) · 3 unmodified lines | |||
| 311 | if (ret) | 310 | if (ret) |
| 312 | goto out; | 311 | goto out; |
| 313 | 312 | ||
| 313 | policy = cpufreq_cpu_get(cpu); | ||
| 314 | if (!policy) { | ||
| 315 | ret = -ENODEV; | ||
| 316 | goto out; | ||
| 317 | } | ||
| 318 | |||
| 319 | down_write(&policy->rwsem); | ||
| 320 | cpudata = policy->driver_data; | ||
| 321 | orig_policy = cpudata->policy; | ||
| 322 | cpudata->policy = CPUFREQ_POLICY_POWERSAVE; | ||
| 323 | |||
| 324 | /* | ||
| 325 | * Disable dynamic EPP before running test. If "orig_dynamic_epp" is | ||
| 326 | * true, the driver will do a redundant switch at the end and there | ||
| 327 | * is no need for enabling it again at the end of the test. | ||
| 328 | */ | ||
| 329 | if (cpudata->dynamic_epp) { | ||
| 330 | pr_debug("Dynamic EPP is enabled, disabling it\n"); | ||
| 331 | amd_pstate_clear_dynamic_epp(policy); | ||
| 332 | } | ||
| 333 | |||
| 314 | for (epp = 0; epp <= U8_MAX; epp++) { | 334 | for (epp = 0; epp <= U8_MAX; epp++) { |
| 315 | u8 val; | 335 | u8 val; |
| 316 | 336 | ||
| @@ -358,6 +378,12 @@ static int amd_pstate_ut_epp(u32 index) · 41 unmodified lines | |||
| 358 | ret = 0; | 378 | ret = 0; |
| 359 | 379 | ||
| 360 | out: | 380 | out: |
| 381 | if (policy) { | ||
| 382 | cpudata->policy = orig_policy; | ||
| 383 | up_write(&policy->rwsem); | ||
| 384 | cpufreq_cpu_put(policy); | ||
| 385 | } | ||
| 386 | |||
| 361 | if (orig_dynamic_epp) { | 387 | if (orig_dynamic_epp) { |
| 362 | int ret2; | 388 | int ret2; |
| 363 | 389 | ||
| @@ -87,11 +87,7 @@ static struct cpufreq_driver amd_pstate_driver; · 86 unmodified lines | |||
| 87 | static struct cpufreq_driver amd_pstate_epp_driver; | 87 | static struct cpufreq_driver amd_pstate_epp_driver; |
| 88 | static int cppc_state = AMD_PSTATE_UNDEFINED; | 88 | static int cppc_state = AMD_PSTATE_UNDEFINED; |
| 89 | static bool amd_pstate_prefcore = true; | 89 | static bool amd_pstate_prefcore = true; |
| 90 | #ifdef CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP | ||
| 91 | static bool dynamic_epp = CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP; | ||
| 92 | #else | ||
| 93 | static bool dynamic_epp; | 90 | static bool dynamic_epp; |
| 94 | #endif | ||
| 95 | static struct quirk_entry *quirks; | 91 | static struct quirk_entry *quirks; |
| 96 | 92 | ||
| 97 | /* | 93 | /* |
| @@ -1291,6 +1287,8 @@ static int amd_pstate_set_dynamic_epp(struct cpufreq_policy *policy) · 1193 unmodified lines | |||
| 1291 | return ret; | 1287 | return ret; |
| 1292 | 1288 | ||
| 1293 | cpudata->profile_name = kasprintf(GFP_KERNEL, "amd-pstate-epp-cpu%d", cpudata->cpu); | 1289 | cpudata->profile_name = kasprintf(GFP_KERNEL, "amd-pstate-epp-cpu%d", cpudata->cpu); |
| 1290 | if (!cpudata->profile_name) | ||
| 1291 | return -ENOMEM; | ||
| 1294 | 1292 | ||
| 1295 | cpudata->ppdev = platform_profile_register(get_cpu_device(policy->cpu), | 1293 | cpudata->ppdev = platform_profile_register(get_cpu_device(policy->cpu), |
| 1296 | cpudata->profile_name, | 1294 | cpudata->profile_name, |
| @@ -1427,7 +1425,7 @@ ssize_t store_energy_performance_preference(struct cpufreq_policy *policy, · 130 unmodified lines | |||
| 1427 | if (ret) | 1425 | if (ret) |
| 1428 | epp = epp_values[ret]; | 1426 | epp = epp_values[ret]; |
| 1429 | else | 1427 | else |
| 1430 | epp = amd_pstate_get_balanced_epp(policy); | 1428 | epp = cpudata->epp_default_dc; |
| 1431 | } | 1429 | } |
| 1432 | 1430 | ||
| 1433 | if (cpudata->policy == CPUFREQ_POLICY_PERFORMANCE) { | 1431 | if (cpudata->policy == CPUFREQ_POLICY_PERFORMANCE) { |
| @@ -1707,6 +1705,8 @@ static int amd_pstate_change_driver_mode(int mode) · 273 unmodified lines | |||
| 1707 | { | 1705 | { |
| 1708 | int ret; | 1706 | int ret; |
| 1709 | 1707 | ||
| 1708 | lockdep_assert_held(&amd_pstate_driver_lock); | ||
| 1709 | |||
| 1710 | ret = amd_pstate_unregister_driver(0); | 1710 | ret = amd_pstate_unregister_driver(0); |
| 1711 | if (ret) | 1711 | if (ret) |
| 1712 | return ret; | 1712 | return ret; |
| @@ -1821,8 +1821,16 @@ static ssize_t dynamic_epp_store(struct device *a, struct device_attribute *b, · 108 unmodified lines | |||
| 1821 | if (ret) | 1821 | if (ret) |
| 1822 | return ret; | 1822 | return ret; |
| 1823 | 1823 | ||
| 1824 | if (dynamic_epp == enabled) | 1824 | guard(mutex)(&amd_pstate_driver_lock); |
| 1825 | |||
| 1826 | if (cppc_state != AMD_PSTATE_ACTIVE) { | ||
| 1827 | pr_debug("dynamic_epp can only be toggled in active mode\n"); | ||
| 1825 | return -EINVAL; | 1828 | return -EINVAL; |
| 1829 | } | ||
| 1830 | |||
| 1831 | /* Nothing to do */ | ||
| 1832 | if (dynamic_epp == enabled) | ||
| 1833 | return count; | ||
| 1826 | 1834 | ||
| 1827 | /* reinitialize with desired dynamic EPP value */ | 1835 | /* reinitialize with desired dynamic EPP value */ |
| 1828 | dynamic_epp = enabled; | 1836 | dynamic_epp = enabled; |
| @@ -1942,7 +1950,7 @@ static int amd_pstate_epp_cpu_init(struct cpufreq_policy *policy) · 113 unmodified lines | |||
| 1942 | if (dynamic_epp) | 1950 | if (dynamic_epp) |
| 1943 | ret = amd_pstate_set_dynamic_epp(policy); | 1951 | ret = amd_pstate_set_dynamic_epp(policy); |
| 1944 | else | 1952 | else |
| 1945 | ret = amd_pstate_set_epp(policy, amd_pstate_get_balanced_epp(policy)); | 1953 | ret = amd_pstate_set_epp(policy, cpudata->epp_default_dc); |
| 1946 | if (ret) | 1954 | if (ret) |
| 1947 | goto free_cpudata1; | 1955 | goto free_cpudata1; |
| 1948 | 1956 | ||
| @@ -1970,12 +1978,13 @@ static void amd_pstate_epp_cpu_exit(struct cpufreq_policy *policy) · 21 unmodified lines | |||
| 1970 | if (cpudata) { | 1978 | if (cpudata) { |
| 1971 | union perf_cached perf = READ_ONCE(cpudata->perf); | 1979 | union perf_cached perf = READ_ONCE(cpudata->perf); |
| 1972 | 1980 | ||
| 1981 | if (cpudata->dynamic_epp) | ||
| 1982 | amd_pstate_clear_dynamic_epp(policy); | ||
| 1983 | |||
| 1973 | /* Reset CPPC_REQ MSR to the BIOS value */ | 1984 | /* Reset CPPC_REQ MSR to the BIOS value */ |
| 1974 | amd_pstate_update_perf(policy, perf.bios_min_perf, 0U, 0U, 0U, false); | 1985 | amd_pstate_update_perf(policy, perf.bios_min_perf, 0U, 0U, 0U, false); |
| 1975 | amd_pstate_set_floor_perf(policy, cpudata->bios_floor_perf); | 1986 | amd_pstate_set_floor_perf(policy, cpudata->bios_floor_perf); |
| 1976 | 1987 | ||
| 1977 | if (cpudata->dynamic_epp) | ||
| 1978 | amd_pstate_clear_dynamic_epp(policy); | ||
| 1979 | kfree(cpudata); | 1988 | kfree(cpudata); |
| 1980 | policy->driver_data = NULL; | 1989 | policy->driver_data = NULL; |
| 1981 | } | 1990 | } |
| @@ -2279,7 +2279,7 @@ static int hwp_get_cpu_scaling(int cpu) · 2278 unmodified lines | |||
| 2279 | * Return the hybrid scaling factor for P-cores and use the | 2279 | * Return the hybrid scaling factor for P-cores and use the |
| 2280 | * default core scaling for E-cores. | 2280 | * default core scaling for E-cores. |
| 2281 | */ | 2281 | */ |
| 2282 | if (hybrid_get_cpu_type(cpu) == INTEL_CPU_TYPE_CORE) | 2282 | if (hybrid_get_cpu_type(cpu) != INTEL_CPU_TYPE_ATOM) |
| 2283 | return hybrid_scaling_factor; | 2283 | return hybrid_scaling_factor; |
| 2284 | 2284 | ||
| 2285 | return core_get_scaling(); | 2285 | return core_get_scaling(); |
| @@ -3734,6 +3734,7 @@ static const struct x86_cpu_id intel_hybrid_scaling_factor[] = { · 1448 unmodified lines | |||
| 3734 | X86_MATCH_VFM(INTEL_RAPTORLAKE, HYBRID_SCALING_FACTOR_ADL), | 3734 | X86_MATCH_VFM(INTEL_RAPTORLAKE, HYBRID_SCALING_FACTOR_ADL), |
| 3735 | X86_MATCH_VFM(INTEL_RAPTORLAKE_P, HYBRID_SCALING_FACTOR_ADL), | 3735 | X86_MATCH_VFM(INTEL_RAPTORLAKE_P, HYBRID_SCALING_FACTOR_ADL), |
| 3736 | X86_MATCH_VFM(INTEL_RAPTORLAKE_S, HYBRID_SCALING_FACTOR_ADL), | 3736 | X86_MATCH_VFM(INTEL_RAPTORLAKE_S, HYBRID_SCALING_FACTOR_ADL), |
| 3737 | X86_MATCH_VFM(INTEL_BARTLETTLAKE, HYBRID_SCALING_FACTOR_ADL), | ||
| 3737 | X86_MATCH_VFM(INTEL_METEORLAKE_L, HYBRID_SCALING_FACTOR_MTL), | 3738 | X86_MATCH_VFM(INTEL_METEORLAKE_L, HYBRID_SCALING_FACTOR_MTL), |
| 3738 | X86_MATCH_VFM(INTEL_LUNARLAKE_M, HYBRID_SCALING_FACTOR_LNL), | 3739 | X86_MATCH_VFM(INTEL_LUNARLAKE_M, HYBRID_SCALING_FACTOR_LNL), |
| 3739 | {} | 3740 | {} |
| @@ -792,9 +792,13 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags) · 791 unmodified lines | |||
| 792 | if (!dmabuf || !dmabuf->file) | 792 | if (!dmabuf || !dmabuf->file) |
| 793 | return -EINVAL; | 793 | return -EINVAL; |
| 794 | 794 | ||
| 795 | fd = FD_ADD(flags, dmabuf->file); | 795 | fd = get_unused_fd_flags(flags); |
| 796 | if (fd < 0) | ||
| 797 | return fd; | ||
| 798 | |||
| 796 | DMA_BUF_TRACE(trace_dma_buf_fd, dmabuf, fd); | 799 | DMA_BUF_TRACE(trace_dma_buf_fd, dmabuf, fd); |
| 797 | 800 | ||
| 801 | fd_install(fd, dmabuf->file); | ||
| 798 | return fd; | 802 | return fd; |
| 799 | } | 803 | } |
| 800 | EXPORT_SYMBOL_NS_GPL(dma_buf_fd, "DMA_BUF"); | 804 | EXPORT_SYMBOL_NS_GPL(dma_buf_fd, "DMA_BUF"); |
| @@ -829,12 +829,21 @@ int dpll_device_delete_ntf(struct dpll_device *dpll) · 828 unmodified lines | |||
| 829 | return dpll_device_event_send(DPLL_CMD_DEVICE_DELETE_NTF, dpll); | 829 | return dpll_device_event_send(DPLL_CMD_DEVICE_DELETE_NTF, dpll); |
| 830 | } | 830 | } |
| 831 | 831 | ||
| 832 | static int | 832 | /** |
| 833 | __dpll_device_change_ntf(struct dpll_device *dpll) | 833 | * __dpll_device_change_ntf - notify that the dpll device has been changed |
| 834 | * @dpll: registered dpll pointer | ||
| 835 | * | ||
| 836 | * Context: caller must hold dpll_lock. Suitable for use inside device | ||
| 837 | * callbacks which are already invoked under dpll_lock. | ||
| 838 | * Return: 0 if succeeds, error code otherwise. | ||
| 839 | */ | ||
| 840 | int __dpll_device_change_ntf(struct dpll_device *dpll) | ||
| 834 | { | 841 | { |
| 842 | lockdep_assert_held(&dpll_lock); | ||
| 835 | dpll_device_notify(dpll, DPLL_DEVICE_CHANGED); | 843 | dpll_device_notify(dpll, DPLL_DEVICE_CHANGED); |
| 836 | return dpll_device_event_send(DPLL_CMD_DEVICE_CHANGE_NTF, dpll); | 844 | return dpll_device_event_send(DPLL_CMD_DEVICE_CHANGE_NTF, dpll); |
| 837 | } | 845 | } |
| 846 | EXPORT_SYMBOL_GPL(__dpll_device_change_ntf); | ||
| 838 | 847 | ||
| 839 | /** | 848 | /** |
| 840 | * dpll_device_change_ntf - notify that the dpll device has been changed | 849 | * dpll_device_change_ntf - notify that the dpll device has been changed |
| @@ -900,11 +909,21 @@ int dpll_pin_delete_ntf(struct dpll_pin *pin) · 59 unmodified lines | |||
| 900 | return dpll_pin_event_send(DPLL_CMD_PIN_DELETE_NTF, pin); | 909 | return dpll_pin_event_send(DPLL_CMD_PIN_DELETE_NTF, pin); |
| 901 | } | 910 | } |
| 902 | 911 | ||
| 912 | /** | ||
| 913 | * __dpll_pin_change_ntf - notify that the pin has been changed | ||
| 914 | * @pin: registered pin pointer | ||
| 915 | * | ||
| 916 | * Context: caller must hold dpll_lock. Suitable for use inside pin | ||
| 917 | * callbacks which are already invoked under dpll_lock. | ||
| 918 | * Return: 0 if succeeds, error code otherwise. | ||
| 919 | */ | ||
| 903 | int __dpll_pin_change_ntf(struct dpll_pin *pin) | 920 | int __dpll_pin_change_ntf(struct dpll_pin *pin) |
| 904 | { | 921 | { |
| 922 | lockdep_assert_held(&dpll_lock); | ||
| 905 | dpll_pin_notify(pin, DPLL_PIN_CHANGED); | 923 | dpll_pin_notify(pin, DPLL_PIN_CHANGED); |
| 906 | return dpll_pin_event_send(DPLL_CMD_PIN_CHANGE_NTF, pin); | 924 | return dpll_pin_event_send(DPLL_CMD_PIN_CHANGE_NTF, pin); |
| 907 | } | 925 | } |
| 926 | EXPORT_SYMBOL_GPL(__dpll_pin_change_ntf); | ||
| 908 | 927 | ||
| 909 | /** | 928 | /** |
| 910 | * dpll_pin_change_ntf - notify that the pin has been changed | 929 | * dpll_pin_change_ntf - notify that the pin has been changed |
| @@ -11,5 +11,3 @@ int dpll_device_delete_ntf(struct dpll_device *dpll); · 10 unmodified lines | |||
| 11 | int dpll_pin_create_ntf(struct dpll_pin *pin); | 11 | int dpll_pin_create_ntf(struct dpll_pin *pin); |
| 12 | 12 | ||
| 13 | int dpll_pin_delete_ntf(struct dpll_pin *pin); | 13 | int dpll_pin_delete_ntf(struct dpll_pin *pin); |
| 14 | |||
| 15 | int __dpll_pin_change_ntf(struct dpll_pin *pin); | ||
| @@ -762,18 +762,15 @@ zl3073x_dev_periodic_work(struct kthread_work *work) · 761 unmodified lines | |||
| 762 | dev_warn(zldev->dev, "Failed to update phase offsets: %pe\n", | 762 | dev_warn(zldev->dev, "Failed to update phase offsets: %pe\n", |
| 763 | ERR_PTR(rc)); | 763 | ERR_PTR(rc)); |
| 764 | 764 | ||
| 765 | /* Update measured input reference frequencies if any DPLL has | 765 | /* Update measured input reference frequencies if frequency |
| 766 | * frequency monitoring enabled. | 766 | * monitoring is enabled. |
| 767 | */ | 767 | */ |
| 768 | list_for_each_entry(zldpll, &zldev->dplls, list) { | 768 | if (zldev->freq_monitor) { |
| 769 | if (zldpll->freq_monitor) { | 769 | rc = zl3073x_ref_freq_meas_update(zldev); |
| 770 | rc = zl3073x_ref_freq_meas_update(zldev); | 770 | if (rc) |
| 771 | if (rc) | 771 | dev_warn(zldev->dev, |
| 772 | dev_warn(zldev->dev, | 772 | "Failed to update measured frequencies: %pe\n", |
| 773 | "Failed to update measured frequencies: %pe\n", | 773 | ERR_PTR(rc)); |
| 774 | ERR_PTR(rc)); | ||
| 775 | break; | ||
| 776 | } | ||
| 777 | } | 774 | } |
| 778 | 775 | ||
| 779 | /* Update references' fractional frequency offsets */ | 776 | /* Update references' fractional frequency offsets */ |
| @@ -57,6 +57,7 @@ struct zl3073x_chip_info { · 56 unmodified lines | |||
| 57 | * @work: periodic work | 57 | * @work: periodic work |
| 58 | * @clock_id: clock id of the device | 58 | * @clock_id: clock id of the device |
| 59 | * @phase_avg_factor: phase offset measurement averaging factor | 59 | * @phase_avg_factor: phase offset measurement averaging factor |
| 60 | * @freq_monitor: is frequency monitor enabled | ||
| 60 | */ | 61 | */ |
| 61 | struct zl3073x_dev { | 62 | struct zl3073x_dev { |
| 62 | struct device *dev; | 63 | struct device *dev; |
| @@ -77,9 +78,10 @@ struct zl3073x_dev { · 14 unmodified lines | |||
| 77 | struct kthread_worker *kworker; | 78 | struct kthread_worker *kworker; |
| 78 | struct kthread_delayed_work work; | 79 | struct kthread_delayed_work work; |
| 79 | 80 | ||
| 80 | /* Devlink parameters */ | 81 | /* Per-chip parameters */ |
| 81 | u64 clock_id; | 82 | u64 clock_id; |
| 82 | u8 phase_avg_factor; | 83 | u8 phase_avg_factor; |
| 84 | bool freq_monitor; | ||
| 83 | }; | 85 | }; |
| 84 | 86 | ||
| 85 | extern const struct regmap_config zl3073x_regmap_config; | 87 | extern const struct regmap_config zl3073x_regmap_config; |
| @@ -1079,15 +1079,6 @@ zl3073x_dpll_phase_offset_avg_factor_get(const struct dpll_device *dpll, · 1078 unmodified lines | |||
| 1079 | return 0; | 1079 | return 0; |
| 1080 | } | 1080 | } |
| 1081 | 1081 | ||
| 1082 | static void | ||
| 1083 | zl3073x_dpll_change_work(struct work_struct *work) | ||
| 1084 | { | ||
| 1085 | struct zl3073x_dpll *zldpll; | ||
| 1086 | |||
| 1087 | zldpll = container_of(work, struct zl3073x_dpll, change_work); | ||
| 1088 | dpll_device_change_ntf(zldpll->dpll_dev); | ||
| 1089 | } | ||
| 1090 | |||
| 1091 | static int | 1082 | static int |
| 1092 | zl3073x_dpll_phase_offset_avg_factor_set(const struct dpll_device *dpll, | 1083 | zl3073x_dpll_phase_offset_avg_factor_set(const struct dpll_device *dpll, |
| 1093 | void *dpll_priv, u32 factor, | 1084 | void *dpll_priv, u32 factor, |
| @@ -1113,8 +1104,10 @@ zl3073x_dpll_phase_offset_avg_factor_set(const struct dpll_device *dpll, · 19 unmodified lines | |||
| 1113 | * we have to send a notification for other DPLL devices. | 1104 | * we have to send a notification for other DPLL devices. |
| 1114 | */ | 1105 | */ |
| 1115 | list_for_each_entry(item, &zldpll->dev->dplls, list) { | 1106 | list_for_each_entry(item, &zldpll->dev->dplls, list) { |
| 1116 | if (item != zldpll) | 1107 | struct dpll_device *dpll_dev = READ_ONCE(item->dpll_dev); |
| 1117 | schedule_work(&item->change_work); | 1108 | |
| 1109 | if (item != zldpll && dpll_dev) | ||
| 1110 | __dpll_device_change_ntf(dpll_dev); | ||
| 1118 | } | 1111 | } |
| 1119 | 1112 | ||
| 1120 | return 0; | 1113 | return 0; |
| @@ -1219,7 +1212,7 @@ zl3073x_dpll_freq_monitor_get(const struct dpll_device *dpll, · 98 unmodified lines | |||
| 1219 | { | 1212 | { |
| 1220 | struct zl3073x_dpll *zldpll = dpll_priv; | 1213 | struct zl3073x_dpll *zldpll = dpll_priv; |
| 1221 | 1214 | ||
| 1222 | if (zldpll->freq_monitor) | 1215 | if (zldpll->dev->freq_monitor) |
| 1223 | *state = DPLL_FEATURE_STATE_ENABLE; | 1216 | *state = DPLL_FEATURE_STATE_ENABLE; |
| 1224 | else | 1217 | else |
| 1225 | *state = DPLL_FEATURE_STATE_DISABLE; | 1218 | *state = DPLL_FEATURE_STATE_DISABLE; |
| @@ -1233,9 +1226,19 @@ zl3073x_dpll_freq_monitor_set(const struct dpll_device *dpll, · 7 unmodified lines | |||
| 1233 | enum dpll_feature_state state, | 1226 | enum dpll_feature_state state, |
| 1234 | struct netlink_ext_ack *extack) | 1227 | struct netlink_ext_ack *extack) |
| 1235 | { | 1228 | { |
| 1236 | struct zl3073x_dpll *zldpll = dpll_priv; | 1229 | struct zl3073x_dpll *item, *zldpll = dpll_priv; |
| 1230 | |||
| 1231 | zldpll->dev->freq_monitor = (state == DPLL_FEATURE_STATE_ENABLE); | ||
| 1232 | |||
| 1233 | /* The frequency monitoring is common for all DPLL channels so after | ||
| 1234 | * change we have to send a notification for other DPLL devices. | ||
| 1235 | */ | ||
| 1236 | list_for_each_entry(item, &zldpll->dev->dplls, list) { | ||
| 1237 | struct dpll_device *dpll_dev = READ_ONCE(item->dpll_dev); | ||
| 1237 | 1238 | ||
| 1238 | zldpll->freq_monitor = (state == DPLL_FEATURE_STATE_ENABLE); | 1239 | if (item != zldpll && dpll_dev) |
| 1240 | __dpll_device_change_ntf(dpll_dev); | ||
| 1241 | } | ||
| 1239 | 1242 | ||
| 1240 | return 0; | 1243 | return 0; |
| 1241 | } | 1244 | } |
| @@ -1394,8 +1397,8 @@ zl3073x_dpll_pin_register(struct zl3073x_dpll_pin *pin, u32 index) · 152 unmodified lines | |||
| 1394 | 1397 | ||
| 1395 | err_register: | 1398 | err_register: |
| 1396 | dpll_pin_put(pin->dpll_pin, &pin->tracker); | 1399 | dpll_pin_put(pin->dpll_pin, &pin->tracker); |
| 1397 | pin->dpll_pin = NULL; | ||
| 1398 | err_pin_get: | 1400 | err_pin_get: |
| 1401 | pin->dpll_pin = NULL; | ||
| 1399 | fwnode_handle_put(pin->fwnode); | 1402 | fwnode_handle_put(pin->fwnode); |
| 1400 | pin->fwnode = NULL; | 1403 | pin->fwnode = NULL; |
| 1401 | zl3073x_pin_props_put(props); | 1404 | zl3073x_pin_props_put(props); |
| @@ -1563,8 +1566,10 @@ zl3073x_dpll_pins_register(struct zl3073x_dpll *zldpll) · 161 unmodified lines | |||
| 1563 | } | 1566 | } |
| 1564 | 1567 | ||
| 1565 | rc = zl3073x_dpll_pin_register(pin, index); | 1568 | rc = zl3073x_dpll_pin_register(pin, index); |
| 1566 | if (rc) | 1569 | if (rc) { |
| 1570 | zl3073x_dpll_pin_free(pin); | ||
| 1567 | goto error; | 1571 | goto error; |
| 1572 | } | ||
| 1568 | 1573 | ||
| 1569 | list_add(&pin->list, &zldpll->pins); | 1574 | list_add(&pin->list, &zldpll->pins); |
| 1570 | } | 1575 | } |
| @@ -1625,13 +1630,13 @@ zl3073x_dpll_device_register(struct zl3073x_dpll *zldpll) · 54 unmodified lines | |||
| 1625 | static void | 1630 | static void |
| 1626 | zl3073x_dpll_device_unregister(struct zl3073x_dpll *zldpll) | 1631 | zl3073x_dpll_device_unregister(struct zl3073x_dpll *zldpll) |
| 1627 | { | 1632 | { |
| 1628 | WARN(!zldpll->dpll_dev, "DPLL device is not registered\n"); | 1633 | struct dpll_device *dpll_dev = READ_ONCE(zldpll->dpll_dev); |
| 1629 | 1634 | ||
| 1630 | cancel_work_sync(&zldpll->change_work); | 1635 | WARN(!dpll_dev, "DPLL device is not registered\n"); |
| 1631 | 1636 | ||
| 1632 | dpll_device_unregister(zldpll->dpll_dev, &zldpll->ops, zldpll); | 1637 | WRITE_ONCE(zldpll->dpll_dev, NULL); |
| 1633 | dpll_device_put(zldpll->dpll_dev, &zldpll->tracker); | 1638 | dpll_device_unregister(dpll_dev, &zldpll->ops, zldpll); |
| 1634 | zldpll->dpll_dev = NULL; | 1639 | dpll_device_put(dpll_dev, &zldpll->tracker); |
| 1635 | } | 1640 | } |
| 1636 | 1641 | ||
| 1637 | /** | 1642 | /** |
| @@ -1750,7 +1755,7 @@ zl3073x_dpll_pin_measured_freq_check(struct zl3073x_dpll_pin *pin) · 112 unmodified lines | |||
| 1750 | u8 ref_id; | 1755 | u8 ref_id; |
| 1751 | u32 freq; | 1756 | u32 freq; |
| 1752 | 1757 | ||
| 1753 | if (!zldpll->freq_monitor) | 1758 | if (!zldpll->dev->freq_monitor) |
| 1754 | return false; | 1759 | return false; |
| 1755 | 1760 | ||
| 1756 | ref_id = zl3073x_input_pin_ref_get(pin->id); | 1761 | ref_id = zl3073x_input_pin_ref_get(pin->id); |
| @@ -1783,10 +1788,8 @@ zl3073x_dpll_changes_check(struct zl3073x_dpll *zldpll) · 26 unmodified lines | |||
| 1783 | struct zl3073x_dev *zldev = zldpll->dev; | 1788 | struct zl3073x_dev *zldev = zldpll->dev; |
| 1784 | enum dpll_lock_status lock_status; | 1789 | enum dpll_lock_status lock_status; |
| 1785 | struct device *dev = zldev->dev; | 1790 | struct device *dev = zldev->dev; |
| 1786 | const struct zl3073x_chan *chan; | ||
| 1787 | struct zl3073x_dpll_pin *pin; | 1791 | struct zl3073x_dpll_pin *pin; |
| 1788 | int rc; | 1792 | int rc; |
| 1789 | u8 mode; | ||
| 1790 | 1793 | ||
| 1791 | zldpll->check_count++; | 1794 | zldpll->check_count++; |
| 1792 | 1795 | ||
| @@ -1805,15 +1808,6 @@ zl3073x_dpll_changes_check(struct zl3073x_dpll *zldpll) · 12 unmodified lines | |||
| 1805 | dpll_device_change_ntf(zldpll->dpll_dev); | 1808 | dpll_device_change_ntf(zldpll->dpll_dev); |
| 1806 | } | 1809 | } |
| 1807 | 1810 | ||
| 1808 | /* Input pin monitoring does make sense only in automatic | ||
| 1809 | * or forced reference modes. | ||
| 1810 | */ | ||
| 1811 | chan = zl3073x_chan_state_get(zldev, zldpll->id); | ||
| 1812 | mode = zl3073x_chan_mode_get(chan); | ||
| 1813 | if (mode != ZL_DPLL_MODE_REFSEL_MODE_AUTO && | ||
| 1814 | mode != ZL_DPLL_MODE_REFSEL_MODE_REFLOCK) | ||
| 1815 | return; | ||
| 1816 | |||
| 1817 | /* Update phase offset latch registers for this DPLL if the phase | 1811 | /* Update phase offset latch registers for this DPLL if the phase |
| 1818 | * offset monitor feature is enabled. | 1812 | * offset monitor feature is enabled. |
| 1819 | */ | 1813 | */ |
| @@ -1924,7 +1918,6 @@ zl3073x_dpll_alloc(struct zl3073x_dev *zldev, u8 ch) · 104 unmodified lines | |||
| 1924 | zldpll->dev = zldev; | 1918 | zldpll->dev = zldev; |
| 1925 | zldpll->id = ch; | 1919 | zldpll->id = ch; |
| 1926 | INIT_LIST_HEAD(&zldpll->pins); | 1920 | INIT_LIST_HEAD(&zldpll->pins); |
| 1927 | INIT_WORK(&zldpll->change_work, zl3073x_dpll_change_work); | ||
| 1928 | 1921 | ||
| 1929 | return zldpll; | 1922 | return zldpll; |
| 1930 | } | 1923 | } |
| @@ -15,13 +15,11 @@ · 14 unmodified lines | |||
| 15 | * @id: DPLL index | 15 | * @id: DPLL index |
| 16 | * @check_count: periodic check counter | 16 | * @check_count: periodic check counter |
| 17 | * @phase_monitor: is phase offset monitor enabled | 17 | * @phase_monitor: is phase offset monitor enabled |
| 18 | * @freq_monitor: is frequency monitor enabled | ||
| 19 | * @ops: DPLL device operations for this instance | 18 | * @ops: DPLL device operations for this instance |
| 20 | * @dpll_dev: pointer to registered DPLL device | 19 | * @dpll_dev: pointer to registered DPLL device |
| 21 | * @tracker: tracking object for the acquired reference | 20 | * @tracker: tracking object for the acquired reference |
| 22 | * @lock_status: last saved DPLL lock status | 21 | * @lock_status: last saved DPLL lock status |
| 23 | * @pins: list of pins | 22 | * @pins: list of pins |
| 24 | * @change_work: device change notification work | ||
| 25 | */ | 23 | */ |
| 26 | struct zl3073x_dpll { | 24 | struct zl3073x_dpll { |
| 27 | struct list_head list; | 25 | struct list_head list; |
| @@ -29,13 +27,11 @@ struct zl3073x_dpll { · 1 unmodified line | |||
| 29 | u8 id; | 27 | u8 id; |
| 30 | u8 check_count; | 28 | u8 check_count; |
| 31 | bool phase_monitor; | 29 | bool phase_monitor; |
| 32 | bool freq_monitor; | ||
| 33 | struct dpll_device_ops ops; | 30 | struct dpll_device_ops ops; |
| 34 | struct dpll_device *dpll_dev; | 31 | struct dpll_device *dpll_dev; |
| 35 | dpll_tracker tracker; | 32 | dpll_tracker tracker; |
| 36 | enum dpll_lock_status lock_status; | 33 | enum dpll_lock_status lock_status; |
| 37 | struct list_head pins; | 34 | struct list_head pins; |
| 38 | struct work_struct change_work; | ||
| 39 | }; | 35 | }; |
| 40 | 36 | ||
| 41 | struct zl3073x_dpll *zl3073x_dpll_alloc(struct zl3073x_dev *zldev, u8 ch); | 37 | struct zl3073x_dpll *zl3073x_dpll_alloc(struct zl3073x_dev *zldev, u8 ch); |
| @@ -777,9 +777,9 @@ static int init_one_mc(struct mc_priv *priv, struct platform_device *pdev, int i · 776 unmodified lines | |||
| 777 | u32 num_chans, rank, dwidth, config; | 777 | u32 num_chans, rank, dwidth, config; |
| 778 | struct edac_mc_layer layers[2]; | 778 | struct edac_mc_layer layers[2]; |
| 779 | struct mem_ctl_info *mci; | 779 | struct mem_ctl_info *mci; |
| 780 | char name[MC_NAME_LEN]; | ||
| 780 | struct device *dev; | 781 | struct device *dev; |
| 781 | enum dev_type dt; | 782 | enum dev_type dt; |
| 782 | char *name; | ||
| 783 | int rc; | 783 | int rc; |
| 784 | 784 | ||
| 785 | config = priv->adec[CONF + i * ADEC_NUM]; | 785 | config = priv->adec[CONF + i * ADEC_NUM]; |
| @@ -813,13 +813,9 @@ static int init_one_mc(struct mc_priv *priv, struct platform_device *pdev, int i · 27 unmodified lines | |||
| 813 | layers[1].is_virt_csrow = false; | 813 | layers[1].is_virt_csrow = false; |
| 814 | 814 | ||
| 815 | rc = -ENOMEM; | 815 | rc = -ENOMEM; |
| 816 | name = kzalloc(MC_NAME_LEN, GFP_KERNEL); | ||
| 817 | if (!name) | ||
| 818 | return rc; | ||
| 819 | |||
| 820 | dev = kzalloc(sizeof(*dev), GFP_KERNEL); | 816 | dev = kzalloc(sizeof(*dev), GFP_KERNEL); |
| 821 | if (!dev) | 817 | if (!dev) |
| 822 | goto err_name_free; | 818 | return rc; |
| 823 | 819 | ||
| 824 | mci = edac_mc_alloc(i, ARRAY_SIZE(layers), layers, sizeof(struct mc_priv)); | 820 | mci = edac_mc_alloc(i, ARRAY_SIZE(layers), layers, sizeof(struct mc_priv)); |
| 825 | if (!mci) { | 821 | if (!mci) { |
| @@ -858,8 +854,6 @@ err_mc_free: · 32 unmodified lines | |||
| 858 | edac_mc_free(mci); | 854 | edac_mc_free(mci); |
| 859 | err_dev_free: | 855 | err_dev_free: |
| 860 | kfree(dev); | 856 | kfree(dev); |
| 861 | err_name_free: | ||
| 862 | kfree(name); | ||
| 863 | 857 | ||
| 864 | return rc; | 858 | return rc; |
| 865 | } | 859 | } |
| @@ -26,6 +26,8 @@ static int ffa_device_match(struct device *dev, const struct device_driver *drv) · 25 unmodified lines | |||
| 26 | 26 | ||
| 27 | id_table = to_ffa_driver(drv)->id_table; | 27 | id_table = to_ffa_driver(drv)->id_table; |
| 28 | ffa_dev = to_ffa_dev(dev); | 28 | ffa_dev = to_ffa_dev(dev); |
| 29 | if (!id_table) | ||
| 30 | return 0; | ||
| 29 | 31 | ||
| 30 | while (!uuid_is_null(&id_table->uuid)) { | 32 | while (!uuid_is_null(&id_table->uuid)) { |
| 31 | /* | 33 | /* |
| @@ -123,7 +125,7 @@ int ffa_driver_register(struct ffa_driver *driver, struct module *owner, · 91 unmodified lines | |||
| 123 | { | 125 | { |
| 124 | int ret; | 126 | int ret; |
| 125 | 127 | ||
| 126 | if (!driver->probe) | 128 | if (!driver->probe || !driver->id_table) |
| 127 | return -EINVAL; | 129 | return -EINVAL; |
| 128 | 130 | ||
| 129 | driver->driver.bus = &ffa_bus_type; | 131 | driver->driver.bus = &ffa_bus_type; |
| @@ -87,6 +87,7 @@ static inline int ffa_to_linux_errno(int errno) · 86 unmodified lines | |||
| 87 | 87 | ||
| 88 | struct ffa_pcpu_irq { | 88 | struct ffa_pcpu_irq { |
| 89 | struct ffa_drv_info *info; | 89 | struct ffa_drv_info *info; |
| 90 | struct work_struct notif_pcpu_work; | ||
| 90 | }; | 91 | }; |
| 91 | 92 | ||
| 92 | struct ffa_drv_info { | 93 | struct ffa_drv_info { |
| @@ -100,13 +101,13 @@ struct ffa_drv_info { · 7 unmodified lines | |||
| 100 | bool mem_ops_native; | 101 | bool mem_ops_native; |
| 101 | bool msg_direct_req2_supp; | 102 | bool msg_direct_req2_supp; |
| 102 | bool bitmap_created; | 103 | bool bitmap_created; |
| 104 | bool bus_notifier_registered; | ||
| 103 | bool notif_enabled; | 105 | bool notif_enabled; |
| 104 | unsigned int sched_recv_irq; | 106 | unsigned int sched_recv_irq; |
| 105 | unsigned int notif_pend_irq; | 107 | unsigned int notif_pend_irq; |
| 106 | unsigned int cpuhp_state; | 108 | unsigned int cpuhp_state; |
| 107 | struct ffa_pcpu_irq __percpu *irq_pcpu; | 109 | struct ffa_pcpu_irq __percpu *irq_pcpu; |
| 108 | struct workqueue_struct *notif_pcpu_wq; | 110 | struct workqueue_struct *notif_pcpu_wq; |
| 109 | struct work_struct notif_pcpu_work; | ||
| 110 | struct work_struct sched_recv_irq_work; | 111 | struct work_struct sched_recv_irq_work; |
| 111 | struct xarray partition_info; | 112 | struct xarray partition_info; |
| 112 | DECLARE_HASHTABLE(notifier_hash, ilog2(FFA_MAX_NOTIFICATIONS)); | 113 | DECLARE_HASHTABLE(notifier_hash, ilog2(FFA_MAX_NOTIFICATIONS)); |
| @@ -322,6 +323,12 @@ __ffa_partition_info_get(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 209 unmodified lines | |||
| 322 | #define PART_INFO_ID_MASK GENMASK(15, 0) | 323 | #define PART_INFO_ID_MASK GENMASK(15, 0) |
| 323 | #define PART_INFO_EXEC_CXT_MASK GENMASK(31, 16) | 324 | #define PART_INFO_EXEC_CXT_MASK GENMASK(31, 16) |
| 324 | #define PART_INFO_PROPS_MASK GENMASK(63, 32) | 325 | #define PART_INFO_PROPS_MASK GENMASK(63, 32) |
| 326 | #define FFA_PART_INFO_GET_REGS_FIRST_REG 3 | ||
| 327 | #define FFA_PART_INFO_GET_REGS_REGS_PER_DESC 3 | ||
| 328 | #define FFA_PART_INFO_GET_REGS_MAX_DESC \ | ||
| 329 | (((sizeof(ffa_value_t) / sizeof_field(ffa_value_t, a0)) - \ | ||
| 330 | FFA_PART_INFO_GET_REGS_FIRST_REG) / \ | ||
| 331 | FFA_PART_INFO_GET_REGS_REGS_PER_DESC) | ||
| 325 | #define PART_INFO_ID(x) ((u16)(FIELD_GET(PART_INFO_ID_MASK, (x)))) | 332 | #define PART_INFO_ID(x) ((u16)(FIELD_GET(PART_INFO_ID_MASK, (x)))) |
| 326 | #define PART_INFO_EXEC_CXT(x) ((u16)(FIELD_GET(PART_INFO_EXEC_CXT_MASK, (x)))) | 333 | #define PART_INFO_EXEC_CXT(x) ((u16)(FIELD_GET(PART_INFO_EXEC_CXT_MASK, (x)))) |
| 327 | #define PART_INFO_PROPERTIES(x) ((u32)(FIELD_GET(PART_INFO_PROPS_MASK, (x)))) | 334 | #define PART_INFO_PROPERTIES(x) ((u32)(FIELD_GET(PART_INFO_PROPS_MASK, (x)))) |
| @@ -329,15 +336,13 @@ static int · 1 unmodified line | |||
| 329 | __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, | 336 | __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, |
| 330 | struct ffa_partition_info *buffer, int num_parts) | 337 | struct ffa_partition_info *buffer, int num_parts) |
| 331 | { | 338 | { |
| 332 | u16 buf_sz, start_idx, cur_idx, count = 0, prev_idx = 0, tag = 0; | 339 | u16 buf_sz, start_idx = 0, cur_idx, count = 0, tag = 0; |
| 333 | struct ffa_partition_info *buf = buffer; | 340 | struct ffa_partition_info *buf = buffer; |
| 334 | ffa_value_t partition_info; | 341 | ffa_value_t partition_info; |
| 335 | 342 | ||
| 336 | do { | 343 | do { |
| 337 | __le64 *regs; | 344 | __le64 *regs; |
| 338 | int idx; | 345 | int idx, nr_desc, buf_idx; |
| 339 | |||
| 340 | start_idx = prev_idx ? prev_idx + 1 : 0; | ||
| 341 | 346 | ||
| 342 | invoke_ffa_fn((ffa_value_t){ | 347 | invoke_ffa_fn((ffa_value_t){ |
| 343 | .a0 = FFA_PARTITION_INFO_GET_REGS, | 348 | .a0 = FFA_PARTITION_INFO_GET_REGS, |
| @@ -353,15 +358,28 @@ __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 9 unmodified lines | |||
| 353 | count = PARTITION_COUNT(partition_info.a2); | 358 | count = PARTITION_COUNT(partition_info.a2); |
| 354 | if (!buffer || !num_parts) /* count only */ | 359 | if (!buffer || !num_parts) /* count only */ |
| 355 | return count; | 360 | return count; |
| 361 | if (count > num_parts) | ||
| 362 | return -EINVAL; | ||
| 356 | 363 | ||
| 357 | cur_idx = CURRENT_INDEX(partition_info.a2); | 364 | cur_idx = CURRENT_INDEX(partition_info.a2); |
| 365 | if (cur_idx < start_idx || cur_idx >= count) | ||
| 366 | return -EINVAL; | ||
| 367 | |||
| 368 | nr_desc = cur_idx - start_idx + 1; | ||
| 369 | if (nr_desc > FFA_PART_INFO_GET_REGS_MAX_DESC) | ||
| 370 | return -EINVAL; | ||
| 371 | |||
| 372 | buf_idx = buf - buffer; | ||
| 373 | if (buf_idx + nr_desc > num_parts) | ||
| 374 | return -EINVAL; | ||
| 375 | |||
| 358 | tag = UUID_INFO_TAG(partition_info.a2); | 376 | tag = UUID_INFO_TAG(partition_info.a2); |
| 359 | buf_sz = PARTITION_INFO_SZ(partition_info.a2); | 377 | buf_sz = PARTITION_INFO_SZ(partition_info.a2); |
| 360 | if (buf_sz > sizeof(*buffer)) | 378 | if (buf_sz > sizeof(*buffer)) |
| 361 | buf_sz = sizeof(*buffer); | 379 | buf_sz = sizeof(*buffer); |
| 362 | 380 | ||
| 363 | regs = (void *)&partition_info.a3; | 381 | regs = (void *)&partition_info.a3; |
| 364 | for (idx = 0; idx < cur_idx - start_idx + 1; idx++, buf++) { | 382 | for (idx = 0; idx < nr_desc; idx++, buf++) { |
| 365 | union { | 383 | union { |
| 366 | uuid_t uuid; | 384 | uuid_t uuid; |
| 367 | u64 regs[2]; | 385 | u64 regs[2]; |
| @@ -379,7 +397,7 @@ __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 11 unmodified lines | |||
| 379 | uuid_copy(&buf->uuid, &uuid_regs.uuid); | 397 | uuid_copy(&buf->uuid, &uuid_regs.uuid); |
| 380 | regs += 3; | 398 | regs += 3; |
| 381 | } | 399 | } |
| 382 | prev_idx = cur_idx; | 400 | start_idx = cur_idx + 1; |
| 383 | 401 | ||
| 384 | } while (cur_idx < (count - 1)); | 402 | } while (cur_idx < (count - 1)); |
| 385 | 403 | ||
| @@ -1189,7 +1207,7 @@ static int · 803 unmodified lines | |||
| 1189 | ffa_sched_recv_cb_update(struct ffa_device *dev, ffa_sched_recv_cb callback, | 1207 | ffa_sched_recv_cb_update(struct ffa_device *dev, ffa_sched_recv_cb callback, |
| 1190 | void *cb_data, bool is_registration) | 1208 | void *cb_data, bool is_registration) |
| 1191 | { | 1209 | { |
| 1192 | struct ffa_dev_part_info *partition = NULL, *tmp; | 1210 | struct ffa_dev_part_info *partition = NULL; |
| 1193 | struct list_head *phead; | 1211 | struct list_head *phead; |
| 1194 | bool cb_valid; | 1212 | bool cb_valid; |
| 1195 | 1213 | ||
| @@ -1202,11 +1220,11 @@ ffa_sched_recv_cb_update(struct ffa_device *dev, ffa_sched_recv_cb callback, · 6 unmodified lines | |||
| 1202 | return -EINVAL; | 1220 | return -EINVAL; |
| 1203 | } | 1221 | } |
| 1204 | 1222 | ||
| 1205 | list_for_each_entry_safe(partition, tmp, phead, node) | 1223 | list_for_each_entry(partition, phead, node) |
| 1206 | if (partition->dev == dev) | 1224 | if (partition->dev == dev) |
| 1207 | break; | 1225 | break; |
| 1208 | 1226 | ||
| 1209 | if (!partition) { | 1227 | if (&partition->node == phead) { |
| 1210 | pr_err("%s: No such partition ID 0x%x\n", __func__, dev->vm_id); | 1228 | pr_err("%s: No such partition ID 0x%x\n", __func__, dev->vm_id); |
| 1211 | return -EINVAL; | 1229 | return -EINVAL; |
| 1212 | } | 1230 | } |
| @@ -1445,20 +1463,25 @@ static int ffa_notify_send(struct ffa_device *dev, int notify_id, · 232 unmodified lines | |||
| 1445 | 1463 | ||
| 1446 | static void handle_notif_callbacks(u64 bitmap, enum notify_type type) | 1464 | static void handle_notif_callbacks(u64 bitmap, enum notify_type type) |
| 1447 | { | 1465 | { |
| 1466 | ffa_notifier_cb cb; | ||
| 1467 | void *cb_data; | ||
| 1448 | int notify_id; | 1468 | int notify_id; |
| 1449 | struct notifier_cb_info *cb_info = NULL; | ||
| 1450 | 1469 | ||
| 1451 | for (notify_id = 0; notify_id <= FFA_MAX_NOTIFICATIONS && bitmap; | 1470 | for (notify_id = 0; notify_id <= FFA_MAX_NOTIFICATIONS && bitmap; |
| 1452 | notify_id++, bitmap >>= 1) { | 1471 | notify_id++, bitmap >>= 1) { |
| 1453 | if (!(bitmap & 1)) | 1472 | if (!(bitmap & 1)) |
| 1454 | continue; | 1473 | continue; |
| 1455 | 1474 | ||
| 1456 | read_lock(&drv_info->notify_lock); | 1475 | scoped_guard(read_lock, &drv_info->notify_lock) { |
| 1457 | cb_info = notifier_hnode_get_by_type(notify_id, type); | 1476 | struct notifier_cb_info *cb_info; |
| 1458 | read_unlock(&drv_info->notify_lock); | ||
| 1459 | 1477 | ||
| 1460 | if (cb_info && cb_info->cb) | 1478 | cb_info = notifier_hnode_get_by_type(notify_id, type); |
| 1461 | cb_info->cb(notify_id, cb_info->cb_data); | 1479 | cb = cb_info ? cb_info->cb : NULL; |
| 1480 | cb_data = cb_info ? cb_info->cb_data : NULL; | ||
| 1481 | } | ||
| 1482 | |||
| 1483 | if (cb) | ||
| 1484 | cb(notify_id, cb_data); | ||
| 1462 | } | 1485 | } |
| 1463 | } | 1486 | } |
| 1464 | 1487 | ||
| @@ -1466,39 +1489,56 @@ static void handle_fwk_notif_callbacks(u32 bitmap) · 1 unmodified line | |||
| 1466 | { | 1489 | { |
| 1467 | void *buf; | 1490 | void *buf; |
| 1468 | uuid_t uuid; | 1491 | uuid_t uuid; |
| 1492 | void *fwk_cb_data; | ||
| 1469 | int notify_id = 0, target; | 1493 | int notify_id = 0, target; |
| 1494 | ffa_fwk_notifier_cb fwk_cb; | ||
| 1470 | struct ffa_indirect_msg_hdr *msg; | 1495 | struct ffa_indirect_msg_hdr *msg; |
| 1471 | struct notifier_cb_info *cb_info = NULL; | 1496 | size_t min_offset = offsetof(struct ffa_indirect_msg_hdr, uuid); |
| 1472 | 1497 | ||
| 1473 | /* Only one framework notification defined and supported for now */ | 1498 | /* Only one framework notification defined and supported for now */ |
| 1474 | if (!(bitmap & FRAMEWORK_NOTIFY_RX_BUFFER_FULL)) | 1499 | if (!(bitmap & FRAMEWORK_NOTIFY_RX_BUFFER_FULL)) |
| 1475 | return; | 1500 | return; |
| 1476 | 1501 | ||
| 1477 | mutex_lock(&drv_info->rx_lock); | 1502 | scoped_guard(mutex, &drv_info->rx_lock) { |
| 1503 | u32 offset, size; | ||
| 1478 | 1504 | ||
| 1479 | msg = drv_info->rx_buffer; | 1505 | msg = drv_info->rx_buffer; |
| 1480 | buf = kmemdup((void *)msg + msg->offset, msg->size, GFP_KERNEL); | 1506 | offset = msg->offset; |
| 1481 | if (!buf) { | 1507 | size = msg->size; |
| 1482 | mutex_unlock(&drv_info->rx_lock); | ||
| 1483 | return; | ||
| 1484 | } | ||
| 1485 | 1508 | ||
| 1486 | target = SENDER_ID(msg->send_recv_id); | 1509 | if (!size || (offset != min_offset && offset < sizeof(*msg)) || |
| 1487 | if (msg->offset >= sizeof(*msg)) | 1510 | offset > drv_info->rxtx_bufsz || |
| 1488 | uuid_copy(&uuid, &msg->uuid); | 1511 | size > drv_info->rxtx_bufsz - offset) { |
| 1489 | else | 1512 | pr_err("invalid framework notification message\n"); |
| 1490 | uuid_copy(&uuid, &uuid_null); | 1513 | ffa_rx_release(); |
| 1514 | return; | ||
| 1515 | } | ||
| 1491 | 1516 | ||
| 1492 | mutex_unlock(&drv_info->rx_lock); | 1517 | buf = kmemdup((void *)msg + offset, size, GFP_KERNEL); |
| 1518 | if (!buf) { | ||
| 1519 | ffa_rx_release(); | ||
| 1520 | return; | ||
| 1521 | } | ||
| 1522 | |||
| 1523 | target = SENDER_ID(msg->send_recv_id); | ||
| 1524 | if (offset >= sizeof(*msg)) | ||
| 1525 | uuid_copy(&uuid, &msg->uuid); | ||
| 1526 | else | ||
| 1527 | uuid_copy(&uuid, &uuid_null); | ||
| 1528 | ffa_rx_release(); | ||
| 1529 | } | ||
| 1493 | 1530 | ||
| 1494 | ffa_rx_release(); | 1531 | scoped_guard(read_lock, &drv_info->notify_lock) { |
| 1532 | struct notifier_cb_info *cb_info; | ||
| 1495 | 1533 | ||
| 1496 | read_lock(&drv_info->notify_lock); | 1534 | cb_info = notifier_hnode_get_by_vmid_uuid(notify_id, target, |
| 1497 | cb_info = notifier_hnode_get_by_vmid_uuid(notify_id, target, &uuid); | 1535 | &uuid); |
| 1498 | read_unlock(&drv_info->notify_lock); | 1536 | fwk_cb = cb_info ? cb_info->fwk_cb : NULL; |
| 1537 | fwk_cb_data = cb_info ? cb_info->cb_data : NULL; | ||
| 1538 | } | ||
| 1499 | 1539 | ||
| 1500 | if (cb_info && cb_info->fwk_cb) | 1540 | if (fwk_cb) |
| 1501 | cb_info->fwk_cb(notify_id, cb_info->cb_data, buf); | 1541 | fwk_cb(notify_id, fwk_cb_data, buf); |
| 1502 | kfree(buf); | 1542 | kfree(buf); |
| 1503 | } | 1543 | } |
| 1504 | 1544 | ||
| @@ -1539,10 +1579,11 @@ ffa_self_notif_handle(u16 vcpu, bool is_per_vcpu, void *cb_data) · 34 unmodified lines | |||
| 1539 | 1579 | ||
| 1540 | static void notif_pcpu_irq_work_fn(struct work_struct *work) | 1580 | static void notif_pcpu_irq_work_fn(struct work_struct *work) |
| 1541 | { | 1581 | { |
| 1542 | struct ffa_drv_info *info = container_of(work, struct ffa_drv_info, | 1582 | struct ffa_pcpu_irq *pcpu = container_of(work, struct ffa_pcpu_irq, |
| 1543 | notif_pcpu_work); | 1583 | notif_pcpu_work); |
| 1584 | struct ffa_drv_info *info = pcpu->info; | ||
| 1544 | 1585 | ||
| 1545 | ffa_self_notif_handle(smp_processor_id(), true, info); | 1586 | notif_get_and_handle(info); |
| 1546 | } | 1587 | } |
| 1547 | 1588 | ||
| 1548 | static const struct ffa_info_ops ffa_drv_info_ops = { | 1589 | static const struct ffa_info_ops ffa_drv_info_ops = { |
| @@ -1629,6 +1670,15 @@ static struct notifier_block ffa_bus_nb = { · 80 unmodified lines | |||
| 1629 | .notifier_call = ffa_bus_notifier, | 1670 | .notifier_call = ffa_bus_notifier, |
| 1630 | }; | 1671 | }; |
| 1631 | 1672 | ||
| 1673 | static void ffa_bus_notifier_unregister(void) | ||
| 1674 | { | ||
| 1675 | if (!drv_info->bus_notifier_registered) | ||
| 1676 | return; | ||
| 1677 | |||
| 1678 | bus_unregister_notifier(&ffa_bus_type, &ffa_bus_nb); | ||
| 1679 | drv_info->bus_notifier_registered = false; | ||
| 1680 | } | ||
| 1681 | |||
| 1632 | static int ffa_xa_add_partition_info(struct ffa_device *dev) | 1682 | static int ffa_xa_add_partition_info(struct ffa_device *dev) |
| 1633 | { | 1683 | { |
| 1634 | struct ffa_dev_part_info *info; | 1684 | struct ffa_dev_part_info *info; |
| @@ -1712,6 +1762,8 @@ static void ffa_partitions_cleanup(void) · 77 unmodified lines | |||
| 1712 | struct list_head *phead; | 1762 | struct list_head *phead; |
| 1713 | unsigned long idx; | 1763 | unsigned long idx; |
| 1714 | 1764 | ||
| 1765 | ffa_bus_notifier_unregister(); | ||
| 1766 | |||
| 1715 | /* Clean up/free all registered devices */ | 1767 | /* Clean up/free all registered devices */ |
| 1716 | ffa_devices_unregister(); | 1768 | ffa_devices_unregister(); |
| 1717 | 1769 | ||
| @@ -1739,11 +1791,14 @@ static int ffa_setup_partitions(void) · 21 unmodified lines | |||
| 1739 | ret = bus_register_notifier(&ffa_bus_type, &ffa_bus_nb); | 1791 | ret = bus_register_notifier(&ffa_bus_type, &ffa_bus_nb); |
| 1740 | if (ret) | 1792 | if (ret) |
| 1741 | pr_err("Failed to register FF-A bus notifiers\n"); | 1793 | pr_err("Failed to register FF-A bus notifiers\n"); |
| 1794 | else | ||
| 1795 | drv_info->bus_notifier_registered = true; | ||
| 1742 | } | 1796 | } |
| 1743 | 1797 | ||
| 1744 | count = ffa_partition_probe(&uuid_null, &pbuf); | 1798 | count = ffa_partition_probe(&uuid_null, &pbuf); |
| 1745 | if (count <= 0) { | 1799 | if (count <= 0) { |
| 1746 | pr_info("%s: No partitions found, error %d\n", __func__, count); | 1800 | pr_info("%s: No partitions found, error %d\n", __func__, count); |
| 1801 | ffa_bus_notifier_unregister(); | ||
| 1747 | return -EINVAL; | 1802 | return -EINVAL; |
| 1748 | } | 1803 | } |
| 1749 | 1804 | ||
| @@ -1811,7 +1866,7 @@ static irqreturn_t notif_pend_irq_handler(int irq, void *irq_data) · 61 unmodified lines | |||
| 1811 | struct ffa_drv_info *info = pcpu->info; | 1866 | struct ffa_drv_info *info = pcpu->info; |
| 1812 | 1867 | ||
| 1813 | queue_work_on(smp_processor_id(), info->notif_pcpu_wq, | 1868 | queue_work_on(smp_processor_id(), info->notif_pcpu_wq, |
| 1814 | &info->notif_pcpu_work); | 1869 | &pcpu->notif_pcpu_work); |
| 1815 | 1870 | ||
| 1816 | return IRQ_HANDLED; | 1871 | return IRQ_HANDLED; |
| 1817 | } | 1872 | } |
| @@ -1928,8 +1983,11 @@ static int ffa_init_pcpu_irq(void) · 110 unmodified lines | |||
| 1928 | if (!irq_pcpu) | 1983 | if (!irq_pcpu) |
| 1929 | return -ENOMEM; | 1984 | return -ENOMEM; |
| 1930 | 1985 | ||
| 1931 | for_each_present_cpu(cpu) | 1986 | for_each_present_cpu(cpu) { |
| 1932 | per_cpu_ptr(irq_pcpu, cpu)->info = drv_info; | 1987 | per_cpu_ptr(irq_pcpu, cpu)->info = drv_info; |
| 1988 | INIT_WORK(&per_cpu_ptr(irq_pcpu, cpu)->notif_pcpu_work, | ||
| 1989 | notif_pcpu_irq_work_fn); | ||
| 1990 | } | ||
| 1933 | 1991 | ||
| 1934 | drv_info->irq_pcpu = irq_pcpu; | 1992 | drv_info->irq_pcpu = irq_pcpu; |
| 1935 | 1993 | ||
| @@ -1958,7 +2016,6 @@ static int ffa_init_pcpu_irq(void) · 22 unmodified lines | |||
| 1958 | } | 2016 | } |
| 1959 | 2017 | ||
| 1960 | INIT_WORK(&drv_info->sched_recv_irq_work, ffa_sched_recv_irq_work_fn); | 2018 | INIT_WORK(&drv_info->sched_recv_irq_work, ffa_sched_recv_irq_work_fn); |
| 1961 | INIT_WORK(&drv_info->notif_pcpu_work, notif_pcpu_irq_work_fn); | ||
| 1962 | drv_info->notif_pcpu_wq = create_workqueue("ffa_pcpu_irq_notification"); | 2019 | drv_info->notif_pcpu_wq = create_workqueue("ffa_pcpu_irq_notification"); |
| 1963 | if (!drv_info->notif_pcpu_wq) | 2020 | if (!drv_info->notif_pcpu_wq) |
| 1964 | return -EINVAL; | 2021 | return -EINVAL; |
| @@ -2063,11 +2120,12 @@ static int __init ffa_init(void) · 98 unmodified lines | |||
| 2063 | rxtx_bufsz = SZ_4K; | 2120 | rxtx_bufsz = SZ_4K; |
| 2064 | } | 2121 | } |
| 2065 | 2122 | ||
| 2123 | rxtx_bufsz = PAGE_ALIGN(rxtx_bufsz); | ||
| 2066 | drv_info->rxtx_bufsz = rxtx_bufsz; | 2124 | drv_info->rxtx_bufsz = rxtx_bufsz; |
| 2067 | drv_info->rx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); | 2125 | drv_info->rx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); |
| 2068 | if (!drv_info->rx_buffer) { | 2126 | if (!drv_info->rx_buffer) { |
| 2069 | ret = -ENOMEM; | 2127 | ret = -ENOMEM; |
| 2070 | goto free_pages; | 2128 | goto free_drv_info; |
| 2071 | } | 2129 | } |
| 2072 | 2130 | ||
| 2073 | drv_info->tx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); | 2131 | drv_info->tx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); |
| @@ -2078,7 +2136,7 @@ static int __init ffa_init(void) · 4 unmodified lines | |||
| 2078 | 2136 | ||
| 2079 | ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer), | 2137 | ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer), |
| 2080 | virt_to_phys(drv_info->rx_buffer), | 2138 | virt_to_phys(drv_info->rx_buffer), |
| 2081 | PAGE_ALIGN(rxtx_bufsz) / FFA_PAGE_SIZE); | 2139 | rxtx_bufsz / FFA_PAGE_SIZE); |
| 2082 | if (ret) { | 2140 | if (ret) { |
| 2083 | pr_err("failed to register FFA RxTx buffers\n"); | 2141 | pr_err("failed to register FFA RxTx buffers\n"); |
| 2084 | goto free_pages; | 2142 | goto free_pages; |
| @@ -60,8 +60,10 @@ static int efi_pstore_open(struct pstore_info *psi) · 59 unmodified lines | |||
| 60 | return err; | 60 | return err; |
| 61 | 61 | ||
| 62 | psi->data = kzalloc(record_size, GFP_KERNEL); | 62 | psi->data = kzalloc(record_size, GFP_KERNEL); |
| 63 | if (!psi->data) | 63 | if (!psi->data) { |
| 64 | efivar_unlock(); | ||
| 64 | return -ENOMEM; | 65 | return -ENOMEM; |
| 66 | } | ||
| 65 | 67 | ||
| 66 | return 0; | 68 | return 0; |
| 67 | } | 69 | } |
| @@ -402,21 +402,11 @@ static void __init efi_debugfs_init(void) · 401 unmodified lines | |||
| 402 | static inline void efi_debugfs_init(void) {} | 402 | static inline void efi_debugfs_init(void) {} |
| 403 | #endif | 403 | #endif |
| 404 | 404 | ||
| 405 | /* | 405 | static int __init efipostcore_init(void) |
| 406 | * We register the efi subsystem with the firmware subsystem and the | ||
| 407 | * efivars subsystem with the efi subsystem, if the system was booted with | ||
| 408 | * EFI. | ||
| 409 | */ | ||
| 410 | static int __init efisubsys_init(void) | ||
| 411 | { | 406 | { |
| 412 | int error; | ||
| 413 | |||
| 414 | if (!efi_enabled(EFI_RUNTIME_SERVICES)) | 407 | if (!efi_enabled(EFI_RUNTIME_SERVICES)) |
| 415 | efi.runtime_supported_mask = 0; | 408 | efi.runtime_supported_mask = 0; |
| 416 | 409 | ||
| 417 | if (!efi_enabled(EFI_BOOT)) | ||
| 418 | return 0; | ||
| 419 | |||
| 420 | if (efi.runtime_supported_mask) { | 410 | if (efi.runtime_supported_mask) { |
| 421 | /* | 411 | /* |
| 422 | * Since we process only one efi_runtime_service() at a time, an | 412 | * Since we process only one efi_runtime_service() at a time, an |
| @@ -428,9 +418,23 @@ static int __init efisubsys_init(void) · 5 unmodified lines | |||
| 428 | pr_err("Creating efi_rts_wq failed, EFI runtime services disabled.\n"); | 418 | pr_err("Creating efi_rts_wq failed, EFI runtime services disabled.\n"); |
| 429 | clear_bit(EFI_RUNTIME_SERVICES, &efi.flags); | 419 | clear_bit(EFI_RUNTIME_SERVICES, &efi.flags); |
| 430 | efi.runtime_supported_mask = 0; | 420 | efi.runtime_supported_mask = 0; |
| 431 | return 0; | ||
| 432 | } | 421 | } |
| 433 | } | 422 | } |
| 423 | return 0; | ||
| 424 | } | ||
| 425 | postcore_initcall(efipostcore_init); | ||
| 426 | |||
| 427 | /* | ||
| 428 | * We register the efi subsystem with the firmware subsystem and the | ||
| 429 | * efivars subsystem with the efi subsystem, if the system was booted with | ||
| 430 | * EFI. | ||
| 431 | */ | ||
| 432 | static int __init efisubsys_init(void) | ||
| 433 | { | ||
| 434 | int error; | ||
| 435 | |||
| 436 | if (!efi_enabled(EFI_BOOT)) | ||
| 437 | return 0; | ||
| 434 | 438 | ||
| 435 | if (efi_rt_services_supported(EFI_RT_SUPPORTED_TIME_SERVICES)) | 439 | if (efi_rt_services_supported(EFI_RT_SUPPORTED_TIME_SERVICES)) |
| 436 | platform_device_register_simple("rtc-efi", 0, NULL, 0); | 440 | platform_device_register_simple("rtc-efi", 0, NULL, 0); |
| @@ -66,7 +66,7 @@ KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__ · 65 unmodified lines | |||
| 66 | lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \ | 66 | lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \ |
| 67 | file.o mem.o random.o randomalloc.o pci.o \ | 67 | file.o mem.o random.o randomalloc.o pci.o \ |
| 68 | skip_spaces.o lib-cmdline.o lib-ctype.o \ | 68 | skip_spaces.o lib-cmdline.o lib-ctype.o \ |
| 69 | alignedmem.o relocate.o printk.o vsprintf.o | 69 | alignedmem.o printk.o vsprintf.o |
| 70 | 70 | ||
| 71 | # include the stub's libfdt dependencies from lib/ when needed | 71 | # include the stub's libfdt dependencies from lib/ when needed |
| 72 | libfdt-deps := fdt_rw.c fdt_ro.c fdt_wip.c fdt.c \ | 72 | libfdt-deps := fdt_rw.c fdt_ro.c fdt_wip.c fdt.c \ |
| @@ -79,6 +79,10 @@ efi_status_t efi_parse_options(char const *cmdline) · 78 unmodified lines | |||
| 79 | efi_noinitrd = true; | 79 | efi_noinitrd = true; |
| 80 | } else if (IS_ENABLED(CONFIG_X86_64) && !strcmp(param, "no5lvl")) { | 80 | } else if (IS_ENABLED(CONFIG_X86_64) && !strcmp(param, "no5lvl")) { |
| 81 | efi_no5lvl = true; | 81 | efi_no5lvl = true; |
| 82 | } else if (IS_ENABLED(CONFIG_LOONGARCH) && | ||
| 83 | IS_ENABLED(CONFIG_HIBERNATION) && | ||
| 84 | !strcmp(param, "resume") && val) { | ||
| 85 | efi_nokaslr = true; /* LoongArch can't KASLR for hibernation */ | ||
| 82 | } else if (IS_ENABLED(CONFIG_ARCH_HAS_MEM_ENCRYPT) && | 86 | } else if (IS_ENABLED(CONFIG_ARCH_HAS_MEM_ENCRYPT) && |
| 83 | !strcmp(param, "mem_encrypt") && val) { | 87 | !strcmp(param, "mem_encrypt") && val) { |
| 84 | if (parse_option_str(val, "on")) | 88 | if (parse_option_str(val, "on")) |
| @@ -1104,13 +1104,6 @@ efi_status_t efi_allocate_pages_aligned(unsigned long size, unsigned long *addr, · 1103 unmodified lines | |||
| 1104 | efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, | 1104 | efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, |
| 1105 | unsigned long *addr, unsigned long min); | 1105 | unsigned long *addr, unsigned long min); |
| 1106 | 1106 | ||
| 1107 | efi_status_t efi_relocate_kernel(unsigned long *image_addr, | ||
| 1108 | unsigned long image_size, | ||
| 1109 | unsigned long alloc_size, | ||
| 1110 | unsigned long preferred_addr, | ||
| 1111 | unsigned long alignment, | ||
| 1112 | unsigned long min_addr); | ||
| 1113 | |||
| 1114 | efi_status_t efi_parse_options(char const *cmdline); | 1107 | efi_status_t efi_parse_options(char const *cmdline); |
| 1115 | 1108 | ||
| 1116 | void efi_parse_option_graphics(char *option); | 1109 | void efi_parse_option_graphics(char *option); |
| @@ -14,6 +14,86 @@ extern int kernel_asize; · 13 unmodified lines | |||
| 14 | extern int kernel_fsize; | 14 | extern int kernel_fsize; |
| 15 | extern int kernel_entry; | 15 | extern int kernel_entry; |
| 16 | 16 | ||
| 17 | /** | ||
| 18 | * efi_relocate_kernel() - copy memory area | ||
| 19 | * @image_addr: pointer to address of memory area to copy | ||
| 20 | * @image_size: size of memory area to copy | ||
| 21 | * @alloc_size: minimum size of memory to allocate, must be greater or | ||
| 22 | * equal to image_size | ||
| 23 | * @preferred_addr: preferred target address | ||
| 24 | * @alignment: minimum alignment of the allocated memory area. It | ||
| 25 | * should be a power of two. | ||
| 26 | * @min_addr: minimum target address | ||
| 27 | * | ||
| 28 | * Copy a memory area to a newly allocated memory area aligned according | ||
| 29 | * to @alignment but at least EFI_ALLOC_ALIGN. If the preferred address | ||
| 30 | * is not available, the allocated address will not be below @min_addr. | ||
| 31 | * On exit, @image_addr is updated to the target copy address that was used. | ||
| 32 | * | ||
| 33 | * This function is used to copy the Linux kernel verbatim. It does not apply | ||
| 34 | * any relocation changes. | ||
| 35 | * | ||
| 36 | * Return: status code | ||
| 37 | */ | ||
| 38 | static | ||
| 39 | efi_status_t efi_relocate_kernel(unsigned long *image_addr, | ||
| 40 | unsigned long image_size, | ||
| 41 | unsigned long alloc_size, | ||
| 42 | unsigned long preferred_addr, | ||
| 43 | unsigned long alignment, | ||
| 44 | unsigned long min_addr) | ||
| 45 | { | ||
| 46 | unsigned long cur_image_addr; | ||
| 47 | unsigned long new_addr = 0; | ||
| 48 | efi_status_t status; | ||
| 49 | unsigned long nr_pages; | ||
| 50 | efi_physical_addr_t efi_addr = preferred_addr; | ||
| 51 | |||
| 52 | if (!image_addr || !image_size || !alloc_size) | ||
| 53 | return EFI_INVALID_PARAMETER; | ||
| 54 | if (alloc_size < image_size) | ||
| 55 | return EFI_INVALID_PARAMETER; | ||
| 56 | |||
| 57 | cur_image_addr = *image_addr; | ||
| 58 | |||
| 59 | /* | ||
| 60 | * The EFI firmware loader could have placed the kernel image | ||
| 61 | * anywhere in memory, but the kernel has restrictions on the | ||
| 62 | * max physical address it can run at. Some architectures | ||
| 63 | * also have a preferred address, so first try to relocate | ||
| 64 | * to the preferred address. If that fails, allocate as low | ||
| 65 | * as possible while respecting the required alignment. | ||
| 66 | */ | ||
| 67 | nr_pages = round_up(alloc_size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; | ||
| 68 | status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | ||
| 69 | EFI_LOADER_DATA, nr_pages, &efi_addr); | ||
| 70 | new_addr = efi_addr; | ||
| 71 | /* | ||
| 72 | * If preferred address allocation failed allocate as low as | ||
| 73 | * possible. | ||
| 74 | */ | ||
| 75 | if (status != EFI_SUCCESS) { | ||
| 76 | status = efi_low_alloc_above(alloc_size, alignment, &new_addr, | ||
| 77 | min_addr); | ||
| 78 | } | ||
| 79 | if (status != EFI_SUCCESS) { | ||
| 80 | efi_err("Failed to allocate usable memory for kernel.\n"); | ||
| 81 | return status; | ||
| 82 | } | ||
| 83 | |||
| 84 | /* | ||
| 85 | * We know source/dest won't overlap since both memory ranges | ||
| 86 | * have been allocated by UEFI, so we can safely use memcpy. | ||
| 87 | */ | ||
| 88 | memcpy((void *)new_addr, (void *)cur_image_addr, image_size); | ||
| 89 | efi_cache_sync_image(new_addr, image_size); | ||
| 90 | |||
| 91 | /* Return the new address of the relocated image. */ | ||
| 92 | *image_addr = new_addr; | ||
| 93 | |||
| 94 | return status; | ||
| 95 | } | ||
| 96 | |||
| 17 | efi_status_t handle_kernel_image(unsigned long *image_addr, | 97 | efi_status_t handle_kernel_image(unsigned long *image_addr, |
| 18 | unsigned long *image_size, | 98 | unsigned long *image_size, |
| 19 | unsigned long *reserve_addr, | 99 | unsigned long *reserve_addr, |
| @@ -18,6 +18,27 @@ efi_status_t check_platform_features(void) · 17 unmodified lines | |||
| 18 | return EFI_SUCCESS; | 18 | return EFI_SUCCESS; |
| 19 | } | 19 | } |
| 20 | 20 | ||
| 21 | void efi_cache_sync_image(unsigned long image_base, unsigned long alloc_size) | ||
| 22 | { | ||
| 23 | asm volatile ("ibar 0" ::: "memory"); | ||
| 24 | } | ||
| 25 | |||
| 26 | unsigned long efi_get_kimg_kaslr_address(void) | ||
| 27 | { | ||
| 28 | unsigned int random_offset = 0; | ||
| 29 | |||
| 30 | #ifdef CONFIG_RANDOMIZE_BASE | ||
| 31 | if (!efi_nokaslr) { | ||
| 32 | efi_get_random_bytes(sizeof(random_offset), (u8 *)&random_offset); | ||
| 33 | random_offset ^= (random_get_entropy() << 16); | ||
| 34 | random_offset &= (CONFIG_RANDOMIZE_BASE_MAX_OFFSET - 1); | ||
| 35 | random_offset = ALIGN(random_offset + SZ_64K, SZ_64K); | ||
| 36 | } | ||
| 37 | #endif | ||
| 38 | |||
| 39 | return PHYSADDR(VMLINUX_LOAD_ADDRESS) + random_offset; | ||
| 40 | } | ||
| 41 | |||
| 21 | struct exit_boot_struct { | 42 | struct exit_boot_struct { |
| 22 | efi_memory_desc_t *runtime_map; | 43 | efi_memory_desc_t *runtime_map; |
| 23 | int runtime_entry_count; | 44 | int runtime_entry_count; |
| @@ -124,3 +124,85 @@ void efi_free(unsigned long size, unsigned long addr) · 123 unmodified lines | |||
| 124 | nr_pages = round_up(size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; | 124 | nr_pages = round_up(size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; |
| 125 | efi_bs_call(free_pages, addr, nr_pages); | 125 | efi_bs_call(free_pages, addr, nr_pages); |
| 126 | } | 126 | } |
| 127 | |||
| 128 | /** | ||
| 129 | * efi_low_alloc_above() - allocate pages at or above given address | ||
| 130 | * @size: size of the memory area to allocate | ||
| 131 | * @align: minimum alignment of the allocated memory area. It should | ||
| 132 | * a power of two. | ||
| 133 | * @addr: on exit the address of the allocated memory | ||
| 134 | * @min: minimum address to used for the memory allocation | ||
| 135 | * | ||
| 136 | * Allocate at the lowest possible address that is not below @min as | ||
| 137 | * EFI_LOADER_DATA. The allocated pages are aligned according to @align but at | ||
| 138 | * least EFI_ALLOC_ALIGN. The first allocated page will not below the address | ||
| 139 | * given by @min. | ||
| 140 | * | ||
| 141 | * Return: status code | ||
| 142 | */ | ||
| 143 | efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, | ||
| 144 | unsigned long *addr, unsigned long min) | ||
| 145 | { | ||
| 146 | struct efi_boot_memmap *map __free(efi_pool) = NULL; | ||
| 147 | efi_status_t status; | ||
| 148 | unsigned long nr_pages; | ||
| 149 | int i; | ||
| 150 | |||
| 151 | status = efi_get_memory_map(&map, false); | ||
| 152 | if (status != EFI_SUCCESS) | ||
| 153 | return status; | ||
| 154 | |||
| 155 | /* | ||
| 156 | * Enforce minimum alignment that EFI or Linux requires when | ||
| 157 | * requesting a specific address. We are doing page-based (or | ||
| 158 | * larger) allocations, and both the address and size must meet | ||
| 159 | * alignment constraints. | ||
| 160 | */ | ||
| 161 | if (align < EFI_ALLOC_ALIGN) | ||
| 162 | align = EFI_ALLOC_ALIGN; | ||
| 163 | |||
| 164 | size = round_up(size, EFI_ALLOC_ALIGN); | ||
| 165 | nr_pages = size / EFI_PAGE_SIZE; | ||
| 166 | for (i = 0; i < map->map_size / map->desc_size; i++) { | ||
| 167 | efi_memory_desc_t *desc; | ||
| 168 | unsigned long m = (unsigned long)map->map; | ||
| 169 | u64 start, end; | ||
| 170 | |||
| 171 | desc = efi_memdesc_ptr(m, map->desc_size, i); | ||
| 172 | |||
| 173 | if (desc->type != EFI_CONVENTIONAL_MEMORY) | ||
| 174 | continue; | ||
| 175 | |||
| 176 | if (desc->attribute & EFI_MEMORY_HOT_PLUGGABLE) | ||
| 177 | continue; | ||
| 178 | |||
| 179 | if (efi_soft_reserve_enabled() && | ||
| 180 | (desc->attribute & EFI_MEMORY_SP)) | ||
| 181 | continue; | ||
| 182 | |||
| 183 | if (desc->num_pages < nr_pages) | ||
| 184 | continue; | ||
| 185 | |||
| 186 | start = desc->phys_addr; | ||
| 187 | end = start + desc->num_pages * EFI_PAGE_SIZE; | ||
| 188 | |||
| 189 | if (start < min) | ||
| 190 | start = min; | ||
| 191 | |||
| 192 | start = round_up(start, align); | ||
| 193 | if ((start + size) > end) | ||
| 194 | continue; | ||
| 195 | |||
| 196 | status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | ||
| 197 | EFI_LOADER_DATA, nr_pages, &start); | ||
| 198 | if (status == EFI_SUCCESS) { | ||
| 199 | *addr = start; | ||
| 200 | break; | ||
| 201 | } | ||
| 202 | } | ||
| 203 | |||
| 204 | if (i == map->map_size / map->desc_size) | ||
| 205 | return EFI_NOT_FOUND; | ||
| 206 | |||
| 207 | return EFI_SUCCESS; | ||
| 208 | } | ||
| @@ -1,166 +0,0 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | ||
| 2 | |||
| 3 | #include <linux/efi.h> | ||
| 4 | #include <asm/efi.h> | ||
| 5 | |||
| 6 | #include "efistub.h" | ||
| 7 | |||
| 8 | /** | ||
| 9 | * efi_low_alloc_above() - allocate pages at or above given address | ||
| 10 | * @size: size of the memory area to allocate | ||
| 11 | * @align: minimum alignment of the allocated memory area. It should | ||
| 12 | * a power of two. | ||
| 13 | * @addr: on exit the address of the allocated memory | ||
| 14 | * @min: minimum address to used for the memory allocation | ||
| 15 | * | ||
| 16 | * Allocate at the lowest possible address that is not below @min as | ||
| 17 | * EFI_LOADER_DATA. The allocated pages are aligned according to @align but at | ||
| 18 | * least EFI_ALLOC_ALIGN. The first allocated page will not below the address | ||
| 19 | * given by @min. | ||
| 20 | * | ||
| 21 | * Return: status code | ||
| 22 | */ | ||
| 23 | efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, | ||
| 24 | unsigned long *addr, unsigned long min) | ||
| 25 | { | ||
| 26 | struct efi_boot_memmap *map __free(efi_pool) = NULL; | ||
| 27 | efi_status_t status; | ||
| 28 | unsigned long nr_pages; | ||
| 29 | int i; | ||
| 30 | |||
| 31 | status = efi_get_memory_map(&map, false); | ||
| 32 | if (status != EFI_SUCCESS) | ||
| 33 | return status; | ||
| 34 | |||
| 35 | /* | ||
| 36 | * Enforce minimum alignment that EFI or Linux requires when | ||
| 37 | * requesting a specific address. We are doing page-based (or | ||
| 38 | * larger) allocations, and both the address and size must meet | ||
| 39 | * alignment constraints. | ||
| 40 | */ | ||
| 41 | if (align < EFI_ALLOC_ALIGN) | ||
| 42 | align = EFI_ALLOC_ALIGN; | ||
| 43 | |||
| 44 | size = round_up(size, EFI_ALLOC_ALIGN); | ||
| 45 | nr_pages = size / EFI_PAGE_SIZE; | ||
| 46 | for (i = 0; i < map->map_size / map->desc_size; i++) { | ||
| 47 | efi_memory_desc_t *desc; | ||
| 48 | unsigned long m = (unsigned long)map->map; | ||
| 49 | u64 start, end; | ||
| 50 | |||
| 51 | desc = efi_memdesc_ptr(m, map->desc_size, i); | ||
| 52 | |||
| 53 | if (desc->type != EFI_CONVENTIONAL_MEMORY) | ||
| 54 | continue; | ||
| 55 | |||
| 56 | if (desc->attribute & EFI_MEMORY_HOT_PLUGGABLE) | ||
| 57 | continue; | ||
| 58 | |||
| 59 | if (efi_soft_reserve_enabled() && | ||
| 60 | (desc->attribute & EFI_MEMORY_SP)) | ||
| 61 | continue; | ||
| 62 | |||
| 63 | if (desc->num_pages < nr_pages) | ||
| 64 | continue; | ||
| 65 | |||
| 66 | start = desc->phys_addr; | ||
| 67 | end = start + desc->num_pages * EFI_PAGE_SIZE; | ||
| 68 | |||
| 69 | if (start < min) | ||
| 70 | start = min; | ||
| 71 | |||
| 72 | start = round_up(start, align); | ||
| 73 | if ((start + size) > end) | ||
| 74 | continue; | ||
| 75 | |||
| 76 | status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | ||
| 77 | EFI_LOADER_DATA, nr_pages, &start); | ||
| 78 | if (status == EFI_SUCCESS) { | ||
| 79 | *addr = start; | ||
| 80 | break; | ||
| 81 | } | ||
| 82 | } | ||
| 83 | |||
| 84 | if (i == map->map_size / map->desc_size) | ||
| 85 | return EFI_NOT_FOUND; | ||
| 86 | |||
| 87 | return EFI_SUCCESS; | ||
| 88 | } | ||
| 89 | |||
| 90 | /** | ||
| 91 | * efi_relocate_kernel() - copy memory area | ||
| 92 | * @image_addr: pointer to address of memory area to copy | ||
| 93 | * @image_size: size of memory area to copy | ||
| 94 | * @alloc_size: minimum size of memory to allocate, must be greater or | ||
| 95 | * equal to image_size | ||
| 96 | * @preferred_addr: preferred target address | ||
| 97 | * @alignment: minimum alignment of the allocated memory area. It | ||
| 98 | * should be a power of two. | ||
| 99 | * @min_addr: minimum target address | ||
| 100 | * | ||
| 101 | * Copy a memory area to a newly allocated memory area aligned according | ||
| 102 | * to @alignment but at least EFI_ALLOC_ALIGN. If the preferred address | ||
| 103 | * is not available, the allocated address will not be below @min_addr. | ||
| 104 | * On exit, @image_addr is updated to the target copy address that was used. | ||
| 105 | * | ||
| 106 | * This function is used to copy the Linux kernel verbatim. It does not apply | ||
| 107 | * any relocation changes. | ||
| 108 | * | ||
| 109 | * Return: status code | ||
| 110 | */ | ||
| 111 | efi_status_t efi_relocate_kernel(unsigned long *image_addr, | ||
| 112 | unsigned long image_size, | ||
| 113 | unsigned long alloc_size, | ||
| 114 | unsigned long preferred_addr, | ||
| 115 | unsigned long alignment, | ||
| 116 | unsigned long min_addr) | ||
| 117 | { | ||
| 118 | unsigned long cur_image_addr; | ||
| 119 | unsigned long new_addr = 0; | ||
| 120 | efi_status_t status; | ||
| 121 | unsigned long nr_pages; | ||
| 122 | efi_physical_addr_t efi_addr = preferred_addr; | ||
| 123 | |||
| 124 | if (!image_addr || !image_size || !alloc_size) | ||
| 125 | return EFI_INVALID_PARAMETER; | ||
| 126 | if (alloc_size < image_size) | ||
| 127 | return EFI_INVALID_PARAMETER; | ||
| 128 | |||
| 129 | cur_image_addr = *image_addr; | ||
| 130 | |||
| 131 | /* | ||
| 132 | * The EFI firmware loader could have placed the kernel image | ||
| 133 | * anywhere in memory, but the kernel has restrictions on the | ||
| 134 | * max physical address it can run at. Some architectures | ||
| 135 | * also have a preferred address, so first try to relocate | ||
| 136 | * to the preferred address. If that fails, allocate as low | ||
| 137 | * as possible while respecting the required alignment. | ||
| 138 | */ | ||
| 139 | nr_pages = round_up(alloc_size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; | ||
| 140 | status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | ||
| 141 | EFI_LOADER_DATA, nr_pages, &efi_addr); | ||
| 142 | new_addr = efi_addr; | ||
| 143 | /* | ||
| 144 | * If preferred address allocation failed allocate as low as | ||
| 145 | * possible. | ||
| 146 | */ | ||
| 147 | if (status != EFI_SUCCESS) { | ||
| 148 | status = efi_low_alloc_above(alloc_size, alignment, &new_addr, | ||
| 149 | min_addr); | ||
| 150 | } | ||
| 151 | if (status != EFI_SUCCESS) { | ||
| 152 | efi_err("Failed to allocate usable memory for kernel.\n"); | ||
| 153 | return status; | ||
| 154 | } | ||
| 155 | |||
| 156 | /* | ||
| 157 | * We know source/dest won't overlap since both memory ranges | ||
| 158 | * have been allocated by UEFI, so we can safely use memcpy. | ||
| 159 | */ | ||
| 160 | memcpy((void *)new_addr, (void *)cur_image_addr, image_size); | ||
| 161 | |||
| 162 | /* Return the new address of the relocated image. */ | ||
| 163 | *image_addr = new_addr; | ||
| 164 | |||
| 165 | return status; | ||
| 166 | } | ||
| @@ -311,11 +311,14 @@ static const struct dmi_system_id efifb_dmi_swap_width_height[] __initconst = { · 310 unmodified lines | |||
| 311 | .callback = efifb_swap_width_height, | 311 | .callback = efifb_swap_width_height, |
| 312 | }, | 312 | }, |
| 313 | { | 313 | { |
| 314 | /* Lenovo IdeaPad Duet 3 10IGL5 with 1200x1920 portrait screen */ | 314 | /* |
| 315 | * Lenovo IdeaPad Duet 3 10IGL5 and 10IGL5-LTE with | ||
| 316 | * 1200x1920 portrait screen | ||
| 317 | */ | ||
| 315 | .matches = { | 318 | .matches = { |
| 316 | DMI_EXACT_MATCH(DMI_SYS_VENDOR, "LENOVO"), | 319 | DMI_EXACT_MATCH(DMI_SYS_VENDOR, "LENOVO"), |
| 317 | DMI_EXACT_MATCH(DMI_PRODUCT_VERSION, | 320 | /* Non exact match to also match the LTE version */ |
| 318 | "IdeaPad Duet 3 10IGL5"), | 321 | DMI_MATCH(DMI_PRODUCT_VERSION, "IdeaPad Duet 3 10IGL5"), |
| 319 | }, | 322 | }, |
| 320 | .callback = efifb_swap_width_height, | 323 | .callback = efifb_swap_width_height, |
| 321 | }, | 324 | }, |
| @@ -539,12 +539,22 @@ static int psci_system_suspend(unsigned long unused) · 538 unmodified lines | |||
| 539 | 539 | ||
| 540 | static int psci_system_suspend_enter(suspend_state_t state) | 540 | static int psci_system_suspend_enter(suspend_state_t state) |
| 541 | { | 541 | { |
| 542 | pm_set_resume_via_firmware(); | ||
| 543 | |||
| 542 | return cpu_suspend(0, psci_system_suspend); | 544 | return cpu_suspend(0, psci_system_suspend); |
| 543 | } | 545 | } |
| 544 | 546 | ||
| 547 | static int psci_system_suspend_begin(suspend_state_t state) | ||
| 548 | { | ||
| 549 | pm_set_suspend_via_firmware(); | ||
| 550 | |||
| 551 | return 0; | ||
| 552 | } | ||
| 553 | |||
| 545 | static const struct platform_suspend_ops psci_suspend_ops = { | 554 | static const struct platform_suspend_ops psci_suspend_ops = { |
| 546 | .valid = suspend_valid_only_mem, | 555 | .valid = suspend_valid_only_mem, |
| 547 | .enter = psci_system_suspend_enter, | 556 | .enter = psci_system_suspend_enter, |
| 557 | .begin = psci_system_suspend_begin, | ||
| 548 | }; | 558 | }; |
| 549 | 559 | ||
| 550 | static void __init psci_init_system_reset2(void) | 560 | static void __init psci_init_system_reset2(void) |
| @@ -362,6 +362,9 @@ static void *pdsfc_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, · 361 unmodified lines | |||
| 362 | void *out = NULL; | 362 | void *out = NULL; |
| 363 | int err; | 363 | int err; |
| 364 | 364 | ||
| 365 | if (in_len < sizeof(*rpc)) | ||
| 366 | return ERR_PTR(-EINVAL); | ||
| 367 | |||
| 365 | err = pdsfc_validate_rpc(pdsfc, rpc, scope); | 368 | err = pdsfc_validate_rpc(pdsfc, rpc, scope); |
| 366 | if (err) | 369 | if (err) |
| 367 | return ERR_PTR(err); | 370 | return ERR_PTR(err); |
| @@ -237,7 +237,9 @@ static irqreturn_t adnp_irq(int irq, void *data) · 236 unmodified lines | |||
| 237 | unsigned long pending; | 237 | unsigned long pending; |
| 238 | int err; | 238 | int err; |
| 239 | 239 | ||
| 240 | scoped_guard(mutex, &adnp->i2c_lock) { | 240 | { |
| 241 | guard(mutex)(&adnp->i2c_lock); | ||
| 242 | |||
| 241 | err = adnp_read(adnp, GPIO_PLR(adnp) + i, &level); | 243 | err = adnp_read(adnp, GPIO_PLR(adnp) + i, &level); |
| 242 | if (err < 0) | 244 | if (err < 0) |
| 243 | continue; | 245 | continue; |
| @@ -968,9 +968,12 @@ static int gpio_aggregator_activate(struct gpio_aggregator *aggr) · 967 unmodified lines | |||
| 968 | } | 968 | } |
| 969 | 969 | ||
| 970 | wait_for_device_probe(); | 970 | wait_for_device_probe(); |
| 971 | if (!device_is_bound(&pdev->dev)) { | 971 | |
| 972 | ret = -ENXIO; | 972 | scoped_guard(device, &pdev->dev) { |
| 973 | goto err_unregister_pdev; | 973 | if (!device_is_bound(&pdev->dev)) { |
| 974 | ret = -ENXIO; | ||
| 975 | goto err_unregister_pdev; | ||
| 976 | } | ||
| 974 | } | 977 | } |
| 975 | 978 | ||
| 976 | aggr->pdev = pdev; | 979 | aggr->pdev = pdev; |
| @@ -979,8 +982,8 @@ static int gpio_aggregator_activate(struct gpio_aggregator *aggr) · 2 unmodified lines | |||
| 979 | err_unregister_pdev: | 982 | err_unregister_pdev: |
| 980 | platform_device_unregister(pdev); | 983 | platform_device_unregister(pdev); |
| 981 | err_remove_lookup_table: | 984 | err_remove_lookup_table: |
| 982 | kfree(aggr->lookups->dev_id); | ||
| 983 | gpiod_remove_lookup_table(aggr->lookups); | 985 | gpiod_remove_lookup_table(aggr->lookups); |
| 986 | kfree(aggr->lookups->dev_id); | ||
| 984 | err_remove_swnode: | 987 | err_remove_swnode: |
| 985 | fwnode_remove_software_node(swnode); | 988 | fwnode_remove_software_node(swnode); |
| 986 | err_remove_lookups: | 989 | err_remove_lookups: |
| @@ -991,11 +994,15 @@ err_remove_lookups: · 4 unmodified lines | |||
| 991 | 994 | ||
| 992 | static void gpio_aggregator_deactivate(struct gpio_aggregator *aggr) | 995 | static void gpio_aggregator_deactivate(struct gpio_aggregator *aggr) |
| 993 | { | 996 | { |
| 997 | struct fwnode_handle *swnode; | ||
| 998 | |||
| 999 | swnode = dev_fwnode(&aggr->pdev->dev); | ||
| 994 | platform_device_unregister(aggr->pdev); | 1000 | platform_device_unregister(aggr->pdev); |
| 995 | aggr->pdev = NULL; | 1001 | aggr->pdev = NULL; |
| 996 | gpiod_remove_lookup_table(aggr->lookups); | 1002 | gpiod_remove_lookup_table(aggr->lookups); |
| 997 | kfree(aggr->lookups->dev_id); | 1003 | kfree(aggr->lookups->dev_id); |
| 998 | kfree(aggr->lookups); | 1004 | kfree(aggr->lookups); |
| 1005 | fwnode_remove_software_node(swnode); | ||
| 999 | } | 1006 | } |
| 1000 | 1007 | ||
| 1001 | static void gpio_aggregator_lockup_configfs(struct gpio_aggregator *aggr, | 1008 | static void gpio_aggregator_lockup_configfs(struct gpio_aggregator *aggr, |
| @@ -469,7 +469,7 @@ static int mxc_gpio_probe(struct platform_device *pdev) · 468 unmodified lines | |||
| 469 | * the handler is needed only once, but doing it for every port | 469 | * the handler is needed only once, but doing it for every port |
| 470 | * is more robust and easier. | 470 | * is more robust and easier. |
| 471 | */ | 471 | */ |
| 472 | port->irq_high = -1; | 472 | port->irq_high = 0; |
| 473 | port->mx_irq_handler = mx2_gpio_irq_handler; | 473 | port->mx_irq_handler = mx2_gpio_irq_handler; |
| 474 | } else | 474 | } else |
| 475 | port->mx_irq_handler = mx3_gpio_irq_handler; | 475 | port->mx_irq_handler = mx3_gpio_irq_handler; |
| @@ -1411,7 +1411,7 @@ static int pca953x_resume(struct device *dev) · 1410 unmodified lines | |||
| 1411 | ret = regulator_enable(chip->regulator); | 1411 | ret = regulator_enable(chip->regulator); |
| 1412 | if (ret) { | 1412 | if (ret) { |
| 1413 | dev_err(dev, "Failed to enable regulator: %d\n", ret); | 1413 | dev_err(dev, "Failed to enable regulator: %d\n", ret); |
| 1414 | return 0; | 1414 | return ret; |
| 1415 | } | 1415 | } |
| 1416 | } | 1416 | } |
| 1417 | 1417 | ||
| @@ -638,10 +638,17 @@ fail: · 637 unmodified lines | |||
| 638 | return ret; | 638 | return ret; |
| 639 | } | 639 | } |
| 640 | 640 | ||
| 641 | static void rockchip_clk_put(void *data) | ||
| 642 | { | ||
| 643 | struct clk *clk = data; | ||
| 644 | |||
| 645 | clk_put(clk); | ||
| 646 | } | ||
| 647 | |||
| 641 | static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) | 648 | static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) |
| 642 | { | 649 | { |
| 643 | struct resource res; | 650 | struct resource res; |
| 644 | int id = 0; | 651 | int id = 0, ret; |
| 645 | 652 | ||
| 646 | if (of_address_to_resource(bank->of_node, 0, &res)) { | 653 | if (of_address_to_resource(bank->of_node, 0, &res)) { |
| 647 | dev_err(bank->dev, "cannot find IO resource for bank\n"); | 654 | dev_err(bank->dev, "cannot find IO resource for bank\n"); |
| @@ -656,11 +663,10 @@ static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) · 8 unmodified lines | |||
| 656 | if (!bank->irq) | 663 | if (!bank->irq) |
| 657 | return -EINVAL; | 664 | return -EINVAL; |
| 658 | 665 | ||
| 659 | bank->clk = of_clk_get(bank->of_node, 0); | 666 | bank->clk = devm_clk_get_enabled(bank->dev, NULL); |
| 660 | if (IS_ERR(bank->clk)) | 667 | if (IS_ERR(bank->clk)) |
| 661 | return PTR_ERR(bank->clk); | 668 | return PTR_ERR(bank->clk); |
| 662 | 669 | ||
| 663 | clk_prepare_enable(bank->clk); | ||
| 664 | id = readl(bank->reg_base + gpio_regs_v2.version_id); | 670 | id = readl(bank->reg_base + gpio_regs_v2.version_id); |
| 665 | 671 | ||
| 666 | switch (id) { | 672 | switch (id) { |
| @@ -672,9 +678,13 @@ static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) · 5 unmodified lines | |||
| 672 | bank->db_clk = of_clk_get(bank->of_node, 1); | 678 | bank->db_clk = of_clk_get(bank->of_node, 1); |
| 673 | if (IS_ERR(bank->db_clk)) { | 679 | if (IS_ERR(bank->db_clk)) { |
| 674 | dev_err(bank->dev, "cannot find debounce clk\n"); | 680 | dev_err(bank->dev, "cannot find debounce clk\n"); |
| 675 | clk_disable_unprepare(bank->clk); | ||
| 676 | return -EINVAL; | 681 | return -EINVAL; |
| 677 | } | 682 | } |
| 683 | |||
| 684 | ret = devm_add_action_or_reset(bank->dev, rockchip_clk_put, | ||
| 685 | bank->db_clk); | ||
| 686 | if (ret) | ||
| 687 | return ret; | ||
| 678 | break; | 688 | break; |
| 679 | case GPIO_TYPE_V1: | 689 | case GPIO_TYPE_V1: |
| 680 | bank->gpio_regs = &gpio_regs_v1; | 690 | bank->gpio_regs = &gpio_regs_v1; |
| @@ -751,7 +761,6 @@ static int rockchip_gpio_probe(struct platform_device *pdev) · 70 unmodified lines | |||
| 751 | 761 | ||
| 752 | ret = rockchip_gpiolib_register(bank); | 762 | ret = rockchip_gpiolib_register(bank); |
| 753 | if (ret) { | 763 | if (ret) { |
| 754 | clk_disable_unprepare(bank->clk); | ||
| 755 | mutex_unlock(&bank->deferred_lock); | 764 | mutex_unlock(&bank->deferred_lock); |
| 756 | return ret; | 765 | return ret; |
| 757 | } | 766 | } |
| @@ -792,7 +801,9 @@ static void rockchip_gpio_remove(struct platform_device *pdev) · 34 unmodified lines | |||
| 792 | { | 801 | { |
| 793 | struct rockchip_pin_bank *bank = platform_get_drvdata(pdev); | 802 | struct rockchip_pin_bank *bank = platform_get_drvdata(pdev); |
| 794 | 803 | ||
| 795 | clk_disable_unprepare(bank->clk); | 804 | irq_set_chained_handler_and_data(bank->irq, NULL, NULL); |
| 805 | if (bank->domain) | ||
| 806 | irq_domain_remove(bank->domain); | ||
| 796 | gpiochip_remove(&bank->gpio_chip); | 807 | gpiochip_remove(&bank->gpio_chip); |
| 797 | } | 808 | } |
| 798 | 809 | ||
| @@ -103,9 +103,18 @@ static void gpio_shared_proxy_free(struct gpio_chip *gc, unsigned int offset) · 102 unmodified lines | |||
| 103 | { | 103 | { |
| 104 | struct gpio_shared_proxy_data *proxy = gpiochip_get_data(gc); | 104 | struct gpio_shared_proxy_data *proxy = gpiochip_get_data(gc); |
| 105 | struct gpio_shared_desc *shared_desc = proxy->shared_desc; | 105 | struct gpio_shared_desc *shared_desc = proxy->shared_desc; |
| 106 | int ret; | ||
| 106 | 107 | ||
| 107 | guard(gpio_shared_desc_lock)(shared_desc); | 108 | guard(gpio_shared_desc_lock)(shared_desc); |
| 108 | 109 | ||
| 110 | if (proxy->voted_high) { | ||
| 111 | ret = gpio_shared_proxy_set_unlocked(proxy, | ||
| 112 | shared_desc->can_sleep ? gpiod_set_value_cansleep : gpiod_set_value, 0); | ||
| 113 | if (ret) | ||
| 114 | dev_err(proxy->dev, | ||
| 115 | "Failed to unset the shared GPIO value on release: %d\n", ret); | ||
| 116 | } | ||
| 117 | |||
| 109 | proxy->shared_desc->usecnt--; | 118 | proxy->shared_desc->usecnt--; |
| 110 | 119 | ||
| 111 | dev_dbg(proxy->dev, "Shared GPIO freed, number of users: %u\n", | 120 | dev_dbg(proxy->dev, "Shared GPIO freed, number of users: %u\n", |
| @@ -901,7 +901,7 @@ static int gpio_sim_device_activate(struct gpio_sim_device *dev) · 900 unmodified lines | |||
| 901 | struct platform_device *pdev; | 901 | struct platform_device *pdev; |
| 902 | struct fwnode_handle *swnode; | 902 | struct fwnode_handle *swnode; |
| 903 | struct gpio_sim_bank *bank; | 903 | struct gpio_sim_bank *bank; |
| 904 | int ret; | 904 | int ret = 0; |
| 905 | 905 | ||
| 906 | lockdep_assert_held(&dev->lock); | 906 | lockdep_assert_held(&dev->lock); |
| 907 | 907 | ||
| @@ -945,9 +945,12 @@ static int gpio_sim_device_activate(struct gpio_sim_device *dev) · 37 unmodified lines | |||
| 945 | } | 945 | } |
| 946 | 946 | ||
| 947 | wait_for_device_probe(); | 947 | wait_for_device_probe(); |
| 948 | if (!device_is_bound(&pdev->dev)) { | 948 | |
| 949 | ret = -ENXIO; | 949 | scoped_guard(device, &pdev->dev) { |
| 950 | goto err_unregister_pdev; | 950 | if (!device_is_bound(&pdev->dev)) { |
| 951 | ret = -ENXIO; | ||
| 952 | goto err_unregister_pdev; | ||
| 953 | } | ||
| 951 | } | 954 | } |
| 952 | 955 | ||
| 953 | dev->pdev = pdev; | 956 | dev->pdev = pdev; |
| @@ -397,7 +397,7 @@ static ssize_t gpio_virtuser_direction_do_write(struct file *file, · 396 unmodified lines | |||
| 397 | char buf[32], *trimmed; | 397 | char buf[32], *trimmed; |
| 398 | int ret, dir, val = 0; | 398 | int ret, dir, val = 0; |
| 399 | 399 | ||
| 400 | if (count >= sizeof(buf)) | 400 | if (*ppos != 0 || count >= sizeof(buf)) |
| 401 | return -EINVAL; | 401 | return -EINVAL; |
| 402 | 402 | ||
| 403 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, ppos, user_buf, count); | 403 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, ppos, user_buf, count); |
| @@ -622,7 +622,7 @@ static ssize_t gpio_virtuser_consumer_write(struct file *file, · 218 unmodified lines | |||
| 622 | char buf[GPIO_VIRTUSER_NAME_BUF_LEN + 2]; | 622 | char buf[GPIO_VIRTUSER_NAME_BUF_LEN + 2]; |
| 623 | int ret; | 623 | int ret; |
| 624 | 624 | ||
| 625 | if (count >= sizeof(buf)) | 625 | if (*ppos != 0 || count >= sizeof(buf)) |
| 626 | return -EINVAL; | 626 | return -EINVAL; |
| 627 | 627 | ||
| 628 | ret = simple_write_to_buffer(buf, GPIO_VIRTUSER_NAME_BUF_LEN, ppos, | 628 | ret = simple_write_to_buffer(buf, GPIO_VIRTUSER_NAME_BUF_LEN, ppos, |
| @@ -1477,9 +1477,12 @@ gpio_virtuser_device_activate(struct gpio_virtuser_device *dev) · 848 unmodified lines | |||
| 1477 | } | 1477 | } |
| 1478 | 1478 | ||
| 1479 | wait_for_device_probe(); | 1479 | wait_for_device_probe(); |
| 1480 | if (!device_is_bound(&pdev->dev)) { | 1480 | |
| 1481 | ret = -ENXIO; | 1481 | scoped_guard(device, &pdev->dev) { |
| 1482 | goto err_unregister_pdev; | 1482 | if (!device_is_bound(&pdev->dev)) { |
| 1483 | ret = -ENXIO; | ||
| 1484 | goto err_unregister_pdev; | ||
| 1485 | } | ||
| 1483 | } | 1486 | } |
| 1484 | 1487 | ||
| 1485 | dev->pdev = pdev; | 1488 | dev->pdev = pdev; |
| @@ -1184,6 +1184,7 @@ static int gpio_v2_line_flags_validate(u64 flags) · 1183 unmodified lines | |||
| 1184 | static int gpio_v2_line_config_validate(struct gpio_v2_line_config *lc, | 1184 | static int gpio_v2_line_config_validate(struct gpio_v2_line_config *lc, |
| 1185 | unsigned int num_lines) | 1185 | unsigned int num_lines) |
| 1186 | { | 1186 | { |
| 1187 | size_t unused_attrs; | ||
| 1187 | unsigned int i; | 1188 | unsigned int i; |
| 1188 | u64 flags; | 1189 | u64 flags; |
| 1189 | int ret; | 1190 | int ret; |
| @@ -1191,9 +1192,21 @@ static int gpio_v2_line_config_validate(struct gpio_v2_line_config *lc, · 1 unmodified line | |||
| 1191 | if (lc->num_attrs > GPIO_V2_LINE_NUM_ATTRS_MAX) | 1192 | if (lc->num_attrs > GPIO_V2_LINE_NUM_ATTRS_MAX) |
| 1192 | return -EINVAL; | 1193 | return -EINVAL; |
| 1193 | 1194 | ||
| 1195 | unused_attrs = GPIO_V2_LINE_NUM_ATTRS_MAX - lc->num_attrs; | ||
| 1196 | |||
| 1194 | if (!mem_is_zero(lc->padding, sizeof(lc->padding))) | 1197 | if (!mem_is_zero(lc->padding, sizeof(lc->padding))) |
| 1195 | return -EINVAL; | 1198 | return -EINVAL; |
| 1196 | 1199 | ||
| 1200 | for (i = 0; i < lc->num_attrs; i++) { | ||
| 1201 | if (lc->attrs[i].attr.padding != 0) | ||
| 1202 | return -EINVAL; | ||
| 1203 | } | ||
| 1204 | |||
| 1205 | if (unused_attrs) { | ||
| 1206 | if (!mem_is_zero(&lc->attrs[lc->num_attrs], unused_attrs * sizeof(*lc->attrs))) | ||
| 1207 | return -EINVAL; | ||
| 1208 | } | ||
| 1209 | |||
| 1197 | for (i = 0; i < num_lines; i++) { | 1210 | for (i = 0; i < num_lines; i++) { |
| 1198 | flags = gpio_v2_line_config_flags(lc, i); | 1211 | flags = gpio_v2_line_config_flags(lc, i); |
| 1199 | ret = gpio_v2_line_flags_validate(flags); | 1212 | ret = gpio_v2_line_flags_validate(flags); |
| @@ -53,7 +53,7 @@ struct gpio_shared_entry { · 52 unmodified lines | |||
| 53 | unsigned int offset; | 53 | unsigned int offset; |
| 54 | /* Index in the property value array. */ | 54 | /* Index in the property value array. */ |
| 55 | size_t index; | 55 | size_t index; |
| 56 | /* Synchronizes the modification of shared_desc. */ | 56 | /* Synchronizes the modification of shared_desc and offset. */ |
| 57 | struct mutex lock; | 57 | struct mutex lock; |
| 58 | struct gpio_shared_desc *shared_desc; | 58 | struct gpio_shared_desc *shared_desc; |
| 59 | struct kref ref; | 59 | struct kref ref; |
| @@ -598,16 +598,13 @@ void gpio_device_teardown_shared(struct gpio_device *gdev) · 538 unmodified lines | |||
| 598 | struct gpio_shared_ref *ref; | 598 | struct gpio_shared_ref *ref; |
| 599 | 599 | ||
| 600 | list_for_each_entry(entry, &gpio_shared_list, list) { | 600 | list_for_each_entry(entry, &gpio_shared_list, list) { |
| 601 | guard(mutex)(&entry->lock); | ||
| 602 | |||
| 603 | if (!device_match_fwnode(&gdev->dev, entry->fwnode)) | 601 | if (!device_match_fwnode(&gdev->dev, entry->fwnode)) |
| 604 | continue; | 602 | continue; |
| 605 | 603 | ||
| 606 | gpiod_free_commit(&gdev->descs[entry->offset]); | 604 | scoped_guard(mutex, &entry->lock) |
| 605 | gpiod_free_commit(&gdev->descs[entry->offset]); | ||
| 607 | 606 | ||
| 608 | list_for_each_entry(ref, &entry->refs, list) { | 607 | list_for_each_entry(ref, &entry->refs, list) { |
| 609 | guard(mutex)(&ref->lock); | ||
| 610 | |||
| 611 | if (ref->lookup) { | 608 | if (ref->lookup) { |
| 612 | gpiod_remove_lookup_table(ref->lookup); | 609 | gpiod_remove_lookup_table(ref->lookup); |
| 613 | kfree(ref->lookup->table[0].key); | 610 | kfree(ref->lookup->table[0].key); |
| @@ -1190,7 +1190,6 @@ struct amdgpu_device { · 1189 unmodified lines | |||
| 1190 | bool apu_prefer_gtt; | 1190 | bool apu_prefer_gtt; |
| 1191 | 1191 | ||
| 1192 | bool userq_halt_for_enforce_isolation; | 1192 | bool userq_halt_for_enforce_isolation; |
| 1193 | struct work_struct userq_reset_work; | ||
| 1194 | struct amdgpu_uid *uid_info; | 1193 | struct amdgpu_uid *uid_info; |
| 1195 | 1194 | ||
| 1196 | struct amdgpu_uma_carveout_info uma_info; | 1195 | struct amdgpu_uma_carveout_info uma_info; |
| @@ -36,6 +36,9 @@ · 35 unmodified lines | |||
| 36 | #include "amdgpu_ras.h" | 36 | #include "amdgpu_ras.h" |
| 37 | #include "amdgpu_umc.h" | 37 | #include "amdgpu_umc.h" |
| 38 | #include "amdgpu_reset.h" | 38 | #include "amdgpu_reset.h" |
| 39 | #if IS_ENABLED(CONFIG_HSA_AMD) | ||
| 40 | #include "kfd_priv.h" | ||
| 41 | #endif | ||
| 39 | 42 | ||
| 40 | /* Total memory size in system memory and all GPU VRAM. Used to | 43 | /* Total memory size in system memory and all GPU VRAM. Used to |
| 41 | * estimate worst case amount of memory to reserve for page tables | 44 | * estimate worst case amount of memory to reserve for page tables |
| @@ -320,6 +323,28 @@ void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev) · 278 unmodified lines | |||
| 320 | (void)amdgpu_reset_domain_schedule(adev->reset_domain, &adev->kfd.reset_work); | 323 | (void)amdgpu_reset_domain_schedule(adev->reset_domain, &adev->kfd.reset_work); |
| 321 | } | 324 | } |
| 322 | 325 | ||
| 326 | void amdgpu_amdkfd_clear_kfd_mapping(struct amdgpu_device *adev) | ||
| 327 | { | ||
| 328 | #if IS_ENABLED(CONFIG_HSA_AMD) | ||
| 329 | struct kfd_dev *kfd = adev->kfd.dev; | ||
| 330 | unsigned int i; | ||
| 331 | |||
| 332 | if (!kfd) | ||
| 333 | return; | ||
| 334 | |||
| 335 | for (i = 0; i < kfd->num_nodes; i++) { | ||
| 336 | struct kfd_node *node = kfd->nodes[i]; | ||
| 337 | |||
| 338 | kfd_dev_unmap_mapping_range(KFD_MMAP_TYPE_DOORBELL | | ||
| 339 | KFD_MMAP_GPU_ID(node->id), | ||
| 340 | kfd_doorbell_process_slice(kfd)); | ||
| 341 | kfd_dev_unmap_mapping_range(KFD_MMAP_TYPE_MMIO | | ||
| 342 | KFD_MMAP_GPU_ID(node->id), | ||
| 343 | PAGE_SIZE); | ||
| 344 | } | ||
| 345 | #endif | ||
| 346 | } | ||
| 347 | |||
| 323 | int amdgpu_amdkfd_alloc_kernel_mem(struct amdgpu_device *adev, size_t size, | 348 | int amdgpu_amdkfd_alloc_kernel_mem(struct amdgpu_device *adev, size_t size, |
| 324 | u32 domain, void **mem_obj, uint64_t *gpu_addr, | 349 | u32 domain, void **mem_obj, uint64_t *gpu_addr, |
| 325 | void **cpu_ptr, bool cp_mqd_gfx9) | 350 | void **cpu_ptr, bool cp_mqd_gfx9) |
| @@ -358,6 +358,7 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, · 357 unmodified lines | |||
| 358 | uint64_t size, u32 alloc_flag, int8_t xcp_id); | 358 | uint64_t size, u32 alloc_flag, int8_t xcp_id); |
| 359 | void amdgpu_amdkfd_unreserve_mem_limit(struct amdgpu_device *adev, | 359 | void amdgpu_amdkfd_unreserve_mem_limit(struct amdgpu_device *adev, |
| 360 | uint64_t size, u32 alloc_flag, int8_t xcp_id); | 360 | uint64_t size, u32 alloc_flag, int8_t xcp_id); |
| 361 | void amdgpu_amdkfd_clear_kfd_mapping(struct amdgpu_device *adev); | ||
| 361 | 362 | ||
| 362 | u64 amdgpu_amdkfd_xcp_memory_size(struct amdgpu_device *adev, int xcp_id); | 363 | u64 amdgpu_amdkfd_xcp_memory_size(struct amdgpu_device *adev, int xcp_id); |
| 363 | 364 | ||
| @@ -2839,8 +2839,12 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 2838 unmodified lines | |||
| 2839 | * that checks whether the PSP is running. A solution for those issues | 2839 | * that checks whether the PSP is running. A solution for those issues |
| 2840 | * in the APU is to trigger a GPU reset, but this should be done during | 2840 | * in the APU is to trigger a GPU reset, but this should be done during |
| 2841 | * the unload phase to avoid adding boot latency and screen flicker. | 2841 | * the unload phase to avoid adding boot latency and screen flicker. |
| 2842 | * GFX V11 has GC block as default off IP. Every time AMDGPU driver sends | ||
| 2843 | * a request to PMFW to unload MP1, PMFW will put GC in reset and power down | ||
| 2844 | * the voltage. Hence, skipping reset for APUs with GFX V11 or later. | ||
| 2842 | */ | 2845 | */ |
| 2843 | if ((adev->flags & AMD_IS_APU) && !adev->gmc.is_app_apu) { | 2846 | if ((adev->flags & AMD_IS_APU) && !adev->gmc.is_app_apu && |
| 2847 | amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 0, 0)) { | ||
| 2844 | r = amdgpu_asic_reset(adev); | 2848 | r = amdgpu_asic_reset(adev); |
| 2845 | if (r) | 2849 | if (r) |
| 2846 | dev_err(adev->dev, "asic reset on %s failed\n", __func__); | 2850 | dev_err(adev->dev, "asic reset on %s failed\n", __func__); |
| @@ -3783,7 +3787,6 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 936 unmodified lines | |||
| 3783 | } | 3787 | } |
| 3784 | 3788 | ||
| 3785 | INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); | 3789 | INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); |
| 3786 | INIT_WORK(&adev->userq_reset_work, amdgpu_userq_reset_work); | ||
| 3787 | 3790 | ||
| 3788 | amdgpu_coredump_init(adev); | 3791 | amdgpu_coredump_init(adev); |
| 3789 | 3792 | ||
| @@ -5474,7 +5477,7 @@ static inline void amdgpu_device_stop_pending_resets(struct amdgpu_device *adev) · 1684 unmodified lines | |||
| 5474 | if (!amdgpu_sriov_vf(adev)) | 5477 | if (!amdgpu_sriov_vf(adev)) |
| 5475 | cancel_work(&adev->reset_work); | 5478 | cancel_work(&adev->reset_work); |
| 5476 | #endif | 5479 | #endif |
| 5477 | cancel_work(&adev->userq_reset_work); | 5480 | amdgpu_userq_mgr_cancel_reset_work(adev); |
| 5478 | 5481 | ||
| 5479 | if (adev->kfd.dev) | 5482 | if (adev->kfd.dev) |
| 5480 | cancel_work(&adev->kfd.reset_work); | 5483 | cancel_work(&adev->kfd.reset_work); |
| @@ -5832,6 +5835,12 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 351 unmodified lines | |||
| 5832 | /* We need to lock reset domain only once both for XGMI and single device */ | 5835 | /* We need to lock reset domain only once both for XGMI and single device */ |
| 5833 | amdgpu_device_recovery_get_reset_lock(adev, &device_list); | 5836 | amdgpu_device_recovery_get_reset_lock(adev, &device_list); |
| 5834 | 5837 | ||
| 5838 | /* unmap all the mappings of doorbell and framebuffer to prevent user space from | ||
| 5839 | * accessing them | ||
| 5840 | */ | ||
| 5841 | unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); | ||
| 5842 | amdgpu_amdkfd_clear_kfd_mapping(adev); | ||
| 5843 | |||
| 5835 | amdgpu_device_halt_activities(adev, job, reset_context, &device_list, | 5844 | amdgpu_device_halt_activities(adev, job, reset_context, &device_list, |
| 5836 | hive, need_emergency_restart); | 5845 | hive, need_emergency_restart); |
| 5837 | if (need_emergency_restart) | 5846 | if (need_emergency_restart) |
| @@ -304,7 +304,7 @@ static int amdgpu_discovery_get_tmr_info(struct amdgpu_device *adev, · 303 unmodified lines | |||
| 304 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].offset; | 304 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].offset; |
| 305 | adev->discovery.size = | 305 | adev->discovery.size = |
| 306 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].size_kb << 10; | 306 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].size_kb << 10; |
| 307 | if (!adev->discovery.offset || !adev->discovery.size) | 307 | if (!adev->discovery.size) |
| 308 | return -EINVAL; | 308 | return -EINVAL; |
| 309 | } else { | 309 | } else { |
| 310 | goto out; | 310 | goto out; |
| @@ -3090,10 +3090,8 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 2779 unmodified lines | |||
| 3090 | case IP_VERSION(11, 5, 1): | 3090 | case IP_VERSION(11, 5, 1): |
| 3091 | case IP_VERSION(11, 5, 2): | 3091 | case IP_VERSION(11, 5, 2): |
| 3092 | case IP_VERSION(11, 5, 3): | 3092 | case IP_VERSION(11, 5, 3): |
| 3093 | adev->family = AMDGPU_FAMILY_GC_11_5_0; | ||
| 3094 | break; | ||
| 3095 | case IP_VERSION(11, 5, 4): | 3093 | case IP_VERSION(11, 5, 4): |
| 3096 | adev->family = AMDGPU_FAMILY_GC_11_5_4; | 3094 | adev->family = AMDGPU_FAMILY_GC_11_5_0; |
| 3097 | break; | 3095 | break; |
| 3098 | case IP_VERSION(12, 0, 0): | 3096 | case IP_VERSION(12, 0, 0): |
| 3099 | case IP_VERSION(12, 0, 1): | 3097 | case IP_VERSION(12, 0, 1): |
| @@ -3149,17 +3149,15 @@ static int __init amdgpu_init(void) · 3148 unmodified lines | |||
| 3149 | 3149 | ||
| 3150 | r = amdgpu_sync_init(); | 3150 | r = amdgpu_sync_init(); |
| 3151 | if (r) | 3151 | if (r) |
| 3152 | goto error_sync; | 3152 | return r; |
| 3153 | |||
| 3154 | r = amdgpu_userq_fence_slab_init(); | ||
| 3155 | if (r) | ||
| 3156 | goto error_fence; | ||
| 3157 | 3153 | ||
| 3158 | amdgpu_register_atpx_handler(); | 3154 | amdgpu_register_atpx_handler(); |
| 3159 | amdgpu_acpi_detect(); | 3155 | amdgpu_acpi_detect(); |
| 3160 | 3156 | ||
| 3161 | /* Ignore KFD init failures. Normal when CONFIG_HSA_AMD is not set. */ | 3157 | /* Ignore KFD init failures when CONFIG_HSA_AMD is not set. */ |
| 3162 | amdgpu_amdkfd_init(); | 3158 | r = amdgpu_amdkfd_init(); |
| 3159 | if (r && r != -ENOENT) | ||
| 3160 | goto error_fini_sync; | ||
| 3163 | 3161 | ||
| 3164 | if (amdgpu_pp_feature_mask & PP_OVERDRIVE_MASK) { | 3162 | if (amdgpu_pp_feature_mask & PP_OVERDRIVE_MASK) { |
| 3165 | add_taint(TAINT_CPU_OUT_OF_SPEC, LOCKDEP_STILL_OK); | 3163 | add_taint(TAINT_CPU_OUT_OF_SPEC, LOCKDEP_STILL_OK); |
| @@ -3170,10 +3168,8 @@ static int __init amdgpu_init(void) · 4 unmodified lines | |||
| 3170 | /* let modprobe override vga console setting */ | 3168 | /* let modprobe override vga console setting */ |
| 3171 | return pci_register_driver(&amdgpu_kms_pci_driver); | 3169 | return pci_register_driver(&amdgpu_kms_pci_driver); |
| 3172 | 3170 | ||
| 3173 | error_fence: | 3171 | error_fini_sync: |
| 3174 | amdgpu_sync_fini(); | 3172 | amdgpu_sync_fini(); |
| 3175 | |||
| 3176 | error_sync: | ||
| 3177 | return r; | 3173 | return r; |
| 3178 | } | 3174 | } |
| 3179 | 3175 | ||
| @@ -3184,7 +3180,6 @@ static void __exit amdgpu_exit(void) · 4 unmodified lines | |||
| 3184 | amdgpu_unregister_atpx_handler(); | 3180 | amdgpu_unregister_atpx_handler(); |
| 3185 | amdgpu_acpi_release(); | 3181 | amdgpu_acpi_release(); |
| 3186 | amdgpu_sync_fini(); | 3182 | amdgpu_sync_fini(); |
| 3187 | amdgpu_userq_fence_slab_fini(); | ||
| 3188 | mmu_notifier_synchronize(); | 3183 | mmu_notifier_synchronize(); |
| 3189 | amdgpu_xcp_drv_release(); | 3184 | amdgpu_xcp_drv_release(); |
| 3190 | } | 3185 | } |
| @@ -262,12 +262,19 @@ void amdgpu_gart_table_ram_free(struct amdgpu_device *adev) · 261 unmodified lines | |||
| 262 | */ | 262 | */ |
| 263 | int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev) | 263 | int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev) |
| 264 | { | 264 | { |
| 265 | int r; | ||
| 266 | |||
| 265 | if (adev->gart.bo != NULL) | 267 | if (adev->gart.bo != NULL) |
| 266 | return 0; | 268 | return 0; |
| 267 | 269 | ||
| 268 | return amdgpu_bo_create_kernel(adev, adev->gart.table_size, PAGE_SIZE, | 270 | r = amdgpu_bo_create_kernel(adev, adev->gart.table_size, PAGE_SIZE, |
| 269 | AMDGPU_GEM_DOMAIN_VRAM, &adev->gart.bo, | 271 | AMDGPU_GEM_DOMAIN_VRAM, &adev->gart.bo, |
| 270 | NULL, (void *)&adev->gart.ptr); | 272 | NULL, (void *)&adev->gart.ptr); |
| 273 | if (r) | ||
| 274 | return r; | ||
| 275 | |||
| 276 | memset_io(adev->gart.ptr, adev->gart.gart_pte_flags, adev->gart.table_size); | ||
| 277 | return 0; | ||
| 271 | } | 278 | } |
| 272 | 279 | ||
| 273 | /** | 280 | /** |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | |||
| 31 | #include <linux/pci.h> | 31 | #include <linux/pci.h> |
| 32 | #include <linux/dma-buf.h> | 32 | #include <linux/dma-buf.h> |
| 33 | #include <linux/dma-fence-unwrap.h> | 33 | #include <linux/dma-fence-unwrap.h> |
| 34 | #include <linux/uaccess.h> | ||
| 34 | 35 | ||
| 35 | #include <drm/amdgpu_drm.h> | 36 | #include <drm/amdgpu_drm.h> |
| 36 | #include <drm/drm_drv.h> | 37 | #include <drm/drm_drv.h> |
| @@ -508,6 +509,9 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, · 471 unmodified lines | |||
| 508 | if (offset_in_page(args->addr | args->size)) | 509 | if (offset_in_page(args->addr | args->size)) |
| 509 | return -EINVAL; | 510 | return -EINVAL; |
| 510 | 511 | ||
| 512 | if (!access_ok((void __user *)(uintptr_t)args->addr, args->size)) | ||
| 513 | return -EFAULT; | ||
| 514 | |||
| 511 | /* reject unknown flag values */ | 515 | /* reject unknown flag values */ |
| 512 | if (args->flags & ~(AMDGPU_GEM_USERPTR_READONLY | | 516 | if (args->flags & ~(AMDGPU_GEM_USERPTR_READONLY | |
| 513 | AMDGPU_GEM_USERPTR_ANONONLY | AMDGPU_GEM_USERPTR_VALIDATE | | 517 | AMDGPU_GEM_USERPTR_ANONONLY | AMDGPU_GEM_USERPTR_VALIDATE | |
| @@ -821,7 +825,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 307 unmodified lines | |||
| 821 | struct drm_syncobj *timeline_syncobj = NULL; | 825 | struct drm_syncobj *timeline_syncobj = NULL; |
| 822 | struct dma_fence_chain *timeline_chain = NULL; | 826 | struct dma_fence_chain *timeline_chain = NULL; |
| 823 | struct drm_exec exec; | 827 | struct drm_exec exec; |
| 824 | uint64_t vm_size; | 828 | uint64_t vm_size, tmp; |
| 825 | int r = 0; | 829 | int r = 0; |
| 826 | 830 | ||
| 827 | /* Validate virtual address range against reserved regions. */ | 831 | /* Validate virtual address range against reserved regions. */ |
| @@ -845,7 +849,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 17 unmodified lines | |||
| 845 | 849 | ||
| 846 | vm_size = adev->vm_manager.max_pfn * AMDGPU_GPU_PAGE_SIZE; | 850 | vm_size = adev->vm_manager.max_pfn * AMDGPU_GPU_PAGE_SIZE; |
| 847 | vm_size -= AMDGPU_VA_RESERVED_TOP; | 851 | vm_size -= AMDGPU_VA_RESERVED_TOP; |
| 848 | if (args->va_address + args->map_size > vm_size) { | 852 | if (check_add_overflow(args->va_address, args->map_size, &tmp) || tmp > vm_size) { |
| 849 | dev_dbg(dev->dev, | 853 | dev_dbg(dev->dev, |
| 850 | "va_address 0x%llx is in top reserved area 0x%llx\n", | 854 | "va_address 0x%llx is in top reserved area 0x%llx\n", |
| 851 | args->va_address + args->map_size, vm_size); | 855 | args->va_address + args->map_size, vm_size); |
| @@ -1089,9 +1093,16 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, · 237 unmodified lines | |||
| 1089 | * If that number is larger than the size of the array, the ioctl must | 1093 | * If that number is larger than the size of the array, the ioctl must |
| 1090 | * be retried. | 1094 | * be retried. |
| 1091 | */ | 1095 | */ |
| 1096 | if (args->num_entries > INT_MAX / sizeof(*vm_entries)) { | ||
| 1097 | r = -EINVAL; | ||
| 1098 | goto out_exec; | ||
| 1099 | } | ||
| 1100 | |||
| 1092 | vm_entries = kvcalloc(args->num_entries, sizeof(*vm_entries), GFP_KERNEL); | 1101 | vm_entries = kvcalloc(args->num_entries, sizeof(*vm_entries), GFP_KERNEL); |
| 1093 | if (!vm_entries) | 1102 | if (!vm_entries) { |
| 1094 | return -ENOMEM; | 1103 | r = -ENOMEM; |
| 1104 | goto out_exec; | ||
| 1105 | } | ||
| 1095 | 1106 | ||
| 1096 | amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) { | 1107 | amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) { |
| 1097 | if (num_mappings < args->num_entries) { | 1108 | if (num_mappings < args->num_entries) { |
| @@ -314,7 +314,10 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc, · 313 unmodified lines | |||
| 314 | mc->gart_start = max_mc_address - mc->gart_size + 1; | 314 | mc->gart_start = max_mc_address - mc->gart_size + 1; |
| 315 | break; | 315 | break; |
| 316 | case AMDGPU_GART_PLACEMENT_LOW: | 316 | case AMDGPU_GART_PLACEMENT_LOW: |
| 317 | mc->gart_start = 0; | 317 | if (size_bf >= mc->gart_size) |
| 318 | mc->gart_start = 0; | ||
| 319 | else | ||
| 320 | mc->gart_start = ALIGN(mc->fb_end, four_gb); | ||
| 318 | break; | 321 | break; |
| 319 | case AMDGPU_GART_PLACEMENT_BEST_FIT: | 322 | case AMDGPU_GART_PLACEMENT_BEST_FIT: |
| 320 | default: | 323 | default: |
| @@ -199,11 +199,18 @@ int amdgpu_gtt_mgr_alloc_entries(struct amdgpu_gtt_mgr *mgr, · 198 unmodified lines | |||
| 199 | enum drm_mm_insert_mode mode) | 199 | enum drm_mm_insert_mode mode) |
| 200 | { | 200 | { |
| 201 | struct amdgpu_device *adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); | 201 | struct amdgpu_device *adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); |
| 202 | u32 alignment = 0; | ||
| 202 | int r; | 203 | int r; |
| 203 | 204 | ||
| 205 | /* Align to TLB L2 cache entry size to work around "V bit HW bug" */ | ||
| 206 | if (adev->asic_type == CHIP_TAHITI) { | ||
| 207 | alignment = 32 * 1024 / AMDGPU_GPU_PAGE_SIZE; | ||
| 208 | num_pages = ALIGN(num_pages, alignment); | ||
| 209 | } | ||
| 210 | |||
| 204 | spin_lock(&mgr->lock); | 211 | spin_lock(&mgr->lock); |
| 205 | r = drm_mm_insert_node_in_range(&mgr->mm, mm_node, num_pages, | 212 | r = drm_mm_insert_node_in_range(&mgr->mm, mm_node, num_pages, |
| 206 | 0, GART_ENTRY_WITHOUT_BO_COLOR, 0, | 213 | alignment, GART_ENTRY_WITHOUT_BO_COLOR, 0, |
| 207 | adev->gmc.gart_size >> PAGE_SHIFT, | 214 | adev->gmc.gart_size >> PAGE_SHIFT, |
| 208 | mode); | 215 | mode); |
| 209 | spin_unlock(&mgr->lock); | 216 | spin_unlock(&mgr->lock); |
| @@ -51,8 +51,6 @@ · 50 unmodified lines | |||
| 51 | #include "amdgpu_amdkfd.h" | 51 | #include "amdgpu_amdkfd.h" |
| 52 | #include "amdgpu_hmm.h" | 52 | #include "amdgpu_hmm.h" |
| 53 | 53 | ||
| 54 | #define MAX_WALK_BYTE (2UL << 30) | ||
| 55 | |||
| 56 | /** | 54 | /** |
| 57 | * amdgpu_hmm_invalidate_gfx - callback to notify about mm change | 55 | * amdgpu_hmm_invalidate_gfx - callback to notify about mm change |
| 58 | * | 56 | * |
| @@ -78,6 +76,7 @@ static bool amdgpu_hmm_invalidate_gfx(struct mmu_interval_notifier *mni, · 19 unmodified lines | |||
| 78 | 76 | ||
| 79 | mmu_interval_set_seq(mni, cur_seq); | 77 | mmu_interval_set_seq(mni, cur_seq); |
| 80 | 78 | ||
| 79 | amdgpu_vm_bo_invalidate(bo, false); | ||
| 81 | r = dma_resv_wait_timeout(bo->tbo.base.resv, DMA_RESV_USAGE_BOOKKEEP, | 80 | r = dma_resv_wait_timeout(bo->tbo.base.resv, DMA_RESV_USAGE_BOOKKEEP, |
| 82 | false, MAX_SCHEDULE_TIMEOUT); | 81 | false, MAX_SCHEDULE_TIMEOUT); |
| 83 | mutex_unlock(&adev->notifier_lock); | 82 | mutex_unlock(&adev->notifier_lock); |
| @@ -170,11 +169,13 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 86 unmodified lines | |||
| 170 | void *owner, | 169 | void *owner, |
| 171 | struct amdgpu_hmm_range *range) | 170 | struct amdgpu_hmm_range *range) |
| 172 | { | 171 | { |
| 173 | unsigned long end; | 172 | const u64 max_bytes = SZ_2G; |
| 173 | |||
| 174 | struct hmm_range *hmm_range = &range->hmm_range; | ||
| 174 | unsigned long timeout; | 175 | unsigned long timeout; |
| 175 | unsigned long *pfns; | 176 | unsigned long *pfns; |
| 176 | int r = 0; | 177 | unsigned long end; |
| 177 | struct hmm_range *hmm_range = &range->hmm_range; | 178 | int r; |
| 178 | 179 | ||
| 179 | pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL); | 180 | pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL); |
| 180 | if (unlikely(!pfns)) { | 181 | if (unlikely(!pfns)) { |
| @@ -191,8 +192,9 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 10 unmodified lines | |||
| 191 | end = start + npages * PAGE_SIZE; | 192 | end = start + npages * PAGE_SIZE; |
| 192 | hmm_range->dev_private_owner = owner; | 193 | hmm_range->dev_private_owner = owner; |
| 193 | 194 | ||
| 195 | hmm_range->notifier_seq = mmu_interval_read_begin(notifier); | ||
| 194 | do { | 196 | do { |
| 195 | hmm_range->end = min(hmm_range->start + MAX_WALK_BYTE, end); | 197 | hmm_range->end = min(hmm_range->start + max_bytes, end); |
| 196 | 198 | ||
| 197 | pr_debug("hmm range: start = 0x%lx, end = 0x%lx", | 199 | pr_debug("hmm range: start = 0x%lx, end = 0x%lx", |
| 198 | hmm_range->start, hmm_range->end); | 200 | hmm_range->start, hmm_range->end); |
| @@ -200,7 +202,6 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 1 unmodified line | |||
| 200 | timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); | 202 | timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); |
| 201 | 203 | ||
| 202 | retry: | 204 | retry: |
| 203 | hmm_range->notifier_seq = mmu_interval_read_begin(notifier); | ||
| 204 | r = hmm_range_fault(hmm_range); | 205 | r = hmm_range_fault(hmm_range); |
| 205 | if (unlikely(r)) { | 206 | if (unlikely(r)) { |
| 206 | if (r == -EBUSY && !time_after(jiffies, timeout)) | 207 | if (r == -EBUSY && !time_after(jiffies, timeout)) |
| @@ -210,7 +211,7 @@ retry: · 3 unmodified lines | |||
| 210 | 211 | ||
| 211 | if (hmm_range->end == end) | 212 | if (hmm_range->end == end) |
| 212 | break; | 213 | break; |
| 213 | hmm_range->hmm_pfns += MAX_WALK_BYTE >> PAGE_SHIFT; | 214 | hmm_range->hmm_pfns += max_bytes >> PAGE_SHIFT; |
| 214 | hmm_range->start = hmm_range->end; | 215 | hmm_range->start = hmm_range->end; |
| 215 | } while (hmm_range->end < end); | 216 | } while (hmm_range->end < end); |
| 216 | 217 | ||
| @@ -873,68 +873,59 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 872 unmodified lines | |||
| 873 | ? -EFAULT : 0; | 873 | ? -EFAULT : 0; |
| 874 | } | 874 | } |
| 875 | case AMDGPU_INFO_READ_MMR_REG: { | 875 | case AMDGPU_INFO_READ_MMR_REG: { |
| 876 | int ret = 0; | ||
| 877 | unsigned int n, alloc_size; | ||
| 878 | uint32_t *regs; | ||
| 879 | unsigned int se_num = (info->read_mmr_reg.instance >> | 876 | unsigned int se_num = (info->read_mmr_reg.instance >> |
| 880 | AMDGPU_INFO_MMR_SE_INDEX_SHIFT) & | 877 | AMDGPU_INFO_MMR_SE_INDEX_SHIFT) & |
| 881 | AMDGPU_INFO_MMR_SE_INDEX_MASK; | 878 | AMDGPU_INFO_MMR_SE_INDEX_MASK; |
| 882 | unsigned int sh_num = (info->read_mmr_reg.instance >> | 879 | unsigned int sh_num = (info->read_mmr_reg.instance >> |
| 883 | AMDGPU_INFO_MMR_SH_INDEX_SHIFT) & | 880 | AMDGPU_INFO_MMR_SH_INDEX_SHIFT) & |
| 884 | AMDGPU_INFO_MMR_SH_INDEX_MASK; | 881 | AMDGPU_INFO_MMR_SH_INDEX_MASK; |
| 885 | 882 | unsigned int alloc_size; | |
| 886 | if (!down_read_trylock(&adev->reset_domain->sem)) | 883 | uint32_t *regs; |
| 887 | return -ENOENT; | 884 | int ret; |
| 888 | 885 | ||
| 889 | /* set full masks if the userspace set all bits | 886 | /* set full masks if the userspace set all bits |
| 890 | * in the bitfields | 887 | * in the bitfields |
| 891 | */ | 888 | */ |
| 892 | if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK) { | 889 | if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK) |
| 893 | se_num = 0xffffffff; | 890 | se_num = 0xffffffff; |
| 894 | } else if (se_num >= AMDGPU_GFX_MAX_SE) { | 891 | else if (se_num >= AMDGPU_GFX_MAX_SE) |
| 895 | ret = -EINVAL; | 892 | return -EINVAL; |
| 896 | goto out; | ||
| 897 | } | ||
| 898 | 893 | ||
| 899 | if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK) { | 894 | if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK) |
| 900 | sh_num = 0xffffffff; | 895 | sh_num = 0xffffffff; |
| 901 | } else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE) { | 896 | else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE) |
| 902 | ret = -EINVAL; | 897 | return -EINVAL; |
| 903 | goto out; | ||
| 904 | } | ||
| 905 | 898 | ||
| 906 | if (info->read_mmr_reg.count > 128) { | 899 | if (info->read_mmr_reg.count > 128) |
| 907 | ret = -EINVAL; | 900 | return -EINVAL; |
| 908 | goto out; | ||
| 909 | } | ||
| 910 | 901 | ||
| 911 | regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs), GFP_KERNEL); | 902 | regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs), |
| 912 | if (!regs) { | 903 | GFP_KERNEL); |
| 913 | ret = -ENOMEM; | 904 | if (!regs) |
| 914 | goto out; | 905 | return -ENOMEM; |
| 915 | } | ||
| 916 | 906 | ||
| 907 | down_read(&adev->reset_domain->sem); | ||
| 917 | alloc_size = info->read_mmr_reg.count * sizeof(*regs); | 908 | alloc_size = info->read_mmr_reg.count * sizeof(*regs); |
| 918 | |||
| 919 | amdgpu_gfx_off_ctrl(adev, false); | 909 | amdgpu_gfx_off_ctrl(adev, false); |
| 910 | ret = 0; | ||
| 920 | for (i = 0; i < info->read_mmr_reg.count; i++) { | 911 | for (i = 0; i < info->read_mmr_reg.count; i++) { |
| 921 | if (amdgpu_asic_read_register(adev, se_num, sh_num, | 912 | if (amdgpu_asic_read_register(adev, se_num, sh_num, |
| 922 | info->read_mmr_reg.dword_offset + i, | 913 | info->read_mmr_reg.dword_offset + i, |
| 923 | ®s[i])) { | 914 | ®s[i])) { |
| 924 | DRM_DEBUG_KMS("unallowed offset %#x\n", | 915 | DRM_DEBUG_KMS("unallowed offset %#x\n", |
| 925 | info->read_mmr_reg.dword_offset + i); | 916 | info->read_mmr_reg.dword_offset + i); |
| 926 | kfree(regs); | ||
| 927 | amdgpu_gfx_off_ctrl(adev, true); | ||
| 928 | ret = -EFAULT; | 917 | ret = -EFAULT; |
| 929 | goto out; | 918 | break; |
| 930 | } | 919 | } |
| 931 | } | 920 | } |
| 932 | amdgpu_gfx_off_ctrl(adev, true); | 921 | amdgpu_gfx_off_ctrl(adev, true); |
| 933 | n = copy_to_user(out, regs, min(size, alloc_size)); | ||
| 934 | kfree(regs); | ||
| 935 | ret = (n ? -EFAULT : 0); | ||
| 936 | out: | ||
| 937 | up_read(&adev->reset_domain->sem); | 922 | up_read(&adev->reset_domain->sem); |
| 923 | |||
| 924 | if (!ret) { | ||
| 925 | ret = copy_to_user(out, regs, min(size, alloc_size)) | ||
| 926 | ? -EFAULT : 0; | ||
| 927 | } | ||
| 928 | kfree(regs); | ||
| 938 | return ret; | 929 | return ret; |
| 939 | } | 930 | } |
| 940 | case AMDGPU_INFO_DEV_INFO: { | 931 | case AMDGPU_INFO_DEV_INFO: { |
| @@ -96,7 +96,8 @@ struct amdgpu_bo_va { · 95 unmodified lines | |||
| 96 | * if non-zero, cannot unmap from GPU because user queues may still access it | 96 | * if non-zero, cannot unmap from GPU because user queues may still access it |
| 97 | */ | 97 | */ |
| 98 | unsigned int queue_refcount; | 98 | unsigned int queue_refcount; |
| 99 | atomic_t userq_va_mapped; | 99 | /* Indicates if this buffer is mapped for any user queue. Once set, never reset. */ |
| 100 | bool userq_va_mapped; | ||
| 100 | }; | 101 | }; |
| 101 | 102 | ||
| 102 | struct amdgpu_bo { | 103 | struct amdgpu_bo { |
| @@ -2280,7 +2280,8 @@ void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev) · 2279 unmodified lines | |||
| 2280 | list_for_each_entry(obj, &con->head, node) { | 2280 | list_for_each_entry(obj, &con->head, node) { |
| 2281 | if (amdgpu_ras_is_supported(adev, obj->head.block) && | 2281 | if (amdgpu_ras_is_supported(adev, obj->head.block) && |
| 2282 | (obj->attr_inuse == 1)) { | 2282 | (obj->attr_inuse == 1)) { |
| 2283 | sprintf(fs_info.debugfs_name, "%s_err_inject", | 2283 | snprintf(fs_info.debugfs_name, sizeof(fs_info.debugfs_name), |
| 2284 | "%s_err_inject", | ||
| 2284 | get_ras_block_str(&obj->head)); | 2285 | get_ras_block_str(&obj->head)); |
| 2285 | fs_info.head = obj->head; | 2286 | fs_info.head = obj->head; |
| 2286 | amdgpu_ras_debugfs_create(adev, &fs_info, dir); | 2287 | amdgpu_ras_debugfs_create(adev, &fs_info, dir); |
| @@ -1950,7 +1950,7 @@ void amdgpu_ras_check_bad_page_status(struct amdgpu_device *adev) · 1949 unmodified lines | |||
| 1950 | if (!control || amdgpu_bad_page_threshold == 0) | 1950 | if (!control || amdgpu_bad_page_threshold == 0) |
| 1951 | return; | 1951 | return; |
| 1952 | 1952 | ||
| 1953 | if (control->ras_num_bad_pages >= ras->bad_page_cnt_threshold) { | 1953 | if (control->ras_num_bad_pages > ras->bad_page_cnt_threshold) { |
| 1954 | if (amdgpu_dpm_send_rma_reason(adev)) | 1954 | if (amdgpu_dpm_send_rma_reason(adev)) |
| 1955 | dev_warn(adev->dev, "Unable to send out-of-band RMA CPER"); | 1955 | dev_warn(adev->dev, "Unable to send out-of-band RMA CPER"); |
| 1956 | else | 1956 | else |
| @@ -552,8 +552,9 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 551 unmodified lines | |||
| 552 | size_t size, loff_t *pos) | 552 | size_t size, loff_t *pos) |
| 553 | { | 553 | { |
| 554 | struct amdgpu_ring *ring = file_inode(f)->i_private; | 554 | struct amdgpu_ring *ring = file_inode(f)->i_private; |
| 555 | uint32_t value, result, early[3]; | 555 | u32 value, result, early[3] = { 0 }; |
| 556 | uint64_t p; | 556 | uint64_t p; |
| 557 | u32 avail_dw, start_dw, read_dw; | ||
| 557 | loff_t i; | 558 | loff_t i; |
| 558 | int r; | 559 | int r; |
| 559 | 560 | ||
| @@ -565,10 +566,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 5 unmodified lines | |||
| 565 | 566 | ||
| 566 | result = 0; | 567 | result = 0; |
| 567 | 568 | ||
| 568 | if (*pos < 12) { | 569 | if (ring->funcs->type == AMDGPU_RING_TYPE_CPER) |
| 569 | if (ring->funcs->type == AMDGPU_RING_TYPE_CPER) | 570 | mutex_lock(&ring->adev->cper.ring_lock); |
| 570 | mutex_lock(&ring->adev->cper.ring_lock); | ||
| 571 | 571 | ||
| 572 | if (*pos < 12) { | ||
| 572 | early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask; | 573 | early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask; |
| 573 | early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask; | 574 | early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask; |
| 574 | early[2] = ring->wptr & ring->buf_mask; | 575 | early[2] = ring->wptr & ring->buf_mask; |
| @@ -600,13 +601,24 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 25 unmodified lines | |||
| 600 | *pos += 4; | 601 | *pos += 4; |
| 601 | } | 602 | } |
| 602 | } else { | 603 | } else { |
| 604 | early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask; | ||
| 605 | early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask; | ||
| 606 | |||
| 603 | p = early[0]; | 607 | p = early[0]; |
| 604 | if (early[0] <= early[1]) | 608 | if (early[0] <= early[1]) |
| 605 | size = (early[1] - early[0]); | 609 | avail_dw = early[1] - early[0]; |
| 606 | else | 610 | else |
| 607 | size = ring->ring_size - (early[0] - early[1]); | 611 | avail_dw = ring->buf_mask + 1 - (early[0] - early[1]); |
| 608 | 612 | ||
| 609 | while (size) { | 613 | start_dw = (*pos > 12) ? ((*pos - 12) >> 2) : 0; |
| 614 | if (start_dw >= avail_dw) | ||
| 615 | goto out; | ||
| 616 | |||
| 617 | p = (p + start_dw) & ring->ptr_mask; | ||
| 618 | avail_dw -= start_dw; | ||
| 619 | read_dw = min_t(u32, avail_dw, size >> 2); | ||
| 620 | |||
| 621 | while (read_dw) { | ||
| 610 | if (p == early[1]) | 622 | if (p == early[1]) |
| 611 | goto out; | 623 | goto out; |
| 612 | 624 | ||
| @@ -619,9 +631,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 6 unmodified lines | |||
| 619 | 631 | ||
| 620 | buf += 4; | 632 | buf += 4; |
| 621 | result += 4; | 633 | result += 4; |
| 622 | size--; | 634 | read_dw--; |
| 623 | p++; | 635 | p++; |
| 624 | p &= ring->ptr_mask; | 636 | p &= ring->ptr_mask; |
| 637 | *pos += 4; | ||
| 625 | } | 638 | } |
| 626 | } | 639 | } |
| 627 | 640 | ||
| @@ -175,11 +175,14 @@ int amdgpu_seq64_alloc(struct amdgpu_device *adev, u64 *va, · 174 unmodified lines | |||
| 175 | { | 175 | { |
| 176 | unsigned long bit_pos; | 176 | unsigned long bit_pos; |
| 177 | 177 | ||
| 178 | bit_pos = find_first_zero_bit(adev->seq64.used, adev->seq64.num_sem); | 178 | for (;;) { |
| 179 | if (bit_pos >= adev->seq64.num_sem) | 179 | bit_pos = find_first_zero_bit(adev->seq64.used, adev->seq64.num_sem); |
| 180 | return -ENOSPC; | 180 | if (bit_pos >= adev->seq64.num_sem) |
| 181 | return -ENOSPC; | ||
| 181 | 182 | ||
| 182 | __set_bit(bit_pos, adev->seq64.used); | 183 | if (!test_and_set_bit(bit_pos, adev->seq64.used)) |
| 184 | break; | ||
| 185 | } | ||
| 183 | 186 | ||
| 184 | *va = bit_pos * sizeof(u64) + amdgpu_seq64_get_va_base(adev); | 187 | *va = bit_pos * sizeof(u64) + amdgpu_seq64_get_va_base(adev); |
| 185 | 188 | ||
| @@ -205,7 +208,7 @@ void amdgpu_seq64_free(struct amdgpu_device *adev, u64 va) · 19 unmodified lines | |||
| 205 | 208 | ||
| 206 | bit_pos = (va - amdgpu_seq64_get_va_base(adev)) / sizeof(u64); | 209 | bit_pos = (va - amdgpu_seq64_get_va_base(adev)) / sizeof(u64); |
| 207 | if (bit_pos < adev->seq64.num_sem) | 210 | if (bit_pos < adev->seq64.num_sem) |
| 208 | __clear_bit(bit_pos, adev->seq64.used); | 211 | clear_bit(bit_pos, adev->seq64.used); |
| 209 | } | 212 | } |
| 210 | 213 | ||
| 211 | /** | 214 | /** |
| @@ -75,6 +75,9 @@ static int amdgpu_ttm_init_on_chip(struct amdgpu_device *adev, · 74 unmodified lines | |||
| 75 | unsigned int type, | 75 | unsigned int type, |
| 76 | uint64_t size_in_page) | 76 | uint64_t size_in_page) |
| 77 | { | 77 | { |
| 78 | if (!size_in_page) | ||
| 79 | return 0; | ||
| 80 | |||
| 78 | return ttm_range_man_init(&adev->mman.bdev, type, | 81 | return ttm_range_man_init(&adev->mman.bdev, type, |
| 79 | false, size_in_page); | 82 | false, size_in_page); |
| 80 | } | 83 | } |
| @@ -130,6 +130,7 @@ void amdgpu_umc_handle_bad_pages(struct amdgpu_device *adev, · 129 unmodified lines | |||
| 130 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && | 130 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 131 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address && | 131 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address && |
| 132 | adev->umc.max_ras_err_cnt_per_query) { | 132 | adev->umc.max_ras_err_cnt_per_query) { |
| 133 | kfree(err_data->err_addr); | ||
| 133 | err_data->err_addr = | 134 | err_data->err_addr = |
| 134 | kzalloc_objs(struct eeprom_table_record, | 135 | kzalloc_objs(struct eeprom_table_record, |
| 135 | adev->umc.max_ras_err_cnt_per_query); | 136 | adev->umc.max_ras_err_cnt_per_query); |
| @@ -160,6 +161,7 @@ void amdgpu_umc_handle_bad_pages(struct amdgpu_device *adev, · 24 unmodified lines | |||
| 160 | if (adev->umc.ras && | 161 | if (adev->umc.ras && |
| 161 | adev->umc.ras->ecc_info_query_ras_error_address && | 162 | adev->umc.ras->ecc_info_query_ras_error_address && |
| 162 | adev->umc.max_ras_err_cnt_per_query) { | 163 | adev->umc.max_ras_err_cnt_per_query) { |
| 164 | kfree(err_data->err_addr); | ||
| 163 | err_data->err_addr = | 165 | err_data->err_addr = |
| 164 | kzalloc_objs(struct eeprom_table_record, | 166 | kzalloc_objs(struct eeprom_table_record, |
| 165 | adev->umc.max_ras_err_cnt_per_query); | 167 | adev->umc.max_ras_err_cnt_per_query); |
| @@ -82,19 +82,11 @@ static bool amdgpu_userq_is_reset_type_supported(struct amdgpu_device *adev, · 81 unmodified lines | |||
| 82 | return false; | 82 | return false; |
| 83 | } | 83 | } |
| 84 | 84 | ||
| 85 | static void amdgpu_userq_gpu_reset(struct amdgpu_device *adev) | 85 | static void amdgpu_userq_mgr_reset_work(struct work_struct *work) |
| 86 | { | ||
| 87 | if (amdgpu_device_should_recover_gpu(adev)) { | ||
| 88 | amdgpu_reset_domain_schedule(adev->reset_domain, | ||
| 89 | &adev->userq_reset_work); | ||
| 90 | /* Wait for the reset job to complete */ | ||
| 91 | flush_work(&adev->userq_reset_work); | ||
| 92 | } | ||
| 93 | } | ||
| 94 | |||
| 95 | static int | ||
| 96 | amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) | ||
| 97 | { | 86 | { |
| 87 | struct amdgpu_userq_mgr *uq_mgr = | ||
| 88 | container_of(work, struct amdgpu_userq_mgr, | ||
| 89 | reset_work); | ||
| 98 | struct amdgpu_device *adev = uq_mgr->adev; | 90 | struct amdgpu_device *adev = uq_mgr->adev; |
| 99 | const int queue_types[] = { | 91 | const int queue_types[] = { |
| 100 | AMDGPU_RING_TYPE_COMPUTE, | 92 | AMDGPU_RING_TYPE_COMPUTE, |
| @@ -103,15 +95,11 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 2 unmodified lines | |||
| 103 | }; | 95 | }; |
| 104 | const int num_queue_types = ARRAY_SIZE(queue_types); | 96 | const int num_queue_types = ARRAY_SIZE(queue_types); |
| 105 | bool gpu_reset = false; | 97 | bool gpu_reset = false; |
| 106 | int r = 0; | 98 | int i, r; |
| 107 | int i; | ||
| 108 | |||
| 109 | /* Warning if current process mutex is not held */ | ||
| 110 | WARN_ON(!mutex_is_locked(&uq_mgr->userq_mutex)); | ||
| 111 | 99 | ||
| 112 | if (unlikely(adev->debug_disable_gpu_ring_reset)) { | 100 | if (unlikely(adev->debug_disable_gpu_ring_reset)) { |
| 113 | dev_err(adev->dev, "userq reset disabled by debug mask\n"); | 101 | dev_err(adev->dev, "userq reset disabled by debug mask\n"); |
| 114 | return 0; | 102 | return; |
| 115 | } | 103 | } |
| 116 | 104 | ||
| 117 | /* | 105 | /* |
| @@ -119,7 +107,7 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | |||
| 119 | * skip all reset detection logic | 107 | * skip all reset detection logic |
| 120 | */ | 108 | */ |
| 121 | if (!amdgpu_gpu_recovery) | 109 | if (!amdgpu_gpu_recovery) |
| 122 | return 0; | 110 | return; |
| 123 | 111 | ||
| 124 | /* | 112 | /* |
| 125 | * Iterate through all queue types to detect and reset problematic queues | 113 | * Iterate through all queue types to detect and reset problematic queues |
| @@ -127,9 +115,11 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | |||
| 127 | */ | 115 | */ |
| 128 | for (i = 0; i < num_queue_types; i++) { | 116 | for (i = 0; i < num_queue_types; i++) { |
| 129 | int ring_type = queue_types[i]; | 117 | int ring_type = queue_types[i]; |
| 130 | const struct amdgpu_userq_funcs *funcs = adev->userq_funcs[ring_type]; | 118 | const struct amdgpu_userq_funcs *funcs = |
| 119 | adev->userq_funcs[ring_type]; | ||
| 131 | 120 | ||
| 132 | if (!amdgpu_userq_is_reset_type_supported(adev, ring_type, AMDGPU_RESET_TYPE_PER_QUEUE)) | 121 | if (!amdgpu_userq_is_reset_type_supported(adev, ring_type, |
| 122 | AMDGPU_RESET_TYPE_PER_QUEUE)) | ||
| 133 | continue; | 123 | continue; |
| 134 | 124 | ||
| 135 | if (atomic_read(&uq_mgr->userq_count[ring_type]) > 0 && | 125 | if (atomic_read(&uq_mgr->userq_count[ring_type]) > 0 && |
| @@ -142,46 +132,43 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 6 unmodified lines | |||
| 142 | } | 132 | } |
| 143 | } | 133 | } |
| 144 | 134 | ||
| 145 | if (gpu_reset) | 135 | if (gpu_reset) { |
| 146 | amdgpu_userq_gpu_reset(adev); | 136 | struct amdgpu_reset_context reset_context; |
| 147 | 137 | ||
| 148 | return r; | 138 | memset(&reset_context, 0, sizeof(reset_context)); |
| 139 | |||
| 140 | reset_context.method = AMD_RESET_METHOD_NONE; | ||
| 141 | reset_context.reset_req_dev = adev; | ||
| 142 | reset_context.src = AMDGPU_RESET_SRC_USERQ; | ||
| 143 | set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | ||
| 144 | /*set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags);*/ | ||
| 145 | |||
| 146 | amdgpu_device_gpu_recover(adev, NULL, &reset_context); | ||
| 147 | } | ||
| 149 | } | 148 | } |
| 150 | 149 | ||
| 151 | static void amdgpu_userq_hang_detect_work(struct work_struct *work) | 150 | static void amdgpu_userq_hang_detect_work(struct work_struct *work) |
| 152 | { | 151 | { |
| 153 | struct amdgpu_usermode_queue *queue = container_of(work, | 152 | struct amdgpu_usermode_queue *queue = |
| 154 | struct amdgpu_usermode_queue, | 153 | container_of(work, struct amdgpu_usermode_queue, |
| 155 | hang_detect_work.work); | 154 | hang_detect_work.work); |
| 156 | struct dma_fence *fence; | ||
| 157 | struct amdgpu_userq_mgr *uq_mgr; | ||
| 158 | |||
| 159 | if (!queue->userq_mgr) | ||
| 160 | return; | ||
| 161 | |||
| 162 | uq_mgr = queue->userq_mgr; | ||
| 163 | fence = READ_ONCE(queue->hang_detect_fence); | ||
| 164 | /* Fence already signaled – no action needed */ | ||
| 165 | if (!fence || dma_fence_is_signaled(fence)) | ||
| 166 | return; | ||
| 167 | 155 | ||
| 168 | mutex_lock(&uq_mgr->userq_mutex); | 156 | /* |
| 169 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | 157 | * Don't schedule the work here! Scheduling or queue work from one reset |
| 170 | mutex_unlock(&uq_mgr->userq_mutex); | 158 | * handler to another is illegal if you don't take extra precautions! |
| 159 | */ | ||
| 160 | amdgpu_userq_mgr_reset_work(&queue->userq_mgr->reset_work); | ||
| 171 | } | 161 | } |
| 172 | 162 | ||
| 173 | /* | 163 | /* |
| 174 | * Start hang detection for a user queue fence. A delayed work will be scheduled | 164 | * Start hang detection for a user queue fence. A delayed work will be scheduled |
| 175 | * to check if the fence is still pending after the timeout period. | 165 | * to reset the queues when the fence doesn't signal in time. |
| 176 | */ | 166 | */ |
| 177 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue) | 167 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue) |
| 178 | { | 168 | { |
| 179 | struct amdgpu_device *adev; | 169 | struct amdgpu_device *adev; |
| 180 | unsigned long timeout_ms; | 170 | unsigned long timeout_ms; |
| 181 | 171 | ||
| 182 | if (!queue || !queue->userq_mgr || !queue->userq_mgr->adev) | ||
| 183 | return; | ||
| 184 | |||
| 185 | adev = queue->userq_mgr->adev; | 172 | adev = queue->userq_mgr->adev; |
| 186 | /* Determine timeout based on queue type */ | 173 | /* Determine timeout based on queue type */ |
| 187 | switch (queue->queue_type) { | 174 | switch (queue->queue_type) { |
| @@ -199,45 +186,44 @@ void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue) · 11 unmodified lines | |||
| 199 | break; | 186 | break; |
| 200 | } | 187 | } |
| 201 | 188 | ||
| 202 | /* Store the fence to monitor and schedule hang detection */ | 189 | queue_delayed_work(adev->reset_domain->wq, &queue->hang_detect_work, |
| 203 | WRITE_ONCE(queue->hang_detect_fence, queue->last_fence); | 190 | msecs_to_jiffies(timeout_ms)); |
| 204 | schedule_delayed_work(&queue->hang_detect_work, | ||
| 205 | msecs_to_jiffies(timeout_ms)); | ||
| 206 | } | ||
| 207 | |||
| 208 | static void amdgpu_userq_init_hang_detect_work(struct amdgpu_usermode_queue *queue) | ||
| 209 | { | ||
| 210 | INIT_DELAYED_WORK(&queue->hang_detect_work, amdgpu_userq_hang_detect_work); | ||
| 211 | queue->hang_detect_fence = NULL; | ||
| 212 | } | 191 | } |
| 213 | 192 | ||
| 214 | static int amdgpu_userq_buffer_va_list_add(struct amdgpu_usermode_queue *queue, | 193 | void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell) |
| 215 | struct amdgpu_bo_va_mapping *va_map, u64 addr) | ||
| 216 | { | 194 | { |
| 217 | struct amdgpu_userq_va_cursor *va_cursor; | 195 | struct xarray *xa = &adev->userq_doorbell_xa; |
| 218 | struct userq_va_list; | 196 | struct amdgpu_usermode_queue *queue; |
| 219 | 197 | unsigned long flags; | |
| 220 | va_cursor = kzalloc_obj(*va_cursor); | 198 | int r; |
| 221 | if (!va_cursor) | ||
| 222 | return -ENOMEM; | ||
| 223 | 199 | ||
| 224 | INIT_LIST_HEAD(&va_cursor->list); | 200 | xa_lock_irqsave(xa, flags); |
| 225 | va_cursor->gpu_addr = addr; | 201 | queue = xa_load(xa, doorbell); |
| 226 | atomic_set(&va_map->bo_va->userq_va_mapped, 1); | 202 | if (queue) { |
| 227 | list_add(&va_cursor->list, &queue->userq_va_list); | 203 | r = amdgpu_userq_fence_driver_process(queue->fence_drv); |
| 204 | /* | ||
| 205 | * We are in interrupt context here, this *can't* wait for | ||
| 206 | * reset work to finish. | ||
| 207 | */ | ||
| 208 | if (r >= 0) | ||
| 209 | cancel_delayed_work(&queue->hang_detect_work); | ||
| 228 | 210 | ||
| 229 | return 0; | 211 | /* Restart the timer when there are still fences pending */ |
| 212 | if (r == 1) | ||
| 213 | amdgpu_userq_start_hang_detect_work(queue); | ||
| 214 | } | ||
| 215 | xa_unlock_irqrestore(xa, flags); | ||
| 230 | } | 216 | } |
| 231 | 217 | ||
| 232 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, | 218 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, |
| 233 | struct amdgpu_usermode_queue *queue, | 219 | struct amdgpu_usermode_queue *queue, |
| 234 | u64 addr, u64 expected_size) | 220 | u64 addr, u64 expected_size, |
| 221 | u64 *va_out) | ||
| 235 | { | 222 | { |
| 236 | struct amdgpu_bo_va_mapping *va_map; | 223 | struct amdgpu_bo_va_mapping *va_map; |
| 237 | struct amdgpu_vm *vm = queue->vm; | 224 | struct amdgpu_vm *vm = queue->vm; |
| 238 | u64 user_addr; | 225 | u64 user_addr; |
| 239 | u64 size; | 226 | u64 size; |
| 240 | int r = 0; | ||
| 241 | 227 | ||
| 242 | /* Caller must hold vm->root.bo reservation */ | 228 | /* Caller must hold vm->root.bo reservation */ |
| 243 | dma_resv_assert_held(queue->vm->root.bo->tbo.base.resv); | 229 | dma_resv_assert_held(queue->vm->root.bo->tbo.base.resv); |
| @@ -246,20 +232,18 @@ int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 246 | size = expected_size >> AMDGPU_GPU_PAGE_SHIFT; | 232 | size = expected_size >> AMDGPU_GPU_PAGE_SHIFT; |
| 247 | 233 | ||
| 248 | va_map = amdgpu_vm_bo_lookup_mapping(vm, user_addr); | 234 | va_map = amdgpu_vm_bo_lookup_mapping(vm, user_addr); |
| 249 | if (!va_map) { | 235 | if (!va_map) |
| 250 | r = -EINVAL; | 236 | return -EINVAL; |
| 251 | goto out_err; | 237 | |
| 252 | } | ||
| 253 | /* Only validate the userq whether resident in the VM mapping range */ | 238 | /* Only validate the userq whether resident in the VM mapping range */ |
| 254 | if (user_addr >= va_map->start && | 239 | if (user_addr >= va_map->start && |
| 255 | va_map->last - user_addr + 1 >= size) { | 240 | va_map->last - user_addr + 1 >= size) { |
| 256 | amdgpu_userq_buffer_va_list_add(queue, va_map, user_addr); | 241 | va_map->bo_va->userq_va_mapped = true; |
| 242 | *va_out = user_addr; | ||
| 257 | return 0; | 243 | return 0; |
| 258 | } | 244 | } |
| 259 | 245 | ||
| 260 | r = -EINVAL; | 246 | return -EINVAL; |
| 261 | out_err: | ||
| 262 | return r; | ||
| 263 | } | 247 | } |
| 264 | 248 | ||
| 265 | static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) | 249 | static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) |
| @@ -270,7 +254,7 @@ static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) · 4 unmodified lines | |||
| 270 | dma_resv_assert_held(vm->root.bo->tbo.base.resv); | 254 | dma_resv_assert_held(vm->root.bo->tbo.base.resv); |
| 271 | 255 | ||
| 272 | mapping = amdgpu_vm_bo_lookup_mapping(vm, addr); | 256 | mapping = amdgpu_vm_bo_lookup_mapping(vm, addr); |
| 273 | if (!IS_ERR_OR_NULL(mapping) && atomic_read(&mapping->bo_va->userq_va_mapped)) | 257 | if (!IS_ERR_OR_NULL(mapping) && mapping->bo_va->userq_va_mapped) |
| 274 | r = true; | 258 | r = true; |
| 275 | else | 259 | else |
| 276 | r = false; | 260 | r = false; |
| @@ -280,14 +264,16 @@ static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) · 3 unmodified lines | |||
| 280 | 264 | ||
| 281 | static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue) | 265 | static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue) |
| 282 | { | 266 | { |
| 283 | struct amdgpu_userq_va_cursor *va_cursor, *tmp; | 267 | int i, r = 0; |
| 284 | int r = 0; | ||
| 285 | 268 | ||
| 286 | list_for_each_entry_safe(va_cursor, tmp, &queue->userq_va_list, list) { | 269 | for (i = 0; i < ARRAY_SIZE(queue->userq_vas.va_array); i++) { |
| 287 | r += amdgpu_userq_buffer_va_mapped(queue->vm, va_cursor->gpu_addr); | 270 | if (!queue->userq_vas.va_array[i]) |
| 271 | continue; | ||
| 272 | r += amdgpu_userq_buffer_va_mapped(queue->vm, | ||
| 273 | queue->userq_vas.va_array[i]); | ||
| 288 | dev_dbg(queue->userq_mgr->adev->dev, | 274 | dev_dbg(queue->userq_mgr->adev->dev, |
| 289 | "validate the userq mapping:%p va:%llx r:%d\n", | 275 | "validate the userq mapping:%p va:%llx r:%d\n", |
| 290 | queue, va_cursor->gpu_addr, r); | 276 | queue, queue->userq_vas.va_array[i], r); |
| 291 | } | 277 | } |
| 292 | 278 | ||
| 293 | if (r != 0) | 279 | if (r != 0) |
| @@ -296,35 +282,7 @@ static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue) · 2 unmodified lines | |||
| 296 | return false; | 282 | return false; |
| 297 | } | 283 | } |
| 298 | 284 | ||
| 299 | static void amdgpu_userq_buffer_va_list_del(struct amdgpu_bo_va_mapping *mapping, | ||
| 300 | struct amdgpu_userq_va_cursor *va_cursor) | ||
| 301 | { | ||
| 302 | atomic_set(&mapping->bo_va->userq_va_mapped, 0); | ||
| 303 | list_del(&va_cursor->list); | ||
| 304 | kfree(va_cursor); | ||
| 305 | } | ||
| 306 | |||
| 307 | static int amdgpu_userq_buffer_vas_list_cleanup(struct amdgpu_device *adev, | ||
| 308 | struct amdgpu_usermode_queue *queue) | ||
| 309 | { | ||
| 310 | struct amdgpu_userq_va_cursor *va_cursor, *tmp; | ||
| 311 | struct amdgpu_bo_va_mapping *mapping; | ||
| 312 | |||
| 313 | /* Caller must hold vm->root.bo reservation */ | ||
| 314 | dma_resv_assert_held(queue->vm->root.bo->tbo.base.resv); | ||
| 315 | |||
| 316 | list_for_each_entry_safe(va_cursor, tmp, &queue->userq_va_list, list) { | ||
| 317 | mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, va_cursor->gpu_addr); | ||
| 318 | if (!mapping) { | ||
| 319 | return -EINVAL; | ||
| 320 | } | ||
| 321 | dev_dbg(adev->dev, "delete the userq:%p va:%llx\n", | ||
| 322 | queue, va_cursor->gpu_addr); | ||
| 323 | amdgpu_userq_buffer_va_list_del(mapping, va_cursor); | ||
| 324 | } | ||
| 325 | 285 | ||
| 326 | return 0; | ||
| 327 | } | ||
| 328 | 286 | ||
| 329 | static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue) | 287 | static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue) |
| 330 | { | 288 | { |
| @@ -332,23 +290,18 @@ static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue) · 1 unmodified line | |||
| 332 | struct amdgpu_device *adev = uq_mgr->adev; | 290 | struct amdgpu_device *adev = uq_mgr->adev; |
| 333 | const struct amdgpu_userq_funcs *userq_funcs = | 291 | const struct amdgpu_userq_funcs *userq_funcs = |
| 334 | adev->userq_funcs[queue->queue_type]; | 292 | adev->userq_funcs[queue->queue_type]; |
| 335 | bool found_hung_queue = false; | 293 | int r; |
| 336 | int r = 0; | ||
| 337 | 294 | ||
| 338 | if (queue->state == AMDGPU_USERQ_STATE_MAPPED) { | 295 | if (queue->state == AMDGPU_USERQ_STATE_MAPPED) { |
| 339 | r = userq_funcs->preempt(queue); | 296 | r = userq_funcs->preempt(queue); |
| 340 | if (r) { | 297 | if (r) { |
| 341 | queue->state = AMDGPU_USERQ_STATE_HUNG; | 298 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 342 | found_hung_queue = true; | 299 | return r; |
| 343 | } else { | 300 | } else { |
| 344 | queue->state = AMDGPU_USERQ_STATE_PREEMPTED; | 301 | queue->state = AMDGPU_USERQ_STATE_PREEMPTED; |
| 345 | } | 302 | } |
| 346 | } | 303 | } |
| 347 | 304 | return 0; | |
| 348 | if (found_hung_queue) | ||
| 349 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | ||
| 350 | |||
| 351 | return r; | ||
| 352 | } | 305 | } |
| 353 | 306 | ||
| 354 | static int amdgpu_userq_restore_helper(struct amdgpu_usermode_queue *queue) | 307 | static int amdgpu_userq_restore_helper(struct amdgpu_usermode_queue *queue) |
| @@ -377,24 +330,21 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue) · 22 unmodified lines | |||
| 377 | struct amdgpu_device *adev = uq_mgr->adev; | 330 | struct amdgpu_device *adev = uq_mgr->adev; |
| 378 | const struct amdgpu_userq_funcs *userq_funcs = | 331 | const struct amdgpu_userq_funcs *userq_funcs = |
| 379 | adev->userq_funcs[queue->queue_type]; | 332 | adev->userq_funcs[queue->queue_type]; |
| 380 | bool found_hung_queue = false; | 333 | int r; |
| 381 | int r = 0; | ||
| 382 | 334 | ||
| 383 | if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) || | 335 | if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) || |
| 384 | (queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) { | 336 | (queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) { |
| 337 | |||
| 385 | r = userq_funcs->unmap(queue); | 338 | r = userq_funcs->unmap(queue); |
| 386 | if (r) { | 339 | if (r) { |
| 387 | queue->state = AMDGPU_USERQ_STATE_HUNG; | 340 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 388 | found_hung_queue = true; | 341 | return r; |
| 389 | } else { | 342 | } else { |
| 390 | queue->state = AMDGPU_USERQ_STATE_UNMAPPED; | 343 | queue->state = AMDGPU_USERQ_STATE_UNMAPPED; |
| 391 | } | 344 | } |
| 392 | } | 345 | } |
| 393 | 346 | ||
| 394 | if (found_hung_queue) | 347 | return 0; |
| 395 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | ||
| 396 | |||
| 397 | return r; | ||
| 398 | } | 348 | } |
| 399 | 349 | ||
| 400 | static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) | 350 | static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) |
| @@ -403,19 +353,19 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) · 2 unmodified lines | |||
| 403 | struct amdgpu_device *adev = uq_mgr->adev; | 353 | struct amdgpu_device *adev = uq_mgr->adev; |
| 404 | const struct amdgpu_userq_funcs *userq_funcs = | 354 | const struct amdgpu_userq_funcs *userq_funcs = |
| 405 | adev->userq_funcs[queue->queue_type]; | 355 | adev->userq_funcs[queue->queue_type]; |
| 406 | int r = 0; | 356 | int r; |
| 407 | 357 | ||
| 408 | if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) { | 358 | if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) { |
| 409 | r = userq_funcs->map(queue); | 359 | r = userq_funcs->map(queue); |
| 410 | if (r) { | 360 | if (r) { |
| 411 | queue->state = AMDGPU_USERQ_STATE_HUNG; | 361 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 412 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | 362 | return r; |
| 413 | } else { | 363 | } else { |
| 414 | queue->state = AMDGPU_USERQ_STATE_MAPPED; | 364 | queue->state = AMDGPU_USERQ_STATE_MAPPED; |
| 415 | } | 365 | } |
| 416 | } | 366 | } |
| 417 | 367 | ||
| 418 | return r; | 368 | return 0; |
| 419 | } | 369 | } |
| 420 | 370 | ||
| 421 | static void amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue) | 371 | static void amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue) |
| @@ -432,18 +382,14 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue) · 10 unmodified lines | |||
| 432 | { | 382 | { |
| 433 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | 383 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; |
| 434 | struct amdgpu_device *adev = uq_mgr->adev; | 384 | struct amdgpu_device *adev = uq_mgr->adev; |
| 435 | const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type]; | ||
| 436 | 385 | ||
| 437 | /* Wait for mode-1 reset to complete */ | 386 | /* Wait for mode-1 reset to complete */ |
| 438 | down_read(&adev->reset_domain->sem); | 387 | down_read(&adev->reset_domain->sem); |
| 439 | 388 | ||
| 440 | uq_funcs->mqd_destroy(queue); | ||
| 441 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ | 389 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ |
| 442 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); | 390 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); |
| 443 | amdgpu_userq_fence_driver_free(queue); | 391 | amdgpu_userq_fence_driver_free(queue); |
| 444 | queue->fence_drv = NULL; | 392 | queue->fence_drv = NULL; |
| 445 | queue->userq_mgr = NULL; | ||
| 446 | list_del(&queue->userq_va_list); | ||
| 447 | 393 | ||
| 448 | up_read(&adev->reset_domain->sem); | 394 | up_read(&adev->reset_domain->sem); |
| 449 | } | 395 | } |
| @@ -482,74 +428,15 @@ retry: · 32 unmodified lines | |||
| 482 | dma_fence_put(ev_fence); | 428 | dma_fence_put(ev_fence); |
| 483 | } | 429 | } |
| 484 | 430 | ||
| 485 | int amdgpu_userq_create_object(struct amdgpu_userq_mgr *uq_mgr, | ||
| 486 | struct amdgpu_userq_obj *userq_obj, | ||
| 487 | int size) | ||
| 488 | { | ||
| 489 | struct amdgpu_device *adev = uq_mgr->adev; | ||
| 490 | struct amdgpu_bo_param bp; | ||
| 491 | int r; | ||
| 492 | |||
| 493 | memset(&bp, 0, sizeof(bp)); | ||
| 494 | bp.byte_align = PAGE_SIZE; | ||
| 495 | bp.domain = AMDGPU_GEM_DOMAIN_GTT; | ||
| 496 | bp.flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS | | ||
| 497 | AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; | ||
| 498 | bp.type = ttm_bo_type_kernel; | ||
| 499 | bp.size = size; | ||
| 500 | bp.resv = NULL; | ||
| 501 | bp.bo_ptr_size = sizeof(struct amdgpu_bo); | ||
| 502 | |||
| 503 | r = amdgpu_bo_create(adev, &bp, &userq_obj->obj); | ||
| 504 | if (r) { | ||
| 505 | drm_file_err(uq_mgr->file, "Failed to allocate BO for userqueue (%d)", r); | ||
| 506 | return r; | ||
| 507 | } | ||
| 508 | |||
| 509 | r = amdgpu_bo_reserve(userq_obj->obj, true); | ||
| 510 | if (r) { | ||
| 511 | drm_file_err(uq_mgr->file, "Failed to reserve BO to map (%d)", r); | ||
| 512 | goto free_obj; | ||
| 513 | } | ||
| 514 | |||
| 515 | r = amdgpu_ttm_alloc_gart(&(userq_obj->obj)->tbo); | ||
| 516 | if (r) { | ||
| 517 | drm_file_err(uq_mgr->file, "Failed to alloc GART for userqueue object (%d)", r); | ||
| 518 | goto unresv; | ||
| 519 | } | ||
| 520 | |||
| 521 | r = amdgpu_bo_kmap(userq_obj->obj, &userq_obj->cpu_ptr); | ||
| 522 | if (r) { | ||
| 523 | drm_file_err(uq_mgr->file, "Failed to map BO for userqueue (%d)", r); | ||
| 524 | goto unresv; | ||
| 525 | } | ||
| 526 | |||
| 527 | userq_obj->gpu_addr = amdgpu_bo_gpu_offset(userq_obj->obj); | ||
| 528 | amdgpu_bo_unreserve(userq_obj->obj); | ||
| 529 | memset(userq_obj->cpu_ptr, 0, size); | ||
| 530 | return 0; | ||
| 531 | 431 | ||
| 532 | unresv: | ||
| 533 | amdgpu_bo_unreserve(userq_obj->obj); | ||
| 534 | |||
| 535 | free_obj: | ||
| 536 | amdgpu_bo_unref(&userq_obj->obj); | ||
| 537 | return r; | ||
| 538 | } | ||
| 539 | |||
| 540 | void amdgpu_userq_destroy_object(struct amdgpu_userq_mgr *uq_mgr, | ||
| 541 | struct amdgpu_userq_obj *userq_obj) | ||
| 542 | { | ||
| 543 | amdgpu_bo_kunmap(userq_obj->obj); | ||
| 544 | amdgpu_bo_unref(&userq_obj->obj); | ||
| 545 | } | ||
| 546 | 432 | ||
| 547 | uint64_t | 433 | static int |
| 548 | amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, | 434 | amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, |
| 549 | struct amdgpu_db_info *db_info, | 435 | struct amdgpu_db_info *db_info, |
| 550 | struct drm_file *filp) | 436 | struct drm_file *filp, |
| 437 | u64 *index) | ||
| 551 | { | 438 | { |
| 552 | uint64_t index; | 439 | u64 doorbell_index; |
| 553 | struct drm_gem_object *gobj; | 440 | struct drm_gem_object *gobj; |
| 554 | struct amdgpu_userq_obj *db_obj = db_info->db_obj; | 441 | struct amdgpu_userq_obj *db_obj = db_info->db_obj; |
| 555 | int r, db_size; | 442 | int r, db_size; |
| @@ -596,12 +483,13 @@ amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, · 40 unmodified lines | |||
| 596 | goto unpin_bo; | 483 | goto unpin_bo; |
| 597 | } | 484 | } |
| 598 | 485 | ||
| 599 | index = amdgpu_doorbell_index_on_bar(uq_mgr->adev, db_obj->obj, | 486 | doorbell_index = amdgpu_doorbell_index_on_bar(uq_mgr->adev, db_obj->obj, |
| 600 | db_info->doorbell_offset, db_size); | 487 | db_info->doorbell_offset, db_size); |
| 601 | drm_dbg_driver(adev_to_drm(uq_mgr->adev), | 488 | drm_dbg_driver(adev_to_drm(uq_mgr->adev), |
| 602 | "[Usermode queues] doorbell index=%lld\n", index); | 489 | "[Usermode queues] doorbell index=%lld\n", doorbell_index); |
| 603 | amdgpu_bo_unreserve(db_obj->obj); | 490 | amdgpu_bo_unreserve(db_obj->obj); |
| 604 | return index; | 491 | *index = doorbell_index; |
| 492 | return 0; | ||
| 605 | 493 | ||
| 606 | unpin_bo: | 494 | unpin_bo: |
| 607 | amdgpu_bo_unpin(db_obj->obj); | 495 | amdgpu_bo_unpin(db_obj->obj); |
| @@ -616,9 +504,7 @@ static int · 8 unmodified lines | |||
| 616 | amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_queue *queue) | 504 | amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_queue *queue) |
| 617 | { | 505 | { |
| 618 | struct amdgpu_device *adev = uq_mgr->adev; | 506 | struct amdgpu_device *adev = uq_mgr->adev; |
| 619 | struct amdgpu_fpriv *fpriv = uq_mgr_to_fpriv(uq_mgr); | 507 | const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type]; |
| 620 | struct amdgpu_vm *vm = &fpriv->vm; | ||
| 621 | |||
| 622 | int r = 0; | 508 | int r = 0; |
| 623 | 509 | ||
| 624 | cancel_delayed_work_sync(&uq_mgr->resume_work); | 510 | cancel_delayed_work_sync(&uq_mgr->resume_work); |
| @@ -626,33 +512,21 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que · 1 unmodified line | |||
| 626 | /* Cancel any pending hang detection work and cleanup */ | 512 | /* Cancel any pending hang detection work and cleanup */ |
| 627 | cancel_delayed_work_sync(&queue->hang_detect_work); | 513 | cancel_delayed_work_sync(&queue->hang_detect_work); |
| 628 | 514 | ||
| 629 | r = amdgpu_bo_reserve(vm->root.bo, false); | ||
| 630 | if (r) { | ||
| 631 | drm_file_err(uq_mgr->file, "Failed to reserve root bo during userqueue destroy\n"); | ||
| 632 | return r; | ||
| 633 | } | ||
| 634 | amdgpu_userq_buffer_vas_list_cleanup(adev, queue); | ||
| 635 | amdgpu_bo_unreserve(vm->root.bo); | ||
| 636 | |||
| 637 | mutex_lock(&uq_mgr->userq_mutex); | 515 | mutex_lock(&uq_mgr->userq_mutex); |
| 638 | queue->hang_detect_fence = NULL; | ||
| 639 | amdgpu_userq_wait_for_last_fence(queue); | 516 | amdgpu_userq_wait_for_last_fence(queue); |
| 640 | 517 | ||
| 641 | #if defined(CONFIG_DEBUG_FS) | 518 | #if defined(CONFIG_DEBUG_FS) |
| 642 | debugfs_remove_recursive(queue->debugfs_queue); | 519 | debugfs_remove_recursive(queue->debugfs_queue); |
| 643 | #endif | 520 | #endif |
| 644 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | ||
| 645 | r = amdgpu_userq_unmap_helper(queue); | 521 | r = amdgpu_userq_unmap_helper(queue); |
| 646 | /*TODO: It requires a reset for userq hw unmap error*/ | ||
| 647 | if (r) { | ||
| 648 | drm_warn(adev_to_drm(uq_mgr->adev), "trying to destroy a HW mapping userq\n"); | ||
| 649 | queue->state = AMDGPU_USERQ_STATE_HUNG; | ||
| 650 | } | ||
| 651 | |||
| 652 | atomic_dec(&uq_mgr->userq_count[queue->queue_type]); | 522 | atomic_dec(&uq_mgr->userq_count[queue->queue_type]); |
| 653 | amdgpu_userq_cleanup(queue); | 523 | amdgpu_userq_cleanup(queue); |
| 654 | mutex_unlock(&uq_mgr->userq_mutex); | 524 | mutex_unlock(&uq_mgr->userq_mutex); |
| 655 | 525 | ||
| 526 | cancel_delayed_work_sync(&queue->hang_detect_work); | ||
| 527 | uq_funcs->mqd_destroy(queue); | ||
| 528 | queue->userq_mgr = NULL; | ||
| 529 | |||
| 656 | amdgpu_bo_reserve(queue->db_obj.obj, true); | 530 | amdgpu_bo_reserve(queue->db_obj.obj, true); |
| 657 | amdgpu_bo_unpin(queue->db_obj.obj); | 531 | amdgpu_bo_unpin(queue->db_obj.obj); |
| 658 | amdgpu_bo_unreserve(queue->db_obj.obj); | 532 | amdgpu_bo_unreserve(queue->db_obj.obj); |
| @@ -724,14 +598,14 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 65 unmodified lines | |||
| 724 | const struct amdgpu_userq_funcs *uq_funcs; | 598 | const struct amdgpu_userq_funcs *uq_funcs; |
| 725 | struct amdgpu_usermode_queue *queue; | 599 | struct amdgpu_usermode_queue *queue; |
| 726 | struct amdgpu_db_info db_info; | 600 | struct amdgpu_db_info db_info; |
| 727 | bool skip_map_queue; | ||
| 728 | u32 qid; | ||
| 729 | uint64_t index; | 601 | uint64_t index; |
| 730 | int r = 0; | 602 | int priority; |
| 731 | int priority = | 603 | u32 qid; |
| 732 | (args->in.flags & AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_MASK) >> | 604 | int r; |
| 733 | AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_SHIFT; | ||
| 734 | 605 | ||
| 606 | priority = | ||
| 607 | (args->in.flags & AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_MASK) | ||
| 608 | >> AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_SHIFT; | ||
| 735 | r = amdgpu_userq_priority_permit(filp, priority); | 609 | r = amdgpu_userq_priority_permit(filp, priority); |
| 736 | if (r) | 610 | if (r) |
| 737 | return r; | 611 | return r; |
| @@ -744,128 +618,121 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 6 unmodified lines | |||
| 744 | 618 | ||
| 745 | uq_funcs = adev->userq_funcs[args->in.ip_type]; | 619 | uq_funcs = adev->userq_funcs[args->in.ip_type]; |
| 746 | if (!uq_funcs) { | 620 | if (!uq_funcs) { |
| 747 | drm_file_err(uq_mgr->file, "Usermode queue is not supported for this IP (%u)\n", | ||
| 748 | args->in.ip_type); | ||
| 749 | r = -EINVAL; | 621 | r = -EINVAL; |
| 750 | goto err_pm_runtime; | 622 | goto err_pm_runtime; |
| 751 | } | 623 | } |
| 752 | 624 | ||
| 753 | queue = kzalloc_obj(struct amdgpu_usermode_queue); | 625 | queue = kzalloc_obj(struct amdgpu_usermode_queue); |
| 754 | if (!queue) { | 626 | if (!queue) { |
| 755 | drm_file_err(uq_mgr->file, "Failed to allocate memory for queue\n"); | ||
| 756 | r = -ENOMEM; | 627 | r = -ENOMEM; |
| 757 | goto err_pm_runtime; | 628 | goto err_pm_runtime; |
| 758 | } | 629 | } |
| 759 | 630 | ||
| 760 | INIT_LIST_HEAD(&queue->userq_va_list); | 631 | kref_init(&queue->refcount); |
| 761 | queue->doorbell_handle = args->in.doorbell_handle; | 632 | queue->doorbell_handle = args->in.doorbell_handle; |
| 762 | queue->queue_type = args->in.ip_type; | 633 | queue->queue_type = args->in.ip_type; |
| 763 | queue->vm = &fpriv->vm; | 634 | queue->vm = &fpriv->vm; |
| 764 | queue->priority = priority; | 635 | queue->priority = priority; |
| 765 | |||
| 766 | db_info.queue_type = queue->queue_type; | ||
| 767 | db_info.doorbell_handle = queue->doorbell_handle; | ||
| 768 | db_info.db_obj = &queue->db_obj; | ||
| 769 | db_info.doorbell_offset = args->in.doorbell_offset; | ||
| 770 | |||
| 771 | queue->userq_mgr = uq_mgr; | 636 | queue->userq_mgr = uq_mgr; |
| 637 | INIT_DELAYED_WORK(&queue->hang_detect_work, | ||
| 638 | amdgpu_userq_hang_detect_work); | ||
| 772 | 639 | ||
| 773 | /* Validate the userq virtual address.*/ | 640 | r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv); |
| 774 | r = amdgpu_bo_reserve(fpriv->vm.root.bo, false); | ||
| 775 | if (r) | 641 | if (r) |
| 776 | goto free_queue; | 642 | goto free_queue; |
| 777 | 643 | ||
| 778 | if (amdgpu_userq_input_va_validate(adev, queue, args->in.queue_va, args->in.queue_size) || | 644 | xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC); |
| 779 | amdgpu_userq_input_va_validate(adev, queue, args->in.rptr_va, AMDGPU_GPU_PAGE_SIZE) || | 645 | mutex_init(&queue->fence_drv_lock); |
| 780 | amdgpu_userq_input_va_validate(adev, queue, args->in.wptr_va, AMDGPU_GPU_PAGE_SIZE)) { | 646 | /* Make sure the queue can actually run with those virtual addresses. */ |
| 647 | r = amdgpu_bo_reserve(fpriv->vm.root.bo, false); | ||
| 648 | if (r) | ||
| 649 | goto free_fence_drv; | ||
| 650 | |||
| 651 | if (amdgpu_userq_input_va_validate(adev, queue, args->in.queue_va, | ||
| 652 | args->in.queue_size, | ||
| 653 | &queue->userq_vas.va.queue_rb) || | ||
| 654 | amdgpu_userq_input_va_validate(adev, queue, args->in.rptr_va, | ||
| 655 | AMDGPU_GPU_PAGE_SIZE, | ||
| 656 | &queue->userq_vas.va.rptr) || | ||
| 657 | amdgpu_userq_input_va_validate(adev, queue, args->in.wptr_va, | ||
| 658 | AMDGPU_GPU_PAGE_SIZE, | ||
| 659 | &queue->userq_vas.va.wptr)) { | ||
| 781 | r = -EINVAL; | 660 | r = -EINVAL; |
| 782 | amdgpu_bo_unreserve(fpriv->vm.root.bo); | 661 | amdgpu_bo_unreserve(fpriv->vm.root.bo); |
| 783 | goto clean_mapping; | 662 | goto free_fence_drv; |
| 784 | } | 663 | } |
| 785 | amdgpu_bo_unreserve(fpriv->vm.root.bo); | 664 | amdgpu_bo_unreserve(fpriv->vm.root.bo); |
| 786 | 665 | ||
| 787 | /* Convert relative doorbell offset into absolute doorbell index */ | 666 | /* Convert relative doorbell offset into absolute doorbell index */ |
| 788 | index = amdgpu_userq_get_doorbell_index(uq_mgr, &db_info, filp); | 667 | db_info.queue_type = queue->queue_type; |
| 789 | if (index == (uint64_t)-EINVAL) { | 668 | db_info.doorbell_handle = queue->doorbell_handle; |
| 669 | db_info.db_obj = &queue->db_obj; | ||
| 670 | db_info.doorbell_offset = args->in.doorbell_offset; | ||
| 671 | r = amdgpu_userq_get_doorbell_index(uq_mgr, &db_info, filp, &index); | ||
| 672 | if (r) { | ||
| 790 | drm_file_err(uq_mgr->file, "Failed to get doorbell for queue\n"); | 673 | drm_file_err(uq_mgr->file, "Failed to get doorbell for queue\n"); |
| 791 | r = -EINVAL; | 674 | goto free_fence_drv; |
| 792 | goto clean_mapping; | ||
| 793 | } | 675 | } |
| 794 | 676 | ||
| 795 | queue->doorbell_index = index; | 677 | queue->doorbell_index = index; |
| 796 | xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC); | ||
| 797 | r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv); | ||
| 798 | if (r) { | ||
| 799 | drm_file_err(uq_mgr->file, "Failed to alloc fence driver\n"); | ||
| 800 | goto clean_mapping; | ||
| 801 | } | ||
| 802 | |||
| 803 | r = uq_funcs->mqd_create(queue, &args->in); | 678 | r = uq_funcs->mqd_create(queue, &args->in); |
| 804 | if (r) { | 679 | if (r) { |
| 805 | drm_file_err(uq_mgr->file, "Failed to create Queue\n"); | 680 | drm_file_err(uq_mgr->file, "Failed to create Queue\n"); |
| 806 | goto clean_fence_driver; | 681 | goto clean_doorbell_bo; |
| 807 | } | 682 | } |
| 808 | 683 | ||
| 684 | /* Update VM owner at userq submit-time for page-fault attribution. */ | ||
| 685 | amdgpu_vm_set_task_info(&fpriv->vm); | ||
| 686 | |||
| 687 | r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, | ||
| 688 | GFP_KERNEL)); | ||
| 689 | if (r) | ||
| 690 | goto clean_mqd; | ||
| 691 | |||
| 809 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); | 692 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); |
| 810 | 693 | ||
| 811 | /* don't map the queue if scheduling is halted */ | 694 | /* don't map the queue if scheduling is halted */ |
| 812 | if (adev->userq_halt_for_enforce_isolation && | 695 | if (!adev->userq_halt_for_enforce_isolation || |
| 813 | ((queue->queue_type == AMDGPU_HW_IP_GFX) || | 696 | ((queue->queue_type != AMDGPU_HW_IP_GFX) && |
| 814 | (queue->queue_type == AMDGPU_HW_IP_COMPUTE))) | 697 | (queue->queue_type != AMDGPU_HW_IP_COMPUTE))) { |
| 815 | skip_map_queue = true; | ||
| 816 | else | ||
| 817 | skip_map_queue = false; | ||
| 818 | if (!skip_map_queue) { | ||
| 819 | r = amdgpu_userq_map_helper(queue); | 698 | r = amdgpu_userq_map_helper(queue); |
| 820 | if (r) { | 699 | if (r) { |
| 821 | drm_file_err(uq_mgr->file, "Failed to map Queue\n"); | 700 | drm_file_err(uq_mgr->file, "Failed to map Queue\n"); |
| 822 | goto clean_mqd; | 701 | mutex_unlock(&uq_mgr->userq_mutex); |
| 702 | goto erase_doorbell; | ||
| 823 | } | 703 | } |
| 824 | } | 704 | } |
| 825 | 705 | ||
| 826 | /* drop this refcount during queue destroy */ | 706 | atomic_inc(&uq_mgr->userq_count[queue->queue_type]); |
| 827 | kref_init(&queue->refcount); | 707 | mutex_unlock(&uq_mgr->userq_mutex); |
| 828 | |||
| 829 | /* Wait for mode-1 reset to complete */ | ||
| 830 | down_read(&adev->reset_domain->sem); | ||
| 831 | 708 | ||
| 832 | r = xa_alloc(&uq_mgr->userq_xa, &qid, queue, | 709 | r = xa_alloc(&uq_mgr->userq_xa, &qid, queue, |
| 833 | XA_LIMIT(1, AMDGPU_MAX_USERQ_COUNT), GFP_KERNEL); | 710 | XA_LIMIT(1, AMDGPU_MAX_USERQ_COUNT), |
| 834 | if (r) { | 711 | GFP_KERNEL); |
| 835 | if (!skip_map_queue) | ||
| 836 | amdgpu_userq_unmap_helper(queue); | ||
| 837 | r = -ENOMEM; | ||
| 838 | goto clean_reset_domain; | ||
| 839 | } | ||
| 840 | |||
| 841 | r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, GFP_KERNEL)); | ||
| 842 | if (r) { | 712 | if (r) { |
| 843 | xa_erase(&uq_mgr->userq_xa, qid); | 713 | /* |
| 844 | if (!skip_map_queue) | 714 | * This drops the last reference which should take care of |
| 845 | amdgpu_userq_unmap_helper(queue); | 715 | * all cleanup. |
| 846 | goto clean_reset_domain; | 716 | */ |
| 717 | amdgpu_userq_put(queue); | ||
| 718 | return r; | ||
| 847 | } | 719 | } |
| 848 | up_read(&adev->reset_domain->sem); | ||
| 849 | 720 | ||
| 850 | amdgpu_debugfs_userq_init(filp, queue, qid); | 721 | amdgpu_debugfs_userq_init(filp, queue, qid); |
| 851 | amdgpu_userq_init_hang_detect_work(queue); | ||
| 852 | |||
| 853 | args->out.queue_id = qid; | 722 | args->out.queue_id = qid; |
| 854 | atomic_inc(&uq_mgr->userq_count[queue->queue_type]); | ||
| 855 | mutex_unlock(&uq_mgr->userq_mutex); | ||
| 856 | return 0; | 723 | return 0; |
| 857 | 724 | ||
| 858 | clean_reset_domain: | 725 | erase_doorbell: |
| 859 | up_read(&adev->reset_domain->sem); | 726 | xa_erase_irq(&adev->userq_doorbell_xa, index); |
| 860 | clean_mqd: | 727 | clean_mqd: |
| 861 | mutex_unlock(&uq_mgr->userq_mutex); | ||
| 862 | uq_funcs->mqd_destroy(queue); | 728 | uq_funcs->mqd_destroy(queue); |
| 863 | clean_fence_driver: | 729 | clean_doorbell_bo: |
| 730 | amdgpu_bo_reserve(queue->db_obj.obj, true); | ||
| 731 | amdgpu_bo_unpin(queue->db_obj.obj); | ||
| 732 | amdgpu_bo_unreserve(queue->db_obj.obj); | ||
| 733 | amdgpu_bo_unref(&queue->db_obj.obj); | ||
| 734 | free_fence_drv: | ||
| 864 | amdgpu_userq_fence_driver_free(queue); | 735 | amdgpu_userq_fence_driver_free(queue); |
| 865 | clean_mapping: | ||
| 866 | amdgpu_bo_reserve(fpriv->vm.root.bo, true); | ||
| 867 | amdgpu_userq_buffer_vas_list_cleanup(adev, queue); | ||
| 868 | amdgpu_bo_unreserve(fpriv->vm.root.bo); | ||
| 869 | free_queue: | 736 | free_queue: |
| 870 | kfree(queue); | 737 | kfree(queue); |
| 871 | err_pm_runtime: | 738 | err_pm_runtime: |
| @@ -1187,7 +1054,7 @@ retry_lock: · 315 unmodified lines | |||
| 1187 | bo = range->bo; | 1054 | bo = range->bo; |
| 1188 | ret = amdgpu_ttm_tt_get_user_pages(bo, range); | 1055 | ret = amdgpu_ttm_tt_get_user_pages(bo, range); |
| 1189 | if (ret) | 1056 | if (ret) |
| 1190 | goto unlock_all; | 1057 | goto free_ranges; |
| 1191 | } | 1058 | } |
| 1192 | 1059 | ||
| 1193 | invalidated = true; | 1060 | invalidated = true; |
| @@ -1214,6 +1081,7 @@ retry_lock: · 20 unmodified lines | |||
| 1214 | 1081 | ||
| 1215 | unlock_all: | 1082 | unlock_all: |
| 1216 | drm_exec_fini(&exec); | 1083 | drm_exec_fini(&exec); |
| 1084 | free_ranges: | ||
| 1217 | xa_for_each(&xa, tmp_key, range) { | 1085 | xa_for_each(&xa, tmp_key, range) { |
| 1218 | if (!range) | 1086 | if (!range) |
| 1219 | continue; | 1087 | continue; |
| @@ -1254,7 +1122,6 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 34 unmodified lines | |||
| 1254 | unsigned long queue_id; | 1122 | unsigned long queue_id; |
| 1255 | int ret = 0, r; | 1123 | int ret = 0, r; |
| 1256 | 1124 | ||
| 1257 | amdgpu_userq_detect_and_reset_queues(uq_mgr); | ||
| 1258 | /* Try to unmap all the queues in this process ctx */ | 1125 | /* Try to unmap all the queues in this process ctx */ |
| 1259 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { | 1126 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1260 | r = amdgpu_userq_preempt_helper(queue); | 1127 | r = amdgpu_userq_preempt_helper(queue); |
| @@ -1262,29 +1129,16 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | |||
| 1262 | ret = r; | 1129 | ret = r; |
| 1263 | } | 1130 | } |
| 1264 | 1131 | ||
| 1265 | if (ret) | 1132 | if (ret) { |
| 1266 | drm_file_err(uq_mgr->file, | 1133 | drm_file_err(uq_mgr->file, |
| 1267 | "Couldn't unmap all the queues, eviction failed ret=%d\n", ret); | 1134 | "Couldn't unmap all the queues, eviction failed ret=%d\n", ret); |
| 1135 | amdgpu_reset_domain_schedule(uq_mgr->adev->reset_domain, | ||
| 1136 | &uq_mgr->reset_work); | ||
| 1137 | flush_work(&uq_mgr->reset_work); | ||
| 1138 | } | ||
| 1268 | return ret; | 1139 | return ret; |
| 1269 | } | 1140 | } |
| 1270 | 1141 | ||
| 1271 | void amdgpu_userq_reset_work(struct work_struct *work) | ||
| 1272 | { | ||
| 1273 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, | ||
| 1274 | userq_reset_work); | ||
| 1275 | struct amdgpu_reset_context reset_context; | ||
| 1276 | |||
| 1277 | memset(&reset_context, 0, sizeof(reset_context)); | ||
| 1278 | |||
| 1279 | reset_context.method = AMD_RESET_METHOD_NONE; | ||
| 1280 | reset_context.reset_req_dev = adev; | ||
| 1281 | reset_context.src = AMDGPU_RESET_SRC_USERQ; | ||
| 1282 | set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | ||
| 1283 | /*set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags);*/ | ||
| 1284 | |||
| 1285 | amdgpu_device_gpu_recover(adev, NULL, &reset_context); | ||
| 1286 | } | ||
| 1287 | |||
| 1288 | static void | 1142 | static void |
| 1289 | amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr) | 1143 | amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr) |
| 1290 | { | 1144 | { |
| @@ -1318,9 +1172,24 @@ int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *f · 27 unmodified lines | |||
| 1318 | userq_mgr->file = file_priv; | 1172 | userq_mgr->file = file_priv; |
| 1319 | 1173 | ||
| 1320 | INIT_DELAYED_WORK(&userq_mgr->resume_work, amdgpu_userq_restore_worker); | 1174 | INIT_DELAYED_WORK(&userq_mgr->resume_work, amdgpu_userq_restore_worker); |
| 1175 | INIT_WORK(&userq_mgr->reset_work, amdgpu_userq_mgr_reset_work); | ||
| 1321 | return 0; | 1176 | return 0; |
| 1322 | } | 1177 | } |
| 1323 | 1178 | ||
| 1179 | void amdgpu_userq_mgr_cancel_reset_work(struct amdgpu_device *adev) | ||
| 1180 | { | ||
| 1181 | struct xarray *xa = &adev->userq_doorbell_xa; | ||
| 1182 | struct amdgpu_usermode_queue *queue; | ||
| 1183 | unsigned long flags, queue_id; | ||
| 1184 | |||
| 1185 | xa_lock_irqsave(xa, flags); | ||
| 1186 | xa_for_each(xa, queue_id, queue) { | ||
| 1187 | cancel_delayed_work(&queue->hang_detect_work); | ||
| 1188 | cancel_work(&queue->userq_mgr->reset_work); | ||
| 1189 | } | ||
| 1190 | xa_unlock_irqrestore(xa, flags); | ||
| 1191 | } | ||
| 1192 | |||
| 1324 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr) | 1193 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr) |
| 1325 | { | 1194 | { |
| 1326 | cancel_delayed_work_sync(&userq_mgr->resume_work); | 1195 | cancel_delayed_work_sync(&userq_mgr->resume_work); |
| @@ -1346,6 +1215,14 @@ void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr) · 19 unmodified lines | |||
| 1346 | } | 1215 | } |
| 1347 | 1216 | ||
| 1348 | xa_destroy(&userq_mgr->userq_xa); | 1217 | xa_destroy(&userq_mgr->userq_xa); |
| 1218 | |||
| 1219 | /* | ||
| 1220 | * Drain any in-flight reset_work. By this point all queues are freed | ||
| 1221 | * and userq_count is 0, so if reset_work starts now it exits early. | ||
| 1222 | * We still need to wait in case it was already executing gpu_recover. | ||
| 1223 | */ | ||
| 1224 | cancel_work_sync(&userq_mgr->reset_work); | ||
| 1225 | |||
| 1349 | mutex_destroy(&userq_mgr->userq_mutex); | 1226 | mutex_destroy(&userq_mgr->userq_mutex); |
| 1350 | } | 1227 | } |
| 1351 | 1228 | ||
| @@ -1364,7 +1241,6 @@ int amdgpu_userq_suspend(struct amdgpu_device *adev) · 12 unmodified lines | |||
| 1364 | uqm = queue->userq_mgr; | 1241 | uqm = queue->userq_mgr; |
| 1365 | cancel_delayed_work_sync(&uqm->resume_work); | 1242 | cancel_delayed_work_sync(&uqm->resume_work); |
| 1366 | guard(mutex)(&uqm->userq_mutex); | 1243 | guard(mutex)(&uqm->userq_mutex); |
| 1367 | amdgpu_userq_detect_and_reset_queues(uqm); | ||
| 1368 | if (adev->in_s0ix) | 1244 | if (adev->in_s0ix) |
| 1369 | r = amdgpu_userq_preempt_helper(queue); | 1245 | r = amdgpu_userq_preempt_helper(queue); |
| 1370 | else | 1246 | else |
| @@ -1423,7 +1299,6 @@ int amdgpu_userq_stop_sched_for_enforce_isolation(struct amdgpu_device *adev, · 52 unmodified lines | |||
| 1423 | if (((queue->queue_type == AMDGPU_HW_IP_GFX) || | 1299 | if (((queue->queue_type == AMDGPU_HW_IP_GFX) || |
| 1424 | (queue->queue_type == AMDGPU_HW_IP_COMPUTE)) && | 1300 | (queue->queue_type == AMDGPU_HW_IP_COMPUTE)) && |
| 1425 | (queue->xcp_id == idx)) { | 1301 | (queue->xcp_id == idx)) { |
| 1426 | amdgpu_userq_detect_and_reset_queues(uqm); | ||
| 1427 | r = amdgpu_userq_preempt_helper(queue); | 1302 | r = amdgpu_userq_preempt_helper(queue); |
| 1428 | if (r) | 1303 | if (r) |
| 1429 | ret = r; | 1304 | ret = r; |
| @@ -1496,23 +1371,21 @@ void amdgpu_userq_pre_reset(struct amdgpu_device *adev) · 66 unmodified lines | |||
| 1496 | { | 1371 | { |
| 1497 | const struct amdgpu_userq_funcs *userq_funcs; | 1372 | const struct amdgpu_userq_funcs *userq_funcs; |
| 1498 | struct amdgpu_usermode_queue *queue; | 1373 | struct amdgpu_usermode_queue *queue; |
| 1499 | struct amdgpu_userq_mgr *uqm; | ||
| 1500 | unsigned long queue_id; | 1374 | unsigned long queue_id; |
| 1501 | 1375 | ||
| 1376 | /* TODO: We probably need a new lock for the queue state */ | ||
| 1502 | xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) { | 1377 | xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) { |
| 1503 | uqm = queue->userq_mgr; | 1378 | if (queue->state != AMDGPU_USERQ_STATE_MAPPED) |
| 1504 | cancel_delayed_work_sync(&uqm->resume_work); | 1379 | continue; |
| 1505 | if (queue->state == AMDGPU_USERQ_STATE_MAPPED) { | 1380 | |
| 1506 | amdgpu_userq_wait_for_last_fence(queue); | 1381 | userq_funcs = adev->userq_funcs[queue->queue_type]; |
| 1507 | userq_funcs = adev->userq_funcs[queue->queue_type]; | 1382 | userq_funcs->unmap(queue); |
| 1508 | userq_funcs->unmap(queue); | 1383 | /* just mark all queues as hung at this point. |
| 1509 | /* just mark all queues as hung at this point. | 1384 | * if unmap succeeds, we could map again |
| 1510 | * if unmap succeeds, we could map again | 1385 | * in amdgpu_userq_post_reset() if vram is not lost |
| 1511 | * in amdgpu_userq_post_reset() if vram is not lost | 1386 | */ |
| 1512 | */ | 1387 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 1513 | queue->state = AMDGPU_USERQ_STATE_HUNG; | 1388 | amdgpu_userq_fence_driver_force_completion(queue); |
| 1514 | amdgpu_userq_fence_driver_force_completion(queue); | ||
| 1515 | } | ||
| 1516 | } | 1389 | } |
| 1517 | } | 1390 | } |
| 1518 | 1391 | ||
| @@ -48,11 +48,6 @@ struct amdgpu_userq_obj { · 47 unmodified lines | |||
| 48 | struct amdgpu_bo *obj; | 48 | struct amdgpu_bo *obj; |
| 49 | }; | 49 | }; |
| 50 | 50 | ||
| 51 | struct amdgpu_userq_va_cursor { | ||
| 52 | u64 gpu_addr; | ||
| 53 | struct list_head list; | ||
| 54 | }; | ||
| 55 | |||
| 56 | struct amdgpu_usermode_queue { | 51 | struct amdgpu_usermode_queue { |
| 57 | int queue_type; | 52 | int queue_type; |
| 58 | enum amdgpu_userq_state state; | 53 | enum amdgpu_userq_state state; |
| @@ -66,17 +61,44 @@ struct amdgpu_usermode_queue { · 7 unmodified lines | |||
| 66 | struct amdgpu_userq_obj db_obj; | 61 | struct amdgpu_userq_obj db_obj; |
| 67 | struct amdgpu_userq_obj fw_obj; | 62 | struct amdgpu_userq_obj fw_obj; |
| 68 | struct amdgpu_userq_obj wptr_obj; | 63 | struct amdgpu_userq_obj wptr_obj; |
| 64 | |||
| 65 | /** | ||
| 66 | * @fence_drv_lock: Protecting @fence_drv_xa. | ||
| 67 | */ | ||
| 68 | struct mutex fence_drv_lock; | ||
| 69 | |||
| 70 | /** | ||
| 71 | * @fence_drv_xa: | ||
| 72 | * | ||
| 73 | * References to the external fence drivers returned by wait_ioctl. | ||
| 74 | * Dropped on the next signaled dma_fence or queue destruction. | ||
| 75 | */ | ||
| 69 | struct xarray fence_drv_xa; | 76 | struct xarray fence_drv_xa; |
| 70 | struct amdgpu_userq_fence_driver *fence_drv; | 77 | struct amdgpu_userq_fence_driver *fence_drv; |
| 71 | struct dma_fence *last_fence; | 78 | struct dma_fence *last_fence; |
| 72 | u32 xcp_id; | 79 | u32 xcp_id; |
| 73 | int priority; | 80 | int priority; |
| 74 | struct dentry *debugfs_queue; | 81 | struct dentry *debugfs_queue; |
| 75 | struct delayed_work hang_detect_work; | 82 | |
| 76 | struct dma_fence *hang_detect_fence; | 83 | /** |
| 84 | * @hang_detect_work: | ||
| 85 | * | ||
| 86 | * Delayed work which runs when userq_fences time out. | ||
| 87 | */ | ||
| 88 | struct delayed_work hang_detect_work; | ||
| 77 | struct kref refcount; | 89 | struct kref refcount; |
| 78 | 90 | ||
| 79 | struct list_head userq_va_list; | 91 | union { |
| 92 | struct { | ||
| 93 | u64 queue_rb; | ||
| 94 | u64 wptr; | ||
| 95 | u64 rptr; | ||
| 96 | u64 eop; | ||
| 97 | u64 shadow; | ||
| 98 | u64 csa; | ||
| 99 | } va; | ||
| 100 | u64 va_array[6]; | ||
| 101 | } userq_vas; | ||
| 80 | }; | 102 | }; |
| 81 | 103 | ||
| 82 | struct amdgpu_userq_funcs { | 104 | struct amdgpu_userq_funcs { |
| @@ -105,6 +127,13 @@ struct amdgpu_userq_mgr { · 22 unmodified lines | |||
| 105 | struct amdgpu_device *adev; | 127 | struct amdgpu_device *adev; |
| 106 | struct delayed_work resume_work; | 128 | struct delayed_work resume_work; |
| 107 | struct drm_file *file; | 129 | struct drm_file *file; |
| 130 | |||
| 131 | /** | ||
| 132 | * @reset_work: | ||
| 133 | * | ||
| 134 | * Reset work which is used when eviction fails. | ||
| 135 | */ | ||
| 136 | struct work_struct reset_work; | ||
| 108 | atomic_t userq_count[AMDGPU_RING_TYPE_MAX]; | 137 | atomic_t userq_count[AMDGPU_RING_TYPE_MAX]; |
| 109 | }; | 138 | }; |
| 110 | 139 | ||
| @@ -123,25 +152,15 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp · 12 unmodified lines | |||
| 123 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, | 152 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, |
| 124 | struct amdgpu_device *adev); | 153 | struct amdgpu_device *adev); |
| 125 | 154 | ||
| 155 | void amdgpu_userq_mgr_cancel_reset_work(struct amdgpu_device *adev); | ||
| 126 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr); | 156 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr); |
| 127 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr); | 157 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr); |
| 128 | 158 | ||
| 129 | int amdgpu_userq_create_object(struct amdgpu_userq_mgr *uq_mgr, | ||
| 130 | struct amdgpu_userq_obj *userq_obj, | ||
| 131 | int size); | ||
| 132 | |||
| 133 | void amdgpu_userq_destroy_object(struct amdgpu_userq_mgr *uq_mgr, | ||
| 134 | struct amdgpu_userq_obj *userq_obj); | ||
| 135 | |||
| 136 | void amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr); | 159 | void amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr); |
| 137 | 160 | ||
| 138 | void amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *userq_mgr, | 161 | void amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *userq_mgr, |
| 139 | struct amdgpu_eviction_fence_mgr *evf_mgr); | 162 | struct amdgpu_eviction_fence_mgr *evf_mgr); |
| 140 | 163 | ||
| 141 | uint64_t amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, | ||
| 142 | struct amdgpu_db_info *db_info, | ||
| 143 | struct drm_file *filp); | ||
| 144 | |||
| 145 | u32 amdgpu_userq_get_supported_ip_mask(struct amdgpu_device *adev); | 164 | u32 amdgpu_userq_get_supported_ip_mask(struct amdgpu_device *adev); |
| 146 | bool amdgpu_userq_enabled(struct drm_device *dev); | 165 | bool amdgpu_userq_enabled(struct drm_device *dev); |
| 147 | 166 | ||
| @@ -156,10 +175,12 @@ void amdgpu_userq_reset_work(struct work_struct *work); · 8 unmodified lines | |||
| 156 | void amdgpu_userq_pre_reset(struct amdgpu_device *adev); | 175 | void amdgpu_userq_pre_reset(struct amdgpu_device *adev); |
| 157 | int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost); | 176 | int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost); |
| 158 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue); | 177 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue); |
| 178 | void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell); | ||
| 159 | 179 | ||
| 160 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, | 180 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, |
| 161 | struct amdgpu_usermode_queue *queue, | 181 | struct amdgpu_usermode_queue *queue, |
| 162 | u64 addr, u64 expected_size); | 182 | u64 addr, u64 expected_size, u64 *va_out); |
| 183 | |||
| 163 | void amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev, | 184 | void amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev, |
| 164 | struct amdgpu_bo_va_mapping *mapping, | 185 | struct amdgpu_bo_va_mapping *mapping, |
| 165 | uint64_t saddr); | 186 | uint64_t saddr); |
| @@ -32,29 +32,9 @@ · 31 unmodified lines | |||
| 32 | #include "amdgpu.h" | 32 | #include "amdgpu.h" |
| 33 | #include "amdgpu_userq_fence.h" | 33 | #include "amdgpu_userq_fence.h" |
| 34 | 34 | ||
| 35 | static const struct dma_fence_ops amdgpu_userq_fence_ops; | ||
| 36 | static struct kmem_cache *amdgpu_userq_fence_slab; | ||
| 37 | |||
| 38 | #define AMDGPU_USERQ_MAX_HANDLES (1U << 16) | 35 | #define AMDGPU_USERQ_MAX_HANDLES (1U << 16) |
| 39 | 36 | ||
| 40 | int amdgpu_userq_fence_slab_init(void) | 37 | static const struct dma_fence_ops amdgpu_userq_fence_ops; |
| 41 | { | ||
| 42 | amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence", | ||
| 43 | sizeof(struct amdgpu_userq_fence), | ||
| 44 | 0, | ||
| 45 | SLAB_HWCACHE_ALIGN, | ||
| 46 | NULL); | ||
| 47 | if (!amdgpu_userq_fence_slab) | ||
| 48 | return -ENOMEM; | ||
| 49 | |||
| 50 | return 0; | ||
| 51 | } | ||
| 52 | |||
| 53 | void amdgpu_userq_fence_slab_fini(void) | ||
| 54 | { | ||
| 55 | rcu_barrier(); | ||
| 56 | kmem_cache_destroy(amdgpu_userq_fence_slab); | ||
| 57 | } | ||
| 58 | 38 | ||
| 59 | static inline struct amdgpu_userq_fence *to_amdgpu_userq_fence(struct dma_fence *f) | 39 | static inline struct amdgpu_userq_fence *to_amdgpu_userq_fence(struct dma_fence *f) |
| 60 | { | 40 | { |
| @@ -141,6 +121,7 @@ amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq) · 80 unmodified lines | |||
| 141 | userq->last_fence = NULL; | 121 | userq->last_fence = NULL; |
| 142 | amdgpu_userq_walk_and_drop_fence_drv(&userq->fence_drv_xa); | 122 | amdgpu_userq_walk_and_drop_fence_drv(&userq->fence_drv_xa); |
| 143 | xa_destroy(&userq->fence_drv_xa); | 123 | xa_destroy(&userq->fence_drv_xa); |
| 124 | mutex_destroy(&userq->fence_drv_lock); | ||
| 144 | /* Drop the queue's ownership reference to fence_drv explicitly */ | 125 | /* Drop the queue's ownership reference to fence_drv explicitly */ |
| 145 | amdgpu_userq_fence_driver_put(userq->fence_drv); | 126 | amdgpu_userq_fence_driver_put(userq->fence_drv); |
| 146 | } | 127 | } |
| @@ -154,7 +135,14 @@ amdgpu_userq_fence_put_fence_drv_array(struct amdgpu_userq_fence *userq_fence) · 7 unmodified lines | |||
| 154 | userq_fence->fence_drv_array_count = 0; | 135 | userq_fence->fence_drv_array_count = 0; |
| 155 | } | 136 | } |
| 156 | 137 | ||
| 157 | void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv) | 138 | /* |
| 139 | * Returns: | ||
| 140 | * -ENOENT when no fences were processes | ||
| 141 | * 1 when more fences are pending | ||
| 142 | * 0 when no fences are pending any more | ||
| 143 | */ | ||
| 144 | int | ||
| 145 | amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv) | ||
| 158 | { | 146 | { |
| 159 | struct amdgpu_userq_fence *userq_fence, *tmp; | 147 | struct amdgpu_userq_fence *userq_fence, *tmp; |
| 160 | LIST_HEAD(to_be_signaled); | 148 | LIST_HEAD(to_be_signaled); |
| @@ -162,9 +150,6 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 1 unmodified line | |||
| 162 | unsigned long flags; | 150 | unsigned long flags; |
| 163 | u64 rptr; | 151 | u64 rptr; |
| 164 | 152 | ||
| 165 | if (!fence_drv) | ||
| 166 | return; | ||
| 167 | |||
| 168 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); | 153 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); |
| 169 | rptr = amdgpu_userq_fence_read(fence_drv); | 154 | rptr = amdgpu_userq_fence_read(fence_drv); |
| 170 | 155 | ||
| @@ -177,6 +162,9 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 6 unmodified lines | |||
| 177 | &userq_fence->link); | 162 | &userq_fence->link); |
| 178 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); | 163 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); |
| 179 | 164 | ||
| 165 | if (list_empty(&to_be_signaled)) | ||
| 166 | return -ENOENT; | ||
| 167 | |||
| 180 | list_for_each_entry_safe(userq_fence, tmp, &to_be_signaled, link) { | 168 | list_for_each_entry_safe(userq_fence, tmp, &to_be_signaled, link) { |
| 181 | fence = &userq_fence->base; | 169 | fence = &userq_fence->base; |
| 182 | list_del_init(&userq_fence->link); | 170 | list_del_init(&userq_fence->link); |
| @@ -188,6 +176,8 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 5 unmodified lines | |||
| 188 | dma_fence_put(fence); | 176 | dma_fence_put(fence); |
| 189 | } | 177 | } |
| 190 | 178 | ||
| 179 | /* That doesn't need to be accurate so no locking */ | ||
| 180 | return list_empty(&fence_drv->fences) ? 0 : 1; | ||
| 191 | } | 181 | } |
| 192 | 182 | ||
| 193 | void amdgpu_userq_fence_driver_destroy(struct kref *ref) | 183 | void amdgpu_userq_fence_driver_destroy(struct kref *ref) |
| @@ -229,80 +219,84 @@ void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv) · 35 unmodified lines | |||
| 229 | kref_put(&fence_drv->refcount, amdgpu_userq_fence_driver_destroy); | 219 | kref_put(&fence_drv->refcount, amdgpu_userq_fence_driver_destroy); |
| 230 | } | 220 | } |
| 231 | 221 | ||
| 232 | static int amdgpu_userq_fence_alloc(struct amdgpu_userq_fence **userq_fence) | 222 | static int amdgpu_userq_fence_alloc(struct amdgpu_usermode_queue *userq, |
| 223 | struct amdgpu_userq_fence **pfence) | ||
| 233 | { | 224 | { |
| 234 | *userq_fence = kmem_cache_alloc(amdgpu_userq_fence_slab, GFP_ATOMIC); | 225 | struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv; |
| 235 | return *userq_fence ? 0 : -ENOMEM; | 226 | struct amdgpu_userq_fence *userq_fence; |
| 227 | void *entry; | ||
| 228 | |||
| 229 | userq_fence = kmalloc(sizeof(*userq_fence), GFP_KERNEL); | ||
| 230 | if (!userq_fence) | ||
| 231 | return -ENOMEM; | ||
| 232 | |||
| 233 | /* | ||
| 234 | * Get the next unused entry, since we fill from the start this can be | ||
| 235 | * used as size to allocate the array. | ||
| 236 | */ | ||
| 237 | mutex_lock(&userq->fence_drv_lock); | ||
| 238 | XA_STATE(xas, &userq->fence_drv_xa, 0); | ||
| 239 | |||
| 240 | rcu_read_lock(); | ||
| 241 | do { | ||
| 242 | entry = xas_find_marked(&xas, ULONG_MAX, XA_FREE_MARK); | ||
| 243 | } while (xas_retry(&xas, entry)); | ||
| 244 | rcu_read_unlock(); | ||
| 245 | |||
| 246 | userq_fence->fence_drv_array = kvmalloc_array(xas.xa_index, | ||
| 247 | sizeof(fence_drv), | ||
| 248 | GFP_KERNEL); | ||
| 249 | if (!userq_fence->fence_drv_array) { | ||
| 250 | mutex_unlock(&userq->fence_drv_lock); | ||
| 251 | kfree(userq_fence); | ||
| 252 | return -ENOMEM; | ||
| 253 | } | ||
| 254 | |||
| 255 | userq_fence->fence_drv_array_count = xas.xa_index; | ||
| 256 | xa_extract(&userq->fence_drv_xa, (void **)userq_fence->fence_drv_array, | ||
| 257 | 0, ULONG_MAX, xas.xa_index, XA_PRESENT); | ||
| 258 | xa_destroy(&userq->fence_drv_xa); | ||
| 259 | |||
| 260 | mutex_unlock(&userq->fence_drv_lock); | ||
| 261 | |||
| 262 | amdgpu_userq_fence_driver_get(fence_drv); | ||
| 263 | userq_fence->fence_drv = fence_drv; | ||
| 264 | |||
| 265 | *pfence = userq_fence; | ||
| 266 | return 0; | ||
| 236 | } | 267 | } |
| 237 | 268 | ||
| 238 | static int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq, | 269 | static void amdgpu_userq_fence_init(struct amdgpu_usermode_queue *userq, |
| 239 | struct amdgpu_userq_fence *userq_fence, | 270 | struct amdgpu_userq_fence *fence, |
| 240 | u64 seq, struct dma_fence **f) | 271 | u64 seq) |
| 241 | { | 272 | { |
| 242 | struct amdgpu_userq_fence_driver *fence_drv; | 273 | struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv; |
| 243 | struct dma_fence *fence; | ||
| 244 | unsigned long flags; | 274 | unsigned long flags; |
| 245 | bool signaled = false; | 275 | bool signaled = false; |
| 246 | 276 | ||
| 247 | fence_drv = userq->fence_drv; | 277 | spin_lock_init(&fence->lock); |
| 248 | if (!fence_drv) | 278 | dma_fence_init64(&fence->base, &amdgpu_userq_fence_ops, &fence->lock, |
| 249 | return -EINVAL; | ||
| 250 | |||
| 251 | spin_lock_init(&userq_fence->lock); | ||
| 252 | INIT_LIST_HEAD(&userq_fence->link); | ||
| 253 | fence = &userq_fence->base; | ||
| 254 | userq_fence->fence_drv = fence_drv; | ||
| 255 | |||
| 256 | dma_fence_init64(fence, &amdgpu_userq_fence_ops, &userq_fence->lock, | ||
| 257 | fence_drv->context, seq); | 279 | fence_drv->context, seq); |
| 258 | 280 | ||
| 259 | amdgpu_userq_fence_driver_get(fence_drv); | 281 | /* Make sure the fence is visible to the hang detect worker */ |
| 260 | dma_fence_get(fence); | 282 | dma_fence_put(userq->last_fence); |
| 261 | 283 | userq->last_fence = dma_fence_get(&fence->base); | |
| 262 | if (!xa_empty(&userq->fence_drv_xa)) { | ||
| 263 | struct amdgpu_userq_fence_driver *stored_fence_drv; | ||
| 264 | unsigned long index, count = 0; | ||
| 265 | int i = 0; | ||
| 266 | |||
| 267 | xa_lock(&userq->fence_drv_xa); | ||
| 268 | xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv) | ||
| 269 | count++; | ||
| 270 | |||
| 271 | userq_fence->fence_drv_array = | ||
| 272 | kvmalloc_objs(struct amdgpu_userq_fence_driver *, count, | ||
| 273 | GFP_ATOMIC); | ||
| 274 | |||
| 275 | if (userq_fence->fence_drv_array) { | ||
| 276 | xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv) { | ||
| 277 | userq_fence->fence_drv_array[i] = stored_fence_drv; | ||
| 278 | __xa_erase(&userq->fence_drv_xa, index); | ||
| 279 | i++; | ||
| 280 | } | ||
| 281 | } | ||
| 282 | |||
| 283 | userq_fence->fence_drv_array_count = i; | ||
| 284 | xa_unlock(&userq->fence_drv_xa); | ||
| 285 | } else { | ||
| 286 | userq_fence->fence_drv_array = NULL; | ||
| 287 | userq_fence->fence_drv_array_count = 0; | ||
| 288 | } | ||
| 289 | 284 | ||
| 290 | /* Check if hardware has already processed the job */ | 285 | /* Check if hardware has already processed the fence */ |
| 291 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); | 286 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); |
| 292 | if (!dma_fence_is_signaled(fence)) { | 287 | if (!dma_fence_is_signaled(&fence->base)) { |
| 293 | list_add_tail(&userq_fence->link, &fence_drv->fences); | 288 | dma_fence_get(&fence->base); |
| 289 | list_add_tail(&fence->link, &fence_drv->fences); | ||
| 294 | } else { | 290 | } else { |
| 291 | INIT_LIST_HEAD(&fence->link); | ||
| 295 | signaled = true; | 292 | signaled = true; |
| 296 | dma_fence_put(fence); | ||
| 297 | } | 293 | } |
| 298 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); | 294 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); |
| 299 | 295 | ||
| 300 | if (signaled) | 296 | if (signaled) |
| 301 | amdgpu_userq_fence_put_fence_drv_array(userq_fence); | 297 | amdgpu_userq_fence_put_fence_drv_array(fence); |
| 302 | 298 | else | |
| 303 | *f = fence; | 299 | amdgpu_userq_start_hang_detect_work(userq); |
| 304 | |||
| 305 | return 0; | ||
| 306 | } | 300 | } |
| 307 | 301 | ||
| 308 | static const char *amdgpu_userq_fence_get_driver_name(struct dma_fence *f) | 302 | static const char *amdgpu_userq_fence_get_driver_name(struct dma_fence *f) |
| @@ -342,7 +336,7 @@ static void amdgpu_userq_fence_free(struct rcu_head *rcu) · 33 unmodified lines | |||
| 342 | amdgpu_userq_fence_driver_put(fence_drv); | 336 | amdgpu_userq_fence_driver_put(fence_drv); |
| 343 | 337 | ||
| 344 | kvfree(userq_fence->fence_drv_array); | 338 | kvfree(userq_fence->fence_drv_array); |
| 345 | kmem_cache_free(amdgpu_userq_fence_slab, userq_fence); | 339 | kfree(userq_fence); |
| 346 | } | 340 | } |
| 347 | 341 | ||
| 348 | static void amdgpu_userq_fence_release(struct dma_fence *f) | 342 | static void amdgpu_userq_fence_release(struct dma_fence *f) |
| @@ -376,56 +370,48 @@ static int amdgpu_userq_fence_read_wptr(struct amdgpu_device *adev, · 27 unmodified lines | |||
| 376 | { | 370 | { |
| 377 | struct amdgpu_bo_va_mapping *mapping; | 371 | struct amdgpu_bo_va_mapping *mapping; |
| 378 | struct amdgpu_bo *bo; | 372 | struct amdgpu_bo *bo; |
| 373 | struct drm_exec exec; | ||
| 379 | u64 addr, *ptr; | 374 | u64 addr, *ptr; |
| 380 | int r; | 375 | int ret; |
| 381 | |||
| 382 | r = amdgpu_bo_reserve(queue->vm->root.bo, false); | ||
| 383 | if (r) | ||
| 384 | return r; | ||
| 385 | 376 | ||
| 386 | addr = queue->userq_prop->wptr_gpu_addr; | 377 | addr = queue->userq_prop->wptr_gpu_addr; |
| 387 | addr &= AMDGPU_GMC_HOLE_MASK; | 378 | addr &= AMDGPU_GMC_HOLE_MASK; |
| 388 | 379 | ||
| 389 | mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, addr >> PAGE_SHIFT); | 380 | drm_exec_init(&exec, DRM_EXEC_IGNORE_DUPLICATES, 2); |
| 390 | if (!mapping) { | 381 | drm_exec_until_all_locked(&exec) { |
| 391 | amdgpu_bo_unreserve(queue->vm->root.bo); | 382 | ret = amdgpu_vm_lock_pd(queue->vm, &exec, 1); |
| 392 | DRM_ERROR("Failed to lookup amdgpu_bo_va_mapping\n"); | 383 | drm_exec_retry_on_contention(&exec); |
| 393 | return -EINVAL; | 384 | if (unlikely(ret)) |
| 394 | } | 385 | goto lock_error; |
| 386 | |||
| 387 | mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, addr >> PAGE_SHIFT); | ||
| 388 | if (!mapping) { | ||
| 389 | ret = -EINVAL; | ||
| 390 | goto lock_error; | ||
| 391 | } | ||
| 395 | 392 | ||
| 396 | bo = amdgpu_bo_ref(mapping->bo_va->base.bo); | 393 | ret = drm_exec_lock_obj(&exec, &mapping->bo_va->base.bo->tbo.base); |
| 397 | amdgpu_bo_unreserve(queue->vm->root.bo); | 394 | drm_exec_retry_on_contention(&exec); |
| 398 | r = amdgpu_bo_reserve(bo, true); | 395 | if (unlikely(ret)) |
| 399 | if (r) { | 396 | goto lock_error; |
| 400 | amdgpu_bo_unref(&bo); | ||
| 401 | DRM_ERROR("Failed to reserve userqueue wptr bo"); | ||
| 402 | return r; | ||
| 403 | } | 397 | } |
| 404 | 398 | ||
| 405 | r = amdgpu_bo_kmap(bo, (void **)&ptr); | 399 | bo = mapping->bo_va->base.bo; |
| 406 | if (r) { | 400 | ret = amdgpu_bo_kmap(bo, (void **)&ptr); |
| 401 | if (ret) { | ||
| 407 | DRM_ERROR("Failed mapping the userqueue wptr bo"); | 402 | DRM_ERROR("Failed mapping the userqueue wptr bo"); |
| 408 | goto map_error; | 403 | goto lock_error; |
| 409 | } | 404 | } |
| 410 | 405 | ||
| 411 | *wptr = le64_to_cpu(*ptr); | 406 | *wptr = le64_to_cpu(*ptr); |
| 412 | 407 | ||
| 413 | amdgpu_bo_kunmap(bo); | 408 | amdgpu_bo_kunmap(bo); |
| 414 | amdgpu_bo_unreserve(bo); | 409 | drm_exec_fini(&exec); |
| 415 | amdgpu_bo_unref(&bo); | ||
| 416 | |||
| 417 | return 0; | 410 | return 0; |
| 418 | 411 | ||
| 419 | map_error: | 412 | lock_error: |
| 420 | amdgpu_bo_unreserve(bo); | 413 | drm_exec_fini(&exec); |
| 421 | amdgpu_bo_unref(&bo); | 414 | return ret; |
| 422 | |||
| 423 | return r; | ||
| 424 | } | ||
| 425 | |||
| 426 | static void amdgpu_userq_fence_cleanup(struct dma_fence *fence) | ||
| 427 | { | ||
| 428 | dma_fence_put(fence); | ||
| 429 | } | 415 | } |
| 430 | 416 | ||
| 431 | static void | 417 | static void |
| @@ -471,13 +457,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 39 unmodified lines | |||
| 471 | const unsigned int num_read_bo_handles = args->num_bo_read_handles; | 457 | const unsigned int num_read_bo_handles = args->num_bo_read_handles; |
| 472 | struct amdgpu_fpriv *fpriv = filp->driver_priv; | 458 | struct amdgpu_fpriv *fpriv = filp->driver_priv; |
| 473 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; | 459 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; |
| 460 | |||
| 474 | struct drm_gem_object **gobj_write, **gobj_read; | 461 | struct drm_gem_object **gobj_write, **gobj_read; |
| 475 | u32 *syncobj_handles, num_syncobj_handles; | 462 | u32 *syncobj_handles, num_syncobj_handles; |
| 476 | struct amdgpu_userq_fence *userq_fence; | 463 | struct amdgpu_usermode_queue *queue; |
| 477 | struct amdgpu_usermode_queue *queue = NULL; | 464 | struct amdgpu_userq_fence *fence; |
| 478 | struct drm_syncobj **syncobj = NULL; | 465 | struct drm_syncobj **syncobj; |
| 479 | struct dma_fence *fence; | ||
| 480 | struct drm_exec exec; | 466 | struct drm_exec exec; |
| 467 | void __user *ptr; | ||
| 481 | int r, i, entry; | 468 | int r, i, entry; |
| 482 | u64 wptr; | 469 | u64 wptr; |
| 483 | 470 | ||
| @@ -489,13 +476,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 5 unmodified lines | |||
| 489 | return -EINVAL; | 476 | return -EINVAL; |
| 490 | 477 | ||
| 491 | num_syncobj_handles = args->num_syncobj_handles; | 478 | num_syncobj_handles = args->num_syncobj_handles; |
| 492 | syncobj_handles = memdup_array_user(u64_to_user_ptr(args->syncobj_handles), | 479 | ptr = u64_to_user_ptr(args->syncobj_handles); |
| 493 | num_syncobj_handles, sizeof(u32)); | 480 | syncobj_handles = memdup_array_user(ptr, num_syncobj_handles, |
| 481 | sizeof(u32)); | ||
| 494 | if (IS_ERR(syncobj_handles)) | 482 | if (IS_ERR(syncobj_handles)) |
| 495 | return PTR_ERR(syncobj_handles); | 483 | return PTR_ERR(syncobj_handles); |
| 496 | 484 | ||
| 497 | /* Array of pointers to the looked up syncobjs */ | 485 | syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), |
| 498 | syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), GFP_KERNEL); | 486 | GFP_KERNEL); |
| 499 | if (!syncobj) { | 487 | if (!syncobj) { |
| 500 | r = -ENOMEM; | 488 | r = -ENOMEM; |
| 501 | goto free_syncobj_handles; | 489 | goto free_syncobj_handles; |
| @@ -509,21 +497,17 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 7 unmodified lines | |||
| 509 | } | 497 | } |
| 510 | } | 498 | } |
| 511 | 499 | ||
| 512 | r = drm_gem_objects_lookup(filp, | 500 | ptr = u64_to_user_ptr(args->bo_read_handles); |
| 513 | u64_to_user_ptr(args->bo_read_handles), | 501 | r = drm_gem_objects_lookup(filp, ptr, num_read_bo_handles, &gobj_read); |
| 514 | num_read_bo_handles, | ||
| 515 | &gobj_read); | ||
| 516 | if (r) | 502 | if (r) |
| 517 | goto free_syncobj; | 503 | goto free_syncobj; |
| 518 | 504 | ||
| 519 | r = drm_gem_objects_lookup(filp, | 505 | ptr = u64_to_user_ptr(args->bo_write_handles); |
| 520 | u64_to_user_ptr(args->bo_write_handles), | 506 | r = drm_gem_objects_lookup(filp, ptr, num_write_bo_handles, |
| 521 | num_write_bo_handles, | ||
| 522 | &gobj_write); | 507 | &gobj_write); |
| 523 | if (r) | 508 | if (r) |
| 524 | goto put_gobj_read; | 509 | goto put_gobj_read; |
| 525 | 510 | ||
| 526 | /* Retrieve the user queue */ | ||
| 527 | queue = amdgpu_userq_get(userq_mgr, args->queue_id); | 511 | queue = amdgpu_userq_get(userq_mgr, args->queue_id); |
| 528 | if (!queue) { | 512 | if (!queue) { |
| 529 | r = -ENOENT; | 513 | r = -ENOENT; |
| @@ -532,73 +516,61 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 2 unmodified lines | |||
| 532 | 516 | ||
| 533 | r = amdgpu_userq_fence_read_wptr(adev, queue, &wptr); | 517 | r = amdgpu_userq_fence_read_wptr(adev, queue, &wptr); |
| 534 | if (r) | 518 | if (r) |
| 535 | goto put_gobj_write; | 519 | goto put_queue; |
| 536 | 520 | ||
| 537 | r = amdgpu_userq_fence_alloc(&userq_fence); | 521 | r = amdgpu_userq_fence_alloc(queue, &fence); |
| 538 | if (r) | 522 | if (r) |
| 539 | goto put_gobj_write; | 523 | goto put_queue; |
| 540 | 524 | ||
| 541 | /* We are here means UQ is active, make sure the eviction fence is valid */ | 525 | /* We are here means UQ is active, make sure the eviction fence is valid */ |
| 542 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); | 526 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); |
| 543 | 527 | ||
| 544 | /* Create a new fence */ | 528 | /* Create the new fence */ |
| 545 | r = amdgpu_userq_fence_create(queue, userq_fence, wptr, &fence); | 529 | amdgpu_userq_fence_init(queue, fence, wptr); |
| 546 | if (r) { | ||
| 547 | mutex_unlock(&userq_mgr->userq_mutex); | ||
| 548 | kmem_cache_free(amdgpu_userq_fence_slab, userq_fence); | ||
| 549 | goto put_gobj_write; | ||
| 550 | } | ||
| 551 | 530 | ||
| 552 | dma_fence_put(queue->last_fence); | ||
| 553 | queue->last_fence = dma_fence_get(fence); | ||
| 554 | amdgpu_userq_start_hang_detect_work(queue); | ||
| 555 | mutex_unlock(&userq_mgr->userq_mutex); | 531 | mutex_unlock(&userq_mgr->userq_mutex); |
| 556 | 532 | ||
| 533 | /* | ||
| 534 | * This needs to come after the fence is created since | ||
| 535 | * amdgpu_userq_ensure_ev_fence() can't be called while holding the resv | ||
| 536 | * locks. | ||
| 537 | */ | ||
| 557 | drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, | 538 | drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, |
| 558 | (num_read_bo_handles + num_write_bo_handles)); | 539 | (num_read_bo_handles + num_write_bo_handles)); |
| 559 | 540 | ||
| 560 | /* Lock all BOs with retry handling */ | ||
| 561 | drm_exec_until_all_locked(&exec) { | 541 | drm_exec_until_all_locked(&exec) { |
| 562 | r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); | 542 | r = drm_exec_prepare_array(&exec, gobj_read, |
| 543 | num_read_bo_handles, 1); | ||
| 563 | drm_exec_retry_on_contention(&exec); | 544 | drm_exec_retry_on_contention(&exec); |
| 564 | if (r) { | 545 | if (r) |
| 565 | amdgpu_userq_fence_cleanup(fence); | ||
| 566 | goto exec_fini; | 546 | goto exec_fini; |
| 567 | } | ||
| 568 | 547 | ||
| 569 | r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); | 548 | r = drm_exec_prepare_array(&exec, gobj_write, |
| 549 | num_write_bo_handles, 1); | ||
| 570 | drm_exec_retry_on_contention(&exec); | 550 | drm_exec_retry_on_contention(&exec); |
| 571 | if (r) { | 551 | if (r) |
| 572 | amdgpu_userq_fence_cleanup(fence); | ||
| 573 | goto exec_fini; | 552 | goto exec_fini; |
| 574 | } | ||
| 575 | } | 553 | } |
| 576 | 554 | ||
| 577 | for (i = 0; i < num_read_bo_handles; i++) { | 555 | /* And publish the new fence in the BOs and syncobj */ |
| 578 | if (!gobj_read || !gobj_read[i]->resv) | 556 | for (i = 0; i < num_read_bo_handles; i++) |
| 579 | continue; | 557 | dma_resv_add_fence(gobj_read[i]->resv, &fence->base, |
| 580 | |||
| 581 | dma_resv_add_fence(gobj_read[i]->resv, fence, | ||
| 582 | DMA_RESV_USAGE_READ); | 558 | DMA_RESV_USAGE_READ); |
| 583 | } | ||
| 584 | 559 | ||
| 585 | for (i = 0; i < num_write_bo_handles; i++) { | 560 | for (i = 0; i < num_write_bo_handles; i++) |
| 586 | if (!gobj_write || !gobj_write[i]->resv) | 561 | dma_resv_add_fence(gobj_write[i]->resv, &fence->base, |
| 587 | continue; | ||
| 588 | |||
| 589 | dma_resv_add_fence(gobj_write[i]->resv, fence, | ||
| 590 | DMA_RESV_USAGE_WRITE); | 562 | DMA_RESV_USAGE_WRITE); |
| 591 | } | ||
| 592 | 563 | ||
| 593 | /* Add the created fence to syncobj/BO's */ | ||
| 594 | for (i = 0; i < num_syncobj_handles; i++) | 564 | for (i = 0; i < num_syncobj_handles; i++) |
| 595 | drm_syncobj_replace_fence(syncobj[i], fence); | 565 | drm_syncobj_replace_fence(syncobj[i], &fence->base); |
| 596 | 566 | ||
| 567 | exec_fini: | ||
| 597 | /* drop the reference acquired in fence creation function */ | 568 | /* drop the reference acquired in fence creation function */ |
| 598 | dma_fence_put(fence); | 569 | dma_fence_put(&fence->base); |
| 599 | 570 | ||
| 600 | exec_fini: | ||
| 601 | drm_exec_fini(&exec); | 571 | drm_exec_fini(&exec); |
| 572 | put_queue: | ||
| 573 | amdgpu_userq_put(queue); | ||
| 602 | put_gobj_write: | 574 | put_gobj_write: |
| 603 | for (i = 0; i < num_write_bo_handles; i++) | 575 | for (i = 0; i < num_write_bo_handles; i++) |
| 604 | drm_gem_object_put(gobj_write[i]); | 576 | drm_gem_object_put(gobj_write[i]); |
| @@ -609,15 +581,11 @@ put_gobj_read: · 4 unmodified lines | |||
| 609 | kvfree(gobj_read); | 581 | kvfree(gobj_read); |
| 610 | free_syncobj: | 582 | free_syncobj: |
| 611 | while (entry-- > 0) | 583 | while (entry-- > 0) |
| 612 | if (syncobj[entry]) | 584 | drm_syncobj_put(syncobj[entry]); |
| 613 | drm_syncobj_put(syncobj[entry]); | ||
| 614 | kfree(syncobj); | 585 | kfree(syncobj); |
| 615 | free_syncobj_handles: | 586 | free_syncobj_handles: |
| 616 | kfree(syncobj_handles); | 587 | kfree(syncobj_handles); |
| 617 | 588 | ||
| 618 | if (queue) | ||
| 619 | amdgpu_userq_put(queue); | ||
| 620 | |||
| 621 | return r; | 589 | return r; |
| 622 | } | 590 | } |
| 623 | 591 | ||
| @@ -892,8 +860,10 @@ amdgpu_userq_wait_return_fence_info(struct drm_file *filp, · 268 unmodified lines | |||
| 892 | * Otherwise, we would gather those references until we don't | 860 | * Otherwise, we would gather those references until we don't |
| 893 | * have any more space left and crash. | 861 | * have any more space left and crash. |
| 894 | */ | 862 | */ |
| 863 | mutex_lock(&waitq->fence_drv_lock); | ||
| 895 | r = xa_alloc(&waitq->fence_drv_xa, &index, fence_drv, | 864 | r = xa_alloc(&waitq->fence_drv_xa, &index, fence_drv, |
| 896 | xa_limit_32b, GFP_KERNEL); | 865 | xa_limit_32b, GFP_KERNEL); |
| 866 | mutex_unlock(&waitq->fence_drv_lock); | ||
| 897 | if (r) | 867 | if (r) |
| 898 | goto put_waitq; | 868 | goto put_waitq; |
| 899 | 869 | ||
| @@ -58,15 +58,12 @@ struct amdgpu_userq_fence_driver { · 57 unmodified lines | |||
| 58 | char timeline_name[TASK_COMM_LEN]; | 58 | char timeline_name[TASK_COMM_LEN]; |
| 59 | }; | 59 | }; |
| 60 | 60 | ||
| 61 | int amdgpu_userq_fence_slab_init(void); | ||
| 62 | void amdgpu_userq_fence_slab_fini(void); | ||
| 63 | |||
| 64 | void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv); | 61 | void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv); |
| 65 | void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv); | 62 | void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv); |
| 66 | int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev, | 63 | int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev, |
| 67 | struct amdgpu_userq_fence_driver **fence_drv_req); | 64 | struct amdgpu_userq_fence_driver **fence_drv_req); |
| 68 | void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq); | 65 | void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq); |
| 69 | void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv); | 66 | int amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv); |
| 70 | void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq); | 67 | void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq); |
| 71 | void amdgpu_userq_fence_driver_destroy(struct kref *ref); | 68 | void amdgpu_userq_fence_driver_destroy(struct kref *ref); |
| 72 | int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, | 69 | int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, |
| @@ -1631,6 +1631,7 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, · 1630 unmodified lines | |||
| 1631 | { | 1631 | { |
| 1632 | struct amdgpu_bo_va *bo_va; | 1632 | struct amdgpu_bo_va *bo_va; |
| 1633 | struct dma_resv *resv; | 1633 | struct dma_resv *resv; |
| 1634 | struct amdgpu_bo *bo; | ||
| 1634 | bool clear, unlock; | 1635 | bool clear, unlock; |
| 1635 | int r; | 1636 | int r; |
| 1636 | 1637 | ||
| @@ -1650,11 +1651,13 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, · 13 unmodified lines | |||
| 1650 | while (!list_empty(&vm->invalidated)) { | 1651 | while (!list_empty(&vm->invalidated)) { |
| 1651 | bo_va = list_first_entry(&vm->invalidated, struct amdgpu_bo_va, | 1652 | bo_va = list_first_entry(&vm->invalidated, struct amdgpu_bo_va, |
| 1652 | base.vm_status); | 1653 | base.vm_status); |
| 1653 | resv = bo_va->base.bo->tbo.base.resv; | 1654 | bo = bo_va->base.bo; |
| 1655 | resv = bo->tbo.base.resv; | ||
| 1654 | spin_unlock(&vm->status_lock); | 1656 | spin_unlock(&vm->status_lock); |
| 1655 | 1657 | ||
| 1656 | /* Try to reserve the BO to avoid clearing its ptes */ | 1658 | /* Try to reserve the BO to avoid clearing its ptes */ |
| 1657 | if (!adev->debug_vm && dma_resv_trylock(resv)) { | 1659 | if (!adev->debug_vm && !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && |
| 1660 | dma_resv_trylock(resv)) { | ||
| 1658 | clear = false; | 1661 | clear = false; |
| 1659 | unlock = true; | 1662 | unlock = true; |
| 1660 | /* The caller is already holding the reservation lock */ | 1663 | /* The caller is already holding the reservation lock */ |
| @@ -2002,7 +2005,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, · 341 unmodified lines | |||
| 2002 | * during user requests GEM unmap IOCTL except for forcing the unmap | 2005 | * during user requests GEM unmap IOCTL except for forcing the unmap |
| 2003 | * from user space. | 2006 | * from user space. |
| 2004 | */ | 2007 | */ |
| 2005 | if (unlikely(atomic_read(&bo_va->userq_va_mapped) > 0)) | 2008 | if (unlikely(bo_va->userq_va_mapped)) |
| 2006 | amdgpu_userq_gem_va_unmap_validate(adev, mapping, saddr); | 2009 | amdgpu_userq_gem_va_unmap_validate(adev, mapping, saddr); |
| 2007 | 2010 | ||
| 2008 | list_del(&mapping->list); | 2011 | list_del(&mapping->list); |
| @@ -3023,11 +3026,22 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · 1014 unmodified lines | |||
| 3023 | 3026 | ||
| 3024 | is_compute_context = vm->is_compute_context; | 3027 | is_compute_context = vm->is_compute_context; |
| 3025 | 3028 | ||
| 3026 | if (is_compute_context && !svm_range_restore_pages(adev, pasid, vmid, | 3029 | if (is_compute_context) { |
| 3027 | node_id, addr >> PAGE_SHIFT, ts, write_fault)) { | 3030 | /* Unreserve root since svm_range_restore_pages might try to reserve it. */ |
| 3031 | /* TODO: rework svm_range_restore_pages so that this isn't necessary. */ | ||
| 3028 | amdgpu_bo_unreserve(root); | 3032 | amdgpu_bo_unreserve(root); |
| 3033 | |||
| 3034 | if (!svm_range_restore_pages(adev, pasid, vmid, | ||
| 3035 | node_id, addr >> PAGE_SHIFT, ts, write_fault)) { | ||
| 3036 | amdgpu_bo_unref(&root); | ||
| 3037 | return true; | ||
| 3038 | } | ||
| 3029 | amdgpu_bo_unref(&root); | 3039 | amdgpu_bo_unref(&root); |
| 3030 | return true; | 3040 | |
| 3041 | /* Re-acquire the VM lock, could be that the VM was freed in between. */ | ||
| 3042 | vm = amdgpu_vm_lock_by_pasid(adev, &root, pasid); | ||
| 3043 | if (!vm) | ||
| 3044 | return false; | ||
| 3031 | } | 3045 | } |
| 3032 | 3046 | ||
| 3033 | addr /= AMDGPU_GPU_PAGE_SIZE; | 3047 | addr /= AMDGPU_GPU_PAGE_SIZE; |
| @@ -562,6 +562,11 @@ static void vpe_ring_emit_fence(struct amdgpu_ring *ring, uint64_t addr, · 561 unmodified lines | |||
| 562 | amdgpu_ring_write(ring, 0); | 562 | amdgpu_ring_write(ring, 0); |
| 563 | } | 563 | } |
| 564 | 564 | ||
| 565 | /* WA: Force sync after TRAP to avoid VPE1 fail to power off */ | ||
| 566 | if (ring->adev->vpe.collaborate_mode) { | ||
| 567 | amdgpu_ring_write(ring, VPE_CMD_HEADER(VPE_CMD_OPCODE_COLLAB_SYNC, 0)); | ||
| 568 | amdgpu_ring_write(ring, 0xabcd); | ||
| 569 | } | ||
| 565 | } | 570 | } |
| 566 | 571 | ||
| 567 | static void vpe_ring_emit_pipeline_sync(struct amdgpu_ring *ring) | 572 | static void vpe_ring_emit_pipeline_sync(struct amdgpu_ring *ring) |
| @@ -968,7 +973,7 @@ static const struct amdgpu_ring_funcs vpe_ring_funcs = { · 400 unmodified lines | |||
| 968 | .emit_frame_size = | 973 | .emit_frame_size = |
| 969 | 5 + /* vpe_ring_init_cond_exec */ | 974 | 5 + /* vpe_ring_init_cond_exec */ |
| 970 | 6 + /* vpe_ring_emit_pipeline_sync */ | 975 | 6 + /* vpe_ring_emit_pipeline_sync */ |
| 971 | 10 + 10 + 10 + /* vpe_ring_emit_fence */ | 976 | 12 + 12 + 12 + /* vpe_ring_emit_fence */ |
| 972 | /* vpe_ring_emit_vm_flush */ | 977 | /* vpe_ring_emit_vm_flush */ |
| 973 | SOC15_FLUSH_GPU_TLB_NUM_WREG * 3 + | 978 | SOC15_FLUSH_GPU_TLB_NUM_WREG * 3 + |
| 974 | SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 6, | 979 | SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 6, |
| @@ -6523,15 +6523,7 @@ static int gfx_v11_0_eop_irq(struct amdgpu_device *adev, · 6522 unmodified lines | |||
| 6523 | DRM_DEBUG("IH: CP EOP\n"); | 6523 | DRM_DEBUG("IH: CP EOP\n"); |
| 6524 | 6524 | ||
| 6525 | if (adev->enable_mes && doorbell_offset) { | 6525 | if (adev->enable_mes && doorbell_offset) { |
| 6526 | struct amdgpu_usermode_queue *queue; | 6526 | amdgpu_userq_process_fence_irq(adev, doorbell_offset); |
| 6527 | struct xarray *xa = &adev->userq_doorbell_xa; | ||
| 6528 | unsigned long flags; | ||
| 6529 | |||
| 6530 | xa_lock_irqsave(xa, flags); | ||
| 6531 | queue = xa_load(xa, doorbell_offset); | ||
| 6532 | if (queue) | ||
| 6533 | amdgpu_userq_fence_driver_process(queue->fence_drv); | ||
| 6534 | xa_unlock_irqrestore(xa, flags); | ||
| 6535 | } else { | 6527 | } else { |
| 6536 | me_id = (entry->ring_id & 0x0c) >> 2; | 6528 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 6537 | pipe_id = (entry->ring_id & 0x03) >> 0; | 6529 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -602,6 +602,13 @@ static int gfx_v12_0_init_microcode(struct amdgpu_device *adev) · 601 unmodified lines | |||
| 602 | "amdgpu/%s_pfp.bin", ucode_prefix); | 602 | "amdgpu/%s_pfp.bin", ucode_prefix); |
| 603 | if (err) | 603 | if (err) |
| 604 | goto out; | 604 | goto out; |
| 605 | |||
| 606 | adev->gfx.rs64_enable = amdgpu_ucode_hdr_version( | ||
| 607 | (union amdgpu_firmware_header *) | ||
| 608 | adev->gfx.pfp_fw->data, 2, 0); | ||
| 609 | if (adev->gfx.rs64_enable) | ||
| 610 | dev_dbg(adev->dev, "CP RS64 enable\n"); | ||
| 611 | |||
| 605 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP); | 612 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP); |
| 606 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP_P0_STACK); | 613 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP_P0_STACK); |
| 607 | 614 | ||
| @@ -4854,15 +4861,7 @@ static int gfx_v12_0_eop_irq(struct amdgpu_device *adev, · 4246 unmodified lines | |||
| 4854 | DRM_DEBUG("IH: CP EOP\n"); | 4861 | DRM_DEBUG("IH: CP EOP\n"); |
| 4855 | 4862 | ||
| 4856 | if (adev->enable_mes && doorbell_offset) { | 4863 | if (adev->enable_mes && doorbell_offset) { |
| 4857 | struct xarray *xa = &adev->userq_doorbell_xa; | 4864 | amdgpu_userq_process_fence_irq(adev, doorbell_offset); |
| 4858 | struct amdgpu_usermode_queue *queue; | ||
| 4859 | unsigned long flags; | ||
| 4860 | |||
| 4861 | xa_lock_irqsave(xa, flags); | ||
| 4862 | queue = xa_load(xa, doorbell_offset); | ||
| 4863 | if (queue) | ||
| 4864 | amdgpu_userq_fence_driver_process(queue->fence_drv); | ||
| 4865 | xa_unlock_irqrestore(xa, flags); | ||
| 4866 | } else { | 4865 | } else { |
| 4867 | me_id = (entry->ring_id & 0x0c) >> 2; | 4866 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 4868 | pipe_id = (entry->ring_id & 0x03) >> 0; | 4867 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -3643,16 +3643,7 @@ static int gfx_v12_1_eop_irq(struct amdgpu_device *adev, · 3642 unmodified lines | |||
| 3643 | DRM_DEBUG("IH: CP EOP\n"); | 3643 | DRM_DEBUG("IH: CP EOP\n"); |
| 3644 | 3644 | ||
| 3645 | if (adev->enable_mes && doorbell_offset) { | 3645 | if (adev->enable_mes && doorbell_offset) { |
| 3646 | struct xarray *xa = &adev->userq_doorbell_xa; | 3646 | amdgpu_userq_process_fence_irq(adev, doorbell_offset); |
| 3647 | struct amdgpu_usermode_queue *queue; | ||
| 3648 | unsigned long flags; | ||
| 3649 | |||
| 3650 | xa_lock_irqsave(xa, flags); | ||
| 3651 | queue = xa_load(xa, doorbell_offset); | ||
| 3652 | if (queue) | ||
| 3653 | amdgpu_userq_fence_driver_process(queue->fence_drv); | ||
| 3654 | |||
| 3655 | xa_unlock_irqrestore(xa, flags); | ||
| 3656 | } else { | 3647 | } else { |
| 3657 | me_id = (entry->ring_id & 0x0c) >> 2; | 3648 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 3658 | pipe_id = (entry->ring_id & 0x03) >> 0; | 3649 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -1571,6 +1571,71 @@ static void gfx_v6_0_setup_spi(struct amdgpu_device *adev) · 1570 unmodified lines | |||
| 1571 | mutex_unlock(&adev->grbm_idx_mutex); | 1571 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1572 | } | 1572 | } |
| 1573 | 1573 | ||
| 1574 | /** | ||
| 1575 | * gfx_v6_0_setup_tcc() - setup which TCCs are used | ||
| 1576 | * | ||
| 1577 | * @adev: amdgpu_device pointer | ||
| 1578 | * | ||
| 1579 | * Verify whether the current GPU has any TCCs disabled, | ||
| 1580 | * which can happen when the GPU is harvested and some | ||
| 1581 | * memory channels are disabled, reducing the memory bus width. | ||
| 1582 | * For example, on the Radeon HD 7870 XT (Tahiti LE). | ||
| 1583 | * | ||
| 1584 | * If some TCCs are disabled, we need to make sure that | ||
| 1585 | * the disabled TCCs are not used, and the remaining TCCs | ||
| 1586 | * are used optimally. | ||
| 1587 | * | ||
| 1588 | * TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels. | ||
| 1589 | * TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used. | ||
| 1590 | * | ||
| 1591 | * For optimal performance: | ||
| 1592 | * - Rely on the CHAN_STEER from the golden registers table, | ||
| 1593 | * only skip disabled TCCs but keep the mapping order. | ||
| 1594 | * - Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing, | ||
| 1595 | * which performs better than using the same TCC twice. | ||
| 1596 | */ | ||
| 1597 | static void gfx_v6_0_setup_tcc(struct amdgpu_device *adev) | ||
| 1598 | { | ||
| 1599 | u32 i, tcc, tcp_addr_config, num_active_tcc = 0; | ||
| 1600 | u64 chan_steer, patched_chan_steer = 0; | ||
| 1601 | const u32 num_max_tcc = adev->gfx.config.max_texture_channel_caches; | ||
| 1602 | const u32 dis_tcc_mask = | ||
| 1603 | amdgpu_gfx_create_bitmask(num_max_tcc) & | ||
| 1604 | (REG_GET_FIELD(RREG32(mmCGTS_TCC_DISABLE), | ||
| 1605 | CGTS_TCC_DISABLE, TCC_DISABLE) | | ||
| 1606 | REG_GET_FIELD(RREG32(mmCGTS_USER_TCC_DISABLE), | ||
| 1607 | CGTS_USER_TCC_DISABLE, TCC_DISABLE)); | ||
| 1608 | |||
| 1609 | /* When no TCC is disabled, the golden registers table already has optimal TCC setup */ | ||
| 1610 | if (!dis_tcc_mask) | ||
| 1611 | return; | ||
| 1612 | |||
| 1613 | /* Each 4-bit nibble contains the index of a TCC used by all TCPs */ | ||
| 1614 | chan_steer = RREG32(mmTCP_CHAN_STEER_LO) | ((u64)RREG32(mmTCP_CHAN_STEER_HI) << 32ull); | ||
| 1615 | |||
| 1616 | /* Patch the TCP to TCC mapping to skip disabled TCCs */ | ||
| 1617 | for (i = 0; i < num_max_tcc; ++i) { | ||
| 1618 | tcc = (chan_steer >> (u64)(4 * i)) & 0xf; | ||
| 1619 | |||
| 1620 | if (!((1 << tcc) & dis_tcc_mask)) { | ||
| 1621 | /* Copy enabled TCC indices to the patched register value. */ | ||
| 1622 | patched_chan_steer |= (u64)tcc << (u64)(4 * num_active_tcc); | ||
| 1623 | ++num_active_tcc; | ||
| 1624 | } | ||
| 1625 | } | ||
| 1626 | |||
| 1627 | WARN_ON(num_active_tcc != num_max_tcc - hweight32(dis_tcc_mask)); | ||
| 1628 | |||
| 1629 | /* Patch number of TCCs used by TCPs */ | ||
| 1630 | tcp_addr_config = REG_SET_FIELD(RREG32(mmTCP_ADDR_CONFIG), | ||
| 1631 | TCP_ADDR_CONFIG, NUM_TCC_BANKS, | ||
| 1632 | num_active_tcc - 1); | ||
| 1633 | |||
| 1634 | WREG32(mmTCP_ADDR_CONFIG, tcp_addr_config); | ||
| 1635 | WREG32(mmTCP_CHAN_STEER_HI, upper_32_bits(patched_chan_steer)); | ||
| 1636 | WREG32(mmTCP_CHAN_STEER_LO, lower_32_bits(patched_chan_steer)); | ||
| 1637 | } | ||
| 1638 | |||
| 1574 | static void gfx_v6_0_config_init(struct amdgpu_device *adev) | 1639 | static void gfx_v6_0_config_init(struct amdgpu_device *adev) |
| 1575 | { | 1640 | { |
| 1576 | adev->gfx.config.double_offchip_lds_buf = 0; | 1641 | adev->gfx.config.double_offchip_lds_buf = 0; |
| @@ -1729,6 +1794,7 @@ static void gfx_v6_0_constants_init(struct amdgpu_device *adev) · 152 unmodified lines | |||
| 1729 | gfx_v6_0_tiling_mode_table_init(adev); | 1794 | gfx_v6_0_tiling_mode_table_init(adev); |
| 1730 | 1795 | ||
| 1731 | gfx_v6_0_setup_rb(adev); | 1796 | gfx_v6_0_setup_rb(adev); |
| 1797 | gfx_v6_0_setup_tcc(adev); | ||
| 1732 | 1798 | ||
| 1733 | gfx_v6_0_setup_spi(adev); | 1799 | gfx_v6_0_setup_spi(adev); |
| 1734 | 1800 | ||
| @@ -5660,9 +5660,6 @@ static void gfx_v9_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, · 5659 unmodified lines | |||
| 5660 | { | 5660 | { |
| 5661 | struct amdgpu_device *adev = ring->adev; | 5661 | struct amdgpu_device *adev = ring->adev; |
| 5662 | 5662 | ||
| 5663 | /* we only allocate 32bit for each seq wb address */ | ||
| 5664 | BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); | ||
| 5665 | |||
| 5666 | /* write fence seq to the "addr" */ | 5663 | /* write fence seq to the "addr" */ |
| 5667 | amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); | 5664 | amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); |
| 5668 | amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | | 5665 | amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | |
| @@ -802,6 +802,7 @@ static const struct amd_ip_funcs jpeg_v2_0_ip_funcs = { · 801 unmodified lines | |||
| 802 | static const struct amdgpu_ring_funcs jpeg_v2_0_dec_ring_vm_funcs = { | 802 | static const struct amdgpu_ring_funcs jpeg_v2_0_dec_ring_vm_funcs = { |
| 803 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 803 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 804 | .align_mask = 0xf, | 804 | .align_mask = 0xf, |
| 805 | .no_user_fence = true, | ||
| 805 | .get_rptr = jpeg_v2_0_dec_ring_get_rptr, | 806 | .get_rptr = jpeg_v2_0_dec_ring_get_rptr, |
| 806 | .get_wptr = jpeg_v2_0_dec_ring_get_wptr, | 807 | .get_wptr = jpeg_v2_0_dec_ring_get_wptr, |
| 807 | .set_wptr = jpeg_v2_0_dec_ring_set_wptr, | 808 | .set_wptr = jpeg_v2_0_dec_ring_set_wptr, |
| @@ -693,6 +693,7 @@ static const struct amd_ip_funcs jpeg_v2_6_ip_funcs = { · 692 unmodified lines | |||
| 693 | static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = { | 693 | static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = { |
| 694 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 694 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 695 | .align_mask = 0xf, | 695 | .align_mask = 0xf, |
| 696 | .no_user_fence = true, | ||
| 696 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, | 697 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, |
| 697 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, | 698 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, |
| 698 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, | 699 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, |
| @@ -724,6 +725,7 @@ static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = { · 25 unmodified lines | |||
| 724 | static const struct amdgpu_ring_funcs jpeg_v2_6_dec_ring_vm_funcs = { | 725 | static const struct amdgpu_ring_funcs jpeg_v2_6_dec_ring_vm_funcs = { |
| 725 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 726 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 726 | .align_mask = 0xf, | 727 | .align_mask = 0xf, |
| 728 | .no_user_fence = true, | ||
| 727 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, | 729 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, |
| 728 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, | 730 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, |
| 729 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, | 731 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, |
| @@ -594,6 +594,7 @@ static const struct amd_ip_funcs jpeg_v3_0_ip_funcs = { · 593 unmodified lines | |||
| 594 | static const struct amdgpu_ring_funcs jpeg_v3_0_dec_ring_vm_funcs = { | 594 | static const struct amdgpu_ring_funcs jpeg_v3_0_dec_ring_vm_funcs = { |
| 595 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 595 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 596 | .align_mask = 0xf, | 596 | .align_mask = 0xf, |
| 597 | .no_user_fence = true, | ||
| 597 | .get_rptr = jpeg_v3_0_dec_ring_get_rptr, | 598 | .get_rptr = jpeg_v3_0_dec_ring_get_rptr, |
| 598 | .get_wptr = jpeg_v3_0_dec_ring_get_wptr, | 599 | .get_wptr = jpeg_v3_0_dec_ring_get_wptr, |
| 599 | .set_wptr = jpeg_v3_0_dec_ring_set_wptr, | 600 | .set_wptr = jpeg_v3_0_dec_ring_set_wptr, |
| @@ -759,6 +759,7 @@ static const struct amd_ip_funcs jpeg_v4_0_ip_funcs = { · 758 unmodified lines | |||
| 759 | static const struct amdgpu_ring_funcs jpeg_v4_0_dec_ring_vm_funcs = { | 759 | static const struct amdgpu_ring_funcs jpeg_v4_0_dec_ring_vm_funcs = { |
| 760 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 760 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 761 | .align_mask = 0xf, | 761 | .align_mask = 0xf, |
| 762 | .no_user_fence = true, | ||
| 762 | .get_rptr = jpeg_v4_0_dec_ring_get_rptr, | 763 | .get_rptr = jpeg_v4_0_dec_ring_get_rptr, |
| 763 | .get_wptr = jpeg_v4_0_dec_ring_get_wptr, | 764 | .get_wptr = jpeg_v4_0_dec_ring_get_wptr, |
| 764 | .set_wptr = jpeg_v4_0_dec_ring_set_wptr, | 765 | .set_wptr = jpeg_v4_0_dec_ring_set_wptr, |
| @@ -1219,6 +1219,7 @@ static const struct amd_ip_funcs jpeg_v4_0_3_ip_funcs = { · 1218 unmodified lines | |||
| 1219 | static const struct amdgpu_ring_funcs jpeg_v4_0_3_dec_ring_vm_funcs = { | 1219 | static const struct amdgpu_ring_funcs jpeg_v4_0_3_dec_ring_vm_funcs = { |
| 1220 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 1220 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 1221 | .align_mask = 0xf, | 1221 | .align_mask = 0xf, |
| 1222 | .no_user_fence = true, | ||
| 1222 | .get_rptr = jpeg_v4_0_3_dec_ring_get_rptr, | 1223 | .get_rptr = jpeg_v4_0_3_dec_ring_get_rptr, |
| 1223 | .get_wptr = jpeg_v4_0_3_dec_ring_get_wptr, | 1224 | .get_wptr = jpeg_v4_0_3_dec_ring_get_wptr, |
| 1224 | .set_wptr = jpeg_v4_0_3_dec_ring_set_wptr, | 1225 | .set_wptr = jpeg_v4_0_3_dec_ring_set_wptr, |
| @@ -804,6 +804,7 @@ static const struct amd_ip_funcs jpeg_v4_0_5_ip_funcs = { · 803 unmodified lines | |||
| 804 | static const struct amdgpu_ring_funcs jpeg_v4_0_5_dec_ring_vm_funcs = { | 804 | static const struct amdgpu_ring_funcs jpeg_v4_0_5_dec_ring_vm_funcs = { |
| 805 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 805 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 806 | .align_mask = 0xf, | 806 | .align_mask = 0xf, |
| 807 | .no_user_fence = true, | ||
| 807 | .get_rptr = jpeg_v4_0_5_dec_ring_get_rptr, | 808 | .get_rptr = jpeg_v4_0_5_dec_ring_get_rptr, |
| 808 | .get_wptr = jpeg_v4_0_5_dec_ring_get_wptr, | 809 | .get_wptr = jpeg_v4_0_5_dec_ring_get_wptr, |
| 809 | .set_wptr = jpeg_v4_0_5_dec_ring_set_wptr, | 810 | .set_wptr = jpeg_v4_0_5_dec_ring_set_wptr, |
| @@ -680,6 +680,7 @@ static const struct amd_ip_funcs jpeg_v5_0_0_ip_funcs = { · 679 unmodified lines | |||
| 680 | static const struct amdgpu_ring_funcs jpeg_v5_0_0_dec_ring_vm_funcs = { | 680 | static const struct amdgpu_ring_funcs jpeg_v5_0_0_dec_ring_vm_funcs = { |
| 681 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 681 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 682 | .align_mask = 0xf, | 682 | .align_mask = 0xf, |
| 683 | .no_user_fence = true, | ||
| 683 | .get_rptr = jpeg_v5_0_0_dec_ring_get_rptr, | 684 | .get_rptr = jpeg_v5_0_0_dec_ring_get_rptr, |
| 684 | .get_wptr = jpeg_v5_0_0_dec_ring_get_wptr, | 685 | .get_wptr = jpeg_v5_0_0_dec_ring_get_wptr, |
| 685 | .set_wptr = jpeg_v5_0_0_dec_ring_set_wptr, | 686 | .set_wptr = jpeg_v5_0_0_dec_ring_set_wptr, |
| @@ -884,6 +884,7 @@ static const struct amd_ip_funcs jpeg_v5_0_1_ip_funcs = { · 883 unmodified lines | |||
| 884 | static const struct amdgpu_ring_funcs jpeg_v5_0_1_dec_ring_vm_funcs = { | 884 | static const struct amdgpu_ring_funcs jpeg_v5_0_1_dec_ring_vm_funcs = { |
| 885 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 885 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 886 | .align_mask = 0xf, | 886 | .align_mask = 0xf, |
| 887 | .no_user_fence = true, | ||
| 887 | .get_rptr = jpeg_v5_0_1_dec_ring_get_rptr, | 888 | .get_rptr = jpeg_v5_0_1_dec_ring_get_rptr, |
| 888 | .get_wptr = jpeg_v5_0_1_dec_ring_get_wptr, | 889 | .get_wptr = jpeg_v5_0_1_dec_ring_get_wptr, |
| 889 | .set_wptr = jpeg_v5_0_1_dec_ring_set_wptr, | 890 | .set_wptr = jpeg_v5_0_1_dec_ring_set_wptr, |
| @@ -703,6 +703,7 @@ static const struct amd_ip_funcs jpeg_v5_0_2_ip_funcs = { · 702 unmodified lines | |||
| 703 | static const struct amdgpu_ring_funcs jpeg_v5_0_2_dec_ring_vm_funcs = { | 703 | static const struct amdgpu_ring_funcs jpeg_v5_0_2_dec_ring_vm_funcs = { |
| 704 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 704 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 705 | .align_mask = 0xf, | 705 | .align_mask = 0xf, |
| 706 | .no_user_fence = true, | ||
| 706 | .get_rptr = jpeg_v5_0_2_dec_ring_get_rptr, | 707 | .get_rptr = jpeg_v5_0_2_dec_ring_get_rptr, |
| 707 | .get_wptr = jpeg_v5_0_2_dec_ring_get_wptr, | 708 | .get_wptr = jpeg_v5_0_2_dec_ring_get_wptr, |
| 708 | .set_wptr = jpeg_v5_0_2_dec_ring_set_wptr, | 709 | .set_wptr = jpeg_v5_0_2_dec_ring_set_wptr, |
| @@ -661,6 +661,7 @@ static const struct amd_ip_funcs jpeg_v5_3_0_ip_funcs = { · 660 unmodified lines | |||
| 661 | static const struct amdgpu_ring_funcs jpeg_v5_3_0_dec_ring_vm_funcs = { | 661 | static const struct amdgpu_ring_funcs jpeg_v5_3_0_dec_ring_vm_funcs = { |
| 662 | .type = AMDGPU_RING_TYPE_VCN_JPEG, | 662 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 663 | .align_mask = 0xf, | 663 | .align_mask = 0xf, |
| 664 | .no_user_fence = true, | ||
| 664 | .get_rptr = jpeg_v5_3_0_dec_ring_get_rptr, | 665 | .get_rptr = jpeg_v5_3_0_dec_ring_get_rptr, |
| 665 | .get_wptr = jpeg_v5_3_0_dec_ring_get_wptr, | 666 | .get_wptr = jpeg_v5_3_0_dec_ring_get_wptr, |
| 666 | .set_wptr = jpeg_v5_3_0_dec_ring_set_wptr, | 667 | .set_wptr = jpeg_v5_3_0_dec_ring_set_wptr, |
| @@ -30,34 +30,6 @@ · 29 unmodified lines | |||
| 30 | #define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE | 30 | #define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE |
| 31 | #define AMDGPU_USERQ_GANG_CTX_SZ PAGE_SIZE | 31 | #define AMDGPU_USERQ_GANG_CTX_SZ PAGE_SIZE |
| 32 | 32 | ||
| 33 | static int | ||
| 34 | mes_userq_map_gtt_bo_to_gart(struct amdgpu_bo *bo) | ||
| 35 | { | ||
| 36 | int ret; | ||
| 37 | |||
| 38 | ret = amdgpu_bo_reserve(bo, true); | ||
| 39 | if (ret) { | ||
| 40 | DRM_ERROR("Failed to reserve bo. ret %d\n", ret); | ||
| 41 | goto err_reserve_bo_failed; | ||
| 42 | } | ||
| 43 | |||
| 44 | ret = amdgpu_ttm_alloc_gart(&bo->tbo); | ||
| 45 | if (ret) { | ||
| 46 | DRM_ERROR("Failed to bind bo to GART. ret %d\n", ret); | ||
| 47 | goto err_map_bo_gart_failed; | ||
| 48 | } | ||
| 49 | |||
| 50 | amdgpu_bo_unreserve(bo); | ||
| 51 | bo = amdgpu_bo_ref(bo); | ||
| 52 | |||
| 53 | return 0; | ||
| 54 | |||
| 55 | err_map_bo_gart_failed: | ||
| 56 | amdgpu_bo_unreserve(bo); | ||
| 57 | err_reserve_bo_failed: | ||
| 58 | return ret; | ||
| 59 | } | ||
| 60 | |||
| 61 | static int | 33 | static int |
| 62 | mes_userq_create_wptr_mapping(struct amdgpu_device *adev, | 34 | mes_userq_create_wptr_mapping(struct amdgpu_device *adev, |
| 63 | struct amdgpu_userq_mgr *uq_mgr, | 35 | struct amdgpu_userq_mgr *uq_mgr, |
| @@ -65,55 +37,64 @@ mes_userq_create_wptr_mapping(struct amdgpu_device *adev, · 1 unmodified line | |||
| 65 | uint64_t wptr) | 37 | uint64_t wptr) |
| 66 | { | 38 | { |
| 67 | struct amdgpu_bo_va_mapping *wptr_mapping; | 39 | struct amdgpu_bo_va_mapping *wptr_mapping; |
| 68 | struct amdgpu_vm *wptr_vm; | ||
| 69 | struct amdgpu_userq_obj *wptr_obj = &queue->wptr_obj; | 40 | struct amdgpu_userq_obj *wptr_obj = &queue->wptr_obj; |
| 41 | struct amdgpu_bo *obj; | ||
| 42 | struct amdgpu_vm *vm = queue->vm; | ||
| 43 | struct drm_exec exec; | ||
| 70 | int ret; | 44 | int ret; |
| 71 | 45 | ||
| 72 | wptr_vm = queue->vm; | ||
| 73 | ret = amdgpu_bo_reserve(wptr_vm->root.bo, false); | ||
| 74 | if (ret) | ||
| 75 | return ret; | ||
| 76 | |||
| 77 | wptr &= AMDGPU_GMC_HOLE_MASK; | 46 | wptr &= AMDGPU_GMC_HOLE_MASK; |
| 78 | wptr_mapping = amdgpu_vm_bo_lookup_mapping(wptr_vm, wptr >> PAGE_SHIFT); | ||
| 79 | amdgpu_bo_unreserve(wptr_vm->root.bo); | ||
| 80 | if (!wptr_mapping) { | ||
| 81 | DRM_ERROR("Failed to lookup wptr bo\n"); | ||
| 82 | return -EINVAL; | ||
| 83 | } | ||
| 84 | 47 | ||
| 85 | wptr_obj->obj = wptr_mapping->bo_va->base.bo; | 48 | drm_exec_init(&exec, DRM_EXEC_IGNORE_DUPLICATES, 2); |
| 86 | if (wptr_obj->obj->tbo.base.size > PAGE_SIZE) { | 49 | drm_exec_until_all_locked(&exec) { |
| 87 | DRM_ERROR("Requested GART mapping for wptr bo larger than one page\n"); | 50 | ret = amdgpu_vm_lock_pd(vm, &exec, 1); |
| 88 | return -EINVAL; | 51 | drm_exec_retry_on_contention(&exec); |
| 89 | } | 52 | if (unlikely(ret)) |
| 53 | goto fail_lock; | ||
| 54 | |||
| 55 | wptr_mapping = amdgpu_vm_bo_lookup_mapping(vm, wptr >> PAGE_SHIFT); | ||
| 56 | if (!wptr_mapping) { | ||
| 57 | ret = -EINVAL; | ||
| 58 | goto fail_lock; | ||
| 59 | } | ||
| 90 | 60 | ||
| 91 | ret = mes_userq_map_gtt_bo_to_gart(wptr_obj->obj); | 61 | obj = wptr_mapping->bo_va->base.bo; |
| 92 | if (ret) { | 62 | ret = drm_exec_lock_obj(&exec, &obj->tbo.base); |
| 93 | DRM_ERROR("Failed to map wptr bo to GART\n"); | 63 | drm_exec_retry_on_contention(&exec); |
| 94 | return ret; | 64 | if (unlikely(ret)) |
| 65 | goto fail_lock; | ||
| 95 | } | 66 | } |
| 96 | 67 | ||
| 97 | ret = amdgpu_bo_reserve(wptr_obj->obj, true); | 68 | wptr_obj->obj = amdgpu_bo_ref(wptr_mapping->bo_va->base.bo); |
| 98 | if (ret) { | 69 | if (wptr_obj->obj->tbo.base.size > PAGE_SIZE) { |
| 99 | DRM_ERROR("Failed to reserve wptr bo\n"); | 70 | ret = -EINVAL; |
| 100 | return ret; | 71 | goto fail_map; |
| 101 | } | 72 | } |
| 102 | 73 | ||
| 103 | /* TODO use eviction fence instead of pinning. */ | 74 | /* TODO use eviction fence instead of pinning. */ |
| 104 | ret = amdgpu_bo_pin(wptr_obj->obj, AMDGPU_GEM_DOMAIN_GTT); | 75 | ret = amdgpu_bo_pin(wptr_obj->obj, AMDGPU_GEM_DOMAIN_GTT); |
| 105 | if (ret) { | 76 | if (ret) { |
| 106 | drm_file_err(uq_mgr->file, "[Usermode queues] Failed to pin wptr bo\n"); | 77 | DRM_ERROR("Failed to pin wptr bo. ret %d\n", ret); |
| 107 | goto unresv_bo; | 78 | goto fail_map; |
| 79 | } | ||
| 80 | |||
| 81 | ret = amdgpu_ttm_alloc_gart(&wptr_obj->obj->tbo); | ||
| 82 | if (ret) { | ||
| 83 | DRM_ERROR("Failed to bind bo to GART. ret %d\n", ret); | ||
| 84 | goto fail_alloc_gart; | ||
| 108 | } | 85 | } |
| 109 | 86 | ||
| 110 | queue->wptr_obj.gpu_addr = amdgpu_bo_gpu_offset(wptr_obj->obj); | 87 | queue->wptr_obj.gpu_addr = amdgpu_bo_gpu_offset(wptr_obj->obj); |
| 111 | amdgpu_bo_unreserve(wptr_obj->obj); | ||
| 112 | 88 | ||
| 89 | drm_exec_fini(&exec); | ||
| 113 | return 0; | 90 | return 0; |
| 114 | 91 | ||
| 115 | unresv_bo: | 92 | fail_alloc_gart: |
| 116 | amdgpu_bo_unreserve(wptr_obj->obj); | 93 | amdgpu_bo_unpin(wptr_obj->obj); |
| 94 | fail_map: | ||
| 95 | amdgpu_bo_unref(&wptr_obj->obj); | ||
| 96 | fail_lock: | ||
| 97 | drm_exec_fini(&exec); | ||
| 117 | return ret; | 98 | return ret; |
| 118 | 99 | ||
| 119 | } | 100 | } |
| @@ -211,12 +192,16 @@ static int mes_userq_create_ctx_space(struct amdgpu_userq_mgr *uq_mgr, · 91 unmodified lines | |||
| 211 | * for the same. | 192 | * for the same. |
| 212 | */ | 193 | */ |
| 213 | size = AMDGPU_USERQ_PROC_CTX_SZ + AMDGPU_USERQ_GANG_CTX_SZ; | 194 | size = AMDGPU_USERQ_PROC_CTX_SZ + AMDGPU_USERQ_GANG_CTX_SZ; |
| 214 | r = amdgpu_userq_create_object(uq_mgr, ctx, size); | 195 | r = amdgpu_bo_create_kernel(uq_mgr->adev, size, 0, |
| 196 | AMDGPU_GEM_DOMAIN_GTT, | ||
| 197 | &ctx->obj, &ctx->gpu_addr, | ||
| 198 | &ctx->cpu_ptr); | ||
| 215 | if (r) { | 199 | if (r) { |
| 216 | DRM_ERROR("Failed to allocate ctx space bo for userqueue, err:%d\n", r); | 200 | DRM_ERROR("Failed to allocate ctx space bo for userqueue, err:%d\n", r); |
| 217 | return r; | 201 | return r; |
| 218 | } | 202 | } |
| 219 | 203 | ||
| 204 | memset(ctx->cpu_ptr, 0, size); | ||
| 220 | return 0; | 205 | return 0; |
| 221 | } | 206 | } |
| 222 | 207 | ||
| @@ -289,13 +274,19 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 66 unmodified lines | |||
| 289 | return -ENOMEM; | 274 | return -ENOMEM; |
| 290 | } | 275 | } |
| 291 | 276 | ||
| 292 | r = amdgpu_userq_create_object(uq_mgr, &queue->mqd, | 277 | r = amdgpu_bo_create_kernel(adev, |
| 293 | AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size)); | 278 | AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size), |
| 279 | 0, AMDGPU_GEM_DOMAIN_GTT, | ||
| 280 | &queue->mqd.obj, &queue->mqd.gpu_addr, | ||
| 281 | &queue->mqd.cpu_ptr); | ||
| 294 | if (r) { | 282 | if (r) { |
| 295 | DRM_ERROR("Failed to create MQD object for userqueue\n"); | 283 | DRM_ERROR("Failed to create MQD object for userqueue\n"); |
| 296 | goto free_props; | 284 | goto free_props; |
| 297 | } | 285 | } |
| 298 | 286 | ||
| 287 | memset(queue->mqd.cpu_ptr, 0, | ||
| 288 | AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size)); | ||
| 289 | |||
| 299 | /* Initialize the MQD BO with user given values */ | 290 | /* Initialize the MQD BO with user given values */ |
| 300 | userq_props->wptr_gpu_addr = mqd_user->wptr_va; | 291 | userq_props->wptr_gpu_addr = mqd_user->wptr_va; |
| 301 | userq_props->rptr_gpu_addr = mqd_user->rptr_va; | 292 | userq_props->rptr_gpu_addr = mqd_user->rptr_va; |
| @@ -327,8 +318,9 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 25 unmodified lines | |||
| 327 | kfree(compute_mqd); | 318 | kfree(compute_mqd); |
| 328 | goto free_mqd; | 319 | goto free_mqd; |
| 329 | } | 320 | } |
| 330 | r = amdgpu_userq_input_va_validate(adev, queue, compute_mqd->eop_va, | 321 | r = amdgpu_userq_input_va_validate(adev, queue, |
| 331 | 2048); | 322 | compute_mqd->eop_va, 2048, |
| 323 | &queue->userq_vas.va.eop); | ||
| 332 | amdgpu_bo_unreserve(queue->vm->root.bo); | 324 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 333 | if (r) { | 325 | if (r) { |
| 334 | kfree(compute_mqd); | 326 | kfree(compute_mqd); |
| @@ -377,7 +369,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 42 unmodified lines | |||
| 377 | goto free_mqd; | 369 | goto free_mqd; |
| 378 | } | 370 | } |
| 379 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->shadow_va, | 371 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->shadow_va, |
| 380 | shadow_info.shadow_size); | 372 | shadow_info.shadow_size, |
| 373 | &queue->userq_vas.va.shadow); | ||
| 381 | if (r) { | 374 | if (r) { |
| 382 | amdgpu_bo_unreserve(queue->vm->root.bo); | 375 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 383 | kfree(mqd_gfx_v11); | 376 | kfree(mqd_gfx_v11); |
| @@ -385,7 +378,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 1 unmodified line | |||
| 385 | } | 378 | } |
| 386 | 379 | ||
| 387 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->csa_va, | 380 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->csa_va, |
| 388 | shadow_info.csa_size); | 381 | shadow_info.csa_size, |
| 382 | &queue->userq_vas.va.csa); | ||
| 389 | amdgpu_bo_unreserve(queue->vm->root.bo); | 383 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 390 | if (r) { | 384 | if (r) { |
| 391 | kfree(mqd_gfx_v11); | 385 | kfree(mqd_gfx_v11); |
| @@ -415,7 +409,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 23 unmodified lines | |||
| 415 | goto free_mqd; | 409 | goto free_mqd; |
| 416 | } | 410 | } |
| 417 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_sdma_v11->csa_va, | 411 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_sdma_v11->csa_va, |
| 418 | 32); | 412 | 32, |
| 413 | &queue->userq_vas.va.csa); | ||
| 419 | amdgpu_bo_unreserve(queue->vm->root.bo); | 414 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 420 | if (r) { | 415 | if (r) { |
| 421 | kfree(mqd_sdma_v11); | 416 | kfree(mqd_sdma_v11); |
| @@ -451,10 +446,12 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 29 unmodified lines | |||
| 451 | return 0; | 446 | return 0; |
| 452 | 447 | ||
| 453 | free_ctx: | 448 | free_ctx: |
| 454 | amdgpu_userq_destroy_object(uq_mgr, &queue->fw_obj); | 449 | amdgpu_bo_free_kernel(&queue->fw_obj.obj, &queue->fw_obj.gpu_addr, |
| 450 | &queue->fw_obj.cpu_ptr); | ||
| 455 | 451 | ||
| 456 | free_mqd: | 452 | free_mqd: |
| 457 | amdgpu_userq_destroy_object(uq_mgr, &queue->mqd); | 453 | amdgpu_bo_free_kernel(&queue->mqd.obj, &queue->mqd.gpu_addr, |
| 454 | &queue->mqd.cpu_ptr); | ||
| 458 | 455 | ||
| 459 | free_props: | 456 | free_props: |
| 460 | kfree(userq_props); | 457 | kfree(userq_props); |
| @@ -464,11 +461,12 @@ free_props: · 3 unmodified lines | |||
| 464 | 461 | ||
| 465 | static void mes_userq_mqd_destroy(struct amdgpu_usermode_queue *queue) | 462 | static void mes_userq_mqd_destroy(struct amdgpu_usermode_queue *queue) |
| 466 | { | 463 | { |
| 467 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | ||
| 468 | 464 | ||
| 469 | amdgpu_userq_destroy_object(uq_mgr, &queue->fw_obj); | 465 | amdgpu_bo_free_kernel(&queue->fw_obj.obj, &queue->fw_obj.gpu_addr, |
| 466 | &queue->fw_obj.cpu_ptr); | ||
| 470 | kfree(queue->userq_prop); | 467 | kfree(queue->userq_prop); |
| 471 | amdgpu_userq_destroy_object(uq_mgr, &queue->mqd); | 468 | amdgpu_bo_free_kernel(&queue->mqd.obj, &queue->mqd.gpu_addr, |
| 469 | &queue->mqd.cpu_ptr); | ||
| 472 | } | 470 | } |
| 473 | 471 | ||
| 474 | static int mes_userq_preempt(struct amdgpu_usermode_queue *queue) | 472 | static int mes_userq_preempt(struct amdgpu_usermode_queue *queue) |
| @@ -889,7 +889,7 @@ static void sdma_v4_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se · 888 unmodified lines | |||
| 889 | /* write the fence */ | 889 | /* write the fence */ |
| 890 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); | 890 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); |
| 891 | /* zero in first two bits */ | 891 | /* zero in first two bits */ |
| 892 | BUG_ON(addr & 0x3); | 892 | WARN_ON(addr & 0x3); |
| 893 | amdgpu_ring_write(ring, lower_32_bits(addr)); | 893 | amdgpu_ring_write(ring, lower_32_bits(addr)); |
| 894 | amdgpu_ring_write(ring, upper_32_bits(addr)); | 894 | amdgpu_ring_write(ring, upper_32_bits(addr)); |
| 895 | amdgpu_ring_write(ring, lower_32_bits(seq)); | 895 | amdgpu_ring_write(ring, lower_32_bits(seq)); |
| @@ -899,7 +899,7 @@ static void sdma_v4_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se · 3 unmodified lines | |||
| 899 | addr += 4; | 899 | addr += 4; |
| 900 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); | 900 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); |
| 901 | /* zero in first two bits */ | 901 | /* zero in first two bits */ |
| 902 | BUG_ON(addr & 0x3); | 902 | WARN_ON(addr & 0x3); |
| 903 | amdgpu_ring_write(ring, lower_32_bits(addr)); | 903 | amdgpu_ring_write(ring, lower_32_bits(addr)); |
| 904 | amdgpu_ring_write(ring, upper_32_bits(addr)); | 904 | amdgpu_ring_write(ring, upper_32_bits(addr)); |
| 905 | amdgpu_ring_write(ring, upper_32_bits(seq)); | 905 | amdgpu_ring_write(ring, upper_32_bits(seq)); |
| @@ -1662,17 +1662,8 @@ static int sdma_v6_0_process_fence_irq(struct amdgpu_device *adev, · 1661 unmodified lines | |||
| 1662 | u32 doorbell_offset = entry->src_data[0]; | 1662 | u32 doorbell_offset = entry->src_data[0]; |
| 1663 | 1663 | ||
| 1664 | if (adev->enable_mes && doorbell_offset) { | 1664 | if (adev->enable_mes && doorbell_offset) { |
| 1665 | struct amdgpu_usermode_queue *queue; | ||
| 1666 | struct xarray *xa = &adev->userq_doorbell_xa; | ||
| 1667 | unsigned long flags; | ||
| 1668 | |||
| 1669 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; | 1665 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; |
| 1670 | 1666 | amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 1671 | xa_lock_irqsave(xa, flags); | ||
| 1672 | queue = xa_load(xa, doorbell_offset); | ||
| 1673 | if (queue) | ||
| 1674 | amdgpu_userq_fence_driver_process(queue->fence_drv); | ||
| 1675 | xa_unlock_irqrestore(xa, flags); | ||
| 1676 | } | 1667 | } |
| 1677 | 1668 | ||
| 1678 | return 0; | 1669 | return 0; |
| @@ -1594,17 +1594,8 @@ static int sdma_v7_0_process_fence_irq(struct amdgpu_device *adev, · 1593 unmodified lines | |||
| 1594 | u32 doorbell_offset = entry->src_data[0]; | 1594 | u32 doorbell_offset = entry->src_data[0]; |
| 1595 | 1595 | ||
| 1596 | if (adev->enable_mes && doorbell_offset) { | 1596 | if (adev->enable_mes && doorbell_offset) { |
| 1597 | struct xarray *xa = &adev->userq_doorbell_xa; | ||
| 1598 | struct amdgpu_usermode_queue *queue; | ||
| 1599 | unsigned long flags; | ||
| 1600 | |||
| 1601 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; | 1597 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; |
| 1602 | 1598 | amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 1603 | xa_lock_irqsave(xa, flags); | ||
| 1604 | queue = xa_load(xa, doorbell_offset); | ||
| 1605 | if (queue) | ||
| 1606 | amdgpu_userq_fence_driver_process(queue->fence_drv); | ||
| 1607 | xa_unlock_irqrestore(xa, flags); | ||
| 1608 | } | 1599 | } |
| 1609 | 1600 | ||
| 1610 | return 0; | 1601 | return 0; |
| @@ -242,6 +242,10 @@ static void uvd_v3_1_mc_resume(struct amdgpu_device *adev) · 241 unmodified lines | |||
| 242 | uint64_t addr; | 242 | uint64_t addr; |
| 243 | uint32_t size; | 243 | uint32_t size; |
| 244 | 244 | ||
| 245 | /* When the keyselect is already set, don't perturb it. */ | ||
| 246 | if (RREG32(mmUVD_FW_START)) | ||
| 247 | return; | ||
| 248 | |||
| 245 | /* program the VCPU memory controller bits 0-27 */ | 249 | /* program the VCPU memory controller bits 0-27 */ |
| 246 | addr = (adev->uvd.inst->gpu_addr + AMDGPU_UVD_FIRMWARE_OFFSET) >> 3; | 250 | addr = (adev->uvd.inst->gpu_addr + AMDGPU_UVD_FIRMWARE_OFFSET) >> 3; |
| 247 | size = AMDGPU_UVD_FIRMWARE_SIZE(adev) >> 3; | 251 | size = AMDGPU_UVD_FIRMWARE_SIZE(adev) >> 3; |
| @@ -284,6 +288,12 @@ static int uvd_v3_1_fw_validate(struct amdgpu_device *adev) · 36 unmodified lines | |||
| 284 | int i; | 288 | int i; |
| 285 | uint32_t keysel = adev->uvd.keyselect; | 289 | uint32_t keysel = adev->uvd.keyselect; |
| 286 | 290 | ||
| 291 | if (RREG32(mmUVD_FW_START) & UVD_FW_STATUS__PASS_MASK) { | ||
| 292 | dev_dbg(adev->dev, "UVD keyselect already set: 0x%x (on CPU: 0x%x)\n", | ||
| 293 | RREG32(mmUVD_FW_START), adev->uvd.keyselect); | ||
| 294 | return 0; | ||
| 295 | } | ||
| 296 | |||
| 287 | WREG32(mmUVD_FW_START, keysel); | 297 | WREG32(mmUVD_FW_START, keysel); |
| 288 | 298 | ||
| 289 | for (i = 0; i < 10; ++i) { | 299 | for (i = 0; i < 10; ++i) { |
| @@ -42,9 +42,10 @@ · 41 unmodified lines | |||
| 42 | #include "oss/oss_1_0_d.h" | 42 | #include "oss/oss_1_0_d.h" |
| 43 | #include "oss/oss_1_0_sh_mask.h" | 43 | #include "oss/oss_1_0_sh_mask.h" |
| 44 | 44 | ||
| 45 | #define VCE_V1_0_ALIGNMENT (32 * 1024) | ||
| 45 | #define VCE_V1_0_FW_SIZE (256 * 1024) | 46 | #define VCE_V1_0_FW_SIZE (256 * 1024) |
| 46 | #define VCE_V1_0_STACK_SIZE (64 * 1024) | 47 | #define VCE_V1_0_STACK_SIZE (64 * 1024) |
| 47 | #define VCE_V1_0_DATA_SIZE (7808 * (AMDGPU_MAX_VCE_HANDLES + 1)) | 48 | #define VCE_V1_0_DATA_SIZE (ALIGN(7808 * (AMDGPU_MAX_VCE_HANDLES + 1), VCE_V1_0_ALIGNMENT)) |
| 48 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 | 49 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 |
| 49 | 50 | ||
| 50 | static void vce_v1_0_set_ring_funcs(struct amdgpu_device *adev); | 51 | static void vce_v1_0_set_ring_funcs(struct amdgpu_device *adev); |
| @@ -177,7 +178,7 @@ static void vce_v1_0_init_cg(struct amdgpu_device *adev) · 126 unmodified lines | |||
| 177 | } | 178 | } |
| 178 | 179 | ||
| 179 | /** | 180 | /** |
| 180 | * vce_v1_0_load_fw_signature - load firmware signature into VCPU BO | 181 | * vce_v1_0_load_fw() - load firmware signature into VCPU BO |
| 181 | * | 182 | * |
| 182 | * @adev: amdgpu_device pointer | 183 | * @adev: amdgpu_device pointer |
| 183 | * | 184 | * |
| @@ -185,21 +186,26 @@ static void vce_v1_0_init_cg(struct amdgpu_device *adev) · 1 unmodified line | |||
| 185 | * This function finds the signature appropriate for the current | 186 | * This function finds the signature appropriate for the current |
| 186 | * ASIC and writes that into the VCPU BO. | 187 | * ASIC and writes that into the VCPU BO. |
| 187 | */ | 188 | */ |
| 188 | static int vce_v1_0_load_fw_signature(struct amdgpu_device *adev) | 189 | static int vce_v1_0_load_fw(struct amdgpu_device *adev) |
| 189 | { | 190 | { |
| 190 | const struct common_firmware_header *hdr; | 191 | const struct common_firmware_header *hdr; |
| 191 | struct vce_v1_0_fw_signature *sign; | 192 | struct vce_v1_0_fw_signature *sign; |
| 192 | unsigned int ucode_offset; | 193 | u32 ucode_offset; |
| 194 | u32 ucode_size; | ||
| 193 | uint32_t chip_id; | 195 | uint32_t chip_id; |
| 194 | u32 *cpu_addr; | 196 | u32 *cpu_addr; |
| 195 | int i; | 197 | int i; |
| 196 | 198 | ||
| 197 | hdr = (const struct common_firmware_header *)adev->vce.fw->data; | 199 | hdr = (const struct common_firmware_header *)adev->vce.fw->data; |
| 198 | ucode_offset = le32_to_cpu(hdr->ucode_array_offset_bytes); | 200 | ucode_offset = le32_to_cpu(hdr->ucode_array_offset_bytes); |
| 201 | ucode_size = hdr->ucode_size_bytes - sizeof(struct vce_v1_0_fw_signature *); | ||
| 199 | cpu_addr = adev->vce.cpu_addr; | 202 | cpu_addr = adev->vce.cpu_addr; |
| 200 | 203 | ||
| 201 | sign = (void *)adev->vce.fw->data + ucode_offset; | 204 | sign = (void *)adev->vce.fw->data + ucode_offset; |
| 202 | 205 | ||
| 206 | if (ucode_size > VCE_V1_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET) | ||
| 207 | return -EINVAL; | ||
| 208 | |||
| 203 | switch (adev->asic_type) { | 209 | switch (adev->asic_type) { |
| 204 | case CHIP_TAHITI: | 210 | case CHIP_TAHITI: |
| 205 | chip_id = 0x01000014; | 211 | chip_id = 0x01000014; |
| @@ -226,12 +232,14 @@ static int vce_v1_0_load_fw_signature(struct amdgpu_device *adev) · 20 unmodified lines | |||
| 226 | return -EINVAL; | 232 | return -EINVAL; |
| 227 | } | 233 | } |
| 228 | 234 | ||
| 235 | memset_io(&cpu_addr[0], 0, amdgpu_bo_size(adev->vce.vcpu_bo)); | ||
| 236 | |||
| 229 | cpu_addr += (256 - 64) / 4; | 237 | cpu_addr += (256 - 64) / 4; |
| 230 | memcpy_toio(&cpu_addr[0], &sign->val[i].nonce[0], 16); | 238 | memcpy_toio(&cpu_addr[0], &sign->val[i].nonce[0], 16); |
| 231 | cpu_addr[4] = cpu_to_le32(le32_to_cpu(sign->length) + 64); | 239 | cpu_addr[4] = cpu_to_le32(le32_to_cpu(sign->length) + 64); |
| 232 | 240 | ||
| 233 | memset_io(&cpu_addr[5], 0, 44); | 241 | memset_io(&cpu_addr[5], 0, 44); |
| 234 | memcpy_toio(&cpu_addr[16], &sign[1], hdr->ucode_size_bytes - sizeof(*sign)); | 242 | memcpy_toio(&cpu_addr[16], &sign[1], ucode_size); |
| 235 | 243 | ||
| 236 | cpu_addr += (le32_to_cpu(sign->length) + 64) / 4; | 244 | cpu_addr += (le32_to_cpu(sign->length) + 64) / 4; |
| 237 | memcpy_toio(&cpu_addr[0], &sign->val[i].sigval[0], 16); | 245 | memcpy_toio(&cpu_addr[0], &sign->val[i].sigval[0], 16); |
| @@ -312,18 +320,23 @@ static int vce_v1_0_mc_resume(struct amdgpu_device *adev) · 74 unmodified lines | |||
| 312 | WREG32(mmVCE_VCPU_SCRATCH7, AMDGPU_MAX_VCE_HANDLES); | 320 | WREG32(mmVCE_VCPU_SCRATCH7, AMDGPU_MAX_VCE_HANDLES); |
| 313 | 321 | ||
| 314 | offset = adev->vce.gpu_addr + AMDGPU_VCE_FIRMWARE_OFFSET; | 322 | offset = adev->vce.gpu_addr + AMDGPU_VCE_FIRMWARE_OFFSET; |
| 315 | size = VCE_V1_0_FW_SIZE; | 323 | size = VCE_V1_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; |
| 316 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); | 324 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset); |
| 317 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); | 325 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 318 | 326 | ||
| 319 | offset += size; | 327 | offset += size; |
| 320 | size = VCE_V1_0_STACK_SIZE; | 328 | size = VCE_V1_0_STACK_SIZE; |
| 321 | WREG32(mmVCE_VCPU_CACHE_OFFSET1, offset & 0x7fffffff); | 329 | WARN_ON(!IS_ALIGNED(offset, VCE_V1_0_ALIGNMENT)); |
| 330 | WARN_ON(!IS_ALIGNED(size, VCE_V1_0_ALIGNMENT)); | ||
| 331 | WREG32(mmVCE_VCPU_CACHE_OFFSET1, offset); | ||
| 322 | WREG32(mmVCE_VCPU_CACHE_SIZE1, size); | 332 | WREG32(mmVCE_VCPU_CACHE_SIZE1, size); |
| 323 | 333 | ||
| 324 | offset += size; | 334 | offset += size; |
| 325 | size = VCE_V1_0_DATA_SIZE; | 335 | size = VCE_V1_0_DATA_SIZE; |
| 326 | WREG32(mmVCE_VCPU_CACHE_OFFSET2, offset & 0x7fffffff); | 336 | WARN_ON(!IS_ALIGNED(offset, VCE_V1_0_ALIGNMENT)); |
| 337 | WARN_ON(!IS_ALIGNED(size, VCE_V1_0_ALIGNMENT)); | ||
| 338 | WARN_ON((offset + size - adev->vce.gpu_addr) > amdgpu_bo_size(adev->vce.vcpu_bo)); | ||
| 339 | WREG32(mmVCE_VCPU_CACHE_OFFSET2, offset); | ||
| 327 | WREG32(mmVCE_VCPU_CACHE_SIZE2, size); | 340 | WREG32(mmVCE_VCPU_CACHE_SIZE2, size); |
| 328 | 341 | ||
| 329 | WREG32_P(mmVCE_LMI_CTRL2, 0x0, ~0x100); | 342 | WREG32_P(mmVCE_LMI_CTRL2, 0x0, ~0x100); |
| @@ -527,22 +540,31 @@ static int vce_v1_0_early_init(struct amdgpu_ip_block *ip_block) · 197 unmodified lines | |||
| 527 | * To accomodate that, we put GART to the LOW address range | 540 | * To accomodate that, we put GART to the LOW address range |
| 528 | * and reserve some GART pages where we map the VCPU BO, | 541 | * and reserve some GART pages where we map the VCPU BO, |
| 529 | * so that it gets a 32-bit address. | 542 | * so that it gets a 32-bit address. |
| 543 | * | ||
| 544 | * The BAR address is zero and we can't change it | ||
| 545 | * due to the firmware validation mechanism. | ||
| 546 | * It seems that it fails to initialize if the address is >= 128 MiB. | ||
| 530 | */ | 547 | */ |
| 531 | static int vce_v1_0_ensure_vcpu_bo_32bit_addr(struct amdgpu_device *adev) | 548 | static int vce_v1_0_ensure_vcpu_bo_32bit_addr(struct amdgpu_device *adev) |
| 532 | { | 549 | { |
| 533 | u64 bo_size = amdgpu_bo_size(adev->vce.vcpu_bo); | 550 | u64 bo_size = amdgpu_bo_size(adev->vce.vcpu_bo); |
| 534 | u64 max_vcpu_bo_addr = 0xffffffff - bo_size; | 551 | u64 max_vcpu_bo_addr = 0x07ffffff - bo_size; |
| 535 | u64 num_pages = ALIGN(bo_size, AMDGPU_GPU_PAGE_SIZE) / AMDGPU_GPU_PAGE_SIZE; | 552 | u64 num_pages = ALIGN(bo_size, AMDGPU_GPU_PAGE_SIZE) / AMDGPU_GPU_PAGE_SIZE; |
| 536 | u64 pa = amdgpu_gmc_vram_pa(adev, adev->vce.vcpu_bo); | 553 | u64 pa = amdgpu_gmc_vram_pa(adev, adev->vce.vcpu_bo); |
| 537 | u64 flags = AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE | AMDGPU_PTE_VALID; | 554 | u64 flags = AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE | AMDGPU_PTE_VALID; |
| 538 | u64 vce_gart_start_offs; | 555 | u64 vce_gart_start_offs; |
| 539 | int r; | 556 | int r; |
| 540 | 557 | ||
| 541 | r = amdgpu_gtt_mgr_alloc_entries(&adev->mman.gtt_mgr, | 558 | if (adev->gmc.vram_start < adev->gmc.gart_start) |
| 542 | &adev->vce.gart_node, num_pages, | 559 | return amdgpu_bo_gpu_offset(adev->vce.vcpu_bo) <= max_vcpu_bo_addr ? 0 : -EINVAL; |
| 543 | DRM_MM_INSERT_LOW); | 560 | |
| 544 | if (r) | 561 | if (!drm_mm_node_allocated(&adev->vce.gart_node)) { |
| 545 | return r; | 562 | r = amdgpu_gtt_mgr_alloc_entries(&adev->mman.gtt_mgr, |
| 563 | &adev->vce.gart_node, num_pages, | ||
| 564 | DRM_MM_INSERT_LOW); | ||
| 565 | if (r) | ||
| 566 | return r; | ||
| 567 | } | ||
| 546 | 568 | ||
| 547 | vce_gart_start_offs = amdgpu_gtt_node_to_byte_offset(&adev->vce.gart_node); | 569 | vce_gart_start_offs = amdgpu_gtt_node_to_byte_offset(&adev->vce.gart_node); |
| 548 | 570 | ||
| @@ -553,8 +575,6 @@ static int vce_v1_0_ensure_vcpu_bo_32bit_addr(struct amdgpu_device *adev) · 4 unmodified lines | |||
| 553 | amdgpu_gart_map_vram_range(adev, pa, adev->vce.gart_node.start, | 575 | amdgpu_gart_map_vram_range(adev, pa, adev->vce.gart_node.start, |
| 554 | num_pages, flags, adev->gart.ptr); | 576 | num_pages, flags, adev->gart.ptr); |
| 555 | adev->vce.gpu_addr = adev->gmc.gart_start + vce_gart_start_offs; | 577 | adev->vce.gpu_addr = adev->gmc.gart_start + vce_gart_start_offs; |
| 556 | if (adev->vce.gpu_addr > max_vcpu_bo_addr) | ||
| 557 | return -EINVAL; | ||
| 558 | 578 | ||
| 559 | return 0; | 579 | return 0; |
| 560 | } | 580 | } |
| @@ -574,10 +594,7 @@ static int vce_v1_0_sw_init(struct amdgpu_ip_block *ip_block) · 13 unmodified lines | |||
| 574 | if (r) | 594 | if (r) |
| 575 | return r; | 595 | return r; |
| 576 | 596 | ||
| 577 | r = amdgpu_vce_resume(adev); | 597 | r = vce_v1_0_load_fw(adev); |
| 578 | if (r) | ||
| 579 | return r; | ||
| 580 | r = vce_v1_0_load_fw_signature(adev); | ||
| 581 | if (r) | 598 | if (r) |
| 582 | return r; | 599 | return r; |
| 583 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); | 600 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); |
| @@ -696,10 +713,7 @@ static int vce_v1_0_resume(struct amdgpu_ip_block *ip_block) · 112 unmodified lines | |||
| 696 | struct amdgpu_device *adev = ip_block->adev; | 713 | struct amdgpu_device *adev = ip_block->adev; |
| 697 | int r; | 714 | int r; |
| 698 | 715 | ||
| 699 | r = amdgpu_vce_resume(adev); | 716 | r = vce_v1_0_load_fw(adev); |
| 700 | if (r) | ||
| 701 | return r; | ||
| 702 | r = vce_v1_0_load_fw_signature(adev); | ||
| 703 | if (r) | 717 | if (r) |
| 704 | return r; | 718 | return r; |
| 705 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); | 719 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); |
| @@ -37,9 +37,14 @@ · 36 unmodified lines | |||
| 37 | #include "oss/oss_2_0_d.h" | 37 | #include "oss/oss_2_0_d.h" |
| 38 | #include "oss/oss_2_0_sh_mask.h" | 38 | #include "oss/oss_2_0_sh_mask.h" |
| 39 | 39 | ||
| 40 | |||
| 41 | /* Use 24K to be safe. The FW supposedly only requires 23744 bytes. */ | ||
| 42 | #define VCE_V2_0_DATA_ENTRY_SIZE (24 * 1024) | ||
| 43 | |||
| 40 | #define VCE_V2_0_FW_SIZE (256 * 1024) | 44 | #define VCE_V2_0_FW_SIZE (256 * 1024) |
| 41 | #define VCE_V2_0_STACK_SIZE (64 * 1024) | 45 | #define VCE_V2_0_STACK_SIZE (64 * 1024) |
| 42 | #define VCE_V2_0_DATA_SIZE (23552 * AMDGPU_MAX_VCE_HANDLES) | 46 | #define VCE_V2_0_DATA_SIZE (VCE_V2_0_DATA_ENTRY_SIZE * (AMDGPU_MAX_VCE_HANDLES + 1)) |
| 47 | |||
| 43 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 | 48 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 |
| 44 | 49 | ||
| 45 | static void vce_v2_0_set_ring_funcs(struct amdgpu_device *adev); | 50 | static void vce_v2_0_set_ring_funcs(struct amdgpu_device *adev); |
| @@ -183,7 +188,7 @@ static void vce_v2_0_mc_resume(struct amdgpu_device *adev) · 137 unmodified lines | |||
| 183 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); | 188 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); |
| 184 | 189 | ||
| 185 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; | 190 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; |
| 186 | size = VCE_V2_0_FW_SIZE; | 191 | size = VCE_V2_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; |
| 187 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); | 192 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); |
| 188 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); | 193 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 189 | 194 | ||
| @@ -574,7 +574,7 @@ static void vce_v3_0_mc_resume(struct amdgpu_device *adev, int idx) · 573 unmodified lines | |||
| 574 | } else | 574 | } else |
| 575 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); | 575 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); |
| 576 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; | 576 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; |
| 577 | size = VCE_V3_0_FW_SIZE; | 577 | size = VCE_V3_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; |
| 578 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); | 578 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); |
| 579 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); | 579 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 580 | 580 | ||
| @@ -2113,6 +2113,7 @@ static const struct amd_ip_funcs vcn_v2_0_ip_funcs = { · 2112 unmodified lines | |||
| 2113 | static const struct amdgpu_ring_funcs vcn_v2_0_dec_ring_vm_funcs = { | 2113 | static const struct amdgpu_ring_funcs vcn_v2_0_dec_ring_vm_funcs = { |
| 2114 | .type = AMDGPU_RING_TYPE_VCN_DEC, | 2114 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 2115 | .align_mask = 0xf, | 2115 | .align_mask = 0xf, |
| 2116 | .no_user_fence = true, | ||
| 2116 | .secure_submission_supported = true, | 2117 | .secure_submission_supported = true, |
| 2117 | .get_rptr = vcn_v2_0_dec_ring_get_rptr, | 2118 | .get_rptr = vcn_v2_0_dec_ring_get_rptr, |
| 2118 | .get_wptr = vcn_v2_0_dec_ring_get_wptr, | 2119 | .get_wptr = vcn_v2_0_dec_ring_get_wptr, |
| @@ -2145,6 +2146,7 @@ static const struct amdgpu_ring_funcs vcn_v2_0_enc_ring_vm_funcs = { · 26 unmodified lines | |||
| 2145 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 2146 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 2146 | .align_mask = 0x3f, | 2147 | .align_mask = 0x3f, |
| 2147 | .nop = VCN_ENC_CMD_NO_OP, | 2148 | .nop = VCN_ENC_CMD_NO_OP, |
| 2149 | .no_user_fence = true, | ||
| 2148 | .get_rptr = vcn_v2_0_enc_ring_get_rptr, | 2150 | .get_rptr = vcn_v2_0_enc_ring_get_rptr, |
| 2149 | .get_wptr = vcn_v2_0_enc_ring_get_wptr, | 2151 | .get_wptr = vcn_v2_0_enc_ring_get_wptr, |
| 2150 | .set_wptr = vcn_v2_0_enc_ring_set_wptr, | 2152 | .set_wptr = vcn_v2_0_enc_ring_set_wptr, |
| @@ -1778,6 +1778,7 @@ static void vcn_v2_5_dec_ring_set_wptr(struct amdgpu_ring *ring) · 1777 unmodified lines | |||
| 1778 | static const struct amdgpu_ring_funcs vcn_v2_5_dec_ring_vm_funcs = { | 1778 | static const struct amdgpu_ring_funcs vcn_v2_5_dec_ring_vm_funcs = { |
| 1779 | .type = AMDGPU_RING_TYPE_VCN_DEC, | 1779 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 1780 | .align_mask = 0xf, | 1780 | .align_mask = 0xf, |
| 1781 | .no_user_fence = true, | ||
| 1781 | .secure_submission_supported = true, | 1782 | .secure_submission_supported = true, |
| 1782 | .get_rptr = vcn_v2_5_dec_ring_get_rptr, | 1783 | .get_rptr = vcn_v2_5_dec_ring_get_rptr, |
| 1783 | .get_wptr = vcn_v2_5_dec_ring_get_wptr, | 1784 | .get_wptr = vcn_v2_5_dec_ring_get_wptr, |
| @@ -1879,6 +1880,7 @@ static const struct amdgpu_ring_funcs vcn_v2_5_enc_ring_vm_funcs = { · 95 unmodified lines | |||
| 1879 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1880 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1880 | .align_mask = 0x3f, | 1881 | .align_mask = 0x3f, |
| 1881 | .nop = VCN_ENC_CMD_NO_OP, | 1882 | .nop = VCN_ENC_CMD_NO_OP, |
| 1883 | .no_user_fence = true, | ||
| 1882 | .get_rptr = vcn_v2_5_enc_ring_get_rptr, | 1884 | .get_rptr = vcn_v2_5_enc_ring_get_rptr, |
| 1883 | .get_wptr = vcn_v2_5_enc_ring_get_wptr, | 1885 | .get_wptr = vcn_v2_5_enc_ring_get_wptr, |
| 1884 | .set_wptr = vcn_v2_5_enc_ring_set_wptr, | 1886 | .set_wptr = vcn_v2_5_enc_ring_set_wptr, |
| @@ -1856,6 +1856,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_dec_sw_ring_vm_funcs = { · 1855 unmodified lines | |||
| 1856 | .type = AMDGPU_RING_TYPE_VCN_DEC, | 1856 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 1857 | .align_mask = 0x3f, | 1857 | .align_mask = 0x3f, |
| 1858 | .nop = VCN_DEC_SW_CMD_NO_OP, | 1858 | .nop = VCN_DEC_SW_CMD_NO_OP, |
| 1859 | .no_user_fence = true, | ||
| 1859 | .secure_submission_supported = true, | 1860 | .secure_submission_supported = true, |
| 1860 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, | 1861 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, |
| 1861 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, | 1862 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, |
| @@ -1972,6 +1973,7 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 110 unmodified lines | |||
| 1972 | 1973 | ||
| 1973 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { | 1974 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { |
| 1974 | uint32_t offset, size, *create; | 1975 | uint32_t offset, size, *create; |
| 1976 | uint64_t buf_end; | ||
| 1975 | 1977 | ||
| 1976 | if (msg[0] != RDECODE_MESSAGE_CREATE) | 1978 | if (msg[0] != RDECODE_MESSAGE_CREATE) |
| 1977 | continue; | 1979 | continue; |
| @@ -1979,7 +1981,8 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1 unmodified line | |||
| 1979 | offset = msg[1]; | 1981 | offset = msg[1]; |
| 1980 | size = msg[2]; | 1982 | size = msg[2]; |
| 1981 | 1983 | ||
| 1982 | if (size < 4 || offset + size > end - addr) { | 1984 | if (size < 4 || check_add_overflow(offset, size, &buf_end) || |
| 1985 | buf_end > end - addr) { | ||
| 1983 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); | 1986 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); |
| 1984 | r = -EINVAL; | 1987 | r = -EINVAL; |
| 1985 | goto out; | 1988 | goto out; |
| @@ -2036,6 +2039,7 @@ static int vcn_v3_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, · 50 unmodified lines | |||
| 2036 | static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = { | 2039 | static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = { |
| 2037 | .type = AMDGPU_RING_TYPE_VCN_DEC, | 2040 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 2038 | .align_mask = 0xf, | 2041 | .align_mask = 0xf, |
| 2042 | .no_user_fence = true, | ||
| 2039 | .secure_submission_supported = true, | 2043 | .secure_submission_supported = true, |
| 2040 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, | 2044 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, |
| 2041 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, | 2045 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, |
| @@ -2138,6 +2142,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_enc_ring_vm_funcs = { · 96 unmodified lines | |||
| 2138 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 2142 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 2139 | .align_mask = 0x3f, | 2143 | .align_mask = 0x3f, |
| 2140 | .nop = VCN_ENC_CMD_NO_OP, | 2144 | .nop = VCN_ENC_CMD_NO_OP, |
| 2145 | .no_user_fence = true, | ||
| 2141 | .get_rptr = vcn_v3_0_enc_ring_get_rptr, | 2146 | .get_rptr = vcn_v3_0_enc_ring_get_rptr, |
| 2142 | .get_wptr = vcn_v3_0_enc_ring_get_wptr, | 2147 | .get_wptr = vcn_v3_0_enc_ring_get_wptr, |
| 2143 | .set_wptr = vcn_v3_0_enc_ring_set_wptr, | 2148 | .set_wptr = vcn_v3_0_enc_ring_set_wptr, |
| @@ -1889,6 +1889,7 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1888 unmodified lines | |||
| 1889 | 1889 | ||
| 1890 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { | 1890 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { |
| 1891 | uint32_t offset, size, *create; | 1891 | uint32_t offset, size, *create; |
| 1892 | uint64_t buf_end; | ||
| 1892 | 1893 | ||
| 1893 | if (msg[0] != RDECODE_MESSAGE_CREATE) | 1894 | if (msg[0] != RDECODE_MESSAGE_CREATE) |
| 1894 | continue; | 1895 | continue; |
| @@ -1896,7 +1897,8 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1 unmodified line | |||
| 1896 | offset = msg[1]; | 1897 | offset = msg[1]; |
| 1897 | size = msg[2]; | 1898 | size = msg[2]; |
| 1898 | 1899 | ||
| 1899 | if (size < 4 || offset + size > end - addr) { | 1900 | if (size < 4 || check_add_overflow(offset, size, &buf_end) || |
| 1901 | buf_end > end - addr) { | ||
| 1900 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); | 1902 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); |
| 1901 | r = -EINVAL; | 1903 | r = -EINVAL; |
| 1902 | goto out; | 1904 | goto out; |
| @@ -1994,6 +1996,7 @@ static struct amdgpu_ring_funcs vcn_v4_0_unified_ring_vm_funcs = { · 91 unmodified lines | |||
| 1994 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1996 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1995 | .align_mask = 0x3f, | 1997 | .align_mask = 0x3f, |
| 1996 | .nop = VCN_ENC_CMD_NO_OP, | 1998 | .nop = VCN_ENC_CMD_NO_OP, |
| 1999 | .no_user_fence = true, | ||
| 1997 | .extra_bytes = sizeof(struct amdgpu_vcn_rb_metadata), | 2000 | .extra_bytes = sizeof(struct amdgpu_vcn_rb_metadata), |
| 1998 | .get_rptr = vcn_v4_0_unified_ring_get_rptr, | 2001 | .get_rptr = vcn_v4_0_unified_ring_get_rptr, |
| 1999 | .get_wptr = vcn_v4_0_unified_ring_get_wptr, | 2002 | .get_wptr = vcn_v4_0_unified_ring_get_wptr, |
| @@ -1775,6 +1775,7 @@ static const struct amdgpu_ring_funcs vcn_v4_0_3_unified_ring_vm_funcs = { · 1774 unmodified lines | |||
| 1775 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1775 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1776 | .align_mask = 0x3f, | 1776 | .align_mask = 0x3f, |
| 1777 | .nop = VCN_ENC_CMD_NO_OP, | 1777 | .nop = VCN_ENC_CMD_NO_OP, |
| 1778 | .no_user_fence = true, | ||
| 1778 | .get_rptr = vcn_v4_0_3_unified_ring_get_rptr, | 1779 | .get_rptr = vcn_v4_0_3_unified_ring_get_rptr, |
| 1779 | .get_wptr = vcn_v4_0_3_unified_ring_get_wptr, | 1780 | .get_wptr = vcn_v4_0_3_unified_ring_get_wptr, |
| 1780 | .set_wptr = vcn_v4_0_3_unified_ring_set_wptr, | 1781 | .set_wptr = vcn_v4_0_3_unified_ring_set_wptr, |
| @@ -1483,6 +1483,7 @@ static struct amdgpu_ring_funcs vcn_v4_0_5_unified_ring_vm_funcs = { · 1482 unmodified lines | |||
| 1483 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1483 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1484 | .align_mask = 0x3f, | 1484 | .align_mask = 0x3f, |
| 1485 | .nop = VCN_ENC_CMD_NO_OP, | 1485 | .nop = VCN_ENC_CMD_NO_OP, |
| 1486 | .no_user_fence = true, | ||
| 1486 | .get_rptr = vcn_v4_0_5_unified_ring_get_rptr, | 1487 | .get_rptr = vcn_v4_0_5_unified_ring_get_rptr, |
| 1487 | .get_wptr = vcn_v4_0_5_unified_ring_get_wptr, | 1488 | .get_wptr = vcn_v4_0_5_unified_ring_get_wptr, |
| 1488 | .set_wptr = vcn_v4_0_5_unified_ring_set_wptr, | 1489 | .set_wptr = vcn_v4_0_5_unified_ring_set_wptr, |
| @@ -1207,6 +1207,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_0_unified_ring_vm_funcs = { · 1206 unmodified lines | |||
| 1207 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1207 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1208 | .align_mask = 0x3f, | 1208 | .align_mask = 0x3f, |
| 1209 | .nop = VCN_ENC_CMD_NO_OP, | 1209 | .nop = VCN_ENC_CMD_NO_OP, |
| 1210 | .no_user_fence = true, | ||
| 1210 | .get_rptr = vcn_v5_0_0_unified_ring_get_rptr, | 1211 | .get_rptr = vcn_v5_0_0_unified_ring_get_rptr, |
| 1211 | .get_wptr = vcn_v5_0_0_unified_ring_get_wptr, | 1212 | .get_wptr = vcn_v5_0_0_unified_ring_get_wptr, |
| 1212 | .set_wptr = vcn_v5_0_0_unified_ring_set_wptr, | 1213 | .set_wptr = vcn_v5_0_0_unified_ring_set_wptr, |
| @@ -1419,6 +1419,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_1_unified_ring_vm_funcs = { · 1418 unmodified lines | |||
| 1419 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 1419 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1420 | .align_mask = 0x3f, | 1420 | .align_mask = 0x3f, |
| 1421 | .nop = VCN_ENC_CMD_NO_OP, | 1421 | .nop = VCN_ENC_CMD_NO_OP, |
| 1422 | .no_user_fence = true, | ||
| 1422 | .get_rptr = vcn_v5_0_1_unified_ring_get_rptr, | 1423 | .get_rptr = vcn_v5_0_1_unified_ring_get_rptr, |
| 1423 | .get_wptr = vcn_v5_0_1_unified_ring_get_wptr, | 1424 | .get_wptr = vcn_v5_0_1_unified_ring_get_wptr, |
| 1424 | .set_wptr = vcn_v5_0_1_unified_ring_set_wptr, | 1425 | .set_wptr = vcn_v5_0_1_unified_ring_set_wptr, |
| @@ -994,6 +994,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_2_unified_ring_vm_funcs = { · 993 unmodified lines | |||
| 994 | .type = AMDGPU_RING_TYPE_VCN_ENC, | 994 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 995 | .align_mask = 0x3f, | 995 | .align_mask = 0x3f, |
| 996 | .nop = VCN_ENC_CMD_NO_OP, | 996 | .nop = VCN_ENC_CMD_NO_OP, |
| 997 | .no_user_fence = true, | ||
| 997 | .get_rptr = vcn_v5_0_2_unified_ring_get_rptr, | 998 | .get_rptr = vcn_v5_0_2_unified_ring_get_rptr, |
| 998 | .get_wptr = vcn_v5_0_2_unified_ring_get_wptr, | 999 | .get_wptr = vcn_v5_0_2_unified_ring_get_wptr, |
| 999 | .set_wptr = vcn_v5_0_2_unified_ring_set_wptr, | 1000 | .set_wptr = vcn_v5_0_2_unified_ring_set_wptr, |
| @@ -25,6 +25,7 @@ · 24 unmodified lines | |||
| 25 | #include <linux/err.h> | 25 | #include <linux/err.h> |
| 26 | #include <linux/fs.h> | 26 | #include <linux/fs.h> |
| 27 | #include <linux/file.h> | 27 | #include <linux/file.h> |
| 28 | #include <linux/overflow.h> | ||
| 28 | #include <linux/sched.h> | 29 | #include <linux/sched.h> |
| 29 | #include <linux/slab.h> | 30 | #include <linux/slab.h> |
| 30 | #include <linux/uaccess.h> | 31 | #include <linux/uaccess.h> |
| @@ -66,6 +67,21 @@ static const struct class kfd_class = { · 35 unmodified lines | |||
| 66 | .name = kfd_dev_name, | 67 | .name = kfd_dev_name, |
| 67 | }; | 68 | }; |
| 68 | 69 | ||
| 70 | /* | ||
| 71 | * Cache the address space of the chardev on first open so that the reset | ||
| 72 | * path can drop all userspace mappings of doorbell and MMIO ranges via | ||
| 73 | * unmap_mapping_range(). | ||
| 74 | */ | ||
| 75 | static struct address_space *kfd_dev_mapping; | ||
| 76 | |||
| 77 | void kfd_dev_unmap_mapping_range(loff_t const holebegin, loff_t const holelen) | ||
| 78 | { | ||
| 79 | struct address_space *mapping = READ_ONCE(kfd_dev_mapping); | ||
| 80 | |||
| 81 | if (mapping) | ||
| 82 | unmap_mapping_range(mapping, holebegin, holelen, 1); | ||
| 83 | } | ||
| 84 | |||
| 69 | static inline struct kfd_process_device *kfd_lock_pdd_by_id(struct kfd_process *p, __u32 gpu_id) | 85 | static inline struct kfd_process_device *kfd_lock_pdd_by_id(struct kfd_process *p, __u32 gpu_id) |
| 70 | { | 86 | { |
| 71 | struct kfd_process_device *pdd; | 87 | struct kfd_process_device *pdd; |
| @@ -132,6 +148,13 @@ static int kfd_open(struct inode *inode, struct file *filep) · 60 unmodified lines | |||
| 132 | if (iminor(inode) != 0) | 148 | if (iminor(inode) != 0) |
| 133 | return -ENODEV; | 149 | return -ENODEV; |
| 134 | 150 | ||
| 151 | /* | ||
| 152 | * /dev/kfd is a single chardev so all opens share one inode. Cache | ||
| 153 | * its address_space on the first open for use by the reset path. | ||
| 154 | */ | ||
| 155 | if (!READ_ONCE(kfd_dev_mapping)) | ||
| 156 | cmpxchg(&kfd_dev_mapping, NULL, inode->i_mapping); | ||
| 157 | |||
| 135 | is_32bit_user_mode = in_compat_syscall(); | 158 | is_32bit_user_mode = in_compat_syscall(); |
| 136 | 159 | ||
| 137 | if (is_32bit_user_mode) { | 160 | if (is_32bit_user_mode) { |
| @@ -1359,7 +1382,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 1221 unmodified lines | |||
| 1359 | peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]); | 1382 | peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]); |
| 1360 | if (WARN_ON_ONCE(!peer_pdd)) | 1383 | if (WARN_ON_ONCE(!peer_pdd)) |
| 1361 | continue; | 1384 | continue; |
| 1362 | kfd_flush_tlb(peer_pdd, TLB_FLUSH_LEGACY); | 1385 | kfd_flush_tlb(peer_pdd); |
| 1363 | } | 1386 | } |
| 1364 | kfree(devices_arr); | 1387 | kfree(devices_arr); |
| 1365 | 1388 | ||
| @@ -1454,7 +1477,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 88 unmodified lines | |||
| 1454 | if (WARN_ON_ONCE(!peer_pdd)) | 1477 | if (WARN_ON_ONCE(!peer_pdd)) |
| 1455 | continue; | 1478 | continue; |
| 1456 | if (flush_tlb) | 1479 | if (flush_tlb) |
| 1457 | kfd_flush_tlb(peer_pdd, TLB_FLUSH_HEAVYWEIGHT); | 1480 | kfd_flush_tlb(peer_pdd); |
| 1458 | 1481 | ||
| 1459 | /* Remove dma mapping after tlb flush to avoid IO_PAGE_FAULT */ | 1482 | /* Remove dma mapping after tlb flush to avoid IO_PAGE_FAULT */ |
| 1460 | err = amdgpu_amdkfd_gpuvm_dmaunmap_mem(mem, peer_pdd->drm_priv); | 1483 | err = amdgpu_amdkfd_gpuvm_dmaunmap_mem(mem, peer_pdd->drm_priv); |
| @@ -1695,6 +1718,16 @@ static int kfd_ioctl_smi_events(struct file *filep, · 234 unmodified lines | |||
| 1695 | return kfd_smi_event_open(pdd->dev, &args->anon_fd); | 1718 | return kfd_smi_event_open(pdd->dev, &args->anon_fd); |
| 1696 | } | 1719 | } |
| 1697 | 1720 | ||
| 1721 | static int kfd_ioctl_svm_validate(void *kdata, unsigned int usize) | ||
| 1722 | { | ||
| 1723 | struct kfd_ioctl_svm_args *args = kdata; | ||
| 1724 | size_t expected = struct_size(args, attrs, args->nattr); | ||
| 1725 | |||
| 1726 | if (expected == SIZE_MAX || usize < expected) | ||
| 1727 | return -EINVAL; | ||
| 1728 | return 0; | ||
| 1729 | } | ||
| 1730 | |||
| 1698 | #if IS_ENABLED(CONFIG_HSA_AMD_SVM) | 1731 | #if IS_ENABLED(CONFIG_HSA_AMD_SVM) |
| 1699 | 1732 | ||
| 1700 | static int kfd_ioctl_set_xnack_mode(struct file *filep, | 1733 | static int kfd_ioctl_set_xnack_mode(struct file *filep, |
| @@ -2267,6 +2300,11 @@ static int criu_restore_devices(struct kfd_process *p, · 566 unmodified lines | |||
| 2267 | ret = -EINVAL; | 2300 | ret = -EINVAL; |
| 2268 | goto exit; | 2301 | goto exit; |
| 2269 | } | 2302 | } |
| 2303 | |||
| 2304 | if (pdd->drm_file) { | ||
| 2305 | ret = -EINVAL; | ||
| 2306 | goto exit; | ||
| 2307 | } | ||
| 2270 | pdd->user_gpu_id = device_buckets[i].user_gpu_id; | 2308 | pdd->user_gpu_id = device_buckets[i].user_gpu_id; |
| 2271 | 2309 | ||
| 2272 | drm_file = fget(device_buckets[i].drm_fd); | 2310 | drm_file = fget(device_buckets[i].drm_fd); |
| @@ -2277,11 +2315,6 @@ static int criu_restore_devices(struct kfd_process *p, · 4 unmodified lines | |||
| 2277 | goto exit; | 2315 | goto exit; |
| 2278 | } | 2316 | } |
| 2279 | 2317 | ||
| 2280 | if (pdd->drm_file) { | ||
| 2281 | ret = -EINVAL; | ||
| 2282 | goto exit; | ||
| 2283 | } | ||
| 2284 | |||
| 2285 | /* create the vm using render nodes for kfd pdd */ | 2318 | /* create the vm using render nodes for kfd pdd */ |
| 2286 | if (kfd_process_device_init_vm(pdd, drm_file)) { | 2319 | if (kfd_process_device_init_vm(pdd, drm_file)) { |
| 2287 | pr_err("could not init vm for given pdd\n"); | 2320 | pr_err("could not init vm for given pdd\n"); |
| @@ -3209,7 +3242,11 @@ static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, v · 921 unmodified lines | |||
| 3209 | 3242 | ||
| 3210 | #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \ | 3243 | #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \ |
| 3211 | [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \ | 3244 | [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \ |
| 3212 | .cmd_drv = 0, .name = #ioctl} | 3245 | .validate = NULL, .cmd_drv = 0, .name = #ioctl} |
| 3246 | |||
| 3247 | #define AMDKFD_IOCTL_DEF_V(ioctl, _func, _validate, _flags) \ | ||
| 3248 | [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \ | ||
| 3249 | .validate = _validate, .cmd_drv = 0, .name = #ioctl} | ||
| 3213 | 3250 | ||
| 3214 | /** Ioctl table */ | 3251 | /** Ioctl table */ |
| 3215 | static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { | 3252 | static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { |
| @@ -3306,7 +3343,8 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { · 90 unmodified lines | |||
| 3306 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SMI_EVENTS, | 3343 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SMI_EVENTS, |
| 3307 | kfd_ioctl_smi_events, 0), | 3344 | kfd_ioctl_smi_events, 0), |
| 3308 | 3345 | ||
| 3309 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SVM, kfd_ioctl_svm, 0), | 3346 | AMDKFD_IOCTL_DEF_V(AMDKFD_IOC_SVM, kfd_ioctl_svm, |
| 3347 | kfd_ioctl_svm_validate, 0), | ||
| 3310 | 3348 | ||
| 3311 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_XNACK_MODE, | 3349 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_XNACK_MODE, |
| 3312 | kfd_ioctl_set_xnack_mode, 0), | 3350 | kfd_ioctl_set_xnack_mode, 0), |
| @@ -3431,6 +3469,12 @@ static long kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 118 unmodified lines | |||
| 3431 | memset(kdata, 0, usize); | 3469 | memset(kdata, 0, usize); |
| 3432 | } | 3470 | } |
| 3433 | 3471 | ||
| 3472 | if (ioctl->validate) { | ||
| 3473 | retcode = ioctl->validate(kdata, usize); | ||
| 3474 | if (retcode) | ||
| 3475 | goto err_i1; | ||
| 3476 | } | ||
| 3477 | |||
| 3434 | retcode = func(filep, process, kdata); | 3478 | retcode = func(filep, process, kdata); |
| 3435 | 3479 | ||
| 3436 | if (cmd & IOC_OUT) | 3480 | if (cmd & IOC_OUT) |
| @@ -1737,37 +1737,6 @@ bool kgd2kfd_vmfault_fast_path(struct amdgpu_device *adev, struct amdgpu_iv_entr · 1736 unmodified lines | |||
| 1737 | return false; | 1737 | return false; |
| 1738 | } | 1738 | } |
| 1739 | 1739 | ||
| 1740 | /* check if there is kfd process still uses adev */ | ||
| 1741 | static bool kgd2kfd_check_device_idle(struct amdgpu_device *adev) | ||
| 1742 | { | ||
| 1743 | struct kfd_process *p; | ||
| 1744 | struct hlist_node *p_temp; | ||
| 1745 | unsigned int temp; | ||
| 1746 | struct kfd_node *dev; | ||
| 1747 | |||
| 1748 | mutex_lock(&kfd_processes_mutex); | ||
| 1749 | |||
| 1750 | if (hash_empty(kfd_processes_table)) { | ||
| 1751 | mutex_unlock(&kfd_processes_mutex); | ||
| 1752 | return true; | ||
| 1753 | } | ||
| 1754 | |||
| 1755 | /* check if there is device still use adev */ | ||
| 1756 | hash_for_each_safe(kfd_processes_table, temp, p_temp, p, kfd_processes) { | ||
| 1757 | for (int i = 0; i < p->n_pdds; i++) { | ||
| 1758 | dev = p->pdds[i]->dev; | ||
| 1759 | if (dev->adev == adev) { | ||
| 1760 | mutex_unlock(&kfd_processes_mutex); | ||
| 1761 | return false; | ||
| 1762 | } | ||
| 1763 | } | ||
| 1764 | } | ||
| 1765 | |||
| 1766 | mutex_unlock(&kfd_processes_mutex); | ||
| 1767 | |||
| 1768 | return true; | ||
| 1769 | } | ||
| 1770 | |||
| 1771 | /** kgd2kfd_teardown_processes - gracefully tear down existing | 1740 | /** kgd2kfd_teardown_processes - gracefully tear down existing |
| 1772 | * kfd processes that use adev | 1741 | * kfd processes that use adev |
| 1773 | * | 1742 | * |
| @@ -1800,7 +1769,7 @@ void kgd2kfd_teardown_processes(struct amdgpu_device *adev) · 26 unmodified lines | |||
| 1800 | mutex_unlock(&kfd_processes_mutex); | 1769 | mutex_unlock(&kfd_processes_mutex); |
| 1801 | 1770 | ||
| 1802 | /* wait all kfd processes use adev terminate */ | 1771 | /* wait all kfd processes use adev terminate */ |
| 1803 | while (!kgd2kfd_check_device_idle(adev)) | 1772 | while (!!atomic_read(&adev->kfd.dev->kfd_processes_count)) |
| 1804 | cond_resched(); | 1773 | cond_resched(); |
| 1805 | } | 1774 | } |
| 1806 | 1775 | ||
| @@ -475,6 +475,9 @@ static int allocate_doorbell(struct qcm_process_device *qpd, · 474 unmodified lines | |||
| 475 | } else { | 475 | } else { |
| 476 | /* For CP queues on SOC15 */ | 476 | /* For CP queues on SOC15 */ |
| 477 | if (restore_id) { | 477 | if (restore_id) { |
| 478 | if (*restore_id >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | ||
| 479 | return -EINVAL; | ||
| 480 | |||
| 478 | /* make sure that ID is free */ | 481 | /* make sure that ID is free */ |
| 479 | if (__test_and_set_bit(*restore_id, qpd->doorbell_bitmap)) | 482 | if (__test_and_set_bit(*restore_id, qpd->doorbell_bitmap)) |
| 480 | return -EINVAL; | 483 | return -EINVAL; |
| @@ -572,7 +575,7 @@ static int allocate_vmid(struct device_queue_manager *dqm, · 91 unmodified lines | |||
| 572 | qpd->vmid, | 575 | qpd->vmid, |
| 573 | qpd->page_table_base); | 576 | qpd->page_table_base); |
| 574 | /* invalidate the VM context after pasid and vmid mapping is set up */ | 577 | /* invalidate the VM context after pasid and vmid mapping is set up */ |
| 575 | kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); | 578 | kfd_flush_tlb(qpd_to_pdd(qpd)); |
| 576 | 579 | ||
| 577 | if (dqm->dev->kfd2kgd->set_scratch_backing_va) | 580 | if (dqm->dev->kfd2kgd->set_scratch_backing_va) |
| 578 | dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->adev, | 581 | dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->adev, |
| @@ -610,7 +613,7 @@ static void deallocate_vmid(struct device_queue_manager *dqm, · 31 unmodified lines | |||
| 610 | if (flush_texture_cache_nocpsch(q->device, qpd)) | 613 | if (flush_texture_cache_nocpsch(q->device, qpd)) |
| 611 | dev_err(dev, "Failed to flush TC\n"); | 614 | dev_err(dev, "Failed to flush TC\n"); |
| 612 | 615 | ||
| 613 | kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); | 616 | kfd_flush_tlb(qpd_to_pdd(qpd)); |
| 614 | 617 | ||
| 615 | /* Release the vmid mapping */ | 618 | /* Release the vmid mapping */ |
| 616 | set_pasid_vmid_mapping(dqm, 0, qpd->vmid); | 619 | set_pasid_vmid_mapping(dqm, 0, qpd->vmid); |
| @@ -1284,7 +1287,7 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, · 667 unmodified lines | |||
| 1284 | dqm->dev->adev, | 1287 | dqm->dev->adev, |
| 1285 | qpd->vmid, | 1288 | qpd->vmid, |
| 1286 | qpd->page_table_base); | 1289 | qpd->page_table_base); |
| 1287 | kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); | 1290 | kfd_flush_tlb(pdd); |
| 1288 | } | 1291 | } |
| 1289 | 1292 | ||
| 1290 | /* Take a safe reference to the mm_struct, which may otherwise | 1293 | /* Take a safe reference to the mm_struct, which may otherwise |
| @@ -1587,6 +1590,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 296 unmodified lines | |||
| 1587 | } | 1590 | } |
| 1588 | 1591 | ||
| 1589 | if (restore_sdma_id) { | 1592 | if (restore_sdma_id) { |
| 1593 | if (*restore_sdma_id >= get_num_sdma_queues(dqm)) | ||
| 1594 | return -EINVAL; | ||
| 1595 | |||
| 1590 | /* Re-use existing sdma_id */ | 1596 | /* Re-use existing sdma_id */ |
| 1591 | if (!test_bit(*restore_sdma_id, dqm->sdma_bitmap)) { | 1597 | if (!test_bit(*restore_sdma_id, dqm->sdma_bitmap)) { |
| 1592 | dev_err(dev, "SDMA queue already in use\n"); | 1598 | dev_err(dev, "SDMA queue already in use\n"); |
| @@ -1613,6 +1619,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 20 unmodified lines | |||
| 1613 | return -ENOMEM; | 1619 | return -ENOMEM; |
| 1614 | } | 1620 | } |
| 1615 | if (restore_sdma_id) { | 1621 | if (restore_sdma_id) { |
| 1622 | if (*restore_sdma_id >= get_num_xgmi_sdma_queues(dqm)) | ||
| 1623 | return -EINVAL; | ||
| 1624 | |||
| 1616 | /* Re-use existing sdma_id */ | 1625 | /* Re-use existing sdma_id */ |
| 1617 | if (!test_bit(*restore_sdma_id, dqm->xgmi_sdma_bitmap)) { | 1626 | if (!test_bit(*restore_sdma_id, dqm->xgmi_sdma_bitmap)) { |
| 1618 | dev_err(dev, "SDMA queue already in use\n"); | 1627 | dev_err(dev, "SDMA queue already in use\n"); |
| @@ -3299,12 +3308,14 @@ static void copy_context_work_handler(struct work_struct *work) · 1680 unmodified lines | |||
| 3299 | 3308 | ||
| 3300 | static uint32_t *get_queue_ids(uint32_t num_queues, uint32_t *usr_queue_id_array) | 3309 | static uint32_t *get_queue_ids(uint32_t num_queues, uint32_t *usr_queue_id_array) |
| 3301 | { | 3310 | { |
| 3302 | size_t array_size = num_queues * sizeof(uint32_t); | ||
| 3303 | |||
| 3304 | if (!usr_queue_id_array) | 3311 | if (!usr_queue_id_array) |
| 3305 | return NULL; | 3312 | return NULL; |
| 3306 | 3313 | ||
| 3307 | return memdup_user(usr_queue_id_array, array_size); | 3314 | if (num_queues > KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) |
| 3315 | return ERR_PTR(-EINVAL); | ||
| 3316 | |||
| 3317 | return memdup_user(usr_queue_id_array, | ||
| 3318 | array_size(num_queues, sizeof(uint32_t))); | ||
| 3308 | } | 3319 | } |
| 3309 | 3320 | ||
| 3310 | int resume_queues(struct kfd_process *p, | 3321 | int resume_queues(struct kfd_process *p, |
| @@ -364,11 +364,15 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd, · 363 unmodified lines | |||
| 364 | { | 364 | { |
| 365 | struct v9_mqd *m; | 365 | struct v9_mqd *m; |
| 366 | struct kfd_context_save_area_header header; | 366 | struct kfd_context_save_area_header header; |
| 367 | u32 cntl_stack_size; | ||
| 368 | u32 cntl_stack_offset; | ||
| 367 | 369 | ||
| 368 | /* Control stack is located one page after MQD. */ | 370 | /* Control stack is located one page after MQD. */ |
| 369 | void *mqd_ctl_stack = (void *)((uintptr_t)mqd + AMDGPU_GPU_PAGE_SIZE); | 371 | void *mqd_ctl_stack = (void *)((uintptr_t)mqd + AMDGPU_GPU_PAGE_SIZE); |
| 370 | 372 | ||
| 371 | m = get_mqd(mqd); | 373 | m = get_mqd(mqd); |
| 374 | cntl_stack_size = min_t(u32, m->cp_hqd_cntl_stack_size, q->ctl_stack_size); | ||
| 375 | cntl_stack_offset = min_t(u32, m->cp_hqd_cntl_stack_offset, cntl_stack_size); | ||
| 372 | 376 | ||
| 373 | *ctl_stack_used_size = m->cp_hqd_cntl_stack_size - | 377 | *ctl_stack_used_size = m->cp_hqd_cntl_stack_size - |
| 374 | m->cp_hqd_cntl_stack_offset; | 378 | m->cp_hqd_cntl_stack_offset; |
| @@ -384,9 +388,10 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd, · 9 unmodified lines | |||
| 384 | if (copy_to_user(ctl_stack, &header, sizeof(header.wave_state))) | 388 | if (copy_to_user(ctl_stack, &header, sizeof(header.wave_state))) |
| 385 | return -EFAULT; | 389 | return -EFAULT; |
| 386 | 390 | ||
| 387 | if (copy_to_user(ctl_stack + m->cp_hqd_cntl_stack_offset, | 391 | *ctl_stack_used_size = cntl_stack_size - cntl_stack_offset; |
| 388 | mqd_ctl_stack + m->cp_hqd_cntl_stack_offset, | 392 | |
| 389 | *ctl_stack_used_size)) | 393 | if (copy_to_user(ctl_stack + cntl_stack_offset, mqd_ctl_stack + cntl_stack_offset, |
| 394 | *ctl_stack_used_size)) | ||
| 390 | return -EFAULT; | 395 | return -EFAULT; |
| 391 | 396 | ||
| 392 | return 0; | 397 | return 0; |
| @@ -395,6 +395,7 @@ enum kfd_mempool { · 394 unmodified lines | |||
| 395 | /* Character device interface */ | 395 | /* Character device interface */ |
| 396 | int kfd_chardev_init(void); | 396 | int kfd_chardev_init(void); |
| 397 | void kfd_chardev_exit(void); | 397 | void kfd_chardev_exit(void); |
| 398 | void kfd_dev_unmap_mapping_range(loff_t const holebegin, loff_t const holelen); | ||
| 398 | 399 | ||
| 399 | /** | 400 | /** |
| 400 | * enum kfd_unmap_queues_filter - Enum for queue filters. | 401 | * enum kfd_unmap_queues_filter - Enum for queue filters. |
| @@ -1047,10 +1048,13 @@ extern struct srcu_struct kfd_processes_srcu; · 646 unmodified lines | |||
| 1047 | typedef int amdkfd_ioctl_t(struct file *filep, struct kfd_process *p, | 1048 | typedef int amdkfd_ioctl_t(struct file *filep, struct kfd_process *p, |
| 1048 | void *data); | 1049 | void *data); |
| 1049 | 1050 | ||
| 1051 | typedef int amdkfd_ioctl_validate_t(void *kdata, unsigned int usize); | ||
| 1052 | |||
| 1050 | struct amdkfd_ioctl_desc { | 1053 | struct amdkfd_ioctl_desc { |
| 1051 | unsigned int cmd; | 1054 | unsigned int cmd; |
| 1052 | int flags; | 1055 | int flags; |
| 1053 | amdkfd_ioctl_t *func; | 1056 | amdkfd_ioctl_t *func; |
| 1057 | amdkfd_ioctl_validate_t *validate; | ||
| 1054 | unsigned int cmd_drv; | 1058 | unsigned int cmd_drv; |
| 1055 | const char *name; | 1059 | const char *name; |
| 1056 | }; | 1060 | }; |
| @@ -1551,13 +1555,13 @@ void kfd_signal_reset_event(struct kfd_node *dev); · 494 unmodified lines | |||
| 1551 | void kfd_signal_poison_consumed_event(struct kfd_node *dev, u32 pasid); | 1555 | void kfd_signal_poison_consumed_event(struct kfd_node *dev, u32 pasid); |
| 1552 | void kfd_signal_process_terminate_event(struct kfd_process *p); | 1556 | void kfd_signal_process_terminate_event(struct kfd_process *p); |
| 1553 | 1557 | ||
| 1554 | static inline void kfd_flush_tlb(struct kfd_process_device *pdd, | 1558 | static inline void kfd_flush_tlb(struct kfd_process_device *pdd) |
| 1555 | enum TLB_FLUSH_TYPE type) | ||
| 1556 | { | 1559 | { |
| 1557 | struct amdgpu_device *adev = pdd->dev->adev; | 1560 | struct amdgpu_device *adev = pdd->dev->adev; |
| 1558 | struct amdgpu_vm *vm = drm_priv_to_vm(pdd->drm_priv); | 1561 | struct amdgpu_vm *vm = drm_priv_to_vm(pdd->drm_priv); |
| 1559 | 1562 | ||
| 1560 | amdgpu_vm_flush_compute_tlb(adev, vm, type, pdd->dev->xcc_mask); | 1563 | amdgpu_vm_flush_compute_tlb(adev, vm, TLB_FLUSH_HEAVYWEIGHT, |
| 1564 | pdd->dev->xcc_mask); | ||
| 1561 | } | 1565 | } |
| 1562 | 1566 | ||
| 1563 | static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev) | 1567 | static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev) |
| @@ -1366,6 +1366,12 @@ svm_range_unmap_from_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 1365 unmodified lines | |||
| 1366 | 1366 | ||
| 1367 | pr_debug("CPU[0x%llx 0x%llx] -> GPU[0x%llx 0x%llx]\n", start, last, | 1367 | pr_debug("CPU[0x%llx 0x%llx] -> GPU[0x%llx 0x%llx]\n", start, last, |
| 1368 | gpu_start, gpu_end); | 1368 | gpu_start, gpu_end); |
| 1369 | |||
| 1370 | if (!amdgpu_vm_ready(vm)) { | ||
| 1371 | pr_debug("VM not ready, canceling unmap\n"); | ||
| 1372 | return -EINVAL; | ||
| 1373 | } | ||
| 1374 | |||
| 1369 | return amdgpu_vm_update_range(adev, vm, false, true, true, false, NULL, gpu_start, | 1375 | return amdgpu_vm_update_range(adev, vm, false, true, true, false, NULL, gpu_start, |
| 1370 | gpu_end, init_pte_value, 0, 0, NULL, NULL, | 1376 | gpu_end, init_pte_value, 0, 0, NULL, NULL, |
| 1371 | fence); | 1377 | fence); |
| @@ -1418,7 +1424,7 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start, · 46 unmodified lines | |||
| 1418 | if (r) | 1424 | if (r) |
| 1419 | break; | 1425 | break; |
| 1420 | } | 1426 | } |
| 1421 | kfd_flush_tlb(pdd, TLB_FLUSH_HEAVYWEIGHT); | 1427 | kfd_flush_tlb(pdd); |
| 1422 | } | 1428 | } |
| 1423 | 1429 | ||
| 1424 | return r; | 1430 | return r; |
| @@ -1443,6 +1449,11 @@ svm_range_map_to_gpu(struct kfd_process_device *pdd, struct svm_range *prange, · 18 unmodified lines | |||
| 1443 | pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms, | 1449 | pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms, |
| 1444 | last_start, last_start + npages - 1, readonly); | 1450 | last_start, last_start + npages - 1, readonly); |
| 1445 | 1451 | ||
| 1452 | if (!amdgpu_vm_ready(vm)) { | ||
| 1453 | pr_debug("VM not ready, canceling map\n"); | ||
| 1454 | return -EINVAL; | ||
| 1455 | } | ||
| 1456 | |||
| 1446 | for (i = offset; i < offset + npages; i++) { | 1457 | for (i = offset; i < offset + npages; i++) { |
| 1447 | uint64_t gpu_start; | 1458 | uint64_t gpu_start; |
| 1448 | uint64_t gpu_end; | 1459 | uint64_t gpu_end; |
| @@ -1560,7 +1571,7 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset, · 111 unmodified lines | |||
| 1560 | } | 1571 | } |
| 1561 | } | 1572 | } |
| 1562 | 1573 | ||
| 1563 | kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); | 1574 | kfd_flush_tlb(pdd); |
| 1564 | } | 1575 | } |
| 1565 | 1576 | ||
| 1566 | return r; | 1577 | return r; |
| @@ -3721,6 +3732,9 @@ svm_range_set_attr(struct kfd_process *p, struct mm_struct *mm, · 2154 unmodified lines | |||
| 3721 | 3732 | ||
| 3722 | svms = &p->svms; | 3733 | svms = &p->svms; |
| 3723 | 3734 | ||
| 3735 | if (!process_info) | ||
| 3736 | return -EINVAL; | ||
| 3737 | |||
| 3724 | mutex_lock(&process_info->lock); | 3738 | mutex_lock(&process_info->lock); |
| 3725 | 3739 | ||
| 3726 | svm_range_list_lock_and_flush_work(svms, mm); | 3740 | svm_range_list_lock_and_flush_work(svms, mm); |
| @@ -1903,7 +1903,11 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) · 1902 unmodified lines | |||
| 1903 | goto error; | 1903 | goto error; |
| 1904 | } | 1904 | } |
| 1905 | 1905 | ||
| 1906 | init_data.asic_id.chip_family = adev->family; | 1906 | /* special handling for early revisions of GC 11.5.4 */ |
| 1907 | if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(11, 5, 4)) | ||
| 1908 | init_data.asic_id.chip_family = AMDGPU_FAMILY_GC_11_5_4; | ||
| 1909 | else | ||
| 1910 | init_data.asic_id.chip_family = adev->family; | ||
| 1907 | 1911 | ||
| 1908 | init_data.asic_id.pci_revision_id = adev->pdev->revision; | 1912 | init_data.asic_id.pci_revision_id = adev->pdev->revision; |
| 1909 | init_data.asic_id.hw_internal_rev = adev->external_rev_id; | 1913 | init_data.asic_id.hw_internal_rev = adev->external_rev_id; |
| @@ -9404,9 +9408,21 @@ static void manage_dm_interrupts(struct amdgpu_device *adev, · 7494 unmodified lines | |||
| 9404 | if (acrtc_state) { | 9408 | if (acrtc_state) { |
| 9405 | timing = &acrtc_state->stream->timing; | 9409 | timing = &acrtc_state->stream->timing; |
| 9406 | 9410 | ||
| 9407 | if (amdgpu_ip_version(adev, DCE_HWIP, 0) < | 9411 | if (amdgpu_ip_version(adev, DCE_HWIP, 0) >= |
| 9408 | IP_VERSION(3, 5, 0) || | 9412 | IP_VERSION(3, 2, 0) && |
| 9409 | !(adev->flags & AMD_IS_APU)) { | 9413 | !(adev->flags & AMD_IS_APU)) { |
| 9414 | /* | ||
| 9415 | * DGPUs NV3x and newer that support idle optimizations | ||
| 9416 | * experience intermittent flip-done timeouts on cursor | ||
| 9417 | * updates. Restore 5s offdelay behavior for now. | ||
| 9418 | * | ||
| 9419 | * Discussion on the issue: | ||
| 9420 | * https://lore.kernel.org/amd-gfx/20260217191632.1243826-1-sysdadmin@m1k.cloud/ | ||
| 9421 | */ | ||
| 9422 | config.offdelay_ms = 5000; | ||
| 9423 | config.disable_immediate = false; | ||
| 9424 | } else if (amdgpu_ip_version(adev, DCE_HWIP, 0) < | ||
| 9425 | IP_VERSION(3, 5, 0)) { | ||
| 9410 | /* | 9426 | /* |
| 9411 | * Older HW and DGPU have issues with instant off; | 9427 | * Older HW and DGPU have issues with instant off; |
| 9412 | * use a 2 frame offdelay. | 9428 | * use a 2 frame offdelay. |
| @@ -1032,6 +1032,45 @@ dm_helpers_read_acpi_edid(struct amdgpu_dm_connector *aconnector) · 1031 unmodified lines | |||
| 1032 | return drm_edid_read_custom(connector, dm_helpers_probe_acpi_edid, connector); | 1032 | return drm_edid_read_custom(connector, dm_helpers_probe_acpi_edid, connector); |
| 1033 | } | 1033 | } |
| 1034 | 1034 | ||
| 1035 | static const struct drm_edid * | ||
| 1036 | dm_helpers_read_vbios_hardcoded_edid(struct dc_link *link, struct amdgpu_dm_connector *aconnector) | ||
| 1037 | { | ||
| 1038 | struct dc_bios *bios = link->ctx->dc_bios; | ||
| 1039 | struct embedded_panel_info info; | ||
| 1040 | const struct drm_edid *edid; | ||
| 1041 | enum bp_result r; | ||
| 1042 | |||
| 1043 | if (!dc_is_embedded_signal(link->connector_signal) || | ||
| 1044 | !bios->funcs->get_embedded_panel_info) | ||
| 1045 | return NULL; | ||
| 1046 | |||
| 1047 | memset(&info, 0, sizeof(info)); | ||
| 1048 | r = bios->funcs->get_embedded_panel_info(bios, &info); | ||
| 1049 | |||
| 1050 | if (r != BP_RESULT_OK) { | ||
| 1051 | dm_error("Error when reading embedded panel info: %u\n", r); | ||
| 1052 | return NULL; | ||
| 1053 | } | ||
| 1054 | |||
| 1055 | if (!info.fake_edid || !info.fake_edid_size) { | ||
| 1056 | dm_error("Embedded panel info doesn't contain an EDID\n"); | ||
| 1057 | return NULL; | ||
| 1058 | } | ||
| 1059 | |||
| 1060 | edid = drm_edid_alloc(info.fake_edid, info.fake_edid_size); | ||
| 1061 | |||
| 1062 | if (!drm_edid_valid(edid)) { | ||
| 1063 | dm_error("EDID from embedded panel info is invalid\n"); | ||
| 1064 | drm_edid_free(edid); | ||
| 1065 | return NULL; | ||
| 1066 | } | ||
| 1067 | |||
| 1068 | aconnector->base.display_info.width_mm = info.panel_width_mm; | ||
| 1069 | aconnector->base.display_info.height_mm = info.panel_height_mm; | ||
| 1070 | |||
| 1071 | return edid; | ||
| 1072 | } | ||
| 1073 | |||
| 1035 | void populate_hdmi_info_from_connector(struct drm_hdmi_info *hdmi, struct dc_edid_caps *edid_caps) | 1074 | void populate_hdmi_info_from_connector(struct drm_hdmi_info *hdmi, struct dc_edid_caps *edid_caps) |
| 1036 | { | 1075 | { |
| 1037 | edid_caps->scdc_present = hdmi->scdc.supported; | 1076 | edid_caps->scdc_present = hdmi->scdc.supported; |
| @@ -1052,6 +1091,9 @@ enum dc_edid_status dm_helpers_read_local_edid( · 14 unmodified lines | |||
| 1052 | 1091 | ||
| 1053 | if (link->aux_mode) | 1092 | if (link->aux_mode) |
| 1054 | ddc = &aconnector->dm_dp_aux.aux.ddc; | 1093 | ddc = &aconnector->dm_dp_aux.aux.ddc; |
| 1094 | else if (link->ddc_hw_inst == GPIO_DDC_LINE_UNKNOWN && | ||
| 1095 | dc_is_embedded_signal(link->connector_signal)) | ||
| 1096 | ddc = NULL; | ||
| 1055 | else | 1097 | else |
| 1056 | ddc = &aconnector->i2c->base; | 1098 | ddc = &aconnector->i2c->base; |
| 1057 | 1099 | ||
| @@ -1065,6 +1107,8 @@ enum dc_edid_status dm_helpers_read_local_edid( · 7 unmodified lines | |||
| 1065 | drm_edid = dm_helpers_read_acpi_edid(aconnector); | 1107 | drm_edid = dm_helpers_read_acpi_edid(aconnector); |
| 1066 | if (drm_edid) | 1108 | if (drm_edid) |
| 1067 | drm_info(connector->dev, "Using ACPI provided EDID for %s\n", connector->name); | 1109 | drm_info(connector->dev, "Using ACPI provided EDID for %s\n", connector->name); |
| 1110 | else if (!ddc) | ||
| 1111 | drm_edid = dm_helpers_read_vbios_hardcoded_edid(link, aconnector); | ||
| 1068 | else | 1112 | else |
| 1069 | drm_edid = drm_edid_read_ddc(connector, ddc); | 1113 | drm_edid = drm_edid_read_ddc(connector, ddc); |
| 1070 | drm_edid_connector_update(connector, drm_edid); | 1114 | drm_edid_connector_update(connector, drm_edid); |
| @@ -794,11 +794,13 @@ static enum bp_result bios_parser_external_encoder_control( · 793 unmodified lines | |||
| 794 | 794 | ||
| 795 | static enum bp_result bios_parser_dac_load_detection( | 795 | static enum bp_result bios_parser_dac_load_detection( |
| 796 | struct dc_bios *dcb, | 796 | struct dc_bios *dcb, |
| 797 | enum engine_id engine_id) | 797 | enum engine_id engine_id, |
| 798 | struct graphics_object_id ext_enc_id) | ||
| 798 | { | 799 | { |
| 799 | struct bios_parser *bp = BP_FROM_DCB(dcb); | 800 | struct bios_parser *bp = BP_FROM_DCB(dcb); |
| 800 | struct dc_context *ctx = dcb->ctx; | 801 | struct dc_context *ctx = dcb->ctx; |
| 801 | struct bp_load_detection_parameters bp_params = {0}; | 802 | struct bp_load_detection_parameters bp_params = {0}; |
| 803 | struct bp_external_encoder_control ext_cntl = {0}; | ||
| 802 | enum bp_result bp_result = BP_RESULT_UNSUPPORTED; | 804 | enum bp_result bp_result = BP_RESULT_UNSUPPORTED; |
| 803 | uint32_t bios_0_scratch; | 805 | uint32_t bios_0_scratch; |
| 804 | uint32_t device_id_mask = 0; | 806 | uint32_t device_id_mask = 0; |
| @@ -824,6 +826,13 @@ static enum bp_result bios_parser_dac_load_detection( · 19 unmodified lines | |||
| 824 | 826 | ||
| 825 | bp_params.engine_id = engine_id; | 827 | bp_params.engine_id = engine_id; |
| 826 | bp_result = bp->cmd_tbl.dac_load_detection(bp, &bp_params); | 828 | bp_result = bp->cmd_tbl.dac_load_detection(bp, &bp_params); |
| 829 | } else if (ext_enc_id.id) { | ||
| 830 | if (!bp->cmd_tbl.external_encoder_control) | ||
| 831 | return BP_RESULT_UNSUPPORTED; | ||
| 832 | |||
| 833 | ext_cntl.action = EXTERNAL_ENCODER_CONTROL_DAC_LOAD_DETECT; | ||
| 834 | ext_cntl.encoder_id = ext_enc_id; | ||
| 835 | bp_result = bp->cmd_tbl.external_encoder_control(bp, &ext_cntl); | ||
| 827 | } | 836 | } |
| 828 | 837 | ||
| 829 | if (bp_result != BP_RESULT_OK) | 838 | if (bp_result != BP_RESULT_OK) |
| @@ -1304,6 +1313,60 @@ static enum bp_result bios_parser_get_embedded_panel_info( · 474 unmodified lines | |||
| 1304 | return BP_RESULT_FAILURE; | 1313 | return BP_RESULT_FAILURE; |
| 1305 | } | 1314 | } |
| 1306 | 1315 | ||
| 1316 | static enum bp_result get_embedded_panel_extra_info( | ||
| 1317 | struct bios_parser *bp, | ||
| 1318 | struct embedded_panel_info *info, | ||
| 1319 | const uint32_t table_offset) | ||
| 1320 | { | ||
| 1321 | uint8_t *record = bios_get_image(&bp->base, table_offset, 1); | ||
| 1322 | ATOM_PANEL_RESOLUTION_PATCH_RECORD *panel_res_record; | ||
| 1323 | ATOM_FAKE_EDID_PATCH_RECORD *fake_edid_record; | ||
| 1324 | |||
| 1325 | while (*record != ATOM_RECORD_END_TYPE) { | ||
| 1326 | switch (*record) { | ||
| 1327 | case LCD_MODE_PATCH_RECORD_MODE_TYPE: | ||
| 1328 | record += sizeof(ATOM_PATCH_RECORD_MODE); | ||
| 1329 | break; | ||
| 1330 | case LCD_RTS_RECORD_TYPE: | ||
| 1331 | record += sizeof(ATOM_LCD_RTS_RECORD); | ||
| 1332 | break; | ||
| 1333 | case LCD_CAP_RECORD_TYPE: | ||
| 1334 | record += sizeof(ATOM_LCD_MODE_CONTROL_CAP); | ||
| 1335 | break; | ||
| 1336 | case LCD_FAKE_EDID_PATCH_RECORD_TYPE: | ||
| 1337 | fake_edid_record = (ATOM_FAKE_EDID_PATCH_RECORD *)record; | ||
| 1338 | if (fake_edid_record->ucFakeEDIDLength) { | ||
| 1339 | if (fake_edid_record->ucFakeEDIDLength == 128) | ||
| 1340 | info->fake_edid_size = | ||
| 1341 | fake_edid_record->ucFakeEDIDLength; | ||
| 1342 | else | ||
| 1343 | info->fake_edid_size = | ||
| 1344 | fake_edid_record->ucFakeEDIDLength * 128; | ||
| 1345 | |||
| 1346 | info->fake_edid = fake_edid_record->ucFakeEDIDString; | ||
| 1347 | |||
| 1348 | record += struct_size(fake_edid_record, | ||
| 1349 | ucFakeEDIDString, | ||
| 1350 | info->fake_edid_size); | ||
| 1351 | } else { | ||
| 1352 | /* empty fake edid record must be 3 bytes long */ | ||
| 1353 | record += sizeof(ATOM_FAKE_EDID_PATCH_RECORD) + 1; | ||
| 1354 | } | ||
| 1355 | break; | ||
| 1356 | case LCD_PANEL_RESOLUTION_RECORD_TYPE: | ||
| 1357 | panel_res_record = (ATOM_PANEL_RESOLUTION_PATCH_RECORD *)record; | ||
| 1358 | info->panel_width_mm = panel_res_record->usHSize; | ||
| 1359 | info->panel_height_mm = panel_res_record->usVSize; | ||
| 1360 | record += sizeof(ATOM_PANEL_RESOLUTION_PATCH_RECORD); | ||
| 1361 | break; | ||
| 1362 | default: | ||
| 1363 | return BP_RESULT_BADBIOSTABLE; | ||
| 1364 | } | ||
| 1365 | } | ||
| 1366 | |||
| 1367 | return BP_RESULT_OK; | ||
| 1368 | } | ||
| 1369 | |||
| 1307 | static enum bp_result get_embedded_panel_info_v1_2( | 1370 | static enum bp_result get_embedded_panel_info_v1_2( |
| 1308 | struct bios_parser *bp, | 1371 | struct bios_parser *bp, |
| 1309 | struct embedded_panel_info *info) | 1372 | struct embedded_panel_info *info) |
| @@ -1420,6 +1483,10 @@ static enum bp_result get_embedded_panel_info_v1_2( · 110 unmodified lines | |||
| 1420 | if (ATOM_PANEL_MISC_API_ENABLED & lvds->ucLVDS_Misc) | 1483 | if (ATOM_PANEL_MISC_API_ENABLED & lvds->ucLVDS_Misc) |
| 1421 | info->lcd_timing.misc_info.API_ENABLED = true; | 1484 | info->lcd_timing.misc_info.API_ENABLED = true; |
| 1422 | 1485 | ||
| 1486 | if (lvds->usExtInfoTableOffset) | ||
| 1487 | return get_embedded_panel_extra_info(bp, info, | ||
| 1488 | le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info)); | ||
| 1489 | |||
| 1423 | return BP_RESULT_OK; | 1490 | return BP_RESULT_OK; |
| 1424 | } | 1491 | } |
| 1425 | 1492 | ||
| @@ -1545,6 +1612,10 @@ static enum bp_result get_embedded_panel_info_v1_3( · 119 unmodified lines | |||
| 1545 | (uint32_t) (ATOM_PANEL_MISC_V13_GREY_LEVEL & | 1612 | (uint32_t) (ATOM_PANEL_MISC_V13_GREY_LEVEL & |
| 1546 | lvds->ucLCD_Misc) >> ATOM_PANEL_MISC_V13_GREY_LEVEL_SHIFT; | 1613 | lvds->ucLCD_Misc) >> ATOM_PANEL_MISC_V13_GREY_LEVEL_SHIFT; |
| 1547 | 1614 | ||
| 1615 | if (lvds->usExtInfoTableOffset) | ||
| 1616 | return get_embedded_panel_extra_info(bp, info, | ||
| 1617 | le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info)); | ||
| 1618 | |||
| 1548 | return BP_RESULT_OK; | 1619 | return BP_RESULT_OK; |
| 1549 | } | 1620 | } |
| 1550 | 1621 | ||
| @@ -493,6 +493,10 @@ static enum bp_result get_gpio_i2c_info( · 492 unmodified lines | |||
| 493 | - sizeof(struct atom_common_table_header)) | 493 | - sizeof(struct atom_common_table_header)) |
| 494 | / sizeof(struct atom_gpio_pin_assignment); | 494 | / sizeof(struct atom_gpio_pin_assignment); |
| 495 | 495 | ||
| 496 | if (!bios_get_image(&bp->base, DATA_TABLES(gpio_pin_lut), | ||
| 497 | le16_to_cpu(header->table_header.structuresize))) | ||
| 498 | return BP_RESULT_BADBIOSTABLE; | ||
| 499 | |||
| 496 | pin = (struct atom_gpio_pin_assignment *) header->gpio_pin; | 500 | pin = (struct atom_gpio_pin_assignment *) header->gpio_pin; |
| 497 | 501 | ||
| 498 | for (table_index = 0; table_index < count; table_index++) { | 502 | for (table_index = 0; table_index < count; table_index++) { |
| @@ -681,6 +685,11 @@ static enum bp_result bios_parser_get_gpio_pin_info( · 182 unmodified lines | |||
| 681 | count = (le16_to_cpu(header->table_header.structuresize) | 685 | count = (le16_to_cpu(header->table_header.structuresize) |
| 682 | - sizeof(struct atom_common_table_header)) | 686 | - sizeof(struct atom_common_table_header)) |
| 683 | / sizeof(struct atom_gpio_pin_assignment); | 687 | / sizeof(struct atom_gpio_pin_assignment); |
| 688 | |||
| 689 | if (!bios_get_image(&bp->base, DATA_TABLES(gpio_pin_lut), | ||
| 690 | le16_to_cpu(header->table_header.structuresize))) | ||
| 691 | return BP_RESULT_BADBIOSTABLE; | ||
| 692 | |||
| 684 | for (i = 0; i < count; ++i) { | 693 | for (i = 0; i < count; ++i) { |
| 685 | if (header->gpio_pin[i].gpio_id != gpio_id) | 694 | if (header->gpio_pin[i].gpio_id != gpio_id) |
| 686 | continue; | 695 | continue; |
| @@ -37,10 +37,13 @@ uint8_t *bios_get_image(struct dc_bios *bp, · 36 unmodified lines | |||
| 37 | uint32_t offset, | 37 | uint32_t offset, |
| 38 | uint32_t size) | 38 | uint32_t size) |
| 39 | { | 39 | { |
| 40 | if (bp->bios && offset + size < bp->bios_size) | 40 | if (!bp->bios) |
| 41 | return bp->bios + offset; | ||
| 42 | else | ||
| 43 | return NULL; | 41 | return NULL; |
| 42 | |||
| 43 | if (offset > bp->bios_size || size > bp->bios_size - offset) | ||
| 44 | return NULL; | ||
| 45 | |||
| 46 | return bp->bios + offset; | ||
| 44 | } | 47 | } |
| 45 | 48 | ||
| 46 | #include "reg_helper.h" | 49 | #include "reg_helper.h" |
| @@ -6071,7 +6071,11 @@ bool dc_process_dmub_aux_transfer_async(struct dc *dc, · 6070 unmodified lines | |||
| 6071 | uint8_t action; | 6071 | uint8_t action; |
| 6072 | union dmub_rb_cmd cmd = {0}; | 6072 | union dmub_rb_cmd cmd = {0}; |
| 6073 | 6073 | ||
| 6074 | ASSERT(payload->length <= 16); | 6074 | if (link_index >= dc->link_count || !dc->links[link_index]) |
| 6075 | return false; | ||
| 6076 | |||
| 6077 | if (payload->length > sizeof(cmd.dp_aux_access.aux_control.dpaux.data)) | ||
| 6078 | return false; | ||
| 6075 | 6079 | ||
| 6076 | cmd.dp_aux_access.header.type = DMUB_CMD__DP_AUX_ACCESS; | 6080 | cmd.dp_aux_access.header.type = DMUB_CMD__DP_AUX_ACCESS; |
| 6077 | cmd.dp_aux_access.header.payload_bytes = 0; | 6081 | cmd.dp_aux_access.header.payload_bytes = 0; |
| @@ -1682,7 +1682,7 @@ struct dc_scratch_space { · 1681 unmodified lines | |||
| 1682 | struct dc_link_training_overrides preferred_training_settings; | 1682 | struct dc_link_training_overrides preferred_training_settings; |
| 1683 | struct dp_audio_test_data audio_test_data; | 1683 | struct dp_audio_test_data audio_test_data; |
| 1684 | 1684 | ||
| 1685 | uint8_t ddc_hw_inst; | 1685 | enum gpio_ddc_line ddc_hw_inst; |
| 1686 | 1686 | ||
| 1687 | uint8_t hpd_src; | 1687 | uint8_t hpd_src; |
| 1688 | 1688 | ||
| @@ -102,7 +102,8 @@ struct dc_vbios_funcs { · 101 unmodified lines | |||
| 102 | struct bp_external_encoder_control *cntl); | 102 | struct bp_external_encoder_control *cntl); |
| 103 | enum bp_result (*dac_load_detection)( | 103 | enum bp_result (*dac_load_detection)( |
| 104 | struct dc_bios *bios, | 104 | struct dc_bios *bios, |
| 105 | enum engine_id engine_id); | 105 | enum engine_id engine_id, |
| 106 | struct graphics_object_id ext_enc_id); | ||
| 106 | enum bp_result (*transmitter_control)( | 107 | enum bp_result (*transmitter_control)( |
| 107 | struct dc_bios *bios, | 108 | struct dc_bios *bios, |
| 108 | struct bp_transmitter_control *cntl); | 109 | struct bp_transmitter_control *cntl); |
| @@ -105,15 +105,26 @@ static void dccg21_update_dpp_dto(struct dccg *dccg, int dpp_inst, int req_dppcl · 104 unmodified lines | |||
| 105 | * dccg2_init() unconditionally overwrites MICROSECOND_TIME_BASE_DIV to | 105 | * dccg2_init() unconditionally overwrites MICROSECOND_TIME_BASE_DIV to |
| 106 | * 0x00120264, destroying the marker before it can be read. | 106 | * 0x00120264, destroying the marker before it can be read. |
| 107 | * | 107 | * |
| 108 | * Guard the call: if the S0i3 marker is present, skip dccg2_init() so the | 108 | * Guard the call: if the S0i3 marker is present, skip init so the |
| 109 | * WA can function correctly. bios_golden_init() will handle init in that case. | 109 | * WA can function correctly. bios_golden_init() will handle init in that case. |
| 110 | * | ||
| 111 | * DCN21 uses 48MHz refclk, not 100MHz, so we must explicitly set the correct | ||
| 112 | * values (48MHz is taken from rn_clk_mgr_construct()). | ||
| 110 | */ | 113 | */ |
| 111 | static void dccg21_init(struct dccg *dccg) | 114 | static void dccg21_init(struct dccg *dccg) |
| 112 | { | 115 | { |
| 116 | struct dcn_dccg *dccg_dcn = TO_DCN_DCCG(dccg); | ||
| 117 | |||
| 113 | if (dccg2_is_s0i3_golden_init_wa_done(dccg)) | 118 | if (dccg2_is_s0i3_golden_init_wa_done(dccg)) |
| 114 | return; | 119 | return; |
| 115 | 120 | ||
| 116 | dccg2_init(dccg); | 121 | /* 48MHz refclk from rn_clk_mgr_construct() */ |
| 122 | REG_WRITE(MICROSECOND_TIME_BASE_DIV, 0x00120230); | ||
| 123 | REG_WRITE(MILLISECOND_TIME_BASE_DIV, 0x0010bb80); | ||
| 124 | REG_WRITE(DISPCLK_FREQ_CHANGE_CNTL, 0x0e01003c); | ||
| 125 | |||
| 126 | if (REG(REFCLK_CNTL)) | ||
| 127 | REG_WRITE(REFCLK_CNTL, 0); | ||
| 117 | } | 128 | } |
| 118 | 129 | ||
| 119 | static const struct dccg_funcs dccg21_funcs = { | 130 | static const struct dccg_funcs dccg21_funcs = { |
| @@ -1102,7 +1102,9 @@ void dce110_link_encoder_hw_init( · 1101 unmodified lines | |||
| 1102 | ASSERT(result == BP_RESULT_OK); | 1102 | ASSERT(result == BP_RESULT_OK); |
| 1103 | 1103 | ||
| 1104 | } | 1104 | } |
| 1105 | aux_initialize(enc110); | 1105 | |
| 1106 | if (enc110->aux_regs) | ||
| 1107 | aux_initialize(enc110); | ||
| 1106 | 1108 | ||
| 1107 | /* reinitialize HPD. | 1109 | /* reinitialize HPD. |
| 1108 | * hpd_initialize() will pass DIG_FE id to HW context. | 1110 | * hpd_initialize() will pass DIG_FE id to HW context. |
| @@ -40,8 +40,8 @@ · 39 unmodified lines | |||
| 40 | #define FN(reg_name, field_name) \ | 40 | #define FN(reg_name, field_name) \ |
| 41 | mcif_wb30->mcif_wb_shift->field_name, mcif_wb30->mcif_wb_mask->field_name | 41 | mcif_wb30->mcif_wb_shift->field_name, mcif_wb30->mcif_wb_mask->field_name |
| 42 | 42 | ||
| 43 | #define MCIF_ADDR(addr) (((unsigned long long)addr & 0xffffffffff) + 0xFE) >> 8 | 43 | #define MCIF_ADDR(addr) ((uint32_t)((((unsigned long long)(addr) & 0xffffffffffULL) + 0xFEULL) >> 8)) |
| 44 | #define MCIF_ADDR_HIGH(addr) (unsigned long long)addr >> 40 | 44 | #define MCIF_ADDR_HIGH(addr) ((uint32_t)(((unsigned long long)(addr)) >> 40)) |
| 45 | 45 | ||
| 46 | /* wbif programming guide: | 46 | /* wbif programming guide: |
| 47 | * 1. set up wbif parameter: | 47 | * 1. set up wbif parameter: |
| @@ -646,6 +646,9 @@ failure: · 645 unmodified lines | |||
| 646 | enum gpio_ddc_line dal_ddc_get_line( | 646 | enum gpio_ddc_line dal_ddc_get_line( |
| 647 | const struct ddc *ddc) | 647 | const struct ddc *ddc) |
| 648 | { | 648 | { |
| 649 | if (!ddc) | ||
| 650 | return GPIO_DDC_LINE_UNKNOWN; | ||
| 651 | |||
| 649 | return (enum gpio_ddc_line)dal_gpio_get_enum(ddc->pin_data); | 652 | return (enum gpio_ddc_line)dal_gpio_get_enum(ddc->pin_data); |
| 650 | } | 653 | } |
| 651 | 654 | ||
| @@ -665,16 +665,45 @@ void dce110_update_info_frame(struct pipe_ctx *pipe_ctx) · 664 unmodified lines | |||
| 665 | } | 665 | } |
| 666 | 666 | ||
| 667 | static void | 667 | static void |
| 668 | dce110_dac_encoder_control(struct pipe_ctx *pipe_ctx, bool enable) | 668 | dce110_external_encoder_control(enum bp_external_encoder_control_action action, |
| 669 | struct dc_link *link, | ||
| 670 | struct dc_crtc_timing *timing) | ||
| 669 | { | 671 | { |
| 670 | struct dc_link *link = pipe_ctx->stream->link; | 672 | struct dc *dc = link->ctx->dc; |
| 671 | struct dc_bios *bios = link->ctx->dc_bios; | 673 | struct dc_bios *bios = link->ctx->dc_bios; |
| 672 | struct bp_encoder_control encoder_control = {0}; | 674 | const struct dc_link_settings *link_settings = &link->cur_link_settings; |
| 675 | enum bp_result bp_result = BP_RESULT_OK; | ||
| 676 | struct bp_external_encoder_control ext_cntl = { | ||
| 677 | .action = action, | ||
| 678 | .connector_obj_id = link->link_enc->connector, | ||
| 679 | .encoder_id = link->ext_enc_id, | ||
| 680 | .lanes_number = link_settings->lane_count, | ||
| 681 | .link_rate = link_settings->link_rate, | ||
| 682 | |||
| 683 | /* Use signal type of the real link encoder, ie. DP */ | ||
| 684 | .signal = link->connector_signal, | ||
| 685 | |||
| 686 | /* We don't know the timing yet when executing the SETUP action, | ||
| 687 | * so use a reasonably high default value. It seems that ENABLE | ||
| 688 | * can change the actual pixel clock but doesn't work with higher | ||
| 689 | * pixel clocks than what SETUP was called with. | ||
| 690 | */ | ||
| 691 | .pixel_clock = timing ? timing->pix_clk_100hz / 10 : 300000, | ||
| 692 | .color_depth = timing ? timing->display_color_depth : COLOR_DEPTH_888, | ||
| 693 | }; | ||
| 694 | DC_LOGGER_INIT(dc->ctx); | ||
| 673 | 695 | ||
| 674 | encoder_control.action = enable ? ENCODER_CONTROL_ENABLE : ENCODER_CONTROL_DISABLE; | 696 | bp_result = bios->funcs->external_encoder_control(bios, &ext_cntl); |
| 675 | encoder_control.engine_id = link->link_enc->analog_engine; | 697 | |
| 676 | encoder_control.pixel_clock = pipe_ctx->stream->timing.pix_clk_100hz / 10; | 698 | if (bp_result != BP_RESULT_OK) |
| 677 | bios->funcs->encoder_control(bios, &encoder_control); | 699 | DC_LOG_ERROR("Failed to execute external encoder action: 0x%x\n", action); |
| 700 | } | ||
| 701 | |||
| 702 | static void | ||
| 703 | dce110_prepare_ddc(struct dc_link *link) | ||
| 704 | { | ||
| 705 | if (link->ext_enc_id.id) | ||
| 706 | dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_DDC_SETUP, link, NULL); | ||
| 678 | } | 707 | } |
| 679 | 708 | ||
| 680 | static bool | 709 | static bool |
| @@ -684,7 +713,8 @@ dce110_dac_load_detect(struct dc_link *link) · 3 unmodified lines | |||
| 684 | struct link_encoder *link_enc = link->link_enc; | 713 | struct link_encoder *link_enc = link->link_enc; |
| 685 | enum bp_result bp_result; | 714 | enum bp_result bp_result; |
| 686 | 715 | ||
| 687 | bp_result = bios->funcs->dac_load_detection(bios, link_enc->analog_engine); | 716 | bp_result = bios->funcs->dac_load_detection( |
| 717 | bios, link_enc->analog_engine, link->ext_enc_id); | ||
| 688 | return bp_result == BP_RESULT_OK; | 718 | return bp_result == BP_RESULT_OK; |
| 689 | } | 719 | } |
| 690 | 720 | ||
| @@ -700,7 +730,6 @@ void dce110_enable_stream(struct pipe_ctx *pipe_ctx) · 9 unmodified lines | |||
| 700 | uint32_t early_control = 0; | 730 | uint32_t early_control = 0; |
| 701 | struct timing_generator *tg = pipe_ctx->stream_res.tg; | 731 | struct timing_generator *tg = pipe_ctx->stream_res.tg; |
| 702 | 732 | ||
| 703 | link_hwss->setup_stream_attribute(pipe_ctx); | ||
| 704 | link_hwss->setup_stream_encoder(pipe_ctx); | 733 | link_hwss->setup_stream_encoder(pipe_ctx); |
| 705 | 734 | ||
| 706 | dc->hwss.update_info_frame(pipe_ctx); | 735 | dc->hwss.update_info_frame(pipe_ctx); |
| @@ -719,8 +748,8 @@ void dce110_enable_stream(struct pipe_ctx *pipe_ctx) · 12 unmodified lines | |||
| 719 | 748 | ||
| 720 | tg->funcs->set_early_control(tg, early_control); | 749 | tg->funcs->set_early_control(tg, early_control); |
| 721 | 750 | ||
| 722 | if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | 751 | if (link->ext_enc_id.id) |
| 723 | dce110_dac_encoder_control(pipe_ctx, true); | 752 | dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_ENABLE, link, timing); |
| 724 | } | 753 | } |
| 725 | 754 | ||
| 726 | static enum bp_result link_transmitter_control( | 755 | static enum bp_result link_transmitter_control( |
| @@ -1219,8 +1248,8 @@ void dce110_disable_stream(struct pipe_ctx *pipe_ctx) · 492 unmodified lines | |||
| 1219 | link_enc->transmitter - TRANSMITTER_UNIPHY_A); | 1248 | link_enc->transmitter - TRANSMITTER_UNIPHY_A); |
| 1220 | } | 1249 | } |
| 1221 | 1250 | ||
| 1222 | if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | 1251 | if (link->ext_enc_id.id) |
| 1223 | dce110_dac_encoder_control(pipe_ctx, false); | 1252 | dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_DISABLE, link, NULL); |
| 1224 | } | 1253 | } |
| 1225 | 1254 | ||
| 1226 | void dce110_unblank_stream(struct pipe_ctx *pipe_ctx, | 1255 | void dce110_unblank_stream(struct pipe_ctx *pipe_ctx, |
| @@ -1603,22 +1632,6 @@ static enum dc_status dce110_enable_stream_timing( · 376 unmodified lines | |||
| 1603 | 1632 | ||
| 1604 | return DC_OK; | 1633 | return DC_OK; |
| 1605 | } | 1634 | } |
| 1606 | static void | ||
| 1607 | dce110_select_crtc_source(struct pipe_ctx *pipe_ctx) | ||
| 1608 | { | ||
| 1609 | struct dc_link *link = pipe_ctx->stream->link; | ||
| 1610 | struct dc_bios *bios = link->ctx->dc_bios; | ||
| 1611 | struct bp_crtc_source_select crtc_source_select = {0}; | ||
| 1612 | enum engine_id engine_id = link->link_enc->preferred_engine; | ||
| 1613 | |||
| 1614 | if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | ||
| 1615 | engine_id = link->link_enc->analog_engine; | ||
| 1616 | crtc_source_select.controller_id = CONTROLLER_ID_D0 + pipe_ctx->stream_res.tg->inst; | ||
| 1617 | crtc_source_select.color_depth = pipe_ctx->stream->timing.display_color_depth; | ||
| 1618 | crtc_source_select.engine_id = engine_id; | ||
| 1619 | crtc_source_select.sink_signal = pipe_ctx->stream->signal; | ||
| 1620 | bios->funcs->select_crtc_source(bios, &crtc_source_select); | ||
| 1621 | } | ||
| 1622 | 1635 | ||
| 1623 | enum dc_status dce110_apply_single_controller_ctx_to_hw( | 1636 | enum dc_status dce110_apply_single_controller_ctx_to_hw( |
| 1624 | struct pipe_ctx *pipe_ctx, | 1637 | struct pipe_ctx *pipe_ctx, |
| @@ -1639,10 +1652,6 @@ enum dc_status dce110_apply_single_controller_ctx_to_hw( · 14 unmodified lines | |||
| 1639 | hws->funcs.disable_stream_gating(dc, pipe_ctx); | 1652 | hws->funcs.disable_stream_gating(dc, pipe_ctx); |
| 1640 | } | 1653 | } |
| 1641 | 1654 | ||
| 1642 | if (pipe_ctx->stream->signal == SIGNAL_TYPE_RGB) { | ||
| 1643 | dce110_select_crtc_source(pipe_ctx); | ||
| 1644 | } | ||
| 1645 | |||
| 1646 | if (pipe_ctx->stream_res.audio != NULL) { | 1655 | if (pipe_ctx->stream_res.audio != NULL) { |
| 1647 | struct audio_output audio_output = {0}; | 1656 | struct audio_output audio_output = {0}; |
| 1648 | 1657 | ||
| @@ -1722,8 +1731,7 @@ enum dc_status dce110_apply_single_controller_ctx_to_hw( · 73 unmodified lines | |||
| 1722 | pipe_ctx->stream_res.tg->funcs->set_static_screen_control( | 1731 | pipe_ctx->stream_res.tg->funcs->set_static_screen_control( |
| 1723 | pipe_ctx->stream_res.tg, event_triggers, 2); | 1732 | pipe_ctx->stream_res.tg, event_triggers, 2); |
| 1724 | 1733 | ||
| 1725 | if (!dc_is_virtual_signal(pipe_ctx->stream->signal) && | 1734 | if (!dc_is_virtual_signal(pipe_ctx->stream->signal)) |
| 1726 | !dc_is_rgb_signal(pipe_ctx->stream->signal)) | ||
| 1727 | pipe_ctx->stream_res.stream_enc->funcs->dig_connect_to_otg( | 1735 | pipe_ctx->stream_res.stream_enc->funcs->dig_connect_to_otg( |
| 1728 | pipe_ctx->stream_res.stream_enc, | 1736 | pipe_ctx->stream_res.stream_enc, |
| 1729 | pipe_ctx->stream_res.tg->inst); | 1737 | pipe_ctx->stream_res.tg->inst); |
| @@ -3376,6 +3384,15 @@ void dce110_enable_tmds_link_output(struct dc_link *link, · 1646 unmodified lines | |||
| 3376 | link->phy_state.symclk_state = SYMCLK_ON_TX_ON; | 3384 | link->phy_state.symclk_state = SYMCLK_ON_TX_ON; |
| 3377 | } | 3385 | } |
| 3378 | 3386 | ||
| 3387 | static void dce110_enable_analog_link_output( | ||
| 3388 | struct dc_link *link, | ||
| 3389 | uint32_t pix_clk_100hz) | ||
| 3390 | { | ||
| 3391 | link->link_enc->funcs->enable_analog_output( | ||
| 3392 | link->link_enc, | ||
| 3393 | pix_clk_100hz); | ||
| 3394 | } | ||
| 3395 | |||
| 3379 | void dce110_enable_dp_link_output( | 3396 | void dce110_enable_dp_link_output( |
| 3380 | struct dc_link *link, | 3397 | struct dc_link *link, |
| 3381 | const struct link_resource *link_res, | 3398 | const struct link_resource *link_res, |
| @@ -3423,6 +3440,11 @@ void dce110_enable_dp_link_output( · 41 unmodified lines | |||
| 3423 | } | 3440 | } |
| 3424 | } | 3441 | } |
| 3425 | 3442 | ||
| 3443 | if (link->ext_enc_id.id) { | ||
| 3444 | dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_INIT, link, NULL); | ||
| 3445 | dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_SETUP, link, NULL); | ||
| 3446 | } | ||
| 3447 | |||
| 3426 | if (dc->link_srv->dp_get_encoding_format(link_settings) == DP_8b_10b_ENCODING) { | 3448 | if (dc->link_srv->dp_get_encoding_format(link_settings) == DP_8b_10b_ENCODING) { |
| 3427 | if (dc->clk_mgr->funcs->notify_link_rate_change) | 3449 | if (dc->clk_mgr->funcs->notify_link_rate_change) |
| 3428 | dc->clk_mgr->funcs->notify_link_rate_change(dc->clk_mgr, link); | 3450 | dc->clk_mgr->funcs->notify_link_rate_change(dc->clk_mgr, link); |
| @@ -3513,8 +3535,10 @@ static const struct hw_sequencer_funcs dce110_funcs = { · 84 unmodified lines | |||
| 3513 | .enable_lvds_link_output = dce110_enable_lvds_link_output, | 3535 | .enable_lvds_link_output = dce110_enable_lvds_link_output, |
| 3514 | .enable_tmds_link_output = dce110_enable_tmds_link_output, | 3536 | .enable_tmds_link_output = dce110_enable_tmds_link_output, |
| 3515 | .enable_dp_link_output = dce110_enable_dp_link_output, | 3537 | .enable_dp_link_output = dce110_enable_dp_link_output, |
| 3538 | .enable_analog_link_output = dce110_enable_analog_link_output, | ||
| 3516 | .disable_link_output = dce110_disable_link_output, | 3539 | .disable_link_output = dce110_disable_link_output, |
| 3517 | .dac_load_detect = dce110_dac_load_detect, | 3540 | .dac_load_detect = dce110_dac_load_detect, |
| 3541 | .prepare_ddc = dce110_prepare_ddc, | ||
| 3518 | }; | 3542 | }; |
| 3519 | 3543 | ||
| 3520 | static const struct hwseq_private_funcs dce110_private_funcs = { | 3544 | static const struct hwseq_private_funcs dce110_private_funcs = { |
| @@ -568,7 +568,9 @@ static bool construct_phy(struct dc_link *link, · 567 unmodified lines | |||
| 568 | goto ddc_create_fail; | 568 | goto ddc_create_fail; |
| 569 | } | 569 | } |
| 570 | 570 | ||
| 571 | if (!link->ddc->ddc_pin) { | 571 | /* Embedded display connectors such as LVDS may not have DDC. */ |
| 572 | if (!link->ddc->ddc_pin && | ||
| 573 | !dc_is_embedded_signal(link->connector_signal)) { | ||
| 572 | DC_ERROR("Failed to get I2C info for connector!\n"); | 574 | DC_ERROR("Failed to get I2C info for connector!\n"); |
| 573 | goto ddc_create_fail; | 575 | goto ddc_create_fail; |
| 574 | } | 576 | } |
| @@ -753,7 +753,8 @@ static struct link_encoder *dce60_link_encoder_create( · 752 unmodified lines | |||
| 753 | enc_init_data, | 753 | enc_init_data, |
| 754 | &link_enc_feature, | 754 | &link_enc_feature, |
| 755 | &link_enc_regs[link_regs_id], | 755 | &link_enc_regs[link_regs_id], |
| 756 | &link_enc_aux_regs[enc_init_data->channel - 1], | 756 | enc_init_data->channel == CHANNEL_ID_UNKNOWN ? |
| 757 | NULL : &link_enc_aux_regs[enc_init_data->channel - 1], | ||
| 757 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? | 758 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? |
| 758 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); | 759 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); |
| 759 | return &enc110->base; | 760 | return &enc110->base; |
| @@ -760,7 +760,8 @@ static struct link_encoder *dce80_link_encoder_create( · 759 unmodified lines | |||
| 760 | enc_init_data, | 760 | enc_init_data, |
| 761 | &link_enc_feature, | 761 | &link_enc_feature, |
| 762 | &link_enc_regs[link_regs_id], | 762 | &link_enc_regs[link_regs_id], |
| 763 | &link_enc_aux_regs[enc_init_data->channel - 1], | 763 | enc_init_data->channel == CHANNEL_ID_UNKNOWN ? |
| 764 | NULL : &link_enc_aux_regs[enc_init_data->channel - 1], | ||
| 764 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? | 765 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? |
| 765 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); | 766 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); |
| 766 | return &enc110->base; | 767 | return &enc110->base; |
| @@ -92,9 +92,14 @@ · 91 unmodified lines | |||
| 92 | #include "dml/dcn32/dcn32_fpu.h" | 92 | #include "dml/dcn32/dcn32_fpu.h" |
| 93 | 93 | ||
| 94 | #include "dc_state_priv.h" | 94 | #include "dc_state_priv.h" |
| 95 | #include "dc_fpu.h" | ||
| 95 | 96 | ||
| 96 | #include "dml2_0/dml2_wrapper.h" | 97 | #include "dml2_0/dml2_wrapper.h" |
| 97 | 98 | ||
| 99 | #if !defined(DC_RUN_WITH_PREEMPTION_ENABLED) | ||
| 100 | #define DC_RUN_WITH_PREEMPTION_ENABLED(code) code | ||
| 101 | #endif | ||
| 102 | |||
| 98 | #define DC_LOGGER_INIT(logger) | 103 | #define DC_LOGGER_INIT(logger) |
| 99 | 104 | ||
| 100 | enum dcn32_clk_src_array_id { | 105 | enum dcn32_clk_src_array_id { |
| @@ -1684,7 +1689,8 @@ static void dcn32_enable_phantom_plane(struct dc *dc, · 1583 unmodified lines | |||
| 1684 | if (curr_pipe->top_pipe && curr_pipe->top_pipe->plane_state == curr_pipe->plane_state) | 1689 | if (curr_pipe->top_pipe && curr_pipe->top_pipe->plane_state == curr_pipe->plane_state) |
| 1685 | phantom_plane = prev_phantom_plane; | 1690 | phantom_plane = prev_phantom_plane; |
| 1686 | else | 1691 | else |
| 1687 | phantom_plane = dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state); | 1692 | DC_RUN_WITH_PREEMPTION_ENABLED(phantom_plane = |
| 1693 | dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state)); | ||
| 1688 | 1694 | ||
| 1689 | if (!phantom_plane) | 1695 | if (!phantom_plane) |
| 1690 | continue; | 1696 | continue; |
| @@ -153,6 +153,10 @@ struct embedded_panel_info { · 152 unmodified lines | |||
| 153 | uint32_t drr_enabled; | 153 | uint32_t drr_enabled; |
| 154 | uint32_t min_drr_refresh_rate; | 154 | uint32_t min_drr_refresh_rate; |
| 155 | bool realtek_eDPToLVDS; | 155 | bool realtek_eDPToLVDS; |
| 156 | uint16_t panel_width_mm; | ||
| 157 | uint16_t panel_height_mm; | ||
| 158 | uint16_t fake_edid_size; | ||
| 159 | const uint8_t *fake_edid; | ||
| 156 | }; | 160 | }; |
| 157 | 161 | ||
| 158 | struct dc_firmware_info { | 162 | struct dc_firmware_info { |
| @@ -3076,6 +3076,10 @@ static bool si_dpm_vblank_too_short(void *handle) · 3075 unmodified lines | |||
| 3076 | /* we never hit the non-gddr5 limit so disable it */ | 3076 | /* we never hit the non-gddr5 limit so disable it */ |
| 3077 | u32 switch_limit = adev->gmc.vram_type == AMDGPU_VRAM_TYPE_GDDR5 ? 450 : 0; | 3077 | u32 switch_limit = adev->gmc.vram_type == AMDGPU_VRAM_TYPE_GDDR5 ? 450 : 0; |
| 3078 | 3078 | ||
| 3079 | /* Disregard vblank time when there are no displays connected */ | ||
| 3080 | if (!adev->pm.pm_display_cfg.num_display) | ||
| 3081 | return false; | ||
| 3082 | |||
| 3079 | /* Consider zero vblank time too short and disable MCLK switching. | 3083 | /* Consider zero vblank time too short and disable MCLK switching. |
| 3080 | * Note that the vblank time is set to maximum when no displays are attached, | 3084 | * Note that the vblank time is set to maximum when no displays are attached, |
| 3081 | * so we'll still enable MCLK switching in that case. | 3085 | * so we'll still enable MCLK switching in that case. |
| @@ -1333,12 +1333,13 @@ static int ci_populate_all_memory_levels(struct pp_hwmgr *hwmgr) · 1332 unmodified lines | |||
| 1333 | 1333 | ||
| 1334 | dev_id = adev->pdev->device; | 1334 | dev_id = adev->pdev->device; |
| 1335 | 1335 | ||
| 1336 | if ((dpm_table->mclk_table.count >= 2) | 1336 | if ((dpm_table->mclk_table.count >= 2) && |
| 1337 | && ((dev_id == 0x67B0) || (dev_id == 0x67B1))) { | 1337 | ((dev_id == 0x67B0) || (dev_id == 0x67B1)) && |
| 1338 | smu_data->smc_state_table.MemoryLevel[1].MinVddci = | 1338 | (adev->pdev->revision == 0)) { |
| 1339 | smu_data->smc_state_table.MemoryLevel[0].MinVddci; | 1339 | smu_data->smc_state_table.MemoryLevel[1].MinVddc = |
| 1340 | smu_data->smc_state_table.MemoryLevel[1].MinMvdd = | 1340 | smu_data->smc_state_table.MemoryLevel[0].MinVddc; |
| 1341 | smu_data->smc_state_table.MemoryLevel[0].MinMvdd; | 1341 | smu_data->smc_state_table.MemoryLevel[1].MinVddcPhases = |
| 1342 | smu_data->smc_state_table.MemoryLevel[0].MinVddcPhases; | ||
| 1342 | } | 1343 | } |
| 1343 | smu_data->smc_state_table.MemoryLevel[0].ActivityLevel = 0x1F; | 1344 | smu_data->smc_state_table.MemoryLevel[0].ActivityLevel = 0x1F; |
| 1344 | CONVERT_FROM_HOST_TO_SMC_US(smu_data->smc_state_table.MemoryLevel[0].ActivityLevel); | 1345 | CONVERT_FROM_HOST_TO_SMC_US(smu_data->smc_state_table.MemoryLevel[0].ActivityLevel); |
| @@ -425,6 +425,7 @@ static int aldebaran_set_default_dpm_table(struct smu_context *smu) · 424 unmodified lines | |||
| 425 | dpm_table->dpm_levels[0].enabled = true; | 425 | dpm_table->dpm_levels[0].enabled = true; |
| 426 | dpm_table->dpm_levels[1].value = pptable->GfxclkFmax; | 426 | dpm_table->dpm_levels[1].value = pptable->GfxclkFmax; |
| 427 | dpm_table->dpm_levels[1].enabled = true; | 427 | dpm_table->dpm_levels[1].enabled = true; |
| 428 | dpm_table->flags |= SMU_DPM_TABLE_FINE_GRAINED; | ||
| 428 | } else { | 429 | } else { |
| 429 | dpm_table->count = 1; | 430 | dpm_table->count = 1; |
| 430 | dpm_table->dpm_levels[0].value = smu->smu_table.boot_values.gfxclk / 100; | 431 | dpm_table->dpm_levels[0].value = smu->smu_table.boot_values.gfxclk / 100; |
| @@ -1129,6 +1129,7 @@ static int smu_v13_0_6_set_default_dpm_table(struct smu_context *smu) · 1128 unmodified lines | |||
| 1129 | /* gfxclk dpm table setup */ | 1129 | /* gfxclk dpm table setup */ |
| 1130 | dpm_table = &dpm_context->dpm_tables.gfx_table; | 1130 | dpm_table = &dpm_context->dpm_tables.gfx_table; |
| 1131 | dpm_table->clk_type = SMU_GFXCLK; | 1131 | dpm_table->clk_type = SMU_GFXCLK; |
| 1132 | dpm_table->flags = SMU_DPM_TABLE_FINE_GRAINED; | ||
| 1132 | if (smu_cmn_feature_is_enabled(smu, SMU_FEATURE_DPM_GFXCLK_BIT)) { | 1133 | if (smu_cmn_feature_is_enabled(smu, SMU_FEATURE_DPM_GFXCLK_BIT)) { |
| 1133 | /* In the case of gfxclk, only fine-grained dpm is honored. | 1134 | /* In the case of gfxclk, only fine-grained dpm is honored. |
| 1134 | * Get min/max values from FW. | 1135 | * Get min/max values from FW. |
| @@ -435,10 +435,12 @@ int smu_v15_0_fini_smc_tables(struct smu_context *smu) · 434 unmodified lines | |||
| 435 | smu_table->watermarks_table = NULL; | 435 | smu_table->watermarks_table = NULL; |
| 436 | smu_table->metrics_time = 0; | 436 | smu_table->metrics_time = 0; |
| 437 | 437 | ||
| 438 | kfree(smu_dpm->dpm_policies); | ||
| 438 | kfree(smu_dpm->dpm_context); | 439 | kfree(smu_dpm->dpm_context); |
| 439 | kfree(smu_dpm->golden_dpm_context); | 440 | kfree(smu_dpm->golden_dpm_context); |
| 440 | kfree(smu_dpm->dpm_current_power_state); | 441 | kfree(smu_dpm->dpm_current_power_state); |
| 441 | kfree(smu_dpm->dpm_request_power_state); | 442 | kfree(smu_dpm->dpm_request_power_state); |
| 443 | smu_dpm->dpm_policies = NULL; | ||
| 442 | smu_dpm->dpm_context = NULL; | 444 | smu_dpm->dpm_context = NULL; |
| 443 | smu_dpm->golden_dpm_context = NULL; | 445 | smu_dpm->golden_dpm_context = NULL; |
| 444 | smu_dpm->dpm_context_size = 0; | 446 | smu_dpm->dpm_context_size = 0; |
| @@ -1370,7 +1370,7 @@ int smu_cmn_print_dpm_clk_levels(struct smu_context *smu, · 1369 unmodified lines | |||
| 1370 | level_index = 1; | 1370 | level_index = 1; |
| 1371 | } | 1371 | } |
| 1372 | 1372 | ||
| 1373 | if (!is_fine_grained) { | 1373 | if (!is_fine_grained || count == 1) { |
| 1374 | for (i = 0; i < count; i++) { | 1374 | for (i = 0; i < count; i++) { |
| 1375 | freq_match = !is_deep_sleep && | 1375 | freq_match = !is_deep_sleep && |
| 1376 | smu_cmn_freqs_match( | 1376 | smu_cmn_freqs_match( |
| @@ -758,7 +758,9 @@ static int chipone_i2c_probe(struct i2c_client *client) · 757 unmodified lines | |||
| 758 | dev_set_drvdata(dev, icn); | 758 | dev_set_drvdata(dev, icn); |
| 759 | i2c_set_clientdata(client, icn); | 759 | i2c_set_clientdata(client, icn); |
| 760 | 760 | ||
| 761 | drm_bridge_add(&icn->bridge); | 761 | ret = devm_drm_bridge_add(dev, &icn->bridge); |
| 762 | if (ret) | ||
| 763 | return ret; | ||
| 762 | 764 | ||
| 763 | return chipone_dsi_host_attach(icn); | 765 | return chipone_dsi_host_attach(icn); |
| 764 | } | 766 | } |
| @@ -222,52 +222,58 @@ static const struct drm_bridge_funcs imx8qxp_pxl2dpi_bridge_funcs = { · 221 unmodified lines | |||
| 222 | imx8qxp_pxl2dpi_bridge_atomic_get_output_bus_fmts, | 222 | imx8qxp_pxl2dpi_bridge_atomic_get_output_bus_fmts, |
| 223 | }; | 223 | }; |
| 224 | 224 | ||
| 225 | static struct device_node * | 225 | static int |
| 226 | imx8qxp_pxl2dpi_get_available_ep_from_port(struct imx8qxp_pxl2dpi *p2d, | 226 | imx8qxp_pxl2dpi_get_available_ep_from_port(struct imx8qxp_pxl2dpi *p2d, |
| 227 | u32 port_id) | 227 | u32 port_id, |
| 228 | struct device_node **ep) | ||
| 228 | { | 229 | { |
| 229 | struct device_node *port, *ep; | 230 | struct device_node *port; |
| 231 | int ret = 0; | ||
| 230 | int ep_cnt; | 232 | int ep_cnt; |
| 231 | 233 | ||
| 234 | *ep = NULL; | ||
| 235 | |||
| 232 | port = of_graph_get_port_by_id(p2d->dev->of_node, port_id); | 236 | port = of_graph_get_port_by_id(p2d->dev->of_node, port_id); |
| 233 | if (!port) { | 237 | if (!port) { |
| 234 | DRM_DEV_ERROR(p2d->dev, "failed to get port@%u\n", port_id); | 238 | DRM_DEV_ERROR(p2d->dev, "failed to get port@%u\n", port_id); |
| 235 | return ERR_PTR(-ENODEV); | 239 | return -ENODEV; |
| 236 | } | 240 | } |
| 237 | 241 | ||
| 238 | ep_cnt = of_get_available_child_count(port); | 242 | ep_cnt = of_get_available_child_count(port); |
| 239 | if (ep_cnt == 0) { | 243 | if (ep_cnt == 0) { |
| 240 | DRM_DEV_ERROR(p2d->dev, "no available endpoints of port@%u\n", | 244 | DRM_DEV_ERROR(p2d->dev, "no available endpoints of port@%u\n", |
| 241 | port_id); | 245 | port_id); |
| 242 | ep = ERR_PTR(-ENODEV); | 246 | ret = -ENODEV; |
| 243 | goto out; | 247 | goto out; |
| 244 | } else if (ep_cnt > 1) { | 248 | } else if (ep_cnt > 1) { |
| 245 | DRM_DEV_ERROR(p2d->dev, | 249 | DRM_DEV_ERROR(p2d->dev, |
| 246 | "invalid available endpoints of port@%u\n", | 250 | "invalid available endpoints of port@%u\n", |
| 247 | port_id); | 251 | port_id); |
| 248 | ep = ERR_PTR(-EINVAL); | 252 | ret = -EINVAL; |
| 249 | goto out; | 253 | goto out; |
| 250 | } | 254 | } |
| 251 | 255 | ||
| 252 | ep = of_get_next_available_child(port, NULL); | 256 | *ep = of_get_next_available_child(port, NULL); |
| 253 | if (!ep) { | 257 | if (!*ep) { |
| 254 | DRM_DEV_ERROR(p2d->dev, | 258 | DRM_DEV_ERROR(p2d->dev, |
| 255 | "failed to get available endpoint of port@%u\n", | 259 | "failed to get available endpoint of port@%u\n", |
| 256 | port_id); | 260 | port_id); |
| 257 | ep = ERR_PTR(-ENODEV); | 261 | ret = -ENODEV; |
| 258 | goto out; | 262 | goto out; |
| 259 | } | 263 | } |
| 260 | out: | 264 | out: |
| 261 | of_node_put(port); | 265 | of_node_put(port); |
| 262 | return ep; | 266 | return ret; |
| 263 | } | 267 | } |
| 264 | 268 | ||
| 265 | static int imx8qxp_pxl2dpi_find_next_bridge(struct imx8qxp_pxl2dpi *p2d) | 269 | static int imx8qxp_pxl2dpi_find_next_bridge(struct imx8qxp_pxl2dpi *p2d) |
| 266 | { | 270 | { |
| 267 | struct device_node *ep __free(device_node) = | 271 | struct device_node *ep __free(device_node) = NULL; |
| 268 | imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1); | 272 | int ret; |
| 269 | if (IS_ERR(ep)) | 273 | |
| 270 | return PTR_ERR(ep); | 274 | ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1, &ep); |
| 275 | if (ret) | ||
| 276 | return ret; | ||
| 271 | 277 | ||
| 272 | struct device_node *remote __free(device_node) = of_graph_get_remote_port_parent(ep); | 278 | struct device_node *remote __free(device_node) = of_graph_get_remote_port_parent(ep); |
| 273 | if (!remote || !of_device_is_available(remote)) { | 279 | if (!remote || !of_device_is_available(remote)) { |
| @@ -291,9 +297,9 @@ static int imx8qxp_pxl2dpi_set_pixel_link_sel(struct imx8qxp_pxl2dpi *p2d) · 17 unmodified lines | |||
| 291 | struct of_endpoint endpoint; | 297 | struct of_endpoint endpoint; |
| 292 | int ret; | 298 | int ret; |
| 293 | 299 | ||
| 294 | ep = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0); | 300 | ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0, &ep); |
| 295 | if (IS_ERR(ep)) | 301 | if (ret) |
| 296 | return PTR_ERR(ep); | 302 | return ret; |
| 297 | 303 | ||
| 298 | ret = of_graph_parse_endpoint(ep, &endpoint); | 304 | ret = of_graph_parse_endpoint(ep, &endpoint); |
| 299 | if (ret) { | 305 | if (ret) { |
| @@ -1559,6 +1559,11 @@ static int it66121_probe(struct i2c_client *client) · 1558 unmodified lines | |||
| 1559 | return ret; | 1559 | return ret; |
| 1560 | } | 1560 | } |
| 1561 | 1561 | ||
| 1562 | ctx->gpio_reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW); | ||
| 1563 | if (IS_ERR(ctx->gpio_reset)) | ||
| 1564 | return dev_err_probe(dev, PTR_ERR(ctx->gpio_reset), | ||
| 1565 | "Failed to get reset GPIO\n"); | ||
| 1566 | |||
| 1562 | it66121_hw_reset(ctx); | 1567 | it66121_hw_reset(ctx); |
| 1563 | 1568 | ||
| 1564 | ctx->regmap = devm_regmap_init_i2c(client, &it66121_regmap_config); | 1569 | ctx->regmap = devm_regmap_init_i2c(client, &it66121_regmap_config); |
| @@ -251,7 +251,6 @@ static void ge_b850v3_lvds_remove(void) · 250 unmodified lines | |||
| 251 | goto out; | 251 | goto out; |
| 252 | 252 | ||
| 253 | drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge); | 253 | drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge); |
| 254 | |||
| 255 | ge_b850v3_lvds_ptr = NULL; | 254 | ge_b850v3_lvds_ptr = NULL; |
| 256 | out: | 255 | out: |
| 257 | mutex_unlock(&ge_b850v3_lvds_dev_mutex); | 256 | mutex_unlock(&ge_b850v3_lvds_dev_mutex); |
| @@ -261,6 +260,7 @@ static int ge_b850v3_register(void) · 3 unmodified lines | |||
| 261 | { | 260 | { |
| 262 | struct i2c_client *stdp4028_i2c = ge_b850v3_lvds_ptr->stdp4028_i2c; | 261 | struct i2c_client *stdp4028_i2c = ge_b850v3_lvds_ptr->stdp4028_i2c; |
| 263 | struct device *dev = &stdp4028_i2c->dev; | 262 | struct device *dev = &stdp4028_i2c->dev; |
| 263 | int ret; | ||
| 264 | 264 | ||
| 265 | /* drm bridge initialization */ | 265 | /* drm bridge initialization */ |
| 266 | ge_b850v3_lvds_ptr->bridge.ops = DRM_BRIDGE_OP_DETECT | | 266 | ge_b850v3_lvds_ptr->bridge.ops = DRM_BRIDGE_OP_DETECT | |
| @@ -277,11 +277,15 @@ static int ge_b850v3_register(void) · 10 unmodified lines | |||
| 277 | if (!stdp4028_i2c->irq) | 277 | if (!stdp4028_i2c->irq) |
| 278 | return 0; | 278 | return 0; |
| 279 | 279 | ||
| 280 | return devm_request_threaded_irq(&stdp4028_i2c->dev, | 280 | ret = devm_request_threaded_irq(&stdp4028_i2c->dev, |
| 281 | stdp4028_i2c->irq, NULL, | 281 | stdp4028_i2c->irq, NULL, |
| 282 | ge_b850v3_lvds_irq_handler, | 282 | ge_b850v3_lvds_irq_handler, |
| 283 | IRQF_TRIGGER_HIGH | IRQF_ONESHOT, | 283 | IRQF_TRIGGER_HIGH | IRQF_ONESHOT, |
| 284 | "ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr); | 284 | "ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr); |
| 285 | if (ret) | ||
| 286 | drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge); | ||
| 287 | |||
| 288 | return ret; | ||
| 285 | } | 289 | } |
| 286 | 290 | ||
| 287 | static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c) | 291 | static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c) |
| @@ -1293,7 +1293,7 @@ static const struct drm_edid *tda998x_edid_read(struct tda998x_priv *priv, · 1292 unmodified lines | |||
| 1293 | * can't handle signals gracefully. | 1293 | * can't handle signals gracefully. |
| 1294 | */ | 1294 | */ |
| 1295 | if (tda998x_edid_delay_wait(priv)) | 1295 | if (tda998x_edid_delay_wait(priv)) |
| 1296 | return 0; | 1296 | return NULL; |
| 1297 | 1297 | ||
| 1298 | if (priv->rev == TDA19988) | 1298 | if (priv->rev == TDA19988) |
| 1299 | reg_clear(priv, REG_TX4, TX4_PD_RAM); | 1299 | reg_clear(priv, REG_TX4, TX4_PD_RAM); |
| @@ -1762,7 +1762,7 @@ static const struct drm_bridge_funcs tda998x_bridge_funcs = { · 462 unmodified lines | |||
| 1762 | static int tda998x_get_audio_ports(struct tda998x_priv *priv, | 1762 | static int tda998x_get_audio_ports(struct tda998x_priv *priv, |
| 1763 | struct device_node *np) | 1763 | struct device_node *np) |
| 1764 | { | 1764 | { |
| 1765 | const u32 *port_data; | 1765 | const __be32 *port_data; |
| 1766 | u32 size; | 1766 | u32 size; |
| 1767 | int i; | 1767 | int i; |
| 1768 | 1768 | ||
| @@ -831,7 +831,7 @@ static void fill_palette_332(struct drm_crtc *crtc, u16 r, u16 g, u16 b, · 830 unmodified lines | |||
| 831 | } | 831 | } |
| 832 | 832 | ||
| 833 | /** | 833 | /** |
| 834 | * drm_crtc_fill_palette_332 - Programs a default palette for R332-like formats | 834 | * drm_crtc_fill_palette_332 - Programs a default palette for RGB332-like formats |
| 835 | * @crtc: The displaying CRTC | 835 | * @crtc: The displaying CRTC |
| 836 | * @set_palette: Callback for programming the hardware gamma LUT | 836 | * @set_palette: Callback for programming the hardware gamma LUT |
| 837 | * | 837 | * |
| @@ -697,6 +697,7 @@ static void drm_dev_init_release(struct drm_device *dev, void *res) · 696 unmodified lines | |||
| 697 | mutex_destroy(&dev->master_mutex); | 697 | mutex_destroy(&dev->master_mutex); |
| 698 | mutex_destroy(&dev->clientlist_mutex); | 698 | mutex_destroy(&dev->clientlist_mutex); |
| 699 | mutex_destroy(&dev->filelist_mutex); | 699 | mutex_destroy(&dev->filelist_mutex); |
| 700 | mutex_destroy(&dev->gem_lru_mutex); | ||
| 700 | } | 701 | } |
| 701 | 702 | ||
| 702 | static int drm_dev_init(struct drm_device *dev, | 703 | static int drm_dev_init(struct drm_device *dev, |
| @@ -738,6 +739,7 @@ static int drm_dev_init(struct drm_device *dev, · 35 unmodified lines | |||
| 738 | INIT_LIST_HEAD(&dev->vblank_event_list); | 739 | INIT_LIST_HEAD(&dev->vblank_event_list); |
| 739 | 740 | ||
| 740 | spin_lock_init(&dev->event_lock); | 741 | spin_lock_init(&dev->event_lock); |
| 742 | mutex_init(&dev->gem_lru_mutex); | ||
| 741 | mutex_init(&dev->filelist_mutex); | 743 | mutex_init(&dev->filelist_mutex); |
| 742 | mutex_init(&dev->clientlist_mutex); | 744 | mutex_init(&dev->clientlist_mutex); |
| 743 | mutex_init(&dev->master_mutex); | 745 | mutex_init(&dev->master_mutex); |
| @@ -70,8 +70,11 @@ static int drm_mode_align_dumb(struct drm_mode_create_dumb *args, · 69 unmodified lines | |||
| 70 | if (!pitch) | 70 | if (!pitch) |
| 71 | return -EINVAL; | 71 | return -EINVAL; |
| 72 | 72 | ||
| 73 | if (hw_pitch_align) | 73 | if (hw_pitch_align) { |
| 74 | pitch = roundup(pitch, hw_pitch_align); | 74 | pitch = roundup(pitch, hw_pitch_align); |
| 75 | if (pitch < hw_pitch_align) | ||
| 76 | return -EINVAL; | ||
| 77 | } | ||
| 75 | 78 | ||
| 76 | if (!hw_size_align) | 79 | if (!hw_size_align) |
| 77 | hw_size_align = PAGE_SIZE; | 80 | hw_size_align = PAGE_SIZE; |
| @@ -80,7 +83,7 @@ static int drm_mode_align_dumb(struct drm_mode_create_dumb *args, · 2 unmodified lines | |||
| 80 | 83 | ||
| 81 | if (check_mul_overflow(args->height, pitch, &size)) | 84 | if (check_mul_overflow(args->height, pitch, &size)) |
| 82 | return -EINVAL; | 85 | return -EINVAL; |
| 83 | size = ALIGN(size, hw_size_align); | 86 | size = roundup(size, hw_size_align); |
| 84 | if (!size) | 87 | if (!size) |
| 85 | return -EINVAL; | 88 | return -EINVAL; |
| 86 | 89 | ||
| @@ -199,6 +202,13 @@ int drm_mode_create_dumb(struct drm_device *dev, · 112 unmodified lines | |||
| 199 | if (!args->width || !args->height || !args->bpp) | 202 | if (!args->width || !args->height || !args->bpp) |
| 200 | return -EINVAL; | 203 | return -EINVAL; |
| 201 | 204 | ||
| 205 | /* Reject unreasonable inputs early. Dumb buffers are for software | ||
| 206 | * rendering; nothing legitimate needs more than 8192x8192 at 32bpp. | ||
| 207 | * This prevents overflows in downstream alignment helpers. | ||
| 208 | */ | ||
| 209 | if (args->width >= 8192 || args->height >= 8192 || args->bpp > 32) | ||
| 210 | return -EINVAL; | ||
| 211 | |||
| 202 | /* overflow checks for 32bit size calculations */ | 212 | /* overflow checks for 32bit size calculations */ |
| 203 | if (args->bpp > U32_MAX - 8) | 213 | if (args->bpp > U32_MAX - 8) |
| 204 | return -EINVAL; | 214 | return -EINVAL; |
| @@ -490,7 +490,7 @@ static void drm_fb_helper_memory_range_to_clip(struct fb_info *info, off_t off, · 489 unmodified lines | |||
| 490 | * the number of horizontal pixels that need an update. | 490 | * the number of horizontal pixels that need an update. |
| 491 | */ | 491 | */ |
| 492 | off_t bit_off = (off % line_length) * 8; | 492 | off_t bit_off = (off % line_length) * 8; |
| 493 | off_t bit_end = (end % line_length) * 8; | 493 | off_t bit_end = bit_off + len * 8; |
| 494 | 494 | ||
| 495 | x1 = bit_off / info->var.bits_per_pixel; | 495 | x1 = bit_off / info->var.bits_per_pixel; |
| 496 | x2 = DIV_ROUND_UP(bit_end, info->var.bits_per_pixel); | 496 | x2 = DIV_ROUND_UP(bit_end, info->var.bits_per_pixel); |
| @@ -1019,7 +1019,7 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 1018 unmodified lines | |||
| 1019 | struct drm_file *file_priv) | 1019 | struct drm_file *file_priv) |
| 1020 | { | 1020 | { |
| 1021 | struct drm_gem_change_handle *args = data; | 1021 | struct drm_gem_change_handle *args = data; |
| 1022 | struct drm_gem_object *obj; | 1022 | struct drm_gem_object *obj, *idrobj; |
| 1023 | int handle, ret; | 1023 | int handle, ret; |
| 1024 | 1024 | ||
| 1025 | if (!drm_core_check_feature(dev, DRIVER_GEM)) | 1025 | if (!drm_core_check_feature(dev, DRIVER_GEM)) |
| @@ -1042,12 +1042,31 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 16 unmodified lines | |||
| 1042 | mutex_lock(&file_priv->prime.lock); | 1042 | mutex_lock(&file_priv->prime.lock); |
| 1043 | 1043 | ||
| 1044 | spin_lock(&file_priv->table_lock); | 1044 | spin_lock(&file_priv->table_lock); |
| 1045 | |||
| 1046 | /* When create_tail allocs an obj idr, it needs to first alloc as NULL, | ||
| 1047 | * then later replace with the correct object. This is not necessary | ||
| 1048 | * here, because the only operations that could race are drm_prime | ||
| 1049 | * bookkeeping, and we hold the prime lock. | ||
| 1050 | */ | ||
| 1045 | ret = idr_alloc(&file_priv->object_idr, obj, handle, handle + 1, | 1051 | ret = idr_alloc(&file_priv->object_idr, obj, handle, handle + 1, |
| 1046 | GFP_NOWAIT); | 1052 | GFP_NOWAIT); |
| 1047 | spin_unlock(&file_priv->table_lock); | ||
| 1048 | 1053 | ||
| 1049 | if (ret < 0) | 1054 | if (ret < 0) { |
| 1050 | goto out_unlock; | 1055 | spin_unlock(&file_priv->table_lock); |
| 1056 | goto out_unlock; | ||
| 1057 | } | ||
| 1058 | |||
| 1059 | idrobj = idr_replace(&file_priv->object_idr, NULL, handle); | ||
| 1060 | if (idrobj != obj) { | ||
| 1061 | idr_replace(&file_priv->object_idr, idrobj, handle); | ||
| 1062 | idr_remove(&file_priv->object_idr, args->new_handle); | ||
| 1063 | spin_unlock(&file_priv->table_lock); | ||
| 1064 | ret = -ENOENT; | ||
| 1065 | goto out_unlock; | ||
| 1066 | } | ||
| 1067 | |||
| 1068 | idr_replace(&file_priv->object_idr, NULL, args->handle); | ||
| 1069 | spin_unlock(&file_priv->table_lock); | ||
| 1051 | 1070 | ||
| 1052 | if (obj->dma_buf) { | 1071 | if (obj->dma_buf) { |
| 1053 | ret = drm_prime_add_buf_handle(&file_priv->prime, obj->dma_buf, | 1072 | ret = drm_prime_add_buf_handle(&file_priv->prime, obj->dma_buf, |
| @@ -1055,6 +1074,7 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 1 unmodified line | |||
| 1055 | if (ret < 0) { | 1074 | if (ret < 0) { |
| 1056 | spin_lock(&file_priv->table_lock); | 1075 | spin_lock(&file_priv->table_lock); |
| 1057 | idr_remove(&file_priv->object_idr, handle); | 1076 | idr_remove(&file_priv->object_idr, handle); |
| 1077 | idr_replace(&file_priv->object_idr, obj, args->handle); | ||
| 1058 | spin_unlock(&file_priv->table_lock); | 1078 | spin_unlock(&file_priv->table_lock); |
| 1059 | goto out_unlock; | 1079 | goto out_unlock; |
| 1060 | } | 1080 | } |
| @@ -1066,7 +1086,9 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 5 unmodified lines | |||
| 1066 | 1086 | ||
| 1067 | spin_lock(&file_priv->table_lock); | 1087 | spin_lock(&file_priv->table_lock); |
| 1068 | idr_remove(&file_priv->object_idr, args->handle); | 1088 | idr_remove(&file_priv->object_idr, args->handle); |
| 1089 | idrobj = idr_replace(&file_priv->object_idr, obj, handle); | ||
| 1069 | spin_unlock(&file_priv->table_lock); | 1090 | spin_unlock(&file_priv->table_lock); |
| 1091 | WARN_ON(idrobj != NULL); | ||
| 1070 | 1092 | ||
| 1071 | out_unlock: | 1093 | out_unlock: |
| 1072 | mutex_unlock(&file_priv->prime.lock); | 1094 | mutex_unlock(&file_priv->prime.lock); |
| @@ -1541,12 +1563,10 @@ EXPORT_SYMBOL(drm_gem_unlock_reservations); · 468 unmodified lines | |||
| 1541 | * drm_gem_lru_init - initialize a LRU | 1563 | * drm_gem_lru_init - initialize a LRU |
| 1542 | * | 1564 | * |
| 1543 | * @lru: The LRU to initialize | 1565 | * @lru: The LRU to initialize |
| 1544 | * @lock: The lock protecting the LRU | ||
| 1545 | */ | 1566 | */ |
| 1546 | void | 1567 | void |
| 1547 | drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock) | 1568 | drm_gem_lru_init(struct drm_gem_lru *lru) |
| 1548 | { | 1569 | { |
| 1549 | lru->lock = lock; | ||
| 1550 | lru->count = 0; | 1570 | lru->count = 0; |
| 1551 | INIT_LIST_HEAD(&lru->list); | 1571 | INIT_LIST_HEAD(&lru->list); |
| 1552 | } | 1572 | } |
| @@ -1571,14 +1591,10 @@ drm_gem_lru_remove_locked(struct drm_gem_object *obj) · 18 unmodified lines | |||
| 1571 | void | 1591 | void |
| 1572 | drm_gem_lru_remove(struct drm_gem_object *obj) | 1592 | drm_gem_lru_remove(struct drm_gem_object *obj) |
| 1573 | { | 1593 | { |
| 1574 | struct drm_gem_lru *lru = obj->lru; | 1594 | mutex_lock(&obj->dev->gem_lru_mutex); |
| 1575 | 1595 | if (obj->lru) | |
| 1576 | if (!lru) | 1596 | drm_gem_lru_remove_locked(obj); |
| 1577 | return; | 1597 | mutex_unlock(&obj->dev->gem_lru_mutex); |
| 1578 | |||
| 1579 | mutex_lock(lru->lock); | ||
| 1580 | drm_gem_lru_remove_locked(obj); | ||
| 1581 | mutex_unlock(lru->lock); | ||
| 1582 | } | 1598 | } |
| 1583 | EXPORT_SYMBOL(drm_gem_lru_remove); | 1599 | EXPORT_SYMBOL(drm_gem_lru_remove); |
| 1584 | 1600 | ||
| @@ -1593,7 +1609,7 @@ EXPORT_SYMBOL(drm_gem_lru_remove); · 8 unmodified lines | |||
| 1593 | void | 1609 | void |
| 1594 | drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj) | 1610 | drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj) |
| 1595 | { | 1611 | { |
| 1596 | lockdep_assert_held_once(lru->lock); | 1612 | lockdep_assert_held_once(&obj->dev->gem_lru_mutex); |
| 1597 | 1613 | ||
| 1598 | if (obj->lru) | 1614 | if (obj->lru) |
| 1599 | drm_gem_lru_remove_locked(obj); | 1615 | drm_gem_lru_remove_locked(obj); |
| @@ -1617,9 +1633,9 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail_locked); · 17 unmodified lines | |||
| 1617 | void | 1633 | void |
| 1618 | drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj) | 1634 | drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj) |
| 1619 | { | 1635 | { |
| 1620 | mutex_lock(lru->lock); | 1636 | mutex_lock(&obj->dev->gem_lru_mutex); |
| 1621 | drm_gem_lru_move_tail_locked(lru, obj); | 1637 | drm_gem_lru_move_tail_locked(lru, obj); |
| 1622 | mutex_unlock(lru->lock); | 1638 | mutex_unlock(&obj->dev->gem_lru_mutex); |
| 1623 | } | 1639 | } |
| 1624 | EXPORT_SYMBOL(drm_gem_lru_move_tail); | 1640 | EXPORT_SYMBOL(drm_gem_lru_move_tail); |
| 1625 | 1641 | ||
| @@ -1633,6 +1649,7 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail); · 7 unmodified lines | |||
| 1633 | * of the shrink callback to check for this (ie. dma_resv_test_signaled()) | 1649 | * of the shrink callback to check for this (ie. dma_resv_test_signaled()) |
| 1634 | * or if necessary block until the buffer becomes idle. | 1650 | * or if necessary block until the buffer becomes idle. |
| 1635 | * | 1651 | * |
| 1652 | * @dev: DRM device the LRU belongs to | ||
| 1636 | * @lru: The LRU to scan | 1653 | * @lru: The LRU to scan |
| 1637 | * @nr_to_scan: The number of pages to try to reclaim | 1654 | * @nr_to_scan: The number of pages to try to reclaim |
| 1638 | * @remaining: The number of pages left to reclaim, should be initialized by caller | 1655 | * @remaining: The number of pages left to reclaim, should be initialized by caller |
| @@ -1640,7 +1657,8 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail); · 1 unmodified line | |||
| 1640 | * @ticket: Optional ww_acquire_ctx context to use for locking | 1657 | * @ticket: Optional ww_acquire_ctx context to use for locking |
| 1641 | */ | 1658 | */ |
| 1642 | unsigned long | 1659 | unsigned long |
| 1643 | drm_gem_lru_scan(struct drm_gem_lru *lru, | 1660 | drm_gem_lru_scan(struct drm_device *dev, |
| 1661 | struct drm_gem_lru *lru, | ||
| 1644 | unsigned int nr_to_scan, | 1662 | unsigned int nr_to_scan, |
| 1645 | unsigned long *remaining, | 1663 | unsigned long *remaining, |
| 1646 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket), | 1664 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket), |
| @@ -1650,9 +1668,9 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 3 unmodified lines | |||
| 1650 | struct drm_gem_object *obj; | 1668 | struct drm_gem_object *obj; |
| 1651 | unsigned freed = 0; | 1669 | unsigned freed = 0; |
| 1652 | 1670 | ||
| 1653 | drm_gem_lru_init(&still_in_lru, lru->lock); | 1671 | drm_gem_lru_init(&still_in_lru); |
| 1654 | 1672 | ||
| 1655 | mutex_lock(lru->lock); | 1673 | mutex_lock(&dev->gem_lru_mutex); |
| 1656 | 1674 | ||
| 1657 | while (freed < nr_to_scan) { | 1675 | while (freed < nr_to_scan) { |
| 1658 | obj = list_first_entry_or_null(&lru->list, typeof(*obj), lru_node); | 1676 | obj = list_first_entry_or_null(&lru->list, typeof(*obj), lru_node); |
| @@ -1675,7 +1693,7 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 16 unmodified lines | |||
| 1675 | * rest of the loop body, to reduce contention with other | 1693 | * rest of the loop body, to reduce contention with other |
| 1676 | * code paths that need the LRU lock | 1694 | * code paths that need the LRU lock |
| 1677 | */ | 1695 | */ |
| 1678 | mutex_unlock(lru->lock); | 1696 | mutex_unlock(&dev->gem_lru_mutex); |
| 1679 | 1697 | ||
| 1680 | if (ticket) | 1698 | if (ticket) |
| 1681 | ww_acquire_init(ticket, &reservation_ww_class); | 1699 | ww_acquire_init(ticket, &reservation_ww_class); |
| @@ -1709,7 +1727,7 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 27 unmodified lines | |||
| 1709 | 1727 | ||
| 1710 | tail: | 1728 | tail: |
| 1711 | drm_gem_object_put(obj); | 1729 | drm_gem_object_put(obj); |
| 1712 | mutex_lock(lru->lock); | 1730 | mutex_lock(&dev->gem_lru_mutex); |
| 1713 | } | 1731 | } |
| 1714 | 1732 | ||
| 1715 | /* | 1733 | /* |
| @@ -1721,7 +1739,7 @@ tail: · 5 unmodified lines | |||
| 1721 | list_splice_tail(&still_in_lru.list, &lru->list); | 1739 | list_splice_tail(&still_in_lru.list, &lru->list); |
| 1722 | lru->count += still_in_lru.count; | 1740 | lru->count += still_in_lru.count; |
| 1723 | 1741 | ||
| 1724 | mutex_unlock(lru->lock); | 1742 | mutex_unlock(&dev->gem_lru_mutex); |
| 1725 | 1743 | ||
| 1726 | return freed; | 1744 | return freed; |
| 1727 | } | 1745 | } |
| @@ -172,8 +172,8 @@ int drm_gem_fb_init_with_funcs(struct drm_device *dev, · 171 unmodified lines | |||
| 172 | } | 172 | } |
| 173 | 173 | ||
| 174 | for (i = 0; i < info->num_planes; i++) { | 174 | for (i = 0; i < info->num_planes; i++) { |
| 175 | unsigned int width = mode_cmd->width / (i ? info->hsub : 1); | 175 | unsigned int width = drm_format_info_plane_width(info, mode_cmd->width, i); |
| 176 | unsigned int height = mode_cmd->height / (i ? info->vsub : 1); | 176 | unsigned int height = drm_format_info_plane_height(info, mode_cmd->height, i); |
| 177 | unsigned int min_size; | 177 | unsigned int min_size; |
| 178 | 178 | ||
| 179 | objs[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]); | 179 | objs[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]); |
| @@ -116,16 +116,18 @@ int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit) · 115 unmodified lines | |||
| 116 | */ | 116 | */ |
| 117 | mutex_lock(&gpu->sched_lock); | 117 | mutex_lock(&gpu->sched_lock); |
| 118 | 118 | ||
| 119 | ret = xa_alloc_cyclic(&gpu->user_fences, &submit->out_fence_id, | ||
| 120 | NULL, xa_limit_32b, &gpu->next_user_fence, | ||
| 121 | GFP_KERNEL); | ||
| 122 | if (ret < 0) | ||
| 123 | goto out_unlock; | ||
| 124 | |||
| 119 | drm_sched_job_arm(&submit->sched_job); | 125 | drm_sched_job_arm(&submit->sched_job); |
| 120 | 126 | ||
| 121 | submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished); | 127 | submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished); |
| 122 | ret = xa_alloc_cyclic(&gpu->user_fences, &submit->out_fence_id, | 128 | |
| 123 | submit->out_fence, xa_limit_32b, | 129 | xa_store(&gpu->user_fences, submit->out_fence_id, |
| 124 | &gpu->next_user_fence, GFP_KERNEL); | 130 | submit->out_fence, GFP_KERNEL); |
| 125 | if (ret < 0) { | ||
| 126 | drm_sched_job_cleanup(&submit->sched_job); | ||
| 127 | goto out_unlock; | ||
| 128 | } | ||
| 129 | 131 | ||
| 130 | /* the scheduler holds on to the job now */ | 132 | /* the scheduler holds on to the job now */ |
| 131 | kref_get(&submit->refcount); | 133 | kref_get(&submit->refcount); |
| @@ -423,7 +423,9 @@ static int exynos_mic_probe(struct platform_device *pdev) · 422 unmodified lines | |||
| 423 | 423 | ||
| 424 | mic->bridge.of_node = dev->of_node; | 424 | mic->bridge.of_node = dev->of_node; |
| 425 | 425 | ||
| 426 | drm_bridge_add(&mic->bridge); | 426 | ret = devm_drm_bridge_add(dev, &mic->bridge); |
| 427 | if (ret) | ||
| 428 | goto err; | ||
| 427 | 429 | ||
| 428 | pm_runtime_enable(dev); | 430 | pm_runtime_enable(dev); |
| 429 | 431 | ||
| @@ -443,12 +445,8 @@ err: · 13 unmodified lines | |||
| 443 | 445 | ||
| 444 | static void exynos_mic_remove(struct platform_device *pdev) | 446 | static void exynos_mic_remove(struct platform_device *pdev) |
| 445 | { | 447 | { |
| 446 | struct exynos_mic *mic = platform_get_drvdata(pdev); | ||
| 447 | |||
| 448 | component_del(&pdev->dev, &exynos_mic_component_ops); | 448 | component_del(&pdev->dev, &exynos_mic_component_ops); |
| 449 | pm_runtime_disable(&pdev->dev); | 449 | pm_runtime_disable(&pdev->dev); |
| 450 | |||
| 451 | drm_bridge_remove(&mic->bridge); | ||
| 452 | } | 450 | } |
| 453 | 451 | ||
| 454 | static const struct of_device_id exynos_mic_of_match[] = { | 452 | static const struct of_device_id exynos_mic_of_match[] = { |
| @@ -580,6 +580,7 @@ static int oaktrail_hdmi_get_modes(struct drm_connector *connector) · 579 unmodified lines | |||
| 580 | } else { | 580 | } else { |
| 581 | edid = (struct edid *)raw_edid; | 581 | edid = (struct edid *)raw_edid; |
| 582 | /* FIXME ? edid = drm_get_edid(connector, i2c_adap); */ | 582 | /* FIXME ? edid = drm_get_edid(connector, i2c_adap); */ |
| 583 | i2c_put_adapter(i2c_adap); | ||
| 583 | } | 584 | } |
| 584 | 585 | ||
| 585 | if (edid) { | 586 | if (edid) { |
| @@ -293,7 +293,7 @@ void oaktrail_lvds_init(struct drm_device *dev, · 292 unmodified lines | |||
| 293 | { | 293 | { |
| 294 | struct gma_encoder *gma_encoder; | 294 | struct gma_encoder *gma_encoder; |
| 295 | struct gma_connector *gma_connector; | 295 | struct gma_connector *gma_connector; |
| 296 | struct gma_i2c_chan *ddc_bus; | 296 | struct gma_i2c_chan *ddc_bus = NULL; |
| 297 | struct drm_connector *connector; | 297 | struct drm_connector *connector; |
| 298 | struct drm_encoder *encoder; | 298 | struct drm_encoder *encoder; |
| 299 | struct drm_psb_private *dev_priv = to_drm_psb_private(dev); | 299 | struct drm_psb_private *dev_priv = to_drm_psb_private(dev); |
| @@ -367,6 +367,8 @@ void oaktrail_lvds_init(struct drm_device *dev, · 67 unmodified lines | |||
| 367 | if (edid == NULL && dev_priv->lpc_gpio_base) { | 367 | if (edid == NULL && dev_priv->lpc_gpio_base) { |
| 368 | ddc_bus = oaktrail_lvds_i2c_init(dev); | 368 | ddc_bus = oaktrail_lvds_i2c_init(dev); |
| 369 | if (!IS_ERR(ddc_bus)) { | 369 | if (!IS_ERR(ddc_bus)) { |
| 370 | if (i2c_adap) | ||
| 371 | i2c_put_adapter(i2c_adap); | ||
| 370 | i2c_adap = &ddc_bus->base; | 372 | i2c_adap = &ddc_bus->base; |
| 371 | edid = drm_get_edid(connector, i2c_adap); | 373 | edid = drm_get_edid(connector, i2c_adap); |
| 372 | } | 374 | } |
| @@ -421,7 +423,10 @@ out: · 48 unmodified lines | |||
| 421 | 423 | ||
| 422 | err_unlock: | 424 | err_unlock: |
| 423 | mutex_unlock(&dev->mode_config.mutex); | 425 | mutex_unlock(&dev->mode_config.mutex); |
| 424 | gma_i2c_destroy(to_gma_i2c_chan(connector->ddc)); | 426 | if (!IS_ERR_OR_NULL(ddc_bus)) |
| 427 | gma_i2c_destroy(ddc_bus); | ||
| 428 | else if (i2c_adap) | ||
| 429 | i2c_put_adapter(i2c_adap); | ||
| 425 | drm_encoder_cleanup(encoder); | 430 | drm_encoder_cleanup(encoder); |
| 426 | err_connector_cleanup: | 431 | err_connector_cleanup: |
| 427 | drm_connector_cleanup(connector); | 432 | drm_connector_cleanup(connector); |
| @@ -391,8 +391,11 @@ static int hyperv_get_supported_resolution(struct hv_device *hdev) · 390 unmodified lines | |||
| 391 | return -ETIMEDOUT; | 391 | return -ETIMEDOUT; |
| 392 | } | 392 | } |
| 393 | 393 | ||
| 394 | if (msg->resolution_resp.resolution_count == 0) { | 394 | if (msg->resolution_resp.resolution_count == 0 || |
| 395 | drm_err(dev, "No supported resolutions\n"); | 395 | msg->resolution_resp.resolution_count > |
| 396 | SYNTHVID_MAX_RESOLUTION_COUNT) { | ||
| 397 | drm_err(dev, "Invalid resolution count: %d\n", | ||
| 398 | msg->resolution_resp.resolution_count); | ||
| 396 | return -ENODEV; | 399 | return -ENODEV; |
| 397 | } | 400 | } |
| 398 | 401 | ||
| @@ -417,30 +420,92 @@ static int hyperv_get_supported_resolution(struct hv_device *hdev) · 18 unmodified lines | |||
| 417 | return 0; | 420 | return 0; |
| 418 | } | 421 | } |
| 419 | 422 | ||
| 420 | static void hyperv_receive_sub(struct hv_device *hdev) | 423 | static void hyperv_receive_sub(struct hv_device *hdev, u32 bytes_recvd) |
| 421 | { | 424 | { |
| 422 | struct hyperv_drm_device *hv = hv_get_drvdata(hdev); | 425 | struct hyperv_drm_device *hv = hv_get_drvdata(hdev); |
| 423 | struct synthvid_msg *msg; | 426 | struct synthvid_msg *msg; |
| 427 | size_t hdr_size; | ||
| 428 | size_t need; | ||
| 424 | 429 | ||
| 425 | if (!hv) | 430 | if (!hv) |
| 426 | return; | 431 | return; |
| 427 | 432 | ||
| 428 | msg = (struct synthvid_msg *)hv->recv_buf; | 433 | hdr_size = sizeof(struct pipe_msg_hdr) + |
| 429 | 434 | sizeof(struct synthvid_msg_hdr); | |
| 430 | /* Complete the wait event */ | 435 | if (bytes_recvd < hdr_size) { |
| 431 | if (msg->vid_hdr.type == SYNTHVID_VERSION_RESPONSE || | 436 | drm_err_ratelimited(&hv->dev, |
| 432 | msg->vid_hdr.type == SYNTHVID_RESOLUTION_RESPONSE || | 437 | "synthvid packet too small for header: %u\n", |
| 433 | msg->vid_hdr.type == SYNTHVID_VRAM_LOCATION_ACK) { | 438 | bytes_recvd); |
| 434 | memcpy(hv->init_buf, msg, VMBUS_MAX_PACKET_SIZE); | ||
| 435 | complete(&hv->wait); | ||
| 436 | return; | 439 | return; |
| 437 | } | 440 | } |
| 438 | 441 | ||
| 439 | if (msg->vid_hdr.type == SYNTHVID_FEATURE_CHANGE) { | 442 | msg = (struct synthvid_msg *)hv->recv_buf; |
| 443 | need = hdr_size; | ||
| 444 | |||
| 445 | switch (msg->vid_hdr.type) { | ||
| 446 | case SYNTHVID_VERSION_RESPONSE: | ||
| 447 | need += sizeof(struct synthvid_version_resp); | ||
| 448 | break; | ||
| 449 | case SYNTHVID_RESOLUTION_RESPONSE: | ||
| 450 | /* | ||
| 451 | * The resolution response is variable length: the host | ||
| 452 | * fills resolution_count entries, not the full | ||
| 453 | * SYNTHVID_MAX_RESOLUTION_COUNT array. Require the fixed | ||
| 454 | * prefix first so resolution_count can be read, then | ||
| 455 | * demand exactly the count-sized array. | ||
| 456 | */ | ||
| 457 | need += offsetof(struct synthvid_supported_resolution_resp, | ||
| 458 | supported_resolution); | ||
| 459 | if (bytes_recvd < need) | ||
| 460 | break; | ||
| 461 | if (msg->resolution_resp.resolution_count > | ||
| 462 | SYNTHVID_MAX_RESOLUTION_COUNT) { | ||
| 463 | drm_err_ratelimited(&hv->dev, | ||
| 464 | "synthvid resolution count too large: %u\n", | ||
| 465 | msg->resolution_resp.resolution_count); | ||
| 466 | return; | ||
| 467 | } | ||
| 468 | need += msg->resolution_resp.resolution_count * | ||
| 469 | sizeof(struct hvd_screen_info); | ||
| 470 | break; | ||
| 471 | case SYNTHVID_VRAM_LOCATION_ACK: | ||
| 472 | need += sizeof(struct synthvid_vram_location_ack); | ||
| 473 | break; | ||
| 474 | case SYNTHVID_FEATURE_CHANGE: | ||
| 475 | /* | ||
| 476 | * Not a completion-driving message: validate its own payload | ||
| 477 | * and consume it here rather than falling through to the | ||
| 478 | * memcpy/complete shared by the wait-event responses. | ||
| 479 | */ | ||
| 480 | if (bytes_recvd < need + | ||
| 481 | sizeof(struct synthvid_feature_change)) { | ||
| 482 | drm_err_ratelimited(&hv->dev, | ||
| 483 | "synthvid feature change packet too small: %u\n", | ||
| 484 | bytes_recvd); | ||
| 485 | return; | ||
| 486 | } | ||
| 440 | hv->dirt_needed = msg->feature_chg.is_dirt_needed; | 487 | hv->dirt_needed = msg->feature_chg.is_dirt_needed; |
| 441 | if (hv->dirt_needed) | 488 | if (hv->dirt_needed) |
| 442 | hyperv_hide_hw_ptr(hv->hdev); | 489 | hyperv_hide_hw_ptr(hv->hdev); |
| 490 | return; | ||
| 491 | default: | ||
| 492 | return; | ||
| 493 | } | ||
| 494 | |||
| 495 | /* | ||
| 496 | * Shared completion path for the wait-event responses | ||
| 497 | * (VERSION_RESPONSE, RESOLUTION_RESPONSE, VRAM_LOCATION_ACK): | ||
| 498 | * require the type-specific payload before handing the buffer to | ||
| 499 | * the waiter. | ||
| 500 | */ | ||
| 501 | if (bytes_recvd < need) { | ||
| 502 | drm_err_ratelimited(&hv->dev, | ||
| 503 | "synthvid packet too small for type %u: %u < %zu\n", | ||
| 504 | msg->vid_hdr.type, bytes_recvd, need); | ||
| 505 | return; | ||
| 443 | } | 506 | } |
| 507 | memcpy(hv->init_buf, msg, bytes_recvd); | ||
| 508 | complete(&hv->wait); | ||
| 444 | } | 509 | } |
| 445 | 510 | ||
| 446 | static void hyperv_receive(void *ctx) | 511 | static void hyperv_receive(void *ctx) |
| @@ -461,9 +526,21 @@ static void hyperv_receive(void *ctx) · 14 unmodified lines | |||
| 461 | ret = vmbus_recvpacket(hdev->channel, recv_buf, | 526 | ret = vmbus_recvpacket(hdev->channel, recv_buf, |
| 462 | VMBUS_MAX_PACKET_SIZE, | 527 | VMBUS_MAX_PACKET_SIZE, |
| 463 | &bytes_recvd, &req_id); | 528 | &bytes_recvd, &req_id); |
| 464 | if (bytes_recvd > 0 && | 529 | if (ret) { |
| 465 | recv_buf->pipe_hdr.type == PIPE_MSG_DATA) | 530 | /* |
| 466 | hyperv_receive_sub(hdev); | 531 | * A nonzero return (e.g. -ENOBUFS for an oversized |
| 532 | * packet) is itself a malformed message: bytes_recvd | ||
| 533 | * then reports the required length rather than a copied | ||
| 534 | * payload, so it must not be forwarded to the | ||
| 535 | * sub-handler. Channel recovery is not attempted. | ||
| 536 | */ | ||
| 537 | drm_err_ratelimited(&hv->dev, | ||
| 538 | "vmbus_recvpacket failed: %d (need %u)\n", | ||
| 539 | ret, bytes_recvd); | ||
| 540 | } else if (bytes_recvd > 0 && | ||
| 541 | recv_buf->pipe_hdr.type == PIPE_MSG_DATA) { | ||
| 542 | hyperv_receive_sub(hdev, bytes_recvd); | ||
| 543 | } | ||
| 467 | } while (bytes_recvd > 0 && ret == 0); | 544 | } while (bytes_recvd > 0 && ret == 0); |
| 468 | } | 545 | } |
| 469 | 546 | ||
| @@ -508,9 +585,13 @@ int hyperv_connect_vsp(struct hv_device *hdev) · 38 unmodified lines | |||
| 508 | ret = hyperv_get_supported_resolution(hdev); | 585 | ret = hyperv_get_supported_resolution(hdev); |
| 509 | if (ret) | 586 | if (ret) |
| 510 | drm_err(dev, "Failed to get supported resolution from host, use default\n"); | 587 | drm_err(dev, "Failed to get supported resolution from host, use default\n"); |
| 511 | } else { | 588 | } |
| 589 | |||
| 590 | if (!hv->screen_width_max) { | ||
| 512 | hv->screen_width_max = SYNTHVID_WIDTH_WIN8; | 591 | hv->screen_width_max = SYNTHVID_WIDTH_WIN8; |
| 513 | hv->screen_height_max = SYNTHVID_HEIGHT_WIN8; | 592 | hv->screen_height_max = SYNTHVID_HEIGHT_WIN8; |
| 593 | hv->preferred_width = SYNTHVID_WIDTH_WIN8; | ||
| 594 | hv->preferred_height = SYNTHVID_HEIGHT_WIN8; | ||
| 514 | } | 595 | } |
| 515 | 596 | ||
| 516 | hv->mmio_megabytes = hdev->channel->offermsg.offer.mmio_megabytes; | 597 | hv->mmio_megabytes = hdev->channel->offermsg.offer.mmio_megabytes; |
| @@ -3976,7 +3976,7 @@ xelpd_program_plane_pre_csc_lut(struct intel_dsb *dsb, · 3975 unmodified lines | |||
| 3976 | intel_de_write_dsb(display, dsb, | 3976 | intel_de_write_dsb(display, dsb, |
| 3977 | PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0), | 3977 | PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0), |
| 3978 | (1 << 24)); | 3978 | (1 << 24)); |
| 3979 | } while (i++ > 130); | 3979 | } while (i++ < 130); |
| 3980 | } else { | 3980 | } else { |
| 3981 | for (i = 0; i < lut_size; i++) { | 3981 | for (i = 0; i < lut_size; i++) { |
| 3982 | u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1); | 3982 | u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1); |
| @@ -497,6 +497,7 @@ struct intel_display { · 496 unmodified lines | |||
| 497 | u8 vblank_enabled; | 497 | u8 vblank_enabled; |
| 498 | 498 | ||
| 499 | int vblank_enable_count; | 499 | int vblank_enable_count; |
| 500 | bool vblank_status_last_notified; | ||
| 500 | 501 | ||
| 501 | struct work_struct vblank_notify_work; | 502 | struct work_struct vblank_notify_work; |
| 502 | 503 | ||
| @@ -1773,8 +1773,12 @@ static void intel_display_vblank_notify_work(struct work_struct *work) · 1772 unmodified lines | |||
| 1773 | struct intel_display *display = | 1773 | struct intel_display *display = |
| 1774 | container_of(work, typeof(*display), irq.vblank_notify_work); | 1774 | container_of(work, typeof(*display), irq.vblank_notify_work); |
| 1775 | int vblank_enable_count = READ_ONCE(display->irq.vblank_enable_count); | 1775 | int vblank_enable_count = READ_ONCE(display->irq.vblank_enable_count); |
| 1776 | bool vblank_status = !!vblank_enable_count; | ||
| 1776 | 1777 | ||
| 1777 | intel_psr_notify_vblank_enable_disable(display, vblank_enable_count); | 1778 | if (display->irq.vblank_status_last_notified != vblank_status) { |
| 1779 | intel_psr_notify_vblank_enable_disable(display, vblank_status); | ||
| 1780 | display->irq.vblank_status_last_notified = vblank_status; | ||
| 1781 | } | ||
| 1778 | } | 1782 | } |
| 1779 | 1783 | ||
| 1780 | int bdw_enable_vblank(struct drm_crtc *_crtc) | 1784 | int bdw_enable_vblank(struct drm_crtc *_crtc) |
| @@ -1787,10 +1791,10 @@ int bdw_enable_vblank(struct drm_crtc *_crtc) · 6 unmodified lines | |||
| 1787 | if (gen11_dsi_configure_te(crtc, true)) | 1791 | if (gen11_dsi_configure_te(crtc, true)) |
| 1788 | return 0; | 1792 | return 0; |
| 1789 | 1793 | ||
| 1794 | spin_lock_irqsave(&display->irq.lock, irqflags); | ||
| 1790 | if (crtc->vblank_psr_notify && display->irq.vblank_enable_count++ == 0) | 1795 | if (crtc->vblank_psr_notify && display->irq.vblank_enable_count++ == 0) |
| 1791 | schedule_work(&display->irq.vblank_notify_work); | 1796 | schedule_work(&display->irq.vblank_notify_work); |
| 1792 | 1797 | ||
| 1793 | spin_lock_irqsave(&display->irq.lock, irqflags); | ||
| 1794 | bdw_enable_pipe_irq(display, pipe, GEN8_PIPE_VBLANK); | 1798 | bdw_enable_pipe_irq(display, pipe, GEN8_PIPE_VBLANK); |
| 1795 | spin_unlock_irqrestore(&display->irq.lock, irqflags); | 1799 | spin_unlock_irqrestore(&display->irq.lock, irqflags); |
| 1796 | 1800 | ||
| @@ -584,6 +584,7 @@ struct intel_connector { · 583 unmodified lines | |||
| 584 | 584 | ||
| 585 | struct { | 585 | struct { |
| 586 | u8 dpcd[EDP_PSR_RECEIVER_CAP_SIZE]; | 586 | u8 dpcd[EDP_PSR_RECEIVER_CAP_SIZE]; |
| 587 | u8 intel_wa_dpcd; | ||
| 587 | 588 | ||
| 588 | bool support; | 589 | bool support; |
| 589 | bool su_support; | 590 | bool su_support; |
| @@ -1789,6 +1790,8 @@ struct intel_psr { · 1199 unmodified lines | |||
| 1789 | u8 active_non_psr_pipes; | 1790 | u8 active_non_psr_pipes; |
| 1790 | 1791 | ||
| 1791 | const char *no_psr_reason; | 1792 | const char *no_psr_reason; |
| 1793 | |||
| 1794 | struct ref_tracker *vblank_wakeref; | ||
| 1792 | }; | 1795 | }; |
| 1793 | 1796 | ||
| 1794 | struct intel_dp { | 1797 | struct intel_dp { |
| @@ -3119,8 +3119,13 @@ static void intel_dp_compute_vsc_colorimetry(const struct intel_crtc_state *crtc · 3118 unmodified lines | |||
| 3119 | drm_WARN_ON(display->drm, | 3119 | drm_WARN_ON(display->drm, |
| 3120 | vsc->bpc == 6 && vsc->pixelformat != DP_PIXELFORMAT_RGB); | 3120 | vsc->bpc == 6 && vsc->pixelformat != DP_PIXELFORMAT_RGB); |
| 3121 | 3121 | ||
| 3122 | /* all YCbCr are always limited range */ | 3122 | /* All YCbCr formats are always limited range. */ |
| 3123 | vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA; | 3123 | if (vsc->pixelformat == DP_PIXELFORMAT_RGB) |
| 3124 | vsc->dynamic_range = crtc_state->limited_color_range ? | ||
| 3125 | DP_DYNAMIC_RANGE_CTA : DP_DYNAMIC_RANGE_VESA; | ||
| 3126 | else | ||
| 3127 | vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA; | ||
| 3128 | |||
| 3124 | vsc->content_type = DP_CONTENT_TYPE_NOT_DEFINED; | 3129 | vsc->content_type = DP_CONTENT_TYPE_NOT_DEFINED; |
| 3125 | } | 3130 | } |
| 3126 | 3131 | ||
| @@ -5298,7 +5303,7 @@ int intel_dp_as_sdp_unpack(struct drm_dp_as_sdp *as_sdp, · 2171 unmodified lines | |||
| 5298 | as_sdp->length = sdp->sdp_header.HB3 & DP_ADAPTIVE_SYNC_SDP_LENGTH; | 5303 | as_sdp->length = sdp->sdp_header.HB3 & DP_ADAPTIVE_SYNC_SDP_LENGTH; |
| 5299 | as_sdp->mode = sdp->db[0] & DP_ADAPTIVE_SYNC_SDP_OPERATION_MODE; | 5304 | as_sdp->mode = sdp->db[0] & DP_ADAPTIVE_SYNC_SDP_OPERATION_MODE; |
| 5300 | as_sdp->vtotal = (sdp->db[2] << 8) | sdp->db[1]; | 5305 | as_sdp->vtotal = (sdp->db[2] << 8) | sdp->db[1]; |
| 5301 | as_sdp->target_rr = (u64)sdp->db[3] | ((u64)sdp->db[4] & 0x3); | 5306 | as_sdp->target_rr = ((sdp->db[4] & 0x3) << 8) | sdp->db[3]; |
| 5302 | as_sdp->target_rr_divider = sdp->db[4] & 0x20 ? true : false; | 5307 | as_sdp->target_rr_divider = sdp->db[4] & 0x20 ? true : false; |
| 5303 | 5308 | ||
| 5304 | return 0; | 5309 | return 0; |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | |||
| 12 | #include "intel_dp.h" | 12 | #include "intel_dp.h" |
| 13 | #include "intel_dp_aux.h" | 13 | #include "intel_dp_aux.h" |
| 14 | #include "intel_dp_aux_regs.h" | 14 | #include "intel_dp_aux_regs.h" |
| 15 | #include "intel_parent.h" | ||
| 15 | #include "intel_pps.h" | 16 | #include "intel_pps.h" |
| 16 | #include "intel_quirks.h" | 17 | #include "intel_quirks.h" |
| 17 | #include "intel_tc.h" | 18 | #include "intel_tc.h" |
| @@ -60,18 +61,29 @@ intel_dp_aux_wait_done(struct intel_dp *intel_dp) · 42 unmodified lines | |||
| 60 | struct intel_display *display = to_intel_display(intel_dp); | 61 | struct intel_display *display = to_intel_display(intel_dp); |
| 61 | i915_reg_t ch_ctl = intel_dp->aux_ch_ctl_reg(intel_dp); | 62 | i915_reg_t ch_ctl = intel_dp->aux_ch_ctl_reg(intel_dp); |
| 62 | const unsigned int timeout_ms = 10; | 63 | const unsigned int timeout_ms = 10; |
| 64 | bool done = true; | ||
| 63 | u32 status; | 65 | u32 status; |
| 64 | bool done; | 66 | int ret; |
| 65 | 67 | ||
| 68 | if (intel_parent_irq_enabled(display)) { | ||
| 66 | #define C (((status = intel_de_read_notrace(display, ch_ctl)) & DP_AUX_CH_CTL_SEND_BUSY) == 0) | 69 | #define C (((status = intel_de_read_notrace(display, ch_ctl)) & DP_AUX_CH_CTL_SEND_BUSY) == 0) |
| 67 | done = wait_event_timeout(display->gmbus.wait_queue, C, | 70 | done = wait_event_timeout(display->gmbus.wait_queue, C, |
| 68 | msecs_to_jiffies_timeout(timeout_ms)); | 71 | msecs_to_jiffies_timeout(timeout_ms)); |
| 72 | |||
| 73 | #undef C | ||
| 74 | } else { | ||
| 75 | ret = intel_de_wait_ms(display, ch_ctl, | ||
| 76 | DP_AUX_CH_CTL_SEND_BUSY, 0, | ||
| 77 | timeout_ms, &status); | ||
| 78 | |||
| 79 | if (ret == -ETIMEDOUT) | ||
| 80 | done = false; | ||
| 81 | } | ||
| 69 | 82 | ||
| 70 | if (!done) | 83 | if (!done) |
| 71 | drm_err(display->drm, | 84 | drm_err(display->drm, |
| 72 | "%s: did not complete or timeout within %ums (status 0x%08x)\n", | 85 | "%s: did not complete or timeout within %ums (status 0x%08x)\n", |
| 73 | intel_dp->aux.name, timeout_ms, status); | 86 | intel_dp->aux.name, timeout_ms, status); |
| 74 | #undef C | ||
| 75 | 87 | ||
| 76 | return status; | 88 | return status; |
| 77 | } | 89 | } |
| @@ -0,0 +1,15 @@ | |||
| 1 | /* SPDX-License-Identifier: MIT */ | ||
| 2 | /* | ||
| 3 | * Copyright © 2026 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #ifndef __INTEL_DPCD_H__ | ||
| 7 | #define __INTEL_DPCD_H__ | ||
| 8 | |||
| 9 | #define INTEL_DPCD_INTEL_WA_REGISTER_CAPS 0x3f0 | ||
| 10 | # define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_EARLYSCANLINE_SDP_SUPPORT_MASK REG_GENMASK(1, 0) | ||
| 11 | # define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_FALL_BACK_TO_PSR1 0 | ||
| 12 | # define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITH_EARLY_SCANLINE 1 | ||
| 13 | # define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITHOUT_EARLY_SCANLINE 2 | ||
| 14 | |||
| 15 | #endif /* __INTEL_DPCD_H__ */ | ||
| @@ -373,7 +373,7 @@ intel_plane_color_copy_uapi_to_hw_state(struct intel_plane_state *plane_state, · 372 unmodified lines | |||
| 373 | bool changed = false; | 373 | bool changed = false; |
| 374 | int i = 0; | 374 | int i = 0; |
| 375 | 375 | ||
| 376 | iter_colorop = plane_state->uapi.color_pipeline; | 376 | iter_colorop = from_plane_state->uapi.color_pipeline; |
| 377 | 377 | ||
| 378 | while (iter_colorop) { | 378 | while (iter_colorop) { |
| 379 | for_each_new_colorop_in_state(state, colorop, new_colorop_state, i) { | 379 | for_each_new_colorop_in_state(state, colorop, new_colorop_state, i) { |
| @@ -43,6 +43,7 @@ · 42 unmodified lines | |||
| 43 | #include "intel_display_wa.h" | 43 | #include "intel_display_wa.h" |
| 44 | #include "intel_dmc.h" | 44 | #include "intel_dmc.h" |
| 45 | #include "intel_dp.h" | 45 | #include "intel_dp.h" |
| 46 | #include "intel_dpcd.h" | ||
| 46 | #include "intel_dp_aux.h" | 47 | #include "intel_dp_aux.h" |
| 47 | #include "intel_dsb.h" | 48 | #include "intel_dsb.h" |
| 48 | #include "intel_frontbuffer.h" | 49 | #include "intel_frontbuffer.h" |
| @@ -716,8 +717,14 @@ static void _psr_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *co · 667 unmodified lines | |||
| 716 | connector->dp.psr_caps.su_support ? "" : "not "); | 717 | connector->dp.psr_caps.su_support ? "" : "not "); |
| 717 | } | 718 | } |
| 718 | 719 | ||
| 719 | if (connector->dp.psr_caps.su_support) | 720 | if (connector->dp.psr_caps.su_support) { |
| 721 | ret = drm_dp_dpcd_read_byte(&intel_dp->aux, | ||
| 722 | INTEL_DPCD_INTEL_WA_REGISTER_CAPS, | ||
| 723 | &connector->dp.psr_caps.intel_wa_dpcd); | ||
| 724 | if (ret < 0) | ||
| 725 | return; | ||
| 720 | _psr_compute_su_granularity(intel_dp, connector); | 726 | _psr_compute_su_granularity(intel_dp, connector); |
| 727 | } | ||
| 721 | } | 728 | } |
| 722 | 729 | ||
| 723 | void intel_psr_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) | 730 | void intel_psr_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) |
| @@ -1358,9 +1365,35 @@ static bool psr2_granularity_check(struct intel_crtc_state *crtc_state, · 634 unmodified lines | |||
| 1358 | return true; | 1365 | return true; |
| 1359 | } | 1366 | } |
| 1360 | 1367 | ||
| 1361 | static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_dp *intel_dp, | 1368 | static bool apply_scanline_indication_wa(struct intel_crtc_state *crtc_state, |
| 1362 | struct intel_crtc_state *crtc_state) | 1369 | struct intel_connector *connector) |
| 1363 | { | 1370 | { |
| 1371 | struct intel_dp *intel_dp = intel_attached_dp(connector); | ||
| 1372 | u8 early_scanline_support = connector->dp.psr_caps.intel_wa_dpcd & | ||
| 1373 | INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_EARLYSCANLINE_SDP_SUPPORT_MASK; | ||
| 1374 | |||
| 1375 | if (intel_dp->edp_dpcd[0] >= DP_EDP_15) | ||
| 1376 | return true; | ||
| 1377 | |||
| 1378 | switch (early_scanline_support) { | ||
| 1379 | case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_FALL_BACK_TO_PSR1: | ||
| 1380 | crtc_state->req_psr2_sdp_prior_scanline = false; | ||
| 1381 | return false; | ||
| 1382 | case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITH_EARLY_SCANLINE: | ||
| 1383 | return true; | ||
| 1384 | case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITHOUT_EARLY_SCANLINE: | ||
| 1385 | crtc_state->req_psr2_sdp_prior_scanline = false; | ||
| 1386 | return true; | ||
| 1387 | default: | ||
| 1388 | MISSING_CASE(early_scanline_support); | ||
| 1389 | return false; | ||
| 1390 | } | ||
| 1391 | } | ||
| 1392 | |||
| 1393 | static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_crtc_state *crtc_state, | ||
| 1394 | struct intel_connector *connector) | ||
| 1395 | { | ||
| 1396 | struct intel_dp *intel_dp = intel_attached_dp(connector); | ||
| 1364 | struct intel_display *display = to_intel_display(intel_dp); | 1397 | struct intel_display *display = to_intel_display(intel_dp); |
| 1365 | const struct drm_display_mode *adjusted_mode = &crtc_state->uapi.adjusted_mode; | 1398 | const struct drm_display_mode *adjusted_mode = &crtc_state->uapi.adjusted_mode; |
| 1366 | u32 hblank_total, hblank_ns, req_ns; | 1399 | u32 hblank_total, hblank_ns, req_ns; |
| @@ -1379,7 +1412,8 @@ static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_dp *intel_d · 12 unmodified lines | |||
| 1379 | return false; | 1412 | return false; |
| 1380 | 1413 | ||
| 1381 | crtc_state->req_psr2_sdp_prior_scanline = true; | 1414 | crtc_state->req_psr2_sdp_prior_scanline = true; |
| 1382 | return true; | 1415 | |
| 1416 | return apply_scanline_indication_wa(crtc_state, connector); | ||
| 1383 | } | 1417 | } |
| 1384 | 1418 | ||
| 1385 | static int intel_psr_entry_setup_frames(struct intel_dp *intel_dp, | 1419 | static int intel_psr_entry_setup_frames(struct intel_dp *intel_dp, |
| @@ -1660,7 +1694,7 @@ static bool intel_sel_update_config_valid(struct intel_crtc_state *crtc_state, · 274 unmodified lines | |||
| 1660 | conn_state)) | 1694 | conn_state)) |
| 1661 | goto unsupported; | 1695 | goto unsupported; |
| 1662 | 1696 | ||
| 1663 | if (!_compute_psr2_sdp_prior_scanline_indication(intel_dp, crtc_state)) { | 1697 | if (!_compute_psr2_sdp_prior_scanline_indication(crtc_state, connector)) { |
| 1664 | drm_dbg_kms(display->drm, | 1698 | drm_dbg_kms(display->drm, |
| 1665 | "Selective update not enabled, SDP indication do not fit in hblank\n"); | 1699 | "Selective update not enabled, SDP indication do not fit in hblank\n"); |
| 1666 | goto unsupported; | 1700 | goto unsupported; |
| @@ -4122,27 +4156,22 @@ void intel_psr_notify_vblank_enable_disable(struct intel_display *display, · 2455 unmodified lines | |||
| 4122 | struct intel_dp *intel_dp = enc_to_intel_dp(encoder); | 4156 | struct intel_dp *intel_dp = enc_to_intel_dp(encoder); |
| 4123 | 4157 | ||
| 4124 | mutex_lock(&intel_dp->psr.lock); | 4158 | mutex_lock(&intel_dp->psr.lock); |
| 4125 | if (intel_dp->psr.panel_replay_enabled) { | 4159 | if (CAN_PANEL_REPLAY(intel_dp)) { |
| 4126 | mutex_unlock(&intel_dp->psr.lock); | 4160 | if (enable) |
| 4127 | break; | 4161 | intel_dp->psr.vblank_wakeref = |
| 4162 | intel_display_power_get(display, | ||
| 4163 | POWER_DOMAIN_DC_OFF); | ||
| 4164 | else | ||
| 4165 | intel_display_power_put(display, POWER_DOMAIN_DC_OFF, | ||
| 4166 | intel_dp->psr.vblank_wakeref); | ||
| 4128 | } | 4167 | } |
| 4129 | 4168 | ||
| 4130 | if (intel_dp->psr.enabled && intel_dp->psr.pkg_c_latency_used) | 4169 | if (intel_dp->psr.enabled && !intel_dp->psr.panel_replay_enabled && |
| 4170 | intel_dp->psr.pkg_c_latency_used) | ||
| 4131 | intel_psr_apply_underrun_on_idle_wa_locked(intel_dp); | 4171 | intel_psr_apply_underrun_on_idle_wa_locked(intel_dp); |
| 4132 | 4172 | ||
| 4133 | mutex_unlock(&intel_dp->psr.lock); | 4173 | mutex_unlock(&intel_dp->psr.lock); |
| 4134 | return; | ||
| 4135 | } | 4174 | } |
| 4136 | |||
| 4137 | /* | ||
| 4138 | * NOTE: intel_display_power_set_target_dc_state is used | ||
| 4139 | * only by PSR * code for DC3CO handling. DC3CO target | ||
| 4140 | * state is currently disabled in * PSR code. If DC3CO | ||
| 4141 | * is taken into use we need take that into account here | ||
| 4142 | * as well. | ||
| 4143 | */ | ||
| 4144 | intel_display_power_set_target_dc_state(display, enable ? DC_STATE_DISABLE : | ||
| 4145 | DC_STATE_EN_UPTO_DC6); | ||
| 4146 | } | 4175 | } |
| 4147 | 4176 | ||
| 4148 | static void | 4177 | static void |
| @@ -419,8 +419,6 @@ void i915_ttm_free_cached_io_rsgt(struct drm_i915_gem_object *obj) · 418 unmodified lines | |||
| 419 | int i915_ttm_purge(struct drm_i915_gem_object *obj) | 419 | int i915_ttm_purge(struct drm_i915_gem_object *obj) |
| 420 | { | 420 | { |
| 421 | struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); | 421 | struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); |
| 422 | struct i915_ttm_tt *i915_tt = | ||
| 423 | container_of(bo->ttm, typeof(*i915_tt), ttm); | ||
| 424 | struct ttm_operation_ctx ctx = { | 422 | struct ttm_operation_ctx ctx = { |
| 425 | .interruptible = true, | 423 | .interruptible = true, |
| 426 | .no_wait_gpu = false, | 424 | .no_wait_gpu = false, |
| @@ -435,16 +433,22 @@ int i915_ttm_purge(struct drm_i915_gem_object *obj) · 8 unmodified lines | |||
| 435 | if (ret) | 433 | if (ret) |
| 436 | return ret; | 434 | return ret; |
| 437 | 435 | ||
| 438 | if (bo->ttm && i915_tt->filp) { | 436 | if (bo->ttm) { |
| 439 | /* | 437 | struct i915_ttm_tt *i915_tt = |
| 440 | * The below fput(which eventually calls shmem_truncate) might | 438 | container_of(bo->ttm, typeof(*i915_tt), ttm); |
| 441 | * be delayed by worker, so when directly called to purge the | 439 | |
| 442 | * pages(like by the shrinker) we should try to be more | 440 | if (i915_tt->filp) { |
| 443 | * aggressive and release the pages immediately. | 441 | /* |
| 444 | */ | 442 | * The below fput(which eventually calls shmem_truncate) |
| 445 | shmem_truncate_range(file_inode(i915_tt->filp), | 443 | * might be delayed by worker, so when directly called |
| 446 | 0, (loff_t)-1); | 444 | * to purge the pages(like by the shrinker) we should |
| 447 | fput(fetch_and_zero(&i915_tt->filp)); | 445 | * try to be more aggressive and release the pages |
| 446 | * immediately. | ||
| 447 | */ | ||
| 448 | shmem_truncate_range(file_inode(i915_tt->filp), | ||
| 449 | 0, (loff_t)-1); | ||
| 450 | fput(fetch_and_zero(&i915_tt->filp)); | ||
| 451 | } | ||
| 448 | } | 452 | } |
| 449 | 453 | ||
| 450 | obj->write_domain = 0; | 454 | obj->write_domain = 0; |
| @@ -132,7 +132,8 @@ void __i915_request_reset(struct i915_request *rq, bool guilty) · 131 unmodified lines | |||
| 132 | rcu_read_lock(); /* protect the GEM context */ | 132 | rcu_read_lock(); /* protect the GEM context */ |
| 133 | if (guilty) { | 133 | if (guilty) { |
| 134 | i915_request_set_error_once(rq, -EIO); | 134 | i915_request_set_error_once(rq, -EIO); |
| 135 | __i915_request_skip(rq); | 135 | if (!i915_request_signaled(rq)) |
| 136 | __i915_request_skip(rq); | ||
| 136 | banned = mark_guilty(rq); | 137 | banned = mark_guilty(rq); |
| 137 | } else { | 138 | } else { |
| 138 | i915_request_set_error_once(rq, -EAGAIN); | 139 | i915_request_set_error_once(rq, -EAGAIN); |
| @@ -750,9 +750,8 @@ static bool has_auxccs(struct drm_device *drm) · 749 unmodified lines | |||
| 750 | { | 750 | { |
| 751 | struct drm_i915_private *i915 = to_i915(drm); | 751 | struct drm_i915_private *i915 = to_i915(drm); |
| 752 | 752 | ||
| 753 | return IS_GRAPHICS_VER(i915, 9, 12) || | 753 | return IS_GRAPHICS_VER(i915, 9, 12) && |
| 754 | IS_ALDERLAKE_P(i915) || | 754 | !HAS_FLAT_CCS(i915); |
| 755 | IS_METEORLAKE(i915); | ||
| 756 | } | 755 | } |
| 757 | 756 | ||
| 758 | static bool has_fenced_regions(struct drm_device *drm) | 757 | static bool has_fenced_regions(struct drm_device *drm) |
| @@ -558,6 +558,6 @@ pvr_fw_trace_debugfs_init(struct pvr_device *pvr_dev, struct dentry *dir) · 557 unmodified lines | |||
| 558 | &pvr_fw_trace_fops); | 558 | &pvr_fw_trace_fops); |
| 559 | } | 559 | } |
| 560 | 560 | ||
| 561 | debugfs_create_file("trace_mask", 0600, dir, fw_trace, | 561 | debugfs_create_file("trace_mask", 0600, dir, pvr_dev, |
| 562 | &pvr_fw_trace_mask_fops); | 562 | &pvr_fw_trace_mask_fops); |
| 563 | } | 563 | } |
| @@ -293,7 +293,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 292 unmodified lines | |||
| 293 | 293 | ||
| 294 | vga_client_register(pdev, lsdc_vga_set_decode); | 294 | vga_client_register(pdev, lsdc_vga_set_decode); |
| 295 | 295 | ||
| 296 | drm_kms_helper_poll_init(ddev); | 296 | drmm_kms_helper_poll_init(ddev); |
| 297 | 297 | ||
| 298 | if (loongson_vblank) { | 298 | if (loongson_vblank) { |
| 299 | ret = drm_vblank_init(ddev, descp->num_of_crtc); | 299 | ret = drm_vblank_init(ddev, descp->num_of_crtc); |
| @@ -240,7 +240,7 @@ static const struct of_device_id mtk_cec_of_ids[] = { · 239 unmodified lines | |||
| 240 | }; | 240 | }; |
| 241 | MODULE_DEVICE_TABLE(of, mtk_cec_of_ids); | 241 | MODULE_DEVICE_TABLE(of, mtk_cec_of_ids); |
| 242 | 242 | ||
| 243 | struct platform_driver mtk_cec_driver = { | 243 | static struct platform_driver mtk_cec_driver = { |
| 244 | .probe = mtk_cec_probe, | 244 | .probe = mtk_cec_probe, |
| 245 | .remove = mtk_cec_remove, | 245 | .remove = mtk_cec_remove, |
| 246 | .driver = { | 246 | .driver = { |
| @@ -328,7 +328,7 @@ static const struct of_device_id mtk_hdmi_ddc_match[] = { · 327 unmodified lines | |||
| 328 | }; | 328 | }; |
| 329 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_match); | 329 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_match); |
| 330 | 330 | ||
| 331 | struct platform_driver mtk_hdmi_ddc_driver = { | 331 | static struct platform_driver mtk_hdmi_ddc_driver = { |
| 332 | .probe = mtk_hdmi_ddc_probe, | 332 | .probe = mtk_hdmi_ddc_probe, |
| 333 | .remove = mtk_hdmi_ddc_remove, | 333 | .remove = mtk_hdmi_ddc_remove, |
| 334 | .driver = { | 334 | .driver = { |
| @@ -389,7 +389,7 @@ static const struct of_device_id mtk_hdmi_ddc_v2_match[] = { · 388 unmodified lines | |||
| 389 | }; | 389 | }; |
| 390 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_v2_match); | 390 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_v2_match); |
| 391 | 391 | ||
| 392 | struct platform_driver mtk_hdmi_ddc_v2_driver = { | 392 | static struct platform_driver mtk_hdmi_ddc_v2_driver = { |
| 393 | .probe = mtk_hdmi_ddc_v2_probe, | 393 | .probe = mtk_hdmi_ddc_v2_probe, |
| 394 | .driver = { | 394 | .driver = { |
| 395 | .name = "mediatek-hdmi-ddc-v2", | 395 | .name = "mediatek-hdmi-ddc-v2", |
| @@ -50,7 +50,7 @@ enum mtk_hdmi_v2_clk_id { · 49 unmodified lines | |||
| 50 | MTK_HDMI_V2_CLK_COUNT, | 50 | MTK_HDMI_V2_CLK_COUNT, |
| 51 | }; | 51 | }; |
| 52 | 52 | ||
| 53 | const char *const mtk_hdmi_v2_clk_names[MTK_HDMI_V2_CLK_COUNT] = { | 53 | static const char *const mtk_hdmi_v2_clk_names[MTK_HDMI_V2_CLK_COUNT] = { |
| 54 | [MTK_HDMI_V2_CLK_HDMI_APB_SEL] = "bus", | 54 | [MTK_HDMI_V2_CLK_HDMI_APB_SEL] = "bus", |
| 55 | [MTK_HDMI_V2_CLK_HDCP_SEL] = "hdcp", | 55 | [MTK_HDMI_V2_CLK_HDCP_SEL] = "hdcp", |
| 56 | [MTK_HDMI_V2_CLK_HDCP_24M_SEL] = "hdcp24m", | 56 | [MTK_HDMI_V2_CLK_HDCP_24M_SEL] = "hdcp24m", |
| @@ -2621,7 +2621,6 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 2620 unmodified lines | |||
| 2621 | struct platform_device *pdev = priv->gpu_pdev; | 2621 | struct platform_device *pdev = priv->gpu_pdev; |
| 2622 | struct adreno_platform_config *config = pdev->dev.platform_data; | 2622 | struct adreno_platform_config *config = pdev->dev.platform_data; |
| 2623 | const struct adreno_info *info = config->info; | 2623 | const struct adreno_info *info = config->info; |
| 2624 | struct device_node *node; | ||
| 2625 | struct a6xx_gpu *a6xx_gpu; | 2624 | struct a6xx_gpu *a6xx_gpu; |
| 2626 | struct adreno_gpu *adreno_gpu; | 2625 | struct adreno_gpu *adreno_gpu; |
| 2627 | struct msm_gpu *gpu; | 2626 | struct msm_gpu *gpu; |
| @@ -2643,7 +2642,8 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 15 unmodified lines | |||
| 2643 | adreno_gpu->registers = NULL; | 2642 | adreno_gpu->registers = NULL; |
| 2644 | 2643 | ||
| 2645 | /* Check if there is a GMU phandle and set it up */ | 2644 | /* Check if there is a GMU phandle and set it up */ |
| 2646 | node = of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0); | 2645 | struct device_node *node __free(device_node) = |
| 2646 | of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0); | ||
| 2647 | /* FIXME: How do we gracefully handle this? */ | 2647 | /* FIXME: How do we gracefully handle this? */ |
| 2648 | BUG_ON(!node); | 2648 | BUG_ON(!node); |
| 2649 | 2649 | ||
| @@ -2690,7 +2690,6 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 40 unmodified lines | |||
| 2690 | ret = a6xx_gmu_wrapper_init(a6xx_gpu, node); | 2690 | ret = a6xx_gmu_wrapper_init(a6xx_gpu, node); |
| 2691 | else | 2691 | else |
| 2692 | ret = a6xx_gmu_init(a6xx_gpu, node); | 2692 | ret = a6xx_gmu_init(a6xx_gpu, node); |
| 2693 | of_node_put(node); | ||
| 2694 | if (ret) { | 2693 | if (ret) { |
| 2695 | a6xx_destroy(&(a6xx_gpu->base.base)); | 2694 | a6xx_destroy(&(a6xx_gpu->base.base)); |
| 2696 | return ERR_PTR(ret); | 2695 | return ERR_PTR(ret); |
| @@ -2740,6 +2739,7 @@ const struct adreno_gpu_funcs a6xx_gpu_funcs = { · 43 unmodified lines | |||
| 2740 | .create_private_vm = a6xx_create_private_vm, | 2739 | .create_private_vm = a6xx_create_private_vm, |
| 2741 | .get_rptr = a6xx_get_rptr, | 2740 | .get_rptr = a6xx_get_rptr, |
| 2742 | .progress = a6xx_progress, | 2741 | .progress = a6xx_progress, |
| 2742 | .sysprof_setup = a6xx_gmu_sysprof_setup, | ||
| 2743 | }, | 2743 | }, |
| 2744 | .init = a6xx_gpu_init, | 2744 | .init = a6xx_gpu_init, |
| 2745 | .get_timestamp = a6xx_gmu_get_timestamp, | 2745 | .get_timestamp = a6xx_gmu_get_timestamp, |
| @@ -2808,6 +2808,7 @@ const struct adreno_gpu_funcs a7xx_gpu_funcs = { · 62 unmodified lines | |||
| 2808 | .create_private_vm = a6xx_create_private_vm, | 2808 | .create_private_vm = a6xx_create_private_vm, |
| 2809 | .get_rptr = a6xx_get_rptr, | 2809 | .get_rptr = a6xx_get_rptr, |
| 2810 | .progress = a6xx_progress, | 2810 | .progress = a6xx_progress, |
| 2811 | .sysprof_setup = a6xx_gmu_sysprof_setup, | ||
| 2811 | }, | 2812 | }, |
| 2812 | .init = a6xx_gpu_init, | 2813 | .init = a6xx_gpu_init, |
| 2813 | .get_timestamp = a6xx_gmu_get_timestamp, | 2814 | .get_timestamp = a6xx_gmu_get_timestamp, |
| @@ -289,6 +289,8 @@ static int a8xx_hfi_send_perf_table(struct a6xx_gmu *gmu) · 288 unmodified lines | |||
| 289 | (gmu->nr_gpu_freqs * num_gx_votes * sizeof(gmu->gx_arc_votes[0])) + | 289 | (gmu->nr_gpu_freqs * num_gx_votes * sizeof(gmu->gx_arc_votes[0])) + |
| 290 | (gmu->nr_gmu_freqs * num_cx_votes * sizeof(gmu->cx_arc_votes[0])); | 290 | (gmu->nr_gmu_freqs * num_cx_votes * sizeof(gmu->cx_arc_votes[0])); |
| 291 | tbl = kzalloc(size, GFP_KERNEL); | 291 | tbl = kzalloc(size, GFP_KERNEL); |
| 292 | if (!tbl) | ||
| 293 | return -ENOMEM; | ||
| 292 | tbl->type = HFI_TABLE_GPU_PERF; | 294 | tbl->type = HFI_TABLE_GPU_PERF; |
| 293 | 295 | ||
| 294 | /* First fill GX votes */ | 296 | /* First fill GX votes */ |
| @@ -17,7 +17,7 @@ MODULE_PARM_DESC(snapshot_debugbus, "Include debugbus sections in GPU devcoredum · 16 unmodified lines | |||
| 17 | module_param_named(snapshot_debugbus, snapshot_debugbus, bool, 0600); | 17 | module_param_named(snapshot_debugbus, snapshot_debugbus, bool, 0600); |
| 18 | 18 | ||
| 19 | int enable_preemption = -1; | 19 | int enable_preemption = -1; |
| 20 | MODULE_PARM_DESC(enable_preemption, "Enable preemption (A7xx only) (1=on , 0=disable, -1=auto (default))"); | 20 | MODULE_PARM_DESC(enable_preemption, "Enable preemption (A7xx+ only) (1=on , 0=disable, -1=auto (default))"); |
| 21 | module_param(enable_preemption, int, 0600); | 21 | module_param(enable_preemption, int, 0600); |
| 22 | 22 | ||
| 23 | bool disable_acd; | 23 | bool disable_acd; |
| @@ -376,7 +376,7 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, · 375 unmodified lines | |||
| 376 | *value = adreno_gpu->info->gmem; | 376 | *value = adreno_gpu->info->gmem; |
| 377 | return 0; | 377 | return 0; |
| 378 | case MSM_PARAM_GMEM_BASE: | 378 | case MSM_PARAM_GMEM_BASE: |
| 379 | if (adreno_gpu->info->family >= ADRENO_6XX_GEN4) | 379 | if (adreno_gpu->info->family >= ADRENO_6XX_GEN3) |
| 380 | *value = 0; | 380 | *value = 0; |
| 381 | else | 381 | else |
| 382 | *value = 0x100000; | 382 | *value = 0x100000; |
| @@ -424,15 +424,21 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, · 41 unmodified lines | |||
| 424 | *value = vm->mm_range; | 424 | *value = vm->mm_range; |
| 425 | return 0; | 425 | return 0; |
| 426 | case MSM_PARAM_HIGHEST_BANK_BIT: | 426 | case MSM_PARAM_HIGHEST_BANK_BIT: |
| 427 | if (!adreno_gpu->ubwc_config) | ||
| 428 | return UERR(ENOENT, drm, "no UBWC on this platform"); | ||
| 427 | *value = adreno_gpu->ubwc_config->highest_bank_bit; | 429 | *value = adreno_gpu->ubwc_config->highest_bank_bit; |
| 428 | return 0; | 430 | return 0; |
| 429 | case MSM_PARAM_RAYTRACING: | 431 | case MSM_PARAM_RAYTRACING: |
| 430 | *value = adreno_gpu->has_ray_tracing; | 432 | *value = adreno_gpu->has_ray_tracing; |
| 431 | return 0; | 433 | return 0; |
| 432 | case MSM_PARAM_UBWC_SWIZZLE: | 434 | case MSM_PARAM_UBWC_SWIZZLE: |
| 435 | if (!adreno_gpu->ubwc_config) | ||
| 436 | return UERR(ENOENT, drm, "no UBWC on this platform"); | ||
| 433 | *value = adreno_gpu->ubwc_config->ubwc_swizzle; | 437 | *value = adreno_gpu->ubwc_config->ubwc_swizzle; |
| 434 | return 0; | 438 | return 0; |
| 435 | case MSM_PARAM_MACROTILE_MODE: | 439 | case MSM_PARAM_MACROTILE_MODE: |
| 440 | if (!adreno_gpu->ubwc_config) | ||
| 441 | return UERR(ENOENT, drm, "no UBWC on this platform"); | ||
| 436 | *value = adreno_gpu->ubwc_config->macrotile_mode; | 442 | *value = adreno_gpu->ubwc_config->macrotile_mode; |
| 437 | return 0; | 443 | return 0; |
| 438 | case MSM_PARAM_UCHE_TRAP_BASE: | 444 | case MSM_PARAM_UCHE_TRAP_BASE: |
| @@ -480,7 +480,7 @@ const struct dpu_mdss_cfg dpu_kaanapali_cfg = { · 479 unmodified lines | |||
| 480 | .wb_count = ARRAY_SIZE(kaanapali_wb), | 480 | .wb_count = ARRAY_SIZE(kaanapali_wb), |
| 481 | .wb = kaanapali_wb, | 481 | .wb = kaanapali_wb, |
| 482 | .cwb_count = ARRAY_SIZE(kaanapali_cwb), | 482 | .cwb_count = ARRAY_SIZE(kaanapali_cwb), |
| 483 | .cwb = sm8650_cwb, | 483 | .cwb = kaanapali_cwb, |
| 484 | .intf_count = ARRAY_SIZE(kaanapali_intf), | 484 | .intf_count = ARRAY_SIZE(kaanapali_intf), |
| 485 | .intf = kaanapali_intf, | 485 | .intf = kaanapali_intf, |
| 486 | .vbif = &sm8650_vbif, | 486 | .vbif = &sm8650_vbif, |
| @@ -61,7 +61,7 @@ static int _dpu_format_populate_plane_sizes_ubwc( · 60 unmodified lines | |||
| 61 | bool meta = MSM_FORMAT_IS_UBWC(fmt); | 61 | bool meta = MSM_FORMAT_IS_UBWC(fmt); |
| 62 | 62 | ||
| 63 | if (MSM_FORMAT_IS_YUV(fmt)) { | 63 | if (MSM_FORMAT_IS_YUV(fmt)) { |
| 64 | unsigned int stride, sclines; | 64 | unsigned int stride, y_sclines, uv_sclines; |
| 65 | unsigned int y_tile_width, y_tile_height; | 65 | unsigned int y_tile_width, y_tile_height; |
| 66 | unsigned int y_meta_stride, y_meta_scanlines; | 66 | unsigned int y_meta_stride, y_meta_scanlines; |
| 67 | unsigned int uv_meta_stride, uv_meta_scanlines; | 67 | unsigned int uv_meta_stride, uv_meta_scanlines; |
| @@ -77,23 +77,25 @@ static int _dpu_format_populate_plane_sizes_ubwc( · 9 unmodified lines | |||
| 77 | y_tile_width = 32; | 77 | y_tile_width = 32; |
| 78 | } | 78 | } |
| 79 | 79 | ||
| 80 | sclines = round_up(fb->height, 16); | 80 | y_sclines = round_up(fb->height, 16); |
| 81 | uv_sclines = round_up((fb->height+1)>>1, 16); | ||
| 81 | y_tile_height = 4; | 82 | y_tile_height = 4; |
| 82 | } else { | 83 | } else { |
| 83 | stride = round_up(fb->width, 128); | 84 | stride = round_up(fb->width, 128); |
| 84 | y_tile_width = 32; | 85 | y_tile_width = 32; |
| 85 | 86 | ||
| 86 | sclines = round_up(fb->height, 32); | 87 | y_sclines = round_up(fb->height, 32); |
| 88 | uv_sclines = round_up((fb->height+1)>>1, 32); | ||
| 87 | y_tile_height = 8; | 89 | y_tile_height = 8; |
| 88 | } | 90 | } |
| 89 | 91 | ||
| 90 | layout->plane_pitch[0] = stride; | 92 | layout->plane_pitch[0] = stride; |
| 91 | layout->plane_size[0] = round_up(layout->plane_pitch[0] * | 93 | layout->plane_size[0] = round_up(layout->plane_pitch[0] * |
| 92 | sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | 94 | y_sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); |
| 93 | 95 | ||
| 94 | layout->plane_pitch[1] = stride; | 96 | layout->plane_pitch[1] = stride; |
| 95 | layout->plane_size[1] = round_up(layout->plane_pitch[1] * | 97 | layout->plane_size[1] = round_up(layout->plane_pitch[1] * |
| 96 | sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | 98 | uv_sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); |
| 97 | 99 | ||
| 98 | if (!meta) | 100 | if (!meta) |
| 99 | return 0; | 101 | return 0; |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include <drm/drm_edid.h> | 6 | #include <drm/drm_edid.h> |
| 7 | #include <drm/drm_framebuffer.h> | 7 | #include <drm/drm_framebuffer.h> |
| 8 | #include <drm/drm_managed.h> | ||
| 8 | 9 | ||
| 9 | #include "dpu_writeback.h" | 10 | #include "dpu_writeback.h" |
| 10 | 11 | ||
| @@ -125,7 +126,7 @@ int dpu_writeback_init(struct drm_device *dev, struct drm_encoder *enc, · 114 unmodified lines | |||
| 125 | struct dpu_wb_connector *dpu_wb_conn; | 126 | struct dpu_wb_connector *dpu_wb_conn; |
| 126 | int rc = 0; | 127 | int rc = 0; |
| 127 | 128 | ||
| 128 | dpu_wb_conn = devm_kzalloc(dev->dev, sizeof(*dpu_wb_conn), GFP_KERNEL); | 129 | dpu_wb_conn = drmm_kzalloc(dev, sizeof(*dpu_wb_conn), GFP_KERNEL); |
| 129 | if (!dpu_wb_conn) | 130 | if (!dpu_wb_conn) |
| 130 | return -ENOMEM; | 131 | return -ENOMEM; |
| 131 | 132 | ||
| @@ -5,11 +5,11 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #define pr_fmt(fmt) "[drm:%s:%d] " fmt, __func__, __LINE__ | 6 | #define pr_fmt(fmt) "[drm:%s:%d] " fmt, __func__, __LINE__ |
| 7 | 7 | ||
| 8 | #include <generated/utsrelease.h> | 8 | #include <linux/utsname.h> |
| 9 | 9 | ||
| 10 | #include "msm_disp_snapshot.h" | 10 | #include "msm_disp_snapshot.h" |
| 11 | 11 | ||
| 12 | static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *base_addr) | 12 | static void msm_disp_state_dump_regs(u32 **reg, u32 len, void __iomem *base_addr) |
| 13 | { | 13 | { |
| 14 | u32 len_padded; | 14 | u32 len_padded; |
| 15 | u32 num_rows; | 15 | u32 num_rows; |
| @@ -19,11 +19,11 @@ static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *b · 3 unmodified lines | |||
| 19 | void __iomem *end_addr; | 19 | void __iomem *end_addr; |
| 20 | int i; | 20 | int i; |
| 21 | 21 | ||
| 22 | len_padded = aligned_len * REG_DUMP_ALIGN; | 22 | len_padded = round_up(len, REG_DUMP_ALIGN); |
| 23 | num_rows = aligned_len / REG_DUMP_ALIGN; | 23 | num_rows = DIV_ROUND_UP(len, REG_DUMP_ALIGN); |
| 24 | 24 | ||
| 25 | addr = base_addr; | 25 | addr = base_addr; |
| 26 | end_addr = base_addr + aligned_len; | 26 | end_addr = base_addr + len; |
| 27 | 27 | ||
| 28 | *reg = kvzalloc(len_padded, GFP_KERNEL); | 28 | *reg = kvzalloc(len_padded, GFP_KERNEL); |
| 29 | if (!*reg) | 29 | if (!*reg) |
| @@ -48,8 +48,8 @@ static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *b · 18 unmodified lines | |||
| 48 | static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, | 48 | static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, |
| 49 | void __iomem *base_addr, struct drm_printer *p) | 49 | void __iomem *base_addr, struct drm_printer *p) |
| 50 | { | 50 | { |
| 51 | void __iomem *addr, *end_addr; | ||
| 51 | int i; | 52 | int i; |
| 52 | void __iomem *addr; | ||
| 53 | u32 num_rows; | 53 | u32 num_rows; |
| 54 | 54 | ||
| 55 | if (!dump_addr) { | 55 | if (!dump_addr) { |
| @@ -58,6 +58,7 @@ static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, · 2 unmodified lines | |||
| 58 | } | 58 | } |
| 59 | 59 | ||
| 60 | addr = base_addr; | 60 | addr = base_addr; |
| 61 | end_addr = base_addr + len; | ||
| 61 | num_rows = len / REG_DUMP_ALIGN; | 62 | num_rows = len / REG_DUMP_ALIGN; |
| 62 | 63 | ||
| 63 | for (i = 0; i < num_rows; i++) { | 64 | for (i = 0; i < num_rows; i++) { |
| @@ -67,6 +68,17 @@ static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, · 3 unmodified lines | |||
| 67 | dump_addr[i * 4 + 2], dump_addr[i * 4 + 3]); | 68 | dump_addr[i * 4 + 2], dump_addr[i * 4 + 3]); |
| 68 | addr += REG_DUMP_ALIGN; | 69 | addr += REG_DUMP_ALIGN; |
| 69 | } | 70 | } |
| 71 | |||
| 72 | if (addr != end_addr) { | ||
| 73 | drm_printf(p, "0x%lx : %08x", | ||
| 74 | (unsigned long)(addr - base_addr), | ||
| 75 | dump_addr[i * 4]); | ||
| 76 | if (addr + 0x4 < end_addr) | ||
| 77 | drm_printf(p, " %08x", dump_addr[i * 4 + 1]); | ||
| 78 | if (addr + 0x8 < end_addr) | ||
| 79 | drm_printf(p, " %08x", dump_addr[i * 4 + 2]); | ||
| 80 | drm_printf(p, "\n"); | ||
| 81 | } | ||
| 70 | } | 82 | } |
| 71 | 83 | ||
| 72 | void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p) | 84 | void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p) |
| @@ -79,7 +91,7 @@ void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p) · 6 unmodified lines | |||
| 79 | } | 91 | } |
| 80 | 92 | ||
| 81 | drm_printf(p, "---\n"); | 93 | drm_printf(p, "---\n"); |
| 82 | drm_printf(p, "kernel: " UTS_RELEASE "\n"); | 94 | drm_printf(p, "kernel: %s\n", init_utsname()->release); |
| 83 | drm_printf(p, "module: " KBUILD_MODNAME "\n"); | 95 | drm_printf(p, "module: " KBUILD_MODNAME "\n"); |
| 84 | drm_printf(p, "dpu devcoredump\n"); | 96 | drm_printf(p, "dpu devcoredump\n"); |
| 85 | drm_printf(p, "time: %ptSp\n", &state->time); | 97 | drm_printf(p, "time: %ptSp\n", &state->time); |
| @@ -185,7 +197,7 @@ void msm_disp_snapshot_add_block(struct msm_disp_state *disp_state, u32 len, · 99 unmodified lines | |||
| 185 | va_end(va); | 197 | va_end(va); |
| 186 | 198 | ||
| 187 | INIT_LIST_HEAD(&new_blk->node); | 199 | INIT_LIST_HEAD(&new_blk->node); |
| 188 | new_blk->size = ALIGN(len, REG_DUMP_ALIGN); | 200 | new_blk->size = len; |
| 189 | new_blk->base_addr = base_addr; | 201 | new_blk->base_addr = base_addr; |
| 190 | 202 | ||
| 191 | msm_disp_state_dump_regs(&new_blk->state, new_blk->size, base_addr); | 203 | msm_disp_state_dump_regs(&new_blk->state, new_blk->size, base_addr); |
| @@ -2033,6 +2033,7 @@ int msm_dsi_host_init(struct msm_dsi *msm_dsi) · 2032 unmodified lines | |||
| 2033 | 2033 | ||
| 2034 | /* fixup base address by io offset */ | 2034 | /* fixup base address by io offset */ |
| 2035 | msm_host->ctrl_base += cfg->io_offset; | 2035 | msm_host->ctrl_base += cfg->io_offset; |
| 2036 | msm_host->ctrl_size -= cfg->io_offset; | ||
| 2036 | 2037 | ||
| 2037 | ret = devm_regulator_bulk_get_const(&pdev->dev, cfg->num_regulators, | 2038 | ret = devm_regulator_bulk_get_const(&pdev->dev, cfg->num_regulators, |
| 2038 | cfg->regulator_data, | 2039 | cfg->regulator_data, |
| @@ -128,11 +128,10 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv, · 127 unmodified lines | |||
| 128 | /* | 128 | /* |
| 129 | * Initialize the LRUs: | 129 | * Initialize the LRUs: |
| 130 | */ | 130 | */ |
| 131 | mutex_init(&priv->lru.lock); | 131 | drm_gem_lru_init(&priv->lru.unbacked); |
| 132 | drm_gem_lru_init(&priv->lru.unbacked, &priv->lru.lock); | 132 | drm_gem_lru_init(&priv->lru.pinned); |
| 133 | drm_gem_lru_init(&priv->lru.pinned, &priv->lru.lock); | 133 | drm_gem_lru_init(&priv->lru.willneed); |
| 134 | drm_gem_lru_init(&priv->lru.willneed, &priv->lru.lock); | 134 | drm_gem_lru_init(&priv->lru.dontneed); |
| 135 | drm_gem_lru_init(&priv->lru.dontneed, &priv->lru.lock); | ||
| 136 | 135 | ||
| 137 | /* Initialize stall-on-fault */ | 136 | /* Initialize stall-on-fault */ |
| 138 | spin_lock_init(&priv->fault_stall_lock); | 137 | spin_lock_init(&priv->fault_stall_lock); |
| @@ -140,7 +139,7 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv, · 1 unmodified line | |||
| 140 | 139 | ||
| 141 | /* Teach lockdep about lock ordering wrt. shrinker: */ | 140 | /* Teach lockdep about lock ordering wrt. shrinker: */ |
| 142 | fs_reclaim_acquire(GFP_KERNEL); | 141 | fs_reclaim_acquire(GFP_KERNEL); |
| 143 | might_lock(&priv->lru.lock); | 142 | might_lock(&ddev->gem_lru_mutex); |
| 144 | fs_reclaim_release(GFP_KERNEL); | 143 | fs_reclaim_release(GFP_KERNEL); |
| 145 | 144 | ||
| 146 | if (priv->kms_init) { | 145 | if (priv->kms_init) { |
| @@ -150,13 +150,6 @@ struct msm_drm_private { · 149 unmodified lines | |||
| 150 | * DONTNEED state (ie. can be purged) | 150 | * DONTNEED state (ie. can be purged) |
| 151 | */ | 151 | */ |
| 152 | struct drm_gem_lru dontneed; | 152 | struct drm_gem_lru dontneed; |
| 153 | |||
| 154 | /** | ||
| 155 | * lock: | ||
| 156 | * | ||
| 157 | * Protects manipulation of all of the LRUs. | ||
| 158 | */ | ||
| 159 | struct mutex lock; | ||
| 160 | } lru; | 153 | } lru; |
| 161 | 154 | ||
| 162 | struct notifier_block vmap_notifier; | 155 | struct notifier_block vmap_notifier; |
| @@ -177,11 +177,11 @@ static void update_lru_locked(struct drm_gem_object *obj) · 176 unmodified lines | |||
| 177 | 177 | ||
| 178 | static void update_lru(struct drm_gem_object *obj) | 178 | static void update_lru(struct drm_gem_object *obj) |
| 179 | { | 179 | { |
| 180 | struct msm_drm_private *priv = obj->dev->dev_private; | 180 | struct drm_device *dev = obj->dev; |
| 181 | 181 | ||
| 182 | mutex_lock(&priv->lru.lock); | 182 | mutex_lock(&dev->gem_lru_mutex); |
| 183 | update_lru_locked(obj); | 183 | update_lru_locked(obj); |
| 184 | mutex_unlock(&priv->lru.lock); | 184 | mutex_unlock(&dev->gem_lru_mutex); |
| 185 | } | 185 | } |
| 186 | 186 | ||
| 187 | static struct page **get_pages(struct drm_gem_object *obj) | 187 | static struct page **get_pages(struct drm_gem_object *obj) |
| @@ -292,11 +292,11 @@ void msm_gem_pin_obj_locked(struct drm_gem_object *obj) · 104 unmodified lines | |||
| 292 | 292 | ||
| 293 | static void pin_obj_locked(struct drm_gem_object *obj) | 293 | static void pin_obj_locked(struct drm_gem_object *obj) |
| 294 | { | 294 | { |
| 295 | struct msm_drm_private *priv = obj->dev->dev_private; | 295 | struct drm_device *dev = obj->dev; |
| 296 | 296 | ||
| 297 | mutex_lock(&priv->lru.lock); | 297 | mutex_lock(&dev->gem_lru_mutex); |
| 298 | msm_gem_pin_obj_locked(obj); | 298 | msm_gem_pin_obj_locked(obj); |
| 299 | mutex_unlock(&priv->lru.lock); | 299 | mutex_unlock(&dev->gem_lru_mutex); |
| 300 | } | 300 | } |
| 301 | 301 | ||
| 302 | struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) | 302 | struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) |
| @@ -487,16 +487,16 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct drm_gpuva *vma) · 184 unmodified lines | |||
| 487 | 487 | ||
| 488 | void msm_gem_unpin_locked(struct drm_gem_object *obj) | 488 | void msm_gem_unpin_locked(struct drm_gem_object *obj) |
| 489 | { | 489 | { |
| 490 | struct msm_drm_private *priv = obj->dev->dev_private; | 490 | struct drm_device *dev = obj->dev; |
| 491 | struct msm_gem_object *msm_obj = to_msm_bo(obj); | 491 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 492 | 492 | ||
| 493 | msm_gem_assert_locked(obj); | 493 | msm_gem_assert_locked(obj); |
| 494 | 494 | ||
| 495 | mutex_lock(&priv->lru.lock); | 495 | mutex_lock(&dev->gem_lru_mutex); |
| 496 | msm_obj->pin_count--; | 496 | msm_obj->pin_count--; |
| 497 | GEM_WARN_ON(msm_obj->pin_count < 0); | 497 | GEM_WARN_ON(msm_obj->pin_count < 0); |
| 498 | update_lru_locked(obj); | 498 | update_lru_locked(obj); |
| 499 | mutex_unlock(&priv->lru.lock); | 499 | mutex_unlock(&dev->gem_lru_mutex); |
| 500 | } | 500 | } |
| 501 | 501 | ||
| 502 | /* Special unpin path for use in fence-signaling path, avoiding the need | 502 | /* Special unpin path for use in fence-signaling path, avoiding the need |
| @@ -507,10 +507,10 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) · 4 unmodified lines | |||
| 507 | */ | 507 | */ |
| 508 | void msm_gem_unpin_active(struct drm_gem_object *obj) | 508 | void msm_gem_unpin_active(struct drm_gem_object *obj) |
| 509 | { | 509 | { |
| 510 | struct msm_drm_private *priv = obj->dev->dev_private; | 510 | struct drm_device *dev = obj->dev; |
| 511 | struct msm_gem_object *msm_obj = to_msm_bo(obj); | 511 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 512 | 512 | ||
| 513 | GEM_WARN_ON(!mutex_is_locked(&priv->lru.lock)); | 513 | GEM_WARN_ON(!mutex_is_locked(&dev->gem_lru_mutex)); |
| 514 | 514 | ||
| 515 | msm_obj->pin_count--; | 515 | msm_obj->pin_count--; |
| 516 | GEM_WARN_ON(msm_obj->pin_count < 0); | 516 | GEM_WARN_ON(msm_obj->pin_count < 0); |
| @@ -797,12 +797,12 @@ void msm_gem_put_vaddr(struct drm_gem_object *obj) · 280 unmodified lines | |||
| 797 | */ | 797 | */ |
| 798 | int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) | 798 | int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) |
| 799 | { | 799 | { |
| 800 | struct msm_drm_private *priv = obj->dev->dev_private; | 800 | struct drm_device *dev = obj->dev; |
| 801 | struct msm_gem_object *msm_obj = to_msm_bo(obj); | 801 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 802 | 802 | ||
| 803 | msm_gem_lock(obj); | 803 | msm_gem_lock(obj); |
| 804 | 804 | ||
| 805 | mutex_lock(&priv->lru.lock); | 805 | mutex_lock(&dev->gem_lru_mutex); |
| 806 | 806 | ||
| 807 | if (msm_obj->madv != __MSM_MADV_PURGED) | 807 | if (msm_obj->madv != __MSM_MADV_PURGED) |
| 808 | msm_obj->madv = madv; | 808 | msm_obj->madv = madv; |
| @@ -814,7 +814,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) · 5 unmodified lines | |||
| 814 | */ | 814 | */ |
| 815 | update_lru_locked(obj); | 815 | update_lru_locked(obj); |
| 816 | 816 | ||
| 817 | mutex_unlock(&priv->lru.lock); | 817 | mutex_unlock(&dev->gem_lru_mutex); |
| 818 | 818 | ||
| 819 | msm_gem_unlock(obj); | 819 | msm_gem_unlock(obj); |
| 820 | 820 | ||
| @@ -824,7 +824,6 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) · 3 unmodified lines | |||
| 824 | void msm_gem_purge(struct drm_gem_object *obj) | 824 | void msm_gem_purge(struct drm_gem_object *obj) |
| 825 | { | 825 | { |
| 826 | struct drm_device *dev = obj->dev; | 826 | struct drm_device *dev = obj->dev; |
| 827 | struct msm_drm_private *priv = obj->dev->dev_private; | ||
| 828 | struct msm_gem_object *msm_obj = to_msm_bo(obj); | 827 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 829 | 828 | ||
| 830 | msm_gem_assert_locked(obj); | 829 | msm_gem_assert_locked(obj); |
| @@ -839,10 +838,10 @@ void msm_gem_purge(struct drm_gem_object *obj) · 8 unmodified lines | |||
| 839 | 838 | ||
| 840 | put_pages(obj); | 839 | put_pages(obj); |
| 841 | 840 | ||
| 842 | mutex_lock(&priv->lru.lock); | 841 | mutex_lock(&dev->gem_lru_mutex); |
| 843 | /* A one-way transition: */ | 842 | /* A one-way transition: */ |
| 844 | msm_obj->madv = __MSM_MADV_PURGED; | 843 | msm_obj->madv = __MSM_MADV_PURGED; |
| 845 | mutex_unlock(&priv->lru.lock); | 844 | mutex_unlock(&dev->gem_lru_mutex); |
| 846 | 845 | ||
| 847 | drm_gem_free_mmap_offset(obj); | 846 | drm_gem_free_mmap_offset(obj); |
| 848 | 847 | ||
| @@ -43,8 +43,7 @@ msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) · 42 unmodified lines | |||
| 43 | } | 43 | } |
| 44 | 44 | ||
| 45 | static bool | 45 | static bool |
| 46 | with_vm_locks(struct ww_acquire_ctx *ticket, | 46 | with_vm_locks(void (*fn)(struct drm_gem_object *obj), |
| 47 | void (*fn)(struct drm_gem_object *obj), | ||
| 48 | struct drm_gem_object *obj) | 47 | struct drm_gem_object *obj) |
| 49 | { | 48 | { |
| 50 | /* | 49 | /* |
| @@ -52,7 +51,7 @@ with_vm_locks(struct ww_acquire_ctx *ticket, · 1 unmodified line | |||
| 52 | * success paths | 51 | * success paths |
| 53 | */ | 52 | */ |
| 54 | struct drm_gpuvm_bo *vm_bo, *last_locked = NULL; | 53 | struct drm_gpuvm_bo *vm_bo, *last_locked = NULL; |
| 55 | int ret = 0; | 54 | bool locked = true; |
| 56 | 55 | ||
| 57 | drm_gem_for_each_gpuvm_bo (vm_bo, obj) { | 56 | drm_gem_for_each_gpuvm_bo (vm_bo, obj) { |
| 58 | struct dma_resv *resv = drm_gpuvm_resv(vm_bo->vm); | 57 | struct dma_resv *resv = drm_gpuvm_resv(vm_bo->vm); |
| @@ -60,23 +59,14 @@ with_vm_locks(struct ww_acquire_ctx *ticket, · 1 unmodified line | |||
| 60 | if (resv == obj->resv) | 59 | if (resv == obj->resv) |
| 61 | continue; | 60 | continue; |
| 62 | 61 | ||
| 63 | ret = dma_resv_lock(resv, ticket); | ||
| 64 | |||
| 65 | /* | ||
| 66 | * Since we already skip the case when the VM and obj | ||
| 67 | * share a resv (ie. _NO_SHARE objs), we don't expect | ||
| 68 | * to hit a double-locking scenario... which the lock | ||
| 69 | * unwinding cannot really cope with. | ||
| 70 | */ | ||
| 71 | WARN_ON(ret == -EALREADY); | ||
| 72 | |||
| 73 | /* | 62 | /* |
| 74 | * Don't bother with slow-lock / backoff / retry sequence, | 63 | * dma_resv_lock can't be used due to acquiring 'ticket' before the |
| 75 | * if we can't get the lock just give up and move on to | 64 | * fs_reclaim lock, which is held in shrinker context |
| 76 | * the next object. | ||
| 77 | */ | 65 | */ |
| 78 | if (ret) | 66 | if (!dma_resv_trylock(resv)) { |
| 67 | locked = false; | ||
| 79 | goto out_unlock; | 68 | goto out_unlock; |
| 69 | } | ||
| 80 | 70 | ||
| 81 | /* | 71 | /* |
| 82 | * Hold a ref to prevent the vm_bo from being freed | 72 | * Hold a ref to prevent the vm_bo from being freed |
| @@ -108,11 +98,11 @@ out_unlock: · 25 unmodified lines | |||
| 108 | } | 98 | } |
| 109 | } | 99 | } |
| 110 | 100 | ||
| 111 | return ret == 0; | 101 | return locked; |
| 112 | } | 102 | } |
| 113 | 103 | ||
| 114 | static bool | 104 | static bool |
| 115 | purge(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) | 105 | purge(struct drm_gem_object *obj, struct ww_acquire_ctx *unused) |
| 116 | { | 106 | { |
| 117 | if (!is_purgeable(to_msm_bo(obj))) | 107 | if (!is_purgeable(to_msm_bo(obj))) |
| 118 | return false; | 108 | return false; |
| @@ -120,11 +110,11 @@ purge(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) · 1 unmodified line | |||
| 120 | if (msm_gem_active(obj)) | 110 | if (msm_gem_active(obj)) |
| 121 | return false; | 111 | return false; |
| 122 | 112 | ||
| 123 | return with_vm_locks(ticket, msm_gem_purge, obj); | 113 | return with_vm_locks(msm_gem_purge, obj); |
| 124 | } | 114 | } |
| 125 | 115 | ||
| 126 | static bool | 116 | static bool |
| 127 | evict(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) | 117 | evict(struct drm_gem_object *obj, struct ww_acquire_ctx *unused) |
| 128 | { | 118 | { |
| 129 | if (is_unevictable(to_msm_bo(obj))) | 119 | if (is_unevictable(to_msm_bo(obj))) |
| 130 | return false; | 120 | return false; |
| @@ -132,7 +122,7 @@ evict(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) · 1 unmodified line | |||
| 132 | if (msm_gem_active(obj)) | 122 | if (msm_gem_active(obj)) |
| 133 | return false; | 123 | return false; |
| 134 | 124 | ||
| 135 | return with_vm_locks(ticket, msm_gem_evict, obj); | 125 | return with_vm_locks(msm_gem_evict, obj); |
| 136 | } | 126 | } |
| 137 | 127 | ||
| 138 | static bool | 128 | static bool |
| @@ -164,7 +154,6 @@ static unsigned long · 25 unmodified lines | |||
| 164 | msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) | 154 | msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) |
| 165 | { | 155 | { |
| 166 | struct msm_drm_private *priv = shrinker->private_data; | 156 | struct msm_drm_private *priv = shrinker->private_data; |
| 167 | struct ww_acquire_ctx ticket; | ||
| 168 | struct { | 157 | struct { |
| 169 | struct drm_gem_lru *lru; | 158 | struct drm_gem_lru *lru; |
| 170 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket); | 159 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket); |
| @@ -185,11 +174,14 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) · 14 unmodified lines | |||
| 185 | for (unsigned i = 0; (nr > 0) && (i < ARRAY_SIZE(stages)); i++) { | 174 | for (unsigned i = 0; (nr > 0) && (i < ARRAY_SIZE(stages)); i++) { |
| 186 | if (!stages[i].cond) | 175 | if (!stages[i].cond) |
| 187 | continue; | 176 | continue; |
| 177 | /* | ||
| 178 | * 'ticket' not needed on trylock paths | ||
| 179 | */ | ||
| 188 | stages[i].freed = | 180 | stages[i].freed = |
| 189 | drm_gem_lru_scan(stages[i].lru, nr, | 181 | drm_gem_lru_scan(priv->dev, stages[i].lru, nr, |
| 190 | &stages[i].remaining, | 182 | &stages[i].remaining, |
| 191 | stages[i].shrink, | 183 | stages[i].shrink, |
| 192 | &ticket); | 184 | NULL); |
| 193 | nr -= stages[i].freed; | 185 | nr -= stages[i].freed; |
| 194 | freed += stages[i].freed; | 186 | freed += stages[i].freed; |
| 195 | remaining += stages[i].remaining; | 187 | remaining += stages[i].remaining; |
| @@ -255,7 +247,7 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) · 59 unmodified lines | |||
| 255 | unsigned long remaining = 0; | 247 | unsigned long remaining = 0; |
| 256 | 248 | ||
| 257 | for (idx = 0; lrus[idx] && unmapped < vmap_shrink_limit; idx++) { | 249 | for (idx = 0; lrus[idx] && unmapped < vmap_shrink_limit; idx++) { |
| 258 | unmapped += drm_gem_lru_scan(lrus[idx], | 250 | unmapped += drm_gem_lru_scan(priv->dev, lrus[idx], |
| 259 | vmap_shrink_limit - unmapped, | 251 | vmap_shrink_limit - unmapped, |
| 260 | &remaining, | 252 | &remaining, |
| 261 | vmap_shrink, | 253 | vmap_shrink, |
| @@ -352,7 +352,7 @@ static int submit_fence_sync(struct msm_gem_submit *submit) · 351 unmodified lines | |||
| 352 | 352 | ||
| 353 | static int submit_pin_objects(struct msm_gem_submit *submit) | 353 | static int submit_pin_objects(struct msm_gem_submit *submit) |
| 354 | { | 354 | { |
| 355 | struct msm_drm_private *priv = submit->dev->dev_private; | 355 | struct drm_device *dev = submit->dev; |
| 356 | int i, ret = 0; | 356 | int i, ret = 0; |
| 357 | 357 | ||
| 358 | for (i = 0; i < submit->nr_bos; i++) { | 358 | for (i = 0; i < submit->nr_bos; i++) { |
| @@ -381,11 +381,11 @@ static int submit_pin_objects(struct msm_gem_submit *submit) · 22 unmodified lines | |||
| 381 | * get_pages() which could trigger reclaim.. and if we held the LRU lock | 381 | * get_pages() which could trigger reclaim.. and if we held the LRU lock |
| 382 | * could trigger deadlock with the shrinker). | 382 | * could trigger deadlock with the shrinker). |
| 383 | */ | 383 | */ |
| 384 | mutex_lock(&priv->lru.lock); | 384 | mutex_lock(&dev->gem_lru_mutex); |
| 385 | for (i = 0; i < submit->nr_bos; i++) { | 385 | for (i = 0; i < submit->nr_bos; i++) { |
| 386 | msm_gem_pin_obj_locked(submit->bos[i].obj); | 386 | msm_gem_pin_obj_locked(submit->bos[i].obj); |
| 387 | } | 387 | } |
| 388 | mutex_unlock(&priv->lru.lock); | 388 | mutex_unlock(&dev->gem_lru_mutex); |
| 389 | 389 | ||
| 390 | submit->bos_pinned = true; | 390 | submit->bos_pinned = true; |
| 391 | 391 | ||
| @@ -702,7 +702,7 @@ static struct dma_fence * · 701 unmodified lines | |||
| 702 | msm_vma_job_run(struct drm_sched_job *_job) | 702 | msm_vma_job_run(struct drm_sched_job *_job) |
| 703 | { | 703 | { |
| 704 | struct msm_vm_bind_job *job = to_msm_vm_bind_job(_job); | 704 | struct msm_vm_bind_job *job = to_msm_vm_bind_job(_job); |
| 705 | struct msm_drm_private *priv = job->vm->drm->dev_private; | 705 | struct drm_device *dev = job->vm->drm; |
| 706 | struct msm_gem_vm *vm = to_msm_vm(job->vm); | 706 | struct msm_gem_vm *vm = to_msm_vm(job->vm); |
| 707 | struct drm_gem_object *obj; | 707 | struct drm_gem_object *obj; |
| 708 | int ret = vm->unusable ? -EINVAL : 0; | 708 | int ret = vm->unusable ? -EINVAL : 0; |
| @@ -745,13 +745,13 @@ msm_vma_job_run(struct drm_sched_job *_job) · 36 unmodified lines | |||
| 745 | if (ret) | 745 | if (ret) |
| 746 | msm_gem_vm_unusable(job->vm); | 746 | msm_gem_vm_unusable(job->vm); |
| 747 | 747 | ||
| 748 | mutex_lock(&priv->lru.lock); | 748 | mutex_lock(&dev->gem_lru_mutex); |
| 749 | 749 | ||
| 750 | job_foreach_bo (obj, job) { | 750 | job_foreach_bo (obj, job) { |
| 751 | msm_gem_unpin_active(obj); | 751 | msm_gem_unpin_active(obj); |
| 752 | } | 752 | } |
| 753 | 753 | ||
| 754 | mutex_unlock(&priv->lru.lock); | 754 | mutex_unlock(&dev->gem_lru_mutex); |
| 755 | 755 | ||
| 756 | /* VM_BIND ops are synchronous, so no fence to wait on: */ | 756 | /* VM_BIND ops are synchronous, so no fence to wait on: */ |
| 757 | return NULL; | 757 | return NULL; |
| @@ -1305,7 +1305,7 @@ vm_bind_job_pin_objects(struct msm_vm_bind_job *job) · 547 unmodified lines | |||
| 1305 | return PTR_ERR(pages); | 1305 | return PTR_ERR(pages); |
| 1306 | } | 1306 | } |
| 1307 | 1307 | ||
| 1308 | struct msm_drm_private *priv = job->vm->drm->dev_private; | 1308 | struct drm_device *dev = job->vm->drm; |
| 1309 | 1309 | ||
| 1310 | /* | 1310 | /* |
| 1311 | * A second loop while holding the LRU lock (a) avoids acquiring/dropping | 1311 | * A second loop while holding the LRU lock (a) avoids acquiring/dropping |
| @@ -1314,10 +1314,10 @@ vm_bind_job_pin_objects(struct msm_vm_bind_job *job) · 2 unmodified lines | |||
| 1314 | * get_pages() which could trigger reclaim.. and if we held the LRU lock | 1314 | * get_pages() which could trigger reclaim.. and if we held the LRU lock |
| 1315 | * could trigger deadlock with the shrinker). | 1315 | * could trigger deadlock with the shrinker). |
| 1316 | */ | 1316 | */ |
| 1317 | mutex_lock(&priv->lru.lock); | 1317 | mutex_lock(&dev->gem_lru_mutex); |
| 1318 | job_foreach_bo (obj, job) | 1318 | job_foreach_bo (obj, job) |
| 1319 | msm_gem_pin_obj_locked(obj); | 1319 | msm_gem_pin_obj_locked(obj); |
| 1320 | mutex_unlock(&priv->lru.lock); | 1320 | mutex_unlock(&dev->gem_lru_mutex); |
| 1321 | 1321 | ||
| 1322 | job->bos_pinned = true; | 1322 | job->bos_pinned = true; |
| 1323 | 1323 | ||
| @@ -13,11 +13,11 @@ · 12 unmodified lines | |||
| 13 | #include "msm_gpu_trace.h" | 13 | #include "msm_gpu_trace.h" |
| 14 | //#include "adreno/adreno_gpu.h" | 14 | //#include "adreno/adreno_gpu.h" |
| 15 | 15 | ||
| 16 | #include <generated/utsrelease.h> | ||
| 17 | #include <linux/string_helpers.h> | 16 | #include <linux/string_helpers.h> |
| 18 | #include <linux/devcoredump.h> | 17 | #include <linux/devcoredump.h> |
| 19 | #include <linux/sched/task.h> | 18 | #include <linux/sched/task.h> |
| 20 | #include <linux/sched/mm.h> | 19 | #include <linux/sched/mm.h> |
| 20 | #include <linux/utsname.h> | ||
| 21 | 21 | ||
| 22 | /* | 22 | /* |
| 23 | * Power Management: | 23 | * Power Management: |
| @@ -196,7 +196,7 @@ static ssize_t msm_gpu_devcoredump_read(char *buffer, loff_t offset, · 172 unmodified lines | |||
| 196 | p = drm_coredump_printer(&iter); | 196 | p = drm_coredump_printer(&iter); |
| 197 | 197 | ||
| 198 | drm_printf(&p, "---\n"); | 198 | drm_printf(&p, "---\n"); |
| 199 | drm_printf(&p, "kernel: " UTS_RELEASE "\n"); | 199 | drm_printf(&p, "kernel: %s\n", init_utsname()->release); |
| 200 | drm_printf(&p, "module: " KBUILD_MODNAME "\n"); | 200 | drm_printf(&p, "module: " KBUILD_MODNAME "\n"); |
| 201 | drm_printf(&p, "time: %ptSp\n", &state->time); | 201 | drm_printf(&p, "time: %ptSp\n", &state->time); |
| 202 | if (state->comm) | 202 | if (state->comm) |
| @@ -677,7 +677,7 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, · 676 unmodified lines | |||
| 677 | int prot) | 677 | int prot) |
| 678 | { | 678 | { |
| 679 | struct msm_iommu *iommu = to_msm_iommu(mmu); | 679 | struct msm_iommu *iommu = to_msm_iommu(mmu); |
| 680 | size_t ret; | 680 | ssize_t ret; |
| 681 | 681 | ||
| 682 | WARN_ON(off != 0); | 682 | WARN_ON(off != 0); |
| 683 | 683 | ||
| @@ -686,7 +686,8 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, · 2 unmodified lines | |||
| 686 | iova |= GENMASK_ULL(63, 49); | 686 | iova |= GENMASK_ULL(63, 49); |
| 687 | 687 | ||
| 688 | ret = iommu_map_sgtable(iommu->domain, iova, sgt, prot); | 688 | ret = iommu_map_sgtable(iommu->domain, iova, sgt, prot); |
| 689 | WARN_ON(!ret); | 689 | if (ret < 0) |
| 690 | return ret; | ||
| 690 | 691 | ||
| 691 | return (ret == len) ? 0 : -EINVAL; | 692 | return (ret == len) ? 0 : -EINVAL; |
| 692 | } | 693 | } |
| @@ -16,13 +16,13 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) · 15 unmodified lines | |||
| 16 | struct msm_gem_submit *submit = to_msm_submit(job); | 16 | struct msm_gem_submit *submit = to_msm_submit(job); |
| 17 | struct msm_fence_context *fctx = submit->ring->fctx; | 17 | struct msm_fence_context *fctx = submit->ring->fctx; |
| 18 | struct msm_gpu *gpu = submit->gpu; | 18 | struct msm_gpu *gpu = submit->gpu; |
| 19 | struct msm_drm_private *priv = gpu->dev->dev_private; | 19 | struct drm_device *dev = gpu->dev; |
| 20 | unsigned nr_cmds = submit->nr_cmds; | 20 | unsigned nr_cmds = submit->nr_cmds; |
| 21 | int i; | 21 | int i; |
| 22 | 22 | ||
| 23 | msm_fence_init(submit->hw_fence, fctx); | 23 | msm_fence_init(submit->hw_fence, fctx); |
| 24 | 24 | ||
| 25 | mutex_lock(&priv->lru.lock); | 25 | mutex_lock(&dev->gem_lru_mutex); |
| 26 | 26 | ||
| 27 | for (i = 0; i < submit->nr_bos; i++) { | 27 | for (i = 0; i < submit->nr_bos; i++) { |
| 28 | struct drm_gem_object *obj = submit->bos[i].obj; | 28 | struct drm_gem_object *obj = submit->bos[i].obj; |
| @@ -32,7 +32,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) · 3 unmodified lines | |||
| 32 | 32 | ||
| 33 | submit->bos_pinned = false; | 33 | submit->bos_pinned = false; |
| 34 | 34 | ||
| 35 | mutex_unlock(&priv->lru.lock); | 35 | mutex_unlock(&dev->gem_lru_mutex); |
| 36 | 36 | ||
| 37 | /* TODO move submit path over to using a per-ring lock.. */ | 37 | /* TODO move submit path over to using a per-ring lock.. */ |
| 38 | mutex_lock(&gpu->lock); | 38 | mutex_lock(&gpu->lock); |
| @@ -2513,6 +2513,7 @@ static const struct nvkm_device_chip · 2512 unmodified lines | |||
| 2513 | nv170_chipset = { | 2513 | nv170_chipset = { |
| 2514 | .name = "GA100", | 2514 | .name = "GA100", |
| 2515 | .bar = { 0x00000001, tu102_bar_new }, | 2515 | .bar = { 0x00000001, tu102_bar_new }, |
| 2516 | .bios = { 0x00000001, nvkm_bios_new }, | ||
| 2516 | .devinit = { 0x00000001, ga100_devinit_new }, | 2517 | .devinit = { 0x00000001, ga100_devinit_new }, |
| 2517 | .fault = { 0x00000001, tu102_fault_new }, | 2518 | .fault = { 0x00000001, tu102_fault_new }, |
| 2518 | .fb = { 0x00000001, ga100_fb_new }, | 2519 | .fb = { 0x00000001, ga100_fb_new }, |
| @@ -2529,7 +2530,6 @@ nv170_chipset = { · 10 unmodified lines | |||
| 2529 | .vfn = { 0x00000001, ga100_vfn_new }, | 2530 | .vfn = { 0x00000001, ga100_vfn_new }, |
| 2530 | .ce = { 0x000003ff, ga100_ce_new }, | 2531 | .ce = { 0x000003ff, ga100_ce_new }, |
| 2531 | .fifo = { 0x00000001, ga100_fifo_new }, | 2532 | .fifo = { 0x00000001, ga100_fifo_new }, |
| 2532 | .sec2 = { 0x00000001, tu102_sec2_new }, | ||
| 2533 | }; | 2533 | }; |
| 2534 | 2534 | ||
| 2535 | static const struct nvkm_device_chip | 2535 | static const struct nvkm_device_chip |
| @@ -3341,7 +3341,6 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 805 unmodified lines | |||
| 3341 | case 0x166: device->chip = &nv166_chipset; break; | 3341 | case 0x166: device->chip = &nv166_chipset; break; |
| 3342 | case 0x167: device->chip = &nv167_chipset; break; | 3342 | case 0x167: device->chip = &nv167_chipset; break; |
| 3343 | case 0x168: device->chip = &nv168_chipset; break; | 3343 | case 0x168: device->chip = &nv168_chipset; break; |
| 3344 | case 0x170: device->chip = &nv170_chipset; break; | ||
| 3345 | case 0x172: device->chip = &nv172_chipset; break; | 3344 | case 0x172: device->chip = &nv172_chipset; break; |
| 3346 | case 0x173: device->chip = &nv173_chipset; break; | 3345 | case 0x173: device->chip = &nv173_chipset; break; |
| 3347 | case 0x174: device->chip = &nv174_chipset; break; | 3346 | case 0x174: device->chip = &nv174_chipset; break; |
| @@ -3361,6 +3360,14 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 13 unmodified lines | |||
| 3361 | case 0x1b6: device->chip = &nv1b6_chipset; break; | 3360 | case 0x1b6: device->chip = &nv1b6_chipset; break; |
| 3362 | case 0x1b7: device->chip = &nv1b7_chipset; break; | 3361 | case 0x1b7: device->chip = &nv1b7_chipset; break; |
| 3363 | default: | 3362 | default: |
| 3363 | if (nvkm_boolopt(device->cfgopt, "NvEnableUnsupportedChipsets", false)) { | ||
| 3364 | switch (device->chipset) { | ||
| 3365 | case 0x170: device->chip = &nv170_chipset; break; | ||
| 3366 | default: | ||
| 3367 | break; | ||
| 3368 | } | ||
| 3369 | } | ||
| 3370 | |||
| 3364 | if (!device->chip) { | 3371 | if (!device->chip) { |
| 3365 | nvdev_error(device, "unknown chipset (%08x)\n", boot0); | 3372 | nvdev_error(device, "unknown chipset (%08x)\n", boot0); |
| 3366 | ret = -ENODEV; | 3373 | ret = -ENODEV; |
| @@ -41,11 +41,15 @@ ga100_gsp_flcn = { · 40 unmodified lines | |||
| 41 | static const struct nvkm_gsp_func | 41 | static const struct nvkm_gsp_func |
| 42 | ga100_gsp = { | 42 | ga100_gsp = { |
| 43 | .flcn = &ga100_gsp_flcn, | 43 | .flcn = &ga100_gsp_flcn, |
| 44 | .fwsec = &tu102_gsp_fwsec, | ||
| 44 | 45 | ||
| 45 | .sig_section = ".fwsignature_ga100", | 46 | .sig_section = ".fwsignature_ga100", |
| 46 | 47 | ||
| 47 | .booter.ctor = tu102_gsp_booter_ctor, | 48 | .booter.ctor = tu102_gsp_booter_ctor, |
| 48 | 49 | ||
| 50 | .fwsec_sb.ctor = tu102_gsp_fwsec_sb_ctor, | ||
| 51 | .fwsec_sb.dtor = tu102_gsp_fwsec_sb_dtor, | ||
| 52 | |||
| 49 | .dtor = r535_gsp_dtor, | 53 | .dtor = r535_gsp_dtor, |
| 50 | .oneinit = tu102_gsp_oneinit, | 54 | .oneinit = tu102_gsp_oneinit, |
| 51 | .init = tu102_gsp_init, | 55 | .init = tu102_gsp_init, |
| @@ -318,13 +318,8 @@ tu102_gsp_oneinit(struct nvkm_gsp *gsp) · 317 unmodified lines | |||
| 318 | if (ret) | 318 | if (ret) |
| 319 | return ret; | 319 | return ret; |
| 320 | 320 | ||
| 321 | /* | 321 | /* Calculate FB layout. */ |
| 322 | * Calculate FB layout. FRTS is a memory region created by the FWSEC-FRTS firmware. | 322 | gsp->fb.wpr2.frts.size = 0x100000; |
| 323 | * FWSEC comes from VBIOS. So on systems with no VBIOS (e.g. GA100), the FRTS does | ||
| 324 | * not exist. Therefore, use the existence of VBIOS to determine whether to reserve | ||
| 325 | * an FRTS region. | ||
| 326 | */ | ||
| 327 | gsp->fb.wpr2.frts.size = device->bios ? 0x100000 : 0; | ||
| 328 | gsp->fb.wpr2.frts.addr = ALIGN_DOWN(gsp->fb.bios.addr, 0x20000) - gsp->fb.wpr2.frts.size; | 323 | gsp->fb.wpr2.frts.addr = ALIGN_DOWN(gsp->fb.bios.addr, 0x20000) - gsp->fb.wpr2.frts.size; |
| 329 | 324 | ||
| 330 | gsp->fb.wpr2.boot.size = gsp->boot.fw.size; | 325 | gsp->fb.wpr2.boot.size = gsp->boot.fw.size; |
| @@ -348,12 +343,9 @@ tu102_gsp_oneinit(struct nvkm_gsp *gsp) · 17 unmodified lines | |||
| 348 | if (ret) | 343 | if (ret) |
| 349 | return ret; | 344 | return ret; |
| 350 | 345 | ||
| 351 | /* Only boot FWSEC-FRTS if it actually exists */ | 346 | ret = nvkm_gsp_fwsec_frts(gsp); |
| 352 | if (gsp->fb.wpr2.frts.size) { | 347 | if (WARN_ON(ret)) |
| 353 | ret = nvkm_gsp_fwsec_frts(gsp); | 348 | return ret; |
| 354 | if (WARN_ON(ret)) | ||
| 355 | return ret; | ||
| 356 | } | ||
| 357 | 349 | ||
| 358 | /* Reset GSP into RISC-V mode. */ | 350 | /* Reset GSP into RISC-V mode. */ |
| 359 | ret = gsp->func->reset(gsp); | 351 | ret = gsp->func->reset(gsp); |
| @@ -208,6 +208,7 @@ config DRM_PANEL_HIMAX_HX83121A · 207 unmodified lines | |||
| 208 | depends on OF | 208 | depends on OF |
| 209 | depends on DRM_MIPI_DSI | 209 | depends on DRM_MIPI_DSI |
| 210 | depends on BACKLIGHT_CLASS_DEVICE | 210 | depends on BACKLIGHT_CLASS_DEVICE |
| 211 | select DRM_DISPLAY_DSC_HELPER | ||
| 211 | select DRM_KMS_HELPER | 212 | select DRM_KMS_HELPER |
| 212 | help | 213 | help |
| 213 | Say Y here if you want to enable support for Himax HX83121A-based | 214 | Say Y here if you want to enable support for Himax HX83121A-based |
| @@ -1324,6 +1324,8 @@ static int boe_panel_disable(struct drm_panel *panel) · 1323 unmodified lines | |||
| 1324 | mipi_dsi_dcs_set_display_off_multi(&ctx); | 1324 | mipi_dsi_dcs_set_display_off_multi(&ctx); |
| 1325 | mipi_dsi_dcs_enter_sleep_mode_multi(&ctx); | 1325 | mipi_dsi_dcs_enter_sleep_mode_multi(&ctx); |
| 1326 | 1326 | ||
| 1327 | boe->dsi->mode_flags |= MIPI_DSI_MODE_LPM; | ||
| 1328 | |||
| 1327 | mipi_dsi_msleep(&ctx, 150); | 1329 | mipi_dsi_msleep(&ctx, 150); |
| 1328 | 1330 | ||
| 1329 | return ctx.accum_err; | 1331 | return ctx.accum_err; |
| @@ -98,9 +98,7 @@ static int feiyang_enable(struct drm_panel *panel) · 97 unmodified lines | |||
| 98 | /* T12 (video & logic signal rise + backlight rise) T12 >= 200ms */ | 98 | /* T12 (video & logic signal rise + backlight rise) T12 >= 200ms */ |
| 99 | msleep(200); | 99 | msleep(200); |
| 100 | 100 | ||
| 101 | mipi_dsi_dcs_set_display_on(ctx->dsi); | 101 | return mipi_dsi_dcs_set_display_on(ctx->dsi); |
| 102 | |||
| 103 | return 0; | ||
| 104 | } | 102 | } |
| 105 | 103 | ||
| 106 | static int feiyang_disable(struct drm_panel *panel) | 104 | static int feiyang_disable(struct drm_panel *panel) |
| @@ -937,6 +937,8 @@ static int hx83102_disable(struct drm_panel *panel) · 936 unmodified lines | |||
| 937 | mipi_dsi_dcs_set_display_off_multi(&dsi_ctx); | 937 | mipi_dsi_dcs_set_display_off_multi(&dsi_ctx); |
| 938 | mipi_dsi_dcs_enter_sleep_mode_multi(&dsi_ctx); | 938 | mipi_dsi_dcs_enter_sleep_mode_multi(&dsi_ctx); |
| 939 | 939 | ||
| 940 | dsi->mode_flags |= MIPI_DSI_MODE_LPM; | ||
| 941 | |||
| 940 | mipi_dsi_msleep(&dsi_ctx, 150); | 942 | mipi_dsi_msleep(&dsi_ctx, 150); |
| 941 | 943 | ||
| 942 | return dsi_ctx.accum_err; | 944 | return dsi_ctx.accum_err; |
| @@ -596,8 +596,8 @@ static int himax_probe(struct mipi_dsi_device *dsi) · 595 unmodified lines | |||
| 596 | 596 | ||
| 597 | ctx = devm_drm_panel_alloc(dev, struct himax, panel, &himax_panel_funcs, | 597 | ctx = devm_drm_panel_alloc(dev, struct himax, panel, &himax_panel_funcs, |
| 598 | DRM_MODE_CONNECTOR_DSI); | 598 | DRM_MODE_CONNECTOR_DSI); |
| 599 | if (!ctx) | 599 | if (IS_ERR(ctx)) |
| 600 | return -ENOMEM; | 600 | return PTR_ERR(ctx); |
| 601 | 601 | ||
| 602 | ret = devm_regulator_bulk_get_const(&dsi->dev, | 602 | ret = devm_regulator_bulk_get_const(&dsi->dev, |
| 603 | ARRAY_SIZE(himax_supplies), | 603 | ARRAY_SIZE(himax_supplies), |
| @@ -390,6 +390,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data, · 389 unmodified lines | |||
| 390 | true, timeout); | 390 | true, timeout); |
| 391 | if (!ret) | 391 | if (!ret) |
| 392 | ret = timeout ? -ETIMEDOUT : -EBUSY; | 392 | ret = timeout ? -ETIMEDOUT : -EBUSY; |
| 393 | else if (ret > 0) | ||
| 394 | ret = 0; | ||
| 393 | 395 | ||
| 394 | drm_gem_object_put(gem_obj); | 396 | drm_gem_object_put(gem_obj); |
| 395 | 397 | ||
| @@ -118,12 +118,13 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 117 unmodified lines | |||
| 118 | /* Complete initialization. */ | 118 | /* Complete initialization. */ |
| 119 | ret = drm_dev_register(&qdev->ddev, ent->driver_data); | 119 | ret = drm_dev_register(&qdev->ddev, ent->driver_data); |
| 120 | if (ret) | 120 | if (ret) |
| 121 | goto modeset_cleanup; | 121 | goto poll_fini; |
| 122 | 122 | ||
| 123 | drm_client_setup(&qdev->ddev, NULL); | 123 | drm_client_setup(&qdev->ddev, NULL); |
| 124 | return 0; | 124 | return 0; |
| 125 | 125 | ||
| 126 | modeset_cleanup: | 126 | poll_fini: |
| 127 | drm_kms_helper_poll_fini(&qdev->ddev); | ||
| 127 | qxl_modeset_fini(qdev); | 128 | qxl_modeset_fini(qdev); |
| 128 | unload: | 129 | unload: |
| 129 | qxl_device_fini(qdev); | 130 | qxl_device_fini(qdev); |
| @@ -154,6 +155,7 @@ qxl_pci_remove(struct pci_dev *pdev) · 24 unmodified lines | |||
| 154 | { | 155 | { |
| 155 | struct drm_device *dev = pci_get_drvdata(pdev); | 156 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 156 | 157 | ||
| 158 | drm_kms_helper_poll_fini(dev); | ||
| 157 | drm_dev_unregister(dev); | 159 | drm_dev_unregister(dev); |
| 158 | drm_atomic_helper_shutdown(dev); | 160 | drm_atomic_helper_shutdown(dev); |
| 159 | if (pci_is_vga(pdev) && pdev->revision < 5) | 161 | if (pci_is_vga(pdev) && pdev->revision < 5) |
| @@ -2461,7 +2461,8 @@ static void ci_register_patching_mc_arb(struct radeon_device *rdev, · 2460 unmodified lines | |||
| 2461 | 2461 | ||
| 2462 | if (patch && | 2462 | if (patch && |
| 2463 | ((rdev->pdev->device == 0x67B0) || | 2463 | ((rdev->pdev->device == 0x67B0) || |
| 2464 | (rdev->pdev->device == 0x67B1))) { | 2464 | (rdev->pdev->device == 0x67B1)) && |
| 2465 | (rdev->pdev->revision == 0)) { | ||
| 2465 | if ((memory_clock > 100000) && (memory_clock <= 125000)) { | 2466 | if ((memory_clock > 100000) && (memory_clock <= 125000)) { |
| 2466 | tmp2 = (((0x31 * engine_clock) / 125000) - 1) & 0xff; | 2467 | tmp2 = (((0x31 * engine_clock) / 125000) - 1) & 0xff; |
| 2467 | *dram_timimg2 &= ~0x00ff0000; | 2468 | *dram_timimg2 &= ~0x00ff0000; |
| @@ -3304,7 +3305,8 @@ static int ci_populate_all_memory_levels(struct radeon_device *rdev) · 836 unmodified lines | |||
| 3304 | pi->smc_state_table.MemoryLevel[0].EnabledForActivity = 1; | 3305 | pi->smc_state_table.MemoryLevel[0].EnabledForActivity = 1; |
| 3305 | 3306 | ||
| 3306 | if ((dpm_table->mclk_table.count >= 2) && | 3307 | if ((dpm_table->mclk_table.count >= 2) && |
| 3307 | ((rdev->pdev->device == 0x67B0) || (rdev->pdev->device == 0x67B1))) { | 3308 | ((rdev->pdev->device == 0x67B0) || (rdev->pdev->device == 0x67B1)) && |
| 3309 | (rdev->pdev->revision == 0)) { | ||
| 3308 | pi->smc_state_table.MemoryLevel[1].MinVddc = | 3310 | pi->smc_state_table.MemoryLevel[1].MinVddc = |
| 3309 | pi->smc_state_table.MemoryLevel[0].MinVddc; | 3311 | pi->smc_state_table.MemoryLevel[0].MinVddc; |
| 3310 | pi->smc_state_table.MemoryLevel[1].MinVddcPhases = | 3312 | pi->smc_state_table.MemoryLevel[1].MinVddcPhases = |
| @@ -4493,7 +4495,8 @@ static int ci_register_patching_mc_seq(struct radeon_device *rdev, · 1182 unmodified lines | |||
| 4493 | 4495 | ||
| 4494 | if (patch && | 4496 | if (patch && |
| 4495 | ((rdev->pdev->device == 0x67B0) || | 4497 | ((rdev->pdev->device == 0x67B0) || |
| 4496 | (rdev->pdev->device == 0x67B1))) { | 4498 | (rdev->pdev->device == 0x67B1)) && |
| 4499 | (rdev->pdev->revision == 0)) { | ||
| 4497 | for (i = 0; i < table->last; i++) { | 4500 | for (i = 0; i < table->last; i++) { |
| 4498 | if (table->last >= SMU7_DISCRETE_MC_REGISTER_ARRAY_SIZE) | 4501 | if (table->last >= SMU7_DISCRETE_MC_REGISTER_ARRAY_SIZE) |
| 4499 | return -EINVAL; | 4502 | return -EINVAL; |
| @@ -312,8 +312,10 @@ static int evergreen_surface_check(struct radeon_cs_parser *p, · 311 unmodified lines | |||
| 312 | case ARRAY_2D_TILED_THIN1: | 312 | case ARRAY_2D_TILED_THIN1: |
| 313 | return evergreen_surface_check_2d(p, surf, prefix); | 313 | return evergreen_surface_check_2d(p, surf, prefix); |
| 314 | default: | 314 | default: |
| 315 | dev_warn(p->dev, "%s:%d %s invalid array mode %d\n", | 315 | if (prefix) { |
| 316 | __func__, __LINE__, prefix, surf->mode); | 316 | dev_warn(p->dev, "%s:%d %s invalid array mode %d\n", |
| 317 | __func__, __LINE__, prefix, surf->mode); | ||
| 318 | } | ||
| 317 | return -EINVAL; | 319 | return -EINVAL; |
| 318 | } | 320 | } |
| 319 | return -EINVAL; | 321 | return -EINVAL; |
| @@ -741,6 +741,7 @@ static int sti_hda_probe(struct platform_device *pdev) · 740 unmodified lines | |||
| 741 | struct device *dev = &pdev->dev; | 741 | struct device *dev = &pdev->dev; |
| 742 | struct sti_hda *hda; | 742 | struct sti_hda *hda; |
| 743 | struct resource *res; | 743 | struct resource *res; |
| 744 | int ret; | ||
| 744 | 745 | ||
| 745 | DRM_INFO("%s\n", __func__); | 746 | DRM_INFO("%s\n", __func__); |
| 746 | 747 | ||
| @@ -779,7 +780,9 @@ static int sti_hda_probe(struct platform_device *pdev) · 32 unmodified lines | |||
| 779 | return PTR_ERR(hda->clk_hddac); | 780 | return PTR_ERR(hda->clk_hddac); |
| 780 | } | 781 | } |
| 781 | 782 | ||
| 782 | drm_bridge_add(&hda->bridge); | 783 | ret = devm_drm_bridge_add(dev, &hda->bridge); |
| 784 | if (ret) | ||
| 785 | return ret; | ||
| 783 | 786 | ||
| 784 | platform_set_drvdata(pdev, hda); | 787 | platform_set_drvdata(pdev, hda); |
| 785 | 788 | ||
| @@ -788,10 +791,7 @@ static int sti_hda_probe(struct platform_device *pdev) · 2 unmodified lines | |||
| 788 | 791 | ||
| 789 | static void sti_hda_remove(struct platform_device *pdev) | 792 | static void sti_hda_remove(struct platform_device *pdev) |
| 790 | { | 793 | { |
| 791 | struct sti_hda *hda = platform_get_drvdata(pdev); | ||
| 792 | |||
| 793 | component_del(&pdev->dev, &sti_hda_ops); | 794 | component_del(&pdev->dev, &sti_hda_ops); |
| 794 | drm_bridge_remove(&hda->bridge); | ||
| 795 | } | 795 | } |
| 796 | 796 | ||
| 797 | static const struct of_device_id hda_of_match[] = { | 797 | static const struct of_device_id hda_of_match[] = { |
| @@ -350,6 +350,7 @@ static void ofdrm_pci_release(void *data) · 349 unmodified lines | |||
| 350 | struct pci_dev *pcidev = data; | 350 | struct pci_dev *pcidev = data; |
| 351 | 351 | ||
| 352 | pci_disable_device(pcidev); | 352 | pci_disable_device(pcidev); |
| 353 | pci_dev_put(pcidev); | ||
| 353 | } | 354 | } |
| 354 | 355 | ||
| 355 | static int ofdrm_device_init_pci(struct ofdrm_device *odev) | 356 | static int ofdrm_device_init_pci(struct ofdrm_device *odev) |
| @@ -375,6 +376,7 @@ static int ofdrm_device_init_pci(struct ofdrm_device *odev) · 19 unmodified lines | |||
| 375 | if (ret) { | 376 | if (ret) { |
| 376 | drm_err(dev, "pci_enable_device(%s) failed: %d\n", | 377 | drm_err(dev, "pci_enable_device(%s) failed: %d\n", |
| 377 | dev_name(&pcidev->dev), ret); | 378 | dev_name(&pcidev->dev), ret); |
| 379 | pci_dev_put(pcidev); | ||
| 378 | return ret; | 380 | return ret; |
| 379 | } | 381 | } |
| 380 | ret = devm_add_action_or_reset(&pdev->dev, ofdrm_pci_release, pcidev); | 382 | ret = devm_add_action_or_reset(&pdev->dev, ofdrm_pci_release, pcidev); |
| @@ -353,7 +353,7 @@ static int appletbdrm_primary_plane_helper_atomic_check(struct drm_plane *plane, · 352 unmodified lines | |||
| 353 | frames_size + | 353 | frames_size + |
| 354 | sizeof(struct appletbdrm_fb_request_footer), 16); | 354 | sizeof(struct appletbdrm_fb_request_footer), 16); |
| 355 | 355 | ||
| 356 | appletbdrm_state->request = kzalloc(request_size, GFP_KERNEL); | 356 | appletbdrm_state->request = kvzalloc(request_size, GFP_KERNEL); |
| 357 | 357 | ||
| 358 | if (!appletbdrm_state->request) | 358 | if (!appletbdrm_state->request) |
| 359 | return -ENOMEM; | 359 | return -ENOMEM; |
| @@ -543,7 +543,7 @@ static void appletbdrm_primary_plane_destroy_state(struct drm_plane *plane, · 183 unmodified lines | |||
| 543 | { | 543 | { |
| 544 | struct appletbdrm_plane_state *appletbdrm_state = to_appletbdrm_plane_state(state); | 544 | struct appletbdrm_plane_state *appletbdrm_state = to_appletbdrm_plane_state(state); |
| 545 | 545 | ||
| 546 | kfree(appletbdrm_state->request); | 546 | kvfree(appletbdrm_state->request); |
| 547 | kfree(appletbdrm_state->response); | 547 | kfree(appletbdrm_state->response); |
| 548 | 548 | ||
| 549 | __drm_gem_destroy_shadow_plane_state(&appletbdrm_state->base); | 549 | __drm_gem_destroy_shadow_plane_state(&appletbdrm_state->base); |
| @@ -761,25 +761,21 @@ static int bochs_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent · 760 unmodified lines | |||
| 761 | 761 | ||
| 762 | ret = pcim_enable_device(pdev); | 762 | ret = pcim_enable_device(pdev); |
| 763 | if (ret) | 763 | if (ret) |
| 764 | goto err_free_dev; | 764 | return ret; |
| 765 | 765 | ||
| 766 | pci_set_drvdata(pdev, dev); | 766 | pci_set_drvdata(pdev, dev); |
| 767 | 767 | ||
| 768 | ret = bochs_load(bochs); | 768 | ret = bochs_load(bochs); |
| 769 | if (ret) | 769 | if (ret) |
| 770 | goto err_free_dev; | 770 | return ret; |
| 771 | 771 | ||
| 772 | ret = drm_dev_register(dev, 0); | 772 | ret = drm_dev_register(dev, 0); |
| 773 | if (ret) | 773 | if (ret) |
| 774 | goto err_free_dev; | 774 | return ret; |
| 775 | 775 | ||
| 776 | drm_client_setup(dev, NULL); | 776 | drm_client_setup(dev, NULL); |
| 777 | 777 | ||
| 778 | return ret; | 778 | return ret; |
| 779 | |||
| 780 | err_free_dev: | ||
| 781 | drm_dev_put(dev); | ||
| 782 | return ret; | ||
| 783 | } | 779 | } |
| 784 | 780 | ||
| 785 | static void bochs_pci_remove(struct pci_dev *pdev) | 781 | static void bochs_pci_remove(struct pci_dev *pdev) |
| @@ -739,7 +739,7 @@ static int ttm_bo_alloc_resource(struct ttm_buffer_object *bo, · 738 unmodified lines | |||
| 739 | may_evict = (force_space && place->mem_type != TTM_PL_SYSTEM); | 739 | may_evict = (force_space && place->mem_type != TTM_PL_SYSTEM); |
| 740 | ret = ttm_resource_alloc(bo, place, res, force_space ? &limit_pool : NULL); | 740 | ret = ttm_resource_alloc(bo, place, res, force_space ? &limit_pool : NULL); |
| 741 | if (ret) { | 741 | if (ret) { |
| 742 | if (ret != -ENOSPC && ret != -EAGAIN) { | 742 | if (ret != -ENOSPC) { |
| 743 | dmem_cgroup_pool_state_put(limit_pool); | 743 | dmem_cgroup_pool_state_put(limit_pool); |
| 744 | return ret; | 744 | return ret; |
| 745 | } | 745 | } |
| @@ -1177,17 +1177,13 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 431 unmodified lines | |||
| 1177 | bdev->funcs->swap_notify(bo); | 1177 | bdev->funcs->swap_notify(bo); |
| 1178 | 1178 | ||
| 1179 | if (ttm_tt_is_populated(tt)) { | 1179 | if (ttm_tt_is_populated(tt)) { |
| 1180 | spin_lock(&bdev->lru_lock); | ||
| 1181 | ttm_resource_del_bulk_move(bo->resource, bo); | ||
| 1182 | spin_unlock(&bdev->lru_lock); | ||
| 1183 | |||
| 1184 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); | 1180 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); |
| 1185 | 1181 | if (!ret) { | |
| 1186 | spin_lock(&bdev->lru_lock); | 1182 | spin_lock(&bdev->lru_lock); |
| 1187 | if (ret) | 1183 | ttm_resource_del_bulk_move_unevictable(bo->resource, bo); |
| 1188 | ttm_resource_add_bulk_move(bo->resource, bo); | 1184 | ttm_resource_move_to_lru_tail(bo->resource); |
| 1189 | ttm_resource_move_to_lru_tail(bo->resource); | 1185 | spin_unlock(&bdev->lru_lock); |
| 1190 | spin_unlock(&bdev->lru_lock); | 1186 | } |
| 1191 | } | 1187 | } |
| 1192 | 1188 | ||
| 1193 | out: | 1189 | out: |
| @@ -1112,19 +1112,14 @@ long ttm_bo_shrink(struct ttm_operation_ctx *ctx, struct ttm_buffer_object *bo, · 1111 unmodified lines | |||
| 1112 | if (lret < 0) | 1112 | if (lret < 0) |
| 1113 | return lret; | 1113 | return lret; |
| 1114 | 1114 | ||
| 1115 | if (bo->bulk_move) { | ||
| 1116 | spin_lock(&bdev->lru_lock); | ||
| 1117 | ttm_resource_del_bulk_move(bo->resource, bo); | ||
| 1118 | spin_unlock(&bdev->lru_lock); | ||
| 1119 | } | ||
| 1120 | |||
| 1121 | lret = ttm_tt_backup(bdev, bo->ttm, (struct ttm_backup_flags) | 1115 | lret = ttm_tt_backup(bdev, bo->ttm, (struct ttm_backup_flags) |
| 1122 | {.purge = flags.purge, | 1116 | {.purge = flags.purge, |
| 1123 | .writeback = flags.writeback}); | 1117 | .writeback = flags.writeback}); |
| 1124 | 1118 | ||
| 1125 | if (lret <= 0 && bo->bulk_move) { | 1119 | if (lret > 0) { |
| 1126 | spin_lock(&bdev->lru_lock); | 1120 | spin_lock(&bdev->lru_lock); |
| 1127 | ttm_resource_add_bulk_move(bo->resource, bo); | 1121 | ttm_resource_del_bulk_move_unevictable(bo->resource, bo); |
| 1122 | ttm_resource_move_to_lru_tail(bo->resource); | ||
| 1128 | spin_unlock(&bdev->lru_lock); | 1123 | spin_unlock(&bdev->lru_lock); |
| 1129 | } | 1124 | } |
| 1130 | 1125 | ||
| @@ -206,6 +206,14 @@ error_free: · 205 unmodified lines | |||
| 206 | return NULL; | 206 | return NULL; |
| 207 | } | 207 | } |
| 208 | 208 | ||
| 209 | static void __free_pages_gpu_account(struct page *p, unsigned int order, | ||
| 210 | bool reclaim) | ||
| 211 | { | ||
| 212 | mod_lruvec_page_state(p, reclaim ? NR_GPU_RECLAIM : NR_GPU_ACTIVE, | ||
| 213 | -(1 << order)); | ||
| 214 | __free_pages(p, order); | ||
| 215 | } | ||
| 216 | |||
| 209 | /* Reset the caching and pages of size 1 << order */ | 217 | /* Reset the caching and pages of size 1 << order */ |
| 210 | static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching caching, | 218 | static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching caching, |
| 211 | unsigned int order, struct page *p, bool reclaim) | 219 | unsigned int order, struct page *p, bool reclaim) |
| @@ -223,9 +231,7 @@ static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching caching, · 11 unmodified lines | |||
| 223 | #endif | 231 | #endif |
| 224 | 232 | ||
| 225 | if (!pool || !ttm_pool_uses_dma_alloc(pool)) { | 233 | if (!pool || !ttm_pool_uses_dma_alloc(pool)) { |
| 226 | mod_lruvec_page_state(p, reclaim ? NR_GPU_RECLAIM : NR_GPU_ACTIVE, | 234 | __free_pages_gpu_account(p, order, reclaim); |
| 227 | -(1 << order)); | ||
| 228 | __free_pages(p, order); | ||
| 229 | return; | 235 | return; |
| 230 | } | 236 | } |
| 231 | 237 | ||
| @@ -606,7 +612,7 @@ static int ttm_pool_restore_commit(struct ttm_pool_tt_restore *restore, · 374 unmodified lines | |||
| 606 | */ | 612 | */ |
| 607 | ttm_pool_split_for_swap(restore->pool, p); | 613 | ttm_pool_split_for_swap(restore->pool, p); |
| 608 | copy_highpage(restore->alloced_page + i, p); | 614 | copy_highpage(restore->alloced_page + i, p); |
| 609 | __free_pages(p, 0); | 615 | __free_pages_gpu_account(p, 0, false); |
| 610 | } | 616 | } |
| 611 | 617 | ||
| 612 | restore->restored_pages++; | 618 | restore->restored_pages++; |
| @@ -1068,7 +1074,7 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, · 455 unmodified lines | |||
| 1068 | if (flags->purge) { | 1074 | if (flags->purge) { |
| 1069 | shrunken += num_pages; | 1075 | shrunken += num_pages; |
| 1070 | page->private = 0; | 1076 | page->private = 0; |
| 1071 | __free_pages(page, order); | 1077 | __free_pages_gpu_account(page, order, false); |
| 1072 | memset(tt->pages + i, 0, | 1078 | memset(tt->pages + i, 0, |
| 1073 | num_pages * sizeof(*tt->pages)); | 1079 | num_pages * sizeof(*tt->pages)); |
| 1074 | } | 1080 | } |
| @@ -1109,7 +1115,7 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, · 34 unmodified lines | |||
| 1109 | } | 1115 | } |
| 1110 | handle = shandle; | 1116 | handle = shandle; |
| 1111 | tt->pages[i] = ttm_backup_handle_to_page_ptr(handle); | 1117 | tt->pages[i] = ttm_backup_handle_to_page_ptr(handle); |
| 1112 | put_page(page); | 1118 | __free_pages_gpu_account(page, 0, false); |
| 1113 | shrunken++; | 1119 | shrunken++; |
| 1114 | } | 1120 | } |
| 1115 | 1121 | ||
| @@ -292,6 +292,19 @@ void ttm_resource_del_bulk_move(struct ttm_resource *res, · 291 unmodified lines | |||
| 292 | ttm_lru_bulk_move_del(bo->bulk_move, res); | 292 | ttm_lru_bulk_move_del(bo->bulk_move, res); |
| 293 | } | 293 | } |
| 294 | 294 | ||
| 295 | /* | ||
| 296 | * Remove a resource from its bulk_move, bypassing the unevictable check. | ||
| 297 | * Use only when the resource is known to still be tracked in the range despite | ||
| 298 | * the BO having just become unevictable; asserts that this is the case. | ||
| 299 | */ | ||
| 300 | void ttm_resource_del_bulk_move_unevictable(struct ttm_resource *res, | ||
| 301 | struct ttm_buffer_object *bo) | ||
| 302 | { | ||
| 303 | WARN_ON_ONCE(!ttm_resource_unevictable(res, bo)); | ||
| 304 | if (bo->bulk_move) | ||
| 305 | ttm_lru_bulk_move_del(bo->bulk_move, res); | ||
| 306 | } | ||
| 307 | |||
| 295 | /* Move a resource to the LRU or bulk tail */ | 308 | /* Move a resource to the LRU or bulk tail */ |
| 296 | void ttm_resource_move_to_lru_tail(struct ttm_resource *res) | 309 | void ttm_resource_move_to_lru_tail(struct ttm_resource *res) |
| 297 | { | 310 | { |
| @@ -385,8 +398,11 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo, · 87 unmodified lines | |||
| 385 | 398 | ||
| 386 | if (man->cg) { | 399 | if (man->cg) { |
| 387 | ret = dmem_cgroup_try_charge(man->cg, bo->base.size, &pool, ret_limit_pool); | 400 | ret = dmem_cgroup_try_charge(man->cg, bo->base.size, &pool, ret_limit_pool); |
| 388 | if (ret) | 401 | if (ret) { |
| 402 | if (ret == -EAGAIN) | ||
| 403 | ret = -ENOSPC; | ||
| 389 | return ret; | 404 | return ret; |
| 405 | } | ||
| 390 | } | 406 | } |
| 391 | 407 | ||
| 392 | ret = man->func->alloc(man, bo, place, res_ptr); | 408 | ret = man->func->alloc(man, bo, place, res_ptr); |
| @@ -285,13 +285,12 @@ static struct urb *udl_get_urb_locked(struct udl_device *udl, long timeout) · 284 unmodified lines | |||
| 285 | return unode->urb; | 285 | return unode->urb; |
| 286 | } | 286 | } |
| 287 | 287 | ||
| 288 | #define GET_URB_TIMEOUT HZ | ||
| 289 | struct urb *udl_get_urb(struct udl_device *udl) | 288 | struct urb *udl_get_urb(struct udl_device *udl) |
| 290 | { | 289 | { |
| 291 | struct urb *urb; | 290 | struct urb *urb; |
| 292 | 291 | ||
| 293 | spin_lock_irq(&udl->urbs.lock); | 292 | spin_lock_irq(&udl->urbs.lock); |
| 294 | urb = udl_get_urb_locked(udl, GET_URB_TIMEOUT); | 293 | urb = udl_get_urb_locked(udl, HZ * 2); |
| 295 | spin_unlock_irq(&udl->urbs.lock); | 294 | spin_unlock_irq(&udl->urbs.lock); |
| 296 | return urb; | 295 | return urb; |
| 297 | } | 296 | } |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | |||
| 21 | #include <drm/drm_gem_framebuffer_helper.h> | 21 | #include <drm/drm_gem_framebuffer_helper.h> |
| 22 | #include <drm/drm_gem_shmem_helper.h> | 22 | #include <drm/drm_gem_shmem_helper.h> |
| 23 | #include <drm/drm_modeset_helper_vtables.h> | 23 | #include <drm/drm_modeset_helper_vtables.h> |
| 24 | #include <drm/drm_print.h> | ||
| 24 | #include <drm/drm_probe_helper.h> | 25 | #include <drm/drm_probe_helper.h> |
| 25 | #include <drm/drm_vblank.h> | 26 | #include <drm/drm_vblank.h> |
| 26 | 27 | ||
| @@ -342,8 +343,10 @@ static void udl_crtc_helper_atomic_enable(struct drm_crtc *crtc, struct drm_atom · 315 unmodified lines | |||
| 342 | return; | 343 | return; |
| 343 | 344 | ||
| 344 | urb = udl_get_urb(udl); | 345 | urb = udl_get_urb(udl); |
| 345 | if (!urb) | 346 | if (!urb) { |
| 347 | drm_err_ratelimited(dev, "get urb failed when enabling crtc\n"); | ||
| 346 | goto out; | 348 | goto out; |
| 349 | } | ||
| 347 | 350 | ||
| 348 | buf = (char *)urb->transfer_buffer; | 351 | buf = (char *)urb->transfer_buffer; |
| 349 | buf = udl_vidreg_lock(buf); | 352 | buf = udl_vidreg_lock(buf); |
| @@ -125,20 +125,6 @@ v3d_performance_query_info_free(struct v3d_performance_query_info *query_info, · 124 unmodified lines | |||
| 125 | } | 125 | } |
| 126 | } | 126 | } |
| 127 | 127 | ||
| 128 | static void | ||
| 129 | v3d_cpu_job_free(struct drm_sched_job *sched_job) | ||
| 130 | { | ||
| 131 | struct v3d_cpu_job *job = to_cpu_job(sched_job); | ||
| 132 | |||
| 133 | v3d_timestamp_query_info_free(&job->timestamp_query, | ||
| 134 | job->timestamp_query.count); | ||
| 135 | |||
| 136 | v3d_performance_query_info_free(&job->performance_query, | ||
| 137 | job->performance_query.count); | ||
| 138 | |||
| 139 | v3d_job_cleanup(&job->base); | ||
| 140 | } | ||
| 141 | |||
| 142 | static void | 128 | static void |
| 143 | v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job) | 129 | v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job) |
| 144 | { | 130 | { |
| @@ -830,7 +816,7 @@ static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = { · 685 unmodified lines | |||
| 830 | 816 | ||
| 831 | static const struct drm_sched_backend_ops v3d_cpu_sched_ops = { | 817 | static const struct drm_sched_backend_ops v3d_cpu_sched_ops = { |
| 832 | .run_job = v3d_cpu_job_run, | 818 | .run_job = v3d_cpu_job_run, |
| 833 | .free_job = v3d_cpu_job_free | 819 | .free_job = v3d_sched_job_free |
| 834 | }; | 820 | }; |
| 835 | 821 | ||
| 836 | static int | 822 | static int |
| @@ -123,6 +123,24 @@ v3d_render_job_free(struct kref *ref) · 122 unmodified lines | |||
| 123 | v3d_job_free(ref); | 123 | v3d_job_free(ref); |
| 124 | } | 124 | } |
| 125 | 125 | ||
| 126 | static void | ||
| 127 | v3d_cpu_job_free(struct kref *ref) | ||
| 128 | { | ||
| 129 | struct v3d_cpu_job *job = container_of(ref, struct v3d_cpu_job, | ||
| 130 | base.refcount); | ||
| 131 | |||
| 132 | v3d_timestamp_query_info_free(&job->timestamp_query, | ||
| 133 | job->timestamp_query.count); | ||
| 134 | |||
| 135 | v3d_performance_query_info_free(&job->performance_query, | ||
| 136 | job->performance_query.count); | ||
| 137 | |||
| 138 | if (job->indirect_csd.indirect) | ||
| 139 | drm_gem_object_put(job->indirect_csd.indirect); | ||
| 140 | |||
| 141 | v3d_job_free(ref); | ||
| 142 | } | ||
| 143 | |||
| 126 | void v3d_job_cleanup(struct v3d_job *job) | 144 | void v3d_job_cleanup(struct v3d_job *job) |
| 127 | { | 145 | { |
| 128 | if (!job) | 146 | if (!job) |
| @@ -1302,7 +1320,7 @@ v3d_submit_cpu_ioctl(struct drm_device *dev, void *data, · 1173 unmodified lines | |||
| 1302 | trace_v3d_submit_cpu_ioctl(&v3d->drm, cpu_job->job_type); | 1320 | trace_v3d_submit_cpu_ioctl(&v3d->drm, cpu_job->job_type); |
| 1303 | 1321 | ||
| 1304 | ret = v3d_job_init(v3d, file_priv, &cpu_job->base, | 1322 | ret = v3d_job_init(v3d, file_priv, &cpu_job->base, |
| 1305 | v3d_job_free, 0, &se, V3D_CPU); | 1323 | v3d_cpu_job_free, 0, &se, V3D_CPU); |
| 1306 | if (ret) { | 1324 | if (ret) { |
| 1307 | v3d_job_deallocate((void *)&cpu_job); | 1325 | v3d_job_deallocate((void *)&cpu_job); |
| 1308 | goto fail; | 1326 | goto fail; |
| @@ -1385,8 +1403,6 @@ fail: · 76 unmodified lines | |||
| 1385 | v3d_job_cleanup((void *)csd_job); | 1403 | v3d_job_cleanup((void *)csd_job); |
| 1386 | v3d_job_cleanup(clean_job); | 1404 | v3d_job_cleanup(clean_job); |
| 1387 | v3d_put_multisync_post_deps(&se); | 1405 | v3d_put_multisync_post_deps(&se); |
| 1388 | kvfree(cpu_job->timestamp_query.queries); | ||
| 1389 | kvfree(cpu_job->performance_query.queries); | ||
| 1390 | 1406 | ||
| 1391 | return ret; | 1407 | return ret; |
| 1392 | } | 1408 | } |
| @@ -317,6 +317,7 @@ virtio_gpu_array_from_handles(struct drm_file *drm_file, u32 *handles, u32 nents · 316 unmodified lines | |||
| 317 | void virtio_gpu_array_add_obj(struct virtio_gpu_object_array *objs, | 317 | void virtio_gpu_array_add_obj(struct virtio_gpu_object_array *objs, |
| 318 | struct drm_gem_object *obj); | 318 | struct drm_gem_object *obj); |
| 319 | int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs); | 319 | int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs); |
| 320 | int virtio_gpu_lock_one_resv_uninterruptible(struct virtio_gpu_object_array *objs); | ||
| 320 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs); | 321 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs); |
| 321 | void virtio_gpu_array_add_fence(struct virtio_gpu_object_array *objs, | 322 | void virtio_gpu_array_add_fence(struct virtio_gpu_object_array *objs, |
| 322 | struct dma_fence *fence); | 323 | struct dma_fence *fence); |
| @@ -238,6 +238,23 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs) · 237 unmodified lines | |||
| 238 | return ret; | 238 | return ret; |
| 239 | } | 239 | } |
| 240 | 240 | ||
| 241 | int virtio_gpu_lock_one_resv_uninterruptible(struct virtio_gpu_object_array *objs) | ||
| 242 | { | ||
| 243 | int ret; | ||
| 244 | |||
| 245 | if (objs->nents != 1) | ||
| 246 | return -EINVAL; | ||
| 247 | |||
| 248 | dma_resv_lock(objs->objs[0]->resv, NULL); | ||
| 249 | |||
| 250 | ret = dma_resv_reserve_fences(objs->objs[0]->resv, 1); | ||
| 251 | if (ret) { | ||
| 252 | virtio_gpu_array_unlock_resv(objs); | ||
| 253 | return ret; | ||
| 254 | } | ||
| 255 | return 0; | ||
| 256 | } | ||
| 257 | |||
| 241 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs) | 258 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs) |
| 242 | { | 259 | { |
| 243 | if (objs->nents == 1) { | 260 | if (objs->nents == 1) { |
| @@ -215,7 +215,10 @@ static void virtio_gpu_resource_flush(struct drm_plane *plane, · 214 unmodified lines | |||
| 215 | if (!objs) | 215 | if (!objs) |
| 216 | return; | 216 | return; |
| 217 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); | 217 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); |
| 218 | virtio_gpu_array_lock_resv(objs); | 218 | if (virtio_gpu_lock_one_resv_uninterruptible(objs)) { |
| 219 | virtio_gpu_array_put_free(objs); | ||
| 220 | return; | ||
| 221 | } | ||
| 219 | virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y, | 222 | virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y, |
| 220 | width, height, objs, | 223 | width, height, objs, |
| 221 | vgplane_st->fence); | 224 | vgplane_st->fence); |
| @@ -459,7 +462,10 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, · 237 unmodified lines | |||
| 459 | if (!objs) | 462 | if (!objs) |
| 460 | return; | 463 | return; |
| 461 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); | 464 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); |
| 462 | virtio_gpu_array_lock_resv(objs); | 465 | if (virtio_gpu_lock_one_resv_uninterruptible(objs)) { |
| 466 | virtio_gpu_array_put_free(objs); | ||
| 467 | return; | ||
| 468 | } | ||
| 463 | virtio_gpu_cmd_transfer_to_host_2d | 469 | virtio_gpu_cmd_transfer_to_host_2d |
| 464 | (vgdev, 0, | 470 | (vgdev, 0, |
| 465 | plane->state->crtc_w, | 471 | plane->state->crtc_w, |
| @@ -88,6 +88,7 @@ xe-y += xe_bb.o \ · 87 unmodified lines | |||
| 88 | xe_irq.o \ | 88 | xe_irq.o \ |
| 89 | xe_late_bind_fw.o \ | 89 | xe_late_bind_fw.o \ |
| 90 | xe_lrc.o \ | 90 | xe_lrc.o \ |
| 91 | xe_mem_pool.o \ | ||
| 91 | xe_migrate.o \ | 92 | xe_migrate.o \ |
| 92 | xe_mmio.o \ | 93 | xe_mmio.o \ |
| 93 | xe_mmio_gem.o \ | 94 | xe_mmio_gem.o \ |
| @@ -37,9 +37,17 @@ static bool intel_hdcp_gsc_check_status(struct drm_device *drm) · 36 unmodified lines | |||
| 37 | struct xe_device *xe = to_xe_device(drm); | 37 | struct xe_device *xe = to_xe_device(drm); |
| 38 | struct xe_tile *tile = xe_device_get_root_tile(xe); | 38 | struct xe_tile *tile = xe_device_get_root_tile(xe); |
| 39 | struct xe_gt *gt = tile->media_gt; | 39 | struct xe_gt *gt = tile->media_gt; |
| 40 | struct xe_gsc *gsc = >->uc.gsc; | 40 | struct xe_gsc *gsc; |
| 41 | |||
| 42 | if (!gt) { | ||
| 43 | drm_dbg_kms(&xe->drm, | ||
| 44 | "not checking GSC status for HDCP2.x: media GT not present or disabled\n"); | ||
| 45 | return false; | ||
| 46 | } | ||
| 47 | |||
| 48 | gsc = >->uc.gsc; | ||
| 41 | 49 | ||
| 42 | if (!gsc || !xe_uc_fw_is_available(&gsc->fw)) { | 50 | if (!xe_uc_fw_is_available(&gsc->fw)) { |
| 43 | drm_dbg_kms(&xe->drm, | 51 | drm_dbg_kms(&xe->drm, |
| 44 | "GSC Components not ready for HDCP2.x\n"); | 52 | "GSC Components not ready for HDCP2.x\n"); |
| 45 | return false; | 53 | return false; |
| @@ -152,10 +152,11 @@ · 151 unmodified lines | |||
| 152 | 152 | ||
| 153 | #define XEHPG_INSTDONE_GEOM_SVGUNIT XE_REG_MCR(0x666c) | 153 | #define XEHPG_INSTDONE_GEOM_SVGUNIT XE_REG_MCR(0x666c) |
| 154 | 154 | ||
| 155 | #define CACHE_MODE_1 XE_REG(0x7004, XE_REG_OPTION_MASKED) | 155 | #define CACHE_MODE_1 XE_REG_MCR(0x7004, XE_REG_OPTION_MASKED) |
| 156 | #define MSAA_OPTIMIZATION_REDUC_DISABLE REG_BIT(11) | 156 | #define MSAA_OPTIMIZATION_REDUC_DISABLE REG_BIT(11) |
| 157 | 157 | ||
| 158 | #define COMMON_SLICE_CHICKEN1 XE_REG(0x7010, XE_REG_OPTION_MASKED) | 158 | #define COMMON_SLICE_CHICKEN1 XE_REG(0x7010, XE_REG_OPTION_MASKED) |
| 159 | #define XEHP_COMMON_SLICE_CHICKEN1 XE_REG_MCR(0x7010, XE_REG_OPTION_MASKED) | ||
| 159 | #define DISABLE_BOTTOM_CLIP_RECTANGLE_TEST REG_BIT(14) | 160 | #define DISABLE_BOTTOM_CLIP_RECTANGLE_TEST REG_BIT(14) |
| 160 | 161 | ||
| 161 | #define HIZ_CHICKEN XE_REG(0x7018, XE_REG_OPTION_MASKED) | 162 | #define HIZ_CHICKEN XE_REG(0x7018, XE_REG_OPTION_MASKED) |
| @@ -178,6 +179,7 @@ · 16 unmodified lines | |||
| 178 | #define XEHPG_SC_INSTDONE_EXTRA2 XE_REG_MCR(0x7108) | 179 | #define XEHPG_SC_INSTDONE_EXTRA2 XE_REG_MCR(0x7108) |
| 179 | 180 | ||
| 180 | #define COMMON_SLICE_CHICKEN4 XE_REG(0x7300, XE_REG_OPTION_MASKED) | 181 | #define COMMON_SLICE_CHICKEN4 XE_REG(0x7300, XE_REG_OPTION_MASKED) |
| 182 | #define XEHP_COMMON_SLICE_CHICKEN4 XE_REG_MCR(0x7300, XE_REG_OPTION_MASKED) | ||
| 181 | #define SBE_PUSH_CONSTANT_BEHIND_FIX_ENABLE REG_BIT(12) | 183 | #define SBE_PUSH_CONSTANT_BEHIND_FIX_ENABLE REG_BIT(12) |
| 182 | #define DISABLE_TDC_LOAD_BALANCING_CALC REG_BIT(6) | 184 | #define DISABLE_TDC_LOAD_BALANCING_CALC REG_BIT(6) |
| 183 | #define HW_FILTERING REG_BIT(5) | 185 | #define HW_FILTERING REG_BIT(5) |
| @@ -583,7 +585,7 @@ · 399 unmodified lines | |||
| 583 | #define DISABLE_128B_EVICTION_COMMAND_UDW REG_BIT(36 - 32) | 585 | #define DISABLE_128B_EVICTION_COMMAND_UDW REG_BIT(36 - 32) |
| 584 | #define LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE REG_BIT(35 - 32) | 586 | #define LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE REG_BIT(35 - 32) |
| 585 | 587 | ||
| 586 | #define ROW_CHICKEN5 XE_REG_MCR(0xe7f0) | 588 | #define ROW_CHICKEN5 XE_REG_MCR(0xe7f0, XE_REG_OPTION_MASKED) |
| 587 | #define CPSS_AWARE_DIS REG_BIT(3) | 589 | #define CPSS_AWARE_DIS REG_BIT(3) |
| 588 | 590 | ||
| 589 | #define SARB_CHICKEN1 XE_REG_MCR(0xe90c) | 591 | #define SARB_CHICKEN1 XE_REG_MCR(0xe90c) |
| @@ -897,10 +897,10 @@ void xe_bo_set_purgeable_state(struct xe_bo *bo, · 896 unmodified lines | |||
| 897 | new_state == XE_MADV_PURGEABLE_PURGED); | 897 | new_state == XE_MADV_PURGEABLE_PURGED); |
| 898 | 898 | ||
| 899 | /* Once purged, always purged - cannot transition out */ | 899 | /* Once purged, always purged - cannot transition out */ |
| 900 | xe_assert(xe, !(bo->madv_purgeable == XE_MADV_PURGEABLE_PURGED && | 900 | xe_assert(xe, !(bo->purgeable.state == XE_MADV_PURGEABLE_PURGED && |
| 901 | new_state != XE_MADV_PURGEABLE_PURGED)); | 901 | new_state != XE_MADV_PURGEABLE_PURGED)); |
| 902 | 902 | ||
| 903 | bo->madv_purgeable = new_state; | 903 | bo->purgeable.state = new_state; |
| 904 | xe_bo_set_purgeable_shrinker(bo, new_state); | 904 | xe_bo_set_purgeable_shrinker(bo, new_state); |
| 905 | } | 905 | } |
| 906 | 906 | ||
| @@ -2322,8 +2322,10 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 1415 unmodified lines | |||
| 2322 | } | 2322 | } |
| 2323 | 2323 | ||
| 2324 | /* XE_BO_FLAG_GGTTx requires XE_BO_FLAG_GGTT also be set */ | 2324 | /* XE_BO_FLAG_GGTTx requires XE_BO_FLAG_GGTT also be set */ |
| 2325 | if ((flags & XE_BO_FLAG_GGTT_ALL) && !(flags & XE_BO_FLAG_GGTT)) | 2325 | if ((flags & XE_BO_FLAG_GGTT_ALL) && !(flags & XE_BO_FLAG_GGTT)) { |
| 2326 | xe_bo_free(bo); | ||
| 2326 | return ERR_PTR(-EINVAL); | 2327 | return ERR_PTR(-EINVAL); |
| 2328 | } | ||
| 2327 | 2329 | ||
| 2328 | if (flags & (XE_BO_FLAG_VRAM_MASK | XE_BO_FLAG_STOLEN) && | 2330 | if (flags & (XE_BO_FLAG_VRAM_MASK | XE_BO_FLAG_STOLEN) && |
| 2329 | !(flags & XE_BO_FLAG_IGNORE_MIN_PAGE_SIZE) && | 2331 | !(flags & XE_BO_FLAG_IGNORE_MIN_PAGE_SIZE) && |
| @@ -2342,8 +2344,10 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 12 unmodified lines | |||
| 2342 | alignment = SZ_4K >> PAGE_SHIFT; | 2344 | alignment = SZ_4K >> PAGE_SHIFT; |
| 2343 | } | 2345 | } |
| 2344 | 2346 | ||
| 2345 | if (type == ttm_bo_type_device && aligned_size != size) | 2347 | if (type == ttm_bo_type_device && aligned_size != size) { |
| 2348 | xe_bo_free(bo); | ||
| 2346 | return ERR_PTR(-EINVAL); | 2349 | return ERR_PTR(-EINVAL); |
| 2350 | } | ||
| 2347 | 2351 | ||
| 2348 | if (!bo) { | 2352 | if (!bo) { |
| 2349 | bo = xe_bo_alloc(); | 2353 | bo = xe_bo_alloc(); |
| @@ -2364,7 +2368,7 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 14 unmodified lines | |||
| 2364 | INIT_LIST_HEAD(&bo->vram_userfault_link); | 2368 | INIT_LIST_HEAD(&bo->vram_userfault_link); |
| 2365 | 2369 | ||
| 2366 | /* Initialize purge advisory state */ | 2370 | /* Initialize purge advisory state */ |
| 2367 | bo->madv_purgeable = XE_MADV_PURGEABLE_WILLNEED; | 2371 | bo->purgeable.state = XE_MADV_PURGEABLE_WILLNEED; |
| 2368 | 2372 | ||
| 2369 | drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size); | 2373 | drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size); |
| 2370 | 2374 | ||
| @@ -251,7 +251,7 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo) · 250 unmodified lines | |||
| 251 | static inline bool xe_bo_is_purged(struct xe_bo *bo) | 251 | static inline bool xe_bo_is_purged(struct xe_bo *bo) |
| 252 | { | 252 | { |
| 253 | xe_bo_assert_held(bo); | 253 | xe_bo_assert_held(bo); |
| 254 | return bo->madv_purgeable == XE_MADV_PURGEABLE_PURGED; | 254 | return bo->purgeable.state == XE_MADV_PURGEABLE_PURGED; |
| 255 | } | 255 | } |
| 256 | 256 | ||
| 257 | /** | 257 | /** |
| @@ -268,11 +268,95 @@ static inline bool xe_bo_is_purged(struct xe_bo *bo) · 10 unmodified lines | |||
| 268 | static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo) | 268 | static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo) |
| 269 | { | 269 | { |
| 270 | xe_bo_assert_held(bo); | 270 | xe_bo_assert_held(bo); |
| 271 | return bo->madv_purgeable == XE_MADV_PURGEABLE_DONTNEED; | 271 | return bo->purgeable.state == XE_MADV_PURGEABLE_DONTNEED; |
| 272 | } | 272 | } |
| 273 | 273 | ||
| 274 | void xe_bo_set_purgeable_state(struct xe_bo *bo, enum xe_madv_purgeable_state new_state); | 274 | void xe_bo_set_purgeable_state(struct xe_bo *bo, enum xe_madv_purgeable_state new_state); |
| 275 | 275 | ||
| 276 | /** | ||
| 277 | * xe_bo_willneed_get_locked() - Acquire a WILLNEED holder on a BO | ||
| 278 | * @bo: Buffer object | ||
| 279 | * | ||
| 280 | * Increments willneed_count and, on a 0->1 transition, promotes the BO | ||
| 281 | * from DONTNEED to WILLNEED. PURGED is terminal and is never modified. | ||
| 282 | * | ||
| 283 | * Caller must hold the BO's dma-resv lock. | ||
| 284 | */ | ||
| 285 | static inline void xe_bo_willneed_get_locked(struct xe_bo *bo) | ||
| 286 | { | ||
| 287 | xe_bo_assert_held(bo); | ||
| 288 | |||
| 289 | /* Imported BOs are owned externally; do not track purgeability. */ | ||
| 290 | if (drm_gem_is_imported(&bo->ttm.base)) | ||
| 291 | return; | ||
| 292 | |||
| 293 | if (bo->purgeable.willneed_count++ == 0 && xe_bo_madv_is_dontneed(bo)) | ||
| 294 | xe_bo_set_purgeable_state(bo, XE_MADV_PURGEABLE_WILLNEED); | ||
| 295 | } | ||
| 296 | |||
| 297 | /** | ||
| 298 | * xe_bo_willneed_put_locked() - Release a WILLNEED holder on a BO | ||
| 299 | * @bo: Buffer object | ||
| 300 | * | ||
| 301 | * Decrements willneed_count and, on a 1->0 transition, marks the BO | ||
| 302 | * DONTNEED only if it still has VMAs (implying all active VMAs are | ||
| 303 | * DONTNEED). If the last VMA is being removed, preserve the current BO | ||
| 304 | * state to match the previous VMA-walk semantics. | ||
| 305 | * | ||
| 306 | * PURGED is terminal and the BO state is never modified. | ||
| 307 | * | ||
| 308 | * Caller must hold the BO's dma-resv lock. | ||
| 309 | */ | ||
| 310 | static inline void xe_bo_willneed_put_locked(struct xe_bo *bo) | ||
| 311 | { | ||
| 312 | xe_bo_assert_held(bo); | ||
| 313 | |||
| 314 | if (drm_gem_is_imported(&bo->ttm.base)) | ||
| 315 | return; | ||
| 316 | |||
| 317 | xe_assert(xe_bo_device(bo), bo->purgeable.willneed_count > 0); | ||
| 318 | if (--bo->purgeable.willneed_count == 0 && bo->purgeable.vma_count > 0 && | ||
| 319 | !xe_bo_is_purged(bo)) | ||
| 320 | xe_bo_set_purgeable_state(bo, XE_MADV_PURGEABLE_DONTNEED); | ||
| 321 | } | ||
| 322 | |||
| 323 | /** | ||
| 324 | * xe_bo_vma_count_inc_locked() - Account a new VMA on a BO | ||
| 325 | * @bo: Buffer object | ||
| 326 | * | ||
| 327 | * Increments vma_count. | ||
| 328 | * | ||
| 329 | * Caller must hold the BO's dma-resv lock. | ||
| 330 | */ | ||
| 331 | static inline void xe_bo_vma_count_inc_locked(struct xe_bo *bo) | ||
| 332 | { | ||
| 333 | xe_bo_assert_held(bo); | ||
| 334 | |||
| 335 | if (drm_gem_is_imported(&bo->ttm.base)) | ||
| 336 | return; | ||
| 337 | |||
| 338 | bo->purgeable.vma_count++; | ||
| 339 | } | ||
| 340 | |||
| 341 | /** | ||
| 342 | * xe_bo_vma_count_dec_locked() - Account a VMA removal on a BO | ||
| 343 | * @bo: Buffer object | ||
| 344 | * | ||
| 345 | * Decrements vma_count. | ||
| 346 | * | ||
| 347 | * Caller must hold the BO's dma-resv lock. | ||
| 348 | */ | ||
| 349 | static inline void xe_bo_vma_count_dec_locked(struct xe_bo *bo) | ||
| 350 | { | ||
| 351 | xe_bo_assert_held(bo); | ||
| 352 | |||
| 353 | if (drm_gem_is_imported(&bo->ttm.base)) | ||
| 354 | return; | ||
| 355 | |||
| 356 | xe_assert(xe_bo_device(bo), bo->purgeable.vma_count > 0); | ||
| 357 | bo->purgeable.vma_count--; | ||
| 358 | } | ||
| 359 | |||
| 276 | static inline void xe_bo_unpin_map_no_vm(struct xe_bo *bo) | 360 | static inline void xe_bo_unpin_map_no_vm(struct xe_bo *bo) |
| 277 | { | 361 | { |
| 278 | if (likely(bo)) { | 362 | if (likely(bo)) { |
| @@ -18,6 +18,7 @@ · 17 unmodified lines | |||
| 18 | #include "xe_ggtt_types.h" | 18 | #include "xe_ggtt_types.h" |
| 19 | 19 | ||
| 20 | struct xe_device; | 20 | struct xe_device; |
| 21 | struct xe_mem_pool_node; | ||
| 21 | struct xe_vm; | 22 | struct xe_vm; |
| 22 | 23 | ||
| 23 | #define XE_BO_MAX_PLACEMENTS 3 | 24 | #define XE_BO_MAX_PLACEMENTS 3 |
| @@ -88,7 +89,7 @@ struct xe_bo { · 64 unmodified lines | |||
| 88 | bool ccs_cleared; | 89 | bool ccs_cleared; |
| 89 | 90 | ||
| 90 | /** @bb_ccs: BB instructions of CCS read/write. Valid only for VF */ | 91 | /** @bb_ccs: BB instructions of CCS read/write. Valid only for VF */ |
| 91 | struct xe_bb *bb_ccs[XE_SRIOV_VF_CCS_CTX_COUNT]; | 92 | struct xe_mem_pool_node *bb_ccs[XE_SRIOV_VF_CCS_CTX_COUNT]; |
| 92 | 93 | ||
| 93 | /** | 94 | /** |
| 94 | * @cpu_caching: CPU caching mode. Currently only used for userspace | 95 | * @cpu_caching: CPU caching mode. Currently only used for userspace |
| @@ -110,10 +111,32 @@ struct xe_bo { · 15 unmodified lines | |||
| 110 | u64 min_align; | 111 | u64 min_align; |
| 111 | 112 | ||
| 112 | /** | 113 | /** |
| 113 | * @madv_purgeable: user space advise on BO purgeability, protected | 114 | * @purgeable: Purgeability state and accounting. |
| 114 | * by BO's dma-resv lock. | 115 | * |
| 116 | * All fields are protected by the BO's dma-resv lock. | ||
| 115 | */ | 117 | */ |
| 116 | u32 madv_purgeable; | 118 | struct { |
| 119 | /** | ||
| 120 | * @purgeable.state: BO purgeability state | ||
| 121 | * (WILLNEED/DONTNEED/PURGED). | ||
| 122 | */ | ||
| 123 | u32 state; | ||
| 124 | |||
| 125 | /** | ||
| 126 | * @purgeable.vma_count: Number of VMAs currently mapping this BO. | ||
| 127 | */ | ||
| 128 | u32 vma_count; | ||
| 129 | |||
| 130 | /** | ||
| 131 | * @purgeable.willneed_count: Number of active WILLNEED holders. | ||
| 132 | * | ||
| 133 | * Counts WILLNEED VMAs plus active dma-buf exports for | ||
| 134 | * non-imported BOs. The BO flips to DONTNEED on a 1->0 | ||
| 135 | * transition only when VMAs still exist; if the last VMA is | ||
| 136 | * removed, the previous BO state is preserved. | ||
| 137 | */ | ||
| 138 | u32 willneed_count; | ||
| 139 | } purgeable; | ||
| 117 | }; | 140 | }; |
| 118 | 141 | ||
| 119 | #endif | 142 | #endif |
| @@ -193,6 +193,18 @@ static int xe_dma_buf_begin_cpu_access(struct dma_buf *dma_buf, · 192 unmodified lines | |||
| 193 | return 0; | 193 | return 0; |
| 194 | } | 194 | } |
| 195 | 195 | ||
| 196 | static void xe_dma_buf_release(struct dma_buf *dmabuf) | ||
| 197 | { | ||
| 198 | struct drm_gem_object *obj = dmabuf->priv; | ||
| 199 | struct xe_bo *bo = gem_to_xe_bo(obj); | ||
| 200 | |||
| 201 | xe_bo_lock(bo, false); | ||
| 202 | xe_bo_willneed_put_locked(bo); | ||
| 203 | xe_bo_unlock(bo); | ||
| 204 | |||
| 205 | drm_gem_dmabuf_release(dmabuf); | ||
| 206 | } | ||
| 207 | |||
| 196 | static const struct dma_buf_ops xe_dmabuf_ops = { | 208 | static const struct dma_buf_ops xe_dmabuf_ops = { |
| 197 | .attach = xe_dma_buf_attach, | 209 | .attach = xe_dma_buf_attach, |
| 198 | .detach = xe_dma_buf_detach, | 210 | .detach = xe_dma_buf_detach, |
| @@ -200,7 +212,7 @@ static const struct dma_buf_ops xe_dmabuf_ops = { · 1 unmodified line | |||
| 200 | .unpin = xe_dma_buf_unpin, | 212 | .unpin = xe_dma_buf_unpin, |
| 201 | .map_dma_buf = xe_dma_buf_map, | 213 | .map_dma_buf = xe_dma_buf_map, |
| 202 | .unmap_dma_buf = xe_dma_buf_unmap, | 214 | .unmap_dma_buf = xe_dma_buf_unmap, |
| 203 | .release = drm_gem_dmabuf_release, | 215 | .release = xe_dma_buf_release, |
| 204 | .begin_cpu_access = xe_dma_buf_begin_cpu_access, | 216 | .begin_cpu_access = xe_dma_buf_begin_cpu_access, |
| 205 | .mmap = drm_gem_dmabuf_mmap, | 217 | .mmap = drm_gem_dmabuf_mmap, |
| 206 | .vmap = drm_gem_dmabuf_vmap, | 218 | .vmap = drm_gem_dmabuf_vmap, |
| @@ -241,26 +253,33 @@ struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags) · 34 unmodified lines | |||
| 241 | ret = -EINVAL; | 253 | ret = -EINVAL; |
| 242 | goto out_unlock; | 254 | goto out_unlock; |
| 243 | } | 255 | } |
| 256 | |||
| 257 | xe_bo_willneed_get_locked(bo); | ||
| 244 | xe_bo_unlock(bo); | 258 | xe_bo_unlock(bo); |
| 245 | 259 | ||
| 246 | ret = ttm_bo_setup_export(&bo->ttm, &ctx); | 260 | ret = ttm_bo_setup_export(&bo->ttm, &ctx); |
| 247 | if (ret) | 261 | if (ret) |
| 248 | return ERR_PTR(ret); | 262 | goto out_put; |
| 249 | 263 | ||
| 250 | buf = drm_gem_prime_export(obj, flags); | 264 | buf = drm_gem_prime_export(obj, flags); |
| 251 | if (!IS_ERR(buf)) | 265 | if (IS_ERR(buf)) { |
| 252 | buf->ops = &xe_dmabuf_ops; | 266 | ret = PTR_ERR(buf); |
| 267 | goto out_put; | ||
| 268 | } | ||
| 253 | 269 | ||
| 270 | buf->ops = &xe_dmabuf_ops; | ||
| 254 | return buf; | 271 | return buf; |
| 255 | 272 | ||
| 273 | out_put: | ||
| 274 | xe_bo_lock(bo, false); | ||
| 275 | xe_bo_willneed_put_locked(bo); | ||
| 256 | out_unlock: | 276 | out_unlock: |
| 257 | xe_bo_unlock(bo); | 277 | xe_bo_unlock(bo); |
| 258 | return ERR_PTR(ret); | 278 | return ERR_PTR(ret); |
| 259 | } | 279 | } |
| 260 | 280 | ||
| 261 | static struct drm_gem_object * | 281 | static struct drm_gem_object * |
| 262 | xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage, | 282 | xe_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf) |
| 263 | struct dma_buf *dma_buf) | ||
| 264 | { | 283 | { |
| 265 | struct dma_resv *resv = dma_buf->resv; | 284 | struct dma_resv *resv = dma_buf->resv; |
| 266 | struct xe_device *xe = to_xe_device(dev); | 285 | struct xe_device *xe = to_xe_device(dev); |
| @@ -281,7 +300,7 @@ xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage, · 14 unmodified lines | |||
| 281 | if (ret) | 300 | if (ret) |
| 282 | break; | 301 | break; |
| 283 | 302 | ||
| 284 | bo = xe_bo_init_locked(xe, storage, NULL, resv, NULL, dma_buf->size, | 303 | bo = xe_bo_init_locked(xe, NULL, NULL, resv, NULL, dma_buf->size, |
| 285 | 0, /* Will require 1way or 2way for vm_bind */ | 304 | 0, /* Will require 1way or 2way for vm_bind */ |
| 286 | ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, &exec); | 305 | ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, &exec); |
| 287 | drm_exec_retry_on_contention(&exec); | 306 | drm_exec_retry_on_contention(&exec); |
| @@ -332,7 +351,6 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 44 unmodified lines | |||
| 332 | const struct dma_buf_attach_ops *attach_ops; | 351 | const struct dma_buf_attach_ops *attach_ops; |
| 333 | struct dma_buf_attachment *attach; | 352 | struct dma_buf_attachment *attach; |
| 334 | struct drm_gem_object *obj; | 353 | struct drm_gem_object *obj; |
| 335 | struct xe_bo *bo; | ||
| 336 | 354 | ||
| 337 | if (dma_buf->ops == &xe_dmabuf_ops) { | 355 | if (dma_buf->ops == &xe_dmabuf_ops) { |
| 338 | obj = dma_buf->priv; | 356 | obj = dma_buf->priv; |
| @@ -348,13 +366,15 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 9 unmodified lines | |||
| 348 | } | 366 | } |
| 349 | 367 | ||
| 350 | /* | 368 | /* |
| 351 | * Don't publish the bo until we have a valid attachment, and a | 369 | * This needs to happen before the attach, since it will create a new |
| 352 | * valid attachment needs the bo address. So pre-create a bo before | 370 | * attachment for this, and add it to the list of attachments, at which |
| 353 | * creating the attachment and publish. | 371 | * point it is globally visible, and at any point the export side can |
| 372 | * call into on invalidate_mappings callback, which require a working | ||
| 373 | * object. | ||
| 354 | */ | 374 | */ |
| 355 | bo = xe_bo_alloc(); | 375 | obj = xe_dma_buf_create_obj(dev, dma_buf); |
| 356 | if (IS_ERR(bo)) | 376 | if (IS_ERR(obj)) |
| 357 | return ERR_CAST(bo); | 377 | return obj; |
| 358 | 378 | ||
| 359 | attach_ops = &xe_dma_buf_attach_ops; | 379 | attach_ops = &xe_dma_buf_attach_ops; |
| 360 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | 380 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) |
| @@ -362,26 +382,15 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 1 unmodified line | |||
| 362 | attach_ops = test->attach_ops; | 382 | attach_ops = test->attach_ops; |
| 363 | #endif | 383 | #endif |
| 364 | 384 | ||
| 365 | attach = dma_buf_dynamic_attach(dma_buf, dev->dev, attach_ops, &bo->ttm.base); | 385 | attach = dma_buf_dynamic_attach(dma_buf, dev->dev, attach_ops, obj); |
| 366 | if (IS_ERR(attach)) { | 386 | if (IS_ERR(attach)) { |
| 367 | obj = ERR_CAST(attach); | 387 | xe_bo_put(gem_to_xe_bo(obj)); |
| 368 | goto out_err; | 388 | return ERR_CAST(attach); |
| 369 | } | 389 | } |
| 370 | 390 | ||
| 371 | /* Errors here will take care of freeing the bo. */ | ||
| 372 | obj = xe_dma_buf_init_obj(dev, bo, dma_buf); | ||
| 373 | if (IS_ERR(obj)) | ||
| 374 | return obj; | ||
| 375 | |||
| 376 | |||
| 377 | get_dma_buf(dma_buf); | 391 | get_dma_buf(dma_buf); |
| 378 | obj->import_attach = attach; | 392 | obj->import_attach = attach; |
| 379 | return obj; | 393 | return obj; |
| 380 | |||
| 381 | out_err: | ||
| 382 | xe_bo_free(bo); | ||
| 383 | |||
| 384 | return obj; | ||
| 385 | } | 394 | } |
| 386 | 395 | ||
| 387 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | 396 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) |
| @@ -869,14 +869,14 @@ static int xe_eu_stall_stream_close(struct inode *inode, struct file *file) · 868 unmodified lines | |||
| 869 | struct xe_eu_stall_data_stream *stream = file->private_data; | 869 | struct xe_eu_stall_data_stream *stream = file->private_data; |
| 870 | struct xe_gt *gt = stream->gt; | 870 | struct xe_gt *gt = stream->gt; |
| 871 | 871 | ||
| 872 | drm_dev_put(>->tile->xe->drm); | ||
| 873 | |||
| 874 | mutex_lock(>->eu_stall->stream_lock); | 872 | mutex_lock(>->eu_stall->stream_lock); |
| 875 | xe_eu_stall_disable_locked(stream); | 873 | xe_eu_stall_disable_locked(stream); |
| 876 | xe_eu_stall_data_buf_destroy(stream); | 874 | xe_eu_stall_data_buf_destroy(stream); |
| 877 | xe_eu_stall_stream_free(stream); | 875 | xe_eu_stall_stream_free(stream); |
| 878 | mutex_unlock(>->eu_stall->stream_lock); | 876 | mutex_unlock(>->eu_stall->stream_lock); |
| 879 | 877 | ||
| 878 | drm_dev_put(>->tile->xe->drm); | ||
| 879 | |||
| 880 | return 0; | 880 | return 0; |
| 881 | } | 881 | } |
| 882 | 882 | ||
| @@ -1405,7 +1405,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 1404 unmodified lines | |||
| 1405 | if (q->vm && q->hwe->hw_engine_group) { | 1405 | if (q->vm && q->hwe->hw_engine_group) { |
| 1406 | err = xe_hw_engine_group_add_exec_queue(q->hwe->hw_engine_group, q); | 1406 | err = xe_hw_engine_group_add_exec_queue(q->hwe->hw_engine_group, q); |
| 1407 | if (err) | 1407 | if (err) |
| 1408 | goto put_exec_queue; | 1408 | goto kill_exec_queue; |
| 1409 | } | 1409 | } |
| 1410 | } | 1410 | } |
| 1411 | 1411 | ||
| @@ -1416,12 +1416,15 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 4 unmodified lines | |||
| 1416 | /* user id alloc must always be last in ioctl to prevent UAF */ | 1416 | /* user id alloc must always be last in ioctl to prevent UAF */ |
| 1417 | err = xa_alloc(&xef->exec_queue.xa, &id, q, xa_limit_32b, GFP_KERNEL); | 1417 | err = xa_alloc(&xef->exec_queue.xa, &id, q, xa_limit_32b, GFP_KERNEL); |
| 1418 | if (err) | 1418 | if (err) |
| 1419 | goto kill_exec_queue; | 1419 | goto del_hw_engine_group; |
| 1420 | 1420 | ||
| 1421 | args->exec_queue_id = id; | 1421 | args->exec_queue_id = id; |
| 1422 | 1422 | ||
| 1423 | return 0; | 1423 | return 0; |
| 1424 | 1424 | ||
| 1425 | del_hw_engine_group: | ||
| 1426 | if (q->vm && q->hwe && q->hwe->hw_engine_group) | ||
| 1427 | xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q); | ||
| 1425 | kill_exec_queue: | 1428 | kill_exec_queue: |
| 1426 | xe_exec_queue_kill(q); | 1429 | xe_exec_queue_kill(q); |
| 1427 | delete_queue_group: | 1430 | delete_queue_group: |
| @@ -1760,7 +1763,7 @@ void xe_exec_queue_tlb_inval_last_fence_put(struct xe_exec_queue *q, · 332 unmodified lines | |||
| 1760 | void xe_exec_queue_tlb_inval_last_fence_put_unlocked(struct xe_exec_queue *q, | 1763 | void xe_exec_queue_tlb_inval_last_fence_put_unlocked(struct xe_exec_queue *q, |
| 1761 | unsigned int type) | 1764 | unsigned int type) |
| 1762 | { | 1765 | { |
| 1763 | xe_assert(q->vm->xe, type == XE_EXEC_QUEUE_TLB_INVAL_MEDIA_GT || | 1766 | xe_assert(gt_to_xe(q->gt), type == XE_EXEC_QUEUE_TLB_INVAL_MEDIA_GT || |
| 1764 | type == XE_EXEC_QUEUE_TLB_INVAL_PRIMARY_GT); | 1767 | type == XE_EXEC_QUEUE_TLB_INVAL_PRIMARY_GT); |
| 1765 | 1768 | ||
| 1766 | dma_fence_put(q->tlb_inval[type].last_fence); | 1769 | dma_fence_put(q->tlb_inval[type].last_fence); |
| @@ -166,7 +166,7 @@ static int query_compatibility_version(struct xe_gsc *gsc) · 165 unmodified lines | |||
| 166 | &rd_offset); | 166 | &rd_offset); |
| 167 | if (err) { | 167 | if (err) { |
| 168 | xe_gt_err(gt, "HuC: invalid GSC reply for version query (err=%d)\n", err); | 168 | xe_gt_err(gt, "HuC: invalid GSC reply for version query (err=%d)\n", err); |
| 169 | return err; | 169 | goto out_bo; |
| 170 | } | 170 | } |
| 171 | 171 | ||
| 172 | compat->major = version_query_rd(xe, &bo->vmap, rd_offset, proj_major); | 172 | compat->major = version_query_rd(xe, &bo->vmap, rd_offset, proj_major); |
| @@ -482,8 +482,7 @@ int xe_gsc_init_post_hwconfig(struct xe_gsc *gsc) · 309 unmodified lines | |||
| 482 | EXEC_QUEUE_FLAG_PERMANENT, 0); | 482 | EXEC_QUEUE_FLAG_PERMANENT, 0); |
| 483 | if (IS_ERR(q)) { | 483 | if (IS_ERR(q)) { |
| 484 | xe_gt_err(gt, "Failed to create queue for GSC submission\n"); | 484 | xe_gt_err(gt, "Failed to create queue for GSC submission\n"); |
| 485 | err = PTR_ERR(q); | 485 | return PTR_ERR(q); |
| 486 | goto out_bo; | ||
| 487 | } | 486 | } |
| 488 | 487 | ||
| 489 | wq = alloc_ordered_workqueue("gsc-ordered-wq", 0); | 488 | wq = alloc_ordered_workqueue("gsc-ordered-wq", 0); |
| @@ -506,8 +505,6 @@ int xe_gsc_init_post_hwconfig(struct xe_gsc *gsc) · 16 unmodified lines | |||
| 506 | 505 | ||
| 507 | out_q: | 506 | out_q: |
| 508 | xe_exec_queue_put(q); | 507 | xe_exec_queue_put(q); |
| 509 | out_bo: | ||
| 510 | xe_bo_unpin_map_no_vm(bo); | ||
| 511 | return err; | 508 | return err; |
| 512 | } | 509 | } |
| 513 | 510 | ||
| @@ -385,10 +385,10 @@ static int pf_migration_mmio_save(struct xe_gt *gt, unsigned int vfid, void *buf · 384 unmodified lines | |||
| 385 | 385 | ||
| 386 | if (xe_gt_is_media_type(gt)) | 386 | if (xe_gt_is_media_type(gt)) |
| 387 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) | 387 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) |
| 388 | regs[n] = xe_mmio_read32(>->mmio, MED_VF_SW_FLAG(n)); | 388 | regs[n] = xe_mmio_read32(&mmio, MED_VF_SW_FLAG(n)); |
| 389 | else | 389 | else |
| 390 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) | 390 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) |
| 391 | regs[n] = xe_mmio_read32(>->mmio, VF_SW_FLAG(n)); | 391 | regs[n] = xe_mmio_read32(&mmio, VF_SW_FLAG(n)); |
| 392 | 392 | ||
| 393 | return 0; | 393 | return 0; |
| 394 | } | 394 | } |
| @@ -407,10 +407,10 @@ static int pf_migration_mmio_restore(struct xe_gt *gt, unsigned int vfid, · 12 unmodified lines | |||
| 407 | 407 | ||
| 408 | if (xe_gt_is_media_type(gt)) | 408 | if (xe_gt_is_media_type(gt)) |
| 409 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) | 409 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) |
| 410 | xe_mmio_write32(>->mmio, MED_VF_SW_FLAG(n), regs[n]); | 410 | xe_mmio_write32(&mmio, MED_VF_SW_FLAG(n), regs[n]); |
| 411 | else | 411 | else |
| 412 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) | 412 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) |
| 413 | xe_mmio_write32(>->mmio, VF_SW_FLAG(n), regs[n]); | 413 | xe_mmio_write32(&mmio, VF_SW_FLAG(n), regs[n]); |
| 414 | 414 | ||
| 415 | return 0; | 415 | return 0; |
| 416 | } | 416 | } |
| @@ -114,8 +114,10 @@ int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt *gt, const u32 *msg, u32 · 113 unmodified lines | |||
| 114 | * VFs with no events are not printed. | 114 | * VFs with no events are not printed. |
| 115 | * | 115 | * |
| 116 | * This function can only be called on PF. | 116 | * This function can only be called on PF. |
| 117 | * | ||
| 118 | * Return: always 0 | ||
| 117 | */ | 119 | */ |
| 118 | void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p) | 120 | int xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p) |
| 119 | { | 121 | { |
| 120 | unsigned int n, total_vfs = xe_gt_sriov_pf_get_totalvfs(gt); | 122 | unsigned int n, total_vfs = xe_gt_sriov_pf_get_totalvfs(gt); |
| 121 | const struct xe_gt_sriov_monitor *data; | 123 | const struct xe_gt_sriov_monitor *data; |
| @@ -144,4 +146,6 @@ void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p · 22 unmodified lines | |||
| 144 | #undef __format | 146 | #undef __format |
| 145 | #undef __value | 147 | #undef __value |
| 146 | } | 148 | } |
| 149 | |||
| 150 | return 0; | ||
| 147 | } | 151 | } |
| @@ -13,7 +13,7 @@ struct drm_printer; · 12 unmodified lines | |||
| 13 | struct xe_gt; | 13 | struct xe_gt; |
| 14 | 14 | ||
| 15 | void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid); | 15 | void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid); |
| 16 | void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p); | 16 | int xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p); |
| 17 | 17 | ||
| 18 | #ifdef CONFIG_PCI_IOV | 18 | #ifdef CONFIG_PCI_IOV |
| 19 | int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt *gt, const u32 *msg, u32 len); | 19 | int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt *gt, const u32 *msg, u32 len); |
| @@ -1137,13 +1137,15 @@ void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val) · 1136 unmodified lines | |||
| 1137 | } | 1137 | } |
| 1138 | 1138 | ||
| 1139 | /** | 1139 | /** |
| 1140 | * xe_gt_sriov_vf_print_config - Print VF self config. | 1140 | * xe_gt_sriov_vf_print_config() - Print VF self config. |
| 1141 | * @gt: the &xe_gt | 1141 | * @gt: the &xe_gt |
| 1142 | * @p: the &drm_printer | 1142 | * @p: the &drm_printer |
| 1143 | * | 1143 | * |
| 1144 | * This function is for VF use only. | 1144 | * This function is for VF use only. |
| 1145 | * | ||
| 1146 | * Return: always 0. | ||
| 1145 | */ | 1147 | */ |
| 1146 | void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) | 1148 | int xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) |
| 1147 | { | 1149 | { |
| 1148 | struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; | 1150 | struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; |
| 1149 | struct xe_device *xe = gt_to_xe(gt); | 1151 | struct xe_device *xe = gt_to_xe(gt); |
| @@ -1170,16 +1172,20 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) · 20 unmodified lines | |||
| 1170 | 1172 | ||
| 1171 | drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs); | 1173 | drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs); |
| 1172 | drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs); | 1174 | drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs); |
| 1175 | |||
| 1176 | return 0; | ||
| 1173 | } | 1177 | } |
| 1174 | 1178 | ||
| 1175 | /** | 1179 | /** |
| 1176 | * xe_gt_sriov_vf_print_runtime - Print VF's runtime regs received from PF. | 1180 | * xe_gt_sriov_vf_print_runtime() - Print VF's runtime regs received from PF. |
| 1177 | * @gt: the &xe_gt | 1181 | * @gt: the &xe_gt |
| 1178 | * @p: the &drm_printer | 1182 | * @p: the &drm_printer |
| 1179 | * | 1183 | * |
| 1180 | * This function is for VF use only. | 1184 | * This function is for VF use only. |
| 1185 | * | ||
| 1186 | * Return: always 0. | ||
| 1181 | */ | 1187 | */ |
| 1182 | void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) | 1188 | int xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) |
| 1183 | { | 1189 | { |
| 1184 | struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs; | 1190 | struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs; |
| 1185 | unsigned int size = gt->sriov.vf.runtime.num_regs; | 1191 | unsigned int size = gt->sriov.vf.runtime.num_regs; |
| @@ -1188,16 +1194,20 @@ void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) · 2 unmodified lines | |||
| 1188 | 1194 | ||
| 1189 | for (; size--; vf_regs++) | 1195 | for (; size--; vf_regs++) |
| 1190 | drm_printf(p, "%#x = %#x\n", vf_regs->offset, vf_regs->value); | 1196 | drm_printf(p, "%#x = %#x\n", vf_regs->offset, vf_regs->value); |
| 1197 | |||
| 1198 | return 0; | ||
| 1191 | } | 1199 | } |
| 1192 | 1200 | ||
| 1193 | /** | 1201 | /** |
| 1194 | * xe_gt_sriov_vf_print_version - Print VF ABI versions. | 1202 | * xe_gt_sriov_vf_print_version() - Print VF ABI versions. |
| 1195 | * @gt: the &xe_gt | 1203 | * @gt: the &xe_gt |
| 1196 | * @p: the &drm_printer | 1204 | * @p: the &drm_printer |
| 1197 | * | 1205 | * |
| 1198 | * This function is for VF use only. | 1206 | * This function is for VF use only. |
| 1207 | * | ||
| 1208 | * Return: always 0. | ||
| 1199 | */ | 1209 | */ |
| 1200 | void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) | 1210 | int xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) |
| 1201 | { | 1211 | { |
| 1202 | struct xe_device *xe = gt_to_xe(gt); | 1212 | struct xe_device *xe = gt_to_xe(gt); |
| 1203 | struct xe_uc_fw_version *guc_version = >->sriov.vf.guc_version; | 1213 | struct xe_uc_fw_version *guc_version = >->sriov.vf.guc_version; |
| @@ -1227,6 +1237,8 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · 23 unmodified lines | |||
| 1227 | GUC_RELAY_VERSION_LATEST_MAJOR, GUC_RELAY_VERSION_LATEST_MINOR); | 1237 | GUC_RELAY_VERSION_LATEST_MAJOR, GUC_RELAY_VERSION_LATEST_MINOR); |
| 1228 | drm_printf(p, "\thandshake:\t%u.%u\n", | 1238 | drm_printf(p, "\thandshake:\t%u.%u\n", |
| 1229 | pf_version->major, pf_version->minor); | 1239 | pf_version->major, pf_version->minor); |
| 1240 | |||
| 1241 | return 0; | ||
| 1230 | } | 1242 | } |
| 1231 | 1243 | ||
| 1232 | static bool vf_post_migration_shutdown(struct xe_gt *gt) | 1244 | static bool vf_post_migration_shutdown(struct xe_gt *gt) |
| @@ -35,9 +35,9 @@ bool xe_gt_sriov_vf_sched_groups_enabled(struct xe_gt *gt); · 34 unmodified lines | |||
| 35 | u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg); | 35 | u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg); |
| 36 | void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val); | 36 | void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val); |
| 37 | 37 | ||
| 38 | void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p); | 38 | int xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p); |
| 39 | void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p); | 39 | int xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p); |
| 40 | void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); | 40 | int xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); |
| 41 | 41 | ||
| 42 | int xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt); | 42 | int xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt); |
| 43 | int xe_vf_migration_fixups_complete_count(struct xe_gt *gt); | 43 | int xe_vf_migration_fixups_complete_count(struct xe_gt *gt); |
| @@ -144,6 +144,13 @@ struct xe_gt { · 143 unmodified lines | |||
| 144 | u8 id; | 144 | u8 id; |
| 145 | /** @info.has_indirect_ring_state: GT has indirect ring state support */ | 145 | /** @info.has_indirect_ring_state: GT has indirect ring state support */ |
| 146 | u8 has_indirect_ring_state:1; | 146 | u8 has_indirect_ring_state:1; |
| 147 | /** | ||
| 148 | * @info.has_xe2_blt_instructions: GT supports Xe2-style MEM_SET | ||
| 149 | * and MEM_COPY blitter functionality. Note that despite the | ||
| 150 | * name, some Xe1 platforms may also support this "Xe2-style" | ||
| 151 | * feature. | ||
| 152 | */ | ||
| 153 | u8 has_xe2_blt_instructions:1; | ||
| 147 | /** | 154 | /** |
| 148 | * @info.num_geometry_xecore_fuse_regs: Number of 32b-bit fuse | 155 | * @info.num_geometry_xecore_fuse_regs: Number of 32b-bit fuse |
| 149 | * registers the geometry XeCore mask spans. | 156 | * registers the geometry XeCore mask spans. |
| @@ -512,12 +512,9 @@ static void guc_golden_lrc_init(struct xe_guc_ads *ads) · 511 unmodified lines | |||
| 512 | * that starts after the execlists LRC registers. This is | 512 | * that starts after the execlists LRC registers. This is |
| 513 | * required to allow the GuC to restore just the engine state | 513 | * required to allow the GuC to restore just the engine state |
| 514 | * when a watchdog reset occurs. | 514 | * when a watchdog reset occurs. |
| 515 | * We calculate the engine state size by removing the size of | ||
| 516 | * what comes before it in the context image (which is identical | ||
| 517 | * on all engines). | ||
| 518 | */ | 515 | */ |
| 519 | ads_blob_write(ads, ads.eng_state_size[guc_class], | 516 | ads_blob_write(ads, ads.eng_state_size[guc_class], |
| 520 | real_size - xe_lrc_skip_size(xe)); | 517 | xe_lrc_engine_state_size(gt, class)); |
| 521 | ads_blob_write(ads, ads.golden_context_lrca[guc_class], | 518 | ads_blob_write(ads, ads.golden_context_lrca[guc_class], |
| 522 | addr_ggtt); | 519 | addr_ggtt); |
| 523 | 520 | ||
| @@ -770,6 +767,11 @@ static unsigned int guc_mmio_regset_write(struct xe_guc_ads *ads, · 246 unmodified lines | |||
| 770 | } | 767 | } |
| 771 | } | 768 | } |
| 772 | 769 | ||
| 770 | if (XE_GT_WA(hwe->gt, 16023105232)) | ||
| 771 | guc_mmio_regset_write_one(ads, regset_map, | ||
| 772 | RING_IDLEDLY(hwe->mmio_base), | ||
| 773 | count++); | ||
| 774 | |||
| 773 | return count; | 775 | return count; |
| 774 | } | 776 | } |
| 775 | 777 | ||
| @@ -259,24 +259,12 @@ static void guc_submit_sw_fini(struct drm_device *drm, void *arg) · 258 unmodified lines | |||
| 259 | } | 259 | } |
| 260 | 260 | ||
| 261 | static void guc_submit_fini(void *arg) | 261 | static void guc_submit_fini(void *arg) |
| 262 | { | ||
| 263 | struct xe_guc *guc = arg; | ||
| 264 | |||
| 265 | /* Forcefully kill any remaining exec queues */ | ||
| 266 | xe_guc_ct_stop(&guc->ct); | ||
| 267 | guc_submit_reset_prepare(guc); | ||
| 268 | xe_guc_softreset(guc); | ||
| 269 | xe_guc_submit_stop(guc); | ||
| 270 | xe_uc_fw_sanitize(&guc->fw); | ||
| 271 | xe_guc_submit_pause_abort(guc); | ||
| 272 | } | ||
| 273 | |||
| 274 | static void guc_submit_wedged_fini(void *arg) | ||
| 275 | { | 262 | { |
| 276 | struct xe_guc *guc = arg; | 263 | struct xe_guc *guc = arg; |
| 277 | struct xe_exec_queue *q; | 264 | struct xe_exec_queue *q; |
| 278 | unsigned long index; | 265 | unsigned long index; |
| 279 | 266 | ||
| 267 | /* Drop any wedged queue refs */ | ||
| 280 | mutex_lock(&guc->submission_state.lock); | 268 | mutex_lock(&guc->submission_state.lock); |
| 281 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) { | 269 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) { |
| 282 | if (exec_queue_wedged(q)) { | 270 | if (exec_queue_wedged(q)) { |
| @@ -286,6 +274,14 @@ static void guc_submit_wedged_fini(void *arg) · 3 unmodified lines | |||
| 286 | } | 274 | } |
| 287 | } | 275 | } |
| 288 | mutex_unlock(&guc->submission_state.lock); | 276 | mutex_unlock(&guc->submission_state.lock); |
| 277 | |||
| 278 | /* Forcefully kill any remaining exec queues */ | ||
| 279 | xe_guc_ct_stop(&guc->ct); | ||
| 280 | guc_submit_reset_prepare(guc); | ||
| 281 | xe_guc_softreset(guc); | ||
| 282 | xe_guc_submit_stop(guc); | ||
| 283 | xe_uc_fw_sanitize(&guc->fw); | ||
| 284 | xe_guc_submit_pause_abort(guc); | ||
| 289 | } | 285 | } |
| 290 | 286 | ||
| 291 | static const struct xe_exec_queue_ops guc_exec_queue_ops; | 287 | static const struct xe_exec_queue_ops guc_exec_queue_ops; |
| @@ -1320,10 +1316,8 @@ static void disable_scheduling_deregister(struct xe_guc *guc, · 1028 unmodified lines | |||
| 1320 | void xe_guc_submit_wedge(struct xe_guc *guc) | 1316 | void xe_guc_submit_wedge(struct xe_guc *guc) |
| 1321 | { | 1317 | { |
| 1322 | struct xe_device *xe = guc_to_xe(guc); | 1318 | struct xe_device *xe = guc_to_xe(guc); |
| 1323 | struct xe_gt *gt = guc_to_gt(guc); | ||
| 1324 | struct xe_exec_queue *q; | 1319 | struct xe_exec_queue *q; |
| 1325 | unsigned long index; | 1320 | unsigned long index; |
| 1326 | int err; | ||
| 1327 | 1321 | ||
| 1328 | xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode); | 1322 | xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode); |
| 1329 | 1323 | ||
| @@ -1335,15 +1329,6 @@ void xe_guc_submit_wedge(struct xe_guc *guc) · 5 unmodified lines | |||
| 1335 | return; | 1329 | return; |
| 1336 | 1330 | ||
| 1337 | if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) { | 1331 | if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) { |
| 1338 | err = devm_add_action_or_reset(guc_to_xe(guc)->drm.dev, | ||
| 1339 | guc_submit_wedged_fini, guc); | ||
| 1340 | if (err) { | ||
| 1341 | xe_gt_err(gt, "Failed to register clean-up on wedged.mode=%s; " | ||
| 1342 | "Although device is wedged.\n", | ||
| 1343 | xe_wedged_mode_to_string(XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)); | ||
| 1344 | return; | ||
| 1345 | } | ||
| 1346 | |||
| 1347 | mutex_lock(&guc->submission_state.lock); | 1332 | mutex_lock(&guc->submission_state.lock); |
| 1348 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) | 1333 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) |
| 1349 | if (xe_exec_queue_get_unless_zero(q)) | 1334 | if (xe_exec_queue_get_unless_zero(q)) |
| @@ -1688,6 +1673,14 @@ static void guc_exec_queue_fini(struct xe_exec_queue *q) · 338 unmodified lines | |||
| 1688 | struct xe_guc_exec_queue *ge = q->guc; | 1673 | struct xe_guc_exec_queue *ge = q->guc; |
| 1689 | struct xe_guc *guc = exec_queue_to_guc(q); | 1674 | struct xe_guc *guc = exec_queue_to_guc(q); |
| 1690 | 1675 | ||
| 1676 | if (xe_exec_queue_is_multi_queue_secondary(q)) { | ||
| 1677 | struct xe_exec_queue_group *group = q->multi_queue.group; | ||
| 1678 | |||
| 1679 | mutex_lock(&group->list_lock); | ||
| 1680 | list_del(&q->multi_queue.link); | ||
| 1681 | mutex_unlock(&group->list_lock); | ||
| 1682 | } | ||
| 1683 | |||
| 1691 | release_guc_id(guc, q); | 1684 | release_guc_id(guc, q); |
| 1692 | xe_sched_entity_fini(&ge->entity); | 1685 | xe_sched_entity_fini(&ge->entity); |
| 1693 | xe_sched_fini(&ge->sched); | 1686 | xe_sched_fini(&ge->sched); |
| @@ -1709,14 +1702,6 @@ static void __guc_exec_queue_destroy_async(struct work_struct *w) · 15 unmodified lines | |||
| 1709 | guard(xe_pm_runtime)(guc_to_xe(guc)); | 1702 | guard(xe_pm_runtime)(guc_to_xe(guc)); |
| 1710 | trace_xe_exec_queue_destroy(q); | 1703 | trace_xe_exec_queue_destroy(q); |
| 1711 | 1704 | ||
| 1712 | if (xe_exec_queue_is_multi_queue_secondary(q)) { | ||
| 1713 | struct xe_exec_queue_group *group = q->multi_queue.group; | ||
| 1714 | |||
| 1715 | mutex_lock(&group->list_lock); | ||
| 1716 | list_del(&q->multi_queue.link); | ||
| 1717 | mutex_unlock(&group->list_lock); | ||
| 1718 | } | ||
| 1719 | |||
| 1720 | /* Confirm no work left behind accessing device structures */ | 1705 | /* Confirm no work left behind accessing device structures */ |
| 1721 | cancel_delayed_work_sync(&ge->sched.base.work_tdr); | 1706 | cancel_delayed_work_sync(&ge->sched.base.work_tdr); |
| 1722 | 1707 | ||
| @@ -746,9 +746,16 @@ size_t xe_lrc_reg_size(struct xe_device *xe) · 745 unmodified lines | |||
| 746 | return 80 * sizeof(u32); | 746 | return 80 * sizeof(u32); |
| 747 | } | 747 | } |
| 748 | 748 | ||
| 749 | size_t xe_lrc_skip_size(struct xe_device *xe) | 749 | /** |
| 750 | * xe_lrc_engine_state_size() - Get size of the engine state within LRC | ||
| 751 | * @gt: the &xe_gt struct instance | ||
| 752 | * @class: Hardware engine class | ||
| 753 | * | ||
| 754 | * Returns: Size of the engine state | ||
| 755 | */ | ||
| 756 | size_t xe_lrc_engine_state_size(struct xe_gt *gt, enum xe_engine_class class) | ||
| 750 | { | 757 | { |
| 751 | return LRC_PPHWSP_SIZE + xe_lrc_reg_size(xe); | 758 | return xe_gt_lrc_hang_replay_size(gt, class) - xe_lrc_reg_size(gt_to_xe(gt)); |
| 752 | } | 759 | } |
| 753 | 760 | ||
| 754 | static inline u32 __xe_lrc_seqno_offset(struct xe_lrc *lrc) | 761 | static inline u32 __xe_lrc_seqno_offset(struct xe_lrc *lrc) |
| @@ -1214,7 +1221,7 @@ static ssize_t setup_invalidate_state_cache_wa(struct xe_lrc *lrc, · 459 unmodified lines | |||
| 1214 | if (xe_gt_WARN_ON(lrc->gt, max_len < 3)) | 1221 | if (xe_gt_WARN_ON(lrc->gt, max_len < 3)) |
| 1215 | return -ENOSPC; | 1222 | return -ENOSPC; |
| 1216 | 1223 | ||
| 1217 | *cmd++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | 1224 | *cmd++ = MI_LOAD_REGISTER_IMM | MI_LRI_LRM_CS_MMIO | MI_LRI_NUM_REGS(1); |
| 1218 | *cmd++ = CS_DEBUG_MODE2(0).addr; | 1225 | *cmd++ = CS_DEBUG_MODE2(0).addr; |
| 1219 | *cmd++ = REG_MASKED_FIELD_ENABLE(INSTRUCTION_STATE_CACHE_INVALIDATE); | 1226 | *cmd++ = REG_MASKED_FIELD_ENABLE(INSTRUCTION_STATE_CACHE_INVALIDATE); |
| 1220 | 1227 | ||
| @@ -130,7 +130,7 @@ u32 xe_lrc_parallel_ggtt_addr(struct xe_lrc *lrc); · 129 unmodified lines | |||
| 130 | struct iosys_map xe_lrc_parallel_map(struct xe_lrc *lrc); | 130 | struct iosys_map xe_lrc_parallel_map(struct xe_lrc *lrc); |
| 131 | 131 | ||
| 132 | size_t xe_lrc_reg_size(struct xe_device *xe); | 132 | size_t xe_lrc_reg_size(struct xe_device *xe); |
| 133 | size_t xe_lrc_skip_size(struct xe_device *xe); | 133 | size_t xe_lrc_engine_state_size(struct xe_gt *gt, enum xe_engine_class class); |
| 134 | 134 | ||
| 135 | void xe_lrc_dump_default(struct drm_printer *p, | 135 | void xe_lrc_dump_default(struct drm_printer *p, |
| 136 | struct xe_gt *gt, | 136 | struct xe_gt *gt, |
| @@ -0,0 +1,403 @@ | |||
| 1 | // SPDX-License-Identifier: MIT | ||
| 2 | /* | ||
| 3 | * Copyright © 2026 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <linux/kernel.h> | ||
| 7 | |||
| 8 | #include <drm/drm_managed.h> | ||
| 9 | |||
| 10 | #include "instructions/xe_mi_commands.h" | ||
| 11 | #include "xe_bo.h" | ||
| 12 | #include "xe_device_types.h" | ||
| 13 | #include "xe_map.h" | ||
| 14 | #include "xe_mem_pool.h" | ||
| 15 | #include "xe_mem_pool_types.h" | ||
| 16 | #include "xe_tile_printk.h" | ||
| 17 | |||
| 18 | /** | ||
| 19 | * struct xe_mem_pool - DRM MM pool for sub-allocating memory from a BO on an | ||
| 20 | * XE tile. | ||
| 21 | * | ||
| 22 | * The XE memory pool is a DRM MM manager that provides sub-allocation of memory | ||
| 23 | * from a backing buffer object (BO) on a specific XE tile. It is designed to | ||
| 24 | * manage memory for GPU workloads, allowing for efficient allocation and | ||
| 25 | * deallocation of memory regions within the BO. | ||
| 26 | * | ||
| 27 | * The memory pool maintains a primary BO that is pinned in the GGTT and mapped | ||
| 28 | * into the CPU address space for direct access. Optionally, it can also maintain | ||
| 29 | * a shadow BO that can be used for atomic updates to the primary BO's contents. | ||
| 30 | * | ||
| 31 | * The API provided by the memory pool allows clients to allocate and free memory | ||
| 32 | * regions, retrieve GPU and CPU addresses, and synchronize data between the | ||
| 33 | * primary and shadow BOs as needed. | ||
| 34 | */ | ||
| 35 | struct xe_mem_pool { | ||
| 36 | /** @base: Range allocator over [0, @size) in bytes */ | ||
| 37 | struct drm_mm base; | ||
| 38 | /** @bo: Active pool BO (GGTT-pinned, CPU-mapped). */ | ||
| 39 | struct xe_bo *bo; | ||
| 40 | /** @shadow: Shadow BO for atomic command updates. */ | ||
363 more lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 140 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 131 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 209 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 554 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 380 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 311 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 386 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 1544 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 303 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 653 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 439 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 168 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 325 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 271 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 415 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 485 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 192 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 279 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 193 lines aren’t shown here.
This diff is large: its 303 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 324 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 201 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 613 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 340 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 236 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 465 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 310 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 1465 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 330 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 228 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 512 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 181 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 208 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 176 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 226 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 784 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 301 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 249 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 686 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 383 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 792 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 213 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 332 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 143 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 295 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 323 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 395 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 184 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 143 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 157 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 450 lines aren’t shown here.
This diff is large: its 210 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 277 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 211 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 204 lines aren’t shown here.
This diff is large: its 429 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 241 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 806 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 726 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 156 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 407 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 874 lines aren’t shown here.
This diff is large: its 267 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 171 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 726 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 468 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 243 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 260 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 296 lines aren’t shown here.
This diff is large: its 291 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 211 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 217 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 622 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.