Linux 7.1-rc6
Unified · Split
| @@ -19,6 +19,7 @@ Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org> · 18 unmodified lines | ||
| 19 | 19 | Ahmad Masri <quic_amasri@quicinc.com> <amasri@codeaurora.org> |
| 20 | 20 | Adam Oldham <oldhamca@gmail.com> |
| 21 | 21 | Adam Radford <aradford@gmail.com> |
| 22 | +Aditya Garg <gargaditya08@proton.me> <gargaditya08@live.com> | |
| 22 | 23 | Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com> |
| 23 | 24 | Adrian Bunk <bunk@stusta.de> |
| 24 | 25 | Ajay Kaher <ajay.kaher@broadcom.com> <akaher@vmware.com> |
| @@ -207,6 +208,7 @@ Claudiu Beznea <claudiu.beznea@tuxon.dev> <claudiu.beznea@microchip.com> · 182 unmodified lines | ||
| 207 | 208 | Colin Ian King <colin.i.king@gmail.com> <colin.king@canonical.com> |
| 208 | 209 | Corey Minyard <minyard@acm.org> |
| 209 | 210 | Damian Hobson-Garcia <dhobsong@igel.co.jp> |
| 211 | +Dan Carpenter <error27@gmail.com> <dan.carpenter@linaro.org> | |
| 210 | 212 | Dan Carpenter <error27@gmail.com> <dan.carpenter@oracle.com> |
| 211 | 213 | Dan Williams <djbw@kernel.org> <dan.j.williams@intel.com> |
| 212 | 214 | Daniel Borkmann <daniel@iogearbox.net> <danborkmann@googlemail.com> |
| @@ -261,8 +263,9 @@ Enric Balletbo i Serra <eballetbo@kernel.org> <enric.balletbo@collabora.com> · 48 unmodified lines | ||
| 261 | 263 | Enric Balletbo i Serra <eballetbo@kernel.org> <eballetbo@iseebcn.com> |
| 262 | 264 | Erik Kaneda <erik.kaneda@intel.com> <erik.schmauss@intel.com> |
| 263 | 265 | Ethan Carter Edwards <ethan@ethancedwards.com> Ethan Edwards <ethancarteredwards@gmail.com> |
| 264 | −Eugen Hristev <eugen.hristev@linaro.org> <eugen.hristev@microchip.com> | |
| 265 | −Eugen Hristev <eugen.hristev@linaro.org> <eugen.hristev@collabora.com> | |
| 266 | +Eugen Hristev <ehristev@kernel.org> <eugen.hristev@microchip.com> | |
| 267 | +Eugen Hristev <ehristev@kernel.org> <eugen.hristev@linaro.org> | |
| 268 | +Eugen Hristev <ehristev@kernel.org> <eugen.hristev@collabora.com> | |
| 266 | 269 | Evgeniy Polyakov <johnpol@2ka.mipt.ru> |
| 267 | 270 | Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> <ezequiel@collabora.com> |
| 268 | 271 | Faith Ekstrand <faith.ekstrand@collabora.com> <jason@jlekstrand.net> |
| @@ -337,6 +340,7 @@ Henrik Rydberg <rydberg@bitmath.org> · 68 unmodified lines | ||
| 337 | 340 | Herbert Xu <herbert@gondor.apana.org.au> |
| 338 | 341 | Huacai Chen <chenhuacai@kernel.org> <chenhc@lemote.com> |
| 339 | 342 | Huacai Chen <chenhuacai@kernel.org> <chenhuacai@loongson.cn> |
| 343 | +Ian Ray <ian.ray@gehealthcare.com> <ian.ray@ge.com> | |
| 340 | 344 | Ignat Korchagin <ignat@linux.win> <ignat@cloudflare.com> |
| 341 | 345 | Igor Korotin <igor.korotin@linux.dev> <igor.korotin.linux@gmail.com> |
| 342 | 346 | Ike Panhc <ikepanhc@gmail.com> <ike.pan@canonical.com> |
| @@ -495,6 +499,7 @@ Leon Romanovsky <leon@kernel.org> <leon@leon.nu> · 152 unmodified lines | ||
| 495 | 499 | Leon Romanovsky <leon@kernel.org> <leonro@mellanox.com> |
| 496 | 500 | Leon Romanovsky <leon@kernel.org> <leonro@nvidia.com> |
| 497 | 501 | Leo Yan <leo.yan@linux.dev> <leo.yan@linaro.org> |
| 502 | +Liam R. Howlett <liam@infradead.org> <Liam.Howlett@oracle.com> | |
| 498 | 503 | Liam Mark <quic_lmark@quicinc.com> <lmark@codeaurora.org> |
| 499 | 504 | Linas Vepstas <linas@austin.ibm.com> |
| 500 | 505 | Linus Lüssing <linus.luessing@c0d3.blue> <linus.luessing@ascom.ch> |
| @@ -505,6 +510,8 @@ Linus Walleij <linusw@kernel.org> <linus.walleij@stericsson.com> · 4 unmodified lines | ||
| 505 | 510 | Linus Walleij <linusw@kernel.org> <linus.walleij@linaro.org> |
| 506 | 511 | Linus Walleij <linusw@kernel.org> <triad@df.lth.se> |
| 507 | 512 | <linux-hardening@vger.kernel.org> <kernel-hardening@lists.openwall.com> |
| 513 | +Li Wang <li.wang@linux.dev> <liwang@redhat.com> | |
| 514 | +Li Wang <li.wang@linux.dev> <wangli.ahau@gmail.com> | |
| 508 | 515 | Li Yang <leoyang.li@nxp.com> <leoli@freescale.com> |
| 509 | 516 | Li Yang <leoyang.li@nxp.com> <leo@zh-kernel.org> |
| 510 | 517 | Lior David <quic_liord@quicinc.com> <liord@codeaurora.org> |
| @@ -579,6 +586,8 @@ Mayuresh Janorkar <mayur@ti.com> · 68 unmodified lines | ||
| 579 | 586 | Md Sadre Alam <quic_mdalam@quicinc.com> <mdalam@codeaurora.org> |
| 580 | 587 | Miaoqing Pan <quic_miaoqing@quicinc.com> <miaoqing@codeaurora.org> |
| 581 | 588 | Michael Buesch <m@bues.ch> |
| 589 | +Michal Grzeschik <mgr@kernel.org> <m.grzeschik@pengutronix.de> | |
| 590 | +Michal Grzeschik <mgr@kernel.org> <mgr@pengutronix.de> | |
| 582 | 591 | Michael Riesch <michael.riesch@collabora.com> <michael.riesch@wolfvision.net> |
| 583 | 592 | Michal Simek <michal.simek@amd.com> <michal.simek@xilinx.com> |
| 584 | 593 | Michel Dänzer <michel@tungstengraphics.com> |
| @@ -677,6 +686,7 @@ Peter A Jonsson <pj@ludd.ltu.se> · 92 unmodified lines | ||
| 677 | 686 | Peter Hilber <peter.hilber@oss.qualcomm.com> <quic_philber@quicinc.com> |
| 678 | 687 | Peter Oruba <peter.oruba@amd.com> |
| 679 | 688 | Peter Oruba <peter@oruba.de> |
| 689 | +Peter Rosin <peda@lysator.liu.se> <peda@axentia.se> | |
| 680 | 690 | Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> <pierre-louis.bossart@linux.intel.com> |
| 681 | 691 | Pratyush Anand <pratyush.anand@gmail.com> <pratyush.anand@st.com> |
| 682 | 692 | Pratyush Yadav <pratyush@kernel.org> <ptyadav@amazon.de> |
| @@ -687,6 +697,7 @@ Punit Agrawal <punitagrawal@gmail.com> <punit.agrawal@arm.com> · 4 unmodified lines | ||
| 687 | 697 | Puranjay Mohan <puranjay@kernel.org> <puranjay12@gmail.com> |
| 688 | 698 | Qais Yousef <qyousef@layalina.io> <qais.yousef@imgtec.com> |
| 689 | 699 | Qais Yousef <qyousef@layalina.io> <qais.yousef@arm.com> |
| 700 | +Qi Zheng <qi.zheng@linux.dev> <zhengqi.arch@bytedance.com> | |
| 690 | 701 | Quentin Monnet <qmo@kernel.org> <quentin.monnet@netronome.com> |
| 691 | 702 | Quentin Monnet <qmo@kernel.org> <quentin@isovalent.com> |
| 692 | 703 | Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com> |
| @@ -850,6 +861,7 @@ Tobias Klauser <tklauser@distanz.ch> <klto@zhaw.ch> · 157 unmodified lines | ||
| 850 | 861 | Tobias Klauser <tklauser@distanz.ch> <tklauser@nuerscht.ch> |
| 851 | 862 | Tobias Klauser <tklauser@distanz.ch> <tklauser@xenon.tklauser.home> |
| 852 | 863 | Todor Tomov <todor.too@gmail.com> <todor.tomov@linaro.org> |
| 864 | +Tomasz Jeznach <tomasz.jeznach@linux.dev> <tjeznach@rivosinc.com> | |
| 853 | 865 | Tony Luck <tony.luck@intel.com> |
| 854 | 866 | Trilok Soni <quic_tsoni@quicinc.com> <tsoni@codeaurora.org> |
| 855 | 867 | TripleX Chung <xxx.phy@gmail.com> <triplex@zh-kernel.org> |
| @@ -786,6 +786,7 @@ networking/altera_tse networking/device_drivers/ethernet/altera/altera_tse · 785 unmodified lines | ||
| 786 | 786 | networking/bpf_flow_dissector bpf/prog_flow_dissector |
| 787 | 787 | networking/cxacru networking/device_drivers/atm/cxacru |
| 788 | 788 | networking/defza networking/device_drivers/fddi/defza |
| 789 | +networking/device_drivers/3com/3c509 networking/device_drivers/ethernet/3com/3c509 | |
| 789 | 790 | networking/device_drivers/3com/vortex networking/device_drivers/ethernet/3com/vortex |
| 790 | 791 | networking/device_drivers/amazon/ena networking/device_drivers/ethernet/amazon/ena |
| 791 | 792 | networking/device_drivers/aquantia/atlantic networking/device_drivers/ethernet/aquantia/atlantic |
File renamed without changes.
| @@ -47,21 +47,19 @@ Please note that implementation details can be changed. · 46 unmodified lines | ||
| 47 | 47 | Called when swp_entry's refcnt goes down to 0. A charge against swap |
| 48 | 48 | disappears. |
| 49 | 49 | |
| 50 | −3. charge-commit-cancel | |
| 50 | +3. charge-commit | |
| 51 | 51 | ======================= |
| 52 | 52 | |
| 53 | 53 | Memcg pages are charged in two steps: |
| 54 | 54 | |
| 55 | 55 | - mem_cgroup_try_charge() |
| 56 | − - mem_cgroup_commit_charge() or mem_cgroup_cancel_charge() | |
| 56 | + - commit_charge() | |
| 57 | 57 | |
| 58 | 58 | At try_charge(), there are no flags to say "this page is charged". |
| 59 | 59 | at this point, usage += PAGE_SIZE. |
| 60 | 60 | |
| 61 | 61 | At commit(), the page is associated with the memcg. |
| 62 | 62 | |
| 63 | − At cancel(), simply usage -= PAGE_SIZE. | |
| 64 | − | |
| 65 | 63 | Under below explanation, we assume CONFIG_SWAP=y. |
| 66 | 64 | |
| 67 | 65 | 4. Anonymous |
| @@ -220,7 +220,7 @@ cgroup v2 currently supports the following mount options. · 219 unmodified lines | ||
| 220 | 220 | memory_hugetlb_accounting |
| 221 | 221 | Count HugeTLB memory usage towards the cgroup's overall |
| 222 | 222 | memory usage for the memory controller (for the purpose of |
| 223 | − statistics reporting and memory protetion). This is a new | |
| 223 | + statistics reporting and memory protection). This is a new | |
| 224 | 224 | behavior that could regress existing setups, so it must be |
| 225 | 225 | explicitly opted in with this mount option. |
| 226 | 226 | |
| @@ -789,24 +789,6 @@ Kernel parameters · 788 unmodified lines | ||
| 789 | 789 | cio_ignore= [S390] |
| 790 | 790 | See Documentation/arch/s390/common_io.rst for details. |
| 791 | 791 | |
| 792 | − clearcpuid=X[,X...] [X86] | |
| 793 | − Disable CPUID feature X for the kernel. See | |
| 794 | − arch/x86/include/asm/cpufeatures.h for the valid bit | |
| 795 | − numbers X. Note the Linux-specific bits are not necessarily | |
| 796 | − stable over kernel options, but the vendor-specific | |
| 797 | − ones should be. | |
| 798 | − X can also be a string as appearing in the flags: line | |
| 799 | − in /proc/cpuinfo which does not have the above | |
| 800 | − instability issue. However, not all features have names | |
| 801 | − in /proc/cpuinfo. | |
| 802 | − Note that using this option will taint your kernel. | |
| 803 | − Also note that user programs calling CPUID directly | |
| 804 | − or using the feature without checking anything | |
| 805 | − will still see it. This just prevents it from | |
| 806 | − being used by the kernel or shown in /proc/cpuinfo. | |
| 807 | − Also note the kernel might malfunction if you disable | |
| 808 | − some critical bits. | |
| 809 | − | |
| 810 | 792 | clk_ignore_unused |
| 811 | 793 | [CLK] |
| 812 | 794 | Prevents the clock framework from automatically gating |
| @@ -43,6 +43,11 @@ Support for changing the platform performance mode is currently not implemented. · 42 unmodified lines | ||
| 43 | 43 | Battery Charging Control |
| 44 | 44 | ------------------------ |
| 45 | 45 | |
| 46 | +.. warning:: Some devices do not properly implement the charging threshold interface. Forcing | |
| 47 | + the driver to enable access to said interface on such devices might damage the | |
| 48 | + battery [1]_. Because of this the driver will not enable said feature even when | |
| 49 | + using the ``force`` module parameter. | |
| 50 | + | |
| 46 | 51 | The ``uniwill-laptop`` driver supports controlling the battery charge limit. This happens over |
| 47 | 52 | the standard ``charge_control_end_threshold`` power supply sysfs attribute. All values |
| 48 | 53 | between 1 and 100 percent are supported. |
| @@ -70,3 +75,8 @@ The ``uniwill-laptop`` driver allows to set the configurable TGP for devices wit · 21 unmodified lines | ||
| 70 | 75 | allow it. |
| 71 | 76 | |
| 72 | 77 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. |
| 78 | + | |
| 79 | +References | |
| 80 | +========== | |
| 81 | + | |
| 82 | +.. [1] https://www.reddit.com/r/XMG_gg/comments/ld9yyf/battery_limit_hidden_function_discovered_on/ | |
| @@ -358,9 +358,9 @@ Dynamic energy performance profile · 357 unmodified lines | ||
| 358 | 358 | The amd-pstate driver supports dynamically selecting the energy performance |
| 359 | 359 | profile based on whether the machine is running on AC or DC power. |
| 360 | 360 | |
| 361 | −Whether this behavior is enabled by default depends on the kernel | |
| 362 | −config option `CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP`. This behavior can also be overridden | |
| 363 | −at runtime by the sysfs file ``/sys/devices/system/cpu/cpufreq/policyX/dynamic_epp``. | |
| 361 | +Whether this behavior is enabled by default depends on the kernel command line option | |
| 362 | +``amd_dynamic_epp`` is set. This behavior can also be overridden | |
| 363 | +at runtime by the sysfs file ``/sys/devices/system/cpu/amd_pstate/dynamic_epp``. | |
| 364 | 364 | |
| 365 | 365 | When set to enabled, the driver will select a different energy performance |
| 366 | 366 | profile when the machine is running on battery or AC power. The driver will |
| @@ -485,9 +485,8 @@ kernel parameter ``amd_prefcore=disable``. · 118 unmodified lines | ||
| 485 | 485 | ``amd_dynamic_epp`` |
| 486 | 486 | |
| 487 | 487 | When AMD pstate is in auto mode, dynamic EPP will control whether the kernel |
| 488 | −autonomously changes the EPP mode. The default is configured by | |
| 489 | −``CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP`` but can be explicitly enabled with | |
| 490 | −``amd_dynamic_epp=enable`` or disabled with ``amd_dynamic_epp=disable``. | |
| 488 | +autonomously changes the EPP mode. The default is disabled. It can be enabled | |
| 489 | +with the kernel parameter ``amd_dynamic_epp=enable``. | |
| 491 | 490 | |
| 492 | 491 | User Space Interface in ``sysfs`` - General |
| 493 | 492 | =========================================== |
| @@ -355,11 +355,12 @@ HyperThreading (HT) in the context of Intel processors, is enabled on at least · 354 unmodified lines | ||
| 355 | 355 | one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely, |
| 356 | 356 | the priority of a given CPU reflects its highest HWP performance level which |
| 357 | 357 | causes the CPU scheduler to generally prefer more performant CPUs, so the less |
| 358 | −performant CPUs are used when the other ones are fully loaded. However, SMT | |
| 359 | −siblings (that is, logical CPUs sharing one physical core) are treated in a | |
| 360 | −special way such that if one of them is in use, the effective priority of the | |
| 361 | −other ones is lowered below the priorities of the CPUs located in the other | |
| 362 | −physical cores. | |
| 358 | +performant CPUs are used when the other ones are fully loaded. SMT siblings | |
| 359 | +(that is, logical CPUs sharing one physical core) are given the same priority. | |
| 360 | +The scheduler can pull tasks from lower-priority cores and place them on any | |
| 361 | +sibling. Since the scheduler spreads tasks among physical cores, tasks will be | |
| 362 | +placed on the SMT siblings of physical cores only after all physical cores are | |
| 363 | +busy. | |
| 363 | 364 | |
| 364 | 365 | This approach maximizes performance in the majority of cases, but unfortunately |
| 365 | 366 | it also leads to excessive energy usage in some important scenarios, like video |
| @@ -21,13 +21,13 @@ call at each patchable function entry, and patches it dynamically at runtime to · 20 unmodified lines | ||
| 21 | 21 | enable or disable the redirection. In the case of RISC-V, 2 instructions, |
| 22 | 22 | AUIPC + JALR, are required to compose a function call. However, it is impossible |
| 23 | 23 | to patch 2 instructions and expect that a concurrent read-side executes them |
| 24 | −without a race condition. This series makes atmoic code patching possible in | |
| 24 | +without a race condition. This series makes atomic code patching possible in | |
| 25 | 25 | RISC-V ftrace. Kernel preemption makes things even worse as it allows the old |
| 26 | 26 | state to persist across the patching process with stop_machine(). |
| 27 | 27 | |
| 28 | 28 | In order to get rid of stop_machine() and run dynamic ftrace with full kernel |
| 29 | 29 | preemption, we partially initialize each patchable function entry at boot-time, |
| 30 | −setting the first instruction to AUIPC, and the second to NOP. Now, atmoic | |
| 30 | +setting the first instruction to AUIPC, and the second to NOP. Now, atomic | |
| 31 | 31 | patching is possible because the kernel only has to update one instruction. |
| 32 | 32 | According to Ziccif, as long as an instruction is naturally aligned, the ISA |
| 33 | 33 | guarantee an atomic update. |
| @@ -36,8 +36,8 @@ By fixing down the first instruction, AUIPC, the range of the ftrace trampoline · 2 unmodified lines | ||
| 36 | 36 | is limited to +-2K from the predetermined target, ftrace_caller, due to the lack |
| 37 | 37 | of immediate encoding space in RISC-V. To address the issue, we introduce |
| 38 | 38 | CALL_OPS, where an 8B naturally align metadata is added in front of each |
| 39 | −pacthable function. The metadata is resolved at the first trampoline, then the | |
| 40 | −execution can be derect to another custom trampoline. | |
| 39 | +patchable function. The metadata is resolved at the first trampoline, then the | |
| 40 | +execution can be directed to another custom trampoline. | |
| 41 | 41 | |
| 42 | 42 | CMODX in the User Space |
| 43 | 43 | ----------------------- |
| @@ -78,7 +78,7 @@ the program. · 77 unmodified lines | ||
| 78 | 78 | |
| 79 | 79 | Per-task indirect branch tracking state can be monitored and |
| 80 | 80 | controlled via the :c:macro:`PR_GET_CFI` and :c:macro:`PR_SET_CFI` |
| 81 | −``prctl()` arguments (respectively), by supplying | |
| 81 | +``prctl()`` arguments (respectively), by supplying | |
| 82 | 82 | :c:macro:`PR_CFI_BRANCH_LANDING_PADS` as the second argument. These |
| 83 | 83 | are architecture-agnostic, and will return -EINVAL if the underlying |
| 84 | 84 | functionality is not supported. |
| @@ -187,6 +187,10 @@ to disable features using the feature number as defined in · 186 unmodified lines | ||
| 187 | 187 | Protection can be disabled using clearcpuid=514. The number 514 is calculated |
| 188 | 188 | from #define X86_FEATURE_UMIP (16*32 + 2). |
| 189 | 189 | |
| 190 | +DO NOT USE this cmdline option in production - it is meant to be used only as | |
| 191 | +a quick'n'dirty debugging aid to rule out a feature-enabling code is the | |
| 192 | +culprit. If you use it, it'll taint the kernel. | |
| 193 | + | |
| 190 | 194 | In addition, there exists a variety of custom command-line parameters that |
| 191 | 195 | disable specific features. The list of parameters includes, but is not limited |
| 192 | 196 | to, nofsgsbase, nosgx, noxsave, etc. 5-level paging can also be disabled using |
| @@ -158,13 +158,22 @@ returned. · 157 unmodified lines | ||
| 158 | 158 | When a message has been received, the location and size of the data with the |
| 159 | 159 | message can be determined by calling:: |
| 160 | 160 | |
| 161 | − void crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | |
| 162 | − enum krb5_crypto_mode mode, | |
| 163 | − size_t *_offset, size_t *_len); | |
| 161 | + int crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | |
| 162 | + enum krb5_crypto_mode mode, | |
| 163 | + size_t *_offset, size_t *_len); | |
| 164 | 164 | |
| 165 | 165 | The caller provides the offset and length of the message to the function, which |
| 166 | 166 | then alters those values to indicate the region containing the data (plus any |
| 167 | −padding). It is up to the caller to determine how much padding there is. | |
| 167 | +padding). It is up to the caller to determine how much padding there is. The | |
| 168 | +function returns an error if the length is too small or if the mode is | |
| 169 | +unsupported. An additional function:: | |
| 170 | + | |
| 171 | + int crypto_krb5_check_data_len(const struct krb5_enctype *krb5, | |
| 172 | + enum krb5_crypto_mode mode, | |
| 173 | + size_t len, size_t min_content); | |
| 174 | + | |
| 175 | +is provided to just do a basic check that the decrypted/verified message would | |
| 176 | +have a sufficient minimum payload. | |
| 168 | 177 | |
| 169 | 178 | Preparation Functions |
| 170 | 179 | --------------------- |
| @@ -219,6 +219,7 @@ allOf: · 218 unmodified lines | ||
| 219 | 219 | - required: |
| 220 | 220 | - "#sound-dai-cells" |
| 221 | 221 | else: |
| 222 | + $ref: /schemas/sound/dai-common.yaml# | |
| 222 | 223 | properties: |
| 223 | 224 | aux-bus: false |
| 224 | 225 | required: |
| @@ -243,7 +244,7 @@ allOf: · 18 unmodified lines | ||
| 243 | 244 | clocks: |
| 244 | 245 | minItems: 5 |
| 245 | 246 | maxItems: 5 |
| 246 | − clocks-names: | |
| 247 | + clock-names: | |
| 247 | 248 | minItems: 5 |
| 248 | 249 | maxItems: 5 |
| 249 | 250 | |
| @@ -264,7 +265,7 @@ allOf: · 14 unmodified lines | ||
| 264 | 265 | clocks: |
| 265 | 266 | minItems: 5 |
| 266 | 267 | maxItems: 6 |
| 267 | − clocks-names: | |
| 268 | + clock-names: | |
| 268 | 269 | minItems: 5 |
| 269 | 270 | maxItems: 6 |
| 270 | 271 | |
| @@ -277,7 +278,6 @@ allOf: · 6 unmodified lines | ||
| 277 | 278 | - qcom,sc8180x-dp |
| 278 | 279 | - qcom,sdm845-dp |
| 279 | 280 | - qcom,sm8350-dp |
| 280 | − - qcom,sm8650-dp | |
| 281 | 281 | then: |
| 282 | 282 | properties: |
| 283 | 283 | reg: |
| @@ -286,6 +286,24 @@ allOf: · 2 unmodified lines | ||
| 286 | 286 | clocks: |
| 287 | 287 | minItems: 6 |
| 288 | 288 | maxItems: 6 |
| 289 | + clock-names: | |
| 290 | + minItems: 6 | |
| 291 | + maxItems: 6 | |
| 292 | + | |
| 293 | + - if: | |
| 294 | + properties: | |
| 295 | + compatible: | |
| 296 | + contains: | |
| 297 | + enum: | |
| 298 | + - qcom,sm8650-dp | |
| 299 | + then: | |
| 300 | + properties: | |
| 301 | + reg: | |
| 302 | + minItems: 5 | |
| 303 | + maxItems: 9 | |
| 304 | + clocks: | |
| 305 | + minItems: 6 | |
| 306 | + maxItems: 6 | |
| 289 | 307 | clocks-names: |
| 290 | 308 | minItems: 6 |
| 291 | 309 | maxItems: 6 |
| @@ -306,7 +324,7 @@ allOf: · 14 unmodified lines | ||
| 306 | 324 | clocks: |
| 307 | 325 | minItems: 6 |
| 308 | 326 | maxItems: 8 |
| 309 | − clocks-names: | |
| 327 | + clock-names: | |
| 310 | 328 | minItems: 6 |
| 311 | 329 | maxItems: 8 |
| 312 | 330 | |
| @@ -326,7 +344,7 @@ allOf: · 13 unmodified lines | ||
| 326 | 344 | clocks: |
| 327 | 345 | minItems: 5 |
| 328 | 346 | maxItems: 6 |
| 329 | − clocks-names: | |
| 347 | + clock-names: | |
| 330 | 348 | minItems: 5 |
| 331 | 349 | maxItems: 6 |
| 332 | 350 | |
| @@ -119,7 +119,7 @@ examples: · 118 unmodified lines | ||
| 119 | 119 | mdss_mdp: display-controller@ae01000 { |
| 120 | 120 | compatible = "qcom,eliza-dpu"; |
| 121 | 121 | reg = <0x0ae01000 0x93000>, |
| 122 | − <0x0aeb0000 0x2008>; | |
| 122 | + <0x0aeb0000 0x3000>; | |
| 123 | 123 | reg-names = "mdp", |
| 124 | 124 | "vbif"; |
| 125 | 125 | |
| @@ -304,7 +304,7 @@ examples: · 178 unmodified lines | ||
| 304 | 304 | mdss_dsi0_phy: phy@ae95000 { |
| 305 | 305 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; |
| 306 | 306 | reg = <0x0ae95000 0x200>, |
| 307 | − <0x0ae95200 0x280>, | |
| 307 | + <0x0ae95200 0x300>, | |
| 308 | 308 | <0x0ae95500 0x400>; |
| 309 | 309 | reg-names = "dsi_phy", |
| 310 | 310 | "dsi_phy_lane", |
| @@ -388,7 +388,7 @@ examples: · 77 unmodified lines | ||
| 388 | 388 | mdss_dsi1_phy: phy@ae97000 { |
| 389 | 389 | compatible = "qcom,eliza-dsi-phy-4nm", "qcom,sm8650-dsi-phy-4nm"; |
| 390 | 390 | reg = <0x0ae97000 0x200>, |
| 391 | − <0x0ae97200 0x280>, | |
| 391 | + <0x0ae97200 0x300>, | |
| 392 | 392 | <0x0ae97500 0x400>; |
| 393 | 393 | reg-names = "dsi_phy", |
| 394 | 394 | "dsi_phy_lane", |
| @@ -407,11 +407,15 @@ examples: · 12 unmodified lines | ||
| 407 | 407 | |
| 408 | 408 | displayport-controller@af54000 { |
| 409 | 409 | compatible = "qcom,eliza-dp", "qcom,sm8650-dp"; |
| 410 | − reg = <0xaf54000 0x104>, | |
| 411 | − <0xaf54200 0xc0>, | |
| 412 | − <0xaf55000 0x770>, | |
| 413 | − <0xaf56000 0x9c>, | |
| 414 | − <0xaf57000 0x9c>; | |
| 410 | + reg = <0x0af54000 0x200>, | |
| 411 | + <0x0af54200 0x200>, | |
| 412 | + <0x0af55000 0xc00>, | |
| 413 | + <0x0af56000 0x400>, | |
| 414 | + <0x0af57000 0x400>, | |
| 415 | + <0x0af58000 0x400>, | |
| 416 | + <0x0af59000 0x400>, | |
| 417 | + <0x0af5a000 0x600>, | |
| 418 | + <0x0af5b000 0x600>; | |
| 415 | 419 | |
| 416 | 420 | interrupts-extended = <&mdss 12>; |
| 417 | 421 | |
| @@ -70,7 +70,7 @@ examples: · 69 unmodified lines | ||
| 70 | 70 | display-controller@ae01000 { |
| 71 | 71 | compatible = "qcom,sm8650-dpu"; |
| 72 | 72 | reg = <0x0ae01000 0x8f000>, |
| 73 | − <0x0aeb0000 0x2008>; | |
| 73 | + <0x0aeb0000 0x3000>; | |
| 74 | 74 | reg-names = "mdp", "vbif"; |
| 75 | 75 | |
| 76 | 76 | clocks = <&gcc_axi_clk>, |
| @@ -112,7 +112,7 @@ examples: · 111 unmodified lines | ||
| 112 | 112 | display-controller@ae01000 { |
| 113 | 113 | compatible = "qcom,sm8650-dpu"; |
| 114 | 114 | reg = <0x0ae01000 0x8f000>, |
| 115 | − <0x0aeb0000 0x2008>; | |
| 115 | + <0x0aeb0000 0x3000>; | |
| 116 | 116 | reg-names = "mdp", "vbif"; |
| 117 | 117 | |
| 118 | 118 | clocks = <&gcc_axi_clk>, |
| @@ -117,7 +117,7 @@ examples: · 116 unmodified lines | ||
| 117 | 117 | display-controller@ae01000 { |
| 118 | 118 | compatible = "qcom,sm8750-dpu"; |
| 119 | 119 | reg = <0x0ae01000 0x93000>, |
| 120 | − <0x0aeb0000 0x2008>; | |
| 120 | + <0x0aeb0000 0x3000>; | |
| 121 | 121 | reg-names = "mdp", |
| 122 | 122 | "vbif"; |
| 123 | 123 | |
| @@ -389,11 +389,15 @@ examples: · 265 unmodified lines | ||
| 389 | 389 | |
| 390 | 390 | displayport-controller@af54000 { |
| 391 | 391 | compatible = "qcom,sm8750-dp", "qcom,sm8650-dp"; |
| 392 | − reg = <0xaf54000 0x104>, | |
| 393 | − <0xaf54200 0xc0>, | |
| 394 | − <0xaf55000 0x770>, | |
| 395 | − <0xaf56000 0x9c>, | |
| 396 | − <0xaf57000 0x9c>; | |
| 392 | + reg = <0x0af54000 0x200>, | |
| 393 | + <0x0af54200 0x200>, | |
| 394 | + <0x0af55000 0xc00>, | |
| 395 | + <0x0af56000 0x400>, | |
| 396 | + <0x0af57000 0x400>, | |
| 397 | + <0x0af58000 0x400>, | |
| 398 | + <0x0af59000 0x400>, | |
| 399 | + <0x0af5a000 0x600>, | |
| 400 | + <0x0af5b000 0x600>; | |
| 397 | 401 | |
| 398 | 402 | interrupts-extended = <&mdss 12>; |
| 399 | 403 | |
| @@ -16,10 +16,15 @@ allOf: · 15 unmodified lines | ||
| 16 | 16 | |
| 17 | 17 | properties: |
| 18 | 18 | compatible: |
| 19 | − enum: | |
| 20 | − - amlogic,meson6-i2c # Meson6, Meson8 and compatible SoCs | |
| 21 | − - amlogic,meson-gxbb-i2c # GXBB and compatible SoCs | |
| 22 | − - amlogic,meson-axg-i2c # AXG and compatible SoCs | |
| 19 | + oneOf: | |
| 20 | + - items: | |
| 21 | + - enum: | |
| 22 | + - amlogic,t7-i2c | |
| 23 | + - const: amlogic,meson-axg-i2c | |
| 24 | + - enum: | |
| 25 | + - amlogic,meson6-i2c # Meson6, Meson8 and compatible SoCs | |
| 26 | + - amlogic,meson-gxbb-i2c # GXBB and compatible SoCs | |
| 27 | + - amlogic,meson-axg-i2c # AXG and compatible SoCs | |
| 23 | 28 | |
| 24 | 29 | reg: |
| 25 | 30 | maxItems: 1 |
| @@ -22,7 +22,9 @@ properties: · 21 unmodified lines | ||
| 22 | 22 | compatible: |
| 23 | 23 | oneOf: |
| 24 | 24 | - items: |
| 25 | − - const: apple,t6020-i2c | |
| 25 | + - enum: | |
| 26 | + - apple,t6020-i2c | |
| 27 | + - apple,t8122-i2c | |
| 26 | 28 | - const: apple,t8103-i2c |
| 27 | 29 | - items: |
| 28 | 30 | - enum: |
| @@ -24,6 +24,7 @@ properties: · 23 unmodified lines | ||
| 24 | 24 | compatible: |
| 25 | 25 | items: |
| 26 | 26 | - enum: |
| 27 | + - qcom,eliza-ipcc | |
| 27 | 28 | - qcom,glymur-ipcc |
| 28 | 29 | - qcom,kaanapali-ipcc |
| 29 | 30 | - qcom,milos-ipcc |
| @@ -73,6 +73,15 @@ properties: · 72 unmodified lines | ||
| 73 | 73 | HSP CSR is to control and get status of different high-speed peripherals |
| 74 | 74 | (such as Ethernet, USB, SATA, etc.) via register, which can tune |
| 75 | 75 | board-level's parameters of PHY, etc. |
| 76 | + | |
| 77 | + Additional background information about the High-Speed Subsystem | |
| 78 | + and the HSP CSR block is available in Chapter 10 ("High-Speed Interface") | |
| 79 | + of the EIC7700X SoC Technical Reference Manual, Part 4 | |
| 80 | + (EIC7700X_SoC_Technical_Reference_Manual_Part4.pdf). The manual is | |
| 81 | + publicly available at | |
| 82 | + https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual/releases | |
| 83 | + | |
| 84 | + This reference is provided for background information only. | |
| 76 | 85 | $ref: /schemas/types.yaml#/definitions/phandle-array |
| 77 | 86 | items: |
| 78 | 87 | - items: |
| @@ -82,6 +91,8 @@ properties: · 3 unmodified lines | ||
| 82 | 91 | - description: Offset of AXI clock controller Low-Power request |
| 83 | 92 | register |
| 84 | 93 | - description: Offset of register controlling TX/RX clock delay |
| 94 | + - description: Optional offset of register controlling TXD delay | |
| 95 | + - description: Optional offset of register controlling RXD delay | |
| 85 | 96 | |
| 86 | 97 | required: |
| 87 | 98 | - compatible |
| @@ -116,7 +127,7 @@ examples: · 28 unmodified lines | ||
| 116 | 127 | reset-names = "stmmaceth"; |
| 117 | 128 | rx-internal-delay-ps = <200>; |
| 118 | 129 | tx-internal-delay-ps = <200>; |
| 119 | − eswin,hsp-sp-csr = <&hsp_sp_csr 0x100 0x108 0x118>; | |
| 130 | + eswin,hsp-sp-csr = <&hsp_sp_csr 0x100 0x108 0x118 0x114 0x11c>; | |
| 120 | 131 | snps,axi-config = <&stmmac_axi_setup>; |
| 121 | 132 | snps,aal; |
| 122 | 133 | snps,fixed-burst; |
| @@ -18,7 +18,9 @@ properties: · 17 unmodified lines | ||
| 18 | 18 | description: Phandles of rt5650 and rt5514 codecs |
| 19 | 19 | items: |
| 20 | 20 | - description: phandle of rt5650 codec |
| 21 | + maxItems: 1 | |
| 21 | 22 | - description: phandle of rt5514 codec |
| 23 | + maxItems: 1 | |
| 22 | 24 | |
| 23 | 25 | mediatek,platform: |
| 24 | 26 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -20,6 +20,9 @@ properties: · 19 unmodified lines | ||
| 20 | 20 | - fsl,ls1021a-qspi |
| 21 | 21 | - fsl,ls2080a-qspi |
| 22 | 22 | - spacemit,k1-qspi |
| 23 | + - items: | |
| 24 | + - const: spacemit,k3-qspi | |
| 25 | + - const: spacemit,k1-qspi | |
| 23 | 26 | - items: |
| 24 | 27 | - enum: |
| 25 | 28 | - fsl,ls1043a-qspi |
| @@ -25,6 +25,7 @@ properties: · 24 unmodified lines | ||
| 25 | 25 | - items: |
| 26 | 26 | - enum: |
| 27 | 27 | - qcom,ipq5018-snand |
| 28 | + - qcom,ipq5210-snand | |
| 28 | 29 | - qcom,ipq5332-snand |
| 29 | 30 | - qcom,ipq5424-snand |
| 30 | 31 | - const: qcom,ipq9574-snand |
| @@ -41,12 +41,13 @@ properties: · 40 unmodified lines | ||
| 41 | 41 | - const: usb_en |
| 42 | 42 | |
| 43 | 43 | resets: |
| 44 | − maxItems: 2 | |
| 44 | + maxItems: 3 | |
| 45 | 45 | |
| 46 | 46 | reset-names: |
| 47 | 47 | items: |
| 48 | 48 | - const: vaux |
| 49 | 49 | - const: usb_rst |
| 50 | + - const: usb_phy | |
| 50 | 51 | |
| 51 | 52 | eswin,hsp-sp-csr: |
| 52 | 53 | description: |
| @@ -85,8 +86,8 @@ examples: · 32 unmodified lines | ||
| 85 | 86 | interrupt-parent = <&plic>; |
| 86 | 87 | interrupts = <85>; |
| 87 | 88 | interrupt-names = "peripheral"; |
| 88 | − resets = <&reset 84>, <&hspcrg 2>; | |
| 89 | − reset-names = "vaux", "usb_rst"; | |
| 89 | + resets = <&reset 84>, <&hspcrg 2>, <&hspcrg 4>; | |
| 90 | + reset-names = "vaux", "usb_rst", "usb_phy"; | |
| 90 | 91 | dr_mode = "peripheral"; |
| 91 | 92 | maximum-speed = "high-speed"; |
| 92 | 93 | phy_type = "utmi"; |
| @@ -81,9 +81,7 @@ properties: · 80 unmodified lines | ||
| 81 | 81 | const: usb2-phy |
| 82 | 82 | |
| 83 | 83 | usb-phy: |
| 84 | − $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 85 | − description: Phandle for the PHY device. | |
| 86 | − deprecated: true | |
| 84 | + maxItems: 1 | |
| 87 | 85 | |
| 88 | 86 | ctrl-module: |
| 89 | 87 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -96,6 +94,9 @@ required: · 6 unmodified lines | ||
| 96 | 94 | - interrupts |
| 97 | 95 | - interrupt-names |
| 98 | 96 | |
| 97 | +allOf: | |
| 98 | + - $ref: usb-hcd.yaml# | |
| 99 | + | |
| 99 | 100 | unevaluatedProperties: false |
| 100 | 101 | |
| 101 | 102 | examples: |
| @@ -57,7 +57,7 @@ Mount options unique to the isofs filesystem. · 56 unmodified lines | ||
| 57 | 57 | Recommended documents about ISO 9660 standard are located at: |
| 58 | 58 | |
| 59 | 59 | - http://www.y-adagio.com/ |
| 60 | −- ftp://ftp.ecma.ch/ecma-st/Ecma-119.pdf | |
| 60 | +- https://ecma-international.org/wp-content/uploads/ECMA-119_2nd_edition_december_1987.pdf | |
| 61 | 61 | |
| 62 | 62 | Quoting from the PDF "This 2nd Edition of Standard ECMA-119 is technically |
| 63 | 63 | identical with ISO 9660.", so it is a valid and gratis substitute of the |
| @@ -22,5 +22,5 @@ The following sensors are supported · 21 unmodified lines | ||
| 22 | 22 | sysfs-Interface |
| 23 | 23 | --------------- |
| 24 | 24 | |
| 25 | −temp0_input | |
| 25 | +temp1_input | |
| 26 | 26 | - Temperature of external NTC (milli-degree C) |
| @@ -135,4 +135,4 @@ References · 134 unmodified lines | ||
| 135 | 135 | |
| 136 | 136 | 4. **Lenovo IdeaPad Laptop Driver:** Reference for DMI-based hardware |
| 137 | 137 | feature gating in Lenovo laptops. |
| 138 | − https://github.com/torvalds/linux/blob/master/drivers/platform/x86/ideapad-laptop.c | |
| 138 | + https://github.com/torvalds/linux/blob/master/drivers/platform/x86/lenovo/ideapad-laptop.c | |
| @@ -69,6 +69,15 @@ properties: · 68 unmodified lines | ||
| 69 | 69 | header: |
| 70 | 70 | description: For C-compatible languages, header which already defines this value. |
| 71 | 71 | type: string |
| 72 | + scope: | |
| 73 | + description: | | |
| 74 | + Visibility of this definition. "uapi" (default) renders into | |
| 75 | + the uAPI header, "kernel" renders into the kernel-side | |
| 76 | + generated header, "user" renders into the user-side | |
| 77 | + generated header. When combined with `header:`, the | |
| 78 | + definition is not rendered, and the named header is | |
| 79 | + included only by code matching the scope. | |
| 80 | + enum: [ uapi, kernel, user ] | |
| 72 | 81 | type: |
| 73 | 82 | enum: [ const, enum, flags ] |
| 74 | 83 | doc: |
| @@ -83,6 +83,15 @@ properties: · 82 unmodified lines | ||
| 83 | 83 | header: |
| 84 | 84 | description: For C-compatible languages, header which already defines this value. |
| 85 | 85 | type: string |
| 86 | + scope: | |
| 87 | + description: | | |
| 88 | + Visibility of this definition. "uapi" (default) renders into | |
| 89 | + the uAPI header, "kernel" renders into the kernel-side | |
| 90 | + generated header, "user" renders into the user-side | |
| 91 | + generated header. When combined with `header:`, the | |
| 92 | + definition is not rendered, and the named header is | |
| 93 | + included only by code matching the scope. | |
| 94 | + enum: [ uapi, kernel, user ] | |
| 86 | 95 | type: |
| 87 | 96 | enum: [ const, enum, flags, struct ] # Trim |
| 88 | 97 | doc: |
| @@ -55,6 +55,15 @@ properties: · 54 unmodified lines | ||
| 55 | 55 | header: |
| 56 | 56 | description: For C-compatible languages, header which already defines this value. |
| 57 | 57 | type: string |
| 58 | + scope: | |
| 59 | + description: | | |
| 60 | + Visibility of this definition. "uapi" (default) renders into | |
| 61 | + the uAPI header, "kernel" renders into the kernel-side | |
| 62 | + generated header, "user" renders into the user-side | |
| 63 | + generated header. When combined with `header:`, the | |
| 64 | + definition is not rendered, and the named header is | |
| 65 | + included only by code matching the scope. | |
| 66 | + enum: [ uapi, kernel, user ] | |
| 58 | 67 | type: |
| 59 | 68 | enum: [ const, enum, flags ] |
| 60 | 69 | doc: |
| @@ -87,6 +87,15 @@ properties: · 86 unmodified lines | ||
| 87 | 87 | header: |
| 88 | 88 | description: For C-compatible languages, header which already defines this value. |
| 89 | 89 | type: string |
| 90 | + scope: | |
| 91 | + description: | | |
| 92 | + Visibility of this definition. "uapi" (default) renders into | |
| 93 | + the uAPI header, "kernel" renders into the kernel-side | |
| 94 | + generated header, "user" renders into the user-side | |
| 95 | + generated header. When combined with `header:`, the | |
| 96 | + definition is not rendered, and the named header is | |
| 97 | + included only by code matching the scope. | |
| 98 | + enum: [ uapi, kernel, user ] | |
| 90 | 99 | type: |
| 91 | 100 | enum: [ const, enum, flags, struct ] # Trim |
| 92 | 101 | doc: |
| @@ -12,6 +12,12 @@ protocol: genetlink · 11 unmodified lines | ||
| 12 | 12 | doc: Netlink protocol to request a transport layer security handshake. |
| 13 | 13 | |
| 14 | 14 | definitions: |
| 15 | + - | |
| 16 | + type: const | |
| 17 | + name: max-errno | |
| 18 | + value: 4095 | |
| 19 | + header: linux/err.h | |
| 20 | + scope: kernel | |
| 15 | 21 | - |
| 16 | 22 | type: enum |
| 17 | 23 | name: handler-class |
| @@ -80,6 +86,8 @@ attribute-sets: · 62 unmodified lines | ||
| 80 | 86 | - |
| 81 | 87 | name: status |
| 82 | 88 | type: u32 |
| 89 | + checks: | |
| 90 | + max: max-errno | |
| 83 | 91 | - |
| 84 | 92 | name: sockfd |
| 85 | 93 | type: s32 |
| @@ -33,6 +33,11 @@ doc: | · 32 unmodified lines | ||
| 33 | 33 | @cap-get operation. |
| 34 | 34 | |
| 35 | 35 | definitions: |
| 36 | + - | |
| 37 | + type: const | |
| 38 | + name: max-handle-id | |
| 39 | + value: 0x3fffffe | |
| 40 | + scope: kernel | |
| 36 | 41 | - |
| 37 | 42 | type: enum |
| 38 | 43 | name: scope |
| @@ -140,6 +145,8 @@ attribute-sets: · 101 unmodified lines | ||
| 140 | 145 | - |
| 141 | 146 | name: id |
| 142 | 147 | type: u32 |
| 148 | + checks: | |
| 149 | + max: max-handle-id | |
| 143 | 150 | doc: | |
| 144 | 151 | Numeric identifier of a shaper. The id semantic depends on |
| 145 | 152 | the scope. For @queue scope it's the queue id and for @node |
| @@ -188,6 +188,7 @@ operations: · 187 unmodified lines | ||
| 188 | 188 | name: dev-set |
| 189 | 189 | doc: Set the configuration of a PSP device. |
| 190 | 190 | attribute-set: dev |
| 191 | + flags: [admin-perm] | |
| 191 | 192 | do: |
| 192 | 193 | request: |
| 193 | 194 | attributes: |
| @@ -207,6 +208,7 @@ operations: · 13 unmodified lines | ||
| 207 | 208 | name: key-rotate |
| 208 | 209 | doc: Rotate the device key. |
| 209 | 210 | attribute-set: dev |
| 211 | + flags: [admin-perm] | |
| 210 | 212 | do: |
| 211 | 213 | request: |
| 212 | 214 | attributes: |
| @@ -0,0 +1,249 @@ | ||
| 1 | +.. SPDX-License-Identifier: GPL-2.0 | |
| 2 | + | |
| 3 | +============================================================================= | |
| 4 | +Linux and the 3Com EtherLink III Series Ethercards (driver v1.18c and higher) | |
| 5 | +============================================================================= | |
| 6 | + | |
| 7 | +This file contains the instructions and caveats for v1.18c and higher versions | |
| 8 | +of the 3c509 driver. You should not use the driver without reading this file. | |
| 9 | + | |
| 10 | +release 1.0 | |
| 11 | + | |
| 12 | +28 February 2002 | |
| 13 | + | |
| 14 | +Current maintainer (corrections to): | |
| 15 | + Maciej W. Rozycki <macro@orcam.me.uk> | |
| 16 | + | |
| 17 | +Introduction | |
| 18 | +============ | |
| 19 | + | |
| 20 | +The following are notes and information on using the 3Com EtherLink III series | |
| 21 | +ethercards in Linux. These cards are commonly known by the most widely-used | |
| 22 | +card's 3Com model number, 3c509. They are all 10mb/s ISA-bus cards and shouldn't | |
| 23 | +be (but sometimes are) confused with the similarly-numbered PCI-bus "3c905" | |
| 24 | +(aka "Vortex" or "Boomerang") series. Kernel support for the 3c509 family is | |
| 25 | +provided by the module 3c509.c, which has code to support all of the following | |
| 26 | +models: | |
| 27 | + | |
| 28 | + - 3c509 (original ISA card) | |
| 29 | + - 3c509B (later revision of the ISA card; supports full-duplex) | |
| 30 | + - 3c589 (PCMCIA) | |
| 31 | + - 3c589B (later revision of the 3c589; supports full-duplex) | |
| 32 | + - 3c579 (EISA) | |
| 33 | + | |
| 34 | +Large portions of this documentation were heavily borrowed from the guide | |
| 35 | +written the original author of the 3c509 driver, Donald Becker. The master | |
| 36 | +copy of that document, which contains notes on older versions of the driver, | |
| 37 | +currently resides on Scyld web server: http://www.scyld.com/. | |
| 38 | + | |
| 39 | + | |
| 40 | +Special Driver Features | |
| 41 | +======================= | |
| 42 | + | |
| 43 | +Overriding card settings | |
| 44 | + | |
| 45 | +The driver allows boot- or load-time overriding of the card's detected IOADDR, | |
| 46 | +IRQ, and transceiver settings, although this capability shouldn't generally be | |
| 47 | +needed except to enable full-duplex mode (see below). An example of the syntax | |
| 48 | +for LILO parameters for doing this:: | |
| 49 | + | |
| 50 | + ether=10,0x310,3,0x3c509,eth0 | |
| 51 | + | |
| 52 | +This configures the first found 3c509 card for IRQ 10, base I/O 0x310, and | |
| 53 | +transceiver type 3 (10base2). The flag "0x3c509" must be set to avoid conflicts | |
| 54 | +with other card types when overriding the I/O address. When the driver is | |
| 55 | +loaded as a module, only the IRQ may be overridden. For example, | |
| 56 | +setting two cards to IRQ10 and IRQ11 is done by using the irq module | |
| 57 | +option:: | |
| 58 | + | |
| 59 | + options 3c509 irq=10,11 | |
| 60 | + | |
| 61 | + | |
| 62 | +Full-duplex mode | |
| 63 | +================ | |
| 64 | + | |
| 65 | +The v1.18c driver added support for the 3c509B's full-duplex capabilities. | |
| 66 | +In order to enable and successfully use full-duplex mode, three conditions | |
| 67 | +must be met: | |
| 68 | + | |
| 69 | +(a) You must have a Etherlink III card model whose hardware supports full- | |
| 70 | +duplex operations. Currently, the only members of the 3c509 family that are | |
| 71 | +positively known to support full-duplex are the 3c509B (ISA bus) and 3c589B | |
| 72 | +(PCMCIA) cards. Cards without the "B" model designation do *not* support | |
| 73 | +full-duplex mode; these include the original 3c509 (no "B"), the original | |
| 74 | +3c589, the 3c529 (MCA bus), and the 3c579 (EISA bus). | |
| 75 | + | |
| 76 | +(b) You must be using your card's 10baseT transceiver (i.e., the RJ-45 | |
| 77 | +connector), not its AUI (thick-net) or 10base2 (thin-net/coax) interfaces. | |
| 78 | +AUI and 10base2 network cabling is physically incapable of full-duplex | |
| 79 | +operation. | |
| 80 | + | |
| 81 | +(c) Most importantly, your 3c509B must be connected to a link partner that is | |
| 82 | +itself full-duplex capable. This is almost certainly one of two things: a full- | |
| 83 | +duplex-capable Ethernet switch (*not* a hub), or a full-duplex-capable NIC on | |
| 84 | +another system that's connected directly to the 3c509B via a crossover cable. | |
| 85 | + | |
| 86 | +Full-duplex mode can be enabled using 'ethtool'. | |
| 87 | + | |
| 88 | +.. warning:: | |
| 89 | + | |
| 90 | + Extremely important caution concerning full-duplex mode | |
| 91 | + | |
| 92 | + Understand that the 3c509B's hardware's full-duplex support is much more | |
| 93 | + limited than that provide by more modern network interface cards. Although | |
| 94 | + at the physical layer of the network it fully supports full-duplex operation, | |
| 95 | + the card was designed before the current Ethernet auto-negotiation (N-way) | |
| 96 | + spec was written. This means that the 3c509B family ***cannot and will not | |
| 97 | + auto-negotiate a full-duplex connection with its link partner under any | |
| 98 | + circumstances, no matter how it is initialized***. If the full-duplex mode | |
| 99 | + of the 3c509B is enabled, its link partner will very likely need to be | |
| 100 | + independently _forced_ into full-duplex mode as well; otherwise various nasty | |
| 101 | + failures will occur - at the very least, you'll see massive numbers of packet | |
| 102 | + collisions. This is one of very rare circumstances where disabling auto- | |
| 103 | + negotiation and forcing the duplex mode of a network interface card or switch | |
| 104 | + would ever be necessary or desirable. | |
| 105 | + | |
| 106 | + | |
| 107 | +Available Transceiver Types | |
| 108 | +=========================== | |
| 109 | + | |
| 110 | +For versions of the driver v1.18c and above, the available transceiver types are: | |
| 111 | + | |
| 112 | +== ========================================================================= | |
| 113 | +0 transceiver type from EEPROM config (normally 10baseT); force half-duplex | |
| 114 | +1 AUI (thick-net / DB15 connector) | |
| 115 | +2 (undefined) | |
| 116 | +3 10base2 (thin-net == coax / BNC connector) | |
| 117 | +4 10baseT (RJ-45 connector); force half-duplex mode | |
| 118 | +8 transceiver type and duplex mode taken from card's EEPROM config settings | |
| 119 | +12 10baseT (RJ-45 connector); force full-duplex mode | |
| 120 | +== ========================================================================= | |
| 121 | + | |
| 122 | +Prior to driver version 1.18c, only transceiver codes 0-4 were supported. Note | |
| 123 | +that the new transceiver codes 8 and 12 are the *only* ones that will enable | |
| 124 | +full-duplex mode, no matter what the card's detected EEPROM settings might be. | |
| 125 | +This insured that merely upgrading the driver from an earlier version would | |
| 126 | +never automatically enable full-duplex mode in an existing installation; | |
| 127 | +it must always be explicitly enabled via one of these code in order to be | |
| 128 | +activated. | |
| 129 | + | |
| 130 | +The transceiver type can be changed using 'ethtool'. | |
| 131 | + | |
| 132 | + | |
| 133 | +Interpretation of error messages and common problems | |
| 134 | +---------------------------------------------------- | |
| 135 | + | |
| 136 | +Error Messages | |
| 137 | +^^^^^^^^^^^^^^ | |
| 138 | + | |
| 139 | +eth0: Infinite loop in interrupt, status 2011. | |
| 140 | +These are "mostly harmless" message indicating that the driver had too much | |
| 141 | +work during that interrupt cycle. With a status of 0x2011 you are receiving | |
| 142 | +packets faster than they can be removed from the card. This should be rare | |
| 143 | +or impossible in normal operation. Possible causes of this error report are: | |
| 144 | + | |
| 145 | + - a "green" mode enabled that slows the processor down when there is no | |
| 146 | + keyboard activity. | |
| 147 | + | |
| 148 | + - some other device or device driver hogging the bus or disabling interrupts. | |
| 149 | + Check /proc/interrupts for excessive interrupt counts. The timer tick | |
| 150 | + interrupt should always be incrementing faster than the others. | |
| 151 | + | |
| 152 | +No received packets | |
| 153 | +^^^^^^^^^^^^^^^^^^^ | |
| 154 | + | |
| 155 | +If a 3c509, 3c562 or 3c589 can successfully transmit packets, but never | |
| 156 | +receives packets (as reported by /proc/net/dev or 'ifconfig') you likely | |
| 157 | +have an interrupt line problem. Check /proc/interrupts to verify that the | |
| 158 | +card is actually generating interrupts. If the interrupt count is not | |
| 159 | +increasing you likely have a physical conflict with two devices trying to | |
| 160 | +use the same ISA IRQ line. The common conflict is with a sound card on IRQ10 | |
| 161 | +or IRQ5, and the easiest solution is to move the 3c509 to a different | |
| 162 | +interrupt line. If the device is receiving packets but 'ping' doesn't work, | |
| 163 | +you have a routing problem. | |
| 164 | + | |
| 165 | +Tx Carrier Errors Reported in /proc/net/dev | |
| 166 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 167 | + | |
| 168 | + | |
| 169 | +If an EtherLink III appears to transmit packets, but the "Tx carrier errors" | |
| 170 | +field in /proc/net/dev increments as quickly as the Tx packet count, you | |
| 171 | +likely have an unterminated network or the incorrect media transceiver selected. | |
| 172 | + | |
| 173 | +3c509B card is not detected on machines with an ISA PnP BIOS. | |
| 174 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 175 | + | |
| 176 | +While the updated driver works with most PnP BIOS programs, it does not work | |
| 177 | +with all. This can be fixed by disabling PnP support using the 3Com-supplied | |
| 178 | +setup program. | |
| 179 | + | |
| 180 | +3c509 card is not detected on overclocked machines | |
| 181 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 182 | + | |
| 183 | +Increase the delay time in id_read_eeprom() from the current value, 500, | |
| 184 | +to an absurdly high value, such as 5000. | |
| 185 | + | |
| 186 | + | |
| 187 | +Decoding Status and Error Messages | |
| 188 | +---------------------------------- | |
| 189 | + | |
| 190 | + | |
| 191 | +The bits in the main status register are: | |
| 192 | + | |
| 193 | +===== ====================================== | |
| 194 | +value description | |
| 195 | +===== ====================================== | |
| 196 | +0x01 Interrupt latch | |
| 197 | +0x02 Tx overrun, or Rx underrun | |
| 198 | +0x04 Tx complete | |
| 199 | +0x08 Tx FIFO room available | |
| 200 | +0x10 A complete Rx packet has arrived | |
| 201 | +0x20 A Rx packet has started to arrive | |
| 202 | +0x40 The driver has requested an interrupt | |
| 203 | +0x80 Statistics counter nearly full | |
| 204 | +===== ====================================== | |
| 205 | + | |
| 206 | +The bits in the transmit (Tx) status word are: | |
| 207 | + | |
| 208 | +===== ============================================ | |
| 209 | +value description | |
| 210 | +===== ============================================ | |
| 211 | +0x02 Out-of-window collision. | |
| 212 | +0x04 Status stack overflow (normally impossible). | |
| 213 | +0x08 16 collisions. | |
| 214 | +0x10 Tx underrun (not enough PCI bus bandwidth). | |
| 215 | +0x20 Tx jabber. | |
| 216 | +0x40 Tx interrupt requested. | |
| 217 | +0x80 Status is valid (this should always be set). | |
| 218 | +===== ============================================ | |
| 219 | + | |
| 220 | + | |
| 221 | +When a transmit error occurs the driver produces a status message such as:: | |
| 222 | + | |
| 223 | + eth0: Transmit error, Tx status register 82 | |
| 224 | + | |
| 225 | +The two values typically seen here are: | |
| 226 | + | |
| 227 | +0x82 | |
| 228 | +^^^^ | |
| 229 | + | |
| 230 | +Out of window collision. This typically occurs when some other Ethernet | |
| 231 | +host is incorrectly set to full duplex on a half duplex network. | |
| 232 | + | |
| 233 | +0x88 | |
| 234 | +^^^^ | |
| 235 | + | |
| 236 | +16 collisions. This typically occurs when the network is exceptionally busy | |
| 237 | +or when another host doesn't correctly back off after a collision. If this | |
| 238 | +error is mixed with 0x82 errors it is the result of a host incorrectly set | |
| 239 | +to full duplex (see above). | |
| 240 | + | |
| 241 | +Both of these errors are the result of network problems that should be | |
| 242 | +corrected. They do not represent driver malfunction. | |
| 243 | + | |
| 244 | + | |
| 245 | +Revision history (this file) | |
| 246 | +============================ | |
| 247 | + | |
| 248 | +28Feb02 v1.0 DR New; major portions based on Becker original 3c509 docs | |
| 249 | + | |
| @@ -10,6 +10,7 @@ Contents: · 9 unmodified lines | ||
| 10 | 10 | .. toctree:: |
| 11 | 11 | :maxdepth: 2 |
| 12 | 12 | |
| 13 | + 3com/3c509 | |
| 13 | 14 | 3com/vortex |
| 14 | 15 | amazon/ena |
| 15 | 16 | altera/altera_tse |
| @@ -86,6 +86,7 @@ regressions and security problems. · 85 unmodified lines | ||
| 86 | 86 | debugging/index |
| 87 | 87 | handling-regressions |
| 88 | 88 | security-bugs |
| 89 | + threat-model | |
| 89 | 90 | cve |
| 90 | 91 | embargoed-hardware-issues |
| 91 | 92 | |
| @@ -66,6 +66,42 @@ In addition, the following information are highly desirable: · 65 unmodified lines | ||
| 66 | 66 | the issue appear. It is useful to share them, as they can be helpful to |
| 67 | 67 | keep end users protected during the time it takes them to apply the fix. |
| 68 | 68 | |
| 69 | +What qualifies as a security bug | |
| 70 | +-------------------------------- | |
| 71 | + | |
| 72 | +It is important that most bugs are handled publicly so as to involve the widest | |
| 73 | +possible audience and find the best solution. By nature, bugs that are handled | |
| 74 | +in closed discussions between a small set of participants are less likely to | |
| 75 | +produce the best possible fix (e.g., risk of missing valid use cases, limited | |
| 76 | +testing abilities). | |
| 77 | + | |
| 78 | +It turns out that the majority of the bugs reported via the security team are | |
| 79 | +just regular bugs that have been improperly qualified as security bugs due to | |
| 80 | +a lack of awareness of the Linux kernel's threat model, as described in | |
| 81 | +Documentation/process/threat-model.rst, and ought to have been sent through | |
| 82 | +the normal channels described in Documentation/admin-guide/reporting-issues.rst | |
| 83 | +instead. | |
| 84 | + | |
| 85 | +The security list exists for urgent bugs that grant an attacker a capability | |
| 86 | +they are not supposed to have on a correctly configured production system, and | |
| 87 | +can be easily exploited, representing an imminent threat to many users. Before | |
| 88 | +reporting, consider whether the issue actually crosses a trust boundary on such | |
| 89 | +a system. | |
| 90 | + | |
| 91 | +**If you resorted to AI assistance to identify a bug, you must treat it as | |
| 92 | +public**. While you may have valid reasons to believe it is not, the security | |
| 93 | +team's experience shows that bugs discovered this way systematically surface | |
| 94 | +simultaneously across multiple researchers, often on the same day. In this | |
| 95 | +case, do not publicly share a reproducer, as this could cause unintended harm; | |
| 96 | +just mention that one is available and maintainers might ask for it privately | |
| 97 | +if they need it. | |
| 98 | + | |
| 99 | +If you are unsure whether an issue qualifies, err on the side of reporting | |
| 100 | +privately: the security team would rather triage a borderline report than miss | |
| 101 | +a real vulnerability. Reporting ordinary bugs to the security list, however, | |
| 102 | +does not make them move faster and consumes triage capacity that other reports | |
| 103 | +need. | |
| 104 | + | |
| 69 | 105 | Identifying contacts |
| 70 | 106 | -------------------- |
| 71 | 107 | |
| @@ -74,7 +110,7 @@ affected subsystem's maintainers and Cc: the Linux kernel security team. Do · 2 unmodified lines | ||
| 74 | 110 | not send it to a public list at this stage, unless you have good reasons to |
| 75 | 111 | consider the issue as being public or trivial to discover (e.g. result of a |
| 76 | 112 | widely available automated vulnerability scanning tool that can be repeated by |
| 77 | −anyone). | |
| 113 | +anyone, or use of AI-based tools). | |
| 78 | 114 | |
| 79 | 115 | If you're sending a report for issues affecting multiple parts in the kernel, |
| 80 | 116 | even if they're fairly similar issues, please send individual messages (think |
| @@ -131,6 +167,64 @@ the Linux kernel security team only. Your message will be triaged, and you · 50 unmodified lines | ||
| 131 | 167 | will receive instructions about whom to contact, if needed. Your message may |
| 132 | 168 | equally be forwarded as-is to the relevant maintainers. |
| 133 | 169 | |
| 170 | +Responsible use of AI to find bugs | |
| 171 | +---------------------------------- | |
| 172 | + | |
| 173 | +A significant fraction of bug reports submitted to the security team are | |
| 174 | +actually the result of code reviews assisted by AI tools. While this can be an | |
| 175 | +efficient means to find bugs in rarely explored areas, it causes an overload on | |
| 176 | +maintainers, who are sometimes forced to ignore such reports due to their poor | |
| 177 | +quality or accuracy. As such, reporters must be particularly cautious about a | |
| 178 | +number of points which tend to make these reports needlessly difficult to | |
| 179 | +handle: | |
| 180 | + | |
| 181 | + * **Length**: AI-generated reports tend to be excessively long, containing | |
| 182 | + multiple sections and excessive detail. This makes it difficult to spot | |
| 183 | + important information such as affected files, versions, and impact. Please | |
| 184 | + ensure that a clear summary of the problem and all critical details are | |
| 185 | + presented first. Do not require triage engineers to scan multiple pages of | |
| 186 | + text. Configure your tools to produce concise, human-style reports. | |
| 187 | + | |
| 188 | + * **Formatting**: Most AI-generated reports are littered with Markdown tags. | |
| 189 | + These decorations complicate the search for important information and do | |
| 190 | + not survive the quoting processes involved in forwarding or replying. | |
| 191 | + Please **always convert your report to plain text** without any formatting | |
| 192 | + decorations before sending it. | |
| 193 | + | |
| 194 | + * **Impact Evaluation**: Many AI-generated reports lack an understanding | |
| 195 | + of the kernel's threat model (see Documentation/process/threat-model.rst) | |
| 196 | + and go to great lengths inventing theoretical consequences. This adds | |
| 197 | + noise and complicates triage. Please stick to verifiable facts (e.g., | |
| 198 | + "this bug permits any user to gain CAP_NET_ADMIN") without enumerating | |
| 199 | + speculative implications. Have your tool read this documentation as | |
| 200 | + part of the evaluation process. | |
| 201 | + | |
| 202 | + * **Reproducer**: AI-based tools are often capable of generating reproducers. | |
| 203 | + Please always ensure your tool provides one and **test it thoroughly**. If | |
| 204 | + the reproducer does not work, or if the tool cannot produce one, the | |
| 205 | + validity of the report should be seriously questioned. Note that since the | |
| 206 | + report will be posted to a public list, the reproducer should only be | |
| 207 | + shared upon maintainers' request. | |
| 208 | + | |
| 209 | + * **Propose a Fix**: Many AI tools are actually better at writing code than | |
| 210 | + evaluating it. Please ask your tool to propose a fix and **test it** before | |
| 211 | + reporting the problem. If the fix cannot be tested because it relies on | |
| 212 | + rare hardware or almost extinct network protocols, the issue is likely not | |
| 213 | + a security bug. In any case, if a fix is proposed, it must adhere to | |
| 214 | + Documentation/process/submitting-patches.rst and include a 'Fixes:' tag | |
| 215 | + designating the commit that introduced the bug. | |
| 216 | + | |
| 217 | +Failure to consider these points exposes your report to the risk of being | |
| 218 | +ignored. | |
| 219 | + | |
| 220 | +Use common sense when evaluating the report. If the affected file has not been | |
| 221 | +touched for more than one year and is maintained by a single individual, it is | |
| 222 | +likely that usage has declined and exposed users are virtually non-existent | |
| 223 | +(e.g., drivers for very old hardware, obsolete filesystems). In such cases, | |
| 224 | +there is no need to consume a maintainer's time with an unimportant report. If | |
| 225 | +the issue is clearly trivial and publicly discoverable, you should report it | |
| 226 | +directly to the public mailing lists. | |
| 227 | + | |
| 134 | 228 | Sending the report |
| 135 | 229 | ------------------ |
| 136 | 230 | |
| @@ -148,7 +242,15 @@ run additional tests. Reports where the reporter does not respond promptly · 11 unmodified lines | ||
| 148 | 242 | or cannot effectively discuss their findings may be abandoned if the |
| 149 | 243 | communication does not quickly improve. |
| 150 | 244 | |
| 151 | −The report must be sent to maintainers, with the security team in ``Cc:``. | |
| 245 | +The report must be sent to maintainers. If there are two or fewer | |
| 246 | +recipients in your message, you must also always Cc: the Linux kernel | |
| 247 | +security team who will ensure the message is delivered to the proper | |
| 248 | +people, and will be able to assist small maintainer teams with processes | |
| 249 | +they may not be familiar with. For larger teams, Cc: the Linux kernel | |
| 250 | +security team for your first few reports or when seeking specific help, | |
| 251 | +such as when resending a message which got no response within a week. | |
| 252 | +Once you have become comfortable with the process for a few reports, it is | |
| 253 | +no longer necessary to Cc: the security list when sending to large teams. | |
| 152 | 254 | The Linux kernel security team can be contacted by email at |
| 153 | 255 | <security@kernel.org>. This is a private list of security officers |
| 154 | 256 | who will help verify the bug report and assist developers working on a fix. |
| @@ -0,0 +1,235 @@ | ||
| 1 | +The Linux Kernel threat model | |
| 2 | +============================= | |
| 3 | + | |
| 4 | +There are a lot of assumptions regarding what the kernel does and does not | |
| 5 | +protect against. These assumptions tend to cause confusion for bug reports | |
| 6 | +(:doc:`security-related ones <security-bugs>` vs :doc:`non-security ones | |
| 7 | +<../admin-guide/reporting-issues>`), and can complicate security enforcement | |
| 8 | +when the responsibilities for some boundaries is not clear between the kernel, | |
| 9 | +distros, administrators and users. | |
| 10 | + | |
| 11 | +This document tries to clarify the responsibilities of the kernel in this | |
| 12 | +domain. | |
| 13 | + | |
| 14 | +The kernel's responsibilities | |
| 15 | +----------------------------- | |
| 16 | + | |
| 17 | +The kernel abstracts access to local hardware resources and to remote systems | |
| 18 | +in a way that allows multiple local users to get a fair share of the available | |
| 19 | +resources granted to them, and, when the underlying hardware permits, to assign | |
| 20 | +a level of confidentiality to their communications and to the data they are | |
| 21 | +processing or storing. | |
| 22 | + | |
| 23 | +The kernel assumes that the underlying hardware behaves according to its | |
| 24 | +specifications. This includes the integrity of the CPU's instruction set, the | |
| 25 | +transparency of the branch prediction unit and the cache units, the consistency | |
| 26 | +of the Memory Management Unit (MMU), the isolation of DMA-capable peripherals | |
| 27 | +(e.g., via IOMMU), state transitions in controllers, ranges of values read from | |
| 28 | +registers, the respect of documented hardware limitations, etc. | |
| 29 | + | |
| 30 | +When hardware fails to maintain its specified isolation (e.g., CPU bugs, | |
| 31 | +side-channels, hardware response to unexpected inputs), the kernel will usually | |
| 32 | +attempt to implement reasonable mitigations. These are best-effort measures | |
| 33 | +intended to reduce the attack surface or elevate the cost of an attack within | |
| 34 | +the limits of the hardware's facilities; they do not constitute a | |
| 35 | +kernel-provided safety guarantee. | |
| 36 | + | |
| 37 | +Users always perform their activities under the authority of an administrator | |
| 38 | +who is able to grant or deny various types of permissions that may affect how | |
| 39 | +users benefit from available resources, or the level of confidentiality of | |
| 40 | +their activities. Administrators may also delegate all or part of their own | |
| 41 | +permissions to some users, particularly via capabilities but not only. All this | |
| 42 | +is performed via configuration (sysctl, file-system permissions etc). | |
| 43 | + | |
| 44 | +The Linux Kernel applies a certain collection of default settings that match | |
| 45 | +its threat model. Distros have their own threat model and will come with their | |
| 46 | +own configuration presets, that the administrator may have to adjust to better | |
| 47 | +suit their expectations (relax or restrict). | |
| 48 | + | |
| 49 | +By default, the Linux Kernel guarantees the following protections when running | |
| 50 | +on common processors featuring privilege levels and memory management units: | |
| 51 | + | |
| 52 | +* **User-based isolation**: an unprivileged user may restrict access to their | |
| 53 | + own data from other unprivileged users running on the same system. This | |
| 54 | + includes: | |
| 55 | + | |
| 56 | + * stored data, via file system permissions | |
| 57 | + * in-memory data (pages are not accessible by default to other users) | |
| 58 | + * process activity (ptrace is not permitted to other users) | |
| 59 | + * inter-process communication (other users may not observe data exchanged via | |
| 60 | + UNIX domain sockets or other IPC mechanisms). | |
| 61 | + * network communications within the same or with other systems | |
| 62 | + | |
| 63 | +* **Capability-based protection**: | |
| 64 | + | |
| 65 | + * users not having elevated capabilities (including but not limited to | |
| 66 | + CAP_SYS_ADMIN) may not alter the | |
| 67 | + kernel's configuration, memory nor state, change other users' view of the | |
| 68 | + file system layout, grant any user capabilities they do not have, nor | |
| 69 | + affect the system's availability (shutdown, reboot, panic, hang, or making | |
| 70 | + the system unresponsive via unbounded resource exhaustion). | |
| 71 | + * users not having the ``CAP_NET_ADMIN`` capability may not alter the network | |
| 72 | + configuration, intercept nor spoof network communications from other users | |
| 73 | + nor systems. | |
| 74 | + * users not having ``CAP_SYS_PTRACE`` may not observe other users' processes | |
| 75 | + activities. | |
| 76 | + | |
| 77 | +When ``CONFIG_USER_NS`` is set, the kernel also permits unprivileged users to | |
| 78 | +create their own user namespace in which they have all capabilities, but with a | |
| 79 | +number of restrictions (they may not perform actions that have impacts on the | |
| 80 | +initial user namespace, such as changing time, loading modules or mounting | |
| 81 | +block devices). Please refer to ``user_namespaces(7)`` for more details, the | |
| 82 | +possibilities of user namespaces are not covered in this document. | |
| 83 | + | |
| 84 | +The kernel also offers a lot of troubleshooting and debugging facilities, which | |
| 85 | +can constitute attack vectors when placed in wrong hands. While some of them | |
| 86 | +are designed to be accessible to regular local users with a low risk (e.g. | |
| 87 | +kernel logs via ``/proc/kmsg``), some would expose enough information to | |
| 88 | +represent a risk in most places and the decision to expose them is under the | |
| 89 | +administrator's responsibility (perf events, traces), and others are not | |
| 90 | +designed to be accessed by non-privileged users (e.g. debugfs). Access to these | |
| 91 | +facilities by a user who has been explicitly granted permission by an | |
| 92 | +administrator does not constitute a security breach. | |
| 93 | + | |
| 94 | +Bugs that permit to violate the principles above constitute security breaches. | |
| 95 | +However, bugs that permit one violation only once another one was already | |
| 96 | +achieved are only weaknesses. The kernel applies a number of self-protection | |
| 97 | +measures whose purpose is to avoid crossing a security boundary when certain | |
| 98 | +classes of bugs are found, but a failure of these extra protections do not | |
| 99 | +constitute a vulnerability alone. | |
| 100 | + | |
| 101 | +What does not constitute a security bug | |
| 102 | +--------------------------------------- | |
| 103 | + | |
| 104 | +In the Linux kernel's threat model, the following classes of problems are | |
| 105 | +**NOT** considered as Linux Kernel security bugs. However, when it is believed | |
| 106 | +that the kernel could do better, they should be reported, so that they can be | |
| 107 | +reviewed and fixed where reasonably possible, but they will be handled as any | |
| 108 | +regular bug: | |
| 109 | + | |
| 110 | +* **Configuration**: | |
| 111 | + | |
| 112 | + * outdated kernels and particularly end-of-life branches are out of the scope | |
| 113 | + of the kernel's threat model: administrators are responsible for keeping | |
| 114 | + their system up to date. For a bug to qualify as a security bug, it must be | |
| 115 | + demonstrated that it affects actively maintained versions. | |
| 116 | + | |
| 117 | + * build-level: changes to the kernel configuration that are explicitly | |
| 118 | + documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or | |
| 119 | + targeted at developers only. | |
| 120 | + | |
| 121 | + * OS-level: changes to command line parameters, sysctls, filesystem | |
| 122 | + permissions, user capabilities, exposure of privileged interfaces, that | |
| 123 | + explicitly increase exposure by either offering non-default access to | |
| 124 | + unprivileged users, or reduce the kernel's ability to enforce some | |
| 125 | + protections or mitigations. Example: write access to procfs or debugfs. | |
| 126 | + | |
| 127 | + * issues triggered only when using features intended for development or | |
| 128 | + debugging (e.g., LOCKDEP, KASAN, FAULT_INJECTION): these features are known | |
| 129 | + to introduce overhead and potential instability and are not intended for | |
| 130 | + production use. | |
| 131 | + | |
| 132 | + * issues affecting drivers exposed under CONFIG_STAGING, as well as features | |
| 133 | + marked EXPERIMENTAL in the configuration. | |
| 134 | + | |
| 135 | + * loading of explicitly insecure/broken/staging modules, and generally any | |
| 136 | + using any subsystem marked as experimental or not intended for production | |
| 137 | + use. | |
| 138 | + | |
| 139 | + * running out-of-tree modules or unofficial kernel forks; these should be | |
| 140 | + reported to the relevant vendor. | |
| 141 | + | |
| 142 | +* **Excess of initial privileges**: | |
| 143 | + | |
| 144 | + * actions performed by a user already possessing the privileges required to | |
| 145 | + perform that action or modify that state (e.g. ``CAP_SYS_ADMIN``, | |
| 146 | + ``CAP_NET_ADMIN``, ``CAP_SYS_RAWIO``, ``CAP_SYS_MODULE`` with no further | |
| 147 | + boundary being crossed). | |
| 148 | + | |
| 149 | + * actions performed in user namespace that do not bypass the restrictions | |
| 150 | + imposed to the initial user (e.g. ptrace usage, signal delivery, resource | |
| 151 | + usage, access to FS/device/sysctl/memory, network binding, system/network | |
| 152 | + configuration etc). | |
| 153 | + | |
| 154 | + * anything performed by the root user in the initial namespace (e.g. kernel | |
| 155 | + oops when writing to a privileged device). | |
| 156 | + | |
| 157 | +* **Out of production use**: | |
| 158 | + | |
| 159 | + This covers theoretical/probabilistic attacks that rely on laboratory | |
| 160 | + conditions with zero system noise, or those requiring an unrealistic number | |
| 161 | + of attempts (e.g., billions of trials) that would be detected by standard | |
| 162 | + system monitoring long before success, such as: | |
| 163 | + | |
| 164 | + * prediction of random numbers that only works in a totally silent | |
| 165 | + environment (such as IP ID, TCP ports or sequence numbers that can only be | |
| 166 | + guessed in a lab). | |
| 167 | + | |
| 168 | + * activity observation and information leaks based on probabilistic | |
| 169 | + approaches that are prone to measurement noise and not realistically | |
| 170 | + reproducible on a production system. | |
| 171 | + | |
| 172 | + * issues that can only be triggered by heavy attacks (e.g. brute force) whose | |
| 173 | + impact on the system makes it unlikely or impossible to remain undetected | |
| 174 | + before they succeed (e.g. consuming all memory before succeeding). | |
| 175 | + | |
| 176 | + * problems seen only under development simulators, emulators, or combinations | |
| 177 | + that do not exist on real systems at the time of reporting (issues | |
| 178 | + involving tens of millions of threads, tens of thousands of CPUs, | |
| 179 | + unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds. | |
| 180 | + | |
| 181 | + * issues whose reproduction requires hardware modification or emulation, | |
| 182 | + including fake USB devices that pretend to be another one. | |
| 183 | + | |
| 184 | + * as well as issues that can be triggered at a cost that is orders of | |
| 185 | + magnitude higher than the expected benefits (e.g. fully functional keyboard | |
| 186 | + emulator only to retrieve 7 uninitialized bytes in a structure, or | |
| 187 | + brute-force method involving millions of connection attempts to guess a | |
| 188 | + port number). | |
| 189 | + | |
| 190 | +* **Hardening failures**: | |
| 191 | + | |
| 192 | + * ability to bypass some of the kernel's hardening measures with no | |
| 193 | + demonstrable exploit path (e.g. ASLR bypass, events timing or probing with | |
| 194 | + no demonstrable consequence). These are just weaknesses, not | |
| 195 | + vulnerabilities. | |
| 196 | + | |
| 197 | + * missing argument checks and failure to report certain errors with no | |
| 198 | + immediate consequence. | |
| 199 | + | |
| 200 | +* **Random information leaks**: | |
| 201 | + | |
| 202 | + This concerns information leaks of small data parts that happen to be there | |
| 203 | + and that cannot be chosen by the attacker, or face access restrictions: | |
| 204 | + | |
| 205 | + * structure padding reported by syscalls or other interfaces. | |
| 206 | + | |
| 207 | + * identifiers, partial data, non-terminated strings reported in error | |
| 208 | + messages. | |
| 209 | + | |
| 210 | + * Leaks of kernel memory addresses/pointers do not constitute an immediately | |
| 211 | + exploitable vector and are not security bugs, though they must be reported | |
| 212 | + and fixed. | |
| 213 | + | |
| 214 | +* **Crafted file system images**: | |
| 215 | + | |
| 216 | + * bugs triggered by mounting a corrupted or maliciously crafted file system | |
| 217 | + image are generally not security bugs, as the kernel assumes the underlying | |
| 218 | + storage media is under the administrator's control, unless the filesystem | |
| 219 | + driver is specifically documented as being hardened against untrusted media. | |
| 220 | + | |
| 221 | + * issues that are resolved, mitigated, or detected by running a filesystem | |
| 222 | + consistency check (fsck) on the image prior to mounting. | |
| 223 | + | |
| 224 | +* **Physical access**: | |
| 225 | + | |
| 226 | + Issues that require physical access to the machine, hardware modification, or | |
| 227 | + the use of specialized hardware (e.g., logic analyzers, DMA-attack tools over | |
| 228 | + PCI-E/Thunderbolt) are out of scope unless the system is explicitly | |
| 229 | + configured with technologies meant to defend against such attacks | |
| 230 | + (e.g. IOMMU). | |
| 231 | + | |
| 232 | +* **Functional and performance regressions**: | |
| 233 | + | |
| 234 | + Any issue that can be mitigated by setting proper permissions and limits | |
| 235 | + doesn't qualify as a security bug. | |
| @@ -40,7 +40,7 @@ There are two drivers in the kernel · 39 unmodified lines | ||
| 40 | 40 | |
| 41 | 41 | *For systems using SoundWire*: sound/soc/codecs/cs35l56.c and associated files |
| 42 | 42 | |
| 43 | −*For systems using HDA*: sound/pci/hda/cs35l56_hda.c | |
| 43 | +*For systems using HDA*: sound/hda/codecs/side-codecs/cs35l56_hda.c | |
| 44 | 44 | |
| 45 | 45 | Firmware |
| 46 | 46 | ======== |
| @@ -24,6 +24,97 @@ Quick access to CPU number, node ID · 23 unmodified lines | ||
| 24 | 24 | Allows to implement per CPU data efficiently. Documentation is in code and |
| 25 | 25 | selftests. :( |
| 26 | 26 | |
| 27 | +Optimized RSEQ V2 | |
| 28 | +----------------- | |
| 29 | + | |
| 30 | +On architectures which utilize the generic entry code and generic TIF bits | |
| 31 | +the kernel supports runtime optimizations for RSEQ, which also enable | |
| 32 | +enhanced features like scheduler time slice extensions. | |
| 33 | + | |
| 34 | +To enable them a task has to register the RSEQ region with at least the | |
| 35 | +length advertised by getauxval(AT_RSEQ_FEATURE_SIZE). | |
| 36 | + | |
| 37 | +If existing binaries register with RSEQ_ORIG_SIZE (32 bytes), the kernel | |
| 38 | +keeps the legacy low performance mode enabled to fulfil the expectations | |
| 39 | +of existing users regarding the original RSEQ implementation behaviour. | |
| 40 | + | |
| 41 | +The following table documents the ABI and behavioral guarantees of the | |
| 42 | +legacy and the optimized V2 mode. | |
| 43 | + | |
| 44 | +.. list-table:: RSEQ modes | |
| 45 | + :header-rows: 1 | |
| 46 | + | |
| 47 | + * - Nr | |
| 48 | + - What | |
| 49 | + | |
| 50 | + - Legacy | |
| 51 | + - Optimized V2 | |
| 52 | + | |
| 53 | + * - 1 | |
| 54 | + - The cpu_id_start, cpu_id, node_id and mm_cid fields (User mode read | |
| 55 | + only) | |
| 56 | + .. Legacy | |
| 57 | + - Updated by the kernel unconditionally after each context switch and | |
| 58 | + before signal delivery | |
| 59 | + .. Optimized V2 | |
| 60 | + - Updated by the kernel if and only if they change, i.e. if the task | |
| 61 | + is migrated or mm_cid changes | |
| 62 | + | |
| 63 | + * - 2 | |
| 64 | + - The rseq_cs critical section field | |
| 65 | + .. Legacy | |
| 66 | + - Evaluated and handled unconditionally after each context switch and | |
| 67 | + before signal delivery | |
| 68 | + .. Optimized V2 | |
| 69 | + - Evaluated and handled conditionally only when user space was | |
| 70 | + interrupted and was scheduled out or before delivering a signal in | |
| 71 | + the interrupted context. | |
| 72 | + | |
| 73 | + * - 3 | |
| 74 | + - Read only fields | |
| 75 | + .. Legacy | |
| 76 | + - No strict enforcement except in debug mode | |
| 77 | + .. Optimized V2 | |
| 78 | + - Strict enforcement | |
| 79 | + | |
| 80 | + * - 4 | |
| 81 | + - membarrier(...RSEQ) | |
| 82 | + .. Legacy | |
| 83 | + - All running threads of the process are interrupted and the ID fields | |
| 84 | + are rewritten and eventually active critical sections are aborted | |
| 85 | + before they return to user space. All threads which are scheduled | |
| 86 | + out whether voluntary or not are covered by #1/#2 above. | |
| 87 | + .. Optimized V2 | |
| 88 | + - All running threads of the process are interrupted and eventually | |
| 89 | + active critical sections are aborted before these threads return to | |
| 90 | + user space. The ID fields are only updated if changed as a | |
| 91 | + consequence of the interrupt. All threads which are scheduled out | |
| 92 | + whether voluntary or not are covered by #1/#2 above. | |
| 93 | + | |
| 94 | + * - 5 | |
| 95 | + - Time slice extensions | |
| 96 | + .. Legacy | |
| 97 | + - Not supported | |
| 98 | + .. Optimized V2 | |
| 99 | + - Supported | |
| 100 | + | |
| 101 | +The legacy mode is obviously less performant as it does unconditional | |
| 102 | +updates and critical section checks even if not strictly required by the | |
| 103 | +ABI contract. That can't be changed anymore as some users depend on that | |
| 104 | +observed behavior, which in turn enables them to violate the ABI and | |
| 105 | +overwrite the cpu_id_start field for their own purposes. This is obviously | |
| 106 | +discouraged as it renders RSEQ incompatible with the intended usage and | |
| 107 | +breaks the expectation of other libraries in the same application. | |
| 108 | + | |
| 109 | +The ABI compliant optimized v2 mode, which respects the read only fields, | |
| 110 | +does not require unconditional updates and therefore is way more | |
| 111 | +performant. The kernel validates the read only fields for compliance. If | |
| 112 | +user space modifies them, the process is killed. Compliant usage allows | |
| 113 | +multiple libraries in the same application to benefit from the RSEQ | |
| 114 | +functionality without disturbing each other. The ABI compliant optimized v2 | |
| 115 | +mode also enables extended RSEQ features like time slice extensions. | |
| 116 | + | |
| 117 | + | |
| 27 | 118 | Scheduler time slice extensions |
| 28 | 119 | ------------------------------- |
| 29 | 120 | |
| @@ -37,7 +128,8 @@ The prerequisites for this functionality are: · 7 unmodified lines | ||
| 37 | 128 | |
| 38 | 129 | * Enabled at boot time (default is enabled) |
| 39 | 130 | |
| 40 | − * A rseq userspace pointer has been registered for the thread | |
| 131 | + * A rseq userspace pointer has been registered for the thread in | |
| 132 | + optimized V2 mode | |
| 41 | 133 | |
| 42 | 134 | The thread has to enable the functionality via prctl(2):: |
| 43 | 135 | |
| @@ -656,8 +656,8 @@ References · 655 unmodified lines | ||
| 656 | 656 | See [white-paper]_, [api-spec]_, [amd-apm]_, [kvm-forum]_, and [snp-fw-abi]_ |
| 657 | 657 | for more info. |
| 658 | 658 | |
| 659 | −.. [white-paper] https://developer.amd.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf | |
| 660 | −.. [api-spec] https://support.amd.com/TechDocs/55766_SEV-KM_API_Specification.pdf | |
| 661 | −.. [amd-apm] https://support.amd.com/TechDocs/24593.pdf (section 15.34) | |
| 659 | +.. [white-paper] https://docs.amd.com/v/u/en-US/memory-encryption-white-paper | |
| 660 | +.. [api-spec] https://docs.amd.com/v/u/en-US/55766_PUB_3.24_SEV_API | |
| 661 | +.. [amd-apm] https://docs.amd.com/v/u/en-US/24593_3.44_APM_Vol2 (section 15.34) | |
| 662 | 662 | .. [kvm-forum] https://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf |
| 663 | −.. [snp-fw-abi] https://www.amd.com/system/files/TechDocs/56860.pdf | |
| 663 | +.. [snp-fw-abi] https://www.amd.com/content/dam/amd/en/documents/developer/56860.pdf | |
| @@ -68,6 +68,12 @@ Maintainers List · 67 unmodified lines | ||
| 68 | 68 | first. When adding to this list, please keep the entries in |
| 69 | 69 | alphabetical order. |
| 70 | 70 | |
| 71 | +3C509 NETWORK DRIVER | |
| 72 | +M: "Maciej W. Rozycki" <macro@orcam.me.uk> | |
| 73 | +L: netdev@vger.kernel.org | |
| 74 | +S: Maintained | |
| 75 | +F: drivers/net/ethernet/3com/3c509.c | |
| 76 | + | |
| 71 | 77 | 3C59X NETWORK DRIVER |
| 72 | 78 | M: Steffen Klassert <klassert@kernel.org> |
| 73 | 79 | L: netdev@vger.kernel.org |
| @@ -1144,8 +1150,9 @@ F: Documentation/arch/x86/amd-hfi.rst · 1070 unmodified lines | ||
| 1144 | 1150 | F: drivers/platform/x86/amd/hfi/ |
| 1145 | 1151 | |
| 1146 | 1152 | AMD IOMMU (AMD-VI) |
| 1147 | −M: Joerg Roedel <joro@8bytes.org> | |
| 1153 | +M: Joerg Roedel (AMD) <joro@8bytes.org> | |
| 1148 | 1154 | R: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> |
| 1155 | +R: Vasant Hegde <vasant.hegde@amd.com> | |
| 1149 | 1156 | L: iommu@lists.linux.dev |
| 1150 | 1157 | S: Maintained |
| 1151 | 1158 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git |
| @@ -2033,7 +2040,7 @@ F: Documentation/hwmon/aquacomputer_d5next.rst · 881 unmodified lines | ||
| 2033 | 2040 | F: drivers/hwmon/aquacomputer_d5next.c |
| 2034 | 2041 | |
| 2035 | 2042 | AQUANTIA ETHERNET DRIVER (atlantic) |
| 2036 | −M: Igor Russkikh <irusskikh@marvell.com> | |
| 2043 | +M: Sukhdeep Singh <sukhdeeps@marvell.com> | |
| 2037 | 2044 | L: netdev@vger.kernel.org |
| 2038 | 2045 | S: Maintained |
| 2039 | 2046 | W: https://www.marvell.com/ |
| @@ -2042,7 +2049,7 @@ F: Documentation/networking/device_drivers/ethernet/aquantia/atlantic.rst · 2 unmodified lines | ||
| 2042 | 2049 | F: drivers/net/ethernet/aquantia/atlantic/ |
| 2043 | 2050 | |
| 2044 | 2051 | AQUANTIA ETHERNET DRIVER PTP SUBSYSTEM |
| 2045 | −M: Egor Pomozov <epomozov@marvell.com> | |
| 2052 | +M: Sukhdeep Singh <sukhdeeps@marvell.com> | |
| 2046 | 2053 | L: netdev@vger.kernel.org |
| 2047 | 2054 | S: Maintained |
| 2048 | 2055 | W: http://www.aquantia.com |
| @@ -2076,7 +2083,7 @@ F: Documentation/devicetree/bindings/display/snps,arcpgu.txt · 27 unmodified lines | ||
| 2076 | 2083 | F: drivers/gpu/drm/tiny/arcpgu.c |
| 2077 | 2084 | |
| 2078 | 2085 | ARCNET NETWORK LAYER |
| 2079 | −M: Michael Grzeschik <m.grzeschik@pengutronix.de> | |
| 2086 | +M: Michael Grzeschik <mgr@kernel.org> | |
| 2080 | 2087 | L: netdev@vger.kernel.org |
| 2081 | 2088 | S: Maintained |
| 2082 | 2089 | F: drivers/net/arcnet/ |
| @@ -3379,7 +3386,9 @@ F: drivers/irqchip/irq-rda-intc.c · 1296 unmodified lines | ||
| 3379 | 3386 | F: drivers/tty/serial/rda-uart.c |
| 3380 | 3387 | |
| 3381 | 3388 | ARM/REALTEK ARCHITECTURE |
| 3382 | −M: Andreas Färber <afaerber@suse.de> | |
| 3389 | +M: James Tai <james.tai@realtek.com> | |
| 3390 | +M: Yu-Chun Lin <eleanor.lin@realtek.com> | |
| 3391 | +R: Andreas Färber <afaerber@suse.com> | |
| 3383 | 3392 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 3384 | 3393 | L: linux-realtek-soc@lists.infradead.org (moderated for non-subscribers) |
| 3385 | 3394 | S: Maintained |
| @@ -3387,6 +3396,7 @@ F: Documentation/devicetree/bindings/arm/realtek.yaml · 1 unmodified line | ||
| 3387 | 3396 | F: arch/arm/boot/dts/realtek/ |
| 3388 | 3397 | F: arch/arm/mach-realtek/ |
| 3389 | 3398 | F: arch/arm64/boot/dts/realtek/ |
| 3399 | +F: drivers/pinctrl/realtek/ | |
| 3390 | 3400 | |
| 3391 | 3401 | ARM/RISC-V/RENESAS ARCHITECTURE |
| 3392 | 3402 | M: Geert Uytterhoeven <geert+renesas@glider.be> |
| @@ -4199,8 +4209,8 @@ F: include/uapi/linux/sonet.h · 806 unmodified lines | ||
| 4199 | 4209 | F: net/atm/ |
| 4200 | 4210 | |
| 4201 | 4211 | ATMEL MACB ETHERNET DRIVER |
| 4202 | −M: Nicolas Ferre <nicolas.ferre@microchip.com> | |
| 4203 | −M: Claudiu Beznea <claudiu.beznea@tuxon.dev> | |
| 4212 | +M: Théo Lebrun <theo.lebrun@bootlin.com> | |
| 4213 | +R: Conor Dooley <conor.dooley@microchip.com> | |
| 4204 | 4214 | S: Maintained |
| 4205 | 4215 | F: drivers/net/ethernet/cadence/ |
| 4206 | 4216 | |
| @@ -4317,18 +4327,16 @@ F: Documentation/devicetree/bindings/leds/backlight/awinic,aw99706.yaml · 110 unmodified lines | ||
| 4317 | 4327 | F: drivers/video/backlight/aw99706.c |
| 4318 | 4328 | |
| 4319 | 4329 | AXENTIA ARM DEVICES |
| 4320 | −M: Peter Rosin <peda@axentia.se> | |
| 4321 | 4330 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 4322 | −S: Maintained | |
| 4331 | +S: Orphan | |
| 4323 | 4332 | F: arch/arm/boot/dts/microchip/at91-linea.dtsi |
| 4324 | 4333 | F: arch/arm/boot/dts/microchip/at91-natte.dtsi |
| 4325 | 4334 | F: arch/arm/boot/dts/microchip/at91-nattis-2-natte-2.dts |
| 4326 | 4335 | F: arch/arm/boot/dts/microchip/at91-tse850-3.dts |
| 4327 | 4336 | |
| 4328 | 4337 | AXENTIA ASOC DRIVERS |
| 4329 | −M: Peter Rosin <peda@axentia.se> | |
| 4330 | 4338 | L: linux-sound@vger.kernel.org |
| 4331 | −S: Maintained | |
| 4339 | +S: Orphan | |
| 4332 | 4340 | F: Documentation/devicetree/bindings/sound/axentia,* |
| 4333 | 4341 | F: sound/soc/atmel/tse850-pcm5142.c |
| 4334 | 4342 | |
| @@ -6383,6 +6391,7 @@ F: include/uapi/linux/comedi.h · 2048 unmodified lines | ||
| 6383 | 6391 | COMMON CLK FRAMEWORK |
| 6384 | 6392 | M: Michael Turquette <mturquette@baylibre.com> |
| 6385 | 6393 | M: Stephen Boyd <sboyd@kernel.org> |
| 6394 | +R: Brian Masney <bmasney@redhat.com> | |
| 6386 | 6395 | L: linux-clk@vger.kernel.org |
| 6387 | 6396 | S: Maintained |
| 6388 | 6397 | Q: http://patchwork.kernel.org/project/linux-clk/list/ |
| @@ -7102,6 +7111,12 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core/debugobjec · 713 unmodified lines | ||
| 7102 | 7111 | F: include/linux/debugobjects.h |
| 7103 | 7112 | F: lib/debugobjects.c |
| 7104 | 7113 | |
| 7114 | +DEC LANCE NETWORK DRIVER | |
| 7115 | +M: "Maciej W. Rozycki" <macro@orcam.me.uk> | |
| 7116 | +L: netdev@vger.kernel.org | |
| 7117 | +S: Maintained | |
| 7118 | +F: drivers/net/ethernet/amd/declance.c | |
| 7119 | + | |
| 7105 | 7120 | DECSTATION PLATFORM SUPPORT |
| 7106 | 7121 | M: "Maciej W. Rozycki" <macro@orcam.me.uk> |
| 7107 | 7122 | L: linux-mips@vger.kernel.org |
| @@ -7898,7 +7913,7 @@ F: drivers/gpu/drm/sun4i/sun8i* · 790 unmodified lines | ||
| 7898 | 7913 | |
| 7899 | 7914 | DRM DRIVER FOR APPLE TOUCH BARS |
| 7900 | 7915 | M: Aun-Ali Zaidi <admin@kodeit.net> |
| 7901 | −M: Aditya Garg <gargaditya08@live.com> | |
| 7916 | +M: Aditya Garg <gargaditya08@proton.me> | |
| 7902 | 7917 | L: dri-devel@lists.freedesktop.org |
| 7903 | 7918 | S: Maintained |
| 7904 | 7919 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git |
| @@ -8218,10 +8233,9 @@ F: include/uapi/drm/nouveau_drm.h · 313 unmodified lines | ||
| 8218 | 8233 | CORE DRIVER FOR NVIDIA GPUS [RUST] |
| 8219 | 8234 | M: Danilo Krummrich <dakr@kernel.org> |
| 8220 | 8235 | M: Alexandre Courbot <acourbot@nvidia.com> |
| 8221 | −L: nouveau@lists.freedesktop.org | |
| 8236 | +L: nova-gpu@lists.linux.dev | |
| 8222 | 8237 | S: Supported |
| 8223 | 8238 | W: https://rust-for-linux.com/nova-gpu-driver |
| 8224 | −Q: https://patchwork.freedesktop.org/project/nouveau/ | |
| 8225 | 8239 | B: https://gitlab.freedesktop.org/drm/nova/-/issues |
| 8226 | 8240 | C: irc://irc.oftc.net/nouveau |
| 8227 | 8241 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next |
| @@ -8230,10 +8244,9 @@ F: drivers/gpu/nova-core/ · 2 unmodified lines | ||
| 8230 | 8244 | |
| 8231 | 8245 | DRM DRIVER FOR NVIDIA GPUS [RUST] |
| 8232 | 8246 | M: Danilo Krummrich <dakr@kernel.org> |
| 8233 | −L: nouveau@lists.freedesktop.org | |
| 8247 | +L: nova-gpu@lists.linux.dev | |
| 8234 | 8248 | S: Supported |
| 8235 | 8249 | W: https://rust-for-linux.com/nova-gpu-driver |
| 8236 | −Q: https://patchwork.freedesktop.org/project/nouveau/ | |
| 8237 | 8250 | B: https://gitlab.freedesktop.org/drm/nova/-/issues |
| 8238 | 8251 | C: irc://irc.oftc.net/nouveau |
| 8239 | 8252 | T: git https://gitlab.freedesktop.org/drm/rust/kernel.git drm-rust-next |
| @@ -10848,7 +10861,7 @@ F: include/linux/generic-radix-tree.h · 2608 unmodified lines | ||
| 10848 | 10861 | F: lib/generic-radix-tree.c |
| 10849 | 10862 | |
| 10850 | 10863 | GENERIC RESISTIVE TOUCHSCREEN ADC DRIVER |
| 10851 | −M: Eugen Hristev <eugen.hristev@microchip.com> | |
| 10864 | +M: Eugen Hristev <ehristev@kernel.org> | |
| 10852 | 10865 | L: linux-input@vger.kernel.org |
| 10853 | 10866 | S: Maintained |
| 10854 | 10867 | F: drivers/input/touchscreen/resistive-adc-touch.c |
| @@ -12071,7 +12084,7 @@ F: Documentation/i2c/busses/i2c-nvidia-gpu.rst · 1216 unmodified lines | ||
| 12071 | 12084 | F: drivers/i2c/busses/i2c-nvidia-gpu.c |
| 12072 | 12085 | |
| 12073 | 12086 | I2C MUXES |
| 12074 | −M: Peter Rosin <peda@axentia.se> | |
| 12087 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 12075 | 12088 | L: linux-i2c@vger.kernel.org |
| 12076 | 12089 | S: Maintained |
| 12077 | 12090 | F: Documentation/devicetree/bindings/i2c/i2c-arb* |
| @@ -12472,7 +12485,7 @@ F: drivers/iio/industrialio-backend.c · 394 unmodified lines | ||
| 12472 | 12485 | F: include/linux/iio/backend.h |
| 12473 | 12486 | |
| 12474 | 12487 | IIO DIGITAL POTENTIOMETER DAC |
| 12475 | −M: Peter Rosin <peda@axentia.se> | |
| 12488 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 12476 | 12489 | L: linux-iio@vger.kernel.org |
| 12477 | 12490 | S: Maintained |
| 12478 | 12491 | F: Documentation/ABI/testing/sysfs-bus-iio-dac-dpot-dac |
| @@ -12480,7 +12493,7 @@ F: Documentation/devicetree/bindings/iio/dac/dpot-dac.yaml · 1 unmodified line | ||
| 12480 | 12493 | F: drivers/iio/dac/dpot-dac.c |
| 12481 | 12494 | |
| 12482 | 12495 | IIO ENVELOPE DETECTOR |
| 12483 | −M: Peter Rosin <peda@axentia.se> | |
| 12496 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 12484 | 12497 | L: linux-iio@vger.kernel.org |
| 12485 | 12498 | S: Maintained |
| 12486 | 12499 | F: Documentation/ABI/testing/sysfs-bus-iio-adc-envelope-detector |
| @@ -12496,7 +12509,7 @@ F: include/linux/iio/iio-gts-helper.h · 9 unmodified lines | ||
| 12496 | 12509 | F: drivers/iio/test/iio-test-gts.c |
| 12497 | 12510 | |
| 12498 | 12511 | IIO MULTIPLEXER |
| 12499 | −M: Peter Rosin <peda@axentia.se> | |
| 12512 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 12500 | 12513 | L: linux-iio@vger.kernel.org |
| 12501 | 12514 | S: Maintained |
| 12502 | 12515 | F: Documentation/devicetree/bindings/iio/multiplexer/io-channel-mux.yaml |
| @@ -12528,7 +12541,7 @@ F: include/linux/iio/ · 25 unmodified lines | ||
| 12528 | 12541 | F: tools/iio/ |
| 12529 | 12542 | |
| 12530 | 12543 | IIO UNIT CONVERTER |
| 12531 | −M: Peter Rosin <peda@axentia.se> | |
| 12544 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 12532 | 12545 | L: linux-iio@vger.kernel.org |
| 12533 | 12546 | S: Maintained |
| 12534 | 12547 | F: Documentation/devicetree/bindings/iio/afe/current-sense-amplifier.yaml |
| @@ -12805,7 +12818,6 @@ M: Cezary Rojewski <cezary.rojewski@intel.com> · 270 unmodified lines | ||
| 12805 | 12818 | M: Liam Girdwood <liam.r.girdwood@linux.intel.com> |
| 12806 | 12819 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> |
| 12807 | 12820 | M: Bard Liao <yung-chuan.liao@linux.intel.com> |
| 12808 | −M: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> | |
| 12809 | 12821 | M: Kai Vehmanen <kai.vehmanen@linux.intel.com> |
| 12810 | 12822 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> |
| 12811 | 12823 | L: linux-sound@vger.kernel.org |
| @@ -13498,7 +13510,7 @@ F: include/linux/iommu-dma.h · 686 unmodified lines | ||
| 13498 | 13510 | F: include/linux/iova.h |
| 13499 | 13511 | |
| 13500 | 13512 | IOMMU SUBSYSTEM |
| 13501 | −M: Joerg Roedel <joro@8bytes.org> | |
| 13513 | +M: Joerg Roedel (AMD) <joro@8bytes.org> | |
| 13502 | 13514 | M: Will Deacon <will@kernel.org> |
| 13503 | 13515 | R: Robin Murphy <robin.murphy@arm.com> |
| 13504 | 13516 | L: iommu@lists.linux.dev |
| @@ -13886,7 +13898,7 @@ M: Pratyush Yadav <pratyush@kernel.org> · 381 unmodified lines | ||
| 13886 | 13898 | R: Dave Young <ruirui.yang@linux.dev> |
| 13887 | 13899 | L: kexec@lists.infradead.org |
| 13888 | 13900 | S: Maintained |
| 13889 | −W: http://lse.sourceforge.net/kdump/ | |
| 13901 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git | |
| 13890 | 13902 | F: Documentation/admin-guide/kdump/ |
| 13891 | 13903 | F: fs/proc/vmcore.c |
| 13892 | 13904 | F: include/linux/crash_core.h |
| @@ -14079,6 +14091,7 @@ KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) · 186 unmodified lines | ||
| 14079 | 14091 | M: Marc Zyngier <maz@kernel.org> |
| 14080 | 14092 | M: Oliver Upton <oupton@kernel.org> |
| 14081 | 14093 | R: Joey Gouly <joey.gouly@arm.com> |
| 14094 | +R: Steffen Eiden <seiden@linux.ibm.com> | |
| 14082 | 14095 | R: Suzuki K Poulose <suzuki.poulose@arm.com> |
| 14083 | 14096 | R: Zenghui Yu <yuzenghui@huawei.com> |
| 14084 | 14097 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| @@ -14203,6 +14216,7 @@ M: Pasha Tatashin <pasha.tatashin@soleen.com> · 118 unmodified lines | ||
| 14203 | 14216 | M: Pratyush Yadav <pratyush@kernel.org> |
| 14204 | 14217 | L: kexec@lists.infradead.org |
| 14205 | 14218 | W: http://kernel.org/pub/linux/utils/kernel/kexec/ |
| 14219 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git | |
| 14206 | 14220 | F: include/linux/kexec.h |
| 14207 | 14221 | F: include/uapi/linux/kexec.h |
| 14208 | 14222 | F: kernel/kexec* |
| @@ -14919,6 +14933,7 @@ LIVE UPDATE · 710 unmodified lines | ||
| 14919 | 14933 | M: Pasha Tatashin <pasha.tatashin@soleen.com> |
| 14920 | 14934 | M: Mike Rapoport <rppt@kernel.org> |
| 14921 | 14935 | M: Pratyush Yadav <pratyush@kernel.org> |
| 14936 | +L: kexec@lists.infradead.org | |
| 14922 | 14937 | L: linux-kernel@vger.kernel.org |
| 14923 | 14938 | S: Maintained |
| 14924 | 14939 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git |
| @@ -15278,7 +15293,7 @@ M: Andrea Cervesato <andrea.cervesato@suse.com> · 353 unmodified lines | ||
| 15278 | 15293 | M: Cyril Hrubis <chrubis@suse.cz> |
| 15279 | 15294 | M: Jan Stancek <jstancek@redhat.com> |
| 15280 | 15295 | M: Petr Vorel <pvorel@suse.cz> |
| 15281 | −M: Li Wang <liwang@redhat.com> | |
| 15296 | +M: Li Wang <li.wang@linux.dev> | |
| 15282 | 15297 | M: Yang Xu <xuyang2018.jy@fujitsu.com> |
| 15283 | 15298 | M: Xiao Yang <yangx.jy@fujitsu.com> |
| 15284 | 15299 | L: ltp@lists.linux.it (subscribers-only) |
| @@ -15425,7 +15440,7 @@ F: include/net/netns/mctp.h · 140 unmodified lines | ||
| 15425 | 15440 | F: net/mctp/ |
| 15426 | 15441 | |
| 15427 | 15442 | MAPLE TREE |
| 15428 | −M: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 15443 | +M: Liam R. Howlett <liam@infradead.org> | |
| 15429 | 15444 | R: Alice Ryhl <aliceryhl@google.com> |
| 15430 | 15445 | R: Andrew Ballance <andrewjballance@gmail.com> |
| 15431 | 15446 | L: maple-tree@lists.infradead.org |
| @@ -15745,7 +15760,7 @@ F: Documentation/devicetree/bindings/media/i2c/maxim,max96717.yaml · 313 unmodified lines | ||
| 15745 | 15760 | F: drivers/media/i2c/max96717.c |
| 15746 | 15761 | |
| 15747 | 15762 | MAX9860 MONO AUDIO VOICE CODEC DRIVER |
| 15748 | −M: Peter Rosin <peda@axentia.se> | |
| 15763 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 15749 | 15764 | L: linux-sound@vger.kernel.org |
| 15750 | 15765 | S: Maintained |
| 15751 | 15766 | F: Documentation/devicetree/bindings/sound/max9860.txt |
| @@ -15960,7 +15975,7 @@ F: Documentation/devicetree/bindings/net/can/microchip,mcp251xfd.yaml · 208 unmodified lines | ||
| 15960 | 15975 | F: drivers/net/can/spi/mcp251xfd/ |
| 15961 | 15976 | |
| 15962 | 15977 | MCP4018 AND MCP4531 MICROCHIP DIGITAL POTENTIOMETER DRIVERS |
| 15963 | −M: Peter Rosin <peda@axentia.se> | |
| 15978 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 15964 | 15979 | L: linux-iio@vger.kernel.org |
| 15965 | 15980 | S: Maintained |
| 15966 | 15981 | F: Documentation/ABI/testing/sysfs-bus-iio-potentiometer-mcp4531 |
| @@ -16518,7 +16533,7 @@ F: drivers/usb/mtu3/ · 551 unmodified lines | ||
| 16518 | 16533 | |
| 16519 | 16534 | MEGACHIPS STDPXXXX-GE-B850V3-FW LVDS/DP++ BRIDGES |
| 16520 | 16535 | M: Peter Senna Tschudin <peter.senna@gmail.com> |
| 16521 | −M: Ian Ray <ian.ray@ge.com> | |
| 16536 | +M: Ian Ray <ian.ray@gehealthcare.com> | |
| 16522 | 16537 | M: Martyn Welch <martyn.welch@collabora.co.uk> |
| 16523 | 16538 | S: Maintained |
| 16524 | 16539 | F: Documentation/devicetree/bindings/display/bridge/megachips-stdpxxxx-ge-b850v3-fw.txt |
| @@ -16785,7 +16800,7 @@ MEMORY MANAGEMENT - CORE · 260 unmodified lines | ||
| 16785 | 16800 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16786 | 16801 | M: David Hildenbrand <david@kernel.org> |
| 16787 | 16802 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16788 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 16803 | +R: Liam R. Howlett <liam@infradead.org> | |
| 16789 | 16804 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16790 | 16805 | R: Mike Rapoport <rppt@kernel.org> |
| 16791 | 16806 | R: Suren Baghdasaryan <surenb@google.com> |
| @@ -16831,7 +16846,7 @@ F: mm/sparse.c · 39 unmodified lines | ||
| 16831 | 16846 | F: mm/util.c |
| 16832 | 16847 | F: mm/vmpressure.c |
| 16833 | 16848 | F: mm/vmstat.c |
| 16834 | −N: include/linux/page[-_]* | |
| 16849 | +N: include\/linux\/page[-_][a-zA-Z]* | |
| 16835 | 16850 | |
| 16836 | 16851 | MEMORY MANAGEMENT - EXECMEM |
| 16837 | 16852 | M: Andrew Morton <akpm@linux-foundation.org> |
| @@ -16921,7 +16936,7 @@ MEMORY MANAGEMENT - MISC · 83 unmodified lines | ||
| 16921 | 16936 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16922 | 16937 | M: David Hildenbrand <david@kernel.org> |
| 16923 | 16938 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 16924 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 16939 | +R: Liam R. Howlett <liam@infradead.org> | |
| 16925 | 16940 | R: Vlastimil Babka <vbabka@kernel.org> |
| 16926 | 16941 | R: Mike Rapoport <rppt@kernel.org> |
| 16927 | 16942 | R: Suren Baghdasaryan <surenb@google.com> |
| @@ -16988,6 +17003,7 @@ S: Maintained · 60 unmodified lines | ||
| 16988 | 17003 | F: include/linux/compaction.h |
| 16989 | 17004 | F: include/linux/gfp.h |
| 16990 | 17005 | F: include/linux/page-isolation.h |
| 17006 | +F: include/linux/pageblock-flags.h | |
| 16991 | 17007 | F: mm/compaction.c |
| 16992 | 17008 | F: mm/debug_page_alloc.c |
| 16993 | 17009 | F: mm/debug_page_ref.c |
| @@ -17009,7 +17025,7 @@ M: Andrew Morton <akpm@linux-foundation.org> · 15 unmodified lines | ||
| 17009 | 17025 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 17010 | 17026 | R: David Hildenbrand <david@kernel.org> |
| 17011 | 17027 | R: Michal Hocko <mhocko@kernel.org> |
| 17012 | −R: Qi Zheng <zhengqi.arch@bytedance.com> | |
| 17028 | +R: Qi Zheng <qi.zheng@linux.dev> | |
| 17013 | 17029 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 17014 | 17030 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 17015 | 17031 | L: linux-mm@kvack.org |
| @@ -17022,7 +17038,7 @@ M: Andrew Morton <akpm@linux-foundation.org> · 6 unmodified lines | ||
| 17022 | 17038 | M: David Hildenbrand <david@kernel.org> |
| 17023 | 17039 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17024 | 17040 | R: Rik van Riel <riel@surriel.com> |
| 17025 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17041 | +R: Liam R. Howlett <liam@infradead.org> | |
| 17026 | 17042 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17027 | 17043 | R: Harry Yoo <harry@kernel.org> |
| 17028 | 17044 | R: Jann Horn <jannh@google.com> |
| @@ -17069,7 +17085,7 @@ M: David Hildenbrand <david@kernel.org> · 40 unmodified lines | ||
| 17069 | 17085 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17070 | 17086 | R: Zi Yan <ziy@nvidia.com> |
| 17071 | 17087 | R: Baolin Wang <baolin.wang@linux.alibaba.com> |
| 17072 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17088 | +R: Liam R. Howlett <liam@infradead.org> | |
| 17073 | 17089 | R: Nico Pache <npache@redhat.com> |
| 17074 | 17090 | R: Ryan Roberts <ryan.roberts@arm.com> |
| 17075 | 17091 | R: Dev Jain <dev.jain@arm.com> |
| @@ -17107,7 +17123,7 @@ F: tools/testing/selftests/mm/uffd-*.[ch] · 31 unmodified lines | ||
| 17107 | 17123 | MEMORY MANAGEMENT - RUST |
| 17108 | 17124 | M: Alice Ryhl <aliceryhl@google.com> |
| 17109 | 17125 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 17110 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17126 | +R: Liam R. Howlett <liam@infradead.org> | |
| 17111 | 17127 | L: linux-mm@kvack.org |
| 17112 | 17128 | L: rust-for-linux@vger.kernel.org |
| 17113 | 17129 | S: Maintained |
| @@ -17121,7 +17137,7 @@ F: rust/kernel/page.rs · 7 unmodified lines | ||
| 17121 | 17137 | |
| 17122 | 17138 | MEMORY MAPPING |
| 17123 | 17139 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17124 | −M: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17140 | +M: Liam R. Howlett <liam@infradead.org> | |
| 17125 | 17141 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17126 | 17142 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17127 | 17143 | R: Jann Horn <jannh@google.com> |
| @@ -17153,7 +17169,7 @@ F: tools/testing/vma/ · 25 unmodified lines | ||
| 17153 | 17169 | MEMORY MAPPING - LOCKING |
| 17154 | 17170 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17155 | 17171 | M: Suren Baghdasaryan <surenb@google.com> |
| 17156 | −M: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17172 | +M: Liam R. Howlett <liam@infradead.org> | |
| 17157 | 17173 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17158 | 17174 | R: Vlastimil Babka <vbabka@kernel.org> |
| 17159 | 17175 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| @@ -17168,7 +17184,7 @@ F: mm/mmap_lock.c · 8 unmodified lines | ||
| 17168 | 17184 | |
| 17169 | 17185 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) |
| 17170 | 17186 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17171 | −M: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 17187 | +M: Liam R. Howlett <liam@infradead.org> | |
| 17172 | 17188 | M: Lorenzo Stoakes <ljs@kernel.org> |
| 17173 | 17189 | M: David Hildenbrand <david@kernel.org> |
| 17174 | 17190 | R: Vlastimil Babka <vbabka@kernel.org> |
| @@ -17363,7 +17379,7 @@ F: Documentation/devicetree/bindings/sound/mikroe,mikroe-proto.txt · 188 unmodified lines | ||
| 17363 | 17379 | F: sound/soc/atmel |
| 17364 | 17380 | |
| 17365 | 17381 | MICROCHIP CSI2DC DRIVER |
| 17366 | −M: Eugen Hristev <eugen.hristev@microchip.com> | |
| 17382 | +M: Eugen Hristev <ehristev@kernel.org> | |
| 17367 | 17383 | L: linux-media@vger.kernel.org |
| 17368 | 17384 | S: Supported |
| 17369 | 17385 | F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml |
| @@ -17390,7 +17406,7 @@ F: drivers/i2c/busses/i2c-at91-*.c · 20 unmodified lines | ||
| 17390 | 17406 | F: drivers/i2c/busses/i2c-at91.h |
| 17391 | 17407 | |
| 17392 | 17408 | MICROCHIP ISC DRIVER |
| 17393 | −M: Eugen Hristev <eugen.hristev@microchip.com> | |
| 17409 | +M: Eugen Hristev <ehristev@kernel.org> | |
| 17394 | 17410 | L: linux-media@vger.kernel.org |
| 17395 | 17411 | S: Supported |
| 17396 | 17412 | F: Documentation/devicetree/bindings/media/atmel,isc.yaml |
| @@ -17402,7 +17418,7 @@ F: drivers/staging/media/deprecated/atmel/atmel-sama*-isc* · 5 unmodified lines | ||
| 17402 | 17418 | F: include/linux/atmel-isc-media.h |
| 17403 | 17419 | |
| 17404 | 17420 | MICROCHIP ISI DRIVER |
| 17405 | −M: Eugen Hristev <eugen.hristev@microchip.com> | |
| 17421 | +M: Eugen Hristev <ehristev@kernel.org> | |
| 17406 | 17422 | L: linux-media@vger.kernel.org |
| 17407 | 17423 | S: Supported |
| 17408 | 17424 | F: drivers/media/platform/atmel/atmel-isi.c |
| @@ -17592,7 +17608,7 @@ F: Documentation/devicetree/bindings/display/bridge/microchip,sam9x75-lvds.yaml · 183 unmodified lines | ||
| 17592 | 17608 | F: drivers/gpu/drm/bridge/microchip-lvds.c |
| 17593 | 17609 | |
| 17594 | 17610 | MICROCHIP SAMA5D2-COMPATIBLE ADC DRIVER |
| 17595 | −M: Eugen Hristev <eugen.hristev@microchip.com> | |
| 17611 | +M: Eugen Hristev <ehristev@kernel.org> | |
| 17596 | 17612 | L: linux-iio@vger.kernel.org |
| 17597 | 17613 | S: Supported |
| 17598 | 17614 | F: Documentation/devicetree/bindings/iio/adc/atmel,sama5d2-adc.yaml |
| @@ -18271,7 +18287,7 @@ F: include/linux/mmc/ · 672 unmodified lines | ||
| 18271 | 18287 | F: include/uapi/linux/mmc/ |
| 18272 | 18288 | |
| 18273 | 18289 | MULTIPLEXER SUBSYSTEM |
| 18274 | −M: Peter Rosin <peda@axentia.se> | |
| 18290 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 18275 | 18291 | S: Odd Fixes |
| 18276 | 18292 | F: Documentation/ABI/testing/sysfs-class-mux* |
| 18277 | 18293 | F: Documentation/devicetree/bindings/mux/ |
| @@ -18655,6 +18671,7 @@ F: tools/testing/selftests/net/ · 377 unmodified lines | ||
| 18655 | 18671 | X: Documentation/networking/mac80211-injection.rst |
| 18656 | 18672 | X: Documentation/networking/mac80211_hwsim/ |
| 18657 | 18673 | X: Documentation/networking/regulatory.rst |
| 18674 | +X: include/net/bluetooth/ | |
| 18658 | 18675 | X: include/net/cfg80211.h |
| 18659 | 18676 | X: include/net/ieee80211_radiotap.h |
| 18660 | 18677 | X: include/net/iw_handler.h |
| @@ -18705,19 +18722,59 @@ F: net/xfrm/ · 44 unmodified lines | ||
| 18705 | 18722 | F: tools/testing/selftests/net/ipsec.c |
| 18706 | 18723 | |
| 18707 | 18724 | NETWORKING [IPv4/IPv6] |
| 18708 | −M: "David S. Miller" <davem@davemloft.net> | |
| 18709 | 18725 | M: David Ahern <dsahern@kernel.org> |
| 18726 | +M: Ido Schimmel <idosch@nvidia.com> | |
| 18710 | 18727 | L: netdev@vger.kernel.org |
| 18711 | 18728 | S: Maintained |
| 18712 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git | |
| 18713 | −F: arch/x86/net/* | |
| 18714 | −F: include/linux/ip.h | |
| 18715 | −F: include/linux/ipv6* | |
| 18729 | +F: Documentation/netlink/specs/rt-addr.yaml | |
| 18730 | +F: Documentation/netlink/specs/rt-neigh.yaml | |
| 18731 | +F: Documentation/netlink/specs/rt-route.yaml | |
| 18732 | +F: Documentation/netlink/specs/rt-rule.yaml | |
| 18733 | +F: include/linux/inetdevice.h | |
| 18734 | +F: include/linux/mroute* | |
| 18735 | +F: include/net/addrconf.h | |
| 18736 | +F: include/net/arp.h | |
| 18716 | 18737 | F: include/net/fib* |
| 18738 | +F: include/net/if_inet6.h | |
| 18739 | +F: include/net/inetpeer.h | |
| 18717 | 18740 | F: include/net/ip* |
| 18741 | +F: include/net/lwtunnel.h | |
| 18742 | +F: include/net/ndisc.h | |
| 18743 | +F: include/net/netns/nexthop.h | |
| 18744 | +F: include/net/nexthop.h | |
| 18718 | 18745 | F: include/net/route.h |
| 18719 | −F: net/ipv4/ | |
| 18720 | −F: net/ipv6/ | |
| 18746 | +F: include/uapi/linux/fib_rules.h | |
| 18747 | +F: include/uapi/linux/in_route.h | |
| 18748 | +F: include/uapi/linux/mroute* | |
| 18749 | +F: include/uapi/linux/nexthop.h | |
| 18750 | +F: net/core/fib* | |
| 18751 | +F: net/core/lwtunnel.c | |
| 18752 | +F: net/ipv4/arp.c | |
| 18753 | +F: net/ipv4/devinet.c | |
| 18754 | +F: net/ipv4/fib* | |
| 18755 | +F: net/ipv4/icmp.c | |
| 18756 | +F: net/ipv4/igmp.c | |
| 18757 | +F: net/ipv4/inet_fragment.c | |
| 18758 | +F: net/ipv4/inetpeer.c | |
| 18759 | +F: net/ipv4/ip* | |
| 18760 | +F: net/ipv4/metrics.c | |
| 18761 | +F: net/ipv4/netlink.c | |
| 18762 | +F: net/ipv4/nexthop.c | |
| 18763 | +F: net/ipv4/route.c | |
| 18764 | +F: net/ipv6/addr* | |
| 18765 | +F: net/ipv6/anycast.c | |
| 18766 | +F: net/ipv6/exthdrs.c | |
| 18767 | +F: net/ipv6/exthdrs_core.c | |
| 18768 | +F: net/ipv6/fib* | |
| 18769 | +F: net/ipv6/icmp.c | |
| 18770 | +F: net/ipv6/ip* | |
| 18771 | +F: net/ipv6/mcast* | |
| 18772 | +F: net/ipv6/ndisc.c | |
| 18773 | +F: net/ipv6/output_core.c | |
| 18774 | +F: net/ipv6/reassembly.c | |
| 18775 | +F: net/ipv6/route.c | |
| 18776 | +F: tools/testing/selftests/net/fib* | |
| 18777 | +F: tools/testing/selftests/net/forwarding/ | |
| 18721 | 18778 | |
| 18722 | 18779 | NETWORKING [LABELED] (NetLabel, Labeled IPsec, SECMARK) |
| 18723 | 18780 | M: Paul Moore <paul@paul-moore.com> |
| @@ -18852,18 +18909,11 @@ F: Documentation/networking/net_failover.rst · 128 unmodified lines | ||
| 18852 | 18909 | F: drivers/net/net_failover.c |
| 18853 | 18910 | F: include/net/net_failover.h |
| 18854 | 18911 | |
| 18855 | −NEXTHOP | |
| 18856 | −M: David Ahern <dsahern@kernel.org> | |
| 18857 | −L: netdev@vger.kernel.org | |
| 18858 | −S: Maintained | |
| 18859 | −F: include/net/netns/nexthop.h | |
| 18860 | −F: include/net/nexthop.h | |
| 18861 | −F: include/uapi/linux/nexthop.h | |
| 18862 | −F: net/ipv4/nexthop.c | |
| 18863 | − | |
| 18864 | 18912 | NFC SUBSYSTEM |
| 18865 | −L: netdev@vger.kernel.org | |
| 18866 | −S: Orphan | |
| 18913 | +M: David Heidelberg <david+nfc@ixit.cz> | |
| 18914 | +L: oe-linux-nfc@lists.linux.dev | |
| 18915 | +S: Maintained | |
| 18916 | +T: git https://codeberg.org/linux-nfc/linux.git | |
| 18867 | 18917 | F: Documentation/devicetree/bindings/net/nfc/ |
| 18868 | 18918 | F: drivers/nfc/ |
| 18869 | 18919 | F: include/net/nfc/ |
| @@ -18931,7 +18981,8 @@ F: drivers/hid/hid-nintendo* · 61 unmodified lines | ||
| 18931 | 18981 | |
| 18932 | 18982 | NIOS2 ARCHITECTURE |
| 18933 | 18983 | M: Dinh Nguyen <dinguyen@kernel.org> |
| 18934 | −S: Maintained | |
| 18984 | +M: Simon Schuster <schuster.simon@siemens-energy.com> | |
| 18985 | +S: Supported | |
| 18935 | 18986 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/dinguyen/linux.git |
| 18936 | 18987 | F: arch/nios2/ |
| 18937 | 18988 | |
| @@ -19347,7 +19398,7 @@ F: include/dt-bindings/display/tda998x.h · 409 unmodified lines | ||
| 19347 | 19398 | K: "nxp,tda998x" |
| 19348 | 19399 | |
| 19349 | 19400 | NXP TFA9879 DRIVER |
| 19350 | −M: Peter Rosin <peda@axentia.se> | |
| 19401 | +M: Peter Rosin <peda@lysator.liu.se> | |
| 19351 | 19402 | L: linux-sound@vger.kernel.org |
| 19352 | 19403 | S: Maintained |
| 19353 | 19404 | F: Documentation/devicetree/bindings/sound/trivial-codec.yaml |
| @@ -19445,7 +19496,6 @@ F: include/misc/ocxl* · 91 unmodified lines | ||
| 19445 | 19496 | F: include/uapi/misc/ocxl.h |
| 19446 | 19497 | |
| 19447 | 19498 | OMAP AUDIO SUPPORT |
| 19448 | −M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | |
| 19449 | 19499 | M: Jarkko Nikula <jarkko.nikula@bitmer.com> |
| 19450 | 19500 | L: linux-sound@vger.kernel.org |
| 19451 | 19501 | L: linux-omap@vger.kernel.org |
| @@ -20348,13 +20398,14 @@ F: Documentation/devicetree/bindings/pci/marvell,armada8k-pcie.yaml · 896 unmodified lines | ||
| 20348 | 20398 | F: drivers/pci/controller/dwc/pcie-armada8k.c |
| 20349 | 20399 | |
| 20350 | 20400 | PCI DRIVER FOR CADENCE PCIE IP |
| 20401 | +R: Aksh Garg <a-garg7@ti.com> | |
| 20351 | 20402 | L: linux-pci@vger.kernel.org |
| 20352 | 20403 | S: Orphan |
| 20353 | 20404 | F: Documentation/devicetree/bindings/pci/cdns,* |
| 20354 | −F: drivers/pci/controller/cadence/*cadence* | |
| 20405 | +F: drivers/pci/controller/cadence/ | |
| 20355 | 20406 | |
| 20356 | 20407 | PCI DRIVER FOR CIX Sky1 |
| 20357 | −M: Hans Zhang <hans.zhang@cixtech.com> | |
| 20408 | +M: Hans Zhang <18255117159@163.com> | |
| 20358 | 20409 | L: linux-pci@vger.kernel.org |
| 20359 | 20410 | S: Maintained |
| 20360 | 20411 | F: Documentation/devicetree/bindings/pci/cix,sky1-pcie-*.yaml |
| @@ -20466,7 +20517,7 @@ F: drivers/pci/controller/plda/pcie-plda-host.c · 105 unmodified lines | ||
| 20466 | 20517 | F: drivers/pci/controller/plda/pcie-plda.h |
| 20467 | 20518 | |
| 20468 | 20519 | PCI DRIVER FOR RENESAS R-CAR |
| 20469 | −M: Marek Vasut <marek.vasut+renesas@gmail.com> | |
| 20520 | +M: Marek Vasut <marek.vasut+renesas@mailbox.org> | |
| 20470 | 20521 | M: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com> |
| 20471 | 20522 | L: linux-pci@vger.kernel.org |
| 20472 | 20523 | L: linux-renesas-soc@vger.kernel.org |
| @@ -20726,15 +20777,13 @@ F: Documentation/devicetree/bindings/pci/intel,keembay-pcie* · 253 unmodified lines | ||
| 20726 | 20777 | F: drivers/pci/controller/dwc/pcie-keembay.c |
| 20727 | 20778 | |
| 20728 | 20779 | PCIE DRIVER FOR INTEL LGM GW SOC |
| 20729 | −M: Chuanhua Lei <lchuanhua@maxlinear.com> | |
| 20730 | 20780 | L: linux-pci@vger.kernel.org |
| 20731 | −S: Maintained | |
| 20781 | +S: Orphan | |
| 20732 | 20782 | F: Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml |
| 20733 | 20783 | F: drivers/pci/controller/dwc/pcie-intel-gw.c |
| 20734 | 20784 | |
| 20735 | 20785 | PCIE DRIVER FOR MEDIATEK |
| 20736 | 20786 | M: Ryder Lee <ryder.lee@mediatek.com> |
| 20737 | −M: Jianjun Wang <jianjun.wang@mediatek.com> | |
| 20738 | 20787 | L: linux-pci@vger.kernel.org |
| 20739 | 20788 | L: linux-mediatek@lists.infradead.org (moderated for non-subscribers) |
| 20740 | 20789 | S: Supported |
| @@ -20807,6 +20856,7 @@ M: Dominik Brodowski <linux@dominikbrodowski.net> · 66 unmodified lines | ||
| 20807 | 20856 | S: Odd Fixes |
| 20808 | 20857 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux.git |
| 20809 | 20858 | F: Documentation/pcmcia/ |
| 20859 | +F: drivers/net/ethernet/8390/pcnet_cs.c | |
| 20810 | 20860 | F: drivers/pcmcia/ |
| 20811 | 20861 | F: include/pcmcia/ |
| 20812 | 20862 | F: tools/pcmcia/ |
| @@ -22939,7 +22989,7 @@ N: riscv · 2126 unmodified lines | ||
| 22939 | 22989 | K: riscv |
| 22940 | 22990 | |
| 22941 | 22991 | RISC-V IOMMU |
| 22942 | −M: Tomasz Jeznach <tjeznach@rivosinc.com> | |
| 22992 | +M: Tomasz Jeznach <tomasz.jeznach@linux.dev> | |
| 22943 | 22993 | L: iommu@lists.linux.dev |
| 22944 | 22994 | L: linux-riscv@lists.infradead.org |
| 22945 | 22995 | S: Maintained |
| @@ -23402,7 +23452,7 @@ RUST [ALLOC] · 456 unmodified lines | ||
| 23402 | 23452 | M: Danilo Krummrich <dakr@kernel.org> |
| 23403 | 23453 | R: Lorenzo Stoakes <ljs@kernel.org> |
| 23404 | 23454 | R: Vlastimil Babka <vbabka@kernel.org> |
| 23405 | −R: Liam R. Howlett <Liam.Howlett@oracle.com> | |
| 23455 | +R: Liam R. Howlett <liam@infradead.org> | |
| 23406 | 23456 | R: Uladzislau Rezki <urezki@gmail.com> |
| 23407 | 23457 | L: rust-for-linux@vger.kernel.org |
| 23408 | 23458 | S: Maintained |
| @@ -23554,7 +23604,7 @@ F: drivers/s390/net/ · 145 unmodified lines | ||
| 23554 | 23604 | |
| 23555 | 23605 | S390 PCI SUBSYSTEM |
| 23556 | 23606 | M: Niklas Schnelle <schnelle@linux.ibm.com> |
| 23557 | −M: Gerald Schaefer <gerald.schaefer@linux.ibm.com> | |
| 23607 | +M: Gerd Bayer <gbayer@linux.ibm.com> | |
| 23558 | 23608 | L: linux-s390@vger.kernel.org |
| 23559 | 23609 | S: Supported |
| 23560 | 23610 | F: Documentation/arch/s390/pci.rst |
| @@ -24108,7 +24158,7 @@ F: drivers/mmc/host/sdhci* · 547 unmodified lines | ||
| 24108 | 24158 | |
| 24109 | 24159 | SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) MICROCHIP DRIVER |
| 24110 | 24160 | M: Aubin Constans <aubin.constans@microchip.com> |
| 24111 | −R: Eugen Hristev <eugen.hristev@collabora.com> | |
| 24161 | +R: Eugen Hristev <ehristev@kernel.org> | |
| 24112 | 24162 | L: linux-mmc@vger.kernel.org |
| 24113 | 24163 | S: Supported |
| 24114 | 24164 | F: drivers/mmc/host/sdhci-of-at91.c |
| @@ -24347,7 +24397,7 @@ F: include/media/i2c/rj54n1cb0c.h · 232 unmodified lines | ||
| 24347 | 24397 | SHRINKER |
| 24348 | 24398 | M: Andrew Morton <akpm@linux-foundation.org> |
| 24349 | 24399 | M: Dave Chinner <david@fromorbit.com> |
| 24350 | −R: Qi Zheng <zhengqi.arch@bytedance.com> | |
| 24400 | +R: Qi Zheng <qi.zheng@linux.dev> | |
| 24351 | 24401 | R: Roman Gushchin <roman.gushchin@linux.dev> |
| 24352 | 24402 | R: Muchun Song <muchun.song@linux.dev> |
| 24353 | 24403 | L: linux-mm@kvack.org |
| @@ -24649,6 +24699,7 @@ S: Maintained · 295 unmodified lines | ||
| 24649 | 24699 | F: fs/smb/client/smbdirect.* |
| 24650 | 24700 | F: fs/smb/smbdirect/ |
| 24651 | 24701 | F: fs/smb/server/transport_rdma.* |
| 24702 | +F: include/linux/smbdirect.h | |
| 24652 | 24703 | |
| 24653 | 24704 | SMC91x ETHERNET DRIVER |
| 24654 | 24705 | M: Nicolas Pitre <nico@fluxnic.net> |
| @@ -24797,6 +24848,7 @@ SOFTWARE RAID (Multiple Disks) SUPPORT · 142 unmodified lines | ||
| 24797 | 24848 | M: Song Liu <song@kernel.org> |
| 24798 | 24849 | M: Yu Kuai <yukuai@fnnas.com> |
| 24799 | 24850 | R: Li Nan <linan122@huawei.com> |
| 24851 | +R: Xiao Ni <xiao@kernel.org> | |
| 24800 | 24852 | L: linux-raid@vger.kernel.org |
| 24801 | 24853 | S: Supported |
| 24802 | 24854 | Q: https://patchwork.kernel.org/project/linux-raid/list/ |
| @@ -25051,7 +25103,6 @@ SOUND - SOUND OPEN FIRMWARE (SOF) DRIVERS · 248 unmodified lines | ||
| 25051 | 25103 | M: Liam Girdwood <lgirdwood@gmail.com> |
| 25052 | 25104 | M: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> |
| 25053 | 25105 | M: Bard Liao <yung-chuan.liao@linux.intel.com> |
| 25054 | −M: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> | |
| 25055 | 25106 | M: Daniel Baluta <daniel.baluta@nxp.com> |
| 25056 | 25107 | R: Kai Vehmanen <kai.vehmanen@linux.intel.com> |
| 25057 | 25108 | R: Pierre-Louis Bossart <pierre-louis.bossart@linux.dev> |
| @@ -26344,7 +26395,7 @@ F: arch/xtensa/ · 1286 unmodified lines | ||
| 26344 | 26395 | F: drivers/irqchip/irq-xtensa-* |
| 26345 | 26396 | |
| 26346 | 26397 | TEXAS INSTRUMENTS ASoC DRIVERS |
| 26347 | −M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | |
| 26398 | +M: Sen Wang <sen@ti.com> | |
| 26348 | 26399 | L: linux-sound@vger.kernel.org |
| 26349 | 26400 | S: Maintained |
| 26350 | 26401 | F: Documentation/devicetree/bindings/sound/davinci-mcasp-audio.yaml |
| @@ -26846,12 +26897,6 @@ S: Maintained · 495 unmodified lines | ||
| 26846 | 26897 | F: Documentation/devicetree/bindings/iio/adc/ti,tsc2046.yaml |
| 26847 | 26898 | F: drivers/iio/adc/ti-tsc2046.c |
| 26848 | 26899 | |
| 26849 | −TI TWL4030 SERIES SOC CODEC DRIVER | |
| 26850 | −M: Peter Ujfalusi <peter.ujfalusi@gmail.com> | |
| 26851 | −L: linux-sound@vger.kernel.org | |
| 26852 | −S: Maintained | |
| 26853 | −F: sound/soc/codecs/twl4030* | |
| 26854 | − | |
| 26855 | 26900 | TI VPE/CAL DRIVERS |
| 26856 | 26901 | M: Yemike Abhilash Chandra <y-abhilashchandra@ti.com> |
| 26857 | 26902 | L: linux-media@vger.kernel.org |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 7 |
| 3 | 3 | PATCHLEVEL = 1 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc1 | |
| 5 | +EXTRAVERSION = -rc6 | |
| 6 | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -486,6 +486,8 @@ export rust_common_flags := --edition=2021 \ · 477 unmodified lines | ||
| 486 | 486 | -Wclippy::as_ptr_cast_mut \ |
| 487 | 487 | -Wclippy::as_underscore \ |
| 488 | 488 | -Wclippy::cast_lossless \ |
| 489 | + -Aclippy::collapsible_if \ | |
| 490 | + -Aclippy::collapsible_match \ | |
| 489 | 491 | -Wclippy::ignored_unit_patterns \ |
| 490 | 492 | -Aclippy::incompatible_msrv \ |
| 491 | 493 | -Wclippy::mut_mut \ |
| @@ -5,4 +5,5 @@ generic-y += agp.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += asm-offsets.h |
| 6 | 6 | generic-y += kvm_para.h |
| 7 | 7 | generic-y += mcs_spinlock.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += text-patching.h |
| @@ -5,5 +5,6 @@ generic-y += extable.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += kvm_para.h |
| 6 | 6 | generic-y += mcs_spinlock.h |
| 7 | 7 | generic-y += parport.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += user.h |
| 9 | 10 | generic-y += text-patching.h |
| @@ -34,9 +34,6 @@ · 33 unmodified lines | ||
| 34 | 34 | clocks = <&mstp9_clks R7S72100_CLK_SPIBSC0>; |
| 35 | 35 | power-domains = <&cpg_clocks>; |
| 36 | 36 | |
| 37 | − #address-cells = <1>; | |
| 38 | − #size-cells = <1>; | |
| 39 | − | |
| 40 | 37 | partitions { |
| 41 | 38 | compatible = "fixed-partitions"; |
| 42 | 39 | #address-cells = <1>; |
| @@ -36,8 +36,6 @@ · 35 unmodified lines | ||
| 36 | 36 | power-domains = <&cpg_clocks>; |
| 37 | 37 | bank-width = <4>; |
| 38 | 38 | device-width = <1>; |
| 39 | − #address-cells = <1>; | |
| 40 | − #size-cells = <1>; | |
| 41 | 39 | |
| 42 | 40 | partitions { |
| 43 | 41 | compatible = "fixed-partitions"; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | clock-div = <3>; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | − bsc: bus { | |
| 40 | + bsc: bus@0 { | |
| 41 | 41 | compatible = "simple-bus"; |
| 42 | 42 | #address-cells = <1>; |
| 43 | 43 | #size-cells = <1>; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | ||
| 40 | 40 | spi2 = &hspi2; |
| 41 | 41 | }; |
| 42 | 42 | |
| 43 | − lbsc: bus { | |
| 43 | + lbsc: bus@0 { | |
| 44 | 44 | compatible = "simple-bus"; |
| 45 | 45 | #address-cells = <1>; |
| 46 | 46 | #size-cells = <1>; |
| @@ -704,7 +704,7 @@ · 703 unmodified lines | ||
| 704 | 704 | }; |
| 705 | 705 | }; |
| 706 | 706 | |
| 707 | − lbsc: bus { | |
| 707 | + lbsc: bus@0 { | |
| 708 | 708 | compatible = "simple-bus"; |
| 709 | 709 | #address-cells = <1>; |
| 710 | 710 | #size-cells = <1>; |
| @@ -86,7 +86,7 @@ · 85 unmodified lines | ||
| 86 | 86 | bootph-all; |
| 87 | 87 | }; |
| 88 | 88 | |
| 89 | − lbsc: bus { | |
| 89 | + lbsc: bus@0 { | |
| 90 | 90 | compatible = "simple-bus"; |
| 91 | 91 | #address-cells = <1>; |
| 92 | 92 | #size-cells = <1>; |
| @@ -3,6 +3,7 @@ generic-y += early_ioremap.h · 2 unmodified lines | ||
| 3 | 3 | generic-y += extable.h |
| 4 | 4 | generic-y += flat.h |
| 5 | 5 | generic-y += parport.h |
| 6 | +generic-y += ring_buffer.h | |
| 6 | 7 | |
| 7 | 8 | generated-y += mach-types.h |
| 8 | 9 | generated-y += unistd-nr.h |
| @@ -86,14 +86,6 @@ static u64 notrace intcp_read_sched_clock(void) · 85 unmodified lines | ||
| 86 | 86 | return val; |
| 87 | 87 | } |
| 88 | 88 | |
| 89 | −static void __init intcp_init_early(void) | |
| 90 | −{ | |
| 91 | − cm_map = syscon_regmap_lookup_by_compatible("arm,core-module-integrator"); | |
| 92 | − if (IS_ERR(cm_map)) | |
| 93 | − return; | |
| 94 | − sched_clock_register(intcp_read_sched_clock, 32, 24000000); | |
| 95 | −} | |
| 96 | − | |
| 97 | 89 | static void __init intcp_init_irq_of(void) |
| 98 | 90 | { |
| 99 | 91 | cm_init(); |
| @@ -119,6 +111,10 @@ static void __init intcp_init_of(void) · 19 unmodified lines | ||
| 119 | 111 | { |
| 120 | 112 | struct device_node *cpcon; |
| 121 | 113 | |
| 114 | + cm_map = syscon_regmap_lookup_by_compatible("arm,core-module-integrator"); | |
| 115 | + if (!IS_ERR(cm_map)) | |
| 116 | + sched_clock_register(intcp_read_sched_clock, 32, 24000000); | |
| 117 | + | |
| 122 | 118 | cpcon = of_find_matching_node(NULL, intcp_syscon_match); |
| 123 | 119 | if (!cpcon) |
| 124 | 120 | return; |
| @@ -138,7 +134,6 @@ static const char * intcp_dt_board_compat[] = { · 13 unmodified lines | ||
| 138 | 134 | DT_MACHINE_START(INTEGRATOR_CP_DT, "ARM Integrator/CP (Device Tree)") |
| 139 | 135 | .reserve = integrator_reserve, |
| 140 | 136 | .map_io = intcp_map_io, |
| 141 | − .init_early = intcp_init_early, | |
| 142 | 137 | .init_irq = intcp_init_irq_of, |
| 143 | 138 | .init_machine = intcp_init_of, |
| 144 | 139 | .dt_compat = intcp_dt_board_compat, |
| @@ -27,7 +27,12 @@ · 26 unmodified lines | ||
| 27 | 27 | status = "okay"; |
| 28 | 28 | |
| 29 | 29 | ports { |
| 30 | + #address-cells = <1>; | |
| 31 | + #size-cells = <0>; | |
| 32 | + | |
| 30 | 33 | port@1 { |
| 34 | + reg = <1>; | |
| 35 | + | |
| 31 | 36 | lvds1_out: endpoint { |
| 32 | 37 | remote-endpoint = <&panel_in>; |
| 33 | 38 | }; |
| @@ -699,7 +699,7 @@ · 698 unmodified lines | ||
| 699 | 699 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 700 | 700 | reg = <0 0xc0700000 0 0x40>; |
| 701 | 701 | interrupts = <GIC_ESPI 10 IRQ_TYPE_LEVEL_HIGH>; |
| 702 | − clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | |
| 702 | + clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; | |
| 703 | 703 | clock-names = "fck", "brg_int", "scif_clk"; |
| 704 | 704 | status = "disabled"; |
| 705 | 705 | }; |
| @@ -709,7 +709,7 @@ · 3 unmodified lines | ||
| 709 | 709 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 710 | 710 | reg = <0 0xc0704000 0 0x40>; |
| 711 | 711 | interrupts = <GIC_ESPI 11 IRQ_TYPE_LEVEL_HIGH>; |
| 712 | − clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | |
| 712 | + clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; | |
| 713 | 713 | clock-names = "fck", "brg_int", "scif_clk"; |
| 714 | 714 | status = "disabled"; |
| 715 | 715 | }; |
| @@ -719,7 +719,7 @@ · 3 unmodified lines | ||
| 719 | 719 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 720 | 720 | reg = <0 0xc0708000 0 0x40>; |
| 721 | 721 | interrupts = <GIC_ESPI 12 IRQ_TYPE_LEVEL_HIGH>; |
| 722 | − clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | |
| 722 | + clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; | |
| 723 | 723 | clock-names = "fck", "brg_int", "scif_clk"; |
| 724 | 724 | status = "disabled"; |
| 725 | 725 | }; |
| @@ -729,7 +729,7 @@ · 3 unmodified lines | ||
| 729 | 729 | "renesas,rcar-gen5-scif", "renesas,scif"; |
| 730 | 730 | reg = <0 0xc070c000 0 0x40>; |
| 731 | 731 | interrupts = <GIC_ESPI 13 IRQ_TYPE_LEVEL_HIGH>; |
| 732 | − clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd16>, <&scif_clk>; | |
| 732 | + clocks = <&dummy_clk_sgasyncd16>, <&dummy_clk_sgasyncd4>, <&scif_clk>; | |
| 733 | 733 | clock-names = "fck", "brg_int", "scif_clk"; |
| 734 | 734 | status = "disabled"; |
| 735 | 735 | }; |
| @@ -1327,6 +1327,7 @@ · 1326 unmodified lines | ||
| 1327 | 1327 | resets = <&cpg 0xaf>; |
| 1328 | 1328 | power-domains = <&cpg>; |
| 1329 | 1329 | #reset-cells = <0>; |
| 1330 | + #mux-state-cells = <1>; | |
| 1330 | 1331 | status = "disabled"; |
| 1331 | 1332 | }; |
| 1332 | 1333 | |
| @@ -1345,6 +1345,7 @@ · 1344 unmodified lines | ||
| 1345 | 1345 | resets = <&cpg 0xaf>; |
| 1346 | 1346 | power-domains = <&cpg>; |
| 1347 | 1347 | #reset-cells = <0>; |
| 1348 | + #mux-state-cells = <1>; | |
| 1348 | 1349 | status = "disabled"; |
| 1349 | 1350 | }; |
| 1350 | 1351 | |
| @@ -1355,6 +1356,7 @@ · 4 unmodified lines | ||
| 1355 | 1356 | resets = <&cpg 0xaf>; |
| 1356 | 1357 | power-domains = <&cpg>; |
| 1357 | 1358 | #reset-cells = <0>; |
| 1359 | + #mux-state-cells = <1>; | |
| 1358 | 1360 | status = "disabled"; |
| 1359 | 1361 | }; |
| 1360 | 1362 | |
| @@ -46,7 +46,12 @@ · 45 unmodified lines | ||
| 46 | 46 | status = "okay"; |
| 47 | 47 | |
| 48 | 48 | ports { |
| 49 | + #address-cells = <1>; | |
| 50 | + #size-cells = <0>; | |
| 51 | + | |
| 49 | 52 | port@0 { |
| 53 | + reg = <0>; | |
| 54 | + | |
| 50 | 55 | csi2_in: endpoint { |
| 51 | 56 | clock-lanes = <0>; |
| 52 | 57 | data-lanes = <1 2>; |
| @@ -26,7 +26,12 @@ · 25 unmodified lines | ||
| 26 | 26 | status = "okay"; |
| 27 | 27 | |
| 28 | 28 | ports { |
| 29 | + #address-cells = <1>; | |
| 30 | + #size-cells = <0>; | |
| 31 | + | |
| 29 | 32 | port@0 { |
| 33 | + reg = <0>; | |
| 34 | + | |
| 30 | 35 | du_out_rgb: endpoint { |
| 31 | 36 | remote-endpoint = <&adv7513_in>; |
| 32 | 37 | }; |
| @@ -27,7 +27,12 @@ · 26 unmodified lines | ||
| 27 | 27 | status = "okay"; |
| 28 | 28 | |
| 29 | 29 | ports { |
| 30 | + #address-cells = <1>; | |
| 31 | + #size-cells = <0>; | |
| 32 | + | |
| 30 | 33 | port@1 { |
| 34 | + reg = <1>; | |
| 35 | + | |
| 31 | 36 | lvds0_out: endpoint { |
| 32 | 37 | remote-endpoint = <&panel_in>; |
| 33 | 38 | }; |
| @@ -409,7 +409,7 @@ __AARCH64_INSN_FUNCS(cbz, 0x7F000000, 0x34000000) · 408 unmodified lines | ||
| 409 | 409 | __AARCH64_INSN_FUNCS(cbnz, 0x7F000000, 0x35000000) |
| 410 | 410 | __AARCH64_INSN_FUNCS(tbz, 0x7F000000, 0x36000000) |
| 411 | 411 | __AARCH64_INSN_FUNCS(tbnz, 0x7F000000, 0x37000000) |
| 412 | −__AARCH64_INSN_FUNCS(bcond, 0xFF000010, 0x54000000) | |
| 412 | +__AARCH64_INSN_FUNCS(bcond, 0xFF000000, 0x54000000) | |
| 413 | 413 | __AARCH64_INSN_FUNCS(svc, 0xFFE0001F, 0xD4000001) |
| 414 | 414 | __AARCH64_INSN_FUNCS(hvc, 0xFFE0001F, 0xD4000002) |
| 415 | 415 | __AARCH64_INSN_FUNCS(smc, 0xFFE0001F, 0xD4000003) |
| @@ -40,7 +40,7 @@ static __always_inline void __pmr_local_irq_enable(void) · 39 unmodified lines | ||
| 40 | 40 | barrier(); |
| 41 | 41 | } |
| 42 | 42 | |
| 43 | −static inline void arch_local_irq_enable(void) | |
| 43 | +static __always_inline void arch_local_irq_enable(void) | |
| 44 | 44 | { |
| 45 | 45 | if (system_uses_irq_prio_masking()) { |
| 46 | 46 | __pmr_local_irq_enable(); |
| @@ -68,7 +68,7 @@ static __always_inline void __pmr_local_irq_disable(void) · 21 unmodified lines | ||
| 68 | 68 | barrier(); |
| 69 | 69 | } |
| 70 | 70 | |
| 71 | −static inline void arch_local_irq_disable(void) | |
| 71 | +static __always_inline void arch_local_irq_disable(void) | |
| 72 | 72 | { |
| 73 | 73 | if (system_uses_irq_prio_masking()) { |
| 74 | 74 | __pmr_local_irq_disable(); |
| @@ -90,7 +90,7 @@ static __always_inline unsigned long __pmr_local_save_flags(void) · 15 unmodified lines | ||
| 90 | 90 | /* |
| 91 | 91 | * Save the current interrupt enable state. |
| 92 | 92 | */ |
| 93 | −static inline unsigned long arch_local_save_flags(void) | |
| 93 | +static __always_inline unsigned long arch_local_save_flags(void) | |
| 94 | 94 | { |
| 95 | 95 | if (system_uses_irq_prio_masking()) { |
| 96 | 96 | return __pmr_local_save_flags(); |
| @@ -109,7 +109,7 @@ static __always_inline bool __pmr_irqs_disabled_flags(unsigned long flags) · 12 unmodified lines | ||
| 109 | 109 | return flags != GIC_PRIO_IRQON; |
| 110 | 110 | } |
| 111 | 111 | |
| 112 | −static inline bool arch_irqs_disabled_flags(unsigned long flags) | |
| 112 | +static __always_inline bool arch_irqs_disabled_flags(unsigned long flags) | |
| 113 | 113 | { |
| 114 | 114 | if (system_uses_irq_prio_masking()) { |
| 115 | 115 | return __pmr_irqs_disabled_flags(flags); |
| @@ -128,7 +128,7 @@ static __always_inline bool __pmr_irqs_disabled(void) · 12 unmodified lines | ||
| 128 | 128 | return __pmr_irqs_disabled_flags(__pmr_local_save_flags()); |
| 129 | 129 | } |
| 130 | 130 | |
| 131 | −static inline bool arch_irqs_disabled(void) | |
| 131 | +static __always_inline bool arch_irqs_disabled(void) | |
| 132 | 132 | { |
| 133 | 133 | if (system_uses_irq_prio_masking()) { |
| 134 | 134 | return __pmr_irqs_disabled(); |
| @@ -160,7 +160,7 @@ static __always_inline unsigned long __pmr_local_irq_save(void) · 25 unmodified lines | ||
| 160 | 160 | return flags; |
| 161 | 161 | } |
| 162 | 162 | |
| 163 | −static inline unsigned long arch_local_irq_save(void) | |
| 163 | +static __always_inline unsigned long arch_local_irq_save(void) | |
| 164 | 164 | { |
| 165 | 165 | if (system_uses_irq_prio_masking()) { |
| 166 | 166 | return __pmr_local_irq_save(); |
| @@ -187,7 +187,7 @@ static __always_inline void __pmr_local_irq_restore(unsigned long flags) · 20 unmodified lines | ||
| 187 | 187 | /* |
| 188 | 188 | * restore saved IRQ state |
| 189 | 189 | */ |
| 190 | −static inline void arch_local_irq_restore(unsigned long flags) | |
| 190 | +static __always_inline void arch_local_irq_restore(unsigned long flags) | |
| 191 | 191 | { |
| 192 | 192 | if (system_uses_irq_prio_masking()) { |
| 193 | 193 | __pmr_local_irq_restore(flags); |
| @@ -68,7 +68,12 @@ · 67 unmodified lines | ||
| 68 | 68 | #define KERNEL_SEGMENT_COUNT 5 |
| 69 | 69 | |
| 70 | 70 | #if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN |
| 71 | −#define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 1) | |
| 71 | +/* | |
| 72 | + * KERNEL_SEGMENT_COUNT counts the permanent kernel VMAs. The early mapping | |
| 73 | + * has one additional split, [_text, _stext). Reserve one more page for the | |
| 74 | + * SWAPPER_BLOCK_SIZE-unaligned boundaries. | |
| 75 | + */ | |
| 76 | +#define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 2) | |
| 72 | 77 | /* |
| 73 | 78 | * The initial ID map consists of the kernel image, mapped as two separate |
| 74 | 79 | * segments, and may appear misaligned wrt the swapper block size. This means |
| @@ -50,6 +50,9 @@ · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | #include <linux/mm.h> |
| 52 | 52 | |
| 53 | +#define MARKER(m) \ | |
| 54 | + m, __after_##m = m - 1 | |
| 55 | + | |
| 53 | 56 | enum __kvm_host_smccc_func { |
| 54 | 57 | /* Hypercalls that are unavailable once pKVM has finalised. */ |
| 55 | 58 | /* __KVM_HOST_SMCCC_FUNC___kvm_hyp_init */ |
| @@ -59,8 +62,10 @@ enum __kvm_host_smccc_func { · 3 unmodified lines | ||
| 59 | 62 | __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, |
| 60 | 63 | __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, |
| 61 | 64 | __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, |
| 65 | + | |
| 66 | + MARKER(__KVM_HOST_SMCCC_FUNC_MIN_PKVM), | |
| 67 | + | |
| 62 | 68 | __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, |
| 63 | − __KVM_HOST_SMCCC_FUNC_MIN_PKVM = __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, | |
| 64 | 69 | |
| 65 | 70 | /* Hypercalls that are always available and common to [nh]VHE/pKVM. */ |
| 66 | 71 | __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, |
| @@ -72,11 +77,20 @@ enum __kvm_host_smccc_func { · 5 unmodified lines | ||
| 72 | 77 | __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_range, |
| 73 | 78 | __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context, |
| 74 | 79 | __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff, |
| 80 | + __KVM_HOST_SMCCC_FUNC___tracing_load, | |
| 81 | + __KVM_HOST_SMCCC_FUNC___tracing_unload, | |
| 82 | + __KVM_HOST_SMCCC_FUNC___tracing_enable, | |
| 83 | + __KVM_HOST_SMCCC_FUNC___tracing_swap_reader, | |
| 84 | + __KVM_HOST_SMCCC_FUNC___tracing_update_clock, | |
| 85 | + __KVM_HOST_SMCCC_FUNC___tracing_reset, | |
| 86 | + __KVM_HOST_SMCCC_FUNC___tracing_enable_event, | |
| 87 | + __KVM_HOST_SMCCC_FUNC___tracing_write_event, | |
| 75 | 88 | __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, |
| 76 | 89 | __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs, |
| 77 | 90 | __KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr, |
| 78 | 91 | __KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr, |
| 79 | − __KVM_HOST_SMCCC_FUNC_MAX_NO_PKVM = __KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr, | |
| 92 | + | |
| 93 | + MARKER(__KVM_HOST_SMCCC_FUNC_PKVM_ONLY), | |
| 80 | 94 | |
| 81 | 95 | /* Hypercalls that are available only when pKVM has finalised. */ |
| 82 | 96 | __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, |
| @@ -100,14 +114,8 @@ enum __kvm_host_smccc_func { · 17 unmodified lines | ||
| 100 | 114 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_load, |
| 101 | 115 | __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_put, |
| 102 | 116 | __KVM_HOST_SMCCC_FUNC___pkvm_tlb_flush_vmid, |
| 103 | − __KVM_HOST_SMCCC_FUNC___tracing_load, | |
| 104 | − __KVM_HOST_SMCCC_FUNC___tracing_unload, | |
| 105 | − __KVM_HOST_SMCCC_FUNC___tracing_enable, | |
| 106 | − __KVM_HOST_SMCCC_FUNC___tracing_swap_reader, | |
| 107 | − __KVM_HOST_SMCCC_FUNC___tracing_update_clock, | |
| 108 | − __KVM_HOST_SMCCC_FUNC___tracing_reset, | |
| 109 | − __KVM_HOST_SMCCC_FUNC___tracing_enable_event, | |
| 110 | − __KVM_HOST_SMCCC_FUNC___tracing_write_event, | |
| 117 | + | |
| 118 | + MARKER(__KVM_HOST_SMCCC_FUNC_MAX) | |
| 111 | 119 | }; |
| 112 | 120 | |
| 113 | 121 | #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] |
| @@ -450,9 +450,6 @@ struct kvm_vcpu_fault_info { · 449 unmodified lines | ||
| 450 | 450 | r = __VNCR_START__ + ((VNCR_ ## r) / 8), \ |
| 451 | 451 | __after_##r = __MAX__(__before_##r - 1, r) |
| 452 | 452 | |
| 453 | −#define MARKER(m) \ | |
| 454 | − m, __after_##m = m - 1 | |
| 455 | − | |
| 456 | 453 | enum vcpu_sysreg { |
| 457 | 454 | __INVALID_SYSREG__, /* 0 is reserved as an invalid value */ |
| 458 | 455 | MPIDR_EL1, /* MultiProcessor Affinity Register */ |
| @@ -514,7 +511,6 @@ enum vcpu_sysreg { · 55 unmodified lines | ||
| 514 | 511 | ACTLR_EL2, /* Auxiliary Control Register (EL2) */ |
| 515 | 512 | CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ |
| 516 | 513 | HACR_EL2, /* Hypervisor Auxiliary Control Register */ |
| 517 | − ZCR_EL2, /* SVE Control Register (EL2) */ | |
| 518 | 514 | TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ |
| 519 | 515 | TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ |
| 520 | 516 | TCR_EL2, /* Translation Control Register (EL2) */ |
| @@ -546,6 +542,7 @@ enum vcpu_sysreg { · 25 unmodified lines | ||
| 546 | 542 | SCTLR2_EL2, /* System Control Register 2 (EL2) */ |
| 547 | 543 | MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */ |
| 548 | 544 | CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */ |
| 545 | + ZCR_EL2, /* SVE Control Register (EL2) */ | |
| 549 | 546 | |
| 550 | 547 | /* Any VNCR-capable reg goes after this point */ |
| 551 | 548 | MARKER(__VNCR_START__), |
| @@ -1548,7 +1545,7 @@ static inline bool __vcpu_has_feature(const struct kvm_arch *ka, int feature) · 996 unmodified lines | ||
| 1548 | 1545 | #define kvm_vcpu_has_feature(k, f) __vcpu_has_feature(&(k)->arch, (f)) |
| 1549 | 1546 | #define vcpu_has_feature(v, f) __vcpu_has_feature(&(v)->kvm->arch, (f)) |
| 1550 | 1547 | |
| 1551 | −#define kvm_vcpu_initialized(v) vcpu_get_flag(vcpu, VCPU_INITIALIZED) | |
| 1548 | +#define kvm_vcpu_initialized(v) vcpu_get_flag(v, VCPU_INITIALIZED) | |
| 1552 | 1549 | |
| 1553 | 1550 | int kvm_trng_call(struct kvm_vcpu *vcpu); |
| 1554 | 1551 | #ifdef CONFIG_KVM |
| @@ -23,6 +23,7 @@ static inline u64 tcr_el2_ps_to_tcr_el1_ips(u64 tcr_el2) · 22 unmodified lines | ||
| 23 | 23 | static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr) |
| 24 | 24 | { |
| 25 | 25 | return TCR_EPD1_MASK | /* disable TTBR1_EL1 */ |
| 26 | + ((tcr & TCR_EL2_DS) ? TCR_DS : 0) | | |
| 26 | 27 | ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) | |
| 27 | 28 | tcr_el2_ps_to_tcr_el1_ips(tcr) | |
| 28 | 29 | (tcr & TCR_EL2_TG0_MASK) | |
| @@ -33,7 +33,7 @@ struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, · 32 unmodified lines | ||
| 33 | 33 | unsigned long vaddr); |
| 34 | 34 | #define vma_alloc_zeroed_movable_folio vma_alloc_zeroed_movable_folio |
| 35 | 35 | |
| 36 | −bool tag_clear_highpages(struct page *to, int numpages); | |
| 36 | +bool tag_clear_highpages(struct page *to, int numpages, bool clear_pages); | |
| 37 | 37 | #define __HAVE_ARCH_TAG_CLEAR_HIGHPAGES |
| 38 | 38 | |
| 39 | 39 | #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) |
| @@ -0,0 +1,10 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0-only */ | |
| 2 | +#ifndef _ASM_ARM64_RING_BUFFER_H | |
| 3 | +#define _ASM_ARM64_RING_BUFFER_H | |
| 4 | + | |
| 5 | +#include <asm/cacheflush.h> | |
| 6 | + | |
| 7 | +/* Flush D-cache on persistent ring buffer */ | |
| 8 | +#define arch_ring_buffer_flush_range(start, end) dcache_clean_pop(start, end) | |
| 9 | + | |
| 10 | +#endif /* _ASM_ARM64_RING_BUFFER_H */ | |
| @@ -844,7 +844,7 @@ · 843 unmodified lines | ||
| 844 | 844 | #define INIT_SCTLR_EL2_MMU_ON \ |
| 845 | 845 | (SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA | SCTLR_ELx_I | \ |
| 846 | 846 | SCTLR_ELx_IESB | SCTLR_ELx_WXN | ENDIAN_SET_EL2 | \ |
| 847 | − SCTLR_ELx_ITFSB | SCTLR_EL2_RES1) | |
| 847 | + SCTLR_ELx_ITFSB | SCTLR_ELx_EIS | SCTLR_ELx_EOS | SCTLR_EL2_RES1) | |
| 848 | 848 | |
| 849 | 849 | #define INIT_SCTLR_EL2_MMU_OFF \ |
| 850 | 850 | (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) |
| @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb) · 52 unmodified lines | ||
| 53 | 53 | static inline void tlb_flush(struct mmu_gather *tlb) |
| 54 | 54 | { |
| 55 | 55 | struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0); |
| 56 | − tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE; | |
| 56 | + tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ? | |
| 57 | + TLBF_NONE : TLBF_NOWALKCACHE; | |
| 57 | 58 | unsigned long stride = tlb_get_unmap_size(tlb); |
| 58 | 59 | int tlb_level = tlb_get_level(tlb); |
| 59 | 60 | |
| @@ -62,6 +62,13 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs, · 61 unmodified lines | ||
| 62 | 62 | irqentry_exit_to_kernel_mode_after_preempt(regs, state); |
| 63 | 63 | } |
| 64 | 64 | |
| 65 | +static __always_inline void arm64_syscall_enter_from_user_mode(struct pt_regs *regs) | |
| 66 | +{ | |
| 67 | + enter_from_user_mode(regs); | |
| 68 | + mte_disable_tco_entry(current); | |
| 69 | + sme_enter_from_user_mode(); | |
| 70 | +} | |
| 71 | + | |
| 65 | 72 | /* |
| 66 | 73 | * Handle IRQ/context state management when entering from user mode. |
| 67 | 74 | * Before this function is called it is not safe to call regular kernel code, |
| @@ -70,20 +77,30 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs, · 2 unmodified lines | ||
| 70 | 77 | static __always_inline void arm64_enter_from_user_mode(struct pt_regs *regs) |
| 71 | 78 | { |
| 72 | 79 | enter_from_user_mode(regs); |
| 80 | + rseq_note_user_irq_entry(); | |
| 73 | 81 | mte_disable_tco_entry(current); |
| 74 | 82 | sme_enter_from_user_mode(); |
| 75 | 83 | } |
| 76 | 84 | |
| 85 | +static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs) | |
| 86 | +{ | |
| 87 | + local_irq_disable(); | |
| 88 | + syscall_exit_to_user_mode_prepare(regs); | |
| 89 | + local_daif_mask(); | |
| 90 | + sme_exit_to_user_mode(); | |
| 91 | + mte_check_tfsr_exit(); | |
| 92 | + exit_to_user_mode(); | |
| 93 | +} | |
| 94 | + | |
| 77 | 95 | /* |
| 78 | 96 | * Handle IRQ/context state management when exiting to user mode. |
| 79 | 97 | * After this function returns it is not safe to call regular kernel code, |
| 80 | 98 | * instrumentable code, or any code which may trigger an exception. |
| 81 | 99 | */ |
| 82 | − | |
| 83 | 100 | static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs) |
| 84 | 101 | { |
| 85 | 102 | local_irq_disable(); |
| 86 | − exit_to_user_mode_prepare_legacy(regs); | |
| 103 | + irqentry_exit_to_user_mode_prepare(regs); | |
| 87 | 104 | local_daif_mask(); |
| 88 | 105 | sme_exit_to_user_mode(); |
| 89 | 106 | mte_check_tfsr_exit(); |
| @@ -92,7 +109,7 @@ static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs) · 2 unmodified lines | ||
| 92 | 109 | |
| 93 | 110 | asmlinkage void noinstr asm_exit_to_user_mode(struct pt_regs *regs) |
| 94 | 111 | { |
| 95 | − arm64_exit_to_user_mode(regs); | |
| 112 | + arm64_syscall_exit_to_user_mode(regs); | |
| 96 | 113 | } |
| 97 | 114 | |
| 98 | 115 | /* |
| @@ -716,12 +733,12 @@ static void noinstr el0_brk64(struct pt_regs *regs, unsigned long esr) · 617 unmodified lines | ||
| 716 | 733 | |
| 717 | 734 | static void noinstr el0_svc(struct pt_regs *regs) |
| 718 | 735 | { |
| 719 | − arm64_enter_from_user_mode(regs); | |
| 736 | + arm64_syscall_enter_from_user_mode(regs); | |
| 720 | 737 | cortex_a76_erratum_1463225_svc_handler(); |
| 721 | 738 | fpsimd_syscall_enter(); |
| 722 | 739 | local_daif_restore(DAIF_PROCCTX); |
| 723 | 740 | do_el0_svc(regs); |
| 724 | − arm64_exit_to_user_mode(regs); | |
| 741 | + arm64_syscall_exit_to_user_mode(regs); | |
| 725 | 742 | fpsimd_syscall_exit(); |
| 726 | 743 | } |
| 727 | 744 | |
| @@ -868,11 +885,11 @@ static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr) · 140 unmodified lines | ||
| 868 | 885 | |
| 869 | 886 | static void noinstr el0_svc_compat(struct pt_regs *regs) |
| 870 | 887 | { |
| 871 | − arm64_enter_from_user_mode(regs); | |
| 888 | + arm64_syscall_enter_from_user_mode(regs); | |
| 872 | 889 | cortex_a76_erratum_1463225_svc_handler(); |
| 873 | 890 | local_daif_restore(DAIF_PROCCTX); |
| 874 | 891 | do_el0_svc_compat(regs); |
| 875 | − arm64_exit_to_user_mode(regs); | |
| 892 | + arm64_syscall_exit_to_user_mode(regs); | |
| 876 | 893 | } |
| 877 | 894 | |
| 878 | 895 | static void noinstr el0_bkpt32(struct pt_regs *regs, unsigned long esr) |
| @@ -196,9 +196,9 @@ static int scs_handle_fde_frame(const struct eh_frame *frame, · 195 unmodified lines | ||
| 196 | 196 | loc += *opcode++ * code_alignment_factor; |
| 197 | 197 | loc += (*opcode++ << 8) * code_alignment_factor; |
| 198 | 198 | loc += (*opcode++ << 16) * code_alignment_factor; |
| 199 | − loc += (*opcode++ << 24) * code_alignment_factor; | |
| 199 | + loc += ((u64)*opcode++ << 24) * code_alignment_factor; | |
| 200 | 200 | size -= 4; |
| 201 | − break; | |
| 201 | + break; | |
| 202 | 202 | |
| 203 | 203 | case DW_CFA_def_cfa: |
| 204 | 204 | case DW_CFA_offset_extended: |
| @@ -983,8 +983,8 @@ static int sve_set_common(struct task_struct *target, · 982 unmodified lines | ||
| 983 | 983 | } |
| 984 | 984 | |
| 985 | 985 | /* Always zero V regs, FPSR, and FPCR */ |
| 986 | − memset(¤t->thread.uw.fpsimd_state, 0, | |
| 987 | − sizeof(current->thread.uw.fpsimd_state)); | |
| 986 | + memset(&target->thread.uw.fpsimd_state, 0, | |
| 987 | + sizeof(target->thread.uw.fpsimd_state)); | |
| 988 | 988 | |
| 989 | 989 | /* Registers: FPSIMD-only case */ |
| 990 | 990 | |
| @@ -67,6 +67,9 @@ struct rt_sigframe_user_layout { · 66 unmodified lines | ||
| 67 | 67 | unsigned long end_offset; |
| 68 | 68 | }; |
| 69 | 69 | |
| 70 | +#define TERMINATOR_SIZE round_up(sizeof(struct _aarch64_ctx), 16) | |
| 71 | +#define EXTRA_CONTEXT_SIZE round_up(sizeof(struct extra_context), 16) | |
| 72 | + | |
| 70 | 73 | /* |
| 71 | 74 | * Holds any EL0-controlled state that influences unprivileged memory accesses. |
| 72 | 75 | * This includes both accesses done in userspace and uaccess done in the kernel. |
| @@ -74,13 +77,35 @@ struct rt_sigframe_user_layout { · 1 unmodified line | ||
| 74 | 77 | * This state needs to be carefully managed to ensure that it doesn't cause |
| 75 | 78 | * uaccess to fail when setting up the signal frame, and the signal handler |
| 76 | 79 | * itself also expects a well-defined state when entered. |
| 80 | + * | |
| 81 | + * The struct should be zero-initialised. Its members should only be accessed | |
| 82 | + * via the accessors below. __valid_fields tracks which of the fields are valid | |
| 83 | + * (have been set to some value). | |
| 77 | 84 | */ |
| 78 | 85 | struct user_access_state { |
| 79 | − u64 por_el0; | |
| 86 | + unsigned int __valid_fields; | |
| 87 | + u64 __por_el0; | |
| 80 | 88 | }; |
| 81 | 89 | |
| 82 | −#define TERMINATOR_SIZE round_up(sizeof(struct _aarch64_ctx), 16) | |
| 83 | −#define EXTRA_CONTEXT_SIZE round_up(sizeof(struct extra_context), 16) | |
| 90 | +#define UA_STATE_HAS_POR_EL0 BIT(0) | |
| 91 | + | |
| 92 | +static void set_ua_state_por_el0(struct user_access_state *ua_state, | |
| 93 | + u64 por_el0) | |
| 94 | +{ | |
| 95 | + ua_state->__por_el0 = por_el0; | |
| 96 | + ua_state->__valid_fields |= UA_STATE_HAS_POR_EL0; | |
| 97 | +} | |
| 98 | + | |
| 99 | +static int get_ua_state_por_el0(const struct user_access_state *ua_state, | |
| 100 | + u64 *por_el0) | |
| 101 | +{ | |
| 102 | + if (ua_state->__valid_fields & UA_STATE_HAS_POR_EL0) { | |
| 103 | + *por_el0 = ua_state->__por_el0; | |
| 104 | + return 0; | |
| 105 | + } | |
| 106 | + | |
| 107 | + return -ENOENT; | |
| 108 | +} | |
| 84 | 109 | |
| 85 | 110 | /* |
| 86 | 111 | * Save the user access state into ua_state and reset it to disable any |
| @@ -94,7 +119,7 @@ static void save_reset_user_access_state(struct user_access_state *ua_state) · 7 unmodified lines | ||
| 94 | 119 | for (int pkey = 0; pkey < arch_max_pkey(); pkey++) |
| 95 | 120 | por_enable_all |= POR_ELx_PERM_PREP(pkey, POE_RWX); |
| 96 | 121 | |
| 97 | − ua_state->por_el0 = read_sysreg_s(SYS_POR_EL0); | |
| 122 | + set_ua_state_por_el0(ua_state, read_sysreg_s(SYS_POR_EL0)); | |
| 98 | 123 | write_sysreg_s(por_enable_all, SYS_POR_EL0); |
| 99 | 124 | /* |
| 100 | 125 | * No ISB required as we can tolerate spurious Overlay faults - |
| @@ -122,8 +147,10 @@ static void set_handler_user_access_state(void) · 21 unmodified lines | ||
| 122 | 147 | */ |
| 123 | 148 | static void restore_user_access_state(const struct user_access_state *ua_state) |
| 124 | 149 | { |
| 125 | − if (system_supports_poe()) | |
| 126 | − write_sysreg_s(ua_state->por_el0, SYS_POR_EL0); | |
| 150 | + u64 por_el0; | |
| 151 | + | |
| 152 | + if (get_ua_state_por_el0(ua_state, &por_el0) == 0) | |
| 153 | + write_sysreg_s(por_el0, SYS_POR_EL0); | |
| 127 | 154 | } |
| 128 | 155 | |
| 129 | 156 | static void init_user_layout(struct rt_sigframe_user_layout *user) |
| @@ -333,11 +360,16 @@ static int restore_fpmr_context(struct user_ctxs *user) · 203 unmodified lines | ||
| 333 | 360 | static int preserve_poe_context(struct poe_context __user *ctx, |
| 334 | 361 | const struct user_access_state *ua_state) |
| 335 | 362 | { |
| 336 | − int err = 0; | |
| 363 | + int err; | |
| 364 | + u64 por_el0; | |
| 365 | + | |
| 366 | + err = get_ua_state_por_el0(ua_state, &por_el0); | |
| 367 | + if (WARN_ON_ONCE(err)) | |
| 368 | + return err; | |
| 337 | 369 | |
| 338 | 370 | __put_user_error(POE_MAGIC, &ctx->head.magic, err); |
| 339 | 371 | __put_user_error(sizeof(*ctx), &ctx->head.size, err); |
| 340 | − __put_user_error(ua_state->por_el0, &ctx->por_el0, err); | |
| 372 | + __put_user_error(por_el0, &ctx->por_el0, err); | |
| 341 | 373 | |
| 342 | 374 | return err; |
| 343 | 375 | } |
| @@ -353,7 +385,7 @@ static int restore_poe_context(struct user_ctxs *user, · 9 unmodified lines | ||
| 353 | 385 | |
| 354 | 386 | __get_user_error(por_el0, &(user->poe->por_el0), err); |
| 355 | 387 | if (!err) |
| 356 | − ua_state->por_el0 = por_el0; | |
| 388 | + set_ua_state_por_el0(ua_state, por_el0); | |
| 357 | 389 | |
| 358 | 390 | return err; |
| 359 | 391 | } |
| @@ -1095,7 +1127,7 @@ SYSCALL_DEFINE0(rt_sigreturn) · 735 unmodified lines | ||
| 1095 | 1127 | { |
| 1096 | 1128 | struct pt_regs *regs = current_pt_regs(); |
| 1097 | 1129 | struct rt_sigframe __user *frame; |
| 1098 | − struct user_access_state ua_state; | |
| 1130 | + struct user_access_state ua_state = {}; | |
| 1099 | 1131 | |
| 1100 | 1132 | /* Always make any pending restarted system calls return -EINTR */ |
| 1101 | 1133 | current->restart_block.fn = do_no_restart_syscall; |
| @@ -1507,7 +1539,7 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set, · 405 unmodified lines | ||
| 1507 | 1539 | { |
| 1508 | 1540 | struct rt_sigframe_user_layout user; |
| 1509 | 1541 | struct rt_sigframe __user *frame; |
| 1510 | − struct user_access_state ua_state; | |
| 1542 | + struct user_access_state ua_state = {}; | |
| 1511 | 1543 | int err = 0; |
| 1512 | 1544 | |
| 1513 | 1545 | fpsimd_save_and_flush_current_state(); |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | * Author: Christoffer Dall <c.dall@virtualopensystems.com> |
| 5 | 5 | */ |
| 6 | 6 | |
| 7 | +#include <linux/arm-smccc.h> | |
| 7 | 8 | #include <linux/bug.h> |
| 8 | 9 | #include <linux/cpu_pm.h> |
| 9 | 10 | #include <linux/errno.h> |
| @@ -554,8 +555,10 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 544 unmodified lines | ||
| 554 | 555 | kvm_destroy_mpidr_data(vcpu->kvm); |
| 555 | 556 | |
| 556 | 557 | err = kvm_vgic_vcpu_init(vcpu); |
| 557 | − if (err) | |
| 558 | + if (err) { | |
| 559 | + kvm_vgic_vcpu_destroy(vcpu); | |
| 558 | 560 | return err; |
| 561 | + } | |
| 559 | 562 | |
| 560 | 563 | err = kvm_share_hyp(vcpu, vcpu + 1); |
| 561 | 564 | if (err) |
| @@ -824,6 +827,10 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) · 262 unmodified lines | ||
| 824 | 827 | { |
| 825 | 828 | bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF | HCR_VSE); |
| 826 | 829 | |
| 830 | + irq_lines |= (!irqchip_in_kernel(v->kvm) && | |
| 831 | + (kvm_timer_should_notify_user(v) || | |
| 832 | + kvm_pmu_should_notify_user(v))); | |
| 833 | + | |
| 827 | 834 | return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) |
| 828 | 835 | && !kvm_arm_vcpu_stopped(v) && !v->arch.pause); |
| 829 | 836 | } |
| @@ -2634,6 +2641,22 @@ static int init_pkvm_host_sve_state(void) · 1804 unmodified lines | ||
| 2634 | 2641 | return 0; |
| 2635 | 2642 | } |
| 2636 | 2643 | |
| 2644 | +static int pkvm_check_sme_dvmsync_fw_call(void) | |
| 2645 | +{ | |
| 2646 | + struct arm_smccc_res res; | |
| 2647 | + | |
| 2648 | + if (!cpus_have_final_cap(ARM64_WORKAROUND_4193714)) | |
| 2649 | + return 0; | |
| 2650 | + | |
| 2651 | + arm_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res); | |
| 2652 | + if (res.a0) { | |
| 2653 | + kvm_err("pKVM requires firmware support for C1-Pro erratum 4193714\n"); | |
| 2654 | + return -ENODEV; | |
| 2655 | + } | |
| 2656 | + | |
| 2657 | + return 0; | |
| 2658 | +} | |
| 2659 | + | |
| 2637 | 2660 | /* |
| 2638 | 2661 | * Finalizes the initialization of hyp mode, once everything else is initialized |
| 2639 | 2662 | * and the initialziation process cannot fail. |
| @@ -2834,6 +2857,10 @@ static int __init init_hyp_mode(void) · 194 unmodified lines | ||
| 2834 | 2857 | if (err) |
| 2835 | 2858 | goto out_err; |
| 2836 | 2859 | |
| 2860 | + err = pkvm_check_sme_dvmsync_fw_call(); | |
| 2861 | + if (err) | |
| 2862 | + goto out_err; | |
| 2863 | + | |
| 2837 | 2864 | err = kvm_hyp_init_protection(hyp_va_bits); |
| 2838 | 2865 | if (err) { |
| 2839 | 2866 | kvm_err("Failed to init hyp memory protection\n"); |
| @@ -131,7 +131,6 @@ struct reg_feat_map_desc { · 130 unmodified lines | ||
| 131 | 131 | } |
| 132 | 132 | |
| 133 | 133 | #define FEAT_SPE ID_AA64DFR0_EL1, PMSVer, IMP |
| 134 | −#define FEAT_SPE_FnE ID_AA64DFR0_EL1, PMSVer, V1P2 | |
| 135 | 134 | #define FEAT_BRBE ID_AA64DFR0_EL1, BRBE, IMP |
| 136 | 135 | #define FEAT_TRC_SR ID_AA64DFR0_EL1, TraceVer, IMP |
| 137 | 136 | #define FEAT_PMUv3 ID_AA64DFR0_EL1, PMUVer, IMP |
| @@ -192,7 +191,7 @@ struct reg_feat_map_desc { · 54 unmodified lines | ||
| 192 | 191 | #define FEAT_SRMASK ID_AA64MMFR4_EL1, SRMASK, IMP |
| 193 | 192 | #define FEAT_PoPS ID_AA64MMFR4_EL1, PoPS, IMP |
| 194 | 193 | #define FEAT_PFAR ID_AA64PFR1_EL1, PFAR, IMP |
| 195 | −#define FEAT_Debugv8p9 ID_AA64DFR0_EL1, PMUVer, V3P9 | |
| 194 | +#define FEAT_Debugv8p9 ID_AA64DFR0_EL1, DebugVer, V8P9 | |
| 196 | 195 | #define FEAT_PMUv3_SS ID_AA64DFR0_EL1, PMSS, IMP |
| 197 | 196 | #define FEAT_SEBEP ID_AA64DFR0_EL1, SEBEP, IMP |
| 198 | 197 | #define FEAT_EBEP ID_AA64DFR1_EL1, EBEP, IMP |
| @@ -283,7 +282,7 @@ static bool feat_anerr(struct kvm *kvm) · 84 unmodified lines | ||
| 283 | 282 | static bool feat_sme_smps(struct kvm *kvm) |
| 284 | 283 | { |
| 285 | 284 | /* |
| 286 | − * Revists this if KVM ever supports SME -- this really should | |
| 285 | + * Revisit this if KVM ever supports SME -- this really should | |
| 287 | 286 | * look at the guest's view of SMIDR_EL1. Funnily enough, this |
| 288 | 287 | * is not captured in the JSON file, but only as a note in the |
| 289 | 288 | * ARM ARM. |
| @@ -295,17 +294,27 @@ static bool feat_sme_smps(struct kvm *kvm) · 5 unmodified lines | ||
| 295 | 294 | static bool feat_spe_fds(struct kvm *kvm) |
| 296 | 295 | { |
| 297 | 296 | /* |
| 298 | − * Revists this if KVM ever supports SPE -- this really should | |
| 297 | + * Revisit this if KVM ever supports SPE -- this really should | |
| 299 | 298 | * look at the guest's view of PMSIDR_EL1. |
| 300 | 299 | */ |
| 301 | 300 | return (kvm_has_feat(kvm, FEAT_SPEv1p4) && |
| 302 | 301 | (read_sysreg_s(SYS_PMSIDR_EL1) & PMSIDR_EL1_FDS)); |
| 303 | 302 | } |
| 304 | 303 | |
| 304 | +static bool feat_spe_fne(struct kvm *kvm) | |
| 305 | +{ | |
| 306 | + /* | |
| 307 | + * Revisit this if KVM ever supports SPE -- this really should | |
| 308 | + * look at the guest's view of PMSIDR_EL1. | |
| 309 | + */ | |
| 310 | + return (kvm_has_feat(kvm, FEAT_SPEv1p2) && | |
| 311 | + (read_sysreg_s(SYS_PMSIDR_EL1) & PMSIDR_EL1_FnE)); | |
| 312 | +} | |
| 313 | + | |
| 305 | 314 | static bool feat_trbe_mpam(struct kvm *kvm) |
| 306 | 315 | { |
| 307 | 316 | /* |
| 308 | − * Revists this if KVM ever supports both MPAM and TRBE -- | |
| 317 | + * Revisit this if KVM ever supports both MPAM and TRBE -- | |
| 309 | 318 | * this really should look at the guest's view of TRBIDR_EL1. |
| 310 | 319 | */ |
| 311 | 320 | return (kvm_has_feat(kvm, FEAT_TRBE) && |
| @@ -537,7 +546,7 @@ static const struct reg_bits_to_feat_map hdfgrtr_feat_map[] = { · 225 unmodified lines | ||
| 537 | 546 | HDFGRTR_EL2_PMBPTR_EL1 | |
| 538 | 547 | HDFGRTR_EL2_PMBLIMITR_EL1, |
| 539 | 548 | FEAT_SPE), |
| 540 | − NEEDS_FEAT(HDFGRTR_EL2_nPMSNEVFR_EL1, FEAT_SPE_FnE), | |
| 549 | + NEEDS_FEAT(HDFGRTR_EL2_nPMSNEVFR_EL1, feat_spe_fne), | |
| 541 | 550 | NEEDS_FEAT(HDFGRTR_EL2_nBRBDATA | |
| 542 | 551 | HDFGRTR_EL2_nBRBCTL | |
| 543 | 552 | HDFGRTR_EL2_nBRBIDR, |
| @@ -605,7 +614,7 @@ static const struct reg_bits_to_feat_map hdfgwtr_feat_map[] = { · 61 unmodified lines | ||
| 605 | 614 | HDFGWTR_EL2_PMBPTR_EL1 | |
| 606 | 615 | HDFGWTR_EL2_PMBLIMITR_EL1, |
| 607 | 616 | FEAT_SPE), |
| 608 | − NEEDS_FEAT(HDFGWTR_EL2_nPMSNEVFR_EL1, FEAT_SPE_FnE), | |
| 617 | + NEEDS_FEAT(HDFGWTR_EL2_nPMSNEVFR_EL1, feat_spe_fne), | |
| 609 | 618 | NEEDS_FEAT(HDFGWTR_EL2_nBRBDATA | |
| 610 | 619 | HDFGWTR_EL2_nBRBCTL, |
| 611 | 620 | FEAT_BRBE), |
| @@ -245,7 +245,7 @@ static inline void __activate_traps_ich_hfgxtr(struct kvm_vcpu *vcpu) · 244 unmodified lines | ||
| 245 | 245 | __activate_fgt(hctxt, vcpu, ICH_HFGITR_EL2); |
| 246 | 246 | } |
| 247 | 247 | |
| 248 | −#define __deactivate_fgt(htcxt, vcpu, reg) \ | |
| 248 | +#define __deactivate_fgt(hctxt, vcpu, reg) \ | |
| 249 | 249 | do { \ |
| 250 | 250 | write_sysreg_s(ctxt_sys_reg(hctxt, reg), \ |
| 251 | 251 | SYS_ ## reg); \ |
| @@ -462,11 +462,13 @@ static inline bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code) · 210 unmodified lines | ||
| 462 | 462 | |
| 463 | 463 | static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) |
| 464 | 464 | { |
| 465 | + u64 zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | |
| 466 | + | |
| 465 | 467 | /* |
| 466 | 468 | * The vCPU's saved SVE state layout always matches the max VL of the |
| 467 | 469 | * vCPU. Start off with the max VL so we can load the SVE state. |
| 468 | 470 | */ |
| 469 | − sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); | |
| 471 | + sve_cond_update_zcr_vq(zcr_el2, SYS_ZCR_EL2); | |
| 470 | 472 | __sve_restore_state(vcpu_sve_pffr(vcpu), |
| 471 | 473 | &vcpu->arch.ctxt.fp_regs.fpsr, |
| 472 | 474 | true); |
| @@ -476,8 +478,10 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) · 3 unmodified lines | ||
| 476 | 478 | * nested guest, as the guest hypervisor could select a smaller VL. Slap |
| 477 | 479 | * that into hardware before wrapping up. |
| 478 | 480 | */ |
| 479 | − if (is_nested_ctxt(vcpu)) | |
| 480 | − sve_cond_update_zcr_vq(__vcpu_sys_reg(vcpu, ZCR_EL2), SYS_ZCR_EL2); | |
| 481 | + if (is_nested_ctxt(vcpu)) { | |
| 482 | + zcr_el2 = min(zcr_el2, __vcpu_sys_reg(vcpu, ZCR_EL2)); | |
| 483 | + sve_cond_update_zcr_vq(zcr_el2, SYS_ZCR_EL2); | |
| 484 | + } | |
| 481 | 485 | |
| 482 | 486 | write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); |
| 483 | 487 | } |
| @@ -501,11 +505,11 @@ static inline void fpsimd_lazy_switch_to_guest(struct kvm_vcpu *vcpu) · 17 unmodified lines | ||
| 501 | 505 | return; |
| 502 | 506 | |
| 503 | 507 | if (vcpu_has_sve(vcpu)) { |
| 508 | + zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | |
| 509 | + | |
| 504 | 510 | /* A guest hypervisor may restrict the effective max VL. */ |
| 505 | 511 | if (is_nested_ctxt(vcpu)) |
| 506 | − zcr_el2 = __vcpu_sys_reg(vcpu, ZCR_EL2); | |
| 507 | − else | |
| 508 | − zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; | |
| 512 | + zcr_el2 = min(zcr_el2, __vcpu_sys_reg(vcpu, ZCR_EL2)); | |
| 509 | 513 | |
| 510 | 514 | write_sysreg_el2(zcr_el2, SYS_ZCR); |
| 511 | 515 | |
| @@ -35,6 +35,9 @@ void trace_clock_update(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cyc) · 34 unmodified lines | ||
| 35 | 35 | struct clock_data *clock = &trace_clock_data; |
| 36 | 36 | u64 bank = clock->cur ^ 1; |
| 37 | 37 | |
| 38 | + if (!mult || shift >= 64) | |
| 39 | + return; | |
| 40 | + | |
| 38 | 41 | clock->data[bank].mult = mult; |
| 39 | 42 | clock->data[bank].shift = shift; |
| 40 | 43 | clock->data[bank].epoch_ns = epoch_ns; |
| @@ -120,7 +120,7 @@ SYM_FUNC_START(__hyp_do_panic) · 119 unmodified lines | ||
| 120 | 120 | |
| 121 | 121 | mov x29, x0 |
| 122 | 122 | |
| 123 | −#ifdef PKVM_DISABLE_STAGE2_ON_PANIC | |
| 123 | +#ifdef CONFIG_PKVM_DISABLE_STAGE2_ON_PANIC | |
| 124 | 124 | /* Ensure host stage-2 is disabled */ |
| 125 | 125 | mrs x0, hcr_el2 |
| 126 | 126 | bic x0, x0, #HCR_VM |
| @@ -709,6 +709,14 @@ static const hcall_t host_hcall[] = { · 708 unmodified lines | ||
| 709 | 709 | HANDLE_FUNC(__kvm_tlb_flush_vmid_range), |
| 710 | 710 | HANDLE_FUNC(__kvm_flush_cpu_context), |
| 711 | 711 | HANDLE_FUNC(__kvm_timer_set_cntvoff), |
| 712 | + HANDLE_FUNC(__tracing_load), | |
| 713 | + HANDLE_FUNC(__tracing_unload), | |
| 714 | + HANDLE_FUNC(__tracing_enable), | |
| 715 | + HANDLE_FUNC(__tracing_swap_reader), | |
| 716 | + HANDLE_FUNC(__tracing_update_clock), | |
| 717 | + HANDLE_FUNC(__tracing_reset), | |
| 718 | + HANDLE_FUNC(__tracing_enable_event), | |
| 719 | + HANDLE_FUNC(__tracing_write_event), | |
| 712 | 720 | HANDLE_FUNC(__vgic_v3_save_aprs), |
| 713 | 721 | HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs), |
| 714 | 722 | HANDLE_FUNC(__vgic_v5_save_apr), |
| @@ -735,22 +743,16 @@ static const hcall_t host_hcall[] = { · 20 unmodified lines | ||
| 735 | 743 | HANDLE_FUNC(__pkvm_vcpu_load), |
| 736 | 744 | HANDLE_FUNC(__pkvm_vcpu_put), |
| 737 | 745 | HANDLE_FUNC(__pkvm_tlb_flush_vmid), |
| 738 | − HANDLE_FUNC(__tracing_load), | |
| 739 | − HANDLE_FUNC(__tracing_unload), | |
| 740 | − HANDLE_FUNC(__tracing_enable), | |
| 741 | − HANDLE_FUNC(__tracing_swap_reader), | |
| 742 | − HANDLE_FUNC(__tracing_update_clock), | |
| 743 | − HANDLE_FUNC(__tracing_reset), | |
| 744 | − HANDLE_FUNC(__tracing_enable_event), | |
| 745 | − HANDLE_FUNC(__tracing_write_event), | |
| 746 | 746 | }; |
| 747 | 747 | |
| 748 | 748 | static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) |
| 749 | 749 | { |
| 750 | 750 | DECLARE_REG(unsigned long, id, host_ctxt, 0); |
| 751 | − unsigned long hcall_min = 0, hcall_max = -1; | |
| 751 | + unsigned long hcall_min = 0, hcall_max = __KVM_HOST_SMCCC_FUNC_MAX; | |
| 752 | 752 | hcall_t hfn; |
| 753 | 753 | |
| 754 | + BUILD_BUG_ON(ARRAY_SIZE(host_hcall) != __KVM_HOST_SMCCC_FUNC_MAX); | |
| 755 | + | |
| 754 | 756 | /* |
| 755 | 757 | * If pKVM has been initialised then reject any calls to the |
| 756 | 758 | * early "privileged" hypercalls. Note that we cannot reject |
| @@ -763,16 +765,14 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) · 6 unmodified lines | ||
| 763 | 765 | if (static_branch_unlikely(&kvm_protected_mode_initialized)) { |
| 764 | 766 | hcall_min = __KVM_HOST_SMCCC_FUNC_MIN_PKVM; |
| 765 | 767 | } else { |
| 766 | − hcall_max = __KVM_HOST_SMCCC_FUNC_MAX_NO_PKVM; | |
| 768 | + hcall_max = __KVM_HOST_SMCCC_FUNC_PKVM_ONLY; | |
| 767 | 769 | } |
| 768 | 770 | |
| 769 | 771 | id &= ~ARM_SMCCC_CALL_HINTS; |
| 770 | 772 | id -= KVM_HOST_SMCCC_ID(0); |
| 771 | 773 | |
| 772 | − if (unlikely(id < hcall_min || id > hcall_max || | |
| 773 | − id >= ARRAY_SIZE(host_hcall))) { | |
| 774 | + if (unlikely(id < hcall_min || id >= hcall_max)) | |
| 774 | 775 | goto inval; |
| 775 | − } | |
| 776 | 776 | |
| 777 | 777 | hfn = host_hcall[id]; |
| 778 | 778 | if (unlikely(!hfn)) |
| @@ -805,6 +805,10 @@ static void handle_host_smc(struct kvm_cpu_context *host_ctxt) · 26 unmodified lines | ||
| 805 | 805 | } |
| 806 | 806 | |
| 807 | 807 | func_id &= ~ARM_SMCCC_CALL_HINTS; |
| 808 | + if (upper_32_bits(func_id)) { | |
| 809 | + cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED; | |
| 810 | + goto exit_skip_instr; | |
| 811 | + } | |
| 808 | 812 | |
| 809 | 813 | handled = kvm_host_psci_handler(host_ctxt, func_id); |
| 810 | 814 | if (!handled) |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | ||
| 5 | 5 | */ |
| 6 | 6 | |
| 7 | 7 | #include <linux/kvm_host.h> |
| 8 | + | |
| 8 | 9 | #include <asm/kvm_emulate.h> |
| 9 | 10 | #include <asm/kvm_hyp.h> |
| 10 | 11 | #include <asm/kvm_mmu.h> |
| @@ -14,6 +15,7 @@ · 3 unmodified lines | ||
| 14 | 15 | |
| 15 | 16 | #include <hyp/fault.h> |
| 16 | 17 | |
| 18 | +#include <nvhe/arm-smccc.h> | |
| 17 | 19 | #include <nvhe/gfp.h> |
| 18 | 20 | #include <nvhe/memory.h> |
| 19 | 21 | #include <nvhe/mem_protect.h> |
| @@ -29,6 +31,19 @@ static struct hyp_pool host_s2_pool; · 9 unmodified lines | ||
| 29 | 31 | static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm); |
| 30 | 32 | #define current_vm (*this_cpu_ptr(&__current_vm)) |
| 31 | 33 | |
| 34 | +static void pkvm_sme_dvmsync_fw_call(void) | |
| 35 | +{ | |
| 36 | + if (alternative_has_cap_unlikely(ARM64_WORKAROUND_4193714)) { | |
| 37 | + struct arm_smccc_res res; | |
| 38 | + | |
| 39 | + /* | |
| 40 | + * Ignore the return value. Probing for the workaround | |
| 41 | + * availability took place in init_hyp_mode(). | |
| 42 | + */ | |
| 43 | + hyp_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res); | |
| 44 | + } | |
| 45 | +} | |
| 46 | + | |
| 32 | 47 | static void guest_lock_component(struct pkvm_hyp_vm *vm) |
| 33 | 48 | { |
| 34 | 49 | hyp_spin_lock(&vm->lock); |
| @@ -574,8 +589,14 @@ static int host_stage2_set_owner_metadata_locked(phys_addr_t addr, u64 size, · 539 unmodified lines | ||
| 574 | 589 | ret = host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, |
| 575 | 590 | addr, size, &host_s2_pool, |
| 576 | 591 | KVM_HOST_INVALID_PTE_TYPE_DONATION, annotation); |
| 577 | − if (!ret) | |
| 592 | + if (!ret) { | |
| 593 | + /* | |
| 594 | + * After stage2 maintenance has happened, but before the page | |
| 595 | + * owner has changed. | |
| 596 | + */ | |
| 597 | + pkvm_sme_dvmsync_fw_call(); | |
| 578 | 598 | __host_update_page_state(addr, size, PKVM_NOPAGE); |
| 599 | + } | |
| 579 | 600 | |
| 580 | 601 | return ret; |
| 581 | 602 | } |
| @@ -1369,6 +1390,22 @@ unlock: · 787 unmodified lines | ||
| 1369 | 1390 | return ret && ret != -EHWPOISON ? ret : 0; |
| 1370 | 1391 | } |
| 1371 | 1392 | |
| 1393 | +/* | |
| 1394 | + * share/donate install at most one stage-2 leaf (PAGE_SIZE, or one | |
| 1395 | + * KVM_PGTABLE_LAST_LEVEL - 1 block for share). kvm_mmu_cache_min_pages() | |
| 1396 | + * bounds the worst-case allocation: exact for the PAGE_SIZE leaf, | |
| 1397 | + * conservative by one for the block. | |
| 1398 | + */ | |
| 1399 | +static int __guest_check_pgtable_memcache(struct pkvm_hyp_vcpu *vcpu) | |
| 1400 | +{ | |
| 1401 | + struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu); | |
| 1402 | + | |
| 1403 | + if (vcpu->vcpu.arch.pkvm_memcache.nr_pages < kvm_mmu_cache_min_pages(vm->pgt.mmu)) | |
| 1404 | + return -ENOMEM; | |
| 1405 | + | |
| 1406 | + return 0; | |
| 1407 | +} | |
| 1408 | + | |
| 1372 | 1409 | int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu) |
| 1373 | 1410 | { |
| 1374 | 1411 | struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu); |
| @@ -1388,6 +1425,10 @@ int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu) · 13 unmodified lines | ||
| 1388 | 1425 | if (ret) |
| 1389 | 1426 | goto unlock; |
| 1390 | 1427 | |
| 1428 | + ret = __guest_check_pgtable_memcache(vcpu); | |
| 1429 | + if (ret) | |
| 1430 | + goto unlock; | |
| 1431 | + | |
| 1391 | 1432 | meta = host_stage2_encode_gfn_meta(vm, gfn); |
| 1392 | 1433 | WARN_ON(host_stage2_set_owner_metadata_locked(phys, PAGE_SIZE, |
| 1393 | 1434 | PKVM_ID_GUEST, meta)); |
| @@ -1453,6 +1494,10 @@ int __pkvm_host_share_guest(u64 pfn, u64 gfn, u64 nr_pages, struct pkvm_hyp_vcpu · 59 unmodified lines | ||
| 1453 | 1494 | } |
| 1454 | 1495 | } |
| 1455 | 1496 | |
| 1497 | + ret = __guest_check_pgtable_memcache(vcpu); | |
| 1498 | + if (ret) | |
| 1499 | + goto unlock; | |
| 1500 | + | |
| 1456 | 1501 | for_each_hyp_page(page, phys, size) { |
| 1457 | 1502 | set_host_state(page, PKVM_PAGE_SHARED_OWNED); |
| 1458 | 1503 | page->host_share_guest_count++; |
| @@ -266,7 +266,8 @@ struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle, · 265 unmodified lines | ||
| 266 | 266 | if (hyp_vm->kvm.created_vcpus <= vcpu_idx) |
| 267 | 267 | goto unlock; |
| 268 | 268 | |
| 269 | − hyp_vcpu = hyp_vm->vcpus[vcpu_idx]; | |
| 269 | + /* Pairs with smp_store_release() in register_hyp_vcpu(). */ | |
| 270 | + hyp_vcpu = smp_load_acquire(&hyp_vm->vcpus[vcpu_idx]); | |
| 270 | 271 | if (!hyp_vcpu) |
| 271 | 272 | goto unlock; |
| 272 | 273 | |
| @@ -751,16 +752,30 @@ static struct pkvm_hyp_vcpu selftest_vcpu = { · 478 unmodified lines | ||
| 751 | 752 | struct pkvm_hyp_vcpu *init_selftest_vm(void *virt) |
| 752 | 753 | { |
| 753 | 754 | struct hyp_page *p = hyp_virt_to_page(virt); |
| 755 | + unsigned long min_pages, seeded = 0; | |
| 754 | 756 | int i; |
| 755 | 757 | |
| 756 | 758 | selftest_vm.kvm.arch.mmu.vtcr = host_mmu.arch.mmu.vtcr; |
| 757 | 759 | WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, virt)); |
| 758 | 760 | |
| 761 | + /* | |
| 762 | + * Mirror pkvm_refill_memcache() for the share/donate pre-checks; | |
| 763 | + * the selftest invokes those functions directly and would | |
| 764 | + * otherwise see an empty memcache. | |
| 765 | + */ | |
| 766 | + min_pages = kvm_mmu_cache_min_pages(&selftest_vm.kvm.arch.mmu); | |
| 767 | + | |
| 759 | 768 | for (i = 0; i < pkvm_selftest_pages(); i++) { |
| 760 | 769 | if (p[i].refcount) |
| 761 | 770 | continue; |
| 762 | 771 | p[i].refcount = 1; |
| 763 | − hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i])); | |
| 772 | + if (seeded < min_pages) { | |
| 773 | + push_hyp_memcache(&selftest_vcpu.vcpu.arch.pkvm_memcache, | |
| 774 | + hyp_page_to_virt(&p[i]), hyp_virt_to_phys); | |
| 775 | + seeded++; | |
| 776 | + } else { | |
| 777 | + hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i])); | |
| 778 | + } | |
| 764 | 779 | } |
| 765 | 780 | |
| 766 | 781 | selftest_vm.kvm.arch.pkvm.handle = __pkvm_reserve_vm(); |
| @@ -860,12 +875,30 @@ err_unpin_kvm: · 93 unmodified lines | ||
| 860 | 875 | * the page-aligned size of 'struct pkvm_hyp_vcpu'. |
| 861 | 876 | * Return 0 on success, negative error code on failure. |
| 862 | 877 | */ |
| 878 | +static int register_hyp_vcpu(struct pkvm_hyp_vm *hyp_vm, | |
| 879 | + struct pkvm_hyp_vcpu *hyp_vcpu) | |
| 880 | +{ | |
| 881 | + unsigned int idx = hyp_vcpu->vcpu.vcpu_idx; | |
| 882 | + | |
| 883 | + if (idx >= hyp_vm->kvm.created_vcpus) | |
| 884 | + return -EINVAL; | |
| 885 | + | |
| 886 | + if (hyp_vm->vcpus[idx]) | |
| 887 | + return -EINVAL; | |
| 888 | + | |
| 889 | + /* | |
| 890 | + * Ensure the hyp_vcpu is initialised before publishing it to | |
| 891 | + * the vCPU-load path via 'hyp_vm->vcpus[]'. | |
| 892 | + */ | |
| 893 | + smp_store_release(&hyp_vm->vcpus[idx], hyp_vcpu); | |
| 894 | + return 0; | |
| 895 | +} | |
| 896 | + | |
| 863 | 897 | int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu, |
| 864 | 898 | unsigned long vcpu_hva) |
| 865 | 899 | { |
| 866 | 900 | struct pkvm_hyp_vcpu *hyp_vcpu; |
| 867 | 901 | struct pkvm_hyp_vm *hyp_vm; |
| 868 | − unsigned int idx; | |
| 869 | 902 | int ret; |
| 870 | 903 | |
| 871 | 904 | hyp_vcpu = map_donated_memory(vcpu_hva, sizeof(*hyp_vcpu)); |
| @@ -884,18 +917,11 @@ int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu, · 12 unmodified lines | ||
| 884 | 917 | if (ret) |
| 885 | 918 | goto unlock; |
| 886 | 919 | |
| 887 | − idx = hyp_vcpu->vcpu.vcpu_idx; | |
| 888 | − if (idx >= hyp_vm->kvm.created_vcpus) { | |
| 889 | − ret = -EINVAL; | |
| 890 | − goto unlock; | |
| 891 | − } | |
| 892 | − | |
| 893 | − if (hyp_vm->vcpus[idx]) { | |
| 894 | − ret = -EINVAL; | |
| 895 | − goto unlock; | |
| 920 | + ret = register_hyp_vcpu(hyp_vm, hyp_vcpu); | |
| 921 | + if (ret) { | |
| 922 | + unpin_host_vcpu(host_vcpu); | |
| 923 | + unpin_host_sve_state(hyp_vcpu); | |
| 896 | 924 | } |
| 897 | − | |
| 898 | − hyp_vm->vcpus[idx] = hyp_vcpu; | |
| 899 | 925 | unlock: |
| 900 | 926 | hyp_spin_unlock(&vm_table_lock); |
| 901 | 927 | |
| @@ -312,15 +312,15 @@ void __noreturn __pkvm_init_finalise(void) · 311 unmodified lines | ||
| 312 | 312 | }; |
| 313 | 313 | pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; |
| 314 | 314 | |
| 315 | − ret = fix_host_ownership(); | |
| 315 | + ret = fix_hyp_pgtable_refcnt(); | |
| 316 | 316 | if (ret) |
| 317 | 317 | goto out; |
| 318 | 318 | |
| 319 | − ret = fix_hyp_pgtable_refcnt(); | |
| 319 | + ret = hyp_create_fixmap(); | |
| 320 | 320 | if (ret) |
| 321 | 321 | goto out; |
| 322 | 322 | |
| 323 | − ret = hyp_create_fixmap(); | |
| 323 | + ret = fix_host_ownership(); | |
| 324 | 324 | if (ret) |
| 325 | 325 | goto out; |
| 326 | 326 | |
| @@ -164,13 +164,16 @@ static int hyp_trace_buffer_load(struct hyp_trace_buffer *trace_buffer, · 163 unmodified lines | ||
| 164 | 164 | return ret; |
| 165 | 165 | } |
| 166 | 166 | |
| 167 | −static bool hyp_trace_desc_validate(struct hyp_trace_desc *desc, size_t desc_size) | |
| 167 | +static bool hyp_trace_desc_is_valid(struct hyp_trace_desc *desc, size_t desc_size) | |
| 168 | 168 | { |
| 169 | 169 | struct ring_buffer_desc *rb_desc; |
| 170 | 170 | unsigned int cpu; |
| 171 | 171 | size_t nr_bpages; |
| 172 | 172 | void *desc_end; |
| 173 | 173 | |
| 174 | + if (!is_protected_kvm_enabled()) | |
| 175 | + return true; | |
| 176 | + | |
| 174 | 177 | /* |
| 175 | 178 | * Both desc_size and bpages_backing_size are untrusted host-provided |
| 176 | 179 | * values. We rely on __pkvm_host_donate_hyp() to enforce their validity. |
| @@ -212,8 +215,10 @@ int __tracing_load(unsigned long desc_hva, size_t desc_size) · 35 unmodified lines | ||
| 212 | 215 | if (ret) |
| 213 | 216 | return ret; |
| 214 | 217 | |
| 215 | − if (!hyp_trace_desc_validate(desc, desc_size)) | |
| 218 | + if (!hyp_trace_desc_is_valid(desc, desc_size)) { | |
| 219 | + ret = -EINVAL; | |
| 216 | 220 | goto err_release_desc; |
| 221 | + } | |
| 217 | 222 | |
| 218 | 223 | hyp_spin_lock(&trace_buffer.lock); |
| 219 | 224 | |
| @@ -663,7 +663,8 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) · 662 unmodified lines | ||
| 663 | 663 | host_ctxt = host_data_ptr(host_ctxt); |
| 664 | 664 | vcpu = host_ctxt->__hyp_running_vcpu; |
| 665 | 665 | |
| 666 | − __deactivate_traps(vcpu); | |
| 666 | + if (vcpu) | |
| 667 | + __deactivate_traps(vcpu); | |
| 667 | 668 | sysreg_restore_host_state_vhe(host_ctxt); |
| 668 | 669 | |
| 669 | 670 | panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", |
| @@ -189,7 +189,7 @@ static void hyp_trace_buffer_unshare_hyp(struct hyp_trace_buffer *trace_buffer, · 188 unmodified lines | ||
| 189 | 189 | if (cpu > last_cpu) |
| 190 | 190 | break; |
| 191 | 191 | |
| 192 | − __share_page(rb_desc->meta_va); | |
| 192 | + __unshare_page(rb_desc->meta_va); | |
| 193 | 193 | for (p = 0; p < rb_desc->nr_page_va; p++) |
| 194 | 194 | __unshare_page(rb_desc->page_va[p]); |
| 195 | 195 | } |
| @@ -212,14 +212,15 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) · 16 unmodified lines | ||
| 212 | 212 | } |
| 213 | 213 | |
| 214 | 214 | if (ret) { |
| 215 | − for (p--; p >= 0; p--) | |
| 215 | + while (--p >= 0) | |
| 216 | 216 | __unshare_page(rb_desc->page_va[p]); |
| 217 | + __unshare_page(rb_desc->meta_va); | |
| 217 | 218 | break; |
| 218 | 219 | } |
| 219 | 220 | } |
| 220 | 221 | |
| 221 | 222 | if (ret) |
| 222 | − hyp_trace_buffer_unshare_hyp(trace_buffer, cpu--); | |
| 223 | + hyp_trace_buffer_unshare_hyp(trace_buffer, --cpu); | |
| 223 | 224 | |
| 224 | 225 | return ret; |
| 225 | 226 | } |
| @@ -248,6 +249,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) · 22 unmodified lines | ||
| 248 | 249 | goto err_free_desc; |
| 249 | 250 | |
| 250 | 251 | trace_buffer->desc = desc; |
| 252 | + trace_buffer->desc_size = desc_size; | |
| 251 | 253 | |
| 252 | 254 | ret = hyp_trace_buffer_alloc_bpages_backing(trace_buffer, size); |
| 253 | 255 | if (ret) |
| @@ -297,6 +299,7 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) · 43 unmodified lines | ||
| 297 | 299 | hyp_trace_buffer_free_bpages_backing(trace_buffer); |
| 298 | 300 | free_pages_exact(trace_buffer->desc, trace_buffer->desc_size); |
| 299 | 301 | trace_buffer->desc = NULL; |
| 302 | + trace_buffer->desc_size = 0; | |
| 300 | 303 | } |
| 301 | 304 | |
| 302 | 305 | static int hyp_trace_enable_tracing(bool enable, void *priv) |
| @@ -1576,21 +1576,24 @@ struct kvm_s2_fault_desc { · 1575 unmodified lines | ||
| 1576 | 1576 | static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) |
| 1577 | 1577 | { |
| 1578 | 1578 | bool write_fault, exec_fault; |
| 1579 | + bool perm_fault = kvm_vcpu_trap_is_permission_fault(s2fd->vcpu); | |
| 1579 | 1580 | enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED; |
| 1580 | 1581 | enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; |
| 1581 | 1582 | struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt; |
| 1582 | 1583 | unsigned long mmu_seq; |
| 1583 | 1584 | struct page *page; |
| 1584 | 1585 | struct kvm *kvm = s2fd->vcpu->kvm; |
| 1585 | − void *memcache; | |
| 1586 | + void *memcache = NULL; | |
| 1586 | 1587 | kvm_pfn_t pfn; |
| 1587 | 1588 | gfn_t gfn; |
| 1588 | 1589 | int ret; |
| 1589 | 1590 | |
| 1590 | − memcache = get_mmu_memcache(s2fd->vcpu); | |
| 1591 | − ret = topup_mmu_memcache(s2fd->vcpu, memcache); | |
| 1592 | − if (ret) | |
| 1593 | − return ret; | |
| 1591 | + if (!perm_fault) { | |
| 1592 | + memcache = get_mmu_memcache(s2fd->vcpu); | |
| 1593 | + ret = topup_mmu_memcache(s2fd->vcpu, memcache); | |
| 1594 | + if (ret) | |
| 1595 | + return ret; | |
| 1596 | + } | |
| 1594 | 1597 | |
| 1595 | 1598 | if (s2fd->nested) |
| 1596 | 1599 | gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT; |
| @@ -1631,9 +1634,19 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) · 34 unmodified lines | ||
| 1631 | 1634 | goto out_unlock; |
| 1632 | 1635 | } |
| 1633 | 1636 | |
| 1634 | − ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE, | |
| 1635 | − __pfn_to_phys(pfn), prot, | |
| 1636 | − memcache, flags); | |
| 1637 | + if (perm_fault) { | |
| 1638 | + /* | |
| 1639 | + * Drop the SW bits in favour of those stored in the | |
| 1640 | + * PTE, which will be preserved. | |
| 1641 | + */ | |
| 1642 | + prot &= ~KVM_NV_GUEST_MAP_SZ; | |
| 1643 | + ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa, | |
| 1644 | + prot, flags); | |
| 1645 | + } else { | |
| 1646 | + ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE, | |
| 1647 | + __pfn_to_phys(pfn), prot, | |
| 1648 | + memcache, flags); | |
| 1649 | + } | |
| 1637 | 1650 | |
| 1638 | 1651 | out_unlock: |
| 1639 | 1652 | kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W); |
| @@ -1834,6 +1834,11 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) · 1833 unmodified lines | ||
| 1834 | 1834 | resx.res1 = VNCR_EL2_RES1; |
| 1835 | 1835 | set_sysreg_masks(kvm, VNCR_EL2, resx); |
| 1836 | 1836 | |
| 1837 | + /* ZCR_EL2 - bits 8:4 are RAZ/WI so treat them as RES0 */ | |
| 1838 | + resx.res0 = ZCR_ELx_RES0 | GENMASK_ULL(8, 4); | |
| 1839 | + resx.res1 = ZCR_ELx_RES1; | |
| 1840 | + set_sysreg_masks(kvm, ZCR_EL2, resx); | |
| 1841 | + | |
| 1837 | 1842 | out: |
| 1838 | 1843 | for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) |
| 1839 | 1844 | __vcpu_rmw_sys_reg(vcpu, sr, |=, 0); |
| @@ -174,8 +174,8 @@ static void kvm_pmu_set_pmc_value(struct kvm_pmc *pmc, u64 val, bool force) · 173 unmodified lines | ||
| 174 | 174 | * action is to use PMCR.P, which will reset them to |
| 175 | 175 | * 0 (the only use of the 'force' parameter). |
| 176 | 176 | */ |
| 177 | − val = __vcpu_sys_reg(vcpu, reg) & GENMASK(63, 32); | |
| 178 | − val |= lower_32_bits(val); | |
| 177 | + val = (__vcpu_sys_reg(vcpu, reg) & GENMASK(63, 32)) | | |
| 178 | + lower_32_bits(val); | |
| 179 | 179 | } |
| 180 | 180 | |
| 181 | 181 | __vcpu_assign_sys_reg(vcpu, reg, val); |
| @@ -2862,21 +2862,16 @@ static bool access_zcr_el2(struct kvm_vcpu *vcpu, · 2861 unmodified lines | ||
| 2862 | 2862 | struct sys_reg_params *p, |
| 2863 | 2863 | const struct sys_reg_desc *r) |
| 2864 | 2864 | { |
| 2865 | − unsigned int vq; | |
| 2866 | − | |
| 2867 | 2865 | if (guest_hyp_sve_traps_enabled(vcpu)) { |
| 2868 | 2866 | kvm_inject_nested_sve_trap(vcpu); |
| 2869 | 2867 | return false; |
| 2870 | 2868 | } |
| 2871 | 2869 | |
| 2872 | − if (!p->is_write) { | |
| 2870 | + if (!p->is_write) | |
| 2873 | 2871 | p->regval = __vcpu_sys_reg(vcpu, ZCR_EL2); |
| 2874 | − return true; | |
| 2875 | − } | |
| 2872 | + else | |
| 2873 | + __vcpu_assign_sys_reg(vcpu, ZCR_EL2, p->regval); | |
| 2876 | 2874 | |
| 2877 | − vq = SYS_FIELD_GET(ZCR_ELx, LEN, p->regval) + 1; | |
| 2878 | − vq = min(vq, vcpu_sve_max_vq(vcpu)); | |
| 2879 | − __vcpu_assign_sys_reg(vcpu, ZCR_EL2, vq - 1); | |
| 2880 | 2875 | return true; |
| 2881 | 2876 | } |
| 2882 | 2877 | |
| @@ -2307,6 +2307,10 @@ static int vgic_its_restore_dte(struct vgic_its *its, u32 id, · 2306 unmodified lines | ||
| 2307 | 2307 | /* dte entry is valid */ |
| 2308 | 2308 | offset = (entry & KVM_ITS_DTE_NEXT_MASK) >> KVM_ITS_DTE_NEXT_SHIFT; |
| 2309 | 2309 | |
| 2310 | + /* Mimic the MAPD behaviour and reject invalid EID bits. */ | |
| 2311 | + if (num_eventid_bits > VITS_TYPER_IDBITS) | |
| 2312 | + return -EINVAL; | |
| 2313 | + | |
| 2310 | 2314 | if (!vgic_its_check_id(its, baser, id, NULL)) |
| 2311 | 2315 | return -EINVAL; |
| 2312 | 2316 | |
| @@ -91,7 +91,7 @@ static int vgic_mmio_uaccess_write_v2_misc(struct kvm_vcpu *vcpu, · 90 unmodified lines | ||
| 91 | 91 | * migration from old kernels to new kernels with legacy |
| 92 | 92 | * userspace. |
| 93 | 93 | */ |
| 94 | − reg = FIELD_GET(GICD_IIDR_REVISION_MASK, reg); | |
| 94 | + reg = FIELD_GET(GICD_IIDR_REVISION_MASK, val); | |
| 95 | 95 | switch (reg) { |
| 96 | 96 | case KVM_VGIC_IMP_REV_2: |
| 97 | 97 | case KVM_VGIC_IMP_REV_3: |
| @@ -194,7 +194,7 @@ static int vgic_mmio_uaccess_write_v3_misc(struct kvm_vcpu *vcpu, · 193 unmodified lines | ||
| 194 | 194 | if ((reg ^ val) & ~GICD_IIDR_REVISION_MASK) |
| 195 | 195 | return -EINVAL; |
| 196 | 196 | |
| 197 | − reg = FIELD_GET(GICD_IIDR_REVISION_MASK, reg); | |
| 197 | + reg = FIELD_GET(GICD_IIDR_REVISION_MASK, val); | |
| 198 | 198 | switch (reg) { |
| 199 | 199 | case KVM_VGIC_IMP_REV_2: |
| 200 | 200 | case KVM_VGIC_IMP_REV_3: |
| @@ -1018,7 +1018,7 @@ struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, · 1017 unmodified lines | ||
| 1018 | 1018 | return vma_alloc_folio(flags, 0, vma, vaddr); |
| 1019 | 1019 | } |
| 1020 | 1020 | |
| 1021 | −bool tag_clear_highpages(struct page *page, int numpages) | |
| 1021 | +bool tag_clear_highpages(struct page *page, int numpages, bool clear_pages) | |
| 1022 | 1022 | { |
| 1023 | 1023 | /* |
| 1024 | 1024 | * Check if MTE is supported and fall back to clear_highpage(). |
| @@ -1026,13 +1026,16 @@ bool tag_clear_highpages(struct page *page, int numpages) · 1 unmodified line | ||
| 1026 | 1026 | * post_alloc_hook() will invoke tag_clear_highpages(). |
| 1027 | 1027 | */ |
| 1028 | 1028 | if (!system_supports_mte()) |
| 1029 | − return false; | |
| 1029 | + return clear_pages; | |
| 1030 | 1030 | |
| 1031 | 1031 | /* Newly allocated pages, shouldn't have been tagged yet */ |
| 1032 | 1032 | for (int i = 0; i < numpages; i++, page++) { |
| 1033 | 1033 | WARN_ON_ONCE(!try_page_mte_tagging(page)); |
| 1034 | − mte_zero_clear_page_tags(page_address(page)); | |
| 1034 | + if (clear_pages) | |
| 1035 | + mte_zero_clear_page_tags(page_address(page)); | |
| 1036 | + else | |
| 1037 | + mte_clear_page_tags(page_address(page)); | |
| 1035 | 1038 | set_page_mte_tagged(page); |
| 1036 | 1039 | } |
| 1037 | − return true; | |
| 1040 | + return false; | |
| 1038 | 1041 | } |
| @@ -9,6 +9,7 @@ generic-y += qrwlock.h · 8 unmodified lines | ||
| 9 | 9 | generic-y += qrwlock_types.h |
| 10 | 10 | generic-y += qspinlock.h |
| 11 | 11 | generic-y += parport.h |
| 12 | +generic-y += ring_buffer.h | |
| 12 | 13 | generic-y += user.h |
| 13 | 14 | generic-y += vmlinux.lds.h |
| 14 | 15 | generic-y += text-patching.h |
| @@ -5,4 +5,5 @@ generic-y += extable.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += iomap.h |
| 6 | 6 | generic-y += kvm_para.h |
| 7 | 7 | generic-y += mcs_spinlock.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += text-patching.h |
| @@ -3,7 +3,7 @@ obj-y += mm/ · 2 unmodified lines | ||
| 3 | 3 | obj-y += net/ |
| 4 | 4 | obj-y += vdso/ |
| 5 | 5 | |
| 6 | −obj-$(CONFIG_KVM) += kvm/ | |
| 6 | +obj-$(subst m,y,$(CONFIG_KVM)) += kvm/ | |
| 7 | 7 | |
| 8 | 8 | # for cleaning |
| 9 | 9 | subdir- += boot |
| @@ -220,6 +220,7 @@ menu "Kernel type and options" · 219 unmodified lines | ||
| 220 | 220 | |
| 221 | 221 | choice |
| 222 | 222 | prompt "Kernel type" |
| 223 | + default 64BIT # Keep existing behavior | |
| 223 | 224 | |
| 224 | 225 | config 32BIT |
| 225 | 226 | bool "32-bit kernel" |
| @@ -55,9 +55,11 @@ endif · 54 unmodified lines | ||
| 55 | 55 | ifdef CONFIG_32BIT |
| 56 | 56 | tool-archpref = $(32bit-tool-archpref) |
| 57 | 57 | UTS_MACHINE := loongarch32 |
| 58 | +cflags-y += $(call cc-option,-m32) | |
| 58 | 59 | else |
| 59 | 60 | tool-archpref = $(64bit-tool-archpref) |
| 60 | 61 | UTS_MACHINE := loongarch64 |
| 62 | +cflags-y += $(call cc-option,-m64) | |
| 61 | 63 | endif |
| 62 | 64 | |
| 63 | 65 | ifneq ($(SUBARCH),$(ARCH)) |
| @@ -10,5 +10,6 @@ generic-y += qrwlock.h · 9 unmodified lines | ||
| 10 | 10 | generic-y += user.h |
| 11 | 11 | generic-y += ioctl.h |
| 12 | 12 | generic-y += mmzone.h |
| 13 | +generic-y += ring_buffer.h | |
| 13 | 14 | generic-y += statfs.h |
| 14 | 15 | generic-y += text-patching.h |
| @@ -20,3 +20,23 @@ asmlinkage void noinstr __no_stack_protector ret_from_kernel_thread(struct task_ · 19 unmodified lines | ||
| 20 | 20 | struct pt_regs *regs, |
| 21 | 21 | int (*fn)(void *), |
| 22 | 22 | void *fn_arg); |
| 23 | + | |
| 24 | +struct kvm_run; | |
| 25 | +struct kvm_vcpu; | |
| 26 | +struct loongarch_fpu; | |
| 27 | + | |
| 28 | +void kvm_exc_entry(void); | |
| 29 | +int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu); | |
| 30 | + | |
| 31 | +void kvm_save_fpu(struct loongarch_fpu *fpu); | |
| 32 | +void kvm_restore_fpu(struct loongarch_fpu *fpu); | |
| 33 | + | |
| 34 | +#ifdef CONFIG_CPU_HAS_LSX | |
| 35 | +void kvm_save_lsx(struct loongarch_fpu *fpu); | |
| 36 | +void kvm_restore_lsx(struct loongarch_fpu *fpu); | |
| 37 | +#endif | |
| 38 | + | |
| 39 | +#ifdef CONFIG_CPU_HAS_LASX | |
| 40 | +void kvm_save_lasx(struct loongarch_fpu *fpu); | |
| 41 | +void kvm_restore_lasx(struct loongarch_fpu *fpu); | |
| 42 | +#endif | |
| @@ -30,6 +30,8 @@ static inline unsigned long efi_get_kimg_min_align(void) · 29 unmodified lines | ||
| 30 | 30 | return SZ_2M; |
| 31 | 31 | } |
| 32 | 32 | |
| 33 | −#define EFI_KIMG_PREFERRED_ADDRESS PHYSADDR(VMLINUX_LOAD_ADDRESS) | |
| 33 | +unsigned long efi_get_kimg_kaslr_address(void); | |
| 34 | + | |
| 35 | +#define EFI_KIMG_PREFERRED_ADDRESS efi_get_kimg_kaslr_address() | |
| 34 | 36 | |
| 35 | 37 | #endif /* _ASM_LOONGARCH_EFI_H */ |
| @@ -87,7 +87,6 @@ struct kvm_context { · 86 unmodified lines | ||
| 87 | 87 | struct kvm_world_switch { |
| 88 | 88 | int (*exc_entry)(void); |
| 89 | 89 | int (*enter_guest)(struct kvm_run *run, struct kvm_vcpu *vcpu); |
| 90 | − unsigned long page_order; | |
| 91 | 90 | }; |
| 92 | 91 | |
| 93 | 92 | #define MAX_PGTABLE_LEVELS 4 |
| @@ -359,8 +358,6 @@ void kvm_exc_entry(void); · 265 unmodified lines | ||
| 359 | 358 | int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu); |
| 360 | 359 | |
| 361 | 360 | extern unsigned long vpid_mask; |
| 362 | −extern const unsigned long kvm_exception_size; | |
| 363 | −extern const unsigned long kvm_enter_guest_size; | |
| 364 | 361 | extern struct kvm_world_switch *kvm_loongarch_ops; |
| 365 | 362 | |
| 366 | 363 | #define SW_GCSR (1 << 0) |
| @@ -69,7 +69,7 @@ · 68 unmodified lines | ||
| 69 | 69 | 9, 10, 11, 12, 13, 14, 15, 16, \ |
| 70 | 70 | 17, 18, 19, 20, 21, 22, 23, 24, \ |
| 71 | 71 | 25, 26, 27, 28, 29, 30, 31; \ |
| 72 | − .cfi_offset \num, SC_REGS + \num * SZREG; \ | |
| 72 | + .cfi_offset \num, SC_REGS + \num * 8; \ | |
| 73 | 73 | .endr; \ |
| 74 | 74 | \ |
| 75 | 75 | nop; \ |
| @@ -4,6 +4,12 @@ · 3 unmodified lines | ||
| 4 | 4 | |
| 5 | 5 | #ifdef CONFIG_PARAVIRT |
| 6 | 6 | |
| 7 | +#include <linux/jump_label.h> | |
| 8 | + | |
| 9 | +DECLARE_STATIC_KEY_FALSE(virt_preempt_key); | |
| 10 | +DECLARE_STATIC_KEY_FALSE(virt_spin_lock_key); | |
| 11 | +DECLARE_PER_CPU(struct kvm_steal_time, steal_time); | |
| 12 | + | |
| 7 | 13 | int __init pv_ipi_init(void); |
| 8 | 14 | int __init pv_time_init(void); |
| 9 | 15 | int __init pv_spinlock_init(void); |
| @@ -3,12 +3,9 @@ · 2 unmodified lines | ||
| 3 | 3 | #define _ASM_LOONGARCH_QSPINLOCK_H |
| 4 | 4 | |
| 5 | 5 | #include <asm/kvm_para.h> |
| 6 | −#include <linux/jump_label.h> | |
| 6 | +#include <asm/paravirt.h> | |
| 7 | 7 | |
| 8 | 8 | #ifdef CONFIG_PARAVIRT |
| 9 | −DECLARE_STATIC_KEY_FALSE(virt_preempt_key); | |
| 10 | −DECLARE_STATIC_KEY_FALSE(virt_spin_lock_key); | |
| 11 | −DECLARE_PER_CPU(struct kvm_steal_time, steal_time); | |
| 12 | 9 | |
| 13 | 10 | #define virt_spin_lock virt_spin_lock |
| 14 | 11 | |
| @@ -85,12 +85,6 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode, · 84 unmodified lines | ||
| 85 | 85 | return count; |
| 86 | 86 | } |
| 87 | 87 | |
| 88 | −static inline bool loongarch_vdso_hres_capable(void) | |
| 89 | −{ | |
| 90 | − return true; | |
| 91 | −} | |
| 92 | −#define __arch_vdso_hres_capable loongarch_vdso_hres_capable | |
| 93 | − | |
| 94 | 88 | #endif /* CONFIG_GENERIC_GETTIMEOFDAY */ |
| 95 | 89 | |
| 96 | 90 | #endif /* !__ASSEMBLER__ */ |
| @@ -60,16 +60,18 @@ NOKPROBE_SYMBOL(arch_prepare_kprobe); · 59 unmodified lines | ||
| 60 | 60 | /* Install breakpoint in text */ |
| 61 | 61 | void arch_arm_kprobe(struct kprobe *p) |
| 62 | 62 | { |
| 63 | − *p->addr = KPROBE_BP_INSN; | |
| 64 | − flush_insn_slot(p); | |
| 63 | + u32 insn = KPROBE_BP_INSN; | |
| 64 | + | |
| 65 | + larch_insn_text_copy(p->addr, &insn, LOONGARCH_INSN_SIZE); | |
| 65 | 66 | } |
| 66 | 67 | NOKPROBE_SYMBOL(arch_arm_kprobe); |
| 67 | 68 | |
| 68 | 69 | /* Remove breakpoint from text */ |
| 69 | 70 | void arch_disarm_kprobe(struct kprobe *p) |
| 70 | 71 | { |
| 71 | − *p->addr = p->opcode; | |
| 72 | − flush_insn_slot(p); | |
| 72 | + u32 insn = p->opcode; | |
| 73 | + | |
| 74 | + larch_insn_text_copy(p->addr, &insn, LOONGARCH_INSN_SIZE); | |
| 73 | 75 | } |
| 74 | 76 | NOKPROBE_SYMBOL(arch_disarm_kprobe); |
| 75 | 77 | |
| @@ -184,16 +186,16 @@ static bool reenter_kprobe(struct kprobe *p, struct pt_regs *regs, · 108 unmodified lines | ||
| 184 | 186 | struct kprobe_ctlblk *kcb) |
| 185 | 187 | { |
| 186 | 188 | switch (kcb->kprobe_status) { |
| 187 | − case KPROBE_HIT_SS: | |
| 188 | 189 | case KPROBE_HIT_SSDONE: |
| 189 | 190 | case KPROBE_HIT_ACTIVE: |
| 190 | 191 | kprobes_inc_nmissed_count(p); |
| 191 | 192 | setup_singlestep(p, regs, kcb, 1); |
| 192 | 193 | break; |
| 194 | + case KPROBE_HIT_SS: | |
| 193 | 195 | case KPROBE_REENTER: |
| 194 | 196 | pr_warn("Failed to recover from reentered kprobes.\n"); |
| 195 | 197 | dump_kprobe(p); |
| 196 | − WARN_ON_ONCE(1); | |
| 198 | + BUG(); | |
| 197 | 199 | break; |
| 198 | 200 | default: |
| 199 | 201 | WARN_ON(1); |
| @@ -134,11 +134,23 @@ early_param("nokaslr", nokaslr); · 133 unmodified lines | ||
| 134 | 134 | |
| 135 | 135 | #define KASLR_DISABLED_MESSAGE "KASLR is disabled by %s in %s cmdline.\n" |
| 136 | 136 | |
| 137 | +/* | |
| 138 | + * Note: strictly-defined KASLR means the kernel's final runtime address | |
| 139 | + * has a random offset from the kernel's load address, which is implemented | |
| 140 | + * in relocate.c; broadly-defined KALSR means the kernel's final runtime | |
| 141 | + * address has a random offset from the kernel's link address (a.k.a. | |
| 142 | + * VMLINUX_LOAD_ADDRESS), which also include the efistlub implementation, | |
| 143 | + * kexec_file implementation and QEMU direct kernel boot. kaslr_disabled() | |
| 144 | + * return true only means strictly-defined KASLR is disabled. | |
| 145 | + */ | |
| 137 | 146 | static inline __init bool kaslr_disabled(void) |
| 138 | 147 | { |
| 139 | 148 | char *str; |
| 140 | 149 | const char *builtin_cmdline = CONFIG_CMDLINE; |
| 141 | 150 | |
| 151 | + if (kaslr_offset()) | |
| 152 | + return true; /* KASLR is performed during early boot. */ | |
| 153 | + | |
| 142 | 154 | str = strstr(builtin_cmdline, "nokaslr"); |
| 143 | 155 | if (str == builtin_cmdline || (str > builtin_cmdline && *(str - 1) == ' ')) { |
| 144 | 156 | pr_info(KASLR_DISABLED_MESSAGE, "\'nokaslr\'", "built-in"); |
| @@ -210,14 +222,52 @@ static inline void __init *determine_relocation_address(void) · 65 unmodified lines | ||
| 210 | 222 | return RELOCATED_KASLR(destination); |
| 211 | 223 | } |
| 212 | 224 | |
| 225 | +static unsigned long __init determine_initrd_address(unsigned long *size) | |
| 226 | +{ | |
| 227 | + unsigned long start = 0; | |
| 228 | + unsigned long key_length; | |
| 229 | + char *p, *endp, *key = "initrd="; | |
| 230 | + | |
| 231 | + key_length = strlen(key); | |
| 232 | + p = strstr(boot_command_line, key); | |
| 233 | + | |
| 234 | + if (!p) { | |
| 235 | + key = "initrdmem="; | |
| 236 | + key_length = strlen(key); | |
| 237 | + p = strstr(boot_command_line, key); | |
| 238 | + } | |
| 239 | + | |
| 240 | + if (p == boot_command_line || (p > boot_command_line && *(p - 1) == ' ')) { | |
| 241 | + p += key_length; | |
| 242 | + start = memparse(p, &endp); | |
| 243 | + if (*endp == ',') | |
| 244 | + *size = memparse(endp + 1, NULL); | |
| 245 | + } | |
| 246 | + | |
| 247 | + return start; | |
| 248 | +} | |
| 249 | + | |
| 213 | 250 | static inline int __init relocation_addr_valid(void *location_new) |
| 214 | 251 | { |
| 252 | + unsigned long kernel_start, kernel_size; | |
| 253 | + unsigned long initrd_start, initrd_size = 0; | |
| 254 | + | |
| 215 | 255 | if ((unsigned long)location_new & 0x00000ffff) |
| 216 | 256 | return 0; /* Inappropriately aligned new location */ |
| 217 | 257 | |
| 218 | 258 | if ((unsigned long)location_new < (unsigned long)_end) |
| 219 | 259 | return 0; /* New location overlaps original kernel */ |
| 220 | 260 | |
| 261 | + initrd_start = determine_initrd_address(&initrd_size); | |
| 262 | + if (initrd_start && initrd_size) { | |
| 263 | + kernel_start = PHYSADDR(location_new); | |
| 264 | + kernel_size = (unsigned long)_end - (unsigned long)_text; | |
| 265 | + | |
| 266 | + if (kernel_start < (initrd_start + initrd_size) && | |
| 267 | + initrd_start < (kernel_start + kernel_size)) | |
| 268 | + return 0; /* initrd/initramfs overlaps kernel */ | |
| 269 | + } | |
| 270 | + | |
| 221 | 271 | return 1; |
| 222 | 272 | } |
| 223 | 273 | #endif |
| @@ -7,11 +7,12 @@ include $(srctree)/virt/kvm/Makefile.kvm · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | obj-$(CONFIG_KVM) += kvm.o |
| 9 | 9 | |
| 10 | +obj-y += switch.o | |
| 11 | + | |
| 10 | 12 | kvm-y += exit.o |
| 11 | 13 | kvm-y += interrupt.o |
| 12 | 14 | kvm-y += main.o |
| 13 | 15 | kvm-y += mmu.o |
| 14 | −kvm-y += switch.o | |
| 15 | 16 | kvm-y += timer.o |
| 16 | 17 | kvm-y += tlb.o |
| 17 | 18 | kvm-y += vcpu.o |
| @@ -390,6 +390,7 @@ int kvm_emu_mmio_read(struct kvm_vcpu *vcpu, larch_inst inst) · 389 unmodified lines | ||
| 390 | 390 | run->mmio.len = 8; |
| 391 | 391 | break; |
| 392 | 392 | default: |
| 393 | + ret = EMULATE_FAIL; | |
| 393 | 394 | break; |
| 394 | 395 | } |
| 395 | 396 | break; |
| @@ -28,23 +28,29 @@ static unsigned int priority_to_irq[EXCCODE_INT_NUM] = { · 27 unmodified lines | ||
| 28 | 28 | static int kvm_irq_deliver(struct kvm_vcpu *vcpu, unsigned int priority) |
| 29 | 29 | { |
| 30 | 30 | unsigned int irq = 0; |
| 31 | + unsigned long old, new; | |
| 31 | 32 | |
| 32 | 33 | clear_bit(priority, &vcpu->arch.irq_pending); |
| 33 | 34 | if (priority < EXCCODE_INT_NUM) |
| 34 | 35 | irq = priority_to_irq[priority]; |
| 35 | 36 | |
| 36 | − if (kvm_guest_has_msgint(&vcpu->arch) && (priority == INT_AVEC)) { | |
| 37 | − dmsintc_inject_irq(vcpu); | |
| 38 | − set_gcsr_estat(irq); | |
| 39 | − return 1; | |
| 40 | − } | |
| 41 | − | |
| 42 | 37 | switch (priority) { |
| 38 | + case INT_AVEC: | |
| 39 | + if (!kvm_guest_has_msgint(&vcpu->arch)) | |
| 40 | + break; | |
| 41 | + dmsintc_inject_irq(vcpu); | |
| 42 | + fallthrough; | |
| 43 | 43 | case INT_TI: |
| 44 | 44 | case INT_IPI: |
| 45 | 45 | case INT_SWI0: |
| 46 | 46 | case INT_SWI1: |
| 47 | + old = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | |
| 47 | 48 | set_gcsr_estat(irq); |
| 49 | + new = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | |
| 50 | + | |
| 51 | + /* Inject TI if TVAL inverted */ | |
| 52 | + if (new > old) | |
| 53 | + set_gcsr_estat(CPU_TIMER); | |
| 48 | 54 | break; |
| 49 | 55 | |
| 50 | 56 | case INT_HWI0 ... INT_HWI7: |
| @@ -61,22 +67,28 @@ static int kvm_irq_deliver(struct kvm_vcpu *vcpu, unsigned int priority) · 10 unmodified lines | ||
| 61 | 67 | static int kvm_irq_clear(struct kvm_vcpu *vcpu, unsigned int priority) |
| 62 | 68 | { |
| 63 | 69 | unsigned int irq = 0; |
| 70 | + unsigned long old, new; | |
| 64 | 71 | |
| 65 | 72 | clear_bit(priority, &vcpu->arch.irq_clear); |
| 66 | 73 | if (priority < EXCCODE_INT_NUM) |
| 67 | 74 | irq = priority_to_irq[priority]; |
| 68 | 75 | |
| 69 | − if (kvm_guest_has_msgint(&vcpu->arch) && (priority == INT_AVEC)) { | |
| 70 | − clear_gcsr_estat(irq); | |
| 71 | − return 1; | |
| 72 | − } | |
| 73 | − | |
| 74 | 76 | switch (priority) { |
| 77 | + case INT_AVEC: | |
| 78 | + if (!kvm_guest_has_msgint(&vcpu->arch)) | |
| 79 | + break; | |
| 80 | + fallthrough; | |
| 75 | 81 | case INT_TI: |
| 76 | 82 | case INT_IPI: |
| 77 | 83 | case INT_SWI0: |
| 78 | 84 | case INT_SWI1: |
| 85 | + old = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | |
| 79 | 86 | clear_gcsr_estat(irq); |
| 87 | + new = kvm_read_hw_gcsr(LOONGARCH_CSR_TVAL); | |
| 88 | + | |
| 89 | + /* Inject TI if TVAL inverted */ | |
| 90 | + if (new > old) | |
| 91 | + set_gcsr_estat(CPU_TIMER); | |
| 80 | 92 | break; |
| 81 | 93 | |
| 82 | 94 | case INT_HWI0 ... INT_HWI7: |
| @@ -348,8 +348,7 @@ void kvm_arch_disable_virtualization_cpu(void) · 347 unmodified lines | ||
| 348 | 348 | |
| 349 | 349 | static int kvm_loongarch_env_init(void) |
| 350 | 350 | { |
| 351 | − int cpu, order, ret; | |
| 352 | − void *addr; | |
| 351 | + int cpu, ret; | |
| 353 | 352 | struct kvm_context *context; |
| 354 | 353 | |
| 355 | 354 | vmcs = alloc_percpu(struct kvm_context); |
| @@ -365,30 +364,8 @@ static int kvm_loongarch_env_init(void) · 9 unmodified lines | ||
| 365 | 364 | return -ENOMEM; |
| 366 | 365 | } |
| 367 | 366 | |
| 368 | − /* | |
| 369 | − * PGD register is shared between root kernel and kvm hypervisor. | |
| 370 | − * So world switch entry should be in DMW area rather than TLB area | |
| 371 | − * to avoid page fault reenter. | |
| 372 | − * | |
| 373 | − * In future if hardware pagetable walking is supported, we won't | |
| 374 | − * need to copy world switch code to DMW area. | |
| 375 | − */ | |
| 376 | − order = get_order(kvm_exception_size + kvm_enter_guest_size); | |
| 377 | − addr = (void *)__get_free_pages(GFP_KERNEL, order); | |
| 378 | − if (!addr) { | |
| 379 | − free_percpu(vmcs); | |
| 380 | − vmcs = NULL; | |
| 381 | − kfree(kvm_loongarch_ops); | |
| 382 | − kvm_loongarch_ops = NULL; | |
| 383 | − return -ENOMEM; | |
| 384 | − } | |
| 385 | − | |
| 386 | − memcpy(addr, kvm_exc_entry, kvm_exception_size); | |
| 387 | − memcpy(addr + kvm_exception_size, kvm_enter_guest, kvm_enter_guest_size); | |
| 388 | − flush_icache_range((unsigned long)addr, (unsigned long)addr + kvm_exception_size + kvm_enter_guest_size); | |
| 389 | − kvm_loongarch_ops->exc_entry = addr; | |
| 390 | − kvm_loongarch_ops->enter_guest = addr + kvm_exception_size; | |
| 391 | − kvm_loongarch_ops->page_order = order; | |
| 367 | + kvm_loongarch_ops->exc_entry = (void *)kvm_exc_entry; | |
| 368 | + kvm_loongarch_ops->enter_guest = (void *)kvm_enter_guest; | |
| 392 | 369 | |
| 393 | 370 | vpid_mask = read_csr_gstat(); |
| 394 | 371 | vpid_mask = (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_GSTAT_GIDBIT_SHIFT; |
| @@ -428,16 +405,10 @@ static int kvm_loongarch_env_init(void) · 33 unmodified lines | ||
| 428 | 405 | |
| 429 | 406 | static void kvm_loongarch_env_exit(void) |
| 430 | 407 | { |
| 431 | − unsigned long addr; | |
| 432 | − | |
| 433 | 408 | if (vmcs) |
| 434 | 409 | free_percpu(vmcs); |
| 435 | 410 | |
| 436 | 411 | if (kvm_loongarch_ops) { |
| 437 | − if (kvm_loongarch_ops->exc_entry) { | |
| 438 | − addr = (unsigned long)kvm_loongarch_ops->exc_entry; | |
| 439 | − free_pages(addr, kvm_loongarch_ops->page_order); | |
| 440 | − } | |
| 441 | 412 | kfree(kvm_loongarch_ops); |
| 442 | 413 | } |
| 443 | 414 | |
| @@ -95,7 +95,7 @@ static int kvm_flush_pte(kvm_pte_t *pte, phys_addr_t addr, kvm_ptw_ctx *ctx) · 94 unmodified lines | ||
| 95 | 95 | else |
| 96 | 96 | kvm->stat.pages--; |
| 97 | 97 | |
| 98 | − *pte = ctx->invalid_entry; | |
| 98 | + kvm_set_pte(pte, ctx->invalid_entry); | |
| 99 | 99 | |
| 100 | 100 | return 1; |
| 101 | 101 | } |
| @@ -4,9 +4,11 @@ · 3 unmodified lines | ||
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | 6 | #include <linux/linkage.h> |
| 7 | +#include <linux/kvm_types.h> | |
| 7 | 8 | #include <asm/asm.h> |
| 8 | 9 | #include <asm/asmmacro.h> |
| 9 | 10 | #include <asm/loongarch.h> |
| 11 | +#include <asm/page.h> | |
| 10 | 12 | #include <asm/regdef.h> |
| 11 | 13 | #include <asm/unwind_hints.h> |
| 12 | 14 | |
| @@ -100,11 +102,16 @@ · 87 unmodified lines | ||
| 100 | 102 | * - is still in guest mode, such as pgd table/vmid registers etc, |
| 101 | 103 | * - will fix with hw page walk enabled in future |
| 102 | 104 | * load kvm_vcpu from reserved CSR KVM_VCPU_KS, and save a2 to KVM_TEMP_KS |
| 105 | + * | |
| 106 | + * PGD register is shared between root kernel and kvm hypervisor. | |
| 107 | + * So world switch entry should be in DMW area rather than TLB area | |
| 108 | + * to avoid page fault re-enter. | |
| 103 | 109 | */ |
| 104 | 110 | .text |
| 111 | + .p2align PAGE_SHIFT | |
| 105 | 112 | .cfi_sections .debug_frame |
| 106 | 113 | SYM_CODE_START(kvm_exc_entry) |
| 107 | − UNWIND_HINT_UNDEFINED | |
| 114 | + UNWIND_HINT_END_OF_STACK | |
| 108 | 115 | csrwr a2, KVM_TEMP_KS |
| 109 | 116 | csrrd a2, KVM_VCPU_KS |
| 110 | 117 | addi.d a2, a2, KVM_VCPU_ARCH |
| @@ -190,8 +197,8 @@ ret_to_host: · 79 unmodified lines | ||
| 190 | 197 | kvm_restore_host_gpr a2 |
| 191 | 198 | jr ra |
| 192 | 199 | |
| 193 | −SYM_INNER_LABEL(kvm_exc_entry_end, SYM_L_LOCAL) | |
| 194 | 200 | SYM_CODE_END(kvm_exc_entry) |
| 201 | +EXPORT_SYMBOL_FOR_KVM(kvm_exc_entry) | |
| 195 | 202 | |
| 196 | 203 | /* |
| 197 | 204 | * int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu) |
| @@ -215,8 +222,8 @@ SYM_FUNC_START(kvm_enter_guest) · 17 unmodified lines | ||
| 215 | 222 | /* Save kvm_vcpu to kscratch */ |
| 216 | 223 | csrwr a1, KVM_VCPU_KS |
| 217 | 224 | kvm_switch_to_guest |
| 218 | −SYM_INNER_LABEL(kvm_enter_guest_end, SYM_L_LOCAL) | |
| 219 | 225 | SYM_FUNC_END(kvm_enter_guest) |
| 226 | +EXPORT_SYMBOL_FOR_KVM(kvm_enter_guest) | |
| 220 | 227 | |
| 221 | 228 | SYM_FUNC_START(kvm_save_fpu) |
| 222 | 229 | fpu_save_csr a0 t1 |
| @@ -224,6 +231,7 @@ SYM_FUNC_START(kvm_save_fpu) · 1 unmodified line | ||
| 224 | 231 | fpu_save_cc a0 t1 t2 |
| 225 | 232 | jr ra |
| 226 | 233 | SYM_FUNC_END(kvm_save_fpu) |
| 234 | +EXPORT_SYMBOL_FOR_KVM(kvm_save_fpu) | |
| 227 | 235 | |
| 228 | 236 | SYM_FUNC_START(kvm_restore_fpu) |
| 229 | 237 | fpu_restore_double a0 t1 |
| @@ -231,6 +239,7 @@ SYM_FUNC_START(kvm_restore_fpu) · 1 unmodified line | ||
| 231 | 239 | fpu_restore_cc a0 t1 t2 |
| 232 | 240 | jr ra |
| 233 | 241 | SYM_FUNC_END(kvm_restore_fpu) |
| 242 | +EXPORT_SYMBOL_FOR_KVM(kvm_restore_fpu) | |
| 234 | 243 | |
| 235 | 244 | #ifdef CONFIG_CPU_HAS_LSX |
| 236 | 245 | SYM_FUNC_START(kvm_save_lsx) |
| @@ -239,6 +248,7 @@ SYM_FUNC_START(kvm_save_lsx) · 2 unmodified lines | ||
| 239 | 248 | lsx_save_data a0 t1 |
| 240 | 249 | jr ra |
| 241 | 250 | SYM_FUNC_END(kvm_save_lsx) |
| 251 | +EXPORT_SYMBOL_FOR_KVM(kvm_save_lsx) | |
| 242 | 252 | |
| 243 | 253 | SYM_FUNC_START(kvm_restore_lsx) |
| 244 | 254 | lsx_restore_data a0 t1 |
| @@ -246,6 +256,7 @@ SYM_FUNC_START(kvm_restore_lsx) · 1 unmodified line | ||
| 246 | 256 | fpu_restore_csr a0 t1 t2 |
| 247 | 257 | jr ra |
| 248 | 258 | SYM_FUNC_END(kvm_restore_lsx) |
| 259 | +EXPORT_SYMBOL_FOR_KVM(kvm_restore_lsx) | |
| 249 | 260 | #endif |
| 250 | 261 | |
| 251 | 262 | #ifdef CONFIG_CPU_HAS_LASX |
| @@ -255,6 +266,7 @@ SYM_FUNC_START(kvm_save_lasx) · 3 unmodified lines | ||
| 255 | 266 | lasx_save_data a0 t1 |
| 256 | 267 | jr ra |
| 257 | 268 | SYM_FUNC_END(kvm_save_lasx) |
| 269 | +EXPORT_SYMBOL_FOR_KVM(kvm_save_lasx) | |
| 258 | 270 | |
| 259 | 271 | SYM_FUNC_START(kvm_restore_lasx) |
| 260 | 272 | lasx_restore_data a0 t1 |
| @@ -262,10 +274,8 @@ SYM_FUNC_START(kvm_restore_lasx) · 1 unmodified line | ||
| 262 | 274 | fpu_restore_csr a0 t1 t2 |
| 263 | 275 | jr ra |
| 264 | 276 | SYM_FUNC_END(kvm_restore_lasx) |
| 277 | +EXPORT_SYMBOL_FOR_KVM(kvm_restore_lasx) | |
| 265 | 278 | #endif |
| 266 | − .section ".rodata" | |
| 267 | −SYM_DATA(kvm_exception_size, .quad kvm_exc_entry_end - kvm_exc_entry) | |
| 268 | −SYM_DATA(kvm_enter_guest_size, .quad kvm_enter_guest_end - kvm_enter_guest) | |
| 269 | 279 | |
| 270 | 280 | #ifdef CONFIG_CPU_HAS_LBT |
| 271 | 281 | STACK_FRAME_NON_STANDARD kvm_restore_fpu |
| @@ -96,15 +96,21 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu) · 95 unmodified lines | ||
| 96 | 96 | * and set CSR TVAL with -1 |
| 97 | 97 | */ |
| 98 | 98 | write_gcsr_timertick(0); |
| 99 | − __delay(2); /* Wait cycles until timer interrupt injected */ | |
| 100 | 99 | |
| 101 | 100 | /* |
| 102 | 101 | * Writing CSR_TINTCLR_TI to LOONGARCH_CSR_TINTCLR will clear |
| 103 | 102 | * timer interrupt, and CSR TVAL keeps unchanged with -1, it |
| 104 | 103 | * avoids spurious timer interrupt |
| 105 | 104 | */ |
| 106 | − if (!(estat & CPU_TIMER)) | |
| 105 | + if (!(estat & CPU_TIMER)) { | |
| 106 | + __delay(2); /* Wait cycles until timer interrupt injected */ | |
| 107 | + | |
| 108 | + /* Write TVAL with max value if no TI shot */ | |
| 109 | + estat = kvm_read_hw_gcsr(LOONGARCH_CSR_ESTAT); | |
| 110 | + if (!(estat & CPU_TIMER)) | |
| 111 | + write_gcsr_timertick(CSR_TCFG_VAL); | |
| 107 | 112 | gcsr_write(CSR_TINTCLR_TI, LOONGARCH_CSR_TINTCLR); |
| 113 | + } | |
| 108 | 114 | return; |
| 109 | 115 | } |
| 110 | 116 | |
| @@ -125,7 +125,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 124 unmodified lines | ||
| 125 | 125 | r = 1; |
| 126 | 126 | break; |
| 127 | 127 | case KVM_CAP_NR_VCPUS: |
| 128 | − r = num_online_cpus(); | |
| 128 | + r = min_t(unsigned int, num_online_cpus(), KVM_MAX_VCPUS); | |
| 129 | 129 | break; |
| 130 | 130 | case KVM_CAP_MAX_VCPUS: |
| 131 | 131 | r = KVM_MAX_VCPUS; |
| @@ -123,11 +123,7 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) · 122 unmodified lines | ||
| 123 | 123 | { |
| 124 | 124 | unsigned long start_pfn = start >> PAGE_SHIFT; |
| 125 | 125 | unsigned long nr_pages = size >> PAGE_SHIFT; |
| 126 | − struct page *page = pfn_to_page(start_pfn); | |
| 127 | 126 | |
| 128 | − /* With altmap the first mapped page is offset from @start */ | |
| 129 | − if (altmap) | |
| 130 | − page += vmem_altmap_offset(altmap); | |
| 131 | 127 | __remove_pages(start_pfn, nr_pages, altmap); |
| 132 | 128 | } |
| 133 | 129 | #endif |
| @@ -61,11 +61,16 @@ static void acpi_release_root_info(struct acpi_pci_root_info *ci) · 60 unmodified lines | ||
| 61 | 61 | static int acpi_prepare_root_resources(struct acpi_pci_root_info *ci) |
| 62 | 62 | { |
| 63 | 63 | int status; |
| 64 | + unsigned long long pci_h = 0; | |
| 64 | 65 | struct resource_entry *entry, *tmp; |
| 65 | 66 | struct acpi_device *device = ci->bridge; |
| 66 | 67 | |
| 67 | 68 | status = acpi_pci_probe_root_resources(ci); |
| 68 | 69 | if (status > 0) { |
| 70 | + acpi_evaluate_integer(device->handle, "PCIH", NULL, &pci_h); | |
| 71 | + if (pci_h) | |
| 72 | + return status; | |
| 73 | + | |
| 69 | 74 | resource_list_for_each_entry_safe(entry, tmp, &ci->resources) { |
| 70 | 75 | if (entry->res->flags & IORESOURCE_MEM) { |
| 71 | 76 | entry->offset = ci->root->mcfg_addr & GENMASK_ULL(63, 40); |
| @@ -132,6 +132,9 @@ static void loongson_gpu_fixup_dma_hang(struct pci_dev *pdev, bool on) · 131 unmodified lines | ||
| 132 | 132 | crtc_reg = regbase; |
| 133 | 133 | crtc_offset = 0x400; |
| 134 | 134 | break; |
| 135 | + default: | |
| 136 | + iounmap(regbase); | |
| 137 | + return; | |
| 135 | 138 | } |
| 136 | 139 | |
| 137 | 140 | for (i = 0; i < CRTC_NUM_MAX; i++, crtc_reg += crtc_offset) { |
| @@ -12,6 +12,8 @@ obj-vdso-$(CONFIG_GENERIC_GETTIMEOFDAY) += vgettimeofday.o · 11 unmodified lines | ||
| 12 | 12 | ccflags-vdso := \ |
| 13 | 13 | $(filter -I%,$(KBUILD_CFLAGS)) \ |
| 14 | 14 | $(filter -E%,$(KBUILD_CFLAGS)) \ |
| 15 | + $(filter -m32,$(KBUILD_CFLAGS)) \ | |
| 16 | + $(filter -m64,$(KBUILD_CFLAGS)) \ | |
| 15 | 17 | $(filter -march=%,$(KBUILD_CFLAGS)) \ |
| 16 | 18 | $(filter -m%-float,$(KBUILD_CFLAGS)) \ |
| 17 | 19 | $(CLANG_FLAGS) \ |
| @@ -3,5 +3,6 @@ generated-y += syscall_table.h · 2 unmodified lines | ||
| 3 | 3 | generic-y += extable.h |
| 4 | 4 | generic-y += kvm_para.h |
| 5 | 5 | generic-y += mcs_spinlock.h |
| 6 | +generic-y += ring_buffer.h | |
| 6 | 7 | generic-y += spinlock.h |
| 7 | 8 | generic-y += text-patching.h |
| @@ -5,6 +5,7 @@ generic-y += extable.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += kvm_para.h |
| 6 | 6 | generic-y += mcs_spinlock.h |
| 7 | 7 | generic-y += parport.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += syscalls.h |
| 9 | 10 | generic-y += tlb.h |
| 10 | 11 | generic-y += user.h |
| @@ -10,6 +10,14 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/mc146818rtc.h> |
| 11 | 11 | #include <linux/platform_device.h> |
| 12 | 12 | |
| 13 | +#include <asm/bootinfo.h> | |
| 14 | + | |
| 15 | +#include <asm/dec/interrupts.h> | |
| 16 | +#include <asm/dec/ioasic_addrs.h> | |
| 17 | +#include <asm/dec/kn01.h> | |
| 18 | +#include <asm/dec/kn02.h> | |
| 19 | +#include <asm/dec/system.h> | |
| 20 | + | |
| 13 | 21 | static struct resource dec_rtc_resources[] = { |
| 14 | 22 | { |
| 15 | 23 | .name = "rtc", |
| @@ -30,11 +38,110 @@ static struct platform_device dec_rtc_device = { · 14 unmodified lines | ||
| 30 | 38 | .num_resources = ARRAY_SIZE(dec_rtc_resources), |
| 31 | 39 | }; |
| 32 | 40 | |
| 41 | +static struct resource dec_dz_resources[] = { | |
| 42 | + { .name = "dz", .flags = IORESOURCE_MEM, }, | |
| 43 | + { .name = "dz", .flags = IORESOURCE_IRQ, }, | |
| 44 | +}; | |
| 45 | + | |
| 46 | +static struct platform_device dec_dz_device = { | |
| 47 | + .name = "dz", | |
| 48 | + .id = PLATFORM_DEVID_NONE, | |
| 49 | + .resource = dec_dz_resources, | |
| 50 | + .num_resources = ARRAY_SIZE(dec_dz_resources), | |
| 51 | +}; | |
| 52 | + | |
| 53 | +static struct platform_device *dec_dz_devices[] __initdata = { | |
| 54 | + &dec_dz_device, | |
| 55 | +}; | |
| 56 | + | |
| 57 | +static struct resource dec_zs_resources[][2] = { | |
| 58 | + { | |
| 59 | + { .name = "scc0", .flags = IORESOURCE_MEM, }, | |
| 60 | + { .name = "scc0", .flags = IORESOURCE_IRQ, }, | |
| 61 | + }, | |
| 62 | + { | |
| 63 | + { .name = "scc1", .flags = IORESOURCE_MEM, }, | |
| 64 | + { .name = "scc1", .flags = IORESOURCE_IRQ, }, | |
| 65 | + }, | |
| 66 | +}; | |
| 67 | + | |
| 68 | +static struct platform_device dec_zs_device[] = { | |
| 69 | + { | |
| 70 | + .name = "zs", | |
| 71 | + .id = 0, | |
| 72 | + .resource = dec_zs_resources[0], | |
| 73 | + .num_resources = ARRAY_SIZE(dec_zs_resources[0]), | |
| 74 | + }, | |
| 75 | + { | |
| 76 | + .name = "zs", | |
| 77 | + .id = 1, | |
| 78 | + .resource = dec_zs_resources[1], | |
| 79 | + .num_resources = ARRAY_SIZE(dec_zs_resources[1]), | |
| 80 | + }, | |
| 81 | +}; | |
| 82 | + | |
| 33 | 83 | static int __init dec_add_devices(void) |
| 34 | 84 | { |
| 85 | + struct platform_device *dec_zs_devices[ARRAY_SIZE(dec_zs_device)]; | |
| 86 | + int ret1, ret2, ret3; | |
| 87 | + int num_dz, num_zs; | |
| 88 | + int irq, i; | |
| 89 | + | |
| 35 | 90 | dec_rtc_resources[0].start = RTC_PORT(0); |
| 36 | 91 | dec_rtc_resources[0].end = RTC_PORT(0) + dec_kn_slot_size - 1; |
| 37 | − return platform_device_register(&dec_rtc_device); | |
| 92 | + | |
| 93 | + i = 0; | |
| 94 | + irq = dec_interrupt[DEC_IRQ_DZ11]; | |
| 95 | + if (IS_ENABLED(CONFIG_32BIT) && irq >= 0) { | |
| 96 | + resource_size_t base; | |
| 97 | + | |
| 98 | + switch (mips_machtype) { | |
| 99 | + case MACH_DS23100: | |
| 100 | + case MACH_DS5100: | |
| 101 | + base = dec_kn_slot_base + KN01_DZ11; | |
| 102 | + break; | |
| 103 | + default: | |
| 104 | + base = dec_kn_slot_base + KN02_DZ11; | |
| 105 | + break; | |
| 106 | + } | |
| 107 | + dec_dz_device.resource[0].start = base; | |
| 108 | + dec_dz_device.resource[0].end = base + dec_kn_slot_size - 1; | |
| 109 | + dec_dz_device.resource[1].start = irq; | |
| 110 | + dec_dz_device.resource[1].end = irq; | |
| 111 | + i++; | |
| 112 | + } | |
| 113 | + num_dz = i; | |
| 114 | + | |
| 115 | + i = 0; | |
| 116 | + irq = dec_interrupt[DEC_IRQ_SCC0]; | |
| 117 | + if (irq >= 0) { | |
| 118 | + resource_size_t base = dec_kn_slot_base + IOASIC_SCC0; | |
| 119 | + | |
| 120 | + dec_zs_device[i].resource[0].start = base; | |
| 121 | + dec_zs_device[i].resource[0].end = base + dec_kn_slot_size - 1; | |
| 122 | + dec_zs_device[i].resource[1].start = irq; | |
| 123 | + dec_zs_device[i].resource[1].end = irq; | |
| 124 | + dec_zs_devices[i] = &dec_zs_device[i]; | |
| 125 | + i++; | |
| 126 | + } | |
| 127 | + irq = dec_interrupt[DEC_IRQ_SCC1]; | |
| 128 | + if (irq >= 0) { | |
| 129 | + resource_size_t base = dec_kn_slot_base + IOASIC_SCC1; | |
| 130 | + | |
| 131 | + dec_zs_device[i].resource[0].start = base; | |
| 132 | + dec_zs_device[i].resource[0].end = base + dec_kn_slot_size - 1; | |
| 133 | + dec_zs_device[i].resource[1].start = irq; | |
| 134 | + dec_zs_device[i].resource[1].end = irq; | |
| 135 | + dec_zs_devices[i] = &dec_zs_device[i]; | |
| 136 | + i++; | |
| 137 | + } | |
| 138 | + num_zs = i; | |
| 139 | + | |
| 140 | + ret1 = platform_device_register(&dec_rtc_device); | |
| 141 | + ret2 = IS_ENABLED(CONFIG_32BIT) ? | |
| 142 | + platform_add_devices(dec_dz_devices, num_dz) : 0; | |
| 143 | + ret3 = platform_add_devices(dec_zs_devices, num_zs); | |
| 144 | + return ret1 ? ret1 : ret2 ? ret2 : ret3; | |
| 38 | 145 | } |
| 39 | 146 | |
| 40 | 147 | device_initcall(dec_add_devices); |
| @@ -12,5 +12,6 @@ generic-y += mcs_spinlock.h · 11 unmodified lines | ||
| 12 | 12 | generic-y += parport.h |
| 13 | 13 | generic-y += qrwlock.h |
| 14 | 14 | generic-y += qspinlock.h |
| 15 | +generic-y += ring_buffer.h | |
| 15 | 16 | generic-y += user.h |
| 16 | 17 | generic-y += text-patching.h |
| @@ -5,6 +5,7 @@ generic-y += cmpxchg.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += extable.h |
| 6 | 6 | generic-y += kvm_para.h |
| 7 | 7 | generic-y += mcs_spinlock.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += spinlock.h |
| 9 | 10 | generic-y += user.h |
| 10 | 11 | generic-y += text-patching.h |
| @@ -12,4 +12,6 @@ · 11 unmodified lines | ||
| 12 | 12 | #define __ALIGN .align 4 |
| 13 | 13 | #define __ALIGN_STR ".align 4" |
| 14 | 14 | |
| 15 | +#define _THIS_IP_ ({ unsigned long __ip; asm volatile("nextpc %0" : "=r" (__ip)); __ip; }) | |
| 16 | + | |
| 15 | 17 | #endif |
| @@ -8,4 +8,5 @@ generic-y += spinlock_types.h · 7 unmodified lines | ||
| 8 | 8 | generic-y += spinlock.h |
| 9 | 9 | generic-y += qrwlock_types.h |
| 10 | 10 | generic-y += qrwlock.h |
| 11 | +generic-y += ring_buffer.h | |
| 11 | 12 | generic-y += user.h |
| @@ -174,15 +174,21 @@ ifeq ($(KBUILD_EXTMOD),) · 173 unmodified lines | ||
| 174 | 174 | # this hack. |
| 175 | 175 | prepare: vdso_prepare |
| 176 | 176 | vdso_prepare: prepare0 |
| 177 | − $(if $(CONFIG_64BIT),$(Q)$(MAKE) \ | |
| 178 | − $(build)=arch/parisc/kernel/vdso64 include/generated/vdso64-offsets.h) | |
| 179 | − $(if $(CONFIG_PA11)$(CONFIG_COMPAT),$(Q)$(MAKE) \ | |
| 177 | +ifdef CONFIG_64BIT | |
| 178 | + $(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso64 include/generated/vdso64-offsets.h | |
| 179 | + $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \ | |
| 180 | 180 | $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h) |
| 181 | +else | |
| 182 | + $(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h | |
| 183 | +endif | |
| 181 | 184 | endif |
| 182 | 185 | |
| 183 | −vdso-install-$(CONFIG_PA11) += arch/parisc/kernel/vdso32/vdso32.so | |
| 186 | +ifdef CONFIG_64BIT | |
| 187 | +vdso-install-y += arch/parisc/kernel/vdso64/vdso64.so | |
| 184 | 188 | vdso-install-$(CONFIG_COMPAT) += arch/parisc/kernel/vdso32/vdso32.so |
| 185 | −vdso-install-$(CONFIG_64BIT) += arch/parisc/kernel/vdso64/vdso64.so | |
| 189 | +else | |
| 190 | +vdso-install-y += arch/parisc/kernel/vdso32/vdso32.so | |
| 191 | +endif | |
| 186 | 192 | |
| 187 | 193 | install: KBUILD_IMAGE := vmlinux |
| 188 | 194 | zinstall: KBUILD_IMAGE := vmlinuz |
| @@ -4,4 +4,5 @@ generated-y += syscall_table_64.h · 3 unmodified lines | ||
| 4 | 4 | generic-y += agp.h |
| 5 | 5 | generic-y += kvm_para.h |
| 6 | 6 | generic-y += mcs_spinlock.h |
| 7 | +generic-y += ring_buffer.h | |
| 7 | 8 | generic-y += user.h |
| @@ -6,13 +6,14 @@ · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | #ifdef CONFIG_64BIT |
| 8 | 8 | #include <generated/vdso64-offsets.h> |
| 9 | +#define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) | |
| 9 | 10 | #endif |
| 10 | 11 | #if !defined(CONFIG_64BIT) || defined(CONFIG_COMPAT) |
| 11 | 12 | #include <generated/vdso32-offsets.h> |
| 12 | −#endif | |
| 13 | − | |
| 14 | −#define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) | |
| 15 | 13 | #define VDSO32_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso32_offset_##name)) |
| 14 | +#else | |
| 15 | +#define VDSO32_SYMBOL(tsk, name) 0UL | |
| 16 | +#endif | |
| 16 | 17 | |
| 17 | 18 | #endif /* __ASSEMBLER__ */ |
| 18 | 19 | |
| @@ -46,6 +46,9 @@ obj-$(CONFIG_KEXEC_FILE) += kexec_file.o · 45 unmodified lines | ||
| 46 | 46 | |
| 47 | 47 | # vdso |
| 48 | 48 | obj-y += vdso.o |
| 49 | −obj-$(CONFIG_64BIT) += vdso64/ | |
| 50 | −obj-$(CONFIG_PA11) += vdso32/ | |
| 49 | +ifdef CONFIG_64BIT | |
| 50 | +obj-y += vdso64/ | |
| 51 | 51 | obj-$(CONFIG_COMPAT) += vdso32/ |
| 52 | +else | |
| 53 | +obj-y += vdso32/ | |
| 54 | +endif | |
| @@ -41,9 +41,7 @@ · 40 unmodified lines | ||
| 41 | 41 | const struct dma_map_ops *hppa_dma_ops __ro_after_init; |
| 42 | 42 | EXPORT_SYMBOL(hppa_dma_ops); |
| 43 | 43 | |
| 44 | −static struct device root = { | |
| 45 | − .init_name = "parisc", | |
| 46 | −}; | |
| 44 | +static struct device *root; | |
| 47 | 45 | |
| 48 | 46 | static inline int check_dev(struct device *dev) |
| 49 | 47 | { |
| @@ -89,7 +87,7 @@ static int for_each_padev(int (*fn)(struct device *, void *), void * data) · 39 unmodified lines | ||
| 89 | 87 | .obj = data, |
| 90 | 88 | .fn = fn, |
| 91 | 89 | }; |
| 92 | − return device_for_each_child(&root, &recurse_data, descend_children); | |
| 90 | + return device_for_each_child(root, &recurse_data, descend_children); | |
| 93 | 91 | } |
| 94 | 92 | |
| 95 | 93 | /** |
| @@ -290,7 +288,7 @@ const struct parisc_device * · 194 unmodified lines | ||
| 290 | 288 | find_pa_parent_type(const struct parisc_device *padev, int type) |
| 291 | 289 | { |
| 292 | 290 | const struct device *dev = &padev->dev; |
| 293 | − while (dev != &root) { | |
| 291 | + while (dev != root) { | |
| 294 | 292 | struct parisc_device *candidate = to_parisc_device(dev); |
| 295 | 293 | if (candidate->id.hw_type == type) |
| 296 | 294 | return candidate; |
| @@ -319,7 +317,7 @@ static void get_node_path(struct device *dev, struct hardware_path *path) · 22 unmodified lines | ||
| 319 | 317 | dev = dev->parent; |
| 320 | 318 | } |
| 321 | 319 | |
| 322 | − while (dev != &root) { | |
| 320 | + while (dev != root) { | |
| 323 | 321 | if (dev_is_pci(dev)) { |
| 324 | 322 | unsigned int devfn = to_pci_dev(dev)->devfn; |
| 325 | 323 | path->bc[i--] = PCI_SLOT(devfn) | (PCI_FUNC(devfn)<< 5); |
| @@ -482,7 +480,7 @@ static struct parisc_device * __init alloc_tree_node( · 156 unmodified lines | ||
| 482 | 480 | static struct parisc_device *create_parisc_device(struct hardware_path *modpath) |
| 483 | 481 | { |
| 484 | 482 | int i; |
| 485 | − struct device *parent = &root; | |
| 483 | + struct device *parent = root; | |
| 486 | 484 | for (i = 0; i < 6; i++) { |
| 487 | 485 | if (modpath->bc[i] == -1) |
| 488 | 486 | continue; |
| @@ -755,7 +753,7 @@ parse_tree_node(struct device *parent, int index, struct hardware_path *modpath) · 266 unmodified lines | ||
| 755 | 753 | struct device *hwpath_to_device(struct hardware_path *modpath) |
| 756 | 754 | { |
| 757 | 755 | int i; |
| 758 | − struct device *parent = &root; | |
| 756 | + struct device *parent = root; | |
| 759 | 757 | for (i = 0; i < 6; i++) { |
| 760 | 758 | if (modpath->bc[i] == -1) |
| 761 | 759 | continue; |
| @@ -880,7 +878,7 @@ void __init walk_central_bus(void) · 118 unmodified lines | ||
| 880 | 878 | { |
| 881 | 879 | walk_native_bus(CENTRAL_BUS_ADDR, |
| 882 | 880 | CENTRAL_BUS_ADDR + (MAX_NATIVE_DEVICES * NATIVE_DEVICE_OFFSET), |
| 883 | − &root); | |
| 881 | + root); | |
| 884 | 882 | } |
| 885 | 883 | |
| 886 | 884 | static __init void print_parisc_device(struct parisc_device *dev) |
| @@ -907,9 +905,10 @@ void __init init_parisc_bus(void) · 20 unmodified lines | ||
| 907 | 905 | { |
| 908 | 906 | if (bus_register(&parisc_bus_type)) |
| 909 | 907 | panic("Could not register PA-RISC bus type\n"); |
| 910 | − if (device_register(&root)) | |
| 908 | + | |
| 909 | + root = root_device_register("parisc"); | |
| 910 | + if (IS_ERR(root)) | |
| 911 | 911 | panic("Could not register PA-RISC root device\n"); |
| 912 | − get_device(&root); | |
| 913 | 912 | } |
| 914 | 913 | |
| 915 | 914 | static __init void qemu_header(void) |
| @@ -83,11 +83,10 @@ config MSI_BITMAP_SELFTEST · 82 unmodified lines | ||
| 83 | 83 | depends on DEBUG_KERNEL |
| 84 | 84 | |
| 85 | 85 | config GUEST_STATE_BUFFER_TEST |
| 86 | − def_tristate n | |
| 86 | + def_tristate KUNIT_ALL_TESTS | |
| 87 | 87 | prompt "Enable Guest State Buffer unit tests" |
| 88 | 88 | depends on KUNIT |
| 89 | 89 | depends on KVM_BOOK3S_HV_POSSIBLE |
| 90 | − default KUNIT_ALL_TESTS | |
| 91 | 90 | help |
| 92 | 91 | The Guest State Buffer is a data format specified in the PAPR. |
| 93 | 92 | It is by hcalls to communicate the state of L2 guests between |
| @@ -76,7 +76,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 75 unmodified lines | ||
| 76 | 76 | # CONFIG_HW_RANDOM is not set |
| 77 | 77 | # CONFIG_HWMON is not set |
| 78 | 78 | CONFIG_FB=y |
| 79 | −CONFIG_FIRMWARE_EDID=y | |
| 80 | 79 | CONFIG_FB_TILEBLITTING=y |
| 81 | 80 | CONFIG_FB_RADEON=y |
| 82 | 81 | CONFIG_FB_3DFX=y |
| @@ -76,7 +76,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 75 unmodified lines | ||
| 76 | 76 | CONFIG_NVRAM=y |
| 77 | 77 | # CONFIG_HWMON is not set |
| 78 | 78 | CONFIG_FB=y |
| 79 | −CONFIG_FIRMWARE_EDID=y | |
| 80 | 79 | CONFIG_FB_OF=y |
| 81 | 80 | CONFIG_FB_MATROX=y |
| 82 | 81 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -85,6 +85,8 @@ CONFIG_PMAC_SMU=y · 84 unmodified lines | ||
| 85 | 85 | CONFIG_MAC_EMUMOUSEBTN=y |
| 86 | 86 | CONFIG_WINDFARM=y |
| 87 | 87 | CONFIG_WINDFARM_PM81=y |
| 88 | +CONFIG_WINDFARM_PM72=y | |
| 89 | +CONFIG_WINDFARM_RM31=y | |
| 88 | 90 | CONFIG_WINDFARM_PM91=y |
| 89 | 91 | CONFIG_WINDFARM_PM112=y |
| 90 | 92 | CONFIG_WINDFARM_PM121=y |
| @@ -121,7 +123,6 @@ CONFIG_I2C_CHARDEV=y · 30 unmodified lines | ||
| 121 | 123 | CONFIG_AGP=m |
| 122 | 124 | CONFIG_AGP_UNINORTH=m |
| 123 | 125 | CONFIG_FB=y |
| 124 | −CONFIG_FIRMWARE_EDID=y | |
| 125 | 126 | CONFIG_FB_TILEBLITTING=y |
| 126 | 127 | CONFIG_FB_OF=y |
| 127 | 128 | CONFIG_FB_NVIDIA=y |
| @@ -98,7 +98,6 @@ CONFIG_SENSORS_LM85=y · 97 unmodified lines | ||
| 98 | 98 | CONFIG_SENSORS_LM90=y |
| 99 | 99 | CONFIG_DRM=y |
| 100 | 100 | CONFIG_DRM_RADEON=y |
| 101 | −CONFIG_FIRMWARE_EDID=y | |
| 102 | 101 | CONFIG_FB_TILEBLITTING=y |
| 103 | 102 | CONFIG_FB_VGA16=y |
| 104 | 103 | CONFIG_FB_NVIDIA=y |
| @@ -196,7 +196,6 @@ CONFIG_I2C_CHARDEV=y · 195 unmodified lines | ||
| 196 | 196 | # CONFIG_PTP_1588_CLOCK is not set |
| 197 | 197 | CONFIG_DRM=y |
| 198 | 198 | CONFIG_DRM_AST=y |
| 199 | −CONFIG_FIRMWARE_EDID=y | |
| 200 | 199 | CONFIG_FB_OF=y |
| 201 | 200 | CONFIG_FB_MATROX=m |
| 202 | 201 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -249,7 +249,6 @@ CONFIG_I2C_CHARDEV=y · 248 unmodified lines | ||
| 249 | 249 | CONFIG_I2C_AMD8111=y |
| 250 | 250 | CONFIG_I2C_PASEMI=y |
| 251 | 251 | CONFIG_FB=y |
| 252 | −CONFIG_FIRMWARE_EDID=y | |
| 253 | 252 | CONFIG_FB_OF=y |
| 254 | 253 | CONFIG_FB_MATROX=y |
| 255 | 254 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -118,7 +118,6 @@ CONFIG_SERIAL_8250_CONSOLE=y · 117 unmodified lines | ||
| 118 | 118 | CONFIG_I2C_CHARDEV=y |
| 119 | 119 | CONFIG_I2C_AMD8111=y |
| 120 | 120 | CONFIG_FB=y |
| 121 | −CONFIG_FIRMWARE_EDID=y | |
| 122 | 121 | CONFIG_FB_OF=y |
| 123 | 122 | CONFIG_FB_MATROX=y |
| 124 | 123 | CONFIG_FB_MATROX_MILLENIUM=y |
| @@ -393,6 +393,7 @@ CONFIG_NETCONSOLE=m · 392 unmodified lines | ||
| 393 | 393 | CONFIG_TUN=m |
| 394 | 394 | CONFIG_VETH=m |
| 395 | 395 | CONFIG_VIRTIO_NET=m |
| 396 | +CONFIG_EL3=m | |
| 396 | 397 | CONFIG_VORTEX=m |
| 397 | 398 | CONFIG_TYPHOON=m |
| 398 | 399 | CONFIG_ADAPTEC_STARFIRE=m |
| @@ -214,7 +214,6 @@ CONFIG_SENSORS_IBMPOWERNV=m · 213 unmodified lines | ||
| 214 | 214 | CONFIG_DRM=m |
| 215 | 215 | CONFIG_DRM_AST=m |
| 216 | 216 | CONFIG_FB=y |
| 217 | −CONFIG_FIRMWARE_EDID=y | |
| 218 | 217 | # CONFIG_VGA_CONSOLE is not set |
| 219 | 218 | CONFIG_FRAMEBUFFER_CONSOLE=y |
| 220 | 219 | CONFIG_LOGO=y |
| @@ -5,4 +5,5 @@ generated-y += syscall_table_spu.h · 4 unmodified lines | ||
| 5 | 5 | generic-y += agp.h |
| 6 | 6 | generic-y += mcs_spinlock.h |
| 7 | 7 | generic-y += qrwlock.h |
| 8 | +generic-y += ring_buffer.h | |
| 8 | 9 | generic-y += early_ioremap.h |
| @@ -79,10 +79,6 @@ extern int pmac_i2c_match_adapter(struct device_node *dev, · 78 unmodified lines | ||
| 79 | 79 | struct i2c_adapter *adapter); |
| 80 | 80 | |
| 81 | 81 | |
| 82 | −/* (legacy) Locking functions exposed to i2c-keywest */ | |
| 83 | −extern int pmac_low_i2c_lock(struct device_node *np); | |
| 84 | −extern int pmac_low_i2c_unlock(struct device_node *np); | |
| 85 | − | |
| 86 | 82 | /* Access functions for platform code */ |
| 87 | 83 | extern int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled); |
| 88 | 84 | extern void pmac_i2c_close(struct pmac_i2c_bus *bus); |
| @@ -458,6 +458,10 @@ DEFINE_PER_CPU(u8, irq_work_pending); · 457 unmodified lines | ||
| 458 | 458 | |
| 459 | 459 | #endif /* 32 vs 64 bit */ |
| 460 | 460 | |
| 461 | +/* | |
| 462 | + * Must be called with preemption disabled since it updates | |
| 463 | + * per-CPU irq_work state and programs the local CPU decrementer. | |
| 464 | + */ | |
| 461 | 465 | void arch_irq_work_raise(void) |
| 462 | 466 | { |
| 463 | 467 | /* |
| @@ -471,10 +475,8 @@ void arch_irq_work_raise(void) · 7 unmodified lines | ||
| 471 | 475 | * which could get tangled up if we're messing with the same state |
| 472 | 476 | * here. |
| 473 | 477 | */ |
| 474 | − preempt_disable(); | |
| 475 | 478 | set_irq_work_pending_flag(); |
| 476 | 479 | set_dec(1); |
| 477 | − preempt_enable(); | |
| 478 | 480 | } |
| 479 | 481 | |
| 480 | 482 | static void set_dec_or_work(u64 val) |
| @@ -62,6 +62,12 @@ CC32FLAGSREMOVE += -fno-stack-clash-protection · 61 unmodified lines | ||
| 62 | 62 | # 32-bit one. clang validates the values passed to these arguments during |
| 63 | 63 | # parsing, even when -fno-stack-protector is passed afterwards. |
| 64 | 64 | CC32FLAGSREMOVE += -mstack-protector-guard% |
| 65 | +# ftrace is disabled for the vdso but arch/powerpc/Makefile adds this define to | |
| 66 | +# KBUILD_CPPFLAGS, which enables use of the 'patchable_function_entry' | |
| 67 | +# attribute in the 'inline' define via 'notrace'. This attribute is not | |
| 68 | +# supported for the powerpcle target, resulting in many instances of | |
| 69 | +# -Wunknown-attributes. | |
| 70 | +CC32FLAGSREMOVE += -DCC_USING_PATCHABLE_FUNCTION_ENTRY | |
| 65 | 71 | endif |
| 66 | 72 | LD32FLAGS := -Wl,-soname=linux-vdso32.so.1 |
| 67 | 73 | AS32FLAGS := -D__VDSO32__ |
| @@ -16,4 +16,4 @@ GCOV_PROFILE_core_$(BITS).o := n · 15 unmodified lines | ||
| 16 | 16 | KCOV_INSTRUMENT_core_$(BITS).o := n |
| 17 | 17 | UBSAN_SANITIZE_core_$(BITS).o := n |
| 18 | 18 | KASAN_SANITIZE_core.o := n |
| 19 | −KASAN_SANITIZE_core_$(BITS) := n | |
| 19 | +KASAN_SANITIZE_core_$(BITS).o := n | |
| @@ -52,7 +52,14 @@ int exit_vmx_usercopy(void) · 51 unmodified lines | ||
| 52 | 52 | } |
| 53 | 53 | EXPORT_SYMBOL(exit_vmx_usercopy); |
| 54 | 54 | |
| 55 | −int enter_vmx_ops(void) | |
| 55 | +/* | |
| 56 | + * Can be called from kexec copy_page() path with MMU off. The kexec | |
| 57 | + * code sets preempt_count to HARDIRQ_OFFSET so we return early here. | |
| 58 | + * Since in_interrupt() is always inline, __no_sanitize_address on this | |
| 59 | + * function is sufficient to avoid KASAN shadow memory accesses in real | |
| 60 | + * mode. | |
| 61 | + */ | |
| 62 | +int __no_sanitize_address enter_vmx_ops(void) | |
| 56 | 63 | { |
| 57 | 64 | if (in_interrupt()) |
| 58 | 65 | return 0; |
| @@ -2242,6 +2242,7 @@ static void record_and_restart(struct perf_event *event, unsigned long val, · 2241 unmodified lines | ||
| 2242 | 2242 | const u64 last_period = event->hw.last_period; |
| 2243 | 2243 | s64 prev, delta, left; |
| 2244 | 2244 | int record = 0; |
| 2245 | + int mark_event = regs->dsisr & MMCRA_SAMPLE_ENABLE; | |
| 2245 | 2246 | |
| 2246 | 2247 | if (event->hw.state & PERF_HES_STOPPED) { |
| 2247 | 2248 | write_pmc(event->hw.idx, 0); |
| @@ -2304,9 +2305,9 @@ static void record_and_restart(struct perf_event *event, unsigned long val, · 56 unmodified lines | ||
| 2304 | 2305 | * In ISA v3.0 and before values "0" and "7" are considered reserved. |
| 2305 | 2306 | * In ISA v3.1, value "7" has been used to indicate "larx/stcx". |
| 2306 | 2307 | * Drop the sample if "type" has reserved values for this field with a |
| 2307 | − * ISA version check. | |
| 2308 | + * ISA version check for marked events. | |
| 2308 | 2309 | */ |
| 2309 | − if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC && | |
| 2310 | + if (mark_event && event->attr.sample_type & PERF_SAMPLE_DATA_SRC && | |
| 2310 | 2311 | ppmu->get_mem_data_src) { |
| 2311 | 2312 | val = (regs->dar & SIER_TYPE_MASK) >> SIER_TYPE_SHIFT; |
| 2312 | 2313 | if (val == 0 || (val == 7 && !cpu_has_feature(CPU_FTR_ARCH_31))) { |
| @@ -210,7 +210,7 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att · 209 unmodified lines | ||
| 210 | 210 | 0, 0, buf, &n, arg); |
| 211 | 211 | |
| 212 | 212 | if (!ret) |
| 213 | − return n; | |
| 213 | + goto out_success; | |
| 214 | 214 | |
| 215 | 215 | if (ret != H_PARAMETER) |
| 216 | 216 | goto out; |
| @@ -244,12 +244,14 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att · 27 unmodified lines | ||
| 244 | 244 | starting_index, 0, buf, &n, arg); |
| 245 | 245 | |
| 246 | 246 | if (!ret) |
| 247 | − return n; | |
| 247 | + goto out_success; | |
| 248 | 248 | |
| 249 | 249 | if (ret != H_PARAMETER) |
| 250 | 250 | goto out; |
| 251 | 251 | } |
| 252 | 252 | |
| 253 | +out_success: | |
| 254 | + put_cpu_var(hv_gpci_reqb); | |
| 253 | 255 | return n; |
| 254 | 256 | |
| 255 | 257 | out: |
| @@ -278,7 +280,7 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute · 22 unmodified lines | ||
| 278 | 280 | 0, 0, buf, &n, arg); |
| 279 | 281 | |
| 280 | 282 | if (!ret) |
| 281 | − return n; | |
| 283 | + goto out_success; | |
| 282 | 284 | |
| 283 | 285 | if (ret != H_PARAMETER) |
| 284 | 286 | goto out; |
| @@ -312,12 +314,14 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute · 27 unmodified lines | ||
| 312 | 314 | starting_index, 0, buf, &n, arg); |
| 313 | 315 | |
| 314 | 316 | if (!ret) |
| 315 | − return n; | |
| 317 | + goto out_success; | |
| 316 | 318 | |
| 317 | 319 | if (ret != H_PARAMETER) |
| 318 | 320 | goto out; |
| 319 | 321 | } |
| 320 | 322 | |
| 323 | +out_success: | |
| 324 | + put_cpu_var(hv_gpci_reqb); | |
| 321 | 325 | return n; |
| 322 | 326 | |
| 323 | 327 | out: |
| @@ -346,7 +350,7 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev, · 22 unmodified lines | ||
| 346 | 350 | 0, 0, buf, &n, arg); |
| 347 | 351 | |
| 348 | 352 | if (!ret) |
| 349 | − return n; | |
| 353 | + goto out_success; | |
| 350 | 354 | |
| 351 | 355 | if (ret != H_PARAMETER) |
| 352 | 356 | goto out; |
| @@ -382,12 +386,14 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev, · 29 unmodified lines | ||
| 382 | 386 | starting_index, secondary_index, buf, &n, arg); |
| 383 | 387 | |
| 384 | 388 | if (!ret) |
| 385 | − return n; | |
| 389 | + goto out_success; | |
| 386 | 390 | |
| 387 | 391 | if (ret != H_PARAMETER) |
| 388 | 392 | goto out; |
| 389 | 393 | } |
| 390 | 394 | |
| 395 | +out_success: | |
| 396 | + put_cpu_var(hv_gpci_reqb); | |
| 391 | 397 | return n; |
| 392 | 398 | |
| 393 | 399 | out: |
| @@ -416,7 +422,7 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device · 22 unmodified lines | ||
| 416 | 422 | 0, 0, buf, &n, arg); |
| 417 | 423 | |
| 418 | 424 | if (!ret) |
| 419 | − return n; | |
| 425 | + goto out_success; | |
| 420 | 426 | |
| 421 | 427 | if (ret != H_PARAMETER) |
| 422 | 428 | goto out; |
| @@ -448,12 +454,14 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device · 25 unmodified lines | ||
| 448 | 454 | starting_index, 0, buf, &n, arg); |
| 449 | 455 | |
| 450 | 456 | if (!ret) |
| 451 | − return n; | |
| 457 | + goto out_success; | |
| 452 | 458 | |
| 453 | 459 | if (ret != H_PARAMETER) |
| 454 | 460 | goto out; |
| 455 | 461 | } |
| 456 | 462 | |
| 463 | +out_success: | |
| 464 | + put_cpu_var(hv_gpci_reqb); | |
| 457 | 465 | return n; |
| 458 | 466 | |
| 459 | 467 | out: |
| @@ -293,6 +293,8 @@ static int pika_dtm_thread(void __iomem *fpga) · 292 unmodified lines | ||
| 293 | 293 | schedule_timeout(HZ); |
| 294 | 294 | } |
| 295 | 295 | |
| 296 | + put_device(&client->dev); | |
| 297 | + | |
| 296 | 298 | return 0; |
| 297 | 299 | } |
| 298 | 300 | |
| @@ -27,8 +27,8 @@ · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | static void __init km82xx_pic_init(void) |
| 29 | 29 | { |
| 30 | − struct device_node *np __free(device_node); | |
| 31 | − np = of_find_compatible_node(NULL, NULL, "fsl,pq2-pic"); | |
| 30 | + struct device_node *np __free(device_node) = of_find_compatible_node(NULL, | |
| 31 | + NULL, "fsl,pq2-pic"); | |
| 32 | 32 | |
| 33 | 33 | if (!np) { |
| 34 | 34 | pr_err("PIC init: can not find cpm-pic node\n"); |
| @@ -477,7 +477,7 @@ int cpm1_gpiochip_add16(struct device *dev) · 476 unmodified lines | ||
| 477 | 477 | struct device_node *np = dev->of_node; |
| 478 | 478 | struct cpm1_gpio16_chip *cpm1_gc; |
| 479 | 479 | struct gpio_chip *gc; |
| 480 | − u16 mask; | |
| 480 | + u32 mask; | |
| 481 | 481 | |
| 482 | 482 | cpm1_gc = devm_kzalloc(dev, sizeof(*cpm1_gc), GFP_KERNEL); |
| 483 | 483 | if (!cpm1_gc) |
| @@ -485,7 +485,7 @@ int cpm1_gpiochip_add16(struct device *dev) · 1 unmodified line | ||
| 485 | 485 | |
| 486 | 486 | spin_lock_init(&cpm1_gc->lock); |
| 487 | 487 | |
| 488 | − if (!of_property_read_u16(np, "fsl,cpm1-gpio-irq-mask", &mask)) { | |
| 488 | + if (!of_property_read_u32(np, "fsl,cpm1-gpio-irq-mask", &mask)) { | |
| 489 | 489 | int i, j; |
| 490 | 490 | |
| 491 | 491 | for (i = 0, j = 0; i < 16; i++) |
| @@ -272,13 +272,12 @@ void __init pas_pci_init(void) · 271 unmodified lines | ||
| 272 | 272 | { |
| 273 | 273 | struct device_node *root = of_find_node_by_path("/"); |
| 274 | 274 | struct device_node *np; |
| 275 | − int res; | |
| 276 | 275 | |
| 277 | 276 | pci_set_flags(PCI_SCAN_ALL_PCIE_DEVS); |
| 278 | 277 | |
| 279 | 278 | np = of_find_compatible_node(root, NULL, "pasemi,rootbus"); |
| 280 | 279 | if (np) { |
| 281 | − res = pas_add_bridge(np); | |
| 280 | + pas_add_bridge(np); | |
| 282 | 281 | of_node_put(np); |
| 283 | 282 | } |
| 284 | 283 | of_node_put(root); |
| @@ -1058,40 +1058,6 @@ int pmac_i2c_match_adapter(struct device_node *dev, struct i2c_adapter *adapter) · 1057 unmodified lines | ||
| 1058 | 1058 | } |
| 1059 | 1059 | EXPORT_SYMBOL_GPL(pmac_i2c_match_adapter); |
| 1060 | 1060 | |
| 1061 | −int pmac_low_i2c_lock(struct device_node *np) | |
| 1062 | −{ | |
| 1063 | − struct pmac_i2c_bus *bus, *found = NULL; | |
| 1064 | − | |
| 1065 | − list_for_each_entry(bus, &pmac_i2c_busses, link) { | |
| 1066 | − if (np == bus->controller) { | |
| 1067 | − found = bus; | |
| 1068 | − break; | |
| 1069 | − } | |
| 1070 | − } | |
| 1071 | − if (!found) | |
| 1072 | − return -ENODEV; | |
| 1073 | − return pmac_i2c_open(bus, 0); | |
| 1074 | −} | |
| 1075 | −EXPORT_SYMBOL_GPL(pmac_low_i2c_lock); | |
| 1076 | − | |
| 1077 | −int pmac_low_i2c_unlock(struct device_node *np) | |
| 1078 | −{ | |
| 1079 | − struct pmac_i2c_bus *bus, *found = NULL; | |
| 1080 | − | |
| 1081 | − list_for_each_entry(bus, &pmac_i2c_busses, link) { | |
| 1082 | − if (np == bus->controller) { | |
| 1083 | − found = bus; | |
| 1084 | − break; | |
| 1085 | − } | |
| 1086 | − } | |
| 1087 | − if (!found) | |
| 1088 | − return -ENODEV; | |
| 1089 | − pmac_i2c_close(bus); | |
| 1090 | − return 0; | |
| 1091 | −} | |
| 1092 | −EXPORT_SYMBOL_GPL(pmac_low_i2c_unlock); | |
| 1093 | − | |
| 1094 | − | |
| 1095 | 1061 | int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled) |
| 1096 | 1062 | { |
| 1097 | 1063 | int rc; |
| @@ -950,8 +950,6 @@ static int __init ps3_start_probe_thread(enum ps3_bus_type bus_type) · 949 unmodified lines | ||
| 950 | 950 | |
| 951 | 951 | static int __init ps3_register_devices(void) |
| 952 | 952 | { |
| 953 | − int result; | |
| 954 | − | |
| 955 | 953 | if (!firmware_has_feature(FW_FEATURE_PS3_LV1)) |
| 956 | 954 | return -ENODEV; |
| 957 | 955 | |
| @@ -959,7 +957,7 @@ static int __init ps3_register_devices(void) · 1 unmodified line | ||
| 959 | 957 | |
| 960 | 958 | /* ps3_repository_dump_bus_info(); */ |
| 961 | 959 | |
| 962 | − result = ps3_start_probe_thread(PS3_BUS_TYPE_STORAGE); | |
| 960 | + ps3_start_probe_thread(PS3_BUS_TYPE_STORAGE); | |
| 963 | 961 | |
| 964 | 962 | ps3_register_vuart_devices(); |
| 965 | 963 | |
| @@ -16,6 +16,7 @@ static void *htm_buf; · 15 unmodified lines | ||
| 16 | 16 | static void *htm_status_buf; |
| 17 | 17 | static void *htm_info_buf; |
| 18 | 18 | static void *htm_caps_buf; |
| 19 | +static void *htm_mem_buf; | |
| 19 | 20 | static u32 nodeindex; |
| 20 | 21 | static u32 nodalchipindex; |
| 21 | 22 | static u32 coreindexonchip; |
| @@ -86,7 +87,7 @@ static ssize_t htm_return_check(long rc) · 64 unmodified lines | ||
| 86 | 87 | static ssize_t htmdump_read(struct file *filp, char __user *ubuf, |
| 87 | 88 | size_t count, loff_t *ppos) |
| 88 | 89 | { |
| 89 | − void *htm_buf = filp->private_data; | |
| 90 | + void *htm_buf_data = filp->private_data; | |
| 90 | 91 | unsigned long page, read_size, available; |
| 91 | 92 | loff_t offset; |
| 92 | 93 | long rc, ret; |
| @@ -100,7 +101,7 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf, · 7 unmodified lines | ||
| 100 | 101 | * - last three values are address, size and offset |
| 101 | 102 | */ |
| 102 | 103 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 103 | − htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf), | |
| 104 | + htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf_data), | |
| 104 | 105 | PAGE_SIZE, page); |
| 105 | 106 | |
| 106 | 107 | ret = htm_return_check(rc); |
| @@ -112,7 +113,61 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf, · 5 unmodified lines | ||
| 112 | 113 | available = PAGE_SIZE; |
| 113 | 114 | read_size = min(count, available); |
| 114 | 115 | *ppos += read_size; |
| 115 | − return simple_read_from_buffer(ubuf, count, &offset, htm_buf, available); | |
| 116 | + return simple_read_from_buffer(ubuf, count, &offset, htm_buf_data, available); | |
| 117 | +} | |
| 118 | + | |
| 119 | +static ssize_t htmsystem_mem_read(struct file *filp, char __user *ubuf, | |
| 120 | + size_t count, loff_t *ppos) | |
| 121 | +{ | |
| 122 | + void *htm_mem_data = filp->private_data; | |
| 123 | + long rc, ret; | |
| 124 | + u64 *num_entries; | |
| 125 | + u64 to_copy = 0; | |
| 126 | + loff_t offset = 0; | |
| 127 | + u64 mem_offset = 0; | |
| 128 | + | |
| 129 | + /* | |
| 130 | + * Invoke H_HTM call with: | |
| 131 | + * - operation as htm status (H_HTM_OP_STATUS) | |
| 132 | + * - last three values as addr, size and offset. "offset" | |
| 133 | + * is value from output buffer header that points to next | |
| 134 | + * entry to dump. 0 is the first entry to dump. next entry | |
| 135 | + * is read from the output bufferbyte offset 0x8. | |
| 136 | + * | |
| 137 | + * When first time hcall is invoked, mem_offset should be | |
| 138 | + * zero because zero is the first entry. | |
| 139 | + * In the next hcall, offset of next entry to read from is | |
| 140 | + * picked from output buffer header itself. So don't fill | |
| 141 | + * mem_offset for first read. | |
| 142 | + * | |
| 143 | + * If there is no further data to read in next iteration, | |
| 144 | + * offset value from output buffer header will point to -1. | |
| 145 | + */ | |
| 146 | + if (*ppos) { | |
| 147 | + mem_offset = *(u64 *)(htm_mem_data + 0x8); | |
| 148 | + if (mem_offset == -1) | |
| 149 | + return 0; | |
| 150 | + } | |
| 151 | + rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, | |
| 152 | + htmtype, H_HTM_OP_DUMP_SYSMEM_CONF, virt_to_phys(htm_mem_data), | |
| 153 | + PAGE_SIZE, be64_to_cpu(mem_offset)); | |
| 154 | + ret = htm_return_check(rc); | |
| 155 | + if (ret <= 0) { | |
| 156 | + pr_debug("H_HTM hcall returned for op: H_HTM_OP_DUMP_SYSMEM_CONF with hcall returning %ld\n", ret); | |
| 157 | + return ret; | |
| 158 | + } | |
| 159 | + | |
| 160 | + /* | |
| 161 | + * HTM system mem buffer, start of buffer + 0x10 gives the | |
| 162 | + * number of HTM entries in the buffer. | |
| 163 | + * So total count to copy is: | |
| 164 | + * 32 bytes (for first 5 fields) + (number of HTM entries * entry size) | |
| 165 | + */ | |
| 166 | + num_entries = htm_mem_data + 0x10; | |
| 167 | + to_copy = 32 + (be64_to_cpu(*num_entries) * 32); | |
| 168 | + | |
| 169 | + *ppos += to_copy; | |
| 170 | + return simple_read_from_buffer(ubuf, count, &offset, htm_mem_data, to_copy); | |
| 116 | 171 | } |
| 117 | 172 | |
| 118 | 173 | static const struct file_operations htmdump_fops = { |
| @@ -121,6 +176,12 @@ static const struct file_operations htmdump_fops = { · 2 unmodified lines | ||
| 121 | 176 | .open = simple_open, |
| 122 | 177 | }; |
| 123 | 178 | |
| 179 | +static const struct file_operations htmsystem_mem_fops = { | |
| 180 | + .llseek = NULL, | |
| 181 | + .read = htmsystem_mem_read, | |
| 182 | + .open = simple_open, | |
| 183 | +}; | |
| 184 | + | |
| 124 | 185 | static int htmconfigure_set(void *data, u64 val) |
| 125 | 186 | { |
| 126 | 187 | long rc, ret; |
| @@ -226,20 +287,31 @@ static int htmstart_get(void *data, u64 *val) · 99 unmodified lines | ||
| 226 | 287 | static ssize_t htmstatus_read(struct file *filp, char __user *ubuf, |
| 227 | 288 | size_t count, loff_t *ppos) |
| 228 | 289 | { |
| 229 | − void *htm_status_buf = filp->private_data; | |
| 290 | + void *htm_status_data = filp->private_data; | |
| 230 | 291 | long rc, ret; |
| 231 | 292 | u64 *num_entries; |
| 232 | 293 | u64 to_copy; |
| 233 | 294 | int htmstatus_flag; |
| 295 | + loff_t offset = 0; | |
| 296 | + u64 status_offset = 0; | |
| 234 | 297 | |
| 235 | 298 | /* |
| 236 | 299 | * Invoke H_HTM call with: |
| 237 | 300 | * - operation as htm status (H_HTM_OP_STATUS) |
| 238 | − * - last three values as addr, size and offset | |
| 301 | + * - last three values as addr, size and offset. | |
| 302 | + * "offset" is value from output buffer header | |
| 303 | + * that points to next entry to dump. 0 is the first | |
| 304 | + * entry to dump. next entry is read from the output | |
| 305 | + * bufferbyte offset 0x8. | |
| 239 | 306 | */ |
| 307 | + if (*ppos) { | |
| 308 | + status_offset = *(u64 *)(htm_status_data + 0x8); | |
| 309 | + if (status_offset == -1) | |
| 310 | + return 0; | |
| 311 | + } | |
| 240 | 312 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 241 | − htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_buf), | |
| 242 | − PAGE_SIZE, 0); | |
| 313 | + htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_data), | |
| 314 | + PAGE_SIZE, be64_to_cpu(status_offset)); | |
| 243 | 315 | |
| 244 | 316 | ret = htm_return_check(rc); |
| 245 | 317 | if (ret <= 0) { |
| @@ -255,13 +327,15 @@ static ssize_t htmstatus_read(struct file *filp, char __user *ubuf, · 9 unmodified lines | ||
| 255 | 327 | * So total count to copy is: |
| 256 | 328 | * 32 bytes (for first 7 fields) + (number of HTM entries * entry size) |
| 257 | 329 | */ |
| 258 | − num_entries = htm_status_buf + 0x10; | |
| 330 | + num_entries = htm_status_data + 0x10; | |
| 259 | 331 | if (htmtype == 0x2) |
| 260 | 332 | htmstatus_flag = 0x8; |
| 261 | 333 | else |
| 262 | 334 | htmstatus_flag = 0x6; |
| 263 | 335 | to_copy = 32 + (be64_to_cpu(*num_entries) * htmstatus_flag); |
| 264 | − return simple_read_from_buffer(ubuf, count, ppos, htm_status_buf, to_copy); | |
| 336 | + *ppos += to_copy; | |
| 337 | + | |
| 338 | + return simple_read_from_buffer(ubuf, count, &offset, htm_status_data, to_copy); | |
| 265 | 339 | } |
| 266 | 340 | |
| 267 | 341 | static const struct file_operations htmstatus_fops = { |
| @@ -273,19 +347,30 @@ static const struct file_operations htmstatus_fops = { · 5 unmodified lines | ||
| 273 | 347 | static ssize_t htminfo_read(struct file *filp, char __user *ubuf, |
| 274 | 348 | size_t count, loff_t *ppos) |
| 275 | 349 | { |
| 276 | − void *htm_info_buf = filp->private_data; | |
| 350 | + void *htm_info_data = filp->private_data; | |
| 277 | 351 | long rc, ret; |
| 278 | 352 | u64 *num_entries; |
| 279 | 353 | u64 to_copy; |
| 354 | + loff_t offset = 0; | |
| 355 | + u64 info_offset = 0; | |
| 280 | 356 | |
| 281 | 357 | /* |
| 282 | 358 | * Invoke H_HTM call with: |
| 283 | 359 | * - operation as htm status (H_HTM_OP_STATUS) |
| 284 | 360 | * - last three values as addr, size and offset |
| 361 | + * "offset" is value from output buffer header | |
| 362 | + * that points to next entry to dump. 0 is the first | |
| 363 | + * entry to dump. next entry is read from the output | |
| 364 | + * bufferbyte offset 0x8. | |
| 285 | 365 | */ |
| 366 | + if (*ppos) { | |
| 367 | + info_offset = *(u64 *)(htm_info_data + 0x8); | |
| 368 | + if (info_offset == -1) | |
| 369 | + return 0; | |
| 370 | + } | |
| 286 | 371 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 287 | − htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_buf), | |
| 288 | − PAGE_SIZE, 0); | |
| 372 | + htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_data), | |
| 373 | + PAGE_SIZE, be64_to_cpu(info_offset)); | |
| 289 | 374 | |
| 290 | 375 | ret = htm_return_check(rc); |
| 291 | 376 | if (ret <= 0) { |
| @@ -301,15 +386,17 @@ static ssize_t htminfo_read(struct file *filp, char __user *ubuf, · 9 unmodified lines | ||
| 301 | 386 | * So total count to copy is: |
| 302 | 387 | * 32 bytes (for first 5 fields) + (number of HTM entries * entry size) |
| 303 | 388 | */ |
| 304 | − num_entries = htm_info_buf + 0x10; | |
| 389 | + num_entries = htm_info_data + 0x10; | |
| 305 | 390 | to_copy = 32 + (be64_to_cpu(*num_entries) * 16); |
| 306 | − return simple_read_from_buffer(ubuf, count, ppos, htm_info_buf, to_copy); | |
| 391 | + | |
| 392 | + *ppos += to_copy; | |
| 393 | + return simple_read_from_buffer(ubuf, count, &offset, htm_info_data, to_copy); | |
| 307 | 394 | } |
| 308 | 395 | |
| 309 | 396 | static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, |
| 310 | 397 | size_t count, loff_t *ppos) |
| 311 | 398 | { |
| 312 | − void *htm_caps_buf = filp->private_data; | |
| 399 | + void *htm_caps_data = filp->private_data; | |
| 313 | 400 | long rc, ret; |
| 314 | 401 | |
| 315 | 402 | /* |
| @@ -319,7 +406,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, · 3 unmodified lines | ||
| 319 | 406 | * and zero |
| 320 | 407 | */ |
| 321 | 408 | rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip, |
| 322 | − htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_buf), | |
| 409 | + htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_data), | |
| 323 | 410 | 0x80, 0); |
| 324 | 411 | |
| 325 | 412 | ret = htm_return_check(rc); |
| @@ -328,7 +415,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf, · 2 unmodified lines | ||
| 328 | 415 | return ret; |
| 329 | 416 | } |
| 330 | 417 | |
| 331 | − return simple_read_from_buffer(ubuf, count, ppos, htm_caps_buf, 0x80); | |
| 418 | + return simple_read_from_buffer(ubuf, count, ppos, htm_caps_data, 0x80); | |
| 332 | 419 | } |
| 333 | 420 | |
| 334 | 421 | static const struct file_operations htminfo_fops = { |
| @@ -457,9 +544,17 @@ static int htmdump_init_debugfs(void) · 122 unmodified lines | ||
| 457 | 544 | return -ENOMEM; |
| 458 | 545 | } |
| 459 | 546 | |
| 547 | + /* Memory to present HTM system memory configuration */ | |
| 548 | + htm_mem_buf = kmalloc(PAGE_SIZE, GFP_KERNEL); | |
| 549 | + if (!htm_mem_buf) { | |
| 550 | + pr_err("Failed to allocate htm mem buf\n"); | |
| 551 | + return -ENOMEM; | |
| 552 | + } | |
| 553 | + | |
| 460 | 554 | debugfs_create_file("htmstatus", 0400, htmdump_debugfs_dir, htm_status_buf, &htmstatus_fops); |
| 461 | 555 | debugfs_create_file("htminfo", 0400, htmdump_debugfs_dir, htm_info_buf, &htminfo_fops); |
| 462 | 556 | debugfs_create_file("htmcaps", 0400, htmdump_debugfs_dir, htm_caps_buf, &htmcaps_fops); |
| 557 | + debugfs_create_file("htmsystem_mem", 0400, htmdump_debugfs_dir, htm_mem_buf, &htmsystem_mem_fops); | |
| 463 | 558 | |
| 464 | 559 | return 0; |
| 465 | 560 | } |
| @@ -482,6 +577,10 @@ static void __exit htmdump_exit(void) · 16 unmodified lines | ||
| 482 | 577 | { |
| 483 | 578 | debugfs_remove_recursive(htmdump_debugfs_dir); |
| 484 | 579 | kfree(htm_buf); |
| 580 | + kfree(htm_status_buf); | |
| 581 | + kfree(htm_info_buf); | |
| 582 | + kfree(htm_caps_buf); | |
| 583 | + kfree(htm_mem_buf); | |
| 485 | 584 | } |
| 486 | 585 | |
| 487 | 586 | module_init(htmdump_init); |
| @@ -190,33 +190,34 @@ static int hvpipe_rtas_recv_msg(char __user *buf, int size) · 189 unmodified lines | ||
| 190 | 190 | return -ENOMEM; |
| 191 | 191 | } |
| 192 | 192 | |
| 193 | − ret = rtas_ibm_receive_hvpipe_msg(work_area, &srcID, | |
| 194 | − &bytes_written); | |
| 195 | − if (!ret) { | |
| 196 | − /* | |
| 197 | − * Recv HVPIPE RTAS is successful. | |
| 198 | − * When releasing FD or no one is waiting on the | |
| 199 | − * specific source, issue recv HVPIPE RTAS call | |
| 200 | − * so that pipe is not blocked - this func is called | |
| 201 | − * with NULL buf. | |
| 202 | − */ | |
| 203 | − if (buf) { | |
| 204 | − if (size < bytes_written) { | |
| 205 | − pr_err("Received the payload size = %d, but the buffer size = %d\n", | |
| 206 | − bytes_written, size); | |
| 207 | − bytes_written = size; | |
| 208 | − } | |
| 209 | − ret = copy_to_user(buf, | |
| 210 | − rtas_work_area_raw_buf(work_area), | |
| 211 | − bytes_written); | |
| 212 | − if (!ret) | |
| 213 | − ret = bytes_written; | |
| 214 | − } | |
| 215 | − } else { | |
| 216 | − pr_err("ibm,receive-hvpipe-msg failed with %d\n", | |
| 217 | − ret); | |
| 193 | + /* | |
| 194 | + * Recv HVPIPE RTAS is successful. | |
| 195 | + * When releasing FD or no one is waiting on the | |
| 196 | + * specific source, issue recv HVPIPE RTAS call | |
| 197 | + * so that pipe is not blocked - this func is called | |
| 198 | + * with NULL buf. | |
| 199 | + */ | |
| 200 | + ret = rtas_ibm_receive_hvpipe_msg(work_area, &srcID, &bytes_written); | |
| 201 | + if (ret) { | |
| 202 | + pr_err("ibm,receive-hvpipe-msg failed with %d\n", ret); | |
| 203 | + goto out; | |
| 218 | 204 | } |
| 219 | 205 | |
| 206 | + if (!buf) | |
| 207 | + goto out; | |
| 208 | + | |
| 209 | + if (size < bytes_written) { | |
| 210 | + pr_err("Received the payload size = %d, but the buffer size = %d\n", | |
| 211 | + bytes_written, size); | |
| 212 | + bytes_written = size; | |
| 213 | + } | |
| 214 | + | |
| 215 | + if (copy_to_user(buf, rtas_work_area_raw_buf(work_area), bytes_written)) | |
| 216 | + ret = -EFAULT; | |
| 217 | + else | |
| 218 | + ret = bytes_written; | |
| 219 | + | |
| 220 | +out: | |
| 220 | 221 | rtas_work_area_free(work_area); |
| 221 | 222 | return ret; |
| 222 | 223 | } |
| @@ -327,8 +328,8 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 104 unmodified lines | ||
| 327 | 328 | { |
| 328 | 329 | |
| 329 | 330 | struct hvpipe_source_info *src_info = file->private_data; |
| 330 | − struct papr_hvpipe_hdr hdr; | |
| 331 | − long ret; | |
| 331 | + struct papr_hvpipe_hdr hdr = {}; | |
| 332 | + ssize_t ret = 0; | |
| 332 | 333 | |
| 333 | 334 | /* |
| 334 | 335 | * Return -ENXIO during migration |
| @@ -376,7 +377,7 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 41 unmodified lines | ||
| 376 | 377 | |
| 377 | 378 | ret = copy_to_user(buf, &hdr, HVPIPE_HDR_LEN); |
| 378 | 379 | if (ret) |
| 379 | − return ret; | |
| 380 | + return -EFAULT; | |
| 380 | 381 | |
| 381 | 382 | /* |
| 382 | 383 | * Message event has payload, so get the payload with |
| @@ -385,19 +386,23 @@ static ssize_t papr_hvpipe_handle_read(struct file *file, · 2 unmodified lines | ||
| 385 | 386 | if (hdr.flags & HVPIPE_MSG_AVAILABLE) { |
| 386 | 387 | ret = hvpipe_rtas_recv_msg(buf + HVPIPE_HDR_LEN, |
| 387 | 388 | size - HVPIPE_HDR_LEN); |
| 388 | − if (ret > 0) { | |
| 389 | + /* | |
| 390 | + * Always clear MSG_AVAILABLE once the RTAS call has drained | |
| 391 | + * the message, regardless of whether copy_to_user succeeded. | |
| 392 | + */ | |
| 393 | + if (ret >= 0 || ret == -EFAULT) | |
| 389 | 394 | src_info->hvpipe_status &= ~HVPIPE_MSG_AVAILABLE; |
| 390 | − ret += HVPIPE_HDR_LEN; | |
| 391 | − } | |
| 392 | 395 | } else if (hdr.flags & HVPIPE_LOST_CONNECTION) { |
| 393 | 396 | /* |
| 394 | 397 | * Hypervisor is closing the pipe for the specific |
| 395 | 398 | * source. So notify user space. |
| 396 | 399 | */ |
| 397 | 400 | src_info->hvpipe_status &= ~HVPIPE_LOST_CONNECTION; |
| 398 | − ret = HVPIPE_HDR_LEN; | |
| 399 | 401 | } |
| 400 | 402 | |
| 403 | + if (ret >= 0) | |
| 404 | + ret += HVPIPE_HDR_LEN; | |
| 405 | + | |
| 401 | 406 | return ret; |
| 402 | 407 | } |
| 403 | 408 | |
| @@ -444,16 +449,18 @@ static int papr_hvpipe_handle_release(struct inode *inode, · 40 unmodified lines | ||
| 444 | 449 | struct file *file) |
| 445 | 450 | { |
| 446 | 451 | struct hvpipe_source_info *src_info; |
| 452 | + unsigned long flags; | |
| 447 | 453 | |
| 448 | 454 | /* |
| 449 | 455 | * Hold the lock, remove source from src_list, reset the |
| 450 | 456 | * hvpipe status and release the lock to prevent any race |
| 451 | 457 | * with message event IRQ. |
| 452 | 458 | */ |
| 453 | − spin_lock(&hvpipe_src_list_lock); | |
| 459 | + spin_lock_irqsave(&hvpipe_src_list_lock, flags); | |
| 454 | 460 | src_info = file->private_data; |
| 455 | 461 | list_del(&src_info->list); |
| 456 | 462 | file->private_data = NULL; |
| 463 | + spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | |
| 457 | 464 | /* |
| 458 | 465 | * If the pipe for this specific source has any pending |
| 459 | 466 | * payload, issue recv HVPIPE RTAS so that pipe will not |
| @@ -461,10 +468,8 @@ static int papr_hvpipe_handle_release(struct inode *inode, · 1 unmodified line | ||
| 461 | 468 | */ |
| 462 | 469 | if (src_info->hvpipe_status & HVPIPE_MSG_AVAILABLE) { |
| 463 | 470 | src_info->hvpipe_status = 0; |
| 464 | − spin_unlock(&hvpipe_src_list_lock); | |
| 465 | 471 | hvpipe_rtas_recv_msg(NULL, 0); |
| 466 | − } else | |
| 467 | − spin_unlock(&hvpipe_src_list_lock); | |
| 472 | + } | |
| 468 | 473 | |
| 469 | 474 | kfree(src_info); |
| 470 | 475 | return 0; |
| @@ -479,50 +484,53 @@ static const struct file_operations papr_hvpipe_handle_ops = { · 8 unmodified lines | ||
| 479 | 484 | |
| 480 | 485 | static int papr_hvpipe_dev_create_handle(u32 srcID) |
| 481 | 486 | { |
| 482 | − struct hvpipe_source_info *src_info __free(kfree) = NULL; | |
| 483 | − | |
| 484 | − spin_lock(&hvpipe_src_list_lock); | |
| 485 | − /* | |
| 486 | − * Do not allow more than one process communicates with | |
| 487 | − * each source. | |
| 488 | − */ | |
| 489 | − src_info = hvpipe_find_source(srcID); | |
| 490 | − if (src_info) { | |
| 491 | − spin_unlock(&hvpipe_src_list_lock); | |
| 492 | − pr_err("pid(%d) is already using the source(%d)\n", | |
| 493 | − src_info->tsk->pid, srcID); | |
| 494 | − return -EALREADY; | |
| 495 | − } | |
| 496 | − spin_unlock(&hvpipe_src_list_lock); | |
| 487 | + struct hvpipe_source_info *src_info; | |
| 488 | + int fd; | |
| 489 | + unsigned long flags; | |
| 497 | 490 | |
| 498 | 491 | src_info = kzalloc_obj(*src_info, GFP_KERNEL_ACCOUNT); |
| 499 | 492 | if (!src_info) |
| 500 | 493 | return -ENOMEM; |
| 501 | 494 | |
| 502 | 495 | src_info->srcID = srcID; |
| 503 | − src_info->tsk = current; | |
| 504 | 496 | init_waitqueue_head(&src_info->recv_wqh); |
| 505 | 497 | |
| 506 | − FD_PREPARE(fdf, O_RDONLY | O_CLOEXEC, | |
| 507 | − anon_inode_getfile("[papr-hvpipe]", &papr_hvpipe_handle_ops, | |
| 508 | − (void *)src_info, O_RDWR)); | |
| 509 | − if (fdf.err) | |
| 510 | − return fdf.err; | |
| 511 | − | |
| 512 | − retain_and_null_ptr(src_info); | |
| 513 | − spin_lock(&hvpipe_src_list_lock); | |
| 514 | 498 | /* |
| 515 | − * If two processes are executing ioctl() for the same | |
| 516 | − * source ID concurrently, prevent the second process to | |
| 517 | − * acquire FD. | |
| 499 | + * Do not allow more than one process communicates with | |
| 500 | + * each source. | |
| 518 | 501 | */ |
| 502 | + spin_lock_irqsave(&hvpipe_src_list_lock, flags); | |
| 519 | 503 | if (hvpipe_find_source(srcID)) { |
| 520 | − spin_unlock(&hvpipe_src_list_lock); | |
| 504 | + spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | |
| 505 | + pr_err("pid(%s:%d) could not get the source(%d)\n", | |
| 506 | + current->comm, task_pid_nr(current), srcID); | |
| 507 | + kfree(src_info); | |
| 521 | 508 | return -EALREADY; |
| 522 | 509 | } |
| 523 | 510 | list_add(&src_info->list, &hvpipe_src_list); |
| 524 | − spin_unlock(&hvpipe_src_list_lock); | |
| 525 | − return fd_publish(fdf); | |
| 511 | + spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | |
| 512 | + | |
| 513 | + fd = FD_ADD(O_RDONLY | O_CLOEXEC, | |
| 514 | + anon_inode_getfile("[papr-hvpipe]", &papr_hvpipe_handle_ops, | |
| 515 | + (void *)src_info, O_RDWR)); | |
| 516 | + if (fd < 0) { | |
| 517 | + spin_lock_irqsave(&hvpipe_src_list_lock, flags); | |
| 518 | + list_del(&src_info->list); | |
| 519 | + spin_unlock_irqrestore(&hvpipe_src_list_lock, flags); | |
| 520 | + /* | |
| 521 | + * if we fail to add FD, that means no userspace program is | |
| 522 | + * polling. In that case if there is a msg pending because the | |
| 523 | + * interrupt was fired after the src_info was added to the | |
| 524 | + * global list, then let's consume it here, to unblock the | |
| 525 | + * hvpipe | |
| 526 | + */ | |
| 527 | + if (src_info->hvpipe_status & HVPIPE_MSG_AVAILABLE) | |
| 528 | + hvpipe_rtas_recv_msg(NULL, 0); | |
| 529 | + kfree(src_info); | |
| 530 | + return fd; | |
| 531 | + } | |
| 532 | + | |
| 533 | + return fd; | |
| 526 | 534 | } |
| 527 | 535 | |
| 528 | 536 | /* |
| @@ -685,20 +693,19 @@ static int __init enable_hvpipe_IRQ(void) · 156 unmodified lines | ||
| 685 | 693 | struct device_node *np; |
| 686 | 694 | |
| 687 | 695 | hvpipe_check_exception_token = rtas_function_token(RTAS_FN_CHECK_EXCEPTION); |
| 688 | − if (hvpipe_check_exception_token == RTAS_UNKNOWN_SERVICE) | |
| 696 | + if (hvpipe_check_exception_token == RTAS_UNKNOWN_SERVICE) | |
| 689 | 697 | return -ENODEV; |
| 690 | 698 | |
| 691 | 699 | /* hvpipe events */ |
| 692 | 700 | np = of_find_node_by_path("/event-sources/ibm,hvpipe-msg-events"); |
| 693 | − if (np != NULL) { | |
| 694 | − request_event_sources_irqs(np, hvpipe_event_interrupt, | |
| 695 | − "HPIPE_EVENT"); | |
| 696 | − of_node_put(np); | |
| 697 | − } else { | |
| 698 | − pr_err("Can not enable hvpipe event IRQ\n"); | |
| 701 | + if (!np) { | |
| 702 | + pr_err("No device node found, could not enable hvpipe event IRQ\n"); | |
| 699 | 703 | return -ENODEV; |
| 700 | 704 | } |
| 701 | 705 | |
| 706 | + request_event_sources_irqs(np, hvpipe_event_interrupt, "HPIPE_EVENT"); | |
| 707 | + of_node_put(np); | |
| 708 | + | |
| 702 | 709 | return 0; |
| 703 | 710 | } |
| 704 | 711 | |
| @@ -775,23 +782,29 @@ static int __init papr_hvpipe_init(void) · 70 unmodified lines | ||
| 775 | 782 | } |
| 776 | 783 | |
| 777 | 784 | ret = enable_hvpipe_IRQ(); |
| 778 | − if (!ret) { | |
| 779 | − ret = set_hvpipe_sys_param(1); | |
| 780 | − if (!ret) | |
| 781 | − ret = misc_register(&papr_hvpipe_dev); | |
| 782 | − } | |
| 785 | + if (ret) | |
| 786 | + goto out_wq; | |
| 783 | 787 | |
| 784 | − if (!ret) { | |
| 785 | − pr_info("hvpipe feature is enabled\n"); | |
| 786 | − hvpipe_feature = true; | |
| 787 | − return 0; | |
| 788 | − } | |
| 788 | + ret = misc_register(&papr_hvpipe_dev); | |
| 789 | + if (ret) | |
| 790 | + goto out_wq; | |
| 789 | 791 | |
| 790 | − pr_err("hvpipe feature is not enabled %d\n", ret); | |
| 792 | + ret = set_hvpipe_sys_param(1); | |
| 793 | + if (ret) | |
| 794 | + goto out_misc; | |
| 795 | + | |
| 796 | + pr_info("hvpipe feature is enabled\n"); | |
| 797 | + hvpipe_feature = true; | |
| 798 | + return 0; | |
| 799 | + | |
| 800 | +out_misc: | |
| 801 | + misc_deregister(&papr_hvpipe_dev); | |
| 802 | +out_wq: | |
| 791 | 803 | destroy_workqueue(papr_hvpipe_wq); |
| 792 | 804 | out: |
| 793 | 805 | kfree(papr_hvpipe_work); |
| 794 | 806 | papr_hvpipe_work = NULL; |
| 807 | + pr_err("hvpipe feature is not enabled %d\n", ret); | |
| 795 | 808 | return ret; |
| 796 | 809 | } |
| 797 | 810 | machine_device_initcall(pseries, papr_hvpipe_init); |
| @@ -21,7 +21,6 @@ struct hvpipe_source_info { · 20 unmodified lines | ||
| 21 | 21 | u32 srcID; |
| 22 | 22 | u32 hvpipe_status; |
| 23 | 23 | wait_queue_head_t recv_wqh; /* wake up poll() waitq */ |
| 24 | − struct task_struct *tsk; | |
| 25 | 24 | }; |
| 26 | 25 | |
| 27 | 26 | /* |
| @@ -937,6 +937,28 @@ config RISCV_VECTOR_MISALIGNED · 936 unmodified lines | ||
| 937 | 937 | help |
| 938 | 938 | Enable detecting support for vector misaligned loads and stores. |
| 939 | 939 | |
| 940 | +config RISCV_SBI_FWFT_DELEGATE_MISALIGNED | |
| 941 | + bool "Request firmware delegation of unaligned access exceptions" | |
| 942 | + depends on RISCV_SBI | |
| 943 | + depends on NONPORTABLE | |
| 944 | + help | |
| 945 | + Use SBI FWFT to request delegation of load address misaligned and | |
| 946 | + store address misaligned exceptions, if possible, and prefer Linux | |
| 947 | + kernel emulation of these accesses to firmware emulation. | |
| 948 | + | |
| 949 | + Unfortunately, Linux's emulation is still incomplete. Namely, it | |
| 950 | + currently does not handle vector instructions and KVM guest accesses. | |
| 951 | + On platforms where these accesses would have been handled by firmware, | |
| 952 | + enabling this causes unexpected kernel oopses, userspaces crashes and | |
| 953 | + KVM guest crashes. If you are sure that these are not a problem for | |
| 954 | + your platform, you can say Y here, which may improve performance. | |
| 955 | + | |
| 956 | + Saying N here will not worsen emulation support for unaligned accesses | |
| 957 | + even in the case where the firmware also has incomplete support. It | |
| 958 | + simply keeps the firmware's emulation enabled. | |
| 959 | + | |
| 960 | + If you don't know what to do here, say N. | |
| 961 | + | |
| 940 | 962 | choice |
| 941 | 963 | prompt "Unaligned Accesses Support" |
| 942 | 964 | default RISCV_PROBE_UNALIGNED_ACCESS |
| @@ -101,16 +101,6 @@ · 100 unmodified lines | ||
| 101 | 101 | status = "okay"; |
| 102 | 102 | }; |
| 103 | 103 | |
| 104 | −&i2c0 { | |
| 105 | − pinctrl-names = "default"; | |
| 106 | − pinctrl-0 = <&i2c0_fabric>; | |
| 107 | −}; | |
| 108 | − | |
| 109 | −&i2c1 { | |
| 110 | − pinctrl-names = "default"; | |
| 111 | − pinctrl-0 = <&i2c1_mssio>; | |
| 112 | −}; | |
| 113 | − | |
| 114 | 104 | &mmuart1 { |
| 115 | 105 | pinctrl-names = "default"; |
| 116 | 106 | pinctrl-0 = <&uart1_fabric>; |
| @@ -14,6 +14,16 @@ · 13 unmodified lines | ||
| 14 | 14 | "microchip,mpfs"; |
| 15 | 15 | }; |
| 16 | 16 | |
| 17 | +&i2c0 { | |
| 18 | + pinctrl-names = "default"; | |
| 19 | + pinctrl-0 = <&i2c0_fabric>; | |
| 20 | +}; | |
| 21 | + | |
| 22 | +&i2c1 { | |
| 23 | + pinctrl-names = "default"; | |
| 24 | + pinctrl-0 = <&i2c1_mssio>; | |
| 25 | +}; | |
| 26 | + | |
| 17 | 27 | &syscontroller { |
| 18 | 28 | microchip,bitstream-flash = <&sys_ctrl_flash>; |
| 19 | 29 | }; |
| @@ -11,3 +11,22 @@ · 10 unmodified lines | ||
| 11 | 11 | "microchip,mpfs-icicle-kit", |
| 12 | 12 | "microchip,mpfs"; |
| 13 | 13 | }; |
| 14 | + | |
| 15 | +&i2c0 { | |
| 16 | + pinctrl-names = "default"; | |
| 17 | + pinctrl-0 = <&i2c0_fabric>; | |
| 18 | +}; | |
| 19 | + | |
| 20 | +/* | |
| 21 | + * Due to silicon errata, routing via MSS IOs doesn't work on ES devices. | |
| 22 | + * Instead, i2c1, appearing on B1/C1, which are normally MSS IOs, is routed | |
| 23 | + * via the fabric and back to B1/C1 via "fabric-test" functionality. | |
| 24 | + * This is done silently by Libero, so the iomux0 setting for i2c1 has to | |
| 25 | + * be fabric IO, despite tooling etc saying that MSS IOs are used. | |
| 26 | + * | |
| 27 | + * See Section 3.3 of https://ww1.microchip.com/downloads/aemDocuments/documents/FPGA/ProductDocuments/Errata/polarfiresoc/microsemi_polarfire_soc_fpga_egineering_samples_errata_er0219_v1.pdf | |
| 28 | + */ | |
| 29 | +&i2c1 { | |
| 30 | + pinctrl-names = "default"; | |
| 31 | + pinctrl-0 = <&i2c1_fabric>; | |
| 32 | +}; | |
| @@ -135,29 +135,6 @@ · 134 unmodified lines | ||
| 135 | 135 | clock-frequency = <49152000>; |
| 136 | 136 | }; |
| 137 | 137 | |
| 138 | −&camss { | |
| 139 | − assigned-clocks = <&ispcrg JH7110_ISPCLK_DOM4_APB_FUNC>, | |
| 140 | − <&ispcrg JH7110_ISPCLK_MIPI_RX0_PXL>; | |
| 141 | − assigned-clock-rates = <49500000>, <198000000>; | |
| 142 | − | |
| 143 | − ports { | |
| 144 | − #address-cells = <1>; | |
| 145 | − #size-cells = <0>; | |
| 146 | − | |
| 147 | − port@0 { | |
| 148 | − reg = <0>; | |
| 149 | − }; | |
| 150 | − | |
| 151 | − port@1 { | |
| 152 | − reg = <1>; | |
| 153 | − | |
| 154 | − camss_from_csi2rx: endpoint { | |
| 155 | − remote-endpoint = <&csi2rx_to_camss>; | |
| 156 | − }; | |
| 157 | − }; | |
| 158 | − }; | |
| 159 | −}; | |
| 160 | − | |
| 161 | 138 | &csi2rx { |
| 162 | 139 | assigned-clocks = <&ispcrg JH7110_ISPCLK_VIN_SYS>; |
| 163 | 140 | assigned-clock-rates = <297000000>; |
| @@ -175,9 +152,7 @@ · 11 unmodified lines | ||
| 175 | 152 | port@1 { |
| 176 | 153 | reg = <1>; |
| 177 | 154 | |
| 178 | − csi2rx_to_camss: endpoint { | |
| 179 | − remote-endpoint = <&camss_from_csi2rx>; | |
| 180 | − }; | |
| 155 | + /* remote CAMSS endpoint */ | |
| 181 | 156 | }; |
| 182 | 157 | }; |
| 183 | 158 | }; |
| @@ -1199,34 +1199,6 @@ · 1198 unmodified lines | ||
| 1199 | 1199 | #phy-cells = <0>; |
| 1200 | 1200 | }; |
| 1201 | 1201 | |
| 1202 | − camss: isp@19840000 { | |
| 1203 | − compatible = "starfive,jh7110-camss"; | |
| 1204 | − reg = <0x0 0x19840000 0x0 0x10000>, | |
| 1205 | − <0x0 0x19870000 0x0 0x30000>; | |
| 1206 | − reg-names = "syscon", "isp"; | |
| 1207 | − clocks = <&ispcrg JH7110_ISPCLK_DOM4_APB_FUNC>, | |
| 1208 | − <&ispcrg JH7110_ISPCLK_ISPV2_TOP_WRAPPER_C>, | |
| 1209 | − <&ispcrg JH7110_ISPCLK_DVP_INV>, | |
| 1210 | − <&ispcrg JH7110_ISPCLK_VIN_P_AXI_WR>, | |
| 1211 | − <&ispcrg JH7110_ISPCLK_MIPI_RX0_PXL>, | |
| 1212 | − <&syscrg JH7110_SYSCLK_ISP_TOP_CORE>, | |
| 1213 | − <&syscrg JH7110_SYSCLK_ISP_TOP_AXI>; | |
| 1214 | − clock-names = "apb_func", "wrapper_clk_c", "dvp_inv", | |
| 1215 | − "axiwr", "mipi_rx0_pxl", "ispcore_2x", | |
| 1216 | − "isp_axi"; | |
| 1217 | − resets = <&ispcrg JH7110_ISPRST_ISPV2_TOP_WRAPPER_P>, | |
| 1218 | − <&ispcrg JH7110_ISPRST_ISPV2_TOP_WRAPPER_C>, | |
| 1219 | − <&ispcrg JH7110_ISPRST_VIN_P_AXI_RD>, | |
| 1220 | − <&ispcrg JH7110_ISPRST_VIN_P_AXI_WR>, | |
| 1221 | − <&syscrg JH7110_SYSRST_ISP_TOP>, | |
| 1222 | − <&syscrg JH7110_SYSRST_ISP_TOP_AXI>; | |
| 1223 | − reset-names = "wrapper_p", "wrapper_c", "axird", | |
| 1224 | − "axiwr", "isp_top_n", "isp_top_axi"; | |
| 1225 | − power-domains = <&pwrc JH7110_PD_ISP>; | |
| 1226 | − interrupts = <92>, <87>, <90>, <88>; | |
| 1227 | − status = "disabled"; | |
| 1228 | − }; | |
| 1229 | − | |
| 1230 | 1202 | voutcrg: clock-controller@295c0000 { |
| 1231 | 1203 | compatible = "starfive,jh7110-voutcrg"; |
| 1232 | 1204 | reg = <0x0 0x295c0000 0x0 0x10000>; |
| @@ -57,7 +57,7 @@ void mips_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, · 56 unmodified lines | ||
| 57 | 57 | } |
| 58 | 58 | |
| 59 | 59 | tmp = (1U << alt->patch_id); |
| 60 | − if (cpu_req_errata && tmp) { | |
| 60 | + if (cpu_req_errata & tmp) { | |
| 61 | 61 | mutex_lock(&text_mutex); |
| 62 | 62 | patch_text_nosync(ALT_OLD_PTR(alt), ALT_ALT_PTR(alt), |
| 63 | 63 | alt->alt_len); |
| @@ -14,5 +14,6 @@ generic-y += ticket_spinlock.h · 13 unmodified lines | ||
| 14 | 14 | generic-y += qrwlock.h |
| 15 | 15 | generic-y += qrwlock_types.h |
| 16 | 16 | generic-y += qspinlock.h |
| 17 | +generic-y += ring_buffer.h | |
| 17 | 18 | generic-y += user.h |
| 18 | 19 | generic-y += vmlinux.lds.h |
| @@ -32,6 +32,10 @@ asmlinkage long __riscv_sys_ni_syscall(const struct pt_regs *); · 31 unmodified lines | ||
| 32 | 32 | __diag_push(); \ |
| 33 | 33 | __diag_ignore(GCC, 8, "-Wattribute-alias", \ |
| 34 | 34 | "Type aliasing is used to sanitize syscall arguments"); \ |
| 35 | + __diag_ignore(clang, 23, "-Wunknown-warning-option", \ | |
| 36 | + "Avoid breaking versions without -Wattribute-alias"); \ | |
| 37 | + __diag_ignore(clang, 23, "-Wattribute-alias", \ | |
| 38 | + "Type aliasing is used to sanitize syscall arguments"); \ | |
| 35 | 39 | static long __se_##prefix##name(ulong, ulong, ulong, ulong, ulong, ulong, \ |
| 36 | 40 | ulong) \ |
| 37 | 41 | __attribute__((alias(__stringify(___se_##prefix##name)))); \ |
| @@ -107,6 +107,8 @@ static long compat_restore_sigcontext(struct pt_regs *regs, · 106 unmodified lines | ||
| 107 | 107 | |
| 108 | 108 | /* sc_regs is structured the same as the start of pt_regs */ |
| 109 | 109 | err = __copy_from_user(&cregs, &sc->sc_regs, sizeof(sc->sc_regs)); |
| 110 | + if (unlikely(err)) | |
| 111 | + return err; | |
| 110 | 112 | |
| 111 | 113 | cregs_to_regs(&cregs, regs); |
| 112 | 114 | |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0 */ |
| 2 | 2 | /* Copyright (C) 2023 Rivos Inc. */ |
| 3 | 3 | |
| 4 | +#include <linux/cfi_types.h> | |
| 4 | 5 | #include <linux/linkage.h> |
| 5 | 6 | #include <asm/asm.h> |
| 6 | 7 | |
| @@ -9,7 +10,7 @@ · 2 unmodified lines | ||
| 9 | 10 | /* void __riscv_copy_words_unaligned(void *, const void *, size_t) */ |
| 10 | 11 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ |
| 11 | 12 | /* Note: The size is truncated to a multiple of 8 * SZREG */ |
| 12 | −SYM_FUNC_START(__riscv_copy_words_unaligned) | |
| 13 | +SYM_TYPED_FUNC_START(__riscv_copy_words_unaligned) | |
| 13 | 14 | andi a4, a2, ~((8*SZREG)-1) |
| 14 | 15 | beqz a4, 2f |
| 15 | 16 | add a3, a1, a4 |
| @@ -41,7 +42,7 @@ SYM_FUNC_END(__riscv_copy_words_unaligned) · 25 unmodified lines | ||
| 41 | 42 | /* void __riscv_copy_bytes_unaligned(void *, const void *, size_t) */ |
| 42 | 43 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ |
| 43 | 44 | /* Note: The size is truncated to a multiple of 8 */ |
| 44 | −SYM_FUNC_START(__riscv_copy_bytes_unaligned) | |
| 45 | +SYM_TYPED_FUNC_START(__riscv_copy_bytes_unaligned) | |
| 45 | 46 | andi a4, a2, ~(8-1) |
| 46 | 47 | beqz a4, 2f |
| 47 | 48 | add a3, a1, a4 |
| @@ -896,10 +896,8 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) · 895 unmodified lines | ||
| 896 | 896 | * CPU cores with the ratified spec will contain non-zero |
| 897 | 897 | * marchid. |
| 898 | 898 | */ |
| 899 | − if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) { | |
| 900 | − this_hwcap &= ~isa2hwcap[RISCV_ISA_EXT_v]; | |
| 899 | + if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) | |
| 901 | 900 | clear_bit(RISCV_ISA_EXT_v, source_isa); |
| 902 | − } | |
| 903 | 901 | |
| 904 | 902 | riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap); |
| 905 | 903 | |
| @@ -1104,16 +1102,16 @@ early_param("riscv_isa_fallback", riscv_isa_fallback_setup); · 198 unmodified lines | ||
| 1104 | 1102 | void __init riscv_fill_hwcap(void) |
| 1105 | 1103 | { |
| 1106 | 1104 | char print_str[NUM_ALPHA_EXTS + 1]; |
| 1107 | − unsigned long isa2hwcap[26] = {0}; | |
| 1105 | + unsigned long isa2hwcap[RISCV_ISA_EXT_BASE] = {0}; | |
| 1108 | 1106 | int i, j; |
| 1109 | 1107 | |
| 1110 | − isa2hwcap['i' - 'a'] = COMPAT_HWCAP_ISA_I; | |
| 1111 | − isa2hwcap['m' - 'a'] = COMPAT_HWCAP_ISA_M; | |
| 1112 | − isa2hwcap['a' - 'a'] = COMPAT_HWCAP_ISA_A; | |
| 1113 | − isa2hwcap['f' - 'a'] = COMPAT_HWCAP_ISA_F; | |
| 1114 | − isa2hwcap['d' - 'a'] = COMPAT_HWCAP_ISA_D; | |
| 1115 | − isa2hwcap['c' - 'a'] = COMPAT_HWCAP_ISA_C; | |
| 1116 | − isa2hwcap['v' - 'a'] = COMPAT_HWCAP_ISA_V; | |
| 1108 | + isa2hwcap[RISCV_ISA_EXT_i] = COMPAT_HWCAP_ISA_I; | |
| 1109 | + isa2hwcap[RISCV_ISA_EXT_m] = COMPAT_HWCAP_ISA_M; | |
| 1110 | + isa2hwcap[RISCV_ISA_EXT_a] = COMPAT_HWCAP_ISA_A; | |
| 1111 | + isa2hwcap[RISCV_ISA_EXT_f] = COMPAT_HWCAP_ISA_F; | |
| 1112 | + isa2hwcap[RISCV_ISA_EXT_d] = COMPAT_HWCAP_ISA_D; | |
| 1113 | + isa2hwcap[RISCV_ISA_EXT_c] = COMPAT_HWCAP_ISA_C; | |
| 1114 | + isa2hwcap[RISCV_ISA_EXT_v] = COMPAT_HWCAP_ISA_V; | |
| 1117 | 1115 | |
| 1118 | 1116 | if (!acpi_disabled) { |
| 1119 | 1117 | riscv_fill_hwcap_from_isa_string(isa2hwcap); |
| @@ -577,8 +577,8 @@ static int compat_riscv_gpr_set(struct task_struct *target, · 576 unmodified lines | ||
| 577 | 577 | struct compat_user_regs_struct cregs; |
| 578 | 578 | |
| 579 | 579 | ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &cregs, 0, -1); |
| 580 | − | |
| 581 | − cregs_to_regs(&cregs, task_pt_regs(target)); | |
| 580 | + if (!ret) | |
| 581 | + cregs_to_regs(&cregs, task_pt_regs(target)); | |
| 582 | 582 | |
| 583 | 583 | return ret; |
| 584 | 584 | } |
| @@ -584,7 +584,7 @@ static int cpu_online_check_unaligned_access_emulated(unsigned int cpu) · 583 unmodified lines | ||
| 584 | 584 | |
| 585 | 585 | static bool misaligned_traps_delegated; |
| 586 | 586 | |
| 587 | −#ifdef CONFIG_RISCV_SBI | |
| 587 | +#if defined(CONFIG_RISCV_SBI_FWFT_DELEGATE_MISALIGNED) | |
| 588 | 588 | |
| 589 | 589 | static int cpu_online_sbi_unaligned_setup(unsigned int cpu) |
| 590 | 590 | { |
| @@ -109,15 +109,16 @@ void set_indir_lp_lock(struct task_struct *task, bool lock) · 108 unmodified lines | ||
| 109 | 109 | task->thread_info.user_cfi_state.ufcfi_locked = lock; |
| 110 | 110 | } |
| 111 | 111 | /* |
| 112 | − * If size is 0, then to be compatible with regular stack we want it to be as big as | |
| 113 | − * regular stack. Else PAGE_ALIGN it and return back | |
| 112 | + * The shadow stack only stores the return address and not any variables | |
| 113 | + * this should be more than sufficient for most applications. | |
| 114 | + * Else PAGE_ALIGN it and return back | |
| 114 | 115 | */ |
| 115 | 116 | static unsigned long calc_shstk_size(unsigned long size) |
| 116 | 117 | { |
| 117 | 118 | if (size) |
| 118 | 119 | return PAGE_ALIGN(size); |
| 119 | 120 | |
| 120 | − return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); | |
| 121 | + return PAGE_ALIGN(min(rlimit(RLIMIT_STACK) / 2, SZ_2G)); | |
| 121 | 122 | } |
| 122 | 123 | |
| 123 | 124 | /* |
| @@ -2,6 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | /* Copyright (C) 2024 Rivos Inc. */ |
| 3 | 3 | |
| 4 | 4 | #include <linux/args.h> |
| 5 | +#include <linux/cfi_types.h> | |
| 5 | 6 | #include <linux/linkage.h> |
| 6 | 7 | #include <asm/asm.h> |
| 7 | 8 | |
| @@ -16,7 +17,7 @@ · 8 unmodified lines | ||
| 16 | 17 | /* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ |
| 17 | 18 | /* Performs a memcpy without aligning buffers, using word loads and stores. */ |
| 18 | 19 | /* Note: The size is truncated to a multiple of WORD_EEW */ |
| 19 | −SYM_FUNC_START(__riscv_copy_vec_words_unaligned) | |
| 20 | +SYM_TYPED_FUNC_START(__riscv_copy_vec_words_unaligned) | |
| 20 | 21 | andi a4, a2, ~(WORD_EEW-1) |
| 21 | 22 | beqz a4, 2f |
| 22 | 23 | add a3, a1, a4 |
| @@ -38,7 +39,7 @@ SYM_FUNC_END(__riscv_copy_vec_words_unaligned) · 15 unmodified lines | ||
| 38 | 39 | /* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ |
| 39 | 40 | /* Performs a memcpy without aligning buffers, using only byte accesses. */ |
| 40 | 41 | /* Note: The size is truncated to a multiple of 8 */ |
| 41 | −SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) | |
| 42 | +SYM_TYPED_FUNC_START(__riscv_copy_vec_bytes_unaligned) | |
| 42 | 43 | andi a4, a2, ~(8-1) |
| 43 | 44 | beqz a4, 2f |
| 44 | 45 | add a3, a1, a4 |
| @@ -415,7 +415,6 @@ int kvm_riscv_vcpu_mmio_load(struct kvm_vcpu *vcpu, struct kvm_run *run, · 414 unmodified lines | ||
| 415 | 415 | shift = 8 * (sizeof(ulong) - len); |
| 416 | 416 | } else if ((insn & INSN_MASK_LBU) == INSN_MATCH_LBU) { |
| 417 | 417 | len = 1; |
| 418 | − shift = 8 * (sizeof(ulong) - len); | |
| 419 | 418 | #ifdef CONFIG_64BIT |
| 420 | 419 | } else if ((insn & INSN_MASK_LD) == INSN_MATCH_LD) { |
| 421 | 420 | len = 8; |
| @@ -649,22 +648,22 @@ int kvm_riscv_vcpu_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · 227 unmodified lines | ||
| 649 | 648 | case 1: |
| 650 | 649 | data8 = *((u8 *)run->mmio.data); |
| 651 | 650 | SET_RD(insn, &vcpu->arch.guest_context, |
| 652 | − (ulong)data8 << shift >> shift); | |
| 651 | + (long)((ulong)data8 << shift) >> shift); | |
| 653 | 652 | break; |
| 654 | 653 | case 2: |
| 655 | 654 | data16 = *((u16 *)run->mmio.data); |
| 656 | 655 | SET_RD(insn, &vcpu->arch.guest_context, |
| 657 | − (ulong)data16 << shift >> shift); | |
| 656 | + (long)((ulong)data16 << shift) >> shift); | |
| 658 | 657 | break; |
| 659 | 658 | case 4: |
| 660 | 659 | data32 = *((u32 *)run->mmio.data); |
| 661 | 660 | SET_RD(insn, &vcpu->arch.guest_context, |
| 662 | − (ulong)data32 << shift >> shift); | |
| 661 | + (long)((ulong)data32 << shift) >> shift); | |
| 663 | 662 | break; |
| 664 | 663 | case 8: |
| 665 | 664 | data64 = *((u64 *)run->mmio.data); |
| 666 | 665 | SET_RD(insn, &vcpu->arch.guest_context, |
| 667 | − (ulong)data64 << shift >> shift); | |
| 666 | + (long)((ulong)data64 << shift) >> shift); | |
| 668 | 667 | break; |
| 669 | 668 | default: |
| 670 | 669 | return -EOPNOTSUPP; |
| @@ -453,8 +453,10 @@ int kvm_riscv_vcpu_pmu_snapshot_set_shmem(struct kvm_vcpu *vcpu, unsigned long s · 452 unmodified lines | ||
| 453 | 453 | } |
| 454 | 454 | |
| 455 | 455 | kvpmu->sdata = kzalloc(snapshot_area_size, GFP_ATOMIC); |
| 456 | − if (!kvpmu->sdata) | |
| 457 | − return -ENOMEM; | |
| 456 | + if (!kvpmu->sdata) { | |
| 457 | + sbiret = SBI_ERR_FAILURE; | |
| 458 | + goto out; | |
| 459 | + } | |
| 458 | 460 | |
| 459 | 461 | /* No need to check writable slot explicitly as kvm_vcpu_write_guest does it internally */ |
| 460 | 462 | if (kvm_vcpu_write_guest(vcpu, saddr, kvpmu->sdata, snapshot_area_size)) { |
| @@ -499,8 +501,10 @@ int kvm_riscv_vcpu_pmu_event_info(struct kvm_vcpu *vcpu, unsigned long saddr_low · 38 unmodified lines | ||
| 499 | 501 | } |
| 500 | 502 | |
| 501 | 503 | einfo = kzalloc(shmem_size, GFP_KERNEL); |
| 502 | − if (!einfo) | |
| 503 | − return -ENOMEM; | |
| 504 | + if (!einfo) { | |
| 505 | + ret = SBI_ERR_FAILURE; | |
| 506 | + goto out; | |
| 507 | + } | |
| 504 | 508 | |
| 505 | 509 | ret = kvm_vcpu_read_guest(vcpu, shmem, einfo, shmem_size); |
| 506 | 510 | if (ret) { |
| @@ -46,7 +46,7 @@ void kvm_riscv_vcpu_record_steal_time(struct kvm_vcpu *vcpu) · 45 unmodified lines | ||
| 46 | 46 | gfn = shmem >> PAGE_SHIFT; |
| 47 | 47 | hva = kvm_vcpu_gfn_to_hva(vcpu, gfn); |
| 48 | 48 | |
| 49 | − if (WARN_ON(kvm_is_error_hva(hva))) { | |
| 49 | + if (kvm_is_error_hva(hva)) { | |
| 50 | 50 | vcpu->arch.sta.shmem = INVALID_GPA; |
| 51 | 51 | return; |
| 52 | 52 | } |
| @@ -55,6 +55,8 @@ static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run, · 54 unmodified lines | ||
| 55 | 55 | |
| 56 | 56 | for_each_set_bit(i, &hmask, BITS_PER_LONG) { |
| 57 | 57 | rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i); |
| 58 | + if (!rvcpu) | |
| 59 | + continue; | |
| 58 | 60 | ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT); |
| 59 | 61 | if (ret < 0) |
| 60 | 62 | break; |
| @@ -792,6 +792,27 @@ static void __init set_mmap_rnd_bits_max(void) · 791 unmodified lines | ||
| 792 | 792 | mmap_rnd_bits_max = MMAP_VA_BITS - PAGE_SHIFT - 3; |
| 793 | 793 | } |
| 794 | 794 | |
| 795 | +static bool __init is_vaddr_valid(unsigned long va) | |
| 796 | +{ | |
| 797 | + unsigned long up = 0; | |
| 798 | + | |
| 799 | + switch (satp_mode) { | |
| 800 | + case SATP_MODE_39: | |
| 801 | + up = 1UL << 38; | |
| 802 | + break; | |
| 803 | + case SATP_MODE_48: | |
| 804 | + up = 1UL << 47; | |
| 805 | + break; | |
| 806 | + case SATP_MODE_57: | |
| 807 | + up = 1UL << 56; | |
| 808 | + break; | |
| 809 | + default: | |
| 810 | + return false; | |
| 811 | + } | |
| 812 | + | |
| 813 | + return (va < up) || (va >= (ULONG_MAX - up + 1)); | |
| 814 | +} | |
| 815 | + | |
| 795 | 816 | /* |
| 796 | 817 | * There is a simple way to determine if 4-level is supported by the |
| 797 | 818 | * underlying hardware: establish 1:1 mapping in 4-level page table mode |
| @@ -833,6 +854,9 @@ static __init void set_satp_mode(uintptr_t dtb_pa) · 35 unmodified lines | ||
| 833 | 854 | set_satp_mode_pmd + PMD_SIZE, |
| 834 | 855 | PMD_SIZE, PAGE_KERNEL_EXEC); |
| 835 | 856 | retry: |
| 857 | + if (!is_vaddr_valid(set_satp_mode_pmd)) | |
| 858 | + goto out; | |
| 859 | + | |
| 836 | 860 | create_pgd_mapping(early_pg_dir, |
| 837 | 861 | set_satp_mode_pmd, |
| 838 | 862 | pgtable_l5_enabled ? |
| @@ -855,6 +879,7 @@ retry: · 16 unmodified lines | ||
| 855 | 879 | disable_pgtable_l4(); |
| 856 | 880 | } |
| 857 | 881 | |
| 882 | +out: | |
| 858 | 883 | memset(early_pg_dir, 0, PAGE_SIZE); |
| 859 | 884 | memset(early_p4d, 0, PAGE_SIZE); |
| 860 | 885 | memset(early_pud, 0, PAGE_SIZE); |
| @@ -7,3 +7,4 @@ generated-y += unistd_nr.h · 6 unmodified lines | ||
| 7 | 7 | generic-y += asm-offsets.h |
| 8 | 8 | generic-y += mcs_spinlock.h |
| 9 | 9 | generic-y += mmzone.h |
| 10 | +generic-y += ring_buffer.h | |
| @@ -1414,6 +1414,9 @@ static inline char *debug_get_user_string(const char __user *user_buf, · 1413 unmodified lines | ||
| 1414 | 1414 | { |
| 1415 | 1415 | char *buffer; |
| 1416 | 1416 | |
| 1417 | + if (!user_len) | |
| 1418 | + return ERR_PTR(-EINVAL); | |
| 1419 | + | |
| 1417 | 1420 | buffer = memdup_user_nul(user_buf, user_len); |
| 1418 | 1421 | if (IS_ERR(buffer)) |
| 1419 | 1422 | return buffer; |
| @@ -1584,6 +1587,11 @@ static int debug_input_flush_fn(debug_info_t *id, struct debug_view *view, · 164 unmodified lines | ||
| 1584 | 1587 | char input_buf[1]; |
| 1585 | 1588 | int rc = user_len; |
| 1586 | 1589 | |
| 1590 | + if (!user_len) { | |
| 1591 | + rc = -EINVAL; | |
| 1592 | + goto out; | |
| 1593 | + } | |
| 1594 | + | |
| 1587 | 1595 | if (user_len > 0x10000) |
| 1588 | 1596 | user_len = 0x10000; |
| 1589 | 1597 | if (*offset != 0) { |
| @@ -186,6 +186,13 @@ static u64 pai_getctr(unsigned long *page, int nr, unsigned long offset) · 185 unmodified lines | ||
| 186 | 186 | return page[nr]; |
| 187 | 187 | } |
| 188 | 188 | |
| 189 | +static void pai_setctr(unsigned long *page, int nr, unsigned long offset, u64 v) | |
| 190 | +{ | |
| 191 | + if (offset) | |
| 192 | + nr += offset / sizeof(*page); | |
| 193 | + page[nr] = v; | |
| 194 | +} | |
| 195 | + | |
| 189 | 196 | /* Read the counter values. Return value from location in CMP. For base |
| 190 | 197 | * event xxx_ALL sum up all events. Returns counter value. |
| 191 | 198 | */ |
| @@ -551,6 +558,8 @@ static void paicrypt_del(struct perf_event *event, int flags) · 359 unmodified lines | ||
| 551 | 558 | /* Create raw data and save it in buffer. Calculate the delta for each |
| 552 | 559 | * counter between this invocation and the last invocation. |
| 553 | 560 | * Returns number of bytes copied. |
| 561 | + * After reading from PAI counter page, save the read value to the old | |
| 562 | + * page to calculate PAI counter deltas. | |
| 554 | 563 | * Saves only entries with positive counter difference of the form |
| 555 | 564 | * 2 bytes: Number of counter |
| 556 | 565 | * 8 bytes: Value of counter |
| @@ -562,16 +571,22 @@ static size_t pai_copy(struct pai_userdata *userdata, unsigned long *page, · 5 unmodified lines | ||
| 562 | 571 | int i, outidx = 0; |
| 563 | 572 | |
| 564 | 573 | for (i = 1; i <= pp->num_avail; i++) { |
| 565 | − u64 val = 0, val_old = 0; | |
| 574 | + u64 val = 0, val_old = 0, val_k = 0, val_old_k = 0; | |
| 566 | 575 | |
| 567 | 576 | if (!exclude_kernel) { |
| 568 | − val += pai_getctr(page, i, pp->kernel_offset); | |
| 569 | − val_old += pai_getctr(page_old, i, pp->kernel_offset); | |
| 577 | + val_k = pai_getctr(page, i, pp->kernel_offset); | |
| 578 | + val_old_k = pai_getctr(page_old, i, pp->kernel_offset); | |
| 579 | + if (val_k != val_old_k) | |
| 580 | + pai_setctr(page_old, i, pp->kernel_offset, val_k); | |
| 570 | 581 | } |
| 571 | 582 | if (!exclude_user) { |
| 572 | − val += pai_getctr(page, i, 0); | |
| 573 | − val_old += pai_getctr(page_old, i, 0); | |
| 583 | + val = pai_getctr(page, i, 0); | |
| 584 | + val_old = pai_getctr(page_old, i, 0); | |
| 585 | + if (val != val_old) | |
| 586 | + pai_setctr(page_old, i, 0, val); | |
| 574 | 587 | } |
| 588 | + val += val_k; | |
| 589 | + val_old += val_old_k; | |
| 575 | 590 | if (val >= val_old) |
| 576 | 591 | val -= val_old; |
| 577 | 592 | else |
| @@ -602,8 +617,6 @@ static size_t pai_copy(struct pai_userdata *userdata, unsigned long *page, · 24 unmodified lines | ||
| 602 | 617 | static int pai_push_sample(size_t rawsize, struct pai_map *cpump, |
| 603 | 618 | struct perf_event *event) |
| 604 | 619 | { |
| 605 | − int idx = PAI_PMU_IDX(event); | |
| 606 | − struct pai_pmu *pp = &pai_pmu[idx]; | |
| 607 | 620 | struct perf_sample_data data; |
| 608 | 621 | struct perf_raw_record raw; |
| 609 | 622 | struct pt_regs regs; |
| @@ -634,8 +647,6 @@ static int pai_push_sample(size_t rawsize, struct pai_map *cpump, · 24 unmodified lines | ||
| 634 | 647 | |
| 635 | 648 | overflow = perf_event_overflow(event, &data, ®s); |
| 636 | 649 | perf_event_update_userpage(event); |
| 637 | − /* Save crypto counter lowcore page after reading event data. */ | |
| 638 | − memcpy((void *)PAI_SAVE_AREA(event), cpump->area, pp->area_size); | |
| 639 | 650 | return overflow; |
| 640 | 651 | } |
| 641 | 652 | |
| @@ -651,7 +662,7 @@ static void pai_have_sample(struct perf_event *event, struct pai_map *cpump) · 9 unmodified lines | ||
| 651 | 662 | rawsize = pai_copy(cpump->save, cpump->area, pp, |
| 652 | 663 | (unsigned long *)PAI_SAVE_AREA(event), |
| 653 | 664 | event->attr.exclude_user, |
| 654 | − event->attr.exclude_kernel); | |
| 665 | + !pp->kernel_offset ? true : event->attr.exclude_kernel); | |
| 655 | 666 | if (rawsize) /* No incremented counters */ |
| 656 | 667 | pai_push_sample(rawsize, cpump, event); |
| 657 | 668 | } |
| @@ -192,17 +192,21 @@ static void tl_to_masks(struct sysinfo_15_1_x *info) · 191 unmodified lines | ||
| 192 | 192 | end = (union topology_entry *)((unsigned long)info + info->length); |
| 193 | 193 | while (tle < end) { |
| 194 | 194 | switch (tle->nl) { |
| 195 | + /* | |
| 196 | + * Adjust drawer_id, book_id, and socked_id so they match the | |
| 197 | + * numbering scheme of e.g. the hardware management console. | |
| 198 | + */ | |
| 195 | 199 | case 3: |
| 196 | 200 | drawer = drawer->next; |
| 197 | − drawer->id = tle->container.id; | |
| 201 | + drawer->id = tle->container.id - 1; | |
| 198 | 202 | break; |
| 199 | 203 | case 2: |
| 200 | 204 | book = book->next; |
| 201 | − book->id = tle->container.id; | |
| 205 | + book->id = tle->container.id - 1; | |
| 202 | 206 | break; |
| 203 | 207 | case 1: |
| 204 | 208 | socket = socket->next; |
| 205 | − socket->id = tle->container.id; | |
| 209 | + socket->id = tle->container.id - 1; | |
| 206 | 210 | break; |
| 207 | 211 | case 0: |
| 208 | 212 | add_cpus_to_mask(&tle->cpu, drawer, book, socket); |
| @@ -267,6 +267,7 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g · 266 unmodified lines | ||
| 267 | 267 | /* No need to take locks as the page table is not installed yet. */ |
| 268 | 268 | pgste_init.prefix_notif = old.s.fc1.prefix_notif; |
| 269 | 269 | pgste_init.vsie_notif = old.s.fc1.vsie_notif; |
| 270 | + pgste_init.vsie_gmem = old.s.fc1.vsie_notif; | |
| 270 | 271 | pgste_init.pcl = uses_skeys && init.h.i; |
| 271 | 272 | dat_init_pgstes(pt, pgste_init.val); |
| 272 | 273 | } else { |
| @@ -145,7 +145,8 @@ union pgste { · 144 unmodified lines | ||
| 145 | 145 | unsigned long cmma_d : 1; /* Dirty flag for CMMA bits */ |
| 146 | 146 | unsigned long prefix_notif : 1; /* Guest prefix invalidation notification */ |
| 147 | 147 | unsigned long vsie_notif : 1; /* Referenced in a shadow table */ |
| 148 | − unsigned long : 5; | |
| 148 | + unsigned long vsie_gmem : 1; /* Contains nested guest memory */ | |
| 149 | + unsigned long : 4; | |
| 149 | 150 | unsigned long : 8; |
| 150 | 151 | }; |
| 151 | 152 | struct { |
| @@ -1445,6 +1445,7 @@ static int _do_shadow_pte(struct gmap *sg, gpa_t raddr, union pte *ptep_h, union · 1444 unmodified lines | ||
| 1445 | 1445 | } else { |
| 1446 | 1446 | pgste = _gmap_ptep_xchg(sg->parent, ptep_h, newpte, pgste, f->gfn, false); |
| 1447 | 1447 | pgste.vsie_notif = 1; |
| 1448 | + pgste.vsie_gmem = 1; | |
| 1448 | 1449 | } |
| 1449 | 1450 | pgste_set_unlock(ptep_h, pgste); |
| 1450 | 1451 | if (rc) |
| @@ -125,7 +125,7 @@ struct gmap *gmap_new_child(struct gmap *parent, gfn_t limit) · 124 unmodified lines | ||
| 125 | 125 | |
| 126 | 126 | int gmap_set_limit(struct gmap *gmap, gfn_t limit) |
| 127 | 127 | { |
| 128 | − struct kvm_s390_mmu_cache *mc; | |
| 128 | + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; | |
| 129 | 129 | int rc, type; |
| 130 | 130 | |
| 131 | 131 | type = gmap_limit_to_type(limit); |
| @@ -142,7 +142,6 @@ int gmap_set_limit(struct gmap *gmap, gfn_t limit) · 10 unmodified lines | ||
| 142 | 142 | rc = dat_set_asce_limit(mc, &gmap->asce, type); |
| 143 | 143 | } while (rc == -ENOMEM); |
| 144 | 144 | |
| 145 | − kvm_s390_free_mmu_cache(mc); | |
| 146 | 145 | return 0; |
| 147 | 146 | } |
| 148 | 147 | |
| @@ -822,8 +821,8 @@ int gmap_ucas_translate(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, gpa_t · 673 unmodified lines | ||
| 822 | 821 | |
| 823 | 822 | int gmap_ucas_map(struct gmap *gmap, gfn_t p_gfn, gfn_t c_gfn, unsigned long count) |
| 824 | 823 | { |
| 825 | − struct kvm_s390_mmu_cache *mc; | |
| 826 | − int rc; | |
| 824 | + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; | |
| 825 | + int rc = 0; | |
| 827 | 826 | |
| 828 | 827 | mc = kvm_s390_new_mmu_cache(); |
| 829 | 828 | if (!mc) |
| @@ -1026,13 +1025,15 @@ int gmap_insert_rmap(struct gmap *sg, gfn_t p_gfn, gfn_t r_gfn, int level) · 196 unmodified lines | ||
| 1026 | 1025 | int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gfn, gfn_t r_gfn, |
| 1027 | 1026 | kvm_pfn_t pfn, int level, bool wr) |
| 1028 | 1027 | { |
| 1028 | + unsigned long bitmask; | |
| 1029 | 1029 | union crste *crstep; |
| 1030 | 1030 | union pgste pgste; |
| 1031 | 1031 | union pte *ptep; |
| 1032 | 1032 | union pte pte; |
| 1033 | 1033 | int flags, rc; |
| 1034 | 1034 | |
| 1035 | − KVM_BUG_ON(!is_shadow(sg), sg->kvm); | |
| 1035 | + if (KVM_BUG_ON(!is_shadow(sg) || level <= TABLE_TYPE_PAGE_TABLE, sg->kvm)) | |
| 1036 | + return -EINVAL; | |
| 1036 | 1037 | lockdep_assert_held(&sg->parent->children_lock); |
| 1037 | 1038 | |
| 1038 | 1039 | flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0); |
| @@ -1041,8 +1042,9 @@ int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gf · 2 unmodified lines | ||
| 1041 | 1042 | if (rc) |
| 1042 | 1043 | return rc; |
| 1043 | 1044 | if (level <= TABLE_TYPE_REGION1) { |
| 1045 | + bitmask = -1UL << (8 + 11 * level); | |
| 1044 | 1046 | scoped_guard(spinlock, &sg->host_to_rmap_lock) |
| 1045 | − rc = gmap_insert_rmap(sg, p_gfn, r_gfn, level); | |
| 1047 | + rc = gmap_insert_rmap(sg, p_gfn, r_gfn & bitmask, level); | |
| 1046 | 1048 | } |
| 1047 | 1049 | if (rc) |
| 1048 | 1050 | return rc; |
| @@ -1143,8 +1145,10 @@ void _gmap_handle_vsie_unshadow_event(struct gmap *parent, gfn_t gfn) · 94 unmodified lines | ||
| 1143 | 1145 | } |
| 1144 | 1146 | scoped_guard(spinlock, &sg->host_to_rmap_lock) |
| 1145 | 1147 | head = radix_tree_delete(&sg->host_to_rmap, gfn); |
| 1146 | − gmap_for_each_rmap_safe(rmap, rnext, head) | |
| 1148 | + gmap_for_each_rmap_safe(rmap, rnext, head) { | |
| 1147 | 1149 | gmap_unshadow_level(sg, rmap->r_gfn, rmap->level); |
| 1150 | + kfree(rmap); | |
| 1151 | + } | |
| 1148 | 1152 | } |
| 1149 | 1153 | } |
| 1150 | 1154 | |
| @@ -167,6 +167,36 @@ static inline bool gmap_unmap_prefix(struct gmap *gmap, gfn_t gfn, gfn_t end) · 166 unmodified lines | ||
| 167 | 167 | return _gmap_unmap_prefix(gmap, gfn, end, false); |
| 168 | 168 | } |
| 169 | 169 | |
| 170 | +/** | |
| 171 | + * pte_needs_unshadow() -- Check if the pte operations triggers unshadowing. | |
| 172 | + * @oldpte: the previous value for the guest pte. | |
| 173 | + * @newpte: the new pte being set. | |
| 174 | + * @pgste: the pgste for the pte entry. | |
| 175 | + * | |
| 176 | + * If the pgste.vsie_notif bit is not set, return false: the page is not | |
| 177 | + * involved in vsie and thus should not trigger an unshadow operation. | |
| 178 | + * | |
| 179 | + * If the pgste.vsie_gmem bit is set, this pte represents shadowed guest | |
| 180 | + * memory. The access rights on g3's memory should be synchronized with g1's | |
| 181 | + * and g2's. Therefore unshadowing is triggered if the new and old pte | |
| 182 | + * differ in protection, or if the new pte is invalid. | |
| 183 | + * | |
| 184 | + * If the pgste.vsie_gmem bit is not set, this pte maps the g2 dat tables | |
| 185 | + * for g3. If the entry becomes writable or absent, it becomes impossible to | |
| 186 | + * guarantee that the shadow mapping will match g2's mapping. In that case, | |
| 187 | + * trigger an unshadow event. | |
| 188 | + * | |
| 189 | + * Return: true if an unshadow event should be triggered, otherwise false. | |
| 190 | + */ | |
| 191 | +static inline bool pte_needs_unshadow(union pte oldpte, union pte newpte, union pgste pgste) | |
| 192 | +{ | |
| 193 | + if (!pgste.vsie_notif) | |
| 194 | + return false; | |
| 195 | + if (pgste.vsie_gmem) | |
| 196 | + return (oldpte.h.p != newpte.h.p) || newpte.h.i; | |
| 197 | + return !newpte.h.p || !newpte.s.pr; | |
| 198 | +} | |
| 199 | + | |
| 170 | 200 | static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, union pte newpte, |
| 171 | 201 | union pgste pgste, gfn_t gfn, bool needs_lock) |
| 172 | 202 | { |
| @@ -180,8 +210,9 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un · 7 unmodified lines | ||
| 180 | 210 | pgste.prefix_notif = 0; |
| 181 | 211 | gmap_unmap_prefix(gmap, gfn, gfn + 1); |
| 182 | 212 | } |
| 183 | − if (pgste.vsie_notif && (ptep->h.p != newpte.h.p || newpte.h.i)) { | |
| 213 | + if (pte_needs_unshadow(*ptep, newpte, pgste)) { | |
| 184 | 214 | pgste.vsie_notif = 0; |
| 215 | + pgste.vsie_gmem = 0; | |
| 185 | 216 | if (needs_lock) |
| 186 | 217 | gmap_handle_vsie_unshadow_event(gmap, gfn); |
| 187 | 218 | else |
| @@ -189,6 +220,7 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un · 1 unmodified line | ||
| 189 | 220 | } |
| 190 | 221 | if (!ptep->s.d && newpte.s.d && !newpte.s.s) |
| 191 | 222 | SetPageDirty(pfn_to_page(newpte.h.pfra)); |
| 223 | + pgste.zero = 0; | |
| 192 | 224 | return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->asce, uses_skeys(gmap)); |
| 193 | 225 | } |
| 194 | 226 | |
| @@ -198,6 +230,30 @@ static inline union pgste gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, uni · 3 unmodified lines | ||
| 198 | 230 | return _gmap_ptep_xchg(gmap, ptep, newpte, pgste, gfn, true); |
| 199 | 231 | } |
| 200 | 232 | |
| 233 | +/** | |
| 234 | + * crste_needs_unshadow() -- Check if the crste operations triggers unshadowing. | |
| 235 | + * @oldcrste: the previous value for the crste. | |
| 236 | + * @newcrste: the new value for the crste. | |
| 237 | + * | |
| 238 | + * If the old crste did not have the vsie_notif bit set, return false: the | |
| 239 | + * page is not involved in vsie and thus should not trigger an unshadow | |
| 240 | + * operation. Conversely, if the bit is set, it can only be g3 memory, since | |
| 241 | + * dat tables are never mapped using large pages. | |
| 242 | + * | |
| 243 | + * Similar to the pgste.vsie_gmem case of pte_needs_unshadow(), if the | |
| 244 | + * protection bit is changing or the new page is invalid, trigger an | |
| 245 | + * unshadow event. Also trigger an unshadow event if the new crste does not | |
| 246 | + * have the vsie_notif bit set. | |
| 247 | + * | |
| 248 | + * Return: true if an unshadow event should be triggered, otherwise false. | |
| 249 | + */ | |
| 250 | +static inline bool crste_needs_unshadow(union crste oldcrste, union crste newcrste) | |
| 251 | +{ | |
| 252 | + if (!oldcrste.s.fc1.vsie_notif) | |
| 253 | + return false; | |
| 254 | + return (newcrste.h.p != oldcrste.h.p) || newcrste.h.i || !newcrste.s.fc1.vsie_notif; | |
| 255 | +} | |
| 256 | + | |
| 201 | 257 | static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, union crste *crstep, |
| 202 | 258 | union crste oldcrste, union crste newcrste, |
| 203 | 259 | gfn_t gfn, bool needs_lock) |
| @@ -216,8 +272,7 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, unio · 12 unmodified lines | ||
| 216 | 272 | newcrste.s.fc1.prefix_notif = 0; |
| 217 | 273 | gmap_unmap_prefix(gmap, gfn, gfn + align); |
| 218 | 274 | } |
| 219 | − if (crste_leaf(oldcrste) && oldcrste.s.fc1.vsie_notif && | |
| 220 | − (newcrste.h.p || newcrste.h.i || !newcrste.s.fc1.vsie_notif)) { | |
| 275 | + if (crste_leaf(oldcrste) && crste_needs_unshadow(oldcrste, newcrste)) { | |
| 221 | 276 | newcrste.s.fc1.vsie_notif = 0; |
| 222 | 277 | if (needs_lock) |
| 223 | 278 | gmap_handle_vsie_unshadow_event(gmap, gfn); |
| @@ -3310,8 +3310,7 @@ static void aen_host_forward(unsigned long si) · 3309 unmodified lines | ||
| 3310 | 3310 | struct zpci_gaite *gaite; |
| 3311 | 3311 | struct kvm *kvm; |
| 3312 | 3312 | |
| 3313 | − gaite = (struct zpci_gaite *)aift->gait + | |
| 3314 | − (si * sizeof(struct zpci_gaite)); | |
| 3313 | + gaite = aift->gait + si; | |
| 3315 | 3314 | if (gaite->count == 0) |
| 3316 | 3315 | return; |
| 3317 | 3316 | if (gaite->aisb != 0) |
| @@ -166,7 +166,7 @@ static int kvm_zpci_set_airq(struct zpci_dev *zdev) · 165 unmodified lines | ||
| 166 | 166 | fib.fmt0.noi = airq_iv_end(zdev->aibv); |
| 167 | 167 | fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector); |
| 168 | 168 | fib.fmt0.aibvo = 0; |
| 169 | − fib.fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); | |
| 169 | + fib.fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; | |
| 170 | 170 | fib.fmt0.aisbo = zdev->aisb & 63; |
| 171 | 171 | fib.gd = zdev->gisa; |
| 172 | 172 | |
| @@ -290,8 +290,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 117 unmodified lines | ||
| 290 | 290 | phys_to_virt(fib->fmt0.aibv)); |
| 291 | 291 | |
| 292 | 292 | spin_lock_irq(&aift->gait_lock); |
| 293 | − gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb * | |
| 294 | − sizeof(struct zpci_gaite)); | |
| 293 | + gaite = aift->gait + zdev->aisb; | |
| 295 | 294 | |
| 296 | 295 | /* If assist not requested, host will get all alerts */ |
| 297 | 296 | if (assist) |
| @@ -309,7 +308,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 11 unmodified lines | ||
| 309 | 308 | |
| 310 | 309 | /* Update guest FIB for re-issue */ |
| 311 | 310 | fib->fmt0.aisbo = zdev->aisb & 63; |
| 312 | − fib->fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); | |
| 311 | + fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; | |
| 313 | 312 | fib->fmt0.isc = gisc; |
| 314 | 313 | |
| 315 | 314 | /* Save some guest fib values in the host for later use */ |
| @@ -357,8 +356,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) · 41 unmodified lines | ||
| 357 | 356 | if (zdev->kzdev->fib.fmt0.aibv == 0) |
| 358 | 357 | goto out; |
| 359 | 358 | spin_lock_irq(&aift->gait_lock); |
| 360 | − gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb * | |
| 361 | − sizeof(struct zpci_gaite)); | |
| 359 | + gaite = aift->gait + zdev->aisb; | |
| 362 | 360 | isc = gaite->gisc; |
| 363 | 361 | gaite->count--; |
| 364 | 362 | if (gaite->count == 0) { |
| @@ -438,7 +438,7 @@ void do_secure_storage_access(struct pt_regs *regs) · 437 unmodified lines | ||
| 438 | 438 | panic("Unexpected PGM 0x3d with TEID bit 61=0"); |
| 439 | 439 | } |
| 440 | 440 | if (is_kernel_fault(regs)) { |
| 441 | − folio = phys_to_folio(addr); | |
| 441 | + folio = virt_to_folio((void *)addr); | |
| 442 | 442 | if (unlikely(!folio_try_get(folio))) |
| 443 | 443 | return; |
| 444 | 444 | rc = uv_convert_from_secure(folio_to_phys(folio)); |
| @@ -3,4 +3,5 @@ generated-y += syscall_table.h · 2 unmodified lines | ||
| 3 | 3 | generic-y += kvm_para.h |
| 4 | 4 | generic-y += mcs_spinlock.h |
| 5 | 5 | generic-y += parport.h |
| 6 | +generic-y += ring_buffer.h | |
| 6 | 7 | generic-y += text-patching.h |
| @@ -7,7 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | /* |
| 8 | 8 | * This is set up by the setup-routine at boot-time |
| 9 | 9 | */ |
| 10 | −extern unsigned char *boot_params_page; | |
| 10 | +extern unsigned char boot_params_page[]; | |
| 11 | 11 | #define PARAM boot_params_page |
| 12 | 12 | |
| 13 | 13 | #define MOUNT_ROOT_RDONLY (*(unsigned long *) (PARAM+0x000)) |
| @@ -4,4 +4,5 @@ generated-y += syscall_table_64.h · 3 unmodified lines | ||
| 4 | 4 | generic-y += agp.h |
| 5 | 5 | generic-y += kvm_para.h |
| 6 | 6 | generic-y += mcs_spinlock.h |
| 7 | +generic-y += ring_buffer.h | |
| 7 | 8 | generic-y += text-patching.h |
| @@ -17,6 +17,7 @@ generic-y += module.lds.h · 16 unmodified lines | ||
| 17 | 17 | generic-y += parport.h |
| 18 | 18 | generic-y += percpu.h |
| 19 | 19 | generic-y += preempt.h |
| 20 | +generic-y += ring_buffer.h | |
| 20 | 21 | generic-y += runtime-const.h |
| 21 | 22 | generic-y += softirq_stack.h |
| 22 | 23 | generic-y += switch_to.h |
| @@ -13,7 +13,7 @@ CFLAGS_REMOVE_syscall_64.o = $(CC_FLAGS_FTRACE) · 12 unmodified lines | ||
| 13 | 13 | CFLAGS_syscall_32.o += -fno-stack-protector |
| 14 | 14 | CFLAGS_syscall_64.o += -fno-stack-protector |
| 15 | 15 | |
| 16 | −obj-y := entry.o entry_$(BITS).o syscall_$(BITS).o | |
| 16 | +obj-y := entry.o entry_$(BITS).o syscall_$(BITS).o common.o | |
| 17 | 17 | |
| 18 | 18 | obj-y += vdso/ |
| 19 | 19 | obj-y += vsyscall/ |
| @@ -0,0 +1,61 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | + | |
| 3 | +#include <linux/entry-common.h> | |
| 4 | +#include <linux/kvm_types.h> | |
| 5 | +#include <linux/hrtimer_rearm.h> | |
| 6 | +#include <asm/fred.h> | |
| 7 | +#include <asm/desc.h> | |
| 8 | + | |
| 9 | +#if IS_ENABLED(CONFIG_KVM_INTEL) | |
| 10 | +/* | |
| 11 | + * On VMX, NMIs and IRQs (as configured by KVM) are acknowledged by hardware as | |
| 12 | + * part of the VM-Exit, i.e. the event itself is consumed as part the VM-Exit. | |
| 13 | + * x86_entry_from_kvm() is invoked by KVM to effectively forward NMIs and IRQs | |
| 14 | + * to the kernel for servicing. On SVM, a.k.a. AMD, the NMI/IRQ VM-Exit is | |
| 15 | + * purely a signal that an NMI/IRQ is pending, i.e. the event that triggered | |
| 16 | + * the VM-Exit is held pending until it's unblocked in the host. | |
| 17 | + */ | |
| 18 | +noinstr void x86_entry_from_kvm(unsigned int event_type, unsigned int vector) | |
| 19 | +{ | |
| 20 | + if (event_type == EVENT_TYPE_EXTINT) { | |
| 21 | +#ifdef CONFIG_X86_64 | |
| 22 | + /* | |
| 23 | + * Use FRED dispatch, even when running IDT. The dispatch | |
| 24 | + * tables are kept in sync between FRED and IDT, and the FRED | |
| 25 | + * dispatch works well with CFI. | |
| 26 | + */ | |
| 27 | + fred_entry_from_kvm(event_type, vector); | |
| 28 | +#else | |
| 29 | + idt_entry_from_kvm(vector); | |
| 30 | +#endif | |
| 31 | + /* | |
| 32 | + * Strictly speaking, only the NMI path requires noinstr. | |
| 33 | + */ | |
| 34 | + instrumentation_begin(); | |
| 35 | + /* | |
| 36 | + * KVM/VMX will dispatch from IRQ-disabled but for a context | |
| 37 | + * that will have IRQs-enabled. This confuses the entry code | |
| 38 | + * and it will not have reprogrammed the timer. Do so now. | |
| 39 | + */ | |
| 40 | + hrtimer_rearm_deferred(); | |
| 41 | + instrumentation_end(); | |
| 42 | + | |
| 43 | + return; | |
| 44 | + } | |
| 45 | + | |
| 46 | + WARN_ON_ONCE(event_type != EVENT_TYPE_NMI); | |
| 47 | + | |
| 48 | +#ifdef CONFIG_X86_64 | |
| 49 | + if (cpu_feature_enabled(X86_FEATURE_FRED)) | |
| 50 | + return fred_entry_from_kvm(event_type, vector); | |
| 51 | +#endif | |
| 52 | + | |
| 53 | + /* | |
| 54 | + * Notably, we must use IDT dispatch for NMI when running in IDT mode. | |
| 55 | + * The FRED NMI context is significantly different and will not work | |
| 56 | + * right (specifically FRED fixed the NMI recursion issue). | |
| 57 | + */ | |
| 58 | + idt_do_nmi_irqoff(); | |
| 59 | +} | |
| 60 | +EXPORT_SYMBOL_FOR_KVM(x86_entry_from_kvm); | |
| 61 | +#endif | |
| @@ -75,3 +75,51 @@ THUNK warn_thunk_thunk, __warn_thunk · 74 unmodified lines | ||
| 75 | 75 | #if defined(CONFIG_STACKPROTECTOR) && defined(CONFIG_SMP) |
| 76 | 76 | EXPORT_SYMBOL(__ref_stack_chk_guard); |
| 77 | 77 | #endif |
| 78 | + | |
| 79 | +#if IS_ENABLED(CONFIG_KVM_INTEL) | |
| 80 | +.macro IDT_DO_EVENT_IRQOFF call_insn call_target | |
| 81 | + /* | |
| 82 | + * Unconditionally create a stack frame, getting the correct RSP on the | |
| 83 | + * stack (for x86-64) would take two instructions anyways, and RBP can | |
| 84 | + * be used to restore RSP to make objtool happy (see below). | |
| 85 | + */ | |
| 86 | + push %_ASM_BP | |
| 87 | + mov %_ASM_SP, %_ASM_BP | |
| 88 | + | |
| 89 | +#ifdef CONFIG_X86_64 | |
| 90 | + /* | |
| 91 | + * Align RSP to a 16-byte boundary (to emulate CPU behavior) before | |
| 92 | + * creating the synthetic interrupt stack frame for the IRQ/NMI. | |
| 93 | + */ | |
| 94 | + and $-16, %rsp | |
| 95 | + push $__KERNEL_DS | |
| 96 | + push %rbp | |
| 97 | +#endif | |
| 98 | + pushf | |
| 99 | + push $__KERNEL_CS | |
| 100 | + \call_insn \call_target | |
| 101 | + | |
| 102 | + /* | |
| 103 | + * "Restore" RSP from RBP, even though IRET has already unwound RSP to | |
| 104 | + * the correct value. objtool doesn't know the callee will IRET and, | |
| 105 | + * without the explicit restore, thinks the stack is getting walloped. | |
| 106 | + * Using an unwind hint is problematic due to x86-64's dynamic alignment. | |
| 107 | + */ | |
| 108 | + leave | |
| 109 | + RET | |
| 110 | +.endm | |
| 111 | + | |
| 112 | +#ifndef CONFIG_X86_64 | |
| 113 | +.pushsection .text, "ax" | |
| 114 | +SYM_FUNC_START(idt_do_interrupt_irqoff) | |
| 115 | + IDT_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 | |
| 116 | +SYM_FUNC_END(idt_do_interrupt_irqoff) | |
| 117 | +.popsection | |
| 118 | +#endif | |
| 119 | + | |
| 120 | +.pushsection .noinstr.text, "ax" | |
| 121 | +SYM_FUNC_START(idt_do_nmi_irqoff) | |
| 122 | + IDT_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx | |
| 123 | +SYM_FUNC_END(idt_do_nmi_irqoff) | |
| 124 | +.popsection | |
| 125 | +#endif | |
| @@ -147,5 +147,4 @@ SYM_FUNC_START(asm_fred_entry_from_kvm) · 146 unmodified lines | ||
| 147 | 147 | RET |
| 148 | 148 | |
| 149 | 149 | SYM_FUNC_END(asm_fred_entry_from_kvm) |
| 150 | −EXPORT_SYMBOL_FOR_KVM(asm_fred_entry_from_kvm); | |
| 151 | 150 | #endif |
| @@ -178,7 +178,7 @@ static int map_vdso(const struct vdso_image *image, unsigned long addr) · 177 unmodified lines | ||
| 178 | 178 | if (IS_ERR(vma)) { |
| 179 | 179 | ret = PTR_ERR(vma); |
| 180 | 180 | do_munmap(mm, text_start, image->size, NULL); |
| 181 | − do_munmap(mm, addr, image->size, NULL); | |
| 181 | + do_munmap(mm, addr, VDSO_NR_PAGES * PAGE_SIZE, NULL); | |
| 182 | 182 | goto up_fail; |
| 183 | 183 | } |
| 184 | 184 | |
| @@ -1294,13 +1294,16 @@ int x86_perf_rdpmc_index(struct perf_event *event) · 1293 unmodified lines | ||
| 1294 | 1294 | return event->hw.event_base_rdpmc; |
| 1295 | 1295 | } |
| 1296 | 1296 | |
| 1297 | −static inline int match_prev_assignment(struct hw_perf_event *hwc, | |
| 1297 | +static inline int match_prev_assignment(struct perf_event *event, | |
| 1298 | 1298 | struct cpu_hw_events *cpuc, |
| 1299 | 1299 | int i) |
| 1300 | 1300 | { |
| 1301 | + struct hw_perf_event *hwc = &event->hw; | |
| 1302 | + | |
| 1301 | 1303 | return hwc->idx == cpuc->assign[i] && |
| 1302 | − hwc->last_cpu == smp_processor_id() && | |
| 1303 | − hwc->last_tag == cpuc->tags[i]; | |
| 1304 | + hwc->last_cpu == smp_processor_id() && | |
| 1305 | + hwc->last_tag == cpuc->tags[i] && | |
| 1306 | + !is_acr_event_group(event); | |
| 1304 | 1307 | } |
| 1305 | 1308 | |
| 1306 | 1309 | static void x86_pmu_start(struct perf_event *event, int flags); |
| @@ -1346,7 +1349,7 @@ static void x86_pmu_enable(struct pmu *pmu) · 39 unmodified lines | ||
| 1346 | 1349 | * - no other event has used the counter since |
| 1347 | 1350 | */ |
| 1348 | 1351 | if (hwc->idx == -1 || |
| 1349 | − match_prev_assignment(hwc, cpuc, i)) | |
| 1352 | + match_prev_assignment(event, cpuc, i)) | |
| 1350 | 1353 | continue; |
| 1351 | 1354 | |
| 1352 | 1355 | /* |
| @@ -1367,7 +1370,7 @@ static void x86_pmu_enable(struct pmu *pmu) · 14 unmodified lines | ||
| 1367 | 1370 | event = cpuc->event_list[i]; |
| 1368 | 1371 | hwc = &event->hw; |
| 1369 | 1372 | |
| 1370 | − if (!match_prev_assignment(hwc, cpuc, i)) | |
| 1373 | + if (!match_prev_assignment(event, cpuc, i)) | |
| 1371 | 1374 | x86_assign_hw_event(event, cpuc, i); |
| 1372 | 1375 | else if (i < n_running) |
| 1373 | 1376 | continue; |
| @@ -3118,11 +3118,11 @@ static void intel_pmu_enable_fixed(struct perf_event *event) · 3117 unmodified lines | ||
| 3118 | 3118 | intel_set_masks(event, idx); |
| 3119 | 3119 | |
| 3120 | 3120 | /* |
| 3121 | − * Enable IRQ generation (0x8), if not PEBS, | |
| 3122 | − * and enable ring-3 counting (0x2) and ring-0 counting (0x1) | |
| 3123 | − * if requested: | |
| 3121 | + * Enable IRQ generation (0x8), if not PEBS or self-reloaded | |
| 3122 | + * ACR event, and enable ring-3 counting (0x2) and ring-0 | |
| 3123 | + * counting (0x1) if requested: | |
| 3124 | 3124 | */ |
| 3125 | − if (!event->attr.precise_ip) | |
| 3125 | + if (!event->attr.precise_ip && !is_acr_self_reload_event(event)) | |
| 3126 | 3126 | bits |= INTEL_FIXED_0_ENABLE_PMI; |
| 3127 | 3127 | if (hwc->config & ARCH_PERFMON_EVENTSEL_USR) |
| 3128 | 3128 | bits |= INTEL_FIXED_0_USER; |
| @@ -3306,6 +3306,15 @@ static void intel_pmu_enable_event(struct perf_event *event) · 177 unmodified lines | ||
| 3306 | 3306 | intel_set_masks(event, idx); |
| 3307 | 3307 | static_call_cond(intel_pmu_enable_acr_event)(event); |
| 3308 | 3308 | static_call_cond(intel_pmu_enable_event_ext)(event); |
| 3309 | + /* | |
| 3310 | + * For self-reloaded ACR event, don't enable PMI since | |
| 3311 | + * HW won't set overflow bit in GLOBAL_STATUS. Otherwise, | |
| 3312 | + * the PMI would be recognized as a suspicious NMI. | |
| 3313 | + */ | |
| 3314 | + if (is_acr_self_reload_event(event)) | |
| 3315 | + hwc->config &= ~ARCH_PERFMON_EVENTSEL_INT; | |
| 3316 | + else if (!event->attr.precise_ip) | |
| 3317 | + hwc->config |= ARCH_PERFMON_EVENTSEL_INT; | |
| 3309 | 3318 | __x86_pmu_enable_event(hwc, enable_mask); |
| 3310 | 3319 | break; |
| 3311 | 3320 | case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS - 1: |
| @@ -3332,23 +3341,41 @@ static void intel_pmu_enable_event(struct perf_event *event) · 20 unmodified lines | ||
| 3332 | 3341 | static void intel_pmu_acr_late_setup(struct cpu_hw_events *cpuc) |
| 3333 | 3342 | { |
| 3334 | 3343 | struct perf_event *event, *leader; |
| 3335 | − int i, j, idx; | |
| 3344 | + int i, j, k, bit, idx; | |
| 3336 | 3345 | |
| 3346 | + /* | |
| 3347 | + * FIXME: ACR mask parsing relies on cpuc->event_list[] (active events only). | |
| 3348 | + * Disabling an ACR event causes bit-shifting errors in the acr_mask of | |
| 3349 | + * remaining group members. As ACR sampling requires all events to be active, | |
| 3350 | + * this limitation is acceptable for now. Revisit if independent event toggling | |
| 3351 | + * is required. | |
| 3352 | + */ | |
| 3337 | 3353 | for (i = 0; i < cpuc->n_events; i++) { |
| 3338 | 3354 | leader = cpuc->event_list[i]; |
| 3339 | 3355 | if (!is_acr_event_group(leader)) |
| 3340 | 3356 | continue; |
| 3341 | 3357 | |
| 3342 | − /* The ACR events must be contiguous. */ | |
| 3358 | + /* Find the last event of the ACR group. */ | |
| 3343 | 3359 | for (j = i; j < cpuc->n_events; j++) { |
| 3344 | 3360 | event = cpuc->event_list[j]; |
| 3345 | 3361 | if (event->group_leader != leader->group_leader) |
| 3346 | 3362 | break; |
| 3347 | − for_each_set_bit(idx, (unsigned long *)&event->attr.config2, X86_PMC_IDX_MAX) { | |
| 3348 | − if (i + idx >= cpuc->n_events || | |
| 3349 | − !is_acr_event_group(cpuc->event_list[i + idx])) | |
| 3350 | − return; | |
| 3351 | − __set_bit(cpuc->assign[i + idx], (unsigned long *)&event->hw.config1); | |
| 3363 | + } | |
| 3364 | + | |
| 3365 | + /* | |
| 3366 | + * Translate the user-space ACR mask (attr.config2) into the physical | |
| 3367 | + * counter bitmask (hw.config1) for each ACR event in the group. | |
| 3368 | + * NOTE: ACR event contiguity is guaranteed by intel_pmu_hw_config(). | |
| 3369 | + */ | |
| 3370 | + for (k = i; k < j; k++) { | |
| 3371 | + event = cpuc->event_list[k]; | |
| 3372 | + event->hw.config1 = 0; | |
| 3373 | + for_each_set_bit(bit, (unsigned long *)&event->attr.config2, X86_PMC_IDX_MAX) { | |
| 3374 | + idx = i + bit; | |
| 3375 | + /* Event index of ACR group must locate in [i, j). */ | |
| 3376 | + if (idx >= j || !is_acr_event_group(cpuc->event_list[idx])) | |
| 3377 | + continue; | |
| 3378 | + __set_bit(cpuc->assign[idx], (unsigned long *)&event->hw.config1); | |
| 3352 | 3379 | } |
| 3353 | 3380 | } |
| 3354 | 3381 | i = j - 1; |
| @@ -7504,6 +7531,7 @@ static __always_inline void intel_pmu_init_pnc(struct pmu *pmu) · 4149 unmodified lines | ||
| 7504 | 7531 | hybrid(pmu, event_constraints) = intel_pnc_event_constraints; |
| 7505 | 7532 | hybrid(pmu, pebs_constraints) = intel_pnc_pebs_event_constraints; |
| 7506 | 7533 | hybrid(pmu, extra_regs) = intel_pnc_extra_regs; |
| 7534 | + static_call_update(intel_pmu_enable_acr_event, intel_pmu_enable_acr); | |
| 7507 | 7535 | } |
| 7508 | 7536 | |
| 7509 | 7537 | static __always_inline void intel_pmu_init_skt(struct pmu *pmu) |
| @@ -137,6 +137,16 @@ static inline bool is_acr_event_group(struct perf_event *event) · 136 unmodified lines | ||
| 137 | 137 | return check_leader_group(event->group_leader, PERF_X86_EVENT_ACR); |
| 138 | 138 | } |
| 139 | 139 | |
| 140 | +static inline bool is_acr_self_reload_event(struct perf_event *event) | |
| 141 | +{ | |
| 142 | + struct hw_perf_event *hwc = &event->hw; | |
| 143 | + | |
| 144 | + if (hwc->idx < 0) | |
| 145 | + return false; | |
| 146 | + | |
| 147 | + return test_bit(hwc->idx, (unsigned long *)&hwc->config1); | |
| 148 | +} | |
| 149 | + | |
| 140 | 150 | struct amd_nb { |
| 141 | 151 | int nb_id; /* NorthBridge id */ |
| 142 | 152 | int refcnt; /* reference count */ |
| @@ -14,3 +14,4 @@ generic-y += early_ioremap.h · 13 unmodified lines | ||
| 14 | 14 | generic-y += fprobe.h |
| 15 | 15 | generic-y += mcs_spinlock.h |
| 16 | 16 | generic-y += mmzone.h |
| 17 | +generic-y += ring_buffer.h | |
| @@ -438,6 +438,10 @@ extern void idt_setup_traps(void); · 437 unmodified lines | ||
| 438 | 438 | extern void idt_setup_apic_and_irq_gates(void); |
| 439 | 439 | extern bool idt_is_f00f_address(unsigned long address); |
| 440 | 440 | |
| 441 | +extern void idt_do_interrupt_irqoff(unsigned long address); | |
| 442 | +extern void idt_do_nmi_irqoff(void); | |
| 443 | +extern void idt_entry_from_kvm(unsigned int vector); | |
| 444 | + | |
| 441 | 445 | #ifdef CONFIG_X86_64 |
| 442 | 446 | extern void idt_setup_early_pf(void); |
| 443 | 447 | #else |
| @@ -145,7 +145,7 @@ struct gate_struct { · 144 unmodified lines | ||
| 145 | 145 | typedef struct gate_struct gate_desc; |
| 146 | 146 | |
| 147 | 147 | #ifndef _SETUP |
| 148 | −static inline unsigned long gate_offset(const gate_desc *g) | |
| 148 | +static __always_inline unsigned long gate_offset(const gate_desc *g) | |
| 149 | 149 | { |
| 150 | 150 | #ifdef CONFIG_X86_64 |
| 151 | 151 | return g->offset_low | ((unsigned long)g->offset_middle << 16) | |
| @@ -137,7 +137,8 @@ extern void __init efi_dump_pagetable(void); · 136 unmodified lines | ||
| 137 | 137 | extern void __init efi_apply_memmap_quirks(void); |
| 138 | 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); |
| 139 | 139 | extern void efi_delete_dummy_variable(void); |
| 140 | −extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); | |
| 140 | +extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr, | |
| 141 | + const struct pt_regs *regs); | |
| 141 | 142 | extern void efi_unmap_boot_services(void); |
| 142 | 143 | |
| 143 | 144 | void arch_efi_call_virt_setup(void); |
| @@ -97,4 +97,6 @@ static __always_inline void arch_exit_to_user_mode(void) · 96 unmodified lines | ||
| 97 | 97 | } |
| 98 | 98 | #define arch_exit_to_user_mode arch_exit_to_user_mode |
| 99 | 99 | |
| 100 | +extern void x86_entry_from_kvm(unsigned int entry_type, unsigned int vector); | |
| 101 | + | |
| 100 | 102 | #endif |
| @@ -110,7 +110,6 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { ret · 109 unmodified lines | ||
| 110 | 110 | static inline void cpu_init_fred_exceptions(void) { } |
| 111 | 111 | static inline void cpu_init_fred_rsps(void) { } |
| 112 | 112 | static inline void fred_complete_exception_setup(void) { } |
| 113 | −static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { } | |
| 114 | 113 | static inline void fred_sync_rsp0(unsigned long rsp0) { } |
| 115 | 114 | static inline void fred_update_rsp0(void) { } |
| 116 | 115 | #endif /* CONFIG_X86_FRED */ |
| @@ -1504,6 +1504,7 @@ struct kvm_arch { · 1503 unmodified lines | ||
| 1504 | 1504 | bool use_master_clock; |
| 1505 | 1505 | u64 master_kernel_ns; |
| 1506 | 1506 | u64 master_cycle_now; |
| 1507 | + struct ratelimit_state kvmclock_update_rs; | |
| 1507 | 1508 | |
| 1508 | 1509 | #ifdef CONFIG_KVM_HYPERV |
| 1509 | 1510 | struct kvm_hv hyperv; |
| @@ -803,9 +803,10 @@ · 802 unmodified lines | ||
| 803 | 803 | #define MSR_AMD64_LBR_SELECT 0xc000010e |
| 804 | 804 | |
| 805 | 805 | /* Zen4 */ |
| 806 | −#define MSR_ZEN4_BP_CFG 0xc001102e | |
| 806 | +#define MSR_ZEN4_BP_CFG 0xc001102e | |
| 807 | 807 | #define MSR_ZEN4_BP_CFG_BP_SPEC_REDUCE_BIT 4 |
| 808 | 808 | #define MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT 5 |
| 809 | +#define MSR_ZEN2_BP_CFG_BUG_FIX_BIT 33 | |
| 809 | 810 | |
| 810 | 811 | /* Fam 19h MSRs */ |
| 811 | 812 | #define MSR_F19H_UMC_PERF_CTL 0xc0010800 |
| @@ -733,6 +733,7 @@ bool xen_set_default_idle(void); · 732 unmodified lines | ||
| 733 | 733 | #endif |
| 734 | 734 | |
| 735 | 735 | void __noreturn stop_this_cpu(void *dummy); |
| 736 | +extern bool x86_hypervisor_present; | |
| 736 | 737 | void microcode_check(struct cpuinfo_x86 *prev_info); |
| 737 | 738 | void store_cpu_caps(struct cpuinfo_x86 *info); |
| 738 | 739 | |
| @@ -88,19 +88,19 @@ static void amd_set_max_freq_ratio(void) · 87 unmodified lines | ||
| 88 | 88 | |
| 89 | 89 | rc = cppc_get_perf_caps(0, &perf_caps); |
| 90 | 90 | if (rc) { |
| 91 | − pr_warn("Could not retrieve perf counters (%d)\n", rc); | |
| 91 | + pr_debug("Could not retrieve perf counters (%d)\n", rc); | |
| 92 | 92 | return; |
| 93 | 93 | } |
| 94 | 94 | |
| 95 | 95 | rc = amd_get_boost_ratio_numerator(0, &numerator); |
| 96 | 96 | if (rc) { |
| 97 | − pr_warn("Could not retrieve highest performance (%d)\n", rc); | |
| 97 | + pr_debug("Could not retrieve highest performance (%d)\n", rc); | |
| 98 | 98 | return; |
| 99 | 99 | } |
| 100 | 100 | nominal_perf = perf_caps.nominal_perf; |
| 101 | 101 | |
| 102 | 102 | if (!nominal_perf) { |
| 103 | − pr_warn("Could not retrieve nominal performance\n"); | |
| 103 | + pr_debug("Could not retrieve nominal performance\n"); | |
| 104 | 104 | return; |
| 105 | 105 | } |
| 106 | 106 | |
| @@ -989,6 +989,9 @@ static void init_amd_zen2(struct cpuinfo_x86 *c) · 988 unmodified lines | ||
| 989 | 989 | |
| 990 | 990 | /* Correct misconfigured CPUID on some clients. */ |
| 991 | 991 | clear_cpu_cap(c, X86_FEATURE_INVLPGB); |
| 992 | + | |
| 993 | + if (!cpu_has(c, X86_FEATURE_HYPERVISOR)) | |
| 994 | + msr_set_bit(MSR_ZEN4_BP_CFG, MSR_ZEN2_BP_CFG_BUG_FIX_BIT); | |
| 992 | 995 | } |
| 993 | 996 | |
| 994 | 997 | static void init_amd_zen3(struct cpuinfo_x86 *c) |
| @@ -92,6 +92,7 @@ static const struct cpuid_dep cpuid_deps[] = { · 91 unmodified lines | ||
| 92 | 92 | { X86_FEATURE_FRED, X86_FEATURE_LKGS }, |
| 93 | 93 | { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, |
| 94 | 94 | { X86_FEATURE_LASS, X86_FEATURE_SMAP }, |
| 95 | + { X86_FEATURE_INVLPGB, X86_FEATURE_PCID }, | |
| 95 | 96 | {} |
| 96 | 97 | }; |
| 97 | 98 | |
| @@ -90,7 +90,6 @@ struct mca_config mca_cfg __read_mostly = { · 89 unmodified lines | ||
| 90 | 90 | }; |
| 91 | 91 | |
| 92 | 92 | static DEFINE_PER_CPU(struct mce_hw_err, hw_errs_seen); |
| 93 | −static unsigned long mce_need_notify; | |
| 94 | 93 | |
| 95 | 94 | /* |
| 96 | 95 | * MCA banks polled by the period polling timer for corrected events. |
| @@ -152,8 +151,10 @@ EXPORT_PER_CPU_SYMBOL_GPL(injectm); · 55 unmodified lines | ||
| 152 | 151 | |
| 153 | 152 | void mce_log(struct mce_hw_err *err) |
| 154 | 153 | { |
| 155 | − if (mce_gen_pool_add(err)) | |
| 154 | + if (mce_gen_pool_add(err)) { | |
| 155 | + pr_info(HW_ERR "Machine check events logged\n"); | |
| 156 | 156 | irq_work_queue(&mce_irq_work); |
| 157 | + } | |
| 157 | 158 | } |
| 158 | 159 | EXPORT_SYMBOL_GPL(mce_log); |
| 159 | 160 | |
| @@ -585,28 +586,6 @@ bool mce_is_correctable(struct mce *m) · 425 unmodified lines | ||
| 585 | 586 | } |
| 586 | 587 | EXPORT_SYMBOL_GPL(mce_is_correctable); |
| 587 | 588 | |
| 588 | −/* | |
| 589 | − * Notify the user(s) about new machine check events. | |
| 590 | − * Can be called from interrupt context, but not from machine check/NMI | |
| 591 | − * context. | |
| 592 | − */ | |
| 593 | −static bool mce_notify_irq(void) | |
| 594 | −{ | |
| 595 | − /* Not more than two messages every minute */ | |
| 596 | − static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2); | |
| 597 | − | |
| 598 | − if (test_and_clear_bit(0, &mce_need_notify)) { | |
| 599 | − mce_work_trigger(); | |
| 600 | − | |
| 601 | − if (__ratelimit(&ratelimit)) | |
| 602 | − pr_info(HW_ERR "Machine check events logged\n"); | |
| 603 | − | |
| 604 | − return true; | |
| 605 | − } | |
| 606 | − | |
| 607 | − return false; | |
| 608 | −} | |
| 609 | − | |
| 610 | 589 | static int mce_early_notifier(struct notifier_block *nb, unsigned long val, |
| 611 | 590 | void *data) |
| 612 | 591 | { |
| @@ -618,9 +597,7 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val, · 5 unmodified lines | ||
| 618 | 597 | /* Emit the trace record: */ |
| 619 | 598 | trace_mce_record(err); |
| 620 | 599 | |
| 621 | − set_bit(0, &mce_need_notify); | |
| 622 | − | |
| 623 | − mce_notify_irq(); | |
| 600 | + mce_work_trigger(); | |
| 624 | 601 | |
| 625 | 602 | return NOTIFY_DONE; |
| 626 | 603 | } |
| @@ -1804,7 +1781,7 @@ static void mce_timer_fn(struct timer_list *t) · 1177 unmodified lines | ||
| 1804 | 1781 | * Alert userspace if needed. If we logged an MCE, reduce the polling |
| 1805 | 1782 | * interval, otherwise increase the polling interval. |
| 1806 | 1783 | */ |
| 1807 | − if (mce_notify_irq()) | |
| 1784 | + if (!mce_gen_pool_empty()) | |
| 1808 | 1785 | iv = max(iv / 2, (unsigned long) HZ/100); |
| 1809 | 1786 | else |
| 1810 | 1787 | iv = min(iv * 2, round_jiffies_relative(check_interval * HZ)); |
| @@ -322,7 +322,7 @@ static u32 get_patch_level(void) · 321 unmodified lines | ||
| 322 | 322 | { |
| 323 | 323 | u32 rev, dummy __always_unused; |
| 324 | 324 | |
| 325 | − if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) { | |
| 325 | + if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) { | |
| 326 | 326 | int cpu = smp_processor_id(); |
| 327 | 327 | |
| 328 | 328 | if (!microcode_rev[cpu]) { |
| @@ -714,7 +714,7 @@ static bool __apply_microcode_amd(struct microcode_amd *mc, u32 *cur_rev, · 385 unmodified lines | ||
| 714 | 714 | invlpg(p_addr_end); |
| 715 | 715 | } |
| 716 | 716 | |
| 717 | − if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) | |
| 717 | + if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) | |
| 718 | 718 | microcode_rev[smp_processor_id()] = mc->hdr.patch_id; |
| 719 | 719 | |
| 720 | 720 | /* verify patch application was successful */ |
| @@ -57,7 +57,7 @@ bool force_minrev = IS_ENABLED(CONFIG_MICROCODE_LATE_FORCE_MINREV); · 56 unmodified lines | ||
| 57 | 57 | u32 base_rev; |
| 58 | 58 | u32 microcode_rev[NR_CPUS] = {}; |
| 59 | 59 | |
| 60 | −bool hypervisor_present; | |
| 60 | +bool __ro_after_init x86_hypervisor_present; | |
| 61 | 61 | |
| 62 | 62 | /* |
| 63 | 63 | * Synchronization. |
| @@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void) · 54 unmodified lines | ||
| 118 | 118 | /* |
| 119 | 119 | * Disable when: |
| 120 | 120 | * |
| 121 | − * 1) The CPU does not support CPUID. | |
| 122 | − */ | |
| 123 | − if (!cpuid_feature()) { | |
| 124 | − dis_ucode_ldr = true; | |
| 125 | − return dis_ucode_ldr; | |
| 126 | − } | |
| 127 | − | |
| 128 | − /* | |
| 121 | + * 1) The CPU does not support CPUID, detected below in | |
| 122 | + * load_ucode_bsp(). | |
| 123 | + * | |
| 129 | 124 | * 2) Bit 31 in CPUID[1]:ECX is clear |
| 130 | 125 | * The bit is reserved for hypervisor use. This is still not |
| 131 | 126 | * completely accurate as XEN PV guests don't see that CPUID bit |
| @@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void) · 3 unmodified lines | ||
| 135 | 130 | * 3) Certain AMD patch levels are not allowed to be |
| 136 | 131 | * overwritten. |
| 137 | 132 | */ |
| 138 | − hypervisor_present = native_cpuid_ecx(1) & BIT(31); | |
| 139 | − | |
| 140 | − if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) || | |
| 133 | + if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) || | |
| 141 | 134 | amd_check_current_patch_level()) |
| 142 | 135 | dis_ucode_ldr = true; |
| 143 | 136 | |
| @@ -179,6 +172,11 @@ void __init load_ucode_bsp(void) · 35 unmodified lines | ||
| 179 | 172 | |
| 180 | 173 | early_parse_cmdline(); |
| 181 | 174 | |
| 175 | + if (!cpuid_feature()) | |
| 176 | + dis_ucode_ldr = true; | |
| 177 | + else | |
| 178 | + x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31); | |
| 179 | + | |
| 182 | 180 | if (microcode_loader_disabled()) |
| 183 | 181 | return; |
| 184 | 182 | |
| @@ -138,6 +138,9 @@ u32 intel_get_platform_id(void) · 137 unmodified lines | ||
| 138 | 138 | { |
| 139 | 139 | unsigned int val[2]; |
| 140 | 140 | |
| 141 | + if (x86_hypervisor_present) | |
| 142 | + return 0; | |
| 143 | + | |
| 141 | 144 | /* |
| 142 | 145 | * This can be called early. Use CPUID directly instead of |
| 143 | 146 | * relying on cpuinfo_x86 which may not be fully initialized. |
| @@ -48,7 +48,6 @@ extern struct early_load_data early_data; · 47 unmodified lines | ||
| 48 | 48 | extern struct ucode_cpu_info ucode_cpu_info[]; |
| 49 | 49 | extern u32 microcode_rev[NR_CPUS]; |
| 50 | 50 | extern u32 base_rev; |
| 51 | −extern bool hypervisor_present; | |
| 52 | 51 | |
| 53 | 52 | struct cpio_data find_microcode_in_initrd(const char *path); |
| 54 | 53 | |
| @@ -450,6 +450,10 @@ __init static int append_e820_table(struct boot_e820_entry *entries, u32 nr_entr · 449 unmodified lines | ||
| 450 | 450 | { |
| 451 | 451 | struct boot_e820_entry *entry = entries; |
| 452 | 452 | |
| 453 | + /* If there aren't any entries, we'll want to fall back to another source: */ | |
| 454 | + if (!nr_entries) | |
| 455 | + return -ENOENT; | |
| 456 | + | |
| 453 | 457 | while (nr_entries) { |
| 454 | 458 | u64 start = entry->addr; |
| 455 | 459 | u64 size = entry->size; |
| @@ -458,7 +462,7 @@ __init static int append_e820_table(struct boot_e820_entry *entries, u32 nr_entr · 2 unmodified lines | ||
| 458 | 462 | |
| 459 | 463 | /* Ignore the remaining entries on 64-bit overflow: */ |
| 460 | 464 | if (start > end && likely(size)) |
| 461 | − return -1; | |
| 465 | + return -EINVAL; | |
| 462 | 466 | |
| 463 | 467 | e820__range_add(start, size, type); |
| 464 | 468 | |
| @@ -27,14 +27,19 @@ · 26 unmodified lines | ||
| 27 | 27 | static inline bool check_xstate_in_sigframe(struct fxregs_state __user *fxbuf, |
| 28 | 28 | struct _fpx_sw_bytes *fx_sw) |
| 29 | 29 | { |
| 30 | + int min_xstate_size = sizeof(struct fxregs_state) + | |
| 31 | + sizeof(struct xstate_header); | |
| 30 | 32 | void __user *fpstate = fxbuf; |
| 31 | 33 | unsigned int magic2; |
| 32 | 34 | |
| 33 | 35 | if (__copy_from_user(fx_sw, &fxbuf->sw_reserved[0], sizeof(*fx_sw))) |
| 34 | 36 | return false; |
| 35 | 37 | |
| 36 | − /* Check for the first magic field */ | |
| 37 | − if (fx_sw->magic1 != FP_XSTATE_MAGIC1) | |
| 38 | + /* Check for the first magic field and other error scenarios. */ | |
| 39 | + if (fx_sw->magic1 != FP_XSTATE_MAGIC1 || | |
| 40 | + fx_sw->xstate_size < min_xstate_size || | |
| 41 | + fx_sw->xstate_size > x86_task_fpu(current)->fpstate->user_size || | |
| 42 | + fx_sw->xstate_size > fx_sw->extended_size) | |
| 38 | 43 | goto setfx; |
| 39 | 44 | |
| 40 | 45 | /* |
| @@ -43,7 +48,7 @@ static inline bool check_xstate_in_sigframe(struct fxregs_state __user *fxbuf, · 2 unmodified lines | ||
| 43 | 48 | * fpstate layout with out copying the extended state information |
| 44 | 49 | * in the memory layout. |
| 45 | 50 | */ |
| 46 | − if (__get_user(magic2, (__u32 __user *)(fpstate + x86_task_fpu(current)->fpstate->user_size))) | |
| 51 | + if (__get_user(magic2, (__u32 __user *)(fpstate + fx_sw->xstate_size))) | |
| 47 | 52 | return false; |
| 48 | 53 | |
| 49 | 54 | if (likely(magic2 == FP_XSTATE_MAGIC2)) |
| @@ -375,6 +375,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) · 374 unmodified lines | ||
| 375 | 375 | goto fail; |
| 376 | 376 | } |
| 377 | 377 | |
| 378 | + /* | |
| 379 | + * Generated trampoline may contain rIP-relative addressing which | |
| 380 | + * displacement needs to be fixed. | |
| 381 | + */ | |
| 382 | + text_poke_apply_relocation(trampoline, trampoline, size, | |
| 383 | + (void *)start_offset, size); | |
| 384 | + | |
| 378 | 385 | /* |
| 379 | 386 | * The address of the ftrace_ops that is used for this trampoline |
| 380 | 387 | * is stored at the end of the trampoline. This will be used to |
| @@ -268,6 +268,13 @@ void __init idt_setup_early_pf(void) · 267 unmodified lines | ||
| 268 | 268 | } |
| 269 | 269 | #endif |
| 270 | 270 | |
| 271 | +#if IS_ENABLED(CONFIG_KVM_INTEL) && !defined(CONFIG_X86_64) | |
| 272 | +void idt_entry_from_kvm(unsigned int vector) | |
| 273 | +{ | |
| 274 | + idt_do_interrupt_irqoff(gate_offset(idt_table + vector)); | |
| 275 | +} | |
| 276 | +#endif | |
| 277 | + | |
| 271 | 278 | static void __init idt_map_in_cea(void) |
| 272 | 279 | { |
| 273 | 280 | /* |
| @@ -614,7 +614,6 @@ DEFINE_IDTENTRY_RAW(exc_nmi_kvm_vmx) · 613 unmodified lines | ||
| 614 | 614 | { |
| 615 | 615 | exc_nmi(regs); |
| 616 | 616 | } |
| 617 | −EXPORT_SYMBOL_FOR_KVM(asm_exc_nmi_kvm_vmx); | |
| 618 | 617 | #endif |
| 619 | 618 | |
| 620 | 619 | #ifdef CONFIG_NMI_CHECK_CPU |
| @@ -136,6 +136,14 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) · 135 unmodified lines | ||
| 136 | 136 | * %r13 original CR4 when relocate_kernel() was invoked |
| 137 | 137 | */ |
| 138 | 138 | |
| 139 | + /* | |
| 140 | + * Set return address to 0 if not preserving context. The purgatory | |
| 141 | + * shipped in kexec-tools will unconditionally look for the return | |
| 142 | + * address on the stack and set a kexec_jump_back_entry= command | |
| 143 | + * line option if it's non-zero. There's no other way that it can | |
| 144 | + * tell a preserve-context (kjump) kexec from a normal one. | |
| 145 | + */ | |
| 146 | + pushq $0 | |
| 139 | 147 | /* store the start address on the stack */ |
| 140 | 148 | pushq %rdx |
| 141 | 149 | |
| @@ -4481,7 +4481,7 @@ static const struct opcode opcode_map_0f_38[256] = { · 4480 unmodified lines | ||
| 4481 | 4481 | X16(N), X16(N), |
| 4482 | 4482 | /* 0x20 - 0x2f */ |
| 4483 | 4483 | X8(N), |
| 4484 | − X2(N), GP(SrcReg | DstMem | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N, | |
| 4484 | + X2(N), GP(SrcMem | DstReg | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N, | |
| 4485 | 4485 | /* 0x30 - 0x7f */ |
| 4486 | 4486 | X16(N), X16(N), X16(N), X16(N), X16(N), |
| 4487 | 4487 | /* 0x80 - 0xef */ |
| @@ -2040,7 +2040,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · 2039 unmodified lines | ||
| 2040 | 2040 | * flush). Translate the address here so the memory can be uniformly |
| 2041 | 2041 | * read with kvm_read_guest(). |
| 2042 | 2042 | */ |
| 2043 | − if (!hc->fast && is_guest_mode(vcpu)) { | |
| 2043 | + if (!hc->fast && mmu_is_nested(vcpu)) { | |
| 2044 | 2044 | hc->ingpa = translate_nested_gpa(vcpu, hc->ingpa, 0, NULL); |
| 2045 | 2045 | if (unlikely(hc->ingpa == INVALID_GPA)) |
| 2046 | 2046 | return HV_STATUS_INVALID_HYPERCALL_INPUT; |
| @@ -667,13 +667,15 @@ bool __kvm_apic_update_irr(unsigned long *pir, void *regs, int *max_irr) · 666 unmodified lines | ||
| 667 | 667 | u32 *__pir = (void *)pir_vals; |
| 668 | 668 | u32 i, vec; |
| 669 | 669 | u32 irr_val, prev_irr_val; |
| 670 | − int max_updated_irr; | |
| 670 | + int max_new_irr; | |
| 671 | 671 | |
| 672 | − max_updated_irr = -1; | |
| 673 | − *max_irr = -1; | |
| 674 | − | |
| 675 | − if (!pi_harvest_pir(pir, pir_vals)) | |
| 672 | + if (!pi_harvest_pir(pir, pir_vals)) { | |
| 673 | + *max_irr = apic_find_highest_vector(regs + APIC_IRR); | |
| 676 | 674 | return false; |
| 675 | + } | |
| 676 | + | |
| 677 | + max_new_irr = -1; | |
| 678 | + *max_irr = -1; | |
| 677 | 679 | |
| 678 | 680 | for (i = vec = 0; i <= 7; i++, vec += 32) { |
| 679 | 681 | u32 *p_irr = (u32 *)(regs + APIC_IRR + i * 0x10); |
| @@ -688,25 +690,25 @@ bool __kvm_apic_update_irr(unsigned long *pir, void *regs, int *max_irr) · 8 unmodified lines | ||
| 688 | 690 | !try_cmpxchg(p_irr, &prev_irr_val, irr_val)); |
| 689 | 691 | |
| 690 | 692 | if (prev_irr_val != irr_val) |
| 691 | − max_updated_irr = __fls(irr_val ^ prev_irr_val) + vec; | |
| 693 | + max_new_irr = __fls(irr_val ^ prev_irr_val) + vec; | |
| 692 | 694 | } |
| 693 | 695 | if (irr_val) |
| 694 | 696 | *max_irr = __fls(irr_val) + vec; |
| 695 | 697 | } |
| 696 | 698 | |
| 697 | − return ((max_updated_irr != -1) && | |
| 698 | − (max_updated_irr == *max_irr)); | |
| 699 | + return max_new_irr != -1 && max_new_irr == *max_irr; | |
| 699 | 700 | } |
| 700 | 701 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(__kvm_apic_update_irr); |
| 701 | 702 | |
| 702 | 703 | bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, unsigned long *pir, int *max_irr) |
| 703 | 704 | { |
| 704 | 705 | struct kvm_lapic *apic = vcpu->arch.apic; |
| 705 | − bool irr_updated = __kvm_apic_update_irr(pir, apic->regs, max_irr); | |
| 706 | + bool max_irr_is_from_pir; | |
| 706 | 707 | |
| 707 | − if (unlikely(!apic->apicv_active && irr_updated)) | |
| 708 | + max_irr_is_from_pir = __kvm_apic_update_irr(pir, apic->regs, max_irr); | |
| 709 | + if (unlikely(!apic->apicv_active && max_irr_is_from_pir)) | |
| 708 | 710 | apic->irr_pending = true; |
| 709 | − return irr_updated; | |
| 711 | + return max_irr_is_from_pir; | |
| 710 | 712 | } |
| 711 | 713 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_apic_update_irr); |
| 712 | 714 | |
| @@ -182,6 +182,8 @@ static struct kmem_cache *pte_list_desc_cache; · 181 unmodified lines | ||
| 182 | 182 | struct kmem_cache *mmu_page_header_cache; |
| 183 | 183 | |
| 184 | 184 | static void mmu_spte_set(u64 *sptep, u64 spte); |
| 185 | +static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp, | |
| 186 | + u64 *spte, struct list_head *invalid_list); | |
| 185 | 187 | |
| 186 | 188 | struct kvm_mmu_role_regs { |
| 187 | 189 | const unsigned long cr0; |
| @@ -1287,19 +1289,6 @@ static void drop_spte(struct kvm *kvm, u64 *sptep) · 1099 unmodified lines | ||
| 1287 | 1289 | rmap_remove(kvm, sptep); |
| 1288 | 1290 | } |
| 1289 | 1291 | |
| 1290 | −static void drop_large_spte(struct kvm *kvm, u64 *sptep, bool flush) | |
| 1291 | −{ | |
| 1292 | − struct kvm_mmu_page *sp; | |
| 1293 | − | |
| 1294 | − sp = sptep_to_sp(sptep); | |
| 1295 | − WARN_ON_ONCE(sp->role.level == PG_LEVEL_4K); | |
| 1296 | − | |
| 1297 | − drop_spte(kvm, sptep); | |
| 1298 | − | |
| 1299 | − if (flush) | |
| 1300 | − kvm_flush_remote_tlbs_sptep(kvm, sptep); | |
| 1301 | −} | |
| 1302 | − | |
| 1303 | 1292 | /* |
| 1304 | 1293 | * Write-protect on the specified @sptep, @pt_protect indicates whether |
| 1305 | 1294 | * spte write-protection is caused by protecting shadow page table. |
| @@ -2466,7 +2455,8 @@ static struct kvm_mmu_page *kvm_mmu_get_child_sp(struct kvm_vcpu *vcpu, · 1160 unmodified lines | ||
| 2466 | 2455 | { |
| 2467 | 2456 | union kvm_mmu_page_role role; |
| 2468 | 2457 | |
| 2469 | − if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep)) | |
| 2458 | + if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep) && | |
| 2459 | + spte_to_child_sp(*sptep) && spte_to_child_sp(*sptep)->gfn == gfn) | |
| 2470 | 2460 | return ERR_PTR(-EEXIST); |
| 2471 | 2461 | |
| 2472 | 2462 | role = kvm_mmu_child_role(sptep, direct, access); |
| @@ -2536,6 +2526,23 @@ static void shadow_walk_next(struct kvm_shadow_walk_iterator *iterator) · 63 unmodified lines | ||
| 2536 | 2526 | __shadow_walk_next(iterator, *iterator->sptep); |
| 2537 | 2527 | } |
| 2538 | 2528 | |
| 2529 | +/* | |
| 2530 | + * Note: while normally KVM uses a "bool flush" return value to let | |
| 2531 | + * the caller batch flushes, __link_shadow_page() flushes immediately | |
| 2532 | + * before populating the parent PTE with the new shadow page. The | |
| 2533 | + * typical callers, direct_map() and FNAME(fetch)(), are not going | |
| 2534 | + * to zap more than one huge SPTE anyway. | |
| 2535 | + * | |
| 2536 | + * The only exception, where @flush can be false, is when a huge SPTE | |
| 2537 | + * is replaced with a shadow page SPTE with a fully populated page table, | |
| 2538 | + * which can happen from shadow_mmu_split_huge_page(). In this case, | |
| 2539 | + * no memory is unmapped across the change to the page tables and no | |
| 2540 | + * immediate flush is needed for correctness. | |
| 2541 | + * | |
| 2542 | + * Even in that case, calls to kvm_mmu_commit_zap_page() are not | |
| 2543 | + * batched. Doing so would require adding an invalid_list argument | |
| 2544 | + * all the way down to __walk_slot_rmaps(). | |
| 2545 | + */ | |
| 2539 | 2546 | static void __link_shadow_page(struct kvm *kvm, |
| 2540 | 2547 | struct kvm_mmu_memory_cache *cache, u64 *sptep, |
| 2541 | 2548 | struct kvm_mmu_page *sp, bool flush) |
| @@ -2544,13 +2551,18 @@ static void __link_shadow_page(struct kvm *kvm, · 2 unmodified lines | ||
| 2544 | 2551 | |
| 2545 | 2552 | BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK); |
| 2546 | 2553 | |
| 2547 | − /* | |
| 2548 | − * If an SPTE is present already, it must be a leaf and therefore | |
| 2549 | − * a large one. Drop it, and flush the TLB if needed, before | |
| 2550 | − * installing sp. | |
| 2551 | − */ | |
| 2552 | − if (is_shadow_present_pte(*sptep)) | |
| 2553 | − drop_large_spte(kvm, sptep, flush); | |
| 2554 | + if (is_shadow_present_pte(*sptep)) { | |
| 2555 | + struct kvm_mmu_page *parent_sp; | |
| 2556 | + LIST_HEAD(invalid_list); | |
| 2557 | + | |
| 2558 | + parent_sp = sptep_to_sp(sptep); | |
| 2559 | + WARN_ON_ONCE(parent_sp->role.level == PG_LEVEL_4K); | |
| 2560 | + | |
| 2561 | + if (mmu_page_zap_pte(kvm, parent_sp, sptep, &invalid_list)) | |
| 2562 | + kvm_mmu_commit_zap_page(kvm, &invalid_list); | |
| 2563 | + else if (flush) | |
| 2564 | + kvm_flush_remote_tlbs_sptep(kvm, sptep); | |
| 2565 | + } | |
| 2554 | 2566 | |
| 2555 | 2567 | spte = make_nonleaf_spte(sp->spt, sp_ad_disabled(sp)); |
| 2556 | 2568 | |
| @@ -206,6 +206,35 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 205 unmodified lines | ||
| 206 | 206 | |
| 207 | 207 | svm_clr_intercept(svm, INTERCEPT_CR8_WRITE); |
| 208 | 208 | |
| 209 | + /* | |
| 210 | + * Flush the TLB when enabling (x2)AVIC and when transitioning between | |
| 211 | + * xAVIC and x2AVIC, as the CPU may have inserted a TLB entry for the | |
| 212 | + * "wrong" mapping. | |
| 213 | + * | |
| 214 | + * KVM uses a per-VM "scratch" page to back the APIC memslot, because | |
| 215 | + * KVM also uses per-VM page tables *and* maintains the page table (NPT | |
| 216 | + * or shadow page) mappings for said memslot even if one or more vCPUs | |
| 217 | + * have their local APIC hardware-disabled or are in x2APIC mode, i.e. | |
| 218 | + * even if one or more vCPUs' APIC MMIO BAR is effectively disabled. | |
| 219 | + * | |
| 220 | + * If xAVIC is fully enabled, hardware ignores the physical address in | |
| 221 | + * KVM's page tables, i.e. in the leaf SPTE for the APIC memslot, and | |
| 222 | + * instead redirects the access to the AVIC backing page, i.e. to the | |
| 223 | + * vCPU's virtual APIC page. If xAVIC is not enabled (APIC is either | |
| 224 | + * hardware-disabled or in x2APIC mode), then guest accesses will use | |
| 225 | + * the page table mapping verbatim, i.e. will access the per-VM scratch | |
| 226 | + * page, as normal memory. | |
| 227 | + * | |
| 228 | + * In both cases, the CPU is allowed to cache TLB entries for the APIC | |
| 229 | + * base GPA. So, KVM needs to flush the TLB when enabling xAVIC, as | |
| 230 | + * accesses need to be redirected to the virtual APIC page, but the TLB | |
| 231 | + * may contain entries pointing at the scratch page. KVM also needs to | |
| 232 | + * flush the TLB when enabling x2AVIC, as accesses need to go to the | |
| 233 | + * scratch page, but the TLB may contain entries tagged as xAVIC, i.e. | |
| 234 | + * entries pointing to the vCPU's virtual APIC page. | |
| 235 | + */ | |
| 236 | + kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu); | |
| 237 | + | |
| 209 | 238 | /* |
| 210 | 239 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR |
| 211 | 240 | * accesses, while interrupt injection to a running vCPU can be |
| @@ -219,12 +248,6 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 7 unmodified lines | ||
| 219 | 248 | /* Disabling MSR intercept for x2APIC registers */ |
| 220 | 249 | avic_set_x2apic_msr_interception(svm, false); |
| 221 | 250 | } else { |
| 222 | − /* | |
| 223 | − * Flush the TLB, the guest may have inserted a non-APIC | |
| 224 | − * mapping into the TLB while AVIC was disabled. | |
| 225 | − */ | |
| 226 | − kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu); | |
| 227 | − | |
| 228 | 251 | /* Enabling MSR intercept for x2APIC registers */ |
| 229 | 252 | avic_set_x2apic_msr_interception(svm, true); |
| 230 | 253 | } |
| @@ -1300,12 +1323,14 @@ bool __init avic_hardware_setup(void) · 1069 unmodified lines | ||
| 1300 | 1323 | } |
| 1301 | 1324 | |
| 1302 | 1325 | /* |
| 1303 | − * Disable IPI virtualization for AMD Family 17h CPUs (Zen1 and Zen2) | |
| 1304 | − * due to erratum 1235, which results in missed VM-Exits on the sender | |
| 1305 | − * and thus missed wake events for blocking vCPUs due to the CPU | |
| 1306 | − * failing to see a software update to clear IsRunning. | |
| 1326 | + * Disable IPI virtualization for AMD Family 17h (Zen1 and Zen2) and | |
| 1327 | + * Hygon Family 18h (derived from AMD Zen1) CPUs due to erratum 1235, | |
| 1328 | + * which results in missed VM-Exits on the sender and thus missed wake | |
| 1329 | + * events for blocking vCPUs due to the CPU failing to see a software | |
| 1330 | + * update to clear IsRunning. | |
| 1307 | 1331 | */ |
| 1308 | − enable_ipiv = enable_ipiv && boot_cpu_data.x86 != 0x17; | |
| 1332 | + if (boot_cpu_data.x86 == 0x17 || boot_cpu_data.x86 == 0x18) | |
| 1333 | + enable_ipiv = false; | |
| 1309 | 1334 | |
| 1310 | 1335 | amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier); |
| 1311 | 1336 | |
| @@ -160,6 +160,16 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm) · 159 unmodified lines | ||
| 160 | 160 | if (!intercept_smi) |
| 161 | 161 | vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI); |
| 162 | 162 | |
| 163 | + /* | |
| 164 | + * Intercept PAUSE if and only if L1 wants to. KVM intercepts PAUSE so | |
| 165 | + * that a vCPU that may be spinning waiting for a lock can be scheduled | |
| 166 | + * out in favor of the vCPU that holds said lock. KVM doesn't support | |
| 167 | + * yielding across L2 vCPUs, as KVM has limited visilibity into which | |
| 168 | + * L2 vCPUs are in the same L2 VM, i.e. may be contending for locks. | |
| 169 | + */ | |
| 170 | + if (!vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_PAUSE)) | |
| 171 | + vmcb_clr_intercept(&vmcb02->control, INTERCEPT_PAUSE); | |
| 172 | + | |
| 163 | 173 | if (nested_vmcb_needs_vls_intercept(svm)) { |
| 164 | 174 | /* |
| 165 | 175 | * If the virtual VMLOAD/VMSAVE is not enabled for the L2, |
| @@ -819,7 +829,6 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) · 653 unmodified lines | ||
| 819 | 829 | struct vmcb *vmcb02 = svm->nested.vmcb02.ptr; |
| 820 | 830 | struct vmcb *vmcb01 = svm->vmcb01.ptr; |
| 821 | 831 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 822 | − u32 pause_count12, pause_thresh12; | |
| 823 | 832 | |
| 824 | 833 | nested_svm_transition_tlb_flush(vcpu); |
| 825 | 834 | |
| @@ -947,31 +956,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) · 121 unmodified lines | ||
| 947 | 956 | vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE; |
| 948 | 957 | |
| 949 | 958 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER)) |
| 950 | − pause_count12 = vmcb12_ctrl->pause_filter_count; | |
| 959 | + vmcb02->control.pause_filter_count = vmcb12_ctrl->pause_filter_count; | |
| 951 | 960 | else |
| 952 | − pause_count12 = 0; | |
| 961 | + vmcb02->control.pause_filter_count = 0; | |
| 953 | 962 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD)) |
| 954 | − pause_thresh12 = vmcb12_ctrl->pause_filter_thresh; | |
| 963 | + vmcb02->control.pause_filter_thresh = vmcb12_ctrl->pause_filter_thresh; | |
| 955 | 964 | else |
| 956 | − pause_thresh12 = 0; | |
| 957 | − if (kvm_pause_in_guest(svm->vcpu.kvm)) { | |
| 958 | − /* use guest values since host doesn't intercept PAUSE */ | |
| 959 | − vmcb02->control.pause_filter_count = pause_count12; | |
| 960 | − vmcb02->control.pause_filter_thresh = pause_thresh12; | |
| 961 | − | |
| 962 | − } else { | |
| 963 | − /* start from host values otherwise */ | |
| 964 | − vmcb02->control.pause_filter_count = vmcb01->control.pause_filter_count; | |
| 965 | − vmcb02->control.pause_filter_thresh = vmcb01->control.pause_filter_thresh; | |
| 966 | − | |
| 967 | − /* ... but ensure filtering is disabled if so requested. */ | |
| 968 | − if (vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_PAUSE)) { | |
| 969 | − if (!pause_count12) | |
| 970 | − vmcb02->control.pause_filter_count = 0; | |
| 971 | − if (!pause_thresh12) | |
| 972 | − vmcb02->control.pause_filter_thresh = 0; | |
| 973 | − } | |
| 974 | − } | |
| 965 | + vmcb02->control.pause_filter_thresh = 0; | |
| 975 | 966 | |
| 976 | 967 | /* |
| 977 | 968 | * Take ALLOW_LARGER_RAP from vmcb12 even though it should be safe to |
| @@ -1298,12 +1289,6 @@ void nested_svm_vmexit(struct vcpu_svm *svm) · 320 unmodified lines | ||
| 1298 | 1289 | /* in case we halted in L2 */ |
| 1299 | 1290 | kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE); |
| 1300 | 1291 | |
| 1301 | − if (!kvm_pause_in_guest(vcpu->kvm)) { | |
| 1302 | − vmcb01->control.pause_filter_count = vmcb02->control.pause_filter_count; | |
| 1303 | − vmcb_mark_dirty(vmcb01, VMCB_INTERCEPTS); | |
| 1304 | − | |
| 1305 | − } | |
| 1306 | − | |
| 1307 | 1292 | /* |
| 1308 | 1293 | * Invalidate last_bus_lock_rip unless KVM is still waiting for the |
| 1309 | 1294 | * guest to make forward progress before re-enabling bus lock detection. |
| @@ -3662,23 +3662,26 @@ int pre_sev_run(struct vcpu_svm *svm, int cpu) · 3661 unmodified lines | ||
| 3662 | 3662 | } |
| 3663 | 3663 | |
| 3664 | 3664 | #define GHCB_SCRATCH_AREA_LIMIT (16ULL * PAGE_SIZE) |
| 3665 | −static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) | |
| 3665 | +static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 min_len) | |
| 3666 | 3666 | { |
| 3667 | 3667 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 3668 | 3668 | u64 ghcb_scratch_beg, ghcb_scratch_end; |
| 3669 | 3669 | u64 scratch_gpa_beg, scratch_gpa_end; |
| 3670 | 3670 | void *scratch_va; |
| 3671 | 3671 | |
| 3672 | + if (WARN_ON_ONCE(!min_len)) | |
| 3673 | + goto e_scratch; | |
| 3674 | + | |
| 3672 | 3675 | scratch_gpa_beg = svm->sev_es.sw_scratch; |
| 3673 | 3676 | if (!scratch_gpa_beg) { |
| 3674 | 3677 | pr_err("vmgexit: scratch gpa not provided\n"); |
| 3675 | 3678 | goto e_scratch; |
| 3676 | 3679 | } |
| 3677 | 3680 | |
| 3678 | − scratch_gpa_end = scratch_gpa_beg + len; | |
| 3681 | + scratch_gpa_end = scratch_gpa_beg + min_len; | |
| 3679 | 3682 | if (scratch_gpa_end < scratch_gpa_beg) { |
| 3680 | 3683 | pr_err("vmgexit: scratch length (%#llx) not valid for scratch address (%#llx)\n", |
| 3681 | − len, scratch_gpa_beg); | |
| 3684 | + min_len, scratch_gpa_beg); | |
| 3682 | 3685 | goto e_scratch; |
| 3683 | 3686 | } |
| 3684 | 3687 | |
| @@ -3702,21 +3705,27 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) · 17 unmodified lines | ||
| 3702 | 3705 | |
| 3703 | 3706 | scratch_va = (void *)svm->sev_es.ghcb; |
| 3704 | 3707 | scratch_va += (scratch_gpa_beg - control->ghcb_gpa); |
| 3708 | + | |
| 3709 | + svm->sev_es.ghcb_sa_len = ghcb_scratch_end - scratch_gpa_beg; | |
| 3705 | 3710 | } else { |
| 3711 | + /* GHCB v2 requires the scratch area to be within the GHCB. */ | |
| 3712 | + if (to_kvm_sev_info(svm->vcpu.kvm)->ghcb_version >= 2) | |
| 3713 | + goto e_scratch; | |
| 3714 | + | |
| 3706 | 3715 | /* |
| 3707 | 3716 | * The guest memory must be read into a kernel buffer, so |
| 3708 | 3717 | * limit the size |
| 3709 | 3718 | */ |
| 3710 | − if (len > GHCB_SCRATCH_AREA_LIMIT) { | |
| 3719 | + if (min_len > GHCB_SCRATCH_AREA_LIMIT) { | |
| 3711 | 3720 | pr_err("vmgexit: scratch area exceeds KVM limits (%#llx requested, %#llx limit)\n", |
| 3712 | − len, GHCB_SCRATCH_AREA_LIMIT); | |
| 3721 | + min_len, GHCB_SCRATCH_AREA_LIMIT); | |
| 3713 | 3722 | goto e_scratch; |
| 3714 | 3723 | } |
| 3715 | − scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT); | |
| 3724 | + scratch_va = kvzalloc(min_len, GFP_KERNEL_ACCOUNT); | |
| 3716 | 3725 | if (!scratch_va) |
| 3717 | 3726 | return -ENOMEM; |
| 3718 | 3727 | |
| 3719 | − if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, len)) { | |
| 3728 | + if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, min_len)) { | |
| 3720 | 3729 | /* Unable to copy scratch area from guest */ |
| 3721 | 3730 | pr_err("vmgexit: kvm_read_guest for scratch area failed\n"); |
| 3722 | 3731 | |
| @@ -3732,11 +3741,10 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) · 9 unmodified lines | ||
| 3732 | 3741 | */ |
| 3733 | 3742 | svm->sev_es.ghcb_sa_sync = sync; |
| 3734 | 3743 | svm->sev_es.ghcb_sa_free = true; |
| 3744 | + svm->sev_es.ghcb_sa_len = min_len; | |
| 3735 | 3745 | } |
| 3736 | 3746 | |
| 3737 | 3747 | svm->sev_es.ghcb_sa = scratch_va; |
| 3738 | − svm->sev_es.ghcb_sa_len = len; | |
| 3739 | − | |
| 3740 | 3748 | return 0; |
| 3741 | 3749 | |
| 3742 | 3750 | e_scratch: |
| @@ -3833,7 +3841,7 @@ struct psc_buffer { · 90 unmodified lines | ||
| 3833 | 3841 | struct psc_entry entries[]; |
| 3834 | 3842 | } __packed; |
| 3835 | 3843 | |
| 3836 | −static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc); | |
| 3844 | +static int snp_begin_psc(struct vcpu_svm *svm); | |
| 3837 | 3845 | |
| 3838 | 3846 | static void snp_complete_psc(struct vcpu_svm *svm, u64 psc_ret) |
| 3839 | 3847 | { |
| @@ -3864,9 +3872,9 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm) · 24 unmodified lines | ||
| 3864 | 3872 | */ |
| 3865 | 3873 | for (idx = svm->sev_es.psc_idx; svm->sev_es.psc_inflight; |
| 3866 | 3874 | svm->sev_es.psc_inflight--, idx++) { |
| 3867 | − struct psc_entry *entry = &entries[idx]; | |
| 3875 | + struct psc_entry entry = READ_ONCE(entries[idx]); | |
| 3868 | 3876 | |
| 3869 | − entry->cur_page = entry->pagesize ? 512 : 1; | |
| 3877 | + entries[idx].cur_page = entry.pagesize ? 512 : 1; | |
| 3870 | 3878 | } |
| 3871 | 3879 | |
| 3872 | 3880 | hdr->cur_entry = idx; |
| @@ -3875,7 +3883,6 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm) · 2 unmodified lines | ||
| 3875 | 3883 | static int snp_complete_one_psc(struct kvm_vcpu *vcpu) |
| 3876 | 3884 | { |
| 3877 | 3885 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3878 | − struct psc_buffer *psc = svm->sev_es.ghcb_sa; | |
| 3879 | 3886 | |
| 3880 | 3887 | if (vcpu->run->hypercall.ret) { |
| 3881 | 3888 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); |
| @@ -3885,16 +3892,18 @@ static int snp_complete_one_psc(struct kvm_vcpu *vcpu) · 3 unmodified lines | ||
| 3885 | 3892 | __snp_complete_one_psc(svm); |
| 3886 | 3893 | |
| 3887 | 3894 | /* Handle the next range (if any). */ |
| 3888 | − return snp_begin_psc(svm, psc); | |
| 3895 | + return snp_begin_psc(svm); | |
| 3889 | 3896 | } |
| 3890 | 3897 | |
| 3891 | −static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc) | |
| 3898 | +static int snp_begin_psc(struct vcpu_svm *svm) | |
| 3892 | 3899 | { |
| 3900 | + struct vcpu_sev_es_state *sev_es = &svm->sev_es; | |
| 3901 | + struct psc_buffer *psc = sev_es->ghcb_sa; | |
| 3893 | 3902 | struct psc_entry *entries = psc->entries; |
| 3894 | 3903 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 3895 | 3904 | struct psc_hdr *hdr = &psc->hdr; |
| 3896 | 3905 | struct psc_entry entry_start; |
| 3897 | − u16 idx, idx_start, idx_end; | |
| 3906 | + u16 idx, idx_start, idx_end, max_nr_entries; | |
| 3898 | 3907 | int npages; |
| 3899 | 3908 | bool huge; |
| 3900 | 3909 | u64 gfn; |
| @@ -3904,6 +3913,19 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc) · 3 unmodified lines | ||
| 3904 | 3913 | return 1; |
| 3905 | 3914 | } |
| 3906 | 3915 | |
| 3916 | + /* | |
| 3917 | + * GHCB v2 requires the scratch area to reside within the GHCB itself, | |
| 3918 | + * and PSC requests are only supported for GHCB v2+. Thus it should be | |
| 3919 | + * impossible to exceed the max PSC entry count (which is derived from | |
| 3920 | + * the size of the shared GHCB buffer). | |
| 3921 | + */ | |
| 3922 | + max_nr_entries = (sev_es->ghcb_sa_len - sizeof(struct psc_hdr)) / | |
| 3923 | + sizeof(struct psc_entry); | |
| 3924 | + if (WARN_ON_ONCE(max_nr_entries > VMGEXIT_PSC_MAX_COUNT)) { | |
| 3925 | + snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); | |
| 3926 | + return 1; | |
| 3927 | + } | |
| 3928 | + | |
| 3907 | 3929 | next_range: |
| 3908 | 3930 | /* There should be no other PSCs in-flight at this point. */ |
| 3909 | 3931 | if (WARN_ON_ONCE(svm->sev_es.psc_inflight)) { |
| @@ -3916,17 +3938,17 @@ next_range: · 6 unmodified lines | ||
| 3916 | 3938 | * validation, so take care to only use validated copies of values used |
| 3917 | 3939 | * for things like array indexing. |
| 3918 | 3940 | */ |
| 3919 | − idx_start = hdr->cur_entry; | |
| 3920 | − idx_end = hdr->end_entry; | |
| 3941 | + idx_start = READ_ONCE(hdr->cur_entry); | |
| 3942 | + idx_end = READ_ONCE(hdr->end_entry); | |
| 3921 | 3943 | |
| 3922 | − if (idx_end >= VMGEXIT_PSC_MAX_COUNT) { | |
| 3944 | + if (idx_end >= max_nr_entries) { | |
| 3923 | 3945 | snp_complete_psc(svm, VMGEXIT_PSC_ERROR_INVALID_HDR); |
| 3924 | 3946 | return 1; |
| 3925 | 3947 | } |
| 3926 | 3948 | |
| 3927 | 3949 | /* Find the start of the next range which needs processing. */ |
| 3928 | 3950 | for (idx = idx_start; idx <= idx_end; idx++, hdr->cur_entry++) { |
| 3929 | − entry_start = entries[idx]; | |
| 3951 | + entry_start = READ_ONCE(entries[idx]); | |
| 3930 | 3952 | |
| 3931 | 3953 | gfn = entry_start.gfn; |
| 3932 | 3954 | huge = entry_start.pagesize; |
| @@ -3970,7 +3992,7 @@ next_range: · 37 unmodified lines | ||
| 3970 | 3992 | * KVM_HC_MAP_GPA_RANGE exit. |
| 3971 | 3993 | */ |
| 3972 | 3994 | while (++idx <= idx_end) { |
| 3973 | − struct psc_entry entry = entries[idx]; | |
| 3995 | + struct psc_entry entry = READ_ONCE(entries[idx]); | |
| 3974 | 3996 | |
| 3975 | 3997 | if (entry.operation != entry_start.operation || |
| 3976 | 3998 | entry.gfn != entry_start.gfn + npages || |
| @@ -4493,13 +4515,22 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 516 unmodified lines | ||
| 4493 | 4515 | case SVM_VMGEXIT_MMIO_READ: |
| 4494 | 4516 | case SVM_VMGEXIT_MMIO_WRITE: { |
| 4495 | 4517 | bool is_write = control->exit_code == SVM_VMGEXIT_MMIO_WRITE; |
| 4518 | + u64 len = control->exit_info_2; | |
| 4496 | 4519 | |
| 4497 | − ret = setup_vmgexit_scratch(svm, !is_write, control->exit_info_2); | |
| 4520 | + if (!len) | |
| 4521 | + return 1; | |
| 4522 | + | |
| 4523 | + if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 && len > 8) { | |
| 4524 | + svm_vmgexit_bad_input(svm, GHCB_ERR_INVALID_INPUT); | |
| 4525 | + return 1; | |
| 4526 | + } | |
| 4527 | + | |
| 4528 | + ret = setup_vmgexit_scratch(svm, !is_write, len); | |
| 4498 | 4529 | if (ret) |
| 4499 | 4530 | break; |
| 4500 | 4531 | |
| 4501 | − ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1, | |
| 4502 | − control->exit_info_2, svm->sev_es.ghcb_sa); | |
| 4532 | + ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1, len, | |
| 4533 | + svm->sev_es.ghcb_sa); | |
| 4503 | 4534 | break; |
| 4504 | 4535 | } |
| 4505 | 4536 | case SVM_VMGEXIT_NMI_COMPLETE: |
| @@ -4546,11 +4577,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 40 unmodified lines | ||
| 4546 | 4577 | vcpu->run->system_event.data[0] = control->ghcb_gpa; |
| 4547 | 4578 | break; |
| 4548 | 4579 | case SVM_VMGEXIT_PSC: |
| 4549 | − ret = setup_vmgexit_scratch(svm, true, control->exit_info_2); | |
| 4580 | + ret = setup_vmgexit_scratch(svm, true, sizeof(struct psc_hdr)); | |
| 4550 | 4581 | if (ret) |
| 4551 | 4582 | break; |
| 4552 | 4583 | |
| 4553 | − ret = snp_begin_psc(svm, svm->sev_es.ghcb_sa); | |
| 4584 | + ret = snp_begin_psc(svm); | |
| 4554 | 4585 | break; |
| 4555 | 4586 | case SVM_VMGEXIT_AP_CREATION: |
| 4556 | 4587 | ret = sev_snp_ap_creation(svm); |
| @@ -4572,6 +4603,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) · 15 unmodified lines | ||
| 4572 | 4603 | control->exit_info_1, control->exit_info_2); |
| 4573 | 4604 | ret = -EINVAL; |
| 4574 | 4605 | break; |
| 4606 | + case SVM_EXIT_IOIO: | |
| 4607 | + if (!((control->exit_info_1 & SVM_IOIO_SIZE_MASK) >> SVM_IOIO_SIZE_SHIFT)) | |
| 4608 | + return 1; | |
| 4609 | + | |
| 4610 | + fallthrough; | |
| 4575 | 4611 | default: |
| 4576 | 4612 | ret = svm_invoke_exit_handler(vcpu, control->exit_code); |
| 4577 | 4613 | } |
| @@ -4592,6 +4628,9 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in) · 14 unmodified lines | ||
| 4592 | 4628 | if (unlikely(check_mul_overflow(count, size, &bytes))) |
| 4593 | 4629 | return -EINVAL; |
| 4594 | 4630 | |
| 4631 | + if (!bytes) | |
| 4632 | + return 1; | |
| 4633 | + | |
| 4595 | 4634 | r = setup_vmgexit_scratch(svm, in, bytes); |
| 4596 | 4635 | if (r) |
| 4597 | 4636 | return r; |
| @@ -913,7 +913,15 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) · 912 unmodified lines | ||
| 913 | 913 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 914 | 914 | int old = control->pause_filter_count; |
| 915 | 915 | |
| 916 | − if (kvm_pause_in_guest(vcpu->kvm)) | |
| 916 | + /* Adjusting pause_filter_count makes no sense if PLE is disabled. */ | |
| 917 | + WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm)); | |
| 918 | + | |
| 919 | + /* | |
| 920 | + * While running L2, KVM should intercept PAUSE if and only if L1 wants | |
| 921 | + * to intercept PAUSE, and L1's intercept should take priority, i.e. | |
| 922 | + * KVM should never handle a PAUSE intercept from L2. | |
| 923 | + */ | |
| 924 | + if (WARN_ON_ONCE(is_guest_mode(vcpu))) | |
| 917 | 925 | return; |
| 918 | 926 | |
| 919 | 927 | control->pause_filter_count = __grow_ple_window(old, |
| @@ -934,7 +942,10 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) · 14 unmodified lines | ||
| 934 | 942 | struct vmcb_control_area *control = &svm->vmcb->control; |
| 935 | 943 | int old = control->pause_filter_count; |
| 936 | 944 | |
| 937 | − if (kvm_pause_in_guest(vcpu->kvm)) | |
| 945 | + /* Adjusting pause_filter_count makes no sense if PLE is disabled. */ | |
| 946 | + WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm)); | |
| 947 | + | |
| 948 | + if (is_guest_mode(vcpu)) | |
| 938 | 949 | return; |
| 939 | 950 | |
| 940 | 951 | control->pause_filter_count = |
| @@ -154,7 +154,7 @@ TRACE_EVENT(kvm_xen_hypercall, · 153 unmodified lines | ||
| 154 | 154 | __entry->a2 = a2; |
| 155 | 155 | __entry->a3 = a3; |
| 156 | 156 | __entry->a4 = a4; |
| 157 | − __entry->a4 = a5; | |
| 157 | + __entry->a5 = a5; | |
| 158 | 158 | ), |
| 159 | 159 | |
| 160 | 160 | TP_printk("cpl %d nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx a5 %lx", |
| @@ -14,6 +14,7 @@ extern bool __read_mostly flexpriority_enabled; · 13 unmodified lines | ||
| 14 | 14 | extern bool __read_mostly enable_ept; |
| 15 | 15 | extern bool __read_mostly enable_unrestricted_guest; |
| 16 | 16 | extern bool __read_mostly enable_ept_ad_bits; |
| 17 | +extern bool __read_mostly enable_cet; | |
| 17 | 18 | extern bool __read_mostly enable_pml; |
| 18 | 19 | extern int __read_mostly pt_mode; |
| 19 | 20 | |
| @@ -31,38 +31,6 @@ · 30 unmodified lines | ||
| 31 | 31 | #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE |
| 32 | 32 | #endif |
| 33 | 33 | |
| 34 | −.macro VMX_DO_EVENT_IRQOFF call_insn call_target | |
| 35 | − /* | |
| 36 | − * Unconditionally create a stack frame, getting the correct RSP on the | |
| 37 | − * stack (for x86-64) would take two instructions anyways, and RBP can | |
| 38 | − * be used to restore RSP to make objtool happy (see below). | |
| 39 | − */ | |
| 40 | − push %_ASM_BP | |
| 41 | − mov %_ASM_SP, %_ASM_BP | |
| 42 | − | |
| 43 | −#ifdef CONFIG_X86_64 | |
| 44 | − /* | |
| 45 | − * Align RSP to a 16-byte boundary (to emulate CPU behavior) before | |
| 46 | − * creating the synthetic interrupt stack frame for the IRQ/NMI. | |
| 47 | − */ | |
| 48 | − and $-16, %rsp | |
| 49 | − push $__KERNEL_DS | |
| 50 | − push %rbp | |
| 51 | −#endif | |
| 52 | − pushf | |
| 53 | − push $__KERNEL_CS | |
| 54 | − \call_insn \call_target | |
| 55 | − | |
| 56 | − /* | |
| 57 | − * "Restore" RSP from RBP, even though IRET has already unwound RSP to | |
| 58 | − * the correct value. objtool doesn't know the callee will IRET and, | |
| 59 | − * without the explicit restore, thinks the stack is getting walloped. | |
| 60 | − * Using an unwind hint is problematic due to x86-64's dynamic alignment. | |
| 61 | − */ | |
| 62 | − leave | |
| 63 | − RET | |
| 64 | −.endm | |
| 65 | − | |
| 66 | 34 | .section .noinstr.text, "ax" |
| 67 | 35 | |
| 68 | 36 | /** |
| @@ -320,10 +288,6 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) · 251 unmodified lines | ||
| 320 | 288 | |
| 321 | 289 | SYM_FUNC_END(__vmx_vcpu_run) |
| 322 | 290 | |
| 323 | −SYM_FUNC_START(vmx_do_nmi_irqoff) | |
| 324 | − VMX_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx | |
| 325 | −SYM_FUNC_END(vmx_do_nmi_irqoff) | |
| 326 | − | |
| 327 | 291 | #ifndef CONFIG_CC_HAS_ASM_GOTO_OUTPUT |
| 328 | 292 | |
| 329 | 293 | /** |
| @@ -375,13 +339,3 @@ SYM_FUNC_START(vmread_error_trampoline) · 45 unmodified lines | ||
| 375 | 339 | RET |
| 376 | 340 | SYM_FUNC_END(vmread_error_trampoline) |
| 377 | 341 | #endif |
| 378 | − | |
| 379 | −.section .text, "ax" | |
| 380 | − | |
| 381 | −#ifndef CONFIG_X86_FRED | |
| 382 | − | |
| 383 | −SYM_FUNC_START(vmx_do_interrupt_irqoff) | |
| 384 | − VMX_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 | |
| 385 | −SYM_FUNC_END(vmx_do_interrupt_irqoff) | |
| 386 | − | |
| 387 | −#endif | |
| @@ -108,6 +108,9 @@ module_param_named(unrestricted_guest, · 107 unmodified lines | ||
| 108 | 108 | bool __read_mostly enable_ept_ad_bits = 1; |
| 109 | 109 | module_param_named(eptad, enable_ept_ad_bits, bool, 0444); |
| 110 | 110 | |
| 111 | +bool __read_mostly enable_cet = 1; | |
| 112 | +module_param_named(cet, enable_cet, bool, 0444); | |
| 113 | + | |
| 111 | 114 | static bool __read_mostly emulate_invalid_guest_state = true; |
| 112 | 115 | module_param(emulate_invalid_guest_state, bool, 0444); |
| 113 | 116 | |
| @@ -4476,7 +4479,7 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) · 4362 unmodified lines | ||
| 4476 | 4479 | * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter |
| 4477 | 4480 | * 3 and 4 for details. |
| 4478 | 4481 | */ |
| 4479 | − if (cpu_has_load_cet_ctrl()) { | |
| 4482 | + if (enable_cet) { | |
| 4480 | 4483 | vmcs_writel(HOST_S_CET, kvm_host.s_cet); |
| 4481 | 4484 | vmcs_writel(HOST_SSP, 0); |
| 4482 | 4485 | vmcs_writel(HOST_INTR_SSP_TABLE, 0); |
| @@ -4532,6 +4535,10 @@ static u32 vmx_get_initial_vmentry_ctrl(void) · 49 unmodified lines | ||
| 4532 | 4535 | if (vmx_pt_mode_is_system()) |
| 4533 | 4536 | vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP | |
| 4534 | 4537 | VM_ENTRY_LOAD_IA32_RTIT_CTL); |
| 4538 | + | |
| 4539 | + if (!enable_cet) | |
| 4540 | + vmentry_ctrl &= ~VM_ENTRY_LOAD_CET_STATE; | |
| 4541 | + | |
| 4535 | 4542 | /* |
| 4536 | 4543 | * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically. |
| 4537 | 4544 | */ |
| @@ -4546,6 +4553,9 @@ static u32 vmx_get_initial_vmexit_ctrl(void) · 8 unmodified lines | ||
| 4546 | 4553 | { |
| 4547 | 4554 | u32 vmexit_ctrl = vmcs_config.vmexit_ctrl; |
| 4548 | 4555 | |
| 4556 | + if (!enable_cet) | |
| 4557 | + vmexit_ctrl &= ~VM_EXIT_LOAD_CET_STATE; | |
| 4558 | + | |
| 4549 | 4559 | /* |
| 4550 | 4560 | * Not used by KVM and never set in vmcs01 or vmcs02, but emulated for |
| 4551 | 4561 | * nested virtualization and thus allowed to be set in vmcs12. |
| @@ -7029,8 +7039,8 @@ static void vmx_set_rvi(int vector) · 2477 unmodified lines | ||
| 7029 | 7039 | int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) |
| 7030 | 7040 | { |
| 7031 | 7041 | struct vcpu_vt *vt = to_vt(vcpu); |
| 7042 | + bool max_irr_is_from_pir; | |
| 7032 | 7043 | int max_irr; |
| 7033 | − bool got_posted_interrupt; | |
| 7034 | 7044 | |
| 7035 | 7045 | if (KVM_BUG_ON(!enable_apicv, vcpu->kvm)) |
| 7036 | 7046 | return -EIO; |
| @@ -7042,17 +7052,22 @@ int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) · 5 unmodified lines | ||
| 7042 | 7052 | * But on x86 this is just a compiler barrier anyway. |
| 7043 | 7053 | */ |
| 7044 | 7054 | smp_mb__after_atomic(); |
| 7045 | − got_posted_interrupt = | |
| 7046 | − kvm_apic_update_irr(vcpu, vt->pi_desc.pir, &max_irr); | |
| 7055 | + max_irr_is_from_pir = kvm_apic_update_irr(vcpu, vt->pi_desc.pir, | |
| 7056 | + &max_irr); | |
| 7047 | 7057 | } else { |
| 7048 | 7058 | max_irr = kvm_lapic_find_highest_irr(vcpu); |
| 7049 | − got_posted_interrupt = false; | |
| 7059 | + max_irr_is_from_pir = false; | |
| 7050 | 7060 | } |
| 7051 | 7061 | |
| 7052 | 7062 | /* |
| 7053 | − * Newly recognized interrupts are injected via either virtual interrupt | |
| 7054 | − * delivery (RVI) or KVM_REQ_EVENT. Virtual interrupt delivery is | |
| 7055 | − * disabled in two cases: | |
| 7063 | + * If APICv is enabled and L2 is not active, then update the Requesting | |
| 7064 | + * Virtual Interrupt (RVI) portion of vmcs01.GUEST_INTR_STATUS with the | |
| 7065 | + * highest priority IRR to deliver the IRQ via Virtual Interrupt | |
| 7066 | + * Delivery. Note, this is required even if the highest priority IRQ | |
| 7067 | + * was already pending in the IRR, as RVI isn't updated in lockstep with | |
| 7068 | + * the IRR (unlike apic->irr_pending). | |
| 7069 | + * | |
| 7070 | + * For the cases where Virtual Interrupt Delivery can't be used: | |
| 7056 | 7071 | * |
| 7057 | 7072 | * 1) If L2 is running and the vCPU has a new pending interrupt. If L1 |
| 7058 | 7073 | * wants to exit on interrupts, KVM_REQ_EVENT is needed to synthesize a |
| @@ -7063,10 +7078,29 @@ int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) · 4 unmodified lines | ||
| 7063 | 7078 | * 2) If APICv is disabled for this vCPU, assigned devices may still |
| 7064 | 7079 | * attempt to post interrupts. The posted interrupt vector will cause |
| 7065 | 7080 | * a VM-Exit and the subsequent entry will call sync_pir_to_irr. |
| 7081 | + * | |
| 7082 | + * In both cases, set KVM_REQ_EVENT if and only if the highest priority | |
| 7083 | + * pending IRQ came from the PIR, as setting KVM_REQ_EVENT if any IRQ | |
| 7084 | + * is pending may put the vCPU into an infinite loop, e.g. if the IRQ | |
| 7085 | + * is blocked, then it will stay pending until an IRQ window is opened. | |
| 7086 | + * | |
| 7087 | + * Note! It's possible that one or more IRQs were moved from the PIR | |
| 7088 | + * to the IRR _without_ max_irr_is_from_pir being true! I.e. if there | |
| 7089 | + * was a higher priority IRQ already pending in the IRR. Not setting | |
| 7090 | + * KVM_REQ_EVENT in this case is intentional and safe. If APICv is | |
| 7091 | + * inactive, or L2 is running with exit-on-interrupt off (in vmcs12), | |
| 7092 | + * i.e. without nested virtual interrupt delivery, then there's no need | |
| 7093 | + * to request an IRQ window as the lower priority IRQ only needs to be | |
| 7094 | + * delivered when the higher priority IRQ is dismissed from the ISR, | |
| 7095 | + * i.e. on the next EOI, and EOIs are always intercepted if APICv is | |
| 7096 | + * disabled or if L2 is running without nested VID. If L2 is running | |
| 7097 | + * exit-on-interrupt on (in vmcs12), then the higher priority IRQ will | |
| 7098 | + * trigger a nested VM-Exit, at which point KVM will re-evaluate L1's | |
| 7099 | + * pending IRQs. | |
| 7066 | 7100 | */ |
| 7067 | 7101 | if (!is_guest_mode(vcpu) && kvm_vcpu_apicv_active(vcpu)) |
| 7068 | 7102 | vmx_set_rvi(max_irr); |
| 7069 | − else if (got_posted_interrupt) | |
| 7103 | + else if (max_irr_is_from_pir) | |
| 7070 | 7104 | kvm_make_request(KVM_REQ_EVENT, vcpu); |
| 7071 | 7105 | |
| 7072 | 7106 | return max_irr; |
| @@ -7083,9 +7117,6 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) · 10 unmodified lines | ||
| 7083 | 7117 | vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); |
| 7084 | 7118 | } |
| 7085 | 7119 | |
| 7086 | −void vmx_do_interrupt_irqoff(unsigned long entry); | |
| 7087 | −void vmx_do_nmi_irqoff(void); | |
| 7088 | − | |
| 7089 | 7120 | static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) |
| 7090 | 7121 | { |
| 7091 | 7122 | /* |
| @@ -7127,17 +7158,8 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, · 35 unmodified lines | ||
| 7127 | 7158 | "unexpected VM-Exit interrupt info: 0x%x", intr_info)) |
| 7128 | 7159 | return; |
| 7129 | 7160 | |
| 7130 | − /* | |
| 7131 | − * Invoke the kernel's IRQ handler for the vector. Use the FRED path | |
| 7132 | − * when it's available even if FRED isn't fully enabled, e.g. even if | |
| 7133 | − * FRED isn't supported in hardware, in order to avoid the indirect | |
| 7134 | − * CALL in the non-FRED path. | |
| 7135 | − */ | |
| 7136 | 7161 | kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); |
| 7137 | − if (IS_ENABLED(CONFIG_X86_FRED)) | |
| 7138 | − fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector); | |
| 7139 | − else | |
| 7140 | − vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector)); | |
| 7162 | + x86_entry_from_kvm(EVENT_TYPE_EXTINT, vector); | |
| 7141 | 7163 | kvm_after_interrupt(vcpu); |
| 7142 | 7164 | |
| 7143 | 7165 | vcpu->arch.at_instruction_boundary = true; |
| @@ -7447,10 +7469,7 @@ noinstr void vmx_handle_nmi(struct kvm_vcpu *vcpu) · 303 unmodified lines | ||
| 7447 | 7469 | return; |
| 7448 | 7470 | |
| 7449 | 7471 | kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); |
| 7450 | − if (cpu_feature_enabled(X86_FEATURE_FRED)) | |
| 7451 | − fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); | |
| 7452 | − else | |
| 7453 | − vmx_do_nmi_irqoff(); | |
| 7472 | + x86_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); | |
| 7454 | 7473 | kvm_after_interrupt(vcpu); |
| 7455 | 7474 | } |
| 7456 | 7475 | |
| @@ -8131,7 +8150,7 @@ static __init void vmx_set_cpu_caps(void) · 674 unmodified lines | ||
| 8131 | 8150 | * VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code |
| 8132 | 8151 | * fails, so disable CET in this case too. |
| 8133 | 8152 | */ |
| 8134 | − if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest || | |
| 8153 | + if (!enable_cet || !enable_unrestricted_guest || | |
| 8135 | 8154 | !cpu_has_vmx_basic_no_hw_errcode_cc()) { |
| 8136 | 8155 | kvm_cpu_cap_clear(X86_FEATURE_SHSTK); |
| 8137 | 8156 | kvm_cpu_cap_clear(X86_FEATURE_IBT); |
| @@ -8606,6 +8625,9 @@ __init int vmx_hardware_setup(void) · 468 unmodified lines | ||
| 8606 | 8625 | !cpu_has_vmx_invept_global()) |
| 8607 | 8626 | enable_ept = 0; |
| 8608 | 8627 | |
| 8628 | + if (!cpu_has_load_cet_ctrl()) | |
| 8629 | + enable_cet = 0; | |
| 8630 | + | |
| 8609 | 8631 | /* NX support is required for shadow paging. */ |
| 8610 | 8632 | if (!enable_ept && !boot_cpu_has(X86_FEATURE_NX)) { |
| 8611 | 8633 | pr_err_ratelimited("NX (Execute Disable) not supported\n"); |
| @@ -4876,7 +4876,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 4875 unmodified lines | ||
| 4876 | 4876 | r = tdp_enabled; |
| 4877 | 4877 | break; |
| 4878 | 4878 | case KVM_CAP_X86_APIC_BUS_CYCLES_NS: |
| 4879 | − r = APIC_BUS_CYCLE_NS_DEFAULT; | |
| 4879 | + r = kvm ? kvm->arch.apic_bus_cycle_ns : APIC_BUS_CYCLE_NS_DEFAULT; | |
| 4880 | 4880 | break; |
| 4881 | 4881 | case KVM_CAP_EXIT_HYPERCALL: |
| 4882 | 4882 | r = KVM_EXIT_HYPERCALL_VALID_MASK; |
| @@ -5227,8 +5227,13 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · 344 unmodified lines | ||
| 5227 | 5227 | * On a host with synchronized TSC, there is no need to update |
| 5228 | 5228 | * kvmclock on vcpu->cpu migration |
| 5229 | 5229 | */ |
| 5230 | − if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1) | |
| 5231 | − kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu); | |
| 5230 | + if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1) { | |
| 5231 | + if (__ratelimit(&vcpu->kvm->arch.kvmclock_update_rs)) | |
| 5232 | + kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu); | |
| 5233 | + else | |
| 5234 | + kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu); | |
| 5235 | + } | |
| 5236 | + | |
| 5232 | 5237 | if (vcpu->cpu != cpu) |
| 5233 | 5238 | kvm_make_request(KVM_REQ_MIGRATE_TIMER, vcpu); |
| 5234 | 5239 | vcpu->cpu = cpu; |
| @@ -13366,6 +13371,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) · 8131 unmodified lines | ||
| 13366 | 13371 | raw_spin_lock_init(&kvm->arch.tsc_write_lock); |
| 13367 | 13372 | mutex_init(&kvm->arch.apic_map_lock); |
| 13368 | 13373 | seqcount_raw_spinlock_init(&kvm->arch.pvclock_sc, &kvm->arch.tsc_write_lock); |
| 13374 | + ratelimit_state_init(&kvm->arch.kvmclock_update_rs, HZ, 10); | |
| 13375 | + ratelimit_set_flags(&kvm->arch.kvmclock_update_rs, RATELIMIT_MSG_ON_RELEASE); | |
| 13369 | 13376 | kvm->arch.kvmclock_offset = -get_kvmclock_base_ns(); |
| 13370 | 13377 | |
| 13371 | 13378 | raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); |
| @@ -14323,7 +14330,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) · 951 unmodified lines | ||
| 14323 | 14330 | * the RAP (Return Address Predicator). |
| 14324 | 14331 | */ |
| 14325 | 14332 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) |
| 14326 | − kvm_register_is_dirty(vcpu, VCPU_EXREG_ERAPS); | |
| 14333 | + kvm_register_mark_dirty(vcpu, VCPU_EXREG_ERAPS); | |
| 14327 | 14334 | |
| 14328 | 14335 | kvm_invalidate_pcid(vcpu, operand.pcid); |
| 14329 | 14336 | return kvm_skip_emulated_instruction(vcpu); |
| @@ -686,7 +686,7 @@ page_fault_oops(struct pt_regs *regs, unsigned long error_code, · 685 unmodified lines | ||
| 686 | 686 | * avoid hanging the system. |
| 687 | 687 | */ |
| 688 | 688 | if (IS_ENABLED(CONFIG_EFI)) |
| 689 | − efi_crash_gracefully_on_page_fault(address); | |
| 689 | + efi_crash_gracefully_on_page_fault(address, regs); | |
| 690 | 690 | |
| 691 | 691 | /* Only not-present faults should be handled by KFENCE. */ |
| 692 | 692 | if (!(error_code & X86_PF_PROT) && |
| @@ -761,7 +761,8 @@ int efi_capsule_setup_info(struct capsule_info *cap_info, void *kbuff, · 760 unmodified lines | ||
| 761 | 761 | * @return: Returns, if the page fault is not handled. This function |
| 762 | 762 | * will never return if the page fault is handled successfully. |
| 763 | 763 | */ |
| 764 | −void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) | |
| 764 | +void efi_crash_gracefully_on_page_fault(unsigned long phys_addr, | |
| 765 | + const struct pt_regs *regs) | |
| 765 | 766 | { |
| 766 | 767 | if (!IS_ENABLED(CONFIG_X86_64)) |
| 767 | 768 | return; |
| @@ -770,7 +771,7 @@ void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) · 2 unmodified lines | ||
| 770 | 771 | * If we get an interrupt/NMI while processing an EFI runtime service |
| 771 | 772 | * then this is a regular OOPS, not an EFI failure. |
| 772 | 773 | */ |
| 773 | − if (in_interrupt()) | |
| 774 | + if (!in_task()) | |
| 774 | 775 | return; |
| 775 | 776 | |
| 776 | 777 | /* |
| @@ -810,6 +811,14 @@ void efi_crash_gracefully_on_page_fault(unsigned long phys_addr) · 33 unmodified lines | ||
| 810 | 811 | return; |
| 811 | 812 | } |
| 812 | 813 | |
| 814 | + /* | |
| 815 | + * The API does not permit entering a kernel mode FPU section with | |
| 816 | + * interrupts enabled and leaving it with interrupts disabled. So | |
| 817 | + * re-enable interrupts now if they were enabled when the page fault | |
| 818 | + * occurred. | |
| 819 | + */ | |
| 820 | + local_irq_restore(regs->flags); | |
| 821 | + | |
| 813 | 822 | /* |
| 814 | 823 | * Before calling EFI Runtime Service, the kernel has switched the |
| 815 | 824 | * calling process to efi_mm. Hence, switch back to task_mm. |
| @@ -49,7 +49,20 @@ static void x86_virt_invoke_kvm_emergency_callback(void) · 48 unmodified lines | ||
| 49 | 49 | { |
| 50 | 50 | cpu_emergency_virt_cb *kvm_callback; |
| 51 | 51 | |
| 52 | − kvm_callback = rcu_dereference(kvm_emergency_callback); | |
| 52 | + /* | |
| 53 | + * RCU may not be watching the crashing CPU here, so rcu_dereference() | |
| 54 | + * triggers a suspicious-RCU-usage splat. In principle, a concurrent | |
| 55 | + * KVM module unload could race with this read; see commit 2baa33a8ddd6 | |
| 56 | + * ("KVM: x86: Leave user-return notifier registered on reboot/shutdown") | |
| 57 | + * which notes that nothing prevents module unload during panic/reboot. | |
| 58 | + * | |
| 59 | + * However, taking a lock here would be riskier than the current race: | |
| 60 | + * the system is going down via NMI shootdown, and any lock could be | |
| 61 | + * held by an already-stopped CPU. Use rcu_dereference_raw() to silence | |
| 62 | + * the lockdep splat and accept the comically small remaining race; | |
| 63 | + * panic context inherently cannot guarantee complete correctness. | |
| 64 | + */ | |
| 65 | + kvm_callback = rcu_dereference_raw(kvm_emergency_callback); | |
| 53 | 66 | if (kvm_callback) |
| 54 | 67 | kvm_callback(); |
| 55 | 68 | } |
| @@ -2145,7 +2145,10 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) · 2144 unmodified lines | ||
| 2145 | 2145 | |
| 2146 | 2146 | static void xen_enter_lazy_mmu(void) |
| 2147 | 2147 | { |
| 2148 | − enter_lazy(XEN_LAZY_MMU); | |
| 2148 | + preempt_disable(); | |
| 2149 | + if (xen_get_lazy_mode() != XEN_LAZY_MMU) | |
| 2150 | + enter_lazy(XEN_LAZY_MMU); | |
| 2151 | + preempt_enable(); | |
| 2149 | 2152 | } |
| 2150 | 2153 | |
| 2151 | 2154 | static void xen_flush_lazy_mmu(void) |
| @@ -2182,7 +2185,8 @@ static void xen_leave_lazy_mmu(void) · 30 unmodified lines | ||
| 2182 | 2185 | { |
| 2183 | 2186 | preempt_disable(); |
| 2184 | 2187 | xen_mc_flush(); |
| 2185 | − leave_lazy(XEN_LAZY_MMU); | |
| 2188 | + if (xen_get_lazy_mode() != XEN_LAZY_NONE) | |
| 2189 | + leave_lazy(XEN_LAZY_MMU); | |
| 2186 | 2190 | preempt_enable(); |
| 2187 | 2191 | } |
| 2188 | 2192 | |
| @@ -655,7 +655,7 @@ static void __init xen_e820_swap_entry_with_ram(struct e820_entry *swap_entry) · 654 unmodified lines | ||
| 655 | 655 | /* Fill new entry (keep size and page offset). */ |
| 656 | 656 | entry->type = swap_entry->type; |
| 657 | 657 | entry->addr = entry_end - swap_size + |
| 658 | − swap_addr - swap_entry->addr; | |
| 658 | + swap_entry->addr - swap_addr; | |
| 659 | 659 | entry->size = swap_entry->size; |
| 660 | 660 | |
| 661 | 661 | /* Convert old entry to RAM, align to pages. */ |
| @@ -695,17 +695,22 @@ static void __init xen_e820_resolve_conflicts(phys_addr_t start, · 33 unmodified lines | ||
| 695 | 695 | return; |
| 696 | 696 | |
| 697 | 697 | end = start + size; |
| 698 | − entry = xen_e820_table.entries; | |
| 698 | + mapcnt = 0; | |
| 699 | 699 | |
| 700 | − for (mapcnt = 0; mapcnt < xen_e820_table.nr_entries; mapcnt++) { | |
| 700 | + while (mapcnt < xen_e820_table.nr_entries) { | |
| 701 | + entry = xen_e820_table.entries + mapcnt; | |
| 701 | 702 | if (entry->addr >= end) |
| 702 | 703 | return; |
| 703 | 704 | |
| 704 | 705 | if (entry->addr + entry->size > start && |
| 705 | − entry->type == E820_TYPE_NVS) | |
| 706 | + entry->type == E820_TYPE_NVS) { | |
| 706 | 707 | xen_e820_swap_entry_with_ram(entry); |
| 708 | + /* E820 map has been changed, restart loop! */ | |
| 709 | + mapcnt = 0; | |
| 710 | + continue; | |
| 711 | + } | |
| 707 | 712 | |
| 708 | − entry++; | |
| 713 | + mapcnt++; | |
| 709 | 714 | } |
| 710 | 715 | } |
| 711 | 716 | |
| @@ -6,5 +6,6 @@ generic-y += mcs_spinlock.h · 5 unmodified lines | ||
| 6 | 6 | generic-y += parport.h |
| 7 | 7 | generic-y += qrwlock.h |
| 8 | 8 | generic-y += qspinlock.h |
| 9 | +generic-y += ring_buffer.h | |
| 9 | 10 | generic-y += user.h |
| 10 | 11 | generic-y += text-patching.h |
| @@ -55,6 +55,10 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size) · 54 unmodified lines | ||
| 55 | 55 | { |
| 56 | 56 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); |
| 57 | 57 | struct bio_integrity_payload *bip = bio_integrity(bio); |
| 58 | + struct bvec_iter data_iter = { | |
| 59 | + .bi_sector = sector, | |
| 60 | + .bi_size = size, | |
| 61 | + }; | |
| 58 | 62 | |
| 59 | 63 | /* |
| 60 | 64 | * Reinitialize bip->bip_iter. |
| @@ -65,7 +69,7 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size) · 4 unmodified lines | ||
| 65 | 69 | memset(&bip->bip_iter, 0, sizeof(bip->bip_iter)); |
| 66 | 70 | bip->bip_iter.bi_sector = sector; |
| 67 | 71 | bip->bip_iter.bi_size = bio_integrity_bytes(bi, size >> SECTOR_SHIFT); |
| 68 | − return blk_status_to_errno(bio_integrity_verify(bio, &bip->bip_iter)); | |
| 72 | + return blk_status_to_errno(bio_integrity_verify(bio, &data_iter)); | |
| 69 | 73 | } |
| 70 | 74 | |
| 71 | 75 | static int __init fs_bio_integrity_init(void) |
| @@ -308,7 +308,6 @@ static int bio_integrity_copy_user(struct bio *bio, struct bio_vec *bvec, · 307 unmodified lines | ||
| 308 | 308 | } |
| 309 | 309 | |
| 310 | 310 | bip->bip_flags |= BIP_COPY_USER; |
| 311 | − bip->bip_vcnt = nr_vecs; | |
| 312 | 311 | return 0; |
| 313 | 312 | free_bip: |
| 314 | 313 | bio_integrity_free(bio); |
| @@ -403,6 +402,24 @@ int bio_integrity_map_user(struct bio *bio, struct iov_iter *iter) · 88 unmodified lines | ||
| 403 | 402 | if (unlikely(ret < 0)) |
| 404 | 403 | goto free_bvec; |
| 405 | 404 | |
| 405 | + /* | |
| 406 | + * Handle partial pinning. This can happen when pin_user_pages_fast() | |
| 407 | + * returns fewer pages than requested. | |
| 408 | + */ | |
| 409 | + if (user_backed_iter(iter) && unlikely(ret != bytes)) { | |
| 410 | + if (ret > 0) { | |
| 411 | + int npinned = DIV_ROUND_UP(offset + ret, PAGE_SIZE); | |
| 412 | + int i; | |
| 413 | + | |
| 414 | + for (i = 0; i < npinned; i++) | |
| 415 | + unpin_user_page(pages[i]); | |
| 416 | + } | |
| 417 | + if (pages != stack_pages) | |
| 418 | + kvfree(pages); | |
| 419 | + ret = -EFAULT; | |
| 420 | + goto free_bvec; | |
| 421 | + } | |
| 422 | + | |
| 406 | 423 | nr_bvecs = bvec_from_pages(bvec, pages, nr_vecs, bytes, offset, |
| 407 | 424 | &is_p2p); |
| 408 | 425 | if (pages != stack_pages) |
| @@ -1279,11 +1279,12 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, · 1278 unmodified lines | ||
| 1279 | 1279 | return bio_iov_iter_align_down(bio, iter, len_align_mask); |
| 1280 | 1280 | } |
| 1281 | 1281 | |
| 1282 | −static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size) | |
| 1282 | +static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size, | |
| 1283 | + size_t minsize) | |
| 1283 | 1284 | { |
| 1284 | 1285 | struct folio *folio; |
| 1285 | 1286 | |
| 1286 | − while (*size > PAGE_SIZE) { | |
| 1287 | + while (*size > minsize) { | |
| 1287 | 1288 | folio = folio_alloc(gfp | __GFP_NORETRY, get_order(*size)); |
| 1288 | 1289 | if (folio) |
| 1289 | 1290 | return folio; |
| @@ -1307,7 +1308,7 @@ static void bio_free_folios(struct bio *bio) · 17 unmodified lines | ||
| 1307 | 1308 | } |
| 1308 | 1309 | |
| 1309 | 1310 | static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, |
| 1310 | − size_t maxlen) | |
| 1311 | + size_t maxlen, size_t minsize) | |
| 1311 | 1312 | { |
| 1312 | 1313 | size_t total_len = min(maxlen, iov_iter_count(iter)); |
| 1313 | 1314 | |
| @@ -1322,13 +1323,13 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, · 8 unmodified lines | ||
| 1322 | 1323 | size_t this_len = min(total_len, SZ_1M); |
| 1323 | 1324 | struct folio *folio; |
| 1324 | 1325 | |
| 1325 | − if (this_len > PAGE_SIZE * 2) | |
| 1326 | + if (this_len > minsize * 2) | |
| 1326 | 1327 | this_len = rounddown_pow_of_two(this_len); |
| 1327 | 1328 | |
| 1328 | 1329 | if (bio->bi_iter.bi_size > BIO_MAX_SIZE - this_len) |
| 1329 | 1330 | break; |
| 1330 | 1331 | |
| 1331 | − folio = folio_alloc_greedy(GFP_KERNEL, &this_len); | |
| 1332 | + folio = folio_alloc_greedy(GFP_KERNEL, &this_len, minsize); | |
| 1332 | 1333 | if (!folio) |
| 1333 | 1334 | break; |
| 1334 | 1335 | bio_add_folio_nofail(bio, folio, this_len, 0); |
| @@ -1344,16 +1345,16 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter, · 9 unmodified lines | ||
| 1344 | 1345 | |
| 1345 | 1346 | if (!bio->bi_iter.bi_size) |
| 1346 | 1347 | return -ENOMEM; |
| 1347 | − return 0; | |
| 1348 | + return bio_iov_iter_align_down(bio, iter, minsize - 1); | |
| 1348 | 1349 | } |
| 1349 | 1350 | |
| 1350 | 1351 | static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, |
| 1351 | − size_t maxlen) | |
| 1352 | + size_t maxlen, size_t minsize) | |
| 1352 | 1353 | { |
| 1353 | 1354 | size_t len = min3(iov_iter_count(iter), maxlen, SZ_1M); |
| 1354 | 1355 | struct folio *folio; |
| 1355 | 1356 | |
| 1356 | − folio = folio_alloc_greedy(GFP_KERNEL, &len); | |
| 1357 | + folio = folio_alloc_greedy(GFP_KERNEL, &len, minsize); | |
| 1357 | 1358 | if (!folio) |
| 1358 | 1359 | return -ENOMEM; |
| 1359 | 1360 | |
| @@ -1382,7 +1383,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 22 unmodified lines | ||
| 1382 | 1383 | bvec_set_folio(&bio->bi_io_vec[0], folio, bio->bi_iter.bi_size, 0); |
| 1383 | 1384 | if (iov_iter_extract_will_pin(iter)) |
| 1384 | 1385 | bio_set_flag(bio, BIO_PAGE_PINNED); |
| 1385 | − return 0; | |
| 1386 | + return bio_iov_iter_align_down(bio, iter, minsize - 1); | |
| 1386 | 1387 | } |
| 1387 | 1388 | |
| 1388 | 1389 | /** |
| @@ -1390,6 +1391,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 1 unmodified line | ||
| 1390 | 1391 | * @bio: bio to send |
| 1391 | 1392 | * @iter: iter to read from / write into |
| 1392 | 1393 | * @maxlen: maximum size to bounce |
| 1394 | + * @minsize: minimum folio allocation size | |
| 1393 | 1395 | * |
| 1394 | 1396 | * Helper for direct I/O implementations that need to bounce buffer because |
| 1395 | 1397 | * we need to checksum the data or perform other operations that require |
| @@ -1397,11 +1399,12 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter, · 1 unmodified line | ||
| 1397 | 1399 | * copies the data into it. Needs to be paired with bio_iov_iter_unbounce() |
| 1398 | 1400 | * called on completion. |
| 1399 | 1401 | */ |
| 1400 | −int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen) | |
| 1402 | +int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen, | |
| 1403 | + size_t minsize) | |
| 1401 | 1404 | { |
| 1402 | 1405 | if (op_is_write(bio_op(bio))) |
| 1403 | − return bio_iov_iter_bounce_write(bio, iter, maxlen); | |
| 1404 | − return bio_iov_iter_bounce_read(bio, iter, maxlen); | |
| 1406 | + return bio_iov_iter_bounce_write(bio, iter, maxlen, minsize); | |
| 1407 | + return bio_iov_iter_bounce_read(bio, iter, maxlen, minsize); | |
| 1405 | 1408 | } |
| 1406 | 1409 | |
| 1407 | 1410 | static void bvec_unpin(struct bio_vec *bv, bool mark_dirty) |
| @@ -2241,7 +2241,7 @@ void blk_cgroup_bio_start(struct bio *bio) · 2240 unmodified lines | ||
| 2241 | 2241 | } |
| 2242 | 2242 | |
| 2243 | 2243 | u64_stats_update_end_irqrestore(&bis->sync, flags); |
| 2244 | − css_rstat_updated(&blkcg->css, cpu); | |
| 2244 | + __css_rstat_updated(&blkcg->css, cpu); | |
| 2245 | 2245 | put_cpu(); |
| 2246 | 2246 | } |
| 2247 | 2247 | |
| @@ -3077,7 +3077,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 3076 unmodified lines | ||
| 3077 | 3077 | /* |
| 3078 | 3078 | * Check if there is a suitable cached request and return it. |
| 3079 | 3079 | */ |
| 3080 | −static struct request *blk_mq_peek_cached_request(struct blk_plug *plug, | |
| 3080 | +static struct request *blk_mq_get_cached_request(struct blk_plug *plug, | |
| 3081 | 3081 | struct request_queue *q, blk_opf_t opf) |
| 3082 | 3082 | { |
| 3083 | 3083 | enum hctx_type type = blk_mq_get_hctx_type(opf); |
| @@ -3093,27 +3093,10 @@ static struct request *blk_mq_peek_cached_request(struct blk_plug *plug, · 9 unmodified lines | ||
| 3093 | 3093 | return NULL; |
| 3094 | 3094 | if (op_is_flush(rq->cmd_flags) != op_is_flush(opf)) |
| 3095 | 3095 | return NULL; |
| 3096 | + rq_list_pop(&plug->cached_rqs); | |
| 3096 | 3097 | return rq; |
| 3097 | 3098 | } |
| 3098 | 3099 | |
| 3099 | −static void blk_mq_use_cached_rq(struct request *rq, struct blk_plug *plug, | |
| 3100 | − struct bio *bio) | |
| 3101 | −{ | |
| 3102 | − if (rq_list_pop(&plug->cached_rqs) != rq) | |
| 3103 | − WARN_ON_ONCE(1); | |
| 3104 | − | |
| 3105 | − /* | |
| 3106 | − * If any qos ->throttle() end up blocking, we will have flushed the | |
| 3107 | − * plug and hence killed the cached_rq list as well. Pop this entry | |
| 3108 | − * before we throttle. | |
| 3109 | − */ | |
| 3110 | − rq_qos_throttle(rq->q, bio); | |
| 3111 | − | |
| 3112 | − blk_mq_rq_time_init(rq, blk_time_get_ns()); | |
| 3113 | − rq->cmd_flags = bio->bi_opf; | |
| 3114 | − INIT_LIST_HEAD(&rq->queuelist); | |
| 3115 | −} | |
| 3116 | − | |
| 3117 | 3100 | static bool bio_unaligned(const struct bio *bio, struct request_queue *q) |
| 3118 | 3101 | { |
| 3119 | 3102 | unsigned int bs_mask = queue_logical_block_size(q) - 1; |
| @@ -3152,7 +3135,7 @@ void blk_mq_submit_bio(struct bio *bio) · 32 unmodified lines | ||
| 3152 | 3135 | /* |
| 3153 | 3136 | * If the plug has a cached request for this queue, try to use it. |
| 3154 | 3137 | */ |
| 3155 | − rq = blk_mq_peek_cached_request(plug, q, bio->bi_opf); | |
| 3138 | + rq = blk_mq_get_cached_request(plug, q, bio->bi_opf); | |
| 3156 | 3139 | |
| 3157 | 3140 | /* |
| 3158 | 3141 | * A BIO that was released from a zone write plug has already been |
| @@ -3211,7 +3194,10 @@ void blk_mq_submit_bio(struct bio *bio) · 52 unmodified lines | ||
| 3211 | 3194 | |
| 3212 | 3195 | new_request: |
| 3213 | 3196 | if (rq) { |
| 3214 | − blk_mq_use_cached_rq(rq, plug, bio); | |
| 3197 | + rq_qos_throttle(rq->q, bio); | |
| 3198 | + blk_mq_rq_time_init(rq, blk_time_get_ns()); | |
| 3199 | + rq->cmd_flags = bio->bi_opf; | |
| 3200 | + INIT_LIST_HEAD(&rq->queuelist); | |
| 3215 | 3201 | } else { |
| 3216 | 3202 | rq = blk_mq_get_new_requests(q, plug, bio); |
| 3217 | 3203 | if (unlikely(!rq)) { |
| @@ -3257,12 +3243,10 @@ new_request: · 39 unmodified lines | ||
| 3257 | 3243 | return; |
| 3258 | 3244 | |
| 3259 | 3245 | queue_exit: |
| 3260 | − /* | |
| 3261 | − * Don't drop the queue reference if we were trying to use a cached | |
| 3262 | − * request and thus didn't acquire one. | |
| 3263 | − */ | |
| 3264 | 3246 | if (!rq) |
| 3265 | 3247 | blk_queue_exit(q); |
| 3248 | + else | |
| 3249 | + rq_list_add_head(&plug->cached_rqs, rq); | |
| 3266 | 3250 | } |
| 3267 | 3251 | |
| 3268 | 3252 | #ifdef CONFIG_BLK_MQ_STACKING |
| @@ -3307,6 +3291,25 @@ blk_status_t blk_insert_cloned_request(struct request *rq) · 38 unmodified lines | ||
| 3307 | 3291 | return BLK_STS_IOERR; |
| 3308 | 3292 | } |
| 3309 | 3293 | |
| 3294 | + /* | |
| 3295 | + * Integrity segment counting depends on the same queue limits | |
| 3296 | + * (virt_boundary_mask, seg_boundary_mask, max_segment_size) that | |
| 3297 | + * vary across stacked queues, so recompute against the bottom | |
| 3298 | + * queue just like nr_phys_segments above. | |
| 3299 | + */ | |
| 3300 | + if (blk_integrity_rq(rq) && rq->bio) { | |
| 3301 | + unsigned short max_int_segs = queue_max_integrity_segments(q); | |
| 3302 | + | |
| 3303 | + rq->nr_integrity_segments = | |
| 3304 | + blk_rq_count_integrity_sg(rq->q, rq->bio); | |
| 3305 | + if (rq->nr_integrity_segments > max_int_segs) { | |
| 3306 | + printk(KERN_ERR "%s: over max integrity segments limit. (%u > %u)\n", | |
| 3307 | + __func__, rq->nr_integrity_segments, | |
| 3308 | + max_int_segs); | |
| 3309 | + return BLK_STS_IOERR; | |
| 3310 | + } | |
| 3311 | + } | |
| 3312 | + | |
| 3310 | 3313 | if (q->disk && should_fail_request(q->disk->part0, blk_rq_bytes(rq))) |
| 3311 | 3314 | return BLK_STS_IOERR; |
| 3312 | 3315 | |
| @@ -623,6 +623,28 @@ static void disk_mark_zone_wplug_dead(struct blk_zone_wplug *zwplug) · 622 unmodified lines | ||
| 623 | 623 | } |
| 624 | 624 | } |
| 625 | 625 | |
| 626 | +static inline bool disk_check_zone_wplug_dead(struct blk_zone_wplug *zwplug) | |
| 627 | +{ | |
| 628 | + if (!(zwplug->flags & BLK_ZONE_WPLUG_DEAD)) | |
| 629 | + return false; | |
| 630 | + | |
| 631 | + /* | |
| 632 | + * If a new write is received right after a zone reset completes and | |
| 633 | + * while the disk_zone_wplugs_worker() thread has not yet released the | |
| 634 | + * reference on the zone write plug after processing the last write to | |
| 635 | + * the zone, then the new write BIO will see the zone write plug marked | |
| 636 | + * as dead. This case is however a false positive and a perfectly valid | |
| 637 | + * pattern. In such case, restore the zone write plug to a live one. | |
| 638 | + */ | |
| 639 | + if (!zwplug->wp_offset && bio_list_empty(&zwplug->bio_list)) { | |
| 640 | + zwplug->flags &= ~BLK_ZONE_WPLUG_DEAD; | |
| 641 | + refcount_inc(&zwplug->ref); | |
| 642 | + return false; | |
| 643 | + } | |
| 644 | + | |
| 645 | + return true; | |
| 646 | +} | |
| 647 | + | |
| 626 | 648 | static bool disk_zone_wplug_submit_bio(struct gendisk *disk, |
| 627 | 649 | struct blk_zone_wplug *zwplug); |
| 628 | 650 | |
| @@ -1444,12 +1466,12 @@ static bool blk_zone_wplug_handle_write(struct bio *bio, unsigned int nr_segs) · 815 unmodified lines | ||
| 1444 | 1466 | spin_lock_irqsave(&zwplug->lock, flags); |
| 1445 | 1467 | |
| 1446 | 1468 | /* |
| 1447 | − * If we got a zone write plug marked as dead, then the user is issuing | |
| 1448 | − * writes to a full zone, or without synchronizing with zone reset or | |
| 1449 | − * zone finish operations. In such case, fail the BIO to signal this | |
| 1450 | − * invalid usage. | |
| 1469 | + * Check if we got a zone write plug marked as dead. If yes, then the | |
| 1470 | + * user is likely issuing writes to a full zone, or without | |
| 1471 | + * synchronizing with zone reset or zone finish operations. In such | |
| 1472 | + * case, fail the BIO to signal this invalid usage. | |
| 1451 | 1473 | */ |
| 1452 | − if (zwplug->flags & BLK_ZONE_WPLUG_DEAD) { | |
| 1474 | + if (disk_check_zone_wplug_dead(zwplug)) { | |
| 1453 | 1475 | spin_unlock_irqrestore(&zwplug->lock, flags); |
| 1454 | 1476 | disk_put_zone_wplug(zwplug); |
| 1455 | 1477 | bio_io_error(bio); |
| @@ -1979,8 +2001,10 @@ static void disk_set_zones_cond_array(struct gendisk *disk, u8 *zones_cond) · 523 unmodified lines | ||
| 1979 | 2001 | |
| 1980 | 2002 | void disk_free_zone_resources(struct gendisk *disk) |
| 1981 | 2003 | { |
| 1982 | − if (disk->zone_wplugs_worker) | |
| 2004 | + if (disk->zone_wplugs_worker) { | |
| 1983 | 2005 | kthread_stop(disk->zone_wplugs_worker); |
| 2006 | + disk->zone_wplugs_worker = NULL; | |
| 2007 | + } | |
| 1984 | 2008 | WARN_ON_ONCE(!list_empty(&disk->zone_wplugs_list)); |
| 1985 | 2009 | |
| 1986 | 2010 | if (disk->zone_wplugs_wq) { |
| @@ -2113,9 +2137,6 @@ commit: · 126 unmodified lines | ||
| 2113 | 2137 | ret = queue_limits_commit_update(q, &lim); |
| 2114 | 2138 | |
| 2115 | 2139 | unfreeze: |
| 2116 | − if (ret) | |
| 2117 | − disk_free_zone_resources(disk); | |
| 2118 | − | |
| 2119 | 2140 | blk_mq_unfreeze_queue(q, memflags); |
| 2120 | 2141 | |
| 2121 | 2142 | return ret; |
| @@ -857,6 +857,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) · 856 unmodified lines | ||
| 857 | 857 | #endif |
| 858 | 858 | |
| 859 | 859 | struct blk_iou_cmd { |
| 860 | + u64 start; | |
| 861 | + u64 len; | |
| 860 | 862 | int res; |
| 861 | 863 | bool nowait; |
| 862 | 864 | }; |
| @@ -946,23 +948,27 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) · 83 unmodified lines | ||
| 946 | 948 | { |
| 947 | 949 | struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); |
| 948 | 950 | struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); |
| 949 | − const struct io_uring_sqe *sqe = cmd->sqe; | |
| 950 | 951 | u32 cmd_op = cmd->cmd_op; |
| 951 | − uint64_t start, len; | |
| 952 | 952 | |
| 953 | − if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || | |
| 954 | − sqe->rw_flags || sqe->file_index)) | |
| 955 | − return -EINVAL; | |
| 953 | + /* Read what we need from the SQE on the first issue */ | |
| 954 | + if (!(issue_flags & IORING_URING_CMD_REISSUE)) { | |
| 955 | + const struct io_uring_sqe *sqe = cmd->sqe; | |
| 956 | + | |
| 957 | + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || | |
| 958 | + sqe->rw_flags || sqe->file_index)) | |
| 959 | + return -EINVAL; | |
| 960 | + | |
| 961 | + bic->start = READ_ONCE(sqe->addr); | |
| 962 | + bic->len = READ_ONCE(sqe->addr3); | |
| 963 | + } | |
| 956 | 964 | |
| 957 | 965 | bic->res = 0; |
| 958 | 966 | bic->nowait = issue_flags & IO_URING_F_NONBLOCK; |
| 959 | 967 | |
| 960 | − start = READ_ONCE(sqe->addr); | |
| 961 | − len = READ_ONCE(sqe->addr3); | |
| 962 | − | |
| 963 | 968 | switch (cmd_op) { |
| 964 | 969 | case BLOCK_URING_CMD_DISCARD: |
| 965 | − return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); | |
| 970 | + return blkdev_cmd_discard(cmd, bdev, bic->start, bic->len, | |
| 971 | + bic->nowait); | |
| 966 | 972 | } |
| 967 | 973 | return -EINVAL; |
| 968 | 974 | } |
| @@ -390,6 +390,11 @@ static int crypto_authenc_esn_create(struct crypto_template *tmpl, · 389 unmodified lines | ||
| 390 | 390 | auth = crypto_spawn_ahash_alg(&ctx->auth); |
| 391 | 391 | auth_base = &auth->base; |
| 392 | 392 | |
| 393 | + if (auth->digestsize > 0 && auth->digestsize < 4) { | |
| 394 | + err = -EINVAL; | |
| 395 | + goto err_free_inst; | |
| 396 | + } | |
| 397 | + | |
| 393 | 398 | err = crypto_grab_skcipher(&ctx->enc, aead_crypto_instance(inst), |
| 394 | 399 | crypto_attr_alg_name(tb[2]), 0, mask); |
| 395 | 400 | if (err) |
| @@ -134,27 +134,69 @@ EXPORT_SYMBOL(crypto_krb5_how_much_data); · 133 unmodified lines | ||
| 134 | 134 | * Find the offset and size of the data in a secure message so that this |
| 135 | 135 | * information can be used in the metadata buffer which will get added to the |
| 136 | 136 | * digest by crypto_krb5_verify_mic(). |
| 137 | + * | |
| 138 | + * Return: 0 if successful, -EBADMSG if the message is too short or -EINVAL if | |
| 139 | + * the mode is unsupported. | |
| 137 | 140 | */ |
| 138 | −void crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | |
| 139 | − enum krb5_crypto_mode mode, | |
| 140 | − size_t *_offset, size_t *_len) | |
| 141 | +int crypto_krb5_where_is_the_data(const struct krb5_enctype *krb5, | |
| 142 | + enum krb5_crypto_mode mode, | |
| 143 | + size_t *_offset, size_t *_len) | |
| 141 | 144 | { |
| 142 | 145 | switch (mode) { |
| 143 | 146 | case KRB5_CHECKSUM_MODE: |
| 147 | + if (*_len < krb5->cksum_len) | |
| 148 | + return -EBADMSG; | |
| 144 | 149 | *_offset += krb5->cksum_len; |
| 145 | 150 | *_len -= krb5->cksum_len; |
| 146 | − return; | |
| 151 | + return 0; | |
| 147 | 152 | case KRB5_ENCRYPT_MODE: |
| 153 | + if (*_len < krb5->conf_len + krb5->cksum_len) | |
| 154 | + return -EBADMSG; | |
| 148 | 155 | *_offset += krb5->conf_len; |
| 149 | 156 | *_len -= krb5->conf_len + krb5->cksum_len; |
| 150 | − return; | |
| 157 | + return 0; | |
| 151 | 158 | default: |
| 152 | 159 | WARN_ON_ONCE(1); |
| 153 | − return; | |
| 160 | + return -EINVAL; | |
| 154 | 161 | } |
| 155 | 162 | } |
| 156 | 163 | EXPORT_SYMBOL(crypto_krb5_where_is_the_data); |
| 157 | 164 | |
| 165 | +/** | |
| 166 | + * crypto_krb5_check_data_len - Check a message is big enough | |
| 167 | + * @krb5: The encoding to use. | |
| 168 | + * @mode: Mode of operation. | |
| 169 | + * @len: The length of the secure blob. | |
| 170 | + * @min_content: Minimum length of the content inside the blob. | |
| 171 | + * | |
| 172 | + * Check that a message is large enough to hold whatever bits the encryption | |
| 173 | + * type wants to glue on (nonce, checksum) plus a minimum amount of content. | |
| 174 | + * | |
| 175 | + * Return: 0 if successful, -EBADMSG if the message is too short or -EINVAL if | |
| 176 | + * the mode is unsupported. | |
| 177 | + */ | |
| 178 | +int crypto_krb5_check_data_len(const struct krb5_enctype *krb5, | |
| 179 | + enum krb5_crypto_mode mode, | |
| 180 | + size_t len, size_t min_content) | |
| 181 | +{ | |
| 182 | + switch (mode) { | |
| 183 | + case KRB5_CHECKSUM_MODE: | |
| 184 | + if (len < krb5->cksum_len || | |
| 185 | + len - krb5->cksum_len < min_content) | |
| 186 | + return -EBADMSG; | |
| 187 | + return 0; | |
| 188 | + case KRB5_ENCRYPT_MODE: | |
| 189 | + if (len < krb5->conf_len + krb5->cksum_len || | |
| 190 | + len - (krb5->conf_len + krb5->cksum_len) < min_content) | |
| 191 | + return -EBADMSG; | |
| 192 | + return 0; | |
| 193 | + default: | |
| 194 | + WARN_ON_ONCE(1); | |
| 195 | + return -EINVAL; | |
| 196 | + } | |
| 197 | +} | |
| 198 | +EXPORT_SYMBOL(crypto_krb5_check_data_len); | |
| 199 | + | |
| 158 | 200 | /* |
| 159 | 201 | * Prepare the encryption with derived key data. |
| 160 | 202 | */ |
| @@ -511,6 +511,11 @@ static int aie2_init(struct amdxdna_dev *xdna) · 510 unmodified lines | ||
| 511 | 511 | return -EINVAL; |
| 512 | 512 | } |
| 513 | 513 | |
| 514 | + if (!xdna->group) { | |
| 515 | + XDNA_ERR(xdna, "Running without IOMMU not supported"); | |
| 516 | + return -EINVAL; | |
| 517 | + } | |
| 518 | + | |
| 514 | 519 | ndev = drmm_kzalloc(&xdna->ddev, sizeof(*ndev), GFP_KERNEL); |
| 515 | 520 | if (!ndev) |
| 516 | 521 | return -ENOMEM; |
| @@ -490,6 +490,9 @@ static struct dma_buf *amdxdna_gem_prime_export(struct drm_gem_object *gobj, int · 489 unmodified lines | ||
| 490 | 490 | struct amdxdna_gem_obj *abo = to_xdna_obj(gobj); |
| 491 | 491 | DEFINE_DMA_BUF_EXPORT_INFO(exp_info); |
| 492 | 492 | |
| 493 | + if (abo->private_buffer) | |
| 494 | + return ERR_PTR(-EOPNOTSUPP); | |
| 495 | + | |
| 493 | 496 | if (abo->dma_buf) { |
| 494 | 497 | get_dma_buf(abo->dma_buf); |
| 495 | 498 | return abo->dma_buf; |
| @@ -685,6 +688,7 @@ amdxdna_gem_create_ubuf_object(struct drm_device *dev, struct amdxdna_drm_create · 189 unmodified lines | ||
| 685 | 688 | { |
| 686 | 689 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 687 | 690 | struct amdxdna_drm_va_tbl va_tbl; |
| 691 | + struct amdxdna_gem_obj *abo; | |
| 688 | 692 | struct drm_gem_object *gobj; |
| 689 | 693 | struct dma_buf *dma_buf; |
| 690 | 694 | |
| @@ -711,7 +715,10 @@ amdxdna_gem_create_ubuf_object(struct drm_device *dev, struct amdxdna_drm_create · 20 unmodified lines | ||
| 711 | 715 | |
| 712 | 716 | dma_buf_put(dma_buf); |
| 713 | 717 | |
| 714 | − return to_xdna_obj(gobj); | |
| 718 | + abo = to_xdna_obj(gobj); | |
| 719 | + abo->private_buffer = true; | |
| 720 | + | |
| 721 | + return abo; | |
| 715 | 722 | } |
| 716 | 723 | |
| 717 | 724 | struct drm_gem_object * |
| @@ -54,6 +54,8 @@ struct amdxdna_gem_obj { · 53 unmodified lines | ||
| 54 | 54 | |
| 55 | 55 | /* True, if BO is managed by XRT, not application */ |
| 56 | 56 | bool internal; |
| 57 | + /* True, if BO is not exportable */ | |
| 58 | + bool private_buffer; | |
| 57 | 59 | }; |
| 58 | 60 | |
| 59 | 61 | #define to_gobj(obj) (&(obj)->base.base) |
| @@ -69,60 +69,10 @@ static void amdxdna_ubuf_release(struct dma_buf *dbuf) · 68 unmodified lines | ||
| 69 | 69 | kfree(ubuf); |
| 70 | 70 | } |
| 71 | 71 | |
| 72 | −static vm_fault_t amdxdna_ubuf_vm_fault(struct vm_fault *vmf) | |
| 73 | −{ | |
| 74 | − struct vm_area_struct *vma = vmf->vma; | |
| 75 | − struct amdxdna_ubuf_priv *ubuf; | |
| 76 | − unsigned long pfn; | |
| 77 | − pgoff_t pgoff; | |
| 78 | − | |
| 79 | − ubuf = vma->vm_private_data; | |
| 80 | − pgoff = (vmf->address - vma->vm_start) >> PAGE_SHIFT; | |
| 81 | − | |
| 82 | − pfn = page_to_pfn(ubuf->pages[pgoff]); | |
| 83 | − return vmf_insert_pfn(vma, vmf->address, pfn); | |
| 84 | −} | |
| 85 | − | |
| 86 | −static const struct vm_operations_struct amdxdna_ubuf_vm_ops = { | |
| 87 | − .fault = amdxdna_ubuf_vm_fault, | |
| 88 | −}; | |
| 89 | − | |
| 90 | −static int amdxdna_ubuf_mmap(struct dma_buf *dbuf, struct vm_area_struct *vma) | |
| 91 | −{ | |
| 92 | − struct amdxdna_ubuf_priv *ubuf = dbuf->priv; | |
| 93 | − | |
| 94 | − vma->vm_ops = &amdxdna_ubuf_vm_ops; | |
| 95 | − vma->vm_private_data = ubuf; | |
| 96 | − vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); | |
| 97 | − | |
| 98 | − return 0; | |
| 99 | −} | |
| 100 | − | |
| 101 | −static int amdxdna_ubuf_vmap(struct dma_buf *dbuf, struct iosys_map *map) | |
| 102 | −{ | |
| 103 | − struct amdxdna_ubuf_priv *ubuf = dbuf->priv; | |
| 104 | − void *kva; | |
| 105 | − | |
| 106 | − kva = vmap(ubuf->pages, ubuf->nr_pages, VM_MAP, PAGE_KERNEL); | |
| 107 | − if (!kva) | |
| 108 | − return -EINVAL; | |
| 109 | − | |
| 110 | − iosys_map_set_vaddr(map, kva); | |
| 111 | − return 0; | |
| 112 | −} | |
| 113 | − | |
| 114 | −static void amdxdna_ubuf_vunmap(struct dma_buf *dbuf, struct iosys_map *map) | |
| 115 | −{ | |
| 116 | − vunmap(map->vaddr); | |
| 117 | −} | |
| 118 | − | |
| 119 | 72 | static const struct dma_buf_ops amdxdna_ubuf_dmabuf_ops = { |
| 120 | 73 | .map_dma_buf = amdxdna_ubuf_map, |
| 121 | 74 | .unmap_dma_buf = amdxdna_ubuf_unmap, |
| 122 | 75 | .release = amdxdna_ubuf_release, |
| 123 | − .mmap = amdxdna_ubuf_mmap, | |
| 124 | − .vmap = amdxdna_ubuf_vmap, | |
| 125 | − .vunmap = amdxdna_ubuf_vunmap, | |
| 126 | 76 | }; |
| 127 | 77 | |
| 128 | 78 | struct dma_buf *amdxdna_get_ubuf(struct drm_device *dev, |
| @@ -450,7 +450,7 @@ priority_bands_fops_write(struct file *file, const char __user *user_buf, size_t · 449 unmodified lines | ||
| 450 | 450 | u32 band; |
| 451 | 451 | int ret; |
| 452 | 452 | |
| 453 | − if (size >= sizeof(buf)) | |
| 453 | + if (*pos != 0 || size >= sizeof(buf)) | |
| 454 | 454 | return -EINVAL; |
| 455 | 455 | |
| 456 | 456 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, pos, user_buf, size); |
| @@ -537,6 +537,26 @@ static const struct file_operations ivpu_fops = { · 536 unmodified lines | ||
| 537 | 537 | #endif |
| 538 | 538 | }; |
| 539 | 539 | |
| 540 | +static int ivpu_gem_prime_handle_to_fd(struct drm_device *dev, struct drm_file *file_priv, | |
| 541 | + u32 handle, u32 flags, int *prime_fd) | |
| 542 | +{ | |
| 543 | + struct drm_gem_object *obj; | |
| 544 | + | |
| 545 | + obj = drm_gem_object_lookup(file_priv, handle); | |
| 546 | + if (!obj) | |
| 547 | + return -ENOENT; | |
| 548 | + | |
| 549 | + if (drm_gem_is_imported(obj)) { | |
| 550 | + /* Do not allow re-exporting */ | |
| 551 | + drm_gem_object_put(obj); | |
| 552 | + return -EOPNOTSUPP; | |
| 553 | + } | |
| 554 | + | |
| 555 | + drm_gem_object_put(obj); | |
| 556 | + | |
| 557 | + return drm_gem_prime_handle_to_fd(dev, file_priv, handle, flags, prime_fd); | |
| 558 | +} | |
| 559 | + | |
| 540 | 560 | static const struct drm_driver driver = { |
| 541 | 561 | .driver_features = DRIVER_GEM | DRIVER_COMPUTE_ACCEL, |
| 542 | 562 | |
| @@ -545,6 +565,7 @@ static const struct drm_driver driver = { · 2 unmodified lines | ||
| 545 | 565 | |
| 546 | 566 | .gem_create_object = ivpu_gem_create_object, |
| 547 | 567 | .gem_prime_import = ivpu_gem_prime_import, |
| 568 | + .prime_handle_to_fd = ivpu_gem_prime_handle_to_fd, | |
| 548 | 569 | |
| 549 | 570 | .ioctls = ivpu_drm_ioctls, |
| 550 | 571 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), |
| @@ -606,8 +606,11 @@ static const struct vm_operations_struct drm_vm_ops = { · 605 unmodified lines | ||
| 606 | 606 | static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) |
| 607 | 607 | { |
| 608 | 608 | struct qaic_bo *bo = to_qaic_bo(obj); |
| 609 | + unsigned long remap_start; | |
| 609 | 610 | unsigned long offset = 0; |
| 611 | + unsigned long remap_end; | |
| 610 | 612 | struct scatterlist *sg; |
| 613 | + unsigned long length; | |
| 611 | 614 | int ret = 0; |
| 612 | 615 | |
| 613 | 616 | if (drm_gem_is_imported(obj)) |
| @@ -615,11 +618,27 @@ static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struc · 1 unmodified line | ||
| 615 | 618 | |
| 616 | 619 | for (sg = bo->sgt->sgl; sg; sg = sg_next(sg)) { |
| 617 | 620 | if (sg_page(sg)) { |
| 621 | + /* if sg is too large for the VMA, so truncate it to fit */ | |
| 622 | + if (check_add_overflow(vma->vm_start, offset, &remap_start)) | |
| 623 | + return -EINVAL; | |
| 624 | + if (check_add_overflow(remap_start, sg->length, &remap_end)) | |
| 625 | + return -EINVAL; | |
| 626 | + | |
| 627 | + if (remap_end > vma->vm_end) { | |
| 628 | + if (check_sub_overflow(vma->vm_end, remap_start, &length)) | |
| 629 | + return -EINVAL; | |
| 630 | + } else { | |
| 631 | + length = sg->length; | |
| 632 | + } | |
| 633 | + | |
| 634 | + if (length == 0) | |
| 635 | + goto out; | |
| 636 | + | |
| 618 | 637 | ret = remap_pfn_range(vma, vma->vm_start + offset, page_to_pfn(sg_page(sg)), |
| 619 | − sg->length, vma->vm_page_prot); | |
| 638 | + length, vma->vm_page_prot); | |
| 620 | 639 | if (ret) |
| 621 | 640 | goto out; |
| 622 | − offset += sg->length; | |
| 641 | + offset += length; | |
| 623 | 642 | } |
| 624 | 643 | } |
| 625 | 644 | |
| @@ -497,11 +497,11 @@ static void decode_ras_msg(struct qaic_device *qdev, struct ras_data *msg) · 496 unmodified lines | ||
| 497 | 497 | qdev->ce_count++; |
| 498 | 498 | break; |
| 499 | 499 | case UE: |
| 500 | − if (qdev->ce_count != UINT_MAX) | |
| 500 | + if (qdev->ue_count != UINT_MAX) | |
| 501 | 501 | qdev->ue_count++; |
| 502 | 502 | break; |
| 503 | 503 | case UE_NF: |
| 504 | − if (qdev->ce_count != UINT_MAX) | |
| 504 | + if (qdev->ue_nf_count != UINT_MAX) | |
| 505 | 505 | qdev->ue_nf_count++; |
| 506 | 506 | break; |
| 507 | 507 | default: |
| @@ -79,11 +79,6 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 78 unmodified lines | ||
| 79 | 79 | rkt_obj->size = args->size; |
| 80 | 80 | rkt_obj->offset = 0; |
| 81 | 81 | |
| 82 | − ret = drm_gem_handle_create(file, gem_obj, &args->handle); | |
| 83 | − drm_gem_object_put(gem_obj); | |
| 84 | − if (ret) | |
| 85 | − goto err; | |
| 86 | − | |
| 87 | 82 | sgt = drm_gem_shmem_get_pages_sgt(shmem_obj); |
| 88 | 83 | if (IS_ERR(sgt)) { |
| 89 | 84 | ret = PTR_ERR(sgt); |
| @@ -95,6 +90,8 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 5 unmodified lines | ||
| 95 | 90 | rkt_obj->size, PAGE_SIZE, |
| 96 | 91 | 0, 0); |
| 97 | 92 | mutex_unlock(&rocket_priv->mm_lock); |
| 93 | + if (ret) | |
| 94 | + goto err; | |
| 98 | 95 | |
| 99 | 96 | ret = iommu_map_sgtable(rocket_priv->domain->domain, |
| 100 | 97 | rkt_obj->mm.start, |
| @@ -112,8 +109,18 @@ int rocket_ioctl_create_bo(struct drm_device *dev, void *data, struct drm_file * · 11 unmodified lines | ||
| 112 | 109 | args->offset = drm_vma_node_offset_addr(&gem_obj->vma_node); |
| 113 | 110 | args->dma_address = rkt_obj->mm.start; |
| 114 | 111 | |
| 112 | + ret = drm_gem_handle_create(file, gem_obj, &args->handle); | |
| 113 | + if (ret) | |
| 114 | + goto err_unmap; | |
| 115 | + | |
| 116 | + drm_gem_object_put(gem_obj); | |
| 117 | + | |
| 115 | 118 | return 0; |
| 116 | 119 | |
| 120 | +err_unmap: | |
| 121 | + iommu_unmap(rocket_priv->domain->domain, | |
| 122 | + rkt_obj->mm.start, rkt_obj->size); | |
| 123 | + | |
| 117 | 124 | err_remove_node: |
| 118 | 125 | mutex_lock(&rocket_priv->mm_lock); |
| 119 | 126 | drm_mm_remove_node(&rkt_obj->mm); |
| @@ -145,6 +152,8 @@ int rocket_ioctl_prep_bo(struct drm_device *dev, void *data, struct drm_file *fi · 25 unmodified lines | ||
| 145 | 152 | ret = dma_resv_wait_timeout(gem_obj->resv, DMA_RESV_USAGE_WRITE, true, timeout); |
| 146 | 153 | if (!ret) |
| 147 | 154 | ret = timeout ? -ETIMEDOUT : -EBUSY; |
| 155 | + else if (ret > 0) | |
| 156 | + ret = 0; | |
| 148 | 157 | |
| 149 | 158 | shmem_obj = &to_rocket_bo(gem_obj)->base; |
| 150 | 159 | |
| @@ -192,11 +192,15 @@ static const struct dmi_system_id ac_dmi_table[] __initconst = { · 191 unmodified lines | ||
| 192 | 192 | |
| 193 | 193 | static int acpi_ac_probe(struct platform_device *pdev) |
| 194 | 194 | { |
| 195 | − struct acpi_device *adev = ACPI_COMPANION(&pdev->dev); | |
| 196 | 195 | struct power_supply_config psy_cfg = {}; |
| 196 | + struct acpi_device *adev; | |
| 197 | 197 | struct acpi_ac *ac; |
| 198 | 198 | int result; |
| 199 | 199 | |
| 200 | + adev = ACPI_COMPANION(&pdev->dev); | |
| 201 | + if (!adev) | |
| 202 | + return -ENODEV; | |
| 203 | + | |
| 200 | 204 | ac = kzalloc_obj(struct acpi_ac); |
| 201 | 205 | if (!ac) |
| 202 | 206 | return -ENOMEM; |
| @@ -423,7 +423,11 @@ static void acpi_pad_notify(acpi_handle handle, u32 event, void *data) · 422 unmodified lines | ||
| 423 | 423 | |
| 424 | 424 | static int acpi_pad_probe(struct platform_device *pdev) |
| 425 | 425 | { |
| 426 | − struct acpi_device *adev = ACPI_COMPANION(&pdev->dev); | |
| 426 | + struct acpi_device *adev; | |
| 427 | + | |
| 428 | + adev = ACPI_COMPANION(&pdev->dev); | |
| 429 | + if (!adev) | |
| 430 | + return -ENODEV; | |
| 427 | 431 | |
| 428 | 432 | return acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, |
| 429 | 433 | acpi_pad_notify, adev); |
| @@ -605,15 +605,12 @@ static umode_t acpi_tad_attr_is_visible(struct kobject *kobj, · 604 unmodified lines | ||
| 605 | 605 | return 0; |
| 606 | 606 | } |
| 607 | 607 | |
| 608 | −static const struct attribute_group acpi_tad_attr_group = { | |
| 608 | +static const struct attribute_group acpi_tad_group = { | |
| 609 | 609 | .attrs = acpi_tad_attrs, |
| 610 | 610 | .is_visible = acpi_tad_attr_is_visible, |
| 611 | 611 | }; |
| 612 | 612 | |
| 613 | −static const struct attribute_group *acpi_tad_attr_groups[] = { | |
| 614 | − &acpi_tad_attr_group, | |
| 615 | − NULL, | |
| 616 | −}; | |
| 613 | +__ATTRIBUTE_GROUPS(acpi_tad); | |
| 617 | 614 | |
| 618 | 615 | #ifdef CONFIG_RTC_CLASS |
| 619 | 616 | /* RTC class device interface */ |
| @@ -683,9 +680,8 @@ static int acpi_tad_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *t) · 63 unmodified lines | ||
| 683 | 680 | |
| 684 | 681 | acpi_tad_rt_to_tm(&rt, &tm_now); |
| 685 | 682 | |
| 686 | − value = ktime_divns(ktime_sub(rtc_tm_to_ktime(t->time), | |
| 687 | − rtc_tm_to_ktime(tm_now)), NSEC_PER_SEC); | |
| 688 | − if (value <= 0 || value > U32_MAX) | |
| 683 | + value = rtc_tm_to_time64(&t->time) - rtc_tm_to_time64(&tm_now); | |
| 684 | + if (value <= 0 || value >= U32_MAX) | |
| 689 | 685 | return -EINVAL; |
| 690 | 686 | } |
| 691 | 687 | |
| @@ -748,8 +744,7 @@ static int acpi_tad_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *t) · 56 unmodified lines | ||
| 748 | 744 | |
| 749 | 745 | if (retval != ACPI_TAD_WAKE_DISABLED) { |
| 750 | 746 | t->enabled = 1; |
| 751 | − t->time = rtc_ktime_to_tm(ktime_add_ns(rtc_tm_to_ktime(tm_now), | |
| 752 | − (u64)retval * NSEC_PER_SEC)); | |
| 747 | + rtc_time64_to_tm(rtc_tm_to_time64(&tm_now) + retval, &t->time); | |
| 753 | 748 | } else { |
| 754 | 749 | t->enabled = 0; |
| 755 | 750 | t->time = tm_now; |
| @@ -795,9 +790,9 @@ static int acpi_tad_disable_timer(struct device *dev, u32 timer_id) · 39 unmodified lines | ||
| 795 | 790 | return acpi_tad_wake_set(dev, "_STV", timer_id, ACPI_TAD_WAKE_DISABLED); |
| 796 | 791 | } |
| 797 | 792 | |
| 798 | −static void acpi_tad_remove(struct platform_device *pdev) | |
| 793 | +static void acpi_tad_remove(void *data) | |
| 799 | 794 | { |
| 800 | − struct device *dev = &pdev->dev; | |
| 795 | + struct device *dev = data; | |
| 801 | 796 | struct acpi_tad_driver_data *dd = dev_get_drvdata(dev); |
| 802 | 797 | |
| 803 | 798 | device_init_wakeup(dev, false); |
| @@ -820,10 +815,15 @@ static void acpi_tad_remove(struct platform_device *pdev) · 16 unmodified lines | ||
| 820 | 815 | static int acpi_tad_probe(struct platform_device *pdev) |
| 821 | 816 | { |
| 822 | 817 | struct device *dev = &pdev->dev; |
| 823 | − acpi_handle handle = ACPI_HANDLE(dev); | |
| 824 | 818 | struct acpi_tad_driver_data *dd; |
| 819 | + acpi_handle handle; | |
| 825 | 820 | acpi_status status; |
| 826 | 821 | unsigned long long caps; |
| 822 | + int ret; | |
| 823 | + | |
| 824 | + handle = ACPI_HANDLE(dev); | |
| 825 | + if (!handle) | |
| 826 | + return -ENODEV; | |
| 827 | 827 | |
| 828 | 828 | /* |
| 829 | 829 | * Initialization failure messages are mostly about firmware issues, so |
| @@ -863,13 +863,21 @@ static int acpi_tad_probe(struct platform_device *pdev) · 33 unmodified lines | ||
| 863 | 863 | } |
| 864 | 864 | |
| 865 | 865 | /* |
| 866 | − * The platform bus type layer tells the ACPI PM domain powers up the | |
| 867 | − * device, so set the runtime PM status of it to "active". | |
| 866 | + * The platform bus type probe callback tells the ACPI PM domain to | |
| 867 | + * power up the device, so set the runtime PM status of it to "active". | |
| 868 | 868 | */ |
| 869 | 869 | pm_runtime_set_active(dev); |
| 870 | 870 | pm_runtime_enable(dev); |
| 871 | 871 | pm_runtime_suspend(dev); |
| 872 | 872 | |
| 873 | + /* | |
| 874 | + * acpi_tad_remove() needs to run after unregistering the RTC class | |
| 875 | + * device to avoid racing with the latter's callbacks. | |
| 876 | + */ | |
| 877 | + ret = devm_add_action_or_reset(&pdev->dev, acpi_tad_remove, &pdev->dev); | |
| 878 | + if (ret) | |
| 879 | + return ret; | |
| 880 | + | |
| 873 | 881 | if (caps & ACPI_TAD_RT) |
| 874 | 882 | acpi_tad_register_rtc(dev, caps); |
| 875 | 883 | |
| @@ -885,10 +893,9 @@ static struct platform_driver acpi_tad_driver = { · 9 unmodified lines | ||
| 885 | 893 | .driver = { |
| 886 | 894 | .name = "acpi-tad", |
| 887 | 895 | .acpi_match_table = acpi_tad_ids, |
| 888 | − .dev_groups = acpi_tad_attr_groups, | |
| 896 | + .dev_groups = acpi_tad_groups, | |
| 889 | 897 | }, |
| 890 | 898 | .probe = acpi_tad_probe, |
| 891 | − .remove = acpi_tad_remove, | |
| 892 | 899 | }; |
| 893 | 900 | MODULE_DEVICE_TABLE(acpi, acpi_tad_ids); |
| 894 | 901 | |
| @@ -78,18 +78,22 @@ ACPI_EXPORT_SYMBOL(acpi_update_all_gpes) · 77 unmodified lines | ||
| 78 | 78 | |
| 79 | 79 | /******************************************************************************* |
| 80 | 80 | * |
| 81 | − * FUNCTION: acpi_enable_gpe | |
| 81 | + * FUNCTION: acpi_enable_gpe_cond | |
| 82 | 82 | * |
| 83 | 83 | * PARAMETERS: gpe_device - Parent GPE Device. NULL for GPE0/GPE1 |
| 84 | 84 | * gpe_number - GPE level within the GPE block |
| 85 | + * dispatch_type - GPE dispatch type to match | |
| 85 | 86 | * |
| 86 | 87 | * RETURN: Status |
| 87 | 88 | * |
| 88 | − * DESCRIPTION: Add a reference to a GPE. On the first reference, the GPE is | |
| 89 | − * hardware-enabled. | |
| 89 | + * DESCRIPTION: Add a reference to a GPE so long as its dispatch type matches | |
| 90 | + * the supplied one, or it is different from ACPI_GPE_DISPATCH_NONE | |
| 91 | + * if the supplied one is ACPI_GPE_DISPATCH_MASK. On the first | |
| 92 | + * reference, the GPE is hardware-enabled. | |
| 90 | 93 | * |
| 91 | 94 | ******************************************************************************/ |
| 92 | −acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) | |
| 95 | +acpi_status acpi_enable_gpe_cond(acpi_handle gpe_device, u32 gpe_number, | |
| 96 | + u8 dispatch_type) | |
| 93 | 97 | { |
| 94 | 98 | acpi_status status = AE_BAD_PARAMETER; |
| 95 | 99 | struct acpi_gpe_event_info *gpe_event_info; |
| @@ -100,14 +104,18 @@ acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) · 4 unmodified lines | ||
| 100 | 104 | flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock); |
| 101 | 105 | |
| 102 | 106 | /* |
| 103 | − * Ensure that we have a valid GPE number and that there is some way | |
| 104 | − * of handling the GPE (handler or a GPE method). In other words, we | |
| 105 | − * won't allow a valid GPE to be enabled if there is no way to handle it. | |
| 107 | + * Ensure that we have a valid GPE number and that the dispatch type of | |
| 108 | + * the GPE matches the supplied one (or it is not ACPI_GPE_DISPATCH_NONE | |
| 109 | + * if the supplied one is ACPI_GPE_DISPATCH_MASK). | |
| 106 | 110 | */ |
| 107 | 111 | gpe_event_info = acpi_ev_get_gpe_event_info(gpe_device, gpe_number); |
| 108 | 112 | if (gpe_event_info) { |
| 109 | − if (ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags) != | |
| 110 | − ACPI_GPE_DISPATCH_NONE) { | |
| 113 | + if (dispatch_type == ACPI_GPE_DISPATCH_MASK) | |
| 114 | + dispatch_type = ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags); | |
| 115 | + else if (dispatch_type != ACPI_GPE_DISPATCH_TYPE(gpe_event_info->flags)) | |
| 116 | + dispatch_type = ACPI_GPE_DISPATCH_NONE; | |
| 117 | + | |
| 118 | + if (dispatch_type != ACPI_GPE_DISPATCH_NONE) { | |
| 111 | 119 | status = acpi_ev_add_gpe_reference(gpe_event_info, TRUE); |
| 112 | 120 | if (ACPI_SUCCESS(status) && |
| 113 | 121 | ACPI_GPE_IS_POLLING_NEEDED(gpe_event_info)) { |
| @@ -128,6 +136,30 @@ acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) · 14 unmodified lines | ||
| 128 | 136 | acpi_os_release_lock(acpi_gbl_gpe_lock, flags); |
| 129 | 137 | return_ACPI_STATUS(status); |
| 130 | 138 | } |
| 139 | +ACPI_EXPORT_SYMBOL(acpi_enable_gpe_cond) | |
| 140 | + | |
| 141 | +/******************************************************************************* | |
| 142 | + * | |
| 143 | + * FUNCTION: acpi_enable_gpe | |
| 144 | + * | |
| 145 | + * PARAMETERS: gpe_device - Parent GPE Device. NULL for GPE0/GPE1 | |
| 146 | + * gpe_number - GPE level within the GPE block | |
| 147 | + * | |
| 148 | + * RETURN: Status | |
| 149 | + * | |
| 150 | + * DESCRIPTION: Add a reference to a GPE. On the first reference, the GPE is | |
| 151 | + * hardware-enabled. | |
| 152 | + * | |
| 153 | + ******************************************************************************/ | |
| 154 | +acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number) | |
| 155 | +{ | |
| 156 | + /* | |
| 157 | + * Ensure that there is some way of handling the GPE (handler or a GPE | |
| 158 | + * method). In other words, we won't allow a valid GPE to be enabled if | |
| 159 | + * there is no way to handle it. | |
| 160 | + */ | |
| 161 | + return acpi_enable_gpe_cond(gpe_device, gpe_number, ACPI_GPE_DISPATCH_MASK); | |
| 162 | +} | |
| 131 | 163 | ACPI_EXPORT_SYMBOL(acpi_enable_gpe) |
| 132 | 164 | |
| 133 | 165 | /******************************************************************************* |
| @@ -401,8 +401,18 @@ static struct acpi_generic_address *einj_get_trigger_parameter_region( · 400 unmodified lines | ||
| 401 | 401 | |
| 402 | 402 | return NULL; |
| 403 | 403 | } |
| 404 | + | |
| 405 | +static bool is_memory_injection(u32 type, u32 flags) | |
| 406 | +{ | |
| 407 | + if (flags & SETWA_FLAGS_EINJV2) | |
| 408 | + return !!(type & ACPI_EINJV2_MEMORY); | |
| 409 | + if (type & ACPI5_VENDOR_BIT) | |
| 410 | + return !!(vendor_flags & SETWA_FLAGS_MEM); | |
| 411 | + return !!(type & MEM_ERROR_MASK) || !!(flags & SETWA_FLAGS_MEM); | |
| 412 | +} | |
| 413 | + | |
| 404 | 414 | /* Execute instructions in trigger error action table */ |
| 405 | −static int __einj_error_trigger(u64 trigger_paddr, u32 type, | |
| 415 | +static int __einj_error_trigger(u64 trigger_paddr, u32 type, u32 flags, | |
| 406 | 416 | u64 param1, u64 param2) |
| 407 | 417 | { |
| 408 | 418 | struct acpi_einj_trigger trigger_tab; |
| @@ -480,7 +490,7 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, · 71 unmodified lines | ||
| 480 | 490 | * This will cause resource conflict with regular memory. So |
| 481 | 491 | * remove it from trigger table resources. |
| 482 | 492 | */ |
| 483 | − if ((param_extension || acpi5) && (type & MEM_ERROR_MASK) && param2) { | |
| 493 | + if ((param_extension || acpi5) && is_memory_injection(type, flags)) { | |
| 484 | 494 | struct apei_resources addr_resources; |
| 485 | 495 | |
| 486 | 496 | apei_resources_init(&addr_resources); |
| @@ -660,7 +670,7 @@ static int __einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2, · 173 unmodified lines | ||
| 660 | 670 | return rc; |
| 661 | 671 | trigger_paddr = apei_exec_ctx_get_output(&ctx); |
| 662 | 672 | if (notrigger == 0) { |
| 663 | − rc = __einj_error_trigger(trigger_paddr, type, param1, param2); | |
| 673 | + rc = __einj_error_trigger(trigger_paddr, type, flags, param1, param2); | |
| 664 | 674 | if (rc) |
| 665 | 675 | return rc; |
| 666 | 676 | } |
| @@ -718,35 +728,30 @@ int einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2, u64 param3, · 51 unmodified lines | ||
| 718 | 728 | SETWA_FLAGS_PCIE_SBDF | SETWA_FLAGS_EINJV2))) |
| 719 | 729 | return -EINVAL; |
| 720 | 730 | |
| 731 | + /* | |
| 732 | + * Injections targeting a CXL 1.0/1.1 port have to be injected | |
| 733 | + * via the einj_cxl_rch_error_inject() path as that does the proper | |
| 734 | + * validation of the given RCRB base (MMIO) address. | |
| 735 | + */ | |
| 736 | + if (einj_is_cxl_error_type(type) && (flags & SETWA_FLAGS_MEM)) | |
| 737 | + return -EINVAL; | |
| 738 | + | |
| 721 | 739 | /* check if type is a valid EINJv2 error type */ |
| 722 | 740 | if (is_v2) { |
| 723 | 741 | if (!(type & available_error_type_v2)) |
| 724 | 742 | return -EINVAL; |
| 725 | 743 | } |
| 726 | − /* | |
| 727 | − * We need extra sanity checks for memory errors. | |
| 728 | − * Other types leap directly to injection. | |
| 729 | − */ | |
| 730 | 744 | |
| 731 | 745 | /* ensure param1/param2 existed */ |
| 732 | 746 | if (!(param_extension || acpi5)) |
| 733 | 747 | goto inject; |
| 734 | 748 | |
| 735 | − /* ensure injection is memory related */ | |
| 736 | − if (type & ACPI5_VENDOR_BIT) { | |
| 737 | − if (vendor_flags != SETWA_FLAGS_MEM) | |
| 738 | − goto inject; | |
| 739 | − } else if (!(type & MEM_ERROR_MASK) && !(flags & SETWA_FLAGS_MEM)) { | |
| 740 | − goto inject; | |
| 741 | − } | |
| 742 | − | |
| 743 | 749 | /* |
| 744 | − * Injections targeting a CXL 1.0/1.1 port have to be injected | |
| 745 | − * via the einj_cxl_rch_error_inject() path as that does the proper | |
| 746 | − * validation of the given RCRB base (MMIO) address. | |
| 750 | + * We need extra sanity checks for memory errors. | |
| 751 | + * Other types leap directly to injection. | |
| 747 | 752 | */ |
| 748 | − if (einj_is_cxl_error_type(type) && (flags & SETWA_FLAGS_MEM)) | |
| 749 | − return -EINVAL; | |
| 753 | + if (!is_memory_injection(type, flags)) | |
| 754 | + goto inject; | |
| 750 | 755 | |
| 751 | 756 | /* |
| 752 | 757 | * Disallow crazy address masks that give BIOS leeway to pick |
| @@ -16,7 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | |
| 17 | 17 | static int psci_acpi_cpu_init_idle(unsigned int cpu) |
| 18 | 18 | { |
| 19 | − int i, count; | |
| 19 | + int i; | |
| 20 | 20 | struct acpi_lpi_state *lpi; |
| 21 | 21 | struct acpi_processor *pr = per_cpu(processors, cpu); |
| 22 | 22 | |
| @@ -30,14 +30,10 @@ static int psci_acpi_cpu_init_idle(unsigned int cpu) · 7 unmodified lines | ||
| 30 | 30 | if (!psci_ops.cpu_suspend) |
| 31 | 31 | return -EOPNOTSUPP; |
| 32 | 32 | |
| 33 | − count = pr->power.count - 1; | |
| 34 | − if (count <= 0) | |
| 35 | − return -ENODEV; | |
| 36 | − | |
| 37 | − for (i = 0; i < count; i++) { | |
| 33 | + for (i = 1; i < pr->power.count; i++) { | |
| 38 | 34 | u32 state; |
| 39 | 35 | |
| 40 | − lpi = &pr->power.lpi_states[i + 1]; | |
| 36 | + lpi = &pr->power.lpi_states[i]; | |
| 41 | 37 | /* |
| 42 | 38 | * Only bits[31:0] represent a PSCI power_state while |
| 43 | 39 | * bits[63:32] must be 0x0 as per ARM ACPI FFH Specification |
| @@ -94,6 +94,7 @@ struct acpi_battery { · 93 unmodified lines | ||
| 94 | 94 | struct power_supply *bat; |
| 95 | 95 | struct power_supply_desc bat_desc; |
| 96 | 96 | struct acpi_device *device; |
| 97 | + struct device *phys_dev; | |
| 97 | 98 | struct notifier_block pm_nb; |
| 98 | 99 | struct list_head list; |
| 99 | 100 | unsigned long update_time; |
| @@ -1033,7 +1034,7 @@ static int acpi_battery_update(struct acpi_battery *battery, bool resume) · 933 unmodified lines | ||
| 1033 | 1034 | if ((battery->state & ACPI_BATTERY_STATE_CRITICAL) || |
| 1034 | 1035 | (test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags) && |
| 1035 | 1036 | (battery->capacity_now <= battery->alarm))) |
| 1036 | − acpi_pm_wakeup_event(&battery->device->dev); | |
| 1037 | + acpi_pm_wakeup_event(battery->phys_dev); | |
| 1037 | 1038 | |
| 1038 | 1039 | return result; |
| 1039 | 1040 | } |
| @@ -1214,10 +1215,14 @@ static void sysfs_battery_cleanup(struct acpi_battery *battery) · 174 unmodified lines | ||
| 1214 | 1215 | |
| 1215 | 1216 | static int acpi_battery_probe(struct platform_device *pdev) |
| 1216 | 1217 | { |
| 1217 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 1218 | 1218 | struct acpi_battery *battery; |
| 1219 | + struct acpi_device *device; | |
| 1219 | 1220 | int result; |
| 1220 | 1221 | |
| 1222 | + device = ACPI_COMPANION(&pdev->dev); | |
| 1223 | + if (!device) | |
| 1224 | + return -ENODEV; | |
| 1225 | + | |
| 1221 | 1226 | if (device->dep_unmet) |
| 1222 | 1227 | return -EPROBE_DEFER; |
| 1223 | 1228 | |
| @@ -1227,6 +1232,7 @@ static int acpi_battery_probe(struct platform_device *pdev) · 3 unmodified lines | ||
| 1227 | 1232 | |
| 1228 | 1233 | platform_set_drvdata(pdev, battery); |
| 1229 | 1234 | |
| 1235 | + battery->phys_dev = &pdev->dev; | |
| 1230 | 1236 | battery->device = device; |
| 1231 | 1237 | |
| 1232 | 1238 | result = devm_mutex_init(&pdev->dev, &battery->update_lock); |
| @@ -179,6 +179,7 @@ struct acpi_button { · 178 unmodified lines | ||
| 179 | 179 | ktime_t last_time; |
| 180 | 180 | bool suspended; |
| 181 | 181 | bool lid_state_initialized; |
| 182 | + bool gpe_enabled; | |
| 182 | 183 | }; |
| 183 | 184 | |
| 184 | 185 | static struct acpi_device *lid_device; |
| @@ -531,15 +532,20 @@ static int acpi_lid_input_open(struct input_dev *input) · 346 unmodified lines | ||
| 531 | 532 | |
| 532 | 533 | static int acpi_button_probe(struct platform_device *pdev) |
| 533 | 534 | { |
| 534 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 535 | 535 | acpi_notify_handler handler; |
| 536 | + struct acpi_device *device; | |
| 536 | 537 | struct acpi_button *button; |
| 537 | 538 | struct input_dev *input; |
| 538 | − const char *hid = acpi_device_hid(device); | |
| 539 | 539 | acpi_status status; |
| 540 | 540 | char *name, *class; |
| 541 | + const char *hid; | |
| 541 | 542 | int error = 0; |
| 542 | 543 | |
| 544 | + device = ACPI_COMPANION(&pdev->dev); | |
| 545 | + if (!device) | |
| 546 | + return -ENODEV; | |
| 547 | + | |
| 548 | + hid = acpi_device_hid(device); | |
| 543 | 549 | if (!strcmp(hid, ACPI_BUTTON_HID_LID) && |
| 544 | 550 | lid_init_state == ACPI_BUTTON_LID_INIT_DISABLED) |
| 545 | 551 | return -ENODEV; |
| @@ -641,6 +647,21 @@ static int acpi_button_probe(struct platform_device *pdev) · 95 unmodified lines | ||
| 641 | 647 | status = acpi_install_notify_handler(device->handle, |
| 642 | 648 | ACPI_ALL_NOTIFY, handler, |
| 643 | 649 | button); |
| 650 | + if (ACPI_SUCCESS(status) && device->wakeup.flags.valid) { | |
| 651 | + acpi_status st; | |
| 652 | + | |
| 653 | + /* | |
| 654 | + * If the wakeup GPE has a handler method, enable it in | |
| 655 | + * case it is also used for signaling runtime events. | |
| 656 | + */ | |
| 657 | + st = acpi_enable_gpe_cond(device->wakeup.gpe_device, | |
| 658 | + device->wakeup.gpe_number, | |
| 659 | + ACPI_GPE_DISPATCH_METHOD); | |
| 660 | + button->gpe_enabled = ACPI_SUCCESS(st); | |
| 661 | + if (button->gpe_enabled) | |
| 662 | + dev_dbg(button->dev, "Enabled ACPI GPE%02llx\n", | |
| 663 | + device->wakeup.gpe_number); | |
| 664 | + } | |
| 644 | 665 | break; |
| 645 | 666 | } |
| 646 | 667 | if (ACPI_FAILURE(status)) { |
| @@ -666,6 +687,7 @@ err_remove_fs: · 19 unmodified lines | ||
| 666 | 687 | acpi_button_remove_fs(button); |
| 667 | 688 | err_free_button: |
| 668 | 689 | kfree(button); |
| 690 | + memset(acpi_device_class(device), 0, sizeof(acpi_device_class)); | |
| 669 | 691 | return error; |
| 670 | 692 | } |
| 671 | 693 | |
| @@ -684,7 +706,13 @@ static void acpi_button_remove(struct platform_device *pdev) · 12 unmodified lines | ||
| 684 | 706 | acpi_button_event); |
| 685 | 707 | break; |
| 686 | 708 | default: |
| 687 | − acpi_remove_notify_handler(adev->handle, ACPI_DEVICE_NOTIFY, | |
| 709 | + if (button->gpe_enabled) { | |
| 710 | + dev_dbg(button->dev, "Disabling ACPI GPE%02llx\n", | |
| 711 | + adev->wakeup.gpe_number); | |
| 712 | + acpi_disable_gpe(adev->wakeup.gpe_device, | |
| 713 | + adev->wakeup.gpe_number); | |
| 714 | + } | |
| 715 | + acpi_remove_notify_handler(adev->handle, ACPI_ALL_NOTIFY, | |
| 688 | 716 | button->type == ACPI_BUTTON_TYPE_LID ? |
| 689 | 717 | acpi_lid_notify : |
| 690 | 718 | acpi_button_notify); |
| @@ -362,7 +362,7 @@ static int send_pcc_cmd(int pcc_ss_id, u16 cmd) · 361 unmodified lines | ||
| 362 | 362 | end: |
| 363 | 363 | if (cmd == CMD_WRITE) { |
| 364 | 364 | if (unlikely(ret)) { |
| 365 | − for_each_online_cpu(i) { | |
| 365 | + for_each_possible_cpu(i) { | |
| 366 | 366 | struct cpc_desc *desc = per_cpu(cpc_desc_ptr, i); |
| 367 | 367 | |
| 368 | 368 | if (!desc) |
| @@ -524,13 +524,13 @@ int acpi_get_psd_map(unsigned int cpu, struct cppc_cpudata *cpu_data) · 155 unmodified lines | ||
| 524 | 524 | else if (pdomain->coord_type == DOMAIN_COORD_TYPE_SW_ANY) |
| 525 | 525 | cpu_data->shared_type = CPUFREQ_SHARED_TYPE_ANY; |
| 526 | 526 | |
| 527 | − for_each_online_cpu(i) { | |
| 527 | + for_each_possible_cpu(i) { | |
| 528 | 528 | if (i == cpu) |
| 529 | 529 | continue; |
| 530 | 530 | |
| 531 | 531 | match_cpc_ptr = per_cpu(cpc_desc_ptr, i); |
| 532 | 532 | if (!match_cpc_ptr) |
| 533 | − goto err_fault; | |
| 533 | + continue; | |
| 534 | 534 | |
| 535 | 535 | match_pdomain = &(match_cpc_ptr->domain_info); |
| 536 | 536 | if (match_pdomain->domain != pdomain->domain) |
| @@ -1676,10 +1676,14 @@ static int acpi_ec_setup(struct acpi_ec *ec, struct acpi_device *device, bool ca · 1675 unmodified lines | ||
| 1676 | 1676 | |
| 1677 | 1677 | static int acpi_ec_probe(struct platform_device *pdev) |
| 1678 | 1678 | { |
| 1679 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 1679 | + struct acpi_device *device; | |
| 1680 | 1680 | struct acpi_ec *ec; |
| 1681 | 1681 | int ret; |
| 1682 | 1682 | |
| 1683 | + device = ACPI_COMPANION(&pdev->dev); | |
| 1684 | + if (!device) | |
| 1685 | + return -ENODEV; | |
| 1686 | + | |
| 1683 | 1687 | if (boot_ec && (boot_ec->handle == device->handle || |
| 1684 | 1688 | !strcmp(acpi_device_hid(device), ACPI_ECDT_HID))) { |
| 1685 | 1689 | /* Fast path: this device corresponds to the boot EC. */ |
| @@ -50,9 +50,13 @@ static void acpi_hed_notify(acpi_handle handle, u32 event, void *data) · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | static int acpi_hed_probe(struct platform_device *pdev) |
| 52 | 52 | { |
| 53 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 53 | + struct acpi_device *device; | |
| 54 | 54 | int err; |
| 55 | 55 | |
| 56 | + device = ACPI_COMPANION(&pdev->dev); | |
| 57 | + if (!device) | |
| 58 | + return -ENODEV; | |
| 59 | + | |
| 56 | 60 | /* Only one hardware error device */ |
| 57 | 61 | if (hed_handle) |
| 58 | 62 | return -EINVAL; |
| @@ -3341,12 +3341,16 @@ static int acpi_nfit_probe(struct platform_device *pdev) · 3340 unmodified lines | ||
| 3341 | 3341 | struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL }; |
| 3342 | 3342 | struct acpi_nfit_desc *acpi_desc; |
| 3343 | 3343 | struct device *dev = &pdev->dev; |
| 3344 | − struct acpi_device *adev = ACPI_COMPANION(dev); | |
| 3345 | 3344 | struct acpi_table_header *tbl; |
| 3345 | + struct acpi_device *adev; | |
| 3346 | 3346 | acpi_status status = AE_OK; |
| 3347 | 3347 | acpi_size sz; |
| 3348 | 3348 | int rc = 0; |
| 3349 | 3349 | |
| 3350 | + adev = ACPI_COMPANION(&pdev->dev); | |
| 3351 | + if (!adev) | |
| 3352 | + return -ENODEV; | |
| 3353 | + | |
| 3350 | 3354 | rc = acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY, |
| 3351 | 3355 | acpi_nfit_notify, dev); |
| 3352 | 3356 | if (rc) |
| @@ -360,10 +360,14 @@ static void pfrt_log_put_idx(void *data) · 359 unmodified lines | ||
| 360 | 360 | |
| 361 | 361 | static int acpi_pfrt_log_probe(struct platform_device *pdev) |
| 362 | 362 | { |
| 363 | − acpi_handle handle = ACPI_HANDLE(&pdev->dev); | |
| 364 | 363 | struct pfrt_log_device *pfrt_log_dev; |
| 364 | + acpi_handle handle; | |
| 365 | 365 | int ret; |
| 366 | 366 | |
| 367 | + handle = ACPI_HANDLE(&pdev->dev); | |
| 368 | + if (!handle) | |
| 369 | + return -ENODEV; | |
| 370 | + | |
| 367 | 371 | if (!acpi_has_method(handle, "_DSM")) { |
| 368 | 372 | dev_dbg(&pdev->dev, "Missing _DSM\n"); |
| 369 | 373 | return -ENODEV; |
| @@ -538,10 +538,14 @@ static void pfru_put_idx(void *data) · 537 unmodified lines | ||
| 538 | 538 | |
| 539 | 539 | static int acpi_pfru_probe(struct platform_device *pdev) |
| 540 | 540 | { |
| 541 | − acpi_handle handle = ACPI_HANDLE(&pdev->dev); | |
| 542 | 541 | struct pfru_device *pfru_dev; |
| 542 | + acpi_handle handle; | |
| 543 | 543 | int ret; |
| 544 | 544 | |
| 545 | + handle = ACPI_HANDLE(&pdev->dev); | |
| 546 | + if (!handle) | |
| 547 | + return -ENODEV; | |
| 548 | + | |
| 545 | 549 | if (!acpi_has_method(handle, "_DSM")) { |
| 546 | 550 | dev_dbg(&pdev->dev, "Missing _DSM\n"); |
| 547 | 551 | return -ENODEV; |
| @@ -629,11 +629,15 @@ static void acpi_sbs_callback(void *context) · 628 unmodified lines | ||
| 629 | 629 | |
| 630 | 630 | static int acpi_sbs_probe(struct platform_device *pdev) |
| 631 | 631 | { |
| 632 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 632 | + struct acpi_device *device; | |
| 633 | 633 | struct acpi_sbs *sbs; |
| 634 | 634 | int result = 0; |
| 635 | 635 | int id; |
| 636 | 636 | |
| 637 | + device = ACPI_COMPANION(&pdev->dev); | |
| 638 | + if (!device) | |
| 639 | + return -ENODEV; | |
| 640 | + | |
| 637 | 641 | sbs = kzalloc_obj(struct acpi_sbs); |
| 638 | 642 | if (!sbs) { |
| 639 | 643 | result = -ENOMEM; |
| @@ -237,11 +237,15 @@ static int smbus_alarm(void *context) · 236 unmodified lines | ||
| 237 | 237 | |
| 238 | 238 | static int acpi_smbus_hc_probe(struct platform_device *pdev) |
| 239 | 239 | { |
| 240 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 240 | + struct acpi_device *device; | |
| 241 | 241 | int status; |
| 242 | 242 | unsigned long long val; |
| 243 | 243 | struct acpi_smb_hc *hc; |
| 244 | 244 | |
| 245 | + device = ACPI_COMPANION(&pdev->dev); | |
| 246 | + if (!device) | |
| 247 | + return -ENODEV; | |
| 248 | + | |
| 245 | 249 | status = acpi_evaluate_integer(device->handle, "_EC", NULL, &val); |
| 246 | 250 | if (ACPI_FAILURE(status)) { |
| 247 | 251 | pr_err("error obtaining _EC.\n"); |
| @@ -789,7 +789,7 @@ static int acpi_thermal_probe(struct platform_device *pdev) · 788 unmodified lines | ||
| 789 | 789 | int i; |
| 790 | 790 | |
| 791 | 791 | if (!device) |
| 792 | − return -EINVAL; | |
| 792 | + return -ENODEV; | |
| 793 | 793 | |
| 794 | 794 | tz = kzalloc_obj(struct acpi_thermal); |
| 795 | 795 | if (!tz) |
| @@ -38,9 +38,13 @@ static u32 acpi_tiny_power_button_event(void *not_used) · 37 unmodified lines | ||
| 38 | 38 | |
| 39 | 39 | static int acpi_tiny_power_button_probe(struct platform_device *pdev) |
| 40 | 40 | { |
| 41 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 41 | + struct acpi_device *device; | |
| 42 | 42 | acpi_status status; |
| 43 | 43 | |
| 44 | + device = ACPI_COMPANION(&pdev->dev); | |
| 45 | + if (!device) | |
| 46 | + return -ENODEV; | |
| 47 | + | |
| 44 | 48 | if (device->device_type == ACPI_BUS_TYPE_POWER_BUTTON) { |
| 45 | 49 | status = acpi_install_fixed_event_handler(ACPI_EVENT_POWER_BUTTON, |
| 46 | 50 | acpi_tiny_power_button_event, |
| @@ -916,6 +916,14 @@ static const struct dmi_system_id video_detect_dmi_table[] = { · 915 unmodified lines | ||
| 916 | 916 | DMI_MATCH(DMI_PRODUCT_NAME, "82K8"), |
| 917 | 917 | }, |
| 918 | 918 | }, |
| 919 | + { | |
| 920 | + .callback = video_detect_force_native, | |
| 921 | + /* HP OMEN Gaming Laptop 16-n0xxx */ | |
| 922 | + .matches = { | |
| 923 | + DMI_MATCH(DMI_SYS_VENDOR, "HP"), | |
| 924 | + DMI_MATCH(DMI_PRODUCT_NAME, "OMEN by HP Gaming Laptop 16-n0xxx"), | |
| 925 | + }, | |
| 926 | + }, | |
| 919 | 927 | |
| 920 | 928 | /* |
| 921 | 929 | * x86 android tablets which directly control the backlight through |
| @@ -157,6 +157,14 @@ impl Allocation { · 156 unmodified lines | ||
| 157 | 157 | self.get_or_init_info().target_node = Some(target_node); |
| 158 | 158 | } |
| 159 | 159 | |
| 160 | + pub(crate) fn take_oneway_node(&mut self) -> Option<DArc<Node>> { | |
| 161 | + if let Some(info) = self.allocation_info.as_mut() { | |
| 162 | + info.oneway_node.take() | |
| 163 | + } else { | |
| 164 | + None | |
| 165 | + } | |
| 166 | + } | |
| 167 | + | |
| 160 | 168 | /// Reserve enough space to push at least `num_fds` fds. |
| 161 | 169 | pub(crate) fn info_add_fd_reserve(&mut self, num_fds: usize) -> Result { |
| 162 | 170 | self.get_or_init_info() |
| @@ -1402,7 +1402,12 @@ impl Process { · 1401 unmodified lines | ||
| 1402 | 1402 | // Clear delivered_deaths list. |
| 1403 | 1403 | // |
| 1404 | 1404 | // Scope ensures that MutexGuard is dropped while executing the body. |
| 1405 | − while let Some(delivered_death) = { self.inner.lock().delivered_deaths.pop_front() } { | |
| 1405 | + while let Some(delivered_death) = { | |
| 1406 | + // Explicitly bind to avoid tail expression lifetime extension of the lockguard | |
| 1407 | + // Can be removed when the kernel moves to edition 2024 | |
| 1408 | + let maybe_death = self.inner.lock().delivered_deaths.pop_front(); | |
| 1409 | + maybe_death | |
| 1410 | + } { | |
| 1406 | 1411 | drop(delivered_death); |
| 1407 | 1412 | } |
| 1408 | 1413 | |
| @@ -204,7 +204,6 @@ impl<T> ArrayRangeAllocator<T> { · 203 unmodified lines | ||
| 204 | 204 | // caller will mark them as unused, which means that they can be freed if the system comes |
| 205 | 205 | // under memory pressure. |
| 206 | 206 | let mut freed_range = FreedRange::interior_pages(offset, size); |
| 207 | − #[expect(clippy::collapsible_if)] // reads better like this | |
| 208 | 207 | if offset % PAGE_SIZE != 0 { |
| 209 | 208 | if i == 0 || self.ranges[i - 1].endpoint() <= (offset & PAGE_MASK) { |
| 210 | 209 | freed_range.start_page_idx -= 1; |
| @@ -270,7 +270,8 @@ impl Transaction { · 269 unmodified lines | ||
| 270 | 270 | /// Not used for replies. |
| 271 | 271 | pub(crate) fn submit(self: DLArc<Self>, info: &mut TransactionInfo) -> BinderResult { |
| 272 | 272 | // Defined before `process_inner` so that the destructor runs after releasing the lock. |
| 273 | − let mut _t_outdated; | |
| 273 | + let _t_outdated; | |
| 274 | + let _oneway_node; | |
| 274 | 275 | |
| 275 | 276 | let oneway = self.flags & TF_ONE_WAY != 0; |
| 276 | 277 | let process = self.to.clone(); |
| @@ -287,6 +288,14 @@ impl Transaction { · 10 unmodified lines | ||
| 287 | 288 | if let Some(t_outdated) = |
| 288 | 289 | target_node.take_outdated_transaction(&self, &mut process_inner) |
| 289 | 290 | { |
| 291 | + let mut alloc_guard = t_outdated.allocation.lock(); | |
| 292 | + if let Some(alloc) = (*alloc_guard).as_mut() { | |
| 293 | + // Take the oneway node to prevent `Allocation::drop` from calling | |
| 294 | + // `pending_oneway_finished()`, which would be incorrect as this | |
| 295 | + // transaction is not being submitted. | |
| 296 | + _oneway_node = alloc.take_oneway_node(); | |
| 297 | + } | |
| 298 | + drop(alloc_guard); | |
| 290 | 299 | // Save the transaction to be dropped after locks are released. |
| 291 | 300 | _t_outdated = t_outdated; |
| 292 | 301 | } |
| @@ -5584,6 +5584,7 @@ void ata_link_init(struct ata_port *ap, struct ata_link *link, int pmp) · 5583 unmodified lines | ||
| 5584 | 5584 | link->pmp = pmp; |
| 5585 | 5585 | link->active_tag = ATA_TAG_POISON; |
| 5586 | 5586 | link->hw_sata_spd_limit = UINT_MAX; |
| 5587 | + INIT_WORK(&link->deferred_qc_work, ata_scsi_deferred_qc_work); | |
| 5587 | 5588 | |
| 5588 | 5589 | /* can't use iterator, ap isn't initialized yet */ |
| 5589 | 5590 | for (i = 0; i < ATA_MAX_DEVICES; i++) { |
| @@ -5666,7 +5667,6 @@ struct ata_port *ata_port_alloc(struct ata_host *host) · 76 unmodified lines | ||
| 5666 | 5667 | mutex_init(&ap->scsi_scan_mutex); |
| 5667 | 5668 | INIT_DELAYED_WORK(&ap->hotplug_task, ata_scsi_hotplug); |
| 5668 | 5669 | INIT_DELAYED_WORK(&ap->scsi_rescan_task, ata_scsi_dev_rescan); |
| 5669 | − INIT_WORK(&ap->deferred_qc_work, ata_scsi_deferred_qc_work); | |
| 5670 | 5670 | INIT_LIST_HEAD(&ap->eh_done_q); |
| 5671 | 5671 | init_waitqueue_head(&ap->eh_wait_q); |
| 5672 | 5672 | init_completion(&ap->park_req_pending); |
| @@ -6291,12 +6291,15 @@ static void ata_port_detach(struct ata_port *ap) · 618 unmodified lines | ||
| 6291 | 6291 | |
| 6292 | 6292 | /* It better be dead now and not have any remaining deferred qc. */ |
| 6293 | 6293 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); |
| 6294 | − WARN_ON(ap->deferred_qc); | |
| 6295 | 6294 | |
| 6296 | − cancel_work_sync(&ap->deferred_qc_work); | |
| 6297 | 6295 | cancel_delayed_work_sync(&ap->hotplug_task); |
| 6298 | 6296 | cancel_delayed_work_sync(&ap->scsi_rescan_task); |
| 6299 | 6297 | |
| 6298 | + ata_for_each_link(link, ap, PMP_FIRST) { | |
| 6299 | + WARN_ON(link->deferred_qc); | |
| 6300 | + cancel_work_sync(&link->deferred_qc_work); | |
| 6301 | + } | |
| 6302 | + | |
| 6300 | 6303 | /* Delete port multiplier link transport devices */ |
| 6301 | 6304 | if (ap->pmp_link) { |
| 6302 | 6305 | int i; |
| @@ -651,11 +651,11 @@ int ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 650 unmodified lines | ||
| 651 | 651 | if (qc->scsicmd != scmd) |
| 652 | 652 | continue; |
| 653 | 653 | if ((qc->flags & ATA_QCFLAG_ACTIVE) || |
| 654 | − qc == ap->deferred_qc) | |
| 654 | + qc == qc->dev->link->deferred_qc) | |
| 655 | 655 | break; |
| 656 | 656 | } |
| 657 | 657 | |
| 658 | − if (i < ATA_MAX_QUEUE && qc == ap->deferred_qc) { | |
| 658 | + if (i < ATA_MAX_QUEUE && qc == qc->dev->link->deferred_qc) { | |
| 659 | 659 | /* |
| 660 | 660 | * This is a deferred command that timed out while |
| 661 | 661 | * waiting for the command queue to drain. Since the qc |
| @@ -666,8 +666,8 @@ int ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 4 unmodified lines | ||
| 666 | 666 | * deferred qc work from issuing this qc. |
| 667 | 667 | */ |
| 668 | 668 | WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); |
| 669 | − ap->deferred_qc = NULL; | |
| 670 | − cancel_work(&ap->deferred_qc_work); | |
| 669 | + qc->dev->link->deferred_qc = NULL; | |
| 670 | + cancel_work(&qc->dev->link->deferred_qc_work); | |
| 671 | 671 | set_host_byte(scmd, DID_TIME_OUT); |
| 672 | 672 | scsi_eh_finish_cmd(scmd, &ap->eh_done_q); |
| 673 | 673 | } else if (i < ATA_MAX_QUEUE) { |
| @@ -110,13 +110,24 @@ int sata_pmp_qc_defer_cmd_switch(struct ata_queued_cmd *qc) · 109 unmodified lines | ||
| 110 | 110 | { |
| 111 | 111 | struct ata_link *link = qc->dev->link; |
| 112 | 112 | struct ata_port *ap = link->ap; |
| 113 | + int ret; | |
| 113 | 114 | |
| 114 | 115 | if (ap->excl_link == NULL || ap->excl_link == link) { |
| 115 | 116 | if (ap->nr_active_links == 0 || ata_link_active(link)) { |
| 116 | 117 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; |
| 117 | − return ata_std_qc_defer(qc); | |
| 118 | + ret = ata_std_qc_defer(qc); | |
| 119 | + if (ret == ATA_DEFER_LINK) | |
| 120 | + return ATA_DEFER_LINK_EXCL; | |
| 121 | + return ret; | |
| 118 | 122 | } |
| 119 | 123 | |
| 124 | + /* | |
| 125 | + * Note: ap->excl_link contains the link that is next in line, | |
| 126 | + * i.e. implicit round robin. If there is only one link | |
| 127 | + * dispatching, ap->excl_link will be left unclaimed, allowing | |
| 128 | + * other links to set ap->excl_link, ensuring that the currently | |
| 129 | + * active link cannot queue any more. | |
| 130 | + */ | |
| 120 | 131 | ap->excl_link = link; |
| 121 | 132 | } |
| 122 | 133 | |
| @@ -571,8 +582,11 @@ static void sata_pmp_detach(struct ata_device *dev) · 448 unmodified lines | ||
| 571 | 582 | if (ap->ops->pmp_detach) |
| 572 | 583 | ap->ops->pmp_detach(ap); |
| 573 | 584 | |
| 574 | − ata_for_each_link(tlink, ap, EDGE) | |
| 585 | + ata_for_each_link(tlink, ap, EDGE) { | |
| 586 | + WARN_ON(tlink->deferred_qc); | |
| 587 | + cancel_work_sync(&tlink->deferred_qc_work); | |
| 575 | 588 | ata_eh_detach_dev(tlink->device); |
| 589 | + } | |
| 576 | 590 | |
| 577 | 591 | spin_lock_irqsave(ap->lock, flags); |
| 578 | 592 | ap->nr_pmp_links = 0; |
| @@ -1664,8 +1664,9 @@ static void ata_scsi_qc_done(struct ata_queued_cmd *qc, bool set_result, · 1663 unmodified lines | ||
| 1664 | 1664 | |
| 1665 | 1665 | void ata_scsi_deferred_qc_work(struct work_struct *work) |
| 1666 | 1666 | { |
| 1667 | − struct ata_port *ap = | |
| 1668 | − container_of(work, struct ata_port, deferred_qc_work); | |
| 1667 | + struct ata_link *link = | |
| 1668 | + container_of(work, struct ata_link, deferred_qc_work); | |
| 1669 | + struct ata_port *ap = link->ap; | |
| 1669 | 1670 | struct ata_queued_cmd *qc; |
| 1670 | 1671 | unsigned long flags; |
| 1671 | 1672 | |
| @@ -1676,10 +1677,10 @@ void ata_scsi_deferred_qc_work(struct work_struct *work) · 4 unmodified lines | ||
| 1676 | 1677 | * such case, we should not need any more deferring the qc, so warn if |
| 1677 | 1678 | * qc_defer() says otherwise. |
| 1678 | 1679 | */ |
| 1679 | − qc = ap->deferred_qc; | |
| 1680 | + qc = link->deferred_qc; | |
| 1680 | 1681 | if (qc && !ata_port_eh_scheduled(ap)) { |
| 1681 | 1682 | WARN_ON_ONCE(ap->ops->qc_defer(qc)); |
| 1682 | − ap->deferred_qc = NULL; | |
| 1683 | + link->deferred_qc = NULL; | |
| 1683 | 1684 | ata_qc_issue(qc); |
| 1684 | 1685 | } |
| 1685 | 1686 | |
| @@ -1688,7 +1689,7 @@ void ata_scsi_deferred_qc_work(struct work_struct *work) · 2 unmodified lines | ||
| 1688 | 1689 | |
| 1689 | 1690 | void ata_scsi_requeue_deferred_qc(struct ata_port *ap) |
| 1690 | 1691 | { |
| 1691 | − struct ata_queued_cmd *qc = ap->deferred_qc; | |
| 1692 | + struct ata_link *link; | |
| 1692 | 1693 | |
| 1693 | 1694 | lockdep_assert_held(ap->lock); |
| 1694 | 1695 | |
| @@ -1697,16 +1698,21 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap) · 2 unmodified lines | ||
| 1697 | 1698 | * do not try to be smart about what to do with this deferred command |
| 1698 | 1699 | * and simply requeue it by completing it with DID_REQUEUE. |
| 1699 | 1700 | */ |
| 1700 | − if (qc) { | |
| 1701 | − ap->deferred_qc = NULL; | |
| 1702 | − cancel_work(&ap->deferred_qc_work); | |
| 1703 | − ata_scsi_qc_done(qc, true, DID_REQUEUE << 16); | |
| 1701 | + ata_for_each_link(link, ap, PMP_FIRST) { | |
| 1702 | + struct ata_queued_cmd *qc = link->deferred_qc; | |
| 1703 | + | |
| 1704 | + if (qc) { | |
| 1705 | + link->deferred_qc = NULL; | |
| 1706 | + cancel_work(&link->deferred_qc_work); | |
| 1707 | + ata_scsi_qc_done(qc, true, DID_REQUEUE << 16); | |
| 1708 | + } | |
| 1704 | 1709 | } |
| 1705 | 1710 | } |
| 1706 | 1711 | |
| 1707 | −static void ata_scsi_schedule_deferred_qc(struct ata_port *ap) | |
| 1712 | +static void ata_scsi_schedule_deferred_qc(struct ata_link *link) | |
| 1708 | 1713 | { |
| 1709 | − struct ata_queued_cmd *qc = ap->deferred_qc; | |
| 1714 | + struct ata_queued_cmd *qc = link->deferred_qc; | |
| 1715 | + struct ata_port *ap = link->ap; | |
| 1710 | 1716 | |
| 1711 | 1717 | lockdep_assert_held(ap->lock); |
| 1712 | 1718 | |
| @@ -1723,12 +1729,12 @@ static void ata_scsi_schedule_deferred_qc(struct ata_port *ap) · 10 unmodified lines | ||
| 1723 | 1729 | return; |
| 1724 | 1730 | } |
| 1725 | 1731 | if (!ap->ops->qc_defer(qc)) |
| 1726 | − queue_work(system_highpri_wq, &ap->deferred_qc_work); | |
| 1732 | + queue_work(system_highpri_wq, &link->deferred_qc_work); | |
| 1727 | 1733 | } |
| 1728 | 1734 | |
| 1729 | 1735 | static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) |
| 1730 | 1736 | { |
| 1731 | − struct ata_port *ap = qc->ap; | |
| 1737 | + struct ata_link *link = qc->dev->link; | |
| 1732 | 1738 | struct scsi_cmnd *cmd = qc->scsicmd; |
| 1733 | 1739 | u8 *cdb = cmd->cmnd; |
| 1734 | 1740 | bool have_sense = qc->flags & ATA_QCFLAG_SENSE_VALID; |
| @@ -1759,22 +1765,23 @@ static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) · 24 unmodified lines | ||
| 1759 | 1765 | |
| 1760 | 1766 | ata_scsi_qc_done(qc, false, 0); |
| 1761 | 1767 | |
| 1762 | − ata_scsi_schedule_deferred_qc(ap); | |
| 1768 | + ata_scsi_schedule_deferred_qc(link); | |
| 1763 | 1769 | } |
| 1764 | 1770 | |
| 1765 | 1771 | static int ata_scsi_qc_issue(struct ata_port *ap, struct ata_queued_cmd *qc) |
| 1766 | 1772 | { |
| 1773 | + struct ata_link *link = qc->dev->link; | |
| 1767 | 1774 | int ret; |
| 1768 | 1775 | |
| 1769 | 1776 | if (!ap->ops->qc_defer) |
| 1770 | − goto issue; | |
| 1777 | + goto issue_qc; | |
| 1771 | 1778 | |
| 1772 | 1779 | /* |
| 1773 | 1780 | * If we already have a deferred qc, then rely on the SCSI layer to |
| 1774 | 1781 | * requeue and defer all incoming commands until the deferred qc is |
| 1775 | 1782 | * processed, once all on-going commands complete. |
| 1776 | 1783 | */ |
| 1777 | − if (ap->deferred_qc) { | |
| 1784 | + if (link->deferred_qc) { | |
| 1778 | 1785 | ata_qc_free(qc); |
| 1779 | 1786 | return SCSI_MLQUEUE_DEVICE_BUSY; |
| 1780 | 1787 | } |
| @@ -1786,38 +1793,46 @@ static int ata_scsi_qc_issue(struct ata_port *ap, struct ata_queued_cmd *qc) · 5 unmodified lines | ||
| 1786 | 1793 | break; |
| 1787 | 1794 | case ATA_DEFER_LINK: |
| 1788 | 1795 | ret = SCSI_MLQUEUE_DEVICE_BUSY; |
| 1789 | − break; | |
| 1796 | + goto defer_qc; | |
| 1797 | + case ATA_DEFER_LINK_EXCL: | |
| 1798 | + /* | |
| 1799 | + * Drivers making use of ap->excl_link cannot store the QC in | |
| 1800 | + * link->deferred_qc, because the ap->excl_link handling is | |
| 1801 | + * incompatible with the link->deferred_qc workqueue handling. | |
| 1802 | + */ | |
| 1803 | + ret = SCSI_MLQUEUE_DEVICE_BUSY; | |
| 1804 | + goto free_qc; | |
| 1790 | 1805 | case ATA_DEFER_PORT: |
| 1791 | 1806 | ret = SCSI_MLQUEUE_HOST_BUSY; |
| 1792 | − break; | |
| 1807 | + goto free_qc; | |
| 1793 | 1808 | default: |
| 1794 | 1809 | WARN_ON_ONCE(1); |
| 1795 | 1810 | ret = SCSI_MLQUEUE_HOST_BUSY; |
| 1796 | − break; | |
| 1811 | + goto free_qc; | |
| 1797 | 1812 | } |
| 1798 | 1813 | |
| 1799 | − if (ret) { | |
| 1800 | − /* | |
| 1801 | − * We must defer this qc: if this is not an NCQ command, keep | |
| 1802 | − * this qc as a deferred one and report to the SCSI layer that | |
| 1803 | − * we issued it so that it is not requeued. The deferred qc will | |
| 1804 | − * be issued with the port deferred_qc_work once all on-going | |
| 1805 | − * commands complete. | |
| 1806 | − */ | |
| 1807 | − if (!ata_is_ncq(qc->tf.protocol)) { | |
| 1808 | − ap->deferred_qc = qc; | |
| 1809 | − return 0; | |
| 1810 | − } | |
| 1814 | +issue_qc: | |
| 1815 | + ata_qc_issue(qc); | |
| 1816 | + return 0; | |
| 1811 | 1817 | |
| 1812 | − /* Force a requeue of the command to defer its execution. */ | |
| 1813 | − ata_qc_free(qc); | |
| 1814 | − return ret; | |
| 1818 | +defer_qc: | |
| 1819 | + /* | |
| 1820 | + * We must defer this qc: if this is not an NCQ command, keep | |
| 1821 | + * this qc as a deferred one and report to the SCSI layer that | |
| 1822 | + * we issued it so that it is not requeued. The deferred qc will | |
| 1823 | + * be issued with the port deferred_qc_work once all on-going | |
| 1824 | + * commands complete. | |
| 1825 | + */ | |
| 1826 | + if (!ata_is_ncq(qc->tf.protocol)) { | |
| 1827 | + link->deferred_qc = qc; | |
| 1828 | + return 0; | |
| 1815 | 1829 | } |
| 1816 | 1830 | |
| 1817 | −issue: | |
| 1818 | − ata_qc_issue(qc); | |
| 1831 | +free_qc: | |
| 1832 | + /* Force a requeue of the command to defer its execution. */ | |
| 1833 | + ata_qc_free(qc); | |
| 1819 | 1834 | |
| 1820 | − return 0; | |
| 1835 | + return ret; | |
| 1821 | 1836 | } |
| 1822 | 1837 | |
| 1823 | 1838 | /** |
| @@ -459,19 +459,11 @@ static void pata_parport_dev_release(struct device *dev) · 458 unmodified lines | ||
| 459 | 459 | kfree(pi); |
| 460 | 460 | } |
| 461 | 461 | |
| 462 | −static void pata_parport_bus_release(struct device *dev) | |
| 463 | −{ | |
| 464 | − /* nothing to do here but required to avoid warning on device removal */ | |
| 465 | −} | |
| 466 | − | |
| 467 | 462 | static const struct bus_type pata_parport_bus_type = { |
| 468 | 463 | .name = DRV_NAME, |
| 469 | 464 | }; |
| 470 | 465 | |
| 471 | −static struct device pata_parport_bus = { | |
| 472 | − .init_name = DRV_NAME, | |
| 473 | − .release = pata_parport_bus_release, | |
| 474 | −}; | |
| 466 | +static struct device *pata_parport_bus; | |
| 475 | 467 | |
| 476 | 468 | static const struct scsi_host_template pata_parport_sht = { |
| 477 | 469 | PATA_PARPORT_SHT("pata_parport") |
| @@ -518,7 +510,7 @@ static struct pi_adapter *pi_init_one(struct parport *parport, · 40 unmodified lines | ||
| 518 | 510 | } |
| 519 | 511 | |
| 520 | 512 | /* set up pi->dev before pi_probe_unit() so it can use dev_printk() */ |
| 521 | − pi->dev.parent = &pata_parport_bus; | |
| 513 | + pi->dev.parent = pata_parport_bus; | |
| 522 | 514 | pi->dev.bus = &pata_parport_bus_type; |
| 523 | 515 | pi->dev.driver = &pr->driver; |
| 524 | 516 | pi->dev.release = pata_parport_dev_release; |
| @@ -780,8 +772,9 @@ static __init int pata_parport_init(void) · 255 unmodified lines | ||
| 780 | 772 | return error; |
| 781 | 773 | } |
| 782 | 774 | |
| 783 | − error = device_register(&pata_parport_bus); | |
| 784 | − if (error) { | |
| 775 | + pata_parport_bus = root_device_register(DRV_NAME); | |
| 776 | + if (IS_ERR(pata_parport_bus)) { | |
| 777 | + error = PTR_ERR(pata_parport_bus); | |
| 785 | 778 | pr_err("failed to register pata_parport bus, error: %d\n", error); |
| 786 | 779 | goto out_unregister_bus; |
| 787 | 780 | } |
| @@ -811,7 +804,7 @@ out_remove_del: · 23 unmodified lines | ||
| 811 | 804 | out_remove_new: |
| 812 | 805 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); |
| 813 | 806 | out_unregister_dev: |
| 814 | − device_unregister(&pata_parport_bus); | |
| 807 | + root_device_unregister(pata_parport_bus); | |
| 815 | 808 | out_unregister_bus: |
| 816 | 809 | bus_unregister(&pata_parport_bus_type); |
| 817 | 810 | return error; |
| @@ -822,7 +815,7 @@ static __exit void pata_parport_exit(void) · 4 unmodified lines | ||
| 822 | 815 | parport_unregister_driver(&pata_parport_driver); |
| 823 | 816 | bus_remove_file(&pata_parport_bus_type, &bus_attr_new_device); |
| 824 | 817 | bus_remove_file(&pata_parport_bus_type, &bus_attr_delete_device); |
| 825 | − device_unregister(&pata_parport_bus); | |
| 818 | + root_device_unregister(pata_parport_bus); | |
| 826 | 819 | bus_unregister(&pata_parport_bus_type); |
| 827 | 820 | } |
| 828 | 821 | |
| @@ -789,6 +789,7 @@ static int sil24_qc_defer(struct ata_queued_cmd *qc) · 788 unmodified lines | ||
| 789 | 789 | struct ata_link *link = qc->dev->link; |
| 790 | 790 | struct ata_port *ap = link->ap; |
| 791 | 791 | u8 prot = qc->tf.protocol; |
| 792 | + int ret; | |
| 792 | 793 | |
| 793 | 794 | /* |
| 794 | 795 | * There is a bug in the chip: |
| @@ -826,7 +827,10 @@ static int sil24_qc_defer(struct ata_queued_cmd *qc) · 31 unmodified lines | ||
| 826 | 827 | qc->flags |= ATA_QCFLAG_CLEAR_EXCL; |
| 827 | 828 | } |
| 828 | 829 | |
| 829 | − return ata_std_qc_defer(qc); | |
| 830 | + ret = ata_std_qc_defer(qc); | |
| 831 | + if (ret == ATA_DEFER_LINK) | |
| 832 | + return ATA_DEFER_LINK_EXCL; | |
| 833 | + return ret; | |
| 830 | 834 | } |
| 831 | 835 | |
| 832 | 836 | static enum ata_completion_errors sil24_qc_prep(struct ata_queued_cmd *qc) |
| @@ -1230,8 +1230,10 @@ void memblk_nr_poison_inc(unsigned long pfn) · 1229 unmodified lines | ||
| 1230 | 1230 | const unsigned long block_id = pfn_to_block_id(pfn); |
| 1231 | 1231 | struct memory_block *mem = find_memory_block_by_id(block_id); |
| 1232 | 1232 | |
| 1233 | − if (mem) | |
| 1233 | + if (mem) { | |
| 1234 | 1234 | atomic_long_inc(&mem->nr_hwpoison); |
| 1235 | + put_device(&mem->dev); | |
| 1236 | + } | |
| 1235 | 1237 | } |
| 1236 | 1238 | |
| 1237 | 1239 | void memblk_nr_poison_sub(unsigned long pfn, long i) |
| @@ -1239,8 +1241,10 @@ void memblk_nr_poison_sub(unsigned long pfn, long i) · 1 unmodified line | ||
| 1239 | 1241 | const unsigned long block_id = pfn_to_block_id(pfn); |
| 1240 | 1242 | struct memory_block *mem = find_memory_block_by_id(block_id); |
| 1241 | 1243 | |
| 1242 | − if (mem) | |
| 1244 | + if (mem) { | |
| 1243 | 1245 | atomic_long_sub(i, &mem->nr_hwpoison); |
| 1246 | + put_device(&mem->dev); | |
| 1247 | + } | |
| 1244 | 1248 | } |
| 1245 | 1249 | |
| 1246 | 1250 | static unsigned long memblk_nr_poison(struct memory_block *mem) |
| @@ -606,6 +606,12 @@ static void platform_device_release(struct device *dev) · 605 unmodified lines | ||
| 606 | 606 | kfree(pa); |
| 607 | 607 | } |
| 608 | 608 | |
| 609 | +static void platform_device_release_full(struct device *dev) | |
| 610 | +{ | |
| 611 | + device_remove_software_node(dev); | |
| 612 | + platform_device_release(dev); | |
| 613 | +} | |
| 614 | + | |
| 609 | 615 | /** |
| 610 | 616 | * platform_device_alloc - create a platform device |
| 611 | 617 | * @name: base name of the device we're adding |
| @@ -848,7 +854,13 @@ struct platform_device *platform_device_register_full(const struct platform_devi · 236 unmodified lines | ||
| 848 | 854 | int ret; |
| 849 | 855 | struct platform_device *pdev; |
| 850 | 856 | |
| 851 | − if (pdevinfo->swnode && pdevinfo->properties) | |
| 857 | + /* | |
| 858 | + * Only one software node per device is allowed. Make sure we don't | |
| 859 | + * accept or create two. | |
| 860 | + */ | |
| 861 | + if ((pdevinfo->swnode && pdevinfo->properties) || | |
| 862 | + (pdevinfo->swnode && is_software_node(pdevinfo->fwnode)) || | |
| 863 | + (pdevinfo->properties && is_software_node(pdevinfo->fwnode))) | |
| 852 | 864 | return ERR_PTR(-EINVAL); |
| 853 | 865 | |
| 854 | 866 | pdev = platform_device_alloc(pdevinfo->name, pdevinfo->id); |
| @@ -878,6 +890,8 @@ struct platform_device *platform_device_register_full(const struct platform_devi · 23 unmodified lines | ||
| 878 | 890 | ret = device_add_software_node(&pdev->dev, pdevinfo->swnode); |
| 879 | 891 | if (ret) |
| 880 | 892 | goto err; |
| 893 | + | |
| 894 | + pdev->dev.release = platform_device_release_full; | |
| 881 | 895 | } else if (pdevinfo->properties) { |
| 882 | 896 | ret = device_create_managed_software_node(&pdev->dev, |
| 883 | 897 | pdevinfo->properties, NULL); |
| @@ -172,7 +172,7 @@ static int regmap_sdw_mbq_read(void *context, unsigned int reg, unsigned int *va · 171 unmodified lines | ||
| 172 | 172 | ret = regmap_sdw_mbq_read_impl(slave, reg, val, mbq_size); |
| 173 | 173 | if (ret == -ENODATA) { |
| 174 | 174 | if (!deferrable) |
| 175 | − dev_warn(dev, "Defer on undeferable control: %x\n", reg); | |
| 175 | + dev_warn(dev, "Defer on undeferrable control: %x\n", reg); | |
| 176 | 176 | |
| 177 | 177 | ret = regmap_sdw_mbq_poll_busy(slave, reg, ctx); |
| 178 | 178 | if (ret) |
| @@ -3257,6 +3257,9 @@ static int _regmap_update_bits(struct regmap *map, unsigned int reg, · 3256 unmodified lines | ||
| 3257 | 3257 | *change = false; |
| 3258 | 3258 | |
| 3259 | 3259 | if (regmap_volatile(map, reg) && map->reg_update_bits) { |
| 3260 | + if (map->cache_only) | |
| 3261 | + return -EBUSY; | |
| 3262 | + | |
| 3260 | 3263 | reg = regmap_reg_addr(map, reg); |
| 3261 | 3264 | ret = map->reg_update_bits(map->bus_context, reg, mask, val); |
| 3262 | 3265 | if (ret == 0 && change) |
| @@ -4565,24 +4565,12 @@ out: · 4564 unmodified lines | ||
| 4565 | 4565 | return ret; |
| 4566 | 4566 | } |
| 4567 | 4567 | |
| 4568 | −static void cancel_tasks_sync(struct rbd_device *rbd_dev) | |
| 4569 | −{ | |
| 4570 | − dout("%s rbd_dev %p\n", __func__, rbd_dev); | |
| 4571 | − | |
| 4572 | − cancel_work_sync(&rbd_dev->acquired_lock_work); | |
| 4573 | − cancel_work_sync(&rbd_dev->released_lock_work); | |
| 4574 | − cancel_delayed_work_sync(&rbd_dev->lock_dwork); | |
| 4575 | − cancel_work_sync(&rbd_dev->unlock_work); | |
| 4576 | −} | |
| 4577 | − | |
| 4578 | 4568 | /* |
| 4579 | 4569 | * header_rwsem must not be held to avoid a deadlock with |
| 4580 | 4570 | * rbd_dev_refresh() when flushing notifies. |
| 4581 | 4571 | */ |
| 4582 | 4572 | static void rbd_unregister_watch(struct rbd_device *rbd_dev) |
| 4583 | 4573 | { |
| 4584 | − cancel_tasks_sync(rbd_dev); | |
| 4585 | − | |
| 4586 | 4574 | mutex_lock(&rbd_dev->watch_mutex); |
| 4587 | 4575 | if (rbd_dev->watch_state == RBD_WATCH_STATE_REGISTERED) |
| 4588 | 4576 | __rbd_unregister_watch(rbd_dev); |
| @@ -6548,10 +6536,18 @@ out_err: · 1959 unmodified lines | ||
| 6548 | 6536 | |
| 6549 | 6537 | static void rbd_dev_image_unlock(struct rbd_device *rbd_dev) |
| 6550 | 6538 | { |
| 6539 | + dout("%s rbd_dev %p\n", __func__, rbd_dev); | |
| 6540 | + | |
| 6541 | + disable_delayed_work_sync(&rbd_dev->lock_dwork); | |
| 6542 | + disable_work_sync(&rbd_dev->unlock_work); | |
| 6543 | + | |
| 6551 | 6544 | down_write(&rbd_dev->lock_rwsem); |
| 6552 | 6545 | if (__rbd_is_lock_owner(rbd_dev)) |
| 6553 | 6546 | __rbd_release_lock(rbd_dev); |
| 6554 | 6547 | up_write(&rbd_dev->lock_rwsem); |
| 6548 | + | |
| 6549 | + flush_work(&rbd_dev->acquired_lock_work); | |
| 6550 | + flush_work(&rbd_dev->released_lock_work); | |
| 6555 | 6551 | } |
| 6556 | 6552 | |
| 6557 | 6553 | /* |
| @@ -900,12 +900,29 @@ static int ublk_validate_params(const struct ublk_device *ub) · 899 unmodified lines | ||
| 900 | 900 | if (p->logical_bs_shift > PAGE_SHIFT || p->logical_bs_shift < 9) |
| 901 | 901 | return -EINVAL; |
| 902 | 902 | |
| 903 | + /* | |
| 904 | + * 256M is a reasonable upper bound for physical block size, | |
| 905 | + * io_min and io_opt; it aligns with the maximum physical | |
| 906 | + * block size possible in NVMe. | |
| 907 | + */ | |
| 908 | + if (p->physical_bs_shift > ilog2(SZ_256M)) | |
| 909 | + return -EINVAL; | |
| 910 | + | |
| 911 | + if (p->io_min_shift > ilog2(SZ_256M)) | |
| 912 | + return -EINVAL; | |
| 913 | + | |
| 914 | + if (p->io_opt_shift > ilog2(SZ_256M)) | |
| 915 | + return -EINVAL; | |
| 916 | + | |
| 903 | 917 | if (p->logical_bs_shift > p->physical_bs_shift) |
| 904 | 918 | return -EINVAL; |
| 905 | 919 | |
| 906 | 920 | if (p->max_sectors > (ub->dev_info.max_io_buf_bytes >> 9)) |
| 907 | 921 | return -EINVAL; |
| 908 | 922 | |
| 923 | + if (p->max_sectors < PAGE_SECTORS) | |
| 924 | + return -EINVAL; | |
| 925 | + | |
| 909 | 926 | if (ublk_dev_is_zoned(ub) && !p->chunk_sectors) |
| 910 | 927 | return -EINVAL; |
| 911 | 928 | } else |
| @@ -2397,8 +2414,14 @@ static void ublk_reset_ch_dev(struct ublk_device *ub) · 1485 unmodified lines | ||
| 2397 | 2414 | { |
| 2398 | 2415 | int i; |
| 2399 | 2416 | |
| 2400 | − for (i = 0; i < ub->dev_info.nr_hw_queues; i++) | |
| 2401 | − ublk_queue_reinit(ub, ublk_get_queue(ub, i)); | |
| 2417 | + for (i = 0; i < ub->dev_info.nr_hw_queues; i++) { | |
| 2418 | + struct ublk_queue *ubq = ublk_get_queue(ub, i); | |
| 2419 | + | |
| 2420 | + /* Sync with ublk_cancel_cmd() */ | |
| 2421 | + spin_lock(&ubq->cancel_lock); | |
| 2422 | + ublk_queue_reinit(ub, ubq); | |
| 2423 | + spin_unlock(&ubq->cancel_lock); | |
| 2424 | + } | |
| 2402 | 2425 | |
| 2403 | 2426 | /* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */ |
| 2404 | 2427 | ub->mm = NULL; |
| @@ -2739,6 +2762,7 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag, · 334 unmodified lines | ||
| 2739 | 2762 | { |
| 2740 | 2763 | struct ublk_io *io = &ubq->ios[tag]; |
| 2741 | 2764 | struct ublk_device *ub = ubq->dev; |
| 2765 | + struct io_uring_cmd *cmd = NULL; | |
| 2742 | 2766 | struct request *req; |
| 2743 | 2767 | bool done; |
| 2744 | 2768 | |
| @@ -2761,12 +2785,15 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag, · 16 unmodified lines | ||
| 2761 | 2785 | |
| 2762 | 2786 | spin_lock(&ubq->cancel_lock); |
| 2763 | 2787 | done = !!(io->flags & UBLK_IO_FLAG_CANCELED); |
| 2764 | − if (!done) | |
| 2788 | + if (!done) { | |
| 2765 | 2789 | io->flags |= UBLK_IO_FLAG_CANCELED; |
| 2790 | + cmd = io->cmd; | |
| 2791 | + io->cmd = NULL; | |
| 2792 | + } | |
| 2766 | 2793 | spin_unlock(&ubq->cancel_lock); |
| 2767 | 2794 | |
| 2768 | − if (!done) | |
| 2769 | − io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, issue_flags); | |
| 2795 | + if (!done && cmd) | |
| 2796 | + io_uring_cmd_done(cmd, UBLK_IO_RES_ABORT, issue_flags); | |
| 2770 | 2797 | } |
| 2771 | 2798 | |
| 2772 | 2799 | /* |
| @@ -3496,8 +3523,10 @@ static void ublk_ch_uring_cmd_cb(struct io_tw_req tw_req, io_tw_token_t tw) · 723 unmodified lines | ||
| 3496 | 3523 | { |
| 3497 | 3524 | unsigned int issue_flags = IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; |
| 3498 | 3525 | struct io_uring_cmd *cmd = io_uring_cmd_from_tw(tw_req); |
| 3499 | − int ret = ublk_ch_uring_cmd_local(cmd, issue_flags); | |
| 3526 | + int ret = -ECANCELED; | |
| 3500 | 3527 | |
| 3528 | + if (!tw.cancel) | |
| 3529 | + ret = ublk_ch_uring_cmd_local(cmd, issue_flags); | |
| 3501 | 3530 | if (ret != -EIOCBQUEUED) |
| 3502 | 3531 | io_uring_cmd_done(cmd, ret, issue_flags); |
| 3503 | 3532 | } |
| @@ -4990,13 +5019,15 @@ static int ublk_ctrl_set_params(struct ublk_device *ub, · 1486 unmodified lines | ||
| 4990 | 5019 | */ |
| 4991 | 5020 | ret = -EACCES; |
| 4992 | 5021 | } else if (copy_from_user(&ub->params, argp, ph.len)) { |
| 5022 | + /* zero out partial copy so no stale params survive */ | |
| 5023 | + memset(&ub->params, 0, sizeof(ub->params)); | |
| 4993 | 5024 | ret = -EFAULT; |
| 4994 | 5025 | } else { |
| 4995 | 5026 | /* clear all we don't support yet */ |
| 4996 | 5027 | ub->params.types &= UBLK_PARAM_TYPE_ALL; |
| 4997 | 5028 | ret = ublk_validate_params(ub); |
| 4998 | 5029 | if (ret) |
| 4999 | − ub->params.types = 0; | |
| 5030 | + memset(&ub->params, 0, sizeof(ub->params)); | |
| 5000 | 5031 | } |
| 5001 | 5032 | mutex_unlock(&ub->mutex); |
| 5002 | 5033 | |
| @@ -33,6 +33,7 @@ · 32 unmodified lines | ||
| 33 | 33 | #include <linux/cpuhotplug.h> |
| 34 | 34 | #include <linux/part_stat.h> |
| 35 | 35 | #include <linux/kernel_read_file.h> |
| 36 | +#include <linux/rcupdate.h> | |
| 36 | 37 | |
| 37 | 38 | #include "zram_drv.h" |
| 38 | 39 | |
| @@ -504,6 +505,7 @@ struct zram_wb_ctl { · 465 unmodified lines | ||
| 504 | 505 | wait_queue_head_t done_wait; |
| 505 | 506 | spinlock_t done_lock; |
| 506 | 507 | atomic_t num_inflight; |
| 508 | + struct rcu_head rcu; | |
| 507 | 509 | }; |
| 508 | 510 | |
| 509 | 511 | struct zram_wb_req { |
| @@ -847,7 +849,7 @@ static void release_wb_ctl(struct zram_wb_ctl *wb_ctl) · 337 unmodified lines | ||
| 847 | 849 | release_wb_req(req); |
| 848 | 850 | } |
| 849 | 851 | |
| 850 | − kfree(wb_ctl); | |
| 852 | + kfree_rcu(wb_ctl, rcu); | |
| 851 | 853 | } |
| 852 | 854 | |
| 853 | 855 | static struct zram_wb_ctl *init_wb_ctl(struct zram *zram) |
| @@ -964,11 +966,13 @@ static void zram_writeback_endio(struct bio *bio) · 110 unmodified lines | ||
| 964 | 966 | struct zram_wb_ctl *wb_ctl = bio->bi_private; |
| 965 | 967 | unsigned long flags; |
| 966 | 968 | |
| 969 | + rcu_read_lock(); | |
| 967 | 970 | spin_lock_irqsave(&wb_ctl->done_lock, flags); |
| 968 | 971 | list_add(&req->entry, &wb_ctl->done_reqs); |
| 969 | 972 | spin_unlock_irqrestore(&wb_ctl->done_lock, flags); |
| 970 | 973 | |
| 971 | 974 | wake_up(&wb_ctl->done_wait); |
| 975 | + rcu_read_unlock(); | |
| 972 | 976 | } |
| 973 | 977 | |
| 974 | 978 | static void zram_submit_wb_request(struct zram *zram, |
| @@ -289,6 +289,9 @@ static inline void btintel_pcie_dump_debug_registers(struct hci_dev *hdev) · 288 unmodified lines | ||
| 289 | 289 | skb_put_data(skb, buf, strlen(buf)); |
| 290 | 290 | data->boot_stage_cache = reg; |
| 291 | 291 | |
| 292 | + if (reg & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING) | |
| 293 | + bt_dev_warn(hdev, "Controller device warning (boot_stage: 0x%8.8x)", reg); | |
| 294 | + | |
| 292 | 295 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_IPC_STATUS_REG); |
| 293 | 296 | snprintf(buf, sizeof(buf), "ipc status: 0x%8.8x", reg); |
| 294 | 297 | skb_put_data(skb, buf, strlen(buf)); |
| @@ -579,12 +582,10 @@ static int btintel_pcie_get_mac_access(struct btintel_pcie_data *data) · 284 unmodified lines | ||
| 579 | 582 | |
| 580 | 583 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| 581 | 584 | |
| 582 | − reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS; | |
| 583 | − reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ; | |
| 584 | − if ((reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_STS) == 0) | |
| 585 | + if (!(reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ)) { | |
| 585 | 586 | reg |= BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; |
| 586 | − | |
| 587 | − btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 587 | + btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 588 | + } | |
| 588 | 589 | |
| 589 | 590 | do { |
| 590 | 591 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| @@ -604,16 +605,10 @@ static void btintel_pcie_release_mac_access(struct btintel_pcie_data *data) · 13 unmodified lines | ||
| 604 | 605 | |
| 605 | 606 | reg = btintel_pcie_rd_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG); |
| 606 | 607 | |
| 607 | − if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ) | |
| 608 | + if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ) { | |
| 608 | 609 | reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ; |
| 609 | − | |
| 610 | − if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS) | |
| 611 | − reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS; | |
| 612 | − | |
| 613 | − if (reg & BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ) | |
| 614 | − reg &= ~BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ; | |
| 615 | − | |
| 616 | − btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 610 | + btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_FUNC_CTRL_REG, reg); | |
| 611 | + } | |
| 617 | 612 | } |
| 618 | 613 | |
| 619 | 614 | static void *btintel_pcie_copy_tlv(void *dest, enum btintel_pcie_tlv_type type, |
| @@ -880,8 +875,11 @@ static inline bool btintel_pcie_in_lockdown(struct btintel_pcie_data *data) · 260 unmodified lines | ||
| 880 | 875 | |
| 881 | 876 | static inline bool btintel_pcie_in_error(struct btintel_pcie_data *data) |
| 882 | 877 | { |
| 883 | − return (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_ERR) || | |
| 884 | − (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER); | |
| 878 | + if (data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING) | |
| 879 | + bt_dev_warn(data->hdev, "Controller device warning (boot_stage: 0x%8.8x)", | |
| 880 | + data->boot_stage_cache); | |
| 881 | + | |
| 882 | + return data->boot_stage_cache & BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER; | |
| 885 | 883 | } |
| 886 | 884 | |
| 887 | 885 | static void btintel_pcie_msix_gp1_handler(struct btintel_pcie_data *data) |
| @@ -914,7 +912,8 @@ static void btintel_pcie_msix_gp0_handler(struct btintel_pcie_data *data) · 26 unmodified lines | ||
| 914 | 912 | data->img_resp_cache = reg; |
| 915 | 913 | |
| 916 | 914 | if (btintel_pcie_in_error(data)) { |
| 917 | − bt_dev_err(data->hdev, "Controller in error state"); | |
| 915 | + bt_dev_err(data->hdev, "Controller in error state (boot_stage: 0x%8.8x)", | |
| 916 | + data->boot_stage_cache); | |
| 918 | 917 | btintel_pcie_dump_debug_registers(data->hdev); |
| 919 | 918 | return; |
| 920 | 919 | } |
| @@ -34,9 +34,6 @@ · 33 unmodified lines | ||
| 34 | 34 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_STS (BIT(20)) |
| 35 | 35 | |
| 36 | 36 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_MAC_ACCESS_REQ (BIT(21)) |
| 37 | −/* Stop MAC Access disconnection request */ | |
| 38 | −#define BTINTEL_PCIE_CSR_FUNC_CTRL_STOP_MAC_ACCESS_DIS (BIT(22)) | |
| 39 | −#define BTINTEL_PCIE_CSR_FUNC_CTRL_XTAL_CLK_REQ (BIT(23)) | |
| 40 | 37 | |
| 41 | 38 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_STS (BIT(28)) |
| 42 | 39 | #define BTINTEL_PCIE_CSR_FUNC_CTRL_BUS_MASTER_DISCON (BIT(29)) |
| @@ -48,7 +45,7 @@ · 5 unmodified lines | ||
| 48 | 45 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_OPFW (BIT(2)) |
| 49 | 46 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ROM_LOCKDOWN (BIT(10)) |
| 50 | 47 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_IML_LOCKDOWN (BIT(11)) |
| 51 | −#define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_ERR (BIT(12)) | |
| 48 | +#define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_WARNING (BIT(12)) | |
| 52 | 49 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_ABORT_HANDLER (BIT(13)) |
| 53 | 50 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_DEVICE_HALTED (BIT(14)) |
| 54 | 51 | #define BTINTEL_PCIE_CSR_BOOT_STAGE_MAC_ACCESS_ON (BIT(16)) |
| @@ -537,6 +537,7 @@ static void btmtk_usb_wmt_recv(struct urb *urb) · 536 unmodified lines | ||
| 537 | 537 | return; |
| 538 | 538 | } else if (urb->status == -ENOENT) { |
| 539 | 539 | /* Avoid suspend failed when usb_kill_urb */ |
| 540 | + kfree(urb->setup_packet); | |
| 540 | 541 | return; |
| 541 | 542 | } |
| 542 | 543 | |
| @@ -610,6 +611,7 @@ static int btmtk_usb_submit_wmt_recv_urb(struct hci_dev *hdev) · 67 unmodified lines | ||
| 610 | 611 | if (err != -EPERM && err != -ENODEV) |
| 611 | 612 | bt_dev_err(hdev, "urb %p submission failed (%d)", |
| 612 | 613 | urb, -err); |
| 614 | + kfree(dr); | |
| 613 | 615 | usb_unanchor_urb(urb); |
| 614 | 616 | } |
| 615 | 617 | |
| @@ -695,8 +697,13 @@ static int btmtk_usb_hci_wmt_sync(struct hci_dev *hdev, · 79 unmodified lines | ||
| 695 | 697 | if (data->evt_skb == NULL) |
| 696 | 698 | goto err_free_wc; |
| 697 | 699 | |
| 698 | − /* Parse and handle the return WMT event */ | |
| 699 | − wmt_evt = (struct btmtk_hci_wmt_evt *)data->evt_skb->data; | |
| 700 | + wmt_evt = skb_pull_data(data->evt_skb, sizeof(*wmt_evt)); | |
| 701 | + if (!wmt_evt) { | |
| 702 | + bt_dev_err(hdev, "WMT event too short (%u bytes)", | |
| 703 | + data->evt_skb->len); | |
| 704 | + err = -EINVAL; | |
| 705 | + goto err_free_skb; | |
| 706 | + } | |
| 700 | 707 | if (wmt_evt->whdr.op != hdr->op) { |
| 701 | 708 | bt_dev_err(hdev, "Wrong op received %d expected %d", |
| 702 | 709 | wmt_evt->whdr.op, hdr->op); |
| @@ -712,6 +719,12 @@ static int btmtk_usb_hci_wmt_sync(struct hci_dev *hdev, · 9 unmodified lines | ||
| 712 | 719 | status = BTMTK_WMT_PATCH_DONE; |
| 713 | 720 | break; |
| 714 | 721 | case BTMTK_WMT_FUNC_CTRL: |
| 722 | + if (!skb_pull_data(data->evt_skb, | |
| 723 | + sizeof(wmt_evt_funcc->status))) { | |
| 724 | + status = BTMTK_WMT_ON_UNDONE; | |
| 725 | + break; | |
| 726 | + } | |
| 727 | + | |
| 715 | 728 | wmt_evt_funcc = (struct btmtk_hci_wmt_evt_funcc *)wmt_evt; |
| 716 | 729 | if (be16_to_cpu(wmt_evt_funcc->status) == 0x404) |
| 717 | 730 | status = BTMTK_WMT_ON_DONE; |
| @@ -3540,7 +3540,13 @@ static int btusb_setup_qca_load_rampatch(struct hci_dev *hdev, · 3539 unmodified lines | ||
| 3540 | 3540 | "firmware rome 0x%x build 0x%x", |
| 3541 | 3541 | rver_rom, rver_patch, ver_rom, ver_patch); |
| 3542 | 3542 | |
| 3543 | − if (rver_rom != ver_rom || rver_patch <= ver_patch) { | |
| 3543 | + /* Allow rampatch when the patch version equals the firmware version. | |
| 3544 | + * A firmware download may be aborted by a transient USB error (e.g. | |
| 3545 | + * disconnect) after the controller updates version info but before | |
| 3546 | + * completion. | |
| 3547 | + * Allowing equal versions enables re-flashing during recovery. | |
| 3548 | + */ | |
| 3549 | + if (rver_rom != ver_rom || rver_patch < ver_patch) { | |
| 3544 | 3550 | bt_dev_err(hdev, "rampatch file version did not match with firmware"); |
| 3545 | 3551 | err = -EINVAL; |
| 3546 | 3552 | goto done; |
| @@ -191,6 +191,9 @@ static int ath_recv(struct hci_uart *hu, const void *data, int count) · 190 unmodified lines | ||
| 191 | 191 | { |
| 192 | 192 | struct ath_struct *ath = hu->priv; |
| 193 | 193 | |
| 194 | + if (!ath) | |
| 195 | + return -ENODEV; | |
| 196 | + | |
| 194 | 197 | ath->rx_skb = h4_recv_buf(hu, ath->rx_skb, data, count, |
| 195 | 198 | ath_recv_pkts, ARRAY_SIZE(ath_recv_pkts)); |
| 196 | 199 | if (IS_ERR(ath->rx_skb)) { |
| @@ -585,6 +585,9 @@ static int bcsp_recv(struct hci_uart *hu, const void *data, int count) · 584 unmodified lines | ||
| 585 | 585 | if (!test_bit(HCI_UART_REGISTERED, &hu->flags)) |
| 586 | 586 | return -EUNATCH; |
| 587 | 587 | |
| 588 | + if (!bcsp) | |
| 589 | + return -ENODEV; | |
| 590 | + | |
| 588 | 591 | BT_DBG("hu %p count %d rx_state %d rx_count %ld", |
| 589 | 592 | hu, count, bcsp->rx_state, bcsp->rx_count); |
| 590 | 593 | |
| @@ -109,6 +109,9 @@ static int h4_recv(struct hci_uart *hu, const void *data, int count) · 108 unmodified lines | ||
| 109 | 109 | { |
| 110 | 110 | struct h4_struct *h4 = hu->priv; |
| 111 | 111 | |
| 112 | + if (!h4) | |
| 113 | + return -ENODEV; | |
| 114 | + | |
| 112 | 115 | h4->rx_skb = h4_recv_buf(hu, h4->rx_skb, data, count, |
| 113 | 116 | h4_recv_pkts, ARRAY_SIZE(h4_recv_pkts)); |
| 114 | 117 | if (IS_ERR(h4->rx_skb)) { |
| @@ -587,6 +587,9 @@ static int h5_recv(struct hci_uart *hu, const void *data, int count) · 586 unmodified lines | ||
| 587 | 587 | struct h5 *h5 = hu->priv; |
| 588 | 588 | const unsigned char *ptr = data; |
| 589 | 589 | |
| 590 | + if (!h5) | |
| 591 | + return -ENODEV; | |
| 592 | + | |
| 590 | 593 | BT_DBG("%s pending %zu count %d", hu->hdev->name, h5->rx_pending, |
| 591 | 594 | count); |
| 592 | 595 | |
| @@ -194,7 +194,15 @@ void hci_uart_init_work(struct work_struct *work) · 193 unmodified lines | ||
| 194 | 194 | err = hci_register_dev(hu->hdev); |
| 195 | 195 | if (err < 0) { |
| 196 | 196 | BT_ERR("Can't register HCI device"); |
| 197 | + | |
| 198 | + percpu_down_write(&hu->proto_lock); | |
| 197 | 199 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); |
| 200 | + percpu_up_write(&hu->proto_lock); | |
| 201 | + | |
| 202 | + /* Safely cancel work after clearing flags */ | |
| 203 | + cancel_work_sync(&hu->write_work); | |
| 204 | + | |
| 205 | + /* Close protocol before freeing hdev */ | |
| 198 | 206 | hu->proto->close(hu); |
| 199 | 207 | hdev = hu->hdev; |
| 200 | 208 | hu->hdev = NULL; |
| @@ -263,8 +271,12 @@ static int hci_uart_open(struct hci_dev *hdev) · 62 unmodified lines | ||
| 263 | 271 | /* Close device */ |
| 264 | 272 | static int hci_uart_close(struct hci_dev *hdev) |
| 265 | 273 | { |
| 274 | + struct hci_uart *hu = hci_get_drvdata(hdev); | |
| 275 | + | |
| 266 | 276 | BT_DBG("hdev %p", hdev); |
| 267 | 277 | |
| 278 | + cancel_work_sync(&hu->write_work); | |
| 279 | + | |
| 268 | 280 | hci_uart_flush(hdev); |
| 269 | 281 | hdev->flush = NULL; |
| 270 | 282 | return 0; |
| @@ -531,6 +543,7 @@ static void hci_uart_tty_close(struct tty_struct *tty) · 260 unmodified lines | ||
| 531 | 543 | { |
| 532 | 544 | struct hci_uart *hu = tty->disc_data; |
| 533 | 545 | struct hci_dev *hdev; |
| 546 | + bool proto_ready; | |
| 534 | 547 | |
| 535 | 548 | BT_DBG("tty %p", tty); |
| 536 | 549 | |
| @@ -540,24 +553,38 @@ static void hci_uart_tty_close(struct tty_struct *tty) · 3 unmodified lines | ||
| 540 | 553 | if (!hu) |
| 541 | 554 | return; |
| 542 | 555 | |
| 543 | − hdev = hu->hdev; | |
| 544 | − if (hdev) | |
| 545 | − hci_uart_close(hdev); | |
| 556 | + /* Wait for init_ready to finish to prevent registration races */ | |
| 557 | + cancel_work_sync(&hu->init_ready); | |
| 546 | 558 | |
| 547 | − if (test_bit(HCI_UART_PROTO_READY, &hu->flags)) { | |
| 559 | + proto_ready = test_bit(HCI_UART_PROTO_READY, &hu->flags); | |
| 560 | + if (proto_ready) { | |
| 548 | 561 | percpu_down_write(&hu->proto_lock); |
| 549 | 562 | clear_bit(HCI_UART_PROTO_READY, &hu->flags); |
| 550 | 563 | percpu_up_write(&hu->proto_lock); |
| 564 | + } | |
| 551 | 565 | |
| 552 | − cancel_work_sync(&hu->init_ready); | |
| 553 | − cancel_work_sync(&hu->write_work); | |
| 566 | + /* | |
| 567 | + * Unconditionally cancel write_work AFTER clearing PROTO_READY. | |
| 568 | + * This ensures that concurrent protocol timers cannot requeue | |
| 569 | + * write_work via hci_uart_tx_wakeup(), permanently preventing | |
| 570 | + * double-free races and UAFs. | |
| 571 | + */ | |
| 572 | + cancel_work_sync(&hu->write_work); | |
| 573 | + | |
| 574 | + hdev = hu->hdev; | |
| 575 | + if (hdev) | |
| 576 | + hci_uart_close(hdev); /* proto->flush is safely skipped */ | |
| 554 | 577 | |
| 578 | + if (proto_ready) { | |
| 555 | 579 | if (hdev) { |
| 556 | 580 | if (test_bit(HCI_UART_REGISTERED, &hu->flags)) |
| 557 | 581 | hci_unregister_dev(hdev); |
| 558 | − hci_free_dev(hdev); | |
| 559 | 582 | } |
| 583 | + /* Close protocol before freeing hdev (intrinsically purges queues) */ | |
| 560 | 584 | hu->proto->close(hu); |
| 585 | + | |
| 586 | + if (hdev) | |
| 587 | + hci_free_dev(hdev); | |
| 561 | 588 | } |
| 562 | 589 | clear_bit(HCI_UART_PROTO_SET, &hu->flags); |
| 563 | 590 | |
| @@ -625,11 +652,12 @@ static void hci_uart_tty_receive(struct tty_struct *tty, const u8 *data, · 61 unmodified lines | ||
| 625 | 652 | * tty caller |
| 626 | 653 | */ |
| 627 | 654 | hu->proto->recv(hu, data, count); |
| 628 | − percpu_up_read(&hu->proto_lock); | |
| 629 | 655 | |
| 630 | 656 | if (hu->hdev) |
| 631 | 657 | hu->hdev->stat.byte_rx += count; |
| 632 | 658 | |
| 659 | + percpu_up_read(&hu->proto_lock); | |
| 660 | + | |
| 633 | 661 | tty_unthrottle(tty); |
| 634 | 662 | } |
| 635 | 663 | |
| @@ -695,6 +723,10 @@ static int hci_uart_register_dev(struct hci_uart *hu) · 59 unmodified lines | ||
| 695 | 723 | percpu_down_write(&hu->proto_lock); |
| 696 | 724 | clear_bit(HCI_UART_PROTO_INIT, &hu->flags); |
| 697 | 725 | percpu_up_write(&hu->proto_lock); |
| 726 | + /* Cancel work after clearing flags */ | |
| 727 | + cancel_work_sync(&hu->write_work); | |
| 728 | + | |
| 729 | + /* Close protocol before freeing hdev */ | |
| 698 | 730 | hu->proto->close(hu); |
| 699 | 731 | hu->hdev = NULL; |
| 700 | 732 | hci_free_dev(hdev); |
| @@ -48,13 +48,12 @@ · 47 unmodified lines | ||
| 48 | 48 | #define HCI_MAX_IBS_SIZE 10 |
| 49 | 49 | |
| 50 | 50 | #define IBS_WAKE_RETRANS_TIMEOUT_MS 100 |
| 51 | −#define IBS_BTSOC_TX_IDLE_TIMEOUT_MS 200 | |
| 51 | +#define IBS_BTSOC_TX_IDLE_TIMEOUT msecs_to_jiffies(200) | |
| 52 | 52 | #define IBS_HOST_TX_IDLE_TIMEOUT_MS 2000 |
| 53 | −#define CMD_TRANS_TIMEOUT_MS 100 | |
| 54 | −#define MEMDUMP_TIMEOUT_MS 8000 | |
| 55 | −#define IBS_DISABLE_SSR_TIMEOUT_MS \ | |
| 56 | − (MEMDUMP_TIMEOUT_MS + FW_DOWNLOAD_TIMEOUT_MS) | |
| 57 | −#define FW_DOWNLOAD_TIMEOUT_MS 3000 | |
| 53 | +#define CMD_TRANS_TIMEOUT msecs_to_jiffies(100) | |
| 54 | +#define MEMDUMP_TIMEOUT msecs_to_jiffies(8000) | |
| 55 | +#define FW_DOWNLOAD_TIMEOUT msecs_to_jiffies(3000) | |
| 56 | +#define IBS_DISABLE_SSR_TIMEOUT (MEMDUMP_TIMEOUT + FW_DOWNLOAD_TIMEOUT) | |
| 58 | 57 | |
| 59 | 58 | /* susclk rate */ |
| 60 | 59 | #define SUSCLK_RATE_32KHZ 32768 |
| @@ -1096,7 +1095,7 @@ static void qca_controller_memdump(struct work_struct *work) · 1035 unmodified lines | ||
| 1096 | 1095 | |
| 1097 | 1096 | queue_delayed_work(qca->workqueue, |
| 1098 | 1097 | &qca->ctrl_memdump_timeout, |
| 1099 | − msecs_to_jiffies(MEMDUMP_TIMEOUT_MS)); | |
| 1098 | + MEMDUMP_TIMEOUT); | |
| 1100 | 1099 | skb_pull(skb, sizeof(qca_memdump->ram_dump_size)); |
| 1101 | 1100 | qca_memdump->current_seq_no = 0; |
| 1102 | 1101 | qca_memdump->received_dump = 0; |
| @@ -1369,7 +1368,7 @@ static int qca_set_baudrate(struct hci_dev *hdev, uint8_t baudrate) · 266 unmodified lines | ||
| 1369 | 1368 | |
| 1370 | 1369 | if (hu->serdev) |
| 1371 | 1370 | serdev_device_wait_until_sent(hu->serdev, |
| 1372 | − msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS)); | |
| 1371 | + CMD_TRANS_TIMEOUT); | |
| 1373 | 1372 | |
| 1374 | 1373 | /* Give the controller time to process the request */ |
| 1375 | 1374 | switch (qca_soc_type(hu)) { |
| @@ -1401,8 +1400,8 @@ static inline void host_set_baudrate(struct hci_uart *hu, unsigned int speed) · 25 unmodified lines | ||
| 1401 | 1400 | |
| 1402 | 1401 | static int qca_send_power_pulse(struct hci_uart *hu, bool on) |
| 1403 | 1402 | { |
| 1403 | + int timeout = CMD_TRANS_TIMEOUT; | |
| 1404 | 1404 | int ret; |
| 1405 | − int timeout = msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS); | |
| 1406 | 1405 | u8 cmd = on ? QCA_WCN3990_POWERON_PULSE : QCA_WCN3990_POWEROFF_PULSE; |
| 1407 | 1406 | |
| 1408 | 1407 | /* These power pulses are single byte command which are sent |
| @@ -1607,7 +1606,7 @@ static void qca_wait_for_dump_collection(struct hci_dev *hdev) · 198 unmodified lines | ||
| 1607 | 1606 | struct qca_data *qca = hu->priv; |
| 1608 | 1607 | |
| 1609 | 1608 | wait_on_bit_timeout(&qca->flags, QCA_MEMDUMP_COLLECTION, |
| 1610 | − TASK_UNINTERRUPTIBLE, MEMDUMP_TIMEOUT_MS); | |
| 1609 | + TASK_UNINTERRUPTIBLE, MEMDUMP_TIMEOUT); | |
| 1611 | 1610 | |
| 1612 | 1611 | clear_bit(QCA_MEMDUMP_COLLECTION, &qca->flags); |
| 1613 | 1612 | } |
| @@ -1681,8 +1680,8 @@ static void qca_hw_error(struct hci_dev *hdev, u8 code) · 67 unmodified lines | ||
| 1681 | 1680 | mod_timer(&qca->tx_idle_timer, jiffies + |
| 1682 | 1681 | msecs_to_jiffies(qca->tx_idle_delay)); |
| 1683 | 1682 | |
| 1684 | − /* Controller reset completion time is 50ms */ | |
| 1685 | − msleep(50); | |
| 1683 | + /* Wait for the controller to load the rampatch and NVM. */ | |
| 1684 | + msleep(100); | |
| 1686 | 1685 | |
| 1687 | 1686 | clear_bit(QCA_SSR_TRIGGERED, &qca->flags); |
| 1688 | 1687 | clear_bit(QCA_IBS_DISABLED, &qca->flags); |
| @@ -2591,7 +2590,7 @@ static void qca_serdev_remove(struct serdev_device *serdev) · 902 unmodified lines | ||
| 2591 | 2590 | static void qca_serdev_shutdown(struct serdev_device *serdev) |
| 2592 | 2591 | { |
| 2593 | 2592 | int ret; |
| 2594 | − int timeout = msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS); | |
| 2593 | + int timeout = CMD_TRANS_TIMEOUT; | |
| 2595 | 2594 | struct qca_serdev *qcadev = serdev_device_get_drvdata(serdev); |
| 2596 | 2595 | struct hci_uart *hu = &qcadev->serdev_hu; |
| 2597 | 2596 | struct hci_dev *hdev = hu->hdev; |
| @@ -2648,7 +2647,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 50 unmodified lines | ||
| 2648 | 2647 | bool tx_pending = false; |
| 2649 | 2648 | int ret = 0; |
| 2650 | 2649 | u8 cmd; |
| 2651 | − u32 wait_timeout = 0; | |
| 2650 | + unsigned long wait_timeout = 0; | |
| 2652 | 2651 | |
| 2653 | 2652 | set_bit(QCA_SUSPENDING, &qca->flags); |
| 2654 | 2653 | |
| @@ -2669,15 +2668,15 @@ static int __maybe_unused qca_suspend(struct device *dev) · 14 unmodified lines | ||
| 2669 | 2668 | if (test_bit(QCA_IBS_DISABLED, &qca->flags) || |
| 2670 | 2669 | test_bit(QCA_SSR_TRIGGERED, &qca->flags)) { |
| 2671 | 2670 | wait_timeout = test_bit(QCA_SSR_TRIGGERED, &qca->flags) ? |
| 2672 | − IBS_DISABLE_SSR_TIMEOUT_MS : | |
| 2673 | − FW_DOWNLOAD_TIMEOUT_MS; | |
| 2671 | + IBS_DISABLE_SSR_TIMEOUT : | |
| 2672 | + FW_DOWNLOAD_TIMEOUT; | |
| 2674 | 2673 | |
| 2675 | 2674 | /* QCA_IBS_DISABLED flag is set to true, During FW download |
| 2676 | 2675 | * and during memory dump collection. It is reset to false, |
| 2677 | 2676 | * After FW download complete. |
| 2678 | 2677 | */ |
| 2679 | 2678 | wait_on_bit_timeout(&qca->flags, QCA_IBS_DISABLED, |
| 2680 | − TASK_UNINTERRUPTIBLE, msecs_to_jiffies(wait_timeout)); | |
| 2679 | + TASK_UNINTERRUPTIBLE, wait_timeout); | |
| 2681 | 2680 | |
| 2682 | 2681 | if (test_bit(QCA_IBS_DISABLED, &qca->flags)) { |
| 2683 | 2682 | bt_dev_err(hu->hdev, "SSR or FW download time out"); |
| @@ -2729,7 +2728,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 45 unmodified lines | ||
| 2729 | 2728 | |
| 2730 | 2729 | if (tx_pending) { |
| 2731 | 2730 | serdev_device_wait_until_sent(hu->serdev, |
| 2732 | − msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS)); | |
| 2731 | + CMD_TRANS_TIMEOUT); | |
| 2733 | 2732 | serial_clock_vote(HCI_IBS_TX_VOTE_CLOCK_OFF, hu); |
| 2734 | 2733 | } |
| 2735 | 2734 | |
| @@ -2738,7 +2737,7 @@ static int __maybe_unused qca_suspend(struct device *dev) · 2 unmodified lines | ||
| 2738 | 2737 | */ |
| 2739 | 2738 | ret = wait_event_interruptible_timeout(qca->suspend_wait_q, |
| 2740 | 2739 | qca->rx_ibs_state == HCI_IBS_RX_ASLEEP, |
| 2741 | − msecs_to_jiffies(IBS_BTSOC_TX_IDLE_TIMEOUT_MS)); | |
| 2740 | + IBS_BTSOC_TX_IDLE_TIMEOUT); | |
| 2742 | 2741 | if (ret == 0) { |
| 2743 | 2742 | ret = -ETIMEDOUT; |
| 2744 | 2743 | goto error; |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <net/bluetooth/hci_core.h> |
| 13 | 13 | |
| 14 | 14 | #define VERSION "0.1" |
| 15 | +#define VIRTBT_RX_BUF_SIZE 1000 | |
| 15 | 16 | |
| 16 | 17 | enum { |
| 17 | 18 | VIRTBT_VQ_TX, |
| @@ -33,11 +34,11 @@ static int virtbt_add_inbuf(struct virtio_bluetooth *vbt) · 15 unmodified lines | ||
| 33 | 34 | struct sk_buff *skb; |
| 34 | 35 | int err; |
| 35 | 36 | |
| 36 | − skb = alloc_skb(1000, GFP_KERNEL); | |
| 37 | + skb = alloc_skb(VIRTBT_RX_BUF_SIZE, GFP_KERNEL); | |
| 37 | 38 | if (!skb) |
| 38 | 39 | return -ENOMEM; |
| 39 | 40 | |
| 40 | − sg_init_one(sg, skb->data, 1000); | |
| 41 | + sg_init_one(sg, skb->data, VIRTBT_RX_BUF_SIZE); | |
| 41 | 42 | |
| 42 | 43 | err = virtqueue_add_inbuf(vq, sg, 1, skb, GFP_KERNEL); |
| 43 | 44 | if (err < 0) { |
| @@ -197,6 +198,7 @@ static int virtbt_shutdown_generic(struct hci_dev *hdev) · 153 unmodified lines | ||
| 197 | 198 | |
| 198 | 199 | static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb) |
| 199 | 200 | { |
| 201 | + size_t min_hdr; | |
| 200 | 202 | __u8 pkt_type; |
| 201 | 203 | |
| 202 | 204 | pkt_type = *((__u8 *) skb->data); |
| @@ -204,16 +206,32 @@ static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb) · 1 unmodified line | ||
| 204 | 206 | |
| 205 | 207 | switch (pkt_type) { |
| 206 | 208 | case HCI_EVENT_PKT: |
| 209 | + min_hdr = sizeof(struct hci_event_hdr); | |
| 210 | + break; | |
| 207 | 211 | case HCI_ACLDATA_PKT: |
| 212 | + min_hdr = sizeof(struct hci_acl_hdr); | |
| 213 | + break; | |
| 208 | 214 | case HCI_SCODATA_PKT: |
| 215 | + min_hdr = sizeof(struct hci_sco_hdr); | |
| 216 | + break; | |
| 209 | 217 | case HCI_ISODATA_PKT: |
| 210 | − hci_skb_pkt_type(skb) = pkt_type; | |
| 211 | − hci_recv_frame(vbt->hdev, skb); | |
| 218 | + min_hdr = sizeof(struct hci_iso_hdr); | |
| 212 | 219 | break; |
| 213 | 220 | default: |
| 214 | 221 | kfree_skb(skb); |
| 215 | − break; | |
| 222 | + return; | |
| 223 | + } | |
| 224 | + | |
| 225 | + if (skb->len < min_hdr) { | |
| 226 | + bt_dev_err_ratelimited(vbt->hdev, | |
| 227 | + "rx pkt_type 0x%02x payload %u < hdr %zu\n", | |
| 228 | + pkt_type, skb->len, min_hdr); | |
| 229 | + kfree_skb(skb); | |
| 230 | + return; | |
| 216 | 231 | } |
| 232 | + | |
| 233 | + hci_skb_pkt_type(skb) = pkt_type; | |
| 234 | + hci_recv_frame(vbt->hdev, skb); | |
| 217 | 235 | } |
| 218 | 236 | |
| 219 | 237 | static void virtbt_rx_work(struct work_struct *work) |
| @@ -227,8 +245,15 @@ static void virtbt_rx_work(struct work_struct *work) · 7 unmodified lines | ||
| 227 | 245 | if (!skb) |
| 228 | 246 | return; |
| 229 | 247 | |
| 230 | − skb_put(skb, len); | |
| 231 | − virtbt_rx_handle(vbt, skb); | |
| 248 | + if (!len || len > VIRTBT_RX_BUF_SIZE) { | |
| 249 | + bt_dev_err_ratelimited(vbt->hdev, | |
| 250 | + "rx reply len %u outside [1, %u]\n", | |
| 251 | + len, VIRTBT_RX_BUF_SIZE); | |
| 252 | + kfree_skb(skb); | |
| 253 | + } else { | |
| 254 | + skb_put(skb, len); | |
| 255 | + virtbt_rx_handle(vbt, skb); | |
| 256 | + } | |
| 232 | 257 | |
| 233 | 258 | if (virtbt_add_inbuf(vbt) < 0) |
| 234 | 259 | return; |
| @@ -631,6 +631,16 @@ int register_cdrom(struct gendisk *disk, struct cdrom_device_info *cdi) · 630 unmodified lines | ||
| 631 | 631 | |
| 632 | 632 | WARN_ON(!cdo->generic_packet); |
| 633 | 633 | |
| 634 | + /* | |
| 635 | + * Propagate the drive's write support to the block layer so BLKROGET | |
| 636 | + * reflects actual write capability. Drivers that use GET CONFIGURATION | |
| 637 | + * features (CDC_MRW_W, CDC_RAM) must have called | |
| 638 | + * cdrom_probe_write_features() before register_cdrom() so the mask is | |
| 639 | + * complete here. | |
| 640 | + */ | |
| 641 | + set_disk_ro(disk, !CDROM_CAN(CDC_DVD_RAM | CDC_MRW_W | CDC_RAM | | |
| 642 | + CDC_CD_RW)); | |
| 643 | + | |
| 634 | 644 | cd_dbg(CD_REG_UNREG, "drive \"/dev/%s\" registered\n", cdi->name); |
| 635 | 645 | mutex_lock(&cdrom_mutex); |
| 636 | 646 | list_add(&cdi->list, &cdrom_list); |
| @@ -742,6 +752,44 @@ static int cdrom_is_random_writable(struct cdrom_device_info *cdi, int *write) · 105 unmodified lines | ||
| 742 | 752 | return 0; |
| 743 | 753 | } |
| 744 | 754 | |
| 755 | +/* | |
| 756 | + * Probe write-related MMC features via GET CONFIGURATION and update | |
| 757 | + * cdi->mask accordingly. Drivers that populate cdi->mask from the MODE SENSE | |
| 758 | + * capabilities page (e.g. sr) should call this after those MODE SENSE bits | |
| 759 | + * have been set but before register_cdrom(), so that the full set of | |
| 760 | + * write-capability bits is known by the time register_cdrom() decides on the | |
| 761 | + * initial read-only state of the disk. | |
| 762 | + */ | |
| 763 | +void cdrom_probe_write_features(struct cdrom_device_info *cdi) | |
| 764 | +{ | |
| 765 | + int mrw, mrw_write, ram_write; | |
| 766 | + | |
| 767 | + mrw = 0; | |
| 768 | + if (!cdrom_is_mrw(cdi, &mrw_write)) | |
| 769 | + mrw = 1; | |
| 770 | + | |
| 771 | + if (CDROM_CAN(CDC_MO_DRIVE)) | |
| 772 | + ram_write = 1; | |
| 773 | + else | |
| 774 | + (void) cdrom_is_random_writable(cdi, &ram_write); | |
| 775 | + | |
| 776 | + if (mrw) | |
| 777 | + cdi->mask &= ~CDC_MRW; | |
| 778 | + else | |
| 779 | + cdi->mask |= CDC_MRW; | |
| 780 | + | |
| 781 | + if (mrw_write) | |
| 782 | + cdi->mask &= ~CDC_MRW_W; | |
| 783 | + else | |
| 784 | + cdi->mask |= CDC_MRW_W; | |
| 785 | + | |
| 786 | + if (ram_write) | |
| 787 | + cdi->mask &= ~CDC_RAM; | |
| 788 | + else | |
| 789 | + cdi->mask |= CDC_RAM; | |
| 790 | +} | |
| 791 | +EXPORT_SYMBOL(cdrom_probe_write_features); | |
| 792 | + | |
| 745 | 793 | static int cdrom_media_erasable(struct cdrom_device_info *cdi) |
| 746 | 794 | { |
| 747 | 795 | disc_information di; |
| @@ -894,33 +942,8 @@ static int cdrom_is_dvd_rw(struct cdrom_device_info *cdi) · 146 unmodified lines | ||
| 894 | 942 | */ |
| 895 | 943 | static int cdrom_open_write(struct cdrom_device_info *cdi) |
| 896 | 944 | { |
| 897 | − int mrw, mrw_write, ram_write; | |
| 898 | 945 | int ret = 1; |
| 899 | 946 | |
| 900 | − mrw = 0; | |
| 901 | − if (!cdrom_is_mrw(cdi, &mrw_write)) | |
| 902 | − mrw = 1; | |
| 903 | − | |
| 904 | − if (CDROM_CAN(CDC_MO_DRIVE)) | |
| 905 | − ram_write = 1; | |
| 906 | − else | |
| 907 | − (void) cdrom_is_random_writable(cdi, &ram_write); | |
| 908 | − | |
| 909 | − if (mrw) | |
| 910 | − cdi->mask &= ~CDC_MRW; | |
| 911 | − else | |
| 912 | − cdi->mask |= CDC_MRW; | |
| 913 | − | |
| 914 | − if (mrw_write) | |
| 915 | − cdi->mask &= ~CDC_MRW_W; | |
| 916 | − else | |
| 917 | − cdi->mask |= CDC_MRW_W; | |
| 918 | − | |
| 919 | − if (ram_write) | |
| 920 | − cdi->mask &= ~CDC_RAM; | |
| 921 | − else | |
| 922 | − cdi->mask |= CDC_RAM; | |
| 923 | − | |
| 924 | 947 | if (CDROM_CAN(CDC_MRW_W)) |
| 925 | 948 | ret = cdrom_mrw_open_write(cdi); |
| 926 | 949 | else if (CDROM_CAN(CDC_DVD_RAM)) |
| @@ -168,6 +168,10 @@ struct smi_info { · 167 unmodified lines | ||
| 168 | 168 | OEM2_DATA_AVAIL) |
| 169 | 169 | unsigned char msg_flags; |
| 170 | 170 | |
| 171 | + /* When requesting events and messages, don't do it forever. */ | |
| 172 | + unsigned int num_requests_in_a_row; | |
| 173 | + bool last_was_flag_fetch; | |
| 174 | + | |
| 171 | 175 | /* Does the BMC have an event buffer? */ |
| 172 | 176 | bool has_event_buffer; |
| 173 | 177 | |
| @@ -410,7 +414,10 @@ static void start_getting_msg_queue(struct smi_info *smi_info) · 236 unmodified lines | ||
| 410 | 414 | |
| 411 | 415 | start_new_msg(smi_info, smi_info->curr_msg->data, |
| 412 | 416 | smi_info->curr_msg->data_size); |
| 413 | − smi_info->si_state = SI_GETTING_MESSAGES; | |
| 417 | + if (smi_info->si_state != SI_GETTING_MESSAGES) { | |
| 418 | + smi_info->num_requests_in_a_row = 0; | |
| 419 | + smi_info->si_state = SI_GETTING_MESSAGES; | |
| 420 | + } | |
| 414 | 421 | } |
| 415 | 422 | |
| 416 | 423 | static void start_getting_events(struct smi_info *smi_info) |
| @@ -421,7 +428,10 @@ static void start_getting_events(struct smi_info *smi_info) · 4 unmodified lines | ||
| 421 | 428 | |
| 422 | 429 | start_new_msg(smi_info, smi_info->curr_msg->data, |
| 423 | 430 | smi_info->curr_msg->data_size); |
| 424 | − smi_info->si_state = SI_GETTING_EVENTS; | |
| 431 | + if (smi_info->si_state != SI_GETTING_EVENTS) { | |
| 432 | + smi_info->num_requests_in_a_row = 0; | |
| 433 | + smi_info->si_state = SI_GETTING_EVENTS; | |
| 434 | + } | |
| 425 | 435 | } |
| 426 | 436 | |
| 427 | 437 | /* |
| @@ -487,15 +497,19 @@ retry: · 59 unmodified lines | ||
| 487 | 497 | } else if (smi_info->msg_flags & RECEIVE_MSG_AVAIL) { |
| 488 | 498 | /* Messages available. */ |
| 489 | 499 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); |
| 490 | − if (!smi_info->curr_msg) | |
| 500 | + if (!smi_info->curr_msg) { | |
| 501 | + smi_info->si_state = SI_NORMAL; | |
| 491 | 502 | return; |
| 503 | + } | |
| 492 | 504 | |
| 493 | 505 | start_getting_msg_queue(smi_info); |
| 494 | 506 | } else if (smi_info->msg_flags & EVENT_MSG_BUFFER_FULL) { |
| 495 | 507 | /* Events available. */ |
| 496 | 508 | smi_info->curr_msg = alloc_msg_handle_irq(smi_info); |
| 497 | − if (!smi_info->curr_msg) | |
| 509 | + if (!smi_info->curr_msg) { | |
| 510 | + smi_info->si_state = SI_NORMAL; | |
| 498 | 511 | return; |
| 512 | + } | |
| 499 | 513 | |
| 500 | 514 | start_getting_events(smi_info); |
| 501 | 515 | } else if (smi_info->msg_flags & OEM_DATA_AVAIL && |
| @@ -595,6 +609,7 @@ static void handle_transaction_done(struct smi_info *smi_info) · 93 unmodified lines | ||
| 595 | 609 | smi_info->si_state = SI_NORMAL; |
| 596 | 610 | } else { |
| 597 | 611 | smi_info->msg_flags = msg[3]; |
| 612 | + smi_info->last_was_flag_fetch = true; | |
| 598 | 613 | handle_flags(smi_info); |
| 599 | 614 | } |
| 600 | 615 | break; |
| @@ -630,7 +645,13 @@ static void handle_transaction_done(struct smi_info *smi_info) · 29 unmodified lines | ||
| 630 | 645 | */ |
| 631 | 646 | msg = smi_info->curr_msg; |
| 632 | 647 | smi_info->curr_msg = NULL; |
| 633 | − if (msg->rsp[2] != 0) { | |
| 648 | + /* | |
| 649 | + * It appears some BMCs, with no event data, return no | |
| 650 | + * data in the message and not a 0x80 error as the | |
| 651 | + * spec says they should. Shut down processing if | |
| 652 | + * the data is not the right length. | |
| 653 | + */ | |
| 654 | + if (msg->rsp[2] != 0 || msg->rsp_size != 19) { | |
| 634 | 655 | /* Error getting event, probably done. */ |
| 635 | 656 | msg->done(msg); |
| 636 | 657 | |
| @@ -640,6 +661,11 @@ static void handle_transaction_done(struct smi_info *smi_info) · 3 unmodified lines | ||
| 640 | 661 | } else { |
| 641 | 662 | smi_inc_stat(smi_info, events); |
| 642 | 663 | |
| 664 | + smi_info->num_requests_in_a_row++; | |
| 665 | + if (smi_info->num_requests_in_a_row > 10) | |
| 666 | + /* Stop if we do this too many times. */ | |
| 667 | + smi_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; | |
| 668 | + | |
| 643 | 669 | /* |
| 644 | 670 | * Do this before we deliver the message |
| 645 | 671 | * because delivering the message releases the |
| @@ -678,6 +704,11 @@ static void handle_transaction_done(struct smi_info *smi_info) · 32 unmodified lines | ||
| 678 | 704 | } else { |
| 679 | 705 | smi_inc_stat(smi_info, incoming_messages); |
| 680 | 706 | |
| 707 | + smi_info->num_requests_in_a_row++; | |
| 708 | + if (smi_info->num_requests_in_a_row > 10) | |
| 709 | + /* Stop if we do this too many times. */ | |
| 710 | + smi_info->msg_flags &= ~RECEIVE_MSG_AVAIL; | |
| 711 | + | |
| 681 | 712 | /* |
| 682 | 713 | * Do this before we deliver the message |
| 683 | 714 | * because delivering the message releases the |
| @@ -819,6 +850,26 @@ restart: · 135 unmodified lines | ||
| 819 | 850 | goto out; |
| 820 | 851 | } |
| 821 | 852 | |
| 853 | + /* | |
| 854 | + * If we are currently idle, or if the last thing that was | |
| 855 | + * done was a flag fetch and there is a message pending, try | |
| 856 | + * to start the next message. | |
| 857 | + * | |
| 858 | + * We do the waiting message check to avoid a stuck flag | |
| 859 | + * completely wedging the driver. Let a message through | |
| 860 | + * in between flag operations if that happens. | |
| 861 | + */ | |
| 862 | + if (si_sm_result == SI_SM_IDLE || | |
| 863 | + (si_sm_result == SI_SM_ATTN && smi_info->waiting_msg && | |
| 864 | + smi_info->last_was_flag_fetch)) { | |
| 865 | + smi_info->last_was_flag_fetch = false; | |
| 866 | + smi_inc_stat(smi_info, idles); | |
| 867 | + | |
| 868 | + si_sm_result = start_next_msg(smi_info); | |
| 869 | + if (si_sm_result != SI_SM_IDLE) | |
| 870 | + goto restart; | |
| 871 | + } | |
| 872 | + | |
| 822 | 873 | /* |
| 823 | 874 | * We prefer handling attn over new messages. But don't do |
| 824 | 875 | * this if there is not yet an upper layer to handle anything. |
| @@ -846,15 +897,6 @@ restart: · 21 unmodified lines | ||
| 846 | 897 | } |
| 847 | 898 | } |
| 848 | 899 | |
| 849 | − /* If we are currently idle, try to start the next message. */ | |
| 850 | − if (si_sm_result == SI_SM_IDLE) { | |
| 851 | − smi_inc_stat(smi_info, idles); | |
| 852 | − | |
| 853 | − si_sm_result = start_next_msg(smi_info); | |
| 854 | − if (si_sm_result != SI_SM_IDLE) | |
| 855 | − goto restart; | |
| 856 | − } | |
| 857 | − | |
| 858 | 900 | if ((si_sm_result == SI_SM_IDLE) |
| 859 | 901 | && (atomic_read(&smi_info->req_events))) { |
| 860 | 902 | /* |
| @@ -225,6 +225,9 @@ struct ssif_info { · 224 unmodified lines | ||
| 225 | 225 | bool has_event_buffer; |
| 226 | 226 | bool supports_alert; |
| 227 | 227 | |
| 228 | + /* When requesting events and messages, don't do it forever. */ | |
| 229 | + unsigned int num_requests_in_a_row; | |
| 230 | + | |
| 228 | 231 | /* |
| 229 | 232 | * Used to tell what we should do with alerts. If we are |
| 230 | 233 | * waiting on a response, read the data immediately. |
| @@ -413,7 +416,10 @@ static void start_event_fetch(struct ssif_info *ssif_info, unsigned long *flags) · 182 unmodified lines | ||
| 413 | 416 | } |
| 414 | 417 | |
| 415 | 418 | ssif_info->curr_msg = msg; |
| 416 | − ssif_info->ssif_state = SSIF_GETTING_EVENTS; | |
| 419 | + if (ssif_info->ssif_state != SSIF_GETTING_EVENTS) { | |
| 420 | + ssif_info->num_requests_in_a_row = 0; | |
| 421 | + ssif_info->ssif_state = SSIF_GETTING_EVENTS; | |
| 422 | + } | |
| 417 | 423 | ipmi_ssif_unlock_cond(ssif_info, flags); |
| 418 | 424 | |
| 419 | 425 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); |
| @@ -436,7 +442,10 @@ static void start_recv_msg_fetch(struct ssif_info *ssif_info, · 16 unmodified lines | ||
| 436 | 442 | } |
| 437 | 443 | |
| 438 | 444 | ssif_info->curr_msg = msg; |
| 439 | − ssif_info->ssif_state = SSIF_GETTING_MESSAGES; | |
| 445 | + if (ssif_info->ssif_state != SSIF_GETTING_MESSAGES) { | |
| 446 | + ssif_info->num_requests_in_a_row = 0; | |
| 447 | + ssif_info->ssif_state = SSIF_GETTING_MESSAGES; | |
| 448 | + } | |
| 440 | 449 | ipmi_ssif_unlock_cond(ssif_info, flags); |
| 441 | 450 | |
| 442 | 451 | msg->data[0] = (IPMI_NETFN_APP_REQUEST << 2); |
| @@ -843,6 +852,11 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result, · 400 unmodified lines | ||
| 843 | 852 | ssif_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; |
| 844 | 853 | handle_flags(ssif_info, flags); |
| 845 | 854 | } else { |
| 855 | + ssif_info->num_requests_in_a_row++; | |
| 856 | + if (ssif_info->num_requests_in_a_row > 10) | |
| 857 | + /* Stop if we do this too many times. */ | |
| 858 | + ssif_info->msg_flags &= ~EVENT_MSG_BUFFER_FULL; | |
| 859 | + | |
| 846 | 860 | handle_flags(ssif_info, flags); |
| 847 | 861 | ssif_inc_stat(ssif_info, events); |
| 848 | 862 | deliver_recv_msg(ssif_info, msg); |
| @@ -876,6 +890,11 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result, · 27 unmodified lines | ||
| 876 | 890 | ssif_info->msg_flags &= ~RECEIVE_MSG_AVAIL; |
| 877 | 891 | handle_flags(ssif_info, flags); |
| 878 | 892 | } else { |
| 893 | + ssif_info->num_requests_in_a_row++; | |
| 894 | + if (ssif_info->num_requests_in_a_row > 10) | |
| 895 | + /* Stop if we do this too many times. */ | |
| 896 | + ssif_info->msg_flags &= ~RECEIVE_MSG_AVAIL; | |
| 897 | + | |
| 879 | 898 | ssif_inc_stat(ssif_info, incoming_messages); |
| 880 | 899 | handle_flags(ssif_info, flags); |
| 881 | 900 | deliver_recv_msg(ssif_info, msg); |
| @@ -1886,6 +1905,7 @@ static int ssif_probe(struct i2c_client *client) · 1004 unmodified lines | ||
| 1886 | 1905 | "kssif%4.4x", thread_num); |
| 1887 | 1906 | if (IS_ERR(ssif_info->thread)) { |
| 1888 | 1907 | rv = PTR_ERR(ssif_info->thread); |
| 1908 | + ssif_info->thread = NULL; | |
| 1889 | 1909 | dev_notice(&ssif_info->client->dev, |
| 1890 | 1910 | "Could not start kernel thread: error %d\n", |
| 1891 | 1911 | rv); |
| @@ -110,6 +110,7 @@ struct eqc_match_data { · 109 unmodified lines | ||
| 110 | 110 | |
| 111 | 111 | const char *reset_auxdev_name; |
| 112 | 112 | const char *pinctrl_auxdev_name; |
| 113 | + const char *eth_phy_auxdev_name; | |
| 113 | 114 | |
| 114 | 115 | unsigned int early_clk_count; |
| 115 | 116 | }; |
| @@ -321,38 +322,18 @@ static void eqc_probe_init_fixed_factors(struct device *dev, · 205 unmodified lines | ||
| 321 | 322 | } |
| 322 | 323 | } |
| 323 | 324 | |
| 324 | −static void eqc_auxdev_release(struct device *dev) | |
| 325 | −{ | |
| 326 | − struct auxiliary_device *adev = to_auxiliary_dev(dev); | |
| 327 | − | |
| 328 | − kfree(adev); | |
| 329 | −} | |
| 330 | − | |
| 331 | −static int eqc_auxdev_create(struct device *dev, void __iomem *base, | |
| 332 | − const char *name, u32 id) | |
| 325 | +static void eqc_auxdev_create_optional(struct device *dev, void __iomem *base, | |
| 326 | + const char *name) | |
| 333 | 327 | { |
| 334 | 328 | struct auxiliary_device *adev; |
| 335 | − int ret; | |
| 336 | − | |
| 337 | − adev = kzalloc_obj(*adev); | |
| 338 | − if (!adev) | |
| 339 | − return -ENOMEM; | |
| 340 | − | |
| 341 | − adev->name = name; | |
| 342 | − adev->dev.parent = dev; | |
| 343 | − adev->dev.platform_data = (void __force *)base; | |
| 344 | − adev->dev.release = eqc_auxdev_release; | |
| 345 | − adev->id = id; | |
| 346 | 329 | |
| 347 | − ret = auxiliary_device_init(adev); | |
| 348 | − if (ret) | |
| 349 | − return ret; | |
| 350 | − | |
| 351 | − ret = auxiliary_device_add(adev); | |
| 352 | − if (ret) | |
| 353 | − auxiliary_device_uninit(adev); | |
| 354 | − | |
| 355 | − return ret; | |
| 330 | + if (name) { | |
| 331 | + adev = devm_auxiliary_device_create(dev, name, | |
| 332 | + (void __force *)base); | |
| 333 | + if (!adev) | |
| 334 | + dev_warn(dev, "failed creating auxiliary device %s.%s\n", | |
| 335 | + KBUILD_MODNAME, name); | |
| 336 | + } | |
| 356 | 337 | } |
| 357 | 338 | |
| 358 | 339 | static int eqc_probe(struct platform_device *pdev) |
| @@ -364,7 +345,6 @@ static int eqc_probe(struct platform_device *pdev) · 5 unmodified lines | ||
| 364 | 345 | unsigned int i, clk_count; |
| 365 | 346 | struct resource *res; |
| 366 | 347 | void __iomem *base; |
| 367 | − int ret; | |
| 368 | 348 | |
| 369 | 349 | data = device_get_match_data(dev); |
| 370 | 350 | if (!data) |
| @@ -378,21 +358,10 @@ static int eqc_probe(struct platform_device *pdev) · 7 unmodified lines | ||
| 378 | 358 | if (!base) |
| 379 | 359 | return -ENOMEM; |
| 380 | 360 | |
| 381 | − /* Init optional reset auxiliary device. */ | |
| 382 | − if (data->reset_auxdev_name) { | |
| 383 | − ret = eqc_auxdev_create(dev, base, data->reset_auxdev_name, 0); | |
| 384 | − if (ret) | |
| 385 | − dev_warn(dev, "failed creating auxiliary device %s.%s: %d\n", | |
| 386 | − KBUILD_MODNAME, data->reset_auxdev_name, ret); | |
| 387 | − } | |
| 388 | − | |
| 389 | − /* Init optional pinctrl auxiliary device. */ | |
| 390 | − if (data->pinctrl_auxdev_name) { | |
| 391 | − ret = eqc_auxdev_create(dev, base, data->pinctrl_auxdev_name, 0); | |
| 392 | − if (ret) | |
| 393 | − dev_warn(dev, "failed creating auxiliary device %s.%s: %d\n", | |
| 394 | − KBUILD_MODNAME, data->pinctrl_auxdev_name, ret); | |
| 395 | − } | |
| 361 | + /* Init optional auxiliary devices. */ | |
| 362 | + eqc_auxdev_create_optional(dev, base, data->reset_auxdev_name); | |
| 363 | + eqc_auxdev_create_optional(dev, base, data->pinctrl_auxdev_name); | |
| 364 | + eqc_auxdev_create_optional(dev, base, data->eth_phy_auxdev_name); | |
| 396 | 365 | |
| 397 | 366 | if (data->pll_count + data->div_count + data->fixed_factor_count == 0) |
| 398 | 367 | return 0; /* Zero clocks, we are done. */ |
| @@ -553,6 +522,7 @@ static const struct eqc_match_data eqc_eyeq5_match_data = { · 154 unmodified lines | ||
| 553 | 522 | |
| 554 | 523 | .reset_auxdev_name = "reset", |
| 555 | 524 | .pinctrl_auxdev_name = "pinctrl", |
| 525 | + .eth_phy_auxdev_name = "phy", | |
| 556 | 526 | |
| 557 | 527 | .early_clk_count = ARRAY_SIZE(eqc_eyeq5_early_plls) + |
| 558 | 528 | ARRAY_SIZE(eqc_eyeq5_early_fixed_factors), |
| @@ -153,7 +153,7 @@ static int rk808_clkout_probe(struct platform_device *pdev) · 152 unmodified lines | ||
| 153 | 153 | struct rk808_clkout *rk808_clkout; |
| 154 | 154 | int ret; |
| 155 | 155 | |
| 156 | − dev->of_node = pdev->dev.parent->of_node; | |
| 156 | + device_set_of_node_from_dev(dev, dev->parent); | |
| 157 | 157 | |
| 158 | 158 | rk808_clkout = devm_kzalloc(dev, |
| 159 | 159 | sizeof(*rk808_clkout), GFP_KERNEL); |
| @@ -846,7 +846,7 @@ static const struct clk_parent_data top_parents[] = { · 845 unmodified lines | ||
| 846 | 846 | CCU_PARENT_HW(pll6_d3), |
| 847 | 847 | }; |
| 848 | 848 | CCU_MUX_DIV_GATE_FC_DEFINE(top_dclk, top_parents, APMU_TOP_DCLK_CTRL, 5, 3, |
| 849 | − BIT(8), 2, 3, BIT(1), 0); | |
| 849 | + BIT(8), 2, 3, BIT(1), CLK_IS_CRITICAL); | |
| 850 | 850 | |
| 851 | 851 | static const struct clk_parent_data ucie_parents[] = { |
| 852 | 852 | CCU_PARENT_HW(pll1_d8_307p2), |
| @@ -274,6 +274,7 @@ static int waveform_ai_cmdtest(struct comedi_device *dev, · 273 unmodified lines | ||
| 274 | 274 | /* Step 2a : make sure trigger sources are unique */ |
| 275 | 275 | |
| 276 | 276 | err |= comedi_check_trigger_is_unique(cmd->convert_src); |
| 277 | + err |= comedi_check_trigger_is_unique(cmd->scan_begin_src); | |
| 277 | 278 | err |= comedi_check_trigger_is_unique(cmd->stop_src); |
| 278 | 279 | |
| 279 | 280 | /* Step 2b : and mutually compatible */ |
| @@ -324,10 +325,10 @@ static int waveform_ai_cmdtest(struct comedi_device *dev, · 44 unmodified lines | ||
| 324 | 325 | arg = min(arg, |
| 325 | 326 | rounddown(UINT_MAX, (unsigned int)NSEC_PER_USEC)); |
| 326 | 327 | arg = NSEC_PER_USEC * DIV_ROUND_CLOSEST(arg, NSEC_PER_USEC); |
| 327 | − if (cmd->scan_begin_arg == TRIG_TIMER) { | |
| 328 | + if (cmd->scan_begin_src == TRIG_TIMER) { | |
| 328 | 329 | /* limit convert_arg to keep scan_begin_arg in range */ |
| 329 | 330 | limit = UINT_MAX / cmd->scan_end_arg; |
| 330 | − limit = rounddown(limit, (unsigned int)NSEC_PER_SEC); | |
| 331 | + limit = rounddown(limit, (unsigned int)NSEC_PER_USEC); | |
| 331 | 332 | arg = min(arg, limit); |
| 332 | 333 | } |
| 333 | 334 | err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg); |
| @@ -124,7 +124,8 @@ struct counter_device *counter_alloc(size_t sizeof_priv) · 123 unmodified lines | ||
| 124 | 124 | |
| 125 | 125 | err_dev_set_name: |
| 126 | 126 | |
| 127 | − counter_chrdev_remove(counter); | |
| 127 | + put_device(dev); | |
| 128 | + return NULL; | |
| 128 | 129 | err_chrdev_add: |
| 129 | 130 | |
| 130 | 131 | ida_free(&counter_ida, dev->id); |
| @@ -70,18 +70,6 @@ config X86_AMD_PSTATE_DEFAULT_MODE · 69 unmodified lines | ||
| 70 | 70 | For details, take a look at: |
| 71 | 71 | <file:Documentation/admin-guide/pm/amd-pstate.rst>. |
| 72 | 72 | |
| 73 | −config X86_AMD_PSTATE_DYNAMIC_EPP | |
| 74 | − bool "AMD Processor P-State dynamic EPP support" | |
| 75 | − depends on X86_AMD_PSTATE | |
| 76 | − default n | |
| 77 | − help | |
| 78 | − Allow the kernel to dynamically change the energy performance | |
| 79 | − value from events like ACPI platform profile and AC adapter plug | |
| 80 | − events. | |
| 81 | − | |
| 82 | − This feature can also be changed at runtime, this configuration | |
| 83 | − option only sets the kernel default value behavior. | |
| 84 | − | |
| 85 | 73 | config X86_AMD_PSTATE_UT |
| 86 | 74 | tristate "selftest for AMD Processor P-State driver" |
| 87 | 75 | depends on X86 && ACPI_PROCESSOR |
| @@ -274,20 +274,21 @@ static int amd_pstate_set_mode(enum amd_pstate_mode mode) · 273 unmodified lines | ||
| 274 | 274 | |
| 275 | 275 | static int amd_pstate_ut_epp(u32 index) |
| 276 | 276 | { |
| 277 | − struct cpufreq_policy *policy __free(put_cpufreq_policy) = NULL; | |
| 278 | − char *buf __free(cleanup_page) = NULL; | |
| 279 | 277 | static const char * const epp_strings[] = { |
| 280 | − "performance", | |
| 281 | − "balance_performance", | |
| 282 | − "balance_power", | |
| 283 | 278 | "power", |
| 279 | + "balance_power", | |
| 280 | + "balance_performance", | |
| 281 | + "performance", | |
| 284 | 282 | }; |
| 285 | − struct amd_cpudata *cpudata; | |
| 283 | + char *buf __free(cleanup_page) = NULL; | |
| 284 | + struct cpufreq_policy *policy = NULL; | |
| 286 | 285 | enum amd_pstate_mode orig_mode; |
| 286 | + struct amd_cpudata *cpudata; | |
| 287 | + unsigned long orig_policy; | |
| 287 | 288 | bool orig_dynamic_epp; |
| 288 | 289 | int ret, cpu = 0; |
| 289 | − int i; | |
| 290 | 290 | u16 epp; |
| 291 | + int i; | |
| 291 | 292 | |
| 292 | 293 | policy = cpufreq_cpu_get(cpu); |
| 293 | 294 | if (!policy) |
| @@ -297,11 +298,9 @@ static int amd_pstate_ut_epp(u32 index) · 3 unmodified lines | ||
| 297 | 298 | orig_mode = amd_pstate_get_status(); |
| 298 | 299 | orig_dynamic_epp = cpudata->dynamic_epp; |
| 299 | 300 | |
| 300 | − /* disable dynamic EPP before running test */ | |
| 301 | − if (cpudata->dynamic_epp) { | |
| 302 | − pr_debug("Dynamic EPP is enabled, disabling it\n"); | |
| 303 | − amd_pstate_clear_dynamic_epp(policy); | |
| 304 | − } | |
| 301 | + /* Drop reference before potential driver change. */ | |
| 302 | + cpufreq_cpu_put(policy); | |
| 303 | + policy = NULL; | |
| 305 | 304 | |
| 306 | 305 | buf = (char *)__get_free_page(GFP_KERNEL); |
| 307 | 306 | if (!buf) |
| @@ -311,6 +310,27 @@ static int amd_pstate_ut_epp(u32 index) · 3 unmodified lines | ||
| 311 | 310 | if (ret) |
| 312 | 311 | goto out; |
| 313 | 312 | |
| 313 | + policy = cpufreq_cpu_get(cpu); | |
| 314 | + if (!policy) { | |
| 315 | + ret = -ENODEV; | |
| 316 | + goto out; | |
| 317 | + } | |
| 318 | + | |
| 319 | + down_write(&policy->rwsem); | |
| 320 | + cpudata = policy->driver_data; | |
| 321 | + orig_policy = cpudata->policy; | |
| 322 | + cpudata->policy = CPUFREQ_POLICY_POWERSAVE; | |
| 323 | + | |
| 324 | + /* | |
| 325 | + * Disable dynamic EPP before running test. If "orig_dynamic_epp" is | |
| 326 | + * true, the driver will do a redundant switch at the end and there | |
| 327 | + * is no need for enabling it again at the end of the test. | |
| 328 | + */ | |
| 329 | + if (cpudata->dynamic_epp) { | |
| 330 | + pr_debug("Dynamic EPP is enabled, disabling it\n"); | |
| 331 | + amd_pstate_clear_dynamic_epp(policy); | |
| 332 | + } | |
| 333 | + | |
| 314 | 334 | for (epp = 0; epp <= U8_MAX; epp++) { |
| 315 | 335 | u8 val; |
| 316 | 336 | |
| @@ -358,6 +378,12 @@ static int amd_pstate_ut_epp(u32 index) · 41 unmodified lines | ||
| 358 | 378 | ret = 0; |
| 359 | 379 | |
| 360 | 380 | out: |
| 381 | + if (policy) { | |
| 382 | + cpudata->policy = orig_policy; | |
| 383 | + up_write(&policy->rwsem); | |
| 384 | + cpufreq_cpu_put(policy); | |
| 385 | + } | |
| 386 | + | |
| 361 | 387 | if (orig_dynamic_epp) { |
| 362 | 388 | int ret2; |
| 363 | 389 | |
| @@ -87,11 +87,7 @@ static struct cpufreq_driver amd_pstate_driver; · 86 unmodified lines | ||
| 87 | 87 | static struct cpufreq_driver amd_pstate_epp_driver; |
| 88 | 88 | static int cppc_state = AMD_PSTATE_UNDEFINED; |
| 89 | 89 | static bool amd_pstate_prefcore = true; |
| 90 | −#ifdef CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP | |
| 91 | −static bool dynamic_epp = CONFIG_X86_AMD_PSTATE_DYNAMIC_EPP; | |
| 92 | −#else | |
| 93 | 90 | static bool dynamic_epp; |
| 94 | −#endif | |
| 95 | 91 | static struct quirk_entry *quirks; |
| 96 | 92 | |
| 97 | 93 | /* |
| @@ -1291,6 +1287,8 @@ static int amd_pstate_set_dynamic_epp(struct cpufreq_policy *policy) · 1193 unmodified lines | ||
| 1291 | 1287 | return ret; |
| 1292 | 1288 | |
| 1293 | 1289 | cpudata->profile_name = kasprintf(GFP_KERNEL, "amd-pstate-epp-cpu%d", cpudata->cpu); |
| 1290 | + if (!cpudata->profile_name) | |
| 1291 | + return -ENOMEM; | |
| 1294 | 1292 | |
| 1295 | 1293 | cpudata->ppdev = platform_profile_register(get_cpu_device(policy->cpu), |
| 1296 | 1294 | cpudata->profile_name, |
| @@ -1427,7 +1425,7 @@ ssize_t store_energy_performance_preference(struct cpufreq_policy *policy, · 130 unmodified lines | ||
| 1427 | 1425 | if (ret) |
| 1428 | 1426 | epp = epp_values[ret]; |
| 1429 | 1427 | else |
| 1430 | − epp = amd_pstate_get_balanced_epp(policy); | |
| 1428 | + epp = cpudata->epp_default_dc; | |
| 1431 | 1429 | } |
| 1432 | 1430 | |
| 1433 | 1431 | if (cpudata->policy == CPUFREQ_POLICY_PERFORMANCE) { |
| @@ -1707,6 +1705,8 @@ static int amd_pstate_change_driver_mode(int mode) · 273 unmodified lines | ||
| 1707 | 1705 | { |
| 1708 | 1706 | int ret; |
| 1709 | 1707 | |
| 1708 | + lockdep_assert_held(&amd_pstate_driver_lock); | |
| 1709 | + | |
| 1710 | 1710 | ret = amd_pstate_unregister_driver(0); |
| 1711 | 1711 | if (ret) |
| 1712 | 1712 | return ret; |
| @@ -1821,8 +1821,16 @@ static ssize_t dynamic_epp_store(struct device *a, struct device_attribute *b, · 108 unmodified lines | ||
| 1821 | 1821 | if (ret) |
| 1822 | 1822 | return ret; |
| 1823 | 1823 | |
| 1824 | − if (dynamic_epp == enabled) | |
| 1824 | + guard(mutex)(&amd_pstate_driver_lock); | |
| 1825 | + | |
| 1826 | + if (cppc_state != AMD_PSTATE_ACTIVE) { | |
| 1827 | + pr_debug("dynamic_epp can only be toggled in active mode\n"); | |
| 1825 | 1828 | return -EINVAL; |
| 1829 | + } | |
| 1830 | + | |
| 1831 | + /* Nothing to do */ | |
| 1832 | + if (dynamic_epp == enabled) | |
| 1833 | + return count; | |
| 1826 | 1834 | |
| 1827 | 1835 | /* reinitialize with desired dynamic EPP value */ |
| 1828 | 1836 | dynamic_epp = enabled; |
| @@ -1942,7 +1950,7 @@ static int amd_pstate_epp_cpu_init(struct cpufreq_policy *policy) · 113 unmodified lines | ||
| 1942 | 1950 | if (dynamic_epp) |
| 1943 | 1951 | ret = amd_pstate_set_dynamic_epp(policy); |
| 1944 | 1952 | else |
| 1945 | − ret = amd_pstate_set_epp(policy, amd_pstate_get_balanced_epp(policy)); | |
| 1953 | + ret = amd_pstate_set_epp(policy, cpudata->epp_default_dc); | |
| 1946 | 1954 | if (ret) |
| 1947 | 1955 | goto free_cpudata1; |
| 1948 | 1956 | |
| @@ -1970,12 +1978,13 @@ static void amd_pstate_epp_cpu_exit(struct cpufreq_policy *policy) · 21 unmodified lines | ||
| 1970 | 1978 | if (cpudata) { |
| 1971 | 1979 | union perf_cached perf = READ_ONCE(cpudata->perf); |
| 1972 | 1980 | |
| 1981 | + if (cpudata->dynamic_epp) | |
| 1982 | + amd_pstate_clear_dynamic_epp(policy); | |
| 1983 | + | |
| 1973 | 1984 | /* Reset CPPC_REQ MSR to the BIOS value */ |
| 1974 | 1985 | amd_pstate_update_perf(policy, perf.bios_min_perf, 0U, 0U, 0U, false); |
| 1975 | 1986 | amd_pstate_set_floor_perf(policy, cpudata->bios_floor_perf); |
| 1976 | 1987 | |
| 1977 | − if (cpudata->dynamic_epp) | |
| 1978 | − amd_pstate_clear_dynamic_epp(policy); | |
| 1979 | 1988 | kfree(cpudata); |
| 1980 | 1989 | policy->driver_data = NULL; |
| 1981 | 1990 | } |
| @@ -2279,7 +2279,7 @@ static int hwp_get_cpu_scaling(int cpu) · 2278 unmodified lines | ||
| 2279 | 2279 | * Return the hybrid scaling factor for P-cores and use the |
| 2280 | 2280 | * default core scaling for E-cores. |
| 2281 | 2281 | */ |
| 2282 | − if (hybrid_get_cpu_type(cpu) == INTEL_CPU_TYPE_CORE) | |
| 2282 | + if (hybrid_get_cpu_type(cpu) != INTEL_CPU_TYPE_ATOM) | |
| 2283 | 2283 | return hybrid_scaling_factor; |
| 2284 | 2284 | |
| 2285 | 2285 | return core_get_scaling(); |
| @@ -3734,6 +3734,7 @@ static const struct x86_cpu_id intel_hybrid_scaling_factor[] = { · 1448 unmodified lines | ||
| 3734 | 3734 | X86_MATCH_VFM(INTEL_RAPTORLAKE, HYBRID_SCALING_FACTOR_ADL), |
| 3735 | 3735 | X86_MATCH_VFM(INTEL_RAPTORLAKE_P, HYBRID_SCALING_FACTOR_ADL), |
| 3736 | 3736 | X86_MATCH_VFM(INTEL_RAPTORLAKE_S, HYBRID_SCALING_FACTOR_ADL), |
| 3737 | + X86_MATCH_VFM(INTEL_BARTLETTLAKE, HYBRID_SCALING_FACTOR_ADL), | |
| 3737 | 3738 | X86_MATCH_VFM(INTEL_METEORLAKE_L, HYBRID_SCALING_FACTOR_MTL), |
| 3738 | 3739 | X86_MATCH_VFM(INTEL_LUNARLAKE_M, HYBRID_SCALING_FACTOR_LNL), |
| 3739 | 3740 | {} |
| @@ -792,9 +792,13 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags) · 791 unmodified lines | ||
| 792 | 792 | if (!dmabuf || !dmabuf->file) |
| 793 | 793 | return -EINVAL; |
| 794 | 794 | |
| 795 | − fd = FD_ADD(flags, dmabuf->file); | |
| 795 | + fd = get_unused_fd_flags(flags); | |
| 796 | + if (fd < 0) | |
| 797 | + return fd; | |
| 798 | + | |
| 796 | 799 | DMA_BUF_TRACE(trace_dma_buf_fd, dmabuf, fd); |
| 797 | 800 | |
| 801 | + fd_install(fd, dmabuf->file); | |
| 798 | 802 | return fd; |
| 799 | 803 | } |
| 800 | 804 | EXPORT_SYMBOL_NS_GPL(dma_buf_fd, "DMA_BUF"); |
| @@ -829,12 +829,21 @@ int dpll_device_delete_ntf(struct dpll_device *dpll) · 828 unmodified lines | ||
| 829 | 829 | return dpll_device_event_send(DPLL_CMD_DEVICE_DELETE_NTF, dpll); |
| 830 | 830 | } |
| 831 | 831 | |
| 832 | −static int | |
| 833 | −__dpll_device_change_ntf(struct dpll_device *dpll) | |
| 832 | +/** | |
| 833 | + * __dpll_device_change_ntf - notify that the dpll device has been changed | |
| 834 | + * @dpll: registered dpll pointer | |
| 835 | + * | |
| 836 | + * Context: caller must hold dpll_lock. Suitable for use inside device | |
| 837 | + * callbacks which are already invoked under dpll_lock. | |
| 838 | + * Return: 0 if succeeds, error code otherwise. | |
| 839 | + */ | |
| 840 | +int __dpll_device_change_ntf(struct dpll_device *dpll) | |
| 834 | 841 | { |
| 842 | + lockdep_assert_held(&dpll_lock); | |
| 835 | 843 | dpll_device_notify(dpll, DPLL_DEVICE_CHANGED); |
| 836 | 844 | return dpll_device_event_send(DPLL_CMD_DEVICE_CHANGE_NTF, dpll); |
| 837 | 845 | } |
| 846 | +EXPORT_SYMBOL_GPL(__dpll_device_change_ntf); | |
| 838 | 847 | |
| 839 | 848 | /** |
| 840 | 849 | * dpll_device_change_ntf - notify that the dpll device has been changed |
| @@ -900,11 +909,21 @@ int dpll_pin_delete_ntf(struct dpll_pin *pin) · 59 unmodified lines | ||
| 900 | 909 | return dpll_pin_event_send(DPLL_CMD_PIN_DELETE_NTF, pin); |
| 901 | 910 | } |
| 902 | 911 | |
| 912 | +/** | |
| 913 | + * __dpll_pin_change_ntf - notify that the pin has been changed | |
| 914 | + * @pin: registered pin pointer | |
| 915 | + * | |
| 916 | + * Context: caller must hold dpll_lock. Suitable for use inside pin | |
| 917 | + * callbacks which are already invoked under dpll_lock. | |
| 918 | + * Return: 0 if succeeds, error code otherwise. | |
| 919 | + */ | |
| 903 | 920 | int __dpll_pin_change_ntf(struct dpll_pin *pin) |
| 904 | 921 | { |
| 922 | + lockdep_assert_held(&dpll_lock); | |
| 905 | 923 | dpll_pin_notify(pin, DPLL_PIN_CHANGED); |
| 906 | 924 | return dpll_pin_event_send(DPLL_CMD_PIN_CHANGE_NTF, pin); |
| 907 | 925 | } |
| 926 | +EXPORT_SYMBOL_GPL(__dpll_pin_change_ntf); | |
| 908 | 927 | |
| 909 | 928 | /** |
| 910 | 929 | * dpll_pin_change_ntf - notify that the pin has been changed |
| @@ -11,5 +11,3 @@ int dpll_device_delete_ntf(struct dpll_device *dpll); · 10 unmodified lines | ||
| 11 | 11 | int dpll_pin_create_ntf(struct dpll_pin *pin); |
| 12 | 12 | |
| 13 | 13 | int dpll_pin_delete_ntf(struct dpll_pin *pin); |
| 14 | − | |
| 15 | −int __dpll_pin_change_ntf(struct dpll_pin *pin); | |
| @@ -762,18 +762,15 @@ zl3073x_dev_periodic_work(struct kthread_work *work) · 761 unmodified lines | ||
| 762 | 762 | dev_warn(zldev->dev, "Failed to update phase offsets: %pe\n", |
| 763 | 763 | ERR_PTR(rc)); |
| 764 | 764 | |
| 765 | − /* Update measured input reference frequencies if any DPLL has | |
| 766 | − * frequency monitoring enabled. | |
| 765 | + /* Update measured input reference frequencies if frequency | |
| 766 | + * monitoring is enabled. | |
| 767 | 767 | */ |
| 768 | − list_for_each_entry(zldpll, &zldev->dplls, list) { | |
| 769 | − if (zldpll->freq_monitor) { | |
| 770 | − rc = zl3073x_ref_freq_meas_update(zldev); | |
| 771 | − if (rc) | |
| 772 | − dev_warn(zldev->dev, | |
| 773 | − "Failed to update measured frequencies: %pe\n", | |
| 774 | − ERR_PTR(rc)); | |
| 775 | − break; | |
| 776 | − } | |
| 768 | + if (zldev->freq_monitor) { | |
| 769 | + rc = zl3073x_ref_freq_meas_update(zldev); | |
| 770 | + if (rc) | |
| 771 | + dev_warn(zldev->dev, | |
| 772 | + "Failed to update measured frequencies: %pe\n", | |
| 773 | + ERR_PTR(rc)); | |
| 777 | 774 | } |
| 778 | 775 | |
| 779 | 776 | /* Update references' fractional frequency offsets */ |
| @@ -57,6 +57,7 @@ struct zl3073x_chip_info { · 56 unmodified lines | ||
| 57 | 57 | * @work: periodic work |
| 58 | 58 | * @clock_id: clock id of the device |
| 59 | 59 | * @phase_avg_factor: phase offset measurement averaging factor |
| 60 | + * @freq_monitor: is frequency monitor enabled | |
| 60 | 61 | */ |
| 61 | 62 | struct zl3073x_dev { |
| 62 | 63 | struct device *dev; |
| @@ -77,9 +78,10 @@ struct zl3073x_dev { · 14 unmodified lines | ||
| 77 | 78 | struct kthread_worker *kworker; |
| 78 | 79 | struct kthread_delayed_work work; |
| 79 | 80 | |
| 80 | − /* Devlink parameters */ | |
| 81 | + /* Per-chip parameters */ | |
| 81 | 82 | u64 clock_id; |
| 82 | 83 | u8 phase_avg_factor; |
| 84 | + bool freq_monitor; | |
| 83 | 85 | }; |
| 84 | 86 | |
| 85 | 87 | extern const struct regmap_config zl3073x_regmap_config; |
| @@ -1079,15 +1079,6 @@ zl3073x_dpll_phase_offset_avg_factor_get(const struct dpll_device *dpll, · 1078 unmodified lines | ||
| 1079 | 1079 | return 0; |
| 1080 | 1080 | } |
| 1081 | 1081 | |
| 1082 | −static void | |
| 1083 | −zl3073x_dpll_change_work(struct work_struct *work) | |
| 1084 | −{ | |
| 1085 | − struct zl3073x_dpll *zldpll; | |
| 1086 | − | |
| 1087 | − zldpll = container_of(work, struct zl3073x_dpll, change_work); | |
| 1088 | − dpll_device_change_ntf(zldpll->dpll_dev); | |
| 1089 | −} | |
| 1090 | − | |
| 1091 | 1082 | static int |
| 1092 | 1083 | zl3073x_dpll_phase_offset_avg_factor_set(const struct dpll_device *dpll, |
| 1093 | 1084 | void *dpll_priv, u32 factor, |
| @@ -1113,8 +1104,10 @@ zl3073x_dpll_phase_offset_avg_factor_set(const struct dpll_device *dpll, · 19 unmodified lines | ||
| 1113 | 1104 | * we have to send a notification for other DPLL devices. |
| 1114 | 1105 | */ |
| 1115 | 1106 | list_for_each_entry(item, &zldpll->dev->dplls, list) { |
| 1116 | − if (item != zldpll) | |
| 1117 | − schedule_work(&item->change_work); | |
| 1107 | + struct dpll_device *dpll_dev = READ_ONCE(item->dpll_dev); | |
| 1108 | + | |
| 1109 | + if (item != zldpll && dpll_dev) | |
| 1110 | + __dpll_device_change_ntf(dpll_dev); | |
| 1118 | 1111 | } |
| 1119 | 1112 | |
| 1120 | 1113 | return 0; |
| @@ -1219,7 +1212,7 @@ zl3073x_dpll_freq_monitor_get(const struct dpll_device *dpll, · 98 unmodified lines | ||
| 1219 | 1212 | { |
| 1220 | 1213 | struct zl3073x_dpll *zldpll = dpll_priv; |
| 1221 | 1214 | |
| 1222 | − if (zldpll->freq_monitor) | |
| 1215 | + if (zldpll->dev->freq_monitor) | |
| 1223 | 1216 | *state = DPLL_FEATURE_STATE_ENABLE; |
| 1224 | 1217 | else |
| 1225 | 1218 | *state = DPLL_FEATURE_STATE_DISABLE; |
| @@ -1233,9 +1226,19 @@ zl3073x_dpll_freq_monitor_set(const struct dpll_device *dpll, · 7 unmodified lines | ||
| 1233 | 1226 | enum dpll_feature_state state, |
| 1234 | 1227 | struct netlink_ext_ack *extack) |
| 1235 | 1228 | { |
| 1236 | − struct zl3073x_dpll *zldpll = dpll_priv; | |
| 1229 | + struct zl3073x_dpll *item, *zldpll = dpll_priv; | |
| 1230 | + | |
| 1231 | + zldpll->dev->freq_monitor = (state == DPLL_FEATURE_STATE_ENABLE); | |
| 1232 | + | |
| 1233 | + /* The frequency monitoring is common for all DPLL channels so after | |
| 1234 | + * change we have to send a notification for other DPLL devices. | |
| 1235 | + */ | |
| 1236 | + list_for_each_entry(item, &zldpll->dev->dplls, list) { | |
| 1237 | + struct dpll_device *dpll_dev = READ_ONCE(item->dpll_dev); | |
| 1237 | 1238 | |
| 1238 | − zldpll->freq_monitor = (state == DPLL_FEATURE_STATE_ENABLE); | |
| 1239 | + if (item != zldpll && dpll_dev) | |
| 1240 | + __dpll_device_change_ntf(dpll_dev); | |
| 1241 | + } | |
| 1239 | 1242 | |
| 1240 | 1243 | return 0; |
| 1241 | 1244 | } |
| @@ -1394,8 +1397,8 @@ zl3073x_dpll_pin_register(struct zl3073x_dpll_pin *pin, u32 index) · 152 unmodified lines | ||
| 1394 | 1397 | |
| 1395 | 1398 | err_register: |
| 1396 | 1399 | dpll_pin_put(pin->dpll_pin, &pin->tracker); |
| 1397 | − pin->dpll_pin = NULL; | |
| 1398 | 1400 | err_pin_get: |
| 1401 | + pin->dpll_pin = NULL; | |
| 1399 | 1402 | fwnode_handle_put(pin->fwnode); |
| 1400 | 1403 | pin->fwnode = NULL; |
| 1401 | 1404 | zl3073x_pin_props_put(props); |
| @@ -1563,8 +1566,10 @@ zl3073x_dpll_pins_register(struct zl3073x_dpll *zldpll) · 161 unmodified lines | ||
| 1563 | 1566 | } |
| 1564 | 1567 | |
| 1565 | 1568 | rc = zl3073x_dpll_pin_register(pin, index); |
| 1566 | − if (rc) | |
| 1569 | + if (rc) { | |
| 1570 | + zl3073x_dpll_pin_free(pin); | |
| 1567 | 1571 | goto error; |
| 1572 | + } | |
| 1568 | 1573 | |
| 1569 | 1574 | list_add(&pin->list, &zldpll->pins); |
| 1570 | 1575 | } |
| @@ -1625,13 +1630,13 @@ zl3073x_dpll_device_register(struct zl3073x_dpll *zldpll) · 54 unmodified lines | ||
| 1625 | 1630 | static void |
| 1626 | 1631 | zl3073x_dpll_device_unregister(struct zl3073x_dpll *zldpll) |
| 1627 | 1632 | { |
| 1628 | − WARN(!zldpll->dpll_dev, "DPLL device is not registered\n"); | |
| 1633 | + struct dpll_device *dpll_dev = READ_ONCE(zldpll->dpll_dev); | |
| 1629 | 1634 | |
| 1630 | − cancel_work_sync(&zldpll->change_work); | |
| 1635 | + WARN(!dpll_dev, "DPLL device is not registered\n"); | |
| 1631 | 1636 | |
| 1632 | − dpll_device_unregister(zldpll->dpll_dev, &zldpll->ops, zldpll); | |
| 1633 | − dpll_device_put(zldpll->dpll_dev, &zldpll->tracker); | |
| 1634 | − zldpll->dpll_dev = NULL; | |
| 1637 | + WRITE_ONCE(zldpll->dpll_dev, NULL); | |
| 1638 | + dpll_device_unregister(dpll_dev, &zldpll->ops, zldpll); | |
| 1639 | + dpll_device_put(dpll_dev, &zldpll->tracker); | |
| 1635 | 1640 | } |
| 1636 | 1641 | |
| 1637 | 1642 | /** |
| @@ -1750,7 +1755,7 @@ zl3073x_dpll_pin_measured_freq_check(struct zl3073x_dpll_pin *pin) · 112 unmodified lines | ||
| 1750 | 1755 | u8 ref_id; |
| 1751 | 1756 | u32 freq; |
| 1752 | 1757 | |
| 1753 | − if (!zldpll->freq_monitor) | |
| 1758 | + if (!zldpll->dev->freq_monitor) | |
| 1754 | 1759 | return false; |
| 1755 | 1760 | |
| 1756 | 1761 | ref_id = zl3073x_input_pin_ref_get(pin->id); |
| @@ -1783,10 +1788,8 @@ zl3073x_dpll_changes_check(struct zl3073x_dpll *zldpll) · 26 unmodified lines | ||
| 1783 | 1788 | struct zl3073x_dev *zldev = zldpll->dev; |
| 1784 | 1789 | enum dpll_lock_status lock_status; |
| 1785 | 1790 | struct device *dev = zldev->dev; |
| 1786 | − const struct zl3073x_chan *chan; | |
| 1787 | 1791 | struct zl3073x_dpll_pin *pin; |
| 1788 | 1792 | int rc; |
| 1789 | − u8 mode; | |
| 1790 | 1793 | |
| 1791 | 1794 | zldpll->check_count++; |
| 1792 | 1795 | |
| @@ -1805,15 +1808,6 @@ zl3073x_dpll_changes_check(struct zl3073x_dpll *zldpll) · 12 unmodified lines | ||
| 1805 | 1808 | dpll_device_change_ntf(zldpll->dpll_dev); |
| 1806 | 1809 | } |
| 1807 | 1810 | |
| 1808 | − /* Input pin monitoring does make sense only in automatic | |
| 1809 | − * or forced reference modes. | |
| 1810 | − */ | |
| 1811 | − chan = zl3073x_chan_state_get(zldev, zldpll->id); | |
| 1812 | − mode = zl3073x_chan_mode_get(chan); | |
| 1813 | − if (mode != ZL_DPLL_MODE_REFSEL_MODE_AUTO && | |
| 1814 | − mode != ZL_DPLL_MODE_REFSEL_MODE_REFLOCK) | |
| 1815 | − return; | |
| 1816 | − | |
| 1817 | 1811 | /* Update phase offset latch registers for this DPLL if the phase |
| 1818 | 1812 | * offset monitor feature is enabled. |
| 1819 | 1813 | */ |
| @@ -1924,7 +1918,6 @@ zl3073x_dpll_alloc(struct zl3073x_dev *zldev, u8 ch) · 104 unmodified lines | ||
| 1924 | 1918 | zldpll->dev = zldev; |
| 1925 | 1919 | zldpll->id = ch; |
| 1926 | 1920 | INIT_LIST_HEAD(&zldpll->pins); |
| 1927 | − INIT_WORK(&zldpll->change_work, zl3073x_dpll_change_work); | |
| 1928 | 1921 | |
| 1929 | 1922 | return zldpll; |
| 1930 | 1923 | } |
| @@ -15,13 +15,11 @@ · 14 unmodified lines | ||
| 15 | 15 | * @id: DPLL index |
| 16 | 16 | * @check_count: periodic check counter |
| 17 | 17 | * @phase_monitor: is phase offset monitor enabled |
| 18 | − * @freq_monitor: is frequency monitor enabled | |
| 19 | 18 | * @ops: DPLL device operations for this instance |
| 20 | 19 | * @dpll_dev: pointer to registered DPLL device |
| 21 | 20 | * @tracker: tracking object for the acquired reference |
| 22 | 21 | * @lock_status: last saved DPLL lock status |
| 23 | 22 | * @pins: list of pins |
| 24 | − * @change_work: device change notification work | |
| 25 | 23 | */ |
| 26 | 24 | struct zl3073x_dpll { |
| 27 | 25 | struct list_head list; |
| @@ -29,13 +27,11 @@ struct zl3073x_dpll { · 1 unmodified line | ||
| 29 | 27 | u8 id; |
| 30 | 28 | u8 check_count; |
| 31 | 29 | bool phase_monitor; |
| 32 | − bool freq_monitor; | |
| 33 | 30 | struct dpll_device_ops ops; |
| 34 | 31 | struct dpll_device *dpll_dev; |
| 35 | 32 | dpll_tracker tracker; |
| 36 | 33 | enum dpll_lock_status lock_status; |
| 37 | 34 | struct list_head pins; |
| 38 | − struct work_struct change_work; | |
| 39 | 35 | }; |
| 40 | 36 | |
| 41 | 37 | struct zl3073x_dpll *zl3073x_dpll_alloc(struct zl3073x_dev *zldev, u8 ch); |
| @@ -777,9 +777,9 @@ static int init_one_mc(struct mc_priv *priv, struct platform_device *pdev, int i · 776 unmodified lines | ||
| 777 | 777 | u32 num_chans, rank, dwidth, config; |
| 778 | 778 | struct edac_mc_layer layers[2]; |
| 779 | 779 | struct mem_ctl_info *mci; |
| 780 | + char name[MC_NAME_LEN]; | |
| 780 | 781 | struct device *dev; |
| 781 | 782 | enum dev_type dt; |
| 782 | − char *name; | |
| 783 | 783 | int rc; |
| 784 | 784 | |
| 785 | 785 | config = priv->adec[CONF + i * ADEC_NUM]; |
| @@ -813,13 +813,9 @@ static int init_one_mc(struct mc_priv *priv, struct platform_device *pdev, int i · 27 unmodified lines | ||
| 813 | 813 | layers[1].is_virt_csrow = false; |
| 814 | 814 | |
| 815 | 815 | rc = -ENOMEM; |
| 816 | − name = kzalloc(MC_NAME_LEN, GFP_KERNEL); | |
| 817 | − if (!name) | |
| 818 | − return rc; | |
| 819 | − | |
| 820 | 816 | dev = kzalloc(sizeof(*dev), GFP_KERNEL); |
| 821 | 817 | if (!dev) |
| 822 | − goto err_name_free; | |
| 818 | + return rc; | |
| 823 | 819 | |
| 824 | 820 | mci = edac_mc_alloc(i, ARRAY_SIZE(layers), layers, sizeof(struct mc_priv)); |
| 825 | 821 | if (!mci) { |
| @@ -858,8 +854,6 @@ err_mc_free: · 32 unmodified lines | ||
| 858 | 854 | edac_mc_free(mci); |
| 859 | 855 | err_dev_free: |
| 860 | 856 | kfree(dev); |
| 861 | −err_name_free: | |
| 862 | − kfree(name); | |
| 863 | 857 | |
| 864 | 858 | return rc; |
| 865 | 859 | } |
| @@ -26,6 +26,8 @@ static int ffa_device_match(struct device *dev, const struct device_driver *drv) · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | id_table = to_ffa_driver(drv)->id_table; |
| 28 | 28 | ffa_dev = to_ffa_dev(dev); |
| 29 | + if (!id_table) | |
| 30 | + return 0; | |
| 29 | 31 | |
| 30 | 32 | while (!uuid_is_null(&id_table->uuid)) { |
| 31 | 33 | /* |
| @@ -123,7 +125,7 @@ int ffa_driver_register(struct ffa_driver *driver, struct module *owner, · 91 unmodified lines | ||
| 123 | 125 | { |
| 124 | 126 | int ret; |
| 125 | 127 | |
| 126 | − if (!driver->probe) | |
| 128 | + if (!driver->probe || !driver->id_table) | |
| 127 | 129 | return -EINVAL; |
| 128 | 130 | |
| 129 | 131 | driver->driver.bus = &ffa_bus_type; |
| @@ -87,6 +87,7 @@ static inline int ffa_to_linux_errno(int errno) · 86 unmodified lines | ||
| 87 | 87 | |
| 88 | 88 | struct ffa_pcpu_irq { |
| 89 | 89 | struct ffa_drv_info *info; |
| 90 | + struct work_struct notif_pcpu_work; | |
| 90 | 91 | }; |
| 91 | 92 | |
| 92 | 93 | struct ffa_drv_info { |
| @@ -100,13 +101,13 @@ struct ffa_drv_info { · 7 unmodified lines | ||
| 100 | 101 | bool mem_ops_native; |
| 101 | 102 | bool msg_direct_req2_supp; |
| 102 | 103 | bool bitmap_created; |
| 104 | + bool bus_notifier_registered; | |
| 103 | 105 | bool notif_enabled; |
| 104 | 106 | unsigned int sched_recv_irq; |
| 105 | 107 | unsigned int notif_pend_irq; |
| 106 | 108 | unsigned int cpuhp_state; |
| 107 | 109 | struct ffa_pcpu_irq __percpu *irq_pcpu; |
| 108 | 110 | struct workqueue_struct *notif_pcpu_wq; |
| 109 | − struct work_struct notif_pcpu_work; | |
| 110 | 111 | struct work_struct sched_recv_irq_work; |
| 111 | 112 | struct xarray partition_info; |
| 112 | 113 | DECLARE_HASHTABLE(notifier_hash, ilog2(FFA_MAX_NOTIFICATIONS)); |
| @@ -322,6 +323,12 @@ __ffa_partition_info_get(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 209 unmodified lines | ||
| 322 | 323 | #define PART_INFO_ID_MASK GENMASK(15, 0) |
| 323 | 324 | #define PART_INFO_EXEC_CXT_MASK GENMASK(31, 16) |
| 324 | 325 | #define PART_INFO_PROPS_MASK GENMASK(63, 32) |
| 326 | +#define FFA_PART_INFO_GET_REGS_FIRST_REG 3 | |
| 327 | +#define FFA_PART_INFO_GET_REGS_REGS_PER_DESC 3 | |
| 328 | +#define FFA_PART_INFO_GET_REGS_MAX_DESC \ | |
| 329 | + (((sizeof(ffa_value_t) / sizeof_field(ffa_value_t, a0)) - \ | |
| 330 | + FFA_PART_INFO_GET_REGS_FIRST_REG) / \ | |
| 331 | + FFA_PART_INFO_GET_REGS_REGS_PER_DESC) | |
| 325 | 332 | #define PART_INFO_ID(x) ((u16)(FIELD_GET(PART_INFO_ID_MASK, (x)))) |
| 326 | 333 | #define PART_INFO_EXEC_CXT(x) ((u16)(FIELD_GET(PART_INFO_EXEC_CXT_MASK, (x)))) |
| 327 | 334 | #define PART_INFO_PROPERTIES(x) ((u32)(FIELD_GET(PART_INFO_PROPS_MASK, (x)))) |
| @@ -329,15 +336,13 @@ static int · 1 unmodified line | ||
| 329 | 336 | __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, |
| 330 | 337 | struct ffa_partition_info *buffer, int num_parts) |
| 331 | 338 | { |
| 332 | − u16 buf_sz, start_idx, cur_idx, count = 0, prev_idx = 0, tag = 0; | |
| 339 | + u16 buf_sz, start_idx = 0, cur_idx, count = 0, tag = 0; | |
| 333 | 340 | struct ffa_partition_info *buf = buffer; |
| 334 | 341 | ffa_value_t partition_info; |
| 335 | 342 | |
| 336 | 343 | do { |
| 337 | 344 | __le64 *regs; |
| 338 | − int idx; | |
| 339 | − | |
| 340 | − start_idx = prev_idx ? prev_idx + 1 : 0; | |
| 345 | + int idx, nr_desc, buf_idx; | |
| 341 | 346 | |
| 342 | 347 | invoke_ffa_fn((ffa_value_t){ |
| 343 | 348 | .a0 = FFA_PARTITION_INFO_GET_REGS, |
| @@ -353,15 +358,28 @@ __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 9 unmodified lines | ||
| 353 | 358 | count = PARTITION_COUNT(partition_info.a2); |
| 354 | 359 | if (!buffer || !num_parts) /* count only */ |
| 355 | 360 | return count; |
| 361 | + if (count > num_parts) | |
| 362 | + return -EINVAL; | |
| 356 | 363 | |
| 357 | 364 | cur_idx = CURRENT_INDEX(partition_info.a2); |
| 365 | + if (cur_idx < start_idx || cur_idx >= count) | |
| 366 | + return -EINVAL; | |
| 367 | + | |
| 368 | + nr_desc = cur_idx - start_idx + 1; | |
| 369 | + if (nr_desc > FFA_PART_INFO_GET_REGS_MAX_DESC) | |
| 370 | + return -EINVAL; | |
| 371 | + | |
| 372 | + buf_idx = buf - buffer; | |
| 373 | + if (buf_idx + nr_desc > num_parts) | |
| 374 | + return -EINVAL; | |
| 375 | + | |
| 358 | 376 | tag = UUID_INFO_TAG(partition_info.a2); |
| 359 | 377 | buf_sz = PARTITION_INFO_SZ(partition_info.a2); |
| 360 | 378 | if (buf_sz > sizeof(*buffer)) |
| 361 | 379 | buf_sz = sizeof(*buffer); |
| 362 | 380 | |
| 363 | 381 | regs = (void *)&partition_info.a3; |
| 364 | − for (idx = 0; idx < cur_idx - start_idx + 1; idx++, buf++) { | |
| 382 | + for (idx = 0; idx < nr_desc; idx++, buf++) { | |
| 365 | 383 | union { |
| 366 | 384 | uuid_t uuid; |
| 367 | 385 | u64 regs[2]; |
| @@ -379,7 +397,7 @@ __ffa_partition_info_get_regs(u32 uuid0, u32 uuid1, u32 uuid2, u32 uuid3, · 11 unmodified lines | ||
| 379 | 397 | uuid_copy(&buf->uuid, &uuid_regs.uuid); |
| 380 | 398 | regs += 3; |
| 381 | 399 | } |
| 382 | − prev_idx = cur_idx; | |
| 400 | + start_idx = cur_idx + 1; | |
| 383 | 401 | |
| 384 | 402 | } while (cur_idx < (count - 1)); |
| 385 | 403 | |
| @@ -1189,7 +1207,7 @@ static int · 803 unmodified lines | ||
| 1189 | 1207 | ffa_sched_recv_cb_update(struct ffa_device *dev, ffa_sched_recv_cb callback, |
| 1190 | 1208 | void *cb_data, bool is_registration) |
| 1191 | 1209 | { |
| 1192 | − struct ffa_dev_part_info *partition = NULL, *tmp; | |
| 1210 | + struct ffa_dev_part_info *partition = NULL; | |
| 1193 | 1211 | struct list_head *phead; |
| 1194 | 1212 | bool cb_valid; |
| 1195 | 1213 | |
| @@ -1202,11 +1220,11 @@ ffa_sched_recv_cb_update(struct ffa_device *dev, ffa_sched_recv_cb callback, · 6 unmodified lines | ||
| 1202 | 1220 | return -EINVAL; |
| 1203 | 1221 | } |
| 1204 | 1222 | |
| 1205 | − list_for_each_entry_safe(partition, tmp, phead, node) | |
| 1223 | + list_for_each_entry(partition, phead, node) | |
| 1206 | 1224 | if (partition->dev == dev) |
| 1207 | 1225 | break; |
| 1208 | 1226 | |
| 1209 | − if (!partition) { | |
| 1227 | + if (&partition->node == phead) { | |
| 1210 | 1228 | pr_err("%s: No such partition ID 0x%x\n", __func__, dev->vm_id); |
| 1211 | 1229 | return -EINVAL; |
| 1212 | 1230 | } |
| @@ -1445,20 +1463,25 @@ static int ffa_notify_send(struct ffa_device *dev, int notify_id, · 232 unmodified lines | ||
| 1445 | 1463 | |
| 1446 | 1464 | static void handle_notif_callbacks(u64 bitmap, enum notify_type type) |
| 1447 | 1465 | { |
| 1466 | + ffa_notifier_cb cb; | |
| 1467 | + void *cb_data; | |
| 1448 | 1468 | int notify_id; |
| 1449 | − struct notifier_cb_info *cb_info = NULL; | |
| 1450 | 1469 | |
| 1451 | 1470 | for (notify_id = 0; notify_id <= FFA_MAX_NOTIFICATIONS && bitmap; |
| 1452 | 1471 | notify_id++, bitmap >>= 1) { |
| 1453 | 1472 | if (!(bitmap & 1)) |
| 1454 | 1473 | continue; |
| 1455 | 1474 | |
| 1456 | − read_lock(&drv_info->notify_lock); | |
| 1457 | − cb_info = notifier_hnode_get_by_type(notify_id, type); | |
| 1458 | − read_unlock(&drv_info->notify_lock); | |
| 1475 | + scoped_guard(read_lock, &drv_info->notify_lock) { | |
| 1476 | + struct notifier_cb_info *cb_info; | |
| 1459 | 1477 | |
| 1460 | − if (cb_info && cb_info->cb) | |
| 1461 | − cb_info->cb(notify_id, cb_info->cb_data); | |
| 1478 | + cb_info = notifier_hnode_get_by_type(notify_id, type); | |
| 1479 | + cb = cb_info ? cb_info->cb : NULL; | |
| 1480 | + cb_data = cb_info ? cb_info->cb_data : NULL; | |
| 1481 | + } | |
| 1482 | + | |
| 1483 | + if (cb) | |
| 1484 | + cb(notify_id, cb_data); | |
| 1462 | 1485 | } |
| 1463 | 1486 | } |
| 1464 | 1487 | |
| @@ -1466,39 +1489,56 @@ static void handle_fwk_notif_callbacks(u32 bitmap) · 1 unmodified line | ||
| 1466 | 1489 | { |
| 1467 | 1490 | void *buf; |
| 1468 | 1491 | uuid_t uuid; |
| 1492 | + void *fwk_cb_data; | |
| 1469 | 1493 | int notify_id = 0, target; |
| 1494 | + ffa_fwk_notifier_cb fwk_cb; | |
| 1470 | 1495 | struct ffa_indirect_msg_hdr *msg; |
| 1471 | − struct notifier_cb_info *cb_info = NULL; | |
| 1496 | + size_t min_offset = offsetof(struct ffa_indirect_msg_hdr, uuid); | |
| 1472 | 1497 | |
| 1473 | 1498 | /* Only one framework notification defined and supported for now */ |
| 1474 | 1499 | if (!(bitmap & FRAMEWORK_NOTIFY_RX_BUFFER_FULL)) |
| 1475 | 1500 | return; |
| 1476 | 1501 | |
| 1477 | − mutex_lock(&drv_info->rx_lock); | |
| 1502 | + scoped_guard(mutex, &drv_info->rx_lock) { | |
| 1503 | + u32 offset, size; | |
| 1478 | 1504 | |
| 1479 | − msg = drv_info->rx_buffer; | |
| 1480 | − buf = kmemdup((void *)msg + msg->offset, msg->size, GFP_KERNEL); | |
| 1481 | − if (!buf) { | |
| 1482 | − mutex_unlock(&drv_info->rx_lock); | |
| 1483 | − return; | |
| 1484 | − } | |
| 1505 | + msg = drv_info->rx_buffer; | |
| 1506 | + offset = msg->offset; | |
| 1507 | + size = msg->size; | |
| 1485 | 1508 | |
| 1486 | − target = SENDER_ID(msg->send_recv_id); | |
| 1487 | − if (msg->offset >= sizeof(*msg)) | |
| 1488 | − uuid_copy(&uuid, &msg->uuid); | |
| 1489 | − else | |
| 1490 | − uuid_copy(&uuid, &uuid_null); | |
| 1509 | + if (!size || (offset != min_offset && offset < sizeof(*msg)) || | |
| 1510 | + offset > drv_info->rxtx_bufsz || | |
| 1511 | + size > drv_info->rxtx_bufsz - offset) { | |
| 1512 | + pr_err("invalid framework notification message\n"); | |
| 1513 | + ffa_rx_release(); | |
| 1514 | + return; | |
| 1515 | + } | |
| 1491 | 1516 | |
| 1492 | − mutex_unlock(&drv_info->rx_lock); | |
| 1517 | + buf = kmemdup((void *)msg + offset, size, GFP_KERNEL); | |
| 1518 | + if (!buf) { | |
| 1519 | + ffa_rx_release(); | |
| 1520 | + return; | |
| 1521 | + } | |
| 1522 | + | |
| 1523 | + target = SENDER_ID(msg->send_recv_id); | |
| 1524 | + if (offset >= sizeof(*msg)) | |
| 1525 | + uuid_copy(&uuid, &msg->uuid); | |
| 1526 | + else | |
| 1527 | + uuid_copy(&uuid, &uuid_null); | |
| 1528 | + ffa_rx_release(); | |
| 1529 | + } | |
| 1493 | 1530 | |
| 1494 | − ffa_rx_release(); | |
| 1531 | + scoped_guard(read_lock, &drv_info->notify_lock) { | |
| 1532 | + struct notifier_cb_info *cb_info; | |
| 1495 | 1533 | |
| 1496 | − read_lock(&drv_info->notify_lock); | |
| 1497 | − cb_info = notifier_hnode_get_by_vmid_uuid(notify_id, target, &uuid); | |
| 1498 | − read_unlock(&drv_info->notify_lock); | |
| 1534 | + cb_info = notifier_hnode_get_by_vmid_uuid(notify_id, target, | |
| 1535 | + &uuid); | |
| 1536 | + fwk_cb = cb_info ? cb_info->fwk_cb : NULL; | |
| 1537 | + fwk_cb_data = cb_info ? cb_info->cb_data : NULL; | |
| 1538 | + } | |
| 1499 | 1539 | |
| 1500 | − if (cb_info && cb_info->fwk_cb) | |
| 1501 | − cb_info->fwk_cb(notify_id, cb_info->cb_data, buf); | |
| 1540 | + if (fwk_cb) | |
| 1541 | + fwk_cb(notify_id, fwk_cb_data, buf); | |
| 1502 | 1542 | kfree(buf); |
| 1503 | 1543 | } |
| 1504 | 1544 | |
| @@ -1539,10 +1579,11 @@ ffa_self_notif_handle(u16 vcpu, bool is_per_vcpu, void *cb_data) · 34 unmodified lines | ||
| 1539 | 1579 | |
| 1540 | 1580 | static void notif_pcpu_irq_work_fn(struct work_struct *work) |
| 1541 | 1581 | { |
| 1542 | − struct ffa_drv_info *info = container_of(work, struct ffa_drv_info, | |
| 1582 | + struct ffa_pcpu_irq *pcpu = container_of(work, struct ffa_pcpu_irq, | |
| 1543 | 1583 | notif_pcpu_work); |
| 1584 | + struct ffa_drv_info *info = pcpu->info; | |
| 1544 | 1585 | |
| 1545 | − ffa_self_notif_handle(smp_processor_id(), true, info); | |
| 1586 | + notif_get_and_handle(info); | |
| 1546 | 1587 | } |
| 1547 | 1588 | |
| 1548 | 1589 | static const struct ffa_info_ops ffa_drv_info_ops = { |
| @@ -1629,6 +1670,15 @@ static struct notifier_block ffa_bus_nb = { · 80 unmodified lines | ||
| 1629 | 1670 | .notifier_call = ffa_bus_notifier, |
| 1630 | 1671 | }; |
| 1631 | 1672 | |
| 1673 | +static void ffa_bus_notifier_unregister(void) | |
| 1674 | +{ | |
| 1675 | + if (!drv_info->bus_notifier_registered) | |
| 1676 | + return; | |
| 1677 | + | |
| 1678 | + bus_unregister_notifier(&ffa_bus_type, &ffa_bus_nb); | |
| 1679 | + drv_info->bus_notifier_registered = false; | |
| 1680 | +} | |
| 1681 | + | |
| 1632 | 1682 | static int ffa_xa_add_partition_info(struct ffa_device *dev) |
| 1633 | 1683 | { |
| 1634 | 1684 | struct ffa_dev_part_info *info; |
| @@ -1712,6 +1762,8 @@ static void ffa_partitions_cleanup(void) · 77 unmodified lines | ||
| 1712 | 1762 | struct list_head *phead; |
| 1713 | 1763 | unsigned long idx; |
| 1714 | 1764 | |
| 1765 | + ffa_bus_notifier_unregister(); | |
| 1766 | + | |
| 1715 | 1767 | /* Clean up/free all registered devices */ |
| 1716 | 1768 | ffa_devices_unregister(); |
| 1717 | 1769 | |
| @@ -1739,11 +1791,14 @@ static int ffa_setup_partitions(void) · 21 unmodified lines | ||
| 1739 | 1791 | ret = bus_register_notifier(&ffa_bus_type, &ffa_bus_nb); |
| 1740 | 1792 | if (ret) |
| 1741 | 1793 | pr_err("Failed to register FF-A bus notifiers\n"); |
| 1794 | + else | |
| 1795 | + drv_info->bus_notifier_registered = true; | |
| 1742 | 1796 | } |
| 1743 | 1797 | |
| 1744 | 1798 | count = ffa_partition_probe(&uuid_null, &pbuf); |
| 1745 | 1799 | if (count <= 0) { |
| 1746 | 1800 | pr_info("%s: No partitions found, error %d\n", __func__, count); |
| 1801 | + ffa_bus_notifier_unregister(); | |
| 1747 | 1802 | return -EINVAL; |
| 1748 | 1803 | } |
| 1749 | 1804 | |
| @@ -1811,7 +1866,7 @@ static irqreturn_t notif_pend_irq_handler(int irq, void *irq_data) · 61 unmodified lines | ||
| 1811 | 1866 | struct ffa_drv_info *info = pcpu->info; |
| 1812 | 1867 | |
| 1813 | 1868 | queue_work_on(smp_processor_id(), info->notif_pcpu_wq, |
| 1814 | − &info->notif_pcpu_work); | |
| 1869 | + &pcpu->notif_pcpu_work); | |
| 1815 | 1870 | |
| 1816 | 1871 | return IRQ_HANDLED; |
| 1817 | 1872 | } |
| @@ -1928,8 +1983,11 @@ static int ffa_init_pcpu_irq(void) · 110 unmodified lines | ||
| 1928 | 1983 | if (!irq_pcpu) |
| 1929 | 1984 | return -ENOMEM; |
| 1930 | 1985 | |
| 1931 | − for_each_present_cpu(cpu) | |
| 1986 | + for_each_present_cpu(cpu) { | |
| 1932 | 1987 | per_cpu_ptr(irq_pcpu, cpu)->info = drv_info; |
| 1988 | + INIT_WORK(&per_cpu_ptr(irq_pcpu, cpu)->notif_pcpu_work, | |
| 1989 | + notif_pcpu_irq_work_fn); | |
| 1990 | + } | |
| 1933 | 1991 | |
| 1934 | 1992 | drv_info->irq_pcpu = irq_pcpu; |
| 1935 | 1993 | |
| @@ -1958,7 +2016,6 @@ static int ffa_init_pcpu_irq(void) · 22 unmodified lines | ||
| 1958 | 2016 | } |
| 1959 | 2017 | |
| 1960 | 2018 | INIT_WORK(&drv_info->sched_recv_irq_work, ffa_sched_recv_irq_work_fn); |
| 1961 | − INIT_WORK(&drv_info->notif_pcpu_work, notif_pcpu_irq_work_fn); | |
| 1962 | 2019 | drv_info->notif_pcpu_wq = create_workqueue("ffa_pcpu_irq_notification"); |
| 1963 | 2020 | if (!drv_info->notif_pcpu_wq) |
| 1964 | 2021 | return -EINVAL; |
| @@ -2063,11 +2120,12 @@ static int __init ffa_init(void) · 98 unmodified lines | ||
| 2063 | 2120 | rxtx_bufsz = SZ_4K; |
| 2064 | 2121 | } |
| 2065 | 2122 | |
| 2123 | + rxtx_bufsz = PAGE_ALIGN(rxtx_bufsz); | |
| 2066 | 2124 | drv_info->rxtx_bufsz = rxtx_bufsz; |
| 2067 | 2125 | drv_info->rx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); |
| 2068 | 2126 | if (!drv_info->rx_buffer) { |
| 2069 | 2127 | ret = -ENOMEM; |
| 2070 | − goto free_pages; | |
| 2128 | + goto free_drv_info; | |
| 2071 | 2129 | } |
| 2072 | 2130 | |
| 2073 | 2131 | drv_info->tx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); |
| @@ -2078,7 +2136,7 @@ static int __init ffa_init(void) · 4 unmodified lines | ||
| 2078 | 2136 | |
| 2079 | 2137 | ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer), |
| 2080 | 2138 | virt_to_phys(drv_info->rx_buffer), |
| 2081 | − PAGE_ALIGN(rxtx_bufsz) / FFA_PAGE_SIZE); | |
| 2139 | + rxtx_bufsz / FFA_PAGE_SIZE); | |
| 2082 | 2140 | if (ret) { |
| 2083 | 2141 | pr_err("failed to register FFA RxTx buffers\n"); |
| 2084 | 2142 | goto free_pages; |
| @@ -60,8 +60,10 @@ static int efi_pstore_open(struct pstore_info *psi) · 59 unmodified lines | ||
| 60 | 60 | return err; |
| 61 | 61 | |
| 62 | 62 | psi->data = kzalloc(record_size, GFP_KERNEL); |
| 63 | − if (!psi->data) | |
| 63 | + if (!psi->data) { | |
| 64 | + efivar_unlock(); | |
| 64 | 65 | return -ENOMEM; |
| 66 | + } | |
| 65 | 67 | |
| 66 | 68 | return 0; |
| 67 | 69 | } |
| @@ -402,21 +402,11 @@ static void __init efi_debugfs_init(void) · 401 unmodified lines | ||
| 402 | 402 | static inline void efi_debugfs_init(void) {} |
| 403 | 403 | #endif |
| 404 | 404 | |
| 405 | −/* | |
| 406 | − * We register the efi subsystem with the firmware subsystem and the | |
| 407 | − * efivars subsystem with the efi subsystem, if the system was booted with | |
| 408 | − * EFI. | |
| 409 | − */ | |
| 410 | −static int __init efisubsys_init(void) | |
| 405 | +static int __init efipostcore_init(void) | |
| 411 | 406 | { |
| 412 | − int error; | |
| 413 | − | |
| 414 | 407 | if (!efi_enabled(EFI_RUNTIME_SERVICES)) |
| 415 | 408 | efi.runtime_supported_mask = 0; |
| 416 | 409 | |
| 417 | − if (!efi_enabled(EFI_BOOT)) | |
| 418 | − return 0; | |
| 419 | − | |
| 420 | 410 | if (efi.runtime_supported_mask) { |
| 421 | 411 | /* |
| 422 | 412 | * Since we process only one efi_runtime_service() at a time, an |
| @@ -428,9 +418,23 @@ static int __init efisubsys_init(void) · 5 unmodified lines | ||
| 428 | 418 | pr_err("Creating efi_rts_wq failed, EFI runtime services disabled.\n"); |
| 429 | 419 | clear_bit(EFI_RUNTIME_SERVICES, &efi.flags); |
| 430 | 420 | efi.runtime_supported_mask = 0; |
| 431 | − return 0; | |
| 432 | 421 | } |
| 433 | 422 | } |
| 423 | + return 0; | |
| 424 | +} | |
| 425 | +postcore_initcall(efipostcore_init); | |
| 426 | + | |
| 427 | +/* | |
| 428 | + * We register the efi subsystem with the firmware subsystem and the | |
| 429 | + * efivars subsystem with the efi subsystem, if the system was booted with | |
| 430 | + * EFI. | |
| 431 | + */ | |
| 432 | +static int __init efisubsys_init(void) | |
| 433 | +{ | |
| 434 | + int error; | |
| 435 | + | |
| 436 | + if (!efi_enabled(EFI_BOOT)) | |
| 437 | + return 0; | |
| 434 | 438 | |
| 435 | 439 | if (efi_rt_services_supported(EFI_RT_SUPPORTED_TIME_SERVICES)) |
| 436 | 440 | platform_device_register_simple("rtc-efi", 0, NULL, 0); |
| @@ -66,7 +66,7 @@ KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__ · 65 unmodified lines | ||
| 66 | 66 | lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \ |
| 67 | 67 | file.o mem.o random.o randomalloc.o pci.o \ |
| 68 | 68 | skip_spaces.o lib-cmdline.o lib-ctype.o \ |
| 69 | − alignedmem.o relocate.o printk.o vsprintf.o | |
| 69 | + alignedmem.o printk.o vsprintf.o | |
| 70 | 70 | |
| 71 | 71 | # include the stub's libfdt dependencies from lib/ when needed |
| 72 | 72 | libfdt-deps := fdt_rw.c fdt_ro.c fdt_wip.c fdt.c \ |
| @@ -79,6 +79,10 @@ efi_status_t efi_parse_options(char const *cmdline) · 78 unmodified lines | ||
| 79 | 79 | efi_noinitrd = true; |
| 80 | 80 | } else if (IS_ENABLED(CONFIG_X86_64) && !strcmp(param, "no5lvl")) { |
| 81 | 81 | efi_no5lvl = true; |
| 82 | + } else if (IS_ENABLED(CONFIG_LOONGARCH) && | |
| 83 | + IS_ENABLED(CONFIG_HIBERNATION) && | |
| 84 | + !strcmp(param, "resume") && val) { | |
| 85 | + efi_nokaslr = true; /* LoongArch can't KASLR for hibernation */ | |
| 82 | 86 | } else if (IS_ENABLED(CONFIG_ARCH_HAS_MEM_ENCRYPT) && |
| 83 | 87 | !strcmp(param, "mem_encrypt") && val) { |
| 84 | 88 | if (parse_option_str(val, "on")) |
| @@ -1104,13 +1104,6 @@ efi_status_t efi_allocate_pages_aligned(unsigned long size, unsigned long *addr, · 1103 unmodified lines | ||
| 1104 | 1104 | efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, |
| 1105 | 1105 | unsigned long *addr, unsigned long min); |
| 1106 | 1106 | |
| 1107 | −efi_status_t efi_relocate_kernel(unsigned long *image_addr, | |
| 1108 | − unsigned long image_size, | |
| 1109 | − unsigned long alloc_size, | |
| 1110 | − unsigned long preferred_addr, | |
| 1111 | − unsigned long alignment, | |
| 1112 | − unsigned long min_addr); | |
| 1113 | − | |
| 1114 | 1107 | efi_status_t efi_parse_options(char const *cmdline); |
| 1115 | 1108 | |
| 1116 | 1109 | void efi_parse_option_graphics(char *option); |
| @@ -14,6 +14,86 @@ extern int kernel_asize; · 13 unmodified lines | ||
| 14 | 14 | extern int kernel_fsize; |
| 15 | 15 | extern int kernel_entry; |
| 16 | 16 | |
| 17 | +/** | |
| 18 | + * efi_relocate_kernel() - copy memory area | |
| 19 | + * @image_addr: pointer to address of memory area to copy | |
| 20 | + * @image_size: size of memory area to copy | |
| 21 | + * @alloc_size: minimum size of memory to allocate, must be greater or | |
| 22 | + * equal to image_size | |
| 23 | + * @preferred_addr: preferred target address | |
| 24 | + * @alignment: minimum alignment of the allocated memory area. It | |
| 25 | + * should be a power of two. | |
| 26 | + * @min_addr: minimum target address | |
| 27 | + * | |
| 28 | + * Copy a memory area to a newly allocated memory area aligned according | |
| 29 | + * to @alignment but at least EFI_ALLOC_ALIGN. If the preferred address | |
| 30 | + * is not available, the allocated address will not be below @min_addr. | |
| 31 | + * On exit, @image_addr is updated to the target copy address that was used. | |
| 32 | + * | |
| 33 | + * This function is used to copy the Linux kernel verbatim. It does not apply | |
| 34 | + * any relocation changes. | |
| 35 | + * | |
| 36 | + * Return: status code | |
| 37 | + */ | |
| 38 | +static | |
| 39 | +efi_status_t efi_relocate_kernel(unsigned long *image_addr, | |
| 40 | + unsigned long image_size, | |
| 41 | + unsigned long alloc_size, | |
| 42 | + unsigned long preferred_addr, | |
| 43 | + unsigned long alignment, | |
| 44 | + unsigned long min_addr) | |
| 45 | +{ | |
| 46 | + unsigned long cur_image_addr; | |
| 47 | + unsigned long new_addr = 0; | |
| 48 | + efi_status_t status; | |
| 49 | + unsigned long nr_pages; | |
| 50 | + efi_physical_addr_t efi_addr = preferred_addr; | |
| 51 | + | |
| 52 | + if (!image_addr || !image_size || !alloc_size) | |
| 53 | + return EFI_INVALID_PARAMETER; | |
| 54 | + if (alloc_size < image_size) | |
| 55 | + return EFI_INVALID_PARAMETER; | |
| 56 | + | |
| 57 | + cur_image_addr = *image_addr; | |
| 58 | + | |
| 59 | + /* | |
| 60 | + * The EFI firmware loader could have placed the kernel image | |
| 61 | + * anywhere in memory, but the kernel has restrictions on the | |
| 62 | + * max physical address it can run at. Some architectures | |
| 63 | + * also have a preferred address, so first try to relocate | |
| 64 | + * to the preferred address. If that fails, allocate as low | |
| 65 | + * as possible while respecting the required alignment. | |
| 66 | + */ | |
| 67 | + nr_pages = round_up(alloc_size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; | |
| 68 | + status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | |
| 69 | + EFI_LOADER_DATA, nr_pages, &efi_addr); | |
| 70 | + new_addr = efi_addr; | |
| 71 | + /* | |
| 72 | + * If preferred address allocation failed allocate as low as | |
| 73 | + * possible. | |
| 74 | + */ | |
| 75 | + if (status != EFI_SUCCESS) { | |
| 76 | + status = efi_low_alloc_above(alloc_size, alignment, &new_addr, | |
| 77 | + min_addr); | |
| 78 | + } | |
| 79 | + if (status != EFI_SUCCESS) { | |
| 80 | + efi_err("Failed to allocate usable memory for kernel.\n"); | |
| 81 | + return status; | |
| 82 | + } | |
| 83 | + | |
| 84 | + /* | |
| 85 | + * We know source/dest won't overlap since both memory ranges | |
| 86 | + * have been allocated by UEFI, so we can safely use memcpy. | |
| 87 | + */ | |
| 88 | + memcpy((void *)new_addr, (void *)cur_image_addr, image_size); | |
| 89 | + efi_cache_sync_image(new_addr, image_size); | |
| 90 | + | |
| 91 | + /* Return the new address of the relocated image. */ | |
| 92 | + *image_addr = new_addr; | |
| 93 | + | |
| 94 | + return status; | |
| 95 | +} | |
| 96 | + | |
| 17 | 97 | efi_status_t handle_kernel_image(unsigned long *image_addr, |
| 18 | 98 | unsigned long *image_size, |
| 19 | 99 | unsigned long *reserve_addr, |
| @@ -18,6 +18,27 @@ efi_status_t check_platform_features(void) · 17 unmodified lines | ||
| 18 | 18 | return EFI_SUCCESS; |
| 19 | 19 | } |
| 20 | 20 | |
| 21 | +void efi_cache_sync_image(unsigned long image_base, unsigned long alloc_size) | |
| 22 | +{ | |
| 23 | + asm volatile ("ibar 0" ::: "memory"); | |
| 24 | +} | |
| 25 | + | |
| 26 | +unsigned long efi_get_kimg_kaslr_address(void) | |
| 27 | +{ | |
| 28 | + unsigned int random_offset = 0; | |
| 29 | + | |
| 30 | +#ifdef CONFIG_RANDOMIZE_BASE | |
| 31 | + if (!efi_nokaslr) { | |
| 32 | + efi_get_random_bytes(sizeof(random_offset), (u8 *)&random_offset); | |
| 33 | + random_offset ^= (random_get_entropy() << 16); | |
| 34 | + random_offset &= (CONFIG_RANDOMIZE_BASE_MAX_OFFSET - 1); | |
| 35 | + random_offset = ALIGN(random_offset + SZ_64K, SZ_64K); | |
| 36 | + } | |
| 37 | +#endif | |
| 38 | + | |
| 39 | + return PHYSADDR(VMLINUX_LOAD_ADDRESS) + random_offset; | |
| 40 | +} | |
| 41 | + | |
| 21 | 42 | struct exit_boot_struct { |
| 22 | 43 | efi_memory_desc_t *runtime_map; |
| 23 | 44 | int runtime_entry_count; |
| @@ -124,3 +124,85 @@ void efi_free(unsigned long size, unsigned long addr) · 123 unmodified lines | ||
| 124 | 124 | nr_pages = round_up(size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; |
| 125 | 125 | efi_bs_call(free_pages, addr, nr_pages); |
| 126 | 126 | } |
| 127 | + | |
| 128 | +/** | |
| 129 | + * efi_low_alloc_above() - allocate pages at or above given address | |
| 130 | + * @size: size of the memory area to allocate | |
| 131 | + * @align: minimum alignment of the allocated memory area. It should | |
| 132 | + * a power of two. | |
| 133 | + * @addr: on exit the address of the allocated memory | |
| 134 | + * @min: minimum address to used for the memory allocation | |
| 135 | + * | |
| 136 | + * Allocate at the lowest possible address that is not below @min as | |
| 137 | + * EFI_LOADER_DATA. The allocated pages are aligned according to @align but at | |
| 138 | + * least EFI_ALLOC_ALIGN. The first allocated page will not below the address | |
| 139 | + * given by @min. | |
| 140 | + * | |
| 141 | + * Return: status code | |
| 142 | + */ | |
| 143 | +efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, | |
| 144 | + unsigned long *addr, unsigned long min) | |
| 145 | +{ | |
| 146 | + struct efi_boot_memmap *map __free(efi_pool) = NULL; | |
| 147 | + efi_status_t status; | |
| 148 | + unsigned long nr_pages; | |
| 149 | + int i; | |
| 150 | + | |
| 151 | + status = efi_get_memory_map(&map, false); | |
| 152 | + if (status != EFI_SUCCESS) | |
| 153 | + return status; | |
| 154 | + | |
| 155 | + /* | |
| 156 | + * Enforce minimum alignment that EFI or Linux requires when | |
| 157 | + * requesting a specific address. We are doing page-based (or | |
| 158 | + * larger) allocations, and both the address and size must meet | |
| 159 | + * alignment constraints. | |
| 160 | + */ | |
| 161 | + if (align < EFI_ALLOC_ALIGN) | |
| 162 | + align = EFI_ALLOC_ALIGN; | |
| 163 | + | |
| 164 | + size = round_up(size, EFI_ALLOC_ALIGN); | |
| 165 | + nr_pages = size / EFI_PAGE_SIZE; | |
| 166 | + for (i = 0; i < map->map_size / map->desc_size; i++) { | |
| 167 | + efi_memory_desc_t *desc; | |
| 168 | + unsigned long m = (unsigned long)map->map; | |
| 169 | + u64 start, end; | |
| 170 | + | |
| 171 | + desc = efi_memdesc_ptr(m, map->desc_size, i); | |
| 172 | + | |
| 173 | + if (desc->type != EFI_CONVENTIONAL_MEMORY) | |
| 174 | + continue; | |
| 175 | + | |
| 176 | + if (desc->attribute & EFI_MEMORY_HOT_PLUGGABLE) | |
| 177 | + continue; | |
| 178 | + | |
| 179 | + if (efi_soft_reserve_enabled() && | |
| 180 | + (desc->attribute & EFI_MEMORY_SP)) | |
| 181 | + continue; | |
| 182 | + | |
| 183 | + if (desc->num_pages < nr_pages) | |
| 184 | + continue; | |
| 185 | + | |
| 186 | + start = desc->phys_addr; | |
| 187 | + end = start + desc->num_pages * EFI_PAGE_SIZE; | |
| 188 | + | |
| 189 | + if (start < min) | |
| 190 | + start = min; | |
| 191 | + | |
| 192 | + start = round_up(start, align); | |
| 193 | + if ((start + size) > end) | |
| 194 | + continue; | |
| 195 | + | |
| 196 | + status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | |
| 197 | + EFI_LOADER_DATA, nr_pages, &start); | |
| 198 | + if (status == EFI_SUCCESS) { | |
| 199 | + *addr = start; | |
| 200 | + break; | |
| 201 | + } | |
| 202 | + } | |
| 203 | + | |
| 204 | + if (i == map->map_size / map->desc_size) | |
| 205 | + return EFI_NOT_FOUND; | |
| 206 | + | |
| 207 | + return EFI_SUCCESS; | |
| 208 | +} | |
| @@ -1,166 +0,0 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0 | |
| 2 | − | |
| 3 | −#include <linux/efi.h> | |
| 4 | −#include <asm/efi.h> | |
| 5 | − | |
| 6 | −#include "efistub.h" | |
| 7 | − | |
| 8 | −/** | |
| 9 | − * efi_low_alloc_above() - allocate pages at or above given address | |
| 10 | − * @size: size of the memory area to allocate | |
| 11 | − * @align: minimum alignment of the allocated memory area. It should | |
| 12 | − * a power of two. | |
| 13 | − * @addr: on exit the address of the allocated memory | |
| 14 | − * @min: minimum address to used for the memory allocation | |
| 15 | − * | |
| 16 | − * Allocate at the lowest possible address that is not below @min as | |
| 17 | − * EFI_LOADER_DATA. The allocated pages are aligned according to @align but at | |
| 18 | − * least EFI_ALLOC_ALIGN. The first allocated page will not below the address | |
| 19 | − * given by @min. | |
| 20 | − * | |
| 21 | − * Return: status code | |
| 22 | − */ | |
| 23 | −efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, | |
| 24 | − unsigned long *addr, unsigned long min) | |
| 25 | −{ | |
| 26 | − struct efi_boot_memmap *map __free(efi_pool) = NULL; | |
| 27 | − efi_status_t status; | |
| 28 | − unsigned long nr_pages; | |
| 29 | − int i; | |
| 30 | − | |
| 31 | − status = efi_get_memory_map(&map, false); | |
| 32 | − if (status != EFI_SUCCESS) | |
| 33 | − return status; | |
| 34 | − | |
| 35 | − /* | |
| 36 | − * Enforce minimum alignment that EFI or Linux requires when | |
| 37 | − * requesting a specific address. We are doing page-based (or | |
| 38 | − * larger) allocations, and both the address and size must meet | |
| 39 | − * alignment constraints. | |
| 40 | − */ | |
| 41 | − if (align < EFI_ALLOC_ALIGN) | |
| 42 | − align = EFI_ALLOC_ALIGN; | |
| 43 | − | |
| 44 | − size = round_up(size, EFI_ALLOC_ALIGN); | |
| 45 | − nr_pages = size / EFI_PAGE_SIZE; | |
| 46 | − for (i = 0; i < map->map_size / map->desc_size; i++) { | |
| 47 | − efi_memory_desc_t *desc; | |
| 48 | − unsigned long m = (unsigned long)map->map; | |
| 49 | − u64 start, end; | |
| 50 | − | |
| 51 | − desc = efi_memdesc_ptr(m, map->desc_size, i); | |
| 52 | − | |
| 53 | − if (desc->type != EFI_CONVENTIONAL_MEMORY) | |
| 54 | − continue; | |
| 55 | − | |
| 56 | − if (desc->attribute & EFI_MEMORY_HOT_PLUGGABLE) | |
| 57 | − continue; | |
| 58 | − | |
| 59 | − if (efi_soft_reserve_enabled() && | |
| 60 | − (desc->attribute & EFI_MEMORY_SP)) | |
| 61 | − continue; | |
| 62 | − | |
| 63 | − if (desc->num_pages < nr_pages) | |
| 64 | − continue; | |
| 65 | − | |
| 66 | − start = desc->phys_addr; | |
| 67 | − end = start + desc->num_pages * EFI_PAGE_SIZE; | |
| 68 | − | |
| 69 | − if (start < min) | |
| 70 | − start = min; | |
| 71 | − | |
| 72 | − start = round_up(start, align); | |
| 73 | − if ((start + size) > end) | |
| 74 | − continue; | |
| 75 | − | |
| 76 | − status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | |
| 77 | − EFI_LOADER_DATA, nr_pages, &start); | |
| 78 | − if (status == EFI_SUCCESS) { | |
| 79 | − *addr = start; | |
| 80 | − break; | |
| 81 | − } | |
| 82 | − } | |
| 83 | − | |
| 84 | − if (i == map->map_size / map->desc_size) | |
| 85 | − return EFI_NOT_FOUND; | |
| 86 | − | |
| 87 | − return EFI_SUCCESS; | |
| 88 | −} | |
| 89 | − | |
| 90 | −/** | |
| 91 | − * efi_relocate_kernel() - copy memory area | |
| 92 | − * @image_addr: pointer to address of memory area to copy | |
| 93 | − * @image_size: size of memory area to copy | |
| 94 | − * @alloc_size: minimum size of memory to allocate, must be greater or | |
| 95 | − * equal to image_size | |
| 96 | − * @preferred_addr: preferred target address | |
| 97 | − * @alignment: minimum alignment of the allocated memory area. It | |
| 98 | − * should be a power of two. | |
| 99 | − * @min_addr: minimum target address | |
| 100 | − * | |
| 101 | − * Copy a memory area to a newly allocated memory area aligned according | |
| 102 | − * to @alignment but at least EFI_ALLOC_ALIGN. If the preferred address | |
| 103 | − * is not available, the allocated address will not be below @min_addr. | |
| 104 | − * On exit, @image_addr is updated to the target copy address that was used. | |
| 105 | − * | |
| 106 | − * This function is used to copy the Linux kernel verbatim. It does not apply | |
| 107 | − * any relocation changes. | |
| 108 | − * | |
| 109 | − * Return: status code | |
| 110 | − */ | |
| 111 | −efi_status_t efi_relocate_kernel(unsigned long *image_addr, | |
| 112 | − unsigned long image_size, | |
| 113 | − unsigned long alloc_size, | |
| 114 | − unsigned long preferred_addr, | |
| 115 | − unsigned long alignment, | |
| 116 | − unsigned long min_addr) | |
| 117 | −{ | |
| 118 | − unsigned long cur_image_addr; | |
| 119 | − unsigned long new_addr = 0; | |
| 120 | − efi_status_t status; | |
| 121 | − unsigned long nr_pages; | |
| 122 | − efi_physical_addr_t efi_addr = preferred_addr; | |
| 123 | − | |
| 124 | − if (!image_addr || !image_size || !alloc_size) | |
| 125 | − return EFI_INVALID_PARAMETER; | |
| 126 | − if (alloc_size < image_size) | |
| 127 | − return EFI_INVALID_PARAMETER; | |
| 128 | − | |
| 129 | − cur_image_addr = *image_addr; | |
| 130 | − | |
| 131 | − /* | |
| 132 | − * The EFI firmware loader could have placed the kernel image | |
| 133 | − * anywhere in memory, but the kernel has restrictions on the | |
| 134 | − * max physical address it can run at. Some architectures | |
| 135 | − * also have a preferred address, so first try to relocate | |
| 136 | − * to the preferred address. If that fails, allocate as low | |
| 137 | − * as possible while respecting the required alignment. | |
| 138 | − */ | |
| 139 | − nr_pages = round_up(alloc_size, EFI_ALLOC_ALIGN) / EFI_PAGE_SIZE; | |
| 140 | − status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS, | |
| 141 | − EFI_LOADER_DATA, nr_pages, &efi_addr); | |
| 142 | − new_addr = efi_addr; | |
| 143 | − /* | |
| 144 | − * If preferred address allocation failed allocate as low as | |
| 145 | − * possible. | |
| 146 | − */ | |
| 147 | − if (status != EFI_SUCCESS) { | |
| 148 | − status = efi_low_alloc_above(alloc_size, alignment, &new_addr, | |
| 149 | − min_addr); | |
| 150 | − } | |
| 151 | − if (status != EFI_SUCCESS) { | |
| 152 | − efi_err("Failed to allocate usable memory for kernel.\n"); | |
| 153 | − return status; | |
| 154 | − } | |
| 155 | − | |
| 156 | − /* | |
| 157 | − * We know source/dest won't overlap since both memory ranges | |
| 158 | − * have been allocated by UEFI, so we can safely use memcpy. | |
| 159 | − */ | |
| 160 | − memcpy((void *)new_addr, (void *)cur_image_addr, image_size); | |
| 161 | − | |
| 162 | − /* Return the new address of the relocated image. */ | |
| 163 | − *image_addr = new_addr; | |
| 164 | − | |
| 165 | − return status; | |
| 166 | −} | |
| @@ -311,11 +311,14 @@ static const struct dmi_system_id efifb_dmi_swap_width_height[] __initconst = { · 310 unmodified lines | ||
| 311 | 311 | .callback = efifb_swap_width_height, |
| 312 | 312 | }, |
| 313 | 313 | { |
| 314 | − /* Lenovo IdeaPad Duet 3 10IGL5 with 1200x1920 portrait screen */ | |
| 314 | + /* | |
| 315 | + * Lenovo IdeaPad Duet 3 10IGL5 and 10IGL5-LTE with | |
| 316 | + * 1200x1920 portrait screen | |
| 317 | + */ | |
| 315 | 318 | .matches = { |
| 316 | 319 | DMI_EXACT_MATCH(DMI_SYS_VENDOR, "LENOVO"), |
| 317 | − DMI_EXACT_MATCH(DMI_PRODUCT_VERSION, | |
| 318 | − "IdeaPad Duet 3 10IGL5"), | |
| 320 | + /* Non exact match to also match the LTE version */ | |
| 321 | + DMI_MATCH(DMI_PRODUCT_VERSION, "IdeaPad Duet 3 10IGL5"), | |
| 319 | 322 | }, |
| 320 | 323 | .callback = efifb_swap_width_height, |
| 321 | 324 | }, |
| @@ -539,12 +539,22 @@ static int psci_system_suspend(unsigned long unused) · 538 unmodified lines | ||
| 539 | 539 | |
| 540 | 540 | static int psci_system_suspend_enter(suspend_state_t state) |
| 541 | 541 | { |
| 542 | + pm_set_resume_via_firmware(); | |
| 543 | + | |
| 542 | 544 | return cpu_suspend(0, psci_system_suspend); |
| 543 | 545 | } |
| 544 | 546 | |
| 547 | +static int psci_system_suspend_begin(suspend_state_t state) | |
| 548 | +{ | |
| 549 | + pm_set_suspend_via_firmware(); | |
| 550 | + | |
| 551 | + return 0; | |
| 552 | +} | |
| 553 | + | |
| 545 | 554 | static const struct platform_suspend_ops psci_suspend_ops = { |
| 546 | 555 | .valid = suspend_valid_only_mem, |
| 547 | 556 | .enter = psci_system_suspend_enter, |
| 557 | + .begin = psci_system_suspend_begin, | |
| 548 | 558 | }; |
| 549 | 559 | |
| 550 | 560 | static void __init psci_init_system_reset2(void) |
| @@ -362,6 +362,9 @@ static void *pdsfc_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, · 361 unmodified lines | ||
| 362 | 362 | void *out = NULL; |
| 363 | 363 | int err; |
| 364 | 364 | |
| 365 | + if (in_len < sizeof(*rpc)) | |
| 366 | + return ERR_PTR(-EINVAL); | |
| 367 | + | |
| 365 | 368 | err = pdsfc_validate_rpc(pdsfc, rpc, scope); |
| 366 | 369 | if (err) |
| 367 | 370 | return ERR_PTR(err); |
| @@ -237,7 +237,9 @@ static irqreturn_t adnp_irq(int irq, void *data) · 236 unmodified lines | ||
| 237 | 237 | unsigned long pending; |
| 238 | 238 | int err; |
| 239 | 239 | |
| 240 | − scoped_guard(mutex, &adnp->i2c_lock) { | |
| 240 | + { | |
| 241 | + guard(mutex)(&adnp->i2c_lock); | |
| 242 | + | |
| 241 | 243 | err = adnp_read(adnp, GPIO_PLR(adnp) + i, &level); |
| 242 | 244 | if (err < 0) |
| 243 | 245 | continue; |
| @@ -968,9 +968,12 @@ static int gpio_aggregator_activate(struct gpio_aggregator *aggr) · 967 unmodified lines | ||
| 968 | 968 | } |
| 969 | 969 | |
| 970 | 970 | wait_for_device_probe(); |
| 971 | − if (!device_is_bound(&pdev->dev)) { | |
| 972 | − ret = -ENXIO; | |
| 973 | − goto err_unregister_pdev; | |
| 971 | + | |
| 972 | + scoped_guard(device, &pdev->dev) { | |
| 973 | + if (!device_is_bound(&pdev->dev)) { | |
| 974 | + ret = -ENXIO; | |
| 975 | + goto err_unregister_pdev; | |
| 976 | + } | |
| 974 | 977 | } |
| 975 | 978 | |
| 976 | 979 | aggr->pdev = pdev; |
| @@ -979,8 +982,8 @@ static int gpio_aggregator_activate(struct gpio_aggregator *aggr) · 2 unmodified lines | ||
| 979 | 982 | err_unregister_pdev: |
| 980 | 983 | platform_device_unregister(pdev); |
| 981 | 984 | err_remove_lookup_table: |
| 982 | − kfree(aggr->lookups->dev_id); | |
| 983 | 985 | gpiod_remove_lookup_table(aggr->lookups); |
| 986 | + kfree(aggr->lookups->dev_id); | |
| 984 | 987 | err_remove_swnode: |
| 985 | 988 | fwnode_remove_software_node(swnode); |
| 986 | 989 | err_remove_lookups: |
| @@ -991,11 +994,15 @@ err_remove_lookups: · 4 unmodified lines | ||
| 991 | 994 | |
| 992 | 995 | static void gpio_aggregator_deactivate(struct gpio_aggregator *aggr) |
| 993 | 996 | { |
| 997 | + struct fwnode_handle *swnode; | |
| 998 | + | |
| 999 | + swnode = dev_fwnode(&aggr->pdev->dev); | |
| 994 | 1000 | platform_device_unregister(aggr->pdev); |
| 995 | 1001 | aggr->pdev = NULL; |
| 996 | 1002 | gpiod_remove_lookup_table(aggr->lookups); |
| 997 | 1003 | kfree(aggr->lookups->dev_id); |
| 998 | 1004 | kfree(aggr->lookups); |
| 1005 | + fwnode_remove_software_node(swnode); | |
| 999 | 1006 | } |
| 1000 | 1007 | |
| 1001 | 1008 | static void gpio_aggregator_lockup_configfs(struct gpio_aggregator *aggr, |
| @@ -469,7 +469,7 @@ static int mxc_gpio_probe(struct platform_device *pdev) · 468 unmodified lines | ||
| 469 | 469 | * the handler is needed only once, but doing it for every port |
| 470 | 470 | * is more robust and easier. |
| 471 | 471 | */ |
| 472 | − port->irq_high = -1; | |
| 472 | + port->irq_high = 0; | |
| 473 | 473 | port->mx_irq_handler = mx2_gpio_irq_handler; |
| 474 | 474 | } else |
| 475 | 475 | port->mx_irq_handler = mx3_gpio_irq_handler; |
| @@ -1411,7 +1411,7 @@ static int pca953x_resume(struct device *dev) · 1410 unmodified lines | ||
| 1411 | 1411 | ret = regulator_enable(chip->regulator); |
| 1412 | 1412 | if (ret) { |
| 1413 | 1413 | dev_err(dev, "Failed to enable regulator: %d\n", ret); |
| 1414 | − return 0; | |
| 1414 | + return ret; | |
| 1415 | 1415 | } |
| 1416 | 1416 | } |
| 1417 | 1417 | |
| @@ -638,10 +638,17 @@ fail: · 637 unmodified lines | ||
| 638 | 638 | return ret; |
| 639 | 639 | } |
| 640 | 640 | |
| 641 | +static void rockchip_clk_put(void *data) | |
| 642 | +{ | |
| 643 | + struct clk *clk = data; | |
| 644 | + | |
| 645 | + clk_put(clk); | |
| 646 | +} | |
| 647 | + | |
| 641 | 648 | static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) |
| 642 | 649 | { |
| 643 | 650 | struct resource res; |
| 644 | − int id = 0; | |
| 651 | + int id = 0, ret; | |
| 645 | 652 | |
| 646 | 653 | if (of_address_to_resource(bank->of_node, 0, &res)) { |
| 647 | 654 | dev_err(bank->dev, "cannot find IO resource for bank\n"); |
| @@ -656,11 +663,10 @@ static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) · 8 unmodified lines | ||
| 656 | 663 | if (!bank->irq) |
| 657 | 664 | return -EINVAL; |
| 658 | 665 | |
| 659 | − bank->clk = of_clk_get(bank->of_node, 0); | |
| 666 | + bank->clk = devm_clk_get_enabled(bank->dev, NULL); | |
| 660 | 667 | if (IS_ERR(bank->clk)) |
| 661 | 668 | return PTR_ERR(bank->clk); |
| 662 | 669 | |
| 663 | − clk_prepare_enable(bank->clk); | |
| 664 | 670 | id = readl(bank->reg_base + gpio_regs_v2.version_id); |
| 665 | 671 | |
| 666 | 672 | switch (id) { |
| @@ -672,9 +678,13 @@ static int rockchip_get_bank_data(struct rockchip_pin_bank *bank) · 5 unmodified lines | ||
| 672 | 678 | bank->db_clk = of_clk_get(bank->of_node, 1); |
| 673 | 679 | if (IS_ERR(bank->db_clk)) { |
| 674 | 680 | dev_err(bank->dev, "cannot find debounce clk\n"); |
| 675 | − clk_disable_unprepare(bank->clk); | |
| 676 | 681 | return -EINVAL; |
| 677 | 682 | } |
| 683 | + | |
| 684 | + ret = devm_add_action_or_reset(bank->dev, rockchip_clk_put, | |
| 685 | + bank->db_clk); | |
| 686 | + if (ret) | |
| 687 | + return ret; | |
| 678 | 688 | break; |
| 679 | 689 | case GPIO_TYPE_V1: |
| 680 | 690 | bank->gpio_regs = &gpio_regs_v1; |
| @@ -751,7 +761,6 @@ static int rockchip_gpio_probe(struct platform_device *pdev) · 70 unmodified lines | ||
| 751 | 761 | |
| 752 | 762 | ret = rockchip_gpiolib_register(bank); |
| 753 | 763 | if (ret) { |
| 754 | − clk_disable_unprepare(bank->clk); | |
| 755 | 764 | mutex_unlock(&bank->deferred_lock); |
| 756 | 765 | return ret; |
| 757 | 766 | } |
| @@ -792,7 +801,9 @@ static void rockchip_gpio_remove(struct platform_device *pdev) · 34 unmodified lines | ||
| 792 | 801 | { |
| 793 | 802 | struct rockchip_pin_bank *bank = platform_get_drvdata(pdev); |
| 794 | 803 | |
| 795 | − clk_disable_unprepare(bank->clk); | |
| 804 | + irq_set_chained_handler_and_data(bank->irq, NULL, NULL); | |
| 805 | + if (bank->domain) | |
| 806 | + irq_domain_remove(bank->domain); | |
| 796 | 807 | gpiochip_remove(&bank->gpio_chip); |
| 797 | 808 | } |
| 798 | 809 | |
| @@ -103,9 +103,18 @@ static void gpio_shared_proxy_free(struct gpio_chip *gc, unsigned int offset) · 102 unmodified lines | ||
| 103 | 103 | { |
| 104 | 104 | struct gpio_shared_proxy_data *proxy = gpiochip_get_data(gc); |
| 105 | 105 | struct gpio_shared_desc *shared_desc = proxy->shared_desc; |
| 106 | + int ret; | |
| 106 | 107 | |
| 107 | 108 | guard(gpio_shared_desc_lock)(shared_desc); |
| 108 | 109 | |
| 110 | + if (proxy->voted_high) { | |
| 111 | + ret = gpio_shared_proxy_set_unlocked(proxy, | |
| 112 | + shared_desc->can_sleep ? gpiod_set_value_cansleep : gpiod_set_value, 0); | |
| 113 | + if (ret) | |
| 114 | + dev_err(proxy->dev, | |
| 115 | + "Failed to unset the shared GPIO value on release: %d\n", ret); | |
| 116 | + } | |
| 117 | + | |
| 109 | 118 | proxy->shared_desc->usecnt--; |
| 110 | 119 | |
| 111 | 120 | dev_dbg(proxy->dev, "Shared GPIO freed, number of users: %u\n", |
| @@ -901,7 +901,7 @@ static int gpio_sim_device_activate(struct gpio_sim_device *dev) · 900 unmodified lines | ||
| 901 | 901 | struct platform_device *pdev; |
| 902 | 902 | struct fwnode_handle *swnode; |
| 903 | 903 | struct gpio_sim_bank *bank; |
| 904 | − int ret; | |
| 904 | + int ret = 0; | |
| 905 | 905 | |
| 906 | 906 | lockdep_assert_held(&dev->lock); |
| 907 | 907 | |
| @@ -945,9 +945,12 @@ static int gpio_sim_device_activate(struct gpio_sim_device *dev) · 37 unmodified lines | ||
| 945 | 945 | } |
| 946 | 946 | |
| 947 | 947 | wait_for_device_probe(); |
| 948 | − if (!device_is_bound(&pdev->dev)) { | |
| 949 | − ret = -ENXIO; | |
| 950 | − goto err_unregister_pdev; | |
| 948 | + | |
| 949 | + scoped_guard(device, &pdev->dev) { | |
| 950 | + if (!device_is_bound(&pdev->dev)) { | |
| 951 | + ret = -ENXIO; | |
| 952 | + goto err_unregister_pdev; | |
| 953 | + } | |
| 951 | 954 | } |
| 952 | 955 | |
| 953 | 956 | dev->pdev = pdev; |
| @@ -397,7 +397,7 @@ static ssize_t gpio_virtuser_direction_do_write(struct file *file, · 396 unmodified lines | ||
| 397 | 397 | char buf[32], *trimmed; |
| 398 | 398 | int ret, dir, val = 0; |
| 399 | 399 | |
| 400 | − if (count >= sizeof(buf)) | |
| 400 | + if (*ppos != 0 || count >= sizeof(buf)) | |
| 401 | 401 | return -EINVAL; |
| 402 | 402 | |
| 403 | 403 | ret = simple_write_to_buffer(buf, sizeof(buf) - 1, ppos, user_buf, count); |
| @@ -622,7 +622,7 @@ static ssize_t gpio_virtuser_consumer_write(struct file *file, · 218 unmodified lines | ||
| 622 | 622 | char buf[GPIO_VIRTUSER_NAME_BUF_LEN + 2]; |
| 623 | 623 | int ret; |
| 624 | 624 | |
| 625 | − if (count >= sizeof(buf)) | |
| 625 | + if (*ppos != 0 || count >= sizeof(buf)) | |
| 626 | 626 | return -EINVAL; |
| 627 | 627 | |
| 628 | 628 | ret = simple_write_to_buffer(buf, GPIO_VIRTUSER_NAME_BUF_LEN, ppos, |
| @@ -1477,9 +1477,12 @@ gpio_virtuser_device_activate(struct gpio_virtuser_device *dev) · 848 unmodified lines | ||
| 1477 | 1477 | } |
| 1478 | 1478 | |
| 1479 | 1479 | wait_for_device_probe(); |
| 1480 | − if (!device_is_bound(&pdev->dev)) { | |
| 1481 | − ret = -ENXIO; | |
| 1482 | − goto err_unregister_pdev; | |
| 1480 | + | |
| 1481 | + scoped_guard(device, &pdev->dev) { | |
| 1482 | + if (!device_is_bound(&pdev->dev)) { | |
| 1483 | + ret = -ENXIO; | |
| 1484 | + goto err_unregister_pdev; | |
| 1485 | + } | |
| 1483 | 1486 | } |
| 1484 | 1487 | |
| 1485 | 1488 | dev->pdev = pdev; |
| @@ -1184,6 +1184,7 @@ static int gpio_v2_line_flags_validate(u64 flags) · 1183 unmodified lines | ||
| 1184 | 1184 | static int gpio_v2_line_config_validate(struct gpio_v2_line_config *lc, |
| 1185 | 1185 | unsigned int num_lines) |
| 1186 | 1186 | { |
| 1187 | + size_t unused_attrs; | |
| 1187 | 1188 | unsigned int i; |
| 1188 | 1189 | u64 flags; |
| 1189 | 1190 | int ret; |
| @@ -1191,9 +1192,21 @@ static int gpio_v2_line_config_validate(struct gpio_v2_line_config *lc, · 1 unmodified line | ||
| 1191 | 1192 | if (lc->num_attrs > GPIO_V2_LINE_NUM_ATTRS_MAX) |
| 1192 | 1193 | return -EINVAL; |
| 1193 | 1194 | |
| 1195 | + unused_attrs = GPIO_V2_LINE_NUM_ATTRS_MAX - lc->num_attrs; | |
| 1196 | + | |
| 1194 | 1197 | if (!mem_is_zero(lc->padding, sizeof(lc->padding))) |
| 1195 | 1198 | return -EINVAL; |
| 1196 | 1199 | |
| 1200 | + for (i = 0; i < lc->num_attrs; i++) { | |
| 1201 | + if (lc->attrs[i].attr.padding != 0) | |
| 1202 | + return -EINVAL; | |
| 1203 | + } | |
| 1204 | + | |
| 1205 | + if (unused_attrs) { | |
| 1206 | + if (!mem_is_zero(&lc->attrs[lc->num_attrs], unused_attrs * sizeof(*lc->attrs))) | |
| 1207 | + return -EINVAL; | |
| 1208 | + } | |
| 1209 | + | |
| 1197 | 1210 | for (i = 0; i < num_lines; i++) { |
| 1198 | 1211 | flags = gpio_v2_line_config_flags(lc, i); |
| 1199 | 1212 | ret = gpio_v2_line_flags_validate(flags); |
| @@ -53,7 +53,7 @@ struct gpio_shared_entry { · 52 unmodified lines | ||
| 53 | 53 | unsigned int offset; |
| 54 | 54 | /* Index in the property value array. */ |
| 55 | 55 | size_t index; |
| 56 | − /* Synchronizes the modification of shared_desc. */ | |
| 56 | + /* Synchronizes the modification of shared_desc and offset. */ | |
| 57 | 57 | struct mutex lock; |
| 58 | 58 | struct gpio_shared_desc *shared_desc; |
| 59 | 59 | struct kref ref; |
| @@ -598,16 +598,13 @@ void gpio_device_teardown_shared(struct gpio_device *gdev) · 538 unmodified lines | ||
| 598 | 598 | struct gpio_shared_ref *ref; |
| 599 | 599 | |
| 600 | 600 | list_for_each_entry(entry, &gpio_shared_list, list) { |
| 601 | − guard(mutex)(&entry->lock); | |
| 602 | − | |
| 603 | 601 | if (!device_match_fwnode(&gdev->dev, entry->fwnode)) |
| 604 | 602 | continue; |
| 605 | 603 | |
| 606 | − gpiod_free_commit(&gdev->descs[entry->offset]); | |
| 604 | + scoped_guard(mutex, &entry->lock) | |
| 605 | + gpiod_free_commit(&gdev->descs[entry->offset]); | |
| 607 | 606 | |
| 608 | 607 | list_for_each_entry(ref, &entry->refs, list) { |
| 609 | − guard(mutex)(&ref->lock); | |
| 610 | − | |
| 611 | 608 | if (ref->lookup) { |
| 612 | 609 | gpiod_remove_lookup_table(ref->lookup); |
| 613 | 610 | kfree(ref->lookup->table[0].key); |
| @@ -1190,7 +1190,6 @@ struct amdgpu_device { · 1189 unmodified lines | ||
| 1190 | 1190 | bool apu_prefer_gtt; |
| 1191 | 1191 | |
| 1192 | 1192 | bool userq_halt_for_enforce_isolation; |
| 1193 | − struct work_struct userq_reset_work; | |
| 1194 | 1193 | struct amdgpu_uid *uid_info; |
| 1195 | 1194 | |
| 1196 | 1195 | struct amdgpu_uma_carveout_info uma_info; |
| @@ -36,6 +36,9 @@ · 35 unmodified lines | ||
| 36 | 36 | #include "amdgpu_ras.h" |
| 37 | 37 | #include "amdgpu_umc.h" |
| 38 | 38 | #include "amdgpu_reset.h" |
| 39 | +#if IS_ENABLED(CONFIG_HSA_AMD) | |
| 40 | +#include "kfd_priv.h" | |
| 41 | +#endif | |
| 39 | 42 | |
| 40 | 43 | /* Total memory size in system memory and all GPU VRAM. Used to |
| 41 | 44 | * estimate worst case amount of memory to reserve for page tables |
| @@ -320,6 +323,28 @@ void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev) · 278 unmodified lines | ||
| 320 | 323 | (void)amdgpu_reset_domain_schedule(adev->reset_domain, &adev->kfd.reset_work); |
| 321 | 324 | } |
| 322 | 325 | |
| 326 | +void amdgpu_amdkfd_clear_kfd_mapping(struct amdgpu_device *adev) | |
| 327 | +{ | |
| 328 | +#if IS_ENABLED(CONFIG_HSA_AMD) | |
| 329 | + struct kfd_dev *kfd = adev->kfd.dev; | |
| 330 | + unsigned int i; | |
| 331 | + | |
| 332 | + if (!kfd) | |
| 333 | + return; | |
| 334 | + | |
| 335 | + for (i = 0; i < kfd->num_nodes; i++) { | |
| 336 | + struct kfd_node *node = kfd->nodes[i]; | |
| 337 | + | |
| 338 | + kfd_dev_unmap_mapping_range(KFD_MMAP_TYPE_DOORBELL | | |
| 339 | + KFD_MMAP_GPU_ID(node->id), | |
| 340 | + kfd_doorbell_process_slice(kfd)); | |
| 341 | + kfd_dev_unmap_mapping_range(KFD_MMAP_TYPE_MMIO | | |
| 342 | + KFD_MMAP_GPU_ID(node->id), | |
| 343 | + PAGE_SIZE); | |
| 344 | + } | |
| 345 | +#endif | |
| 346 | +} | |
| 347 | + | |
| 323 | 348 | int amdgpu_amdkfd_alloc_kernel_mem(struct amdgpu_device *adev, size_t size, |
| 324 | 349 | u32 domain, void **mem_obj, uint64_t *gpu_addr, |
| 325 | 350 | void **cpu_ptr, bool cp_mqd_gfx9) |
| @@ -358,6 +358,7 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, · 357 unmodified lines | ||
| 358 | 358 | uint64_t size, u32 alloc_flag, int8_t xcp_id); |
| 359 | 359 | void amdgpu_amdkfd_unreserve_mem_limit(struct amdgpu_device *adev, |
| 360 | 360 | uint64_t size, u32 alloc_flag, int8_t xcp_id); |
| 361 | +void amdgpu_amdkfd_clear_kfd_mapping(struct amdgpu_device *adev); | |
| 361 | 362 | |
| 362 | 363 | u64 amdgpu_amdkfd_xcp_memory_size(struct amdgpu_device *adev, int xcp_id); |
| 363 | 364 | |
| @@ -2839,8 +2839,12 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 2838 unmodified lines | ||
| 2839 | 2839 | * that checks whether the PSP is running. A solution for those issues |
| 2840 | 2840 | * in the APU is to trigger a GPU reset, but this should be done during |
| 2841 | 2841 | * the unload phase to avoid adding boot latency and screen flicker. |
| 2842 | + * GFX V11 has GC block as default off IP. Every time AMDGPU driver sends | |
| 2843 | + * a request to PMFW to unload MP1, PMFW will put GC in reset and power down | |
| 2844 | + * the voltage. Hence, skipping reset for APUs with GFX V11 or later. | |
| 2842 | 2845 | */ |
| 2843 | − if ((adev->flags & AMD_IS_APU) && !adev->gmc.is_app_apu) { | |
| 2846 | + if ((adev->flags & AMD_IS_APU) && !adev->gmc.is_app_apu && | |
| 2847 | + amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 0, 0)) { | |
| 2844 | 2848 | r = amdgpu_asic_reset(adev); |
| 2845 | 2849 | if (r) |
| 2846 | 2850 | dev_err(adev->dev, "asic reset on %s failed\n", __func__); |
| @@ -3783,7 +3787,6 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 936 unmodified lines | ||
| 3783 | 3787 | } |
| 3784 | 3788 | |
| 3785 | 3789 | INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); |
| 3786 | − INIT_WORK(&adev->userq_reset_work, amdgpu_userq_reset_work); | |
| 3787 | 3790 | |
| 3788 | 3791 | amdgpu_coredump_init(adev); |
| 3789 | 3792 | |
| @@ -5474,7 +5477,7 @@ static inline void amdgpu_device_stop_pending_resets(struct amdgpu_device *adev) · 1684 unmodified lines | ||
| 5474 | 5477 | if (!amdgpu_sriov_vf(adev)) |
| 5475 | 5478 | cancel_work(&adev->reset_work); |
| 5476 | 5479 | #endif |
| 5477 | − cancel_work(&adev->userq_reset_work); | |
| 5480 | + amdgpu_userq_mgr_cancel_reset_work(adev); | |
| 5478 | 5481 | |
| 5479 | 5482 | if (adev->kfd.dev) |
| 5480 | 5483 | cancel_work(&adev->kfd.reset_work); |
| @@ -5832,6 +5835,12 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 351 unmodified lines | ||
| 5832 | 5835 | /* We need to lock reset domain only once both for XGMI and single device */ |
| 5833 | 5836 | amdgpu_device_recovery_get_reset_lock(adev, &device_list); |
| 5834 | 5837 | |
| 5838 | + /* unmap all the mappings of doorbell and framebuffer to prevent user space from | |
| 5839 | + * accessing them | |
| 5840 | + */ | |
| 5841 | + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); | |
| 5842 | + amdgpu_amdkfd_clear_kfd_mapping(adev); | |
| 5843 | + | |
| 5835 | 5844 | amdgpu_device_halt_activities(adev, job, reset_context, &device_list, |
| 5836 | 5845 | hive, need_emergency_restart); |
| 5837 | 5846 | if (need_emergency_restart) |
| @@ -304,7 +304,7 @@ static int amdgpu_discovery_get_tmr_info(struct amdgpu_device *adev, · 303 unmodified lines | ||
| 304 | 304 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].offset; |
| 305 | 305 | adev->discovery.size = |
| 306 | 306 | adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_IPD_TABLE_ID].size_kb << 10; |
| 307 | − if (!adev->discovery.offset || !adev->discovery.size) | |
| 307 | + if (!adev->discovery.size) | |
| 308 | 308 | return -EINVAL; |
| 309 | 309 | } else { |
| 310 | 310 | goto out; |
| @@ -3090,10 +3090,8 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 2779 unmodified lines | ||
| 3090 | 3090 | case IP_VERSION(11, 5, 1): |
| 3091 | 3091 | case IP_VERSION(11, 5, 2): |
| 3092 | 3092 | case IP_VERSION(11, 5, 3): |
| 3093 | − adev->family = AMDGPU_FAMILY_GC_11_5_0; | |
| 3094 | − break; | |
| 3095 | 3093 | case IP_VERSION(11, 5, 4): |
| 3096 | − adev->family = AMDGPU_FAMILY_GC_11_5_4; | |
| 3094 | + adev->family = AMDGPU_FAMILY_GC_11_5_0; | |
| 3097 | 3095 | break; |
| 3098 | 3096 | case IP_VERSION(12, 0, 0): |
| 3099 | 3097 | case IP_VERSION(12, 0, 1): |
| @@ -3149,17 +3149,15 @@ static int __init amdgpu_init(void) · 3148 unmodified lines | ||
| 3149 | 3149 | |
| 3150 | 3150 | r = amdgpu_sync_init(); |
| 3151 | 3151 | if (r) |
| 3152 | − goto error_sync; | |
| 3153 | − | |
| 3154 | − r = amdgpu_userq_fence_slab_init(); | |
| 3155 | − if (r) | |
| 3156 | − goto error_fence; | |
| 3152 | + return r; | |
| 3157 | 3153 | |
| 3158 | 3154 | amdgpu_register_atpx_handler(); |
| 3159 | 3155 | amdgpu_acpi_detect(); |
| 3160 | 3156 | |
| 3161 | − /* Ignore KFD init failures. Normal when CONFIG_HSA_AMD is not set. */ | |
| 3162 | − amdgpu_amdkfd_init(); | |
| 3157 | + /* Ignore KFD init failures when CONFIG_HSA_AMD is not set. */ | |
| 3158 | + r = amdgpu_amdkfd_init(); | |
| 3159 | + if (r && r != -ENOENT) | |
| 3160 | + goto error_fini_sync; | |
| 3163 | 3161 | |
| 3164 | 3162 | if (amdgpu_pp_feature_mask & PP_OVERDRIVE_MASK) { |
| 3165 | 3163 | add_taint(TAINT_CPU_OUT_OF_SPEC, LOCKDEP_STILL_OK); |
| @@ -3170,10 +3168,8 @@ static int __init amdgpu_init(void) · 4 unmodified lines | ||
| 3170 | 3168 | /* let modprobe override vga console setting */ |
| 3171 | 3169 | return pci_register_driver(&amdgpu_kms_pci_driver); |
| 3172 | 3170 | |
| 3173 | −error_fence: | |
| 3171 | +error_fini_sync: | |
| 3174 | 3172 | amdgpu_sync_fini(); |
| 3175 | − | |
| 3176 | −error_sync: | |
| 3177 | 3173 | return r; |
| 3178 | 3174 | } |
| 3179 | 3175 | |
| @@ -3184,7 +3180,6 @@ static void __exit amdgpu_exit(void) · 4 unmodified lines | ||
| 3184 | 3180 | amdgpu_unregister_atpx_handler(); |
| 3185 | 3181 | amdgpu_acpi_release(); |
| 3186 | 3182 | amdgpu_sync_fini(); |
| 3187 | − amdgpu_userq_fence_slab_fini(); | |
| 3188 | 3183 | mmu_notifier_synchronize(); |
| 3189 | 3184 | amdgpu_xcp_drv_release(); |
| 3190 | 3185 | } |
| @@ -262,12 +262,19 @@ void amdgpu_gart_table_ram_free(struct amdgpu_device *adev) · 261 unmodified lines | ||
| 262 | 262 | */ |
| 263 | 263 | int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev) |
| 264 | 264 | { |
| 265 | + int r; | |
| 266 | + | |
| 265 | 267 | if (adev->gart.bo != NULL) |
| 266 | 268 | return 0; |
| 267 | 269 | |
| 268 | − return amdgpu_bo_create_kernel(adev, adev->gart.table_size, PAGE_SIZE, | |
| 269 | − AMDGPU_GEM_DOMAIN_VRAM, &adev->gart.bo, | |
| 270 | − NULL, (void *)&adev->gart.ptr); | |
| 270 | + r = amdgpu_bo_create_kernel(adev, adev->gart.table_size, PAGE_SIZE, | |
| 271 | + AMDGPU_GEM_DOMAIN_VRAM, &adev->gart.bo, | |
| 272 | + NULL, (void *)&adev->gart.ptr); | |
| 273 | + if (r) | |
| 274 | + return r; | |
| 275 | + | |
| 276 | + memset_io(adev->gart.ptr, adev->gart.gart_pte_flags, adev->gart.table_size); | |
| 277 | + return 0; | |
| 271 | 278 | } |
| 272 | 279 | |
| 273 | 280 | /** |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <linux/pci.h> |
| 32 | 32 | #include <linux/dma-buf.h> |
| 33 | 33 | #include <linux/dma-fence-unwrap.h> |
| 34 | +#include <linux/uaccess.h> | |
| 34 | 35 | |
| 35 | 36 | #include <drm/amdgpu_drm.h> |
| 36 | 37 | #include <drm/drm_drv.h> |
| @@ -508,6 +509,9 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, · 471 unmodified lines | ||
| 508 | 509 | if (offset_in_page(args->addr | args->size)) |
| 509 | 510 | return -EINVAL; |
| 510 | 511 | |
| 512 | + if (!access_ok((void __user *)(uintptr_t)args->addr, args->size)) | |
| 513 | + return -EFAULT; | |
| 514 | + | |
| 511 | 515 | /* reject unknown flag values */ |
| 512 | 516 | if (args->flags & ~(AMDGPU_GEM_USERPTR_READONLY | |
| 513 | 517 | AMDGPU_GEM_USERPTR_ANONONLY | AMDGPU_GEM_USERPTR_VALIDATE | |
| @@ -821,7 +825,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 307 unmodified lines | ||
| 821 | 825 | struct drm_syncobj *timeline_syncobj = NULL; |
| 822 | 826 | struct dma_fence_chain *timeline_chain = NULL; |
| 823 | 827 | struct drm_exec exec; |
| 824 | − uint64_t vm_size; | |
| 828 | + uint64_t vm_size, tmp; | |
| 825 | 829 | int r = 0; |
| 826 | 830 | |
| 827 | 831 | /* Validate virtual address range against reserved regions. */ |
| @@ -845,7 +849,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 17 unmodified lines | ||
| 845 | 849 | |
| 846 | 850 | vm_size = adev->vm_manager.max_pfn * AMDGPU_GPU_PAGE_SIZE; |
| 847 | 851 | vm_size -= AMDGPU_VA_RESERVED_TOP; |
| 848 | − if (args->va_address + args->map_size > vm_size) { | |
| 852 | + if (check_add_overflow(args->va_address, args->map_size, &tmp) || tmp > vm_size) { | |
| 849 | 853 | dev_dbg(dev->dev, |
| 850 | 854 | "va_address 0x%llx is in top reserved area 0x%llx\n", |
| 851 | 855 | args->va_address + args->map_size, vm_size); |
| @@ -1089,9 +1093,16 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, · 237 unmodified lines | ||
| 1089 | 1093 | * If that number is larger than the size of the array, the ioctl must |
| 1090 | 1094 | * be retried. |
| 1091 | 1095 | */ |
| 1096 | + if (args->num_entries > INT_MAX / sizeof(*vm_entries)) { | |
| 1097 | + r = -EINVAL; | |
| 1098 | + goto out_exec; | |
| 1099 | + } | |
| 1100 | + | |
| 1092 | 1101 | vm_entries = kvcalloc(args->num_entries, sizeof(*vm_entries), GFP_KERNEL); |
| 1093 | − if (!vm_entries) | |
| 1094 | − return -ENOMEM; | |
| 1102 | + if (!vm_entries) { | |
| 1103 | + r = -ENOMEM; | |
| 1104 | + goto out_exec; | |
| 1105 | + } | |
| 1095 | 1106 | |
| 1096 | 1107 | amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) { |
| 1097 | 1108 | if (num_mappings < args->num_entries) { |
| @@ -314,7 +314,10 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc, · 313 unmodified lines | ||
| 314 | 314 | mc->gart_start = max_mc_address - mc->gart_size + 1; |
| 315 | 315 | break; |
| 316 | 316 | case AMDGPU_GART_PLACEMENT_LOW: |
| 317 | − mc->gart_start = 0; | |
| 317 | + if (size_bf >= mc->gart_size) | |
| 318 | + mc->gart_start = 0; | |
| 319 | + else | |
| 320 | + mc->gart_start = ALIGN(mc->fb_end, four_gb); | |
| 318 | 321 | break; |
| 319 | 322 | case AMDGPU_GART_PLACEMENT_BEST_FIT: |
| 320 | 323 | default: |
| @@ -199,11 +199,18 @@ int amdgpu_gtt_mgr_alloc_entries(struct amdgpu_gtt_mgr *mgr, · 198 unmodified lines | ||
| 199 | 199 | enum drm_mm_insert_mode mode) |
| 200 | 200 | { |
| 201 | 201 | struct amdgpu_device *adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); |
| 202 | + u32 alignment = 0; | |
| 202 | 203 | int r; |
| 203 | 204 | |
| 205 | + /* Align to TLB L2 cache entry size to work around "V bit HW bug" */ | |
| 206 | + if (adev->asic_type == CHIP_TAHITI) { | |
| 207 | + alignment = 32 * 1024 / AMDGPU_GPU_PAGE_SIZE; | |
| 208 | + num_pages = ALIGN(num_pages, alignment); | |
| 209 | + } | |
| 210 | + | |
| 204 | 211 | spin_lock(&mgr->lock); |
| 205 | 212 | r = drm_mm_insert_node_in_range(&mgr->mm, mm_node, num_pages, |
| 206 | − 0, GART_ENTRY_WITHOUT_BO_COLOR, 0, | |
| 213 | + alignment, GART_ENTRY_WITHOUT_BO_COLOR, 0, | |
| 207 | 214 | adev->gmc.gart_size >> PAGE_SHIFT, |
| 208 | 215 | mode); |
| 209 | 216 | spin_unlock(&mgr->lock); |
| @@ -51,8 +51,6 @@ · 50 unmodified lines | ||
| 51 | 51 | #include "amdgpu_amdkfd.h" |
| 52 | 52 | #include "amdgpu_hmm.h" |
| 53 | 53 | |
| 54 | −#define MAX_WALK_BYTE (2UL << 30) | |
| 55 | − | |
| 56 | 54 | /** |
| 57 | 55 | * amdgpu_hmm_invalidate_gfx - callback to notify about mm change |
| 58 | 56 | * |
| @@ -78,6 +76,7 @@ static bool amdgpu_hmm_invalidate_gfx(struct mmu_interval_notifier *mni, · 19 unmodified lines | ||
| 78 | 76 | |
| 79 | 77 | mmu_interval_set_seq(mni, cur_seq); |
| 80 | 78 | |
| 79 | + amdgpu_vm_bo_invalidate(bo, false); | |
| 81 | 80 | r = dma_resv_wait_timeout(bo->tbo.base.resv, DMA_RESV_USAGE_BOOKKEEP, |
| 82 | 81 | false, MAX_SCHEDULE_TIMEOUT); |
| 83 | 82 | mutex_unlock(&adev->notifier_lock); |
| @@ -170,11 +169,13 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 86 unmodified lines | ||
| 170 | 169 | void *owner, |
| 171 | 170 | struct amdgpu_hmm_range *range) |
| 172 | 171 | { |
| 173 | − unsigned long end; | |
| 172 | + const u64 max_bytes = SZ_2G; | |
| 173 | + | |
| 174 | + struct hmm_range *hmm_range = &range->hmm_range; | |
| 174 | 175 | unsigned long timeout; |
| 175 | 176 | unsigned long *pfns; |
| 176 | − int r = 0; | |
| 177 | − struct hmm_range *hmm_range = &range->hmm_range; | |
| 177 | + unsigned long end; | |
| 178 | + int r; | |
| 178 | 179 | |
| 179 | 180 | pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL); |
| 180 | 181 | if (unlikely(!pfns)) { |
| @@ -191,8 +192,9 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 10 unmodified lines | ||
| 191 | 192 | end = start + npages * PAGE_SIZE; |
| 192 | 193 | hmm_range->dev_private_owner = owner; |
| 193 | 194 | |
| 195 | + hmm_range->notifier_seq = mmu_interval_read_begin(notifier); | |
| 194 | 196 | do { |
| 195 | − hmm_range->end = min(hmm_range->start + MAX_WALK_BYTE, end); | |
| 197 | + hmm_range->end = min(hmm_range->start + max_bytes, end); | |
| 196 | 198 | |
| 197 | 199 | pr_debug("hmm range: start = 0x%lx, end = 0x%lx", |
| 198 | 200 | hmm_range->start, hmm_range->end); |
| @@ -200,7 +202,6 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier, · 1 unmodified line | ||
| 200 | 202 | timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); |
| 201 | 203 | |
| 202 | 204 | retry: |
| 203 | − hmm_range->notifier_seq = mmu_interval_read_begin(notifier); | |
| 204 | 205 | r = hmm_range_fault(hmm_range); |
| 205 | 206 | if (unlikely(r)) { |
| 206 | 207 | if (r == -EBUSY && !time_after(jiffies, timeout)) |
| @@ -210,7 +211,7 @@ retry: · 3 unmodified lines | ||
| 210 | 211 | |
| 211 | 212 | if (hmm_range->end == end) |
| 212 | 213 | break; |
| 213 | − hmm_range->hmm_pfns += MAX_WALK_BYTE >> PAGE_SHIFT; | |
| 214 | + hmm_range->hmm_pfns += max_bytes >> PAGE_SHIFT; | |
| 214 | 215 | hmm_range->start = hmm_range->end; |
| 215 | 216 | } while (hmm_range->end < end); |
| 216 | 217 | |
| @@ -873,68 +873,59 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 872 unmodified lines | ||
| 873 | 873 | ? -EFAULT : 0; |
| 874 | 874 | } |
| 875 | 875 | case AMDGPU_INFO_READ_MMR_REG: { |
| 876 | − int ret = 0; | |
| 877 | − unsigned int n, alloc_size; | |
| 878 | − uint32_t *regs; | |
| 879 | 876 | unsigned int se_num = (info->read_mmr_reg.instance >> |
| 880 | 877 | AMDGPU_INFO_MMR_SE_INDEX_SHIFT) & |
| 881 | 878 | AMDGPU_INFO_MMR_SE_INDEX_MASK; |
| 882 | 879 | unsigned int sh_num = (info->read_mmr_reg.instance >> |
| 883 | 880 | AMDGPU_INFO_MMR_SH_INDEX_SHIFT) & |
| 884 | 881 | AMDGPU_INFO_MMR_SH_INDEX_MASK; |
| 885 | − | |
| 886 | − if (!down_read_trylock(&adev->reset_domain->sem)) | |
| 887 | − return -ENOENT; | |
| 882 | + unsigned int alloc_size; | |
| 883 | + uint32_t *regs; | |
| 884 | + int ret; | |
| 888 | 885 | |
| 889 | 886 | /* set full masks if the userspace set all bits |
| 890 | 887 | * in the bitfields |
| 891 | 888 | */ |
| 892 | − if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK) { | |
| 889 | + if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK) | |
| 893 | 890 | se_num = 0xffffffff; |
| 894 | − } else if (se_num >= AMDGPU_GFX_MAX_SE) { | |
| 895 | − ret = -EINVAL; | |
| 896 | − goto out; | |
| 897 | − } | |
| 891 | + else if (se_num >= AMDGPU_GFX_MAX_SE) | |
| 892 | + return -EINVAL; | |
| 898 | 893 | |
| 899 | − if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK) { | |
| 894 | + if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK) | |
| 900 | 895 | sh_num = 0xffffffff; |
| 901 | − } else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE) { | |
| 902 | − ret = -EINVAL; | |
| 903 | − goto out; | |
| 904 | − } | |
| 896 | + else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE) | |
| 897 | + return -EINVAL; | |
| 905 | 898 | |
| 906 | − if (info->read_mmr_reg.count > 128) { | |
| 907 | − ret = -EINVAL; | |
| 908 | − goto out; | |
| 909 | − } | |
| 899 | + if (info->read_mmr_reg.count > 128) | |
| 900 | + return -EINVAL; | |
| 910 | 901 | |
| 911 | − regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs), GFP_KERNEL); | |
| 912 | − if (!regs) { | |
| 913 | − ret = -ENOMEM; | |
| 914 | − goto out; | |
| 915 | − } | |
| 902 | + regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs), | |
| 903 | + GFP_KERNEL); | |
| 904 | + if (!regs) | |
| 905 | + return -ENOMEM; | |
| 916 | 906 | |
| 907 | + down_read(&adev->reset_domain->sem); | |
| 917 | 908 | alloc_size = info->read_mmr_reg.count * sizeof(*regs); |
| 918 | − | |
| 919 | 909 | amdgpu_gfx_off_ctrl(adev, false); |
| 910 | + ret = 0; | |
| 920 | 911 | for (i = 0; i < info->read_mmr_reg.count; i++) { |
| 921 | 912 | if (amdgpu_asic_read_register(adev, se_num, sh_num, |
| 922 | 913 | info->read_mmr_reg.dword_offset + i, |
| 923 | 914 | ®s[i])) { |
| 924 | 915 | DRM_DEBUG_KMS("unallowed offset %#x\n", |
| 925 | 916 | info->read_mmr_reg.dword_offset + i); |
| 926 | − kfree(regs); | |
| 927 | − amdgpu_gfx_off_ctrl(adev, true); | |
| 928 | 917 | ret = -EFAULT; |
| 929 | − goto out; | |
| 918 | + break; | |
| 930 | 919 | } |
| 931 | 920 | } |
| 932 | 921 | amdgpu_gfx_off_ctrl(adev, true); |
| 933 | − n = copy_to_user(out, regs, min(size, alloc_size)); | |
| 934 | − kfree(regs); | |
| 935 | − ret = (n ? -EFAULT : 0); | |
| 936 | −out: | |
| 937 | 922 | up_read(&adev->reset_domain->sem); |
| 923 | + | |
| 924 | + if (!ret) { | |
| 925 | + ret = copy_to_user(out, regs, min(size, alloc_size)) | |
| 926 | + ? -EFAULT : 0; | |
| 927 | + } | |
| 928 | + kfree(regs); | |
| 938 | 929 | return ret; |
| 939 | 930 | } |
| 940 | 931 | case AMDGPU_INFO_DEV_INFO: { |
| @@ -96,7 +96,8 @@ struct amdgpu_bo_va { · 95 unmodified lines | ||
| 96 | 96 | * if non-zero, cannot unmap from GPU because user queues may still access it |
| 97 | 97 | */ |
| 98 | 98 | unsigned int queue_refcount; |
| 99 | − atomic_t userq_va_mapped; | |
| 99 | + /* Indicates if this buffer is mapped for any user queue. Once set, never reset. */ | |
| 100 | + bool userq_va_mapped; | |
| 100 | 101 | }; |
| 101 | 102 | |
| 102 | 103 | struct amdgpu_bo { |
| @@ -2280,7 +2280,8 @@ void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev) · 2279 unmodified lines | ||
| 2280 | 2280 | list_for_each_entry(obj, &con->head, node) { |
| 2281 | 2281 | if (amdgpu_ras_is_supported(adev, obj->head.block) && |
| 2282 | 2282 | (obj->attr_inuse == 1)) { |
| 2283 | − sprintf(fs_info.debugfs_name, "%s_err_inject", | |
| 2283 | + snprintf(fs_info.debugfs_name, sizeof(fs_info.debugfs_name), | |
| 2284 | + "%s_err_inject", | |
| 2284 | 2285 | get_ras_block_str(&obj->head)); |
| 2285 | 2286 | fs_info.head = obj->head; |
| 2286 | 2287 | amdgpu_ras_debugfs_create(adev, &fs_info, dir); |
| @@ -1950,7 +1950,7 @@ void amdgpu_ras_check_bad_page_status(struct amdgpu_device *adev) · 1949 unmodified lines | ||
| 1950 | 1950 | if (!control || amdgpu_bad_page_threshold == 0) |
| 1951 | 1951 | return; |
| 1952 | 1952 | |
| 1953 | − if (control->ras_num_bad_pages >= ras->bad_page_cnt_threshold) { | |
| 1953 | + if (control->ras_num_bad_pages > ras->bad_page_cnt_threshold) { | |
| 1954 | 1954 | if (amdgpu_dpm_send_rma_reason(adev)) |
| 1955 | 1955 | dev_warn(adev->dev, "Unable to send out-of-band RMA CPER"); |
| 1956 | 1956 | else |
| @@ -552,8 +552,9 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 551 unmodified lines | ||
| 552 | 552 | size_t size, loff_t *pos) |
| 553 | 553 | { |
| 554 | 554 | struct amdgpu_ring *ring = file_inode(f)->i_private; |
| 555 | − uint32_t value, result, early[3]; | |
| 555 | + u32 value, result, early[3] = { 0 }; | |
| 556 | 556 | uint64_t p; |
| 557 | + u32 avail_dw, start_dw, read_dw; | |
| 557 | 558 | loff_t i; |
| 558 | 559 | int r; |
| 559 | 560 | |
| @@ -565,10 +566,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 5 unmodified lines | ||
| 565 | 566 | |
| 566 | 567 | result = 0; |
| 567 | 568 | |
| 568 | − if (*pos < 12) { | |
| 569 | − if (ring->funcs->type == AMDGPU_RING_TYPE_CPER) | |
| 570 | − mutex_lock(&ring->adev->cper.ring_lock); | |
| 569 | + if (ring->funcs->type == AMDGPU_RING_TYPE_CPER) | |
| 570 | + mutex_lock(&ring->adev->cper.ring_lock); | |
| 571 | 571 | |
| 572 | + if (*pos < 12) { | |
| 572 | 573 | early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask; |
| 573 | 574 | early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask; |
| 574 | 575 | early[2] = ring->wptr & ring->buf_mask; |
| @@ -600,13 +601,24 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 25 unmodified lines | ||
| 600 | 601 | *pos += 4; |
| 601 | 602 | } |
| 602 | 603 | } else { |
| 604 | + early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask; | |
| 605 | + early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask; | |
| 606 | + | |
| 603 | 607 | p = early[0]; |
| 604 | 608 | if (early[0] <= early[1]) |
| 605 | − size = (early[1] - early[0]); | |
| 609 | + avail_dw = early[1] - early[0]; | |
| 606 | 610 | else |
| 607 | − size = ring->ring_size - (early[0] - early[1]); | |
| 611 | + avail_dw = ring->buf_mask + 1 - (early[0] - early[1]); | |
| 608 | 612 | |
| 609 | − while (size) { | |
| 613 | + start_dw = (*pos > 12) ? ((*pos - 12) >> 2) : 0; | |
| 614 | + if (start_dw >= avail_dw) | |
| 615 | + goto out; | |
| 616 | + | |
| 617 | + p = (p + start_dw) & ring->ptr_mask; | |
| 618 | + avail_dw -= start_dw; | |
| 619 | + read_dw = min_t(u32, avail_dw, size >> 2); | |
| 620 | + | |
| 621 | + while (read_dw) { | |
| 610 | 622 | if (p == early[1]) |
| 611 | 623 | goto out; |
| 612 | 624 | |
| @@ -619,9 +631,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 6 unmodified lines | ||
| 619 | 631 | |
| 620 | 632 | buf += 4; |
| 621 | 633 | result += 4; |
| 622 | − size--; | |
| 634 | + read_dw--; | |
| 623 | 635 | p++; |
| 624 | 636 | p &= ring->ptr_mask; |
| 637 | + *pos += 4; | |
| 625 | 638 | } |
| 626 | 639 | } |
| 627 | 640 | |
| @@ -175,11 +175,14 @@ int amdgpu_seq64_alloc(struct amdgpu_device *adev, u64 *va, · 174 unmodified lines | ||
| 175 | 175 | { |
| 176 | 176 | unsigned long bit_pos; |
| 177 | 177 | |
| 178 | − bit_pos = find_first_zero_bit(adev->seq64.used, adev->seq64.num_sem); | |
| 179 | − if (bit_pos >= adev->seq64.num_sem) | |
| 180 | − return -ENOSPC; | |
| 178 | + for (;;) { | |
| 179 | + bit_pos = find_first_zero_bit(adev->seq64.used, adev->seq64.num_sem); | |
| 180 | + if (bit_pos >= adev->seq64.num_sem) | |
| 181 | + return -ENOSPC; | |
| 181 | 182 | |
| 182 | − __set_bit(bit_pos, adev->seq64.used); | |
| 183 | + if (!test_and_set_bit(bit_pos, adev->seq64.used)) | |
| 184 | + break; | |
| 185 | + } | |
| 183 | 186 | |
| 184 | 187 | *va = bit_pos * sizeof(u64) + amdgpu_seq64_get_va_base(adev); |
| 185 | 188 | |
| @@ -205,7 +208,7 @@ void amdgpu_seq64_free(struct amdgpu_device *adev, u64 va) · 19 unmodified lines | ||
| 205 | 208 | |
| 206 | 209 | bit_pos = (va - amdgpu_seq64_get_va_base(adev)) / sizeof(u64); |
| 207 | 210 | if (bit_pos < adev->seq64.num_sem) |
| 208 | − __clear_bit(bit_pos, adev->seq64.used); | |
| 211 | + clear_bit(bit_pos, adev->seq64.used); | |
| 209 | 212 | } |
| 210 | 213 | |
| 211 | 214 | /** |
| @@ -75,6 +75,9 @@ static int amdgpu_ttm_init_on_chip(struct amdgpu_device *adev, · 74 unmodified lines | ||
| 75 | 75 | unsigned int type, |
| 76 | 76 | uint64_t size_in_page) |
| 77 | 77 | { |
| 78 | + if (!size_in_page) | |
| 79 | + return 0; | |
| 80 | + | |
| 78 | 81 | return ttm_range_man_init(&adev->mman.bdev, type, |
| 79 | 82 | false, size_in_page); |
| 80 | 83 | } |
| @@ -130,6 +130,7 @@ void amdgpu_umc_handle_bad_pages(struct amdgpu_device *adev, · 129 unmodified lines | ||
| 130 | 130 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 131 | 131 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address && |
| 132 | 132 | adev->umc.max_ras_err_cnt_per_query) { |
| 133 | + kfree(err_data->err_addr); | |
| 133 | 134 | err_data->err_addr = |
| 134 | 135 | kzalloc_objs(struct eeprom_table_record, |
| 135 | 136 | adev->umc.max_ras_err_cnt_per_query); |
| @@ -160,6 +161,7 @@ void amdgpu_umc_handle_bad_pages(struct amdgpu_device *adev, · 24 unmodified lines | ||
| 160 | 161 | if (adev->umc.ras && |
| 161 | 162 | adev->umc.ras->ecc_info_query_ras_error_address && |
| 162 | 163 | adev->umc.max_ras_err_cnt_per_query) { |
| 164 | + kfree(err_data->err_addr); | |
| 163 | 165 | err_data->err_addr = |
| 164 | 166 | kzalloc_objs(struct eeprom_table_record, |
| 165 | 167 | adev->umc.max_ras_err_cnt_per_query); |
| @@ -82,19 +82,11 @@ static bool amdgpu_userq_is_reset_type_supported(struct amdgpu_device *adev, · 81 unmodified lines | ||
| 82 | 82 | return false; |
| 83 | 83 | } |
| 84 | 84 | |
| 85 | −static void amdgpu_userq_gpu_reset(struct amdgpu_device *adev) | |
| 86 | −{ | |
| 87 | − if (amdgpu_device_should_recover_gpu(adev)) { | |
| 88 | − amdgpu_reset_domain_schedule(adev->reset_domain, | |
| 89 | − &adev->userq_reset_work); | |
| 90 | − /* Wait for the reset job to complete */ | |
| 91 | − flush_work(&adev->userq_reset_work); | |
| 92 | − } | |
| 93 | −} | |
| 94 | − | |
| 95 | −static int | |
| 96 | −amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) | |
| 85 | +static void amdgpu_userq_mgr_reset_work(struct work_struct *work) | |
| 97 | 86 | { |
| 87 | + struct amdgpu_userq_mgr *uq_mgr = | |
| 88 | + container_of(work, struct amdgpu_userq_mgr, | |
| 89 | + reset_work); | |
| 98 | 90 | struct amdgpu_device *adev = uq_mgr->adev; |
| 99 | 91 | const int queue_types[] = { |
| 100 | 92 | AMDGPU_RING_TYPE_COMPUTE, |
| @@ -103,15 +95,11 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 2 unmodified lines | ||
| 103 | 95 | }; |
| 104 | 96 | const int num_queue_types = ARRAY_SIZE(queue_types); |
| 105 | 97 | bool gpu_reset = false; |
| 106 | − int r = 0; | |
| 107 | − int i; | |
| 108 | − | |
| 109 | − /* Warning if current process mutex is not held */ | |
| 110 | − WARN_ON(!mutex_is_locked(&uq_mgr->userq_mutex)); | |
| 98 | + int i, r; | |
| 111 | 99 | |
| 112 | 100 | if (unlikely(adev->debug_disable_gpu_ring_reset)) { |
| 113 | 101 | dev_err(adev->dev, "userq reset disabled by debug mask\n"); |
| 114 | − return 0; | |
| 102 | + return; | |
| 115 | 103 | } |
| 116 | 104 | |
| 117 | 105 | /* |
| @@ -119,7 +107,7 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | ||
| 119 | 107 | * skip all reset detection logic |
| 120 | 108 | */ |
| 121 | 109 | if (!amdgpu_gpu_recovery) |
| 122 | − return 0; | |
| 110 | + return; | |
| 123 | 111 | |
| 124 | 112 | /* |
| 125 | 113 | * Iterate through all queue types to detect and reset problematic queues |
| @@ -127,9 +115,11 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | ||
| 127 | 115 | */ |
| 128 | 116 | for (i = 0; i < num_queue_types; i++) { |
| 129 | 117 | int ring_type = queue_types[i]; |
| 130 | − const struct amdgpu_userq_funcs *funcs = adev->userq_funcs[ring_type]; | |
| 118 | + const struct amdgpu_userq_funcs *funcs = | |
| 119 | + adev->userq_funcs[ring_type]; | |
| 131 | 120 | |
| 132 | − if (!amdgpu_userq_is_reset_type_supported(adev, ring_type, AMDGPU_RESET_TYPE_PER_QUEUE)) | |
| 121 | + if (!amdgpu_userq_is_reset_type_supported(adev, ring_type, | |
| 122 | + AMDGPU_RESET_TYPE_PER_QUEUE)) | |
| 133 | 123 | continue; |
| 134 | 124 | |
| 135 | 125 | if (atomic_read(&uq_mgr->userq_count[ring_type]) > 0 && |
| @@ -142,46 +132,43 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr) · 6 unmodified lines | ||
| 142 | 132 | } |
| 143 | 133 | } |
| 144 | 134 | |
| 145 | − if (gpu_reset) | |
| 146 | − amdgpu_userq_gpu_reset(adev); | |
| 135 | + if (gpu_reset) { | |
| 136 | + struct amdgpu_reset_context reset_context; | |
| 147 | 137 | |
| 148 | − return r; | |
| 138 | + memset(&reset_context, 0, sizeof(reset_context)); | |
| 139 | + | |
| 140 | + reset_context.method = AMD_RESET_METHOD_NONE; | |
| 141 | + reset_context.reset_req_dev = adev; | |
| 142 | + reset_context.src = AMDGPU_RESET_SRC_USERQ; | |
| 143 | + set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | |
| 144 | + /*set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags);*/ | |
| 145 | + | |
| 146 | + amdgpu_device_gpu_recover(adev, NULL, &reset_context); | |
| 147 | + } | |
| 149 | 148 | } |
| 150 | 149 | |
| 151 | 150 | static void amdgpu_userq_hang_detect_work(struct work_struct *work) |
| 152 | 151 | { |
| 153 | − struct amdgpu_usermode_queue *queue = container_of(work, | |
| 154 | − struct amdgpu_usermode_queue, | |
| 155 | − hang_detect_work.work); | |
| 156 | − struct dma_fence *fence; | |
| 157 | − struct amdgpu_userq_mgr *uq_mgr; | |
| 158 | − | |
| 159 | − if (!queue->userq_mgr) | |
| 160 | − return; | |
| 161 | − | |
| 162 | − uq_mgr = queue->userq_mgr; | |
| 163 | − fence = READ_ONCE(queue->hang_detect_fence); | |
| 164 | − /* Fence already signaled – no action needed */ | |
| 165 | − if (!fence || dma_fence_is_signaled(fence)) | |
| 166 | − return; | |
| 152 | + struct amdgpu_usermode_queue *queue = | |
| 153 | + container_of(work, struct amdgpu_usermode_queue, | |
| 154 | + hang_detect_work.work); | |
| 167 | 155 | |
| 168 | − mutex_lock(&uq_mgr->userq_mutex); | |
| 169 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 170 | − mutex_unlock(&uq_mgr->userq_mutex); | |
| 156 | + /* | |
| 157 | + * Don't schedule the work here! Scheduling or queue work from one reset | |
| 158 | + * handler to another is illegal if you don't take extra precautions! | |
| 159 | + */ | |
| 160 | + amdgpu_userq_mgr_reset_work(&queue->userq_mgr->reset_work); | |
| 171 | 161 | } |
| 172 | 162 | |
| 173 | 163 | /* |
| 174 | 164 | * Start hang detection for a user queue fence. A delayed work will be scheduled |
| 175 | − * to check if the fence is still pending after the timeout period. | |
| 176 | −*/ | |
| 165 | + * to reset the queues when the fence doesn't signal in time. | |
| 166 | + */ | |
| 177 | 167 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue) |
| 178 | 168 | { |
| 179 | 169 | struct amdgpu_device *adev; |
| 180 | 170 | unsigned long timeout_ms; |
| 181 | 171 | |
| 182 | − if (!queue || !queue->userq_mgr || !queue->userq_mgr->adev) | |
| 183 | − return; | |
| 184 | − | |
| 185 | 172 | adev = queue->userq_mgr->adev; |
| 186 | 173 | /* Determine timeout based on queue type */ |
| 187 | 174 | switch (queue->queue_type) { |
| @@ -199,45 +186,44 @@ void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue) · 11 unmodified lines | ||
| 199 | 186 | break; |
| 200 | 187 | } |
| 201 | 188 | |
| 202 | − /* Store the fence to monitor and schedule hang detection */ | |
| 203 | − WRITE_ONCE(queue->hang_detect_fence, queue->last_fence); | |
| 204 | − schedule_delayed_work(&queue->hang_detect_work, | |
| 205 | − msecs_to_jiffies(timeout_ms)); | |
| 206 | −} | |
| 207 | − | |
| 208 | −static void amdgpu_userq_init_hang_detect_work(struct amdgpu_usermode_queue *queue) | |
| 209 | −{ | |
| 210 | − INIT_DELAYED_WORK(&queue->hang_detect_work, amdgpu_userq_hang_detect_work); | |
| 211 | − queue->hang_detect_fence = NULL; | |
| 189 | + queue_delayed_work(adev->reset_domain->wq, &queue->hang_detect_work, | |
| 190 | + msecs_to_jiffies(timeout_ms)); | |
| 212 | 191 | } |
| 213 | 192 | |
| 214 | −static int amdgpu_userq_buffer_va_list_add(struct amdgpu_usermode_queue *queue, | |
| 215 | − struct amdgpu_bo_va_mapping *va_map, u64 addr) | |
| 193 | +void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell) | |
| 216 | 194 | { |
| 217 | − struct amdgpu_userq_va_cursor *va_cursor; | |
| 218 | − struct userq_va_list; | |
| 219 | − | |
| 220 | − va_cursor = kzalloc_obj(*va_cursor); | |
| 221 | − if (!va_cursor) | |
| 222 | − return -ENOMEM; | |
| 195 | + struct xarray *xa = &adev->userq_doorbell_xa; | |
| 196 | + struct amdgpu_usermode_queue *queue; | |
| 197 | + unsigned long flags; | |
| 198 | + int r; | |
| 223 | 199 | |
| 224 | − INIT_LIST_HEAD(&va_cursor->list); | |
| 225 | − va_cursor->gpu_addr = addr; | |
| 226 | − atomic_set(&va_map->bo_va->userq_va_mapped, 1); | |
| 227 | − list_add(&va_cursor->list, &queue->userq_va_list); | |
| 200 | + xa_lock_irqsave(xa, flags); | |
| 201 | + queue = xa_load(xa, doorbell); | |
| 202 | + if (queue) { | |
| 203 | + r = amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 204 | + /* | |
| 205 | + * We are in interrupt context here, this *can't* wait for | |
| 206 | + * reset work to finish. | |
| 207 | + */ | |
| 208 | + if (r >= 0) | |
| 209 | + cancel_delayed_work(&queue->hang_detect_work); | |
| 228 | 210 | |
| 229 | − return 0; | |
| 211 | + /* Restart the timer when there are still fences pending */ | |
| 212 | + if (r == 1) | |
| 213 | + amdgpu_userq_start_hang_detect_work(queue); | |
| 214 | + } | |
| 215 | + xa_unlock_irqrestore(xa, flags); | |
| 230 | 216 | } |
| 231 | 217 | |
| 232 | 218 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, |
| 233 | 219 | struct amdgpu_usermode_queue *queue, |
| 234 | − u64 addr, u64 expected_size) | |
| 220 | + u64 addr, u64 expected_size, | |
| 221 | + u64 *va_out) | |
| 235 | 222 | { |
| 236 | 223 | struct amdgpu_bo_va_mapping *va_map; |
| 237 | 224 | struct amdgpu_vm *vm = queue->vm; |
| 238 | 225 | u64 user_addr; |
| 239 | 226 | u64 size; |
| 240 | − int r = 0; | |
| 241 | 227 | |
| 242 | 228 | /* Caller must hold vm->root.bo reservation */ |
| 243 | 229 | dma_resv_assert_held(queue->vm->root.bo->tbo.base.resv); |
| @@ -246,20 +232,18 @@ int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 246 | 232 | size = expected_size >> AMDGPU_GPU_PAGE_SHIFT; |
| 247 | 233 | |
| 248 | 234 | va_map = amdgpu_vm_bo_lookup_mapping(vm, user_addr); |
| 249 | − if (!va_map) { | |
| 250 | − r = -EINVAL; | |
| 251 | − goto out_err; | |
| 252 | − } | |
| 235 | + if (!va_map) | |
| 236 | + return -EINVAL; | |
| 237 | + | |
| 253 | 238 | /* Only validate the userq whether resident in the VM mapping range */ |
| 254 | 239 | if (user_addr >= va_map->start && |
| 255 | 240 | va_map->last - user_addr + 1 >= size) { |
| 256 | − amdgpu_userq_buffer_va_list_add(queue, va_map, user_addr); | |
| 241 | + va_map->bo_va->userq_va_mapped = true; | |
| 242 | + *va_out = user_addr; | |
| 257 | 243 | return 0; |
| 258 | 244 | } |
| 259 | 245 | |
| 260 | − r = -EINVAL; | |
| 261 | −out_err: | |
| 262 | − return r; | |
| 246 | + return -EINVAL; | |
| 263 | 247 | } |
| 264 | 248 | |
| 265 | 249 | static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) |
| @@ -270,7 +254,7 @@ static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) · 4 unmodified lines | ||
| 270 | 254 | dma_resv_assert_held(vm->root.bo->tbo.base.resv); |
| 271 | 255 | |
| 272 | 256 | mapping = amdgpu_vm_bo_lookup_mapping(vm, addr); |
| 273 | − if (!IS_ERR_OR_NULL(mapping) && atomic_read(&mapping->bo_va->userq_va_mapped)) | |
| 257 | + if (!IS_ERR_OR_NULL(mapping) && mapping->bo_va->userq_va_mapped) | |
| 274 | 258 | r = true; |
| 275 | 259 | else |
| 276 | 260 | r = false; |
| @@ -280,14 +264,16 @@ static bool amdgpu_userq_buffer_va_mapped(struct amdgpu_vm *vm, u64 addr) · 3 unmodified lines | ||
| 280 | 264 | |
| 281 | 265 | static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue) |
| 282 | 266 | { |
| 283 | − struct amdgpu_userq_va_cursor *va_cursor, *tmp; | |
| 284 | − int r = 0; | |
| 267 | + int i, r = 0; | |
| 285 | 268 | |
| 286 | − list_for_each_entry_safe(va_cursor, tmp, &queue->userq_va_list, list) { | |
| 287 | − r += amdgpu_userq_buffer_va_mapped(queue->vm, va_cursor->gpu_addr); | |
| 269 | + for (i = 0; i < ARRAY_SIZE(queue->userq_vas.va_array); i++) { | |
| 270 | + if (!queue->userq_vas.va_array[i]) | |
| 271 | + continue; | |
| 272 | + r += amdgpu_userq_buffer_va_mapped(queue->vm, | |
| 273 | + queue->userq_vas.va_array[i]); | |
| 288 | 274 | dev_dbg(queue->userq_mgr->adev->dev, |
| 289 | 275 | "validate the userq mapping:%p va:%llx r:%d\n", |
| 290 | − queue, va_cursor->gpu_addr, r); | |
| 276 | + queue, queue->userq_vas.va_array[i], r); | |
| 291 | 277 | } |
| 292 | 278 | |
| 293 | 279 | if (r != 0) |
| @@ -296,35 +282,7 @@ static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue) · 2 unmodified lines | ||
| 296 | 282 | return false; |
| 297 | 283 | } |
| 298 | 284 | |
| 299 | −static void amdgpu_userq_buffer_va_list_del(struct amdgpu_bo_va_mapping *mapping, | |
| 300 | − struct amdgpu_userq_va_cursor *va_cursor) | |
| 301 | −{ | |
| 302 | − atomic_set(&mapping->bo_va->userq_va_mapped, 0); | |
| 303 | − list_del(&va_cursor->list); | |
| 304 | − kfree(va_cursor); | |
| 305 | −} | |
| 306 | − | |
| 307 | −static int amdgpu_userq_buffer_vas_list_cleanup(struct amdgpu_device *adev, | |
| 308 | − struct amdgpu_usermode_queue *queue) | |
| 309 | −{ | |
| 310 | − struct amdgpu_userq_va_cursor *va_cursor, *tmp; | |
| 311 | − struct amdgpu_bo_va_mapping *mapping; | |
| 312 | − | |
| 313 | − /* Caller must hold vm->root.bo reservation */ | |
| 314 | − dma_resv_assert_held(queue->vm->root.bo->tbo.base.resv); | |
| 315 | − | |
| 316 | − list_for_each_entry_safe(va_cursor, tmp, &queue->userq_va_list, list) { | |
| 317 | − mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, va_cursor->gpu_addr); | |
| 318 | − if (!mapping) { | |
| 319 | − return -EINVAL; | |
| 320 | − } | |
| 321 | − dev_dbg(adev->dev, "delete the userq:%p va:%llx\n", | |
| 322 | − queue, va_cursor->gpu_addr); | |
| 323 | − amdgpu_userq_buffer_va_list_del(mapping, va_cursor); | |
| 324 | − } | |
| 325 | 285 | |
| 326 | − return 0; | |
| 327 | −} | |
| 328 | 286 | |
| 329 | 287 | static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue) |
| 330 | 288 | { |
| @@ -332,23 +290,18 @@ static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue) · 1 unmodified line | ||
| 332 | 290 | struct amdgpu_device *adev = uq_mgr->adev; |
| 333 | 291 | const struct amdgpu_userq_funcs *userq_funcs = |
| 334 | 292 | adev->userq_funcs[queue->queue_type]; |
| 335 | − bool found_hung_queue = false; | |
| 336 | − int r = 0; | |
| 293 | + int r; | |
| 337 | 294 | |
| 338 | 295 | if (queue->state == AMDGPU_USERQ_STATE_MAPPED) { |
| 339 | 296 | r = userq_funcs->preempt(queue); |
| 340 | 297 | if (r) { |
| 341 | 298 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 342 | − found_hung_queue = true; | |
| 299 | + return r; | |
| 343 | 300 | } else { |
| 344 | 301 | queue->state = AMDGPU_USERQ_STATE_PREEMPTED; |
| 345 | 302 | } |
| 346 | 303 | } |
| 347 | − | |
| 348 | − if (found_hung_queue) | |
| 349 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 350 | − | |
| 351 | − return r; | |
| 304 | + return 0; | |
| 352 | 305 | } |
| 353 | 306 | |
| 354 | 307 | static int amdgpu_userq_restore_helper(struct amdgpu_usermode_queue *queue) |
| @@ -377,24 +330,21 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue) · 22 unmodified lines | ||
| 377 | 330 | struct amdgpu_device *adev = uq_mgr->adev; |
| 378 | 331 | const struct amdgpu_userq_funcs *userq_funcs = |
| 379 | 332 | adev->userq_funcs[queue->queue_type]; |
| 380 | − bool found_hung_queue = false; | |
| 381 | − int r = 0; | |
| 333 | + int r; | |
| 382 | 334 | |
| 383 | 335 | if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) || |
| 384 | − (queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) { | |
| 336 | + (queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) { | |
| 337 | + | |
| 385 | 338 | r = userq_funcs->unmap(queue); |
| 386 | 339 | if (r) { |
| 387 | 340 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 388 | − found_hung_queue = true; | |
| 341 | + return r; | |
| 389 | 342 | } else { |
| 390 | 343 | queue->state = AMDGPU_USERQ_STATE_UNMAPPED; |
| 391 | 344 | } |
| 392 | 345 | } |
| 393 | 346 | |
| 394 | − if (found_hung_queue) | |
| 395 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 396 | − | |
| 397 | − return r; | |
| 347 | + return 0; | |
| 398 | 348 | } |
| 399 | 349 | |
| 400 | 350 | static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) |
| @@ -403,19 +353,19 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) · 2 unmodified lines | ||
| 403 | 353 | struct amdgpu_device *adev = uq_mgr->adev; |
| 404 | 354 | const struct amdgpu_userq_funcs *userq_funcs = |
| 405 | 355 | adev->userq_funcs[queue->queue_type]; |
| 406 | − int r = 0; | |
| 356 | + int r; | |
| 407 | 357 | |
| 408 | 358 | if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) { |
| 409 | 359 | r = userq_funcs->map(queue); |
| 410 | 360 | if (r) { |
| 411 | 361 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 412 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 362 | + return r; | |
| 413 | 363 | } else { |
| 414 | 364 | queue->state = AMDGPU_USERQ_STATE_MAPPED; |
| 415 | 365 | } |
| 416 | 366 | } |
| 417 | 367 | |
| 418 | − return r; | |
| 368 | + return 0; | |
| 419 | 369 | } |
| 420 | 370 | |
| 421 | 371 | static void amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue) |
| @@ -432,18 +382,14 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue) · 10 unmodified lines | ||
| 432 | 382 | { |
| 433 | 383 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; |
| 434 | 384 | struct amdgpu_device *adev = uq_mgr->adev; |
| 435 | − const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type]; | |
| 436 | 385 | |
| 437 | 386 | /* Wait for mode-1 reset to complete */ |
| 438 | 387 | down_read(&adev->reset_domain->sem); |
| 439 | 388 | |
| 440 | − uq_funcs->mqd_destroy(queue); | |
| 441 | 389 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ |
| 442 | 390 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); |
| 443 | 391 | amdgpu_userq_fence_driver_free(queue); |
| 444 | 392 | queue->fence_drv = NULL; |
| 445 | − queue->userq_mgr = NULL; | |
| 446 | − list_del(&queue->userq_va_list); | |
| 447 | 393 | |
| 448 | 394 | up_read(&adev->reset_domain->sem); |
| 449 | 395 | } |
| @@ -482,74 +428,15 @@ retry: · 32 unmodified lines | ||
| 482 | 428 | dma_fence_put(ev_fence); |
| 483 | 429 | } |
| 484 | 430 | |
| 485 | −int amdgpu_userq_create_object(struct amdgpu_userq_mgr *uq_mgr, | |
| 486 | − struct amdgpu_userq_obj *userq_obj, | |
| 487 | − int size) | |
| 488 | −{ | |
| 489 | − struct amdgpu_device *adev = uq_mgr->adev; | |
| 490 | − struct amdgpu_bo_param bp; | |
| 491 | − int r; | |
| 492 | − | |
| 493 | − memset(&bp, 0, sizeof(bp)); | |
| 494 | − bp.byte_align = PAGE_SIZE; | |
| 495 | − bp.domain = AMDGPU_GEM_DOMAIN_GTT; | |
| 496 | − bp.flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS | | |
| 497 | − AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; | |
| 498 | − bp.type = ttm_bo_type_kernel; | |
| 499 | − bp.size = size; | |
| 500 | − bp.resv = NULL; | |
| 501 | − bp.bo_ptr_size = sizeof(struct amdgpu_bo); | |
| 502 | − | |
| 503 | − r = amdgpu_bo_create(adev, &bp, &userq_obj->obj); | |
| 504 | − if (r) { | |
| 505 | − drm_file_err(uq_mgr->file, "Failed to allocate BO for userqueue (%d)", r); | |
| 506 | − return r; | |
| 507 | − } | |
| 508 | − | |
| 509 | − r = amdgpu_bo_reserve(userq_obj->obj, true); | |
| 510 | − if (r) { | |
| 511 | − drm_file_err(uq_mgr->file, "Failed to reserve BO to map (%d)", r); | |
| 512 | − goto free_obj; | |
| 513 | − } | |
| 514 | − | |
| 515 | − r = amdgpu_ttm_alloc_gart(&(userq_obj->obj)->tbo); | |
| 516 | − if (r) { | |
| 517 | − drm_file_err(uq_mgr->file, "Failed to alloc GART for userqueue object (%d)", r); | |
| 518 | − goto unresv; | |
| 519 | − } | |
| 520 | − | |
| 521 | − r = amdgpu_bo_kmap(userq_obj->obj, &userq_obj->cpu_ptr); | |
| 522 | − if (r) { | |
| 523 | − drm_file_err(uq_mgr->file, "Failed to map BO for userqueue (%d)", r); | |
| 524 | − goto unresv; | |
| 525 | − } | |
| 526 | − | |
| 527 | − userq_obj->gpu_addr = amdgpu_bo_gpu_offset(userq_obj->obj); | |
| 528 | − amdgpu_bo_unreserve(userq_obj->obj); | |
| 529 | − memset(userq_obj->cpu_ptr, 0, size); | |
| 530 | − return 0; | |
| 531 | 431 | |
| 532 | −unresv: | |
| 533 | − amdgpu_bo_unreserve(userq_obj->obj); | |
| 534 | − | |
| 535 | −free_obj: | |
| 536 | − amdgpu_bo_unref(&userq_obj->obj); | |
| 537 | − return r; | |
| 538 | −} | |
| 539 | − | |
| 540 | −void amdgpu_userq_destroy_object(struct amdgpu_userq_mgr *uq_mgr, | |
| 541 | − struct amdgpu_userq_obj *userq_obj) | |
| 542 | −{ | |
| 543 | − amdgpu_bo_kunmap(userq_obj->obj); | |
| 544 | − amdgpu_bo_unref(&userq_obj->obj); | |
| 545 | −} | |
| 546 | 432 | |
| 547 | −uint64_t | |
| 433 | +static int | |
| 548 | 434 | amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, |
| 549 | 435 | struct amdgpu_db_info *db_info, |
| 550 | − struct drm_file *filp) | |
| 436 | + struct drm_file *filp, | |
| 437 | + u64 *index) | |
| 551 | 438 | { |
| 552 | − uint64_t index; | |
| 439 | + u64 doorbell_index; | |
| 553 | 440 | struct drm_gem_object *gobj; |
| 554 | 441 | struct amdgpu_userq_obj *db_obj = db_info->db_obj; |
| 555 | 442 | int r, db_size; |
| @@ -596,12 +483,13 @@ amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, · 40 unmodified lines | ||
| 596 | 483 | goto unpin_bo; |
| 597 | 484 | } |
| 598 | 485 | |
| 599 | − index = amdgpu_doorbell_index_on_bar(uq_mgr->adev, db_obj->obj, | |
| 600 | − db_info->doorbell_offset, db_size); | |
| 486 | + doorbell_index = amdgpu_doorbell_index_on_bar(uq_mgr->adev, db_obj->obj, | |
| 487 | + db_info->doorbell_offset, db_size); | |
| 601 | 488 | drm_dbg_driver(adev_to_drm(uq_mgr->adev), |
| 602 | − "[Usermode queues] doorbell index=%lld\n", index); | |
| 489 | + "[Usermode queues] doorbell index=%lld\n", doorbell_index); | |
| 603 | 490 | amdgpu_bo_unreserve(db_obj->obj); |
| 604 | − return index; | |
| 491 | + *index = doorbell_index; | |
| 492 | + return 0; | |
| 605 | 493 | |
| 606 | 494 | unpin_bo: |
| 607 | 495 | amdgpu_bo_unpin(db_obj->obj); |
| @@ -616,9 +504,7 @@ static int · 8 unmodified lines | ||
| 616 | 504 | amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_queue *queue) |
| 617 | 505 | { |
| 618 | 506 | struct amdgpu_device *adev = uq_mgr->adev; |
| 619 | − struct amdgpu_fpriv *fpriv = uq_mgr_to_fpriv(uq_mgr); | |
| 620 | − struct amdgpu_vm *vm = &fpriv->vm; | |
| 621 | − | |
| 507 | + const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type]; | |
| 622 | 508 | int r = 0; |
| 623 | 509 | |
| 624 | 510 | cancel_delayed_work_sync(&uq_mgr->resume_work); |
| @@ -626,33 +512,21 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que · 1 unmodified line | ||
| 626 | 512 | /* Cancel any pending hang detection work and cleanup */ |
| 627 | 513 | cancel_delayed_work_sync(&queue->hang_detect_work); |
| 628 | 514 | |
| 629 | − r = amdgpu_bo_reserve(vm->root.bo, false); | |
| 630 | − if (r) { | |
| 631 | − drm_file_err(uq_mgr->file, "Failed to reserve root bo during userqueue destroy\n"); | |
| 632 | − return r; | |
| 633 | − } | |
| 634 | − amdgpu_userq_buffer_vas_list_cleanup(adev, queue); | |
| 635 | − amdgpu_bo_unreserve(vm->root.bo); | |
| 636 | − | |
| 637 | 515 | mutex_lock(&uq_mgr->userq_mutex); |
| 638 | − queue->hang_detect_fence = NULL; | |
| 639 | 516 | amdgpu_userq_wait_for_last_fence(queue); |
| 640 | 517 | |
| 641 | 518 | #if defined(CONFIG_DEBUG_FS) |
| 642 | 519 | debugfs_remove_recursive(queue->debugfs_queue); |
| 643 | 520 | #endif |
| 644 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 645 | 521 | r = amdgpu_userq_unmap_helper(queue); |
| 646 | − /*TODO: It requires a reset for userq hw unmap error*/ | |
| 647 | − if (r) { | |
| 648 | − drm_warn(adev_to_drm(uq_mgr->adev), "trying to destroy a HW mapping userq\n"); | |
| 649 | − queue->state = AMDGPU_USERQ_STATE_HUNG; | |
| 650 | − } | |
| 651 | − | |
| 652 | 522 | atomic_dec(&uq_mgr->userq_count[queue->queue_type]); |
| 653 | 523 | amdgpu_userq_cleanup(queue); |
| 654 | 524 | mutex_unlock(&uq_mgr->userq_mutex); |
| 655 | 525 | |
| 526 | + cancel_delayed_work_sync(&queue->hang_detect_work); | |
| 527 | + uq_funcs->mqd_destroy(queue); | |
| 528 | + queue->userq_mgr = NULL; | |
| 529 | + | |
| 656 | 530 | amdgpu_bo_reserve(queue->db_obj.obj, true); |
| 657 | 531 | amdgpu_bo_unpin(queue->db_obj.obj); |
| 658 | 532 | amdgpu_bo_unreserve(queue->db_obj.obj); |
| @@ -724,14 +598,14 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 65 unmodified lines | ||
| 724 | 598 | const struct amdgpu_userq_funcs *uq_funcs; |
| 725 | 599 | struct amdgpu_usermode_queue *queue; |
| 726 | 600 | struct amdgpu_db_info db_info; |
| 727 | − bool skip_map_queue; | |
| 728 | − u32 qid; | |
| 729 | 601 | uint64_t index; |
| 730 | − int r = 0; | |
| 731 | − int priority = | |
| 732 | − (args->in.flags & AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_MASK) >> | |
| 733 | − AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_SHIFT; | |
| 602 | + int priority; | |
| 603 | + u32 qid; | |
| 604 | + int r; | |
| 734 | 605 | |
| 606 | + priority = | |
| 607 | + (args->in.flags & AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_MASK) | |
| 608 | + >> AMDGPU_USERQ_CREATE_FLAGS_QUEUE_PRIORITY_SHIFT; | |
| 735 | 609 | r = amdgpu_userq_priority_permit(filp, priority); |
| 736 | 610 | if (r) |
| 737 | 611 | return r; |
| @@ -744,128 +618,121 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 6 unmodified lines | ||
| 744 | 618 | |
| 745 | 619 | uq_funcs = adev->userq_funcs[args->in.ip_type]; |
| 746 | 620 | if (!uq_funcs) { |
| 747 | − drm_file_err(uq_mgr->file, "Usermode queue is not supported for this IP (%u)\n", | |
| 748 | − args->in.ip_type); | |
| 749 | 621 | r = -EINVAL; |
| 750 | 622 | goto err_pm_runtime; |
| 751 | 623 | } |
| 752 | 624 | |
| 753 | 625 | queue = kzalloc_obj(struct amdgpu_usermode_queue); |
| 754 | 626 | if (!queue) { |
| 755 | − drm_file_err(uq_mgr->file, "Failed to allocate memory for queue\n"); | |
| 756 | 627 | r = -ENOMEM; |
| 757 | 628 | goto err_pm_runtime; |
| 758 | 629 | } |
| 759 | 630 | |
| 760 | − INIT_LIST_HEAD(&queue->userq_va_list); | |
| 631 | + kref_init(&queue->refcount); | |
| 761 | 632 | queue->doorbell_handle = args->in.doorbell_handle; |
| 762 | 633 | queue->queue_type = args->in.ip_type; |
| 763 | 634 | queue->vm = &fpriv->vm; |
| 764 | 635 | queue->priority = priority; |
| 765 | − | |
| 766 | − db_info.queue_type = queue->queue_type; | |
| 767 | − db_info.doorbell_handle = queue->doorbell_handle; | |
| 768 | − db_info.db_obj = &queue->db_obj; | |
| 769 | − db_info.doorbell_offset = args->in.doorbell_offset; | |
| 770 | − | |
| 771 | 636 | queue->userq_mgr = uq_mgr; |
| 637 | + INIT_DELAYED_WORK(&queue->hang_detect_work, | |
| 638 | + amdgpu_userq_hang_detect_work); | |
| 772 | 639 | |
| 773 | − /* Validate the userq virtual address.*/ | |
| 774 | − r = amdgpu_bo_reserve(fpriv->vm.root.bo, false); | |
| 640 | + r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv); | |
| 775 | 641 | if (r) |
| 776 | 642 | goto free_queue; |
| 777 | 643 | |
| 778 | − if (amdgpu_userq_input_va_validate(adev, queue, args->in.queue_va, args->in.queue_size) || | |
| 779 | − amdgpu_userq_input_va_validate(adev, queue, args->in.rptr_va, AMDGPU_GPU_PAGE_SIZE) || | |
| 780 | − amdgpu_userq_input_va_validate(adev, queue, args->in.wptr_va, AMDGPU_GPU_PAGE_SIZE)) { | |
| 644 | + xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC); | |
| 645 | + mutex_init(&queue->fence_drv_lock); | |
| 646 | + /* Make sure the queue can actually run with those virtual addresses. */ | |
| 647 | + r = amdgpu_bo_reserve(fpriv->vm.root.bo, false); | |
| 648 | + if (r) | |
| 649 | + goto free_fence_drv; | |
| 650 | + | |
| 651 | + if (amdgpu_userq_input_va_validate(adev, queue, args->in.queue_va, | |
| 652 | + args->in.queue_size, | |
| 653 | + &queue->userq_vas.va.queue_rb) || | |
| 654 | + amdgpu_userq_input_va_validate(adev, queue, args->in.rptr_va, | |
| 655 | + AMDGPU_GPU_PAGE_SIZE, | |
| 656 | + &queue->userq_vas.va.rptr) || | |
| 657 | + amdgpu_userq_input_va_validate(adev, queue, args->in.wptr_va, | |
| 658 | + AMDGPU_GPU_PAGE_SIZE, | |
| 659 | + &queue->userq_vas.va.wptr)) { | |
| 781 | 660 | r = -EINVAL; |
| 782 | 661 | amdgpu_bo_unreserve(fpriv->vm.root.bo); |
| 783 | − goto clean_mapping; | |
| 662 | + goto free_fence_drv; | |
| 784 | 663 | } |
| 785 | 664 | amdgpu_bo_unreserve(fpriv->vm.root.bo); |
| 786 | 665 | |
| 787 | 666 | /* Convert relative doorbell offset into absolute doorbell index */ |
| 788 | − index = amdgpu_userq_get_doorbell_index(uq_mgr, &db_info, filp); | |
| 789 | − if (index == (uint64_t)-EINVAL) { | |
| 667 | + db_info.queue_type = queue->queue_type; | |
| 668 | + db_info.doorbell_handle = queue->doorbell_handle; | |
| 669 | + db_info.db_obj = &queue->db_obj; | |
| 670 | + db_info.doorbell_offset = args->in.doorbell_offset; | |
| 671 | + r = amdgpu_userq_get_doorbell_index(uq_mgr, &db_info, filp, &index); | |
| 672 | + if (r) { | |
| 790 | 673 | drm_file_err(uq_mgr->file, "Failed to get doorbell for queue\n"); |
| 791 | − r = -EINVAL; | |
| 792 | − goto clean_mapping; | |
| 674 | + goto free_fence_drv; | |
| 793 | 675 | } |
| 794 | 676 | |
| 795 | 677 | queue->doorbell_index = index; |
| 796 | − xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC); | |
| 797 | − r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv); | |
| 798 | − if (r) { | |
| 799 | − drm_file_err(uq_mgr->file, "Failed to alloc fence driver\n"); | |
| 800 | − goto clean_mapping; | |
| 801 | − } | |
| 802 | − | |
| 803 | 678 | r = uq_funcs->mqd_create(queue, &args->in); |
| 804 | 679 | if (r) { |
| 805 | 680 | drm_file_err(uq_mgr->file, "Failed to create Queue\n"); |
| 806 | − goto clean_fence_driver; | |
| 681 | + goto clean_doorbell_bo; | |
| 807 | 682 | } |
| 808 | 683 | |
| 684 | + /* Update VM owner at userq submit-time for page-fault attribution. */ | |
| 685 | + amdgpu_vm_set_task_info(&fpriv->vm); | |
| 686 | + | |
| 687 | + r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, | |
| 688 | + GFP_KERNEL)); | |
| 689 | + if (r) | |
| 690 | + goto clean_mqd; | |
| 691 | + | |
| 809 | 692 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); |
| 810 | 693 | |
| 811 | 694 | /* don't map the queue if scheduling is halted */ |
| 812 | − if (adev->userq_halt_for_enforce_isolation && | |
| 813 | − ((queue->queue_type == AMDGPU_HW_IP_GFX) || | |
| 814 | − (queue->queue_type == AMDGPU_HW_IP_COMPUTE))) | |
| 815 | − skip_map_queue = true; | |
| 816 | − else | |
| 817 | − skip_map_queue = false; | |
| 818 | − if (!skip_map_queue) { | |
| 695 | + if (!adev->userq_halt_for_enforce_isolation || | |
| 696 | + ((queue->queue_type != AMDGPU_HW_IP_GFX) && | |
| 697 | + (queue->queue_type != AMDGPU_HW_IP_COMPUTE))) { | |
| 819 | 698 | r = amdgpu_userq_map_helper(queue); |
| 820 | 699 | if (r) { |
| 821 | 700 | drm_file_err(uq_mgr->file, "Failed to map Queue\n"); |
| 822 | − goto clean_mqd; | |
| 701 | + mutex_unlock(&uq_mgr->userq_mutex); | |
| 702 | + goto erase_doorbell; | |
| 823 | 703 | } |
| 824 | 704 | } |
| 825 | 705 | |
| 826 | − /* drop this refcount during queue destroy */ | |
| 827 | − kref_init(&queue->refcount); | |
| 828 | − | |
| 829 | − /* Wait for mode-1 reset to complete */ | |
| 830 | − down_read(&adev->reset_domain->sem); | |
| 706 | + atomic_inc(&uq_mgr->userq_count[queue->queue_type]); | |
| 707 | + mutex_unlock(&uq_mgr->userq_mutex); | |
| 831 | 708 | |
| 832 | 709 | r = xa_alloc(&uq_mgr->userq_xa, &qid, queue, |
| 833 | − XA_LIMIT(1, AMDGPU_MAX_USERQ_COUNT), GFP_KERNEL); | |
| 834 | − if (r) { | |
| 835 | − if (!skip_map_queue) | |
| 836 | − amdgpu_userq_unmap_helper(queue); | |
| 837 | − r = -ENOMEM; | |
| 838 | − goto clean_reset_domain; | |
| 839 | − } | |
| 840 | − | |
| 841 | − r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, GFP_KERNEL)); | |
| 710 | + XA_LIMIT(1, AMDGPU_MAX_USERQ_COUNT), | |
| 711 | + GFP_KERNEL); | |
| 842 | 712 | if (r) { |
| 843 | − xa_erase(&uq_mgr->userq_xa, qid); | |
| 844 | − if (!skip_map_queue) | |
| 845 | − amdgpu_userq_unmap_helper(queue); | |
| 846 | − goto clean_reset_domain; | |
| 713 | + /* | |
| 714 | + * This drops the last reference which should take care of | |
| 715 | + * all cleanup. | |
| 716 | + */ | |
| 717 | + amdgpu_userq_put(queue); | |
| 718 | + return r; | |
| 847 | 719 | } |
| 848 | − up_read(&adev->reset_domain->sem); | |
| 849 | 720 | |
| 850 | 721 | amdgpu_debugfs_userq_init(filp, queue, qid); |
| 851 | − amdgpu_userq_init_hang_detect_work(queue); | |
| 852 | − | |
| 853 | 722 | args->out.queue_id = qid; |
| 854 | − atomic_inc(&uq_mgr->userq_count[queue->queue_type]); | |
| 855 | − mutex_unlock(&uq_mgr->userq_mutex); | |
| 856 | 723 | return 0; |
| 857 | 724 | |
| 858 | −clean_reset_domain: | |
| 859 | − up_read(&adev->reset_domain->sem); | |
| 725 | +erase_doorbell: | |
| 726 | + xa_erase_irq(&adev->userq_doorbell_xa, index); | |
| 860 | 727 | clean_mqd: |
| 861 | − mutex_unlock(&uq_mgr->userq_mutex); | |
| 862 | 728 | uq_funcs->mqd_destroy(queue); |
| 863 | −clean_fence_driver: | |
| 729 | +clean_doorbell_bo: | |
| 730 | + amdgpu_bo_reserve(queue->db_obj.obj, true); | |
| 731 | + amdgpu_bo_unpin(queue->db_obj.obj); | |
| 732 | + amdgpu_bo_unreserve(queue->db_obj.obj); | |
| 733 | + amdgpu_bo_unref(&queue->db_obj.obj); | |
| 734 | +free_fence_drv: | |
| 864 | 735 | amdgpu_userq_fence_driver_free(queue); |
| 865 | −clean_mapping: | |
| 866 | − amdgpu_bo_reserve(fpriv->vm.root.bo, true); | |
| 867 | − amdgpu_userq_buffer_vas_list_cleanup(adev, queue); | |
| 868 | − amdgpu_bo_unreserve(fpriv->vm.root.bo); | |
| 869 | 736 | free_queue: |
| 870 | 737 | kfree(queue); |
| 871 | 738 | err_pm_runtime: |
| @@ -1187,7 +1054,7 @@ retry_lock: · 315 unmodified lines | ||
| 1187 | 1054 | bo = range->bo; |
| 1188 | 1055 | ret = amdgpu_ttm_tt_get_user_pages(bo, range); |
| 1189 | 1056 | if (ret) |
| 1190 | − goto unlock_all; | |
| 1057 | + goto free_ranges; | |
| 1191 | 1058 | } |
| 1192 | 1059 | |
| 1193 | 1060 | invalidated = true; |
| @@ -1214,6 +1081,7 @@ retry_lock: · 20 unmodified lines | ||
| 1214 | 1081 | |
| 1215 | 1082 | unlock_all: |
| 1216 | 1083 | drm_exec_fini(&exec); |
| 1084 | +free_ranges: | |
| 1217 | 1085 | xa_for_each(&xa, tmp_key, range) { |
| 1218 | 1086 | if (!range) |
| 1219 | 1087 | continue; |
| @@ -1254,7 +1122,6 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 34 unmodified lines | ||
| 1254 | 1122 | unsigned long queue_id; |
| 1255 | 1123 | int ret = 0, r; |
| 1256 | 1124 | |
| 1257 | − amdgpu_userq_detect_and_reset_queues(uq_mgr); | |
| 1258 | 1125 | /* Try to unmap all the queues in this process ctx */ |
| 1259 | 1126 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1260 | 1127 | r = amdgpu_userq_preempt_helper(queue); |
| @@ -1262,29 +1129,16 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 1 unmodified line | ||
| 1262 | 1129 | ret = r; |
| 1263 | 1130 | } |
| 1264 | 1131 | |
| 1265 | − if (ret) | |
| 1132 | + if (ret) { | |
| 1266 | 1133 | drm_file_err(uq_mgr->file, |
| 1267 | 1134 | "Couldn't unmap all the queues, eviction failed ret=%d\n", ret); |
| 1135 | + amdgpu_reset_domain_schedule(uq_mgr->adev->reset_domain, | |
| 1136 | + &uq_mgr->reset_work); | |
| 1137 | + flush_work(&uq_mgr->reset_work); | |
| 1138 | + } | |
| 1268 | 1139 | return ret; |
| 1269 | 1140 | } |
| 1270 | 1141 | |
| 1271 | −void amdgpu_userq_reset_work(struct work_struct *work) | |
| 1272 | −{ | |
| 1273 | − struct amdgpu_device *adev = container_of(work, struct amdgpu_device, | |
| 1274 | − userq_reset_work); | |
| 1275 | − struct amdgpu_reset_context reset_context; | |
| 1276 | − | |
| 1277 | − memset(&reset_context, 0, sizeof(reset_context)); | |
| 1278 | − | |
| 1279 | − reset_context.method = AMD_RESET_METHOD_NONE; | |
| 1280 | − reset_context.reset_req_dev = adev; | |
| 1281 | − reset_context.src = AMDGPU_RESET_SRC_USERQ; | |
| 1282 | − set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | |
| 1283 | − /*set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags);*/ | |
| 1284 | − | |
| 1285 | − amdgpu_device_gpu_recover(adev, NULL, &reset_context); | |
| 1286 | −} | |
| 1287 | − | |
| 1288 | 1142 | static void |
| 1289 | 1143 | amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr) |
| 1290 | 1144 | { |
| @@ -1318,9 +1172,24 @@ int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *f · 27 unmodified lines | ||
| 1318 | 1172 | userq_mgr->file = file_priv; |
| 1319 | 1173 | |
| 1320 | 1174 | INIT_DELAYED_WORK(&userq_mgr->resume_work, amdgpu_userq_restore_worker); |
| 1175 | + INIT_WORK(&userq_mgr->reset_work, amdgpu_userq_mgr_reset_work); | |
| 1321 | 1176 | return 0; |
| 1322 | 1177 | } |
| 1323 | 1178 | |
| 1179 | +void amdgpu_userq_mgr_cancel_reset_work(struct amdgpu_device *adev) | |
| 1180 | +{ | |
| 1181 | + struct xarray *xa = &adev->userq_doorbell_xa; | |
| 1182 | + struct amdgpu_usermode_queue *queue; | |
| 1183 | + unsigned long flags, queue_id; | |
| 1184 | + | |
| 1185 | + xa_lock_irqsave(xa, flags); | |
| 1186 | + xa_for_each(xa, queue_id, queue) { | |
| 1187 | + cancel_delayed_work(&queue->hang_detect_work); | |
| 1188 | + cancel_work(&queue->userq_mgr->reset_work); | |
| 1189 | + } | |
| 1190 | + xa_unlock_irqrestore(xa, flags); | |
| 1191 | +} | |
| 1192 | + | |
| 1324 | 1193 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr) |
| 1325 | 1194 | { |
| 1326 | 1195 | cancel_delayed_work_sync(&userq_mgr->resume_work); |
| @@ -1346,6 +1215,14 @@ void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr) · 19 unmodified lines | ||
| 1346 | 1215 | } |
| 1347 | 1216 | |
| 1348 | 1217 | xa_destroy(&userq_mgr->userq_xa); |
| 1218 | + | |
| 1219 | + /* | |
| 1220 | + * Drain any in-flight reset_work. By this point all queues are freed | |
| 1221 | + * and userq_count is 0, so if reset_work starts now it exits early. | |
| 1222 | + * We still need to wait in case it was already executing gpu_recover. | |
| 1223 | + */ | |
| 1224 | + cancel_work_sync(&userq_mgr->reset_work); | |
| 1225 | + | |
| 1349 | 1226 | mutex_destroy(&userq_mgr->userq_mutex); |
| 1350 | 1227 | } |
| 1351 | 1228 | |
| @@ -1364,7 +1241,6 @@ int amdgpu_userq_suspend(struct amdgpu_device *adev) · 12 unmodified lines | ||
| 1364 | 1241 | uqm = queue->userq_mgr; |
| 1365 | 1242 | cancel_delayed_work_sync(&uqm->resume_work); |
| 1366 | 1243 | guard(mutex)(&uqm->userq_mutex); |
| 1367 | − amdgpu_userq_detect_and_reset_queues(uqm); | |
| 1368 | 1244 | if (adev->in_s0ix) |
| 1369 | 1245 | r = amdgpu_userq_preempt_helper(queue); |
| 1370 | 1246 | else |
| @@ -1423,7 +1299,6 @@ int amdgpu_userq_stop_sched_for_enforce_isolation(struct amdgpu_device *adev, · 52 unmodified lines | ||
| 1423 | 1299 | if (((queue->queue_type == AMDGPU_HW_IP_GFX) || |
| 1424 | 1300 | (queue->queue_type == AMDGPU_HW_IP_COMPUTE)) && |
| 1425 | 1301 | (queue->xcp_id == idx)) { |
| 1426 | − amdgpu_userq_detect_and_reset_queues(uqm); | |
| 1427 | 1302 | r = amdgpu_userq_preempt_helper(queue); |
| 1428 | 1303 | if (r) |
| 1429 | 1304 | ret = r; |
| @@ -1496,23 +1371,21 @@ void amdgpu_userq_pre_reset(struct amdgpu_device *adev) · 66 unmodified lines | ||
| 1496 | 1371 | { |
| 1497 | 1372 | const struct amdgpu_userq_funcs *userq_funcs; |
| 1498 | 1373 | struct amdgpu_usermode_queue *queue; |
| 1499 | − struct amdgpu_userq_mgr *uqm; | |
| 1500 | 1374 | unsigned long queue_id; |
| 1501 | 1375 | |
| 1376 | + /* TODO: We probably need a new lock for the queue state */ | |
| 1502 | 1377 | xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) { |
| 1503 | − uqm = queue->userq_mgr; | |
| 1504 | − cancel_delayed_work_sync(&uqm->resume_work); | |
| 1505 | − if (queue->state == AMDGPU_USERQ_STATE_MAPPED) { | |
| 1506 | − amdgpu_userq_wait_for_last_fence(queue); | |
| 1507 | − userq_funcs = adev->userq_funcs[queue->queue_type]; | |
| 1508 | − userq_funcs->unmap(queue); | |
| 1509 | − /* just mark all queues as hung at this point. | |
| 1510 | − * if unmap succeeds, we could map again | |
| 1511 | − * in amdgpu_userq_post_reset() if vram is not lost | |
| 1512 | − */ | |
| 1513 | − queue->state = AMDGPU_USERQ_STATE_HUNG; | |
| 1514 | − amdgpu_userq_fence_driver_force_completion(queue); | |
| 1515 | − } | |
| 1378 | + if (queue->state != AMDGPU_USERQ_STATE_MAPPED) | |
| 1379 | + continue; | |
| 1380 | + | |
| 1381 | + userq_funcs = adev->userq_funcs[queue->queue_type]; | |
| 1382 | + userq_funcs->unmap(queue); | |
| 1383 | + /* just mark all queues as hung at this point. | |
| 1384 | + * if unmap succeeds, we could map again | |
| 1385 | + * in amdgpu_userq_post_reset() if vram is not lost | |
| 1386 | + */ | |
| 1387 | + queue->state = AMDGPU_USERQ_STATE_HUNG; | |
| 1388 | + amdgpu_userq_fence_driver_force_completion(queue); | |
| 1516 | 1389 | } |
| 1517 | 1390 | } |
| 1518 | 1391 | |
| @@ -48,11 +48,6 @@ struct amdgpu_userq_obj { · 47 unmodified lines | ||
| 48 | 48 | struct amdgpu_bo *obj; |
| 49 | 49 | }; |
| 50 | 50 | |
| 51 | −struct amdgpu_userq_va_cursor { | |
| 52 | − u64 gpu_addr; | |
| 53 | − struct list_head list; | |
| 54 | −}; | |
| 55 | − | |
| 56 | 51 | struct amdgpu_usermode_queue { |
| 57 | 52 | int queue_type; |
| 58 | 53 | enum amdgpu_userq_state state; |
| @@ -66,17 +61,44 @@ struct amdgpu_usermode_queue { · 7 unmodified lines | ||
| 66 | 61 | struct amdgpu_userq_obj db_obj; |
| 67 | 62 | struct amdgpu_userq_obj fw_obj; |
| 68 | 63 | struct amdgpu_userq_obj wptr_obj; |
| 64 | + | |
| 65 | + /** | |
| 66 | + * @fence_drv_lock: Protecting @fence_drv_xa. | |
| 67 | + */ | |
| 68 | + struct mutex fence_drv_lock; | |
| 69 | + | |
| 70 | + /** | |
| 71 | + * @fence_drv_xa: | |
| 72 | + * | |
| 73 | + * References to the external fence drivers returned by wait_ioctl. | |
| 74 | + * Dropped on the next signaled dma_fence or queue destruction. | |
| 75 | + */ | |
| 69 | 76 | struct xarray fence_drv_xa; |
| 70 | 77 | struct amdgpu_userq_fence_driver *fence_drv; |
| 71 | 78 | struct dma_fence *last_fence; |
| 72 | 79 | u32 xcp_id; |
| 73 | 80 | int priority; |
| 74 | 81 | struct dentry *debugfs_queue; |
| 75 | − struct delayed_work hang_detect_work; | |
| 76 | − struct dma_fence *hang_detect_fence; | |
| 82 | + | |
| 83 | + /** | |
| 84 | + * @hang_detect_work: | |
| 85 | + * | |
| 86 | + * Delayed work which runs when userq_fences time out. | |
| 87 | + */ | |
| 88 | + struct delayed_work hang_detect_work; | |
| 77 | 89 | struct kref refcount; |
| 78 | 90 | |
| 79 | − struct list_head userq_va_list; | |
| 91 | + union { | |
| 92 | + struct { | |
| 93 | + u64 queue_rb; | |
| 94 | + u64 wptr; | |
| 95 | + u64 rptr; | |
| 96 | + u64 eop; | |
| 97 | + u64 shadow; | |
| 98 | + u64 csa; | |
| 99 | + } va; | |
| 100 | + u64 va_array[6]; | |
| 101 | + } userq_vas; | |
| 80 | 102 | }; |
| 81 | 103 | |
| 82 | 104 | struct amdgpu_userq_funcs { |
| @@ -105,6 +127,13 @@ struct amdgpu_userq_mgr { · 22 unmodified lines | ||
| 105 | 127 | struct amdgpu_device *adev; |
| 106 | 128 | struct delayed_work resume_work; |
| 107 | 129 | struct drm_file *file; |
| 130 | + | |
| 131 | + /** | |
| 132 | + * @reset_work: | |
| 133 | + * | |
| 134 | + * Reset work which is used when eviction fails. | |
| 135 | + */ | |
| 136 | + struct work_struct reset_work; | |
| 108 | 137 | atomic_t userq_count[AMDGPU_RING_TYPE_MAX]; |
| 109 | 138 | }; |
| 110 | 139 | |
| @@ -123,25 +152,15 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp · 12 unmodified lines | ||
| 123 | 152 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, |
| 124 | 153 | struct amdgpu_device *adev); |
| 125 | 154 | |
| 155 | +void amdgpu_userq_mgr_cancel_reset_work(struct amdgpu_device *adev); | |
| 126 | 156 | void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr); |
| 127 | 157 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr); |
| 128 | 158 | |
| 129 | −int amdgpu_userq_create_object(struct amdgpu_userq_mgr *uq_mgr, | |
| 130 | − struct amdgpu_userq_obj *userq_obj, | |
| 131 | − int size); | |
| 132 | − | |
| 133 | −void amdgpu_userq_destroy_object(struct amdgpu_userq_mgr *uq_mgr, | |
| 134 | − struct amdgpu_userq_obj *userq_obj); | |
| 135 | − | |
| 136 | 159 | void amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr); |
| 137 | 160 | |
| 138 | 161 | void amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *userq_mgr, |
| 139 | 162 | struct amdgpu_eviction_fence_mgr *evf_mgr); |
| 140 | 163 | |
| 141 | −uint64_t amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, | |
| 142 | − struct amdgpu_db_info *db_info, | |
| 143 | − struct drm_file *filp); | |
| 144 | − | |
| 145 | 164 | u32 amdgpu_userq_get_supported_ip_mask(struct amdgpu_device *adev); |
| 146 | 165 | bool amdgpu_userq_enabled(struct drm_device *dev); |
| 147 | 166 | |
| @@ -156,10 +175,12 @@ void amdgpu_userq_reset_work(struct work_struct *work); · 8 unmodified lines | ||
| 156 | 175 | void amdgpu_userq_pre_reset(struct amdgpu_device *adev); |
| 157 | 176 | int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost); |
| 158 | 177 | void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue); |
| 178 | +void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell); | |
| 159 | 179 | |
| 160 | 180 | int amdgpu_userq_input_va_validate(struct amdgpu_device *adev, |
| 161 | 181 | struct amdgpu_usermode_queue *queue, |
| 162 | − u64 addr, u64 expected_size); | |
| 182 | + u64 addr, u64 expected_size, u64 *va_out); | |
| 183 | + | |
| 163 | 184 | void amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev, |
| 164 | 185 | struct amdgpu_bo_va_mapping *mapping, |
| 165 | 186 | uint64_t saddr); |
| @@ -32,29 +32,9 @@ · 31 unmodified lines | ||
| 32 | 32 | #include "amdgpu.h" |
| 33 | 33 | #include "amdgpu_userq_fence.h" |
| 34 | 34 | |
| 35 | −static const struct dma_fence_ops amdgpu_userq_fence_ops; | |
| 36 | −static struct kmem_cache *amdgpu_userq_fence_slab; | |
| 37 | − | |
| 38 | 35 | #define AMDGPU_USERQ_MAX_HANDLES (1U << 16) |
| 39 | 36 | |
| 40 | −int amdgpu_userq_fence_slab_init(void) | |
| 41 | −{ | |
| 42 | − amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence", | |
| 43 | − sizeof(struct amdgpu_userq_fence), | |
| 44 | − 0, | |
| 45 | − SLAB_HWCACHE_ALIGN, | |
| 46 | − NULL); | |
| 47 | − if (!amdgpu_userq_fence_slab) | |
| 48 | − return -ENOMEM; | |
| 49 | − | |
| 50 | − return 0; | |
| 51 | −} | |
| 52 | − | |
| 53 | −void amdgpu_userq_fence_slab_fini(void) | |
| 54 | −{ | |
| 55 | − rcu_barrier(); | |
| 56 | − kmem_cache_destroy(amdgpu_userq_fence_slab); | |
| 57 | −} | |
| 37 | +static const struct dma_fence_ops amdgpu_userq_fence_ops; | |
| 58 | 38 | |
| 59 | 39 | static inline struct amdgpu_userq_fence *to_amdgpu_userq_fence(struct dma_fence *f) |
| 60 | 40 | { |
| @@ -141,6 +121,7 @@ amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq) · 80 unmodified lines | ||
| 141 | 121 | userq->last_fence = NULL; |
| 142 | 122 | amdgpu_userq_walk_and_drop_fence_drv(&userq->fence_drv_xa); |
| 143 | 123 | xa_destroy(&userq->fence_drv_xa); |
| 124 | + mutex_destroy(&userq->fence_drv_lock); | |
| 144 | 125 | /* Drop the queue's ownership reference to fence_drv explicitly */ |
| 145 | 126 | amdgpu_userq_fence_driver_put(userq->fence_drv); |
| 146 | 127 | } |
| @@ -154,7 +135,14 @@ amdgpu_userq_fence_put_fence_drv_array(struct amdgpu_userq_fence *userq_fence) · 7 unmodified lines | ||
| 154 | 135 | userq_fence->fence_drv_array_count = 0; |
| 155 | 136 | } |
| 156 | 137 | |
| 157 | −void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv) | |
| 138 | +/* | |
| 139 | + * Returns: | |
| 140 | + * -ENOENT when no fences were processes | |
| 141 | + * 1 when more fences are pending | |
| 142 | + * 0 when no fences are pending any more | |
| 143 | + */ | |
| 144 | +int | |
| 145 | +amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv) | |
| 158 | 146 | { |
| 159 | 147 | struct amdgpu_userq_fence *userq_fence, *tmp; |
| 160 | 148 | LIST_HEAD(to_be_signaled); |
| @@ -162,9 +150,6 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 1 unmodified line | ||
| 162 | 150 | unsigned long flags; |
| 163 | 151 | u64 rptr; |
| 164 | 152 | |
| 165 | − if (!fence_drv) | |
| 166 | − return; | |
| 167 | − | |
| 168 | 153 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); |
| 169 | 154 | rptr = amdgpu_userq_fence_read(fence_drv); |
| 170 | 155 | |
| @@ -177,6 +162,9 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 6 unmodified lines | ||
| 177 | 162 | &userq_fence->link); |
| 178 | 163 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); |
| 179 | 164 | |
| 165 | + if (list_empty(&to_be_signaled)) | |
| 166 | + return -ENOENT; | |
| 167 | + | |
| 180 | 168 | list_for_each_entry_safe(userq_fence, tmp, &to_be_signaled, link) { |
| 181 | 169 | fence = &userq_fence->base; |
| 182 | 170 | list_del_init(&userq_fence->link); |
| @@ -188,6 +176,8 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d · 5 unmodified lines | ||
| 188 | 176 | dma_fence_put(fence); |
| 189 | 177 | } |
| 190 | 178 | |
| 179 | + /* That doesn't need to be accurate so no locking */ | |
| 180 | + return list_empty(&fence_drv->fences) ? 0 : 1; | |
| 191 | 181 | } |
| 192 | 182 | |
| 193 | 183 | void amdgpu_userq_fence_driver_destroy(struct kref *ref) |
| @@ -229,80 +219,84 @@ void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv) · 35 unmodified lines | ||
| 229 | 219 | kref_put(&fence_drv->refcount, amdgpu_userq_fence_driver_destroy); |
| 230 | 220 | } |
| 231 | 221 | |
| 232 | −static int amdgpu_userq_fence_alloc(struct amdgpu_userq_fence **userq_fence) | |
| 222 | +static int amdgpu_userq_fence_alloc(struct amdgpu_usermode_queue *userq, | |
| 223 | + struct amdgpu_userq_fence **pfence) | |
| 233 | 224 | { |
| 234 | − *userq_fence = kmem_cache_alloc(amdgpu_userq_fence_slab, GFP_ATOMIC); | |
| 235 | − return *userq_fence ? 0 : -ENOMEM; | |
| 225 | + struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv; | |
| 226 | + struct amdgpu_userq_fence *userq_fence; | |
| 227 | + void *entry; | |
| 228 | + | |
| 229 | + userq_fence = kmalloc(sizeof(*userq_fence), GFP_KERNEL); | |
| 230 | + if (!userq_fence) | |
| 231 | + return -ENOMEM; | |
| 232 | + | |
| 233 | + /* | |
| 234 | + * Get the next unused entry, since we fill from the start this can be | |
| 235 | + * used as size to allocate the array. | |
| 236 | + */ | |
| 237 | + mutex_lock(&userq->fence_drv_lock); | |
| 238 | + XA_STATE(xas, &userq->fence_drv_xa, 0); | |
| 239 | + | |
| 240 | + rcu_read_lock(); | |
| 241 | + do { | |
| 242 | + entry = xas_find_marked(&xas, ULONG_MAX, XA_FREE_MARK); | |
| 243 | + } while (xas_retry(&xas, entry)); | |
| 244 | + rcu_read_unlock(); | |
| 245 | + | |
| 246 | + userq_fence->fence_drv_array = kvmalloc_array(xas.xa_index, | |
| 247 | + sizeof(fence_drv), | |
| 248 | + GFP_KERNEL); | |
| 249 | + if (!userq_fence->fence_drv_array) { | |
| 250 | + mutex_unlock(&userq->fence_drv_lock); | |
| 251 | + kfree(userq_fence); | |
| 252 | + return -ENOMEM; | |
| 253 | + } | |
| 254 | + | |
| 255 | + userq_fence->fence_drv_array_count = xas.xa_index; | |
| 256 | + xa_extract(&userq->fence_drv_xa, (void **)userq_fence->fence_drv_array, | |
| 257 | + 0, ULONG_MAX, xas.xa_index, XA_PRESENT); | |
| 258 | + xa_destroy(&userq->fence_drv_xa); | |
| 259 | + | |
| 260 | + mutex_unlock(&userq->fence_drv_lock); | |
| 261 | + | |
| 262 | + amdgpu_userq_fence_driver_get(fence_drv); | |
| 263 | + userq_fence->fence_drv = fence_drv; | |
| 264 | + | |
| 265 | + *pfence = userq_fence; | |
| 266 | + return 0; | |
| 236 | 267 | } |
| 237 | 268 | |
| 238 | −static int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq, | |
| 239 | − struct amdgpu_userq_fence *userq_fence, | |
| 240 | − u64 seq, struct dma_fence **f) | |
| 269 | +static void amdgpu_userq_fence_init(struct amdgpu_usermode_queue *userq, | |
| 270 | + struct amdgpu_userq_fence *fence, | |
| 271 | + u64 seq) | |
| 241 | 272 | { |
| 242 | − struct amdgpu_userq_fence_driver *fence_drv; | |
| 243 | − struct dma_fence *fence; | |
| 273 | + struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv; | |
| 244 | 274 | unsigned long flags; |
| 245 | 275 | bool signaled = false; |
| 246 | 276 | |
| 247 | − fence_drv = userq->fence_drv; | |
| 248 | − if (!fence_drv) | |
| 249 | − return -EINVAL; | |
| 250 | − | |
| 251 | − spin_lock_init(&userq_fence->lock); | |
| 252 | − INIT_LIST_HEAD(&userq_fence->link); | |
| 253 | − fence = &userq_fence->base; | |
| 254 | − userq_fence->fence_drv = fence_drv; | |
| 255 | − | |
| 256 | − dma_fence_init64(fence, &amdgpu_userq_fence_ops, &userq_fence->lock, | |
| 277 | + spin_lock_init(&fence->lock); | |
| 278 | + dma_fence_init64(&fence->base, &amdgpu_userq_fence_ops, &fence->lock, | |
| 257 | 279 | fence_drv->context, seq); |
| 258 | 280 | |
| 259 | − amdgpu_userq_fence_driver_get(fence_drv); | |
| 260 | − dma_fence_get(fence); | |
| 261 | − | |
| 262 | − if (!xa_empty(&userq->fence_drv_xa)) { | |
| 263 | − struct amdgpu_userq_fence_driver *stored_fence_drv; | |
| 264 | − unsigned long index, count = 0; | |
| 265 | − int i = 0; | |
| 266 | − | |
| 267 | − xa_lock(&userq->fence_drv_xa); | |
| 268 | − xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv) | |
| 269 | − count++; | |
| 270 | − | |
| 271 | − userq_fence->fence_drv_array = | |
| 272 | − kvmalloc_objs(struct amdgpu_userq_fence_driver *, count, | |
| 273 | − GFP_ATOMIC); | |
| 274 | − | |
| 275 | − if (userq_fence->fence_drv_array) { | |
| 276 | − xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv) { | |
| 277 | − userq_fence->fence_drv_array[i] = stored_fence_drv; | |
| 278 | − __xa_erase(&userq->fence_drv_xa, index); | |
| 279 | − i++; | |
| 280 | − } | |
| 281 | − } | |
| 282 | − | |
| 283 | − userq_fence->fence_drv_array_count = i; | |
| 284 | − xa_unlock(&userq->fence_drv_xa); | |
| 285 | − } else { | |
| 286 | − userq_fence->fence_drv_array = NULL; | |
| 287 | − userq_fence->fence_drv_array_count = 0; | |
| 288 | − } | |
| 281 | + /* Make sure the fence is visible to the hang detect worker */ | |
| 282 | + dma_fence_put(userq->last_fence); | |
| 283 | + userq->last_fence = dma_fence_get(&fence->base); | |
| 289 | 284 | |
| 290 | − /* Check if hardware has already processed the job */ | |
| 285 | + /* Check if hardware has already processed the fence */ | |
| 291 | 286 | spin_lock_irqsave(&fence_drv->fence_list_lock, flags); |
| 292 | − if (!dma_fence_is_signaled(fence)) { | |
| 293 | − list_add_tail(&userq_fence->link, &fence_drv->fences); | |
| 287 | + if (!dma_fence_is_signaled(&fence->base)) { | |
| 288 | + dma_fence_get(&fence->base); | |
| 289 | + list_add_tail(&fence->link, &fence_drv->fences); | |
| 294 | 290 | } else { |
| 291 | + INIT_LIST_HEAD(&fence->link); | |
| 295 | 292 | signaled = true; |
| 296 | − dma_fence_put(fence); | |
| 297 | 293 | } |
| 298 | 294 | spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); |
| 299 | 295 | |
| 300 | 296 | if (signaled) |
| 301 | − amdgpu_userq_fence_put_fence_drv_array(userq_fence); | |
| 302 | − | |
| 303 | − *f = fence; | |
| 304 | − | |
| 305 | − return 0; | |
| 297 | + amdgpu_userq_fence_put_fence_drv_array(fence); | |
| 298 | + else | |
| 299 | + amdgpu_userq_start_hang_detect_work(userq); | |
| 306 | 300 | } |
| 307 | 301 | |
| 308 | 302 | static const char *amdgpu_userq_fence_get_driver_name(struct dma_fence *f) |
| @@ -342,7 +336,7 @@ static void amdgpu_userq_fence_free(struct rcu_head *rcu) · 33 unmodified lines | ||
| 342 | 336 | amdgpu_userq_fence_driver_put(fence_drv); |
| 343 | 337 | |
| 344 | 338 | kvfree(userq_fence->fence_drv_array); |
| 345 | − kmem_cache_free(amdgpu_userq_fence_slab, userq_fence); | |
| 339 | + kfree(userq_fence); | |
| 346 | 340 | } |
| 347 | 341 | |
| 348 | 342 | static void amdgpu_userq_fence_release(struct dma_fence *f) |
| @@ -376,56 +370,48 @@ static int amdgpu_userq_fence_read_wptr(struct amdgpu_device *adev, · 27 unmodified lines | ||
| 376 | 370 | { |
| 377 | 371 | struct amdgpu_bo_va_mapping *mapping; |
| 378 | 372 | struct amdgpu_bo *bo; |
| 373 | + struct drm_exec exec; | |
| 379 | 374 | u64 addr, *ptr; |
| 380 | − int r; | |
| 381 | − | |
| 382 | − r = amdgpu_bo_reserve(queue->vm->root.bo, false); | |
| 383 | − if (r) | |
| 384 | − return r; | |
| 375 | + int ret; | |
| 385 | 376 | |
| 386 | 377 | addr = queue->userq_prop->wptr_gpu_addr; |
| 387 | 378 | addr &= AMDGPU_GMC_HOLE_MASK; |
| 388 | 379 | |
| 389 | − mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, addr >> PAGE_SHIFT); | |
| 390 | − if (!mapping) { | |
| 391 | − amdgpu_bo_unreserve(queue->vm->root.bo); | |
| 392 | − DRM_ERROR("Failed to lookup amdgpu_bo_va_mapping\n"); | |
| 393 | − return -EINVAL; | |
| 394 | − } | |
| 380 | + drm_exec_init(&exec, DRM_EXEC_IGNORE_DUPLICATES, 2); | |
| 381 | + drm_exec_until_all_locked(&exec) { | |
| 382 | + ret = amdgpu_vm_lock_pd(queue->vm, &exec, 1); | |
| 383 | + drm_exec_retry_on_contention(&exec); | |
| 384 | + if (unlikely(ret)) | |
| 385 | + goto lock_error; | |
| 386 | + | |
| 387 | + mapping = amdgpu_vm_bo_lookup_mapping(queue->vm, addr >> PAGE_SHIFT); | |
| 388 | + if (!mapping) { | |
| 389 | + ret = -EINVAL; | |
| 390 | + goto lock_error; | |
| 391 | + } | |
| 395 | 392 | |
| 396 | − bo = amdgpu_bo_ref(mapping->bo_va->base.bo); | |
| 397 | − amdgpu_bo_unreserve(queue->vm->root.bo); | |
| 398 | − r = amdgpu_bo_reserve(bo, true); | |
| 399 | − if (r) { | |
| 400 | − amdgpu_bo_unref(&bo); | |
| 401 | − DRM_ERROR("Failed to reserve userqueue wptr bo"); | |
| 402 | − return r; | |
| 393 | + ret = drm_exec_lock_obj(&exec, &mapping->bo_va->base.bo->tbo.base); | |
| 394 | + drm_exec_retry_on_contention(&exec); | |
| 395 | + if (unlikely(ret)) | |
| 396 | + goto lock_error; | |
| 403 | 397 | } |
| 404 | 398 | |
| 405 | − r = amdgpu_bo_kmap(bo, (void **)&ptr); | |
| 406 | − if (r) { | |
| 399 | + bo = mapping->bo_va->base.bo; | |
| 400 | + ret = amdgpu_bo_kmap(bo, (void **)&ptr); | |
| 401 | + if (ret) { | |
| 407 | 402 | DRM_ERROR("Failed mapping the userqueue wptr bo"); |
| 408 | − goto map_error; | |
| 403 | + goto lock_error; | |
| 409 | 404 | } |
| 410 | 405 | |
| 411 | 406 | *wptr = le64_to_cpu(*ptr); |
| 412 | 407 | |
| 413 | 408 | amdgpu_bo_kunmap(bo); |
| 414 | − amdgpu_bo_unreserve(bo); | |
| 415 | − amdgpu_bo_unref(&bo); | |
| 416 | − | |
| 409 | + drm_exec_fini(&exec); | |
| 417 | 410 | return 0; |
| 418 | 411 | |
| 419 | −map_error: | |
| 420 | − amdgpu_bo_unreserve(bo); | |
| 421 | − amdgpu_bo_unref(&bo); | |
| 422 | − | |
| 423 | − return r; | |
| 424 | −} | |
| 425 | − | |
| 426 | −static void amdgpu_userq_fence_cleanup(struct dma_fence *fence) | |
| 427 | −{ | |
| 428 | − dma_fence_put(fence); | |
| 412 | +lock_error: | |
| 413 | + drm_exec_fini(&exec); | |
| 414 | + return ret; | |
| 429 | 415 | } |
| 430 | 416 | |
| 431 | 417 | static void |
| @@ -471,13 +457,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 39 unmodified lines | ||
| 471 | 457 | const unsigned int num_read_bo_handles = args->num_bo_read_handles; |
| 472 | 458 | struct amdgpu_fpriv *fpriv = filp->driver_priv; |
| 473 | 459 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; |
| 460 | + | |
| 474 | 461 | struct drm_gem_object **gobj_write, **gobj_read; |
| 475 | 462 | u32 *syncobj_handles, num_syncobj_handles; |
| 476 | − struct amdgpu_userq_fence *userq_fence; | |
| 477 | − struct amdgpu_usermode_queue *queue = NULL; | |
| 478 | − struct drm_syncobj **syncobj = NULL; | |
| 479 | − struct dma_fence *fence; | |
| 463 | + struct amdgpu_usermode_queue *queue; | |
| 464 | + struct amdgpu_userq_fence *fence; | |
| 465 | + struct drm_syncobj **syncobj; | |
| 480 | 466 | struct drm_exec exec; |
| 467 | + void __user *ptr; | |
| 481 | 468 | int r, i, entry; |
| 482 | 469 | u64 wptr; |
| 483 | 470 | |
| @@ -489,13 +476,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 5 unmodified lines | ||
| 489 | 476 | return -EINVAL; |
| 490 | 477 | |
| 491 | 478 | num_syncobj_handles = args->num_syncobj_handles; |
| 492 | − syncobj_handles = memdup_array_user(u64_to_user_ptr(args->syncobj_handles), | |
| 493 | − num_syncobj_handles, sizeof(u32)); | |
| 479 | + ptr = u64_to_user_ptr(args->syncobj_handles); | |
| 480 | + syncobj_handles = memdup_array_user(ptr, num_syncobj_handles, | |
| 481 | + sizeof(u32)); | |
| 494 | 482 | if (IS_ERR(syncobj_handles)) |
| 495 | 483 | return PTR_ERR(syncobj_handles); |
| 496 | 484 | |
| 497 | − /* Array of pointers to the looked up syncobjs */ | |
| 498 | − syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), GFP_KERNEL); | |
| 485 | + syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), | |
| 486 | + GFP_KERNEL); | |
| 499 | 487 | if (!syncobj) { |
| 500 | 488 | r = -ENOMEM; |
| 501 | 489 | goto free_syncobj_handles; |
| @@ -509,21 +497,17 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 7 unmodified lines | ||
| 509 | 497 | } |
| 510 | 498 | } |
| 511 | 499 | |
| 512 | − r = drm_gem_objects_lookup(filp, | |
| 513 | − u64_to_user_ptr(args->bo_read_handles), | |
| 514 | − num_read_bo_handles, | |
| 515 | − &gobj_read); | |
| 500 | + ptr = u64_to_user_ptr(args->bo_read_handles); | |
| 501 | + r = drm_gem_objects_lookup(filp, ptr, num_read_bo_handles, &gobj_read); | |
| 516 | 502 | if (r) |
| 517 | 503 | goto free_syncobj; |
| 518 | 504 | |
| 519 | − r = drm_gem_objects_lookup(filp, | |
| 520 | − u64_to_user_ptr(args->bo_write_handles), | |
| 521 | − num_write_bo_handles, | |
| 505 | + ptr = u64_to_user_ptr(args->bo_write_handles); | |
| 506 | + r = drm_gem_objects_lookup(filp, ptr, num_write_bo_handles, | |
| 522 | 507 | &gobj_write); |
| 523 | 508 | if (r) |
| 524 | 509 | goto put_gobj_read; |
| 525 | 510 | |
| 526 | − /* Retrieve the user queue */ | |
| 527 | 511 | queue = amdgpu_userq_get(userq_mgr, args->queue_id); |
| 528 | 512 | if (!queue) { |
| 529 | 513 | r = -ENOENT; |
| @@ -532,73 +516,61 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 2 unmodified lines | ||
| 532 | 516 | |
| 533 | 517 | r = amdgpu_userq_fence_read_wptr(adev, queue, &wptr); |
| 534 | 518 | if (r) |
| 535 | − goto put_gobj_write; | |
| 519 | + goto put_queue; | |
| 536 | 520 | |
| 537 | − r = amdgpu_userq_fence_alloc(&userq_fence); | |
| 521 | + r = amdgpu_userq_fence_alloc(queue, &fence); | |
| 538 | 522 | if (r) |
| 539 | − goto put_gobj_write; | |
| 523 | + goto put_queue; | |
| 540 | 524 | |
| 541 | 525 | /* We are here means UQ is active, make sure the eviction fence is valid */ |
| 542 | 526 | amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr); |
| 543 | 527 | |
| 544 | − /* Create a new fence */ | |
| 545 | − r = amdgpu_userq_fence_create(queue, userq_fence, wptr, &fence); | |
| 546 | − if (r) { | |
| 547 | − mutex_unlock(&userq_mgr->userq_mutex); | |
| 548 | − kmem_cache_free(amdgpu_userq_fence_slab, userq_fence); | |
| 549 | − goto put_gobj_write; | |
| 550 | − } | |
| 528 | + /* Create the new fence */ | |
| 529 | + amdgpu_userq_fence_init(queue, fence, wptr); | |
| 551 | 530 | |
| 552 | − dma_fence_put(queue->last_fence); | |
| 553 | − queue->last_fence = dma_fence_get(fence); | |
| 554 | − amdgpu_userq_start_hang_detect_work(queue); | |
| 555 | 531 | mutex_unlock(&userq_mgr->userq_mutex); |
| 556 | 532 | |
| 533 | + /* | |
| 534 | + * This needs to come after the fence is created since | |
| 535 | + * amdgpu_userq_ensure_ev_fence() can't be called while holding the resv | |
| 536 | + * locks. | |
| 537 | + */ | |
| 557 | 538 | drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, |
| 558 | 539 | (num_read_bo_handles + num_write_bo_handles)); |
| 559 | 540 | |
| 560 | − /* Lock all BOs with retry handling */ | |
| 561 | 541 | drm_exec_until_all_locked(&exec) { |
| 562 | − r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); | |
| 542 | + r = drm_exec_prepare_array(&exec, gobj_read, | |
| 543 | + num_read_bo_handles, 1); | |
| 563 | 544 | drm_exec_retry_on_contention(&exec); |
| 564 | − if (r) { | |
| 565 | − amdgpu_userq_fence_cleanup(fence); | |
| 545 | + if (r) | |
| 566 | 546 | goto exec_fini; |
| 567 | − } | |
| 568 | 547 | |
| 569 | − r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); | |
| 548 | + r = drm_exec_prepare_array(&exec, gobj_write, | |
| 549 | + num_write_bo_handles, 1); | |
| 570 | 550 | drm_exec_retry_on_contention(&exec); |
| 571 | − if (r) { | |
| 572 | − amdgpu_userq_fence_cleanup(fence); | |
| 551 | + if (r) | |
| 573 | 552 | goto exec_fini; |
| 574 | − } | |
| 575 | 553 | } |
| 576 | 554 | |
| 577 | − for (i = 0; i < num_read_bo_handles; i++) { | |
| 578 | − if (!gobj_read || !gobj_read[i]->resv) | |
| 579 | − continue; | |
| 580 | − | |
| 581 | − dma_resv_add_fence(gobj_read[i]->resv, fence, | |
| 555 | + /* And publish the new fence in the BOs and syncobj */ | |
| 556 | + for (i = 0; i < num_read_bo_handles; i++) | |
| 557 | + dma_resv_add_fence(gobj_read[i]->resv, &fence->base, | |
| 582 | 558 | DMA_RESV_USAGE_READ); |
| 583 | − } | |
| 584 | 559 | |
| 585 | − for (i = 0; i < num_write_bo_handles; i++) { | |
| 586 | − if (!gobj_write || !gobj_write[i]->resv) | |
| 587 | − continue; | |
| 588 | − | |
| 589 | − dma_resv_add_fence(gobj_write[i]->resv, fence, | |
| 560 | + for (i = 0; i < num_write_bo_handles; i++) | |
| 561 | + dma_resv_add_fence(gobj_write[i]->resv, &fence->base, | |
| 590 | 562 | DMA_RESV_USAGE_WRITE); |
| 591 | − } | |
| 592 | 563 | |
| 593 | − /* Add the created fence to syncobj/BO's */ | |
| 594 | 564 | for (i = 0; i < num_syncobj_handles; i++) |
| 595 | − drm_syncobj_replace_fence(syncobj[i], fence); | |
| 565 | + drm_syncobj_replace_fence(syncobj[i], &fence->base); | |
| 596 | 566 | |
| 567 | +exec_fini: | |
| 597 | 568 | /* drop the reference acquired in fence creation function */ |
| 598 | − dma_fence_put(fence); | |
| 569 | + dma_fence_put(&fence->base); | |
| 599 | 570 | |
| 600 | −exec_fini: | |
| 601 | 571 | drm_exec_fini(&exec); |
| 572 | +put_queue: | |
| 573 | + amdgpu_userq_put(queue); | |
| 602 | 574 | put_gobj_write: |
| 603 | 575 | for (i = 0; i < num_write_bo_handles; i++) |
| 604 | 576 | drm_gem_object_put(gobj_write[i]); |
| @@ -609,15 +581,11 @@ put_gobj_read: · 4 unmodified lines | ||
| 609 | 581 | kvfree(gobj_read); |
| 610 | 582 | free_syncobj: |
| 611 | 583 | while (entry-- > 0) |
| 612 | − if (syncobj[entry]) | |
| 613 | − drm_syncobj_put(syncobj[entry]); | |
| 584 | + drm_syncobj_put(syncobj[entry]); | |
| 614 | 585 | kfree(syncobj); |
| 615 | 586 | free_syncobj_handles: |
| 616 | 587 | kfree(syncobj_handles); |
| 617 | 588 | |
| 618 | − if (queue) | |
| 619 | − amdgpu_userq_put(queue); | |
| 620 | − | |
| 621 | 589 | return r; |
| 622 | 590 | } |
| 623 | 591 | |
| @@ -892,8 +860,10 @@ amdgpu_userq_wait_return_fence_info(struct drm_file *filp, · 268 unmodified lines | ||
| 892 | 860 | * Otherwise, we would gather those references until we don't |
| 893 | 861 | * have any more space left and crash. |
| 894 | 862 | */ |
| 863 | + mutex_lock(&waitq->fence_drv_lock); | |
| 895 | 864 | r = xa_alloc(&waitq->fence_drv_xa, &index, fence_drv, |
| 896 | 865 | xa_limit_32b, GFP_KERNEL); |
| 866 | + mutex_unlock(&waitq->fence_drv_lock); | |
| 897 | 867 | if (r) |
| 898 | 868 | goto put_waitq; |
| 899 | 869 | |
| @@ -58,15 +58,12 @@ struct amdgpu_userq_fence_driver { · 57 unmodified lines | ||
| 58 | 58 | char timeline_name[TASK_COMM_LEN]; |
| 59 | 59 | }; |
| 60 | 60 | |
| 61 | −int amdgpu_userq_fence_slab_init(void); | |
| 62 | −void amdgpu_userq_fence_slab_fini(void); | |
| 63 | − | |
| 64 | 61 | void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv); |
| 65 | 62 | void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv); |
| 66 | 63 | int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev, |
| 67 | 64 | struct amdgpu_userq_fence_driver **fence_drv_req); |
| 68 | 65 | void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq); |
| 69 | −void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv); | |
| 66 | +int amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv); | |
| 70 | 67 | void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq); |
| 71 | 68 | void amdgpu_userq_fence_driver_destroy(struct kref *ref); |
| 72 | 69 | int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, |
| @@ -1631,6 +1631,7 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, · 1630 unmodified lines | ||
| 1631 | 1631 | { |
| 1632 | 1632 | struct amdgpu_bo_va *bo_va; |
| 1633 | 1633 | struct dma_resv *resv; |
| 1634 | + struct amdgpu_bo *bo; | |
| 1634 | 1635 | bool clear, unlock; |
| 1635 | 1636 | int r; |
| 1636 | 1637 | |
| @@ -1650,11 +1651,13 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 1650 | 1651 | while (!list_empty(&vm->invalidated)) { |
| 1651 | 1652 | bo_va = list_first_entry(&vm->invalidated, struct amdgpu_bo_va, |
| 1652 | 1653 | base.vm_status); |
| 1653 | − resv = bo_va->base.bo->tbo.base.resv; | |
| 1654 | + bo = bo_va->base.bo; | |
| 1655 | + resv = bo->tbo.base.resv; | |
| 1654 | 1656 | spin_unlock(&vm->status_lock); |
| 1655 | 1657 | |
| 1656 | 1658 | /* Try to reserve the BO to avoid clearing its ptes */ |
| 1657 | − if (!adev->debug_vm && dma_resv_trylock(resv)) { | |
| 1659 | + if (!adev->debug_vm && !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && | |
| 1660 | + dma_resv_trylock(resv)) { | |
| 1658 | 1661 | clear = false; |
| 1659 | 1662 | unlock = true; |
| 1660 | 1663 | /* The caller is already holding the reservation lock */ |
| @@ -2002,7 +2005,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, · 341 unmodified lines | ||
| 2002 | 2005 | * during user requests GEM unmap IOCTL except for forcing the unmap |
| 2003 | 2006 | * from user space. |
| 2004 | 2007 | */ |
| 2005 | − if (unlikely(atomic_read(&bo_va->userq_va_mapped) > 0)) | |
| 2008 | + if (unlikely(bo_va->userq_va_mapped)) | |
| 2006 | 2009 | amdgpu_userq_gem_va_unmap_validate(adev, mapping, saddr); |
| 2007 | 2010 | |
| 2008 | 2011 | list_del(&mapping->list); |
| @@ -3023,11 +3026,22 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · 1014 unmodified lines | ||
| 3023 | 3026 | |
| 3024 | 3027 | is_compute_context = vm->is_compute_context; |
| 3025 | 3028 | |
| 3026 | − if (is_compute_context && !svm_range_restore_pages(adev, pasid, vmid, | |
| 3027 | − node_id, addr >> PAGE_SHIFT, ts, write_fault)) { | |
| 3029 | + if (is_compute_context) { | |
| 3030 | + /* Unreserve root since svm_range_restore_pages might try to reserve it. */ | |
| 3031 | + /* TODO: rework svm_range_restore_pages so that this isn't necessary. */ | |
| 3028 | 3032 | amdgpu_bo_unreserve(root); |
| 3033 | + | |
| 3034 | + if (!svm_range_restore_pages(adev, pasid, vmid, | |
| 3035 | + node_id, addr >> PAGE_SHIFT, ts, write_fault)) { | |
| 3036 | + amdgpu_bo_unref(&root); | |
| 3037 | + return true; | |
| 3038 | + } | |
| 3029 | 3039 | amdgpu_bo_unref(&root); |
| 3030 | − return true; | |
| 3040 | + | |
| 3041 | + /* Re-acquire the VM lock, could be that the VM was freed in between. */ | |
| 3042 | + vm = amdgpu_vm_lock_by_pasid(adev, &root, pasid); | |
| 3043 | + if (!vm) | |
| 3044 | + return false; | |
| 3031 | 3045 | } |
| 3032 | 3046 | |
| 3033 | 3047 | addr /= AMDGPU_GPU_PAGE_SIZE; |
| @@ -562,6 +562,11 @@ static void vpe_ring_emit_fence(struct amdgpu_ring *ring, uint64_t addr, · 561 unmodified lines | ||
| 562 | 562 | amdgpu_ring_write(ring, 0); |
| 563 | 563 | } |
| 564 | 564 | |
| 565 | + /* WA: Force sync after TRAP to avoid VPE1 fail to power off */ | |
| 566 | + if (ring->adev->vpe.collaborate_mode) { | |
| 567 | + amdgpu_ring_write(ring, VPE_CMD_HEADER(VPE_CMD_OPCODE_COLLAB_SYNC, 0)); | |
| 568 | + amdgpu_ring_write(ring, 0xabcd); | |
| 569 | + } | |
| 565 | 570 | } |
| 566 | 571 | |
| 567 | 572 | static void vpe_ring_emit_pipeline_sync(struct amdgpu_ring *ring) |
| @@ -968,7 +973,7 @@ static const struct amdgpu_ring_funcs vpe_ring_funcs = { · 400 unmodified lines | ||
| 968 | 973 | .emit_frame_size = |
| 969 | 974 | 5 + /* vpe_ring_init_cond_exec */ |
| 970 | 975 | 6 + /* vpe_ring_emit_pipeline_sync */ |
| 971 | − 10 + 10 + 10 + /* vpe_ring_emit_fence */ | |
| 976 | + 12 + 12 + 12 + /* vpe_ring_emit_fence */ | |
| 972 | 977 | /* vpe_ring_emit_vm_flush */ |
| 973 | 978 | SOC15_FLUSH_GPU_TLB_NUM_WREG * 3 + |
| 974 | 979 | SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 6, |
| @@ -6523,15 +6523,7 @@ static int gfx_v11_0_eop_irq(struct amdgpu_device *adev, · 6522 unmodified lines | ||
| 6523 | 6523 | DRM_DEBUG("IH: CP EOP\n"); |
| 6524 | 6524 | |
| 6525 | 6525 | if (adev->enable_mes && doorbell_offset) { |
| 6526 | − struct amdgpu_usermode_queue *queue; | |
| 6527 | − struct xarray *xa = &adev->userq_doorbell_xa; | |
| 6528 | − unsigned long flags; | |
| 6529 | − | |
| 6530 | − xa_lock_irqsave(xa, flags); | |
| 6531 | − queue = xa_load(xa, doorbell_offset); | |
| 6532 | − if (queue) | |
| 6533 | − amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 6534 | − xa_unlock_irqrestore(xa, flags); | |
| 6526 | + amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 6535 | 6527 | } else { |
| 6536 | 6528 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 6537 | 6529 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -602,6 +602,13 @@ static int gfx_v12_0_init_microcode(struct amdgpu_device *adev) · 601 unmodified lines | ||
| 602 | 602 | "amdgpu/%s_pfp.bin", ucode_prefix); |
| 603 | 603 | if (err) |
| 604 | 604 | goto out; |
| 605 | + | |
| 606 | + adev->gfx.rs64_enable = amdgpu_ucode_hdr_version( | |
| 607 | + (union amdgpu_firmware_header *) | |
| 608 | + adev->gfx.pfp_fw->data, 2, 0); | |
| 609 | + if (adev->gfx.rs64_enable) | |
| 610 | + dev_dbg(adev->dev, "CP RS64 enable\n"); | |
| 611 | + | |
| 605 | 612 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP); |
| 606 | 613 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP_P0_STACK); |
| 607 | 614 | |
| @@ -4854,15 +4861,7 @@ static int gfx_v12_0_eop_irq(struct amdgpu_device *adev, · 4246 unmodified lines | ||
| 4854 | 4861 | DRM_DEBUG("IH: CP EOP\n"); |
| 4855 | 4862 | |
| 4856 | 4863 | if (adev->enable_mes && doorbell_offset) { |
| 4857 | − struct xarray *xa = &adev->userq_doorbell_xa; | |
| 4858 | − struct amdgpu_usermode_queue *queue; | |
| 4859 | − unsigned long flags; | |
| 4860 | − | |
| 4861 | − xa_lock_irqsave(xa, flags); | |
| 4862 | − queue = xa_load(xa, doorbell_offset); | |
| 4863 | − if (queue) | |
| 4864 | − amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 4865 | − xa_unlock_irqrestore(xa, flags); | |
| 4864 | + amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 4866 | 4865 | } else { |
| 4867 | 4866 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 4868 | 4867 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -3643,16 +3643,7 @@ static int gfx_v12_1_eop_irq(struct amdgpu_device *adev, · 3642 unmodified lines | ||
| 3643 | 3643 | DRM_DEBUG("IH: CP EOP\n"); |
| 3644 | 3644 | |
| 3645 | 3645 | if (adev->enable_mes && doorbell_offset) { |
| 3646 | − struct xarray *xa = &adev->userq_doorbell_xa; | |
| 3647 | − struct amdgpu_usermode_queue *queue; | |
| 3648 | − unsigned long flags; | |
| 3649 | − | |
| 3650 | − xa_lock_irqsave(xa, flags); | |
| 3651 | − queue = xa_load(xa, doorbell_offset); | |
| 3652 | − if (queue) | |
| 3653 | − amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 3654 | − | |
| 3655 | − xa_unlock_irqrestore(xa, flags); | |
| 3646 | + amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 3656 | 3647 | } else { |
| 3657 | 3648 | me_id = (entry->ring_id & 0x0c) >> 2; |
| 3658 | 3649 | pipe_id = (entry->ring_id & 0x03) >> 0; |
| @@ -1571,6 +1571,71 @@ static void gfx_v6_0_setup_spi(struct amdgpu_device *adev) · 1570 unmodified lines | ||
| 1571 | 1571 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1572 | 1572 | } |
| 1573 | 1573 | |
| 1574 | +/** | |
| 1575 | + * gfx_v6_0_setup_tcc() - setup which TCCs are used | |
| 1576 | + * | |
| 1577 | + * @adev: amdgpu_device pointer | |
| 1578 | + * | |
| 1579 | + * Verify whether the current GPU has any TCCs disabled, | |
| 1580 | + * which can happen when the GPU is harvested and some | |
| 1581 | + * memory channels are disabled, reducing the memory bus width. | |
| 1582 | + * For example, on the Radeon HD 7870 XT (Tahiti LE). | |
| 1583 | + * | |
| 1584 | + * If some TCCs are disabled, we need to make sure that | |
| 1585 | + * the disabled TCCs are not used, and the remaining TCCs | |
| 1586 | + * are used optimally. | |
| 1587 | + * | |
| 1588 | + * TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels. | |
| 1589 | + * TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used. | |
| 1590 | + * | |
| 1591 | + * For optimal performance: | |
| 1592 | + * - Rely on the CHAN_STEER from the golden registers table, | |
| 1593 | + * only skip disabled TCCs but keep the mapping order. | |
| 1594 | + * - Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing, | |
| 1595 | + * which performs better than using the same TCC twice. | |
| 1596 | + */ | |
| 1597 | +static void gfx_v6_0_setup_tcc(struct amdgpu_device *adev) | |
| 1598 | +{ | |
| 1599 | + u32 i, tcc, tcp_addr_config, num_active_tcc = 0; | |
| 1600 | + u64 chan_steer, patched_chan_steer = 0; | |
| 1601 | + const u32 num_max_tcc = adev->gfx.config.max_texture_channel_caches; | |
| 1602 | + const u32 dis_tcc_mask = | |
| 1603 | + amdgpu_gfx_create_bitmask(num_max_tcc) & | |
| 1604 | + (REG_GET_FIELD(RREG32(mmCGTS_TCC_DISABLE), | |
| 1605 | + CGTS_TCC_DISABLE, TCC_DISABLE) | | |
| 1606 | + REG_GET_FIELD(RREG32(mmCGTS_USER_TCC_DISABLE), | |
| 1607 | + CGTS_USER_TCC_DISABLE, TCC_DISABLE)); | |
| 1608 | + | |
| 1609 | + /* When no TCC is disabled, the golden registers table already has optimal TCC setup */ | |
| 1610 | + if (!dis_tcc_mask) | |
| 1611 | + return; | |
| 1612 | + | |
| 1613 | + /* Each 4-bit nibble contains the index of a TCC used by all TCPs */ | |
| 1614 | + chan_steer = RREG32(mmTCP_CHAN_STEER_LO) | ((u64)RREG32(mmTCP_CHAN_STEER_HI) << 32ull); | |
| 1615 | + | |
| 1616 | + /* Patch the TCP to TCC mapping to skip disabled TCCs */ | |
| 1617 | + for (i = 0; i < num_max_tcc; ++i) { | |
| 1618 | + tcc = (chan_steer >> (u64)(4 * i)) & 0xf; | |
| 1619 | + | |
| 1620 | + if (!((1 << tcc) & dis_tcc_mask)) { | |
| 1621 | + /* Copy enabled TCC indices to the patched register value. */ | |
| 1622 | + patched_chan_steer |= (u64)tcc << (u64)(4 * num_active_tcc); | |
| 1623 | + ++num_active_tcc; | |
| 1624 | + } | |
| 1625 | + } | |
| 1626 | + | |
| 1627 | + WARN_ON(num_active_tcc != num_max_tcc - hweight32(dis_tcc_mask)); | |
| 1628 | + | |
| 1629 | + /* Patch number of TCCs used by TCPs */ | |
| 1630 | + tcp_addr_config = REG_SET_FIELD(RREG32(mmTCP_ADDR_CONFIG), | |
| 1631 | + TCP_ADDR_CONFIG, NUM_TCC_BANKS, | |
| 1632 | + num_active_tcc - 1); | |
| 1633 | + | |
| 1634 | + WREG32(mmTCP_ADDR_CONFIG, tcp_addr_config); | |
| 1635 | + WREG32(mmTCP_CHAN_STEER_HI, upper_32_bits(patched_chan_steer)); | |
| 1636 | + WREG32(mmTCP_CHAN_STEER_LO, lower_32_bits(patched_chan_steer)); | |
| 1637 | +} | |
| 1638 | + | |
| 1574 | 1639 | static void gfx_v6_0_config_init(struct amdgpu_device *adev) |
| 1575 | 1640 | { |
| 1576 | 1641 | adev->gfx.config.double_offchip_lds_buf = 0; |
| @@ -1729,6 +1794,7 @@ static void gfx_v6_0_constants_init(struct amdgpu_device *adev) · 152 unmodified lines | ||
| 1729 | 1794 | gfx_v6_0_tiling_mode_table_init(adev); |
| 1730 | 1795 | |
| 1731 | 1796 | gfx_v6_0_setup_rb(adev); |
| 1797 | + gfx_v6_0_setup_tcc(adev); | |
| 1732 | 1798 | |
| 1733 | 1799 | gfx_v6_0_setup_spi(adev); |
| 1734 | 1800 | |
| @@ -5660,9 +5660,6 @@ static void gfx_v9_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, · 5659 unmodified lines | ||
| 5660 | 5660 | { |
| 5661 | 5661 | struct amdgpu_device *adev = ring->adev; |
| 5662 | 5662 | |
| 5663 | − /* we only allocate 32bit for each seq wb address */ | |
| 5664 | − BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); | |
| 5665 | − | |
| 5666 | 5663 | /* write fence seq to the "addr" */ |
| 5667 | 5664 | amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); |
| 5668 | 5665 | amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | |
| @@ -802,6 +802,7 @@ static const struct amd_ip_funcs jpeg_v2_0_ip_funcs = { · 801 unmodified lines | ||
| 802 | 802 | static const struct amdgpu_ring_funcs jpeg_v2_0_dec_ring_vm_funcs = { |
| 803 | 803 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 804 | 804 | .align_mask = 0xf, |
| 805 | + .no_user_fence = true, | |
| 805 | 806 | .get_rptr = jpeg_v2_0_dec_ring_get_rptr, |
| 806 | 807 | .get_wptr = jpeg_v2_0_dec_ring_get_wptr, |
| 807 | 808 | .set_wptr = jpeg_v2_0_dec_ring_set_wptr, |
| @@ -693,6 +693,7 @@ static const struct amd_ip_funcs jpeg_v2_6_ip_funcs = { · 692 unmodified lines | ||
| 693 | 693 | static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = { |
| 694 | 694 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 695 | 695 | .align_mask = 0xf, |
| 696 | + .no_user_fence = true, | |
| 696 | 697 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, |
| 697 | 698 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, |
| 698 | 699 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, |
| @@ -724,6 +725,7 @@ static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = { · 25 unmodified lines | ||
| 724 | 725 | static const struct amdgpu_ring_funcs jpeg_v2_6_dec_ring_vm_funcs = { |
| 725 | 726 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 726 | 727 | .align_mask = 0xf, |
| 728 | + .no_user_fence = true, | |
| 727 | 729 | .get_rptr = jpeg_v2_5_dec_ring_get_rptr, |
| 728 | 730 | .get_wptr = jpeg_v2_5_dec_ring_get_wptr, |
| 729 | 731 | .set_wptr = jpeg_v2_5_dec_ring_set_wptr, |
| @@ -594,6 +594,7 @@ static const struct amd_ip_funcs jpeg_v3_0_ip_funcs = { · 593 unmodified lines | ||
| 594 | 594 | static const struct amdgpu_ring_funcs jpeg_v3_0_dec_ring_vm_funcs = { |
| 595 | 595 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 596 | 596 | .align_mask = 0xf, |
| 597 | + .no_user_fence = true, | |
| 597 | 598 | .get_rptr = jpeg_v3_0_dec_ring_get_rptr, |
| 598 | 599 | .get_wptr = jpeg_v3_0_dec_ring_get_wptr, |
| 599 | 600 | .set_wptr = jpeg_v3_0_dec_ring_set_wptr, |
| @@ -759,6 +759,7 @@ static const struct amd_ip_funcs jpeg_v4_0_ip_funcs = { · 758 unmodified lines | ||
| 759 | 759 | static const struct amdgpu_ring_funcs jpeg_v4_0_dec_ring_vm_funcs = { |
| 760 | 760 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 761 | 761 | .align_mask = 0xf, |
| 762 | + .no_user_fence = true, | |
| 762 | 763 | .get_rptr = jpeg_v4_0_dec_ring_get_rptr, |
| 763 | 764 | .get_wptr = jpeg_v4_0_dec_ring_get_wptr, |
| 764 | 765 | .set_wptr = jpeg_v4_0_dec_ring_set_wptr, |
| @@ -1219,6 +1219,7 @@ static const struct amd_ip_funcs jpeg_v4_0_3_ip_funcs = { · 1218 unmodified lines | ||
| 1219 | 1219 | static const struct amdgpu_ring_funcs jpeg_v4_0_3_dec_ring_vm_funcs = { |
| 1220 | 1220 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 1221 | 1221 | .align_mask = 0xf, |
| 1222 | + .no_user_fence = true, | |
| 1222 | 1223 | .get_rptr = jpeg_v4_0_3_dec_ring_get_rptr, |
| 1223 | 1224 | .get_wptr = jpeg_v4_0_3_dec_ring_get_wptr, |
| 1224 | 1225 | .set_wptr = jpeg_v4_0_3_dec_ring_set_wptr, |
| @@ -804,6 +804,7 @@ static const struct amd_ip_funcs jpeg_v4_0_5_ip_funcs = { · 803 unmodified lines | ||
| 804 | 804 | static const struct amdgpu_ring_funcs jpeg_v4_0_5_dec_ring_vm_funcs = { |
| 805 | 805 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 806 | 806 | .align_mask = 0xf, |
| 807 | + .no_user_fence = true, | |
| 807 | 808 | .get_rptr = jpeg_v4_0_5_dec_ring_get_rptr, |
| 808 | 809 | .get_wptr = jpeg_v4_0_5_dec_ring_get_wptr, |
| 809 | 810 | .set_wptr = jpeg_v4_0_5_dec_ring_set_wptr, |
| @@ -680,6 +680,7 @@ static const struct amd_ip_funcs jpeg_v5_0_0_ip_funcs = { · 679 unmodified lines | ||
| 680 | 680 | static const struct amdgpu_ring_funcs jpeg_v5_0_0_dec_ring_vm_funcs = { |
| 681 | 681 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 682 | 682 | .align_mask = 0xf, |
| 683 | + .no_user_fence = true, | |
| 683 | 684 | .get_rptr = jpeg_v5_0_0_dec_ring_get_rptr, |
| 684 | 685 | .get_wptr = jpeg_v5_0_0_dec_ring_get_wptr, |
| 685 | 686 | .set_wptr = jpeg_v5_0_0_dec_ring_set_wptr, |
| @@ -884,6 +884,7 @@ static const struct amd_ip_funcs jpeg_v5_0_1_ip_funcs = { · 883 unmodified lines | ||
| 884 | 884 | static const struct amdgpu_ring_funcs jpeg_v5_0_1_dec_ring_vm_funcs = { |
| 885 | 885 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 886 | 886 | .align_mask = 0xf, |
| 887 | + .no_user_fence = true, | |
| 887 | 888 | .get_rptr = jpeg_v5_0_1_dec_ring_get_rptr, |
| 888 | 889 | .get_wptr = jpeg_v5_0_1_dec_ring_get_wptr, |
| 889 | 890 | .set_wptr = jpeg_v5_0_1_dec_ring_set_wptr, |
| @@ -703,6 +703,7 @@ static const struct amd_ip_funcs jpeg_v5_0_2_ip_funcs = { · 702 unmodified lines | ||
| 703 | 703 | static const struct amdgpu_ring_funcs jpeg_v5_0_2_dec_ring_vm_funcs = { |
| 704 | 704 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 705 | 705 | .align_mask = 0xf, |
| 706 | + .no_user_fence = true, | |
| 706 | 707 | .get_rptr = jpeg_v5_0_2_dec_ring_get_rptr, |
| 707 | 708 | .get_wptr = jpeg_v5_0_2_dec_ring_get_wptr, |
| 708 | 709 | .set_wptr = jpeg_v5_0_2_dec_ring_set_wptr, |
| @@ -661,6 +661,7 @@ static const struct amd_ip_funcs jpeg_v5_3_0_ip_funcs = { · 660 unmodified lines | ||
| 661 | 661 | static const struct amdgpu_ring_funcs jpeg_v5_3_0_dec_ring_vm_funcs = { |
| 662 | 662 | .type = AMDGPU_RING_TYPE_VCN_JPEG, |
| 663 | 663 | .align_mask = 0xf, |
| 664 | + .no_user_fence = true, | |
| 664 | 665 | .get_rptr = jpeg_v5_3_0_dec_ring_get_rptr, |
| 665 | 666 | .get_wptr = jpeg_v5_3_0_dec_ring_get_wptr, |
| 666 | 667 | .set_wptr = jpeg_v5_3_0_dec_ring_set_wptr, |
| @@ -30,34 +30,6 @@ · 29 unmodified lines | ||
| 30 | 30 | #define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE |
| 31 | 31 | #define AMDGPU_USERQ_GANG_CTX_SZ PAGE_SIZE |
| 32 | 32 | |
| 33 | −static int | |
| 34 | −mes_userq_map_gtt_bo_to_gart(struct amdgpu_bo *bo) | |
| 35 | −{ | |
| 36 | − int ret; | |
| 37 | − | |
| 38 | − ret = amdgpu_bo_reserve(bo, true); | |
| 39 | − if (ret) { | |
| 40 | − DRM_ERROR("Failed to reserve bo. ret %d\n", ret); | |
| 41 | − goto err_reserve_bo_failed; | |
| 42 | − } | |
| 43 | − | |
| 44 | − ret = amdgpu_ttm_alloc_gart(&bo->tbo); | |
| 45 | − if (ret) { | |
| 46 | − DRM_ERROR("Failed to bind bo to GART. ret %d\n", ret); | |
| 47 | − goto err_map_bo_gart_failed; | |
| 48 | − } | |
| 49 | − | |
| 50 | − amdgpu_bo_unreserve(bo); | |
| 51 | − bo = amdgpu_bo_ref(bo); | |
| 52 | − | |
| 53 | − return 0; | |
| 54 | − | |
| 55 | −err_map_bo_gart_failed: | |
| 56 | − amdgpu_bo_unreserve(bo); | |
| 57 | −err_reserve_bo_failed: | |
| 58 | − return ret; | |
| 59 | −} | |
| 60 | − | |
| 61 | 33 | static int |
| 62 | 34 | mes_userq_create_wptr_mapping(struct amdgpu_device *adev, |
| 63 | 35 | struct amdgpu_userq_mgr *uq_mgr, |
| @@ -65,55 +37,64 @@ mes_userq_create_wptr_mapping(struct amdgpu_device *adev, · 1 unmodified line | ||
| 65 | 37 | uint64_t wptr) |
| 66 | 38 | { |
| 67 | 39 | struct amdgpu_bo_va_mapping *wptr_mapping; |
| 68 | − struct amdgpu_vm *wptr_vm; | |
| 69 | 40 | struct amdgpu_userq_obj *wptr_obj = &queue->wptr_obj; |
| 41 | + struct amdgpu_bo *obj; | |
| 42 | + struct amdgpu_vm *vm = queue->vm; | |
| 43 | + struct drm_exec exec; | |
| 70 | 44 | int ret; |
| 71 | 45 | |
| 72 | − wptr_vm = queue->vm; | |
| 73 | − ret = amdgpu_bo_reserve(wptr_vm->root.bo, false); | |
| 74 | − if (ret) | |
| 75 | − return ret; | |
| 76 | − | |
| 77 | 46 | wptr &= AMDGPU_GMC_HOLE_MASK; |
| 78 | − wptr_mapping = amdgpu_vm_bo_lookup_mapping(wptr_vm, wptr >> PAGE_SHIFT); | |
| 79 | − amdgpu_bo_unreserve(wptr_vm->root.bo); | |
| 80 | − if (!wptr_mapping) { | |
| 81 | − DRM_ERROR("Failed to lookup wptr bo\n"); | |
| 82 | − return -EINVAL; | |
| 83 | − } | |
| 84 | 47 | |
| 85 | − wptr_obj->obj = wptr_mapping->bo_va->base.bo; | |
| 86 | − if (wptr_obj->obj->tbo.base.size > PAGE_SIZE) { | |
| 87 | − DRM_ERROR("Requested GART mapping for wptr bo larger than one page\n"); | |
| 88 | − return -EINVAL; | |
| 89 | − } | |
| 48 | + drm_exec_init(&exec, DRM_EXEC_IGNORE_DUPLICATES, 2); | |
| 49 | + drm_exec_until_all_locked(&exec) { | |
| 50 | + ret = amdgpu_vm_lock_pd(vm, &exec, 1); | |
| 51 | + drm_exec_retry_on_contention(&exec); | |
| 52 | + if (unlikely(ret)) | |
| 53 | + goto fail_lock; | |
| 54 | + | |
| 55 | + wptr_mapping = amdgpu_vm_bo_lookup_mapping(vm, wptr >> PAGE_SHIFT); | |
| 56 | + if (!wptr_mapping) { | |
| 57 | + ret = -EINVAL; | |
| 58 | + goto fail_lock; | |
| 59 | + } | |
| 90 | 60 | |
| 91 | − ret = mes_userq_map_gtt_bo_to_gart(wptr_obj->obj); | |
| 92 | − if (ret) { | |
| 93 | − DRM_ERROR("Failed to map wptr bo to GART\n"); | |
| 94 | − return ret; | |
| 61 | + obj = wptr_mapping->bo_va->base.bo; | |
| 62 | + ret = drm_exec_lock_obj(&exec, &obj->tbo.base); | |
| 63 | + drm_exec_retry_on_contention(&exec); | |
| 64 | + if (unlikely(ret)) | |
| 65 | + goto fail_lock; | |
| 95 | 66 | } |
| 96 | 67 | |
| 97 | − ret = amdgpu_bo_reserve(wptr_obj->obj, true); | |
| 98 | − if (ret) { | |
| 99 | − DRM_ERROR("Failed to reserve wptr bo\n"); | |
| 100 | − return ret; | |
| 68 | + wptr_obj->obj = amdgpu_bo_ref(wptr_mapping->bo_va->base.bo); | |
| 69 | + if (wptr_obj->obj->tbo.base.size > PAGE_SIZE) { | |
| 70 | + ret = -EINVAL; | |
| 71 | + goto fail_map; | |
| 101 | 72 | } |
| 102 | 73 | |
| 103 | 74 | /* TODO use eviction fence instead of pinning. */ |
| 104 | 75 | ret = amdgpu_bo_pin(wptr_obj->obj, AMDGPU_GEM_DOMAIN_GTT); |
| 105 | 76 | if (ret) { |
| 106 | − drm_file_err(uq_mgr->file, "[Usermode queues] Failed to pin wptr bo\n"); | |
| 107 | − goto unresv_bo; | |
| 77 | + DRM_ERROR("Failed to pin wptr bo. ret %d\n", ret); | |
| 78 | + goto fail_map; | |
| 79 | + } | |
| 80 | + | |
| 81 | + ret = amdgpu_ttm_alloc_gart(&wptr_obj->obj->tbo); | |
| 82 | + if (ret) { | |
| 83 | + DRM_ERROR("Failed to bind bo to GART. ret %d\n", ret); | |
| 84 | + goto fail_alloc_gart; | |
| 108 | 85 | } |
| 109 | 86 | |
| 110 | 87 | queue->wptr_obj.gpu_addr = amdgpu_bo_gpu_offset(wptr_obj->obj); |
| 111 | − amdgpu_bo_unreserve(wptr_obj->obj); | |
| 112 | 88 | |
| 89 | + drm_exec_fini(&exec); | |
| 113 | 90 | return 0; |
| 114 | 91 | |
| 115 | −unresv_bo: | |
| 116 | − amdgpu_bo_unreserve(wptr_obj->obj); | |
| 92 | +fail_alloc_gart: | |
| 93 | + amdgpu_bo_unpin(wptr_obj->obj); | |
| 94 | +fail_map: | |
| 95 | + amdgpu_bo_unref(&wptr_obj->obj); | |
| 96 | +fail_lock: | |
| 97 | + drm_exec_fini(&exec); | |
| 117 | 98 | return ret; |
| 118 | 99 | |
| 119 | 100 | } |
| @@ -211,12 +192,16 @@ static int mes_userq_create_ctx_space(struct amdgpu_userq_mgr *uq_mgr, · 91 unmodified lines | ||
| 211 | 192 | * for the same. |
| 212 | 193 | */ |
| 213 | 194 | size = AMDGPU_USERQ_PROC_CTX_SZ + AMDGPU_USERQ_GANG_CTX_SZ; |
| 214 | − r = amdgpu_userq_create_object(uq_mgr, ctx, size); | |
| 195 | + r = amdgpu_bo_create_kernel(uq_mgr->adev, size, 0, | |
| 196 | + AMDGPU_GEM_DOMAIN_GTT, | |
| 197 | + &ctx->obj, &ctx->gpu_addr, | |
| 198 | + &ctx->cpu_ptr); | |
| 215 | 199 | if (r) { |
| 216 | 200 | DRM_ERROR("Failed to allocate ctx space bo for userqueue, err:%d\n", r); |
| 217 | 201 | return r; |
| 218 | 202 | } |
| 219 | 203 | |
| 204 | + memset(ctx->cpu_ptr, 0, size); | |
| 220 | 205 | return 0; |
| 221 | 206 | } |
| 222 | 207 | |
| @@ -289,13 +274,19 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 66 unmodified lines | ||
| 289 | 274 | return -ENOMEM; |
| 290 | 275 | } |
| 291 | 276 | |
| 292 | − r = amdgpu_userq_create_object(uq_mgr, &queue->mqd, | |
| 293 | − AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size)); | |
| 277 | + r = amdgpu_bo_create_kernel(adev, | |
| 278 | + AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size), | |
| 279 | + 0, AMDGPU_GEM_DOMAIN_GTT, | |
| 280 | + &queue->mqd.obj, &queue->mqd.gpu_addr, | |
| 281 | + &queue->mqd.cpu_ptr); | |
| 294 | 282 | if (r) { |
| 295 | 283 | DRM_ERROR("Failed to create MQD object for userqueue\n"); |
| 296 | 284 | goto free_props; |
| 297 | 285 | } |
| 298 | 286 | |
| 287 | + memset(queue->mqd.cpu_ptr, 0, | |
| 288 | + AMDGPU_MQD_SIZE_ALIGN(mqd_hw_default->mqd_size)); | |
| 289 | + | |
| 299 | 290 | /* Initialize the MQD BO with user given values */ |
| 300 | 291 | userq_props->wptr_gpu_addr = mqd_user->wptr_va; |
| 301 | 292 | userq_props->rptr_gpu_addr = mqd_user->rptr_va; |
| @@ -327,8 +318,9 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 25 unmodified lines | ||
| 327 | 318 | kfree(compute_mqd); |
| 328 | 319 | goto free_mqd; |
| 329 | 320 | } |
| 330 | − r = amdgpu_userq_input_va_validate(adev, queue, compute_mqd->eop_va, | |
| 331 | − 2048); | |
| 321 | + r = amdgpu_userq_input_va_validate(adev, queue, | |
| 322 | + compute_mqd->eop_va, 2048, | |
| 323 | + &queue->userq_vas.va.eop); | |
| 332 | 324 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 333 | 325 | if (r) { |
| 334 | 326 | kfree(compute_mqd); |
| @@ -377,7 +369,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 42 unmodified lines | ||
| 377 | 369 | goto free_mqd; |
| 378 | 370 | } |
| 379 | 371 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->shadow_va, |
| 380 | − shadow_info.shadow_size); | |
| 372 | + shadow_info.shadow_size, | |
| 373 | + &queue->userq_vas.va.shadow); | |
| 381 | 374 | if (r) { |
| 382 | 375 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 383 | 376 | kfree(mqd_gfx_v11); |
| @@ -385,7 +378,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 1 unmodified line | ||
| 385 | 378 | } |
| 386 | 379 | |
| 387 | 380 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_gfx_v11->csa_va, |
| 388 | − shadow_info.csa_size); | |
| 381 | + shadow_info.csa_size, | |
| 382 | + &queue->userq_vas.va.csa); | |
| 389 | 383 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 390 | 384 | if (r) { |
| 391 | 385 | kfree(mqd_gfx_v11); |
| @@ -415,7 +409,8 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 23 unmodified lines | ||
| 415 | 409 | goto free_mqd; |
| 416 | 410 | } |
| 417 | 411 | r = amdgpu_userq_input_va_validate(adev, queue, mqd_sdma_v11->csa_va, |
| 418 | − 32); | |
| 412 | + 32, | |
| 413 | + &queue->userq_vas.va.csa); | |
| 419 | 414 | amdgpu_bo_unreserve(queue->vm->root.bo); |
| 420 | 415 | if (r) { |
| 421 | 416 | kfree(mqd_sdma_v11); |
| @@ -451,10 +446,12 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, · 29 unmodified lines | ||
| 451 | 446 | return 0; |
| 452 | 447 | |
| 453 | 448 | free_ctx: |
| 454 | − amdgpu_userq_destroy_object(uq_mgr, &queue->fw_obj); | |
| 449 | + amdgpu_bo_free_kernel(&queue->fw_obj.obj, &queue->fw_obj.gpu_addr, | |
| 450 | + &queue->fw_obj.cpu_ptr); | |
| 455 | 451 | |
| 456 | 452 | free_mqd: |
| 457 | − amdgpu_userq_destroy_object(uq_mgr, &queue->mqd); | |
| 453 | + amdgpu_bo_free_kernel(&queue->mqd.obj, &queue->mqd.gpu_addr, | |
| 454 | + &queue->mqd.cpu_ptr); | |
| 458 | 455 | |
| 459 | 456 | free_props: |
| 460 | 457 | kfree(userq_props); |
| @@ -464,11 +461,12 @@ free_props: · 3 unmodified lines | ||
| 464 | 461 | |
| 465 | 462 | static void mes_userq_mqd_destroy(struct amdgpu_usermode_queue *queue) |
| 466 | 463 | { |
| 467 | − struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | |
| 468 | 464 | |
| 469 | − amdgpu_userq_destroy_object(uq_mgr, &queue->fw_obj); | |
| 465 | + amdgpu_bo_free_kernel(&queue->fw_obj.obj, &queue->fw_obj.gpu_addr, | |
| 466 | + &queue->fw_obj.cpu_ptr); | |
| 470 | 467 | kfree(queue->userq_prop); |
| 471 | − amdgpu_userq_destroy_object(uq_mgr, &queue->mqd); | |
| 468 | + amdgpu_bo_free_kernel(&queue->mqd.obj, &queue->mqd.gpu_addr, | |
| 469 | + &queue->mqd.cpu_ptr); | |
| 472 | 470 | } |
| 473 | 471 | |
| 474 | 472 | static int mes_userq_preempt(struct amdgpu_usermode_queue *queue) |
| @@ -889,7 +889,7 @@ static void sdma_v4_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se · 888 unmodified lines | ||
| 889 | 889 | /* write the fence */ |
| 890 | 890 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); |
| 891 | 891 | /* zero in first two bits */ |
| 892 | − BUG_ON(addr & 0x3); | |
| 892 | + WARN_ON(addr & 0x3); | |
| 893 | 893 | amdgpu_ring_write(ring, lower_32_bits(addr)); |
| 894 | 894 | amdgpu_ring_write(ring, upper_32_bits(addr)); |
| 895 | 895 | amdgpu_ring_write(ring, lower_32_bits(seq)); |
| @@ -899,7 +899,7 @@ static void sdma_v4_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se · 3 unmodified lines | ||
| 899 | 899 | addr += 4; |
| 900 | 900 | amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); |
| 901 | 901 | /* zero in first two bits */ |
| 902 | − BUG_ON(addr & 0x3); | |
| 902 | + WARN_ON(addr & 0x3); | |
| 903 | 903 | amdgpu_ring_write(ring, lower_32_bits(addr)); |
| 904 | 904 | amdgpu_ring_write(ring, upper_32_bits(addr)); |
| 905 | 905 | amdgpu_ring_write(ring, upper_32_bits(seq)); |
| @@ -1662,17 +1662,8 @@ static int sdma_v6_0_process_fence_irq(struct amdgpu_device *adev, · 1661 unmodified lines | ||
| 1662 | 1662 | u32 doorbell_offset = entry->src_data[0]; |
| 1663 | 1663 | |
| 1664 | 1664 | if (adev->enable_mes && doorbell_offset) { |
| 1665 | − struct amdgpu_usermode_queue *queue; | |
| 1666 | − struct xarray *xa = &adev->userq_doorbell_xa; | |
| 1667 | − unsigned long flags; | |
| 1668 | − | |
| 1669 | 1665 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; |
| 1670 | − | |
| 1671 | − xa_lock_irqsave(xa, flags); | |
| 1672 | − queue = xa_load(xa, doorbell_offset); | |
| 1673 | − if (queue) | |
| 1674 | − amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 1675 | − xa_unlock_irqrestore(xa, flags); | |
| 1666 | + amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 1676 | 1667 | } |
| 1677 | 1668 | |
| 1678 | 1669 | return 0; |
| @@ -1594,17 +1594,8 @@ static int sdma_v7_0_process_fence_irq(struct amdgpu_device *adev, · 1593 unmodified lines | ||
| 1594 | 1594 | u32 doorbell_offset = entry->src_data[0]; |
| 1595 | 1595 | |
| 1596 | 1596 | if (adev->enable_mes && doorbell_offset) { |
| 1597 | − struct xarray *xa = &adev->userq_doorbell_xa; | |
| 1598 | − struct amdgpu_usermode_queue *queue; | |
| 1599 | − unsigned long flags; | |
| 1600 | − | |
| 1601 | 1597 | doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT; |
| 1602 | − | |
| 1603 | − xa_lock_irqsave(xa, flags); | |
| 1604 | − queue = xa_load(xa, doorbell_offset); | |
| 1605 | − if (queue) | |
| 1606 | − amdgpu_userq_fence_driver_process(queue->fence_drv); | |
| 1607 | − xa_unlock_irqrestore(xa, flags); | |
| 1598 | + amdgpu_userq_process_fence_irq(adev, doorbell_offset); | |
| 1608 | 1599 | } |
| 1609 | 1600 | |
| 1610 | 1601 | return 0; |
| @@ -242,6 +242,10 @@ static void uvd_v3_1_mc_resume(struct amdgpu_device *adev) · 241 unmodified lines | ||
| 242 | 242 | uint64_t addr; |
| 243 | 243 | uint32_t size; |
| 244 | 244 | |
| 245 | + /* When the keyselect is already set, don't perturb it. */ | |
| 246 | + if (RREG32(mmUVD_FW_START)) | |
| 247 | + return; | |
| 248 | + | |
| 245 | 249 | /* program the VCPU memory controller bits 0-27 */ |
| 246 | 250 | addr = (adev->uvd.inst->gpu_addr + AMDGPU_UVD_FIRMWARE_OFFSET) >> 3; |
| 247 | 251 | size = AMDGPU_UVD_FIRMWARE_SIZE(adev) >> 3; |
| @@ -284,6 +288,12 @@ static int uvd_v3_1_fw_validate(struct amdgpu_device *adev) · 36 unmodified lines | ||
| 284 | 288 | int i; |
| 285 | 289 | uint32_t keysel = adev->uvd.keyselect; |
| 286 | 290 | |
| 291 | + if (RREG32(mmUVD_FW_START) & UVD_FW_STATUS__PASS_MASK) { | |
| 292 | + dev_dbg(adev->dev, "UVD keyselect already set: 0x%x (on CPU: 0x%x)\n", | |
| 293 | + RREG32(mmUVD_FW_START), adev->uvd.keyselect); | |
| 294 | + return 0; | |
| 295 | + } | |
| 296 | + | |
| 287 | 297 | WREG32(mmUVD_FW_START, keysel); |
| 288 | 298 | |
| 289 | 299 | for (i = 0; i < 10; ++i) { |
| @@ -42,9 +42,10 @@ · 41 unmodified lines | ||
| 42 | 42 | #include "oss/oss_1_0_d.h" |
| 43 | 43 | #include "oss/oss_1_0_sh_mask.h" |
| 44 | 44 | |
| 45 | +#define VCE_V1_0_ALIGNMENT (32 * 1024) | |
| 45 | 46 | #define VCE_V1_0_FW_SIZE (256 * 1024) |
| 46 | 47 | #define VCE_V1_0_STACK_SIZE (64 * 1024) |
| 47 | −#define VCE_V1_0_DATA_SIZE (7808 * (AMDGPU_MAX_VCE_HANDLES + 1)) | |
| 48 | +#define VCE_V1_0_DATA_SIZE (ALIGN(7808 * (AMDGPU_MAX_VCE_HANDLES + 1), VCE_V1_0_ALIGNMENT)) | |
| 48 | 49 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 |
| 49 | 50 | |
| 50 | 51 | static void vce_v1_0_set_ring_funcs(struct amdgpu_device *adev); |
| @@ -177,7 +178,7 @@ static void vce_v1_0_init_cg(struct amdgpu_device *adev) · 126 unmodified lines | ||
| 177 | 178 | } |
| 178 | 179 | |
| 179 | 180 | /** |
| 180 | − * vce_v1_0_load_fw_signature - load firmware signature into VCPU BO | |
| 181 | + * vce_v1_0_load_fw() - load firmware signature into VCPU BO | |
| 181 | 182 | * |
| 182 | 183 | * @adev: amdgpu_device pointer |
| 183 | 184 | * |
| @@ -185,21 +186,26 @@ static void vce_v1_0_init_cg(struct amdgpu_device *adev) · 1 unmodified line | ||
| 185 | 186 | * This function finds the signature appropriate for the current |
| 186 | 187 | * ASIC and writes that into the VCPU BO. |
| 187 | 188 | */ |
| 188 | −static int vce_v1_0_load_fw_signature(struct amdgpu_device *adev) | |
| 189 | +static int vce_v1_0_load_fw(struct amdgpu_device *adev) | |
| 189 | 190 | { |
| 190 | 191 | const struct common_firmware_header *hdr; |
| 191 | 192 | struct vce_v1_0_fw_signature *sign; |
| 192 | − unsigned int ucode_offset; | |
| 193 | + u32 ucode_offset; | |
| 194 | + u32 ucode_size; | |
| 193 | 195 | uint32_t chip_id; |
| 194 | 196 | u32 *cpu_addr; |
| 195 | 197 | int i; |
| 196 | 198 | |
| 197 | 199 | hdr = (const struct common_firmware_header *)adev->vce.fw->data; |
| 198 | 200 | ucode_offset = le32_to_cpu(hdr->ucode_array_offset_bytes); |
| 201 | + ucode_size = hdr->ucode_size_bytes - sizeof(struct vce_v1_0_fw_signature *); | |
| 199 | 202 | cpu_addr = adev->vce.cpu_addr; |
| 200 | 203 | |
| 201 | 204 | sign = (void *)adev->vce.fw->data + ucode_offset; |
| 202 | 205 | |
| 206 | + if (ucode_size > VCE_V1_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET) | |
| 207 | + return -EINVAL; | |
| 208 | + | |
| 203 | 209 | switch (adev->asic_type) { |
| 204 | 210 | case CHIP_TAHITI: |
| 205 | 211 | chip_id = 0x01000014; |
| @@ -226,12 +232,14 @@ static int vce_v1_0_load_fw_signature(struct amdgpu_device *adev) · 20 unmodified lines | ||
| 226 | 232 | return -EINVAL; |
| 227 | 233 | } |
| 228 | 234 | |
| 235 | + memset_io(&cpu_addr[0], 0, amdgpu_bo_size(adev->vce.vcpu_bo)); | |
| 236 | + | |
| 229 | 237 | cpu_addr += (256 - 64) / 4; |
| 230 | 238 | memcpy_toio(&cpu_addr[0], &sign->val[i].nonce[0], 16); |
| 231 | 239 | cpu_addr[4] = cpu_to_le32(le32_to_cpu(sign->length) + 64); |
| 232 | 240 | |
| 233 | 241 | memset_io(&cpu_addr[5], 0, 44); |
| 234 | − memcpy_toio(&cpu_addr[16], &sign[1], hdr->ucode_size_bytes - sizeof(*sign)); | |
| 242 | + memcpy_toio(&cpu_addr[16], &sign[1], ucode_size); | |
| 235 | 243 | |
| 236 | 244 | cpu_addr += (le32_to_cpu(sign->length) + 64) / 4; |
| 237 | 245 | memcpy_toio(&cpu_addr[0], &sign->val[i].sigval[0], 16); |
| @@ -312,18 +320,23 @@ static int vce_v1_0_mc_resume(struct amdgpu_device *adev) · 74 unmodified lines | ||
| 312 | 320 | WREG32(mmVCE_VCPU_SCRATCH7, AMDGPU_MAX_VCE_HANDLES); |
| 313 | 321 | |
| 314 | 322 | offset = adev->vce.gpu_addr + AMDGPU_VCE_FIRMWARE_OFFSET; |
| 315 | − size = VCE_V1_0_FW_SIZE; | |
| 316 | − WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); | |
| 323 | + size = VCE_V1_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; | |
| 324 | + WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset); | |
| 317 | 325 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 318 | 326 | |
| 319 | 327 | offset += size; |
| 320 | 328 | size = VCE_V1_0_STACK_SIZE; |
| 321 | − WREG32(mmVCE_VCPU_CACHE_OFFSET1, offset & 0x7fffffff); | |
| 329 | + WARN_ON(!IS_ALIGNED(offset, VCE_V1_0_ALIGNMENT)); | |
| 330 | + WARN_ON(!IS_ALIGNED(size, VCE_V1_0_ALIGNMENT)); | |
| 331 | + WREG32(mmVCE_VCPU_CACHE_OFFSET1, offset); | |
| 322 | 332 | WREG32(mmVCE_VCPU_CACHE_SIZE1, size); |
| 323 | 333 | |
| 324 | 334 | offset += size; |
| 325 | 335 | size = VCE_V1_0_DATA_SIZE; |
| 326 | − WREG32(mmVCE_VCPU_CACHE_OFFSET2, offset & 0x7fffffff); | |
| 336 | + WARN_ON(!IS_ALIGNED(offset, VCE_V1_0_ALIGNMENT)); | |
| 337 | + WARN_ON(!IS_ALIGNED(size, VCE_V1_0_ALIGNMENT)); | |
| 338 | + WARN_ON((offset + size - adev->vce.gpu_addr) > amdgpu_bo_size(adev->vce.vcpu_bo)); | |
| 339 | + WREG32(mmVCE_VCPU_CACHE_OFFSET2, offset); | |
| 327 | 340 | WREG32(mmVCE_VCPU_CACHE_SIZE2, size); |
| 328 | 341 | |
| 329 | 342 | WREG32_P(mmVCE_LMI_CTRL2, 0x0, ~0x100); |
| @@ -527,22 +540,31 @@ static int vce_v1_0_early_init(struct amdgpu_ip_block *ip_block) · 197 unmodified lines | ||
| 527 | 540 | * To accomodate that, we put GART to the LOW address range |
| 528 | 541 | * and reserve some GART pages where we map the VCPU BO, |
| 529 | 542 | * so that it gets a 32-bit address. |
| 543 | + * | |
| 544 | + * The BAR address is zero and we can't change it | |
| 545 | + * due to the firmware validation mechanism. | |
| 546 | + * It seems that it fails to initialize if the address is >= 128 MiB. | |
| 530 | 547 | */ |
| 531 | 548 | static int vce_v1_0_ensure_vcpu_bo_32bit_addr(struct amdgpu_device *adev) |
| 532 | 549 | { |
| 533 | 550 | u64 bo_size = amdgpu_bo_size(adev->vce.vcpu_bo); |
| 534 | − u64 max_vcpu_bo_addr = 0xffffffff - bo_size; | |
| 551 | + u64 max_vcpu_bo_addr = 0x07ffffff - bo_size; | |
| 535 | 552 | u64 num_pages = ALIGN(bo_size, AMDGPU_GPU_PAGE_SIZE) / AMDGPU_GPU_PAGE_SIZE; |
| 536 | 553 | u64 pa = amdgpu_gmc_vram_pa(adev, adev->vce.vcpu_bo); |
| 537 | 554 | u64 flags = AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE | AMDGPU_PTE_VALID; |
| 538 | 555 | u64 vce_gart_start_offs; |
| 539 | 556 | int r; |
| 540 | 557 | |
| 541 | − r = amdgpu_gtt_mgr_alloc_entries(&adev->mman.gtt_mgr, | |
| 542 | − &adev->vce.gart_node, num_pages, | |
| 543 | − DRM_MM_INSERT_LOW); | |
| 544 | − if (r) | |
| 545 | − return r; | |
| 558 | + if (adev->gmc.vram_start < adev->gmc.gart_start) | |
| 559 | + return amdgpu_bo_gpu_offset(adev->vce.vcpu_bo) <= max_vcpu_bo_addr ? 0 : -EINVAL; | |
| 560 | + | |
| 561 | + if (!drm_mm_node_allocated(&adev->vce.gart_node)) { | |
| 562 | + r = amdgpu_gtt_mgr_alloc_entries(&adev->mman.gtt_mgr, | |
| 563 | + &adev->vce.gart_node, num_pages, | |
| 564 | + DRM_MM_INSERT_LOW); | |
| 565 | + if (r) | |
| 566 | + return r; | |
| 567 | + } | |
| 546 | 568 | |
| 547 | 569 | vce_gart_start_offs = amdgpu_gtt_node_to_byte_offset(&adev->vce.gart_node); |
| 548 | 570 | |
| @@ -553,8 +575,6 @@ static int vce_v1_0_ensure_vcpu_bo_32bit_addr(struct amdgpu_device *adev) · 4 unmodified lines | ||
| 553 | 575 | amdgpu_gart_map_vram_range(adev, pa, adev->vce.gart_node.start, |
| 554 | 576 | num_pages, flags, adev->gart.ptr); |
| 555 | 577 | adev->vce.gpu_addr = adev->gmc.gart_start + vce_gart_start_offs; |
| 556 | − if (adev->vce.gpu_addr > max_vcpu_bo_addr) | |
| 557 | − return -EINVAL; | |
| 558 | 578 | |
| 559 | 579 | return 0; |
| 560 | 580 | } |
| @@ -574,10 +594,7 @@ static int vce_v1_0_sw_init(struct amdgpu_ip_block *ip_block) · 13 unmodified lines | ||
| 574 | 594 | if (r) |
| 575 | 595 | return r; |
| 576 | 596 | |
| 577 | − r = amdgpu_vce_resume(adev); | |
| 578 | − if (r) | |
| 579 | − return r; | |
| 580 | − r = vce_v1_0_load_fw_signature(adev); | |
| 597 | + r = vce_v1_0_load_fw(adev); | |
| 581 | 598 | if (r) |
| 582 | 599 | return r; |
| 583 | 600 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); |
| @@ -696,10 +713,7 @@ static int vce_v1_0_resume(struct amdgpu_ip_block *ip_block) · 112 unmodified lines | ||
| 696 | 713 | struct amdgpu_device *adev = ip_block->adev; |
| 697 | 714 | int r; |
| 698 | 715 | |
| 699 | − r = amdgpu_vce_resume(adev); | |
| 700 | − if (r) | |
| 701 | − return r; | |
| 702 | − r = vce_v1_0_load_fw_signature(adev); | |
| 716 | + r = vce_v1_0_load_fw(adev); | |
| 703 | 717 | if (r) |
| 704 | 718 | return r; |
| 705 | 719 | r = vce_v1_0_ensure_vcpu_bo_32bit_addr(adev); |
| @@ -37,9 +37,14 @@ · 36 unmodified lines | ||
| 37 | 37 | #include "oss/oss_2_0_d.h" |
| 38 | 38 | #include "oss/oss_2_0_sh_mask.h" |
| 39 | 39 | |
| 40 | + | |
| 41 | +/* Use 24K to be safe. The FW supposedly only requires 23744 bytes. */ | |
| 42 | +#define VCE_V2_0_DATA_ENTRY_SIZE (24 * 1024) | |
| 43 | + | |
| 40 | 44 | #define VCE_V2_0_FW_SIZE (256 * 1024) |
| 41 | 45 | #define VCE_V2_0_STACK_SIZE (64 * 1024) |
| 42 | −#define VCE_V2_0_DATA_SIZE (23552 * AMDGPU_MAX_VCE_HANDLES) | |
| 46 | +#define VCE_V2_0_DATA_SIZE (VCE_V2_0_DATA_ENTRY_SIZE * (AMDGPU_MAX_VCE_HANDLES + 1)) | |
| 47 | + | |
| 43 | 48 | #define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02 |
| 44 | 49 | |
| 45 | 50 | static void vce_v2_0_set_ring_funcs(struct amdgpu_device *adev); |
| @@ -183,7 +188,7 @@ static void vce_v2_0_mc_resume(struct amdgpu_device *adev) · 137 unmodified lines | ||
| 183 | 188 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); |
| 184 | 189 | |
| 185 | 190 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; |
| 186 | − size = VCE_V2_0_FW_SIZE; | |
| 191 | + size = VCE_V2_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; | |
| 187 | 192 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); |
| 188 | 193 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 189 | 194 | |
| @@ -574,7 +574,7 @@ static void vce_v3_0_mc_resume(struct amdgpu_device *adev, int idx) · 573 unmodified lines | ||
| 574 | 574 | } else |
| 575 | 575 | WREG32(mmVCE_LMI_VCPU_CACHE_40BIT_BAR, (adev->vce.gpu_addr >> 8)); |
| 576 | 576 | offset = AMDGPU_VCE_FIRMWARE_OFFSET; |
| 577 | − size = VCE_V3_0_FW_SIZE; | |
| 577 | + size = VCE_V3_0_FW_SIZE - AMDGPU_VCE_FIRMWARE_OFFSET; | |
| 578 | 578 | WREG32(mmVCE_VCPU_CACHE_OFFSET0, offset & 0x7fffffff); |
| 579 | 579 | WREG32(mmVCE_VCPU_CACHE_SIZE0, size); |
| 580 | 580 | |
| @@ -2113,6 +2113,7 @@ static const struct amd_ip_funcs vcn_v2_0_ip_funcs = { · 2112 unmodified lines | ||
| 2113 | 2113 | static const struct amdgpu_ring_funcs vcn_v2_0_dec_ring_vm_funcs = { |
| 2114 | 2114 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 2115 | 2115 | .align_mask = 0xf, |
| 2116 | + .no_user_fence = true, | |
| 2116 | 2117 | .secure_submission_supported = true, |
| 2117 | 2118 | .get_rptr = vcn_v2_0_dec_ring_get_rptr, |
| 2118 | 2119 | .get_wptr = vcn_v2_0_dec_ring_get_wptr, |
| @@ -2145,6 +2146,7 @@ static const struct amdgpu_ring_funcs vcn_v2_0_enc_ring_vm_funcs = { · 26 unmodified lines | ||
| 2145 | 2146 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 2146 | 2147 | .align_mask = 0x3f, |
| 2147 | 2148 | .nop = VCN_ENC_CMD_NO_OP, |
| 2149 | + .no_user_fence = true, | |
| 2148 | 2150 | .get_rptr = vcn_v2_0_enc_ring_get_rptr, |
| 2149 | 2151 | .get_wptr = vcn_v2_0_enc_ring_get_wptr, |
| 2150 | 2152 | .set_wptr = vcn_v2_0_enc_ring_set_wptr, |
| @@ -1778,6 +1778,7 @@ static void vcn_v2_5_dec_ring_set_wptr(struct amdgpu_ring *ring) · 1777 unmodified lines | ||
| 1778 | 1778 | static const struct amdgpu_ring_funcs vcn_v2_5_dec_ring_vm_funcs = { |
| 1779 | 1779 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 1780 | 1780 | .align_mask = 0xf, |
| 1781 | + .no_user_fence = true, | |
| 1781 | 1782 | .secure_submission_supported = true, |
| 1782 | 1783 | .get_rptr = vcn_v2_5_dec_ring_get_rptr, |
| 1783 | 1784 | .get_wptr = vcn_v2_5_dec_ring_get_wptr, |
| @@ -1879,6 +1880,7 @@ static const struct amdgpu_ring_funcs vcn_v2_5_enc_ring_vm_funcs = { · 95 unmodified lines | ||
| 1879 | 1880 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1880 | 1881 | .align_mask = 0x3f, |
| 1881 | 1882 | .nop = VCN_ENC_CMD_NO_OP, |
| 1883 | + .no_user_fence = true, | |
| 1882 | 1884 | .get_rptr = vcn_v2_5_enc_ring_get_rptr, |
| 1883 | 1885 | .get_wptr = vcn_v2_5_enc_ring_get_wptr, |
| 1884 | 1886 | .set_wptr = vcn_v2_5_enc_ring_set_wptr, |
| @@ -1856,6 +1856,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_dec_sw_ring_vm_funcs = { · 1855 unmodified lines | ||
| 1856 | 1856 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 1857 | 1857 | .align_mask = 0x3f, |
| 1858 | 1858 | .nop = VCN_DEC_SW_CMD_NO_OP, |
| 1859 | + .no_user_fence = true, | |
| 1859 | 1860 | .secure_submission_supported = true, |
| 1860 | 1861 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, |
| 1861 | 1862 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, |
| @@ -1972,6 +1973,7 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 110 unmodified lines | ||
| 1972 | 1973 | |
| 1973 | 1974 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { |
| 1974 | 1975 | uint32_t offset, size, *create; |
| 1976 | + uint64_t buf_end; | |
| 1975 | 1977 | |
| 1976 | 1978 | if (msg[0] != RDECODE_MESSAGE_CREATE) |
| 1977 | 1979 | continue; |
| @@ -1979,7 +1981,8 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1 unmodified line | ||
| 1979 | 1981 | offset = msg[1]; |
| 1980 | 1982 | size = msg[2]; |
| 1981 | 1983 | |
| 1982 | − if (size < 4 || offset + size > end - addr) { | |
| 1984 | + if (size < 4 || check_add_overflow(offset, size, &buf_end) || | |
| 1985 | + buf_end > end - addr) { | |
| 1983 | 1986 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); |
| 1984 | 1987 | r = -EINVAL; |
| 1985 | 1988 | goto out; |
| @@ -2036,6 +2039,7 @@ static int vcn_v3_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, · 50 unmodified lines | ||
| 2036 | 2039 | static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = { |
| 2037 | 2040 | .type = AMDGPU_RING_TYPE_VCN_DEC, |
| 2038 | 2041 | .align_mask = 0xf, |
| 2042 | + .no_user_fence = true, | |
| 2039 | 2043 | .secure_submission_supported = true, |
| 2040 | 2044 | .get_rptr = vcn_v3_0_dec_ring_get_rptr, |
| 2041 | 2045 | .get_wptr = vcn_v3_0_dec_ring_get_wptr, |
| @@ -2138,6 +2142,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_enc_ring_vm_funcs = { · 96 unmodified lines | ||
| 2138 | 2142 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 2139 | 2143 | .align_mask = 0x3f, |
| 2140 | 2144 | .nop = VCN_ENC_CMD_NO_OP, |
| 2145 | + .no_user_fence = true, | |
| 2141 | 2146 | .get_rptr = vcn_v3_0_enc_ring_get_rptr, |
| 2142 | 2147 | .get_wptr = vcn_v3_0_enc_ring_get_wptr, |
| 2143 | 2148 | .set_wptr = vcn_v3_0_enc_ring_set_wptr, |
| @@ -1889,6 +1889,7 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1888 unmodified lines | ||
| 1889 | 1889 | |
| 1890 | 1890 | for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) { |
| 1891 | 1891 | uint32_t offset, size, *create; |
| 1892 | + uint64_t buf_end; | |
| 1892 | 1893 | |
| 1893 | 1894 | if (msg[0] != RDECODE_MESSAGE_CREATE) |
| 1894 | 1895 | continue; |
| @@ -1896,7 +1897,8 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, · 1 unmodified line | ||
| 1896 | 1897 | offset = msg[1]; |
| 1897 | 1898 | size = msg[2]; |
| 1898 | 1899 | |
| 1899 | − if (size < 4 || offset + size > end - addr) { | |
| 1900 | + if (size < 4 || check_add_overflow(offset, size, &buf_end) || | |
| 1901 | + buf_end > end - addr) { | |
| 1900 | 1902 | DRM_ERROR("VCN message buffer exceeds BO bounds!\n"); |
| 1901 | 1903 | r = -EINVAL; |
| 1902 | 1904 | goto out; |
| @@ -1994,6 +1996,7 @@ static struct amdgpu_ring_funcs vcn_v4_0_unified_ring_vm_funcs = { · 91 unmodified lines | ||
| 1994 | 1996 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1995 | 1997 | .align_mask = 0x3f, |
| 1996 | 1998 | .nop = VCN_ENC_CMD_NO_OP, |
| 1999 | + .no_user_fence = true, | |
| 1997 | 2000 | .extra_bytes = sizeof(struct amdgpu_vcn_rb_metadata), |
| 1998 | 2001 | .get_rptr = vcn_v4_0_unified_ring_get_rptr, |
| 1999 | 2002 | .get_wptr = vcn_v4_0_unified_ring_get_wptr, |
| @@ -1775,6 +1775,7 @@ static const struct amdgpu_ring_funcs vcn_v4_0_3_unified_ring_vm_funcs = { · 1774 unmodified lines | ||
| 1775 | 1775 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1776 | 1776 | .align_mask = 0x3f, |
| 1777 | 1777 | .nop = VCN_ENC_CMD_NO_OP, |
| 1778 | + .no_user_fence = true, | |
| 1778 | 1779 | .get_rptr = vcn_v4_0_3_unified_ring_get_rptr, |
| 1779 | 1780 | .get_wptr = vcn_v4_0_3_unified_ring_get_wptr, |
| 1780 | 1781 | .set_wptr = vcn_v4_0_3_unified_ring_set_wptr, |
| @@ -1483,6 +1483,7 @@ static struct amdgpu_ring_funcs vcn_v4_0_5_unified_ring_vm_funcs = { · 1482 unmodified lines | ||
| 1483 | 1483 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1484 | 1484 | .align_mask = 0x3f, |
| 1485 | 1485 | .nop = VCN_ENC_CMD_NO_OP, |
| 1486 | + .no_user_fence = true, | |
| 1486 | 1487 | .get_rptr = vcn_v4_0_5_unified_ring_get_rptr, |
| 1487 | 1488 | .get_wptr = vcn_v4_0_5_unified_ring_get_wptr, |
| 1488 | 1489 | .set_wptr = vcn_v4_0_5_unified_ring_set_wptr, |
| @@ -1207,6 +1207,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_0_unified_ring_vm_funcs = { · 1206 unmodified lines | ||
| 1207 | 1207 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1208 | 1208 | .align_mask = 0x3f, |
| 1209 | 1209 | .nop = VCN_ENC_CMD_NO_OP, |
| 1210 | + .no_user_fence = true, | |
| 1210 | 1211 | .get_rptr = vcn_v5_0_0_unified_ring_get_rptr, |
| 1211 | 1212 | .get_wptr = vcn_v5_0_0_unified_ring_get_wptr, |
| 1212 | 1213 | .set_wptr = vcn_v5_0_0_unified_ring_set_wptr, |
| @@ -1419,6 +1419,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_1_unified_ring_vm_funcs = { · 1418 unmodified lines | ||
| 1419 | 1419 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 1420 | 1420 | .align_mask = 0x3f, |
| 1421 | 1421 | .nop = VCN_ENC_CMD_NO_OP, |
| 1422 | + .no_user_fence = true, | |
| 1422 | 1423 | .get_rptr = vcn_v5_0_1_unified_ring_get_rptr, |
| 1423 | 1424 | .get_wptr = vcn_v5_0_1_unified_ring_get_wptr, |
| 1424 | 1425 | .set_wptr = vcn_v5_0_1_unified_ring_set_wptr, |
| @@ -994,6 +994,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_2_unified_ring_vm_funcs = { · 993 unmodified lines | ||
| 994 | 994 | .type = AMDGPU_RING_TYPE_VCN_ENC, |
| 995 | 995 | .align_mask = 0x3f, |
| 996 | 996 | .nop = VCN_ENC_CMD_NO_OP, |
| 997 | + .no_user_fence = true, | |
| 997 | 998 | .get_rptr = vcn_v5_0_2_unified_ring_get_rptr, |
| 998 | 999 | .get_wptr = vcn_v5_0_2_unified_ring_get_wptr, |
| 999 | 1000 | .set_wptr = vcn_v5_0_2_unified_ring_set_wptr, |
| @@ -25,6 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/err.h> |
| 26 | 26 | #include <linux/fs.h> |
| 27 | 27 | #include <linux/file.h> |
| 28 | +#include <linux/overflow.h> | |
| 28 | 29 | #include <linux/sched.h> |
| 29 | 30 | #include <linux/slab.h> |
| 30 | 31 | #include <linux/uaccess.h> |
| @@ -66,6 +67,21 @@ static const struct class kfd_class = { · 35 unmodified lines | ||
| 66 | 67 | .name = kfd_dev_name, |
| 67 | 68 | }; |
| 68 | 69 | |
| 70 | +/* | |
| 71 | + * Cache the address space of the chardev on first open so that the reset | |
| 72 | + * path can drop all userspace mappings of doorbell and MMIO ranges via | |
| 73 | + * unmap_mapping_range(). | |
| 74 | + */ | |
| 75 | +static struct address_space *kfd_dev_mapping; | |
| 76 | + | |
| 77 | +void kfd_dev_unmap_mapping_range(loff_t const holebegin, loff_t const holelen) | |
| 78 | +{ | |
| 79 | + struct address_space *mapping = READ_ONCE(kfd_dev_mapping); | |
| 80 | + | |
| 81 | + if (mapping) | |
| 82 | + unmap_mapping_range(mapping, holebegin, holelen, 1); | |
| 83 | +} | |
| 84 | + | |
| 69 | 85 | static inline struct kfd_process_device *kfd_lock_pdd_by_id(struct kfd_process *p, __u32 gpu_id) |
| 70 | 86 | { |
| 71 | 87 | struct kfd_process_device *pdd; |
| @@ -132,6 +148,13 @@ static int kfd_open(struct inode *inode, struct file *filep) · 60 unmodified lines | ||
| 132 | 148 | if (iminor(inode) != 0) |
| 133 | 149 | return -ENODEV; |
| 134 | 150 | |
| 151 | + /* | |
| 152 | + * /dev/kfd is a single chardev so all opens share one inode. Cache | |
| 153 | + * its address_space on the first open for use by the reset path. | |
| 154 | + */ | |
| 155 | + if (!READ_ONCE(kfd_dev_mapping)) | |
| 156 | + cmpxchg(&kfd_dev_mapping, NULL, inode->i_mapping); | |
| 157 | + | |
| 135 | 158 | is_32bit_user_mode = in_compat_syscall(); |
| 136 | 159 | |
| 137 | 160 | if (is_32bit_user_mode) { |
| @@ -1359,7 +1382,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 1221 unmodified lines | ||
| 1359 | 1382 | peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]); |
| 1360 | 1383 | if (WARN_ON_ONCE(!peer_pdd)) |
| 1361 | 1384 | continue; |
| 1362 | − kfd_flush_tlb(peer_pdd, TLB_FLUSH_LEGACY); | |
| 1385 | + kfd_flush_tlb(peer_pdd); | |
| 1363 | 1386 | } |
| 1364 | 1387 | kfree(devices_arr); |
| 1365 | 1388 | |
| @@ -1454,7 +1477,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 88 unmodified lines | ||
| 1454 | 1477 | if (WARN_ON_ONCE(!peer_pdd)) |
| 1455 | 1478 | continue; |
| 1456 | 1479 | if (flush_tlb) |
| 1457 | − kfd_flush_tlb(peer_pdd, TLB_FLUSH_HEAVYWEIGHT); | |
| 1480 | + kfd_flush_tlb(peer_pdd); | |
| 1458 | 1481 | |
| 1459 | 1482 | /* Remove dma mapping after tlb flush to avoid IO_PAGE_FAULT */ |
| 1460 | 1483 | err = amdgpu_amdkfd_gpuvm_dmaunmap_mem(mem, peer_pdd->drm_priv); |
| @@ -1695,6 +1718,16 @@ static int kfd_ioctl_smi_events(struct file *filep, · 234 unmodified lines | ||
| 1695 | 1718 | return kfd_smi_event_open(pdd->dev, &args->anon_fd); |
| 1696 | 1719 | } |
| 1697 | 1720 | |
| 1721 | +static int kfd_ioctl_svm_validate(void *kdata, unsigned int usize) | |
| 1722 | +{ | |
| 1723 | + struct kfd_ioctl_svm_args *args = kdata; | |
| 1724 | + size_t expected = struct_size(args, attrs, args->nattr); | |
| 1725 | + | |
| 1726 | + if (expected == SIZE_MAX || usize < expected) | |
| 1727 | + return -EINVAL; | |
| 1728 | + return 0; | |
| 1729 | +} | |
| 1730 | + | |
| 1698 | 1731 | #if IS_ENABLED(CONFIG_HSA_AMD_SVM) |
| 1699 | 1732 | |
| 1700 | 1733 | static int kfd_ioctl_set_xnack_mode(struct file *filep, |
| @@ -2267,6 +2300,11 @@ static int criu_restore_devices(struct kfd_process *p, · 566 unmodified lines | ||
| 2267 | 2300 | ret = -EINVAL; |
| 2268 | 2301 | goto exit; |
| 2269 | 2302 | } |
| 2303 | + | |
| 2304 | + if (pdd->drm_file) { | |
| 2305 | + ret = -EINVAL; | |
| 2306 | + goto exit; | |
| 2307 | + } | |
| 2270 | 2308 | pdd->user_gpu_id = device_buckets[i].user_gpu_id; |
| 2271 | 2309 | |
| 2272 | 2310 | drm_file = fget(device_buckets[i].drm_fd); |
| @@ -2277,11 +2315,6 @@ static int criu_restore_devices(struct kfd_process *p, · 4 unmodified lines | ||
| 2277 | 2315 | goto exit; |
| 2278 | 2316 | } |
| 2279 | 2317 | |
| 2280 | − if (pdd->drm_file) { | |
| 2281 | − ret = -EINVAL; | |
| 2282 | − goto exit; | |
| 2283 | − } | |
| 2284 | − | |
| 2285 | 2318 | /* create the vm using render nodes for kfd pdd */ |
| 2286 | 2319 | if (kfd_process_device_init_vm(pdd, drm_file)) { |
| 2287 | 2320 | pr_err("could not init vm for given pdd\n"); |
| @@ -3209,7 +3242,11 @@ static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, v · 921 unmodified lines | ||
| 3209 | 3242 | |
| 3210 | 3243 | #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \ |
| 3211 | 3244 | [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \ |
| 3212 | − .cmd_drv = 0, .name = #ioctl} | |
| 3245 | + .validate = NULL, .cmd_drv = 0, .name = #ioctl} | |
| 3246 | + | |
| 3247 | +#define AMDKFD_IOCTL_DEF_V(ioctl, _func, _validate, _flags) \ | |
| 3248 | + [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \ | |
| 3249 | + .validate = _validate, .cmd_drv = 0, .name = #ioctl} | |
| 3213 | 3250 | |
| 3214 | 3251 | /** Ioctl table */ |
| 3215 | 3252 | static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { |
| @@ -3306,7 +3343,8 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { · 90 unmodified lines | ||
| 3306 | 3343 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SMI_EVENTS, |
| 3307 | 3344 | kfd_ioctl_smi_events, 0), |
| 3308 | 3345 | |
| 3309 | − AMDKFD_IOCTL_DEF(AMDKFD_IOC_SVM, kfd_ioctl_svm, 0), | |
| 3346 | + AMDKFD_IOCTL_DEF_V(AMDKFD_IOC_SVM, kfd_ioctl_svm, | |
| 3347 | + kfd_ioctl_svm_validate, 0), | |
| 3310 | 3348 | |
| 3311 | 3349 | AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_XNACK_MODE, |
| 3312 | 3350 | kfd_ioctl_set_xnack_mode, 0), |
| @@ -3431,6 +3469,12 @@ static long kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 118 unmodified lines | ||
| 3431 | 3469 | memset(kdata, 0, usize); |
| 3432 | 3470 | } |
| 3433 | 3471 | |
| 3472 | + if (ioctl->validate) { | |
| 3473 | + retcode = ioctl->validate(kdata, usize); | |
| 3474 | + if (retcode) | |
| 3475 | + goto err_i1; | |
| 3476 | + } | |
| 3477 | + | |
| 3434 | 3478 | retcode = func(filep, process, kdata); |
| 3435 | 3479 | |
| 3436 | 3480 | if (cmd & IOC_OUT) |
| @@ -1737,37 +1737,6 @@ bool kgd2kfd_vmfault_fast_path(struct amdgpu_device *adev, struct amdgpu_iv_entr · 1736 unmodified lines | ||
| 1737 | 1737 | return false; |
| 1738 | 1738 | } |
| 1739 | 1739 | |
| 1740 | −/* check if there is kfd process still uses adev */ | |
| 1741 | −static bool kgd2kfd_check_device_idle(struct amdgpu_device *adev) | |
| 1742 | −{ | |
| 1743 | − struct kfd_process *p; | |
| 1744 | − struct hlist_node *p_temp; | |
| 1745 | − unsigned int temp; | |
| 1746 | − struct kfd_node *dev; | |
| 1747 | − | |
| 1748 | − mutex_lock(&kfd_processes_mutex); | |
| 1749 | − | |
| 1750 | − if (hash_empty(kfd_processes_table)) { | |
| 1751 | − mutex_unlock(&kfd_processes_mutex); | |
| 1752 | − return true; | |
| 1753 | − } | |
| 1754 | − | |
| 1755 | − /* check if there is device still use adev */ | |
| 1756 | − hash_for_each_safe(kfd_processes_table, temp, p_temp, p, kfd_processes) { | |
| 1757 | − for (int i = 0; i < p->n_pdds; i++) { | |
| 1758 | − dev = p->pdds[i]->dev; | |
| 1759 | − if (dev->adev == adev) { | |
| 1760 | − mutex_unlock(&kfd_processes_mutex); | |
| 1761 | − return false; | |
| 1762 | − } | |
| 1763 | − } | |
| 1764 | − } | |
| 1765 | − | |
| 1766 | − mutex_unlock(&kfd_processes_mutex); | |
| 1767 | − | |
| 1768 | − return true; | |
| 1769 | −} | |
| 1770 | − | |
| 1771 | 1740 | /** kgd2kfd_teardown_processes - gracefully tear down existing |
| 1772 | 1741 | * kfd processes that use adev |
| 1773 | 1742 | * |
| @@ -1800,7 +1769,7 @@ void kgd2kfd_teardown_processes(struct amdgpu_device *adev) · 26 unmodified lines | ||
| 1800 | 1769 | mutex_unlock(&kfd_processes_mutex); |
| 1801 | 1770 | |
| 1802 | 1771 | /* wait all kfd processes use adev terminate */ |
| 1803 | − while (!kgd2kfd_check_device_idle(adev)) | |
| 1772 | + while (!!atomic_read(&adev->kfd.dev->kfd_processes_count)) | |
| 1804 | 1773 | cond_resched(); |
| 1805 | 1774 | } |
| 1806 | 1775 | |
| @@ -475,6 +475,9 @@ static int allocate_doorbell(struct qcm_process_device *qpd, · 474 unmodified lines | ||
| 475 | 475 | } else { |
| 476 | 476 | /* For CP queues on SOC15 */ |
| 477 | 477 | if (restore_id) { |
| 478 | + if (*restore_id >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | |
| 479 | + return -EINVAL; | |
| 480 | + | |
| 478 | 481 | /* make sure that ID is free */ |
| 479 | 482 | if (__test_and_set_bit(*restore_id, qpd->doorbell_bitmap)) |
| 480 | 483 | return -EINVAL; |
| @@ -572,7 +575,7 @@ static int allocate_vmid(struct device_queue_manager *dqm, · 91 unmodified lines | ||
| 572 | 575 | qpd->vmid, |
| 573 | 576 | qpd->page_table_base); |
| 574 | 577 | /* invalidate the VM context after pasid and vmid mapping is set up */ |
| 575 | − kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); | |
| 578 | + kfd_flush_tlb(qpd_to_pdd(qpd)); | |
| 576 | 579 | |
| 577 | 580 | if (dqm->dev->kfd2kgd->set_scratch_backing_va) |
| 578 | 581 | dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->adev, |
| @@ -610,7 +613,7 @@ static void deallocate_vmid(struct device_queue_manager *dqm, · 31 unmodified lines | ||
| 610 | 613 | if (flush_texture_cache_nocpsch(q->device, qpd)) |
| 611 | 614 | dev_err(dev, "Failed to flush TC\n"); |
| 612 | 615 | |
| 613 | − kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); | |
| 616 | + kfd_flush_tlb(qpd_to_pdd(qpd)); | |
| 614 | 617 | |
| 615 | 618 | /* Release the vmid mapping */ |
| 616 | 619 | set_pasid_vmid_mapping(dqm, 0, qpd->vmid); |
| @@ -1284,7 +1287,7 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, · 667 unmodified lines | ||
| 1284 | 1287 | dqm->dev->adev, |
| 1285 | 1288 | qpd->vmid, |
| 1286 | 1289 | qpd->page_table_base); |
| 1287 | − kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); | |
| 1290 | + kfd_flush_tlb(pdd); | |
| 1288 | 1291 | } |
| 1289 | 1292 | |
| 1290 | 1293 | /* Take a safe reference to the mm_struct, which may otherwise |
| @@ -1587,6 +1590,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 296 unmodified lines | ||
| 1587 | 1590 | } |
| 1588 | 1591 | |
| 1589 | 1592 | if (restore_sdma_id) { |
| 1593 | + if (*restore_sdma_id >= get_num_sdma_queues(dqm)) | |
| 1594 | + return -EINVAL; | |
| 1595 | + | |
| 1590 | 1596 | /* Re-use existing sdma_id */ |
| 1591 | 1597 | if (!test_bit(*restore_sdma_id, dqm->sdma_bitmap)) { |
| 1592 | 1598 | dev_err(dev, "SDMA queue already in use\n"); |
| @@ -1613,6 +1619,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 20 unmodified lines | ||
| 1613 | 1619 | return -ENOMEM; |
| 1614 | 1620 | } |
| 1615 | 1621 | if (restore_sdma_id) { |
| 1622 | + if (*restore_sdma_id >= get_num_xgmi_sdma_queues(dqm)) | |
| 1623 | + return -EINVAL; | |
| 1624 | + | |
| 1616 | 1625 | /* Re-use existing sdma_id */ |
| 1617 | 1626 | if (!test_bit(*restore_sdma_id, dqm->xgmi_sdma_bitmap)) { |
| 1618 | 1627 | dev_err(dev, "SDMA queue already in use\n"); |
| @@ -3299,12 +3308,14 @@ static void copy_context_work_handler(struct work_struct *work) · 1680 unmodified lines | ||
| 3299 | 3308 | |
| 3300 | 3309 | static uint32_t *get_queue_ids(uint32_t num_queues, uint32_t *usr_queue_id_array) |
| 3301 | 3310 | { |
| 3302 | − size_t array_size = num_queues * sizeof(uint32_t); | |
| 3303 | − | |
| 3304 | 3311 | if (!usr_queue_id_array) |
| 3305 | 3312 | return NULL; |
| 3306 | 3313 | |
| 3307 | − return memdup_user(usr_queue_id_array, array_size); | |
| 3314 | + if (num_queues > KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | |
| 3315 | + return ERR_PTR(-EINVAL); | |
| 3316 | + | |
| 3317 | + return memdup_user(usr_queue_id_array, | |
| 3318 | + array_size(num_queues, sizeof(uint32_t))); | |
| 3308 | 3319 | } |
| 3309 | 3320 | |
| 3310 | 3321 | int resume_queues(struct kfd_process *p, |
| @@ -364,11 +364,15 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd, · 363 unmodified lines | ||
| 364 | 364 | { |
| 365 | 365 | struct v9_mqd *m; |
| 366 | 366 | struct kfd_context_save_area_header header; |
| 367 | + u32 cntl_stack_size; | |
| 368 | + u32 cntl_stack_offset; | |
| 367 | 369 | |
| 368 | 370 | /* Control stack is located one page after MQD. */ |
| 369 | 371 | void *mqd_ctl_stack = (void *)((uintptr_t)mqd + AMDGPU_GPU_PAGE_SIZE); |
| 370 | 372 | |
| 371 | 373 | m = get_mqd(mqd); |
| 374 | + cntl_stack_size = min_t(u32, m->cp_hqd_cntl_stack_size, q->ctl_stack_size); | |
| 375 | + cntl_stack_offset = min_t(u32, m->cp_hqd_cntl_stack_offset, cntl_stack_size); | |
| 372 | 376 | |
| 373 | 377 | *ctl_stack_used_size = m->cp_hqd_cntl_stack_size - |
| 374 | 378 | m->cp_hqd_cntl_stack_offset; |
| @@ -384,9 +388,10 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd, · 9 unmodified lines | ||
| 384 | 388 | if (copy_to_user(ctl_stack, &header, sizeof(header.wave_state))) |
| 385 | 389 | return -EFAULT; |
| 386 | 390 | |
| 387 | − if (copy_to_user(ctl_stack + m->cp_hqd_cntl_stack_offset, | |
| 388 | − mqd_ctl_stack + m->cp_hqd_cntl_stack_offset, | |
| 389 | − *ctl_stack_used_size)) | |
| 391 | + *ctl_stack_used_size = cntl_stack_size - cntl_stack_offset; | |
| 392 | + | |
| 393 | + if (copy_to_user(ctl_stack + cntl_stack_offset, mqd_ctl_stack + cntl_stack_offset, | |
| 394 | + *ctl_stack_used_size)) | |
| 390 | 395 | return -EFAULT; |
| 391 | 396 | |
| 392 | 397 | return 0; |
| @@ -395,6 +395,7 @@ enum kfd_mempool { · 394 unmodified lines | ||
| 395 | 395 | /* Character device interface */ |
| 396 | 396 | int kfd_chardev_init(void); |
| 397 | 397 | void kfd_chardev_exit(void); |
| 398 | +void kfd_dev_unmap_mapping_range(loff_t const holebegin, loff_t const holelen); | |
| 398 | 399 | |
| 399 | 400 | /** |
| 400 | 401 | * enum kfd_unmap_queues_filter - Enum for queue filters. |
| @@ -1047,10 +1048,13 @@ extern struct srcu_struct kfd_processes_srcu; · 646 unmodified lines | ||
| 1047 | 1048 | typedef int amdkfd_ioctl_t(struct file *filep, struct kfd_process *p, |
| 1048 | 1049 | void *data); |
| 1049 | 1050 | |
| 1051 | +typedef int amdkfd_ioctl_validate_t(void *kdata, unsigned int usize); | |
| 1052 | + | |
| 1050 | 1053 | struct amdkfd_ioctl_desc { |
| 1051 | 1054 | unsigned int cmd; |
| 1052 | 1055 | int flags; |
| 1053 | 1056 | amdkfd_ioctl_t *func; |
| 1057 | + amdkfd_ioctl_validate_t *validate; | |
| 1054 | 1058 | unsigned int cmd_drv; |
| 1055 | 1059 | const char *name; |
| 1056 | 1060 | }; |
| @@ -1551,13 +1555,13 @@ void kfd_signal_reset_event(struct kfd_node *dev); · 494 unmodified lines | ||
| 1551 | 1555 | void kfd_signal_poison_consumed_event(struct kfd_node *dev, u32 pasid); |
| 1552 | 1556 | void kfd_signal_process_terminate_event(struct kfd_process *p); |
| 1553 | 1557 | |
| 1554 | −static inline void kfd_flush_tlb(struct kfd_process_device *pdd, | |
| 1555 | − enum TLB_FLUSH_TYPE type) | |
| 1558 | +static inline void kfd_flush_tlb(struct kfd_process_device *pdd) | |
| 1556 | 1559 | { |
| 1557 | 1560 | struct amdgpu_device *adev = pdd->dev->adev; |
| 1558 | 1561 | struct amdgpu_vm *vm = drm_priv_to_vm(pdd->drm_priv); |
| 1559 | 1562 | |
| 1560 | − amdgpu_vm_flush_compute_tlb(adev, vm, type, pdd->dev->xcc_mask); | |
| 1563 | + amdgpu_vm_flush_compute_tlb(adev, vm, TLB_FLUSH_HEAVYWEIGHT, | |
| 1564 | + pdd->dev->xcc_mask); | |
| 1561 | 1565 | } |
| 1562 | 1566 | |
| 1563 | 1567 | static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev) |
| @@ -1366,6 +1366,12 @@ svm_range_unmap_from_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 1365 unmodified lines | ||
| 1366 | 1366 | |
| 1367 | 1367 | pr_debug("CPU[0x%llx 0x%llx] -> GPU[0x%llx 0x%llx]\n", start, last, |
| 1368 | 1368 | gpu_start, gpu_end); |
| 1369 | + | |
| 1370 | + if (!amdgpu_vm_ready(vm)) { | |
| 1371 | + pr_debug("VM not ready, canceling unmap\n"); | |
| 1372 | + return -EINVAL; | |
| 1373 | + } | |
| 1374 | + | |
| 1369 | 1375 | return amdgpu_vm_update_range(adev, vm, false, true, true, false, NULL, gpu_start, |
| 1370 | 1376 | gpu_end, init_pte_value, 0, 0, NULL, NULL, |
| 1371 | 1377 | fence); |
| @@ -1418,7 +1424,7 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start, · 46 unmodified lines | ||
| 1418 | 1424 | if (r) |
| 1419 | 1425 | break; |
| 1420 | 1426 | } |
| 1421 | − kfd_flush_tlb(pdd, TLB_FLUSH_HEAVYWEIGHT); | |
| 1427 | + kfd_flush_tlb(pdd); | |
| 1422 | 1428 | } |
| 1423 | 1429 | |
| 1424 | 1430 | return r; |
| @@ -1443,6 +1449,11 @@ svm_range_map_to_gpu(struct kfd_process_device *pdd, struct svm_range *prange, · 18 unmodified lines | ||
| 1443 | 1449 | pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms, |
| 1444 | 1450 | last_start, last_start + npages - 1, readonly); |
| 1445 | 1451 | |
| 1452 | + if (!amdgpu_vm_ready(vm)) { | |
| 1453 | + pr_debug("VM not ready, canceling map\n"); | |
| 1454 | + return -EINVAL; | |
| 1455 | + } | |
| 1456 | + | |
| 1446 | 1457 | for (i = offset; i < offset + npages; i++) { |
| 1447 | 1458 | uint64_t gpu_start; |
| 1448 | 1459 | uint64_t gpu_end; |
| @@ -1560,7 +1571,7 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset, · 111 unmodified lines | ||
| 1560 | 1571 | } |
| 1561 | 1572 | } |
| 1562 | 1573 | |
| 1563 | − kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); | |
| 1574 | + kfd_flush_tlb(pdd); | |
| 1564 | 1575 | } |
| 1565 | 1576 | |
| 1566 | 1577 | return r; |
| @@ -3721,6 +3732,9 @@ svm_range_set_attr(struct kfd_process *p, struct mm_struct *mm, · 2154 unmodified lines | ||
| 3721 | 3732 | |
| 3722 | 3733 | svms = &p->svms; |
| 3723 | 3734 | |
| 3735 | + if (!process_info) | |
| 3736 | + return -EINVAL; | |
| 3737 | + | |
| 3724 | 3738 | mutex_lock(&process_info->lock); |
| 3725 | 3739 | |
| 3726 | 3740 | svm_range_list_lock_and_flush_work(svms, mm); |
| @@ -1903,7 +1903,11 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) · 1902 unmodified lines | ||
| 1903 | 1903 | goto error; |
| 1904 | 1904 | } |
| 1905 | 1905 | |
| 1906 | − init_data.asic_id.chip_family = adev->family; | |
| 1906 | + /* special handling for early revisions of GC 11.5.4 */ | |
| 1907 | + if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(11, 5, 4)) | |
| 1908 | + init_data.asic_id.chip_family = AMDGPU_FAMILY_GC_11_5_4; | |
| 1909 | + else | |
| 1910 | + init_data.asic_id.chip_family = adev->family; | |
| 1907 | 1911 | |
| 1908 | 1912 | init_data.asic_id.pci_revision_id = adev->pdev->revision; |
| 1909 | 1913 | init_data.asic_id.hw_internal_rev = adev->external_rev_id; |
| @@ -9404,9 +9408,21 @@ static void manage_dm_interrupts(struct amdgpu_device *adev, · 7494 unmodified lines | ||
| 9404 | 9408 | if (acrtc_state) { |
| 9405 | 9409 | timing = &acrtc_state->stream->timing; |
| 9406 | 9410 | |
| 9407 | − if (amdgpu_ip_version(adev, DCE_HWIP, 0) < | |
| 9408 | − IP_VERSION(3, 5, 0) || | |
| 9409 | − !(adev->flags & AMD_IS_APU)) { | |
| 9411 | + if (amdgpu_ip_version(adev, DCE_HWIP, 0) >= | |
| 9412 | + IP_VERSION(3, 2, 0) && | |
| 9413 | + !(adev->flags & AMD_IS_APU)) { | |
| 9414 | + /* | |
| 9415 | + * DGPUs NV3x and newer that support idle optimizations | |
| 9416 | + * experience intermittent flip-done timeouts on cursor | |
| 9417 | + * updates. Restore 5s offdelay behavior for now. | |
| 9418 | + * | |
| 9419 | + * Discussion on the issue: | |
| 9420 | + * https://lore.kernel.org/amd-gfx/20260217191632.1243826-1-sysdadmin@m1k.cloud/ | |
| 9421 | + */ | |
| 9422 | + config.offdelay_ms = 5000; | |
| 9423 | + config.disable_immediate = false; | |
| 9424 | + } else if (amdgpu_ip_version(adev, DCE_HWIP, 0) < | |
| 9425 | + IP_VERSION(3, 5, 0)) { | |
| 9410 | 9426 | /* |
| 9411 | 9427 | * Older HW and DGPU have issues with instant off; |
| 9412 | 9428 | * use a 2 frame offdelay. |
| @@ -1032,6 +1032,45 @@ dm_helpers_read_acpi_edid(struct amdgpu_dm_connector *aconnector) · 1031 unmodified lines | ||
| 1032 | 1032 | return drm_edid_read_custom(connector, dm_helpers_probe_acpi_edid, connector); |
| 1033 | 1033 | } |
| 1034 | 1034 | |
| 1035 | +static const struct drm_edid * | |
| 1036 | +dm_helpers_read_vbios_hardcoded_edid(struct dc_link *link, struct amdgpu_dm_connector *aconnector) | |
| 1037 | +{ | |
| 1038 | + struct dc_bios *bios = link->ctx->dc_bios; | |
| 1039 | + struct embedded_panel_info info; | |
| 1040 | + const struct drm_edid *edid; | |
| 1041 | + enum bp_result r; | |
| 1042 | + | |
| 1043 | + if (!dc_is_embedded_signal(link->connector_signal) || | |
| 1044 | + !bios->funcs->get_embedded_panel_info) | |
| 1045 | + return NULL; | |
| 1046 | + | |
| 1047 | + memset(&info, 0, sizeof(info)); | |
| 1048 | + r = bios->funcs->get_embedded_panel_info(bios, &info); | |
| 1049 | + | |
| 1050 | + if (r != BP_RESULT_OK) { | |
| 1051 | + dm_error("Error when reading embedded panel info: %u\n", r); | |
| 1052 | + return NULL; | |
| 1053 | + } | |
| 1054 | + | |
| 1055 | + if (!info.fake_edid || !info.fake_edid_size) { | |
| 1056 | + dm_error("Embedded panel info doesn't contain an EDID\n"); | |
| 1057 | + return NULL; | |
| 1058 | + } | |
| 1059 | + | |
| 1060 | + edid = drm_edid_alloc(info.fake_edid, info.fake_edid_size); | |
| 1061 | + | |
| 1062 | + if (!drm_edid_valid(edid)) { | |
| 1063 | + dm_error("EDID from embedded panel info is invalid\n"); | |
| 1064 | + drm_edid_free(edid); | |
| 1065 | + return NULL; | |
| 1066 | + } | |
| 1067 | + | |
| 1068 | + aconnector->base.display_info.width_mm = info.panel_width_mm; | |
| 1069 | + aconnector->base.display_info.height_mm = info.panel_height_mm; | |
| 1070 | + | |
| 1071 | + return edid; | |
| 1072 | +} | |
| 1073 | + | |
| 1035 | 1074 | void populate_hdmi_info_from_connector(struct drm_hdmi_info *hdmi, struct dc_edid_caps *edid_caps) |
| 1036 | 1075 | { |
| 1037 | 1076 | edid_caps->scdc_present = hdmi->scdc.supported; |
| @@ -1052,6 +1091,9 @@ enum dc_edid_status dm_helpers_read_local_edid( · 14 unmodified lines | ||
| 1052 | 1091 | |
| 1053 | 1092 | if (link->aux_mode) |
| 1054 | 1093 | ddc = &aconnector->dm_dp_aux.aux.ddc; |
| 1094 | + else if (link->ddc_hw_inst == GPIO_DDC_LINE_UNKNOWN && | |
| 1095 | + dc_is_embedded_signal(link->connector_signal)) | |
| 1096 | + ddc = NULL; | |
| 1055 | 1097 | else |
| 1056 | 1098 | ddc = &aconnector->i2c->base; |
| 1057 | 1099 | |
| @@ -1065,6 +1107,8 @@ enum dc_edid_status dm_helpers_read_local_edid( · 7 unmodified lines | ||
| 1065 | 1107 | drm_edid = dm_helpers_read_acpi_edid(aconnector); |
| 1066 | 1108 | if (drm_edid) |
| 1067 | 1109 | drm_info(connector->dev, "Using ACPI provided EDID for %s\n", connector->name); |
| 1110 | + else if (!ddc) | |
| 1111 | + drm_edid = dm_helpers_read_vbios_hardcoded_edid(link, aconnector); | |
| 1068 | 1112 | else |
| 1069 | 1113 | drm_edid = drm_edid_read_ddc(connector, ddc); |
| 1070 | 1114 | drm_edid_connector_update(connector, drm_edid); |
| @@ -794,11 +794,13 @@ static enum bp_result bios_parser_external_encoder_control( · 793 unmodified lines | ||
| 794 | 794 | |
| 795 | 795 | static enum bp_result bios_parser_dac_load_detection( |
| 796 | 796 | struct dc_bios *dcb, |
| 797 | − enum engine_id engine_id) | |
| 797 | + enum engine_id engine_id, | |
| 798 | + struct graphics_object_id ext_enc_id) | |
| 798 | 799 | { |
| 799 | 800 | struct bios_parser *bp = BP_FROM_DCB(dcb); |
| 800 | 801 | struct dc_context *ctx = dcb->ctx; |
| 801 | 802 | struct bp_load_detection_parameters bp_params = {0}; |
| 803 | + struct bp_external_encoder_control ext_cntl = {0}; | |
| 802 | 804 | enum bp_result bp_result = BP_RESULT_UNSUPPORTED; |
| 803 | 805 | uint32_t bios_0_scratch; |
| 804 | 806 | uint32_t device_id_mask = 0; |
| @@ -824,6 +826,13 @@ static enum bp_result bios_parser_dac_load_detection( · 19 unmodified lines | ||
| 824 | 826 | |
| 825 | 827 | bp_params.engine_id = engine_id; |
| 826 | 828 | bp_result = bp->cmd_tbl.dac_load_detection(bp, &bp_params); |
| 829 | + } else if (ext_enc_id.id) { | |
| 830 | + if (!bp->cmd_tbl.external_encoder_control) | |
| 831 | + return BP_RESULT_UNSUPPORTED; | |
| 832 | + | |
| 833 | + ext_cntl.action = EXTERNAL_ENCODER_CONTROL_DAC_LOAD_DETECT; | |
| 834 | + ext_cntl.encoder_id = ext_enc_id; | |
| 835 | + bp_result = bp->cmd_tbl.external_encoder_control(bp, &ext_cntl); | |
| 827 | 836 | } |
| 828 | 837 | |
| 829 | 838 | if (bp_result != BP_RESULT_OK) |
| @@ -1304,6 +1313,60 @@ static enum bp_result bios_parser_get_embedded_panel_info( · 474 unmodified lines | ||
| 1304 | 1313 | return BP_RESULT_FAILURE; |
| 1305 | 1314 | } |
| 1306 | 1315 | |
| 1316 | +static enum bp_result get_embedded_panel_extra_info( | |
| 1317 | + struct bios_parser *bp, | |
| 1318 | + struct embedded_panel_info *info, | |
| 1319 | + const uint32_t table_offset) | |
| 1320 | +{ | |
| 1321 | + uint8_t *record = bios_get_image(&bp->base, table_offset, 1); | |
| 1322 | + ATOM_PANEL_RESOLUTION_PATCH_RECORD *panel_res_record; | |
| 1323 | + ATOM_FAKE_EDID_PATCH_RECORD *fake_edid_record; | |
| 1324 | + | |
| 1325 | + while (*record != ATOM_RECORD_END_TYPE) { | |
| 1326 | + switch (*record) { | |
| 1327 | + case LCD_MODE_PATCH_RECORD_MODE_TYPE: | |
| 1328 | + record += sizeof(ATOM_PATCH_RECORD_MODE); | |
| 1329 | + break; | |
| 1330 | + case LCD_RTS_RECORD_TYPE: | |
| 1331 | + record += sizeof(ATOM_LCD_RTS_RECORD); | |
| 1332 | + break; | |
| 1333 | + case LCD_CAP_RECORD_TYPE: | |
| 1334 | + record += sizeof(ATOM_LCD_MODE_CONTROL_CAP); | |
| 1335 | + break; | |
| 1336 | + case LCD_FAKE_EDID_PATCH_RECORD_TYPE: | |
| 1337 | + fake_edid_record = (ATOM_FAKE_EDID_PATCH_RECORD *)record; | |
| 1338 | + if (fake_edid_record->ucFakeEDIDLength) { | |
| 1339 | + if (fake_edid_record->ucFakeEDIDLength == 128) | |
| 1340 | + info->fake_edid_size = | |
| 1341 | + fake_edid_record->ucFakeEDIDLength; | |
| 1342 | + else | |
| 1343 | + info->fake_edid_size = | |
| 1344 | + fake_edid_record->ucFakeEDIDLength * 128; | |
| 1345 | + | |
| 1346 | + info->fake_edid = fake_edid_record->ucFakeEDIDString; | |
| 1347 | + | |
| 1348 | + record += struct_size(fake_edid_record, | |
| 1349 | + ucFakeEDIDString, | |
| 1350 | + info->fake_edid_size); | |
| 1351 | + } else { | |
| 1352 | + /* empty fake edid record must be 3 bytes long */ | |
| 1353 | + record += sizeof(ATOM_FAKE_EDID_PATCH_RECORD) + 1; | |
| 1354 | + } | |
| 1355 | + break; | |
| 1356 | + case LCD_PANEL_RESOLUTION_RECORD_TYPE: | |
| 1357 | + panel_res_record = (ATOM_PANEL_RESOLUTION_PATCH_RECORD *)record; | |
| 1358 | + info->panel_width_mm = panel_res_record->usHSize; | |
| 1359 | + info->panel_height_mm = panel_res_record->usVSize; | |
| 1360 | + record += sizeof(ATOM_PANEL_RESOLUTION_PATCH_RECORD); | |
| 1361 | + break; | |
| 1362 | + default: | |
| 1363 | + return BP_RESULT_BADBIOSTABLE; | |
| 1364 | + } | |
| 1365 | + } | |
| 1366 | + | |
| 1367 | + return BP_RESULT_OK; | |
| 1368 | +} | |
| 1369 | + | |
| 1307 | 1370 | static enum bp_result get_embedded_panel_info_v1_2( |
| 1308 | 1371 | struct bios_parser *bp, |
| 1309 | 1372 | struct embedded_panel_info *info) |
| @@ -1420,6 +1483,10 @@ static enum bp_result get_embedded_panel_info_v1_2( · 110 unmodified lines | ||
| 1420 | 1483 | if (ATOM_PANEL_MISC_API_ENABLED & lvds->ucLVDS_Misc) |
| 1421 | 1484 | info->lcd_timing.misc_info.API_ENABLED = true; |
| 1422 | 1485 | |
| 1486 | + if (lvds->usExtInfoTableOffset) | |
| 1487 | + return get_embedded_panel_extra_info(bp, info, | |
| 1488 | + le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info)); | |
| 1489 | + | |
| 1423 | 1490 | return BP_RESULT_OK; |
| 1424 | 1491 | } |
| 1425 | 1492 | |
| @@ -1545,6 +1612,10 @@ static enum bp_result get_embedded_panel_info_v1_3( · 119 unmodified lines | ||
| 1545 | 1612 | (uint32_t) (ATOM_PANEL_MISC_V13_GREY_LEVEL & |
| 1546 | 1613 | lvds->ucLCD_Misc) >> ATOM_PANEL_MISC_V13_GREY_LEVEL_SHIFT; |
| 1547 | 1614 | |
| 1615 | + if (lvds->usExtInfoTableOffset) | |
| 1616 | + return get_embedded_panel_extra_info(bp, info, | |
| 1617 | + le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info)); | |
| 1618 | + | |
| 1548 | 1619 | return BP_RESULT_OK; |
| 1549 | 1620 | } |
| 1550 | 1621 | |
| @@ -493,6 +493,10 @@ static enum bp_result get_gpio_i2c_info( · 492 unmodified lines | ||
| 493 | 493 | - sizeof(struct atom_common_table_header)) |
| 494 | 494 | / sizeof(struct atom_gpio_pin_assignment); |
| 495 | 495 | |
| 496 | + if (!bios_get_image(&bp->base, DATA_TABLES(gpio_pin_lut), | |
| 497 | + le16_to_cpu(header->table_header.structuresize))) | |
| 498 | + return BP_RESULT_BADBIOSTABLE; | |
| 499 | + | |
| 496 | 500 | pin = (struct atom_gpio_pin_assignment *) header->gpio_pin; |
| 497 | 501 | |
| 498 | 502 | for (table_index = 0; table_index < count; table_index++) { |
| @@ -681,6 +685,11 @@ static enum bp_result bios_parser_get_gpio_pin_info( · 182 unmodified lines | ||
| 681 | 685 | count = (le16_to_cpu(header->table_header.structuresize) |
| 682 | 686 | - sizeof(struct atom_common_table_header)) |
| 683 | 687 | / sizeof(struct atom_gpio_pin_assignment); |
| 688 | + | |
| 689 | + if (!bios_get_image(&bp->base, DATA_TABLES(gpio_pin_lut), | |
| 690 | + le16_to_cpu(header->table_header.structuresize))) | |
| 691 | + return BP_RESULT_BADBIOSTABLE; | |
| 692 | + | |
| 684 | 693 | for (i = 0; i < count; ++i) { |
| 685 | 694 | if (header->gpio_pin[i].gpio_id != gpio_id) |
| 686 | 695 | continue; |
| @@ -37,10 +37,13 @@ uint8_t *bios_get_image(struct dc_bios *bp, · 36 unmodified lines | ||
| 37 | 37 | uint32_t offset, |
| 38 | 38 | uint32_t size) |
| 39 | 39 | { |
| 40 | − if (bp->bios && offset + size < bp->bios_size) | |
| 41 | − return bp->bios + offset; | |
| 42 | − else | |
| 40 | + if (!bp->bios) | |
| 43 | 41 | return NULL; |
| 42 | + | |
| 43 | + if (offset > bp->bios_size || size > bp->bios_size - offset) | |
| 44 | + return NULL; | |
| 45 | + | |
| 46 | + return bp->bios + offset; | |
| 44 | 47 | } |
| 45 | 48 | |
| 46 | 49 | #include "reg_helper.h" |
| @@ -6071,7 +6071,11 @@ bool dc_process_dmub_aux_transfer_async(struct dc *dc, · 6070 unmodified lines | ||
| 6071 | 6071 | uint8_t action; |
| 6072 | 6072 | union dmub_rb_cmd cmd = {0}; |
| 6073 | 6073 | |
| 6074 | − ASSERT(payload->length <= 16); | |
| 6074 | + if (link_index >= dc->link_count || !dc->links[link_index]) | |
| 6075 | + return false; | |
| 6076 | + | |
| 6077 | + if (payload->length > sizeof(cmd.dp_aux_access.aux_control.dpaux.data)) | |
| 6078 | + return false; | |
| 6075 | 6079 | |
| 6076 | 6080 | cmd.dp_aux_access.header.type = DMUB_CMD__DP_AUX_ACCESS; |
| 6077 | 6081 | cmd.dp_aux_access.header.payload_bytes = 0; |
| @@ -1682,7 +1682,7 @@ struct dc_scratch_space { · 1681 unmodified lines | ||
| 1682 | 1682 | struct dc_link_training_overrides preferred_training_settings; |
| 1683 | 1683 | struct dp_audio_test_data audio_test_data; |
| 1684 | 1684 | |
| 1685 | − uint8_t ddc_hw_inst; | |
| 1685 | + enum gpio_ddc_line ddc_hw_inst; | |
| 1686 | 1686 | |
| 1687 | 1687 | uint8_t hpd_src; |
| 1688 | 1688 | |
| @@ -102,7 +102,8 @@ struct dc_vbios_funcs { · 101 unmodified lines | ||
| 102 | 102 | struct bp_external_encoder_control *cntl); |
| 103 | 103 | enum bp_result (*dac_load_detection)( |
| 104 | 104 | struct dc_bios *bios, |
| 105 | − enum engine_id engine_id); | |
| 105 | + enum engine_id engine_id, | |
| 106 | + struct graphics_object_id ext_enc_id); | |
| 106 | 107 | enum bp_result (*transmitter_control)( |
| 107 | 108 | struct dc_bios *bios, |
| 108 | 109 | struct bp_transmitter_control *cntl); |
| @@ -105,15 +105,26 @@ static void dccg21_update_dpp_dto(struct dccg *dccg, int dpp_inst, int req_dppcl · 104 unmodified lines | ||
| 105 | 105 | * dccg2_init() unconditionally overwrites MICROSECOND_TIME_BASE_DIV to |
| 106 | 106 | * 0x00120264, destroying the marker before it can be read. |
| 107 | 107 | * |
| 108 | − * Guard the call: if the S0i3 marker is present, skip dccg2_init() so the | |
| 108 | + * Guard the call: if the S0i3 marker is present, skip init so the | |
| 109 | 109 | * WA can function correctly. bios_golden_init() will handle init in that case. |
| 110 | + * | |
| 111 | + * DCN21 uses 48MHz refclk, not 100MHz, so we must explicitly set the correct | |
| 112 | + * values (48MHz is taken from rn_clk_mgr_construct()). | |
| 110 | 113 | */ |
| 111 | 114 | static void dccg21_init(struct dccg *dccg) |
| 112 | 115 | { |
| 116 | + struct dcn_dccg *dccg_dcn = TO_DCN_DCCG(dccg); | |
| 117 | + | |
| 113 | 118 | if (dccg2_is_s0i3_golden_init_wa_done(dccg)) |
| 114 | 119 | return; |
| 115 | 120 | |
| 116 | − dccg2_init(dccg); | |
| 121 | + /* 48MHz refclk from rn_clk_mgr_construct() */ | |
| 122 | + REG_WRITE(MICROSECOND_TIME_BASE_DIV, 0x00120230); | |
| 123 | + REG_WRITE(MILLISECOND_TIME_BASE_DIV, 0x0010bb80); | |
| 124 | + REG_WRITE(DISPCLK_FREQ_CHANGE_CNTL, 0x0e01003c); | |
| 125 | + | |
| 126 | + if (REG(REFCLK_CNTL)) | |
| 127 | + REG_WRITE(REFCLK_CNTL, 0); | |
| 117 | 128 | } |
| 118 | 129 | |
| 119 | 130 | static const struct dccg_funcs dccg21_funcs = { |
| @@ -1102,7 +1102,9 @@ void dce110_link_encoder_hw_init( · 1101 unmodified lines | ||
| 1102 | 1102 | ASSERT(result == BP_RESULT_OK); |
| 1103 | 1103 | |
| 1104 | 1104 | } |
| 1105 | − aux_initialize(enc110); | |
| 1105 | + | |
| 1106 | + if (enc110->aux_regs) | |
| 1107 | + aux_initialize(enc110); | |
| 1106 | 1108 | |
| 1107 | 1109 | /* reinitialize HPD. |
| 1108 | 1110 | * hpd_initialize() will pass DIG_FE id to HW context. |
| @@ -40,8 +40,8 @@ · 39 unmodified lines | ||
| 40 | 40 | #define FN(reg_name, field_name) \ |
| 41 | 41 | mcif_wb30->mcif_wb_shift->field_name, mcif_wb30->mcif_wb_mask->field_name |
| 42 | 42 | |
| 43 | −#define MCIF_ADDR(addr) (((unsigned long long)addr & 0xffffffffff) + 0xFE) >> 8 | |
| 44 | −#define MCIF_ADDR_HIGH(addr) (unsigned long long)addr >> 40 | |
| 43 | +#define MCIF_ADDR(addr) ((uint32_t)((((unsigned long long)(addr) & 0xffffffffffULL) + 0xFEULL) >> 8)) | |
| 44 | +#define MCIF_ADDR_HIGH(addr) ((uint32_t)(((unsigned long long)(addr)) >> 40)) | |
| 45 | 45 | |
| 46 | 46 | /* wbif programming guide: |
| 47 | 47 | * 1. set up wbif parameter: |
| @@ -646,6 +646,9 @@ failure: · 645 unmodified lines | ||
| 646 | 646 | enum gpio_ddc_line dal_ddc_get_line( |
| 647 | 647 | const struct ddc *ddc) |
| 648 | 648 | { |
| 649 | + if (!ddc) | |
| 650 | + return GPIO_DDC_LINE_UNKNOWN; | |
| 651 | + | |
| 649 | 652 | return (enum gpio_ddc_line)dal_gpio_get_enum(ddc->pin_data); |
| 650 | 653 | } |
| 651 | 654 | |
| @@ -665,16 +665,45 @@ void dce110_update_info_frame(struct pipe_ctx *pipe_ctx) · 664 unmodified lines | ||
| 665 | 665 | } |
| 666 | 666 | |
| 667 | 667 | static void |
| 668 | −dce110_dac_encoder_control(struct pipe_ctx *pipe_ctx, bool enable) | |
| 668 | +dce110_external_encoder_control(enum bp_external_encoder_control_action action, | |
| 669 | + struct dc_link *link, | |
| 670 | + struct dc_crtc_timing *timing) | |
| 669 | 671 | { |
| 670 | − struct dc_link *link = pipe_ctx->stream->link; | |
| 672 | + struct dc *dc = link->ctx->dc; | |
| 671 | 673 | struct dc_bios *bios = link->ctx->dc_bios; |
| 672 | − struct bp_encoder_control encoder_control = {0}; | |
| 674 | + const struct dc_link_settings *link_settings = &link->cur_link_settings; | |
| 675 | + enum bp_result bp_result = BP_RESULT_OK; | |
| 676 | + struct bp_external_encoder_control ext_cntl = { | |
| 677 | + .action = action, | |
| 678 | + .connector_obj_id = link->link_enc->connector, | |
| 679 | + .encoder_id = link->ext_enc_id, | |
| 680 | + .lanes_number = link_settings->lane_count, | |
| 681 | + .link_rate = link_settings->link_rate, | |
| 682 | + | |
| 683 | + /* Use signal type of the real link encoder, ie. DP */ | |
| 684 | + .signal = link->connector_signal, | |
| 685 | + | |
| 686 | + /* We don't know the timing yet when executing the SETUP action, | |
| 687 | + * so use a reasonably high default value. It seems that ENABLE | |
| 688 | + * can change the actual pixel clock but doesn't work with higher | |
| 689 | + * pixel clocks than what SETUP was called with. | |
| 690 | + */ | |
| 691 | + .pixel_clock = timing ? timing->pix_clk_100hz / 10 : 300000, | |
| 692 | + .color_depth = timing ? timing->display_color_depth : COLOR_DEPTH_888, | |
| 693 | + }; | |
| 694 | + DC_LOGGER_INIT(dc->ctx); | |
| 673 | 695 | |
| 674 | − encoder_control.action = enable ? ENCODER_CONTROL_ENABLE : ENCODER_CONTROL_DISABLE; | |
| 675 | − encoder_control.engine_id = link->link_enc->analog_engine; | |
| 676 | − encoder_control.pixel_clock = pipe_ctx->stream->timing.pix_clk_100hz / 10; | |
| 677 | − bios->funcs->encoder_control(bios, &encoder_control); | |
| 696 | + bp_result = bios->funcs->external_encoder_control(bios, &ext_cntl); | |
| 697 | + | |
| 698 | + if (bp_result != BP_RESULT_OK) | |
| 699 | + DC_LOG_ERROR("Failed to execute external encoder action: 0x%x\n", action); | |
| 700 | +} | |
| 701 | + | |
| 702 | +static void | |
| 703 | +dce110_prepare_ddc(struct dc_link *link) | |
| 704 | +{ | |
| 705 | + if (link->ext_enc_id.id) | |
| 706 | + dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_DDC_SETUP, link, NULL); | |
| 678 | 707 | } |
| 679 | 708 | |
| 680 | 709 | static bool |
| @@ -684,7 +713,8 @@ dce110_dac_load_detect(struct dc_link *link) · 3 unmodified lines | ||
| 684 | 713 | struct link_encoder *link_enc = link->link_enc; |
| 685 | 714 | enum bp_result bp_result; |
| 686 | 715 | |
| 687 | − bp_result = bios->funcs->dac_load_detection(bios, link_enc->analog_engine); | |
| 716 | + bp_result = bios->funcs->dac_load_detection( | |
| 717 | + bios, link_enc->analog_engine, link->ext_enc_id); | |
| 688 | 718 | return bp_result == BP_RESULT_OK; |
| 689 | 719 | } |
| 690 | 720 | |
| @@ -700,7 +730,6 @@ void dce110_enable_stream(struct pipe_ctx *pipe_ctx) · 9 unmodified lines | ||
| 700 | 730 | uint32_t early_control = 0; |
| 701 | 731 | struct timing_generator *tg = pipe_ctx->stream_res.tg; |
| 702 | 732 | |
| 703 | − link_hwss->setup_stream_attribute(pipe_ctx); | |
| 704 | 733 | link_hwss->setup_stream_encoder(pipe_ctx); |
| 705 | 734 | |
| 706 | 735 | dc->hwss.update_info_frame(pipe_ctx); |
| @@ -719,8 +748,8 @@ void dce110_enable_stream(struct pipe_ctx *pipe_ctx) · 12 unmodified lines | ||
| 719 | 748 | |
| 720 | 749 | tg->funcs->set_early_control(tg, early_control); |
| 721 | 750 | |
| 722 | − if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | |
| 723 | − dce110_dac_encoder_control(pipe_ctx, true); | |
| 751 | + if (link->ext_enc_id.id) | |
| 752 | + dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_ENABLE, link, timing); | |
| 724 | 753 | } |
| 725 | 754 | |
| 726 | 755 | static enum bp_result link_transmitter_control( |
| @@ -1219,8 +1248,8 @@ void dce110_disable_stream(struct pipe_ctx *pipe_ctx) · 492 unmodified lines | ||
| 1219 | 1248 | link_enc->transmitter - TRANSMITTER_UNIPHY_A); |
| 1220 | 1249 | } |
| 1221 | 1250 | |
| 1222 | − if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | |
| 1223 | − dce110_dac_encoder_control(pipe_ctx, false); | |
| 1251 | + if (link->ext_enc_id.id) | |
| 1252 | + dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_DISABLE, link, NULL); | |
| 1224 | 1253 | } |
| 1225 | 1254 | |
| 1226 | 1255 | void dce110_unblank_stream(struct pipe_ctx *pipe_ctx, |
| @@ -1603,22 +1632,6 @@ static enum dc_status dce110_enable_stream_timing( · 376 unmodified lines | ||
| 1603 | 1632 | |
| 1604 | 1633 | return DC_OK; |
| 1605 | 1634 | } |
| 1606 | −static void | |
| 1607 | −dce110_select_crtc_source(struct pipe_ctx *pipe_ctx) | |
| 1608 | −{ | |
| 1609 | − struct dc_link *link = pipe_ctx->stream->link; | |
| 1610 | − struct dc_bios *bios = link->ctx->dc_bios; | |
| 1611 | − struct bp_crtc_source_select crtc_source_select = {0}; | |
| 1612 | − enum engine_id engine_id = link->link_enc->preferred_engine; | |
| 1613 | − | |
| 1614 | − if (dc_is_rgb_signal(pipe_ctx->stream->signal)) | |
| 1615 | − engine_id = link->link_enc->analog_engine; | |
| 1616 | − crtc_source_select.controller_id = CONTROLLER_ID_D0 + pipe_ctx->stream_res.tg->inst; | |
| 1617 | − crtc_source_select.color_depth = pipe_ctx->stream->timing.display_color_depth; | |
| 1618 | − crtc_source_select.engine_id = engine_id; | |
| 1619 | − crtc_source_select.sink_signal = pipe_ctx->stream->signal; | |
| 1620 | − bios->funcs->select_crtc_source(bios, &crtc_source_select); | |
| 1621 | −} | |
| 1622 | 1635 | |
| 1623 | 1636 | enum dc_status dce110_apply_single_controller_ctx_to_hw( |
| 1624 | 1637 | struct pipe_ctx *pipe_ctx, |
| @@ -1639,10 +1652,6 @@ enum dc_status dce110_apply_single_controller_ctx_to_hw( · 14 unmodified lines | ||
| 1639 | 1652 | hws->funcs.disable_stream_gating(dc, pipe_ctx); |
| 1640 | 1653 | } |
| 1641 | 1654 | |
| 1642 | − if (pipe_ctx->stream->signal == SIGNAL_TYPE_RGB) { | |
| 1643 | − dce110_select_crtc_source(pipe_ctx); | |
| 1644 | − } | |
| 1645 | − | |
| 1646 | 1655 | if (pipe_ctx->stream_res.audio != NULL) { |
| 1647 | 1656 | struct audio_output audio_output = {0}; |
| 1648 | 1657 | |
| @@ -1722,8 +1731,7 @@ enum dc_status dce110_apply_single_controller_ctx_to_hw( · 73 unmodified lines | ||
| 1722 | 1731 | pipe_ctx->stream_res.tg->funcs->set_static_screen_control( |
| 1723 | 1732 | pipe_ctx->stream_res.tg, event_triggers, 2); |
| 1724 | 1733 | |
| 1725 | − if (!dc_is_virtual_signal(pipe_ctx->stream->signal) && | |
| 1726 | − !dc_is_rgb_signal(pipe_ctx->stream->signal)) | |
| 1734 | + if (!dc_is_virtual_signal(pipe_ctx->stream->signal)) | |
| 1727 | 1735 | pipe_ctx->stream_res.stream_enc->funcs->dig_connect_to_otg( |
| 1728 | 1736 | pipe_ctx->stream_res.stream_enc, |
| 1729 | 1737 | pipe_ctx->stream_res.tg->inst); |
| @@ -3376,6 +3384,15 @@ void dce110_enable_tmds_link_output(struct dc_link *link, · 1646 unmodified lines | ||
| 3376 | 3384 | link->phy_state.symclk_state = SYMCLK_ON_TX_ON; |
| 3377 | 3385 | } |
| 3378 | 3386 | |
| 3387 | +static void dce110_enable_analog_link_output( | |
| 3388 | + struct dc_link *link, | |
| 3389 | + uint32_t pix_clk_100hz) | |
| 3390 | +{ | |
| 3391 | + link->link_enc->funcs->enable_analog_output( | |
| 3392 | + link->link_enc, | |
| 3393 | + pix_clk_100hz); | |
| 3394 | +} | |
| 3395 | + | |
| 3379 | 3396 | void dce110_enable_dp_link_output( |
| 3380 | 3397 | struct dc_link *link, |
| 3381 | 3398 | const struct link_resource *link_res, |
| @@ -3423,6 +3440,11 @@ void dce110_enable_dp_link_output( · 41 unmodified lines | ||
| 3423 | 3440 | } |
| 3424 | 3441 | } |
| 3425 | 3442 | |
| 3443 | + if (link->ext_enc_id.id) { | |
| 3444 | + dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_INIT, link, NULL); | |
| 3445 | + dce110_external_encoder_control(EXTERNAL_ENCODER_CONTROL_SETUP, link, NULL); | |
| 3446 | + } | |
| 3447 | + | |
| 3426 | 3448 | if (dc->link_srv->dp_get_encoding_format(link_settings) == DP_8b_10b_ENCODING) { |
| 3427 | 3449 | if (dc->clk_mgr->funcs->notify_link_rate_change) |
| 3428 | 3450 | dc->clk_mgr->funcs->notify_link_rate_change(dc->clk_mgr, link); |
| @@ -3513,8 +3535,10 @@ static const struct hw_sequencer_funcs dce110_funcs = { · 84 unmodified lines | ||
| 3513 | 3535 | .enable_lvds_link_output = dce110_enable_lvds_link_output, |
| 3514 | 3536 | .enable_tmds_link_output = dce110_enable_tmds_link_output, |
| 3515 | 3537 | .enable_dp_link_output = dce110_enable_dp_link_output, |
| 3538 | + .enable_analog_link_output = dce110_enable_analog_link_output, | |
| 3516 | 3539 | .disable_link_output = dce110_disable_link_output, |
| 3517 | 3540 | .dac_load_detect = dce110_dac_load_detect, |
| 3541 | + .prepare_ddc = dce110_prepare_ddc, | |
| 3518 | 3542 | }; |
| 3519 | 3543 | |
| 3520 | 3544 | static const struct hwseq_private_funcs dce110_private_funcs = { |
| @@ -568,7 +568,9 @@ static bool construct_phy(struct dc_link *link, · 567 unmodified lines | ||
| 568 | 568 | goto ddc_create_fail; |
| 569 | 569 | } |
| 570 | 570 | |
| 571 | − if (!link->ddc->ddc_pin) { | |
| 571 | + /* Embedded display connectors such as LVDS may not have DDC. */ | |
| 572 | + if (!link->ddc->ddc_pin && | |
| 573 | + !dc_is_embedded_signal(link->connector_signal)) { | |
| 572 | 574 | DC_ERROR("Failed to get I2C info for connector!\n"); |
| 573 | 575 | goto ddc_create_fail; |
| 574 | 576 | } |
| @@ -753,7 +753,8 @@ static struct link_encoder *dce60_link_encoder_create( · 752 unmodified lines | ||
| 753 | 753 | enc_init_data, |
| 754 | 754 | &link_enc_feature, |
| 755 | 755 | &link_enc_regs[link_regs_id], |
| 756 | − &link_enc_aux_regs[enc_init_data->channel - 1], | |
| 756 | + enc_init_data->channel == CHANNEL_ID_UNKNOWN ? | |
| 757 | + NULL : &link_enc_aux_regs[enc_init_data->channel - 1], | |
| 757 | 758 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? |
| 758 | 759 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); |
| 759 | 760 | return &enc110->base; |
| @@ -760,7 +760,8 @@ static struct link_encoder *dce80_link_encoder_create( · 759 unmodified lines | ||
| 760 | 760 | enc_init_data, |
| 761 | 761 | &link_enc_feature, |
| 762 | 762 | &link_enc_regs[link_regs_id], |
| 763 | − &link_enc_aux_regs[enc_init_data->channel - 1], | |
| 763 | + enc_init_data->channel == CHANNEL_ID_UNKNOWN ? | |
| 764 | + NULL : &link_enc_aux_regs[enc_init_data->channel - 1], | |
| 764 | 765 | enc_init_data->hpd_source >= ARRAY_SIZE(link_enc_hpd_regs) ? |
| 765 | 766 | NULL : &link_enc_hpd_regs[enc_init_data->hpd_source]); |
| 766 | 767 | return &enc110->base; |
| @@ -92,9 +92,14 @@ · 91 unmodified lines | ||
| 92 | 92 | #include "dml/dcn32/dcn32_fpu.h" |
| 93 | 93 | |
| 94 | 94 | #include "dc_state_priv.h" |
| 95 | +#include "dc_fpu.h" | |
| 95 | 96 | |
| 96 | 97 | #include "dml2_0/dml2_wrapper.h" |
| 97 | 98 | |
| 99 | +#if !defined(DC_RUN_WITH_PREEMPTION_ENABLED) | |
| 100 | +#define DC_RUN_WITH_PREEMPTION_ENABLED(code) code | |
| 101 | +#endif | |
| 102 | + | |
| 98 | 103 | #define DC_LOGGER_INIT(logger) |
| 99 | 104 | |
| 100 | 105 | enum dcn32_clk_src_array_id { |
| @@ -1684,7 +1689,8 @@ static void dcn32_enable_phantom_plane(struct dc *dc, · 1583 unmodified lines | ||
| 1684 | 1689 | if (curr_pipe->top_pipe && curr_pipe->top_pipe->plane_state == curr_pipe->plane_state) |
| 1685 | 1690 | phantom_plane = prev_phantom_plane; |
| 1686 | 1691 | else |
| 1687 | − phantom_plane = dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state); | |
| 1692 | + DC_RUN_WITH_PREEMPTION_ENABLED(phantom_plane = | |
| 1693 | + dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state)); | |
| 1688 | 1694 | |
| 1689 | 1695 | if (!phantom_plane) |
| 1690 | 1696 | continue; |
| @@ -153,6 +153,10 @@ struct embedded_panel_info { · 152 unmodified lines | ||
| 153 | 153 | uint32_t drr_enabled; |
| 154 | 154 | uint32_t min_drr_refresh_rate; |
| 155 | 155 | bool realtek_eDPToLVDS; |
| 156 | + uint16_t panel_width_mm; | |
| 157 | + uint16_t panel_height_mm; | |
| 158 | + uint16_t fake_edid_size; | |
| 159 | + const uint8_t *fake_edid; | |
| 156 | 160 | }; |
| 157 | 161 | |
| 158 | 162 | struct dc_firmware_info { |
| @@ -3076,6 +3076,10 @@ static bool si_dpm_vblank_too_short(void *handle) · 3075 unmodified lines | ||
| 3076 | 3076 | /* we never hit the non-gddr5 limit so disable it */ |
| 3077 | 3077 | u32 switch_limit = adev->gmc.vram_type == AMDGPU_VRAM_TYPE_GDDR5 ? 450 : 0; |
| 3078 | 3078 | |
| 3079 | + /* Disregard vblank time when there are no displays connected */ | |
| 3080 | + if (!adev->pm.pm_display_cfg.num_display) | |
| 3081 | + return false; | |
| 3082 | + | |
| 3079 | 3083 | /* Consider zero vblank time too short and disable MCLK switching. |
| 3080 | 3084 | * Note that the vblank time is set to maximum when no displays are attached, |
| 3081 | 3085 | * so we'll still enable MCLK switching in that case. |
| @@ -1333,12 +1333,13 @@ static int ci_populate_all_memory_levels(struct pp_hwmgr *hwmgr) · 1332 unmodified lines | ||
| 1333 | 1333 | |
| 1334 | 1334 | dev_id = adev->pdev->device; |
| 1335 | 1335 | |
| 1336 | − if ((dpm_table->mclk_table.count >= 2) | |
| 1337 | − && ((dev_id == 0x67B0) || (dev_id == 0x67B1))) { | |
| 1338 | − smu_data->smc_state_table.MemoryLevel[1].MinVddci = | |
| 1339 | − smu_data->smc_state_table.MemoryLevel[0].MinVddci; | |
| 1340 | − smu_data->smc_state_table.MemoryLevel[1].MinMvdd = | |
| 1341 | − smu_data->smc_state_table.MemoryLevel[0].MinMvdd; | |
| 1336 | + if ((dpm_table->mclk_table.count >= 2) && | |
| 1337 | + ((dev_id == 0x67B0) || (dev_id == 0x67B1)) && | |
| 1338 | + (adev->pdev->revision == 0)) { | |
| 1339 | + smu_data->smc_state_table.MemoryLevel[1].MinVddc = | |
| 1340 | + smu_data->smc_state_table.MemoryLevel[0].MinVddc; | |
| 1341 | + smu_data->smc_state_table.MemoryLevel[1].MinVddcPhases = | |
| 1342 | + smu_data->smc_state_table.MemoryLevel[0].MinVddcPhases; | |
| 1342 | 1343 | } |
| 1343 | 1344 | smu_data->smc_state_table.MemoryLevel[0].ActivityLevel = 0x1F; |
| 1344 | 1345 | CONVERT_FROM_HOST_TO_SMC_US(smu_data->smc_state_table.MemoryLevel[0].ActivityLevel); |
| @@ -425,6 +425,7 @@ static int aldebaran_set_default_dpm_table(struct smu_context *smu) · 424 unmodified lines | ||
| 425 | 425 | dpm_table->dpm_levels[0].enabled = true; |
| 426 | 426 | dpm_table->dpm_levels[1].value = pptable->GfxclkFmax; |
| 427 | 427 | dpm_table->dpm_levels[1].enabled = true; |
| 428 | + dpm_table->flags |= SMU_DPM_TABLE_FINE_GRAINED; | |
| 428 | 429 | } else { |
| 429 | 430 | dpm_table->count = 1; |
| 430 | 431 | dpm_table->dpm_levels[0].value = smu->smu_table.boot_values.gfxclk / 100; |
| @@ -1129,6 +1129,7 @@ static int smu_v13_0_6_set_default_dpm_table(struct smu_context *smu) · 1128 unmodified lines | ||
| 1129 | 1129 | /* gfxclk dpm table setup */ |
| 1130 | 1130 | dpm_table = &dpm_context->dpm_tables.gfx_table; |
| 1131 | 1131 | dpm_table->clk_type = SMU_GFXCLK; |
| 1132 | + dpm_table->flags = SMU_DPM_TABLE_FINE_GRAINED; | |
| 1132 | 1133 | if (smu_cmn_feature_is_enabled(smu, SMU_FEATURE_DPM_GFXCLK_BIT)) { |
| 1133 | 1134 | /* In the case of gfxclk, only fine-grained dpm is honored. |
| 1134 | 1135 | * Get min/max values from FW. |
| @@ -435,10 +435,12 @@ int smu_v15_0_fini_smc_tables(struct smu_context *smu) · 434 unmodified lines | ||
| 435 | 435 | smu_table->watermarks_table = NULL; |
| 436 | 436 | smu_table->metrics_time = 0; |
| 437 | 437 | |
| 438 | + kfree(smu_dpm->dpm_policies); | |
| 438 | 439 | kfree(smu_dpm->dpm_context); |
| 439 | 440 | kfree(smu_dpm->golden_dpm_context); |
| 440 | 441 | kfree(smu_dpm->dpm_current_power_state); |
| 441 | 442 | kfree(smu_dpm->dpm_request_power_state); |
| 443 | + smu_dpm->dpm_policies = NULL; | |
| 442 | 444 | smu_dpm->dpm_context = NULL; |
| 443 | 445 | smu_dpm->golden_dpm_context = NULL; |
| 444 | 446 | smu_dpm->dpm_context_size = 0; |
| @@ -1370,7 +1370,7 @@ int smu_cmn_print_dpm_clk_levels(struct smu_context *smu, · 1369 unmodified lines | ||
| 1370 | 1370 | level_index = 1; |
| 1371 | 1371 | } |
| 1372 | 1372 | |
| 1373 | − if (!is_fine_grained) { | |
| 1373 | + if (!is_fine_grained || count == 1) { | |
| 1374 | 1374 | for (i = 0; i < count; i++) { |
| 1375 | 1375 | freq_match = !is_deep_sleep && |
| 1376 | 1376 | smu_cmn_freqs_match( |
| @@ -758,7 +758,9 @@ static int chipone_i2c_probe(struct i2c_client *client) · 757 unmodified lines | ||
| 758 | 758 | dev_set_drvdata(dev, icn); |
| 759 | 759 | i2c_set_clientdata(client, icn); |
| 760 | 760 | |
| 761 | − drm_bridge_add(&icn->bridge); | |
| 761 | + ret = devm_drm_bridge_add(dev, &icn->bridge); | |
| 762 | + if (ret) | |
| 763 | + return ret; | |
| 762 | 764 | |
| 763 | 765 | return chipone_dsi_host_attach(icn); |
| 764 | 766 | } |
| @@ -222,52 +222,58 @@ static const struct drm_bridge_funcs imx8qxp_pxl2dpi_bridge_funcs = { · 221 unmodified lines | ||
| 222 | 222 | imx8qxp_pxl2dpi_bridge_atomic_get_output_bus_fmts, |
| 223 | 223 | }; |
| 224 | 224 | |
| 225 | −static struct device_node * | |
| 225 | +static int | |
| 226 | 226 | imx8qxp_pxl2dpi_get_available_ep_from_port(struct imx8qxp_pxl2dpi *p2d, |
| 227 | − u32 port_id) | |
| 227 | + u32 port_id, | |
| 228 | + struct device_node **ep) | |
| 228 | 229 | { |
| 229 | − struct device_node *port, *ep; | |
| 230 | + struct device_node *port; | |
| 231 | + int ret = 0; | |
| 230 | 232 | int ep_cnt; |
| 231 | 233 | |
| 234 | + *ep = NULL; | |
| 235 | + | |
| 232 | 236 | port = of_graph_get_port_by_id(p2d->dev->of_node, port_id); |
| 233 | 237 | if (!port) { |
| 234 | 238 | DRM_DEV_ERROR(p2d->dev, "failed to get port@%u\n", port_id); |
| 235 | − return ERR_PTR(-ENODEV); | |
| 239 | + return -ENODEV; | |
| 236 | 240 | } |
| 237 | 241 | |
| 238 | 242 | ep_cnt = of_get_available_child_count(port); |
| 239 | 243 | if (ep_cnt == 0) { |
| 240 | 244 | DRM_DEV_ERROR(p2d->dev, "no available endpoints of port@%u\n", |
| 241 | 245 | port_id); |
| 242 | − ep = ERR_PTR(-ENODEV); | |
| 246 | + ret = -ENODEV; | |
| 243 | 247 | goto out; |
| 244 | 248 | } else if (ep_cnt > 1) { |
| 245 | 249 | DRM_DEV_ERROR(p2d->dev, |
| 246 | 250 | "invalid available endpoints of port@%u\n", |
| 247 | 251 | port_id); |
| 248 | − ep = ERR_PTR(-EINVAL); | |
| 252 | + ret = -EINVAL; | |
| 249 | 253 | goto out; |
| 250 | 254 | } |
| 251 | 255 | |
| 252 | − ep = of_get_next_available_child(port, NULL); | |
| 253 | − if (!ep) { | |
| 256 | + *ep = of_get_next_available_child(port, NULL); | |
| 257 | + if (!*ep) { | |
| 254 | 258 | DRM_DEV_ERROR(p2d->dev, |
| 255 | 259 | "failed to get available endpoint of port@%u\n", |
| 256 | 260 | port_id); |
| 257 | − ep = ERR_PTR(-ENODEV); | |
| 261 | + ret = -ENODEV; | |
| 258 | 262 | goto out; |
| 259 | 263 | } |
| 260 | 264 | out: |
| 261 | 265 | of_node_put(port); |
| 262 | − return ep; | |
| 266 | + return ret; | |
| 263 | 267 | } |
| 264 | 268 | |
| 265 | 269 | static int imx8qxp_pxl2dpi_find_next_bridge(struct imx8qxp_pxl2dpi *p2d) |
| 266 | 270 | { |
| 267 | − struct device_node *ep __free(device_node) = | |
| 268 | − imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1); | |
| 269 | − if (IS_ERR(ep)) | |
| 270 | − return PTR_ERR(ep); | |
| 271 | + struct device_node *ep __free(device_node) = NULL; | |
| 272 | + int ret; | |
| 273 | + | |
| 274 | + ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1, &ep); | |
| 275 | + if (ret) | |
| 276 | + return ret; | |
| 271 | 277 | |
| 272 | 278 | struct device_node *remote __free(device_node) = of_graph_get_remote_port_parent(ep); |
| 273 | 279 | if (!remote || !of_device_is_available(remote)) { |
| @@ -291,9 +297,9 @@ static int imx8qxp_pxl2dpi_set_pixel_link_sel(struct imx8qxp_pxl2dpi *p2d) · 17 unmodified lines | ||
| 291 | 297 | struct of_endpoint endpoint; |
| 292 | 298 | int ret; |
| 293 | 299 | |
| 294 | − ep = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0); | |
| 295 | − if (IS_ERR(ep)) | |
| 296 | − return PTR_ERR(ep); | |
| 300 | + ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0, &ep); | |
| 301 | + if (ret) | |
| 302 | + return ret; | |
| 297 | 303 | |
| 298 | 304 | ret = of_graph_parse_endpoint(ep, &endpoint); |
| 299 | 305 | if (ret) { |
| @@ -1559,6 +1559,11 @@ static int it66121_probe(struct i2c_client *client) · 1558 unmodified lines | ||
| 1559 | 1559 | return ret; |
| 1560 | 1560 | } |
| 1561 | 1561 | |
| 1562 | + ctx->gpio_reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW); | |
| 1563 | + if (IS_ERR(ctx->gpio_reset)) | |
| 1564 | + return dev_err_probe(dev, PTR_ERR(ctx->gpio_reset), | |
| 1565 | + "Failed to get reset GPIO\n"); | |
| 1566 | + | |
| 1562 | 1567 | it66121_hw_reset(ctx); |
| 1563 | 1568 | |
| 1564 | 1569 | ctx->regmap = devm_regmap_init_i2c(client, &it66121_regmap_config); |
| @@ -251,7 +251,6 @@ static void ge_b850v3_lvds_remove(void) · 250 unmodified lines | ||
| 251 | 251 | goto out; |
| 252 | 252 | |
| 253 | 253 | drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge); |
| 254 | − | |
| 255 | 254 | ge_b850v3_lvds_ptr = NULL; |
| 256 | 255 | out: |
| 257 | 256 | mutex_unlock(&ge_b850v3_lvds_dev_mutex); |
| @@ -261,6 +260,7 @@ static int ge_b850v3_register(void) · 3 unmodified lines | ||
| 261 | 260 | { |
| 262 | 261 | struct i2c_client *stdp4028_i2c = ge_b850v3_lvds_ptr->stdp4028_i2c; |
| 263 | 262 | struct device *dev = &stdp4028_i2c->dev; |
| 263 | + int ret; | |
| 264 | 264 | |
| 265 | 265 | /* drm bridge initialization */ |
| 266 | 266 | ge_b850v3_lvds_ptr->bridge.ops = DRM_BRIDGE_OP_DETECT | |
| @@ -277,11 +277,15 @@ static int ge_b850v3_register(void) · 10 unmodified lines | ||
| 277 | 277 | if (!stdp4028_i2c->irq) |
| 278 | 278 | return 0; |
| 279 | 279 | |
| 280 | − return devm_request_threaded_irq(&stdp4028_i2c->dev, | |
| 281 | − stdp4028_i2c->irq, NULL, | |
| 282 | − ge_b850v3_lvds_irq_handler, | |
| 283 | − IRQF_TRIGGER_HIGH | IRQF_ONESHOT, | |
| 284 | − "ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr); | |
| 280 | + ret = devm_request_threaded_irq(&stdp4028_i2c->dev, | |
| 281 | + stdp4028_i2c->irq, NULL, | |
| 282 | + ge_b850v3_lvds_irq_handler, | |
| 283 | + IRQF_TRIGGER_HIGH | IRQF_ONESHOT, | |
| 284 | + "ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr); | |
| 285 | + if (ret) | |
| 286 | + drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge); | |
| 287 | + | |
| 288 | + return ret; | |
| 285 | 289 | } |
| 286 | 290 | |
| 287 | 291 | static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c) |
| @@ -1293,7 +1293,7 @@ static const struct drm_edid *tda998x_edid_read(struct tda998x_priv *priv, · 1292 unmodified lines | ||
| 1293 | 1293 | * can't handle signals gracefully. |
| 1294 | 1294 | */ |
| 1295 | 1295 | if (tda998x_edid_delay_wait(priv)) |
| 1296 | − return 0; | |
| 1296 | + return NULL; | |
| 1297 | 1297 | |
| 1298 | 1298 | if (priv->rev == TDA19988) |
| 1299 | 1299 | reg_clear(priv, REG_TX4, TX4_PD_RAM); |
| @@ -1762,7 +1762,7 @@ static const struct drm_bridge_funcs tda998x_bridge_funcs = { · 462 unmodified lines | ||
| 1762 | 1762 | static int tda998x_get_audio_ports(struct tda998x_priv *priv, |
| 1763 | 1763 | struct device_node *np) |
| 1764 | 1764 | { |
| 1765 | − const u32 *port_data; | |
| 1765 | + const __be32 *port_data; | |
| 1766 | 1766 | u32 size; |
| 1767 | 1767 | int i; |
| 1768 | 1768 | |
| @@ -831,7 +831,7 @@ static void fill_palette_332(struct drm_crtc *crtc, u16 r, u16 g, u16 b, · 830 unmodified lines | ||
| 831 | 831 | } |
| 832 | 832 | |
| 833 | 833 | /** |
| 834 | − * drm_crtc_fill_palette_332 - Programs a default palette for R332-like formats | |
| 834 | + * drm_crtc_fill_palette_332 - Programs a default palette for RGB332-like formats | |
| 835 | 835 | * @crtc: The displaying CRTC |
| 836 | 836 | * @set_palette: Callback for programming the hardware gamma LUT |
| 837 | 837 | * |
| @@ -697,6 +697,7 @@ static void drm_dev_init_release(struct drm_device *dev, void *res) · 696 unmodified lines | ||
| 697 | 697 | mutex_destroy(&dev->master_mutex); |
| 698 | 698 | mutex_destroy(&dev->clientlist_mutex); |
| 699 | 699 | mutex_destroy(&dev->filelist_mutex); |
| 700 | + mutex_destroy(&dev->gem_lru_mutex); | |
| 700 | 701 | } |
| 701 | 702 | |
| 702 | 703 | static int drm_dev_init(struct drm_device *dev, |
| @@ -738,6 +739,7 @@ static int drm_dev_init(struct drm_device *dev, · 35 unmodified lines | ||
| 738 | 739 | INIT_LIST_HEAD(&dev->vblank_event_list); |
| 739 | 740 | |
| 740 | 741 | spin_lock_init(&dev->event_lock); |
| 742 | + mutex_init(&dev->gem_lru_mutex); | |
| 741 | 743 | mutex_init(&dev->filelist_mutex); |
| 742 | 744 | mutex_init(&dev->clientlist_mutex); |
| 743 | 745 | mutex_init(&dev->master_mutex); |
| @@ -70,8 +70,11 @@ static int drm_mode_align_dumb(struct drm_mode_create_dumb *args, · 69 unmodified lines | ||
| 70 | 70 | if (!pitch) |
| 71 | 71 | return -EINVAL; |
| 72 | 72 | |
| 73 | − if (hw_pitch_align) | |
| 73 | + if (hw_pitch_align) { | |
| 74 | 74 | pitch = roundup(pitch, hw_pitch_align); |
| 75 | + if (pitch < hw_pitch_align) | |
| 76 | + return -EINVAL; | |
| 77 | + } | |
| 75 | 78 | |
| 76 | 79 | if (!hw_size_align) |
| 77 | 80 | hw_size_align = PAGE_SIZE; |
| @@ -80,7 +83,7 @@ static int drm_mode_align_dumb(struct drm_mode_create_dumb *args, · 2 unmodified lines | ||
| 80 | 83 | |
| 81 | 84 | if (check_mul_overflow(args->height, pitch, &size)) |
| 82 | 85 | return -EINVAL; |
| 83 | − size = ALIGN(size, hw_size_align); | |
| 86 | + size = roundup(size, hw_size_align); | |
| 84 | 87 | if (!size) |
| 85 | 88 | return -EINVAL; |
| 86 | 89 | |
| @@ -199,6 +202,13 @@ int drm_mode_create_dumb(struct drm_device *dev, · 112 unmodified lines | ||
| 199 | 202 | if (!args->width || !args->height || !args->bpp) |
| 200 | 203 | return -EINVAL; |
| 201 | 204 | |
| 205 | + /* Reject unreasonable inputs early. Dumb buffers are for software | |
| 206 | + * rendering; nothing legitimate needs more than 8192x8192 at 32bpp. | |
| 207 | + * This prevents overflows in downstream alignment helpers. | |
| 208 | + */ | |
| 209 | + if (args->width >= 8192 || args->height >= 8192 || args->bpp > 32) | |
| 210 | + return -EINVAL; | |
| 211 | + | |
| 202 | 212 | /* overflow checks for 32bit size calculations */ |
| 203 | 213 | if (args->bpp > U32_MAX - 8) |
| 204 | 214 | return -EINVAL; |
| @@ -490,7 +490,7 @@ static void drm_fb_helper_memory_range_to_clip(struct fb_info *info, off_t off, · 489 unmodified lines | ||
| 490 | 490 | * the number of horizontal pixels that need an update. |
| 491 | 491 | */ |
| 492 | 492 | off_t bit_off = (off % line_length) * 8; |
| 493 | − off_t bit_end = (end % line_length) * 8; | |
| 493 | + off_t bit_end = bit_off + len * 8; | |
| 494 | 494 | |
| 495 | 495 | x1 = bit_off / info->var.bits_per_pixel; |
| 496 | 496 | x2 = DIV_ROUND_UP(bit_end, info->var.bits_per_pixel); |
| @@ -1019,7 +1019,7 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 1018 unmodified lines | ||
| 1019 | 1019 | struct drm_file *file_priv) |
| 1020 | 1020 | { |
| 1021 | 1021 | struct drm_gem_change_handle *args = data; |
| 1022 | − struct drm_gem_object *obj; | |
| 1022 | + struct drm_gem_object *obj, *idrobj; | |
| 1023 | 1023 | int handle, ret; |
| 1024 | 1024 | |
| 1025 | 1025 | if (!drm_core_check_feature(dev, DRIVER_GEM)) |
| @@ -1042,12 +1042,31 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 16 unmodified lines | ||
| 1042 | 1042 | mutex_lock(&file_priv->prime.lock); |
| 1043 | 1043 | |
| 1044 | 1044 | spin_lock(&file_priv->table_lock); |
| 1045 | + | |
| 1046 | + /* When create_tail allocs an obj idr, it needs to first alloc as NULL, | |
| 1047 | + * then later replace with the correct object. This is not necessary | |
| 1048 | + * here, because the only operations that could race are drm_prime | |
| 1049 | + * bookkeeping, and we hold the prime lock. | |
| 1050 | + */ | |
| 1045 | 1051 | ret = idr_alloc(&file_priv->object_idr, obj, handle, handle + 1, |
| 1046 | 1052 | GFP_NOWAIT); |
| 1047 | − spin_unlock(&file_priv->table_lock); | |
| 1048 | 1053 | |
| 1049 | − if (ret < 0) | |
| 1050 | − goto out_unlock; | |
| 1054 | + if (ret < 0) { | |
| 1055 | + spin_unlock(&file_priv->table_lock); | |
| 1056 | + goto out_unlock; | |
| 1057 | + } | |
| 1058 | + | |
| 1059 | + idrobj = idr_replace(&file_priv->object_idr, NULL, handle); | |
| 1060 | + if (idrobj != obj) { | |
| 1061 | + idr_replace(&file_priv->object_idr, idrobj, handle); | |
| 1062 | + idr_remove(&file_priv->object_idr, args->new_handle); | |
| 1063 | + spin_unlock(&file_priv->table_lock); | |
| 1064 | + ret = -ENOENT; | |
| 1065 | + goto out_unlock; | |
| 1066 | + } | |
| 1067 | + | |
| 1068 | + idr_replace(&file_priv->object_idr, NULL, args->handle); | |
| 1069 | + spin_unlock(&file_priv->table_lock); | |
| 1051 | 1070 | |
| 1052 | 1071 | if (obj->dma_buf) { |
| 1053 | 1072 | ret = drm_prime_add_buf_handle(&file_priv->prime, obj->dma_buf, |
| @@ -1055,6 +1074,7 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 1 unmodified line | ||
| 1055 | 1074 | if (ret < 0) { |
| 1056 | 1075 | spin_lock(&file_priv->table_lock); |
| 1057 | 1076 | idr_remove(&file_priv->object_idr, handle); |
| 1077 | + idr_replace(&file_priv->object_idr, obj, args->handle); | |
| 1058 | 1078 | spin_unlock(&file_priv->table_lock); |
| 1059 | 1079 | goto out_unlock; |
| 1060 | 1080 | } |
| @@ -1066,7 +1086,9 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data, · 5 unmodified lines | ||
| 1066 | 1086 | |
| 1067 | 1087 | spin_lock(&file_priv->table_lock); |
| 1068 | 1088 | idr_remove(&file_priv->object_idr, args->handle); |
| 1089 | + idrobj = idr_replace(&file_priv->object_idr, obj, handle); | |
| 1069 | 1090 | spin_unlock(&file_priv->table_lock); |
| 1091 | + WARN_ON(idrobj != NULL); | |
| 1070 | 1092 | |
| 1071 | 1093 | out_unlock: |
| 1072 | 1094 | mutex_unlock(&file_priv->prime.lock); |
| @@ -1541,12 +1563,10 @@ EXPORT_SYMBOL(drm_gem_unlock_reservations); · 468 unmodified lines | ||
| 1541 | 1563 | * drm_gem_lru_init - initialize a LRU |
| 1542 | 1564 | * |
| 1543 | 1565 | * @lru: The LRU to initialize |
| 1544 | − * @lock: The lock protecting the LRU | |
| 1545 | 1566 | */ |
| 1546 | 1567 | void |
| 1547 | −drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock) | |
| 1568 | +drm_gem_lru_init(struct drm_gem_lru *lru) | |
| 1548 | 1569 | { |
| 1549 | − lru->lock = lock; | |
| 1550 | 1570 | lru->count = 0; |
| 1551 | 1571 | INIT_LIST_HEAD(&lru->list); |
| 1552 | 1572 | } |
| @@ -1571,14 +1591,10 @@ drm_gem_lru_remove_locked(struct drm_gem_object *obj) · 18 unmodified lines | ||
| 1571 | 1591 | void |
| 1572 | 1592 | drm_gem_lru_remove(struct drm_gem_object *obj) |
| 1573 | 1593 | { |
| 1574 | − struct drm_gem_lru *lru = obj->lru; | |
| 1575 | − | |
| 1576 | − if (!lru) | |
| 1577 | − return; | |
| 1578 | − | |
| 1579 | − mutex_lock(lru->lock); | |
| 1580 | − drm_gem_lru_remove_locked(obj); | |
| 1581 | − mutex_unlock(lru->lock); | |
| 1594 | + mutex_lock(&obj->dev->gem_lru_mutex); | |
| 1595 | + if (obj->lru) | |
| 1596 | + drm_gem_lru_remove_locked(obj); | |
| 1597 | + mutex_unlock(&obj->dev->gem_lru_mutex); | |
| 1582 | 1598 | } |
| 1583 | 1599 | EXPORT_SYMBOL(drm_gem_lru_remove); |
| 1584 | 1600 | |
| @@ -1593,7 +1609,7 @@ EXPORT_SYMBOL(drm_gem_lru_remove); · 8 unmodified lines | ||
| 1593 | 1609 | void |
| 1594 | 1610 | drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj) |
| 1595 | 1611 | { |
| 1596 | − lockdep_assert_held_once(lru->lock); | |
| 1612 | + lockdep_assert_held_once(&obj->dev->gem_lru_mutex); | |
| 1597 | 1613 | |
| 1598 | 1614 | if (obj->lru) |
| 1599 | 1615 | drm_gem_lru_remove_locked(obj); |
| @@ -1617,9 +1633,9 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail_locked); · 17 unmodified lines | ||
| 1617 | 1633 | void |
| 1618 | 1634 | drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj) |
| 1619 | 1635 | { |
| 1620 | − mutex_lock(lru->lock); | |
| 1636 | + mutex_lock(&obj->dev->gem_lru_mutex); | |
| 1621 | 1637 | drm_gem_lru_move_tail_locked(lru, obj); |
| 1622 | − mutex_unlock(lru->lock); | |
| 1638 | + mutex_unlock(&obj->dev->gem_lru_mutex); | |
| 1623 | 1639 | } |
| 1624 | 1640 | EXPORT_SYMBOL(drm_gem_lru_move_tail); |
| 1625 | 1641 | |
| @@ -1633,6 +1649,7 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail); · 7 unmodified lines | ||
| 1633 | 1649 | * of the shrink callback to check for this (ie. dma_resv_test_signaled()) |
| 1634 | 1650 | * or if necessary block until the buffer becomes idle. |
| 1635 | 1651 | * |
| 1652 | + * @dev: DRM device the LRU belongs to | |
| 1636 | 1653 | * @lru: The LRU to scan |
| 1637 | 1654 | * @nr_to_scan: The number of pages to try to reclaim |
| 1638 | 1655 | * @remaining: The number of pages left to reclaim, should be initialized by caller |
| @@ -1640,7 +1657,8 @@ EXPORT_SYMBOL(drm_gem_lru_move_tail); · 1 unmodified line | ||
| 1640 | 1657 | * @ticket: Optional ww_acquire_ctx context to use for locking |
| 1641 | 1658 | */ |
| 1642 | 1659 | unsigned long |
| 1643 | −drm_gem_lru_scan(struct drm_gem_lru *lru, | |
| 1660 | +drm_gem_lru_scan(struct drm_device *dev, | |
| 1661 | + struct drm_gem_lru *lru, | |
| 1644 | 1662 | unsigned int nr_to_scan, |
| 1645 | 1663 | unsigned long *remaining, |
| 1646 | 1664 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket), |
| @@ -1650,9 +1668,9 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 3 unmodified lines | ||
| 1650 | 1668 | struct drm_gem_object *obj; |
| 1651 | 1669 | unsigned freed = 0; |
| 1652 | 1670 | |
| 1653 | − drm_gem_lru_init(&still_in_lru, lru->lock); | |
| 1671 | + drm_gem_lru_init(&still_in_lru); | |
| 1654 | 1672 | |
| 1655 | − mutex_lock(lru->lock); | |
| 1673 | + mutex_lock(&dev->gem_lru_mutex); | |
| 1656 | 1674 | |
| 1657 | 1675 | while (freed < nr_to_scan) { |
| 1658 | 1676 | obj = list_first_entry_or_null(&lru->list, typeof(*obj), lru_node); |
| @@ -1675,7 +1693,7 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 16 unmodified lines | ||
| 1675 | 1693 | * rest of the loop body, to reduce contention with other |
| 1676 | 1694 | * code paths that need the LRU lock |
| 1677 | 1695 | */ |
| 1678 | − mutex_unlock(lru->lock); | |
| 1696 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 1679 | 1697 | |
| 1680 | 1698 | if (ticket) |
| 1681 | 1699 | ww_acquire_init(ticket, &reservation_ww_class); |
| @@ -1709,7 +1727,7 @@ drm_gem_lru_scan(struct drm_gem_lru *lru, · 27 unmodified lines | ||
| 1709 | 1727 | |
| 1710 | 1728 | tail: |
| 1711 | 1729 | drm_gem_object_put(obj); |
| 1712 | − mutex_lock(lru->lock); | |
| 1730 | + mutex_lock(&dev->gem_lru_mutex); | |
| 1713 | 1731 | } |
| 1714 | 1732 | |
| 1715 | 1733 | /* |
| @@ -1721,7 +1739,7 @@ tail: · 5 unmodified lines | ||
| 1721 | 1739 | list_splice_tail(&still_in_lru.list, &lru->list); |
| 1722 | 1740 | lru->count += still_in_lru.count; |
| 1723 | 1741 | |
| 1724 | − mutex_unlock(lru->lock); | |
| 1742 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 1725 | 1743 | |
| 1726 | 1744 | return freed; |
| 1727 | 1745 | } |
| @@ -172,8 +172,8 @@ int drm_gem_fb_init_with_funcs(struct drm_device *dev, · 171 unmodified lines | ||
| 172 | 172 | } |
| 173 | 173 | |
| 174 | 174 | for (i = 0; i < info->num_planes; i++) { |
| 175 | − unsigned int width = mode_cmd->width / (i ? info->hsub : 1); | |
| 176 | − unsigned int height = mode_cmd->height / (i ? info->vsub : 1); | |
| 175 | + unsigned int width = drm_format_info_plane_width(info, mode_cmd->width, i); | |
| 176 | + unsigned int height = drm_format_info_plane_height(info, mode_cmd->height, i); | |
| 177 | 177 | unsigned int min_size; |
| 178 | 178 | |
| 179 | 179 | objs[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]); |
| @@ -116,16 +116,18 @@ int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit) · 115 unmodified lines | ||
| 116 | 116 | */ |
| 117 | 117 | mutex_lock(&gpu->sched_lock); |
| 118 | 118 | |
| 119 | + ret = xa_alloc_cyclic(&gpu->user_fences, &submit->out_fence_id, | |
| 120 | + NULL, xa_limit_32b, &gpu->next_user_fence, | |
| 121 | + GFP_KERNEL); | |
| 122 | + if (ret < 0) | |
| 123 | + goto out_unlock; | |
| 124 | + | |
| 119 | 125 | drm_sched_job_arm(&submit->sched_job); |
| 120 | 126 | |
| 121 | 127 | submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished); |
| 122 | − ret = xa_alloc_cyclic(&gpu->user_fences, &submit->out_fence_id, | |
| 123 | − submit->out_fence, xa_limit_32b, | |
| 124 | − &gpu->next_user_fence, GFP_KERNEL); | |
| 125 | − if (ret < 0) { | |
| 126 | − drm_sched_job_cleanup(&submit->sched_job); | |
| 127 | − goto out_unlock; | |
| 128 | − } | |
| 128 | + | |
| 129 | + xa_store(&gpu->user_fences, submit->out_fence_id, | |
| 130 | + submit->out_fence, GFP_KERNEL); | |
| 129 | 131 | |
| 130 | 132 | /* the scheduler holds on to the job now */ |
| 131 | 133 | kref_get(&submit->refcount); |
| @@ -423,7 +423,9 @@ static int exynos_mic_probe(struct platform_device *pdev) · 422 unmodified lines | ||
| 423 | 423 | |
| 424 | 424 | mic->bridge.of_node = dev->of_node; |
| 425 | 425 | |
| 426 | − drm_bridge_add(&mic->bridge); | |
| 426 | + ret = devm_drm_bridge_add(dev, &mic->bridge); | |
| 427 | + if (ret) | |
| 428 | + goto err; | |
| 427 | 429 | |
| 428 | 430 | pm_runtime_enable(dev); |
| 429 | 431 | |
| @@ -443,12 +445,8 @@ err: · 13 unmodified lines | ||
| 443 | 445 | |
| 444 | 446 | static void exynos_mic_remove(struct platform_device *pdev) |
| 445 | 447 | { |
| 446 | − struct exynos_mic *mic = platform_get_drvdata(pdev); | |
| 447 | − | |
| 448 | 448 | component_del(&pdev->dev, &exynos_mic_component_ops); |
| 449 | 449 | pm_runtime_disable(&pdev->dev); |
| 450 | − | |
| 451 | − drm_bridge_remove(&mic->bridge); | |
| 452 | 450 | } |
| 453 | 451 | |
| 454 | 452 | static const struct of_device_id exynos_mic_of_match[] = { |
| @@ -580,6 +580,7 @@ static int oaktrail_hdmi_get_modes(struct drm_connector *connector) · 579 unmodified lines | ||
| 580 | 580 | } else { |
| 581 | 581 | edid = (struct edid *)raw_edid; |
| 582 | 582 | /* FIXME ? edid = drm_get_edid(connector, i2c_adap); */ |
| 583 | + i2c_put_adapter(i2c_adap); | |
| 583 | 584 | } |
| 584 | 585 | |
| 585 | 586 | if (edid) { |
| @@ -293,7 +293,7 @@ void oaktrail_lvds_init(struct drm_device *dev, · 292 unmodified lines | ||
| 293 | 293 | { |
| 294 | 294 | struct gma_encoder *gma_encoder; |
| 295 | 295 | struct gma_connector *gma_connector; |
| 296 | − struct gma_i2c_chan *ddc_bus; | |
| 296 | + struct gma_i2c_chan *ddc_bus = NULL; | |
| 297 | 297 | struct drm_connector *connector; |
| 298 | 298 | struct drm_encoder *encoder; |
| 299 | 299 | struct drm_psb_private *dev_priv = to_drm_psb_private(dev); |
| @@ -367,6 +367,8 @@ void oaktrail_lvds_init(struct drm_device *dev, · 67 unmodified lines | ||
| 367 | 367 | if (edid == NULL && dev_priv->lpc_gpio_base) { |
| 368 | 368 | ddc_bus = oaktrail_lvds_i2c_init(dev); |
| 369 | 369 | if (!IS_ERR(ddc_bus)) { |
| 370 | + if (i2c_adap) | |
| 371 | + i2c_put_adapter(i2c_adap); | |
| 370 | 372 | i2c_adap = &ddc_bus->base; |
| 371 | 373 | edid = drm_get_edid(connector, i2c_adap); |
| 372 | 374 | } |
| @@ -421,7 +423,10 @@ out: · 48 unmodified lines | ||
| 421 | 423 | |
| 422 | 424 | err_unlock: |
| 423 | 425 | mutex_unlock(&dev->mode_config.mutex); |
| 424 | − gma_i2c_destroy(to_gma_i2c_chan(connector->ddc)); | |
| 426 | + if (!IS_ERR_OR_NULL(ddc_bus)) | |
| 427 | + gma_i2c_destroy(ddc_bus); | |
| 428 | + else if (i2c_adap) | |
| 429 | + i2c_put_adapter(i2c_adap); | |
| 425 | 430 | drm_encoder_cleanup(encoder); |
| 426 | 431 | err_connector_cleanup: |
| 427 | 432 | drm_connector_cleanup(connector); |
| @@ -391,8 +391,11 @@ static int hyperv_get_supported_resolution(struct hv_device *hdev) · 390 unmodified lines | ||
| 391 | 391 | return -ETIMEDOUT; |
| 392 | 392 | } |
| 393 | 393 | |
| 394 | − if (msg->resolution_resp.resolution_count == 0) { | |
| 395 | − drm_err(dev, "No supported resolutions\n"); | |
| 394 | + if (msg->resolution_resp.resolution_count == 0 || | |
| 395 | + msg->resolution_resp.resolution_count > | |
| 396 | + SYNTHVID_MAX_RESOLUTION_COUNT) { | |
| 397 | + drm_err(dev, "Invalid resolution count: %d\n", | |
| 398 | + msg->resolution_resp.resolution_count); | |
| 396 | 399 | return -ENODEV; |
| 397 | 400 | } |
| 398 | 401 | |
| @@ -417,30 +420,92 @@ static int hyperv_get_supported_resolution(struct hv_device *hdev) · 18 unmodified lines | ||
| 417 | 420 | return 0; |
| 418 | 421 | } |
| 419 | 422 | |
| 420 | −static void hyperv_receive_sub(struct hv_device *hdev) | |
| 423 | +static void hyperv_receive_sub(struct hv_device *hdev, u32 bytes_recvd) | |
| 421 | 424 | { |
| 422 | 425 | struct hyperv_drm_device *hv = hv_get_drvdata(hdev); |
| 423 | 426 | struct synthvid_msg *msg; |
| 427 | + size_t hdr_size; | |
| 428 | + size_t need; | |
| 424 | 429 | |
| 425 | 430 | if (!hv) |
| 426 | 431 | return; |
| 427 | 432 | |
| 428 | − msg = (struct synthvid_msg *)hv->recv_buf; | |
| 429 | − | |
| 430 | − /* Complete the wait event */ | |
| 431 | − if (msg->vid_hdr.type == SYNTHVID_VERSION_RESPONSE || | |
| 432 | − msg->vid_hdr.type == SYNTHVID_RESOLUTION_RESPONSE || | |
| 433 | − msg->vid_hdr.type == SYNTHVID_VRAM_LOCATION_ACK) { | |
| 434 | − memcpy(hv->init_buf, msg, VMBUS_MAX_PACKET_SIZE); | |
| 435 | − complete(&hv->wait); | |
| 433 | + hdr_size = sizeof(struct pipe_msg_hdr) + | |
| 434 | + sizeof(struct synthvid_msg_hdr); | |
| 435 | + if (bytes_recvd < hdr_size) { | |
| 436 | + drm_err_ratelimited(&hv->dev, | |
| 437 | + "synthvid packet too small for header: %u\n", | |
| 438 | + bytes_recvd); | |
| 436 | 439 | return; |
| 437 | 440 | } |
| 438 | 441 | |
| 439 | − if (msg->vid_hdr.type == SYNTHVID_FEATURE_CHANGE) { | |
| 442 | + msg = (struct synthvid_msg *)hv->recv_buf; | |
| 443 | + need = hdr_size; | |
| 444 | + | |
| 445 | + switch (msg->vid_hdr.type) { | |
| 446 | + case SYNTHVID_VERSION_RESPONSE: | |
| 447 | + need += sizeof(struct synthvid_version_resp); | |
| 448 | + break; | |
| 449 | + case SYNTHVID_RESOLUTION_RESPONSE: | |
| 450 | + /* | |
| 451 | + * The resolution response is variable length: the host | |
| 452 | + * fills resolution_count entries, not the full | |
| 453 | + * SYNTHVID_MAX_RESOLUTION_COUNT array. Require the fixed | |
| 454 | + * prefix first so resolution_count can be read, then | |
| 455 | + * demand exactly the count-sized array. | |
| 456 | + */ | |
| 457 | + need += offsetof(struct synthvid_supported_resolution_resp, | |
| 458 | + supported_resolution); | |
| 459 | + if (bytes_recvd < need) | |
| 460 | + break; | |
| 461 | + if (msg->resolution_resp.resolution_count > | |
| 462 | + SYNTHVID_MAX_RESOLUTION_COUNT) { | |
| 463 | + drm_err_ratelimited(&hv->dev, | |
| 464 | + "synthvid resolution count too large: %u\n", | |
| 465 | + msg->resolution_resp.resolution_count); | |
| 466 | + return; | |
| 467 | + } | |
| 468 | + need += msg->resolution_resp.resolution_count * | |
| 469 | + sizeof(struct hvd_screen_info); | |
| 470 | + break; | |
| 471 | + case SYNTHVID_VRAM_LOCATION_ACK: | |
| 472 | + need += sizeof(struct synthvid_vram_location_ack); | |
| 473 | + break; | |
| 474 | + case SYNTHVID_FEATURE_CHANGE: | |
| 475 | + /* | |
| 476 | + * Not a completion-driving message: validate its own payload | |
| 477 | + * and consume it here rather than falling through to the | |
| 478 | + * memcpy/complete shared by the wait-event responses. | |
| 479 | + */ | |
| 480 | + if (bytes_recvd < need + | |
| 481 | + sizeof(struct synthvid_feature_change)) { | |
| 482 | + drm_err_ratelimited(&hv->dev, | |
| 483 | + "synthvid feature change packet too small: %u\n", | |
| 484 | + bytes_recvd); | |
| 485 | + return; | |
| 486 | + } | |
| 440 | 487 | hv->dirt_needed = msg->feature_chg.is_dirt_needed; |
| 441 | 488 | if (hv->dirt_needed) |
| 442 | 489 | hyperv_hide_hw_ptr(hv->hdev); |
| 490 | + return; | |
| 491 | + default: | |
| 492 | + return; | |
| 493 | + } | |
| 494 | + | |
| 495 | + /* | |
| 496 | + * Shared completion path for the wait-event responses | |
| 497 | + * (VERSION_RESPONSE, RESOLUTION_RESPONSE, VRAM_LOCATION_ACK): | |
| 498 | + * require the type-specific payload before handing the buffer to | |
| 499 | + * the waiter. | |
| 500 | + */ | |
| 501 | + if (bytes_recvd < need) { | |
| 502 | + drm_err_ratelimited(&hv->dev, | |
| 503 | + "synthvid packet too small for type %u: %u < %zu\n", | |
| 504 | + msg->vid_hdr.type, bytes_recvd, need); | |
| 505 | + return; | |
| 443 | 506 | } |
| 507 | + memcpy(hv->init_buf, msg, bytes_recvd); | |
| 508 | + complete(&hv->wait); | |
| 444 | 509 | } |
| 445 | 510 | |
| 446 | 511 | static void hyperv_receive(void *ctx) |
| @@ -461,9 +526,21 @@ static void hyperv_receive(void *ctx) · 14 unmodified lines | ||
| 461 | 526 | ret = vmbus_recvpacket(hdev->channel, recv_buf, |
| 462 | 527 | VMBUS_MAX_PACKET_SIZE, |
| 463 | 528 | &bytes_recvd, &req_id); |
| 464 | − if (bytes_recvd > 0 && | |
| 465 | − recv_buf->pipe_hdr.type == PIPE_MSG_DATA) | |
| 466 | − hyperv_receive_sub(hdev); | |
| 529 | + if (ret) { | |
| 530 | + /* | |
| 531 | + * A nonzero return (e.g. -ENOBUFS for an oversized | |
| 532 | + * packet) is itself a malformed message: bytes_recvd | |
| 533 | + * then reports the required length rather than a copied | |
| 534 | + * payload, so it must not be forwarded to the | |
| 535 | + * sub-handler. Channel recovery is not attempted. | |
| 536 | + */ | |
| 537 | + drm_err_ratelimited(&hv->dev, | |
| 538 | + "vmbus_recvpacket failed: %d (need %u)\n", | |
| 539 | + ret, bytes_recvd); | |
| 540 | + } else if (bytes_recvd > 0 && | |
| 541 | + recv_buf->pipe_hdr.type == PIPE_MSG_DATA) { | |
| 542 | + hyperv_receive_sub(hdev, bytes_recvd); | |
| 543 | + } | |
| 467 | 544 | } while (bytes_recvd > 0 && ret == 0); |
| 468 | 545 | } |
| 469 | 546 | |
| @@ -508,9 +585,13 @@ int hyperv_connect_vsp(struct hv_device *hdev) · 38 unmodified lines | ||
| 508 | 585 | ret = hyperv_get_supported_resolution(hdev); |
| 509 | 586 | if (ret) |
| 510 | 587 | drm_err(dev, "Failed to get supported resolution from host, use default\n"); |
| 511 | − } else { | |
| 588 | + } | |
| 589 | + | |
| 590 | + if (!hv->screen_width_max) { | |
| 512 | 591 | hv->screen_width_max = SYNTHVID_WIDTH_WIN8; |
| 513 | 592 | hv->screen_height_max = SYNTHVID_HEIGHT_WIN8; |
| 593 | + hv->preferred_width = SYNTHVID_WIDTH_WIN8; | |
| 594 | + hv->preferred_height = SYNTHVID_HEIGHT_WIN8; | |
| 514 | 595 | } |
| 515 | 596 | |
| 516 | 597 | hv->mmio_megabytes = hdev->channel->offermsg.offer.mmio_megabytes; |
| @@ -3976,7 +3976,7 @@ xelpd_program_plane_pre_csc_lut(struct intel_dsb *dsb, · 3975 unmodified lines | ||
| 3976 | 3976 | intel_de_write_dsb(display, dsb, |
| 3977 | 3977 | PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0), |
| 3978 | 3978 | (1 << 24)); |
| 3979 | − } while (i++ > 130); | |
| 3979 | + } while (i++ < 130); | |
| 3980 | 3980 | } else { |
| 3981 | 3981 | for (i = 0; i < lut_size; i++) { |
| 3982 | 3982 | u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1); |
| @@ -497,6 +497,7 @@ struct intel_display { · 496 unmodified lines | ||
| 497 | 497 | u8 vblank_enabled; |
| 498 | 498 | |
| 499 | 499 | int vblank_enable_count; |
| 500 | + bool vblank_status_last_notified; | |
| 500 | 501 | |
| 501 | 502 | struct work_struct vblank_notify_work; |
| 502 | 503 | |
| @@ -1773,8 +1773,12 @@ static void intel_display_vblank_notify_work(struct work_struct *work) · 1772 unmodified lines | ||
| 1773 | 1773 | struct intel_display *display = |
| 1774 | 1774 | container_of(work, typeof(*display), irq.vblank_notify_work); |
| 1775 | 1775 | int vblank_enable_count = READ_ONCE(display->irq.vblank_enable_count); |
| 1776 | + bool vblank_status = !!vblank_enable_count; | |
| 1776 | 1777 | |
| 1777 | − intel_psr_notify_vblank_enable_disable(display, vblank_enable_count); | |
| 1778 | + if (display->irq.vblank_status_last_notified != vblank_status) { | |
| 1779 | + intel_psr_notify_vblank_enable_disable(display, vblank_status); | |
| 1780 | + display->irq.vblank_status_last_notified = vblank_status; | |
| 1781 | + } | |
| 1778 | 1782 | } |
| 1779 | 1783 | |
| 1780 | 1784 | int bdw_enable_vblank(struct drm_crtc *_crtc) |
| @@ -1787,10 +1791,10 @@ int bdw_enable_vblank(struct drm_crtc *_crtc) · 6 unmodified lines | ||
| 1787 | 1791 | if (gen11_dsi_configure_te(crtc, true)) |
| 1788 | 1792 | return 0; |
| 1789 | 1793 | |
| 1794 | + spin_lock_irqsave(&display->irq.lock, irqflags); | |
| 1790 | 1795 | if (crtc->vblank_psr_notify && display->irq.vblank_enable_count++ == 0) |
| 1791 | 1796 | schedule_work(&display->irq.vblank_notify_work); |
| 1792 | 1797 | |
| 1793 | − spin_lock_irqsave(&display->irq.lock, irqflags); | |
| 1794 | 1798 | bdw_enable_pipe_irq(display, pipe, GEN8_PIPE_VBLANK); |
| 1795 | 1799 | spin_unlock_irqrestore(&display->irq.lock, irqflags); |
| 1796 | 1800 | |
| @@ -584,6 +584,7 @@ struct intel_connector { · 583 unmodified lines | ||
| 584 | 584 | |
| 585 | 585 | struct { |
| 586 | 586 | u8 dpcd[EDP_PSR_RECEIVER_CAP_SIZE]; |
| 587 | + u8 intel_wa_dpcd; | |
| 587 | 588 | |
| 588 | 589 | bool support; |
| 589 | 590 | bool su_support; |
| @@ -1789,6 +1790,8 @@ struct intel_psr { · 1199 unmodified lines | ||
| 1789 | 1790 | u8 active_non_psr_pipes; |
| 1790 | 1791 | |
| 1791 | 1792 | const char *no_psr_reason; |
| 1793 | + | |
| 1794 | + struct ref_tracker *vblank_wakeref; | |
| 1792 | 1795 | }; |
| 1793 | 1796 | |
| 1794 | 1797 | struct intel_dp { |
| @@ -3119,8 +3119,13 @@ static void intel_dp_compute_vsc_colorimetry(const struct intel_crtc_state *crtc · 3118 unmodified lines | ||
| 3119 | 3119 | drm_WARN_ON(display->drm, |
| 3120 | 3120 | vsc->bpc == 6 && vsc->pixelformat != DP_PIXELFORMAT_RGB); |
| 3121 | 3121 | |
| 3122 | − /* all YCbCr are always limited range */ | |
| 3123 | − vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA; | |
| 3122 | + /* All YCbCr formats are always limited range. */ | |
| 3123 | + if (vsc->pixelformat == DP_PIXELFORMAT_RGB) | |
| 3124 | + vsc->dynamic_range = crtc_state->limited_color_range ? | |
| 3125 | + DP_DYNAMIC_RANGE_CTA : DP_DYNAMIC_RANGE_VESA; | |
| 3126 | + else | |
| 3127 | + vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA; | |
| 3128 | + | |
| 3124 | 3129 | vsc->content_type = DP_CONTENT_TYPE_NOT_DEFINED; |
| 3125 | 3130 | } |
| 3126 | 3131 | |
| @@ -5298,7 +5303,7 @@ int intel_dp_as_sdp_unpack(struct drm_dp_as_sdp *as_sdp, · 2171 unmodified lines | ||
| 5298 | 5303 | as_sdp->length = sdp->sdp_header.HB3 & DP_ADAPTIVE_SYNC_SDP_LENGTH; |
| 5299 | 5304 | as_sdp->mode = sdp->db[0] & DP_ADAPTIVE_SYNC_SDP_OPERATION_MODE; |
| 5300 | 5305 | as_sdp->vtotal = (sdp->db[2] << 8) | sdp->db[1]; |
| 5301 | − as_sdp->target_rr = (u64)sdp->db[3] | ((u64)sdp->db[4] & 0x3); | |
| 5306 | + as_sdp->target_rr = ((sdp->db[4] & 0x3) << 8) | sdp->db[3]; | |
| 5302 | 5307 | as_sdp->target_rr_divider = sdp->db[4] & 0x20 ? true : false; |
| 5303 | 5308 | |
| 5304 | 5309 | return 0; |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include "intel_dp.h" |
| 13 | 13 | #include "intel_dp_aux.h" |
| 14 | 14 | #include "intel_dp_aux_regs.h" |
| 15 | +#include "intel_parent.h" | |
| 15 | 16 | #include "intel_pps.h" |
| 16 | 17 | #include "intel_quirks.h" |
| 17 | 18 | #include "intel_tc.h" |
| @@ -60,18 +61,29 @@ intel_dp_aux_wait_done(struct intel_dp *intel_dp) · 42 unmodified lines | ||
| 60 | 61 | struct intel_display *display = to_intel_display(intel_dp); |
| 61 | 62 | i915_reg_t ch_ctl = intel_dp->aux_ch_ctl_reg(intel_dp); |
| 62 | 63 | const unsigned int timeout_ms = 10; |
| 64 | + bool done = true; | |
| 63 | 65 | u32 status; |
| 64 | − bool done; | |
| 66 | + int ret; | |
| 65 | 67 | |
| 68 | + if (intel_parent_irq_enabled(display)) { | |
| 66 | 69 | #define C (((status = intel_de_read_notrace(display, ch_ctl)) & DP_AUX_CH_CTL_SEND_BUSY) == 0) |
| 67 | − done = wait_event_timeout(display->gmbus.wait_queue, C, | |
| 68 | − msecs_to_jiffies_timeout(timeout_ms)); | |
| 70 | + done = wait_event_timeout(display->gmbus.wait_queue, C, | |
| 71 | + msecs_to_jiffies_timeout(timeout_ms)); | |
| 72 | + | |
| 73 | +#undef C | |
| 74 | + } else { | |
| 75 | + ret = intel_de_wait_ms(display, ch_ctl, | |
| 76 | + DP_AUX_CH_CTL_SEND_BUSY, 0, | |
| 77 | + timeout_ms, &status); | |
| 78 | + | |
| 79 | + if (ret == -ETIMEDOUT) | |
| 80 | + done = false; | |
| 81 | + } | |
| 69 | 82 | |
| 70 | 83 | if (!done) |
| 71 | 84 | drm_err(display->drm, |
| 72 | 85 | "%s: did not complete or timeout within %ums (status 0x%08x)\n", |
| 73 | 86 | intel_dp->aux.name, timeout_ms, status); |
| 74 | −#undef C | |
| 75 | 87 | |
| 76 | 88 | return status; |
| 77 | 89 | } |
| @@ -0,0 +1,15 @@ | ||
| 1 | +/* SPDX-License-Identifier: MIT */ | |
| 2 | +/* | |
| 3 | + * Copyright © 2026 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#ifndef __INTEL_DPCD_H__ | |
| 7 | +#define __INTEL_DPCD_H__ | |
| 8 | + | |
| 9 | +#define INTEL_DPCD_INTEL_WA_REGISTER_CAPS 0x3f0 | |
| 10 | +# define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_EARLYSCANLINE_SDP_SUPPORT_MASK REG_GENMASK(1, 0) | |
| 11 | +# define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_FALL_BACK_TO_PSR1 0 | |
| 12 | +# define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITH_EARLY_SCANLINE 1 | |
| 13 | +# define INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITHOUT_EARLY_SCANLINE 2 | |
| 14 | + | |
| 15 | +#endif /* __INTEL_DPCD_H__ */ | |
| @@ -373,7 +373,7 @@ intel_plane_color_copy_uapi_to_hw_state(struct intel_plane_state *plane_state, · 372 unmodified lines | ||
| 373 | 373 | bool changed = false; |
| 374 | 374 | int i = 0; |
| 375 | 375 | |
| 376 | − iter_colorop = plane_state->uapi.color_pipeline; | |
| 376 | + iter_colorop = from_plane_state->uapi.color_pipeline; | |
| 377 | 377 | |
| 378 | 378 | while (iter_colorop) { |
| 379 | 379 | for_each_new_colorop_in_state(state, colorop, new_colorop_state, i) { |
| @@ -43,6 +43,7 @@ · 42 unmodified lines | ||
| 43 | 43 | #include "intel_display_wa.h" |
| 44 | 44 | #include "intel_dmc.h" |
| 45 | 45 | #include "intel_dp.h" |
| 46 | +#include "intel_dpcd.h" | |
| 46 | 47 | #include "intel_dp_aux.h" |
| 47 | 48 | #include "intel_dsb.h" |
| 48 | 49 | #include "intel_frontbuffer.h" |
| @@ -716,8 +717,14 @@ static void _psr_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *co · 667 unmodified lines | ||
| 716 | 717 | connector->dp.psr_caps.su_support ? "" : "not "); |
| 717 | 718 | } |
| 718 | 719 | |
| 719 | − if (connector->dp.psr_caps.su_support) | |
| 720 | + if (connector->dp.psr_caps.su_support) { | |
| 721 | + ret = drm_dp_dpcd_read_byte(&intel_dp->aux, | |
| 722 | + INTEL_DPCD_INTEL_WA_REGISTER_CAPS, | |
| 723 | + &connector->dp.psr_caps.intel_wa_dpcd); | |
| 724 | + if (ret < 0) | |
| 725 | + return; | |
| 720 | 726 | _psr_compute_su_granularity(intel_dp, connector); |
| 727 | + } | |
| 721 | 728 | } |
| 722 | 729 | |
| 723 | 730 | void intel_psr_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) |
| @@ -1358,9 +1365,35 @@ static bool psr2_granularity_check(struct intel_crtc_state *crtc_state, · 634 unmodified lines | ||
| 1358 | 1365 | return true; |
| 1359 | 1366 | } |
| 1360 | 1367 | |
| 1361 | −static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_dp *intel_dp, | |
| 1362 | − struct intel_crtc_state *crtc_state) | |
| 1368 | +static bool apply_scanline_indication_wa(struct intel_crtc_state *crtc_state, | |
| 1369 | + struct intel_connector *connector) | |
| 1363 | 1370 | { |
| 1371 | + struct intel_dp *intel_dp = intel_attached_dp(connector); | |
| 1372 | + u8 early_scanline_support = connector->dp.psr_caps.intel_wa_dpcd & | |
| 1373 | + INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_EARLYSCANLINE_SDP_SUPPORT_MASK; | |
| 1374 | + | |
| 1375 | + if (intel_dp->edp_dpcd[0] >= DP_EDP_15) | |
| 1376 | + return true; | |
| 1377 | + | |
| 1378 | + switch (early_scanline_support) { | |
| 1379 | + case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_FALL_BACK_TO_PSR1: | |
| 1380 | + crtc_state->req_psr2_sdp_prior_scanline = false; | |
| 1381 | + return false; | |
| 1382 | + case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITH_EARLY_SCANLINE: | |
| 1383 | + return true; | |
| 1384 | + case INTEL_DPCD_INTEL_WA_REGISTER_CAPS_PSR2_WITHOUT_EARLY_SCANLINE: | |
| 1385 | + crtc_state->req_psr2_sdp_prior_scanline = false; | |
| 1386 | + return true; | |
| 1387 | + default: | |
| 1388 | + MISSING_CASE(early_scanline_support); | |
| 1389 | + return false; | |
| 1390 | + } | |
| 1391 | +} | |
| 1392 | + | |
| 1393 | +static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_crtc_state *crtc_state, | |
| 1394 | + struct intel_connector *connector) | |
| 1395 | +{ | |
| 1396 | + struct intel_dp *intel_dp = intel_attached_dp(connector); | |
| 1364 | 1397 | struct intel_display *display = to_intel_display(intel_dp); |
| 1365 | 1398 | const struct drm_display_mode *adjusted_mode = &crtc_state->uapi.adjusted_mode; |
| 1366 | 1399 | u32 hblank_total, hblank_ns, req_ns; |
| @@ -1379,7 +1412,8 @@ static bool _compute_psr2_sdp_prior_scanline_indication(struct intel_dp *intel_d · 12 unmodified lines | ||
| 1379 | 1412 | return false; |
| 1380 | 1413 | |
| 1381 | 1414 | crtc_state->req_psr2_sdp_prior_scanline = true; |
| 1382 | − return true; | |
| 1415 | + | |
| 1416 | + return apply_scanline_indication_wa(crtc_state, connector); | |
| 1383 | 1417 | } |
| 1384 | 1418 | |
| 1385 | 1419 | static int intel_psr_entry_setup_frames(struct intel_dp *intel_dp, |
| @@ -1660,7 +1694,7 @@ static bool intel_sel_update_config_valid(struct intel_crtc_state *crtc_state, · 274 unmodified lines | ||
| 1660 | 1694 | conn_state)) |
| 1661 | 1695 | goto unsupported; |
| 1662 | 1696 | |
| 1663 | − if (!_compute_psr2_sdp_prior_scanline_indication(intel_dp, crtc_state)) { | |
| 1697 | + if (!_compute_psr2_sdp_prior_scanline_indication(crtc_state, connector)) { | |
| 1664 | 1698 | drm_dbg_kms(display->drm, |
| 1665 | 1699 | "Selective update not enabled, SDP indication do not fit in hblank\n"); |
| 1666 | 1700 | goto unsupported; |
| @@ -4122,27 +4156,22 @@ void intel_psr_notify_vblank_enable_disable(struct intel_display *display, · 2455 unmodified lines | ||
| 4122 | 4156 | struct intel_dp *intel_dp = enc_to_intel_dp(encoder); |
| 4123 | 4157 | |
| 4124 | 4158 | mutex_lock(&intel_dp->psr.lock); |
| 4125 | − if (intel_dp->psr.panel_replay_enabled) { | |
| 4126 | − mutex_unlock(&intel_dp->psr.lock); | |
| 4127 | − break; | |
| 4159 | + if (CAN_PANEL_REPLAY(intel_dp)) { | |
| 4160 | + if (enable) | |
| 4161 | + intel_dp->psr.vblank_wakeref = | |
| 4162 | + intel_display_power_get(display, | |
| 4163 | + POWER_DOMAIN_DC_OFF); | |
| 4164 | + else | |
| 4165 | + intel_display_power_put(display, POWER_DOMAIN_DC_OFF, | |
| 4166 | + intel_dp->psr.vblank_wakeref); | |
| 4128 | 4167 | } |
| 4129 | 4168 | |
| 4130 | − if (intel_dp->psr.enabled && intel_dp->psr.pkg_c_latency_used) | |
| 4169 | + if (intel_dp->psr.enabled && !intel_dp->psr.panel_replay_enabled && | |
| 4170 | + intel_dp->psr.pkg_c_latency_used) | |
| 4131 | 4171 | intel_psr_apply_underrun_on_idle_wa_locked(intel_dp); |
| 4132 | 4172 | |
| 4133 | 4173 | mutex_unlock(&intel_dp->psr.lock); |
| 4134 | − return; | |
| 4135 | 4174 | } |
| 4136 | − | |
| 4137 | − /* | |
| 4138 | − * NOTE: intel_display_power_set_target_dc_state is used | |
| 4139 | − * only by PSR * code for DC3CO handling. DC3CO target | |
| 4140 | − * state is currently disabled in * PSR code. If DC3CO | |
| 4141 | − * is taken into use we need take that into account here | |
| 4142 | − * as well. | |
| 4143 | − */ | |
| 4144 | − intel_display_power_set_target_dc_state(display, enable ? DC_STATE_DISABLE : | |
| 4145 | − DC_STATE_EN_UPTO_DC6); | |
| 4146 | 4175 | } |
| 4147 | 4176 | |
| 4148 | 4177 | static void |
| @@ -419,8 +419,6 @@ void i915_ttm_free_cached_io_rsgt(struct drm_i915_gem_object *obj) · 418 unmodified lines | ||
| 419 | 419 | int i915_ttm_purge(struct drm_i915_gem_object *obj) |
| 420 | 420 | { |
| 421 | 421 | struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); |
| 422 | − struct i915_ttm_tt *i915_tt = | |
| 423 | − container_of(bo->ttm, typeof(*i915_tt), ttm); | |
| 424 | 422 | struct ttm_operation_ctx ctx = { |
| 425 | 423 | .interruptible = true, |
| 426 | 424 | .no_wait_gpu = false, |
| @@ -435,16 +433,22 @@ int i915_ttm_purge(struct drm_i915_gem_object *obj) · 8 unmodified lines | ||
| 435 | 433 | if (ret) |
| 436 | 434 | return ret; |
| 437 | 435 | |
| 438 | − if (bo->ttm && i915_tt->filp) { | |
| 439 | − /* | |
| 440 | − * The below fput(which eventually calls shmem_truncate) might | |
| 441 | − * be delayed by worker, so when directly called to purge the | |
| 442 | − * pages(like by the shrinker) we should try to be more | |
| 443 | − * aggressive and release the pages immediately. | |
| 444 | − */ | |
| 445 | − shmem_truncate_range(file_inode(i915_tt->filp), | |
| 446 | − 0, (loff_t)-1); | |
| 447 | − fput(fetch_and_zero(&i915_tt->filp)); | |
| 436 | + if (bo->ttm) { | |
| 437 | + struct i915_ttm_tt *i915_tt = | |
| 438 | + container_of(bo->ttm, typeof(*i915_tt), ttm); | |
| 439 | + | |
| 440 | + if (i915_tt->filp) { | |
| 441 | + /* | |
| 442 | + * The below fput(which eventually calls shmem_truncate) | |
| 443 | + * might be delayed by worker, so when directly called | |
| 444 | + * to purge the pages(like by the shrinker) we should | |
| 445 | + * try to be more aggressive and release the pages | |
| 446 | + * immediately. | |
| 447 | + */ | |
| 448 | + shmem_truncate_range(file_inode(i915_tt->filp), | |
| 449 | + 0, (loff_t)-1); | |
| 450 | + fput(fetch_and_zero(&i915_tt->filp)); | |
| 451 | + } | |
| 448 | 452 | } |
| 449 | 453 | |
| 450 | 454 | obj->write_domain = 0; |
| @@ -132,7 +132,8 @@ void __i915_request_reset(struct i915_request *rq, bool guilty) · 131 unmodified lines | ||
| 132 | 132 | rcu_read_lock(); /* protect the GEM context */ |
| 133 | 133 | if (guilty) { |
| 134 | 134 | i915_request_set_error_once(rq, -EIO); |
| 135 | − __i915_request_skip(rq); | |
| 135 | + if (!i915_request_signaled(rq)) | |
| 136 | + __i915_request_skip(rq); | |
| 136 | 137 | banned = mark_guilty(rq); |
| 137 | 138 | } else { |
| 138 | 139 | i915_request_set_error_once(rq, -EAGAIN); |
| @@ -750,9 +750,8 @@ static bool has_auxccs(struct drm_device *drm) · 749 unmodified lines | ||
| 750 | 750 | { |
| 751 | 751 | struct drm_i915_private *i915 = to_i915(drm); |
| 752 | 752 | |
| 753 | − return IS_GRAPHICS_VER(i915, 9, 12) || | |
| 754 | − IS_ALDERLAKE_P(i915) || | |
| 755 | − IS_METEORLAKE(i915); | |
| 753 | + return IS_GRAPHICS_VER(i915, 9, 12) && | |
| 754 | + !HAS_FLAT_CCS(i915); | |
| 756 | 755 | } |
| 757 | 756 | |
| 758 | 757 | static bool has_fenced_regions(struct drm_device *drm) |
| @@ -558,6 +558,6 @@ pvr_fw_trace_debugfs_init(struct pvr_device *pvr_dev, struct dentry *dir) · 557 unmodified lines | ||
| 558 | 558 | &pvr_fw_trace_fops); |
| 559 | 559 | } |
| 560 | 560 | |
| 561 | − debugfs_create_file("trace_mask", 0600, dir, fw_trace, | |
| 561 | + debugfs_create_file("trace_mask", 0600, dir, pvr_dev, | |
| 562 | 562 | &pvr_fw_trace_mask_fops); |
| 563 | 563 | } |
| @@ -293,7 +293,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 292 unmodified lines | ||
| 293 | 293 | |
| 294 | 294 | vga_client_register(pdev, lsdc_vga_set_decode); |
| 295 | 295 | |
| 296 | − drm_kms_helper_poll_init(ddev); | |
| 296 | + drmm_kms_helper_poll_init(ddev); | |
| 297 | 297 | |
| 298 | 298 | if (loongson_vblank) { |
| 299 | 299 | ret = drm_vblank_init(ddev, descp->num_of_crtc); |
| @@ -240,7 +240,7 @@ static const struct of_device_id mtk_cec_of_ids[] = { · 239 unmodified lines | ||
| 240 | 240 | }; |
| 241 | 241 | MODULE_DEVICE_TABLE(of, mtk_cec_of_ids); |
| 242 | 242 | |
| 243 | −struct platform_driver mtk_cec_driver = { | |
| 243 | +static struct platform_driver mtk_cec_driver = { | |
| 244 | 244 | .probe = mtk_cec_probe, |
| 245 | 245 | .remove = mtk_cec_remove, |
| 246 | 246 | .driver = { |
| @@ -328,7 +328,7 @@ static const struct of_device_id mtk_hdmi_ddc_match[] = { · 327 unmodified lines | ||
| 328 | 328 | }; |
| 329 | 329 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_match); |
| 330 | 330 | |
| 331 | −struct platform_driver mtk_hdmi_ddc_driver = { | |
| 331 | +static struct platform_driver mtk_hdmi_ddc_driver = { | |
| 332 | 332 | .probe = mtk_hdmi_ddc_probe, |
| 333 | 333 | .remove = mtk_hdmi_ddc_remove, |
| 334 | 334 | .driver = { |
| @@ -389,7 +389,7 @@ static const struct of_device_id mtk_hdmi_ddc_v2_match[] = { · 388 unmodified lines | ||
| 389 | 389 | }; |
| 390 | 390 | MODULE_DEVICE_TABLE(of, mtk_hdmi_ddc_v2_match); |
| 391 | 391 | |
| 392 | −struct platform_driver mtk_hdmi_ddc_v2_driver = { | |
| 392 | +static struct platform_driver mtk_hdmi_ddc_v2_driver = { | |
| 393 | 393 | .probe = mtk_hdmi_ddc_v2_probe, |
| 394 | 394 | .driver = { |
| 395 | 395 | .name = "mediatek-hdmi-ddc-v2", |
| @@ -50,7 +50,7 @@ enum mtk_hdmi_v2_clk_id { · 49 unmodified lines | ||
| 50 | 50 | MTK_HDMI_V2_CLK_COUNT, |
| 51 | 51 | }; |
| 52 | 52 | |
| 53 | −const char *const mtk_hdmi_v2_clk_names[MTK_HDMI_V2_CLK_COUNT] = { | |
| 53 | +static const char *const mtk_hdmi_v2_clk_names[MTK_HDMI_V2_CLK_COUNT] = { | |
| 54 | 54 | [MTK_HDMI_V2_CLK_HDMI_APB_SEL] = "bus", |
| 55 | 55 | [MTK_HDMI_V2_CLK_HDCP_SEL] = "hdcp", |
| 56 | 56 | [MTK_HDMI_V2_CLK_HDCP_24M_SEL] = "hdcp24m", |
| @@ -2621,7 +2621,6 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 2620 unmodified lines | ||
| 2621 | 2621 | struct platform_device *pdev = priv->gpu_pdev; |
| 2622 | 2622 | struct adreno_platform_config *config = pdev->dev.platform_data; |
| 2623 | 2623 | const struct adreno_info *info = config->info; |
| 2624 | − struct device_node *node; | |
| 2625 | 2624 | struct a6xx_gpu *a6xx_gpu; |
| 2626 | 2625 | struct adreno_gpu *adreno_gpu; |
| 2627 | 2626 | struct msm_gpu *gpu; |
| @@ -2643,7 +2642,8 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 15 unmodified lines | ||
| 2643 | 2642 | adreno_gpu->registers = NULL; |
| 2644 | 2643 | |
| 2645 | 2644 | /* Check if there is a GMU phandle and set it up */ |
| 2646 | − node = of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0); | |
| 2645 | + struct device_node *node __free(device_node) = | |
| 2646 | + of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0); | |
| 2647 | 2647 | /* FIXME: How do we gracefully handle this? */ |
| 2648 | 2648 | BUG_ON(!node); |
| 2649 | 2649 | |
| @@ -2690,7 +2690,6 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) · 40 unmodified lines | ||
| 2690 | 2690 | ret = a6xx_gmu_wrapper_init(a6xx_gpu, node); |
| 2691 | 2691 | else |
| 2692 | 2692 | ret = a6xx_gmu_init(a6xx_gpu, node); |
| 2693 | − of_node_put(node); | |
| 2694 | 2693 | if (ret) { |
| 2695 | 2694 | a6xx_destroy(&(a6xx_gpu->base.base)); |
| 2696 | 2695 | return ERR_PTR(ret); |
| @@ -2740,6 +2739,7 @@ const struct adreno_gpu_funcs a6xx_gpu_funcs = { · 43 unmodified lines | ||
| 2740 | 2739 | .create_private_vm = a6xx_create_private_vm, |
| 2741 | 2740 | .get_rptr = a6xx_get_rptr, |
| 2742 | 2741 | .progress = a6xx_progress, |
| 2742 | + .sysprof_setup = a6xx_gmu_sysprof_setup, | |
| 2743 | 2743 | }, |
| 2744 | 2744 | .init = a6xx_gpu_init, |
| 2745 | 2745 | .get_timestamp = a6xx_gmu_get_timestamp, |
| @@ -2808,6 +2808,7 @@ const struct adreno_gpu_funcs a7xx_gpu_funcs = { · 62 unmodified lines | ||
| 2808 | 2808 | .create_private_vm = a6xx_create_private_vm, |
| 2809 | 2809 | .get_rptr = a6xx_get_rptr, |
| 2810 | 2810 | .progress = a6xx_progress, |
| 2811 | + .sysprof_setup = a6xx_gmu_sysprof_setup, | |
| 2811 | 2812 | }, |
| 2812 | 2813 | .init = a6xx_gpu_init, |
| 2813 | 2814 | .get_timestamp = a6xx_gmu_get_timestamp, |
| @@ -289,6 +289,8 @@ static int a8xx_hfi_send_perf_table(struct a6xx_gmu *gmu) · 288 unmodified lines | ||
| 289 | 289 | (gmu->nr_gpu_freqs * num_gx_votes * sizeof(gmu->gx_arc_votes[0])) + |
| 290 | 290 | (gmu->nr_gmu_freqs * num_cx_votes * sizeof(gmu->cx_arc_votes[0])); |
| 291 | 291 | tbl = kzalloc(size, GFP_KERNEL); |
| 292 | + if (!tbl) | |
| 293 | + return -ENOMEM; | |
| 292 | 294 | tbl->type = HFI_TABLE_GPU_PERF; |
| 293 | 295 | |
| 294 | 296 | /* First fill GX votes */ |
| @@ -17,7 +17,7 @@ MODULE_PARM_DESC(snapshot_debugbus, "Include debugbus sections in GPU devcoredum · 16 unmodified lines | ||
| 17 | 17 | module_param_named(snapshot_debugbus, snapshot_debugbus, bool, 0600); |
| 18 | 18 | |
| 19 | 19 | int enable_preemption = -1; |
| 20 | −MODULE_PARM_DESC(enable_preemption, "Enable preemption (A7xx only) (1=on , 0=disable, -1=auto (default))"); | |
| 20 | +MODULE_PARM_DESC(enable_preemption, "Enable preemption (A7xx+ only) (1=on , 0=disable, -1=auto (default))"); | |
| 21 | 21 | module_param(enable_preemption, int, 0600); |
| 22 | 22 | |
| 23 | 23 | bool disable_acd; |
| @@ -376,7 +376,7 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, · 375 unmodified lines | ||
| 376 | 376 | *value = adreno_gpu->info->gmem; |
| 377 | 377 | return 0; |
| 378 | 378 | case MSM_PARAM_GMEM_BASE: |
| 379 | − if (adreno_gpu->info->family >= ADRENO_6XX_GEN4) | |
| 379 | + if (adreno_gpu->info->family >= ADRENO_6XX_GEN3) | |
| 380 | 380 | *value = 0; |
| 381 | 381 | else |
| 382 | 382 | *value = 0x100000; |
| @@ -424,15 +424,21 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, · 41 unmodified lines | ||
| 424 | 424 | *value = vm->mm_range; |
| 425 | 425 | return 0; |
| 426 | 426 | case MSM_PARAM_HIGHEST_BANK_BIT: |
| 427 | + if (!adreno_gpu->ubwc_config) | |
| 428 | + return UERR(ENOENT, drm, "no UBWC on this platform"); | |
| 427 | 429 | *value = adreno_gpu->ubwc_config->highest_bank_bit; |
| 428 | 430 | return 0; |
| 429 | 431 | case MSM_PARAM_RAYTRACING: |
| 430 | 432 | *value = adreno_gpu->has_ray_tracing; |
| 431 | 433 | return 0; |
| 432 | 434 | case MSM_PARAM_UBWC_SWIZZLE: |
| 435 | + if (!adreno_gpu->ubwc_config) | |
| 436 | + return UERR(ENOENT, drm, "no UBWC on this platform"); | |
| 433 | 437 | *value = adreno_gpu->ubwc_config->ubwc_swizzle; |
| 434 | 438 | return 0; |
| 435 | 439 | case MSM_PARAM_MACROTILE_MODE: |
| 440 | + if (!adreno_gpu->ubwc_config) | |
| 441 | + return UERR(ENOENT, drm, "no UBWC on this platform"); | |
| 436 | 442 | *value = adreno_gpu->ubwc_config->macrotile_mode; |
| 437 | 443 | return 0; |
| 438 | 444 | case MSM_PARAM_UCHE_TRAP_BASE: |
| @@ -480,7 +480,7 @@ const struct dpu_mdss_cfg dpu_kaanapali_cfg = { · 479 unmodified lines | ||
| 480 | 480 | .wb_count = ARRAY_SIZE(kaanapali_wb), |
| 481 | 481 | .wb = kaanapali_wb, |
| 482 | 482 | .cwb_count = ARRAY_SIZE(kaanapali_cwb), |
| 483 | − .cwb = sm8650_cwb, | |
| 483 | + .cwb = kaanapali_cwb, | |
| 484 | 484 | .intf_count = ARRAY_SIZE(kaanapali_intf), |
| 485 | 485 | .intf = kaanapali_intf, |
| 486 | 486 | .vbif = &sm8650_vbif, |
| @@ -61,7 +61,7 @@ static int _dpu_format_populate_plane_sizes_ubwc( · 60 unmodified lines | ||
| 61 | 61 | bool meta = MSM_FORMAT_IS_UBWC(fmt); |
| 62 | 62 | |
| 63 | 63 | if (MSM_FORMAT_IS_YUV(fmt)) { |
| 64 | − unsigned int stride, sclines; | |
| 64 | + unsigned int stride, y_sclines, uv_sclines; | |
| 65 | 65 | unsigned int y_tile_width, y_tile_height; |
| 66 | 66 | unsigned int y_meta_stride, y_meta_scanlines; |
| 67 | 67 | unsigned int uv_meta_stride, uv_meta_scanlines; |
| @@ -77,23 +77,25 @@ static int _dpu_format_populate_plane_sizes_ubwc( · 9 unmodified lines | ||
| 77 | 77 | y_tile_width = 32; |
| 78 | 78 | } |
| 79 | 79 | |
| 80 | − sclines = round_up(fb->height, 16); | |
| 80 | + y_sclines = round_up(fb->height, 16); | |
| 81 | + uv_sclines = round_up((fb->height+1)>>1, 16); | |
| 81 | 82 | y_tile_height = 4; |
| 82 | 83 | } else { |
| 83 | 84 | stride = round_up(fb->width, 128); |
| 84 | 85 | y_tile_width = 32; |
| 85 | 86 | |
| 86 | − sclines = round_up(fb->height, 32); | |
| 87 | + y_sclines = round_up(fb->height, 32); | |
| 88 | + uv_sclines = round_up((fb->height+1)>>1, 32); | |
| 87 | 89 | y_tile_height = 8; |
| 88 | 90 | } |
| 89 | 91 | |
| 90 | 92 | layout->plane_pitch[0] = stride; |
| 91 | 93 | layout->plane_size[0] = round_up(layout->plane_pitch[0] * |
| 92 | − sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | |
| 94 | + y_sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | |
| 93 | 95 | |
| 94 | 96 | layout->plane_pitch[1] = stride; |
| 95 | 97 | layout->plane_size[1] = round_up(layout->plane_pitch[1] * |
| 96 | − sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | |
| 98 | + uv_sclines, DPU_UBWC_PLANE_SIZE_ALIGNMENT); | |
| 97 | 99 | |
| 98 | 100 | if (!meta) |
| 99 | 101 | return 0; |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include <drm/drm_edid.h> |
| 7 | 7 | #include <drm/drm_framebuffer.h> |
| 8 | +#include <drm/drm_managed.h> | |
| 8 | 9 | |
| 9 | 10 | #include "dpu_writeback.h" |
| 10 | 11 | |
| @@ -125,7 +126,7 @@ int dpu_writeback_init(struct drm_device *dev, struct drm_encoder *enc, · 114 unmodified lines | ||
| 125 | 126 | struct dpu_wb_connector *dpu_wb_conn; |
| 126 | 127 | int rc = 0; |
| 127 | 128 | |
| 128 | − dpu_wb_conn = devm_kzalloc(dev->dev, sizeof(*dpu_wb_conn), GFP_KERNEL); | |
| 129 | + dpu_wb_conn = drmm_kzalloc(dev, sizeof(*dpu_wb_conn), GFP_KERNEL); | |
| 129 | 130 | if (!dpu_wb_conn) |
| 130 | 131 | return -ENOMEM; |
| 131 | 132 | |
| @@ -5,11 +5,11 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #define pr_fmt(fmt) "[drm:%s:%d] " fmt, __func__, __LINE__ |
| 7 | 7 | |
| 8 | −#include <generated/utsrelease.h> | |
| 8 | +#include <linux/utsname.h> | |
| 9 | 9 | |
| 10 | 10 | #include "msm_disp_snapshot.h" |
| 11 | 11 | |
| 12 | −static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *base_addr) | |
| 12 | +static void msm_disp_state_dump_regs(u32 **reg, u32 len, void __iomem *base_addr) | |
| 13 | 13 | { |
| 14 | 14 | u32 len_padded; |
| 15 | 15 | u32 num_rows; |
| @@ -19,11 +19,11 @@ static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *b · 3 unmodified lines | ||
| 19 | 19 | void __iomem *end_addr; |
| 20 | 20 | int i; |
| 21 | 21 | |
| 22 | − len_padded = aligned_len * REG_DUMP_ALIGN; | |
| 23 | − num_rows = aligned_len / REG_DUMP_ALIGN; | |
| 22 | + len_padded = round_up(len, REG_DUMP_ALIGN); | |
| 23 | + num_rows = DIV_ROUND_UP(len, REG_DUMP_ALIGN); | |
| 24 | 24 | |
| 25 | 25 | addr = base_addr; |
| 26 | − end_addr = base_addr + aligned_len; | |
| 26 | + end_addr = base_addr + len; | |
| 27 | 27 | |
| 28 | 28 | *reg = kvzalloc(len_padded, GFP_KERNEL); |
| 29 | 29 | if (!*reg) |
| @@ -48,8 +48,8 @@ static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *b · 18 unmodified lines | ||
| 48 | 48 | static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, |
| 49 | 49 | void __iomem *base_addr, struct drm_printer *p) |
| 50 | 50 | { |
| 51 | + void __iomem *addr, *end_addr; | |
| 51 | 52 | int i; |
| 52 | − void __iomem *addr; | |
| 53 | 53 | u32 num_rows; |
| 54 | 54 | |
| 55 | 55 | if (!dump_addr) { |
| @@ -58,6 +58,7 @@ static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, · 2 unmodified lines | ||
| 58 | 58 | } |
| 59 | 59 | |
| 60 | 60 | addr = base_addr; |
| 61 | + end_addr = base_addr + len; | |
| 61 | 62 | num_rows = len / REG_DUMP_ALIGN; |
| 62 | 63 | |
| 63 | 64 | for (i = 0; i < num_rows; i++) { |
| @@ -67,6 +68,17 @@ static void msm_disp_state_print_regs(const u32 *dump_addr, u32 len, · 3 unmodified lines | ||
| 67 | 68 | dump_addr[i * 4 + 2], dump_addr[i * 4 + 3]); |
| 68 | 69 | addr += REG_DUMP_ALIGN; |
| 69 | 70 | } |
| 71 | + | |
| 72 | + if (addr != end_addr) { | |
| 73 | + drm_printf(p, "0x%lx : %08x", | |
| 74 | + (unsigned long)(addr - base_addr), | |
| 75 | + dump_addr[i * 4]); | |
| 76 | + if (addr + 0x4 < end_addr) | |
| 77 | + drm_printf(p, " %08x", dump_addr[i * 4 + 1]); | |
| 78 | + if (addr + 0x8 < end_addr) | |
| 79 | + drm_printf(p, " %08x", dump_addr[i * 4 + 2]); | |
| 80 | + drm_printf(p, "\n"); | |
| 81 | + } | |
| 70 | 82 | } |
| 71 | 83 | |
| 72 | 84 | void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p) |
| @@ -79,7 +91,7 @@ void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p) · 6 unmodified lines | ||
| 79 | 91 | } |
| 80 | 92 | |
| 81 | 93 | drm_printf(p, "---\n"); |
| 82 | − drm_printf(p, "kernel: " UTS_RELEASE "\n"); | |
| 94 | + drm_printf(p, "kernel: %s\n", init_utsname()->release); | |
| 83 | 95 | drm_printf(p, "module: " KBUILD_MODNAME "\n"); |
| 84 | 96 | drm_printf(p, "dpu devcoredump\n"); |
| 85 | 97 | drm_printf(p, "time: %ptSp\n", &state->time); |
| @@ -185,7 +197,7 @@ void msm_disp_snapshot_add_block(struct msm_disp_state *disp_state, u32 len, · 99 unmodified lines | ||
| 185 | 197 | va_end(va); |
| 186 | 198 | |
| 187 | 199 | INIT_LIST_HEAD(&new_blk->node); |
| 188 | − new_blk->size = ALIGN(len, REG_DUMP_ALIGN); | |
| 200 | + new_blk->size = len; | |
| 189 | 201 | new_blk->base_addr = base_addr; |
| 190 | 202 | |
| 191 | 203 | msm_disp_state_dump_regs(&new_blk->state, new_blk->size, base_addr); |
| @@ -2033,6 +2033,7 @@ int msm_dsi_host_init(struct msm_dsi *msm_dsi) · 2032 unmodified lines | ||
| 2033 | 2033 | |
| 2034 | 2034 | /* fixup base address by io offset */ |
| 2035 | 2035 | msm_host->ctrl_base += cfg->io_offset; |
| 2036 | + msm_host->ctrl_size -= cfg->io_offset; | |
| 2036 | 2037 | |
| 2037 | 2038 | ret = devm_regulator_bulk_get_const(&pdev->dev, cfg->num_regulators, |
| 2038 | 2039 | cfg->regulator_data, |
| @@ -128,11 +128,10 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv, · 127 unmodified lines | ||
| 128 | 128 | /* |
| 129 | 129 | * Initialize the LRUs: |
| 130 | 130 | */ |
| 131 | − mutex_init(&priv->lru.lock); | |
| 132 | − drm_gem_lru_init(&priv->lru.unbacked, &priv->lru.lock); | |
| 133 | − drm_gem_lru_init(&priv->lru.pinned, &priv->lru.lock); | |
| 134 | − drm_gem_lru_init(&priv->lru.willneed, &priv->lru.lock); | |
| 135 | − drm_gem_lru_init(&priv->lru.dontneed, &priv->lru.lock); | |
| 131 | + drm_gem_lru_init(&priv->lru.unbacked); | |
| 132 | + drm_gem_lru_init(&priv->lru.pinned); | |
| 133 | + drm_gem_lru_init(&priv->lru.willneed); | |
| 134 | + drm_gem_lru_init(&priv->lru.dontneed); | |
| 136 | 135 | |
| 137 | 136 | /* Initialize stall-on-fault */ |
| 138 | 137 | spin_lock_init(&priv->fault_stall_lock); |
| @@ -140,7 +139,7 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv, · 1 unmodified line | ||
| 140 | 139 | |
| 141 | 140 | /* Teach lockdep about lock ordering wrt. shrinker: */ |
| 142 | 141 | fs_reclaim_acquire(GFP_KERNEL); |
| 143 | − might_lock(&priv->lru.lock); | |
| 142 | + might_lock(&ddev->gem_lru_mutex); | |
| 144 | 143 | fs_reclaim_release(GFP_KERNEL); |
| 145 | 144 | |
| 146 | 145 | if (priv->kms_init) { |
| @@ -150,13 +150,6 @@ struct msm_drm_private { · 149 unmodified lines | ||
| 150 | 150 | * DONTNEED state (ie. can be purged) |
| 151 | 151 | */ |
| 152 | 152 | struct drm_gem_lru dontneed; |
| 153 | − | |
| 154 | − /** | |
| 155 | − * lock: | |
| 156 | − * | |
| 157 | − * Protects manipulation of all of the LRUs. | |
| 158 | − */ | |
| 159 | − struct mutex lock; | |
| 160 | 153 | } lru; |
| 161 | 154 | |
| 162 | 155 | struct notifier_block vmap_notifier; |
| @@ -177,11 +177,11 @@ static void update_lru_locked(struct drm_gem_object *obj) · 176 unmodified lines | ||
| 177 | 177 | |
| 178 | 178 | static void update_lru(struct drm_gem_object *obj) |
| 179 | 179 | { |
| 180 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 180 | + struct drm_device *dev = obj->dev; | |
| 181 | 181 | |
| 182 | − mutex_lock(&priv->lru.lock); | |
| 182 | + mutex_lock(&dev->gem_lru_mutex); | |
| 183 | 183 | update_lru_locked(obj); |
| 184 | − mutex_unlock(&priv->lru.lock); | |
| 184 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 185 | 185 | } |
| 186 | 186 | |
| 187 | 187 | static struct page **get_pages(struct drm_gem_object *obj) |
| @@ -292,11 +292,11 @@ void msm_gem_pin_obj_locked(struct drm_gem_object *obj) · 104 unmodified lines | ||
| 292 | 292 | |
| 293 | 293 | static void pin_obj_locked(struct drm_gem_object *obj) |
| 294 | 294 | { |
| 295 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 295 | + struct drm_device *dev = obj->dev; | |
| 296 | 296 | |
| 297 | − mutex_lock(&priv->lru.lock); | |
| 297 | + mutex_lock(&dev->gem_lru_mutex); | |
| 298 | 298 | msm_gem_pin_obj_locked(obj); |
| 299 | − mutex_unlock(&priv->lru.lock); | |
| 299 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 300 | 300 | } |
| 301 | 301 | |
| 302 | 302 | struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) |
| @@ -487,16 +487,16 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct drm_gpuva *vma) · 184 unmodified lines | ||
| 487 | 487 | |
| 488 | 488 | void msm_gem_unpin_locked(struct drm_gem_object *obj) |
| 489 | 489 | { |
| 490 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 490 | + struct drm_device *dev = obj->dev; | |
| 491 | 491 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 492 | 492 | |
| 493 | 493 | msm_gem_assert_locked(obj); |
| 494 | 494 | |
| 495 | − mutex_lock(&priv->lru.lock); | |
| 495 | + mutex_lock(&dev->gem_lru_mutex); | |
| 496 | 496 | msm_obj->pin_count--; |
| 497 | 497 | GEM_WARN_ON(msm_obj->pin_count < 0); |
| 498 | 498 | update_lru_locked(obj); |
| 499 | − mutex_unlock(&priv->lru.lock); | |
| 499 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 500 | 500 | } |
| 501 | 501 | |
| 502 | 502 | /* Special unpin path for use in fence-signaling path, avoiding the need |
| @@ -507,10 +507,10 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) · 4 unmodified lines | ||
| 507 | 507 | */ |
| 508 | 508 | void msm_gem_unpin_active(struct drm_gem_object *obj) |
| 509 | 509 | { |
| 510 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 510 | + struct drm_device *dev = obj->dev; | |
| 511 | 511 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 512 | 512 | |
| 513 | − GEM_WARN_ON(!mutex_is_locked(&priv->lru.lock)); | |
| 513 | + GEM_WARN_ON(!mutex_is_locked(&dev->gem_lru_mutex)); | |
| 514 | 514 | |
| 515 | 515 | msm_obj->pin_count--; |
| 516 | 516 | GEM_WARN_ON(msm_obj->pin_count < 0); |
| @@ -797,12 +797,12 @@ void msm_gem_put_vaddr(struct drm_gem_object *obj) · 280 unmodified lines | ||
| 797 | 797 | */ |
| 798 | 798 | int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) |
| 799 | 799 | { |
| 800 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 800 | + struct drm_device *dev = obj->dev; | |
| 801 | 801 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 802 | 802 | |
| 803 | 803 | msm_gem_lock(obj); |
| 804 | 804 | |
| 805 | − mutex_lock(&priv->lru.lock); | |
| 805 | + mutex_lock(&dev->gem_lru_mutex); | |
| 806 | 806 | |
| 807 | 807 | if (msm_obj->madv != __MSM_MADV_PURGED) |
| 808 | 808 | msm_obj->madv = madv; |
| @@ -814,7 +814,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) · 5 unmodified lines | ||
| 814 | 814 | */ |
| 815 | 815 | update_lru_locked(obj); |
| 816 | 816 | |
| 817 | − mutex_unlock(&priv->lru.lock); | |
| 817 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 818 | 818 | |
| 819 | 819 | msm_gem_unlock(obj); |
| 820 | 820 | |
| @@ -824,7 +824,6 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) · 3 unmodified lines | ||
| 824 | 824 | void msm_gem_purge(struct drm_gem_object *obj) |
| 825 | 825 | { |
| 826 | 826 | struct drm_device *dev = obj->dev; |
| 827 | − struct msm_drm_private *priv = obj->dev->dev_private; | |
| 828 | 827 | struct msm_gem_object *msm_obj = to_msm_bo(obj); |
| 829 | 828 | |
| 830 | 829 | msm_gem_assert_locked(obj); |
| @@ -839,10 +838,10 @@ void msm_gem_purge(struct drm_gem_object *obj) · 8 unmodified lines | ||
| 839 | 838 | |
| 840 | 839 | put_pages(obj); |
| 841 | 840 | |
| 842 | − mutex_lock(&priv->lru.lock); | |
| 841 | + mutex_lock(&dev->gem_lru_mutex); | |
| 843 | 842 | /* A one-way transition: */ |
| 844 | 843 | msm_obj->madv = __MSM_MADV_PURGED; |
| 845 | − mutex_unlock(&priv->lru.lock); | |
| 844 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 846 | 845 | |
| 847 | 846 | drm_gem_free_mmap_offset(obj); |
| 848 | 847 | |
| @@ -43,8 +43,7 @@ msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) · 42 unmodified lines | ||
| 43 | 43 | } |
| 44 | 44 | |
| 45 | 45 | static bool |
| 46 | −with_vm_locks(struct ww_acquire_ctx *ticket, | |
| 47 | − void (*fn)(struct drm_gem_object *obj), | |
| 46 | +with_vm_locks(void (*fn)(struct drm_gem_object *obj), | |
| 48 | 47 | struct drm_gem_object *obj) |
| 49 | 48 | { |
| 50 | 49 | /* |
| @@ -52,7 +51,7 @@ with_vm_locks(struct ww_acquire_ctx *ticket, · 1 unmodified line | ||
| 52 | 51 | * success paths |
| 53 | 52 | */ |
| 54 | 53 | struct drm_gpuvm_bo *vm_bo, *last_locked = NULL; |
| 55 | − int ret = 0; | |
| 54 | + bool locked = true; | |
| 56 | 55 | |
| 57 | 56 | drm_gem_for_each_gpuvm_bo (vm_bo, obj) { |
| 58 | 57 | struct dma_resv *resv = drm_gpuvm_resv(vm_bo->vm); |
| @@ -60,23 +59,14 @@ with_vm_locks(struct ww_acquire_ctx *ticket, · 1 unmodified line | ||
| 60 | 59 | if (resv == obj->resv) |
| 61 | 60 | continue; |
| 62 | 61 | |
| 63 | − ret = dma_resv_lock(resv, ticket); | |
| 64 | − | |
| 65 | − /* | |
| 66 | − * Since we already skip the case when the VM and obj | |
| 67 | − * share a resv (ie. _NO_SHARE objs), we don't expect | |
| 68 | − * to hit a double-locking scenario... which the lock | |
| 69 | − * unwinding cannot really cope with. | |
| 70 | − */ | |
| 71 | − WARN_ON(ret == -EALREADY); | |
| 72 | − | |
| 73 | 62 | /* |
| 74 | − * Don't bother with slow-lock / backoff / retry sequence, | |
| 75 | − * if we can't get the lock just give up and move on to | |
| 76 | − * the next object. | |
| 63 | + * dma_resv_lock can't be used due to acquiring 'ticket' before the | |
| 64 | + * fs_reclaim lock, which is held in shrinker context | |
| 77 | 65 | */ |
| 78 | − if (ret) | |
| 66 | + if (!dma_resv_trylock(resv)) { | |
| 67 | + locked = false; | |
| 79 | 68 | goto out_unlock; |
| 69 | + } | |
| 80 | 70 | |
| 81 | 71 | /* |
| 82 | 72 | * Hold a ref to prevent the vm_bo from being freed |
| @@ -108,11 +98,11 @@ out_unlock: · 25 unmodified lines | ||
| 108 | 98 | } |
| 109 | 99 | } |
| 110 | 100 | |
| 111 | − return ret == 0; | |
| 101 | + return locked; | |
| 112 | 102 | } |
| 113 | 103 | |
| 114 | 104 | static bool |
| 115 | −purge(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) | |
| 105 | +purge(struct drm_gem_object *obj, struct ww_acquire_ctx *unused) | |
| 116 | 106 | { |
| 117 | 107 | if (!is_purgeable(to_msm_bo(obj))) |
| 118 | 108 | return false; |
| @@ -120,11 +110,11 @@ purge(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) · 1 unmodified line | ||
| 120 | 110 | if (msm_gem_active(obj)) |
| 121 | 111 | return false; |
| 122 | 112 | |
| 123 | − return with_vm_locks(ticket, msm_gem_purge, obj); | |
| 113 | + return with_vm_locks(msm_gem_purge, obj); | |
| 124 | 114 | } |
| 125 | 115 | |
| 126 | 116 | static bool |
| 127 | −evict(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) | |
| 117 | +evict(struct drm_gem_object *obj, struct ww_acquire_ctx *unused) | |
| 128 | 118 | { |
| 129 | 119 | if (is_unevictable(to_msm_bo(obj))) |
| 130 | 120 | return false; |
| @@ -132,7 +122,7 @@ evict(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket) · 1 unmodified line | ||
| 132 | 122 | if (msm_gem_active(obj)) |
| 133 | 123 | return false; |
| 134 | 124 | |
| 135 | − return with_vm_locks(ticket, msm_gem_evict, obj); | |
| 125 | + return with_vm_locks(msm_gem_evict, obj); | |
| 136 | 126 | } |
| 137 | 127 | |
| 138 | 128 | static bool |
| @@ -164,7 +154,6 @@ static unsigned long · 25 unmodified lines | ||
| 164 | 154 | msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) |
| 165 | 155 | { |
| 166 | 156 | struct msm_drm_private *priv = shrinker->private_data; |
| 167 | − struct ww_acquire_ctx ticket; | |
| 168 | 157 | struct { |
| 169 | 158 | struct drm_gem_lru *lru; |
| 170 | 159 | bool (*shrink)(struct drm_gem_object *obj, struct ww_acquire_ctx *ticket); |
| @@ -185,11 +174,14 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) · 14 unmodified lines | ||
| 185 | 174 | for (unsigned i = 0; (nr > 0) && (i < ARRAY_SIZE(stages)); i++) { |
| 186 | 175 | if (!stages[i].cond) |
| 187 | 176 | continue; |
| 177 | + /* | |
| 178 | + * 'ticket' not needed on trylock paths | |
| 179 | + */ | |
| 188 | 180 | stages[i].freed = |
| 189 | − drm_gem_lru_scan(stages[i].lru, nr, | |
| 181 | + drm_gem_lru_scan(priv->dev, stages[i].lru, nr, | |
| 190 | 182 | &stages[i].remaining, |
| 191 | 183 | stages[i].shrink, |
| 192 | − &ticket); | |
| 184 | + NULL); | |
| 193 | 185 | nr -= stages[i].freed; |
| 194 | 186 | freed += stages[i].freed; |
| 195 | 187 | remaining += stages[i].remaining; |
| @@ -255,7 +247,7 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) · 59 unmodified lines | ||
| 255 | 247 | unsigned long remaining = 0; |
| 256 | 248 | |
| 257 | 249 | for (idx = 0; lrus[idx] && unmapped < vmap_shrink_limit; idx++) { |
| 258 | − unmapped += drm_gem_lru_scan(lrus[idx], | |
| 250 | + unmapped += drm_gem_lru_scan(priv->dev, lrus[idx], | |
| 259 | 251 | vmap_shrink_limit - unmapped, |
| 260 | 252 | &remaining, |
| 261 | 253 | vmap_shrink, |
| @@ -352,7 +352,7 @@ static int submit_fence_sync(struct msm_gem_submit *submit) · 351 unmodified lines | ||
| 352 | 352 | |
| 353 | 353 | static int submit_pin_objects(struct msm_gem_submit *submit) |
| 354 | 354 | { |
| 355 | − struct msm_drm_private *priv = submit->dev->dev_private; | |
| 355 | + struct drm_device *dev = submit->dev; | |
| 356 | 356 | int i, ret = 0; |
| 357 | 357 | |
| 358 | 358 | for (i = 0; i < submit->nr_bos; i++) { |
| @@ -381,11 +381,11 @@ static int submit_pin_objects(struct msm_gem_submit *submit) · 22 unmodified lines | ||
| 381 | 381 | * get_pages() which could trigger reclaim.. and if we held the LRU lock |
| 382 | 382 | * could trigger deadlock with the shrinker). |
| 383 | 383 | */ |
| 384 | − mutex_lock(&priv->lru.lock); | |
| 384 | + mutex_lock(&dev->gem_lru_mutex); | |
| 385 | 385 | for (i = 0; i < submit->nr_bos; i++) { |
| 386 | 386 | msm_gem_pin_obj_locked(submit->bos[i].obj); |
| 387 | 387 | } |
| 388 | − mutex_unlock(&priv->lru.lock); | |
| 388 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 389 | 389 | |
| 390 | 390 | submit->bos_pinned = true; |
| 391 | 391 | |
| @@ -702,7 +702,7 @@ static struct dma_fence * · 701 unmodified lines | ||
| 702 | 702 | msm_vma_job_run(struct drm_sched_job *_job) |
| 703 | 703 | { |
| 704 | 704 | struct msm_vm_bind_job *job = to_msm_vm_bind_job(_job); |
| 705 | − struct msm_drm_private *priv = job->vm->drm->dev_private; | |
| 705 | + struct drm_device *dev = job->vm->drm; | |
| 706 | 706 | struct msm_gem_vm *vm = to_msm_vm(job->vm); |
| 707 | 707 | struct drm_gem_object *obj; |
| 708 | 708 | int ret = vm->unusable ? -EINVAL : 0; |
| @@ -745,13 +745,13 @@ msm_vma_job_run(struct drm_sched_job *_job) · 36 unmodified lines | ||
| 745 | 745 | if (ret) |
| 746 | 746 | msm_gem_vm_unusable(job->vm); |
| 747 | 747 | |
| 748 | − mutex_lock(&priv->lru.lock); | |
| 748 | + mutex_lock(&dev->gem_lru_mutex); | |
| 749 | 749 | |
| 750 | 750 | job_foreach_bo (obj, job) { |
| 751 | 751 | msm_gem_unpin_active(obj); |
| 752 | 752 | } |
| 753 | 753 | |
| 754 | − mutex_unlock(&priv->lru.lock); | |
| 754 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 755 | 755 | |
| 756 | 756 | /* VM_BIND ops are synchronous, so no fence to wait on: */ |
| 757 | 757 | return NULL; |
| @@ -1305,7 +1305,7 @@ vm_bind_job_pin_objects(struct msm_vm_bind_job *job) · 547 unmodified lines | ||
| 1305 | 1305 | return PTR_ERR(pages); |
| 1306 | 1306 | } |
| 1307 | 1307 | |
| 1308 | − struct msm_drm_private *priv = job->vm->drm->dev_private; | |
| 1308 | + struct drm_device *dev = job->vm->drm; | |
| 1309 | 1309 | |
| 1310 | 1310 | /* |
| 1311 | 1311 | * A second loop while holding the LRU lock (a) avoids acquiring/dropping |
| @@ -1314,10 +1314,10 @@ vm_bind_job_pin_objects(struct msm_vm_bind_job *job) · 2 unmodified lines | ||
| 1314 | 1314 | * get_pages() which could trigger reclaim.. and if we held the LRU lock |
| 1315 | 1315 | * could trigger deadlock with the shrinker). |
| 1316 | 1316 | */ |
| 1317 | − mutex_lock(&priv->lru.lock); | |
| 1317 | + mutex_lock(&dev->gem_lru_mutex); | |
| 1318 | 1318 | job_foreach_bo (obj, job) |
| 1319 | 1319 | msm_gem_pin_obj_locked(obj); |
| 1320 | − mutex_unlock(&priv->lru.lock); | |
| 1320 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 1321 | 1321 | |
| 1322 | 1322 | job->bos_pinned = true; |
| 1323 | 1323 | |
| @@ -13,11 +13,11 @@ · 12 unmodified lines | ||
| 13 | 13 | #include "msm_gpu_trace.h" |
| 14 | 14 | //#include "adreno/adreno_gpu.h" |
| 15 | 15 | |
| 16 | −#include <generated/utsrelease.h> | |
| 17 | 16 | #include <linux/string_helpers.h> |
| 18 | 17 | #include <linux/devcoredump.h> |
| 19 | 18 | #include <linux/sched/task.h> |
| 20 | 19 | #include <linux/sched/mm.h> |
| 20 | +#include <linux/utsname.h> | |
| 21 | 21 | |
| 22 | 22 | /* |
| 23 | 23 | * Power Management: |
| @@ -196,7 +196,7 @@ static ssize_t msm_gpu_devcoredump_read(char *buffer, loff_t offset, · 172 unmodified lines | ||
| 196 | 196 | p = drm_coredump_printer(&iter); |
| 197 | 197 | |
| 198 | 198 | drm_printf(&p, "---\n"); |
| 199 | − drm_printf(&p, "kernel: " UTS_RELEASE "\n"); | |
| 199 | + drm_printf(&p, "kernel: %s\n", init_utsname()->release); | |
| 200 | 200 | drm_printf(&p, "module: " KBUILD_MODNAME "\n"); |
| 201 | 201 | drm_printf(&p, "time: %ptSp\n", &state->time); |
| 202 | 202 | if (state->comm) |
| @@ -677,7 +677,7 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, · 676 unmodified lines | ||
| 677 | 677 | int prot) |
| 678 | 678 | { |
| 679 | 679 | struct msm_iommu *iommu = to_msm_iommu(mmu); |
| 680 | − size_t ret; | |
| 680 | + ssize_t ret; | |
| 681 | 681 | |
| 682 | 682 | WARN_ON(off != 0); |
| 683 | 683 | |
| @@ -686,7 +686,8 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, · 2 unmodified lines | ||
| 686 | 686 | iova |= GENMASK_ULL(63, 49); |
| 687 | 687 | |
| 688 | 688 | ret = iommu_map_sgtable(iommu->domain, iova, sgt, prot); |
| 689 | − WARN_ON(!ret); | |
| 689 | + if (ret < 0) | |
| 690 | + return ret; | |
| 690 | 691 | |
| 691 | 692 | return (ret == len) ? 0 : -EINVAL; |
| 692 | 693 | } |
| @@ -16,13 +16,13 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) · 15 unmodified lines | ||
| 16 | 16 | struct msm_gem_submit *submit = to_msm_submit(job); |
| 17 | 17 | struct msm_fence_context *fctx = submit->ring->fctx; |
| 18 | 18 | struct msm_gpu *gpu = submit->gpu; |
| 19 | − struct msm_drm_private *priv = gpu->dev->dev_private; | |
| 19 | + struct drm_device *dev = gpu->dev; | |
| 20 | 20 | unsigned nr_cmds = submit->nr_cmds; |
| 21 | 21 | int i; |
| 22 | 22 | |
| 23 | 23 | msm_fence_init(submit->hw_fence, fctx); |
| 24 | 24 | |
| 25 | − mutex_lock(&priv->lru.lock); | |
| 25 | + mutex_lock(&dev->gem_lru_mutex); | |
| 26 | 26 | |
| 27 | 27 | for (i = 0; i < submit->nr_bos; i++) { |
| 28 | 28 | struct drm_gem_object *obj = submit->bos[i].obj; |
| @@ -32,7 +32,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) · 3 unmodified lines | ||
| 32 | 32 | |
| 33 | 33 | submit->bos_pinned = false; |
| 34 | 34 | |
| 35 | − mutex_unlock(&priv->lru.lock); | |
| 35 | + mutex_unlock(&dev->gem_lru_mutex); | |
| 36 | 36 | |
| 37 | 37 | /* TODO move submit path over to using a per-ring lock.. */ |
| 38 | 38 | mutex_lock(&gpu->lock); |
| @@ -2513,6 +2513,7 @@ static const struct nvkm_device_chip · 2512 unmodified lines | ||
| 2513 | 2513 | nv170_chipset = { |
| 2514 | 2514 | .name = "GA100", |
| 2515 | 2515 | .bar = { 0x00000001, tu102_bar_new }, |
| 2516 | + .bios = { 0x00000001, nvkm_bios_new }, | |
| 2516 | 2517 | .devinit = { 0x00000001, ga100_devinit_new }, |
| 2517 | 2518 | .fault = { 0x00000001, tu102_fault_new }, |
| 2518 | 2519 | .fb = { 0x00000001, ga100_fb_new }, |
| @@ -2529,7 +2530,6 @@ nv170_chipset = { · 10 unmodified lines | ||
| 2529 | 2530 | .vfn = { 0x00000001, ga100_vfn_new }, |
| 2530 | 2531 | .ce = { 0x000003ff, ga100_ce_new }, |
| 2531 | 2532 | .fifo = { 0x00000001, ga100_fifo_new }, |
| 2532 | − .sec2 = { 0x00000001, tu102_sec2_new }, | |
| 2533 | 2533 | }; |
| 2534 | 2534 | |
| 2535 | 2535 | static const struct nvkm_device_chip |
| @@ -3341,7 +3341,6 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 805 unmodified lines | ||
| 3341 | 3341 | case 0x166: device->chip = &nv166_chipset; break; |
| 3342 | 3342 | case 0x167: device->chip = &nv167_chipset; break; |
| 3343 | 3343 | case 0x168: device->chip = &nv168_chipset; break; |
| 3344 | − case 0x170: device->chip = &nv170_chipset; break; | |
| 3345 | 3344 | case 0x172: device->chip = &nv172_chipset; break; |
| 3346 | 3345 | case 0x173: device->chip = &nv173_chipset; break; |
| 3347 | 3346 | case 0x174: device->chip = &nv174_chipset; break; |
| @@ -3361,6 +3360,14 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 13 unmodified lines | ||
| 3361 | 3360 | case 0x1b6: device->chip = &nv1b6_chipset; break; |
| 3362 | 3361 | case 0x1b7: device->chip = &nv1b7_chipset; break; |
| 3363 | 3362 | default: |
| 3363 | + if (nvkm_boolopt(device->cfgopt, "NvEnableUnsupportedChipsets", false)) { | |
| 3364 | + switch (device->chipset) { | |
| 3365 | + case 0x170: device->chip = &nv170_chipset; break; | |
| 3366 | + default: | |
| 3367 | + break; | |
| 3368 | + } | |
| 3369 | + } | |
| 3370 | + | |
| 3364 | 3371 | if (!device->chip) { |
| 3365 | 3372 | nvdev_error(device, "unknown chipset (%08x)\n", boot0); |
| 3366 | 3373 | ret = -ENODEV; |
| @@ -41,11 +41,15 @@ ga100_gsp_flcn = { · 40 unmodified lines | ||
| 41 | 41 | static const struct nvkm_gsp_func |
| 42 | 42 | ga100_gsp = { |
| 43 | 43 | .flcn = &ga100_gsp_flcn, |
| 44 | + .fwsec = &tu102_gsp_fwsec, | |
| 44 | 45 | |
| 45 | 46 | .sig_section = ".fwsignature_ga100", |
| 46 | 47 | |
| 47 | 48 | .booter.ctor = tu102_gsp_booter_ctor, |
| 48 | 49 | |
| 50 | + .fwsec_sb.ctor = tu102_gsp_fwsec_sb_ctor, | |
| 51 | + .fwsec_sb.dtor = tu102_gsp_fwsec_sb_dtor, | |
| 52 | + | |
| 49 | 53 | .dtor = r535_gsp_dtor, |
| 50 | 54 | .oneinit = tu102_gsp_oneinit, |
| 51 | 55 | .init = tu102_gsp_init, |
| @@ -318,13 +318,8 @@ tu102_gsp_oneinit(struct nvkm_gsp *gsp) · 317 unmodified lines | ||
| 318 | 318 | if (ret) |
| 319 | 319 | return ret; |
| 320 | 320 | |
| 321 | − /* | |
| 322 | − * Calculate FB layout. FRTS is a memory region created by the FWSEC-FRTS firmware. | |
| 323 | − * FWSEC comes from VBIOS. So on systems with no VBIOS (e.g. GA100), the FRTS does | |
| 324 | − * not exist. Therefore, use the existence of VBIOS to determine whether to reserve | |
| 325 | − * an FRTS region. | |
| 326 | − */ | |
| 327 | − gsp->fb.wpr2.frts.size = device->bios ? 0x100000 : 0; | |
| 321 | + /* Calculate FB layout. */ | |
| 322 | + gsp->fb.wpr2.frts.size = 0x100000; | |
| 328 | 323 | gsp->fb.wpr2.frts.addr = ALIGN_DOWN(gsp->fb.bios.addr, 0x20000) - gsp->fb.wpr2.frts.size; |
| 329 | 324 | |
| 330 | 325 | gsp->fb.wpr2.boot.size = gsp->boot.fw.size; |
| @@ -348,12 +343,9 @@ tu102_gsp_oneinit(struct nvkm_gsp *gsp) · 17 unmodified lines | ||
| 348 | 343 | if (ret) |
| 349 | 344 | return ret; |
| 350 | 345 | |
| 351 | − /* Only boot FWSEC-FRTS if it actually exists */ | |
| 352 | − if (gsp->fb.wpr2.frts.size) { | |
| 353 | − ret = nvkm_gsp_fwsec_frts(gsp); | |
| 354 | − if (WARN_ON(ret)) | |
| 355 | − return ret; | |
| 356 | − } | |
| 346 | + ret = nvkm_gsp_fwsec_frts(gsp); | |
| 347 | + if (WARN_ON(ret)) | |
| 348 | + return ret; | |
| 357 | 349 | |
| 358 | 350 | /* Reset GSP into RISC-V mode. */ |
| 359 | 351 | ret = gsp->func->reset(gsp); |
| @@ -208,6 +208,7 @@ config DRM_PANEL_HIMAX_HX83121A · 207 unmodified lines | ||
| 208 | 208 | depends on OF |
| 209 | 209 | depends on DRM_MIPI_DSI |
| 210 | 210 | depends on BACKLIGHT_CLASS_DEVICE |
| 211 | + select DRM_DISPLAY_DSC_HELPER | |
| 211 | 212 | select DRM_KMS_HELPER |
| 212 | 213 | help |
| 213 | 214 | Say Y here if you want to enable support for Himax HX83121A-based |
| @@ -1324,6 +1324,8 @@ static int boe_panel_disable(struct drm_panel *panel) · 1323 unmodified lines | ||
| 1324 | 1324 | mipi_dsi_dcs_set_display_off_multi(&ctx); |
| 1325 | 1325 | mipi_dsi_dcs_enter_sleep_mode_multi(&ctx); |
| 1326 | 1326 | |
| 1327 | + boe->dsi->mode_flags |= MIPI_DSI_MODE_LPM; | |
| 1328 | + | |
| 1327 | 1329 | mipi_dsi_msleep(&ctx, 150); |
| 1328 | 1330 | |
| 1329 | 1331 | return ctx.accum_err; |
| @@ -98,9 +98,7 @@ static int feiyang_enable(struct drm_panel *panel) · 97 unmodified lines | ||
| 98 | 98 | /* T12 (video & logic signal rise + backlight rise) T12 >= 200ms */ |
| 99 | 99 | msleep(200); |
| 100 | 100 | |
| 101 | − mipi_dsi_dcs_set_display_on(ctx->dsi); | |
| 102 | − | |
| 103 | − return 0; | |
| 101 | + return mipi_dsi_dcs_set_display_on(ctx->dsi); | |
| 104 | 102 | } |
| 105 | 103 | |
| 106 | 104 | static int feiyang_disable(struct drm_panel *panel) |
| @@ -937,6 +937,8 @@ static int hx83102_disable(struct drm_panel *panel) · 936 unmodified lines | ||
| 937 | 937 | mipi_dsi_dcs_set_display_off_multi(&dsi_ctx); |
| 938 | 938 | mipi_dsi_dcs_enter_sleep_mode_multi(&dsi_ctx); |
| 939 | 939 | |
| 940 | + dsi->mode_flags |= MIPI_DSI_MODE_LPM; | |
| 941 | + | |
| 940 | 942 | mipi_dsi_msleep(&dsi_ctx, 150); |
| 941 | 943 | |
| 942 | 944 | return dsi_ctx.accum_err; |
| @@ -596,8 +596,8 @@ static int himax_probe(struct mipi_dsi_device *dsi) · 595 unmodified lines | ||
| 596 | 596 | |
| 597 | 597 | ctx = devm_drm_panel_alloc(dev, struct himax, panel, &himax_panel_funcs, |
| 598 | 598 | DRM_MODE_CONNECTOR_DSI); |
| 599 | − if (!ctx) | |
| 600 | − return -ENOMEM; | |
| 599 | + if (IS_ERR(ctx)) | |
| 600 | + return PTR_ERR(ctx); | |
| 601 | 601 | |
| 602 | 602 | ret = devm_regulator_bulk_get_const(&dsi->dev, |
| 603 | 603 | ARRAY_SIZE(himax_supplies), |
| @@ -390,6 +390,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data, · 389 unmodified lines | ||
| 390 | 390 | true, timeout); |
| 391 | 391 | if (!ret) |
| 392 | 392 | ret = timeout ? -ETIMEDOUT : -EBUSY; |
| 393 | + else if (ret > 0) | |
| 394 | + ret = 0; | |
| 393 | 395 | |
| 394 | 396 | drm_gem_object_put(gem_obj); |
| 395 | 397 | |
| @@ -118,12 +118,13 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 117 unmodified lines | ||
| 118 | 118 | /* Complete initialization. */ |
| 119 | 119 | ret = drm_dev_register(&qdev->ddev, ent->driver_data); |
| 120 | 120 | if (ret) |
| 121 | − goto modeset_cleanup; | |
| 121 | + goto poll_fini; | |
| 122 | 122 | |
| 123 | 123 | drm_client_setup(&qdev->ddev, NULL); |
| 124 | 124 | return 0; |
| 125 | 125 | |
| 126 | −modeset_cleanup: | |
| 126 | +poll_fini: | |
| 127 | + drm_kms_helper_poll_fini(&qdev->ddev); | |
| 127 | 128 | qxl_modeset_fini(qdev); |
| 128 | 129 | unload: |
| 129 | 130 | qxl_device_fini(qdev); |
| @@ -154,6 +155,7 @@ qxl_pci_remove(struct pci_dev *pdev) · 24 unmodified lines | ||
| 154 | 155 | { |
| 155 | 156 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 156 | 157 | |
| 158 | + drm_kms_helper_poll_fini(dev); | |
| 157 | 159 | drm_dev_unregister(dev); |
| 158 | 160 | drm_atomic_helper_shutdown(dev); |
| 159 | 161 | if (pci_is_vga(pdev) && pdev->revision < 5) |
| @@ -2461,7 +2461,8 @@ static void ci_register_patching_mc_arb(struct radeon_device *rdev, · 2460 unmodified lines | ||
| 2461 | 2461 | |
| 2462 | 2462 | if (patch && |
| 2463 | 2463 | ((rdev->pdev->device == 0x67B0) || |
| 2464 | − (rdev->pdev->device == 0x67B1))) { | |
| 2464 | + (rdev->pdev->device == 0x67B1)) && | |
| 2465 | + (rdev->pdev->revision == 0)) { | |
| 2465 | 2466 | if ((memory_clock > 100000) && (memory_clock <= 125000)) { |
| 2466 | 2467 | tmp2 = (((0x31 * engine_clock) / 125000) - 1) & 0xff; |
| 2467 | 2468 | *dram_timimg2 &= ~0x00ff0000; |
| @@ -3304,7 +3305,8 @@ static int ci_populate_all_memory_levels(struct radeon_device *rdev) · 836 unmodified lines | ||
| 3304 | 3305 | pi->smc_state_table.MemoryLevel[0].EnabledForActivity = 1; |
| 3305 | 3306 | |
| 3306 | 3307 | if ((dpm_table->mclk_table.count >= 2) && |
| 3307 | − ((rdev->pdev->device == 0x67B0) || (rdev->pdev->device == 0x67B1))) { | |
| 3308 | + ((rdev->pdev->device == 0x67B0) || (rdev->pdev->device == 0x67B1)) && | |
| 3309 | + (rdev->pdev->revision == 0)) { | |
| 3308 | 3310 | pi->smc_state_table.MemoryLevel[1].MinVddc = |
| 3309 | 3311 | pi->smc_state_table.MemoryLevel[0].MinVddc; |
| 3310 | 3312 | pi->smc_state_table.MemoryLevel[1].MinVddcPhases = |
| @@ -4493,7 +4495,8 @@ static int ci_register_patching_mc_seq(struct radeon_device *rdev, · 1182 unmodified lines | ||
| 4493 | 4495 | |
| 4494 | 4496 | if (patch && |
| 4495 | 4497 | ((rdev->pdev->device == 0x67B0) || |
| 4496 | − (rdev->pdev->device == 0x67B1))) { | |
| 4498 | + (rdev->pdev->device == 0x67B1)) && | |
| 4499 | + (rdev->pdev->revision == 0)) { | |
| 4497 | 4500 | for (i = 0; i < table->last; i++) { |
| 4498 | 4501 | if (table->last >= SMU7_DISCRETE_MC_REGISTER_ARRAY_SIZE) |
| 4499 | 4502 | return -EINVAL; |
| @@ -312,8 +312,10 @@ static int evergreen_surface_check(struct radeon_cs_parser *p, · 311 unmodified lines | ||
| 312 | 312 | case ARRAY_2D_TILED_THIN1: |
| 313 | 313 | return evergreen_surface_check_2d(p, surf, prefix); |
| 314 | 314 | default: |
| 315 | − dev_warn(p->dev, "%s:%d %s invalid array mode %d\n", | |
| 316 | − __func__, __LINE__, prefix, surf->mode); | |
| 315 | + if (prefix) { | |
| 316 | + dev_warn(p->dev, "%s:%d %s invalid array mode %d\n", | |
| 317 | + __func__, __LINE__, prefix, surf->mode); | |
| 318 | + } | |
| 317 | 319 | return -EINVAL; |
| 318 | 320 | } |
| 319 | 321 | return -EINVAL; |
| @@ -741,6 +741,7 @@ static int sti_hda_probe(struct platform_device *pdev) · 740 unmodified lines | ||
| 741 | 741 | struct device *dev = &pdev->dev; |
| 742 | 742 | struct sti_hda *hda; |
| 743 | 743 | struct resource *res; |
| 744 | + int ret; | |
| 744 | 745 | |
| 745 | 746 | DRM_INFO("%s\n", __func__); |
| 746 | 747 | |
| @@ -779,7 +780,9 @@ static int sti_hda_probe(struct platform_device *pdev) · 32 unmodified lines | ||
| 779 | 780 | return PTR_ERR(hda->clk_hddac); |
| 780 | 781 | } |
| 781 | 782 | |
| 782 | − drm_bridge_add(&hda->bridge); | |
| 783 | + ret = devm_drm_bridge_add(dev, &hda->bridge); | |
| 784 | + if (ret) | |
| 785 | + return ret; | |
| 783 | 786 | |
| 784 | 787 | platform_set_drvdata(pdev, hda); |
| 785 | 788 | |
| @@ -788,10 +791,7 @@ static int sti_hda_probe(struct platform_device *pdev) · 2 unmodified lines | ||
| 788 | 791 | |
| 789 | 792 | static void sti_hda_remove(struct platform_device *pdev) |
| 790 | 793 | { |
| 791 | − struct sti_hda *hda = platform_get_drvdata(pdev); | |
| 792 | − | |
| 793 | 794 | component_del(&pdev->dev, &sti_hda_ops); |
| 794 | − drm_bridge_remove(&hda->bridge); | |
| 795 | 795 | } |
| 796 | 796 | |
| 797 | 797 | static const struct of_device_id hda_of_match[] = { |
| @@ -350,6 +350,7 @@ static void ofdrm_pci_release(void *data) · 349 unmodified lines | ||
| 350 | 350 | struct pci_dev *pcidev = data; |
| 351 | 351 | |
| 352 | 352 | pci_disable_device(pcidev); |
| 353 | + pci_dev_put(pcidev); | |
| 353 | 354 | } |
| 354 | 355 | |
| 355 | 356 | static int ofdrm_device_init_pci(struct ofdrm_device *odev) |
| @@ -375,6 +376,7 @@ static int ofdrm_device_init_pci(struct ofdrm_device *odev) · 19 unmodified lines | ||
| 375 | 376 | if (ret) { |
| 376 | 377 | drm_err(dev, "pci_enable_device(%s) failed: %d\n", |
| 377 | 378 | dev_name(&pcidev->dev), ret); |
| 379 | + pci_dev_put(pcidev); | |
| 378 | 380 | return ret; |
| 379 | 381 | } |
| 380 | 382 | ret = devm_add_action_or_reset(&pdev->dev, ofdrm_pci_release, pcidev); |
| @@ -353,7 +353,7 @@ static int appletbdrm_primary_plane_helper_atomic_check(struct drm_plane *plane, · 352 unmodified lines | ||
| 353 | 353 | frames_size + |
| 354 | 354 | sizeof(struct appletbdrm_fb_request_footer), 16); |
| 355 | 355 | |
| 356 | − appletbdrm_state->request = kzalloc(request_size, GFP_KERNEL); | |
| 356 | + appletbdrm_state->request = kvzalloc(request_size, GFP_KERNEL); | |
| 357 | 357 | |
| 358 | 358 | if (!appletbdrm_state->request) |
| 359 | 359 | return -ENOMEM; |
| @@ -543,7 +543,7 @@ static void appletbdrm_primary_plane_destroy_state(struct drm_plane *plane, · 183 unmodified lines | ||
| 543 | 543 | { |
| 544 | 544 | struct appletbdrm_plane_state *appletbdrm_state = to_appletbdrm_plane_state(state); |
| 545 | 545 | |
| 546 | − kfree(appletbdrm_state->request); | |
| 546 | + kvfree(appletbdrm_state->request); | |
| 547 | 547 | kfree(appletbdrm_state->response); |
| 548 | 548 | |
| 549 | 549 | __drm_gem_destroy_shadow_plane_state(&appletbdrm_state->base); |
| @@ -761,25 +761,21 @@ static int bochs_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent · 760 unmodified lines | ||
| 761 | 761 | |
| 762 | 762 | ret = pcim_enable_device(pdev); |
| 763 | 763 | if (ret) |
| 764 | − goto err_free_dev; | |
| 764 | + return ret; | |
| 765 | 765 | |
| 766 | 766 | pci_set_drvdata(pdev, dev); |
| 767 | 767 | |
| 768 | 768 | ret = bochs_load(bochs); |
| 769 | 769 | if (ret) |
| 770 | − goto err_free_dev; | |
| 770 | + return ret; | |
| 771 | 771 | |
| 772 | 772 | ret = drm_dev_register(dev, 0); |
| 773 | 773 | if (ret) |
| 774 | − goto err_free_dev; | |
| 774 | + return ret; | |
| 775 | 775 | |
| 776 | 776 | drm_client_setup(dev, NULL); |
| 777 | 777 | |
| 778 | 778 | return ret; |
| 779 | − | |
| 780 | −err_free_dev: | |
| 781 | − drm_dev_put(dev); | |
| 782 | − return ret; | |
| 783 | 779 | } |
| 784 | 780 | |
| 785 | 781 | static void bochs_pci_remove(struct pci_dev *pdev) |
| @@ -739,7 +739,7 @@ static int ttm_bo_alloc_resource(struct ttm_buffer_object *bo, · 738 unmodified lines | ||
| 739 | 739 | may_evict = (force_space && place->mem_type != TTM_PL_SYSTEM); |
| 740 | 740 | ret = ttm_resource_alloc(bo, place, res, force_space ? &limit_pool : NULL); |
| 741 | 741 | if (ret) { |
| 742 | − if (ret != -ENOSPC && ret != -EAGAIN) { | |
| 742 | + if (ret != -ENOSPC) { | |
| 743 | 743 | dmem_cgroup_pool_state_put(limit_pool); |
| 744 | 744 | return ret; |
| 745 | 745 | } |
| @@ -1177,17 +1177,13 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 431 unmodified lines | ||
| 1177 | 1177 | bdev->funcs->swap_notify(bo); |
| 1178 | 1178 | |
| 1179 | 1179 | if (ttm_tt_is_populated(tt)) { |
| 1180 | − spin_lock(&bdev->lru_lock); | |
| 1181 | − ttm_resource_del_bulk_move(bo->resource, bo); | |
| 1182 | − spin_unlock(&bdev->lru_lock); | |
| 1183 | − | |
| 1184 | 1180 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); |
| 1185 | − | |
| 1186 | − spin_lock(&bdev->lru_lock); | |
| 1187 | − if (ret) | |
| 1188 | − ttm_resource_add_bulk_move(bo->resource, bo); | |
| 1189 | − ttm_resource_move_to_lru_tail(bo->resource); | |
| 1190 | − spin_unlock(&bdev->lru_lock); | |
| 1181 | + if (!ret) { | |
| 1182 | + spin_lock(&bdev->lru_lock); | |
| 1183 | + ttm_resource_del_bulk_move_unevictable(bo->resource, bo); | |
| 1184 | + ttm_resource_move_to_lru_tail(bo->resource); | |
| 1185 | + spin_unlock(&bdev->lru_lock); | |
| 1186 | + } | |
| 1191 | 1187 | } |
| 1192 | 1188 | |
| 1193 | 1189 | out: |
| @@ -1112,19 +1112,14 @@ long ttm_bo_shrink(struct ttm_operation_ctx *ctx, struct ttm_buffer_object *bo, · 1111 unmodified lines | ||
| 1112 | 1112 | if (lret < 0) |
| 1113 | 1113 | return lret; |
| 1114 | 1114 | |
| 1115 | − if (bo->bulk_move) { | |
| 1116 | − spin_lock(&bdev->lru_lock); | |
| 1117 | − ttm_resource_del_bulk_move(bo->resource, bo); | |
| 1118 | − spin_unlock(&bdev->lru_lock); | |
| 1119 | − } | |
| 1120 | − | |
| 1121 | 1115 | lret = ttm_tt_backup(bdev, bo->ttm, (struct ttm_backup_flags) |
| 1122 | 1116 | {.purge = flags.purge, |
| 1123 | 1117 | .writeback = flags.writeback}); |
| 1124 | 1118 | |
| 1125 | − if (lret <= 0 && bo->bulk_move) { | |
| 1119 | + if (lret > 0) { | |
| 1126 | 1120 | spin_lock(&bdev->lru_lock); |
| 1127 | − ttm_resource_add_bulk_move(bo->resource, bo); | |
| 1121 | + ttm_resource_del_bulk_move_unevictable(bo->resource, bo); | |
| 1122 | + ttm_resource_move_to_lru_tail(bo->resource); | |
| 1128 | 1123 | spin_unlock(&bdev->lru_lock); |
| 1129 | 1124 | } |
| 1130 | 1125 | |
| @@ -206,6 +206,14 @@ error_free: · 205 unmodified lines | ||
| 206 | 206 | return NULL; |
| 207 | 207 | } |
| 208 | 208 | |
| 209 | +static void __free_pages_gpu_account(struct page *p, unsigned int order, | |
| 210 | + bool reclaim) | |
| 211 | +{ | |
| 212 | + mod_lruvec_page_state(p, reclaim ? NR_GPU_RECLAIM : NR_GPU_ACTIVE, | |
| 213 | + -(1 << order)); | |
| 214 | + __free_pages(p, order); | |
| 215 | +} | |
| 216 | + | |
| 209 | 217 | /* Reset the caching and pages of size 1 << order */ |
| 210 | 218 | static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching caching, |
| 211 | 219 | unsigned int order, struct page *p, bool reclaim) |
| @@ -223,9 +231,7 @@ static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching caching, · 11 unmodified lines | ||
| 223 | 231 | #endif |
| 224 | 232 | |
| 225 | 233 | if (!pool || !ttm_pool_uses_dma_alloc(pool)) { |
| 226 | − mod_lruvec_page_state(p, reclaim ? NR_GPU_RECLAIM : NR_GPU_ACTIVE, | |
| 227 | − -(1 << order)); | |
| 228 | − __free_pages(p, order); | |
| 234 | + __free_pages_gpu_account(p, order, reclaim); | |
| 229 | 235 | return; |
| 230 | 236 | } |
| 231 | 237 | |
| @@ -606,7 +612,7 @@ static int ttm_pool_restore_commit(struct ttm_pool_tt_restore *restore, · 374 unmodified lines | ||
| 606 | 612 | */ |
| 607 | 613 | ttm_pool_split_for_swap(restore->pool, p); |
| 608 | 614 | copy_highpage(restore->alloced_page + i, p); |
| 609 | − __free_pages(p, 0); | |
| 615 | + __free_pages_gpu_account(p, 0, false); | |
| 610 | 616 | } |
| 611 | 617 | |
| 612 | 618 | restore->restored_pages++; |
| @@ -1068,7 +1074,7 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, · 455 unmodified lines | ||
| 1068 | 1074 | if (flags->purge) { |
| 1069 | 1075 | shrunken += num_pages; |
| 1070 | 1076 | page->private = 0; |
| 1071 | − __free_pages(page, order); | |
| 1077 | + __free_pages_gpu_account(page, order, false); | |
| 1072 | 1078 | memset(tt->pages + i, 0, |
| 1073 | 1079 | num_pages * sizeof(*tt->pages)); |
| 1074 | 1080 | } |
| @@ -1109,7 +1115,7 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, · 34 unmodified lines | ||
| 1109 | 1115 | } |
| 1110 | 1116 | handle = shandle; |
| 1111 | 1117 | tt->pages[i] = ttm_backup_handle_to_page_ptr(handle); |
| 1112 | − put_page(page); | |
| 1118 | + __free_pages_gpu_account(page, 0, false); | |
| 1113 | 1119 | shrunken++; |
| 1114 | 1120 | } |
| 1115 | 1121 | |
| @@ -292,6 +292,19 @@ void ttm_resource_del_bulk_move(struct ttm_resource *res, · 291 unmodified lines | ||
| 292 | 292 | ttm_lru_bulk_move_del(bo->bulk_move, res); |
| 293 | 293 | } |
| 294 | 294 | |
| 295 | +/* | |
| 296 | + * Remove a resource from its bulk_move, bypassing the unevictable check. | |
| 297 | + * Use only when the resource is known to still be tracked in the range despite | |
| 298 | + * the BO having just become unevictable; asserts that this is the case. | |
| 299 | + */ | |
| 300 | +void ttm_resource_del_bulk_move_unevictable(struct ttm_resource *res, | |
| 301 | + struct ttm_buffer_object *bo) | |
| 302 | +{ | |
| 303 | + WARN_ON_ONCE(!ttm_resource_unevictable(res, bo)); | |
| 304 | + if (bo->bulk_move) | |
| 305 | + ttm_lru_bulk_move_del(bo->bulk_move, res); | |
| 306 | +} | |
| 307 | + | |
| 295 | 308 | /* Move a resource to the LRU or bulk tail */ |
| 296 | 309 | void ttm_resource_move_to_lru_tail(struct ttm_resource *res) |
| 297 | 310 | { |
| @@ -385,8 +398,11 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo, · 87 unmodified lines | ||
| 385 | 398 | |
| 386 | 399 | if (man->cg) { |
| 387 | 400 | ret = dmem_cgroup_try_charge(man->cg, bo->base.size, &pool, ret_limit_pool); |
| 388 | − if (ret) | |
| 401 | + if (ret) { | |
| 402 | + if (ret == -EAGAIN) | |
| 403 | + ret = -ENOSPC; | |
| 389 | 404 | return ret; |
| 405 | + } | |
| 390 | 406 | } |
| 391 | 407 | |
| 392 | 408 | ret = man->func->alloc(man, bo, place, res_ptr); |
| @@ -285,13 +285,12 @@ static struct urb *udl_get_urb_locked(struct udl_device *udl, long timeout) · 284 unmodified lines | ||
| 285 | 285 | return unode->urb; |
| 286 | 286 | } |
| 287 | 287 | |
| 288 | −#define GET_URB_TIMEOUT HZ | |
| 289 | 288 | struct urb *udl_get_urb(struct udl_device *udl) |
| 290 | 289 | { |
| 291 | 290 | struct urb *urb; |
| 292 | 291 | |
| 293 | 292 | spin_lock_irq(&udl->urbs.lock); |
| 294 | − urb = udl_get_urb_locked(udl, GET_URB_TIMEOUT); | |
| 293 | + urb = udl_get_urb_locked(udl, HZ * 2); | |
| 295 | 294 | spin_unlock_irq(&udl->urbs.lock); |
| 296 | 295 | return urb; |
| 297 | 296 | } |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #include <drm/drm_gem_framebuffer_helper.h> |
| 22 | 22 | #include <drm/drm_gem_shmem_helper.h> |
| 23 | 23 | #include <drm/drm_modeset_helper_vtables.h> |
| 24 | +#include <drm/drm_print.h> | |
| 24 | 25 | #include <drm/drm_probe_helper.h> |
| 25 | 26 | #include <drm/drm_vblank.h> |
| 26 | 27 | |
| @@ -342,8 +343,10 @@ static void udl_crtc_helper_atomic_enable(struct drm_crtc *crtc, struct drm_atom · 315 unmodified lines | ||
| 342 | 343 | return; |
| 343 | 344 | |
| 344 | 345 | urb = udl_get_urb(udl); |
| 345 | − if (!urb) | |
| 346 | + if (!urb) { | |
| 347 | + drm_err_ratelimited(dev, "get urb failed when enabling crtc\n"); | |
| 346 | 348 | goto out; |
| 349 | + } | |
| 347 | 350 | |
| 348 | 351 | buf = (char *)urb->transfer_buffer; |
| 349 | 352 | buf = udl_vidreg_lock(buf); |
| @@ -125,20 +125,6 @@ v3d_performance_query_info_free(struct v3d_performance_query_info *query_info, · 124 unmodified lines | ||
| 125 | 125 | } |
| 126 | 126 | } |
| 127 | 127 | |
| 128 | −static void | |
| 129 | −v3d_cpu_job_free(struct drm_sched_job *sched_job) | |
| 130 | −{ | |
| 131 | − struct v3d_cpu_job *job = to_cpu_job(sched_job); | |
| 132 | − | |
| 133 | − v3d_timestamp_query_info_free(&job->timestamp_query, | |
| 134 | − job->timestamp_query.count); | |
| 135 | − | |
| 136 | − v3d_performance_query_info_free(&job->performance_query, | |
| 137 | − job->performance_query.count); | |
| 138 | − | |
| 139 | − v3d_job_cleanup(&job->base); | |
| 140 | −} | |
| 141 | − | |
| 142 | 128 | static void |
| 143 | 129 | v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job) |
| 144 | 130 | { |
| @@ -830,7 +816,7 @@ static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = { · 685 unmodified lines | ||
| 830 | 816 | |
| 831 | 817 | static const struct drm_sched_backend_ops v3d_cpu_sched_ops = { |
| 832 | 818 | .run_job = v3d_cpu_job_run, |
| 833 | − .free_job = v3d_cpu_job_free | |
| 819 | + .free_job = v3d_sched_job_free | |
| 834 | 820 | }; |
| 835 | 821 | |
| 836 | 822 | static int |
| @@ -123,6 +123,24 @@ v3d_render_job_free(struct kref *ref) · 122 unmodified lines | ||
| 123 | 123 | v3d_job_free(ref); |
| 124 | 124 | } |
| 125 | 125 | |
| 126 | +static void | |
| 127 | +v3d_cpu_job_free(struct kref *ref) | |
| 128 | +{ | |
| 129 | + struct v3d_cpu_job *job = container_of(ref, struct v3d_cpu_job, | |
| 130 | + base.refcount); | |
| 131 | + | |
| 132 | + v3d_timestamp_query_info_free(&job->timestamp_query, | |
| 133 | + job->timestamp_query.count); | |
| 134 | + | |
| 135 | + v3d_performance_query_info_free(&job->performance_query, | |
| 136 | + job->performance_query.count); | |
| 137 | + | |
| 138 | + if (job->indirect_csd.indirect) | |
| 139 | + drm_gem_object_put(job->indirect_csd.indirect); | |
| 140 | + | |
| 141 | + v3d_job_free(ref); | |
| 142 | +} | |
| 143 | + | |
| 126 | 144 | void v3d_job_cleanup(struct v3d_job *job) |
| 127 | 145 | { |
| 128 | 146 | if (!job) |
| @@ -1302,7 +1320,7 @@ v3d_submit_cpu_ioctl(struct drm_device *dev, void *data, · 1173 unmodified lines | ||
| 1302 | 1320 | trace_v3d_submit_cpu_ioctl(&v3d->drm, cpu_job->job_type); |
| 1303 | 1321 | |
| 1304 | 1322 | ret = v3d_job_init(v3d, file_priv, &cpu_job->base, |
| 1305 | − v3d_job_free, 0, &se, V3D_CPU); | |
| 1323 | + v3d_cpu_job_free, 0, &se, V3D_CPU); | |
| 1306 | 1324 | if (ret) { |
| 1307 | 1325 | v3d_job_deallocate((void *)&cpu_job); |
| 1308 | 1326 | goto fail; |
| @@ -1385,8 +1403,6 @@ fail: · 76 unmodified lines | ||
| 1385 | 1403 | v3d_job_cleanup((void *)csd_job); |
| 1386 | 1404 | v3d_job_cleanup(clean_job); |
| 1387 | 1405 | v3d_put_multisync_post_deps(&se); |
| 1388 | − kvfree(cpu_job->timestamp_query.queries); | |
| 1389 | − kvfree(cpu_job->performance_query.queries); | |
| 1390 | 1406 | |
| 1391 | 1407 | return ret; |
| 1392 | 1408 | } |
| @@ -317,6 +317,7 @@ virtio_gpu_array_from_handles(struct drm_file *drm_file, u32 *handles, u32 nents · 316 unmodified lines | ||
| 317 | 317 | void virtio_gpu_array_add_obj(struct virtio_gpu_object_array *objs, |
| 318 | 318 | struct drm_gem_object *obj); |
| 319 | 319 | int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs); |
| 320 | +int virtio_gpu_lock_one_resv_uninterruptible(struct virtio_gpu_object_array *objs); | |
| 320 | 321 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs); |
| 321 | 322 | void virtio_gpu_array_add_fence(struct virtio_gpu_object_array *objs, |
| 322 | 323 | struct dma_fence *fence); |
| @@ -238,6 +238,23 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs) · 237 unmodified lines | ||
| 238 | 238 | return ret; |
| 239 | 239 | } |
| 240 | 240 | |
| 241 | +int virtio_gpu_lock_one_resv_uninterruptible(struct virtio_gpu_object_array *objs) | |
| 242 | +{ | |
| 243 | + int ret; | |
| 244 | + | |
| 245 | + if (objs->nents != 1) | |
| 246 | + return -EINVAL; | |
| 247 | + | |
| 248 | + dma_resv_lock(objs->objs[0]->resv, NULL); | |
| 249 | + | |
| 250 | + ret = dma_resv_reserve_fences(objs->objs[0]->resv, 1); | |
| 251 | + if (ret) { | |
| 252 | + virtio_gpu_array_unlock_resv(objs); | |
| 253 | + return ret; | |
| 254 | + } | |
| 255 | + return 0; | |
| 256 | +} | |
| 257 | + | |
| 241 | 258 | void virtio_gpu_array_unlock_resv(struct virtio_gpu_object_array *objs) |
| 242 | 259 | { |
| 243 | 260 | if (objs->nents == 1) { |
| @@ -215,7 +215,10 @@ static void virtio_gpu_resource_flush(struct drm_plane *plane, · 214 unmodified lines | ||
| 215 | 215 | if (!objs) |
| 216 | 216 | return; |
| 217 | 217 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); |
| 218 | − virtio_gpu_array_lock_resv(objs); | |
| 218 | + if (virtio_gpu_lock_one_resv_uninterruptible(objs)) { | |
| 219 | + virtio_gpu_array_put_free(objs); | |
| 220 | + return; | |
| 221 | + } | |
| 219 | 222 | virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y, |
| 220 | 223 | width, height, objs, |
| 221 | 224 | vgplane_st->fence); |
| @@ -459,7 +462,10 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, · 237 unmodified lines | ||
| 459 | 462 | if (!objs) |
| 460 | 463 | return; |
| 461 | 464 | virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); |
| 462 | − virtio_gpu_array_lock_resv(objs); | |
| 465 | + if (virtio_gpu_lock_one_resv_uninterruptible(objs)) { | |
| 466 | + virtio_gpu_array_put_free(objs); | |
| 467 | + return; | |
| 468 | + } | |
| 463 | 469 | virtio_gpu_cmd_transfer_to_host_2d |
| 464 | 470 | (vgdev, 0, |
| 465 | 471 | plane->state->crtc_w, |
| @@ -88,6 +88,7 @@ xe-y += xe_bb.o \ · 87 unmodified lines | ||
| 88 | 88 | xe_irq.o \ |
| 89 | 89 | xe_late_bind_fw.o \ |
| 90 | 90 | xe_lrc.o \ |
| 91 | + xe_mem_pool.o \ | |
| 91 | 92 | xe_migrate.o \ |
| 92 | 93 | xe_mmio.o \ |
| 93 | 94 | xe_mmio_gem.o \ |
| @@ -37,9 +37,17 @@ static bool intel_hdcp_gsc_check_status(struct drm_device *drm) · 36 unmodified lines | ||
| 37 | 37 | struct xe_device *xe = to_xe_device(drm); |
| 38 | 38 | struct xe_tile *tile = xe_device_get_root_tile(xe); |
| 39 | 39 | struct xe_gt *gt = tile->media_gt; |
| 40 | − struct xe_gsc *gsc = >->uc.gsc; | |
| 40 | + struct xe_gsc *gsc; | |
| 41 | + | |
| 42 | + if (!gt) { | |
| 43 | + drm_dbg_kms(&xe->drm, | |
| 44 | + "not checking GSC status for HDCP2.x: media GT not present or disabled\n"); | |
| 45 | + return false; | |
| 46 | + } | |
| 47 | + | |
| 48 | + gsc = >->uc.gsc; | |
| 41 | 49 | |
| 42 | − if (!gsc || !xe_uc_fw_is_available(&gsc->fw)) { | |
| 50 | + if (!xe_uc_fw_is_available(&gsc->fw)) { | |
| 43 | 51 | drm_dbg_kms(&xe->drm, |
| 44 | 52 | "GSC Components not ready for HDCP2.x\n"); |
| 45 | 53 | return false; |
| @@ -152,10 +152,11 @@ · 151 unmodified lines | ||
| 152 | 152 | |
| 153 | 153 | #define XEHPG_INSTDONE_GEOM_SVGUNIT XE_REG_MCR(0x666c) |
| 154 | 154 | |
| 155 | −#define CACHE_MODE_1 XE_REG(0x7004, XE_REG_OPTION_MASKED) | |
| 155 | +#define CACHE_MODE_1 XE_REG_MCR(0x7004, XE_REG_OPTION_MASKED) | |
| 156 | 156 | #define MSAA_OPTIMIZATION_REDUC_DISABLE REG_BIT(11) |
| 157 | 157 | |
| 158 | 158 | #define COMMON_SLICE_CHICKEN1 XE_REG(0x7010, XE_REG_OPTION_MASKED) |
| 159 | +#define XEHP_COMMON_SLICE_CHICKEN1 XE_REG_MCR(0x7010, XE_REG_OPTION_MASKED) | |
| 159 | 160 | #define DISABLE_BOTTOM_CLIP_RECTANGLE_TEST REG_BIT(14) |
| 160 | 161 | |
| 161 | 162 | #define HIZ_CHICKEN XE_REG(0x7018, XE_REG_OPTION_MASKED) |
| @@ -178,6 +179,7 @@ · 16 unmodified lines | ||
| 178 | 179 | #define XEHPG_SC_INSTDONE_EXTRA2 XE_REG_MCR(0x7108) |
| 179 | 180 | |
| 180 | 181 | #define COMMON_SLICE_CHICKEN4 XE_REG(0x7300, XE_REG_OPTION_MASKED) |
| 182 | +#define XEHP_COMMON_SLICE_CHICKEN4 XE_REG_MCR(0x7300, XE_REG_OPTION_MASKED) | |
| 181 | 183 | #define SBE_PUSH_CONSTANT_BEHIND_FIX_ENABLE REG_BIT(12) |
| 182 | 184 | #define DISABLE_TDC_LOAD_BALANCING_CALC REG_BIT(6) |
| 183 | 185 | #define HW_FILTERING REG_BIT(5) |
| @@ -583,7 +585,7 @@ · 399 unmodified lines | ||
| 583 | 585 | #define DISABLE_128B_EVICTION_COMMAND_UDW REG_BIT(36 - 32) |
| 584 | 586 | #define LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE REG_BIT(35 - 32) |
| 585 | 587 | |
| 586 | −#define ROW_CHICKEN5 XE_REG_MCR(0xe7f0) | |
| 588 | +#define ROW_CHICKEN5 XE_REG_MCR(0xe7f0, XE_REG_OPTION_MASKED) | |
| 587 | 589 | #define CPSS_AWARE_DIS REG_BIT(3) |
| 588 | 590 | |
| 589 | 591 | #define SARB_CHICKEN1 XE_REG_MCR(0xe90c) |
| @@ -897,10 +897,10 @@ void xe_bo_set_purgeable_state(struct xe_bo *bo, · 896 unmodified lines | ||
| 897 | 897 | new_state == XE_MADV_PURGEABLE_PURGED); |
| 898 | 898 | |
| 899 | 899 | /* Once purged, always purged - cannot transition out */ |
| 900 | − xe_assert(xe, !(bo->madv_purgeable == XE_MADV_PURGEABLE_PURGED && | |
| 900 | + xe_assert(xe, !(bo->purgeable.state == XE_MADV_PURGEABLE_PURGED && | |
| 901 | 901 | new_state != XE_MADV_PURGEABLE_PURGED)); |
| 902 | 902 | |
| 903 | − bo->madv_purgeable = new_state; | |
| 903 | + bo->purgeable.state = new_state; | |
| 904 | 904 | xe_bo_set_purgeable_shrinker(bo, new_state); |
| 905 | 905 | } |
| 906 | 906 | |
| @@ -2322,8 +2322,10 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 1415 unmodified lines | ||
| 2322 | 2322 | } |
| 2323 | 2323 | |
| 2324 | 2324 | /* XE_BO_FLAG_GGTTx requires XE_BO_FLAG_GGTT also be set */ |
| 2325 | − if ((flags & XE_BO_FLAG_GGTT_ALL) && !(flags & XE_BO_FLAG_GGTT)) | |
| 2325 | + if ((flags & XE_BO_FLAG_GGTT_ALL) && !(flags & XE_BO_FLAG_GGTT)) { | |
| 2326 | + xe_bo_free(bo); | |
| 2326 | 2327 | return ERR_PTR(-EINVAL); |
| 2328 | + } | |
| 2327 | 2329 | |
| 2328 | 2330 | if (flags & (XE_BO_FLAG_VRAM_MASK | XE_BO_FLAG_STOLEN) && |
| 2329 | 2331 | !(flags & XE_BO_FLAG_IGNORE_MIN_PAGE_SIZE) && |
| @@ -2342,8 +2344,10 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 12 unmodified lines | ||
| 2342 | 2344 | alignment = SZ_4K >> PAGE_SHIFT; |
| 2343 | 2345 | } |
| 2344 | 2346 | |
| 2345 | − if (type == ttm_bo_type_device && aligned_size != size) | |
| 2347 | + if (type == ttm_bo_type_device && aligned_size != size) { | |
| 2348 | + xe_bo_free(bo); | |
| 2346 | 2349 | return ERR_PTR(-EINVAL); |
| 2350 | + } | |
| 2347 | 2351 | |
| 2348 | 2352 | if (!bo) { |
| 2349 | 2353 | bo = xe_bo_alloc(); |
| @@ -2364,7 +2368,7 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, · 14 unmodified lines | ||
| 2364 | 2368 | INIT_LIST_HEAD(&bo->vram_userfault_link); |
| 2365 | 2369 | |
| 2366 | 2370 | /* Initialize purge advisory state */ |
| 2367 | − bo->madv_purgeable = XE_MADV_PURGEABLE_WILLNEED; | |
| 2371 | + bo->purgeable.state = XE_MADV_PURGEABLE_WILLNEED; | |
| 2368 | 2372 | |
| 2369 | 2373 | drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size); |
| 2370 | 2374 | |
| @@ -251,7 +251,7 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo) · 250 unmodified lines | ||
| 251 | 251 | static inline bool xe_bo_is_purged(struct xe_bo *bo) |
| 252 | 252 | { |
| 253 | 253 | xe_bo_assert_held(bo); |
| 254 | − return bo->madv_purgeable == XE_MADV_PURGEABLE_PURGED; | |
| 254 | + return bo->purgeable.state == XE_MADV_PURGEABLE_PURGED; | |
| 255 | 255 | } |
| 256 | 256 | |
| 257 | 257 | /** |
| @@ -268,11 +268,95 @@ static inline bool xe_bo_is_purged(struct xe_bo *bo) · 10 unmodified lines | ||
| 268 | 268 | static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo) |
| 269 | 269 | { |
| 270 | 270 | xe_bo_assert_held(bo); |
| 271 | − return bo->madv_purgeable == XE_MADV_PURGEABLE_DONTNEED; | |
| 271 | + return bo->purgeable.state == XE_MADV_PURGEABLE_DONTNEED; | |
| 272 | 272 | } |
| 273 | 273 | |
| 274 | 274 | void xe_bo_set_purgeable_state(struct xe_bo *bo, enum xe_madv_purgeable_state new_state); |
| 275 | 275 | |
| 276 | +/** | |
| 277 | + * xe_bo_willneed_get_locked() - Acquire a WILLNEED holder on a BO | |
| 278 | + * @bo: Buffer object | |
| 279 | + * | |
| 280 | + * Increments willneed_count and, on a 0->1 transition, promotes the BO | |
| 281 | + * from DONTNEED to WILLNEED. PURGED is terminal and is never modified. | |
| 282 | + * | |
| 283 | + * Caller must hold the BO's dma-resv lock. | |
| 284 | + */ | |
| 285 | +static inline void xe_bo_willneed_get_locked(struct xe_bo *bo) | |
| 286 | +{ | |
| 287 | + xe_bo_assert_held(bo); | |
| 288 | + | |
| 289 | + /* Imported BOs are owned externally; do not track purgeability. */ | |
| 290 | + if (drm_gem_is_imported(&bo->ttm.base)) | |
| 291 | + return; | |
| 292 | + | |
| 293 | + if (bo->purgeable.willneed_count++ == 0 && xe_bo_madv_is_dontneed(bo)) | |
| 294 | + xe_bo_set_purgeable_state(bo, XE_MADV_PURGEABLE_WILLNEED); | |
| 295 | +} | |
| 296 | + | |
| 297 | +/** | |
| 298 | + * xe_bo_willneed_put_locked() - Release a WILLNEED holder on a BO | |
| 299 | + * @bo: Buffer object | |
| 300 | + * | |
| 301 | + * Decrements willneed_count and, on a 1->0 transition, marks the BO | |
| 302 | + * DONTNEED only if it still has VMAs (implying all active VMAs are | |
| 303 | + * DONTNEED). If the last VMA is being removed, preserve the current BO | |
| 304 | + * state to match the previous VMA-walk semantics. | |
| 305 | + * | |
| 306 | + * PURGED is terminal and the BO state is never modified. | |
| 307 | + * | |
| 308 | + * Caller must hold the BO's dma-resv lock. | |
| 309 | + */ | |
| 310 | +static inline void xe_bo_willneed_put_locked(struct xe_bo *bo) | |
| 311 | +{ | |
| 312 | + xe_bo_assert_held(bo); | |
| 313 | + | |
| 314 | + if (drm_gem_is_imported(&bo->ttm.base)) | |
| 315 | + return; | |
| 316 | + | |
| 317 | + xe_assert(xe_bo_device(bo), bo->purgeable.willneed_count > 0); | |
| 318 | + if (--bo->purgeable.willneed_count == 0 && bo->purgeable.vma_count > 0 && | |
| 319 | + !xe_bo_is_purged(bo)) | |
| 320 | + xe_bo_set_purgeable_state(bo, XE_MADV_PURGEABLE_DONTNEED); | |
| 321 | +} | |
| 322 | + | |
| 323 | +/** | |
| 324 | + * xe_bo_vma_count_inc_locked() - Account a new VMA on a BO | |
| 325 | + * @bo: Buffer object | |
| 326 | + * | |
| 327 | + * Increments vma_count. | |
| 328 | + * | |
| 329 | + * Caller must hold the BO's dma-resv lock. | |
| 330 | + */ | |
| 331 | +static inline void xe_bo_vma_count_inc_locked(struct xe_bo *bo) | |
| 332 | +{ | |
| 333 | + xe_bo_assert_held(bo); | |
| 334 | + | |
| 335 | + if (drm_gem_is_imported(&bo->ttm.base)) | |
| 336 | + return; | |
| 337 | + | |
| 338 | + bo->purgeable.vma_count++; | |
| 339 | +} | |
| 340 | + | |
| 341 | +/** | |
| 342 | + * xe_bo_vma_count_dec_locked() - Account a VMA removal on a BO | |
| 343 | + * @bo: Buffer object | |
| 344 | + * | |
| 345 | + * Decrements vma_count. | |
| 346 | + * | |
| 347 | + * Caller must hold the BO's dma-resv lock. | |
| 348 | + */ | |
| 349 | +static inline void xe_bo_vma_count_dec_locked(struct xe_bo *bo) | |
| 350 | +{ | |
| 351 | + xe_bo_assert_held(bo); | |
| 352 | + | |
| 353 | + if (drm_gem_is_imported(&bo->ttm.base)) | |
| 354 | + return; | |
| 355 | + | |
| 356 | + xe_assert(xe_bo_device(bo), bo->purgeable.vma_count > 0); | |
| 357 | + bo->purgeable.vma_count--; | |
| 358 | +} | |
| 359 | + | |
| 276 | 360 | static inline void xe_bo_unpin_map_no_vm(struct xe_bo *bo) |
| 277 | 361 | { |
| 278 | 362 | if (likely(bo)) { |
| @@ -18,6 +18,7 @@ · 17 unmodified lines | ||
| 18 | 18 | #include "xe_ggtt_types.h" |
| 19 | 19 | |
| 20 | 20 | struct xe_device; |
| 21 | +struct xe_mem_pool_node; | |
| 21 | 22 | struct xe_vm; |
| 22 | 23 | |
| 23 | 24 | #define XE_BO_MAX_PLACEMENTS 3 |
| @@ -88,7 +89,7 @@ struct xe_bo { · 64 unmodified lines | ||
| 88 | 89 | bool ccs_cleared; |
| 89 | 90 | |
| 90 | 91 | /** @bb_ccs: BB instructions of CCS read/write. Valid only for VF */ |
| 91 | − struct xe_bb *bb_ccs[XE_SRIOV_VF_CCS_CTX_COUNT]; | |
| 92 | + struct xe_mem_pool_node *bb_ccs[XE_SRIOV_VF_CCS_CTX_COUNT]; | |
| 92 | 93 | |
| 93 | 94 | /** |
| 94 | 95 | * @cpu_caching: CPU caching mode. Currently only used for userspace |
| @@ -110,10 +111,32 @@ struct xe_bo { · 15 unmodified lines | ||
| 110 | 111 | u64 min_align; |
| 111 | 112 | |
| 112 | 113 | /** |
| 113 | − * @madv_purgeable: user space advise on BO purgeability, protected | |
| 114 | − * by BO's dma-resv lock. | |
| 114 | + * @purgeable: Purgeability state and accounting. | |
| 115 | + * | |
| 116 | + * All fields are protected by the BO's dma-resv lock. | |
| 115 | 117 | */ |
| 116 | − u32 madv_purgeable; | |
| 118 | + struct { | |
| 119 | + /** | |
| 120 | + * @purgeable.state: BO purgeability state | |
| 121 | + * (WILLNEED/DONTNEED/PURGED). | |
| 122 | + */ | |
| 123 | + u32 state; | |
| 124 | + | |
| 125 | + /** | |
| 126 | + * @purgeable.vma_count: Number of VMAs currently mapping this BO. | |
| 127 | + */ | |
| 128 | + u32 vma_count; | |
| 129 | + | |
| 130 | + /** | |
| 131 | + * @purgeable.willneed_count: Number of active WILLNEED holders. | |
| 132 | + * | |
| 133 | + * Counts WILLNEED VMAs plus active dma-buf exports for | |
| 134 | + * non-imported BOs. The BO flips to DONTNEED on a 1->0 | |
| 135 | + * transition only when VMAs still exist; if the last VMA is | |
| 136 | + * removed, the previous BO state is preserved. | |
| 137 | + */ | |
| 138 | + u32 willneed_count; | |
| 139 | + } purgeable; | |
| 117 | 140 | }; |
| 118 | 141 | |
| 119 | 142 | #endif |
| @@ -193,6 +193,18 @@ static int xe_dma_buf_begin_cpu_access(struct dma_buf *dma_buf, · 192 unmodified lines | ||
| 193 | 193 | return 0; |
| 194 | 194 | } |
| 195 | 195 | |
| 196 | +static void xe_dma_buf_release(struct dma_buf *dmabuf) | |
| 197 | +{ | |
| 198 | + struct drm_gem_object *obj = dmabuf->priv; | |
| 199 | + struct xe_bo *bo = gem_to_xe_bo(obj); | |
| 200 | + | |
| 201 | + xe_bo_lock(bo, false); | |
| 202 | + xe_bo_willneed_put_locked(bo); | |
| 203 | + xe_bo_unlock(bo); | |
| 204 | + | |
| 205 | + drm_gem_dmabuf_release(dmabuf); | |
| 206 | +} | |
| 207 | + | |
| 196 | 208 | static const struct dma_buf_ops xe_dmabuf_ops = { |
| 197 | 209 | .attach = xe_dma_buf_attach, |
| 198 | 210 | .detach = xe_dma_buf_detach, |
| @@ -200,7 +212,7 @@ static const struct dma_buf_ops xe_dmabuf_ops = { · 1 unmodified line | ||
| 200 | 212 | .unpin = xe_dma_buf_unpin, |
| 201 | 213 | .map_dma_buf = xe_dma_buf_map, |
| 202 | 214 | .unmap_dma_buf = xe_dma_buf_unmap, |
| 203 | − .release = drm_gem_dmabuf_release, | |
| 215 | + .release = xe_dma_buf_release, | |
| 204 | 216 | .begin_cpu_access = xe_dma_buf_begin_cpu_access, |
| 205 | 217 | .mmap = drm_gem_dmabuf_mmap, |
| 206 | 218 | .vmap = drm_gem_dmabuf_vmap, |
| @@ -241,26 +253,33 @@ struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags) · 34 unmodified lines | ||
| 241 | 253 | ret = -EINVAL; |
| 242 | 254 | goto out_unlock; |
| 243 | 255 | } |
| 256 | + | |
| 257 | + xe_bo_willneed_get_locked(bo); | |
| 244 | 258 | xe_bo_unlock(bo); |
| 245 | 259 | |
| 246 | 260 | ret = ttm_bo_setup_export(&bo->ttm, &ctx); |
| 247 | 261 | if (ret) |
| 248 | − return ERR_PTR(ret); | |
| 262 | + goto out_put; | |
| 249 | 263 | |
| 250 | 264 | buf = drm_gem_prime_export(obj, flags); |
| 251 | − if (!IS_ERR(buf)) | |
| 252 | − buf->ops = &xe_dmabuf_ops; | |
| 265 | + if (IS_ERR(buf)) { | |
| 266 | + ret = PTR_ERR(buf); | |
| 267 | + goto out_put; | |
| 268 | + } | |
| 253 | 269 | |
| 270 | + buf->ops = &xe_dmabuf_ops; | |
| 254 | 271 | return buf; |
| 255 | 272 | |
| 273 | +out_put: | |
| 274 | + xe_bo_lock(bo, false); | |
| 275 | + xe_bo_willneed_put_locked(bo); | |
| 256 | 276 | out_unlock: |
| 257 | 277 | xe_bo_unlock(bo); |
| 258 | 278 | return ERR_PTR(ret); |
| 259 | 279 | } |
| 260 | 280 | |
| 261 | 281 | static struct drm_gem_object * |
| 262 | −xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage, | |
| 263 | − struct dma_buf *dma_buf) | |
| 282 | +xe_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf) | |
| 264 | 283 | { |
| 265 | 284 | struct dma_resv *resv = dma_buf->resv; |
| 266 | 285 | struct xe_device *xe = to_xe_device(dev); |
| @@ -281,7 +300,7 @@ xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage, · 14 unmodified lines | ||
| 281 | 300 | if (ret) |
| 282 | 301 | break; |
| 283 | 302 | |
| 284 | − bo = xe_bo_init_locked(xe, storage, NULL, resv, NULL, dma_buf->size, | |
| 303 | + bo = xe_bo_init_locked(xe, NULL, NULL, resv, NULL, dma_buf->size, | |
| 285 | 304 | 0, /* Will require 1way or 2way for vm_bind */ |
| 286 | 305 | ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, &exec); |
| 287 | 306 | drm_exec_retry_on_contention(&exec); |
| @@ -332,7 +351,6 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 44 unmodified lines | ||
| 332 | 351 | const struct dma_buf_attach_ops *attach_ops; |
| 333 | 352 | struct dma_buf_attachment *attach; |
| 334 | 353 | struct drm_gem_object *obj; |
| 335 | − struct xe_bo *bo; | |
| 336 | 354 | |
| 337 | 355 | if (dma_buf->ops == &xe_dmabuf_ops) { |
| 338 | 356 | obj = dma_buf->priv; |
| @@ -348,13 +366,15 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 9 unmodified lines | ||
| 348 | 366 | } |
| 349 | 367 | |
| 350 | 368 | /* |
| 351 | − * Don't publish the bo until we have a valid attachment, and a | |
| 352 | − * valid attachment needs the bo address. So pre-create a bo before | |
| 353 | − * creating the attachment and publish. | |
| 369 | + * This needs to happen before the attach, since it will create a new | |
| 370 | + * attachment for this, and add it to the list of attachments, at which | |
| 371 | + * point it is globally visible, and at any point the export side can | |
| 372 | + * call into on invalidate_mappings callback, which require a working | |
| 373 | + * object. | |
| 354 | 374 | */ |
| 355 | − bo = xe_bo_alloc(); | |
| 356 | − if (IS_ERR(bo)) | |
| 357 | − return ERR_CAST(bo); | |
| 375 | + obj = xe_dma_buf_create_obj(dev, dma_buf); | |
| 376 | + if (IS_ERR(obj)) | |
| 377 | + return obj; | |
| 358 | 378 | |
| 359 | 379 | attach_ops = &xe_dma_buf_attach_ops; |
| 360 | 380 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) |
| @@ -362,26 +382,15 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev, · 1 unmodified line | ||
| 362 | 382 | attach_ops = test->attach_ops; |
| 363 | 383 | #endif |
| 364 | 384 | |
| 365 | − attach = dma_buf_dynamic_attach(dma_buf, dev->dev, attach_ops, &bo->ttm.base); | |
| 385 | + attach = dma_buf_dynamic_attach(dma_buf, dev->dev, attach_ops, obj); | |
| 366 | 386 | if (IS_ERR(attach)) { |
| 367 | − obj = ERR_CAST(attach); | |
| 368 | − goto out_err; | |
| 387 | + xe_bo_put(gem_to_xe_bo(obj)); | |
| 388 | + return ERR_CAST(attach); | |
| 369 | 389 | } |
| 370 | 390 | |
| 371 | − /* Errors here will take care of freeing the bo. */ | |
| 372 | − obj = xe_dma_buf_init_obj(dev, bo, dma_buf); | |
| 373 | − if (IS_ERR(obj)) | |
| 374 | − return obj; | |
| 375 | − | |
| 376 | − | |
| 377 | 391 | get_dma_buf(dma_buf); |
| 378 | 392 | obj->import_attach = attach; |
| 379 | 393 | return obj; |
| 380 | − | |
| 381 | −out_err: | |
| 382 | − xe_bo_free(bo); | |
| 383 | − | |
| 384 | − return obj; | |
| 385 | 394 | } |
| 386 | 395 | |
| 387 | 396 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) |
| @@ -869,14 +869,14 @@ static int xe_eu_stall_stream_close(struct inode *inode, struct file *file) · 868 unmodified lines | ||
| 869 | 869 | struct xe_eu_stall_data_stream *stream = file->private_data; |
| 870 | 870 | struct xe_gt *gt = stream->gt; |
| 871 | 871 | |
| 872 | − drm_dev_put(>->tile->xe->drm); | |
| 873 | − | |
| 874 | 872 | mutex_lock(>->eu_stall->stream_lock); |
| 875 | 873 | xe_eu_stall_disable_locked(stream); |
| 876 | 874 | xe_eu_stall_data_buf_destroy(stream); |
| 877 | 875 | xe_eu_stall_stream_free(stream); |
| 878 | 876 | mutex_unlock(>->eu_stall->stream_lock); |
| 879 | 877 | |
| 878 | + drm_dev_put(>->tile->xe->drm); | |
| 879 | + | |
| 880 | 880 | return 0; |
| 881 | 881 | } |
| 882 | 882 | |
| @@ -1405,7 +1405,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 1404 unmodified lines | ||
| 1405 | 1405 | if (q->vm && q->hwe->hw_engine_group) { |
| 1406 | 1406 | err = xe_hw_engine_group_add_exec_queue(q->hwe->hw_engine_group, q); |
| 1407 | 1407 | if (err) |
| 1408 | − goto put_exec_queue; | |
| 1408 | + goto kill_exec_queue; | |
| 1409 | 1409 | } |
| 1410 | 1410 | } |
| 1411 | 1411 | |
| @@ -1416,12 +1416,15 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 4 unmodified lines | ||
| 1416 | 1416 | /* user id alloc must always be last in ioctl to prevent UAF */ |
| 1417 | 1417 | err = xa_alloc(&xef->exec_queue.xa, &id, q, xa_limit_32b, GFP_KERNEL); |
| 1418 | 1418 | if (err) |
| 1419 | − goto kill_exec_queue; | |
| 1419 | + goto del_hw_engine_group; | |
| 1420 | 1420 | |
| 1421 | 1421 | args->exec_queue_id = id; |
| 1422 | 1422 | |
| 1423 | 1423 | return 0; |
| 1424 | 1424 | |
| 1425 | +del_hw_engine_group: | |
| 1426 | + if (q->vm && q->hwe && q->hwe->hw_engine_group) | |
| 1427 | + xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q); | |
| 1425 | 1428 | kill_exec_queue: |
| 1426 | 1429 | xe_exec_queue_kill(q); |
| 1427 | 1430 | delete_queue_group: |
| @@ -1760,7 +1763,7 @@ void xe_exec_queue_tlb_inval_last_fence_put(struct xe_exec_queue *q, · 332 unmodified lines | ||
| 1760 | 1763 | void xe_exec_queue_tlb_inval_last_fence_put_unlocked(struct xe_exec_queue *q, |
| 1761 | 1764 | unsigned int type) |
| 1762 | 1765 | { |
| 1763 | − xe_assert(q->vm->xe, type == XE_EXEC_QUEUE_TLB_INVAL_MEDIA_GT || | |
| 1766 | + xe_assert(gt_to_xe(q->gt), type == XE_EXEC_QUEUE_TLB_INVAL_MEDIA_GT || | |
| 1764 | 1767 | type == XE_EXEC_QUEUE_TLB_INVAL_PRIMARY_GT); |
| 1765 | 1768 | |
| 1766 | 1769 | dma_fence_put(q->tlb_inval[type].last_fence); |
| @@ -166,7 +166,7 @@ static int query_compatibility_version(struct xe_gsc *gsc) · 165 unmodified lines | ||
| 166 | 166 | &rd_offset); |
| 167 | 167 | if (err) { |
| 168 | 168 | xe_gt_err(gt, "HuC: invalid GSC reply for version query (err=%d)\n", err); |
| 169 | − return err; | |
| 169 | + goto out_bo; | |
| 170 | 170 | } |
| 171 | 171 | |
| 172 | 172 | compat->major = version_query_rd(xe, &bo->vmap, rd_offset, proj_major); |
| @@ -482,8 +482,7 @@ int xe_gsc_init_post_hwconfig(struct xe_gsc *gsc) · 309 unmodified lines | ||
| 482 | 482 | EXEC_QUEUE_FLAG_PERMANENT, 0); |
| 483 | 483 | if (IS_ERR(q)) { |
| 484 | 484 | xe_gt_err(gt, "Failed to create queue for GSC submission\n"); |
| 485 | − err = PTR_ERR(q); | |
| 486 | − goto out_bo; | |
| 485 | + return PTR_ERR(q); | |
| 487 | 486 | } |
| 488 | 487 | |
| 489 | 488 | wq = alloc_ordered_workqueue("gsc-ordered-wq", 0); |
| @@ -506,8 +505,6 @@ int xe_gsc_init_post_hwconfig(struct xe_gsc *gsc) · 16 unmodified lines | ||
| 506 | 505 | |
| 507 | 506 | out_q: |
| 508 | 507 | xe_exec_queue_put(q); |
| 509 | −out_bo: | |
| 510 | − xe_bo_unpin_map_no_vm(bo); | |
| 511 | 508 | return err; |
| 512 | 509 | } |
| 513 | 510 | |
| @@ -385,10 +385,10 @@ static int pf_migration_mmio_save(struct xe_gt *gt, unsigned int vfid, void *buf · 384 unmodified lines | ||
| 385 | 385 | |
| 386 | 386 | if (xe_gt_is_media_type(gt)) |
| 387 | 387 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) |
| 388 | − regs[n] = xe_mmio_read32(>->mmio, MED_VF_SW_FLAG(n)); | |
| 388 | + regs[n] = xe_mmio_read32(&mmio, MED_VF_SW_FLAG(n)); | |
| 389 | 389 | else |
| 390 | 390 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) |
| 391 | − regs[n] = xe_mmio_read32(>->mmio, VF_SW_FLAG(n)); | |
| 391 | + regs[n] = xe_mmio_read32(&mmio, VF_SW_FLAG(n)); | |
| 392 | 392 | |
| 393 | 393 | return 0; |
| 394 | 394 | } |
| @@ -407,10 +407,10 @@ static int pf_migration_mmio_restore(struct xe_gt *gt, unsigned int vfid, · 12 unmodified lines | ||
| 407 | 407 | |
| 408 | 408 | if (xe_gt_is_media_type(gt)) |
| 409 | 409 | for (n = 0; n < MED_VF_SW_FLAG_COUNT; n++) |
| 410 | − xe_mmio_write32(>->mmio, MED_VF_SW_FLAG(n), regs[n]); | |
| 410 | + xe_mmio_write32(&mmio, MED_VF_SW_FLAG(n), regs[n]); | |
| 411 | 411 | else |
| 412 | 412 | for (n = 0; n < VF_SW_FLAG_COUNT; n++) |
| 413 | − xe_mmio_write32(>->mmio, VF_SW_FLAG(n), regs[n]); | |
| 413 | + xe_mmio_write32(&mmio, VF_SW_FLAG(n), regs[n]); | |
| 414 | 414 | |
| 415 | 415 | return 0; |
| 416 | 416 | } |
| @@ -114,8 +114,10 @@ int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt *gt, const u32 *msg, u32 · 113 unmodified lines | ||
| 114 | 114 | * VFs with no events are not printed. |
| 115 | 115 | * |
| 116 | 116 | * This function can only be called on PF. |
| 117 | + * | |
| 118 | + * Return: always 0 | |
| 117 | 119 | */ |
| 118 | −void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p) | |
| 120 | +int xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p) | |
| 119 | 121 | { |
| 120 | 122 | unsigned int n, total_vfs = xe_gt_sriov_pf_get_totalvfs(gt); |
| 121 | 123 | const struct xe_gt_sriov_monitor *data; |
| @@ -144,4 +146,6 @@ void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p · 22 unmodified lines | ||
| 144 | 146 | #undef __format |
| 145 | 147 | #undef __value |
| 146 | 148 | } |
| 149 | + | |
| 150 | + return 0; | |
| 147 | 151 | } |
| @@ -13,7 +13,7 @@ struct drm_printer; · 12 unmodified lines | ||
| 13 | 13 | struct xe_gt; |
| 14 | 14 | |
| 15 | 15 | void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid); |
| 16 | −void xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p); | |
| 16 | +int xe_gt_sriov_pf_monitor_print_events(struct xe_gt *gt, struct drm_printer *p); | |
| 17 | 17 | |
| 18 | 18 | #ifdef CONFIG_PCI_IOV |
| 19 | 19 | int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt *gt, const u32 *msg, u32 len); |
| @@ -1137,13 +1137,15 @@ void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val) · 1136 unmodified lines | ||
| 1137 | 1137 | } |
| 1138 | 1138 | |
| 1139 | 1139 | /** |
| 1140 | − * xe_gt_sriov_vf_print_config - Print VF self config. | |
| 1140 | + * xe_gt_sriov_vf_print_config() - Print VF self config. | |
| 1141 | 1141 | * @gt: the &xe_gt |
| 1142 | 1142 | * @p: the &drm_printer |
| 1143 | 1143 | * |
| 1144 | 1144 | * This function is for VF use only. |
| 1145 | + * | |
| 1146 | + * Return: always 0. | |
| 1145 | 1147 | */ |
| 1146 | −void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) | |
| 1148 | +int xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) | |
| 1147 | 1149 | { |
| 1148 | 1150 | struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; |
| 1149 | 1151 | struct xe_device *xe = gt_to_xe(gt); |
| @@ -1170,16 +1172,20 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) · 20 unmodified lines | ||
| 1170 | 1172 | |
| 1171 | 1173 | drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs); |
| 1172 | 1174 | drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs); |
| 1175 | + | |
| 1176 | + return 0; | |
| 1173 | 1177 | } |
| 1174 | 1178 | |
| 1175 | 1179 | /** |
| 1176 | − * xe_gt_sriov_vf_print_runtime - Print VF's runtime regs received from PF. | |
| 1180 | + * xe_gt_sriov_vf_print_runtime() - Print VF's runtime regs received from PF. | |
| 1177 | 1181 | * @gt: the &xe_gt |
| 1178 | 1182 | * @p: the &drm_printer |
| 1179 | 1183 | * |
| 1180 | 1184 | * This function is for VF use only. |
| 1185 | + * | |
| 1186 | + * Return: always 0. | |
| 1181 | 1187 | */ |
| 1182 | −void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) | |
| 1188 | +int xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) | |
| 1183 | 1189 | { |
| 1184 | 1190 | struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs; |
| 1185 | 1191 | unsigned int size = gt->sriov.vf.runtime.num_regs; |
| @@ -1188,16 +1194,20 @@ void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p) · 2 unmodified lines | ||
| 1188 | 1194 | |
| 1189 | 1195 | for (; size--; vf_regs++) |
| 1190 | 1196 | drm_printf(p, "%#x = %#x\n", vf_regs->offset, vf_regs->value); |
| 1197 | + | |
| 1198 | + return 0; | |
| 1191 | 1199 | } |
| 1192 | 1200 | |
| 1193 | 1201 | /** |
| 1194 | − * xe_gt_sriov_vf_print_version - Print VF ABI versions. | |
| 1202 | + * xe_gt_sriov_vf_print_version() - Print VF ABI versions. | |
| 1195 | 1203 | * @gt: the &xe_gt |
| 1196 | 1204 | * @p: the &drm_printer |
| 1197 | 1205 | * |
| 1198 | 1206 | * This function is for VF use only. |
| 1207 | + * | |
| 1208 | + * Return: always 0. | |
| 1199 | 1209 | */ |
| 1200 | −void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) | |
| 1210 | +int xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) | |
| 1201 | 1211 | { |
| 1202 | 1212 | struct xe_device *xe = gt_to_xe(gt); |
| 1203 | 1213 | struct xe_uc_fw_version *guc_version = >->sriov.vf.guc_version; |
| @@ -1227,6 +1237,8 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · 23 unmodified lines | ||
| 1227 | 1237 | GUC_RELAY_VERSION_LATEST_MAJOR, GUC_RELAY_VERSION_LATEST_MINOR); |
| 1228 | 1238 | drm_printf(p, "\thandshake:\t%u.%u\n", |
| 1229 | 1239 | pf_version->major, pf_version->minor); |
| 1240 | + | |
| 1241 | + return 0; | |
| 1230 | 1242 | } |
| 1231 | 1243 | |
| 1232 | 1244 | static bool vf_post_migration_shutdown(struct xe_gt *gt) |
| @@ -35,9 +35,9 @@ bool xe_gt_sriov_vf_sched_groups_enabled(struct xe_gt *gt); · 34 unmodified lines | ||
| 35 | 35 | u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg); |
| 36 | 36 | void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val); |
| 37 | 37 | |
| 38 | −void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p); | |
| 39 | −void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p); | |
| 40 | −void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); | |
| 38 | +int xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p); | |
| 39 | +int xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p); | |
| 40 | +int xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); | |
| 41 | 41 | |
| 42 | 42 | int xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt); |
| 43 | 43 | int xe_vf_migration_fixups_complete_count(struct xe_gt *gt); |
| @@ -144,6 +144,13 @@ struct xe_gt { · 143 unmodified lines | ||
| 144 | 144 | u8 id; |
| 145 | 145 | /** @info.has_indirect_ring_state: GT has indirect ring state support */ |
| 146 | 146 | u8 has_indirect_ring_state:1; |
| 147 | + /** | |
| 148 | + * @info.has_xe2_blt_instructions: GT supports Xe2-style MEM_SET | |
| 149 | + * and MEM_COPY blitter functionality. Note that despite the | |
| 150 | + * name, some Xe1 platforms may also support this "Xe2-style" | |
| 151 | + * feature. | |
| 152 | + */ | |
| 153 | + u8 has_xe2_blt_instructions:1; | |
| 147 | 154 | /** |
| 148 | 155 | * @info.num_geometry_xecore_fuse_regs: Number of 32b-bit fuse |
| 149 | 156 | * registers the geometry XeCore mask spans. |
| @@ -512,12 +512,9 @@ static void guc_golden_lrc_init(struct xe_guc_ads *ads) · 511 unmodified lines | ||
| 512 | 512 | * that starts after the execlists LRC registers. This is |
| 513 | 513 | * required to allow the GuC to restore just the engine state |
| 514 | 514 | * when a watchdog reset occurs. |
| 515 | − * We calculate the engine state size by removing the size of | |
| 516 | − * what comes before it in the context image (which is identical | |
| 517 | − * on all engines). | |
| 518 | 515 | */ |
| 519 | 516 | ads_blob_write(ads, ads.eng_state_size[guc_class], |
| 520 | − real_size - xe_lrc_skip_size(xe)); | |
| 517 | + xe_lrc_engine_state_size(gt, class)); | |
| 521 | 518 | ads_blob_write(ads, ads.golden_context_lrca[guc_class], |
| 522 | 519 | addr_ggtt); |
| 523 | 520 | |
| @@ -770,6 +767,11 @@ static unsigned int guc_mmio_regset_write(struct xe_guc_ads *ads, · 246 unmodified lines | ||
| 770 | 767 | } |
| 771 | 768 | } |
| 772 | 769 | |
| 770 | + if (XE_GT_WA(hwe->gt, 16023105232)) | |
| 771 | + guc_mmio_regset_write_one(ads, regset_map, | |
| 772 | + RING_IDLEDLY(hwe->mmio_base), | |
| 773 | + count++); | |
| 774 | + | |
| 773 | 775 | return count; |
| 774 | 776 | } |
| 775 | 777 | |
| @@ -259,24 +259,12 @@ static void guc_submit_sw_fini(struct drm_device *drm, void *arg) · 258 unmodified lines | ||
| 259 | 259 | } |
| 260 | 260 | |
| 261 | 261 | static void guc_submit_fini(void *arg) |
| 262 | −{ | |
| 263 | − struct xe_guc *guc = arg; | |
| 264 | − | |
| 265 | − /* Forcefully kill any remaining exec queues */ | |
| 266 | − xe_guc_ct_stop(&guc->ct); | |
| 267 | − guc_submit_reset_prepare(guc); | |
| 268 | − xe_guc_softreset(guc); | |
| 269 | − xe_guc_submit_stop(guc); | |
| 270 | − xe_uc_fw_sanitize(&guc->fw); | |
| 271 | − xe_guc_submit_pause_abort(guc); | |
| 272 | −} | |
| 273 | − | |
| 274 | −static void guc_submit_wedged_fini(void *arg) | |
| 275 | 262 | { |
| 276 | 263 | struct xe_guc *guc = arg; |
| 277 | 264 | struct xe_exec_queue *q; |
| 278 | 265 | unsigned long index; |
| 279 | 266 | |
| 267 | + /* Drop any wedged queue refs */ | |
| 280 | 268 | mutex_lock(&guc->submission_state.lock); |
| 281 | 269 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) { |
| 282 | 270 | if (exec_queue_wedged(q)) { |
| @@ -286,6 +274,14 @@ static void guc_submit_wedged_fini(void *arg) · 3 unmodified lines | ||
| 286 | 274 | } |
| 287 | 275 | } |
| 288 | 276 | mutex_unlock(&guc->submission_state.lock); |
| 277 | + | |
| 278 | + /* Forcefully kill any remaining exec queues */ | |
| 279 | + xe_guc_ct_stop(&guc->ct); | |
| 280 | + guc_submit_reset_prepare(guc); | |
| 281 | + xe_guc_softreset(guc); | |
| 282 | + xe_guc_submit_stop(guc); | |
| 283 | + xe_uc_fw_sanitize(&guc->fw); | |
| 284 | + xe_guc_submit_pause_abort(guc); | |
| 289 | 285 | } |
| 290 | 286 | |
| 291 | 287 | static const struct xe_exec_queue_ops guc_exec_queue_ops; |
| @@ -1320,10 +1316,8 @@ static void disable_scheduling_deregister(struct xe_guc *guc, · 1028 unmodified lines | ||
| 1320 | 1316 | void xe_guc_submit_wedge(struct xe_guc *guc) |
| 1321 | 1317 | { |
| 1322 | 1318 | struct xe_device *xe = guc_to_xe(guc); |
| 1323 | − struct xe_gt *gt = guc_to_gt(guc); | |
| 1324 | 1319 | struct xe_exec_queue *q; |
| 1325 | 1320 | unsigned long index; |
| 1326 | − int err; | |
| 1327 | 1321 | |
| 1328 | 1322 | xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode); |
| 1329 | 1323 | |
| @@ -1335,15 +1329,6 @@ void xe_guc_submit_wedge(struct xe_guc *guc) · 5 unmodified lines | ||
| 1335 | 1329 | return; |
| 1336 | 1330 | |
| 1337 | 1331 | if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) { |
| 1338 | − err = devm_add_action_or_reset(guc_to_xe(guc)->drm.dev, | |
| 1339 | − guc_submit_wedged_fini, guc); | |
| 1340 | − if (err) { | |
| 1341 | − xe_gt_err(gt, "Failed to register clean-up on wedged.mode=%s; " | |
| 1342 | − "Although device is wedged.\n", | |
| 1343 | − xe_wedged_mode_to_string(XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)); | |
| 1344 | − return; | |
| 1345 | − } | |
| 1346 | − | |
| 1347 | 1332 | mutex_lock(&guc->submission_state.lock); |
| 1348 | 1333 | xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) |
| 1349 | 1334 | if (xe_exec_queue_get_unless_zero(q)) |
| @@ -1688,6 +1673,14 @@ static void guc_exec_queue_fini(struct xe_exec_queue *q) · 338 unmodified lines | ||
| 1688 | 1673 | struct xe_guc_exec_queue *ge = q->guc; |
| 1689 | 1674 | struct xe_guc *guc = exec_queue_to_guc(q); |
| 1690 | 1675 | |
| 1676 | + if (xe_exec_queue_is_multi_queue_secondary(q)) { | |
| 1677 | + struct xe_exec_queue_group *group = q->multi_queue.group; | |
| 1678 | + | |
| 1679 | + mutex_lock(&group->list_lock); | |
| 1680 | + list_del(&q->multi_queue.link); | |
| 1681 | + mutex_unlock(&group->list_lock); | |
| 1682 | + } | |
| 1683 | + | |
| 1691 | 1684 | release_guc_id(guc, q); |
| 1692 | 1685 | xe_sched_entity_fini(&ge->entity); |
| 1693 | 1686 | xe_sched_fini(&ge->sched); |
| @@ -1709,14 +1702,6 @@ static void __guc_exec_queue_destroy_async(struct work_struct *w) · 15 unmodified lines | ||
| 1709 | 1702 | guard(xe_pm_runtime)(guc_to_xe(guc)); |
| 1710 | 1703 | trace_xe_exec_queue_destroy(q); |
| 1711 | 1704 | |
| 1712 | − if (xe_exec_queue_is_multi_queue_secondary(q)) { | |
| 1713 | − struct xe_exec_queue_group *group = q->multi_queue.group; | |
| 1714 | − | |
| 1715 | − mutex_lock(&group->list_lock); | |
| 1716 | − list_del(&q->multi_queue.link); | |
| 1717 | − mutex_unlock(&group->list_lock); | |
| 1718 | − } | |
| 1719 | − | |
| 1720 | 1705 | /* Confirm no work left behind accessing device structures */ |
| 1721 | 1706 | cancel_delayed_work_sync(&ge->sched.base.work_tdr); |
| 1722 | 1707 | |
| @@ -746,9 +746,16 @@ size_t xe_lrc_reg_size(struct xe_device *xe) · 745 unmodified lines | ||
| 746 | 746 | return 80 * sizeof(u32); |
| 747 | 747 | } |
| 748 | 748 | |
| 749 | −size_t xe_lrc_skip_size(struct xe_device *xe) | |
| 749 | +/** | |
| 750 | + * xe_lrc_engine_state_size() - Get size of the engine state within LRC | |
| 751 | + * @gt: the &xe_gt struct instance | |
| 752 | + * @class: Hardware engine class | |
| 753 | + * | |
| 754 | + * Returns: Size of the engine state | |
| 755 | + */ | |
| 756 | +size_t xe_lrc_engine_state_size(struct xe_gt *gt, enum xe_engine_class class) | |
| 750 | 757 | { |
| 751 | − return LRC_PPHWSP_SIZE + xe_lrc_reg_size(xe); | |
| 758 | + return xe_gt_lrc_hang_replay_size(gt, class) - xe_lrc_reg_size(gt_to_xe(gt)); | |
| 752 | 759 | } |
| 753 | 760 | |
| 754 | 761 | static inline u32 __xe_lrc_seqno_offset(struct xe_lrc *lrc) |
| @@ -1214,7 +1221,7 @@ static ssize_t setup_invalidate_state_cache_wa(struct xe_lrc *lrc, · 459 unmodified lines | ||
| 1214 | 1221 | if (xe_gt_WARN_ON(lrc->gt, max_len < 3)) |
| 1215 | 1222 | return -ENOSPC; |
| 1216 | 1223 | |
| 1217 | − *cmd++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | |
| 1224 | + *cmd++ = MI_LOAD_REGISTER_IMM | MI_LRI_LRM_CS_MMIO | MI_LRI_NUM_REGS(1); | |
| 1218 | 1225 | *cmd++ = CS_DEBUG_MODE2(0).addr; |
| 1219 | 1226 | *cmd++ = REG_MASKED_FIELD_ENABLE(INSTRUCTION_STATE_CACHE_INVALIDATE); |
| 1220 | 1227 | |
| @@ -130,7 +130,7 @@ u32 xe_lrc_parallel_ggtt_addr(struct xe_lrc *lrc); · 129 unmodified lines | ||
| 130 | 130 | struct iosys_map xe_lrc_parallel_map(struct xe_lrc *lrc); |
| 131 | 131 | |
| 132 | 132 | size_t xe_lrc_reg_size(struct xe_device *xe); |
| 133 | −size_t xe_lrc_skip_size(struct xe_device *xe); | |
| 133 | +size_t xe_lrc_engine_state_size(struct xe_gt *gt, enum xe_engine_class class); | |
| 134 | 134 | |
| 135 | 135 | void xe_lrc_dump_default(struct drm_printer *p, |
| 136 | 136 | struct xe_gt *gt, |
| @@ -0,0 +1,403 @@ | ||
| 1 | +// SPDX-License-Identifier: MIT | |
| 2 | +/* | |
| 3 | + * Copyright © 2026 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#include <linux/kernel.h> | |
| 7 | + | |
| 8 | +#include <drm/drm_managed.h> | |
| 9 | + | |
| 10 | +#include "instructions/xe_mi_commands.h" | |
| 11 | +#include "xe_bo.h" | |
| 12 | +#include "xe_device_types.h" | |
| 13 | +#include "xe_map.h" | |
| 14 | +#include "xe_mem_pool.h" | |
| 15 | +#include "xe_mem_pool_types.h" | |
| 16 | +#include "xe_tile_printk.h" | |
| 17 | + | |
| 18 | +/** | |
| 19 | + * struct xe_mem_pool - DRM MM pool for sub-allocating memory from a BO on an | |
| 20 | + * XE tile. | |
| 21 | + * | |
| 22 | + * The XE memory pool is a DRM MM manager that provides sub-allocation of memory | |
| 23 | + * from a backing buffer object (BO) on a specific XE tile. It is designed to | |
| 24 | + * manage memory for GPU workloads, allowing for efficient allocation and | |
| 25 | + * deallocation of memory regions within the BO. | |
| 26 | + * | |
| 27 | + * The memory pool maintains a primary BO that is pinned in the GGTT and mapped | |
| 28 | + * into the CPU address space for direct access. Optionally, it can also maintain | |
| 29 | + * a shadow BO that can be used for atomic updates to the primary BO's contents. | |
| 30 | + * | |
| 31 | + * The API provided by the memory pool allows clients to allocate and free memory | |
| 32 | + * regions, retrieve GPU and CPU addresses, and synchronize data between the | |
| 33 | + * primary and shadow BOs as needed. | |
| 34 | + */ | |
| 35 | +struct xe_mem_pool { | |
| 36 | + /** @base: Range allocator over [0, @size) in bytes */ | |
| 37 | + struct drm_mm base; | |
| 38 | + /** @bo: Active pool BO (GGTT-pinned, CPU-mapped). */ | |
| 39 | + struct xe_bo *bo; | |
| 40 | + /** @shadow: Shadow BO for atomic command updates. */ | |
363 more lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 140 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 131 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 209 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 554 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 380 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 311 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 386 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 1544 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 303 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 653 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 439 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 168 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 325 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 271 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 415 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 485 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 192 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 279 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 193 lines aren’t shown here.
This diff is large: its 303 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 324 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 201 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 613 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 340 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 236 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 465 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 310 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 1465 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 330 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 228 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 512 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 181 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 208 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 176 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 226 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 784 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 301 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 249 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 686 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 383 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 792 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 213 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 332 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 143 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 295 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 323 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 395 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 184 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 143 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 157 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 450 lines aren’t shown here.
This diff is large: its 210 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 277 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 211 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 204 lines aren’t shown here.
This diff is large: its 429 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 241 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 806 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 726 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 156 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 407 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 874 lines aren’t shown here.
This diff is large: its 267 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 171 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 726 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 468 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 243 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 260 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 296 lines aren’t shown here.
This diff is large: its 291 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 211 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 217 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 622 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.