We are applying further, lower-prio fixes on top of two ww_mutex fixes in locking/urgent.
Unified · Split
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | |||
| 23 | - enum: | 23 | - enum: |
| 24 | - ingenic,jz4775-intc | 24 | - ingenic,jz4775-intc |
| 25 | - ingenic,jz4770-intc | 25 | - ingenic,jz4770-intc |
| 26 | - ingenic,jz4760b-intc | ||
| 26 | - const: ingenic,jz4760-intc | 27 | - const: ingenic,jz4760-intc |
| 27 | - items: | 28 | - items: |
| 28 | - const: ingenic,x1000-intc | 29 | - const: ingenic,x1000-intc |
| @@ -613,6 +613,27 @@ Some of these date from the very introduction of KMS in 2008 ... · 612 unmodified lines | |||
| 613 | 613 | ||
| 614 | Level: Intermediate | 614 | Level: Intermediate |
| 615 | 615 | ||
| 616 | Remove automatic page mapping from dma-buf importing | ||
| 617 | ---------------------------------------------------- | ||
| 618 | |||
| 619 | When importing dma-bufs, the dma-buf and PRIME frameworks automatically map | ||
| 620 | imported pages into the importer's DMA area. drm_gem_prime_fd_to_handle() and | ||
| 621 | drm_gem_prime_handle_to_fd() require that importers call dma_buf_attach() | ||
| 622 | even if they never do actual device DMA, but only CPU access through | ||
| 623 | dma_buf_vmap(). This is a problem for USB devices, which do not support DMA | ||
| 624 | operations. | ||
| 625 | |||
| 626 | To fix the issue, automatic page mappings should be removed from the | ||
| 627 | buffer-sharing code. Fixing this is a bit more involved, since the import/export | ||
| 628 | cache is also tied to &drm_gem_object.import_attach. Meanwhile we paper over | ||
| 629 | this problem for USB devices by fishing out the USB host controller device, as | ||
| 630 | long as that supports DMA. Otherwise importing can still needlessly fail. | ||
| 631 | |||
| 632 | Contact: Thomas Zimmermann <tzimmermann@suse.de>, Daniel Vetter | ||
| 633 | |||
| 634 | Level: Advanced | ||
| 635 | |||
| 636 | |||
| 616 | Better Testing | 637 | Better Testing |
| 617 | ============== | 638 | ============== |
| 618 | 639 | ||
| @@ -1988,7 +1988,7 @@ netif_carrier. · 1987 unmodified lines | |||
| 1988 | If use_carrier is 0, then the MII monitor will first query the | 1988 | If use_carrier is 0, then the MII monitor will first query the |
| 1989 | device's (via ioctl) MII registers and check the link state. If that | 1989 | device's (via ioctl) MII registers and check the link state. If that |
| 1990 | request fails (not just that it returns carrier down), then the MII | 1990 | request fails (not just that it returns carrier down), then the MII |
| 1991 | monitor will make an ethtool ETHOOL_GLINK request to attempt to obtain | 1991 | monitor will make an ethtool ETHTOOL_GLINK request to attempt to obtain |
| 1992 | the same information. If both methods fail (i.e., the driver either | 1992 | the same information. If both methods fail (i.e., the driver either |
| 1993 | does not support or had some error in processing both the MII register | 1993 | does not support or had some error in processing both the MII register |
| 1994 | and ethtool requests), then the MII monitor will assume the link is | 1994 | and ethtool requests), then the MII monitor will assume the link is |
| @@ -142,73 +142,13 @@ Please send incremental versions on top of what has been merged in order to fix · 141 unmodified lines | |||
| 142 | the patches the way they would look like if your latest patch series was to be | 142 | the patches the way they would look like if your latest patch series was to be |
| 143 | merged. | 143 | merged. |
| 144 | 144 | ||
| 145 | How can I tell what patches are queued up for backporting to the various stable releases? | 145 | Are there special rules regarding stable submissions on netdev? |
| 146 | ----------------------------------------------------------------------------------------- | ||
| 147 | Normally Greg Kroah-Hartman collects stable commits himself, but for | ||
| 148 | networking, Dave collects up patches he deems critical for the | ||
| 149 | networking subsystem, and then hands them off to Greg. | ||
| 150 | |||
| 151 | There is a patchworks queue that you can see here: | ||
| 152 | |||
| 153 | https://patchwork.kernel.org/bundle/netdev/stable/?state=* | ||
| 154 | |||
| 155 | It contains the patches which Dave has selected, but not yet handed off | ||
| 156 | to Greg. If Greg already has the patch, then it will be here: | ||
| 157 | |||
| 158 | https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git | ||
| 159 | |||
| 160 | A quick way to find whether the patch is in this stable-queue is to | ||
| 161 | simply clone the repo, and then git grep the mainline commit ID, e.g. | ||
| 162 | :: | ||
| 163 | |||
| 164 | stable-queue$ git grep -l 284041ef21fdf2e | ||
| 165 | releases/3.0.84/ipv6-fix-possible-crashes-in-ip6_cork_release.patch | ||
| 166 | releases/3.4.51/ipv6-fix-possible-crashes-in-ip6_cork_release.patch | ||
| 167 | releases/3.9.8/ipv6-fix-possible-crashes-in-ip6_cork_release.patch | ||
| 168 | stable/stable-queue$ | ||
| 169 | |||
| 170 | I see a network patch and I think it should be backported to stable. Should I request it via stable@vger.kernel.org like the references in the kernel's Documentation/process/stable-kernel-rules.rst file say? | ||
| 171 | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ||
| 172 | No, not for networking. Check the stable queues as per above first | ||
| 173 | to see if it is already queued. If not, then send a mail to netdev, | ||
| 174 | listing the upstream commit ID and why you think it should be a stable | ||
| 175 | candidate. | ||
| 176 | |||
| 177 | Before you jump to go do the above, do note that the normal stable rules | ||
| 178 | in :ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>` | ||
| 179 | still apply. So you need to explicitly indicate why it is a critical | ||
| 180 | fix and exactly what users are impacted. In addition, you need to | ||
| 181 | convince yourself that you *really* think it has been overlooked, | ||
| 182 | vs. having been considered and rejected. | ||
| 183 | |||
| 184 | Generally speaking, the longer it has had a chance to "soak" in | ||
| 185 | mainline, the better the odds that it is an OK candidate for stable. So | ||
| 186 | scrambling to request a commit be added the day after it appears should | ||
| 187 | be avoided. | ||
| 188 | |||
| 189 | I have created a network patch and I think it should be backported to stable. Should I add a Cc: stable@vger.kernel.org like the references in the kernel's Documentation/ directory say? | ||
| 190 | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ||
| 191 | No. See above answer. In short, if you think it really belongs in | ||
| 192 | stable, then ensure you write a decent commit log that describes who | ||
| 193 | gets impacted by the bug fix and how it manifests itself, and when the | ||
| 194 | bug was introduced. If you do that properly, then the commit will get | ||
| 195 | handled appropriately and most likely get put in the patchworks stable | ||
| 196 | queue if it really warrants it. | ||
| 197 | |||
| 198 | If you think there is some valid information relating to it being in | ||
| 199 | stable that does *not* belong in the commit log, then use the three dash | ||
| 200 | marker line as described in | ||
| 201 | :ref:`Documentation/process/submitting-patches.rst <the_canonical_patch_format>` | ||
| 202 | to temporarily embed that information into the patch that you send. | ||
| 203 | |||
| 204 | Are all networking bug fixes backported to all stable releases? | ||
| 205 | --------------------------------------------------------------- | 146 | --------------------------------------------------------------- |
| 206 | Due to capacity, Dave could only take care of the backports for the | 147 | While it used to be the case that netdev submissions were not supposed |
| 207 | last two stable releases. For earlier stable releases, each stable | 148 | to carry explicit ``CC: stable@vger.kernel.org`` tags that is no longer |
| 208 | branch maintainer is supposed to take care of them. If you find any | 149 | the case today. Please follow the standard stable rules in |
| 209 | patch is missing from an earlier stable branch, please notify | 150 | :ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>`, |
| 210 | stable@vger.kernel.org with either a commit ID or a formal patch | 151 | and make sure you include appropriate Fixes tags! |
| 211 | backported, and CC Dave and other relevant networking developers. | ||
| 212 | 152 | ||
| 213 | Is the comment style convention different for the networking content? | 153 | Is the comment style convention different for the networking content? |
| 214 | --------------------------------------------------------------------- | 154 | --------------------------------------------------------------------- |
| @@ -35,12 +35,6 @@ Rules on what kind of patches are accepted, and which ones are not, into the · 34 unmodified lines | |||
| 35 | Procedure for submitting patches to the -stable tree | 35 | Procedure for submitting patches to the -stable tree |
| 36 | ---------------------------------------------------- | 36 | ---------------------------------------------------- |
| 37 | 37 | ||
| 38 | - If the patch covers files in net/ or drivers/net please follow netdev stable | ||
| 39 | submission guidelines as described in | ||
| 40 | :ref:`Documentation/networking/netdev-FAQ.rst <netdev-FAQ>` | ||
| 41 | after first checking the stable networking queue at | ||
| 42 | https://patchwork.kernel.org/bundle/netdev/stable/?state=* | ||
| 43 | to ensure the requested patch is not already queued up. | ||
| 44 | - Security patches should not be handled (solely) by the -stable review | 38 | - Security patches should not be handled (solely) by the -stable review |
| 45 | process but should follow the procedures in | 39 | process but should follow the procedures in |
| 46 | :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`. | 40 | :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`. |
| @@ -250,11 +250,6 @@ should also read · 249 unmodified lines | |||
| 250 | :ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>` | 250 | :ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>` |
| 251 | in addition to this file. | 251 | in addition to this file. |
| 252 | 252 | ||
| 253 | Note, however, that some subsystem maintainers want to come to their own | ||
| 254 | conclusions on which patches should go to the stable trees. The networking | ||
| 255 | maintainer, in particular, would rather not see individual developers | ||
| 256 | adding lines like the above to their patches. | ||
| 257 | |||
| 258 | If changes affect userland-kernel interfaces, please send the MAN-PAGES | 253 | If changes affect userland-kernel interfaces, please send the MAN-PAGES |
| 259 | maintainer (as listed in the MAINTAINERS file) a man-pages patch, or at | 254 | maintainer (as listed in the MAINTAINERS file) a man-pages patch, or at |
| 260 | least a notification of the change, so that some information makes its way | 255 | least a notification of the change, so that some information makes its way |
| @@ -182,6 +182,9 @@ is dependent on the CPU capability and the kernel configuration. The limit can · 181 unmodified lines | |||
| 182 | be retrieved using KVM_CAP_ARM_VM_IPA_SIZE of the KVM_CHECK_EXTENSION | 182 | be retrieved using KVM_CAP_ARM_VM_IPA_SIZE of the KVM_CHECK_EXTENSION |
| 183 | ioctl() at run-time. | 183 | ioctl() at run-time. |
| 184 | 184 | ||
| 185 | Creation of the VM will fail if the requested IPA size (whether it is | ||
| 186 | implicit or explicit) is unsupported on the host. | ||
| 187 | |||
| 185 | Please note that configuring the IPA size does not affect the capability | 188 | Please note that configuring the IPA size does not affect the capability |
| 186 | exposed by the guest CPUs in ID_AA64MMFR0_EL1[PARange]. It only affects | 189 | exposed by the guest CPUs in ID_AA64MMFR0_EL1[PARange]. It only affects |
| 187 | size of the address translated by the stage2 level (guest physical to | 190 | size of the address translated by the stage2 level (guest physical to |
| @@ -261,8 +261,8 @@ ABI/API · 260 unmodified lines | |||
| 261 | L: linux-api@vger.kernel.org | 261 | L: linux-api@vger.kernel.org |
| 262 | F: include/linux/syscalls.h | 262 | F: include/linux/syscalls.h |
| 263 | F: kernel/sys_ni.c | 263 | F: kernel/sys_ni.c |
| 264 | F: include/uapi/ | 264 | X: include/uapi/ |
| 265 | F: arch/*/include/uapi/ | 265 | X: arch/*/include/uapi/ |
| 266 | 266 | ||
| 267 | ABIT UGURU 1,2 HARDWARE MONITOR DRIVER | 267 | ABIT UGURU 1,2 HARDWARE MONITOR DRIVER |
| 268 | M: Hans de Goede <hdegoede@redhat.com> | 268 | M: Hans de Goede <hdegoede@redhat.com> |
| @@ -5835,7 +5835,7 @@ M: David Airlie <airlied@linux.ie> · 5566 unmodified lines | |||
| 5835 | M: Daniel Vetter <daniel@ffwll.ch> | 5835 | M: Daniel Vetter <daniel@ffwll.ch> |
| 5836 | L: dri-devel@lists.freedesktop.org | 5836 | L: dri-devel@lists.freedesktop.org |
| 5837 | S: Maintained | 5837 | S: Maintained |
| 5838 | B: https://bugs.freedesktop.org/ | 5838 | B: https://gitlab.freedesktop.org/drm |
| 5839 | C: irc://chat.freenode.net/dri-devel | 5839 | C: irc://chat.freenode.net/dri-devel |
| 5840 | T: git git://anongit.freedesktop.org/drm/drm | 5840 | T: git git://anongit.freedesktop.org/drm/drm |
| 5841 | F: Documentation/devicetree/bindings/display/ | 5841 | F: Documentation/devicetree/bindings/display/ |
| @@ -10716,7 +10716,8 @@ F: drivers/net/ethernet/marvell/mvpp2/ · 4874 unmodified lines | |||
| 10716 | 10716 | ||
| 10717 | MARVELL MWIFIEX WIRELESS DRIVER | 10717 | MARVELL MWIFIEX WIRELESS DRIVER |
| 10718 | M: Amitkumar Karwar <amitkarwar@gmail.com> | 10718 | M: Amitkumar Karwar <amitkarwar@gmail.com> |
| 10719 | M: Ganapathi Bhat <ganapathi.bhat@nxp.com> | 10719 | M: Ganapathi Bhat <ganapathi017@gmail.com> |
| 10720 | M: Sharvari Harisangam <sharvari.harisangam@nxp.com> | ||
| 10720 | M: Xinming Hu <huxinming820@gmail.com> | 10721 | M: Xinming Hu <huxinming820@gmail.com> |
| 10721 | L: linux-wireless@vger.kernel.org | 10722 | L: linux-wireless@vger.kernel.org |
| 10722 | S: Maintained | 10723 | S: Maintained |
| @@ -19165,7 +19166,7 @@ S: Maintained · 8442 unmodified lines | |||
| 19165 | F: drivers/infiniband/hw/vmw_pvrdma/ | 19166 | F: drivers/infiniband/hw/vmw_pvrdma/ |
| 19166 | 19167 | ||
| 19167 | VMware PVSCSI driver | 19168 | VMware PVSCSI driver |
| 19168 | M: Jim Gill <jgill@vmware.com> | 19169 | M: Vishal Bhakta <vbhakta@vmware.com> |
| 19169 | M: VMware PV-Drivers <pv-drivers@vmware.com> | 19170 | M: VMware PV-Drivers <pv-drivers@vmware.com> |
| 19170 | L: linux-scsi@vger.kernel.org | 19171 | L: linux-scsi@vger.kernel.org |
| 19171 | S: Maintained | 19172 | S: Maintained |
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | VERSION = 5 | 2 | VERSION = 5 |
| 3 | PATCHLEVEL = 12 | 3 | PATCHLEVEL = 12 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc2 | 5 | EXTRAVERSION = -rc3 |
| 6 | NAME = Frozen Wasteland | 6 | NAME = Frozen Wasteland |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| @@ -264,7 +264,8 @@ no-dot-config-targets := $(clean-targets) \ · 255 unmodified lines | |||
| 264 | $(version_h) headers headers_% archheaders archscripts \ | 264 | $(version_h) headers headers_% archheaders archscripts \ |
| 265 | %asm-generic kernelversion %src-pkg dt_binding_check \ | 265 | %asm-generic kernelversion %src-pkg dt_binding_check \ |
| 266 | outputmakefile | 266 | outputmakefile |
| 267 | no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease | 267 | no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease \ |
| 268 | image_name | ||
| 268 | single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/ | 269 | single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/ |
| 269 | 270 | ||
| 270 | config-build := | 271 | config-build := |
| @@ -478,6 +479,7 @@ USERINCLUDE := \ · 207 unmodified lines | |||
| 478 | -I$(objtree)/arch/$(SRCARCH)/include/generated/uapi \ | 479 | -I$(objtree)/arch/$(SRCARCH)/include/generated/uapi \ |
| 479 | -I$(srctree)/include/uapi \ | 480 | -I$(srctree)/include/uapi \ |
| 480 | -I$(objtree)/include/generated/uapi \ | 481 | -I$(objtree)/include/generated/uapi \ |
| 482 | -include $(srctree)/include/linux/compiler-version.h \ | ||
| 481 | -include $(srctree)/include/linux/kconfig.h | 483 | -include $(srctree)/include/linux/kconfig.h |
| 482 | 484 | ||
| 483 | # Use LINUXINCLUDE when you must reference the include/ directory. | 485 | # Use LINUXINCLUDE when you must reference the include/ directory. |
| @@ -632,13 +632,12 @@ config HAS_LTO_CLANG · 631 unmodified lines | |||
| 632 | def_bool y | 632 | def_bool y |
| 633 | # Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510 | 633 | # Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510 |
| 634 | depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD | 634 | depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD |
| 635 | depends on $(success,test $(LLVM) -eq 1) | ||
| 636 | depends on $(success,test $(LLVM_IAS) -eq 1) | 635 | depends on $(success,test $(LLVM_IAS) -eq 1) |
| 637 | depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm) | 636 | depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm) |
| 638 | depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm) | 637 | depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm) |
| 639 | depends on ARCH_SUPPORTS_LTO_CLANG | 638 | depends on ARCH_SUPPORTS_LTO_CLANG |
| 640 | depends on !FTRACE_MCOUNT_USE_RECORDMCOUNT | 639 | depends on !FTRACE_MCOUNT_USE_RECORDMCOUNT |
| 641 | depends on !KASAN | 640 | depends on !KASAN || KASAN_HW_TAGS |
| 642 | depends on !GCOV_KERNEL | 641 | depends on !GCOV_KERNEL |
| 643 | help | 642 | help |
| 644 | The compiler and Kconfig options support building with Clang's | 643 | The compiler and Kconfig options support building with Clang's |
| @@ -348,6 +348,7 @@ config ARCH_EP93XX · 347 unmodified lines | |||
| 348 | select ARM_AMBA | 348 | select ARM_AMBA |
| 349 | imply ARM_PATCH_PHYS_VIRT | 349 | imply ARM_PATCH_PHYS_VIRT |
| 350 | select ARM_VIC | 350 | select ARM_VIC |
| 351 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 351 | select AUTO_ZRELADDR | 352 | select AUTO_ZRELADDR |
| 352 | select CLKDEV_LOOKUP | 353 | select CLKDEV_LOOKUP |
| 353 | select CLKSRC_MMIO | 354 | select CLKSRC_MMIO |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | |||
| 11 | 11 | ||
| 12 | #include <xen/xen.h> | 12 | #include <xen/xen.h> |
| 13 | #include <xen/interface/memory.h> | 13 | #include <xen/interface/memory.h> |
| 14 | #include <xen/grant_table.h> | ||
| 14 | #include <xen/page.h> | 15 | #include <xen/page.h> |
| 15 | #include <xen/swiotlb-xen.h> | 16 | #include <xen/swiotlb-xen.h> |
| 16 | 17 | ||
| @@ -109,7 +110,7 @@ int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, · 92 unmodified lines | |||
| 109 | map_ops[i].status = GNTST_general_error; | 110 | map_ops[i].status = GNTST_general_error; |
| 110 | unmap.host_addr = map_ops[i].host_addr, | 111 | unmap.host_addr = map_ops[i].host_addr, |
| 111 | unmap.handle = map_ops[i].handle; | 112 | unmap.handle = map_ops[i].handle; |
| 112 | map_ops[i].handle = ~0; | 113 | map_ops[i].handle = INVALID_GRANT_HANDLE; |
| 113 | if (map_ops[i].flags & GNTMAP_device_map) | 114 | if (map_ops[i].flags & GNTMAP_device_map) |
| 114 | unmap.dev_bus_addr = map_ops[i].dev_bus_addr; | 115 | unmap.dev_bus_addr = map_ops[i].dev_bus_addr; |
| 115 | else | 116 | else |
| @@ -130,7 +131,6 @@ int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, · 14 unmodified lines | |||
| 130 | 131 | ||
| 131 | return 0; | 132 | return 0; |
| 132 | } | 133 | } |
| 133 | EXPORT_SYMBOL_GPL(set_foreign_p2m_mapping); | ||
| 134 | 134 | ||
| 135 | int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, | 135 | int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, |
| 136 | struct gnttab_unmap_grant_ref *kunmap_ops, | 136 | struct gnttab_unmap_grant_ref *kunmap_ops, |
| @@ -145,7 +145,6 @@ int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, · 8 unmodified lines | |||
| 145 | 145 | ||
| 146 | return 0; | 146 | return 0; |
| 147 | } | 147 | } |
| 148 | EXPORT_SYMBOL_GPL(clear_foreign_p2m_mapping); | ||
| 149 | 148 | ||
| 150 | bool __set_phys_to_machine_multi(unsigned long pfn, | 149 | bool __set_phys_to_machine_multi(unsigned long pfn, |
| 151 | unsigned long mfn, unsigned long nr_pages) | 150 | unsigned long mfn, unsigned long nr_pages) |
| @@ -1055,8 +1055,6 @@ config HW_PERF_EVENTS · 1054 unmodified lines | |||
| 1055 | config SYS_SUPPORTS_HUGETLBFS | 1055 | config SYS_SUPPORTS_HUGETLBFS |
| 1056 | def_bool y | 1056 | def_bool y |
| 1057 | 1057 | ||
| 1058 | config ARCH_WANT_HUGE_PMD_SHARE | ||
| 1059 | |||
| 1060 | config ARCH_HAS_CACHE_LINE_SIZE | 1058 | config ARCH_HAS_CACHE_LINE_SIZE |
| 1061 | def_bool y | 1059 | def_bool y |
| 1062 | 1060 | ||
| @@ -1157,8 +1155,8 @@ config XEN · 94 unmodified lines | |||
| 1157 | 1155 | ||
| 1158 | config FORCE_MAX_ZONEORDER | 1156 | config FORCE_MAX_ZONEORDER |
| 1159 | int | 1157 | int |
| 1160 | default "14" if (ARM64_64K_PAGES && TRANSPARENT_HUGEPAGE) | 1158 | default "14" if ARM64_64K_PAGES |
| 1161 | default "12" if (ARM64_16K_PAGES && TRANSPARENT_HUGEPAGE) | 1159 | default "12" if ARM64_16K_PAGES |
| 1162 | default "11" | 1160 | default "11" |
| 1163 | help | 1161 | help |
| 1164 | The kernel memory allocator divides physically contiguous memory | 1162 | The kernel memory allocator divides physically contiguous memory |
| @@ -1855,12 +1853,6 @@ config CMDLINE_FROM_BOOTLOADER · 690 unmodified lines | |||
| 1855 | the boot loader doesn't provide any, the default kernel command | 1853 | the boot loader doesn't provide any, the default kernel command |
| 1856 | string provided in CMDLINE will be used. | 1854 | string provided in CMDLINE will be used. |
| 1857 | 1855 | ||
| 1858 | config CMDLINE_EXTEND | ||
| 1859 | bool "Extend bootloader kernel arguments" | ||
| 1860 | help | ||
| 1861 | The command-line arguments provided by the boot loader will be | ||
| 1862 | appended to the default kernel command string. | ||
| 1863 | |||
| 1864 | config CMDLINE_FORCE | 1856 | config CMDLINE_FORCE |
| 1865 | bool "Always use the default kernel command string" | 1857 | bool "Always use the default kernel command string" |
| 1866 | help | 1858 | help |
| @@ -47,10 +47,10 @@ · 46 unmodified lines | |||
| 47 | #define __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context 2 | 47 | #define __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context 2 |
| 48 | #define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa 3 | 48 | #define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa 3 |
| 49 | #define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid 4 | 49 | #define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid 4 |
| 50 | #define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_local_vmid 5 | 50 | #define __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context 5 |
| 51 | #define __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff 6 | 51 | #define __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff 6 |
| 52 | #define __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs 7 | 52 | #define __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs 7 |
| 53 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_get_ich_vtr_el2 8 | 53 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config 8 |
| 54 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr 9 | 54 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr 9 |
| 55 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr 10 | 55 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr 10 |
| 56 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs 11 | 56 | #define __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs 11 |
| @@ -183,16 +183,16 @@ DECLARE_KVM_HYP_SYM(__bp_harden_hyp_vecs); · 126 unmodified lines | |||
| 183 | #define __bp_harden_hyp_vecs CHOOSE_HYP_SYM(__bp_harden_hyp_vecs) | 183 | #define __bp_harden_hyp_vecs CHOOSE_HYP_SYM(__bp_harden_hyp_vecs) |
| 184 | 184 | ||
| 185 | extern void __kvm_flush_vm_context(void); | 185 | extern void __kvm_flush_vm_context(void); |
| 186 | extern void __kvm_flush_cpu_context(struct kvm_s2_mmu *mmu); | ||
| 186 | extern void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa, | 187 | extern void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa, |
| 187 | int level); | 188 | int level); |
| 188 | extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu); | 189 | extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu); |
| 189 | extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu); | ||
| 190 | 190 | ||
| 191 | extern void __kvm_timer_set_cntvoff(u64 cntvoff); | 191 | extern void __kvm_timer_set_cntvoff(u64 cntvoff); |
| 192 | 192 | ||
| 193 | extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu); | 193 | extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu); |
| 194 | 194 | ||
| 195 | extern u64 __vgic_v3_get_ich_vtr_el2(void); | 195 | extern u64 __vgic_v3_get_gic_config(void); |
| 196 | extern u64 __vgic_v3_read_vmcr(void); | 196 | extern u64 __vgic_v3_read_vmcr(void); |
| 197 | extern void __vgic_v3_write_vmcr(u32 vmcr); | 197 | extern void __vgic_v3_write_vmcr(u32 vmcr); |
| 198 | extern void __vgic_v3_init_lrs(void); | 198 | extern void __vgic_v3_init_lrs(void); |
| @@ -83,6 +83,11 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt); · 82 unmodified lines | |||
| 83 | void __debug_switch_to_guest(struct kvm_vcpu *vcpu); | 83 | void __debug_switch_to_guest(struct kvm_vcpu *vcpu); |
| 84 | void __debug_switch_to_host(struct kvm_vcpu *vcpu); | 84 | void __debug_switch_to_host(struct kvm_vcpu *vcpu); |
| 85 | 85 | ||
| 86 | #ifdef __KVM_NVHE_HYPERVISOR__ | ||
| 87 | void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu); | ||
| 88 | void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu); | ||
| 89 | #endif | ||
| 90 | |||
| 86 | void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); | 91 | void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); |
| 87 | void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); | 92 | void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); |
| 88 | 93 | ||
| @@ -97,7 +102,8 @@ bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt); · 8 unmodified lines | |||
| 97 | 102 | ||
| 98 | void __noreturn hyp_panic(void); | 103 | void __noreturn hyp_panic(void); |
| 99 | #ifdef __KVM_NVHE_HYPERVISOR__ | 104 | #ifdef __KVM_NVHE_HYPERVISOR__ |
| 100 | void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); | 105 | void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, |
| 106 | u64 elr, u64 par); | ||
| 101 | #endif | 107 | #endif |
| 102 | 108 | ||
| 103 | #endif /* __ARM64_KVM_HYP_H__ */ | 109 | #endif /* __ARM64_KVM_HYP_H__ */ |
| @@ -328,6 +328,11 @@ static inline void *phys_to_virt(phys_addr_t x) · 327 unmodified lines | |||
| 328 | #define ARCH_PFN_OFFSET ((unsigned long)PHYS_PFN_OFFSET) | 328 | #define ARCH_PFN_OFFSET ((unsigned long)PHYS_PFN_OFFSET) |
| 329 | 329 | ||
| 330 | #if !defined(CONFIG_SPARSEMEM_VMEMMAP) || defined(CONFIG_DEBUG_VIRTUAL) | 330 | #if !defined(CONFIG_SPARSEMEM_VMEMMAP) || defined(CONFIG_DEBUG_VIRTUAL) |
| 331 | #define page_to_virt(x) ({ \ | ||
| 332 | __typeof__(x) __page = x; \ | ||
| 333 | void *__addr = __va(page_to_phys(__page)); \ | ||
| 334 | (void *)__tag_set((const void *)__addr, page_kasan_tag(__page));\ | ||
| 335 | }) | ||
| 331 | #define virt_to_page(x) pfn_to_page(virt_to_pfn(x)) | 336 | #define virt_to_page(x) pfn_to_page(virt_to_pfn(x)) |
| 332 | #else | 337 | #else |
| 333 | #define page_to_virt(x) ({ \ | 338 | #define page_to_virt(x) ({ \ |
| @@ -63,23 +63,6 @@ static inline void cpu_switch_mm(pgd_t *pgd, struct mm_struct *mm) · 62 unmodified lines | |||
| 63 | extern u64 idmap_t0sz; | 63 | extern u64 idmap_t0sz; |
| 64 | extern u64 idmap_ptrs_per_pgd; | 64 | extern u64 idmap_ptrs_per_pgd; |
| 65 | 65 | ||
| 66 | static inline bool __cpu_uses_extended_idmap(void) | ||
| 67 | { | ||
| 68 | if (IS_ENABLED(CONFIG_ARM64_VA_BITS_52)) | ||
| 69 | return false; | ||
| 70 | |||
| 71 | return unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)); | ||
| 72 | } | ||
| 73 | |||
| 74 | /* | ||
| 75 | * True if the extended ID map requires an extra level of translation table | ||
| 76 | * to be configured. | ||
| 77 | */ | ||
| 78 | static inline bool __cpu_uses_extended_idmap_level(void) | ||
| 79 | { | ||
| 80 | return ARM64_HW_PGTABLE_LEVELS(64 - idmap_t0sz) > CONFIG_PGTABLE_LEVELS; | ||
| 81 | } | ||
| 82 | |||
| 83 | /* | 66 | /* |
| 84 | * Ensure TCR.T0SZ is set to the provided value. | 67 | * Ensure TCR.T0SZ is set to the provided value. |
| 85 | */ | 68 | */ |
| @@ -66,7 +66,6 @@ extern bool arm64_use_ng_mappings; · 65 unmodified lines | |||
| 66 | #define _PAGE_DEFAULT (_PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) | 66 | #define _PAGE_DEFAULT (_PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) |
| 67 | 67 | ||
| 68 | #define PAGE_KERNEL __pgprot(PROT_NORMAL) | 68 | #define PAGE_KERNEL __pgprot(PROT_NORMAL) |
| 69 | #define PAGE_KERNEL_TAGGED __pgprot(PROT_NORMAL_TAGGED) | ||
| 70 | #define PAGE_KERNEL_RO __pgprot((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) | 69 | #define PAGE_KERNEL_RO __pgprot((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) |
| 71 | #define PAGE_KERNEL_ROX __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) | 70 | #define PAGE_KERNEL_ROX __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) |
| 72 | #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) | 71 | #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) |
| @@ -486,6 +486,9 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) · 485 unmodified lines | |||
| 486 | __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_NC) | PTE_PXN | PTE_UXN) | 486 | __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_NC) | PTE_PXN | PTE_UXN) |
| 487 | #define pgprot_device(prot) \ | 487 | #define pgprot_device(prot) \ |
| 488 | __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_DEVICE_nGnRE) | PTE_PXN | PTE_UXN) | 488 | __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_DEVICE_nGnRE) | PTE_PXN | PTE_UXN) |
| 489 | #define pgprot_tagged(prot) \ | ||
| 490 | __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_TAGGED)) | ||
| 491 | #define pgprot_mhp pgprot_tagged | ||
| 489 | /* | 492 | /* |
| 490 | * DMA allocations for non-coherent devices use what the Arm architecture calls | 493 | * DMA allocations for non-coherent devices use what the Arm architecture calls |
| 491 | * "Normal non-cacheable" memory, which permits speculation, unaligned accesses | 494 | * "Normal non-cacheable" memory, which permits speculation, unaligned accesses |
| @@ -796,6 +796,11 @@ · 795 unmodified lines | |||
| 796 | #define ID_AA64MMFR0_PARANGE_48 0x5 | 796 | #define ID_AA64MMFR0_PARANGE_48 0x5 |
| 797 | #define ID_AA64MMFR0_PARANGE_52 0x6 | 797 | #define ID_AA64MMFR0_PARANGE_52 0x6 |
| 798 | 798 | ||
| 799 | #define ID_AA64MMFR0_TGRAN_2_SUPPORTED_DEFAULT 0x0 | ||
| 800 | #define ID_AA64MMFR0_TGRAN_2_SUPPORTED_NONE 0x1 | ||
| 801 | #define ID_AA64MMFR0_TGRAN_2_SUPPORTED_MIN 0x2 | ||
| 802 | #define ID_AA64MMFR0_TGRAN_2_SUPPORTED_MAX 0x7 | ||
| 803 | |||
| 799 | #ifdef CONFIG_ARM64_PA_BITS_52 | 804 | #ifdef CONFIG_ARM64_PA_BITS_52 |
| 800 | #define ID_AA64MMFR0_PARANGE_MAX ID_AA64MMFR0_PARANGE_52 | 805 | #define ID_AA64MMFR0_PARANGE_MAX ID_AA64MMFR0_PARANGE_52 |
| 801 | #else | 806 | #else |
| @@ -961,14 +966,17 @@ · 159 unmodified lines | |||
| 961 | #define ID_PFR1_PROGMOD_SHIFT 0 | 966 | #define ID_PFR1_PROGMOD_SHIFT 0 |
| 962 | 967 | ||
| 963 | #if defined(CONFIG_ARM64_4K_PAGES) | 968 | #if defined(CONFIG_ARM64_4K_PAGES) |
| 964 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN4_SHIFT | 969 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN4_SHIFT |
| 965 | #define ID_AA64MMFR0_TGRAN_SUPPORTED ID_AA64MMFR0_TGRAN4_SUPPORTED | 970 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_TGRAN4_SUPPORTED |
| 971 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MAX 0x7 | ||
| 966 | #elif defined(CONFIG_ARM64_16K_PAGES) | 972 | #elif defined(CONFIG_ARM64_16K_PAGES) |
| 967 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN16_SHIFT | 973 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN16_SHIFT |
| 968 | #define ID_AA64MMFR0_TGRAN_SUPPORTED ID_AA64MMFR0_TGRAN16_SUPPORTED | 974 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_TGRAN16_SUPPORTED |
| 975 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MAX 0xF | ||
| 969 | #elif defined(CONFIG_ARM64_64K_PAGES) | 976 | #elif defined(CONFIG_ARM64_64K_PAGES) |
| 970 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN64_SHIFT | 977 | #define ID_AA64MMFR0_TGRAN_SHIFT ID_AA64MMFR0_TGRAN64_SHIFT |
| 971 | #define ID_AA64MMFR0_TGRAN_SUPPORTED ID_AA64MMFR0_TGRAN64_SUPPORTED | 978 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_TGRAN64_SUPPORTED |
| 979 | #define ID_AA64MMFR0_TGRAN_SUPPORTED_MAX 0x7 | ||
| 972 | #endif | 980 | #endif |
| 973 | 981 | ||
| 974 | #define MVFR2_FPMISC_SHIFT 4 | 982 | #define MVFR2_FPMISC_SHIFT 4 |
| @@ -319,7 +319,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) · 318 unmodified lines | |||
| 319 | */ | 319 | */ |
| 320 | adrp x5, __idmap_text_end | 320 | adrp x5, __idmap_text_end |
| 321 | clz x5, x5 | 321 | clz x5, x5 |
| 322 | cmp x5, TCR_T0SZ(VA_BITS) // default T0SZ small enough? | 322 | cmp x5, TCR_T0SZ(VA_BITS_MIN) // default T0SZ small enough? |
| 323 | b.ge 1f // .. then skip VA range extension | 323 | b.ge 1f // .. then skip VA range extension |
| 324 | 324 | ||
| 325 | adr_l x6, idmap_t0sz | 325 | adr_l x6, idmap_t0sz |
| @@ -655,8 +655,10 @@ SYM_FUNC_END(__secondary_too_slow) · 329 unmodified lines | |||
| 655 | SYM_FUNC_START(__enable_mmu) | 655 | SYM_FUNC_START(__enable_mmu) |
| 656 | mrs x2, ID_AA64MMFR0_EL1 | 656 | mrs x2, ID_AA64MMFR0_EL1 |
| 657 | ubfx x2, x2, #ID_AA64MMFR0_TGRAN_SHIFT, 4 | 657 | ubfx x2, x2, #ID_AA64MMFR0_TGRAN_SHIFT, 4 |
| 658 | cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED | 658 | cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED_MIN |
| 659 | b.ne __no_granule_support | 659 | b.lt __no_granule_support |
| 660 | cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED_MAX | ||
| 661 | b.gt __no_granule_support | ||
| 660 | update_early_cpu_boot_status 0, x2, x3 | 662 | update_early_cpu_boot_status 0, x2, x3 |
| 661 | adrp x2, idmap_pg_dir | 663 | adrp x2, idmap_pg_dir |
| 662 | phys_to_ttbr x1, x1 | 664 | phys_to_ttbr x1, x1 |
| @@ -163,33 +163,36 @@ static __init void __parse_cmdline(const char *cmdline, bool parse_aliases) · 162 unmodified lines | |||
| 163 | } while (1); | 163 | } while (1); |
| 164 | } | 164 | } |
| 165 | 165 | ||
| 166 | static __init void parse_cmdline(void) | 166 | static __init const u8 *get_bootargs_cmdline(void) |
| 167 | { | 167 | { |
| 168 | if (!IS_ENABLED(CONFIG_CMDLINE_FORCE)) { | 168 | const u8 *prop; |
| 169 | const u8 *prop; | 169 | void *fdt; |
| 170 | void *fdt; | 170 | int node; |
| 171 | int node; | ||
| 172 | 171 | ||
| 173 | fdt = get_early_fdt_ptr(); | 172 | fdt = get_early_fdt_ptr(); |
| 174 | if (!fdt) | 173 | if (!fdt) |
| 175 | goto out; | 174 | return NULL; |
| 176 | 175 | ||
| 177 | node = fdt_path_offset(fdt, "/chosen"); | 176 | node = fdt_path_offset(fdt, "/chosen"); |
| 178 | if (node < 0) | 177 | if (node < 0) |
| 179 | goto out; | 178 | return NULL; |
| 180 | 179 | ||
| 181 | prop = fdt_getprop(fdt, node, "bootargs", NULL); | 180 | prop = fdt_getprop(fdt, node, "bootargs", NULL); |
| 182 | if (!prop) | 181 | if (!prop) |
| 183 | goto out; | 182 | return NULL; |
| 184 | 183 | ||
| 185 | __parse_cmdline(prop, true); | 184 | return strlen(prop) ? prop : NULL; |
| 185 | } | ||
| 186 | 186 | ||
| 187 | if (!IS_ENABLED(CONFIG_CMDLINE_EXTEND)) | 187 | static __init void parse_cmdline(void) |
| 188 | return; | 188 | { |
| 189 | } | 189 | const u8 *prop = get_bootargs_cmdline(); |
| 190 | 190 | ||
| 191 | out: | 191 | if (IS_ENABLED(CONFIG_CMDLINE_FORCE) || !prop) |
| 192 | __parse_cmdline(CONFIG_CMDLINE, true); | 192 | __parse_cmdline(CONFIG_CMDLINE, true); |
| 193 | |||
| 194 | if (!IS_ENABLED(CONFIG_CMDLINE_FORCE) && prop) | ||
| 195 | __parse_cmdline(prop, true); | ||
| 193 | } | 196 | } |
| 194 | 197 | ||
| 195 | /* Keep checkers quiet */ | 198 | /* Keep checkers quiet */ |
| @@ -101,6 +101,9 @@ KVM_NVHE_ALIAS(__stop___kvm_ex_table); · 100 unmodified lines | |||
| 101 | /* Array containing bases of nVHE per-CPU memory regions. */ | 101 | /* Array containing bases of nVHE per-CPU memory regions. */ |
| 102 | KVM_NVHE_ALIAS(kvm_arm_hyp_percpu_base); | 102 | KVM_NVHE_ALIAS(kvm_arm_hyp_percpu_base); |
| 103 | 103 | ||
| 104 | /* PMU available static key */ | ||
| 105 | KVM_NVHE_ALIAS(kvm_arm_pmu_available); | ||
| 106 | |||
| 104 | #endif /* CONFIG_KVM */ | 107 | #endif /* CONFIG_KVM */ |
| 105 | 108 | ||
| 106 | #endif /* __ARM64_KERNEL_IMAGE_VARS_H */ | 109 | #endif /* __ARM64_KERNEL_IMAGE_VARS_H */ |
| @@ -460,7 +460,7 @@ static inline int armv8pmu_counter_has_overflowed(u32 pmnc, int idx) · 459 unmodified lines | |||
| 460 | return pmnc & BIT(ARMV8_IDX_TO_COUNTER(idx)); | 460 | return pmnc & BIT(ARMV8_IDX_TO_COUNTER(idx)); |
| 461 | } | 461 | } |
| 462 | 462 | ||
| 463 | static inline u32 armv8pmu_read_evcntr(int idx) | 463 | static inline u64 armv8pmu_read_evcntr(int idx) |
| 464 | { | 464 | { |
| 465 | u32 counter = ARMV8_IDX_TO_COUNTER(idx); | 465 | u32 counter = ARMV8_IDX_TO_COUNTER(idx); |
| 466 | 466 | ||
| @@ -385,11 +385,16 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · 384 unmodified lines | |||
| 385 | last_ran = this_cpu_ptr(mmu->last_vcpu_ran); | 385 | last_ran = this_cpu_ptr(mmu->last_vcpu_ran); |
| 386 | 386 | ||
| 387 | /* | 387 | /* |
| 388 | * We guarantee that both TLBs and I-cache are private to each | ||
| 389 | * vcpu. If detecting that a vcpu from the same VM has | ||
| 390 | * previously run on the same physical CPU, call into the | ||
| 391 | * hypervisor code to nuke the relevant contexts. | ||
| 392 | * | ||
| 388 | * We might get preempted before the vCPU actually runs, but | 393 | * We might get preempted before the vCPU actually runs, but |
| 389 | * over-invalidation doesn't affect correctness. | 394 | * over-invalidation doesn't affect correctness. |
| 390 | */ | 395 | */ |
| 391 | if (*last_ran != vcpu->vcpu_id) { | 396 | if (*last_ran != vcpu->vcpu_id) { |
| 392 | kvm_call_hyp(__kvm_tlb_flush_local_vmid, mmu); | 397 | kvm_call_hyp(__kvm_flush_cpu_context, mmu); |
| 393 | *last_ran = vcpu->vcpu_id; | 398 | *last_ran = vcpu->vcpu_id; |
| 394 | } | 399 | } |
| 395 | 400 | ||
| @@ -85,8 +85,10 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) · 84 unmodified lines | |||
| 85 | 85 | ||
| 86 | // If the hyp context is loaded, go straight to hyp_panic | 86 | // If the hyp context is loaded, go straight to hyp_panic |
| 87 | get_loaded_vcpu x0, x1 | 87 | get_loaded_vcpu x0, x1 |
| 88 | cbz x0, hyp_panic | 88 | cbnz x0, 1f |
| 89 | b hyp_panic | ||
| 89 | 90 | ||
| 91 | 1: | ||
| 90 | // The hyp context is saved so make sure it is restored to allow | 92 | // The hyp context is saved so make sure it is restored to allow |
| 91 | // hyp_panic to run at hyp and, subsequently, panic to run in the host. | 93 | // hyp_panic to run at hyp and, subsequently, panic to run in the host. |
| 92 | // This makes use of __guest_exit to avoid duplication but sets the | 94 | // This makes use of __guest_exit to avoid duplication but sets the |
| @@ -94,7 +96,7 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) · 1 unmodified line | |||
| 94 | // current state is saved to the guest context but it will only be | 96 | // current state is saved to the guest context but it will only be |
| 95 | // accurate if the guest had been completely restored. | 97 | // accurate if the guest had been completely restored. |
| 96 | adr_this_cpu x0, kvm_hyp_ctxt, x1 | 98 | adr_this_cpu x0, kvm_hyp_ctxt, x1 |
| 97 | adr x1, hyp_panic | 99 | adr_l x1, hyp_panic |
| 98 | str x1, [x0, #CPU_XREG_OFFSET(30)] | 100 | str x1, [x0, #CPU_XREG_OFFSET(30)] |
| 99 | 101 | ||
| 100 | get_vcpu_ptr x1, x0 | 102 | get_vcpu_ptr x1, x0 |
| @@ -146,7 +148,7 @@ SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL) · 45 unmodified lines | |||
| 146 | // Now restore the hyp regs | 148 | // Now restore the hyp regs |
| 147 | restore_callee_saved_regs x2 | 149 | restore_callee_saved_regs x2 |
| 148 | 150 | ||
| 149 | set_loaded_vcpu xzr, x1, x2 | 151 | set_loaded_vcpu xzr, x2, x3 |
| 150 | 152 | ||
| 151 | alternative_if ARM64_HAS_RAS_EXTN | 153 | alternative_if ARM64_HAS_RAS_EXTN |
| 152 | // If we have the RAS extensions we can consume a pending error | 154 | // If we have the RAS extensions we can consume a pending error |
| @@ -90,15 +90,18 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu) · 89 unmodified lines | |||
| 90 | * counter, which could make a PMXEVCNTR_EL0 access UNDEF at | 90 | * counter, which could make a PMXEVCNTR_EL0 access UNDEF at |
| 91 | * EL1 instead of being trapped to EL2. | 91 | * EL1 instead of being trapped to EL2. |
| 92 | */ | 92 | */ |
| 93 | write_sysreg(0, pmselr_el0); | 93 | if (kvm_arm_support_pmu_v3()) { |
| 94 | write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0); | 94 | write_sysreg(0, pmselr_el0); |
| 95 | write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0); | ||
| 96 | } | ||
| 95 | write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2); | 97 | write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2); |
| 96 | } | 98 | } |
| 97 | 99 | ||
| 98 | static inline void __deactivate_traps_common(void) | 100 | static inline void __deactivate_traps_common(void) |
| 99 | { | 101 | { |
| 100 | write_sysreg(0, hstr_el2); | 102 | write_sysreg(0, hstr_el2); |
| 101 | write_sysreg(0, pmuserenr_el0); | 103 | if (kvm_arm_support_pmu_v3()) |
| 104 | write_sysreg(0, pmuserenr_el0); | ||
| 102 | } | 105 | } |
| 103 | 106 | ||
| 104 | static inline void ___activate_traps(struct kvm_vcpu *vcpu) | 107 | static inline void ___activate_traps(struct kvm_vcpu *vcpu) |
| @@ -58,16 +58,24 @@ static void __debug_restore_spe(u64 pmscr_el1) · 57 unmodified lines | |||
| 58 | write_sysreg_s(pmscr_el1, SYS_PMSCR_EL1); | 58 | write_sysreg_s(pmscr_el1, SYS_PMSCR_EL1); |
| 59 | } | 59 | } |
| 60 | 60 | ||
| 61 | void __debug_switch_to_guest(struct kvm_vcpu *vcpu) | 61 | void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu) |
| 62 | { | 62 | { |
| 63 | /* Disable and flush SPE data generation */ | 63 | /* Disable and flush SPE data generation */ |
| 64 | __debug_save_spe(&vcpu->arch.host_debug_state.pmscr_el1); | 64 | __debug_save_spe(&vcpu->arch.host_debug_state.pmscr_el1); |
| 65 | } | ||
| 66 | |||
| 67 | void __debug_switch_to_guest(struct kvm_vcpu *vcpu) | ||
| 68 | { | ||
| 65 | __debug_switch_to_guest_common(vcpu); | 69 | __debug_switch_to_guest_common(vcpu); |
| 66 | } | 70 | } |
| 67 | 71 | ||
| 68 | void __debug_switch_to_host(struct kvm_vcpu *vcpu) | 72 | void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu) |
| 69 | { | 73 | { |
| 70 | __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1); | 74 | __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1); |
| 75 | } | ||
| 76 | |||
| 77 | void __debug_switch_to_host(struct kvm_vcpu *vcpu) | ||
| 78 | { | ||
| 71 | __debug_switch_to_host_common(vcpu); | 79 | __debug_switch_to_host_common(vcpu); |
| 72 | } | 80 | } |
| 73 | 81 | ||
| @@ -71,7 +71,8 @@ SYM_FUNC_START(__host_enter) · 70 unmodified lines | |||
| 71 | SYM_FUNC_END(__host_enter) | 71 | SYM_FUNC_END(__host_enter) |
| 72 | 72 | ||
| 73 | /* | 73 | /* |
| 74 | * void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); | 74 | * void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, |
| 75 | * u64 elr, u64 par); | ||
| 75 | */ | 76 | */ |
| 76 | SYM_FUNC_START(__hyp_do_panic) | 77 | SYM_FUNC_START(__hyp_do_panic) |
| 77 | /* Prepare and exit to the host's panic funciton. */ | 78 | /* Prepare and exit to the host's panic funciton. */ |
| @@ -82,9 +83,11 @@ SYM_FUNC_START(__hyp_do_panic) · 4 unmodified lines | |||
| 82 | hyp_kimg_va lr, x6 | 83 | hyp_kimg_va lr, x6 |
| 83 | msr elr_el2, lr | 84 | msr elr_el2, lr |
| 84 | 85 | ||
| 85 | /* Set the panic format string. Use the, now free, LR as scratch. */ | 86 | mov x29, x0 |
| 86 | ldr lr, =__hyp_panic_string | 87 | |
| 87 | hyp_kimg_va lr, x6 | 88 | /* Load the format string into x0 and arguments into x1-7 */ |
| 89 | ldr x0, =__hyp_panic_string | ||
| 90 | hyp_kimg_va x0, x6 | ||
| 88 | 91 | ||
| 89 | /* Load the format arguments into x1-7. */ | 92 | /* Load the format arguments into x1-7. */ |
| 90 | mov x6, x3 | 93 | mov x6, x3 |
| @@ -94,9 +97,7 @@ SYM_FUNC_START(__hyp_do_panic) · 3 unmodified lines | |||
| 94 | mrs x5, hpfar_el2 | 97 | mrs x5, hpfar_el2 |
| 95 | 98 | ||
| 96 | /* Enter the host, conditionally restoring the host context. */ | 99 | /* Enter the host, conditionally restoring the host context. */ |
| 97 | cmp x0, xzr | 100 | cbz x29, __host_enter_without_restoring |
| 98 | mov x0, lr | ||
| 99 | b.eq __host_enter_without_restoring | ||
| 100 | b __host_enter_for_panic | 101 | b __host_enter_for_panic |
| 101 | SYM_FUNC_END(__hyp_do_panic) | 102 | SYM_FUNC_END(__hyp_do_panic) |
| 102 | 103 | ||
| @@ -46,11 +46,11 @@ static void handle___kvm_tlb_flush_vmid(struct kvm_cpu_context *host_ctxt) · 45 unmodified lines | |||
| 46 | __kvm_tlb_flush_vmid(kern_hyp_va(mmu)); | 46 | __kvm_tlb_flush_vmid(kern_hyp_va(mmu)); |
| 47 | } | 47 | } |
| 48 | 48 | ||
| 49 | static void handle___kvm_tlb_flush_local_vmid(struct kvm_cpu_context *host_ctxt) | 49 | static void handle___kvm_flush_cpu_context(struct kvm_cpu_context *host_ctxt) |
| 50 | { | 50 | { |
| 51 | DECLARE_REG(struct kvm_s2_mmu *, mmu, host_ctxt, 1); | 51 | DECLARE_REG(struct kvm_s2_mmu *, mmu, host_ctxt, 1); |
| 52 | 52 | ||
| 53 | __kvm_tlb_flush_local_vmid(kern_hyp_va(mmu)); | 53 | __kvm_flush_cpu_context(kern_hyp_va(mmu)); |
| 54 | } | 54 | } |
| 55 | 55 | ||
| 56 | static void handle___kvm_timer_set_cntvoff(struct kvm_cpu_context *host_ctxt) | 56 | static void handle___kvm_timer_set_cntvoff(struct kvm_cpu_context *host_ctxt) |
| @@ -67,9 +67,9 @@ static void handle___kvm_enable_ssbs(struct kvm_cpu_context *host_ctxt) · 10 unmodified lines | |||
| 67 | write_sysreg_el2(tmp, SYS_SCTLR); | 67 | write_sysreg_el2(tmp, SYS_SCTLR); |
| 68 | } | 68 | } |
| 69 | 69 | ||
| 70 | static void handle___vgic_v3_get_ich_vtr_el2(struct kvm_cpu_context *host_ctxt) | 70 | static void handle___vgic_v3_get_gic_config(struct kvm_cpu_context *host_ctxt) |
| 71 | { | 71 | { |
| 72 | cpu_reg(host_ctxt, 1) = __vgic_v3_get_ich_vtr_el2(); | 72 | cpu_reg(host_ctxt, 1) = __vgic_v3_get_gic_config(); |
| 73 | } | 73 | } |
| 74 | 74 | ||
| 75 | static void handle___vgic_v3_read_vmcr(struct kvm_cpu_context *host_ctxt) | 75 | static void handle___vgic_v3_read_vmcr(struct kvm_cpu_context *host_ctxt) |
| @@ -115,10 +115,10 @@ static const hcall_t host_hcall[] = { · 39 unmodified lines | |||
| 115 | HANDLE_FUNC(__kvm_flush_vm_context), | 115 | HANDLE_FUNC(__kvm_flush_vm_context), |
| 116 | HANDLE_FUNC(__kvm_tlb_flush_vmid_ipa), | 116 | HANDLE_FUNC(__kvm_tlb_flush_vmid_ipa), |
| 117 | HANDLE_FUNC(__kvm_tlb_flush_vmid), | 117 | HANDLE_FUNC(__kvm_tlb_flush_vmid), |
| 118 | HANDLE_FUNC(__kvm_tlb_flush_local_vmid), | 118 | HANDLE_FUNC(__kvm_flush_cpu_context), |
| 119 | HANDLE_FUNC(__kvm_timer_set_cntvoff), | 119 | HANDLE_FUNC(__kvm_timer_set_cntvoff), |
| 120 | HANDLE_FUNC(__kvm_enable_ssbs), | 120 | HANDLE_FUNC(__kvm_enable_ssbs), |
| 121 | HANDLE_FUNC(__vgic_v3_get_ich_vtr_el2), | 121 | HANDLE_FUNC(__vgic_v3_get_gic_config), |
| 122 | HANDLE_FUNC(__vgic_v3_read_vmcr), | 122 | HANDLE_FUNC(__vgic_v3_read_vmcr), |
| 123 | HANDLE_FUNC(__vgic_v3_write_vmcr), | 123 | HANDLE_FUNC(__vgic_v3_write_vmcr), |
| 124 | HANDLE_FUNC(__vgic_v3_init_lrs), | 124 | HANDLE_FUNC(__vgic_v3_init_lrs), |
| @@ -192,6 +192,14 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) · 191 unmodified lines | |||
| 192 | pmu_switch_needed = __pmu_switch_to_guest(host_ctxt); | 192 | pmu_switch_needed = __pmu_switch_to_guest(host_ctxt); |
| 193 | 193 | ||
| 194 | __sysreg_save_state_nvhe(host_ctxt); | 194 | __sysreg_save_state_nvhe(host_ctxt); |
| 195 | /* | ||
| 196 | * We must flush and disable the SPE buffer for nVHE, as | ||
| 197 | * the translation regime(EL1&0) is going to be loaded with | ||
| 198 | * that of the guest. And we must do this before we change the | ||
| 199 | * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and | ||
| 200 | * before we load guest Stage1. | ||
| 201 | */ | ||
| 202 | __debug_save_host_buffers_nvhe(vcpu); | ||
| 195 | 203 | ||
| 196 | __adjust_pc(vcpu); | 204 | __adjust_pc(vcpu); |
| 197 | 205 | ||
| @@ -234,11 +242,12 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) · 36 unmodified lines | |||
| 234 | if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) | 242 | if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) |
| 235 | __fpsimd_save_fpexc32(vcpu); | 243 | __fpsimd_save_fpexc32(vcpu); |
| 236 | 244 | ||
| 245 | __debug_switch_to_host(vcpu); | ||
| 237 | /* | 246 | /* |
| 238 | * This must come after restoring the host sysregs, since a non-VHE | 247 | * This must come after restoring the host sysregs, since a non-VHE |
| 239 | * system may enable SPE here and make use of the TTBRs. | 248 | * system may enable SPE here and make use of the TTBRs. |
| 240 | */ | 249 | */ |
| 241 | __debug_switch_to_host(vcpu); | 250 | __debug_restore_host_buffers_nvhe(vcpu); |
| 242 | 251 | ||
| 243 | if (pmu_switch_needed) | 252 | if (pmu_switch_needed) |
| 244 | __pmu_switch_to_host(host_ctxt); | 253 | __pmu_switch_to_host(host_ctxt); |
| @@ -257,7 +266,6 @@ void __noreturn hyp_panic(void) · 12 unmodified lines | |||
| 257 | u64 spsr = read_sysreg_el2(SYS_SPSR); | 266 | u64 spsr = read_sysreg_el2(SYS_SPSR); |
| 258 | u64 elr = read_sysreg_el2(SYS_ELR); | 267 | u64 elr = read_sysreg_el2(SYS_ELR); |
| 259 | u64 par = read_sysreg_par(); | 268 | u64 par = read_sysreg_par(); |
| 260 | bool restore_host = true; | ||
| 261 | struct kvm_cpu_context *host_ctxt; | 269 | struct kvm_cpu_context *host_ctxt; |
| 262 | struct kvm_vcpu *vcpu; | 270 | struct kvm_vcpu *vcpu; |
| 263 | 271 | ||
| @@ -271,7 +279,7 @@ void __noreturn hyp_panic(void) · 7 unmodified lines | |||
| 271 | __sysreg_restore_state_nvhe(host_ctxt); | 279 | __sysreg_restore_state_nvhe(host_ctxt); |
| 272 | } | 280 | } |
| 273 | 281 | ||
| 274 | __hyp_do_panic(restore_host, spsr, elr, par); | 282 | __hyp_do_panic(host_ctxt, spsr, elr, par); |
| 275 | unreachable(); | 283 | unreachable(); |
| 276 | } | 284 | } |
| 277 | 285 | ||
| @@ -123,7 +123,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 122 unmodified lines | |||
| 123 | __tlb_switch_to_host(&cxt); | 123 | __tlb_switch_to_host(&cxt); |
| 124 | } | 124 | } |
| 125 | 125 | ||
| 126 | void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu) | 126 | void __kvm_flush_cpu_context(struct kvm_s2_mmu *mmu) |
| 127 | { | 127 | { |
| 128 | struct tlb_inv_context cxt; | 128 | struct tlb_inv_context cxt; |
| 129 | 129 | ||
| @@ -131,6 +131,7 @@ void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu) · 1 unmodified line | |||
| 131 | __tlb_switch_to_guest(mmu, &cxt); | 131 | __tlb_switch_to_guest(mmu, &cxt); |
| 132 | 132 | ||
| 133 | __tlbi(vmalle1); | 133 | __tlbi(vmalle1); |
| 134 | asm volatile("ic iallu"); | ||
| 134 | dsb(nsh); | 135 | dsb(nsh); |
| 135 | isb(); | 136 | isb(); |
| 136 | 137 | ||
| @@ -223,6 +223,7 @@ static inline int __kvm_pgtable_visit(struct kvm_pgtable_walk_data *data, · 222 unmodified lines | |||
| 223 | goto out; | 223 | goto out; |
| 224 | 224 | ||
| 225 | if (!table) { | 225 | if (!table) { |
| 226 | data->addr = ALIGN_DOWN(data->addr, kvm_granule_size(level)); | ||
| 226 | data->addr += kvm_granule_size(level); | 227 | data->addr += kvm_granule_size(level); |
| 227 | goto out; | 228 | goto out; |
| 228 | } | 229 | } |
| @@ -405,9 +405,45 @@ void __vgic_v3_init_lrs(void) · 404 unmodified lines | |||
| 405 | __gic_v3_set_lr(0, i); | 405 | __gic_v3_set_lr(0, i); |
| 406 | } | 406 | } |
| 407 | 407 | ||
| 408 | u64 __vgic_v3_get_ich_vtr_el2(void) | 408 | /* |
| 409 | * Return the GIC CPU configuration: | ||
| 410 | * - [31:0] ICH_VTR_EL2 | ||
| 411 | * - [62:32] RES0 | ||
| 412 | * - [63] MMIO (GICv2) capable | ||
| 413 | */ | ||
| 414 | u64 __vgic_v3_get_gic_config(void) | ||
| 409 | { | 415 | { |
| 410 | return read_gicreg(ICH_VTR_EL2); | 416 | u64 val, sre = read_gicreg(ICC_SRE_EL1); |
| 417 | unsigned long flags = 0; | ||
| 418 | |||
| 419 | /* | ||
| 420 | * To check whether we have a MMIO-based (GICv2 compatible) | ||
| 421 | * CPU interface, we need to disable the system register | ||
| 422 | * view. To do that safely, we have to prevent any interrupt | ||
| 423 | * from firing (which would be deadly). | ||
| 424 | * | ||
| 425 | * Note that this only makes sense on VHE, as interrupts are | ||
| 426 | * already masked for nVHE as part of the exception entry to | ||
| 427 | * EL2. | ||
| 428 | */ | ||
| 429 | if (has_vhe()) | ||
| 430 | flags = local_daif_save(); | ||
| 431 | |||
| 432 | write_gicreg(0, ICC_SRE_EL1); | ||
| 433 | isb(); | ||
| 434 | |||
| 435 | val = read_gicreg(ICC_SRE_EL1); | ||
| 436 | |||
| 437 | write_gicreg(sre, ICC_SRE_EL1); | ||
| 438 | isb(); | ||
| 439 | |||
| 440 | if (has_vhe()) | ||
| 441 | local_daif_restore(flags); | ||
| 442 | |||
| 443 | val = (val & ICC_SRE_EL1_SRE) ? 0 : (1ULL << 63); | ||
| 444 | val |= read_gicreg(ICH_VTR_EL2); | ||
| 445 | |||
| 446 | return val; | ||
| 411 | } | 447 | } |
| 412 | 448 | ||
| 413 | u64 __vgic_v3_read_vmcr(void) | 449 | u64 __vgic_v3_read_vmcr(void) |
| @@ -127,7 +127,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 126 unmodified lines | |||
| 127 | __tlb_switch_to_host(&cxt); | 127 | __tlb_switch_to_host(&cxt); |
| 128 | } | 128 | } |
| 129 | 129 | ||
| 130 | void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu) | 130 | void __kvm_flush_cpu_context(struct kvm_s2_mmu *mmu) |
| 131 | { | 131 | { |
| 132 | struct tlb_inv_context cxt; | 132 | struct tlb_inv_context cxt; |
| 133 | 133 | ||
| @@ -135,6 +135,7 @@ void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu) · 1 unmodified line | |||
| 135 | __tlb_switch_to_guest(mmu, &cxt); | 135 | __tlb_switch_to_guest(mmu, &cxt); |
| 136 | 136 | ||
| 137 | __tlbi(vmalle1); | 137 | __tlbi(vmalle1); |
| 138 | asm volatile("ic iallu"); | ||
| 138 | dsb(nsh); | 139 | dsb(nsh); |
| 139 | isb(); | 140 | isb(); |
| 140 | 141 | ||
| @@ -1312,8 +1312,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, · 1311 unmodified lines | |||
| 1312 | * Prevent userspace from creating a memory region outside of the IPA | 1312 | * Prevent userspace from creating a memory region outside of the IPA |
| 1313 | * space addressable by the KVM guest IPA space. | 1313 | * space addressable by the KVM guest IPA space. |
| 1314 | */ | 1314 | */ |
| 1315 | if (memslot->base_gfn + memslot->npages >= | 1315 | if ((memslot->base_gfn + memslot->npages) > (kvm_phys_size(kvm) >> PAGE_SHIFT)) |
| 1316 | (kvm_phys_size(kvm) >> PAGE_SHIFT)) | ||
| 1317 | return -EFAULT; | 1316 | return -EFAULT; |
| 1318 | 1317 | ||
| 1319 | mmap_read_lock(current->mm); | 1318 | mmap_read_lock(current->mm); |
| @@ -11,6 +11,8 @@ · 10 unmodified lines | |||
| 11 | 11 | ||
| 12 | #include <asm/kvm_emulate.h> | 12 | #include <asm/kvm_emulate.h> |
| 13 | 13 | ||
| 14 | DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available); | ||
| 15 | |||
| 14 | static int kvm_is_in_guest(void) | 16 | static int kvm_is_in_guest(void) |
| 15 | { | 17 | { |
| 16 | return kvm_get_running_vcpu() != NULL; | 18 | return kvm_get_running_vcpu() != NULL; |
| @@ -48,6 +50,14 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = { · 31 unmodified lines | |||
| 48 | 50 | ||
| 49 | int kvm_perf_init(void) | 51 | int kvm_perf_init(void) |
| 50 | { | 52 | { |
| 53 | /* | ||
| 54 | * Check if HW_PERF_EVENTS are supported by checking the number of | ||
| 55 | * hardware performance counters. This could ensure the presence of | ||
| 56 | * a physical PMU and CONFIG_PERF_EVENT is selected. | ||
| 57 | */ | ||
| 58 | if (IS_ENABLED(CONFIG_ARM_PMU) && perf_num_counters() > 0) | ||
| 59 | static_branch_enable(&kvm_arm_pmu_available); | ||
| 60 | |||
| 51 | return perf_register_guest_info_callbacks(&kvm_guest_cbs); | 61 | return perf_register_guest_info_callbacks(&kvm_guest_cbs); |
| 52 | } | 62 | } |
| 53 | 63 | ||
| @@ -823,16 +823,6 @@ u64 kvm_pmu_get_pmceid(struct kvm_vcpu *vcpu, bool pmceid1) · 822 unmodified lines | |||
| 823 | return val & mask; | 823 | return val & mask; |
| 824 | } | 824 | } |
| 825 | 825 | ||
| 826 | bool kvm_arm_support_pmu_v3(void) | ||
| 827 | { | ||
| 828 | /* | ||
| 829 | * Check if HW_PERF_EVENTS are supported by checking the number of | ||
| 830 | * hardware performance counters. This could ensure the presence of | ||
| 831 | * a physical PMU and CONFIG_PERF_EVENT is selected. | ||
| 832 | */ | ||
| 833 | return (perf_num_counters() > 0); | ||
| 834 | } | ||
| 835 | |||
| 836 | int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu) | 826 | int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu) |
| 837 | { | 827 | { |
| 838 | if (!kvm_vcpu_has_pmu(vcpu)) | 828 | if (!kvm_vcpu_has_pmu(vcpu)) |
| @@ -311,23 +311,24 @@ int kvm_set_ipa_limit(void) · 310 unmodified lines | |||
| 311 | } | 311 | } |
| 312 | 312 | ||
| 313 | switch (cpuid_feature_extract_unsigned_field(mmfr0, tgran_2)) { | 313 | switch (cpuid_feature_extract_unsigned_field(mmfr0, tgran_2)) { |
| 314 | default: | 314 | case ID_AA64MMFR0_TGRAN_2_SUPPORTED_NONE: |
| 315 | case 1: | ||
| 316 | kvm_err("PAGE_SIZE not supported at Stage-2, giving up\n"); | 315 | kvm_err("PAGE_SIZE not supported at Stage-2, giving up\n"); |
| 317 | return -EINVAL; | 316 | return -EINVAL; |
| 318 | case 0: | 317 | case ID_AA64MMFR0_TGRAN_2_SUPPORTED_DEFAULT: |
| 319 | kvm_debug("PAGE_SIZE supported at Stage-2 (default)\n"); | 318 | kvm_debug("PAGE_SIZE supported at Stage-2 (default)\n"); |
| 320 | break; | 319 | break; |
| 321 | case 2: | 320 | case ID_AA64MMFR0_TGRAN_2_SUPPORTED_MIN ... ID_AA64MMFR0_TGRAN_2_SUPPORTED_MAX: |
| 322 | kvm_debug("PAGE_SIZE supported at Stage-2 (advertised)\n"); | 321 | kvm_debug("PAGE_SIZE supported at Stage-2 (advertised)\n"); |
| 323 | break; | 322 | break; |
| 323 | default: | ||
| 324 | kvm_err("Unsupported value for TGRAN_2, giving up\n"); | ||
| 325 | return -EINVAL; | ||
| 324 | } | 326 | } |
| 325 | 327 | ||
| 326 | kvm_ipa_limit = id_aa64mmfr0_parange_to_phys_shift(parange); | 328 | kvm_ipa_limit = id_aa64mmfr0_parange_to_phys_shift(parange); |
| 327 | WARN(kvm_ipa_limit < KVM_PHYS_SHIFT, | 329 | kvm_info("IPA Size Limit: %d bits%s\n", kvm_ipa_limit, |
| 328 | "KVM IPA Size Limit (%d bits) is smaller than default size\n", | 330 | ((kvm_ipa_limit < KVM_PHYS_SHIFT) ? |
| 329 | kvm_ipa_limit); | 331 | " (Reduced IPA size, limited VM/VMM compatibility)" : "")); |
| 330 | kvm_info("IPA Size Limit: %d bits\n", kvm_ipa_limit); | ||
| 331 | 332 | ||
| 332 | return 0; | 333 | return 0; |
| 333 | } | 334 | } |
| @@ -356,6 +357,11 @@ int kvm_arm_setup_stage2(struct kvm *kvm, unsigned long type) · 22 unmodified lines | |||
| 356 | return -EINVAL; | 357 | return -EINVAL; |
| 357 | } else { | 358 | } else { |
| 358 | phys_shift = KVM_PHYS_SHIFT; | 359 | phys_shift = KVM_PHYS_SHIFT; |
| 360 | if (phys_shift > kvm_ipa_limit) { | ||
| 361 | pr_warn_once("%s using unsupported default IPA limit, upgrade your VMM\n", | ||
| 362 | current->comm); | ||
| 363 | return -EINVAL; | ||
| 364 | } | ||
| 359 | } | 365 | } |
| 360 | 366 | ||
| 361 | mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1); | 367 | mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1); |
| @@ -574,9 +574,13 @@ early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable); · 573 unmodified lines | |||
| 574 | */ | 574 | */ |
| 575 | int vgic_v3_probe(const struct gic_kvm_info *info) | 575 | int vgic_v3_probe(const struct gic_kvm_info *info) |
| 576 | { | 576 | { |
| 577 | u32 ich_vtr_el2 = kvm_call_hyp_ret(__vgic_v3_get_ich_vtr_el2); | 577 | u64 ich_vtr_el2 = kvm_call_hyp_ret(__vgic_v3_get_gic_config); |
| 578 | bool has_v2; | ||
| 578 | int ret; | 579 | int ret; |
| 579 | 580 | ||
| 581 | has_v2 = ich_vtr_el2 >> 63; | ||
| 582 | ich_vtr_el2 = (u32)ich_vtr_el2; | ||
| 583 | |||
| 580 | /* | 584 | /* |
| 581 | * The ListRegs field is 5 bits, but there is an architectural | 585 | * The ListRegs field is 5 bits, but there is an architectural |
| 582 | * maximum of 16 list registers. Just ignore bit 4... | 586 | * maximum of 16 list registers. Just ignore bit 4... |
| @@ -594,13 +598,15 @@ int vgic_v3_probe(const struct gic_kvm_info *info) · 11 unmodified lines | |||
| 594 | gicv4_enable ? "en" : "dis"); | 598 | gicv4_enable ? "en" : "dis"); |
| 595 | } | 599 | } |
| 596 | 600 | ||
| 601 | kvm_vgic_global_state.vcpu_base = 0; | ||
| 602 | |||
| 597 | if (!info->vcpu.start) { | 603 | if (!info->vcpu.start) { |
| 598 | kvm_info("GICv3: no GICV resource entry\n"); | 604 | kvm_info("GICv3: no GICV resource entry\n"); |
| 599 | kvm_vgic_global_state.vcpu_base = 0; | 605 | } else if (!has_v2) { |
| 606 | pr_warn(FW_BUG "CPU interface incapable of MMIO access\n"); | ||
| 600 | } else if (!PAGE_ALIGNED(info->vcpu.start)) { | 607 | } else if (!PAGE_ALIGNED(info->vcpu.start)) { |
| 601 | pr_warn("GICV physical address 0x%llx not page aligned\n", | 608 | pr_warn("GICV physical address 0x%llx not page aligned\n", |
| 602 | (unsigned long long)info->vcpu.start); | 609 | (unsigned long long)info->vcpu.start); |
| 603 | kvm_vgic_global_state.vcpu_base = 0; | ||
| 604 | } else { | 610 | } else { |
| 605 | kvm_vgic_global_state.vcpu_base = info->vcpu.start; | 611 | kvm_vgic_global_state.vcpu_base = info->vcpu.start; |
| 606 | kvm_vgic_global_state.can_emulate_gicv2 = true; | 612 | kvm_vgic_global_state.can_emulate_gicv2 = true; |
| @@ -219,17 +219,40 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max) · 218 unmodified lines | |||
| 219 | 219 | ||
| 220 | int pfn_valid(unsigned long pfn) | 220 | int pfn_valid(unsigned long pfn) |
| 221 | { | 221 | { |
| 222 | phys_addr_t addr = pfn << PAGE_SHIFT; | 222 | phys_addr_t addr = PFN_PHYS(pfn); |
| 223 | 223 | ||
| 224 | if ((addr >> PAGE_SHIFT) != pfn) | 224 | /* |
| 225 | * Ensure the upper PAGE_SHIFT bits are clear in the | ||
| 226 | * pfn. Else it might lead to false positives when | ||
| 227 | * some of the upper bits are set, but the lower bits | ||
| 228 | * match a valid pfn. | ||
| 229 | */ | ||
| 230 | if (PHYS_PFN(addr) != pfn) | ||
| 225 | return 0; | 231 | return 0; |
| 226 | 232 | ||
| 227 | #ifdef CONFIG_SPARSEMEM | 233 | #ifdef CONFIG_SPARSEMEM |
| 234 | { | ||
| 235 | struct mem_section *ms; | ||
| 236 | |||
| 228 | if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) | 237 | if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) |
| 229 | return 0; | 238 | return 0; |
| 230 | 239 | ||
| 231 | if (!valid_section(__pfn_to_section(pfn))) | 240 | ms = __pfn_to_section(pfn); |
| 241 | if (!valid_section(ms)) | ||
| 232 | return 0; | 242 | return 0; |
| 243 | |||
| 244 | /* | ||
| 245 | * ZONE_DEVICE memory does not have the memblock entries. | ||
| 246 | * memblock_is_map_memory() check for ZONE_DEVICE based | ||
| 247 | * addresses will always fail. Even the normal hotplugged | ||
| 248 | * memory will never have MEMBLOCK_NOMAP flag set in their | ||
| 249 | * memblock entries. Skip memblock search for all non early | ||
| 250 | * memory sections covering all of hotplug memory including | ||
| 251 | * both normal and ZONE_DEVICE based. | ||
| 252 | */ | ||
| 253 | if (!early_section(ms)) | ||
| 254 | return pfn_section_valid(ms, pfn); | ||
| 255 | } | ||
| 233 | #endif | 256 | #endif |
| 234 | return memblock_is_map_memory(addr); | 257 | return memblock_is_map_memory(addr); |
| 235 | } | 258 | } |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | |||
| 40 | #define NO_BLOCK_MAPPINGS BIT(0) | 40 | #define NO_BLOCK_MAPPINGS BIT(0) |
| 41 | #define NO_CONT_MAPPINGS BIT(1) | 41 | #define NO_CONT_MAPPINGS BIT(1) |
| 42 | 42 | ||
| 43 | u64 idmap_t0sz = TCR_T0SZ(VA_BITS); | 43 | u64 idmap_t0sz = TCR_T0SZ(VA_BITS_MIN); |
| 44 | u64 idmap_ptrs_per_pgd = PTRS_PER_PGD; | 44 | u64 idmap_ptrs_per_pgd = PTRS_PER_PGD; |
| 45 | 45 | ||
| 46 | u64 __section(".mmuoff.data.write") vabits_actual; | 46 | u64 __section(".mmuoff.data.write") vabits_actual; |
| @@ -512,7 +512,8 @@ static void __init map_mem(pgd_t *pgdp) · 465 unmodified lines | |||
| 512 | * if MTE is present. Otherwise, it has the same attributes as | 512 | * if MTE is present. Otherwise, it has the same attributes as |
| 513 | * PAGE_KERNEL. | 513 | * PAGE_KERNEL. |
| 514 | */ | 514 | */ |
| 515 | __map_memblock(pgdp, start, end, PAGE_KERNEL_TAGGED, flags); | 515 | __map_memblock(pgdp, start, end, pgprot_tagged(PAGE_KERNEL), |
| 516 | flags); | ||
| 516 | } | 517 | } |
| 517 | 518 | ||
| 518 | /* | 519 | /* |
| @@ -32,7 +32,7 @@ static inline void syscall_rollback(struct task_struct *task, · 31 unmodified lines | |||
| 32 | static inline long syscall_get_error(struct task_struct *task, | 32 | static inline long syscall_get_error(struct task_struct *task, |
| 33 | struct pt_regs *regs) | 33 | struct pt_regs *regs) |
| 34 | { | 34 | { |
| 35 | return regs->r10 == -1 ? regs->r8:0; | 35 | return regs->r10 == -1 ? -regs->r8:0; |
| 36 | } | 36 | } |
| 37 | 37 | ||
| 38 | static inline long syscall_get_return_value(struct task_struct *task, | 38 | static inline long syscall_get_return_value(struct task_struct *task, |
| @@ -2013,27 +2013,39 @@ static void syscall_get_set_args_cb(struct unw_frame_info *info, void *data) · 2012 unmodified lines | |||
| 2013 | { | 2013 | { |
| 2014 | struct syscall_get_set_args *args = data; | 2014 | struct syscall_get_set_args *args = data; |
| 2015 | struct pt_regs *pt = args->regs; | 2015 | struct pt_regs *pt = args->regs; |
| 2016 | unsigned long *krbs, cfm, ndirty; | 2016 | unsigned long *krbs, cfm, ndirty, nlocals, nouts; |
| 2017 | int i, count; | 2017 | int i, count; |
| 2018 | 2018 | ||
| 2019 | if (unw_unwind_to_user(info) < 0) | 2019 | if (unw_unwind_to_user(info) < 0) |
| 2020 | return; | 2020 | return; |
| 2021 | 2021 | ||
| 2022 | /* | ||
| 2023 | * We get here via a few paths: | ||
| 2024 | * - break instruction: cfm is shared with caller. | ||
| 2025 | * syscall args are in out= regs, locals are non-empty. | ||
| 2026 | * - epsinstruction: cfm is set by br.call | ||
| 2027 | * locals don't exist. | ||
| 2028 | * | ||
| 2029 | * For both cases argguments are reachable in cfm.sof - cfm.sol. | ||
| 2030 | * CFM: [ ... | sor: 17..14 | sol : 13..7 | sof : 6..0 ] | ||
| 2031 | */ | ||
| 2022 | cfm = pt->cr_ifs; | 2032 | cfm = pt->cr_ifs; |
| 2033 | nlocals = (cfm >> 7) & 0x7f; /* aka sol */ | ||
| 2034 | nouts = (cfm & 0x7f) - nlocals; /* aka sof - sol */ | ||
| 2023 | krbs = (unsigned long *)info->task + IA64_RBS_OFFSET/8; | 2035 | krbs = (unsigned long *)info->task + IA64_RBS_OFFSET/8; |
| 2024 | ndirty = ia64_rse_num_regs(krbs, krbs + (pt->loadrs >> 19)); | 2036 | ndirty = ia64_rse_num_regs(krbs, krbs + (pt->loadrs >> 19)); |
| 2025 | 2037 | ||
| 2026 | count = 0; | 2038 | count = 0; |
| 2027 | if (in_syscall(pt)) | 2039 | if (in_syscall(pt)) |
| 2028 | count = min_t(int, args->n, cfm & 0x7f); | 2040 | count = min_t(int, args->n, nouts); |
| 2029 | 2041 | ||
| 2042 | /* Iterate over outs. */ | ||
| 2030 | for (i = 0; i < count; i++) { | 2043 | for (i = 0; i < count; i++) { |
| 2044 | int j = ndirty + nlocals + i + args->i; | ||
| 2031 | if (args->rw) | 2045 | if (args->rw) |
| 2032 | *ia64_rse_skip_regs(krbs, ndirty + i + args->i) = | 2046 | *ia64_rse_skip_regs(krbs, j) = args->args[i]; |
| 2033 | args->args[i]; | ||
| 2034 | else | 2047 | else |
| 2035 | args->args[i] = *ia64_rse_skip_regs(krbs, | 2048 | args->args[i] = *ia64_rse_skip_regs(krbs, j); |
| 2036 | ndirty + i + args->i); | ||
| 2037 | } | 2049 | } |
| 2038 | 2050 | ||
| 2039 | if (!args->rw) { | 2051 | if (!args->rw) { |
| @@ -171,7 +171,7 @@ static inline __attribute_const__ int __virt_to_node_shift(void) · 170 unmodified lines | |||
| 171 | #include <asm-generic/memory_model.h> | 171 | #include <asm-generic/memory_model.h> |
| 172 | #endif | 172 | #endif |
| 173 | 173 | ||
| 174 | #define virt_addr_valid(kaddr) ((void *)(kaddr) >= (void *)PAGE_OFFSET && (void *)(kaddr) < high_memory) | 174 | #define virt_addr_valid(kaddr) ((unsigned long)(kaddr) >= PAGE_OFFSET && (unsigned long)(kaddr) < (unsigned long)high_memory) |
| 175 | #define pfn_valid(pfn) virt_addr_valid(pfn_to_virt(pfn)) | 175 | #define pfn_valid(pfn) virt_addr_valid(pfn_to_virt(pfn)) |
| 176 | 176 | ||
| 177 | #endif /* __ASSEMBLY__ */ | 177 | #endif /* __ASSEMBLY__ */ |
| @@ -30,8 +30,8 @@ extern unsigned long memory_end; · 29 unmodified lines | |||
| 30 | #define page_to_pfn(page) virt_to_pfn(page_to_virt(page)) | 30 | #define page_to_pfn(page) virt_to_pfn(page_to_virt(page)) |
| 31 | #define pfn_valid(pfn) ((pfn) < max_mapnr) | 31 | #define pfn_valid(pfn) ((pfn) < max_mapnr) |
| 32 | 32 | ||
| 33 | #define virt_addr_valid(kaddr) (((void *)(kaddr) >= (void *)PAGE_OFFSET) && \ | 33 | #define virt_addr_valid(kaddr) (((unsigned long)(kaddr) >= PAGE_OFFSET) && \ |
| 34 | ((void *)(kaddr) < (void *)memory_end)) | 34 | ((unsigned long)(kaddr) < memory_end)) |
| 35 | 35 | ||
| 36 | #endif /* __ASSEMBLY__ */ | 36 | #endif /* __ASSEMBLY__ */ |
| 37 | 37 | ||
| @@ -14,6 +14,7 @@ · 13 unmodified lines | |||
| 14 | 14 | ||
| 15 | #include <asm/addrspace.h> | 15 | #include <asm/addrspace.h> |
| 16 | #include <asm/unaligned.h> | 16 | #include <asm/unaligned.h> |
| 17 | #include <asm-generic/vmlinux.lds.h> | ||
| 17 | 18 | ||
| 18 | /* | 19 | /* |
| 19 | * These two variables specify the free mem region | 20 | * These two variables specify the free mem region |
| @@ -120,6 +121,13 @@ void decompress_kernel(unsigned long boot_heap_start) · 100 unmodified lines | |||
| 120 | /* last four bytes is always image size in little endian */ | 121 | /* last four bytes is always image size in little endian */ |
| 121 | image_size = get_unaligned_le32((void *)&__image_end - 4); | 122 | image_size = get_unaligned_le32((void *)&__image_end - 4); |
| 122 | 123 | ||
| 124 | /* The device tree's address must be properly aligned */ | ||
| 125 | image_size = ALIGN(image_size, STRUCT_ALIGNMENT); | ||
| 126 | |||
| 127 | puts("Copy device tree to address "); | ||
| 128 | puthex(VMLINUX_LOAD_ADDRESS_ULL + image_size); | ||
| 129 | puts("\n"); | ||
| 130 | |||
| 123 | /* copy dtb to where the booted kernel will expect it */ | 131 | /* copy dtb to where the booted kernel will expect it */ |
| 124 | memcpy((void *)VMLINUX_LOAD_ADDRESS_ULL + image_size, | 132 | memcpy((void *)VMLINUX_LOAD_ADDRESS_ULL + image_size, |
| 125 | __appended_dtb, dtb_size); | 133 | __appended_dtb, dtb_size); |
| @@ -12,8 +12,8 @@ AFLAGS_chacha-core.o += -O2 # needed to fill branch delay slots · 11 unmodified lines | |||
| 12 | obj-$(CONFIG_CRYPTO_POLY1305_MIPS) += poly1305-mips.o | 12 | obj-$(CONFIG_CRYPTO_POLY1305_MIPS) += poly1305-mips.o |
| 13 | poly1305-mips-y := poly1305-core.o poly1305-glue.o | 13 | poly1305-mips-y := poly1305-core.o poly1305-glue.o |
| 14 | 14 | ||
| 15 | perlasm-flavour-$(CONFIG_CPU_MIPS32) := o32 | 15 | perlasm-flavour-$(CONFIG_32BIT) := o32 |
| 16 | perlasm-flavour-$(CONFIG_CPU_MIPS64) := 64 | 16 | perlasm-flavour-$(CONFIG_64BIT) := 64 |
| 17 | 17 | ||
| 18 | quiet_cmd_perlasm = PERLASM $@ | 18 | quiet_cmd_perlasm = PERLASM $@ |
| 19 | cmd_perlasm = $(PERL) $(<) $(perlasm-flavour-y) $(@) | 19 | cmd_perlasm = $(PERL) $(<) $(perlasm-flavour-y) $(@) |
| @@ -24,8 +24,11 @@ extern void (*board_ebase_setup)(void); · 23 unmodified lines | |||
| 24 | extern void (*board_cache_error_setup)(void); | 24 | extern void (*board_cache_error_setup)(void); |
| 25 | 25 | ||
| 26 | extern int register_nmi_notifier(struct notifier_block *nb); | 26 | extern int register_nmi_notifier(struct notifier_block *nb); |
| 27 | extern void reserve_exception_space(phys_addr_t addr, unsigned long size); | ||
| 27 | extern char except_vec_nmi[]; | 28 | extern char except_vec_nmi[]; |
| 28 | 29 | ||
| 30 | #define VECTORSPACING 0x100 /* for EI/VI mode */ | ||
| 31 | |||
| 29 | #define nmi_notifier(fn, pri) \ | 32 | #define nmi_notifier(fn, pri) \ |
| 30 | ({ \ | 33 | ({ \ |
| 31 | static struct notifier_block fn##_nb = { \ | 34 | static struct notifier_block fn##_nb = { \ |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | |||
| 26 | #include <asm/elf.h> | 26 | #include <asm/elf.h> |
| 27 | #include <asm/pgtable-bits.h> | 27 | #include <asm/pgtable-bits.h> |
| 28 | #include <asm/spram.h> | 28 | #include <asm/spram.h> |
| 29 | #include <asm/traps.h> | ||
| 29 | #include <linux/uaccess.h> | 30 | #include <linux/uaccess.h> |
| 30 | 31 | ||
| 31 | #include "fpu-probe.h" | 32 | #include "fpu-probe.h" |
| @@ -1628,6 +1629,7 @@ static inline void cpu_probe_broadcom(struct cpuinfo_mips *c, unsigned int cpu) · 1596 unmodified lines | |||
| 1628 | c->cputype = CPU_BMIPS3300; | 1629 | c->cputype = CPU_BMIPS3300; |
| 1629 | __cpu_name[cpu] = "Broadcom BMIPS3300"; | 1630 | __cpu_name[cpu] = "Broadcom BMIPS3300"; |
| 1630 | set_elf_platform(cpu, "bmips3300"); | 1631 | set_elf_platform(cpu, "bmips3300"); |
| 1632 | reserve_exception_space(0x400, VECTORSPACING * 64); | ||
| 1631 | break; | 1633 | break; |
| 1632 | case PRID_IMP_BMIPS43XX: { | 1634 | case PRID_IMP_BMIPS43XX: { |
| 1633 | int rev = c->processor_id & PRID_REV_MASK; | 1635 | int rev = c->processor_id & PRID_REV_MASK; |
| @@ -1638,6 +1640,7 @@ static inline void cpu_probe_broadcom(struct cpuinfo_mips *c, unsigned int cpu) · 4 unmodified lines | |||
| 1638 | __cpu_name[cpu] = "Broadcom BMIPS4380"; | 1640 | __cpu_name[cpu] = "Broadcom BMIPS4380"; |
| 1639 | set_elf_platform(cpu, "bmips4380"); | 1641 | set_elf_platform(cpu, "bmips4380"); |
| 1640 | c->options |= MIPS_CPU_RIXI; | 1642 | c->options |= MIPS_CPU_RIXI; |
| 1643 | reserve_exception_space(0x400, VECTORSPACING * 64); | ||
| 1641 | } else { | 1644 | } else { |
| 1642 | c->cputype = CPU_BMIPS4350; | 1645 | c->cputype = CPU_BMIPS4350; |
| 1643 | __cpu_name[cpu] = "Broadcom BMIPS4350"; | 1646 | __cpu_name[cpu] = "Broadcom BMIPS4350"; |
| @@ -1654,6 +1657,7 @@ static inline void cpu_probe_broadcom(struct cpuinfo_mips *c, unsigned int cpu) · 10 unmodified lines | |||
| 1654 | __cpu_name[cpu] = "Broadcom BMIPS5000"; | 1657 | __cpu_name[cpu] = "Broadcom BMIPS5000"; |
| 1655 | set_elf_platform(cpu, "bmips5000"); | 1658 | set_elf_platform(cpu, "bmips5000"); |
| 1656 | c->options |= MIPS_CPU_ULRI | MIPS_CPU_RIXI; | 1659 | c->options |= MIPS_CPU_ULRI | MIPS_CPU_RIXI; |
| 1660 | reserve_exception_space(0x1000, VECTORSPACING * 64); | ||
| 1657 | break; | 1661 | break; |
| 1658 | } | 1662 | } |
| 1659 | } | 1663 | } |
| @@ -2133,6 +2137,8 @@ void cpu_probe(void) · 473 unmodified lines | |||
| 2133 | if (cpu == 0) | 2137 | if (cpu == 0) |
| 2134 | __ua_limit = ~((1ull << cpu_vmbits) - 1); | 2138 | __ua_limit = ~((1ull << cpu_vmbits) - 1); |
| 2135 | #endif | 2139 | #endif |
| 2140 | |||
| 2141 | reserve_exception_space(0, 0x1000); | ||
| 2136 | } | 2142 | } |
| 2137 | 2143 | ||
| 2138 | void cpu_report(void) | 2144 | void cpu_report(void) |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | |||
| 21 | #include <asm/fpu.h> | 21 | #include <asm/fpu.h> |
| 22 | #include <asm/mipsregs.h> | 22 | #include <asm/mipsregs.h> |
| 23 | #include <asm/elf.h> | 23 | #include <asm/elf.h> |
| 24 | #include <asm/traps.h> | ||
| 24 | 25 | ||
| 25 | #include "fpu-probe.h" | 26 | #include "fpu-probe.h" |
| 26 | 27 | ||
| @@ -158,6 +159,8 @@ void cpu_probe(void) · 131 unmodified lines | |||
| 158 | cpu_set_fpu_opts(c); | 159 | cpu_set_fpu_opts(c); |
| 159 | else | 160 | else |
| 160 | cpu_set_nofpu_opts(c); | 161 | cpu_set_nofpu_opts(c); |
| 162 | |||
| 163 | reserve_exception_space(0, 0x400); | ||
| 161 | } | 164 | } |
| 162 | 165 | ||
| 163 | void cpu_report(void) | 166 | void cpu_report(void) |
| @@ -2009,13 +2009,16 @@ void __noreturn nmi_exception_handler(struct pt_regs *regs) · 2008 unmodified lines | |||
| 2009 | nmi_exit(); | 2009 | nmi_exit(); |
| 2010 | } | 2010 | } |
| 2011 | 2011 | ||
| 2012 | #define VECTORSPACING 0x100 /* for EI/VI mode */ | ||
| 2013 | |||
| 2014 | unsigned long ebase; | 2012 | unsigned long ebase; |
| 2015 | EXPORT_SYMBOL_GPL(ebase); | 2013 | EXPORT_SYMBOL_GPL(ebase); |
| 2016 | unsigned long exception_handlers[32]; | 2014 | unsigned long exception_handlers[32]; |
| 2017 | unsigned long vi_handlers[64]; | 2015 | unsigned long vi_handlers[64]; |
| 2018 | 2016 | ||
| 2017 | void reserve_exception_space(phys_addr_t addr, unsigned long size) | ||
| 2018 | { | ||
| 2019 | memblock_reserve(addr, size); | ||
| 2020 | } | ||
| 2021 | |||
| 2019 | void __init *set_except_vector(int n, void *addr) | 2022 | void __init *set_except_vector(int n, void *addr) |
| 2020 | { | 2023 | { |
| 2021 | unsigned long handler = (unsigned long) addr; | 2024 | unsigned long handler = (unsigned long) addr; |
| @@ -2367,10 +2370,7 @@ void __init trap_init(void) · 345 unmodified lines | |||
| 2367 | 2370 | ||
| 2368 | if (!cpu_has_mips_r2_r6) { | 2371 | if (!cpu_has_mips_r2_r6) { |
| 2369 | ebase = CAC_BASE; | 2372 | ebase = CAC_BASE; |
| 2370 | ebase_pa = virt_to_phys((void *)ebase); | ||
| 2371 | vec_size = 0x400; | 2373 | vec_size = 0x400; |
| 2372 | |||
| 2373 | memblock_reserve(ebase_pa, vec_size); | ||
| 2374 | } else { | 2374 | } else { |
| 2375 | if (cpu_has_veic || cpu_has_vint) | 2375 | if (cpu_has_veic || cpu_has_vint) |
| 2376 | vec_size = 0x200 + VECTORSPACING*64; | 2376 | vec_size = 0x200 + VECTORSPACING*64; |
| @@ -145,6 +145,7 @@ SECTIONS · 144 unmodified lines | |||
| 145 | } | 145 | } |
| 146 | 146 | ||
| 147 | #ifdef CONFIG_MIPS_ELF_APPENDED_DTB | 147 | #ifdef CONFIG_MIPS_ELF_APPENDED_DTB |
| 148 | STRUCT_ALIGN(); | ||
| 148 | .appended_dtb : AT(ADDR(.appended_dtb) - LOAD_OFFSET) { | 149 | .appended_dtb : AT(ADDR(.appended_dtb) - LOAD_OFFSET) { |
| 149 | *(.appended_dtb) | 150 | *(.appended_dtb) |
| 150 | KEEP(*(.appended_dtb)) | 151 | KEEP(*(.appended_dtb)) |
| @@ -172,6 +173,11 @@ SECTIONS · 21 unmodified lines | |||
| 172 | #endif | 173 | #endif |
| 173 | 174 | ||
| 174 | #ifdef CONFIG_MIPS_RAW_APPENDED_DTB | 175 | #ifdef CONFIG_MIPS_RAW_APPENDED_DTB |
| 176 | .fill : { | ||
| 177 | FILL(0); | ||
| 178 | BYTE(0); | ||
| 179 | . = ALIGN(8); | ||
| 180 | } | ||
| 175 | __appended_dtb = .; | 181 | __appended_dtb = .; |
| 176 | /* leave space for appended DTB */ | 182 | /* leave space for appended DTB */ |
| 177 | . += 0x100000; | 183 | . += 0x100000; |
| @@ -203,9 +203,12 @@ config PREFETCH · 202 unmodified lines | |||
| 203 | def_bool y | 203 | def_bool y |
| 204 | depends on PA8X00 || PA7200 | 204 | depends on PA8X00 || PA7200 |
| 205 | 205 | ||
| 206 | config PARISC_HUGE_KERNEL | ||
| 207 | def_bool y if !MODULES || UBSAN || FTRACE || COMPILE_TEST | ||
| 208 | |||
| 206 | config MLONGCALLS | 209 | config MLONGCALLS |
| 207 | def_bool y if !MODULES || UBSAN || FTRACE | 210 | def_bool y if PARISC_HUGE_KERNEL |
| 208 | bool "Enable the -mlong-calls compiler option for big kernels" if MODULES && !UBSAN && !FTRACE | 211 | bool "Enable the -mlong-calls compiler option for big kernels" if !PARISC_HUGE_KERNEL |
| 209 | depends on PA8X00 | 212 | depends on PA8X00 |
| 210 | help | 213 | help |
| 211 | If you configure the kernel to include many drivers built-in instead | 214 | If you configure the kernel to include many drivers built-in instead |
| @@ -567,8 +567,6 @@ static const struct user_regset_view user_parisc_native_view = { · 566 unmodified lines | |||
| 567 | }; | 567 | }; |
| 568 | 568 | ||
| 569 | #ifdef CONFIG_64BIT | 569 | #ifdef CONFIG_64BIT |
| 570 | #include <linux/compat.h> | ||
| 571 | |||
| 572 | static int gpr32_get(struct task_struct *target, | 570 | static int gpr32_get(struct task_struct *target, |
| 573 | const struct user_regset *regset, | 571 | const struct user_regset *regset, |
| 574 | struct membuf to) | 572 | struct membuf to) |
| @@ -73,9 +73,10 @@ void __patch_exception(int exc, unsigned long addr); · 72 unmodified lines | |||
| 73 | #endif | 73 | #endif |
| 74 | 74 | ||
| 75 | #define OP_RT_RA_MASK 0xffff0000UL | 75 | #define OP_RT_RA_MASK 0xffff0000UL |
| 76 | #define LIS_R2 0x3c020000UL | 76 | #define LIS_R2 (PPC_INST_ADDIS | __PPC_RT(R2)) |
| 77 | #define ADDIS_R2_R12 0x3c4c0000UL | 77 | #define ADDIS_R2_R12 (PPC_INST_ADDIS | __PPC_RT(R2) | __PPC_RA(R12)) |
| 78 | #define ADDI_R2_R2 0x38420000UL | 78 | #define ADDI_R2_R2 (PPC_INST_ADDI | __PPC_RT(R2) | __PPC_RA(R2)) |
| 79 | |||
| 79 | 80 | ||
| 80 | static inline unsigned long ppc_function_entry(void *func) | 81 | static inline unsigned long ppc_function_entry(void *func) |
| 81 | { | 82 | { |
| @@ -53,8 +53,8 @@ static inline void mtdcrx(unsigned int reg, unsigned int val) · 52 unmodified lines | |||
| 53 | #define mfdcr(rn) \ | 53 | #define mfdcr(rn) \ |
| 54 | ({unsigned int rval; \ | 54 | ({unsigned int rval; \ |
| 55 | if (__builtin_constant_p(rn) && rn < 1024) \ | 55 | if (__builtin_constant_p(rn) && rn < 1024) \ |
| 56 | asm volatile("mfdcr %0," __stringify(rn) \ | 56 | asm volatile("mfdcr %0, %1" : "=r" (rval) \ |
| 57 | : "=r" (rval)); \ | 57 | : "n" (rn)); \ |
| 58 | else if (likely(cpu_has_feature(CPU_FTR_INDEXED_DCR))) \ | 58 | else if (likely(cpu_has_feature(CPU_FTR_INDEXED_DCR))) \ |
| 59 | rval = mfdcrx(rn); \ | 59 | rval = mfdcrx(rn); \ |
| 60 | else \ | 60 | else \ |
| @@ -64,8 +64,8 @@ static inline void mtdcrx(unsigned int reg, unsigned int val) · 3 unmodified lines | |||
| 64 | #define mtdcr(rn, v) \ | 64 | #define mtdcr(rn, v) \ |
| 65 | do { \ | 65 | do { \ |
| 66 | if (__builtin_constant_p(rn) && rn < 1024) \ | 66 | if (__builtin_constant_p(rn) && rn < 1024) \ |
| 67 | asm volatile("mtdcr " __stringify(rn) ",%0" \ | 67 | asm volatile("mtdcr %0, %1" \ |
| 68 | : : "r" (v)); \ | 68 | : : "n" (rn), "r" (v)); \ |
| 69 | else if (likely(cpu_has_feature(CPU_FTR_INDEXED_DCR))) \ | 69 | else if (likely(cpu_has_feature(CPU_FTR_INDEXED_DCR))) \ |
| 70 | mtdcrx(rn, v); \ | 70 | mtdcrx(rn, v); \ |
| 71 | else \ | 71 | else \ |
| @@ -410,7 +410,6 @@ DECLARE_INTERRUPT_HANDLER(altivec_assist_exception); · 409 unmodified lines | |||
| 410 | DECLARE_INTERRUPT_HANDLER(CacheLockingException); | 410 | DECLARE_INTERRUPT_HANDLER(CacheLockingException); |
| 411 | DECLARE_INTERRUPT_HANDLER(SPEFloatingPointException); | 411 | DECLARE_INTERRUPT_HANDLER(SPEFloatingPointException); |
| 412 | DECLARE_INTERRUPT_HANDLER(SPEFloatingPointRoundException); | 412 | DECLARE_INTERRUPT_HANDLER(SPEFloatingPointRoundException); |
| 413 | DECLARE_INTERRUPT_HANDLER(unrecoverable_exception); | ||
| 414 | DECLARE_INTERRUPT_HANDLER(WatchdogException); | 413 | DECLARE_INTERRUPT_HANDLER(WatchdogException); |
| 415 | DECLARE_INTERRUPT_HANDLER(kernel_bad_stack); | 414 | DECLARE_INTERRUPT_HANDLER(kernel_bad_stack); |
| 416 | 415 | ||
| @@ -437,6 +436,8 @@ DECLARE_INTERRUPT_HANDLER_NMI(hmi_exception_realmode); · 20 unmodified lines | |||
| 437 | 436 | ||
| 438 | DECLARE_INTERRUPT_HANDLER_ASYNC(TAUException); | 437 | DECLARE_INTERRUPT_HANDLER_ASYNC(TAUException); |
| 439 | 438 | ||
| 439 | void unrecoverable_exception(struct pt_regs *regs); | ||
| 440 | |||
| 440 | void replay_system_reset(void); | 441 | void replay_system_reset(void); |
| 441 | void replay_soft_interrupts(void); | 442 | void replay_soft_interrupts(void); |
| 442 | 443 | ||
| @@ -228,7 +228,7 @@ enum { · 227 unmodified lines | |||
| 228 | #define MMU_FTRS_ALWAYS 0 | 228 | #define MMU_FTRS_ALWAYS 0 |
| 229 | #endif | 229 | #endif |
| 230 | 230 | ||
| 231 | static inline bool early_mmu_has_feature(unsigned long feature) | 231 | static __always_inline bool early_mmu_has_feature(unsigned long feature) |
| 232 | { | 232 | { |
| 233 | if (MMU_FTRS_ALWAYS & feature) | 233 | if (MMU_FTRS_ALWAYS & feature) |
| 234 | return true; | 234 | return true; |
| @@ -286,7 +286,7 @@ static inline void mmu_feature_keys_init(void) · 51 unmodified lines | |||
| 286 | 286 | ||
| 287 | } | 287 | } |
| 288 | 288 | ||
| 289 | static inline bool mmu_has_feature(unsigned long feature) | 289 | static __always_inline bool mmu_has_feature(unsigned long feature) |
| 290 | { | 290 | { |
| 291 | return early_mmu_has_feature(feature); | 291 | return early_mmu_has_feature(feature); |
| 292 | } | 292 | } |
| @@ -195,7 +195,7 @@ static inline void regs_set_return_value(struct pt_regs *regs, unsigned long rc) · 194 unmodified lines | |||
| 195 | #define TRAP_FLAGS_MASK 0x11 | 195 | #define TRAP_FLAGS_MASK 0x11 |
| 196 | #define TRAP(regs) ((regs)->trap & ~TRAP_FLAGS_MASK) | 196 | #define TRAP(regs) ((regs)->trap & ~TRAP_FLAGS_MASK) |
| 197 | #define FULL_REGS(regs) (((regs)->trap & 1) == 0) | 197 | #define FULL_REGS(regs) (((regs)->trap & 1) == 0) |
| 198 | #define SET_FULL_REGS(regs) ((regs)->trap |= 1) | 198 | #define SET_FULL_REGS(regs) ((regs)->trap &= ~1) |
| 199 | #endif | 199 | #endif |
| 200 | #define CHECK_FULL_REGS(regs) BUG_ON(!FULL_REGS(regs)) | 200 | #define CHECK_FULL_REGS(regs) BUG_ON(!FULL_REGS(regs)) |
| 201 | #define NV_REG_POISON 0xdeadbeefdeadbeefUL | 201 | #define NV_REG_POISON 0xdeadbeefdeadbeefUL |
| @@ -210,7 +210,7 @@ static inline void regs_set_return_value(struct pt_regs *regs, unsigned long rc) · 8 unmodified lines | |||
| 210 | #define TRAP_FLAGS_MASK 0x1F | 210 | #define TRAP_FLAGS_MASK 0x1F |
| 211 | #define TRAP(regs) ((regs)->trap & ~TRAP_FLAGS_MASK) | 211 | #define TRAP(regs) ((regs)->trap & ~TRAP_FLAGS_MASK) |
| 212 | #define FULL_REGS(regs) (((regs)->trap & 1) == 0) | 212 | #define FULL_REGS(regs) (((regs)->trap & 1) == 0) |
| 213 | #define SET_FULL_REGS(regs) ((regs)->trap |= 1) | 213 | #define SET_FULL_REGS(regs) ((regs)->trap &= ~1) |
| 214 | #define IS_CRITICAL_EXC(regs) (((regs)->trap & 2) != 0) | 214 | #define IS_CRITICAL_EXC(regs) (((regs)->trap & 2) != 0) |
| 215 | #define IS_MCHECK_EXC(regs) (((regs)->trap & 4) != 0) | 215 | #define IS_MCHECK_EXC(regs) (((regs)->trap & 4) != 0) |
| 216 | #define IS_DEBUG_EXC(regs) (((regs)->trap & 8) != 0) | 216 | #define IS_DEBUG_EXC(regs) (((regs)->trap & 8) != 0) |
| @@ -71,6 +71,16 @@ static inline void disable_kernel_vsx(void) · 70 unmodified lines | |||
| 71 | { | 71 | { |
| 72 | msr_check_and_clear(MSR_FP|MSR_VEC|MSR_VSX); | 72 | msr_check_and_clear(MSR_FP|MSR_VEC|MSR_VSX); |
| 73 | } | 73 | } |
| 74 | #else | ||
| 75 | static inline void enable_kernel_vsx(void) | ||
| 76 | { | ||
| 77 | BUILD_BUG(); | ||
| 78 | } | ||
| 79 | |||
| 80 | static inline void disable_kernel_vsx(void) | ||
| 81 | { | ||
| 82 | BUILD_BUG(); | ||
| 83 | } | ||
| 74 | #endif | 84 | #endif |
| 75 | 85 | ||
| 76 | #ifdef CONFIG_SPE | 86 | #ifdef CONFIG_SPE |
| @@ -113,7 +113,7 @@ struct vio_driver { · 112 unmodified lines | |||
| 113 | const char *name; | 113 | const char *name; |
| 114 | const struct vio_device_id *id_table; | 114 | const struct vio_device_id *id_table; |
| 115 | int (*probe)(struct vio_dev *dev, const struct vio_device_id *id); | 115 | int (*probe)(struct vio_dev *dev, const struct vio_device_id *id); |
| 116 | int (*remove)(struct vio_dev *dev); | 116 | void (*remove)(struct vio_dev *dev); |
| 117 | /* A driver must have a get_desired_dma() function to | 117 | /* A driver must have a get_desired_dma() function to |
| 118 | * be loaded in a CMO environment if it uses DMA. | 118 | * be loaded in a CMO environment if it uses DMA. |
| 119 | */ | 119 | */ |
| @@ -466,7 +466,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real) · 465 unmodified lines | |||
| 466 | 466 | ||
| 467 | ld r10,PACAKMSR(r13) /* get MSR value for kernel */ | 467 | ld r10,PACAKMSR(r13) /* get MSR value for kernel */ |
| 468 | /* MSR[RI] is clear iff using SRR regs */ | 468 | /* MSR[RI] is clear iff using SRR regs */ |
| 469 | .if IHSRR == EXC_HV_OR_STD | 469 | .if IHSRR_IF_HVMODE |
| 470 | BEGIN_FTR_SECTION | 470 | BEGIN_FTR_SECTION |
| 471 | xori r10,r10,MSR_RI | 471 | xori r10,r10,MSR_RI |
| 472 | END_FTR_SECTION_IFCLR(CPU_FTR_HVMODE) | 472 | END_FTR_SECTION_IFCLR(CPU_FTR_HVMODE) |
| @@ -457,11 +457,12 @@ InstructionTLBMiss: · 456 unmodified lines | |||
| 457 | cmplw 0,r1,r3 | 457 | cmplw 0,r1,r3 |
| 458 | #endif | 458 | #endif |
| 459 | mfspr r2, SPRN_SDR1 | 459 | mfspr r2, SPRN_SDR1 |
| 460 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | 460 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | _PAGE_USER |
| 461 | rlwinm r2, r2, 28, 0xfffff000 | 461 | rlwinm r2, r2, 28, 0xfffff000 |
| 462 | #ifdef CONFIG_MODULES | 462 | #ifdef CONFIG_MODULES |
| 463 | bgt- 112f | 463 | bgt- 112f |
| 464 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ | 464 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ |
| 465 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | ||
| 465 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ | 466 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ |
| 466 | #endif | 467 | #endif |
| 467 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ | 468 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ |
| @@ -520,10 +521,11 @@ DataLoadTLBMiss: · 52 unmodified lines | |||
| 520 | lis r1, TASK_SIZE@h /* check if kernel address */ | 521 | lis r1, TASK_SIZE@h /* check if kernel address */ |
| 521 | cmplw 0,r1,r3 | 522 | cmplw 0,r1,r3 |
| 522 | mfspr r2, SPRN_SDR1 | 523 | mfspr r2, SPRN_SDR1 |
| 523 | li r1, _PAGE_PRESENT | _PAGE_ACCESSED | 524 | li r1, _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_USER |
| 524 | rlwinm r2, r2, 28, 0xfffff000 | 525 | rlwinm r2, r2, 28, 0xfffff000 |
| 525 | bgt- 112f | 526 | bgt- 112f |
| 526 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ | 527 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ |
| 528 | li r1, _PAGE_PRESENT | _PAGE_ACCESSED | ||
| 527 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ | 529 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ |
| 528 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ | 530 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ |
| 529 | lwz r2,0(r2) /* get pmd entry */ | 531 | lwz r2,0(r2) /* get pmd entry */ |
| @@ -597,10 +599,11 @@ DataStoreTLBMiss: · 67 unmodified lines | |||
| 597 | lis r1, TASK_SIZE@h /* check if kernel address */ | 599 | lis r1, TASK_SIZE@h /* check if kernel address */ |
| 598 | cmplw 0,r1,r3 | 600 | cmplw 0,r1,r3 |
| 599 | mfspr r2, SPRN_SDR1 | 601 | mfspr r2, SPRN_SDR1 |
| 600 | li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED | 602 | li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_USER |
| 601 | rlwinm r2, r2, 28, 0xfffff000 | 603 | rlwinm r2, r2, 28, 0xfffff000 |
| 602 | bgt- 112f | 604 | bgt- 112f |
| 603 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ | 605 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ |
| 606 | li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED | ||
| 604 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ | 607 | addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */ |
| 605 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ | 608 | 112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */ |
| 606 | lwz r2,0(r2) /* get pmd entry */ | 609 | lwz r2,0(r2) /* get pmd entry */ |
| @@ -149,7 +149,7 @@ notrace long system_call_exception(long r3, long r4, long r5, · 148 unmodified lines | |||
| 149 | * enabled when the interrupt handler returns (indicating a process-context / | 149 | * enabled when the interrupt handler returns (indicating a process-context / |
| 150 | * synchronous interrupt) then irqs_enabled should be true. | 150 | * synchronous interrupt) then irqs_enabled should be true. |
| 151 | */ | 151 | */ |
| 152 | static notrace inline bool __prep_irq_for_enabled_exit(bool clear_ri) | 152 | static notrace __always_inline bool __prep_irq_for_enabled_exit(bool clear_ri) |
| 153 | { | 153 | { |
| 154 | /* This must be done with RI=1 because tracing may touch vmaps */ | 154 | /* This must be done with RI=1 because tracing may touch vmaps */ |
| 155 | trace_hardirqs_on(); | 155 | trace_hardirqs_on(); |
| @@ -436,7 +436,6 @@ again: · 280 unmodified lines | |||
| 436 | return ret; | 436 | return ret; |
| 437 | } | 437 | } |
| 438 | 438 | ||
| 439 | void unrecoverable_exception(struct pt_regs *regs); | ||
| 440 | void preempt_schedule_irq(void); | 439 | void preempt_schedule_irq(void); |
| 441 | 440 | ||
| 442 | notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr) | 441 | notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr) |
| @@ -2170,7 +2170,7 @@ DEFINE_INTERRUPT_HANDLER(SPEFloatingPointRoundException) · 2169 unmodified lines | |||
| 2170 | * in the MSR is 0. This indicates that SRR0/1 are live, and that | 2170 | * in the MSR is 0. This indicates that SRR0/1 are live, and that |
| 2171 | * we therefore lost state by taking this exception. | 2171 | * we therefore lost state by taking this exception. |
| 2172 | */ | 2172 | */ |
| 2173 | DEFINE_INTERRUPT_HANDLER(unrecoverable_exception) | 2173 | void unrecoverable_exception(struct pt_regs *regs) |
| 2174 | { | 2174 | { |
| 2175 | pr_emerg("Unrecoverable exception %lx at %lx (msr=%lx)\n", | 2175 | pr_emerg("Unrecoverable exception %lx at %lx (msr=%lx)\n", |
| 2176 | regs->trap, regs->nip, regs->msr); | 2176 | regs->trap, regs->nip, regs->msr); |
| @@ -904,7 +904,7 @@ static nokprobe_inline int do_vsx_load(struct instruction_op *op, · 903 unmodified lines | |||
| 904 | if (!address_ok(regs, ea, size) || copy_mem_in(mem, ea, size, regs)) | 904 | if (!address_ok(regs, ea, size) || copy_mem_in(mem, ea, size, regs)) |
| 905 | return -EFAULT; | 905 | return -EFAULT; |
| 906 | 906 | ||
| 907 | nr_vsx_regs = size / sizeof(__vector128); | 907 | nr_vsx_regs = max(1ul, size / sizeof(__vector128)); |
| 908 | emulate_vsx_load(op, buf, mem, cross_endian); | 908 | emulate_vsx_load(op, buf, mem, cross_endian); |
| 909 | preempt_disable(); | 909 | preempt_disable(); |
| 910 | if (reg < 32) { | 910 | if (reg < 32) { |
| @@ -951,7 +951,7 @@ static nokprobe_inline int do_vsx_store(struct instruction_op *op, · 40 unmodified lines | |||
| 951 | if (!address_ok(regs, ea, size)) | 951 | if (!address_ok(regs, ea, size)) |
| 952 | return -EFAULT; | 952 | return -EFAULT; |
| 953 | 953 | ||
| 954 | nr_vsx_regs = size / sizeof(__vector128); | 954 | nr_vsx_regs = max(1ul, size / sizeof(__vector128)); |
| 955 | preempt_disable(); | 955 | preempt_disable(); |
| 956 | if (reg < 32) { | 956 | if (reg < 32) { |
| 957 | /* FP regs + extensions */ | 957 | /* FP regs + extensions */ |
| @@ -222,7 +222,7 @@ static inline void perf_get_data_addr(struct perf_event *event, struct pt_regs * · 221 unmodified lines | |||
| 222 | if (!(mmcra & MMCRA_SAMPLE_ENABLE) || sdar_valid) | 222 | if (!(mmcra & MMCRA_SAMPLE_ENABLE) || sdar_valid) |
| 223 | *addrp = mfspr(SPRN_SDAR); | 223 | *addrp = mfspr(SPRN_SDAR); |
| 224 | 224 | ||
| 225 | if (is_kernel_addr(mfspr(SPRN_SDAR)) && perf_allow_kernel(&event->attr) != 0) | 225 | if (is_kernel_addr(mfspr(SPRN_SDAR)) && event->attr.exclude_kernel) |
| 226 | *addrp = 0; | 226 | *addrp = 0; |
| 227 | } | 227 | } |
| 228 | 228 | ||
| @@ -507,7 +507,7 @@ static void power_pmu_bhrb_read(struct perf_event *event, struct cpu_hw_events * · 278 unmodified lines | |||
| 507 | * addresses, hence include a check before filtering code | 507 | * addresses, hence include a check before filtering code |
| 508 | */ | 508 | */ |
| 509 | if (!(ppmu->flags & PPMU_ARCH_31) && | 509 | if (!(ppmu->flags & PPMU_ARCH_31) && |
| 510 | is_kernel_addr(addr) && perf_allow_kernel(&event->attr) != 0) | 510 | is_kernel_addr(addr) && event->attr.exclude_kernel) |
| 511 | continue; | 511 | continue; |
| 512 | 512 | ||
| 513 | /* Branches are read most recent first (ie. mfbhrb 0 is | 513 | /* Branches are read most recent first (ie. mfbhrb 0 is |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | |||
| 4 | * Copyright 2006-2007 Michael Ellerman, IBM Corp. | 4 | * Copyright 2006-2007 Michael Ellerman, IBM Corp. |
| 5 | */ | 5 | */ |
| 6 | 6 | ||
| 7 | #include <linux/crash_dump.h> | ||
| 7 | #include <linux/device.h> | 8 | #include <linux/device.h> |
| 8 | #include <linux/irq.h> | 9 | #include <linux/irq.h> |
| 9 | #include <linux/msi.h> | 10 | #include <linux/msi.h> |
| @@ -458,8 +459,28 @@ again: · 448 unmodified lines | |||
| 458 | return hwirq; | 459 | return hwirq; |
| 459 | } | 460 | } |
| 460 | 461 | ||
| 461 | virq = irq_create_mapping_affinity(NULL, hwirq, | 462 | /* |
| 462 | entry->affinity); | 463 | * Depending on the number of online CPUs in the original |
| 464 | * kernel, it is likely for CPU #0 to be offline in a kdump | ||
| 465 | * kernel. The associated IRQs in the affinity mappings | ||
| 466 | * provided by irq_create_affinity_masks() are thus not | ||
| 467 | * started by irq_startup(), as per-design for managed IRQs. | ||
| 468 | * This can be a problem with multi-queue block devices driven | ||
| 469 | * by blk-mq : such a non-started IRQ is very likely paired | ||
| 470 | * with the single queue enforced by blk-mq during kdump (see | ||
| 471 | * blk_mq_alloc_tag_set()). This causes the device to remain | ||
| 472 | * silent and likely hangs the guest at some point. | ||
| 473 | * | ||
| 474 | * We don't really care for fine-grained affinity when doing | ||
| 475 | * kdump actually : simply ignore the pre-computed affinity | ||
| 476 | * masks in this case and let the default mask with all CPUs | ||
| 477 | * be used when creating the IRQ mappings. | ||
| 478 | */ | ||
| 479 | if (is_kdump_kernel()) | ||
| 480 | virq = irq_create_mapping(NULL, hwirq); | ||
| 481 | else | ||
| 482 | virq = irq_create_mapping_affinity(NULL, hwirq, | ||
| 483 | entry->affinity); | ||
| 463 | 484 | ||
| 464 | if (!virq) { | 485 | if (!virq) { |
| 465 | pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq); | 486 | pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq); |
| @@ -1261,7 +1261,6 @@ static int vio_bus_remove(struct device *dev) · 1260 unmodified lines | |||
| 1261 | struct vio_dev *viodev = to_vio_dev(dev); | 1261 | struct vio_dev *viodev = to_vio_dev(dev); |
| 1262 | struct vio_driver *viodrv = to_vio_driver(dev->driver); | 1262 | struct vio_driver *viodrv = to_vio_driver(dev->driver); |
| 1263 | struct device *devptr; | 1263 | struct device *devptr; |
| 1264 | int ret = 1; | ||
| 1265 | 1264 | ||
| 1266 | /* | 1265 | /* |
| 1267 | * Hold a reference to the device after the remove function is called | 1266 | * Hold a reference to the device after the remove function is called |
| @@ -1270,13 +1269,13 @@ static int vio_bus_remove(struct device *dev) · 2 unmodified lines | |||
| 1270 | devptr = get_device(dev); | 1269 | devptr = get_device(dev); |
| 1271 | 1270 | ||
| 1272 | if (viodrv->remove) | 1271 | if (viodrv->remove) |
| 1273 | ret = viodrv->remove(viodev); | 1272 | viodrv->remove(viodev); |
| 1274 | 1273 | ||
| 1275 | if (!ret && firmware_has_feature(FW_FEATURE_CMO)) | 1274 | if (firmware_has_feature(FW_FEATURE_CMO)) |
| 1276 | vio_cmo_bus_remove(viodev); | 1275 | vio_cmo_bus_remove(viodev); |
| 1277 | 1276 | ||
| 1278 | put_device(devptr); | 1277 | put_device(devptr); |
| 1279 | return ret; | 1278 | return 0; |
| 1280 | } | 1279 | } |
| 1281 | 1280 | ||
| 1282 | /** | 1281 | /** |
| @@ -275,9 +275,9 @@ CONFIG_IP_VS_DH=m · 274 unmodified lines | |||
| 275 | CONFIG_IP_VS_SH=m | 275 | CONFIG_IP_VS_SH=m |
| 276 | CONFIG_IP_VS_SED=m | 276 | CONFIG_IP_VS_SED=m |
| 277 | CONFIG_IP_VS_NQ=m | 277 | CONFIG_IP_VS_NQ=m |
| 278 | CONFIG_IP_VS_TWOS=m | ||
| 278 | CONFIG_IP_VS_FTP=m | 279 | CONFIG_IP_VS_FTP=m |
| 279 | CONFIG_IP_VS_PE_SIP=m | 280 | CONFIG_IP_VS_PE_SIP=m |
| 280 | CONFIG_NF_TABLES_IPV4=y | ||
| 281 | CONFIG_NFT_FIB_IPV4=m | 281 | CONFIG_NFT_FIB_IPV4=m |
| 282 | CONFIG_NF_TABLES_ARP=y | 282 | CONFIG_NF_TABLES_ARP=y |
| 283 | CONFIG_IP_NF_IPTABLES=m | 283 | CONFIG_IP_NF_IPTABLES=m |
| @@ -298,7 +298,6 @@ CONFIG_IP_NF_SECURITY=m · 14 unmodified lines | |||
| 298 | CONFIG_IP_NF_ARPTABLES=m | 298 | CONFIG_IP_NF_ARPTABLES=m |
| 299 | CONFIG_IP_NF_ARPFILTER=m | 299 | CONFIG_IP_NF_ARPFILTER=m |
| 300 | CONFIG_IP_NF_ARP_MANGLE=m | 300 | CONFIG_IP_NF_ARP_MANGLE=m |
| 301 | CONFIG_NF_TABLES_IPV6=y | ||
| 302 | CONFIG_NFT_FIB_IPV6=m | 301 | CONFIG_NFT_FIB_IPV6=m |
| 303 | CONFIG_IP6_NF_IPTABLES=m | 302 | CONFIG_IP6_NF_IPTABLES=m |
| 304 | CONFIG_IP6_NF_MATCH_AH=m | 303 | CONFIG_IP6_NF_MATCH_AH=m |
| @@ -481,7 +480,6 @@ CONFIG_NLMON=m · 176 unmodified lines | |||
| 481 | # CONFIG_NET_VENDOR_AQUANTIA is not set | 480 | # CONFIG_NET_VENDOR_AQUANTIA is not set |
| 482 | # CONFIG_NET_VENDOR_ARC is not set | 481 | # CONFIG_NET_VENDOR_ARC is not set |
| 483 | # CONFIG_NET_VENDOR_ATHEROS is not set | 482 | # CONFIG_NET_VENDOR_ATHEROS is not set |
| 484 | # CONFIG_NET_VENDOR_AURORA is not set | ||
| 485 | # CONFIG_NET_VENDOR_BROADCOM is not set | 483 | # CONFIG_NET_VENDOR_BROADCOM is not set |
| 486 | # CONFIG_NET_VENDOR_BROCADE is not set | 484 | # CONFIG_NET_VENDOR_BROCADE is not set |
| 487 | # CONFIG_NET_VENDOR_CADENCE is not set | 485 | # CONFIG_NET_VENDOR_CADENCE is not set |
| @@ -581,7 +579,6 @@ CONFIG_VIRTIO_BALLOON=m · 93 unmodified lines | |||
| 581 | CONFIG_VIRTIO_INPUT=y | 579 | CONFIG_VIRTIO_INPUT=y |
| 582 | CONFIG_VHOST_NET=m | 580 | CONFIG_VHOST_NET=m |
| 583 | CONFIG_VHOST_VSOCK=m | 581 | CONFIG_VHOST_VSOCK=m |
| 584 | # CONFIG_SURFACE_PLATFORMS is not set | ||
| 585 | CONFIG_S390_CCW_IOMMU=y | 582 | CONFIG_S390_CCW_IOMMU=y |
| 586 | CONFIG_S390_AP_IOMMU=y | 583 | CONFIG_S390_AP_IOMMU=y |
| 587 | CONFIG_EXT4_FS=y | 584 | CONFIG_EXT4_FS=y |
| @@ -635,6 +632,7 @@ CONFIG_NTFS_RW=y · 47 unmodified lines | |||
| 635 | CONFIG_PROC_KCORE=y | 632 | CONFIG_PROC_KCORE=y |
| 636 | CONFIG_TMPFS=y | 633 | CONFIG_TMPFS=y |
| 637 | CONFIG_TMPFS_POSIX_ACL=y | 634 | CONFIG_TMPFS_POSIX_ACL=y |
| 635 | CONFIG_TMPFS_INODE64=y | ||
| 638 | CONFIG_HUGETLBFS=y | 636 | CONFIG_HUGETLBFS=y |
| 639 | CONFIG_CONFIGFS_FS=m | 637 | CONFIG_CONFIGFS_FS=m |
| 640 | CONFIG_ECRYPT_FS=m | 638 | CONFIG_ECRYPT_FS=m |
| @@ -714,12 +712,8 @@ CONFIG_CRYPTO_VMAC=m · 73 unmodified lines | |||
| 714 | CONFIG_CRYPTO_CRC32=m | 712 | CONFIG_CRYPTO_CRC32=m |
| 715 | CONFIG_CRYPTO_BLAKE2S=m | 713 | CONFIG_CRYPTO_BLAKE2S=m |
| 716 | CONFIG_CRYPTO_MICHAEL_MIC=m | 714 | CONFIG_CRYPTO_MICHAEL_MIC=m |
| 717 | CONFIG_CRYPTO_RMD128=m | ||
| 718 | CONFIG_CRYPTO_RMD160=m | 715 | CONFIG_CRYPTO_RMD160=m |
| 719 | CONFIG_CRYPTO_RMD256=m | ||
| 720 | CONFIG_CRYPTO_RMD320=m | ||
| 721 | CONFIG_CRYPTO_SHA3=m | 716 | CONFIG_CRYPTO_SHA3=m |
| 722 | CONFIG_CRYPTO_TGR192=m | ||
| 723 | CONFIG_CRYPTO_WP512=m | 717 | CONFIG_CRYPTO_WP512=m |
| 724 | CONFIG_CRYPTO_AES_TI=m | 718 | CONFIG_CRYPTO_AES_TI=m |
| 725 | CONFIG_CRYPTO_ANUBIS=m | 719 | CONFIG_CRYPTO_ANUBIS=m |
| @@ -731,7 +725,6 @@ CONFIG_CRYPTO_CAST6=m · 5 unmodified lines | |||
| 731 | CONFIG_CRYPTO_DES=m | 725 | CONFIG_CRYPTO_DES=m |
| 732 | CONFIG_CRYPTO_FCRYPT=m | 726 | CONFIG_CRYPTO_FCRYPT=m |
| 733 | CONFIG_CRYPTO_KHAZAD=m | 727 | CONFIG_CRYPTO_KHAZAD=m |
| 734 | CONFIG_CRYPTO_SALSA20=m | ||
| 735 | CONFIG_CRYPTO_SEED=m | 728 | CONFIG_CRYPTO_SEED=m |
| 736 | CONFIG_CRYPTO_SERPENT=m | 729 | CONFIG_CRYPTO_SERPENT=m |
| 737 | CONFIG_CRYPTO_SM4=m | 730 | CONFIG_CRYPTO_SM4=m |
| @@ -796,12 +789,9 @@ CONFIG_DEBUG_OBJECTS_RCU_HEAD=y · 58 unmodified lines | |||
| 796 | CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER=y | 789 | CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER=y |
| 797 | CONFIG_SLUB_DEBUG_ON=y | 790 | CONFIG_SLUB_DEBUG_ON=y |
| 798 | CONFIG_SLUB_STATS=y | 791 | CONFIG_SLUB_STATS=y |
| 799 | CONFIG_DEBUG_KMEMLEAK=y | ||
| 800 | CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y | ||
| 801 | CONFIG_DEBUG_STACK_USAGE=y | 792 | CONFIG_DEBUG_STACK_USAGE=y |
| 802 | CONFIG_DEBUG_VM=y | 793 | CONFIG_DEBUG_VM=y |
| 803 | CONFIG_DEBUG_VM_VMACACHE=y | 794 | CONFIG_DEBUG_VM_VMACACHE=y |
| 804 | CONFIG_DEBUG_VM_RB=y | ||
| 805 | CONFIG_DEBUG_VM_PGFLAGS=y | 795 | CONFIG_DEBUG_VM_PGFLAGS=y |
| 806 | CONFIG_DEBUG_MEMORY_INIT=y | 796 | CONFIG_DEBUG_MEMORY_INIT=y |
| 807 | CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m | 797 | CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m |
| @@ -838,6 +828,7 @@ CONFIG_BPF_KPROBE_OVERRIDE=y · 30 unmodified lines | |||
| 838 | CONFIG_HIST_TRIGGERS=y | 828 | CONFIG_HIST_TRIGGERS=y |
| 839 | CONFIG_FTRACE_STARTUP_TEST=y | 829 | CONFIG_FTRACE_STARTUP_TEST=y |
| 840 | # CONFIG_EVENT_TRACE_STARTUP_TEST is not set | 830 | # CONFIG_EVENT_TRACE_STARTUP_TEST is not set |
| 831 | CONFIG_DEBUG_ENTRY=y | ||
| 841 | CONFIG_NOTIFIER_ERROR_INJECTION=m | 832 | CONFIG_NOTIFIER_ERROR_INJECTION=m |
| 842 | CONFIG_NETDEV_NOTIFIER_ERROR_INJECT=m | 833 | CONFIG_NETDEV_NOTIFIER_ERROR_INJECT=m |
| 843 | CONFIG_FAULT_INJECTION=y | 834 | CONFIG_FAULT_INJECTION=y |
| @@ -861,4 +852,3 @@ CONFIG_PERCPU_TEST=m · 17 unmodified lines | |||
| 861 | CONFIG_ATOMIC64_SELFTEST=y | 852 | CONFIG_ATOMIC64_SELFTEST=y |
| 862 | CONFIG_TEST_BITOPS=m | 853 | CONFIG_TEST_BITOPS=m |
| 863 | CONFIG_TEST_BPF=m | 854 | CONFIG_TEST_BPF=m |
| 864 | CONFIG_DEBUG_ENTRY=y | ||
| @@ -266,9 +266,9 @@ CONFIG_IP_VS_DH=m · 265 unmodified lines | |||
| 266 | CONFIG_IP_VS_SH=m | 266 | CONFIG_IP_VS_SH=m |
| 267 | CONFIG_IP_VS_SED=m | 267 | CONFIG_IP_VS_SED=m |
| 268 | CONFIG_IP_VS_NQ=m | 268 | CONFIG_IP_VS_NQ=m |
| 269 | CONFIG_IP_VS_TWOS=m | ||
| 269 | CONFIG_IP_VS_FTP=m | 270 | CONFIG_IP_VS_FTP=m |
| 270 | CONFIG_IP_VS_PE_SIP=m | 271 | CONFIG_IP_VS_PE_SIP=m |
| 271 | CONFIG_NF_TABLES_IPV4=y | ||
| 272 | CONFIG_NFT_FIB_IPV4=m | 272 | CONFIG_NFT_FIB_IPV4=m |
| 273 | CONFIG_NF_TABLES_ARP=y | 273 | CONFIG_NF_TABLES_ARP=y |
| 274 | CONFIG_IP_NF_IPTABLES=m | 274 | CONFIG_IP_NF_IPTABLES=m |
| @@ -289,7 +289,6 @@ CONFIG_IP_NF_SECURITY=m · 14 unmodified lines | |||
| 289 | CONFIG_IP_NF_ARPTABLES=m | 289 | CONFIG_IP_NF_ARPTABLES=m |
| 290 | CONFIG_IP_NF_ARPFILTER=m | 290 | CONFIG_IP_NF_ARPFILTER=m |
| 291 | CONFIG_IP_NF_ARP_MANGLE=m | 291 | CONFIG_IP_NF_ARP_MANGLE=m |
| 292 | CONFIG_NF_TABLES_IPV6=y | ||
| 293 | CONFIG_NFT_FIB_IPV6=m | 292 | CONFIG_NFT_FIB_IPV6=m |
| 294 | CONFIG_IP6_NF_IPTABLES=m | 293 | CONFIG_IP6_NF_IPTABLES=m |
| 295 | CONFIG_IP6_NF_MATCH_AH=m | 294 | CONFIG_IP6_NF_MATCH_AH=m |
| @@ -473,7 +472,6 @@ CONFIG_NLMON=m · 177 unmodified lines | |||
| 473 | # CONFIG_NET_VENDOR_AQUANTIA is not set | 472 | # CONFIG_NET_VENDOR_AQUANTIA is not set |
| 474 | # CONFIG_NET_VENDOR_ARC is not set | 473 | # CONFIG_NET_VENDOR_ARC is not set |
| 475 | # CONFIG_NET_VENDOR_ATHEROS is not set | 474 | # CONFIG_NET_VENDOR_ATHEROS is not set |
| 476 | # CONFIG_NET_VENDOR_AURORA is not set | ||
| 477 | # CONFIG_NET_VENDOR_BROADCOM is not set | 475 | # CONFIG_NET_VENDOR_BROADCOM is not set |
| 478 | # CONFIG_NET_VENDOR_BROCADE is not set | 476 | # CONFIG_NET_VENDOR_BROCADE is not set |
| 479 | # CONFIG_NET_VENDOR_CADENCE is not set | 477 | # CONFIG_NET_VENDOR_CADENCE is not set |
| @@ -573,7 +571,6 @@ CONFIG_VIRTIO_BALLOON=m · 93 unmodified lines | |||
| 573 | CONFIG_VIRTIO_INPUT=y | 571 | CONFIG_VIRTIO_INPUT=y |
| 574 | CONFIG_VHOST_NET=m | 572 | CONFIG_VHOST_NET=m |
| 575 | CONFIG_VHOST_VSOCK=m | 573 | CONFIG_VHOST_VSOCK=m |
| 576 | # CONFIG_SURFACE_PLATFORMS is not set | ||
| 577 | CONFIG_S390_CCW_IOMMU=y | 574 | CONFIG_S390_CCW_IOMMU=y |
| 578 | CONFIG_S390_AP_IOMMU=y | 575 | CONFIG_S390_AP_IOMMU=y |
| 579 | CONFIG_EXT4_FS=y | 576 | CONFIG_EXT4_FS=y |
| @@ -623,6 +620,7 @@ CONFIG_NTFS_RW=y · 43 unmodified lines | |||
| 623 | CONFIG_PROC_KCORE=y | 620 | CONFIG_PROC_KCORE=y |
| 624 | CONFIG_TMPFS=y | 621 | CONFIG_TMPFS=y |
| 625 | CONFIG_TMPFS_POSIX_ACL=y | 622 | CONFIG_TMPFS_POSIX_ACL=y |
| 623 | CONFIG_TMPFS_INODE64=y | ||
| 626 | CONFIG_HUGETLBFS=y | 624 | CONFIG_HUGETLBFS=y |
| 627 | CONFIG_CONFIGFS_FS=m | 625 | CONFIG_CONFIGFS_FS=m |
| 628 | CONFIG_ECRYPT_FS=m | 626 | CONFIG_ECRYPT_FS=m |
| @@ -703,12 +701,8 @@ CONFIG_CRYPTO_VMAC=m · 74 unmodified lines | |||
| 703 | CONFIG_CRYPTO_CRC32=m | 701 | CONFIG_CRYPTO_CRC32=m |
| 704 | CONFIG_CRYPTO_BLAKE2S=m | 702 | CONFIG_CRYPTO_BLAKE2S=m |
| 705 | CONFIG_CRYPTO_MICHAEL_MIC=m | 703 | CONFIG_CRYPTO_MICHAEL_MIC=m |
| 706 | CONFIG_CRYPTO_RMD128=m | ||
| 707 | CONFIG_CRYPTO_RMD160=m | 704 | CONFIG_CRYPTO_RMD160=m |
| 708 | CONFIG_CRYPTO_RMD256=m | ||
| 709 | CONFIG_CRYPTO_RMD320=m | ||
| 710 | CONFIG_CRYPTO_SHA3=m | 705 | CONFIG_CRYPTO_SHA3=m |
| 711 | CONFIG_CRYPTO_TGR192=m | ||
| 712 | CONFIG_CRYPTO_WP512=m | 706 | CONFIG_CRYPTO_WP512=m |
| 713 | CONFIG_CRYPTO_AES_TI=m | 707 | CONFIG_CRYPTO_AES_TI=m |
| 714 | CONFIG_CRYPTO_ANUBIS=m | 708 | CONFIG_CRYPTO_ANUBIS=m |
| @@ -720,7 +714,6 @@ CONFIG_CRYPTO_CAST6=m · 5 unmodified lines | |||
| 720 | CONFIG_CRYPTO_DES=m | 714 | CONFIG_CRYPTO_DES=m |
| 721 | CONFIG_CRYPTO_FCRYPT=m | 715 | CONFIG_CRYPTO_FCRYPT=m |
| 722 | CONFIG_CRYPTO_KHAZAD=m | 716 | CONFIG_CRYPTO_KHAZAD=m |
| 723 | CONFIG_CRYPTO_SALSA20=m | ||
| 724 | CONFIG_CRYPTO_SEED=m | 717 | CONFIG_CRYPTO_SEED=m |
| 725 | CONFIG_CRYPTO_SERPENT=m | 718 | CONFIG_CRYPTO_SERPENT=m |
| 726 | CONFIG_CRYPTO_SM4=m | 719 | CONFIG_CRYPTO_SM4=m |
| @@ -26,7 +26,6 @@ CONFIG_CRASH_DUMP=y · 25 unmodified lines | |||
| 26 | # CONFIG_SECCOMP is not set | 26 | # CONFIG_SECCOMP is not set |
| 27 | # CONFIG_GCC_PLUGINS is not set | 27 | # CONFIG_GCC_PLUGINS is not set |
| 28 | CONFIG_PARTITION_ADVANCED=y | 28 | CONFIG_PARTITION_ADVANCED=y |
| 29 | CONFIG_IBM_PARTITION=y | ||
| 30 | # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set | 29 | # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set |
| 31 | # CONFIG_COMPACTION is not set | 30 | # CONFIG_COMPACTION is not set |
| 32 | # CONFIG_MIGRATION is not set | 31 | # CONFIG_MIGRATION is not set |
| @@ -61,11 +60,9 @@ CONFIG_RAW_DRIVER=y · 28 unmodified lines | |||
| 61 | # CONFIG_HID is not set | 60 | # CONFIG_HID is not set |
| 62 | # CONFIG_VIRTIO_MENU is not set | 61 | # CONFIG_VIRTIO_MENU is not set |
| 63 | # CONFIG_VHOST_MENU is not set | 62 | # CONFIG_VHOST_MENU is not set |
| 64 | # CONFIG_SURFACE_PLATFORMS is not set | ||
| 65 | # CONFIG_IOMMU_SUPPORT is not set | 63 | # CONFIG_IOMMU_SUPPORT is not set |
| 66 | # CONFIG_DNOTIFY is not set | 64 | # CONFIG_DNOTIFY is not set |
| 67 | # CONFIG_INOTIFY_USER is not set | 65 | # CONFIG_INOTIFY_USER is not set |
| 68 | CONFIG_CONFIGFS_FS=y | ||
| 69 | # CONFIG_MISC_FILESYSTEMS is not set | 66 | # CONFIG_MISC_FILESYSTEMS is not set |
| 70 | # CONFIG_NETWORK_FILESYSTEMS is not set | 67 | # CONFIG_NETWORK_FILESYSTEMS is not set |
| 71 | CONFIG_LSM="yama,loadpin,safesetid,integrity" | 68 | CONFIG_LSM="yama,loadpin,safesetid,integrity" |
| @@ -14,12 +14,12 @@ · 13 unmodified lines | |||
| 14 | 14 | ||
| 15 | struct s390_idle_data { | 15 | struct s390_idle_data { |
| 16 | seqcount_t seqcount; | 16 | seqcount_t seqcount; |
| 17 | unsigned long long idle_count; | 17 | unsigned long idle_count; |
| 18 | unsigned long long idle_time; | 18 | unsigned long idle_time; |
| 19 | unsigned long long clock_idle_enter; | 19 | unsigned long clock_idle_enter; |
| 20 | unsigned long long clock_idle_exit; | 20 | unsigned long clock_idle_exit; |
| 21 | unsigned long long timer_idle_enter; | 21 | unsigned long timer_idle_enter; |
| 22 | unsigned long long timer_idle_exit; | 22 | unsigned long timer_idle_exit; |
| 23 | unsigned long mt_cycles_enter[8]; | 23 | unsigned long mt_cycles_enter[8]; |
| 24 | }; | 24 | }; |
| 25 | 25 | ||
| @@ -98,10 +98,10 @@ extern unsigned char ptff_function_mask[16]; · 97 unmodified lines | |||
| 98 | 98 | ||
| 99 | /* Query TOD offset result */ | 99 | /* Query TOD offset result */ |
| 100 | struct ptff_qto { | 100 | struct ptff_qto { |
| 101 | unsigned long long physical_clock; | 101 | unsigned long physical_clock; |
| 102 | unsigned long long tod_offset; | 102 | unsigned long tod_offset; |
| 103 | unsigned long long logical_tod_offset; | 103 | unsigned long logical_tod_offset; |
| 104 | unsigned long long tod_epoch_difference; | 104 | unsigned long tod_epoch_difference; |
| 105 | } __packed; | 105 | } __packed; |
| 106 | 106 | ||
| 107 | static inline int ptff_query(unsigned int nr) | 107 | static inline int ptff_query(unsigned int nr) |
| @@ -151,9 +151,9 @@ struct ptff_qui { · 43 unmodified lines | |||
| 151 | rc; \ | 151 | rc; \ |
| 152 | }) | 152 | }) |
| 153 | 153 | ||
| 154 | static inline unsigned long long local_tick_disable(void) | 154 | static inline unsigned long local_tick_disable(void) |
| 155 | { | 155 | { |
| 156 | unsigned long long old; | 156 | unsigned long old; |
| 157 | 157 | ||
| 158 | old = S390_lowcore.clock_comparator; | 158 | old = S390_lowcore.clock_comparator; |
| 159 | S390_lowcore.clock_comparator = clock_comparator_max; | 159 | S390_lowcore.clock_comparator = clock_comparator_max; |
| @@ -161,7 +161,7 @@ static inline unsigned long long local_tick_disable(void) · 1 unmodified line | |||
| 161 | return old; | 161 | return old; |
| 162 | } | 162 | } |
| 163 | 163 | ||
| 164 | static inline void local_tick_enable(unsigned long long comp) | 164 | static inline void local_tick_enable(unsigned long comp) |
| 165 | { | 165 | { |
| 166 | S390_lowcore.clock_comparator = comp; | 166 | S390_lowcore.clock_comparator = comp; |
| 167 | set_clock_comparator(S390_lowcore.clock_comparator); | 167 | set_clock_comparator(S390_lowcore.clock_comparator); |
| @@ -169,9 +169,9 @@ static inline void local_tick_enable(unsigned long long comp) · 1 unmodified line | |||
| 169 | 169 | ||
| 170 | #define CLOCK_TICK_RATE 1193180 /* Underlying HZ */ | 170 | #define CLOCK_TICK_RATE 1193180 /* Underlying HZ */ |
| 171 | 171 | ||
| 172 | typedef unsigned long long cycles_t; | 172 | typedef unsigned long cycles_t; |
| 173 | 173 | ||
| 174 | static inline unsigned long long get_tod_clock(void) | 174 | static inline unsigned long get_tod_clock(void) |
| 175 | { | 175 | { |
| 176 | union tod_clock clk; | 176 | union tod_clock clk; |
| 177 | 177 | ||
| @@ -179,10 +179,10 @@ static inline unsigned long long get_tod_clock(void) · 1 unmodified line | |||
| 179 | return clk.tod; | 179 | return clk.tod; |
| 180 | } | 180 | } |
| 181 | 181 | ||
| 182 | static inline unsigned long long get_tod_clock_fast(void) | 182 | static inline unsigned long get_tod_clock_fast(void) |
| 183 | { | 183 | { |
| 184 | #ifdef CONFIG_HAVE_MARCH_Z9_109_FEATURES | 184 | #ifdef CONFIG_HAVE_MARCH_Z9_109_FEATURES |
| 185 | unsigned long long clk; | 185 | unsigned long clk; |
| 186 | 186 | ||
| 187 | asm volatile("stckf %0" : "=Q" (clk) : : "cc"); | 187 | asm volatile("stckf %0" : "=Q" (clk) : : "cc"); |
| 188 | return clk; | 188 | return clk; |
| @@ -208,9 +208,9 @@ extern union tod_clock tod_clock_base; · 19 unmodified lines | |||
| 208 | * Therefore preemption must be disabled, otherwise the returned | 208 | * Therefore preemption must be disabled, otherwise the returned |
| 209 | * value is not guaranteed to be monotonic. | 209 | * value is not guaranteed to be monotonic. |
| 210 | */ | 210 | */ |
| 211 | static inline unsigned long long get_tod_clock_monotonic(void) | 211 | static inline unsigned long get_tod_clock_monotonic(void) |
| 212 | { | 212 | { |
| 213 | unsigned long long tod; | 213 | unsigned long tod; |
| 214 | 214 | ||
| 215 | preempt_disable_notrace(); | 215 | preempt_disable_notrace(); |
| 216 | tod = get_tod_clock() - tod_clock_base.tod; | 216 | tod = get_tod_clock() - tod_clock_base.tod; |
| @@ -237,7 +237,7 @@ static inline unsigned long long get_tod_clock_monotonic(void) · 20 unmodified lines | |||
| 237 | * -> ns = (th * 125) + ((tl * 125) >> 9); | 237 | * -> ns = (th * 125) + ((tl * 125) >> 9); |
| 238 | * | 238 | * |
| 239 | */ | 239 | */ |
| 240 | static inline unsigned long long tod_to_ns(unsigned long long todval) | 240 | static inline unsigned long tod_to_ns(unsigned long todval) |
| 241 | { | 241 | { |
| 242 | return ((todval >> 9) * 125) + (((todval & 0x1ff) * 125) >> 9); | 242 | return ((todval >> 9) * 125) + (((todval & 0x1ff) * 125) >> 9); |
| 243 | } | 243 | } |
| @@ -249,10 +249,10 @@ static inline unsigned long long tod_to_ns(unsigned long long todval) · 5 unmodified lines | |||
| 249 | * | 249 | * |
| 250 | * Returns: true if a is later than b | 250 | * Returns: true if a is later than b |
| 251 | */ | 251 | */ |
| 252 | static inline int tod_after(unsigned long long a, unsigned long long b) | 252 | static inline int tod_after(unsigned long a, unsigned long b) |
| 253 | { | 253 | { |
| 254 | if (MACHINE_HAS_SCC) | 254 | if (MACHINE_HAS_SCC) |
| 255 | return (long long) a > (long long) b; | 255 | return (long) a > (long) b; |
| 256 | return a > b; | 256 | return a > b; |
| 257 | } | 257 | } |
| 258 | 258 | ||
| @@ -263,10 +263,10 @@ static inline int tod_after(unsigned long long a, unsigned long long b) · 4 unmodified lines | |||
| 263 | * | 263 | * |
| 264 | * Returns: true if a is later than b | 264 | * Returns: true if a is later than b |
| 265 | */ | 265 | */ |
| 266 | static inline int tod_after_eq(unsigned long long a, unsigned long long b) | 266 | static inline int tod_after_eq(unsigned long a, unsigned long b) |
| 267 | { | 267 | { |
| 268 | if (MACHINE_HAS_SCC) | 268 | if (MACHINE_HAS_SCC) |
| 269 | return (long long) a >= (long long) b; | 269 | return (long) a >= (long) b; |
| 270 | return a >= b; | 270 | return a >= b; |
| 271 | } | 271 | } |
| 272 | 272 | ||
File renamed without changes.
| @@ -47,7 +47,7 @@ void account_idle_time_irq(void) · 46 unmodified lines | |||
| 47 | void arch_cpu_idle(void) | 47 | void arch_cpu_idle(void) |
| 48 | { | 48 | { |
| 49 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); | 49 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); |
| 50 | unsigned long long idle_time; | 50 | unsigned long idle_time; |
| 51 | unsigned long psw_mask; | 51 | unsigned long psw_mask; |
| 52 | 52 | ||
| 53 | /* Wait for external, I/O or machine check interrupt. */ | 53 | /* Wait for external, I/O or machine check interrupt. */ |
| @@ -73,7 +73,7 @@ static ssize_t show_idle_count(struct device *dev, · 19 unmodified lines | |||
| 73 | struct device_attribute *attr, char *buf) | 73 | struct device_attribute *attr, char *buf) |
| 74 | { | 74 | { |
| 75 | struct s390_idle_data *idle = &per_cpu(s390_idle, dev->id); | 75 | struct s390_idle_data *idle = &per_cpu(s390_idle, dev->id); |
| 76 | unsigned long long idle_count; | 76 | unsigned long idle_count; |
| 77 | unsigned int seq; | 77 | unsigned int seq; |
| 78 | 78 | ||
| 79 | do { | 79 | do { |
| @@ -82,14 +82,14 @@ static ssize_t show_idle_count(struct device *dev, · 2 unmodified lines | |||
| 82 | if (READ_ONCE(idle->clock_idle_enter)) | 82 | if (READ_ONCE(idle->clock_idle_enter)) |
| 83 | idle_count++; | 83 | idle_count++; |
| 84 | } while (read_seqcount_retry(&idle->seqcount, seq)); | 84 | } while (read_seqcount_retry(&idle->seqcount, seq)); |
| 85 | return sprintf(buf, "%llu\n", idle_count); | 85 | return sprintf(buf, "%lu\n", idle_count); |
| 86 | } | 86 | } |
| 87 | DEVICE_ATTR(idle_count, 0444, show_idle_count, NULL); | 87 | DEVICE_ATTR(idle_count, 0444, show_idle_count, NULL); |
| 88 | 88 | ||
| 89 | static ssize_t show_idle_time(struct device *dev, | 89 | static ssize_t show_idle_time(struct device *dev, |
| 90 | struct device_attribute *attr, char *buf) | 90 | struct device_attribute *attr, char *buf) |
| 91 | { | 91 | { |
| 92 | unsigned long long now, idle_time, idle_enter, idle_exit, in_idle; | 92 | unsigned long now, idle_time, idle_enter, idle_exit, in_idle; |
| 93 | struct s390_idle_data *idle = &per_cpu(s390_idle, dev->id); | 93 | struct s390_idle_data *idle = &per_cpu(s390_idle, dev->id); |
| 94 | unsigned int seq; | 94 | unsigned int seq; |
| 95 | 95 | ||
| @@ -109,14 +109,14 @@ static ssize_t show_idle_time(struct device *dev, · 13 unmodified lines | |||
| 109 | } | 109 | } |
| 110 | } | 110 | } |
| 111 | idle_time += in_idle; | 111 | idle_time += in_idle; |
| 112 | return sprintf(buf, "%llu\n", idle_time >> 12); | 112 | return sprintf(buf, "%lu\n", idle_time >> 12); |
| 113 | } | 113 | } |
| 114 | DEVICE_ATTR(idle_time_us, 0444, show_idle_time, NULL); | 114 | DEVICE_ATTR(idle_time_us, 0444, show_idle_time, NULL); |
| 115 | 115 | ||
| 116 | u64 arch_cpu_idle_time(int cpu) | 116 | u64 arch_cpu_idle_time(int cpu) |
| 117 | { | 117 | { |
| 118 | struct s390_idle_data *idle = &per_cpu(s390_idle, cpu); | 118 | struct s390_idle_data *idle = &per_cpu(s390_idle, cpu); |
| 119 | unsigned long long now, idle_enter, idle_exit, in_idle; | 119 | unsigned long now, idle_enter, idle_exit, in_idle; |
| 120 | unsigned int seq; | 120 | unsigned int seq; |
| 121 | 121 | ||
| 122 | do { | 122 | do { |
| @@ -269,7 +269,7 @@ static int __hw_perf_event_init(struct perf_event *event, unsigned int type) · 268 unmodified lines | |||
| 269 | case CPUMF_CTR_SET_MAX: | 269 | case CPUMF_CTR_SET_MAX: |
| 270 | /* The counter could not be associated to a counter set */ | 270 | /* The counter could not be associated to a counter set */ |
| 271 | return -EINVAL; | 271 | return -EINVAL; |
| 272 | }; | 272 | } |
| 273 | 273 | ||
| 274 | /* Initialize for using the CPU-measurement counter facility */ | 274 | /* Initialize for using the CPU-measurement counter facility */ |
| 275 | if (!atomic_inc_not_zero(&num_events)) { | 275 | if (!atomic_inc_not_zero(&num_events)) { |
| @@ -26,12 +26,10 @@ · 25 unmodified lines | |||
| 26 | #include <asm/timex.h> | 26 | #include <asm/timex.h> |
| 27 | #include <asm/debug.h> | 27 | #include <asm/debug.h> |
| 28 | 28 | ||
| 29 | #include <asm/perf_cpum_cf_diag.h> | 29 | #include <asm/hwctrset.h> |
| 30 | 30 | ||
| 31 | #define CF_DIAG_CTRSET_DEF 0xfeef /* Counter set header mark */ | 31 | #define CF_DIAG_CTRSET_DEF 0xfeef /* Counter set header mark */ |
| 32 | #define CF_DIAG_MIN_INTERVAL 60 /* Minimum counter set read */ | ||
| 33 | /* interval in seconds */ | 32 | /* interval in seconds */ |
| 34 | static unsigned long cf_diag_interval = CF_DIAG_MIN_INTERVAL; | ||
| 35 | static unsigned int cf_diag_cpu_speed; | 33 | static unsigned int cf_diag_cpu_speed; |
| 36 | static debug_info_t *cf_diag_dbg; | 34 | static debug_info_t *cf_diag_dbg; |
| 37 | 35 | ||
| @@ -729,7 +727,6 @@ static DEFINE_MUTEX(cf_diag_ctrset_mutex); · 691 unmodified lines | |||
| 729 | static struct cf_diag_ctrset { | 727 | static struct cf_diag_ctrset { |
| 730 | unsigned long ctrset; /* Bit mask of counter set to read */ | 728 | unsigned long ctrset; /* Bit mask of counter set to read */ |
| 731 | cpumask_t mask; /* CPU mask to read from */ | 729 | cpumask_t mask; /* CPU mask to read from */ |
| 732 | time64_t lastread; /* Epoch counter set last read */ | ||
| 733 | } cf_diag_ctrset; | 730 | } cf_diag_ctrset; |
| 734 | 731 | ||
| 735 | static void cf_diag_ctrset_clear(void) | 732 | static void cf_diag_ctrset_clear(void) |
| @@ -866,27 +863,16 @@ static int cf_diag_all_read(unsigned long arg) · 130 unmodified lines | |||
| 866 | { | 863 | { |
| 867 | struct cf_diag_call_on_cpu_parm p; | 864 | struct cf_diag_call_on_cpu_parm p; |
| 868 | cpumask_var_t mask; | 865 | cpumask_var_t mask; |
| 869 | time64_t now; | 866 | int rc; |
| 870 | int rc = 0; | ||
| 871 | 867 | ||
| 872 | debug_sprintf_event(cf_diag_dbg, 5, "%s\n", __func__); | 868 | debug_sprintf_event(cf_diag_dbg, 5, "%s\n", __func__); |
| 873 | if (!alloc_cpumask_var(&mask, GFP_KERNEL)) | 869 | if (!alloc_cpumask_var(&mask, GFP_KERNEL)) |
| 874 | return -ENOMEM; | 870 | return -ENOMEM; |
| 875 | now = ktime_get_seconds(); | 871 | |
| 876 | if (cf_diag_ctrset.lastread + cf_diag_interval > now) { | ||
| 877 | debug_sprintf_event(cf_diag_dbg, 5, "%s now %lld " | ||
| 878 | " lastread %lld\n", __func__, now, | ||
| 879 | cf_diag_ctrset.lastread); | ||
| 880 | rc = -EAGAIN; | ||
| 881 | goto out; | ||
| 882 | } else { | ||
| 883 | cf_diag_ctrset.lastread = now; | ||
| 884 | } | ||
| 885 | p.sets = cf_diag_ctrset.ctrset; | 872 | p.sets = cf_diag_ctrset.ctrset; |
| 886 | cpumask_and(mask, &cf_diag_ctrset.mask, cpu_online_mask); | 873 | cpumask_and(mask, &cf_diag_ctrset.mask, cpu_online_mask); |
| 887 | on_each_cpu_mask(mask, cf_diag_cpu_read, &p, 1); | 874 | on_each_cpu_mask(mask, cf_diag_cpu_read, &p, 1); |
| 888 | rc = cf_diag_all_copy(arg, mask); | 875 | rc = cf_diag_all_copy(arg, mask); |
| 889 | out: | ||
| 890 | free_cpumask_var(mask); | 876 | free_cpumask_var(mask); |
| 891 | debug_sprintf_event(cf_diag_dbg, 5, "%s rc %d\n", __func__, rc); | 877 | debug_sprintf_event(cf_diag_dbg, 5, "%s rc %d\n", __func__, rc); |
| 892 | return rc; | 878 | return rc; |
| @@ -68,10 +68,10 @@ EXPORT_SYMBOL(s390_epoch_delta_notifier); · 67 unmodified lines | |||
| 68 | 68 | ||
| 69 | unsigned char ptff_function_mask[16]; | 69 | unsigned char ptff_function_mask[16]; |
| 70 | 70 | ||
| 71 | static unsigned long long lpar_offset; | 71 | static unsigned long lpar_offset; |
| 72 | static unsigned long long initial_leap_seconds; | 72 | static unsigned long initial_leap_seconds; |
| 73 | static unsigned long long tod_steering_end; | 73 | static unsigned long tod_steering_end; |
| 74 | static long long tod_steering_delta; | 74 | static long tod_steering_delta; |
| 75 | 75 | ||
| 76 | /* | 76 | /* |
| 77 | * Get time offsets with PTFF | 77 | * Get time offsets with PTFF |
| @@ -96,7 +96,7 @@ void __init time_early_init(void) · 18 unmodified lines | |||
| 96 | 96 | ||
| 97 | /* get initial leap seconds */ | 97 | /* get initial leap seconds */ |
| 98 | if (ptff_query(PTFF_QUI) && ptff(&qui, sizeof(qui), PTFF_QUI) == 0) | 98 | if (ptff_query(PTFF_QUI) && ptff(&qui, sizeof(qui), PTFF_QUI) == 0) |
| 99 | initial_leap_seconds = (unsigned long long) | 99 | initial_leap_seconds = (unsigned long) |
| 100 | ((long) qui.old_leap * 4096000000L); | 100 | ((long) qui.old_leap * 4096000000L); |
| 101 | } | 101 | } |
| 102 | 102 | ||
| @@ -222,7 +222,7 @@ void __init read_persistent_wall_and_boot_offset(struct timespec64 *wall_time, · 119 unmodified lines | |||
| 222 | 222 | ||
| 223 | static u64 read_tod_clock(struct clocksource *cs) | 223 | static u64 read_tod_clock(struct clocksource *cs) |
| 224 | { | 224 | { |
| 225 | unsigned long long now, adj; | 225 | unsigned long now, adj; |
| 226 | 226 | ||
| 227 | preempt_disable(); /* protect from changes to steering parameters */ | 227 | preempt_disable(); /* protect from changes to steering parameters */ |
| 228 | now = get_tod_clock(); | 228 | now = get_tod_clock(); |
| @@ -362,7 +362,7 @@ static inline int check_sync_clock(void) · 133 unmodified lines | |||
| 362 | * Apply clock delta to the global data structures. | 362 | * Apply clock delta to the global data structures. |
| 363 | * This is called once on the CPU that performed the clock sync. | 363 | * This is called once on the CPU that performed the clock sync. |
| 364 | */ | 364 | */ |
| 365 | static void clock_sync_global(unsigned long long delta) | 365 | static void clock_sync_global(unsigned long delta) |
| 366 | { | 366 | { |
| 367 | unsigned long now, adj; | 367 | unsigned long now, adj; |
| 368 | struct ptff_qto qto; | 368 | struct ptff_qto qto; |
| @@ -378,7 +378,7 @@ static void clock_sync_global(unsigned long long delta) · 9 unmodified lines | |||
| 378 | -(adj >> 15) : (adj >> 15); | 378 | -(adj >> 15) : (adj >> 15); |
| 379 | tod_steering_delta += delta; | 379 | tod_steering_delta += delta; |
| 380 | if ((abs(tod_steering_delta) >> 48) != 0) | 380 | if ((abs(tod_steering_delta) >> 48) != 0) |
| 381 | panic("TOD clock sync offset %lli is too large to drift\n", | 381 | panic("TOD clock sync offset %li is too large to drift\n", |
| 382 | tod_steering_delta); | 382 | tod_steering_delta); |
| 383 | tod_steering_end = now + (abs(tod_steering_delta) << 15); | 383 | tod_steering_end = now + (abs(tod_steering_delta) << 15); |
| 384 | vdso_data->arch_data.tod_steering_end = tod_steering_end; | 384 | vdso_data->arch_data.tod_steering_end = tod_steering_end; |
| @@ -394,7 +394,7 @@ static void clock_sync_global(unsigned long long delta) · 9 unmodified lines | |||
| 394 | * Apply clock delta to the per-CPU data structures of this CPU. | 394 | * Apply clock delta to the per-CPU data structures of this CPU. |
| 395 | * This is called for each online CPU after the call to clock_sync_global. | 395 | * This is called for each online CPU after the call to clock_sync_global. |
| 396 | */ | 396 | */ |
| 397 | static void clock_sync_local(unsigned long long delta) | 397 | static void clock_sync_local(unsigned long delta) |
| 398 | { | 398 | { |
| 399 | /* Add the delta to the clock comparator. */ | 399 | /* Add the delta to the clock comparator. */ |
| 400 | if (S390_lowcore.clock_comparator != clock_comparator_max) { | 400 | if (S390_lowcore.clock_comparator != clock_comparator_max) { |
| @@ -418,7 +418,7 @@ static void __init time_init_wq(void) · 17 unmodified lines | |||
| 418 | struct clock_sync_data { | 418 | struct clock_sync_data { |
| 419 | atomic_t cpus; | 419 | atomic_t cpus; |
| 420 | int in_sync; | 420 | int in_sync; |
| 421 | unsigned long long clock_delta; | 421 | unsigned long clock_delta; |
| 422 | }; | 422 | }; |
| 423 | 423 | ||
| 424 | /* | 424 | /* |
| @@ -538,7 +538,7 @@ static int stpinfo_valid(void) · 113 unmodified lines | |||
| 538 | static int stp_sync_clock(void *data) | 538 | static int stp_sync_clock(void *data) |
| 539 | { | 539 | { |
| 540 | struct clock_sync_data *sync = data; | 540 | struct clock_sync_data *sync = data; |
| 541 | unsigned long long clock_delta, flags; | 541 | u64 clock_delta, flags; |
| 542 | static int first; | 542 | static int first; |
| 543 | int rc; | 543 | int rc; |
| 544 | 544 | ||
| @@ -720,8 +720,8 @@ static ssize_t ctn_id_show(struct device *dev, · 175 unmodified lines | |||
| 720 | 720 | ||
| 721 | mutex_lock(&stp_mutex); | 721 | mutex_lock(&stp_mutex); |
| 722 | if (stpinfo_valid()) | 722 | if (stpinfo_valid()) |
| 723 | ret = sprintf(buf, "%016llx\n", | 723 | ret = sprintf(buf, "%016lx\n", |
| 724 | *(unsigned long long *) stp_info.ctnid); | 724 | *(unsigned long *) stp_info.ctnid); |
| 725 | mutex_unlock(&stp_mutex); | 725 | mutex_unlock(&stp_mutex); |
| 726 | return ret; | 726 | return ret; |
| 727 | } | 727 | } |
| @@ -794,7 +794,7 @@ static ssize_t leap_seconds_scheduled_show(struct device *dev, · 66 unmodified lines | |||
| 794 | if (!stzi.lsoib.p) | 794 | if (!stzi.lsoib.p) |
| 795 | return sprintf(buf, "0,0\n"); | 795 | return sprintf(buf, "0,0\n"); |
| 796 | 796 | ||
| 797 | return sprintf(buf, "%llu,%d\n", | 797 | return sprintf(buf, "%lu,%d\n", |
| 798 | tod_to_ns(stzi.lsoib.nlsout - TOD_UNIX_EPOCH) / NSEC_PER_SEC, | 798 | tod_to_ns(stzi.lsoib.nlsout - TOD_UNIX_EPOCH) / NSEC_PER_SEC, |
| 799 | stzi.lsoib.nlso - stzi.lsoib.also); | 799 | stzi.lsoib.nlso - stzi.lsoib.also); |
| 800 | } | 800 | } |
| @@ -76,8 +76,6 @@ static void cpu_group_map(cpumask_t *dst, struct mask_info *info, unsigned int c · 75 unmodified lines | |||
| 76 | } | 76 | } |
| 77 | info = info->next; | 77 | info = info->next; |
| 78 | } | 78 | } |
| 79 | if (cpumask_empty(&mask)) | ||
| 80 | cpumask_copy(&mask, cpumask_of(cpu)); | ||
| 81 | break; | 79 | break; |
| 82 | case TOPOLOGY_MODE_PACKAGE: | 80 | case TOPOLOGY_MODE_PACKAGE: |
| 83 | cpumask_copy(&mask, cpu_present_mask); | 81 | cpumask_copy(&mask, cpu_present_mask); |
| @@ -1287,7 +1287,7 @@ static u64 __calculate_sltime(struct kvm_vcpu *vcpu) · 1286 unmodified lines | |||
| 1287 | /* already expired? */ | 1287 | /* already expired? */ |
| 1288 | if (cputm >> 63) | 1288 | if (cputm >> 63) |
| 1289 | return 0; | 1289 | return 0; |
| 1290 | return min(sltime, tod_to_ns(cputm)); | 1290 | return min_t(u64, sltime, tod_to_ns(cputm)); |
| 1291 | } | 1291 | } |
| 1292 | } else if (cpu_timer_interrupts_enabled(vcpu)) { | 1292 | } else if (cpu_timer_interrupts_enabled(vcpu)) { |
| 1293 | sltime = kvm_s390_get_cpu_timer(vcpu); | 1293 | sltime = kvm_s390_get_cpu_timer(vcpu); |
| @@ -93,7 +93,7 @@ CONFIG_NETDEVICES=y · 92 unmodified lines | |||
| 93 | CONFIG_NET_ETHERNET=y | 93 | CONFIG_NET_ETHERNET=y |
| 94 | CONFIG_MII=m | 94 | CONFIG_MII=m |
| 95 | CONFIG_SUNLANCE=m | 95 | CONFIG_SUNLANCE=m |
| 96 | CONFIG_HAPPYMEAL=m | 96 | CONFIG_HAPPYMEAL=y |
| 97 | CONFIG_SUNGEM=m | 97 | CONFIG_SUNGEM=m |
| 98 | CONFIG_SUNVNET=m | 98 | CONFIG_SUNVNET=m |
| 99 | CONFIG_LDMVSW=m | 99 | CONFIG_LDMVSW=m |
| @@ -234,9 +234,7 @@ CONFIG_CRYPTO_TWOFISH=m · 134 unmodified lines | |||
| 234 | CONFIG_CRC16=m | 234 | CONFIG_CRC16=m |
| 235 | CONFIG_LIBCRC32C=m | 235 | CONFIG_LIBCRC32C=m |
| 236 | CONFIG_VCC=m | 236 | CONFIG_VCC=m |
| 237 | CONFIG_ATA=y | ||
| 238 | CONFIG_PATA_CMD64X=y | 237 | CONFIG_PATA_CMD64X=y |
| 239 | CONFIG_HAPPYMEAL=y | ||
| 240 | CONFIG_IP_PNP=y | 238 | CONFIG_IP_PNP=y |
| 241 | CONFIG_IP_PNP_DHCP=y | 239 | CONFIG_IP_PNP_DHCP=y |
| 242 | CONFIG_DEVTMPFS=y | 240 | CONFIG_DEVTMPFS=y |
| @@ -8,7 +8,6 @@ · 7 unmodified lines | |||
| 8 | 8 | ||
| 9 | #include <asm/ptrace.h> | 9 | #include <asm/ptrace.h> |
| 10 | #include <asm/processor.h> | 10 | #include <asm/processor.h> |
| 11 | #include <asm/extable_64.h> | ||
| 12 | #include <asm/spitfire.h> | 11 | #include <asm/spitfire.h> |
| 13 | #include <asm/adi.h> | 12 | #include <asm/adi.h> |
| 14 | 13 | ||
| @@ -1,6 +1,6 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | 1 | /* SPDX-License-Identifier: GPL-2.0 */ |
| 2 | #ifndef __ASM_EXTABLE64_H | 2 | #ifndef __ASM_EXTABLE_H |
| 3 | #define __ASM_EXTABLE64_H | 3 | #define __ASM_EXTABLE_H |
| 4 | /* | 4 | /* |
| 5 | * The exception table consists of pairs of addresses: the first is the | 5 | * The exception table consists of pairs of addresses: the first is the |
| 6 | * address of an instruction that is allowed to fault, and the second is | 6 | * address of an instruction that is allowed to fault, and the second is |
| @@ -50,16 +50,12 @@ struct thread_struct { · 49 unmodified lines | |||
| 50 | unsigned long fsr; | 50 | unsigned long fsr; |
| 51 | unsigned long fpqdepth; | 51 | unsigned long fpqdepth; |
| 52 | struct fpq fpqueue[16]; | 52 | struct fpq fpqueue[16]; |
| 53 | unsigned long flags; | ||
| 54 | mm_segment_t current_ds; | 53 | mm_segment_t current_ds; |
| 55 | }; | 54 | }; |
| 56 | 55 | ||
| 57 | #define SPARC_FLAG_KTHREAD 0x1 /* task is a kernel thread */ | ||
| 58 | #define SPARC_FLAG_UNALIGNED 0x2 /* is allowed to do unaligned accesses */ | ||
| 59 | |||
| 60 | #define INIT_THREAD { \ | 56 | #define INIT_THREAD { \ |
| 61 | .flags = SPARC_FLAG_KTHREAD, \ | ||
| 62 | .current_ds = KERNEL_DS, \ | 57 | .current_ds = KERNEL_DS, \ |
| 58 | .kregs = (struct pt_regs *)(init_stack+THREAD_SIZE)-1 \ | ||
| 63 | } | 59 | } |
| 64 | 60 | ||
| 65 | /* Do necessary setup to start up a newly executed thread. */ | 61 | /* Do necessary setup to start up a newly executed thread. */ |
| @@ -118,6 +118,7 @@ struct thread_info { · 117 unmodified lines | |||
| 118 | .task = &tsk, \ | 118 | .task = &tsk, \ |
| 119 | .current_ds = ASI_P, \ | 119 | .current_ds = ASI_P, \ |
| 120 | .preempt_count = INIT_PREEMPT_COUNT, \ | 120 | .preempt_count = INIT_PREEMPT_COUNT, \ |
| 121 | .kregs = (struct pt_regs *)(init_stack+THREAD_SIZE)-1 \ | ||
| 121 | } | 122 | } |
| 122 | 123 | ||
| 123 | /* how to get the thread information struct from C */ | 124 | /* how to get the thread information struct from C */ |
| @@ -1,6 +1,9 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | 1 | /* SPDX-License-Identifier: GPL-2.0 */ |
| 2 | #ifndef ___ASM_SPARC_UACCESS_H | 2 | #ifndef ___ASM_SPARC_UACCESS_H |
| 3 | #define ___ASM_SPARC_UACCESS_H | 3 | #define ___ASM_SPARC_UACCESS_H |
| 4 | |||
| 5 | #include <asm/extable.h> | ||
| 6 | |||
| 4 | #if defined(__sparc__) && defined(__arch64__) | 7 | #if defined(__sparc__) && defined(__arch64__) |
| 5 | #include <asm/uaccess_64.h> | 8 | #include <asm/uaccess_64.h> |
| 6 | #else | 9 | #else |
| @@ -13,9 +13,6 @@ · 12 unmodified lines | |||
| 13 | 13 | ||
| 14 | #include <asm/processor.h> | 14 | #include <asm/processor.h> |
| 15 | 15 | ||
| 16 | #define ARCH_HAS_SORT_EXTABLE | ||
| 17 | #define ARCH_HAS_SEARCH_EXTABLE | ||
| 18 | |||
| 19 | /* Sparc is not segmented, however we need to be able to fool access_ok() | 16 | /* Sparc is not segmented, however we need to be able to fool access_ok() |
| 20 | * when doing system calls from kernel mode legitimately. | 17 | * when doing system calls from kernel mode legitimately. |
| 21 | * | 18 | * |
| @@ -40,36 +37,6 @@ · 18 unmodified lines | |||
| 40 | #define __access_ok(addr, size) (__user_ok((addr) & get_fs().seg, (size))) | 37 | #define __access_ok(addr, size) (__user_ok((addr) & get_fs().seg, (size))) |
| 41 | #define access_ok(addr, size) __access_ok((unsigned long)(addr), size) | 38 | #define access_ok(addr, size) __access_ok((unsigned long)(addr), size) |
| 42 | 39 | ||
| 43 | /* | ||
| 44 | * The exception table consists of pairs of addresses: the first is the | ||
| 45 | * address of an instruction that is allowed to fault, and the second is | ||
| 46 | * the address at which the program should continue. No registers are | ||
| 47 | * modified, so it is entirely up to the continuation code to figure out | ||
| 48 | * what to do. | ||
| 49 | * | ||
| 50 | * All the routines below use bits of fixup code that are out of line | ||
| 51 | * with the main instruction path. This means when everything is well, | ||
| 52 | * we don't even have to jump over them. Further, they do not intrude | ||
| 53 | * on our cache or tlb entries. | ||
| 54 | * | ||
| 55 | * There is a special way how to put a range of potentially faulting | ||
| 56 | * insns (like twenty ldd/std's with now intervening other instructions) | ||
| 57 | * You specify address of first in insn and 0 in fixup and in the next | ||
| 58 | * exception_table_entry you specify last potentially faulting insn + 1 | ||
| 59 | * and in fixup the routine which should handle the fault. | ||
| 60 | * That fixup code will get | ||
| 61 | * (faulting_insn_address - first_insn_in_the_range_address)/4 | ||
| 62 | * in %g2 (ie. index of the faulting instruction in the range). | ||
| 63 | */ | ||
| 64 | |||
| 65 | struct exception_table_entry | ||
| 66 | { | ||
| 67 | unsigned long insn, fixup; | ||
| 68 | }; | ||
| 69 | |||
| 70 | /* Returns 0 if exception not found and fixup otherwise. */ | ||
| 71 | unsigned long search_extables_range(unsigned long addr, unsigned long *g2); | ||
| 72 | |||
| 73 | /* Uh, these should become the main single-value transfer routines.. | 40 | /* Uh, these should become the main single-value transfer routines.. |
| 74 | * They automatically use the right size if we just have the right | 41 | * They automatically use the right size if we just have the right |
| 75 | * pointer type.. | 42 | * pointer type.. |
| @@ -252,12 +219,7 @@ static inline unsigned long __clear_user(void __user *addr, unsigned long size) · 176 unmodified lines | |||
| 252 | unsigned long ret; | 219 | unsigned long ret; |
| 253 | 220 | ||
| 254 | __asm__ __volatile__ ( | 221 | __asm__ __volatile__ ( |
| 255 | ".section __ex_table,#alloc\n\t" | ||
| 256 | ".align 4\n\t" | ||
| 257 | ".word 1f,3\n\t" | ||
| 258 | ".previous\n\t" | ||
| 259 | "mov %2, %%o1\n" | 222 | "mov %2, %%o1\n" |
| 260 | "1:\n\t" | ||
| 261 | "call __bzero\n\t" | 223 | "call __bzero\n\t" |
| 262 | " mov %1, %%o0\n\t" | 224 | " mov %1, %%o0\n\t" |
| 263 | "mov %%o0, %0\n" | 225 | "mov %%o0, %0\n" |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | |||
| 10 | #include <linux/string.h> | 10 | #include <linux/string.h> |
| 11 | #include <asm/asi.h> | 11 | #include <asm/asi.h> |
| 12 | #include <asm/spitfire.h> | 12 | #include <asm/spitfire.h> |
| 13 | #include <asm/extable_64.h> | ||
| 14 | 13 | ||
| 15 | #include <asm/processor.h> | 14 | #include <asm/processor.h> |
| 16 | 15 | ||
| @@ -515,7 +515,7 @@ continue_boot: · 514 unmodified lines | |||
| 515 | 515 | ||
| 516 | /* I want a kernel stack NOW! */ | 516 | /* I want a kernel stack NOW! */ |
| 517 | set init_thread_union, %g1 | 517 | set init_thread_union, %g1 |
| 518 | set (THREAD_SIZE - STACKFRAME_SZ), %g2 | 518 | set (THREAD_SIZE - STACKFRAME_SZ - TRACEREG_SZ), %g2 |
| 519 | add %g1, %g2, %sp | 519 | add %g1, %g2, %sp |
| 520 | mov 0, %fp /* And for good luck */ | 520 | mov 0, %fp /* And for good luck */ |
| 521 | 521 | ||
| @@ -706,7 +706,7 @@ tlb_fixup_done: · 705 unmodified lines | |||
| 706 | wr %g0, ASI_P, %asi | 706 | wr %g0, ASI_P, %asi |
| 707 | mov 1, %g1 | 707 | mov 1, %g1 |
| 708 | sllx %g1, THREAD_SHIFT, %g1 | 708 | sllx %g1, THREAD_SHIFT, %g1 |
| 709 | sub %g1, (STACKFRAME_SZ + STACK_BIAS), %g1 | 709 | sub %g1, (STACKFRAME_SZ + STACK_BIAS + TRACEREG_SZ), %g1 |
| 710 | add %g6, %g1, %sp | 710 | add %g6, %g1, %sp |
| 711 | 711 | ||
| 712 | /* Set per-cpu pointer initially to zero, this makes | 712 | /* Set per-cpu pointer initially to zero, this makes |
| @@ -216,16 +216,6 @@ void flush_thread(void) · 215 unmodified lines | |||
| 216 | clear_thread_flag(TIF_USEDFPU); | 216 | clear_thread_flag(TIF_USEDFPU); |
| 217 | #endif | 217 | #endif |
| 218 | } | 218 | } |
| 219 | |||
| 220 | /* This task is no longer a kernel thread. */ | ||
| 221 | if (current->thread.flags & SPARC_FLAG_KTHREAD) { | ||
| 222 | current->thread.flags &= ~SPARC_FLAG_KTHREAD; | ||
| 223 | |||
| 224 | /* We must fixup kregs as well. */ | ||
| 225 | /* XXX This was not fixed for ti for a while, worked. Unused? */ | ||
| 226 | current->thread.kregs = (struct pt_regs *) | ||
| 227 | (task_stack_page(current) + (THREAD_SIZE - TRACEREG_SZ)); | ||
| 228 | } | ||
| 229 | } | 219 | } |
| 230 | 220 | ||
| 231 | static inline struct sparc_stackf __user * | 221 | static inline struct sparc_stackf __user * |
| @@ -313,7 +303,6 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, unsigned long arg, · 81 unmodified lines | |||
| 313 | extern int nwindows; | 303 | extern int nwindows; |
| 314 | unsigned long psr; | 304 | unsigned long psr; |
| 315 | memset(new_stack, 0, STACKFRAME_SZ + TRACEREG_SZ); | 305 | memset(new_stack, 0, STACKFRAME_SZ + TRACEREG_SZ); |
| 316 | p->thread.flags |= SPARC_FLAG_KTHREAD; | ||
| 317 | p->thread.current_ds = KERNEL_DS; | 306 | p->thread.current_ds = KERNEL_DS; |
| 318 | ti->kpc = (((unsigned long) ret_from_kernel_thread) - 0x8); | 307 | ti->kpc = (((unsigned long) ret_from_kernel_thread) - 0x8); |
| 319 | childregs->u_regs[UREG_G1] = sp; /* function */ | 308 | childregs->u_regs[UREG_G1] = sp; /* function */ |
| @@ -325,7 +314,6 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, unsigned long arg, · 5 unmodified lines | |||
| 325 | } | 314 | } |
| 326 | memcpy(new_stack, (char *)regs - STACKFRAME_SZ, STACKFRAME_SZ + TRACEREG_SZ); | 315 | memcpy(new_stack, (char *)regs - STACKFRAME_SZ, STACKFRAME_SZ + TRACEREG_SZ); |
| 327 | childregs->u_regs[UREG_FP] = sp; | 316 | childregs->u_regs[UREG_FP] = sp; |
| 328 | p->thread.flags &= ~SPARC_FLAG_KTHREAD; | ||
| 329 | p->thread.current_ds = USER_DS; | 317 | p->thread.current_ds = USER_DS; |
| 330 | ti->kpc = (((unsigned long) ret_from_fork) - 0x8); | 318 | ti->kpc = (((unsigned long) ret_from_fork) - 0x8); |
| 331 | ti->kpsr = current->thread.fork_kpsr | PSR_PIL; | 319 | ti->kpsr = current->thread.fork_kpsr | PSR_PIL; |
| @@ -266,7 +266,6 @@ static __init void leon_patch(void) · 265 unmodified lines | |||
| 266 | } | 266 | } |
| 267 | 267 | ||
| 268 | struct tt_entry *sparc_ttable; | 268 | struct tt_entry *sparc_ttable; |
| 269 | static struct pt_regs fake_swapper_regs; | ||
| 270 | 269 | ||
| 271 | /* Called from head_32.S - before we have setup anything | 270 | /* Called from head_32.S - before we have setup anything |
| 272 | * in the kernel. Be very careful with what you do here. | 271 | * in the kernel. Be very careful with what you do here. |
| @@ -363,8 +362,6 @@ void __init setup_arch(char **cmdline_p) · 90 unmodified lines | |||
| 363 | (*(linux_dbvec->teach_debugger))(); | 362 | (*(linux_dbvec->teach_debugger))(); |
| 364 | } | 363 | } |
| 365 | 364 | ||
| 366 | init_task.thread.kregs = &fake_swapper_regs; | ||
| 367 | |||
| 368 | /* Run-time patch instructions to match the cpu model */ | 365 | /* Run-time patch instructions to match the cpu model */ |
| 369 | per_cpu_patch(); | 366 | per_cpu_patch(); |
| 370 | 367 | ||
| @@ -165,8 +165,6 @@ extern int root_mountflags; · 164 unmodified lines | |||
| 165 | 165 | ||
| 166 | char reboot_command[COMMAND_LINE_SIZE]; | 166 | char reboot_command[COMMAND_LINE_SIZE]; |
| 167 | 167 | ||
| 168 | static struct pt_regs fake_swapper_regs = { { 0, }, 0, 0, 0, 0 }; | ||
| 169 | |||
| 170 | static void __init per_cpu_patch(void) | 168 | static void __init per_cpu_patch(void) |
| 171 | { | 169 | { |
| 172 | struct cpuid_patch_entry *p; | 170 | struct cpuid_patch_entry *p; |
| @@ -661,8 +659,6 @@ void __init setup_arch(char **cmdline_p) · 488 unmodified lines | |||
| 661 | rd_image_start = ram_flags & RAMDISK_IMAGE_START_MASK; | 659 | rd_image_start = ram_flags & RAMDISK_IMAGE_START_MASK; |
| 662 | #endif | 660 | #endif |
| 663 | 661 | ||
| 664 | task_thread_info(&init_task)->kregs = &fake_swapper_regs; | ||
| 665 | |||
| 666 | #ifdef CONFIG_IP_PNP | 662 | #ifdef CONFIG_IP_PNP |
| 667 | if (!ic_set_manually) { | 663 | if (!ic_set_manually) { |
| 668 | phandle chosen = prom_finddevice("/chosen"); | 664 | phandle chosen = prom_finddevice("/chosen"); |
| @@ -275,14 +275,13 @@ bool is_no_fault_exception(struct pt_regs *regs) · 274 unmodified lines | |||
| 275 | asi = (regs->tstate >> 24); /* saved %asi */ | 275 | asi = (regs->tstate >> 24); /* saved %asi */ |
| 276 | else | 276 | else |
| 277 | asi = (insn >> 5); /* immediate asi */ | 277 | asi = (insn >> 5); /* immediate asi */ |
| 278 | if ((asi & 0xf2) == ASI_PNF) { | 278 | if ((asi & 0xf6) == ASI_PNF) { |
| 279 | if (insn & 0x1000000) { /* op3[5:4]=3 */ | 279 | if (insn & 0x200000) /* op3[2], stores */ |
| 280 | handle_ldf_stq(insn, regs); | ||
| 281 | return true; | ||
| 282 | } else if (insn & 0x200000) { /* op3[2], stores */ | ||
| 283 | return false; | 280 | return false; |
| 284 | } | 281 | if (insn & 0x1000000) /* op3[5:4]=3 (fp) */ |
| 285 | handle_ld_nf(insn, regs); | 282 | handle_ldf_stq(insn, regs); |
| 283 | else | ||
| 284 | handle_ld_nf(insn, regs); | ||
| 286 | return true; | 285 | return true; |
| 287 | } | 286 | } |
| 288 | } | 287 | } |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | |||
| 16 | #include <linux/uaccess.h> | 16 | #include <linux/uaccess.h> |
| 17 | #include <linux/smp.h> | 17 | #include <linux/smp.h> |
| 18 | #include <linux/perf_event.h> | 18 | #include <linux/perf_event.h> |
| 19 | #include <linux/extable.h> | ||
| 19 | 20 | ||
| 20 | #include <asm/setup.h> | 21 | #include <asm/setup.h> |
| 21 | 22 | ||
| @@ -213,10 +214,10 @@ static inline int ok_for_kernel(unsigned int insn) · 191 unmodified lines | |||
| 213 | 214 | ||
| 214 | static void kernel_mna_trap_fault(struct pt_regs *regs, unsigned int insn) | 215 | static void kernel_mna_trap_fault(struct pt_regs *regs, unsigned int insn) |
| 215 | { | 216 | { |
| 216 | unsigned long g2 = regs->u_regs [UREG_G2]; | 217 | const struct exception_table_entry *entry; |
| 217 | unsigned long fixup = search_extables_range(regs->pc, &g2); | ||
| 218 | 218 | ||
| 219 | if (!fixup) { | 219 | entry = search_exception_tables(regs->pc); |
| 220 | if (!entry) { | ||
| 220 | unsigned long address = compute_effective_address(regs, insn); | 221 | unsigned long address = compute_effective_address(regs, insn); |
| 221 | if(address < PAGE_SIZE) { | 222 | if(address < PAGE_SIZE) { |
| 222 | printk(KERN_ALERT "Unable to handle kernel NULL pointer dereference in mna handler"); | 223 | printk(KERN_ALERT "Unable to handle kernel NULL pointer dereference in mna handler"); |
| @@ -232,9 +233,8 @@ static void kernel_mna_trap_fault(struct pt_regs *regs, unsigned int insn) · 9 unmodified lines | |||
| 232 | die_if_kernel("Oops", regs); | 233 | die_if_kernel("Oops", regs); |
| 233 | /* Not reached */ | 234 | /* Not reached */ |
| 234 | } | 235 | } |
| 235 | regs->pc = fixup; | 236 | regs->pc = entry->fixup; |
| 236 | regs->npc = regs->pc + 4; | 237 | regs->npc = regs->pc + 4; |
| 237 | regs->u_regs [UREG_G2] = g2; | ||
| 238 | } | 238 | } |
| 239 | 239 | ||
| 240 | asmlinkage void kernel_unaligned_trap(struct pt_regs *regs, unsigned int insn) | 240 | asmlinkage void kernel_unaligned_trap(struct pt_regs *regs, unsigned int insn) |
| @@ -274,103 +274,9 @@ asmlinkage void kernel_unaligned_trap(struct pt_regs *regs, unsigned int insn) · 33 unmodified lines | |||
| 274 | } | 274 | } |
| 275 | } | 275 | } |
| 276 | 276 | ||
| 277 | static inline int ok_for_user(struct pt_regs *regs, unsigned int insn, | 277 | asmlinkage void user_unaligned_trap(struct pt_regs *regs, unsigned int insn) |
| 278 | enum direction dir) | ||
| 279 | { | ||
| 280 | unsigned int reg; | ||
| 281 | int size = ((insn >> 19) & 3) == 3 ? 8 : 4; | ||
| 282 | |||
| 283 | if ((regs->pc | regs->npc) & 3) | ||
| 284 | return 0; | ||
| 285 | |||
| 286 | /* Must access_ok() in all the necessary places. */ | ||
| 287 | #define WINREG_ADDR(regnum) \ | ||
| 288 | ((void __user *)(((unsigned long *)regs->u_regs[UREG_FP])+(regnum))) | ||
| 289 | |||
| 290 | reg = (insn >> 25) & 0x1f; | ||
| 291 | if (reg >= 16) { | ||
| 292 | if (!access_ok(WINREG_ADDR(reg - 16), size)) | ||
| 293 | return -EFAULT; | ||
| 294 | } | ||
| 295 | reg = (insn >> 14) & 0x1f; | ||
| 296 | if (reg >= 16) { | ||
| 297 | if (!access_ok(WINREG_ADDR(reg - 16), size)) | ||
| 298 | return -EFAULT; | ||
| 299 | } | ||
| 300 | if (!(insn & 0x2000)) { | ||
| 301 | reg = (insn & 0x1f); | ||
| 302 | if (reg >= 16) { | ||
| 303 | if (!access_ok(WINREG_ADDR(reg - 16), size)) | ||
| 304 | return -EFAULT; | ||
| 305 | } | ||
| 306 | } | ||
| 307 | #undef WINREG_ADDR | ||
| 308 | return 0; | ||
| 309 | } | ||
| 310 | |||
| 311 | static void user_mna_trap_fault(struct pt_regs *regs, unsigned int insn) | ||
| 312 | { | 278 | { |
| 313 | send_sig_fault(SIGBUS, BUS_ADRALN, | 279 | send_sig_fault(SIGBUS, BUS_ADRALN, |
| 314 | (void __user *)safe_compute_effective_address(regs, insn), | 280 | (void __user *)safe_compute_effective_address(regs, insn), |
| 315 | 0, current); | 281 | 0, current); |
| 316 | } | 282 | } |
| 317 | |||
| 318 | asmlinkage void user_unaligned_trap(struct pt_regs *regs, unsigned int insn) | ||
| 319 | { | ||
| 320 | enum direction dir; | ||
| 321 | |||
| 322 | if(!(current->thread.flags & SPARC_FLAG_UNALIGNED) || | ||
| 323 | (((insn >> 30) & 3) != 3)) | ||
| 324 | goto kill_user; | ||
| 325 | dir = decode_direction(insn); | ||
| 326 | if(!ok_for_user(regs, insn, dir)) { | ||
| 327 | goto kill_user; | ||
| 328 | } else { | ||
| 329 | int err, size = decode_access_size(insn); | ||
| 330 | unsigned long addr; | ||
| 331 | |||
| 332 | if(floating_point_load_or_store_p(insn)) { | ||
| 333 | printk("User FPU load/store unaligned unsupported.\n"); | ||
| 334 | goto kill_user; | ||
| 335 | } | ||
| 336 | |||
| 337 | addr = compute_effective_address(regs, insn); | ||
| 338 | perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, addr); | ||
| 339 | switch(dir) { | ||
| 340 | case load: | ||
| 341 | err = do_int_load(fetch_reg_addr(((insn>>25)&0x1f), | ||
| 342 | regs), | ||
| 343 | size, (unsigned long *) addr, | ||
| 344 | decode_signedness(insn)); | ||
| 345 | break; | ||
| 346 | |||
| 347 | case store: | ||
| 348 | err = do_int_store(((insn>>25)&0x1f), size, | ||
| 349 | (unsigned long *) addr, regs); | ||
| 350 | break; | ||
| 351 | |||
| 352 | case both: | ||
| 353 | /* | ||
| 354 | * This was supported in 2.4. However, we question | ||
| 355 | * the value of SWAP instruction across word boundaries. | ||
| 356 | */ | ||
| 357 | printk("Unaligned SWAP unsupported.\n"); | ||
| 358 | err = -EFAULT; | ||
| 359 | break; | ||
| 360 | |||
| 361 | default: | ||
| 362 | unaligned_panic("Impossible user unaligned trap."); | ||
| 363 | goto out; | ||
| 364 | } | ||
| 365 | if (err) | ||
| 366 | goto kill_user; | ||
| 367 | else | ||
| 368 | advance(regs); | ||
| 369 | goto out; | ||
| 370 | } | ||
| 371 | |||
| 372 | kill_user: | ||
| 373 | user_mna_trap_fault(regs, insn); | ||
| 374 | out: | ||
| 375 | ; | ||
| 376 | } | ||
| @@ -155,13 +155,6 @@ cpout: retl ! get outta here · 154 unmodified lines | |||
| 155 | .text; \ | 155 | .text; \ |
| 156 | .align 4 | 156 | .align 4 |
| 157 | 157 | ||
| 158 | #define EXT(start,end) \ | ||
| 159 | .section __ex_table,ALLOC; \ | ||
| 160 | .align 4; \ | ||
| 161 | .word start, 0, end, cc_fault; \ | ||
| 162 | .text; \ | ||
| 163 | .align 4 | ||
| 164 | |||
| 165 | /* This aligned version executes typically in 8.5 superscalar cycles, this | 158 | /* This aligned version executes typically in 8.5 superscalar cycles, this |
| 166 | * is the best I can do. I say 8.5 because the final add will pair with | 159 | * is the best I can do. I say 8.5 because the final add will pair with |
| 167 | * the next ldd in the main unrolled loop. Thus the pipe is always full. | 160 | * the next ldd in the main unrolled loop. Thus the pipe is always full. |
| @@ -169,20 +162,20 @@ cpout: retl ! get outta here · 1 unmodified line | |||
| 169 | * please check the fixup code below as well. | 162 | * please check the fixup code below as well. |
| 170 | */ | 163 | */ |
| 171 | #define CSUMCOPY_BIGCHUNK_ALIGNED(src, dst, sum, off, t0, t1, t2, t3, t4, t5, t6, t7) \ | 164 | #define CSUMCOPY_BIGCHUNK_ALIGNED(src, dst, sum, off, t0, t1, t2, t3, t4, t5, t6, t7) \ |
| 172 | ldd [src + off + 0x00], t0; \ | 165 | EX(ldd [src + off + 0x00], t0); \ |
| 173 | ldd [src + off + 0x08], t2; \ | 166 | EX(ldd [src + off + 0x08], t2); \ |
| 174 | addxcc t0, sum, sum; \ | 167 | addxcc t0, sum, sum; \ |
| 175 | ldd [src + off + 0x10], t4; \ | 168 | EX(ldd [src + off + 0x10], t4); \ |
| 176 | addxcc t1, sum, sum; \ | 169 | addxcc t1, sum, sum; \ |
| 177 | ldd [src + off + 0x18], t6; \ | 170 | EX(ldd [src + off + 0x18], t6); \ |
| 178 | addxcc t2, sum, sum; \ | 171 | addxcc t2, sum, sum; \ |
| 179 | std t0, [dst + off + 0x00]; \ | 172 | EX(std t0, [dst + off + 0x00]); \ |
| 180 | addxcc t3, sum, sum; \ | 173 | addxcc t3, sum, sum; \ |
| 181 | std t2, [dst + off + 0x08]; \ | 174 | EX(std t2, [dst + off + 0x08]); \ |
| 182 | addxcc t4, sum, sum; \ | 175 | addxcc t4, sum, sum; \ |
| 183 | std t4, [dst + off + 0x10]; \ | 176 | EX(std t4, [dst + off + 0x10]); \ |
| 184 | addxcc t5, sum, sum; \ | 177 | addxcc t5, sum, sum; \ |
| 185 | std t6, [dst + off + 0x18]; \ | 178 | EX(std t6, [dst + off + 0x18]); \ |
| 186 | addxcc t6, sum, sum; \ | 179 | addxcc t6, sum, sum; \ |
| 187 | addxcc t7, sum, sum; | 180 | addxcc t7, sum, sum; |
| 188 | 181 | ||
| @@ -191,39 +184,39 @@ cpout: retl ! get outta here · 2 unmodified lines | |||
| 191 | * Viking MXCC into streaming mode. Ho hum... | 184 | * Viking MXCC into streaming mode. Ho hum... |
| 192 | */ | 185 | */ |
| 193 | #define CSUMCOPY_BIGCHUNK(src, dst, sum, off, t0, t1, t2, t3, t4, t5, t6, t7) \ | 186 | #define CSUMCOPY_BIGCHUNK(src, dst, sum, off, t0, t1, t2, t3, t4, t5, t6, t7) \ |
| 194 | ldd [src + off + 0x00], t0; \ | 187 | EX(ldd [src + off + 0x00], t0); \ |
| 195 | ldd [src + off + 0x08], t2; \ | 188 | EX(ldd [src + off + 0x08], t2); \ |
| 196 | ldd [src + off + 0x10], t4; \ | 189 | EX(ldd [src + off + 0x10], t4); \ |
| 197 | ldd [src + off + 0x18], t6; \ | 190 | EX(ldd [src + off + 0x18], t6); \ |
| 198 | st t0, [dst + off + 0x00]; \ | 191 | EX(st t0, [dst + off + 0x00]); \ |
| 199 | addxcc t0, sum, sum; \ | 192 | addxcc t0, sum, sum; \ |
| 200 | st t1, [dst + off + 0x04]; \ | 193 | EX(st t1, [dst + off + 0x04]); \ |
| 201 | addxcc t1, sum, sum; \ | 194 | addxcc t1, sum, sum; \ |
| 202 | st t2, [dst + off + 0x08]; \ | 195 | EX(st t2, [dst + off + 0x08]); \ |
| 203 | addxcc t2, sum, sum; \ | 196 | addxcc t2, sum, sum; \ |
| 204 | st t3, [dst + off + 0x0c]; \ | 197 | EX(st t3, [dst + off + 0x0c]); \ |
| 205 | addxcc t3, sum, sum; \ | 198 | addxcc t3, sum, sum; \ |
| 206 | st t4, [dst + off + 0x10]; \ | 199 | EX(st t4, [dst + off + 0x10]); \ |
| 207 | addxcc t4, sum, sum; \ | 200 | addxcc t4, sum, sum; \ |
| 208 | st t5, [dst + off + 0x14]; \ | 201 | EX(st t5, [dst + off + 0x14]); \ |
| 209 | addxcc t5, sum, sum; \ | 202 | addxcc t5, sum, sum; \ |
| 210 | st t6, [dst + off + 0x18]; \ | 203 | EX(st t6, [dst + off + 0x18]); \ |
| 211 | addxcc t6, sum, sum; \ | 204 | addxcc t6, sum, sum; \ |
| 212 | st t7, [dst + off + 0x1c]; \ | 205 | EX(st t7, [dst + off + 0x1c]); \ |
| 213 | addxcc t7, sum, sum; | 206 | addxcc t7, sum, sum; |
| 214 | 207 | ||
| 215 | /* Yuck, 6 superscalar cycles... */ | 208 | /* Yuck, 6 superscalar cycles... */ |
| 216 | #define CSUMCOPY_LASTCHUNK(src, dst, sum, off, t0, t1, t2, t3) \ | 209 | #define CSUMCOPY_LASTCHUNK(src, dst, sum, off, t0, t1, t2, t3) \ |
| 217 | ldd [src - off - 0x08], t0; \ | 210 | EX(ldd [src - off - 0x08], t0); \ |
| 218 | ldd [src - off - 0x00], t2; \ | 211 | EX(ldd [src - off - 0x00], t2); \ |
| 219 | addxcc t0, sum, sum; \ | 212 | addxcc t0, sum, sum; \ |
| 220 | st t0, [dst - off - 0x08]; \ | 213 | EX(st t0, [dst - off - 0x08]); \ |
| 221 | addxcc t1, sum, sum; \ | 214 | addxcc t1, sum, sum; \ |
| 222 | st t1, [dst - off - 0x04]; \ | 215 | EX(st t1, [dst - off - 0x04]); \ |
| 223 | addxcc t2, sum, sum; \ | 216 | addxcc t2, sum, sum; \ |
| 224 | st t2, [dst - off - 0x00]; \ | 217 | EX(st t2, [dst - off - 0x00]); \ |
| 225 | addxcc t3, sum, sum; \ | 218 | addxcc t3, sum, sum; \ |
| 226 | st t3, [dst - off + 0x04]; | 219 | EX(st t3, [dst - off + 0x04]); |
| 227 | 220 | ||
| 228 | /* Handle the end cruft code out of band for better cache patterns. */ | 221 | /* Handle the end cruft code out of band for better cache patterns. */ |
| 229 | cc_end_cruft: | 222 | cc_end_cruft: |
| @@ -331,7 +324,6 @@ __csum_partial_copy_sparc_generic: · 101 unmodified lines | |||
| 331 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x20,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 324 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x20,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 332 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x40,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 325 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x40,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 333 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x60,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 326 | CSUMCOPY_BIGCHUNK(%o0,%o1,%g7,0x60,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 334 | 10: EXT(5b, 10b) ! note for exception handling | ||
| 335 | sub %g1, 128, %g1 ! detract from length | 327 | sub %g1, 128, %g1 ! detract from length |
| 336 | addx %g0, %g7, %g7 ! add in last carry bit | 328 | addx %g0, %g7, %g7 ! add in last carry bit |
| 337 | andcc %g1, 0xffffff80, %g0 ! more to csum? | 329 | andcc %g1, 0xffffff80, %g0 ! more to csum? |
| @@ -356,8 +348,7 @@ cctbl: CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x68,%g2,%g3,%g4,%g5) · 18 unmodified lines | |||
| 356 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x28,%g2,%g3,%g4,%g5) | 348 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x28,%g2,%g3,%g4,%g5) |
| 357 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x18,%g2,%g3,%g4,%g5) | 349 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x18,%g2,%g3,%g4,%g5) |
| 358 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x08,%g2,%g3,%g4,%g5) | 350 | CSUMCOPY_LASTCHUNK(%o0,%o1,%g7,0x08,%g2,%g3,%g4,%g5) |
| 359 | 12: EXT(cctbl, 12b) ! note for exception table handling | 351 | 12: addx %g0, %g7, %g7 |
| 360 | addx %g0, %g7, %g7 | ||
| 361 | andcc %o3, 0xf, %g0 ! check for low bits set | 352 | andcc %o3, 0xf, %g0 ! check for low bits set |
| 362 | ccte: bne cc_end_cruft ! something left, handle it out of band | 353 | ccte: bne cc_end_cruft ! something left, handle it out of band |
| 363 | andcc %o3, 8, %g0 ! begin checks for that code | 354 | andcc %o3, 8, %g0 ! begin checks for that code |
| @@ -367,7 +358,6 @@ ccdbl: CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x00,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o · 3 unmodified lines | |||
| 367 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x20,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 358 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x20,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 368 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x40,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 359 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x40,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 369 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x60,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) | 360 | CSUMCOPY_BIGCHUNK_ALIGNED(%o0,%o1,%g7,0x60,%o4,%o5,%g2,%g3,%g4,%g5,%o2,%o3) |
| 370 | 11: EXT(ccdbl, 11b) ! note for exception table handling | ||
| 371 | sub %g1, 128, %g1 ! detract from length | 361 | sub %g1, 128, %g1 ! detract from length |
| 372 | addx %g0, %g7, %g7 ! add in last carry bit | 362 | addx %g0, %g7, %g7 ! add in last carry bit |
| 373 | andcc %g1, 0xffffff80, %g0 ! more to csum? | 363 | andcc %g1, 0xffffff80, %g0 ! more to csum? |
| @@ -21,98 +21,134 @@ · 20 unmodified lines | |||
| 21 | /* Work around cpp -rob */ | 21 | /* Work around cpp -rob */ |
| 22 | #define ALLOC #alloc | 22 | #define ALLOC #alloc |
| 23 | #define EXECINSTR #execinstr | 23 | #define EXECINSTR #execinstr |
| 24 | |||
| 25 | #define EX_ENTRY(l1, l2) \ | ||
| 26 | .section __ex_table,ALLOC; \ | ||
| 27 | .align 4; \ | ||
| 28 | .word l1, l2; \ | ||
| 29 | .text; | ||
| 30 | |||
| 24 | #define EX(x,y,a,b) \ | 31 | #define EX(x,y,a,b) \ |
| 25 | 98: x,y; \ | 32 | 98: x,y; \ |
| 26 | .section .fixup,ALLOC,EXECINSTR; \ | 33 | .section .fixup,ALLOC,EXECINSTR; \ |
| 27 | .align 4; \ | 34 | .align 4; \ |
| 28 | 99: ba fixupretl; \ | 35 | 99: retl; \ |
| 29 | a, b, %g3; \ | 36 | a, b, %o0; \ |
| 30 | .section __ex_table,ALLOC; \ | 37 | EX_ENTRY(98b, 99b) |
| 31 | .align 4; \ | ||
| 32 | .word 98b, 99b; \ | ||
| 33 | .text; \ | ||
| 34 | .align 4 | ||
| 35 | 38 | ||
| 36 | #define EX2(x,y,c,d,e,a,b) \ | 39 | #define EX2(x,y,c,d,e,a,b) \ |
| 37 | 98: x,y; \ | 40 | 98: x,y; \ |
| 38 | .section .fixup,ALLOC,EXECINSTR; \ | 41 | .section .fixup,ALLOC,EXECINSTR; \ |
| 39 | .align 4; \ | 42 | .align 4; \ |
| 40 | 99: c, d, e; \ | 43 | 99: c, d, e; \ |
| 41 | ba fixupretl; \ | 44 | retl; \ |
| 42 | a, b, %g3; \ | 45 | a, b, %o0; \ |
| 43 | .section __ex_table,ALLOC; \ | 46 | EX_ENTRY(98b, 99b) |
| 44 | .align 4; \ | ||
| 45 | .word 98b, 99b; \ | ||
| 46 | .text; \ | ||
| 47 | .align 4 | ||
| 48 | 47 | ||
| 49 | #define EXO2(x,y) \ | 48 | #define EXO2(x,y) \ |
| 50 | 98: x, y; \ | 49 | 98: x, y; \ |
| 51 | .section __ex_table,ALLOC; \ | 50 | EX_ENTRY(98b, 97f) |
| 52 | .align 4; \ | ||
| 53 | .word 98b, 97f; \ | ||
| 54 | .text; \ | ||
| 55 | .align 4 | ||
| 56 | 51 | ||
| 57 | #define EXT(start,end,handler) \ | 52 | #define LD(insn, src, offset, reg, label) \ |
| 58 | .section __ex_table,ALLOC; \ | 53 | 98: insn [%src + (offset)], %reg; \ |
| 59 | .align 4; \ | 54 | .section .fixup,ALLOC,EXECINSTR; \ |
| 60 | .word start, 0, end, handler; \ | 55 | 99: ba label; \ |
| 61 | .text; \ | 56 | mov offset, %g5; \ |
| 62 | .align 4 | 57 | EX_ENTRY(98b, 99b) |
| 63 | 58 | ||
| 64 | /* Please do not change following macros unless you change logic used | 59 | #define ST(insn, dst, offset, reg, label) \ |
| 65 | * in .fixup at the end of this file as well | 60 | 98: insn %reg, [%dst + (offset)]; \ |
| 66 | */ | 61 | .section .fixup,ALLOC,EXECINSTR; \ |
| 62 | 99: ba label; \ | ||
| 63 | mov offset, %g5; \ | ||
| 64 | EX_ENTRY(98b, 99b) | ||
| 67 | 65 | ||
| 68 | /* Both these macros have to start with exactly the same insn */ | 66 | /* Both these macros have to start with exactly the same insn */ |
| 67 | /* left: g7 + (g1 % 128) - offset */ | ||
| 69 | #define MOVE_BIGCHUNK(src, dst, offset, t0, t1, t2, t3, t4, t5, t6, t7) \ | 68 | #define MOVE_BIGCHUNK(src, dst, offset, t0, t1, t2, t3, t4, t5, t6, t7) \ |
| 70 | ldd [%src + (offset) + 0x00], %t0; \ | 69 | LD(ldd, src, offset + 0x00, t0, bigchunk_fault) \ |
| 71 | ldd [%src + (offset) + 0x08], %t2; \ | 70 | LD(ldd, src, offset + 0x08, t2, bigchunk_fault) \ |
| 72 | ldd [%src + (offset) + 0x10], %t4; \ | 71 | LD(ldd, src, offset + 0x10, t4, bigchunk_fault) \ |
| 73 | ldd [%src + (offset) + 0x18], %t6; \ | 72 | LD(ldd, src, offset + 0x18, t6, bigchunk_fault) \ |
| 74 | st %t0, [%dst + (offset) + 0x00]; \ | 73 | ST(st, dst, offset + 0x00, t0, bigchunk_fault) \ |
| 75 | st %t1, [%dst + (offset) + 0x04]; \ | 74 | ST(st, dst, offset + 0x04, t1, bigchunk_fault) \ |
| 76 | st %t2, [%dst + (offset) + 0x08]; \ | 75 | ST(st, dst, offset + 0x08, t2, bigchunk_fault) \ |
| 77 | st %t3, [%dst + (offset) + 0x0c]; \ | 76 | ST(st, dst, offset + 0x0c, t3, bigchunk_fault) \ |
| 78 | st %t4, [%dst + (offset) + 0x10]; \ | 77 | ST(st, dst, offset + 0x10, t4, bigchunk_fault) \ |
| 79 | st %t5, [%dst + (offset) + 0x14]; \ | 78 | ST(st, dst, offset + 0x14, t5, bigchunk_fault) \ |
| 80 | st %t6, [%dst + (offset) + 0x18]; \ | 79 | ST(st, dst, offset + 0x18, t6, bigchunk_fault) \ |
| 81 | st %t7, [%dst + (offset) + 0x1c]; | 80 | ST(st, dst, offset + 0x1c, t7, bigchunk_fault) |
| 82 | 81 | ||
| 82 | /* left: g7 + (g1 % 128) - offset */ | ||
| 83 | #define MOVE_BIGALIGNCHUNK(src, dst, offset, t0, t1, t2, t3, t4, t5, t6, t7) \ | 83 | #define MOVE_BIGALIGNCHUNK(src, dst, offset, t0, t1, t2, t3, t4, t5, t6, t7) \ |
| 84 | ldd [%src + (offset) + 0x00], %t0; \ | 84 | LD(ldd, src, offset + 0x00, t0, bigchunk_fault) \ |
| 85 | ldd [%src + (offset) + 0x08], %t2; \ | 85 | LD(ldd, src, offset + 0x08, t2, bigchunk_fault) \ |
| 86 | ldd [%src + (offset) + 0x10], %t4; \ | 86 | LD(ldd, src, offset + 0x10, t4, bigchunk_fault) \ |
| 87 | ldd [%src + (offset) + 0x18], %t6; \ | 87 | LD(ldd, src, offset + 0x18, t6, bigchunk_fault) \ |
| 88 | std %t0, [%dst + (offset) + 0x00]; \ | 88 | ST(std, dst, offset + 0x00, t0, bigchunk_fault) \ |
| 89 | std %t2, [%dst + (offset) + 0x08]; \ | 89 | ST(std, dst, offset + 0x08, t2, bigchunk_fault) \ |
| 90 | std %t4, [%dst + (offset) + 0x10]; \ | 90 | ST(std, dst, offset + 0x10, t4, bigchunk_fault) \ |
| 91 | std %t6, [%dst + (offset) + 0x18]; | 91 | ST(std, dst, offset + 0x18, t6, bigchunk_fault) |
| 92 | 92 | ||
| 93 | .section .fixup,#alloc,#execinstr | ||
| 94 | bigchunk_fault: | ||
| 95 | sub %g7, %g5, %o0 | ||
| 96 | and %g1, 127, %g1 | ||
| 97 | retl | ||
| 98 | add %o0, %g1, %o0 | ||
| 99 | |||
| 100 | /* left: offset + 16 + (g1 % 16) */ | ||
| 93 | #define MOVE_LASTCHUNK(src, dst, offset, t0, t1, t2, t3) \ | 101 | #define MOVE_LASTCHUNK(src, dst, offset, t0, t1, t2, t3) \ |
| 94 | ldd [%src - (offset) - 0x10], %t0; \ | 102 | LD(ldd, src, -(offset + 0x10), t0, lastchunk_fault) \ |
| 95 | ldd [%src - (offset) - 0x08], %t2; \ | 103 | LD(ldd, src, -(offset + 0x08), t2, lastchunk_fault) \ |
| 96 | st %t0, [%dst - (offset) - 0x10]; \ | 104 | ST(st, dst, -(offset + 0x10), t0, lastchunk_fault) \ |
| 97 | st %t1, [%dst - (offset) - 0x0c]; \ | 105 | ST(st, dst, -(offset + 0x0c), t1, lastchunk_fault) \ |
| 98 | st %t2, [%dst - (offset) - 0x08]; \ | 106 | ST(st, dst, -(offset + 0x08), t2, lastchunk_fault) \ |
| 99 | st %t3, [%dst - (offset) - 0x04]; | 107 | ST(st, dst, -(offset + 0x04), t3, lastchunk_fault) |
| 100 | 108 | ||
| 101 | #define MOVE_HALFCHUNK(src, dst, offset, t0, t1, t2, t3) \ | 109 | .section .fixup,#alloc,#execinstr |
| 102 | lduh [%src + (offset) + 0x00], %t0; \ | 110 | lastchunk_fault: |
| 103 | lduh [%src + (offset) + 0x02], %t1; \ | 111 | and %g1, 15, %g1 |
| 104 | lduh [%src + (offset) + 0x04], %t2; \ | 112 | retl |
| 105 | lduh [%src + (offset) + 0x06], %t3; \ | 113 | sub %g1, %g5, %o0 |
| 106 | sth %t0, [%dst + (offset) + 0x00]; \ | ||
| 107 | sth %t1, [%dst + (offset) + 0x02]; \ | ||
| 108 | sth %t2, [%dst + (offset) + 0x04]; \ | ||
| 109 | sth %t3, [%dst + (offset) + 0x06]; | ||
| 110 | 114 | ||
| 115 | /* left: o3 + (o2 % 16) - offset */ | ||
| 116 | #define MOVE_HALFCHUNK(src, dst, offset, t0, t1, t2, t3) \ | ||
| 117 | LD(lduh, src, offset + 0x00, t0, halfchunk_fault) \ | ||
| 118 | LD(lduh, src, offset + 0x02, t1, halfchunk_fault) \ | ||
| 119 | LD(lduh, src, offset + 0x04, t2, halfchunk_fault) \ | ||
| 120 | LD(lduh, src, offset + 0x06, t3, halfchunk_fault) \ | ||
| 121 | ST(sth, dst, offset + 0x00, t0, halfchunk_fault) \ | ||
| 122 | ST(sth, dst, offset + 0x02, t1, halfchunk_fault) \ | ||
| 123 | ST(sth, dst, offset + 0x04, t2, halfchunk_fault) \ | ||
| 124 | ST(sth, dst, offset + 0x06, t3, halfchunk_fault) | ||
| 125 | |||
| 126 | /* left: o3 + (o2 % 16) + offset + 2 */ | ||
| 111 | #define MOVE_SHORTCHUNK(src, dst, offset, t0, t1) \ | 127 | #define MOVE_SHORTCHUNK(src, dst, offset, t0, t1) \ |
| 112 | ldub [%src - (offset) - 0x02], %t0; \ | 128 | LD(ldub, src, -(offset + 0x02), t0, halfchunk_fault) \ |
| 113 | ldub [%src - (offset) - 0x01], %t1; \ | 129 | LD(ldub, src, -(offset + 0x01), t1, halfchunk_fault) \ |
| 114 | stb %t0, [%dst - (offset) - 0x02]; \ | 130 | ST(stb, dst, -(offset + 0x02), t0, halfchunk_fault) \ |
| 115 | stb %t1, [%dst - (offset) - 0x01]; | 131 | ST(stb, dst, -(offset + 0x01), t1, halfchunk_fault) |
| 132 | |||
| 133 | .section .fixup,#alloc,#execinstr | ||
| 134 | halfchunk_fault: | ||
| 135 | and %o2, 15, %o2 | ||
| 136 | sub %o3, %g5, %o3 | ||
| 137 | retl | ||
| 138 | add %o2, %o3, %o0 | ||
| 139 | |||
| 140 | /* left: offset + 2 + (o2 % 2) */ | ||
| 141 | #define MOVE_LAST_SHORTCHUNK(src, dst, offset, t0, t1) \ | ||
| 142 | LD(ldub, src, -(offset + 0x02), t0, last_shortchunk_fault) \ | ||
| 143 | LD(ldub, src, -(offset + 0x01), t1, last_shortchunk_fault) \ | ||
| 144 | ST(stb, dst, -(offset + 0x02), t0, last_shortchunk_fault) \ | ||
| 145 | ST(stb, dst, -(offset + 0x01), t1, last_shortchunk_fault) | ||
| 146 | |||
| 147 | .section .fixup,#alloc,#execinstr | ||
| 148 | last_shortchunk_fault: | ||
| 149 | and %o2, 1, %o2 | ||
| 150 | retl | ||
| 151 | sub %o2, %g5, %o0 | ||
| 116 | 152 | ||
| 117 | .text | 153 | .text |
| 118 | .align 4 | 154 | .align 4 |
| @@ -182,8 +218,6 @@ __copy_user: /* %o0=dst %o1=src %o2=len */ · 63 unmodified lines | |||
| 182 | MOVE_BIGCHUNK(o1, o0, 0x20, o2, o3, o4, o5, g2, g3, g4, g5) | 218 | MOVE_BIGCHUNK(o1, o0, 0x20, o2, o3, o4, o5, g2, g3, g4, g5) |
| 183 | MOVE_BIGCHUNK(o1, o0, 0x40, o2, o3, o4, o5, g2, g3, g4, g5) | 219 | MOVE_BIGCHUNK(o1, o0, 0x40, o2, o3, o4, o5, g2, g3, g4, g5) |
| 184 | MOVE_BIGCHUNK(o1, o0, 0x60, o2, o3, o4, o5, g2, g3, g4, g5) | 220 | MOVE_BIGCHUNK(o1, o0, 0x60, o2, o3, o4, o5, g2, g3, g4, g5) |
| 185 | 80: | ||
| 186 | EXT(5b, 80b, 50f) | ||
| 187 | subcc %g7, 128, %g7 | 221 | subcc %g7, 128, %g7 |
| 188 | add %o1, 128, %o1 | 222 | add %o1, 128, %o1 |
| 189 | bne 5b | 223 | bne 5b |
| @@ -201,7 +235,6 @@ __copy_user: /* %o0=dst %o1=src %o2=len */ · 11 unmodified lines | |||
| 201 | jmpl %o5 + %lo(copy_user_table_end), %g0 | 235 | jmpl %o5 + %lo(copy_user_table_end), %g0 |
| 202 | add %o0, %g7, %o0 | 236 | add %o0, %g7, %o0 |
| 203 | 237 | ||
| 204 | copy_user_table: | ||
| 205 | MOVE_LASTCHUNK(o1, o0, 0x60, g2, g3, g4, g5) | 238 | MOVE_LASTCHUNK(o1, o0, 0x60, g2, g3, g4, g5) |
| 206 | MOVE_LASTCHUNK(o1, o0, 0x50, g2, g3, g4, g5) | 239 | MOVE_LASTCHUNK(o1, o0, 0x50, g2, g3, g4, g5) |
| 207 | MOVE_LASTCHUNK(o1, o0, 0x40, g2, g3, g4, g5) | 240 | MOVE_LASTCHUNK(o1, o0, 0x40, g2, g3, g4, g5) |
| @@ -210,7 +243,6 @@ copy_user_table: · 2 unmodified lines | |||
| 210 | MOVE_LASTCHUNK(o1, o0, 0x10, g2, g3, g4, g5) | 243 | MOVE_LASTCHUNK(o1, o0, 0x10, g2, g3, g4, g5) |
| 211 | MOVE_LASTCHUNK(o1, o0, 0x00, g2, g3, g4, g5) | 244 | MOVE_LASTCHUNK(o1, o0, 0x00, g2, g3, g4, g5) |
| 212 | copy_user_table_end: | 245 | copy_user_table_end: |
| 213 | EXT(copy_user_table, copy_user_table_end, 51f) | ||
| 214 | be copy_user_last7 | 246 | be copy_user_last7 |
| 215 | andcc %g1, 4, %g0 | 247 | andcc %g1, 4, %g0 |
| 216 | 248 | ||
| @@ -250,8 +282,6 @@ ldd_std: · 33 unmodified lines | |||
| 250 | MOVE_BIGALIGNCHUNK(o1, o0, 0x20, o2, o3, o4, o5, g2, g3, g4, g5) | 282 | MOVE_BIGALIGNCHUNK(o1, o0, 0x20, o2, o3, o4, o5, g2, g3, g4, g5) |
| 251 | MOVE_BIGALIGNCHUNK(o1, o0, 0x40, o2, o3, o4, o5, g2, g3, g4, g5) | 283 | MOVE_BIGALIGNCHUNK(o1, o0, 0x40, o2, o3, o4, o5, g2, g3, g4, g5) |
| 252 | MOVE_BIGALIGNCHUNK(o1, o0, 0x60, o2, o3, o4, o5, g2, g3, g4, g5) | 284 | MOVE_BIGALIGNCHUNK(o1, o0, 0x60, o2, o3, o4, o5, g2, g3, g4, g5) |
| 253 | 81: | ||
| 254 | EXT(ldd_std, 81b, 52f) | ||
| 255 | subcc %g7, 128, %g7 | 285 | subcc %g7, 128, %g7 |
| 256 | add %o1, 128, %o1 | 286 | add %o1, 128, %o1 |
| 257 | bne ldd_std | 287 | bne ldd_std |
| @@ -290,8 +320,6 @@ cannot_optimize: · 32 unmodified lines | |||
| 290 | 10: | 320 | 10: |
| 291 | MOVE_HALFCHUNK(o1, o0, 0x00, g2, g3, g4, g5) | 321 | MOVE_HALFCHUNK(o1, o0, 0x00, g2, g3, g4, g5) |
| 292 | MOVE_HALFCHUNK(o1, o0, 0x08, g2, g3, g4, g5) | 322 | MOVE_HALFCHUNK(o1, o0, 0x08, g2, g3, g4, g5) |
| 293 | 82: | ||
| 294 | EXT(10b, 82b, 53f) | ||
| 295 | subcc %o3, 0x10, %o3 | 323 | subcc %o3, 0x10, %o3 |
| 296 | add %o1, 0x10, %o1 | 324 | add %o1, 0x10, %o1 |
| 297 | bne 10b | 325 | bne 10b |
| @@ -308,8 +336,6 @@ byte_chunk: · 10 unmodified lines | |||
| 308 | MOVE_SHORTCHUNK(o1, o0, -0x0c, g2, g3) | 336 | MOVE_SHORTCHUNK(o1, o0, -0x0c, g2, g3) |
| 309 | MOVE_SHORTCHUNK(o1, o0, -0x0e, g2, g3) | 337 | MOVE_SHORTCHUNK(o1, o0, -0x0e, g2, g3) |
| 310 | MOVE_SHORTCHUNK(o1, o0, -0x10, g2, g3) | 338 | MOVE_SHORTCHUNK(o1, o0, -0x10, g2, g3) |
| 311 | 83: | ||
| 312 | EXT(byte_chunk, 83b, 54f) | ||
| 313 | subcc %o3, 0x10, %o3 | 339 | subcc %o3, 0x10, %o3 |
| 314 | add %o1, 0x10, %o1 | 340 | add %o1, 0x10, %o1 |
| 315 | bne byte_chunk | 341 | bne byte_chunk |
| @@ -325,16 +351,14 @@ short_end: · 9 unmodified lines | |||
| 325 | add %o1, %o3, %o1 | 351 | add %o1, %o3, %o1 |
| 326 | jmpl %o5 + %lo(short_table_end), %g0 | 352 | jmpl %o5 + %lo(short_table_end), %g0 |
| 327 | andcc %o2, 1, %g0 | 353 | andcc %o2, 1, %g0 |
| 328 | 84: | 354 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x0c, g2, g3) |
| 329 | MOVE_SHORTCHUNK(o1, o0, 0x0c, g2, g3) | 355 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x0a, g2, g3) |
| 330 | MOVE_SHORTCHUNK(o1, o0, 0x0a, g2, g3) | 356 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x08, g2, g3) |
| 331 | MOVE_SHORTCHUNK(o1, o0, 0x08, g2, g3) | 357 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x06, g2, g3) |
| 332 | MOVE_SHORTCHUNK(o1, o0, 0x06, g2, g3) | 358 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x04, g2, g3) |
| 333 | MOVE_SHORTCHUNK(o1, o0, 0x04, g2, g3) | 359 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x02, g2, g3) |
| 334 | MOVE_SHORTCHUNK(o1, o0, 0x02, g2, g3) | 360 | MOVE_LAST_SHORTCHUNK(o1, o0, 0x00, g2, g3) |
| 335 | MOVE_SHORTCHUNK(o1, o0, 0x00, g2, g3) | ||
| 336 | short_table_end: | 361 | short_table_end: |
| 337 | EXT(84b, short_table_end, 55f) | ||
| 338 | be 1f | 362 | be 1f |
| 339 | nop | 363 | nop |
| 340 | EX(ldub [%o1], %g2, add %g0, 1) | 364 | EX(ldub [%o1], %g2, add %g0, 1) |
| @@ -363,123 +387,8 @@ short_aligned_end: · 22 unmodified lines | |||
| 363 | .section .fixup,#alloc,#execinstr | 387 | .section .fixup,#alloc,#execinstr |
| 364 | .align 4 | 388 | .align 4 |
| 365 | 97: | 389 | 97: |
| 366 | mov %o2, %g3 | ||
| 367 | fixupretl: | ||
| 368 | retl | 390 | retl |
| 369 | mov %g3, %o0 | 391 | mov %o2, %o0 |
| 370 | |||
| 371 | /* exception routine sets %g2 to (broken_insn - first_insn)>>2 */ | ||
| 372 | 50: | ||
| 373 | /* This magic counts how many bytes are left when crash in MOVE_BIGCHUNK | ||
| 374 | * happens. This is derived from the amount ldd reads, st stores, etc. | ||
| 375 | * x = g2 % 12; | ||
| 376 | * g3 = g1 + g7 - ((g2 / 12) * 32 + (x < 4) ? 0 : (x - 4) * 4); | ||
| 377 | * o0 += (g2 / 12) * 32; | ||
| 378 | */ | ||
| 379 | cmp %g2, 12 | ||
| 380 | add %o0, %g7, %o0 | ||
| 381 | bcs 1f | ||
| 382 | cmp %g2, 24 | ||
| 383 | bcs 2f | ||
| 384 | cmp %g2, 36 | ||
| 385 | bcs 3f | ||
| 386 | nop | ||
| 387 | sub %g2, 12, %g2 | ||
| 388 | sub %g7, 32, %g7 | ||
| 389 | 3: sub %g2, 12, %g2 | ||
| 390 | sub %g7, 32, %g7 | ||
| 391 | 2: sub %g2, 12, %g2 | ||
| 392 | sub %g7, 32, %g7 | ||
| 393 | 1: cmp %g2, 4 | ||
| 394 | bcs,a 60f | ||
| 395 | clr %g2 | ||
| 396 | sub %g2, 4, %g2 | ||
| 397 | sll %g2, 2, %g2 | ||
| 398 | 60: and %g1, 0x7f, %g3 | ||
| 399 | sub %o0, %g7, %o0 | ||
| 400 | add %g3, %g7, %g3 | ||
| 401 | ba fixupretl | ||
| 402 | sub %g3, %g2, %g3 | ||
| 403 | 51: | ||
| 404 | /* i = 41 - g2; j = i % 6; | ||
| 405 | * g3 = (g1 & 15) + (i / 6) * 16 + (j < 4) ? (j + 1) * 4 : 16; | ||
| 406 | * o0 -= (i / 6) * 16 + 16; | ||
| 407 | */ | ||
| 408 | neg %g2 | ||
| 409 | and %g1, 0xf, %g1 | ||
| 410 | add %g2, 41, %g2 | ||
| 411 | add %o0, %g1, %o0 | ||
| 412 | 1: cmp %g2, 6 | ||
| 413 | bcs,a 2f | ||
| 414 | cmp %g2, 4 | ||
| 415 | add %g1, 16, %g1 | ||
| 416 | b 1b | ||
| 417 | sub %g2, 6, %g2 | ||
| 418 | 2: bcc,a 2f | ||
| 419 | mov 16, %g2 | ||
| 420 | inc %g2 | ||
| 421 | sll %g2, 2, %g2 | ||
| 422 | 2: add %g1, %g2, %g3 | ||
| 423 | ba fixupretl | ||
| 424 | sub %o0, %g3, %o0 | ||
| 425 | 52: | ||
| 426 | /* g3 = g1 + g7 - (g2 / 8) * 32 + (g2 & 4) ? (g2 & 3) * 8 : 0; | ||
| 427 | o0 += (g2 / 8) * 32 */ | ||
| 428 | andn %g2, 7, %g4 | ||
| 429 | add %o0, %g7, %o0 | ||
| 430 | andcc %g2, 4, %g0 | ||
| 431 | and %g2, 3, %g2 | ||
| 432 | sll %g4, 2, %g4 | ||
| 433 | sll %g2, 3, %g2 | ||
| 434 | bne 60b | ||
| 435 | sub %g7, %g4, %g7 | ||
| 436 | ba 60b | ||
| 437 | clr %g2 | ||
| 438 | 53: | ||
| 439 | /* g3 = o3 + (o2 & 15) - (g2 & 8) - (g2 & 4) ? (g2 & 3) * 2 : 0; | ||
| 440 | o0 += (g2 & 8) */ | ||
| 441 | and %g2, 3, %g4 | ||
| 442 | andcc %g2, 4, %g0 | ||
| 443 | and %g2, 8, %g2 | ||
| 444 | sll %g4, 1, %g4 | ||
| 445 | be 1f | ||
| 446 | add %o0, %g2, %o0 | ||
| 447 | add %g2, %g4, %g2 | ||
| 448 | 1: and %o2, 0xf, %g3 | ||
| 449 | add %g3, %o3, %g3 | ||
| 450 | ba fixupretl | ||
| 451 | sub %g3, %g2, %g3 | ||
| 452 | 54: | ||
| 453 | /* g3 = o3 + (o2 & 15) - (g2 / 4) * 2 - (g2 & 2) ? (g2 & 1) : 0; | ||
| 454 | o0 += (g2 / 4) * 2 */ | ||
| 455 | srl %g2, 2, %o4 | ||
| 456 | and %g2, 1, %o5 | ||
| 457 | srl %g2, 1, %g2 | ||
| 458 | add %o4, %o4, %o4 | ||
| 459 | and %o5, %g2, %o5 | ||
| 460 | and %o2, 0xf, %o2 | ||
| 461 | add %o0, %o4, %o0 | ||
| 462 | sub %o3, %o5, %o3 | ||
| 463 | sub %o2, %o4, %o2 | ||
| 464 | ba fixupretl | ||
| 465 | add %o2, %o3, %g3 | ||
| 466 | 55: | ||
| 467 | /* i = 27 - g2; | ||
| 468 | g3 = (o2 & 1) + i / 4 * 2 + !(i & 3); | ||
| 469 | o0 -= i / 4 * 2 + 1 */ | ||
| 470 | neg %g2 | ||
| 471 | and %o2, 1, %o2 | ||
| 472 | add %g2, 27, %g2 | ||
| 473 | srl %g2, 2, %o5 | ||
| 474 | andcc %g2, 3, %g0 | ||
| 475 | mov 1, %g2 | ||
| 476 | add %o5, %o5, %o5 | ||
| 477 | be,a 1f | ||
| 478 | clr %g2 | ||
| 479 | 1: add %g2, %o5, %g3 | ||
| 480 | sub %o0, %g3, %o0 | ||
| 481 | ba fixupretl | ||
| 482 | add %g3, %o2, %g3 | ||
| 483 | 392 | ||
| 484 | .globl __copy_user_end | 393 | .globl __copy_user_end |
| 485 | __copy_user_end: | 394 | __copy_user_end: |
| @@ -19,7 +19,7 @@ · 18 unmodified lines | |||
| 19 | 98: x,y; \ | 19 | 98: x,y; \ |
| 20 | .section .fixup,ALLOC,EXECINSTR; \ | 20 | .section .fixup,ALLOC,EXECINSTR; \ |
| 21 | .align 4; \ | 21 | .align 4; \ |
| 22 | 99: ba 30f; \ | 22 | 99: retl; \ |
| 23 | a, b, %o0; \ | 23 | a, b, %o0; \ |
| 24 | .section __ex_table,ALLOC; \ | 24 | .section __ex_table,ALLOC; \ |
| 25 | .align 4; \ | 25 | .align 4; \ |
| @@ -27,35 +27,44 @@ · 1 unmodified line | |||
| 27 | .text; \ | 27 | .text; \ |
| 28 | .align 4 | 28 | .align 4 |
| 29 | 29 | ||
| 30 | #define EXT(start,end,handler) \ | 30 | #define STORE(source, base, offset, n) \ |
| 31 | 98: std source, [base + offset + n]; \ | ||
| 32 | .section .fixup,ALLOC,EXECINSTR; \ | ||
| 33 | .align 4; \ | ||
| 34 | 99: ba 30f; \ | ||
| 35 | sub %o3, n - offset, %o3; \ | ||
| 31 | .section __ex_table,ALLOC; \ | 36 | .section __ex_table,ALLOC; \ |
| 32 | .align 4; \ | 37 | .align 4; \ |
| 33 | .word start, 0, end, handler; \ | 38 | .word 98b, 99b; \ |
| 34 | .text; \ | 39 | .text; \ |
| 35 | .align 4 | 40 | .align 4; |
| 41 | |||
| 42 | #define STORE_LAST(source, base, offset, n) \ | ||
| 43 | EX(std source, [base - offset - n], \ | ||
| 44 | add %o1, offset + n); | ||
| 36 | 45 | ||
| 37 | /* Please don't change these macros, unless you change the logic | 46 | /* Please don't change these macros, unless you change the logic |
| 38 | * in the .fixup section below as well. | 47 | * in the .fixup section below as well. |
| 39 | * Store 64 bytes at (BASE + OFFSET) using value SOURCE. */ | 48 | * Store 64 bytes at (BASE + OFFSET) using value SOURCE. */ |
| 40 | #define ZERO_BIG_BLOCK(base, offset, source) \ | 49 | #define ZERO_BIG_BLOCK(base, offset, source) \ |
| 41 | std source, [base + offset + 0x00]; \ | 50 | STORE(source, base, offset, 0x00); \ |
| 42 | std source, [base + offset + 0x08]; \ | 51 | STORE(source, base, offset, 0x08); \ |
| 43 | std source, [base + offset + 0x10]; \ | 52 | STORE(source, base, offset, 0x10); \ |
| 44 | std source, [base + offset + 0x18]; \ | 53 | STORE(source, base, offset, 0x18); \ |
| 45 | std source, [base + offset + 0x20]; \ | 54 | STORE(source, base, offset, 0x20); \ |
| 46 | std source, [base + offset + 0x28]; \ | 55 | STORE(source, base, offset, 0x28); \ |
| 47 | std source, [base + offset + 0x30]; \ | 56 | STORE(source, base, offset, 0x30); \ |
| 48 | std source, [base + offset + 0x38]; | 57 | STORE(source, base, offset, 0x38); |
| 49 | 58 | ||
| 50 | #define ZERO_LAST_BLOCKS(base, offset, source) \ | 59 | #define ZERO_LAST_BLOCKS(base, offset, source) \ |
| 51 | std source, [base - offset - 0x38]; \ | 60 | STORE_LAST(source, base, offset, 0x38); \ |
| 52 | std source, [base - offset - 0x30]; \ | 61 | STORE_LAST(source, base, offset, 0x30); \ |
| 53 | std source, [base - offset - 0x28]; \ | 62 | STORE_LAST(source, base, offset, 0x28); \ |
| 54 | std source, [base - offset - 0x20]; \ | 63 | STORE_LAST(source, base, offset, 0x20); \ |
| 55 | std source, [base - offset - 0x18]; \ | 64 | STORE_LAST(source, base, offset, 0x18); \ |
| 56 | std source, [base - offset - 0x10]; \ | 65 | STORE_LAST(source, base, offset, 0x10); \ |
| 57 | std source, [base - offset - 0x08]; \ | 66 | STORE_LAST(source, base, offset, 0x08); \ |
| 58 | std source, [base - offset - 0x00]; | 67 | STORE_LAST(source, base, offset, 0x00); |
| 59 | 68 | ||
| 60 | .text | 69 | .text |
| 61 | .align 4 | 70 | .align 4 |
| @@ -68,8 +77,6 @@ __bzero_begin: · 6 unmodified lines | |||
| 68 | .globl memset | 77 | .globl memset |
| 69 | EXPORT_SYMBOL(__bzero) | 78 | EXPORT_SYMBOL(__bzero) |
| 70 | EXPORT_SYMBOL(memset) | 79 | EXPORT_SYMBOL(memset) |
| 71 | .globl __memset_start, __memset_end | ||
| 72 | __memset_start: | ||
| 73 | memset: | 80 | memset: |
| 74 | mov %o0, %g1 | 81 | mov %o0, %g1 |
| 75 | mov 1, %g4 | 82 | mov 1, %g4 |
| @@ -122,8 +129,6 @@ __bzero: · 46 unmodified lines | |||
| 122 | ZERO_BIG_BLOCK(%o0, 0x00, %g2) | 129 | ZERO_BIG_BLOCK(%o0, 0x00, %g2) |
| 123 | subcc %o3, 128, %o3 | 130 | subcc %o3, 128, %o3 |
| 124 | ZERO_BIG_BLOCK(%o0, 0x40, %g2) | 131 | ZERO_BIG_BLOCK(%o0, 0x40, %g2) |
| 125 | 11: | ||
| 126 | EXT(10b, 11b, 20f) | ||
| 127 | bne 10b | 132 | bne 10b |
| 128 | add %o0, 128, %o0 | 133 | add %o0, 128, %o0 |
| 129 | 134 | ||
| @@ -138,11 +143,9 @@ __bzero: · 8 unmodified lines | |||
| 138 | jmp %o4 | 143 | jmp %o4 |
| 139 | add %o0, %o2, %o0 | 144 | add %o0, %o2, %o0 |
| 140 | 145 | ||
| 141 | 12: | ||
| 142 | ZERO_LAST_BLOCKS(%o0, 0x48, %g2) | 146 | ZERO_LAST_BLOCKS(%o0, 0x48, %g2) |
| 143 | ZERO_LAST_BLOCKS(%o0, 0x08, %g2) | 147 | ZERO_LAST_BLOCKS(%o0, 0x08, %g2) |
| 144 | 13: | 148 | 13: |
| 145 | EXT(12b, 13b, 21f) | ||
| 146 | be 8f | 149 | be 8f |
| 147 | andcc %o1, 4, %g0 | 150 | andcc %o1, 4, %g0 |
| 148 | 151 | ||
| @@ -182,37 +185,13 @@ __bzero: · 33 unmodified lines | |||
| 182 | 5: | 185 | 5: |
| 183 | retl | 186 | retl |
| 184 | clr %o0 | 187 | clr %o0 |
| 185 | __memset_end: | ||
| 186 | 188 | ||
| 187 | .section .fixup,#alloc,#execinstr | 189 | .section .fixup,#alloc,#execinstr |
| 188 | .align 4 | 190 | .align 4 |
| 189 | 20: | ||
| 190 | cmp %g2, 8 | ||
| 191 | bleu 1f | ||
| 192 | and %o1, 0x7f, %o1 | ||
| 193 | sub %g2, 9, %g2 | ||
| 194 | add %o3, 64, %o3 | ||
| 195 | 1: | ||
| 196 | sll %g2, 3, %g2 | ||
| 197 | add %o3, %o1, %o0 | ||
| 198 | b 30f | ||
| 199 | sub %o0, %g2, %o0 | ||
| 200 | 21: | ||
| 201 | mov 8, %o0 | ||
| 202 | and %o1, 7, %o1 | ||
| 203 | sub %o0, %g2, %o0 | ||
| 204 | sll %o0, 3, %o0 | ||
| 205 | b 30f | ||
| 206 | add %o0, %o1, %o0 | ||
| 207 | 30: | 191 | 30: |
| 208 | /* %o4 is faulting address, %o5 is %pc where fault occurred */ | 192 | and %o1, 0x7f, %o1 |
| 209 | save %sp, -104, %sp | 193 | retl |
| 210 | mov %i5, %o0 | 194 | add %o3, %o1, %o0 |
| 211 | mov %i7, %o1 | ||
| 212 | call lookup_fault | ||
| 213 | mov %i4, %o2 | ||
| 214 | ret | ||
| 215 | restore | ||
| 216 | 195 | ||
| 217 | .globl __bzero_end | 196 | .globl __bzero_end |
| 218 | __bzero_end: | 197 | __bzero_end: |
| @@ -8,7 +8,7 @@ ccflags-y := -Werror · 7 unmodified lines | |||
| 8 | obj-$(CONFIG_SPARC64) += ultra.o tlb.o tsb.o | 8 | obj-$(CONFIG_SPARC64) += ultra.o tlb.o tsb.o |
| 9 | obj-y += fault_$(BITS).o | 9 | obj-y += fault_$(BITS).o |
| 10 | obj-y += init_$(BITS).o | 10 | obj-y += init_$(BITS).o |
| 11 | obj-$(CONFIG_SPARC32) += extable.o srmmu.o iommu.o io-unit.o | 11 | obj-$(CONFIG_SPARC32) += srmmu.o iommu.o io-unit.o |
| 12 | obj-$(CONFIG_SPARC32) += srmmu_access.o | 12 | obj-$(CONFIG_SPARC32) += srmmu_access.o |
| 13 | obj-$(CONFIG_SPARC32) += hypersparc.o viking.o tsunami.o swift.o | 13 | obj-$(CONFIG_SPARC32) += hypersparc.o viking.o tsunami.o swift.o |
| 14 | obj-$(CONFIG_SPARC32) += leon_mm.o | 14 | obj-$(CONFIG_SPARC32) += leon_mm.o |
| @@ -1,107 +0,0 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | ||
| 2 | /* | ||
| 3 | * linux/arch/sparc/mm/extable.c | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <linux/module.h> | ||
| 7 | #include <linux/extable.h> | ||
| 8 | #include <linux/uaccess.h> | ||
| 9 | |||
| 10 | void sort_extable(struct exception_table_entry *start, | ||
| 11 | struct exception_table_entry *finish) | ||
| 12 | { | ||
| 13 | } | ||
| 14 | |||
| 15 | /* Caller knows they are in a range if ret->fixup == 0 */ | ||
| 16 | const struct exception_table_entry * | ||
| 17 | search_extable(const struct exception_table_entry *base, | ||
| 18 | const size_t num, | ||
| 19 | unsigned long value) | ||
| 20 | { | ||
| 21 | int i; | ||
| 22 | |||
| 23 | /* Single insn entries are encoded as: | ||
| 24 | * word 1: insn address | ||
| 25 | * word 2: fixup code address | ||
| 26 | * | ||
| 27 | * Range entries are encoded as: | ||
| 28 | * word 1: first insn address | ||
| 29 | * word 2: 0 | ||
| 30 | * word 3: last insn address + 4 bytes | ||
| 31 | * word 4: fixup code address | ||
| 32 | * | ||
| 33 | * Deleted entries are encoded as: | ||
| 34 | * word 1: unused | ||
| 35 | * word 2: -1 | ||
| 36 | * | ||
| 37 | * See asm/uaccess.h for more details. | ||
| 38 | */ | ||
| 39 | |||
| 40 | /* 1. Try to find an exact match. */ | ||
| 41 | for (i = 0; i < num; i++) { | ||
| 42 | if (base[i].fixup == 0) { | ||
| 43 | /* A range entry, skip both parts. */ | ||
| 44 | i++; | ||
| 45 | continue; | ||
| 46 | } | ||
| 47 | |||
| 48 | /* A deleted entry; see trim_init_extable */ | ||
| 49 | if (base[i].fixup == -1) | ||
| 50 | continue; | ||
| 51 | |||
| 52 | if (base[i].insn == value) | ||
| 53 | return &base[i]; | ||
| 54 | } | ||
| 55 | |||
| 56 | /* 2. Try to find a range match. */ | ||
| 57 | for (i = 0; i < (num - 1); i++) { | ||
| 58 | if (base[i].fixup) | ||
| 59 | continue; | ||
| 60 | |||
| 61 | if (base[i].insn <= value && base[i + 1].insn > value) | ||
| 62 | return &base[i]; | ||
| 63 | |||
| 64 | i++; | ||
| 65 | } | ||
| 66 | |||
| 67 | return NULL; | ||
| 68 | } | ||
| 69 | |||
| 70 | #ifdef CONFIG_MODULES | ||
| 71 | /* We could memmove them around; easier to mark the trimmed ones. */ | ||
| 72 | void trim_init_extable(struct module *m) | ||
| 73 | { | ||
| 74 | unsigned int i; | ||
| 75 | bool range; | ||
| 76 | |||
| 77 | for (i = 0; i < m->num_exentries; i += range ? 2 : 1) { | ||
| 78 | range = m->extable[i].fixup == 0; | ||
| 79 | |||
| 80 | if (within_module_init(m->extable[i].insn, m)) { | ||
| 81 | m->extable[i].fixup = -1; | ||
| 82 | if (range) | ||
| 83 | m->extable[i+1].fixup = -1; | ||
| 84 | } | ||
| 85 | if (range) | ||
| 86 | i++; | ||
| 87 | } | ||
| 88 | } | ||
| 89 | #endif /* CONFIG_MODULES */ | ||
| 90 | |||
| 91 | /* Special extable search, which handles ranges. Returns fixup */ | ||
| 92 | unsigned long search_extables_range(unsigned long addr, unsigned long *g2) | ||
| 93 | { | ||
| 94 | const struct exception_table_entry *entry; | ||
| 95 | |||
| 96 | entry = search_exception_tables(addr); | ||
| 97 | if (!entry) | ||
| 98 | return 0; | ||
| 99 | |||
| 100 | /* Inside range? Fix g2 and return correct fixup */ | ||
| 101 | if (!entry->fixup) { | ||
| 102 | *g2 = (addr - entry->insn) / 4; | ||
| 103 | return (entry + 1)->fixup; | ||
| 104 | } | ||
| 105 | |||
| 106 | return entry->fixup; | ||
| 107 | } | ||
| @@ -23,6 +23,7 @@ · 22 unmodified lines | |||
| 23 | #include <linux/interrupt.h> | 23 | #include <linux/interrupt.h> |
| 24 | #include <linux/kdebug.h> | 24 | #include <linux/kdebug.h> |
| 25 | #include <linux/uaccess.h> | 25 | #include <linux/uaccess.h> |
| 26 | #include <linux/extable.h> | ||
| 26 | 27 | ||
| 27 | #include <asm/page.h> | 28 | #include <asm/page.h> |
| 28 | #include <asm/openprom.h> | 29 | #include <asm/openprom.h> |
| @@ -54,54 +55,6 @@ static void __noreturn unhandled_fault(unsigned long address, · 25 unmodified lines | |||
| 54 | die_if_kernel("Oops", regs); | 55 | die_if_kernel("Oops", regs); |
| 55 | } | 56 | } |
| 56 | 57 | ||
| 57 | asmlinkage int lookup_fault(unsigned long pc, unsigned long ret_pc, | ||
| 58 | unsigned long address) | ||
| 59 | { | ||
| 60 | struct pt_regs regs; | ||
| 61 | unsigned long g2; | ||
| 62 | unsigned int insn; | ||
| 63 | int i; | ||
| 64 | |||
| 65 | i = search_extables_range(ret_pc, &g2); | ||
| 66 | switch (i) { | ||
| 67 | case 3: | ||
| 68 | /* load & store will be handled by fixup */ | ||
| 69 | return 3; | ||
| 70 | |||
| 71 | case 1: | ||
| 72 | /* store will be handled by fixup, load will bump out */ | ||
| 73 | /* for _to_ macros */ | ||
| 74 | insn = *((unsigned int *) pc); | ||
| 75 | if ((insn >> 21) & 1) | ||
| 76 | return 1; | ||
| 77 | break; | ||
| 78 | |||
| 79 | case 2: | ||
| 80 | /* load will be handled by fixup, store will bump out */ | ||
| 81 | /* for _from_ macros */ | ||
| 82 | insn = *((unsigned int *) pc); | ||
| 83 | if (!((insn >> 21) & 1) || ((insn>>19)&0x3f) == 15) | ||
| 84 | return 2; | ||
| 85 | break; | ||
| 86 | |||
| 87 | default: | ||
| 88 | break; | ||
| 89 | } | ||
| 90 | |||
| 91 | memset(®s, 0, sizeof(regs)); | ||
| 92 | regs.pc = pc; | ||
| 93 | regs.npc = pc + 4; | ||
| 94 | __asm__ __volatile__( | ||
| 95 | "rd %%psr, %0\n\t" | ||
| 96 | "nop\n\t" | ||
| 97 | "nop\n\t" | ||
| 98 | "nop\n" : "=r" (regs.psr)); | ||
| 99 | unhandled_fault(address, current, ®s); | ||
| 100 | |||
| 101 | /* Not reached */ | ||
| 102 | return 0; | ||
| 103 | } | ||
| 104 | |||
| 105 | static inline void | 58 | static inline void |
| 106 | show_signal_msg(struct pt_regs *regs, int sig, int code, | 59 | show_signal_msg(struct pt_regs *regs, int sig, int code, |
| 107 | unsigned long address, struct task_struct *tsk) | 60 | unsigned long address, struct task_struct *tsk) |
| @@ -162,8 +115,6 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, · 54 unmodified lines | |||
| 162 | struct vm_area_struct *vma; | 115 | struct vm_area_struct *vma; |
| 163 | struct task_struct *tsk = current; | 116 | struct task_struct *tsk = current; |
| 164 | struct mm_struct *mm = tsk->mm; | 117 | struct mm_struct *mm = tsk->mm; |
| 165 | unsigned int fixup; | ||
| 166 | unsigned long g2; | ||
| 167 | int from_user = !(regs->psr & PSR_PS); | 118 | int from_user = !(regs->psr & PSR_PS); |
| 168 | int code; | 119 | int code; |
| 169 | vm_fault_t fault; | 120 | vm_fault_t fault; |
| @@ -281,30 +232,19 @@ bad_area_nosemaphore: · 111 unmodified lines | |||
| 281 | 232 | ||
| 282 | /* Is this in ex_table? */ | 233 | /* Is this in ex_table? */ |
| 283 | no_context: | 234 | no_context: |
| 284 | g2 = regs->u_regs[UREG_G2]; | ||
| 285 | if (!from_user) { | 235 | if (!from_user) { |
| 286 | fixup = search_extables_range(regs->pc, &g2); | 236 | const struct exception_table_entry *entry; |
| 287 | /* Values below 10 are reserved for other things */ | ||
| 288 | if (fixup > 10) { | ||
| 289 | extern const unsigned int __memset_start[]; | ||
| 290 | extern const unsigned int __memset_end[]; | ||
| 291 | 237 | ||
| 238 | entry = search_exception_tables(regs->pc); | ||
| 292 | #ifdef DEBUG_EXCEPTIONS | 239 | #ifdef DEBUG_EXCEPTIONS |
| 293 | printk("Exception: PC<%08lx> faddr<%08lx>\n", | 240 | printk("Exception: PC<%08lx> faddr<%08lx>\n", |
| 294 | regs->pc, address); | 241 | regs->pc, address); |
| 295 | printk("EX_TABLE: insn<%08lx> fixup<%08x> g2<%08lx>\n", | 242 | printk("EX_TABLE: insn<%08lx> fixup<%08x>\n", |
| 296 | regs->pc, fixup, g2); | 243 | regs->pc, entry->fixup); |
| 297 | #endif | 244 | #endif |
| 298 | if ((regs->pc >= (unsigned long)__memset_start && | 245 | regs->pc = entry->fixup; |
| 299 | regs->pc < (unsigned long)__memset_end)) { | 246 | regs->npc = regs->pc + 4; |
| 300 | regs->u_regs[UREG_I4] = address; | 247 | return; |
| 301 | regs->u_regs[UREG_I5] = regs->pc; | ||
| 302 | } | ||
| 303 | regs->u_regs[UREG_G2] = g2; | ||
| 304 | regs->pc = fixup; | ||
| 305 | regs->npc = regs->pc + 4; | ||
| 306 | return; | ||
| 307 | } | ||
| 308 | } | 248 | } |
| 309 | 249 | ||
| 310 | unhandled_fault(address, tsk, regs); | 250 | unhandled_fault(address, tsk, regs); |
| @@ -1,7 +1,5 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | 1 | /* SPDX-License-Identifier: GPL-2.0 */ |
| 2 | /* fault_32.c - visible as they are called from assembler */ | 2 | /* fault_32.c - visible as they are called from assembler */ |
| 3 | asmlinkage int lookup_fault(unsigned long pc, unsigned long ret_pc, | ||
| 4 | unsigned long address); | ||
| 5 | asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, | 3 | asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, |
| 6 | unsigned long address); | 4 | unsigned long address); |
| 7 | 5 | ||
| @@ -128,7 +128,8 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs) · 127 unmodified lines | |||
| 128 | regs->ax = -EFAULT; | 128 | regs->ax = -EFAULT; |
| 129 | 129 | ||
| 130 | instrumentation_end(); | 130 | instrumentation_end(); |
| 131 | syscall_exit_to_user_mode(regs); | 131 | local_irq_disable(); |
| 132 | irqentry_exit_to_user_mode(regs); | ||
| 132 | return false; | 133 | return false; |
| 133 | } | 134 | } |
| 134 | 135 | ||
| @@ -210,6 +210,8 @@ SYM_CODE_START(entry_SYSCALL_compat) · 209 unmodified lines | |||
| 210 | /* Switch to the kernel stack */ | 210 | /* Switch to the kernel stack */ |
| 211 | movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp | 211 | movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp |
| 212 | 212 | ||
| 213 | SYM_INNER_LABEL(entry_SYSCALL_compat_safe_stack, SYM_L_GLOBAL) | ||
| 214 | |||
| 213 | /* Construct struct pt_regs on stack */ | 215 | /* Construct struct pt_regs on stack */ |
| 214 | pushq $__USER32_DS /* pt_regs->ss */ | 216 | pushq $__USER32_DS /* pt_regs->ss */ |
| 215 | pushq %r8 /* pt_regs->sp */ | 217 | pushq %r8 /* pt_regs->sp */ |
| @@ -81,7 +81,11 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_swap_task_ctx, *x86_pmu.swap_task_ctx); · 80 unmodified lines | |||
| 81 | DEFINE_STATIC_CALL_NULL(x86_pmu_drain_pebs, *x86_pmu.drain_pebs); | 81 | DEFINE_STATIC_CALL_NULL(x86_pmu_drain_pebs, *x86_pmu.drain_pebs); |
| 82 | DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, *x86_pmu.pebs_aliases); | 82 | DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, *x86_pmu.pebs_aliases); |
| 83 | 83 | ||
| 84 | DEFINE_STATIC_CALL_NULL(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs); | 84 | /* |
| 85 | * This one is magic, it will get called even when PMU init fails (because | ||
| 86 | * there is no PMU), in which case it should simply return NULL. | ||
| 87 | */ | ||
| 88 | DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs); | ||
| 85 | 89 | ||
| 86 | u64 __read_mostly hw_cache_event_ids | 90 | u64 __read_mostly hw_cache_event_ids |
| 87 | [PERF_COUNT_HW_CACHE_MAX] | 91 | [PERF_COUNT_HW_CACHE_MAX] |
| @@ -1944,13 +1948,6 @@ static void _x86_pmu_read(struct perf_event *event) · 1856 unmodified lines | |||
| 1944 | x86_perf_event_update(event); | 1948 | x86_perf_event_update(event); |
| 1945 | } | 1949 | } |
| 1946 | 1950 | ||
| 1947 | static inline struct perf_guest_switch_msr * | ||
| 1948 | perf_guest_get_msrs_nop(int *nr) | ||
| 1949 | { | ||
| 1950 | *nr = 0; | ||
| 1951 | return NULL; | ||
| 1952 | } | ||
| 1953 | |||
| 1954 | static int __init init_hw_perf_events(void) | 1951 | static int __init init_hw_perf_events(void) |
| 1955 | { | 1952 | { |
| 1956 | struct x86_pmu_quirk *quirk; | 1953 | struct x86_pmu_quirk *quirk; |
| @@ -2025,7 +2022,7 @@ static int __init init_hw_perf_events(void) · 68 unmodified lines | |||
| 2025 | x86_pmu.read = _x86_pmu_read; | 2022 | x86_pmu.read = _x86_pmu_read; |
| 2026 | 2023 | ||
| 2027 | if (!x86_pmu.guest_get_msrs) | 2024 | if (!x86_pmu.guest_get_msrs) |
| 2028 | x86_pmu.guest_get_msrs = perf_guest_get_msrs_nop; | 2025 | x86_pmu.guest_get_msrs = (void *)&__static_call_return0; |
| 2029 | 2026 | ||
| 2030 | x86_pmu_static_call_update(); | 2027 | x86_pmu_static_call_update(); |
| 2031 | 2028 | ||
| @@ -3662,8 +3662,10 @@ static int intel_pmu_hw_config(struct perf_event *event) · 3661 unmodified lines | |||
| 3662 | if (!(event->attr.freq || (event->attr.wakeup_events && !event->attr.watermark))) { | 3662 | if (!(event->attr.freq || (event->attr.wakeup_events && !event->attr.watermark))) { |
| 3663 | event->hw.flags |= PERF_X86_EVENT_AUTO_RELOAD; | 3663 | event->hw.flags |= PERF_X86_EVENT_AUTO_RELOAD; |
| 3664 | if (!(event->attr.sample_type & | 3664 | if (!(event->attr.sample_type & |
| 3665 | ~intel_pmu_large_pebs_flags(event))) | 3665 | ~intel_pmu_large_pebs_flags(event))) { |
| 3666 | event->hw.flags |= PERF_X86_EVENT_LARGE_PEBS; | 3666 | event->hw.flags |= PERF_X86_EVENT_LARGE_PEBS; |
| 3667 | event->attach_state |= PERF_ATTACH_SCHED_CB; | ||
| 3668 | } | ||
| 3667 | } | 3669 | } |
| 3668 | if (x86_pmu.pebs_aliases) | 3670 | if (x86_pmu.pebs_aliases) |
| 3669 | x86_pmu.pebs_aliases(event); | 3671 | x86_pmu.pebs_aliases(event); |
| @@ -3676,6 +3678,7 @@ static int intel_pmu_hw_config(struct perf_event *event) · 6 unmodified lines | |||
| 3676 | ret = intel_pmu_setup_lbr_filter(event); | 3678 | ret = intel_pmu_setup_lbr_filter(event); |
| 3677 | if (ret) | 3679 | if (ret) |
| 3678 | return ret; | 3680 | return ret; |
| 3681 | event->attach_state |= PERF_ATTACH_SCHED_CB; | ||
| 3679 | 3682 | ||
| 3680 | /* | 3683 | /* |
| 3681 | * BTS is set up earlier in this path, so don't account twice | 3684 | * BTS is set up earlier in this path, so don't account twice |
| @@ -23,6 +23,8 @@ unsigned long insn_get_seg_base(struct pt_regs *regs, int seg_reg_idx); · 22 unmodified lines | |||
| 23 | int insn_get_code_seg_params(struct pt_regs *regs); | 23 | int insn_get_code_seg_params(struct pt_regs *regs); |
| 24 | int insn_fetch_from_user(struct pt_regs *regs, | 24 | int insn_fetch_from_user(struct pt_regs *regs, |
| 25 | unsigned char buf[MAX_INSN_SIZE]); | 25 | unsigned char buf[MAX_INSN_SIZE]); |
| 26 | int insn_fetch_from_user_inatomic(struct pt_regs *regs, | ||
| 27 | unsigned char buf[MAX_INSN_SIZE]); | ||
| 26 | bool insn_decode(struct insn *insn, struct pt_regs *regs, | 28 | bool insn_decode(struct insn *insn, struct pt_regs *regs, |
| 27 | unsigned char buf[MAX_INSN_SIZE], int buf_size); | 29 | unsigned char buf[MAX_INSN_SIZE], int buf_size); |
| 28 | 30 | ||
| @@ -963,7 +963,7 @@ struct kvm_arch { · 962 unmodified lines | |||
| 963 | struct kvm_pit *vpit; | 963 | struct kvm_pit *vpit; |
| 964 | atomic_t vapics_in_nmi_mode; | 964 | atomic_t vapics_in_nmi_mode; |
| 965 | struct mutex apic_map_lock; | 965 | struct mutex apic_map_lock; |
| 966 | struct kvm_apic_map *apic_map; | 966 | struct kvm_apic_map __rcu *apic_map; |
| 967 | atomic_t apic_map_dirty; | 967 | atomic_t apic_map_dirty; |
| 968 | 968 | ||
| 969 | bool apic_access_page_done; | 969 | bool apic_access_page_done; |
| @@ -1036,7 +1036,7 @@ struct kvm_arch { · 66 unmodified lines | |||
| 1036 | 1036 | ||
| 1037 | bool bus_lock_detection_enabled; | 1037 | bool bus_lock_detection_enabled; |
| 1038 | 1038 | ||
| 1039 | struct kvm_pmu_event_filter *pmu_event_filter; | 1039 | struct kvm_pmu_event_filter __rcu *pmu_event_filter; |
| 1040 | struct task_struct *nx_lpage_recovery_thread; | 1040 | struct task_struct *nx_lpage_recovery_thread; |
| 1041 | 1041 | ||
| 1042 | #ifdef CONFIG_X86_64 | 1042 | #ifdef CONFIG_X86_64 |
| @@ -25,6 +25,7 @@ void __end_SYSENTER_singlestep_region(void); · 24 unmodified lines | |||
| 25 | void entry_SYSENTER_compat(void); | 25 | void entry_SYSENTER_compat(void); |
| 26 | void __end_entry_SYSENTER_compat(void); | 26 | void __end_entry_SYSENTER_compat(void); |
| 27 | void entry_SYSCALL_compat(void); | 27 | void entry_SYSCALL_compat(void); |
| 28 | void entry_SYSCALL_compat_safe_stack(void); | ||
| 28 | void entry_INT80_compat(void); | 29 | void entry_INT80_compat(void); |
| 29 | #ifdef CONFIG_XEN_PV | 30 | #ifdef CONFIG_XEN_PV |
| 30 | void xen_entry_INT80_compat(void); | 31 | void xen_entry_INT80_compat(void); |
| @@ -94,6 +94,8 @@ struct pt_regs { · 93 unmodified lines | |||
| 94 | #include <asm/paravirt_types.h> | 94 | #include <asm/paravirt_types.h> |
| 95 | #endif | 95 | #endif |
| 96 | 96 | ||
| 97 | #include <asm/proto.h> | ||
| 98 | |||
| 97 | struct cpuinfo_x86; | 99 | struct cpuinfo_x86; |
| 98 | struct task_struct; | 100 | struct task_struct; |
| 99 | 101 | ||
| @@ -175,6 +177,19 @@ static inline bool any_64bit_mode(struct pt_regs *regs) · 75 unmodified lines | |||
| 175 | #ifdef CONFIG_X86_64 | 177 | #ifdef CONFIG_X86_64 |
| 176 | #define current_user_stack_pointer() current_pt_regs()->sp | 178 | #define current_user_stack_pointer() current_pt_regs()->sp |
| 177 | #define compat_user_stack_pointer() current_pt_regs()->sp | 179 | #define compat_user_stack_pointer() current_pt_regs()->sp |
| 180 | |||
| 181 | static inline bool ip_within_syscall_gap(struct pt_regs *regs) | ||
| 182 | { | ||
| 183 | bool ret = (regs->ip >= (unsigned long)entry_SYSCALL_64 && | ||
| 184 | regs->ip < (unsigned long)entry_SYSCALL_64_safe_stack); | ||
| 185 | |||
| 186 | #ifdef CONFIG_IA32_EMULATION | ||
| 187 | ret = ret || (regs->ip >= (unsigned long)entry_SYSCALL_compat && | ||
| 188 | regs->ip < (unsigned long)entry_SYSCALL_compat_safe_stack); | ||
| 189 | #endif | ||
| 190 | |||
| 191 | return ret; | ||
| 192 | } | ||
| 178 | #endif | 193 | #endif |
| 179 | 194 | ||
| 180 | static inline unsigned long kernel_stack_pointer(struct pt_regs *regs) | 195 | static inline unsigned long kernel_stack_pointer(struct pt_regs *regs) |
| @@ -58,9 +58,8 @@ static __always_inline unsigned long smap_save(void) · 57 unmodified lines | |||
| 58 | unsigned long flags; | 58 | unsigned long flags; |
| 59 | 59 | ||
| 60 | asm volatile ("# smap_save\n\t" | 60 | asm volatile ("# smap_save\n\t" |
| 61 | ALTERNATIVE("jmp 1f", "", X86_FEATURE_SMAP) | 61 | ALTERNATIVE("", "pushf; pop %0; " __ASM_CLAC "\n\t", |
| 62 | "pushf; pop %0; " __ASM_CLAC "\n\t" | 62 | X86_FEATURE_SMAP) |
| 63 | "1:" | ||
| 64 | : "=rm" (flags) : : "memory", "cc"); | 63 | : "=rm" (flags) : : "memory", "cc"); |
| 65 | 64 | ||
| 66 | return flags; | 65 | return flags; |
| @@ -69,9 +68,8 @@ static __always_inline unsigned long smap_save(void) · 2 unmodified lines | |||
| 69 | static __always_inline void smap_restore(unsigned long flags) | 68 | static __always_inline void smap_restore(unsigned long flags) |
| 70 | { | 69 | { |
| 71 | asm volatile ("# smap_restore\n\t" | 70 | asm volatile ("# smap_restore\n\t" |
| 72 | ALTERNATIVE("jmp 1f", "", X86_FEATURE_SMAP) | 71 | ALTERNATIVE("", "push %0; popf\n\t", |
| 73 | "push %0; popf\n\t" | 72 | X86_FEATURE_SMAP) |
| 74 | "1:" | ||
| 75 | : : "g" (flags) : "memory", "cc"); | 73 | : : "g" (flags) : "memory", "cc"); |
| 76 | } | 74 | } |
| 77 | 75 | ||
| @@ -268,21 +268,20 @@ static void __init kvmclock_init_mem(void) · 267 unmodified lines | |||
| 268 | 268 | ||
| 269 | static int __init kvm_setup_vsyscall_timeinfo(void) | 269 | static int __init kvm_setup_vsyscall_timeinfo(void) |
| 270 | { | 270 | { |
| 271 | #ifdef CONFIG_X86_64 | 271 | kvmclock_init_mem(); |
| 272 | u8 flags; | ||
| 273 | 272 | ||
| 274 | if (!per_cpu(hv_clock_per_cpu, 0) || !kvmclock_vsyscall) | 273 | #ifdef CONFIG_X86_64 |
| 275 | return 0; | 274 | if (per_cpu(hv_clock_per_cpu, 0) && kvmclock_vsyscall) { |
| 275 | u8 flags; | ||
| 276 | 276 | ||
| 277 | flags = pvclock_read_flags(&hv_clock_boot[0].pvti); | 277 | flags = pvclock_read_flags(&hv_clock_boot[0].pvti); |
| 278 | if (!(flags & PVCLOCK_TSC_STABLE_BIT)) | 278 | if (!(flags & PVCLOCK_TSC_STABLE_BIT)) |
| 279 | return 0; | 279 | return 0; |
| 280 | 280 | ||
| 281 | kvm_clock.vdso_clock_mode = VDSO_CLOCKMODE_PVCLOCK; | 281 | kvm_clock.vdso_clock_mode = VDSO_CLOCKMODE_PVCLOCK; |
| 282 | } | ||
| 282 | #endif | 283 | #endif |
| 283 | 284 | ||
| 284 | kvmclock_init_mem(); | ||
| 285 | |||
| 286 | return 0; | 285 | return 0; |
| 287 | } | 286 | } |
| 288 | early_initcall(kvm_setup_vsyscall_timeinfo); | 287 | early_initcall(kvm_setup_vsyscall_timeinfo); |
| @@ -121,8 +121,18 @@ static void __init setup_vc_stacks(int cpu) · 120 unmodified lines | |||
| 121 | cea_set_pte((void *)vaddr, pa, PAGE_KERNEL); | 121 | cea_set_pte((void *)vaddr, pa, PAGE_KERNEL); |
| 122 | } | 122 | } |
| 123 | 123 | ||
| 124 | static __always_inline bool on_vc_stack(unsigned long sp) | 124 | static __always_inline bool on_vc_stack(struct pt_regs *regs) |
| 125 | { | 125 | { |
| 126 | unsigned long sp = regs->sp; | ||
| 127 | |||
| 128 | /* User-mode RSP is not trusted */ | ||
| 129 | if (user_mode(regs)) | ||
| 130 | return false; | ||
| 131 | |||
| 132 | /* SYSCALL gap still has user-mode RSP */ | ||
| 133 | if (ip_within_syscall_gap(regs)) | ||
| 134 | return false; | ||
| 135 | |||
| 126 | return ((sp >= __this_cpu_ist_bottom_va(VC)) && (sp < __this_cpu_ist_top_va(VC))); | 136 | return ((sp >= __this_cpu_ist_bottom_va(VC)) && (sp < __this_cpu_ist_top_va(VC))); |
| 127 | } | 137 | } |
| 128 | 138 | ||
| @@ -144,7 +154,7 @@ void noinstr __sev_es_ist_enter(struct pt_regs *regs) · 15 unmodified lines | |||
| 144 | old_ist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]); | 154 | old_ist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]); |
| 145 | 155 | ||
| 146 | /* Make room on the IST stack */ | 156 | /* Make room on the IST stack */ |
| 147 | if (on_vc_stack(regs->sp)) | 157 | if (on_vc_stack(regs)) |
| 148 | new_ist = ALIGN_DOWN(regs->sp, 8) - sizeof(old_ist); | 158 | new_ist = ALIGN_DOWN(regs->sp, 8) - sizeof(old_ist); |
| 149 | else | 159 | else |
| 150 | new_ist = old_ist - sizeof(old_ist); | 160 | new_ist = old_ist - sizeof(old_ist); |
| @@ -248,7 +258,7 @@ static enum es_result vc_decode_insn(struct es_em_ctxt *ctxt) · 97 unmodified lines | |||
| 248 | int res; | 258 | int res; |
| 249 | 259 | ||
| 250 | if (user_mode(ctxt->regs)) { | 260 | if (user_mode(ctxt->regs)) { |
| 251 | res = insn_fetch_from_user(ctxt->regs, buffer); | 261 | res = insn_fetch_from_user_inatomic(ctxt->regs, buffer); |
| 252 | if (!res) { | 262 | if (!res) { |
| 253 | ctxt->fi.vector = X86_TRAP_PF; | 263 | ctxt->fi.vector = X86_TRAP_PF; |
| 254 | ctxt->fi.error_code = X86_PF_INSTR | X86_PF_USER; | 264 | ctxt->fi.error_code = X86_PF_INSTR | X86_PF_USER; |
| @@ -1248,13 +1258,12 @@ static __always_inline bool on_vc_fallback_stack(struct pt_regs *regs) · 993 unmodified lines | |||
| 1248 | DEFINE_IDTENTRY_VC_SAFE_STACK(exc_vmm_communication) | 1258 | DEFINE_IDTENTRY_VC_SAFE_STACK(exc_vmm_communication) |
| 1249 | { | 1259 | { |
| 1250 | struct sev_es_runtime_data *data = this_cpu_read(runtime_data); | 1260 | struct sev_es_runtime_data *data = this_cpu_read(runtime_data); |
| 1261 | irqentry_state_t irq_state; | ||
| 1251 | struct ghcb_state state; | 1262 | struct ghcb_state state; |
| 1252 | struct es_em_ctxt ctxt; | 1263 | struct es_em_ctxt ctxt; |
| 1253 | enum es_result result; | 1264 | enum es_result result; |
| 1254 | struct ghcb *ghcb; | 1265 | struct ghcb *ghcb; |
| 1255 | 1266 | ||
| 1256 | lockdep_assert_irqs_disabled(); | ||
| 1257 | |||
| 1258 | /* | 1267 | /* |
| 1259 | * Handle #DB before calling into !noinstr code to avoid recursive #DB. | 1268 | * Handle #DB before calling into !noinstr code to avoid recursive #DB. |
| 1260 | */ | 1269 | */ |
| @@ -1263,6 +1272,8 @@ DEFINE_IDTENTRY_VC_SAFE_STACK(exc_vmm_communication) · 2 unmodified lines | |||
| 1263 | return; | 1272 | return; |
| 1264 | } | 1273 | } |
| 1265 | 1274 | ||
| 1275 | irq_state = irqentry_nmi_enter(regs); | ||
| 1276 | lockdep_assert_irqs_disabled(); | ||
| 1266 | instrumentation_begin(); | 1277 | instrumentation_begin(); |
| 1267 | 1278 | ||
| 1268 | /* | 1279 | /* |
| @@ -1325,6 +1336,7 @@ DEFINE_IDTENTRY_VC_SAFE_STACK(exc_vmm_communication) · 56 unmodified lines | |||
| 1325 | 1336 | ||
| 1326 | out: | 1337 | out: |
| 1327 | instrumentation_end(); | 1338 | instrumentation_end(); |
| 1339 | irqentry_nmi_exit(regs, irq_state); | ||
| 1328 | 1340 | ||
| 1329 | return; | 1341 | return; |
| 1330 | 1342 | ||
| @@ -694,8 +694,7 @@ asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *r · 693 unmodified lines | |||
| 694 | * In the SYSCALL entry path the RSP value comes from user-space - don't | 694 | * In the SYSCALL entry path the RSP value comes from user-space - don't |
| 695 | * trust it and switch to the current kernel stack | 695 | * trust it and switch to the current kernel stack |
| 696 | */ | 696 | */ |
| 697 | if (regs->ip >= (unsigned long)entry_SYSCALL_64 && | 697 | if (ip_within_syscall_gap(regs)) { |
| 698 | regs->ip < (unsigned long)entry_SYSCALL_64_safe_stack) { | ||
| 699 | sp = this_cpu_read(cpu_current_top_of_stack); | 698 | sp = this_cpu_read(cpu_current_top_of_stack); |
| 700 | goto sync; | 699 | goto sync; |
| 701 | } | 700 | } |
| @@ -13,7 +13,7 @@ · 12 unmodified lines | |||
| 13 | 13 | ||
| 14 | #define orc_warn_current(args...) \ | 14 | #define orc_warn_current(args...) \ |
| 15 | ({ \ | 15 | ({ \ |
| 16 | if (state->task == current) \ | 16 | if (state->task == current && !state->error) \ |
| 17 | orc_warn(args); \ | 17 | orc_warn(args); \ |
| 18 | }) | 18 | }) |
| 19 | 19 | ||
| @@ -367,8 +367,8 @@ static bool deref_stack_regs(struct unwind_state *state, unsigned long addr, · 347 unmodified lines | |||
| 367 | if (!stack_access_ok(state, addr, sizeof(struct pt_regs))) | 367 | if (!stack_access_ok(state, addr, sizeof(struct pt_regs))) |
| 368 | return false; | 368 | return false; |
| 369 | 369 | ||
| 370 | *ip = regs->ip; | 370 | *ip = READ_ONCE_NOCHECK(regs->ip); |
| 371 | *sp = regs->sp; | 371 | *sp = READ_ONCE_NOCHECK(regs->sp); |
| 372 | return true; | 372 | return true; |
| 373 | } | 373 | } |
| 374 | 374 | ||
| @@ -380,8 +380,8 @@ static bool deref_stack_iret_regs(struct unwind_state *state, unsigned long addr · 5 unmodified lines | |||
| 380 | if (!stack_access_ok(state, addr, IRET_FRAME_SIZE)) | 380 | if (!stack_access_ok(state, addr, IRET_FRAME_SIZE)) |
| 381 | return false; | 381 | return false; |
| 382 | 382 | ||
| 383 | *ip = regs->ip; | 383 | *ip = READ_ONCE_NOCHECK(regs->ip); |
| 384 | *sp = regs->sp; | 384 | *sp = READ_ONCE_NOCHECK(regs->sp); |
| 385 | return true; | 385 | return true; |
| 386 | } | 386 | } |
| 387 | 387 | ||
| @@ -402,12 +402,12 @@ static bool get_reg(struct unwind_state *state, unsigned int reg_off, · 14 unmodified lines | |||
| 402 | return false; | 402 | return false; |
| 403 | 403 | ||
| 404 | if (state->full_regs) { | 404 | if (state->full_regs) { |
| 405 | *val = ((unsigned long *)state->regs)[reg]; | 405 | *val = READ_ONCE_NOCHECK(((unsigned long *)state->regs)[reg]); |
| 406 | return true; | 406 | return true; |
| 407 | } | 407 | } |
| 408 | 408 | ||
| 409 | if (state->prev_regs) { | 409 | if (state->prev_regs) { |
| 410 | *val = ((unsigned long *)state->prev_regs)[reg]; | 410 | *val = READ_ONCE_NOCHECK(((unsigned long *)state->prev_regs)[reg]); |
| 411 | return true; | 411 | return true; |
| 412 | } | 412 | } |
| 413 | 413 | ||
| @@ -1642,7 +1642,16 @@ static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) · 1641 unmodified lines | |||
| 1642 | } | 1642 | } |
| 1643 | 1643 | ||
| 1644 | if (kvm_use_posted_timer_interrupt(apic->vcpu)) { | 1644 | if (kvm_use_posted_timer_interrupt(apic->vcpu)) { |
| 1645 | kvm_wait_lapic_expire(vcpu); | 1645 | /* |
| 1646 | * Ensure the guest's timer has truly expired before posting an | ||
| 1647 | * interrupt. Open code the relevant checks to avoid querying | ||
| 1648 | * lapic_timer_int_injected(), which will be false since the | ||
| 1649 | * interrupt isn't yet injected. Waiting until after injecting | ||
| 1650 | * is not an option since that won't help a posted interrupt. | ||
| 1651 | */ | ||
| 1652 | if (vcpu->arch.apic->lapic_timer.expired_tscdeadline && | ||
| 1653 | vcpu->arch.apic->lapic_timer.timer_advance_ns) | ||
| 1654 | __kvm_wait_lapic_expire(vcpu); | ||
| 1646 | kvm_apic_inject_pending_timer_irqs(apic); | 1655 | kvm_apic_inject_pending_timer_irqs(apic); |
| 1647 | return; | 1656 | return; |
| 1648 | } | 1657 | } |
| @@ -2595,6 +2604,7 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s) · 946 unmodified lines | |||
| 2595 | 2604 | ||
| 2596 | apic_update_ppr(apic); | 2605 | apic_update_ppr(apic); |
| 2597 | hrtimer_cancel(&apic->lapic_timer.timer); | 2606 | hrtimer_cancel(&apic->lapic_timer.timer); |
| 2607 | apic->lapic_timer.expired_tscdeadline = 0; | ||
| 2598 | apic_update_lvtt(apic); | 2608 | apic_update_lvtt(apic); |
| 2599 | apic_manage_nmi_watchdog(apic, kvm_lapic_get_reg(apic, APIC_LVT0)); | 2609 | apic_manage_nmi_watchdog(apic, kvm_lapic_get_reg(apic, APIC_LVT0)); |
| 2600 | update_divide_count(apic); | 2610 | update_divide_count(apic); |
| @@ -337,7 +337,18 @@ static void handle_removed_tdp_mmu_page(struct kvm *kvm, u64 *pt, · 336 unmodified lines | |||
| 337 | cpu_relax(); | 337 | cpu_relax(); |
| 338 | } | 338 | } |
| 339 | } else { | 339 | } else { |
| 340 | /* | ||
| 341 | * If the SPTE is not MMU-present, there is no backing | ||
| 342 | * page associated with the SPTE and so no side effects | ||
| 343 | * that need to be recorded, and exclusive ownership of | ||
| 344 | * mmu_lock ensures the SPTE can't be made present. | ||
| 345 | * Note, zapping MMIO SPTEs is also unnecessary as they | ||
| 346 | * are guarded by the memslots generation, not by being | ||
| 347 | * unreachable. | ||
| 348 | */ | ||
| 340 | old_child_spte = READ_ONCE(*sptep); | 349 | old_child_spte = READ_ONCE(*sptep); |
| 350 | if (!is_shadow_present_pte(old_child_spte)) | ||
| 351 | continue; | ||
| 341 | 352 | ||
| 342 | /* | 353 | /* |
| 343 | * Marking the SPTE as a removed SPTE is not | 354 | * Marking the SPTE as a removed SPTE is not |
| @@ -115,13 +115,6 @@ static const struct svm_direct_access_msrs { · 114 unmodified lines | |||
| 115 | { .index = MSR_INVALID, .always = false }, | 115 | { .index = MSR_INVALID, .always = false }, |
| 116 | }; | 116 | }; |
| 117 | 117 | ||
| 118 | /* enable NPT for AMD64 and X86 with PAE */ | ||
| 119 | #if defined(CONFIG_X86_64) || defined(CONFIG_X86_PAE) | ||
| 120 | bool npt_enabled = true; | ||
| 121 | #else | ||
| 122 | bool npt_enabled; | ||
| 123 | #endif | ||
| 124 | |||
| 125 | /* | 118 | /* |
| 126 | * These 2 parameters are used to config the controls for Pause-Loop Exiting: | 119 | * These 2 parameters are used to config the controls for Pause-Loop Exiting: |
| 127 | * pause_filter_count: On processors that support Pause filtering(indicated | 120 | * pause_filter_count: On processors that support Pause filtering(indicated |
| @@ -170,9 +163,12 @@ module_param(pause_filter_count_shrink, ushort, 0444); · 42 unmodified lines | |||
| 170 | static unsigned short pause_filter_count_max = KVM_SVM_DEFAULT_PLE_WINDOW_MAX; | 163 | static unsigned short pause_filter_count_max = KVM_SVM_DEFAULT_PLE_WINDOW_MAX; |
| 171 | module_param(pause_filter_count_max, ushort, 0444); | 164 | module_param(pause_filter_count_max, ushort, 0444); |
| 172 | 165 | ||
| 173 | /* allow nested paging (virtualized MMU) for all guests */ | 166 | /* |
| 174 | static int npt = true; | 167 | * Use nested page tables by default. Note, NPT may get forced off by |
| 175 | module_param(npt, int, S_IRUGO); | 168 | * svm_hardware_setup() if it's unsupported by hardware or the host kernel. |
| 169 | */ | ||
| 170 | bool npt_enabled = true; | ||
| 171 | module_param_named(npt, npt_enabled, bool, 0444); | ||
| 176 | 172 | ||
| 177 | /* allow nested virtualization in KVM/SVM */ | 173 | /* allow nested virtualization in KVM/SVM */ |
| 178 | static int nested = true; | 174 | static int nested = true; |
| @@ -988,10 +984,15 @@ static __init int svm_hardware_setup(void) · 809 unmodified lines | |||
| 988 | goto err; | 984 | goto err; |
| 989 | } | 985 | } |
| 990 | 986 | ||
| 991 | if (!boot_cpu_has(X86_FEATURE_NPT)) | 987 | /* |
| 988 | * KVM's MMU doesn't support using 2-level paging for itself, and thus | ||
| 989 | * NPT isn't supported if the host is using 2-level paging since host | ||
| 990 | * CR4 is unchanged on VMRUN. | ||
| 991 | */ | ||
| 992 | if (!IS_ENABLED(CONFIG_X86_64) && !IS_ENABLED(CONFIG_X86_PAE)) | ||
| 992 | npt_enabled = false; | 993 | npt_enabled = false; |
| 993 | 994 | ||
| 994 | if (npt_enabled && !npt) | 995 | if (!boot_cpu_has(X86_FEATURE_NPT)) |
| 995 | npt_enabled = false; | 996 | npt_enabled = false; |
| 996 | 997 | ||
| 997 | kvm_configure_mmu(npt_enabled, get_max_npt_level(), PG_LEVEL_1G); | 998 | kvm_configure_mmu(npt_enabled, get_max_npt_level(), PG_LEVEL_1G); |
| @@ -6580,8 +6580,8 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · 6579 unmodified lines | |||
| 6580 | int i, nr_msrs; | 6580 | int i, nr_msrs; |
| 6581 | struct perf_guest_switch_msr *msrs; | 6581 | struct perf_guest_switch_msr *msrs; |
| 6582 | 6582 | ||
| 6583 | /* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */ | ||
| 6583 | msrs = perf_guest_get_msrs(&nr_msrs); | 6584 | msrs = perf_guest_get_msrs(&nr_msrs); |
| 6584 | |||
| 6585 | if (!msrs) | 6585 | if (!msrs) |
| 6586 | return; | 6586 | return; |
| 6587 | 6587 | ||
| @@ -10601,7 +10601,7 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, · 10600 unmodified lines | |||
| 10601 | return (void __user *)hva; | 10601 | return (void __user *)hva; |
| 10602 | } else { | 10602 | } else { |
| 10603 | if (!slot || !slot->npages) | 10603 | if (!slot || !slot->npages) |
| 10604 | return 0; | 10604 | return NULL; |
| 10605 | 10605 | ||
| 10606 | old_npages = slot->npages; | 10606 | old_npages = slot->npages; |
| 10607 | hva = slot->userspace_addr; | 10607 | hva = slot->userspace_addr; |
| @@ -1415,6 +1415,25 @@ void __user *insn_get_addr_ref(struct insn *insn, struct pt_regs *regs) · 1414 unmodified lines | |||
| 1415 | } | 1415 | } |
| 1416 | } | 1416 | } |
| 1417 | 1417 | ||
| 1418 | static unsigned long insn_get_effective_ip(struct pt_regs *regs) | ||
| 1419 | { | ||
| 1420 | unsigned long seg_base = 0; | ||
| 1421 | |||
| 1422 | /* | ||
| 1423 | * If not in user-space long mode, a custom code segment could be in | ||
| 1424 | * use. This is true in protected mode (if the process defined a local | ||
| 1425 | * descriptor table), or virtual-8086 mode. In most of the cases | ||
| 1426 | * seg_base will be zero as in USER_CS. | ||
| 1427 | */ | ||
| 1428 | if (!user_64bit_mode(regs)) { | ||
| 1429 | seg_base = insn_get_seg_base(regs, INAT_SEG_REG_CS); | ||
| 1430 | if (seg_base == -1L) | ||
| 1431 | return 0; | ||
| 1432 | } | ||
| 1433 | |||
| 1434 | return seg_base + regs->ip; | ||
| 1435 | } | ||
| 1436 | |||
| 1418 | /** | 1437 | /** |
| 1419 | * insn_fetch_from_user() - Copy instruction bytes from user-space memory | 1438 | * insn_fetch_from_user() - Copy instruction bytes from user-space memory |
| 1420 | * @regs: Structure with register values as seen when entering kernel mode | 1439 | * @regs: Structure with register values as seen when entering kernel mode |
| @@ -1431,24 +1450,43 @@ void __user *insn_get_addr_ref(struct insn *insn, struct pt_regs *regs) · 10 unmodified lines | |||
| 1431 | */ | 1450 | */ |
| 1432 | int insn_fetch_from_user(struct pt_regs *regs, unsigned char buf[MAX_INSN_SIZE]) | 1451 | int insn_fetch_from_user(struct pt_regs *regs, unsigned char buf[MAX_INSN_SIZE]) |
| 1433 | { | 1452 | { |
| 1434 | unsigned long seg_base = 0; | 1453 | unsigned long ip; |
| 1435 | int not_copied; | 1454 | int not_copied; |
| 1436 | 1455 | ||
| 1437 | /* | 1456 | ip = insn_get_effective_ip(regs); |
| 1438 | * If not in user-space long mode, a custom code segment could be in | 1457 | if (!ip) |
| 1439 | * use. This is true in protected mode (if the process defined a local | 1458 | return 0; |
| 1440 | * descriptor table), or virtual-8086 mode. In most of the cases | 1459 | |
| 1441 | * seg_base will be zero as in USER_CS. | 1460 | not_copied = copy_from_user(buf, (void __user *)ip, MAX_INSN_SIZE); |
| 1442 | */ | ||
| 1443 | if (!user_64bit_mode(regs)) { | ||
| 1444 | seg_base = insn_get_seg_base(regs, INAT_SEG_REG_CS); | ||
| 1445 | if (seg_base == -1L) | ||
| 1446 | return 0; | ||
| 1447 | } | ||
| 1448 | 1461 | ||
| 1462 | return MAX_INSN_SIZE - not_copied; | ||
| 1463 | } | ||
| 1464 | |||
| 1465 | /** | ||
| 1466 | * insn_fetch_from_user_inatomic() - Copy instruction bytes from user-space memory | ||
| 1467 | * while in atomic code | ||
| 1468 | * @regs: Structure with register values as seen when entering kernel mode | ||
| 1469 | * @buf: Array to store the fetched instruction | ||
| 1470 | * | ||
| 1471 | * Gets the linear address of the instruction and copies the instruction bytes | ||
| 1472 | * to the buf. This function must be used in atomic context. | ||
| 1473 | * | ||
| 1474 | * Returns: | ||
| 1475 | * | ||
| 1476 | * Number of instruction bytes copied. | ||
| 1477 | * | ||
| 1478 | * 0 if nothing was copied. | ||
| 1479 | */ | ||
| 1480 | int insn_fetch_from_user_inatomic(struct pt_regs *regs, unsigned char buf[MAX_INSN_SIZE]) | ||
| 1481 | { | ||
| 1482 | unsigned long ip; | ||
| 1483 | int not_copied; | ||
| 1484 | |||
| 1485 | ip = insn_get_effective_ip(regs); | ||
| 1486 | if (!ip) | ||
| 1487 | return 0; | ||
| 1449 | 1488 | ||
| 1450 | not_copied = copy_from_user(buf, (void __user *)(seg_base + regs->ip), | 1489 | not_copied = __copy_from_user_inatomic(buf, (void __user *)ip, MAX_INSN_SIZE); |
| 1451 | MAX_INSN_SIZE); | ||
| 1452 | 1490 | ||
| 1453 | return MAX_INSN_SIZE - not_copied; | 1491 | return MAX_INSN_SIZE - not_copied; |
| 1454 | } | 1492 | } |
| @@ -1349,6 +1349,7 @@ st: if (is_imm8(insn->off)) · 1348 unmodified lines | |||
| 1349 | insn->imm == (BPF_XOR | BPF_FETCH)) { | 1349 | insn->imm == (BPF_XOR | BPF_FETCH)) { |
| 1350 | u8 *branch_target; | 1350 | u8 *branch_target; |
| 1351 | bool is64 = BPF_SIZE(insn->code) == BPF_DW; | 1351 | bool is64 = BPF_SIZE(insn->code) == BPF_DW; |
| 1352 | u32 real_src_reg = src_reg; | ||
| 1352 | 1353 | ||
| 1353 | /* | 1354 | /* |
| 1354 | * Can't be implemented with a single x86 insn. | 1355 | * Can't be implemented with a single x86 insn. |
| @@ -1357,6 +1358,9 @@ st: if (is_imm8(insn->off)) · 2 unmodified lines | |||
| 1357 | 1358 | ||
| 1358 | /* Will need RAX as a CMPXCHG operand so save R0 */ | 1359 | /* Will need RAX as a CMPXCHG operand so save R0 */ |
| 1359 | emit_mov_reg(&prog, true, BPF_REG_AX, BPF_REG_0); | 1360 | emit_mov_reg(&prog, true, BPF_REG_AX, BPF_REG_0); |
| 1361 | if (src_reg == BPF_REG_0) | ||
| 1362 | real_src_reg = BPF_REG_AX; | ||
| 1363 | |||
| 1360 | branch_target = prog; | 1364 | branch_target = prog; |
| 1361 | /* Load old value */ | 1365 | /* Load old value */ |
| 1362 | emit_ldx(&prog, BPF_SIZE(insn->code), | 1366 | emit_ldx(&prog, BPF_SIZE(insn->code), |
| @@ -1366,9 +1370,9 @@ st: if (is_imm8(insn->off)) · 3 unmodified lines | |||
| 1366 | * put the result in the AUX_REG. | 1370 | * put the result in the AUX_REG. |
| 1367 | */ | 1371 | */ |
| 1368 | emit_mov_reg(&prog, is64, AUX_REG, BPF_REG_0); | 1372 | emit_mov_reg(&prog, is64, AUX_REG, BPF_REG_0); |
| 1369 | maybe_emit_mod(&prog, AUX_REG, src_reg, is64); | 1373 | maybe_emit_mod(&prog, AUX_REG, real_src_reg, is64); |
| 1370 | EMIT2(simple_alu_opcodes[BPF_OP(insn->imm)], | 1374 | EMIT2(simple_alu_opcodes[BPF_OP(insn->imm)], |
| 1371 | add_2reg(0xC0, AUX_REG, src_reg)); | 1375 | add_2reg(0xC0, AUX_REG, real_src_reg)); |
| 1372 | /* Attempt to swap in new value */ | 1376 | /* Attempt to swap in new value */ |
| 1373 | err = emit_atomic(&prog, BPF_CMPXCHG, | 1377 | err = emit_atomic(&prog, BPF_CMPXCHG, |
| 1374 | dst_reg, AUX_REG, insn->off, | 1378 | dst_reg, AUX_REG, insn->off, |
| @@ -1381,7 +1385,7 @@ st: if (is_imm8(insn->off)) · 6 unmodified lines | |||
| 1381 | */ | 1385 | */ |
| 1382 | EMIT2(X86_JNE, -(prog - branch_target) - 2); | 1386 | EMIT2(X86_JNE, -(prog - branch_target) - 2); |
| 1383 | /* Return the pre-modification value */ | 1387 | /* Return the pre-modification value */ |
| 1384 | emit_mov_reg(&prog, is64, src_reg, BPF_REG_0); | 1388 | emit_mov_reg(&prog, is64, real_src_reg, BPF_REG_0); |
| 1385 | /* Restore R0 after clobbering RAX */ | 1389 | /* Restore R0 after clobbering RAX */ |
| 1386 | emit_mov_reg(&prog, true, BPF_REG_0, BPF_REG_AX); | 1390 | emit_mov_reg(&prog, true, BPF_REG_0, BPF_REG_AX); |
| 1387 | break; | 1391 | break; |
| @@ -741,7 +741,7 @@ int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, · 740 unmodified lines | |||
| 741 | map_ops[i].status = GNTST_general_error; | 741 | map_ops[i].status = GNTST_general_error; |
| 742 | unmap[0].host_addr = map_ops[i].host_addr, | 742 | unmap[0].host_addr = map_ops[i].host_addr, |
| 743 | unmap[0].handle = map_ops[i].handle; | 743 | unmap[0].handle = map_ops[i].handle; |
| 744 | map_ops[i].handle = ~0; | 744 | map_ops[i].handle = INVALID_GRANT_HANDLE; |
| 745 | if (map_ops[i].flags & GNTMAP_device_map) | 745 | if (map_ops[i].flags & GNTMAP_device_map) |
| 746 | unmap[0].dev_bus_addr = map_ops[i].dev_bus_addr; | 746 | unmap[0].dev_bus_addr = map_ops[i].dev_bus_addr; |
| 747 | else | 747 | else |
| @@ -751,7 +751,7 @@ int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, · 3 unmodified lines | |||
| 751 | kmap_ops[i].status = GNTST_general_error; | 751 | kmap_ops[i].status = GNTST_general_error; |
| 752 | unmap[1].host_addr = kmap_ops[i].host_addr, | 752 | unmap[1].host_addr = kmap_ops[i].host_addr, |
| 753 | unmap[1].handle = kmap_ops[i].handle; | 753 | unmap[1].handle = kmap_ops[i].handle; |
| 754 | kmap_ops[i].handle = ~0; | 754 | kmap_ops[i].handle = INVALID_GRANT_HANDLE; |
| 755 | if (kmap_ops[i].flags & GNTMAP_device_map) | 755 | if (kmap_ops[i].flags & GNTMAP_device_map) |
| 756 | unmap[1].dev_bus_addr = kmap_ops[i].dev_bus_addr; | 756 | unmap[1].dev_bus_addr = kmap_ops[i].dev_bus_addr; |
| 757 | else | 757 | else |
| @@ -776,7 +776,6 @@ int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, · 18 unmodified lines | |||
| 776 | out: | 776 | out: |
| 777 | return ret; | 777 | return ret; |
| 778 | } | 778 | } |
| 779 | EXPORT_SYMBOL_GPL(set_foreign_p2m_mapping); | ||
| 780 | 779 | ||
| 781 | int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, | 780 | int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, |
| 782 | struct gnttab_unmap_grant_ref *kunmap_ops, | 781 | struct gnttab_unmap_grant_ref *kunmap_ops, |
| @@ -802,7 +801,6 @@ int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, · 19 unmodified lines | |||
| 802 | 801 | ||
| 803 | return ret; | 802 | return ret; |
| 804 | } | 803 | } |
| 805 | EXPORT_SYMBOL_GPL(clear_foreign_p2m_mapping); | ||
| 806 | 804 | ||
| 807 | #ifdef CONFIG_XEN_DEBUG_FS | 805 | #ifdef CONFIG_XEN_DEBUG_FS |
| 808 | #include <linux/debugfs.h> | 806 | #include <linux/debugfs.h> |
| @@ -33,7 +33,7 @@ static struct biovec_slab { · 32 unmodified lines | |||
| 33 | { .nr_vecs = 16, .name = "biovec-16" }, | 33 | { .nr_vecs = 16, .name = "biovec-16" }, |
| 34 | { .nr_vecs = 64, .name = "biovec-64" }, | 34 | { .nr_vecs = 64, .name = "biovec-64" }, |
| 35 | { .nr_vecs = 128, .name = "biovec-128" }, | 35 | { .nr_vecs = 128, .name = "biovec-128" }, |
| 36 | { .nr_vecs = BIO_MAX_PAGES, .name = "biovec-max" }, | 36 | { .nr_vecs = BIO_MAX_VECS, .name = "biovec-max" }, |
| 37 | }; | 37 | }; |
| 38 | 38 | ||
| 39 | static struct biovec_slab *biovec_slab(unsigned short nr_vecs) | 39 | static struct biovec_slab *biovec_slab(unsigned short nr_vecs) |
| @@ -46,7 +46,7 @@ static struct biovec_slab *biovec_slab(unsigned short nr_vecs) · 6 unmodified lines | |||
| 46 | return &bvec_slabs[1]; | 46 | return &bvec_slabs[1]; |
| 47 | case 65 ... 128: | 47 | case 65 ... 128: |
| 48 | return &bvec_slabs[2]; | 48 | return &bvec_slabs[2]; |
| 49 | case 129 ... BIO_MAX_PAGES: | 49 | case 129 ... BIO_MAX_VECS: |
| 50 | return &bvec_slabs[3]; | 50 | return &bvec_slabs[3]; |
| 51 | default: | 51 | default: |
| 52 | BUG(); | 52 | BUG(); |
| @@ -151,9 +151,9 @@ out: · 98 unmodified lines | |||
| 151 | 151 | ||
| 152 | void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned short nr_vecs) | 152 | void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned short nr_vecs) |
| 153 | { | 153 | { |
| 154 | BIO_BUG_ON(nr_vecs > BIO_MAX_PAGES); | 154 | BIO_BUG_ON(nr_vecs > BIO_MAX_VECS); |
| 155 | 155 | ||
| 156 | if (nr_vecs == BIO_MAX_PAGES) | 156 | if (nr_vecs == BIO_MAX_VECS) |
| 157 | mempool_free(bv, pool); | 157 | mempool_free(bv, pool); |
| 158 | else if (nr_vecs > BIO_INLINE_VECS) | 158 | else if (nr_vecs > BIO_INLINE_VECS) |
| 159 | kmem_cache_free(biovec_slab(nr_vecs)->slab, bv); | 159 | kmem_cache_free(biovec_slab(nr_vecs)->slab, bv); |
| @@ -186,15 +186,15 @@ struct bio_vec *bvec_alloc(mempool_t *pool, unsigned short *nr_vecs, · 26 unmodified lines | |||
| 186 | /* | 186 | /* |
| 187 | * Try a slab allocation first for all smaller allocations. If that | 187 | * Try a slab allocation first for all smaller allocations. If that |
| 188 | * fails and __GFP_DIRECT_RECLAIM is set retry with the mempool. | 188 | * fails and __GFP_DIRECT_RECLAIM is set retry with the mempool. |
| 189 | * The mempool is sized to handle up to BIO_MAX_PAGES entries. | 189 | * The mempool is sized to handle up to BIO_MAX_VECS entries. |
| 190 | */ | 190 | */ |
| 191 | if (*nr_vecs < BIO_MAX_PAGES) { | 191 | if (*nr_vecs < BIO_MAX_VECS) { |
| 192 | struct bio_vec *bvl; | 192 | struct bio_vec *bvl; |
| 193 | 193 | ||
| 194 | bvl = kmem_cache_alloc(bvs->slab, bvec_alloc_gfp(gfp_mask)); | 194 | bvl = kmem_cache_alloc(bvs->slab, bvec_alloc_gfp(gfp_mask)); |
| 195 | if (likely(bvl) || !(gfp_mask & __GFP_DIRECT_RECLAIM)) | 195 | if (likely(bvl) || !(gfp_mask & __GFP_DIRECT_RECLAIM)) |
| 196 | return bvl; | 196 | return bvl; |
| 197 | *nr_vecs = BIO_MAX_PAGES; | 197 | *nr_vecs = BIO_MAX_VECS; |
| 198 | } | 198 | } |
| 199 | 199 | ||
| 200 | return mempool_alloc(pool, gfp_mask); | 200 | return mempool_alloc(pool, gfp_mask); |
| @@ -109,6 +109,7 @@ void blkg_rwstat_recursive_sum(struct blkcg_gq *blkg, struct blkcg_policy *pol, · 108 unmodified lines | |||
| 109 | 109 | ||
| 110 | lockdep_assert_held(&blkg->q->queue_lock); | 110 | lockdep_assert_held(&blkg->q->queue_lock); |
| 111 | 111 | ||
| 112 | memset(sum, 0, sizeof(*sum)); | ||
| 112 | rcu_read_lock(); | 113 | rcu_read_lock(); |
| 113 | blkg_for_each_descendant_pre(pos_blkg, pos_css, blkg) { | 114 | blkg_for_each_descendant_pre(pos_blkg, pos_css, blkg) { |
| 114 | struct blkg_rwstat *rwstat; | 115 | struct blkg_rwstat *rwstat; |
| @@ -122,7 +123,7 @@ void blkg_rwstat_recursive_sum(struct blkcg_gq *blkg, struct blkcg_policy *pol, · 7 unmodified lines | |||
| 122 | rwstat = (void *)pos_blkg + off; | 123 | rwstat = (void *)pos_blkg + off; |
| 123 | 124 | ||
| 124 | for (i = 0; i < BLKG_RWSTAT_NR; i++) | 125 | for (i = 0; i < BLKG_RWSTAT_NR; i++) |
| 125 | sum->cnt[i] = blkg_rwstat_read_counter(rwstat, i); | 126 | sum->cnt[i] += blkg_rwstat_read_counter(rwstat, i); |
| 126 | } | 127 | } |
| 127 | rcu_read_unlock(); | 128 | rcu_read_unlock(); |
| 128 | } | 129 | } |
| @@ -219,7 +219,7 @@ static bool blk_crypto_split_bio_if_needed(struct bio **bio_ptr) · 218 unmodified lines | |||
| 219 | 219 | ||
| 220 | bio_for_each_segment(bv, bio, iter) { | 220 | bio_for_each_segment(bv, bio, iter) { |
| 221 | num_sectors += bv.bv_len >> SECTOR_SHIFT; | 221 | num_sectors += bv.bv_len >> SECTOR_SHIFT; |
| 222 | if (++i == BIO_MAX_PAGES) | 222 | if (++i == BIO_MAX_VECS) |
| 223 | break; | 223 | break; |
| 224 | } | 224 | } |
| 225 | if (num_sectors < bio_sectors(bio)) { | 225 | if (num_sectors < bio_sectors(bio)) { |
| @@ -296,7 +296,7 @@ static unsigned int __blkdev_sectors_to_bio_pages(sector_t nr_sects) · 295 unmodified lines | |||
| 296 | { | 296 | { |
| 297 | sector_t pages = DIV_ROUND_UP_SECTOR_T(nr_sects, PAGE_SIZE / 512); | 297 | sector_t pages = DIV_ROUND_UP_SECTOR_T(nr_sects, PAGE_SIZE / 512); |
| 298 | 298 | ||
| 299 | return min(pages, (sector_t)BIO_MAX_PAGES); | 299 | return min(pages, (sector_t)BIO_MAX_VECS); |
| 300 | } | 300 | } |
| 301 | 301 | ||
| 302 | static int __blkdev_issue_zero_pages(struct block_device *bdev, | 302 | static int __blkdev_issue_zero_pages(struct block_device *bdev, |
| @@ -249,7 +249,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, · 248 unmodified lines | |||
| 249 | if (!iov_iter_count(iter)) | 249 | if (!iov_iter_count(iter)) |
| 250 | return -EINVAL; | 250 | return -EINVAL; |
| 251 | 251 | ||
| 252 | bio = bio_kmalloc(gfp_mask, iov_iter_npages(iter, BIO_MAX_PAGES)); | 252 | bio = bio_kmalloc(gfp_mask, iov_iter_npages(iter, BIO_MAX_VECS)); |
| 253 | if (!bio) | 253 | if (!bio) |
| 254 | return -ENOMEM; | 254 | return -ENOMEM; |
| 255 | bio->bi_opf |= req_op(rq); | 255 | bio->bi_opf |= req_op(rq); |
| @@ -240,7 +240,7 @@ int blkdev_zone_mgmt(struct block_device *bdev, enum req_opf op, · 239 unmodified lines | |||
| 240 | */ | 240 | */ |
| 241 | if (op == REQ_OP_ZONE_RESET && | 241 | if (op == REQ_OP_ZONE_RESET && |
| 242 | blkdev_allow_reset_all_zones(bdev, sector, nr_sectors)) { | 242 | blkdev_allow_reset_all_zones(bdev, sector, nr_sectors)) { |
| 243 | bio->bi_opf = REQ_OP_ZONE_RESET_ALL; | 243 | bio->bi_opf = REQ_OP_ZONE_RESET_ALL | REQ_SYNC; |
| 244 | break; | 244 | break; |
| 245 | } | 245 | } |
| 246 | 246 | ||
| @@ -318,6 +318,22 @@ int blkdev_report_zones_ioctl(struct block_device *bdev, fmode_t mode, · 71 unmodified lines | |||
| 318 | return 0; | 318 | return 0; |
| 319 | } | 319 | } |
| 320 | 320 | ||
| 321 | static int blkdev_truncate_zone_range(struct block_device *bdev, fmode_t mode, | ||
| 322 | const struct blk_zone_range *zrange) | ||
| 323 | { | ||
| 324 | loff_t start, end; | ||
| 325 | |||
| 326 | if (zrange->sector + zrange->nr_sectors <= zrange->sector || | ||
| 327 | zrange->sector + zrange->nr_sectors > get_capacity(bdev->bd_disk)) | ||
| 328 | /* Out of range */ | ||
| 329 | return -EINVAL; | ||
| 330 | |||
| 331 | start = zrange->sector << SECTOR_SHIFT; | ||
| 332 | end = ((zrange->sector + zrange->nr_sectors) << SECTOR_SHIFT) - 1; | ||
| 333 | |||
| 334 | return truncate_bdev_range(bdev, mode, start, end); | ||
| 335 | } | ||
| 336 | |||
| 321 | /* | 337 | /* |
| 322 | * BLKRESETZONE, BLKOPENZONE, BLKCLOSEZONE and BLKFINISHZONE ioctl processing. | 338 | * BLKRESETZONE, BLKOPENZONE, BLKCLOSEZONE and BLKFINISHZONE ioctl processing. |
| 323 | * Called from blkdev_ioctl. | 339 | * Called from blkdev_ioctl. |
| @@ -329,6 +345,7 @@ int blkdev_zone_mgmt_ioctl(struct block_device *bdev, fmode_t mode, · 5 unmodified lines | |||
| 329 | struct request_queue *q; | 345 | struct request_queue *q; |
| 330 | struct blk_zone_range zrange; | 346 | struct blk_zone_range zrange; |
| 331 | enum req_opf op; | 347 | enum req_opf op; |
| 348 | int ret; | ||
| 332 | 349 | ||
| 333 | if (!argp) | 350 | if (!argp) |
| 334 | return -EINVAL; | 351 | return -EINVAL; |
| @@ -352,6 +369,11 @@ int blkdev_zone_mgmt_ioctl(struct block_device *bdev, fmode_t mode, · 17 unmodified lines | |||
| 352 | switch (cmd) { | 369 | switch (cmd) { |
| 353 | case BLKRESETZONE: | 370 | case BLKRESETZONE: |
| 354 | op = REQ_OP_ZONE_RESET; | 371 | op = REQ_OP_ZONE_RESET; |
| 372 | |||
| 373 | /* Invalidate the page cache, including dirty pages. */ | ||
| 374 | ret = blkdev_truncate_zone_range(bdev, mode, &zrange); | ||
| 375 | if (ret) | ||
| 376 | return ret; | ||
| 355 | break; | 377 | break; |
| 356 | case BLKOPENZONE: | 378 | case BLKOPENZONE: |
| 357 | op = REQ_OP_ZONE_OPEN; | 379 | op = REQ_OP_ZONE_OPEN; |
| @@ -366,8 +388,20 @@ int blkdev_zone_mgmt_ioctl(struct block_device *bdev, fmode_t mode, · 8 unmodified lines | |||
| 366 | return -ENOTTY; | 388 | return -ENOTTY; |
| 367 | } | 389 | } |
| 368 | 390 | ||
| 369 | return blkdev_zone_mgmt(bdev, op, zrange.sector, zrange.nr_sectors, | 391 | ret = blkdev_zone_mgmt(bdev, op, zrange.sector, zrange.nr_sectors, |
| 370 | GFP_KERNEL); | 392 | GFP_KERNEL); |
| 393 | |||
| 394 | /* | ||
| 395 | * Invalidate the page cache again for zone reset: writes can only be | ||
| 396 | * direct for zoned devices so concurrent writes would not add any page | ||
| 397 | * to the page cache after/during reset. The page cache may be filled | ||
| 398 | * again due to concurrent reads though and dropping the pages for | ||
| 399 | * these is fine. | ||
| 400 | */ | ||
| 401 | if (!ret && cmd == BLKRESETZONE) | ||
| 402 | ret = blkdev_truncate_zone_range(bdev, mode, &zrange); | ||
| 403 | |||
| 404 | return ret; | ||
| 371 | } | 405 | } |
| 372 | 406 | ||
| 373 | static inline unsigned long *blk_alloc_zone_bitmap(int node, | 407 | static inline unsigned long *blk_alloc_zone_bitmap(int node, |
| @@ -229,10 +229,10 @@ static struct bio *bounce_clone_bio(struct bio *bio_src) · 228 unmodified lines | |||
| 229 | * - The point of cloning the biovec is to produce a bio with a biovec | 229 | * - The point of cloning the biovec is to produce a bio with a biovec |
| 230 | * the caller can modify: bi_idx and bi_bvec_done should be 0. | 230 | * the caller can modify: bi_idx and bi_bvec_done should be 0. |
| 231 | * | 231 | * |
| 232 | * - The original bio could've had more than BIO_MAX_PAGES biovecs; if | 232 | * - The original bio could've had more than BIO_MAX_VECS biovecs; if |
| 233 | * we tried to clone the whole thing bio_alloc_bioset() would fail. | 233 | * we tried to clone the whole thing bio_alloc_bioset() would fail. |
| 234 | * But the clone should succeed as long as the number of biovecs we | 234 | * But the clone should succeed as long as the number of biovecs we |
| 235 | * actually need to allocate is fewer than BIO_MAX_PAGES. | 235 | * actually need to allocate is fewer than BIO_MAX_VECS. |
| 236 | * | 236 | * |
| 237 | * - Lastly, bi_vcnt should not be looked at or relied upon by code | 237 | * - Lastly, bi_vcnt should not be looked at or relied upon by code |
| 238 | * that does not own the bio - reason being drivers don't use it for | 238 | * that does not own the bio - reason being drivers don't use it for |
| @@ -299,7 +299,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, · 60 unmodified lines | |||
| 299 | int sectors = 0; | 299 | int sectors = 0; |
| 300 | 300 | ||
| 301 | bio_for_each_segment(from, *bio_orig, iter) { | 301 | bio_for_each_segment(from, *bio_orig, iter) { |
| 302 | if (i++ < BIO_MAX_PAGES) | 302 | if (i++ < BIO_MAX_VECS) |
| 303 | sectors += from.bv_len >> 9; | 303 | sectors += from.bv_len >> 9; |
| 304 | if (page_to_pfn(from.bv_page) > q->limits.bounce_pfn) | 304 | if (page_to_pfn(from.bv_page) > q->limits.bounce_pfn) |
| 305 | bounce = true; | 305 | bounce = true; |
| @@ -534,10 +534,8 @@ static void register_disk(struct device *parent, struct gendisk *disk, · 533 unmodified lines | |||
| 534 | kobject_create_and_add("holders", &ddev->kobj); | 534 | kobject_create_and_add("holders", &ddev->kobj); |
| 535 | disk->slave_dir = kobject_create_and_add("slaves", &ddev->kobj); | 535 | disk->slave_dir = kobject_create_and_add("slaves", &ddev->kobj); |
| 536 | 536 | ||
| 537 | if (disk->flags & GENHD_FL_HIDDEN) { | 537 | if (disk->flags & GENHD_FL_HIDDEN) |
| 538 | dev_set_uevent_suppress(ddev, 0); | ||
| 539 | return; | 538 | return; |
| 540 | } | ||
| 541 | 539 | ||
| 542 | disk_scan_partitions(disk); | 540 | disk_scan_partitions(disk); |
| 543 | 541 | ||
| @@ -767,7 +767,7 @@ config CRYPTO_POLY1305_X86_64 · 766 unmodified lines | |||
| 767 | 767 | ||
| 768 | config CRYPTO_POLY1305_MIPS | 768 | config CRYPTO_POLY1305_MIPS |
| 769 | tristate "Poly1305 authenticator algorithm (MIPS optimized)" | 769 | tristate "Poly1305 authenticator algorithm (MIPS optimized)" |
| 770 | depends on CPU_MIPS32 || (CPU_MIPS64 && 64BIT) | 770 | depends on MIPS |
| 771 | select CRYPTO_ARCH_HAVE_LIB_POLY1305 | 771 | select CRYPTO_ARCH_HAVE_LIB_POLY1305 |
| 772 | 772 | ||
| 773 | config CRYPTO_MD4 | 773 | config CRYPTO_MD4 |
| @@ -2260,7 +2260,8 @@ out: · 2259 unmodified lines | |||
| 2260 | return rc; | 2260 | return rc; |
| 2261 | 2261 | ||
| 2262 | err_eni_release: | 2262 | err_eni_release: |
| 2263 | eni_do_release(dev); | 2263 | dev->phy = NULL; |
| 2264 | iounmap(ENI_DEV(dev)->ioaddr); | ||
| 2264 | err_unregister: | 2265 | err_unregister: |
| 2265 | atm_dev_deregister(dev); | 2266 | atm_dev_deregister(dev); |
| 2266 | err_free_consistent: | 2267 | err_free_consistent: |
| @@ -262,7 +262,7 @@ static int idt77105_start(struct atm_dev *dev) · 261 unmodified lines | |||
| 262 | { | 262 | { |
| 263 | unsigned long flags; | 263 | unsigned long flags; |
| 264 | 264 | ||
| 265 | if (!(dev->dev_data = kmalloc(sizeof(struct idt77105_priv),GFP_KERNEL))) | 265 | if (!(dev->phy_data = kmalloc(sizeof(struct idt77105_priv),GFP_KERNEL))) |
| 266 | return -ENOMEM; | 266 | return -ENOMEM; |
| 267 | PRIV(dev)->dev = dev; | 267 | PRIV(dev)->dev = dev; |
| 268 | spin_lock_irqsave(&idt77105_priv_lock, flags); | 268 | spin_lock_irqsave(&idt77105_priv_lock, flags); |
| @@ -337,7 +337,7 @@ static int idt77105_stop(struct atm_dev *dev) · 68 unmodified lines | |||
| 337 | else | 337 | else |
| 338 | idt77105_all = walk->next; | 338 | idt77105_all = walk->next; |
| 339 | dev->phy = NULL; | 339 | dev->phy = NULL; |
| 340 | dev->dev_data = NULL; | 340 | dev->phy_data = NULL; |
| 341 | kfree(walk); | 341 | kfree(walk); |
| 342 | break; | 342 | break; |
| 343 | } | 343 | } |
| @@ -2233,6 +2233,7 @@ static int lanai_dev_open(struct atm_dev *atmdev) · 2232 unmodified lines | |||
| 2233 | conf1_write(lanai); | 2233 | conf1_write(lanai); |
| 2234 | #endif | 2234 | #endif |
| 2235 | iounmap(lanai->base); | 2235 | iounmap(lanai->base); |
| 2236 | lanai->base = NULL; | ||
| 2236 | error_pci: | 2237 | error_pci: |
| 2237 | pci_disable_device(lanai->pci); | 2238 | pci_disable_device(lanai->pci); |
| 2238 | error: | 2239 | error: |
| @@ -2245,6 +2246,8 @@ static int lanai_dev_open(struct atm_dev *atmdev) · 6 unmodified lines | |||
| 2245 | static void lanai_dev_close(struct atm_dev *atmdev) | 2246 | static void lanai_dev_close(struct atm_dev *atmdev) |
| 2246 | { | 2247 | { |
| 2247 | struct lanai_dev *lanai = (struct lanai_dev *) atmdev->dev_data; | 2248 | struct lanai_dev *lanai = (struct lanai_dev *) atmdev->dev_data; |
| 2249 | if (lanai->base==NULL) | ||
| 2250 | return; | ||
| 2248 | printk(KERN_INFO DEV_LABEL "(itf %d): shutting down interface\n", | 2251 | printk(KERN_INFO DEV_LABEL "(itf %d): shutting down interface\n", |
| 2249 | lanai->number); | 2252 | lanai->number); |
| 2250 | lanai_timed_poll_stop(lanai); | 2253 | lanai_timed_poll_stop(lanai); |
| @@ -2552,7 +2555,7 @@ static int lanai_init_one(struct pci_dev *pci, · 301 unmodified lines | |||
| 2552 | struct atm_dev *atmdev; | 2555 | struct atm_dev *atmdev; |
| 2553 | int result; | 2556 | int result; |
| 2554 | 2557 | ||
| 2555 | lanai = kmalloc(sizeof(*lanai), GFP_KERNEL); | 2558 | lanai = kzalloc(sizeof(*lanai), GFP_KERNEL); |
| 2556 | if (lanai == NULL) { | 2559 | if (lanai == NULL) { |
| 2557 | printk(KERN_ERR DEV_LABEL | 2560 | printk(KERN_ERR DEV_LABEL |
| 2558 | ": couldn't allocate dev_data structure!\n"); | 2561 | ": couldn't allocate dev_data structure!\n"); |
| @@ -211,7 +211,7 @@ static void uPD98402_int(struct atm_dev *dev) · 210 unmodified lines | |||
| 211 | static int uPD98402_start(struct atm_dev *dev) | 211 | static int uPD98402_start(struct atm_dev *dev) |
| 212 | { | 212 | { |
| 213 | DPRINTK("phy_start\n"); | 213 | DPRINTK("phy_start\n"); |
| 214 | if (!(dev->dev_data = kmalloc(sizeof(struct uPD98402_priv),GFP_KERNEL))) | 214 | if (!(dev->phy_data = kmalloc(sizeof(struct uPD98402_priv),GFP_KERNEL))) |
| 215 | return -ENOMEM; | 215 | return -ENOMEM; |
| 216 | spin_lock_init(&PRIV(dev)->lock); | 216 | spin_lock_init(&PRIV(dev)->lock); |
| 217 | memset(&PRIV(dev)->sonet_stats,0,sizeof(struct k_sonet_stats)); | 217 | memset(&PRIV(dev)->sonet_stats,0,sizeof(struct k_sonet_stats)); |
| @@ -938,6 +938,9 @@ int software_node_register(const struct software_node *node) · 937 unmodified lines | |||
| 938 | if (software_node_to_swnode(node)) | 938 | if (software_node_to_swnode(node)) |
| 939 | return -EEXIST; | 939 | return -EEXIST; |
| 940 | 940 | ||
| 941 | if (node->parent && !parent) | ||
| 942 | return -EINVAL; | ||
| 943 | |||
| 941 | return PTR_ERR_OR_ZERO(swnode_register(node, parent, 0)); | 944 | return PTR_ERR_OR_ZERO(swnode_register(node, parent, 0)); |
| 942 | } | 945 | } |
| 943 | EXPORT_SYMBOL_GPL(software_node_register); | 946 | EXPORT_SYMBOL_GPL(software_node_register); |
| @@ -1002,25 +1005,33 @@ EXPORT_SYMBOL_GPL(fwnode_remove_software_node); · 58 unmodified lines | |||
| 1002 | /** | 1005 | /** |
| 1003 | * device_add_software_node - Assign software node to a device | 1006 | * device_add_software_node - Assign software node to a device |
| 1004 | * @dev: The device the software node is meant for. | 1007 | * @dev: The device the software node is meant for. |
| 1005 | * @swnode: The software node. | 1008 | * @node: The software node. |
| 1006 | * | 1009 | * |
| 1007 | * This function will register @swnode and make it the secondary firmware node | 1010 | * This function will make @node the secondary firmware node pointer of @dev. If |
| 1008 | * pointer of @dev. If @dev has no primary node, then @swnode will become the primary | 1011 | * @dev has no primary node, then @node will become the primary node. The |
| 1009 | * node. | 1012 | * function will register @node automatically if it wasn't already registered. |
| 1010 | */ | 1013 | */ |
| 1011 | int device_add_software_node(struct device *dev, const struct software_node *swnode) | 1014 | int device_add_software_node(struct device *dev, const struct software_node *node) |
| 1012 | { | 1015 | { |
| 1016 | struct swnode *swnode; | ||
| 1013 | int ret; | 1017 | int ret; |
| 1014 | 1018 | ||
| 1015 | /* Only one software node per device. */ | 1019 | /* Only one software node per device. */ |
| 1016 | if (dev_to_swnode(dev)) | 1020 | if (dev_to_swnode(dev)) |
| 1017 | return -EBUSY; | 1021 | return -EBUSY; |
| 1018 | 1022 | ||
| 1019 | ret = software_node_register(swnode); | 1023 | swnode = software_node_to_swnode(node); |
| 1020 | if (ret) | 1024 | if (swnode) { |
| 1021 | return ret; | 1025 | kobject_get(&swnode->kobj); |
| 1026 | } else { | ||
| 1027 | ret = software_node_register(node); | ||
| 1028 | if (ret) | ||
| 1029 | return ret; | ||
| 1030 | |||
| 1031 | swnode = software_node_to_swnode(node); | ||
| 1032 | } | ||
| 1022 | 1033 | ||
| 1023 | set_secondary_fwnode(dev, software_node_fwnode(swnode)); | 1034 | set_secondary_fwnode(dev, &swnode->fwnode); |
| 1024 | 1035 | ||
| 1025 | return 0; | 1036 | return 0; |
| 1026 | } | 1037 | } |
| @@ -1324,7 +1324,7 @@ struct bm_extent { · 1323 unmodified lines | |||
| 1324 | * A followup commit may allow even bigger BIO sizes, | 1324 | * A followup commit may allow even bigger BIO sizes, |
| 1325 | * once we thought that through. */ | 1325 | * once we thought that through. */ |
| 1326 | #define DRBD_MAX_BIO_SIZE (1U << 20) | 1326 | #define DRBD_MAX_BIO_SIZE (1U << 20) |
| 1327 | #if DRBD_MAX_BIO_SIZE > (BIO_MAX_PAGES << PAGE_SHIFT) | 1327 | #if DRBD_MAX_BIO_SIZE > (BIO_MAX_VECS << PAGE_SHIFT) |
| 1328 | #error Architecture not supported: DRBD_MAX_BIO_SIZE > BIO_MAX_SIZE | 1328 | #error Architecture not supported: DRBD_MAX_BIO_SIZE > BIO_MAX_SIZE |
| 1329 | #endif | 1329 | #endif |
| 1330 | #define DRBD_MAX_BIO_SIZE_SAFE (1U << 12) /* Works always = 4k */ | 1330 | #define DRBD_MAX_BIO_SIZE_SAFE (1U << 12) /* Works always = 4k */ |
| @@ -871,6 +871,7 @@ static int rsxx_pci_probe(struct pci_dev *dev, · 870 unmodified lines | |||
| 871 | card->event_wq = create_singlethread_workqueue(DRIVER_NAME"_event"); | 871 | card->event_wq = create_singlethread_workqueue(DRIVER_NAME"_event"); |
| 872 | if (!card->event_wq) { | 872 | if (!card->event_wq) { |
| 873 | dev_err(CARD_TO_DEV(card), "Failed card event setup.\n"); | 873 | dev_err(CARD_TO_DEV(card), "Failed card event setup.\n"); |
| 874 | st = -ENOMEM; | ||
| 874 | goto failed_event_handler; | 875 | goto failed_event_handler; |
| 875 | } | 876 | } |
| 876 | 877 | ||
| @@ -877,6 +877,7 @@ static int mm_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) · 876 unmodified lines | |||
| 877 | if (card->mm_pages[0].desc == NULL || | 877 | if (card->mm_pages[0].desc == NULL || |
| 878 | card->mm_pages[1].desc == NULL) { | 878 | card->mm_pages[1].desc == NULL) { |
| 879 | dev_printk(KERN_ERR, &card->dev->dev, "alloc failed\n"); | 879 | dev_printk(KERN_ERR, &card->dev->dev, "alloc failed\n"); |
| 880 | ret = -ENOMEM; | ||
| 880 | goto failed_alloc; | 881 | goto failed_alloc; |
| 881 | } | 882 | } |
| 882 | reset_page(&card->mm_pages[0]); | 883 | reset_page(&card->mm_pages[0]); |
| @@ -888,8 +889,10 @@ static int mm_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) · 5 unmodified lines | |||
| 888 | spin_lock_init(&card->lock); | 889 | spin_lock_init(&card->lock); |
| 889 | 890 | ||
| 890 | card->queue = blk_alloc_queue(NUMA_NO_NODE); | 891 | card->queue = blk_alloc_queue(NUMA_NO_NODE); |
| 891 | if (!card->queue) | 892 | if (!card->queue) { |
| 893 | ret = -ENOMEM; | ||
| 892 | goto failed_alloc; | 894 | goto failed_alloc; |
| 895 | } | ||
| 893 | 896 | ||
| 894 | tasklet_init(&card->tasklet, process_page, (unsigned long)card); | 897 | tasklet_init(&card->tasklet, process_page, (unsigned long)card); |
| 895 | 898 | ||
| @@ -627,7 +627,7 @@ static ssize_t writeback_store(struct device *dev, · 626 unmodified lines | |||
| 627 | struct bio_vec bio_vec; | 627 | struct bio_vec bio_vec; |
| 628 | struct page *page; | 628 | struct page *page; |
| 629 | ssize_t ret = len; | 629 | ssize_t ret = len; |
| 630 | int mode; | 630 | int mode, err; |
| 631 | unsigned long blk_idx = 0; | 631 | unsigned long blk_idx = 0; |
| 632 | 632 | ||
| 633 | if (sysfs_streq(buf, "idle")) | 633 | if (sysfs_streq(buf, "idle")) |
| @@ -638,8 +638,8 @@ static ssize_t writeback_store(struct device *dev, · 4 unmodified lines | |||
| 638 | if (strncmp(buf, PAGE_WB_SIG, sizeof(PAGE_WB_SIG) - 1)) | 638 | if (strncmp(buf, PAGE_WB_SIG, sizeof(PAGE_WB_SIG) - 1)) |
| 639 | return -EINVAL; | 639 | return -EINVAL; |
| 640 | 640 | ||
| 641 | ret = kstrtol(buf + sizeof(PAGE_WB_SIG) - 1, 10, &index); | 641 | if (kstrtol(buf + sizeof(PAGE_WB_SIG) - 1, 10, &index) || |
| 642 | if (ret || index >= nr_pages) | 642 | index >= nr_pages) |
| 643 | return -EINVAL; | 643 | return -EINVAL; |
| 644 | 644 | ||
| 645 | nr_pages = 1; | 645 | nr_pages = 1; |
| @@ -663,7 +663,7 @@ static ssize_t writeback_store(struct device *dev, · 17 unmodified lines | |||
| 663 | goto release_init_lock; | 663 | goto release_init_lock; |
| 664 | } | 664 | } |
| 665 | 665 | ||
| 666 | while (nr_pages--) { | 666 | for (; nr_pages != 0; index++, nr_pages--) { |
| 667 | struct bio_vec bvec; | 667 | struct bio_vec bvec; |
| 668 | 668 | ||
| 669 | bvec.bv_page = page; | 669 | bvec.bv_page = page; |
| @@ -728,12 +728,17 @@ static ssize_t writeback_store(struct device *dev, · 58 unmodified lines | |||
| 728 | * XXX: A single page IO would be inefficient for write | 728 | * XXX: A single page IO would be inefficient for write |
| 729 | * but it would be not bad as starter. | 729 | * but it would be not bad as starter. |
| 730 | */ | 730 | */ |
| 731 | ret = submit_bio_wait(&bio); | 731 | err = submit_bio_wait(&bio); |
| 732 | if (ret) { | 732 | if (err) { |
| 733 | zram_slot_lock(zram, index); | 733 | zram_slot_lock(zram, index); |
| 734 | zram_clear_flag(zram, index, ZRAM_UNDER_WB); | 734 | zram_clear_flag(zram, index, ZRAM_UNDER_WB); |
| 735 | zram_clear_flag(zram, index, ZRAM_IDLE); | 735 | zram_clear_flag(zram, index, ZRAM_IDLE); |
| 736 | zram_slot_unlock(zram, index); | 736 | zram_slot_unlock(zram, index); |
| 737 | /* | ||
| 738 | * Return last IO error unless every IO were | ||
| 739 | * not suceeded. | ||
| 740 | */ | ||
| 741 | ret = err; | ||
| 737 | continue; | 742 | continue; |
| 738 | } | 743 | } |
| 739 | 744 | ||
| @@ -54,10 +54,9 @@ static int pseries_rng_probe(struct vio_dev *dev, · 53 unmodified lines | |||
| 54 | return hwrng_register(&pseries_rng); | 54 | return hwrng_register(&pseries_rng); |
| 55 | } | 55 | } |
| 56 | 56 | ||
| 57 | static int pseries_rng_remove(struct vio_dev *dev) | 57 | static void pseries_rng_remove(struct vio_dev *dev) |
| 58 | { | 58 | { |
| 59 | hwrng_unregister(&pseries_rng); | 59 | hwrng_unregister(&pseries_rng); |
| 60 | return 0; | ||
| 61 | } | 60 | } |
| 62 | 61 | ||
| 63 | static const struct vio_device_id pseries_rng_driver_ids[] = { | 62 | static const struct vio_device_id pseries_rng_driver_ids[] = { |
| @@ -343,7 +343,7 @@ static int ibmvtpm_crq_send_init_complete(struct ibmvtpm_dev *ibmvtpm) · 342 unmodified lines | |||
| 343 | * | 343 | * |
| 344 | * Return: Always 0. | 344 | * Return: Always 0. |
| 345 | */ | 345 | */ |
| 346 | static int tpm_ibmvtpm_remove(struct vio_dev *vdev) | 346 | static void tpm_ibmvtpm_remove(struct vio_dev *vdev) |
| 347 | { | 347 | { |
| 348 | struct tpm_chip *chip = dev_get_drvdata(&vdev->dev); | 348 | struct tpm_chip *chip = dev_get_drvdata(&vdev->dev); |
| 349 | struct ibmvtpm_dev *ibmvtpm = dev_get_drvdata(&chip->dev); | 349 | struct ibmvtpm_dev *ibmvtpm = dev_get_drvdata(&chip->dev); |
| @@ -372,8 +372,6 @@ static int tpm_ibmvtpm_remove(struct vio_dev *vdev) · 22 unmodified lines | |||
| 372 | kfree(ibmvtpm); | 372 | kfree(ibmvtpm); |
| 373 | /* For tpm_ibmvtpm_get_desired_dma */ | 373 | /* For tpm_ibmvtpm_get_desired_dma */ |
| 374 | dev_set_drvdata(&vdev->dev, NULL); | 374 | dev_set_drvdata(&vdev->dev, NULL); |
| 375 | |||
| 376 | return 0; | ||
| 377 | } | 375 | } |
| 378 | 376 | ||
| 379 | /** | 377 | /** |
| @@ -103,6 +103,8 @@ static const struct of_device_id whitelist[] __initconst = { · 102 unmodified lines | |||
| 103 | static const struct of_device_id blacklist[] __initconst = { | 103 | static const struct of_device_id blacklist[] __initconst = { |
| 104 | { .compatible = "allwinner,sun50i-h6", }, | 104 | { .compatible = "allwinner,sun50i-h6", }, |
| 105 | 105 | ||
| 106 | { .compatible = "arm,vexpress", }, | ||
| 107 | |||
| 106 | { .compatible = "calxeda,highbank", }, | 108 | { .compatible = "calxeda,highbank", }, |
| 107 | { .compatible = "calxeda,ecx-2000", }, | 109 | { .compatible = "calxeda,ecx-2000", }, |
| 108 | 110 | ||
| @@ -317,9 +317,9 @@ static int qcom_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) · 316 unmodified lines | |||
| 317 | } | 317 | } |
| 318 | 318 | ||
| 319 | base = ioremap(res->start, resource_size(res)); | 319 | base = ioremap(res->start, resource_size(res)); |
| 320 | if (IS_ERR(base)) { | 320 | if (!base) { |
| 321 | dev_err(dev, "failed to map resource %pR\n", res); | 321 | dev_err(dev, "failed to map resource %pR\n", res); |
| 322 | ret = PTR_ERR(base); | 322 | ret = -ENOMEM; |
| 323 | goto release_region; | 323 | goto release_region; |
| 324 | } | 324 | } |
| 325 | 325 | ||
| @@ -374,7 +374,7 @@ static int qcom_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) · 48 unmodified lines | |||
| 374 | error: | 374 | error: |
| 375 | kfree(data); | 375 | kfree(data); |
| 376 | unmap_base: | 376 | unmap_base: |
| 377 | iounmap(data->base); | 377 | iounmap(base); |
| 378 | release_region: | 378 | release_region: |
| 379 | release_mem_region(res->start, resource_size(res)); | 379 | release_mem_region(res->start, resource_size(res)); |
| 380 | return ret; | 380 | return ret; |
| @@ -1042,7 +1042,7 @@ error: · 1041 unmodified lines | |||
| 1042 | return ret; | 1042 | return ret; |
| 1043 | } | 1043 | } |
| 1044 | 1044 | ||
| 1045 | static int nx842_remove(struct vio_dev *viodev) | 1045 | static void nx842_remove(struct vio_dev *viodev) |
| 1046 | { | 1046 | { |
| 1047 | struct nx842_devdata *old_devdata; | 1047 | struct nx842_devdata *old_devdata; |
| 1048 | unsigned long flags; | 1048 | unsigned long flags; |
| @@ -1063,8 +1063,6 @@ static int nx842_remove(struct vio_dev *viodev) · 14 unmodified lines | |||
| 1063 | if (old_devdata) | 1063 | if (old_devdata) |
| 1064 | kfree(old_devdata->counters); | 1064 | kfree(old_devdata->counters); |
| 1065 | kfree(old_devdata); | 1065 | kfree(old_devdata); |
| 1066 | |||
| 1067 | return 0; | ||
| 1068 | } | 1066 | } |
| 1069 | 1067 | ||
| 1070 | static const struct vio_device_id nx842_vio_driver_ids[] = { | 1068 | static const struct vio_device_id nx842_vio_driver_ids[] = { |
| @@ -783,7 +783,7 @@ static int nx_probe(struct vio_dev *viodev, const struct vio_device_id *id) · 782 unmodified lines | |||
| 783 | return nx_register_algs(); | 783 | return nx_register_algs(); |
| 784 | } | 784 | } |
| 785 | 785 | ||
| 786 | static int nx_remove(struct vio_dev *viodev) | 786 | static void nx_remove(struct vio_dev *viodev) |
| 787 | { | 787 | { |
| 788 | dev_dbg(&viodev->dev, "entering nx_remove for UA 0x%x\n", | 788 | dev_dbg(&viodev->dev, "entering nx_remove for UA 0x%x\n", |
| 789 | viodev->unit_address); | 789 | viodev->unit_address); |
| @@ -811,8 +811,6 @@ static int nx_remove(struct vio_dev *viodev) · 21 unmodified lines | |||
| 811 | nx_unregister_skcipher(&nx_ecb_aes_alg, NX_FC_AES, | 811 | nx_unregister_skcipher(&nx_ecb_aes_alg, NX_FC_AES, |
| 812 | NX_MODE_AES_ECB); | 812 | NX_MODE_AES_ECB); |
| 813 | } | 813 | } |
| 814 | |||
| 815 | return 0; | ||
| 816 | } | 814 | } |
| 817 | 815 | ||
| 818 | 816 | ||
| @@ -24,7 +24,7 @@ efi_status_t check_platform_features(void) · 23 unmodified lines | |||
| 24 | return EFI_SUCCESS; | 24 | return EFI_SUCCESS; |
| 25 | 25 | ||
| 26 | tg = (read_cpuid(ID_AA64MMFR0_EL1) >> ID_AA64MMFR0_TGRAN_SHIFT) & 0xf; | 26 | tg = (read_cpuid(ID_AA64MMFR0_EL1) >> ID_AA64MMFR0_TGRAN_SHIFT) & 0xf; |
| 27 | if (tg != ID_AA64MMFR0_TGRAN_SUPPORTED) { | 27 | if (tg < ID_AA64MMFR0_TGRAN_SUPPORTED_MIN || tg > ID_AA64MMFR0_TGRAN_SUPPORTED_MAX) { |
| 28 | if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) | 28 | if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) |
| 29 | efi_err("This 64 KB granular kernel is not supported by your CPU\n"); | 29 | efi_err("This 64 KB granular kernel is not supported by your CPU\n"); |
| 30 | else | 30 | else |
| @@ -96,6 +96,18 @@ static void install_memreserve_table(void) · 95 unmodified lines | |||
| 96 | efi_err("Failed to install memreserve config table!\n"); | 96 | efi_err("Failed to install memreserve config table!\n"); |
| 97 | } | 97 | } |
| 98 | 98 | ||
| 99 | static u32 get_supported_rt_services(void) | ||
| 100 | { | ||
| 101 | const efi_rt_properties_table_t *rt_prop_table; | ||
| 102 | u32 supported = EFI_RT_SUPPORTED_ALL; | ||
| 103 | |||
| 104 | rt_prop_table = get_efi_config_table(EFI_RT_PROPERTIES_TABLE_GUID); | ||
| 105 | if (rt_prop_table) | ||
| 106 | supported &= rt_prop_table->runtime_services_supported; | ||
| 107 | |||
| 108 | return supported; | ||
| 109 | } | ||
| 110 | |||
| 99 | /* | 111 | /* |
| 100 | * EFI entry point for the arm/arm64 EFI stubs. This is the entrypoint | 112 | * EFI entry point for the arm/arm64 EFI stubs. This is the entrypoint |
| 101 | * that is described in the PE/COFF header. Most of the code is the same | 113 | * that is described in the PE/COFF header. Most of the code is the same |
| @@ -250,6 +262,10 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, · 148 unmodified lines | |||
| 250 | (prop_tbl->memory_protection_attribute & | 262 | (prop_tbl->memory_protection_attribute & |
| 251 | EFI_PROPERTIES_RUNTIME_MEMORY_PROTECTION_NON_EXECUTABLE_PE_DATA); | 263 | EFI_PROPERTIES_RUNTIME_MEMORY_PROTECTION_NON_EXECUTABLE_PE_DATA); |
| 252 | 264 | ||
| 265 | /* force efi_novamap if SetVirtualAddressMap() is unsupported */ | ||
| 266 | efi_novamap |= !(get_supported_rt_services() & | ||
| 267 | EFI_RT_SUPPORTED_SET_VIRTUAL_ADDRESS_MAP); | ||
| 268 | |||
| 253 | /* hibernation expects the runtime regions to stay in the same place */ | 269 | /* hibernation expects the runtime regions to stay in the same place */ |
| 254 | if (!IS_ENABLED(CONFIG_HIBERNATION) && !efi_nokaslr && !flat_va_mapping) { | 270 | if (!IS_ENABLED(CONFIG_HIBERNATION) && !efi_nokaslr && !flat_va_mapping) { |
| 255 | /* | 271 | /* |
| @@ -113,8 +113,29 @@ MODULE_DEVICE_TABLE(i2c, pca953x_id); · 112 unmodified lines | |||
| 113 | #ifdef CONFIG_GPIO_PCA953X_IRQ | 113 | #ifdef CONFIG_GPIO_PCA953X_IRQ |
| 114 | 114 | ||
| 115 | #include <linux/dmi.h> | 115 | #include <linux/dmi.h> |
| 116 | #include <linux/gpio.h> | 116 | |
| 117 | #include <linux/list.h> | 117 | static const struct acpi_gpio_params pca953x_irq_gpios = { 0, 0, true }; |
| 118 | |||
| 119 | static const struct acpi_gpio_mapping pca953x_acpi_irq_gpios[] = { | ||
| 120 | { "irq-gpios", &pca953x_irq_gpios, 1, ACPI_GPIO_QUIRK_ABSOLUTE_NUMBER }, | ||
| 121 | { } | ||
| 122 | }; | ||
| 123 | |||
| 124 | static int pca953x_acpi_get_irq(struct device *dev) | ||
| 125 | { | ||
| 126 | int ret; | ||
| 127 | |||
| 128 | ret = devm_acpi_dev_add_driver_gpios(dev, pca953x_acpi_irq_gpios); | ||
| 129 | if (ret) | ||
| 130 | dev_warn(dev, "can't add GPIO ACPI mapping\n"); | ||
| 131 | |||
| 132 | ret = acpi_dev_gpio_irq_get_by(ACPI_COMPANION(dev), "irq-gpios", 0); | ||
| 133 | if (ret < 0) | ||
| 134 | return ret; | ||
| 135 | |||
| 136 | dev_info(dev, "ACPI interrupt quirk (IRQ %d)\n", ret); | ||
| 137 | return ret; | ||
| 138 | } | ||
| 118 | 139 | ||
| 119 | static const struct dmi_system_id pca953x_dmi_acpi_irq_info[] = { | 140 | static const struct dmi_system_id pca953x_dmi_acpi_irq_info[] = { |
| 120 | { | 141 | { |
| @@ -133,59 +154,6 @@ static const struct dmi_system_id pca953x_dmi_acpi_irq_info[] = { · 12 unmodified lines | |||
| 133 | }, | 154 | }, |
| 134 | {} | 155 | {} |
| 135 | }; | 156 | }; |
| 136 | |||
| 137 | #ifdef CONFIG_ACPI | ||
| 138 | static int pca953x_acpi_get_pin(struct acpi_resource *ares, void *data) | ||
| 139 | { | ||
| 140 | struct acpi_resource_gpio *agpio; | ||
| 141 | int *pin = data; | ||
| 142 | |||
| 143 | if (acpi_gpio_get_irq_resource(ares, &agpio)) | ||
| 144 | *pin = agpio->pin_table[0]; | ||
| 145 | return 1; | ||
| 146 | } | ||
| 147 | |||
| 148 | static int pca953x_acpi_find_pin(struct device *dev) | ||
| 149 | { | ||
| 150 | struct acpi_device *adev = ACPI_COMPANION(dev); | ||
| 151 | int pin = -ENOENT, ret; | ||
| 152 | LIST_HEAD(r); | ||
| 153 | |||
| 154 | ret = acpi_dev_get_resources(adev, &r, pca953x_acpi_get_pin, &pin); | ||
| 155 | acpi_dev_free_resource_list(&r); | ||
| 156 | if (ret < 0) | ||
| 157 | return ret; | ||
| 158 | |||
| 159 | return pin; | ||
| 160 | } | ||
| 161 | #else | ||
| 162 | static inline int pca953x_acpi_find_pin(struct device *dev) { return -ENXIO; } | ||
| 163 | #endif | ||
| 164 | |||
| 165 | static int pca953x_acpi_get_irq(struct device *dev) | ||
| 166 | { | ||
| 167 | int pin, ret; | ||
| 168 | |||
| 169 | pin = pca953x_acpi_find_pin(dev); | ||
| 170 | if (pin < 0) | ||
| 171 | return pin; | ||
| 172 | |||
| 173 | dev_info(dev, "Applying ACPI interrupt quirk (GPIO %d)\n", pin); | ||
| 174 | |||
| 175 | if (!gpio_is_valid(pin)) | ||
| 176 | return -EINVAL; | ||
| 177 | |||
| 178 | ret = gpio_request(pin, "pca953x interrupt"); | ||
| 179 | if (ret) | ||
| 180 | return ret; | ||
| 181 | |||
| 182 | ret = gpio_to_irq(pin); | ||
| 183 | |||
| 184 | /* When pin is used as an IRQ, no need to keep it requested */ | ||
| 185 | gpio_free(pin); | ||
| 186 | |||
| 187 | return ret; | ||
| 188 | } | ||
| 189 | #endif | 157 | #endif |
| 190 | 158 | ||
| 191 | static const struct acpi_device_id pca953x_acpi_ids[] = { | 159 | static const struct acpi_device_id pca953x_acpi_ids[] = { |
| @@ -174,7 +174,7 @@ static void acpi_gpiochip_request_irq(struct acpi_gpio_chip *acpi_gpio, · 173 unmodified lines | |||
| 174 | int ret, value; | 174 | int ret, value; |
| 175 | 175 | ||
| 176 | ret = request_threaded_irq(event->irq, NULL, event->handler, | 176 | ret = request_threaded_irq(event->irq, NULL, event->handler, |
| 177 | event->irqflags, "ACPI:Event", event); | 177 | event->irqflags | IRQF_ONESHOT, "ACPI:Event", event); |
| 178 | if (ret) { | 178 | if (ret) { |
| 179 | dev_err(acpi_gpio->chip->parent, | 179 | dev_err(acpi_gpio->chip->parent, |
| 180 | "Failed to setup interrupt handler for %d\n", | 180 | "Failed to setup interrupt handler for %d\n", |
| @@ -677,6 +677,7 @@ static int acpi_populate_gpio_lookup(struct acpi_resource *ares, void *data) · 496 unmodified lines | |||
| 677 | if (!lookup->desc) { | 677 | if (!lookup->desc) { |
| 678 | const struct acpi_resource_gpio *agpio = &ares->data.gpio; | 678 | const struct acpi_resource_gpio *agpio = &ares->data.gpio; |
| 679 | bool gpioint = agpio->connection_type == ACPI_RESOURCE_GPIO_TYPE_INT; | 679 | bool gpioint = agpio->connection_type == ACPI_RESOURCE_GPIO_TYPE_INT; |
| 680 | struct gpio_desc *desc; | ||
| 680 | u16 pin_index; | 681 | u16 pin_index; |
| 681 | 682 | ||
| 682 | if (lookup->info.quirks & ACPI_GPIO_QUIRK_ONLY_GPIOIO && gpioint) | 683 | if (lookup->info.quirks & ACPI_GPIO_QUIRK_ONLY_GPIOIO && gpioint) |
| @@ -689,8 +690,12 @@ static int acpi_populate_gpio_lookup(struct acpi_resource *ares, void *data) · 6 unmodified lines | |||
| 689 | if (pin_index >= agpio->pin_table_length) | 690 | if (pin_index >= agpio->pin_table_length) |
| 690 | return 1; | 691 | return 1; |
| 691 | 692 | ||
| 692 | lookup->desc = acpi_get_gpiod(agpio->resource_source.string_ptr, | 693 | if (lookup->info.quirks & ACPI_GPIO_QUIRK_ABSOLUTE_NUMBER) |
| 694 | desc = gpio_to_desc(agpio->pin_table[pin_index]); | ||
| 695 | else | ||
| 696 | desc = acpi_get_gpiod(agpio->resource_source.string_ptr, | ||
| 693 | agpio->pin_table[pin_index]); | 697 | agpio->pin_table[pin_index]); |
| 698 | lookup->desc = desc; | ||
| 694 | lookup->info.pin_config = agpio->pin_config; | 699 | lookup->info.pin_config = agpio->pin_config; |
| 695 | lookup->info.debounce = agpio->debounce_timeout; | 700 | lookup->info.debounce = agpio->debounce_timeout; |
| 696 | lookup->info.gpioint = gpioint; | 701 | lookup->info.gpioint = gpioint; |
| @@ -940,8 +945,9 @@ struct gpio_desc *acpi_node_get_gpiod(struct fwnode_handle *fwnode, · 243 unmodified lines | |||
| 940 | } | 945 | } |
| 941 | 946 | ||
| 942 | /** | 947 | /** |
| 943 | * acpi_dev_gpio_irq_get() - Find GpioInt and translate it to Linux IRQ number | 948 | * acpi_dev_gpio_irq_get_by() - Find GpioInt and translate it to Linux IRQ number |
| 944 | * @adev: pointer to a ACPI device to get IRQ from | 949 | * @adev: pointer to a ACPI device to get IRQ from |
| 950 | * @name: optional name of GpioInt resource | ||
| 945 | * @index: index of GpioInt resource (starting from %0) | 951 | * @index: index of GpioInt resource (starting from %0) |
| 946 | * | 952 | * |
| 947 | * If the device has one or more GpioInt resources, this function can be | 953 | * If the device has one or more GpioInt resources, this function can be |
| @@ -951,9 +957,12 @@ struct gpio_desc *acpi_node_get_gpiod(struct fwnode_handle *fwnode, · 3 unmodified lines | |||
| 951 | * The function is idempotent, though each time it runs it will configure GPIO | 957 | * The function is idempotent, though each time it runs it will configure GPIO |
| 952 | * pin direction according to the flags in GpioInt resource. | 958 | * pin direction according to the flags in GpioInt resource. |
| 953 | * | 959 | * |
| 960 | * The function takes optional @name parameter. If the resource has a property | ||
| 961 | * name, then only those will be taken into account. | ||
| 962 | * | ||
| 954 | * Return: Linux IRQ number (> %0) on success, negative errno on failure. | 963 | * Return: Linux IRQ number (> %0) on success, negative errno on failure. |
| 955 | */ | 964 | */ |
| 956 | int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index) | 965 | int acpi_dev_gpio_irq_get_by(struct acpi_device *adev, const char *name, int index) |
| 957 | { | 966 | { |
| 958 | int idx, i; | 967 | int idx, i; |
| 959 | unsigned int irq_flags; | 968 | unsigned int irq_flags; |
| @@ -963,7 +972,7 @@ int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index) · 3 unmodified lines | |||
| 963 | struct acpi_gpio_info info; | 972 | struct acpi_gpio_info info; |
| 964 | struct gpio_desc *desc; | 973 | struct gpio_desc *desc; |
| 965 | 974 | ||
| 966 | desc = acpi_get_gpiod_by_index(adev, NULL, i, &info); | 975 | desc = acpi_get_gpiod_by_index(adev, name, i, &info); |
| 967 | 976 | ||
| 968 | /* Ignore -EPROBE_DEFER, it only matters if idx matches */ | 977 | /* Ignore -EPROBE_DEFER, it only matters if idx matches */ |
| 969 | if (IS_ERR(desc) && PTR_ERR(desc) != -EPROBE_DEFER) | 978 | if (IS_ERR(desc) && PTR_ERR(desc) != -EPROBE_DEFER) |
| @@ -1008,7 +1017,7 @@ int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index) · 38 unmodified lines | |||
| 1008 | } | 1017 | } |
| 1009 | return -ENOENT; | 1018 | return -ENOENT; |
| 1010 | } | 1019 | } |
| 1011 | EXPORT_SYMBOL_GPL(acpi_dev_gpio_irq_get); | 1020 | EXPORT_SYMBOL_GPL(acpi_dev_gpio_irq_get_by); |
| 1012 | 1021 | ||
| 1013 | static acpi_status | 1022 | static acpi_status |
| 1014 | acpi_gpio_adr_space_handler(u32 function, acpi_physical_address address, | 1023 | acpi_gpio_adr_space_handler(u32 function, acpi_physical_address address, |
| @@ -367,22 +367,18 @@ static int gpiochip_set_desc_names(struct gpio_chip *gc) · 366 unmodified lines | |||
| 367 | * | 367 | * |
| 368 | * Looks for device property "gpio-line-names" and if it exists assigns | 368 | * Looks for device property "gpio-line-names" and if it exists assigns |
| 369 | * GPIO line names for the chip. The memory allocated for the assigned | 369 | * GPIO line names for the chip. The memory allocated for the assigned |
| 370 | * names belong to the underlying software node and should not be released | 370 | * names belong to the underlying firmware node and should not be released |
| 371 | * by the caller. | 371 | * by the caller. |
| 372 | */ | 372 | */ |
| 373 | static int devprop_gpiochip_set_names(struct gpio_chip *chip) | 373 | static int devprop_gpiochip_set_names(struct gpio_chip *chip) |
| 374 | { | 374 | { |
| 375 | struct gpio_device *gdev = chip->gpiodev; | 375 | struct gpio_device *gdev = chip->gpiodev; |
| 376 | struct device *dev = chip->parent; | 376 | struct fwnode_handle *fwnode = dev_fwnode(&gdev->dev); |
| 377 | const char **names; | 377 | const char **names; |
| 378 | int ret, i; | 378 | int ret, i; |
| 379 | int count; | 379 | int count; |
| 380 | 380 | ||
| 381 | /* GPIO chip may not have a parent device whose properties we inspect. */ | 381 | count = fwnode_property_string_array_count(fwnode, "gpio-line-names"); |
| 382 | if (!dev) | ||
| 383 | return 0; | ||
| 384 | |||
| 385 | count = device_property_string_array_count(dev, "gpio-line-names"); | ||
| 386 | if (count < 0) | 382 | if (count < 0) |
| 387 | return 0; | 383 | return 0; |
| 388 | 384 | ||
| @@ -396,7 +392,7 @@ static int devprop_gpiochip_set_names(struct gpio_chip *chip) · 7 unmodified lines | |||
| 396 | if (!names) | 392 | if (!names) |
| 397 | return -ENOMEM; | 393 | return -ENOMEM; |
| 398 | 394 | ||
| 399 | ret = device_property_read_string_array(dev, "gpio-line-names", | 395 | ret = fwnode_property_read_string_array(fwnode, "gpio-line-names", |
| 400 | names, count); | 396 | names, count); |
| 401 | if (ret < 0) { | 397 | if (ret < 0) { |
| 402 | dev_warn(&gdev->dev, "failed to read GPIO line names\n"); | 398 | dev_warn(&gdev->dev, "failed to read GPIO line names\n"); |
| @@ -474,9 +470,13 @@ EXPORT_SYMBOL_GPL(gpiochip_line_is_valid); · 71 unmodified lines | |||
| 474 | 470 | ||
| 475 | static void gpiodevice_release(struct device *dev) | 471 | static void gpiodevice_release(struct device *dev) |
| 476 | { | 472 | { |
| 477 | struct gpio_device *gdev = dev_get_drvdata(dev); | 473 | struct gpio_device *gdev = container_of(dev, struct gpio_device, dev); |
| 474 | unsigned long flags; | ||
| 478 | 475 | ||
| 476 | spin_lock_irqsave(&gpio_lock, flags); | ||
| 479 | list_del(&gdev->list); | 477 | list_del(&gdev->list); |
| 478 | spin_unlock_irqrestore(&gpio_lock, flags); | ||
| 479 | |||
| 480 | ida_free(&gpio_ida, gdev->id); | 480 | ida_free(&gpio_ida, gdev->id); |
| 481 | kfree_const(gdev->label); | 481 | kfree_const(gdev->label); |
| 482 | kfree(gdev->descs); | 482 | kfree(gdev->descs); |
| @@ -605,7 +605,6 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data, · 122 unmodified lines | |||
| 605 | goto err_free_ida; | 605 | goto err_free_ida; |
| 606 | 606 | ||
| 607 | device_initialize(&gdev->dev); | 607 | device_initialize(&gdev->dev); |
| 608 | dev_set_drvdata(&gdev->dev, gdev); | ||
| 609 | if (gc->parent && gc->parent->driver) | 608 | if (gc->parent && gc->parent->driver) |
| 610 | gdev->owner = gc->parent->driver->owner; | 609 | gdev->owner = gc->parent->driver->owner; |
| 611 | else if (gc->owner) | 610 | else if (gc->owner) |
| @@ -228,6 +228,7 @@ source "drivers/gpu/drm/arm/Kconfig" · 227 unmodified lines | |||
| 228 | config DRM_RADEON | 228 | config DRM_RADEON |
| 229 | tristate "ATI Radeon" | 229 | tristate "ATI Radeon" |
| 230 | depends on DRM && PCI && MMU | 230 | depends on DRM && PCI && MMU |
| 231 | depends on AGP || !AGP | ||
| 231 | select FW_LOADER | 232 | select FW_LOADER |
| 232 | select DRM_KMS_HELPER | 233 | select DRM_KMS_HELPER |
| 233 | select DRM_TTM | 234 | select DRM_TTM |
| @@ -180,6 +180,7 @@ extern uint amdgpu_smu_memory_pool_size; · 179 unmodified lines | |||
| 180 | extern uint amdgpu_dc_feature_mask; | 180 | extern uint amdgpu_dc_feature_mask; |
| 181 | extern uint amdgpu_dc_debug_mask; | 181 | extern uint amdgpu_dc_debug_mask; |
| 182 | extern uint amdgpu_dm_abm_level; | 182 | extern uint amdgpu_dm_abm_level; |
| 183 | extern int amdgpu_backlight; | ||
| 183 | extern struct amdgpu_mgpu_info mgpu_info; | 184 | extern struct amdgpu_mgpu_info mgpu_info; |
| 184 | extern int amdgpu_ras_enable; | 185 | extern int amdgpu_ras_enable; |
| 185 | extern uint amdgpu_ras_mask; | 186 | extern uint amdgpu_ras_mask; |
| @@ -903,7 +903,7 @@ void amdgpu_acpi_fini(struct amdgpu_device *adev) · 902 unmodified lines | |||
| 903 | */ | 903 | */ |
| 904 | bool amdgpu_acpi_is_s0ix_supported(struct amdgpu_device *adev) | 904 | bool amdgpu_acpi_is_s0ix_supported(struct amdgpu_device *adev) |
| 905 | { | 905 | { |
| 906 | #if defined(CONFIG_AMD_PMC) | 906 | #if defined(CONFIG_AMD_PMC) || defined(CONFIG_AMD_PMC_MODULE) |
| 907 | if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) { | 907 | if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) { |
| 908 | if (adev->flags & AMD_IS_APU) | 908 | if (adev->flags & AMD_IS_APU) |
| 909 | return true; | 909 | return true; |
| @@ -781,6 +781,10 @@ uint amdgpu_dm_abm_level; · 780 unmodified lines | |||
| 781 | MODULE_PARM_DESC(abmlevel, "ABM level (0 = off (default), 1-4 = backlight reduction level) "); | 781 | MODULE_PARM_DESC(abmlevel, "ABM level (0 = off (default), 1-4 = backlight reduction level) "); |
| 782 | module_param_named(abmlevel, amdgpu_dm_abm_level, uint, 0444); | 782 | module_param_named(abmlevel, amdgpu_dm_abm_level, uint, 0444); |
| 783 | 783 | ||
| 784 | int amdgpu_backlight = -1; | ||
| 785 | MODULE_PARM_DESC(backlight, "Backlight control (0 = pwm, 1 = aux, -1 auto (default))"); | ||
| 786 | module_param_named(backlight, amdgpu_backlight, bint, 0444); | ||
| 787 | |||
| 784 | /** | 788 | /** |
| 785 | * DOC: tmz (int) | 789 | * DOC: tmz (int) |
| 786 | * Trusted Memory Zone (TMZ) is a method to protect data being written | 790 | * Trusted Memory Zone (TMZ) is a method to protect data being written |
| @@ -146,7 +146,7 @@ static int amdgpufb_create_pinned_object(struct amdgpu_fbdev *rfbdev, · 145 unmodified lines | |||
| 146 | size = mode_cmd->pitches[0] * height; | 146 | size = mode_cmd->pitches[0] * height; |
| 147 | aligned_size = ALIGN(size, PAGE_SIZE); | 147 | aligned_size = ALIGN(size, PAGE_SIZE); |
| 148 | ret = amdgpu_gem_object_create(adev, aligned_size, 0, domain, flags, | 148 | ret = amdgpu_gem_object_create(adev, aligned_size, 0, domain, flags, |
| 149 | ttm_bo_type_kernel, NULL, &gobj); | 149 | ttm_bo_type_device, NULL, &gobj); |
| 150 | if (ret) { | 150 | if (ret) { |
| 151 | pr_err("failed to allocate framebuffer (%d)\n", aligned_size); | 151 | pr_err("failed to allocate framebuffer (%d)\n", aligned_size); |
| 152 | return -ENOMEM; | 152 | return -ENOMEM; |
| @@ -2267,6 +2267,11 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector) · 2266 unmodified lines | |||
| 2267 | caps->ext_caps->bits.hdr_aux_backlight_control == 1) | 2267 | caps->ext_caps->bits.hdr_aux_backlight_control == 1) |
| 2268 | caps->aux_support = true; | 2268 | caps->aux_support = true; |
| 2269 | 2269 | ||
| 2270 | if (amdgpu_backlight == 0) | ||
| 2271 | caps->aux_support = false; | ||
| 2272 | else if (amdgpu_backlight == 1) | ||
| 2273 | caps->aux_support = true; | ||
| 2274 | |||
| 2270 | /* From the specification (CTA-861-G), for calculating the maximum | 2275 | /* From the specification (CTA-861-G), for calculating the maximum |
| 2271 | * luminance we need to use: | 2276 | * luminance we need to use: |
| 2272 | * Luminance = 50*2**(CV/32) | 2277 | * Luminance = 50*2**(CV/32) |
| @@ -3185,19 +3190,6 @@ static void amdgpu_dm_update_backlight_caps(struct amdgpu_display_manager *dm) · 912 unmodified lines | |||
| 3185 | #endif | 3190 | #endif |
| 3186 | } | 3191 | } |
| 3187 | 3192 | ||
| 3188 | static int set_backlight_via_aux(struct dc_link *link, uint32_t brightness) | ||
| 3189 | { | ||
| 3190 | bool rc; | ||
| 3191 | |||
| 3192 | if (!link) | ||
| 3193 | return 1; | ||
| 3194 | |||
| 3195 | rc = dc_link_set_backlight_level_nits(link, true, brightness, | ||
| 3196 | AUX_BL_DEFAULT_TRANSITION_TIME_MS); | ||
| 3197 | |||
| 3198 | return rc ? 0 : 1; | ||
| 3199 | } | ||
| 3200 | |||
| 3201 | static int get_brightness_range(const struct amdgpu_dm_backlight_caps *caps, | 3193 | static int get_brightness_range(const struct amdgpu_dm_backlight_caps *caps, |
| 3202 | unsigned *min, unsigned *max) | 3194 | unsigned *min, unsigned *max) |
| 3203 | { | 3195 | { |
| @@ -3260,9 +3252,10 @@ static int amdgpu_dm_backlight_update_status(struct backlight_device *bd) · 56 unmodified lines | |||
| 3260 | brightness = convert_brightness_from_user(&caps, bd->props.brightness); | 3252 | brightness = convert_brightness_from_user(&caps, bd->props.brightness); |
| 3261 | // Change brightness based on AUX property | 3253 | // Change brightness based on AUX property |
| 3262 | if (caps.aux_support) | 3254 | if (caps.aux_support) |
| 3263 | return set_backlight_via_aux(link, brightness); | 3255 | rc = dc_link_set_backlight_level_nits(link, true, brightness, |
| 3264 | 3256 | AUX_BL_DEFAULT_TRANSITION_TIME_MS); | |
| 3265 | rc = dc_link_set_backlight_level(dm->backlight_link, brightness, 0); | 3257 | else |
| 3258 | rc = dc_link_set_backlight_level(dm->backlight_link, brightness, 0); | ||
| 3266 | 3259 | ||
| 3267 | return rc ? 0 : 1; | 3260 | return rc ? 0 : 1; |
| 3268 | } | 3261 | } |
| @@ -3270,11 +3263,27 @@ static int amdgpu_dm_backlight_update_status(struct backlight_device *bd) · 1 unmodified line | |||
| 3270 | static int amdgpu_dm_backlight_get_brightness(struct backlight_device *bd) | 3263 | static int amdgpu_dm_backlight_get_brightness(struct backlight_device *bd) |
| 3271 | { | 3264 | { |
| 3272 | struct amdgpu_display_manager *dm = bl_get_data(bd); | 3265 | struct amdgpu_display_manager *dm = bl_get_data(bd); |
| 3273 | int ret = dc_link_get_backlight_level(dm->backlight_link); | 3266 | struct amdgpu_dm_backlight_caps caps; |
| 3267 | |||
| 3268 | amdgpu_dm_update_backlight_caps(dm); | ||
| 3269 | caps = dm->backlight_caps; | ||
| 3270 | |||
| 3271 | if (caps.aux_support) { | ||
| 3272 | struct dc_link *link = (struct dc_link *)dm->backlight_link; | ||
| 3273 | u32 avg, peak; | ||
| 3274 | bool rc; | ||
| 3274 | 3275 | ||
| 3275 | if (ret == DC_ERROR_UNEXPECTED) | 3276 | rc = dc_link_get_backlight_level_nits(link, &avg, &peak); |
| 3276 | return bd->props.brightness; | 3277 | if (!rc) |
| 3277 | return convert_brightness_to_user(&dm->backlight_caps, ret); | 3278 | return bd->props.brightness; |
| 3279 | return convert_brightness_to_user(&caps, avg); | ||
| 3280 | } else { | ||
| 3281 | int ret = dc_link_get_backlight_level(dm->backlight_link); | ||
| 3282 | |||
| 3283 | if (ret == DC_ERROR_UNEXPECTED) | ||
| 3284 | return bd->props.brightness; | ||
| 3285 | return convert_brightness_to_user(&caps, ret); | ||
| 3286 | } | ||
| 3278 | } | 3287 | } |
| 3279 | 3288 | ||
| 3280 | static const struct backlight_ops amdgpu_dm_backlight_ops = { | 3289 | static const struct backlight_ops amdgpu_dm_backlight_ops = { |
| @@ -4716,6 +4725,7 @@ static int fill_dc_plane_attributes(struct amdgpu_device *adev, · 1435 unmodified lines | |||
| 4716 | dc_plane_state->global_alpha_value = plane_info.global_alpha_value; | 4725 | dc_plane_state->global_alpha_value = plane_info.global_alpha_value; |
| 4717 | dc_plane_state->dcc = plane_info.dcc; | 4726 | dc_plane_state->dcc = plane_info.dcc; |
| 4718 | dc_plane_state->layer_index = plane_info.layer_index; // Always returns 0 | 4727 | dc_plane_state->layer_index = plane_info.layer_index; // Always returns 0 |
| 4728 | dc_plane_state->flip_int_enabled = true; | ||
| 4719 | 4729 | ||
| 4720 | /* | 4730 | /* |
| 4721 | * Always set input transfer function, since plane state is refreshed | 4731 | * Always set input transfer function, since plane state is refreshed |
| @@ -2602,7 +2602,6 @@ bool dc_link_set_backlight_level(const struct dc_link *link, · 2601 unmodified lines | |||
| 2602 | if (pipe_ctx->plane_state == NULL) | 2602 | if (pipe_ctx->plane_state == NULL) |
| 2603 | frame_ramp = 0; | 2603 | frame_ramp = 0; |
| 2604 | } else { | 2604 | } else { |
| 2605 | ASSERT(false); | ||
| 2606 | return false; | 2605 | return false; |
| 2607 | } | 2606 | } |
| 2608 | 2607 | ||
| @@ -887,6 +887,7 @@ struct dc_plane_state { · 886 unmodified lines | |||
| 887 | int layer_index; | 887 | int layer_index; |
| 888 | 888 | ||
| 889 | union surface_update_flags update_flags; | 889 | union surface_update_flags update_flags; |
| 890 | bool flip_int_enabled; | ||
| 890 | /* private to DC core */ | 891 | /* private to DC core */ |
| 891 | struct dc_plane_status status; | 892 | struct dc_plane_status status; |
| 892 | struct dc_context *ctx; | 893 | struct dc_context *ctx; |
| @@ -1257,6 +1257,16 @@ void hubp1_soft_reset(struct hubp *hubp, bool reset) · 1256 unmodified lines | |||
| 1257 | REG_UPDATE(DCHUBP_CNTL, HUBP_DISABLE, reset ? 1 : 0); | 1257 | REG_UPDATE(DCHUBP_CNTL, HUBP_DISABLE, reset ? 1 : 0); |
| 1258 | } | 1258 | } |
| 1259 | 1259 | ||
| 1260 | void hubp1_set_flip_int(struct hubp *hubp) | ||
| 1261 | { | ||
| 1262 | struct dcn10_hubp *hubp1 = TO_DCN10_HUBP(hubp); | ||
| 1263 | |||
| 1264 | REG_UPDATE(DCSURF_SURFACE_FLIP_INTERRUPT, | ||
| 1265 | SURFACE_FLIP_INT_MASK, 1); | ||
| 1266 | |||
| 1267 | return; | ||
| 1268 | } | ||
| 1269 | |||
| 1260 | void hubp1_init(struct hubp *hubp) | 1270 | void hubp1_init(struct hubp *hubp) |
| 1261 | { | 1271 | { |
| 1262 | //do nothing | 1272 | //do nothing |
| @@ -1290,6 +1300,7 @@ static const struct hubp_funcs dcn10_hubp_funcs = { · 27 unmodified lines | |||
| 1290 | .dmdata_load = NULL, | 1300 | .dmdata_load = NULL, |
| 1291 | .hubp_soft_reset = hubp1_soft_reset, | 1301 | .hubp_soft_reset = hubp1_soft_reset, |
| 1292 | .hubp_in_blank = hubp1_in_blank, | 1302 | .hubp_in_blank = hubp1_in_blank, |
| 1303 | .hubp_set_flip_int = hubp1_set_flip_int, | ||
| 1293 | }; | 1304 | }; |
| 1294 | 1305 | ||
| 1295 | /*****************************************/ | 1306 | /*****************************************/ |
| @@ -74,6 +74,7 @@ · 73 unmodified lines | |||
| 74 | SRI(DCSURF_SURFACE_EARLIEST_INUSE_C, HUBPREQ, id),\ | 74 | SRI(DCSURF_SURFACE_EARLIEST_INUSE_C, HUBPREQ, id),\ |
| 75 | SRI(DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C, HUBPREQ, id),\ | 75 | SRI(DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C, HUBPREQ, id),\ |
| 76 | SRI(DCSURF_SURFACE_CONTROL, HUBPREQ, id),\ | 76 | SRI(DCSURF_SURFACE_CONTROL, HUBPREQ, id),\ |
| 77 | SRI(DCSURF_SURFACE_FLIP_INTERRUPT, HUBPREQ, id),\ | ||
| 77 | SRI(HUBPRET_CONTROL, HUBPRET, id),\ | 78 | SRI(HUBPRET_CONTROL, HUBPRET, id),\ |
| 78 | SRI(DCN_EXPANSION_MODE, HUBPREQ, id),\ | 79 | SRI(DCN_EXPANSION_MODE, HUBPREQ, id),\ |
| 79 | SRI(DCHUBP_REQ_SIZE_CONFIG, HUBP, id),\ | 80 | SRI(DCHUBP_REQ_SIZE_CONFIG, HUBP, id),\ |
| @@ -183,6 +184,7 @@ · 103 unmodified lines | |||
| 183 | uint32_t DCSURF_SURFACE_EARLIEST_INUSE_C; \ | 184 | uint32_t DCSURF_SURFACE_EARLIEST_INUSE_C; \ |
| 184 | uint32_t DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C; \ | 185 | uint32_t DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C; \ |
| 185 | uint32_t DCSURF_SURFACE_CONTROL; \ | 186 | uint32_t DCSURF_SURFACE_CONTROL; \ |
| 187 | uint32_t DCSURF_SURFACE_FLIP_INTERRUPT; \ | ||
| 186 | uint32_t HUBPRET_CONTROL; \ | 188 | uint32_t HUBPRET_CONTROL; \ |
| 187 | uint32_t DCN_EXPANSION_MODE; \ | 189 | uint32_t DCN_EXPANSION_MODE; \ |
| 188 | uint32_t DCHUBP_REQ_SIZE_CONFIG; \ | 190 | uint32_t DCHUBP_REQ_SIZE_CONFIG; \ |
| @@ -332,6 +334,7 @@ · 143 unmodified lines | |||
| 332 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_META_SURFACE_TMZ_C, mask_sh),\ | 334 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_META_SURFACE_TMZ_C, mask_sh),\ |
| 333 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_EN, mask_sh),\ | 335 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_EN, mask_sh),\ |
| 334 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_64B_BLK, mask_sh),\ | 336 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_64B_BLK, mask_sh),\ |
| 337 | HUBP_SF(HUBPREQ0_DCSURF_SURFACE_FLIP_INTERRUPT, SURFACE_FLIP_INT_MASK, mask_sh),\ | ||
| 335 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, DET_BUF_PLANE1_BASE_ADDRESS, mask_sh),\ | 338 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, DET_BUF_PLANE1_BASE_ADDRESS, mask_sh),\ |
| 336 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CB_B, mask_sh),\ | 339 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CB_B, mask_sh),\ |
| 337 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CR_R, mask_sh),\ | 340 | HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CR_R, mask_sh),\ |
| @@ -531,6 +534,7 @@ · 193 unmodified lines | |||
| 531 | type PRIMARY_SURFACE_DCC_IND_64B_BLK;\ | 534 | type PRIMARY_SURFACE_DCC_IND_64B_BLK;\ |
| 532 | type SECONDARY_SURFACE_DCC_EN;\ | 535 | type SECONDARY_SURFACE_DCC_EN;\ |
| 533 | type SECONDARY_SURFACE_DCC_IND_64B_BLK;\ | 536 | type SECONDARY_SURFACE_DCC_IND_64B_BLK;\ |
| 537 | type SURFACE_FLIP_INT_MASK;\ | ||
| 534 | type DET_BUF_PLANE1_BASE_ADDRESS;\ | 538 | type DET_BUF_PLANE1_BASE_ADDRESS;\ |
| 535 | type CROSSBAR_SRC_CB_B;\ | 539 | type CROSSBAR_SRC_CB_B;\ |
| 536 | type CROSSBAR_SRC_CR_R;\ | 540 | type CROSSBAR_SRC_CR_R;\ |
| @@ -777,4 +781,6 @@ void hubp1_read_state_common(struct hubp *hubp); · 240 unmodified lines | |||
| 777 | bool hubp1_in_blank(struct hubp *hubp); | 781 | bool hubp1_in_blank(struct hubp *hubp); |
| 778 | void hubp1_soft_reset(struct hubp *hubp, bool reset); | 782 | void hubp1_soft_reset(struct hubp *hubp, bool reset); |
| 779 | 783 | ||
| 784 | void hubp1_set_flip_int(struct hubp *hubp); | ||
| 785 | |||
| 780 | #endif | 786 | #endif |
| @@ -2196,6 +2196,13 @@ static void dcn10_enable_plane( · 2195 unmodified lines | |||
| 2196 | if (dc->debug.sanity_checks) { | 2196 | if (dc->debug.sanity_checks) { |
| 2197 | hws->funcs.verify_allow_pstate_change_high(dc); | 2197 | hws->funcs.verify_allow_pstate_change_high(dc); |
| 2198 | } | 2198 | } |
| 2199 | |||
| 2200 | if (!pipe_ctx->top_pipe | ||
| 2201 | && pipe_ctx->plane_state | ||
| 2202 | && pipe_ctx->plane_state->flip_int_enabled | ||
| 2203 | && pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int) | ||
| 2204 | pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int(pipe_ctx->plane_res.hubp); | ||
| 2205 | |||
| 2199 | } | 2206 | } |
| 2200 | 2207 | ||
| 2201 | void dcn10_program_gamut_remap(struct pipe_ctx *pipe_ctx) | 2208 | void dcn10_program_gamut_remap(struct pipe_ctx *pipe_ctx) |
| @@ -1597,6 +1597,7 @@ static struct hubp_funcs dcn20_hubp_funcs = { · 1596 unmodified lines | |||
| 1597 | .validate_dml_output = hubp2_validate_dml_output, | 1597 | .validate_dml_output = hubp2_validate_dml_output, |
| 1598 | .hubp_in_blank = hubp1_in_blank, | 1598 | .hubp_in_blank = hubp1_in_blank, |
| 1599 | .hubp_soft_reset = hubp1_soft_reset, | 1599 | .hubp_soft_reset = hubp1_soft_reset, |
| 1600 | .hubp_set_flip_int = hubp1_set_flip_int, | ||
| 1600 | }; | 1601 | }; |
| 1601 | 1602 | ||
| 1602 | 1603 | ||
| @@ -1146,6 +1146,12 @@ void dcn20_enable_plane( · 1145 unmodified lines | |||
| 1146 | pipe_ctx->plane_res.hubp->funcs->hubp_set_vm_system_aperture_settings(pipe_ctx->plane_res.hubp, &apt); | 1146 | pipe_ctx->plane_res.hubp->funcs->hubp_set_vm_system_aperture_settings(pipe_ctx->plane_res.hubp, &apt); |
| 1147 | } | 1147 | } |
| 1148 | 1148 | ||
| 1149 | if (!pipe_ctx->top_pipe | ||
| 1150 | && pipe_ctx->plane_state | ||
| 1151 | && pipe_ctx->plane_state->flip_int_enabled | ||
| 1152 | && pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int) | ||
| 1153 | pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int(pipe_ctx->plane_res.hubp); | ||
| 1154 | |||
| 1149 | // if (dc->debug.sanity_checks) { | 1155 | // if (dc->debug.sanity_checks) { |
| 1150 | // dcn10_verify_allow_pstate_change_high(dc); | 1156 | // dcn10_verify_allow_pstate_change_high(dc); |
| 1151 | // } | 1157 | // } |
| @@ -838,6 +838,7 @@ static struct hubp_funcs dcn21_hubp_funcs = { · 837 unmodified lines | |||
| 838 | .hubp_set_flip_control_surface_gsl = hubp2_set_flip_control_surface_gsl, | 838 | .hubp_set_flip_control_surface_gsl = hubp2_set_flip_control_surface_gsl, |
| 839 | .hubp_init = hubp21_init, | 839 | .hubp_init = hubp21_init, |
| 840 | .validate_dml_output = hubp21_validate_dml_output, | 840 | .validate_dml_output = hubp21_validate_dml_output, |
| 841 | .hubp_set_flip_int = hubp1_set_flip_int, | ||
| 841 | }; | 842 | }; |
| 842 | 843 | ||
| 843 | bool hubp21_construct( | 844 | bool hubp21_construct( |
| @@ -296,7 +296,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn2_1_soc = { · 295 unmodified lines | |||
| 296 | .num_banks = 8, | 296 | .num_banks = 8, |
| 297 | .num_chans = 4, | 297 | .num_chans = 4, |
| 298 | .vmm_page_size_bytes = 4096, | 298 | .vmm_page_size_bytes = 4096, |
| 299 | .dram_clock_change_latency_us = 11.72, | 299 | .dram_clock_change_latency_us = 23.84, |
| 300 | .return_bus_width_bytes = 64, | 300 | .return_bus_width_bytes = 64, |
| 301 | .dispclk_dppclk_vco_speed_mhz = 3600, | 301 | .dispclk_dppclk_vco_speed_mhz = 3600, |
| 302 | .xfc_bus_transport_time_us = 4, | 302 | .xfc_bus_transport_time_us = 4, |
| @@ -1062,8 +1062,6 @@ static void patch_bounding_box(struct dc *dc, struct _vcs_dpi_soc_bounding_box_s · 759 unmodified lines | |||
| 1062 | { | 1062 | { |
| 1063 | int i; | 1063 | int i; |
| 1064 | 1064 | ||
| 1065 | DC_FP_START(); | ||
| 1066 | |||
| 1067 | if (dc->bb_overrides.sr_exit_time_ns) { | 1065 | if (dc->bb_overrides.sr_exit_time_ns) { |
| 1068 | for (i = 0; i < WM_SET_COUNT; i++) { | 1066 | for (i = 0; i < WM_SET_COUNT; i++) { |
| 1069 | dc->clk_mgr->bw_params->wm_table.entries[i].sr_exit_time_us = | 1067 | dc->clk_mgr->bw_params->wm_table.entries[i].sr_exit_time_us = |
| @@ -1088,8 +1086,6 @@ static void patch_bounding_box(struct dc *dc, struct _vcs_dpi_soc_bounding_box_s · 18 unmodified lines | |||
| 1088 | dc->bb_overrides.dram_clock_change_latency_ns / 1000.0; | 1086 | dc->bb_overrides.dram_clock_change_latency_ns / 1000.0; |
| 1089 | } | 1087 | } |
| 1090 | } | 1088 | } |
| 1091 | |||
| 1092 | DC_FP_END(); | ||
| 1093 | } | 1089 | } |
| 1094 | 1090 | ||
| 1095 | void dcn21_calculate_wm( | 1091 | void dcn21_calculate_wm( |
| @@ -1339,7 +1335,7 @@ static noinline bool dcn21_validate_bandwidth_fp(struct dc *dc, · 243 unmodified lines | |||
| 1339 | int vlevel = 0; | 1335 | int vlevel = 0; |
| 1340 | int pipe_split_from[MAX_PIPES]; | 1336 | int pipe_split_from[MAX_PIPES]; |
| 1341 | int pipe_cnt = 0; | 1337 | int pipe_cnt = 0; |
| 1342 | display_e2e_pipe_params_st *pipes = kzalloc(dc->res_pool->pipe_count * sizeof(display_e2e_pipe_params_st), GFP_KERNEL); | 1338 | display_e2e_pipe_params_st *pipes = kzalloc(dc->res_pool->pipe_count * sizeof(display_e2e_pipe_params_st), GFP_ATOMIC); |
| 1343 | DC_LOGGER_INIT(dc->ctx->logger); | 1339 | DC_LOGGER_INIT(dc->ctx->logger); |
| 1344 | 1340 | ||
| 1345 | BW_VAL_TRACE_COUNT(); | 1341 | BW_VAL_TRACE_COUNT(); |
| @@ -511,6 +511,7 @@ static struct hubp_funcs dcn30_hubp_funcs = { · 510 unmodified lines | |||
| 511 | .hubp_init = hubp3_init, | 511 | .hubp_init = hubp3_init, |
| 512 | .hubp_in_blank = hubp1_in_blank, | 512 | .hubp_in_blank = hubp1_in_blank, |
| 513 | .hubp_soft_reset = hubp1_soft_reset, | 513 | .hubp_soft_reset = hubp1_soft_reset, |
| 514 | .hubp_set_flip_int = hubp1_set_flip_int, | ||
| 514 | }; | 515 | }; |
| 515 | 516 | ||
| 516 | bool hubp3_construct( | 517 | bool hubp3_construct( |
| @@ -2508,6 +2508,19 @@ static const struct resource_funcs dcn30_res_pool_funcs = { · 2507 unmodified lines | |||
| 2508 | .patch_unknown_plane_state = dcn20_patch_unknown_plane_state, | 2508 | .patch_unknown_plane_state = dcn20_patch_unknown_plane_state, |
| 2509 | }; | 2509 | }; |
| 2510 | 2510 | ||
| 2511 | #define CTX ctx | ||
| 2512 | |||
| 2513 | #define REG(reg_name) \ | ||
| 2514 | (DCN_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name) | ||
| 2515 | |||
| 2516 | static uint32_t read_pipe_fuses(struct dc_context *ctx) | ||
| 2517 | { | ||
| 2518 | uint32_t value = REG_READ(CC_DC_PIPE_DIS); | ||
| 2519 | /* Support for max 6 pipes */ | ||
| 2520 | value = value & 0x3f; | ||
| 2521 | return value; | ||
| 2522 | } | ||
| 2523 | |||
| 2511 | static bool dcn30_resource_construct( | 2524 | static bool dcn30_resource_construct( |
| 2512 | uint8_t num_virtual_links, | 2525 | uint8_t num_virtual_links, |
| 2513 | struct dc *dc, | 2526 | struct dc *dc, |
| @@ -2517,6 +2530,15 @@ static bool dcn30_resource_construct( · 3 unmodified lines | |||
| 2517 | struct dc_context *ctx = dc->ctx; | 2530 | struct dc_context *ctx = dc->ctx; |
| 2518 | struct irq_service_init_data init_data; | 2531 | struct irq_service_init_data init_data; |
| 2519 | struct ddc_service_init_data ddc_init_data; | 2532 | struct ddc_service_init_data ddc_init_data; |
| 2533 | uint32_t pipe_fuses = read_pipe_fuses(ctx); | ||
| 2534 | uint32_t num_pipes = 0; | ||
| 2535 | |||
| 2536 | if (!(pipe_fuses == 0 || pipe_fuses == 0x3e)) { | ||
| 2537 | BREAK_TO_DEBUGGER(); | ||
| 2538 | dm_error("DC: Unexpected fuse recipe for navi2x !\n"); | ||
| 2539 | /* fault to single pipe */ | ||
| 2540 | pipe_fuses = 0x3e; | ||
| 2541 | } | ||
| 2520 | 2542 | ||
| 2521 | DC_FP_START(); | 2543 | DC_FP_START(); |
| 2522 | 2544 | ||
| @@ -2650,6 +2672,15 @@ static bool dcn30_resource_construct( · 127 unmodified lines | |||
| 2650 | /* PP Lib and SMU interfaces */ | 2672 | /* PP Lib and SMU interfaces */ |
| 2651 | init_soc_bounding_box(dc, pool); | 2673 | init_soc_bounding_box(dc, pool); |
| 2652 | 2674 | ||
| 2675 | num_pipes = dcn3_0_ip.max_num_dpp; | ||
| 2676 | |||
| 2677 | for (i = 0; i < dcn3_0_ip.max_num_dpp; i++) | ||
| 2678 | if (pipe_fuses & 1 << i) | ||
| 2679 | num_pipes--; | ||
| 2680 | |||
| 2681 | dcn3_0_ip.max_num_dpp = num_pipes; | ||
| 2682 | dcn3_0_ip.max_num_otg = num_pipes; | ||
| 2683 | |||
| 2653 | dml_init_instance(&dc->dml, &dcn3_0_soc, &dcn3_0_ip, DML_PROJECT_DCN30); | 2684 | dml_init_instance(&dc->dml, &dcn3_0_soc, &dcn3_0_ip, DML_PROJECT_DCN30); |
| 2654 | 2685 | ||
| 2655 | /* IRQ */ | 2686 | /* IRQ */ |
| @@ -1619,12 +1619,106 @@ static void dcn301_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *b · 1618 unmodified lines | |||
| 1619 | dml_init_instance(&dc->dml, &dcn3_01_soc, &dcn3_01_ip, DML_PROJECT_DCN30); | 1619 | dml_init_instance(&dc->dml, &dcn3_01_soc, &dcn3_01_ip, DML_PROJECT_DCN30); |
| 1620 | } | 1620 | } |
| 1621 | 1621 | ||
| 1622 | static void calculate_wm_set_for_vlevel( | ||
| 1623 | int vlevel, | ||
| 1624 | struct wm_range_table_entry *table_entry, | ||
| 1625 | struct dcn_watermarks *wm_set, | ||
| 1626 | struct display_mode_lib *dml, | ||
| 1627 | display_e2e_pipe_params_st *pipes, | ||
| 1628 | int pipe_cnt) | ||
| 1629 | { | ||
| 1630 | double dram_clock_change_latency_cached = dml->soc.dram_clock_change_latency_us; | ||
| 1631 | |||
| 1632 | ASSERT(vlevel < dml->soc.num_states); | ||
| 1633 | /* only pipe 0 is read for voltage and dcf/soc clocks */ | ||
| 1634 | pipes[0].clks_cfg.voltage = vlevel; | ||
| 1635 | pipes[0].clks_cfg.dcfclk_mhz = dml->soc.clock_limits[vlevel].dcfclk_mhz; | ||
| 1636 | pipes[0].clks_cfg.socclk_mhz = dml->soc.clock_limits[vlevel].socclk_mhz; | ||
| 1637 | |||
| 1638 | dml->soc.dram_clock_change_latency_us = table_entry->pstate_latency_us; | ||
| 1639 | dml->soc.sr_exit_time_us = table_entry->sr_exit_time_us; | ||
| 1640 | dml->soc.sr_enter_plus_exit_time_us = table_entry->sr_enter_plus_exit_time_us; | ||
| 1641 | |||
| 1642 | wm_set->urgent_ns = get_wm_urgent(dml, pipes, pipe_cnt) * 1000; | ||
| 1643 | wm_set->cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(dml, pipes, pipe_cnt) * 1000; | ||
| 1644 | wm_set->cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(dml, pipes, pipe_cnt) * 1000; | ||
| 1645 | wm_set->cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(dml, pipes, pipe_cnt) * 1000; | ||
| 1646 | wm_set->pte_meta_urgent_ns = get_wm_memory_trip(dml, pipes, pipe_cnt) * 1000; | ||
| 1647 | wm_set->frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(dml, pipes, pipe_cnt) * 1000; | ||
| 1648 | wm_set->frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(dml, pipes, pipe_cnt) * 1000; | ||
| 1649 | wm_set->urgent_latency_ns = get_urgent_latency(dml, pipes, pipe_cnt) * 1000; | ||
| 1650 | dml->soc.dram_clock_change_latency_us = dram_clock_change_latency_cached; | ||
| 1651 | |||
| 1652 | } | ||
| 1653 | |||
| 1654 | static void dcn301_calculate_wm_and_dlg( | ||
| 1655 | struct dc *dc, struct dc_state *context, | ||
| 1656 | display_e2e_pipe_params_st *pipes, | ||
| 1657 | int pipe_cnt, | ||
| 1658 | int vlevel_req) | ||
| 1659 | { | ||
| 1660 | int i, pipe_idx; | ||
| 1661 | int vlevel, vlevel_max; | ||
| 1662 | struct wm_range_table_entry *table_entry; | ||
| 1663 | struct clk_bw_params *bw_params = dc->clk_mgr->bw_params; | ||
| 1664 | |||
| 1665 | ASSERT(bw_params); | ||
| 1666 | |||
| 1667 | vlevel_max = bw_params->clk_table.num_entries - 1; | ||
| 1668 | |||
| 1669 | /* WM Set D */ | ||
| 1670 | table_entry = &bw_params->wm_table.entries[WM_D]; | ||
| 1671 | if (table_entry->wm_type == WM_TYPE_RETRAINING) | ||
| 1672 | vlevel = 0; | ||
| 1673 | else | ||
| 1674 | vlevel = vlevel_max; | ||
| 1675 | calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.d, | ||
| 1676 | &context->bw_ctx.dml, pipes, pipe_cnt); | ||
| 1677 | /* WM Set C */ | ||
| 1678 | table_entry = &bw_params->wm_table.entries[WM_C]; | ||
| 1679 | vlevel = min(max(vlevel_req, 2), vlevel_max); | ||
| 1680 | calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.c, | ||
| 1681 | &context->bw_ctx.dml, pipes, pipe_cnt); | ||
| 1682 | /* WM Set B */ | ||
| 1683 | table_entry = &bw_params->wm_table.entries[WM_B]; | ||
| 1684 | vlevel = min(max(vlevel_req, 1), vlevel_max); | ||
| 1685 | calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.b, | ||
| 1686 | &context->bw_ctx.dml, pipes, pipe_cnt); | ||
| 1687 | |||
| 1688 | /* WM Set A */ | ||
| 1689 | table_entry = &bw_params->wm_table.entries[WM_A]; | ||
| 1690 | vlevel = min(vlevel_req, vlevel_max); | ||
| 1691 | calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.a, | ||
| 1692 | &context->bw_ctx.dml, pipes, pipe_cnt); | ||
| 1693 | |||
| 1694 | for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) { | ||
| 1695 | if (!context->res_ctx.pipe_ctx[i].stream) | ||
| 1696 | continue; | ||
| 1697 | |||
| 1698 | pipes[pipe_idx].clks_cfg.dispclk_mhz = get_dispclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt); | ||
| 1699 | pipes[pipe_idx].clks_cfg.dppclk_mhz = get_dppclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx); | ||
| 1700 | |||
| 1701 | if (dc->config.forced_clocks) { | ||
| 1702 | pipes[pipe_idx].clks_cfg.dispclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dispclk_mhz; | ||
| 1703 | pipes[pipe_idx].clks_cfg.dppclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dppclk_mhz; | ||
| 1704 | } | ||
| 1705 | if (dc->debug.min_disp_clk_khz > pipes[pipe_idx].clks_cfg.dispclk_mhz * 1000) | ||
| 1706 | pipes[pipe_idx].clks_cfg.dispclk_mhz = dc->debug.min_disp_clk_khz / 1000.0; | ||
| 1707 | if (dc->debug.min_dpp_clk_khz > pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000) | ||
| 1708 | pipes[pipe_idx].clks_cfg.dppclk_mhz = dc->debug.min_dpp_clk_khz / 1000.0; | ||
| 1709 | |||
| 1710 | pipe_idx++; | ||
| 1711 | } | ||
| 1712 | |||
| 1713 | dcn20_calculate_dlg_params(dc, context, pipes, pipe_cnt, vlevel); | ||
| 1714 | } | ||
| 1715 | |||
| 1622 | static struct resource_funcs dcn301_res_pool_funcs = { | 1716 | static struct resource_funcs dcn301_res_pool_funcs = { |
| 1623 | .destroy = dcn301_destroy_resource_pool, | 1717 | .destroy = dcn301_destroy_resource_pool, |
| 1624 | .link_enc_create = dcn301_link_encoder_create, | 1718 | .link_enc_create = dcn301_link_encoder_create, |
| 1625 | .panel_cntl_create = dcn301_panel_cntl_create, | 1719 | .panel_cntl_create = dcn301_panel_cntl_create, |
| 1626 | .validate_bandwidth = dcn30_validate_bandwidth, | 1720 | .validate_bandwidth = dcn30_validate_bandwidth, |
| 1627 | .calculate_wm_and_dlg = dcn30_calculate_wm_and_dlg, | 1721 | .calculate_wm_and_dlg = dcn301_calculate_wm_and_dlg, |
| 1628 | .populate_dml_pipes = dcn30_populate_dml_pipes_from_context, | 1722 | .populate_dml_pipes = dcn30_populate_dml_pipes_from_context, |
| 1629 | .acquire_idle_pipe_for_layer = dcn20_acquire_idle_pipe_for_layer, | 1723 | .acquire_idle_pipe_for_layer = dcn20_acquire_idle_pipe_for_layer, |
| 1630 | .add_stream_to_ctx = dcn30_add_stream_to_ctx, | 1724 | .add_stream_to_ctx = dcn30_add_stream_to_ctx, |
| @@ -191,6 +191,8 @@ struct hubp_funcs { · 190 unmodified lines | |||
| 191 | bool (*hubp_in_blank)(struct hubp *hubp); | 191 | bool (*hubp_in_blank)(struct hubp *hubp); |
| 192 | void (*hubp_soft_reset)(struct hubp *hubp, bool reset); | 192 | void (*hubp_soft_reset)(struct hubp *hubp, bool reset); |
| 193 | 193 | ||
| 194 | void (*hubp_set_flip_int)(struct hubp *hubp); | ||
| 195 | |||
| 194 | }; | 196 | }; |
| 195 | 197 | ||
| 196 | #endif | 198 | #endif |
| @@ -5216,10 +5216,10 @@ static int smu7_set_watermarks_for_clocks_ranges(struct pp_hwmgr *hwmgr, · 5215 unmodified lines | |||
| 5216 | for (j = 0; j < dep_sclk_table->count; j++) { | 5216 | for (j = 0; j < dep_sclk_table->count; j++) { |
| 5217 | valid_entry = false; | 5217 | valid_entry = false; |
| 5218 | for (k = 0; k < watermarks->num_wm_sets; k++) { | 5218 | for (k = 0; k < watermarks->num_wm_sets; k++) { |
| 5219 | if (dep_sclk_table->entries[i].clk / 10 >= watermarks->wm_clk_ranges[k].wm_min_eng_clk_in_khz && | 5219 | if (dep_sclk_table->entries[i].clk >= watermarks->wm_clk_ranges[k].wm_min_eng_clk_in_khz / 10 && |
| 5220 | dep_sclk_table->entries[i].clk / 10 < watermarks->wm_clk_ranges[k].wm_max_eng_clk_in_khz && | 5220 | dep_sclk_table->entries[i].clk < watermarks->wm_clk_ranges[k].wm_max_eng_clk_in_khz / 10 && |
| 5221 | dep_mclk_table->entries[i].clk / 10 >= watermarks->wm_clk_ranges[k].wm_min_mem_clk_in_khz && | 5221 | dep_mclk_table->entries[i].clk >= watermarks->wm_clk_ranges[k].wm_min_mem_clk_in_khz / 10 && |
| 5222 | dep_mclk_table->entries[i].clk / 10 < watermarks->wm_clk_ranges[k].wm_max_mem_clk_in_khz) { | 5222 | dep_mclk_table->entries[i].clk < watermarks->wm_clk_ranges[k].wm_max_mem_clk_in_khz / 10) { |
| 5223 | valid_entry = true; | 5223 | valid_entry = true; |
| 5224 | table->DisplayWatermark[i][j] = watermarks->wm_clk_ranges[k].wm_set_id; | 5224 | table->DisplayWatermark[i][j] = watermarks->wm_clk_ranges[k].wm_set_id; |
| 5225 | break; | 5225 | break; |
| @@ -1505,6 +1505,48 @@ static int vega10_populate_single_lclk_level(struct pp_hwmgr *hwmgr, · 1504 unmodified lines | |||
| 1505 | return 0; | 1505 | return 0; |
| 1506 | } | 1506 | } |
| 1507 | 1507 | ||
| 1508 | static int vega10_override_pcie_parameters(struct pp_hwmgr *hwmgr) | ||
| 1509 | { | ||
| 1510 | struct amdgpu_device *adev = (struct amdgpu_device *)(hwmgr->adev); | ||
| 1511 | struct vega10_hwmgr *data = | ||
| 1512 | (struct vega10_hwmgr *)(hwmgr->backend); | ||
| 1513 | uint32_t pcie_gen = 0, pcie_width = 0; | ||
| 1514 | PPTable_t *pp_table = &(data->smc_state_table.pp_table); | ||
| 1515 | int i; | ||
| 1516 | |||
| 1517 | if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN4) | ||
| 1518 | pcie_gen = 3; | ||
| 1519 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3) | ||
| 1520 | pcie_gen = 2; | ||
| 1521 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN2) | ||
| 1522 | pcie_gen = 1; | ||
| 1523 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN1) | ||
| 1524 | pcie_gen = 0; | ||
| 1525 | |||
| 1526 | if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X16) | ||
| 1527 | pcie_width = 6; | ||
| 1528 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X12) | ||
| 1529 | pcie_width = 5; | ||
| 1530 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X8) | ||
| 1531 | pcie_width = 4; | ||
| 1532 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X4) | ||
| 1533 | pcie_width = 3; | ||
| 1534 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X2) | ||
| 1535 | pcie_width = 2; | ||
| 1536 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X1) | ||
| 1537 | pcie_width = 1; | ||
| 1538 | |||
| 1539 | for (i = 0; i < NUM_LINK_LEVELS; i++) { | ||
| 1540 | if (pp_table->PcieGenSpeed[i] > pcie_gen) | ||
| 1541 | pp_table->PcieGenSpeed[i] = pcie_gen; | ||
| 1542 | |||
| 1543 | if (pp_table->PcieLaneCount[i] > pcie_width) | ||
| 1544 | pp_table->PcieLaneCount[i] = pcie_width; | ||
| 1545 | } | ||
| 1546 | |||
| 1547 | return 0; | ||
| 1548 | } | ||
| 1549 | |||
| 1508 | static int vega10_populate_smc_link_levels(struct pp_hwmgr *hwmgr) | 1550 | static int vega10_populate_smc_link_levels(struct pp_hwmgr *hwmgr) |
| 1509 | { | 1551 | { |
| 1510 | int result = -1; | 1552 | int result = -1; |
| @@ -2556,6 +2598,11 @@ static int vega10_init_smc_table(struct pp_hwmgr *hwmgr) · 1045 unmodified lines | |||
| 2556 | "Failed to initialize Link Level!", | 2598 | "Failed to initialize Link Level!", |
| 2557 | return result); | 2599 | return result); |
| 2558 | 2600 | ||
| 2601 | result = vega10_override_pcie_parameters(hwmgr); | ||
| 2602 | PP_ASSERT_WITH_CODE(!result, | ||
| 2603 | "Failed to override pcie parameters!", | ||
| 2604 | return result); | ||
| 2605 | |||
| 2559 | result = vega10_populate_all_graphic_levels(hwmgr); | 2606 | result = vega10_populate_all_graphic_levels(hwmgr); |
| 2560 | PP_ASSERT_WITH_CODE(!result, | 2607 | PP_ASSERT_WITH_CODE(!result, |
| 2561 | "Failed to initialize Graphics Level!", | 2608 | "Failed to initialize Graphics Level!", |
| @@ -2922,6 +2969,7 @@ static int vega10_start_dpm(struct pp_hwmgr *hwmgr, uint32_t bitmap) · 360 unmodified lines | |||
| 2922 | return 0; | 2969 | return 0; |
| 2923 | } | 2970 | } |
| 2924 | 2971 | ||
| 2972 | |||
| 2925 | static int vega10_enable_disable_PCC_limit_feature(struct pp_hwmgr *hwmgr, bool enable) | 2973 | static int vega10_enable_disable_PCC_limit_feature(struct pp_hwmgr *hwmgr, bool enable) |
| 2926 | { | 2974 | { |
| 2927 | struct vega10_hwmgr *data = hwmgr->backend; | 2975 | struct vega10_hwmgr *data = hwmgr->backend; |
| @@ -481,6 +481,67 @@ static void vega12_init_dpm_state(struct vega12_dpm_state *dpm_state) · 480 unmodified lines | |||
| 481 | dpm_state->hard_max_level = 0xffff; | 481 | dpm_state->hard_max_level = 0xffff; |
| 482 | } | 482 | } |
| 483 | 483 | ||
| 484 | static int vega12_override_pcie_parameters(struct pp_hwmgr *hwmgr) | ||
| 485 | { | ||
| 486 | struct amdgpu_device *adev = (struct amdgpu_device *)(hwmgr->adev); | ||
| 487 | struct vega12_hwmgr *data = | ||
| 488 | (struct vega12_hwmgr *)(hwmgr->backend); | ||
| 489 | uint32_t pcie_gen = 0, pcie_width = 0, smu_pcie_arg, pcie_gen_arg, pcie_width_arg; | ||
| 490 | PPTable_t *pp_table = &(data->smc_state_table.pp_table); | ||
| 491 | int i; | ||
| 492 | int ret; | ||
| 493 | |||
| 494 | if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN4) | ||
| 495 | pcie_gen = 3; | ||
| 496 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3) | ||
| 497 | pcie_gen = 2; | ||
| 498 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN2) | ||
| 499 | pcie_gen = 1; | ||
| 500 | else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN1) | ||
| 501 | pcie_gen = 0; | ||
| 502 | |||
| 503 | if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X16) | ||
| 504 | pcie_width = 6; | ||
| 505 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X12) | ||
| 506 | pcie_width = 5; | ||
| 507 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X8) | ||
| 508 | pcie_width = 4; | ||
| 509 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X4) | ||
| 510 | pcie_width = 3; | ||
| 511 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X2) | ||
| 512 | pcie_width = 2; | ||
| 513 | else if (adev->pm.pcie_mlw_mask & CAIL_PCIE_LINK_WIDTH_SUPPORT_X1) | ||
| 514 | pcie_width = 1; | ||
| 515 | |||
| 516 | /* Bit 31:16: LCLK DPM level. 0 is DPM0, and 1 is DPM1 | ||
| 517 | * Bit 15:8: PCIE GEN, 0 to 3 corresponds to GEN1 to GEN4 | ||
| 518 | * Bit 7:0: PCIE lane width, 1 to 7 corresponds is x1 to x32 | ||
| 519 | */ | ||
| 520 | for (i = 0; i < NUM_LINK_LEVELS; i++) { | ||
| 521 | pcie_gen_arg = (pp_table->PcieGenSpeed[i] > pcie_gen) ? pcie_gen : | ||
| 522 | pp_table->PcieGenSpeed[i]; | ||
| 523 | pcie_width_arg = (pp_table->PcieLaneCount[i] > pcie_width) ? pcie_width : | ||
| 524 | pp_table->PcieLaneCount[i]; | ||
| 525 | |||
| 526 | if (pcie_gen_arg != pp_table->PcieGenSpeed[i] || pcie_width_arg != | ||
| 527 | pp_table->PcieLaneCount[i]) { | ||
| 528 | smu_pcie_arg = (i << 16) | (pcie_gen_arg << 8) | pcie_width_arg; | ||
| 529 | ret = smum_send_msg_to_smc_with_parameter(hwmgr, | ||
| 530 | PPSMC_MSG_OverridePcieParameters, smu_pcie_arg, | ||
| 531 | NULL); | ||
| 532 | PP_ASSERT_WITH_CODE(!ret, | ||
| 533 | "[OverridePcieParameters] Attempt to override pcie params failed!", | ||
| 534 | return ret); | ||
| 535 | } | ||
| 536 | |||
| 537 | /* update the pptable */ | ||
| 538 | pp_table->PcieGenSpeed[i] = pcie_gen_arg; | ||
| 539 | pp_table->PcieLaneCount[i] = pcie_width_arg; | ||
| 540 | } | ||
| 541 | |||
| 542 | return 0; | ||
| 543 | } | ||
| 544 | |||
| 484 | static int vega12_get_number_of_dpm_level(struct pp_hwmgr *hwmgr, | 545 | static int vega12_get_number_of_dpm_level(struct pp_hwmgr *hwmgr, |
| 485 | PPCLK_e clk_id, uint32_t *num_of_levels) | 546 | PPCLK_e clk_id, uint32_t *num_of_levels) |
| 486 | { | 547 | { |
| @@ -968,6 +1029,11 @@ static int vega12_enable_dpm_tasks(struct pp_hwmgr *hwmgr) · 481 unmodified lines | |||
| 968 | "Failed to enable all smu features!", | 1029 | "Failed to enable all smu features!", |
| 969 | return result); | 1030 | return result); |
| 970 | 1031 | ||
| 1032 | result = vega12_override_pcie_parameters(hwmgr); | ||
| 1033 | PP_ASSERT_WITH_CODE(!result, | ||
| 1034 | "[EnableDPMTasks] Failed to override pcie parameters!", | ||
| 1035 | return result); | ||
| 1036 | |||
| 971 | tmp_result = vega12_power_control_set_level(hwmgr); | 1037 | tmp_result = vega12_power_control_set_level(hwmgr); |
| 972 | PP_ASSERT_WITH_CODE(!tmp_result, | 1038 | PP_ASSERT_WITH_CODE(!tmp_result, |
| 973 | "Failed to power control set level!", | 1039 | "Failed to power control set level!", |
| @@ -831,7 +831,9 @@ static int vega20_override_pcie_parameters(struct pp_hwmgr *hwmgr) · 830 unmodified lines | |||
| 831 | struct amdgpu_device *adev = (struct amdgpu_device *)(hwmgr->adev); | 831 | struct amdgpu_device *adev = (struct amdgpu_device *)(hwmgr->adev); |
| 832 | struct vega20_hwmgr *data = | 832 | struct vega20_hwmgr *data = |
| 833 | (struct vega20_hwmgr *)(hwmgr->backend); | 833 | (struct vega20_hwmgr *)(hwmgr->backend); |
| 834 | uint32_t pcie_gen = 0, pcie_width = 0, smu_pcie_arg; | 834 | uint32_t pcie_gen = 0, pcie_width = 0, smu_pcie_arg, pcie_gen_arg, pcie_width_arg; |
| 835 | PPTable_t *pp_table = &(data->smc_state_table.pp_table); | ||
| 836 | int i; | ||
| 835 | int ret; | 837 | int ret; |
| 836 | 838 | ||
| 837 | if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN4) | 839 | if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN4) |
| @@ -860,17 +862,27 @@ static int vega20_override_pcie_parameters(struct pp_hwmgr *hwmgr) · 22 unmodified lines | |||
| 860 | * Bit 15:8: PCIE GEN, 0 to 3 corresponds to GEN1 to GEN4 | 862 | * Bit 15:8: PCIE GEN, 0 to 3 corresponds to GEN1 to GEN4 |
| 861 | * Bit 7:0: PCIE lane width, 1 to 7 corresponds is x1 to x32 | 863 | * Bit 7:0: PCIE lane width, 1 to 7 corresponds is x1 to x32 |
| 862 | */ | 864 | */ |
| 863 | smu_pcie_arg = (1 << 16) | (pcie_gen << 8) | pcie_width; | 865 | for (i = 0; i < NUM_LINK_LEVELS; i++) { |
| 864 | ret = smum_send_msg_to_smc_with_parameter(hwmgr, | 866 | pcie_gen_arg = (pp_table->PcieGenSpeed[i] > pcie_gen) ? pcie_gen : |
| 865 | PPSMC_MSG_OverridePcieParameters, smu_pcie_arg, | 867 | pp_table->PcieGenSpeed[i]; |
| 866 | NULL); | 868 | pcie_width_arg = (pp_table->PcieLaneCount[i] > pcie_width) ? pcie_width : |
| 867 | PP_ASSERT_WITH_CODE(!ret, | 869 | pp_table->PcieLaneCount[i]; |
| 868 | "[OverridePcieParameters] Attempt to override pcie params failed!", | 870 | |
| 869 | return ret); | 871 | if (pcie_gen_arg != pp_table->PcieGenSpeed[i] || pcie_width_arg != |
| 872 | pp_table->PcieLaneCount[i]) { | ||
| 873 | smu_pcie_arg = (i << 16) | (pcie_gen_arg << 8) | pcie_width_arg; | ||
| 874 | ret = smum_send_msg_to_smc_with_parameter(hwmgr, | ||
| 875 | PPSMC_MSG_OverridePcieParameters, smu_pcie_arg, | ||
| 876 | NULL); | ||
| 877 | PP_ASSERT_WITH_CODE(!ret, | ||
| 878 | "[OverridePcieParameters] Attempt to override pcie params failed!", | ||
| 879 | return ret); | ||
| 880 | } | ||
| 870 | 881 | ||
| 871 | data->pcie_parameters_override = true; | 882 | /* update the pptable */ |
| 872 | data->pcie_gen_level1 = pcie_gen; | 883 | pp_table->PcieGenSpeed[i] = pcie_gen_arg; |
| 873 | data->pcie_width_level1 = pcie_width; | 884 | pp_table->PcieLaneCount[i] = pcie_width_arg; |
| 885 | } | ||
| 874 | 886 | ||
| 875 | return 0; | 887 | return 0; |
| 876 | } | 888 | } |
| @@ -3319,9 +3331,7 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, · 2442 unmodified lines | |||
| 3319 | data->od8_settings.od8_settings_array; | 3331 | data->od8_settings.od8_settings_array; |
| 3320 | OverDriveTable_t *od_table = | 3332 | OverDriveTable_t *od_table = |
| 3321 | &(data->smc_state_table.overdrive_table); | 3333 | &(data->smc_state_table.overdrive_table); |
| 3322 | struct phm_ppt_v3_information *pptable_information = | 3334 | PPTable_t *pptable = &(data->smc_state_table.pp_table); |
| 3323 | (struct phm_ppt_v3_information *)hwmgr->pptable; | ||
| 3324 | PPTable_t *pptable = (PPTable_t *)pptable_information->smc_pptable; | ||
| 3325 | struct pp_clock_levels_with_latency clocks; | 3335 | struct pp_clock_levels_with_latency clocks; |
| 3326 | struct vega20_single_dpm_table *fclk_dpm_table = | 3336 | struct vega20_single_dpm_table *fclk_dpm_table = |
| 3327 | &(data->dpm_table.fclk_table); | 3337 | &(data->dpm_table.fclk_table); |
| @@ -3420,13 +3430,9 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, · 92 unmodified lines | |||
| 3420 | current_lane_width = | 3430 | current_lane_width = |
| 3421 | vega20_get_current_pcie_link_width_level(hwmgr); | 3431 | vega20_get_current_pcie_link_width_level(hwmgr); |
| 3422 | for (i = 0; i < NUM_LINK_LEVELS; i++) { | 3432 | for (i = 0; i < NUM_LINK_LEVELS; i++) { |
| 3423 | if (i == 1 && data->pcie_parameters_override) { | 3433 | gen_speed = pptable->PcieGenSpeed[i]; |
| 3424 | gen_speed = data->pcie_gen_level1; | 3434 | lane_width = pptable->PcieLaneCount[i]; |
| 3425 | lane_width = data->pcie_width_level1; | 3435 | |
| 3426 | } else { | ||
| 3427 | gen_speed = pptable->PcieGenSpeed[i]; | ||
| 3428 | lane_width = pptable->PcieLaneCount[i]; | ||
| 3429 | } | ||
| 3430 | size += sprintf(buf + size, "%d: %s %s %dMhz %s\n", i, | 3436 | size += sprintf(buf + size, "%d: %s %s %dMhz %s\n", i, |
| 3431 | (gen_speed == 0) ? "2.5GT/s," : | 3437 | (gen_speed == 0) ? "2.5GT/s," : |
| 3432 | (gen_speed == 1) ? "5.0GT/s," : | 3438 | (gen_speed == 1) ? "5.0GT/s," : |
| @@ -2048,7 +2048,7 @@ static void drm_fbdev_cleanup(struct drm_fb_helper *fb_helper) · 2047 unmodified lines | |||
| 2048 | 2048 | ||
| 2049 | if (shadow) | 2049 | if (shadow) |
| 2050 | vfree(shadow); | 2050 | vfree(shadow); |
| 2051 | else | 2051 | else if (fb_helper->buffer) |
| 2052 | drm_client_buffer_vunmap(fb_helper->buffer); | 2052 | drm_client_buffer_vunmap(fb_helper->buffer); |
| 2053 | 2053 | ||
| 2054 | drm_client_framebuffer_delete(fb_helper->buffer); | 2054 | drm_client_framebuffer_delete(fb_helper->buffer); |
| @@ -357,13 +357,14 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem, · 356 unmodified lines | |||
| 357 | if (--shmem->vmap_use_count > 0) | 357 | if (--shmem->vmap_use_count > 0) |
| 358 | return; | 358 | return; |
| 359 | 359 | ||
| 360 | if (obj->import_attach) | 360 | if (obj->import_attach) { |
| 361 | dma_buf_vunmap(obj->import_attach->dmabuf, map); | 361 | dma_buf_vunmap(obj->import_attach->dmabuf, map); |
| 362 | else | 362 | } else { |
| 363 | vunmap(shmem->vaddr); | 363 | vunmap(shmem->vaddr); |
| 364 | drm_gem_shmem_put_pages(shmem); | ||
| 365 | } | ||
| 364 | 366 | ||
| 365 | shmem->vaddr = NULL; | 367 | shmem->vaddr = NULL; |
| 366 | drm_gem_shmem_put_pages(shmem); | ||
| 367 | } | 368 | } |
| 368 | 369 | ||
| 369 | /* | 370 | /* |
| @@ -525,14 +526,28 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf) · 155 unmodified lines | |||
| 525 | struct drm_gem_object *obj = vma->vm_private_data; | 526 | struct drm_gem_object *obj = vma->vm_private_data; |
| 526 | struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj); | 527 | struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj); |
| 527 | loff_t num_pages = obj->size >> PAGE_SHIFT; | 528 | loff_t num_pages = obj->size >> PAGE_SHIFT; |
| 529 | vm_fault_t ret; | ||
| 528 | struct page *page; | 530 | struct page *page; |
| 531 | pgoff_t page_offset; | ||
| 529 | 532 | ||
| 530 | if (vmf->pgoff >= num_pages || WARN_ON_ONCE(!shmem->pages)) | 533 | /* We don't use vmf->pgoff since that has the fake offset */ |
| 531 | return VM_FAULT_SIGBUS; | 534 | page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT; |
| 532 | 535 | ||
| 533 | page = shmem->pages[vmf->pgoff]; | 536 | mutex_lock(&shmem->pages_lock); |
| 534 | 537 | ||
| 535 | return vmf_insert_page(vma, vmf->address, page); | 538 | if (page_offset >= num_pages || |
| 539 | WARN_ON_ONCE(!shmem->pages) || | ||
| 540 | shmem->madv < 0) { | ||
| 541 | ret = VM_FAULT_SIGBUS; | ||
| 542 | } else { | ||
| 543 | page = shmem->pages[page_offset]; | ||
| 544 | |||
| 545 | ret = vmf_insert_page(vma, vmf->address, page); | ||
| 546 | } | ||
| 547 | |||
| 548 | mutex_unlock(&shmem->pages_lock); | ||
| 549 | |||
| 550 | return ret; | ||
| 536 | } | 551 | } |
| 537 | 552 | ||
| 538 | static void drm_gem_shmem_vm_open(struct vm_area_struct *vma) | 553 | static void drm_gem_shmem_vm_open(struct vm_area_struct *vma) |
| @@ -581,9 +596,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) · 42 unmodified lines | |||
| 581 | struct drm_gem_shmem_object *shmem; | 596 | struct drm_gem_shmem_object *shmem; |
| 582 | int ret; | 597 | int ret; |
| 583 | 598 | ||
| 584 | /* Remove the fake offset */ | ||
| 585 | vma->vm_pgoff -= drm_vma_node_start(&obj->vma_node); | ||
| 586 | |||
| 587 | if (obj->import_attach) { | 599 | if (obj->import_attach) { |
| 588 | /* Drop the reference drm_gem_mmap_obj() acquired.*/ | 600 | /* Drop the reference drm_gem_mmap_obj() acquired.*/ |
| 589 | drm_gem_object_put(obj); | 601 | drm_gem_object_put(obj); |
| @@ -99,6 +99,8 @@ static int compat_drm_version(struct file *file, unsigned int cmd, · 98 unmodified lines | |||
| 99 | if (copy_from_user(&v32, (void __user *)arg, sizeof(v32))) | 99 | if (copy_from_user(&v32, (void __user *)arg, sizeof(v32))) |
| 100 | return -EFAULT; | 100 | return -EFAULT; |
| 101 | 101 | ||
| 102 | memset(&v, 0, sizeof(v)); | ||
| 103 | |||
| 102 | v = (struct drm_version) { | 104 | v = (struct drm_version) { |
| 103 | .name_len = v32.name_len, | 105 | .name_len = v32.name_len, |
| 104 | .name = compat_ptr(v32.name), | 106 | .name = compat_ptr(v32.name), |
| @@ -137,6 +139,9 @@ static int compat_drm_getunique(struct file *file, unsigned int cmd, · 32 unmodified lines | |||
| 137 | 139 | ||
| 138 | if (copy_from_user(&uq32, (void __user *)arg, sizeof(uq32))) | 140 | if (copy_from_user(&uq32, (void __user *)arg, sizeof(uq32))) |
| 139 | return -EFAULT; | 141 | return -EFAULT; |
| 142 | |||
| 143 | memset(&uq, 0, sizeof(uq)); | ||
| 144 | |||
| 140 | uq = (struct drm_unique){ | 145 | uq = (struct drm_unique){ |
| 141 | .unique_len = uq32.unique_len, | 146 | .unique_len = uq32.unique_len, |
| 142 | .unique = compat_ptr(uq32.unique), | 147 | .unique = compat_ptr(uq32.unique), |
| @@ -265,6 +270,8 @@ static int compat_drm_getclient(struct file *file, unsigned int cmd, · 122 unmodified lines | |||
| 265 | if (copy_from_user(&c32, argp, sizeof(c32))) | 270 | if (copy_from_user(&c32, argp, sizeof(c32))) |
| 266 | return -EFAULT; | 271 | return -EFAULT; |
| 267 | 272 | ||
| 273 | memset(&client, 0, sizeof(client)); | ||
| 274 | |||
| 268 | client.idx = c32.idx; | 275 | client.idx = c32.idx; |
| 269 | 276 | ||
| 270 | err = drm_ioctl_kernel(file, drm_getclient, &client, 0); | 277 | err = drm_ioctl_kernel(file, drm_getclient, &client, 0); |
| @@ -852,6 +859,8 @@ static int compat_drm_wait_vblank(struct file *file, unsigned int cmd, · 581 unmodified lines | |||
| 852 | if (copy_from_user(&req32, argp, sizeof(req32))) | 859 | if (copy_from_user(&req32, argp, sizeof(req32))) |
| 853 | return -EFAULT; | 860 | return -EFAULT; |
| 854 | 861 | ||
| 862 | memset(&req, 0, sizeof(req)); | ||
| 863 | |||
| 855 | req.request.type = req32.request.type; | 864 | req.request.type = req32.request.type; |
| 856 | req.request.sequence = req32.request.sequence; | 865 | req.request.sequence = req32.request.sequence; |
| 857 | req.request.signal = req32.request.signal; | 866 | req.request.signal = req32.request.signal; |
| @@ -889,6 +898,8 @@ static int compat_drm_mode_addfb2(struct file *file, unsigned int cmd, · 31 unmodified lines | |||
| 889 | struct drm_mode_fb_cmd2 req64; | 898 | struct drm_mode_fb_cmd2 req64; |
| 890 | int err; | 899 | int err; |
| 891 | 900 | ||
| 901 | memset(&req64, 0, sizeof(req64)); | ||
| 902 | |||
| 892 | if (copy_from_user(&req64, argp, | 903 | if (copy_from_user(&req64, argp, |
| 893 | offsetof(drm_mode_fb_cmd232_t, modifier))) | 904 | offsetof(drm_mode_fb_cmd232_t, modifier))) |
| 894 | return -EFAULT; | 905 | return -EFAULT; |
| @@ -713,9 +713,12 @@ static int engine_setup_common(struct intel_engine_cs *engine) · 712 unmodified lines | |||
| 713 | goto err_status; | 713 | goto err_status; |
| 714 | } | 714 | } |
| 715 | 715 | ||
| 716 | err = intel_engine_init_cmd_parser(engine); | ||
| 717 | if (err) | ||
| 718 | goto err_cmd_parser; | ||
| 719 | |||
| 716 | intel_engine_init_active(engine, ENGINE_PHYSICAL); | 720 | intel_engine_init_active(engine, ENGINE_PHYSICAL); |
| 717 | intel_engine_init_execlists(engine); | 721 | intel_engine_init_execlists(engine); |
| 718 | intel_engine_init_cmd_parser(engine); | ||
| 719 | intel_engine_init__pm(engine); | 722 | intel_engine_init__pm(engine); |
| 720 | intel_engine_init_retire(engine); | 723 | intel_engine_init_retire(engine); |
| 721 | 724 | ||
| @@ -732,6 +735,8 @@ static int engine_setup_common(struct intel_engine_cs *engine) · 10 unmodified lines | |||
| 732 | 735 | ||
| 733 | return 0; | 736 | return 0; |
| 734 | 737 | ||
| 738 | err_cmd_parser: | ||
| 739 | intel_breadcrumbs_free(engine->breadcrumbs); | ||
| 735 | err_status: | 740 | err_status: |
| 736 | cleanup_status_page(engine); | 741 | cleanup_status_page(engine); |
| 737 | return err; | 742 | return err; |
| @@ -940,7 +940,7 @@ static void fini_hash_table(struct intel_engine_cs *engine) · 939 unmodified lines | |||
| 940 | * struct intel_engine_cs based on whether the platform requires software | 940 | * struct intel_engine_cs based on whether the platform requires software |
| 941 | * command parsing. | 941 | * command parsing. |
| 942 | */ | 942 | */ |
| 943 | void intel_engine_init_cmd_parser(struct intel_engine_cs *engine) | 943 | int intel_engine_init_cmd_parser(struct intel_engine_cs *engine) |
| 944 | { | 944 | { |
| 945 | const struct drm_i915_cmd_table *cmd_tables; | 945 | const struct drm_i915_cmd_table *cmd_tables; |
| 946 | int cmd_table_count; | 946 | int cmd_table_count; |
| @@ -948,7 +948,7 @@ void intel_engine_init_cmd_parser(struct intel_engine_cs *engine) · 1 unmodified line | |||
| 948 | 948 | ||
| 949 | if (!IS_GEN(engine->i915, 7) && !(IS_GEN(engine->i915, 9) && | 949 | if (!IS_GEN(engine->i915, 7) && !(IS_GEN(engine->i915, 9) && |
| 950 | engine->class == COPY_ENGINE_CLASS)) | 950 | engine->class == COPY_ENGINE_CLASS)) |
| 951 | return; | 951 | return 0; |
| 952 | 952 | ||
| 953 | switch (engine->class) { | 953 | switch (engine->class) { |
| 954 | case RENDER_CLASS: | 954 | case RENDER_CLASS: |
| @@ -1013,19 +1013,19 @@ void intel_engine_init_cmd_parser(struct intel_engine_cs *engine) · 58 unmodified lines | |||
| 1013 | break; | 1013 | break; |
| 1014 | default: | 1014 | default: |
| 1015 | MISSING_CASE(engine->class); | 1015 | MISSING_CASE(engine->class); |
| 1016 | return; | 1016 | goto out; |
| 1017 | } | 1017 | } |
| 1018 | 1018 | ||
| 1019 | if (!validate_cmds_sorted(engine, cmd_tables, cmd_table_count)) { | 1019 | if (!validate_cmds_sorted(engine, cmd_tables, cmd_table_count)) { |
| 1020 | drm_err(&engine->i915->drm, | 1020 | drm_err(&engine->i915->drm, |
| 1021 | "%s: command descriptions are not sorted\n", | 1021 | "%s: command descriptions are not sorted\n", |
| 1022 | engine->name); | 1022 | engine->name); |
| 1023 | return; | 1023 | goto out; |
| 1024 | } | 1024 | } |
| 1025 | if (!validate_regs_sorted(engine)) { | 1025 | if (!validate_regs_sorted(engine)) { |
| 1026 | drm_err(&engine->i915->drm, | 1026 | drm_err(&engine->i915->drm, |
| 1027 | "%s: registers are not sorted\n", engine->name); | 1027 | "%s: registers are not sorted\n", engine->name); |
| 1028 | return; | 1028 | goto out; |
| 1029 | } | 1029 | } |
| 1030 | 1030 | ||
| 1031 | ret = init_hash_table(engine, cmd_tables, cmd_table_count); | 1031 | ret = init_hash_table(engine, cmd_tables, cmd_table_count); |
| @@ -1033,10 +1033,17 @@ void intel_engine_init_cmd_parser(struct intel_engine_cs *engine) · 1 unmodified line | |||
| 1033 | drm_err(&engine->i915->drm, | 1033 | drm_err(&engine->i915->drm, |
| 1034 | "%s: initialised failed!\n", engine->name); | 1034 | "%s: initialised failed!\n", engine->name); |
| 1035 | fini_hash_table(engine); | 1035 | fini_hash_table(engine); |
| 1036 | return; | 1036 | goto out; |
| 1037 | } | 1037 | } |
| 1038 | 1038 | ||
| 1039 | engine->flags |= I915_ENGINE_USING_CMD_PARSER; | 1039 | engine->flags |= I915_ENGINE_USING_CMD_PARSER; |
| 1040 | |||
| 1041 | out: | ||
| 1042 | if (intel_engine_requires_cmd_parser(engine) && | ||
| 1043 | !intel_engine_using_cmd_parser(engine)) | ||
| 1044 | return -EINVAL; | ||
| 1045 | |||
| 1046 | return 0; | ||
| 1040 | } | 1047 | } |
| 1041 | 1048 | ||
| 1042 | /** | 1049 | /** |
| @@ -1952,7 +1952,7 @@ const char *i915_cache_level_str(struct drm_i915_private *i915, int type); · 1951 unmodified lines | |||
| 1952 | 1952 | ||
| 1953 | /* i915_cmd_parser.c */ | 1953 | /* i915_cmd_parser.c */ |
| 1954 | int i915_cmd_parser_get_version(struct drm_i915_private *dev_priv); | 1954 | int i915_cmd_parser_get_version(struct drm_i915_private *dev_priv); |
| 1955 | void intel_engine_init_cmd_parser(struct intel_engine_cs *engine); | 1955 | int intel_engine_init_cmd_parser(struct intel_engine_cs *engine); |
| 1956 | void intel_engine_cleanup_cmd_parser(struct intel_engine_cs *engine); | 1956 | void intel_engine_cleanup_cmd_parser(struct intel_engine_cs *engine); |
| 1957 | int intel_engine_cmd_parser(struct intel_engine_cs *engine, | 1957 | int intel_engine_cmd_parser(struct intel_engine_cs *engine, |
| 1958 | struct i915_vma *batch, | 1958 | struct i915_vma *batch, |
| @@ -482,6 +482,16 @@ static int meson_probe_remote(struct platform_device *pdev, · 481 unmodified lines | |||
| 482 | return count; | 482 | return count; |
| 483 | } | 483 | } |
| 484 | 484 | ||
| 485 | static void meson_drv_shutdown(struct platform_device *pdev) | ||
| 486 | { | ||
| 487 | struct meson_drm *priv = dev_get_drvdata(&pdev->dev); | ||
| 488 | struct drm_device *drm = priv->drm; | ||
| 489 | |||
| 490 | DRM_DEBUG_DRIVER("\n"); | ||
| 491 | drm_kms_helper_poll_fini(drm); | ||
| 492 | drm_atomic_helper_shutdown(drm); | ||
| 493 | } | ||
| 494 | |||
| 485 | static int meson_drv_probe(struct platform_device *pdev) | 495 | static int meson_drv_probe(struct platform_device *pdev) |
| 486 | { | 496 | { |
| 487 | struct component_match *match = NULL; | 497 | struct component_match *match = NULL; |
| @@ -553,6 +563,7 @@ static const struct dev_pm_ops meson_drv_pm_ops = { · 65 unmodified lines | |||
| 553 | 563 | ||
| 554 | static struct platform_driver meson_drm_platform_driver = { | 564 | static struct platform_driver meson_drm_platform_driver = { |
| 555 | .probe = meson_drv_probe, | 565 | .probe = meson_drv_probe, |
| 566 | .shutdown = meson_drv_shutdown, | ||
| 556 | .driver = { | 567 | .driver = { |
| 557 | .name = "meson-drm", | 568 | .name = "meson-drm", |
| 558 | .of_match_table = dt_match, | 569 | .of_match_table = dt_match, |
| @@ -556,7 +556,8 @@ nouveau_bo_sync_for_device(struct nouveau_bo *nvbo) · 555 unmodified lines | |||
| 556 | if (nvbo->force_coherent) | 556 | if (nvbo->force_coherent) |
| 557 | return; | 557 | return; |
| 558 | 558 | ||
| 559 | for (i = 0; i < ttm_dma->num_pages; ++i) { | 559 | i = 0; |
| 560 | while (i < ttm_dma->num_pages) { | ||
| 560 | struct page *p = ttm_dma->pages[i]; | 561 | struct page *p = ttm_dma->pages[i]; |
| 561 | size_t num_pages = 1; | 562 | size_t num_pages = 1; |
| 562 | 563 | ||
| @@ -587,7 +588,8 @@ nouveau_bo_sync_for_cpu(struct nouveau_bo *nvbo) · 24 unmodified lines | |||
| 587 | if (nvbo->force_coherent) | 588 | if (nvbo->force_coherent) |
| 588 | return; | 589 | return; |
| 589 | 590 | ||
| 590 | for (i = 0; i < ttm_dma->num_pages; ++i) { | 591 | i = 0; |
| 592 | while (i < ttm_dma->num_pages) { | ||
| 591 | struct page *p = ttm_dma->pages[i]; | 593 | struct page *p = ttm_dma->pages[i]; |
| 592 | size_t num_pages = 1; | 594 | size_t num_pages = 1; |
| 593 | 595 | ||
| @@ -328,6 +328,7 @@ static void qxl_crtc_update_monitors_config(struct drm_crtc *crtc, · 327 unmodified lines | |||
| 328 | 328 | ||
| 329 | head.id = i; | 329 | head.id = i; |
| 330 | head.flags = 0; | 330 | head.flags = 0; |
| 331 | head.surface_id = 0; | ||
| 331 | oldcount = qdev->monitors_config->count; | 332 | oldcount = qdev->monitors_config->count; |
| 332 | if (crtc->state->active) { | 333 | if (crtc->state->active) { |
| 333 | struct drm_display_mode *mode = &crtc->mode; | 334 | struct drm_display_mode *mode = &crtc->mode; |
| @@ -321,7 +321,7 @@ int qxl_alloc_release_reserved(struct qxl_device *qdev, unsigned long size, · 320 unmodified lines | |||
| 321 | int type, struct qxl_release **release, | 321 | int type, struct qxl_release **release, |
| 322 | struct qxl_bo **rbo) | 322 | struct qxl_bo **rbo) |
| 323 | { | 323 | { |
| 324 | struct qxl_bo *bo; | 324 | struct qxl_bo *bo, *free_bo = NULL; |
| 325 | int idr_ret; | 325 | int idr_ret; |
| 326 | int ret = 0; | 326 | int ret = 0; |
| 327 | union qxl_release_info *info; | 327 | union qxl_release_info *info; |
| @@ -347,7 +347,7 @@ int qxl_alloc_release_reserved(struct qxl_device *qdev, unsigned long size, · 19 unmodified lines | |||
| 347 | 347 | ||
| 348 | mutex_lock(&qdev->release_mutex); | 348 | mutex_lock(&qdev->release_mutex); |
| 349 | if (qdev->current_release_bo_offset[cur_idx] + 1 >= releases_per_bo[cur_idx]) { | 349 | if (qdev->current_release_bo_offset[cur_idx] + 1 >= releases_per_bo[cur_idx]) { |
| 350 | qxl_bo_unref(&qdev->current_release_bo[cur_idx]); | 350 | free_bo = qdev->current_release_bo[cur_idx]; |
| 351 | qdev->current_release_bo_offset[cur_idx] = 0; | 351 | qdev->current_release_bo_offset[cur_idx] = 0; |
| 352 | qdev->current_release_bo[cur_idx] = NULL; | 352 | qdev->current_release_bo[cur_idx] = NULL; |
| 353 | } | 353 | } |
| @@ -355,6 +355,10 @@ int qxl_alloc_release_reserved(struct qxl_device *qdev, unsigned long size, · 1 unmodified line | |||
| 355 | ret = qxl_release_bo_alloc(qdev, &qdev->current_release_bo[cur_idx]); | 355 | ret = qxl_release_bo_alloc(qdev, &qdev->current_release_bo[cur_idx]); |
| 356 | if (ret) { | 356 | if (ret) { |
| 357 | mutex_unlock(&qdev->release_mutex); | 357 | mutex_unlock(&qdev->release_mutex); |
| 358 | if (free_bo) { | ||
| 359 | qxl_bo_unpin(free_bo); | ||
| 360 | qxl_bo_unref(&free_bo); | ||
| 361 | } | ||
| 358 | qxl_release_free(qdev, *release); | 362 | qxl_release_free(qdev, *release); |
| 359 | return ret; | 363 | return ret; |
| 360 | } | 364 | } |
| @@ -370,6 +374,10 @@ int qxl_alloc_release_reserved(struct qxl_device *qdev, unsigned long size, · 9 unmodified lines | |||
| 370 | *rbo = bo; | 374 | *rbo = bo; |
| 371 | 375 | ||
| 372 | mutex_unlock(&qdev->release_mutex); | 376 | mutex_unlock(&qdev->release_mutex); |
| 377 | if (free_bo) { | ||
| 378 | qxl_bo_unpin(free_bo); | ||
| 379 | qxl_bo_unref(&free_bo); | ||
| 380 | } | ||
| 373 | 381 | ||
| 374 | ret = qxl_release_list_add(*release, bo); | 382 | ret = qxl_release_list_add(*release, bo); |
| 375 | qxl_bo_unref(&bo); | 383 | qxl_bo_unref(&bo); |
| @@ -574,6 +574,8 @@ struct radeon_gem { · 573 unmodified lines | |||
| 574 | struct list_head objects; | 574 | struct list_head objects; |
| 575 | }; | 575 | }; |
| 576 | 576 | ||
| 577 | extern const struct drm_gem_object_funcs radeon_gem_object_funcs; | ||
| 578 | |||
| 577 | int radeon_gem_init(struct radeon_device *rdev); | 579 | int radeon_gem_init(struct radeon_device *rdev); |
| 578 | void radeon_gem_fini(struct radeon_device *rdev); | 580 | void radeon_gem_fini(struct radeon_device *rdev); |
| 579 | int radeon_gem_object_create(struct radeon_device *rdev, unsigned long size, | 581 | int radeon_gem_object_create(struct radeon_device *rdev, unsigned long size, |
| @@ -43,7 +43,7 @@ struct sg_table *radeon_gem_prime_get_sg_table(struct drm_gem_object *obj); · 42 unmodified lines | |||
| 43 | int radeon_gem_prime_pin(struct drm_gem_object *obj); | 43 | int radeon_gem_prime_pin(struct drm_gem_object *obj); |
| 44 | void radeon_gem_prime_unpin(struct drm_gem_object *obj); | 44 | void radeon_gem_prime_unpin(struct drm_gem_object *obj); |
| 45 | 45 | ||
| 46 | static const struct drm_gem_object_funcs radeon_gem_object_funcs; | 46 | const struct drm_gem_object_funcs radeon_gem_object_funcs; |
| 47 | 47 | ||
| 48 | static void radeon_gem_object_free(struct drm_gem_object *gobj) | 48 | static void radeon_gem_object_free(struct drm_gem_object *gobj) |
| 49 | { | 49 | { |
| @@ -227,7 +227,7 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r) · 177 unmodified lines | |||
| 227 | return r; | 227 | return r; |
| 228 | } | 228 | } |
| 229 | 229 | ||
| 230 | static const struct drm_gem_object_funcs radeon_gem_object_funcs = { | 230 | const struct drm_gem_object_funcs radeon_gem_object_funcs = { |
| 231 | .free = radeon_gem_object_free, | 231 | .free = radeon_gem_object_free, |
| 232 | .open = radeon_gem_object_open, | 232 | .open = radeon_gem_object_open, |
| 233 | .close = radeon_gem_object_close, | 233 | .close = radeon_gem_object_close, |
| @@ -56,6 +56,8 @@ struct drm_gem_object *radeon_gem_prime_import_sg_table(struct drm_device *dev, · 55 unmodified lines | |||
| 56 | if (ret) | 56 | if (ret) |
| 57 | return ERR_PTR(ret); | 57 | return ERR_PTR(ret); |
| 58 | 58 | ||
| 59 | bo->tbo.base.funcs = &radeon_gem_object_funcs; | ||
| 60 | |||
| 59 | mutex_lock(&rdev->gem.mutex); | 61 | mutex_lock(&rdev->gem.mutex); |
| 60 | list_add_tail(&bo->list, &rdev->gem.objects); | 62 | list_add_tail(&bo->list, &rdev->gem.objects); |
| 61 | mutex_unlock(&rdev->gem.mutex); | 63 | mutex_unlock(&rdev->gem.mutex); |
| @@ -83,6 +83,7 @@ MODULE_PARM_DESC(eco_mode, "Turn on Eco mode (less bright, more silent)"); · 82 unmodified lines | |||
| 83 | 83 | ||
| 84 | struct gm12u320_device { | 84 | struct gm12u320_device { |
| 85 | struct drm_device dev; | 85 | struct drm_device dev; |
| 86 | struct device *dmadev; | ||
| 86 | struct drm_simple_display_pipe pipe; | 87 | struct drm_simple_display_pipe pipe; |
| 87 | struct drm_connector conn; | 88 | struct drm_connector conn; |
| 88 | unsigned char *cmd_buf; | 89 | unsigned char *cmd_buf; |
| @@ -601,6 +602,22 @@ static const uint64_t gm12u320_pipe_modifiers[] = { · 512 unmodified lines | |||
| 601 | DRM_FORMAT_MOD_INVALID | 602 | DRM_FORMAT_MOD_INVALID |
| 602 | }; | 603 | }; |
| 603 | 604 | ||
| 605 | /* | ||
| 606 | * FIXME: Dma-buf sharing requires DMA support by the importing device. | ||
| 607 | * This function is a workaround to make USB devices work as well. | ||
| 608 | * See todo.rst for how to fix the issue in the dma-buf framework. | ||
| 609 | */ | ||
| 610 | static struct drm_gem_object *gm12u320_gem_prime_import(struct drm_device *dev, | ||
| 611 | struct dma_buf *dma_buf) | ||
| 612 | { | ||
| 613 | struct gm12u320_device *gm12u320 = to_gm12u320(dev); | ||
| 614 | |||
| 615 | if (!gm12u320->dmadev) | ||
| 616 | return ERR_PTR(-ENODEV); | ||
| 617 | |||
| 618 | return drm_gem_prime_import_dev(dev, dma_buf, gm12u320->dmadev); | ||
| 619 | } | ||
| 620 | |||
| 604 | DEFINE_DRM_GEM_FOPS(gm12u320_fops); | 621 | DEFINE_DRM_GEM_FOPS(gm12u320_fops); |
| 605 | 622 | ||
| 606 | static const struct drm_driver gm12u320_drm_driver = { | 623 | static const struct drm_driver gm12u320_drm_driver = { |
| @@ -614,6 +631,7 @@ static const struct drm_driver gm12u320_drm_driver = { · 7 unmodified lines | |||
| 614 | 631 | ||
| 615 | .fops = &gm12u320_fops, | 632 | .fops = &gm12u320_fops, |
| 616 | DRM_GEM_SHMEM_DRIVER_OPS, | 633 | DRM_GEM_SHMEM_DRIVER_OPS, |
| 634 | .gem_prime_import = gm12u320_gem_prime_import, | ||
| 617 | }; | 635 | }; |
| 618 | 636 | ||
| 619 | static const struct drm_mode_config_funcs gm12u320_mode_config_funcs = { | 637 | static const struct drm_mode_config_funcs gm12u320_mode_config_funcs = { |
| @@ -640,15 +658,18 @@ static int gm12u320_usb_probe(struct usb_interface *interface, · 20 unmodified lines | |||
| 640 | struct gm12u320_device, dev); | 658 | struct gm12u320_device, dev); |
| 641 | if (IS_ERR(gm12u320)) | 659 | if (IS_ERR(gm12u320)) |
| 642 | return PTR_ERR(gm12u320); | 660 | return PTR_ERR(gm12u320); |
| 661 | dev = &gm12u320->dev; | ||
| 662 | |||
| 663 | gm12u320->dmadev = usb_intf_get_dma_device(to_usb_interface(dev->dev)); | ||
| 664 | if (!gm12u320->dmadev) | ||
| 665 | drm_warn(dev, "buffer sharing not supported"); /* not an error */ | ||
| 643 | 666 | ||
| 644 | INIT_DELAYED_WORK(&gm12u320->fb_update.work, gm12u320_fb_update_work); | 667 | INIT_DELAYED_WORK(&gm12u320->fb_update.work, gm12u320_fb_update_work); |
| 645 | mutex_init(&gm12u320->fb_update.lock); | 668 | mutex_init(&gm12u320->fb_update.lock); |
| 646 | 669 | ||
| 647 | dev = &gm12u320->dev; | ||
| 648 | |||
| 649 | ret = drmm_mode_config_init(dev); | 670 | ret = drmm_mode_config_init(dev); |
| 650 | if (ret) | 671 | if (ret) |
| 651 | return ret; | 672 | goto err_put_device; |
| 652 | 673 | ||
| 653 | dev->mode_config.min_width = GM12U320_USER_WIDTH; | 674 | dev->mode_config.min_width = GM12U320_USER_WIDTH; |
| 654 | dev->mode_config.max_width = GM12U320_USER_WIDTH; | 675 | dev->mode_config.max_width = GM12U320_USER_WIDTH; |
| @@ -658,15 +679,15 @@ static int gm12u320_usb_probe(struct usb_interface *interface, · 3 unmodified lines | |||
| 658 | 679 | ||
| 659 | ret = gm12u320_usb_alloc(gm12u320); | 680 | ret = gm12u320_usb_alloc(gm12u320); |
| 660 | if (ret) | 681 | if (ret) |
| 661 | return ret; | 682 | goto err_put_device; |
| 662 | 683 | ||
| 663 | ret = gm12u320_set_ecomode(gm12u320); | 684 | ret = gm12u320_set_ecomode(gm12u320); |
| 664 | if (ret) | 685 | if (ret) |
| 665 | return ret; | 686 | goto err_put_device; |
| 666 | 687 | ||
| 667 | ret = gm12u320_conn_init(gm12u320); | 688 | ret = gm12u320_conn_init(gm12u320); |
| 668 | if (ret) | 689 | if (ret) |
| 669 | return ret; | 690 | goto err_put_device; |
| 670 | 691 | ||
| 671 | ret = drm_simple_display_pipe_init(&gm12u320->dev, | 692 | ret = drm_simple_display_pipe_init(&gm12u320->dev, |
| 672 | &gm12u320->pipe, | 693 | &gm12u320->pipe, |
| @@ -676,24 +697,31 @@ static int gm12u320_usb_probe(struct usb_interface *interface, · 3 unmodified lines | |||
| 676 | gm12u320_pipe_modifiers, | 697 | gm12u320_pipe_modifiers, |
| 677 | &gm12u320->conn); | 698 | &gm12u320->conn); |
| 678 | if (ret) | 699 | if (ret) |
| 679 | return ret; | 700 | goto err_put_device; |
| 680 | 701 | ||
| 681 | drm_mode_config_reset(dev); | 702 | drm_mode_config_reset(dev); |
| 682 | 703 | ||
| 683 | usb_set_intfdata(interface, dev); | 704 | usb_set_intfdata(interface, dev); |
| 684 | ret = drm_dev_register(dev, 0); | 705 | ret = drm_dev_register(dev, 0); |
| 685 | if (ret) | 706 | if (ret) |
| 686 | return ret; | 707 | goto err_put_device; |
| 687 | 708 | ||
| 688 | drm_fbdev_generic_setup(dev, 0); | 709 | drm_fbdev_generic_setup(dev, 0); |
| 689 | 710 | ||
| 690 | return 0; | 711 | return 0; |
| 712 | |||
| 713 | err_put_device: | ||
| 714 | put_device(gm12u320->dmadev); | ||
| 715 | return ret; | ||
| 691 | } | 716 | } |
| 692 | 717 | ||
| 693 | static void gm12u320_usb_disconnect(struct usb_interface *interface) | 718 | static void gm12u320_usb_disconnect(struct usb_interface *interface) |
| 694 | { | 719 | { |
| 695 | struct drm_device *dev = usb_get_intfdata(interface); | 720 | struct drm_device *dev = usb_get_intfdata(interface); |
| 721 | struct gm12u320_device *gm12u320 = to_gm12u320(dev); | ||
| 696 | 722 | ||
| 723 | put_device(gm12u320->dmadev); | ||
| 724 | gm12u320->dmadev = NULL; | ||
| 697 | drm_dev_unplug(dev); | 725 | drm_dev_unplug(dev); |
| 698 | drm_atomic_helper_shutdown(dev); | 726 | drm_atomic_helper_shutdown(dev); |
| 699 | } | 727 | } |
| @@ -136,7 +136,8 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, · 135 unmodified lines | |||
| 136 | struct ttm_bo_device *bdev = bo->bdev; | 136 | struct ttm_bo_device *bdev = bo->bdev; |
| 137 | struct ttm_resource_manager *man; | 137 | struct ttm_resource_manager *man; |
| 138 | 138 | ||
| 139 | dma_resv_assert_held(bo->base.resv); | 139 | if (!bo->deleted) |
| 140 | dma_resv_assert_held(bo->base.resv); | ||
| 140 | 141 | ||
| 141 | if (bo->pin_count) { | 142 | if (bo->pin_count) { |
| 142 | ttm_bo_del_from_lru(bo); | 143 | ttm_bo_del_from_lru(bo); |
| @@ -508,8 +509,11 @@ static void ttm_bo_release(struct kref *kref) · 365 unmodified lines | |||
| 508 | * Make pinned bos immediately available to | 509 | * Make pinned bos immediately available to |
| 509 | * shrinkers, now that they are queued for | 510 | * shrinkers, now that they are queued for |
| 510 | * destruction. | 511 | * destruction. |
| 512 | * | ||
| 513 | * FIXME: QXL is triggering this. Can be removed when the | ||
| 514 | * driver is fixed. | ||
| 511 | */ | 515 | */ |
| 512 | if (WARN_ON(bo->pin_count)) { | 516 | if (WARN_ON_ONCE(bo->pin_count)) { |
| 513 | bo->pin_count = 0; | 517 | bo->pin_count = 0; |
| 514 | ttm_bo_move_to_lru_tail(bo, &bo->mem, NULL); | 518 | ttm_bo_move_to_lru_tail(bo, &bo->mem, NULL); |
| 515 | } | 519 | } |
| @@ -268,13 +268,13 @@ static void ttm_pool_type_init(struct ttm_pool_type *pt, struct ttm_pool *pool, · 267 unmodified lines | |||
| 268 | /* Remove a pool_type from the global shrinker list and free all pages */ | 268 | /* Remove a pool_type from the global shrinker list and free all pages */ |
| 269 | static void ttm_pool_type_fini(struct ttm_pool_type *pt) | 269 | static void ttm_pool_type_fini(struct ttm_pool_type *pt) |
| 270 | { | 270 | { |
| 271 | struct page *p, *tmp; | 271 | struct page *p; |
| 272 | 272 | ||
| 273 | mutex_lock(&shrinker_lock); | 273 | mutex_lock(&shrinker_lock); |
| 274 | list_del(&pt->shrinker_list); | 274 | list_del(&pt->shrinker_list); |
| 275 | mutex_unlock(&shrinker_lock); | 275 | mutex_unlock(&shrinker_lock); |
| 276 | 276 | ||
| 277 | list_for_each_entry_safe(p, tmp, &pt->pages, lru) | 277 | while ((p = ttm_pool_type_take(pt))) |
| 278 | ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); | 278 | ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); |
| 279 | } | 279 | } |
| 280 | 280 | ||
| @@ -32,6 +32,22 @@ static int udl_usb_resume(struct usb_interface *interface) · 31 unmodified lines | |||
| 32 | return drm_mode_config_helper_resume(dev); | 32 | return drm_mode_config_helper_resume(dev); |
| 33 | } | 33 | } |
| 34 | 34 | ||
| 35 | /* | ||
| 36 | * FIXME: Dma-buf sharing requires DMA support by the importing device. | ||
| 37 | * This function is a workaround to make USB devices work as well. | ||
| 38 | * See todo.rst for how to fix the issue in the dma-buf framework. | ||
| 39 | */ | ||
| 40 | static struct drm_gem_object *udl_driver_gem_prime_import(struct drm_device *dev, | ||
| 41 | struct dma_buf *dma_buf) | ||
| 42 | { | ||
| 43 | struct udl_device *udl = to_udl(dev); | ||
| 44 | |||
| 45 | if (!udl->dmadev) | ||
| 46 | return ERR_PTR(-ENODEV); | ||
| 47 | |||
| 48 | return drm_gem_prime_import_dev(dev, dma_buf, udl->dmadev); | ||
| 49 | } | ||
| 50 | |||
| 35 | DEFINE_DRM_GEM_FOPS(udl_driver_fops); | 51 | DEFINE_DRM_GEM_FOPS(udl_driver_fops); |
| 36 | 52 | ||
| 37 | static const struct drm_driver driver = { | 53 | static const struct drm_driver driver = { |
| @@ -40,6 +56,7 @@ static const struct drm_driver driver = { · 2 unmodified lines | |||
| 40 | /* GEM hooks */ | 56 | /* GEM hooks */ |
| 41 | .fops = &udl_driver_fops, | 57 | .fops = &udl_driver_fops, |
| 42 | DRM_GEM_SHMEM_DRIVER_OPS, | 58 | DRM_GEM_SHMEM_DRIVER_OPS, |
| 59 | .gem_prime_import = udl_driver_gem_prime_import, | ||
| 43 | 60 | ||
| 44 | .name = DRIVER_NAME, | 61 | .name = DRIVER_NAME, |
| 45 | .desc = DRIVER_DESC, | 62 | .desc = DRIVER_DESC, |
| @@ -50,6 +50,7 @@ struct urb_list { · 49 unmodified lines | |||
| 50 | struct udl_device { | 50 | struct udl_device { |
| 51 | struct drm_device drm; | 51 | struct drm_device drm; |
| 52 | struct device *dev; | 52 | struct device *dev; |
| 53 | struct device *dmadev; | ||
| 53 | 54 | ||
| 54 | struct drm_simple_display_pipe display_pipe; | 55 | struct drm_simple_display_pipe display_pipe; |
| 55 | 56 | ||
| @@ -315,6 +315,10 @@ int udl_init(struct udl_device *udl) · 314 unmodified lines | |||
| 315 | 315 | ||
| 316 | DRM_DEBUG("\n"); | 316 | DRM_DEBUG("\n"); |
| 317 | 317 | ||
| 318 | udl->dmadev = usb_intf_get_dma_device(to_usb_interface(dev->dev)); | ||
| 319 | if (!udl->dmadev) | ||
| 320 | drm_warn(dev, "buffer sharing not supported"); /* not an error */ | ||
| 321 | |||
| 318 | mutex_init(&udl->gem_lock); | 322 | mutex_init(&udl->gem_lock); |
| 319 | 323 | ||
| 320 | if (!udl_parse_vendor_descriptor(udl)) { | 324 | if (!udl_parse_vendor_descriptor(udl)) { |
| @@ -343,12 +347,18 @@ int udl_init(struct udl_device *udl) · 22 unmodified lines | |||
| 343 | err: | 347 | err: |
| 344 | if (udl->urbs.count) | 348 | if (udl->urbs.count) |
| 345 | udl_free_urb_list(dev); | 349 | udl_free_urb_list(dev); |
| 350 | put_device(udl->dmadev); | ||
| 346 | DRM_ERROR("%d\n", ret); | 351 | DRM_ERROR("%d\n", ret); |
| 347 | return ret; | 352 | return ret; |
| 348 | } | 353 | } |
| 349 | 354 | ||
| 350 | int udl_drop_usb(struct drm_device *dev) | 355 | int udl_drop_usb(struct drm_device *dev) |
| 351 | { | 356 | { |
| 357 | struct udl_device *udl = to_udl(dev); | ||
| 358 | |||
| 352 | udl_free_urb_list(dev); | 359 | udl_free_urb_list(dev); |
| 360 | put_device(udl->dmadev); | ||
| 361 | udl->dmadev = NULL; | ||
| 362 | |||
| 353 | return 0; | 363 | return 0; |
| 354 | } | 364 | } |
| @@ -500,8 +500,6 @@ vm_fault_t vmw_bo_vm_huge_fault(struct vm_fault *vmf, · 499 unmodified lines | |||
| 500 | vm_fault_t ret; | 500 | vm_fault_t ret; |
| 501 | pgoff_t fault_page_size; | 501 | pgoff_t fault_page_size; |
| 502 | bool write = vmf->flags & FAULT_FLAG_WRITE; | 502 | bool write = vmf->flags & FAULT_FLAG_WRITE; |
| 503 | bool is_cow_mapping = | ||
| 504 | (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; | ||
| 505 | 503 | ||
| 506 | switch (pe_size) { | 504 | switch (pe_size) { |
| 507 | case PE_SIZE_PMD: | 505 | case PE_SIZE_PMD: |
| @@ -518,7 +516,7 @@ vm_fault_t vmw_bo_vm_huge_fault(struct vm_fault *vmf, · 10 unmodified lines | |||
| 518 | } | 516 | } |
| 519 | 517 | ||
| 520 | /* Always do write dirty-tracking and COW on PTE level. */ | 518 | /* Always do write dirty-tracking and COW on PTE level. */ |
| 521 | if (write && (READ_ONCE(vbo->dirty) || is_cow_mapping)) | 519 | if (write && (READ_ONCE(vbo->dirty) || is_cow_mapping(vma->vm_flags))) |
| 522 | return VM_FAULT_FALLBACK; | 520 | return VM_FAULT_FALLBACK; |
| 523 | 521 | ||
| 524 | ret = ttm_bo_vm_reserve(bo, vmf); | 522 | ret = ttm_bo_vm_reserve(bo, vmf); |
| @@ -49,7 +49,7 @@ int vmw_mmap(struct file *filp, struct vm_area_struct *vma) · 48 unmodified lines | |||
| 49 | vma->vm_ops = &vmw_vm_ops; | 49 | vma->vm_ops = &vmw_vm_ops; |
| 50 | 50 | ||
| 51 | /* Use VM_PFNMAP rather than VM_MIXEDMAP if not a COW mapping */ | 51 | /* Use VM_PFNMAP rather than VM_MIXEDMAP if not a COW mapping */ |
| 52 | if ((vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) != VM_MAYWRITE) | 52 | if (!is_cow_mapping(vma->vm_flags)) |
| 53 | vma->vm_flags = (vma->vm_flags & ~VM_MIXEDMAP) | VM_PFNMAP; | 53 | vma->vm_flags = (vma->vm_flags & ~VM_MIXEDMAP) | VM_PFNMAP; |
| 54 | 54 | ||
| 55 | return 0; | 55 | return 0; |
| @@ -8,7 +8,6 @@ config IRQCHIP · 7 unmodified lines | |||
| 8 | config ARM_GIC | 8 | config ARM_GIC |
| 9 | bool | 9 | bool |
| 10 | select IRQ_DOMAIN_HIERARCHY | 10 | select IRQ_DOMAIN_HIERARCHY |
| 11 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 12 | select GENERIC_IRQ_EFFECTIVE_AFF_MASK | 11 | select GENERIC_IRQ_EFFECTIVE_AFF_MASK |
| 13 | 12 | ||
| 14 | config ARM_GIC_PM | 13 | config ARM_GIC_PM |
| @@ -33,7 +32,6 @@ config GIC_NON_BANKED · 18 unmodified lines | |||
| 33 | 32 | ||
| 34 | config ARM_GIC_V3 | 33 | config ARM_GIC_V3 |
| 35 | bool | 34 | bool |
| 36 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 37 | select IRQ_DOMAIN_HIERARCHY | 35 | select IRQ_DOMAIN_HIERARCHY |
| 38 | select PARTITION_PERCPU | 36 | select PARTITION_PERCPU |
| 39 | select GENERIC_IRQ_EFFECTIVE_AFF_MASK | 37 | select GENERIC_IRQ_EFFECTIVE_AFF_MASK |
| @@ -64,7 +62,6 @@ config ARM_NVIC · 24 unmodified lines | |||
| 64 | config ARM_VIC | 62 | config ARM_VIC |
| 65 | bool | 63 | bool |
| 66 | select IRQ_DOMAIN | 64 | select IRQ_DOMAIN |
| 67 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 68 | 65 | ||
| 69 | config ARM_VIC_NR | 66 | config ARM_VIC_NR |
| 70 | int | 67 | int |
| @@ -99,14 +96,12 @@ config ATMEL_AIC_IRQ · 28 unmodified lines | |||
| 99 | bool | 96 | bool |
| 100 | select GENERIC_IRQ_CHIP | 97 | select GENERIC_IRQ_CHIP |
| 101 | select IRQ_DOMAIN | 98 | select IRQ_DOMAIN |
| 102 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 103 | select SPARSE_IRQ | 99 | select SPARSE_IRQ |
| 104 | 100 | ||
| 105 | config ATMEL_AIC5_IRQ | 101 | config ATMEL_AIC5_IRQ |
| 106 | bool | 102 | bool |
| 107 | select GENERIC_IRQ_CHIP | 103 | select GENERIC_IRQ_CHIP |
| 108 | select IRQ_DOMAIN | 104 | select IRQ_DOMAIN |
| 109 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 110 | select SPARSE_IRQ | 105 | select SPARSE_IRQ |
| 111 | 106 | ||
| 112 | config I8259 | 107 | config I8259 |
| @@ -153,7 +148,6 @@ config DW_APB_ICTL · 40 unmodified lines | |||
| 153 | config FARADAY_FTINTC010 | 148 | config FARADAY_FTINTC010 |
| 154 | bool | 149 | bool |
| 155 | select IRQ_DOMAIN | 150 | select IRQ_DOMAIN |
| 156 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 157 | select SPARSE_IRQ | 151 | select SPARSE_IRQ |
| 158 | 152 | ||
| 159 | config HISILICON_IRQ_MBIGEN | 153 | config HISILICON_IRQ_MBIGEN |
| @@ -169,7 +163,6 @@ config IMGPDC_IRQ · 9 unmodified lines | |||
| 169 | config IXP4XX_IRQ | 163 | config IXP4XX_IRQ |
| 170 | bool | 164 | bool |
| 171 | select IRQ_DOMAIN | 165 | select IRQ_DOMAIN |
| 172 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 173 | select SPARSE_IRQ | 166 | select SPARSE_IRQ |
| 174 | 167 | ||
| 175 | config MADERA_IRQ | 168 | config MADERA_IRQ |
| @@ -186,7 +179,6 @@ config CLPS711X_IRQCHIP · 10 unmodified lines | |||
| 186 | bool | 179 | bool |
| 187 | depends on ARCH_CLPS711X | 180 | depends on ARCH_CLPS711X |
| 188 | select IRQ_DOMAIN | 181 | select IRQ_DOMAIN |
| 189 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 190 | select SPARSE_IRQ | 182 | select SPARSE_IRQ |
| 191 | default y | 183 | default y |
| 192 | 184 | ||
| @@ -205,7 +197,6 @@ config OMAP_IRQCHIP · 12 unmodified lines | |||
| 205 | config ORION_IRQCHIP | 197 | config ORION_IRQCHIP |
| 206 | bool | 198 | bool |
| 207 | select IRQ_DOMAIN | 199 | select IRQ_DOMAIN |
| 208 | select GENERIC_IRQ_MULTI_HANDLER | ||
| 209 | 200 | ||
| 210 | config PIC32_EVIC | 201 | config PIC32_EVIC |
| 211 | bool | 202 | bool |
| @@ -179,5 +179,6 @@ err_free_tcu: · 178 unmodified lines | |||
| 179 | } | 179 | } |
| 180 | IRQCHIP_DECLARE(jz4740_tcu_irq, "ingenic,jz4740-tcu", ingenic_tcu_irq_init); | 180 | IRQCHIP_DECLARE(jz4740_tcu_irq, "ingenic,jz4740-tcu", ingenic_tcu_irq_init); |
| 181 | IRQCHIP_DECLARE(jz4725b_tcu_irq, "ingenic,jz4725b-tcu", ingenic_tcu_irq_init); | 181 | IRQCHIP_DECLARE(jz4725b_tcu_irq, "ingenic,jz4725b-tcu", ingenic_tcu_irq_init); |
| 182 | IRQCHIP_DECLARE(jz4760_tcu_irq, "ingenic,jz4760-tcu", ingenic_tcu_irq_init); | ||
| 182 | IRQCHIP_DECLARE(jz4770_tcu_irq, "ingenic,jz4770-tcu", ingenic_tcu_irq_init); | 183 | IRQCHIP_DECLARE(jz4770_tcu_irq, "ingenic,jz4770-tcu", ingenic_tcu_irq_init); |
| 183 | IRQCHIP_DECLARE(x1000_tcu_irq, "ingenic,x1000-tcu", ingenic_tcu_irq_init); | 184 | IRQCHIP_DECLARE(x1000_tcu_irq, "ingenic,x1000-tcu", ingenic_tcu_irq_init); |
| @@ -155,6 +155,7 @@ static int __init intc_2chip_of_init(struct device_node *node, · 154 unmodified lines | |||
| 155 | { | 155 | { |
| 156 | return ingenic_intc_of_init(node, 2); | 156 | return ingenic_intc_of_init(node, 2); |
| 157 | } | 157 | } |
| 158 | IRQCHIP_DECLARE(jz4760_intc, "ingenic,jz4760-intc", intc_2chip_of_init); | ||
| 158 | IRQCHIP_DECLARE(jz4770_intc, "ingenic,jz4770-intc", intc_2chip_of_init); | 159 | IRQCHIP_DECLARE(jz4770_intc, "ingenic,jz4770-intc", intc_2chip_of_init); |
| 159 | IRQCHIP_DECLARE(jz4775_intc, "ingenic,jz4775-intc", intc_2chip_of_init); | 160 | IRQCHIP_DECLARE(jz4775_intc, "ingenic,jz4775-intc", intc_2chip_of_init); |
| 160 | IRQCHIP_DECLARE(jz4780_intc, "ingenic,jz4780-intc", intc_2chip_of_init); | 161 | IRQCHIP_DECLARE(jz4780_intc, "ingenic,jz4780-intc", intc_2chip_of_init); |
| @@ -51,10 +51,8 @@ static ssize_t ttyname_store(struct device *dev, · 50 unmodified lines | |||
| 51 | 51 | ||
| 52 | if (size) { | 52 | if (size) { |
| 53 | ttyname = kmemdup_nul(buf, size, GFP_KERNEL); | 53 | ttyname = kmemdup_nul(buf, size, GFP_KERNEL); |
| 54 | if (!ttyname) { | 54 | if (!ttyname) |
| 55 | ret = -ENOMEM; | 55 | return -ENOMEM; |
| 56 | goto out_unlock; | ||
| 57 | } | ||
| 58 | } else { | 56 | } else { |
| 59 | ttyname = NULL; | 57 | ttyname = NULL; |
| 60 | } | 58 | } |
| @@ -69,7 +67,6 @@ static ssize_t ttyname_store(struct device *dev, · 8 unmodified lines | |||
| 69 | 67 | ||
| 70 | trigger_data->ttyname = ttyname; | 68 | trigger_data->ttyname = ttyname; |
| 71 | 69 | ||
| 72 | out_unlock: | ||
| 73 | mutex_unlock(&trigger_data->mutex); | 70 | mutex_unlock(&trigger_data->mutex); |
| 74 | 71 | ||
| 75 | if (ttyname && !running) | 72 | if (ttyname && !running) |
| @@ -125,12 +122,12 @@ static void ledtrig_tty_work(struct work_struct *work) · 49 unmodified lines | |||
| 125 | 122 | ||
| 126 | if (icount.rx != trigger_data->rx || | 123 | if (icount.rx != trigger_data->rx || |
| 127 | icount.tx != trigger_data->tx) { | 124 | icount.tx != trigger_data->tx) { |
| 128 | led_set_brightness(trigger_data->led_cdev, LED_ON); | 125 | led_set_brightness_sync(trigger_data->led_cdev, LED_ON); |
| 129 | 126 | ||
| 130 | trigger_data->rx = icount.rx; | 127 | trigger_data->rx = icount.rx; |
| 131 | trigger_data->tx = icount.tx; | 128 | trigger_data->tx = icount.tx; |
| 132 | } else { | 129 | } else { |
| 133 | led_set_brightness(trigger_data->led_cdev, LED_OFF); | 130 | led_set_brightness_sync(trigger_data->led_cdev, LED_OFF); |
| 134 | } | 131 | } |
| 135 | 132 | ||
| 136 | out: | 133 | out: |
| @@ -965,7 +965,7 @@ static int bcache_device_init(struct bcache_device *d, unsigned int block_size, · 964 unmodified lines | |||
| 965 | q->limits.max_hw_sectors = UINT_MAX; | 965 | q->limits.max_hw_sectors = UINT_MAX; |
| 966 | q->limits.max_sectors = UINT_MAX; | 966 | q->limits.max_sectors = UINT_MAX; |
| 967 | q->limits.max_segment_size = UINT_MAX; | 967 | q->limits.max_segment_size = UINT_MAX; |
| 968 | q->limits.max_segments = BIO_MAX_PAGES; | 968 | q->limits.max_segments = BIO_MAX_VECS; |
| 969 | blk_queue_max_discard_sectors(q, UINT_MAX); | 969 | blk_queue_max_discard_sectors(q, UINT_MAX); |
| 970 | q->limits.discard_granularity = 512; | 970 | q->limits.discard_granularity = 512; |
| 971 | q->limits.io_min = block_size; | 971 | q->limits.io_min = block_size; |
| @@ -229,7 +229,7 @@ static DEFINE_SPINLOCK(dm_crypt_clients_lock); · 228 unmodified lines | |||
| 229 | static unsigned dm_crypt_clients_n = 0; | 229 | static unsigned dm_crypt_clients_n = 0; |
| 230 | static volatile unsigned long dm_crypt_pages_per_client; | 230 | static volatile unsigned long dm_crypt_pages_per_client; |
| 231 | #define DM_CRYPT_MEMORY_PERCENT 2 | 231 | #define DM_CRYPT_MEMORY_PERCENT 2 |
| 232 | #define DM_CRYPT_MIN_PAGES_PER_CLIENT (BIO_MAX_PAGES * 16) | 232 | #define DM_CRYPT_MIN_PAGES_PER_CLIENT (BIO_MAX_VECS * 16) |
| 233 | 233 | ||
| 234 | static void clone_init(struct dm_crypt_io *, struct bio *); | 234 | static void clone_init(struct dm_crypt_io *, struct bio *); |
| 235 | static void kcryptd_queue_crypt(struct dm_crypt_io *io); | 235 | static void kcryptd_queue_crypt(struct dm_crypt_io *io); |
| @@ -3246,7 +3246,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) · 3010 unmodified lines | |||
| 3246 | ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, | 3246 | ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, |
| 3247 | ARCH_KMALLOC_MINALIGN); | 3247 | ARCH_KMALLOC_MINALIGN); |
| 3248 | 3248 | ||
| 3249 | ret = mempool_init(&cc->page_pool, BIO_MAX_PAGES, crypt_page_alloc, crypt_page_free, cc); | 3249 | ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); |
| 3250 | if (ret) { | 3250 | if (ret) { |
| 3251 | ti->error = "Cannot allocate page mempool"; | 3251 | ti->error = "Cannot allocate page mempool"; |
| 3252 | goto bad; | 3252 | goto bad; |
| @@ -3373,9 +3373,9 @@ static int crypt_map(struct dm_target *ti, struct bio *bio) · 120 unmodified lines | |||
| 3373 | /* | 3373 | /* |
| 3374 | * Check if bio is too large, split as needed. | 3374 | * Check if bio is too large, split as needed. |
| 3375 | */ | 3375 | */ |
| 3376 | if (unlikely(bio->bi_iter.bi_size > (BIO_MAX_PAGES << PAGE_SHIFT)) && | 3376 | if (unlikely(bio->bi_iter.bi_size > (BIO_MAX_VECS << PAGE_SHIFT)) && |
| 3377 | (bio_data_dir(bio) == WRITE || cc->on_disk_tag_size)) | 3377 | (bio_data_dir(bio) == WRITE || cc->on_disk_tag_size)) |
| 3378 | dm_accept_partial_bio(bio, ((BIO_MAX_PAGES << PAGE_SHIFT) >> SECTOR_SHIFT)); | 3378 | dm_accept_partial_bio(bio, ((BIO_MAX_VECS << PAGE_SHIFT) >> SECTOR_SHIFT)); |
| 3379 | 3379 | ||
| 3380 | /* | 3380 | /* |
| 3381 | * Ensure that bio is a multiple of internal sector encryption size | 3381 | * Ensure that bio is a multiple of internal sector encryption size |
| @@ -1892,10 +1892,10 @@ restart: · 1891 unmodified lines | |||
| 1892 | list_add(&g->lru, &wbl.list); | 1892 | list_add(&g->lru, &wbl.list); |
| 1893 | wbl.size++; | 1893 | wbl.size++; |
| 1894 | g->write_in_progress = true; | 1894 | g->write_in_progress = true; |
| 1895 | g->wc_list_contiguous = BIO_MAX_PAGES; | 1895 | g->wc_list_contiguous = BIO_MAX_VECS; |
| 1896 | f = g; | 1896 | f = g; |
| 1897 | e->wc_list_contiguous++; | 1897 | e->wc_list_contiguous++; |
| 1898 | if (unlikely(e->wc_list_contiguous == BIO_MAX_PAGES)) { | 1898 | if (unlikely(e->wc_list_contiguous == BIO_MAX_VECS)) { |
| 1899 | if (unlikely(wc->writeback_all)) { | 1899 | if (unlikely(wc->writeback_all)) { |
| 1900 | next_node = rb_next(&f->rb_node); | 1900 | next_node = rb_next(&f->rb_node); |
| 1901 | if (likely(next_node)) | 1901 | if (likely(next_node)) |
| @@ -735,7 +735,7 @@ static void r5l_submit_current_io(struct r5l_log *log) · 734 unmodified lines | |||
| 735 | 735 | ||
| 736 | static struct bio *r5l_bio_alloc(struct r5l_log *log) | 736 | static struct bio *r5l_bio_alloc(struct r5l_log *log) |
| 737 | { | 737 | { |
| 738 | struct bio *bio = bio_alloc_bioset(GFP_NOIO, BIO_MAX_PAGES, &log->bs); | 738 | struct bio *bio = bio_alloc_bioset(GFP_NOIO, BIO_MAX_VECS, &log->bs); |
| 739 | 739 | ||
| 740 | bio_set_op_attrs(bio, REQ_OP_WRITE, 0); | 740 | bio_set_op_attrs(bio, REQ_OP_WRITE, 0); |
| 741 | bio_set_dev(bio, log->rdev->bdev); | 741 | bio_set_dev(bio, log->rdev->bdev); |
| @@ -1634,7 +1634,7 @@ static int r5l_recovery_allocate_ra_pool(struct r5l_log *log, · 892 unmodified lines | |||
| 1634 | { | 1634 | { |
| 1635 | struct page *page; | 1635 | struct page *page; |
| 1636 | 1636 | ||
| 1637 | ctx->ra_bio = bio_alloc_bioset(GFP_KERNEL, BIO_MAX_PAGES, &log->bs); | 1637 | ctx->ra_bio = bio_alloc_bioset(GFP_KERNEL, BIO_MAX_VECS, &log->bs); |
| 1638 | if (!ctx->ra_bio) | 1638 | if (!ctx->ra_bio) |
| 1639 | return -ENOMEM; | 1639 | return -ENOMEM; |
| 1640 | 1640 | ||
| @@ -496,7 +496,7 @@ static void ppl_submit_iounit(struct ppl_io_unit *io) · 495 unmodified lines | |||
| 496 | if (!bio_add_page(bio, sh->ppl_page, PAGE_SIZE, 0)) { | 496 | if (!bio_add_page(bio, sh->ppl_page, PAGE_SIZE, 0)) { |
| 497 | struct bio *prev = bio; | 497 | struct bio *prev = bio; |
| 498 | 498 | ||
| 499 | bio = bio_alloc_bioset(GFP_NOIO, BIO_MAX_PAGES, | 499 | bio = bio_alloc_bioset(GFP_NOIO, BIO_MAX_VECS, |
| 500 | &ppl_conf->bs); | 500 | &ppl_conf->bs); |
| 501 | bio->bi_opf = prev->bi_opf; | 501 | bio->bi_opf = prev->bi_opf; |
| 502 | bio->bi_write_hint = prev->bi_write_hint; | 502 | bio->bi_write_hint = prev->bi_write_hint; |
| @@ -1288,7 +1288,6 @@ static void rkisp1_params_config_parameter(struct rkisp1_params *params) · 1287 unmodified lines | |||
| 1288 | memset(hst.hist_weight, 0x01, sizeof(hst.hist_weight)); | 1288 | memset(hst.hist_weight, 0x01, sizeof(hst.hist_weight)); |
| 1289 | rkisp1_hst_config(params, &hst); | 1289 | rkisp1_hst_config(params, &hst); |
| 1290 | rkisp1_param_set_bits(params, RKISP1_CIF_ISP_HIST_PROP, | 1290 | rkisp1_param_set_bits(params, RKISP1_CIF_ISP_HIST_PROP, |
| 1291 | ~RKISP1_CIF_ISP_HIST_PROP_MODE_MASK | | ||
| 1292 | rkisp1_hst_params_default_config.mode); | 1291 | rkisp1_hst_params_default_config.mode); |
| 1293 | 1292 | ||
| 1294 | /* set the range */ | 1293 | /* set the range */ |
| @@ -245,7 +245,7 @@ static int vsp1_du_pipeline_setup_brx(struct vsp1_device *vsp1, · 244 unmodified lines | |||
| 245 | brx = &vsp1->bru->entity; | 245 | brx = &vsp1->bru->entity; |
| 246 | else if (pipe->brx && !drm_pipe->force_brx_release) | 246 | else if (pipe->brx && !drm_pipe->force_brx_release) |
| 247 | brx = pipe->brx; | 247 | brx = pipe->brx; |
| 248 | else if (!vsp1->bru->entity.pipe) | 248 | else if (vsp1_feature(vsp1, VSP1_HAS_BRU) && !vsp1->bru->entity.pipe) |
| 249 | brx = &vsp1->bru->entity; | 249 | brx = &vsp1->bru->entity; |
| 250 | else | 250 | else |
| 251 | brx = &vsp1->brs->entity; | 251 | brx = &vsp1->brs->entity; |
| @@ -462,9 +462,9 @@ static int vsp1_du_pipeline_setup_inputs(struct vsp1_device *vsp1, · 210 unmodified lines | |||
| 462 | * make sure it is present in the pipeline's list of entities if it | 462 | * make sure it is present in the pipeline's list of entities if it |
| 463 | * wasn't already. | 463 | * wasn't already. |
| 464 | */ | 464 | */ |
| 465 | if (!use_uif) { | 465 | if (drm_pipe->uif && !use_uif) { |
| 466 | drm_pipe->uif->pipe = NULL; | 466 | drm_pipe->uif->pipe = NULL; |
| 467 | } else if (!drm_pipe->uif->pipe) { | 467 | } else if (drm_pipe->uif && !drm_pipe->uif->pipe) { |
| 468 | drm_pipe->uif->pipe = pipe; | 468 | drm_pipe->uif->pipe = pipe; |
| 469 | list_add_tail(&drm_pipe->uif->list_pipe, &pipe->entities); | 469 | list_add_tail(&drm_pipe->uif->list_pipe, &pipe->entities); |
| 470 | } | 470 | } |
| @@ -5,6 +5,7 @@ obj-y += keymaps/ · 4 unmodified lines | |||
| 5 | obj-$(CONFIG_RC_CORE) += rc-core.o | 5 | obj-$(CONFIG_RC_CORE) += rc-core.o |
| 6 | rc-core-y := rc-main.o rc-ir-raw.o | 6 | rc-core-y := rc-main.o rc-ir-raw.o |
| 7 | rc-core-$(CONFIG_LIRC) += lirc_dev.o | 7 | rc-core-$(CONFIG_LIRC) += lirc_dev.o |
| 8 | rc-core-$(CONFIG_MEDIA_CEC_RC) += keymaps/rc-cec.o | ||
| 8 | rc-core-$(CONFIG_BPF_LIRC_MODE2) += bpf-lirc.o | 9 | rc-core-$(CONFIG_BPF_LIRC_MODE2) += bpf-lirc.o |
| 9 | obj-$(CONFIG_IR_NEC_DECODER) += ir-nec-decoder.o | 10 | obj-$(CONFIG_IR_NEC_DECODER) += ir-nec-decoder.o |
| 10 | obj-$(CONFIG_IR_RC5_DECODER) += ir-rc5-decoder.o | 11 | obj-$(CONFIG_IR_RC5_DECODER) += ir-rc5-decoder.o |
| @@ -21,7 +21,6 @@ obj-$(CONFIG_RC_MAP) += rc-adstech-dvb-t-pci.o \ · 20 unmodified lines | |||
| 21 | rc-behold.o \ | 21 | rc-behold.o \ |
| 22 | rc-behold-columbus.o \ | 22 | rc-behold-columbus.o \ |
| 23 | rc-budget-ci-old.o \ | 23 | rc-budget-ci-old.o \ |
| 24 | rc-cec.o \ | ||
| 25 | rc-cinergy-1400.o \ | 24 | rc-cinergy-1400.o \ |
| 26 | rc-cinergy.o \ | 25 | rc-cinergy.o \ |
| 27 | rc-d680-dmb.o \ | 26 | rc-d680-dmb.o \ |
| @@ -1,5 +1,15 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0-or-later | 1 | // SPDX-License-Identifier: GPL-2.0-or-later |
| 2 | /* Keytable for the CEC remote control | 2 | /* Keytable for the CEC remote control |
| 3 | * | ||
| 4 | * This keymap is unusual in that it can't be built as a module, | ||
| 5 | * instead it is registered directly in rc-main.c if CONFIG_MEDIA_CEC_RC | ||
| 6 | * is set. This is because it can be called from drm_dp_cec_set_edid() via | ||
| 7 | * cec_register_adapter() in an asynchronous context, and it is not | ||
| 8 | * allowed to use request_module() to load rc-cec.ko in that case. | ||
| 9 | * | ||
| 10 | * Since this keymap is only used if CONFIG_MEDIA_CEC_RC is set, we | ||
| 11 | * just compile this keymap into the rc-core module and never as a | ||
| 12 | * separate module. | ||
| 3 | * | 13 | * |
| 4 | * Copyright (c) 2015 by Kamil Debski | 14 | * Copyright (c) 2015 by Kamil Debski |
| 5 | */ | 15 | */ |
| @@ -152,7 +162,7 @@ static struct rc_map_table cec[] = { · 146 unmodified lines | |||
| 152 | /* 0x77-0xff: Reserved */ | 162 | /* 0x77-0xff: Reserved */ |
| 153 | }; | 163 | }; |
| 154 | 164 | ||
| 155 | static struct rc_map_list cec_map = { | 165 | struct rc_map_list cec_map = { |
| 156 | .map = { | 166 | .map = { |
| 157 | .scan = cec, | 167 | .scan = cec, |
| 158 | .size = ARRAY_SIZE(cec), | 168 | .size = ARRAY_SIZE(cec), |
| @@ -160,19 +170,3 @@ static struct rc_map_list cec_map = { · 1 unmodified line | |||
| 160 | .name = RC_MAP_CEC, | 170 | .name = RC_MAP_CEC, |
| 161 | } | 171 | } |
| 162 | }; | 172 | }; |
| 163 | |||
| 164 | static int __init init_rc_map_cec(void) | ||
| 165 | { | ||
| 166 | return rc_map_register(&cec_map); | ||
| 167 | } | ||
| 168 | |||
| 169 | static void __exit exit_rc_map_cec(void) | ||
| 170 | { | ||
| 171 | rc_map_unregister(&cec_map); | ||
| 172 | } | ||
| 173 | |||
| 174 | module_init(init_rc_map_cec); | ||
| 175 | module_exit(exit_rc_map_cec); | ||
| 176 | |||
| 177 | MODULE_LICENSE("GPL"); | ||
| 178 | MODULE_AUTHOR("Kamil Debski"); | ||
| @@ -2069,6 +2069,9 @@ static int __init rc_core_init(void) · 2068 unmodified lines | |||
| 2069 | 2069 | ||
| 2070 | led_trigger_register_simple("rc-feedback", &led_feedback); | 2070 | led_trigger_register_simple("rc-feedback", &led_feedback); |
| 2071 | rc_map_register(&empty_map); | 2071 | rc_map_register(&empty_map); |
| 2072 | #ifdef CONFIG_MEDIA_CEC_RC | ||
| 2073 | rc_map_register(&cec_map); | ||
| 2074 | #endif | ||
| 2072 | 2075 | ||
| 2073 | return 0; | 2076 | return 0; |
| 2074 | } | 2077 | } |
| @@ -2078,6 +2081,9 @@ static void __exit rc_core_exit(void) · 3 unmodified lines | |||
| 2078 | lirc_dev_exit(); | 2081 | lirc_dev_exit(); |
| 2079 | class_unregister(&rc_class); | 2082 | class_unregister(&rc_class); |
| 2080 | led_trigger_unregister_simple(led_feedback); | 2083 | led_trigger_unregister_simple(led_feedback); |
| 2084 | #ifdef CONFIG_MEDIA_CEC_RC | ||
| 2085 | rc_map_unregister(&cec_map); | ||
| 2086 | #endif | ||
| 2081 | rc_map_unregister(&empty_map); | 2087 | rc_map_unregister(&empty_map); |
| 2082 | } | 2088 | } |
| 2083 | 2089 | ||
| @@ -371,7 +371,7 @@ void usbtv_audio_free(struct usbtv *usbtv) · 370 unmodified lines | |||
| 371 | cancel_work_sync(&usbtv->snd_trigger); | 371 | cancel_work_sync(&usbtv->snd_trigger); |
| 372 | 372 | ||
| 373 | if (usbtv->snd && usbtv->udev) { | 373 | if (usbtv->snd && usbtv->udev) { |
| 374 | snd_card_free(usbtv->snd); | 374 | snd_card_free_when_closed(usbtv->snd); |
| 375 | usbtv->snd = NULL; | 375 | usbtv->snd = NULL; |
| 376 | } | 376 | } |
| 377 | } | 377 | } |
| @@ -950,6 +950,11 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, · 949 unmodified lines | |||
| 950 | if (!fl->cctx->rpdev) | 950 | if (!fl->cctx->rpdev) |
| 951 | return -EPIPE; | 951 | return -EPIPE; |
| 952 | 952 | ||
| 953 | if (handle == FASTRPC_INIT_HANDLE && !kernel) { | ||
| 954 | dev_warn_ratelimited(fl->sctx->dev, "user app trying to send a kernel RPC message (%d)\n", handle); | ||
| 955 | return -EPERM; | ||
| 956 | } | ||
| 957 | |||
| 953 | ctx = fastrpc_context_alloc(fl, kernel, sc, args); | 958 | ctx = fastrpc_context_alloc(fl, kernel, sc, args); |
| 954 | if (IS_ERR(ctx)) | 959 | if (IS_ERR(ctx)) |
| 955 | return PTR_ERR(ctx); | 960 | return PTR_ERR(ctx); |
| @@ -992,7 +992,6 @@ void hl_debugfs_add_device(struct hl_device *hdev) · 991 unmodified lines | |||
| 992 | struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; | 992 | struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; |
| 993 | int count = ARRAY_SIZE(hl_debugfs_list); | 993 | int count = ARRAY_SIZE(hl_debugfs_list); |
| 994 | struct hl_debugfs_entry *entry; | 994 | struct hl_debugfs_entry *entry; |
| 995 | struct dentry *ent; | ||
| 996 | int i; | 995 | int i; |
| 997 | 996 | ||
| 998 | dev_entry->hdev = hdev; | 997 | dev_entry->hdev = hdev; |
| @@ -1105,13 +1104,11 @@ void hl_debugfs_add_device(struct hl_device *hdev) · 106 unmodified lines | |||
| 1105 | &hl_security_violations_fops); | 1104 | &hl_security_violations_fops); |
| 1106 | 1105 | ||
| 1107 | for (i = 0, entry = dev_entry->entry_arr ; i < count ; i++, entry++) { | 1106 | for (i = 0, entry = dev_entry->entry_arr ; i < count ; i++, entry++) { |
| 1108 | 1107 | debugfs_create_file(hl_debugfs_list[i].name, | |
| 1109 | ent = debugfs_create_file(hl_debugfs_list[i].name, | ||
| 1110 | 0444, | 1108 | 0444, |
| 1111 | dev_entry->root, | 1109 | dev_entry->root, |
| 1112 | entry, | 1110 | entry, |
| 1113 | &hl_debugfs_fops); | 1111 | &hl_debugfs_fops); |
| 1114 | entry->dent = ent; | ||
| 1115 | entry->info_ent = &hl_debugfs_list[i]; | 1112 | entry->info_ent = &hl_debugfs_list[i]; |
| 1116 | entry->dev_entry = dev_entry; | 1113 | entry->dev_entry = dev_entry; |
| 1117 | } | 1114 | } |
| @@ -93,12 +93,19 @@ void hl_hpriv_put(struct hl_fpriv *hpriv) · 92 unmodified lines | |||
| 93 | static int hl_device_release(struct inode *inode, struct file *filp) | 93 | static int hl_device_release(struct inode *inode, struct file *filp) |
| 94 | { | 94 | { |
| 95 | struct hl_fpriv *hpriv = filp->private_data; | 95 | struct hl_fpriv *hpriv = filp->private_data; |
| 96 | struct hl_device *hdev = hpriv->hdev; | ||
| 97 | |||
| 98 | filp->private_data = NULL; | ||
| 99 | |||
| 100 | if (!hdev) { | ||
| 101 | pr_crit("Closing FD after device was removed. Memory leak will occur and it is advised to reboot.\n"); | ||
| 102 | put_pid(hpriv->taskpid); | ||
| 103 | return 0; | ||
| 104 | } | ||
| 96 | 105 | ||
| 97 | hl_cb_mgr_fini(hpriv->hdev, &hpriv->cb_mgr); | 106 | hl_cb_mgr_fini(hpriv->hdev, &hpriv->cb_mgr); |
| 98 | hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr); | 107 | hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr); |
| 99 | 108 | ||
| 100 | filp->private_data = NULL; | ||
| 101 | |||
| 102 | hl_hpriv_put(hpriv); | 109 | hl_hpriv_put(hpriv); |
| 103 | 110 | ||
| 104 | return 0; | 111 | return 0; |
| @@ -107,15 +114,20 @@ static int hl_device_release(struct inode *inode, struct file *filp) · 2 unmodified lines | |||
| 107 | static int hl_device_release_ctrl(struct inode *inode, struct file *filp) | 114 | static int hl_device_release_ctrl(struct inode *inode, struct file *filp) |
| 108 | { | 115 | { |
| 109 | struct hl_fpriv *hpriv = filp->private_data; | 116 | struct hl_fpriv *hpriv = filp->private_data; |
| 110 | struct hl_device *hdev; | 117 | struct hl_device *hdev = hpriv->hdev; |
| 111 | 118 | ||
| 112 | filp->private_data = NULL; | 119 | filp->private_data = NULL; |
| 113 | 120 | ||
| 114 | hdev = hpriv->hdev; | 121 | if (!hdev) { |
| 122 | pr_err("Closing FD after device was removed\n"); | ||
| 123 | goto out; | ||
| 124 | } | ||
| 115 | 125 | ||
| 116 | mutex_lock(&hdev->fpriv_list_lock); | 126 | mutex_lock(&hdev->fpriv_list_lock); |
| 117 | list_del(&hpriv->dev_node); | 127 | list_del(&hpriv->dev_node); |
| 118 | mutex_unlock(&hdev->fpriv_list_lock); | 128 | mutex_unlock(&hdev->fpriv_list_lock); |
| 129 | out: | ||
| 130 | put_pid(hpriv->taskpid); | ||
| 119 | 131 | ||
| 120 | kfree(hpriv); | 132 | kfree(hpriv); |
| 121 | 133 | ||
| @@ -134,8 +146,14 @@ static int hl_device_release_ctrl(struct inode *inode, struct file *filp) · 12 unmodified lines | |||
| 134 | static int hl_mmap(struct file *filp, struct vm_area_struct *vma) | 146 | static int hl_mmap(struct file *filp, struct vm_area_struct *vma) |
| 135 | { | 147 | { |
| 136 | struct hl_fpriv *hpriv = filp->private_data; | 148 | struct hl_fpriv *hpriv = filp->private_data; |
| 149 | struct hl_device *hdev = hpriv->hdev; | ||
| 137 | unsigned long vm_pgoff; | 150 | unsigned long vm_pgoff; |
| 138 | 151 | ||
| 152 | if (!hdev) { | ||
| 153 | pr_err_ratelimited("Trying to mmap after device was removed! Please close FD\n"); | ||
| 154 | return -ENODEV; | ||
| 155 | } | ||
| 156 | |||
| 139 | vm_pgoff = vma->vm_pgoff; | 157 | vm_pgoff = vma->vm_pgoff; |
| 140 | vma->vm_pgoff = HL_MMAP_OFFSET_VALUE_GET(vm_pgoff); | 158 | vma->vm_pgoff = HL_MMAP_OFFSET_VALUE_GET(vm_pgoff); |
| 141 | 159 | ||
| @@ -883,6 +901,16 @@ wait_for_processes: · 741 unmodified lines | |||
| 883 | return -EBUSY; | 901 | return -EBUSY; |
| 884 | } | 902 | } |
| 885 | 903 | ||
| 904 | static void device_disable_open_processes(struct hl_device *hdev) | ||
| 905 | { | ||
| 906 | struct hl_fpriv *hpriv; | ||
| 907 | |||
| 908 | mutex_lock(&hdev->fpriv_list_lock); | ||
| 909 | list_for_each_entry(hpriv, &hdev->fpriv_list, dev_node) | ||
| 910 | hpriv->hdev = NULL; | ||
| 911 | mutex_unlock(&hdev->fpriv_list_lock); | ||
| 912 | } | ||
| 913 | |||
| 886 | /* | 914 | /* |
| 887 | * hl_device_reset - reset the device | 915 | * hl_device_reset - reset the device |
| 888 | * | 916 | * |
| @@ -1556,8 +1584,10 @@ void hl_device_fini(struct hl_device *hdev) · 667 unmodified lines | |||
| 1556 | HL_PENDING_RESET_LONG_SEC); | 1584 | HL_PENDING_RESET_LONG_SEC); |
| 1557 | 1585 | ||
| 1558 | rc = device_kill_open_processes(hdev, HL_PENDING_RESET_LONG_SEC); | 1586 | rc = device_kill_open_processes(hdev, HL_PENDING_RESET_LONG_SEC); |
| 1559 | if (rc) | 1587 | if (rc) { |
| 1560 | dev_crit(hdev->dev, "Failed to kill all open processes\n"); | 1588 | dev_crit(hdev->dev, "Failed to kill all open processes\n"); |
| 1589 | device_disable_open_processes(hdev); | ||
| 1590 | } | ||
| 1561 | 1591 | ||
| 1562 | hl_cb_pool_fini(hdev); | 1592 | hl_cb_pool_fini(hdev); |
| 1563 | 1593 | ||
| @@ -1465,12 +1465,10 @@ struct hl_info_list { · 1464 unmodified lines | |||
| 1465 | 1465 | ||
| 1466 | /** | 1466 | /** |
| 1467 | * struct hl_debugfs_entry - debugfs dentry wrapper. | 1467 | * struct hl_debugfs_entry - debugfs dentry wrapper. |
| 1468 | * @dent: base debugfs entry structure. | ||
| 1469 | * @info_ent: dentry realted ops. | 1468 | * @info_ent: dentry realted ops. |
| 1470 | * @dev_entry: ASIC specific debugfs manager. | 1469 | * @dev_entry: ASIC specific debugfs manager. |
| 1471 | */ | 1470 | */ |
| 1472 | struct hl_debugfs_entry { | 1471 | struct hl_debugfs_entry { |
| 1473 | struct dentry *dent; | ||
| 1474 | const struct hl_info_list *info_ent; | 1472 | const struct hl_info_list *info_ent; |
| 1475 | struct hl_dbg_device_entry *dev_entry; | 1473 | struct hl_dbg_device_entry *dev_entry; |
| 1476 | }; | 1474 | }; |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | |||
| 5 | * All Rights Reserved. | 5 | * All Rights Reserved. |
| 6 | */ | 6 | */ |
| 7 | 7 | ||
| 8 | #define pr_fmt(fmt) "habanalabs: " fmt | ||
| 9 | |||
| 8 | #include <uapi/misc/habanalabs.h> | 10 | #include <uapi/misc/habanalabs.h> |
| 9 | #include "habanalabs.h" | 11 | #include "habanalabs.h" |
| 10 | 12 | ||
| @@ -682,6 +684,11 @@ long hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 671 unmodified lines | |||
| 682 | const struct hl_ioctl_desc *ioctl = NULL; | 684 | const struct hl_ioctl_desc *ioctl = NULL; |
| 683 | unsigned int nr = _IOC_NR(cmd); | 685 | unsigned int nr = _IOC_NR(cmd); |
| 684 | 686 | ||
| 687 | if (!hdev) { | ||
| 688 | pr_err_ratelimited("Sending ioctl after device was removed! Please close FD\n"); | ||
| 689 | return -ENODEV; | ||
| 690 | } | ||
| 691 | |||
| 685 | if ((nr >= HL_COMMAND_START) && (nr < HL_COMMAND_END)) { | 692 | if ((nr >= HL_COMMAND_START) && (nr < HL_COMMAND_END)) { |
| 686 | ioctl = &hl_ioctls[nr]; | 693 | ioctl = &hl_ioctls[nr]; |
| 687 | } else { | 694 | } else { |
| @@ -700,6 +707,11 @@ long hl_ioctl_control(struct file *filep, unsigned int cmd, unsigned long arg) · 12 unmodified lines | |||
| 700 | const struct hl_ioctl_desc *ioctl = NULL; | 707 | const struct hl_ioctl_desc *ioctl = NULL; |
| 701 | unsigned int nr = _IOC_NR(cmd); | 708 | unsigned int nr = _IOC_NR(cmd); |
| 702 | 709 | ||
| 710 | if (!hdev) { | ||
| 711 | pr_err_ratelimited("Sending ioctl after device was removed! Please close FD\n"); | ||
| 712 | return -ENODEV; | ||
| 713 | } | ||
| 714 | |||
| 703 | if (nr == _IOC_NR(HL_IOCTL_INFO)) { | 715 | if (nr == _IOC_NR(HL_IOCTL_INFO)) { |
| 704 | ioctl = &hl_ioctls_control[nr]; | 716 | ioctl = &hl_ioctls_control[nr]; |
| 705 | } else { | 717 | } else { |
| @@ -47,7 +47,7 @@ inline u32 hl_cq_inc_ptr(u32 ptr) · 46 unmodified lines | |||
| 47 | * Increment ptr by 1. If it reaches the number of event queue | 47 | * Increment ptr by 1. If it reaches the number of event queue |
| 48 | * entries, set it to 0 | 48 | * entries, set it to 0 |
| 49 | */ | 49 | */ |
| 50 | inline u32 hl_eq_inc_ptr(u32 ptr) | 50 | static inline u32 hl_eq_inc_ptr(u32 ptr) |
| 51 | { | 51 | { |
| 52 | ptr++; | 52 | ptr++; |
| 53 | if (unlikely(ptr == HL_EQ_LENGTH)) | 53 | if (unlikely(ptr == HL_EQ_LENGTH)) |
| @@ -499,18 +499,32 @@ static void hl_mmu_pa_page_with_offset(struct hl_ctx *ctx, u64 virt_addr, · 498 unmodified lines | |||
| 499 | else /* HL_VA_RANGE_TYPE_DRAM */ | 499 | else /* HL_VA_RANGE_TYPE_DRAM */ |
| 500 | p = &prop->dmmu; | 500 | p = &prop->dmmu; |
| 501 | 501 | ||
| 502 | /* | 502 | if ((hops->range_type == HL_VA_RANGE_TYPE_DRAM) && |
| 503 | * find the correct hop shift field in hl_mmu_properties structure | 503 | !is_power_of_2(prop->dram_page_size)) { |
| 504 | * in order to determine the right maks for the page offset. | 504 | u32 bit; |
| 505 | */ | 505 | u64 page_offset_mask; |
| 506 | hop0_shift_off = offsetof(struct hl_mmu_properties, hop0_shift); | 506 | u64 phys_addr_mask; |
| 507 | p = (char *)p + hop0_shift_off; | 507 | |
| 508 | p = (char *)p + ((hops->used_hops - 1) * sizeof(u64)); | 508 | bit = __ffs64((u64)prop->dram_page_size); |
| 509 | hop_shift = *(u64 *)p; | 509 | page_offset_mask = ((1ull << bit) - 1); |
| 510 | offset_mask = (1ull << hop_shift) - 1; | 510 | phys_addr_mask = ~page_offset_mask; |
| 511 | addr_mask = ~(offset_mask); | 511 | *phys_addr = (tmp_phys_addr & phys_addr_mask) | |
| 512 | *phys_addr = (tmp_phys_addr & addr_mask) | | 512 | (virt_addr & page_offset_mask); |
| 513 | (virt_addr & offset_mask); | 513 | } else { |
| 514 | /* | ||
| 515 | * find the correct hop shift field in hl_mmu_properties | ||
| 516 | * structure in order to determine the right masks | ||
| 517 | * for the page offset. | ||
| 518 | */ | ||
| 519 | hop0_shift_off = offsetof(struct hl_mmu_properties, hop0_shift); | ||
| 520 | p = (char *)p + hop0_shift_off; | ||
| 521 | p = (char *)p + ((hops->used_hops - 1) * sizeof(u64)); | ||
| 522 | hop_shift = *(u64 *)p; | ||
| 523 | offset_mask = (1ull << hop_shift) - 1; | ||
| 524 | addr_mask = ~(offset_mask); | ||
| 525 | *phys_addr = (tmp_phys_addr & addr_mask) | | ||
| 526 | (virt_addr & offset_mask); | ||
| 527 | } | ||
| 514 | } | 528 | } |
| 515 | 529 | ||
| 516 | int hl_mmu_va_to_pa(struct hl_ctx *ctx, u64 virt_addr, u64 *phys_addr) | 530 | int hl_mmu_va_to_pa(struct hl_ctx *ctx, u64 virt_addr, u64 *phys_addr) |
| @@ -2288,15 +2288,13 @@ crq_failed: · 2287 unmodified lines | |||
| 2288 | return -EPERM; | 2288 | return -EPERM; |
| 2289 | } | 2289 | } |
| 2290 | 2290 | ||
| 2291 | static int ibmvmc_remove(struct vio_dev *vdev) | 2291 | static void ibmvmc_remove(struct vio_dev *vdev) |
| 2292 | { | 2292 | { |
| 2293 | struct crq_server_adapter *adapter = dev_get_drvdata(&vdev->dev); | 2293 | struct crq_server_adapter *adapter = dev_get_drvdata(&vdev->dev); |
| 2294 | 2294 | ||
| 2295 | dev_info(adapter->dev, "Entering remove for UA 0x%x\n", | 2295 | dev_info(adapter->dev, "Entering remove for UA 0x%x\n", |
| 2296 | vdev->unit_address); | 2296 | vdev->unit_address); |
| 2297 | ibmvmc_release_crq_queue(adapter); | 2297 | ibmvmc_release_crq_queue(adapter); |
| 2298 | |||
| 2299 | return 0; | ||
| 2300 | } | 2298 | } |
| 2301 | 2299 | ||
| 2302 | static struct vio_device_id ibmvmc_device_table[] = { | 2300 | static struct vio_device_id ibmvmc_device_table[] = { |
| @@ -140,6 +140,7 @@ static const struct of_device_id pvpanic_mmio_match[] = { · 139 unmodified lines | |||
| 140 | { .compatible = "qemu,pvpanic-mmio", }, | 140 | { .compatible = "qemu,pvpanic-mmio", }, |
| 141 | {} | 141 | {} |
| 142 | }; | 142 | }; |
| 143 | MODULE_DEVICE_TABLE(of, pvpanic_mmio_match); | ||
| 143 | 144 | ||
| 144 | static const struct acpi_device_id pvpanic_device_ids[] = { | 145 | static const struct acpi_device_id pvpanic_device_ids[] = { |
| 145 | { "QEMU0001", 0 }, | 146 | { "QEMU0001", 0 }, |
| @@ -399,11 +399,6 @@ void mmc_remove_card(struct mmc_card *card) · 398 unmodified lines | |||
| 399 | mmc_remove_card_debugfs(card); | 399 | mmc_remove_card_debugfs(card); |
| 400 | #endif | 400 | #endif |
| 401 | 401 | ||
| 402 | if (host->cqe_enabled) { | ||
| 403 | host->cqe_ops->cqe_disable(host); | ||
| 404 | host->cqe_enabled = false; | ||
| 405 | } | ||
| 406 | |||
| 407 | if (mmc_card_present(card)) { | 402 | if (mmc_card_present(card)) { |
| 408 | if (mmc_host_is_spi(card->host)) { | 403 | if (mmc_host_is_spi(card->host)) { |
| 409 | pr_info("%s: SPI card removed\n", | 404 | pr_info("%s: SPI card removed\n", |
| @@ -416,6 +411,10 @@ void mmc_remove_card(struct mmc_card *card) · 6 unmodified lines | |||
| 416 | of_node_put(card->dev.of_node); | 411 | of_node_put(card->dev.of_node); |
| 417 | } | 412 | } |
| 418 | 413 | ||
| 414 | if (host->cqe_enabled) { | ||
| 415 | host->cqe_ops->cqe_disable(host); | ||
| 416 | host->cqe_enabled = false; | ||
| 417 | } | ||
| 418 | |||
| 419 | put_device(&card->dev); | 419 | put_device(&card->dev); |
| 420 | } | 420 | } |
| 421 | |||
| @@ -423,10 +423,6 @@ static int mmc_decode_ext_csd(struct mmc_card *card, u8 *ext_csd) · 422 unmodified lines | |||
| 423 | 423 | ||
| 424 | /* EXT_CSD value is in units of 10ms, but we store in ms */ | 424 | /* EXT_CSD value is in units of 10ms, but we store in ms */ |
| 425 | card->ext_csd.part_time = 10 * ext_csd[EXT_CSD_PART_SWITCH_TIME]; | 425 | card->ext_csd.part_time = 10 * ext_csd[EXT_CSD_PART_SWITCH_TIME]; |
| 426 | /* Some eMMC set the value too low so set a minimum */ | ||
| 427 | if (card->ext_csd.part_time && | ||
| 428 | card->ext_csd.part_time < MMC_MIN_PART_SWITCH_TIME) | ||
| 429 | card->ext_csd.part_time = MMC_MIN_PART_SWITCH_TIME; | ||
| 430 | 426 | ||
| 431 | /* Sleep / awake timeout in 100ns units */ | 427 | /* Sleep / awake timeout in 100ns units */ |
| 432 | if (sa_shift > 0 && sa_shift <= 0x17) | 428 | if (sa_shift > 0 && sa_shift <= 0x17) |
| @@ -616,6 +612,17 @@ static int mmc_decode_ext_csd(struct mmc_card *card, u8 *ext_csd) · 183 unmodified lines | |||
| 616 | card->ext_csd.data_sector_size = 512; | 612 | card->ext_csd.data_sector_size = 512; |
| 617 | } | 613 | } |
| 618 | 614 | ||
| 615 | /* | ||
| 616 | * GENERIC_CMD6_TIME is to be used "unless a specific timeout is defined | ||
| 617 | * when accessing a specific field", so use it here if there is no | ||
| 618 | * PARTITION_SWITCH_TIME. | ||
| 619 | */ | ||
| 620 | if (!card->ext_csd.part_time) | ||
| 621 | card->ext_csd.part_time = card->ext_csd.generic_cmd6_time; | ||
| 622 | /* Some eMMC set the value too low so set a minimum */ | ||
| 623 | if (card->ext_csd.part_time < MMC_MIN_PART_SWITCH_TIME) | ||
| 624 | card->ext_csd.part_time = MMC_MIN_PART_SWITCH_TIME; | ||
| 625 | |||
| 619 | /* eMMC v5 or later */ | 626 | /* eMMC v5 or later */ |
| 620 | if (card->ext_csd.rev >= 7) { | 627 | if (card->ext_csd.rev >= 7) { |
| 621 | memcpy(card->ext_csd.fwrev, &ext_csd[EXT_CSD_FIRMWARE_VERSION], | 628 | memcpy(card->ext_csd.fwrev, &ext_csd[EXT_CSD_FIRMWARE_VERSION], |
| @@ -1242,7 +1242,11 @@ mmci_start_command(struct mmci_host *host, struct mmc_command *cmd, u32 c) · 1241 unmodified lines | |||
| 1242 | if (!cmd->busy_timeout) | 1242 | if (!cmd->busy_timeout) |
| 1243 | cmd->busy_timeout = 10 * MSEC_PER_SEC; | 1243 | cmd->busy_timeout = 10 * MSEC_PER_SEC; |
| 1244 | 1244 | ||
| 1245 | clks = (unsigned long long)cmd->busy_timeout * host->cclk; | 1245 | if (cmd->busy_timeout > host->mmc->max_busy_timeout) |
| 1246 | clks = (unsigned long long)host->mmc->max_busy_timeout * host->cclk; | ||
| 1247 | else | ||
| 1248 | clks = (unsigned long long)cmd->busy_timeout * host->cclk; | ||
| 1249 | |||
| 1246 | do_div(clks, MSEC_PER_SEC); | 1250 | do_div(clks, MSEC_PER_SEC); |
| 1247 | writel_relaxed(clks, host->base + MMCIDATATIMER); | 1251 | writel_relaxed(clks, host->base + MMCIDATATIMER); |
| 1248 | } | 1252 | } |
| @@ -2151,6 +2155,10 @@ static int mmci_probe(struct amba_device *dev, · 902 unmodified lines | |||
| 2151 | mmc->caps |= MMC_CAP_WAIT_WHILE_BUSY; | 2155 | mmc->caps |= MMC_CAP_WAIT_WHILE_BUSY; |
| 2152 | } | 2156 | } |
| 2153 | 2157 | ||
| 2158 | /* Variants with mandatory busy timeout in HW needs R1B responses. */ | ||
| 2159 | if (variant->busy_timeout) | ||
| 2160 | mmc->caps |= MMC_CAP_NEED_RSP_BUSY; | ||
| 2161 | |||
| 2154 | /* Prepare a CMD12 - needed to clear the DPSM on some variants. */ | 2162 | /* Prepare a CMD12 - needed to clear the DPSM on some variants. */ |
| 2155 | host->stop_abort.opcode = MMC_STOP_TRANSMISSION; | 2163 | host->stop_abort.opcode = MMC_STOP_TRANSMISSION; |
| 2156 | host->stop_abort.arg = 0; | 2164 | host->stop_abort.arg = 0; |
| @@ -94,7 +94,7 @@ config WIREGUARD · 93 unmodified lines | |||
| 94 | select CRYPTO_BLAKE2S_ARM if ARM | 94 | select CRYPTO_BLAKE2S_ARM if ARM |
| 95 | select CRYPTO_CURVE25519_NEON if ARM && KERNEL_MODE_NEON | 95 | select CRYPTO_CURVE25519_NEON if ARM && KERNEL_MODE_NEON |
| 96 | select CRYPTO_CHACHA_MIPS if CPU_MIPS32_R2 | 96 | select CRYPTO_CHACHA_MIPS if CPU_MIPS32_R2 |
| 97 | select CRYPTO_POLY1305_MIPS if CPU_MIPS32 || (CPU_MIPS64 && 64BIT) | 97 | select CRYPTO_POLY1305_MIPS if MIPS |
| 98 | help | 98 | help |
| 99 | WireGuard is a secure, fast, and easy to use replacement for IPSec | 99 | WireGuard is a secure, fast, and easy to use replacement for IPSec |
| 100 | that uses modern cryptography and clever networking tricks. It's | 100 | that uses modern cryptography and clever networking tricks. It's |
| @@ -3978,11 +3978,15 @@ static int bond_neigh_init(struct neighbour *n) · 3977 unmodified lines | |||
| 3978 | 3978 | ||
| 3979 | rcu_read_lock(); | 3979 | rcu_read_lock(); |
| 3980 | slave = bond_first_slave_rcu(bond); | 3980 | slave = bond_first_slave_rcu(bond); |
| 3981 | if (!slave) | 3981 | if (!slave) { |
| 3982 | ret = -EINVAL; | ||
| 3982 | goto out; | 3983 | goto out; |
| 3984 | } | ||
| 3983 | slave_ops = slave->dev->netdev_ops; | 3985 | slave_ops = slave->dev->netdev_ops; |
| 3984 | if (!slave_ops->ndo_neigh_setup) | 3986 | if (!slave_ops->ndo_neigh_setup) { |
| 3987 | ret = -EINVAL; | ||
| 3985 | goto out; | 3988 | goto out; |
| 3989 | } | ||
| 3986 | 3990 | ||
| 3987 | /* TODO: find another way [1] to implement this. | 3991 | /* TODO: find another way [1] to implement this. |
| 3988 | * Passing a zeroed structure is fragile, | 3992 | * Passing a zeroed structure is fragile, |
| @@ -701,7 +701,7 @@ static int flexcan_chip_freeze(struct flexcan_priv *priv) · 700 unmodified lines | |||
| 701 | u32 reg; | 701 | u32 reg; |
| 702 | 702 | ||
| 703 | reg = priv->read(®s->mcr); | 703 | reg = priv->read(®s->mcr); |
| 704 | reg |= FLEXCAN_MCR_HALT; | 704 | reg |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT; |
| 705 | priv->write(reg, ®s->mcr); | 705 | priv->write(reg, ®s->mcr); |
| 706 | 706 | ||
| 707 | while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK)) | 707 | while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK)) |
| @@ -1480,10 +1480,13 @@ static int flexcan_chip_start(struct net_device *dev) · 772 unmodified lines | |||
| 1480 | 1480 | ||
| 1481 | flexcan_set_bittiming(dev); | 1481 | flexcan_set_bittiming(dev); |
| 1482 | 1482 | ||
| 1483 | /* set freeze, halt */ | ||
| 1484 | err = flexcan_chip_freeze(priv); | ||
| 1485 | if (err) | ||
| 1486 | goto out_chip_disable; | ||
| 1487 | |||
| 1483 | /* MCR | 1488 | /* MCR |
| 1484 | * | 1489 | * |
| 1485 | * enable freeze | ||
| 1486 | * halt now | ||
| 1487 | * only supervisor access | 1490 | * only supervisor access |
| 1488 | * enable warning int | 1491 | * enable warning int |
| 1489 | * enable individual RX masking | 1492 | * enable individual RX masking |
| @@ -1492,9 +1495,8 @@ static int flexcan_chip_start(struct net_device *dev) · 2 unmodified lines | |||
| 1492 | */ | 1495 | */ |
| 1493 | reg_mcr = priv->read(®s->mcr); | 1496 | reg_mcr = priv->read(®s->mcr); |
| 1494 | reg_mcr &= ~FLEXCAN_MCR_MAXMB(0xff); | 1497 | reg_mcr &= ~FLEXCAN_MCR_MAXMB(0xff); |
| 1495 | reg_mcr |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT | FLEXCAN_MCR_SUPV | | 1498 | reg_mcr |= FLEXCAN_MCR_SUPV | FLEXCAN_MCR_WRN_EN | FLEXCAN_MCR_IRMQ | |
| 1496 | FLEXCAN_MCR_WRN_EN | FLEXCAN_MCR_IRMQ | FLEXCAN_MCR_IDAM_C | | 1499 | FLEXCAN_MCR_IDAM_C | FLEXCAN_MCR_MAXMB(priv->tx_mb_idx); |
| 1497 | FLEXCAN_MCR_MAXMB(priv->tx_mb_idx); | ||
| 1498 | 1500 | ||
| 1499 | /* MCR | 1501 | /* MCR |
| 1500 | * | 1502 | * |
| @@ -1865,10 +1867,14 @@ static int register_flexcandev(struct net_device *dev) · 364 unmodified lines | |||
| 1865 | if (err) | 1867 | if (err) |
| 1866 | goto out_chip_disable; | 1868 | goto out_chip_disable; |
| 1867 | 1869 | ||
| 1868 | /* set freeze, halt and activate FIFO, restrict register access */ | 1870 | /* set freeze, halt */ |
| 1871 | err = flexcan_chip_freeze(priv); | ||
| 1872 | if (err) | ||
| 1873 | goto out_chip_disable; | ||
| 1874 | |||
| 1875 | /* activate FIFO, restrict register access */ | ||
| 1869 | reg = priv->read(®s->mcr); | 1876 | reg = priv->read(®s->mcr); |
| 1870 | reg |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT | | 1877 | reg |= FLEXCAN_MCR_FEN | FLEXCAN_MCR_SUPV; |
| 1871 | FLEXCAN_MCR_FEN | FLEXCAN_MCR_SUPV; | ||
| 1872 | priv->write(reg, ®s->mcr); | 1878 | priv->write(reg, ®s->mcr); |
| 1873 | 1879 | ||
| 1874 | /* Currently we only support newer versions of this core | 1880 | /* Currently we only support newer versions of this core |
| @@ -237,14 +237,14 @@ static int tcan4x5x_init(struct m_can_classdev *cdev) · 236 unmodified lines | |||
| 237 | if (ret) | 237 | if (ret) |
| 238 | return ret; | 238 | return ret; |
| 239 | 239 | ||
| 240 | /* Zero out the MCAN buffers */ | ||
| 241 | m_can_init_ram(cdev); | ||
| 242 | |||
| 240 | ret = regmap_update_bits(tcan4x5x->regmap, TCAN4X5X_CONFIG, | 243 | ret = regmap_update_bits(tcan4x5x->regmap, TCAN4X5X_CONFIG, |
| 241 | TCAN4X5X_MODE_SEL_MASK, TCAN4X5X_MODE_NORMAL); | 244 | TCAN4X5X_MODE_SEL_MASK, TCAN4X5X_MODE_NORMAL); |
| 242 | if (ret) | 245 | if (ret) |
| 243 | return ret; | 246 | return ret; |
| 244 | 247 | ||
| 245 | /* Zero out the MCAN buffers */ | ||
| 246 | m_can_init_ram(cdev); | ||
| 247 | |||
| 248 | return ret; | 248 | return ret; |
| 249 | } | 249 | } |
| 250 | 250 | ||
| @@ -335,8 +335,6 @@ static void mcp251xfd_ring_init(struct mcp251xfd_priv *priv) · 334 unmodified lines | |||
| 335 | u8 len; | 335 | u8 len; |
| 336 | int i, j; | 336 | int i, j; |
| 337 | 337 | ||
| 338 | netdev_reset_queue(priv->ndev); | ||
| 339 | |||
| 340 | /* TEF */ | 338 | /* TEF */ |
| 341 | tef_ring = priv->tef; | 339 | tef_ring = priv->tef; |
| 342 | tef_ring->head = 0; | 340 | tef_ring->head = 0; |
| @@ -1249,8 +1247,7 @@ mcp251xfd_handle_tefif_recover(const struct mcp251xfd_priv *priv, const u32 seq) · 906 unmodified lines | |||
| 1249 | 1247 | ||
| 1250 | static int | 1248 | static int |
| 1251 | mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv, | 1249 | mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv, |
| 1252 | const struct mcp251xfd_hw_tef_obj *hw_tef_obj, | 1250 | const struct mcp251xfd_hw_tef_obj *hw_tef_obj) |
| 1253 | unsigned int *frame_len_ptr) | ||
| 1254 | { | 1251 | { |
| 1255 | struct net_device_stats *stats = &priv->ndev->stats; | 1252 | struct net_device_stats *stats = &priv->ndev->stats; |
| 1256 | u32 seq, seq_masked, tef_tail_masked; | 1253 | u32 seq, seq_masked, tef_tail_masked; |
| @@ -1272,8 +1269,7 @@ mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv, · 15 unmodified lines | |||
| 1272 | stats->tx_bytes += | 1269 | stats->tx_bytes += |
| 1273 | can_rx_offload_get_echo_skb(&priv->offload, | 1270 | can_rx_offload_get_echo_skb(&priv->offload, |
| 1274 | mcp251xfd_get_tef_tail(priv), | 1271 | mcp251xfd_get_tef_tail(priv), |
| 1275 | hw_tef_obj->ts, | 1272 | hw_tef_obj->ts, NULL); |
| 1276 | frame_len_ptr); | ||
| 1277 | stats->tx_packets++; | 1273 | stats->tx_packets++; |
| 1278 | priv->tef->tail++; | 1274 | priv->tef->tail++; |
| 1279 | 1275 | ||
| @@ -1331,7 +1327,6 @@ mcp251xfd_tef_obj_read(const struct mcp251xfd_priv *priv, · 51 unmodified lines | |||
| 1331 | static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv) | 1327 | static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv) |
| 1332 | { | 1328 | { |
| 1333 | struct mcp251xfd_hw_tef_obj hw_tef_obj[MCP251XFD_TX_OBJ_NUM_MAX]; | 1329 | struct mcp251xfd_hw_tef_obj hw_tef_obj[MCP251XFD_TX_OBJ_NUM_MAX]; |
| 1334 | unsigned int total_frame_len = 0; | ||
| 1335 | u8 tef_tail, len, l; | 1330 | u8 tef_tail, len, l; |
| 1336 | int err, i; | 1331 | int err, i; |
| 1337 | 1332 | ||
| @@ -1353,9 +1348,7 @@ static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv) · 15 unmodified lines | |||
| 1353 | } | 1348 | } |
| 1354 | 1349 | ||
| 1355 | for (i = 0; i < len; i++) { | 1350 | for (i = 0; i < len; i++) { |
| 1356 | unsigned int frame_len; | 1351 | err = mcp251xfd_handle_tefif_one(priv, &hw_tef_obj[i]); |
| 1357 | |||
| 1358 | err = mcp251xfd_handle_tefif_one(priv, &hw_tef_obj[i], &frame_len); | ||
| 1359 | /* -EAGAIN means the Sequence Number in the TEF | 1352 | /* -EAGAIN means the Sequence Number in the TEF |
| 1360 | * doesn't match our tef_tail. This can happen if we | 1353 | * doesn't match our tef_tail. This can happen if we |
| 1361 | * read the TEF objects too early. Leave loop let the | 1354 | * read the TEF objects too early. Leave loop let the |
| @@ -1365,8 +1358,6 @@ static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv) · 3 unmodified lines | |||
| 1365 | goto out_netif_wake_queue; | 1358 | goto out_netif_wake_queue; |
| 1366 | if (err) | 1359 | if (err) |
| 1367 | return err; | 1360 | return err; |
| 1368 | |||
| 1369 | total_frame_len += frame_len; | ||
| 1370 | } | 1361 | } |
| 1371 | 1362 | ||
| 1372 | out_netif_wake_queue: | 1363 | out_netif_wake_queue: |
| @@ -1397,7 +1388,6 @@ static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv) · 24 unmodified lines | |||
| 1397 | return err; | 1388 | return err; |
| 1398 | 1389 | ||
| 1399 | tx_ring->tail += len; | 1390 | tx_ring->tail += len; |
| 1400 | netdev_completed_queue(priv->ndev, len, total_frame_len); | ||
| 1401 | 1391 | ||
| 1402 | err = mcp251xfd_check_tef_tail(priv); | 1392 | err = mcp251xfd_check_tef_tail(priv); |
| 1403 | if (err) | 1393 | if (err) |
| @@ -2443,7 +2433,6 @@ static netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb, · 1039 unmodified lines | |||
| 2443 | struct mcp251xfd_priv *priv = netdev_priv(ndev); | 2433 | struct mcp251xfd_priv *priv = netdev_priv(ndev); |
| 2444 | struct mcp251xfd_tx_ring *tx_ring = priv->tx; | 2434 | struct mcp251xfd_tx_ring *tx_ring = priv->tx; |
| 2445 | struct mcp251xfd_tx_obj *tx_obj; | 2435 | struct mcp251xfd_tx_obj *tx_obj; |
| 2446 | unsigned int frame_len; | ||
| 2447 | u8 tx_head; | 2436 | u8 tx_head; |
| 2448 | int err; | 2437 | int err; |
| 2449 | 2438 | ||
| @@ -2462,9 +2451,7 @@ static netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb, · 12 unmodified lines | |||
| 2462 | if (mcp251xfd_get_tx_free(tx_ring) == 0) | 2451 | if (mcp251xfd_get_tx_free(tx_ring) == 0) |
| 2463 | netif_stop_queue(ndev); | 2452 | netif_stop_queue(ndev); |
| 2464 | 2453 | ||
| 2465 | frame_len = can_skb_get_frame_len(skb); | 2454 | can_put_echo_skb(skb, ndev, tx_head, 0); |
| 2466 | can_put_echo_skb(skb, ndev, tx_head, frame_len); | ||
| 2467 | netdev_sent_queue(priv->ndev, frame_len); | ||
| 2468 | 2455 | ||
| 2469 | err = mcp251xfd_tx_obj_write(priv, tx_obj); | 2456 | err = mcp251xfd_tx_obj_write(priv, tx_obj); |
| 2470 | if (err) | 2457 | if (err) |
| @@ -406,7 +406,7 @@ static int bcm_sf2_sw_rst(struct bcm_sf2_priv *priv) · 405 unmodified lines | |||
| 406 | /* The watchdog reset does not work on 7278, we need to hit the | 406 | /* The watchdog reset does not work on 7278, we need to hit the |
| 407 | * "external" reset line through the reset controller. | 407 | * "external" reset line through the reset controller. |
| 408 | */ | 408 | */ |
| 409 | if (priv->type == BCM7278_DEVICE_ID && !IS_ERR(priv->rcdev)) { | 409 | if (priv->type == BCM7278_DEVICE_ID) { |
| 410 | ret = reset_control_assert(priv->rcdev); | 410 | ret = reset_control_assert(priv->rcdev); |
| 411 | if (ret) | 411 | if (ret) |
| 412 | return ret; | 412 | return ret; |
| @@ -1265,7 +1265,7 @@ static int bcm_sf2_sw_probe(struct platform_device *pdev) · 852 unmodified lines | |||
| 1265 | 1265 | ||
| 1266 | priv->rcdev = devm_reset_control_get_optional_exclusive(&pdev->dev, | 1266 | priv->rcdev = devm_reset_control_get_optional_exclusive(&pdev->dev, |
| 1267 | "switch"); | 1267 | "switch"); |
| 1268 | if (PTR_ERR(priv->rcdev) == -EPROBE_DEFER) | 1268 | if (IS_ERR(priv->rcdev)) |
| 1269 | return PTR_ERR(priv->rcdev); | 1269 | return PTR_ERR(priv->rcdev); |
| 1270 | 1270 | ||
| 1271 | /* Auto-detection using standard registers will not work, so | 1271 | /* Auto-detection using standard registers will not work, so |
| @@ -1426,7 +1426,7 @@ static int bcm_sf2_sw_remove(struct platform_device *pdev) · 154 unmodified lines | |||
| 1426 | bcm_sf2_mdio_unregister(priv); | 1426 | bcm_sf2_mdio_unregister(priv); |
| 1427 | clk_disable_unprepare(priv->clk_mdiv); | 1427 | clk_disable_unprepare(priv->clk_mdiv); |
| 1428 | clk_disable_unprepare(priv->clk); | 1428 | clk_disable_unprepare(priv->clk); |
| 1429 | if (priv->type == BCM7278_DEVICE_ID && !IS_ERR(priv->rcdev)) | 1429 | if (priv->type == BCM7278_DEVICE_ID) |
| 1430 | reset_control_assert(priv->rcdev); | 1430 | reset_control_assert(priv->rcdev); |
| 1431 | 1431 | ||
| 1432 | return 0; | 1432 | return 0; |
| @@ -1624,6 +1624,7 @@ mtk_get_tag_protocol(struct dsa_switch *ds, int port, · 1623 unmodified lines | |||
| 1624 | } | 1624 | } |
| 1625 | } | 1625 | } |
| 1626 | 1626 | ||
| 1627 | #ifdef CONFIG_GPIOLIB | ||
| 1627 | static inline u32 | 1628 | static inline u32 |
| 1628 | mt7530_gpio_to_bit(unsigned int offset) | 1629 | mt7530_gpio_to_bit(unsigned int offset) |
| 1629 | { | 1630 | { |
| @@ -1726,6 +1727,7 @@ mt7530_setup_gpio(struct mt7530_priv *priv) · 96 unmodified lines | |||
| 1726 | 1727 | ||
| 1727 | return devm_gpiochip_add_data(dev, gc, priv); | 1728 | return devm_gpiochip_add_data(dev, gc, priv); |
| 1728 | } | 1729 | } |
| 1730 | #endif /* CONFIG_GPIOLIB */ | ||
| 1729 | 1731 | ||
| 1730 | static int | 1732 | static int |
| 1731 | mt7530_setup(struct dsa_switch *ds) | 1733 | mt7530_setup(struct dsa_switch *ds) |
| @@ -1868,11 +1870,13 @@ mt7530_setup(struct dsa_switch *ds) · 136 unmodified lines | |||
| 1868 | } | 1870 | } |
| 1869 | } | 1871 | } |
| 1870 | 1872 | ||
| 1873 | #ifdef CONFIG_GPIOLIB | ||
| 1871 | if (of_property_read_bool(priv->dev->of_node, "gpio-controller")) { | 1874 | if (of_property_read_bool(priv->dev->of_node, "gpio-controller")) { |
| 1872 | ret = mt7530_setup_gpio(priv); | 1875 | ret = mt7530_setup_gpio(priv); |
| 1873 | if (ret) | 1876 | if (ret) |
| 1874 | return ret; | 1877 | return ret; |
| 1875 | } | 1878 | } |
| 1879 | #endif /* CONFIG_GPIOLIB */ | ||
| 1876 | 1880 | ||
| 1877 | mt7530_setup_port5(ds, interface); | 1881 | mt7530_setup_port5(ds, interface); |
| 1878 | 1882 | ||
| @@ -1922,7 +1922,7 @@ out_unlock_ptp: · 1921 unmodified lines | |||
| 1922 | speed = SPEED_1000; | 1922 | speed = SPEED_1000; |
| 1923 | else if (bmcr & BMCR_SPEED100) | 1923 | else if (bmcr & BMCR_SPEED100) |
| 1924 | speed = SPEED_100; | 1924 | speed = SPEED_100; |
| 1925 | else if (bmcr & BMCR_SPEED10) | 1925 | else |
| 1926 | speed = SPEED_10; | 1926 | speed = SPEED_10; |
| 1927 | 1927 | ||
| 1928 | sja1105_sgmii_pcs_force_speed(priv, speed); | 1928 | sja1105_sgmii_pcs_force_speed(priv, speed); |
| @@ -3369,14 +3369,14 @@ static int sja1105_port_ucast_bcast_flood(struct sja1105_private *priv, int to, · 1440 unmodified lines | |||
| 3369 | if (flags.val & BR_FLOOD) | 3369 | if (flags.val & BR_FLOOD) |
| 3370 | priv->ucast_egress_floods |= BIT(to); | 3370 | priv->ucast_egress_floods |= BIT(to); |
| 3371 | else | 3371 | else |
| 3372 | priv->ucast_egress_floods |= BIT(to); | 3372 | priv->ucast_egress_floods &= ~BIT(to); |
| 3373 | } | 3373 | } |
| 3374 | 3374 | ||
| 3375 | if (flags.mask & BR_BCAST_FLOOD) { | 3375 | if (flags.mask & BR_BCAST_FLOOD) { |
| 3376 | if (flags.val & BR_BCAST_FLOOD) | 3376 | if (flags.val & BR_BCAST_FLOOD) |
| 3377 | priv->bcast_egress_floods |= BIT(to); | 3377 | priv->bcast_egress_floods |= BIT(to); |
| 3378 | else | 3378 | else |
| 3379 | priv->bcast_egress_floods |= BIT(to); | 3379 | priv->bcast_egress_floods &= ~BIT(to); |
| 3380 | } | 3380 | } |
| 3381 | 3381 | ||
| 3382 | return sja1105_manage_flood_domains(priv); | 3382 | return sja1105_manage_flood_domains(priv); |
| @@ -528,7 +528,10 @@ static int xrs700x_hsr_join(struct dsa_switch *ds, int port, · 527 unmodified lines | |||
| 528 | return -EOPNOTSUPP; | 528 | return -EOPNOTSUPP; |
| 529 | 529 | ||
| 530 | dsa_hsr_foreach_port(dp, ds, hsr) { | 530 | dsa_hsr_foreach_port(dp, ds, hsr) { |
| 531 | partner = dp; | 531 | if (dp->index != port) { |
| 532 | partner = dp; | ||
| 533 | break; | ||
| 534 | } | ||
| 532 | } | 535 | } |
| 533 | 536 | ||
| 534 | /* We can't enable redundancy on the switch until both | 537 | /* We can't enable redundancy on the switch until both |
| @@ -582,7 +585,10 @@ static int xrs700x_hsr_leave(struct dsa_switch *ds, int port, · 47 unmodified lines | |||
| 582 | unsigned int val; | 585 | unsigned int val; |
| 583 | 586 | ||
| 584 | dsa_hsr_foreach_port(dp, ds, hsr) { | 587 | dsa_hsr_foreach_port(dp, ds, hsr) { |
| 585 | partner = dp; | 588 | if (dp->index != port) { |
| 589 | partner = dp; | ||
| 590 | break; | ||
| 591 | } | ||
| 586 | } | 592 | } |
| 587 | 593 | ||
| 588 | if (!partner) | 594 | if (!partner) |
| @@ -1894,13 +1894,16 @@ static int alx_resume(struct device *dev) · 1893 unmodified lines | |||
| 1894 | 1894 | ||
| 1895 | if (!netif_running(alx->dev)) | 1895 | if (!netif_running(alx->dev)) |
| 1896 | return 0; | 1896 | return 0; |
| 1897 | netif_device_attach(alx->dev); | ||
| 1898 | 1897 | ||
| 1899 | rtnl_lock(); | 1898 | rtnl_lock(); |
| 1900 | err = __alx_open(alx, true); | 1899 | err = __alx_open(alx, true); |
| 1901 | rtnl_unlock(); | 1900 | rtnl_unlock(); |
| 1901 | if (err) | ||
| 1902 | return err; | ||
| 1902 | 1903 | ||
| 1903 | return err; | 1904 | netif_device_attach(alx->dev); |
| 1905 | |||
| 1906 | return 0; | ||
| 1904 | } | 1907 | } |
| 1905 | 1908 | ||
| 1906 | static SIMPLE_DEV_PM_OPS(alx_pm_ops, alx_suspend, alx_resume); | 1909 | static SIMPLE_DEV_PM_OPS(alx_pm_ops, alx_suspend, alx_resume); |
| @@ -592,6 +592,9 @@ static int bcm4908_enet_poll(struct napi_struct *napi, int weight) · 591 unmodified lines | |||
| 592 | bcm4908_enet_intrs_on(enet); | 592 | bcm4908_enet_intrs_on(enet); |
| 593 | } | 593 | } |
| 594 | 594 | ||
| 595 | /* Hardware could disable ring if it run out of descriptors */ | ||
| 596 | bcm4908_enet_dma_rx_ring_enable(enet, &enet->rx_ring); | ||
| 597 | |||
| 595 | return handled; | 598 | return handled; |
| 596 | } | 599 | } |
| 597 | 600 | ||
| @@ -8556,10 +8556,18 @@ static void bnxt_setup_inta(struct bnxt *bp) · 8555 unmodified lines | |||
| 8556 | bp->irq_tbl[0].handler = bnxt_inta; | 8556 | bp->irq_tbl[0].handler = bnxt_inta; |
| 8557 | } | 8557 | } |
| 8558 | 8558 | ||
| 8559 | static int bnxt_init_int_mode(struct bnxt *bp); | ||
| 8560 | |||
| 8559 | static int bnxt_setup_int_mode(struct bnxt *bp) | 8561 | static int bnxt_setup_int_mode(struct bnxt *bp) |
| 8560 | { | 8562 | { |
| 8561 | int rc; | 8563 | int rc; |
| 8562 | 8564 | ||
| 8565 | if (!bp->irq_tbl) { | ||
| 8566 | rc = bnxt_init_int_mode(bp); | ||
| 8567 | if (rc || !bp->irq_tbl) | ||
| 8568 | return rc ?: -ENODEV; | ||
| 8569 | } | ||
| 8570 | |||
| 8563 | if (bp->flags & BNXT_FLAG_USING_MSIX) | 8571 | if (bp->flags & BNXT_FLAG_USING_MSIX) |
| 8564 | bnxt_setup_msix(bp); | 8572 | bnxt_setup_msix(bp); |
| 8565 | else | 8573 | else |
| @@ -8744,7 +8752,7 @@ static int bnxt_init_inta(struct bnxt *bp) · 178 unmodified lines | |||
| 8744 | 8752 | ||
| 8745 | static int bnxt_init_int_mode(struct bnxt *bp) | 8753 | static int bnxt_init_int_mode(struct bnxt *bp) |
| 8746 | { | 8754 | { |
| 8747 | int rc = 0; | 8755 | int rc = -ENODEV; |
| 8748 | 8756 | ||
| 8749 | if (bp->flags & BNXT_FLAG_MSIX_CAP) | 8757 | if (bp->flags & BNXT_FLAG_MSIX_CAP) |
| 8750 | rc = bnxt_init_msix(bp); | 8758 | rc = bnxt_init_msix(bp); |
| @@ -9514,7 +9522,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) · 763 unmodified lines | |||
| 9514 | { | 9522 | { |
| 9515 | struct hwrm_func_drv_if_change_output *resp = bp->hwrm_cmd_resp_addr; | 9523 | struct hwrm_func_drv_if_change_output *resp = bp->hwrm_cmd_resp_addr; |
| 9516 | struct hwrm_func_drv_if_change_input req = {0}; | 9524 | struct hwrm_func_drv_if_change_input req = {0}; |
| 9517 | bool resc_reinit = false, fw_reset = false; | 9525 | bool fw_reset = !bp->irq_tbl; |
| 9526 | bool resc_reinit = false; | ||
| 9518 | int rc, retry = 0; | 9527 | int rc, retry = 0; |
| 9519 | u32 flags = 0; | 9528 | u32 flags = 0; |
| 9520 | 9529 | ||
| @@ -9557,6 +9566,7 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) · 36 unmodified lines | |||
| 9557 | 9566 | ||
| 9558 | if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state) && !fw_reset) { | 9567 | if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state) && !fw_reset) { |
| 9559 | netdev_err(bp->dev, "RESET_DONE not set during FW reset.\n"); | 9568 | netdev_err(bp->dev, "RESET_DONE not set during FW reset.\n"); |
| 9569 | set_bit(BNXT_STATE_ABORT_ERR, &bp->state); | ||
| 9560 | return -ENODEV; | 9570 | return -ENODEV; |
| 9561 | } | 9571 | } |
| 9562 | if (resc_reinit || fw_reset) { | 9572 | if (resc_reinit || fw_reset) { |
| @@ -9890,6 +9900,9 @@ static int bnxt_reinit_after_abort(struct bnxt *bp) · 327 unmodified lines | |||
| 9890 | if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) | 9900 | if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) |
| 9891 | return -EBUSY; | 9901 | return -EBUSY; |
| 9892 | 9902 | ||
| 9903 | if (bp->dev->reg_state == NETREG_UNREGISTERED) | ||
| 9904 | return -ENODEV; | ||
| 9905 | |||
| 9893 | rc = bnxt_fw_init_one(bp); | 9906 | rc = bnxt_fw_init_one(bp); |
| 9894 | if (!rc) { | 9907 | if (!rc) { |
| 9895 | bnxt_clear_int_mode(bp); | 9908 | bnxt_clear_int_mode(bp); |
| @@ -3954,6 +3954,13 @@ static int macb_init(struct platform_device *pdev) · 3953 unmodified lines | |||
| 3954 | return 0; | 3954 | return 0; |
| 3955 | } | 3955 | } |
| 3956 | 3956 | ||
| 3957 | static const struct macb_usrio_config macb_default_usrio = { | ||
| 3958 | .mii = MACB_BIT(MII), | ||
| 3959 | .rmii = MACB_BIT(RMII), | ||
| 3960 | .rgmii = GEM_BIT(RGMII), | ||
| 3961 | .refclk = MACB_BIT(CLKEN), | ||
| 3962 | }; | ||
| 3963 | |||
| 3957 | #if defined(CONFIG_OF) | 3964 | #if defined(CONFIG_OF) |
| 3958 | /* 1518 rounded up */ | 3965 | /* 1518 rounded up */ |
| 3959 | #define AT91ETHER_MAX_RBUFF_SZ 0x600 | 3966 | #define AT91ETHER_MAX_RBUFF_SZ 0x600 |
| @@ -4439,13 +4446,6 @@ static int fu540_c000_init(struct platform_device *pdev) · 479 unmodified lines | |||
| 4439 | return macb_init(pdev); | 4446 | return macb_init(pdev); |
| 4440 | } | 4447 | } |
| 4441 | 4448 | ||
| 4442 | static const struct macb_usrio_config macb_default_usrio = { | ||
| 4443 | .mii = MACB_BIT(MII), | ||
| 4444 | .rmii = MACB_BIT(RMII), | ||
| 4445 | .rgmii = GEM_BIT(RGMII), | ||
| 4446 | .refclk = MACB_BIT(CLKEN), | ||
| 4447 | }; | ||
| 4448 | |||
| 4449 | static const struct macb_usrio_config sama7g5_usrio = { | 4449 | static const struct macb_usrio_config sama7g5_usrio = { |
| 4450 | .mii = 0, | 4450 | .mii = 0, |
| 4451 | .rmii = 1, | 4451 | .rmii = 1, |
| @@ -4594,6 +4594,7 @@ static const struct macb_config default_gem_config = { · 142 unmodified lines | |||
| 4594 | .dma_burst_length = 16, | 4594 | .dma_burst_length = 16, |
| 4595 | .clk_init = macb_clk_init, | 4595 | .clk_init = macb_clk_init, |
| 4596 | .init = macb_init, | 4596 | .init = macb_init, |
| 4597 | .usrio = &macb_default_usrio, | ||
| 4597 | .jumbo_max_len = 10240, | 4598 | .jumbo_max_len = 10240, |
| 4598 | }; | 4599 | }; |
| 4599 | 4600 | ||
| @@ -672,7 +672,7 @@ static int chcr_ktls_cpl_act_open_rpl(struct adapter *adap, · 671 unmodified lines | |||
| 672 | if (tx_info->pending_close) { | 672 | if (tx_info->pending_close) { |
| 673 | spin_unlock(&tx_info->lock); | 673 | spin_unlock(&tx_info->lock); |
| 674 | if (!status) { | 674 | if (!status) { |
| 675 | /* it's a late success, tcb status is establised, | 675 | /* it's a late success, tcb status is established, |
| 676 | * mark it close. | 676 | * mark it close. |
| 677 | */ | 677 | */ |
| 678 | chcr_ktls_mark_tcb_close(tx_info); | 678 | chcr_ktls_mark_tcb_close(tx_info); |
| @@ -930,7 +930,7 @@ chcr_ktls_get_tx_flits(u32 nr_frags, unsigned int key_ctx_len) · 251 unmodified lines | |||
| 930 | } | 930 | } |
| 931 | 931 | ||
| 932 | /* | 932 | /* |
| 933 | * chcr_ktls_check_tcp_options: To check if there is any TCP option availbale | 933 | * chcr_ktls_check_tcp_options: To check if there is any TCP option available |
| 934 | * other than timestamp. | 934 | * other than timestamp. |
| 935 | * @skb - skb contains partial record.. | 935 | * @skb - skb contains partial record.. |
| 936 | * return: 1 / 0 | 936 | * return: 1 / 0 |
| @@ -1115,7 +1115,7 @@ static int chcr_ktls_xmit_wr_complete(struct sk_buff *skb, · 178 unmodified lines | |||
| 1115 | } | 1115 | } |
| 1116 | 1116 | ||
| 1117 | if (unlikely(credits < ETHTXQ_STOP_THRES)) { | 1117 | if (unlikely(credits < ETHTXQ_STOP_THRES)) { |
| 1118 | /* Credits are below the threshold vaues, stop the queue after | 1118 | /* Credits are below the threshold values, stop the queue after |
| 1119 | * injecting the Work Request for this packet. | 1119 | * injecting the Work Request for this packet. |
| 1120 | */ | 1120 | */ |
| 1121 | chcr_eth_txq_stop(q); | 1121 | chcr_eth_txq_stop(q); |
| @@ -2006,7 +2006,7 @@ static int chcr_ktls_xmit(struct sk_buff *skb, struct net_device *dev) · 884 unmodified lines | |||
| 2006 | 2006 | ||
| 2007 | /* TCP segments can be in received either complete or partial. | 2007 | /* TCP segments can be in received either complete or partial. |
| 2008 | * chcr_end_part_handler will handle cases if complete record or end | 2008 | * chcr_end_part_handler will handle cases if complete record or end |
| 2009 | * part of the record is received. Incase of partial end part of record, | 2009 | * part of the record is received. In case of partial end part of record, |
| 2010 | * we will send the complete record again. | 2010 | * we will send the complete record again. |
| 2011 | */ | 2011 | */ |
| 2012 | 2012 | ||
| @@ -133,6 +133,8 @@ struct board_info { · 132 unmodified lines | |||
| 133 | u32 wake_state; | 133 | u32 wake_state; |
| 134 | 134 | ||
| 135 | int ip_summed; | 135 | int ip_summed; |
| 136 | |||
| 137 | struct regulator *power_supply; | ||
| 136 | }; | 138 | }; |
| 137 | 139 | ||
| 138 | /* debug code */ | 140 | /* debug code */ |
| @@ -1449,7 +1451,7 @@ dm9000_probe(struct platform_device *pdev) · 1310 unmodified lines | |||
| 1449 | if (ret) { | 1451 | if (ret) { |
| 1450 | dev_err(dev, "failed to request reset gpio %d: %d\n", | 1452 | dev_err(dev, "failed to request reset gpio %d: %d\n", |
| 1451 | reset_gpios, ret); | 1453 | reset_gpios, ret); |
| 1452 | return -ENODEV; | 1454 | goto out_regulator_disable; |
| 1453 | } | 1455 | } |
| 1454 | 1456 | ||
| 1455 | /* According to manual PWRST# Low Period Min 1ms */ | 1457 | /* According to manual PWRST# Low Period Min 1ms */ |
| @@ -1461,8 +1463,10 @@ dm9000_probe(struct platform_device *pdev) · 5 unmodified lines | |||
| 1461 | 1463 | ||
| 1462 | if (!pdata) { | 1464 | if (!pdata) { |
| 1463 | pdata = dm9000_parse_dt(&pdev->dev); | 1465 | pdata = dm9000_parse_dt(&pdev->dev); |
| 1464 | if (IS_ERR(pdata)) | 1466 | if (IS_ERR(pdata)) { |
| 1465 | return PTR_ERR(pdata); | 1467 | ret = PTR_ERR(pdata); |
| 1468 | goto out_regulator_disable; | ||
| 1469 | } | ||
| 1466 | } | 1470 | } |
| 1467 | 1471 | ||
| 1468 | /* Init network device */ | 1472 | /* Init network device */ |
| @@ -1479,6 +1483,8 @@ dm9000_probe(struct platform_device *pdev) · 10 unmodified lines | |||
| 1479 | 1483 | ||
| 1480 | db->dev = &pdev->dev; | 1484 | db->dev = &pdev->dev; |
| 1481 | db->ndev = ndev; | 1485 | db->ndev = ndev; |
| 1486 | if (!IS_ERR(power)) | ||
| 1487 | db->power_supply = power; | ||
| 1482 | 1488 | ||
| 1483 | spin_lock_init(&db->lock); | 1489 | spin_lock_init(&db->lock); |
| 1484 | mutex_init(&db->addr_lock); | 1490 | mutex_init(&db->addr_lock); |
| @@ -1501,7 +1507,7 @@ dm9000_probe(struct platform_device *pdev) · 16 unmodified lines | |||
| 1501 | goto out; | 1507 | goto out; |
| 1502 | } | 1508 | } |
| 1503 | 1509 | ||
| 1504 | db->irq_wake = platform_get_irq(pdev, 1); | 1510 | db->irq_wake = platform_get_irq_optional(pdev, 1); |
| 1505 | if (db->irq_wake >= 0) { | 1511 | if (db->irq_wake >= 0) { |
| 1506 | dev_dbg(db->dev, "wakeup irq %d\n", db->irq_wake); | 1512 | dev_dbg(db->dev, "wakeup irq %d\n", db->irq_wake); |
| 1507 | 1513 | ||
| @@ -1703,6 +1709,10 @@ out: · 195 unmodified lines | |||
| 1703 | dm9000_release_board(pdev, db); | 1709 | dm9000_release_board(pdev, db); |
| 1704 | free_netdev(ndev); | 1710 | free_netdev(ndev); |
| 1705 | 1711 | ||
| 1712 | out_regulator_disable: | ||
| 1713 | if (!IS_ERR(power)) | ||
| 1714 | regulator_disable(power); | ||
| 1715 | |||
| 1706 | return ret; | 1716 | return ret; |
| 1707 | } | 1717 | } |
| 1708 | 1718 | ||
| @@ -1760,10 +1770,13 @@ static int · 51 unmodified lines | |||
| 1760 | dm9000_drv_remove(struct platform_device *pdev) | 1770 | dm9000_drv_remove(struct platform_device *pdev) |
| 1761 | { | 1771 | { |
| 1762 | struct net_device *ndev = platform_get_drvdata(pdev); | 1772 | struct net_device *ndev = platform_get_drvdata(pdev); |
| 1773 | struct board_info *dm = to_dm9000_board(ndev); | ||
| 1763 | 1774 | ||
| 1764 | unregister_netdev(ndev); | 1775 | unregister_netdev(ndev); |
| 1765 | dm9000_release_board(pdev, netdev_priv(ndev)); | 1776 | dm9000_release_board(pdev, dm); |
| 1766 | free_netdev(ndev); /* free device structure */ | 1777 | free_netdev(ndev); /* free device structure */ |
| 1778 | if (dm->power_supply) | ||
| 1779 | regulator_disable(dm->power_supply); | ||
| 1767 | 1780 | ||
| 1768 | dev_dbg(&pdev->dev, "released and freed device\n"); | 1781 | dev_dbg(&pdev->dev, "released and freed device\n"); |
| 1769 | return 0; | 1782 | return 0; |
| @@ -281,6 +281,8 @@ static int enetc_poll(struct napi_struct *napi, int budget) · 280 unmodified lines | |||
| 281 | int work_done; | 281 | int work_done; |
| 282 | int i; | 282 | int i; |
| 283 | 283 | ||
| 284 | enetc_lock_mdio(); | ||
| 285 | |||
| 284 | for (i = 0; i < v->count_tx_rings; i++) | 286 | for (i = 0; i < v->count_tx_rings; i++) |
| 285 | if (!enetc_clean_tx_ring(&v->tx_ring[i], budget)) | 287 | if (!enetc_clean_tx_ring(&v->tx_ring[i], budget)) |
| 286 | complete = false; | 288 | complete = false; |
| @@ -291,8 +293,10 @@ static int enetc_poll(struct napi_struct *napi, int budget) · 4 unmodified lines | |||
| 291 | if (work_done) | 293 | if (work_done) |
| 292 | v->rx_napi_work = true; | 294 | v->rx_napi_work = true; |
| 293 | 295 | ||
| 294 | if (!complete) | 296 | if (!complete) { |
| 297 | enetc_unlock_mdio(); | ||
| 295 | return budget; | 298 | return budget; |
| 299 | } | ||
| 296 | 300 | ||
| 297 | napi_complete_done(napi, work_done); | 301 | napi_complete_done(napi, work_done); |
| 298 | 302 | ||
| @@ -301,8 +305,6 @@ static int enetc_poll(struct napi_struct *napi, int budget) · 2 unmodified lines | |||
| 301 | 305 | ||
| 302 | v->rx_napi_work = false; | 306 | v->rx_napi_work = false; |
| 303 | 307 | ||
| 304 | enetc_lock_mdio(); | ||
| 305 | |||
| 306 | /* enable interrupts */ | 308 | /* enable interrupts */ |
| 307 | enetc_wr_reg_hot(v->rbier, ENETC_RBIER_RXTIE); | 309 | enetc_wr_reg_hot(v->rbier, ENETC_RBIER_RXTIE); |
| 308 | 310 | ||
| @@ -327,8 +329,8 @@ static void enetc_get_tx_tstamp(struct enetc_hw *hw, union enetc_tx_bd *txbd, · 18 unmodified lines | |||
| 327 | { | 329 | { |
| 328 | u32 lo, hi, tstamp_lo; | 330 | u32 lo, hi, tstamp_lo; |
| 329 | 331 | ||
| 330 | lo = enetc_rd(hw, ENETC_SICTR0); | 332 | lo = enetc_rd_hot(hw, ENETC_SICTR0); |
| 331 | hi = enetc_rd(hw, ENETC_SICTR1); | 333 | hi = enetc_rd_hot(hw, ENETC_SICTR1); |
| 332 | tstamp_lo = le32_to_cpu(txbd->wb.tstamp); | 334 | tstamp_lo = le32_to_cpu(txbd->wb.tstamp); |
| 333 | if (lo <= tstamp_lo) | 335 | if (lo <= tstamp_lo) |
| 334 | hi -= 1; | 336 | hi -= 1; |
| @@ -342,6 +344,12 @@ static void enetc_tstamp_tx(struct sk_buff *skb, u64 tstamp) · 7 unmodified lines | |||
| 342 | if (skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS) { | 344 | if (skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS) { |
| 343 | memset(&shhwtstamps, 0, sizeof(shhwtstamps)); | 345 | memset(&shhwtstamps, 0, sizeof(shhwtstamps)); |
| 344 | shhwtstamps.hwtstamp = ns_to_ktime(tstamp); | 346 | shhwtstamps.hwtstamp = ns_to_ktime(tstamp); |
| 347 | /* Ensure skb_mstamp_ns, which might have been populated with | ||
| 348 | * the txtime, is not mistaken for a software timestamp, | ||
| 349 | * because this will prevent the dispatch of our hardware | ||
| 350 | * timestamp to the socket. | ||
| 351 | */ | ||
| 352 | skb->tstamp = ktime_set(0, 0); | ||
| 345 | skb_tstamp_tx(skb, &shhwtstamps); | 353 | skb_tstamp_tx(skb, &shhwtstamps); |
| 346 | } | 354 | } |
| 347 | } | 355 | } |
| @@ -358,9 +366,7 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) · 10 unmodified lines | |||
| 358 | i = tx_ring->next_to_clean; | 366 | i = tx_ring->next_to_clean; |
| 359 | tx_swbd = &tx_ring->tx_swbd[i]; | 367 | tx_swbd = &tx_ring->tx_swbd[i]; |
| 360 | 368 | ||
| 361 | enetc_lock_mdio(); | ||
| 362 | bds_to_clean = enetc_bd_ready_count(tx_ring, i); | 369 | bds_to_clean = enetc_bd_ready_count(tx_ring, i); |
| 363 | enetc_unlock_mdio(); | ||
| 364 | 370 | ||
| 365 | do_tstamp = false; | 371 | do_tstamp = false; |
| 366 | 372 | ||
| @@ -403,8 +409,6 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) · 36 unmodified lines | |||
| 403 | tx_swbd = tx_ring->tx_swbd; | 409 | tx_swbd = tx_ring->tx_swbd; |
| 404 | } | 410 | } |
| 405 | 411 | ||
| 406 | enetc_lock_mdio(); | ||
| 407 | |||
| 408 | /* BD iteration loop end */ | 412 | /* BD iteration loop end */ |
| 409 | if (is_eof) { | 413 | if (is_eof) { |
| 410 | tx_frm_cnt++; | 414 | tx_frm_cnt++; |
| @@ -415,8 +419,6 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) · 4 unmodified lines | |||
| 415 | 419 | ||
| 416 | if (unlikely(!bds_to_clean)) | 420 | if (unlikely(!bds_to_clean)) |
| 417 | bds_to_clean = enetc_bd_ready_count(tx_ring, i); | 421 | bds_to_clean = enetc_bd_ready_count(tx_ring, i); |
| 418 | |||
| 419 | enetc_unlock_mdio(); | ||
| 420 | } | 422 | } |
| 421 | 423 | ||
| 422 | tx_ring->next_to_clean = i; | 424 | tx_ring->next_to_clean = i; |
| @@ -527,9 +529,8 @@ static void enetc_get_rx_tstamp(struct net_device *ndev, · 104 unmodified lines | |||
| 527 | static void enetc_get_offloads(struct enetc_bdr *rx_ring, | 529 | static void enetc_get_offloads(struct enetc_bdr *rx_ring, |
| 528 | union enetc_rx_bd *rxbd, struct sk_buff *skb) | 530 | union enetc_rx_bd *rxbd, struct sk_buff *skb) |
| 529 | { | 531 | { |
| 530 | #ifdef CONFIG_FSL_ENETC_PTP_CLOCK | ||
| 531 | struct enetc_ndev_priv *priv = netdev_priv(rx_ring->ndev); | 532 | struct enetc_ndev_priv *priv = netdev_priv(rx_ring->ndev); |
| 532 | #endif | 533 | |
| 533 | /* TODO: hashing */ | 534 | /* TODO: hashing */ |
| 534 | if (rx_ring->ndev->features & NETIF_F_RXCSUM) { | 535 | if (rx_ring->ndev->features & NETIF_F_RXCSUM) { |
| 535 | u16 inet_csum = le16_to_cpu(rxbd->r.inet_csum); | 536 | u16 inet_csum = le16_to_cpu(rxbd->r.inet_csum); |
| @@ -538,12 +539,31 @@ static void enetc_get_offloads(struct enetc_bdr *rx_ring, · 2 unmodified lines | |||
| 538 | skb->ip_summed = CHECKSUM_COMPLETE; | 539 | skb->ip_summed = CHECKSUM_COMPLETE; |
| 539 | } | 540 | } |
| 540 | 541 | ||
| 541 | /* copy VLAN to skb, if one is extracted, for now we assume it's a | 542 | if (le16_to_cpu(rxbd->r.flags) & ENETC_RXBD_FLAG_VLAN) { |
| 542 | * standard TPID, but HW also supports custom values | 543 | __be16 tpid = 0; |
| 543 | */ | 544 | |
| 544 | if (le16_to_cpu(rxbd->r.flags) & ENETC_RXBD_FLAG_VLAN) | 545 | switch (le16_to_cpu(rxbd->r.flags) & ENETC_RXBD_FLAG_TPID) { |
| 545 | __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), | 546 | case 0: |
| 546 | le16_to_cpu(rxbd->r.vlan_opt)); | 547 | tpid = htons(ETH_P_8021Q); |
| 548 | break; | ||
| 549 | case 1: | ||
| 550 | tpid = htons(ETH_P_8021AD); | ||
| 551 | break; | ||
| 552 | case 2: | ||
| 553 | tpid = htons(enetc_port_rd(&priv->si->hw, | ||
| 554 | ENETC_PCVLANR1)); | ||
| 555 | break; | ||
| 556 | case 3: | ||
| 557 | tpid = htons(enetc_port_rd(&priv->si->hw, | ||
| 558 | ENETC_PCVLANR2)); | ||
| 559 | break; | ||
| 560 | default: | ||
| 561 | break; | ||
| 562 | } | ||
| 563 | |||
| 564 | __vlan_hwaccel_put_tag(skb, tpid, le16_to_cpu(rxbd->r.vlan_opt)); | ||
| 565 | } | ||
| 566 | |||
| 547 | #ifdef CONFIG_FSL_ENETC_PTP_CLOCK | 567 | #ifdef CONFIG_FSL_ENETC_PTP_CLOCK |
| 548 | if (priv->active_offloads & ENETC_F_RX_TSTAMP) | 568 | if (priv->active_offloads & ENETC_F_RX_TSTAMP) |
| 549 | enetc_get_rx_tstamp(rx_ring->ndev, rxbd, skb); | 569 | enetc_get_rx_tstamp(rx_ring->ndev, rxbd, skb); |
| @@ -660,8 +680,6 @@ static int enetc_clean_rx_ring(struct enetc_bdr *rx_ring, · 110 unmodified lines | |||
| 660 | u32 bd_status; | 680 | u32 bd_status; |
| 661 | u16 size; | 681 | u16 size; |
| 662 | 682 | ||
| 663 | enetc_lock_mdio(); | ||
| 664 | |||
| 665 | if (cleaned_cnt >= ENETC_RXBD_BUNDLE) { | 683 | if (cleaned_cnt >= ENETC_RXBD_BUNDLE) { |
| 666 | int count = enetc_refill_rx_ring(rx_ring, cleaned_cnt); | 684 | int count = enetc_refill_rx_ring(rx_ring, cleaned_cnt); |
| 667 | 685 | ||
| @@ -672,19 +690,15 @@ static int enetc_clean_rx_ring(struct enetc_bdr *rx_ring, · 4 unmodified lines | |||
| 672 | 690 | ||
| 673 | rxbd = enetc_rxbd(rx_ring, i); | 691 | rxbd = enetc_rxbd(rx_ring, i); |
| 674 | bd_status = le32_to_cpu(rxbd->r.lstatus); | 692 | bd_status = le32_to_cpu(rxbd->r.lstatus); |
| 675 | if (!bd_status) { | 693 | if (!bd_status) |
| 676 | enetc_unlock_mdio(); | ||
| 677 | break; | 694 | break; |
| 678 | } | ||
| 679 | 695 | ||
| 680 | enetc_wr_reg_hot(rx_ring->idr, BIT(rx_ring->index)); | 696 | enetc_wr_reg_hot(rx_ring->idr, BIT(rx_ring->index)); |
| 681 | dma_rmb(); /* for reading other rxbd fields */ | 697 | dma_rmb(); /* for reading other rxbd fields */ |
| 682 | size = le16_to_cpu(rxbd->r.buf_len); | 698 | size = le16_to_cpu(rxbd->r.buf_len); |
| 683 | skb = enetc_map_rx_buff_to_skb(rx_ring, i, size); | 699 | skb = enetc_map_rx_buff_to_skb(rx_ring, i, size); |
| 684 | if (!skb) { | 700 | if (!skb) |
| 685 | enetc_unlock_mdio(); | ||
| 686 | break; | 701 | break; |
| 687 | } | ||
| 688 | 702 | ||
| 689 | enetc_get_offloads(rx_ring, rxbd, skb); | 703 | enetc_get_offloads(rx_ring, rxbd, skb); |
| 690 | 704 | ||
| @@ -696,7 +710,6 @@ static int enetc_clean_rx_ring(struct enetc_bdr *rx_ring, · 5 unmodified lines | |||
| 696 | 710 | ||
| 697 | if (unlikely(bd_status & | 711 | if (unlikely(bd_status & |
| 698 | ENETC_RXBD_LSTATUS(ENETC_RXBD_ERR_MASK))) { | 712 | ENETC_RXBD_LSTATUS(ENETC_RXBD_ERR_MASK))) { |
| 699 | enetc_unlock_mdio(); | ||
| 700 | dev_kfree_skb(skb); | 713 | dev_kfree_skb(skb); |
| 701 | while (!(bd_status & ENETC_RXBD_LSTATUS_F)) { | 714 | while (!(bd_status & ENETC_RXBD_LSTATUS_F)) { |
| 702 | dma_rmb(); | 715 | dma_rmb(); |
| @@ -736,8 +749,6 @@ static int enetc_clean_rx_ring(struct enetc_bdr *rx_ring, · 33 unmodified lines | |||
| 736 | 749 | ||
| 737 | enetc_process_skb(rx_ring, skb); | 750 | enetc_process_skb(rx_ring, skb); |
| 738 | 751 | ||
| 739 | enetc_unlock_mdio(); | ||
| 740 | |||
| 741 | napi_gro_receive(napi, skb); | 752 | napi_gro_receive(napi, skb); |
| 742 | 753 | ||
| 743 | rx_frm_cnt++; | 754 | rx_frm_cnt++; |
| @@ -984,7 +995,7 @@ static void enetc_free_rxtx_rings(struct enetc_ndev_priv *priv) · 240 unmodified lines | |||
| 984 | enetc_free_tx_ring(priv->tx_ring[i]); | 995 | enetc_free_tx_ring(priv->tx_ring[i]); |
| 985 | } | 996 | } |
| 986 | 997 | ||
| 987 | static int enetc_alloc_cbdr(struct device *dev, struct enetc_cbdr *cbdr) | 998 | int enetc_alloc_cbdr(struct device *dev, struct enetc_cbdr *cbdr) |
| 988 | { | 999 | { |
| 989 | int size = cbdr->bd_count * sizeof(struct enetc_cbd); | 1000 | int size = cbdr->bd_count * sizeof(struct enetc_cbd); |
| 990 | 1001 | ||
| @@ -1005,7 +1016,7 @@ static int enetc_alloc_cbdr(struct device *dev, struct enetc_cbdr *cbdr) · 14 unmodified lines | |||
| 1005 | return 0; | 1016 | return 0; |
| 1006 | } | 1017 | } |
| 1007 | 1018 | ||
| 1008 | static void enetc_free_cbdr(struct device *dev, struct enetc_cbdr *cbdr) | 1019 | void enetc_free_cbdr(struct device *dev, struct enetc_cbdr *cbdr) |
| 1009 | { | 1020 | { |
| 1010 | int size = cbdr->bd_count * sizeof(struct enetc_cbd); | 1021 | int size = cbdr->bd_count * sizeof(struct enetc_cbd); |
| 1011 | 1022 | ||
| @@ -1013,7 +1024,7 @@ static void enetc_free_cbdr(struct device *dev, struct enetc_cbdr *cbdr) · 1 unmodified line | |||
| 1013 | cbdr->bd_base = NULL; | 1024 | cbdr->bd_base = NULL; |
| 1014 | } | 1025 | } |
| 1015 | 1026 | ||
| 1016 | static void enetc_setup_cbdr(struct enetc_hw *hw, struct enetc_cbdr *cbdr) | 1027 | void enetc_setup_cbdr(struct enetc_hw *hw, struct enetc_cbdr *cbdr) |
| 1017 | { | 1028 | { |
| 1018 | /* set CBDR cache attributes */ | 1029 | /* set CBDR cache attributes */ |
| 1019 | enetc_wr(hw, ENETC_SICAR2, | 1030 | enetc_wr(hw, ENETC_SICAR2, |
| @@ -1033,7 +1044,7 @@ static void enetc_setup_cbdr(struct enetc_hw *hw, struct enetc_cbdr *cbdr) · 13 unmodified lines | |||
| 1033 | cbdr->cir = hw->reg + ENETC_SICBDRCIR; | 1044 | cbdr->cir = hw->reg + ENETC_SICBDRCIR; |
| 1034 | } | 1045 | } |
| 1035 | 1046 | ||
| 1036 | static void enetc_clear_cbdr(struct enetc_hw *hw) | 1047 | void enetc_clear_cbdr(struct enetc_hw *hw) |
| 1037 | { | 1048 | { |
| 1038 | enetc_wr(hw, ENETC_SICBDRMR, 0); | 1049 | enetc_wr(hw, ENETC_SICBDRMR, 0); |
| 1039 | } | 1050 | } |
| @@ -1058,13 +1069,12 @@ static int enetc_setup_default_rss_table(struct enetc_si *si, int num_groups) · 18 unmodified lines | |||
| 1058 | return 0; | 1069 | return 0; |
| 1059 | } | 1070 | } |
| 1060 | 1071 | ||
| 1061 | static int enetc_configure_si(struct enetc_ndev_priv *priv) | 1072 | int enetc_configure_si(struct enetc_ndev_priv *priv) |
| 1062 | { | 1073 | { |
| 1063 | struct enetc_si *si = priv->si; | 1074 | struct enetc_si *si = priv->si; |
| 1064 | struct enetc_hw *hw = &si->hw; | 1075 | struct enetc_hw *hw = &si->hw; |
| 1065 | int err; | 1076 | int err; |
| 1066 | 1077 | ||
| 1067 | enetc_setup_cbdr(hw, &si->cbd_ring); | ||
| 1068 | /* set SI cache attributes */ | 1078 | /* set SI cache attributes */ |
| 1069 | enetc_wr(hw, ENETC_SICAR0, | 1079 | enetc_wr(hw, ENETC_SICAR0, |
| 1070 | ENETC_SICAR_RD_COHERENT | ENETC_SICAR_WR_COHERENT); | 1080 | ENETC_SICAR_RD_COHERENT | ENETC_SICAR_WR_COHERENT); |
| @@ -1112,6 +1122,8 @@ int enetc_alloc_si_resources(struct enetc_ndev_priv *priv) · 41 unmodified lines | |||
| 1112 | if (err) | 1122 | if (err) |
| 1113 | return err; | 1123 | return err; |
| 1114 | 1124 | ||
| 1125 | enetc_setup_cbdr(&si->hw, &si->cbd_ring); | ||
| 1126 | |||
| 1115 | priv->cls_rules = kcalloc(si->num_fs_entries, sizeof(*priv->cls_rules), | 1127 | priv->cls_rules = kcalloc(si->num_fs_entries, sizeof(*priv->cls_rules), |
| 1116 | GFP_KERNEL); | 1128 | GFP_KERNEL); |
| 1117 | if (!priv->cls_rules) { | 1129 | if (!priv->cls_rules) { |
| @@ -1119,14 +1131,8 @@ int enetc_alloc_si_resources(struct enetc_ndev_priv *priv) · 1 unmodified line | |||
| 1119 | goto err_alloc_cls; | 1131 | goto err_alloc_cls; |
| 1120 | } | 1132 | } |
| 1121 | 1133 | ||
| 1122 | err = enetc_configure_si(priv); | ||
| 1123 | if (err) | ||
| 1124 | goto err_config_si; | ||
| 1125 | |||
| 1126 | return 0; | 1134 | return 0; |
| 1127 | 1135 | ||
| 1128 | err_config_si: | ||
| 1129 | kfree(priv->cls_rules); | ||
| 1130 | err_alloc_cls: | 1136 | err_alloc_cls: |
| 1131 | enetc_clear_cbdr(&si->hw); | 1137 | enetc_clear_cbdr(&si->hw); |
| 1132 | enetc_free_cbdr(priv->dev, &si->cbd_ring); | 1138 | enetc_free_cbdr(priv->dev, &si->cbd_ring); |
| @@ -1212,7 +1218,8 @@ static void enetc_setup_rxbdr(struct enetc_hw *hw, struct enetc_bdr *rx_ring) · 79 unmodified lines | |||
| 1212 | rx_ring->idr = hw->reg + ENETC_SIRXIDR; | 1218 | rx_ring->idr = hw->reg + ENETC_SIRXIDR; |
| 1213 | 1219 | ||
| 1214 | enetc_refill_rx_ring(rx_ring, enetc_bd_unused(rx_ring)); | 1220 | enetc_refill_rx_ring(rx_ring, enetc_bd_unused(rx_ring)); |
| 1215 | enetc_wr(hw, ENETC_SIRXIDR, rx_ring->next_to_use); | 1221 | /* update ENETC's consumer index */ |
| 1222 | enetc_rxbdr_wr(hw, idx, ENETC_RBCIR, rx_ring->next_to_use); | ||
| 1216 | 1223 | ||
| 1217 | /* enable ring */ | 1224 | /* enable ring */ |
| 1218 | enetc_rxbdr_wr(hw, idx, ENETC_RBMR, rbmr); | 1225 | enetc_rxbdr_wr(hw, idx, ENETC_RBMR, rbmr); |
| @@ -292,6 +292,7 @@ void enetc_get_si_caps(struct enetc_si *si); · 291 unmodified lines | |||
| 292 | void enetc_init_si_rings_params(struct enetc_ndev_priv *priv); | 292 | void enetc_init_si_rings_params(struct enetc_ndev_priv *priv); |
| 293 | int enetc_alloc_si_resources(struct enetc_ndev_priv *priv); | 293 | int enetc_alloc_si_resources(struct enetc_ndev_priv *priv); |
| 294 | void enetc_free_si_resources(struct enetc_ndev_priv *priv); | 294 | void enetc_free_si_resources(struct enetc_ndev_priv *priv); |
| 295 | int enetc_configure_si(struct enetc_ndev_priv *priv); | ||
| 295 | 296 | ||
| 296 | int enetc_open(struct net_device *ndev); | 297 | int enetc_open(struct net_device *ndev); |
| 297 | int enetc_close(struct net_device *ndev); | 298 | int enetc_close(struct net_device *ndev); |
| @@ -309,6 +310,10 @@ int enetc_setup_tc(struct net_device *ndev, enum tc_setup_type type, · 11 unmodified lines | |||
| 309 | void enetc_set_ethtool_ops(struct net_device *ndev); | 310 | void enetc_set_ethtool_ops(struct net_device *ndev); |
| 310 | 311 | ||
| 311 | /* control buffer descriptor ring (CBDR) */ | 312 | /* control buffer descriptor ring (CBDR) */ |
| 313 | int enetc_alloc_cbdr(struct device *dev, struct enetc_cbdr *cbdr); | ||
| 314 | void enetc_free_cbdr(struct device *dev, struct enetc_cbdr *cbdr); | ||
| 315 | void enetc_setup_cbdr(struct enetc_hw *hw, struct enetc_cbdr *cbdr); | ||
| 316 | void enetc_clear_cbdr(struct enetc_hw *hw); | ||
| 312 | int enetc_set_mac_flt_entry(struct enetc_si *si, int index, | 317 | int enetc_set_mac_flt_entry(struct enetc_si *si, int index, |
| 313 | char *mac_addr, int si_map); | 318 | char *mac_addr, int si_map); |
| 314 | int enetc_clear_mac_flt_entry(struct enetc_si *si, int index); | 319 | int enetc_clear_mac_flt_entry(struct enetc_si *si, int index); |
| @@ -172,6 +172,8 @@ enum enetc_bdr_type {TX, RX}; · 171 unmodified lines | |||
| 172 | #define ENETC_PSIPMAR0(n) (0x0100 + (n) * 0x8) /* n = SI index */ | 172 | #define ENETC_PSIPMAR0(n) (0x0100 + (n) * 0x8) /* n = SI index */ |
| 173 | #define ENETC_PSIPMAR1(n) (0x0104 + (n) * 0x8) | 173 | #define ENETC_PSIPMAR1(n) (0x0104 + (n) * 0x8) |
| 174 | #define ENETC_PVCLCTR 0x0208 | 174 | #define ENETC_PVCLCTR 0x0208 |
| 175 | #define ENETC_PCVLANR1 0x0210 | ||
| 176 | #define ENETC_PCVLANR2 0x0214 | ||
| 175 | #define ENETC_VLAN_TYPE_C BIT(0) | 177 | #define ENETC_VLAN_TYPE_C BIT(0) |
| 176 | #define ENETC_VLAN_TYPE_S BIT(1) | 178 | #define ENETC_VLAN_TYPE_S BIT(1) |
| 177 | #define ENETC_PVCLCTR_OVTPIDL(bmp) ((bmp) & 0xff) /* VLAN_TYPE */ | 179 | #define ENETC_PVCLCTR_OVTPIDL(bmp) ((bmp) & 0xff) /* VLAN_TYPE */ |
| @@ -232,14 +234,23 @@ enum enetc_bdr_type {TX, RX}; · 54 unmodified lines | |||
| 232 | #define ENETC_PM0_MAXFRM 0x8014 | 234 | #define ENETC_PM0_MAXFRM 0x8014 |
| 233 | #define ENETC_SET_TX_MTU(val) ((val) << 16) | 235 | #define ENETC_SET_TX_MTU(val) ((val) << 16) |
| 234 | #define ENETC_SET_MAXFRM(val) ((val) & 0xffff) | 236 | #define ENETC_SET_MAXFRM(val) ((val) & 0xffff) |
| 237 | #define ENETC_PM0_RX_FIFO 0x801c | ||
| 238 | #define ENETC_PM0_RX_FIFO_VAL 1 | ||
| 235 | 239 | ||
| 236 | #define ENETC_PM_IMDIO_BASE 0x8030 | 240 | #define ENETC_PM_IMDIO_BASE 0x8030 |
| 237 | 241 | ||
| 238 | #define ENETC_PM0_IF_MODE 0x8300 | 242 | #define ENETC_PM0_IF_MODE 0x8300 |
| 239 | #define ENETC_PMO_IFM_RG BIT(2) | 243 | #define ENETC_PM0_IFM_RG BIT(2) |
| 240 | #define ENETC_PM0_IFM_RLP (BIT(5) | BIT(11)) | 244 | #define ENETC_PM0_IFM_RLP (BIT(5) | BIT(11)) |
| 241 | #define ENETC_PM0_IFM_RGAUTO (BIT(15) | ENETC_PMO_IFM_RG | BIT(1)) | 245 | #define ENETC_PM0_IFM_EN_AUTO BIT(15) |
| 242 | #define ENETC_PM0_IFM_XGMII BIT(12) | 246 | #define ENETC_PM0_IFM_SSP_MASK GENMASK(14, 13) |
| 247 | #define ENETC_PM0_IFM_SSP_1000 (2 << 13) | ||
| 248 | #define ENETC_PM0_IFM_SSP_100 (0 << 13) | ||
| 249 | #define ENETC_PM0_IFM_SSP_10 (1 << 13) | ||
| 250 | #define ENETC_PM0_IFM_FULL_DPX BIT(12) | ||
| 251 | #define ENETC_PM0_IFM_IFMODE_MASK GENMASK(1, 0) | ||
| 252 | #define ENETC_PM0_IFM_IFMODE_XGMII 0 | ||
| 253 | #define ENETC_PM0_IFM_IFMODE_GMII 2 | ||
| 243 | #define ENETC_PSIDCAPR 0x1b08 | 254 | #define ENETC_PSIDCAPR 0x1b08 |
| 244 | #define ENETC_PSIDCAPR_MSK GENMASK(15, 0) | 255 | #define ENETC_PSIDCAPR_MSK GENMASK(15, 0) |
| 245 | #define ENETC_PSFCAPR 0x1b18 | 256 | #define ENETC_PSFCAPR 0x1b18 |
| @@ -453,6 +464,8 @@ static inline u64 _enetc_rd_reg64_wa(void __iomem *reg) · 207 unmodified lines | |||
| 453 | #define enetc_wr_reg(reg, val) _enetc_wr_reg_wa((reg), (val)) | 464 | #define enetc_wr_reg(reg, val) _enetc_wr_reg_wa((reg), (val)) |
| 454 | #define enetc_rd(hw, off) enetc_rd_reg((hw)->reg + (off)) | 465 | #define enetc_rd(hw, off) enetc_rd_reg((hw)->reg + (off)) |
| 455 | #define enetc_wr(hw, off, val) enetc_wr_reg((hw)->reg + (off), val) | 466 | #define enetc_wr(hw, off, val) enetc_wr_reg((hw)->reg + (off), val) |
| 467 | #define enetc_rd_hot(hw, off) enetc_rd_reg_hot((hw)->reg + (off)) | ||
| 468 | #define enetc_wr_hot(hw, off, val) enetc_wr_reg_hot((hw)->reg + (off), val) | ||
| 456 | #define enetc_rd64(hw, off) _enetc_rd_reg64_wa((hw)->reg + (off)) | 469 | #define enetc_rd64(hw, off) _enetc_rd_reg64_wa((hw)->reg + (off)) |
| 457 | /* port register accessors - PF only */ | 470 | /* port register accessors - PF only */ |
| 458 | #define enetc_port_rd(hw, off) enetc_rd_reg((hw)->port + (off)) | 471 | #define enetc_port_rd(hw, off) enetc_rd_reg((hw)->port + (off)) |
| @@ -568,6 +581,7 @@ union enetc_rx_bd { · 109 unmodified lines | |||
| 568 | #define ENETC_RXBD_LSTATUS(flags) ((flags) << 16) | 581 | #define ENETC_RXBD_LSTATUS(flags) ((flags) << 16) |
| 569 | #define ENETC_RXBD_FLAG_VLAN BIT(9) | 582 | #define ENETC_RXBD_FLAG_VLAN BIT(9) |
| 570 | #define ENETC_RXBD_FLAG_TSTMP BIT(10) | 583 | #define ENETC_RXBD_FLAG_TSTMP BIT(10) |
| 584 | #define ENETC_RXBD_FLAG_TPID GENMASK(1, 0) | ||
| 571 | 585 | ||
| 572 | #define ENETC_MAC_ADDR_FILT_CNT 8 /* # of supported entries per port */ | 586 | #define ENETC_MAC_ADDR_FILT_CNT 8 /* # of supported entries per port */ |
| 573 | #define EMETC_MAC_ADDR_FILT_RES 3 /* # of reserved entries at the beginning */ | 587 | #define EMETC_MAC_ADDR_FILT_RES 3 /* # of reserved entries at the beginning */ |
| @@ -190,7 +190,6 @@ static void enetc_pf_set_rx_mode(struct net_device *ndev) · 189 unmodified lines | |||
| 190 | { | 190 | { |
| 191 | struct enetc_ndev_priv *priv = netdev_priv(ndev); | 191 | struct enetc_ndev_priv *priv = netdev_priv(ndev); |
| 192 | struct enetc_pf *pf = enetc_si_priv(priv->si); | 192 | struct enetc_pf *pf = enetc_si_priv(priv->si); |
| 193 | char vlan_promisc_simap = pf->vlan_promisc_simap; | ||
| 194 | struct enetc_hw *hw = &priv->si->hw; | 193 | struct enetc_hw *hw = &priv->si->hw; |
| 195 | bool uprom = false, mprom = false; | 194 | bool uprom = false, mprom = false; |
| 196 | struct enetc_mac_filter *filter; | 195 | struct enetc_mac_filter *filter; |
| @@ -203,16 +202,12 @@ static void enetc_pf_set_rx_mode(struct net_device *ndev) · 6 unmodified lines | |||
| 203 | psipmr = ENETC_PSIPMR_SET_UP(0) | ENETC_PSIPMR_SET_MP(0); | 202 | psipmr = ENETC_PSIPMR_SET_UP(0) | ENETC_PSIPMR_SET_MP(0); |
| 204 | uprom = true; | 203 | uprom = true; |
| 205 | mprom = true; | 204 | mprom = true; |
| 206 | /* Enable VLAN promiscuous mode for SI0 (PF) */ | ||
| 207 | vlan_promisc_simap |= BIT(0); | ||
| 208 | } else if (ndev->flags & IFF_ALLMULTI) { | 205 | } else if (ndev->flags & IFF_ALLMULTI) { |
| 209 | /* enable multi cast promisc mode for SI0 (PF) */ | 206 | /* enable multi cast promisc mode for SI0 (PF) */ |
| 210 | psipmr = ENETC_PSIPMR_SET_MP(0); | 207 | psipmr = ENETC_PSIPMR_SET_MP(0); |
| 211 | mprom = true; | 208 | mprom = true; |
| 212 | } | 209 | } |
| 213 | 210 | ||
| 214 | enetc_set_vlan_promisc(&pf->si->hw, vlan_promisc_simap); | ||
| 215 | |||
| 216 | /* first 2 filter entries belong to PF */ | 211 | /* first 2 filter entries belong to PF */ |
| 217 | if (!uprom) { | 212 | if (!uprom) { |
| 218 | /* Update unicast filters */ | 213 | /* Update unicast filters */ |
| @@ -320,7 +315,7 @@ static void enetc_set_loopback(struct net_device *ndev, bool en) · 101 unmodified lines | |||
| 320 | u32 reg; | 315 | u32 reg; |
| 321 | 316 | ||
| 322 | reg = enetc_port_rd(hw, ENETC_PM0_IF_MODE); | 317 | reg = enetc_port_rd(hw, ENETC_PM0_IF_MODE); |
| 323 | if (reg & ENETC_PMO_IFM_RG) { | 318 | if (reg & ENETC_PM0_IFM_RG) { |
| 324 | /* RGMII mode */ | 319 | /* RGMII mode */ |
| 325 | reg = (reg & ~ENETC_PM0_IFM_RLP) | | 320 | reg = (reg & ~ENETC_PM0_IFM_RLP) | |
| 326 | (en ? ENETC_PM0_IFM_RLP : 0); | 321 | (en ? ENETC_PM0_IFM_RLP : 0); |
| @@ -495,17 +490,30 @@ static void enetc_configure_port_mac(struct enetc_hw *hw) · 168 unmodified lines | |||
| 495 | 490 | ||
| 496 | enetc_port_wr(hw, ENETC_PM1_CMD_CFG, ENETC_PM0_CMD_PHY_TX_EN | | 491 | enetc_port_wr(hw, ENETC_PM1_CMD_CFG, ENETC_PM0_CMD_PHY_TX_EN | |
| 497 | ENETC_PM0_CMD_TXP | ENETC_PM0_PROMISC); | 492 | ENETC_PM0_CMD_TXP | ENETC_PM0_PROMISC); |
| 493 | |||
| 494 | /* On LS1028A, the MAC RX FIFO defaults to 2, which is too high | ||
| 495 | * and may lead to RX lock-up under traffic. Set it to 1 instead, | ||
| 496 | * as recommended by the hardware team. | ||
| 497 | */ | ||
| 498 | enetc_port_wr(hw, ENETC_PM0_RX_FIFO, ENETC_PM0_RX_FIFO_VAL); | ||
| 498 | } | 499 | } |
| 499 | 500 | ||
| 500 | static void enetc_mac_config(struct enetc_hw *hw, phy_interface_t phy_mode) | 501 | static void enetc_mac_config(struct enetc_hw *hw, phy_interface_t phy_mode) |
| 501 | { | 502 | { |
| 502 | /* set auto-speed for RGMII */ | 503 | u32 val; |
| 503 | if (enetc_port_rd(hw, ENETC_PM0_IF_MODE) & ENETC_PMO_IFM_RG || | 504 | |
| 504 | phy_interface_mode_is_rgmii(phy_mode)) | 505 | if (phy_interface_mode_is_rgmii(phy_mode)) { |
| 505 | enetc_port_wr(hw, ENETC_PM0_IF_MODE, ENETC_PM0_IFM_RGAUTO); | 506 | val = enetc_port_rd(hw, ENETC_PM0_IF_MODE); |
| 507 | val &= ~ENETC_PM0_IFM_EN_AUTO; | ||
| 508 | val &= ENETC_PM0_IFM_IFMODE_MASK; | ||
| 509 | val |= ENETC_PM0_IFM_IFMODE_GMII | ENETC_PM0_IFM_RG; | ||
| 510 | enetc_port_wr(hw, ENETC_PM0_IF_MODE, val); | ||
| 511 | } | ||
| 506 | 512 | ||
| 507 | if (phy_mode == PHY_INTERFACE_MODE_USXGMII) | 513 | if (phy_mode == PHY_INTERFACE_MODE_USXGMII) { |
| 508 | enetc_port_wr(hw, ENETC_PM0_IF_MODE, ENETC_PM0_IFM_XGMII); | 514 | val = ENETC_PM0_IFM_FULL_DPX | ENETC_PM0_IFM_IFMODE_XGMII; |
| 515 | enetc_port_wr(hw, ENETC_PM0_IF_MODE, val); | ||
| 516 | } | ||
| 509 | } | 517 | } |
| 510 | 518 | ||
| 511 | static void enetc_mac_enable(struct enetc_hw *hw, bool en) | 519 | static void enetc_mac_enable(struct enetc_hw *hw, bool en) |
| @@ -937,6 +945,34 @@ static void enetc_pl_mac_config(struct phylink_config *config, · 425 unmodified lines | |||
| 937 | phylink_set_pcs(priv->phylink, &pf->pcs->pcs); | 945 | phylink_set_pcs(priv->phylink, &pf->pcs->pcs); |
| 938 | } | 946 | } |
| 939 | 947 | ||
| 948 | static void enetc_force_rgmii_mac(struct enetc_hw *hw, int speed, int duplex) | ||
| 949 | { | ||
| 950 | u32 old_val, val; | ||
| 951 | |||
| 952 | old_val = val = enetc_port_rd(hw, ENETC_PM0_IF_MODE); | ||
| 953 | |||
| 954 | if (speed == SPEED_1000) { | ||
| 955 | val &= ~ENETC_PM0_IFM_SSP_MASK; | ||
| 956 | val |= ENETC_PM0_IFM_SSP_1000; | ||
| 957 | } else if (speed == SPEED_100) { | ||
| 958 | val &= ~ENETC_PM0_IFM_SSP_MASK; | ||
| 959 | val |= ENETC_PM0_IFM_SSP_100; | ||
| 960 | } else if (speed == SPEED_10) { | ||
| 961 | val &= ~ENETC_PM0_IFM_SSP_MASK; | ||
| 962 | val |= ENETC_PM0_IFM_SSP_10; | ||
| 963 | } | ||
| 964 | |||
| 965 | if (duplex == DUPLEX_FULL) | ||
| 966 | val |= ENETC_PM0_IFM_FULL_DPX; | ||
| 967 | else | ||
| 968 | val &= ~ENETC_PM0_IFM_FULL_DPX; | ||
| 969 | |||
| 970 | if (val == old_val) | ||
| 971 | return; | ||
| 972 | |||
| 973 | enetc_port_wr(hw, ENETC_PM0_IF_MODE, val); | ||
| 974 | } | ||
| 975 | |||
| 940 | static void enetc_pl_mac_link_up(struct phylink_config *config, | 976 | static void enetc_pl_mac_link_up(struct phylink_config *config, |
| 941 | struct phy_device *phy, unsigned int mode, | 977 | struct phy_device *phy, unsigned int mode, |
| 942 | phy_interface_t interface, int speed, | 978 | phy_interface_t interface, int speed, |
| @@ -949,6 +985,10 @@ static void enetc_pl_mac_link_up(struct phylink_config *config, · 6 unmodified lines | |||
| 949 | if (priv->active_offloads & ENETC_F_QBV) | 985 | if (priv->active_offloads & ENETC_F_QBV) |
| 950 | enetc_sched_speed_set(priv, speed); | 986 | enetc_sched_speed_set(priv, speed); |
| 951 | 987 | ||
| 988 | if (!phylink_autoneg_inband(mode) && | ||
| 989 | phy_interface_mode_is_rgmii(interface)) | ||
| 990 | enetc_force_rgmii_mac(&pf->si->hw, speed, duplex); | ||
| 991 | |||
| 952 | enetc_mac_enable(&pf->si->hw, true); | 992 | enetc_mac_enable(&pf->si->hw, true); |
| 953 | } | 993 | } |
| 954 | 994 | ||
| @@ -1041,6 +1081,26 @@ static int enetc_init_port_rss_memory(struct enetc_si *si) · 86 unmodified lines | |||
| 1041 | return err; | 1081 | return err; |
| 1042 | } | 1082 | } |
| 1043 | 1083 | ||
| 1084 | static void enetc_init_unused_port(struct enetc_si *si) | ||
| 1085 | { | ||
| 1086 | struct device *dev = &si->pdev->dev; | ||
| 1087 | struct enetc_hw *hw = &si->hw; | ||
| 1088 | int err; | ||
| 1089 | |||
| 1090 | si->cbd_ring.bd_count = ENETC_CBDR_DEFAULT_SIZE; | ||
| 1091 | err = enetc_alloc_cbdr(dev, &si->cbd_ring); | ||
| 1092 | if (err) | ||
| 1093 | return; | ||
| 1094 | |||
| 1095 | enetc_setup_cbdr(hw, &si->cbd_ring); | ||
| 1096 | |||
| 1097 | enetc_init_port_rfs_memory(si); | ||
| 1098 | enetc_init_port_rss_memory(si); | ||
| 1099 | |||
| 1100 | enetc_clear_cbdr(hw); | ||
| 1101 | enetc_free_cbdr(dev, &si->cbd_ring); | ||
| 1102 | } | ||
| 1103 | |||
| 1044 | static int enetc_pf_probe(struct pci_dev *pdev, | 1104 | static int enetc_pf_probe(struct pci_dev *pdev, |
| 1045 | const struct pci_device_id *ent) | 1105 | const struct pci_device_id *ent) |
| 1046 | { | 1106 | { |
| @@ -1051,11 +1111,6 @@ static int enetc_pf_probe(struct pci_dev *pdev, · 4 unmodified lines | |||
| 1051 | struct enetc_pf *pf; | 1111 | struct enetc_pf *pf; |
| 1052 | int err; | 1112 | int err; |
| 1053 | 1113 | ||
| 1054 | if (node && !of_device_is_available(node)) { | ||
| 1055 | dev_info(&pdev->dev, "device is disabled, skipping\n"); | ||
| 1056 | return -ENODEV; | ||
| 1057 | } | ||
| 1058 | |||
| 1059 | err = enetc_pci_probe(pdev, KBUILD_MODNAME, sizeof(*pf)); | 1114 | err = enetc_pci_probe(pdev, KBUILD_MODNAME, sizeof(*pf)); |
| 1060 | if (err) { | 1115 | if (err) { |
| 1061 | dev_err(&pdev->dev, "PCI probing failed\n"); | 1116 | dev_err(&pdev->dev, "PCI probing failed\n"); |
| @@ -1069,6 +1124,13 @@ static int enetc_pf_probe(struct pci_dev *pdev, · 7 unmodified lines | |||
| 1069 | goto err_map_pf_space; | 1124 | goto err_map_pf_space; |
| 1070 | } | 1125 | } |
| 1071 | 1126 | ||
| 1127 | if (node && !of_device_is_available(node)) { | ||
| 1128 | enetc_init_unused_port(si); | ||
| 1129 | dev_info(&pdev->dev, "device is disabled, skipping\n"); | ||
| 1130 | err = -ENODEV; | ||
| 1131 | goto err_device_disabled; | ||
| 1132 | } | ||
| 1133 | |||
| 1072 | pf = enetc_si_priv(si); | 1134 | pf = enetc_si_priv(si); |
| 1073 | pf->si = si; | 1135 | pf->si = si; |
| 1074 | pf->total_vfs = pci_sriov_get_totalvfs(pdev); | 1136 | pf->total_vfs = pci_sriov_get_totalvfs(pdev); |
| @@ -1108,6 +1170,12 @@ static int enetc_pf_probe(struct pci_dev *pdev, · 33 unmodified lines | |||
| 1108 | goto err_init_port_rss; | 1170 | goto err_init_port_rss; |
| 1109 | } | 1171 | } |
| 1110 | 1172 | ||
| 1173 | err = enetc_configure_si(priv); | ||
| 1174 | if (err) { | ||
| 1175 | dev_err(&pdev->dev, "Failed to configure SI\n"); | ||
| 1176 | goto err_config_si; | ||
| 1177 | } | ||
| 1178 | |||
| 1111 | err = enetc_alloc_msix(priv); | 1179 | err = enetc_alloc_msix(priv); |
| 1112 | if (err) { | 1180 | if (err) { |
| 1113 | dev_err(&pdev->dev, "MSIX alloc failed\n"); | 1181 | dev_err(&pdev->dev, "MSIX alloc failed\n"); |
| @@ -1136,6 +1204,7 @@ err_phylink_create: · 22 unmodified lines | |||
| 1136 | enetc_mdiobus_destroy(pf); | 1204 | enetc_mdiobus_destroy(pf); |
| 1137 | err_mdiobus_create: | 1205 | err_mdiobus_create: |
| 1138 | enetc_free_msix(priv); | 1206 | enetc_free_msix(priv); |
| 1207 | err_config_si: | ||
| 1139 | err_init_port_rss: | 1208 | err_init_port_rss: |
| 1140 | err_init_port_rfs: | 1209 | err_init_port_rfs: |
| 1141 | err_alloc_msix: | 1210 | err_alloc_msix: |
| @@ -1144,6 +1213,7 @@ err_alloc_si_res: · 2 unmodified lines | |||
| 1144 | si->ndev = NULL; | 1213 | si->ndev = NULL; |
| 1145 | free_netdev(ndev); | 1214 | free_netdev(ndev); |
| 1146 | err_alloc_netdev: | 1215 | err_alloc_netdev: |
| 1216 | err_device_disabled: | ||
| 1147 | err_map_pf_space: | 1217 | err_map_pf_space: |
| 1148 | enetc_pci_remove(pdev); | 1218 | enetc_pci_remove(pdev); |
| 1149 | 1219 | ||
| @@ -171,6 +171,12 @@ static int enetc_vf_probe(struct pci_dev *pdev, · 170 unmodified lines | |||
| 171 | goto err_alloc_si_res; | 171 | goto err_alloc_si_res; |
| 172 | } | 172 | } |
| 173 | 173 | ||
| 174 | err = enetc_configure_si(priv); | ||
| 175 | if (err) { | ||
| 176 | dev_err(&pdev->dev, "Failed to configure SI\n"); | ||
| 177 | goto err_config_si; | ||
| 178 | } | ||
| 179 | |||
| 174 | err = enetc_alloc_msix(priv); | 180 | err = enetc_alloc_msix(priv); |
| 175 | if (err) { | 181 | if (err) { |
| 176 | dev_err(&pdev->dev, "MSIX alloc failed\n"); | 182 | dev_err(&pdev->dev, "MSIX alloc failed\n"); |
| @@ -187,6 +193,7 @@ static int enetc_vf_probe(struct pci_dev *pdev, · 10 unmodified lines | |||
| 187 | 193 | ||
| 188 | err_reg_netdev: | 194 | err_reg_netdev: |
| 189 | enetc_free_msix(priv); | 195 | enetc_free_msix(priv); |
| 196 | err_config_si: | ||
| 190 | err_alloc_msix: | 197 | err_alloc_msix: |
| 191 | enetc_free_si_resources(priv); | 198 | enetc_free_si_resources(priv); |
| 192 | err_alloc_si_res: | 199 | err_alloc_si_res: |
| @@ -377,9 +377,16 @@ static int fec_ptp_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts) · 376 unmodified lines | |||
| 377 | u64 ns; | 377 | u64 ns; |
| 378 | unsigned long flags; | 378 | unsigned long flags; |
| 379 | 379 | ||
| 380 | mutex_lock(&adapter->ptp_clk_mutex); | ||
| 381 | /* Check the ptp clock */ | ||
| 382 | if (!adapter->ptp_clk_on) { | ||
| 383 | mutex_unlock(&adapter->ptp_clk_mutex); | ||
| 384 | return -EINVAL; | ||
| 385 | } | ||
| 380 | spin_lock_irqsave(&adapter->tmreg_lock, flags); | 386 | spin_lock_irqsave(&adapter->tmreg_lock, flags); |
| 381 | ns = timecounter_read(&adapter->tc); | 387 | ns = timecounter_read(&adapter->tc); |
| 382 | spin_unlock_irqrestore(&adapter->tmreg_lock, flags); | 388 | spin_unlock_irqrestore(&adapter->tmreg_lock, flags); |
| 389 | mutex_unlock(&adapter->ptp_clk_mutex); | ||
| 383 | 390 | ||
| 384 | *ts = ns_to_timespec64(ns); | 391 | *ts = ns_to_timespec64(ns); |
| 385 | 392 | ||
| @@ -2390,6 +2390,10 @@ static bool gfar_add_rx_frag(struct gfar_rx_buff *rxb, u32 lstatus, · 2389 unmodified lines | |||
| 2390 | if (lstatus & BD_LFLAG(RXBD_LAST)) | 2390 | if (lstatus & BD_LFLAG(RXBD_LAST)) |
| 2391 | size -= skb->len; | 2391 | size -= skb->len; |
| 2392 | 2392 | ||
| 2393 | WARN(size < 0, "gianfar: rx fragment size underflow"); | ||
| 2394 | if (size < 0) | ||
| 2395 | return false; | ||
| 2396 | |||
| 2393 | skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, | 2397 | skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, |
| 2394 | rxb->page_offset + RXBUF_ALIGNMENT, | 2398 | rxb->page_offset + RXBUF_ALIGNMENT, |
| 2395 | size, GFAR_RXB_TRUESIZE); | 2399 | size, GFAR_RXB_TRUESIZE); |
| @@ -2552,6 +2556,17 @@ static int gfar_clean_rx_ring(struct gfar_priv_rx_q *rx_queue, · 156 unmodified lines | |||
| 2552 | if (lstatus & BD_LFLAG(RXBD_EMPTY)) | 2556 | if (lstatus & BD_LFLAG(RXBD_EMPTY)) |
| 2553 | break; | 2557 | break; |
| 2554 | 2558 | ||
| 2559 | /* lost RXBD_LAST descriptor due to overrun */ | ||
| 2560 | if (skb && | ||
| 2561 | (lstatus & BD_LFLAG(RXBD_FIRST))) { | ||
| 2562 | /* discard faulty buffer */ | ||
| 2563 | dev_kfree_skb(skb); | ||
| 2564 | skb = NULL; | ||
| 2565 | rx_queue->stats.rx_dropped++; | ||
| 2566 | |||
| 2567 | /* can continue normally */ | ||
| 2568 | } | ||
| 2569 | |||
| 2555 | /* order rx buffer descriptor reads */ | 2570 | /* order rx buffer descriptor reads */ |
| 2556 | rmb(); | 2571 | rmb(); |
| 2557 | 2572 | ||
| @@ -1663,8 +1663,10 @@ static int hns_nic_clear_all_rx_fetch(struct net_device *ndev) · 1662 unmodified lines | |||
| 1663 | for (j = 0; j < fetch_num; j++) { | 1663 | for (j = 0; j < fetch_num; j++) { |
| 1664 | /* alloc one skb and init */ | 1664 | /* alloc one skb and init */ |
| 1665 | skb = hns_assemble_skb(ndev); | 1665 | skb = hns_assemble_skb(ndev); |
| 1666 | if (!skb) | 1666 | if (!skb) { |
| 1667 | ret = -ENOMEM; | ||
| 1667 | goto out; | 1668 | goto out; |
| 1669 | } | ||
| 1668 | rd = &tx_ring_data(priv, skb->queue_mapping); | 1670 | rd = &tx_ring_data(priv, skb->queue_mapping); |
| 1669 | hns_nic_net_xmit_hw(ndev, skb, rd); | 1671 | hns_nic_net_xmit_hw(ndev, skb, rd); |
| 1670 | 1672 | ||
| @@ -1053,16 +1053,16 @@ struct hclge_fd_tcam_config_3_cmd { · 1052 unmodified lines | |||
| 1053 | #define HCLGE_FD_AD_DROP_B 0 | 1053 | #define HCLGE_FD_AD_DROP_B 0 |
| 1054 | #define HCLGE_FD_AD_DIRECT_QID_B 1 | 1054 | #define HCLGE_FD_AD_DIRECT_QID_B 1 |
| 1055 | #define HCLGE_FD_AD_QID_S 2 | 1055 | #define HCLGE_FD_AD_QID_S 2 |
| 1056 | #define HCLGE_FD_AD_QID_M GENMASK(12, 2) | 1056 | #define HCLGE_FD_AD_QID_M GENMASK(11, 2) |
| 1057 | #define HCLGE_FD_AD_USE_COUNTER_B 12 | 1057 | #define HCLGE_FD_AD_USE_COUNTER_B 12 |
| 1058 | #define HCLGE_FD_AD_COUNTER_NUM_S 13 | 1058 | #define HCLGE_FD_AD_COUNTER_NUM_S 13 |
| 1059 | #define HCLGE_FD_AD_COUNTER_NUM_M GENMASK(20, 13) | 1059 | #define HCLGE_FD_AD_COUNTER_NUM_M GENMASK(20, 13) |
| 1060 | #define HCLGE_FD_AD_NXT_STEP_B 20 | 1060 | #define HCLGE_FD_AD_NXT_STEP_B 20 |
| 1061 | #define HCLGE_FD_AD_NXT_KEY_S 21 | 1061 | #define HCLGE_FD_AD_NXT_KEY_S 21 |
| 1062 | #define HCLGE_FD_AD_NXT_KEY_M GENMASK(26, 21) | 1062 | #define HCLGE_FD_AD_NXT_KEY_M GENMASK(25, 21) |
| 1063 | #define HCLGE_FD_AD_WR_RULE_ID_B 0 | 1063 | #define HCLGE_FD_AD_WR_RULE_ID_B 0 |
| 1064 | #define HCLGE_FD_AD_RULE_ID_S 1 | 1064 | #define HCLGE_FD_AD_RULE_ID_S 1 |
| 1065 | #define HCLGE_FD_AD_RULE_ID_M GENMASK(13, 1) | 1065 | #define HCLGE_FD_AD_RULE_ID_M GENMASK(12, 1) |
| 1066 | #define HCLGE_FD_AD_TC_OVRD_B 16 | 1066 | #define HCLGE_FD_AD_TC_OVRD_B 16 |
| 1067 | #define HCLGE_FD_AD_TC_SIZE_S 17 | 1067 | #define HCLGE_FD_AD_TC_SIZE_S 17 |
| 1068 | #define HCLGE_FD_AD_TC_SIZE_M GENMASK(20, 17) | 1068 | #define HCLGE_FD_AD_TC_SIZE_M GENMASK(20, 17) |
| @@ -5245,9 +5245,9 @@ static bool hclge_fd_convert_tuple(u32 tuple_bit, u8 *key_x, u8 *key_y, · 5244 unmodified lines | |||
| 5245 | case BIT(INNER_SRC_MAC): | 5245 | case BIT(INNER_SRC_MAC): |
| 5246 | for (i = 0; i < ETH_ALEN; i++) { | 5246 | for (i = 0; i < ETH_ALEN; i++) { |
| 5247 | calc_x(key_x[ETH_ALEN - 1 - i], rule->tuples.src_mac[i], | 5247 | calc_x(key_x[ETH_ALEN - 1 - i], rule->tuples.src_mac[i], |
| 5248 | rule->tuples.src_mac[i]); | 5248 | rule->tuples_mask.src_mac[i]); |
| 5249 | calc_y(key_y[ETH_ALEN - 1 - i], rule->tuples.src_mac[i], | 5249 | calc_y(key_y[ETH_ALEN - 1 - i], rule->tuples.src_mac[i], |
| 5250 | rule->tuples.src_mac[i]); | 5250 | rule->tuples_mask.src_mac[i]); |
| 5251 | } | 5251 | } |
| 5252 | 5252 | ||
| 5253 | return true; | 5253 | return true; |
| @@ -6330,8 +6330,7 @@ static void hclge_fd_get_ext_info(struct ethtool_rx_flow_spec *fs, · 1076 unmodified lines | |||
| 6330 | fs->h_ext.vlan_tci = cpu_to_be16(rule->tuples.vlan_tag1); | 6330 | fs->h_ext.vlan_tci = cpu_to_be16(rule->tuples.vlan_tag1); |
| 6331 | fs->m_ext.vlan_tci = | 6331 | fs->m_ext.vlan_tci = |
| 6332 | rule->unused_tuple & BIT(INNER_VLAN_TAG_FST) ? | 6332 | rule->unused_tuple & BIT(INNER_VLAN_TAG_FST) ? |
| 6333 | cpu_to_be16(VLAN_VID_MASK) : | 6333 | 0 : cpu_to_be16(rule->tuples_mask.vlan_tag1); |
| 6334 | cpu_to_be16(rule->tuples_mask.vlan_tag1); | ||
| 6335 | } | 6334 | } |
| 6336 | 6335 | ||
| 6337 | if (fs->flow_type & FLOW_MAC_EXT) { | 6336 | if (fs->flow_type & FLOW_MAC_EXT) { |
| @@ -1758,7 +1758,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · 1757 unmodified lines | |||
| 1758 | return 0; | 1758 | return 0; |
| 1759 | } | 1759 | } |
| 1760 | 1760 | ||
| 1761 | static int ibmveth_remove(struct vio_dev *dev) | 1761 | static void ibmveth_remove(struct vio_dev *dev) |
| 1762 | { | 1762 | { |
| 1763 | struct net_device *netdev = dev_get_drvdata(&dev->dev); | 1763 | struct net_device *netdev = dev_get_drvdata(&dev->dev); |
| 1764 | struct ibmveth_adapter *adapter = netdev_priv(netdev); | 1764 | struct ibmveth_adapter *adapter = netdev_priv(netdev); |
| @@ -1771,8 +1771,6 @@ static int ibmveth_remove(struct vio_dev *dev) · 6 unmodified lines | |||
| 1771 | 1771 | ||
| 1772 | free_netdev(netdev); | 1772 | free_netdev(netdev); |
| 1773 | dev_set_drvdata(&dev->dev, NULL); | 1773 | dev_set_drvdata(&dev->dev, NULL); |
| 1774 | |||
| 1775 | return 0; | ||
| 1776 | } | 1774 | } |
| 1777 | 1775 | ||
| 1778 | static struct attribute veth_active_attr; | 1776 | static struct attribute veth_active_attr; |
| @@ -78,7 +78,6 @@ MODULE_LICENSE("GPL"); · 77 unmodified lines | |||
| 78 | MODULE_VERSION(IBMVNIC_DRIVER_VERSION); | 78 | MODULE_VERSION(IBMVNIC_DRIVER_VERSION); |
| 79 | 79 | ||
| 80 | static int ibmvnic_version = IBMVNIC_INITIAL_VERSION; | 80 | static int ibmvnic_version = IBMVNIC_INITIAL_VERSION; |
| 81 | static int ibmvnic_remove(struct vio_dev *); | ||
| 82 | static void release_sub_crqs(struct ibmvnic_adapter *, bool); | 81 | static void release_sub_crqs(struct ibmvnic_adapter *, bool); |
| 83 | static int ibmvnic_reset_crq(struct ibmvnic_adapter *); | 82 | static int ibmvnic_reset_crq(struct ibmvnic_adapter *); |
| 84 | static int ibmvnic_send_crq_init(struct ibmvnic_adapter *); | 83 | static int ibmvnic_send_crq_init(struct ibmvnic_adapter *); |
| @@ -1906,10 +1905,9 @@ static int ibmvnic_set_mac(struct net_device *netdev, void *p) · 1821 unmodified lines | |||
| 1906 | if (!is_valid_ether_addr(addr->sa_data)) | 1905 | if (!is_valid_ether_addr(addr->sa_data)) |
| 1907 | return -EADDRNOTAVAIL; | 1906 | return -EADDRNOTAVAIL; |
| 1908 | 1907 | ||
| 1909 | if (adapter->state != VNIC_PROBED) { | 1908 | ether_addr_copy(adapter->mac_addr, addr->sa_data); |
| 1910 | ether_addr_copy(adapter->mac_addr, addr->sa_data); | 1909 | if (adapter->state != VNIC_PROBED) |
| 1911 | rc = __ibmvnic_set_mac(netdev, addr->sa_data); | 1910 | rc = __ibmvnic_set_mac(netdev, addr->sa_data); |
| 1912 | } | ||
| 1913 | 1911 | ||
| 1914 | return rc; | 1912 | return rc; |
| 1915 | } | 1913 | } |
| @@ -5219,16 +5217,14 @@ static int ibmvnic_reset_init(struct ibmvnic_adapter *adapter, bool reset) · 3303 unmodified lines | |||
| 5219 | { | 5217 | { |
| 5220 | struct device *dev = &adapter->vdev->dev; | 5218 | struct device *dev = &adapter->vdev->dev; |
| 5221 | unsigned long timeout = msecs_to_jiffies(20000); | 5219 | unsigned long timeout = msecs_to_jiffies(20000); |
| 5222 | u64 old_num_rx_queues, old_num_tx_queues; | 5220 | u64 old_num_rx_queues = adapter->req_rx_queues; |
| 5221 | u64 old_num_tx_queues = adapter->req_tx_queues; | ||
| 5223 | int rc; | 5222 | int rc; |
| 5224 | 5223 | ||
| 5225 | adapter->from_passive_init = false; | 5224 | adapter->from_passive_init = false; |
| 5226 | 5225 | ||
| 5227 | if (reset) { | 5226 | if (reset) |
| 5228 | old_num_rx_queues = adapter->req_rx_queues; | ||
| 5229 | old_num_tx_queues = adapter->req_tx_queues; | ||
| 5230 | reinit_completion(&adapter->init_done); | 5227 | reinit_completion(&adapter->init_done); |
| 5231 | } | ||
| 5232 | 5228 | ||
| 5233 | adapter->init_done_rc = 0; | 5229 | adapter->init_done_rc = 0; |
| 5234 | rc = ibmvnic_send_crq_init(adapter); | 5230 | rc = ibmvnic_send_crq_init(adapter); |
| @@ -5396,7 +5392,7 @@ ibmvnic_init_fail: · 161 unmodified lines | |||
| 5396 | return rc; | 5392 | return rc; |
| 5397 | } | 5393 | } |
| 5398 | 5394 | ||
| 5399 | static int ibmvnic_remove(struct vio_dev *dev) | 5395 | static void ibmvnic_remove(struct vio_dev *dev) |
| 5400 | { | 5396 | { |
| 5401 | struct net_device *netdev = dev_get_drvdata(&dev->dev); | 5397 | struct net_device *netdev = dev_get_drvdata(&dev->dev); |
| 5402 | struct ibmvnic_adapter *adapter = netdev_priv(netdev); | 5398 | struct ibmvnic_adapter *adapter = netdev_priv(netdev); |
| @@ -5411,9 +5407,9 @@ static int ibmvnic_remove(struct vio_dev *dev) · 8 unmodified lines | |||
| 5411 | * after setting state, so __ibmvnic_reset() which is called | 5407 | * after setting state, so __ibmvnic_reset() which is called |
| 5412 | * from the flush_work() below, can make progress. | 5408 | * from the flush_work() below, can make progress. |
| 5413 | */ | 5409 | */ |
| 5414 | spin_lock_irqsave(&adapter->rwi_lock, flags); | 5410 | spin_lock(&adapter->rwi_lock); |
| 5415 | adapter->state = VNIC_REMOVING; | 5411 | adapter->state = VNIC_REMOVING; |
| 5416 | spin_unlock_irqrestore(&adapter->rwi_lock, flags); | 5412 | spin_unlock(&adapter->rwi_lock); |
| 5417 | 5413 | ||
| 5418 | spin_unlock_irqrestore(&adapter->state_lock, flags); | 5414 | spin_unlock_irqrestore(&adapter->state_lock, flags); |
| 5419 | 5415 | ||
| @@ -5437,8 +5433,6 @@ static int ibmvnic_remove(struct vio_dev *dev) · 17 unmodified lines | |||
| 5437 | device_remove_file(&dev->dev, &dev_attr_failover); | 5433 | device_remove_file(&dev->dev, &dev_attr_failover); |
| 5438 | free_netdev(netdev); | 5434 | free_netdev(netdev); |
| 5439 | dev_set_drvdata(&dev->dev, NULL); | 5435 | dev_set_drvdata(&dev->dev, NULL); |
| 5440 | |||
| 5441 | return 0; | ||
| 5442 | } | 5436 | } |
| 5443 | 5437 | ||
| 5444 | static ssize_t failover_store(struct device *dev, struct device_attribute *attr, | 5438 | static ssize_t failover_store(struct device *dev, struct device_attribute *attr, |
| @@ -1776,7 +1776,8 @@ static int iavf_init_get_resources(struct iavf_adapter *adapter) · 1775 unmodified lines | |||
| 1776 | goto err_alloc; | 1776 | goto err_alloc; |
| 1777 | } | 1777 | } |
| 1778 | 1778 | ||
| 1779 | if (iavf_process_config(adapter)) | 1779 | err = iavf_process_config(adapter); |
| 1780 | if (err) | ||
| 1780 | goto err_alloc; | 1781 | goto err_alloc; |
| 1781 | adapter->current_op = VIRTCHNL_OP_UNKNOWN; | 1782 | adapter->current_op = VIRTCHNL_OP_UNKNOWN; |
| 1782 | 1783 | ||
| @@ -575,6 +575,11 @@ static int ixgbe_ipsec_add_sa(struct xfrm_state *xs) · 574 unmodified lines | |||
| 575 | return -EINVAL; | 575 | return -EINVAL; |
| 576 | } | 576 | } |
| 577 | 577 | ||
| 578 | if (xs->props.mode != XFRM_MODE_TRANSPORT) { | ||
| 579 | netdev_err(dev, "Unsupported mode for ipsec offload\n"); | ||
| 580 | return -EINVAL; | ||
| 581 | } | ||
| 582 | |||
| 578 | if (ixgbe_ipsec_check_mgmt_ip(xs)) { | 583 | if (ixgbe_ipsec_check_mgmt_ip(xs)) { |
| 579 | netdev_err(dev, "IPsec IP addr clash with mgmt filters\n"); | 584 | netdev_err(dev, "IPsec IP addr clash with mgmt filters\n"); |
| 580 | return -EINVAL; | 585 | return -EINVAL; |
| @@ -9565,8 +9565,10 @@ static int ixgbe_configure_clsu32(struct ixgbe_adapter *adapter, · 9564 unmodified lines | |||
| 9565 | ixgbe_atr_compute_perfect_hash_82599(&input->filter, mask); | 9565 | ixgbe_atr_compute_perfect_hash_82599(&input->filter, mask); |
| 9566 | err = ixgbe_fdir_write_perfect_filter_82599(hw, &input->filter, | 9566 | err = ixgbe_fdir_write_perfect_filter_82599(hw, &input->filter, |
| 9567 | input->sw_idx, queue); | 9567 | input->sw_idx, queue); |
| 9568 | if (!err) | 9568 | if (err) |
| 9569 | ixgbe_update_ethtool_fdir_entry(adapter, input, input->sw_idx); | 9569 | goto err_out_w_lock; |
| 9570 | |||
| 9571 | ixgbe_update_ethtool_fdir_entry(adapter, input, input->sw_idx); | ||
| 9570 | spin_unlock(&adapter->fdir_perfect_lock); | 9572 | spin_unlock(&adapter->fdir_perfect_lock); |
| 9571 | 9573 | ||
| 9572 | if ((uhtid != 0x800) && (adapter->jump_tables[uhtid])) | 9574 | if ((uhtid != 0x800) && (adapter->jump_tables[uhtid])) |
| @@ -272,6 +272,11 @@ static int ixgbevf_ipsec_add_sa(struct xfrm_state *xs) · 271 unmodified lines | |||
| 272 | return -EINVAL; | 272 | return -EINVAL; |
| 273 | } | 273 | } |
| 274 | 274 | ||
| 275 | if (xs->props.mode != XFRM_MODE_TRANSPORT) { | ||
| 276 | netdev_err(dev, "Unsupported mode for ipsec offload\n"); | ||
| 277 | return -EINVAL; | ||
| 278 | } | ||
| 279 | |||
| 275 | if (xs->xso.flags & XFRM_OFFLOAD_INBOUND) { | 280 | if (xs->xso.flags & XFRM_OFFLOAD_INBOUND) { |
| 276 | struct rx_sa rsa; | 281 | struct rx_sa rsa; |
| 277 | 282 | ||
| @@ -56,7 +56,9 @@ static bool is_dev_rpm(void *cgxd) · 55 unmodified lines | |||
| 56 | 56 | ||
| 57 | bool is_lmac_valid(struct cgx *cgx, int lmac_id) | 57 | bool is_lmac_valid(struct cgx *cgx, int lmac_id) |
| 58 | { | 58 | { |
| 59 | return cgx && test_bit(lmac_id, &cgx->lmac_bmap); | 59 | if (!cgx || lmac_id < 0 || lmac_id >= MAX_LMAC_PER_CGX) |
| 60 | return false; | ||
| 61 | return test_bit(lmac_id, &cgx->lmac_bmap); | ||
| 60 | } | 62 | } |
| 61 | 63 | ||
| 62 | struct mac_ops *get_mac_ops(void *cgxd) | 64 | struct mac_ops *get_mac_ops(void *cgxd) |
| @@ -1225,8 +1225,6 @@ static int mtk_star_receive_packet(struct mtk_star_priv *priv) · 1224 unmodified lines | |||
| 1225 | goto push_new_skb; | 1225 | goto push_new_skb; |
| 1226 | } | 1226 | } |
| 1227 | 1227 | ||
| 1228 | desc_data.dma_addr = new_dma_addr; | ||
| 1229 | |||
| 1230 | /* We can't fail anymore at this point: it's safe to unmap the skb. */ | 1228 | /* We can't fail anymore at this point: it's safe to unmap the skb. */ |
| 1231 | mtk_star_dma_unmap_rx(priv, &desc_data); | 1229 | mtk_star_dma_unmap_rx(priv, &desc_data); |
| 1232 | 1230 | ||
| @@ -1236,6 +1234,9 @@ static int mtk_star_receive_packet(struct mtk_star_priv *priv) · 3 unmodified lines | |||
| 1236 | desc_data.skb->dev = ndev; | 1234 | desc_data.skb->dev = ndev; |
| 1237 | netif_receive_skb(desc_data.skb); | 1235 | netif_receive_skb(desc_data.skb); |
| 1238 | 1236 | ||
| 1237 | /* update dma_addr for new skb */ | ||
| 1238 | desc_data.dma_addr = new_dma_addr; | ||
| 1239 | |||
| 1239 | push_new_skb: | 1240 | push_new_skb: |
| 1240 | desc_data.len = skb_tailroom(new_skb); | 1241 | desc_data.len = skb_tailroom(new_skb); |
| 1241 | desc_data.skb = new_skb; | 1242 | desc_data.skb = new_skb; |
| @@ -47,7 +47,7 @@ · 46 unmodified lines | |||
| 47 | #define EN_ETHTOOL_SHORT_MASK cpu_to_be16(0xffff) | 47 | #define EN_ETHTOOL_SHORT_MASK cpu_to_be16(0xffff) |
| 48 | #define EN_ETHTOOL_WORD_MASK cpu_to_be32(0xffffffff) | 48 | #define EN_ETHTOOL_WORD_MASK cpu_to_be32(0xffffffff) |
| 49 | 49 | ||
| 50 | static int mlx4_en_moderation_update(struct mlx4_en_priv *priv) | 50 | int mlx4_en_moderation_update(struct mlx4_en_priv *priv) |
| 51 | { | 51 | { |
| 52 | int i, t; | 52 | int i, t; |
| 53 | int err = 0; | 53 | int err = 0; |
| @@ -3554,6 +3554,8 @@ int mlx4_en_reset_config(struct net_device *dev, · 3553 unmodified lines | |||
| 3554 | en_err(priv, "Failed starting port\n"); | 3554 | en_err(priv, "Failed starting port\n"); |
| 3555 | } | 3555 | } |
| 3556 | 3556 | ||
| 3557 | if (!err) | ||
| 3558 | err = mlx4_en_moderation_update(priv); | ||
| 3557 | out: | 3559 | out: |
| 3558 | mutex_unlock(&mdev->state_lock); | 3560 | mutex_unlock(&mdev->state_lock); |
| 3559 | kfree(tmp); | 3561 | kfree(tmp); |
| @@ -775,6 +775,7 @@ void mlx4_en_ptp_overflow_check(struct mlx4_en_dev *mdev); · 774 unmodified lines | |||
| 775 | #define DEV_FEATURE_CHANGED(dev, new_features, feature) \ | 775 | #define DEV_FEATURE_CHANGED(dev, new_features, feature) \ |
| 776 | ((dev->features & feature) ^ (new_features & feature)) | 776 | ((dev->features & feature) ^ (new_features & feature)) |
| 777 | 777 | ||
| 778 | int mlx4_en_moderation_update(struct mlx4_en_priv *priv); | ||
| 778 | int mlx4_en_reset_config(struct net_device *dev, | 779 | int mlx4_en_reset_config(struct net_device *dev, |
| 779 | struct hwtstamp_config ts_config, | 780 | struct hwtstamp_config ts_config, |
| 780 | netdev_features_t new_features); | 781 | netdev_features_t new_features); |
| @@ -4430,6 +4430,7 @@ MLXSW_ITEM32(reg, ptys, ext_eth_proto_cap, 0x08, 0, 32); · 4429 unmodified lines | |||
| 4430 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4 BIT(20) | 4430 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4 BIT(20) |
| 4431 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_SR4 BIT(21) | 4431 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_SR4 BIT(21) |
| 4432 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_KR4 BIT(22) | 4432 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_KR4 BIT(22) |
| 4433 | #define MLXSW_REG_PTYS_ETH_SPEED_100GBASE_LR4_ER4 BIT(23) | ||
| 4433 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_CR BIT(27) | 4434 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_CR BIT(27) |
| 4434 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_KR BIT(28) | 4435 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_KR BIT(28) |
| 4435 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_SR BIT(29) | 4436 | #define MLXSW_REG_PTYS_ETH_SPEED_25GBASE_SR BIT(29) |
| @@ -1169,6 +1169,11 @@ static const struct mlxsw_sp1_port_link_mode mlxsw_sp1_port_link_mode[] = { · 1168 unmodified lines | |||
| 1169 | .mask_ethtool = ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT, | 1169 | .mask_ethtool = ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT, |
| 1170 | .speed = SPEED_100000, | 1170 | .speed = SPEED_100000, |
| 1171 | }, | 1171 | }, |
| 1172 | { | ||
| 1173 | .mask = MLXSW_REG_PTYS_ETH_SPEED_100GBASE_LR4_ER4, | ||
| 1174 | .mask_ethtool = ETHTOOL_LINK_MODE_100000baseLR4_ER4_Full_BIT, | ||
| 1175 | .speed = SPEED_100000, | ||
| 1176 | }, | ||
| 1172 | }; | 1177 | }; |
| 1173 | 1178 | ||
| 1174 | #define MLXSW_SP1_PORT_LINK_MODE_LEN ARRAY_SIZE(mlxsw_sp1_port_link_mode) | 1179 | #define MLXSW_SP1_PORT_LINK_MODE_LEN ARRAY_SIZE(mlxsw_sp1_port_link_mode) |
| @@ -5951,6 +5951,10 @@ mlxsw_sp_router_fib4_replace(struct mlxsw_sp *mlxsw_sp, · 5950 unmodified lines | |||
| 5951 | if (mlxsw_sp->router->aborted) | 5951 | if (mlxsw_sp->router->aborted) |
| 5952 | return 0; | 5952 | return 0; |
| 5953 | 5953 | ||
| 5954 | if (fen_info->fi->nh && | ||
| 5955 | !mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp, fen_info->fi->nh->id)) | ||
| 5956 | return 0; | ||
| 5957 | |||
| 5954 | fib_node = mlxsw_sp_fib_node_get(mlxsw_sp, fen_info->tb_id, | 5958 | fib_node = mlxsw_sp_fib_node_get(mlxsw_sp, fen_info->tb_id, |
| 5955 | &fen_info->dst, sizeof(fen_info->dst), | 5959 | &fen_info->dst, sizeof(fen_info->dst), |
| 5956 | fen_info->dst_len, | 5960 | fen_info->dst_len, |
| @@ -6601,6 +6605,9 @@ static int mlxsw_sp_router_fib6_replace(struct mlxsw_sp *mlxsw_sp, · 644 unmodified lines | |||
| 6601 | if (mlxsw_sp_fib6_rt_should_ignore(rt)) | 6605 | if (mlxsw_sp_fib6_rt_should_ignore(rt)) |
| 6602 | return 0; | 6606 | return 0; |
| 6603 | 6607 | ||
| 6608 | if (rt->nh && !mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp, rt->nh->id)) | ||
| 6609 | return 0; | ||
| 6610 | |||
| 6604 | fib_node = mlxsw_sp_fib_node_get(mlxsw_sp, rt->fib6_table->tb6_id, | 6611 | fib_node = mlxsw_sp_fib_node_get(mlxsw_sp, rt->fib6_table->tb6_id, |
| 6605 | &rt->fib6_dst.addr, | 6612 | &rt->fib6_dst.addr, |
| 6606 | sizeof(rt->fib6_dst.addr), | 6613 | sizeof(rt->fib6_dst.addr), |
| @@ -613,7 +613,8 @@ static const struct mlxsw_sx_port_link_mode mlxsw_sx_port_link_mode[] = { · 612 unmodified lines | |||
| 613 | { | 613 | { |
| 614 | .mask = MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4 | | 614 | .mask = MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4 | |
| 615 | MLXSW_REG_PTYS_ETH_SPEED_100GBASE_SR4 | | 615 | MLXSW_REG_PTYS_ETH_SPEED_100GBASE_SR4 | |
| 616 | MLXSW_REG_PTYS_ETH_SPEED_100GBASE_KR4, | 616 | MLXSW_REG_PTYS_ETH_SPEED_100GBASE_KR4 | |
| 617 | MLXSW_REG_PTYS_ETH_SPEED_100GBASE_LR4_ER4, | ||
| 617 | .speed = 100000, | 618 | .speed = 100000, |
| 618 | }, | 619 | }, |
| 619 | }; | 620 | }; |
| @@ -2040,7 +2040,7 @@ lan743x_rx_trim_skb(struct sk_buff *skb, int frame_length) · 2039 unmodified lines | |||
| 2040 | dev_kfree_skb_irq(skb); | 2040 | dev_kfree_skb_irq(skb); |
| 2041 | return NULL; | 2041 | return NULL; |
| 2042 | } | 2042 | } |
| 2043 | frame_length = max_t(int, 0, frame_length - RX_HEAD_PADDING - 2); | 2043 | frame_length = max_t(int, 0, frame_length - RX_HEAD_PADDING - 4); |
| 2044 | if (skb->len > frame_length) { | 2044 | if (skb->len > frame_length) { |
| 2045 | skb->tail -= skb->len - frame_length; | 2045 | skb->tail -= skb->len - frame_length; |
| 2046 | skb->len = frame_length; | 2046 | skb->len = frame_length; |
| @@ -13,6 +13,7 @@ if NET_VENDOR_MICROSEMI · 12 unmodified lines | |||
| 13 | 13 | ||
| 14 | # Users should depend on NET_SWITCHDEV, HAS_IOMEM | 14 | # Users should depend on NET_SWITCHDEV, HAS_IOMEM |
| 15 | config MSCC_OCELOT_SWITCH_LIB | 15 | config MSCC_OCELOT_SWITCH_LIB |
| 16 | select NET_DEVLINK | ||
| 16 | select REGMAP_MMIO | 17 | select REGMAP_MMIO |
| 17 | select PACKING | 18 | select PACKING |
| 18 | select PHYLIB | 19 | select PHYLIB |
| @@ -540,13 +540,14 @@ ocelot_flower_parse_key(struct ocelot *ocelot, int port, bool ingress, · 539 unmodified lines | |||
| 540 | return -EOPNOTSUPP; | 540 | return -EOPNOTSUPP; |
| 541 | } | 541 | } |
| 542 | 542 | ||
| 543 | flow_rule_match_ipv4_addrs(rule, &match); | ||
| 544 | |||
| 543 | if (filter->block_id == VCAP_IS1 && *(u32 *)&match.mask->dst) { | 545 | if (filter->block_id == VCAP_IS1 && *(u32 *)&match.mask->dst) { |
| 544 | NL_SET_ERR_MSG_MOD(extack, | 546 | NL_SET_ERR_MSG_MOD(extack, |
| 545 | "Key type S1_NORMAL cannot match on destination IP"); | 547 | "Key type S1_NORMAL cannot match on destination IP"); |
| 546 | return -EOPNOTSUPP; | 548 | return -EOPNOTSUPP; |
| 547 | } | 549 | } |
| 548 | 550 | ||
| 549 | flow_rule_match_ipv4_addrs(rule, &match); | ||
| 550 | tmp = &filter->key.ipv4.sip.value.addr[0]; | 551 | tmp = &filter->key.ipv4.sip.value.addr[0]; |
| 551 | memcpy(tmp, &match.key->src, 4); | 552 | memcpy(tmp, &match.key->src, 4); |
| 552 | 553 | ||
| @@ -767,7 +767,7 @@ static void r8168fp_adjust_ocp_cmd(struct rtl8169_private *tp, u32 *cmd, int typ · 766 unmodified lines | |||
| 767 | if (type == ERIAR_OOB && | 767 | if (type == ERIAR_OOB && |
| 768 | (tp->mac_version == RTL_GIGA_MAC_VER_52 || | 768 | (tp->mac_version == RTL_GIGA_MAC_VER_52 || |
| 769 | tp->mac_version == RTL_GIGA_MAC_VER_53)) | 769 | tp->mac_version == RTL_GIGA_MAC_VER_53)) |
| 770 | *cmd |= 0x7f0 << 18; | 770 | *cmd |= 0xf70 << 18; |
| 771 | } | 771 | } |
| 772 | 772 | ||
| 773 | DECLARE_RTL_COND(rtl_eriar_cond) | 773 | DECLARE_RTL_COND(rtl_eriar_cond) |
| @@ -560,6 +560,8 @@ static struct sh_eth_cpu_data r7s72100_data = { · 559 unmodified lines | |||
| 560 | EESR_TDE, | 560 | EESR_TDE, |
| 561 | .fdr_value = 0x0000070f, | 561 | .fdr_value = 0x0000070f, |
| 562 | 562 | ||
| 563 | .trscer_err_mask = DESC_I_RINT8 | DESC_I_RINT5, | ||
| 564 | |||
| 563 | .no_psr = 1, | 565 | .no_psr = 1, |
| 564 | .apr = 1, | 566 | .apr = 1, |
| 565 | .mpr = 1, | 567 | .mpr = 1, |
| @@ -780,6 +782,8 @@ static struct sh_eth_cpu_data r7s9210_data = { · 214 unmodified lines | |||
| 780 | 782 | ||
| 781 | .fdr_value = 0x0000070f, | 783 | .fdr_value = 0x0000070f, |
| 782 | 784 | ||
| 785 | .trscer_err_mask = DESC_I_RINT8 | DESC_I_RINT5, | ||
| 786 | |||
| 783 | .apr = 1, | 787 | .apr = 1, |
| 784 | .mpr = 1, | 788 | .mpr = 1, |
| 785 | .tpauser = 1, | 789 | .tpauser = 1, |
| @@ -1089,6 +1093,9 @@ static struct sh_eth_cpu_data sh771x_data = { · 303 unmodified lines | |||
| 1089 | EESIPR_CEEFIP | EESIPR_CELFIP | | 1093 | EESIPR_CEEFIP | EESIPR_CELFIP | |
| 1090 | EESIPR_RRFIP | EESIPR_RTLFIP | EESIPR_RTSFIP | | 1094 | EESIPR_RRFIP | EESIPR_RTLFIP | EESIPR_RTSFIP | |
| 1091 | EESIPR_PREIP | EESIPR_CERFIP, | 1095 | EESIPR_PREIP | EESIPR_CERFIP, |
| 1096 | |||
| 1097 | .trscer_err_mask = DESC_I_RINT8, | ||
| 1098 | |||
| 1092 | .tsu = 1, | 1099 | .tsu = 1, |
| 1093 | .dual_port = 1, | 1100 | .dual_port = 1, |
| 1094 | }; | 1101 | }; |
| @@ -233,6 +233,7 @@ static void common_default_data(struct plat_stmmacenet_data *plat) · 232 unmodified lines | |||
| 233 | static int intel_mgbe_common_data(struct pci_dev *pdev, | 233 | static int intel_mgbe_common_data(struct pci_dev *pdev, |
| 234 | struct plat_stmmacenet_data *plat) | 234 | struct plat_stmmacenet_data *plat) |
| 235 | { | 235 | { |
| 236 | char clk_name[20]; | ||
| 236 | int ret; | 237 | int ret; |
| 237 | int i; | 238 | int i; |
| 238 | 239 | ||
| @@ -301,8 +302,10 @@ static int intel_mgbe_common_data(struct pci_dev *pdev, · 62 unmodified lines | |||
| 301 | plat->eee_usecs_rate = plat->clk_ptp_rate; | 302 | plat->eee_usecs_rate = plat->clk_ptp_rate; |
| 302 | 303 | ||
| 303 | /* Set system clock */ | 304 | /* Set system clock */ |
| 305 | sprintf(clk_name, "%s-%s", "stmmac", pci_name(pdev)); | ||
| 306 | |||
| 304 | plat->stmmac_clk = clk_register_fixed_rate(&pdev->dev, | 307 | plat->stmmac_clk = clk_register_fixed_rate(&pdev->dev, |
| 305 | "stmmac-clk", NULL, 0, | 308 | clk_name, NULL, 0, |
| 306 | plat->clk_ptp_rate); | 309 | plat->clk_ptp_rate); |
| 307 | 310 | ||
| 308 | if (IS_ERR(plat->stmmac_clk)) { | 311 | if (IS_ERR(plat->stmmac_clk)) { |
| @@ -446,8 +449,8 @@ static int tgl_common_data(struct pci_dev *pdev, · 137 unmodified lines | |||
| 446 | return intel_mgbe_common_data(pdev, plat); | 449 | return intel_mgbe_common_data(pdev, plat); |
| 447 | } | 450 | } |
| 448 | 451 | ||
| 449 | static int tgl_sgmii_data(struct pci_dev *pdev, | 452 | static int tgl_sgmii_phy0_data(struct pci_dev *pdev, |
| 450 | struct plat_stmmacenet_data *plat) | 453 | struct plat_stmmacenet_data *plat) |
| 451 | { | 454 | { |
| 452 | plat->bus_id = 1; | 455 | plat->bus_id = 1; |
| 453 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; | 456 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; |
| @@ -456,12 +459,26 @@ static int tgl_sgmii_data(struct pci_dev *pdev, · 2 unmodified lines | |||
| 456 | return tgl_common_data(pdev, plat); | 459 | return tgl_common_data(pdev, plat); |
| 457 | } | 460 | } |
| 458 | 461 | ||
| 459 | static struct stmmac_pci_info tgl_sgmii1g_info = { | 462 | static struct stmmac_pci_info tgl_sgmii1g_phy0_info = { |
| 460 | .setup = tgl_sgmii_data, | 463 | .setup = tgl_sgmii_phy0_data, |
| 461 | }; | 464 | }; |
| 462 | 465 | ||
| 463 | static int adls_sgmii_data(struct pci_dev *pdev, | 466 | static int tgl_sgmii_phy1_data(struct pci_dev *pdev, |
| 464 | struct plat_stmmacenet_data *plat) | 467 | struct plat_stmmacenet_data *plat) |
| 468 | { | ||
| 469 | plat->bus_id = 2; | ||
| 470 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; | ||
| 471 | plat->serdes_powerup = intel_serdes_powerup; | ||
| 472 | plat->serdes_powerdown = intel_serdes_powerdown; | ||
| 473 | return tgl_common_data(pdev, plat); | ||
| 474 | } | ||
| 475 | |||
| 476 | static struct stmmac_pci_info tgl_sgmii1g_phy1_info = { | ||
| 477 | .setup = tgl_sgmii_phy1_data, | ||
| 478 | }; | ||
| 479 | |||
| 480 | static int adls_sgmii_phy0_data(struct pci_dev *pdev, | ||
| 481 | struct plat_stmmacenet_data *plat) | ||
| 465 | { | 482 | { |
| 466 | plat->bus_id = 1; | 483 | plat->bus_id = 1; |
| 467 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; | 484 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; |
| @@ -471,10 +488,24 @@ static int adls_sgmii_data(struct pci_dev *pdev, · 3 unmodified lines | |||
| 471 | return tgl_common_data(pdev, plat); | 488 | return tgl_common_data(pdev, plat); |
| 472 | } | 489 | } |
| 473 | 490 | ||
| 474 | static struct stmmac_pci_info adls_sgmii1g_info = { | 491 | static struct stmmac_pci_info adls_sgmii1g_phy0_info = { |
| 475 | .setup = adls_sgmii_data, | 492 | .setup = adls_sgmii_phy0_data, |
| 476 | }; | 493 | }; |
| 477 | 494 | ||
| 495 | static int adls_sgmii_phy1_data(struct pci_dev *pdev, | ||
| 496 | struct plat_stmmacenet_data *plat) | ||
| 497 | { | ||
| 498 | plat->bus_id = 2; | ||
| 499 | plat->phy_interface = PHY_INTERFACE_MODE_SGMII; | ||
| 500 | |||
| 501 | /* SerDes power up and power down are done in BIOS for ADL */ | ||
| 502 | |||
| 503 | return tgl_common_data(pdev, plat); | ||
| 504 | } | ||
| 505 | |||
| 506 | static struct stmmac_pci_info adls_sgmii1g_phy1_info = { | ||
| 507 | .setup = adls_sgmii_phy1_data, | ||
| 508 | }; | ||
| 478 | static const struct stmmac_pci_func_data galileo_stmmac_func_data[] = { | 509 | static const struct stmmac_pci_func_data galileo_stmmac_func_data[] = { |
| 479 | { | 510 | { |
| 480 | .func = 6, | 511 | .func = 6, |
| @@ -756,11 +787,11 @@ static const struct pci_device_id intel_eth_pci_id_table[] = { · 275 unmodified lines | |||
| 756 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_RGMII1G_ID, &ehl_pse1_rgmii1g_info) }, | 787 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_RGMII1G_ID, &ehl_pse1_rgmii1g_info) }, |
| 757 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_SGMII1G_ID, &ehl_pse1_sgmii1g_info) }, | 788 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_SGMII1G_ID, &ehl_pse1_sgmii1g_info) }, |
| 758 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_SGMII2G5_ID, &ehl_pse1_sgmii1g_info) }, | 789 | { PCI_DEVICE_DATA(INTEL, EHL_PSE1_SGMII2G5_ID, &ehl_pse1_sgmii1g_info) }, |
| 759 | { PCI_DEVICE_DATA(INTEL, TGL_SGMII1G_ID, &tgl_sgmii1g_info) }, | 790 | { PCI_DEVICE_DATA(INTEL, TGL_SGMII1G_ID, &tgl_sgmii1g_phy0_info) }, |
| 760 | { PCI_DEVICE_DATA(INTEL, TGLH_SGMII1G_0_ID, &tgl_sgmii1g_info) }, | 791 | { PCI_DEVICE_DATA(INTEL, TGLH_SGMII1G_0_ID, &tgl_sgmii1g_phy0_info) }, |
| 761 | { PCI_DEVICE_DATA(INTEL, TGLH_SGMII1G_1_ID, &tgl_sgmii1g_info) }, | 792 | { PCI_DEVICE_DATA(INTEL, TGLH_SGMII1G_1_ID, &tgl_sgmii1g_phy1_info) }, |
| 762 | { PCI_DEVICE_DATA(INTEL, ADLS_SGMII1G_0_ID, &adls_sgmii1g_info) }, | 793 | { PCI_DEVICE_DATA(INTEL, ADLS_SGMII1G_0_ID, &adls_sgmii1g_phy0_info) }, |
| 763 | { PCI_DEVICE_DATA(INTEL, ADLS_SGMII1G_1_ID, &adls_sgmii1g_info) }, | 794 | { PCI_DEVICE_DATA(INTEL, ADLS_SGMII1G_1_ID, &adls_sgmii1g_phy1_info) }, |
| 764 | {} | 795 | {} |
| 765 | }; | 796 | }; |
| 766 | MODULE_DEVICE_TABLE(pci, intel_eth_pci_id_table); | 797 | MODULE_DEVICE_TABLE(pci, intel_eth_pci_id_table); |
| @@ -402,19 +402,53 @@ static void dwmac4_rd_set_tx_ic(struct dma_desc *p) · 401 unmodified lines | |||
| 402 | p->des2 |= cpu_to_le32(TDES2_INTERRUPT_ON_COMPLETION); | 402 | p->des2 |= cpu_to_le32(TDES2_INTERRUPT_ON_COMPLETION); |
| 403 | } | 403 | } |
| 404 | 404 | ||
| 405 | static void dwmac4_display_ring(void *head, unsigned int size, bool rx) | 405 | static void dwmac4_display_ring(void *head, unsigned int size, bool rx, |
| 406 | dma_addr_t dma_rx_phy, unsigned int desc_size) | ||
| 406 | { | 407 | { |
| 407 | struct dma_desc *p = (struct dma_desc *)head; | 408 | dma_addr_t dma_addr; |
| 408 | int i; | 409 | int i; |
| 409 | 410 | ||
| 410 | pr_info("%s descriptor ring:\n", rx ? "RX" : "TX"); | 411 | pr_info("%s descriptor ring:\n", rx ? "RX" : "TX"); |
| 411 | 412 | ||
| 412 | for (i = 0; i < size; i++) { | 413 | if (desc_size == sizeof(struct dma_desc)) { |
| 413 | pr_info("%03d [0x%x]: 0x%x 0x%x 0x%x 0x%x\n", | 414 | struct dma_desc *p = (struct dma_desc *)head; |
| 414 | i, (unsigned int)virt_to_phys(p), | 415 | |
| 415 | le32_to_cpu(p->des0), le32_to_cpu(p->des1), | 416 | for (i = 0; i < size; i++) { |
| 416 | le32_to_cpu(p->des2), le32_to_cpu(p->des3)); | 417 | dma_addr = dma_rx_phy + i * sizeof(*p); |
| 417 | p++; | 418 | pr_info("%03d [%pad]: 0x%x 0x%x 0x%x 0x%x\n", |
| 419 | i, &dma_addr, | ||
| 420 | le32_to_cpu(p->des0), le32_to_cpu(p->des1), | ||
| 421 | le32_to_cpu(p->des2), le32_to_cpu(p->des3)); | ||
| 422 | p++; | ||
| 423 | } | ||
| 424 | } else if (desc_size == sizeof(struct dma_extended_desc)) { | ||
| 425 | struct dma_extended_desc *extp = (struct dma_extended_desc *)head; | ||
| 426 | |||
| 427 | for (i = 0; i < size; i++) { | ||
| 428 | dma_addr = dma_rx_phy + i * sizeof(*extp); | ||
| 429 | pr_info("%03d [%pad]: 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x\n", | ||
| 430 | i, &dma_addr, | ||
| 431 | le32_to_cpu(extp->basic.des0), le32_to_cpu(extp->basic.des1), | ||
| 432 | le32_to_cpu(extp->basic.des2), le32_to_cpu(extp->basic.des3), | ||
| 433 | le32_to_cpu(extp->des4), le32_to_cpu(extp->des5), | ||
| 434 | le32_to_cpu(extp->des6), le32_to_cpu(extp->des7)); | ||
| 435 | extp++; | ||
| 436 | } | ||
| 437 | } else if (desc_size == sizeof(struct dma_edesc)) { | ||
| 438 | struct dma_edesc *ep = (struct dma_edesc *)head; | ||
| 439 | |||
| 440 | for (i = 0; i < size; i++) { | ||
| 441 | dma_addr = dma_rx_phy + i * sizeof(*ep); | ||
| 442 | pr_info("%03d [%pad]: 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x\n", | ||
| 443 | i, &dma_addr, | ||
| 444 | le32_to_cpu(ep->des4), le32_to_cpu(ep->des5), | ||
| 445 | le32_to_cpu(ep->des6), le32_to_cpu(ep->des7), | ||
| 446 | le32_to_cpu(ep->basic.des0), le32_to_cpu(ep->basic.des1), | ||
| 447 | le32_to_cpu(ep->basic.des2), le32_to_cpu(ep->basic.des3)); | ||
| 448 | ep++; | ||
| 449 | } | ||
| 450 | } else { | ||
| 451 | pr_err("unsupported descriptor!"); | ||
| 418 | } | 452 | } |
| 419 | } | 453 | } |
| 420 | 454 | ||
| @@ -499,10 +533,15 @@ static void dwmac4_get_rx_header_len(struct dma_desc *p, unsigned int *len) · 78 unmodified lines | |||
| 499 | *len = le32_to_cpu(p->des2) & RDES2_HL; | 533 | *len = le32_to_cpu(p->des2) & RDES2_HL; |
| 500 | } | 534 | } |
| 501 | 535 | ||
| 502 | static void dwmac4_set_sec_addr(struct dma_desc *p, dma_addr_t addr) | 536 | static void dwmac4_set_sec_addr(struct dma_desc *p, dma_addr_t addr, bool buf2_valid) |
| 503 | { | 537 | { |
| 504 | p->des2 = cpu_to_le32(lower_32_bits(addr)); | 538 | p->des2 = cpu_to_le32(lower_32_bits(addr)); |
| 505 | p->des3 = cpu_to_le32(upper_32_bits(addr) | RDES3_BUFFER2_VALID_ADDR); | 539 | p->des3 = cpu_to_le32(upper_32_bits(addr)); |
| 540 | |||
| 541 | if (buf2_valid) | ||
| 542 | p->des3 |= cpu_to_le32(RDES3_BUFFER2_VALID_ADDR); | ||
| 543 | else | ||
| 544 | p->des3 &= cpu_to_le32(~RDES3_BUFFER2_VALID_ADDR); | ||
| 506 | } | 545 | } |
| 507 | 546 | ||
| 508 | static void dwmac4_set_tbs(struct dma_edesc *p, u32 sec, u32 nsec) | 547 | static void dwmac4_set_tbs(struct dma_edesc *p, u32 sec, u32 nsec) |
| @@ -124,6 +124,23 @@ static void dwmac4_dma_init_channel(void __iomem *ioaddr, · 123 unmodified lines | |||
| 124 | ioaddr + DMA_CHAN_INTR_ENA(chan)); | 124 | ioaddr + DMA_CHAN_INTR_ENA(chan)); |
| 125 | } | 125 | } |
| 126 | 126 | ||
| 127 | static void dwmac410_dma_init_channel(void __iomem *ioaddr, | ||
| 128 | struct stmmac_dma_cfg *dma_cfg, u32 chan) | ||
| 129 | { | ||
| 130 | u32 value; | ||
| 131 | |||
| 132 | /* common channel control register config */ | ||
| 133 | value = readl(ioaddr + DMA_CHAN_CONTROL(chan)); | ||
| 134 | if (dma_cfg->pblx8) | ||
| 135 | value = value | DMA_BUS_MODE_PBL; | ||
| 136 | |||
| 137 | writel(value, ioaddr + DMA_CHAN_CONTROL(chan)); | ||
| 138 | |||
| 139 | /* Mask interrupts by writing to CSR7 */ | ||
| 140 | writel(DMA_CHAN_INTR_DEFAULT_MASK_4_10, | ||
| 141 | ioaddr + DMA_CHAN_INTR_ENA(chan)); | ||
| 142 | } | ||
| 143 | |||
| 127 | static void dwmac4_dma_init(void __iomem *ioaddr, | 144 | static void dwmac4_dma_init(void __iomem *ioaddr, |
| 128 | struct stmmac_dma_cfg *dma_cfg, int atds) | 145 | struct stmmac_dma_cfg *dma_cfg, int atds) |
| 129 | { | 146 | { |
| @@ -523,7 +540,7 @@ const struct stmmac_dma_ops dwmac4_dma_ops = { · 393 unmodified lines | |||
| 523 | const struct stmmac_dma_ops dwmac410_dma_ops = { | 540 | const struct stmmac_dma_ops dwmac410_dma_ops = { |
| 524 | .reset = dwmac4_dma_reset, | 541 | .reset = dwmac4_dma_reset, |
| 525 | .init = dwmac4_dma_init, | 542 | .init = dwmac4_dma_init, |
| 526 | .init_chan = dwmac4_dma_init_channel, | 543 | .init_chan = dwmac410_dma_init_channel, |
| 527 | .init_rx_chan = dwmac4_dma_init_rx_chan, | 544 | .init_rx_chan = dwmac4_dma_init_rx_chan, |
| 528 | .init_tx_chan = dwmac4_dma_init_tx_chan, | 545 | .init_tx_chan = dwmac4_dma_init_tx_chan, |
| 529 | .axi = dwmac4_dma_axi, | 546 | .axi = dwmac4_dma_axi, |
| @@ -53,10 +53,6 @@ void dwmac4_dma_stop_tx(void __iomem *ioaddr, u32 chan) · 52 unmodified lines | |||
| 53 | 53 | ||
| 54 | value &= ~DMA_CONTROL_ST; | 54 | value &= ~DMA_CONTROL_ST; |
| 55 | writel(value, ioaddr + DMA_CHAN_TX_CONTROL(chan)); | 55 | writel(value, ioaddr + DMA_CHAN_TX_CONTROL(chan)); |
| 56 | |||
| 57 | value = readl(ioaddr + GMAC_CONFIG); | ||
| 58 | value &= ~GMAC_CONFIG_TE; | ||
| 59 | writel(value, ioaddr + GMAC_CONFIG); | ||
| 60 | } | 56 | } |
| 61 | 57 | ||
| 62 | void dwmac4_dma_start_rx(void __iomem *ioaddr, u32 chan) | 58 | void dwmac4_dma_start_rx(void __iomem *ioaddr, u32 chan) |
| @@ -292,7 +292,7 @@ static void dwxgmac2_get_rx_header_len(struct dma_desc *p, unsigned int *len) · 291 unmodified lines | |||
| 292 | *len = le32_to_cpu(p->des2) & XGMAC_RDES2_HL; | 292 | *len = le32_to_cpu(p->des2) & XGMAC_RDES2_HL; |
| 293 | } | 293 | } |
| 294 | 294 | ||
| 295 | static void dwxgmac2_set_sec_addr(struct dma_desc *p, dma_addr_t addr) | 295 | static void dwxgmac2_set_sec_addr(struct dma_desc *p, dma_addr_t addr, bool is_valid) |
| 296 | { | 296 | { |
| 297 | p->des2 = cpu_to_le32(lower_32_bits(addr)); | 297 | p->des2 = cpu_to_le32(lower_32_bits(addr)); |
| 298 | p->des3 = cpu_to_le32(upper_32_bits(addr)); | 298 | p->des3 = cpu_to_le32(upper_32_bits(addr)); |
| @@ -417,19 +417,22 @@ static int enh_desc_get_rx_timestamp_status(void *desc, void *next_desc, · 416 unmodified lines | |||
| 417 | } | 417 | } |
| 418 | } | 418 | } |
| 419 | 419 | ||
| 420 | static void enh_desc_display_ring(void *head, unsigned int size, bool rx) | 420 | static void enh_desc_display_ring(void *head, unsigned int size, bool rx, |
| 421 | dma_addr_t dma_rx_phy, unsigned int desc_size) | ||
| 421 | { | 422 | { |
| 422 | struct dma_extended_desc *ep = (struct dma_extended_desc *)head; | 423 | struct dma_extended_desc *ep = (struct dma_extended_desc *)head; |
| 424 | dma_addr_t dma_addr; | ||
| 423 | int i; | 425 | int i; |
| 424 | 426 | ||
| 425 | pr_info("Extended %s descriptor ring:\n", rx ? "RX" : "TX"); | 427 | pr_info("Extended %s descriptor ring:\n", rx ? "RX" : "TX"); |
| 426 | 428 | ||
| 427 | for (i = 0; i < size; i++) { | 429 | for (i = 0; i < size; i++) { |
| 428 | u64 x; | 430 | u64 x; |
| 431 | dma_addr = dma_rx_phy + i * sizeof(*ep); | ||
| 429 | 432 | ||
| 430 | x = *(u64 *)ep; | 433 | x = *(u64 *)ep; |
| 431 | pr_info("%03d [0x%x]: 0x%x 0x%x 0x%x 0x%x\n", | 434 | pr_info("%03d [%pad]: 0x%x 0x%x 0x%x 0x%x\n", |
| 432 | i, (unsigned int)virt_to_phys(ep), | 435 | i, &dma_addr, |
| 433 | (unsigned int)x, (unsigned int)(x >> 32), | 436 | (unsigned int)x, (unsigned int)(x >> 32), |
| 434 | ep->basic.des2, ep->basic.des3); | 437 | ep->basic.des2, ep->basic.des3); |
| 435 | ep++; | 438 | ep++; |
| @@ -78,7 +78,8 @@ struct stmmac_desc_ops { · 77 unmodified lines | |||
| 78 | /* get rx timestamp status */ | 78 | /* get rx timestamp status */ |
| 79 | int (*get_rx_timestamp_status)(void *desc, void *next_desc, u32 ats); | 79 | int (*get_rx_timestamp_status)(void *desc, void *next_desc, u32 ats); |
| 80 | /* Display ring */ | 80 | /* Display ring */ |
| 81 | void (*display_ring)(void *head, unsigned int size, bool rx); | 81 | void (*display_ring)(void *head, unsigned int size, bool rx, |
| 82 | dma_addr_t dma_rx_phy, unsigned int desc_size); | ||
| 82 | /* set MSS via context descriptor */ | 83 | /* set MSS via context descriptor */ |
| 83 | void (*set_mss)(struct dma_desc *p, unsigned int mss); | 84 | void (*set_mss)(struct dma_desc *p, unsigned int mss); |
| 84 | /* get descriptor skbuff address */ | 85 | /* get descriptor skbuff address */ |
| @@ -91,7 +92,7 @@ struct stmmac_desc_ops { · 6 unmodified lines | |||
| 91 | int (*get_rx_hash)(struct dma_desc *p, u32 *hash, | 92 | int (*get_rx_hash)(struct dma_desc *p, u32 *hash, |
| 92 | enum pkt_hash_types *type); | 93 | enum pkt_hash_types *type); |
| 93 | void (*get_rx_header_len)(struct dma_desc *p, unsigned int *len); | 94 | void (*get_rx_header_len)(struct dma_desc *p, unsigned int *len); |
| 94 | void (*set_sec_addr)(struct dma_desc *p, dma_addr_t addr); | 95 | void (*set_sec_addr)(struct dma_desc *p, dma_addr_t addr, bool buf2_valid); |
| 95 | void (*set_sarc)(struct dma_desc *p, u32 sarc_type); | 96 | void (*set_sarc)(struct dma_desc *p, u32 sarc_type); |
| 96 | void (*set_vlan_tag)(struct dma_desc *p, u16 tag, u16 inner_tag, | 97 | void (*set_vlan_tag)(struct dma_desc *p, u16 tag, u16 inner_tag, |
| 97 | u32 inner_type); | 98 | u32 inner_type); |
| @@ -269,19 +269,22 @@ static int ndesc_get_rx_timestamp_status(void *desc, void *next_desc, u32 ats) · 268 unmodified lines | |||
| 269 | return 1; | 269 | return 1; |
| 270 | } | 270 | } |
| 271 | 271 | ||
| 272 | static void ndesc_display_ring(void *head, unsigned int size, bool rx) | 272 | static void ndesc_display_ring(void *head, unsigned int size, bool rx, |
| 273 | dma_addr_t dma_rx_phy, unsigned int desc_size) | ||
| 273 | { | 274 | { |
| 274 | struct dma_desc *p = (struct dma_desc *)head; | 275 | struct dma_desc *p = (struct dma_desc *)head; |
| 276 | dma_addr_t dma_addr; | ||
| 275 | int i; | 277 | int i; |
| 276 | 278 | ||
| 277 | pr_info("%s descriptor ring:\n", rx ? "RX" : "TX"); | 279 | pr_info("%s descriptor ring:\n", rx ? "RX" : "TX"); |
| 278 | 280 | ||
| 279 | for (i = 0; i < size; i++) { | 281 | for (i = 0; i < size; i++) { |
| 280 | u64 x; | 282 | u64 x; |
| 283 | dma_addr = dma_rx_phy + i * sizeof(*p); | ||
| 281 | 284 | ||
| 282 | x = *(u64 *)p; | 285 | x = *(u64 *)p; |
| 283 | pr_info("%03d [0x%x]: 0x%x 0x%x 0x%x 0x%x", | 286 | pr_info("%03d [%pad]: 0x%x 0x%x 0x%x 0x%x", |
| 284 | i, (unsigned int)virt_to_phys(p), | 287 | i, &dma_addr, |
| 285 | (unsigned int)x, (unsigned int)(x >> 32), | 288 | (unsigned int)x, (unsigned int)(x >> 32), |
| 286 | p->des2, p->des3); | 289 | p->des2, p->des3); |
| 287 | p++; | 290 | p++; |
| @@ -1133,6 +1133,7 @@ static int stmmac_phy_setup(struct stmmac_priv *priv) · 1132 unmodified lines | |||
| 1133 | static void stmmac_display_rx_rings(struct stmmac_priv *priv) | 1133 | static void stmmac_display_rx_rings(struct stmmac_priv *priv) |
| 1134 | { | 1134 | { |
| 1135 | u32 rx_cnt = priv->plat->rx_queues_to_use; | 1135 | u32 rx_cnt = priv->plat->rx_queues_to_use; |
| 1136 | unsigned int desc_size; | ||
| 1136 | void *head_rx; | 1137 | void *head_rx; |
| 1137 | u32 queue; | 1138 | u32 queue; |
| 1138 | 1139 | ||
| @@ -1142,19 +1143,24 @@ static void stmmac_display_rx_rings(struct stmmac_priv *priv) · 3 unmodified lines | |||
| 1142 | 1143 | ||
| 1143 | pr_info("\tRX Queue %u rings\n", queue); | 1144 | pr_info("\tRX Queue %u rings\n", queue); |
| 1144 | 1145 | ||
| 1145 | if (priv->extend_desc) | 1146 | if (priv->extend_desc) { |
| 1146 | head_rx = (void *)rx_q->dma_erx; | 1147 | head_rx = (void *)rx_q->dma_erx; |
| 1147 | else | 1148 | desc_size = sizeof(struct dma_extended_desc); |
| 1149 | } else { | ||
| 1148 | head_rx = (void *)rx_q->dma_rx; | 1150 | head_rx = (void *)rx_q->dma_rx; |
| 1151 | desc_size = sizeof(struct dma_desc); | ||
| 1152 | } | ||
| 1149 | 1153 | ||
| 1150 | /* Display RX ring */ | 1154 | /* Display RX ring */ |
| 1151 | stmmac_display_ring(priv, head_rx, priv->dma_rx_size, true); | 1155 | stmmac_display_ring(priv, head_rx, priv->dma_rx_size, true, |
| 1156 | rx_q->dma_rx_phy, desc_size); | ||
| 1152 | } | 1157 | } |
| 1153 | } | 1158 | } |
| 1154 | 1159 | ||
| 1155 | static void stmmac_display_tx_rings(struct stmmac_priv *priv) | 1160 | static void stmmac_display_tx_rings(struct stmmac_priv *priv) |
| 1156 | { | 1161 | { |
| 1157 | u32 tx_cnt = priv->plat->tx_queues_to_use; | 1162 | u32 tx_cnt = priv->plat->tx_queues_to_use; |
| 1163 | unsigned int desc_size; | ||
| 1158 | void *head_tx; | 1164 | void *head_tx; |
| 1159 | u32 queue; | 1165 | u32 queue; |
| 1160 | 1166 | ||
| @@ -1164,14 +1170,19 @@ static void stmmac_display_tx_rings(struct stmmac_priv *priv) · 3 unmodified lines | |||
| 1164 | 1170 | ||
| 1165 | pr_info("\tTX Queue %d rings\n", queue); | 1171 | pr_info("\tTX Queue %d rings\n", queue); |
| 1166 | 1172 | ||
| 1167 | if (priv->extend_desc) | 1173 | if (priv->extend_desc) { |
| 1168 | head_tx = (void *)tx_q->dma_etx; | 1174 | head_tx = (void *)tx_q->dma_etx; |
| 1169 | else if (tx_q->tbs & STMMAC_TBS_AVAIL) | 1175 | desc_size = sizeof(struct dma_extended_desc); |
| 1176 | } else if (tx_q->tbs & STMMAC_TBS_AVAIL) { | ||
| 1170 | head_tx = (void *)tx_q->dma_entx; | 1177 | head_tx = (void *)tx_q->dma_entx; |
| 1171 | else | 1178 | desc_size = sizeof(struct dma_edesc); |
| 1179 | } else { | ||
| 1172 | head_tx = (void *)tx_q->dma_tx; | 1180 | head_tx = (void *)tx_q->dma_tx; |
| 1181 | desc_size = sizeof(struct dma_desc); | ||
| 1182 | } | ||
| 1173 | 1183 | ||
| 1174 | stmmac_display_ring(priv, head_tx, priv->dma_tx_size, false); | 1184 | stmmac_display_ring(priv, head_tx, priv->dma_tx_size, false, |
| 1185 | tx_q->dma_tx_phy, desc_size); | ||
| 1175 | } | 1186 | } |
| 1176 | } | 1187 | } |
| 1177 | 1188 | ||
| @@ -1303,9 +1314,10 @@ static int stmmac_init_rx_buffers(struct stmmac_priv *priv, struct dma_desc *p, · 125 unmodified lines | |||
| 1303 | return -ENOMEM; | 1314 | return -ENOMEM; |
| 1304 | 1315 | ||
| 1305 | buf->sec_addr = page_pool_get_dma_addr(buf->sec_page); | 1316 | buf->sec_addr = page_pool_get_dma_addr(buf->sec_page); |
| 1306 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr); | 1317 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, true); |
| 1307 | } else { | 1318 | } else { |
| 1308 | buf->sec_page = NULL; | 1319 | buf->sec_page = NULL; |
| 1320 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, false); | ||
| 1309 | } | 1321 | } |
| 1310 | 1322 | ||
| 1311 | buf->addr = page_pool_get_dma_addr(buf->page); | 1323 | buf->addr = page_pool_get_dma_addr(buf->page); |
| @@ -1367,6 +1379,88 @@ static void stmmac_free_tx_buffer(struct stmmac_priv *priv, u32 queue, int i) · 55 unmodified lines | |||
| 1367 | } | 1379 | } |
| 1368 | } | 1380 | } |
| 1369 | 1381 | ||
| 1382 | /** | ||
| 1383 | * stmmac_reinit_rx_buffers - reinit the RX descriptor buffer. | ||
| 1384 | * @priv: driver private structure | ||
| 1385 | * Description: this function is called to re-allocate a receive buffer, perform | ||
| 1386 | * the DMA mapping and init the descriptor. | ||
| 1387 | */ | ||
| 1388 | static void stmmac_reinit_rx_buffers(struct stmmac_priv *priv) | ||
| 1389 | { | ||
| 1390 | u32 rx_count = priv->plat->rx_queues_to_use; | ||
| 1391 | u32 queue; | ||
| 1392 | int i; | ||
| 1393 | |||
| 1394 | for (queue = 0; queue < rx_count; queue++) { | ||
| 1395 | struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; | ||
| 1396 | |||
| 1397 | for (i = 0; i < priv->dma_rx_size; i++) { | ||
| 1398 | struct stmmac_rx_buffer *buf = &rx_q->buf_pool[i]; | ||
| 1399 | |||
| 1400 | if (buf->page) { | ||
| 1401 | page_pool_recycle_direct(rx_q->page_pool, buf->page); | ||
| 1402 | buf->page = NULL; | ||
| 1403 | } | ||
| 1404 | |||
| 1405 | if (priv->sph && buf->sec_page) { | ||
| 1406 | page_pool_recycle_direct(rx_q->page_pool, buf->sec_page); | ||
| 1407 | buf->sec_page = NULL; | ||
| 1408 | } | ||
| 1409 | } | ||
| 1410 | } | ||
| 1411 | |||
| 1412 | for (queue = 0; queue < rx_count; queue++) { | ||
| 1413 | struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; | ||
| 1414 | |||
| 1415 | for (i = 0; i < priv->dma_rx_size; i++) { | ||
| 1416 | struct stmmac_rx_buffer *buf = &rx_q->buf_pool[i]; | ||
| 1417 | struct dma_desc *p; | ||
| 1418 | |||
| 1419 | if (priv->extend_desc) | ||
| 1420 | p = &((rx_q->dma_erx + i)->basic); | ||
| 1421 | else | ||
| 1422 | p = rx_q->dma_rx + i; | ||
| 1423 | |||
| 1424 | if (!buf->page) { | ||
| 1425 | buf->page = page_pool_dev_alloc_pages(rx_q->page_pool); | ||
| 1426 | if (!buf->page) | ||
| 1427 | goto err_reinit_rx_buffers; | ||
| 1428 | |||
| 1429 | buf->addr = page_pool_get_dma_addr(buf->page); | ||
| 1430 | } | ||
| 1431 | |||
| 1432 | if (priv->sph && !buf->sec_page) { | ||
| 1433 | buf->sec_page = page_pool_dev_alloc_pages(rx_q->page_pool); | ||
| 1434 | if (!buf->sec_page) | ||
| 1435 | goto err_reinit_rx_buffers; | ||
| 1436 | |||
| 1437 | buf->sec_addr = page_pool_get_dma_addr(buf->sec_page); | ||
| 1438 | } | ||
| 1439 | |||
| 1440 | stmmac_set_desc_addr(priv, p, buf->addr); | ||
| 1441 | if (priv->sph) | ||
| 1442 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, true); | ||
| 1443 | else | ||
| 1444 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, false); | ||
| 1445 | if (priv->dma_buf_sz == BUF_SIZE_16KiB) | ||
| 1446 | stmmac_init_desc3(priv, p); | ||
| 1447 | } | ||
| 1448 | } | ||
| 1449 | |||
| 1450 | return; | ||
| 1451 | |||
| 1452 | err_reinit_rx_buffers: | ||
| 1453 | do { | ||
| 1454 | while (--i >= 0) | ||
| 1455 | stmmac_free_rx_buffer(priv, queue, i); | ||
| 1456 | |||
| 1457 | if (queue == 0) | ||
| 1458 | break; | ||
| 1459 | |||
| 1460 | i = priv->dma_rx_size; | ||
| 1461 | } while (queue-- > 0); | ||
| 1462 | } | ||
| 1463 | |||
| 1370 | /** | 1464 | /** |
| 1371 | * init_dma_rx_desc_rings - init the RX descriptor rings | 1465 | * init_dma_rx_desc_rings - init the RX descriptor rings |
| 1372 | * @dev: net device structure | 1466 | * @dev: net device structure |
| @@ -3648,7 +3742,10 @@ static inline void stmmac_rx_refill(struct stmmac_priv *priv, u32 queue) · 2275 unmodified lines | |||
| 3648 | DMA_FROM_DEVICE); | 3742 | DMA_FROM_DEVICE); |
| 3649 | 3743 | ||
| 3650 | stmmac_set_desc_addr(priv, p, buf->addr); | 3744 | stmmac_set_desc_addr(priv, p, buf->addr); |
| 3651 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr); | 3745 | if (priv->sph) |
| 3746 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, true); | ||
| 3747 | else | ||
| 3748 | stmmac_set_desc_sec_addr(priv, p, buf->sec_addr, false); | ||
| 3652 | stmmac_refill_desc3(priv, rx_q, p); | 3749 | stmmac_refill_desc3(priv, rx_q, p); |
| 3653 | 3750 | ||
| 3654 | rx_q->rx_count_frames++; | 3751 | rx_q->rx_count_frames++; |
| @@ -3736,18 +3833,23 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue) · 81 unmodified lines | |||
| 3736 | unsigned int count = 0, error = 0, len = 0; | 3833 | unsigned int count = 0, error = 0, len = 0; |
| 3737 | int status = 0, coe = priv->hw->rx_csum; | 3834 | int status = 0, coe = priv->hw->rx_csum; |
| 3738 | unsigned int next_entry = rx_q->cur_rx; | 3835 | unsigned int next_entry = rx_q->cur_rx; |
| 3836 | unsigned int desc_size; | ||
| 3739 | struct sk_buff *skb = NULL; | 3837 | struct sk_buff *skb = NULL; |
| 3740 | 3838 | ||
| 3741 | if (netif_msg_rx_status(priv)) { | 3839 | if (netif_msg_rx_status(priv)) { |
| 3742 | void *rx_head; | 3840 | void *rx_head; |
| 3743 | 3841 | ||
| 3744 | netdev_dbg(priv->dev, "%s: descriptor ring:\n", __func__); | 3842 | netdev_dbg(priv->dev, "%s: descriptor ring:\n", __func__); |
| 3745 | if (priv->extend_desc) | 3843 | if (priv->extend_desc) { |
| 3746 | rx_head = (void *)rx_q->dma_erx; | 3844 | rx_head = (void *)rx_q->dma_erx; |
| 3747 | else | 3845 | desc_size = sizeof(struct dma_extended_desc); |
| 3846 | } else { | ||
| 3748 | rx_head = (void *)rx_q->dma_rx; | 3847 | rx_head = (void *)rx_q->dma_rx; |
| 3848 | desc_size = sizeof(struct dma_desc); | ||
| 3849 | } | ||
| 3749 | 3850 | ||
| 3750 | stmmac_display_ring(priv, rx_head, priv->dma_rx_size, true); | 3851 | stmmac_display_ring(priv, rx_head, priv->dma_rx_size, true, |
| 3852 | rx_q->dma_rx_phy, desc_size); | ||
| 3751 | } | 3853 | } |
| 3752 | while (count < limit) { | 3854 | while (count < limit) { |
| 3753 | unsigned int buf1_len = 0, buf2_len = 0; | 3855 | unsigned int buf1_len = 0, buf2_len = 0; |
| @@ -4315,24 +4417,27 @@ static int stmmac_set_mac_address(struct net_device *ndev, void *addr) · 561 unmodified lines | |||
| 4315 | static struct dentry *stmmac_fs_dir; | 4417 | static struct dentry *stmmac_fs_dir; |
| 4316 | 4418 | ||
| 4317 | static void sysfs_display_ring(void *head, int size, int extend_desc, | 4419 | static void sysfs_display_ring(void *head, int size, int extend_desc, |
| 4318 | struct seq_file *seq) | 4420 | struct seq_file *seq, dma_addr_t dma_phy_addr) |
| 4319 | { | 4421 | { |
| 4320 | int i; | 4422 | int i; |
| 4321 | struct dma_extended_desc *ep = (struct dma_extended_desc *)head; | 4423 | struct dma_extended_desc *ep = (struct dma_extended_desc *)head; |
| 4322 | struct dma_desc *p = (struct dma_desc *)head; | 4424 | struct dma_desc *p = (struct dma_desc *)head; |
| 4425 | dma_addr_t dma_addr; | ||
| 4323 | 4426 | ||
| 4324 | for (i = 0; i < size; i++) { | 4427 | for (i = 0; i < size; i++) { |
| 4325 | if (extend_desc) { | 4428 | if (extend_desc) { |
| 4326 | seq_printf(seq, "%d [0x%x]: 0x%x 0x%x 0x%x 0x%x\n", | 4429 | dma_addr = dma_phy_addr + i * sizeof(*ep); |
| 4327 | i, (unsigned int)virt_to_phys(ep), | 4430 | seq_printf(seq, "%d [%pad]: 0x%x 0x%x 0x%x 0x%x\n", |
| 4431 | i, &dma_addr, | ||
| 4328 | le32_to_cpu(ep->basic.des0), | 4432 | le32_to_cpu(ep->basic.des0), |
| 4329 | le32_to_cpu(ep->basic.des1), | 4433 | le32_to_cpu(ep->basic.des1), |
| 4330 | le32_to_cpu(ep->basic.des2), | 4434 | le32_to_cpu(ep->basic.des2), |
| 4331 | le32_to_cpu(ep->basic.des3)); | 4435 | le32_to_cpu(ep->basic.des3)); |
| 4332 | ep++; | 4436 | ep++; |
| 4333 | } else { | 4437 | } else { |
| 4334 | seq_printf(seq, "%d [0x%x]: 0x%x 0x%x 0x%x 0x%x\n", | 4438 | dma_addr = dma_phy_addr + i * sizeof(*p); |
| 4335 | i, (unsigned int)virt_to_phys(p), | 4439 | seq_printf(seq, "%d [%pad]: 0x%x 0x%x 0x%x 0x%x\n", |
| 4440 | i, &dma_addr, | ||
| 4336 | le32_to_cpu(p->des0), le32_to_cpu(p->des1), | 4441 | le32_to_cpu(p->des0), le32_to_cpu(p->des1), |
| 4337 | le32_to_cpu(p->des2), le32_to_cpu(p->des3)); | 4442 | le32_to_cpu(p->des2), le32_to_cpu(p->des3)); |
| 4338 | p++; | 4443 | p++; |
| @@ -4360,11 +4465,11 @@ static int stmmac_rings_status_show(struct seq_file *seq, void *v) · 21 unmodified lines | |||
| 4360 | if (priv->extend_desc) { | 4465 | if (priv->extend_desc) { |
| 4361 | seq_printf(seq, "Extended descriptor ring:\n"); | 4466 | seq_printf(seq, "Extended descriptor ring:\n"); |
| 4362 | sysfs_display_ring((void *)rx_q->dma_erx, | 4467 | sysfs_display_ring((void *)rx_q->dma_erx, |
| 4363 | priv->dma_rx_size, 1, seq); | 4468 | priv->dma_rx_size, 1, seq, rx_q->dma_rx_phy); |
| 4364 | } else { | 4469 | } else { |
| 4365 | seq_printf(seq, "Descriptor ring:\n"); | 4470 | seq_printf(seq, "Descriptor ring:\n"); |
| 4366 | sysfs_display_ring((void *)rx_q->dma_rx, | 4471 | sysfs_display_ring((void *)rx_q->dma_rx, |
| 4367 | priv->dma_rx_size, 0, seq); | 4472 | priv->dma_rx_size, 0, seq, rx_q->dma_rx_phy); |
| 4368 | } | 4473 | } |
| 4369 | } | 4474 | } |
| 4370 | 4475 | ||
| @@ -4376,11 +4481,11 @@ static int stmmac_rings_status_show(struct seq_file *seq, void *v) · 5 unmodified lines | |||
| 4376 | if (priv->extend_desc) { | 4481 | if (priv->extend_desc) { |
| 4377 | seq_printf(seq, "Extended descriptor ring:\n"); | 4482 | seq_printf(seq, "Extended descriptor ring:\n"); |
| 4378 | sysfs_display_ring((void *)tx_q->dma_etx, | 4483 | sysfs_display_ring((void *)tx_q->dma_etx, |
| 4379 | priv->dma_tx_size, 1, seq); | 4484 | priv->dma_tx_size, 1, seq, tx_q->dma_tx_phy); |
| 4380 | } else if (!(tx_q->tbs & STMMAC_TBS_AVAIL)) { | 4485 | } else if (!(tx_q->tbs & STMMAC_TBS_AVAIL)) { |
| 4381 | seq_printf(seq, "Descriptor ring:\n"); | 4486 | seq_printf(seq, "Descriptor ring:\n"); |
| 4382 | sysfs_display_ring((void *)tx_q->dma_tx, | 4487 | sysfs_display_ring((void *)tx_q->dma_tx, |
| 4383 | priv->dma_tx_size, 0, seq); | 4488 | priv->dma_tx_size, 0, seq, tx_q->dma_tx_phy); |
| 4384 | } | 4489 | } |
| 4385 | } | 4490 | } |
| 4386 | 4491 | ||
| @@ -5144,13 +5249,16 @@ int stmmac_dvr_remove(struct device *dev) · 757 unmodified lines | |||
| 5144 | netdev_info(priv->dev, "%s: removing driver", __func__); | 5249 | netdev_info(priv->dev, "%s: removing driver", __func__); |
| 5145 | 5250 | ||
| 5146 | stmmac_stop_all_dma(priv); | 5251 | stmmac_stop_all_dma(priv); |
| 5252 | stmmac_mac_set(priv, priv->ioaddr, false); | ||
| 5253 | netif_carrier_off(ndev); | ||
| 5254 | unregister_netdev(ndev); | ||
| 5147 | 5255 | ||
| 5256 | /* Serdes power down needs to happen after VLAN filter | ||
| 5257 | * is deleted that is triggered by unregister_netdev(). | ||
| 5258 | */ | ||
| 5148 | if (priv->plat->serdes_powerdown) | 5259 | if (priv->plat->serdes_powerdown) |
| 5149 | priv->plat->serdes_powerdown(ndev, priv->plat->bsp_priv); | 5260 | priv->plat->serdes_powerdown(ndev, priv->plat->bsp_priv); |
| 5150 | 5261 | ||
| 5151 | stmmac_mac_set(priv, priv->ioaddr, false); | ||
| 5152 | netif_carrier_off(ndev); | ||
| 5153 | unregister_netdev(ndev); | ||
| 5154 | #ifdef CONFIG_DEBUG_FS | 5262 | #ifdef CONFIG_DEBUG_FS |
| 5155 | stmmac_exit_fs(ndev); | 5263 | stmmac_exit_fs(ndev); |
| 5156 | #endif | 5264 | #endif |
| @@ -5257,6 +5365,8 @@ static void stmmac_reset_queues_param(struct stmmac_priv *priv) · 100 unmodified lines | |||
| 5257 | tx_q->cur_tx = 0; | 5365 | tx_q->cur_tx = 0; |
| 5258 | tx_q->dirty_tx = 0; | 5366 | tx_q->dirty_tx = 0; |
| 5259 | tx_q->mss = 0; | 5367 | tx_q->mss = 0; |
| 5368 | |||
| 5369 | netdev_tx_reset_queue(netdev_get_tx_queue(priv->dev, queue)); | ||
| 5260 | } | 5370 | } |
| 5261 | } | 5371 | } |
| 5262 | 5372 | ||
| @@ -5318,7 +5428,7 @@ int stmmac_resume(struct device *dev) · 55 unmodified lines | |||
| 5318 | mutex_lock(&priv->lock); | 5428 | mutex_lock(&priv->lock); |
| 5319 | 5429 | ||
| 5320 | stmmac_reset_queues_param(priv); | 5430 | stmmac_reset_queues_param(priv); |
| 5321 | 5431 | stmmac_reinit_rx_buffers(priv); | |
| 5322 | stmmac_free_tx_skbufs(priv); | 5432 | stmmac_free_tx_skbufs(priv); |
| 5323 | stmmac_clear_descriptors(priv); | 5433 | stmmac_clear_descriptors(priv); |
| 5324 | 5434 | ||
| @@ -3931,8 +3931,6 @@ static void niu_xmac_interrupt(struct niu *np) · 3930 unmodified lines | |||
| 3931 | mp->rx_mcasts += RXMAC_MC_FRM_CNT_COUNT; | 3931 | mp->rx_mcasts += RXMAC_MC_FRM_CNT_COUNT; |
| 3932 | if (val & XRXMAC_STATUS_RXBCAST_CNT_EXP) | 3932 | if (val & XRXMAC_STATUS_RXBCAST_CNT_EXP) |
| 3933 | mp->rx_bcasts += RXMAC_BC_FRM_CNT_COUNT; | 3933 | mp->rx_bcasts += RXMAC_BC_FRM_CNT_COUNT; |
| 3934 | if (val & XRXMAC_STATUS_RXBCAST_CNT_EXP) | ||
| 3935 | mp->rx_bcasts += RXMAC_BC_FRM_CNT_COUNT; | ||
| 3936 | if (val & XRXMAC_STATUS_RXHIST1_CNT_EXP) | 3934 | if (val & XRXMAC_STATUS_RXHIST1_CNT_EXP) |
| 3937 | mp->rx_hist_cnt1 += RXMAC_HIST_CNT1_COUNT; | 3935 | mp->rx_hist_cnt1 += RXMAC_HIST_CNT1_COUNT; |
| 3938 | if (val & XRXMAC_STATUS_RXHIST2_CNT_EXP) | 3936 | if (val & XRXMAC_STATUS_RXHIST2_CNT_EXP) |
| @@ -2044,6 +2044,7 @@ bdx_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 2043 unmodified lines | |||
| 2044 | /*bdx_hw_reset(priv); */ | 2044 | /*bdx_hw_reset(priv); */ |
| 2045 | if (bdx_read_mac(priv)) { | 2045 | if (bdx_read_mac(priv)) { |
| 2046 | pr_err("load MAC address failed\n"); | 2046 | pr_err("load MAC address failed\n"); |
| 2047 | err = -EFAULT; | ||
| 2047 | goto err_out_iomap; | 2048 | goto err_out_iomap; |
| 2048 | } | 2049 | } |
| 2049 | SET_NETDEV_DEV(ndev, &pdev->dev); | 2050 | SET_NETDEV_DEV(ndev, &pdev->dev); |
| @@ -171,11 +171,6 @@ static void sp_encaps(struct sixpack *sp, unsigned char *icp, int len) · 170 unmodified lines | |||
| 171 | goto out_drop; | 171 | goto out_drop; |
| 172 | } | 172 | } |
| 173 | 173 | ||
| 174 | if (len > sp->mtu) { /* sp->mtu = AX25_MTU = max. PACLEN = 256 */ | ||
| 175 | msg = "oversized transmit packet!"; | ||
| 176 | goto out_drop; | ||
| 177 | } | ||
| 178 | |||
| 179 | if (p[0] > 5) { | 174 | if (p[0] > 5) { |
| 180 | msg = "invalid KISS command"; | 175 | msg = "invalid KISS command"; |
| 181 | goto out_drop; | 176 | goto out_drop; |
| @@ -229,7 +229,7 @@ int netvsc_send(struct net_device *net, · 228 unmodified lines | |||
| 229 | bool xdp_tx); | 229 | bool xdp_tx); |
| 230 | void netvsc_linkstatus_callback(struct net_device *net, | 230 | void netvsc_linkstatus_callback(struct net_device *net, |
| 231 | struct rndis_message *resp, | 231 | struct rndis_message *resp, |
| 232 | void *data); | 232 | void *data, u32 data_buflen); |
| 233 | int netvsc_recv_callback(struct net_device *net, | 233 | int netvsc_recv_callback(struct net_device *net, |
| 234 | struct netvsc_device *nvdev, | 234 | struct netvsc_device *nvdev, |
| 235 | struct netvsc_channel *nvchan); | 235 | struct netvsc_channel *nvchan); |
| @@ -744,7 +744,7 @@ static netdev_tx_t netvsc_start_xmit(struct sk_buff *skb, · 743 unmodified lines | |||
| 744 | */ | 744 | */ |
| 745 | void netvsc_linkstatus_callback(struct net_device *net, | 745 | void netvsc_linkstatus_callback(struct net_device *net, |
| 746 | struct rndis_message *resp, | 746 | struct rndis_message *resp, |
| 747 | void *data) | 747 | void *data, u32 data_buflen) |
| 748 | { | 748 | { |
| 749 | struct rndis_indicate_status *indicate = &resp->msg.indicate_status; | 749 | struct rndis_indicate_status *indicate = &resp->msg.indicate_status; |
| 750 | struct net_device_context *ndev_ctx = netdev_priv(net); | 750 | struct net_device_context *ndev_ctx = netdev_priv(net); |
| @@ -765,11 +765,16 @@ void netvsc_linkstatus_callback(struct net_device *net, · 14 unmodified lines | |||
| 765 | if (indicate->status == RNDIS_STATUS_LINK_SPEED_CHANGE) { | 765 | if (indicate->status == RNDIS_STATUS_LINK_SPEED_CHANGE) { |
| 766 | u32 speed; | 766 | u32 speed; |
| 767 | 767 | ||
| 768 | /* Validate status_buf_offset */ | 768 | /* Validate status_buf_offset and status_buflen. |
| 769 | * | ||
| 770 | * Certain (pre-Fe) implementations of Hyper-V's vSwitch didn't account | ||
| 771 | * for the status buffer field in resp->msg_len; perform the validation | ||
| 772 | * using data_buflen (>= resp->msg_len). | ||
| 773 | */ | ||
| 769 | if (indicate->status_buflen < sizeof(speed) || | 774 | if (indicate->status_buflen < sizeof(speed) || |
| 770 | indicate->status_buf_offset < sizeof(*indicate) || | 775 | indicate->status_buf_offset < sizeof(*indicate) || |
| 771 | resp->msg_len - RNDIS_HEADER_SIZE < indicate->status_buf_offset || | 776 | data_buflen - RNDIS_HEADER_SIZE < indicate->status_buf_offset || |
| 772 | resp->msg_len - RNDIS_HEADER_SIZE - indicate->status_buf_offset | 777 | data_buflen - RNDIS_HEADER_SIZE - indicate->status_buf_offset |
| 773 | < indicate->status_buflen) { | 778 | < indicate->status_buflen) { |
| 774 | netdev_err(net, "invalid rndis_indicate_status packet\n"); | 779 | netdev_err(net, "invalid rndis_indicate_status packet\n"); |
| 775 | return; | 780 | return; |
| @@ -620,7 +620,7 @@ int rndis_filter_receive(struct net_device *ndev, · 619 unmodified lines | |||
| 620 | 620 | ||
| 621 | case RNDIS_MSG_INDICATE: | 621 | case RNDIS_MSG_INDICATE: |
| 622 | /* notification msgs */ | 622 | /* notification msgs */ |
| 623 | netvsc_linkstatus_callback(ndev, rndis_msg, data); | 623 | netvsc_linkstatus_callback(ndev, rndis_msg, data, buflen); |
| 624 | break; | 624 | break; |
| 625 | default: | 625 | default: |
| 626 | netdev_err(ndev, | 626 | netdev_err(ndev, |
| @@ -294,6 +294,7 @@ nsim_create(struct nsim_dev *nsim_dev, struct nsim_dev_port *nsim_dev_port) · 293 unmodified lines | |||
| 294 | dev_net_set(dev, nsim_dev_net(nsim_dev)); | 294 | dev_net_set(dev, nsim_dev_net(nsim_dev)); |
| 295 | ns = netdev_priv(dev); | 295 | ns = netdev_priv(dev); |
| 296 | ns->netdev = dev; | 296 | ns->netdev = dev; |
| 297 | u64_stats_init(&ns->syncp); | ||
| 297 | ns->nsim_dev = nsim_dev; | 298 | ns->nsim_dev = nsim_dev; |
| 298 | ns->nsim_dev_port = nsim_dev_port; | 299 | ns->nsim_dev_port = nsim_dev_port; |
| 299 | ns->nsim_bus_dev = nsim_dev->nsim_bus_dev; | 300 | ns->nsim_bus_dev = nsim_dev->nsim_bus_dev; |
| @@ -290,6 +290,7 @@ static int dp83822_config_intr(struct phy_device *phydev) · 289 unmodified lines | |||
| 290 | 290 | ||
| 291 | static irqreturn_t dp83822_handle_interrupt(struct phy_device *phydev) | 291 | static irqreturn_t dp83822_handle_interrupt(struct phy_device *phydev) |
| 292 | { | 292 | { |
| 293 | bool trigger_machine = false; | ||
| 293 | int irq_status; | 294 | int irq_status; |
| 294 | 295 | ||
| 295 | /* The MISR1 and MISR2 registers are holding the interrupt status in | 296 | /* The MISR1 and MISR2 registers are holding the interrupt status in |
| @@ -305,7 +306,7 @@ static irqreturn_t dp83822_handle_interrupt(struct phy_device *phydev) · 9 unmodified lines | |||
| 305 | return IRQ_NONE; | 306 | return IRQ_NONE; |
| 306 | } | 307 | } |
| 307 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) | 308 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) |
| 308 | goto trigger_machine; | 309 | trigger_machine = true; |
| 309 | 310 | ||
| 310 | irq_status = phy_read(phydev, MII_DP83822_MISR2); | 311 | irq_status = phy_read(phydev, MII_DP83822_MISR2); |
| 311 | if (irq_status < 0) { | 312 | if (irq_status < 0) { |
| @@ -313,11 +314,11 @@ static irqreturn_t dp83822_handle_interrupt(struct phy_device *phydev) · 1 unmodified line | |||
| 313 | return IRQ_NONE; | 314 | return IRQ_NONE; |
| 314 | } | 315 | } |
| 315 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) | 316 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) |
| 316 | goto trigger_machine; | 317 | trigger_machine = true; |
| 317 | 318 | ||
| 318 | return IRQ_NONE; | 319 | if (!trigger_machine) |
| 320 | return IRQ_NONE; | ||
| 319 | 321 | ||
| 320 | trigger_machine: | ||
| 321 | phy_trigger_machine(phydev); | 322 | phy_trigger_machine(phydev); |
| 322 | 323 | ||
| 323 | return IRQ_HANDLED; | 324 | return IRQ_HANDLED; |
| @@ -264,6 +264,7 @@ static int dp83811_config_intr(struct phy_device *phydev) · 263 unmodified lines | |||
| 264 | 264 | ||
| 265 | static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev) | 265 | static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev) |
| 266 | { | 266 | { |
| 267 | bool trigger_machine = false; | ||
| 267 | int irq_status; | 268 | int irq_status; |
| 268 | 269 | ||
| 269 | /* The INT_STAT registers 1, 2 and 3 are holding the interrupt status | 270 | /* The INT_STAT registers 1, 2 and 3 are holding the interrupt status |
| @@ -279,7 +280,7 @@ static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev) · 9 unmodified lines | |||
| 279 | return IRQ_NONE; | 280 | return IRQ_NONE; |
| 280 | } | 281 | } |
| 281 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) | 282 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) |
| 282 | goto trigger_machine; | 283 | trigger_machine = true; |
| 283 | 284 | ||
| 284 | irq_status = phy_read(phydev, MII_DP83811_INT_STAT2); | 285 | irq_status = phy_read(phydev, MII_DP83811_INT_STAT2); |
| 285 | if (irq_status < 0) { | 286 | if (irq_status < 0) { |
| @@ -287,7 +288,7 @@ static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev) · 1 unmodified line | |||
| 287 | return IRQ_NONE; | 288 | return IRQ_NONE; |
| 288 | } | 289 | } |
| 289 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) | 290 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) |
| 290 | goto trigger_machine; | 291 | trigger_machine = true; |
| 291 | 292 | ||
| 292 | irq_status = phy_read(phydev, MII_DP83811_INT_STAT3); | 293 | irq_status = phy_read(phydev, MII_DP83811_INT_STAT3); |
| 293 | if (irq_status < 0) { | 294 | if (irq_status < 0) { |
| @@ -295,11 +296,11 @@ static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev) · 1 unmodified line | |||
| 295 | return IRQ_NONE; | 296 | return IRQ_NONE; |
| 296 | } | 297 | } |
| 297 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) | 298 | if (irq_status & ((irq_status & GENMASK(7, 0)) << 8)) |
| 298 | goto trigger_machine; | 299 | trigger_machine = true; |
| 299 | 300 | ||
| 300 | return IRQ_NONE; | 301 | if (!trigger_machine) |
| 302 | return IRQ_NONE; | ||
| 301 | 303 | ||
| 302 | trigger_machine: | ||
| 303 | phy_trigger_machine(phydev); | 304 | phy_trigger_machine(phydev); |
| 304 | 305 | ||
| 305 | return IRQ_HANDLED; | 306 | return IRQ_HANDLED; |
| @@ -276,14 +276,16 @@ int phy_ethtool_ksettings_set(struct phy_device *phydev, · 275 unmodified lines | |||
| 276 | 276 | ||
| 277 | phydev->autoneg = autoneg; | 277 | phydev->autoneg = autoneg; |
| 278 | 278 | ||
| 279 | phydev->speed = speed; | 279 | if (autoneg == AUTONEG_DISABLE) { |
| 280 | phydev->speed = speed; | ||
| 281 | phydev->duplex = duplex; | ||
| 282 | } | ||
| 280 | 283 | ||
| 281 | linkmode_copy(phydev->advertising, advertising); | 284 | linkmode_copy(phydev->advertising, advertising); |
| 282 | 285 | ||
| 283 | linkmode_mod_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, | 286 | linkmode_mod_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, |
| 284 | phydev->advertising, autoneg == AUTONEG_ENABLE); | 287 | phydev->advertising, autoneg == AUTONEG_ENABLE); |
| 285 | 288 | ||
| 286 | phydev->duplex = duplex; | ||
| 287 | phydev->master_slave_set = cmd->base.master_slave_cfg; | 289 | phydev->master_slave_set = cmd->base.master_slave_cfg; |
| 288 | phydev->mdix_ctrl = cmd->base.eth_tp_mdix_ctrl; | 290 | phydev->mdix_ctrl = cmd->base.eth_tp_mdix_ctrl; |
| 289 | 291 | ||
| @@ -230,7 +230,6 @@ static struct phy_driver genphy_driver; · 229 unmodified lines | |||
| 230 | static LIST_HEAD(phy_fixup_list); | 230 | static LIST_HEAD(phy_fixup_list); |
| 231 | static DEFINE_MUTEX(phy_fixup_lock); | 231 | static DEFINE_MUTEX(phy_fixup_lock); |
| 232 | 232 | ||
| 233 | #ifdef CONFIG_PM | ||
| 234 | static bool mdio_bus_phy_may_suspend(struct phy_device *phydev) | 233 | static bool mdio_bus_phy_may_suspend(struct phy_device *phydev) |
| 235 | { | 234 | { |
| 236 | struct device_driver *drv = phydev->mdio.dev.driver; | 235 | struct device_driver *drv = phydev->mdio.dev.driver; |
| @@ -270,7 +269,7 @@ out: · 33 unmodified lines | |||
| 270 | return !phydev->suspended; | 269 | return !phydev->suspended; |
| 271 | } | 270 | } |
| 272 | 271 | ||
| 273 | static int mdio_bus_phy_suspend(struct device *dev) | 272 | static __maybe_unused int mdio_bus_phy_suspend(struct device *dev) |
| 274 | { | 273 | { |
| 275 | struct phy_device *phydev = to_phy_device(dev); | 274 | struct phy_device *phydev = to_phy_device(dev); |
| 276 | 275 | ||
| @@ -290,7 +289,7 @@ static int mdio_bus_phy_suspend(struct device *dev) · 13 unmodified lines | |||
| 290 | return phy_suspend(phydev); | 289 | return phy_suspend(phydev); |
| 291 | } | 290 | } |
| 292 | 291 | ||
| 293 | static int mdio_bus_phy_resume(struct device *dev) | 292 | static __maybe_unused int mdio_bus_phy_resume(struct device *dev) |
| 294 | { | 293 | { |
| 295 | struct phy_device *phydev = to_phy_device(dev); | 294 | struct phy_device *phydev = to_phy_device(dev); |
| 296 | int ret; | 295 | int ret; |
| @@ -316,7 +315,6 @@ no_resume: · 19 unmodified lines | |||
| 316 | 315 | ||
| 317 | static SIMPLE_DEV_PM_OPS(mdio_bus_phy_pm_ops, mdio_bus_phy_suspend, | 316 | static SIMPLE_DEV_PM_OPS(mdio_bus_phy_pm_ops, mdio_bus_phy_suspend, |
| 318 | mdio_bus_phy_resume); | 317 | mdio_bus_phy_resume); |
| 319 | #endif /* CONFIG_PM */ | ||
| 320 | 318 | ||
| 321 | /** | 319 | /** |
| 322 | * phy_register_fixup - creates a new phy_fixup and adds it to the list | 320 | * phy_register_fixup - creates a new phy_fixup and adds it to the list |
| @@ -851,17 +851,17 @@ int cdc_ncm_bind_common(struct usbnet *dev, struct usb_interface *intf, u8 data_ · 850 unmodified lines | |||
| 851 | 851 | ||
| 852 | /* check if we got everything */ | 852 | /* check if we got everything */ |
| 853 | if (!ctx->data) { | 853 | if (!ctx->data) { |
| 854 | dev_dbg(&intf->dev, "CDC Union missing and no IAD found\n"); | 854 | dev_err(&intf->dev, "CDC Union missing and no IAD found\n"); |
| 855 | goto error; | 855 | goto error; |
| 856 | } | 856 | } |
| 857 | if (cdc_ncm_comm_intf_is_mbim(intf->cur_altsetting)) { | 857 | if (cdc_ncm_comm_intf_is_mbim(intf->cur_altsetting)) { |
| 858 | if (!ctx->mbim_desc) { | 858 | if (!ctx->mbim_desc) { |
| 859 | dev_dbg(&intf->dev, "MBIM functional descriptor missing\n"); | 859 | dev_err(&intf->dev, "MBIM functional descriptor missing\n"); |
| 860 | goto error; | 860 | goto error; |
| 861 | } | 861 | } |
| 862 | } else { | 862 | } else { |
| 863 | if (!ctx->ether_desc || !ctx->func_desc) { | 863 | if (!ctx->ether_desc || !ctx->func_desc) { |
| 864 | dev_dbg(&intf->dev, "NCM or ECM functional descriptors missing\n"); | 864 | dev_err(&intf->dev, "NCM or ECM functional descriptors missing\n"); |
| 865 | goto error; | 865 | goto error; |
| 866 | } | 866 | } |
| 867 | } | 867 | } |
| @@ -870,7 +870,7 @@ int cdc_ncm_bind_common(struct usbnet *dev, struct usb_interface *intf, u8 data_ · 2 unmodified lines | |||
| 870 | if (ctx->data != ctx->control) { | 870 | if (ctx->data != ctx->control) { |
| 871 | temp = usb_driver_claim_interface(driver, ctx->data, dev); | 871 | temp = usb_driver_claim_interface(driver, ctx->data, dev); |
| 872 | if (temp) { | 872 | if (temp) { |
| 873 | dev_dbg(&intf->dev, "failed to claim data intf\n"); | 873 | dev_err(&intf->dev, "failed to claim data intf\n"); |
| 874 | goto error; | 874 | goto error; |
| 875 | } | 875 | } |
| 876 | } | 876 | } |
| @@ -926,7 +926,7 @@ int cdc_ncm_bind_common(struct usbnet *dev, struct usb_interface *intf, u8 data_ · 49 unmodified lines | |||
| 926 | if (ctx->ether_desc) { | 926 | if (ctx->ether_desc) { |
| 927 | temp = usbnet_get_ethernet_addr(dev, ctx->ether_desc->iMACAddress); | 927 | temp = usbnet_get_ethernet_addr(dev, ctx->ether_desc->iMACAddress); |
| 928 | if (temp) { | 928 | if (temp) { |
| 929 | dev_dbg(&intf->dev, "failed to get mac address\n"); | 929 | dev_err(&intf->dev, "failed to get mac address\n"); |
| 930 | goto error2; | 930 | goto error2; |
| 931 | } | 931 | } |
| 932 | dev_info(&intf->dev, "MAC-Address: %pM\n", dev->net->dev_addr); | 932 | dev_info(&intf->dev, "MAC-Address: %pM\n", dev->net->dev_addr); |
| @@ -429,13 +429,6 @@ static ssize_t add_mux_store(struct device *d, struct device_attribute *attr, c · 428 unmodified lines | |||
| 429 | goto err; | 429 | goto err; |
| 430 | } | 430 | } |
| 431 | 431 | ||
| 432 | /* we don't want to modify a running netdev */ | ||
| 433 | if (netif_running(dev->net)) { | ||
| 434 | netdev_err(dev->net, "Cannot change a running device\n"); | ||
| 435 | ret = -EBUSY; | ||
| 436 | goto err; | ||
| 437 | } | ||
| 438 | |||
| 439 | ret = qmimux_register_device(dev->net, mux_id); | 432 | ret = qmimux_register_device(dev->net, mux_id); |
| 440 | if (!ret) { | 433 | if (!ret) { |
| 441 | info->flags |= QMI_WWAN_FLAG_MUX; | 434 | info->flags |= QMI_WWAN_FLAG_MUX; |
| @@ -465,13 +458,6 @@ static ssize_t del_mux_store(struct device *d, struct device_attribute *attr, c · 23 unmodified lines | |||
| 465 | if (!rtnl_trylock()) | 458 | if (!rtnl_trylock()) |
| 466 | return restart_syscall(); | 459 | return restart_syscall(); |
| 467 | 460 | ||
| 468 | /* we don't want to modify a running netdev */ | ||
| 469 | if (netif_running(dev->net)) { | ||
| 470 | netdev_err(dev->net, "Cannot change a running device\n"); | ||
| 471 | ret = -EBUSY; | ||
| 472 | goto err; | ||
| 473 | } | ||
| 474 | |||
| 475 | del_dev = qmimux_find_dev(dev, mux_id); | 461 | del_dev = qmimux_find_dev(dev, mux_id); |
| 476 | if (!del_dev) { | 462 | if (!del_dev) { |
| 477 | netdev_err(dev->net, "mux_id not present\n"); | 463 | netdev_err(dev->net, "mux_id not present\n"); |
| @@ -3021,29 +3021,6 @@ static void __rtl_set_wol(struct r8152 *tp, u32 wolopts) · 3020 unmodified lines | |||
| 3021 | device_set_wakeup_enable(&tp->udev->dev, false); | 3021 | device_set_wakeup_enable(&tp->udev->dev, false); |
| 3022 | } | 3022 | } |
| 3023 | 3023 | ||
| 3024 | static void r8153_mac_clk_spd(struct r8152 *tp, bool enable) | ||
| 3025 | { | ||
| 3026 | /* MAC clock speed down */ | ||
| 3027 | if (enable) { | ||
| 3028 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL, | ||
| 3029 | ALDPS_SPDWN_RATIO); | ||
| 3030 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL2, | ||
| 3031 | EEE_SPDWN_RATIO); | ||
| 3032 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL3, | ||
| 3033 | PKT_AVAIL_SPDWN_EN | SUSPEND_SPDWN_EN | | ||
| 3034 | U1U2_SPDWN_EN | L1_SPDWN_EN); | ||
| 3035 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL4, | ||
| 3036 | PWRSAVE_SPDWN_EN | RXDV_SPDWN_EN | TX10MIDLE_EN | | ||
| 3037 | TP100_SPDWN_EN | TP500_SPDWN_EN | EEE_SPDWN_EN | | ||
| 3038 | TP1000_SPDWN_EN); | ||
| 3039 | } else { | ||
| 3040 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL, 0); | ||
| 3041 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL2, 0); | ||
| 3042 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL3, 0); | ||
| 3043 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL4, 0); | ||
| 3044 | } | ||
| 3045 | } | ||
| 3046 | |||
| 3047 | static void r8153_u1u2en(struct r8152 *tp, bool enable) | 3024 | static void r8153_u1u2en(struct r8152 *tp, bool enable) |
| 3048 | { | 3025 | { |
| 3049 | u8 u1u2[8]; | 3026 | u8 u1u2[8]; |
| @@ -3338,11 +3315,9 @@ static void rtl8153_runtime_enable(struct r8152 *tp, bool enable) · 288 unmodified lines | |||
| 3338 | if (enable) { | 3315 | if (enable) { |
| 3339 | r8153_u1u2en(tp, false); | 3316 | r8153_u1u2en(tp, false); |
| 3340 | r8153_u2p3en(tp, false); | 3317 | r8153_u2p3en(tp, false); |
| 3341 | r8153_mac_clk_spd(tp, true); | ||
| 3342 | rtl_runtime_suspend_enable(tp, true); | 3318 | rtl_runtime_suspend_enable(tp, true); |
| 3343 | } else { | 3319 | } else { |
| 3344 | rtl_runtime_suspend_enable(tp, false); | 3320 | rtl_runtime_suspend_enable(tp, false); |
| 3345 | r8153_mac_clk_spd(tp, false); | ||
| 3346 | 3321 | ||
| 3347 | switch (tp->version) { | 3322 | switch (tp->version) { |
| 3348 | case RTL_VER_03: | 3323 | case RTL_VER_03: |
| @@ -4718,7 +4693,6 @@ static void r8153_first_init(struct r8152 *tp) · 1369 unmodified lines | |||
| 4718 | { | 4693 | { |
| 4719 | u32 ocp_data; | 4694 | u32 ocp_data; |
| 4720 | 4695 | ||
| 4721 | r8153_mac_clk_spd(tp, false); | ||
| 4722 | rxdy_gated_en(tp, true); | 4696 | rxdy_gated_en(tp, true); |
| 4723 | r8153_teredo_off(tp); | 4697 | r8153_teredo_off(tp); |
| 4724 | 4698 | ||
| @@ -4769,8 +4743,6 @@ static void r8153_enter_oob(struct r8152 *tp) · 44 unmodified lines | |||
| 4769 | { | 4743 | { |
| 4770 | u32 ocp_data; | 4744 | u32 ocp_data; |
| 4771 | 4745 | ||
| 4772 | r8153_mac_clk_spd(tp, true); | ||
| 4773 | |||
| 4774 | ocp_data = ocp_read_byte(tp, MCU_TYPE_PLA, PLA_OOB_CTRL); | 4746 | ocp_data = ocp_read_byte(tp, MCU_TYPE_PLA, PLA_OOB_CTRL); |
| 4775 | ocp_data &= ~NOW_IS_OOB; | 4747 | ocp_data &= ~NOW_IS_OOB; |
| 4776 | ocp_write_byte(tp, MCU_TYPE_PLA, PLA_OOB_CTRL, ocp_data); | 4748 | ocp_write_byte(tp, MCU_TYPE_PLA, PLA_OOB_CTRL, ocp_data); |
| @@ -5496,10 +5468,15 @@ static void r8153_init(struct r8152 *tp) · 719 unmodified lines | |||
| 5496 | 5468 | ||
| 5497 | ocp_write_word(tp, MCU_TYPE_USB, USB_CONNECT_TIMER, 0x0001); | 5469 | ocp_write_word(tp, MCU_TYPE_USB, USB_CONNECT_TIMER, 0x0001); |
| 5498 | 5470 | ||
| 5471 | /* MAC clock speed down */ | ||
| 5472 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL, 0); | ||
| 5473 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL2, 0); | ||
| 5474 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL3, 0); | ||
| 5475 | ocp_write_word(tp, MCU_TYPE_PLA, PLA_MAC_PWR_CTRL4, 0); | ||
| 5476 | |||
| 5499 | r8153_power_cut_en(tp, false); | 5477 | r8153_power_cut_en(tp, false); |
| 5500 | rtl_runtime_suspend_enable(tp, false); | 5478 | rtl_runtime_suspend_enable(tp, false); |
| 5501 | r8153_u1u2en(tp, true); | 5479 | r8153_u1u2en(tp, true); |
| 5502 | r8153_mac_clk_spd(tp, false); | ||
| 5503 | usb_enable_lpm(tp->udev); | 5480 | usb_enable_lpm(tp->udev); |
| 5504 | 5481 | ||
| 5505 | ocp_data = ocp_read_byte(tp, MCU_TYPE_PLA, PLA_CONFIG6); | 5482 | ocp_data = ocp_read_byte(tp, MCU_TYPE_PLA, PLA_CONFIG6); |
| @@ -887,7 +887,7 @@ int usbnet_open (struct net_device *net) · 886 unmodified lines | |||
| 887 | 887 | ||
| 888 | // insist peer be connected | 888 | // insist peer be connected |
| 889 | if (info->check_connect && (retval = info->check_connect (dev)) < 0) { | 889 | if (info->check_connect && (retval = info->check_connect (dev)) < 0) { |
| 890 | netif_dbg(dev, ifup, dev->net, "can't open; %d\n", retval); | 890 | netif_err(dev, ifup, dev->net, "can't open; %d\n", retval); |
| 891 | goto done; | 891 | goto done; |
| 892 | } | 892 | } |
| 893 | 893 | ||
| @@ -204,14 +204,18 @@ static int uhdlc_init(struct ucc_hdlc_private *priv) · 203 unmodified lines | |||
| 204 | priv->rx_skbuff = kcalloc(priv->rx_ring_size, | 204 | priv->rx_skbuff = kcalloc(priv->rx_ring_size, |
| 205 | sizeof(*priv->rx_skbuff), | 205 | sizeof(*priv->rx_skbuff), |
| 206 | GFP_KERNEL); | 206 | GFP_KERNEL); |
| 207 | if (!priv->rx_skbuff) | 207 | if (!priv->rx_skbuff) { |
| 208 | ret = -ENOMEM; | ||
| 208 | goto free_ucc_pram; | 209 | goto free_ucc_pram; |
| 210 | } | ||
| 209 | 211 | ||
| 210 | priv->tx_skbuff = kcalloc(priv->tx_ring_size, | 212 | priv->tx_skbuff = kcalloc(priv->tx_ring_size, |
| 211 | sizeof(*priv->tx_skbuff), | 213 | sizeof(*priv->tx_skbuff), |
| 212 | GFP_KERNEL); | 214 | GFP_KERNEL); |
| 213 | if (!priv->tx_skbuff) | 215 | if (!priv->tx_skbuff) { |
| 216 | ret = -ENOMEM; | ||
| 214 | goto free_rx_skbuff; | 217 | goto free_rx_skbuff; |
| 218 | } | ||
| 215 | 219 | ||
| 216 | priv->skb_curtx = 0; | 220 | priv->skb_curtx = 0; |
| 217 | priv->skb_dirtytx = 0; | 221 | priv->skb_dirtytx = 0; |
| @@ -292,7 +292,6 @@ static int lapbeth_open(struct net_device *dev) · 291 unmodified lines | |||
| 292 | return -ENODEV; | 292 | return -ENODEV; |
| 293 | } | 293 | } |
| 294 | 294 | ||
| 295 | netif_start_queue(dev); | ||
| 296 | return 0; | 295 | return 0; |
| 297 | } | 296 | } |
| 298 | 297 | ||
| @@ -300,8 +299,6 @@ static int lapbeth_close(struct net_device *dev) · 1 unmodified line | |||
| 300 | { | 299 | { |
| 301 | int err; | 300 | int err; |
| 302 | 301 | ||
| 303 | netif_stop_queue(dev); | ||
| 304 | |||
| 305 | if ((err = lapb_unregister(dev)) != LAPB_OK) | 302 | if ((err = lapb_unregister(dev)) != LAPB_OK) |
| 306 | pr_err("lapb_unregister error: %d\n", err); | 303 | pr_err("lapb_unregister error: %d\n", err); |
| 307 | 304 | ||
| @@ -5450,8 +5450,8 @@ ath11k_mac_op_assign_vif_chanctx(struct ieee80211_hw *hw, · 5449 unmodified lines | |||
| 5450 | } | 5450 | } |
| 5451 | 5451 | ||
| 5452 | if (ab->hw_params.vdev_start_delay && | 5452 | if (ab->hw_params.vdev_start_delay && |
| 5453 | (arvif->vdev_type == WMI_VDEV_TYPE_AP || | 5453 | arvif->vdev_type != WMI_VDEV_TYPE_AP && |
| 5454 | arvif->vdev_type == WMI_VDEV_TYPE_MONITOR)) { | 5454 | arvif->vdev_type != WMI_VDEV_TYPE_MONITOR) { |
| 5455 | param.vdev_id = arvif->vdev_id; | 5455 | param.vdev_id = arvif->vdev_id; |
| 5456 | param.peer_type = WMI_PEER_TYPE_DEFAULT; | 5456 | param.peer_type = WMI_PEER_TYPE_DEFAULT; |
| 5457 | param.peer_addr = ar->mac_addr; | 5457 | param.peer_addr = ar->mac_addr; |
| @@ -1687,8 +1687,8 @@ static int ath11k_qmi_respond_fw_mem_request(struct ath11k_base *ab) · 1686 unmodified lines | |||
| 1687 | req->mem_seg[i].size = ab->qmi.target_mem[i].size; | 1687 | req->mem_seg[i].size = ab->qmi.target_mem[i].size; |
| 1688 | req->mem_seg[i].type = ab->qmi.target_mem[i].type; | 1688 | req->mem_seg[i].type = ab->qmi.target_mem[i].type; |
| 1689 | ath11k_dbg(ab, ATH11K_DBG_QMI, | 1689 | ath11k_dbg(ab, ATH11K_DBG_QMI, |
| 1690 | "qmi req mem_seg[%d] 0x%llx %u %u\n", i, | 1690 | "qmi req mem_seg[%d] %pad %u %u\n", i, |
| 1691 | ab->qmi.target_mem[i].paddr, | 1691 | &ab->qmi.target_mem[i].paddr, |
| 1692 | ab->qmi.target_mem[i].size, | 1692 | ab->qmi.target_mem[i].size, |
| 1693 | ab->qmi.target_mem[i].type); | 1693 | ab->qmi.target_mem[i].type); |
| 1694 | } | 1694 | } |
| @@ -177,7 +177,8 @@ struct ath_frame_info { · 176 unmodified lines | |||
| 177 | s8 txq; | 177 | s8 txq; |
| 178 | u8 keyix; | 178 | u8 keyix; |
| 179 | u8 rtscts_rate; | 179 | u8 rtscts_rate; |
| 180 | u8 retries : 7; | 180 | u8 retries : 6; |
| 181 | u8 dyn_smps : 1; | ||
| 181 | u8 baw_tracked : 1; | 182 | u8 baw_tracked : 1; |
| 182 | u8 tx_power; | 183 | u8 tx_power; |
| 183 | enum ath9k_key_type keytype:2; | 184 | enum ath9k_key_type keytype:2; |
| @@ -1271,6 +1271,11 @@ static void ath_buf_set_rate(struct ath_softc *sc, struct ath_buf *bf, · 1270 unmodified lines | |||
| 1271 | is_40, is_sgi, is_sp); | 1271 | is_40, is_sgi, is_sp); |
| 1272 | if (rix < 8 && (tx_info->flags & IEEE80211_TX_CTL_STBC)) | 1272 | if (rix < 8 && (tx_info->flags & IEEE80211_TX_CTL_STBC)) |
| 1273 | info->rates[i].RateFlags |= ATH9K_RATESERIES_STBC; | 1273 | info->rates[i].RateFlags |= ATH9K_RATESERIES_STBC; |
| 1274 | if (rix >= 8 && fi->dyn_smps) { | ||
| 1275 | info->rates[i].RateFlags |= | ||
| 1276 | ATH9K_RATESERIES_RTS_CTS; | ||
| 1277 | info->flags |= ATH9K_TXDESC_CTSENA; | ||
| 1278 | } | ||
| 1274 | 1279 | ||
| 1275 | info->txpower[i] = ath_get_rate_txpower(sc, bf, rix, | 1280 | info->txpower[i] = ath_get_rate_txpower(sc, bf, rix, |
| 1276 | is_40, false); | 1281 | is_40, false); |
| @@ -2114,6 +2119,7 @@ static void setup_frame_info(struct ieee80211_hw *hw, · 837 unmodified lines | |||
| 2114 | fi->keyix = an->ps_key; | 2119 | fi->keyix = an->ps_key; |
| 2115 | else | 2120 | else |
| 2116 | fi->keyix = ATH9K_TXKEYIX_INVALID; | 2121 | fi->keyix = ATH9K_TXKEYIX_INVALID; |
| 2122 | fi->dyn_smps = sta && sta->smps_mode == IEEE80211_SMPS_DYNAMIC; | ||
| 2117 | fi->keytype = keytype; | 2123 | fi->keytype = keytype; |
| 2118 | fi->framelen = framelen; | 2124 | fi->framelen = framelen; |
| 2119 | fi->tx_power = txpower; | 2125 | fi->tx_power = txpower; |
| @@ -271,12 +271,12 @@ static int iwl_pnvm_get_from_efi(struct iwl_trans *trans, · 270 unmodified lines | |||
| 271 | err = efivar_entry_get(pnvm_efivar, NULL, &package_size, package); | 271 | err = efivar_entry_get(pnvm_efivar, NULL, &package_size, package); |
| 272 | if (err) { | 272 | if (err) { |
| 273 | IWL_DEBUG_FW(trans, | 273 | IWL_DEBUG_FW(trans, |
| 274 | "PNVM UEFI variable not found %d (len %zd)\n", | 274 | "PNVM UEFI variable not found %d (len %lu)\n", |
| 275 | err, package_size); | 275 | err, package_size); |
| 276 | goto out; | 276 | goto out; |
| 277 | } | 277 | } |
| 278 | 278 | ||
| 279 | IWL_DEBUG_FW(trans, "Read PNVM fro UEFI with size %zd\n", package_size); | 279 | IWL_DEBUG_FW(trans, "Read PNVM fro UEFI with size %lu\n", package_size); |
| 280 | 280 | ||
| 281 | *data = kmemdup(package->data, *len, GFP_KERNEL); | 281 | *data = kmemdup(package->data, *len, GFP_KERNEL); |
| 282 | if (!*data) | 282 | if (!*data) |
| @@ -205,6 +205,8 @@ static inline void iwl_op_mode_time_point(struct iwl_op_mode *op_mode, · 204 unmodified lines | |||
| 205 | enum iwl_fw_ini_time_point tp_id, | 205 | enum iwl_fw_ini_time_point tp_id, |
| 206 | union iwl_dbg_tlv_tp_data *tp_data) | 206 | union iwl_dbg_tlv_tp_data *tp_data) |
| 207 | { | 207 | { |
| 208 | if (!op_mode || !op_mode->ops || !op_mode->ops->time_point) | ||
| 209 | return; | ||
| 208 | op_mode->ops->time_point(op_mode, tp_id, tp_data); | 210 | op_mode->ops->time_point(op_mode, tp_id, tp_data); |
| 209 | } | 211 | } |
| 210 | 212 | ||
| @@ -1083,6 +1083,7 @@ static const struct dmi_system_id dmi_ppag_approved_list[] = { · 1082 unmodified lines | |||
| 1083 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTek COMPUTER INC."), | 1083 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTek COMPUTER INC."), |
| 1084 | }, | 1084 | }, |
| 1085 | }, | 1085 | }, |
| 1086 | {} | ||
| 1086 | }; | 1087 | }; |
| 1087 | 1088 | ||
| 1088 | static int iwl_mvm_ppag_init(struct iwl_mvm *mvm) | 1089 | static int iwl_mvm_ppag_init(struct iwl_mvm *mvm) |
| @@ -1106,6 +1106,8 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 1105 unmodified lines | |||
| 1106 | } | 1106 | } |
| 1107 | } | 1107 | } |
| 1108 | 1108 | ||
| 1109 | #if IS_ENABLED(CONFIG_IWLMVM) | ||
| 1110 | |||
| 1109 | /* | 1111 | /* |
| 1110 | * Workaround for problematic SnJ device: sometimes when | 1112 | * Workaround for problematic SnJ device: sometimes when |
| 1111 | * certain RF modules are connected to SnJ, the device ID | 1113 | * certain RF modules are connected to SnJ, the device ID |
| @@ -1116,7 +1118,6 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 4 unmodified lines | |||
| 1116 | if (CSR_HW_REV_TYPE(iwl_trans->hw_rev) == IWL_CFG_MAC_TYPE_SNJ) | 1118 | if (CSR_HW_REV_TYPE(iwl_trans->hw_rev) == IWL_CFG_MAC_TYPE_SNJ) |
| 1117 | iwl_trans->trans_cfg = &iwl_so_trans_cfg; | 1119 | iwl_trans->trans_cfg = &iwl_so_trans_cfg; |
| 1118 | 1120 | ||
| 1119 | #if IS_ENABLED(CONFIG_IWLMVM) | ||
| 1120 | /* | 1121 | /* |
| 1121 | * special-case 7265D, it has the same PCI IDs. | 1122 | * special-case 7265D, it has the same PCI IDs. |
| 1122 | * | 1123 | * |
| @@ -1129,6 +1129,8 @@ static int _iwl_pcie_rx_init(struct iwl_trans *trans) · 1128 unmodified lines | |||
| 1129 | 1129 | ||
| 1130 | iwl_pcie_rx_init_rxb_lists(rxq); | 1130 | iwl_pcie_rx_init_rxb_lists(rxq); |
| 1131 | 1131 | ||
| 1132 | spin_unlock_bh(&rxq->lock); | ||
| 1133 | |||
| 1132 | if (!rxq->napi.poll) { | 1134 | if (!rxq->napi.poll) { |
| 1133 | int (*poll)(struct napi_struct *, int) = iwl_pcie_napi_poll; | 1135 | int (*poll)(struct napi_struct *, int) = iwl_pcie_napi_poll; |
| 1134 | 1136 | ||
| @@ -1149,7 +1151,6 @@ static int _iwl_pcie_rx_init(struct iwl_trans *trans) · 14 unmodified lines | |||
| 1149 | napi_enable(&rxq->napi); | 1151 | napi_enable(&rxq->napi); |
| 1150 | } | 1152 | } |
| 1151 | 1153 | ||
| 1152 | spin_unlock_bh(&rxq->lock); | ||
| 1153 | } | 1154 | } |
| 1154 | 1155 | ||
| 1155 | /* move the pool to the default queue and allocator ownerships */ | 1156 | /* move the pool to the default queue and allocator ownerships */ |
| @@ -345,7 +345,6 @@ mt76_dma_tx_queue_skb(struct mt76_dev *dev, struct mt76_queue *q, · 344 unmodified lines | |||
| 345 | }; | 345 | }; |
| 346 | struct ieee80211_hw *hw; | 346 | struct ieee80211_hw *hw; |
| 347 | int len, n = 0, ret = -ENOMEM; | 347 | int len, n = 0, ret = -ENOMEM; |
| 348 | struct mt76_queue_entry e; | ||
| 349 | struct mt76_txwi_cache *t; | 348 | struct mt76_txwi_cache *t; |
| 350 | struct sk_buff *iter; | 349 | struct sk_buff *iter; |
| 351 | dma_addr_t addr; | 350 | dma_addr_t addr; |
| @@ -387,6 +386,11 @@ mt76_dma_tx_queue_skb(struct mt76_dev *dev, struct mt76_queue *q, · 35 unmodified lines | |||
| 387 | } | 386 | } |
| 388 | tx_info.nbuf = n; | 387 | tx_info.nbuf = n; |
| 389 | 388 | ||
| 389 | if (q->queued + (tx_info.nbuf + 1) / 2 >= q->ndesc - 1) { | ||
| 390 | ret = -ENOMEM; | ||
| 391 | goto unmap; | ||
| 392 | } | ||
| 393 | |||
| 390 | dma_sync_single_for_cpu(dev->dev, t->dma_addr, dev->drv->txwi_size, | 394 | dma_sync_single_for_cpu(dev->dev, t->dma_addr, dev->drv->txwi_size, |
| 391 | DMA_TO_DEVICE); | 395 | DMA_TO_DEVICE); |
| 392 | ret = dev->drv->tx_prepare_skb(dev, txwi, q->qid, wcid, sta, &tx_info); | 396 | ret = dev->drv->tx_prepare_skb(dev, txwi, q->qid, wcid, sta, &tx_info); |
| @@ -395,11 +399,6 @@ mt76_dma_tx_queue_skb(struct mt76_dev *dev, struct mt76_queue *q, · 2 unmodified lines | |||
| 395 | if (ret < 0) | 399 | if (ret < 0) |
| 396 | goto unmap; | 400 | goto unmap; |
| 397 | 401 | ||
| 398 | if (q->queued + (tx_info.nbuf + 1) / 2 >= q->ndesc - 1) { | ||
| 399 | ret = -ENOMEM; | ||
| 400 | goto unmap; | ||
| 401 | } | ||
| 402 | |||
| 403 | return mt76_dma_add_buf(dev, q, tx_info.buf, tx_info.nbuf, | 402 | return mt76_dma_add_buf(dev, q, tx_info.buf, tx_info.nbuf, |
| 404 | tx_info.info, tx_info.skb, t); | 403 | tx_info.info, tx_info.skb, t); |
| 405 | 404 | ||
| @@ -419,9 +418,7 @@ free: · 13 unmodified lines | |||
| 419 | } | 418 | } |
| 420 | #endif | 419 | #endif |
| 421 | 420 | ||
| 422 | e.skb = tx_info.skb; | 421 | dev_kfree_skb(tx_info.skb); |
| 423 | e.txwi = t; | ||
| 424 | dev->drv->tx_complete_skb(dev, &e); | ||
| 425 | mt76_put_txwi(dev, t); | 422 | mt76_put_txwi(dev, t); |
| 426 | return ret; | 423 | return ret; |
| 427 | } | 424 | } |
| @@ -515,13 +512,13 @@ mt76_add_fragment(struct mt76_dev *dev, struct mt76_queue *q, void *data, · 87 unmodified lines | |||
| 515 | { | 512 | { |
| 516 | struct sk_buff *skb = q->rx_head; | 513 | struct sk_buff *skb = q->rx_head; |
| 517 | struct skb_shared_info *shinfo = skb_shinfo(skb); | 514 | struct skb_shared_info *shinfo = skb_shinfo(skb); |
| 515 | int nr_frags = shinfo->nr_frags; | ||
| 518 | 516 | ||
| 519 | if (shinfo->nr_frags < ARRAY_SIZE(shinfo->frags)) { | 517 | if (nr_frags < ARRAY_SIZE(shinfo->frags)) { |
| 520 | struct page *page = virt_to_head_page(data); | 518 | struct page *page = virt_to_head_page(data); |
| 521 | int offset = data - page_address(page) + q->buf_offset; | 519 | int offset = data - page_address(page) + q->buf_offset; |
| 522 | 520 | ||
| 523 | skb_add_rx_frag(skb, shinfo->nr_frags, page, offset, len, | 521 | skb_add_rx_frag(skb, nr_frags, page, offset, len, q->buf_size); |
| 524 | q->buf_size); | ||
| 525 | } else { | 522 | } else { |
| 526 | skb_free_frag(data); | 523 | skb_free_frag(data); |
| 527 | } | 524 | } |
| @@ -530,7 +527,10 @@ mt76_add_fragment(struct mt76_dev *dev, struct mt76_queue *q, void *data, · 2 unmodified lines | |||
| 530 | return; | 527 | return; |
| 531 | 528 | ||
| 532 | q->rx_head = NULL; | 529 | q->rx_head = NULL; |
| 533 | dev->drv->rx_skb(dev, q - dev->q_rx, skb); | 530 | if (nr_frags < ARRAY_SIZE(shinfo->frags)) |
| 531 | dev->drv->rx_skb(dev, q - dev->q_rx, skb); | ||
| 532 | else | ||
| 533 | dev_kfree_skb(skb); | ||
| 534 | } | 534 | } |
| 535 | 535 | ||
| 536 | static int | 536 | static int |
| @@ -967,11 +967,6 @@ int mt7915_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, · 966 unmodified lines | |||
| 967 | } | 967 | } |
| 968 | txp->nbuf = nbuf; | 968 | txp->nbuf = nbuf; |
| 969 | 969 | ||
| 970 | /* pass partial skb header to fw */ | ||
| 971 | tx_info->buf[1].len = MT_CT_PARSE_LEN; | ||
| 972 | tx_info->buf[1].skip_unmap = true; | ||
| 973 | tx_info->nbuf = MT_CT_DMA_BUF_NUM; | ||
| 974 | |||
| 975 | txp->flags = cpu_to_le16(MT_CT_INFO_APPLY_TXD | MT_CT_INFO_FROM_HOST); | 970 | txp->flags = cpu_to_le16(MT_CT_INFO_APPLY_TXD | MT_CT_INFO_FROM_HOST); |
| 976 | 971 | ||
| 977 | if (!key) | 972 | if (!key) |
| @@ -1009,6 +1004,11 @@ int mt7915_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, · 31 unmodified lines | |||
| 1009 | txp->rept_wds_wcid = cpu_to_le16(0x3ff); | 1004 | txp->rept_wds_wcid = cpu_to_le16(0x3ff); |
| 1010 | tx_info->skb = DMA_DUMMY_DATA; | 1005 | tx_info->skb = DMA_DUMMY_DATA; |
| 1011 | 1006 | ||
| 1007 | /* pass partial skb header to fw */ | ||
| 1008 | tx_info->buf[1].len = MT_CT_PARSE_LEN; | ||
| 1009 | tx_info->buf[1].skip_unmap = true; | ||
| 1010 | tx_info->nbuf = MT_CT_DMA_BUF_NUM; | ||
| 1011 | |||
| 1012 | return 0; | 1012 | return 0; |
| 1013 | } | 1013 | } |
| 1014 | 1014 | ||
| @@ -543,7 +543,7 @@ mt7915_tm_set_tx_cont(struct mt7915_phy *phy, bool en) · 542 unmodified lines | |||
| 543 | tx_cont->bw = CMD_CBW_20MHZ; | 543 | tx_cont->bw = CMD_CBW_20MHZ; |
| 544 | break; | 544 | break; |
| 545 | default: | 545 | default: |
| 546 | break; | 546 | return -EINVAL; |
| 547 | } | 547 | } |
| 548 | 548 | ||
| 549 | if (!en) { | 549 | if (!en) { |
| @@ -591,7 +591,7 @@ mt7915_tm_set_tx_cont(struct mt7915_phy *phy, bool en) · 41 unmodified lines | |||
| 591 | mode = MT_PHY_TYPE_HE_MU; | 591 | mode = MT_PHY_TYPE_HE_MU; |
| 592 | break; | 592 | break; |
| 593 | default: | 593 | default: |
| 594 | break; | 594 | return -EINVAL; |
| 595 | } | 595 | } |
| 596 | 596 | ||
| 597 | rateval = mode << 6 | rate_idx; | 597 | rateval = mode << 6 | rate_idx; |
| @@ -405,10 +405,8 @@ mt7921_mcu_tx_rate_report(struct mt7921_dev *dev, struct sk_buff *skb, · 404 unmodified lines | |||
| 405 | if (wlan_idx >= MT76_N_WCIDS) | 405 | if (wlan_idx >= MT76_N_WCIDS) |
| 406 | return; | 406 | return; |
| 407 | wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); | 407 | wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); |
| 408 | if (!wcid) { | 408 | if (!wcid) |
| 409 | stats->tx_rate = rate; | ||
| 410 | return; | 409 | return; |
| 411 | } | ||
| 412 | 410 | ||
| 413 | msta = container_of(wcid, struct mt7921_sta, wcid); | 411 | msta = container_of(wcid, struct mt7921_sta, wcid); |
| 414 | stats = &msta->stats; | 412 | stats = &msta->stats; |
| @@ -557,8 +557,8 @@ check_frags: · 556 unmodified lines | |||
| 557 | } | 557 | } |
| 558 | 558 | ||
| 559 | if (skb_has_frag_list(skb) && !first_shinfo) { | 559 | if (skb_has_frag_list(skb) && !first_shinfo) { |
| 560 | first_shinfo = skb_shinfo(skb); | 560 | first_shinfo = shinfo; |
| 561 | shinfo = skb_shinfo(skb_shinfo(skb)->frag_list); | 561 | shinfo = skb_shinfo(shinfo->frag_list); |
| 562 | nr_frags = shinfo->nr_frags; | 562 | nr_frags = shinfo->nr_frags; |
| 563 | 563 | ||
| 564 | goto check_frags; | 564 | goto check_frags; |
| @@ -380,6 +380,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved) · 379 unmodified lines | |||
| 380 | return true; | 380 | return true; |
| 381 | 381 | ||
| 382 | nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD; | 382 | nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD; |
| 383 | nvme_req(req)->flags |= NVME_REQ_CANCELLED; | ||
| 383 | blk_mq_complete_request(req); | 384 | blk_mq_complete_request(req); |
| 384 | return true; | 385 | return true; |
| 385 | } | 386 | } |
| @@ -1440,7 +1441,7 @@ static int nvme_identify_ns(struct nvme_ctrl *ctrl, unsigned nsid, · 1054 unmodified lines | |||
| 1440 | goto out_free_id; | 1441 | goto out_free_id; |
| 1441 | } | 1442 | } |
| 1442 | 1443 | ||
| 1443 | error = -ENODEV; | 1444 | error = NVME_SC_INVALID_NS | NVME_SC_DNR; |
| 1444 | if ((*id)->ncap == 0) /* namespace not allocated or attached */ | 1445 | if ((*id)->ncap == 0) /* namespace not allocated or attached */ |
| 1445 | goto out_free_id; | 1446 | goto out_free_id; |
| 1446 | 1447 | ||
| @@ -4038,7 +4039,7 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid) · 2591 unmodified lines | |||
| 4038 | static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids) | 4039 | static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids) |
| 4039 | { | 4040 | { |
| 4040 | struct nvme_id_ns *id; | 4041 | struct nvme_id_ns *id; |
| 4041 | int ret = -ENODEV; | 4042 | int ret = NVME_SC_INVALID_NS | NVME_SC_DNR; |
| 4042 | 4043 | ||
| 4043 | if (test_bit(NVME_NS_DEAD, &ns->flags)) | 4044 | if (test_bit(NVME_NS_DEAD, &ns->flags)) |
| 4044 | goto out; | 4045 | goto out; |
| @@ -4047,7 +4048,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids) · 2 unmodified lines | |||
| 4047 | if (ret) | 4048 | if (ret) |
| 4048 | goto out; | 4049 | goto out; |
| 4049 | 4050 | ||
| 4050 | ret = -ENODEV; | 4051 | ret = NVME_SC_INVALID_NS | NVME_SC_DNR; |
| 4051 | if (!nvme_ns_ids_equal(&ns->head->ids, ids)) { | 4052 | if (!nvme_ns_ids_equal(&ns->head->ids, ids)) { |
| 4052 | dev_err(ns->ctrl->device, | 4053 | dev_err(ns->ctrl->device, |
| 4053 | "identifiers changed for nsid %d\n", ns->head->ns_id); | 4054 | "identifiers changed for nsid %d\n", ns->head->ns_id); |
| @@ -4065,7 +4066,7 @@ out: · 11 unmodified lines | |||
| 4065 | * | 4066 | * |
| 4066 | * TODO: we should probably schedule a delayed retry here. | 4067 | * TODO: we should probably schedule a delayed retry here. |
| 4067 | */ | 4068 | */ |
| 4068 | if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR))) | 4069 | if (ret > 0 && (ret & NVME_SC_DNR)) |
| 4069 | nvme_ns_remove(ns); | 4070 | nvme_ns_remove(ns); |
| 4070 | } | 4071 | } |
| 4071 | 4072 | ||
| @@ -4095,6 +4096,12 @@ static void nvme_validate_or_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) · 23 unmodified lines | |||
| 4095 | nsid); | 4096 | nsid); |
| 4096 | break; | 4097 | break; |
| 4097 | } | 4098 | } |
| 4099 | if (!nvme_multi_css(ctrl)) { | ||
| 4100 | dev_warn(ctrl->device, | ||
| 4101 | "command set not reported for nsid: %d\n", | ||
| 4102 | nsid); | ||
| 4103 | break; | ||
| 4104 | } | ||
| 4098 | nvme_alloc_ns(ctrl, nsid, &ids); | 4105 | nvme_alloc_ns(ctrl, nsid, &ids); |
| 4099 | break; | 4106 | break; |
| 4100 | default: | 4107 | default: |
| @@ -1956,7 +1956,7 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req) · 1955 unmodified lines | |||
| 1956 | sizeof(op->rsp_iu), DMA_FROM_DEVICE); | 1956 | sizeof(op->rsp_iu), DMA_FROM_DEVICE); |
| 1957 | 1957 | ||
| 1958 | if (opstate == FCPOP_STATE_ABORTED) | 1958 | if (opstate == FCPOP_STATE_ABORTED) |
| 1959 | status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1); | 1959 | status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1); |
| 1960 | else if (freq->status) { | 1960 | else if (freq->status) { |
| 1961 | status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1); | 1961 | status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1); |
| 1962 | dev_info(ctrl->ctrl.device, | 1962 | dev_info(ctrl->ctrl.device, |
| @@ -2055,7 +2055,7 @@ done: · 92 unmodified lines | |||
| 2055 | nvme_fc_complete_rq(rq); | 2055 | nvme_fc_complete_rq(rq); |
| 2056 | 2056 | ||
| 2057 | check_error: | 2057 | check_error: |
| 2058 | if (terminate_assoc) | 2058 | if (terminate_assoc && ctrl->ctrl.state != NVME_CTRL_RESETTING) |
| 2059 | queue_work(nvme_reset_wq, &ctrl->ioerr_work); | 2059 | queue_work(nvme_reset_wq, &ctrl->ioerr_work); |
| 2060 | } | 2060 | } |
| 2061 | 2061 | ||
| @@ -2443,6 +2443,7 @@ nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved) · 381 unmodified lines | |||
| 2443 | struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl); | 2443 | struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl); |
| 2444 | struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req); | 2444 | struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req); |
| 2445 | 2445 | ||
| 2446 | op->nreq.flags |= NVME_REQ_CANCELLED; | ||
| 2446 | __nvme_fc_abort_op(ctrl, op); | 2447 | __nvme_fc_abort_op(ctrl, op); |
| 2447 | return true; | 2448 | return true; |
| 2448 | } | 2449 | } |
| @@ -3246,6 +3246,7 @@ static const struct pci_device_id nvme_id_table[] = { · 3245 unmodified lines | |||
| 3246 | .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, }, | 3246 | .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, }, |
| 3247 | { PCI_DEVICE(0x144d, 0xa822), /* Samsung PM1725a */ | 3247 | { PCI_DEVICE(0x144d, 0xa822), /* Samsung PM1725a */ |
| 3248 | .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY | | 3248 | .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY | |
| 3249 | NVME_QUIRK_DISABLE_WRITE_ZEROES| | ||
| 3249 | NVME_QUIRK_IGNORE_DEV_SUBNQN, }, | 3250 | NVME_QUIRK_IGNORE_DEV_SUBNQN, }, |
| 3250 | { PCI_DEVICE(0x1987, 0x5016), /* Phison E16 */ | 3251 | { PCI_DEVICE(0x1987, 0x5016), /* Phison E16 */ |
| 3251 | .driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, }, | 3252 | .driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, }, |
| @@ -9,7 +9,13 @@ · 8 unmodified lines | |||
| 9 | 9 | ||
| 10 | int nvme_revalidate_zones(struct nvme_ns *ns) | 10 | int nvme_revalidate_zones(struct nvme_ns *ns) |
| 11 | { | 11 | { |
| 12 | return blk_revalidate_disk_zones(ns->disk, NULL); | 12 | struct request_queue *q = ns->queue; |
| 13 | int ret; | ||
| 14 | |||
| 15 | ret = blk_revalidate_disk_zones(ns->disk, NULL); | ||
| 16 | if (!ret) | ||
| 17 | blk_queue_max_zone_append_sectors(q, ns->ctrl->max_zone_append); | ||
| 18 | return ret; | ||
| 13 | } | 19 | } |
| 14 | 20 | ||
| 15 | static int nvme_set_max_append(struct nvme_ctrl *ctrl) | 21 | static int nvme_set_max_append(struct nvme_ctrl *ctrl) |
| @@ -107,7 +113,6 @@ int nvme_update_zone_info(struct nvme_ns *ns, unsigned lbaf) · 91 unmodified lines | |||
| 107 | blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q); | 113 | blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q); |
| 108 | blk_queue_max_open_zones(q, le32_to_cpu(id->mor) + 1); | 114 | blk_queue_max_open_zones(q, le32_to_cpu(id->mor) + 1); |
| 109 | blk_queue_max_active_zones(q, le32_to_cpu(id->mar) + 1); | 115 | blk_queue_max_active_zones(q, le32_to_cpu(id->mar) + 1); |
| 110 | blk_queue_max_zone_append_sectors(q, ns->ctrl->max_zone_append); | ||
| 111 | free_data: | 116 | free_data: |
| 112 | kfree(id); | 117 | kfree(id); |
| 113 | return status; | 118 | return status; |
| @@ -50,9 +50,9 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req) · 49 unmodified lines | |||
| 50 | 50 | ||
| 51 | /* | 51 | /* |
| 52 | * nvmet_passthru_map_sg is limitted to using a single bio so limit | 52 | * nvmet_passthru_map_sg is limitted to using a single bio so limit |
| 53 | * the mdts based on BIO_MAX_PAGES as well | 53 | * the mdts based on BIO_MAX_VECS as well |
| 54 | */ | 54 | */ |
| 55 | max_hw_sectors = min_not_zero(BIO_MAX_PAGES << (PAGE_SHIFT - 9), | 55 | max_hw_sectors = min_not_zero(BIO_MAX_VECS << (PAGE_SHIFT - 9), |
| 56 | max_hw_sectors); | 56 | max_hw_sectors); |
| 57 | 57 | ||
| 58 | page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12; | 58 | page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12; |
| @@ -191,7 +191,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq) · 132 unmodified lines | |||
| 191 | struct bio *bio; | 191 | struct bio *bio; |
| 192 | int i; | 192 | int i; |
| 193 | 193 | ||
| 194 | if (req->sg_cnt > BIO_MAX_PAGES) | 194 | if (req->sg_cnt > BIO_MAX_VECS) |
| 195 | return -EINVAL; | 195 | return -EINVAL; |
| 196 | 196 | ||
| 197 | if (req->transfer_len <= NVMET_MAX_INLINE_DATA_LEN) { | 197 | if (req->transfer_len <= NVMET_MAX_INLINE_DATA_LEN) { |
| @@ -802,9 +802,8 @@ static void nvmet_rdma_write_data_done(struct ib_cq *cq, struct ib_wc *wc) · 801 unmodified lines | |||
| 802 | nvmet_req_uninit(&rsp->req); | 802 | nvmet_req_uninit(&rsp->req); |
| 803 | nvmet_rdma_release_rsp(rsp); | 803 | nvmet_rdma_release_rsp(rsp); |
| 804 | if (wc->status != IB_WC_WR_FLUSH_ERR) { | 804 | if (wc->status != IB_WC_WR_FLUSH_ERR) { |
| 805 | pr_info("RDMA WRITE for CQE 0x%p failed with status %s (%d).\n", | 805 | pr_info("RDMA WRITE for CQE failed with status %s (%d).\n", |
| 806 | wc->wr_cqe, ib_wc_status_msg(wc->status), | 806 | ib_wc_status_msg(wc->status), wc->status); |
| 807 | wc->status); | ||
| 808 | nvmet_rdma_error_comp(queue); | 807 | nvmet_rdma_error_comp(queue); |
| 809 | } | 808 | } |
| 810 | return; | 809 | return; |
| @@ -1492,7 +1492,11 @@ static struct dev_pm_opp *_opp_get_next(struct opp_table *opp_table, · 1491 unmodified lines | |||
| 1492 | 1492 | ||
| 1493 | mutex_lock(&opp_table->lock); | 1493 | mutex_lock(&opp_table->lock); |
| 1494 | list_for_each_entry(temp, &opp_table->opp_list, node) { | 1494 | list_for_each_entry(temp, &opp_table->opp_list, node) { |
| 1495 | if (dynamic == temp->dynamic) { | 1495 | /* |
| 1496 | * Refcount must be dropped only once for each OPP by OPP core, | ||
| 1497 | * do that with help of "removed" flag. | ||
| 1498 | */ | ||
| 1499 | if (!temp->removed && dynamic == temp->dynamic) { | ||
| 1496 | opp = temp; | 1500 | opp = temp; |
| 1497 | break; | 1501 | break; |
| 1498 | } | 1502 | } |
| @@ -1502,10 +1506,27 @@ static struct dev_pm_opp *_opp_get_next(struct opp_table *opp_table, · 3 unmodified lines | |||
| 1502 | return opp; | 1506 | return opp; |
| 1503 | } | 1507 | } |
| 1504 | 1508 | ||
| 1505 | bool _opp_remove_all_static(struct opp_table *opp_table) | 1509 | /* |
| 1510 | * Can't call dev_pm_opp_put() from under the lock as debugfs removal needs to | ||
| 1511 | * happen lock less to avoid circular dependency issues. This routine must be | ||
| 1512 | * called without the opp_table->lock held. | ||
| 1513 | */ | ||
| 1514 | static void _opp_remove_all(struct opp_table *opp_table, bool dynamic) | ||
| 1506 | { | 1515 | { |
| 1507 | struct dev_pm_opp *opp; | 1516 | struct dev_pm_opp *opp; |
| 1508 | 1517 | ||
| 1518 | while ((opp = _opp_get_next(opp_table, dynamic))) { | ||
| 1519 | opp->removed = true; | ||
| 1520 | dev_pm_opp_put(opp); | ||
| 1521 | |||
| 1522 | /* Drop the references taken by dev_pm_opp_add() */ | ||
| 1523 | if (dynamic) | ||
| 1524 | dev_pm_opp_put_opp_table(opp_table); | ||
| 1525 | } | ||
| 1526 | } | ||
| 1527 | |||
| 1528 | bool _opp_remove_all_static(struct opp_table *opp_table) | ||
| 1529 | { | ||
| 1509 | mutex_lock(&opp_table->lock); | 1530 | mutex_lock(&opp_table->lock); |
| 1510 | 1531 | ||
| 1511 | if (!opp_table->parsed_static_opps) { | 1532 | if (!opp_table->parsed_static_opps) { |
| @@ -1520,13 +1541,7 @@ bool _opp_remove_all_static(struct opp_table *opp_table) · 8 unmodified lines | |||
| 1520 | 1541 | ||
| 1521 | mutex_unlock(&opp_table->lock); | 1542 | mutex_unlock(&opp_table->lock); |
| 1522 | 1543 | ||
| 1523 | /* | 1544 | _opp_remove_all(opp_table, false); |
| 1524 | * Can't remove the OPP from under the lock, debugfs removal needs to | ||
| 1525 | * happen lock less to avoid circular dependency issues. | ||
| 1526 | */ | ||
| 1527 | while ((opp = _opp_get_next(opp_table, false))) | ||
| 1528 | dev_pm_opp_put(opp); | ||
| 1529 | |||
| 1530 | return true; | 1545 | return true; |
| 1531 | } | 1546 | } |
| 1532 | 1547 | ||
| @@ -1539,25 +1554,12 @@ bool _opp_remove_all_static(struct opp_table *opp_table) · 6 unmodified lines | |||
| 1539 | void dev_pm_opp_remove_all_dynamic(struct device *dev) | 1554 | void dev_pm_opp_remove_all_dynamic(struct device *dev) |
| 1540 | { | 1555 | { |
| 1541 | struct opp_table *opp_table; | 1556 | struct opp_table *opp_table; |
| 1542 | struct dev_pm_opp *opp; | ||
| 1543 | int count = 0; | ||
| 1544 | 1557 | ||
| 1545 | opp_table = _find_opp_table(dev); | 1558 | opp_table = _find_opp_table(dev); |
| 1546 | if (IS_ERR(opp_table)) | 1559 | if (IS_ERR(opp_table)) |
| 1547 | return; | 1560 | return; |
| 1548 | 1561 | ||
| 1549 | /* | 1562 | _opp_remove_all(opp_table, true); |
| 1550 | * Can't remove the OPP from under the lock, debugfs removal needs to | ||
| 1551 | * happen lock less to avoid circular dependency issues. | ||
| 1552 | */ | ||
| 1553 | while ((opp = _opp_get_next(opp_table, true))) { | ||
| 1554 | dev_pm_opp_put(opp); | ||
| 1555 | count++; | ||
| 1556 | } | ||
| 1557 | |||
| 1558 | /* Drop the references taken by dev_pm_opp_add() */ | ||
| 1559 | while (count--) | ||
| 1560 | dev_pm_opp_put_opp_table(opp_table); | ||
| 1561 | 1563 | ||
| 1562 | /* Drop the reference taken by _find_opp_table() */ | 1564 | /* Drop the reference taken by _find_opp_table() */ |
| 1563 | dev_pm_opp_put_opp_table(opp_table); | 1565 | dev_pm_opp_put_opp_table(opp_table); |
| @@ -56,6 +56,7 @@ extern struct list_head opp_tables, lazy_opp_tables; · 55 unmodified lines | |||
| 56 | * @dynamic: not-created from static DT entries. | 56 | * @dynamic: not-created from static DT entries. |
| 57 | * @turbo: true if turbo (boost) OPP | 57 | * @turbo: true if turbo (boost) OPP |
| 58 | * @suspend: true if suspend OPP | 58 | * @suspend: true if suspend OPP |
| 59 | * @removed: flag indicating that OPP's reference is dropped by OPP core. | ||
| 59 | * @pstate: Device's power domain's performance state. | 60 | * @pstate: Device's power domain's performance state. |
| 60 | * @rate: Frequency in hertz | 61 | * @rate: Frequency in hertz |
| 61 | * @level: Performance level | 62 | * @level: Performance level |
| @@ -78,6 +79,7 @@ struct dev_pm_opp { · 16 unmodified lines | |||
| 78 | bool dynamic; | 79 | bool dynamic; |
| 79 | bool turbo; | 80 | bool turbo; |
| 80 | bool suspend; | 81 | bool suspend; |
| 82 | bool removed; | ||
| 81 | unsigned int pstate; | 83 | unsigned int pstate; |
| 82 | unsigned long rate; | 84 | unsigned long rate; |
| 83 | unsigned int level; | 85 | unsigned int level; |
| @@ -26,7 +26,7 @@ · 25 unmodified lines | |||
| 26 | #include <xen/platform_pci.h> | 26 | #include <xen/platform_pci.h> |
| 27 | 27 | ||
| 28 | #include <asm/xen/swiotlb-xen.h> | 28 | #include <asm/xen/swiotlb-xen.h> |
| 29 | #define INVALID_GRANT_REF (0) | 29 | |
| 30 | #define INVALID_EVTCHN (-1) | 30 | #define INVALID_EVTCHN (-1) |
| 31 | 31 | ||
| 32 | struct pci_bus_entry { | 32 | struct pci_bus_entry { |
| @@ -42,7 +42,7 @@ struct pcifront_device { · 9 unmodified lines | |||
| 42 | struct list_head root_buses; | 42 | struct list_head root_buses; |
| 43 | 43 | ||
| 44 | int evtchn; | 44 | int evtchn; |
| 45 | int gnt_ref; | 45 | grant_ref_t gnt_ref; |
| 46 | 46 | ||
| 47 | int irq; | 47 | int irq; |
| 48 | 48 | ||
| @@ -681,6 +681,7 @@ static int dmc620_pmu_device_probe(struct platform_device *pdev) · 680 unmodified lines | |||
| 681 | if (!name) { | 681 | if (!name) { |
| 682 | dev_err(&pdev->dev, | 682 | dev_err(&pdev->dev, |
| 683 | "Create name failed, PMU @%pa\n", &res->start); | 683 | "Create name failed, PMU @%pa\n", &res->start); |
| 684 | ret = -ENOMEM; | ||
| 684 | goto out_teardown_dev; | 685 | goto out_teardown_dev; |
| 685 | } | 686 | } |
| 686 | 687 | ||
| @@ -41,7 +41,7 @@ struct mt6315_chip { · 40 unmodified lines | |||
| 41 | .type = REGULATOR_VOLTAGE, \ | 41 | .type = REGULATOR_VOLTAGE, \ |
| 42 | .id = _bid, \ | 42 | .id = _bid, \ |
| 43 | .owner = THIS_MODULE, \ | 43 | .owner = THIS_MODULE, \ |
| 44 | .n_voltages = 0xbf, \ | 44 | .n_voltages = 0xc0, \ |
| 45 | .linear_ranges = mt_volt_range1, \ | 45 | .linear_ranges = mt_volt_range1, \ |
| 46 | .n_linear_ranges = ARRAY_SIZE(mt_volt_range1), \ | 46 | .n_linear_ranges = ARRAY_SIZE(mt_volt_range1), \ |
| 47 | .vsel_reg = _vsel, \ | 47 | .vsel_reg = _vsel, \ |
| @@ -69,7 +69,7 @@ static unsigned int mt6315_map_mode(u32 mode) · 21 unmodified lines | |||
| 69 | case MT6315_BUCK_MODE_LP: | 69 | case MT6315_BUCK_MODE_LP: |
| 70 | return REGULATOR_MODE_IDLE; | 70 | return REGULATOR_MODE_IDLE; |
| 71 | default: | 71 | default: |
| 72 | return -EINVAL; | 72 | return REGULATOR_MODE_INVALID; |
| 73 | } | 73 | } |
| 74 | } | 74 | } |
| 75 | 75 | ||
| @@ -797,6 +797,14 @@ static int pca9450_i2c_probe(struct i2c_client *i2c, · 796 unmodified lines | |||
| 797 | return ret; | 797 | return ret; |
| 798 | } | 798 | } |
| 799 | 799 | ||
| 800 | /* Clear PRESET_EN bit in BUCK123_DVS to use DVS registers */ | ||
| 801 | ret = regmap_clear_bits(pca9450->regmap, PCA9450_REG_BUCK123_DVS, | ||
| 802 | BUCK123_PRESET_EN); | ||
| 803 | if (ret) { | ||
| 804 | dev_err(&i2c->dev, "Failed to clear PRESET_EN bit: %d\n", ret); | ||
| 805 | return ret; | ||
| 806 | } | ||
| 807 | |||
| 800 | /* Set reset behavior on assertion of WDOG_B signal */ | 808 | /* Set reset behavior on assertion of WDOG_B signal */ |
| 801 | ret = regmap_update_bits(pca9450->regmap, PCA9450_REG_RESET_CTRL, | 809 | ret = regmap_update_bits(pca9450->regmap, PCA9450_REG_RESET_CTRL, |
| 802 | WDOG_B_CFG_MASK, WDOG_B_CFG_COLD_LDO12); | 810 | WDOG_B_CFG_MASK, WDOG_B_CFG_COLD_LDO12); |
| @@ -814,7 +822,7 @@ static int pca9450_i2c_probe(struct i2c_client *i2c, · 11 unmodified lines | |||
| 814 | 822 | ||
| 815 | if (IS_ERR(pca9450->sd_vsel_gpio)) { | 823 | if (IS_ERR(pca9450->sd_vsel_gpio)) { |
| 816 | dev_err(&i2c->dev, "Failed to get SD_VSEL GPIO\n"); | 824 | dev_err(&i2c->dev, "Failed to get SD_VSEL GPIO\n"); |
| 817 | return ret; | 825 | return PTR_ERR(pca9450->sd_vsel_gpio); |
| 818 | } | 826 | } |
| 819 | 827 | ||
| 820 | dev_info(&i2c->dev, "%s probed.\n", | 828 | dev_info(&i2c->dev, "%s probed.\n", |
| @@ -726,8 +726,8 @@ static const struct rpmh_vreg_hw_data pmic5_ftsmps510 = { · 725 unmodified lines | |||
| 726 | static const struct rpmh_vreg_hw_data pmic5_hfsmps515 = { | 726 | static const struct rpmh_vreg_hw_data pmic5_hfsmps515 = { |
| 727 | .regulator_type = VRM, | 727 | .regulator_type = VRM, |
| 728 | .ops = &rpmh_regulator_vrm_ops, | 728 | .ops = &rpmh_regulator_vrm_ops, |
| 729 | .voltage_range = REGULATOR_LINEAR_RANGE(2800000, 0, 4, 16000), | 729 | .voltage_range = REGULATOR_LINEAR_RANGE(320000, 0, 235, 16000), |
| 730 | .n_voltages = 5, | 730 | .n_voltages = 236, |
| 731 | .pmic_mode_map = pmic_mode_map_pmic5_smps, | 731 | .pmic_mode_map = pmic_mode_map_pmic5_smps, |
| 732 | .of_map_mode = rpmh_regulator_pmic4_smps_of_map_mode, | 732 | .of_map_mode = rpmh_regulator_pmic4_smps_of_map_mode, |
| 733 | }; | 733 | }; |
| @@ -901,7 +901,7 @@ static const struct rpmh_vreg_init_data pm8350_vreg_data[] = { · 167 unmodified lines | |||
| 901 | }; | 901 | }; |
| 902 | 902 | ||
| 903 | static const struct rpmh_vreg_init_data pm8350c_vreg_data[] = { | 903 | static const struct rpmh_vreg_init_data pm8350c_vreg_data[] = { |
| 904 | RPMH_VREG("smps1", "smp%s1", &pmic5_hfsmps510, "vdd-s1"), | 904 | RPMH_VREG("smps1", "smp%s1", &pmic5_hfsmps515, "vdd-s1"), |
| 905 | RPMH_VREG("smps2", "smp%s2", &pmic5_ftsmps510, "vdd-s2"), | 905 | RPMH_VREG("smps2", "smp%s2", &pmic5_ftsmps510, "vdd-s2"), |
| 906 | RPMH_VREG("smps3", "smp%s3", &pmic5_ftsmps510, "vdd-s3"), | 906 | RPMH_VREG("smps3", "smp%s3", &pmic5_ftsmps510, "vdd-s3"), |
| 907 | RPMH_VREG("smps4", "smp%s4", &pmic5_ftsmps510, "vdd-s4"), | 907 | RPMH_VREG("smps4", "smp%s4", &pmic5_ftsmps510, "vdd-s4"), |
| @@ -153,9 +153,9 @@ static int rt4831_regulator_probe(struct platform_device *pdev) · 152 unmodified lines | |||
| 153 | int i, ret; | 153 | int i, ret; |
| 154 | 154 | ||
| 155 | regmap = dev_get_regmap(pdev->dev.parent, NULL); | 155 | regmap = dev_get_regmap(pdev->dev.parent, NULL); |
| 156 | if (IS_ERR(regmap)) { | 156 | if (!regmap) { |
| 157 | dev_err(&pdev->dev, "Failed to init regmap\n"); | 157 | dev_err(&pdev->dev, "Failed to init regmap\n"); |
| 158 | return PTR_ERR(regmap); | 158 | return -ENODEV; |
| 159 | } | 159 | } |
| 160 | 160 | ||
| 161 | /* Configure DSV mode to normal by default */ | 161 | /* Configure DSV mode to normal by default */ |
| @@ -3052,7 +3052,8 @@ static blk_status_t do_dasd_request(struct blk_mq_hw_ctx *hctx, · 3051 unmodified lines | |||
| 3052 | 3052 | ||
| 3053 | basedev = block->base; | 3053 | basedev = block->base; |
| 3054 | spin_lock_irq(&dq->lock); | 3054 | spin_lock_irq(&dq->lock); |
| 3055 | if (basedev->state < DASD_STATE_READY) { | 3055 | if (basedev->state < DASD_STATE_READY || |
| 3056 | test_bit(DASD_FLAG_OFFLINE, &basedev->flags)) { | ||
| 3056 | DBF_DEV_EVENT(DBF_ERR, basedev, | 3057 | DBF_DEV_EVENT(DBF_ERR, basedev, |
| 3057 | "device not ready for request %p", req); | 3058 | "device not ready for request %p", req); |
| 3058 | rc = BLK_STS_IOERR; | 3059 | rc = BLK_STS_IOERR; |
| @@ -3487,8 +3488,6 @@ void dasd_generic_remove(struct ccw_device *cdev) · 428 unmodified lines | |||
| 3487 | struct dasd_device *device; | 3488 | struct dasd_device *device; |
| 3488 | struct dasd_block *block; | 3489 | struct dasd_block *block; |
| 3489 | 3490 | ||
| 3490 | cdev->handler = NULL; | ||
| 3491 | |||
| 3492 | device = dasd_device_from_cdev(cdev); | 3491 | device = dasd_device_from_cdev(cdev); |
| 3493 | if (IS_ERR(device)) { | 3492 | if (IS_ERR(device)) { |
| 3494 | dasd_remove_sysfs_files(cdev); | 3493 | dasd_remove_sysfs_files(cdev); |
| @@ -3507,6 +3506,7 @@ void dasd_generic_remove(struct ccw_device *cdev) · 12 unmodified lines | |||
| 3507 | * no quite down yet. | 3506 | * no quite down yet. |
| 3508 | */ | 3507 | */ |
| 3509 | dasd_set_target_state(device, DASD_STATE_NEW); | 3508 | dasd_set_target_state(device, DASD_STATE_NEW); |
| 3509 | cdev->handler = NULL; | ||
| 3510 | /* dasd_delete_device destroys the device reference. */ | 3510 | /* dasd_delete_device destroys the device reference. */ |
| 3511 | block = device->block; | 3511 | block = device->block; |
| 3512 | dasd_delete_device(device); | 3512 | dasd_delete_device(device); |
| @@ -424,8 +424,10 @@ tty3270_update(struct timer_list *t) · 423 unmodified lines | |||
| 424 | * last output position matches the start address | 424 | * last output position matches the start address |
| 425 | * of this line. | 425 | * of this line. |
| 426 | */ | 426 | */ |
| 427 | if (s->string[1] == sba[0] && s->string[2] == sba[1]) | 427 | if (s->string[1] == sba[0] && s->string[2] == sba[1]) { |
| 428 | str += 3, len -= 3; | 428 | str += 3; |
| 429 | len -= 3; | ||
| 430 | } | ||
| 429 | if (raw3270_request_add_data(wrq, str, len) != 0) | 431 | if (raw3270_request_add_data(wrq, str, len) != 0) |
| 430 | break; | 432 | break; |
| 431 | list_del_init(&s->update); | 433 | list_del_init(&s->update); |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | |||
| 15 | #include <linux/init.h> | 15 | #include <linux/init.h> |
| 16 | #include <linux/slab.h> | 16 | #include <linux/slab.h> |
| 17 | #include <linux/debugfs.h> | 17 | #include <linux/debugfs.h> |
| 18 | #include <linux/reboot.h> | ||
| 18 | 19 | ||
| 19 | #include <asm/asm-offsets.h> | 20 | #include <asm/asm-offsets.h> |
| 20 | #include <asm/ipl.h> | 21 | #include <asm/ipl.h> |
| @@ -238,6 +239,28 @@ static int __init zcore_reipl_init(void) · 217 unmodified lines | |||
| 238 | return 0; | 239 | return 0; |
| 239 | } | 240 | } |
| 240 | 241 | ||
| 242 | static int zcore_reboot_and_on_panic_handler(struct notifier_block *self, | ||
| 243 | unsigned long event, | ||
| 244 | void *data) | ||
| 245 | { | ||
| 246 | if (hsa_available) | ||
| 247 | release_hsa(); | ||
| 248 | |||
| 249 | return NOTIFY_OK; | ||
| 250 | } | ||
| 251 | |||
| 252 | static struct notifier_block zcore_reboot_notifier = { | ||
| 253 | .notifier_call = zcore_reboot_and_on_panic_handler, | ||
| 254 | /* we need to be notified before reipl and kdump */ | ||
| 255 | .priority = INT_MAX, | ||
| 256 | }; | ||
| 257 | |||
| 258 | static struct notifier_block zcore_on_panic_notifier = { | ||
| 259 | .notifier_call = zcore_reboot_and_on_panic_handler, | ||
| 260 | /* we need to be notified before reipl and kdump */ | ||
| 261 | .priority = INT_MAX, | ||
| 262 | }; | ||
| 263 | |||
| 241 | static int __init zcore_init(void) | 264 | static int __init zcore_init(void) |
| 242 | { | 265 | { |
| 243 | unsigned char arch; | 266 | unsigned char arch; |
| @@ -293,28 +316,15 @@ static int __init zcore_init(void) · 49 unmodified lines | |||
| 293 | goto fail; | 316 | goto fail; |
| 294 | 317 | ||
| 295 | zcore_dir = debugfs_create_dir("zcore" , NULL); | 318 | zcore_dir = debugfs_create_dir("zcore" , NULL); |
| 296 | if (!zcore_dir) { | ||
| 297 | rc = -ENOMEM; | ||
| 298 | goto fail; | ||
| 299 | } | ||
| 300 | zcore_reipl_file = debugfs_create_file("reipl", S_IRUSR, zcore_dir, | 319 | zcore_reipl_file = debugfs_create_file("reipl", S_IRUSR, zcore_dir, |
| 301 | NULL, &zcore_reipl_fops); | 320 | NULL, &zcore_reipl_fops); |
| 302 | if (!zcore_reipl_file) { | ||
| 303 | rc = -ENOMEM; | ||
| 304 | goto fail_dir; | ||
| 305 | } | ||
| 306 | zcore_hsa_file = debugfs_create_file("hsa", S_IRUSR|S_IWUSR, zcore_dir, | 321 | zcore_hsa_file = debugfs_create_file("hsa", S_IRUSR|S_IWUSR, zcore_dir, |
| 307 | NULL, &zcore_hsa_fops); | 322 | NULL, &zcore_hsa_fops); |
| 308 | if (!zcore_hsa_file) { | ||
| 309 | rc = -ENOMEM; | ||
| 310 | goto fail_reipl_file; | ||
| 311 | } | ||
| 312 | return 0; | ||
| 313 | 323 | ||
| 314 | fail_reipl_file: | 324 | register_reboot_notifier(&zcore_reboot_notifier); |
| 315 | debugfs_remove(zcore_reipl_file); | 325 | atomic_notifier_chain_register(&panic_notifier_list, &zcore_on_panic_notifier); |
| 316 | fail_dir: | 326 | |
| 317 | debugfs_remove(zcore_dir); | 327 | return 0; |
| 318 | fail: | 328 | fail: |
| 319 | diag308(DIAG308_REL_HSA, NULL); | 329 | diag308(DIAG308_REL_HSA, NULL); |
| 320 | return rc; | 330 | return rc; |
| @@ -47,7 +47,7 @@ static void ccw_timeout_log(struct ccw_device *cdev) · 46 unmodified lines | |||
| 47 | orb = &private->orb; | 47 | orb = &private->orb; |
| 48 | cc = stsch(sch->schid, &schib); | 48 | cc = stsch(sch->schid, &schib); |
| 49 | 49 | ||
| 50 | printk(KERN_WARNING "cio: ccw device timeout occurred at %llx, " | 50 | printk(KERN_WARNING "cio: ccw device timeout occurred at %lx, " |
| 51 | "device information:\n", get_tod_clock()); | 51 | "device information:\n", get_tod_clock()); |
| 52 | printk(KERN_WARNING "cio: orb:\n"); | 52 | printk(KERN_WARNING "cio: orb:\n"); |
| 53 | print_hex_dump(KERN_WARNING, "cio: ", DUMP_PREFIX_NONE, 16, 1, | 53 | print_hex_dump(KERN_WARNING, "cio: ", DUMP_PREFIX_NONE, 16, 1, |
| @@ -543,7 +543,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev, · 542 unmodified lines | |||
| 543 | if (ret) | 543 | if (ret) |
| 544 | return ret; | 544 | return ret; |
| 545 | 545 | ||
| 546 | return copy_to_user((void __user *)arg, &info, minsz); | 546 | return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; |
| 547 | } | 547 | } |
| 548 | case VFIO_DEVICE_GET_REGION_INFO: | 548 | case VFIO_DEVICE_GET_REGION_INFO: |
| 549 | { | 549 | { |
| @@ -561,7 +561,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev, · 11 unmodified lines | |||
| 561 | if (ret) | 561 | if (ret) |
| 562 | return ret; | 562 | return ret; |
| 563 | 563 | ||
| 564 | return copy_to_user((void __user *)arg, &info, minsz); | 564 | return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; |
| 565 | } | 565 | } |
| 566 | case VFIO_DEVICE_GET_IRQ_INFO: | 566 | case VFIO_DEVICE_GET_IRQ_INFO: |
| 567 | { | 567 | { |
| @@ -582,7 +582,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev, · 14 unmodified lines | |||
| 582 | if (info.count == -1) | 582 | if (info.count == -1) |
| 583 | return -EINVAL; | 583 | return -EINVAL; |
| 584 | 584 | ||
| 585 | return copy_to_user((void __user *)arg, &info, minsz); | 585 | return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; |
| 586 | } | 586 | } |
| 587 | case VFIO_DEVICE_SET_IRQS: | 587 | case VFIO_DEVICE_SET_IRQS: |
| 588 | { | 588 | { |
| @@ -1286,7 +1286,7 @@ static int vfio_ap_mdev_get_device_info(unsigned long arg) · 1285 unmodified lines | |||
| 1286 | info.num_regions = 0; | 1286 | info.num_regions = 0; |
| 1287 | info.num_irqs = 0; | 1287 | info.num_irqs = 0; |
| 1288 | 1288 | ||
| 1289 | return copy_to_user((void __user *)arg, &info, minsz); | 1289 | return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; |
| 1290 | } | 1290 | } |
| 1291 | 1291 | ||
| 1292 | static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev, | 1292 | static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev, |
| @@ -436,7 +436,7 @@ struct qeth_qdio_out_buffer { · 435 unmodified lines | |||
| 436 | int is_header[QDIO_MAX_ELEMENTS_PER_BUFFER]; | 436 | int is_header[QDIO_MAX_ELEMENTS_PER_BUFFER]; |
| 437 | 437 | ||
| 438 | struct qeth_qdio_out_q *q; | 438 | struct qeth_qdio_out_q *q; |
| 439 | struct qeth_qdio_out_buffer *next_pending; | 439 | struct list_head list_entry; |
| 440 | }; | 440 | }; |
| 441 | 441 | ||
| 442 | struct qeth_card; | 442 | struct qeth_card; |
| @@ -500,6 +500,7 @@ struct qeth_qdio_out_q { · 57 unmodified lines | |||
| 500 | struct qdio_buffer *qdio_bufs[QDIO_MAX_BUFFERS_PER_Q]; | 500 | struct qdio_buffer *qdio_bufs[QDIO_MAX_BUFFERS_PER_Q]; |
| 501 | struct qeth_qdio_out_buffer *bufs[QDIO_MAX_BUFFERS_PER_Q]; | 501 | struct qeth_qdio_out_buffer *bufs[QDIO_MAX_BUFFERS_PER_Q]; |
| 502 | struct qdio_outbuf_state *bufstates; /* convenience pointer */ | 502 | struct qdio_outbuf_state *bufstates; /* convenience pointer */ |
| 503 | struct list_head pending_bufs; | ||
| 503 | struct qeth_out_q_stats stats; | 504 | struct qeth_out_q_stats stats; |
| 504 | spinlock_t lock; | 505 | spinlock_t lock; |
| 505 | unsigned int priority; | 506 | unsigned int priority; |
| @@ -73,8 +73,6 @@ static void qeth_free_qdio_queues(struct qeth_card *card); · 72 unmodified lines | |||
| 73 | static void qeth_notify_skbs(struct qeth_qdio_out_q *queue, | 73 | static void qeth_notify_skbs(struct qeth_qdio_out_q *queue, |
| 74 | struct qeth_qdio_out_buffer *buf, | 74 | struct qeth_qdio_out_buffer *buf, |
| 75 | enum iucv_tx_notify notification); | 75 | enum iucv_tx_notify notification); |
| 76 | static void qeth_tx_complete_buf(struct qeth_qdio_out_buffer *buf, bool error, | ||
| 77 | int budget); | ||
| 78 | 76 | ||
| 79 | static void qeth_close_dev_handler(struct work_struct *work) | 77 | static void qeth_close_dev_handler(struct work_struct *work) |
| 80 | { | 78 | { |
| @@ -465,41 +463,6 @@ static enum iucv_tx_notify qeth_compute_cq_notification(int sbalf15, · 384 unmodified lines | |||
| 465 | return n; | 463 | return n; |
| 466 | } | 464 | } |
| 467 | 465 | ||
| 468 | static void qeth_cleanup_handled_pending(struct qeth_qdio_out_q *q, int bidx, | ||
| 469 | int forced_cleanup) | ||
| 470 | { | ||
| 471 | if (q->card->options.cq != QETH_CQ_ENABLED) | ||
| 472 | return; | ||
| 473 | |||
| 474 | if (q->bufs[bidx]->next_pending != NULL) { | ||
| 475 | struct qeth_qdio_out_buffer *head = q->bufs[bidx]; | ||
| 476 | struct qeth_qdio_out_buffer *c = q->bufs[bidx]->next_pending; | ||
| 477 | |||
| 478 | while (c) { | ||
| 479 | if (forced_cleanup || | ||
| 480 | atomic_read(&c->state) == QETH_QDIO_BUF_EMPTY) { | ||
| 481 | struct qeth_qdio_out_buffer *f = c; | ||
| 482 | |||
| 483 | QETH_CARD_TEXT(f->q->card, 5, "fp"); | ||
| 484 | QETH_CARD_TEXT_(f->q->card, 5, "%lx", (long) f); | ||
| 485 | /* release here to avoid interleaving between | ||
| 486 | outbound tasklet and inbound tasklet | ||
| 487 | regarding notifications and lifecycle */ | ||
| 488 | qeth_tx_complete_buf(c, forced_cleanup, 0); | ||
| 489 | |||
| 490 | c = f->next_pending; | ||
| 491 | WARN_ON_ONCE(head->next_pending != f); | ||
| 492 | head->next_pending = c; | ||
| 493 | kmem_cache_free(qeth_qdio_outbuf_cache, f); | ||
| 494 | } else { | ||
| 495 | head = c; | ||
| 496 | c = c->next_pending; | ||
| 497 | } | ||
| 498 | |||
| 499 | } | ||
| 500 | } | ||
| 501 | } | ||
| 502 | |||
| 503 | static void qeth_qdio_handle_aob(struct qeth_card *card, | 466 | static void qeth_qdio_handle_aob(struct qeth_card *card, |
| 504 | unsigned long phys_aob_addr) | 467 | unsigned long phys_aob_addr) |
| 505 | { | 468 | { |
| @@ -507,6 +470,7 @@ static void qeth_qdio_handle_aob(struct qeth_card *card, · 1 unmodified line | |||
| 507 | struct qaob *aob; | 470 | struct qaob *aob; |
| 508 | struct qeth_qdio_out_buffer *buffer; | 471 | struct qeth_qdio_out_buffer *buffer; |
| 509 | enum iucv_tx_notify notification; | 472 | enum iucv_tx_notify notification; |
| 473 | struct qeth_qdio_out_q *queue; | ||
| 510 | unsigned int i; | 474 | unsigned int i; |
| 511 | 475 | ||
| 512 | aob = (struct qaob *) phys_to_virt(phys_aob_addr); | 476 | aob = (struct qaob *) phys_to_virt(phys_aob_addr); |
| @@ -537,7 +501,7 @@ static void qeth_qdio_handle_aob(struct qeth_card *card, · 24 unmodified lines | |||
| 537 | qeth_notify_skbs(buffer->q, buffer, notification); | 501 | qeth_notify_skbs(buffer->q, buffer, notification); |
| 538 | 502 | ||
| 539 | /* Free dangling allocations. The attached skbs are handled by | 503 | /* Free dangling allocations. The attached skbs are handled by |
| 540 | * qeth_cleanup_handled_pending(). | 504 | * qeth_tx_complete_pending_bufs(). |
| 541 | */ | 505 | */ |
| 542 | for (i = 0; | 506 | for (i = 0; |
| 543 | i < aob->sb_count && i < QETH_MAX_BUFFER_ELEMENTS(card); | 507 | i < aob->sb_count && i < QETH_MAX_BUFFER_ELEMENTS(card); |
| @@ -549,7 +513,9 @@ static void qeth_qdio_handle_aob(struct qeth_card *card, · 5 unmodified lines | |||
| 549 | buffer->is_header[i] = 0; | 513 | buffer->is_header[i] = 0; |
| 550 | } | 514 | } |
| 551 | 515 | ||
| 516 | queue = buffer->q; | ||
| 552 | atomic_set(&buffer->state, QETH_QDIO_BUF_EMPTY); | 517 | atomic_set(&buffer->state, QETH_QDIO_BUF_EMPTY); |
| 518 | napi_schedule(&queue->napi); | ||
| 553 | break; | 519 | break; |
| 554 | default: | 520 | default: |
| 555 | WARN_ON_ONCE(1); | 521 | WARN_ON_ONCE(1); |
| @@ -1424,9 +1390,6 @@ static void qeth_tx_complete_buf(struct qeth_qdio_out_buffer *buf, bool error, · 868 unmodified lines | |||
| 1424 | struct qeth_qdio_out_q *queue = buf->q; | 1390 | struct qeth_qdio_out_q *queue = buf->q; |
| 1425 | struct sk_buff *skb; | 1391 | struct sk_buff *skb; |
| 1426 | 1392 | ||
| 1427 | if (atomic_read(&buf->state) == QETH_QDIO_BUF_PENDING) | ||
| 1428 | qeth_notify_skbs(queue, buf, TX_NOTIFY_GENERALERROR); | ||
| 1429 | |||
| 1430 | /* Empty buffer? */ | 1393 | /* Empty buffer? */ |
| 1431 | if (buf->next_element_to_fill == 0) | 1394 | if (buf->next_element_to_fill == 0) |
| 1432 | return; | 1395 | return; |
| @@ -1488,14 +1451,38 @@ static void qeth_clear_output_buffer(struct qeth_qdio_out_q *queue, · 55 unmodified lines | |||
| 1488 | atomic_set(&buf->state, QETH_QDIO_BUF_EMPTY); | 1451 | atomic_set(&buf->state, QETH_QDIO_BUF_EMPTY); |
| 1489 | } | 1452 | } |
| 1490 | 1453 | ||
| 1454 | static void qeth_tx_complete_pending_bufs(struct qeth_card *card, | ||
| 1455 | struct qeth_qdio_out_q *queue, | ||
| 1456 | bool drain) | ||
| 1457 | { | ||
| 1458 | struct qeth_qdio_out_buffer *buf, *tmp; | ||
| 1459 | |||
| 1460 | list_for_each_entry_safe(buf, tmp, &queue->pending_bufs, list_entry) { | ||
| 1461 | if (drain || atomic_read(&buf->state) == QETH_QDIO_BUF_EMPTY) { | ||
| 1462 | QETH_CARD_TEXT(card, 5, "fp"); | ||
| 1463 | QETH_CARD_TEXT_(card, 5, "%lx", (long) buf); | ||
| 1464 | |||
| 1465 | if (drain) | ||
| 1466 | qeth_notify_skbs(queue, buf, | ||
| 1467 | TX_NOTIFY_GENERALERROR); | ||
| 1468 | qeth_tx_complete_buf(buf, drain, 0); | ||
| 1469 | |||
| 1470 | list_del(&buf->list_entry); | ||
| 1471 | kmem_cache_free(qeth_qdio_outbuf_cache, buf); | ||
| 1472 | } | ||
| 1473 | } | ||
| 1474 | } | ||
| 1475 | |||
| 1491 | static void qeth_drain_output_queue(struct qeth_qdio_out_q *q, bool free) | 1476 | static void qeth_drain_output_queue(struct qeth_qdio_out_q *q, bool free) |
| 1492 | { | 1477 | { |
| 1493 | int j; | 1478 | int j; |
| 1494 | 1479 | ||
| 1480 | qeth_tx_complete_pending_bufs(q->card, q, true); | ||
| 1481 | |||
| 1495 | for (j = 0; j < QDIO_MAX_BUFFERS_PER_Q; ++j) { | 1482 | for (j = 0; j < QDIO_MAX_BUFFERS_PER_Q; ++j) { |
| 1496 | if (!q->bufs[j]) | 1483 | if (!q->bufs[j]) |
| 1497 | continue; | 1484 | continue; |
| 1498 | qeth_cleanup_handled_pending(q, j, 1); | 1485 | |
| 1499 | qeth_clear_output_buffer(q, q->bufs[j], true, 0); | 1486 | qeth_clear_output_buffer(q, q->bufs[j], true, 0); |
| 1500 | if (free) { | 1487 | if (free) { |
| 1501 | kmem_cache_free(qeth_qdio_outbuf_cache, q->bufs[j]); | 1488 | kmem_cache_free(qeth_qdio_outbuf_cache, q->bufs[j]); |
| @@ -2615,7 +2602,6 @@ static int qeth_init_qdio_out_buf(struct qeth_qdio_out_q *q, int bidx) · 1113 unmodified lines | |||
| 2615 | skb_queue_head_init(&newbuf->skb_list); | 2602 | skb_queue_head_init(&newbuf->skb_list); |
| 2616 | lockdep_set_class(&newbuf->skb_list.lock, &qdio_out_skb_queue_key); | 2603 | lockdep_set_class(&newbuf->skb_list.lock, &qdio_out_skb_queue_key); |
| 2617 | newbuf->q = q; | 2604 | newbuf->q = q; |
| 2618 | newbuf->next_pending = q->bufs[bidx]; | ||
| 2619 | atomic_set(&newbuf->state, QETH_QDIO_BUF_EMPTY); | 2605 | atomic_set(&newbuf->state, QETH_QDIO_BUF_EMPTY); |
| 2620 | q->bufs[bidx] = newbuf; | 2606 | q->bufs[bidx] = newbuf; |
| 2621 | return 0; | 2607 | return 0; |
| @@ -2634,15 +2620,28 @@ static void qeth_free_output_queue(struct qeth_qdio_out_q *q) · 12 unmodified lines | |||
| 2634 | static struct qeth_qdio_out_q *qeth_alloc_output_queue(void) | 2620 | static struct qeth_qdio_out_q *qeth_alloc_output_queue(void) |
| 2635 | { | 2621 | { |
| 2636 | struct qeth_qdio_out_q *q = kzalloc(sizeof(*q), GFP_KERNEL); | 2622 | struct qeth_qdio_out_q *q = kzalloc(sizeof(*q), GFP_KERNEL); |
| 2623 | unsigned int i; | ||
| 2637 | 2624 | ||
| 2638 | if (!q) | 2625 | if (!q) |
| 2639 | return NULL; | 2626 | return NULL; |
| 2640 | 2627 | ||
| 2641 | if (qdio_alloc_buffers(q->qdio_bufs, QDIO_MAX_BUFFERS_PER_Q)) { | 2628 | if (qdio_alloc_buffers(q->qdio_bufs, QDIO_MAX_BUFFERS_PER_Q)) |
| 2642 | kfree(q); | 2629 | goto err_qdio_bufs; |
| 2643 | return NULL; | 2630 | |
| 2631 | for (i = 0; i < QDIO_MAX_BUFFERS_PER_Q; i++) { | ||
| 2632 | if (qeth_init_qdio_out_buf(q, i)) | ||
| 2633 | goto err_out_bufs; | ||
| 2644 | } | 2634 | } |
| 2635 | |||
| 2645 | return q; | 2636 | return q; |
| 2637 | |||
| 2638 | err_out_bufs: | ||
| 2639 | while (i > 0) | ||
| 2640 | kmem_cache_free(qeth_qdio_outbuf_cache, q->bufs[--i]); | ||
| 2641 | qdio_free_buffers(q->qdio_bufs, QDIO_MAX_BUFFERS_PER_Q); | ||
| 2642 | err_qdio_bufs: | ||
| 2643 | kfree(q); | ||
| 2644 | return NULL; | ||
| 2646 | } | 2645 | } |
| 2647 | 2646 | ||
| 2648 | static void qeth_tx_completion_timer(struct timer_list *timer) | 2647 | static void qeth_tx_completion_timer(struct timer_list *timer) |
| @@ -2655,7 +2654,7 @@ static void qeth_tx_completion_timer(struct timer_list *timer) · 6 unmodified lines | |||
| 2655 | 2654 | ||
| 2656 | static int qeth_alloc_qdio_queues(struct qeth_card *card) | 2655 | static int qeth_alloc_qdio_queues(struct qeth_card *card) |
| 2657 | { | 2656 | { |
| 2658 | int i, j; | 2657 | unsigned int i; |
| 2659 | 2658 | ||
| 2660 | QETH_CARD_TEXT(card, 2, "allcqdbf"); | 2659 | QETH_CARD_TEXT(card, 2, "allcqdbf"); |
| 2661 | 2660 | ||
| @@ -2684,18 +2683,12 @@ static int qeth_alloc_qdio_queues(struct qeth_card *card) · 22 unmodified lines | |||
| 2684 | card->qdio.out_qs[i] = queue; | 2683 | card->qdio.out_qs[i] = queue; |
| 2685 | queue->card = card; | 2684 | queue->card = card; |
| 2686 | queue->queue_no = i; | 2685 | queue->queue_no = i; |
| 2686 | INIT_LIST_HEAD(&queue->pending_bufs); | ||
| 2687 | spin_lock_init(&queue->lock); | 2687 | spin_lock_init(&queue->lock); |
| 2688 | timer_setup(&queue->timer, qeth_tx_completion_timer, 0); | 2688 | timer_setup(&queue->timer, qeth_tx_completion_timer, 0); |
| 2689 | queue->coalesce_usecs = QETH_TX_COALESCE_USECS; | 2689 | queue->coalesce_usecs = QETH_TX_COALESCE_USECS; |
| 2690 | queue->max_coalesced_frames = QETH_TX_MAX_COALESCED_FRAMES; | 2690 | queue->max_coalesced_frames = QETH_TX_MAX_COALESCED_FRAMES; |
| 2691 | queue->priority = QETH_QIB_PQUE_PRIO_DEFAULT; | 2691 | queue->priority = QETH_QIB_PQUE_PRIO_DEFAULT; |
| 2692 | |||
| 2693 | /* give outbound qeth_qdio_buffers their qdio_buffers */ | ||
| 2694 | for (j = 0; j < QDIO_MAX_BUFFERS_PER_Q; ++j) { | ||
| 2695 | WARN_ON(queue->bufs[j]); | ||
| 2696 | if (qeth_init_qdio_out_buf(queue, j)) | ||
| 2697 | goto out_freeoutqbufs; | ||
| 2698 | } | ||
| 2699 | } | 2692 | } |
| 2700 | 2693 | ||
| 2701 | /* completion */ | 2694 | /* completion */ |
| @@ -2704,13 +2697,6 @@ static int qeth_alloc_qdio_queues(struct qeth_card *card) · 2 unmodified lines | |||
| 2704 | 2697 | ||
| 2705 | return 0; | 2698 | return 0; |
| 2706 | 2699 | ||
| 2707 | out_freeoutqbufs: | ||
| 2708 | while (j > 0) { | ||
| 2709 | --j; | ||
| 2710 | kmem_cache_free(qeth_qdio_outbuf_cache, | ||
| 2711 | card->qdio.out_qs[i]->bufs[j]); | ||
| 2712 | card->qdio.out_qs[i]->bufs[j] = NULL; | ||
| 2713 | } | ||
| 2714 | out_freeoutq: | 2700 | out_freeoutq: |
| 2715 | while (i > 0) { | 2701 | while (i > 0) { |
| 2716 | qeth_free_output_queue(card->qdio.out_qs[--i]); | 2702 | qeth_free_output_queue(card->qdio.out_qs[--i]); |
| @@ -6107,6 +6093,8 @@ static void qeth_iqd_tx_complete(struct qeth_qdio_out_q *queue, · 3390 unmodified lines | |||
| 6107 | qeth_schedule_recovery(card); | 6093 | qeth_schedule_recovery(card); |
| 6108 | } | 6094 | } |
| 6109 | 6095 | ||
| 6096 | list_add(&buffer->list_entry, | ||
| 6097 | &queue->pending_bufs); | ||
| 6110 | /* Skip clearing the buffer: */ | 6098 | /* Skip clearing the buffer: */ |
| 6111 | return; | 6099 | return; |
| 6112 | case QETH_QDIO_BUF_QAOB_OK: | 6100 | case QETH_QDIO_BUF_QAOB_OK: |
| @@ -6162,6 +6150,8 @@ static int qeth_tx_poll(struct napi_struct *napi, int budget) · 49 unmodified lines | |||
| 6162 | unsigned int bytes = 0; | 6150 | unsigned int bytes = 0; |
| 6163 | int completed; | 6151 | int completed; |
| 6164 | 6152 | ||
| 6153 | qeth_tx_complete_pending_bufs(card, queue, false); | ||
| 6154 | |||
| 6165 | if (qeth_out_queue_is_empty(queue)) { | 6155 | if (qeth_out_queue_is_empty(queue)) { |
| 6166 | napi_complete(napi); | 6156 | napi_complete(napi); |
| 6167 | return 0; | 6157 | return 0; |
| @@ -6194,7 +6184,6 @@ static int qeth_tx_poll(struct napi_struct *napi, int budget) · 26 unmodified lines | |||
| 6194 | 6184 | ||
| 6195 | qeth_handle_send_error(card, buffer, error); | 6185 | qeth_handle_send_error(card, buffer, error); |
| 6196 | qeth_iqd_tx_complete(queue, bidx, error, budget); | 6186 | qeth_iqd_tx_complete(queue, bidx, error, budget); |
| 6197 | qeth_cleanup_handled_pending(queue, bidx, false); | ||
| 6198 | } | 6187 | } |
| 6199 | 6188 | ||
| 6200 | netdev_tx_completed_queue(txq, packets, bytes); | 6189 | netdev_tx_completed_queue(txq, packets, bytes); |
| @@ -7249,9 +7238,7 @@ int qeth_open(struct net_device *dev) · 1048 unmodified lines | |||
| 7249 | card->data.state = CH_STATE_UP; | 7238 | card->data.state = CH_STATE_UP; |
| 7250 | netif_tx_start_all_queues(dev); | 7239 | netif_tx_start_all_queues(dev); |
| 7251 | 7240 | ||
| 7252 | napi_enable(&card->napi); | ||
| 7253 | local_bh_disable(); | 7241 | local_bh_disable(); |
| 7254 | napi_schedule(&card->napi); | ||
| 7255 | if (IS_IQD(card)) { | 7242 | if (IS_IQD(card)) { |
| 7256 | struct qeth_qdio_out_q *queue; | 7243 | struct qeth_qdio_out_q *queue; |
| 7257 | unsigned int i; | 7244 | unsigned int i; |
| @@ -7263,8 +7250,12 @@ int qeth_open(struct net_device *dev) · 5 unmodified lines | |||
| 7263 | napi_schedule(&queue->napi); | 7250 | napi_schedule(&queue->napi); |
| 7264 | } | 7251 | } |
| 7265 | } | 7252 | } |
| 7253 | |||
| 7254 | napi_enable(&card->napi); | ||
| 7255 | napi_schedule(&card->napi); | ||
| 7266 | /* kick-start the NAPI softirq: */ | 7256 | /* kick-start the NAPI softirq: */ |
| 7267 | local_bh_enable(); | 7257 | local_bh_enable(); |
| 7258 | |||
| 7268 | return 0; | 7259 | return 0; |
| 7269 | } | 7260 | } |
| 7270 | EXPORT_SYMBOL_GPL(qeth_open); | 7261 | EXPORT_SYMBOL_GPL(qeth_open); |
| @@ -7274,6 +7265,11 @@ int qeth_stop(struct net_device *dev) · 3 unmodified lines | |||
| 7274 | struct qeth_card *card = dev->ml_priv; | 7265 | struct qeth_card *card = dev->ml_priv; |
| 7275 | 7266 | ||
| 7276 | QETH_CARD_TEXT(card, 4, "qethstop"); | 7267 | QETH_CARD_TEXT(card, 4, "qethstop"); |
| 7268 | |||
| 7269 | napi_disable(&card->napi); | ||
| 7270 | cancel_delayed_work_sync(&card->buffer_reclaim_work); | ||
| 7271 | qdio_stop_irq(CARD_DDEV(card)); | ||
| 7272 | |||
| 7277 | if (IS_IQD(card)) { | 7273 | if (IS_IQD(card)) { |
| 7278 | struct qeth_qdio_out_q *queue; | 7274 | struct qeth_qdio_out_q *queue; |
| 7279 | unsigned int i; | 7275 | unsigned int i; |
| @@ -7294,10 +7290,6 @@ int qeth_stop(struct net_device *dev) · 14 unmodified lines | |||
| 7294 | netif_tx_disable(dev); | 7290 | netif_tx_disable(dev); |
| 7295 | } | 7291 | } |
| 7296 | 7292 | ||
| 7297 | napi_disable(&card->napi); | ||
| 7298 | cancel_delayed_work_sync(&card->buffer_reclaim_work); | ||
| 7299 | qdio_stop_irq(CARD_DDEV(card)); | ||
| 7300 | |||
| 7301 | return 0; | 7293 | return 0; |
| 7302 | } | 7294 | } |
| 7303 | EXPORT_SYMBOL_GPL(qeth_stop); | 7295 | EXPORT_SYMBOL_GPL(qeth_stop); |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | |||
| 21 | #include <linux/bsg-lib.h> | 21 | #include <linux/bsg-lib.h> |
| 22 | #include <asm/firmware.h> | 22 | #include <asm/firmware.h> |
| 23 | #include <asm/irq.h> | 23 | #include <asm/irq.h> |
| 24 | #include <asm/rtas.h> | ||
| 24 | #include <asm/vio.h> | 25 | #include <asm/vio.h> |
| 25 | #include <scsi/scsi.h> | 26 | #include <scsi/scsi.h> |
| 26 | #include <scsi/scsi_cmnd.h> | 27 | #include <scsi/scsi_cmnd.h> |
| @@ -158,6 +159,9 @@ static void ibmvfc_npiv_logout(struct ibmvfc_host *); · 131 unmodified lines | |||
| 158 | static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *); | 159 | static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *); |
| 159 | static void ibmvfc_tgt_move_login(struct ibmvfc_target *); | 160 | static void ibmvfc_tgt_move_login(struct ibmvfc_target *); |
| 160 | 161 | ||
| 162 | static void ibmvfc_release_sub_crqs(struct ibmvfc_host *); | ||
| 163 | static void ibmvfc_init_sub_crqs(struct ibmvfc_host *); | ||
| 164 | |||
| 161 | static const char *unknown_error = "unknown error"; | 165 | static const char *unknown_error = "unknown error"; |
| 162 | 166 | ||
| 163 | static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba, | 167 | static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba, |
| @@ -899,6 +903,9 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) · 735 unmodified lines | |||
| 899 | { | 903 | { |
| 900 | int rc = 0; | 904 | int rc = 0; |
| 901 | struct vio_dev *vdev = to_vio_dev(vhost->dev); | 905 | struct vio_dev *vdev = to_vio_dev(vhost->dev); |
| 906 | unsigned long flags; | ||
| 907 | |||
| 908 | ibmvfc_release_sub_crqs(vhost); | ||
| 902 | 909 | ||
| 903 | /* Re-enable the CRQ */ | 910 | /* Re-enable the CRQ */ |
| 904 | do { | 911 | do { |
| @@ -910,6 +917,15 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) · 5 unmodified lines | |||
| 910 | if (rc) | 917 | if (rc) |
| 911 | dev_err(vhost->dev, "Error enabling adapter (rc=%d)\n", rc); | 918 | dev_err(vhost->dev, "Error enabling adapter (rc=%d)\n", rc); |
| 912 | 919 | ||
| 920 | spin_lock_irqsave(vhost->host->host_lock, flags); | ||
| 921 | spin_lock(vhost->crq.q_lock); | ||
| 922 | vhost->do_enquiry = 1; | ||
| 923 | vhost->using_channels = 0; | ||
| 924 | spin_unlock(vhost->crq.q_lock); | ||
| 925 | spin_unlock_irqrestore(vhost->host->host_lock, flags); | ||
| 926 | |||
| 927 | ibmvfc_init_sub_crqs(vhost); | ||
| 928 | |||
| 913 | return rc; | 929 | return rc; |
| 914 | } | 930 | } |
| 915 | 931 | ||
| @@ -926,8 +942,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) · 10 unmodified lines | |||
| 926 | unsigned long flags; | 942 | unsigned long flags; |
| 927 | struct vio_dev *vdev = to_vio_dev(vhost->dev); | 943 | struct vio_dev *vdev = to_vio_dev(vhost->dev); |
| 928 | struct ibmvfc_queue *crq = &vhost->crq; | 944 | struct ibmvfc_queue *crq = &vhost->crq; |
| 929 | struct ibmvfc_queue *scrq; | 945 | |
| 930 | int i; | 946 | ibmvfc_release_sub_crqs(vhost); |
| 931 | 947 | ||
| 932 | /* Close the CRQ */ | 948 | /* Close the CRQ */ |
| 933 | do { | 949 | do { |
| @@ -947,16 +963,6 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) · 13 unmodified lines | |||
| 947 | memset(crq->msgs.crq, 0, PAGE_SIZE); | 963 | memset(crq->msgs.crq, 0, PAGE_SIZE); |
| 948 | crq->cur = 0; | 964 | crq->cur = 0; |
| 949 | 965 | ||
| 950 | if (vhost->scsi_scrqs.scrqs) { | ||
| 951 | for (i = 0; i < nr_scsi_hw_queues; i++) { | ||
| 952 | scrq = &vhost->scsi_scrqs.scrqs[i]; | ||
| 953 | spin_lock(scrq->q_lock); | ||
| 954 | memset(scrq->msgs.scrq, 0, PAGE_SIZE); | ||
| 955 | scrq->cur = 0; | ||
| 956 | spin_unlock(scrq->q_lock); | ||
| 957 | } | ||
| 958 | } | ||
| 959 | |||
| 960 | /* And re-open it again */ | 966 | /* And re-open it again */ |
| 961 | rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, | 967 | rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, |
| 962 | crq->msg_token, PAGE_SIZE); | 968 | crq->msg_token, PAGE_SIZE); |
| @@ -966,9 +972,12 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) · 3 unmodified lines | |||
| 966 | dev_warn(vhost->dev, "Partner adapter not ready\n"); | 972 | dev_warn(vhost->dev, "Partner adapter not ready\n"); |
| 967 | else if (rc != 0) | 973 | else if (rc != 0) |
| 968 | dev_warn(vhost->dev, "Couldn't register crq (rc=%d)\n", rc); | 974 | dev_warn(vhost->dev, "Couldn't register crq (rc=%d)\n", rc); |
| 975 | |||
| 969 | spin_unlock(vhost->crq.q_lock); | 976 | spin_unlock(vhost->crq.q_lock); |
| 970 | spin_unlock_irqrestore(vhost->host->host_lock, flags); | 977 | spin_unlock_irqrestore(vhost->host->host_lock, flags); |
| 971 | 978 | ||
| 979 | ibmvfc_init_sub_crqs(vhost); | ||
| 980 | |||
| 972 | return rc; | 981 | return rc; |
| 973 | } | 982 | } |
| 974 | 983 | ||
| @@ -5642,7 +5651,8 @@ static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, · 4667 unmodified lines | |||
| 5642 | rc = h_reg_sub_crq(vdev->unit_address, scrq->msg_token, PAGE_SIZE, | 5651 | rc = h_reg_sub_crq(vdev->unit_address, scrq->msg_token, PAGE_SIZE, |
| 5643 | &scrq->cookie, &scrq->hw_irq); | 5652 | &scrq->cookie, &scrq->hw_irq); |
| 5644 | 5653 | ||
| 5645 | if (rc) { | 5654 | /* H_CLOSED indicates successful register, but no CRQ partner */ |
| 5655 | if (rc && rc != H_CLOSED) { | ||
| 5646 | dev_warn(dev, "Error registering sub-crq: %d\n", rc); | 5656 | dev_warn(dev, "Error registering sub-crq: %d\n", rc); |
| 5647 | if (rc == H_PARAMETER) | 5657 | if (rc == H_PARAMETER) |
| 5648 | dev_warn_once(dev, "Firmware may not support MQ\n"); | 5658 | dev_warn_once(dev, "Firmware may not support MQ\n"); |
| @@ -5675,8 +5685,8 @@ static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, · 26 unmodified lines | |||
| 5675 | 5685 | ||
| 5676 | irq_failed: | 5686 | irq_failed: |
| 5677 | do { | 5687 | do { |
| 5678 | plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); | 5688 | rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); |
| 5679 | } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); | 5689 | } while (rtas_busy_delay(rc)); |
| 5680 | reg_failed: | 5690 | reg_failed: |
| 5681 | ibmvfc_free_queue(vhost, scrq); | 5691 | ibmvfc_free_queue(vhost, scrq); |
| 5682 | LEAVE; | 5692 | LEAVE; |
| @@ -5694,6 +5704,7 @@ static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) · 11 unmodified lines | |||
| 5694 | 5704 | ||
| 5695 | free_irq(scrq->irq, scrq); | 5705 | free_irq(scrq->irq, scrq); |
| 5696 | irq_dispose_mapping(scrq->irq); | 5706 | irq_dispose_mapping(scrq->irq); |
| 5707 | scrq->irq = 0; | ||
| 5697 | 5708 | ||
| 5698 | do { | 5709 | do { |
| 5699 | rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, | 5710 | rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, |
| @@ -5707,17 +5718,21 @@ static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) · 7 unmodified lines | |||
| 5707 | LEAVE; | 5718 | LEAVE; |
| 5708 | } | 5719 | } |
| 5709 | 5720 | ||
| 5710 | static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) | 5721 | static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) |
| 5711 | { | 5722 | { |
| 5712 | int i, j; | 5723 | int i, j; |
| 5713 | 5724 | ||
| 5714 | ENTER; | 5725 | ENTER; |
| 5726 | if (!vhost->mq_enabled) | ||
| 5727 | return; | ||
| 5715 | 5728 | ||
| 5716 | vhost->scsi_scrqs.scrqs = kcalloc(nr_scsi_hw_queues, | 5729 | vhost->scsi_scrqs.scrqs = kcalloc(nr_scsi_hw_queues, |
| 5717 | sizeof(*vhost->scsi_scrqs.scrqs), | 5730 | sizeof(*vhost->scsi_scrqs.scrqs), |
| 5718 | GFP_KERNEL); | 5731 | GFP_KERNEL); |
| 5719 | if (!vhost->scsi_scrqs.scrqs) | 5732 | if (!vhost->scsi_scrqs.scrqs) { |
| 5720 | return -1; | 5733 | vhost->do_enquiry = 0; |
| 5734 | return; | ||
| 5735 | } | ||
| 5721 | 5736 | ||
| 5722 | for (i = 0; i < nr_scsi_hw_queues; i++) { | 5737 | for (i = 0; i < nr_scsi_hw_queues; i++) { |
| 5723 | if (ibmvfc_register_scsi_channel(vhost, i)) { | 5738 | if (ibmvfc_register_scsi_channel(vhost, i)) { |
| @@ -5726,13 +5741,12 @@ static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) · 2 unmodified lines | |||
| 5726 | kfree(vhost->scsi_scrqs.scrqs); | 5741 | kfree(vhost->scsi_scrqs.scrqs); |
| 5727 | vhost->scsi_scrqs.scrqs = NULL; | 5742 | vhost->scsi_scrqs.scrqs = NULL; |
| 5728 | vhost->scsi_scrqs.active_queues = 0; | 5743 | vhost->scsi_scrqs.active_queues = 0; |
| 5729 | LEAVE; | 5744 | vhost->do_enquiry = 0; |
| 5730 | return -1; | 5745 | break; |
| 5731 | } | 5746 | } |
| 5732 | } | 5747 | } |
| 5733 | 5748 | ||
| 5734 | LEAVE; | 5749 | LEAVE; |
| 5735 | return 0; | ||
| 5736 | } | 5750 | } |
| 5737 | 5751 | ||
| 5738 | static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) | 5752 | static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) |
| @@ -5999,11 +6013,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) · 260 unmodified lines | |||
| 5999 | goto remove_shost; | 6013 | goto remove_shost; |
| 6000 | } | 6014 | } |
| 6001 | 6015 | ||
| 6002 | if (vhost->mq_enabled) { | 6016 | ibmvfc_init_sub_crqs(vhost); |
| 6003 | rc = ibmvfc_init_sub_crqs(vhost); | ||
| 6004 | if (rc) | ||
| 6005 | dev_warn(dev, "Failed to allocate Sub-CRQs. rc=%d\n", rc); | ||
| 6006 | } | ||
| 6007 | 6017 | ||
| 6008 | if (shost_to_fc_host(shost)->rqst_q) | 6018 | if (shost_to_fc_host(shost)->rqst_q) |
| 6009 | blk_queue_max_segments(shost_to_fc_host(shost)->rqst_q, 1); | 6019 | blk_queue_max_segments(shost_to_fc_host(shost)->rqst_q, 1); |
| @@ -6038,7 +6048,7 @@ out: · 28 unmodified lines | |||
| 6038 | * Return value: | 6048 | * Return value: |
| 6039 | * 0 | 6049 | * 0 |
| 6040 | **/ | 6050 | **/ |
| 6041 | static int ibmvfc_remove(struct vio_dev *vdev) | 6051 | static void ibmvfc_remove(struct vio_dev *vdev) |
| 6042 | { | 6052 | { |
| 6043 | struct ibmvfc_host *vhost = dev_get_drvdata(&vdev->dev); | 6053 | struct ibmvfc_host *vhost = dev_get_drvdata(&vdev->dev); |
| 6044 | LIST_HEAD(purge); | 6054 | LIST_HEAD(purge); |
| @@ -6070,7 +6080,6 @@ static int ibmvfc_remove(struct vio_dev *vdev) · 25 unmodified lines | |||
| 6070 | spin_unlock(&ibmvfc_driver_lock); | 6080 | spin_unlock(&ibmvfc_driver_lock); |
| 6071 | scsi_host_put(vhost->host); | 6081 | scsi_host_put(vhost->host); |
| 6072 | LEAVE; | 6082 | LEAVE; |
| 6073 | return 0; | ||
| 6074 | } | 6083 | } |
| 6075 | 6084 | ||
| 6076 | /** | 6085 | /** |
| @@ -2335,7 +2335,7 @@ static int ibmvscsi_probe(struct vio_dev *vdev, const struct vio_device_id *id) · 2334 unmodified lines | |||
| 2335 | return -1; | 2335 | return -1; |
| 2336 | } | 2336 | } |
| 2337 | 2337 | ||
| 2338 | static int ibmvscsi_remove(struct vio_dev *vdev) | 2338 | static void ibmvscsi_remove(struct vio_dev *vdev) |
| 2339 | { | 2339 | { |
| 2340 | struct ibmvscsi_host_data *hostdata = dev_get_drvdata(&vdev->dev); | 2340 | struct ibmvscsi_host_data *hostdata = dev_get_drvdata(&vdev->dev); |
| 2341 | 2341 | ||
| @@ -2356,8 +2356,6 @@ static int ibmvscsi_remove(struct vio_dev *vdev) · 14 unmodified lines | |||
| 2356 | spin_unlock(&ibmvscsi_driver_lock); | 2356 | spin_unlock(&ibmvscsi_driver_lock); |
| 2357 | 2357 | ||
| 2358 | scsi_host_put(hostdata->host); | 2358 | scsi_host_put(hostdata->host); |
| 2359 | |||
| 2360 | return 0; | ||
| 2361 | } | 2359 | } |
| 2362 | 2360 | ||
| 2363 | /** | 2361 | /** |
| @@ -3595,7 +3595,7 @@ free_adapter: · 3594 unmodified lines | |||
| 3595 | return rc; | 3595 | return rc; |
| 3596 | } | 3596 | } |
| 3597 | 3597 | ||
| 3598 | static int ibmvscsis_remove(struct vio_dev *vdev) | 3598 | static void ibmvscsis_remove(struct vio_dev *vdev) |
| 3599 | { | 3599 | { |
| 3600 | struct scsi_info *vscsi = dev_get_drvdata(&vdev->dev); | 3600 | struct scsi_info *vscsi = dev_get_drvdata(&vdev->dev); |
| 3601 | 3601 | ||
| @@ -3622,8 +3622,6 @@ static int ibmvscsis_remove(struct vio_dev *vdev) · 20 unmodified lines | |||
| 3622 | list_del(&vscsi->list); | 3622 | list_del(&vscsi->list); |
| 3623 | spin_unlock_bh(&ibmvscsis_dev_lock); | 3623 | spin_unlock_bh(&ibmvscsis_dev_lock); |
| 3624 | kfree(vscsi); | 3624 | kfree(vscsi); |
| 3625 | |||
| 3626 | return 0; | ||
| 3627 | } | 3625 | } |
| 3628 | 3626 | ||
| 3629 | static ssize_t system_id_show(struct device *dev, | 3627 | static ssize_t system_id_show(struct device *dev, |
| @@ -253,12 +253,17 @@ static int ufs_qcom_host_reset(struct ufs_hba *hba) · 252 unmodified lines | |||
| 253 | { | 253 | { |
| 254 | int ret = 0; | 254 | int ret = 0; |
| 255 | struct ufs_qcom_host *host = ufshcd_get_variant(hba); | 255 | struct ufs_qcom_host *host = ufshcd_get_variant(hba); |
| 256 | bool reenable_intr = false; | ||
| 256 | 257 | ||
| 257 | if (!host->core_reset) { | 258 | if (!host->core_reset) { |
| 258 | dev_warn(hba->dev, "%s: reset control not set\n", __func__); | 259 | dev_warn(hba->dev, "%s: reset control not set\n", __func__); |
| 259 | goto out; | 260 | goto out; |
| 260 | } | 261 | } |
| 261 | 262 | ||
| 263 | reenable_intr = hba->is_irq_enabled; | ||
| 264 | disable_irq(hba->irq); | ||
| 265 | hba->is_irq_enabled = false; | ||
| 266 | |||
| 262 | ret = reset_control_assert(host->core_reset); | 267 | ret = reset_control_assert(host->core_reset); |
| 263 | if (ret) { | 268 | if (ret) { |
| 264 | dev_err(hba->dev, "%s: core_reset assert failed, err = %d\n", | 269 | dev_err(hba->dev, "%s: core_reset assert failed, err = %d\n", |
| @@ -280,6 +285,11 @@ static int ufs_qcom_host_reset(struct ufs_hba *hba) · 15 unmodified lines | |||
| 280 | 285 | ||
| 281 | usleep_range(1000, 1100); | 286 | usleep_range(1000, 1100); |
| 282 | 287 | ||
| 288 | if (reenable_intr) { | ||
| 289 | enable_irq(hba->irq); | ||
| 290 | hba->is_irq_enabled = true; | ||
| 291 | } | ||
| 292 | |||
| 283 | out: | 293 | out: |
| 284 | return ret; | 294 | return ret; |
| 285 | } | 295 | } |
| @@ -95,8 +95,6 @@ · 94 unmodified lines | |||
| 95 | 16, 4, buf, __len, false); \ | 95 | 16, 4, buf, __len, false); \ |
| 96 | } while (0) | 96 | } while (0) |
| 97 | 97 | ||
| 98 | static bool early_suspend; | ||
| 99 | |||
| 100 | int ufshcd_dump_regs(struct ufs_hba *hba, size_t offset, size_t len, | 98 | int ufshcd_dump_regs(struct ufs_hba *hba, size_t offset, size_t len, |
| 101 | const char *prefix) | 99 | const char *prefix) |
| 102 | { | 100 | { |
| @@ -1535,7 +1533,7 @@ static ssize_t ufshcd_clkscale_enable_show(struct device *dev, · 1432 unmodified lines | |||
| 1535 | { | 1533 | { |
| 1536 | struct ufs_hba *hba = dev_get_drvdata(dev); | 1534 | struct ufs_hba *hba = dev_get_drvdata(dev); |
| 1537 | 1535 | ||
| 1538 | return snprintf(buf, PAGE_SIZE, "%d\n", hba->clk_scaling.is_enabled); | 1536 | return sysfs_emit(buf, "%d\n", hba->clk_scaling.is_enabled); |
| 1539 | } | 1537 | } |
| 1540 | 1538 | ||
| 1541 | static ssize_t ufshcd_clkscale_enable_store(struct device *dev, | 1539 | static ssize_t ufshcd_clkscale_enable_store(struct device *dev, |
| @@ -4987,6 +4985,7 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · 3445 unmodified lines | |||
| 4987 | * UFS device needs urgent BKOPs. | 4985 | * UFS device needs urgent BKOPs. |
| 4988 | */ | 4986 | */ |
| 4989 | if (!hba->pm_op_in_progress && | 4987 | if (!hba->pm_op_in_progress && |
| 4988 | !ufshcd_eh_in_progress(hba) && | ||
| 4990 | ufshcd_is_exception_event(lrbp->ucd_rsp_ptr) && | 4989 | ufshcd_is_exception_event(lrbp->ucd_rsp_ptr) && |
| 4991 | schedule_work(&hba->eeh_work)) { | 4990 | schedule_work(&hba->eeh_work)) { |
| 4992 | /* | 4991 | /* |
| @@ -5784,13 +5783,20 @@ static void ufshcd_err_handling_prepare(struct ufs_hba *hba) · 791 unmodified lines | |||
| 5784 | ufshcd_suspend_clkscaling(hba); | 5783 | ufshcd_suspend_clkscaling(hba); |
| 5785 | ufshcd_clk_scaling_allow(hba, false); | 5784 | ufshcd_clk_scaling_allow(hba, false); |
| 5786 | } | 5785 | } |
| 5786 | ufshcd_scsi_block_requests(hba); | ||
| 5787 | /* Drain ufshcd_queuecommand() */ | ||
| 5788 | down_write(&hba->clk_scaling_lock); | ||
| 5789 | up_write(&hba->clk_scaling_lock); | ||
| 5790 | cancel_work_sync(&hba->eeh_work); | ||
| 5787 | } | 5791 | } |
| 5788 | 5792 | ||
| 5789 | static void ufshcd_err_handling_unprepare(struct ufs_hba *hba) | 5793 | static void ufshcd_err_handling_unprepare(struct ufs_hba *hba) |
| 5790 | { | 5794 | { |
| 5795 | ufshcd_scsi_unblock_requests(hba); | ||
| 5791 | ufshcd_release(hba); | 5796 | ufshcd_release(hba); |
| 5792 | if (ufshcd_is_clkscaling_supported(hba)) | 5797 | if (ufshcd_is_clkscaling_supported(hba)) |
| 5793 | ufshcd_clk_scaling_suspend(hba, false); | 5798 | ufshcd_clk_scaling_suspend(hba, false); |
| 5799 | ufshcd_clear_ua_wluns(hba); | ||
| 5794 | pm_runtime_put(hba->dev); | 5800 | pm_runtime_put(hba->dev); |
| 5795 | } | 5801 | } |
| 5796 | 5802 | ||
| @@ -5882,8 +5888,8 @@ static void ufshcd_err_handler(struct work_struct *work) · 85 unmodified lines | |||
| 5882 | spin_unlock_irqrestore(hba->host->host_lock, flags); | 5888 | spin_unlock_irqrestore(hba->host->host_lock, flags); |
| 5883 | ufshcd_err_handling_prepare(hba); | 5889 | ufshcd_err_handling_prepare(hba); |
| 5884 | spin_lock_irqsave(hba->host->host_lock, flags); | 5890 | spin_lock_irqsave(hba->host->host_lock, flags); |
| 5885 | ufshcd_scsi_block_requests(hba); | 5891 | if (hba->ufshcd_state != UFSHCD_STATE_ERROR) |
| 5886 | hba->ufshcd_state = UFSHCD_STATE_RESET; | 5892 | hba->ufshcd_state = UFSHCD_STATE_RESET; |
| 5887 | 5893 | ||
| 5888 | /* Complete requests that have door-bell cleared by h/w */ | 5894 | /* Complete requests that have door-bell cleared by h/w */ |
| 5889 | ufshcd_complete_requests(hba); | 5895 | ufshcd_complete_requests(hba); |
| @@ -6042,12 +6048,8 @@ skip_err_handling: · 152 unmodified lines | |||
| 6042 | } | 6048 | } |
| 6043 | ufshcd_clear_eh_in_progress(hba); | 6049 | ufshcd_clear_eh_in_progress(hba); |
| 6044 | spin_unlock_irqrestore(hba->host->host_lock, flags); | 6050 | spin_unlock_irqrestore(hba->host->host_lock, flags); |
| 6045 | ufshcd_scsi_unblock_requests(hba); | ||
| 6046 | ufshcd_err_handling_unprepare(hba); | 6051 | ufshcd_err_handling_unprepare(hba); |
| 6047 | up(&hba->host_sem); | 6052 | up(&hba->host_sem); |
| 6048 | |||
| 6049 | if (!err && needs_reset) | ||
| 6050 | ufshcd_clear_ua_wluns(hba); | ||
| 6051 | } | 6053 | } |
| 6052 | 6054 | ||
| 6053 | /** | 6055 | /** |
| @@ -7858,6 +7860,8 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool async) · 1804 unmodified lines | |||
| 7858 | unsigned long flags; | 7860 | unsigned long flags; |
| 7859 | ktime_t start = ktime_get(); | 7861 | ktime_t start = ktime_get(); |
| 7860 | 7862 | ||
| 7863 | hba->ufshcd_state = UFSHCD_STATE_RESET; | ||
| 7864 | |||
| 7861 | ret = ufshcd_link_startup(hba); | 7865 | ret = ufshcd_link_startup(hba); |
| 7862 | if (ret) | 7866 | if (ret) |
| 7863 | goto out; | 7867 | goto out; |
| @@ -8972,11 +8976,6 @@ int ufshcd_system_suspend(struct ufs_hba *hba) · 1108 unmodified lines | |||
| 8972 | int ret = 0; | 8976 | int ret = 0; |
| 8973 | ktime_t start = ktime_get(); | 8977 | ktime_t start = ktime_get(); |
| 8974 | 8978 | ||
| 8975 | if (!hba) { | ||
| 8976 | early_suspend = true; | ||
| 8977 | return 0; | ||
| 8978 | } | ||
| 8979 | |||
| 8980 | down(&hba->host_sem); | 8979 | down(&hba->host_sem); |
| 8981 | 8980 | ||
| 8982 | if (!hba->is_powered) | 8981 | if (!hba->is_powered) |
| @@ -9028,14 +9027,6 @@ int ufshcd_system_resume(struct ufs_hba *hba) · 45 unmodified lines | |||
| 9028 | int ret = 0; | 9027 | int ret = 0; |
| 9029 | ktime_t start = ktime_get(); | 9028 | ktime_t start = ktime_get(); |
| 9030 | 9029 | ||
| 9031 | if (!hba) | ||
| 9032 | return -EINVAL; | ||
| 9033 | |||
| 9034 | if (unlikely(early_suspend)) { | ||
| 9035 | early_suspend = false; | ||
| 9036 | down(&hba->host_sem); | ||
| 9037 | } | ||
| 9038 | |||
| 9039 | if (!hba->is_powered || pm_runtime_suspended(hba->dev)) | 9030 | if (!hba->is_powered || pm_runtime_suspended(hba->dev)) |
| 9040 | /* | 9031 | /* |
| 9041 | * Let the runtime resume take care of resuming | 9032 | * Let the runtime resume take care of resuming |
| @@ -9068,9 +9059,6 @@ int ufshcd_runtime_suspend(struct ufs_hba *hba) · 26 unmodified lines | |||
| 9068 | int ret = 0; | 9059 | int ret = 0; |
| 9069 | ktime_t start = ktime_get(); | 9060 | ktime_t start = ktime_get(); |
| 9070 | 9061 | ||
| 9071 | if (!hba) | ||
| 9072 | return -EINVAL; | ||
| 9073 | |||
| 9074 | if (!hba->is_powered) | 9062 | if (!hba->is_powered) |
| 9075 | goto out; | 9063 | goto out; |
| 9076 | else | 9064 | else |
| @@ -9109,9 +9097,6 @@ int ufshcd_runtime_resume(struct ufs_hba *hba) · 32 unmodified lines | |||
| 9109 | int ret = 0; | 9097 | int ret = 0; |
| 9110 | ktime_t start = ktime_get(); | 9098 | ktime_t start = ktime_get(); |
| 9111 | 9099 | ||
| 9112 | if (!hba) | ||
| 9113 | return -EINVAL; | ||
| 9114 | |||
| 9115 | if (!hba->is_powered) | 9100 | if (!hba->is_powered) |
| 9116 | goto out; | 9101 | goto out; |
| 9117 | else | 9102 | else |
| @@ -17,8 +17,6 @@ · 16 unmodified lines | |||
| 17 | * along with this program; if not, write to the Free Software | 17 | * along with this program; if not, write to the Free Software |
| 18 | * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. | 18 | * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
| 19 | * | 19 | * |
| 20 | * Maintained by: Jim Gill <jgill@vmware.com> | ||
| 21 | * | ||
| 22 | */ | 20 | */ |
| 23 | 21 | ||
| 24 | #include <linux/kernel.h> | 22 | #include <linux/kernel.h> |
| @@ -17,8 +17,6 @@ · 16 unmodified lines | |||
| 17 | * along with this program; if not, write to the Free Software | 17 | * along with this program; if not, write to the Free Software |
| 18 | * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. | 18 | * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
| 19 | * | 19 | * |
| 20 | * Maintained by: Jim Gill <jgill@vmware.com> | ||
| 21 | * | ||
| 22 | */ | 20 | */ |
| 23 | 21 | ||
| 24 | #ifndef _VMW_PVSCSI_H_ | 22 | #ifndef _VMW_PVSCSI_H_ |
| @@ -260,6 +260,7 @@ static irqreturn_t apci1032_interrupt(int irq, void *d) · 259 unmodified lines | |||
| 260 | struct apci1032_private *devpriv = dev->private; | 260 | struct apci1032_private *devpriv = dev->private; |
| 261 | struct comedi_subdevice *s = dev->read_subdev; | 261 | struct comedi_subdevice *s = dev->read_subdev; |
| 262 | unsigned int ctrl; | 262 | unsigned int ctrl; |
| 263 | unsigned short val; | ||
| 263 | 264 | ||
| 264 | /* check interrupt is from this device */ | 265 | /* check interrupt is from this device */ |
| 265 | if ((inl(devpriv->amcc_iobase + AMCC_OP_REG_INTCSR) & | 266 | if ((inl(devpriv->amcc_iobase + AMCC_OP_REG_INTCSR) & |
| @@ -275,7 +276,8 @@ static irqreturn_t apci1032_interrupt(int irq, void *d) · 9 unmodified lines | |||
| 275 | outl(ctrl & ~APCI1032_CTRL_INT_ENA, dev->iobase + APCI1032_CTRL_REG); | 276 | outl(ctrl & ~APCI1032_CTRL_INT_ENA, dev->iobase + APCI1032_CTRL_REG); |
| 276 | 277 | ||
| 277 | s->state = inl(dev->iobase + APCI1032_STATUS_REG) & 0xffff; | 278 | s->state = inl(dev->iobase + APCI1032_STATUS_REG) & 0xffff; |
| 278 | comedi_buf_write_samples(s, &s->state, 1); | 279 | val = s->state; |
| 280 | comedi_buf_write_samples(s, &val, 1); | ||
| 279 | comedi_handle_events(dev, s); | 281 | comedi_handle_events(dev, s); |
| 280 | 282 | ||
| 281 | /* enable the interrupt */ | 283 | /* enable the interrupt */ |
| @@ -208,7 +208,7 @@ static irqreturn_t apci1500_interrupt(int irq, void *d) · 207 unmodified lines | |||
| 208 | struct comedi_device *dev = d; | 208 | struct comedi_device *dev = d; |
| 209 | struct apci1500_private *devpriv = dev->private; | 209 | struct apci1500_private *devpriv = dev->private; |
| 210 | struct comedi_subdevice *s = dev->read_subdev; | 210 | struct comedi_subdevice *s = dev->read_subdev; |
| 211 | unsigned int status = 0; | 211 | unsigned short status = 0; |
| 212 | unsigned int val; | 212 | unsigned int val; |
| 213 | 213 | ||
| 214 | val = inl(devpriv->amcc + AMCC_OP_REG_INTCSR); | 214 | val = inl(devpriv->amcc + AMCC_OP_REG_INTCSR); |
| @@ -238,14 +238,14 @@ static irqreturn_t apci1500_interrupt(int irq, void *d) · 23 unmodified lines | |||
| 238 | * | 238 | * |
| 239 | * Mask Meaning | 239 | * Mask Meaning |
| 240 | * ---------- ------------------------------------------ | 240 | * ---------- ------------------------------------------ |
| 241 | * 0x00000001 Event 1 has occurred | 241 | * 0b00000001 Event 1 has occurred |
| 242 | * 0x00000010 Event 2 has occurred | 242 | * 0b00000010 Event 2 has occurred |
| 243 | * 0x00000100 Counter/timer 1 has run down (not implemented) | 243 | * 0b00000100 Counter/timer 1 has run down (not implemented) |
| 244 | * 0x00001000 Counter/timer 2 has run down (not implemented) | 244 | * 0b00001000 Counter/timer 2 has run down (not implemented) |
| 245 | * 0x00010000 Counter 3 has run down (not implemented) | 245 | * 0b00010000 Counter 3 has run down (not implemented) |
| 246 | * 0x00100000 Watchdog has run down (not implemented) | 246 | * 0b00100000 Watchdog has run down (not implemented) |
| 247 | * 0x01000000 Voltage error | 247 | * 0b01000000 Voltage error |
| 248 | * 0x10000000 Short-circuit error | 248 | * 0b10000000 Short-circuit error |
| 249 | */ | 249 | */ |
| 250 | comedi_buf_write_samples(s, &status, 1); | 250 | comedi_buf_write_samples(s, &status, 1); |
| 251 | comedi_handle_events(dev, s); | 251 | comedi_handle_events(dev, s); |
| @@ -300,11 +300,11 @@ static int pci1710_ai_eoc(struct comedi_device *dev, · 299 unmodified lines | |||
| 300 | static int pci1710_ai_read_sample(struct comedi_device *dev, | 300 | static int pci1710_ai_read_sample(struct comedi_device *dev, |
| 301 | struct comedi_subdevice *s, | 301 | struct comedi_subdevice *s, |
| 302 | unsigned int cur_chan, | 302 | unsigned int cur_chan, |
| 303 | unsigned int *val) | 303 | unsigned short *val) |
| 304 | { | 304 | { |
| 305 | const struct boardtype *board = dev->board_ptr; | 305 | const struct boardtype *board = dev->board_ptr; |
| 306 | struct pci1710_private *devpriv = dev->private; | 306 | struct pci1710_private *devpriv = dev->private; |
| 307 | unsigned int sample; | 307 | unsigned short sample; |
| 308 | unsigned int chan; | 308 | unsigned int chan; |
| 309 | 309 | ||
| 310 | sample = inw(dev->iobase + PCI171X_AD_DATA_REG); | 310 | sample = inw(dev->iobase + PCI171X_AD_DATA_REG); |
| @@ -345,7 +345,7 @@ static int pci1710_ai_insn_read(struct comedi_device *dev, · 34 unmodified lines | |||
| 345 | pci1710_ai_setup_chanlist(dev, s, &insn->chanspec, 1, 1); | 345 | pci1710_ai_setup_chanlist(dev, s, &insn->chanspec, 1, 1); |
| 346 | 346 | ||
| 347 | for (i = 0; i < insn->n; i++) { | 347 | for (i = 0; i < insn->n; i++) { |
| 348 | unsigned int val; | 348 | unsigned short val; |
| 349 | 349 | ||
| 350 | /* start conversion */ | 350 | /* start conversion */ |
| 351 | outw(0, dev->iobase + PCI171X_SOFTTRG_REG); | 351 | outw(0, dev->iobase + PCI171X_SOFTTRG_REG); |
| @@ -395,7 +395,7 @@ static void pci1710_handle_every_sample(struct comedi_device *dev, · 43 unmodified lines | |||
| 395 | { | 395 | { |
| 396 | struct comedi_cmd *cmd = &s->async->cmd; | 396 | struct comedi_cmd *cmd = &s->async->cmd; |
| 397 | unsigned int status; | 397 | unsigned int status; |
| 398 | unsigned int val; | 398 | unsigned short val; |
| 399 | int ret; | 399 | int ret; |
| 400 | 400 | ||
| 401 | status = inw(dev->iobase + PCI171X_STATUS_REG); | 401 | status = inw(dev->iobase + PCI171X_STATUS_REG); |
| @@ -455,7 +455,7 @@ static void pci1710_handle_fifo(struct comedi_device *dev, · 53 unmodified lines | |||
| 455 | } | 455 | } |
| 456 | 456 | ||
| 457 | for (i = 0; i < devpriv->max_samples; i++) { | 457 | for (i = 0; i < devpriv->max_samples; i++) { |
| 458 | unsigned int val; | 458 | unsigned short val; |
| 459 | int ret; | 459 | int ret; |
| 460 | 460 | ||
| 461 | ret = pci1710_ai_read_sample(dev, s, s->async->cur_chan, &val); | 461 | ret = pci1710_ai_read_sample(dev, s, s->async->cur_chan, &val); |
| @@ -126,7 +126,9 @@ static irqreturn_t pc236_interrupt(int irq, void *d) · 125 unmodified lines | |||
| 126 | 126 | ||
| 127 | handled = pc236_intr_check(dev); | 127 | handled = pc236_intr_check(dev); |
| 128 | if (dev->attached && handled) { | 128 | if (dev->attached && handled) { |
| 129 | comedi_buf_write_samples(s, &s->state, 1); | 129 | unsigned short val = 0; |
| 130 | |||
| 131 | comedi_buf_write_samples(s, &val, 1); | ||
| 130 | comedi_handle_events(dev, s); | 132 | comedi_handle_events(dev, s); |
| 131 | } | 133 | } |
| 132 | return IRQ_RETVAL(handled); | 134 | return IRQ_RETVAL(handled); |
| @@ -210,12 +210,13 @@ static irqreturn_t parport_interrupt(int irq, void *d) · 209 unmodified lines | |||
| 210 | struct comedi_device *dev = d; | 210 | struct comedi_device *dev = d; |
| 211 | struct comedi_subdevice *s = dev->read_subdev; | 211 | struct comedi_subdevice *s = dev->read_subdev; |
| 212 | unsigned int ctrl; | 212 | unsigned int ctrl; |
| 213 | unsigned short val = 0; | ||
| 213 | 214 | ||
| 214 | ctrl = inb(dev->iobase + PARPORT_CTRL_REG); | 215 | ctrl = inb(dev->iobase + PARPORT_CTRL_REG); |
| 215 | if (!(ctrl & PARPORT_CTRL_IRQ_ENA)) | 216 | if (!(ctrl & PARPORT_CTRL_IRQ_ENA)) |
| 216 | return IRQ_NONE; | 217 | return IRQ_NONE; |
| 217 | 218 | ||
| 218 | comedi_buf_write_samples(s, &s->state, 1); | 219 | comedi_buf_write_samples(s, &val, 1); |
| 219 | comedi_handle_events(dev, s); | 220 | comedi_handle_events(dev, s); |
| 220 | 221 | ||
| 221 | return IRQ_HANDLED; | 222 | return IRQ_HANDLED; |
| @@ -186,7 +186,7 @@ static irqreturn_t das6402_interrupt(int irq, void *d) · 185 unmodified lines | |||
| 186 | if (status & DAS6402_STATUS_FFULL) { | 186 | if (status & DAS6402_STATUS_FFULL) { |
| 187 | async->events |= COMEDI_CB_OVERFLOW; | 187 | async->events |= COMEDI_CB_OVERFLOW; |
| 188 | } else if (status & DAS6402_STATUS_FFNE) { | 188 | } else if (status & DAS6402_STATUS_FFNE) { |
| 189 | unsigned int val; | 189 | unsigned short val; |
| 190 | 190 | ||
| 191 | val = das6402_ai_read_sample(dev, s); | 191 | val = das6402_ai_read_sample(dev, s); |
| 192 | comedi_buf_write_samples(s, &val, 1); | 192 | comedi_buf_write_samples(s, &val, 1); |
| @@ -427,7 +427,7 @@ static irqreturn_t das800_interrupt(int irq, void *d) · 426 unmodified lines | |||
| 427 | struct comedi_cmd *cmd; | 427 | struct comedi_cmd *cmd; |
| 428 | unsigned long irq_flags; | 428 | unsigned long irq_flags; |
| 429 | unsigned int status; | 429 | unsigned int status; |
| 430 | unsigned int val; | 430 | unsigned short val; |
| 431 | bool fifo_empty; | 431 | bool fifo_empty; |
| 432 | bool fifo_overflow; | 432 | bool fifo_overflow; |
| 433 | int i; | 433 | int i; |
| @@ -404,7 +404,7 @@ static irqreturn_t dmm32at_isr(int irq, void *d) · 403 unmodified lines | |||
| 404 | { | 404 | { |
| 405 | struct comedi_device *dev = d; | 405 | struct comedi_device *dev = d; |
| 406 | unsigned char intstat; | 406 | unsigned char intstat; |
| 407 | unsigned int val; | 407 | unsigned short val; |
| 408 | int i; | 408 | int i; |
| 409 | 409 | ||
| 410 | if (!dev->attached) { | 410 | if (!dev->attached) { |
| @@ -924,7 +924,7 @@ static irqreturn_t me4000_ai_isr(int irq, void *dev_id) · 923 unmodified lines | |||
| 924 | struct comedi_subdevice *s = dev->read_subdev; | 924 | struct comedi_subdevice *s = dev->read_subdev; |
| 925 | int i; | 925 | int i; |
| 926 | int c = 0; | 926 | int c = 0; |
| 927 | unsigned int lval; | 927 | unsigned short lval; |
| 928 | 928 | ||
| 929 | if (!dev->attached) | 929 | if (!dev->attached) |
| 930 | return IRQ_NONE; | 930 | return IRQ_NONE; |
| @@ -195,7 +195,9 @@ static irqreturn_t ni6527_interrupt(int irq, void *d) · 194 unmodified lines | |||
| 195 | return IRQ_NONE; | 195 | return IRQ_NONE; |
| 196 | 196 | ||
| 197 | if (status & NI6527_STATUS_EDGE) { | 197 | if (status & NI6527_STATUS_EDGE) { |
| 198 | comedi_buf_write_samples(s, &s->state, 1); | 198 | unsigned short val = 0; |
| 199 | |||
| 200 | comedi_buf_write_samples(s, &val, 1); | ||
| 199 | comedi_handle_events(dev, s); | 201 | comedi_handle_events(dev, s); |
| 200 | } | 202 | } |
| 201 | 203 | ||
| @@ -472,6 +472,7 @@ static irqreturn_t ni_65xx_interrupt(int irq, void *d) · 471 unmodified lines | |||
| 472 | struct comedi_device *dev = d; | 472 | struct comedi_device *dev = d; |
| 473 | struct comedi_subdevice *s = dev->read_subdev; | 473 | struct comedi_subdevice *s = dev->read_subdev; |
| 474 | unsigned int status; | 474 | unsigned int status; |
| 475 | unsigned short val = 0; | ||
| 475 | 476 | ||
| 476 | status = readb(dev->mmio + NI_65XX_STATUS_REG); | 477 | status = readb(dev->mmio + NI_65XX_STATUS_REG); |
| 477 | if ((status & NI_65XX_STATUS_INT) == 0) | 478 | if ((status & NI_65XX_STATUS_INT) == 0) |
| @@ -482,7 +483,7 @@ static irqreturn_t ni_65xx_interrupt(int irq, void *d) · 4 unmodified lines | |||
| 482 | writeb(NI_65XX_CLR_EDGE_INT | NI_65XX_CLR_OVERFLOW_INT, | 483 | writeb(NI_65XX_CLR_EDGE_INT | NI_65XX_CLR_OVERFLOW_INT, |
| 483 | dev->mmio + NI_65XX_CLR_REG); | 484 | dev->mmio + NI_65XX_CLR_REG); |
| 484 | 485 | ||
| 485 | comedi_buf_write_samples(s, &s->state, 1); | 486 | comedi_buf_write_samples(s, &val, 1); |
| 486 | comedi_handle_events(dev, s); | 487 | comedi_handle_events(dev, s); |
| 487 | 488 | ||
| 488 | return IRQ_HANDLED; | 489 | return IRQ_HANDLED; |
| @@ -184,7 +184,7 @@ static irqreturn_t pcl711_interrupt(int irq, void *d) · 183 unmodified lines | |||
| 184 | struct comedi_device *dev = d; | 184 | struct comedi_device *dev = d; |
| 185 | struct comedi_subdevice *s = dev->read_subdev; | 185 | struct comedi_subdevice *s = dev->read_subdev; |
| 186 | struct comedi_cmd *cmd = &s->async->cmd; | 186 | struct comedi_cmd *cmd = &s->async->cmd; |
| 187 | unsigned int data; | 187 | unsigned short data; |
| 188 | 188 | ||
| 189 | if (!dev->attached) { | 189 | if (!dev->attached) { |
| 190 | dev_err(dev->class_dev, "spurious interrupt\n"); | 190 | dev_err(dev->class_dev, "spurious interrupt\n"); |
| @@ -220,9 +220,11 @@ static irqreturn_t pcl726_interrupt(int irq, void *d) · 219 unmodified lines | |||
| 220 | struct pcl726_private *devpriv = dev->private; | 220 | struct pcl726_private *devpriv = dev->private; |
| 221 | 221 | ||
| 222 | if (devpriv->cmd_running) { | 222 | if (devpriv->cmd_running) { |
| 223 | unsigned short val = 0; | ||
| 224 | |||
| 223 | pcl726_intr_cancel(dev, s); | 225 | pcl726_intr_cancel(dev, s); |
| 224 | 226 | ||
| 225 | comedi_buf_write_samples(s, &s->state, 1); | 227 | comedi_buf_write_samples(s, &val, 1); |
| 226 | comedi_handle_events(dev, s); | 228 | comedi_handle_events(dev, s); |
| 227 | } | 229 | } |
| 228 | 230 | ||
| @@ -423,7 +423,7 @@ static int pcl818_ai_eoc(struct comedi_device *dev, · 422 unmodified lines | |||
| 423 | 423 | ||
| 424 | static bool pcl818_ai_write_sample(struct comedi_device *dev, | 424 | static bool pcl818_ai_write_sample(struct comedi_device *dev, |
| 425 | struct comedi_subdevice *s, | 425 | struct comedi_subdevice *s, |
| 426 | unsigned int chan, unsigned int val) | 426 | unsigned int chan, unsigned short val) |
| 427 | { | 427 | { |
| 428 | struct pcl818_private *devpriv = dev->private; | 428 | struct pcl818_private *devpriv = dev->private; |
| 429 | struct comedi_cmd *cmd = &s->async->cmd; | 429 | struct comedi_cmd *cmd = &s->async->cmd; |
| @@ -1120,6 +1120,7 @@ static int ks_wlan_set_scan(struct net_device *dev, · 1119 unmodified lines | |||
| 1120 | { | 1120 | { |
| 1121 | struct ks_wlan_private *priv = netdev_priv(dev); | 1121 | struct ks_wlan_private *priv = netdev_priv(dev); |
| 1122 | struct iw_scan_req *req = NULL; | 1122 | struct iw_scan_req *req = NULL; |
| 1123 | int len; | ||
| 1123 | 1124 | ||
| 1124 | if (priv->sleep_mode == SLP_SLEEP) | 1125 | if (priv->sleep_mode == SLP_SLEEP) |
| 1125 | return -EPERM; | 1126 | return -EPERM; |
| @@ -1129,8 +1130,9 @@ static int ks_wlan_set_scan(struct net_device *dev, · 3 unmodified lines | |||
| 1129 | if (wrqu->data.length == sizeof(struct iw_scan_req) && | 1130 | if (wrqu->data.length == sizeof(struct iw_scan_req) && |
| 1130 | wrqu->data.flags & IW_SCAN_THIS_ESSID) { | 1131 | wrqu->data.flags & IW_SCAN_THIS_ESSID) { |
| 1131 | req = (struct iw_scan_req *)extra; | 1132 | req = (struct iw_scan_req *)extra; |
| 1132 | priv->scan_ssid_len = req->essid_len; | 1133 | len = min_t(int, req->essid_len, IW_ESSID_MAX_SIZE); |
| 1133 | memcpy(priv->scan_ssid, req->essid, priv->scan_ssid_len); | 1134 | priv->scan_ssid_len = len; |
| 1135 | memcpy(priv->scan_ssid, req->essid, len); | ||
| 1134 | } else { | 1136 | } else { |
| 1135 | priv->scan_ssid_len = 0; | 1137 | priv->scan_ssid_len = 0; |
| 1136 | } | 1138 | } |
| @@ -791,6 +791,7 @@ int rtw_check_beacon_data(struct adapter *padapter, u8 *pbuf, int len) · 790 unmodified lines | |||
| 791 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_SSID, &ie_len, | 791 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_SSID, &ie_len, |
| 792 | pbss_network->ie_length - _BEACON_IE_OFFSET_); | 792 | pbss_network->ie_length - _BEACON_IE_OFFSET_); |
| 793 | if (p && ie_len > 0) { | 793 | if (p && ie_len > 0) { |
| 794 | ie_len = min_t(int, ie_len, sizeof(pbss_network->ssid.ssid)); | ||
| 794 | memset(&pbss_network->ssid, 0, sizeof(struct ndis_802_11_ssid)); | 795 | memset(&pbss_network->ssid, 0, sizeof(struct ndis_802_11_ssid)); |
| 795 | memcpy(pbss_network->ssid.ssid, p + 2, ie_len); | 796 | memcpy(pbss_network->ssid.ssid, p + 2, ie_len); |
| 796 | pbss_network->ssid.ssid_length = ie_len; | 797 | pbss_network->ssid.ssid_length = ie_len; |
| @@ -811,6 +812,7 @@ int rtw_check_beacon_data(struct adapter *padapter, u8 *pbuf, int len) · 14 unmodified lines | |||
| 811 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_SUPP_RATES, &ie_len, | 812 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_SUPP_RATES, &ie_len, |
| 812 | pbss_network->ie_length - _BEACON_IE_OFFSET_); | 813 | pbss_network->ie_length - _BEACON_IE_OFFSET_); |
| 813 | if (p) { | 814 | if (p) { |
| 815 | ie_len = min_t(int, ie_len, NDIS_802_11_LENGTH_RATES_EX); | ||
| 814 | memcpy(supportRate, p + 2, ie_len); | 816 | memcpy(supportRate, p + 2, ie_len); |
| 815 | supportRateNum = ie_len; | 817 | supportRateNum = ie_len; |
| 816 | } | 818 | } |
| @@ -819,6 +821,8 @@ int rtw_check_beacon_data(struct adapter *padapter, u8 *pbuf, int len) · 2 unmodified lines | |||
| 819 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_EXT_SUPP_RATES, | 821 | p = rtw_get_ie(ie + _BEACON_IE_OFFSET_, WLAN_EID_EXT_SUPP_RATES, |
| 820 | &ie_len, pbss_network->ie_length - _BEACON_IE_OFFSET_); | 822 | &ie_len, pbss_network->ie_length - _BEACON_IE_OFFSET_); |
| 821 | if (p) { | 823 | if (p) { |
| 824 | ie_len = min_t(int, ie_len, | ||
| 825 | NDIS_802_11_LENGTH_RATES_EX - supportRateNum); | ||
| 822 | memcpy(supportRate + supportRateNum, p + 2, ie_len); | 826 | memcpy(supportRate + supportRateNum, p + 2, ie_len); |
| 823 | supportRateNum += ie_len; | 827 | supportRateNum += ie_len; |
| 824 | } | 828 | } |
| @@ -934,6 +938,7 @@ int rtw_check_beacon_data(struct adapter *padapter, u8 *pbuf, int len) · 109 unmodified lines | |||
| 934 | 938 | ||
| 935 | pht_cap->mcs.rx_mask[0] = 0xff; | 939 | pht_cap->mcs.rx_mask[0] = 0xff; |
| 936 | pht_cap->mcs.rx_mask[1] = 0x0; | 940 | pht_cap->mcs.rx_mask[1] = 0x0; |
| 941 | ie_len = min_t(int, ie_len, sizeof(pmlmepriv->htpriv.ht_cap)); | ||
| 937 | memcpy(&pmlmepriv->htpriv.ht_cap, p + 2, ie_len); | 942 | memcpy(&pmlmepriv->htpriv.ht_cap, p + 2, ie_len); |
| 938 | } | 943 | } |
| 939 | 944 | ||
| @@ -1133,9 +1133,11 @@ static int rtw_wx_set_scan(struct net_device *dev, struct iw_request_info *a, · 1132 unmodified lines | |||
| 1133 | break; | 1133 | break; |
| 1134 | } | 1134 | } |
| 1135 | sec_len = *(pos++); len -= 1; | 1135 | sec_len = *(pos++); len -= 1; |
| 1136 | if (sec_len > 0 && sec_len <= len) { | 1136 | if (sec_len > 0 && |
| 1137 | sec_len <= len && | ||
| 1138 | sec_len <= 32) { | ||
| 1137 | ssid[ssid_index].ssid_length = sec_len; | 1139 | ssid[ssid_index].ssid_length = sec_len; |
| 1138 | memcpy(ssid[ssid_index].ssid, pos, ssid[ssid_index].ssid_length); | 1140 | memcpy(ssid[ssid_index].ssid, pos, sec_len); |
| 1139 | ssid_index++; | 1141 | ssid_index++; |
| 1140 | } | 1142 | } |
| 1141 | pos += sec_len; | 1143 | pos += sec_len; |
| @@ -27,6 +27,7 @@ config RTLLIB_CRYPTO_CCMP · 26 unmodified lines | |||
| 27 | config RTLLIB_CRYPTO_TKIP | 27 | config RTLLIB_CRYPTO_TKIP |
| 28 | tristate "Support for rtllib TKIP crypto" | 28 | tristate "Support for rtllib TKIP crypto" |
| 29 | depends on RTLLIB | 29 | depends on RTLLIB |
| 30 | select CRYPTO | ||
| 30 | select CRYPTO_LIB_ARC4 | 31 | select CRYPTO_LIB_ARC4 |
| 31 | select CRYPTO_MICHAEL_MIC | 32 | select CRYPTO_MICHAEL_MIC |
| 32 | default y | 33 | default y |
| @@ -406,9 +406,10 @@ static int _rtl92e_wx_set_scan(struct net_device *dev, · 405 unmodified lines | |||
| 406 | struct iw_scan_req *req = (struct iw_scan_req *)b; | 406 | struct iw_scan_req *req = (struct iw_scan_req *)b; |
| 407 | 407 | ||
| 408 | if (req->essid_len) { | 408 | if (req->essid_len) { |
| 409 | ieee->current_network.ssid_len = req->essid_len; | 409 | int len = min_t(int, req->essid_len, IW_ESSID_MAX_SIZE); |
| 410 | memcpy(ieee->current_network.ssid, req->essid, | 410 | |
| 411 | req->essid_len); | 411 | ieee->current_network.ssid_len = len; |
| 412 | memcpy(ieee->current_network.ssid, req->essid, len); | ||
| 412 | } | 413 | } |
| 413 | } | 414 | } |
| 414 | 415 | ||
| @@ -331,8 +331,10 @@ static int r8192_wx_set_scan(struct net_device *dev, struct iw_request_info *a, · 330 unmodified lines | |||
| 331 | struct iw_scan_req *req = (struct iw_scan_req *)b; | 331 | struct iw_scan_req *req = (struct iw_scan_req *)b; |
| 332 | 332 | ||
| 333 | if (req->essid_len) { | 333 | if (req->essid_len) { |
| 334 | ieee->current_network.ssid_len = req->essid_len; | 334 | int len = min_t(int, req->essid_len, IW_ESSID_MAX_SIZE); |
| 335 | memcpy(ieee->current_network.ssid, req->essid, req->essid_len); | 335 | |
| 336 | ieee->current_network.ssid_len = len; | ||
| 337 | memcpy(ieee->current_network.ssid, req->essid, len); | ||
| 336 | } | 338 | } |
| 337 | } | 339 | } |
| 338 | 340 | ||
| @@ -192,8 +192,10 @@ u8 r8712_sitesurvey_cmd(struct _adapter *padapter, · 191 unmodified lines | |||
| 192 | psurveyPara->ss_ssidlen = 0; | 192 | psurveyPara->ss_ssidlen = 0; |
| 193 | memset(psurveyPara->ss_ssid, 0, IW_ESSID_MAX_SIZE + 1); | 193 | memset(psurveyPara->ss_ssid, 0, IW_ESSID_MAX_SIZE + 1); |
| 194 | if (pssid && pssid->SsidLength) { | 194 | if (pssid && pssid->SsidLength) { |
| 195 | memcpy(psurveyPara->ss_ssid, pssid->Ssid, pssid->SsidLength); | 195 | int len = min_t(int, pssid->SsidLength, IW_ESSID_MAX_SIZE); |
| 196 | psurveyPara->ss_ssidlen = cpu_to_le32(pssid->SsidLength); | 196 | |
| 197 | memcpy(psurveyPara->ss_ssid, pssid->Ssid, len); | ||
| 198 | psurveyPara->ss_ssidlen = cpu_to_le32(len); | ||
| 197 | } | 199 | } |
| 198 | set_fwstate(pmlmepriv, _FW_UNDER_SURVEY); | 200 | set_fwstate(pmlmepriv, _FW_UNDER_SURVEY); |
| 199 | r8712_enqueue_cmd(pcmdpriv, ph2c); | 201 | r8712_enqueue_cmd(pcmdpriv, ph2c); |
| @@ -924,7 +924,7 @@ static int r871x_wx_set_priv(struct net_device *dev, · 923 unmodified lines | |||
| 924 | struct iw_point *dwrq = (struct iw_point *)awrq; | 924 | struct iw_point *dwrq = (struct iw_point *)awrq; |
| 925 | 925 | ||
| 926 | len = dwrq->length; | 926 | len = dwrq->length; |
| 927 | ext = memdup_user(dwrq->pointer, len); | 927 | ext = strndup_user(dwrq->pointer, len); |
| 928 | if (IS_ERR(ext)) | 928 | if (IS_ERR(ext)) |
| 929 | return PTR_ERR(ext); | 929 | return PTR_ERR(ext); |
| 930 | 930 | ||
| @@ -5,6 +5,7 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/gpio/consumer.h> | ||
| 8 | #include <net/mac80211.h> | 9 | #include <net/mac80211.h> |
| 9 | 10 | ||
| 10 | #include "bh.h" | 11 | #include "bh.h" |
| @@ -8,6 +8,10 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_BH_H | 8 | #ifndef WFX_BH_H |
| 9 | #define WFX_BH_H | 9 | #define WFX_BH_H |
| 10 | 10 | ||
| 11 | #include <linux/atomic.h> | ||
| 12 | #include <linux/wait.h> | ||
| 13 | #include <linux/workqueue.h> | ||
| 14 | |||
| 11 | struct wfx_dev; | 15 | struct wfx_dev; |
| 12 | 16 | ||
| 13 | struct wfx_hif { | 17 | struct wfx_hif { |
| @@ -8,6 +8,9 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_BUS_H | 8 | #ifndef WFX_BUS_H |
| 9 | #define WFX_BUS_H | 9 | #define WFX_BUS_H |
| 10 | 10 | ||
| 11 | #include <linux/mmc/sdio_func.h> | ||
| 12 | #include <linux/spi/spi.h> | ||
| 13 | |||
| 11 | #define WFX_REG_CONFIG 0x0 | 14 | #define WFX_REG_CONFIG 0x0 |
| 12 | #define WFX_REG_CONTROL 0x1 | 15 | #define WFX_REG_CONTROL 0x1 |
| 13 | #define WFX_REG_IN_OUT_QUEUE 0x2 | 16 | #define WFX_REG_IN_OUT_QUEUE 0x2 |
| @@ -5,13 +5,19 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/module.h> | ||
| 8 | #include <linux/mmc/sdio.h> | 9 | #include <linux/mmc/sdio.h> |
| 9 | #include <linux/mmc/sdio_func.h> | 10 | #include <linux/mmc/sdio_func.h> |
| 10 | #include <linux/mmc/card.h> | 11 | #include <linux/mmc/card.h> |
| 12 | #include <linux/interrupt.h> | ||
| 11 | #include <linux/of_irq.h> | 13 | #include <linux/of_irq.h> |
| 14 | #include <linux/irq.h> | ||
| 12 | 15 | ||
| 13 | #include "bus.h" | 16 | #include "bus.h" |
| 14 | #include "wfx.h" | 17 | #include "wfx.h" |
| 18 | #include "hwio.h" | ||
| 19 | #include "main.h" | ||
| 20 | #include "bh.h" | ||
| 15 | 21 | ||
| 16 | static const struct wfx_platform_data wfx_sdio_pdata = { | 22 | static const struct wfx_platform_data wfx_sdio_pdata = { |
| 17 | .file_fw = "wfm_wf200", | 23 | .file_fw = "wfm_wf200", |
| @@ -6,12 +6,19 @@ · 5 unmodified lines | |||
| 6 | * Copyright (c) 2011, Sagrad Inc. | 6 | * Copyright (c) 2011, Sagrad Inc. |
| 7 | * Copyright (c) 2010, ST-Ericsson | 7 | * Copyright (c) 2010, ST-Ericsson |
| 8 | */ | 8 | */ |
| 9 | #include <linux/module.h> | ||
| 10 | #include <linux/delay.h> | ||
| 11 | #include <linux/gpio/consumer.h> | ||
| 9 | #include <linux/spi/spi.h> | 12 | #include <linux/spi/spi.h> |
| 13 | #include <linux/interrupt.h> | ||
| 10 | #include <linux/irq.h> | 14 | #include <linux/irq.h> |
| 11 | #include <linux/of.h> | 15 | #include <linux/of.h> |
| 12 | 16 | ||
| 13 | #include "bus.h" | 17 | #include "bus.h" |
| 14 | #include "wfx.h" | 18 | #include "wfx.h" |
| 19 | #include "hwio.h" | ||
| 20 | #include "main.h" | ||
| 21 | #include "bh.h" | ||
| 15 | 22 | ||
| 16 | #define SET_WRITE 0x7FFF /* usage: and operation */ | 23 | #define SET_WRITE 0x7FFF /* usage: and operation */ |
| 17 | #define SET_READ 0x8000 /* usage: or operation */ | 24 | #define SET_READ 0x8000 /* usage: or operation */ |
| @@ -5,8 +5,13 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/etherdevice.h> | ||
| 9 | #include <net/mac80211.h> | ||
| 10 | |||
| 8 | #include "data_rx.h" | 11 | #include "data_rx.h" |
| 9 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "bh.h" | ||
| 14 | #include "sta.h" | ||
| 10 | 15 | ||
| 11 | static void wfx_rx_handle_ba(struct wfx_vif *wvif, struct ieee80211_mgmt *mgmt) | 16 | static void wfx_rx_handle_ba(struct wfx_vif *wvif, struct ieee80211_mgmt *mgmt) |
| 12 | { | 17 | { |
| @@ -6,9 +6,14 @@ · 5 unmodified lines | |||
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <net/mac80211.h> | 8 | #include <net/mac80211.h> |
| 9 | #include <linux/etherdevice.h> | ||
| 9 | 10 | ||
| 11 | #include "data_tx.h" | ||
| 10 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "bh.h" | ||
| 11 | #include "sta.h" | 14 | #include "sta.h" |
| 15 | #include "queue.h" | ||
| 16 | #include "debug.h" | ||
| 12 | #include "traces.h" | 17 | #include "traces.h" |
| 13 | #include "hif_tx_mib.h" | 18 | #include "hif_tx_mib.h" |
| 14 | 19 | ||
| @@ -8,6 +8,9 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_DATA_TX_H | 8 | #ifndef WFX_DATA_TX_H |
| 9 | #define WFX_DATA_TX_H | 9 | #define WFX_DATA_TX_H |
| 10 | 10 | ||
| 11 | #include <linux/list.h> | ||
| 12 | #include <net/mac80211.h> | ||
| 13 | |||
| 11 | #include "hif_api_cmd.h" | 14 | #include "hif_api_cmd.h" |
| 12 | #include "hif_api_mib.h" | 15 | #include "hif_api_mib.h" |
| 13 | 16 | ||
| @@ -5,9 +5,15 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/debugfs.h> | ||
| 9 | #include <linux/seq_file.h> | ||
| 10 | #include <linux/crc32.h> | ||
| 11 | |||
| 8 | #include "debug.h" | 12 | #include "debug.h" |
| 9 | #include "wfx.h" | 13 | #include "wfx.h" |
| 10 | #include "sta.h" | 14 | #include "sta.h" |
| 15 | #include "main.h" | ||
| 16 | #include "hif_tx.h" | ||
| 11 | #include "hif_tx_mib.h" | 17 | #include "hif_tx_mib.h" |
| 12 | 18 | ||
| 13 | #define CREATE_TRACE_POINTS | 19 | #define CREATE_TRACE_POINTS |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | |||
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/firmware.h> | 8 | #include <linux/firmware.h> |
| 9 | #include <linux/slab.h> | ||
| 10 | #include <linux/mm.h> | ||
| 9 | #include <linux/bitfield.h> | 11 | #include <linux/bitfield.h> |
| 10 | 12 | ||
| 11 | #include "fwio.h" | 13 | #include "fwio.h" |
| @@ -8,6 +8,10 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_HIF_API_CMD_H | 8 | #ifndef WFX_HIF_API_CMD_H |
| 9 | #define WFX_HIF_API_CMD_H | 9 | #define WFX_HIF_API_CMD_H |
| 10 | 10 | ||
| 11 | #include <linux/ieee80211.h> | ||
| 12 | |||
| 13 | #include "hif_api_general.h" | ||
| 14 | |||
| 11 | enum hif_requests_ids { | 15 | enum hif_requests_ids { |
| 12 | HIF_REQ_ID_RESET = 0x0a, | 16 | HIF_REQ_ID_RESET = 0x0a, |
| 13 | HIF_REQ_ID_READ_MIB = 0x05, | 17 | HIF_REQ_ID_READ_MIB = 0x05, |
| @@ -8,6 +8,15 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_HIF_API_GENERAL_H | 8 | #ifndef WFX_HIF_API_GENERAL_H |
| 9 | #define WFX_HIF_API_GENERAL_H | 9 | #define WFX_HIF_API_GENERAL_H |
| 10 | 10 | ||
| 11 | #ifdef __KERNEL__ | ||
| 12 | #include <linux/types.h> | ||
| 13 | #include <linux/if_ether.h> | ||
| 14 | #else | ||
| 15 | #include <net/ethernet.h> | ||
| 16 | #include <stdint.h> | ||
| 17 | #define __packed __attribute__((__packed__)) | ||
| 18 | #endif | ||
| 19 | |||
| 11 | #define HIF_ID_IS_INDICATION 0x80 | 20 | #define HIF_ID_IS_INDICATION 0x80 |
| 12 | #define HIF_COUNTER_MAX 7 | 21 | #define HIF_COUNTER_MAX 7 |
| 13 | 22 | ||
| @@ -6,7 +6,11 @@ · 5 unmodified lines | |||
| 6 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 6 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 7 | * Copyright (c) 2010, ST-Ericsson | 7 | * Copyright (c) 2010, ST-Ericsson |
| 8 | */ | 8 | */ |
| 9 | #include <linux/etherdevice.h> | ||
| 10 | |||
| 11 | #include "hif_tx.h" | ||
| 9 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "bh.h" | ||
| 10 | #include "hwio.h" | 14 | #include "hwio.h" |
| 11 | #include "debug.h" | 15 | #include "debug.h" |
| 12 | #include "sta.h" | 16 | #include "sta.h" |
| @@ -6,8 +6,13 @@ · 5 unmodified lines | |||
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | * Copyright (C) 2010, ST-Ericsson SA | 7 | * Copyright (C) 2010, ST-Ericsson SA |
| 8 | */ | 8 | */ |
| 9 | |||
| 10 | #include <linux/etherdevice.h> | ||
| 11 | |||
| 9 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "hif_tx.h" | ||
| 10 | #include "hif_tx_mib.h" | 14 | #include "hif_tx_mib.h" |
| 15 | #include "hif_api_mib.h" | ||
| 11 | 16 | ||
| 12 | int hif_set_output_power(struct wfx_vif *wvif, int val) | 17 | int hif_set_output_power(struct wfx_vif *wvif, int val) |
| 13 | { | 18 | { |
| @@ -5,10 +5,13 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/kernel.h> | ||
| 9 | #include <linux/delay.h> | ||
| 8 | #include <linux/slab.h> | 10 | #include <linux/slab.h> |
| 9 | 11 | ||
| 10 | #include "hwio.h" | 12 | #include "hwio.h" |
| 11 | #include "wfx.h" | 13 | #include "wfx.h" |
| 14 | #include "bus.h" | ||
| 12 | #include "traces.h" | 15 | #include "traces.h" |
| 13 | 16 | ||
| 14 | /* | 17 | /* |
| @@ -8,6 +8,8 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_HWIO_H | 8 | #ifndef WFX_HWIO_H |
| 9 | #define WFX_HWIO_H | 9 | #define WFX_HWIO_H |
| 10 | 10 | ||
| 11 | #include <linux/types.h> | ||
| 12 | |||
| 11 | struct wfx_dev; | 13 | struct wfx_dev; |
| 12 | 14 | ||
| 13 | int wfx_data_read(struct wfx_dev *wdev, void *buf, size_t buf_len); | 15 | int wfx_data_read(struct wfx_dev *wdev, void *buf, size_t buf_len); |
| @@ -5,10 +5,12 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/etherdevice.h> | ||
| 8 | #include <net/mac80211.h> | 9 | #include <net/mac80211.h> |
| 9 | 10 | ||
| 10 | #include "key.h" | 11 | #include "key.h" |
| 11 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "hif_tx_mib.h" | ||
| 12 | 14 | ||
| 13 | static int wfx_alloc_key(struct wfx_dev *wdev) | 15 | static int wfx_alloc_key(struct wfx_dev *wdev) |
| 14 | { | 16 | { |
| @@ -8,6 +8,8 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_KEY_H | 8 | #ifndef WFX_KEY_H |
| 9 | #define WFX_KEY_H | 9 | #define WFX_KEY_H |
| 10 | 10 | ||
| 11 | #include <net/mac80211.h> | ||
| 12 | |||
| 11 | struct wfx_dev; | 13 | struct wfx_dev; |
| 12 | struct wfx_vif; | 14 | struct wfx_vif; |
| 13 | 15 | ||
| @@ -10,21 +10,28 @@ · 9 unmodified lines | |||
| 10 | * Copyright (c) 2006, Michael Wu <flamingice@sourmilk.net> | 10 | * Copyright (c) 2006, Michael Wu <flamingice@sourmilk.net> |
| 11 | * Copyright (c) 2004-2006 Jean-Baptiste Note <jbnote@gmail.com>, et al. | 11 | * Copyright (c) 2004-2006 Jean-Baptiste Note <jbnote@gmail.com>, et al. |
| 12 | */ | 12 | */ |
| 13 | #include <linux/module.h> | ||
| 13 | #include <linux/of.h> | 14 | #include <linux/of.h> |
| 14 | #include <linux/of_net.h> | 15 | #include <linux/of_net.h> |
| 16 | #include <linux/gpio/consumer.h> | ||
| 15 | #include <linux/mmc/sdio_func.h> | 17 | #include <linux/mmc/sdio_func.h> |
| 16 | #include <linux/spi/spi.h> | 18 | #include <linux/spi/spi.h> |
| 19 | #include <linux/etherdevice.h> | ||
| 17 | #include <linux/firmware.h> | 20 | #include <linux/firmware.h> |
| 18 | 21 | ||
| 22 | #include "main.h" | ||
| 19 | #include "wfx.h" | 23 | #include "wfx.h" |
| 20 | #include "fwio.h" | 24 | #include "fwio.h" |
| 21 | #include "hwio.h" | 25 | #include "hwio.h" |
| 22 | #include "bus.h" | 26 | #include "bus.h" |
| 27 | #include "bh.h" | ||
| 23 | #include "sta.h" | 28 | #include "sta.h" |
| 24 | #include "key.h" | 29 | #include "key.h" |
| 25 | #include "scan.h" | 30 | #include "scan.h" |
| 26 | #include "debug.h" | 31 | #include "debug.h" |
| 32 | #include "data_tx.h" | ||
| 27 | #include "hif_tx_mib.h" | 33 | #include "hif_tx_mib.h" |
| 34 | #include "hif_api_cmd.h" | ||
| 28 | 35 | ||
| 29 | #define WFX_PDS_MAX_SIZE 1500 | 36 | #define WFX_PDS_MAX_SIZE 1500 |
| 30 | 37 | ||
| @@ -10,8 +10,11 @@ · 9 unmodified lines | |||
| 10 | #ifndef WFX_MAIN_H | 10 | #ifndef WFX_MAIN_H |
| 11 | #define WFX_MAIN_H | 11 | #define WFX_MAIN_H |
| 12 | 12 | ||
| 13 | #include <linux/device.h> | ||
| 13 | #include <linux/gpio/consumer.h> | 14 | #include <linux/gpio/consumer.h> |
| 14 | 15 | ||
| 16 | #include "hif_api_general.h" | ||
| 17 | |||
| 15 | struct wfx_dev; | 18 | struct wfx_dev; |
| 16 | struct hwbus_ops; | 19 | struct hwbus_ops; |
| 17 | 20 | ||
| @@ -5,9 +5,13 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/sched.h> | ||
| 8 | #include <net/mac80211.h> | 9 | #include <net/mac80211.h> |
| 9 | 10 | ||
| 11 | #include "queue.h" | ||
| 10 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "sta.h" | ||
| 14 | #include "data_tx.h" | ||
| 11 | #include "traces.h" | 15 | #include "traces.h" |
| 12 | 16 | ||
| 13 | void wfx_tx_lock(struct wfx_dev *wdev) | 17 | void wfx_tx_lock(struct wfx_dev *wdev) |
| @@ -8,6 +8,9 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_QUEUE_H | 8 | #ifndef WFX_QUEUE_H |
| 9 | #define WFX_QUEUE_H | 9 | #define WFX_QUEUE_H |
| 10 | 10 | ||
| 11 | #include <linux/skbuff.h> | ||
| 12 | #include <linux/atomic.h> | ||
| 13 | |||
| 11 | struct wfx_dev; | 14 | struct wfx_dev; |
| 12 | struct wfx_vif; | 15 | struct wfx_vif; |
| 13 | 16 | ||
| @@ -8,6 +8,8 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_SCAN_H | 8 | #ifndef WFX_SCAN_H |
| 9 | #define WFX_SCAN_H | 9 | #define WFX_SCAN_H |
| 10 | 10 | ||
| 11 | #include <net/mac80211.h> | ||
| 12 | |||
| 11 | struct wfx_dev; | 13 | struct wfx_dev; |
| 12 | struct wfx_vif; | 14 | struct wfx_vif; |
| 13 | 15 | ||
| @@ -5,11 +5,17 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. | 5 | * Copyright (c) 2017-2020, Silicon Laboratories, Inc. |
| 6 | * Copyright (c) 2010, ST-Ericsson | 6 | * Copyright (c) 2010, ST-Ericsson |
| 7 | */ | 7 | */ |
| 8 | #include <linux/etherdevice.h> | ||
| 8 | #include <net/mac80211.h> | 9 | #include <net/mac80211.h> |
| 9 | 10 | ||
| 10 | #include "sta.h" | 11 | #include "sta.h" |
| 11 | #include "wfx.h" | 12 | #include "wfx.h" |
| 13 | #include "fwio.h" | ||
| 14 | #include "bh.h" | ||
| 15 | #include "key.h" | ||
| 12 | #include "scan.h" | 16 | #include "scan.h" |
| 17 | #include "debug.h" | ||
| 18 | #include "hif_tx.h" | ||
| 13 | #include "hif_tx_mib.h" | 19 | #include "hif_tx_mib.h" |
| 14 | 20 | ||
| 15 | #define HIF_MAX_ARP_IP_ADDRTABLE_ENTRIES 2 | 21 | #define HIF_MAX_ARP_IP_ADDRTABLE_ENTRIES 2 |
| @@ -8,6 +8,8 @@ · 7 unmodified lines | |||
| 8 | #ifndef WFX_STA_H | 8 | #ifndef WFX_STA_H |
| 9 | #define WFX_STA_H | 9 | #define WFX_STA_H |
| 10 | 10 | ||
| 11 | #include <net/mac80211.h> | ||
| 12 | |||
| 11 | struct wfx_dev; | 13 | struct wfx_dev; |
| 12 | struct wfx_vif; | 14 | struct wfx_vif; |
| 13 | 15 | ||
| @@ -12,8 +12,11 @@ · 11 unmodified lines | |||
| 12 | #define _WFX_TRACE_H | 12 | #define _WFX_TRACE_H |
| 13 | 13 | ||
| 14 | #include <linux/tracepoint.h> | 14 | #include <linux/tracepoint.h> |
| 15 | #include <net/mac80211.h> | ||
| 15 | 16 | ||
| 16 | #include "bus.h" | 17 | #include "bus.h" |
| 18 | #include "hif_api_cmd.h" | ||
| 19 | #include "hif_api_mib.h" | ||
| 17 | 20 | ||
| 18 | /* The hell below need some explanations. For each symbolic number, we need to | 21 | /* The hell below need some explanations. For each symbolic number, we need to |
| 19 | * define it with TRACE_DEFINE_ENUM() and in a list for __print_symbolic. | 22 | * define it with TRACE_DEFINE_ENUM() and in a list for __print_symbolic. |
| @@ -10,6 +10,9 @@ · 9 unmodified lines | |||
| 10 | #ifndef WFX_H | 10 | #ifndef WFX_H |
| 11 | #define WFX_H | 11 | #define WFX_H |
| 12 | 12 | ||
| 13 | #include <linux/completion.h> | ||
| 14 | #include <linux/workqueue.h> | ||
| 15 | #include <linux/mutex.h> | ||
| 13 | #include <linux/nospec.h> | 16 | #include <linux/nospec.h> |
| 14 | #include <net/mac80211.h> | 17 | #include <net/mac80211.h> |
| 15 | 18 | ||
| @@ -317,7 +317,6 @@ static void hvcs_hangup(struct tty_struct * tty); · 316 unmodified lines | |||
| 317 | 317 | ||
| 318 | static int hvcs_probe(struct vio_dev *dev, | 318 | static int hvcs_probe(struct vio_dev *dev, |
| 319 | const struct vio_device_id *id); | 319 | const struct vio_device_id *id); |
| 320 | static int hvcs_remove(struct vio_dev *dev); | ||
| 321 | static int __init hvcs_module_init(void); | 320 | static int __init hvcs_module_init(void); |
| 322 | static void __exit hvcs_module_exit(void); | 321 | static void __exit hvcs_module_exit(void); |
| 323 | static int hvcs_initialize(void); | 322 | static int hvcs_initialize(void); |
| @@ -819,7 +818,7 @@ static int hvcs_probe( · 495 unmodified lines | |||
| 819 | return 0; | 818 | return 0; |
| 820 | } | 819 | } |
| 821 | 820 | ||
| 822 | static int hvcs_remove(struct vio_dev *dev) | 821 | static void hvcs_remove(struct vio_dev *dev) |
| 823 | { | 822 | { |
| 824 | struct hvcs_struct *hvcsd = dev_get_drvdata(&dev->dev); | 823 | struct hvcs_struct *hvcsd = dev_get_drvdata(&dev->dev); |
| 825 | unsigned long flags; | 824 | unsigned long flags; |
| @@ -849,7 +848,6 @@ static int hvcs_remove(struct vio_dev *dev) · 23 unmodified lines | |||
| 849 | 848 | ||
| 850 | printk(KERN_INFO "HVCS: vty-server@%X removed from the" | 849 | printk(KERN_INFO "HVCS: vty-server@%X removed from the" |
| 851 | " vio bus.\n", dev->unit_address); | 850 | " vio bus.\n", dev->unit_address); |
| 852 | return 0; | ||
| 853 | }; | 851 | }; |
| 854 | 852 | ||
| 855 | static struct vio_driver hvcs_vio_driver = { | 853 | static struct vio_driver hvcs_vio_driver = { |
| @@ -66,8 +66,7 @@ static void pty_close(struct tty_struct *tty, struct file *filp) · 65 unmodified lines | |||
| 66 | wake_up_interruptible(&tty->link->read_wait); | 66 | wake_up_interruptible(&tty->link->read_wait); |
| 67 | wake_up_interruptible(&tty->link->write_wait); | 67 | wake_up_interruptible(&tty->link->write_wait); |
| 68 | if (tty->driver->subtype == PTY_TYPE_MASTER) { | 68 | if (tty->driver->subtype == PTY_TYPE_MASTER) { |
| 69 | struct file *f; | 69 | set_bit(TTY_OTHER_CLOSED, &tty->flags); |
| 70 | |||
| 71 | #ifdef CONFIG_UNIX98_PTYS | 70 | #ifdef CONFIG_UNIX98_PTYS |
| 72 | if (tty->driver == ptm_driver) { | 71 | if (tty->driver == ptm_driver) { |
| 73 | mutex_lock(&devpts_mutex); | 72 | mutex_lock(&devpts_mutex); |
| @@ -76,17 +75,7 @@ static void pty_close(struct tty_struct *tty, struct file *filp) · 2 unmodified lines | |||
| 76 | mutex_unlock(&devpts_mutex); | 75 | mutex_unlock(&devpts_mutex); |
| 77 | } | 76 | } |
| 78 | #endif | 77 | #endif |
| 79 | 78 | tty_vhangup(tty->link); | |
| 80 | /* | ||
| 81 | * This hack is required because a program can open a | ||
| 82 | * pty and redirect a console to it, but if the pty is | ||
| 83 | * closed and the console is not released, then the | ||
| 84 | * slave side will never close. So release the | ||
| 85 | * redirect when the master closes. | ||
| 86 | */ | ||
| 87 | f = tty_release_redirect(tty->link); | ||
| 88 | if (f) | ||
| 89 | fput(f); | ||
| 90 | } | 79 | } |
| 91 | } | 80 | } |
| 92 | 81 | ||
| @@ -1056,9 +1056,9 @@ static int max310x_startup(struct uart_port *port) · 1055 unmodified lines | |||
| 1056 | max310x_port_update(port, MAX310X_MODE1_REG, | 1056 | max310x_port_update(port, MAX310X_MODE1_REG, |
| 1057 | MAX310X_MODE1_TRNSCVCTRL_BIT, 0); | 1057 | MAX310X_MODE1_TRNSCVCTRL_BIT, 0); |
| 1058 | 1058 | ||
| 1059 | /* Reset FIFOs */ | 1059 | /* Configure MODE2 register & Reset FIFOs*/ |
| 1060 | max310x_port_write(port, MAX310X_MODE2_REG, | 1060 | val = MAX310X_MODE2_RXEMPTINV_BIT | MAX310X_MODE2_FIFORST_BIT; |
| 1061 | MAX310X_MODE2_FIFORST_BIT); | 1061 | max310x_port_write(port, MAX310X_MODE2_REG, val); |
| 1062 | max310x_port_update(port, MAX310X_MODE2_REG, | 1062 | max310x_port_update(port, MAX310X_MODE2_REG, |
| 1063 | MAX310X_MODE2_FIFORST_BIT, 0); | 1063 | MAX310X_MODE2_FIFORST_BIT, 0); |
| 1064 | 1064 | ||
| @@ -1086,27 +1086,8 @@ static int max310x_startup(struct uart_port *port) · 21 unmodified lines | |||
| 1086 | /* Clear IRQ status register */ | 1086 | /* Clear IRQ status register */ |
| 1087 | max310x_port_read(port, MAX310X_IRQSTS_REG); | 1087 | max310x_port_read(port, MAX310X_IRQSTS_REG); |
| 1088 | 1088 | ||
| 1089 | /* | 1089 | /* Enable RX, TX, CTS change interrupts */ |
| 1090 | * Let's ask for an interrupt after a timeout equivalent to | 1090 | val = MAX310X_IRQ_RXEMPTY_BIT | MAX310X_IRQ_TXEMPTY_BIT; |
| 1091 | * the receiving time of 4 characters after the last character | ||
| 1092 | * has been received. | ||
| 1093 | */ | ||
| 1094 | max310x_port_write(port, MAX310X_RXTO_REG, 4); | ||
| 1095 | |||
| 1096 | /* | ||
| 1097 | * Make sure we also get RX interrupts when the RX FIFO is | ||
| 1098 | * filling up quickly, so get an interrupt when half of the RX | ||
| 1099 | * FIFO has been filled in. | ||
| 1100 | */ | ||
| 1101 | max310x_port_write(port, MAX310X_FIFOTRIGLVL_REG, | ||
| 1102 | MAX310X_FIFOTRIGLVL_RX(MAX310X_FIFO_SIZE / 2)); | ||
| 1103 | |||
| 1104 | /* Enable RX timeout interrupt in LSR */ | ||
| 1105 | max310x_port_write(port, MAX310X_LSR_IRQEN_REG, | ||
| 1106 | MAX310X_LSR_RXTO_BIT); | ||
| 1107 | |||
| 1108 | /* Enable LSR, RX FIFO trigger, CTS change interrupts */ | ||
| 1109 | val = MAX310X_IRQ_LSR_BIT | MAX310X_IRQ_RXFIFO_BIT | MAX310X_IRQ_TXEMPTY_BIT; | ||
| 1110 | max310x_port_write(port, MAX310X_IRQEN_REG, val | MAX310X_IRQ_CTS_BIT); | 1091 | max310x_port_write(port, MAX310X_IRQEN_REG, val | MAX310X_IRQ_CTS_BIT); |
| 1111 | 1092 | ||
| 1112 | return 0; | 1093 | return 0; |
| @@ -544,9 +544,7 @@ EXPORT_SYMBOL_GPL(tty_wakeup); · 543 unmodified lines | |||
| 544 | * @tty: tty device | 544 | * @tty: tty device |
| 545 | * | 545 | * |
| 546 | * This is available to the pty code so if the master closes, if the | 546 | * This is available to the pty code so if the master closes, if the |
| 547 | * slave is a redirect it can release the redirect. It returns the | 547 | * slave is a redirect it can release the redirect. |
| 548 | * filp for the redirect, which must be fput when the operations on | ||
| 549 | * the tty are completed. | ||
| 550 | */ | 548 | */ |
| 551 | struct file *tty_release_redirect(struct tty_struct *tty) | 549 | struct file *tty_release_redirect(struct tty_struct *tty) |
| 552 | { | 550 | { |
| @@ -561,6 +559,7 @@ struct file *tty_release_redirect(struct tty_struct *tty) · 8 unmodified lines | |||
| 561 | 559 | ||
| 562 | return f; | 560 | return f; |
| 563 | } | 561 | } |
| 562 | EXPORT_SYMBOL_GPL(tty_release_redirect); | ||
| 564 | 563 | ||
| 565 | /** | 564 | /** |
| 566 | * __tty_hangup - actual handler for hangup events | 565 | * __tty_hangup - actual handler for hangup events |
| @@ -1935,6 +1935,11 @@ static const struct usb_device_id acm_ids[] = { · 1934 unmodified lines | |||
| 1935 | .driver_info = SEND_ZERO_PACKET, | 1935 | .driver_info = SEND_ZERO_PACKET, |
| 1936 | }, | 1936 | }, |
| 1937 | 1937 | ||
| 1938 | /* Exclude Goodix Fingerprint Reader */ | ||
| 1939 | { USB_DEVICE(0x27c6, 0x5395), | ||
| 1940 | .driver_info = IGNORE_DEVICE, | ||
| 1941 | }, | ||
| 1942 | |||
| 1938 | /* control interfaces without any protocol set */ | 1943 | /* control interfaces without any protocol set */ |
| 1939 | { USB_INTERFACE_INFO(USB_CLASS_COMM, USB_CDC_SUBCLASS_ACM, | 1944 | { USB_INTERFACE_INFO(USB_CLASS_COMM, USB_CDC_SUBCLASS_ACM, |
| 1940 | USB_CDC_PROTO_NONE) }, | 1945 | USB_CDC_PROTO_NONE) }, |
| @@ -494,16 +494,24 @@ static int usblp_release(struct inode *inode, struct file *file) · 493 unmodified lines | |||
| 494 | /* No kernel lock - fine */ | 494 | /* No kernel lock - fine */ |
| 495 | static __poll_t usblp_poll(struct file *file, struct poll_table_struct *wait) | 495 | static __poll_t usblp_poll(struct file *file, struct poll_table_struct *wait) |
| 496 | { | 496 | { |
| 497 | __poll_t ret; | 497 | struct usblp *usblp = file->private_data; |
| 498 | __poll_t ret = 0; | ||
| 498 | unsigned long flags; | 499 | unsigned long flags; |
| 499 | 500 | ||
| 500 | struct usblp *usblp = file->private_data; | ||
| 501 | /* Should we check file->f_mode & FMODE_WRITE before poll_wait()? */ | 501 | /* Should we check file->f_mode & FMODE_WRITE before poll_wait()? */ |
| 502 | poll_wait(file, &usblp->rwait, wait); | 502 | poll_wait(file, &usblp->rwait, wait); |
| 503 | poll_wait(file, &usblp->wwait, wait); | 503 | poll_wait(file, &usblp->wwait, wait); |
| 504 | |||
| 505 | mutex_lock(&usblp->mut); | ||
| 506 | if (!usblp->present) | ||
| 507 | ret |= EPOLLHUP; | ||
| 508 | mutex_unlock(&usblp->mut); | ||
| 509 | |||
| 504 | spin_lock_irqsave(&usblp->lock, flags); | 510 | spin_lock_irqsave(&usblp->lock, flags); |
| 505 | ret = ((usblp->bidir && usblp->rcomplete) ? EPOLLIN | EPOLLRDNORM : 0) | | 511 | if (usblp->bidir && usblp->rcomplete) |
| 506 | ((usblp->no_paper || usblp->wcomplete) ? EPOLLOUT | EPOLLWRNORM : 0); | 512 | ret |= EPOLLIN | EPOLLRDNORM; |
| 513 | if (usblp->no_paper || usblp->wcomplete) | ||
| 514 | ret |= EPOLLOUT | EPOLLWRNORM; | ||
| 507 | spin_unlock_irqrestore(&usblp->lock, flags); | 515 | spin_unlock_irqrestore(&usblp->lock, flags); |
| 508 | return ret; | 516 | return ret; |
| 509 | } | 517 | } |
| @@ -748,6 +748,38 @@ void usb_put_intf(struct usb_interface *intf) · 747 unmodified lines | |||
| 748 | } | 748 | } |
| 749 | EXPORT_SYMBOL_GPL(usb_put_intf); | 749 | EXPORT_SYMBOL_GPL(usb_put_intf); |
| 750 | 750 | ||
| 751 | /** | ||
| 752 | * usb_intf_get_dma_device - acquire a reference on the usb interface's DMA endpoint | ||
| 753 | * @intf: the usb interface | ||
| 754 | * | ||
| 755 | * While a USB device cannot perform DMA operations by itself, many USB | ||
| 756 | * controllers can. A call to usb_intf_get_dma_device() returns the DMA endpoint | ||
| 757 | * for the given USB interface, if any. The returned device structure must be | ||
| 758 | * released with put_device(). | ||
| 759 | * | ||
| 760 | * See also usb_get_dma_device(). | ||
| 761 | * | ||
| 762 | * Returns: A reference to the usb interface's DMA endpoint; or NULL if none | ||
| 763 | * exists. | ||
| 764 | */ | ||
| 765 | struct device *usb_intf_get_dma_device(struct usb_interface *intf) | ||
| 766 | { | ||
| 767 | struct usb_device *udev = interface_to_usbdev(intf); | ||
| 768 | struct device *dmadev; | ||
| 769 | |||
| 770 | if (!udev->bus) | ||
| 771 | return NULL; | ||
| 772 | |||
| 773 | dmadev = get_device(udev->bus->sysdev); | ||
| 774 | if (!dmadev || !dmadev->dma_mask) { | ||
| 775 | put_device(dmadev); | ||
| 776 | return NULL; | ||
| 777 | } | ||
| 778 | |||
| 779 | return dmadev; | ||
| 780 | } | ||
| 781 | EXPORT_SYMBOL_GPL(usb_intf_get_dma_device); | ||
| 782 | |||
| 751 | /* USB device locking | 783 | /* USB device locking |
| 752 | * | 784 | * |
| 753 | * USB devices and interfaces are locked using the semaphore in their | 785 | * USB devices and interfaces are locked using the semaphore in their |
| @@ -358,8 +358,10 @@ static int dwc3_qcom_suspend(struct dwc3_qcom *qcom) · 357 unmodified lines | |||
| 358 | if (ret) | 358 | if (ret) |
| 359 | dev_warn(qcom->dev, "failed to disable interconnect: %d\n", ret); | 359 | dev_warn(qcom->dev, "failed to disable interconnect: %d\n", ret); |
| 360 | 360 | ||
| 361 | if (device_may_wakeup(qcom->dev)) | ||
| 362 | dwc3_qcom_enable_interrupts(qcom); | ||
| 363 | |||
| 361 | qcom->is_suspended = true; | 364 | qcom->is_suspended = true; |
| 362 | dwc3_qcom_enable_interrupts(qcom); | ||
| 363 | 365 | ||
| 364 | return 0; | 366 | return 0; |
| 365 | } | 367 | } |
| @@ -372,7 +374,8 @@ static int dwc3_qcom_resume(struct dwc3_qcom *qcom) · 6 unmodified lines | |||
| 372 | if (!qcom->is_suspended) | 374 | if (!qcom->is_suspended) |
| 373 | return 0; | 375 | return 0; |
| 374 | 376 | ||
| 375 | dwc3_qcom_disable_interrupts(qcom); | 377 | if (device_may_wakeup(qcom->dev)) |
| 378 | dwc3_qcom_disable_interrupts(qcom); | ||
| 376 | 379 | ||
| 377 | for (i = 0; i < qcom->num_clocks; i++) { | 380 | for (i = 0; i < qcom->num_clocks; i++) { |
| 378 | ret = clk_prepare_enable(qcom->clks[i]); | 381 | ret = clk_prepare_enable(qcom->clks[i]); |
| @@ -650,16 +653,19 @@ static int dwc3_qcom_of_register_core(struct platform_device *pdev) · 271 unmodified lines | |||
| 650 | ret = of_platform_populate(np, NULL, NULL, dev); | 653 | ret = of_platform_populate(np, NULL, NULL, dev); |
| 651 | if (ret) { | 654 | if (ret) { |
| 652 | dev_err(dev, "failed to register dwc3 core - %d\n", ret); | 655 | dev_err(dev, "failed to register dwc3 core - %d\n", ret); |
| 653 | return ret; | 656 | goto node_put; |
| 654 | } | 657 | } |
| 655 | 658 | ||
| 656 | qcom->dwc3 = of_find_device_by_node(dwc3_np); | 659 | qcom->dwc3 = of_find_device_by_node(dwc3_np); |
| 657 | if (!qcom->dwc3) { | 660 | if (!qcom->dwc3) { |
| 661 | ret = -ENODEV; | ||
| 658 | dev_err(dev, "failed to get dwc3 platform device\n"); | 662 | dev_err(dev, "failed to get dwc3 platform device\n"); |
| 659 | return -ENODEV; | ||
| 660 | } | 663 | } |
| 661 | 664 | ||
| 662 | return 0; | 665 | node_put: |
| 666 | of_node_put(dwc3_np); | ||
| 667 | |||
| 668 | return ret; | ||
| 663 | } | 669 | } |
| 664 | 670 | ||
| 665 | static struct platform_device * | 671 | static struct platform_device * |
| @@ -938,6 +944,8 @@ static const struct dwc3_acpi_pdata sdm845_acpi_urs_pdata = { · 272 unmodified lines | |||
| 938 | static const struct acpi_device_id dwc3_qcom_acpi_match[] = { | 944 | static const struct acpi_device_id dwc3_qcom_acpi_match[] = { |
| 939 | { "QCOM2430", (unsigned long)&sdm845_acpi_pdata }, | 945 | { "QCOM2430", (unsigned long)&sdm845_acpi_pdata }, |
| 940 | { "QCOM0304", (unsigned long)&sdm845_acpi_urs_pdata }, | 946 | { "QCOM0304", (unsigned long)&sdm845_acpi_urs_pdata }, |
| 947 | { "QCOM0497", (unsigned long)&sdm845_acpi_urs_pdata }, | ||
| 948 | { "QCOM04A6", (unsigned long)&sdm845_acpi_pdata }, | ||
| 941 | { }, | 949 | { }, |
| 942 | }; | 950 | }; |
| 943 | MODULE_DEVICE_TABLE(acpi, dwc3_qcom_acpi_match); | 951 | MODULE_DEVICE_TABLE(acpi, dwc3_qcom_acpi_match); |
| @@ -499,6 +499,7 @@ static void f_audio_disable(struct usb_function *f) · 498 unmodified lines | |||
| 499 | uac1->as_out_alt = 0; | 499 | uac1->as_out_alt = 0; |
| 500 | uac1->as_in_alt = 0; | 500 | uac1->as_in_alt = 0; |
| 501 | 501 | ||
| 502 | u_audio_stop_playback(&uac1->g_audio); | ||
| 502 | u_audio_stop_capture(&uac1->g_audio); | 503 | u_audio_stop_capture(&uac1->g_audio); |
| 503 | } | 504 | } |
| 504 | 505 | ||
| @@ -478,7 +478,7 @@ static int set_ep_max_packet_size(const struct f_uac2_opts *uac2_opts, · 477 unmodified lines | |||
| 478 | } | 478 | } |
| 479 | 479 | ||
| 480 | max_size_bw = num_channels(chmask) * ssize * | 480 | max_size_bw = num_channels(chmask) * ssize * |
| 481 | DIV_ROUND_UP(srate, factor / (1 << (ep_desc->bInterval - 1))); | 481 | ((srate / (factor / (1 << (ep_desc->bInterval - 1)))) + 1); |
| 482 | ep_desc->wMaxPacketSize = cpu_to_le16(min_t(u16, max_size_bw, | 482 | ep_desc->wMaxPacketSize = cpu_to_le16(min_t(u16, max_size_bw, |
| 483 | max_size_ep)); | 483 | max_size_ep)); |
| 484 | 484 | ||
| @@ -182,12 +182,11 @@ out: \ · 181 unmodified lines | |||
| 182 | size_t len) \ | 182 | size_t len) \ |
| 183 | { \ | 183 | { \ |
| 184 | struct f_##_f_##_opts *opts = to_f_##_f_##_opts(item); \ | 184 | struct f_##_f_##_opts *opts = to_f_##_f_##_opts(item); \ |
| 185 | int ret; \ | 185 | int ret = -EINVAL; \ |
| 186 | u8 val; \ | 186 | u8 val; \ |
| 187 | \ | 187 | \ |
| 188 | mutex_lock(&opts->lock); \ | 188 | mutex_lock(&opts->lock); \ |
| 189 | ret = sscanf(page, "%02hhx", &val); \ | 189 | if (sscanf(page, "%02hhx", &val) > 0) { \ |
| 190 | if (ret > 0) { \ | ||
| 191 | opts->_n_ = val; \ | 190 | opts->_n_ = val; \ |
| 192 | ret = len; \ | 191 | ret = len; \ |
| 193 | } \ | 192 | } \ |
| @@ -1773,8 +1773,8 @@ static int s3c2410_udc_probe(struct platform_device *pdev) · 1772 unmodified lines | |||
| 1773 | udc_info = dev_get_platdata(&pdev->dev); | 1773 | udc_info = dev_get_platdata(&pdev->dev); |
| 1774 | 1774 | ||
| 1775 | base_addr = devm_platform_ioremap_resource(pdev, 0); | 1775 | base_addr = devm_platform_ioremap_resource(pdev, 0); |
| 1776 | if (!base_addr) { | 1776 | if (IS_ERR(base_addr)) { |
| 1777 | retval = -ENOMEM; | 1777 | retval = PTR_ERR(base_addr); |
| 1778 | goto err_mem; | 1778 | goto err_mem; |
| 1779 | } | 1779 | } |
| 1780 | 1780 | ||
| @@ -66,6 +66,7 @@ · 65 unmodified lines | |||
| 66 | #define PCI_DEVICE_ID_ASMEDIA_1042A_XHCI 0x1142 | 66 | #define PCI_DEVICE_ID_ASMEDIA_1042A_XHCI 0x1142 |
| 67 | #define PCI_DEVICE_ID_ASMEDIA_1142_XHCI 0x1242 | 67 | #define PCI_DEVICE_ID_ASMEDIA_1142_XHCI 0x1242 |
| 68 | #define PCI_DEVICE_ID_ASMEDIA_2142_XHCI 0x2142 | 68 | #define PCI_DEVICE_ID_ASMEDIA_2142_XHCI 0x2142 |
| 69 | #define PCI_DEVICE_ID_ASMEDIA_3242_XHCI 0x3242 | ||
| 69 | 70 | ||
| 70 | static const char hcd_name[] = "xhci_hcd"; | 71 | static const char hcd_name[] = "xhci_hcd"; |
| 71 | 72 | ||
| @@ -276,11 +277,14 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci) · 204 unmodified lines | |||
| 276 | pdev->device == PCI_DEVICE_ID_ASMEDIA_1042_XHCI) | 277 | pdev->device == PCI_DEVICE_ID_ASMEDIA_1042_XHCI) |
| 277 | xhci->quirks |= XHCI_BROKEN_STREAMS; | 278 | xhci->quirks |= XHCI_BROKEN_STREAMS; |
| 278 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && | 279 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && |
| 279 | pdev->device == PCI_DEVICE_ID_ASMEDIA_1042A_XHCI) | 280 | pdev->device == PCI_DEVICE_ID_ASMEDIA_1042A_XHCI) { |
| 280 | xhci->quirks |= XHCI_TRUST_TX_LENGTH; | 281 | xhci->quirks |= XHCI_TRUST_TX_LENGTH; |
| 282 | xhci->quirks |= XHCI_NO_64BIT_SUPPORT; | ||
| 283 | } | ||
| 281 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && | 284 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && |
| 282 | (pdev->device == PCI_DEVICE_ID_ASMEDIA_1142_XHCI || | 285 | (pdev->device == PCI_DEVICE_ID_ASMEDIA_1142_XHCI || |
| 283 | pdev->device == PCI_DEVICE_ID_ASMEDIA_2142_XHCI)) | 286 | pdev->device == PCI_DEVICE_ID_ASMEDIA_2142_XHCI || |
| 287 | pdev->device == PCI_DEVICE_ID_ASMEDIA_3242_XHCI)) | ||
| 284 | xhci->quirks |= XHCI_NO_64BIT_SUPPORT; | 288 | xhci->quirks |= XHCI_NO_64BIT_SUPPORT; |
| 285 | 289 | ||
| 286 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && | 290 | if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && |
| @@ -295,6 +299,11 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci) · 8 unmodified lines | |||
| 295 | pdev->device == 0x9026) | 299 | pdev->device == 0x9026) |
| 296 | xhci->quirks |= XHCI_RESET_PLL_ON_DISCONNECT; | 300 | xhci->quirks |= XHCI_RESET_PLL_ON_DISCONNECT; |
| 297 | 301 | ||
| 302 | if (pdev->vendor == PCI_VENDOR_ID_AMD && | ||
| 303 | (pdev->device == PCI_DEVICE_ID_AMD_PROMONTORYA_2 || | ||
| 304 | pdev->device == PCI_DEVICE_ID_AMD_PROMONTORYA_4)) | ||
| 305 | xhci->quirks |= XHCI_NO_SOFT_RETRY; | ||
| 306 | |||
| 298 | if (xhci->quirks & XHCI_RESET_ON_RESUME) | 307 | if (xhci->quirks & XHCI_RESET_ON_RESUME) |
| 299 | xhci_dbg_trace(xhci, trace_xhci_dbg_quirks, | 308 | xhci_dbg_trace(xhci, trace_xhci_dbg_quirks, |
| 300 | "QUIRK: Resetting on resume"); | 309 | "QUIRK: Resetting on resume"); |
| @@ -2484,7 +2484,8 @@ static int process_bulk_intr_td(struct xhci_hcd *xhci, struct xhci_td *td, · 2483 unmodified lines | |||
| 2484 | remaining = 0; | 2484 | remaining = 0; |
| 2485 | break; | 2485 | break; |
| 2486 | case COMP_USB_TRANSACTION_ERROR: | 2486 | case COMP_USB_TRANSACTION_ERROR: |
| 2487 | if ((ep_ring->err_count++ > MAX_SOFT_RETRY) || | 2487 | if (xhci->quirks & XHCI_NO_SOFT_RETRY || |
| 2488 | (ep_ring->err_count++ > MAX_SOFT_RETRY) || | ||
| 2488 | le32_to_cpu(slot_ctx->tt_info) & TT_SLOT) | 2489 | le32_to_cpu(slot_ctx->tt_info) & TT_SLOT) |
| 2489 | break; | 2490 | break; |
| 2490 | 2491 | ||
| @@ -883,44 +883,42 @@ static void xhci_clear_command_ring(struct xhci_hcd *xhci) · 882 unmodified lines | |||
| 883 | xhci_set_cmd_ring_deq(xhci); | 883 | xhci_set_cmd_ring_deq(xhci); |
| 884 | } | 884 | } |
| 885 | 885 | ||
| 886 | static void xhci_disable_port_wake_on_bits(struct xhci_hcd *xhci) | 886 | /* |
| 887 | * Disable port wake bits if do_wakeup is not set. | ||
| 888 | * | ||
| 889 | * Also clear a possible internal port wake state left hanging for ports that | ||
| 890 | * detected termination but never successfully enumerated (trained to 0U). | ||
| 891 | * Internal wake causes immediate xHCI wake after suspend. PORT_CSC write done | ||
| 892 | * at enumeration clears this wake, force one here as well for unconnected ports | ||
| 893 | */ | ||
| 894 | |||
| 895 | static void xhci_disable_hub_port_wake(struct xhci_hcd *xhci, | ||
| 896 | struct xhci_hub *rhub, | ||
| 897 | bool do_wakeup) | ||
| 887 | { | 898 | { |
| 888 | struct xhci_port **ports; | ||
| 889 | int port_index; | ||
| 890 | unsigned long flags; | 899 | unsigned long flags; |
| 891 | u32 t1, t2, portsc; | 900 | u32 t1, t2, portsc; |
| 901 | int i; | ||
| 892 | 902 | ||
| 893 | spin_lock_irqsave(&xhci->lock, flags); | 903 | spin_lock_irqsave(&xhci->lock, flags); |
| 894 | 904 | ||
| 895 | /* disable usb3 ports Wake bits */ | 905 | for (i = 0; i < rhub->num_ports; i++) { |
| 896 | port_index = xhci->usb3_rhub.num_ports; | 906 | portsc = readl(rhub->ports[i]->addr); |
| 897 | ports = xhci->usb3_rhub.ports; | 907 | t1 = xhci_port_state_to_neutral(portsc); |
| 898 | while (port_index--) { | 908 | t2 = t1; |
| 899 | t1 = readl(ports[port_index]->addr); | 909 | |
| 900 | portsc = t1; | 910 | /* clear wake bits if do_wake is not set */ |
| 901 | t1 = xhci_port_state_to_neutral(t1); | 911 | if (!do_wakeup) |
| 902 | t2 = t1 & ~PORT_WAKE_BITS; | 912 | t2 &= ~PORT_WAKE_BITS; |
| 903 | if (t1 != t2) { | 913 | |
| 904 | writel(t2, ports[port_index]->addr); | 914 | /* Don't touch csc bit if connected or connect change is set */ |
| 905 | xhci_dbg(xhci, "disable wake bits port %d-%d, portsc: 0x%x, write: 0x%x\n", | 915 | if (!(portsc & (PORT_CSC | PORT_CONNECT))) |
| 906 | xhci->usb3_rhub.hcd->self.busnum, | 916 | t2 |= PORT_CSC; |
| 907 | port_index + 1, portsc, t2); | ||
| 908 | } | ||
| 909 | } | ||
| 910 | 917 | ||
| 911 | /* disable usb2 ports Wake bits */ | ||
| 912 | port_index = xhci->usb2_rhub.num_ports; | ||
| 913 | ports = xhci->usb2_rhub.ports; | ||
| 914 | while (port_index--) { | ||
| 915 | t1 = readl(ports[port_index]->addr); | ||
| 916 | portsc = t1; | ||
| 917 | t1 = xhci_port_state_to_neutral(t1); | ||
| 918 | t2 = t1 & ~PORT_WAKE_BITS; | ||
| 919 | if (t1 != t2) { | 918 | if (t1 != t2) { |
| 920 | writel(t2, ports[port_index]->addr); | 919 | writel(t2, rhub->ports[i]->addr); |
| 921 | xhci_dbg(xhci, "disable wake bits port %d-%d, portsc: 0x%x, write: 0x%x\n", | 920 | xhci_dbg(xhci, "config port %d-%d wake bits, portsc: 0x%x, write: 0x%x\n", |
| 922 | xhci->usb2_rhub.hcd->self.busnum, | 921 | rhub->hcd->self.busnum, i + 1, portsc, t2); |
| 923 | port_index + 1, portsc, t2); | ||
| 924 | } | 922 | } |
| 925 | } | 923 | } |
| 926 | spin_unlock_irqrestore(&xhci->lock, flags); | 924 | spin_unlock_irqrestore(&xhci->lock, flags); |
| @@ -983,8 +981,8 @@ int xhci_suspend(struct xhci_hcd *xhci, bool do_wakeup) · 56 unmodified lines | |||
| 983 | return -EINVAL; | 981 | return -EINVAL; |
| 984 | 982 | ||
| 985 | /* Clear root port wake on bits if wakeup not allowed. */ | 983 | /* Clear root port wake on bits if wakeup not allowed. */ |
| 986 | if (!do_wakeup) | 984 | xhci_disable_hub_port_wake(xhci, &xhci->usb3_rhub, do_wakeup); |
| 987 | xhci_disable_port_wake_on_bits(xhci); | 985 | xhci_disable_hub_port_wake(xhci, &xhci->usb2_rhub, do_wakeup); |
| 988 | 986 | ||
| 989 | if (!HCD_HW_ACCESSIBLE(hcd)) | 987 | if (!HCD_HW_ACCESSIBLE(hcd)) |
| 990 | return 0; | 988 | return 0; |
| @@ -1088,6 +1086,7 @@ int xhci_resume(struct xhci_hcd *xhci, bool hibernated) · 97 unmodified lines | |||
| 1088 | struct usb_hcd *secondary_hcd; | 1086 | struct usb_hcd *secondary_hcd; |
| 1089 | int retval = 0; | 1087 | int retval = 0; |
| 1090 | bool comp_timer_running = false; | 1088 | bool comp_timer_running = false; |
| 1089 | bool pending_portevent = false; | ||
| 1091 | 1090 | ||
| 1092 | if (!hcd->state) | 1091 | if (!hcd->state) |
| 1093 | return 0; | 1092 | return 0; |
| @@ -1226,13 +1225,22 @@ int xhci_resume(struct xhci_hcd *xhci, bool hibernated) · 132 unmodified lines | |||
| 1226 | 1225 | ||
| 1227 | done: | 1226 | done: |
| 1228 | if (retval == 0) { | 1227 | if (retval == 0) { |
| 1229 | /* Resume root hubs only when have pending events. */ | 1228 | /* |
| 1230 | if (xhci_pending_portevent(xhci)) { | 1229 | * Resume roothubs only if there are pending events. |
| 1230 | * USB 3 devices resend U3 LFPS wake after a 100ms delay if | ||
| 1231 | * the first wake signalling failed, give it that chance. | ||
| 1232 | */ | ||
| 1233 | pending_portevent = xhci_pending_portevent(xhci); | ||
| 1234 | if (!pending_portevent) { | ||
| 1235 | msleep(120); | ||
| 1236 | pending_portevent = xhci_pending_portevent(xhci); | ||
| 1237 | } | ||
| 1238 | |||
| 1239 | if (pending_portevent) { | ||
| 1231 | usb_hcd_resume_root_hub(xhci->shared_hcd); | 1240 | usb_hcd_resume_root_hub(xhci->shared_hcd); |
| 1232 | usb_hcd_resume_root_hub(hcd); | 1241 | usb_hcd_resume_root_hub(hcd); |
| 1233 | } | 1242 | } |
| 1234 | } | 1243 | } |
| 1235 | |||
| 1236 | /* | 1244 | /* |
| 1237 | * If system is subject to the Quirk, Compliance Mode Timer needs to | 1245 | * If system is subject to the Quirk, Compliance Mode Timer needs to |
| 1238 | * be re-initialized Always after a system resume. Ports are subject | 1246 | * be re-initialized Always after a system resume. Ports are subject |
| @@ -1891,6 +1891,7 @@ struct xhci_hcd { · 1890 unmodified lines | |||
| 1891 | #define XHCI_SKIP_PHY_INIT BIT_ULL(37) | 1891 | #define XHCI_SKIP_PHY_INIT BIT_ULL(37) |
| 1892 | #define XHCI_DISABLE_SPARSE BIT_ULL(38) | 1892 | #define XHCI_DISABLE_SPARSE BIT_ULL(38) |
| 1893 | #define XHCI_SG_TRB_CACHE_SIZE_QUIRK BIT_ULL(39) | 1893 | #define XHCI_SG_TRB_CACHE_SIZE_QUIRK BIT_ULL(39) |
| 1894 | #define XHCI_NO_SOFT_RETRY BIT_ULL(40) | ||
| 1894 | 1895 | ||
| 1895 | unsigned int num_active_eps; | 1896 | unsigned int num_active_eps; |
| 1896 | unsigned int limit_active_eps; | 1897 | unsigned int limit_active_eps; |
| @@ -746,6 +746,8 @@ struct usbhs_pipe *usbhs_pipe_malloc(struct usbhs_priv *priv, · 745 unmodified lines | |||
| 746 | 746 | ||
| 747 | void usbhs_pipe_free(struct usbhs_pipe *pipe) | 747 | void usbhs_pipe_free(struct usbhs_pipe *pipe) |
| 748 | { | 748 | { |
| 749 | usbhsp_pipe_select(pipe); | ||
| 750 | usbhsp_pipe_cfg_set(pipe, 0xFFFF, 0); | ||
| 749 | usbhsp_put_pipe(pipe); | 751 | usbhsp_put_pipe(pipe); |
| 750 | } | 752 | } |
| 751 | 753 | ||
| @@ -86,6 +86,7 @@ static const struct usb_device_id id_table[] = { · 85 unmodified lines | |||
| 86 | { USB_DEVICE(0x1a86, 0x7522) }, | 86 | { USB_DEVICE(0x1a86, 0x7522) }, |
| 87 | { USB_DEVICE(0x1a86, 0x7523) }, | 87 | { USB_DEVICE(0x1a86, 0x7523) }, |
| 88 | { USB_DEVICE(0x4348, 0x5523) }, | 88 | { USB_DEVICE(0x4348, 0x5523) }, |
| 89 | { USB_DEVICE(0x9986, 0x7523) }, | ||
| 89 | { }, | 90 | { }, |
| 90 | }; | 91 | }; |
| 91 | MODULE_DEVICE_TABLE(usb, id_table); | 92 | MODULE_DEVICE_TABLE(usb, id_table); |
| @@ -145,6 +145,7 @@ static const struct usb_device_id id_table[] = { · 144 unmodified lines | |||
| 145 | { USB_DEVICE(0x10C4, 0x8857) }, /* CEL EM357 ZigBee USB Stick */ | 145 | { USB_DEVICE(0x10C4, 0x8857) }, /* CEL EM357 ZigBee USB Stick */ |
| 146 | { USB_DEVICE(0x10C4, 0x88A4) }, /* MMB Networks ZigBee USB Device */ | 146 | { USB_DEVICE(0x10C4, 0x88A4) }, /* MMB Networks ZigBee USB Device */ |
| 147 | { USB_DEVICE(0x10C4, 0x88A5) }, /* Planet Innovation Ingeni ZigBee USB Device */ | 147 | { USB_DEVICE(0x10C4, 0x88A5) }, /* Planet Innovation Ingeni ZigBee USB Device */ |
| 148 | { USB_DEVICE(0x10C4, 0x88D8) }, /* Acuity Brands nLight Air Adapter */ | ||
| 148 | { USB_DEVICE(0x10C4, 0x88FB) }, /* CESINEL MEDCAL STII Network Analyzer */ | 149 | { USB_DEVICE(0x10C4, 0x88FB) }, /* CESINEL MEDCAL STII Network Analyzer */ |
| 149 | { USB_DEVICE(0x10C4, 0x8938) }, /* CESINEL MEDCAL S II Network Analyzer */ | 150 | { USB_DEVICE(0x10C4, 0x8938) }, /* CESINEL MEDCAL S II Network Analyzer */ |
| 150 | { USB_DEVICE(0x10C4, 0x8946) }, /* Ketra N1 Wireless Interface */ | 151 | { USB_DEVICE(0x10C4, 0x8946) }, /* Ketra N1 Wireless Interface */ |
| @@ -201,6 +202,8 @@ static const struct usb_device_id id_table[] = { · 50 unmodified lines | |||
| 201 | { USB_DEVICE(0x1901, 0x0194) }, /* GE Healthcare Remote Alarm Box */ | 202 | { USB_DEVICE(0x1901, 0x0194) }, /* GE Healthcare Remote Alarm Box */ |
| 202 | { USB_DEVICE(0x1901, 0x0195) }, /* GE B850/B650/B450 CP2104 DP UART interface */ | 203 | { USB_DEVICE(0x1901, 0x0195) }, /* GE B850/B650/B450 CP2104 DP UART interface */ |
| 203 | { USB_DEVICE(0x1901, 0x0196) }, /* GE B850 CP2105 DP UART interface */ | 204 | { USB_DEVICE(0x1901, 0x0196) }, /* GE B850 CP2105 DP UART interface */ |
| 205 | { USB_DEVICE(0x1901, 0x0197) }, /* GE CS1000 Display serial interface */ | ||
| 206 | { USB_DEVICE(0x1901, 0x0198) }, /* GE CS1000 M.2 Key E serial interface */ | ||
| 204 | { USB_DEVICE(0x199B, 0xBA30) }, /* LORD WSDA-200-USB */ | 207 | { USB_DEVICE(0x199B, 0xBA30) }, /* LORD WSDA-200-USB */ |
| 205 | { USB_DEVICE(0x19CF, 0x3000) }, /* Parrot NMEA GPS Flight Recorder */ | 208 | { USB_DEVICE(0x19CF, 0x3000) }, /* Parrot NMEA GPS Flight Recorder */ |
| 206 | { USB_DEVICE(0x1ADB, 0x0001) }, /* Schweitzer Engineering C662 Cable */ | 209 | { USB_DEVICE(0x1ADB, 0x0001) }, /* Schweitzer Engineering C662 Cable */ |
| @@ -3003,26 +3003,32 @@ static int edge_startup(struct usb_serial *serial) · 3002 unmodified lines | |||
| 3003 | response = -ENODEV; | 3003 | response = -ENODEV; |
| 3004 | } | 3004 | } |
| 3005 | 3005 | ||
| 3006 | usb_free_urb(edge_serial->interrupt_read_urb); | 3006 | goto error; |
| 3007 | kfree(edge_serial->interrupt_in_buffer); | ||
| 3008 | |||
| 3009 | usb_free_urb(edge_serial->read_urb); | ||
| 3010 | kfree(edge_serial->bulk_in_buffer); | ||
| 3011 | |||
| 3012 | kfree(edge_serial); | ||
| 3013 | |||
| 3014 | return response; | ||
| 3015 | } | 3007 | } |
| 3016 | 3008 | ||
| 3017 | /* start interrupt read for this edgeport this interrupt will | 3009 | /* start interrupt read for this edgeport this interrupt will |
| 3018 | * continue as long as the edgeport is connected */ | 3010 | * continue as long as the edgeport is connected */ |
| 3019 | response = usb_submit_urb(edge_serial->interrupt_read_urb, | 3011 | response = usb_submit_urb(edge_serial->interrupt_read_urb, |
| 3020 | GFP_KERNEL); | 3012 | GFP_KERNEL); |
| 3021 | if (response) | 3013 | if (response) { |
| 3022 | dev_err(ddev, "%s - Error %d submitting control urb\n", | 3014 | dev_err(ddev, "%s - Error %d submitting control urb\n", |
| 3023 | __func__, response); | 3015 | __func__, response); |
| 3016 | |||
| 3017 | goto error; | ||
| 3018 | } | ||
| 3024 | } | 3019 | } |
| 3025 | return response; | 3020 | return response; |
| 3021 | |||
| 3022 | error: | ||
| 3023 | usb_free_urb(edge_serial->interrupt_read_urb); | ||
| 3024 | kfree(edge_serial->interrupt_in_buffer); | ||
| 3025 | |||
| 3026 | usb_free_urb(edge_serial->read_urb); | ||
| 3027 | kfree(edge_serial->bulk_in_buffer); | ||
| 3028 | |||
| 3029 | kfree(edge_serial); | ||
| 3030 | |||
| 3031 | return response; | ||
| 3026 | } | 3032 | } |
| 3027 | 3033 | ||
| 3028 | 3034 | ||
| @@ -545,37 +545,13 @@ static void xr_close(struct usb_serial_port *port) · 544 unmodified lines | |||
| 545 | 545 | ||
| 546 | static int xr_probe(struct usb_serial *serial, const struct usb_device_id *id) | 546 | static int xr_probe(struct usb_serial *serial, const struct usb_device_id *id) |
| 547 | { | 547 | { |
| 548 | struct usb_driver *driver = serial->type->usb_driver; | ||
| 549 | struct usb_interface *control_interface; | ||
| 550 | int ret; | ||
| 551 | |||
| 552 | /* Don't bind to control interface */ | 548 | /* Don't bind to control interface */ |
| 553 | if (serial->interface->cur_altsetting->desc.bInterfaceNumber == 0) | 549 | if (serial->interface->cur_altsetting->desc.bInterfaceNumber == 0) |
| 554 | return -ENODEV; | 550 | return -ENODEV; |
| 555 | 551 | ||
| 556 | /* But claim the control interface during data interface probe */ | ||
| 557 | control_interface = usb_ifnum_to_if(serial->dev, 0); | ||
| 558 | if (!control_interface) | ||
| 559 | return -ENODEV; | ||
| 560 | |||
| 561 | ret = usb_driver_claim_interface(driver, control_interface, NULL); | ||
| 562 | if (ret) { | ||
| 563 | dev_err(&serial->interface->dev, "Failed to claim control interface\n"); | ||
| 564 | return ret; | ||
| 565 | } | ||
| 566 | |||
| 567 | return 0; | 552 | return 0; |
| 568 | } | 553 | } |
| 569 | 554 | ||
| 570 | static void xr_disconnect(struct usb_serial *serial) | ||
| 571 | { | ||
| 572 | struct usb_driver *driver = serial->type->usb_driver; | ||
| 573 | struct usb_interface *control_interface; | ||
| 574 | |||
| 575 | control_interface = usb_ifnum_to_if(serial->dev, 0); | ||
| 576 | usb_driver_release_interface(driver, control_interface); | ||
| 577 | } | ||
| 578 | |||
| 579 | static const struct usb_device_id id_table[] = { | 555 | static const struct usb_device_id id_table[] = { |
| 580 | { USB_DEVICE(0x04e2, 0x1410) }, /* XR21V141X */ | 556 | { USB_DEVICE(0x04e2, 0x1410) }, /* XR21V141X */ |
| 581 | { } | 557 | { } |
| @@ -590,7 +566,6 @@ static struct usb_serial_driver xr_device = { · 8 unmodified lines | |||
| 590 | .id_table = id_table, | 566 | .id_table = id_table, |
| 591 | .num_ports = 1, | 567 | .num_ports = 1, |
| 592 | .probe = xr_probe, | 568 | .probe = xr_probe, |
| 593 | .disconnect = xr_disconnect, | ||
| 594 | .open = xr_open, | 569 | .open = xr_open, |
| 595 | .close = xr_close, | 570 | .close = xr_close, |
| 596 | .break_ctl = xr_break_ctl, | 571 | .break_ctl = xr_break_ctl, |
| @@ -46,6 +46,8 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 45 unmodified lines | |||
| 46 | int sockfd = 0; | 46 | int sockfd = 0; |
| 47 | struct socket *socket; | 47 | struct socket *socket; |
| 48 | int rv; | 48 | int rv; |
| 49 | struct task_struct *tcp_rx = NULL; | ||
| 50 | struct task_struct *tcp_tx = NULL; | ||
| 49 | 51 | ||
| 50 | if (!sdev) { | 52 | if (!sdev) { |
| 51 | dev_err(dev, "sdev is null\n"); | 53 | dev_err(dev, "sdev is null\n"); |
| @@ -69,23 +71,47 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 17 unmodified lines | |||
| 69 | } | 71 | } |
| 70 | 72 | ||
| 71 | socket = sockfd_lookup(sockfd, &err); | 73 | socket = sockfd_lookup(sockfd, &err); |
| 72 | if (!socket) | 74 | if (!socket) { |
| 75 | dev_err(dev, "failed to lookup sock"); | ||
| 73 | goto err; | 76 | goto err; |
| 77 | } | ||
| 74 | 78 | ||
| 75 | sdev->ud.tcp_socket = socket; | 79 | if (socket->type != SOCK_STREAM) { |
| 76 | sdev->ud.sockfd = sockfd; | 80 | dev_err(dev, "Expecting SOCK_STREAM - found %d", |
| 81 | socket->type); | ||
| 82 | goto sock_err; | ||
| 83 | } | ||
| 77 | 84 | ||
| 85 | /* unlock and create threads and get tasks */ | ||
| 78 | spin_unlock_irq(&sdev->ud.lock); | 86 | spin_unlock_irq(&sdev->ud.lock); |
| 87 | tcp_rx = kthread_create(stub_rx_loop, &sdev->ud, "stub_rx"); | ||
| 88 | if (IS_ERR(tcp_rx)) { | ||
| 89 | sockfd_put(socket); | ||
| 90 | return -EINVAL; | ||
| 91 | } | ||
| 92 | tcp_tx = kthread_create(stub_tx_loop, &sdev->ud, "stub_tx"); | ||
| 93 | if (IS_ERR(tcp_tx)) { | ||
| 94 | kthread_stop(tcp_rx); | ||
| 95 | sockfd_put(socket); | ||
| 96 | return -EINVAL; | ||
| 97 | } | ||
| 79 | 98 | ||
| 80 | sdev->ud.tcp_rx = kthread_get_run(stub_rx_loop, &sdev->ud, | 99 | /* get task structs now */ |
| 81 | "stub_rx"); | 100 | get_task_struct(tcp_rx); |
| 82 | sdev->ud.tcp_tx = kthread_get_run(stub_tx_loop, &sdev->ud, | 101 | get_task_struct(tcp_tx); |
| 83 | "stub_tx"); | ||
| 84 | 102 | ||
| 103 | /* lock and update sdev->ud state */ | ||
| 85 | spin_lock_irq(&sdev->ud.lock); | 104 | spin_lock_irq(&sdev->ud.lock); |
| 105 | sdev->ud.tcp_socket = socket; | ||
| 106 | sdev->ud.sockfd = sockfd; | ||
| 107 | sdev->ud.tcp_rx = tcp_rx; | ||
| 108 | sdev->ud.tcp_tx = tcp_tx; | ||
| 86 | sdev->ud.status = SDEV_ST_USED; | 109 | sdev->ud.status = SDEV_ST_USED; |
| 87 | spin_unlock_irq(&sdev->ud.lock); | 110 | spin_unlock_irq(&sdev->ud.lock); |
| 88 | 111 | ||
| 112 | wake_up_process(sdev->ud.tcp_rx); | ||
| 113 | wake_up_process(sdev->ud.tcp_tx); | ||
| 114 | |||
| 89 | } else { | 115 | } else { |
| 90 | dev_info(dev, "stub down\n"); | 116 | dev_info(dev, "stub down\n"); |
| 91 | 117 | ||
| @@ -100,6 +126,8 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 8 unmodified lines | |||
| 100 | 126 | ||
| 101 | return count; | 127 | return count; |
| 102 | 128 | ||
| 129 | sock_err: | ||
| 130 | sockfd_put(socket); | ||
| 103 | err: | 131 | err: |
| 104 | spin_unlock_irq(&sdev->ud.lock); | 132 | spin_unlock_irq(&sdev->ud.lock); |
| 105 | return -EINVAL; | 133 | return -EINVAL; |
| @@ -312,6 +312,8 @@ static ssize_t attach_store(struct device *dev, struct device_attribute *attr, · 311 unmodified lines | |||
| 312 | struct vhci *vhci; | 312 | struct vhci *vhci; |
| 313 | int err; | 313 | int err; |
| 314 | unsigned long flags; | 314 | unsigned long flags; |
| 315 | struct task_struct *tcp_rx = NULL; | ||
| 316 | struct task_struct *tcp_tx = NULL; | ||
| 315 | 317 | ||
| 316 | /* | 318 | /* |
| 317 | * @rhport: port number of vhci_hcd | 319 | * @rhport: port number of vhci_hcd |
| @@ -349,12 +351,35 @@ static ssize_t attach_store(struct device *dev, struct device_attribute *attr, · 31 unmodified lines | |||
| 349 | 351 | ||
| 350 | /* Extract socket from fd. */ | 352 | /* Extract socket from fd. */ |
| 351 | socket = sockfd_lookup(sockfd, &err); | 353 | socket = sockfd_lookup(sockfd, &err); |
| 352 | if (!socket) | 354 | if (!socket) { |
| 355 | dev_err(dev, "failed to lookup sock"); | ||
| 353 | return -EINVAL; | 356 | return -EINVAL; |
| 357 | } | ||
| 358 | if (socket->type != SOCK_STREAM) { | ||
| 359 | dev_err(dev, "Expecting SOCK_STREAM - found %d", | ||
| 360 | socket->type); | ||
| 361 | sockfd_put(socket); | ||
| 362 | return -EINVAL; | ||
| 363 | } | ||
| 364 | |||
| 365 | /* create threads before locking */ | ||
| 366 | tcp_rx = kthread_create(vhci_rx_loop, &vdev->ud, "vhci_rx"); | ||
| 367 | if (IS_ERR(tcp_rx)) { | ||
| 368 | sockfd_put(socket); | ||
| 369 | return -EINVAL; | ||
| 370 | } | ||
| 371 | tcp_tx = kthread_create(vhci_tx_loop, &vdev->ud, "vhci_tx"); | ||
| 372 | if (IS_ERR(tcp_tx)) { | ||
| 373 | kthread_stop(tcp_rx); | ||
| 374 | sockfd_put(socket); | ||
| 375 | return -EINVAL; | ||
| 376 | } | ||
| 354 | 377 | ||
| 355 | /* now need lock until setting vdev status as used */ | 378 | /* get task structs now */ |
| 379 | get_task_struct(tcp_rx); | ||
| 380 | get_task_struct(tcp_tx); | ||
| 356 | 381 | ||
| 357 | /* begin a lock */ | 382 | /* now begin lock until setting vdev status set */ |
| 358 | spin_lock_irqsave(&vhci->lock, flags); | 383 | spin_lock_irqsave(&vhci->lock, flags); |
| 359 | spin_lock(&vdev->ud.lock); | 384 | spin_lock(&vdev->ud.lock); |
| 360 | 385 | ||
| @@ -364,6 +389,8 @@ static ssize_t attach_store(struct device *dev, struct device_attribute *attr, · 3 unmodified lines | |||
| 364 | spin_unlock_irqrestore(&vhci->lock, flags); | 389 | spin_unlock_irqrestore(&vhci->lock, flags); |
| 365 | 390 | ||
| 366 | sockfd_put(socket); | 391 | sockfd_put(socket); |
| 392 | kthread_stop_put(tcp_rx); | ||
| 393 | kthread_stop_put(tcp_tx); | ||
| 367 | 394 | ||
| 368 | dev_err(dev, "port %d already used\n", rhport); | 395 | dev_err(dev, "port %d already used\n", rhport); |
| 369 | /* | 396 | /* |
| @@ -382,6 +409,8 @@ static ssize_t attach_store(struct device *dev, struct device_attribute *attr, · 12 unmodified lines | |||
| 382 | vdev->speed = speed; | 409 | vdev->speed = speed; |
| 383 | vdev->ud.sockfd = sockfd; | 410 | vdev->ud.sockfd = sockfd; |
| 384 | vdev->ud.tcp_socket = socket; | 411 | vdev->ud.tcp_socket = socket; |
| 412 | vdev->ud.tcp_rx = tcp_rx; | ||
| 413 | vdev->ud.tcp_tx = tcp_tx; | ||
| 385 | vdev->ud.status = VDEV_ST_NOTASSIGNED; | 414 | vdev->ud.status = VDEV_ST_NOTASSIGNED; |
| 386 | usbip_kcov_handle_init(&vdev->ud); | 415 | usbip_kcov_handle_init(&vdev->ud); |
| 387 | 416 | ||
| @@ -389,8 +418,8 @@ static ssize_t attach_store(struct device *dev, struct device_attribute *attr, · 1 unmodified line | |||
| 389 | spin_unlock_irqrestore(&vhci->lock, flags); | 418 | spin_unlock_irqrestore(&vhci->lock, flags); |
| 390 | /* end the lock */ | 419 | /* end the lock */ |
| 391 | 420 | ||
| 392 | vdev->ud.tcp_rx = kthread_get_run(vhci_rx_loop, &vdev->ud, "vhci_rx"); | 421 | wake_up_process(vdev->ud.tcp_rx); |
| 393 | vdev->ud.tcp_tx = kthread_get_run(vhci_tx_loop, &vdev->ud, "vhci_tx"); | 422 | wake_up_process(vdev->ud.tcp_tx); |
| 394 | 423 | ||
| 395 | rh_port_connect(vdev, speed); | 424 | rh_port_connect(vdev, speed); |
| 396 | 425 | ||
| @@ -90,8 +90,9 @@ unlock: · 89 unmodified lines | |||
| 90 | } | 90 | } |
| 91 | static BIN_ATTR_RO(dev_desc, sizeof(struct usb_device_descriptor)); | 91 | static BIN_ATTR_RO(dev_desc, sizeof(struct usb_device_descriptor)); |
| 92 | 92 | ||
| 93 | static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *attr, | 93 | static ssize_t usbip_sockfd_store(struct device *dev, |
| 94 | const char *in, size_t count) | 94 | struct device_attribute *attr, |
| 95 | const char *in, size_t count) | ||
| 95 | { | 96 | { |
| 96 | struct vudc *udc = (struct vudc *) dev_get_drvdata(dev); | 97 | struct vudc *udc = (struct vudc *) dev_get_drvdata(dev); |
| 97 | int rv; | 98 | int rv; |
| @@ -100,6 +101,8 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 2 unmodified lines | |||
| 100 | struct socket *socket; | 101 | struct socket *socket; |
| 101 | unsigned long flags; | 102 | unsigned long flags; |
| 102 | int ret; | 103 | int ret; |
| 104 | struct task_struct *tcp_rx = NULL; | ||
| 105 | struct task_struct *tcp_tx = NULL; | ||
| 103 | 106 | ||
| 104 | rv = kstrtoint(in, 0, &sockfd); | 107 | rv = kstrtoint(in, 0, &sockfd); |
| 105 | if (rv != 0) | 108 | if (rv != 0) |
| @@ -138,24 +141,54 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 32 unmodified lines | |||
| 138 | goto unlock_ud; | 141 | goto unlock_ud; |
| 139 | } | 142 | } |
| 140 | 143 | ||
| 141 | udc->ud.tcp_socket = socket; | 144 | if (socket->type != SOCK_STREAM) { |
| 145 | dev_err(dev, "Expecting SOCK_STREAM - found %d", | ||
| 146 | socket->type); | ||
| 147 | ret = -EINVAL; | ||
| 148 | goto sock_err; | ||
| 149 | } | ||
| 142 | 150 | ||
| 151 | /* unlock and create threads and get tasks */ | ||
| 143 | spin_unlock_irq(&udc->ud.lock); | 152 | spin_unlock_irq(&udc->ud.lock); |
| 144 | spin_unlock_irqrestore(&udc->lock, flags); | 153 | spin_unlock_irqrestore(&udc->lock, flags); |
| 145 | 154 | ||
| 146 | udc->ud.tcp_rx = kthread_get_run(&v_rx_loop, | 155 | tcp_rx = kthread_create(&v_rx_loop, &udc->ud, "vudc_rx"); |
| 147 | &udc->ud, "vudc_rx"); | 156 | if (IS_ERR(tcp_rx)) { |
| 148 | udc->ud.tcp_tx = kthread_get_run(&v_tx_loop, | 157 | sockfd_put(socket); |
| 149 | &udc->ud, "vudc_tx"); | 158 | return -EINVAL; |
| 159 | } | ||
| 160 | tcp_tx = kthread_create(&v_tx_loop, &udc->ud, "vudc_tx"); | ||
| 161 | if (IS_ERR(tcp_tx)) { | ||
| 162 | kthread_stop(tcp_rx); | ||
| 163 | sockfd_put(socket); | ||
| 164 | return -EINVAL; | ||
| 165 | } | ||
| 166 | |||
| 167 | /* get task structs now */ | ||
| 168 | get_task_struct(tcp_rx); | ||
| 169 | get_task_struct(tcp_tx); | ||
| 150 | 170 | ||
| 171 | /* lock and update udc->ud state */ | ||
| 151 | spin_lock_irqsave(&udc->lock, flags); | 172 | spin_lock_irqsave(&udc->lock, flags); |
| 152 | spin_lock_irq(&udc->ud.lock); | 173 | spin_lock_irq(&udc->ud.lock); |
| 174 | |||
| 175 | udc->ud.tcp_socket = socket; | ||
| 176 | udc->ud.tcp_rx = tcp_rx; | ||
| 177 | udc->ud.tcp_rx = tcp_tx; | ||
| 153 | udc->ud.status = SDEV_ST_USED; | 178 | udc->ud.status = SDEV_ST_USED; |
| 179 | |||
| 154 | spin_unlock_irq(&udc->ud.lock); | 180 | spin_unlock_irq(&udc->ud.lock); |
| 155 | 181 | ||
| 156 | ktime_get_ts64(&udc->start_time); | 182 | ktime_get_ts64(&udc->start_time); |
| 157 | v_start_timer(udc); | 183 | v_start_timer(udc); |
| 158 | udc->connected = 1; | 184 | udc->connected = 1; |
| 185 | |||
| 186 | spin_unlock_irqrestore(&udc->lock, flags); | ||
| 187 | |||
| 188 | wake_up_process(udc->ud.tcp_rx); | ||
| 189 | wake_up_process(udc->ud.tcp_tx); | ||
| 190 | return count; | ||
| 191 | |||
| 159 | } else { | 192 | } else { |
| 160 | if (!udc->connected) { | 193 | if (!udc->connected) { |
| 161 | dev_err(dev, "Device not connected"); | 194 | dev_err(dev, "Device not connected"); |
| @@ -177,6 +210,8 @@ static ssize_t usbip_sockfd_store(struct device *dev, struct device_attribute *a · 15 unmodified lines | |||
| 177 | 210 | ||
| 178 | return count; | 211 | return count; |
| 179 | 212 | ||
| 213 | sock_err: | ||
| 214 | sockfd_put(socket); | ||
| 180 | unlock_ud: | 215 | unlock_ud: |
| 181 | spin_unlock_irq(&udc->ud.lock); | 216 | spin_unlock_irq(&udc->ud.lock); |
| 182 | unlock: | 217 | unlock: |
| @@ -287,11 +287,8 @@ static inline void aty_st_8(int regindex, u8 val, const struct atyfb_par *par) · 286 unmodified lines | |||
| 287 | #endif | 287 | #endif |
| 288 | } | 288 | } |
| 289 | 289 | ||
| 290 | #if defined(CONFIG_PMAC_BACKLIGHT) || defined (CONFIG_FB_ATY_GENERIC_LCD) || \ | ||
| 291 | defined (CONFIG_FB_ATY_BACKLIGHT) | ||
| 292 | extern void aty_st_lcd(int index, u32 val, const struct atyfb_par *par); | 290 | extern void aty_st_lcd(int index, u32 val, const struct atyfb_par *par); |
| 293 | extern u32 aty_ld_lcd(int index, const struct atyfb_par *par); | 291 | extern u32 aty_ld_lcd(int index, const struct atyfb_par *par); |
| 294 | #endif | ||
| 295 | 292 | ||
| 296 | /* | 293 | /* |
| 297 | * DAC operations | 294 | * DAC operations |
| @@ -133,7 +133,7 @@ · 132 unmodified lines | |||
| 133 | #define PRINTKE(fmt, args...) printk(KERN_ERR "atyfb: " fmt, ## args) | 133 | #define PRINTKE(fmt, args...) printk(KERN_ERR "atyfb: " fmt, ## args) |
| 134 | 134 | ||
| 135 | #if defined(CONFIG_PMAC_BACKLIGHT) || defined(CONFIG_FB_ATY_GENERIC_LCD) || \ | 135 | #if defined(CONFIG_PMAC_BACKLIGHT) || defined(CONFIG_FB_ATY_GENERIC_LCD) || \ |
| 136 | defined(CONFIG_FB_ATY_BACKLIGHT) | 136 | defined(CONFIG_FB_ATY_BACKLIGHT) || defined (CONFIG_PPC_PMAC) |
| 137 | static const u32 lt_lcd_regs[] = { | 137 | static const u32 lt_lcd_regs[] = { |
| 138 | CNFG_PANEL_LG, | 138 | CNFG_PANEL_LG, |
| 139 | LCD_GEN_CNTL_LG, | 139 | LCD_GEN_CNTL_LG, |
| @@ -175,8 +175,8 @@ u32 aty_ld_lcd(int index, const struct atyfb_par *par) · 35 unmodified lines | |||
| 175 | return aty_ld_le32(LCD_DATA, par); | 175 | return aty_ld_le32(LCD_DATA, par); |
| 176 | } | 176 | } |
| 177 | } | 177 | } |
| 178 | #else /* defined(CONFIG_PMAC_BACKLIGHT) || defined(CONFIG_FB_ATY_BACKLIGHT) \ | 178 | #else /* defined(CONFIG_PMAC_BACKLIGHT) || defined(CONFIG_FB_ATY_BACKLIGHT) || |
| 179 | defined(CONFIG_FB_ATY_GENERIC_LCD) */ | 179 | defined(CONFIG_FB_ATY_GENERIC_LCD) || defined(CONFIG_PPC_PMAC) */ |
| 180 | void aty_st_lcd(int index, u32 val, const struct atyfb_par *par) | 180 | void aty_st_lcd(int index, u32 val, const struct atyfb_par *par) |
| 181 | { } | 181 | { } |
| 182 | 182 | ||
| @@ -184,7 +184,8 @@ u32 aty_ld_lcd(int index, const struct atyfb_par *par) · 1 unmodified line | |||
| 184 | { | 184 | { |
| 185 | return 0; | 185 | return 0; |
| 186 | } | 186 | } |
| 187 | #endif /* defined(CONFIG_PMAC_BACKLIGHT) || defined (CONFIG_FB_ATY_GENERIC_LCD) */ | 187 | #endif /* defined(CONFIG_PMAC_BACKLIGHT) || defined(CONFIG_FB_ATY_BACKLIGHT) || |
| 188 | defined (CONFIG_FB_ATY_GENERIC_LCD) || defined(CONFIG_PPC_PMAC) */ | ||
| 188 | 189 | ||
| 189 | #ifdef CONFIG_FB_ATY_GENERIC_LCD | 190 | #ifdef CONFIG_FB_ATY_GENERIC_LCD |
| 190 | /* | 191 | /* |
| @@ -333,7 +333,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, · 332 unmodified lines | |||
| 333 | acrn_ioreq_request_clear(vm); | 333 | acrn_ioreq_request_clear(vm); |
| 334 | break; | 334 | break; |
| 335 | case ACRN_IOCTL_PM_GET_CPU_STATE: | 335 | case ACRN_IOCTL_PM_GET_CPU_STATE: |
| 336 | if (copy_from_user(&cstate_cmd, (void *)ioctl_param, | 336 | if (copy_from_user(&cstate_cmd, (void __user *)ioctl_param, |
| 337 | sizeof(cstate_cmd))) | 337 | sizeof(cstate_cmd))) |
| 338 | return -EFAULT; | 338 | return -EFAULT; |
| 339 | 339 | ||
| @@ -404,6 +404,14 @@ fail_remove: · 64 unmodified lines | |||
| 404 | } | 404 | } |
| 405 | static DEVICE_ATTR_WO(remove_cpu); | 405 | static DEVICE_ATTR_WO(remove_cpu); |
| 406 | 406 | ||
| 407 | static umode_t acrn_attr_visible(struct kobject *kobj, struct attribute *a, int n) | ||
| 408 | { | ||
| 409 | if (a == &dev_attr_remove_cpu.attr) | ||
| 410 | return IS_ENABLED(CONFIG_HOTPLUG_CPU) ? a->mode : 0; | ||
| 411 | |||
| 412 | return a->mode; | ||
| 413 | } | ||
| 414 | |||
| 407 | static struct attribute *acrn_attrs[] = { | 415 | static struct attribute *acrn_attrs[] = { |
| 408 | &dev_attr_remove_cpu.attr, | 416 | &dev_attr_remove_cpu.attr, |
| 409 | NULL | 417 | NULL |
| @@ -411,6 +419,7 @@ static struct attribute *acrn_attrs[] = { · 1 unmodified line | |||
| 411 | 419 | ||
| 412 | static struct attribute_group acrn_attr_group = { | 420 | static struct attribute_group acrn_attr_group = { |
| 413 | .attrs = acrn_attrs, | 421 | .attrs = acrn_attrs, |
| 422 | .is_visible = acrn_attr_visible, | ||
| 414 | }; | 423 | }; |
| 415 | 424 | ||
| 416 | static const struct attribute_group *acrn_attr_groups[] = { | 425 | static const struct attribute_group *acrn_attr_groups[] = { |
| @@ -112,7 +112,7 @@ static int acrn_irqfd_assign(struct acrn_vm *vm, struct acrn_irqfd *args) · 111 unmodified lines | |||
| 112 | { | 112 | { |
| 113 | struct eventfd_ctx *eventfd = NULL; | 113 | struct eventfd_ctx *eventfd = NULL; |
| 114 | struct hsm_irqfd *irqfd, *tmp; | 114 | struct hsm_irqfd *irqfd, *tmp; |
| 115 | unsigned int events; | 115 | __poll_t events; |
| 116 | struct fd f; | 116 | struct fd f; |
| 117 | int ret = 0; | 117 | int ret = 0; |
| 118 | 118 | ||
| @@ -158,9 +158,9 @@ static int acrn_irqfd_assign(struct acrn_vm *vm, struct acrn_irqfd *args) · 39 unmodified lines | |||
| 158 | mutex_unlock(&vm->irqfds_lock); | 158 | mutex_unlock(&vm->irqfds_lock); |
| 159 | 159 | ||
| 160 | /* Check the pending event in this stage */ | 160 | /* Check the pending event in this stage */ |
| 161 | events = f.file->f_op->poll(f.file, &irqfd->pt); | 161 | events = vfs_poll(f.file, &irqfd->pt); |
| 162 | 162 | ||
| 163 | if (events & POLLIN) | 163 | if (events & EPOLLIN) |
| 164 | acrn_irqfd_inject(irqfd); | 164 | acrn_irqfd_inject(irqfd); |
| 165 | 165 | ||
| 166 | fdput(f); | 166 | fdput(f); |
| @@ -47,6 +47,11 @@ static unsigned evtchn_2l_max_channels(void) · 46 unmodified lines | |||
| 47 | return EVTCHN_2L_NR_CHANNELS; | 47 | return EVTCHN_2L_NR_CHANNELS; |
| 48 | } | 48 | } |
| 49 | 49 | ||
| 50 | static void evtchn_2l_remove(evtchn_port_t evtchn, unsigned int cpu) | ||
| 51 | { | ||
| 52 | clear_bit(evtchn, BM(per_cpu(cpu_evtchn_mask, cpu))); | ||
| 53 | } | ||
| 54 | |||
| 50 | static void evtchn_2l_bind_to_cpu(evtchn_port_t evtchn, unsigned int cpu, | 55 | static void evtchn_2l_bind_to_cpu(evtchn_port_t evtchn, unsigned int cpu, |
| 51 | unsigned int old_cpu) | 56 | unsigned int old_cpu) |
| 52 | { | 57 | { |
| @@ -72,12 +77,6 @@ static bool evtchn_2l_is_pending(evtchn_port_t port) · 19 unmodified lines | |||
| 72 | return sync_test_bit(port, BM(&s->evtchn_pending[0])); | 77 | return sync_test_bit(port, BM(&s->evtchn_pending[0])); |
| 73 | } | 78 | } |
| 74 | 79 | ||
| 75 | static bool evtchn_2l_test_and_set_mask(evtchn_port_t port) | ||
| 76 | { | ||
| 77 | struct shared_info *s = HYPERVISOR_shared_info; | ||
| 78 | return sync_test_and_set_bit(port, BM(&s->evtchn_mask[0])); | ||
| 79 | } | ||
| 80 | |||
| 81 | static void evtchn_2l_mask(evtchn_port_t port) | 80 | static void evtchn_2l_mask(evtchn_port_t port) |
| 82 | { | 81 | { |
| 83 | struct shared_info *s = HYPERVISOR_shared_info; | 82 | struct shared_info *s = HYPERVISOR_shared_info; |
| @@ -355,18 +354,27 @@ static void evtchn_2l_resume(void) · 271 unmodified lines | |||
| 355 | EVTCHN_2L_NR_CHANNELS/BITS_PER_EVTCHN_WORD); | 354 | EVTCHN_2L_NR_CHANNELS/BITS_PER_EVTCHN_WORD); |
| 356 | } | 355 | } |
| 357 | 356 | ||
| 357 | static int evtchn_2l_percpu_deinit(unsigned int cpu) | ||
| 358 | { | ||
| 359 | memset(per_cpu(cpu_evtchn_mask, cpu), 0, sizeof(xen_ulong_t) * | ||
| 360 | EVTCHN_2L_NR_CHANNELS/BITS_PER_EVTCHN_WORD); | ||
| 361 | |||
| 362 | return 0; | ||
| 363 | } | ||
| 364 | |||
| 358 | static const struct evtchn_ops evtchn_ops_2l = { | 365 | static const struct evtchn_ops evtchn_ops_2l = { |
| 359 | .max_channels = evtchn_2l_max_channels, | 366 | .max_channels = evtchn_2l_max_channels, |
| 360 | .nr_channels = evtchn_2l_max_channels, | 367 | .nr_channels = evtchn_2l_max_channels, |
| 368 | .remove = evtchn_2l_remove, | ||
| 361 | .bind_to_cpu = evtchn_2l_bind_to_cpu, | 369 | .bind_to_cpu = evtchn_2l_bind_to_cpu, |
| 362 | .clear_pending = evtchn_2l_clear_pending, | 370 | .clear_pending = evtchn_2l_clear_pending, |
| 363 | .set_pending = evtchn_2l_set_pending, | 371 | .set_pending = evtchn_2l_set_pending, |
| 364 | .is_pending = evtchn_2l_is_pending, | 372 | .is_pending = evtchn_2l_is_pending, |
| 365 | .test_and_set_mask = evtchn_2l_test_and_set_mask, | ||
| 366 | .mask = evtchn_2l_mask, | 373 | .mask = evtchn_2l_mask, |
| 367 | .unmask = evtchn_2l_unmask, | 374 | .unmask = evtchn_2l_unmask, |
| 368 | .handle_events = evtchn_2l_handle_events, | 375 | .handle_events = evtchn_2l_handle_events, |
| 369 | .resume = evtchn_2l_resume, | 376 | .resume = evtchn_2l_resume, |
| 377 | .percpu_deinit = evtchn_2l_percpu_deinit, | ||
| 370 | }; | 378 | }; |
| 371 | 379 | ||
| 372 | void __init xen_evtchn_2l_init(void) | 380 | void __init xen_evtchn_2l_init(void) |
| @@ -98,13 +98,19 @@ struct irq_info { · 97 unmodified lines | |||
| 98 | short refcnt; | 98 | short refcnt; |
| 99 | u8 spurious_cnt; | 99 | u8 spurious_cnt; |
| 100 | u8 is_accounted; | 100 | u8 is_accounted; |
| 101 | enum xen_irq_type type; /* type */ | 101 | short type; /* type: IRQT_* */ |
| 102 | u8 mask_reason; /* Why is event channel masked */ | ||
| 103 | #define EVT_MASK_REASON_EXPLICIT 0x01 | ||
| 104 | #define EVT_MASK_REASON_TEMPORARY 0x02 | ||
| 105 | #define EVT_MASK_REASON_EOI_PENDING 0x04 | ||
| 106 | u8 is_active; /* Is event just being handled? */ | ||
| 102 | unsigned irq; | 107 | unsigned irq; |
| 103 | evtchn_port_t evtchn; /* event channel */ | 108 | evtchn_port_t evtchn; /* event channel */ |
| 104 | unsigned short cpu; /* cpu bound */ | 109 | unsigned short cpu; /* cpu bound */ |
| 105 | unsigned short eoi_cpu; /* EOI must happen on this cpu-1 */ | 110 | unsigned short eoi_cpu; /* EOI must happen on this cpu-1 */ |
| 106 | unsigned int irq_epoch; /* If eoi_cpu valid: irq_epoch of event */ | 111 | unsigned int irq_epoch; /* If eoi_cpu valid: irq_epoch of event */ |
| 107 | u64 eoi_time; /* Time in jiffies when to EOI. */ | 112 | u64 eoi_time; /* Time in jiffies when to EOI. */ |
| 113 | spinlock_t lock; | ||
| 108 | 114 | ||
| 109 | union { | 115 | union { |
| 110 | unsigned short virq; | 116 | unsigned short virq; |
| @@ -154,6 +160,7 @@ static DEFINE_RWLOCK(evtchn_rwlock); · 43 unmodified lines | |||
| 154 | * evtchn_rwlock | 160 | * evtchn_rwlock |
| 155 | * IRQ-desc lock | 161 | * IRQ-desc lock |
| 156 | * percpu eoi_list_lock | 162 | * percpu eoi_list_lock |
| 163 | * irq_info->lock | ||
| 157 | */ | 164 | */ |
| 158 | 165 | ||
| 159 | static LIST_HEAD(xen_irq_list_head); | 166 | static LIST_HEAD(xen_irq_list_head); |
| @@ -304,6 +311,8 @@ static int xen_irq_info_common_setup(struct irq_info *info, · 144 unmodified lines | |||
| 304 | info->irq = irq; | 311 | info->irq = irq; |
| 305 | info->evtchn = evtchn; | 312 | info->evtchn = evtchn; |
| 306 | info->cpu = cpu; | 313 | info->cpu = cpu; |
| 314 | info->mask_reason = EVT_MASK_REASON_EXPLICIT; | ||
| 315 | spin_lock_init(&info->lock); | ||
| 307 | 316 | ||
| 308 | ret = set_evtchn_to_irq(evtchn, irq); | 317 | ret = set_evtchn_to_irq(evtchn, irq); |
| 309 | if (ret < 0) | 318 | if (ret < 0) |
| @@ -377,6 +386,7 @@ static int xen_irq_info_pirq_setup(unsigned irq, · 67 unmodified lines | |||
| 377 | static void xen_irq_info_cleanup(struct irq_info *info) | 386 | static void xen_irq_info_cleanup(struct irq_info *info) |
| 378 | { | 387 | { |
| 379 | set_evtchn_to_irq(info->evtchn, -1); | 388 | set_evtchn_to_irq(info->evtchn, -1); |
| 389 | xen_evtchn_port_remove(info->evtchn, info->cpu); | ||
| 380 | info->evtchn = 0; | 390 | info->evtchn = 0; |
| 381 | channels_on_cpu_dec(info); | 391 | channels_on_cpu_dec(info); |
| 382 | } | 392 | } |
| @@ -458,6 +468,34 @@ unsigned int cpu_from_evtchn(evtchn_port_t evtchn) · 75 unmodified lines | |||
| 458 | return ret; | 468 | return ret; |
| 459 | } | 469 | } |
| 460 | 470 | ||
| 471 | static void do_mask(struct irq_info *info, u8 reason) | ||
| 472 | { | ||
| 473 | unsigned long flags; | ||
| 474 | |||
| 475 | spin_lock_irqsave(&info->lock, flags); | ||
| 476 | |||
| 477 | if (!info->mask_reason) | ||
| 478 | mask_evtchn(info->evtchn); | ||
| 479 | |||
| 480 | info->mask_reason |= reason; | ||
| 481 | |||
| 482 | spin_unlock_irqrestore(&info->lock, flags); | ||
| 483 | } | ||
| 484 | |||
| 485 | static void do_unmask(struct irq_info *info, u8 reason) | ||
| 486 | { | ||
| 487 | unsigned long flags; | ||
| 488 | |||
| 489 | spin_lock_irqsave(&info->lock, flags); | ||
| 490 | |||
| 491 | info->mask_reason &= ~reason; | ||
| 492 | |||
| 493 | if (!info->mask_reason) | ||
| 494 | unmask_evtchn(info->evtchn); | ||
| 495 | |||
| 496 | spin_unlock_irqrestore(&info->lock, flags); | ||
| 497 | } | ||
| 498 | |||
| 461 | #ifdef CONFIG_X86 | 499 | #ifdef CONFIG_X86 |
| 462 | static bool pirq_check_eoi_map(unsigned irq) | 500 | static bool pirq_check_eoi_map(unsigned irq) |
| 463 | { | 501 | { |
| @@ -604,7 +642,7 @@ static void xen_irq_lateeoi_locked(struct irq_info *info, bool spurious) · 140 unmodified lines | |||
| 604 | } | 642 | } |
| 605 | 643 | ||
| 606 | info->eoi_time = 0; | 644 | info->eoi_time = 0; |
| 607 | unmask_evtchn(evtchn); | 645 | do_unmask(info, EVT_MASK_REASON_EOI_PENDING); |
| 608 | } | 646 | } |
| 609 | 647 | ||
| 610 | static void xen_irq_lateeoi_worker(struct work_struct *work) | 648 | static void xen_irq_lateeoi_worker(struct work_struct *work) |
| @@ -773,6 +811,12 @@ static void xen_evtchn_close(evtchn_port_t port) · 162 unmodified lines | |||
| 773 | BUG(); | 811 | BUG(); |
| 774 | } | 812 | } |
| 775 | 813 | ||
| 814 | static void event_handler_exit(struct irq_info *info) | ||
| 815 | { | ||
| 816 | smp_store_release(&info->is_active, 0); | ||
| 817 | clear_evtchn(info->evtchn); | ||
| 818 | } | ||
| 819 | |||
| 776 | static void pirq_query_unmask(int irq) | 820 | static void pirq_query_unmask(int irq) |
| 777 | { | 821 | { |
| 778 | struct physdev_irq_status_query irq_status; | 822 | struct physdev_irq_status_query irq_status; |
| @@ -791,14 +835,15 @@ static void pirq_query_unmask(int irq) · 12 unmodified lines | |||
| 791 | 835 | ||
| 792 | static void eoi_pirq(struct irq_data *data) | 836 | static void eoi_pirq(struct irq_data *data) |
| 793 | { | 837 | { |
| 794 | evtchn_port_t evtchn = evtchn_from_irq(data->irq); | 838 | struct irq_info *info = info_for_irq(data->irq); |
| 839 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 795 | struct physdev_eoi eoi = { .irq = pirq_from_irq(data->irq) }; | 840 | struct physdev_eoi eoi = { .irq = pirq_from_irq(data->irq) }; |
| 796 | int rc = 0; | 841 | int rc = 0; |
| 797 | 842 | ||
| 798 | if (!VALID_EVTCHN(evtchn)) | 843 | if (!VALID_EVTCHN(evtchn)) |
| 799 | return; | 844 | return; |
| 800 | 845 | ||
| 801 | clear_evtchn(evtchn); | 846 | event_handler_exit(info); |
| 802 | 847 | ||
| 803 | if (pirq_needs_eoi(data->irq)) { | 848 | if (pirq_needs_eoi(data->irq)) { |
| 804 | rc = HYPERVISOR_physdev_op(PHYSDEVOP_eoi, &eoi); | 849 | rc = HYPERVISOR_physdev_op(PHYSDEVOP_eoi, &eoi); |
| @@ -849,7 +894,8 @@ static unsigned int __startup_pirq(unsigned int irq) · 44 unmodified lines | |||
| 849 | goto err; | 894 | goto err; |
| 850 | 895 | ||
| 851 | out: | 896 | out: |
| 852 | unmask_evtchn(evtchn); | 897 | do_unmask(info, EVT_MASK_REASON_EXPLICIT); |
| 898 | |||
| 853 | eoi_pirq(irq_get_irq_data(irq)); | 899 | eoi_pirq(irq_get_irq_data(irq)); |
| 854 | 900 | ||
| 855 | return 0; | 901 | return 0; |
| @@ -876,7 +922,7 @@ static void shutdown_pirq(struct irq_data *data) · 20 unmodified lines | |||
| 876 | if (!VALID_EVTCHN(evtchn)) | 922 | if (!VALID_EVTCHN(evtchn)) |
| 877 | return; | 923 | return; |
| 878 | 924 | ||
| 879 | mask_evtchn(evtchn); | 925 | do_mask(info, EVT_MASK_REASON_EXPLICIT); |
| 880 | xen_evtchn_close(evtchn); | 926 | xen_evtchn_close(evtchn); |
| 881 | xen_irq_info_cleanup(info); | 927 | xen_irq_info_cleanup(info); |
| 882 | } | 928 | } |
| @@ -1628,6 +1674,8 @@ void handle_irq_for_port(evtchn_port_t port, struct evtchn_loop_ctrl *ctrl) · 745 unmodified lines | |||
| 1628 | } | 1674 | } |
| 1629 | 1675 | ||
| 1630 | info = info_for_irq(irq); | 1676 | info = info_for_irq(irq); |
| 1677 | if (xchg_acquire(&info->is_active, 1)) | ||
| 1678 | return; | ||
| 1631 | 1679 | ||
| 1632 | dev = (info->type == IRQT_EVTCHN) ? info->u.interdomain : NULL; | 1680 | dev = (info->type == IRQT_EVTCHN) ? info->u.interdomain : NULL; |
| 1633 | if (dev) | 1681 | if (dev) |
| @@ -1720,10 +1768,10 @@ void rebind_evtchn_irq(evtchn_port_t evtchn, int irq) · 86 unmodified lines | |||
| 1720 | } | 1768 | } |
| 1721 | 1769 | ||
| 1722 | /* Rebind an evtchn so that it gets delivered to a specific cpu */ | 1770 | /* Rebind an evtchn so that it gets delivered to a specific cpu */ |
| 1723 | static int xen_rebind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int tcpu) | 1771 | static int xen_rebind_evtchn_to_cpu(struct irq_info *info, unsigned int tcpu) |
| 1724 | { | 1772 | { |
| 1725 | struct evtchn_bind_vcpu bind_vcpu; | 1773 | struct evtchn_bind_vcpu bind_vcpu; |
| 1726 | int masked; | 1774 | evtchn_port_t evtchn = info ? info->evtchn : 0; |
| 1727 | 1775 | ||
| 1728 | if (!VALID_EVTCHN(evtchn)) | 1776 | if (!VALID_EVTCHN(evtchn)) |
| 1729 | return -1; | 1777 | return -1; |
| @@ -1739,7 +1787,7 @@ static int xen_rebind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int tcpu) · 9 unmodified lines | |||
| 1739 | * Mask the event while changing the VCPU binding to prevent | 1787 | * Mask the event while changing the VCPU binding to prevent |
| 1740 | * it being delivered on an unexpected VCPU. | 1788 | * it being delivered on an unexpected VCPU. |
| 1741 | */ | 1789 | */ |
| 1742 | masked = test_and_set_mask(evtchn); | 1790 | do_mask(info, EVT_MASK_REASON_TEMPORARY); |
| 1743 | 1791 | ||
| 1744 | /* | 1792 | /* |
| 1745 | * If this fails, it usually just indicates that we're dealing with a | 1793 | * If this fails, it usually just indicates that we're dealing with a |
| @@ -1749,8 +1797,7 @@ static int xen_rebind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int tcpu) · 3 unmodified lines | |||
| 1749 | if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_vcpu, &bind_vcpu) >= 0) | 1797 | if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_vcpu, &bind_vcpu) >= 0) |
| 1750 | bind_evtchn_to_cpu(evtchn, tcpu, false); | 1798 | bind_evtchn_to_cpu(evtchn, tcpu, false); |
| 1751 | 1799 | ||
| 1752 | if (!masked) | 1800 | do_unmask(info, EVT_MASK_REASON_TEMPORARY); |
| 1753 | unmask_evtchn(evtchn); | ||
| 1754 | 1801 | ||
| 1755 | return 0; | 1802 | return 0; |
| 1756 | } | 1803 | } |
| @@ -1789,7 +1836,7 @@ static int set_affinity_irq(struct irq_data *data, const struct cpumask *dest, · 32 unmodified lines | |||
| 1789 | unsigned int tcpu = select_target_cpu(dest); | 1836 | unsigned int tcpu = select_target_cpu(dest); |
| 1790 | int ret; | 1837 | int ret; |
| 1791 | 1838 | ||
| 1792 | ret = xen_rebind_evtchn_to_cpu(evtchn_from_irq(data->irq), tcpu); | 1839 | ret = xen_rebind_evtchn_to_cpu(info_for_irq(data->irq), tcpu); |
| 1793 | if (!ret) | 1840 | if (!ret) |
| 1794 | irq_data_update_effective_affinity(data, cpumask_of(tcpu)); | 1841 | irq_data_update_effective_affinity(data, cpumask_of(tcpu)); |
| 1795 | 1842 | ||
| @@ -1798,28 +1845,29 @@ static int set_affinity_irq(struct irq_data *data, const struct cpumask *dest, · 2 unmodified lines | |||
| 1798 | 1845 | ||
| 1799 | static void enable_dynirq(struct irq_data *data) | 1846 | static void enable_dynirq(struct irq_data *data) |
| 1800 | { | 1847 | { |
| 1801 | evtchn_port_t evtchn = evtchn_from_irq(data->irq); | 1848 | struct irq_info *info = info_for_irq(data->irq); |
| 1849 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1802 | 1850 | ||
| 1803 | if (VALID_EVTCHN(evtchn)) | 1851 | if (VALID_EVTCHN(evtchn)) |
| 1804 | unmask_evtchn(evtchn); | 1852 | do_unmask(info, EVT_MASK_REASON_EXPLICIT); |
| 1805 | } | 1853 | } |
| 1806 | 1854 | ||
| 1807 | static void disable_dynirq(struct irq_data *data) | 1855 | static void disable_dynirq(struct irq_data *data) |
| 1808 | { | 1856 | { |
| 1809 | evtchn_port_t evtchn = evtchn_from_irq(data->irq); | 1857 | struct irq_info *info = info_for_irq(data->irq); |
| 1858 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1810 | 1859 | ||
| 1811 | if (VALID_EVTCHN(evtchn)) | 1860 | if (VALID_EVTCHN(evtchn)) |
| 1812 | mask_evtchn(evtchn); | 1861 | do_mask(info, EVT_MASK_REASON_EXPLICIT); |
| 1813 | } | 1862 | } |
| 1814 | 1863 | ||
| 1815 | static void ack_dynirq(struct irq_data *data) | 1864 | static void ack_dynirq(struct irq_data *data) |
| 1816 | { | 1865 | { |
| 1817 | evtchn_port_t evtchn = evtchn_from_irq(data->irq); | 1866 | struct irq_info *info = info_for_irq(data->irq); |
| 1867 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1818 | 1868 | ||
| 1819 | if (!VALID_EVTCHN(evtchn)) | 1869 | if (VALID_EVTCHN(evtchn)) |
| 1820 | return; | 1870 | event_handler_exit(info); |
| 1821 | |||
| 1822 | clear_evtchn(evtchn); | ||
| 1823 | } | 1871 | } |
| 1824 | 1872 | ||
| 1825 | static void mask_ack_dynirq(struct irq_data *data) | 1873 | static void mask_ack_dynirq(struct irq_data *data) |
| @@ -1828,18 +1876,39 @@ static void mask_ack_dynirq(struct irq_data *data) · 2 unmodified lines | |||
| 1828 | ack_dynirq(data); | 1876 | ack_dynirq(data); |
| 1829 | } | 1877 | } |
| 1830 | 1878 | ||
| 1879 | static void lateeoi_ack_dynirq(struct irq_data *data) | ||
| 1880 | { | ||
| 1881 | struct irq_info *info = info_for_irq(data->irq); | ||
| 1882 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1883 | |||
| 1884 | if (VALID_EVTCHN(evtchn)) { | ||
| 1885 | do_mask(info, EVT_MASK_REASON_EOI_PENDING); | ||
| 1886 | event_handler_exit(info); | ||
| 1887 | } | ||
| 1888 | } | ||
| 1889 | |||
| 1890 | static void lateeoi_mask_ack_dynirq(struct irq_data *data) | ||
| 1891 | { | ||
| 1892 | struct irq_info *info = info_for_irq(data->irq); | ||
| 1893 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1894 | |||
| 1895 | if (VALID_EVTCHN(evtchn)) { | ||
| 1896 | do_mask(info, EVT_MASK_REASON_EXPLICIT); | ||
| 1897 | event_handler_exit(info); | ||
| 1898 | } | ||
| 1899 | } | ||
| 1900 | |||
| 1831 | static int retrigger_dynirq(struct irq_data *data) | 1901 | static int retrigger_dynirq(struct irq_data *data) |
| 1832 | { | 1902 | { |
| 1833 | evtchn_port_t evtchn = evtchn_from_irq(data->irq); | 1903 | struct irq_info *info = info_for_irq(data->irq); |
| 1834 | int masked; | 1904 | evtchn_port_t evtchn = info ? info->evtchn : 0; |
| 1835 | 1905 | ||
| 1836 | if (!VALID_EVTCHN(evtchn)) | 1906 | if (!VALID_EVTCHN(evtchn)) |
| 1837 | return 0; | 1907 | return 0; |
| 1838 | 1908 | ||
| 1839 | masked = test_and_set_mask(evtchn); | 1909 | do_mask(info, EVT_MASK_REASON_TEMPORARY); |
| 1840 | set_evtchn(evtchn); | 1910 | set_evtchn(evtchn); |
| 1841 | if (!masked) | 1911 | do_unmask(info, EVT_MASK_REASON_TEMPORARY); |
| 1842 | unmask_evtchn(evtchn); | ||
| 1843 | 1912 | ||
| 1844 | return 1; | 1913 | return 1; |
| 1845 | } | 1914 | } |
| @@ -1938,10 +2007,11 @@ static void restore_cpu_ipis(unsigned int cpu) · 92 unmodified lines | |||
| 1938 | /* Clear an irq's pending state, in preparation for polling on it */ | 2007 | /* Clear an irq's pending state, in preparation for polling on it */ |
| 1939 | void xen_clear_irq_pending(int irq) | 2008 | void xen_clear_irq_pending(int irq) |
| 1940 | { | 2009 | { |
| 1941 | evtchn_port_t evtchn = evtchn_from_irq(irq); | 2010 | struct irq_info *info = info_for_irq(irq); |
| 2011 | evtchn_port_t evtchn = info ? info->evtchn : 0; | ||
| 1942 | 2012 | ||
| 1943 | if (VALID_EVTCHN(evtchn)) | 2013 | if (VALID_EVTCHN(evtchn)) |
| 1944 | clear_evtchn(evtchn); | 2014 | event_handler_exit(info); |
| 1945 | } | 2015 | } |
| 1946 | EXPORT_SYMBOL(xen_clear_irq_pending); | 2016 | EXPORT_SYMBOL(xen_clear_irq_pending); |
| 1947 | void xen_set_irq_pending(int irq) | 2017 | void xen_set_irq_pending(int irq) |
| @@ -2053,8 +2123,8 @@ static struct irq_chip xen_lateeoi_chip __read_mostly = { · 105 unmodified lines | |||
| 2053 | .irq_mask = disable_dynirq, | 2123 | .irq_mask = disable_dynirq, |
| 2054 | .irq_unmask = enable_dynirq, | 2124 | .irq_unmask = enable_dynirq, |
| 2055 | 2125 | ||
| 2056 | .irq_ack = mask_ack_dynirq, | 2126 | .irq_ack = lateeoi_ack_dynirq, |
| 2057 | .irq_mask_ack = mask_ack_dynirq, | 2127 | .irq_mask_ack = lateeoi_mask_ack_dynirq, |
| 2058 | 2128 | ||
| 2059 | .irq_set_affinity = set_affinity_irq, | 2129 | .irq_set_affinity = set_affinity_irq, |
| 2060 | .irq_retrigger = retrigger_dynirq, | 2130 | .irq_retrigger = retrigger_dynirq, |
| @@ -209,12 +209,6 @@ static bool evtchn_fifo_is_pending(evtchn_port_t port) · 208 unmodified lines | |||
| 209 | return sync_test_bit(EVTCHN_FIFO_BIT(PENDING, word), BM(word)); | 209 | return sync_test_bit(EVTCHN_FIFO_BIT(PENDING, word), BM(word)); |
| 210 | } | 210 | } |
| 211 | 211 | ||
| 212 | static bool evtchn_fifo_test_and_set_mask(evtchn_port_t port) | ||
| 213 | { | ||
| 214 | event_word_t *word = event_word_from_port(port); | ||
| 215 | return sync_test_and_set_bit(EVTCHN_FIFO_BIT(MASKED, word), BM(word)); | ||
| 216 | } | ||
| 217 | |||
| 218 | static void evtchn_fifo_mask(evtchn_port_t port) | 212 | static void evtchn_fifo_mask(evtchn_port_t port) |
| 219 | { | 213 | { |
| 220 | event_word_t *word = event_word_from_port(port); | 214 | event_word_t *word = event_word_from_port(port); |
| @@ -423,7 +417,6 @@ static const struct evtchn_ops evtchn_ops_fifo = { · 202 unmodified lines | |||
| 423 | .clear_pending = evtchn_fifo_clear_pending, | 417 | .clear_pending = evtchn_fifo_clear_pending, |
| 424 | .set_pending = evtchn_fifo_set_pending, | 418 | .set_pending = evtchn_fifo_set_pending, |
| 425 | .is_pending = evtchn_fifo_is_pending, | 419 | .is_pending = evtchn_fifo_is_pending, |
| 426 | .test_and_set_mask = evtchn_fifo_test_and_set_mask, | ||
| 427 | .mask = evtchn_fifo_mask, | 420 | .mask = evtchn_fifo_mask, |
| 428 | .unmask = evtchn_fifo_unmask, | 421 | .unmask = evtchn_fifo_unmask, |
| 429 | .handle_events = evtchn_fifo_handle_events, | 422 | .handle_events = evtchn_fifo_handle_events, |
| @@ -14,13 +14,13 @@ struct evtchn_ops { · 13 unmodified lines | |||
| 14 | unsigned (*nr_channels)(void); | 14 | unsigned (*nr_channels)(void); |
| 15 | 15 | ||
| 16 | int (*setup)(evtchn_port_t port); | 16 | int (*setup)(evtchn_port_t port); |
| 17 | void (*remove)(evtchn_port_t port, unsigned int cpu); | ||
| 17 | void (*bind_to_cpu)(evtchn_port_t evtchn, unsigned int cpu, | 18 | void (*bind_to_cpu)(evtchn_port_t evtchn, unsigned int cpu, |
| 18 | unsigned int old_cpu); | 19 | unsigned int old_cpu); |
| 19 | 20 | ||
| 20 | void (*clear_pending)(evtchn_port_t port); | 21 | void (*clear_pending)(evtchn_port_t port); |
| 21 | void (*set_pending)(evtchn_port_t port); | 22 | void (*set_pending)(evtchn_port_t port); |
| 22 | bool (*is_pending)(evtchn_port_t port); | 23 | bool (*is_pending)(evtchn_port_t port); |
| 23 | bool (*test_and_set_mask)(evtchn_port_t port); | ||
| 24 | void (*mask)(evtchn_port_t port); | 24 | void (*mask)(evtchn_port_t port); |
| 25 | void (*unmask)(evtchn_port_t port); | 25 | void (*unmask)(evtchn_port_t port); |
| 26 | 26 | ||
| @@ -54,6 +54,13 @@ static inline int xen_evtchn_port_setup(evtchn_port_t evtchn) · 27 unmodified lines | |||
| 54 | return 0; | 54 | return 0; |
| 55 | } | 55 | } |
| 56 | 56 | ||
| 57 | static inline void xen_evtchn_port_remove(evtchn_port_t evtchn, | ||
| 58 | unsigned int cpu) | ||
| 59 | { | ||
| 60 | if (evtchn_ops->remove) | ||
| 61 | evtchn_ops->remove(evtchn, cpu); | ||
| 62 | } | ||
| 63 | |||
| 57 | static inline void xen_evtchn_port_bind_to_cpu(evtchn_port_t evtchn, | 64 | static inline void xen_evtchn_port_bind_to_cpu(evtchn_port_t evtchn, |
| 58 | unsigned int cpu, | 65 | unsigned int cpu, |
| 59 | unsigned int old_cpu) | 66 | unsigned int old_cpu) |
| @@ -76,11 +83,6 @@ static inline bool test_evtchn(evtchn_port_t port) · 16 unmodified lines | |||
| 76 | return evtchn_ops->is_pending(port); | 83 | return evtchn_ops->is_pending(port); |
| 77 | } | 84 | } |
| 78 | 85 | ||
| 79 | static inline bool test_and_set_mask(evtchn_port_t port) | ||
| 80 | { | ||
| 81 | return evtchn_ops->test_and_set_mask(port); | ||
| 82 | } | ||
| 83 | |||
| 84 | static inline void mask_evtchn(evtchn_port_t port) | 86 | static inline void mask_evtchn(evtchn_port_t port) |
| 85 | { | 87 | { |
| 86 | return evtchn_ops->mask(port); | 88 | return evtchn_ops->mask(port); |
| @@ -133,20 +133,26 @@ struct gntdev_grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count, · 132 unmodified lines | |||
| 133 | if (NULL == add) | 133 | if (NULL == add) |
| 134 | return NULL; | 134 | return NULL; |
| 135 | 135 | ||
| 136 | add->grants = kvcalloc(count, sizeof(add->grants[0]), GFP_KERNEL); | 136 | add->grants = kvmalloc_array(count, sizeof(add->grants[0]), |
| 137 | add->map_ops = kvcalloc(count, sizeof(add->map_ops[0]), GFP_KERNEL); | 137 | GFP_KERNEL); |
| 138 | add->unmap_ops = kvcalloc(count, sizeof(add->unmap_ops[0]), GFP_KERNEL); | 138 | add->map_ops = kvmalloc_array(count, sizeof(add->map_ops[0]), |
| 139 | add->kmap_ops = kvcalloc(count, sizeof(add->kmap_ops[0]), GFP_KERNEL); | 139 | GFP_KERNEL); |
| 140 | add->kunmap_ops = kvcalloc(count, | 140 | add->unmap_ops = kvmalloc_array(count, sizeof(add->unmap_ops[0]), |
| 141 | sizeof(add->kunmap_ops[0]), GFP_KERNEL); | 141 | GFP_KERNEL); |
| 142 | add->pages = kvcalloc(count, sizeof(add->pages[0]), GFP_KERNEL); | 142 | add->pages = kvcalloc(count, sizeof(add->pages[0]), GFP_KERNEL); |
| 143 | if (NULL == add->grants || | 143 | if (NULL == add->grants || |
| 144 | NULL == add->map_ops || | 144 | NULL == add->map_ops || |
| 145 | NULL == add->unmap_ops || | 145 | NULL == add->unmap_ops || |
| 146 | NULL == add->kmap_ops || | ||
| 147 | NULL == add->kunmap_ops || | ||
| 148 | NULL == add->pages) | 146 | NULL == add->pages) |
| 149 | goto err; | 147 | goto err; |
| 148 | if (use_ptemod) { | ||
| 149 | add->kmap_ops = kvmalloc_array(count, sizeof(add->kmap_ops[0]), | ||
| 150 | GFP_KERNEL); | ||
| 151 | add->kunmap_ops = kvmalloc_array(count, sizeof(add->kunmap_ops[0]), | ||
| 152 | GFP_KERNEL); | ||
| 153 | if (NULL == add->kmap_ops || NULL == add->kunmap_ops) | ||
| 154 | goto err; | ||
| 155 | } | ||
| 150 | 156 | ||
| 151 | #ifdef CONFIG_XEN_GRANT_DMA_ALLOC | 157 | #ifdef CONFIG_XEN_GRANT_DMA_ALLOC |
| 152 | add->dma_flags = dma_flags; | 158 | add->dma_flags = dma_flags; |
| @@ -183,10 +189,14 @@ struct gntdev_grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count, · 30 unmodified lines | |||
| 183 | goto err; | 189 | goto err; |
| 184 | 190 | ||
| 185 | for (i = 0; i < count; i++) { | 191 | for (i = 0; i < count; i++) { |
| 186 | add->map_ops[i].handle = -1; | 192 | add->grants[i].domid = DOMID_INVALID; |
| 187 | add->unmap_ops[i].handle = -1; | 193 | add->grants[i].ref = INVALID_GRANT_REF; |
| 188 | add->kmap_ops[i].handle = -1; | 194 | add->map_ops[i].handle = INVALID_GRANT_HANDLE; |
| 189 | add->kunmap_ops[i].handle = -1; | 195 | add->unmap_ops[i].handle = INVALID_GRANT_HANDLE; |
| 196 | if (use_ptemod) { | ||
| 197 | add->kmap_ops[i].handle = INVALID_GRANT_HANDLE; | ||
| 198 | add->kunmap_ops[i].handle = INVALID_GRANT_HANDLE; | ||
| 199 | } | ||
| 190 | } | 200 | } |
| 191 | 201 | ||
| 192 | add->index = 0; | 202 | add->index = 0; |
| @@ -274,7 +284,7 @@ static int find_grant_ptes(pte_t *pte, unsigned long addr, void *data) · 81 unmodified lines | |||
| 274 | map->grants[pgnr].ref, | 284 | map->grants[pgnr].ref, |
| 275 | map->grants[pgnr].domid); | 285 | map->grants[pgnr].domid); |
| 276 | gnttab_set_unmap_op(&map->unmap_ops[pgnr], pte_maddr, flags, | 286 | gnttab_set_unmap_op(&map->unmap_ops[pgnr], pte_maddr, flags, |
| 277 | -1 /* handle */); | 287 | INVALID_GRANT_HANDLE); |
| 278 | return 0; | 288 | return 0; |
| 279 | } | 289 | } |
| 280 | 290 | ||
| @@ -292,7 +302,7 @@ int gntdev_map_grant_pages(struct gntdev_grant_map *map) · 11 unmodified lines | |||
| 292 | 302 | ||
| 293 | if (!use_ptemod) { | 303 | if (!use_ptemod) { |
| 294 | /* Note: it could already be mapped */ | 304 | /* Note: it could already be mapped */ |
| 295 | if (map->map_ops[0].handle != -1) | 305 | if (map->map_ops[0].handle != INVALID_GRANT_HANDLE) |
| 296 | return 0; | 306 | return 0; |
| 297 | for (i = 0; i < map->count; i++) { | 307 | for (i = 0; i < map->count; i++) { |
| 298 | unsigned long addr = (unsigned long) | 308 | unsigned long addr = (unsigned long) |
| @@ -301,7 +311,7 @@ int gntdev_map_grant_pages(struct gntdev_grant_map *map) · 2 unmodified lines | |||
| 301 | map->grants[i].ref, | 311 | map->grants[i].ref, |
| 302 | map->grants[i].domid); | 312 | map->grants[i].domid); |
| 303 | gnttab_set_unmap_op(&map->unmap_ops[i], addr, | 313 | gnttab_set_unmap_op(&map->unmap_ops[i], addr, |
| 304 | map->flags, -1 /* handle */); | 314 | map->flags, INVALID_GRANT_HANDLE); |
| 305 | } | 315 | } |
| 306 | } else { | 316 | } else { |
| 307 | /* | 317 | /* |
| @@ -327,13 +337,13 @@ int gntdev_map_grant_pages(struct gntdev_grant_map *map) · 19 unmodified lines | |||
| 327 | map->grants[i].ref, | 337 | map->grants[i].ref, |
| 328 | map->grants[i].domid); | 338 | map->grants[i].domid); |
| 329 | gnttab_set_unmap_op(&map->kunmap_ops[i], address, | 339 | gnttab_set_unmap_op(&map->kunmap_ops[i], address, |
| 330 | flags, -1); | 340 | flags, INVALID_GRANT_HANDLE); |
| 331 | } | 341 | } |
| 332 | } | 342 | } |
| 333 | 343 | ||
| 334 | pr_debug("map %d+%d\n", map->index, map->count); | 344 | pr_debug("map %d+%d\n", map->index, map->count); |
| 335 | err = gnttab_map_refs(map->map_ops, use_ptemod ? map->kmap_ops : NULL, | 345 | err = gnttab_map_refs(map->map_ops, map->kmap_ops, map->pages, |
| 336 | map->pages, map->count); | 346 | map->count); |
| 337 | 347 | ||
| 338 | for (i = 0; i < map->count; i++) { | 348 | for (i = 0; i < map->count; i++) { |
| 339 | if (map->map_ops[i].status == GNTST_okay) | 349 | if (map->map_ops[i].status == GNTST_okay) |
| @@ -385,7 +395,7 @@ static int __unmap_grant_pages(struct gntdev_grant_map *map, int offset, · 45 unmodified lines | |||
| 385 | pr_debug("unmap handle=%d st=%d\n", | 395 | pr_debug("unmap handle=%d st=%d\n", |
| 386 | map->unmap_ops[offset+i].handle, | 396 | map->unmap_ops[offset+i].handle, |
| 387 | map->unmap_ops[offset+i].status); | 397 | map->unmap_ops[offset+i].status); |
| 388 | map->unmap_ops[offset+i].handle = -1; | 398 | map->unmap_ops[offset+i].handle = INVALID_GRANT_HANDLE; |
| 389 | } | 399 | } |
| 390 | return err; | 400 | return err; |
| 391 | } | 401 | } |
| @@ -401,13 +411,15 @@ static int unmap_grant_pages(struct gntdev_grant_map *map, int offset, · 9 unmodified lines | |||
| 401 | * already unmapped some of the grants. Only unmap valid ranges. | 411 | * already unmapped some of the grants. Only unmap valid ranges. |
| 402 | */ | 412 | */ |
| 403 | while (pages && !err) { | 413 | while (pages && !err) { |
| 404 | while (pages && map->unmap_ops[offset].handle == -1) { | 414 | while (pages && |
| 415 | map->unmap_ops[offset].handle == INVALID_GRANT_HANDLE) { | ||
| 405 | offset++; | 416 | offset++; |
| 406 | pages--; | 417 | pages--; |
| 407 | } | 418 | } |
| 408 | range = 0; | 419 | range = 0; |
| 409 | while (range < pages) { | 420 | while (range < pages) { |
| 410 | if (map->unmap_ops[offset+range].handle == -1) | 421 | if (map->unmap_ops[offset + range].handle == |
| 422 | INVALID_GRANT_HANDLE) | ||
| 411 | break; | 423 | break; |
| 412 | range++; | 424 | range++; |
| 413 | } | 425 | } |
| @@ -203,7 +203,7 @@ config TMPFS_XATTR · 202 unmodified lines | |||
| 203 | 203 | ||
| 204 | config TMPFS_INODE64 | 204 | config TMPFS_INODE64 |
| 205 | bool "Use 64-bit ino_t by default in tmpfs" | 205 | bool "Use 64-bit ino_t by default in tmpfs" |
| 206 | depends on TMPFS && 64BIT && !(S390 || ALPHA) | 206 | depends on TMPFS && 64BIT |
| 207 | default n | 207 | default n |
| 208 | help | 208 | help |
| 209 | tmpfs has historically used only inode numbers as wide as an unsigned | 209 | tmpfs has historically used only inode numbers as wide as an unsigned |
| @@ -70,7 +70,6 @@ const struct inode_operations afs_dir_inode_operations = { · 69 unmodified lines | |||
| 70 | .permission = afs_permission, | 70 | .permission = afs_permission, |
| 71 | .getattr = afs_getattr, | 71 | .getattr = afs_getattr, |
| 72 | .setattr = afs_setattr, | 72 | .setattr = afs_setattr, |
| 73 | .listxattr = afs_listxattr, | ||
| 74 | }; | 73 | }; |
| 75 | 74 | ||
| 76 | const struct address_space_operations afs_dir_aops = { | 75 | const struct address_space_operations afs_dir_aops = { |
| @@ -43,7 +43,6 @@ const struct inode_operations afs_file_inode_operations = { · 42 unmodified lines | |||
| 43 | .getattr = afs_getattr, | 43 | .getattr = afs_getattr, |
| 44 | .setattr = afs_setattr, | 44 | .setattr = afs_setattr, |
| 45 | .permission = afs_permission, | 45 | .permission = afs_permission, |
| 46 | .listxattr = afs_listxattr, | ||
| 47 | }; | 46 | }; |
| 48 | 47 | ||
| 49 | const struct address_space_operations afs_fs_aops = { | 48 | const struct address_space_operations afs_fs_aops = { |
| @@ -181,10 +181,13 @@ void afs_wait_for_operation(struct afs_operation *op) · 180 unmodified lines | |||
| 181 | if (test_bit(AFS_SERVER_FL_IS_YFS, &op->server->flags) && | 181 | if (test_bit(AFS_SERVER_FL_IS_YFS, &op->server->flags) && |
| 182 | op->ops->issue_yfs_rpc) | 182 | op->ops->issue_yfs_rpc) |
| 183 | op->ops->issue_yfs_rpc(op); | 183 | op->ops->issue_yfs_rpc(op); |
| 184 | else | 184 | else if (op->ops->issue_afs_rpc) |
| 185 | op->ops->issue_afs_rpc(op); | 185 | op->ops->issue_afs_rpc(op); |
| 186 | else | ||
| 187 | op->ac.error = -ENOTSUPP; | ||
| 186 | 188 | ||
| 187 | op->error = afs_wait_for_call_to_complete(op->call, &op->ac); | 189 | if (op->call) |
| 190 | op->error = afs_wait_for_call_to_complete(op->call, &op->ac); | ||
| 188 | } | 191 | } |
| 189 | 192 | ||
| 190 | switch (op->error) { | 193 | switch (op->error) { |
| @@ -27,7 +27,6 @@ · 26 unmodified lines | |||
| 27 | 27 | ||
| 28 | static const struct inode_operations afs_symlink_inode_operations = { | 28 | static const struct inode_operations afs_symlink_inode_operations = { |
| 29 | .get_link = page_get_link, | 29 | .get_link = page_get_link, |
| 30 | .listxattr = afs_listxattr, | ||
| 31 | }; | 30 | }; |
| 32 | 31 | ||
| 33 | static noinline void dump_vnode(struct afs_vnode *vnode, struct afs_vnode *parent_vnode) | 32 | static noinline void dump_vnode(struct afs_vnode *vnode, struct afs_vnode *parent_vnode) |
| @@ -1509,7 +1509,6 @@ extern int afs_launder_page(struct page *); · 1508 unmodified lines | |||
| 1509 | * xattr.c | 1509 | * xattr.c |
| 1510 | */ | 1510 | */ |
| 1511 | extern const struct xattr_handler *afs_xattr_handlers[]; | 1511 | extern const struct xattr_handler *afs_xattr_handlers[]; |
| 1512 | extern ssize_t afs_listxattr(struct dentry *, char *, size_t); | ||
| 1513 | 1512 | ||
| 1514 | /* | 1513 | /* |
| 1515 | * yfsclient.c | 1514 | * yfsclient.c |
| @@ -32,7 +32,6 @@ const struct inode_operations afs_mntpt_inode_operations = { · 31 unmodified lines | |||
| 32 | .lookup = afs_mntpt_lookup, | 32 | .lookup = afs_mntpt_lookup, |
| 33 | .readlink = page_readlink, | 33 | .readlink = page_readlink, |
| 34 | .getattr = afs_getattr, | 34 | .getattr = afs_getattr, |
| 35 | .listxattr = afs_listxattr, | ||
| 36 | }; | 35 | }; |
| 37 | 36 | ||
| 38 | const struct inode_operations afs_autocell_inode_operations = { | 37 | const struct inode_operations afs_autocell_inode_operations = { |
| @@ -11,29 +11,6 @@ · 10 unmodified lines | |||
| 11 | #include <linux/xattr.h> | 11 | #include <linux/xattr.h> |
| 12 | #include "internal.h" | 12 | #include "internal.h" |
| 13 | 13 | ||
| 14 | static const char afs_xattr_list[] = | ||
| 15 | "afs.acl\0" | ||
| 16 | "afs.cell\0" | ||
| 17 | "afs.fid\0" | ||
| 18 | "afs.volume\0" | ||
| 19 | "afs.yfs.acl\0" | ||
| 20 | "afs.yfs.acl_inherited\0" | ||
| 21 | "afs.yfs.acl_num_cleaned\0" | ||
| 22 | "afs.yfs.vol_acl"; | ||
| 23 | |||
| 24 | /* | ||
| 25 | * Retrieve a list of the supported xattrs. | ||
| 26 | */ | ||
| 27 | ssize_t afs_listxattr(struct dentry *dentry, char *buffer, size_t size) | ||
| 28 | { | ||
| 29 | if (size == 0) | ||
| 30 | return sizeof(afs_xattr_list); | ||
| 31 | if (size < sizeof(afs_xattr_list)) | ||
| 32 | return -ERANGE; | ||
| 33 | memcpy(buffer, afs_xattr_list, sizeof(afs_xattr_list)); | ||
| 34 | return sizeof(afs_xattr_list); | ||
| 35 | } | ||
| 36 | |||
| 37 | /* | 14 | /* |
| 38 | * Deal with the result of a successful fetch ACL operation. | 15 | * Deal with the result of a successful fetch ACL operation. |
| 39 | */ | 16 | */ |
| @@ -231,6 +208,8 @@ static int afs_xattr_get_yfs(const struct xattr_handler *handler, · 191 unmodified lines | |||
| 231 | else | 208 | else |
| 232 | ret = -ERANGE; | 209 | ret = -ERANGE; |
| 233 | } | 210 | } |
| 211 | } else if (ret == -ENOTSUPP) { | ||
| 212 | ret = -ENODATA; | ||
| 234 | } | 213 | } |
| 235 | 214 | ||
| 236 | error_yacl: | 215 | error_yacl: |
| @@ -256,6 +235,7 @@ static int afs_xattr_set_yfs(const struct xattr_handler *handler, · 19 unmodified lines | |||
| 256 | { | 235 | { |
| 257 | struct afs_operation *op; | 236 | struct afs_operation *op; |
| 258 | struct afs_vnode *vnode = AFS_FS_I(inode); | 237 | struct afs_vnode *vnode = AFS_FS_I(inode); |
| 238 | int ret; | ||
| 259 | 239 | ||
| 260 | if (flags == XATTR_CREATE || | 240 | if (flags == XATTR_CREATE || |
| 261 | strcmp(name, "acl") != 0) | 241 | strcmp(name, "acl") != 0) |
| @@ -270,7 +250,10 @@ static int afs_xattr_set_yfs(const struct xattr_handler *handler, · 8 unmodified lines | |||
| 270 | return afs_put_operation(op); | 250 | return afs_put_operation(op); |
| 271 | 251 | ||
| 272 | op->ops = &yfs_store_opaque_acl2_operation; | 252 | op->ops = &yfs_store_opaque_acl2_operation; |
| 273 | return afs_do_sync_operation(op); | 253 | ret = afs_do_sync_operation(op); |
| 254 | if (ret == -ENOTSUPP) | ||
| 255 | ret = -ENODATA; | ||
| 256 | return ret; | ||
| 274 | } | 257 | } |
| 275 | 258 | ||
| 276 | static const struct xattr_handler afs_xattr_yfs_handler = { | 259 | static const struct xattr_handler afs_xattr_yfs_handler = { |
| @@ -649,12 +649,24 @@ static ssize_t bm_register_write(struct file *file, const char __user *buffer, · 648 unmodified lines | |||
| 649 | struct super_block *sb = file_inode(file)->i_sb; | 649 | struct super_block *sb = file_inode(file)->i_sb; |
| 650 | struct dentry *root = sb->s_root, *dentry; | 650 | struct dentry *root = sb->s_root, *dentry; |
| 651 | int err = 0; | 651 | int err = 0; |
| 652 | struct file *f = NULL; | ||
| 652 | 653 | ||
| 653 | e = create_entry(buffer, count); | 654 | e = create_entry(buffer, count); |
| 654 | 655 | ||
| 655 | if (IS_ERR(e)) | 656 | if (IS_ERR(e)) |
| 656 | return PTR_ERR(e); | 657 | return PTR_ERR(e); |
| 657 | 658 | ||
| 659 | if (e->flags & MISC_FMT_OPEN_FILE) { | ||
| 660 | f = open_exec(e->interpreter); | ||
| 661 | if (IS_ERR(f)) { | ||
| 662 | pr_notice("register: failed to install interpreter file %s\n", | ||
| 663 | e->interpreter); | ||
| 664 | kfree(e); | ||
| 665 | return PTR_ERR(f); | ||
| 666 | } | ||
| 667 | e->interp_file = f; | ||
| 668 | } | ||
| 669 | |||
| 658 | inode_lock(d_inode(root)); | 670 | inode_lock(d_inode(root)); |
| 659 | dentry = lookup_one_len(e->name, root, strlen(e->name)); | 671 | dentry = lookup_one_len(e->name, root, strlen(e->name)); |
| 660 | err = PTR_ERR(dentry); | 672 | err = PTR_ERR(dentry); |
| @@ -678,21 +690,6 @@ static ssize_t bm_register_write(struct file *file, const char __user *buffer, · 17 unmodified lines | |||
| 678 | goto out2; | 690 | goto out2; |
| 679 | } | 691 | } |
| 680 | 692 | ||
| 681 | if (e->flags & MISC_FMT_OPEN_FILE) { | ||
| 682 | struct file *f; | ||
| 683 | |||
| 684 | f = open_exec(e->interpreter); | ||
| 685 | if (IS_ERR(f)) { | ||
| 686 | err = PTR_ERR(f); | ||
| 687 | pr_notice("register: failed to install interpreter file %s\n", e->interpreter); | ||
| 688 | simple_release_fs(&bm_mnt, &entry_count); | ||
| 689 | iput(inode); | ||
| 690 | inode = NULL; | ||
| 691 | goto out2; | ||
| 692 | } | ||
| 693 | e->interp_file = f; | ||
| 694 | } | ||
| 695 | |||
| 696 | e->dentry = dget(dentry); | 693 | e->dentry = dget(dentry); |
| 697 | inode->i_private = e; | 694 | inode->i_private = e; |
| 698 | inode->i_fop = &bm_entry_operations; | 695 | inode->i_fop = &bm_entry_operations; |
| @@ -709,6 +706,8 @@ out: · 10 unmodified lines | |||
| 709 | inode_unlock(d_inode(root)); | 706 | inode_unlock(d_inode(root)); |
| 710 | 707 | ||
| 711 | if (err) { | 708 | if (err) { |
| 709 | if (f) | ||
| 710 | filp_close(f, NULL); | ||
| 712 | kfree(e); | 711 | kfree(e); |
| 713 | return err; | 712 | return err; |
| 714 | } | 713 | } |
| @@ -118,13 +118,22 @@ int truncate_bdev_range(struct block_device *bdev, fmode_t mode, · 117 unmodified lines | |||
| 118 | if (!(mode & FMODE_EXCL)) { | 118 | if (!(mode & FMODE_EXCL)) { |
| 119 | int err = bd_prepare_to_claim(bdev, truncate_bdev_range); | 119 | int err = bd_prepare_to_claim(bdev, truncate_bdev_range); |
| 120 | if (err) | 120 | if (err) |
| 121 | return err; | 121 | goto invalidate; |
| 122 | } | 122 | } |
| 123 | 123 | ||
| 124 | truncate_inode_pages_range(bdev->bd_inode->i_mapping, lstart, lend); | 124 | truncate_inode_pages_range(bdev->bd_inode->i_mapping, lstart, lend); |
| 125 | if (!(mode & FMODE_EXCL)) | 125 | if (!(mode & FMODE_EXCL)) |
| 126 | bd_abort_claiming(bdev, truncate_bdev_range); | 126 | bd_abort_claiming(bdev, truncate_bdev_range); |
| 127 | return 0; | 127 | return 0; |
| 128 | |||
| 129 | invalidate: | ||
| 130 | /* | ||
| 131 | * Someone else has handle exclusively open. Try invalidating instead. | ||
| 132 | * The 'end' argument is inclusive so the rounding is safe. | ||
| 133 | */ | ||
| 134 | return invalidate_inode_pages2_range(bdev->bd_inode->i_mapping, | ||
| 135 | lstart >> PAGE_SHIFT, | ||
| 136 | lend >> PAGE_SHIFT); | ||
| 128 | } | 137 | } |
| 129 | 138 | ||
| 130 | static void set_init_blocksize(struct block_device *bdev) | 139 | static void set_init_blocksize(struct block_device *bdev) |
| @@ -423,7 +432,7 @@ static ssize_t __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, · 292 unmodified lines | |||
| 423 | dio->size += bio->bi_iter.bi_size; | 432 | dio->size += bio->bi_iter.bi_size; |
| 424 | pos += bio->bi_iter.bi_size; | 433 | pos += bio->bi_iter.bi_size; |
| 425 | 434 | ||
| 426 | nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_PAGES); | 435 | nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS); |
| 427 | if (!nr_pages) { | 436 | if (!nr_pages) { |
| 428 | bool polled = false; | 437 | bool polled = false; |
| 429 | 438 | ||
| @@ -491,8 +500,8 @@ blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter) · 61 unmodified lines | |||
| 491 | if (!iov_iter_count(iter)) | 500 | if (!iov_iter_count(iter)) |
| 492 | return 0; | 501 | return 0; |
| 493 | 502 | ||
| 494 | nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_PAGES + 1); | 503 | nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS + 1); |
| 495 | if (is_sync_kiocb(iocb) && nr_pages <= BIO_MAX_PAGES) | 504 | if (is_sync_kiocb(iocb) && nr_pages <= BIO_MAX_VECS) |
| 496 | return __blkdev_direct_IO_simple(iocb, iter, nr_pages); | 505 | return __blkdev_direct_IO_simple(iocb, iter, nr_pages); |
| 497 | 506 | ||
| 498 | return __blkdev_direct_IO(iocb, iter, bio_max_segs(nr_pages)); | 507 | return __blkdev_direct_IO(iocb, iter, bio_max_segs(nr_pages)); |
| @@ -3059,7 +3059,7 @@ struct bio *btrfs_bio_alloc(u64 first_byte) · 3058 unmodified lines | |||
| 3059 | { | 3059 | { |
| 3060 | struct bio *bio; | 3060 | struct bio *bio; |
| 3061 | 3061 | ||
| 3062 | bio = bio_alloc_bioset(GFP_NOFS, BIO_MAX_PAGES, &btrfs_bioset); | 3062 | bio = bio_alloc_bioset(GFP_NOFS, BIO_MAX_VECS, &btrfs_bioset); |
| 3063 | bio->bi_iter.bi_sector = first_byte >> 9; | 3063 | bio->bi_iter.bi_sector = first_byte >> 9; |
| 3064 | btrfs_io_bio_init(btrfs_io_bio(bio)); | 3064 | btrfs_io_bio_init(btrfs_io_bio(bio)); |
| 3065 | return bio; | 3065 | return bio; |
| @@ -1428,7 +1428,7 @@ static void scrub_recheck_block_on_raid56(struct btrfs_fs_info *fs_info, · 1427 unmodified lines | |||
| 1428 | if (!first_page->dev->bdev) | 1428 | if (!first_page->dev->bdev) |
| 1429 | goto out; | 1429 | goto out; |
| 1430 | 1430 | ||
| 1431 | bio = btrfs_io_bio_alloc(BIO_MAX_PAGES); | 1431 | bio = btrfs_io_bio_alloc(BIO_MAX_VECS); |
| 1432 | bio_set_dev(bio, first_page->dev->bdev); | 1432 | bio_set_dev(bio, first_page->dev->bdev); |
| 1433 | 1433 | ||
| 1434 | for (page_num = 0; page_num < sblock->page_count; page_num++) { | 1434 | for (page_num = 0; page_num < sblock->page_count; page_num++) { |
| @@ -207,7 +207,7 @@ static int cifs_debug_files_proc_show(struct seq_file *m, void *v) · 206 unmodified lines | |||
| 207 | from_kuid(&init_user_ns, cfile->uid), | 207 | from_kuid(&init_user_ns, cfile->uid), |
| 208 | cfile->dentry); | 208 | cfile->dentry); |
| 209 | #ifdef CONFIG_CIFS_DEBUG2 | 209 | #ifdef CONFIG_CIFS_DEBUG2 |
| 210 | seq_printf(m, " 0x%llx\n", cfile->fid.mid); | 210 | seq_printf(m, " %llu\n", cfile->fid.mid); |
| 211 | #else | 211 | #else |
| 212 | seq_printf(m, "\n"); | 212 | seq_printf(m, "\n"); |
| 213 | #endif /* CIFS_DEBUG2 */ | 213 | #endif /* CIFS_DEBUG2 */ |
| @@ -290,7 +290,7 @@ cifs_statfs(struct dentry *dentry, struct kstatfs *buf) · 289 unmodified lines | |||
| 290 | rc = server->ops->queryfs(xid, tcon, cifs_sb, buf); | 290 | rc = server->ops->queryfs(xid, tcon, cifs_sb, buf); |
| 291 | 291 | ||
| 292 | free_xid(xid); | 292 | free_xid(xid); |
| 293 | return 0; | 293 | return rc; |
| 294 | } | 294 | } |
| 295 | 295 | ||
| 296 | static long cifs_fallocate(struct file *file, int mode, loff_t off, loff_t len) | 296 | static long cifs_fallocate(struct file *file, int mode, loff_t off, loff_t len) |
| @@ -257,7 +257,7 @@ struct smb_version_operations { · 256 unmodified lines | |||
| 257 | /* verify the message */ | 257 | /* verify the message */ |
| 258 | int (*check_message)(char *, unsigned int, struct TCP_Server_Info *); | 258 | int (*check_message)(char *, unsigned int, struct TCP_Server_Info *); |
| 259 | bool (*is_oplock_break)(char *, struct TCP_Server_Info *); | 259 | bool (*is_oplock_break)(char *, struct TCP_Server_Info *); |
| 260 | int (*handle_cancelled_mid)(char *, struct TCP_Server_Info *); | 260 | int (*handle_cancelled_mid)(struct mid_q_entry *, struct TCP_Server_Info *); |
| 261 | void (*downgrade_oplock)(struct TCP_Server_Info *server, | 261 | void (*downgrade_oplock)(struct TCP_Server_Info *server, |
| 262 | struct cifsInodeInfo *cinode, __u32 oplock, | 262 | struct cifsInodeInfo *cinode, __u32 oplock, |
| 263 | unsigned int epoch, bool *purge_cache); | 263 | unsigned int epoch, bool *purge_cache); |
| @@ -1705,16 +1705,17 @@ static inline bool is_retryable_error(int error) · 1441 unmodified lines | |||
| 1705 | #define CIFS_NO_RSP_BUF 0x040 /* no response buffer required */ | 1705 | #define CIFS_NO_RSP_BUF 0x040 /* no response buffer required */ |
| 1706 | 1706 | ||
| 1707 | /* Type of request operation */ | 1707 | /* Type of request operation */ |
| 1708 | #define CIFS_ECHO_OP 0x080 /* echo request */ | 1708 | #define CIFS_ECHO_OP 0x080 /* echo request */ |
| 1709 | #define CIFS_OBREAK_OP 0x0100 /* oplock break request */ | 1709 | #define CIFS_OBREAK_OP 0x0100 /* oplock break request */ |
| 1710 | #define CIFS_NEG_OP 0x0200 /* negotiate request */ | 1710 | #define CIFS_NEG_OP 0x0200 /* negotiate request */ |
| 1711 | #define CIFS_CP_CREATE_CLOSE_OP 0x0400 /* compound create+close request */ | ||
| 1711 | /* Lower bitmask values are reserved by others below. */ | 1712 | /* Lower bitmask values are reserved by others below. */ |
| 1712 | #define CIFS_SESS_OP 0x2000 /* session setup request */ | 1713 | #define CIFS_SESS_OP 0x2000 /* session setup request */ |
| 1713 | #define CIFS_OP_MASK 0x2380 /* mask request type */ | 1714 | #define CIFS_OP_MASK 0x2780 /* mask request type */ |
| 1714 | 1715 | ||
| 1715 | #define CIFS_HAS_CREDITS 0x0400 /* already has credits */ | 1716 | #define CIFS_HAS_CREDITS 0x0400 /* already has credits */ |
| 1716 | #define CIFS_TRANSFORM_REQ 0x0800 /* transform request before sending */ | 1717 | #define CIFS_TRANSFORM_REQ 0x0800 /* transform request before sending */ |
| 1717 | #define CIFS_NO_SRV_RSP 0x1000 /* there is no server response */ | 1718 | #define CIFS_NO_SRV_RSP 0x1000 /* there is no server response */ |
| 1718 | 1719 | ||
| 1719 | /* Security Flags: indicate type of session setup needed */ | 1720 | /* Security Flags: indicate type of session setup needed */ |
| 1720 | #define CIFSSEC_MAY_SIGN 0x00001 | 1721 | #define CIFSSEC_MAY_SIGN 0x00001 |
| @@ -741,7 +741,7 @@ static void clean_demultiplex_info(struct TCP_Server_Info *server) · 740 unmodified lines | |||
| 741 | spin_lock(&GlobalMid_Lock); | 741 | spin_lock(&GlobalMid_Lock); |
| 742 | list_for_each_safe(tmp, tmp2, &server->pending_mid_q) { | 742 | list_for_each_safe(tmp, tmp2, &server->pending_mid_q) { |
| 743 | mid_entry = list_entry(tmp, struct mid_q_entry, qhead); | 743 | mid_entry = list_entry(tmp, struct mid_q_entry, qhead); |
| 744 | cifs_dbg(FYI, "Clearing mid 0x%llx\n", mid_entry->mid); | 744 | cifs_dbg(FYI, "Clearing mid %llu\n", mid_entry->mid); |
| 745 | kref_get(&mid_entry->refcount); | 745 | kref_get(&mid_entry->refcount); |
| 746 | mid_entry->mid_state = MID_SHUTDOWN; | 746 | mid_entry->mid_state = MID_SHUTDOWN; |
| 747 | list_move(&mid_entry->qhead, &dispose_list); | 747 | list_move(&mid_entry->qhead, &dispose_list); |
| @@ -752,7 +752,7 @@ static void clean_demultiplex_info(struct TCP_Server_Info *server) · 4 unmodified lines | |||
| 752 | /* now walk dispose list and issue callbacks */ | 752 | /* now walk dispose list and issue callbacks */ |
| 753 | list_for_each_safe(tmp, tmp2, &dispose_list) { | 753 | list_for_each_safe(tmp, tmp2, &dispose_list) { |
| 754 | mid_entry = list_entry(tmp, struct mid_q_entry, qhead); | 754 | mid_entry = list_entry(tmp, struct mid_q_entry, qhead); |
| 755 | cifs_dbg(FYI, "Callback mid 0x%llx\n", mid_entry->mid); | 755 | cifs_dbg(FYI, "Callback mid %llu\n", mid_entry->mid); |
| 756 | list_del_init(&mid_entry->qhead); | 756 | list_del_init(&mid_entry->qhead); |
| 757 | mid_entry->callback(mid_entry); | 757 | mid_entry->callback(mid_entry); |
| 758 | cifs_mid_q_entry_release(mid_entry); | 758 | cifs_mid_q_entry_release(mid_entry); |
| @@ -1429,6 +1429,11 @@ smbd_connected: · 670 unmodified lines | |||
| 1429 | tcp_ses->min_offload = ctx->min_offload; | 1429 | tcp_ses->min_offload = ctx->min_offload; |
| 1430 | tcp_ses->tcpStatus = CifsNeedNegotiate; | 1430 | tcp_ses->tcpStatus = CifsNeedNegotiate; |
| 1431 | 1431 | ||
| 1432 | if ((ctx->max_credits < 20) || (ctx->max_credits > 60000)) | ||
| 1433 | tcp_ses->max_credits = SMB2_MAX_CREDITS_AVAILABLE; | ||
| 1434 | else | ||
| 1435 | tcp_ses->max_credits = ctx->max_credits; | ||
| 1436 | |||
| 1432 | tcp_ses->nr_targets = 1; | 1437 | tcp_ses->nr_targets = 1; |
| 1433 | tcp_ses->ignore_signature = ctx->ignore_signature; | 1438 | tcp_ses->ignore_signature = ctx->ignore_signature; |
| 1434 | /* thread spawned, put it on the list */ | 1439 | /* thread spawned, put it on the list */ |
| @@ -2832,11 +2837,6 @@ static int mount_get_conns(struct smb3_fs_context *ctx, struct cifs_sb_info *cif · 1397 unmodified lines | |||
| 2832 | 2837 | ||
| 2833 | *nserver = server; | 2838 | *nserver = server; |
| 2834 | 2839 | ||
| 2835 | if ((ctx->max_credits < 20) || (ctx->max_credits > 60000)) | ||
| 2836 | server->max_credits = SMB2_MAX_CREDITS_AVAILABLE; | ||
| 2837 | else | ||
| 2838 | server->max_credits = ctx->max_credits; | ||
| 2839 | |||
| 2840 | /* get a reference to a SMB session */ | 2840 | /* get a reference to a SMB session */ |
| 2841 | ses = cifs_get_smb_ses(server, ctx); | 2841 | ses = cifs_get_smb_ses(server, ctx); |
| 2842 | if (IS_ERR(ses)) { | 2842 | if (IS_ERR(ses)) { |
| @@ -230,6 +230,7 @@ cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses, · 229 unmodified lines | |||
| 230 | ctx.noautotune = ses->server->noautotune; | 230 | ctx.noautotune = ses->server->noautotune; |
| 231 | ctx.sockopt_tcp_nodelay = ses->server->tcp_nodelay; | 231 | ctx.sockopt_tcp_nodelay = ses->server->tcp_nodelay; |
| 232 | ctx.echo_interval = ses->server->echo_interval / HZ; | 232 | ctx.echo_interval = ses->server->echo_interval / HZ; |
| 233 | ctx.max_credits = ses->server->max_credits; | ||
| 233 | 234 | ||
| 234 | /* | 235 | /* |
| 235 | * This will be used for encoding/decoding user/domain/pw | 236 | * This will be used for encoding/decoding user/domain/pw |
| @@ -358,6 +358,7 @@ smb2_compound_op(const unsigned int xid, struct cifs_tcon *tcon, · 357 unmodified lines | |||
| 358 | if (cfile) | 358 | if (cfile) |
| 359 | goto after_close; | 359 | goto after_close; |
| 360 | /* Close */ | 360 | /* Close */ |
| 361 | flags |= CIFS_CP_CREATE_CLOSE_OP; | ||
| 361 | rqst[num_rqst].rq_iov = &vars->close_iov[0]; | 362 | rqst[num_rqst].rq_iov = &vars->close_iov[0]; |
| 362 | rqst[num_rqst].rq_nvec = 1; | 363 | rqst[num_rqst].rq_nvec = 1; |
| 363 | rc = SMB2_close_init(tcon, server, | 364 | rc = SMB2_close_init(tcon, server, |
| @@ -767,7 +767,7 @@ smb2_cancelled_close_fid(struct work_struct *work) · 766 unmodified lines | |||
| 767 | int rc; | 767 | int rc; |
| 768 | 768 | ||
| 769 | if (cancelled->mid) | 769 | if (cancelled->mid) |
| 770 | cifs_tcon_dbg(VFS, "Close unmatched open for MID:%llx\n", | 770 | cifs_tcon_dbg(VFS, "Close unmatched open for MID:%llu\n", |
| 771 | cancelled->mid); | 771 | cancelled->mid); |
| 772 | else | 772 | else |
| 773 | cifs_tcon_dbg(VFS, "Close interrupted close\n"); | 773 | cifs_tcon_dbg(VFS, "Close interrupted close\n"); |
| @@ -844,14 +844,14 @@ smb2_handle_cancelled_close(struct cifs_tcon *tcon, __u64 persistent_fid, · 70 unmodified lines | |||
| 844 | } | 844 | } |
| 845 | 845 | ||
| 846 | int | 846 | int |
| 847 | smb2_handle_cancelled_mid(char *buffer, struct TCP_Server_Info *server) | 847 | smb2_handle_cancelled_mid(struct mid_q_entry *mid, struct TCP_Server_Info *server) |
| 848 | { | 848 | { |
| 849 | struct smb2_sync_hdr *sync_hdr = (struct smb2_sync_hdr *)buffer; | 849 | struct smb2_sync_hdr *sync_hdr = mid->resp_buf; |
| 850 | struct smb2_create_rsp *rsp = (struct smb2_create_rsp *)buffer; | 850 | struct smb2_create_rsp *rsp = mid->resp_buf; |
| 851 | struct cifs_tcon *tcon; | 851 | struct cifs_tcon *tcon; |
| 852 | int rc; | 852 | int rc; |
| 853 | 853 | ||
| 854 | if (sync_hdr->Command != SMB2_CREATE || | 854 | if ((mid->optype & CIFS_CP_CREATE_CLOSE_OP) || sync_hdr->Command != SMB2_CREATE || |
| 855 | sync_hdr->Status != STATUS_SUCCESS) | 855 | sync_hdr->Status != STATUS_SUCCESS) |
| 856 | return 0; | 856 | return 0; |
| 857 | 857 | ||
| @@ -1195,7 +1195,7 @@ smb2_set_ea(const unsigned int xid, struct cifs_tcon *tcon, · 1194 unmodified lines | |||
| 1195 | struct TCP_Server_Info *server = cifs_pick_channel(ses); | 1195 | struct TCP_Server_Info *server = cifs_pick_channel(ses); |
| 1196 | __le16 *utf16_path = NULL; | 1196 | __le16 *utf16_path = NULL; |
| 1197 | int ea_name_len = strlen(ea_name); | 1197 | int ea_name_len = strlen(ea_name); |
| 1198 | int flags = 0; | 1198 | int flags = CIFS_CP_CREATE_CLOSE_OP; |
| 1199 | int len; | 1199 | int len; |
| 1200 | struct smb_rqst rqst[3]; | 1200 | struct smb_rqst rqst[3]; |
| 1201 | int resp_buftype[3]; | 1201 | int resp_buftype[3]; |
| @@ -1573,7 +1573,7 @@ smb2_ioctl_query_info(const unsigned int xid, · 371 unmodified lines | |||
| 1573 | struct smb_query_info qi; | 1573 | struct smb_query_info qi; |
| 1574 | struct smb_query_info __user *pqi; | 1574 | struct smb_query_info __user *pqi; |
| 1575 | int rc = 0; | 1575 | int rc = 0; |
| 1576 | int flags = 0; | 1576 | int flags = CIFS_CP_CREATE_CLOSE_OP; |
| 1577 | struct smb2_query_info_rsp *qi_rsp = NULL; | 1577 | struct smb2_query_info_rsp *qi_rsp = NULL; |
| 1578 | struct smb2_ioctl_rsp *io_rsp = NULL; | 1578 | struct smb2_ioctl_rsp *io_rsp = NULL; |
| 1579 | void *buffer = NULL; | 1579 | void *buffer = NULL; |
| @@ -2577,7 +2577,7 @@ smb2_query_info_compound(const unsigned int xid, struct cifs_tcon *tcon, · 997 unmodified lines | |||
| 2577 | { | 2577 | { |
| 2578 | struct cifs_ses *ses = tcon->ses; | 2578 | struct cifs_ses *ses = tcon->ses; |
| 2579 | struct TCP_Server_Info *server = cifs_pick_channel(ses); | 2579 | struct TCP_Server_Info *server = cifs_pick_channel(ses); |
| 2580 | int flags = 0; | 2580 | int flags = CIFS_CP_CREATE_CLOSE_OP; |
| 2581 | struct smb_rqst rqst[3]; | 2581 | struct smb_rqst rqst[3]; |
| 2582 | int resp_buftype[3]; | 2582 | int resp_buftype[3]; |
| 2583 | struct kvec rsp_iov[3]; | 2583 | struct kvec rsp_iov[3]; |
| @@ -2975,7 +2975,7 @@ smb2_query_symlink(const unsigned int xid, struct cifs_tcon *tcon, · 391 unmodified lines | |||
| 2975 | unsigned int sub_offset; | 2975 | unsigned int sub_offset; |
| 2976 | unsigned int print_len; | 2976 | unsigned int print_len; |
| 2977 | unsigned int print_offset; | 2977 | unsigned int print_offset; |
| 2978 | int flags = 0; | 2978 | int flags = CIFS_CP_CREATE_CLOSE_OP; |
| 2979 | struct smb_rqst rqst[3]; | 2979 | struct smb_rqst rqst[3]; |
| 2980 | int resp_buftype[3]; | 2980 | int resp_buftype[3]; |
| 2981 | struct kvec rsp_iov[3]; | 2981 | struct kvec rsp_iov[3]; |
| @@ -3157,7 +3157,7 @@ smb2_query_reparse_tag(const unsigned int xid, struct cifs_tcon *tcon, · 175 unmodified lines | |||
| 3157 | struct cifs_open_parms oparms; | 3157 | struct cifs_open_parms oparms; |
| 3158 | struct cifs_fid fid; | 3158 | struct cifs_fid fid; |
| 3159 | struct TCP_Server_Info *server = cifs_pick_channel(tcon->ses); | 3159 | struct TCP_Server_Info *server = cifs_pick_channel(tcon->ses); |
| 3160 | int flags = 0; | 3160 | int flags = CIFS_CP_CREATE_CLOSE_OP; |
| 3161 | struct smb_rqst rqst[3]; | 3161 | struct smb_rqst rqst[3]; |
| 3162 | int resp_buftype[3]; | 3162 | int resp_buftype[3]; |
| 3163 | struct kvec rsp_iov[3]; | 3163 | struct kvec rsp_iov[3]; |
| @@ -4041,8 +4041,7 @@ smb2_async_readv(struct cifs_readdata *rdata) · 4040 unmodified lines | |||
| 4041 | if (rdata->credits.value > 0) { | 4041 | if (rdata->credits.value > 0) { |
| 4042 | shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(rdata->bytes, | 4042 | shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(rdata->bytes, |
| 4043 | SMB2_MAX_BUFFER_SIZE)); | 4043 | SMB2_MAX_BUFFER_SIZE)); |
| 4044 | shdr->CreditRequest = | 4044 | shdr->CreditRequest = cpu_to_le16(le16_to_cpu(shdr->CreditCharge) + 8); |
| 4045 | cpu_to_le16(le16_to_cpu(shdr->CreditCharge) + 1); | ||
| 4046 | 4045 | ||
| 4047 | rc = adjust_credits(server, &rdata->credits, rdata->bytes); | 4046 | rc = adjust_credits(server, &rdata->credits, rdata->bytes); |
| 4048 | if (rc) | 4047 | if (rc) |
| @@ -4348,8 +4347,7 @@ smb2_async_writev(struct cifs_writedata *wdata, · 299 unmodified lines | |||
| 4348 | if (wdata->credits.value > 0) { | 4347 | if (wdata->credits.value > 0) { |
| 4349 | shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(wdata->bytes, | 4348 | shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(wdata->bytes, |
| 4350 | SMB2_MAX_BUFFER_SIZE)); | 4349 | SMB2_MAX_BUFFER_SIZE)); |
| 4351 | shdr->CreditRequest = | 4350 | shdr->CreditRequest = cpu_to_le16(le16_to_cpu(shdr->CreditCharge) + 8); |
| 4352 | cpu_to_le16(le16_to_cpu(shdr->CreditCharge) + 1); | ||
| 4353 | 4351 | ||
| 4354 | rc = adjust_credits(server, &wdata->credits, wdata->bytes); | 4352 | rc = adjust_credits(server, &wdata->credits, wdata->bytes); |
| 4355 | if (rc) | 4353 | if (rc) |
| @@ -246,8 +246,7 @@ extern int SMB2_oplock_break(const unsigned int xid, struct cifs_tcon *tcon, · 245 unmodified lines | |||
| 246 | extern int smb2_handle_cancelled_close(struct cifs_tcon *tcon, | 246 | extern int smb2_handle_cancelled_close(struct cifs_tcon *tcon, |
| 247 | __u64 persistent_fid, | 247 | __u64 persistent_fid, |
| 248 | __u64 volatile_fid); | 248 | __u64 volatile_fid); |
| 249 | extern int smb2_handle_cancelled_mid(char *buffer, | 249 | extern int smb2_handle_cancelled_mid(struct mid_q_entry *mid, struct TCP_Server_Info *server); |
| 250 | struct TCP_Server_Info *server); | ||
| 251 | void smb2_cancelled_close_fid(struct work_struct *work); | 250 | void smb2_cancelled_close_fid(struct work_struct *work); |
| 252 | extern int SMB2_QFS_info(const unsigned int xid, struct cifs_tcon *tcon, | 251 | extern int SMB2_QFS_info(const unsigned int xid, struct cifs_tcon *tcon, |
| 253 | u64 persistent_file_id, u64 volatile_file_id, | 252 | u64 persistent_file_id, u64 volatile_file_id, |
| @@ -101,7 +101,7 @@ static void _cifs_mid_q_entry_release(struct kref *refcount) · 100 unmodified lines | |||
| 101 | if (midEntry->resp_buf && (midEntry->mid_flags & MID_WAIT_CANCELLED) && | 101 | if (midEntry->resp_buf && (midEntry->mid_flags & MID_WAIT_CANCELLED) && |
| 102 | midEntry->mid_state == MID_RESPONSE_RECEIVED && | 102 | midEntry->mid_state == MID_RESPONSE_RECEIVED && |
| 103 | server->ops->handle_cancelled_mid) | 103 | server->ops->handle_cancelled_mid) |
| 104 | server->ops->handle_cancelled_mid(midEntry->resp_buf, server); | 104 | server->ops->handle_cancelled_mid(midEntry, server); |
| 105 | 105 | ||
| 106 | midEntry->mid_state = MID_FREE; | 106 | midEntry->mid_state = MID_FREE; |
| 107 | atomic_dec(&midCount); | 107 | atomic_dec(&midCount); |
| @@ -1207,7 +1207,7 @@ compound_send_recv(const unsigned int xid, struct cifs_ses *ses, · 1099 unmodified lines | |||
| 1207 | } | 1207 | } |
| 1208 | if (rc != 0) { | 1208 | if (rc != 0) { |
| 1209 | for (; i < num_rqst; i++) { | 1209 | for (; i < num_rqst; i++) { |
| 1210 | cifs_server_dbg(VFS, "Cancelling wait for mid %llu cmd: %d\n", | 1210 | cifs_server_dbg(FYI, "Cancelling wait for mid %llu cmd: %d\n", |
| 1211 | midQ[i]->mid, le16_to_cpu(midQ[i]->command)); | 1211 | midQ[i]->mid, le16_to_cpu(midQ[i]->command)); |
| 1212 | send_cancel(server, &rqst[i], midQ[i]); | 1212 | send_cancel(server, &rqst[i], midQ[i]); |
| 1213 | spin_lock(&GlobalMid_Lock); | 1213 | spin_lock(&GlobalMid_Lock); |
| @@ -378,7 +378,7 @@ static int __configfs_open_file(struct inode *inode, struct file *file, int type · 377 unmodified lines | |||
| 378 | 378 | ||
| 379 | attr = to_attr(dentry); | 379 | attr = to_attr(dentry); |
| 380 | if (!attr) | 380 | if (!attr) |
| 381 | goto out_put_item; | 381 | goto out_free_buffer; |
| 382 | 382 | ||
| 383 | if (type & CONFIGFS_ITEM_BIN_ATTR) { | 383 | if (type & CONFIGFS_ITEM_BIN_ATTR) { |
| 384 | buffer->bin_attr = to_bin_attr(dentry); | 384 | buffer->bin_attr = to_bin_attr(dentry); |
| @@ -391,7 +391,7 @@ static int __configfs_open_file(struct inode *inode, struct file *file, int type · 6 unmodified lines | |||
| 391 | /* Grab the module reference for this attribute if we have one */ | 391 | /* Grab the module reference for this attribute if we have one */ |
| 392 | error = -ENODEV; | 392 | error = -ENODEV; |
| 393 | if (!try_module_get(buffer->owner)) | 393 | if (!try_module_get(buffer->owner)) |
| 394 | goto out_put_item; | 394 | goto out_free_buffer; |
| 395 | 395 | ||
| 396 | error = -EACCES; | 396 | error = -EACCES; |
| 397 | if (!buffer->item->ci_type) | 397 | if (!buffer->item->ci_type) |
| @@ -435,8 +435,6 @@ static int __configfs_open_file(struct inode *inode, struct file *file, int type · 37 unmodified lines | |||
| 435 | 435 | ||
| 436 | out_put_module: | 436 | out_put_module: |
| 437 | module_put(buffer->owner); | 437 | module_put(buffer->owner); |
| 438 | out_put_item: | ||
| 439 | config_item_put(buffer->item); | ||
| 440 | out_free_buffer: | 438 | out_free_buffer: |
| 441 | up_read(&frag->frag_sem); | 439 | up_read(&frag->frag_sem); |
| 442 | kfree(buffer); | 440 | kfree(buffer); |
| @@ -52,7 +52,7 @@ static int fscrypt_zeroout_range_inline_crypt(const struct inode *inode, · 51 unmodified lines | |||
| 52 | int num_pages = 0; | 52 | int num_pages = 0; |
| 53 | 53 | ||
| 54 | /* This always succeeds since __GFP_DIRECT_RECLAIM is set. */ | 54 | /* This always succeeds since __GFP_DIRECT_RECLAIM is set. */ |
| 55 | bio = bio_alloc(GFP_NOFS, BIO_MAX_PAGES); | 55 | bio = bio_alloc(GFP_NOFS, BIO_MAX_VECS); |
| 56 | 56 | ||
| 57 | while (len) { | 57 | while (len) { |
| 58 | unsigned int blocks_this_page = min(len, blocks_per_page); | 58 | unsigned int blocks_this_page = min(len, blocks_per_page); |
| @@ -74,7 +74,7 @@ static int fscrypt_zeroout_range_inline_crypt(const struct inode *inode, · 15 unmodified lines | |||
| 74 | len -= blocks_this_page; | 74 | len -= blocks_this_page; |
| 75 | lblk += blocks_this_page; | 75 | lblk += blocks_this_page; |
| 76 | pblk += blocks_this_page; | 76 | pblk += blocks_this_page; |
| 77 | if (num_pages == BIO_MAX_PAGES || !len || | 77 | if (num_pages == BIO_MAX_VECS || !len || |
| 78 | !fscrypt_mergeable_bio(bio, inode, lblk)) { | 78 | !fscrypt_mergeable_bio(bio, inode, lblk)) { |
| 79 | err = submit_bio_wait(bio); | 79 | err = submit_bio_wait(bio); |
| 80 | if (err) | 80 | if (err) |
| @@ -126,7 +126,7 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, · 45 unmodified lines | |||
| 126 | return fscrypt_zeroout_range_inline_crypt(inode, lblk, pblk, | 126 | return fscrypt_zeroout_range_inline_crypt(inode, lblk, pblk, |
| 127 | len); | 127 | len); |
| 128 | 128 | ||
| 129 | BUILD_BUG_ON(ARRAY_SIZE(pages) > BIO_MAX_PAGES); | 129 | BUILD_BUG_ON(ARRAY_SIZE(pages) > BIO_MAX_VECS); |
| 130 | nr_pages = min_t(unsigned int, ARRAY_SIZE(pages), | 130 | nr_pages = min_t(unsigned int, ARRAY_SIZE(pages), |
| 131 | (len + blocks_per_page - 1) >> blocks_per_page_bits); | 131 | (len + blocks_per_page - 1) >> blocks_per_page_bits); |
| 132 | 132 | ||
| @@ -129,6 +129,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio, · 128 unmodified lines | |||
| 129 | struct page *page, | 129 | struct page *page, |
| 130 | erofs_off_t *last_block, | 130 | erofs_off_t *last_block, |
| 131 | unsigned int nblocks, | 131 | unsigned int nblocks, |
| 132 | unsigned int *eblks, | ||
| 132 | bool ra) | 133 | bool ra) |
| 133 | { | 134 | { |
| 134 | struct inode *const inode = mapping->host; | 135 | struct inode *const inode = mapping->host; |
| @@ -145,8 +146,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio, · 10 unmodified lines | |||
| 145 | 146 | ||
| 146 | /* note that for readpage case, bio also equals to NULL */ | 147 | /* note that for readpage case, bio also equals to NULL */ |
| 147 | if (bio && | 148 | if (bio && |
| 148 | /* not continuous */ | 149 | (*last_block + 1 != current_block || !*eblks)) { |
| 149 | *last_block + 1 != current_block) { | ||
| 150 | submit_bio_retry: | 150 | submit_bio_retry: |
| 151 | submit_bio(bio); | 151 | submit_bio(bio); |
| 152 | bio = NULL; | 152 | bio = NULL; |
| @@ -216,7 +216,8 @@ submit_bio_retry: · 63 unmodified lines | |||
| 216 | if (nblocks > DIV_ROUND_UP(map.m_plen, PAGE_SIZE)) | 216 | if (nblocks > DIV_ROUND_UP(map.m_plen, PAGE_SIZE)) |
| 217 | nblocks = DIV_ROUND_UP(map.m_plen, PAGE_SIZE); | 217 | nblocks = DIV_ROUND_UP(map.m_plen, PAGE_SIZE); |
| 218 | 218 | ||
| 219 | bio = bio_alloc(GFP_NOIO, bio_max_segs(nblocks)); | 219 | *eblks = bio_max_segs(nblocks); |
| 220 | bio = bio_alloc(GFP_NOIO, *eblks); | ||
| 220 | 221 | ||
| 221 | bio->bi_end_io = erofs_readendio; | 222 | bio->bi_end_io = erofs_readendio; |
| 222 | bio_set_dev(bio, sb->s_bdev); | 223 | bio_set_dev(bio, sb->s_bdev); |
| @@ -229,16 +230,8 @@ submit_bio_retry: · 6 unmodified lines | |||
| 229 | /* out of the extent or bio is full */ | 230 | /* out of the extent or bio is full */ |
| 230 | if (err < PAGE_SIZE) | 231 | if (err < PAGE_SIZE) |
| 231 | goto submit_bio_retry; | 232 | goto submit_bio_retry; |
| 232 | 233 | --*eblks; | |
| 233 | *last_block = current_block; | 234 | *last_block = current_block; |
| 234 | |||
| 235 | /* shift in advance in case of it followed by too many gaps */ | ||
| 236 | if (bio->bi_iter.bi_size >= bio->bi_max_vecs * PAGE_SIZE) { | ||
| 237 | /* err should reassign to 0 after submitting */ | ||
| 238 | err = 0; | ||
| 239 | goto submit_bio_out; | ||
| 240 | } | ||
| 241 | |||
| 242 | return bio; | 235 | return bio; |
| 243 | 236 | ||
| 244 | err_out: | 237 | err_out: |
| @@ -252,7 +245,6 @@ has_updated: · 7 unmodified lines | |||
| 252 | 245 | ||
| 253 | /* if updated manually, continuous pages has a gap */ | 246 | /* if updated manually, continuous pages has a gap */ |
| 254 | if (bio) | 247 | if (bio) |
| 255 | submit_bio_out: | ||
| 256 | submit_bio(bio); | 248 | submit_bio(bio); |
| 257 | return err ? ERR_PTR(err) : NULL; | 249 | return err ? ERR_PTR(err) : NULL; |
| 258 | } | 250 | } |
| @@ -264,23 +256,26 @@ submit_bio_out: · 5 unmodified lines | |||
| 264 | static int erofs_raw_access_readpage(struct file *file, struct page *page) | 256 | static int erofs_raw_access_readpage(struct file *file, struct page *page) |
| 265 | { | 257 | { |
| 266 | erofs_off_t last_block; | 258 | erofs_off_t last_block; |
| 259 | unsigned int eblks; | ||
| 267 | struct bio *bio; | 260 | struct bio *bio; |
| 268 | 261 | ||
| 269 | trace_erofs_readpage(page, true); | 262 | trace_erofs_readpage(page, true); |
| 270 | 263 | ||
| 271 | bio = erofs_read_raw_page(NULL, page->mapping, | 264 | bio = erofs_read_raw_page(NULL, page->mapping, |
| 272 | page, &last_block, 1, false); | 265 | page, &last_block, 1, &eblks, false); |
| 273 | 266 | ||
| 274 | if (IS_ERR(bio)) | 267 | if (IS_ERR(bio)) |
| 275 | return PTR_ERR(bio); | 268 | return PTR_ERR(bio); |
| 276 | 269 | ||
| 277 | DBG_BUGON(bio); /* since we have only one bio -- must be NULL */ | 270 | if (bio) |
| 271 | submit_bio(bio); | ||
| 278 | return 0; | 272 | return 0; |
| 279 | } | 273 | } |
| 280 | 274 | ||
| 281 | static void erofs_raw_access_readahead(struct readahead_control *rac) | 275 | static void erofs_raw_access_readahead(struct readahead_control *rac) |
| 282 | { | 276 | { |
| 283 | erofs_off_t last_block; | 277 | erofs_off_t last_block; |
| 278 | unsigned int eblks; | ||
| 284 | struct bio *bio = NULL; | 279 | struct bio *bio = NULL; |
| 285 | struct page *page; | 280 | struct page *page; |
| 286 | 281 | ||
| @@ -291,7 +286,7 @@ static void erofs_raw_access_readahead(struct readahead_control *rac) · 4 unmodified lines | |||
| 291 | prefetchw(&page->flags); | 286 | prefetchw(&page->flags); |
| 292 | 287 | ||
| 293 | bio = erofs_read_raw_page(bio, rac->mapping, page, &last_block, | 288 | bio = erofs_read_raw_page(bio, rac->mapping, page, &last_block, |
| 294 | readahead_count(rac), true); | 289 | readahead_count(rac), &eblks, true); |
| 295 | 290 | ||
| 296 | /* all the page errors are ignored when readahead */ | 291 | /* all the page errors are ignored when readahead */ |
| 297 | if (IS_ERR(bio)) { | 292 | if (IS_ERR(bio)) { |
| @@ -305,7 +300,6 @@ static void erofs_raw_access_readahead(struct readahead_control *rac) · 7 unmodified lines | |||
| 305 | put_page(page); | 300 | put_page(page); |
| 306 | } | 301 | } |
| 307 | 302 | ||
| 308 | /* the rare case (end in gaps) */ | ||
| 309 | if (bio) | 303 | if (bio) |
| 310 | submit_bio(bio); | 304 | submit_bio(bio); |
| 311 | } | 305 | } |
| @@ -1235,7 +1235,7 @@ submit_bio_retry: · 1234 unmodified lines | |||
| 1235 | } | 1235 | } |
| 1236 | 1236 | ||
| 1237 | if (!bio) { | 1237 | if (!bio) { |
| 1238 | bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES); | 1238 | bio = bio_alloc(GFP_NOIO, BIO_MAX_VECS); |
| 1239 | 1239 | ||
| 1240 | bio->bi_end_io = z_erofs_decompressqueue_endio; | 1240 | bio->bi_end_io = z_erofs_decompressqueue_endio; |
| 1241 | bio_set_dev(bio, sb->s_bdev); | 1241 | bio_set_dev(bio, sb->s_bdev); |
| @@ -398,7 +398,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io, · 397 unmodified lines | |||
| 398 | * bio_alloc will _always_ be able to allocate a bio if | 398 | * bio_alloc will _always_ be able to allocate a bio if |
| 399 | * __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). | 399 | * __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). |
| 400 | */ | 400 | */ |
| 401 | bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES); | 401 | bio = bio_alloc(GFP_NOIO, BIO_MAX_VECS); |
| 402 | fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO); | 402 | fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO); |
| 403 | bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9); | 403 | bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9); |
| 404 | bio_set_dev(bio, bh->b_bdev); | 404 | bio_set_dev(bio, bh->b_bdev); |
| @@ -292,7 +292,7 @@ void f2fs_ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index) · 291 unmodified lines | |||
| 292 | f2fs_put_page(page, 0); | 292 | f2fs_put_page(page, 0); |
| 293 | 293 | ||
| 294 | if (readahead) | 294 | if (readahead) |
| 295 | f2fs_ra_meta_pages(sbi, index, BIO_MAX_PAGES, META_POR, true); | 295 | f2fs_ra_meta_pages(sbi, index, BIO_MAX_VECS, META_POR, true); |
| 296 | } | 296 | } |
| 297 | 297 | ||
| 298 | static int __f2fs_write_meta_page(struct page *page, | 298 | static int __f2fs_write_meta_page(struct page *page, |
| @@ -857,7 +857,7 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio) · 856 unmodified lines | |||
| 857 | f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL); | 857 | f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL); |
| 858 | alloc_new: | 858 | alloc_new: |
| 859 | if (!bio) { | 859 | if (!bio) { |
| 860 | bio = __bio_alloc(fio, BIO_MAX_PAGES); | 860 | bio = __bio_alloc(fio, BIO_MAX_VECS); |
| 861 | __attach_io_flag(fio); | 861 | __attach_io_flag(fio); |
| 862 | f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host, | 862 | f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host, |
| 863 | fio->page->index, fio, GFP_NOIO); | 863 | fio->page->index, fio, GFP_NOIO); |
| @@ -932,7 +932,7 @@ alloc_new: · 68 unmodified lines | |||
| 932 | fio->retry = true; | 932 | fio->retry = true; |
| 933 | goto skip; | 933 | goto skip; |
| 934 | } | 934 | } |
| 935 | io->bio = __bio_alloc(fio, BIO_MAX_PAGES); | 935 | io->bio = __bio_alloc(fio, BIO_MAX_VECS); |
| 936 | f2fs_set_bio_crypt_ctx(io->bio, fio->page->mapping->host, | 936 | f2fs_set_bio_crypt_ctx(io->bio, fio->page->mapping->host, |
| 937 | bio_page->index, fio, GFP_NOIO); | 937 | bio_page->index, fio, GFP_NOIO); |
| 938 | io->fio = *fio; | 938 | io->fio = *fio; |
| @@ -4381,7 +4381,7 @@ static int build_sit_entries(struct f2fs_sb_info *sbi) · 4380 unmodified lines | |||
| 4381 | block_t total_node_blocks = 0; | 4381 | block_t total_node_blocks = 0; |
| 4382 | 4382 | ||
| 4383 | do { | 4383 | do { |
| 4384 | readed = f2fs_ra_meta_pages(sbi, start_blk, BIO_MAX_PAGES, | 4384 | readed = f2fs_ra_meta_pages(sbi, start_blk, BIO_MAX_VECS, |
| 4385 | META_SIT, true); | 4385 | META_SIT, true); |
| 4386 | 4386 | ||
| 4387 | start = start_blk * sit_i->sents_per_block; | 4387 | start = start_blk * sit_i->sents_per_block; |
| @@ -851,7 +851,7 @@ static inline int nr_pages_to_skip(struct f2fs_sb_info *sbi, int type) · 850 unmodified lines | |||
| 851 | else if (type == NODE) | 851 | else if (type == NODE) |
| 852 | return 8 * sbi->blocks_per_seg; | 852 | return 8 * sbi->blocks_per_seg; |
| 853 | else if (type == META) | 853 | else if (type == META) |
| 854 | return 8 * BIO_MAX_PAGES; | 854 | return 8 * BIO_MAX_VECS; |
| 855 | else | 855 | else |
| 856 | return 0; | 856 | return 0; |
| 857 | } | 857 | } |
| @@ -868,7 +868,7 @@ static inline long nr_pages_to_write(struct f2fs_sb_info *sbi, int type, · 10 unmodified lines | |||
| 868 | return 0; | 868 | return 0; |
| 869 | 869 | ||
| 870 | nr_to_write = wbc->nr_to_write; | 870 | nr_to_write = wbc->nr_to_write; |
| 871 | desired = BIO_MAX_PAGES; | 871 | desired = BIO_MAX_VECS; |
| 872 | if (type == NODE) | 872 | if (type == NODE) |
| 873 | desired <<= 1; | 873 | desired <<= 1; |
| 874 | 874 | ||
| @@ -753,9 +753,9 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount) · 752 unmodified lines | |||
| 753 | case Opt_io_size_bits: | 753 | case Opt_io_size_bits: |
| 754 | if (args->from && match_int(args, &arg)) | 754 | if (args->from && match_int(args, &arg)) |
| 755 | return -EINVAL; | 755 | return -EINVAL; |
| 756 | if (arg <= 0 || arg > __ilog2_u32(BIO_MAX_PAGES)) { | 756 | if (arg <= 0 || arg > __ilog2_u32(BIO_MAX_VECS)) { |
| 757 | f2fs_warn(sbi, "Not support %d, larger than %d", | 757 | f2fs_warn(sbi, "Not support %d, larger than %d", |
| 758 | 1 << arg, BIO_MAX_PAGES); | 758 | 1 << arg, BIO_MAX_VECS); |
| 759 | return -EINVAL; | 759 | return -EINVAL; |
| 760 | } | 760 | } |
| 761 | F2FS_OPTION(sbi).write_io_size_bits = arg; | 761 | F2FS_OPTION(sbi).write_io_size_bits = arg; |
| @@ -2229,19 +2229,21 @@ static int fuse_device_clone(struct fuse_conn *fc, struct file *new) · 2228 unmodified lines | |||
| 2229 | static long fuse_dev_ioctl(struct file *file, unsigned int cmd, | 2229 | static long fuse_dev_ioctl(struct file *file, unsigned int cmd, |
| 2230 | unsigned long arg) | 2230 | unsigned long arg) |
| 2231 | { | 2231 | { |
| 2232 | int err = -ENOTTY; | 2232 | int res; |
| 2233 | int oldfd; | ||
| 2234 | struct fuse_dev *fud = NULL; | ||
| 2233 | 2235 | ||
| 2234 | if (cmd == FUSE_DEV_IOC_CLONE) { | 2236 | if (_IOC_TYPE(cmd) != FUSE_DEV_IOC_MAGIC) |
| 2235 | int oldfd; | 2237 | return -ENOTTY; |
| 2236 | 2238 | ||
| 2237 | err = -EFAULT; | 2239 | switch (_IOC_NR(cmd)) { |
| 2238 | if (!get_user(oldfd, (__u32 __user *) arg)) { | 2240 | case _IOC_NR(FUSE_DEV_IOC_CLONE): |
| 2241 | res = -EFAULT; | ||
| 2242 | if (!get_user(oldfd, (__u32 __user *)arg)) { | ||
| 2239 | struct file *old = fget(oldfd); | 2243 | struct file *old = fget(oldfd); |
| 2240 | 2244 | ||
| 2241 | err = -EINVAL; | 2245 | res = -EINVAL; |
| 2242 | if (old) { | 2246 | if (old) { |
| 2243 | struct fuse_dev *fud = NULL; | ||
| 2244 | |||
| 2245 | /* | 2247 | /* |
| 2246 | * Check against file->f_op because CUSE | 2248 | * Check against file->f_op because CUSE |
| 2247 | * uses the same ioctl handler. | 2249 | * uses the same ioctl handler. |
| @@ -2252,14 +2254,18 @@ static long fuse_dev_ioctl(struct file *file, unsigned int cmd, · 4 unmodified lines | |||
| 2252 | 2254 | ||
| 2253 | if (fud) { | 2255 | if (fud) { |
| 2254 | mutex_lock(&fuse_mutex); | 2256 | mutex_lock(&fuse_mutex); |
| 2255 | err = fuse_device_clone(fud->fc, file); | 2257 | res = fuse_device_clone(fud->fc, file); |
| 2256 | mutex_unlock(&fuse_mutex); | 2258 | mutex_unlock(&fuse_mutex); |
| 2257 | } | 2259 | } |
| 2258 | fput(old); | 2260 | fput(old); |
| 2259 | } | 2261 | } |
| 2260 | } | 2262 | } |
| 2263 | break; | ||
| 2264 | default: | ||
| 2265 | res = -ENOTTY; | ||
| 2266 | break; | ||
| 2261 | } | 2267 | } |
| 2262 | return err; | 2268 | return res; |
| 2263 | } | 2269 | } |
| 2264 | 2270 | ||
| 2265 | const struct file_operations fuse_dev_operations = { | 2271 | const struct file_operations fuse_dev_operations = { |
| @@ -863,6 +863,7 @@ static inline u64 fuse_get_attr_version(struct fuse_conn *fc) · 862 unmodified lines | |||
| 863 | 863 | ||
| 864 | static inline void fuse_make_bad(struct inode *inode) | 864 | static inline void fuse_make_bad(struct inode *inode) |
| 865 | { | 865 | { |
| 866 | remove_inode_hash(inode); | ||
| 866 | set_bit(FUSE_I_BAD, &get_fuse_inode(inode)->state); | 867 | set_bit(FUSE_I_BAD, &get_fuse_inode(inode)->state); |
| 867 | } | 868 | } |
| 868 | 869 | ||
| @@ -1324,8 +1324,15 @@ static int virtio_fs_fill_super(struct super_block *sb, struct fs_context *fsc) · 1323 unmodified lines | |||
| 1324 | 1324 | ||
| 1325 | /* virtiofs allocates and installs its own fuse devices */ | 1325 | /* virtiofs allocates and installs its own fuse devices */ |
| 1326 | ctx->fudptr = NULL; | 1326 | ctx->fudptr = NULL; |
| 1327 | if (ctx->dax) | 1327 | if (ctx->dax) { |
| 1328 | if (!fs->dax_dev) { | ||
| 1329 | err = -EINVAL; | ||
| 1330 | pr_err("virtio-fs: dax can't be enabled as filesystem" | ||
| 1331 | " device does not support it.\n"); | ||
| 1332 | goto err_free_fuse_devs; | ||
| 1333 | } | ||
| 1328 | ctx->dax_dev = fs->dax_dev; | 1334 | ctx->dax_dev = fs->dax_dev; |
| 1335 | } | ||
| 1329 | err = fuse_fill_super_common(sb, ctx); | 1336 | err = fuse_fill_super_common(sb, ctx); |
| 1330 | if (err < 0) | 1337 | if (err < 0) |
| 1331 | goto err_free_fuse_devs; | 1338 | goto err_free_fuse_devs; |
| @@ -998,12 +998,16 @@ static void trans_drain(struct gfs2_trans *tr) · 997 unmodified lines | |||
| 998 | while (!list_empty(head)) { | 998 | while (!list_empty(head)) { |
| 999 | bd = list_first_entry(head, struct gfs2_bufdata, bd_list); | 999 | bd = list_first_entry(head, struct gfs2_bufdata, bd_list); |
| 1000 | list_del_init(&bd->bd_list); | 1000 | list_del_init(&bd->bd_list); |
| 1001 | if (!list_empty(&bd->bd_ail_st_list)) | ||
| 1002 | gfs2_remove_from_ail(bd); | ||
| 1001 | kmem_cache_free(gfs2_bufdata_cachep, bd); | 1003 | kmem_cache_free(gfs2_bufdata_cachep, bd); |
| 1002 | } | 1004 | } |
| 1003 | head = &tr->tr_databuf; | 1005 | head = &tr->tr_databuf; |
| 1004 | while (!list_empty(head)) { | 1006 | while (!list_empty(head)) { |
| 1005 | bd = list_first_entry(head, struct gfs2_bufdata, bd_list); | 1007 | bd = list_first_entry(head, struct gfs2_bufdata, bd_list); |
| 1006 | list_del_init(&bd->bd_list); | 1008 | list_del_init(&bd->bd_list); |
| 1009 | if (!list_empty(&bd->bd_ail_st_list)) | ||
| 1010 | gfs2_remove_from_ail(bd); | ||
| 1007 | kmem_cache_free(gfs2_bufdata_cachep, bd); | 1011 | kmem_cache_free(gfs2_bufdata_cachep, bd); |
| 1008 | } | 1012 | } |
| 1009 | } | 1013 | } |
| @@ -1032,7 +1036,7 @@ repeat: · 22 unmodified lines | |||
| 1032 | * Do this check while holding the log_flush_lock to prevent new | 1036 | * Do this check while holding the log_flush_lock to prevent new |
| 1033 | * buffers from being added to the ail via gfs2_pin() | 1037 | * buffers from being added to the ail via gfs2_pin() |
| 1034 | */ | 1038 | */ |
| 1035 | if (gfs2_withdrawn(sdp)) | 1039 | if (gfs2_withdrawn(sdp) || !test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags)) |
| 1036 | goto out; | 1040 | goto out; |
| 1037 | 1041 | ||
| 1038 | /* Log might have been flushed while we waited for the flush lock */ | 1042 | /* Log might have been flushed while we waited for the flush lock */ |
| @@ -267,7 +267,7 @@ static struct bio *gfs2_log_alloc_bio(struct gfs2_sbd *sdp, u64 blkno, · 266 unmodified lines | |||
| 267 | bio_end_io_t *end_io) | 267 | bio_end_io_t *end_io) |
| 268 | { | 268 | { |
| 269 | struct super_block *sb = sdp->sd_vfs; | 269 | struct super_block *sb = sdp->sd_vfs; |
| 270 | struct bio *bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES); | 270 | struct bio *bio = bio_alloc(GFP_NOIO, BIO_MAX_VECS); |
| 271 | 271 | ||
| 272 | bio->bi_iter.bi_sector = blkno << sdp->sd_fsb2bb_shift; | 272 | bio->bi_iter.bi_sector = blkno << sdp->sd_fsb2bb_shift; |
| 273 | bio_set_dev(bio, sb->s_bdev); | 273 | bio_set_dev(bio, sb->s_bdev); |
| @@ -1539,9 +1539,7 @@ static int gfs2_reconfigure(struct fs_context *fc) · 1538 unmodified lines | |||
| 1539 | return -EINVAL; | 1539 | return -EINVAL; |
| 1540 | 1540 | ||
| 1541 | if (fc->sb_flags & SB_RDONLY) { | 1541 | if (fc->sb_flags & SB_RDONLY) { |
| 1542 | error = gfs2_make_fs_ro(sdp); | 1542 | gfs2_make_fs_ro(sdp); |
| 1543 | if (error) | ||
| 1544 | errorfc(fc, "unable to remount read-only"); | ||
| 1545 | } else { | 1543 | } else { |
| 1546 | error = gfs2_make_fs_rw(sdp); | 1544 | error = gfs2_make_fs_rw(sdp); |
| 1547 | if (error) | 1545 | if (error) |
| @@ -587,9 +587,8 @@ out: · 586 unmodified lines | |||
| 587 | * Returns: errno | 587 | * Returns: errno |
| 588 | */ | 588 | */ |
| 589 | 589 | ||
| 590 | int gfs2_make_fs_ro(struct gfs2_sbd *sdp) | 590 | void gfs2_make_fs_ro(struct gfs2_sbd *sdp) |
| 591 | { | 591 | { |
| 592 | int error = 0; | ||
| 593 | int log_write_allowed = test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags); | 592 | int log_write_allowed = test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags); |
| 594 | 593 | ||
| 595 | gfs2_flush_delete_work(sdp); | 594 | gfs2_flush_delete_work(sdp); |
| @@ -624,8 +623,6 @@ int gfs2_make_fs_ro(struct gfs2_sbd *sdp) · 28 unmodified lines | |||
| 624 | 623 | ||
| 625 | if (!log_write_allowed) | 624 | if (!log_write_allowed) |
| 626 | sdp->sd_vfs->s_flags |= SB_RDONLY; | 625 | sdp->sd_vfs->s_flags |= SB_RDONLY; |
| 627 | |||
| 628 | return error; | ||
| 629 | } | 626 | } |
| 630 | 627 | ||
| 631 | /** | 628 | /** |
| @@ -637,7 +634,6 @@ int gfs2_make_fs_ro(struct gfs2_sbd *sdp) · 5 unmodified lines | |||
| 637 | static void gfs2_put_super(struct super_block *sb) | 634 | static void gfs2_put_super(struct super_block *sb) |
| 638 | { | 635 | { |
| 639 | struct gfs2_sbd *sdp = sb->s_fs_info; | 636 | struct gfs2_sbd *sdp = sb->s_fs_info; |
| 640 | int error; | ||
| 641 | struct gfs2_jdesc *jd; | 637 | struct gfs2_jdesc *jd; |
| 642 | 638 | ||
| 643 | /* No more recovery requests */ | 639 | /* No more recovery requests */ |
| @@ -658,9 +654,7 @@ restart: · 14 unmodified lines | |||
| 658 | spin_unlock(&sdp->sd_jindex_spin); | 654 | spin_unlock(&sdp->sd_jindex_spin); |
| 659 | 655 | ||
| 660 | if (!sb_rdonly(sb)) { | 656 | if (!sb_rdonly(sb)) { |
| 661 | error = gfs2_make_fs_ro(sdp); | 657 | gfs2_make_fs_ro(sdp); |
| 662 | if (error) | ||
| 663 | gfs2_io_error(sdp); | ||
| 664 | } | 658 | } |
| 665 | WARN_ON(gfs2_withdrawing(sdp)); | 659 | WARN_ON(gfs2_withdrawing(sdp)); |
| 666 | 660 | ||
| @@ -34,7 +34,7 @@ extern int gfs2_lookup_in_master_dir(struct gfs2_sbd *sdp, char *filename, · 33 unmodified lines | |||
| 34 | struct gfs2_inode **ipp); | 34 | struct gfs2_inode **ipp); |
| 35 | 35 | ||
| 36 | extern int gfs2_make_fs_rw(struct gfs2_sbd *sdp); | 36 | extern int gfs2_make_fs_rw(struct gfs2_sbd *sdp); |
| 37 | extern int gfs2_make_fs_ro(struct gfs2_sbd *sdp); | 37 | extern void gfs2_make_fs_ro(struct gfs2_sbd *sdp); |
| 38 | extern void gfs2_online_uevent(struct gfs2_sbd *sdp); | 38 | extern void gfs2_online_uevent(struct gfs2_sbd *sdp); |
| 39 | extern int gfs2_statfs_init(struct gfs2_sbd *sdp); | 39 | extern int gfs2_statfs_init(struct gfs2_sbd *sdp); |
| 40 | extern void gfs2_statfs_change(struct gfs2_sbd *sdp, s64 total, s64 free, | 40 | extern void gfs2_statfs_change(struct gfs2_sbd *sdp, s64 total, s64 free, |
| @@ -169,6 +169,8 @@ static struct gfs2_bufdata *gfs2_alloc_bufdata(struct gfs2_glock *gl, · 168 unmodified lines | |||
| 169 | bd->bd_bh = bh; | 169 | bd->bd_bh = bh; |
| 170 | bd->bd_gl = gl; | 170 | bd->bd_gl = gl; |
| 171 | INIT_LIST_HEAD(&bd->bd_list); | 171 | INIT_LIST_HEAD(&bd->bd_list); |
| 172 | INIT_LIST_HEAD(&bd->bd_ail_st_list); | ||
| 173 | INIT_LIST_HEAD(&bd->bd_ail_gl_list); | ||
| 172 | bh->b_private = bd; | 174 | bh->b_private = bd; |
| 173 | return bd; | 175 | return bd; |
| 174 | } | 176 | } |
| @@ -119,17 +119,22 @@ void gfs2_freeze_unlock(struct gfs2_holder *freeze_gh) · 118 unmodified lines | |||
| 119 | static void signal_our_withdraw(struct gfs2_sbd *sdp) | 119 | static void signal_our_withdraw(struct gfs2_sbd *sdp) |
| 120 | { | 120 | { |
| 121 | struct gfs2_glock *live_gl = sdp->sd_live_gh.gh_gl; | 121 | struct gfs2_glock *live_gl = sdp->sd_live_gh.gh_gl; |
| 122 | struct inode *inode = sdp->sd_jdesc->jd_inode; | 122 | struct inode *inode; |
| 123 | struct gfs2_inode *ip = GFS2_I(inode); | 123 | struct gfs2_inode *ip; |
| 124 | struct gfs2_glock *i_gl = ip->i_gl; | 124 | struct gfs2_glock *i_gl; |
| 125 | u64 no_formal_ino = ip->i_no_formal_ino; | 125 | u64 no_formal_ino; |
| 126 | int log_write_allowed = test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags); | 126 | int log_write_allowed = test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags); |
| 127 | int ret = 0; | 127 | int ret = 0; |
| 128 | int tries; | 128 | int tries; |
| 129 | 129 | ||
| 130 | if (test_bit(SDF_NORECOVERY, &sdp->sd_flags)) | 130 | if (test_bit(SDF_NORECOVERY, &sdp->sd_flags) || !sdp->sd_jdesc) |
| 131 | return; | 131 | return; |
| 132 | 132 | ||
| 133 | inode = sdp->sd_jdesc->jd_inode; | ||
| 134 | ip = GFS2_I(inode); | ||
| 135 | i_gl = ip->i_gl; | ||
| 136 | no_formal_ino = ip->i_no_formal_ino; | ||
| 137 | |||
| 133 | /* Prevent any glock dq until withdraw recovery is complete */ | 138 | /* Prevent any glock dq until withdraw recovery is complete */ |
| 134 | set_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags); | 139 | set_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags); |
| 135 | /* | 140 | /* |
| @@ -156,7 +161,7 @@ static void signal_our_withdraw(struct gfs2_sbd *sdp) · 20 unmodified lines | |||
| 156 | ret = 0; | 161 | ret = 0; |
| 157 | } | 162 | } |
| 158 | if (!ret) | 163 | if (!ret) |
| 159 | ret = gfs2_make_fs_ro(sdp); | 164 | gfs2_make_fs_ro(sdp); |
| 160 | gfs2_freeze_unlock(&freeze_gh); | 165 | gfs2_freeze_unlock(&freeze_gh); |
| 161 | } | 166 | } |
| 162 | 167 | ||
| @@ -110,7 +110,6 @@ struct io_wq { · 109 unmodified lines | |||
| 110 | io_wq_work_fn *do_work; | 110 | io_wq_work_fn *do_work; |
| 111 | 111 | ||
| 112 | struct task_struct *manager; | 112 | struct task_struct *manager; |
| 113 | struct user_struct *user; | ||
| 114 | 113 | ||
| 115 | struct io_wq_hash *hash; | 114 | struct io_wq_hash *hash; |
| 116 | 115 | ||
| @@ -592,7 +591,7 @@ static bool create_io_worker(struct io_wq *wq, struct io_wqe *wqe, int index) · 475 unmodified lines | |||
| 592 | tsk->pf_io_worker = worker; | 591 | tsk->pf_io_worker = worker; |
| 593 | worker->task = tsk; | 592 | worker->task = tsk; |
| 594 | set_cpus_allowed_ptr(tsk, cpumask_of_node(wqe->node)); | 593 | set_cpus_allowed_ptr(tsk, cpumask_of_node(wqe->node)); |
| 595 | tsk->flags |= PF_NOFREEZE | PF_NO_SETAFFINITY; | 594 | tsk->flags |= PF_NO_SETAFFINITY; |
| 596 | 595 | ||
| 597 | raw_spin_lock_irq(&wqe->lock); | 596 | raw_spin_lock_irq(&wqe->lock); |
| 598 | hlist_nulls_add_head_rcu(&worker->nulls_node, &wqe->free_list); | 597 | hlist_nulls_add_head_rcu(&worker->nulls_node, &wqe->free_list); |
| @@ -710,7 +709,6 @@ static int io_wq_manager(void *data) · 111 unmodified lines | |||
| 710 | set_current_state(TASK_INTERRUPTIBLE); | 709 | set_current_state(TASK_INTERRUPTIBLE); |
| 711 | io_wq_check_workers(wq); | 710 | io_wq_check_workers(wq); |
| 712 | schedule_timeout(HZ); | 711 | schedule_timeout(HZ); |
| 713 | try_to_freeze(); | ||
| 714 | if (fatal_signal_pending(current)) | 712 | if (fatal_signal_pending(current)) |
| 715 | set_bit(IO_WQ_BIT_EXIT, &wq->state); | 713 | set_bit(IO_WQ_BIT_EXIT, &wq->state); |
| 716 | } while (!test_bit(IO_WQ_BIT_EXIT, &wq->state)); | 714 | } while (!test_bit(IO_WQ_BIT_EXIT, &wq->state)); |
| @@ -722,9 +720,9 @@ static int io_wq_manager(void *data) · 5 unmodified lines | |||
| 722 | io_wq_for_each_worker(wq->wqes[node], io_wq_worker_wake, NULL); | 720 | io_wq_for_each_worker(wq->wqes[node], io_wq_worker_wake, NULL); |
| 723 | rcu_read_unlock(); | 721 | rcu_read_unlock(); |
| 724 | 722 | ||
| 725 | /* we might not ever have created any workers */ | 723 | if (atomic_dec_and_test(&wq->worker_refs)) |
| 726 | if (atomic_read(&wq->worker_refs)) | 724 | complete(&wq->worker_done); |
| 727 | wait_for_completion(&wq->worker_done); | 725 | wait_for_completion(&wq->worker_done); |
| 728 | 726 | ||
| 729 | spin_lock_irq(&wq->hash->wait.lock); | 727 | spin_lock_irq(&wq->hash->wait.lock); |
| 730 | for_each_node(node) | 728 | for_each_node(node) |
| @@ -774,7 +772,10 @@ static int io_wq_fork_manager(struct io_wq *wq) · 43 unmodified lines | |||
| 774 | if (wq->manager) | 772 | if (wq->manager) |
| 775 | return 0; | 773 | return 0; |
| 776 | 774 | ||
| 777 | reinit_completion(&wq->worker_done); | 775 | WARN_ON_ONCE(test_bit(IO_WQ_BIT_EXIT, &wq->state)); |
| 776 | |||
| 777 | init_completion(&wq->worker_done); | ||
| 778 | atomic_set(&wq->worker_refs, 1); | ||
| 778 | tsk = create_io_thread(io_wq_manager, wq, NUMA_NO_NODE); | 779 | tsk = create_io_thread(io_wq_manager, wq, NUMA_NO_NODE); |
| 779 | if (!IS_ERR(tsk)) { | 780 | if (!IS_ERR(tsk)) { |
| 780 | wq->manager = get_task_struct(tsk); | 781 | wq->manager = get_task_struct(tsk); |
| @@ -782,6 +783,9 @@ static int io_wq_fork_manager(struct io_wq *wq) · 1 unmodified line | |||
| 782 | return 0; | 783 | return 0; |
| 783 | } | 784 | } |
| 784 | 785 | ||
| 786 | if (atomic_dec_and_test(&wq->worker_refs)) | ||
| 787 | complete(&wq->worker_done); | ||
| 788 | |||
| 785 | return PTR_ERR(tsk); | 789 | return PTR_ERR(tsk); |
| 786 | } | 790 | } |
| 787 | 791 | ||
| @@ -794,8 +798,7 @@ static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) · 6 unmodified lines | |||
| 794 | /* Can only happen if manager creation fails after exec */ | 798 | /* Can only happen if manager creation fails after exec */ |
| 795 | if (io_wq_fork_manager(wqe->wq) || | 799 | if (io_wq_fork_manager(wqe->wq) || |
| 796 | test_bit(IO_WQ_BIT_EXIT, &wqe->wq->state)) { | 800 | test_bit(IO_WQ_BIT_EXIT, &wqe->wq->state)) { |
| 797 | work->flags |= IO_WQ_WORK_CANCEL; | 801 | io_run_cancel(work, wqe); |
| 798 | wqe->wq->do_work(work); | ||
| 799 | return; | 802 | return; |
| 800 | } | 803 | } |
| 801 | 804 | ||
| @@ -1018,13 +1021,9 @@ struct io_wq *io_wq_create(unsigned bounded, struct io_wq_data *data) · 216 unmodified lines | |||
| 1018 | init_completion(&wq->exited); | 1021 | init_completion(&wq->exited); |
| 1019 | refcount_set(&wq->refs, 1); | 1022 | refcount_set(&wq->refs, 1); |
| 1020 | 1023 | ||
| 1021 | init_completion(&wq->worker_done); | ||
| 1022 | atomic_set(&wq->worker_refs, 0); | ||
| 1023 | |||
| 1024 | ret = io_wq_fork_manager(wq); | 1024 | ret = io_wq_fork_manager(wq); |
| 1025 | if (!ret) | 1025 | if (!ret) |
| 1026 | return wq; | 1026 | return wq; |
| 1027 | |||
| 1028 | err: | 1027 | err: |
| 1029 | io_wq_put_hash(data->hash); | 1028 | io_wq_put_hash(data->hash); |
| 1030 | cpuhp_state_remove_instance_nocalls(io_wq_online, &wq->cpuhp_node); | 1029 | cpuhp_state_remove_instance_nocalls(io_wq_online, &wq->cpuhp_node); |
| @@ -79,8 +79,8 @@ static inline void wq_list_del(struct io_wq_work_list *list, · 78 unmodified lines | |||
| 79 | 79 | ||
| 80 | struct io_wq_work { | 80 | struct io_wq_work { |
| 81 | struct io_wq_work_node list; | 81 | struct io_wq_work_node list; |
| 82 | const struct cred *creds; | ||
| 82 | unsigned flags; | 83 | unsigned flags; |
| 83 | unsigned short personality; | ||
| 84 | }; | 84 | }; |
| 85 | 85 | ||
| 86 | static inline struct io_wq_work *wq_next_work(struct io_wq_work *work) | 86 | static inline struct io_wq_work *wq_next_work(struct io_wq_work *work) |
| @@ -258,12 +258,10 @@ enum { · 257 unmodified lines | |||
| 258 | 258 | ||
| 259 | struct io_sq_data { | 259 | struct io_sq_data { |
| 260 | refcount_t refs; | 260 | refcount_t refs; |
| 261 | struct mutex lock; | 261 | struct rw_semaphore rw_lock; |
| 262 | 262 | ||
| 263 | /* ctx's that are using this sqd */ | 263 | /* ctx's that are using this sqd */ |
| 264 | struct list_head ctx_list; | 264 | struct list_head ctx_list; |
| 265 | struct list_head ctx_new_list; | ||
| 266 | struct mutex ctx_lock; | ||
| 267 | 265 | ||
| 268 | struct task_struct *thread; | 266 | struct task_struct *thread; |
| 269 | struct wait_queue_head wait; | 267 | struct wait_queue_head wait; |
| @@ -271,10 +269,9 @@ struct io_sq_data { · 1 unmodified line | |||
| 271 | unsigned sq_thread_idle; | 269 | unsigned sq_thread_idle; |
| 272 | int sq_cpu; | 270 | int sq_cpu; |
| 273 | pid_t task_pid; | 271 | pid_t task_pid; |
| 272 | pid_t task_tgid; | ||
| 274 | 273 | ||
| 275 | unsigned long state; | 274 | unsigned long state; |
| 276 | struct completion startup; | ||
| 277 | struct completion parked; | ||
| 278 | struct completion exited; | 275 | struct completion exited; |
| 279 | }; | 276 | }; |
| 280 | 277 | ||
| @@ -336,7 +333,6 @@ struct io_ring_ctx { · 55 unmodified lines | |||
| 336 | unsigned int drain_next: 1; | 333 | unsigned int drain_next: 1; |
| 337 | unsigned int eventfd_async: 1; | 334 | unsigned int eventfd_async: 1; |
| 338 | unsigned int restricted: 1; | 335 | unsigned int restricted: 1; |
| 339 | unsigned int sqo_exec: 1; | ||
| 340 | 336 | ||
| 341 | /* | 337 | /* |
| 342 | * Ring buffer of indices into array of io_uring_sqe, which is | 338 | * Ring buffer of indices into array of io_uring_sqe, which is |
| @@ -380,6 +376,7 @@ struct io_ring_ctx { · 37 unmodified lines | |||
| 380 | /* Only used for accounting purposes */ | 376 | /* Only used for accounting purposes */ |
| 381 | struct mm_struct *mm_account; | 377 | struct mm_struct *mm_account; |
| 382 | 378 | ||
| 379 | const struct cred *sq_creds; /* cred used for __io_sq_thread() */ | ||
| 383 | struct io_sq_data *sq_data; /* if using sq thread polling */ | 380 | struct io_sq_data *sq_data; /* if using sq thread polling */ |
| 384 | 381 | ||
| 385 | struct wait_queue_head sqo_sq_wait; | 382 | struct wait_queue_head sqo_sq_wait; |
| @@ -400,7 +397,6 @@ struct io_ring_ctx { · 14 unmodified lines | |||
| 400 | struct user_struct *user; | 397 | struct user_struct *user; |
| 401 | 398 | ||
| 402 | struct completion ref_comp; | 399 | struct completion ref_comp; |
| 403 | struct completion sq_thread_comp; | ||
| 404 | 400 | ||
| 405 | #if defined(CONFIG_UNIX) | 401 | #if defined(CONFIG_UNIX) |
| 406 | struct socket *ring_sock; | 402 | struct socket *ring_sock; |
| @@ -408,7 +404,8 @@ struct io_ring_ctx { · 1 unmodified line | |||
| 408 | 404 | ||
| 409 | struct idr io_buffer_idr; | 405 | struct idr io_buffer_idr; |
| 410 | 406 | ||
| 411 | struct idr personality_idr; | 407 | struct xarray personalities; |
| 408 | u32 pers_next; | ||
| 412 | 409 | ||
| 413 | struct { | 410 | struct { |
| 414 | unsigned cached_cq_tail; | 411 | unsigned cached_cq_tail; |
| @@ -454,6 +451,7 @@ struct io_ring_ctx { · 39 unmodified lines | |||
| 454 | 451 | ||
| 455 | /* Keep this last, we don't need it for the fast path */ | 452 | /* Keep this last, we don't need it for the fast path */ |
| 456 | struct work_struct exit_work; | 453 | struct work_struct exit_work; |
| 454 | struct list_head tctx_list; | ||
| 457 | }; | 455 | }; |
| 458 | 456 | ||
| 459 | /* | 457 | /* |
| @@ -805,6 +803,12 @@ struct io_kiocb { · 345 unmodified lines | |||
| 805 | struct io_wq_work work; | 803 | struct io_wq_work work; |
| 806 | }; | 804 | }; |
| 807 | 805 | ||
| 806 | struct io_tctx_node { | ||
| 807 | struct list_head ctx_node; | ||
| 808 | struct task_struct *task; | ||
| 809 | struct io_ring_ctx *ctx; | ||
| 810 | }; | ||
| 811 | |||
| 808 | struct io_defer_entry { | 812 | struct io_defer_entry { |
| 809 | struct list_head list; | 813 | struct list_head list; |
| 810 | struct io_kiocb *req; | 814 | struct io_kiocb *req; |
| @@ -979,6 +983,8 @@ static const struct io_op_def io_op_defs[] = { · 168 unmodified lines | |||
| 979 | [IORING_OP_UNLINKAT] = {}, | 983 | [IORING_OP_UNLINKAT] = {}, |
| 980 | }; | 984 | }; |
| 981 | 985 | ||
| 986 | static bool io_disarm_next(struct io_kiocb *req); | ||
| 987 | static void io_uring_del_task_file(unsigned long index); | ||
| 982 | static void io_uring_try_cancel_requests(struct io_ring_ctx *ctx, | 988 | static void io_uring_try_cancel_requests(struct io_ring_ctx *ctx, |
| 983 | struct task_struct *task, | 989 | struct task_struct *task, |
| 984 | struct files_struct *files); | 990 | struct files_struct *files); |
| @@ -1129,9 +1135,8 @@ static struct io_ring_ctx *io_ring_ctx_alloc(struct io_uring_params *p) · 144 unmodified lines | |||
| 1129 | init_waitqueue_head(&ctx->cq_wait); | 1135 | init_waitqueue_head(&ctx->cq_wait); |
| 1130 | INIT_LIST_HEAD(&ctx->cq_overflow_list); | 1136 | INIT_LIST_HEAD(&ctx->cq_overflow_list); |
| 1131 | init_completion(&ctx->ref_comp); | 1137 | init_completion(&ctx->ref_comp); |
| 1132 | init_completion(&ctx->sq_thread_comp); | ||
| 1133 | idr_init(&ctx->io_buffer_idr); | 1138 | idr_init(&ctx->io_buffer_idr); |
| 1134 | idr_init(&ctx->personality_idr); | 1139 | xa_init_flags(&ctx->personalities, XA_FLAGS_ALLOC1); |
| 1135 | mutex_init(&ctx->uring_lock); | 1140 | mutex_init(&ctx->uring_lock); |
| 1136 | init_waitqueue_head(&ctx->wait); | 1141 | init_waitqueue_head(&ctx->wait); |
| 1137 | spin_lock_init(&ctx->completion_lock); | 1142 | spin_lock_init(&ctx->completion_lock); |
| @@ -1144,6 +1149,7 @@ static struct io_ring_ctx *io_ring_ctx_alloc(struct io_uring_params *p) · 6 unmodified lines | |||
| 1144 | INIT_LIST_HEAD(&ctx->rsrc_ref_list); | 1149 | INIT_LIST_HEAD(&ctx->rsrc_ref_list); |
| 1145 | INIT_DELAYED_WORK(&ctx->rsrc_put_work, io_rsrc_put_work); | 1150 | INIT_DELAYED_WORK(&ctx->rsrc_put_work, io_rsrc_put_work); |
| 1146 | init_llist_head(&ctx->rsrc_put_llist); | 1151 | init_llist_head(&ctx->rsrc_put_llist); |
| 1152 | INIT_LIST_HEAD(&ctx->tctx_list); | ||
| 1147 | INIT_LIST_HEAD(&ctx->submit_state.comp.free_list); | 1153 | INIT_LIST_HEAD(&ctx->submit_state.comp.free_list); |
| 1148 | INIT_LIST_HEAD(&ctx->submit_state.comp.locked_free_list); | 1154 | INIT_LIST_HEAD(&ctx->submit_state.comp.locked_free_list); |
| 1149 | return ctx; | 1155 | return ctx; |
| @@ -1183,6 +1189,9 @@ static void io_prep_async_work(struct io_kiocb *req) · 33 unmodified lines | |||
| 1183 | const struct io_op_def *def = &io_op_defs[req->opcode]; | 1189 | const struct io_op_def *def = &io_op_defs[req->opcode]; |
| 1184 | struct io_ring_ctx *ctx = req->ctx; | 1190 | struct io_ring_ctx *ctx = req->ctx; |
| 1185 | 1191 | ||
| 1192 | if (!req->work.creds) | ||
| 1193 | req->work.creds = get_current_cred(); | ||
| 1194 | |||
| 1186 | if (req->flags & REQ_F_FORCE_ASYNC) | 1195 | if (req->flags & REQ_F_FORCE_ASYNC) |
| 1187 | req->work.flags |= IO_WQ_WORK_CONCURRENT; | 1196 | req->work.flags |= IO_WQ_WORK_CONCURRENT; |
| 1188 | 1197 | ||
| @@ -1514,15 +1523,14 @@ static void io_cqring_fill_event(struct io_kiocb *req, long res) · 325 unmodified lines | |||
| 1514 | __io_cqring_fill_event(req, res, 0); | 1523 | __io_cqring_fill_event(req, res, 0); |
| 1515 | } | 1524 | } |
| 1516 | 1525 | ||
| 1517 | static inline void io_req_complete_post(struct io_kiocb *req, long res, | 1526 | static void io_req_complete_post(struct io_kiocb *req, long res, |
| 1518 | unsigned int cflags) | 1527 | unsigned int cflags) |
| 1519 | { | 1528 | { |
| 1520 | struct io_ring_ctx *ctx = req->ctx; | 1529 | struct io_ring_ctx *ctx = req->ctx; |
| 1521 | unsigned long flags; | 1530 | unsigned long flags; |
| 1522 | 1531 | ||
| 1523 | spin_lock_irqsave(&ctx->completion_lock, flags); | 1532 | spin_lock_irqsave(&ctx->completion_lock, flags); |
| 1524 | __io_cqring_fill_event(req, res, cflags); | 1533 | __io_cqring_fill_event(req, res, cflags); |
| 1525 | io_commit_cqring(ctx); | ||
| 1526 | /* | 1534 | /* |
| 1527 | * If we're the last reference to this request, add to our locked | 1535 | * If we're the last reference to this request, add to our locked |
| 1528 | * free_list cache. | 1536 | * free_list cache. |
| @@ -1530,19 +1538,26 @@ static inline void io_req_complete_post(struct io_kiocb *req, long res, · 1 unmodified line | |||
| 1530 | if (refcount_dec_and_test(&req->refs)) { | 1538 | if (refcount_dec_and_test(&req->refs)) { |
| 1531 | struct io_comp_state *cs = &ctx->submit_state.comp; | 1539 | struct io_comp_state *cs = &ctx->submit_state.comp; |
| 1532 | 1540 | ||
| 1541 | if (req->flags & (REQ_F_LINK | REQ_F_HARDLINK)) { | ||
| 1542 | if (req->flags & (REQ_F_LINK_TIMEOUT | REQ_F_FAIL_LINK)) | ||
| 1543 | io_disarm_next(req); | ||
| 1544 | if (req->link) { | ||
| 1545 | io_req_task_queue(req->link); | ||
| 1546 | req->link = NULL; | ||
| 1547 | } | ||
| 1548 | } | ||
| 1533 | io_dismantle_req(req); | 1549 | io_dismantle_req(req); |
| 1534 | io_put_task(req->task, 1); | 1550 | io_put_task(req->task, 1); |
| 1535 | list_add(&req->compl.list, &cs->locked_free_list); | 1551 | list_add(&req->compl.list, &cs->locked_free_list); |
| 1536 | cs->locked_free_nr++; | 1552 | cs->locked_free_nr++; |
| 1537 | } else | 1553 | } else |
| 1538 | req = NULL; | 1554 | req = NULL; |
| 1555 | io_commit_cqring(ctx); | ||
| 1539 | spin_unlock_irqrestore(&ctx->completion_lock, flags); | 1556 | spin_unlock_irqrestore(&ctx->completion_lock, flags); |
| 1540 | |||
| 1541 | io_cqring_ev_posted(ctx); | 1557 | io_cqring_ev_posted(ctx); |
| 1542 | if (req) { | 1558 | |
| 1543 | io_queue_next(req); | 1559 | if (req) |
| 1544 | percpu_ref_put(&ctx->refs); | 1560 | percpu_ref_put(&ctx->refs); |
| 1545 | } | ||
| 1546 | } | 1561 | } |
| 1547 | 1562 | ||
| 1548 | static void io_req_complete_state(struct io_kiocb *req, long res, | 1563 | static void io_req_complete_state(struct io_kiocb *req, long res, |
| @@ -1648,6 +1663,10 @@ static void io_dismantle_req(struct io_kiocb *req) · 99 unmodified lines | |||
| 1648 | io_put_file(req, req->file, (req->flags & REQ_F_FIXED_FILE)); | 1663 | io_put_file(req, req->file, (req->flags & REQ_F_FIXED_FILE)); |
| 1649 | if (req->fixed_rsrc_refs) | 1664 | if (req->fixed_rsrc_refs) |
| 1650 | percpu_ref_put(req->fixed_rsrc_refs); | 1665 | percpu_ref_put(req->fixed_rsrc_refs); |
| 1666 | if (req->work.creds) { | ||
| 1667 | put_cred(req->work.creds); | ||
| 1668 | req->work.creds = NULL; | ||
| 1669 | } | ||
| 1651 | 1670 | ||
| 1652 | if (req->flags & REQ_F_INFLIGHT) { | 1671 | if (req->flags & REQ_F_INFLIGHT) { |
| 1653 | struct io_ring_ctx *ctx = req->ctx; | 1672 | struct io_ring_ctx *ctx = req->ctx; |
| @@ -1690,15 +1709,11 @@ static inline void io_remove_next_linked(struct io_kiocb *req) · 36 unmodified lines | |||
| 1690 | nxt->link = NULL; | 1709 | nxt->link = NULL; |
| 1691 | } | 1710 | } |
| 1692 | 1711 | ||
| 1693 | static void io_kill_linked_timeout(struct io_kiocb *req) | 1712 | static bool io_kill_linked_timeout(struct io_kiocb *req) |
| 1713 | __must_hold(&req->ctx->completion_lock) | ||
| 1694 | { | 1714 | { |
| 1695 | struct io_ring_ctx *ctx = req->ctx; | 1715 | struct io_kiocb *link = req->link; |
| 1696 | struct io_kiocb *link; | ||
| 1697 | bool cancelled = false; | 1716 | bool cancelled = false; |
| 1698 | unsigned long flags; | ||
| 1699 | |||
| 1700 | spin_lock_irqsave(&ctx->completion_lock, flags); | ||
| 1701 | link = req->link; | ||
| 1702 | 1717 | ||
| 1703 | /* | 1718 | /* |
| 1704 | * Can happen if a linked timeout fired and link had been like | 1719 | * Can happen if a linked timeout fired and link had been like |
| @@ -1713,50 +1728,48 @@ static void io_kill_linked_timeout(struct io_kiocb *req) · 8 unmodified lines | |||
| 1713 | ret = hrtimer_try_to_cancel(&io->timer); | 1728 | ret = hrtimer_try_to_cancel(&io->timer); |
| 1714 | if (ret != -1) { | 1729 | if (ret != -1) { |
| 1715 | io_cqring_fill_event(link, -ECANCELED); | 1730 | io_cqring_fill_event(link, -ECANCELED); |
| 1716 | io_commit_cqring(ctx); | 1731 | io_put_req_deferred(link, 1); |
| 1717 | cancelled = true; | 1732 | cancelled = true; |
| 1718 | } | 1733 | } |
| 1719 | } | 1734 | } |
| 1720 | req->flags &= ~REQ_F_LINK_TIMEOUT; | 1735 | req->flags &= ~REQ_F_LINK_TIMEOUT; |
| 1721 | spin_unlock_irqrestore(&ctx->completion_lock, flags); | 1736 | return cancelled; |
| 1722 | |||
| 1723 | if (cancelled) { | ||
| 1724 | io_cqring_ev_posted(ctx); | ||
| 1725 | io_put_req(link); | ||
| 1726 | } | ||
| 1727 | } | 1737 | } |
| 1728 | 1738 | ||
| 1729 | |||
| 1730 | static void io_fail_links(struct io_kiocb *req) | 1739 | static void io_fail_links(struct io_kiocb *req) |
| 1740 | __must_hold(&req->ctx->completion_lock) | ||
| 1731 | { | 1741 | { |
| 1732 | struct io_kiocb *link, *nxt; | 1742 | struct io_kiocb *nxt, *link = req->link; |
| 1733 | struct io_ring_ctx *ctx = req->ctx; | ||
| 1734 | unsigned long flags; | ||
| 1735 | 1743 | ||
| 1736 | spin_lock_irqsave(&ctx->completion_lock, flags); | ||
| 1737 | link = req->link; | ||
| 1738 | req->link = NULL; | 1744 | req->link = NULL; |
| 1739 | |||
| 1740 | while (link) { | 1745 | while (link) { |
| 1741 | nxt = link->link; | 1746 | nxt = link->link; |
| 1742 | link->link = NULL; | 1747 | link->link = NULL; |
| 1743 | 1748 | ||
| 1744 | trace_io_uring_fail_link(req, link); | 1749 | trace_io_uring_fail_link(req, link); |
| 1745 | io_cqring_fill_event(link, -ECANCELED); | 1750 | io_cqring_fill_event(link, -ECANCELED); |
| 1746 | |||
| 1747 | io_put_req_deferred(link, 2); | 1751 | io_put_req_deferred(link, 2); |
| 1748 | link = nxt; | 1752 | link = nxt; |
| 1749 | } | 1753 | } |
| 1750 | io_commit_cqring(ctx); | 1754 | } |
| 1751 | spin_unlock_irqrestore(&ctx->completion_lock, flags); | ||
| 1752 | 1755 | ||
| 1753 | io_cqring_ev_posted(ctx); | 1756 | static bool io_disarm_next(struct io_kiocb *req) |
| 1757 | __must_hold(&req->ctx->completion_lock) | ||
| 1758 | { | ||
| 1759 | bool posted = false; | ||
| 1760 | |||
| 1761 | if (likely(req->flags & REQ_F_LINK_TIMEOUT)) | ||
| 1762 | posted = io_kill_linked_timeout(req); | ||
| 1763 | if (unlikely(req->flags & REQ_F_FAIL_LINK)) { | ||
| 1764 | posted |= (req->link != NULL); | ||
| 1765 | io_fail_links(req); | ||
| 1766 | } | ||
| 1767 | return posted; | ||
| 1754 | } | 1768 | } |
| 1755 | 1769 | ||
| 1756 | static struct io_kiocb *__io_req_find_next(struct io_kiocb *req) | 1770 | static struct io_kiocb *__io_req_find_next(struct io_kiocb *req) |
| 1757 | { | 1771 | { |
| 1758 | if (req->flags & REQ_F_LINK_TIMEOUT) | 1772 | struct io_kiocb *nxt; |
| 1759 | io_kill_linked_timeout(req); | ||
| 1760 | 1773 | ||
| 1761 | /* | 1774 | /* |
| 1762 | * If LINK is set, we have dependent requests in this chain. If we | 1775 | * If LINK is set, we have dependent requests in this chain. If we |
| @@ -1764,14 +1777,22 @@ static struct io_kiocb *__io_req_find_next(struct io_kiocb *req) · 1 unmodified line | |||
| 1764 | * dependencies to the next request. In case of failure, fail the rest | 1777 | * dependencies to the next request. In case of failure, fail the rest |
| 1765 | * of the chain. | 1778 | * of the chain. |
| 1766 | */ | 1779 | */ |
| 1767 | if (likely(!(req->flags & REQ_F_FAIL_LINK))) { | 1780 | if (req->flags & (REQ_F_LINK_TIMEOUT | REQ_F_FAIL_LINK)) { |
| 1768 | struct io_kiocb *nxt = req->link; | 1781 | struct io_ring_ctx *ctx = req->ctx; |
| 1782 | unsigned long flags; | ||
| 1783 | bool posted; | ||
| 1769 | 1784 | ||
| 1770 | req->link = NULL; | 1785 | spin_lock_irqsave(&ctx->completion_lock, flags); |
| 1771 | return nxt; | 1786 | posted = io_disarm_next(req); |
| 1787 | if (posted) | ||
| 1788 | io_commit_cqring(req->ctx); | ||
| 1789 | spin_unlock_irqrestore(&ctx->completion_lock, flags); | ||
| 1790 | if (posted) | ||
| 1791 | io_cqring_ev_posted(ctx); | ||
| 1772 | } | 1792 | } |
| 1773 | io_fail_links(req); | 1793 | nxt = req->link; |
| 1774 | return NULL; | 1794 | req->link = NULL; |
| 1795 | return nxt; | ||
| 1775 | } | 1796 | } |
| 1776 | 1797 | ||
| 1777 | static inline struct io_kiocb *io_req_find_next(struct io_kiocb *req) | 1798 | static inline struct io_kiocb *io_req_find_next(struct io_kiocb *req) |
| @@ -5559,22 +5580,30 @@ add: · 3781 unmodified lines | |||
| 5559 | return 0; | 5580 | return 0; |
| 5560 | } | 5581 | } |
| 5561 | 5582 | ||
| 5583 | struct io_cancel_data { | ||
| 5584 | struct io_ring_ctx *ctx; | ||
| 5585 | u64 user_data; | ||
| 5586 | }; | ||
| 5587 | |||
| 5562 | static bool io_cancel_cb(struct io_wq_work *work, void *data) | 5588 | static bool io_cancel_cb(struct io_wq_work *work, void *data) |
| 5563 | { | 5589 | { |
| 5564 | struct io_kiocb *req = container_of(work, struct io_kiocb, work); | 5590 | struct io_kiocb *req = container_of(work, struct io_kiocb, work); |
| 5591 | struct io_cancel_data *cd = data; | ||
| 5565 | 5592 | ||
| 5566 | return req->user_data == (unsigned long) data; | 5593 | return req->ctx == cd->ctx && req->user_data == cd->user_data; |
| 5567 | } | 5594 | } |
| 5568 | 5595 | ||
| 5569 | static int io_async_cancel_one(struct io_uring_task *tctx, void *sqe_addr) | 5596 | static int io_async_cancel_one(struct io_uring_task *tctx, u64 user_data, |
| 5597 | struct io_ring_ctx *ctx) | ||
| 5570 | { | 5598 | { |
| 5599 | struct io_cancel_data data = { .ctx = ctx, .user_data = user_data, }; | ||
| 5571 | enum io_wq_cancel cancel_ret; | 5600 | enum io_wq_cancel cancel_ret; |
| 5572 | int ret = 0; | 5601 | int ret = 0; |
| 5573 | 5602 | ||
| 5574 | if (!tctx->io_wq) | 5603 | if (!tctx || !tctx->io_wq) |
| 5575 | return -ENOENT; | 5604 | return -ENOENT; |
| 5576 | 5605 | ||
| 5577 | cancel_ret = io_wq_cancel_cb(tctx->io_wq, io_cancel_cb, sqe_addr, false); | 5606 | cancel_ret = io_wq_cancel_cb(tctx->io_wq, io_cancel_cb, &data, false); |
| 5578 | switch (cancel_ret) { | 5607 | switch (cancel_ret) { |
| 5579 | case IO_WQ_CANCEL_OK: | 5608 | case IO_WQ_CANCEL_OK: |
| 5580 | ret = 0; | 5609 | ret = 0; |
| @@ -5597,8 +5626,7 @@ static void io_async_find_and_cancel(struct io_ring_ctx *ctx, · 16 unmodified lines | |||
| 5597 | unsigned long flags; | 5626 | unsigned long flags; |
| 5598 | int ret; | 5627 | int ret; |
| 5599 | 5628 | ||
| 5600 | ret = io_async_cancel_one(req->task->io_uring, | 5629 | ret = io_async_cancel_one(req->task->io_uring, sqe_addr, ctx); |
| 5601 | (void *) (unsigned long) sqe_addr); | ||
| 5602 | if (ret != -ENOENT) { | 5630 | if (ret != -ENOENT) { |
| 5603 | spin_lock_irqsave(&ctx->completion_lock, flags); | 5631 | spin_lock_irqsave(&ctx->completion_lock, flags); |
| 5604 | goto done; | 5632 | goto done; |
| @@ -5639,8 +5667,47 @@ static int io_async_cancel_prep(struct io_kiocb *req, · 34 unmodified lines | |||
| 5639 | static int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags) | 5667 | static int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags) |
| 5640 | { | 5668 | { |
| 5641 | struct io_ring_ctx *ctx = req->ctx; | 5669 | struct io_ring_ctx *ctx = req->ctx; |
| 5670 | u64 sqe_addr = req->cancel.addr; | ||
| 5671 | struct io_tctx_node *node; | ||
| 5672 | int ret; | ||
| 5673 | |||
| 5674 | /* tasks should wait for their io-wq threads, so safe w/o sync */ | ||
| 5675 | ret = io_async_cancel_one(req->task->io_uring, sqe_addr, ctx); | ||
| 5676 | spin_lock_irq(&ctx->completion_lock); | ||
| 5677 | if (ret != -ENOENT) | ||
| 5678 | goto done; | ||
| 5679 | ret = io_timeout_cancel(ctx, sqe_addr); | ||
| 5680 | if (ret != -ENOENT) | ||
| 5681 | goto done; | ||
| 5682 | ret = io_poll_cancel(ctx, sqe_addr); | ||
| 5683 | if (ret != -ENOENT) | ||
| 5684 | goto done; | ||
| 5685 | spin_unlock_irq(&ctx->completion_lock); | ||
| 5686 | |||
| 5687 | /* slow path, try all io-wq's */ | ||
| 5688 | io_ring_submit_lock(ctx, !(issue_flags & IO_URING_F_NONBLOCK)); | ||
| 5689 | ret = -ENOENT; | ||
| 5690 | list_for_each_entry(node, &ctx->tctx_list, ctx_node) { | ||
| 5691 | struct io_uring_task *tctx = node->task->io_uring; | ||
| 5692 | |||
| 5693 | if (!tctx || !tctx->io_wq) | ||
| 5694 | continue; | ||
| 5695 | ret = io_async_cancel_one(tctx, req->cancel.addr, ctx); | ||
| 5696 | if (ret != -ENOENT) | ||
| 5697 | break; | ||
| 5698 | } | ||
| 5699 | io_ring_submit_unlock(ctx, !(issue_flags & IO_URING_F_NONBLOCK)); | ||
| 5642 | 5700 | ||
| 5643 | io_async_find_and_cancel(ctx, req, req->cancel.addr, 0); | 5701 | spin_lock_irq(&ctx->completion_lock); |
| 5702 | done: | ||
| 5703 | io_cqring_fill_event(req, ret); | ||
| 5704 | io_commit_cqring(ctx); | ||
| 5705 | spin_unlock_irq(&ctx->completion_lock); | ||
| 5706 | io_cqring_ev_posted(ctx); | ||
| 5707 | |||
| 5708 | if (ret < 0) | ||
| 5709 | req_set_fail_links(req); | ||
| 5710 | io_put_req(req); | ||
| 5644 | return 0; | 5711 | return 0; |
| 5645 | } | 5712 | } |
| 5646 | 5713 | ||
| @@ -5916,18 +5983,8 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags) · 269 unmodified lines | |||
| 5916 | const struct cred *creds = NULL; | 5983 | const struct cred *creds = NULL; |
| 5917 | int ret; | 5984 | int ret; |
| 5918 | 5985 | ||
| 5919 | if (req->work.personality) { | 5986 | if (req->work.creds && req->work.creds != current_cred()) |
| 5920 | const struct cred *new_creds; | 5987 | creds = override_creds(req->work.creds); |
| 5921 | |||
| 5922 | if (!(issue_flags & IO_URING_F_NONBLOCK)) | ||
| 5923 | mutex_lock(&ctx->uring_lock); | ||
| 5924 | new_creds = idr_find(&ctx->personality_idr, req->work.personality); | ||
| 5925 | if (!(issue_flags & IO_URING_F_NONBLOCK)) | ||
| 5926 | mutex_unlock(&ctx->uring_lock); | ||
| 5927 | if (!new_creds) | ||
| 5928 | return -EINVAL; | ||
| 5929 | creds = override_creds(new_creds); | ||
| 5930 | } | ||
| 5931 | 5988 | ||
| 5932 | switch (req->opcode) { | 5989 | switch (req->opcode) { |
| 5933 | case IORING_OP_NOP: | 5990 | case IORING_OP_NOP: |
| @@ -6291,7 +6348,7 @@ static int io_init_req(struct io_ring_ctx *ctx, struct io_kiocb *req, · 357 unmodified lines | |||
| 6291 | { | 6348 | { |
| 6292 | struct io_submit_state *state; | 6349 | struct io_submit_state *state; |
| 6293 | unsigned int sqe_flags; | 6350 | unsigned int sqe_flags; |
| 6294 | int ret = 0; | 6351 | int personality, ret = 0; |
| 6295 | 6352 | ||
| 6296 | req->opcode = READ_ONCE(sqe->opcode); | 6353 | req->opcode = READ_ONCE(sqe->opcode); |
| 6297 | /* same numerical values with corresponding REQ_F_*, safe to copy */ | 6354 | /* same numerical values with corresponding REQ_F_*, safe to copy */ |
| @@ -6306,6 +6363,9 @@ static int io_init_req(struct io_ring_ctx *ctx, struct io_kiocb *req, · 8 unmodified lines | |||
| 6306 | refcount_set(&req->refs, 2); | 6363 | refcount_set(&req->refs, 2); |
| 6307 | req->task = current; | 6364 | req->task = current; |
| 6308 | req->result = 0; | 6365 | req->result = 0; |
| 6366 | req->work.list.next = NULL; | ||
| 6367 | req->work.creds = NULL; | ||
| 6368 | req->work.flags = 0; | ||
| 6309 | 6369 | ||
| 6310 | /* enforce forwards compatibility on users */ | 6370 | /* enforce forwards compatibility on users */ |
| 6311 | if (unlikely(sqe_flags & ~SQE_VALID_FLAGS)) { | 6371 | if (unlikely(sqe_flags & ~SQE_VALID_FLAGS)) { |
| @@ -6323,9 +6383,13 @@ static int io_init_req(struct io_ring_ctx *ctx, struct io_kiocb *req, · 11 unmodified lines | |||
| 6323 | !io_op_defs[req->opcode].buffer_select) | 6383 | !io_op_defs[req->opcode].buffer_select) |
| 6324 | return -EOPNOTSUPP; | 6384 | return -EOPNOTSUPP; |
| 6325 | 6385 | ||
| 6326 | req->work.list.next = NULL; | 6386 | personality = READ_ONCE(sqe->personality); |
| 6327 | req->work.flags = 0; | 6387 | if (personality) { |
| 6328 | req->work.personality = READ_ONCE(sqe->personality); | 6388 | req->work.creds = xa_load(&ctx->personalities, personality); |
| 6389 | if (!req->work.creds) | ||
| 6390 | return -EINVAL; | ||
| 6391 | get_cred(req->work.creds); | ||
| 6392 | } | ||
| 6329 | state = &ctx->submit_state; | 6393 | state = &ctx->submit_state; |
| 6330 | 6394 | ||
| 6331 | /* | 6395 | /* |
| @@ -6587,7 +6651,8 @@ static int __io_sq_thread(struct io_ring_ctx *ctx, bool cap_entries) · 255 unmodified lines | |||
| 6587 | if (!list_empty(&ctx->iopoll_list)) | 6651 | if (!list_empty(&ctx->iopoll_list)) |
| 6588 | io_do_iopoll(ctx, &nr_events, 0); | 6652 | io_do_iopoll(ctx, &nr_events, 0); |
| 6589 | 6653 | ||
| 6590 | if (to_submit && likely(!percpu_ref_is_dying(&ctx->refs))) | 6654 | if (to_submit && likely(!percpu_ref_is_dying(&ctx->refs)) && |
| 6655 | !(ctx->flags & IORING_SETUP_R_DISABLED)) | ||
| 6591 | ret = io_submit_sqes(ctx, to_submit); | 6656 | ret = io_submit_sqes(ctx, to_submit); |
| 6592 | mutex_unlock(&ctx->uring_lock); | 6657 | mutex_unlock(&ctx->uring_lock); |
| 6593 | } | 6658 | } |
| @@ -6611,58 +6676,6 @@ static void io_sqd_update_thread_idle(struct io_sq_data *sqd) · 17 unmodified lines | |||
| 6611 | sqd->sq_thread_idle = sq_thread_idle; | 6676 | sqd->sq_thread_idle = sq_thread_idle; |
| 6612 | } | 6677 | } |
| 6613 | 6678 | ||
| 6614 | static void io_sqd_init_new(struct io_sq_data *sqd) | ||
| 6615 | { | ||
| 6616 | struct io_ring_ctx *ctx; | ||
| 6617 | |||
| 6618 | while (!list_empty(&sqd->ctx_new_list)) { | ||
| 6619 | ctx = list_first_entry(&sqd->ctx_new_list, struct io_ring_ctx, sqd_list); | ||
| 6620 | list_move_tail(&ctx->sqd_list, &sqd->ctx_list); | ||
| 6621 | complete(&ctx->sq_thread_comp); | ||
| 6622 | } | ||
| 6623 | |||
| 6624 | io_sqd_update_thread_idle(sqd); | ||
| 6625 | } | ||
| 6626 | |||
| 6627 | static bool io_sq_thread_should_stop(struct io_sq_data *sqd) | ||
| 6628 | { | ||
| 6629 | return test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); | ||
| 6630 | } | ||
| 6631 | |||
| 6632 | static bool io_sq_thread_should_park(struct io_sq_data *sqd) | ||
| 6633 | { | ||
| 6634 | return test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); | ||
| 6635 | } | ||
| 6636 | |||
| 6637 | static void io_sq_thread_parkme(struct io_sq_data *sqd) | ||
| 6638 | { | ||
| 6639 | for (;;) { | ||
| 6640 | /* | ||
| 6641 | * TASK_PARKED is a special state; we must serialize against | ||
| 6642 | * possible pending wakeups to avoid store-store collisions on | ||
| 6643 | * task->state. | ||
| 6644 | * | ||
| 6645 | * Such a collision might possibly result in the task state | ||
| 6646 | * changin from TASK_PARKED and us failing the | ||
| 6647 | * wait_task_inactive() in kthread_park(). | ||
| 6648 | */ | ||
| 6649 | set_special_state(TASK_PARKED); | ||
| 6650 | if (!test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state)) | ||
| 6651 | break; | ||
| 6652 | |||
| 6653 | /* | ||
| 6654 | * Thread is going to call schedule(), do not preempt it, | ||
| 6655 | * or the caller of kthread_park() may spend more time in | ||
| 6656 | * wait_task_inactive(). | ||
| 6657 | */ | ||
| 6658 | preempt_disable(); | ||
| 6659 | complete(&sqd->parked); | ||
| 6660 | schedule_preempt_disabled(); | ||
| 6661 | preempt_enable(); | ||
| 6662 | } | ||
| 6663 | __set_current_state(TASK_RUNNING); | ||
| 6664 | } | ||
| 6665 | |||
| 6666 | static int io_sq_thread(void *data) | 6679 | static int io_sq_thread(void *data) |
| 6667 | { | 6680 | { |
| 6668 | struct io_sq_data *sqd = data; | 6681 | struct io_sq_data *sqd = data; |
| @@ -6681,31 +6694,32 @@ static int io_sq_thread(void *data) · 12 unmodified lines | |||
| 6681 | set_cpus_allowed_ptr(current, cpu_online_mask); | 6694 | set_cpus_allowed_ptr(current, cpu_online_mask); |
| 6682 | current->flags |= PF_NO_SETAFFINITY; | 6695 | current->flags |= PF_NO_SETAFFINITY; |
| 6683 | 6696 | ||
| 6684 | wait_for_completion(&sqd->startup); | 6697 | down_read(&sqd->rw_lock); |
| 6685 | 6698 | ||
| 6686 | while (!io_sq_thread_should_stop(sqd)) { | 6699 | while (!test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state)) { |
| 6687 | int ret; | 6700 | int ret; |
| 6688 | bool cap_entries, sqt_spin, needs_sched; | 6701 | bool cap_entries, sqt_spin, needs_sched; |
| 6689 | 6702 | ||
| 6690 | /* | 6703 | if (test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state)) { |
| 6691 | * Any changes to the sqd lists are synchronized through the | 6704 | up_read(&sqd->rw_lock); |
| 6692 | * thread parking. This synchronizes the thread vs users, | 6705 | cond_resched(); |
| 6693 | * the users are synchronized on the sqd->ctx_lock. | 6706 | down_read(&sqd->rw_lock); |
| 6694 | */ | 6707 | io_run_task_work(); |
| 6695 | if (io_sq_thread_should_park(sqd)) { | ||
| 6696 | io_sq_thread_parkme(sqd); | ||
| 6697 | continue; | ||
| 6698 | } | ||
| 6699 | if (unlikely(!list_empty(&sqd->ctx_new_list))) { | ||
| 6700 | io_sqd_init_new(sqd); | ||
| 6701 | timeout = jiffies + sqd->sq_thread_idle; | 6708 | timeout = jiffies + sqd->sq_thread_idle; |
| 6709 | continue; | ||
| 6702 | } | 6710 | } |
| 6703 | if (fatal_signal_pending(current)) | 6711 | if (fatal_signal_pending(current)) |
| 6704 | break; | 6712 | break; |
| 6705 | sqt_spin = false; | 6713 | sqt_spin = false; |
| 6706 | cap_entries = !list_is_singular(&sqd->ctx_list); | 6714 | cap_entries = !list_is_singular(&sqd->ctx_list); |
| 6707 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) { | 6715 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) { |
| 6716 | const struct cred *creds = NULL; | ||
| 6717 | |||
| 6718 | if (ctx->sq_creds != current_cred()) | ||
| 6719 | creds = override_creds(ctx->sq_creds); | ||
| 6708 | ret = __io_sq_thread(ctx, cap_entries); | 6720 | ret = __io_sq_thread(ctx, cap_entries); |
| 6721 | if (creds) | ||
| 6722 | revert_creds(creds); | ||
| 6709 | if (!sqt_spin && (ret > 0 || !list_empty(&ctx->iopoll_list))) | 6723 | if (!sqt_spin && (ret > 0 || !list_empty(&ctx->iopoll_list))) |
| 6710 | sqt_spin = true; | 6724 | sqt_spin = true; |
| 6711 | } | 6725 | } |
| @@ -6732,12 +6746,13 @@ static int io_sq_thread(void *data) · 20 unmodified lines | |||
| 6732 | } | 6746 | } |
| 6733 | } | 6747 | } |
| 6734 | 6748 | ||
| 6735 | if (needs_sched && !io_sq_thread_should_park(sqd)) { | 6749 | if (needs_sched && !test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state)) { |
| 6736 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) | 6750 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) |
| 6737 | io_ring_set_wakeup_flag(ctx); | 6751 | io_ring_set_wakeup_flag(ctx); |
| 6738 | 6752 | ||
| 6753 | up_read(&sqd->rw_lock); | ||
| 6739 | schedule(); | 6754 | schedule(); |
| 6740 | try_to_freeze(); | 6755 | down_read(&sqd->rw_lock); |
| 6741 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) | 6756 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) |
| 6742 | io_ring_clear_wakeup_flag(ctx); | 6757 | io_ring_clear_wakeup_flag(ctx); |
| 6743 | } | 6758 | } |
| @@ -6745,32 +6760,23 @@ static int io_sq_thread(void *data) · 1 unmodified line | |||
| 6745 | finish_wait(&sqd->wait, &wait); | 6760 | finish_wait(&sqd->wait, &wait); |
| 6746 | timeout = jiffies + sqd->sq_thread_idle; | 6761 | timeout = jiffies + sqd->sq_thread_idle; |
| 6747 | } | 6762 | } |
| 6748 | 6763 | up_read(&sqd->rw_lock); | |
| 6749 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) | 6764 | down_write(&sqd->rw_lock); |
| 6750 | io_uring_cancel_sqpoll(ctx); | ||
| 6751 | |||
| 6752 | io_run_task_work(); | ||
| 6753 | |||
| 6754 | /* | 6765 | /* |
| 6755 | * Ensure that we park properly if racing with someone trying to park | 6766 | * someone may have parked and added a cancellation task_work, run |
| 6756 | * while we're exiting. If we fail to grab the lock, check park and | 6767 | * it first because we don't want it in io_uring_cancel_sqpoll() |
| 6757 | * park if necessary. The ordering with the park bit and the lock | ||
| 6758 | * ensures that we catch this reliably. | ||
| 6759 | */ | 6768 | */ |
| 6760 | if (!mutex_trylock(&sqd->lock)) { | 6769 | io_run_task_work(); |
| 6761 | if (io_sq_thread_should_park(sqd)) | ||
| 6762 | io_sq_thread_parkme(sqd); | ||
| 6763 | mutex_lock(&sqd->lock); | ||
| 6764 | } | ||
| 6765 | 6770 | ||
| 6771 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) | ||
| 6772 | io_uring_cancel_sqpoll(ctx); | ||
| 6766 | sqd->thread = NULL; | 6773 | sqd->thread = NULL; |
| 6767 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) { | 6774 | list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) |
| 6768 | ctx->sqo_exec = 1; | ||
| 6769 | io_ring_set_wakeup_flag(ctx); | 6775 | io_ring_set_wakeup_flag(ctx); |
| 6770 | } | 6776 | up_write(&sqd->rw_lock); |
| 6771 | 6777 | ||
| 6778 | io_run_task_work(); | ||
| 6772 | complete(&sqd->exited); | 6779 | complete(&sqd->exited); |
| 6773 | mutex_unlock(&sqd->lock); | ||
| 6774 | do_exit(0); | 6780 | do_exit(0); |
| 6775 | } | 6781 | } |
| 6776 | 6782 | ||
| @@ -7069,44 +7075,37 @@ static int io_sqe_files_unregister(struct io_ring_ctx *ctx) · 292 unmodified lines | |||
| 7069 | } | 7075 | } |
| 7070 | 7076 | ||
| 7071 | static void io_sq_thread_unpark(struct io_sq_data *sqd) | 7077 | static void io_sq_thread_unpark(struct io_sq_data *sqd) |
| 7072 | __releases(&sqd->lock) | 7078 | __releases(&sqd->rw_lock) |
| 7073 | { | 7079 | { |
| 7074 | if (sqd->thread == current) | 7080 | WARN_ON_ONCE(sqd->thread == current); |
| 7075 | return; | 7081 | |
| 7076 | clear_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); | 7082 | clear_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); |
| 7077 | if (sqd->thread) | 7083 | up_write(&sqd->rw_lock); |
| 7078 | wake_up_state(sqd->thread, TASK_PARKED); | ||
| 7079 | mutex_unlock(&sqd->lock); | ||
| 7080 | } | 7084 | } |
| 7081 | 7085 | ||
| 7082 | static void io_sq_thread_park(struct io_sq_data *sqd) | 7086 | static void io_sq_thread_park(struct io_sq_data *sqd) |
| 7083 | __acquires(&sqd->lock) | 7087 | __acquires(&sqd->rw_lock) |
| 7084 | { | 7088 | { |
| 7085 | if (sqd->thread == current) | 7089 | WARN_ON_ONCE(sqd->thread == current); |
| 7086 | return; | 7090 | |
| 7087 | set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); | 7091 | set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); |
| 7088 | mutex_lock(&sqd->lock); | 7092 | down_write(&sqd->rw_lock); |
| 7089 | if (sqd->thread) { | 7093 | /* set again for consistency, in case concurrent parks are happening */ |
| 7094 | set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state); | ||
| 7095 | if (sqd->thread) | ||
| 7090 | wake_up_process(sqd->thread); | 7096 | wake_up_process(sqd->thread); |
| 7091 | wait_for_completion(&sqd->parked); | ||
| 7092 | } | ||
| 7093 | } | 7097 | } |
| 7094 | 7098 | ||
| 7095 | static void io_sq_thread_stop(struct io_sq_data *sqd) | 7099 | static void io_sq_thread_stop(struct io_sq_data *sqd) |
| 7096 | { | 7100 | { |
| 7097 | if (test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state)) | 7101 | WARN_ON_ONCE(sqd->thread == current); |
| 7098 | return; | 7102 | |
| 7099 | mutex_lock(&sqd->lock); | 7103 | down_write(&sqd->rw_lock); |
| 7100 | if (sqd->thread) { | 7104 | set_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); |
| 7101 | set_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); | 7105 | if (sqd->thread) |
| 7102 | WARN_ON_ONCE(test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state)); | ||
| 7103 | wake_up_process(sqd->thread); | 7106 | wake_up_process(sqd->thread); |
| 7104 | mutex_unlock(&sqd->lock); | 7107 | up_write(&sqd->rw_lock); |
| 7105 | wait_for_completion(&sqd->exited); | 7108 | wait_for_completion(&sqd->exited); |
| 7106 | WARN_ON_ONCE(sqd->thread); | ||
| 7107 | } else { | ||
| 7108 | mutex_unlock(&sqd->lock); | ||
| 7109 | } | ||
| 7110 | } | 7109 | } |
| 7111 | 7110 | ||
| 7112 | static void io_put_sq_data(struct io_sq_data *sqd) | 7111 | static void io_put_sq_data(struct io_sq_data *sqd) |
| @@ -7122,22 +7121,15 @@ static void io_sq_thread_finish(struct io_ring_ctx *ctx) · 9 unmodified lines | |||
| 7122 | struct io_sq_data *sqd = ctx->sq_data; | 7121 | struct io_sq_data *sqd = ctx->sq_data; |
| 7123 | 7122 | ||
| 7124 | if (sqd) { | 7123 | if (sqd) { |
| 7125 | complete(&sqd->startup); | 7124 | io_sq_thread_park(sqd); |
| 7126 | if (sqd->thread) { | 7125 | list_del_init(&ctx->sqd_list); |
| 7127 | wait_for_completion(&ctx->sq_thread_comp); | ||
| 7128 | io_sq_thread_park(sqd); | ||
| 7129 | } | ||
| 7130 | |||
| 7131 | mutex_lock(&sqd->ctx_lock); | ||
| 7132 | list_del(&ctx->sqd_list); | ||
| 7133 | io_sqd_update_thread_idle(sqd); | 7126 | io_sqd_update_thread_idle(sqd); |
| 7134 | mutex_unlock(&sqd->ctx_lock); | 7127 | io_sq_thread_unpark(sqd); |
| 7135 | |||
| 7136 | if (sqd->thread) | ||
| 7137 | io_sq_thread_unpark(sqd); | ||
| 7138 | 7128 | ||
| 7139 | io_put_sq_data(sqd); | 7129 | io_put_sq_data(sqd); |
| 7140 | ctx->sq_data = NULL; | 7130 | ctx->sq_data = NULL; |
| 7131 | if (ctx->sq_creds) | ||
| 7132 | put_cred(ctx->sq_creds); | ||
| 7141 | } | 7133 | } |
| 7142 | } | 7134 | } |
| 7143 | 7135 | ||
| @@ -7161,18 +7153,32 @@ static struct io_sq_data *io_attach_sq_data(struct io_uring_params *p) · 17 unmodified lines | |||
| 7161 | fdput(f); | 7153 | fdput(f); |
| 7162 | return ERR_PTR(-EINVAL); | 7154 | return ERR_PTR(-EINVAL); |
| 7163 | } | 7155 | } |
| 7156 | if (sqd->task_tgid != current->tgid) { | ||
| 7157 | fdput(f); | ||
| 7158 | return ERR_PTR(-EPERM); | ||
| 7159 | } | ||
| 7164 | 7160 | ||
| 7165 | refcount_inc(&sqd->refs); | 7161 | refcount_inc(&sqd->refs); |
| 7166 | fdput(f); | 7162 | fdput(f); |
| 7167 | return sqd; | 7163 | return sqd; |
| 7168 | } | 7164 | } |
| 7169 | 7165 | ||
| 7170 | static struct io_sq_data *io_get_sq_data(struct io_uring_params *p) | 7166 | static struct io_sq_data *io_get_sq_data(struct io_uring_params *p, |
| 7167 | bool *attached) | ||
| 7171 | { | 7168 | { |
| 7172 | struct io_sq_data *sqd; | 7169 | struct io_sq_data *sqd; |
| 7173 | 7170 | ||
| 7174 | if (p->flags & IORING_SETUP_ATTACH_WQ) | 7171 | *attached = false; |
| 7175 | return io_attach_sq_data(p); | 7172 | if (p->flags & IORING_SETUP_ATTACH_WQ) { |
| 7173 | sqd = io_attach_sq_data(p); | ||
| 7174 | if (!IS_ERR(sqd)) { | ||
| 7175 | *attached = true; | ||
| 7176 | return sqd; | ||
| 7177 | } | ||
| 7178 | /* fall through for EPERM case, setup new sqd/task */ | ||
| 7179 | if (PTR_ERR(sqd) != -EPERM) | ||
| 7180 | return sqd; | ||
| 7181 | } | ||
| 7176 | 7182 | ||
| 7177 | sqd = kzalloc(sizeof(*sqd), GFP_KERNEL); | 7183 | sqd = kzalloc(sizeof(*sqd), GFP_KERNEL); |
| 7178 | if (!sqd) | 7184 | if (!sqd) |
| @@ -7180,12 +7186,8 @@ static struct io_sq_data *io_get_sq_data(struct io_uring_params *p) · 1 unmodified line | |||
| 7180 | 7186 | ||
| 7181 | refcount_set(&sqd->refs, 1); | 7187 | refcount_set(&sqd->refs, 1); |
| 7182 | INIT_LIST_HEAD(&sqd->ctx_list); | 7188 | INIT_LIST_HEAD(&sqd->ctx_list); |
| 7183 | INIT_LIST_HEAD(&sqd->ctx_new_list); | 7189 | init_rwsem(&sqd->rw_lock); |
| 7184 | mutex_init(&sqd->ctx_lock); | ||
| 7185 | mutex_init(&sqd->lock); | ||
| 7186 | init_waitqueue_head(&sqd->wait); | 7190 | init_waitqueue_head(&sqd->wait); |
| 7187 | init_completion(&sqd->startup); | ||
| 7188 | init_completion(&sqd->parked); | ||
| 7189 | init_completion(&sqd->exited); | 7191 | init_completion(&sqd->exited); |
| 7190 | return sqd; | 7192 | return sqd; |
| 7191 | } | 7193 | } |
| @@ -7802,7 +7804,6 @@ static int io_uring_alloc_task_context(struct task_struct *task, · 610 unmodified lines | |||
| 7802 | init_waitqueue_head(&tctx->wait); | 7804 | init_waitqueue_head(&tctx->wait); |
| 7803 | tctx->last = NULL; | 7805 | tctx->last = NULL; |
| 7804 | atomic_set(&tctx->in_idle, 0); | 7806 | atomic_set(&tctx->in_idle, 0); |
| 7805 | tctx->sqpoll = false; | ||
| 7806 | task->io_uring = tctx; | 7807 | task->io_uring = tctx; |
| 7807 | spin_lock_init(&tctx->task_lock); | 7808 | spin_lock_init(&tctx->task_lock); |
| 7808 | INIT_WQ_LIST(&tctx->task_list); | 7809 | INIT_WQ_LIST(&tctx->task_list); |
| @@ -7823,26 +7824,6 @@ void __io_uring_free(struct task_struct *tsk) · 14 unmodified lines | |||
| 7823 | tsk->io_uring = NULL; | 7824 | tsk->io_uring = NULL; |
| 7824 | } | 7825 | } |
| 7825 | 7826 | ||
| 7826 | static int io_sq_thread_fork(struct io_sq_data *sqd, struct io_ring_ctx *ctx) | ||
| 7827 | { | ||
| 7828 | struct task_struct *tsk; | ||
| 7829 | int ret; | ||
| 7830 | |||
| 7831 | clear_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); | ||
| 7832 | reinit_completion(&sqd->parked); | ||
| 7833 | ctx->sqo_exec = 0; | ||
| 7834 | sqd->task_pid = current->pid; | ||
| 7835 | tsk = create_io_thread(io_sq_thread, sqd, NUMA_NO_NODE); | ||
| 7836 | if (IS_ERR(tsk)) | ||
| 7837 | return PTR_ERR(tsk); | ||
| 7838 | ret = io_uring_alloc_task_context(tsk, ctx); | ||
| 7839 | if (ret) | ||
| 7840 | set_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); | ||
| 7841 | sqd->thread = tsk; | ||
| 7842 | wake_up_new_task(tsk); | ||
| 7843 | return ret; | ||
| 7844 | } | ||
| 7845 | |||
| 7846 | static int io_sq_offload_create(struct io_ring_ctx *ctx, | 7827 | static int io_sq_offload_create(struct io_ring_ctx *ctx, |
| 7847 | struct io_uring_params *p) | 7828 | struct io_uring_params *p) |
| 7848 | { | 7829 | { |
| @@ -7865,39 +7846,51 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx, · 16 unmodified lines | |||
| 7865 | if (ctx->flags & IORING_SETUP_SQPOLL) { | 7846 | if (ctx->flags & IORING_SETUP_SQPOLL) { |
| 7866 | struct task_struct *tsk; | 7847 | struct task_struct *tsk; |
| 7867 | struct io_sq_data *sqd; | 7848 | struct io_sq_data *sqd; |
| 7849 | bool attached; | ||
| 7868 | 7850 | ||
| 7869 | ret = -EPERM; | 7851 | ret = -EPERM; |
| 7870 | if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_NICE)) | 7852 | if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_NICE)) |
| 7871 | goto err; | 7853 | goto err; |
| 7872 | 7854 | ||
| 7873 | sqd = io_get_sq_data(p); | 7855 | sqd = io_get_sq_data(p, &attached); |
| 7874 | if (IS_ERR(sqd)) { | 7856 | if (IS_ERR(sqd)) { |
| 7875 | ret = PTR_ERR(sqd); | 7857 | ret = PTR_ERR(sqd); |
| 7876 | goto err; | 7858 | goto err; |
| 7877 | } | 7859 | } |
| 7878 | 7860 | ||
| 7861 | ctx->sq_creds = get_current_cred(); | ||
| 7879 | ctx->sq_data = sqd; | 7862 | ctx->sq_data = sqd; |
| 7880 | io_sq_thread_park(sqd); | ||
| 7881 | mutex_lock(&sqd->ctx_lock); | ||
| 7882 | list_add(&ctx->sqd_list, &sqd->ctx_new_list); | ||
| 7883 | mutex_unlock(&sqd->ctx_lock); | ||
| 7884 | io_sq_thread_unpark(sqd); | ||
| 7885 | |||
| 7886 | ctx->sq_thread_idle = msecs_to_jiffies(p->sq_thread_idle); | 7863 | ctx->sq_thread_idle = msecs_to_jiffies(p->sq_thread_idle); |
| 7887 | if (!ctx->sq_thread_idle) | 7864 | if (!ctx->sq_thread_idle) |
| 7888 | ctx->sq_thread_idle = HZ; | 7865 | ctx->sq_thread_idle = HZ; |
| 7889 | 7866 | ||
| 7890 | if (sqd->thread) | 7867 | ret = 0; |
| 7868 | io_sq_thread_park(sqd); | ||
| 7869 | /* don't attach to a dying SQPOLL thread, would be racy */ | ||
| 7870 | if (attached && !sqd->thread) { | ||
| 7871 | ret = -ENXIO; | ||
| 7872 | } else { | ||
| 7873 | list_add(&ctx->sqd_list, &sqd->ctx_list); | ||
| 7874 | io_sqd_update_thread_idle(sqd); | ||
| 7875 | } | ||
| 7876 | io_sq_thread_unpark(sqd); | ||
| 7877 | |||
| 7878 | if (ret < 0) { | ||
| 7879 | io_put_sq_data(sqd); | ||
| 7880 | ctx->sq_data = NULL; | ||
| 7881 | return ret; | ||
| 7882 | } else if (attached) { | ||
| 7891 | return 0; | 7883 | return 0; |
| 7884 | } | ||
| 7892 | 7885 | ||
| 7893 | if (p->flags & IORING_SETUP_SQ_AFF) { | 7886 | if (p->flags & IORING_SETUP_SQ_AFF) { |
| 7894 | int cpu = p->sq_thread_cpu; | 7887 | int cpu = p->sq_thread_cpu; |
| 7895 | 7888 | ||
| 7896 | ret = -EINVAL; | 7889 | ret = -EINVAL; |
| 7897 | if (cpu >= nr_cpu_ids) | 7890 | if (cpu >= nr_cpu_ids) |
| 7898 | goto err; | 7891 | goto err_sqpoll; |
| 7899 | if (!cpu_online(cpu)) | 7892 | if (!cpu_online(cpu)) |
| 7900 | goto err; | 7893 | goto err_sqpoll; |
| 7901 | 7894 | ||
| 7902 | sqd->sq_cpu = cpu; | 7895 | sqd->sq_cpu = cpu; |
| 7903 | } else { | 7896 | } else { |
| @@ -7905,15 +7898,15 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx, · 1 unmodified line | |||
| 7905 | } | 7898 | } |
| 7906 | 7899 | ||
| 7907 | sqd->task_pid = current->pid; | 7900 | sqd->task_pid = current->pid; |
| 7901 | sqd->task_tgid = current->tgid; | ||
| 7908 | tsk = create_io_thread(io_sq_thread, sqd, NUMA_NO_NODE); | 7902 | tsk = create_io_thread(io_sq_thread, sqd, NUMA_NO_NODE); |
| 7909 | if (IS_ERR(tsk)) { | 7903 | if (IS_ERR(tsk)) { |
| 7910 | ret = PTR_ERR(tsk); | 7904 | ret = PTR_ERR(tsk); |
| 7911 | goto err; | 7905 | goto err_sqpoll; |
| 7912 | } | 7906 | } |
| 7913 | ret = io_uring_alloc_task_context(tsk, ctx); | 7907 | |
| 7914 | if (ret) | ||
| 7915 | set_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state); | ||
| 7916 | sqd->thread = tsk; | 7908 | sqd->thread = tsk; |
| 7909 | ret = io_uring_alloc_task_context(tsk, ctx); | ||
| 7917 | wake_up_new_task(tsk); | 7910 | wake_up_new_task(tsk); |
| 7918 | if (ret) | 7911 | if (ret) |
| 7919 | goto err; | 7912 | goto err; |
| @@ -7927,15 +7920,9 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx, · 7 unmodified lines | |||
| 7927 | err: | 7920 | err: |
| 7928 | io_sq_thread_finish(ctx); | 7921 | io_sq_thread_finish(ctx); |
| 7929 | return ret; | 7922 | return ret; |
| 7930 | } | 7923 | err_sqpoll: |
| 7931 | 7924 | complete(&ctx->sq_data->exited); | |
| 7932 | static void io_sq_offload_start(struct io_ring_ctx *ctx) | 7925 | goto err; |
| 7933 | { | ||
| 7934 | struct io_sq_data *sqd = ctx->sq_data; | ||
| 7935 | |||
| 7936 | ctx->flags &= ~IORING_SETUP_R_DISABLED; | ||
| 7937 | if (ctx->flags & IORING_SETUP_SQPOLL) | ||
| 7938 | complete(&sqd->startup); | ||
| 7939 | } | 7926 | } |
| 7940 | 7927 | ||
| 7941 | static inline void __io_unaccount_mem(struct user_struct *user, | 7928 | static inline void __io_unaccount_mem(struct user_struct *user, |
| @@ -8418,7 +8405,6 @@ static void io_ring_ctx_free(struct io_ring_ctx *ctx) · 476 unmodified lines | |||
| 8418 | mutex_unlock(&ctx->uring_lock); | 8405 | mutex_unlock(&ctx->uring_lock); |
| 8419 | io_eventfd_unregister(ctx); | 8406 | io_eventfd_unregister(ctx); |
| 8420 | io_destroy_buffers(ctx); | 8407 | io_destroy_buffers(ctx); |
| 8421 | idr_destroy(&ctx->personality_idr); | ||
| 8422 | 8408 | ||
| 8423 | #if defined(CONFIG_UNIX) | 8409 | #if defined(CONFIG_UNIX) |
| 8424 | if (ctx->ring_sock) { | 8410 | if (ctx->ring_sock) { |
| @@ -8483,7 +8469,7 @@ static int io_unregister_personality(struct io_ring_ctx *ctx, unsigned id) · 58 unmodified lines | |||
| 8483 | { | 8469 | { |
| 8484 | const struct cred *creds; | 8470 | const struct cred *creds; |
| 8485 | 8471 | ||
| 8486 | creds = idr_remove(&ctx->personality_idr, id); | 8472 | creds = xa_erase(&ctx->personalities, id); |
| 8487 | if (creds) { | 8473 | if (creds) { |
| 8488 | put_cred(creds); | 8474 | put_cred(creds); |
| 8489 | return 0; | 8475 | return 0; |
| @@ -8492,14 +8478,6 @@ static int io_unregister_personality(struct io_ring_ctx *ctx, unsigned id) · 2 unmodified lines | |||
| 8492 | return -EINVAL; | 8478 | return -EINVAL; |
| 8493 | } | 8479 | } |
| 8494 | 8480 | ||
| 8495 | static int io_remove_personalities(int id, void *p, void *data) | ||
| 8496 | { | ||
| 8497 | struct io_ring_ctx *ctx = data; | ||
| 8498 | |||
| 8499 | io_unregister_personality(ctx, id); | ||
| 8500 | return 0; | ||
| 8501 | } | ||
| 8502 | |||
| 8503 | static bool io_run_ctx_fallback(struct io_ring_ctx *ctx) | 8481 | static bool io_run_ctx_fallback(struct io_ring_ctx *ctx) |
| 8504 | { | 8482 | { |
| 8505 | struct callback_head *work, *next; | 8483 | struct callback_head *work, *next; |
| @@ -8522,10 +8500,34 @@ static bool io_run_ctx_fallback(struct io_ring_ctx *ctx) · 16 unmodified lines | |||
| 8522 | return executed; | 8500 | return executed; |
| 8523 | } | 8501 | } |
| 8524 | 8502 | ||
| 8503 | struct io_tctx_exit { | ||
| 8504 | struct callback_head task_work; | ||
| 8505 | struct completion completion; | ||
| 8506 | struct io_ring_ctx *ctx; | ||
| 8507 | }; | ||
| 8508 | |||
| 8509 | static void io_tctx_exit_cb(struct callback_head *cb) | ||
| 8510 | { | ||
| 8511 | struct io_uring_task *tctx = current->io_uring; | ||
| 8512 | struct io_tctx_exit *work; | ||
| 8513 | |||
| 8514 | work = container_of(cb, struct io_tctx_exit, task_work); | ||
| 8515 | /* | ||
| 8516 | * When @in_idle, we're in cancellation and it's racy to remove the | ||
| 8517 | * node. It'll be removed by the end of cancellation, just ignore it. | ||
| 8518 | */ | ||
| 8519 | if (!atomic_read(&tctx->in_idle)) | ||
| 8520 | io_uring_del_task_file((unsigned long)work->ctx); | ||
| 8521 | complete(&work->completion); | ||
| 8522 | } | ||
| 8523 | |||
| 8525 | static void io_ring_exit_work(struct work_struct *work) | 8524 | static void io_ring_exit_work(struct work_struct *work) |
| 8526 | { | 8525 | { |
| 8527 | struct io_ring_ctx *ctx = container_of(work, struct io_ring_ctx, | 8526 | struct io_ring_ctx *ctx = container_of(work, struct io_ring_ctx, exit_work); |
| 8528 | exit_work); | 8527 | unsigned long timeout = jiffies + HZ * 60 * 5; |
| 8528 | struct io_tctx_exit exit; | ||
| 8529 | struct io_tctx_node *node; | ||
| 8530 | int ret; | ||
| 8529 | 8531 | ||
| 8530 | /* | 8532 | /* |
| 8531 | * If we're doing polled IO and end up having requests being | 8533 | * If we're doing polled IO and end up having requests being |
| @@ -8535,19 +8537,47 @@ static void io_ring_exit_work(struct work_struct *work) · 3 unmodified lines | |||
| 8535 | */ | 8537 | */ |
| 8536 | do { | 8538 | do { |
| 8537 | io_uring_try_cancel_requests(ctx, NULL, NULL); | 8539 | io_uring_try_cancel_requests(ctx, NULL, NULL); |
| 8540 | |||
| 8541 | WARN_ON_ONCE(time_after(jiffies, timeout)); | ||
| 8538 | } while (!wait_for_completion_timeout(&ctx->ref_comp, HZ/20)); | 8542 | } while (!wait_for_completion_timeout(&ctx->ref_comp, HZ/20)); |
| 8543 | |||
| 8544 | mutex_lock(&ctx->uring_lock); | ||
| 8545 | while (!list_empty(&ctx->tctx_list)) { | ||
| 8546 | WARN_ON_ONCE(time_after(jiffies, timeout)); | ||
| 8547 | |||
| 8548 | node = list_first_entry(&ctx->tctx_list, struct io_tctx_node, | ||
| 8549 | ctx_node); | ||
| 8550 | exit.ctx = ctx; | ||
| 8551 | init_completion(&exit.completion); | ||
| 8552 | init_task_work(&exit.task_work, io_tctx_exit_cb); | ||
| 8553 | ret = task_work_add(node->task, &exit.task_work, TWA_SIGNAL); | ||
| 8554 | if (WARN_ON_ONCE(ret)) | ||
| 8555 | continue; | ||
| 8556 | wake_up_process(node->task); | ||
| 8557 | |||
| 8558 | mutex_unlock(&ctx->uring_lock); | ||
| 8559 | wait_for_completion(&exit.completion); | ||
| 8560 | cond_resched(); | ||
| 8561 | mutex_lock(&ctx->uring_lock); | ||
| 8562 | } | ||
| 8563 | mutex_unlock(&ctx->uring_lock); | ||
| 8564 | |||
| 8539 | io_ring_ctx_free(ctx); | 8565 | io_ring_ctx_free(ctx); |
| 8540 | } | 8566 | } |
| 8541 | 8567 | ||
| 8542 | static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx) | 8568 | static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx) |
| 8543 | { | 8569 | { |
| 8570 | unsigned long index; | ||
| 8571 | struct creds *creds; | ||
| 8572 | |||
| 8544 | mutex_lock(&ctx->uring_lock); | 8573 | mutex_lock(&ctx->uring_lock); |
| 8545 | percpu_ref_kill(&ctx->refs); | 8574 | percpu_ref_kill(&ctx->refs); |
| 8546 | /* if force is set, the ring is going away. always drop after that */ | 8575 | /* if force is set, the ring is going away. always drop after that */ |
| 8547 | ctx->cq_overflow_flushed = 1; | 8576 | ctx->cq_overflow_flushed = 1; |
| 8548 | if (ctx->rings) | 8577 | if (ctx->rings) |
| 8549 | __io_cqring_overflow_flush(ctx, true, NULL, NULL); | 8578 | __io_cqring_overflow_flush(ctx, true, NULL, NULL); |
| 8550 | idr_for_each(&ctx->personality_idr, io_remove_personalities, ctx); | 8579 | xa_for_each(&ctx->personalities, index, creds) |
| 8580 | io_unregister_personality(ctx, index); | ||
| 8551 | mutex_unlock(&ctx->uring_lock); | 8581 | mutex_unlock(&ctx->uring_lock); |
| 8552 | 8582 | ||
| 8553 | io_kill_timeouts(ctx, NULL, NULL); | 8583 | io_kill_timeouts(ctx, NULL, NULL); |
| @@ -8600,11 +8630,11 @@ static bool io_cancel_task_cb(struct io_wq_work *work, void *data) · 46 unmodified lines | |||
| 8600 | return ret; | 8630 | return ret; |
| 8601 | } | 8631 | } |
| 8602 | 8632 | ||
| 8603 | static void io_cancel_defer_files(struct io_ring_ctx *ctx, | 8633 | static bool io_cancel_defer_files(struct io_ring_ctx *ctx, |
| 8604 | struct task_struct *task, | 8634 | struct task_struct *task, |
| 8605 | struct files_struct *files) | 8635 | struct files_struct *files) |
| 8606 | { | 8636 | { |
| 8607 | struct io_defer_entry *de = NULL; | 8637 | struct io_defer_entry *de; |
| 8608 | LIST_HEAD(list); | 8638 | LIST_HEAD(list); |
| 8609 | 8639 | ||
| 8610 | spin_lock_irq(&ctx->completion_lock); | 8640 | spin_lock_irq(&ctx->completion_lock); |
| @@ -8615,6 +8645,8 @@ static void io_cancel_defer_files(struct io_ring_ctx *ctx, · 4 unmodified lines | |||
| 8615 | } | 8645 | } |
| 8616 | } | 8646 | } |
| 8617 | spin_unlock_irq(&ctx->completion_lock); | 8647 | spin_unlock_irq(&ctx->completion_lock); |
| 8648 | if (list_empty(&list)) | ||
| 8649 | return false; | ||
| 8618 | 8650 | ||
| 8619 | while (!list_empty(&list)) { | 8651 | while (!list_empty(&list)) { |
| 8620 | de = list_first_entry(&list, struct io_defer_entry, list); | 8652 | de = list_first_entry(&list, struct io_defer_entry, list); |
| @@ -8624,6 +8656,38 @@ static void io_cancel_defer_files(struct io_ring_ctx *ctx, · 3 unmodified lines | |||
| 8624 | io_req_complete(de->req, -ECANCELED); | 8656 | io_req_complete(de->req, -ECANCELED); |
| 8625 | kfree(de); | 8657 | kfree(de); |
| 8626 | } | 8658 | } |
| 8659 | return true; | ||
| 8660 | } | ||
| 8661 | |||
| 8662 | static bool io_cancel_ctx_cb(struct io_wq_work *work, void *data) | ||
| 8663 | { | ||
| 8664 | struct io_kiocb *req = container_of(work, struct io_kiocb, work); | ||
| 8665 | |||
| 8666 | return req->ctx == data; | ||
| 8667 | } | ||
| 8668 | |||
| 8669 | static bool io_uring_try_cancel_iowq(struct io_ring_ctx *ctx) | ||
| 8670 | { | ||
| 8671 | struct io_tctx_node *node; | ||
| 8672 | enum io_wq_cancel cret; | ||
| 8673 | bool ret = false; | ||
| 8674 | |||
| 8675 | mutex_lock(&ctx->uring_lock); | ||
| 8676 | list_for_each_entry(node, &ctx->tctx_list, ctx_node) { | ||
| 8677 | struct io_uring_task *tctx = node->task->io_uring; | ||
| 8678 | |||
| 8679 | /* | ||
| 8680 | * io_wq will stay alive while we hold uring_lock, because it's | ||
| 8681 | * killed after ctx nodes, which requires to take the lock. | ||
| 8682 | */ | ||
| 8683 | if (!tctx || !tctx->io_wq) | ||
| 8684 | continue; | ||
| 8685 | cret = io_wq_cancel_cb(tctx->io_wq, io_cancel_ctx_cb, ctx, true); | ||
| 8686 | ret |= (cret != IO_WQ_CANCEL_NOTFOUND); | ||
| 8687 | } | ||
| 8688 | mutex_unlock(&ctx->uring_lock); | ||
| 8689 | |||
| 8690 | return ret; | ||
| 8627 | } | 8691 | } |
| 8628 | 8692 | ||
| 8629 | static void io_uring_try_cancel_requests(struct io_ring_ctx *ctx, | 8693 | static void io_uring_try_cancel_requests(struct io_ring_ctx *ctx, |
| @@ -8631,27 +8695,34 @@ static void io_uring_try_cancel_requests(struct io_ring_ctx *ctx, · 1 unmodified line | |||
| 8631 | struct files_struct *files) | 8695 | struct files_struct *files) |
| 8632 | { | 8696 | { |
| 8633 | struct io_task_cancel cancel = { .task = task, .files = files, }; | 8697 | struct io_task_cancel cancel = { .task = task, .files = files, }; |
| 8634 | struct task_struct *tctx_task = task ?: current; | 8698 | struct io_uring_task *tctx = task ? task->io_uring : NULL; |
| 8635 | struct io_uring_task *tctx = tctx_task->io_uring; | ||
| 8636 | 8699 | ||
| 8637 | while (1) { | 8700 | while (1) { |
| 8638 | enum io_wq_cancel cret; | 8701 | enum io_wq_cancel cret; |
| 8639 | bool ret = false; | 8702 | bool ret = false; |
| 8640 | 8703 | ||
| 8641 | if (tctx && tctx->io_wq) { | 8704 | if (!task) { |
| 8705 | ret |= io_uring_try_cancel_iowq(ctx); | ||
| 8706 | } else if (tctx && tctx->io_wq) { | ||
| 8707 | /* | ||
| 8708 | * Cancels requests of all rings, not only @ctx, but | ||
| 8709 | * it's fine as the task is in exit/exec. | ||
| 8710 | */ | ||
| 8642 | cret = io_wq_cancel_cb(tctx->io_wq, io_cancel_task_cb, | 8711 | cret = io_wq_cancel_cb(tctx->io_wq, io_cancel_task_cb, |
| 8643 | &cancel, true); | 8712 | &cancel, true); |
| 8644 | ret |= (cret != IO_WQ_CANCEL_NOTFOUND); | 8713 | ret |= (cret != IO_WQ_CANCEL_NOTFOUND); |
| 8645 | } | 8714 | } |
| 8646 | 8715 | ||
| 8647 | /* SQPOLL thread does its own polling */ | 8716 | /* SQPOLL thread does its own polling */ |
| 8648 | if (!(ctx->flags & IORING_SETUP_SQPOLL) && !files) { | 8717 | if ((!(ctx->flags & IORING_SETUP_SQPOLL) && !files) || |
| 8718 | (ctx->sq_data && ctx->sq_data->thread == current)) { | ||
| 8649 | while (!list_empty_careful(&ctx->iopoll_list)) { | 8719 | while (!list_empty_careful(&ctx->iopoll_list)) { |
| 8650 | io_iopoll_try_reap_events(ctx); | 8720 | io_iopoll_try_reap_events(ctx); |
| 8651 | ret = true; | 8721 | ret = true; |
| 8652 | } | 8722 | } |
| 8653 | } | 8723 | } |
| 8654 | 8724 | ||
| 8725 | ret |= io_cancel_defer_files(ctx, task, files); | ||
| 8655 | ret |= io_poll_remove_all(ctx, task, files); | 8726 | ret |= io_poll_remove_all(ctx, task, files); |
| 8656 | ret |= io_kill_timeouts(ctx, task, files); | 8727 | ret |= io_kill_timeouts(ctx, task, files); |
| 8657 | ret |= io_run_task_work(); | 8728 | ret |= io_run_task_work(); |
| @@ -8691,58 +8762,21 @@ static void io_uring_cancel_files(struct io_ring_ctx *ctx, · 33 unmodified lines | |||
| 8691 | 8762 | ||
| 8692 | io_uring_try_cancel_requests(ctx, task, files); | 8763 | io_uring_try_cancel_requests(ctx, task, files); |
| 8693 | 8764 | ||
| 8694 | if (ctx->sq_data) | ||
| 8695 | io_sq_thread_unpark(ctx->sq_data); | ||
| 8696 | prepare_to_wait(&task->io_uring->wait, &wait, | 8765 | prepare_to_wait(&task->io_uring->wait, &wait, |
| 8697 | TASK_UNINTERRUPTIBLE); | 8766 | TASK_UNINTERRUPTIBLE); |
| 8698 | if (inflight == io_uring_count_inflight(ctx, task, files)) | 8767 | if (inflight == io_uring_count_inflight(ctx, task, files)) |
| 8699 | schedule(); | 8768 | schedule(); |
| 8700 | finish_wait(&task->io_uring->wait, &wait); | 8769 | finish_wait(&task->io_uring->wait, &wait); |
| 8701 | if (ctx->sq_data) | ||
| 8702 | io_sq_thread_park(ctx->sq_data); | ||
| 8703 | } | ||
| 8704 | } | ||
| 8705 | |||
| 8706 | /* | ||
| 8707 | * We need to iteratively cancel requests, in case a request has dependent | ||
| 8708 | * hard links. These persist even for failure of cancelations, hence keep | ||
| 8709 | * looping until none are found. | ||
| 8710 | */ | ||
| 8711 | static void io_uring_cancel_task_requests(struct io_ring_ctx *ctx, | ||
| 8712 | struct files_struct *files) | ||
| 8713 | { | ||
| 8714 | struct task_struct *task = current; | ||
| 8715 | |||
| 8716 | if ((ctx->flags & IORING_SETUP_SQPOLL) && ctx->sq_data) { | ||
| 8717 | /* never started, nothing to cancel */ | ||
| 8718 | if (ctx->flags & IORING_SETUP_R_DISABLED) { | ||
| 8719 | io_sq_offload_start(ctx); | ||
| 8720 | return; | ||
| 8721 | } | ||
| 8722 | io_sq_thread_park(ctx->sq_data); | ||
| 8723 | task = ctx->sq_data->thread; | ||
| 8724 | if (task) | ||
| 8725 | atomic_inc(&task->io_uring->in_idle); | ||
| 8726 | } | 8770 | } |
| 8727 | |||
| 8728 | io_cancel_defer_files(ctx, task, files); | ||
| 8729 | |||
| 8730 | io_uring_cancel_files(ctx, task, files); | ||
| 8731 | if (!files) | ||
| 8732 | io_uring_try_cancel_requests(ctx, task, NULL); | ||
| 8733 | |||
| 8734 | if (task) | ||
| 8735 | atomic_dec(&task->io_uring->in_idle); | ||
| 8736 | if (ctx->sq_data) | ||
| 8737 | io_sq_thread_unpark(ctx->sq_data); | ||
| 8738 | } | 8771 | } |
| 8739 | 8772 | ||
| 8740 | /* | 8773 | /* |
| 8741 | * Note that this task has used io_uring. We use it for cancelation purposes. | 8774 | * Note that this task has used io_uring. We use it for cancelation purposes. |
| 8742 | */ | 8775 | */ |
| 8743 | static int io_uring_add_task_file(struct io_ring_ctx *ctx, struct file *file) | 8776 | static int io_uring_add_task_file(struct io_ring_ctx *ctx) |
| 8744 | { | 8777 | { |
| 8745 | struct io_uring_task *tctx = current->io_uring; | 8778 | struct io_uring_task *tctx = current->io_uring; |
| 8779 | struct io_tctx_node *node; | ||
| 8746 | int ret; | 8780 | int ret; |
| 8747 | 8781 | ||
| 8748 | if (unlikely(!tctx)) { | 8782 | if (unlikely(!tctx)) { |
| @@ -8751,102 +8785,151 @@ static int io_uring_add_task_file(struct io_ring_ctx *ctx, struct file *file) · 2 unmodified lines | |||
| 8751 | return ret; | 8785 | return ret; |
| 8752 | tctx = current->io_uring; | 8786 | tctx = current->io_uring; |
| 8753 | } | 8787 | } |
| 8754 | if (tctx->last != file) { | 8788 | if (tctx->last != ctx) { |
| 8755 | void *old = xa_load(&tctx->xa, (unsigned long)file); | 8789 | void *old = xa_load(&tctx->xa, (unsigned long)ctx); |
| 8756 | 8790 | ||
| 8757 | if (!old) { | 8791 | if (!old) { |
| 8758 | get_file(file); | 8792 | node = kmalloc(sizeof(*node), GFP_KERNEL); |
| 8759 | ret = xa_err(xa_store(&tctx->xa, (unsigned long)file, | 8793 | if (!node) |
| 8760 | file, GFP_KERNEL)); | 8794 | return -ENOMEM; |
| 8795 | node->ctx = ctx; | ||
| 8796 | node->task = current; | ||
| 8797 | |||
| 8798 | ret = xa_err(xa_store(&tctx->xa, (unsigned long)ctx, | ||
| 8799 | node, GFP_KERNEL)); | ||
| 8761 | if (ret) { | 8800 | if (ret) { |
| 8762 | fput(file); | 8801 | kfree(node); |
| 8763 | return ret; | 8802 | return ret; |
| 8764 | } | 8803 | } |
| 8804 | |||
| 8805 | mutex_lock(&ctx->uring_lock); | ||
| 8806 | list_add(&node->ctx_node, &ctx->tctx_list); | ||
| 8807 | mutex_unlock(&ctx->uring_lock); | ||
| 8765 | } | 8808 | } |
| 8766 | tctx->last = file; | 8809 | tctx->last = ctx; |
| 8767 | } | 8810 | } |
| 8768 | |||
| 8769 | /* | ||
| 8770 | * This is race safe in that the task itself is doing this, hence it | ||
| 8771 | * cannot be going through the exit/cancel paths at the same time. | ||
| 8772 | * This cannot be modified while exit/cancel is running. | ||
| 8773 | */ | ||
| 8774 | if (!tctx->sqpoll && (ctx->flags & IORING_SETUP_SQPOLL)) | ||
| 8775 | tctx->sqpoll = true; | ||
| 8776 | |||
| 8777 | return 0; | 8811 | return 0; |
| 8778 | } | 8812 | } |
| 8779 | 8813 | ||
| 8780 | /* | 8814 | /* |
| 8781 | * Remove this io_uring_file -> task mapping. | 8815 | * Remove this io_uring_file -> task mapping. |
| 8782 | */ | 8816 | */ |
| 8783 | static void io_uring_del_task_file(struct file *file) | 8817 | static void io_uring_del_task_file(unsigned long index) |
| 8784 | { | 8818 | { |
| 8785 | struct io_uring_task *tctx = current->io_uring; | 8819 | struct io_uring_task *tctx = current->io_uring; |
| 8820 | struct io_tctx_node *node; | ||
| 8786 | 8821 | ||
| 8787 | if (tctx->last == file) | 8822 | if (!tctx) |
| 8823 | return; | ||
| 8824 | node = xa_erase(&tctx->xa, index); | ||
| 8825 | if (!node) | ||
| 8826 | return; | ||
| 8827 | |||
| 8828 | WARN_ON_ONCE(current != node->task); | ||
| 8829 | WARN_ON_ONCE(list_empty(&node->ctx_node)); | ||
| 8830 | |||
| 8831 | mutex_lock(&node->ctx->uring_lock); | ||
| 8832 | list_del(&node->ctx_node); | ||
| 8833 | mutex_unlock(&node->ctx->uring_lock); | ||
| 8834 | |||
| 8835 | if (tctx->last == node->ctx) | ||
| 8788 | tctx->last = NULL; | 8836 | tctx->last = NULL; |
| 8789 | file = xa_erase(&tctx->xa, (unsigned long)file); | 8837 | kfree(node); |
| 8790 | if (file) | ||
| 8791 | fput(file); | ||
| 8792 | } | 8838 | } |
| 8793 | 8839 | ||
| 8794 | static void io_uring_clean_tctx(struct io_uring_task *tctx) | 8840 | static void io_uring_clean_tctx(struct io_uring_task *tctx) |
| 8795 | { | 8841 | { |
| 8796 | struct file *file; | 8842 | struct io_tctx_node *node; |
| 8797 | unsigned long index; | 8843 | unsigned long index; |
| 8798 | 8844 | ||
| 8799 | xa_for_each(&tctx->xa, index, file) | 8845 | xa_for_each(&tctx->xa, index, node) |
| 8800 | io_uring_del_task_file(file); | 8846 | io_uring_del_task_file(index); |
| 8801 | if (tctx->io_wq) { | 8847 | if (tctx->io_wq) { |
| 8802 | io_wq_put_and_exit(tctx->io_wq); | 8848 | io_wq_put_and_exit(tctx->io_wq); |
| 8803 | tctx->io_wq = NULL; | 8849 | tctx->io_wq = NULL; |
| 8804 | } | 8850 | } |
| 8805 | } | 8851 | } |
| 8806 | 8852 | ||
| 8853 | static s64 tctx_inflight(struct io_uring_task *tctx) | ||
| 8854 | { | ||
| 8855 | return percpu_counter_sum(&tctx->inflight); | ||
| 8856 | } | ||
| 8857 | |||
| 8858 | static void io_sqpoll_cancel_cb(struct callback_head *cb) | ||
| 8859 | { | ||
| 8860 | struct io_tctx_exit *work = container_of(cb, struct io_tctx_exit, task_work); | ||
| 8861 | struct io_ring_ctx *ctx = work->ctx; | ||
| 8862 | struct io_sq_data *sqd = ctx->sq_data; | ||
| 8863 | |||
| 8864 | if (sqd->thread) | ||
| 8865 | io_uring_cancel_sqpoll(ctx); | ||
| 8866 | complete(&work->completion); | ||
| 8867 | } | ||
| 8868 | |||
| 8869 | static void io_sqpoll_cancel_sync(struct io_ring_ctx *ctx) | ||
| 8870 | { | ||
| 8871 | struct io_sq_data *sqd = ctx->sq_data; | ||
| 8872 | struct io_tctx_exit work = { .ctx = ctx, }; | ||
| 8873 | struct task_struct *task; | ||
| 8874 | |||
| 8875 | io_sq_thread_park(sqd); | ||
| 8876 | list_del_init(&ctx->sqd_list); | ||
| 8877 | io_sqd_update_thread_idle(sqd); | ||
| 8878 | task = sqd->thread; | ||
| 8879 | if (task) { | ||
| 8880 | init_completion(&work.completion); | ||
| 8881 | init_task_work(&work.task_work, io_sqpoll_cancel_cb); | ||
| 8882 | WARN_ON_ONCE(task_work_add(task, &work.task_work, TWA_SIGNAL)); | ||
| 8883 | wake_up_process(task); | ||
| 8884 | } | ||
| 8885 | io_sq_thread_unpark(sqd); | ||
| 8886 | |||
| 8887 | if (task) | ||
| 8888 | wait_for_completion(&work.completion); | ||
| 8889 | } | ||
| 8890 | |||
| 8807 | void __io_uring_files_cancel(struct files_struct *files) | 8891 | void __io_uring_files_cancel(struct files_struct *files) |
| 8808 | { | 8892 | { |
| 8809 | struct io_uring_task *tctx = current->io_uring; | 8893 | struct io_uring_task *tctx = current->io_uring; |
| 8810 | struct file *file; | 8894 | struct io_tctx_node *node; |
| 8811 | unsigned long index; | 8895 | unsigned long index; |
| 8812 | 8896 | ||
| 8813 | /* make sure overflow events are dropped */ | 8897 | /* make sure overflow events are dropped */ |
| 8814 | atomic_inc(&tctx->in_idle); | 8898 | atomic_inc(&tctx->in_idle); |
| 8815 | xa_for_each(&tctx->xa, index, file) | 8899 | xa_for_each(&tctx->xa, index, node) { |
| 8816 | io_uring_cancel_task_requests(file->private_data, files); | 8900 | struct io_ring_ctx *ctx = node->ctx; |
| 8901 | |||
| 8902 | if (ctx->sq_data) { | ||
| 8903 | io_sqpoll_cancel_sync(ctx); | ||
| 8904 | continue; | ||
| 8905 | } | ||
| 8906 | io_uring_cancel_files(ctx, current, files); | ||
| 8907 | if (!files) | ||
| 8908 | io_uring_try_cancel_requests(ctx, current, NULL); | ||
| 8909 | } | ||
| 8817 | atomic_dec(&tctx->in_idle); | 8910 | atomic_dec(&tctx->in_idle); |
| 8818 | 8911 | ||
| 8819 | if (files) | 8912 | if (files) |
| 8820 | io_uring_clean_tctx(tctx); | 8913 | io_uring_clean_tctx(tctx); |
| 8821 | } | 8914 | } |
| 8822 | 8915 | ||
| 8823 | static s64 tctx_inflight(struct io_uring_task *tctx) | 8916 | /* should only be called by SQPOLL task */ |
| 8824 | { | ||
| 8825 | return percpu_counter_sum(&tctx->inflight); | ||
| 8826 | } | ||
| 8827 | |||
| 8828 | static void io_uring_cancel_sqpoll(struct io_ring_ctx *ctx) | 8917 | static void io_uring_cancel_sqpoll(struct io_ring_ctx *ctx) |
| 8829 | { | 8918 | { |
| 8830 | struct io_sq_data *sqd = ctx->sq_data; | 8919 | struct io_sq_data *sqd = ctx->sq_data; |
| 8831 | struct io_uring_task *tctx; | 8920 | struct io_uring_task *tctx = current->io_uring; |
| 8832 | s64 inflight; | 8921 | s64 inflight; |
| 8833 | DEFINE_WAIT(wait); | 8922 | DEFINE_WAIT(wait); |
| 8834 | 8923 | ||
| 8835 | if (!sqd) | 8924 | WARN_ON_ONCE(!sqd || ctx->sq_data->thread != current); |
| 8836 | return; | 8925 | |
| 8837 | io_sq_thread_park(sqd); | ||
| 8838 | if (!sqd->thread || !sqd->thread->io_uring) { | ||
| 8839 | io_sq_thread_unpark(sqd); | ||
| 8840 | return; | ||
| 8841 | } | ||
| 8842 | tctx = ctx->sq_data->thread->io_uring; | ||
| 8843 | atomic_inc(&tctx->in_idle); | 8926 | atomic_inc(&tctx->in_idle); |
| 8844 | do { | 8927 | do { |
| 8845 | /* read completions before cancelations */ | 8928 | /* read completions before cancelations */ |
| 8846 | inflight = tctx_inflight(tctx); | 8929 | inflight = tctx_inflight(tctx); |
| 8847 | if (!inflight) | 8930 | if (!inflight) |
| 8848 | break; | 8931 | break; |
| 8849 | io_uring_cancel_task_requests(ctx, NULL); | 8932 | io_uring_try_cancel_requests(ctx, current, NULL); |
| 8850 | 8933 | ||
| 8851 | prepare_to_wait(&tctx->wait, &wait, TASK_UNINTERRUPTIBLE); | 8934 | prepare_to_wait(&tctx->wait, &wait, TASK_UNINTERRUPTIBLE); |
| 8852 | /* | 8935 | /* |
| @@ -8859,7 +8942,6 @@ static void io_uring_cancel_sqpoll(struct io_ring_ctx *ctx) · 6 unmodified lines | |||
| 8859 | finish_wait(&tctx->wait, &wait); | 8942 | finish_wait(&tctx->wait, &wait); |
| 8860 | } while (1); | 8943 | } while (1); |
| 8861 | atomic_dec(&tctx->in_idle); | 8944 | atomic_dec(&tctx->in_idle); |
| 8862 | io_sq_thread_unpark(sqd); | ||
| 8863 | } | 8945 | } |
| 8864 | 8946 | ||
| 8865 | /* | 8947 | /* |
| @@ -8874,15 +8956,6 @@ void __io_uring_task_cancel(void) · 8 unmodified lines | |||
| 8874 | 8956 | ||
| 8875 | /* make sure overflow events are dropped */ | 8957 | /* make sure overflow events are dropped */ |
| 8876 | atomic_inc(&tctx->in_idle); | 8958 | atomic_inc(&tctx->in_idle); |
| 8877 | |||
| 8878 | if (tctx->sqpoll) { | ||
| 8879 | struct file *file; | ||
| 8880 | unsigned long index; | ||
| 8881 | |||
| 8882 | xa_for_each(&tctx->xa, index, file) | ||
| 8883 | io_uring_cancel_sqpoll(file->private_data); | ||
| 8884 | } | ||
| 8885 | |||
| 8886 | do { | 8959 | do { |
| 8887 | /* read completions before cancelations */ | 8960 | /* read completions before cancelations */ |
| 8888 | inflight = tctx_inflight(tctx); | 8961 | inflight = tctx_inflight(tctx); |
| @@ -8981,7 +9054,6 @@ static unsigned long io_uring_nommu_get_unmapped_area(struct file *file, · 92 unmodified lines | |||
| 8981 | 9054 | ||
| 8982 | static int io_sqpoll_wait_sq(struct io_ring_ctx *ctx) | 9055 | static int io_sqpoll_wait_sq(struct io_ring_ctx *ctx) |
| 8983 | { | 9056 | { |
| 8984 | int ret = 0; | ||
| 8985 | DEFINE_WAIT(wait); | 9057 | DEFINE_WAIT(wait); |
| 8986 | 9058 | ||
| 8987 | do { | 9059 | do { |
| @@ -8995,7 +9067,7 @@ static int io_sqpoll_wait_sq(struct io_ring_ctx *ctx) · 7 unmodified lines | |||
| 8995 | } while (!signal_pending(current)); | 9067 | } while (!signal_pending(current)); |
| 8996 | 9068 | ||
| 8997 | finish_wait(&ctx->sqo_sq_wait, &wait); | 9069 | finish_wait(&ctx->sqo_sq_wait, &wait); |
| 8998 | return ret; | 9070 | return 0; |
| 8999 | } | 9071 | } |
| 9000 | 9072 | ||
| 9001 | static int io_get_ext_arg(unsigned flags, const void __user *argp, size_t *argsz, | 9073 | static int io_get_ext_arg(unsigned flags, const void __user *argp, size_t *argsz, |
| @@ -9069,13 +9141,10 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, · 67 unmodified lines | |||
| 9069 | if (ctx->flags & IORING_SETUP_SQPOLL) { | 9141 | if (ctx->flags & IORING_SETUP_SQPOLL) { |
| 9070 | io_cqring_overflow_flush(ctx, false, NULL, NULL); | 9142 | io_cqring_overflow_flush(ctx, false, NULL, NULL); |
| 9071 | 9143 | ||
| 9072 | if (unlikely(ctx->sqo_exec)) { | ||
| 9073 | ret = io_sq_thread_fork(ctx->sq_data, ctx); | ||
| 9074 | if (ret) | ||
| 9075 | goto out; | ||
| 9076 | ctx->sqo_exec = 0; | ||
| 9077 | } | ||
| 9078 | ret = -EOWNERDEAD; | 9144 | ret = -EOWNERDEAD; |
| 9145 | if (unlikely(ctx->sq_data->thread == NULL)) { | ||
| 9146 | goto out; | ||
| 9147 | } | ||
| 9079 | if (flags & IORING_ENTER_SQ_WAKEUP) | 9148 | if (flags & IORING_ENTER_SQ_WAKEUP) |
| 9080 | wake_up(&ctx->sq_data->wait); | 9149 | wake_up(&ctx->sq_data->wait); |
| 9081 | if (flags & IORING_ENTER_SQ_WAIT) { | 9150 | if (flags & IORING_ENTER_SQ_WAIT) { |
| @@ -9085,7 +9154,7 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, · 3 unmodified lines | |||
| 9085 | } | 9154 | } |
| 9086 | submitted = to_submit; | 9155 | submitted = to_submit; |
| 9087 | } else if (to_submit) { | 9156 | } else if (to_submit) { |
| 9088 | ret = io_uring_add_task_file(ctx, f.file); | 9157 | ret = io_uring_add_task_file(ctx); |
| 9089 | if (unlikely(ret)) | 9158 | if (unlikely(ret)) |
| 9090 | goto out; | 9159 | goto out; |
| 9091 | mutex_lock(&ctx->uring_lock); | 9160 | mutex_lock(&ctx->uring_lock); |
| @@ -9127,10 +9196,9 @@ out_fput: · 35 unmodified lines | |||
| 9127 | } | 9196 | } |
| 9128 | 9197 | ||
| 9129 | #ifdef CONFIG_PROC_FS | 9198 | #ifdef CONFIG_PROC_FS |
| 9130 | static int io_uring_show_cred(int id, void *p, void *data) | 9199 | static int io_uring_show_cred(struct seq_file *m, unsigned int id, |
| 9200 | const struct cred *cred) | ||
| 9131 | { | 9201 | { |
| 9132 | const struct cred *cred = p; | ||
| 9133 | struct seq_file *m = data; | ||
| 9134 | struct user_namespace *uns = seq_user_ns(m); | 9202 | struct user_namespace *uns = seq_user_ns(m); |
| 9135 | struct group_info *gi; | 9203 | struct group_info *gi; |
| 9136 | kernel_cap_t cap; | 9204 | kernel_cap_t cap; |
| @@ -9198,9 +9266,13 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m) · 61 unmodified lines | |||
| 9198 | seq_printf(m, "%5u: 0x%llx/%u\n", i, buf->ubuf, | 9266 | seq_printf(m, "%5u: 0x%llx/%u\n", i, buf->ubuf, |
| 9199 | (unsigned int) buf->len); | 9267 | (unsigned int) buf->len); |
| 9200 | } | 9268 | } |
| 9201 | if (has_lock && !idr_is_empty(&ctx->personality_idr)) { | 9269 | if (has_lock && !xa_empty(&ctx->personalities)) { |
| 9270 | unsigned long index; | ||
| 9271 | const struct cred *cred; | ||
| 9272 | |||
| 9202 | seq_printf(m, "Personalities:\n"); | 9273 | seq_printf(m, "Personalities:\n"); |
| 9203 | idr_for_each(&ctx->personality_idr, io_uring_show_cred, m); | 9274 | xa_for_each(&ctx->personalities, index, cred) |
| 9275 | io_uring_show_cred(m, index, cred); | ||
| 9204 | } | 9276 | } |
| 9205 | seq_printf(m, "PollList:\n"); | 9277 | seq_printf(m, "PollList:\n"); |
| 9206 | spin_lock_irq(&ctx->completion_lock); | 9278 | spin_lock_irq(&ctx->completion_lock); |
| @@ -9294,7 +9366,7 @@ static int io_uring_install_fd(struct io_ring_ctx *ctx, struct file *file) · 87 unmodified lines | |||
| 9294 | if (fd < 0) | 9366 | if (fd < 0) |
| 9295 | return fd; | 9367 | return fd; |
| 9296 | 9368 | ||
| 9297 | ret = io_uring_add_task_file(ctx, file); | 9369 | ret = io_uring_add_task_file(ctx); |
| 9298 | if (ret) { | 9370 | if (ret) { |
| 9299 | put_unused_fd(fd); | 9371 | put_unused_fd(fd); |
| 9300 | return ret; | 9372 | return ret; |
| @@ -9402,9 +9474,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, · 101 unmodified lines | |||
| 9402 | if (ret) | 9474 | if (ret) |
| 9403 | goto err; | 9475 | goto err; |
| 9404 | 9476 | ||
| 9405 | if (!(p->flags & IORING_SETUP_R_DISABLED)) | ||
| 9406 | io_sq_offload_start(ctx); | ||
| 9407 | |||
| 9408 | memset(&p->sq_off, 0, sizeof(p->sq_off)); | 9477 | memset(&p->sq_off, 0, sizeof(p->sq_off)); |
| 9409 | p->sq_off.head = offsetof(struct io_rings, sq.head); | 9478 | p->sq_off.head = offsetof(struct io_rings, sq.head); |
| 9410 | p->sq_off.tail = offsetof(struct io_rings, sq.tail); | 9479 | p->sq_off.tail = offsetof(struct io_rings, sq.tail); |
| @@ -9532,14 +9601,16 @@ out: · 121 unmodified lines | |||
| 9532 | static int io_register_personality(struct io_ring_ctx *ctx) | 9601 | static int io_register_personality(struct io_ring_ctx *ctx) |
| 9533 | { | 9602 | { |
| 9534 | const struct cred *creds; | 9603 | const struct cred *creds; |
| 9604 | u32 id; | ||
| 9535 | int ret; | 9605 | int ret; |
| 9536 | 9606 | ||
| 9537 | creds = get_current_cred(); | 9607 | creds = get_current_cred(); |
| 9538 | 9608 | ||
| 9539 | ret = idr_alloc_cyclic(&ctx->personality_idr, (void *) creds, 1, | 9609 | ret = xa_alloc_cyclic(&ctx->personalities, &id, (void *)creds, |
| 9540 | USHRT_MAX, GFP_KERNEL); | 9610 | XA_LIMIT(0, USHRT_MAX), &ctx->pers_next, GFP_KERNEL); |
| 9541 | if (ret < 0) | 9611 | if (!ret) |
| 9542 | put_cred(creds); | 9612 | return id; |
| 9613 | put_cred(creds); | ||
| 9543 | return ret; | 9614 | return ret; |
| 9544 | } | 9615 | } |
| 9545 | 9616 | ||
| @@ -9621,7 +9692,9 @@ static int io_register_enable_rings(struct io_ring_ctx *ctx) · 75 unmodified lines | |||
| 9621 | if (ctx->restrictions.registered) | 9692 | if (ctx->restrictions.registered) |
| 9622 | ctx->restricted = 1; | 9693 | ctx->restricted = 1; |
| 9623 | 9694 | ||
| 9624 | io_sq_offload_start(ctx); | 9695 | ctx->flags &= ~IORING_SETUP_R_DISABLED; |
| 9696 | if (ctx->sq_data && wq_has_sleeper(&ctx->sq_data->wait)) | ||
| 9697 | wake_up(&ctx->sq_data->wait); | ||
| 9625 | return 0; | 9698 | return 0; |
| 9626 | } | 9699 | } |
| 9627 | 9700 | ||
| @@ -1221,7 +1221,7 @@ iomap_alloc_ioend(struct inode *inode, struct iomap_writepage_ctx *wpc, · 1220 unmodified lines | |||
| 1221 | struct iomap_ioend *ioend; | 1221 | struct iomap_ioend *ioend; |
| 1222 | struct bio *bio; | 1222 | struct bio *bio; |
| 1223 | 1223 | ||
| 1224 | bio = bio_alloc_bioset(GFP_NOFS, BIO_MAX_PAGES, &iomap_ioend_bioset); | 1224 | bio = bio_alloc_bioset(GFP_NOFS, BIO_MAX_VECS, &iomap_ioend_bioset); |
| 1225 | bio_set_dev(bio, wpc->iomap.bdev); | 1225 | bio_set_dev(bio, wpc->iomap.bdev); |
| 1226 | bio->bi_iter.bi_sector = sector; | 1226 | bio->bi_iter.bi_sector = sector; |
| 1227 | bio->bi_opf = REQ_OP_WRITE | wbc_to_write_flags(wbc); | 1227 | bio->bi_opf = REQ_OP_WRITE | wbc_to_write_flags(wbc); |
| @@ -1252,7 +1252,7 @@ iomap_chain_bio(struct bio *prev) · 24 unmodified lines | |||
| 1252 | { | 1252 | { |
| 1253 | struct bio *new; | 1253 | struct bio *new; |
| 1254 | 1254 | ||
| 1255 | new = bio_alloc(GFP_NOFS, BIO_MAX_PAGES); | 1255 | new = bio_alloc(GFP_NOFS, BIO_MAX_VECS); |
| 1256 | bio_copy_dev(new, prev);/* also copies over blkcg information */ | 1256 | bio_copy_dev(new, prev);/* also copies over blkcg information */ |
| 1257 | new->bi_iter.bi_sector = bio_end_sector(prev); | 1257 | new->bi_iter.bi_sector = bio_end_sector(prev); |
| 1258 | new->bi_opf = prev->bi_opf; | 1258 | new->bi_opf = prev->bi_opf; |
| @@ -296,7 +296,7 @@ iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length, · 295 unmodified lines | |||
| 296 | */ | 296 | */ |
| 297 | bio_opf = iomap_dio_bio_opflags(dio, iomap, use_fua); | 297 | bio_opf = iomap_dio_bio_opflags(dio, iomap, use_fua); |
| 298 | 298 | ||
| 299 | nr_pages = bio_iov_vecs_to_alloc(dio->submit.iter, BIO_MAX_PAGES); | 299 | nr_pages = bio_iov_vecs_to_alloc(dio->submit.iter, BIO_MAX_VECS); |
| 300 | do { | 300 | do { |
| 301 | size_t n; | 301 | size_t n; |
| 302 | if (dio->error) { | 302 | if (dio->error) { |
| @@ -338,7 +338,7 @@ iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length, · 35 unmodified lines | |||
| 338 | copied += n; | 338 | copied += n; |
| 339 | 339 | ||
| 340 | nr_pages = bio_iov_vecs_to_alloc(dio->submit.iter, | 340 | nr_pages = bio_iov_vecs_to_alloc(dio->submit.iter, |
| 341 | BIO_MAX_PAGES); | 341 | BIO_MAX_VECS); |
| 342 | iomap_dio_submit_bio(dio, iomap, bio, pos); | 342 | iomap_dio_submit_bio(dio, iomap, bio, pos); |
| 343 | pos += n; | 343 | pos += n; |
| 344 | } while (nr_pages); | 344 | } while (nr_pages); |
| @@ -1808,9 +1808,6 @@ check_conflicting_open(struct file *filp, const long arg, int flags) · 1807 unmodified lines | |||
| 1808 | 1808 | ||
| 1809 | if (flags & FL_LAYOUT) | 1809 | if (flags & FL_LAYOUT) |
| 1810 | return 0; | 1810 | return 0; |
| 1811 | if (flags & FL_DELEG) | ||
| 1812 | /* We leave these checks to the caller. */ | ||
| 1813 | return 0; | ||
| 1814 | 1811 | ||
| 1815 | if (arg == F_RDLCK) | 1812 | if (arg == F_RDLCK) |
| 1816 | return inode_is_open_for_write(inode) ? -EAGAIN : 0; | 1813 | return inode_is_open_for_write(inode) ? -EAGAIN : 0; |
| @@ -616,7 +616,7 @@ alloc_new: · 615 unmodified lines | |||
| 616 | goto out; | 616 | goto out; |
| 617 | } | 617 | } |
| 618 | bio = mpage_alloc(bdev, blocks[0] << (blkbits - 9), | 618 | bio = mpage_alloc(bdev, blocks[0] << (blkbits - 9), |
| 619 | BIO_MAX_PAGES, GFP_NOFS|__GFP_HIGH); | 619 | BIO_MAX_VECS, GFP_NOFS|__GFP_HIGH); |
| 620 | if (bio == NULL) | 620 | if (bio == NULL) |
| 621 | goto confused; | 621 | goto confused; |
| 622 | 622 | ||
| @@ -127,7 +127,7 @@ config PNFS_BLOCK · 126 unmodified lines | |||
| 127 | config PNFS_FLEXFILE_LAYOUT | 127 | config PNFS_FLEXFILE_LAYOUT |
| 128 | tristate | 128 | tristate |
| 129 | depends on NFS_V4_1 && NFS_V3 | 129 | depends on NFS_V4_1 && NFS_V3 |
| 130 | default m | 130 | default NFS_V4 |
| 131 | 131 | ||
| 132 | config NFS_V4_1_IMPLEMENTATION_ID_DOMAIN | 132 | config NFS_V4_1_IMPLEMENTATION_ID_DOMAIN |
| 133 | string "NFSv4.1 Implementation ID Domain" | 133 | string "NFSv4.1 Implementation ID Domain" |
| @@ -81,8 +81,9 @@ static struct nfs_open_dir_context *alloc_nfs_open_dir_context(struct inode *dir · 80 unmodified lines | |||
| 81 | spin_lock(&dir->i_lock); | 81 | spin_lock(&dir->i_lock); |
| 82 | if (list_empty(&nfsi->open_files) && | 82 | if (list_empty(&nfsi->open_files) && |
| 83 | (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER)) | 83 | (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER)) |
| 84 | nfsi->cache_validity |= NFS_INO_INVALID_DATA | | 84 | nfs_set_cache_invalid(dir, |
| 85 | NFS_INO_REVAL_FORCED; | 85 | NFS_INO_INVALID_DATA | |
| 86 | NFS_INO_REVAL_FORCED); | ||
| 86 | list_add(&ctx->list, &nfsi->open_files); | 87 | list_add(&ctx->list, &nfsi->open_files); |
| 87 | spin_unlock(&dir->i_lock); | 88 | spin_unlock(&dir->i_lock); |
| 88 | return ctx; | 89 | return ctx; |
| @@ -1401,6 +1402,13 @@ out_force: · 1312 unmodified lines | |||
| 1401 | goto out; | 1402 | goto out; |
| 1402 | } | 1403 | } |
| 1403 | 1404 | ||
| 1405 | static void nfs_mark_dir_for_revalidate(struct inode *inode) | ||
| 1406 | { | ||
| 1407 | spin_lock(&inode->i_lock); | ||
| 1408 | nfs_set_cache_invalid(inode, NFS_INO_REVAL_PAGECACHE); | ||
| 1409 | spin_unlock(&inode->i_lock); | ||
| 1410 | } | ||
| 1411 | |||
| 1404 | /* | 1412 | /* |
| 1405 | * We judge how long we want to trust negative | 1413 | * We judge how long we want to trust negative |
| 1406 | * dentries by looking at the parent inode mtime. | 1414 | * dentries by looking at the parent inode mtime. |
| @@ -1435,19 +1443,14 @@ nfs_lookup_revalidate_done(struct inode *dir, struct dentry *dentry, · 28 unmodified lines | |||
| 1435 | __func__, dentry); | 1443 | __func__, dentry); |
| 1436 | return 1; | 1444 | return 1; |
| 1437 | case 0: | 1445 | case 0: |
| 1438 | nfs_mark_for_revalidate(dir); | 1446 | /* |
| 1439 | if (inode && S_ISDIR(inode->i_mode)) { | 1447 | * We can't d_drop the root of a disconnected tree: |
| 1440 | /* Purge readdir caches. */ | 1448 | * its d_hash is on the s_anon list and d_drop() would hide |
| 1441 | nfs_zap_caches(inode); | 1449 | * it from shrink_dcache_for_unmount(), leading to busy |
| 1442 | /* | 1450 | * inodes on unmount and further oopses. |
| 1443 | * We can't d_drop the root of a disconnected tree: | 1451 | */ |
| 1444 | * its d_hash is on the s_anon list and d_drop() would hide | 1452 | if (inode && IS_ROOT(dentry)) |
| 1445 | * it from shrink_dcache_for_unmount(), leading to busy | 1453 | return 1; |
| 1446 | * inodes on unmount and further oopses. | ||
| 1447 | */ | ||
| 1448 | if (IS_ROOT(dentry)) | ||
| 1449 | return 1; | ||
| 1450 | } | ||
| 1451 | dfprintk(LOOKUPCACHE, "NFS: %s(%pd2) is invalid\n", | 1454 | dfprintk(LOOKUPCACHE, "NFS: %s(%pd2) is invalid\n", |
| 1452 | __func__, dentry); | 1455 | __func__, dentry); |
| 1453 | return 0; | 1456 | return 0; |
| @@ -1525,6 +1528,13 @@ out: · 71 unmodified lines | |||
| 1525 | nfs_free_fattr(fattr); | 1528 | nfs_free_fattr(fattr); |
| 1526 | nfs_free_fhandle(fhandle); | 1529 | nfs_free_fhandle(fhandle); |
| 1527 | nfs4_label_free(label); | 1530 | nfs4_label_free(label); |
| 1531 | |||
| 1532 | /* | ||
| 1533 | * If the lookup failed despite the dentry change attribute being | ||
| 1534 | * a match, then we should revalidate the directory cache. | ||
| 1535 | */ | ||
| 1536 | if (!ret && nfs_verify_change_attribute(dir, dentry->d_time)) | ||
| 1537 | nfs_mark_dir_for_revalidate(dir); | ||
| 1528 | return nfs_lookup_revalidate_done(dir, dentry, inode, ret); | 1538 | return nfs_lookup_revalidate_done(dir, dentry, inode, ret); |
| 1529 | } | 1539 | } |
| 1530 | 1540 | ||
| @@ -1567,7 +1577,7 @@ nfs_do_lookup_revalidate(struct inode *dir, struct dentry *dentry, · 36 unmodified lines | |||
| 1567 | error = nfs_lookup_verify_inode(inode, flags); | 1577 | error = nfs_lookup_verify_inode(inode, flags); |
| 1568 | if (error) { | 1578 | if (error) { |
| 1569 | if (error == -ESTALE) | 1579 | if (error == -ESTALE) |
| 1570 | nfs_zap_caches(dir); | 1580 | nfs_mark_dir_for_revalidate(dir); |
| 1571 | goto out_bad; | 1581 | goto out_bad; |
| 1572 | } | 1582 | } |
| 1573 | nfs_advise_use_readdirplus(dir); | 1583 | nfs_advise_use_readdirplus(dir); |
| @@ -1691,10 +1701,9 @@ static void nfs_drop_nlink(struct inode *inode) · 117 unmodified lines | |||
| 1691 | if (inode->i_nlink > 0) | 1701 | if (inode->i_nlink > 0) |
| 1692 | drop_nlink(inode); | 1702 | drop_nlink(inode); |
| 1693 | NFS_I(inode)->attr_gencount = nfs_inc_attr_generation_counter(); | 1703 | NFS_I(inode)->attr_gencount = nfs_inc_attr_generation_counter(); |
| 1694 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_CHANGE | 1704 | nfs_set_cache_invalid( |
| 1695 | | NFS_INO_INVALID_CTIME | 1705 | inode, NFS_INO_INVALID_CHANGE | NFS_INO_INVALID_CTIME | |
| 1696 | | NFS_INO_INVALID_OTHER | 1706 | NFS_INO_INVALID_OTHER | NFS_INO_REVAL_FORCED); |
| 1697 | | NFS_INO_REVAL_FORCED; | ||
| 1698 | spin_unlock(&inode->i_lock); | 1707 | spin_unlock(&inode->i_lock); |
| 1699 | } | 1708 | } |
| 1700 | 1709 | ||
| @@ -1706,7 +1715,7 @@ static void nfs_dentry_iput(struct dentry *dentry, struct inode *inode) · 5 unmodified lines | |||
| 1706 | { | 1715 | { |
| 1707 | if (S_ISDIR(inode->i_mode)) | 1716 | if (S_ISDIR(inode->i_mode)) |
| 1708 | /* drop any readdir cache as it could easily be old */ | 1717 | /* drop any readdir cache as it could easily be old */ |
| 1709 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_DATA; | 1718 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_DATA); |
| 1710 | 1719 | ||
| 1711 | if (dentry->d_flags & DCACHE_NFSFS_RENAMED) { | 1720 | if (dentry->d_flags & DCACHE_NFSFS_RENAMED) { |
| 1712 | nfs_complete_unlink(dentry, inode); | 1721 | nfs_complete_unlink(dentry, inode); |
| @@ -2064,7 +2073,6 @@ out: · 351 unmodified lines | |||
| 2064 | dput(parent); | 2073 | dput(parent); |
| 2065 | return d; | 2074 | return d; |
| 2066 | out_error: | 2075 | out_error: |
| 2067 | nfs_mark_for_revalidate(dir); | ||
| 2068 | d = ERR_PTR(error); | 2076 | d = ERR_PTR(error); |
| 2069 | goto out; | 2077 | goto out; |
| 2070 | } | 2078 | } |
| @@ -2473,9 +2481,9 @@ int nfs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, · 402 unmodified lines | |||
| 2473 | if (error == 0) { | 2481 | if (error == 0) { |
| 2474 | spin_lock(&old_inode->i_lock); | 2482 | spin_lock(&old_inode->i_lock); |
| 2475 | NFS_I(old_inode)->attr_gencount = nfs_inc_attr_generation_counter(); | 2483 | NFS_I(old_inode)->attr_gencount = nfs_inc_attr_generation_counter(); |
| 2476 | NFS_I(old_inode)->cache_validity |= NFS_INO_INVALID_CHANGE | 2484 | nfs_set_cache_invalid(old_inode, NFS_INO_INVALID_CHANGE | |
| 2477 | | NFS_INO_INVALID_CTIME | 2485 | NFS_INO_INVALID_CTIME | |
| 2478 | | NFS_INO_REVAL_FORCED; | 2486 | NFS_INO_REVAL_FORCED); |
| 2479 | spin_unlock(&old_inode->i_lock); | 2487 | spin_unlock(&old_inode->i_lock); |
| 2480 | } | 2488 | } |
| 2481 | out: | 2489 | out: |
| @@ -207,7 +207,7 @@ static bool nfs_has_xattr_cache(const struct nfs_inode *nfsi) · 206 unmodified lines | |||
| 207 | } | 207 | } |
| 208 | #endif | 208 | #endif |
| 209 | 209 | ||
| 210 | static void nfs_set_cache_invalid(struct inode *inode, unsigned long flags) | 210 | void nfs_set_cache_invalid(struct inode *inode, unsigned long flags) |
| 211 | { | 211 | { |
| 212 | struct nfs_inode *nfsi = NFS_I(inode); | 212 | struct nfs_inode *nfsi = NFS_I(inode); |
| 213 | bool have_delegation = NFS_PROTO(inode)->have_delegation(inode, FMODE_READ); | 213 | bool have_delegation = NFS_PROTO(inode)->have_delegation(inode, FMODE_READ); |
| @@ -229,6 +229,7 @@ static void nfs_set_cache_invalid(struct inode *inode, unsigned long flags) · 15 unmodified lines | |||
| 229 | if (flags & NFS_INO_INVALID_DATA) | 229 | if (flags & NFS_INO_INVALID_DATA) |
| 230 | nfs_fscache_invalidate(inode); | 230 | nfs_fscache_invalidate(inode); |
| 231 | } | 231 | } |
| 232 | EXPORT_SYMBOL_GPL(nfs_set_cache_invalid); | ||
| 232 | 233 | ||
| 233 | /* | 234 | /* |
| 234 | * Invalidate the local caches | 235 | * Invalidate the local caches |
| @@ -1067,8 +1068,8 @@ void nfs_inode_attach_open_context(struct nfs_open_context *ctx) · 832 unmodified lines | |||
| 1067 | spin_lock(&inode->i_lock); | 1068 | spin_lock(&inode->i_lock); |
| 1068 | if (list_empty(&nfsi->open_files) && | 1069 | if (list_empty(&nfsi->open_files) && |
| 1069 | (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER)) | 1070 | (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER)) |
| 1070 | nfsi->cache_validity |= NFS_INO_INVALID_DATA | | 1071 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_DATA | |
| 1071 | NFS_INO_REVAL_FORCED; | 1072 | NFS_INO_REVAL_FORCED); |
| 1072 | list_add_tail_rcu(&ctx->list, &nfsi->open_files); | 1073 | list_add_tail_rcu(&ctx->list, &nfsi->open_files); |
| 1073 | spin_unlock(&inode->i_lock); | 1074 | spin_unlock(&inode->i_lock); |
| 1074 | } | 1075 | } |
| @@ -411,7 +411,8 @@ extern int nfs_write_inode(struct inode *, struct writeback_control *); · 410 unmodified lines | |||
| 411 | extern int nfs_drop_inode(struct inode *); | 411 | extern int nfs_drop_inode(struct inode *); |
| 412 | extern void nfs_clear_inode(struct inode *); | 412 | extern void nfs_clear_inode(struct inode *); |
| 413 | extern void nfs_evict_inode(struct inode *); | 413 | extern void nfs_evict_inode(struct inode *); |
| 414 | void nfs_zap_acl_cache(struct inode *inode); | 414 | extern void nfs_zap_acl_cache(struct inode *inode); |
| 415 | extern void nfs_set_cache_invalid(struct inode *inode, unsigned long flags); | ||
| 415 | extern bool nfs_check_cache_invalid(struct inode *, unsigned long); | 416 | extern bool nfs_check_cache_invalid(struct inode *, unsigned long); |
| 416 | extern int nfs_wait_bit_killable(struct wait_bit_key *key, int mode); | 417 | extern int nfs_wait_bit_killable(struct wait_bit_key *key, int mode); |
| 417 | extern int nfs_wait_atomic_killable(atomic_t *p, unsigned int mode); | 418 | extern int nfs_wait_atomic_killable(atomic_t *p, unsigned int mode); |
| @@ -36,6 +36,7 @@ · 35 unmodified lines | |||
| 36 | #define NFS3_pagepad_sz (1) /* Page padding */ | 36 | #define NFS3_pagepad_sz (1) /* Page padding */ |
| 37 | #define NFS3_fhandle_sz (1+16) | 37 | #define NFS3_fhandle_sz (1+16) |
| 38 | #define NFS3_fh_sz (NFS3_fhandle_sz) /* shorthand */ | 38 | #define NFS3_fh_sz (NFS3_fhandle_sz) /* shorthand */ |
| 39 | #define NFS3_post_op_fh_sz (1+NFS3_fh_sz) | ||
| 39 | #define NFS3_sattr_sz (15) | 40 | #define NFS3_sattr_sz (15) |
| 40 | #define NFS3_filename_sz (1+(NFS3_MAXNAMLEN>>2)) | 41 | #define NFS3_filename_sz (1+(NFS3_MAXNAMLEN>>2)) |
| 41 | #define NFS3_path_sz (1+(NFS3_MAXPATHLEN>>2)) | 42 | #define NFS3_path_sz (1+(NFS3_MAXPATHLEN>>2)) |
| @@ -73,7 +74,7 @@ · 31 unmodified lines | |||
| 73 | #define NFS3_readlinkres_sz (1+NFS3_post_op_attr_sz+1+NFS3_pagepad_sz) | 74 | #define NFS3_readlinkres_sz (1+NFS3_post_op_attr_sz+1+NFS3_pagepad_sz) |
| 74 | #define NFS3_readres_sz (1+NFS3_post_op_attr_sz+3+NFS3_pagepad_sz) | 75 | #define NFS3_readres_sz (1+NFS3_post_op_attr_sz+3+NFS3_pagepad_sz) |
| 75 | #define NFS3_writeres_sz (1+NFS3_wcc_data_sz+4) | 76 | #define NFS3_writeres_sz (1+NFS3_wcc_data_sz+4) |
| 76 | #define NFS3_createres_sz (1+NFS3_fh_sz+NFS3_post_op_attr_sz+NFS3_wcc_data_sz) | 77 | #define NFS3_createres_sz (1+NFS3_post_op_fh_sz+NFS3_post_op_attr_sz+NFS3_wcc_data_sz) |
| 77 | #define NFS3_renameres_sz (1+(2 * NFS3_wcc_data_sz)) | 78 | #define NFS3_renameres_sz (1+(2 * NFS3_wcc_data_sz)) |
| 78 | #define NFS3_linkres_sz (1+NFS3_post_op_attr_sz+NFS3_wcc_data_sz) | 79 | #define NFS3_linkres_sz (1+NFS3_post_op_attr_sz+NFS3_wcc_data_sz) |
| 79 | #define NFS3_readdirres_sz (1+NFS3_post_op_attr_sz+2+NFS3_pagepad_sz) | 80 | #define NFS3_readdirres_sz (1+NFS3_post_op_attr_sz+2+NFS3_pagepad_sz) |
| @@ -357,13 +357,15 @@ static ssize_t _nfs42_proc_copy(struct file *src, · 356 unmodified lines | |||
| 357 | truncate_pagecache_range(dst_inode, pos_dst, | 357 | truncate_pagecache_range(dst_inode, pos_dst, |
| 358 | pos_dst + res->write_res.count); | 358 | pos_dst + res->write_res.count); |
| 359 | spin_lock(&dst_inode->i_lock); | 359 | spin_lock(&dst_inode->i_lock); |
| 360 | NFS_I(dst_inode)->cache_validity |= (NFS_INO_REVAL_PAGECACHE | | 360 | nfs_set_cache_invalid( |
| 361 | NFS_INO_REVAL_FORCED | NFS_INO_INVALID_SIZE | | 361 | dst_inode, NFS_INO_REVAL_PAGECACHE | NFS_INO_REVAL_FORCED | |
| 362 | NFS_INO_INVALID_ATTR | NFS_INO_INVALID_DATA); | 362 | NFS_INO_INVALID_SIZE | NFS_INO_INVALID_ATTR | |
| 363 | NFS_INO_INVALID_DATA); | ||
| 363 | spin_unlock(&dst_inode->i_lock); | 364 | spin_unlock(&dst_inode->i_lock); |
| 364 | spin_lock(&src_inode->i_lock); | 365 | spin_lock(&src_inode->i_lock); |
| 365 | NFS_I(src_inode)->cache_validity |= (NFS_INO_REVAL_PAGECACHE | | 366 | nfs_set_cache_invalid(src_inode, NFS_INO_REVAL_PAGECACHE | |
| 366 | NFS_INO_REVAL_FORCED | NFS_INO_INVALID_ATIME); | 367 | NFS_INO_REVAL_FORCED | |
| 368 | NFS_INO_INVALID_ATIME); | ||
| 367 | spin_unlock(&src_inode->i_lock); | 369 | spin_unlock(&src_inode->i_lock); |
| 368 | status = res->write_res.count; | 370 | status = res->write_res.count; |
| 369 | out: | 371 | out: |
| @@ -1169,14 +1169,14 @@ int nfs4_call_sync(struct rpc_clnt *clnt, · 1168 unmodified lines | |||
| 1169 | static void | 1169 | static void |
| 1170 | nfs4_inc_nlink_locked(struct inode *inode) | 1170 | nfs4_inc_nlink_locked(struct inode *inode) |
| 1171 | { | 1171 | { |
| 1172 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_OTHER; | 1172 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_OTHER); |
| 1173 | inc_nlink(inode); | 1173 | inc_nlink(inode); |
| 1174 | } | 1174 | } |
| 1175 | 1175 | ||
| 1176 | static void | 1176 | static void |
| 1177 | nfs4_dec_nlink_locked(struct inode *inode) | 1177 | nfs4_dec_nlink_locked(struct inode *inode) |
| 1178 | { | 1178 | { |
| 1179 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_OTHER; | 1179 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_OTHER); |
| 1180 | drop_nlink(inode); | 1180 | drop_nlink(inode); |
| 1181 | } | 1181 | } |
| 1182 | 1182 | ||
| @@ -1187,35 +1187,31 @@ nfs4_update_changeattr_locked(struct inode *inode, · 4 unmodified lines | |||
| 1187 | { | 1187 | { |
| 1188 | struct nfs_inode *nfsi = NFS_I(inode); | 1188 | struct nfs_inode *nfsi = NFS_I(inode); |
| 1189 | 1189 | ||
| 1190 | nfsi->cache_validity |= NFS_INO_INVALID_CTIME | 1190 | cache_validity |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME; |
| 1191 | | NFS_INO_INVALID_MTIME | ||
| 1192 | | cache_validity; | ||
| 1193 | 1191 | ||
| 1194 | if (cinfo->atomic && cinfo->before == inode_peek_iversion_raw(inode)) { | 1192 | if (cinfo->atomic && cinfo->before == inode_peek_iversion_raw(inode)) { |
| 1195 | nfsi->cache_validity &= ~NFS_INO_REVAL_PAGECACHE; | 1193 | nfsi->cache_validity &= ~NFS_INO_REVAL_PAGECACHE; |
| 1196 | nfsi->attrtimeo_timestamp = jiffies; | 1194 | nfsi->attrtimeo_timestamp = jiffies; |
| 1197 | } else { | 1195 | } else { |
| 1198 | if (S_ISDIR(inode->i_mode)) { | 1196 | if (S_ISDIR(inode->i_mode)) { |
| 1199 | nfsi->cache_validity |= NFS_INO_INVALID_DATA; | 1197 | cache_validity |= NFS_INO_INVALID_DATA; |
| 1200 | nfs_force_lookup_revalidate(inode); | 1198 | nfs_force_lookup_revalidate(inode); |
| 1201 | } else { | 1199 | } else { |
| 1202 | if (!NFS_PROTO(inode)->have_delegation(inode, | 1200 | if (!NFS_PROTO(inode)->have_delegation(inode, |
| 1203 | FMODE_READ)) | 1201 | FMODE_READ)) |
| 1204 | nfsi->cache_validity |= NFS_INO_REVAL_PAGECACHE; | 1202 | cache_validity |= NFS_INO_REVAL_PAGECACHE; |
| 1205 | } | 1203 | } |
| 1206 | 1204 | ||
| 1207 | if (cinfo->before != inode_peek_iversion_raw(inode)) | 1205 | if (cinfo->before != inode_peek_iversion_raw(inode)) |
| 1208 | nfsi->cache_validity |= NFS_INO_INVALID_ACCESS | | 1206 | cache_validity |= NFS_INO_INVALID_ACCESS | |
| 1209 | NFS_INO_INVALID_ACL | | 1207 | NFS_INO_INVALID_ACL | |
| 1210 | NFS_INO_INVALID_XATTR; | 1208 | NFS_INO_INVALID_XATTR; |
| 1211 | } | 1209 | } |
| 1212 | inode_set_iversion_raw(inode, cinfo->after); | 1210 | inode_set_iversion_raw(inode, cinfo->after); |
| 1213 | nfsi->read_cache_jiffies = timestamp; | 1211 | nfsi->read_cache_jiffies = timestamp; |
| 1214 | nfsi->attr_gencount = nfs_inc_attr_generation_counter(); | 1212 | nfsi->attr_gencount = nfs_inc_attr_generation_counter(); |
| 1213 | nfs_set_cache_invalid(inode, cache_validity); | ||
| 1215 | nfsi->cache_validity &= ~NFS_INO_INVALID_CHANGE; | 1214 | nfsi->cache_validity &= ~NFS_INO_INVALID_CHANGE; |
| 1216 | |||
| 1217 | if (nfsi->cache_validity & NFS_INO_INVALID_DATA) | ||
| 1218 | nfs_fscache_invalidate(inode); | ||
| 1219 | } | 1215 | } |
| 1220 | 1216 | ||
| 1221 | void | 1217 | void |
| @@ -5893,6 +5889,9 @@ static int __nfs4_proc_set_acl(struct inode *inode, const void *buf, size_t bufl · 4671 unmodified lines | |||
| 5893 | unsigned int npages = DIV_ROUND_UP(buflen, PAGE_SIZE); | 5889 | unsigned int npages = DIV_ROUND_UP(buflen, PAGE_SIZE); |
| 5894 | int ret, i; | 5890 | int ret, i; |
| 5895 | 5891 | ||
| 5892 | /* You can't remove system.nfs4_acl: */ | ||
| 5893 | if (buflen == 0) | ||
| 5894 | return -EINVAL; | ||
| 5896 | if (!nfs4_server_supports_acls(server)) | 5895 | if (!nfs4_server_supports_acls(server)) |
| 5897 | return -EOPNOTSUPP; | 5896 | return -EOPNOTSUPP; |
| 5898 | if (npages > ARRAY_SIZE(pages)) | 5897 | if (npages > ARRAY_SIZE(pages)) |
| @@ -5915,9 +5914,9 @@ static int __nfs4_proc_set_acl(struct inode *inode, const void *buf, size_t bufl · 16 unmodified lines | |||
| 5915 | * so mark the attribute cache invalid. | 5914 | * so mark the attribute cache invalid. |
| 5916 | */ | 5915 | */ |
| 5917 | spin_lock(&inode->i_lock); | 5916 | spin_lock(&inode->i_lock); |
| 5918 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_CHANGE | 5917 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_CHANGE | |
| 5919 | | NFS_INO_INVALID_CTIME | 5918 | NFS_INO_INVALID_CTIME | |
| 5920 | | NFS_INO_REVAL_FORCED; | 5919 | NFS_INO_REVAL_FORCED); |
| 5921 | spin_unlock(&inode->i_lock); | 5920 | spin_unlock(&inode->i_lock); |
| 5922 | nfs_access_zap_cache(inode); | 5921 | nfs_access_zap_cache(inode); |
| 5923 | nfs_zap_acl_cache(inode); | 5922 | nfs_zap_acl_cache(inode); |
| @@ -5969,7 +5968,7 @@ static int _nfs4_get_security_label(struct inode *inode, void *buf, · 45 unmodified lines | |||
| 5969 | return ret; | 5968 | return ret; |
| 5970 | if (!(fattr.valid & NFS_ATTR_FATTR_V4_SECURITY_LABEL)) | 5969 | if (!(fattr.valid & NFS_ATTR_FATTR_V4_SECURITY_LABEL)) |
| 5971 | return -ENOENT; | 5970 | return -ENOENT; |
| 5972 | return 0; | 5971 | return label.len; |
| 5973 | } | 5972 | } |
| 5974 | 5973 | ||
| 5975 | static int nfs4_get_security_label(struct inode *inode, void *buf, | 5974 | static int nfs4_get_security_label(struct inode *inode, void *buf, |
| @@ -500,9 +500,9 @@ nfs_sillyrename(struct inode *dir, struct dentry *dentry) · 499 unmodified lines | |||
| 500 | nfs_set_verifier(dentry, nfs_save_change_attribute(dir)); | 500 | nfs_set_verifier(dentry, nfs_save_change_attribute(dir)); |
| 501 | spin_lock(&inode->i_lock); | 501 | spin_lock(&inode->i_lock); |
| 502 | NFS_I(inode)->attr_gencount = nfs_inc_attr_generation_counter(); | 502 | NFS_I(inode)->attr_gencount = nfs_inc_attr_generation_counter(); |
| 503 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_CHANGE | 503 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_CHANGE | |
| 504 | | NFS_INO_INVALID_CTIME | 504 | NFS_INO_INVALID_CTIME | |
| 505 | | NFS_INO_REVAL_FORCED; | 505 | NFS_INO_REVAL_FORCED); |
| 506 | spin_unlock(&inode->i_lock); | 506 | spin_unlock(&inode->i_lock); |
| 507 | d_move(dentry, sdentry); | 507 | d_move(dentry, sdentry); |
| 508 | break; | 508 | break; |
| @@ -303,9 +303,9 @@ static void nfs_set_pageerror(struct address_space *mapping) · 302 unmodified lines | |||
| 303 | nfs_zap_mapping(mapping->host, mapping); | 303 | nfs_zap_mapping(mapping->host, mapping); |
| 304 | /* Force file size revalidation */ | 304 | /* Force file size revalidation */ |
| 305 | spin_lock(&inode->i_lock); | 305 | spin_lock(&inode->i_lock); |
| 306 | NFS_I(inode)->cache_validity |= NFS_INO_REVAL_FORCED | | 306 | nfs_set_cache_invalid(inode, NFS_INO_REVAL_FORCED | |
| 307 | NFS_INO_REVAL_PAGECACHE | | 307 | NFS_INO_REVAL_PAGECACHE | |
| 308 | NFS_INO_INVALID_SIZE; | 308 | NFS_INO_INVALID_SIZE); |
| 309 | spin_unlock(&inode->i_lock); | 309 | spin_unlock(&inode->i_lock); |
| 310 | } | 310 | } |
| 311 | 311 | ||
| @@ -1604,7 +1604,7 @@ static int nfs_writeback_done(struct rpc_task *task, · 1292 unmodified lines | |||
| 1604 | /* Deal with the suid/sgid bit corner case */ | 1604 | /* Deal with the suid/sgid bit corner case */ |
| 1605 | if (nfs_should_remove_suid(inode)) { | 1605 | if (nfs_should_remove_suid(inode)) { |
| 1606 | spin_lock(&inode->i_lock); | 1606 | spin_lock(&inode->i_lock); |
| 1607 | NFS_I(inode)->cache_validity |= NFS_INO_INVALID_OTHER; | 1607 | nfs_set_cache_invalid(inode, NFS_INO_INVALID_OTHER); |
| 1608 | spin_unlock(&inode->i_lock); | 1608 | spin_unlock(&inode->i_lock); |
| 1609 | } | 1609 | } |
| 1610 | return 0; | 1610 | return 0; |
| @@ -73,6 +73,7 @@ config NFSD_V4 · 72 unmodified lines | |||
| 73 | select NFSD_V3 | 73 | select NFSD_V3 |
| 74 | select FS_POSIX_ACL | 74 | select FS_POSIX_ACL |
| 75 | select SUNRPC_GSS | 75 | select SUNRPC_GSS |
| 76 | select CRYPTO | ||
| 76 | select CRYPTO_MD5 | 77 | select CRYPTO_MD5 |
| 77 | select CRYPTO_SHA256 | 78 | select CRYPTO_SHA256 |
| 78 | select GRACE_PERIOD | 79 | select GRACE_PERIOD |
| @@ -898,6 +898,8 @@ nfsd_file_find_locked(struct inode *inode, unsigned int may_flags, · 897 unmodified lines | |||
| 898 | continue; | 898 | continue; |
| 899 | if (!nfsd_match_cred(nf->nf_cred, current_cred())) | 899 | if (!nfsd_match_cred(nf->nf_cred, current_cred())) |
| 900 | continue; | 900 | continue; |
| 901 | if (!test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) | ||
| 902 | continue; | ||
| 901 | if (nfsd_file_get(nf) != NULL) | 903 | if (nfsd_file_get(nf) != NULL) |
| 902 | return nf; | 904 | return nf; |
| 903 | } | 905 | } |
| @@ -1189,6 +1189,7 @@ static void nfsd4_cb_done(struct rpc_task *task, void *calldata) · 1188 unmodified lines | |||
| 1189 | switch (task->tk_status) { | 1189 | switch (task->tk_status) { |
| 1190 | case -EIO: | 1190 | case -EIO: |
| 1191 | case -ETIMEDOUT: | 1191 | case -ETIMEDOUT: |
| 1192 | case -EACCES: | ||
| 1192 | nfsd4_mark_cb_down(clp, task->tk_status); | 1193 | nfsd4_mark_cb_down(clp, task->tk_status); |
| 1193 | } | 1194 | } |
| 1194 | break; | 1195 | break; |
| @@ -1302,7 +1302,7 @@ nfsd4_cleanup_inter_ssc(struct vfsmount *ss_mnt, struct nfsd_file *src, · 1301 unmodified lines | |||
| 1302 | struct nfsd_file *dst) | 1302 | struct nfsd_file *dst) |
| 1303 | { | 1303 | { |
| 1304 | nfs42_ssc_close(src->nf_file); | 1304 | nfs42_ssc_close(src->nf_file); |
| 1305 | /* 'src' is freed by nfsd4_do_async_copy */ | 1305 | fput(src->nf_file); |
| 1306 | nfsd_file_put(dst); | 1306 | nfsd_file_put(dst); |
| 1307 | mntput(ss_mnt); | 1307 | mntput(ss_mnt); |
| 1308 | } | 1308 | } |
| @@ -4940,31 +4940,6 @@ static struct file_lock *nfs4_alloc_init_lease(struct nfs4_delegation *dp, · 4939 unmodified lines | |||
| 4940 | return fl; | 4940 | return fl; |
| 4941 | } | 4941 | } |
| 4942 | 4942 | ||
| 4943 | static int nfsd4_check_conflicting_opens(struct nfs4_client *clp, | ||
| 4944 | struct nfs4_file *fp) | ||
| 4945 | { | ||
| 4946 | struct nfs4_clnt_odstate *co; | ||
| 4947 | struct file *f = fp->fi_deleg_file->nf_file; | ||
| 4948 | struct inode *ino = locks_inode(f); | ||
| 4949 | int writes = atomic_read(&ino->i_writecount); | ||
| 4950 | |||
| 4951 | if (fp->fi_fds[O_WRONLY]) | ||
| 4952 | writes--; | ||
| 4953 | if (fp->fi_fds[O_RDWR]) | ||
| 4954 | writes--; | ||
| 4955 | if (writes > 0) | ||
| 4956 | return -EAGAIN; | ||
| 4957 | spin_lock(&fp->fi_lock); | ||
| 4958 | list_for_each_entry(co, &fp->fi_clnt_odstate, co_perfile) { | ||
| 4959 | if (co->co_client != clp) { | ||
| 4960 | spin_unlock(&fp->fi_lock); | ||
| 4961 | return -EAGAIN; | ||
| 4962 | } | ||
| 4963 | } | ||
| 4964 | spin_unlock(&fp->fi_lock); | ||
| 4965 | return 0; | ||
| 4966 | } | ||
| 4967 | |||
| 4968 | static struct nfs4_delegation * | 4943 | static struct nfs4_delegation * |
| 4969 | nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh, | 4944 | nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh, |
| 4970 | struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate) | 4945 | struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate) |
| @@ -4984,12 +4959,9 @@ nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh, · 13 unmodified lines | |||
| 4984 | 4959 | ||
| 4985 | nf = find_readable_file(fp); | 4960 | nf = find_readable_file(fp); |
| 4986 | if (!nf) { | 4961 | if (!nf) { |
| 4987 | /* | 4962 | /* We should always have a readable file here */ |
| 4988 | * We probably could attempt another open and get a read | 4963 | WARN_ON_ONCE(1); |
| 4989 | * delegation, but for now, don't bother until the | 4964 | return ERR_PTR(-EBADF); |
| 4990 | * client actually sends us one. | ||
| 4991 | */ | ||
| 4992 | return ERR_PTR(-EAGAIN); | ||
| 4993 | } | 4965 | } |
| 4994 | spin_lock(&state_lock); | 4966 | spin_lock(&state_lock); |
| 4995 | spin_lock(&fp->fi_lock); | 4967 | spin_lock(&fp->fi_lock); |
| @@ -5019,19 +4991,11 @@ nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh, · 23 unmodified lines | |||
| 5019 | if (!fl) | 4991 | if (!fl) |
| 5020 | goto out_clnt_odstate; | 4992 | goto out_clnt_odstate; |
| 5021 | 4993 | ||
| 5022 | status = nfsd4_check_conflicting_opens(clp, fp); | ||
| 5023 | if (status) { | ||
| 5024 | locks_free_lock(fl); | ||
| 5025 | goto out_clnt_odstate; | ||
| 5026 | } | ||
| 5027 | status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL); | 4994 | status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL); |
| 5028 | if (fl) | 4995 | if (fl) |
| 5029 | locks_free_lock(fl); | 4996 | locks_free_lock(fl); |
| 5030 | if (status) | 4997 | if (status) |
| 5031 | goto out_clnt_odstate; | 4998 | goto out_clnt_odstate; |
| 5032 | status = nfsd4_check_conflicting_opens(clp, fp); | ||
| 5033 | if (status) | ||
| 5034 | goto out_clnt_odstate; | ||
| 5035 | 4999 | ||
| 5036 | spin_lock(&state_lock); | 5000 | spin_lock(&state_lock); |
| 5037 | spin_lock(&fp->fi_lock); | 5001 | spin_lock(&fp->fi_lock); |
| @@ -5113,6 +5077,17 @@ nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open, · 75 unmodified lines | |||
| 5113 | goto out_no_deleg; | 5077 | goto out_no_deleg; |
| 5114 | if (!cb_up || !(oo->oo_flags & NFS4_OO_CONFIRMED)) | 5078 | if (!cb_up || !(oo->oo_flags & NFS4_OO_CONFIRMED)) |
| 5115 | goto out_no_deleg; | 5079 | goto out_no_deleg; |
| 5080 | /* | ||
| 5081 | * Also, if the file was opened for write or | ||
| 5082 | * create, there's a good chance the client's | ||
| 5083 | * about to write to it, resulting in an | ||
| 5084 | * immediate recall (since we don't support | ||
| 5085 | * write delegations): | ||
| 5086 | */ | ||
| 5087 | if (open->op_share_access & NFS4_SHARE_ACCESS_WRITE) | ||
| 5088 | goto out_no_deleg; | ||
| 5089 | if (open->op_create == NFS4_OPEN_CREATE) | ||
| 5090 | goto out_no_deleg; | ||
| 5116 | break; | 5091 | break; |
| 5117 | default: | 5092 | default: |
| 5118 | goto out_no_deleg; | 5093 | goto out_no_deleg; |
| @@ -5389,7 +5364,7 @@ nfs4_laundromat(struct nfsd_net *nn) · 270 unmodified lines | |||
| 5389 | idr_for_each_entry(&nn->s2s_cp_stateids, cps_t, i) { | 5364 | idr_for_each_entry(&nn->s2s_cp_stateids, cps_t, i) { |
| 5390 | cps = container_of(cps_t, struct nfs4_cpntf_state, cp_stateid); | 5365 | cps = container_of(cps_t, struct nfs4_cpntf_state, cp_stateid); |
| 5391 | if (cps->cp_stateid.sc_type == NFS4_COPYNOTIFY_STID && | 5366 | if (cps->cp_stateid.sc_type == NFS4_COPYNOTIFY_STID && |
| 5392 | cps->cpntf_time > cutoff) | 5367 | cps->cpntf_time < cutoff) |
| 5393 | _free_cpntf_state_locked(nn, cps); | 5368 | _free_cpntf_state_locked(nn, cps); |
| 5394 | } | 5369 | } |
| 5395 | spin_unlock(&nn->s2s_cp_lock); | 5370 | spin_unlock(&nn->s2s_cp_lock); |
| @@ -399,7 +399,7 @@ static void nilfs_segbuf_prepare_write(struct nilfs_segment_buffer *segbuf, · 398 unmodified lines | |||
| 399 | { | 399 | { |
| 400 | wi->bio = NULL; | 400 | wi->bio = NULL; |
| 401 | wi->rest_blocks = segbuf->sb_sum.nblocks; | 401 | wi->rest_blocks = segbuf->sb_sum.nblocks; |
| 402 | wi->max_pages = BIO_MAX_PAGES; | 402 | wi->max_pages = BIO_MAX_VECS; |
| 403 | wi->nr_vecs = min(wi->max_pages, wi->rest_blocks); | 403 | wi->nr_vecs = min(wi->max_pages, wi->rest_blocks); |
| 404 | wi->start = wi->end = 0; | 404 | wi->start = wi->end = 0; |
| 405 | wi->blocknr = segbuf->sb_pseg_start; | 405 | wi->blocknr = segbuf->sb_pseg_start; |
| @@ -12,7 +12,7 @@ · 11 unmodified lines | |||
| 12 | 12 | ||
| 13 | #define IS_MNT_SHARED(m) ((m)->mnt.mnt_flags & MNT_SHARED) | 13 | #define IS_MNT_SHARED(m) ((m)->mnt.mnt_flags & MNT_SHARED) |
| 14 | #define IS_MNT_SLAVE(m) ((m)->mnt_master) | 14 | #define IS_MNT_SLAVE(m) ((m)->mnt_master) |
| 15 | #define IS_MNT_NEW(m) (!(m)->mnt_ns) | 15 | #define IS_MNT_NEW(m) (!(m)->mnt_ns || is_anon_ns((m)->mnt_ns)) |
| 16 | #define CLEAR_MNT_SHARED(m) ((m)->mnt.mnt_flags &= ~MNT_SHARED) | 16 | #define CLEAR_MNT_SHARED(m) ((m)->mnt.mnt_flags &= ~MNT_SHARED) |
| 17 | #define IS_MNT_UNBINDABLE(m) ((m)->mnt.mnt_flags & MNT_UNBINDABLE) | 17 | #define IS_MNT_UNBINDABLE(m) ((m)->mnt.mnt_flags & MNT_UNBINDABLE) |
| 18 | #define IS_MNT_MARKED(m) ((m)->mnt.mnt_flags & MNT_MARKED) | 18 | #define IS_MNT_MARKED(m) ((m)->mnt.mnt_flags & MNT_MARKED) |
| @@ -1036,8 +1036,6 @@ struct clear_refs_private { · 1035 unmodified lines | |||
| 1036 | 1036 | ||
| 1037 | #ifdef CONFIG_MEM_SOFT_DIRTY | 1037 | #ifdef CONFIG_MEM_SOFT_DIRTY |
| 1038 | 1038 | ||
| 1039 | #define is_cow_mapping(flags) (((flags) & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE) | ||
| 1040 | |||
| 1041 | static inline bool pte_is_pinned(struct vm_area_struct *vma, unsigned long addr, pte_t pte) | 1039 | static inline bool pte_is_pinned(struct vm_area_struct *vma, unsigned long addr, pte_t pte) |
| 1042 | { | 1040 | { |
| 1043 | struct page *page; | 1041 | struct page *page; |
| @@ -87,7 +87,7 @@ static int squashfs_bio_read(struct super_block *sb, u64 index, int length, · 86 unmodified lines | |||
| 87 | int error, i; | 87 | int error, i; |
| 88 | struct bio *bio; | 88 | struct bio *bio; |
| 89 | 89 | ||
| 90 | if (page_count <= BIO_MAX_PAGES) | 90 | if (page_count <= BIO_MAX_VECS) |
| 91 | bio = bio_alloc(GFP_NOIO, page_count); | 91 | bio = bio_alloc(GFP_NOIO, page_count); |
| 92 | else | 92 | else |
| 93 | bio = bio_kmalloc(GFP_NOIO, page_count); | 93 | bio = bio_kmalloc(GFP_NOIO, page_count); |
| @@ -684,7 +684,7 @@ static ssize_t zonefs_file_dio_append(struct kiocb *iocb, struct iov_iter *from) · 683 unmodified lines | |||
| 684 | max = ALIGN_DOWN(max << SECTOR_SHIFT, inode->i_sb->s_blocksize); | 684 | max = ALIGN_DOWN(max << SECTOR_SHIFT, inode->i_sb->s_blocksize); |
| 685 | iov_iter_truncate(from, max); | 685 | iov_iter_truncate(from, max); |
| 686 | 686 | ||
| 687 | nr_pages = iov_iter_npages(from, BIO_MAX_PAGES); | 687 | nr_pages = iov_iter_npages(from, BIO_MAX_VECS); |
| 688 | if (!nr_pages) | 688 | if (!nr_pages) |
| 689 | return 0; | 689 | return 0; |
| 690 | 690 | ||
| @@ -13,6 +13,13 @@ · 12 unmodified lines | |||
| 13 | #define ARMV8_PMU_CYCLE_IDX (ARMV8_PMU_MAX_COUNTERS - 1) | 13 | #define ARMV8_PMU_CYCLE_IDX (ARMV8_PMU_MAX_COUNTERS - 1) |
| 14 | #define ARMV8_PMU_MAX_COUNTER_PAIRS ((ARMV8_PMU_MAX_COUNTERS + 1) >> 1) | 14 | #define ARMV8_PMU_MAX_COUNTER_PAIRS ((ARMV8_PMU_MAX_COUNTERS + 1) >> 1) |
| 15 | 15 | ||
| 16 | DECLARE_STATIC_KEY_FALSE(kvm_arm_pmu_available); | ||
| 17 | |||
| 18 | static __always_inline bool kvm_arm_support_pmu_v3(void) | ||
| 19 | { | ||
| 20 | return static_branch_likely(&kvm_arm_pmu_available); | ||
| 21 | } | ||
| 22 | |||
| 16 | #ifdef CONFIG_HW_PERF_EVENTS | 23 | #ifdef CONFIG_HW_PERF_EVENTS |
| 17 | 24 | ||
| 18 | struct kvm_pmc { | 25 | struct kvm_pmc { |
| @@ -47,7 +54,6 @@ void kvm_pmu_software_increment(struct kvm_vcpu *vcpu, u64 val); · 28 unmodified lines | |||
| 47 | void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val); | 54 | void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val); |
| 48 | void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, u64 data, | 55 | void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, u64 data, |
| 49 | u64 select_idx); | 56 | u64 select_idx); |
| 50 | bool kvm_arm_support_pmu_v3(void); | ||
| 51 | int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, | 57 | int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, |
| 52 | struct kvm_device_attr *attr); | 58 | struct kvm_device_attr *attr); |
| 53 | int kvm_arm_pmu_v3_get_attr(struct kvm_vcpu *vcpu, | 59 | int kvm_arm_pmu_v3_get_attr(struct kvm_vcpu *vcpu, |
| @@ -87,7 +93,6 @@ static inline void kvm_pmu_software_increment(struct kvm_vcpu *vcpu, u64 val) {} · 33 unmodified lines | |||
| 87 | static inline void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val) {} | 93 | static inline void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val) {} |
| 88 | static inline void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, | 94 | static inline void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, |
| 89 | u64 data, u64 select_idx) {} | 95 | u64 data, u64 select_idx) {} |
| 90 | static inline bool kvm_arm_support_pmu_v3(void) { return false; } | ||
| 91 | static inline int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, | 96 | static inline int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, |
| 92 | struct kvm_device_attr *attr) | 97 | struct kvm_device_attr *attr) |
| 93 | { | 98 | { |
| @@ -1079,19 +1079,25 @@ void __acpi_handle_debug(struct _ddebug *descriptor, acpi_handle handle, const c · 1078 unmodified lines | |||
| 1079 | #if defined(CONFIG_ACPI) && defined(CONFIG_GPIOLIB) | 1079 | #if defined(CONFIG_ACPI) && defined(CONFIG_GPIOLIB) |
| 1080 | bool acpi_gpio_get_irq_resource(struct acpi_resource *ares, | 1080 | bool acpi_gpio_get_irq_resource(struct acpi_resource *ares, |
| 1081 | struct acpi_resource_gpio **agpio); | 1081 | struct acpi_resource_gpio **agpio); |
| 1082 | int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index); | 1082 | int acpi_dev_gpio_irq_get_by(struct acpi_device *adev, const char *name, int index); |
| 1083 | #else | 1083 | #else |
| 1084 | static inline bool acpi_gpio_get_irq_resource(struct acpi_resource *ares, | 1084 | static inline bool acpi_gpio_get_irq_resource(struct acpi_resource *ares, |
| 1085 | struct acpi_resource_gpio **agpio) | 1085 | struct acpi_resource_gpio **agpio) |
| 1086 | { | 1086 | { |
| 1087 | return false; | 1087 | return false; |
| 1088 | } | 1088 | } |
| 1089 | static inline int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index) | 1089 | static inline int acpi_dev_gpio_irq_get_by(struct acpi_device *adev, |
| 1090 | const char *name, int index) | ||
| 1090 | { | 1091 | { |
| 1091 | return -ENXIO; | 1092 | return -ENXIO; |
| 1092 | } | 1093 | } |
| 1093 | #endif | 1094 | #endif |
| 1094 | 1095 | ||
| 1096 | static inline int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index) | ||
| 1097 | { | ||
| 1098 | return acpi_dev_gpio_irq_get_by(adev, NULL, index); | ||
| 1099 | } | ||
| 1100 | |||
| 1095 | /* Device properties */ | 1101 | /* Device properties */ |
| 1096 | 1102 | ||
| 1097 | #ifdef CONFIG_ACPI | 1103 | #ifdef CONFIG_ACPI |
| @@ -151,7 +151,7 @@ struct atm_dev { · 150 unmodified lines | |||
| 151 | const char *type; /* device type name */ | 151 | const char *type; /* device type name */ |
| 152 | int number; /* device index */ | 152 | int number; /* device index */ |
| 153 | void *dev_data; /* per-device data */ | 153 | void *dev_data; /* per-device data */ |
| 154 | void *phy_data; /* private PHY date */ | 154 | void *phy_data; /* private PHY data */ |
| 155 | unsigned long flags; /* device flags (ATM_DF_*) */ | 155 | unsigned long flags; /* device flags (ATM_DF_*) */ |
| 156 | struct list_head local; /* local ATM addresses */ | 156 | struct list_head local; /* local ATM addresses */ |
| 157 | struct list_head lecs; /* LECS ATM addresses learned via ILMI */ | 157 | struct list_head lecs; /* LECS ATM addresses learned via ILMI */ |
| @@ -20,11 +20,11 @@ · 19 unmodified lines | |||
| 20 | #define BIO_BUG_ON | 20 | #define BIO_BUG_ON |
| 21 | #endif | 21 | #endif |
| 22 | 22 | ||
| 23 | #define BIO_MAX_PAGES 256U | 23 | #define BIO_MAX_VECS 256U |
| 24 | 24 | ||
| 25 | static inline unsigned int bio_max_segs(unsigned int nr_segs) | 25 | static inline unsigned int bio_max_segs(unsigned int nr_segs) |
| 26 | { | 26 | { |
| 27 | return min(nr_segs, BIO_MAX_PAGES); | 27 | return min(nr_segs, BIO_MAX_VECS); |
| 28 | } | 28 | } |
| 29 | 29 | ||
| 30 | #define bio_prio(bio) (bio)->bi_ioprio | 30 | #define bio_prio(bio) (bio)->bi_ioprio |
| @@ -65,8 +65,12 @@ static inline void can_skb_reserve(struct sk_buff *skb) · 64 unmodified lines | |||
| 65 | 65 | ||
| 66 | static inline void can_skb_set_owner(struct sk_buff *skb, struct sock *sk) | 66 | static inline void can_skb_set_owner(struct sk_buff *skb, struct sock *sk) |
| 67 | { | 67 | { |
| 68 | if (sk) { | 68 | /* If the socket has already been closed by user space, the |
| 69 | sock_hold(sk); | 69 | * refcount may already be 0 (and the socket will be freed |
| 70 | * after the last TX skb has been freed). So only increase | ||
| 71 | * socket refcount if the refcount is > 0. | ||
| 72 | */ | ||
| 73 | if (sk && refcount_inc_not_zero(&sk->sk_refcnt)) { | ||
| 70 | skb->destructor = sock_efree; | 74 | skb->destructor = sock_efree; |
| 71 | skb->sk = sk; | 75 | skb->sk = sk; |
| 72 | } | 76 | } |
| @@ -31,6 +31,12 @@ · 30 unmodified lines | |||
| 31 | #define __no_sanitize_thread | 31 | #define __no_sanitize_thread |
| 32 | #endif | 32 | #endif |
| 33 | 33 | ||
| 34 | #if defined(CONFIG_ARCH_USE_BUILTIN_BSWAP) | ||
| 35 | #define __HAVE_BUILTIN_BSWAP32__ | ||
| 36 | #define __HAVE_BUILTIN_BSWAP64__ | ||
| 37 | #define __HAVE_BUILTIN_BSWAP16__ | ||
| 38 | #endif /* CONFIG_ARCH_USE_BUILTIN_BSWAP */ | ||
| 39 | |||
| 34 | #if __has_feature(undefined_behavior_sanitizer) | 40 | #if __has_feature(undefined_behavior_sanitizer) |
| 35 | /* GCC does not have __SANITIZE_UNDEFINED__ */ | 41 | /* GCC does not have __SANITIZE_UNDEFINED__ */ |
| 36 | #define __no_sanitize_undefined \ | 42 | #define __no_sanitize_undefined \ |
| @@ -0,0 +1,14 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0-only */ | ||
| 2 | |||
| 3 | #ifdef __LINUX_COMPILER_VERSION_H | ||
| 4 | #error "Please do not include <linux/compiler-version.h>. This is done by the build system." | ||
| 5 | #endif | ||
| 6 | #define __LINUX_COMPILER_VERSION_H | ||
| 7 | |||
| 8 | /* | ||
| 9 | * This header exists to force full rebuild when the compiler is upgraded. | ||
| 10 | * | ||
| 11 | * When fixdep scans this, it will find this string "CONFIG_CC_VERSION_TEXT" | ||
| 12 | * and add dependency on include/config/cc/version/text.h, which is touched | ||
| 13 | * by Kconfig when the version string from the compiler changes. | ||
| 14 | */ | ||
| @@ -108,6 +108,8 @@ static inline void cpu_maps_update_done(void) · 107 unmodified lines | |||
| 108 | { | 108 | { |
| 109 | } | 109 | } |
| 110 | 110 | ||
| 111 | static inline int add_cpu(unsigned int cpu) { return 0;} | ||
| 112 | |||
| 111 | #endif /* CONFIG_SMP */ | 113 | #endif /* CONFIG_SMP */ |
| 112 | extern struct bus_type cpu_subsys; | 114 | extern struct bus_type cpu_subsys; |
| 113 | 115 | ||
| @@ -137,6 +139,7 @@ static inline int cpus_read_trylock(void) { return true; } · 23 unmodified lines | |||
| 137 | static inline void lockdep_assert_cpus_held(void) { } | 139 | static inline void lockdep_assert_cpus_held(void) { } |
| 138 | static inline void cpu_hotplug_disable(void) { } | 140 | static inline void cpu_hotplug_disable(void) { } |
| 139 | static inline void cpu_hotplug_enable(void) { } | 141 | static inline void cpu_hotplug_enable(void) { } |
| 142 | static inline int remove_cpu(unsigned int cpu) { return -EPERM; } | ||
| 140 | static inline void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) { } | 143 | static inline void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) { } |
| 141 | #endif /* !CONFIG_HOTPLUG_CPU */ | 144 | #endif /* !CONFIG_HOTPLUG_CPU */ |
| 142 | 145 | ||
| @@ -674,6 +674,8 @@ struct acpi_gpio_mapping { · 673 unmodified lines | |||
| 674 | * get GpioIo type explicitly, this quirk may be used. | 674 | * get GpioIo type explicitly, this quirk may be used. |
| 675 | */ | 675 | */ |
| 676 | #define ACPI_GPIO_QUIRK_ONLY_GPIOIO BIT(1) | 676 | #define ACPI_GPIO_QUIRK_ONLY_GPIOIO BIT(1) |
| 677 | /* Use given pin as an absolute GPIO number in the system */ | ||
| 678 | #define ACPI_GPIO_QUIRK_ABSOLUTE_NUMBER BIT(2) | ||
| 677 | 679 | ||
| 678 | unsigned int quirks; | 680 | unsigned int quirks; |
| 679 | }; | 681 | }; |
| @@ -18,7 +18,7 @@ struct io_uring_task { · 17 unmodified lines | |||
| 18 | /* submission side */ | 18 | /* submission side */ |
| 19 | struct xarray xa; | 19 | struct xarray xa; |
| 20 | struct wait_queue_head wait; | 20 | struct wait_queue_head wait; |
| 21 | struct file *last; | 21 | void *last; |
| 22 | void *io_wq; | 22 | void *io_wq; |
| 23 | struct percpu_counter inflight; | 23 | struct percpu_counter inflight; |
| 24 | atomic_t in_idle; | 24 | atomic_t in_idle; |
| @@ -150,7 +150,6 @@ struct irq_domain_chip_generic; · 149 unmodified lines | |||
| 150 | * setting up one or more generic chips for interrupt controllers | 150 | * setting up one or more generic chips for interrupt controllers |
| 151 | * drivers using the generic chip library which uses this pointer. | 151 | * drivers using the generic chip library which uses this pointer. |
| 152 | * @parent: Pointer to parent irq_domain to support hierarchy irq_domains | 152 | * @parent: Pointer to parent irq_domain to support hierarchy irq_domains |
| 153 | * @debugfs_file: dentry for the domain debugfs file | ||
| 154 | * | 153 | * |
| 155 | * Revmap data, used internally by irq_domain | 154 | * Revmap data, used internally by irq_domain |
| 156 | * @revmap_direct_max_irq: The largest hwirq that can be set for controllers that | 155 | * @revmap_direct_max_irq: The largest hwirq that can be set for controllers that |
| @@ -174,9 +173,6 @@ struct irq_domain { · 17 unmodified lines | |||
| 174 | #ifdef CONFIG_IRQ_DOMAIN_HIERARCHY | 173 | #ifdef CONFIG_IRQ_DOMAIN_HIERARCHY |
| 175 | struct irq_domain *parent; | 174 | struct irq_domain *parent; |
| 176 | #endif | 175 | #endif |
| 177 | #ifdef CONFIG_GENERIC_IRQ_DEBUGFS | ||
| 178 | struct dentry *debugfs_file; | ||
| 179 | #endif | ||
| 180 | 176 | ||
| 181 | /* reverse map data. The linear map gets appended to the irq_domain */ | 177 | /* reverse map data. The linear map gets appended to the irq_domain */ |
| 182 | irq_hw_number_t hwirq_max; | 178 | irq_hw_number_t hwirq_max; |
| @@ -2,8 +2,6 @@ · 1 unmodified line | |||
| 2 | #ifndef __LINUX_KCONFIG_H | 2 | #ifndef __LINUX_KCONFIG_H |
| 3 | #define __LINUX_KCONFIG_H | 3 | #define __LINUX_KCONFIG_H |
| 4 | 4 | ||
| 5 | /* CONFIG_CC_VERSION_TEXT (Do not delete this comment. See help in Kconfig) */ | ||
| 6 | |||
| 7 | #include <generated/autoconf.h> | 5 | #include <generated/autoconf.h> |
| 8 | 6 | ||
| 9 | #ifdef CONFIG_CPU_BIG_ENDIAN | 7 | #ifdef CONFIG_CPU_BIG_ENDIAN |
| @@ -460,7 +460,7 @@ static inline void memblock_free_late(phys_addr_t base, phys_addr_t size) · 459 unmodified lines | |||
| 460 | /* | 460 | /* |
| 461 | * Set the allocation direction to bottom-up or top-down. | 461 | * Set the allocation direction to bottom-up or top-down. |
| 462 | */ | 462 | */ |
| 463 | static inline void memblock_set_bottom_up(bool enable) | 463 | static inline __init void memblock_set_bottom_up(bool enable) |
| 464 | { | 464 | { |
| 465 | memblock.bottom_up = enable; | 465 | memblock.bottom_up = enable; |
| 466 | } | 466 | } |
| @@ -470,7 +470,7 @@ static inline void memblock_set_bottom_up(bool enable) · 3 unmodified lines | |||
| 470 | * if this is true, that said, memblock will allocate memory | 470 | * if this is true, that said, memblock will allocate memory |
| 471 | * in bottom-up direction. | 471 | * in bottom-up direction. |
| 472 | */ | 472 | */ |
| 473 | static inline bool memblock_bottom_up(void) | 473 | static inline __init bool memblock_bottom_up(void) |
| 474 | { | 474 | { |
| 475 | return memblock.bottom_up; | 475 | return memblock.bottom_up; |
| 476 | } | 476 | } |
| @@ -1061,9 +1061,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, · 1060 unmodified lines | |||
| 1061 | rcu_read_unlock(); | 1061 | rcu_read_unlock(); |
| 1062 | } | 1062 | } |
| 1063 | 1063 | ||
| 1064 | #ifdef CONFIG_TRANSPARENT_HUGEPAGE | 1064 | void split_page_memcg(struct page *head, unsigned int nr); |
| 1065 | void mem_cgroup_split_huge_fixup(struct page *head); | ||
| 1066 | #endif | ||
| 1067 | 1065 | ||
| 1068 | #else /* CONFIG_MEMCG */ | 1066 | #else /* CONFIG_MEMCG */ |
| 1069 | 1067 | ||
| @@ -1400,7 +1398,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, · 330 unmodified lines | |||
| 1400 | return 0; | 1398 | return 0; |
| 1401 | } | 1399 | } |
| 1402 | 1400 | ||
| 1403 | static inline void mem_cgroup_split_huge_fixup(struct page *head) | 1401 | static inline void split_page_memcg(struct page *head, unsigned int nr) |
| 1404 | { | 1402 | { |
| 1405 | } | 1403 | } |
| 1406 | 1404 | ||
| @@ -1300,6 +1300,27 @@ static inline bool page_maybe_dma_pinned(struct page *page) · 1299 unmodified lines | |||
| 1300 | GUP_PIN_COUNTING_BIAS; | 1300 | GUP_PIN_COUNTING_BIAS; |
| 1301 | } | 1301 | } |
| 1302 | 1302 | ||
| 1303 | static inline bool is_cow_mapping(vm_flags_t flags) | ||
| 1304 | { | ||
| 1305 | return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; | ||
| 1306 | } | ||
| 1307 | |||
| 1308 | /* | ||
| 1309 | * This should most likely only be called during fork() to see whether we | ||
| 1310 | * should break the cow immediately for a page on the src mm. | ||
| 1311 | */ | ||
| 1312 | static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma, | ||
| 1313 | struct page *page) | ||
| 1314 | { | ||
| 1315 | if (!is_cow_mapping(vma->vm_flags)) | ||
| 1316 | return false; | ||
| 1317 | |||
| 1318 | if (!atomic_read(&vma->vm_mm->has_pinned)) | ||
| 1319 | return false; | ||
| 1320 | |||
| 1321 | return page_maybe_dma_pinned(page); | ||
| 1322 | } | ||
| 1323 | |||
| 1303 | #if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP) | 1324 | #if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP) |
| 1304 | #define SECTION_IN_PAGE_FLAGS | 1325 | #define SECTION_IN_PAGE_FLAGS |
| 1305 | #endif | 1326 | #endif |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | |||
| 23 | #endif | 23 | #endif |
| 24 | #define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1)) | 24 | #define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1)) |
| 25 | 25 | ||
| 26 | #define INIT_PASID 0 | ||
| 26 | 27 | ||
| 27 | struct address_space; | 28 | struct address_space; |
| 28 | struct mem_cgroup; | 29 | struct mem_cgroup; |
| @@ -3959,8 +3959,6 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, · 3958 unmodified lines | |||
| 3959 | int bpf_xdp_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); | 3959 | int bpf_xdp_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); |
| 3960 | u32 dev_xdp_prog_id(struct net_device *dev, enum bpf_xdp_mode mode); | 3960 | u32 dev_xdp_prog_id(struct net_device *dev, enum bpf_xdp_mode mode); |
| 3961 | 3961 | ||
| 3962 | int xdp_umem_query(struct net_device *dev, u16 queue_id); | ||
| 3963 | |||
| 3964 | int __dev_forward_skb(struct net_device *dev, struct sk_buff *skb); | 3962 | int __dev_forward_skb(struct net_device *dev, struct sk_buff *skb); |
| 3965 | int dev_forward_skb(struct net_device *dev, struct sk_buff *skb); | 3963 | int dev_forward_skb(struct net_device *dev, struct sk_buff *skb); |
| 3966 | int dev_forward_skb_nomtu(struct net_device *dev, struct sk_buff *skb); | 3964 | int dev_forward_skb_nomtu(struct net_device *dev, struct sk_buff *skb); |
| @@ -606,6 +606,7 @@ struct swevent_hlist { · 605 unmodified lines | |||
| 606 | #define PERF_ATTACH_TASK 0x04 | 606 | #define PERF_ATTACH_TASK 0x04 |
| 607 | #define PERF_ATTACH_TASK_DATA 0x08 | 607 | #define PERF_ATTACH_TASK_DATA 0x08 |
| 608 | #define PERF_ATTACH_ITRACE 0x10 | 608 | #define PERF_ATTACH_ITRACE 0x10 |
| 609 | #define PERF_ATTACH_SCHED_CB 0x20 | ||
| 609 | 610 | ||
| 610 | struct perf_cgroup; | 611 | struct perf_cgroup; |
| 611 | struct perf_buffer; | 612 | struct perf_buffer; |
| @@ -872,6 +873,7 @@ struct perf_cpu_context { · 260 unmodified lines | |||
| 872 | struct list_head cgrp_cpuctx_entry; | 873 | struct list_head cgrp_cpuctx_entry; |
| 873 | #endif | 874 | #endif |
| 874 | 875 | ||
| 876 | struct list_head sched_cb_entry; | ||
| 875 | int sched_cb_usage; | 877 | int sched_cb_usage; |
| 876 | 878 | ||
| 877 | int online; | 879 | int online; |
| @@ -904,6 +904,10 @@ static inline void ptep_modify_prot_commit(struct vm_area_struct *vma, · 903 unmodified lines | |||
| 904 | #define pgprot_device pgprot_noncached | 904 | #define pgprot_device pgprot_noncached |
| 905 | #endif | 905 | #endif |
| 906 | 906 | ||
| 907 | #ifndef pgprot_mhp | ||
| 908 | #define pgprot_mhp(prot) (prot) | ||
| 909 | #endif | ||
| 910 | |||
| 907 | #ifdef CONFIG_MMU | 911 | #ifdef CONFIG_MMU |
| 908 | #ifndef pgprot_modify | 912 | #ifndef pgprot_modify |
| 909 | #define pgprot_modify pgprot_modify | 913 | #define pgprot_modify pgprot_modify |
| @@ -488,7 +488,7 @@ fwnode_create_software_node(const struct property_entry *properties, · 487 unmodified lines | |||
| 488 | const struct fwnode_handle *parent); | 488 | const struct fwnode_handle *parent); |
| 489 | void fwnode_remove_software_node(struct fwnode_handle *fwnode); | 489 | void fwnode_remove_software_node(struct fwnode_handle *fwnode); |
| 490 | 490 | ||
| 491 | int device_add_software_node(struct device *dev, const struct software_node *swnode); | 491 | int device_add_software_node(struct device *dev, const struct software_node *node); |
| 492 | void device_remove_software_node(struct device *dev); | 492 | void device_remove_software_node(struct device *dev); |
| 493 | 493 | ||
| 494 | int device_create_managed_software_node(struct device *dev, | 494 | int device_create_managed_software_node(struct device *dev, |
| @@ -147,6 +147,9 @@ enum { · 146 unmodified lines | |||
| 147 | #define BUCK6_FPWM 0x04 | 147 | #define BUCK6_FPWM 0x04 |
| 148 | #define BUCK6_ENMODE_MASK 0x03 | 148 | #define BUCK6_ENMODE_MASK 0x03 |
| 149 | 149 | ||
| 150 | /* PCA9450_REG_BUCK123_PRESET_EN bit */ | ||
| 151 | #define BUCK123_PRESET_EN 0x80 | ||
| 152 | |||
| 150 | /* PCA9450_BUCK1OUT_DVS0 bits */ | 153 | /* PCA9450_BUCK1OUT_DVS0 bits */ |
| 151 | #define BUCK1OUT_DVS0_MASK 0x7F | 154 | #define BUCK1OUT_DVS0_MASK 0x7F |
| 152 | #define BUCK1OUT_DVS0_DEFAULT 0x14 | 155 | #define BUCK1OUT_DVS0_DEFAULT 0x14 |
| @@ -140,7 +140,8 @@ static inline bool in_vfork(struct task_struct *tsk) · 139 unmodified lines | |||
| 140 | * another oom-unkillable task does this it should blame itself. | 140 | * another oom-unkillable task does this it should blame itself. |
| 141 | */ | 141 | */ |
| 142 | rcu_read_lock(); | 142 | rcu_read_lock(); |
| 143 | ret = tsk->vfork_done && tsk->real_parent->mm == tsk->mm; | 143 | ret = tsk->vfork_done && |
| 144 | rcu_dereference(tsk->real_parent)->mm == tsk->mm; | ||
| 144 | rcu_read_unlock(); | 145 | rcu_read_unlock(); |
| 145 | 146 | ||
| 146 | return ret; | 147 | return ret; |
| @@ -664,10 +664,7 @@ typedef struct { · 663 unmodified lines | |||
| 664 | * seqcount_latch_init() - runtime initializer for seqcount_latch_t | 664 | * seqcount_latch_init() - runtime initializer for seqcount_latch_t |
| 665 | * @s: Pointer to the seqcount_latch_t instance | 665 | * @s: Pointer to the seqcount_latch_t instance |
| 666 | */ | 666 | */ |
| 667 | static inline void seqcount_latch_init(seqcount_latch_t *s) | 667 | #define seqcount_latch_init(s) seqcount_init(&(s)->seqcount) |
| 668 | { | ||
| 669 | seqcount_init(&s->seqcount); | ||
| 670 | } | ||
| 671 | 668 | ||
| 672 | /** | 669 | /** |
| 673 | * raw_read_seqcount_latch() - pick even/odd latch data copy | 670 | * raw_read_seqcount_latch() - pick even/odd latch data copy |
| @@ -128,7 +128,7 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data, · 127 unmodified lines | |||
| 128 | const struct cpumask *cpus); | 128 | const struct cpumask *cpus); |
| 129 | #else /* CONFIG_SMP || CONFIG_HOTPLUG_CPU */ | 129 | #else /* CONFIG_SMP || CONFIG_HOTPLUG_CPU */ |
| 130 | 130 | ||
| 131 | static inline int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data, | 131 | static __always_inline int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data, |
| 132 | const struct cpumask *cpus) | 132 | const struct cpumask *cpus) |
| 133 | { | 133 | { |
| 134 | unsigned long flags; | 134 | unsigned long flags; |
| @@ -139,14 +139,15 @@ static inline int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data, · 4 unmodified lines | |||
| 139 | return ret; | 139 | return ret; |
| 140 | } | 140 | } |
| 141 | 141 | ||
| 142 | static inline int stop_machine(cpu_stop_fn_t fn, void *data, | 142 | static __always_inline int |
| 143 | const struct cpumask *cpus) | 143 | stop_machine(cpu_stop_fn_t fn, void *data, const struct cpumask *cpus) |
| 144 | { | 144 | { |
| 145 | return stop_machine_cpuslocked(fn, data, cpus); | 145 | return stop_machine_cpuslocked(fn, data, cpus); |
| 146 | } | 146 | } |
| 147 | 147 | ||
| 148 | static inline int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data, | 148 | static __always_inline int |
| 149 | const struct cpumask *cpus) | 149 | stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data, |
| 150 | const struct cpumask *cpus) | ||
| 150 | { | 151 | { |
| 151 | return stop_machine(fn, data, cpus); | 152 | return stop_machine(fn, data, cpus); |
| 152 | } | 153 | } |
| @@ -104,7 +104,6 @@ struct svcxprt_rdma { · 103 unmodified lines | |||
| 104 | 104 | ||
| 105 | wait_queue_head_t sc_send_wait; /* SQ exhaustion waitlist */ | 105 | wait_queue_head_t sc_send_wait; /* SQ exhaustion waitlist */ |
| 106 | unsigned long sc_flags; | 106 | unsigned long sc_flags; |
| 107 | u32 sc_pending_recvs; | ||
| 108 | struct list_head sc_read_complete_q; | 107 | struct list_head sc_read_complete_q; |
| 109 | struct work_struct sc_work; | 108 | struct work_struct sc_work; |
| 110 | 109 | ||
| @@ -23,7 +23,7 @@ struct ts_config; · 22 unmodified lines | |||
| 23 | struct ts_state | 23 | struct ts_state |
| 24 | { | 24 | { |
| 25 | unsigned int offset; | 25 | unsigned int offset; |
| 26 | char cb[40]; | 26 | char cb[48]; |
| 27 | }; | 27 | }; |
| 28 | 28 | ||
| 29 | /** | 29 | /** |
| @@ -115,12 +115,13 @@ static inline void u64_stats_inc(u64_stats_t *p) · 114 unmodified lines | |||
| 115 | } | 115 | } |
| 116 | #endif | 116 | #endif |
| 117 | 117 | ||
| 118 | #if BITS_PER_LONG == 32 && defined(CONFIG_SMP) | ||
| 119 | #define u64_stats_init(syncp) seqcount_init(&(syncp)->seq) | ||
| 120 | #else | ||
| 118 | static inline void u64_stats_init(struct u64_stats_sync *syncp) | 121 | static inline void u64_stats_init(struct u64_stats_sync *syncp) |
| 119 | { | 122 | { |
| 120 | #if BITS_PER_LONG == 32 && defined(CONFIG_SMP) | ||
| 121 | seqcount_init(&syncp->seq); | ||
| 122 | #endif | ||
| 123 | } | 123 | } |
| 124 | #endif | ||
| 124 | 125 | ||
| 125 | static inline void u64_stats_update_begin(struct u64_stats_sync *syncp) | 126 | static inline void u64_stats_update_begin(struct u64_stats_sync *syncp) |
| 126 | { | 127 | { |
| @@ -746,6 +746,8 @@ extern int usb_lock_device_for_reset(struct usb_device *udev, · 745 unmodified lines | |||
| 746 | extern int usb_reset_device(struct usb_device *dev); | 746 | extern int usb_reset_device(struct usb_device *dev); |
| 747 | extern void usb_queue_reset_device(struct usb_interface *dev); | 747 | extern void usb_queue_reset_device(struct usb_interface *dev); |
| 748 | 748 | ||
| 749 | extern struct device *usb_intf_get_dma_device(struct usb_interface *intf); | ||
| 750 | |||
| 749 | #ifdef CONFIG_ACPI | 751 | #ifdef CONFIG_ACPI |
| 750 | extern int usb_acpi_set_power_state(struct usb_device *hdev, int index, | 752 | extern int usb_acpi_set_power_state(struct usb_device *hdev, int index, |
| 751 | bool enable); | 753 | bool enable); |
| @@ -79,8 +79,13 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, · 78 unmodified lines | |||
| 79 | if (gso_type && skb->network_header) { | 79 | if (gso_type && skb->network_header) { |
| 80 | struct flow_keys_basic keys; | 80 | struct flow_keys_basic keys; |
| 81 | 81 | ||
| 82 | if (!skb->protocol) | 82 | if (!skb->protocol) { |
| 83 | __be16 protocol = dev_parse_header_protocol(skb); | ||
| 84 | |||
| 83 | virtio_net_hdr_set_proto(skb, hdr); | 85 | virtio_net_hdr_set_proto(skb, hdr); |
| 86 | if (protocol && protocol != skb->protocol) | ||
| 87 | return -EINVAL; | ||
| 88 | } | ||
| 84 | retry: | 89 | retry: |
| 85 | if (!skb_flow_dissect_flow_keys_basic(NULL, skb, &keys, | 90 | if (!skb_flow_dissect_flow_keys_basic(NULL, skb, &keys, |
| 86 | NULL, 0, 0, 0, | 91 | NULL, 0, 0, 0, |
| @@ -173,9 +173,10 @@ static inline void ww_acquire_done(struct ww_acquire_ctx *ctx) · 172 unmodified lines | |||
| 173 | */ | 173 | */ |
| 174 | static inline void ww_acquire_fini(struct ww_acquire_ctx *ctx) | 174 | static inline void ww_acquire_fini(struct ww_acquire_ctx *ctx) |
| 175 | { | 175 | { |
| 176 | #ifdef CONFIG_DEBUG_MUTEXES | 176 | #ifdef CONFIG_DEBUG_LOCK_ALLOC |
| 177 | mutex_release(&ctx->dep_map, _THIS_IP_); | 177 | mutex_release(&ctx->dep_map, _THIS_IP_); |
| 178 | 178 | #endif | |
| 179 | #ifdef CONFIG_DEBUG_MUTEXES | ||
| 179 | DEBUG_LOCKS_WARN_ON(ctx->acquired); | 180 | DEBUG_LOCKS_WARN_ON(ctx->acquired); |
| 180 | if (!IS_ENABLED(CONFIG_PROVE_LOCKING)) | 181 | if (!IS_ENABLED(CONFIG_PROVE_LOCKING)) |
| 181 | /* | 182 | /* |
| @@ -175,6 +175,13 @@ struct rc_map_list { · 174 unmodified lines | |||
| 175 | struct rc_map map; | 175 | struct rc_map map; |
| 176 | }; | 176 | }; |
| 177 | 177 | ||
| 178 | #ifdef CONFIG_MEDIA_CEC_RC | ||
| 179 | /* | ||
| 180 | * rc_map_list from rc-cec.c | ||
| 181 | */ | ||
| 182 | extern struct rc_map_list cec_map; | ||
| 183 | #endif | ||
| 184 | |||
| 178 | /* Routines from rc-map.c */ | 185 | /* Routines from rc-map.c */ |
| 179 | 186 | ||
| 180 | /** | 187 | /** |
| @@ -3850,7 +3850,6 @@ union bpf_attr { · 3849 unmodified lines | |||
| 3850 | * | 3850 | * |
| 3851 | * long bpf_check_mtu(void *ctx, u32 ifindex, u32 *mtu_len, s32 len_diff, u64 flags) | 3851 | * long bpf_check_mtu(void *ctx, u32 ifindex, u32 *mtu_len, s32 len_diff, u64 flags) |
| 3852 | * Description | 3852 | * Description |
| 3853 | |||
| 3854 | * Check ctx packet size against exceeding MTU of net device (based | 3853 | * Check ctx packet size against exceeding MTU of net device (based |
| 3855 | * on *ifindex*). This helper will likely be used in combination | 3854 | * on *ifindex*). This helper will likely be used in combination |
| 3856 | * with helpers that adjust/change the packet size. | 3855 | * with helpers that adjust/change the packet size. |
| @@ -903,7 +903,8 @@ struct fuse_notify_retrieve_in { · 902 unmodified lines | |||
| 903 | }; | 903 | }; |
| 904 | 904 | ||
| 905 | /* Device ioctls: */ | 905 | /* Device ioctls: */ |
| 906 | #define FUSE_DEV_IOC_CLONE _IOR(229, 0, uint32_t) | 906 | #define FUSE_DEV_IOC_MAGIC 229 |
| 907 | #define FUSE_DEV_IOC_CLONE _IOR(FUSE_DEV_IOC_MAGIC, 0, uint32_t) | ||
| 907 | 908 | ||
| 908 | struct fuse_lseek_in { | 909 | struct fuse_lseek_in { |
| 909 | uint64_t fh; | 910 | uint64_t fh; |
| @@ -145,6 +145,7 @@ enum { · 144 unmodified lines | |||
| 145 | L2TP_ATTR_RX_ERRORS, /* u64 */ | 145 | L2TP_ATTR_RX_ERRORS, /* u64 */ |
| 146 | L2TP_ATTR_STATS_PAD, | 146 | L2TP_ATTR_STATS_PAD, |
| 147 | L2TP_ATTR_RX_COOKIE_DISCARDS, /* u64 */ | 147 | L2TP_ATTR_RX_COOKIE_DISCARDS, /* u64 */ |
| 148 | L2TP_ATTR_RX_INVALID, /* u64 */ | ||
| 148 | __L2TP_ATTR_STATS_MAX, | 149 | __L2TP_ATTR_STATS_MAX, |
| 149 | }; | 150 | }; |
| 150 | 151 | ||
| @@ -5,7 +5,7 @@ · 4 unmodified lines | |||
| 5 | #define NFCT_HELPER_STATUS_DISABLED 0 | 5 | #define NFCT_HELPER_STATUS_DISABLED 0 |
| 6 | #define NFCT_HELPER_STATUS_ENABLED 1 | 6 | #define NFCT_HELPER_STATUS_ENABLED 1 |
| 7 | 7 | ||
| 8 | enum nfnl_acct_msg_types { | 8 | enum nfnl_cthelper_msg_types { |
| 9 | NFNL_MSG_CTHELPER_NEW, | 9 | NFNL_MSG_CTHELPER_NEW, |
| 10 | NFNL_MSG_CTHELPER_GET, | 10 | NFNL_MSG_CTHELPER_GET, |
| 11 | NFNL_MSG_CTHELPER_DEL, | 11 | NFNL_MSG_CTHELPER_DEL, |
| @@ -50,6 +50,13 @@ · 49 unmodified lines | |||
| 50 | #include <linux/page-flags.h> | 50 | #include <linux/page-flags.h> |
| 51 | #include <linux/kernel.h> | 51 | #include <linux/kernel.h> |
| 52 | 52 | ||
| 53 | /* | ||
| 54 | * Technically there's no reliably invalid grant reference or grant handle, | ||
| 55 | * so pick the value that is the most unlikely one to be observed valid. | ||
| 56 | */ | ||
| 57 | #define INVALID_GRANT_REF ((grant_ref_t)-1) | ||
| 58 | #define INVALID_GRANT_HANDLE ((grant_handle_t)-1) | ||
| 59 | |||
| 53 | #define GNTTAB_RESERVED_XENSTORE 1 | 60 | #define GNTTAB_RESERVED_XENSTORE 1 |
| 54 | 61 | ||
| 55 | /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ | 62 | /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ |
| @@ -51,7 +51,6 @@ · 50 unmodified lines | |||
| 51 | 51 | ||
| 52 | #define XENBUS_MAX_RING_GRANT_ORDER 4 | 52 | #define XENBUS_MAX_RING_GRANT_ORDER 4 |
| 53 | #define XENBUS_MAX_RING_GRANTS (1U << XENBUS_MAX_RING_GRANT_ORDER) | 53 | #define XENBUS_MAX_RING_GRANTS (1U << XENBUS_MAX_RING_GRANT_ORDER) |
| 54 | #define INVALID_GRANT_HANDLE (~0U) | ||
| 55 | 54 | ||
| 56 | /* Register callback to watch this node. */ | 55 | /* Register callback to watch this node. */ |
| 57 | struct xenbus_watch | 56 | struct xenbus_watch |
| @@ -20,10 +20,10 @@ config CC_VERSION_TEXT · 19 unmodified lines | |||
| 20 | When the compiler is updated, Kconfig will be invoked. | 20 | When the compiler is updated, Kconfig will be invoked. |
| 21 | 21 | ||
| 22 | - Ensure full rebuild when the compiler is updated | 22 | - Ensure full rebuild when the compiler is updated |
| 23 | include/linux/kconfig.h contains this option in the comment line so | 23 | include/linux/compiler-version.h contains this option in the comment |
| 24 | fixdep adds include/config/cc/version/text.h into the auto-generated | 24 | line so fixdep adds include/config/cc/version/text.h into the |
| 25 | dependency. When the compiler is updated, syncconfig will touch it | 25 | auto-generated dependency. When the compiler is updated, syncconfig |
| 26 | and then every file will be rebuilt. | 26 | will touch it and then every file will be rebuilt. |
| 27 | 27 | ||
| 28 | config CC_IS_GCC | 28 | config CC_IS_GCC |
| 29 | def_bool $(success,test "$(cc-name)" = GCC) | 29 | def_bool $(success,test "$(cc-name)" = GCC) |
| @@ -119,8 +119,7 @@ config INIT_ENV_ARG_LIMIT · 89 unmodified lines | |||
| 119 | 119 | ||
| 120 | config COMPILE_TEST | 120 | config COMPILE_TEST |
| 121 | bool "Compile also drivers which will not load" | 121 | bool "Compile also drivers which will not load" |
| 122 | depends on !UML && !S390 | 122 | depends on HAS_IOMEM |
| 123 | default n | ||
| 124 | help | 123 | help |
| 125 | Some drivers can be compiled on a different platform than they are | 124 | Some drivers can be compiled on a different platform than they are |
| 126 | intended to be run on. Despite they cannot be loaded there (or even | 125 | intended to be run on. Despite they cannot be loaded there (or even |
| @@ -4321,8 +4321,6 @@ btf_get_prog_ctx_type(struct bpf_verifier_log *log, struct btf *btf, · 4320 unmodified lines | |||
| 4321 | * is not supported yet. | 4321 | * is not supported yet. |
| 4322 | * BPF_PROG_TYPE_RAW_TRACEPOINT is fine. | 4322 | * BPF_PROG_TYPE_RAW_TRACEPOINT is fine. |
| 4323 | */ | 4323 | */ |
| 4324 | if (log->level & BPF_LOG_LEVEL) | ||
| 4325 | bpf_log(log, "arg#%d type is not a struct\n", arg); | ||
| 4326 | return NULL; | 4324 | return NULL; |
| 4327 | } | 4325 | } |
| 4328 | tname = btf_name_by_offset(btf, t->name_off); | 4326 | tname = btf_name_by_offset(btf, t->name_off); |
| @@ -1118,6 +1118,8 @@ static void bpf_prog_clone_free(struct bpf_prog *fp) · 1117 unmodified lines | |||
| 1118 | * clone is guaranteed to not be locked. | 1118 | * clone is guaranteed to not be locked. |
| 1119 | */ | 1119 | */ |
| 1120 | fp->aux = NULL; | 1120 | fp->aux = NULL; |
| 1121 | fp->stats = NULL; | ||
| 1122 | fp->active = NULL; | ||
| 1121 | __bpf_prog_free(fp); | 1123 | __bpf_prog_free(fp); |
| 1122 | } | 1124 | } |
| 1123 | 1125 | ||
| @@ -2342,6 +2344,10 @@ bool __weak bpf_helper_changes_pkt_data(void *func) · 1218 unmodified lines | |||
| 2342 | /* Return TRUE if the JIT backend wants verifier to enable sub-register usage | 2344 | /* Return TRUE if the JIT backend wants verifier to enable sub-register usage |
| 2343 | * analysis code and wants explicit zero extension inserted by verifier. | 2345 | * analysis code and wants explicit zero extension inserted by verifier. |
| 2344 | * Otherwise, return FALSE. | 2346 | * Otherwise, return FALSE. |
| 2347 | * | ||
| 2348 | * The verifier inserts an explicit zero extension after BPF_CMPXCHGs even if | ||
| 2349 | * you don't override this. JITs that don't want these extra insns can detect | ||
| 2350 | * them using insn_is_zext. | ||
| 2345 | */ | 2351 | */ |
| 2346 | bool __weak bpf_jit_needs_zext(void) | 2352 | bool __weak bpf_jit_needs_zext(void) |
| 2347 | { | 2353 | { |
| @@ -504,6 +504,13 @@ static bool is_ptr_cast_function(enum bpf_func_id func_id) · 503 unmodified lines | |||
| 504 | func_id == BPF_FUNC_skc_to_tcp_request_sock; | 504 | func_id == BPF_FUNC_skc_to_tcp_request_sock; |
| 505 | } | 505 | } |
| 506 | 506 | ||
| 507 | static bool is_cmpxchg_insn(const struct bpf_insn *insn) | ||
| 508 | { | ||
| 509 | return BPF_CLASS(insn->code) == BPF_STX && | ||
| 510 | BPF_MODE(insn->code) == BPF_ATOMIC && | ||
| 511 | insn->imm == BPF_CMPXCHG; | ||
| 512 | } | ||
| 513 | |||
| 507 | /* string representation of 'enum bpf_reg_type' */ | 514 | /* string representation of 'enum bpf_reg_type' */ |
| 508 | static const char * const reg_type_str[] = { | 515 | static const char * const reg_type_str[] = { |
| 509 | [NOT_INIT] = "?", | 516 | [NOT_INIT] = "?", |
| @@ -1120,7 +1127,7 @@ static void mark_ptr_not_null_reg(struct bpf_reg_state *reg) · 610 unmodified lines | |||
| 1120 | reg->type = PTR_TO_RDWR_BUF; | 1127 | reg->type = PTR_TO_RDWR_BUF; |
| 1121 | break; | 1128 | break; |
| 1122 | default: | 1129 | default: |
| 1123 | WARN_ON("unknown nullable register type"); | 1130 | WARN_ONCE(1, "unknown nullable register type"); |
| 1124 | } | 1131 | } |
| 1125 | } | 1132 | } |
| 1126 | 1133 | ||
| @@ -1703,7 +1710,11 @@ static bool is_reg64(struct bpf_verifier_env *env, struct bpf_insn *insn, · 576 unmodified lines | |||
| 1703 | } | 1710 | } |
| 1704 | 1711 | ||
| 1705 | if (class == BPF_STX) { | 1712 | if (class == BPF_STX) { |
| 1706 | if (reg->type != SCALAR_VALUE) | 1713 | /* BPF_STX (including atomic variants) has multiple source |
| 1714 | * operands, one of which is a ptr. Check whether the caller is | ||
| 1715 | * asking about it. | ||
| 1716 | */ | ||
| 1717 | if (t == SRC_OP && reg->type != SCALAR_VALUE) | ||
| 1707 | return true; | 1718 | return true; |
| 1708 | return BPF_SIZE(code) == BPF_DW; | 1719 | return BPF_SIZE(code) == BPF_DW; |
| 1709 | } | 1720 | } |
| @@ -1735,22 +1746,38 @@ static bool is_reg64(struct bpf_verifier_env *env, struct bpf_insn *insn, · 25 unmodified lines | |||
| 1735 | return true; | 1746 | return true; |
| 1736 | } | 1747 | } |
| 1737 | 1748 | ||
| 1738 | /* Return TRUE if INSN doesn't have explicit value define. */ | 1749 | /* Return the regno defined by the insn, or -1. */ |
| 1739 | static bool insn_no_def(struct bpf_insn *insn) | 1750 | static int insn_def_regno(const struct bpf_insn *insn) |
| 1740 | { | 1751 | { |
| 1741 | u8 class = BPF_CLASS(insn->code); | 1752 | switch (BPF_CLASS(insn->code)) { |
| 1742 | 1753 | case BPF_JMP: | |
| 1743 | return (class == BPF_JMP || class == BPF_JMP32 || | 1754 | case BPF_JMP32: |
| 1744 | class == BPF_STX || class == BPF_ST); | 1755 | case BPF_ST: |
| 1756 | return -1; | ||
| 1757 | case BPF_STX: | ||
| 1758 | if (BPF_MODE(insn->code) == BPF_ATOMIC && | ||
| 1759 | (insn->imm & BPF_FETCH)) { | ||
| 1760 | if (insn->imm == BPF_CMPXCHG) | ||
| 1761 | return BPF_REG_0; | ||
| 1762 | else | ||
| 1763 | return insn->src_reg; | ||
| 1764 | } else { | ||
| 1765 | return -1; | ||
| 1766 | } | ||
| 1767 | default: | ||
| 1768 | return insn->dst_reg; | ||
| 1769 | } | ||
| 1745 | } | 1770 | } |
| 1746 | 1771 | ||
| 1747 | /* Return TRUE if INSN has defined any 32-bit value explicitly. */ | 1772 | /* Return TRUE if INSN has defined any 32-bit value explicitly. */ |
| 1748 | static bool insn_has_def32(struct bpf_verifier_env *env, struct bpf_insn *insn) | 1773 | static bool insn_has_def32(struct bpf_verifier_env *env, struct bpf_insn *insn) |
| 1749 | { | 1774 | { |
| 1750 | if (insn_no_def(insn)) | 1775 | int dst_reg = insn_def_regno(insn); |
| 1776 | |||
| 1777 | if (dst_reg == -1) | ||
| 1751 | return false; | 1778 | return false; |
| 1752 | 1779 | ||
| 1753 | return !is_reg64(env, insn, insn->dst_reg, NULL, DST_OP); | 1780 | return !is_reg64(env, insn, dst_reg, NULL, DST_OP); |
| 1754 | } | 1781 | } |
| 1755 | 1782 | ||
| 1756 | static void mark_insn_zext(struct bpf_verifier_env *env, | 1783 | static void mark_insn_zext(struct bpf_verifier_env *env, |
| @@ -11006,9 +11033,10 @@ static int opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env, · 9249 unmodified lines | |||
| 11006 | for (i = 0; i < len; i++) { | 11033 | for (i = 0; i < len; i++) { |
| 11007 | int adj_idx = i + delta; | 11034 | int adj_idx = i + delta; |
| 11008 | struct bpf_insn insn; | 11035 | struct bpf_insn insn; |
| 11009 | u8 load_reg; | 11036 | int load_reg; |
| 11010 | 11037 | ||
| 11011 | insn = insns[adj_idx]; | 11038 | insn = insns[adj_idx]; |
| 11039 | load_reg = insn_def_regno(&insn); | ||
| 11012 | if (!aux[adj_idx].zext_dst) { | 11040 | if (!aux[adj_idx].zext_dst) { |
| 11013 | u8 code, class; | 11041 | u8 code, class; |
| 11014 | u32 imm_rnd; | 11042 | u32 imm_rnd; |
| @@ -11018,14 +11046,14 @@ static int opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env, · 3 unmodified lines | |||
| 11018 | 11046 | ||
| 11019 | code = insn.code; | 11047 | code = insn.code; |
| 11020 | class = BPF_CLASS(code); | 11048 | class = BPF_CLASS(code); |
| 11021 | if (insn_no_def(&insn)) | 11049 | if (load_reg == -1) |
| 11022 | continue; | 11050 | continue; |
| 11023 | 11051 | ||
| 11024 | /* NOTE: arg "reg" (the fourth one) is only used for | 11052 | /* NOTE: arg "reg" (the fourth one) is only used for |
| 11025 | * BPF_STX which has been ruled out in above | 11053 | * BPF_STX + SRC_OP, so it is safe to pass NULL |
| 11026 | * check, it is safe to pass NULL here. | 11054 | * here. |
| 11027 | */ | 11055 | */ |
| 11028 | if (is_reg64(env, &insn, insn.dst_reg, NULL, DST_OP)) { | 11056 | if (is_reg64(env, &insn, load_reg, NULL, DST_OP)) { |
| 11029 | if (class == BPF_LD && | 11057 | if (class == BPF_LD && |
| 11030 | BPF_MODE(code) == BPF_IMM) | 11058 | BPF_MODE(code) == BPF_IMM) |
| 11031 | i++; | 11059 | i++; |
| @@ -11040,31 +11068,28 @@ static int opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env, · 8 unmodified lines | |||
| 11040 | imm_rnd = get_random_int(); | 11068 | imm_rnd = get_random_int(); |
| 11041 | rnd_hi32_patch[0] = insn; | 11069 | rnd_hi32_patch[0] = insn; |
| 11042 | rnd_hi32_patch[1].imm = imm_rnd; | 11070 | rnd_hi32_patch[1].imm = imm_rnd; |
| 11043 | rnd_hi32_patch[3].dst_reg = insn.dst_reg; | 11071 | rnd_hi32_patch[3].dst_reg = load_reg; |
| 11044 | patch = rnd_hi32_patch; | 11072 | patch = rnd_hi32_patch; |
| 11045 | patch_len = 4; | 11073 | patch_len = 4; |
| 11046 | goto apply_patch_buffer; | 11074 | goto apply_patch_buffer; |
| 11047 | } | 11075 | } |
| 11048 | 11076 | ||
| 11049 | if (!bpf_jit_needs_zext()) | 11077 | /* Add in an zero-extend instruction if a) the JIT has requested |
| 11078 | * it or b) it's a CMPXCHG. | ||
| 11079 | * | ||
| 11080 | * The latter is because: BPF_CMPXCHG always loads a value into | ||
| 11081 | * R0, therefore always zero-extends. However some archs' | ||
| 11082 | * equivalent instruction only does this load when the | ||
| 11083 | * comparison is successful. This detail of CMPXCHG is | ||
| 11084 | * orthogonal to the general zero-extension behaviour of the | ||
| 11085 | * CPU, so it's treated independently of bpf_jit_needs_zext. | ||
| 11086 | */ | ||
| 11087 | if (!bpf_jit_needs_zext() && !is_cmpxchg_insn(&insn)) | ||
| 11050 | continue; | 11088 | continue; |
| 11051 | 11089 | ||
| 11052 | /* zext_dst means that we want to zero-extend whatever register | 11090 | if (WARN_ON(load_reg == -1)) { |
| 11053 | * the insn defines, which is dst_reg most of the time, with | 11091 | verbose(env, "verifier bug. zext_dst is set, but no reg is defined\n"); |
| 11054 | * the notable exception of BPF_STX + BPF_ATOMIC + BPF_FETCH. | 11092 | return -EFAULT; |
| 11055 | */ | ||
| 11056 | if (BPF_CLASS(insn.code) == BPF_STX && | ||
| 11057 | BPF_MODE(insn.code) == BPF_ATOMIC) { | ||
| 11058 | /* BPF_STX + BPF_ATOMIC insns without BPF_FETCH do not | ||
| 11059 | * define any registers, therefore zext_dst cannot be | ||
| 11060 | * set. | ||
| 11061 | */ | ||
| 11062 | if (WARN_ON(!(insn.imm & BPF_FETCH))) | ||
| 11063 | return -EINVAL; | ||
| 11064 | load_reg = insn.imm == BPF_CMPXCHG ? BPF_REG_0 | ||
| 11065 | : insn.src_reg; | ||
| 11066 | } else { | ||
| 11067 | load_reg = insn.dst_reg; | ||
| 11068 | } | 11093 | } |
| 11069 | 11094 | ||
| 11070 | zext_patch[0] = insn; | 11095 | zext_patch[0] = insn; |
| @@ -386,6 +386,7 @@ static DEFINE_MUTEX(perf_sched_mutex); · 385 unmodified lines | |||
| 386 | static atomic_t perf_sched_count; | 386 | static atomic_t perf_sched_count; |
| 387 | 387 | ||
| 388 | static DEFINE_PER_CPU(atomic_t, perf_cgroup_events); | 388 | static DEFINE_PER_CPU(atomic_t, perf_cgroup_events); |
| 389 | static DEFINE_PER_CPU(int, perf_sched_cb_usages); | ||
| 389 | static DEFINE_PER_CPU(struct pmu_event_list, pmu_sb_events); | 390 | static DEFINE_PER_CPU(struct pmu_event_list, pmu_sb_events); |
| 390 | 391 | ||
| 391 | static atomic_t nr_mmap_events __read_mostly; | 392 | static atomic_t nr_mmap_events __read_mostly; |
| @@ -3461,11 +3462,16 @@ unlock: · 3069 unmodified lines | |||
| 3461 | } | 3462 | } |
| 3462 | } | 3463 | } |
| 3463 | 3464 | ||
| 3465 | static DEFINE_PER_CPU(struct list_head, sched_cb_list); | ||
| 3466 | |||
| 3464 | void perf_sched_cb_dec(struct pmu *pmu) | 3467 | void perf_sched_cb_dec(struct pmu *pmu) |
| 3465 | { | 3468 | { |
| 3466 | struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context); | 3469 | struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context); |
| 3467 | 3470 | ||
| 3468 | --cpuctx->sched_cb_usage; | 3471 | this_cpu_dec(perf_sched_cb_usages); |
| 3472 | |||
| 3473 | if (!--cpuctx->sched_cb_usage) | ||
| 3474 | list_del(&cpuctx->sched_cb_entry); | ||
| 3469 | } | 3475 | } |
| 3470 | 3476 | ||
| 3471 | 3477 | ||
| @@ -3473,7 +3479,10 @@ void perf_sched_cb_inc(struct pmu *pmu) · 1 unmodified line | |||
| 3473 | { | 3479 | { |
| 3474 | struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context); | 3480 | struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context); |
| 3475 | 3481 | ||
| 3476 | cpuctx->sched_cb_usage++; | 3482 | if (!cpuctx->sched_cb_usage++) |
| 3483 | list_add(&cpuctx->sched_cb_entry, this_cpu_ptr(&sched_cb_list)); | ||
| 3484 | |||
| 3485 | this_cpu_inc(perf_sched_cb_usages); | ||
| 3477 | } | 3486 | } |
| 3478 | 3487 | ||
| 3479 | /* | 3488 | /* |
| @@ -3502,6 +3511,24 @@ static void __perf_pmu_sched_task(struct perf_cpu_context *cpuctx, bool sched_in · 22 unmodified lines | |||
| 3502 | perf_ctx_unlock(cpuctx, cpuctx->task_ctx); | 3511 | perf_ctx_unlock(cpuctx, cpuctx->task_ctx); |
| 3503 | } | 3512 | } |
| 3504 | 3513 | ||
| 3514 | static void perf_pmu_sched_task(struct task_struct *prev, | ||
| 3515 | struct task_struct *next, | ||
| 3516 | bool sched_in) | ||
| 3517 | { | ||
| 3518 | struct perf_cpu_context *cpuctx; | ||
| 3519 | |||
| 3520 | if (prev == next) | ||
| 3521 | return; | ||
| 3522 | |||
| 3523 | list_for_each_entry(cpuctx, this_cpu_ptr(&sched_cb_list), sched_cb_entry) { | ||
| 3524 | /* will be handled in perf_event_context_sched_in/out */ | ||
| 3525 | if (cpuctx->task_ctx) | ||
| 3526 | continue; | ||
| 3527 | |||
| 3528 | __perf_pmu_sched_task(cpuctx, sched_in); | ||
| 3529 | } | ||
| 3530 | } | ||
| 3531 | |||
| 3505 | static void perf_event_switch(struct task_struct *task, | 3532 | static void perf_event_switch(struct task_struct *task, |
| 3506 | struct task_struct *next_prev, bool sched_in); | 3533 | struct task_struct *next_prev, bool sched_in); |
| 3507 | 3534 | ||
| @@ -3524,6 +3551,9 @@ void __perf_event_task_sched_out(struct task_struct *task, · 16 unmodified lines | |||
| 3524 | { | 3551 | { |
| 3525 | int ctxn; | 3552 | int ctxn; |
| 3526 | 3553 | ||
| 3554 | if (__this_cpu_read(perf_sched_cb_usages)) | ||
| 3555 | perf_pmu_sched_task(task, next, false); | ||
| 3556 | |||
| 3527 | if (atomic_read(&nr_switch_events)) | 3557 | if (atomic_read(&nr_switch_events)) |
| 3528 | perf_event_switch(task, next, false); | 3558 | perf_event_switch(task, next, false); |
| 3529 | 3559 | ||
| @@ -3832,6 +3862,9 @@ void __perf_event_task_sched_in(struct task_struct *prev, · 302 unmodified lines | |||
| 3832 | 3862 | ||
| 3833 | if (atomic_read(&nr_switch_events)) | 3863 | if (atomic_read(&nr_switch_events)) |
| 3834 | perf_event_switch(task, prev, true); | 3864 | perf_event_switch(task, prev, true); |
| 3865 | |||
| 3866 | if (__this_cpu_read(perf_sched_cb_usages)) | ||
| 3867 | perf_pmu_sched_task(prev, task, true); | ||
| 3835 | } | 3868 | } |
| 3836 | 3869 | ||
| 3837 | static u64 perf_calculate_period(struct perf_event *event, u64 nsec, u64 count) | 3870 | static u64 perf_calculate_period(struct perf_event *event, u64 nsec, u64 count) |
| @@ -4656,7 +4689,7 @@ static void unaccount_event(struct perf_event *event) · 818 unmodified lines | |||
| 4656 | if (event->parent) | 4689 | if (event->parent) |
| 4657 | return; | 4690 | return; |
| 4658 | 4691 | ||
| 4659 | if (event->attach_state & PERF_ATTACH_TASK) | 4692 | if (event->attach_state & (PERF_ATTACH_TASK | PERF_ATTACH_SCHED_CB)) |
| 4660 | dec = true; | 4693 | dec = true; |
| 4661 | if (event->attr.mmap || event->attr.mmap_data) | 4694 | if (event->attr.mmap || event->attr.mmap_data) |
| 4662 | atomic_dec(&nr_mmap_events); | 4695 | atomic_dec(&nr_mmap_events); |
| @@ -11175,7 +11208,7 @@ static void account_event(struct perf_event *event) · 6512 unmodified lines | |||
| 11175 | if (event->parent) | 11208 | if (event->parent) |
| 11176 | return; | 11209 | return; |
| 11177 | 11210 | ||
| 11178 | if (event->attach_state & PERF_ATTACH_TASK) | 11211 | if (event->attach_state & (PERF_ATTACH_TASK | PERF_ATTACH_SCHED_CB)) |
| 11179 | inc = true; | 11212 | inc = true; |
| 11180 | if (event->attr.mmap || event->attr.mmap_data) | 11213 | if (event->attr.mmap || event->attr.mmap_data) |
| 11181 | atomic_inc(&nr_mmap_events); | 11214 | atomic_inc(&nr_mmap_events); |
| @@ -12972,6 +13005,7 @@ static void __init perf_event_init_all_cpus(void) · 1790 unmodified lines | |||
| 12972 | #ifdef CONFIG_CGROUP_PERF | 13005 | #ifdef CONFIG_CGROUP_PERF |
| 12973 | INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu)); | 13006 | INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu)); |
| 12974 | #endif | 13007 | #endif |
| 13008 | INIT_LIST_HEAD(&per_cpu(sched_cb_list, cpu)); | ||
| 12975 | } | 13009 | } |
| 12976 | } | 13010 | } |
| 12977 | 13011 | ||
| @@ -994,6 +994,13 @@ static void mm_init_owner(struct mm_struct *mm, struct task_struct *p) · 993 unmodified lines | |||
| 994 | #endif | 994 | #endif |
| 995 | } | 995 | } |
| 996 | 996 | ||
| 997 | static void mm_init_pasid(struct mm_struct *mm) | ||
| 998 | { | ||
| 999 | #ifdef CONFIG_IOMMU_SUPPORT | ||
| 1000 | mm->pasid = INIT_PASID; | ||
| 1001 | #endif | ||
| 1002 | } | ||
| 1003 | |||
| 997 | static void mm_init_uprobes_state(struct mm_struct *mm) | 1004 | static void mm_init_uprobes_state(struct mm_struct *mm) |
| 998 | { | 1005 | { |
| 999 | #ifdef CONFIG_UPROBES | 1006 | #ifdef CONFIG_UPROBES |
| @@ -1024,6 +1031,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, · 24 unmodified lines | |||
| 1024 | mm_init_cpumask(mm); | 1031 | mm_init_cpumask(mm); |
| 1025 | mm_init_aio(mm); | 1032 | mm_init_aio(mm); |
| 1026 | mm_init_owner(mm, p); | 1033 | mm_init_owner(mm, p); |
| 1034 | mm_init_pasid(mm); | ||
| 1027 | RCU_INIT_POINTER(mm->exe_file, NULL); | 1035 | RCU_INIT_POINTER(mm->exe_file, NULL); |
| 1028 | mmu_notifier_subscriptions_init(mm); | 1036 | mmu_notifier_subscriptions_init(mm); |
| 1029 | init_tlb_flush_pending(mm); | 1037 | init_tlb_flush_pending(mm); |
| @@ -2436,6 +2444,7 @@ struct task_struct *create_io_thread(int (*fn)(void *), void *arg, int node) · 1406 unmodified lines | |||
| 2436 | if (!IS_ERR(tsk)) { | 2444 | if (!IS_ERR(tsk)) { |
| 2437 | sigfillset(&tsk->blocked); | 2445 | sigfillset(&tsk->blocked); |
| 2438 | sigdelsetmask(&tsk->blocked, sigmask(SIGKILL)); | 2446 | sigdelsetmask(&tsk->blocked, sigmask(SIGKILL)); |
| 2447 | tsk->flags |= PF_NOFREEZE; | ||
| 2439 | } | 2448 | } |
| 2440 | return tsk; | 2449 | return tsk; |
| 2441 | } | 2450 | } |
| @@ -1898,16 +1898,15 @@ DEFINE_SHOW_ATTRIBUTE(irq_domain_debug); · 1897 unmodified lines | |||
| 1898 | 1898 | ||
| 1899 | static void debugfs_add_domain_dir(struct irq_domain *d) | 1899 | static void debugfs_add_domain_dir(struct irq_domain *d) |
| 1900 | { | 1900 | { |
| 1901 | if (!d->name || !domain_dir || d->debugfs_file) | 1901 | if (!d->name || !domain_dir) |
| 1902 | return; | 1902 | return; |
| 1903 | d->debugfs_file = debugfs_create_file(d->name, 0444, domain_dir, d, | 1903 | debugfs_create_file(d->name, 0444, domain_dir, d, |
| 1904 | &irq_domain_debug_fops); | 1904 | &irq_domain_debug_fops); |
| 1905 | } | 1905 | } |
| 1906 | 1906 | ||
| 1907 | static void debugfs_remove_domain_dir(struct irq_domain *d) | 1907 | static void debugfs_remove_domain_dir(struct irq_domain *d) |
| 1908 | { | 1908 | { |
| 1909 | debugfs_remove(d->debugfs_file); | 1909 | debugfs_remove(debugfs_lookup(d->name, domain_dir)); |
| 1910 | d->debugfs_file = NULL; | ||
| 1911 | } | 1910 | } |
| 1912 | 1911 | ||
| 1913 | void __init irq_domain_debugfs_init(struct dentry *root) | 1912 | void __init irq_domain_debugfs_init(struct dentry *root) |
| @@ -626,7 +626,7 @@ static inline int mutex_can_spin_on_owner(struct mutex *lock) · 625 unmodified lines | |||
| 626 | */ | 626 | */ |
| 627 | static __always_inline bool | 627 | static __always_inline bool |
| 628 | mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx, | 628 | mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx, |
| 629 | const bool use_ww_ctx, struct mutex_waiter *waiter) | 629 | struct mutex_waiter *waiter) |
| 630 | { | 630 | { |
| 631 | if (!waiter) { | 631 | if (!waiter) { |
| 632 | /* | 632 | /* |
| @@ -702,7 +702,7 @@ fail: · 69 unmodified lines | |||
| 702 | #else | 702 | #else |
| 703 | static __always_inline bool | 703 | static __always_inline bool |
| 704 | mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx, | 704 | mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx, |
| 705 | const bool use_ww_ctx, struct mutex_waiter *waiter) | 705 | struct mutex_waiter *waiter) |
| 706 | { | 706 | { |
| 707 | return false; | 707 | return false; |
| 708 | } | 708 | } |
| @@ -922,6 +922,9 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 213 unmodified lines | |||
| 922 | struct ww_mutex *ww; | 922 | struct ww_mutex *ww; |
| 923 | int ret; | 923 | int ret; |
| 924 | 924 | ||
| 925 | if (!use_ww_ctx) | ||
| 926 | ww_ctx = NULL; | ||
| 927 | |||
| 925 | might_sleep(); | 928 | might_sleep(); |
| 926 | 929 | ||
| 927 | #ifdef CONFIG_DEBUG_MUTEXES | 930 | #ifdef CONFIG_DEBUG_MUTEXES |
| @@ -929,7 +932,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 1 unmodified line | |||
| 929 | #endif | 932 | #endif |
| 930 | 933 | ||
| 931 | ww = container_of(lock, struct ww_mutex, base); | 934 | ww = container_of(lock, struct ww_mutex, base); |
| 932 | if (use_ww_ctx && ww_ctx) { | 935 | if (ww_ctx) { |
| 933 | if (unlikely(ww_ctx == READ_ONCE(ww->ctx))) | 936 | if (unlikely(ww_ctx == READ_ONCE(ww->ctx))) |
| 934 | return -EALREADY; | 937 | return -EALREADY; |
| 935 | 938 | ||
| @@ -946,10 +949,10 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 10 unmodified lines | |||
| 946 | mutex_acquire_nest(&lock->dep_map, subclass, 0, nest_lock, ip); | 949 | mutex_acquire_nest(&lock->dep_map, subclass, 0, nest_lock, ip); |
| 947 | 950 | ||
| 948 | if (__mutex_trylock(lock) || | 951 | if (__mutex_trylock(lock) || |
| 949 | mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, NULL)) { | 952 | mutex_optimistic_spin(lock, ww_ctx, NULL)) { |
| 950 | /* got the lock, yay! */ | 953 | /* got the lock, yay! */ |
| 951 | lock_acquired(&lock->dep_map, ip); | 954 | lock_acquired(&lock->dep_map, ip); |
| 952 | if (use_ww_ctx && ww_ctx) | 955 | if (ww_ctx) |
| 953 | ww_mutex_set_context_fastpath(ww, ww_ctx); | 956 | ww_mutex_set_context_fastpath(ww, ww_ctx); |
| 954 | preempt_enable(); | 957 | preempt_enable(); |
| 955 | return 0; | 958 | return 0; |
| @@ -960,7 +963,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 4 unmodified lines | |||
| 960 | * After waiting to acquire the wait_lock, try again. | 963 | * After waiting to acquire the wait_lock, try again. |
| 961 | */ | 964 | */ |
| 962 | if (__mutex_trylock(lock)) { | 965 | if (__mutex_trylock(lock)) { |
| 963 | if (use_ww_ctx && ww_ctx) | 966 | if (ww_ctx) |
| 964 | __ww_mutex_check_waiters(lock, ww_ctx); | 967 | __ww_mutex_check_waiters(lock, ww_ctx); |
| 965 | 968 | ||
| 966 | goto skip_wait; | 969 | goto skip_wait; |
| @@ -1013,7 +1016,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 46 unmodified lines | |||
| 1013 | goto err; | 1016 | goto err; |
| 1014 | } | 1017 | } |
| 1015 | 1018 | ||
| 1016 | if (use_ww_ctx && ww_ctx) { | 1019 | if (ww_ctx) { |
| 1017 | ret = __ww_mutex_check_kill(lock, &waiter, ww_ctx); | 1020 | ret = __ww_mutex_check_kill(lock, &waiter, ww_ctx); |
| 1018 | if (ret) | 1021 | if (ret) |
| 1019 | goto err; | 1022 | goto err; |
| @@ -1026,7 +1029,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 6 unmodified lines | |||
| 1026 | * ww_mutex needs to always recheck its position since its waiter | 1029 | * ww_mutex needs to always recheck its position since its waiter |
| 1027 | * list is not FIFO ordered. | 1030 | * list is not FIFO ordered. |
| 1028 | */ | 1031 | */ |
| 1029 | if ((use_ww_ctx && ww_ctx) || !first) { | 1032 | if (ww_ctx || !first) { |
| 1030 | first = __mutex_waiter_is_first(lock, &waiter); | 1033 | first = __mutex_waiter_is_first(lock, &waiter); |
| 1031 | if (first) | 1034 | if (first) |
| 1032 | __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); | 1035 | __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); |
| @@ -1039,7 +1042,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 6 unmodified lines | |||
| 1039 | * or we must see its unlock and acquire. | 1042 | * or we must see its unlock and acquire. |
| 1040 | */ | 1043 | */ |
| 1041 | if (__mutex_trylock(lock) || | 1044 | if (__mutex_trylock(lock) || |
| 1042 | (first && mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, &waiter))) | 1045 | (first && mutex_optimistic_spin(lock, ww_ctx, &waiter))) |
| 1043 | break; | 1046 | break; |
| 1044 | 1047 | ||
| 1045 | spin_lock(&lock->wait_lock); | 1048 | spin_lock(&lock->wait_lock); |
| @@ -1048,7 +1051,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, · 2 unmodified lines | |||
| 1048 | acquired: | 1051 | acquired: |
| 1049 | __set_current_state(TASK_RUNNING); | 1052 | __set_current_state(TASK_RUNNING); |
| 1050 | 1053 | ||
| 1051 | if (use_ww_ctx && ww_ctx) { | 1054 | if (ww_ctx) { |
| 1052 | /* | 1055 | /* |
| 1053 | * Wound-Wait; we stole the lock (!first_waiter), check the | 1056 | * Wound-Wait; we stole the lock (!first_waiter), check the |
| 1054 | * waiters as anyone might want to wound us. | 1057 | * waiters as anyone might want to wound us. |
| @@ -1068,7 +1071,7 @@ skip_wait: · 13 unmodified lines | |||
| 1068 | /* got the lock - cleanup and rejoice! */ | 1071 | /* got the lock - cleanup and rejoice! */ |
| 1069 | lock_acquired(&lock->dep_map, ip); | 1072 | lock_acquired(&lock->dep_map, ip); |
| 1070 | 1073 | ||
| 1071 | if (use_ww_ctx && ww_ctx) | 1074 | if (ww_ctx) |
| 1072 | ww_mutex_lock_acquired(ww, ww_ctx); | 1075 | ww_mutex_lock_acquired(ww, ww_ctx); |
| 1073 | 1076 | ||
| 1074 | spin_unlock(&lock->wait_lock); | 1077 | spin_unlock(&lock->wait_lock); |
| @@ -1862,8 +1862,13 @@ struct migration_arg { · 1861 unmodified lines | |||
| 1862 | struct set_affinity_pending *pending; | 1862 | struct set_affinity_pending *pending; |
| 1863 | }; | 1863 | }; |
| 1864 | 1864 | ||
| 1865 | /* | ||
| 1866 | * @refs: number of wait_for_completion() | ||
| 1867 | * @stop_pending: is @stop_work in use | ||
| 1868 | */ | ||
| 1865 | struct set_affinity_pending { | 1869 | struct set_affinity_pending { |
| 1866 | refcount_t refs; | 1870 | refcount_t refs; |
| 1871 | unsigned int stop_pending; | ||
| 1867 | struct completion done; | 1872 | struct completion done; |
| 1868 | struct cpu_stop_work stop_work; | 1873 | struct cpu_stop_work stop_work; |
| 1869 | struct migration_arg arg; | 1874 | struct migration_arg arg; |
| @@ -1898,8 +1903,8 @@ static struct rq *__migrate_task(struct rq *rq, struct rq_flags *rf, · 28 unmodified lines | |||
| 1898 | */ | 1903 | */ |
| 1899 | static int migration_cpu_stop(void *data) | 1904 | static int migration_cpu_stop(void *data) |
| 1900 | { | 1905 | { |
| 1901 | struct set_affinity_pending *pending; | ||
| 1902 | struct migration_arg *arg = data; | 1906 | struct migration_arg *arg = data; |
| 1907 | struct set_affinity_pending *pending = arg->pending; | ||
| 1903 | struct task_struct *p = arg->task; | 1908 | struct task_struct *p = arg->task; |
| 1904 | int dest_cpu = arg->dest_cpu; | 1909 | int dest_cpu = arg->dest_cpu; |
| 1905 | struct rq *rq = this_rq(); | 1910 | struct rq *rq = this_rq(); |
| @@ -1921,7 +1926,6 @@ static int migration_cpu_stop(void *data) · 15 unmodified lines | |||
| 1921 | raw_spin_lock(&p->pi_lock); | 1926 | raw_spin_lock(&p->pi_lock); |
| 1922 | rq_lock(rq, &rf); | 1927 | rq_lock(rq, &rf); |
| 1923 | 1928 | ||
| 1924 | pending = p->migration_pending; | ||
| 1925 | /* | 1929 | /* |
| 1926 | * If task_rq(p) != rq, it cannot be migrated here, because we're | 1930 | * If task_rq(p) != rq, it cannot be migrated here, because we're |
| 1927 | * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because | 1931 | * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because |
| @@ -1932,21 +1936,14 @@ static int migration_cpu_stop(void *data) · 4 unmodified lines | |||
| 1932 | goto out; | 1936 | goto out; |
| 1933 | 1937 | ||
| 1934 | if (pending) { | 1938 | if (pending) { |
| 1935 | p->migration_pending = NULL; | 1939 | if (p->migration_pending == pending) |
| 1940 | p->migration_pending = NULL; | ||
| 1936 | complete = true; | 1941 | complete = true; |
| 1937 | } | 1942 | } |
| 1938 | 1943 | ||
| 1939 | /* migrate_enable() -- we must not race against SCA */ | ||
| 1940 | if (dest_cpu < 0) { | 1944 | if (dest_cpu < 0) { |
| 1941 | /* | 1945 | if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) |
| 1942 | * When this was migrate_enable() but we no longer | ||
| 1943 | * have a @pending, a concurrent SCA 'fixed' things | ||
| 1944 | * and we should be valid again. Nothing to do. | ||
| 1945 | */ | ||
| 1946 | if (!pending) { | ||
| 1947 | WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask)); | ||
| 1948 | goto out; | 1946 | goto out; |
| 1949 | } | ||
| 1950 | 1947 | ||
| 1951 | dest_cpu = cpumask_any_distribute(&p->cpus_mask); | 1948 | dest_cpu = cpumask_any_distribute(&p->cpus_mask); |
| 1952 | } | 1949 | } |
| @@ -1956,7 +1953,14 @@ static int migration_cpu_stop(void *data) · 3 unmodified lines | |||
| 1956 | else | 1953 | else |
| 1957 | p->wake_cpu = dest_cpu; | 1954 | p->wake_cpu = dest_cpu; |
| 1958 | 1955 | ||
| 1959 | } else if (dest_cpu < 0 || pending) { | 1956 | /* |
| 1957 | * XXX __migrate_task() can fail, at which point we might end | ||
| 1958 | * up running on a dodgy CPU, AFAICT this can only happen | ||
| 1959 | * during CPU hotplug, at which point we'll get pushed out | ||
| 1960 | * anyway, so it's probably not a big deal. | ||
| 1961 | */ | ||
| 1962 | |||
| 1963 | } else if (pending) { | ||
| 1960 | /* | 1964 | /* |
| 1961 | * This happens when we get migrated between migrate_enable()'s | 1965 | * This happens when we get migrated between migrate_enable()'s |
| 1962 | * preempt_enable() and scheduling the stopper task. At that | 1966 | * preempt_enable() and scheduling the stopper task. At that |
| @@ -1971,43 +1975,32 @@ static int migration_cpu_stop(void *data) · 8 unmodified lines | |||
| 1971 | * ->pi_lock, so the allowed mask is stable - if it got | 1975 | * ->pi_lock, so the allowed mask is stable - if it got |
| 1972 | * somewhere allowed, we're done. | 1976 | * somewhere allowed, we're done. |
| 1973 | */ | 1977 | */ |
| 1974 | if (pending && cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) { | 1978 | if (cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) { |
| 1975 | p->migration_pending = NULL; | 1979 | if (p->migration_pending == pending) |
| 1980 | p->migration_pending = NULL; | ||
| 1976 | complete = true; | 1981 | complete = true; |
| 1977 | goto out; | 1982 | goto out; |
| 1978 | } | 1983 | } |
| 1979 | 1984 | ||
| 1980 | /* | ||
| 1981 | * When this was migrate_enable() but we no longer have an | ||
| 1982 | * @pending, a concurrent SCA 'fixed' things and we should be | ||
| 1983 | * valid again. Nothing to do. | ||
| 1984 | */ | ||
| 1985 | if (!pending) { | ||
| 1986 | WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask)); | ||
| 1987 | goto out; | ||
| 1988 | } | ||
| 1989 | |||
| 1990 | /* | 1985 | /* |
| 1991 | * When migrate_enable() hits a rq mis-match we can't reliably | 1986 | * When migrate_enable() hits a rq mis-match we can't reliably |
| 1992 | * determine is_migration_disabled() and so have to chase after | 1987 | * determine is_migration_disabled() and so have to chase after |
| 1993 | * it. | 1988 | * it. |
| 1994 | */ | 1989 | */ |
| 1990 | WARN_ON_ONCE(!pending->stop_pending); | ||
| 1995 | task_rq_unlock(rq, p, &rf); | 1991 | task_rq_unlock(rq, p, &rf); |
| 1996 | stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop, | 1992 | stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop, |
| 1997 | &pending->arg, &pending->stop_work); | 1993 | &pending->arg, &pending->stop_work); |
| 1998 | return 0; | 1994 | return 0; |
| 1999 | } | 1995 | } |
| 2000 | out: | 1996 | out: |
| 1997 | if (pending) | ||
| 1998 | pending->stop_pending = false; | ||
| 2001 | task_rq_unlock(rq, p, &rf); | 1999 | task_rq_unlock(rq, p, &rf); |
| 2002 | 2000 | ||
| 2003 | if (complete) | 2001 | if (complete) |
| 2004 | complete_all(&pending->done); | 2002 | complete_all(&pending->done); |
| 2005 | 2003 | ||
| 2006 | /* For pending->{arg,stop_work} */ | ||
| 2007 | pending = arg->pending; | ||
| 2008 | if (pending && refcount_dec_and_test(&pending->refs)) | ||
| 2009 | wake_up_var(&pending->refs); | ||
| 2010 | |||
| 2011 | return 0; | 2004 | return 0; |
| 2012 | } | 2005 | } |
| 2013 | 2006 | ||
| @@ -2194,11 +2187,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag · 180 unmodified lines | |||
| 2194 | int dest_cpu, unsigned int flags) | 2187 | int dest_cpu, unsigned int flags) |
| 2195 | { | 2188 | { |
| 2196 | struct set_affinity_pending my_pending = { }, *pending = NULL; | 2189 | struct set_affinity_pending my_pending = { }, *pending = NULL; |
| 2197 | struct migration_arg arg = { | 2190 | bool stop_pending, complete = false; |
| 2198 | .task = p, | ||
| 2199 | .dest_cpu = dest_cpu, | ||
| 2200 | }; | ||
| 2201 | bool complete = false; | ||
| 2202 | 2191 | ||
| 2203 | /* Can the task run on the task's current CPU? If so, we're done */ | 2192 | /* Can the task run on the task's current CPU? If so, we're done */ |
| 2204 | if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) { | 2193 | if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) { |
| @@ -2210,12 +2199,16 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag · 5 unmodified lines | |||
| 2210 | push_task = get_task_struct(p); | 2199 | push_task = get_task_struct(p); |
| 2211 | } | 2200 | } |
| 2212 | 2201 | ||
| 2202 | /* | ||
| 2203 | * If there are pending waiters, but no pending stop_work, | ||
| 2204 | * then complete now. | ||
| 2205 | */ | ||
| 2213 | pending = p->migration_pending; | 2206 | pending = p->migration_pending; |
| 2214 | if (pending) { | 2207 | if (pending && !pending->stop_pending) { |
| 2215 | refcount_inc(&pending->refs); | ||
| 2216 | p->migration_pending = NULL; | 2208 | p->migration_pending = NULL; |
| 2217 | complete = true; | 2209 | complete = true; |
| 2218 | } | 2210 | } |
| 2211 | |||
| 2219 | task_rq_unlock(rq, p, rf); | 2212 | task_rq_unlock(rq, p, rf); |
| 2220 | 2213 | ||
| 2221 | if (push_task) { | 2214 | if (push_task) { |
| @@ -2224,7 +2217,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag · 2 unmodified lines | |||
| 2224 | } | 2217 | } |
| 2225 | 2218 | ||
| 2226 | if (complete) | 2219 | if (complete) |
| 2227 | goto do_complete; | 2220 | complete_all(&pending->done); |
| 2228 | 2221 | ||
| 2229 | return 0; | 2222 | return 0; |
| 2230 | } | 2223 | } |
| @@ -2235,6 +2228,12 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag · 4 unmodified lines | |||
| 2235 | /* Install the request */ | 2228 | /* Install the request */ |
| 2236 | refcount_set(&my_pending.refs, 1); | 2229 | refcount_set(&my_pending.refs, 1); |
| 2237 | init_completion(&my_pending.done); | 2230 | init_completion(&my_pending.done); |
| 2231 | my_pending.arg = (struct migration_arg) { | ||
| 2232 | .task = p, | ||
| 2233 | .dest_cpu = -1, /* any */ | ||
| 2234 | .pending = &my_pending, | ||
| 2235 | }; | ||
| 2236 | |||
| 2238 | p->migration_pending = &my_pending; | 2237 | p->migration_pending = &my_pending; |
| 2239 | } else { | 2238 | } else { |
| 2240 | pending = p->migration_pending; | 2239 | pending = p->migration_pending; |
| @@ -2259,45 +2258,41 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag · 18 unmodified lines | |||
| 2259 | return -EINVAL; | 2258 | return -EINVAL; |
| 2260 | } | 2259 | } |
| 2261 | 2260 | ||
| 2262 | if (flags & SCA_MIGRATE_ENABLE) { | ||
| 2263 | |||
| 2264 | refcount_inc(&pending->refs); /* pending->{arg,stop_work} */ | ||
| 2265 | p->migration_flags &= ~MDF_PUSH; | ||
| 2266 | task_rq_unlock(rq, p, rf); | ||
| 2267 | |||
| 2268 | pending->arg = (struct migration_arg) { | ||
| 2269 | .task = p, | ||
| 2270 | .dest_cpu = -1, | ||
| 2271 | .pending = pending, | ||
| 2272 | }; | ||
| 2273 | |||
| 2274 | stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, | ||
| 2275 | &pending->arg, &pending->stop_work); | ||
| 2276 | |||
| 2277 | return 0; | ||
| 2278 | } | ||
| 2279 | |||
| 2280 | if (task_running(rq, p) || p->state == TASK_WAKING) { | 2261 | if (task_running(rq, p) || p->state == TASK_WAKING) { |
| 2281 | /* | 2262 | /* |
| 2282 | * Lessen races (and headaches) by delegating | 2263 | * MIGRATE_ENABLE gets here because 'p == current', but for |
| 2283 | * is_migration_disabled(p) checks to the stopper, which will | 2264 | * anything else we cannot do is_migration_disabled(), punt |
| 2284 | * run on the same CPU as said p. | 2265 | * and have the stopper function handle it all race-free. |
| 2285 | */ | 2266 | */ |
| 2267 | stop_pending = pending->stop_pending; | ||
| 2268 | if (!stop_pending) | ||
| 2269 | pending->stop_pending = true; | ||
| 2270 | |||
| 2271 | if (flags & SCA_MIGRATE_ENABLE) | ||
| 2272 | p->migration_flags &= ~MDF_PUSH; | ||
| 2273 | |||
| 2286 | task_rq_unlock(rq, p, rf); | 2274 | task_rq_unlock(rq, p, rf); |
| 2287 | stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); | ||
| 2288 | 2275 | ||
| 2276 | if (!stop_pending) { | ||
| 2277 | stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, | ||
| 2278 | &pending->arg, &pending->stop_work); | ||
| 2279 | } | ||
| 2280 | |||
| 2281 | if (flags & SCA_MIGRATE_ENABLE) | ||
| 2282 | return 0; | ||
| 2289 | } else { | 2283 | } else { |
| 2290 | 2284 | ||
| 2291 | if (!is_migration_disabled(p)) { | 2285 | if (!is_migration_disabled(p)) { |
| 2292 | if (task_on_rq_queued(p)) | 2286 | if (task_on_rq_queued(p)) |
| 2293 | rq = move_queued_task(rq, rf, p, dest_cpu); | 2287 | rq = move_queued_task(rq, rf, p, dest_cpu); |
| 2294 | 2288 | ||
| 2295 | p->migration_pending = NULL; | 2289 | if (!pending->stop_pending) { |
| 2296 | complete = true; | 2290 | p->migration_pending = NULL; |
| 2291 | complete = true; | ||
| 2292 | } | ||
| 2297 | } | 2293 | } |
| 2298 | task_rq_unlock(rq, p, rf); | 2294 | task_rq_unlock(rq, p, rf); |
| 2299 | 2295 | ||
| 2300 | do_complete: | ||
| 2301 | if (complete) | 2296 | if (complete) |
| 2302 | complete_all(&pending->done); | 2297 | complete_all(&pending->done); |
| 2303 | } | 2298 | } |
| @@ -2305,7 +2300,7 @@ do_complete: · 1 unmodified line | |||
| 2305 | wait_for_completion(&pending->done); | 2300 | wait_for_completion(&pending->done); |
| 2306 | 2301 | ||
| 2307 | if (refcount_dec_and_test(&pending->refs)) | 2302 | if (refcount_dec_and_test(&pending->refs)) |
| 2308 | wake_up_var(&pending->refs); | 2303 | wake_up_var(&pending->refs); /* No UaF, just an address */ |
| 2309 | 2304 | ||
| 2310 | /* | 2305 | /* |
| 2311 | * Block the original owner of &pending until all subsequent callers | 2306 | * Block the original owner of &pending until all subsequent callers |
| @@ -2313,6 +2308,9 @@ do_complete: · 1 unmodified line | |||
| 2313 | */ | 2308 | */ |
| 2314 | wait_var_event(&my_pending.refs, !refcount_read(&my_pending.refs)); | 2309 | wait_var_event(&my_pending.refs, !refcount_read(&my_pending.refs)); |
| 2315 | 2310 | ||
| 2311 | /* ARGH */ | ||
| 2312 | WARN_ON_ONCE(my_pending.stop_pending); | ||
| 2313 | |||
| 2316 | return 0; | 2314 | return 0; |
| 2317 | } | 2315 | } |
| 2318 | 2316 | ||
| @@ -471,9 +471,7 @@ static int sync_runqueues_membarrier_state(struct mm_struct *mm) · 470 unmodified lines | |||
| 471 | } | 471 | } |
| 472 | rcu_read_unlock(); | 472 | rcu_read_unlock(); |
| 473 | 473 | ||
| 474 | preempt_disable(); | 474 | on_each_cpu_mask(tmpmask, ipi_sync_rq_state, mm, true); |
| 475 | smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1); | ||
| 476 | preempt_enable(); | ||
| 477 | 475 | ||
| 478 | free_cpumask_var(tmpmask); | 476 | free_cpumask_var(tmpmask); |
| 479 | cpus_read_unlock(); | 477 | cpus_read_unlock(); |
| @@ -2079,7 +2079,7 @@ static int prctl_set_auxv(struct mm_struct *mm, unsigned long addr, · 2078 unmodified lines | |||
| 2079 | * up to the caller to provide sane values here, otherwise userspace | 2079 | * up to the caller to provide sane values here, otherwise userspace |
| 2080 | * tools which use this vector might be unhappy. | 2080 | * tools which use this vector might be unhappy. |
| 2081 | */ | 2081 | */ |
| 2082 | unsigned long user_auxv[AT_VECTOR_SIZE]; | 2082 | unsigned long user_auxv[AT_VECTOR_SIZE] = {}; |
| 2083 | 2083 | ||
| 2084 | if (len > sizeof(user_auxv)) | 2084 | if (len > sizeof(user_auxv)) |
| 2085 | return -EINVAL; | 2085 | return -EINVAL; |
| @@ -546,8 +546,11 @@ static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base, · 545 unmodified lines | |||
| 546 | } | 546 | } |
| 547 | 547 | ||
| 548 | /* | 548 | /* |
| 549 | * Recomputes cpu_base::*next_timer and returns the earliest expires_next but | 549 | * Recomputes cpu_base::*next_timer and returns the earliest expires_next |
| 550 | * does not set cpu_base::*expires_next, that is done by hrtimer_reprogram. | 550 | * but does not set cpu_base::*expires_next, that is done by |
| 551 | * hrtimer[_force]_reprogram and hrtimer_interrupt only. When updating | ||
| 552 | * cpu_base::*expires_next right away, reprogramming logic would no longer | ||
| 553 | * work. | ||
| 551 | * | 554 | * |
| 552 | * When a softirq is pending, we can ignore the HRTIMER_ACTIVE_SOFT bases, | 555 | * When a softirq is pending, we can ignore the HRTIMER_ACTIVE_SOFT bases, |
| 553 | * those timers will get run whenever the softirq gets handled, at the end of | 556 | * those timers will get run whenever the softirq gets handled, at the end of |
| @@ -588,6 +591,37 @@ __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base, unsigned int active_ · 34 unmodified lines | |||
| 588 | return expires_next; | 591 | return expires_next; |
| 589 | } | 592 | } |
| 590 | 593 | ||
| 594 | static ktime_t hrtimer_update_next_event(struct hrtimer_cpu_base *cpu_base) | ||
| 595 | { | ||
| 596 | ktime_t expires_next, soft = KTIME_MAX; | ||
| 597 | |||
| 598 | /* | ||
| 599 | * If the soft interrupt has already been activated, ignore the | ||
| 600 | * soft bases. They will be handled in the already raised soft | ||
| 601 | * interrupt. | ||
| 602 | */ | ||
| 603 | if (!cpu_base->softirq_activated) { | ||
| 604 | soft = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_SOFT); | ||
| 605 | /* | ||
| 606 | * Update the soft expiry time. clock_settime() might have | ||
| 607 | * affected it. | ||
| 608 | */ | ||
| 609 | cpu_base->softirq_expires_next = soft; | ||
| 610 | } | ||
| 611 | |||
| 612 | expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD); | ||
| 613 | /* | ||
| 614 | * If a softirq timer is expiring first, update cpu_base->next_timer | ||
| 615 | * and program the hardware with the soft expiry time. | ||
| 616 | */ | ||
| 617 | if (expires_next > soft) { | ||
| 618 | cpu_base->next_timer = cpu_base->softirq_next_timer; | ||
| 619 | expires_next = soft; | ||
| 620 | } | ||
| 621 | |||
| 622 | return expires_next; | ||
| 623 | } | ||
| 624 | |||
| 591 | static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base) | 625 | static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base) |
| 592 | { | 626 | { |
| 593 | ktime_t *offs_real = &base->clock_base[HRTIMER_BASE_REALTIME].offset; | 627 | ktime_t *offs_real = &base->clock_base[HRTIMER_BASE_REALTIME].offset; |
| @@ -628,23 +662,7 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal) · 34 unmodified lines | |||
| 628 | { | 662 | { |
| 629 | ktime_t expires_next; | 663 | ktime_t expires_next; |
| 630 | 664 | ||
| 631 | /* | 665 | expires_next = hrtimer_update_next_event(cpu_base); |
| 632 | * Find the current next expiration time. | ||
| 633 | */ | ||
| 634 | expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL); | ||
| 635 | |||
| 636 | if (cpu_base->next_timer && cpu_base->next_timer->is_soft) { | ||
| 637 | /* | ||
| 638 | * When the softirq is activated, hrtimer has to be | ||
| 639 | * programmed with the first hard hrtimer because soft | ||
| 640 | * timer interrupt could occur too late. | ||
| 641 | */ | ||
| 642 | if (cpu_base->softirq_activated) | ||
| 643 | expires_next = __hrtimer_get_next_event(cpu_base, | ||
| 644 | HRTIMER_ACTIVE_HARD); | ||
| 645 | else | ||
| 646 | cpu_base->softirq_expires_next = expires_next; | ||
| 647 | } | ||
| 648 | 666 | ||
| 649 | if (skip_equal && expires_next == cpu_base->expires_next) | 667 | if (skip_equal && expires_next == cpu_base->expires_next) |
| 650 | return; | 668 | return; |
| @@ -1644,8 +1662,8 @@ retry: · 993 unmodified lines | |||
| 1644 | 1662 | ||
| 1645 | __hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD); | 1663 | __hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD); |
| 1646 | 1664 | ||
| 1647 | /* Reevaluate the clock bases for the next expiry */ | 1665 | /* Reevaluate the clock bases for the [soft] next expiry */ |
| 1648 | expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL); | 1666 | expires_next = hrtimer_update_next_event(cpu_base); |
| 1649 | /* | 1667 | /* |
| 1650 | * Store the new expiry value so the migration code can verify | 1668 | * Store the new expiry value so the migration code can verify |
| 1651 | * against it. | 1669 | * against it. |
| @@ -156,6 +156,7 @@ config KASAN_STACK_ENABLE · 155 unmodified lines | |||
| 156 | 156 | ||
| 157 | config KASAN_STACK | 157 | config KASAN_STACK |
| 158 | int | 158 | int |
| 159 | depends on KASAN_GENERIC || KASAN_SW_TAGS | ||
| 159 | default 1 if KASAN_STACK_ENABLE || CC_IS_GCC | 160 | default 1 if KASAN_STACK_ENABLE || CC_IS_GCC |
| 160 | default 0 | 161 | default 0 |
| 161 | 162 | ||
| @@ -21,7 +21,6 @@ static inline unsigned long ex_to_insn(const struct exception_table_entry *x) · 20 unmodified lines | |||
| 21 | } | 21 | } |
| 22 | #endif | 22 | #endif |
| 23 | 23 | ||
| 24 | #ifndef ARCH_HAS_SORT_EXTABLE | ||
| 25 | #ifndef ARCH_HAS_RELATIVE_EXTABLE | 24 | #ifndef ARCH_HAS_RELATIVE_EXTABLE |
| 26 | #define swap_ex NULL | 25 | #define swap_ex NULL |
| 27 | #else | 26 | #else |
| @@ -88,9 +87,6 @@ void trim_init_extable(struct module *m) · 60 unmodified lines | |||
| 88 | m->num_exentries--; | 87 | m->num_exentries--; |
| 89 | } | 88 | } |
| 90 | #endif /* CONFIG_MODULES */ | 89 | #endif /* CONFIG_MODULES */ |
| 91 | #endif /* !ARCH_HAS_SORT_EXTABLE */ | ||
| 92 | |||
| 93 | #ifndef ARCH_HAS_SEARCH_EXTABLE | ||
| 94 | 90 | ||
| 95 | static int cmp_ex_search(const void *key, const void *elt) | 91 | static int cmp_ex_search(const void *key, const void *elt) |
| 96 | { | 92 | { |
| @@ -120,4 +116,3 @@ search_extable(const struct exception_table_entry *base, · 23 unmodified lines | |||
| 120 | return bsearch(&value, base, num, | 116 | return bsearch(&value, base, num, |
| 121 | sizeof(struct exception_table_entry), cmp_ex_search); | 117 | sizeof(struct exception_table_entry), cmp_ex_search); |
| 122 | } | 118 | } |
| 123 | #endif | ||
| @@ -368,20 +368,24 @@ void zero_user_segments(struct page *page, unsigned start1, unsigned end1, · 367 unmodified lines | |||
| 368 | 368 | ||
| 369 | BUG_ON(end1 > page_size(page) || end2 > page_size(page)); | 369 | BUG_ON(end1 > page_size(page) || end2 > page_size(page)); |
| 370 | 370 | ||
| 371 | if (start1 >= end1) | ||
| 372 | start1 = end1 = 0; | ||
| 373 | if (start2 >= end2) | ||
| 374 | start2 = end2 = 0; | ||
| 375 | |||
| 371 | for (i = 0; i < compound_nr(page); i++) { | 376 | for (i = 0; i < compound_nr(page); i++) { |
| 372 | void *kaddr = NULL; | 377 | void *kaddr = NULL; |
| 373 | 378 | ||
| 374 | if (start1 < PAGE_SIZE || start2 < PAGE_SIZE) | ||
| 375 | kaddr = kmap_atomic(page + i); | ||
| 376 | |||
| 377 | if (start1 >= PAGE_SIZE) { | 379 | if (start1 >= PAGE_SIZE) { |
| 378 | start1 -= PAGE_SIZE; | 380 | start1 -= PAGE_SIZE; |
| 379 | end1 -= PAGE_SIZE; | 381 | end1 -= PAGE_SIZE; |
| 380 | } else { | 382 | } else { |
| 381 | unsigned this_end = min_t(unsigned, end1, PAGE_SIZE); | 383 | unsigned this_end = min_t(unsigned, end1, PAGE_SIZE); |
| 382 | 384 | ||
| 383 | if (end1 > start1) | 385 | if (end1 > start1) { |
| 386 | kaddr = kmap_atomic(page + i); | ||
| 384 | memset(kaddr + start1, 0, this_end - start1); | 387 | memset(kaddr + start1, 0, this_end - start1); |
| 388 | } | ||
| 385 | end1 -= this_end; | 389 | end1 -= this_end; |
| 386 | start1 = 0; | 390 | start1 = 0; |
| 387 | } | 391 | } |
| @@ -392,8 +396,11 @@ void zero_user_segments(struct page *page, unsigned start1, unsigned end1, · 4 unmodified lines | |||
| 392 | } else { | 396 | } else { |
| 393 | unsigned this_end = min_t(unsigned, end2, PAGE_SIZE); | 397 | unsigned this_end = min_t(unsigned, end2, PAGE_SIZE); |
| 394 | 398 | ||
| 395 | if (end2 > start2) | 399 | if (end2 > start2) { |
| 400 | if (!kaddr) | ||
| 401 | kaddr = kmap_atomic(page + i); | ||
| 396 | memset(kaddr + start2, 0, this_end - start2); | 402 | memset(kaddr + start2, 0, this_end - start2); |
| 403 | } | ||
| 397 | end2 -= this_end; | 404 | end2 -= this_end; |
| 398 | start2 = 0; | 405 | start2 = 0; |
| 399 | } | 406 | } |
| @@ -1100,9 +1100,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, · 1099 unmodified lines | |||
| 1100 | * best effort that the pinned pages won't be replaced by another | 1100 | * best effort that the pinned pages won't be replaced by another |
| 1101 | * random page during the coming copy-on-write. | 1101 | * random page during the coming copy-on-write. |
| 1102 | */ | 1102 | */ |
| 1103 | if (unlikely(is_cow_mapping(vma->vm_flags) && | 1103 | if (unlikely(page_needs_cow_for_dma(vma, src_page))) { |
| 1104 | atomic_read(&src_mm->has_pinned) && | ||
| 1105 | page_maybe_dma_pinned(src_page))) { | ||
| 1106 | pte_free(dst_mm, pgtable); | 1104 | pte_free(dst_mm, pgtable); |
| 1107 | spin_unlock(src_ptl); | 1105 | spin_unlock(src_ptl); |
| 1108 | spin_unlock(dst_ptl); | 1106 | spin_unlock(dst_ptl); |
| @@ -1214,9 +1212,7 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, · 105 unmodified lines | |||
| 1214 | } | 1212 | } |
| 1215 | 1213 | ||
| 1216 | /* Please refer to comments in copy_huge_pmd() */ | 1214 | /* Please refer to comments in copy_huge_pmd() */ |
| 1217 | if (unlikely(is_cow_mapping(vma->vm_flags) && | 1215 | if (unlikely(page_needs_cow_for_dma(vma, pud_page(pud)))) { |
| 1218 | atomic_read(&src_mm->has_pinned) && | ||
| 1219 | page_maybe_dma_pinned(pud_page(pud)))) { | ||
| 1220 | spin_unlock(src_ptl); | 1216 | spin_unlock(src_ptl); |
| 1221 | spin_unlock(dst_ptl); | 1217 | spin_unlock(dst_ptl); |
| 1222 | __split_huge_pud(vma, src_pud, addr); | 1218 | __split_huge_pud(vma, src_pud, addr); |
| @@ -2471,7 +2467,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, · 1248 unmodified lines | |||
| 2471 | int i; | 2467 | int i; |
| 2472 | 2468 | ||
| 2473 | /* complete memcg works before add pages to LRU */ | 2469 | /* complete memcg works before add pages to LRU */ |
| 2474 | mem_cgroup_split_huge_fixup(head); | 2470 | split_page_memcg(head, nr); |
| 2475 | 2471 | ||
| 2476 | if (PageAnon(head) && PageSwapCache(head)) { | 2472 | if (PageAnon(head) && PageSwapCache(head)) { |
| 2477 | swp_entry_t entry = { .val = page_private(head) }; | 2473 | swp_entry_t entry = { .val = page_private(head) }; |
| @@ -331,6 +331,24 @@ static void coalesce_file_region(struct resv_map *resv, struct file_region *rg) · 330 unmodified lines | |||
| 331 | } | 331 | } |
| 332 | } | 332 | } |
| 333 | 333 | ||
| 334 | static inline long | ||
| 335 | hugetlb_resv_map_add(struct resv_map *map, struct file_region *rg, long from, | ||
| 336 | long to, struct hstate *h, struct hugetlb_cgroup *cg, | ||
| 337 | long *regions_needed) | ||
| 338 | { | ||
| 339 | struct file_region *nrg; | ||
| 340 | |||
| 341 | if (!regions_needed) { | ||
| 342 | nrg = get_file_region_entry_from_cache(map, from, to); | ||
| 343 | record_hugetlb_cgroup_uncharge_info(cg, h, map, nrg); | ||
| 344 | list_add(&nrg->link, rg->link.prev); | ||
| 345 | coalesce_file_region(map, nrg); | ||
| 346 | } else | ||
| 347 | *regions_needed += 1; | ||
| 348 | |||
| 349 | return to - from; | ||
| 350 | } | ||
| 351 | |||
| 334 | /* | 352 | /* |
| 335 | * Must be called with resv->lock held. | 353 | * Must be called with resv->lock held. |
| 336 | * | 354 | * |
| @@ -346,7 +364,7 @@ static long add_reservation_in_range(struct resv_map *resv, long f, long t, · 9 unmodified lines | |||
| 346 | long add = 0; | 364 | long add = 0; |
| 347 | struct list_head *head = &resv->regions; | 365 | struct list_head *head = &resv->regions; |
| 348 | long last_accounted_offset = f; | 366 | long last_accounted_offset = f; |
| 349 | struct file_region *rg = NULL, *trg = NULL, *nrg = NULL; | 367 | struct file_region *rg = NULL, *trg = NULL; |
| 350 | 368 | ||
| 351 | if (regions_needed) | 369 | if (regions_needed) |
| 352 | *regions_needed = 0; | 370 | *regions_needed = 0; |
| @@ -369,24 +387,17 @@ static long add_reservation_in_range(struct resv_map *resv, long f, long t, · 16 unmodified lines | |||
| 369 | /* When we find a region that starts beyond our range, we've | 387 | /* When we find a region that starts beyond our range, we've |
| 370 | * finished. | 388 | * finished. |
| 371 | */ | 389 | */ |
| 372 | if (rg->from > t) | 390 | if (rg->from >= t) |
| 373 | break; | 391 | break; |
| 374 | 392 | ||
| 375 | /* Add an entry for last_accounted_offset -> rg->from, and | 393 | /* Add an entry for last_accounted_offset -> rg->from, and |
| 376 | * update last_accounted_offset. | 394 | * update last_accounted_offset. |
| 377 | */ | 395 | */ |
| 378 | if (rg->from > last_accounted_offset) { | 396 | if (rg->from > last_accounted_offset) |
| 379 | add += rg->from - last_accounted_offset; | 397 | add += hugetlb_resv_map_add(resv, rg, |
| 380 | if (!regions_needed) { | 398 | last_accounted_offset, |
| 381 | nrg = get_file_region_entry_from_cache( | 399 | rg->from, h, h_cg, |
| 382 | resv, last_accounted_offset, rg->from); | 400 | regions_needed); |
| 383 | record_hugetlb_cgroup_uncharge_info(h_cg, h, | ||
| 384 | resv, nrg); | ||
| 385 | list_add(&nrg->link, rg->link.prev); | ||
| 386 | coalesce_file_region(resv, nrg); | ||
| 387 | } else | ||
| 388 | *regions_needed += 1; | ||
| 389 | } | ||
| 390 | 401 | ||
| 391 | last_accounted_offset = rg->to; | 402 | last_accounted_offset = rg->to; |
| 392 | } | 403 | } |
| @@ -394,17 +405,9 @@ static long add_reservation_in_range(struct resv_map *resv, long f, long t, · 1 unmodified line | |||
| 394 | /* Handle the case where our range extends beyond | 405 | /* Handle the case where our range extends beyond |
| 395 | * last_accounted_offset. | 406 | * last_accounted_offset. |
| 396 | */ | 407 | */ |
| 397 | if (last_accounted_offset < t) { | 408 | if (last_accounted_offset < t) |
| 398 | add += t - last_accounted_offset; | 409 | add += hugetlb_resv_map_add(resv, rg, last_accounted_offset, |
| 399 | if (!regions_needed) { | 410 | t, h, h_cg, regions_needed); |
| 400 | nrg = get_file_region_entry_from_cache( | ||
| 401 | resv, last_accounted_offset, t); | ||
| 402 | record_hugetlb_cgroup_uncharge_info(h_cg, h, resv, nrg); | ||
| 403 | list_add(&nrg->link, rg->link.prev); | ||
| 404 | coalesce_file_region(resv, nrg); | ||
| 405 | } else | ||
| 406 | *regions_needed += 1; | ||
| 407 | } | ||
| 408 | 411 | ||
| 409 | VM_BUG_ON(add < 0); | 412 | VM_BUG_ON(add < 0); |
| 410 | return add; | 413 | return add; |
| @@ -3725,21 +3728,32 @@ static bool is_hugetlb_entry_hwpoisoned(pte_t pte) · 3314 unmodified lines | |||
| 3725 | return false; | 3728 | return false; |
| 3726 | } | 3729 | } |
| 3727 | 3730 | ||
| 3731 | static void | ||
| 3732 | hugetlb_install_page(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr, | ||
| 3733 | struct page *new_page) | ||
| 3734 | { | ||
| 3735 | __SetPageUptodate(new_page); | ||
| 3736 | set_huge_pte_at(vma->vm_mm, addr, ptep, make_huge_pte(vma, new_page, 1)); | ||
| 3737 | hugepage_add_new_anon_rmap(new_page, vma, addr); | ||
| 3738 | hugetlb_count_add(pages_per_huge_page(hstate_vma(vma)), vma->vm_mm); | ||
| 3739 | ClearHPageRestoreReserve(new_page); | ||
| 3740 | SetHPageMigratable(new_page); | ||
| 3741 | } | ||
| 3742 | |||
| 3728 | int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, | 3743 | int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, |
| 3729 | struct vm_area_struct *vma) | 3744 | struct vm_area_struct *vma) |
| 3730 | { | 3745 | { |
| 3731 | pte_t *src_pte, *dst_pte, entry, dst_entry; | 3746 | pte_t *src_pte, *dst_pte, entry, dst_entry; |
| 3732 | struct page *ptepage; | 3747 | struct page *ptepage; |
| 3733 | unsigned long addr; | 3748 | unsigned long addr; |
| 3734 | int cow; | 3749 | bool cow = is_cow_mapping(vma->vm_flags); |
| 3735 | struct hstate *h = hstate_vma(vma); | 3750 | struct hstate *h = hstate_vma(vma); |
| 3736 | unsigned long sz = huge_page_size(h); | 3751 | unsigned long sz = huge_page_size(h); |
| 3752 | unsigned long npages = pages_per_huge_page(h); | ||
| 3737 | struct address_space *mapping = vma->vm_file->f_mapping; | 3753 | struct address_space *mapping = vma->vm_file->f_mapping; |
| 3738 | struct mmu_notifier_range range; | 3754 | struct mmu_notifier_range range; |
| 3739 | int ret = 0; | 3755 | int ret = 0; |
| 3740 | 3756 | ||
| 3741 | cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; | ||
| 3742 | |||
| 3743 | if (cow) { | 3757 | if (cow) { |
| 3744 | mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, src, | 3758 | mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, src, |
| 3745 | vma->vm_start, | 3759 | vma->vm_start, |
| @@ -3784,6 +3798,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · 38 unmodified lines | |||
| 3784 | spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); | 3798 | spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); |
| 3785 | entry = huge_ptep_get(src_pte); | 3799 | entry = huge_ptep_get(src_pte); |
| 3786 | dst_entry = huge_ptep_get(dst_pte); | 3800 | dst_entry = huge_ptep_get(dst_pte); |
| 3801 | again: | ||
| 3787 | if (huge_pte_none(entry) || !huge_pte_none(dst_entry)) { | 3802 | if (huge_pte_none(entry) || !huge_pte_none(dst_entry)) { |
| 3788 | /* | 3803 | /* |
| 3789 | * Skip if src entry none. Also, skip in the | 3804 | * Skip if src entry none. Also, skip in the |
| @@ -3807,6 +3822,52 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · 17 unmodified lines | |||
| 3807 | } | 3822 | } |
| 3808 | set_huge_swap_pte_at(dst, addr, dst_pte, entry, sz); | 3823 | set_huge_swap_pte_at(dst, addr, dst_pte, entry, sz); |
| 3809 | } else { | 3824 | } else { |
| 3825 | entry = huge_ptep_get(src_pte); | ||
| 3826 | ptepage = pte_page(entry); | ||
| 3827 | get_page(ptepage); | ||
| 3828 | |||
| 3829 | /* | ||
| 3830 | * This is a rare case where we see pinned hugetlb | ||
| 3831 | * pages while they're prone to COW. We need to do the | ||
| 3832 | * COW earlier during fork. | ||
| 3833 | * | ||
| 3834 | * When pre-allocating the page or copying data, we | ||
| 3835 | * need to be without the pgtable locks since we could | ||
| 3836 | * sleep during the process. | ||
| 3837 | */ | ||
| 3838 | if (unlikely(page_needs_cow_for_dma(vma, ptepage))) { | ||
| 3839 | pte_t src_pte_old = entry; | ||
| 3840 | struct page *new; | ||
| 3841 | |||
| 3842 | spin_unlock(src_ptl); | ||
| 3843 | spin_unlock(dst_ptl); | ||
| 3844 | /* Do not use reserve as it's private owned */ | ||
| 3845 | new = alloc_huge_page(vma, addr, 1); | ||
| 3846 | if (IS_ERR(new)) { | ||
| 3847 | put_page(ptepage); | ||
| 3848 | ret = PTR_ERR(new); | ||
| 3849 | break; | ||
| 3850 | } | ||
| 3851 | copy_user_huge_page(new, ptepage, addr, vma, | ||
| 3852 | npages); | ||
| 3853 | put_page(ptepage); | ||
| 3854 | |||
| 3855 | /* Install the new huge page if src pte stable */ | ||
| 3856 | dst_ptl = huge_pte_lock(h, dst, dst_pte); | ||
| 3857 | src_ptl = huge_pte_lockptr(h, src, src_pte); | ||
| 3858 | spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); | ||
| 3859 | entry = huge_ptep_get(src_pte); | ||
| 3860 | if (!pte_same(src_pte_old, entry)) { | ||
| 3861 | put_page(new); | ||
| 3862 | /* dst_entry won't change as in child */ | ||
| 3863 | goto again; | ||
| 3864 | } | ||
| 3865 | hugetlb_install_page(vma, dst_pte, addr, new); | ||
| 3866 | spin_unlock(src_ptl); | ||
| 3867 | spin_unlock(dst_ptl); | ||
| 3868 | continue; | ||
| 3869 | } | ||
| 3870 | |||
| 3810 | if (cow) { | 3871 | if (cow) { |
| 3811 | /* | 3872 | /* |
| 3812 | * No need to notify as we are downgrading page | 3873 | * No need to notify as we are downgrading page |
| @@ -3817,12 +3878,10 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · 4 unmodified lines | |||
| 3817 | */ | 3878 | */ |
| 3818 | huge_ptep_set_wrprotect(src, addr, src_pte); | 3879 | huge_ptep_set_wrprotect(src, addr, src_pte); |
| 3819 | } | 3880 | } |
| 3820 | entry = huge_ptep_get(src_pte); | 3881 | |
| 3821 | ptepage = pte_page(entry); | ||
| 3822 | get_page(ptepage); | ||
| 3823 | page_dup_rmap(ptepage, true); | 3882 | page_dup_rmap(ptepage, true); |
| 3824 | set_huge_pte_at(dst, addr, dst_pte, entry); | 3883 | set_huge_pte_at(dst, addr, dst_pte, entry); |
| 3825 | hugetlb_count_add(pages_per_huge_page(h), dst); | 3884 | hugetlb_count_add(npages, dst); |
| 3826 | } | 3885 | } |
| 3827 | spin_unlock(src_ptl); | 3886 | spin_unlock(src_ptl); |
| 3828 | spin_unlock(dst_ptl); | 3887 | spin_unlock(dst_ptl); |
| @@ -296,11 +296,6 @@ static inline unsigned int buddy_order(struct page *page) · 295 unmodified lines | |||
| 296 | */ | 296 | */ |
| 297 | #define buddy_order_unsafe(page) READ_ONCE(page_private(page)) | 297 | #define buddy_order_unsafe(page) READ_ONCE(page_private(page)) |
| 298 | 298 | ||
| 299 | static inline bool is_cow_mapping(vm_flags_t flags) | ||
| 300 | { | ||
| 301 | return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; | ||
| 302 | } | ||
| 303 | |||
| 304 | /* | 299 | /* |
| 305 | * These three helpers classifies VMAs for virtual memory accounting. | 300 | * These three helpers classifies VMAs for virtual memory accounting. |
| 306 | */ | 301 | */ |
| @@ -20,6 +20,11 @@ · 19 unmodified lines | |||
| 20 | 20 | ||
| 21 | #include "kfence.h" | 21 | #include "kfence.h" |
| 22 | 22 | ||
| 23 | /* May be overridden by <asm/kfence.h>. */ | ||
| 24 | #ifndef ARCH_FUNC_PREFIX | ||
| 25 | #define ARCH_FUNC_PREFIX "" | ||
| 26 | #endif | ||
| 27 | |||
| 23 | extern bool no_hash_pointers; | 28 | extern bool no_hash_pointers; |
| 24 | 29 | ||
| 25 | /* Helper function to either print to a seq_file or to console. */ | 30 | /* Helper function to either print to a seq_file or to console. */ |
| @@ -67,8 +72,9 @@ static int get_stack_skipnr(const unsigned long stack_entries[], int num_entries · 41 unmodified lines | |||
| 67 | for (skipnr = 0; skipnr < num_entries; skipnr++) { | 72 | for (skipnr = 0; skipnr < num_entries; skipnr++) { |
| 68 | int len = scnprintf(buf, sizeof(buf), "%ps", (void *)stack_entries[skipnr]); | 73 | int len = scnprintf(buf, sizeof(buf), "%ps", (void *)stack_entries[skipnr]); |
| 69 | 74 | ||
| 70 | if (str_has_prefix(buf, "kfence_") || str_has_prefix(buf, "__kfence_") || | 75 | if (str_has_prefix(buf, ARCH_FUNC_PREFIX "kfence_") || |
| 71 | !strncmp(buf, "__slab_free", len)) { | 76 | str_has_prefix(buf, ARCH_FUNC_PREFIX "__kfence_") || |
| 77 | !strncmp(buf, ARCH_FUNC_PREFIX "__slab_free", len)) { | ||
| 72 | /* | 78 | /* |
| 73 | * In case of tail calls from any of the below | 79 | * In case of tail calls from any of the below |
| 74 | * to any of the above. | 80 | * to any of the above. |
| @@ -77,10 +83,10 @@ static int get_stack_skipnr(const unsigned long stack_entries[], int num_entries · 2 unmodified lines | |||
| 77 | } | 83 | } |
| 78 | 84 | ||
| 79 | /* Also the *_bulk() variants by only checking prefixes. */ | 85 | /* Also the *_bulk() variants by only checking prefixes. */ |
| 80 | if (str_has_prefix(buf, "kfree") || | 86 | if (str_has_prefix(buf, ARCH_FUNC_PREFIX "kfree") || |
| 81 | str_has_prefix(buf, "kmem_cache_free") || | 87 | str_has_prefix(buf, ARCH_FUNC_PREFIX "kmem_cache_free") || |
| 82 | str_has_prefix(buf, "__kmalloc") || | 88 | str_has_prefix(buf, ARCH_FUNC_PREFIX "__kmalloc") || |
| 83 | str_has_prefix(buf, "kmem_cache_alloc")) | 89 | str_has_prefix(buf, ARCH_FUNC_PREFIX "kmem_cache_alloc")) |
| 84 | goto found; | 90 | goto found; |
| 85 | } | 91 | } |
| 86 | if (fallback < num_entries) | 92 | if (fallback < num_entries) |
| @@ -116,12 +122,12 @@ void kfence_print_object(struct seq_file *seq, const struct kfence_metadata *met · 29 unmodified lines | |||
| 116 | lockdep_assert_held(&meta->lock); | 122 | lockdep_assert_held(&meta->lock); |
| 117 | 123 | ||
| 118 | if (meta->state == KFENCE_OBJECT_UNUSED) { | 124 | if (meta->state == KFENCE_OBJECT_UNUSED) { |
| 119 | seq_con_printf(seq, "kfence-#%zd unused\n", meta - kfence_metadata); | 125 | seq_con_printf(seq, "kfence-#%td unused\n", meta - kfence_metadata); |
| 120 | return; | 126 | return; |
| 121 | } | 127 | } |
| 122 | 128 | ||
| 123 | seq_con_printf(seq, | 129 | seq_con_printf(seq, |
| 124 | "kfence-#%zd [0x%p-0x%p" | 130 | "kfence-#%td [0x%p-0x%p" |
| 125 | ", size=%d, cache=%s] allocated by task %d:\n", | 131 | ", size=%d, cache=%s] allocated by task %d:\n", |
| 126 | meta - kfence_metadata, (void *)start, (void *)(start + size - 1), size, | 132 | meta - kfence_metadata, (void *)start, (void *)(start + size - 1), size, |
| 127 | (cache && cache->name) ? cache->name : "<destroyed>", meta->alloc_track.pid); | 133 | (cache && cache->name) ? cache->name : "<destroyed>", meta->alloc_track.pid); |
| @@ -204,7 +210,7 @@ void kfence_report_error(unsigned long address, bool is_write, struct pt_regs *r · 76 unmodified lines | |||
| 204 | 210 | ||
| 205 | pr_err("BUG: KFENCE: out-of-bounds %s in %pS\n\n", get_access_type(is_write), | 211 | pr_err("BUG: KFENCE: out-of-bounds %s in %pS\n\n", get_access_type(is_write), |
| 206 | (void *)stack_entries[skipnr]); | 212 | (void *)stack_entries[skipnr]); |
| 207 | pr_err("Out-of-bounds %s at 0x%p (%luB %s of kfence-#%zd):\n", | 213 | pr_err("Out-of-bounds %s at 0x%p (%luB %s of kfence-#%td):\n", |
| 208 | get_access_type(is_write), (void *)address, | 214 | get_access_type(is_write), (void *)address, |
| 209 | left_of_object ? meta->addr - address : address - meta->addr, | 215 | left_of_object ? meta->addr - address : address - meta->addr, |
| 210 | left_of_object ? "left" : "right", object_index); | 216 | left_of_object ? "left" : "right", object_index); |
| @@ -213,14 +219,14 @@ void kfence_report_error(unsigned long address, bool is_write, struct pt_regs *r · 2 unmodified lines | |||
| 213 | case KFENCE_ERROR_UAF: | 219 | case KFENCE_ERROR_UAF: |
| 214 | pr_err("BUG: KFENCE: use-after-free %s in %pS\n\n", get_access_type(is_write), | 220 | pr_err("BUG: KFENCE: use-after-free %s in %pS\n\n", get_access_type(is_write), |
| 215 | (void *)stack_entries[skipnr]); | 221 | (void *)stack_entries[skipnr]); |
| 216 | pr_err("Use-after-free %s at 0x%p (in kfence-#%zd):\n", | 222 | pr_err("Use-after-free %s at 0x%p (in kfence-#%td):\n", |
| 217 | get_access_type(is_write), (void *)address, object_index); | 223 | get_access_type(is_write), (void *)address, object_index); |
| 218 | break; | 224 | break; |
| 219 | case KFENCE_ERROR_CORRUPTION: | 225 | case KFENCE_ERROR_CORRUPTION: |
| 220 | pr_err("BUG: KFENCE: memory corruption in %pS\n\n", (void *)stack_entries[skipnr]); | 226 | pr_err("BUG: KFENCE: memory corruption in %pS\n\n", (void *)stack_entries[skipnr]); |
| 221 | pr_err("Corrupted memory at 0x%p ", (void *)address); | 227 | pr_err("Corrupted memory at 0x%p ", (void *)address); |
| 222 | print_diff_canary(address, 16, meta); | 228 | print_diff_canary(address, 16, meta); |
| 223 | pr_cont(" (in kfence-#%zd):\n", object_index); | 229 | pr_cont(" (in kfence-#%td):\n", object_index); |
| 224 | break; | 230 | break; |
| 225 | case KFENCE_ERROR_INVALID: | 231 | case KFENCE_ERROR_INVALID: |
| 226 | pr_err("BUG: KFENCE: invalid %s in %pS\n\n", get_access_type(is_write), | 232 | pr_err("BUG: KFENCE: invalid %s in %pS\n\n", get_access_type(is_write), |
| @@ -230,7 +236,7 @@ void kfence_report_error(unsigned long address, bool is_write, struct pt_regs *r · 3 unmodified lines | |||
| 230 | break; | 236 | break; |
| 231 | case KFENCE_ERROR_INVALID_FREE: | 237 | case KFENCE_ERROR_INVALID_FREE: |
| 232 | pr_err("BUG: KFENCE: invalid free in %pS\n\n", (void *)stack_entries[skipnr]); | 238 | pr_err("BUG: KFENCE: invalid free in %pS\n\n", (void *)stack_entries[skipnr]); |
| 233 | pr_err("Invalid free of 0x%p (in kfence-#%zd):\n", (void *)address, | 239 | pr_err("Invalid free of 0x%p (in kfence-#%td):\n", (void *)address, |
| 234 | object_index); | 240 | object_index); |
| 235 | break; | 241 | break; |
| 236 | } | 242 | } |
| @@ -1198,12 +1198,22 @@ SYSCALL_DEFINE5(process_madvise, int, pidfd, const struct iovec __user *, vec, · 1197 unmodified lines | |||
| 1198 | goto release_task; | 1198 | goto release_task; |
| 1199 | } | 1199 | } |
| 1200 | 1200 | ||
| 1201 | mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); | 1201 | /* Require PTRACE_MODE_READ to avoid leaking ASLR metadata. */ |
| 1202 | mm = mm_access(task, PTRACE_MODE_READ_FSCREDS); | ||
| 1202 | if (IS_ERR_OR_NULL(mm)) { | 1203 | if (IS_ERR_OR_NULL(mm)) { |
| 1203 | ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; | 1204 | ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; |
| 1204 | goto release_task; | 1205 | goto release_task; |
| 1205 | } | 1206 | } |
| 1206 | 1207 | ||
| 1208 | /* | ||
| 1209 | * Require CAP_SYS_NICE for influencing process performance. Note that | ||
| 1210 | * only non-destructive hints are currently supported. | ||
| 1211 | */ | ||
| 1212 | if (!capable(CAP_SYS_NICE)) { | ||
| 1213 | ret = -EPERM; | ||
| 1214 | goto release_mm; | ||
| 1215 | } | ||
| 1216 | |||
| 1207 | total_len = iov_iter_count(&iter); | 1217 | total_len = iov_iter_count(&iter); |
| 1208 | 1218 | ||
| 1209 | while (iov_iter_count(&iter)) { | 1219 | while (iov_iter_count(&iter)) { |
| @@ -1218,6 +1228,7 @@ SYSCALL_DEFINE5(process_madvise, int, pidfd, const struct iovec __user *, vec, · 8 unmodified lines | |||
| 1218 | if (ret == 0) | 1228 | if (ret == 0) |
| 1219 | ret = total_len - iov_iter_count(&iter); | 1229 | ret = total_len - iov_iter_count(&iter); |
| 1220 | 1230 | ||
| 1231 | release_mm: | ||
| 1221 | mmput(mm); | 1232 | mmput(mm); |
| 1222 | release_task: | 1233 | release_task: |
| 1223 | put_task_struct(task); | 1234 | put_task_struct(task); |
| @@ -3287,24 +3287,21 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size) · 3286 unmodified lines | |||
| 3287 | 3287 | ||
| 3288 | #endif /* CONFIG_MEMCG_KMEM */ | 3288 | #endif /* CONFIG_MEMCG_KMEM */ |
| 3289 | 3289 | ||
| 3290 | #ifdef CONFIG_TRANSPARENT_HUGEPAGE | ||
| 3291 | /* | 3290 | /* |
| 3292 | * Because page_memcg(head) is not set on compound tails, set it now. | 3291 | * Because page_memcg(head) is not set on tails, set it now. |
| 3293 | */ | 3292 | */ |
| 3294 | void mem_cgroup_split_huge_fixup(struct page *head) | 3293 | void split_page_memcg(struct page *head, unsigned int nr) |
| 3295 | { | 3294 | { |
| 3296 | struct mem_cgroup *memcg = page_memcg(head); | 3295 | struct mem_cgroup *memcg = page_memcg(head); |
| 3297 | int i; | 3296 | int i; |
| 3298 | 3297 | ||
| 3299 | if (mem_cgroup_disabled()) | 3298 | if (mem_cgroup_disabled() || !memcg) |
| 3300 | return; | 3299 | return; |
| 3301 | 3300 | ||
| 3302 | for (i = 1; i < HPAGE_PMD_NR; i++) { | 3301 | for (i = 1; i < nr; i++) |
| 3303 | css_get(&memcg->css); | 3302 | head[i].memcg_data = head->memcg_data; |
| 3304 | head[i].memcg_data = (unsigned long)memcg; | 3303 | css_get_many(&memcg->css, nr - 1); |
| 3305 | } | ||
| 3306 | } | 3304 | } |
| 3307 | #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ | ||
| 3308 | 3305 | ||
| 3309 | #ifdef CONFIG_MEMCG_SWAP | 3306 | #ifdef CONFIG_MEMCG_SWAP |
| 3310 | /** | 3307 | /** |
| @@ -809,12 +809,8 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma · 808 unmodified lines | |||
| 809 | pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, | 809 | pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, |
| 810 | struct page **prealloc, pte_t pte, struct page *page) | 810 | struct page **prealloc, pte_t pte, struct page *page) |
| 811 | { | 811 | { |
| 812 | struct mm_struct *src_mm = src_vma->vm_mm; | ||
| 813 | struct page *new_page; | 812 | struct page *new_page; |
| 814 | 813 | ||
| 815 | if (!is_cow_mapping(src_vma->vm_flags)) | ||
| 816 | return 1; | ||
| 817 | |||
| 818 | /* | 814 | /* |
| 819 | * What we want to do is to check whether this page may | 815 | * What we want to do is to check whether this page may |
| 820 | * have been pinned by the parent process. If so, | 816 | * have been pinned by the parent process. If so, |
| @@ -828,9 +824,7 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma · 7 unmodified lines | |||
| 828 | * the page count. That might give false positives for | 824 | * the page count. That might give false positives for |
| 829 | * for pinning, but it will work correctly. | 825 | * for pinning, but it will work correctly. |
| 830 | */ | 826 | */ |
| 831 | if (likely(!atomic_read(&src_mm->has_pinned))) | 827 | if (likely(!page_needs_cow_for_dma(src_vma, page))) |
| 832 | return 1; | ||
| 833 | if (likely(!page_maybe_dma_pinned(page))) | ||
| 834 | return 1; | 828 | return 1; |
| 835 | 829 | ||
| 836 | new_page = *prealloc; | 830 | new_page = *prealloc; |
| @@ -3103,6 +3097,14 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) · 2266 unmodified lines | |||
| 3103 | return handle_userfault(vmf, VM_UFFD_WP); | 3097 | return handle_userfault(vmf, VM_UFFD_WP); |
| 3104 | } | 3098 | } |
| 3105 | 3099 | ||
| 3100 | /* | ||
| 3101 | * Userfaultfd write-protect can defer flushes. Ensure the TLB | ||
| 3102 | * is flushed in this case before copying. | ||
| 3103 | */ | ||
| 3104 | if (unlikely(userfaultfd_wp(vmf->vma) && | ||
| 3105 | mm_tlb_flush_pending(vmf->vma->vm_mm))) | ||
| 3106 | flush_tlb_page(vmf->vma, vmf->address); | ||
| 3107 | |||
| 3106 | vmf->page = vm_normal_page(vma, vmf->address, vmf->orig_pte); | 3108 | vmf->page = vm_normal_page(vma, vmf->address, vmf->orig_pte); |
| 3107 | if (!vmf->page) { | 3109 | if (!vmf->page) { |
| 3108 | /* | 3110 | /* |
| @@ -1072,7 +1072,7 @@ static int online_memory_block(struct memory_block *mem, void *arg) · 1071 unmodified lines | |||
| 1072 | */ | 1072 | */ |
| 1073 | int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags) | 1073 | int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags) |
| 1074 | { | 1074 | { |
| 1075 | struct mhp_params params = { .pgprot = PAGE_KERNEL }; | 1075 | struct mhp_params params = { .pgprot = pgprot_mhp(PAGE_KERNEL) }; |
| 1076 | u64 start, size; | 1076 | u64 start, size; |
| 1077 | bool new_node = false; | 1077 | bool new_node = false; |
| 1078 | int ret; | 1078 | int ret; |
| @@ -1281,6 +1281,12 @@ static __always_inline bool free_pages_prepare(struct page *page, · 1280 unmodified lines | |||
| 1281 | 1281 | ||
| 1282 | kernel_poison_pages(page, 1 << order); | 1282 | kernel_poison_pages(page, 1 << order); |
| 1283 | 1283 | ||
| 1284 | /* | ||
| 1285 | * With hardware tag-based KASAN, memory tags must be set before the | ||
| 1286 | * page becomes unavailable via debug_pagealloc or arch_free_page. | ||
| 1287 | */ | ||
| 1288 | kasan_free_nondeferred_pages(page, order); | ||
| 1289 | |||
| 1284 | /* | 1290 | /* |
| 1285 | * arch_free_page() can make the page's contents inaccessible. s390 | 1291 | * arch_free_page() can make the page's contents inaccessible. s390 |
| 1286 | * does this. So nothing which can access the page's contents should | 1292 | * does this. So nothing which can access the page's contents should |
| @@ -1290,8 +1296,6 @@ static __always_inline bool free_pages_prepare(struct page *page, · 3 unmodified lines | |||
| 1290 | 1296 | ||
| 1291 | debug_pagealloc_unmap_pages(page, 1 << order); | 1297 | debug_pagealloc_unmap_pages(page, 1 << order); |
| 1292 | 1298 | ||
| 1293 | kasan_free_nondeferred_pages(page, order); | ||
| 1294 | |||
| 1295 | return true; | 1299 | return true; |
| 1296 | } | 1300 | } |
| 1297 | 1301 | ||
| @@ -3310,6 +3314,7 @@ void split_page(struct page *page, unsigned int order) · 2012 unmodified lines | |||
| 3310 | for (i = 1; i < (1 << order); i++) | 3314 | for (i = 1; i < (1 << order); i++) |
| 3311 | set_page_refcounted(page + i); | 3315 | set_page_refcounted(page + i); |
| 3312 | split_page_owner(page, 1 << order); | 3316 | split_page_owner(page, 1 << order); |
| 3317 | split_page_memcg(page, 1 << order); | ||
| 3313 | } | 3318 | } |
| 3314 | EXPORT_SYMBOL_GPL(split_page); | 3319 | EXPORT_SYMBOL_GPL(split_page); |
| 3315 | 3320 | ||
| @@ -6259,12 +6264,65 @@ static void __meminit zone_init_free_lists(struct zone *zone) · 2943 unmodified lines | |||
| 6259 | } | 6264 | } |
| 6260 | } | 6265 | } |
| 6261 | 6266 | ||
| 6267 | #if !defined(CONFIG_FLAT_NODE_MEM_MAP) | ||
| 6268 | /* | ||
| 6269 | * Only struct pages that correspond to ranges defined by memblock.memory | ||
| 6270 | * are zeroed and initialized by going through __init_single_page() during | ||
| 6271 | * memmap_init_zone(). | ||
| 6272 | * | ||
| 6273 | * But, there could be struct pages that correspond to holes in | ||
| 6274 | * memblock.memory. This can happen because of the following reasons: | ||
| 6275 | * - physical memory bank size is not necessarily the exact multiple of the | ||
| 6276 | * arbitrary section size | ||
| 6277 | * - early reserved memory may not be listed in memblock.memory | ||
| 6278 | * - memory layouts defined with memmap= kernel parameter may not align | ||
| 6279 | * nicely with memmap sections | ||
| 6280 | * | ||
| 6281 | * Explicitly initialize those struct pages so that: | ||
| 6282 | * - PG_Reserved is set | ||
| 6283 | * - zone and node links point to zone and node that span the page if the | ||
| 6284 | * hole is in the middle of a zone | ||
| 6285 | * - zone and node links point to adjacent zone/node if the hole falls on | ||
| 6286 | * the zone boundary; the pages in such holes will be prepended to the | ||
| 6287 | * zone/node above the hole except for the trailing pages in the last | ||
| 6288 | * section that will be appended to the zone/node below. | ||
| 6289 | */ | ||
| 6290 | static u64 __meminit init_unavailable_range(unsigned long spfn, | ||
| 6291 | unsigned long epfn, | ||
| 6292 | int zone, int node) | ||
| 6293 | { | ||
| 6294 | unsigned long pfn; | ||
| 6295 | u64 pgcnt = 0; | ||
| 6296 | |||
| 6297 | for (pfn = spfn; pfn < epfn; pfn++) { | ||
| 6298 | if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { | ||
| 6299 | pfn = ALIGN_DOWN(pfn, pageblock_nr_pages) | ||
| 6300 | + pageblock_nr_pages - 1; | ||
| 6301 | continue; | ||
| 6302 | } | ||
| 6303 | __init_single_page(pfn_to_page(pfn), pfn, zone, node); | ||
| 6304 | __SetPageReserved(pfn_to_page(pfn)); | ||
| 6305 | pgcnt++; | ||
| 6306 | } | ||
| 6307 | |||
| 6308 | return pgcnt; | ||
| 6309 | } | ||
| 6310 | #else | ||
| 6311 | static inline u64 init_unavailable_range(unsigned long spfn, unsigned long epfn, | ||
| 6312 | int zone, int node) | ||
| 6313 | { | ||
| 6314 | return 0; | ||
| 6315 | } | ||
| 6316 | #endif | ||
| 6317 | |||
| 6262 | void __meminit __weak memmap_init_zone(struct zone *zone) | 6318 | void __meminit __weak memmap_init_zone(struct zone *zone) |
| 6263 | { | 6319 | { |
| 6264 | unsigned long zone_start_pfn = zone->zone_start_pfn; | 6320 | unsigned long zone_start_pfn = zone->zone_start_pfn; |
| 6265 | unsigned long zone_end_pfn = zone_start_pfn + zone->spanned_pages; | 6321 | unsigned long zone_end_pfn = zone_start_pfn + zone->spanned_pages; |
| 6266 | int i, nid = zone_to_nid(zone), zone_id = zone_idx(zone); | 6322 | int i, nid = zone_to_nid(zone), zone_id = zone_idx(zone); |
| 6323 | static unsigned long hole_pfn; | ||
| 6267 | unsigned long start_pfn, end_pfn; | 6324 | unsigned long start_pfn, end_pfn; |
| 6325 | u64 pgcnt = 0; | ||
| 6268 | 6326 | ||
| 6269 | for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { | 6327 | for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { |
| 6270 | start_pfn = clamp(start_pfn, zone_start_pfn, zone_end_pfn); | 6328 | start_pfn = clamp(start_pfn, zone_start_pfn, zone_end_pfn); |
| @@ -6274,7 +6332,29 @@ void __meminit __weak memmap_init_zone(struct zone *zone) · 3 unmodified lines | |||
| 6274 | memmap_init_range(end_pfn - start_pfn, nid, | 6332 | memmap_init_range(end_pfn - start_pfn, nid, |
| 6275 | zone_id, start_pfn, zone_end_pfn, | 6333 | zone_id, start_pfn, zone_end_pfn, |
| 6276 | MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); | 6334 | MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); |
| 6335 | |||
| 6336 | if (hole_pfn < start_pfn) | ||
| 6337 | pgcnt += init_unavailable_range(hole_pfn, start_pfn, | ||
| 6338 | zone_id, nid); | ||
| 6339 | hole_pfn = end_pfn; | ||
| 6277 | } | 6340 | } |
| 6341 | |||
| 6342 | #ifdef CONFIG_SPARSEMEM | ||
| 6343 | /* | ||
| 6344 | * Initialize the hole in the range [zone_end_pfn, section_end]. | ||
| 6345 | * If zone boundary falls in the middle of a section, this hole | ||
| 6346 | * will be re-initialized during the call to this function for the | ||
| 6347 | * higher zone. | ||
| 6348 | */ | ||
| 6349 | end_pfn = round_up(zone_end_pfn, PAGES_PER_SECTION); | ||
| 6350 | if (hole_pfn < end_pfn) | ||
| 6351 | pgcnt += init_unavailable_range(hole_pfn, end_pfn, | ||
| 6352 | zone_id, nid); | ||
| 6353 | #endif | ||
| 6354 | |||
| 6355 | if (pgcnt) | ||
| 6356 | pr_info(" %s zone: %llu pages in unavailable ranges\n", | ||
| 6357 | zone->name, pgcnt); | ||
| 6278 | } | 6358 | } |
| 6279 | 6359 | ||
| 6280 | static int zone_batchsize(struct zone *zone) | 6360 | static int zone_batchsize(struct zone *zone) |
| @@ -7071,88 +7151,6 @@ void __init free_area_init_memoryless_node(int nid) · 790 unmodified lines | |||
| 7071 | free_area_init_node(nid); | 7151 | free_area_init_node(nid); |
| 7072 | } | 7152 | } |
| 7073 | 7153 | ||
| 7074 | #if !defined(CONFIG_FLAT_NODE_MEM_MAP) | ||
| 7075 | /* | ||
| 7076 | * Initialize all valid struct pages in the range [spfn, epfn) and mark them | ||
| 7077 | * PageReserved(). Return the number of struct pages that were initialized. | ||
| 7078 | */ | ||
| 7079 | static u64 __init init_unavailable_range(unsigned long spfn, unsigned long epfn) | ||
| 7080 | { | ||
| 7081 | unsigned long pfn; | ||
| 7082 | u64 pgcnt = 0; | ||
| 7083 | |||
| 7084 | for (pfn = spfn; pfn < epfn; pfn++) { | ||
| 7085 | if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { | ||
| 7086 | pfn = ALIGN_DOWN(pfn, pageblock_nr_pages) | ||
| 7087 | + pageblock_nr_pages - 1; | ||
| 7088 | continue; | ||
| 7089 | } | ||
| 7090 | /* | ||
| 7091 | * Use a fake node/zone (0) for now. Some of these pages | ||
| 7092 | * (in memblock.reserved but not in memblock.memory) will | ||
| 7093 | * get re-initialized via reserve_bootmem_region() later. | ||
| 7094 | */ | ||
| 7095 | __init_single_page(pfn_to_page(pfn), pfn, 0, 0); | ||
| 7096 | __SetPageReserved(pfn_to_page(pfn)); | ||
| 7097 | pgcnt++; | ||
| 7098 | } | ||
| 7099 | |||
| 7100 | return pgcnt; | ||
| 7101 | } | ||
| 7102 | |||
| 7103 | /* | ||
| 7104 | * Only struct pages that are backed by physical memory are zeroed and | ||
| 7105 | * initialized by going through __init_single_page(). But, there are some | ||
| 7106 | * struct pages which are reserved in memblock allocator and their fields | ||
| 7107 | * may be accessed (for example page_to_pfn() on some configuration accesses | ||
| 7108 | * flags). We must explicitly initialize those struct pages. | ||
| 7109 | * | ||
| 7110 | * This function also addresses a similar issue where struct pages are left | ||
| 7111 | * uninitialized because the physical address range is not covered by | ||
| 7112 | * memblock.memory or memblock.reserved. That could happen when memblock | ||
| 7113 | * layout is manually configured via memmap=, or when the highest physical | ||
| 7114 | * address (max_pfn) does not end on a section boundary. | ||
| 7115 | */ | ||
| 7116 | static void __init init_unavailable_mem(void) | ||
| 7117 | { | ||
| 7118 | phys_addr_t start, end; | ||
| 7119 | u64 i, pgcnt; | ||
| 7120 | phys_addr_t next = 0; | ||
| 7121 | |||
| 7122 | /* | ||
| 7123 | * Loop through unavailable ranges not covered by memblock.memory. | ||
| 7124 | */ | ||
| 7125 | pgcnt = 0; | ||
| 7126 | for_each_mem_range(i, &start, &end) { | ||
| 7127 | if (next < start) | ||
| 7128 | pgcnt += init_unavailable_range(PFN_DOWN(next), | ||
| 7129 | PFN_UP(start)); | ||
| 7130 | next = end; | ||
| 7131 | } | ||
| 7132 | |||
| 7133 | /* | ||
| 7134 | * Early sections always have a fully populated memmap for the whole | ||
| 7135 | * section - see pfn_valid(). If the last section has holes at the | ||
| 7136 | * end and that section is marked "online", the memmap will be | ||
| 7137 | * considered initialized. Make sure that memmap has a well defined | ||
| 7138 | * state. | ||
| 7139 | */ | ||
| 7140 | pgcnt += init_unavailable_range(PFN_DOWN(next), | ||
| 7141 | round_up(max_pfn, PAGES_PER_SECTION)); | ||
| 7142 | |||
| 7143 | /* | ||
| 7144 | * Struct pages that do not have backing memory. This could be because | ||
| 7145 | * firmware is using some of this memory, or for some other reasons. | ||
| 7146 | */ | ||
| 7147 | if (pgcnt) | ||
| 7148 | pr_info("Zeroed struct page in unavailable ranges: %lld pages", pgcnt); | ||
| 7149 | } | ||
| 7150 | #else | ||
| 7151 | static inline void __init init_unavailable_mem(void) | ||
| 7152 | { | ||
| 7153 | } | ||
| 7154 | #endif /* !CONFIG_FLAT_NODE_MEM_MAP */ | ||
| 7155 | |||
| 7156 | #if MAX_NUMNODES > 1 | 7154 | #if MAX_NUMNODES > 1 |
| 7157 | /* | 7155 | /* |
| 7158 | * Figure out the number of possible node ids. | 7156 | * Figure out the number of possible node ids. |
| @@ -7576,7 +7574,6 @@ void __init free_area_init(unsigned long *max_zone_pfn) · 417 unmodified lines | |||
| 7576 | /* Initialise every node */ | 7574 | /* Initialise every node */ |
| 7577 | mminit_verify_pageflags_layout(); | 7575 | mminit_verify_pageflags_layout(); |
| 7578 | setup_nr_node_ids(); | 7576 | setup_nr_node_ids(); |
| 7579 | init_unavailable_mem(); | ||
| 7580 | for_each_online_node(nid) { | 7577 | for_each_online_node(nid) { |
| 7581 | pg_data_t *pgdat = NODE_DATA(nid); | 7578 | pg_data_t *pgdat = NODE_DATA(nid); |
| 7582 | free_area_init_node(nid); | 7579 | free_area_init_node(nid); |
| @@ -2992,7 +2992,7 @@ static void *cache_alloc_debugcheck_after(struct kmem_cache *cachep, · 2991 unmodified lines | |||
| 2992 | gfp_t flags, void *objp, unsigned long caller) | 2992 | gfp_t flags, void *objp, unsigned long caller) |
| 2993 | { | 2993 | { |
| 2994 | WARN_ON_ONCE(cachep->ctor && (flags & __GFP_ZERO)); | 2994 | WARN_ON_ONCE(cachep->ctor && (flags & __GFP_ZERO)); |
| 2995 | if (!objp) | 2995 | if (!objp || is_kfence_address(objp)) |
| 2996 | return objp; | 2996 | return objp; |
| 2997 | if (cachep->flags & SLAB_POISON) { | 2997 | if (cachep->flags & SLAB_POISON) { |
| 2998 | check_poison_obj(cachep, objp); | 2998 | check_poison_obj(cachep, objp); |
| @@ -1993,7 +1993,7 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n, · 1992 unmodified lines | |||
| 1993 | 1993 | ||
| 1994 | t = acquire_slab(s, n, page, object == NULL, &objects); | 1994 | t = acquire_slab(s, n, page, object == NULL, &objects); |
| 1995 | if (!t) | 1995 | if (!t) |
| 1996 | continue; /* cmpxchg raced */ | 1996 | break; |
| 1997 | 1997 | ||
| 1998 | available += objects; | 1998 | available += objects; |
| 1999 | if (!object) { | 1999 | if (!object) { |
| @@ -1623,10 +1623,6 @@ p9_client_read_once(struct p9_fid *fid, u64 offset, struct iov_iter *to, · 1622 unmodified lines | |||
| 1623 | } | 1623 | } |
| 1624 | 1624 | ||
| 1625 | p9_debug(P9_DEBUG_9P, "<<< RREAD count %d\n", count); | 1625 | p9_debug(P9_DEBUG_9P, "<<< RREAD count %d\n", count); |
| 1626 | if (!count) { | ||
| 1627 | p9_tag_remove(clnt, req); | ||
| 1628 | return 0; | ||
| 1629 | } | ||
| 1630 | 1626 | ||
| 1631 | if (non_zc) { | 1627 | if (non_zc) { |
| 1632 | int n = copy_to_iter(dataptr, count, to); | 1628 | int n = copy_to_iter(dataptr, count, to); |
| @@ -2147,7 +2147,7 @@ out: · 2146 unmodified lines | |||
| 2147 | out_err: | 2147 | out_err: |
| 2148 | cb->args[1] = idx; | 2148 | cb->args[1] = idx; |
| 2149 | cb->args[0] = h; | 2149 | cb->args[0] = h; |
| 2150 | cb->seq = net->dev_base_seq; | 2150 | cb->seq = tgt_net->dev_base_seq; |
| 2151 | nl_dump_check_consistent(cb, nlmsg_hdr(skb)); | 2151 | nl_dump_check_consistent(cb, nlmsg_hdr(skb)); |
| 2152 | if (netnsid >= 0) | 2152 | if (netnsid >= 0) |
| 2153 | put_net(tgt_net); | 2153 | put_net(tgt_net); |
| @@ -3659,6 +3659,8 @@ unsigned int skb_find_text(struct sk_buff *skb, unsigned int from, · 3658 unmodified lines | |||
| 3659 | struct ts_state state; | 3659 | struct ts_state state; |
| 3660 | unsigned int ret; | 3660 | unsigned int ret; |
| 3661 | 3661 | ||
| 3662 | BUILD_BUG_ON(sizeof(struct skb_seq_state) > sizeof(state.cb)); | ||
| 3663 | |||
| 3662 | config->get_next_block = skb_ts_get_next_block; | 3664 | config->get_next_block = skb_ts_get_next_block; |
| 3663 | config->finish = skb_ts_finish; | 3665 | config->finish = skb_ts_finish; |
| 3664 | 3666 | ||
| @@ -118,6 +118,8 @@ config NET_DSA_TAG_OCELOT · 117 unmodified lines | |||
| 118 | 118 | ||
| 119 | config NET_DSA_TAG_OCELOT_8021Q | 119 | config NET_DSA_TAG_OCELOT_8021Q |
| 120 | tristate "Tag driver for Ocelot family of switches, using VLAN" | 120 | tristate "Tag driver for Ocelot family of switches, using VLAN" |
| 121 | depends on MSCC_OCELOT_SWITCH_LIB || \ | ||
| 122 | (MSCC_OCELOT_SWITCH_LIB=n && COMPILE_TEST) | ||
| 121 | select NET_DSA_TAG_8021Q | 123 | select NET_DSA_TAG_8021Q |
| 122 | help | 124 | help |
| 123 | Say Y or M if you want to enable support for tagging frames with a | 125 | Say Y or M if you want to enable support for tagging frames with a |
| @@ -230,8 +230,8 @@ int dsa_port_hsr_join(struct dsa_port *dp, struct net_device *hsr); · 229 unmodified lines | |||
| 230 | void dsa_port_hsr_leave(struct dsa_port *dp, struct net_device *hsr); | 230 | void dsa_port_hsr_leave(struct dsa_port *dp, struct net_device *hsr); |
| 231 | extern const struct phylink_mac_ops dsa_port_phylink_mac_ops; | 231 | extern const struct phylink_mac_ops dsa_port_phylink_mac_ops; |
| 232 | 232 | ||
| 233 | static inline bool dsa_port_offloads_netdev(struct dsa_port *dp, | 233 | static inline bool dsa_port_offloads_bridge_port(struct dsa_port *dp, |
| 234 | struct net_device *dev) | 234 | struct net_device *dev) |
| 235 | { | 235 | { |
| 236 | /* Switchdev offloading can be configured on: */ | 236 | /* Switchdev offloading can be configured on: */ |
| 237 | 237 | ||
| @@ -241,12 +241,6 @@ static inline bool dsa_port_offloads_netdev(struct dsa_port *dp, · 3 unmodified lines | |||
| 241 | */ | 241 | */ |
| 242 | return true; | 242 | return true; |
| 243 | 243 | ||
| 244 | if (dp->bridge_dev == dev) | ||
| 245 | /* DSA ports connected to a bridge, and event was emitted | ||
| 246 | * for the bridge. | ||
| 247 | */ | ||
| 248 | return true; | ||
| 249 | |||
| 250 | if (dp->lag_dev == dev) | 244 | if (dp->lag_dev == dev) |
| 251 | /* DSA ports connected to a bridge via a LAG */ | 245 | /* DSA ports connected to a bridge via a LAG */ |
| 252 | return true; | 246 | return true; |
| @@ -254,14 +248,23 @@ static inline bool dsa_port_offloads_netdev(struct dsa_port *dp, · 1 unmodified line | |||
| 254 | return false; | 248 | return false; |
| 255 | } | 249 | } |
| 256 | 250 | ||
| 251 | static inline bool dsa_port_offloads_bridge(struct dsa_port *dp, | ||
| 252 | struct net_device *bridge_dev) | ||
| 253 | { | ||
| 254 | /* DSA ports connected to a bridge, and event was emitted | ||
| 255 | * for the bridge. | ||
| 256 | */ | ||
| 257 | return dp->bridge_dev == bridge_dev; | ||
| 258 | } | ||
| 259 | |||
| 257 | /* Returns true if any port of this tree offloads the given net_device */ | 260 | /* Returns true if any port of this tree offloads the given net_device */ |
| 258 | static inline bool dsa_tree_offloads_netdev(struct dsa_switch_tree *dst, | 261 | static inline bool dsa_tree_offloads_bridge_port(struct dsa_switch_tree *dst, |
| 259 | struct net_device *dev) | 262 | struct net_device *dev) |
| 260 | { | 263 | { |
| 261 | struct dsa_port *dp; | 264 | struct dsa_port *dp; |
| 262 | 265 | ||
| 263 | list_for_each_entry(dp, &dst->ports, list) | 266 | list_for_each_entry(dp, &dst->ports, list) |
| 264 | if (dsa_port_offloads_netdev(dp, dev)) | 267 | if (dsa_port_offloads_bridge_port(dp, dev)) |
| 265 | return true; | 268 | return true; |
| 266 | 269 | ||
| 267 | return false; | 270 | return false; |
| @@ -278,28 +278,43 @@ static int dsa_slave_port_attr_set(struct net_device *dev, · 277 unmodified lines | |||
| 278 | struct dsa_port *dp = dsa_slave_to_port(dev); | 278 | struct dsa_port *dp = dsa_slave_to_port(dev); |
| 279 | int ret; | 279 | int ret; |
| 280 | 280 | ||
| 281 | if (!dsa_port_offloads_netdev(dp, attr->orig_dev)) | ||
| 282 | return -EOPNOTSUPP; | ||
| 283 | |||
| 284 | switch (attr->id) { | 281 | switch (attr->id) { |
| 285 | case SWITCHDEV_ATTR_ID_PORT_STP_STATE: | 282 | case SWITCHDEV_ATTR_ID_PORT_STP_STATE: |
| 283 | if (!dsa_port_offloads_bridge_port(dp, attr->orig_dev)) | ||
| 284 | return -EOPNOTSUPP; | ||
| 285 | |||
| 286 | ret = dsa_port_set_state(dp, attr->u.stp_state); | 286 | ret = dsa_port_set_state(dp, attr->u.stp_state); |
| 287 | break; | 287 | break; |
| 288 | case SWITCHDEV_ATTR_ID_BRIDGE_VLAN_FILTERING: | 288 | case SWITCHDEV_ATTR_ID_BRIDGE_VLAN_FILTERING: |
| 289 | if (!dsa_port_offloads_bridge(dp, attr->orig_dev)) | ||
| 290 | return -EOPNOTSUPP; | ||
| 291 | |||
| 289 | ret = dsa_port_vlan_filtering(dp, attr->u.vlan_filtering, | 292 | ret = dsa_port_vlan_filtering(dp, attr->u.vlan_filtering, |
| 290 | extack); | 293 | extack); |
| 291 | break; | 294 | break; |
| 292 | case SWITCHDEV_ATTR_ID_BRIDGE_AGEING_TIME: | 295 | case SWITCHDEV_ATTR_ID_BRIDGE_AGEING_TIME: |
| 296 | if (!dsa_port_offloads_bridge(dp, attr->orig_dev)) | ||
| 297 | return -EOPNOTSUPP; | ||
| 298 | |||
| 293 | ret = dsa_port_ageing_time(dp, attr->u.ageing_time); | 299 | ret = dsa_port_ageing_time(dp, attr->u.ageing_time); |
| 294 | break; | 300 | break; |
| 295 | case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS: | 301 | case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS: |
| 302 | if (!dsa_port_offloads_bridge_port(dp, attr->orig_dev)) | ||
| 303 | return -EOPNOTSUPP; | ||
| 304 | |||
| 296 | ret = dsa_port_pre_bridge_flags(dp, attr->u.brport_flags, | 305 | ret = dsa_port_pre_bridge_flags(dp, attr->u.brport_flags, |
| 297 | extack); | 306 | extack); |
| 298 | break; | 307 | break; |
| 299 | case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS: | 308 | case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS: |
| 309 | if (!dsa_port_offloads_bridge_port(dp, attr->orig_dev)) | ||
| 310 | return -EOPNOTSUPP; | ||
| 311 | |||
| 300 | ret = dsa_port_bridge_flags(dp, attr->u.brport_flags, extack); | 312 | ret = dsa_port_bridge_flags(dp, attr->u.brport_flags, extack); |
| 301 | break; | 313 | break; |
| 302 | case SWITCHDEV_ATTR_ID_BRIDGE_MROUTER: | 314 | case SWITCHDEV_ATTR_ID_BRIDGE_MROUTER: |
| 315 | if (!dsa_port_offloads_bridge(dp, attr->orig_dev)) | ||
| 316 | return -EOPNOTSUPP; | ||
| 317 | |||
| 303 | ret = dsa_port_mrouter(dp->cpu_dp, attr->u.mrouter, extack); | 318 | ret = dsa_port_mrouter(dp->cpu_dp, attr->u.mrouter, extack); |
| 304 | break; | 319 | break; |
| 305 | default: | 320 | default: |
| @@ -341,9 +356,6 @@ static int dsa_slave_vlan_add(struct net_device *dev, · 35 unmodified lines | |||
| 341 | struct switchdev_obj_port_vlan vlan; | 356 | struct switchdev_obj_port_vlan vlan; |
| 342 | int err; | 357 | int err; |
| 343 | 358 | ||
| 344 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | ||
| 345 | return -EOPNOTSUPP; | ||
| 346 | |||
| 347 | if (dsa_port_skip_vlan_configuration(dp)) { | 359 | if (dsa_port_skip_vlan_configuration(dp)) { |
| 348 | NL_SET_ERR_MSG_MOD(extack, "skipping configuration of VLAN"); | 360 | NL_SET_ERR_MSG_MOD(extack, "skipping configuration of VLAN"); |
| 349 | return 0; | 361 | return 0; |
| @@ -391,27 +403,36 @@ static int dsa_slave_port_obj_add(struct net_device *dev, · 41 unmodified lines | |||
| 391 | 403 | ||
| 392 | switch (obj->id) { | 404 | switch (obj->id) { |
| 393 | case SWITCHDEV_OBJ_ID_PORT_MDB: | 405 | case SWITCHDEV_OBJ_ID_PORT_MDB: |
| 394 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 406 | if (!dsa_port_offloads_bridge_port(dp, obj->orig_dev)) |
| 395 | return -EOPNOTSUPP; | 407 | return -EOPNOTSUPP; |
| 408 | |||
| 396 | err = dsa_port_mdb_add(dp, SWITCHDEV_OBJ_PORT_MDB(obj)); | 409 | err = dsa_port_mdb_add(dp, SWITCHDEV_OBJ_PORT_MDB(obj)); |
| 397 | break; | 410 | break; |
| 398 | case SWITCHDEV_OBJ_ID_HOST_MDB: | 411 | case SWITCHDEV_OBJ_ID_HOST_MDB: |
| 412 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) | ||
| 413 | return -EOPNOTSUPP; | ||
| 414 | |||
| 399 | /* DSA can directly translate this to a normal MDB add, | 415 | /* DSA can directly translate this to a normal MDB add, |
| 400 | * but on the CPU port. | 416 | * but on the CPU port. |
| 401 | */ | 417 | */ |
| 402 | err = dsa_port_mdb_add(dp->cpu_dp, SWITCHDEV_OBJ_PORT_MDB(obj)); | 418 | err = dsa_port_mdb_add(dp->cpu_dp, SWITCHDEV_OBJ_PORT_MDB(obj)); |
| 403 | break; | 419 | break; |
| 404 | case SWITCHDEV_OBJ_ID_PORT_VLAN: | 420 | case SWITCHDEV_OBJ_ID_PORT_VLAN: |
| 421 | if (!dsa_port_offloads_bridge_port(dp, obj->orig_dev)) | ||
| 422 | return -EOPNOTSUPP; | ||
| 423 | |||
| 405 | err = dsa_slave_vlan_add(dev, obj, extack); | 424 | err = dsa_slave_vlan_add(dev, obj, extack); |
| 406 | break; | 425 | break; |
| 407 | case SWITCHDEV_OBJ_ID_MRP: | 426 | case SWITCHDEV_OBJ_ID_MRP: |
| 408 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 427 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) |
| 409 | return -EOPNOTSUPP; | 428 | return -EOPNOTSUPP; |
| 429 | |||
| 410 | err = dsa_port_mrp_add(dp, SWITCHDEV_OBJ_MRP(obj)); | 430 | err = dsa_port_mrp_add(dp, SWITCHDEV_OBJ_MRP(obj)); |
| 411 | break; | 431 | break; |
| 412 | case SWITCHDEV_OBJ_ID_RING_ROLE_MRP: | 432 | case SWITCHDEV_OBJ_ID_RING_ROLE_MRP: |
| 413 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 433 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) |
| 414 | return -EOPNOTSUPP; | 434 | return -EOPNOTSUPP; |
| 435 | |||
| 415 | err = dsa_port_mrp_add_ring_role(dp, | 436 | err = dsa_port_mrp_add_ring_role(dp, |
| 416 | SWITCHDEV_OBJ_RING_ROLE_MRP(obj)); | 437 | SWITCHDEV_OBJ_RING_ROLE_MRP(obj)); |
| 417 | break; | 438 | break; |
| @@ -431,9 +452,6 @@ static int dsa_slave_vlan_del(struct net_device *dev, · 13 unmodified lines | |||
| 431 | struct switchdev_obj_port_vlan *vlan; | 452 | struct switchdev_obj_port_vlan *vlan; |
| 432 | int err; | 453 | int err; |
| 433 | 454 | ||
| 434 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | ||
| 435 | return -EOPNOTSUPP; | ||
| 436 | |||
| 437 | if (dsa_port_skip_vlan_configuration(dp)) | 455 | if (dsa_port_skip_vlan_configuration(dp)) |
| 438 | return 0; | 456 | return 0; |
| 439 | 457 | ||
| @@ -459,27 +477,36 @@ static int dsa_slave_port_obj_del(struct net_device *dev, · 19 unmodified lines | |||
| 459 | 477 | ||
| 460 | switch (obj->id) { | 478 | switch (obj->id) { |
| 461 | case SWITCHDEV_OBJ_ID_PORT_MDB: | 479 | case SWITCHDEV_OBJ_ID_PORT_MDB: |
| 462 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 480 | if (!dsa_port_offloads_bridge_port(dp, obj->orig_dev)) |
| 463 | return -EOPNOTSUPP; | 481 | return -EOPNOTSUPP; |
| 482 | |||
| 464 | err = dsa_port_mdb_del(dp, SWITCHDEV_OBJ_PORT_MDB(obj)); | 483 | err = dsa_port_mdb_del(dp, SWITCHDEV_OBJ_PORT_MDB(obj)); |
| 465 | break; | 484 | break; |
| 466 | case SWITCHDEV_OBJ_ID_HOST_MDB: | 485 | case SWITCHDEV_OBJ_ID_HOST_MDB: |
| 486 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) | ||
| 487 | return -EOPNOTSUPP; | ||
| 488 | |||
| 467 | /* DSA can directly translate this to a normal MDB add, | 489 | /* DSA can directly translate this to a normal MDB add, |
| 468 | * but on the CPU port. | 490 | * but on the CPU port. |
| 469 | */ | 491 | */ |
| 470 | err = dsa_port_mdb_del(dp->cpu_dp, SWITCHDEV_OBJ_PORT_MDB(obj)); | 492 | err = dsa_port_mdb_del(dp->cpu_dp, SWITCHDEV_OBJ_PORT_MDB(obj)); |
| 471 | break; | 493 | break; |
| 472 | case SWITCHDEV_OBJ_ID_PORT_VLAN: | 494 | case SWITCHDEV_OBJ_ID_PORT_VLAN: |
| 495 | if (!dsa_port_offloads_bridge_port(dp, obj->orig_dev)) | ||
| 496 | return -EOPNOTSUPP; | ||
| 497 | |||
| 473 | err = dsa_slave_vlan_del(dev, obj); | 498 | err = dsa_slave_vlan_del(dev, obj); |
| 474 | break; | 499 | break; |
| 475 | case SWITCHDEV_OBJ_ID_MRP: | 500 | case SWITCHDEV_OBJ_ID_MRP: |
| 476 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 501 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) |
| 477 | return -EOPNOTSUPP; | 502 | return -EOPNOTSUPP; |
| 503 | |||
| 478 | err = dsa_port_mrp_del(dp, SWITCHDEV_OBJ_MRP(obj)); | 504 | err = dsa_port_mrp_del(dp, SWITCHDEV_OBJ_MRP(obj)); |
| 479 | break; | 505 | break; |
| 480 | case SWITCHDEV_OBJ_ID_RING_ROLE_MRP: | 506 | case SWITCHDEV_OBJ_ID_RING_ROLE_MRP: |
| 481 | if (!dsa_port_offloads_netdev(dp, obj->orig_dev)) | 507 | if (!dsa_port_offloads_bridge(dp, obj->orig_dev)) |
| 482 | return -EOPNOTSUPP; | 508 | return -EOPNOTSUPP; |
| 509 | |||
| 483 | err = dsa_port_mrp_del_ring_role(dp, | 510 | err = dsa_port_mrp_del_ring_role(dp, |
| 484 | SWITCHDEV_OBJ_RING_ROLE_MRP(obj)); | 511 | SWITCHDEV_OBJ_RING_ROLE_MRP(obj)); |
| 485 | break; | 512 | break; |
| @@ -2298,7 +2325,7 @@ static int dsa_slave_switchdev_event(struct notifier_block *unused, · 1812 unmodified lines | |||
| 2298 | * other ports bridged with the LAG should be able to | 2325 | * other ports bridged with the LAG should be able to |
| 2299 | * autonomously forward towards it. | 2326 | * autonomously forward towards it. |
| 2300 | */ | 2327 | */ |
| 2301 | if (dsa_tree_offloads_netdev(dp->ds->dst, dev)) | 2328 | if (dsa_tree_offloads_bridge_port(dp->ds->dst, dev)) |
| 2302 | return NOTIFY_DONE; | 2329 | return NOTIFY_DONE; |
| 2303 | } | 2330 | } |
| 2304 | 2331 | ||
| @@ -13,6 +13,7 @@ · 12 unmodified lines | |||
| 13 | #define MTK_HDR_LEN 4 | 13 | #define MTK_HDR_LEN 4 |
| 14 | #define MTK_HDR_XMIT_UNTAGGED 0 | 14 | #define MTK_HDR_XMIT_UNTAGGED 0 |
| 15 | #define MTK_HDR_XMIT_TAGGED_TPID_8100 1 | 15 | #define MTK_HDR_XMIT_TAGGED_TPID_8100 1 |
| 16 | #define MTK_HDR_XMIT_TAGGED_TPID_88A8 2 | ||
| 16 | #define MTK_HDR_RECV_SOURCE_PORT_MASK GENMASK(2, 0) | 17 | #define MTK_HDR_RECV_SOURCE_PORT_MASK GENMASK(2, 0) |
| 17 | #define MTK_HDR_XMIT_DP_BIT_MASK GENMASK(5, 0) | 18 | #define MTK_HDR_XMIT_DP_BIT_MASK GENMASK(5, 0) |
| 18 | #define MTK_HDR_XMIT_SA_DIS BIT(6) | 19 | #define MTK_HDR_XMIT_SA_DIS BIT(6) |
| @@ -21,8 +22,8 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb, · 2 unmodified lines | |||
| 21 | struct net_device *dev) | 22 | struct net_device *dev) |
| 22 | { | 23 | { |
| 23 | struct dsa_port *dp = dsa_slave_to_port(dev); | 24 | struct dsa_port *dp = dsa_slave_to_port(dev); |
| 25 | u8 xmit_tpid; | ||
| 24 | u8 *mtk_tag; | 26 | u8 *mtk_tag; |
| 25 | bool is_vlan_skb = true; | ||
| 26 | unsigned char *dest = eth_hdr(skb)->h_dest; | 27 | unsigned char *dest = eth_hdr(skb)->h_dest; |
| 27 | bool is_multicast_skb = is_multicast_ether_addr(dest) && | 28 | bool is_multicast_skb = is_multicast_ether_addr(dest) && |
| 28 | !is_broadcast_ether_addr(dest); | 29 | !is_broadcast_ether_addr(dest); |
| @@ -33,10 +34,17 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb, · 4 unmodified lines | |||
| 33 | * the both special and VLAN tag at the same time and then look up VLAN | 34 | * the both special and VLAN tag at the same time and then look up VLAN |
| 34 | * table with VID. | 35 | * table with VID. |
| 35 | */ | 36 | */ |
| 36 | if (!skb_vlan_tagged(skb)) { | 37 | switch (skb->protocol) { |
| 38 | case htons(ETH_P_8021Q): | ||
| 39 | xmit_tpid = MTK_HDR_XMIT_TAGGED_TPID_8100; | ||
| 40 | break; | ||
| 41 | case htons(ETH_P_8021AD): | ||
| 42 | xmit_tpid = MTK_HDR_XMIT_TAGGED_TPID_88A8; | ||
| 43 | break; | ||
| 44 | default: | ||
| 45 | xmit_tpid = MTK_HDR_XMIT_UNTAGGED; | ||
| 37 | skb_push(skb, MTK_HDR_LEN); | 46 | skb_push(skb, MTK_HDR_LEN); |
| 38 | memmove(skb->data, skb->data + MTK_HDR_LEN, 2 * ETH_ALEN); | 47 | memmove(skb->data, skb->data + MTK_HDR_LEN, 2 * ETH_ALEN); |
| 39 | is_vlan_skb = false; | ||
| 40 | } | 48 | } |
| 41 | 49 | ||
| 42 | mtk_tag = skb->data + 2 * ETH_ALEN; | 50 | mtk_tag = skb->data + 2 * ETH_ALEN; |
| @@ -44,8 +52,7 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb, · 1 unmodified line | |||
| 44 | /* Mark tag attribute on special tag insertion to notify hardware | 52 | /* Mark tag attribute on special tag insertion to notify hardware |
| 45 | * whether that's a combined special tag with 802.1Q header. | 53 | * whether that's a combined special tag with 802.1Q header. |
| 46 | */ | 54 | */ |
| 47 | mtk_tag[0] = is_vlan_skb ? MTK_HDR_XMIT_TAGGED_TPID_8100 : | 55 | mtk_tag[0] = xmit_tpid; |
| 48 | MTK_HDR_XMIT_UNTAGGED; | ||
| 49 | mtk_tag[1] = (1 << dp->index) & MTK_HDR_XMIT_DP_BIT_MASK; | 56 | mtk_tag[1] = (1 << dp->index) & MTK_HDR_XMIT_DP_BIT_MASK; |
| 50 | 57 | ||
| 51 | /* Disable SA learning for multicast frames */ | 58 | /* Disable SA learning for multicast frames */ |
| @@ -53,7 +60,7 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb, · 1 unmodified line | |||
| 53 | mtk_tag[1] |= MTK_HDR_XMIT_SA_DIS; | 60 | mtk_tag[1] |= MTK_HDR_XMIT_SA_DIS; |
| 54 | 61 | ||
| 55 | /* Tag control information is kept for 802.1Q */ | 62 | /* Tag control information is kept for 802.1Q */ |
| 56 | if (!is_vlan_skb) { | 63 | if (xmit_tpid == MTK_HDR_XMIT_UNTAGGED) { |
| 57 | mtk_tag[2] = 0; | 64 | mtk_tag[2] = 0; |
| 58 | mtk_tag[3] = 0; | 65 | mtk_tag[3] = 0; |
| 59 | } | 66 | } |
| @@ -35,14 +35,12 @@ static struct sk_buff *rtl4a_tag_xmit(struct sk_buff *skb, · 34 unmodified lines | |||
| 35 | struct net_device *dev) | 35 | struct net_device *dev) |
| 36 | { | 36 | { |
| 37 | struct dsa_port *dp = dsa_slave_to_port(dev); | 37 | struct dsa_port *dp = dsa_slave_to_port(dev); |
| 38 | __be16 *p; | ||
| 38 | u8 *tag; | 39 | u8 *tag; |
| 39 | u16 *p; | ||
| 40 | u16 out; | 40 | u16 out; |
| 41 | 41 | ||
| 42 | /* Pad out to at least 60 bytes */ | 42 | /* Pad out to at least 60 bytes */ |
| 43 | if (unlikely(eth_skb_pad(skb))) | 43 | if (unlikely(__skb_put_padto(skb, ETH_ZLEN, false))) |
| 44 | return NULL; | ||
| 45 | if (skb_cow_head(skb, RTL4_A_HDR_LEN) < 0) | ||
| 46 | return NULL; | 44 | return NULL; |
| 47 | 45 | ||
| 48 | netdev_dbg(dev, "add realtek tag to package to port %d\n", | 46 | netdev_dbg(dev, "add realtek tag to package to port %d\n", |
| @@ -53,13 +51,13 @@ static struct sk_buff *rtl4a_tag_xmit(struct sk_buff *skb, · 4 unmodified lines | |||
| 53 | tag = skb->data + 2 * ETH_ALEN; | 51 | tag = skb->data + 2 * ETH_ALEN; |
| 54 | 52 | ||
| 55 | /* Set Ethertype */ | 53 | /* Set Ethertype */ |
| 56 | p = (u16 *)tag; | 54 | p = (__be16 *)tag; |
| 57 | *p = htons(RTL4_A_ETHERTYPE); | 55 | *p = htons(RTL4_A_ETHERTYPE); |
| 58 | 56 | ||
| 59 | out = (RTL4_A_PROTOCOL_RTL8366RB << 12) | (2 << 8); | 57 | out = (RTL4_A_PROTOCOL_RTL8366RB << 12) | (2 << 8); |
| 60 | /* The lower bits is the port numer */ | 58 | /* The lower bits is the port number */ |
| 61 | out |= (u8)dp->index; | 59 | out |= (u8)dp->index; |
| 62 | p = (u16 *)(tag + 2); | 60 | p = (__be16 *)(tag + 2); |
| 63 | *p = htons(out); | 61 | *p = htons(out); |
| 64 | 62 | ||
| 65 | return skb; | 63 | return skb; |
| @@ -116,10 +116,9 @@ int ethnl_set_channels(struct sk_buff *skb, struct genl_info *info) · 115 unmodified lines | |||
| 116 | struct ethtool_channels channels = {}; | 116 | struct ethtool_channels channels = {}; |
| 117 | struct ethnl_req_info req_info = {}; | 117 | struct ethnl_req_info req_info = {}; |
| 118 | struct nlattr **tb = info->attrs; | 118 | struct nlattr **tb = info->attrs; |
| 119 | const struct nlattr *err_attr; | 119 | u32 err_attr, max_rx_in_use = 0; |
| 120 | const struct ethtool_ops *ops; | 120 | const struct ethtool_ops *ops; |
| 121 | struct net_device *dev; | 121 | struct net_device *dev; |
| 122 | u32 max_rx_in_use = 0; | ||
| 123 | int ret; | 122 | int ret; |
| 124 | 123 | ||
| 125 | ret = ethnl_parse_header_dev_get(&req_info, | 124 | ret = ethnl_parse_header_dev_get(&req_info, |
| @@ -157,34 +156,35 @@ int ethnl_set_channels(struct sk_buff *skb, struct genl_info *info) · 31 unmodified lines | |||
| 157 | 156 | ||
| 158 | /* ensure new channel counts are within limits */ | 157 | /* ensure new channel counts are within limits */ |
| 159 | if (channels.rx_count > channels.max_rx) | 158 | if (channels.rx_count > channels.max_rx) |
| 160 | err_attr = tb[ETHTOOL_A_CHANNELS_RX_COUNT]; | 159 | err_attr = ETHTOOL_A_CHANNELS_RX_COUNT; |
| 161 | else if (channels.tx_count > channels.max_tx) | 160 | else if (channels.tx_count > channels.max_tx) |
| 162 | err_attr = tb[ETHTOOL_A_CHANNELS_TX_COUNT]; | 161 | err_attr = ETHTOOL_A_CHANNELS_TX_COUNT; |
| 163 | else if (channels.other_count > channels.max_other) | 162 | else if (channels.other_count > channels.max_other) |
| 164 | err_attr = tb[ETHTOOL_A_CHANNELS_OTHER_COUNT]; | 163 | err_attr = ETHTOOL_A_CHANNELS_OTHER_COUNT; |
| 165 | else if (channels.combined_count > channels.max_combined) | 164 | else if (channels.combined_count > channels.max_combined) |
| 166 | err_attr = tb[ETHTOOL_A_CHANNELS_COMBINED_COUNT]; | 165 | err_attr = ETHTOOL_A_CHANNELS_COMBINED_COUNT; |
| 167 | else | 166 | else |
| 168 | err_attr = NULL; | 167 | err_attr = 0; |
| 169 | if (err_attr) { | 168 | if (err_attr) { |
| 170 | ret = -EINVAL; | 169 | ret = -EINVAL; |
| 171 | NL_SET_ERR_MSG_ATTR(info->extack, err_attr, | 170 | NL_SET_ERR_MSG_ATTR(info->extack, tb[err_attr], |
| 172 | "requested channel count exceeds maximum"); | 171 | "requested channel count exceeds maximum"); |
| 173 | goto out_ops; | 172 | goto out_ops; |
| 174 | } | 173 | } |
| 175 | 174 | ||
| 176 | /* ensure there is at least one RX and one TX channel */ | 175 | /* ensure there is at least one RX and one TX channel */ |
| 177 | if (!channels.combined_count && !channels.rx_count) | 176 | if (!channels.combined_count && !channels.rx_count) |
| 178 | err_attr = tb[ETHTOOL_A_CHANNELS_RX_COUNT]; | 177 | err_attr = ETHTOOL_A_CHANNELS_RX_COUNT; |
| 179 | else if (!channels.combined_count && !channels.tx_count) | 178 | else if (!channels.combined_count && !channels.tx_count) |
| 180 | err_attr = tb[ETHTOOL_A_CHANNELS_TX_COUNT]; | 179 | err_attr = ETHTOOL_A_CHANNELS_TX_COUNT; |
| 181 | else | 180 | else |
| 182 | err_attr = NULL; | 181 | err_attr = 0; |
| 183 | if (err_attr) { | 182 | if (err_attr) { |
| 184 | if (mod_combined) | 183 | if (mod_combined) |
| 185 | err_attr = tb[ETHTOOL_A_CHANNELS_COMBINED_COUNT]; | 184 | err_attr = ETHTOOL_A_CHANNELS_COMBINED_COUNT; |
| 186 | ret = -EINVAL; | 185 | ret = -EINVAL; |
| 187 | NL_SET_ERR_MSG_ATTR(info->extack, err_attr, "requested channel counts would result in no RX or TX channel being configured"); | 186 | NL_SET_ERR_MSG_ATTR(info->extack, tb[err_attr], |
| 187 | "requested channel counts would result in no RX or TX channel being configured"); | ||
| 188 | goto out_ops; | 188 | goto out_ops; |
| 189 | } | 189 | } |
| 190 | 190 | ||
| @@ -519,16 +519,10 @@ int cipso_v4_doi_remove(u32 doi, struct netlbl_audit *audit_info) · 518 unmodified lines | |||
| 519 | ret_val = -ENOENT; | 519 | ret_val = -ENOENT; |
| 520 | goto doi_remove_return; | 520 | goto doi_remove_return; |
| 521 | } | 521 | } |
| 522 | if (!refcount_dec_and_test(&doi_def->refcount)) { | ||
| 523 | spin_unlock(&cipso_v4_doi_list_lock); | ||
| 524 | ret_val = -EBUSY; | ||
| 525 | goto doi_remove_return; | ||
| 526 | } | ||
| 527 | list_del_rcu(&doi_def->list); | 522 | list_del_rcu(&doi_def->list); |
| 528 | spin_unlock(&cipso_v4_doi_list_lock); | 523 | spin_unlock(&cipso_v4_doi_list_lock); |
| 529 | 524 | ||
| 530 | cipso_v4_cache_invalidate(); | 525 | cipso_v4_doi_putdef(doi_def); |
| 531 | call_rcu(&doi_def->rcu, cipso_v4_doi_free_rcu); | ||
| 532 | ret_val = 0; | 526 | ret_val = 0; |
| 533 | 527 | ||
| 534 | doi_remove_return: | 528 | doi_remove_return: |
| @@ -585,9 +579,6 @@ void cipso_v4_doi_putdef(struct cipso_v4_doi *doi_def) · 50 unmodified lines | |||
| 585 | 579 | ||
| 586 | if (!refcount_dec_and_test(&doi_def->refcount)) | 580 | if (!refcount_dec_and_test(&doi_def->refcount)) |
| 587 | return; | 581 | return; |
| 588 | spin_lock(&cipso_v4_doi_list_lock); | ||
| 589 | list_del_rcu(&doi_def->list); | ||
| 590 | spin_unlock(&cipso_v4_doi_list_lock); | ||
| 591 | 582 | ||
| 592 | cipso_v4_cache_invalidate(); | 583 | cipso_v4_cache_invalidate(); |
| 593 | call_rcu(&doi_def->rcu, cipso_v4_doi_free_rcu); | 584 | call_rcu(&doi_def->rcu, cipso_v4_doi_free_rcu); |
| @@ -1162,7 +1153,7 @@ static void cipso_v4_gentag_hdr(const struct cipso_v4_doi *doi_def, · 568 unmodified lines | |||
| 1162 | { | 1153 | { |
| 1163 | buf[0] = IPOPT_CIPSO; | 1154 | buf[0] = IPOPT_CIPSO; |
| 1164 | buf[1] = CIPSO_V4_HDR_LEN + len; | 1155 | buf[1] = CIPSO_V4_HDR_LEN + len; |
| 1165 | *(__be32 *)&buf[2] = htonl(doi_def->doi); | 1156 | put_unaligned_be32(doi_def->doi, &buf[2]); |
| 1166 | } | 1157 | } |
| 1167 | 1158 | ||
| 1168 | /** | 1159 | /** |
| @@ -65,7 +65,7 @@ EXPORT_SYMBOL_GPL(inet_peer_base_init); · 64 unmodified lines | |||
| 65 | #define PEER_MAX_GC 32 | 65 | #define PEER_MAX_GC 32 |
| 66 | 66 | ||
| 67 | /* Exported for sysctl_net_ipv4. */ | 67 | /* Exported for sysctl_net_ipv4. */ |
| 68 | int inet_peer_threshold __read_mostly = 65536 + 128; /* start to throw entries more | 68 | int inet_peer_threshold __read_mostly; /* start to throw entries more |
| 69 | * aggressively at this stage */ | 69 | * aggressively at this stage */ |
| 70 | int inet_peer_minttl __read_mostly = 120 * HZ; /* TTL under high load: 120 sec */ | 70 | int inet_peer_minttl __read_mostly = 120 * HZ; /* TTL under high load: 120 sec */ |
| 71 | int inet_peer_maxttl __read_mostly = 10 * 60 * HZ; /* usual time to live: 10 min */ | 71 | int inet_peer_maxttl __read_mostly = 10 * 60 * HZ; /* usual time to live: 10 min */ |
| @@ -73,20 +73,13 @@ int inet_peer_maxttl __read_mostly = 10 * 60 * HZ; /* usual time to live: 10 min · 1 unmodified line | |||
| 73 | /* Called from ip_output.c:ip_init */ | 73 | /* Called from ip_output.c:ip_init */ |
| 74 | void __init inet_initpeers(void) | 74 | void __init inet_initpeers(void) |
| 75 | { | 75 | { |
| 76 | struct sysinfo si; | 76 | u64 nr_entries; |
| 77 | 77 | ||
| 78 | /* Use the straight interface to information about memory. */ | 78 | /* 1% of physical memory */ |
| 79 | si_meminfo(&si); | 79 | nr_entries = div64_ul((u64)totalram_pages() << PAGE_SHIFT, |
| 80 | /* The values below were suggested by Alexey Kuznetsov | 80 | 100 * L1_CACHE_ALIGN(sizeof(struct inet_peer))); |
| 81 | * <kuznet@ms2.inr.ac.ru>. I don't have any opinion about the values | 81 | |
| 82 | * myself. --SAW | 82 | inet_peer_threshold = clamp_val(nr_entries, 4096, 65536 + 128); |
| 83 | */ | ||
| 84 | if (si.totalram <= (32768*1024)/PAGE_SIZE) | ||
| 85 | inet_peer_threshold >>= 1; /* max pool size about 1MB on IA32 */ | ||
| 86 | if (si.totalram <= (16384*1024)/PAGE_SIZE) | ||
| 87 | inet_peer_threshold >>= 1; /* about 512KB */ | ||
| 88 | if (si.totalram <= (8192*1024)/PAGE_SIZE) | ||
| 89 | inet_peer_threshold >>= 2; /* about 128KB */ | ||
| 90 | 83 | ||
| 91 | peer_cachep = kmem_cache_create("inet_peer_cache", | 84 | peer_cachep = kmem_cache_create("inet_peer_cache", |
| 92 | sizeof(struct inet_peer), | 85 | sizeof(struct inet_peer), |
| @@ -502,8 +502,7 @@ static int tnl_update_pmtu(struct net_device *dev, struct sk_buff *skb, · 501 unmodified lines | |||
| 502 | if (!skb_is_gso(skb) && | 502 | if (!skb_is_gso(skb) && |
| 503 | (inner_iph->frag_off & htons(IP_DF)) && | 503 | (inner_iph->frag_off & htons(IP_DF)) && |
| 504 | mtu < pkt_size) { | 504 | mtu < pkt_size) { |
| 505 | memset(IPCB(skb), 0, sizeof(*IPCB(skb))); | 505 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, htonl(mtu)); |
| 506 | icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, htonl(mtu)); | ||
| 507 | return -E2BIG; | 506 | return -E2BIG; |
| 508 | } | 507 | } |
| 509 | } | 508 | } |
| @@ -527,7 +526,7 @@ static int tnl_update_pmtu(struct net_device *dev, struct sk_buff *skb, · 17 unmodified lines | |||
| 527 | 526 | ||
| 528 | if (!skb_is_gso(skb) && mtu >= IPV6_MIN_MTU && | 527 | if (!skb_is_gso(skb) && mtu >= IPV6_MIN_MTU && |
| 529 | mtu < pkt_size) { | 528 | mtu < pkt_size) { |
| 530 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 529 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 531 | return -E2BIG; | 530 | return -E2BIG; |
| 532 | } | 531 | } |
| 533 | } | 532 | } |
| @@ -238,13 +238,13 @@ static netdev_tx_t vti_xmit(struct sk_buff *skb, struct net_device *dev, · 237 unmodified lines | |||
| 238 | if (skb->len > mtu) { | 238 | if (skb->len > mtu) { |
| 239 | skb_dst_update_pmtu_no_confirm(skb, mtu); | 239 | skb_dst_update_pmtu_no_confirm(skb, mtu); |
| 240 | if (skb->protocol == htons(ETH_P_IP)) { | 240 | if (skb->protocol == htons(ETH_P_IP)) { |
| 241 | icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, | 241 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, |
| 242 | htonl(mtu)); | 242 | htonl(mtu)); |
| 243 | } else { | 243 | } else { |
| 244 | if (mtu < IPV6_MIN_MTU) | 244 | if (mtu < IPV6_MIN_MTU) |
| 245 | mtu = IPV6_MIN_MTU; | 245 | mtu = IPV6_MIN_MTU; |
| 246 | 246 | ||
| 247 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 247 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 248 | } | 248 | } |
| 249 | 249 | ||
| 250 | dst_release(dst); | 250 | dst_release(dst); |
| @@ -1399,7 +1399,7 @@ out: · 1398 unmodified lines | |||
| 1399 | 1399 | ||
| 1400 | /* rtnl */ | 1400 | /* rtnl */ |
| 1401 | /* remove all nexthops tied to a device being deleted */ | 1401 | /* remove all nexthops tied to a device being deleted */ |
| 1402 | static void nexthop_flush_dev(struct net_device *dev) | 1402 | static void nexthop_flush_dev(struct net_device *dev, unsigned long event) |
| 1403 | { | 1403 | { |
| 1404 | unsigned int hash = nh_dev_hashfn(dev->ifindex); | 1404 | unsigned int hash = nh_dev_hashfn(dev->ifindex); |
| 1405 | struct net *net = dev_net(dev); | 1405 | struct net *net = dev_net(dev); |
| @@ -1411,6 +1411,10 @@ static void nexthop_flush_dev(struct net_device *dev) · 5 unmodified lines | |||
| 1411 | if (nhi->fib_nhc.nhc_dev != dev) | 1411 | if (nhi->fib_nhc.nhc_dev != dev) |
| 1412 | continue; | 1412 | continue; |
| 1413 | 1413 | ||
| 1414 | if (nhi->reject_nh && | ||
| 1415 | (event == NETDEV_DOWN || event == NETDEV_CHANGE)) | ||
| 1416 | continue; | ||
| 1417 | |||
| 1414 | remove_nexthop(net, nhi->nh_parent, NULL); | 1418 | remove_nexthop(net, nhi->nh_parent, NULL); |
| 1415 | } | 1419 | } |
| 1416 | } | 1420 | } |
| @@ -2189,11 +2193,11 @@ static int nh_netdev_event(struct notifier_block *this, · 772 unmodified lines | |||
| 2189 | switch (event) { | 2193 | switch (event) { |
| 2190 | case NETDEV_DOWN: | 2194 | case NETDEV_DOWN: |
| 2191 | case NETDEV_UNREGISTER: | 2195 | case NETDEV_UNREGISTER: |
| 2192 | nexthop_flush_dev(dev); | 2196 | nexthop_flush_dev(dev, event); |
| 2193 | break; | 2197 | break; |
| 2194 | case NETDEV_CHANGE: | 2198 | case NETDEV_CHANGE: |
| 2195 | if (!(dev_get_flags(dev) & (IFF_RUNNING | IFF_LOWER_UP))) | 2199 | if (!(dev_get_flags(dev) & (IFF_RUNNING | IFF_LOWER_UP))) |
| 2196 | nexthop_flush_dev(dev); | 2200 | nexthop_flush_dev(dev, event); |
| 2197 | break; | 2201 | break; |
| 2198 | case NETDEV_CHANGEMTU: | 2202 | case NETDEV_CHANGEMTU: |
| 2199 | info_ext = ptr; | 2203 | info_ext = ptr; |
| @@ -3469,16 +3469,23 @@ static int do_tcp_setsockopt(struct sock *sk, int level, int optname, · 3468 unmodified lines | |||
| 3469 | break; | 3469 | break; |
| 3470 | 3470 | ||
| 3471 | case TCP_QUEUE_SEQ: | 3471 | case TCP_QUEUE_SEQ: |
| 3472 | if (sk->sk_state != TCP_CLOSE) | 3472 | if (sk->sk_state != TCP_CLOSE) { |
| 3473 | err = -EPERM; | 3473 | err = -EPERM; |
| 3474 | else if (tp->repair_queue == TCP_SEND_QUEUE) | 3474 | } else if (tp->repair_queue == TCP_SEND_QUEUE) { |
| 3475 | WRITE_ONCE(tp->write_seq, val); | 3475 | if (!tcp_rtx_queue_empty(sk)) |
| 3476 | else if (tp->repair_queue == TCP_RECV_QUEUE) { | 3476 | err = -EPERM; |
| 3477 | WRITE_ONCE(tp->rcv_nxt, val); | 3477 | else |
| 3478 | WRITE_ONCE(tp->copied_seq, val); | 3478 | WRITE_ONCE(tp->write_seq, val); |
| 3479 | } | 3479 | } else if (tp->repair_queue == TCP_RECV_QUEUE) { |
| 3480 | else | 3480 | if (tp->rcv_nxt != tp->copied_seq) { |
| 3481 | err = -EPERM; | ||
| 3482 | } else { | ||
| 3483 | WRITE_ONCE(tp->rcv_nxt, val); | ||
| 3484 | WRITE_ONCE(tp->copied_seq, val); | ||
| 3485 | } | ||
| 3486 | } else { | ||
| 3481 | err = -EINVAL; | 3487 | err = -EINVAL; |
| 3488 | } | ||
| 3482 | break; | 3489 | break; |
| 3483 | 3490 | ||
| 3484 | case TCP_REPAIR_OPTIONS: | 3491 | case TCP_REPAIR_OPTIONS: |
| @@ -4143,7 +4150,8 @@ static int do_tcp_getsockopt(struct sock *sk, int level, · 658 unmodified lines | |||
| 4143 | 4150 | ||
| 4144 | if (get_user(len, optlen)) | 4151 | if (get_user(len, optlen)) |
| 4145 | return -EFAULT; | 4152 | return -EFAULT; |
| 4146 | if (len < offsetofend(struct tcp_zerocopy_receive, length)) | 4153 | if (len < 0 || |
| 4154 | len < offsetofend(struct tcp_zerocopy_receive, length)) | ||
| 4147 | return -EINVAL; | 4155 | return -EINVAL; |
| 4148 | if (unlikely(len > sizeof(zc))) { | 4156 | if (unlikely(len > sizeof(zc))) { |
| 4149 | err = check_zeroed_user(optval + sizeof(zc), | 4157 | err = check_zeroed_user(optval + sizeof(zc), |
| @@ -526,7 +526,7 @@ struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb, · 525 unmodified lines | |||
| 526 | } | 526 | } |
| 527 | 527 | ||
| 528 | if (!sk || NAPI_GRO_CB(skb)->encap_mark || | 528 | if (!sk || NAPI_GRO_CB(skb)->encap_mark || |
| 529 | (skb->ip_summed != CHECKSUM_PARTIAL && | 529 | (uh->check && skb->ip_summed != CHECKSUM_PARTIAL && |
| 530 | NAPI_GRO_CB(skb)->csum_cnt == 0 && | 530 | NAPI_GRO_CB(skb)->csum_cnt == 0 && |
| 531 | !NAPI_GRO_CB(skb)->csum_valid) || | 531 | !NAPI_GRO_CB(skb)->csum_valid) || |
| 532 | !udp_sk(sk)->gro_receive) | 532 | !udp_sk(sk)->gro_receive) |
| @@ -83,6 +83,9 @@ struct calipso_map_cache_entry { · 82 unmodified lines | |||
| 83 | 83 | ||
| 84 | static struct calipso_map_cache_bkt *calipso_cache; | 84 | static struct calipso_map_cache_bkt *calipso_cache; |
| 85 | 85 | ||
| 86 | static void calipso_cache_invalidate(void); | ||
| 87 | static void calipso_doi_putdef(struct calipso_doi *doi_def); | ||
| 88 | |||
| 86 | /* Label Mapping Cache Functions | 89 | /* Label Mapping Cache Functions |
| 87 | */ | 90 | */ |
| 88 | 91 | ||
| @@ -444,15 +447,10 @@ static int calipso_doi_remove(u32 doi, struct netlbl_audit *audit_info) · 355 unmodified lines | |||
| 444 | ret_val = -ENOENT; | 447 | ret_val = -ENOENT; |
| 445 | goto doi_remove_return; | 448 | goto doi_remove_return; |
| 446 | } | 449 | } |
| 447 | if (!refcount_dec_and_test(&doi_def->refcount)) { | ||
| 448 | spin_unlock(&calipso_doi_list_lock); | ||
| 449 | ret_val = -EBUSY; | ||
| 450 | goto doi_remove_return; | ||
| 451 | } | ||
| 452 | list_del_rcu(&doi_def->list); | 450 | list_del_rcu(&doi_def->list); |
| 453 | spin_unlock(&calipso_doi_list_lock); | 451 | spin_unlock(&calipso_doi_list_lock); |
| 454 | 452 | ||
| 455 | call_rcu(&doi_def->rcu, calipso_doi_free_rcu); | 453 | calipso_doi_putdef(doi_def); |
| 456 | ret_val = 0; | 454 | ret_val = 0; |
| 457 | 455 | ||
| 458 | doi_remove_return: | 456 | doi_remove_return: |
| @@ -508,10 +506,8 @@ static void calipso_doi_putdef(struct calipso_doi *doi_def) · 49 unmodified lines | |||
| 508 | 506 | ||
| 509 | if (!refcount_dec_and_test(&doi_def->refcount)) | 507 | if (!refcount_dec_and_test(&doi_def->refcount)) |
| 510 | return; | 508 | return; |
| 511 | spin_lock(&calipso_doi_list_lock); | ||
| 512 | list_del_rcu(&doi_def->list); | ||
| 513 | spin_unlock(&calipso_doi_list_lock); | ||
| 514 | 509 | ||
| 510 | calipso_cache_invalidate(); | ||
| 515 | call_rcu(&doi_def->rcu, calipso_doi_free_rcu); | 511 | call_rcu(&doi_def->rcu, calipso_doi_free_rcu); |
| 516 | } | 512 | } |
| 517 | 513 | ||
| @@ -678,8 +678,8 @@ static int prepare_ip6gre_xmit_ipv6(struct sk_buff *skb, · 677 unmodified lines | |||
| 678 | 678 | ||
| 679 | tel = (struct ipv6_tlv_tnl_enc_lim *)&skb_network_header(skb)[offset]; | 679 | tel = (struct ipv6_tlv_tnl_enc_lim *)&skb_network_header(skb)[offset]; |
| 680 | if (tel->encap_limit == 0) { | 680 | if (tel->encap_limit == 0) { |
| 681 | icmpv6_send(skb, ICMPV6_PARAMPROB, | 681 | icmpv6_ndo_send(skb, ICMPV6_PARAMPROB, |
| 682 | ICMPV6_HDR_FIELD, offset + 2); | 682 | ICMPV6_HDR_FIELD, offset + 2); |
| 683 | return -1; | 683 | return -1; |
| 684 | } | 684 | } |
| 685 | *encap_limit = tel->encap_limit - 1; | 685 | *encap_limit = tel->encap_limit - 1; |
| @@ -805,8 +805,8 @@ static inline int ip6gre_xmit_ipv4(struct sk_buff *skb, struct net_device *dev) · 119 unmodified lines | |||
| 805 | if (err != 0) { | 805 | if (err != 0) { |
| 806 | /* XXX: send ICMP error even if DF is not set. */ | 806 | /* XXX: send ICMP error even if DF is not set. */ |
| 807 | if (err == -EMSGSIZE) | 807 | if (err == -EMSGSIZE) |
| 808 | icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, | 808 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, |
| 809 | htonl(mtu)); | 809 | htonl(mtu)); |
| 810 | return -1; | 810 | return -1; |
| 811 | } | 811 | } |
| 812 | 812 | ||
| @@ -837,7 +837,7 @@ static inline int ip6gre_xmit_ipv6(struct sk_buff *skb, struct net_device *dev) · 24 unmodified lines | |||
| 837 | &mtu, skb->protocol); | 837 | &mtu, skb->protocol); |
| 838 | if (err != 0) { | 838 | if (err != 0) { |
| 839 | if (err == -EMSGSIZE) | 839 | if (err == -EMSGSIZE) |
| 840 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 840 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 841 | return -1; | 841 | return -1; |
| 842 | } | 842 | } |
| 843 | 843 | ||
| @@ -1063,10 +1063,10 @@ static netdev_tx_t ip6erspan_tunnel_xmit(struct sk_buff *skb, · 219 unmodified lines | |||
| 1063 | /* XXX: send ICMP error even if DF is not set. */ | 1063 | /* XXX: send ICMP error even if DF is not set. */ |
| 1064 | if (err == -EMSGSIZE) { | 1064 | if (err == -EMSGSIZE) { |
| 1065 | if (skb->protocol == htons(ETH_P_IP)) | 1065 | if (skb->protocol == htons(ETH_P_IP)) |
| 1066 | icmp_send(skb, ICMP_DEST_UNREACH, | 1066 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, |
| 1067 | ICMP_FRAG_NEEDED, htonl(mtu)); | 1067 | ICMP_FRAG_NEEDED, htonl(mtu)); |
| 1068 | else | 1068 | else |
| 1069 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 1069 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 1070 | } | 1070 | } |
| 1071 | 1071 | ||
| 1072 | goto tx_err; | 1072 | goto tx_err; |
| @@ -1332,8 +1332,8 @@ ipxip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev, · 1331 unmodified lines | |||
| 1332 | 1332 | ||
| 1333 | tel = (void *)&skb_network_header(skb)[offset]; | 1333 | tel = (void *)&skb_network_header(skb)[offset]; |
| 1334 | if (tel->encap_limit == 0) { | 1334 | if (tel->encap_limit == 0) { |
| 1335 | icmpv6_send(skb, ICMPV6_PARAMPROB, | 1335 | icmpv6_ndo_send(skb, ICMPV6_PARAMPROB, |
| 1336 | ICMPV6_HDR_FIELD, offset + 2); | 1336 | ICMPV6_HDR_FIELD, offset + 2); |
| 1337 | return -1; | 1337 | return -1; |
| 1338 | } | 1338 | } |
| 1339 | encap_limit = tel->encap_limit - 1; | 1339 | encap_limit = tel->encap_limit - 1; |
| @@ -1385,11 +1385,11 @@ ipxip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev, · 45 unmodified lines | |||
| 1385 | if (err == -EMSGSIZE) | 1385 | if (err == -EMSGSIZE) |
| 1386 | switch (protocol) { | 1386 | switch (protocol) { |
| 1387 | case IPPROTO_IPIP: | 1387 | case IPPROTO_IPIP: |
| 1388 | icmp_send(skb, ICMP_DEST_UNREACH, | 1388 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, |
| 1389 | ICMP_FRAG_NEEDED, htonl(mtu)); | 1389 | ICMP_FRAG_NEEDED, htonl(mtu)); |
| 1390 | break; | 1390 | break; |
| 1391 | case IPPROTO_IPV6: | 1391 | case IPPROTO_IPV6: |
| 1392 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 1392 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 1393 | break; | 1393 | break; |
| 1394 | default: | 1394 | default: |
| 1395 | break; | 1395 | break; |
| @@ -521,10 +521,10 @@ vti6_xmit(struct sk_buff *skb, struct net_device *dev, struct flowi *fl) · 520 unmodified lines | |||
| 521 | if (mtu < IPV6_MIN_MTU) | 521 | if (mtu < IPV6_MIN_MTU) |
| 522 | mtu = IPV6_MIN_MTU; | 522 | mtu = IPV6_MIN_MTU; |
| 523 | 523 | ||
| 524 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 524 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 525 | } else { | 525 | } else { |
| 526 | icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, | 526 | icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED, |
| 527 | htonl(mtu)); | 527 | htonl(mtu)); |
| 528 | } | 528 | } |
| 529 | 529 | ||
| 530 | err = -EMSGSIZE; | 530 | err = -EMSGSIZE; |
| @@ -987,7 +987,7 @@ static netdev_tx_t ipip6_tunnel_xmit(struct sk_buff *skb, · 986 unmodified lines | |||
| 987 | skb_dst_update_pmtu_no_confirm(skb, mtu); | 987 | skb_dst_update_pmtu_no_confirm(skb, mtu); |
| 988 | 988 | ||
| 989 | if (skb->len > mtu && !skb_is_gso(skb)) { | 989 | if (skb->len > mtu && !skb_is_gso(skb)) { |
| 990 | icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); | 990 | icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu); |
| 991 | ip_rt_put(rt); | 991 | ip_rt_put(rt); |
| 992 | goto tx_error; | 992 | goto tx_error; |
| 993 | } | 993 | } |
| @@ -649,9 +649,9 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb, · 648 unmodified lines | |||
| 649 | /* Parse and check optional cookie */ | 649 | /* Parse and check optional cookie */ |
| 650 | if (session->peer_cookie_len > 0) { | 650 | if (session->peer_cookie_len > 0) { |
| 651 | if (memcmp(ptr, &session->peer_cookie[0], session->peer_cookie_len)) { | 651 | if (memcmp(ptr, &session->peer_cookie[0], session->peer_cookie_len)) { |
| 652 | pr_warn_ratelimited("%s: cookie mismatch (%u/%u). Discarding.\n", | 652 | pr_debug_ratelimited("%s: cookie mismatch (%u/%u). Discarding.\n", |
| 653 | tunnel->name, tunnel->tunnel_id, | 653 | tunnel->name, tunnel->tunnel_id, |
| 654 | session->session_id); | 654 | session->session_id); |
| 655 | atomic_long_inc(&session->stats.rx_cookie_discards); | 655 | atomic_long_inc(&session->stats.rx_cookie_discards); |
| 656 | goto discard; | 656 | goto discard; |
| 657 | } | 657 | } |
| @@ -702,8 +702,8 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb, · 44 unmodified lines | |||
| 702 | * If user has configured mandatory sequence numbers, discard. | 702 | * If user has configured mandatory sequence numbers, discard. |
| 703 | */ | 703 | */ |
| 704 | if (session->recv_seq) { | 704 | if (session->recv_seq) { |
| 705 | pr_warn_ratelimited("%s: recv data has no seq numbers when required. Discarding.\n", | 705 | pr_debug_ratelimited("%s: recv data has no seq numbers when required. Discarding.\n", |
| 706 | session->name); | 706 | session->name); |
| 707 | atomic_long_inc(&session->stats.rx_seq_discards); | 707 | atomic_long_inc(&session->stats.rx_seq_discards); |
| 708 | goto discard; | 708 | goto discard; |
| 709 | } | 709 | } |
| @@ -718,8 +718,8 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb, · 8 unmodified lines | |||
| 718 | session->send_seq = 0; | 718 | session->send_seq = 0; |
| 719 | l2tp_session_set_header_len(session, tunnel->version); | 719 | l2tp_session_set_header_len(session, tunnel->version); |
| 720 | } else if (session->send_seq) { | 720 | } else if (session->send_seq) { |
| 721 | pr_warn_ratelimited("%s: recv data has no seq numbers when required. Discarding.\n", | 721 | pr_debug_ratelimited("%s: recv data has no seq numbers when required. Discarding.\n", |
| 722 | session->name); | 722 | session->name); |
| 723 | atomic_long_inc(&session->stats.rx_seq_discards); | 723 | atomic_long_inc(&session->stats.rx_seq_discards); |
| 724 | goto discard; | 724 | goto discard; |
| 725 | } | 725 | } |
| @@ -809,9 +809,9 @@ static int l2tp_udp_recv_core(struct l2tp_tunnel *tunnel, struct sk_buff *skb) · 83 unmodified lines | |||
| 809 | 809 | ||
| 810 | /* Short packet? */ | 810 | /* Short packet? */ |
| 811 | if (!pskb_may_pull(skb, L2TP_HDR_SIZE_MAX)) { | 811 | if (!pskb_may_pull(skb, L2TP_HDR_SIZE_MAX)) { |
| 812 | pr_warn_ratelimited("%s: recv short packet (len=%d)\n", | 812 | pr_debug_ratelimited("%s: recv short packet (len=%d)\n", |
| 813 | tunnel->name, skb->len); | 813 | tunnel->name, skb->len); |
| 814 | goto error; | 814 | goto invalid; |
| 815 | } | 815 | } |
| 816 | 816 | ||
| 817 | /* Point to L2TP header */ | 817 | /* Point to L2TP header */ |
| @@ -824,9 +824,9 @@ static int l2tp_udp_recv_core(struct l2tp_tunnel *tunnel, struct sk_buff *skb) · 6 unmodified lines | |||
| 824 | /* Check protocol version */ | 824 | /* Check protocol version */ |
| 825 | version = hdrflags & L2TP_HDR_VER_MASK; | 825 | version = hdrflags & L2TP_HDR_VER_MASK; |
| 826 | if (version != tunnel->version) { | 826 | if (version != tunnel->version) { |
| 827 | pr_warn_ratelimited("%s: recv protocol version mismatch: got %d expected %d\n", | 827 | pr_debug_ratelimited("%s: recv protocol version mismatch: got %d expected %d\n", |
| 828 | tunnel->name, version, tunnel->version); | 828 | tunnel->name, version, tunnel->version); |
| 829 | goto error; | 829 | goto invalid; |
| 830 | } | 830 | } |
| 831 | 831 | ||
| 832 | /* Get length of L2TP packet */ | 832 | /* Get length of L2TP packet */ |
| @@ -834,7 +834,7 @@ static int l2tp_udp_recv_core(struct l2tp_tunnel *tunnel, struct sk_buff *skb) · 1 unmodified line | |||
| 834 | 834 | ||
| 835 | /* If type is control packet, it is handled by userspace. */ | 835 | /* If type is control packet, it is handled by userspace. */ |
| 836 | if (hdrflags & L2TP_HDRFLAG_T) | 836 | if (hdrflags & L2TP_HDRFLAG_T) |
| 837 | goto error; | 837 | goto pass; |
| 838 | 838 | ||
| 839 | /* Skip flags */ | 839 | /* Skip flags */ |
| 840 | ptr += 2; | 840 | ptr += 2; |
| @@ -863,21 +863,24 @@ static int l2tp_udp_recv_core(struct l2tp_tunnel *tunnel, struct sk_buff *skb) · 22 unmodified lines | |||
| 863 | l2tp_session_dec_refcount(session); | 863 | l2tp_session_dec_refcount(session); |
| 864 | 864 | ||
| 865 | /* Not found? Pass to userspace to deal with */ | 865 | /* Not found? Pass to userspace to deal with */ |
| 866 | pr_warn_ratelimited("%s: no session found (%u/%u). Passing up.\n", | 866 | pr_debug_ratelimited("%s: no session found (%u/%u). Passing up.\n", |
| 867 | tunnel->name, tunnel_id, session_id); | 867 | tunnel->name, tunnel_id, session_id); |
| 868 | goto error; | 868 | goto pass; |
| 869 | } | 869 | } |
| 870 | 870 | ||
| 871 | if (tunnel->version == L2TP_HDR_VER_3 && | 871 | if (tunnel->version == L2TP_HDR_VER_3 && |
| 872 | l2tp_v3_ensure_opt_in_linear(session, skb, &ptr, &optr)) | 872 | l2tp_v3_ensure_opt_in_linear(session, skb, &ptr, &optr)) |
| 873 | goto error; | 873 | goto invalid; |
| 874 | 874 | ||
| 875 | l2tp_recv_common(session, skb, ptr, optr, hdrflags, length); | 875 | l2tp_recv_common(session, skb, ptr, optr, hdrflags, length); |
| 876 | l2tp_session_dec_refcount(session); | 876 | l2tp_session_dec_refcount(session); |
| 877 | 877 | ||
| 878 | return 0; | 878 | return 0; |
| 879 | 879 | ||
| 880 | error: | 880 | invalid: |
| 881 | atomic_long_inc(&tunnel->stats.rx_invalid); | ||
| 882 | |||
| 883 | pass: | ||
| 881 | /* Put UDP header back */ | 884 | /* Put UDP header back */ |
| 882 | __skb_push(skb, sizeof(struct udphdr)); | 885 | __skb_push(skb, sizeof(struct udphdr)); |
| 883 | 886 | ||
| @@ -39,6 +39,7 @@ struct l2tp_stats { · 38 unmodified lines | |||
| 39 | atomic_long_t rx_oos_packets; | 39 | atomic_long_t rx_oos_packets; |
| 40 | atomic_long_t rx_errors; | 40 | atomic_long_t rx_errors; |
| 41 | atomic_long_t rx_cookie_discards; | 41 | atomic_long_t rx_cookie_discards; |
| 42 | atomic_long_t rx_invalid; | ||
| 42 | }; | 43 | }; |
| 43 | 44 | ||
| 44 | struct l2tp_tunnel; | 45 | struct l2tp_tunnel; |
| @@ -428,6 +428,9 @@ static int l2tp_nl_tunnel_send(struct sk_buff *skb, u32 portid, u32 seq, int fla · 427 unmodified lines | |||
| 428 | L2TP_ATTR_STATS_PAD) || | 428 | L2TP_ATTR_STATS_PAD) || |
| 429 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_ERRORS, | 429 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_ERRORS, |
| 430 | atomic_long_read(&tunnel->stats.rx_errors), | 430 | atomic_long_read(&tunnel->stats.rx_errors), |
| 431 | L2TP_ATTR_STATS_PAD) || | ||
| 432 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_INVALID, | ||
| 433 | atomic_long_read(&tunnel->stats.rx_invalid), | ||
| 431 | L2TP_ATTR_STATS_PAD)) | 434 | L2TP_ATTR_STATS_PAD)) |
| 432 | goto nla_put_failure; | 435 | goto nla_put_failure; |
| 433 | nla_nest_end(skb, nest); | 436 | nla_nest_end(skb, nest); |
| @@ -771,6 +774,9 @@ static int l2tp_nl_session_send(struct sk_buff *skb, u32 portid, u32 seq, int fl · 337 unmodified lines | |||
| 771 | L2TP_ATTR_STATS_PAD) || | 774 | L2TP_ATTR_STATS_PAD) || |
| 772 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_ERRORS, | 775 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_ERRORS, |
| 773 | atomic_long_read(&session->stats.rx_errors), | 776 | atomic_long_read(&session->stats.rx_errors), |
| 777 | L2TP_ATTR_STATS_PAD) || | ||
| 778 | nla_put_u64_64bit(skb, L2TP_ATTR_RX_INVALID, | ||
| 779 | atomic_long_read(&session->stats.rx_invalid), | ||
| 774 | L2TP_ATTR_STATS_PAD)) | 780 | L2TP_ATTR_STATS_PAD)) |
| 775 | goto nla_put_failure; | 781 | goto nla_put_failure; |
| 776 | nla_nest_end(skb, nest); | 782 | nla_nest_end(skb, nest); |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | |||
| 14 | #include <linux/netdev_features.h> | 14 | #include <linux/netdev_features.h> |
| 15 | #include <linux/netdevice.h> | 15 | #include <linux/netdevice.h> |
| 16 | #include <linux/skbuff.h> | 16 | #include <linux/skbuff.h> |
| 17 | #include <net/mpls.h> | ||
| 17 | 18 | ||
| 18 | static struct sk_buff *mpls_gso_segment(struct sk_buff *skb, | 19 | static struct sk_buff *mpls_gso_segment(struct sk_buff *skb, |
| 19 | netdev_features_t features) | 20 | netdev_features_t features) |
| @@ -27,6 +28,8 @@ static struct sk_buff *mpls_gso_segment(struct sk_buff *skb, · 7 unmodified lines | |||
| 27 | 28 | ||
| 28 | skb_reset_network_header(skb); | 29 | skb_reset_network_header(skb); |
| 29 | mpls_hlen = skb_inner_network_header(skb) - skb_network_header(skb); | 30 | mpls_hlen = skb_inner_network_header(skb) - skb_network_header(skb); |
| 31 | if (unlikely(!mpls_hlen || mpls_hlen % MPLS_HLEN)) | ||
| 32 | goto out; | ||
| 30 | if (unlikely(!pskb_may_pull(skb, mpls_hlen))) | 33 | if (unlikely(!pskb_may_pull(skb, mpls_hlen))) |
| 31 | goto out; | 34 | goto out; |
| 32 | 35 | ||
| @@ -1061,6 +1061,12 @@ out: · 1060 unmodified lines | |||
| 1061 | } | 1061 | } |
| 1062 | } | 1062 | } |
| 1063 | 1063 | ||
| 1064 | static void __mptcp_clean_una_wakeup(struct sock *sk) | ||
| 1065 | { | ||
| 1066 | __mptcp_clean_una(sk); | ||
| 1067 | mptcp_write_space(sk); | ||
| 1068 | } | ||
| 1069 | |||
| 1064 | static void mptcp_enter_memory_pressure(struct sock *sk) | 1070 | static void mptcp_enter_memory_pressure(struct sock *sk) |
| 1065 | { | 1071 | { |
| 1066 | struct mptcp_subflow_context *subflow; | 1072 | struct mptcp_subflow_context *subflow; |
| @@ -1189,6 +1195,7 @@ static bool mptcp_tx_cache_refill(struct sock *sk, int size, · 122 unmodified lines | |||
| 1189 | */ | 1195 | */ |
| 1190 | while (skbs->qlen > 1) { | 1196 | while (skbs->qlen > 1) { |
| 1191 | skb = __skb_dequeue_tail(skbs); | 1197 | skb = __skb_dequeue_tail(skbs); |
| 1198 | *total_ts -= skb->truesize; | ||
| 1192 | __kfree_skb(skb); | 1199 | __kfree_skb(skb); |
| 1193 | } | 1200 | } |
| 1194 | return skbs->qlen > 0; | 1201 | return skbs->qlen > 0; |
| @@ -1444,7 +1451,7 @@ static void mptcp_push_release(struct sock *sk, struct sock *ssk, · 249 unmodified lines | |||
| 1444 | release_sock(ssk); | 1451 | release_sock(ssk); |
| 1445 | } | 1452 | } |
| 1446 | 1453 | ||
| 1447 | static void mptcp_push_pending(struct sock *sk, unsigned int flags) | 1454 | static void __mptcp_push_pending(struct sock *sk, unsigned int flags) |
| 1448 | { | 1455 | { |
| 1449 | struct sock *prev_ssk = NULL, *ssk = NULL; | 1456 | struct sock *prev_ssk = NULL, *ssk = NULL; |
| 1450 | struct mptcp_sock *msk = mptcp_sk(sk); | 1457 | struct mptcp_sock *msk = mptcp_sk(sk); |
| @@ -1696,14 +1703,14 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) · 245 unmodified lines | |||
| 1696 | 1703 | ||
| 1697 | wait_for_memory: | 1704 | wait_for_memory: |
| 1698 | mptcp_set_nospace(sk); | 1705 | mptcp_set_nospace(sk); |
| 1699 | mptcp_push_pending(sk, msg->msg_flags); | 1706 | __mptcp_push_pending(sk, msg->msg_flags); |
| 1700 | ret = sk_stream_wait_memory(sk, &timeo); | 1707 | ret = sk_stream_wait_memory(sk, &timeo); |
| 1701 | if (ret) | 1708 | if (ret) |
| 1702 | goto out; | 1709 | goto out; |
| 1703 | } | 1710 | } |
| 1704 | 1711 | ||
| 1705 | if (copied) | 1712 | if (copied) |
| 1706 | mptcp_push_pending(sk, msg->msg_flags); | 1713 | __mptcp_push_pending(sk, msg->msg_flags); |
| 1707 | 1714 | ||
| 1708 | out: | 1715 | out: |
| 1709 | release_sock(sk); | 1716 | release_sock(sk); |
| @@ -2115,6 +2122,14 @@ static struct sock *mptcp_subflow_get_retrans(const struct mptcp_sock *msk) · 405 unmodified lines | |||
| 2115 | return backup; | 2122 | return backup; |
| 2116 | } | 2123 | } |
| 2117 | 2124 | ||
| 2125 | static void mptcp_dispose_initial_subflow(struct mptcp_sock *msk) | ||
| 2126 | { | ||
| 2127 | if (msk->subflow) { | ||
| 2128 | iput(SOCK_INODE(msk->subflow)); | ||
| 2129 | msk->subflow = NULL; | ||
| 2130 | } | ||
| 2131 | } | ||
| 2132 | |||
| 2118 | /* subflow sockets can be either outgoing (connect) or incoming | 2133 | /* subflow sockets can be either outgoing (connect) or incoming |
| 2119 | * (accept). | 2134 | * (accept). |
| 2120 | * | 2135 | * |
| @@ -2126,6 +2141,8 @@ static struct sock *mptcp_subflow_get_retrans(const struct mptcp_sock *msk) · 5 unmodified lines | |||
| 2126 | static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, | 2141 | static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, |
| 2127 | struct mptcp_subflow_context *subflow) | 2142 | struct mptcp_subflow_context *subflow) |
| 2128 | { | 2143 | { |
| 2144 | struct mptcp_sock *msk = mptcp_sk(sk); | ||
| 2145 | |||
| 2129 | list_del(&subflow->node); | 2146 | list_del(&subflow->node); |
| 2130 | 2147 | ||
| 2131 | lock_sock_nested(ssk, SINGLE_DEPTH_NESTING); | 2148 | lock_sock_nested(ssk, SINGLE_DEPTH_NESTING); |
| @@ -2154,6 +2171,18 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, · 22 unmodified lines | |||
| 2154 | release_sock(ssk); | 2171 | release_sock(ssk); |
| 2155 | 2172 | ||
| 2156 | sock_put(ssk); | 2173 | sock_put(ssk); |
| 2174 | |||
| 2175 | if (ssk == msk->last_snd) | ||
| 2176 | msk->last_snd = NULL; | ||
| 2177 | |||
| 2178 | if (ssk == msk->ack_hint) | ||
| 2179 | msk->ack_hint = NULL; | ||
| 2180 | |||
| 2181 | if (ssk == msk->first) | ||
| 2182 | msk->first = NULL; | ||
| 2183 | |||
| 2184 | if (msk->subflow && ssk == msk->subflow->sk) | ||
| 2185 | mptcp_dispose_initial_subflow(msk); | ||
| 2157 | } | 2186 | } |
| 2158 | 2187 | ||
| 2159 | void mptcp_close_ssk(struct sock *sk, struct sock *ssk, | 2188 | void mptcp_close_ssk(struct sock *sk, struct sock *ssk, |
| @@ -2238,14 +2267,58 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk) · 78 unmodified lines | |||
| 2238 | mptcp_close_wake_up(sk); | 2267 | mptcp_close_wake_up(sk); |
| 2239 | } | 2268 | } |
| 2240 | 2269 | ||
| 2241 | static void mptcp_worker(struct work_struct *work) | 2270 | static void __mptcp_retrans(struct sock *sk) |
| 2242 | { | 2271 | { |
| 2243 | struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work); | 2272 | struct mptcp_sock *msk = mptcp_sk(sk); |
| 2244 | struct sock *ssk, *sk = &msk->sk.icsk_inet.sk; | ||
| 2245 | struct mptcp_sendmsg_info info = {}; | 2273 | struct mptcp_sendmsg_info info = {}; |
| 2246 | struct mptcp_data_frag *dfrag; | 2274 | struct mptcp_data_frag *dfrag; |
| 2247 | size_t copied = 0; | 2275 | size_t copied = 0; |
| 2248 | int state, ret; | 2276 | struct sock *ssk; |
| 2277 | int ret; | ||
| 2278 | |||
| 2279 | __mptcp_clean_una_wakeup(sk); | ||
| 2280 | dfrag = mptcp_rtx_head(sk); | ||
| 2281 | if (!dfrag) | ||
| 2282 | return; | ||
| 2283 | |||
| 2284 | ssk = mptcp_subflow_get_retrans(msk); | ||
| 2285 | if (!ssk) | ||
| 2286 | goto reset_timer; | ||
| 2287 | |||
| 2288 | lock_sock(ssk); | ||
| 2289 | |||
| 2290 | /* limit retransmission to the bytes already sent on some subflows */ | ||
| 2291 | info.sent = 0; | ||
| 2292 | info.limit = dfrag->already_sent; | ||
| 2293 | while (info.sent < dfrag->already_sent) { | ||
| 2294 | if (!mptcp_alloc_tx_skb(sk, ssk)) | ||
| 2295 | break; | ||
| 2296 | |||
| 2297 | ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info); | ||
| 2298 | if (ret <= 0) | ||
| 2299 | break; | ||
| 2300 | |||
| 2301 | MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RETRANSSEGS); | ||
| 2302 | copied += ret; | ||
| 2303 | info.sent += ret; | ||
| 2304 | } | ||
| 2305 | if (copied) | ||
| 2306 | tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle, | ||
| 2307 | info.size_goal); | ||
| 2308 | |||
| 2309 | mptcp_set_timeout(sk, ssk); | ||
| 2310 | release_sock(ssk); | ||
| 2311 | |||
| 2312 | reset_timer: | ||
| 2313 | if (!mptcp_timer_pending(sk)) | ||
| 2314 | mptcp_reset_timer(sk); | ||
| 2315 | } | ||
| 2316 | |||
| 2317 | static void mptcp_worker(struct work_struct *work) | ||
| 2318 | { | ||
| 2319 | struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work); | ||
| 2320 | struct sock *sk = &msk->sk.icsk_inet.sk; | ||
| 2321 | int state; | ||
| 2249 | 2322 | ||
| 2250 | lock_sock(sk); | 2323 | lock_sock(sk); |
| 2251 | state = sk->sk_state; | 2324 | state = sk->sk_state; |
| @@ -2280,45 +2353,8 @@ static void mptcp_worker(struct work_struct *work) · 28 unmodified lines | |||
| 2280 | if (test_and_clear_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags)) | 2353 | if (test_and_clear_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags)) |
| 2281 | __mptcp_close_subflow(msk); | 2354 | __mptcp_close_subflow(msk); |
| 2282 | 2355 | ||
| 2283 | if (!test_and_clear_bit(MPTCP_WORK_RTX, &msk->flags)) | 2356 | if (test_and_clear_bit(MPTCP_WORK_RTX, &msk->flags)) |
| 2284 | goto unlock; | 2357 | __mptcp_retrans(sk); |
| 2285 | |||
| 2286 | __mptcp_clean_una(sk); | ||
| 2287 | dfrag = mptcp_rtx_head(sk); | ||
| 2288 | if (!dfrag) | ||
| 2289 | goto unlock; | ||
| 2290 | |||
| 2291 | ssk = mptcp_subflow_get_retrans(msk); | ||
| 2292 | if (!ssk) | ||
| 2293 | goto reset_unlock; | ||
| 2294 | |||
| 2295 | lock_sock(ssk); | ||
| 2296 | |||
| 2297 | /* limit retransmission to the bytes already sent on some subflows */ | ||
| 2298 | info.sent = 0; | ||
| 2299 | info.limit = dfrag->already_sent; | ||
| 2300 | while (info.sent < dfrag->already_sent) { | ||
| 2301 | if (!mptcp_alloc_tx_skb(sk, ssk)) | ||
| 2302 | break; | ||
| 2303 | |||
| 2304 | ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info); | ||
| 2305 | if (ret <= 0) | ||
| 2306 | break; | ||
| 2307 | |||
| 2308 | MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RETRANSSEGS); | ||
| 2309 | copied += ret; | ||
| 2310 | info.sent += ret; | ||
| 2311 | } | ||
| 2312 | if (copied) | ||
| 2313 | tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle, | ||
| 2314 | info.size_goal); | ||
| 2315 | |||
| 2316 | mptcp_set_timeout(sk, ssk); | ||
| 2317 | release_sock(ssk); | ||
| 2318 | |||
| 2319 | reset_unlock: | ||
| 2320 | if (!mptcp_timer_pending(sk)) | ||
| 2321 | mptcp_reset_timer(sk); | ||
| 2322 | 2358 | ||
| 2323 | unlock: | 2359 | unlock: |
| 2324 | release_sock(sk); | 2360 | release_sock(sk); |
| @@ -2523,12 +2559,6 @@ static void __mptcp_destroy_sock(struct sock *sk) · 198 unmodified lines | |||
| 2523 | 2559 | ||
| 2524 | might_sleep(); | 2560 | might_sleep(); |
| 2525 | 2561 | ||
| 2526 | /* dispose the ancillatory tcp socket, if any */ | ||
| 2527 | if (msk->subflow) { | ||
| 2528 | iput(SOCK_INODE(msk->subflow)); | ||
| 2529 | msk->subflow = NULL; | ||
| 2530 | } | ||
| 2531 | |||
| 2532 | /* be sure to always acquire the join list lock, to sync vs | 2562 | /* be sure to always acquire the join list lock, to sync vs |
| 2533 | * mptcp_finish_join(). | 2563 | * mptcp_finish_join(). |
| 2534 | */ | 2564 | */ |
| @@ -2553,6 +2583,7 @@ static void __mptcp_destroy_sock(struct sock *sk) · 18 unmodified lines | |||
| 2553 | sk_stream_kill_queues(sk); | 2583 | sk_stream_kill_queues(sk); |
| 2554 | xfrm_sk_free_policy(sk); | 2584 | xfrm_sk_free_policy(sk); |
| 2555 | sk_refcnt_debug_release(sk); | 2585 | sk_refcnt_debug_release(sk); |
| 2586 | mptcp_dispose_initial_subflow(msk); | ||
| 2556 | sock_put(sk); | 2587 | sock_put(sk); |
| 2557 | } | 2588 | } |
| 2558 | 2589 | ||
| @@ -2934,13 +2965,14 @@ static void mptcp_release_cb(struct sock *sk) · 375 unmodified lines | |||
| 2934 | { | 2965 | { |
| 2935 | unsigned long flags, nflags; | 2966 | unsigned long flags, nflags; |
| 2936 | 2967 | ||
| 2937 | /* push_pending may touch wmem_reserved, do it before the later | 2968 | for (;;) { |
| 2938 | * cleanup | 2969 | flags = 0; |
| 2939 | */ | 2970 | if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags)) |
| 2940 | if (test_and_clear_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags)) | 2971 | flags |= MPTCP_PUSH_PENDING; |
| 2941 | __mptcp_clean_una(sk); | 2972 | if (!flags) |
| 2942 | if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags)) { | 2973 | break; |
| 2943 | /* mptcp_push_pending() acquires the subflow socket lock | 2974 | |
| 2975 | /* the following actions acquire the subflow socket lock | ||
| 2944 | * | 2976 | * |
| 2945 | * 1) can't be invoked in atomic scope | 2977 | * 1) can't be invoked in atomic scope |
| 2946 | * 2) must avoid ABBA deadlock with msk socket spinlock: the RX | 2978 | * 2) must avoid ABBA deadlock with msk socket spinlock: the RX |
| @@ -2949,13 +2981,21 @@ static void mptcp_release_cb(struct sock *sk) · 2 unmodified lines | |||
| 2949 | */ | 2981 | */ |
| 2950 | 2982 | ||
| 2951 | spin_unlock_bh(&sk->sk_lock.slock); | 2983 | spin_unlock_bh(&sk->sk_lock.slock); |
| 2952 | mptcp_push_pending(sk, 0); | 2984 | if (flags & MPTCP_PUSH_PENDING) |
| 2985 | __mptcp_push_pending(sk, 0); | ||
| 2986 | |||
| 2987 | cond_resched(); | ||
| 2953 | spin_lock_bh(&sk->sk_lock.slock); | 2988 | spin_lock_bh(&sk->sk_lock.slock); |
| 2954 | } | 2989 | } |
| 2990 | |||
| 2991 | if (test_and_clear_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags)) | ||
| 2992 | __mptcp_clean_una_wakeup(sk); | ||
| 2955 | if (test_and_clear_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags)) | 2993 | if (test_and_clear_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags)) |
| 2956 | __mptcp_error_report(sk); | 2994 | __mptcp_error_report(sk); |
| 2957 | 2995 | ||
| 2958 | /* clear any wmem reservation and errors */ | 2996 | /* push_pending may touch wmem_reserved, ensure we do the cleanup |
| 2997 | * later | ||
| 2998 | */ | ||
| 2959 | __mptcp_update_wmem(sk); | 2999 | __mptcp_update_wmem(sk); |
| 2960 | __mptcp_update_rmem(sk); | 3000 | __mptcp_update_rmem(sk); |
| 2961 | 3001 | ||
| @@ -3285,6 +3325,9 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock, · 323 unmodified lines | |||
| 3285 | /* PM/worker can now acquire the first subflow socket | 3325 | /* PM/worker can now acquire the first subflow socket |
| 3286 | * lock without racing with listener queue cleanup, | 3326 | * lock without racing with listener queue cleanup, |
| 3287 | * we can notify it, if needed. | 3327 | * we can notify it, if needed. |
| 3328 | * | ||
| 3329 | * Even if remote has reset the initial subflow by now | ||
| 3330 | * the refcnt is still at least one. | ||
| 3288 | */ | 3331 | */ |
| 3289 | subflow = mptcp_subflow_ctx(msk->first); | 3332 | subflow = mptcp_subflow_ctx(msk->first); |
| 3290 | list_add(&subflow->node, &msk->conn_list); | 3333 | list_add(&subflow->node, &msk->conn_list); |
| @@ -52,14 +52,15 @@ · 51 unmodified lines | |||
| 52 | #define TCPOLEN_MPTCP_DSS_MAP64 14 | 52 | #define TCPOLEN_MPTCP_DSS_MAP64 14 |
| 53 | #define TCPOLEN_MPTCP_DSS_CHECKSUM 2 | 53 | #define TCPOLEN_MPTCP_DSS_CHECKSUM 2 |
| 54 | #define TCPOLEN_MPTCP_ADD_ADDR 16 | 54 | #define TCPOLEN_MPTCP_ADD_ADDR 16 |
| 55 | #define TCPOLEN_MPTCP_ADD_ADDR_PORT 20 | 55 | #define TCPOLEN_MPTCP_ADD_ADDR_PORT 18 |
| 56 | #define TCPOLEN_MPTCP_ADD_ADDR_BASE 8 | 56 | #define TCPOLEN_MPTCP_ADD_ADDR_BASE 8 |
| 57 | #define TCPOLEN_MPTCP_ADD_ADDR_BASE_PORT 12 | 57 | #define TCPOLEN_MPTCP_ADD_ADDR_BASE_PORT 10 |
| 58 | #define TCPOLEN_MPTCP_ADD_ADDR6 28 | 58 | #define TCPOLEN_MPTCP_ADD_ADDR6 28 |
| 59 | #define TCPOLEN_MPTCP_ADD_ADDR6_PORT 32 | 59 | #define TCPOLEN_MPTCP_ADD_ADDR6_PORT 30 |
| 60 | #define TCPOLEN_MPTCP_ADD_ADDR6_BASE 20 | 60 | #define TCPOLEN_MPTCP_ADD_ADDR6_BASE 20 |
| 61 | #define TCPOLEN_MPTCP_ADD_ADDR6_BASE_PORT 24 | 61 | #define TCPOLEN_MPTCP_ADD_ADDR6_BASE_PORT 22 |
| 62 | #define TCPOLEN_MPTCP_PORT_LEN 4 | 62 | #define TCPOLEN_MPTCP_PORT_LEN 2 |
| 63 | #define TCPOLEN_MPTCP_PORT_ALIGN 2 | ||
| 63 | #define TCPOLEN_MPTCP_RM_ADDR_BASE 4 | 64 | #define TCPOLEN_MPTCP_RM_ADDR_BASE 4 |
| 64 | #define TCPOLEN_MPTCP_PRIO 3 | 65 | #define TCPOLEN_MPTCP_PRIO 3 |
| 65 | #define TCPOLEN_MPTCP_PRIO_ALIGN 4 | 66 | #define TCPOLEN_MPTCP_PRIO_ALIGN 4 |
| @@ -701,8 +702,9 @@ static inline unsigned int mptcp_add_addr_len(int family, bool echo, bool port) · 635 unmodified lines | |||
| 701 | len = TCPOLEN_MPTCP_ADD_ADDR6_BASE; | 702 | len = TCPOLEN_MPTCP_ADD_ADDR6_BASE; |
| 702 | if (!echo) | 703 | if (!echo) |
| 703 | len += MPTCPOPT_THMAC_LEN; | 704 | len += MPTCPOPT_THMAC_LEN; |
| 705 | /* account for 2 trailing 'nop' options */ | ||
| 704 | if (port) | 706 | if (port) |
| 705 | len += TCPOLEN_MPTCP_PORT_LEN; | 707 | len += TCPOLEN_MPTCP_PORT_LEN + TCPOLEN_MPTCP_PORT_ALIGN; |
| 706 | 708 | ||
| 707 | return len; | 709 | return len; |
| 708 | } | 710 | } |
| @@ -687,11 +687,6 @@ create_child: · 686 unmodified lines | |||
| 687 | /* move the msk reference ownership to the subflow */ | 687 | /* move the msk reference ownership to the subflow */ |
| 688 | subflow_req->msk = NULL; | 688 | subflow_req->msk = NULL; |
| 689 | ctx->conn = (struct sock *)owner; | 689 | ctx->conn = (struct sock *)owner; |
| 690 | if (!mptcp_finish_join(child)) | ||
| 691 | goto dispose_child; | ||
| 692 | |||
| 693 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX); | ||
| 694 | tcp_rsk(req)->drop_req = true; | ||
| 695 | 690 | ||
| 696 | if (subflow_use_different_sport(owner, sk)) { | 691 | if (subflow_use_different_sport(owner, sk)) { |
| 697 | pr_debug("ack inet_sport=%d %d", | 692 | pr_debug("ack inet_sport=%d %d", |
| @@ -699,10 +694,16 @@ create_child: · 1 unmodified line | |||
| 699 | ntohs(inet_sk((struct sock *)owner)->inet_sport)); | 694 | ntohs(inet_sk((struct sock *)owner)->inet_sport)); |
| 700 | if (!mptcp_pm_sport_in_anno_list(owner, sk)) { | 695 | if (!mptcp_pm_sport_in_anno_list(owner, sk)) { |
| 701 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_MISMATCHPORTACKRX); | 696 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_MISMATCHPORTACKRX); |
| 702 | goto out; | 697 | goto dispose_child; |
| 703 | } | 698 | } |
| 704 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINPORTACKRX); | 699 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINPORTACKRX); |
| 705 | } | 700 | } |
| 701 | |||
| 702 | if (!mptcp_finish_join(child)) | ||
| 703 | goto dispose_child; | ||
| 704 | |||
| 705 | SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX); | ||
| 706 | tcp_rsk(req)->drop_req = true; | ||
| 706 | } | 707 | } |
| 707 | } | 708 | } |
| 708 | 709 | ||
| @@ -1297,6 +1298,7 @@ failed_unlink: · 588 unmodified lines | |||
| 1297 | spin_lock_bh(&msk->join_list_lock); | 1298 | spin_lock_bh(&msk->join_list_lock); |
| 1298 | list_del(&subflow->node); | 1299 | list_del(&subflow->node); |
| 1299 | spin_unlock_bh(&msk->join_list_lock); | 1300 | spin_unlock_bh(&msk->join_list_lock); |
| 1301 | sock_put(mptcp_subflow_tcp_sock(subflow)); | ||
| 1300 | 1302 | ||
| 1301 | failed: | 1303 | failed: |
| 1302 | subflow->disposable = 1; | 1304 | subflow->disposable = 1; |
| @@ -219,7 +219,7 @@ nf_ct_lookup_helper(struct nf_conn *ct, struct net *net) · 218 unmodified lines | |||
| 219 | return NULL; | 219 | return NULL; |
| 220 | pr_info("nf_conntrack: default automatic helper assignment " | 220 | pr_info("nf_conntrack: default automatic helper assignment " |
| 221 | "has been turned off for security reasons and CT-based " | 221 | "has been turned off for security reasons and CT-based " |
| 222 | " firewall rule not found. Use the iptables CT target " | 222 | "firewall rule not found. Use the iptables CT target " |
| 223 | "to attach helpers instead.\n"); | 223 | "to attach helpers instead.\n"); |
| 224 | net->ct.auto_assign_helper_warned = 1; | 224 | net->ct.auto_assign_helper_warned = 1; |
| 225 | return NULL; | 225 | return NULL; |
| @@ -228,7 +228,6 @@ nf_ct_lookup_helper(struct nf_conn *ct, struct net *net) · 2 unmodified lines | |||
| 228 | return __nf_ct_helper_find(&ct->tuplehash[IP_CT_DIR_REPLY].tuple); | 228 | return __nf_ct_helper_find(&ct->tuplehash[IP_CT_DIR_REPLY].tuple); |
| 229 | } | 229 | } |
| 230 | 230 | ||
| 231 | |||
| 232 | int __nf_ct_try_assign_helper(struct nf_conn *ct, struct nf_conn *tmpl, | 231 | int __nf_ct_try_assign_helper(struct nf_conn *ct, struct nf_conn *tmpl, |
| 233 | gfp_t flags) | 232 | gfp_t flags) |
| 234 | { | 233 | { |
| @@ -982,8 +982,10 @@ int nf_conntrack_tcp_packet(struct nf_conn *ct, · 981 unmodified lines | |||
| 982 | IP_CT_EXP_CHALLENGE_ACK; | 982 | IP_CT_EXP_CHALLENGE_ACK; |
| 983 | } | 983 | } |
| 984 | spin_unlock_bh(&ct->lock); | 984 | spin_unlock_bh(&ct->lock); |
| 985 | nf_ct_l4proto_log_invalid(skb, ct, "invalid packet ignored in " | 985 | nf_ct_l4proto_log_invalid(skb, ct, |
| 986 | "state %s ", tcp_conntrack_names[old_state]); | 986 | "packet (index %d) in dir %d ignored, state %s", |
| 987 | index, dir, | ||
| 988 | tcp_conntrack_names[old_state]); | ||
| 987 | return NF_ACCEPT; | 989 | return NF_ACCEPT; |
| 988 | case TCP_CONNTRACK_MAX: | 990 | case TCP_CONNTRACK_MAX: |
| 989 | /* Special case for SYN proxy: when the SYN to the server or | 991 | /* Special case for SYN proxy: when the SYN to the server or |
| @@ -646,8 +646,8 @@ nf_nat_ipv4_fn(void *priv, struct sk_buff *skb, · 645 unmodified lines | |||
| 646 | } | 646 | } |
| 647 | 647 | ||
| 648 | static unsigned int | 648 | static unsigned int |
| 649 | nf_nat_ipv4_in(void *priv, struct sk_buff *skb, | 649 | nf_nat_ipv4_pre_routing(void *priv, struct sk_buff *skb, |
| 650 | const struct nf_hook_state *state) | 650 | const struct nf_hook_state *state) |
| 651 | { | 651 | { |
| 652 | unsigned int ret; | 652 | unsigned int ret; |
| 653 | __be32 daddr = ip_hdr(skb)->daddr; | 653 | __be32 daddr = ip_hdr(skb)->daddr; |
| @@ -659,6 +659,23 @@ nf_nat_ipv4_in(void *priv, struct sk_buff *skb, · 5 unmodified lines | |||
| 659 | return ret; | 659 | return ret; |
| 660 | } | 660 | } |
| 661 | 661 | ||
| 662 | static unsigned int | ||
| 663 | nf_nat_ipv4_local_in(void *priv, struct sk_buff *skb, | ||
| 664 | const struct nf_hook_state *state) | ||
| 665 | { | ||
| 666 | __be32 saddr = ip_hdr(skb)->saddr; | ||
| 667 | struct sock *sk = skb->sk; | ||
| 668 | unsigned int ret; | ||
| 669 | |||
| 670 | ret = nf_nat_ipv4_fn(priv, skb, state); | ||
| 671 | |||
| 672 | if (ret == NF_ACCEPT && sk && saddr != ip_hdr(skb)->saddr && | ||
| 673 | !inet_sk_transparent(sk)) | ||
| 674 | skb_orphan(skb); /* TCP edemux obtained wrong socket */ | ||
| 675 | |||
| 676 | return ret; | ||
| 677 | } | ||
| 678 | |||
| 662 | static unsigned int | 679 | static unsigned int |
| 663 | nf_nat_ipv4_out(void *priv, struct sk_buff *skb, | 680 | nf_nat_ipv4_out(void *priv, struct sk_buff *skb, |
| 664 | const struct nf_hook_state *state) | 681 | const struct nf_hook_state *state) |
| @@ -736,7 +753,7 @@ nf_nat_ipv4_local_fn(void *priv, struct sk_buff *skb, · 71 unmodified lines | |||
| 736 | static const struct nf_hook_ops nf_nat_ipv4_ops[] = { | 753 | static const struct nf_hook_ops nf_nat_ipv4_ops[] = { |
| 737 | /* Before packet filtering, change destination */ | 754 | /* Before packet filtering, change destination */ |
| 738 | { | 755 | { |
| 739 | .hook = nf_nat_ipv4_in, | 756 | .hook = nf_nat_ipv4_pre_routing, |
| 740 | .pf = NFPROTO_IPV4, | 757 | .pf = NFPROTO_IPV4, |
| 741 | .hooknum = NF_INET_PRE_ROUTING, | 758 | .hooknum = NF_INET_PRE_ROUTING, |
| 742 | .priority = NF_IP_PRI_NAT_DST, | 759 | .priority = NF_IP_PRI_NAT_DST, |
| @@ -757,7 +774,7 @@ static const struct nf_hook_ops nf_nat_ipv4_ops[] = { · 14 unmodified lines | |||
| 757 | }, | 774 | }, |
| 758 | /* After packet filtering, change source */ | 775 | /* After packet filtering, change source */ |
| 759 | { | 776 | { |
| 760 | .hook = nf_nat_ipv4_fn, | 777 | .hook = nf_nat_ipv4_local_in, |
| 761 | .pf = NFPROTO_IPV4, | 778 | .pf = NFPROTO_IPV4, |
| 762 | .hooknum = NF_INET_LOCAL_IN, | 779 | .hooknum = NF_INET_LOCAL_IN, |
| 763 | .priority = NF_IP_PRI_NAT_SRC, | 780 | .priority = NF_IP_PRI_NAT_SRC, |
| @@ -916,6 +916,12 @@ static int nf_tables_updtable(struct nft_ctx *ctx) · 915 unmodified lines | |||
| 916 | if (flags == ctx->table->flags) | 916 | if (flags == ctx->table->flags) |
| 917 | return 0; | 917 | return 0; |
| 918 | 918 | ||
| 919 | if ((nft_table_has_owner(ctx->table) && | ||
| 920 | !(flags & NFT_TABLE_F_OWNER)) || | ||
| 921 | (!nft_table_has_owner(ctx->table) && | ||
| 922 | flags & NFT_TABLE_F_OWNER)) | ||
| 923 | return -EOPNOTSUPP; | ||
| 924 | |||
| 919 | trans = nft_trans_alloc(ctx, NFT_MSG_NEWTABLE, | 925 | trans = nft_trans_alloc(ctx, NFT_MSG_NEWTABLE, |
| 920 | sizeof(struct nft_trans_table)); | 926 | sizeof(struct nft_trans_table)); |
| 921 | if (trans == NULL) | 927 | if (trans == NULL) |
| @@ -9022,8 +9028,12 @@ static void __nft_release_hooks(struct net *net) · 8100 unmodified lines | |||
| 9022 | { | 9028 | { |
| 9023 | struct nft_table *table; | 9029 | struct nft_table *table; |
| 9024 | 9030 | ||
| 9025 | list_for_each_entry(table, &net->nft.tables, list) | 9031 | list_for_each_entry(table, &net->nft.tables, list) { |
| 9032 | if (nft_table_has_owner(table)) | ||
| 9033 | continue; | ||
| 9034 | |||
| 9026 | __nft_release_hook(net, table); | 9035 | __nft_release_hook(net, table); |
| 9036 | } | ||
| 9027 | } | 9037 | } |
| 9028 | 9038 | ||
| 9029 | static void __nft_release_table(struct net *net, struct nft_table *table) | 9039 | static void __nft_release_table(struct net *net, struct nft_table *table) |
| @@ -9073,13 +9083,12 @@ static void __nft_release_table(struct net *net, struct nft_table *table) · 43 unmodified lines | |||
| 9073 | nf_tables_table_destroy(&ctx); | 9083 | nf_tables_table_destroy(&ctx); |
| 9074 | } | 9084 | } |
| 9075 | 9085 | ||
| 9076 | static void __nft_release_tables(struct net *net, u32 nlpid) | 9086 | static void __nft_release_tables(struct net *net) |
| 9077 | { | 9087 | { |
| 9078 | struct nft_table *table, *nt; | 9088 | struct nft_table *table, *nt; |
| 9079 | 9089 | ||
| 9080 | list_for_each_entry_safe(table, nt, &net->nft.tables, list) { | 9090 | list_for_each_entry_safe(table, nt, &net->nft.tables, list) { |
| 9081 | if (nft_table_has_owner(table) && | 9091 | if (nft_table_has_owner(table)) |
| 9082 | nlpid != table->nlpid) | ||
| 9083 | continue; | 9092 | continue; |
| 9084 | 9093 | ||
| 9085 | __nft_release_table(net, table); | 9094 | __nft_release_table(net, table); |
| @@ -9145,7 +9154,7 @@ static void __net_exit nf_tables_exit_net(struct net *net) · 59 unmodified lines | |||
| 9145 | mutex_lock(&net->nft.commit_mutex); | 9154 | mutex_lock(&net->nft.commit_mutex); |
| 9146 | if (!list_empty(&net->nft.commit_list)) | 9155 | if (!list_empty(&net->nft.commit_list)) |
| 9147 | __nf_tables_abort(net, NFNL_ABORT_NONE); | 9156 | __nf_tables_abort(net, NFNL_ABORT_NONE); |
| 9148 | __nft_release_tables(net, 0); | 9157 | __nft_release_tables(net); |
| 9149 | mutex_unlock(&net->nft.commit_mutex); | 9158 | mutex_unlock(&net->nft.commit_mutex); |
| 9150 | WARN_ON_ONCE(!list_empty(&net->nft.tables)); | 9159 | WARN_ON_ONCE(!list_empty(&net->nft.tables)); |
| 9151 | WARN_ON_ONCE(!list_empty(&net->nft.module_list)); | 9160 | WARN_ON_ONCE(!list_empty(&net->nft.module_list)); |
| @@ -330,6 +330,7 @@ static int match_revfn(u8 af, const char *name, u8 revision, int *bestp) · 329 unmodified lines | |||
| 330 | const struct xt_match *m; | 330 | const struct xt_match *m; |
| 331 | int have_rev = 0; | 331 | int have_rev = 0; |
| 332 | 332 | ||
| 333 | mutex_lock(&xt[af].mutex); | ||
| 333 | list_for_each_entry(m, &xt[af].match, list) { | 334 | list_for_each_entry(m, &xt[af].match, list) { |
| 334 | if (strcmp(m->name, name) == 0) { | 335 | if (strcmp(m->name, name) == 0) { |
| 335 | if (m->revision > *bestp) | 336 | if (m->revision > *bestp) |
| @@ -338,6 +339,7 @@ static int match_revfn(u8 af, const char *name, u8 revision, int *bestp) · 2 unmodified lines | |||
| 338 | have_rev = 1; | 339 | have_rev = 1; |
| 339 | } | 340 | } |
| 340 | } | 341 | } |
| 342 | mutex_unlock(&xt[af].mutex); | ||
| 341 | 343 | ||
| 342 | if (af != NFPROTO_UNSPEC && !have_rev) | 344 | if (af != NFPROTO_UNSPEC && !have_rev) |
| 343 | return match_revfn(NFPROTO_UNSPEC, name, revision, bestp); | 345 | return match_revfn(NFPROTO_UNSPEC, name, revision, bestp); |
| @@ -350,6 +352,7 @@ static int target_revfn(u8 af, const char *name, u8 revision, int *bestp) · 6 unmodified lines | |||
| 350 | const struct xt_target *t; | 352 | const struct xt_target *t; |
| 351 | int have_rev = 0; | 353 | int have_rev = 0; |
| 352 | 354 | ||
| 355 | mutex_lock(&xt[af].mutex); | ||
| 353 | list_for_each_entry(t, &xt[af].target, list) { | 356 | list_for_each_entry(t, &xt[af].target, list) { |
| 354 | if (strcmp(t->name, name) == 0) { | 357 | if (strcmp(t->name, name) == 0) { |
| 355 | if (t->revision > *bestp) | 358 | if (t->revision > *bestp) |
| @@ -358,6 +361,7 @@ static int target_revfn(u8 af, const char *name, u8 revision, int *bestp) · 2 unmodified lines | |||
| 358 | have_rev = 1; | 361 | have_rev = 1; |
| 359 | } | 362 | } |
| 360 | } | 363 | } |
| 364 | mutex_unlock(&xt[af].mutex); | ||
| 361 | 365 | ||
| 362 | if (af != NFPROTO_UNSPEC && !have_rev) | 366 | if (af != NFPROTO_UNSPEC && !have_rev) |
| 363 | return target_revfn(NFPROTO_UNSPEC, name, revision, bestp); | 367 | return target_revfn(NFPROTO_UNSPEC, name, revision, bestp); |
| @@ -371,12 +375,10 @@ int xt_find_revision(u8 af, const char *name, u8 revision, int target, · 7 unmodified lines | |||
| 371 | { | 375 | { |
| 372 | int have_rev, best = -1; | 376 | int have_rev, best = -1; |
| 373 | 377 | ||
| 374 | mutex_lock(&xt[af].mutex); | ||
| 375 | if (target == 1) | 378 | if (target == 1) |
| 376 | have_rev = target_revfn(af, name, revision, &best); | 379 | have_rev = target_revfn(af, name, revision, &best); |
| 377 | else | 380 | else |
| 378 | have_rev = match_revfn(af, name, revision, &best); | 381 | have_rev = match_revfn(af, name, revision, &best); |
| 379 | mutex_unlock(&xt[af].mutex); | ||
| 380 | 382 | ||
| 381 | /* Nothing at all? Return 0 to try loading module. */ | 383 | /* Nothing at all? Return 0 to try loading module. */ |
| 382 | if (best == -1) { | 384 | if (best == -1) { |
| @@ -575,6 +575,7 @@ list_start: · 574 unmodified lines | |||
| 575 | 575 | ||
| 576 | break; | 576 | break; |
| 577 | } | 577 | } |
| 578 | cipso_v4_doi_putdef(doi_def); | ||
| 578 | rcu_read_unlock(); | 579 | rcu_read_unlock(); |
| 579 | 580 | ||
| 580 | genlmsg_end(ans_skb, data); | 581 | genlmsg_end(ans_skb, data); |
| @@ -583,12 +584,14 @@ list_start: · 2 unmodified lines | |||
| 583 | list_retry: | 584 | list_retry: |
| 584 | /* XXX - this limit is a guesstimate */ | 585 | /* XXX - this limit is a guesstimate */ |
| 585 | if (nlsze_mult < 4) { | 586 | if (nlsze_mult < 4) { |
| 587 | cipso_v4_doi_putdef(doi_def); | ||
| 586 | rcu_read_unlock(); | 588 | rcu_read_unlock(); |
| 587 | kfree_skb(ans_skb); | 589 | kfree_skb(ans_skb); |
| 588 | nlsze_mult *= 2; | 590 | nlsze_mult *= 2; |
| 589 | goto list_start; | 591 | goto list_start; |
| 590 | } | 592 | } |
| 591 | list_failure_lock: | 593 | list_failure_lock: |
| 594 | cipso_v4_doi_putdef(doi_def); | ||
| 592 | rcu_read_unlock(); | 595 | rcu_read_unlock(); |
| 593 | list_failure: | 596 | list_failure: |
| 594 | kfree_skb(ans_skb); | 597 | kfree_skb(ans_skb); |
| @@ -439,7 +439,7 @@ int qrtr_endpoint_post(struct qrtr_endpoint *ep, const void *data, size_t len) · 438 unmodified lines | |||
| 439 | if (len == 0 || len & 3) | 439 | if (len == 0 || len & 3) |
| 440 | return -EINVAL; | 440 | return -EINVAL; |
| 441 | 441 | ||
| 442 | skb = netdev_alloc_skb(NULL, len); | 442 | skb = __netdev_alloc_skb(NULL, len, GFP_ATOMIC | __GFP_NOWARN); |
| 443 | if (!skb) | 443 | if (!skb) |
| 444 | return -ENOMEM; | 444 | return -ENOMEM; |
| 445 | 445 | ||
| @@ -958,8 +958,10 @@ static int qrtr_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) · 512 unmodified lines | |||
| 958 | plen = (len + 3) & ~3; | 958 | plen = (len + 3) & ~3; |
| 959 | skb = sock_alloc_send_skb(sk, plen + QRTR_HDR_MAX_SIZE, | 959 | skb = sock_alloc_send_skb(sk, plen + QRTR_HDR_MAX_SIZE, |
| 960 | msg->msg_flags & MSG_DONTWAIT, &rc); | 960 | msg->msg_flags & MSG_DONTWAIT, &rc); |
| 961 | if (!skb) | 961 | if (!skb) { |
| 962 | rc = -ENOMEM; | ||
| 962 | goto out_node; | 963 | goto out_node; |
| 964 | } | ||
| 963 | 965 | ||
| 964 | skb_reserve(skb, QRTR_HDR_MAX_SIZE); | 966 | skb_reserve(skb, QRTR_HDR_MAX_SIZE); |
| 965 | 967 | ||
| @@ -2168,7 +2168,7 @@ static int tc_dump_tclass_qdisc(struct Qdisc *q, struct sk_buff *skb, · 2167 unmodified lines | |||
| 2168 | 2168 | ||
| 2169 | static int tc_dump_tclass_root(struct Qdisc *root, struct sk_buff *skb, | 2169 | static int tc_dump_tclass_root(struct Qdisc *root, struct sk_buff *skb, |
| 2170 | struct tcmsg *tcm, struct netlink_callback *cb, | 2170 | struct tcmsg *tcm, struct netlink_callback *cb, |
| 2171 | int *t_p, int s_t) | 2171 | int *t_p, int s_t, bool recur) |
| 2172 | { | 2172 | { |
| 2173 | struct Qdisc *q; | 2173 | struct Qdisc *q; |
| 2174 | int b; | 2174 | int b; |
| @@ -2179,7 +2179,7 @@ static int tc_dump_tclass_root(struct Qdisc *root, struct sk_buff *skb, · 4 unmodified lines | |||
| 2179 | if (tc_dump_tclass_qdisc(root, skb, tcm, cb, t_p, s_t) < 0) | 2179 | if (tc_dump_tclass_qdisc(root, skb, tcm, cb, t_p, s_t) < 0) |
| 2180 | return -1; | 2180 | return -1; |
| 2181 | 2181 | ||
| 2182 | if (!qdisc_dev(root)) | 2182 | if (!qdisc_dev(root) || !recur) |
| 2183 | return 0; | 2183 | return 0; |
| 2184 | 2184 | ||
| 2185 | if (tcm->tcm_parent) { | 2185 | if (tcm->tcm_parent) { |
| @@ -2214,13 +2214,13 @@ static int tc_dump_tclass(struct sk_buff *skb, struct netlink_callback *cb) · 28 unmodified lines | |||
| 2214 | s_t = cb->args[0]; | 2214 | s_t = cb->args[0]; |
| 2215 | t = 0; | 2215 | t = 0; |
| 2216 | 2216 | ||
| 2217 | if (tc_dump_tclass_root(dev->qdisc, skb, tcm, cb, &t, s_t) < 0) | 2217 | if (tc_dump_tclass_root(dev->qdisc, skb, tcm, cb, &t, s_t, true) < 0) |
| 2218 | goto done; | 2218 | goto done; |
| 2219 | 2219 | ||
| 2220 | dev_queue = dev_ingress_queue(dev); | 2220 | dev_queue = dev_ingress_queue(dev); |
| 2221 | if (dev_queue && | 2221 | if (dev_queue && |
| 2222 | tc_dump_tclass_root(dev_queue->qdisc_sleeping, skb, tcm, cb, | 2222 | tc_dump_tclass_root(dev_queue->qdisc_sleeping, skb, tcm, cb, |
| 2223 | &t, s_t) < 0) | 2223 | &t, s_t, false) < 0) |
| 2224 | goto done; | 2224 | goto done; |
| 2225 | 2225 | ||
| 2226 | done: | 2226 | done: |
| @@ -75,7 +75,7 @@ int sctp_tsnmap_check(const struct sctp_tsnmap *map, __u32 tsn) · 74 unmodified lines | |||
| 75 | return 1; | 75 | return 1; |
| 76 | 76 | ||
| 77 | /* Verify that we can hold this TSN and that it will not | 77 | /* Verify that we can hold this TSN and that it will not |
| 78 | * overlfow our map | 78 | * overflow our map |
| 79 | */ | 79 | */ |
| 80 | if (!TSN_lt(tsn, map->base_tsn + SCTP_TSN_MAP_SIZE)) | 80 | if (!TSN_lt(tsn, map->base_tsn + SCTP_TSN_MAP_SIZE)) |
| 81 | return -1; | 81 | return -1; |
| @@ -1825,11 +1825,14 @@ static int · 1824 unmodified lines | |||
| 1825 | svcauth_gss_release(struct svc_rqst *rqstp) | 1825 | svcauth_gss_release(struct svc_rqst *rqstp) |
| 1826 | { | 1826 | { |
| 1827 | struct gss_svc_data *gsd = (struct gss_svc_data *)rqstp->rq_auth_data; | 1827 | struct gss_svc_data *gsd = (struct gss_svc_data *)rqstp->rq_auth_data; |
| 1828 | struct rpc_gss_wire_cred *gc = &gsd->clcred; | 1828 | struct rpc_gss_wire_cred *gc; |
| 1829 | struct xdr_buf *resbuf = &rqstp->rq_res; | 1829 | struct xdr_buf *resbuf = &rqstp->rq_res; |
| 1830 | int stat = -EINVAL; | 1830 | int stat = -EINVAL; |
| 1831 | struct sunrpc_net *sn = net_generic(SVC_NET(rqstp), sunrpc_net_id); | 1831 | struct sunrpc_net *sn = net_generic(SVC_NET(rqstp), sunrpc_net_id); |
| 1832 | 1832 | ||
| 1833 | if (!gsd) | ||
| 1834 | goto out; | ||
| 1835 | gc = &gsd->clcred; | ||
| 1833 | if (gc->gc_proc != RPC_GSS_PROC_DATA) | 1836 | if (gc->gc_proc != RPC_GSS_PROC_DATA) |
| 1834 | goto out; | 1837 | goto out; |
| 1835 | /* Release can be called twice, but we only wrap once. */ | 1838 | /* Release can be called twice, but we only wrap once. */ |
| @@ -1870,10 +1873,10 @@ out_err: · 34 unmodified lines | |||
| 1870 | if (rqstp->rq_cred.cr_group_info) | 1873 | if (rqstp->rq_cred.cr_group_info) |
| 1871 | put_group_info(rqstp->rq_cred.cr_group_info); | 1874 | put_group_info(rqstp->rq_cred.cr_group_info); |
| 1872 | rqstp->rq_cred.cr_group_info = NULL; | 1875 | rqstp->rq_cred.cr_group_info = NULL; |
| 1873 | if (gsd->rsci) | 1876 | if (gsd && gsd->rsci) { |
| 1874 | cache_put(&gsd->rsci->h, sn->rsc_cache); | 1877 | cache_put(&gsd->rsci->h, sn->rsc_cache); |
| 1875 | gsd->rsci = NULL; | 1878 | gsd->rsci = NULL; |
| 1876 | 1879 | } | |
| 1877 | return stat; | 1880 | return stat; |
| 1878 | } | 1881 | } |
| 1879 | 1882 | ||
| @@ -963,8 +963,11 @@ void rpc_execute(struct rpc_task *task) · 962 unmodified lines | |||
| 963 | 963 | ||
| 964 | rpc_set_active(task); | 964 | rpc_set_active(task); |
| 965 | rpc_make_runnable(rpciod_workqueue, task); | 965 | rpc_make_runnable(rpciod_workqueue, task); |
| 966 | if (!is_async) | 966 | if (!is_async) { |
| 967 | unsigned int pflags = memalloc_nofs_save(); | ||
| 967 | __rpc_execute(task); | 968 | __rpc_execute(task); |
| 969 | memalloc_nofs_restore(pflags); | ||
| 970 | } | ||
| 968 | } | 971 | } |
| 969 | 972 | ||
| 970 | static void rpc_async_schedule(struct work_struct *work) | 973 | static void rpc_async_schedule(struct work_struct *work) |
| @@ -1413,7 +1413,7 @@ svc_process_common(struct svc_rqst *rqstp, struct kvec *argv, struct kvec *resv) · 1412 unmodified lines | |||
| 1413 | 1413 | ||
| 1414 | sendit: | 1414 | sendit: |
| 1415 | if (svc_authorise(rqstp)) | 1415 | if (svc_authorise(rqstp)) |
| 1416 | goto close; | 1416 | goto close_xprt; |
| 1417 | return 1; /* Caller can now send it */ | 1417 | return 1; /* Caller can now send it */ |
| 1418 | 1418 | ||
| 1419 | release_dropit: | 1419 | release_dropit: |
| @@ -1425,6 +1425,8 @@ release_dropit: · 5 unmodified lines | |||
| 1425 | return 0; | 1425 | return 0; |
| 1426 | 1426 | ||
| 1427 | close: | 1427 | close: |
| 1428 | svc_authorise(rqstp); | ||
| 1429 | close_xprt: | ||
| 1428 | if (rqstp->rq_xprt && test_bit(XPT_TEMP, &rqstp->rq_xprt->xpt_flags)) | 1430 | if (rqstp->rq_xprt && test_bit(XPT_TEMP, &rqstp->rq_xprt->xpt_flags)) |
| 1429 | svc_close_xprt(rqstp->rq_xprt); | 1431 | svc_close_xprt(rqstp->rq_xprt); |
| 1430 | dprintk("svc: svc_process close\n"); | 1432 | dprintk("svc: svc_process close\n"); |
| @@ -1433,7 +1435,7 @@ release_dropit: · 2 unmodified lines | |||
| 1433 | err_short_len: | 1435 | err_short_len: |
| 1434 | svc_printk(rqstp, "short len %zd, dropping request\n", | 1436 | svc_printk(rqstp, "short len %zd, dropping request\n", |
| 1435 | argv->iov_len); | 1437 | argv->iov_len); |
| 1436 | goto close; | 1438 | goto close_xprt; |
| 1437 | 1439 | ||
| 1438 | err_bad_rpc: | 1440 | err_bad_rpc: |
| 1439 | serv->sv_stats->rpcbadfmt++; | 1441 | serv->sv_stats->rpcbadfmt++; |
| @@ -1060,7 +1060,7 @@ static int svc_close_list(struct svc_serv *serv, struct list_head *xprt_list, st · 1059 unmodified lines | |||
| 1060 | struct svc_xprt *xprt; | 1060 | struct svc_xprt *xprt; |
| 1061 | int ret = 0; | 1061 | int ret = 0; |
| 1062 | 1062 | ||
| 1063 | spin_lock(&serv->sv_lock); | 1063 | spin_lock_bh(&serv->sv_lock); |
| 1064 | list_for_each_entry(xprt, xprt_list, xpt_list) { | 1064 | list_for_each_entry(xprt, xprt_list, xpt_list) { |
| 1065 | if (xprt->xpt_net != net) | 1065 | if (xprt->xpt_net != net) |
| 1066 | continue; | 1066 | continue; |
| @@ -1068,7 +1068,7 @@ static int svc_close_list(struct svc_serv *serv, struct list_head *xprt_list, st · 1 unmodified line | |||
| 1068 | set_bit(XPT_CLOSE, &xprt->xpt_flags); | 1068 | set_bit(XPT_CLOSE, &xprt->xpt_flags); |
| 1069 | svc_xprt_enqueue(xprt); | 1069 | svc_xprt_enqueue(xprt); |
| 1070 | } | 1070 | } |
| 1071 | spin_unlock(&serv->sv_lock); | 1071 | spin_unlock_bh(&serv->sv_lock); |
| 1072 | return ret; | 1072 | return ret; |
| 1073 | } | 1073 | } |
| 1074 | 1074 | ||
| @@ -252,9 +252,9 @@ xprt_setup_rdma_bc(struct xprt_create *args) · 251 unmodified lines | |||
| 252 | xprt->timeout = &xprt_rdma_bc_timeout; | 252 | xprt->timeout = &xprt_rdma_bc_timeout; |
| 253 | xprt_set_bound(xprt); | 253 | xprt_set_bound(xprt); |
| 254 | xprt_set_connected(xprt); | 254 | xprt_set_connected(xprt); |
| 255 | xprt->bind_timeout = RPCRDMA_BIND_TO; | 255 | xprt->bind_timeout = 0; |
| 256 | xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO; | 256 | xprt->reestablish_timeout = 0; |
| 257 | xprt->idle_timeout = RPCRDMA_IDLE_DISC_TO; | 257 | xprt->idle_timeout = 0; |
| 258 | 258 | ||
| 259 | xprt->prot = XPRT_TRANSPORT_BC_RDMA; | 259 | xprt->prot = XPRT_TRANSPORT_BC_RDMA; |
| 260 | xprt->ops = &xprt_rdma_bc_procs; | 260 | xprt->ops = &xprt_rdma_bc_procs; |
| @@ -266,46 +266,33 @@ void svc_rdma_release_rqst(struct svc_rqst *rqstp) · 265 unmodified lines | |||
| 266 | svc_rdma_recv_ctxt_put(rdma, ctxt); | 266 | svc_rdma_recv_ctxt_put(rdma, ctxt); |
| 267 | } | 267 | } |
| 268 | 268 | ||
| 269 | static bool svc_rdma_refresh_recvs(struct svcxprt_rdma *rdma, | 269 | static int __svc_rdma_post_recv(struct svcxprt_rdma *rdma, |
| 270 | unsigned int wanted, bool temp) | 270 | struct svc_rdma_recv_ctxt *ctxt) |
| 271 | { | 271 | { |
| 272 | const struct ib_recv_wr *bad_wr = NULL; | ||
| 273 | struct svc_rdma_recv_ctxt *ctxt; | ||
| 274 | struct ib_recv_wr *recv_chain; | ||
| 275 | int ret; | 272 | int ret; |
| 276 | 273 | ||
| 277 | recv_chain = NULL; | 274 | trace_svcrdma_post_recv(ctxt); |
| 278 | while (wanted--) { | 275 | ret = ib_post_recv(rdma->sc_qp, &ctxt->rc_recv_wr, NULL); |
| 279 | ctxt = svc_rdma_recv_ctxt_get(rdma); | ||
| 280 | if (!ctxt) | ||
| 281 | break; | ||
| 282 | |||
| 283 | trace_svcrdma_post_recv(ctxt); | ||
| 284 | ctxt->rc_temp = temp; | ||
| 285 | ctxt->rc_recv_wr.next = recv_chain; | ||
| 286 | recv_chain = &ctxt->rc_recv_wr; | ||
| 287 | rdma->sc_pending_recvs++; | ||
| 288 | } | ||
| 289 | if (!recv_chain) | ||
| 290 | return false; | ||
| 291 | |||
| 292 | ret = ib_post_recv(rdma->sc_qp, recv_chain, &bad_wr); | ||
| 293 | if (ret) | 276 | if (ret) |
| 294 | goto err_post; | 277 | goto err_post; |
| 295 | return true; | 278 | return 0; |
| 296 | 279 | ||
| 297 | err_post: | 280 | err_post: |
| 298 | while (bad_wr) { | ||
| 299 | ctxt = container_of(bad_wr, struct svc_rdma_recv_ctxt, | ||
| 300 | rc_recv_wr); | ||
| 301 | bad_wr = bad_wr->next; | ||
| 302 | svc_rdma_recv_ctxt_put(rdma, ctxt); | ||
| 303 | } | ||
| 304 | |||
| 305 | trace_svcrdma_rq_post_err(rdma, ret); | 281 | trace_svcrdma_rq_post_err(rdma, ret); |
| 306 | /* Since we're destroying the xprt, no need to reset | 282 | svc_rdma_recv_ctxt_put(rdma, ctxt); |
| 307 | * sc_pending_recvs. */ | 283 | return ret; |
| 308 | return false; | 284 | } |
| 285 | |||
| 286 | static int svc_rdma_post_recv(struct svcxprt_rdma *rdma) | ||
| 287 | { | ||
| 288 | struct svc_rdma_recv_ctxt *ctxt; | ||
| 289 | |||
| 290 | if (test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags)) | ||
| 291 | return 0; | ||
| 292 | ctxt = svc_rdma_recv_ctxt_get(rdma); | ||
| 293 | if (!ctxt) | ||
| 294 | return -ENOMEM; | ||
| 295 | return __svc_rdma_post_recv(rdma, ctxt); | ||
| 309 | } | 296 | } |
| 310 | 297 | ||
| 311 | /** | 298 | /** |
| @@ -316,7 +303,20 @@ err_post: · 4 unmodified lines | |||
| 316 | */ | 303 | */ |
| 317 | bool svc_rdma_post_recvs(struct svcxprt_rdma *rdma) | 304 | bool svc_rdma_post_recvs(struct svcxprt_rdma *rdma) |
| 318 | { | 305 | { |
| 319 | return svc_rdma_refresh_recvs(rdma, rdma->sc_max_requests, true); | 306 | struct svc_rdma_recv_ctxt *ctxt; |
| 307 | unsigned int i; | ||
| 308 | int ret; | ||
| 309 | |||
| 310 | for (i = 0; i < rdma->sc_max_requests; i++) { | ||
| 311 | ctxt = svc_rdma_recv_ctxt_get(rdma); | ||
| 312 | if (!ctxt) | ||
| 313 | return false; | ||
| 314 | ctxt->rc_temp = true; | ||
| 315 | ret = __svc_rdma_post_recv(rdma, ctxt); | ||
| 316 | if (ret) | ||
| 317 | return false; | ||
| 318 | } | ||
| 319 | return true; | ||
| 320 | } | 320 | } |
| 321 | 321 | ||
| 322 | /** | 322 | /** |
| @@ -324,6 +324,8 @@ bool svc_rdma_post_recvs(struct svcxprt_rdma *rdma) · 1 unmodified line | |||
| 324 | * @cq: Completion Queue context | 324 | * @cq: Completion Queue context |
| 325 | * @wc: Work Completion object | 325 | * @wc: Work Completion object |
| 326 | * | 326 | * |
| 327 | * NB: The svc_xprt/svcxprt_rdma is pinned whenever it's possible that | ||
| 328 | * the Receive completion handler could be running. | ||
| 327 | */ | 329 | */ |
| 328 | static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) | 330 | static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) |
| 329 | { | 331 | { |
| @@ -331,8 +333,6 @@ static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) · 1 unmodified line | |||
| 331 | struct ib_cqe *cqe = wc->wr_cqe; | 333 | struct ib_cqe *cqe = wc->wr_cqe; |
| 332 | struct svc_rdma_recv_ctxt *ctxt; | 334 | struct svc_rdma_recv_ctxt *ctxt; |
| 333 | 335 | ||
| 334 | rdma->sc_pending_recvs--; | ||
| 335 | |||
| 336 | /* WARNING: Only wc->wr_cqe and wc->status are reliable */ | 336 | /* WARNING: Only wc->wr_cqe and wc->status are reliable */ |
| 337 | ctxt = container_of(cqe, struct svc_rdma_recv_ctxt, rc_cqe); | 337 | ctxt = container_of(cqe, struct svc_rdma_recv_ctxt, rc_cqe); |
| 338 | 338 | ||
| @@ -340,6 +340,9 @@ static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) · 1 unmodified line | |||
| 340 | if (wc->status != IB_WC_SUCCESS) | 340 | if (wc->status != IB_WC_SUCCESS) |
| 341 | goto flushed; | 341 | goto flushed; |
| 342 | 342 | ||
| 343 | if (svc_rdma_post_recv(rdma)) | ||
| 344 | goto post_err; | ||
| 345 | |||
| 343 | /* All wc fields are now known to be valid */ | 346 | /* All wc fields are now known to be valid */ |
| 344 | ctxt->rc_byte_len = wc->byte_len; | 347 | ctxt->rc_byte_len = wc->byte_len; |
| 345 | 348 | ||
| @@ -350,18 +353,11 @@ static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) · 4 unmodified lines | |||
| 350 | spin_unlock(&rdma->sc_rq_dto_lock); | 353 | spin_unlock(&rdma->sc_rq_dto_lock); |
| 351 | if (!test_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags)) | 354 | if (!test_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags)) |
| 352 | svc_xprt_enqueue(&rdma->sc_xprt); | 355 | svc_xprt_enqueue(&rdma->sc_xprt); |
| 353 | |||
| 354 | if (!test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags) && | ||
| 355 | rdma->sc_pending_recvs < rdma->sc_max_requests) | ||
| 356 | if (!svc_rdma_refresh_recvs(rdma, RPCRDMA_MAX_RECV_BATCH, | ||
| 357 | false)) | ||
| 358 | goto post_err; | ||
| 359 | |||
| 360 | return; | 356 | return; |
| 361 | 357 | ||
| 362 | flushed: | 358 | flushed: |
| 363 | svc_rdma_recv_ctxt_put(rdma, ctxt); | ||
| 364 | post_err: | 359 | post_err: |
| 360 | svc_rdma_recv_ctxt_put(rdma, ctxt); | ||
| 365 | set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags); | 361 | set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags); |
| 366 | svc_xprt_enqueue(&rdma->sc_xprt); | 362 | svc_xprt_enqueue(&rdma->sc_xprt); |
| 367 | } | 363 | } |
| @@ -1699,5 +1699,7 @@ int main(int argc, char **argv) · 1698 unmodified lines | |||
| 1699 | 1699 | ||
| 1700 | xdpsock_cleanup(); | 1700 | xdpsock_cleanup(); |
| 1701 | 1701 | ||
| 1702 | munmap(bufs, NUM_FRAMES * opt_xsk_frame_size); | ||
| 1703 | |||
| 1702 | return 0; | 1704 | return 0; |
| 1703 | } | 1705 | } |
| @@ -327,7 +327,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE · 326 unmodified lines | |||
| 327 | 327 | ||
| 328 | quiet_cmd_dtc = DTC $@ | 328 | quiet_cmd_dtc = DTC $@ |
| 329 | cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \ | 329 | cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \ |
| 330 | $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \ | 330 | $(DTC) -o $@ -b 0 \ |
| 331 | $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \ | 331 | $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \ |
| 332 | -d $(depfile).dtc.tmp $(dtc-tmp) ; \ | 332 | -d $(depfile).dtc.tmp $(dtc-tmp) ; \ |
| 333 | cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile) | 333 | cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile) |
| @@ -352,7 +352,7 @@ define rule_dtc · 18 unmodified lines | |||
| 352 | endef | 352 | endef |
| 353 | 353 | ||
| 354 | $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE | 354 | $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE |
| 355 | $(call if_changed_rule,dtc,yaml) | 355 | $(call if_changed_rule,dtc) |
| 356 | 356 | ||
| 357 | dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp) | 357 | dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp) |
| 358 | 358 | ||
| @@ -57,9 +57,9 @@ if arg_contain --version "$@"; then · 56 unmodified lines | |||
| 57 | fi | 57 | fi |
| 58 | 58 | ||
| 59 | if arg_contain -E "$@"; then | 59 | if arg_contain -E "$@"; then |
| 60 | # For scripts/gcc-version.sh; This emulates GCC 20.0.0 | 60 | # For scripts/cc-version.sh; This emulates GCC 20.0.0 |
| 61 | if arg_contain - "$@"; then | 61 | if arg_contain - "$@"; then |
| 62 | sed 's/^__GNUC__$/20/; s/^__GNUC_MINOR__$/0/; s/^__GNUC_PATCHLEVEL__$/0/' | 62 | sed -n '/^GCC/{s/__GNUC__/20/; s/__GNUC_MINOR__/0/; s/__GNUC_PATCHLEVEL__/0/; p;}' |
| 63 | exit 0 | 63 | exit 0 |
| 64 | else | 64 | else |
| 65 | echo "no input files" >&2 | 65 | echo "no input files" >&2 |
| @@ -73,6 +73,15 @@ if arg_contain -S "$@"; then · 7 unmodified lines | |||
| 73 | echo "%gs" | 73 | echo "%gs" |
| 74 | exit 0 | 74 | exit 0 |
| 75 | fi | 75 | fi |
| 76 | |||
| 77 | # For arch/powerpc/tools/gcc-check-mprofile-kernel.sh | ||
| 78 | if arg_contain -m64 "$@" && arg_contain -mlittle-endian "$@" && | ||
| 79 | arg_contain -mprofile-kernel "$@"; then | ||
| 80 | if ! test -t 0 && ! grep -q notrace; then | ||
| 81 | echo "_mcount" | ||
| 82 | fi | ||
| 83 | exit 0 | ||
| 84 | fi | ||
| 76 | fi | 85 | fi |
| 77 | 86 | ||
| 78 | # To set GCC_PLUGINS | 87 | # To set GCC_PLUGINS |
| @@ -85,3 +94,8 @@ if arg_contain -print-file-name=plugin "$@"; then · 6 unmodified lines | |||
| 85 | echo $plugin_dir | 94 | echo $plugin_dir |
| 86 | exit 0 | 95 | exit 0 |
| 87 | fi | 96 | fi |
| 97 | |||
| 98 | # inverted return value | ||
| 99 | if arg_contain -D__SIZEOF_INT128__=0 "$@"; then | ||
| 100 | exit 1 | ||
| 101 | fi | ||
| @@ -22,6 +22,7 @@ always-y += $(GCC_PLUGIN) · 21 unmodified lines | |||
| 22 | GCC_PLUGINS_DIR = $(shell $(CC) -print-file-name=plugin) | 22 | GCC_PLUGINS_DIR = $(shell $(CC) -print-file-name=plugin) |
| 23 | 23 | ||
| 24 | plugin_cxxflags = -Wp,-MMD,$(depfile) $(KBUILD_HOSTCXXFLAGS) -fPIC \ | 24 | plugin_cxxflags = -Wp,-MMD,$(depfile) $(KBUILD_HOSTCXXFLAGS) -fPIC \ |
| 25 | -include $(srctree)/include/linux/compiler-version.h \ | ||
| 25 | -I $(GCC_PLUGINS_DIR)/include -I $(obj) -std=gnu++11 \ | 26 | -I $(GCC_PLUGINS_DIR)/include -I $(obj) -std=gnu++11 \ |
| 26 | -fno-rtti -fno-exceptions -fasynchronous-unwind-tables \ | 27 | -fno-rtti -fno-exceptions -fasynchronous-unwind-tables \ |
| 27 | -ggdb -Wno-narrowing -Wno-unused-variable \ | 28 | -ggdb -Wno-narrowing -Wno-unused-variable \ |
| @@ -29,7 +29,7 @@ orig_args="$@" · 28 unmodified lines | |||
| 29 | # Get the first line of the --version output. | 29 | # Get the first line of the --version output. |
| 30 | IFS=' | 30 | IFS=' |
| 31 | ' | 31 | ' |
| 32 | set -- $("$@" --version) | 32 | set -- $(LC_ALL=C "$@" --version) |
| 33 | 33 | ||
| 34 | # Split the line on spaces. | 34 | # Split the line on spaces. |
| 35 | IFS=' ' | 35 | IFS=' ' |
| @@ -44,14 +44,20 @@ if [ "$1" = GNU -a "$2" = ld ]; then · 8 unmodified lines | |||
| 44 | elif [ "$1" = GNU -a "$2" = gold ]; then | 44 | elif [ "$1" = GNU -a "$2" = gold ]; then |
| 45 | echo "gold linker is not supported as it is not capable of linking the kernel proper." >&2 | 45 | echo "gold linker is not supported as it is not capable of linking the kernel proper." >&2 |
| 46 | exit 1 | 46 | exit 1 |
| 47 | elif [ "$1" = LLD ]; then | ||
| 48 | version=$2 | ||
| 49 | min_version=$lld_min_version | ||
| 50 | name=LLD | ||
| 51 | disp_name=LLD | ||
| 52 | else | 47 | else |
| 53 | echo "$orig_args: unknown linker" >&2 | 48 | while [ $# -gt 1 -a "$1" != "LLD" ]; do |
| 54 | exit 1 | 49 | shift |
| 50 | done | ||
| 51 | |||
| 52 | if [ "$1" = LLD ]; then | ||
| 53 | version=$2 | ||
| 54 | min_version=$lld_min_version | ||
| 55 | name=LLD | ||
| 56 | disp_name=LLD | ||
| 57 | else | ||
| 58 | echo "$orig_args: unknown linker" >&2 | ||
| 59 | exit 1 | ||
| 60 | fi | ||
| 55 | fi | 61 | fi |
| 56 | 62 | ||
| 57 | # Some distributions append a package release number, as in 2.34-4.fc32 | 63 | # Some distributions append a package release number, as in 2.34-4.fc32 |
| @@ -543,8 +543,7 @@ int cap_convert_nscap(struct user_namespace *mnt_userns, struct dentry *dentry, · 542 unmodified lines | |||
| 543 | __u32 magic, nsmagic; | 543 | __u32 magic, nsmagic; |
| 544 | struct inode *inode = d_backing_inode(dentry); | 544 | struct inode *inode = d_backing_inode(dentry); |
| 545 | struct user_namespace *task_ns = current_user_ns(), | 545 | struct user_namespace *task_ns = current_user_ns(), |
| 546 | *fs_ns = inode->i_sb->s_user_ns, | 546 | *fs_ns = inode->i_sb->s_user_ns; |
| 547 | *ancestor; | ||
| 548 | kuid_t rootid; | 547 | kuid_t rootid; |
| 549 | size_t newsize; | 548 | size_t newsize; |
| 550 | 549 | ||
| @@ -567,15 +566,6 @@ int cap_convert_nscap(struct user_namespace *mnt_userns, struct dentry *dentry, · 16 unmodified lines | |||
| 567 | if (nsrootid == -1) | 566 | if (nsrootid == -1) |
| 568 | return -EINVAL; | 567 | return -EINVAL; |
| 569 | 568 | ||
| 570 | /* | ||
| 571 | * Do not allow allow adding a v3 filesystem capability xattr | ||
| 572 | * if the rootid field is ambiguous. | ||
| 573 | */ | ||
| 574 | for (ancestor = task_ns->parent; ancestor; ancestor = ancestor->parent) { | ||
| 575 | if (from_kuid(ancestor, rootid) == 0) | ||
| 576 | return -EINVAL; | ||
| 577 | } | ||
| 578 | |||
| 579 | newsize = sizeof(struct vfs_ns_cap_data); | 569 | newsize = sizeof(struct vfs_ns_cap_data); |
| 580 | nscap = kmalloc(newsize, GFP_ATOMIC); | 570 | nscap = kmalloc(newsize, GFP_ATOMIC); |
| 581 | if (!nscap) | 571 | if (!nscap) |
| @@ -39,6 +39,11 @@ int intel_nhlt_get_dmic_geo(struct device *dev, struct nhlt_acpi_table *nhlt) · 38 unmodified lines | |||
| 39 | if (!nhlt) | 39 | if (!nhlt) |
| 40 | return 0; | 40 | return 0; |
| 41 | 41 | ||
| 42 | if (nhlt->header.length <= sizeof(struct acpi_table_header)) { | ||
| 43 | dev_warn(dev, "Invalid DMIC description table\n"); | ||
| 44 | return 0; | ||
| 45 | } | ||
| 46 | |||
| 42 | for (j = 0, epnt = nhlt->desc; j < nhlt->endpoint_count; j++, | 47 | for (j = 0, epnt = nhlt->desc; j < nhlt->endpoint_count; j++, |
| 43 | epnt = (struct nhlt_endpoint *)((u8 *)epnt + epnt->length)) { | 48 | epnt = (struct nhlt_endpoint *)((u8 *)epnt + epnt->length)) { |
| 44 | 49 | ||
| @@ -47,6 +47,10 @@ static void hda_codec_unsol_event(struct hdac_device *dev, unsigned int ev) · 46 unmodified lines | |||
| 47 | if (codec->bus->shutdown) | 47 | if (codec->bus->shutdown) |
| 48 | return; | 48 | return; |
| 49 | 49 | ||
| 50 | /* ignore unsol events during system suspend/resume */ | ||
| 51 | if (codec->core.dev.power.power_state.event != PM_EVENT_ON) | ||
| 52 | return; | ||
| 53 | |||
| 50 | if (codec->patch_ops.unsol_event) | 54 | if (codec->patch_ops.unsol_event) |
| 51 | codec->patch_ops.unsol_event(codec, ev); | 55 | codec->patch_ops.unsol_event(codec, ev); |
| 52 | } | 56 | } |
| @@ -609,13 +609,6 @@ static int azx_pcm_open(struct snd_pcm_substream *substream) · 608 unmodified lines | |||
| 609 | 20, | 609 | 20, |
| 610 | 178000000); | 610 | 178000000); |
| 611 | 611 | ||
| 612 | /* by some reason, the playback stream stalls on PulseAudio with | ||
| 613 | * tsched=1 when a capture stream triggers. Until we figure out the | ||
| 614 | * real cause, disable tsched mode by telling the PCM info flag. | ||
| 615 | */ | ||
| 616 | if (chip->driver_caps & AZX_DCAPS_AMD_WORKAROUND) | ||
| 617 | runtime->hw.info |= SNDRV_PCM_INFO_BATCH; | ||
| 618 | |||
| 619 | if (chip->align_buffer_size) | 612 | if (chip->align_buffer_size) |
| 620 | /* constrain buffer sizes to be multiple of 128 | 613 | /* constrain buffer sizes to be multiple of 128 |
| 621 | bytes. This is more efficient in terms of memory | 614 | bytes. This is more efficient in terms of memory |
| @@ -1026,6 +1026,8 @@ static int azx_prepare(struct device *dev) · 1025 unmodified lines | |||
| 1026 | chip = card->private_data; | 1026 | chip = card->private_data; |
| 1027 | chip->pm_prepared = 1; | 1027 | chip->pm_prepared = 1; |
| 1028 | 1028 | ||
| 1029 | flush_work(&azx_bus(chip)->unsol_work); | ||
| 1030 | |||
| 1029 | /* HDA controller always requires different WAKEEN for runtime suspend | 1031 | /* HDA controller always requires different WAKEEN for runtime suspend |
| 1030 | * and system suspend, so don't use direct-complete here. | 1032 | * and system suspend, so don't use direct-complete here. |
| 1031 | */ | 1033 | */ |
| @@ -1309,6 +1309,7 @@ static const struct snd_pci_quirk ca0132_quirks[] = { · 1308 unmodified lines | |||
| 1309 | SND_PCI_QUIRK(0x1102, 0x0013, "Recon3D", QUIRK_R3D), | 1309 | SND_PCI_QUIRK(0x1102, 0x0013, "Recon3D", QUIRK_R3D), |
| 1310 | SND_PCI_QUIRK(0x1102, 0x0018, "Recon3D", QUIRK_R3D), | 1310 | SND_PCI_QUIRK(0x1102, 0x0018, "Recon3D", QUIRK_R3D), |
| 1311 | SND_PCI_QUIRK(0x1102, 0x0051, "Sound Blaster AE-5", QUIRK_AE5), | 1311 | SND_PCI_QUIRK(0x1102, 0x0051, "Sound Blaster AE-5", QUIRK_AE5), |
| 1312 | SND_PCI_QUIRK(0x1102, 0x0191, "Sound Blaster AE-5 Plus", QUIRK_AE5), | ||
| 1312 | SND_PCI_QUIRK(0x1102, 0x0081, "Sound Blaster AE-7", QUIRK_AE7), | 1313 | SND_PCI_QUIRK(0x1102, 0x0081, "Sound Blaster AE-7", QUIRK_AE7), |
| 1313 | {} | 1314 | {} |
| 1314 | }; | 1315 | }; |
| @@ -149,6 +149,21 @@ static int cx_auto_vmaster_mute_led(struct led_classdev *led_cdev, · 148 unmodified lines | |||
| 149 | return 0; | 149 | return 0; |
| 150 | } | 150 | } |
| 151 | 151 | ||
| 152 | static void cxt_init_gpio_led(struct hda_codec *codec) | ||
| 153 | { | ||
| 154 | struct conexant_spec *spec = codec->spec; | ||
| 155 | unsigned int mask = spec->gpio_mute_led_mask | spec->gpio_mic_led_mask; | ||
| 156 | |||
| 157 | if (mask) { | ||
| 158 | snd_hda_codec_write(codec, 0x01, 0, AC_VERB_SET_GPIO_MASK, | ||
| 159 | mask); | ||
| 160 | snd_hda_codec_write(codec, 0x01, 0, AC_VERB_SET_GPIO_DIRECTION, | ||
| 161 | mask); | ||
| 162 | snd_hda_codec_write(codec, 0x01, 0, AC_VERB_SET_GPIO_DATA, | ||
| 163 | spec->gpio_led); | ||
| 164 | } | ||
| 165 | } | ||
| 166 | |||
| 152 | static int cx_auto_init(struct hda_codec *codec) | 167 | static int cx_auto_init(struct hda_codec *codec) |
| 153 | { | 168 | { |
| 154 | struct conexant_spec *spec = codec->spec; | 169 | struct conexant_spec *spec = codec->spec; |
| @@ -156,6 +171,7 @@ static int cx_auto_init(struct hda_codec *codec) · 1 unmodified line | |||
| 156 | if (!spec->dynamic_eapd) | 171 | if (!spec->dynamic_eapd) |
| 157 | cx_auto_turn_eapd(codec, spec->num_eapds, spec->eapds, true); | 172 | cx_auto_turn_eapd(codec, spec->num_eapds, spec->eapds, true); |
| 158 | 173 | ||
| 174 | cxt_init_gpio_led(codec); | ||
| 159 | snd_hda_apply_fixup(codec, HDA_FIXUP_ACT_INIT); | 175 | snd_hda_apply_fixup(codec, HDA_FIXUP_ACT_INIT); |
| 160 | 176 | ||
| 161 | return 0; | 177 | return 0; |
| @@ -215,6 +231,7 @@ enum { · 53 unmodified lines | |||
| 215 | CXT_FIXUP_HP_SPECTRE, | 231 | CXT_FIXUP_HP_SPECTRE, |
| 216 | CXT_FIXUP_HP_GATE_MIC, | 232 | CXT_FIXUP_HP_GATE_MIC, |
| 217 | CXT_FIXUP_MUTE_LED_GPIO, | 233 | CXT_FIXUP_MUTE_LED_GPIO, |
| 234 | CXT_FIXUP_HP_ZBOOK_MUTE_LED, | ||
| 218 | CXT_FIXUP_HEADSET_MIC, | 235 | CXT_FIXUP_HEADSET_MIC, |
| 219 | CXT_FIXUP_HP_MIC_NO_PRESENCE, | 236 | CXT_FIXUP_HP_MIC_NO_PRESENCE, |
| 220 | }; | 237 | }; |
| @@ -654,31 +671,36 @@ static int cxt_gpio_micmute_update(struct led_classdev *led_cdev, · 433 unmodified lines | |||
| 654 | return 0; | 671 | return 0; |
| 655 | } | 672 | } |
78 more lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 402 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 4 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.