Unified · Split
| @@ -653,6 +653,9 @@ · 652 unmodified lines | ||
| 653 | 653 | cpuidle.off=1 [CPU_IDLE] |
| 654 | 654 | disable the cpuidle sub-system |
| 655 | 655 | |
| 656 | + cpufreq.off=1 [CPU_FREQ] | |
| 657 | + disable the cpufreq sub-system | |
| 658 | + | |
| 656 | 659 | cpu_init_udelay=N |
| 657 | 660 | [X86] Delay for N microsec between assert and de-assert |
| 658 | 661 | of APIC INIT to start processors. This delay occurs |
| @@ -1183,6 +1186,12 @@ · 524 unmodified lines | ||
| 1183 | 1186 | functions that can be changed at run time by the |
| 1184 | 1187 | set_graph_notrace file in the debugfs tracing directory. |
| 1185 | 1188 | |
| 1189 | + ftrace_graph_max_depth=<uint> | |
| 1190 | + [FTRACE] Used with the function graph tracer. This is | |
| 1191 | + the max depth it will trace into a function. This value | |
| 1192 | + can be changed at run time by the max_graph_depth file | |
| 1193 | + in the tracefs tracing directory. default: 0 (no limit) | |
| 1194 | + | |
| 1186 | 1195 | gamecon.map[2|3]= |
| 1187 | 1196 | [HW,JOY] Multisystem joystick and NES/SNES/PSX pad |
| 1188 | 1197 | support via parallel port (up to 5 devices per port) |
| @@ -68,3 +68,4 @@ stable kernels. · 67 unmodified lines | ||
| 68 | 68 | | | | | | |
| 69 | 69 | | Qualcomm Tech. | Falkor v1 | E1003 | QCOM_FALKOR_ERRATUM_1003 | |
| 70 | 70 | | Qualcomm Tech. | Falkor v1 | E1009 | QCOM_FALKOR_ERRATUM_1009 | |
| 71 | +| Qualcomm Tech. | QDF2400 ITS | E0065 | QCOM_QDF2400_ERRATUM_0065 | | |
| @@ -1142,16 +1142,17 @@ used by the kernel. · 1141 unmodified lines | ||
| 1142 | 1142 | |
| 1143 | 1143 | pids.max |
| 1144 | 1144 | |
| 1145 | − A read-write single value file which exists on non-root cgroups. The | |
| 1146 | − default is "max". | |
| 1145 | + A read-write single value file which exists on non-root | |
| 1146 | + cgroups. The default is "max". | |
| 1147 | 1147 | |
| 1148 | − Hard limit of number of processes. | |
| 1148 | + Hard limit of number of processes. | |
| 1149 | 1149 | |
| 1150 | 1150 | pids.current |
| 1151 | 1151 | |
| 1152 | − A read-only single value file which exists on all cgroups. | |
| 1152 | + A read-only single value file which exists on all cgroups. | |
| 1153 | 1153 | |
| 1154 | − The number of processes currently in the cgroup and its descendants. | |
| 1154 | + The number of processes currently in the cgroup and its | |
| 1155 | + descendants. | |
| 1155 | 1156 | |
| 1156 | 1157 | Organisational operations are not blocked by cgroup policies, so it is |
| 1157 | 1158 | possible to have pids.current > pids.max. This can be done by either |
| @@ -10,7 +10,7 @@ Note that kcov does not aim to collect as much coverage as possible. It aims · 9 unmodified lines | ||
| 10 | 10 | to collect more or less stable coverage that is function of syscall inputs. |
| 11 | 11 | To achieve this goal it does not collect coverage in soft/hard interrupts |
| 12 | 12 | and instrumentation of some inherently non-deterministic parts of kernel is |
| 13 | −disbled (e.g. scheduler, locking). | |
| 13 | +disabled (e.g. scheduler, locking). | |
| 14 | 14 | |
| 15 | 15 | Usage |
| 16 | 16 | ----- |
| @@ -71,6 +71,9 @@ · 70 unmodified lines | ||
| 71 | 71 | For Axon it can be absent, though my current driver |
| 72 | 72 | doesn't handle phy-address yet so for now, keep |
| 73 | 73 | 0x00ffffff in it. |
| 74 | + - phy-handle : Used to describe configurations where a external PHY | |
| 75 | + is used. Please refer to: | |
| 76 | + Documentation/devicetree/bindings/net/ethernet.txt | |
| 74 | 77 | - rx-fifo-size-gige : 1 cell, Rx fifo size in bytes for 1000 Mb/sec |
| 75 | 78 | operations (if absent the value is the same as |
| 76 | 79 | rx-fifo-size). For Axon, either absent or 2048. |
| @@ -81,8 +84,22 @@ · 4 unmodified lines | ||
| 81 | 84 | offload, phandle of the TAH device node. |
| 82 | 85 | - tah-channel : 1 cell, optional. If appropriate, channel used on the |
| 83 | 86 | TAH engine. |
| 87 | + - fixed-link : Fixed-link subnode describing a link to a non-MDIO | |
| 88 | + managed entity. See | |
| 89 | + Documentation/devicetree/bindings/net/fixed-link.txt | |
| 90 | + for details. | |
| 91 | + - mdio subnode : When the EMAC has a phy connected to its local | |
| 92 | + mdio, which us supported by the kernel's network | |
| 93 | + PHY library in drivers/net/phy, there must be device | |
| 94 | + tree subnode with the following required properties: | |
| 95 | + - #address-cells: Must be <1>. | |
| 96 | + - #size-cells: Must be <0>. | |
| 84 | 97 | |
| 85 | − Example: | |
| 98 | + For PHY definitions: Please refer to | |
| 99 | + Documentation/devicetree/bindings/net/phy.txt and | |
| 100 | + Documentation/devicetree/bindings/net/ethernet.txt | |
| 101 | + | |
| 102 | + Examples: | |
| 86 | 103 | |
| 87 | 104 | EMAC0: ethernet@40000800 { |
| 88 | 105 | device_type = "network"; |
| @@ -104,6 +121,48 @@ · 15 unmodified lines | ||
| 104 | 121 | zmii-channel = <0>; |
| 105 | 122 | }; |
| 106 | 123 | |
| 124 | + EMAC1: ethernet@ef600c00 { | |
| 125 | + device_type = "network"; | |
| 126 | + compatible = "ibm,emac-apm821xx", "ibm,emac4sync"; | |
| 127 | + interrupt-parent = <&EMAC1>; | |
| 128 | + interrupts = <0 1>; | |
| 129 | + #interrupt-cells = <1>; | |
| 130 | + #address-cells = <0>; | |
| 131 | + #size-cells = <0>; | |
| 132 | + interrupt-map = <0 &UIC2 0x10 IRQ_TYPE_LEVEL_HIGH /* Status */ | |
| 133 | + 1 &UIC2 0x14 IRQ_TYPE_LEVEL_HIGH /* Wake */>; | |
| 134 | + reg = <0xef600c00 0x000000c4>; | |
| 135 | + local-mac-address = [000000000000]; /* Filled in by U-Boot */ | |
| 136 | + mal-device = <&MAL0>; | |
| 137 | + mal-tx-channel = <0>; | |
| 138 | + mal-rx-channel = <0>; | |
| 139 | + cell-index = <0>; | |
| 140 | + max-frame-size = <9000>; | |
| 141 | + rx-fifo-size = <16384>; | |
| 142 | + tx-fifo-size = <2048>; | |
| 143 | + fifo-entry-size = <10>; | |
| 144 | + phy-mode = "rgmii"; | |
| 145 | + phy-handle = <&phy0>; | |
| 146 | + phy-map = <0x00000000>; | |
| 147 | + rgmii-device = <&RGMII0>; | |
| 148 | + rgmii-channel = <0>; | |
| 149 | + tah-device = <&TAH0>; | |
| 150 | + tah-channel = <0>; | |
| 151 | + has-inverted-stacr-oc; | |
| 152 | + has-new-stacr-staopc; | |
| 153 | + | |
| 154 | + mdio { | |
| 155 | + #address-cells = <1>; | |
| 156 | + #size-cells = <0>; | |
| 157 | + | |
| 158 | + phy0: ethernet-phy@0 { | |
| 159 | + compatible = "ethernet-phy-ieee802.3-c22"; | |
| 160 | + reg = <0>; | |
| 161 | + }; | |
| 162 | + }; | |
| 163 | + }; | |
| 164 | + | |
| 165 | + | |
| 107 | 166 | ii) McMAL node |
| 108 | 167 | |
| 109 | 168 | Required properties: |
| @@ -145,4 +204,3 @@ · 35 unmodified lines | ||
| 145 | 204 | - revision : as provided by the RGMII new version register if |
| 146 | 205 | available. |
| 147 | 206 | For Axon: 0x0000012a |
| 148 | − | |
| @@ -45,7 +45,7 @@ Required Properties: · 44 unmodified lines | ||
| 45 | 45 | Optional Properties: |
| 46 | 46 | - reg-names: In addition to the required properties, the following are optional |
| 47 | 47 | - "efuse-address" - Contains efuse base address used to pick up ABB info. |
| 48 | − - "ldo-address" - Contains address of ABB LDO overide register address. | |
| 48 | + - "ldo-address" - Contains address of ABB LDO override register. | |
| 49 | 49 | "efuse-address" is required for this. |
| 50 | 50 | - ti,ldovbb-vset-mask - Required if ldo-address is set, mask for LDO override |
| 51 | 51 | register to provide override vset value. |
| @@ -7,18 +7,18 @@ Required properties : · 6 unmodified lines | ||
| 7 | 7 | - compatible : Should be "microchip,usb251xb" or one of the specific types: |
| 8 | 8 | "microchip,usb2512b", "microchip,usb2512bi", "microchip,usb2513b", |
| 9 | 9 | "microchip,usb2513bi", "microchip,usb2514b", "microchip,usb2514bi" |
| 10 | − - hub-reset-gpios : Should specify the gpio for hub reset | |
| 10 | + - reset-gpios : Should specify the gpio for hub reset | |
| 11 | + - reg : I2C address on the selected bus (default is <0x2C>) | |
| 11 | 12 | |
| 12 | 13 | Optional properties : |
| 13 | − - reg : I2C address on the selected bus (default is <0x2C>) | |
| 14 | 14 | - skip-config : Skip Hub configuration, but only send the USB-Attach command |
| 15 | − - vendor-id : USB Vendor ID of the hub (16 bit, default is 0x0424) | |
| 16 | − - product-id : USB Product ID of the hub (16 bit, default depends on type) | |
| 17 | − - device-id : USB Device ID of the hub (16 bit, default is 0x0bb3) | |
| 18 | − - language-id : USB Language ID (16 bit, default is 0x0000) | |
| 19 | − - manufacturer : USB Manufacturer string (max 31 characters long) | |
| 20 | − - product : USB Product string (max 31 characters long) | |
| 21 | − - serial : USB Serial string (max 31 characters long) | |
| 15 | + - vendor-id : Set USB Vendor ID of the hub (16 bit, default is 0x0424) | |
| 16 | + - product-id : Set USB Product ID of the hub (16 bit, default depends on type) | |
| 17 | + - device-id : Set USB Device ID of the hub (16 bit, default is 0x0bb3) | |
| 18 | + - language-id : Set USB Language ID (16 bit, default is 0x0000) | |
| 19 | + - manufacturer : Set USB Manufacturer string (max 31 characters long) | |
| 20 | + - product : Set USB Product string (max 31 characters long) | |
| 21 | + - serial : Set USB Serial string (max 31 characters long) | |
| 22 | 22 | - {bus,self}-powered : selects between self- and bus-powered operation (default |
| 23 | 23 | is self-powered) |
| 24 | 24 | - disable-hi-speed : disable USB Hi-Speed support |
| @@ -31,8 +31,10 @@ Optional properties : · 6 unmodified lines | ||
| 31 | 31 | (default is individual) |
| 32 | 32 | - dynamic-power-switching : enable auto-switching from self- to bus-powered |
| 33 | 33 | operation if the local power source is removed or unavailable |
| 34 | − - oc-delay-{100us,4ms,8ms,16ms} : set over current timer delay (default is 8ms) | |
| 35 | − - compound-device : indicated the hub is part of a compound device | |
| 34 | + - oc-delay-us : Delay time (in microseconds) for filtering the over-current | |
| 35 | + sense inputs. Valid values are 100, 4000, 8000 (default) and 16000. If | |
| 36 | + an invalid value is given, the default is used instead. | |
| 37 | + - compound-device : indicate the hub is part of a compound device | |
| 36 | 38 | - port-mapping-mode : enable port mapping mode |
| 37 | 39 | - string-support : enable string descriptor support (required for manufacturer, |
| 38 | 40 | product and serial string configuration) |
| @@ -40,34 +42,15 @@ Optional properties : · 1 unmodified line | ||
| 40 | 42 | device connected. |
| 41 | 43 | - sp-disabled-ports : Specifies the ports which will be self-power disabled |
| 42 | 44 | - bp-disabled-ports : Specifies the ports which will be bus-power disabled |
| 43 | − - max-sp-power : Specifies the maximum current the hub consumes from an | |
| 44 | − upstream port when operating as self-powered hub including the power | |
| 45 | − consumption of a permanently attached peripheral if the hub is | |
| 46 | − configured as a compound device. The value is given in mA in a 0 - 500 | |
| 47 | − range (default is 2). | |
| 48 | − - max-bp-power : Specifies the maximum current the hub consumes from an | |
| 49 | − upstream port when operating as bus-powered hub including the power | |
| 50 | − consumption of a permanently attached peripheral if the hub is | |
| 51 | − configured as a compound device. The value is given in mA in a 0 - 500 | |
| 52 | − range (default is 100). | |
| 53 | − - max-sp-current : Specifies the maximum current the hub consumes from an | |
| 54 | − upstream port when operating as self-powered hub EXCLUDING the power | |
| 55 | − consumption of a permanently attached peripheral if the hub is | |
| 56 | − configured as a compound device. The value is given in mA in a 0 - 500 | |
| 57 | − range (default is 2). | |
| 58 | − - max-bp-current : Specifies the maximum current the hub consumes from an | |
| 59 | − upstream port when operating as bus-powered hub EXCLUDING the power | |
| 60 | − consumption of a permanently attached peripheral if the hub is | |
| 61 | − configured as a compound device. The value is given in mA in a 0 - 500 | |
| 62 | − range (default is 100). | |
| 63 | − - power-on-time : Specifies the time it takes from the time the host initiates | |
| 64 | − the power-on sequence to a port until the port has adequate power. The | |
| 65 | − value is given in ms in a 0 - 510 range (default is 100ms). | |
| 45 | + - power-on-time-ms : Specifies the time it takes from the time the host | |
| 46 | + initiates the power-on sequence to a port until the port has adequate | |
| 47 | + power. The value is given in ms in a 0 - 510 range (default is 100ms). | |
| 66 | 48 | |
| 67 | 49 | Examples: |
| 68 | 50 | usb2512b@2c { |
| 69 | 51 | compatible = "microchip,usb2512b"; |
| 70 | − hub-reset-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>; | |
| 52 | + reg = <0x2c>; | |
| 53 | + reset-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>; | |
| 71 | 54 | }; |
| 72 | 55 | |
| 73 | 56 | usb2514b@2c { |
| @@ -1006,7 +1006,8 @@ accept_redirects - BOOLEAN · 1005 unmodified lines | ||
| 1006 | 1006 | FALSE (router) |
| 1007 | 1007 | |
| 1008 | 1008 | forwarding - BOOLEAN |
| 1009 | − Enable IP forwarding on this interface. | |
| 1009 | + Enable IP forwarding on this interface. This controls whether packets | |
| 1010 | + received _on_ this interface can be forwarded. | |
| 1010 | 1011 | |
| 1011 | 1012 | mc_forwarding - BOOLEAN |
| 1012 | 1013 | Do multicast routing. The kernel needs to be compiled with CONFIG_MROUTE |
| @@ -951,6 +951,10 @@ This ioctl allows the user to create or modify a guest physical memory · 950 unmodified lines | ||
| 951 | 951 | slot. When changing an existing slot, it may be moved in the guest |
| 952 | 952 | physical memory space, or its flags may be modified. It may not be |
| 953 | 953 | resized. Slots may not overlap in guest physical address space. |
| 954 | +Bits 0-15 of "slot" specifies the slot id and this value should be | |
| 955 | +less than the maximum number of user memory slots supported per VM. | |
| 956 | +The maximum allowed slots can be queried using KVM_CAP_NR_MEMSLOTS, | |
| 957 | +if this capability is supported by the architecture. | |
| 954 | 958 | |
| 955 | 959 | If KVM_CAP_MULTI_ADDRESS_SPACE is available, bits 16-31 of "slot" |
| 956 | 960 | specifies the address space which is being modified. They must be |
| @@ -172,10 +172,6 @@ the same read(2) protocol as for the page fault notifications. The · 171 unmodified lines | ||
| 172 | 172 | manager has to explicitly enable these events by setting appropriate |
| 173 | 173 | bits in uffdio_api.features passed to UFFDIO_API ioctl: |
| 174 | 174 | |
| 175 | −UFFD_FEATURE_EVENT_EXIT - enable notification about exit() of the | |
| 176 | −non-cooperative process. When the monitored process exits, the uffd | |
| 177 | −manager will get UFFD_EVENT_EXIT. | |
| 178 | − | |
| 179 | 175 | UFFD_FEATURE_EVENT_FORK - enable userfaultfd hooks for fork(). When |
| 180 | 176 | this feature is enabled, the userfaultfd context of the parent process |
| 181 | 177 | is duplicated into the newly created process. The manager receives |
| @@ -8307,7 +8307,6 @@ M: Richard Leitner <richard.leitner@skidata.com> · 8306 unmodified lines | ||
| 8307 | 8307 | L: linux-usb@vger.kernel.org |
| 8308 | 8308 | S: Maintained |
| 8309 | 8309 | F: drivers/usb/misc/usb251xb.c |
| 8310 | −F: include/linux/platform_data/usb251xb.h | |
| 8311 | 8310 | F: Documentation/devicetree/bindings/usb/usb251xb.txt |
| 8312 | 8311 | |
| 8313 | 8312 | MICROSOFT SURFACE PRO 3 BUTTON DRIVER |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | VERSION = 4 |
| 2 | 2 | PATCHLEVEL = 11 |
| 3 | 3 | SUBLEVEL = 0 |
| 4 | −EXTRAVERSION = -rc1 | |
| 4 | +EXTRAVERSION = -rc2 | |
| 5 | 5 | NAME = Fearless Coyote |
| 6 | 6 | |
| 7 | 7 | # *DOCUMENTATION* |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #define _ASM_ARC_HUGEPAGE_H |
| 12 | 12 | |
| 13 | 13 | #include <linux/types.h> |
| 14 | +#define __ARCH_USE_5LEVEL_HACK | |
| 14 | 15 | #include <asm-generic/pgtable-nopmd.h> |
| 15 | 16 | |
| 16 | 17 | static inline pte_t pmd_pte(pmd_t pmd) |
| @@ -37,6 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | |
| 38 | 38 | #include <asm/page.h> |
| 39 | 39 | #include <asm/mmu.h> |
| 40 | +#define __ARCH_USE_5LEVEL_HACK | |
| 40 | 41 | #include <asm-generic/pgtable-nopmd.h> |
| 41 | 42 | #include <linux/const.h> |
| 42 | 43 | |
| @@ -209,6 +209,7 @@ · 208 unmodified lines | ||
| 209 | 209 | #define HSR_EC_IABT_HYP (0x21) |
| 210 | 210 | #define HSR_EC_DABT (0x24) |
| 211 | 211 | #define HSR_EC_DABT_HYP (0x25) |
| 212 | +#define HSR_EC_MAX (0x3f) | |
| 212 | 213 | |
| 213 | 214 | #define HSR_WFI_IS_WFE (_AC(1, UL) << 0) |
| 214 | 215 | |
| @@ -30,7 +30,6 @@ · 29 unmodified lines | ||
| 30 | 30 | #define __KVM_HAVE_ARCH_INTC_INITIALIZED |
| 31 | 31 | |
| 32 | 32 | #define KVM_USER_MEM_SLOTS 32 |
| 33 | −#define KVM_PRIVATE_MEM_SLOTS 4 | |
| 34 | 33 | #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 |
| 35 | 34 | #define KVM_HAVE_ONE_REG |
| 36 | 35 | #define KVM_HALT_POLL_NS_DEFAULT 500000 |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | #else |
| 22 | 22 | |
| 23 | +#define __ARCH_USE_5LEVEL_HACK | |
| 23 | 24 | #include <asm-generic/pgtable-nopud.h> |
| 24 | 25 | #include <asm/memory.h> |
| 25 | 26 | #include <asm/pgtable-hwdef.h> |
| @@ -221,6 +221,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 220 unmodified lines | ||
| 221 | 221 | case KVM_CAP_MAX_VCPUS: |
| 222 | 222 | r = KVM_MAX_VCPUS; |
| 223 | 223 | break; |
| 224 | + case KVM_CAP_NR_MEMSLOTS: | |
| 225 | + r = KVM_USER_MEM_SLOTS; | |
| 226 | + break; | |
| 224 | 227 | case KVM_CAP_MSI_DEVID: |
| 225 | 228 | if (!kvm) |
| 226 | 229 | r = -EINVAL; |
| @@ -79,7 +79,19 @@ static int kvm_handle_wfx(struct kvm_vcpu *vcpu, struct kvm_run *run) · 78 unmodified lines | ||
| 79 | 79 | return 1; |
| 80 | 80 | } |
| 81 | 81 | |
| 82 | +static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run) | |
| 83 | +{ | |
| 84 | + u32 hsr = kvm_vcpu_get_hsr(vcpu); | |
| 85 | + | |
| 86 | + kvm_pr_unimpl("Unknown exception class: hsr: %#08x\n", | |
| 87 | + hsr); | |
| 88 | + | |
| 89 | + kvm_inject_undefined(vcpu); | |
| 90 | + return 1; | |
| 91 | +} | |
| 92 | + | |
| 82 | 93 | static exit_handle_fn arm_exit_handlers[] = { |
| 94 | + [0 ... HSR_EC_MAX] = kvm_handle_unknown_ec, | |
| 83 | 95 | [HSR_EC_WFI] = kvm_handle_wfx, |
| 84 | 96 | [HSR_EC_CP15_32] = kvm_handle_cp15_32, |
| 85 | 97 | [HSR_EC_CP15_64] = kvm_handle_cp15_64, |
| @@ -98,13 +110,6 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu) · 12 unmodified lines | ||
| 98 | 110 | { |
| 99 | 111 | u8 hsr_ec = kvm_vcpu_trap_get_class(vcpu); |
| 100 | 112 | |
| 101 | − if (hsr_ec >= ARRAY_SIZE(arm_exit_handlers) || | |
| 102 | − !arm_exit_handlers[hsr_ec]) { | |
| 103 | − kvm_err("Unknown exception class: hsr: %#08x\n", | |
| 104 | − (unsigned int)kvm_vcpu_get_hsr(vcpu)); | |
| 105 | − BUG(); | |
| 106 | − } | |
| 107 | − | |
| 108 | 113 | return arm_exit_handlers[hsr_ec]; |
| 109 | 114 | } |
| 110 | 115 | |
| @@ -198,6 +198,8 @@ static const struct dma_map_ops xen_swiotlb_dma_ops = { · 197 unmodified lines | ||
| 198 | 198 | .unmap_page = xen_swiotlb_unmap_page, |
| 199 | 199 | .dma_supported = xen_swiotlb_dma_supported, |
| 200 | 200 | .set_dma_mask = xen_swiotlb_set_dma_mask, |
| 201 | + .mmap = xen_swiotlb_dma_mmap, | |
| 202 | + .get_sgtable = xen_swiotlb_get_sgtable, | |
| 201 | 203 | }; |
| 202 | 204 | |
| 203 | 205 | int __init xen_mm_init(void) |
| @@ -508,6 +508,16 @@ config QCOM_FALKOR_ERRATUM_1009 · 507 unmodified lines | ||
| 508 | 508 | |
| 509 | 509 | If unsure, say Y. |
| 510 | 510 | |
| 511 | +config QCOM_QDF2400_ERRATUM_0065 | |
| 512 | + bool "QDF2400 E0065: Incorrect GITS_TYPER.ITT_Entry_size" | |
| 513 | + default y | |
| 514 | + help | |
| 515 | + On Qualcomm Datacenter Technologies QDF2400 SoC, ITS hardware reports | |
| 516 | + ITE size incorrectly. The GITS_TYPER.ITT_Entry_size field should have | |
| 517 | + been indicated as 16Bytes (0xf), not 8Bytes (0x7). | |
| 518 | + | |
| 519 | + If unsure, say Y. | |
| 520 | + | |
| 511 | 521 | endmenu |
| 512 | 522 | |
| 513 | 523 | |
| @@ -30,8 +30,7 @@ · 29 unmodified lines | ||
| 30 | 30 | |
| 31 | 31 | #define __KVM_HAVE_ARCH_INTC_INITIALIZED |
| 32 | 32 | |
| 33 | −#define KVM_USER_MEM_SLOTS 32 | |
| 34 | −#define KVM_PRIVATE_MEM_SLOTS 4 | |
| 33 | +#define KVM_USER_MEM_SLOTS 512 | |
| 35 | 34 | #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 |
| 36 | 35 | #define KVM_HALT_POLL_NS_DEFAULT 500000 |
| 37 | 36 | |
| @@ -55,9 +55,13 @@ typedef struct { pteval_t pgprot; } pgprot_t; · 54 unmodified lines | ||
| 55 | 55 | #define __pgprot(x) ((pgprot_t) { (x) } ) |
| 56 | 56 | |
| 57 | 57 | #if CONFIG_PGTABLE_LEVELS == 2 |
| 58 | +#define __ARCH_USE_5LEVEL_HACK | |
| 58 | 59 | #include <asm-generic/pgtable-nopmd.h> |
| 59 | 60 | #elif CONFIG_PGTABLE_LEVELS == 3 |
| 61 | +#define __ARCH_USE_5LEVEL_HACK | |
| 60 | 62 | #include <asm-generic/pgtable-nopud.h> |
| 63 | +#elif CONFIG_PGTABLE_LEVELS == 4 | |
| 64 | +#include <asm-generic/5level-fixup.h> | |
| 61 | 65 | #endif |
| 62 | 66 | |
| 63 | 67 | #endif /* __ASM_PGTABLE_TYPES_H */ |
| @@ -135,7 +135,19 @@ static int kvm_handle_guest_debug(struct kvm_vcpu *vcpu, struct kvm_run *run) · 134 unmodified lines | ||
| 135 | 135 | return ret; |
| 136 | 136 | } |
| 137 | 137 | |
| 138 | +static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run) | |
| 139 | +{ | |
| 140 | + u32 hsr = kvm_vcpu_get_hsr(vcpu); | |
| 141 | + | |
| 142 | + kvm_pr_unimpl("Unknown exception class: hsr: %#08x -- %s\n", | |
| 143 | + hsr, esr_get_class_string(hsr)); | |
| 144 | + | |
| 145 | + kvm_inject_undefined(vcpu); | |
| 146 | + return 1; | |
| 147 | +} | |
| 148 | + | |
| 138 | 149 | static exit_handle_fn arm_exit_handlers[] = { |
| 150 | + [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec, | |
| 139 | 151 | [ESR_ELx_EC_WFx] = kvm_handle_wfx, |
| 140 | 152 | [ESR_ELx_EC_CP15_32] = kvm_handle_cp15_32, |
| 141 | 153 | [ESR_ELx_EC_CP15_64] = kvm_handle_cp15_64, |
| @@ -162,13 +174,6 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu) · 20 unmodified lines | ||
| 162 | 174 | u32 hsr = kvm_vcpu_get_hsr(vcpu); |
| 163 | 175 | u8 hsr_ec = ESR_ELx_EC(hsr); |
| 164 | 176 | |
| 165 | − if (hsr_ec >= ARRAY_SIZE(arm_exit_handlers) || | |
| 166 | − !arm_exit_handlers[hsr_ec]) { | |
| 167 | − kvm_err("Unknown exception class: hsr: %#08x -- %s\n", | |
| 168 | − hsr, esr_get_class_string(hsr)); | |
| 169 | − BUG(); | |
| 170 | − } | |
| 171 | − | |
| 172 | 177 | return arm_exit_handlers[hsr_ec]; |
| 173 | 178 | } |
| 174 | 179 | |
| @@ -18,14 +18,62 @@ · 17 unmodified lines | ||
| 18 | 18 | #include <asm/kvm_hyp.h> |
| 19 | 19 | #include <asm/tlbflush.h> |
| 20 | 20 | |
| 21 | +static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm) | |
| 22 | +{ | |
| 23 | + u64 val; | |
| 24 | + | |
| 25 | + /* | |
| 26 | + * With VHE enabled, we have HCR_EL2.{E2H,TGE} = {1,1}, and | |
| 27 | + * most TLB operations target EL2/EL0. In order to affect the | |
| 28 | + * guest TLBs (EL1/EL0), we need to change one of these two | |
| 29 | + * bits. Changing E2H is impossible (goodbye TTBR1_EL2), so | |
| 30 | + * let's flip TGE before executing the TLB operation. | |
| 31 | + */ | |
| 32 | + write_sysreg(kvm->arch.vttbr, vttbr_el2); | |
| 33 | + val = read_sysreg(hcr_el2); | |
| 34 | + val &= ~HCR_TGE; | |
| 35 | + write_sysreg(val, hcr_el2); | |
| 36 | + isb(); | |
| 37 | +} | |
| 38 | + | |
| 39 | +static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm) | |
| 40 | +{ | |
| 41 | + write_sysreg(kvm->arch.vttbr, vttbr_el2); | |
| 42 | + isb(); | |
| 43 | +} | |
| 44 | + | |
| 45 | +static hyp_alternate_select(__tlb_switch_to_guest, | |
| 46 | + __tlb_switch_to_guest_nvhe, | |
| 47 | + __tlb_switch_to_guest_vhe, | |
| 48 | + ARM64_HAS_VIRT_HOST_EXTN); | |
| 49 | + | |
| 50 | +static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm) | |
| 51 | +{ | |
| 52 | + /* | |
| 53 | + * We're done with the TLB operation, let's restore the host's | |
| 54 | + * view of HCR_EL2. | |
| 55 | + */ | |
| 56 | + write_sysreg(0, vttbr_el2); | |
| 57 | + write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2); | |
| 58 | +} | |
| 59 | + | |
| 60 | +static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm) | |
| 61 | +{ | |
| 62 | + write_sysreg(0, vttbr_el2); | |
| 63 | +} | |
| 64 | + | |
| 65 | +static hyp_alternate_select(__tlb_switch_to_host, | |
| 66 | + __tlb_switch_to_host_nvhe, | |
| 67 | + __tlb_switch_to_host_vhe, | |
| 68 | + ARM64_HAS_VIRT_HOST_EXTN); | |
| 69 | + | |
| 21 | 70 | void __hyp_text __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa) |
| 22 | 71 | { |
| 23 | 72 | dsb(ishst); |
| 24 | 73 | |
| 25 | 74 | /* Switch to requested VMID */ |
| 26 | 75 | kvm = kern_hyp_va(kvm); |
| 27 | − write_sysreg(kvm->arch.vttbr, vttbr_el2); | |
| 28 | − isb(); | |
| 76 | + __tlb_switch_to_guest()(kvm); | |
| 29 | 77 | |
| 30 | 78 | /* |
| 31 | 79 | * We could do so much better if we had the VA as well. |
| @@ -46,7 +94,7 @@ void __hyp_text __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa) · 14 unmodified lines | ||
| 46 | 94 | dsb(ish); |
| 47 | 95 | isb(); |
| 48 | 96 | |
| 49 | − write_sysreg(0, vttbr_el2); | |
| 97 | + __tlb_switch_to_host()(kvm); | |
| 50 | 98 | } |
| 51 | 99 | |
| 52 | 100 | void __hyp_text __kvm_tlb_flush_vmid(struct kvm *kvm) |
| @@ -55,14 +103,13 @@ void __hyp_text __kvm_tlb_flush_vmid(struct kvm *kvm) · 2 unmodified lines | ||
| 55 | 103 | |
| 56 | 104 | /* Switch to requested VMID */ |
| 57 | 105 | kvm = kern_hyp_va(kvm); |
| 58 | − write_sysreg(kvm->arch.vttbr, vttbr_el2); | |
| 59 | − isb(); | |
| 106 | + __tlb_switch_to_guest()(kvm); | |
| 60 | 107 | |
| 61 | 108 | __tlbi(vmalls12e1is); |
| 62 | 109 | dsb(ish); |
| 63 | 110 | isb(); |
| 64 | 111 | |
| 65 | − write_sysreg(0, vttbr_el2); | |
| 112 | + __tlb_switch_to_host()(kvm); | |
| 66 | 113 | } |
| 67 | 114 | |
| 68 | 115 | void __hyp_text __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu) |
| @@ -70,14 +117,13 @@ void __hyp_text __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu) · 1 unmodified line | ||
| 70 | 117 | struct kvm *kvm = kern_hyp_va(kern_hyp_va(vcpu)->kvm); |
| 71 | 118 | |
| 72 | 119 | /* Switch to requested VMID */ |
| 73 | − write_sysreg(kvm->arch.vttbr, vttbr_el2); | |
| 74 | − isb(); | |
| 120 | + __tlb_switch_to_guest()(kvm); | |
| 75 | 121 | |
| 76 | 122 | __tlbi(vmalle1); |
| 77 | 123 | dsb(nsh); |
| 78 | 124 | isb(); |
| 79 | 125 | |
| 80 | − write_sysreg(0, vttbr_el2); | |
| 126 | + __tlb_switch_to_host()(kvm); | |
| 81 | 127 | } |
| 82 | 128 | |
| 83 | 129 | void __hyp_text __kvm_flush_vm_context(void) |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #ifndef __ASM_AVR32_PGTABLE_2LEVEL_H |
| 9 | 9 | #define __ASM_AVR32_PGTABLE_2LEVEL_H |
| 10 | 10 | |
| 11 | +#define __ARCH_USE_5LEVEL_HACK | |
| 11 | 12 | #include <asm-generic/pgtable-nopmd.h> |
| 12 | 13 | |
| 13 | 14 | /* |
| @@ -14,7 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | */ |
| 15 | 15 | |
| 16 | 16 | #include <linux/oprofile.h> |
| 17 | −#include <linux/sched.h> | |
| 17 | +#include <linux/ptrace.h> | |
| 18 | 18 | #include <linux/uaccess.h> |
| 19 | 19 | |
| 20 | 20 | /* The first two words of each frame on the stack look like this if we have |
| @@ -2086,7 +2086,7 @@ static void cryptocop_job_queue_close(void) · 2085 unmodified lines | ||
| 2086 | 2086 | dma_in_cfg.en = regk_dma_no; |
| 2087 | 2087 | REG_WR(dma, IN_DMA_INST, rw_cfg, dma_in_cfg); |
| 2088 | 2088 | |
| 2089 | − /* Disble the cryptocop. */ | |
| 2089 | + /* Disable the cryptocop. */ | |
| 2090 | 2090 | rw_cfg = REG_RD(strcop, regi_strcop, rw_cfg); |
| 2091 | 2091 | rw_cfg.en = 0; |
| 2092 | 2092 | REG_WR(strcop, regi_strcop, rw_cfg, rw_cfg); |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #define _CRIS_PGTABLE_H |
| 7 | 7 | |
| 8 | 8 | #include <asm/page.h> |
| 9 | +#define __ARCH_USE_5LEVEL_HACK | |
| 9 | 10 | #include <asm-generic/pgtable-nopmd.h> |
| 10 | 11 | |
| 11 | 12 | #ifndef __ASSEMBLY__ |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #ifndef _ASM_PGTABLE_H |
| 17 | 17 | #define _ASM_PGTABLE_H |
| 18 | 18 | |
| 19 | +#include <asm-generic/5level-fixup.h> | |
| 19 | 20 | #include <asm/mem-layout.h> |
| 20 | 21 | #include <asm/setup.h> |
| 21 | 22 | #include <asm/processor.h> |
| @@ -1,5 +1,6 @@ | ||
| 1 | 1 | #ifndef _H8300_PGTABLE_H |
| 2 | 2 | #define _H8300_PGTABLE_H |
| 3 | +#define __ARCH_USE_5LEVEL_HACK | |
| 3 | 4 | #include <asm-generic/pgtable-nopud.h> |
| 4 | 5 | #include <asm-generic/pgtable.h> |
| 5 | 6 | #define pgtable_cache_init() do { } while (0) |
| @@ -9,7 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | */ |
| 10 | 10 | |
| 11 | 11 | #include <linux/linkage.h> |
| 12 | −#include <linux/sched.h> | |
| 12 | +#include <linux/sched/signal.h> | |
| 13 | 13 | #include <asm/ptrace.h> |
| 14 | 14 | |
| 15 | 15 | #define BREAKINST 0x5730 /* trapa #3 */ |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | */ |
| 27 | 27 | #include <linux/swap.h> |
| 28 | 28 | #include <asm/page.h> |
| 29 | +#define __ARCH_USE_5LEVEL_HACK | |
| 29 | 30 | #include <asm-generic/pgtable-nopmd.h> |
| 30 | 31 | |
| 31 | 32 | /* A handy thing to have if one has the RAM. Declared in head.S */ |
| @@ -587,8 +587,10 @@ extern struct page *zero_page_memmap_ptr; · 586 unmodified lines | ||
| 587 | 587 | |
| 588 | 588 | |
| 589 | 589 | #if CONFIG_PGTABLE_LEVELS == 3 |
| 590 | +#define __ARCH_USE_5LEVEL_HACK | |
| 590 | 591 | #include <asm-generic/pgtable-nopud.h> |
| 591 | 592 | #endif |
| 593 | +#include <asm-generic/5level-fixup.h> | |
| 592 | 594 | #include <asm-generic/pgtable.h> |
| 593 | 595 | |
| 594 | 596 | #endif /* _ASM_IA64_PGTABLE_H */ |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #define _METAG_PGTABLE_H |
| 7 | 7 | |
| 8 | 8 | #include <asm/pgtable-bits.h> |
| 9 | +#define __ARCH_USE_5LEVEL_HACK | |
| 9 | 10 | #include <asm-generic/pgtable-nopmd.h> |
| 10 | 11 | |
| 11 | 12 | /* Invalid regions on Meta: 0x00000000-0x001FFFFF and 0xFFFF0000-0xFFFFFFFF */ |
| @@ -95,7 +95,8 @@ typedef struct { unsigned long pgd; } pgd_t; · 94 unmodified lines | ||
| 95 | 95 | # else /* CONFIG_MMU */ |
| 96 | 96 | typedef struct { unsigned long ste[64]; } pmd_t; |
| 97 | 97 | typedef struct { pmd_t pue[1]; } pud_t; |
| 98 | −typedef struct { pud_t pge[1]; } pgd_t; | |
| 98 | +typedef struct { pud_t p4e[1]; } p4d_t; | |
| 99 | +typedef struct { p4d_t pge[1]; } pgd_t; | |
| 99 | 100 | # endif /* CONFIG_MMU */ |
| 100 | 101 | |
| 101 | 102 | # define pte_val(x) ((x).pte) |
| @@ -10,7 +10,9 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/irqflags.h> |
| 11 | 11 | #include <linux/notifier.h> |
| 12 | 12 | #include <linux/prefetch.h> |
| 13 | +#include <linux/ptrace.h> | |
| 13 | 14 | #include <linux/sched.h> |
| 15 | +#include <linux/sched/task_stack.h> | |
| 14 | 16 | |
| 15 | 17 | #include <asm/cop2.h> |
| 16 | 18 | #include <asm/current.h> |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <asm/cop2.h> |
| 10 | 10 | #include <linux/export.h> |
| 11 | 11 | #include <linux/interrupt.h> |
| 12 | +#include <linux/sched/task_stack.h> | |
| 12 | 13 | |
| 13 | 14 | #include "octeon-crypto.h" |
| 14 | 15 | |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/kernel_stat.h> |
| 13 | 13 | #include <linux/sched.h> |
| 14 | 14 | #include <linux/sched/hotplug.h> |
| 15 | +#include <linux/sched/task_stack.h> | |
| 15 | 16 | #include <linux/init.h> |
| 16 | 17 | #include <linux/export.h> |
| 17 | 18 | |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | |
| 13 | 13 | #include <linux/sched.h> |
| 14 | 14 | #include <linux/sched/task_stack.h> |
| 15 | +#include <linux/ptrace.h> | |
| 15 | 16 | #include <linux/thread_info.h> |
| 16 | 17 | #include <linux/bitops.h> |
| 17 | 18 | |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <asm/cachectl.h> |
| 17 | 17 | #include <asm/fixmap.h> |
| 18 | 18 | |
| 19 | +#define __ARCH_USE_5LEVEL_HACK | |
| 19 | 20 | #include <asm-generic/pgtable-nopmd.h> |
| 20 | 21 | |
| 21 | 22 | extern int temp_tlb_entry; |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | #include <asm/cachectl.h> |
| 18 | 18 | #include <asm/fixmap.h> |
| 19 | 19 | |
| 20 | +#define __ARCH_USE_5LEVEL_HACK | |
| 20 | 21 | #if defined(CONFIG_PAGE_SIZE_64KB) && !defined(CONFIG_MIPS_VA_BITS_48) |
| 21 | 22 | #include <asm-generic/pgtable-nopmd.h> |
| 22 | 23 | #else |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/init.h> |
| 12 | 12 | #include <linux/sched.h> |
| 13 | 13 | #include <linux/sched/hotplug.h> |
| 14 | +#include <linux/sched/task_stack.h> | |
| 14 | 15 | #include <linux/mm.h> |
| 15 | 16 | #include <linux/delay.h> |
| 16 | 17 | #include <linux/smp.h> |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | #include <linux/interrupt.h> |
| 24 | 24 | #include <linux/irqchip/mips-gic.h> |
| 25 | 25 | #include <linux/compiler.h> |
| 26 | +#include <linux/sched/task_stack.h> | |
| 26 | 27 | #include <linux/smp.h> |
| 27 | 28 | |
| 28 | 29 | #include <linux/atomic.h> |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/init.h> |
| 14 | 14 | #include <linux/sched.h> |
| 15 | 15 | #include <linux/notifier.h> |
| 16 | +#include <linux/ptrace.h> | |
| 16 | 17 | |
| 17 | 18 | #include <asm/fpu.h> |
| 18 | 19 | #include <asm/cop2.h> |
| @@ -35,6 +35,7 @@ · 34 unmodified lines | ||
| 35 | 35 | #include <linux/kernel.h> |
| 36 | 36 | #include <linux/delay.h> |
| 37 | 37 | #include <linux/init.h> |
| 38 | +#include <linux/sched/task_stack.h> | |
| 38 | 39 | #include <linux/smp.h> |
| 39 | 40 | #include <linux/irq.h> |
| 40 | 41 | |
| @@ -9,11 +9,14 @@ · 8 unmodified lines | ||
| 9 | 9 | * Copyright (C) 2009 Wind River Systems, |
| 10 | 10 | * written by Ralf Baechle <ralf@linux-mips.org> |
| 11 | 11 | */ |
| 12 | +#include <linux/capability.h> | |
| 12 | 13 | #include <linux/init.h> |
| 13 | 14 | #include <linux/irqflags.h> |
| 14 | 15 | #include <linux/notifier.h> |
| 15 | 16 | #include <linux/prefetch.h> |
| 17 | +#include <linux/ptrace.h> | |
| 16 | 18 | #include <linux/sched.h> |
| 19 | +#include <linux/sched/task_stack.h> | |
| 17 | 20 | |
| 18 | 21 | #include <asm/cop2.h> |
| 19 | 22 | #include <asm/current.h> |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <linux/kernel.h> |
| 10 | 10 | #include <linux/sched.h> |
| 11 | 11 | #include <linux/sched/debug.h> |
| 12 | +#include <linux/sched/signal.h> | |
| 12 | 13 | #include <linux/seq_file.h> |
| 13 | 14 | |
| 14 | 15 | #include <asm/addrspace.h> |
| @@ -12,7 +12,9 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/signal.h> /* for SIGBUS */ |
| 13 | 13 | #include <linux/sched.h> /* schow_regs(), force_sig() */ |
| 14 | 14 | #include <linux/sched/debug.h> |
| 15 | +#include <linux/sched/signal.h> | |
| 15 | 16 | |
| 17 | +#include <asm/ptrace.h> | |
| 16 | 18 | #include <asm/sn/addrs.h> |
| 17 | 19 | #include <asm/sn/arch.h> |
| 18 | 20 | #include <asm/sn/sn0/hub.h> |
| @@ -8,10 +8,13 @@ · 7 unmodified lines | ||
| 8 | 8 | */ |
| 9 | 9 | #include <linux/init.h> |
| 10 | 10 | #include <linux/sched.h> |
| 11 | +#include <linux/sched/task_stack.h> | |
| 11 | 12 | #include <linux/topology.h> |
| 12 | 13 | #include <linux/nodemask.h> |
| 14 | + | |
| 13 | 15 | #include <asm/page.h> |
| 14 | 16 | #include <asm/processor.h> |
| 17 | +#include <asm/ptrace.h> | |
| 15 | 18 | #include <asm/sn/arch.h> |
| 16 | 19 | #include <asm/sn/gda.h> |
| 17 | 20 | #include <asm/sn/intr.h> |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/kernel.h> |
| 12 | 12 | #include <linux/sched.h> |
| 13 | 13 | #include <linux/sched/debug.h> |
| 14 | +#include <linux/sched/signal.h> | |
| 14 | 15 | #include <asm/traps.h> |
| 15 | 16 | #include <linux/uaccess.h> |
| 16 | 17 | #include <asm/addrspace.h> |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/kernel.h> |
| 14 | 14 | #include <linux/module.h> |
| 15 | 15 | #include <linux/sched.h> |
| 16 | +#include <linux/sched/signal.h> | |
| 16 | 17 | #include <linux/notifier.h> |
| 17 | 18 | #include <linux/delay.h> |
| 18 | 19 | #include <linux/rtc/ds1685.h> |
| @@ -57,6 +57,7 @@ typedef struct page *pgtable_t; · 56 unmodified lines | ||
| 57 | 57 | #define __pgd(x) ((pgd_t) { (x) }) |
| 58 | 58 | #define __pgprot(x) ((pgprot_t) { (x) }) |
| 59 | 59 | |
| 60 | +#define __ARCH_USE_5LEVEL_HACK | |
| 60 | 61 | #include <asm-generic/pgtable-nopmd.h> |
| 61 | 62 | |
| 62 | 63 | #endif /* !__ASSEMBLY__ */ |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include <asm/tlbflush.h> |
| 23 | 23 | |
| 24 | 24 | #include <asm/pgtable-bits.h> |
| 25 | +#define __ARCH_USE_5LEVEL_HACK | |
| 25 | 26 | #include <asm-generic/pgtable-nopmd.h> |
| 26 | 27 | |
| 27 | 28 | #define FIRST_USER_ADDRESS 0UL |
| @@ -25,6 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | #ifndef __ASM_OPENRISC_PGTABLE_H |
| 26 | 26 | #define __ASM_OPENRISC_PGTABLE_H |
| 27 | 27 | |
| 28 | +#define __ARCH_USE_5LEVEL_HACK | |
| 28 | 29 | #include <asm-generic/pgtable-nopmd.h> |
| 29 | 30 | |
| 30 | 31 | #ifndef __ASSEMBLY__ |
| @@ -80,93 +80,99 @@ config ARCH_HAS_DMA_SET_COHERENT_MASK · 79 unmodified lines | ||
| 80 | 80 | config PPC |
| 81 | 81 | bool |
| 82 | 82 | default y |
| 83 | − select BUILDTIME_EXTABLE_SORT | |
| 83 | + # | |
| 84 | + # Please keep this list sorted alphabetically. | |
| 85 | + # | |
| 86 | + select ARCH_HAS_DEVMEM_IS_ALLOWED | |
| 87 | + select ARCH_HAS_DMA_SET_COHERENT_MASK | |
| 88 | + select ARCH_HAS_ELF_RANDOMIZE | |
| 89 | + select ARCH_HAS_GCOV_PROFILE_ALL | |
| 90 | + select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE | |
| 91 | + select ARCH_HAS_SG_CHAIN | |
| 92 | + select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST | |
| 93 | + select ARCH_HAS_UBSAN_SANITIZE_ALL | |
| 94 | + select ARCH_HAVE_NMI_SAFE_CMPXCHG | |
| 84 | 95 | select ARCH_MIGHT_HAVE_PC_PARPORT |
| 85 | 96 | select ARCH_MIGHT_HAVE_PC_SERIO |
| 97 | + select ARCH_SUPPORTS_ATOMIC_RMW | |
| 98 | + select ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT | |
| 99 | + select ARCH_USE_BUILTIN_BSWAP | |
| 100 | + select ARCH_USE_CMPXCHG_LOCKREF if PPC64 | |
| 101 | + select ARCH_WANT_IPC_PARSE_VERSION | |
| 86 | 102 | select BINFMT_ELF |
| 87 | − select ARCH_HAS_ELF_RANDOMIZE | |
| 88 | − select OF | |
| 89 | − select OF_EARLY_FLATTREE | |
| 90 | − select OF_RESERVED_MEM | |
| 91 | − select HAVE_FTRACE_MCOUNT_RECORD | |
| 103 | + select BUILDTIME_EXTABLE_SORT | |
| 104 | + select CLONE_BACKWARDS | |
| 105 | + select DCACHE_WORD_ACCESS if PPC64 && CPU_LITTLE_ENDIAN | |
| 106 | + select EDAC_ATOMIC_SCRUB | |
| 107 | + select EDAC_SUPPORT | |
| 108 | + select GENERIC_ATOMIC64 if PPC32 | |
| 109 | + select GENERIC_CLOCKEVENTS | |
| 110 | + select GENERIC_CLOCKEVENTS_BROADCAST if SMP | |
| 111 | + select GENERIC_CMOS_UPDATE | |
| 112 | + select GENERIC_CPU_AUTOPROBE | |
| 113 | + select GENERIC_IRQ_SHOW | |
| 114 | + select GENERIC_IRQ_SHOW_LEVEL | |
| 115 | + select GENERIC_SMP_IDLE_THREAD | |
| 116 | + select GENERIC_STRNCPY_FROM_USER | |
| 117 | + select GENERIC_STRNLEN_USER | |
| 118 | + select GENERIC_TIME_VSYSCALL_OLD | |
| 119 | + select HAVE_ARCH_AUDITSYSCALL | |
| 120 | + select HAVE_ARCH_HARDENED_USERCOPY | |
| 121 | + select HAVE_ARCH_JUMP_LABEL | |
| 122 | + select HAVE_ARCH_KGDB | |
| 123 | + select HAVE_ARCH_SECCOMP_FILTER | |
| 124 | + select HAVE_ARCH_TRACEHOOK | |
| 125 | + select HAVE_CBPF_JIT if !PPC64 | |
| 126 | + select HAVE_CONTEXT_TRACKING if PPC64 | |
| 127 | + select HAVE_DEBUG_KMEMLEAK | |
| 128 | + select HAVE_DEBUG_STACKOVERFLOW | |
| 129 | + select HAVE_DMA_API_DEBUG | |
| 92 | 130 | select HAVE_DYNAMIC_FTRACE |
| 93 | − select HAVE_DYNAMIC_FTRACE_WITH_REGS if MPROFILE_KERNEL | |
| 94 | − select HAVE_FUNCTION_TRACER | |
| 131 | + select HAVE_DYNAMIC_FTRACE_WITH_REGS if MPROFILE_KERNEL | |
| 132 | + select HAVE_EBPF_JIT if PPC64 | |
| 133 | + select HAVE_EFFICIENT_UNALIGNED_ACCESS if !(CPU_LITTLE_ENDIAN && POWER7_CPU) | |
| 134 | + select HAVE_FTRACE_MCOUNT_RECORD | |
| 95 | 135 | select HAVE_FUNCTION_GRAPH_TRACER |
| 136 | + select HAVE_FUNCTION_TRACER | |
| 96 | 137 | select HAVE_GCC_PLUGINS |
| 97 | − select SYSCTL_EXCEPTION_TRACE | |
| 98 | − select VIRT_TO_BUS if !PPC64 | |
| 138 | + select HAVE_GENERIC_RCU_GUP | |
| 139 | + select HAVE_HW_BREAKPOINT if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx) | |
| 99 | 140 | select HAVE_IDE |
| 100 | 141 | select HAVE_IOREMAP_PROT |
| 101 | − select HAVE_EFFICIENT_UNALIGNED_ACCESS if !(CPU_LITTLE_ENDIAN && POWER7_CPU) | |
| 142 | + select HAVE_IRQ_EXIT_ON_IRQ_STACK | |
| 143 | + select HAVE_KERNEL_GZIP | |
| 102 | 144 | select HAVE_KPROBES |
| 103 | − select HAVE_OPTPROBES if PPC64 | |
| 104 | − select HAVE_ARCH_KGDB | |
| 105 | 145 | select HAVE_KRETPROBES |
| 106 | − select HAVE_ARCH_TRACEHOOK | |
| 146 | + select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS | |
| 107 | 147 | select HAVE_MEMBLOCK |
| 108 | 148 | select HAVE_MEMBLOCK_NODE_MAP |
| 109 | − select HAVE_DMA_API_DEBUG | |
| 149 | + select HAVE_MOD_ARCH_SPECIFIC | |
| 150 | + select HAVE_NMI if PERF_EVENTS | |
| 110 | 151 | select HAVE_OPROFILE |
| 111 | − select HAVE_DEBUG_KMEMLEAK | |
| 112 | − select ARCH_HAS_SG_CHAIN | |
| 113 | − select GENERIC_ATOMIC64 if PPC32 | |
| 152 | + select HAVE_OPTPROBES if PPC64 | |
| 114 | 153 | select HAVE_PERF_EVENTS |
| 154 | + select HAVE_PERF_EVENTS_NMI if PPC64 | |
| 115 | 155 | select HAVE_PERF_REGS |
| 116 | 156 | select HAVE_PERF_USER_STACK_DUMP |
| 157 | + select HAVE_RCU_TABLE_FREE if SMP | |
| 117 | 158 | select HAVE_REGS_AND_STACK_ACCESS_API |
| 118 | − select HAVE_HW_BREAKPOINT if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx) | |
| 119 | − select ARCH_WANT_IPC_PARSE_VERSION | |
| 120 | − select SPARSE_IRQ | |
| 159 | + select HAVE_SYSCALL_TRACEPOINTS | |
| 160 | + select HAVE_VIRT_CPU_ACCOUNTING | |
| 121 | 161 | select IRQ_DOMAIN |
| 122 | − select GENERIC_IRQ_SHOW | |
| 123 | − select GENERIC_IRQ_SHOW_LEVEL | |
| 124 | 162 | select IRQ_FORCED_THREADING |
| 125 | − select HAVE_RCU_TABLE_FREE if SMP | |
| 126 | − select HAVE_SYSCALL_TRACEPOINTS | |
| 127 | − select HAVE_CBPF_JIT if !PPC64 | |
| 128 | − select HAVE_EBPF_JIT if PPC64 | |
| 129 | − select HAVE_ARCH_JUMP_LABEL | |
| 130 | − select ARCH_HAVE_NMI_SAFE_CMPXCHG | |
| 131 | − select ARCH_HAS_GCOV_PROFILE_ALL | |
| 132 | − select GENERIC_SMP_IDLE_THREAD | |
| 133 | − select GENERIC_CMOS_UPDATE | |
| 134 | − select GENERIC_TIME_VSYSCALL_OLD | |
| 135 | − select GENERIC_CLOCKEVENTS | |
| 136 | − select GENERIC_CLOCKEVENTS_BROADCAST if SMP | |
| 137 | − select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST | |
| 138 | − select GENERIC_STRNCPY_FROM_USER | |
| 139 | − select GENERIC_STRNLEN_USER | |
| 140 | − select HAVE_MOD_ARCH_SPECIFIC | |
| 141 | 163 | select MODULES_USE_ELF_RELA |
| 142 | − select CLONE_BACKWARDS | |
| 143 | − select ARCH_USE_BUILTIN_BSWAP | |
| 144 | − select OLD_SIGSUSPEND | |
| 145 | − select OLD_SIGACTION if PPC32 | |
| 146 | − select HAVE_DEBUG_STACKOVERFLOW | |
| 147 | − select HAVE_IRQ_EXIT_ON_IRQ_STACK | |
| 148 | − select ARCH_USE_CMPXCHG_LOCKREF if PPC64 | |
| 149 | − select HAVE_ARCH_AUDITSYSCALL | |
| 150 | − select ARCH_SUPPORTS_ATOMIC_RMW | |
| 151 | − select DCACHE_WORD_ACCESS if PPC64 && CPU_LITTLE_ENDIAN | |
| 152 | 164 | select NO_BOOTMEM |
| 153 | − select HAVE_GENERIC_RCU_GUP | |
| 154 | − select HAVE_PERF_EVENTS_NMI if PPC64 | |
| 155 | − select HAVE_NMI if PERF_EVENTS | |
| 156 | − select EDAC_SUPPORT | |
| 157 | − select EDAC_ATOMIC_SCRUB | |
| 158 | − select ARCH_HAS_DMA_SET_COHERENT_MASK | |
| 159 | − select ARCH_HAS_DEVMEM_IS_ALLOWED | |
| 160 | − select HAVE_ARCH_SECCOMP_FILTER | |
| 161 | − select ARCH_HAS_UBSAN_SANITIZE_ALL | |
| 162 | − select ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT | |
| 163 | − select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS | |
| 164 | − select GENERIC_CPU_AUTOPROBE | |
| 165 | − select HAVE_VIRT_CPU_ACCOUNTING | |
| 166 | − select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE | |
| 167 | − select HAVE_ARCH_HARDENED_USERCOPY | |
| 168 | − select HAVE_KERNEL_GZIP | |
| 169 | − select HAVE_CONTEXT_TRACKING if PPC64 | |
| 165 | + select OF | |
| 166 | + select OF_EARLY_FLATTREE | |
| 167 | + select OF_RESERVED_MEM | |
| 168 | + select OLD_SIGACTION if PPC32 | |
| 169 | + select OLD_SIGSUSPEND | |
| 170 | + select SPARSE_IRQ | |
| 171 | + select SYSCTL_EXCEPTION_TRACE | |
| 172 | + select VIRT_TO_BUS if !PPC64 | |
| 173 | + # | |
| 174 | + # Please keep this list sorted alphabetically. | |
| 175 | + # | |
| 170 | 176 | |
| 171 | 177 | config GENERIC_CSUM |
| 172 | 178 | def_bool n |
| @@ -72,8 +72,15 @@ GNUTARGET := powerpc · 71 unmodified lines | ||
| 72 | 72 | MULTIPLEWORD := -mmultiple |
| 73 | 73 | endif |
| 74 | 74 | |
| 75 | −cflags-$(CONFIG_CPU_BIG_ENDIAN) += $(call cc-option,-mbig-endian) | |
| 75 | +ifdef CONFIG_PPC64 | |
| 76 | +cflags-$(CONFIG_CPU_BIG_ENDIAN) += $(call cc-option,-mabi=elfv1) | |
| 77 | +cflags-$(CONFIG_CPU_BIG_ENDIAN) += $(call cc-option,-mcall-aixdesc) | |
| 78 | +aflags-$(CONFIG_CPU_BIG_ENDIAN) += $(call cc-option,-mabi=elfv1) | |
| 79 | +aflags-$(CONFIG_CPU_LITTLE_ENDIAN) += -mabi=elfv2 | |
| 80 | +endif | |
| 81 | + | |
| 76 | 82 | cflags-$(CONFIG_CPU_LITTLE_ENDIAN) += -mlittle-endian |
| 83 | +cflags-$(CONFIG_CPU_BIG_ENDIAN) += $(call cc-option,-mbig-endian) | |
| 77 | 84 | ifneq ($(cc-name),clang) |
| 78 | 85 | cflags-$(CONFIG_CPU_LITTLE_ENDIAN) += -mno-strict-align |
| 79 | 86 | endif |
| @@ -113,7 +120,9 @@ ifeq ($(CONFIG_CPU_LITTLE_ENDIAN),y) · 33 unmodified lines | ||
| 113 | 120 | CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv2,$(call cc-option,-mcall-aixdesc)) |
| 114 | 121 | AFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv2) |
| 115 | 122 | else |
| 123 | +CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv1) | |
| 116 | 124 | CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mcall-aixdesc) |
| 125 | +AFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv1) | |
| 117 | 126 | endif |
| 118 | 127 | CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mcmodel=medium,$(call cc-option,-mminimal-toc)) |
| 119 | 128 | CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mno-pointers-to-nested-functions) |
| @@ -68,6 +68,7 @@ SECTIONS · 67 unmodified lines | ||
| 68 | 68 | } |
| 69 | 69 | |
| 70 | 70 | #ifdef CONFIG_PPC64_BOOT_WRAPPER |
| 71 | + . = ALIGN(256); | |
| 71 | 72 | .got : |
| 72 | 73 | { |
| 73 | 74 | __toc_start = .; |
| @@ -52,7 +52,7 @@ static int crc32c_vpmsum_cra_init(struct crypto_tfm *tfm) · 51 unmodified lines | ||
| 52 | 52 | { |
| 53 | 53 | u32 *key = crypto_tfm_ctx(tfm); |
| 54 | 54 | |
| 55 | − *key = 0; | |
| 55 | + *key = ~0; | |
| 56 | 56 | |
| 57 | 57 | return 0; |
| 58 | 58 | } |
| @@ -51,6 +51,10 @@ · 50 unmodified lines | ||
| 51 | 51 | #define PPC_BIT(bit) (1UL << PPC_BITLSHIFT(bit)) |
| 52 | 52 | #define PPC_BITMASK(bs, be) ((PPC_BIT(bs) - PPC_BIT(be)) | PPC_BIT(bs)) |
| 53 | 53 | |
| 54 | +/* Put a PPC bit into a "normal" bit position */ | |
| 55 | +#define PPC_BITEXTRACT(bits, ppc_bit, dst_bit) \ | |
| 56 | + ((((bits) >> PPC_BITLSHIFT(ppc_bit)) & 1) << (dst_bit)) | |
| 57 | + | |
| 54 | 58 | #include <asm/barrier.h> |
| 55 | 59 | |
| 56 | 60 | /* Macro for generating the ***_bits() functions */ |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | #ifndef _ASM_POWERPC_BOOK3S_32_PGTABLE_H |
| 2 | 2 | #define _ASM_POWERPC_BOOK3S_32_PGTABLE_H |
| 3 | 3 | |
| 4 | +#define __ARCH_USE_5LEVEL_HACK | |
| 4 | 5 | #include <asm-generic/pgtable-nopmd.h> |
| 5 | 6 | |
| 6 | 7 | #include <asm/book3s/32/hash.h> |
| @@ -1,9 +1,12 @@ | ||
| 1 | 1 | #ifndef _ASM_POWERPC_BOOK3S_64_PGTABLE_H_ |
| 2 | 2 | #define _ASM_POWERPC_BOOK3S_64_PGTABLE_H_ |
| 3 | 3 | |
| 4 | +#include <asm-generic/5level-fixup.h> | |
| 5 | + | |
| 4 | 6 | #ifndef __ASSEMBLY__ |
| 5 | 7 | #include <linux/mmdebug.h> |
| 6 | 8 | #endif |
| 9 | + | |
| 7 | 10 | /* |
| 8 | 11 | * Common bits between hash and Radix page table |
| 9 | 12 | */ |
| @@ -347,23 +350,58 @@ static inline int __ptep_test_and_clear_young(struct mm_struct *mm, · 337 unmodified lines | ||
| 347 | 350 | __r; \ |
| 348 | 351 | }) |
| 349 | 352 | |
| 353 | +static inline int __pte_write(pte_t pte) | |
| 354 | +{ | |
| 355 | + return !!(pte_raw(pte) & cpu_to_be64(_PAGE_WRITE)); | |
| 356 | +} | |
| 357 | + | |
| 358 | +#ifdef CONFIG_NUMA_BALANCING | |
| 359 | +#define pte_savedwrite pte_savedwrite | |
| 360 | +static inline bool pte_savedwrite(pte_t pte) | |
| 361 | +{ | |
| 362 | + /* | |
| 363 | + * Saved write ptes are prot none ptes that doesn't have | |
| 364 | + * privileged bit sit. We mark prot none as one which has | |
| 365 | + * present and pviliged bit set and RWX cleared. To mark | |
| 366 | + * protnone which used to have _PAGE_WRITE set we clear | |
| 367 | + * the privileged bit. | |
| 368 | + */ | |
| 369 | + return !(pte_raw(pte) & cpu_to_be64(_PAGE_RWX | _PAGE_PRIVILEGED)); | |
| 370 | +} | |
| 371 | +#else | |
| 372 | +#define pte_savedwrite pte_savedwrite | |
| 373 | +static inline bool pte_savedwrite(pte_t pte) | |
| 374 | +{ | |
| 375 | + return false; | |
| 376 | +} | |
| 377 | +#endif | |
| 378 | + | |
| 379 | +static inline int pte_write(pte_t pte) | |
| 380 | +{ | |
| 381 | + return __pte_write(pte) || pte_savedwrite(pte); | |
| 382 | +} | |
| 383 | + | |
| 350 | 384 | #define __HAVE_ARCH_PTEP_SET_WRPROTECT |
| 351 | 385 | static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, |
| 352 | 386 | pte_t *ptep) |
| 353 | 387 | { |
| 354 | − if ((pte_raw(*ptep) & cpu_to_be64(_PAGE_WRITE)) == 0) | |
| 355 | − return; | |
| 356 | − | |
| 357 | − pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 0); | |
| 388 | + if (__pte_write(*ptep)) | |
| 389 | + pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 0); | |
| 390 | + else if (unlikely(pte_savedwrite(*ptep))) | |
| 391 | + pte_update(mm, addr, ptep, 0, _PAGE_PRIVILEGED, 0); | |
| 358 | 392 | } |
| 359 | 393 | |
| 360 | 394 | static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, |
| 361 | 395 | unsigned long addr, pte_t *ptep) |
| 362 | 396 | { |
| 363 | − if ((pte_raw(*ptep) & cpu_to_be64(_PAGE_WRITE)) == 0) | |
| 364 | − return; | |
| 365 | − | |
| 366 | − pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 1); | |
| 397 | + /* | |
| 398 | + * We should not find protnone for hugetlb, but this complete the | |
| 399 | + * interface. | |
| 400 | + */ | |
| 401 | + if (__pte_write(*ptep)) | |
| 402 | + pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 1); | |
| 403 | + else if (unlikely(pte_savedwrite(*ptep))) | |
| 404 | + pte_update(mm, addr, ptep, 0, _PAGE_PRIVILEGED, 1); | |
| 367 | 405 | } |
| 368 | 406 | |
| 369 | 407 | #define __HAVE_ARCH_PTEP_GET_AND_CLEAR |
| @@ -397,11 +435,6 @@ static inline void pte_clear(struct mm_struct *mm, unsigned long addr, · 27 unmodified lines | ||
| 397 | 435 | pte_update(mm, addr, ptep, ~0UL, 0, 0); |
| 398 | 436 | } |
| 399 | 437 | |
| 400 | −static inline int pte_write(pte_t pte) | |
| 401 | −{ | |
| 402 | − return !!(pte_raw(pte) & cpu_to_be64(_PAGE_WRITE)); | |
| 403 | −} | |
| 404 | − | |
| 405 | 438 | static inline int pte_dirty(pte_t pte) |
| 406 | 439 | { |
| 407 | 440 | return !!(pte_raw(pte) & cpu_to_be64(_PAGE_DIRTY)); |
| @@ -465,19 +498,12 @@ static inline pte_t pte_clear_savedwrite(pte_t pte) · 57 unmodified lines | ||
| 465 | 498 | VM_BUG_ON(!pte_protnone(pte)); |
| 466 | 499 | return __pte(pte_val(pte) | _PAGE_PRIVILEGED); |
| 467 | 500 | } |
| 468 | − | |
| 469 | −#define pte_savedwrite pte_savedwrite | |
| 470 | −static inline bool pte_savedwrite(pte_t pte) | |
| 501 | +#else | |
| 502 | +#define pte_clear_savedwrite pte_clear_savedwrite | |
| 503 | +static inline pte_t pte_clear_savedwrite(pte_t pte) | |
| 471 | 504 | { |
| 472 | − /* | |
| 473 | − * Saved write ptes are prot none ptes that doesn't have | |
| 474 | − * privileged bit sit. We mark prot none as one which has | |
| 475 | − * present and pviliged bit set and RWX cleared. To mark | |
| 476 | − * protnone which used to have _PAGE_WRITE set we clear | |
| 477 | − * the privileged bit. | |
| 478 | − */ | |
| 479 | − VM_BUG_ON(!pte_protnone(pte)); | |
| 480 | − return !(pte_raw(pte) & cpu_to_be64(_PAGE_RWX | _PAGE_PRIVILEGED)); | |
| 505 | + VM_WARN_ON(1); | |
| 506 | + return __pte(pte_val(pte) & ~_PAGE_WRITE); | |
| 481 | 507 | } |
| 482 | 508 | #endif /* CONFIG_NUMA_BALANCING */ |
| 483 | 509 | |
| @@ -506,6 +532,8 @@ static inline unsigned long pte_pfn(pte_t pte) · 22 unmodified lines | ||
| 506 | 532 | /* Generic modifiers for PTE bits */ |
| 507 | 533 | static inline pte_t pte_wrprotect(pte_t pte) |
| 508 | 534 | { |
| 535 | + if (unlikely(pte_savedwrite(pte))) | |
| 536 | + return pte_clear_savedwrite(pte); | |
| 509 | 537 | return __pte(pte_val(pte) & ~_PAGE_WRITE); |
| 510 | 538 | } |
| 511 | 539 | |
| @@ -926,6 +954,7 @@ static inline int pmd_protnone(pmd_t pmd) · 414 unmodified lines | ||
| 926 | 954 | |
| 927 | 955 | #define __HAVE_ARCH_PMD_WRITE |
| 928 | 956 | #define pmd_write(pmd) pte_write(pmd_pte(pmd)) |
| 957 | +#define __pmd_write(pmd) __pte_write(pmd_pte(pmd)) | |
| 929 | 958 | #define pmd_savedwrite(pmd) pte_savedwrite(pmd_pte(pmd)) |
| 930 | 959 | |
| 931 | 960 | #ifdef CONFIG_TRANSPARENT_HUGEPAGE |
| @@ -982,11 +1011,10 @@ static inline int __pmdp_test_and_clear_young(struct mm_struct *mm, · 50 unmodified lines | ||
| 982 | 1011 | static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr, |
| 983 | 1012 | pmd_t *pmdp) |
| 984 | 1013 | { |
| 985 | − | |
| 986 | − if ((pmd_raw(*pmdp) & cpu_to_be64(_PAGE_WRITE)) == 0) | |
| 987 | − return; | |
| 988 | − | |
| 989 | − pmd_hugepage_update(mm, addr, pmdp, _PAGE_WRITE, 0); | |
| 1014 | + if (__pmd_write((*pmdp))) | |
| 1015 | + pmd_hugepage_update(mm, addr, pmdp, _PAGE_WRITE, 0); | |
| 1016 | + else if (unlikely(pmd_savedwrite(*pmdp))) | |
| 1017 | + pmd_hugepage_update(mm, addr, pmdp, 0, _PAGE_PRIVILEGED); | |
| 990 | 1018 | } |
| 991 | 1019 | |
| 992 | 1020 | static inline int pmd_trans_huge(pmd_t pmd) |
| @@ -112,7 +112,7 @@ static inline __wsum csum_add(__wsum csum, __wsum addend) · 111 unmodified lines | ||
| 112 | 112 | |
| 113 | 113 | #ifdef __powerpc64__ |
| 114 | 114 | res += (__force u64)addend; |
| 115 | − return (__force __wsum)((u32)res + (res >> 32)); | |
| 115 | + return (__force __wsum) from64to32(res); | |
| 116 | 116 | #else |
| 117 | 117 | asm("addc %0,%0,%1;" |
| 118 | 118 | "addze %0,%0;" |
| @@ -70,8 +70,8 @@ static inline void report_invalid_psscr_val(u64 psscr_val, int err) · 69 unmodified lines | ||
| 70 | 70 | std r0,0(r1); \ |
| 71 | 71 | ptesync; \ |
| 72 | 72 | ld r0,0(r1); \ |
| 73 | −1: cmpd cr0,r0,r0; \ | |
| 74 | − bne 1b; \ | |
| 73 | +236: cmpd cr0,r0,r0; \ | |
| 74 | + bne 236b; \ | |
| 75 | 75 | IDLE_INST; \ |
| 76 | 76 | |
| 77 | 77 | #define IDLE_STATE_ENTER_SEQ_NORET(IDLE_INST) \ |
| @@ -144,8 +144,8 @@ extern int arch_setup_additional_pages(struct linux_binprm *bprm, · 143 unmodified lines | ||
| 144 | 144 | #define ARCH_DLINFO_CACHE_GEOMETRY \ |
| 145 | 145 | NEW_AUX_ENT(AT_L1I_CACHESIZE, ppc64_caches.l1i.size); \ |
| 146 | 146 | NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY, get_cache_geometry(l1i)); \ |
| 147 | − NEW_AUX_ENT(AT_L1D_CACHESIZE, ppc64_caches.l1i.size); \ | |
| 148 | − NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, get_cache_geometry(l1i)); \ | |
| 147 | + NEW_AUX_ENT(AT_L1D_CACHESIZE, ppc64_caches.l1d.size); \ | |
| 148 | + NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, get_cache_geometry(l1d)); \ | |
| 149 | 149 | NEW_AUX_ENT(AT_L2_CACHESIZE, ppc64_caches.l2.size); \ |
| 150 | 150 | NEW_AUX_ENT(AT_L2_CACHEGEOMETRY, get_cache_geometry(l2)); \ |
| 151 | 151 | NEW_AUX_ENT(AT_L3_CACHESIZE, ppc64_caches.l3.size); \ |
| @@ -66,6 +66,55 @@ · 65 unmodified lines | ||
| 66 | 66 | |
| 67 | 67 | #define P8_DSISR_MC_SLB_ERRORS (P7_DSISR_MC_SLB_ERRORS | \ |
| 68 | 68 | P8_DSISR_MC_ERAT_MULTIHIT_SEC) |
| 69 | + | |
| 70 | +/* | |
| 71 | + * Machine Check bits on power9 | |
| 72 | + */ | |
| 73 | +#define P9_SRR1_MC_LOADSTORE(srr1) (((srr1) >> PPC_BITLSHIFT(42)) & 1) | |
| 74 | + | |
| 75 | +#define P9_SRR1_MC_IFETCH(srr1) ( \ | |
| 76 | + PPC_BITEXTRACT(srr1, 45, 0) | \ | |
| 77 | + PPC_BITEXTRACT(srr1, 44, 1) | \ | |
| 78 | + PPC_BITEXTRACT(srr1, 43, 2) | \ | |
| 79 | + PPC_BITEXTRACT(srr1, 36, 3) ) | |
| 80 | + | |
| 81 | +/* 0 is reserved */ | |
| 82 | +#define P9_SRR1_MC_IFETCH_UE 1 | |
| 83 | +#define P9_SRR1_MC_IFETCH_SLB_PARITY 2 | |
| 84 | +#define P9_SRR1_MC_IFETCH_SLB_MULTIHIT 3 | |
| 85 | +#define P9_SRR1_MC_IFETCH_ERAT_MULTIHIT 4 | |
| 86 | +#define P9_SRR1_MC_IFETCH_TLB_MULTIHIT 5 | |
| 87 | +#define P9_SRR1_MC_IFETCH_UE_TLB_RELOAD 6 | |
| 88 | +/* 7 is reserved */ | |
| 89 | +#define P9_SRR1_MC_IFETCH_LINK_TIMEOUT 8 | |
| 90 | +#define P9_SRR1_MC_IFETCH_LINK_TABLEWALK_TIMEOUT 9 | |
| 91 | +/* 10 ? */ | |
| 92 | +#define P9_SRR1_MC_IFETCH_RA 11 | |
| 93 | +#define P9_SRR1_MC_IFETCH_RA_TABLEWALK 12 | |
| 94 | +#define P9_SRR1_MC_IFETCH_RA_ASYNC_STORE 13 | |
| 95 | +#define P9_SRR1_MC_IFETCH_LINK_ASYNC_STORE_TIMEOUT 14 | |
| 96 | +#define P9_SRR1_MC_IFETCH_RA_TABLEWALK_FOREIGN 15 | |
| 97 | + | |
| 98 | +/* DSISR bits for machine check (On Power9) */ | |
| 99 | +#define P9_DSISR_MC_UE (PPC_BIT(48)) | |
| 100 | +#define P9_DSISR_MC_UE_TABLEWALK (PPC_BIT(49)) | |
| 101 | +#define P9_DSISR_MC_LINK_LOAD_TIMEOUT (PPC_BIT(50)) | |
| 102 | +#define P9_DSISR_MC_LINK_TABLEWALK_TIMEOUT (PPC_BIT(51)) | |
| 103 | +#define P9_DSISR_MC_ERAT_MULTIHIT (PPC_BIT(52)) | |
| 104 | +#define P9_DSISR_MC_TLB_MULTIHIT_MFTLB (PPC_BIT(53)) | |
| 105 | +#define P9_DSISR_MC_USER_TLBIE (PPC_BIT(54)) | |
| 106 | +#define P9_DSISR_MC_SLB_PARITY_MFSLB (PPC_BIT(55)) | |
| 107 | +#define P9_DSISR_MC_SLB_MULTIHIT_MFSLB (PPC_BIT(56)) | |
| 108 | +#define P9_DSISR_MC_RA_LOAD (PPC_BIT(57)) | |
| 109 | +#define P9_DSISR_MC_RA_TABLEWALK (PPC_BIT(58)) | |
| 110 | +#define P9_DSISR_MC_RA_TABLEWALK_FOREIGN (PPC_BIT(59)) | |
| 111 | +#define P9_DSISR_MC_RA_FOREIGN (PPC_BIT(60)) | |
| 112 | + | |
| 113 | +/* SLB error bits */ | |
| 114 | +#define P9_DSISR_MC_SLB_ERRORS (P9_DSISR_MC_ERAT_MULTIHIT | \ | |
| 115 | + P9_DSISR_MC_SLB_PARITY_MFSLB | \ | |
| 116 | + P9_DSISR_MC_SLB_MULTIHIT_MFSLB) | |
| 117 | + | |
| 69 | 118 | enum MCE_Version { |
| 70 | 119 | MCE_V1 = 1, |
| 71 | 120 | }; |
| @@ -93,6 +142,9 @@ enum MCE_ErrorType { · 21 unmodified lines | ||
| 93 | 142 | MCE_ERROR_TYPE_SLB = 2, |
| 94 | 143 | MCE_ERROR_TYPE_ERAT = 3, |
| 95 | 144 | MCE_ERROR_TYPE_TLB = 4, |
| 145 | + MCE_ERROR_TYPE_USER = 5, | |
| 146 | + MCE_ERROR_TYPE_RA = 6, | |
| 147 | + MCE_ERROR_TYPE_LINK = 7, | |
| 96 | 148 | }; |
| 97 | 149 | |
| 98 | 150 | enum MCE_UeErrorType { |
| @@ -121,6 +173,32 @@ enum MCE_TlbErrorType { · 22 unmodified lines | ||
| 121 | 173 | MCE_TLB_ERROR_MULTIHIT = 2, |
| 122 | 174 | }; |
| 123 | 175 | |
| 176 | +enum MCE_UserErrorType { | |
| 177 | + MCE_USER_ERROR_INDETERMINATE = 0, | |
| 178 | + MCE_USER_ERROR_TLBIE = 1, | |
| 179 | +}; | |
| 180 | + | |
| 181 | +enum MCE_RaErrorType { | |
| 182 | + MCE_RA_ERROR_INDETERMINATE = 0, | |
| 183 | + MCE_RA_ERROR_IFETCH = 1, | |
| 184 | + MCE_RA_ERROR_PAGE_TABLE_WALK_IFETCH = 2, | |
| 185 | + MCE_RA_ERROR_PAGE_TABLE_WALK_IFETCH_FOREIGN = 3, | |
| 186 | + MCE_RA_ERROR_LOAD = 4, | |
| 187 | + MCE_RA_ERROR_STORE = 5, | |
| 188 | + MCE_RA_ERROR_PAGE_TABLE_WALK_LOAD_STORE = 6, | |
| 189 | + MCE_RA_ERROR_PAGE_TABLE_WALK_LOAD_STORE_FOREIGN = 7, | |
| 190 | + MCE_RA_ERROR_LOAD_STORE_FOREIGN = 8, | |
| 191 | +}; | |
| 192 | + | |
| 193 | +enum MCE_LinkErrorType { | |
| 194 | + MCE_LINK_ERROR_INDETERMINATE = 0, | |
| 195 | + MCE_LINK_ERROR_IFETCH_TIMEOUT = 1, | |
| 196 | + MCE_LINK_ERROR_PAGE_TABLE_WALK_IFETCH_TIMEOUT = 2, | |
| 197 | + MCE_LINK_ERROR_LOAD_TIMEOUT = 3, | |
| 198 | + MCE_LINK_ERROR_STORE_TIMEOUT = 4, | |
| 199 | + MCE_LINK_ERROR_PAGE_TABLE_WALK_LOAD_STORE_TIMEOUT = 5, | |
| 200 | +}; | |
| 201 | + | |
| 124 | 202 | struct machine_check_event { |
| 125 | 203 | enum MCE_Version version:8; /* 0x00 */ |
| 126 | 204 | uint8_t in_use; /* 0x01 */ |
| @@ -166,6 +244,30 @@ struct machine_check_event { · 39 unmodified lines | ||
| 166 | 244 | uint64_t effective_address; |
| 167 | 245 | uint8_t reserved_2[16]; |
| 168 | 246 | } tlb_error; |
| 247 | + | |
| 248 | + struct { | |
| 249 | + enum MCE_UserErrorType user_error_type:8; | |
| 250 | + uint8_t effective_address_provided; | |
| 251 | + uint8_t reserved_1[6]; | |
| 252 | + uint64_t effective_address; | |
| 253 | + uint8_t reserved_2[16]; | |
| 254 | + } user_error; | |
| 255 | + | |
| 256 | + struct { | |
| 257 | + enum MCE_RaErrorType ra_error_type:8; | |
| 258 | + uint8_t effective_address_provided; | |
| 259 | + uint8_t reserved_1[6]; | |
| 260 | + uint64_t effective_address; | |
| 261 | + uint8_t reserved_2[16]; | |
| 262 | + } ra_error; | |
| 263 | + | |
| 264 | + struct { | |
| 265 | + enum MCE_LinkErrorType link_error_type:8; | |
| 266 | + uint8_t effective_address_provided; | |
| 267 | + uint8_t reserved_1[6]; | |
| 268 | + uint64_t effective_address; | |
| 269 | + uint8_t reserved_2[16]; | |
| 270 | + } link_error; | |
| 169 | 271 | } u; |
| 170 | 272 | }; |
| 171 | 273 | |
| @@ -176,8 +278,12 @@ struct mce_error_info { · 4 unmodified lines | ||
| 176 | 278 | enum MCE_SlbErrorType slb_error_type:8; |
| 177 | 279 | enum MCE_EratErrorType erat_error_type:8; |
| 178 | 280 | enum MCE_TlbErrorType tlb_error_type:8; |
| 281 | + enum MCE_UserErrorType user_error_type:8; | |
| 282 | + enum MCE_RaErrorType ra_error_type:8; | |
| 283 | + enum MCE_LinkErrorType link_error_type:8; | |
| 179 | 284 | } u; |
| 180 | − uint8_t reserved[2]; | |
| 285 | + enum MCE_Severity severity:8; | |
| 286 | + enum MCE_Initiator initiator:8; | |
| 181 | 287 | }; |
| 182 | 288 | |
| 183 | 289 | #define MAX_MC_EVT 100 |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | #ifndef _ASM_POWERPC_NOHASH_32_PGTABLE_H |
| 2 | 2 | #define _ASM_POWERPC_NOHASH_32_PGTABLE_H |
| 3 | 3 | |
| 4 | +#define __ARCH_USE_5LEVEL_HACK | |
| 4 | 5 | #include <asm-generic/pgtable-nopmd.h> |
| 5 | 6 | |
| 6 | 7 | #ifndef __ASSEMBLY__ |
| @@ -1,5 +1,8 @@ | ||
| 1 | 1 | #ifndef _ASM_POWERPC_NOHASH_64_PGTABLE_4K_H |
| 2 | 2 | #define _ASM_POWERPC_NOHASH_64_PGTABLE_4K_H |
| 3 | + | |
| 4 | +#include <asm-generic/5level-fixup.h> | |
| 5 | + | |
| 3 | 6 | /* |
| 4 | 7 | * Entries per page directory level. The PTE level must use a 64b record |
| 5 | 8 | * for each page table entry. The PMD and PGD level use a 32b record for |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | #ifndef _ASM_POWERPC_NOHASH_64_PGTABLE_64K_H |
| 2 | 2 | #define _ASM_POWERPC_NOHASH_64_PGTABLE_64K_H |
| 3 | 3 | |
| 4 | +#define __ARCH_USE_5LEVEL_HACK | |
| 4 | 5 | #include <asm-generic/pgtable-nopud.h> |
| 5 | 6 | |
| 6 | 7 | |
| @@ -230,7 +230,7 @@ static inline int hugepd_ok(hugepd_t hpd) · 229 unmodified lines | ||
| 230 | 230 | return ((hpd_val(hpd) & 0x4) != 0); |
| 231 | 231 | #else |
| 232 | 232 | /* We clear the top bit to indicate hugepd */ |
| 233 | − return ((hpd_val(hpd) & PD_HUGE) == 0); | |
| 233 | + return (hpd_val(hpd) && (hpd_val(hpd) & PD_HUGE) == 0); | |
| 234 | 234 | #endif |
| 235 | 235 | } |
| 236 | 236 | |
| @@ -284,6 +284,13 @@ · 283 unmodified lines | ||
| 284 | 284 | #define PPC_INST_BRANCH_COND 0x40800000 |
| 285 | 285 | #define PPC_INST_LBZCIX 0x7c0006aa |
| 286 | 286 | #define PPC_INST_STBCIX 0x7c0007aa |
| 287 | +#define PPC_INST_LWZX 0x7c00002e | |
| 288 | +#define PPC_INST_LFSX 0x7c00042e | |
| 289 | +#define PPC_INST_STFSX 0x7c00052e | |
| 290 | +#define PPC_INST_LFDX 0x7c0004ae | |
| 291 | +#define PPC_INST_STFDX 0x7c0005ae | |
| 292 | +#define PPC_INST_LVX 0x7c0000ce | |
| 293 | +#define PPC_INST_STVX 0x7c0001ce | |
| 287 | 294 | |
| 288 | 295 | /* macros to insert fields into opcodes */ |
| 289 | 296 | #define ___PPC_RA(a) (((a) & 0x1f) << 16) |
| @@ -160,12 +160,18 @@ struct of_drconf_cell { · 159 unmodified lines | ||
| 160 | 160 | #define OV5_PFO_HW_ENCR 0x1120 /* PFO Encryption Accelerator */ |
| 161 | 161 | #define OV5_SUB_PROCESSORS 0x1501 /* 1,2,or 4 Sub-Processors supported */ |
| 162 | 162 | #define OV5_XIVE_EXPLOIT 0x1701 /* XIVE exploitation supported */ |
| 163 | −#define OV5_MMU_RADIX_300 0x1880 /* ISA v3.00 radix MMU supported */ | |
| 164 | −#define OV5_MMU_HASH_300 0x1840 /* ISA v3.00 hash MMU supported */ | |
| 165 | −#define OV5_MMU_SEGM_RADIX 0x1820 /* radix mode (no segmentation) */ | |
| 166 | −#define OV5_MMU_PROC_TBL 0x1810 /* hcall selects SLB or proc table */ | |
| 167 | −#define OV5_MMU_SLB 0x1800 /* always use SLB */ | |
| 168 | −#define OV5_MMU_GTSE 0x1808 /* Guest translation shootdown */ | |
| 163 | +/* MMU Base Architecture */ | |
| 164 | +#define OV5_MMU_SUPPORT 0x18C0 /* MMU Mode Support Mask */ | |
| 165 | +#define OV5_MMU_HASH 0x1800 /* Hash MMU Only */ | |
| 166 | +#define OV5_MMU_RADIX 0x1840 /* Radix MMU Only */ | |
| 167 | +#define OV5_MMU_EITHER 0x1880 /* Hash or Radix Supported */ | |
| 168 | +#define OV5_MMU_DYNAMIC 0x18C0 /* Hash or Radix Can Switch Later */ | |
| 169 | +#define OV5_NMMU 0x1820 /* Nest MMU Available */ | |
| 170 | +/* Hash Table Extensions */ | |
| 171 | +#define OV5_HASH_SEG_TBL 0x1980 /* In Memory Segment Tables Available */ | |
| 172 | +#define OV5_HASH_GTSE 0x1940 /* Guest Translation Shoot Down Avail */ | |
| 173 | +/* Radix Table Extensions */ | |
| 174 | +#define OV5_RADIX_GTSE 0x1A40 /* Guest Translation Shoot Down Avail */ | |
| 169 | 175 | |
| 170 | 176 | /* Option Vector 6: IBM PAPR hints */ |
| 171 | 177 | #define OV6_LINUX 0x02 /* Linux is our OS */ |
| @@ -77,6 +77,7 @@ extern void __flush_tlb_power8(unsigned int action); · 76 unmodified lines | ||
| 77 | 77 | extern void __flush_tlb_power9(unsigned int action); |
| 78 | 78 | extern long __machine_check_early_realmode_p7(struct pt_regs *regs); |
| 79 | 79 | extern long __machine_check_early_realmode_p8(struct pt_regs *regs); |
| 80 | +extern long __machine_check_early_realmode_p9(struct pt_regs *regs); | |
| 80 | 81 | #endif /* CONFIG_PPC64 */ |
| 81 | 82 | #if defined(CONFIG_E500) |
| 82 | 83 | extern void __setup_cpu_e5500(unsigned long offset, struct cpu_spec* spec); |
| @@ -540,6 +541,7 @@ static struct cpu_spec __initdata cpu_specs[] = { · 457 unmodified lines | ||
| 540 | 541 | .cpu_setup = __setup_cpu_power9, |
| 541 | 542 | .cpu_restore = __restore_cpu_power9, |
| 542 | 543 | .flush_tlb = __flush_tlb_power9, |
| 544 | + .machine_check_early = __machine_check_early_realmode_p9, | |
| 543 | 545 | .platform = "power9", |
| 544 | 546 | }, |
| 545 | 547 | { /* Power9 */ |
| @@ -559,6 +561,7 @@ static struct cpu_spec __initdata cpu_specs[] = { · 13 unmodified lines | ||
| 559 | 561 | .cpu_setup = __setup_cpu_power9, |
| 560 | 562 | .cpu_restore = __restore_cpu_power9, |
| 561 | 563 | .flush_tlb = __flush_tlb_power9, |
| 564 | + .machine_check_early = __machine_check_early_realmode_p9, | |
| 562 | 565 | .platform = "power9", |
| 563 | 566 | }, |
| 564 | 567 | { /* Cell Broadband Engine */ |
| @@ -276,19 +276,21 @@ power_enter_stop: · 275 unmodified lines | ||
| 276 | 276 | */ |
| 277 | 277 | andis. r4,r3,PSSCR_EC_ESL_MASK_SHIFTED |
| 278 | 278 | clrldi r3,r3,60 /* r3 = Bits[60:63] = Requested Level (RL) */ |
| 279 | − bne 1f | |
| 279 | + bne .Lhandle_esl_ec_set | |
| 280 | 280 | IDLE_STATE_ENTER_SEQ(PPC_STOP) |
| 281 | 281 | li r3,0 /* Since we didn't lose state, return 0 */ |
| 282 | 282 | b pnv_wakeup_noloss |
| 283 | + | |
| 284 | +.Lhandle_esl_ec_set: | |
| 283 | 285 | /* |
| 284 | 286 | * Check if the requested state is a deep idle state. |
| 285 | 287 | */ |
| 286 | −1: LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state) | |
| 288 | + LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state) | |
| 287 | 289 | ld r4,ADDROFF(pnv_first_deep_stop_state)(r5) |
| 288 | 290 | cmpd r3,r4 |
| 289 | − bge 2f | |
| 291 | + bge .Lhandle_deep_stop | |
| 290 | 292 | IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP) |
| 291 | −2: | |
| 293 | +.Lhandle_deep_stop: | |
| 292 | 294 | /* |
| 293 | 295 | * Entering deep idle state. |
| 294 | 296 | * Clear thread bit in PACA_CORE_IDLE_STATE, save SPRs to |
| @@ -58,6 +58,15 @@ static void mce_set_error_info(struct machine_check_event *mce, · 57 unmodified lines | ||
| 58 | 58 | case MCE_ERROR_TYPE_TLB: |
| 59 | 59 | mce->u.tlb_error.tlb_error_type = mce_err->u.tlb_error_type; |
| 60 | 60 | break; |
| 61 | + case MCE_ERROR_TYPE_USER: | |
| 62 | + mce->u.user_error.user_error_type = mce_err->u.user_error_type; | |
| 63 | + break; | |
| 64 | + case MCE_ERROR_TYPE_RA: | |
| 65 | + mce->u.ra_error.ra_error_type = mce_err->u.ra_error_type; | |
| 66 | + break; | |
| 67 | + case MCE_ERROR_TYPE_LINK: | |
| 68 | + mce->u.link_error.link_error_type = mce_err->u.link_error_type; | |
| 69 | + break; | |
| 61 | 70 | case MCE_ERROR_TYPE_UNKNOWN: |
| 62 | 71 | default: |
| 63 | 72 | break; |
| @@ -90,13 +99,14 @@ void save_mce_event(struct pt_regs *regs, long handled, · 26 unmodified lines | ||
| 90 | 99 | mce->gpr3 = regs->gpr[3]; |
| 91 | 100 | mce->in_use = 1; |
| 92 | 101 | |
| 93 | − mce->initiator = MCE_INITIATOR_CPU; | |
| 94 | 102 | /* Mark it recovered if we have handled it and MSR(RI=1). */ |
| 95 | 103 | if (handled && (regs->msr & MSR_RI)) |
| 96 | 104 | mce->disposition = MCE_DISPOSITION_RECOVERED; |
| 97 | 105 | else |
| 98 | 106 | mce->disposition = MCE_DISPOSITION_NOT_RECOVERED; |
| 99 | − mce->severity = MCE_SEV_ERROR_SYNC; | |
| 107 | + | |
| 108 | + mce->initiator = mce_err->initiator; | |
| 109 | + mce->severity = mce_err->severity; | |
| 100 | 110 | |
| 101 | 111 | /* |
| 102 | 112 | * Populate the mce error_type and type-specific error_type. |
| @@ -115,6 +125,15 @@ void save_mce_event(struct pt_regs *regs, long handled, · 12 unmodified lines | ||
| 115 | 125 | } else if (mce->error_type == MCE_ERROR_TYPE_ERAT) { |
| 116 | 126 | mce->u.erat_error.effective_address_provided = true; |
| 117 | 127 | mce->u.erat_error.effective_address = addr; |
| 128 | + } else if (mce->error_type == MCE_ERROR_TYPE_USER) { | |
| 129 | + mce->u.user_error.effective_address_provided = true; | |
| 130 | + mce->u.user_error.effective_address = addr; | |
| 131 | + } else if (mce->error_type == MCE_ERROR_TYPE_RA) { | |
| 132 | + mce->u.ra_error.effective_address_provided = true; | |
| 133 | + mce->u.ra_error.effective_address = addr; | |
| 134 | + } else if (mce->error_type == MCE_ERROR_TYPE_LINK) { | |
| 135 | + mce->u.link_error.effective_address_provided = true; | |
| 136 | + mce->u.link_error.effective_address = addr; | |
| 118 | 137 | } else if (mce->error_type == MCE_ERROR_TYPE_UE) { |
| 119 | 138 | mce->u.ue_error.effective_address_provided = true; |
| 120 | 139 | mce->u.ue_error.effective_address = addr; |
| @@ -239,6 +258,29 @@ void machine_check_print_event_info(struct machine_check_event *evt) · 118 unmodified lines | ||
| 239 | 258 | "Parity", |
| 240 | 259 | "Multihit", |
| 241 | 260 | }; |
| 261 | + static const char *mc_user_types[] = { | |
| 262 | + "Indeterminate", | |
| 263 | + "tlbie(l) invalid", | |
| 264 | + }; | |
| 265 | + static const char *mc_ra_types[] = { | |
| 266 | + "Indeterminate", | |
| 267 | + "Instruction fetch (bad)", | |
| 268 | + "Page table walk ifetch (bad)", | |
| 269 | + "Page table walk ifetch (foreign)", | |
| 270 | + "Load (bad)", | |
| 271 | + "Store (bad)", | |
| 272 | + "Page table walk Load/Store (bad)", | |
| 273 | + "Page table walk Load/Store (foreign)", | |
| 274 | + "Load/Store (foreign)", | |
| 275 | + }; | |
| 276 | + static const char *mc_link_types[] = { | |
| 277 | + "Indeterminate", | |
| 278 | + "Instruction fetch (timeout)", | |
| 279 | + "Page table walk ifetch (timeout)", | |
| 280 | + "Load (timeout)", | |
| 281 | + "Store (timeout)", | |
| 282 | + "Page table walk Load/Store (timeout)", | |
| 283 | + }; | |
| 242 | 284 | |
| 243 | 285 | /* Print things out */ |
| 244 | 286 | if (evt->version != MCE_V1) { |
| @@ -315,6 +357,36 @@ void machine_check_print_event_info(struct machine_check_event *evt) · 70 unmodified lines | ||
| 315 | 357 | printk("%s Effective address: %016llx\n", |
| 316 | 358 | level, evt->u.tlb_error.effective_address); |
| 317 | 359 | break; |
| 360 | + case MCE_ERROR_TYPE_USER: | |
| 361 | + subtype = evt->u.user_error.user_error_type < | |
| 362 | + ARRAY_SIZE(mc_user_types) ? | |
| 363 | + mc_user_types[evt->u.user_error.user_error_type] | |
| 364 | + : "Unknown"; | |
| 365 | + printk("%s Error type: User [%s]\n", level, subtype); | |
| 366 | + if (evt->u.user_error.effective_address_provided) | |
| 367 | + printk("%s Effective address: %016llx\n", | |
| 368 | + level, evt->u.user_error.effective_address); | |
| 369 | + break; | |
| 370 | + case MCE_ERROR_TYPE_RA: | |
| 371 | + subtype = evt->u.ra_error.ra_error_type < | |
| 372 | + ARRAY_SIZE(mc_ra_types) ? | |
| 373 | + mc_ra_types[evt->u.ra_error.ra_error_type] | |
| 374 | + : "Unknown"; | |
| 375 | + printk("%s Error type: Real address [%s]\n", level, subtype); | |
| 376 | + if (evt->u.ra_error.effective_address_provided) | |
| 377 | + printk("%s Effective address: %016llx\n", | |
| 378 | + level, evt->u.ra_error.effective_address); | |
| 379 | + break; | |
| 380 | + case MCE_ERROR_TYPE_LINK: | |
| 381 | + subtype = evt->u.link_error.link_error_type < | |
| 382 | + ARRAY_SIZE(mc_link_types) ? | |
| 383 | + mc_link_types[evt->u.link_error.link_error_type] | |
| 384 | + : "Unknown"; | |
| 385 | + printk("%s Error type: Link [%s]\n", level, subtype); | |
| 386 | + if (evt->u.link_error.effective_address_provided) | |
| 387 | + printk("%s Effective address: %016llx\n", | |
| 388 | + level, evt->u.link_error.effective_address); | |
| 389 | + break; | |
| 318 | 390 | default: |
| 319 | 391 | case MCE_ERROR_TYPE_UNKNOWN: |
| 320 | 392 | printk("%s Error type: Unknown\n", level); |
| @@ -341,6 +413,18 @@ uint64_t get_mce_fault_addr(struct machine_check_event *evt) · 20 unmodified lines | ||
| 341 | 413 | if (evt->u.tlb_error.effective_address_provided) |
| 342 | 414 | return evt->u.tlb_error.effective_address; |
| 343 | 415 | break; |
| 416 | + case MCE_ERROR_TYPE_USER: | |
| 417 | + if (evt->u.user_error.effective_address_provided) | |
| 418 | + return evt->u.user_error.effective_address; | |
| 419 | + break; | |
| 420 | + case MCE_ERROR_TYPE_RA: | |
| 421 | + if (evt->u.ra_error.effective_address_provided) | |
| 422 | + return evt->u.ra_error.effective_address; | |
| 423 | + break; | |
| 424 | + case MCE_ERROR_TYPE_LINK: | |
| 425 | + if (evt->u.link_error.effective_address_provided) | |
| 426 | + return evt->u.link_error.effective_address; | |
| 427 | + break; | |
| 344 | 428 | default: |
| 345 | 429 | case MCE_ERROR_TYPE_UNKNOWN: |
| 346 | 430 | break; |
| @@ -116,6 +116,51 @@ static void flush_and_reload_slb(void) · 115 unmodified lines | ||
| 116 | 116 | } |
| 117 | 117 | #endif |
| 118 | 118 | |
| 119 | +static void flush_erat(void) | |
| 120 | +{ | |
| 121 | + asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); | |
| 122 | +} | |
| 123 | + | |
| 124 | +#define MCE_FLUSH_SLB 1 | |
| 125 | +#define MCE_FLUSH_TLB 2 | |
| 126 | +#define MCE_FLUSH_ERAT 3 | |
| 127 | + | |
| 128 | +static int mce_flush(int what) | |
| 129 | +{ | |
| 130 | +#ifdef CONFIG_PPC_STD_MMU_64 | |
| 131 | + if (what == MCE_FLUSH_SLB) { | |
| 132 | + flush_and_reload_slb(); | |
| 133 | + return 1; | |
| 134 | + } | |
| 135 | +#endif | |
| 136 | + if (what == MCE_FLUSH_ERAT) { | |
| 137 | + flush_erat(); | |
| 138 | + return 1; | |
| 139 | + } | |
| 140 | + if (what == MCE_FLUSH_TLB) { | |
| 141 | + if (cur_cpu_spec && cur_cpu_spec->flush_tlb) { | |
| 142 | + cur_cpu_spec->flush_tlb(TLB_INVAL_SCOPE_GLOBAL); | |
| 143 | + return 1; | |
| 144 | + } | |
| 145 | + } | |
| 146 | + | |
| 147 | + return 0; | |
| 148 | +} | |
| 149 | + | |
| 150 | +static int mce_handle_flush_derrors(uint64_t dsisr, uint64_t slb, uint64_t tlb, uint64_t erat) | |
| 151 | +{ | |
| 152 | + if ((dsisr & slb) && mce_flush(MCE_FLUSH_SLB)) | |
| 153 | + dsisr &= ~slb; | |
| 154 | + if ((dsisr & erat) && mce_flush(MCE_FLUSH_ERAT)) | |
| 155 | + dsisr &= ~erat; | |
| 156 | + if ((dsisr & tlb) && mce_flush(MCE_FLUSH_TLB)) | |
| 157 | + dsisr &= ~tlb; | |
| 158 | + /* Any other errors we don't understand? */ | |
| 159 | + if (dsisr) | |
| 160 | + return 0; | |
| 161 | + return 1; | |
| 162 | +} | |
| 163 | + | |
| 119 | 164 | static long mce_handle_derror(uint64_t dsisr, uint64_t slb_error_bits) |
| 120 | 165 | { |
| 121 | 166 | long handled = 1; |
| @@ -281,6 +326,9 @@ long __machine_check_early_realmode_p7(struct pt_regs *regs) · 159 unmodified lines | ||
| 281 | 326 | long handled = 1; |
| 282 | 327 | struct mce_error_info mce_error_info = { 0 }; |
| 283 | 328 | |
| 329 | + mce_error_info.severity = MCE_SEV_ERROR_SYNC; | |
| 330 | + mce_error_info.initiator = MCE_INITIATOR_CPU; | |
| 331 | + | |
| 284 | 332 | srr1 = regs->msr; |
| 285 | 333 | nip = regs->nip; |
| 286 | 334 | |
| @@ -352,6 +400,9 @@ long __machine_check_early_realmode_p8(struct pt_regs *regs) · 65 unmodified lines | ||
| 352 | 400 | long handled = 1; |
| 353 | 401 | struct mce_error_info mce_error_info = { 0 }; |
| 354 | 402 | |
| 403 | + mce_error_info.severity = MCE_SEV_ERROR_SYNC; | |
| 404 | + mce_error_info.initiator = MCE_INITIATOR_CPU; | |
| 405 | + | |
| 355 | 406 | srr1 = regs->msr; |
| 356 | 407 | nip = regs->nip; |
| 357 | 408 | |
| @@ -372,3 +423,189 @@ long __machine_check_early_realmode_p8(struct pt_regs *regs) · 14 unmodified lines | ||
| 372 | 423 | save_mce_event(regs, handled, &mce_error_info, nip, addr); |
| 373 | 424 | return handled; |
| 374 | 425 | } |
| 426 | + | |
| 427 | +static int mce_handle_derror_p9(struct pt_regs *regs) | |
| 428 | +{ | |
| 429 | + uint64_t dsisr = regs->dsisr; | |
| 430 | + | |
| 431 | + return mce_handle_flush_derrors(dsisr, | |
| 432 | + P9_DSISR_MC_SLB_PARITY_MFSLB | | |
| 433 | + P9_DSISR_MC_SLB_MULTIHIT_MFSLB, | |
| 434 | + | |
| 435 | + P9_DSISR_MC_TLB_MULTIHIT_MFTLB, | |
| 436 | + | |
| 437 | + P9_DSISR_MC_ERAT_MULTIHIT); | |
| 438 | +} | |
| 439 | + | |
| 440 | +static int mce_handle_ierror_p9(struct pt_regs *regs) | |
| 441 | +{ | |
| 442 | + uint64_t srr1 = regs->msr; | |
| 443 | + | |
| 444 | + switch (P9_SRR1_MC_IFETCH(srr1)) { | |
| 445 | + case P9_SRR1_MC_IFETCH_SLB_PARITY: | |
| 446 | + case P9_SRR1_MC_IFETCH_SLB_MULTIHIT: | |
| 447 | + return mce_flush(MCE_FLUSH_SLB); | |
| 448 | + case P9_SRR1_MC_IFETCH_TLB_MULTIHIT: | |
| 449 | + return mce_flush(MCE_FLUSH_TLB); | |
| 450 | + case P9_SRR1_MC_IFETCH_ERAT_MULTIHIT: | |
| 451 | + return mce_flush(MCE_FLUSH_ERAT); | |
| 452 | + default: | |
| 453 | + return 0; | |
| 454 | + } | |
| 455 | +} | |
| 456 | + | |
| 457 | +static void mce_get_derror_p9(struct pt_regs *regs, | |
| 458 | + struct mce_error_info *mce_err, uint64_t *addr) | |
| 459 | +{ | |
| 460 | + uint64_t dsisr = regs->dsisr; | |
| 461 | + | |
| 462 | + mce_err->severity = MCE_SEV_ERROR_SYNC; | |
| 463 | + mce_err->initiator = MCE_INITIATOR_CPU; | |
| 464 | + | |
| 465 | + if (dsisr & P9_DSISR_MC_USER_TLBIE) | |
| 466 | + *addr = regs->nip; | |
| 467 | + else | |
| 468 | + *addr = regs->dar; | |
| 469 | + | |
| 470 | + if (dsisr & P9_DSISR_MC_UE) { | |
| 471 | + mce_err->error_type = MCE_ERROR_TYPE_UE; | |
| 472 | + mce_err->u.ue_error_type = MCE_UE_ERROR_LOAD_STORE; | |
| 473 | + } else if (dsisr & P9_DSISR_MC_UE_TABLEWALK) { | |
| 474 | + mce_err->error_type = MCE_ERROR_TYPE_UE; | |
| 475 | + mce_err->u.ue_error_type = MCE_UE_ERROR_PAGE_TABLE_WALK_LOAD_STORE; | |
| 476 | + } else if (dsisr & P9_DSISR_MC_LINK_LOAD_TIMEOUT) { | |
| 477 | + mce_err->error_type = MCE_ERROR_TYPE_LINK; | |
| 478 | + mce_err->u.link_error_type = MCE_LINK_ERROR_LOAD_TIMEOUT; | |
| 479 | + } else if (dsisr & P9_DSISR_MC_LINK_TABLEWALK_TIMEOUT) { | |
| 480 | + mce_err->error_type = MCE_ERROR_TYPE_LINK; | |
| 481 | + mce_err->u.link_error_type = MCE_LINK_ERROR_PAGE_TABLE_WALK_LOAD_STORE_TIMEOUT; | |
| 482 | + } else if (dsisr & P9_DSISR_MC_ERAT_MULTIHIT) { | |
| 483 | + mce_err->error_type = MCE_ERROR_TYPE_ERAT; | |
| 484 | + mce_err->u.erat_error_type = MCE_ERAT_ERROR_MULTIHIT; | |
| 485 | + } else if (dsisr & P9_DSISR_MC_TLB_MULTIHIT_MFTLB) { | |
| 486 | + mce_err->error_type = MCE_ERROR_TYPE_TLB; | |
| 487 | + mce_err->u.tlb_error_type = MCE_TLB_ERROR_MULTIHIT; | |
| 488 | + } else if (dsisr & P9_DSISR_MC_USER_TLBIE) { | |
| 489 | + mce_err->error_type = MCE_ERROR_TYPE_USER; | |
| 490 | + mce_err->u.user_error_type = MCE_USER_ERROR_TLBIE; | |
| 491 | + } else if (dsisr & P9_DSISR_MC_SLB_PARITY_MFSLB) { | |
| 492 | + mce_err->error_type = MCE_ERROR_TYPE_SLB; | |
| 493 | + mce_err->u.slb_error_type = MCE_SLB_ERROR_PARITY; | |
| 494 | + } else if (dsisr & P9_DSISR_MC_SLB_MULTIHIT_MFSLB) { | |
| 495 | + mce_err->error_type = MCE_ERROR_TYPE_SLB; | |
| 496 | + mce_err->u.slb_error_type = MCE_SLB_ERROR_MULTIHIT; | |
| 497 | + } else if (dsisr & P9_DSISR_MC_RA_LOAD) { | |
| 498 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 499 | + mce_err->u.ra_error_type = MCE_RA_ERROR_LOAD; | |
| 500 | + } else if (dsisr & P9_DSISR_MC_RA_TABLEWALK) { | |
| 501 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 502 | + mce_err->u.ra_error_type = MCE_RA_ERROR_PAGE_TABLE_WALK_LOAD_STORE; | |
| 503 | + } else if (dsisr & P9_DSISR_MC_RA_TABLEWALK_FOREIGN) { | |
| 504 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 505 | + mce_err->u.ra_error_type = MCE_RA_ERROR_PAGE_TABLE_WALK_LOAD_STORE_FOREIGN; | |
| 506 | + } else if (dsisr & P9_DSISR_MC_RA_FOREIGN) { | |
| 507 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 508 | + mce_err->u.ra_error_type = MCE_RA_ERROR_LOAD_STORE_FOREIGN; | |
| 509 | + } | |
| 510 | +} | |
| 511 | + | |
| 512 | +static void mce_get_ierror_p9(struct pt_regs *regs, | |
| 513 | + struct mce_error_info *mce_err, uint64_t *addr) | |
| 514 | +{ | |
| 515 | + uint64_t srr1 = regs->msr; | |
| 516 | + | |
| 517 | + switch (P9_SRR1_MC_IFETCH(srr1)) { | |
| 518 | + case P9_SRR1_MC_IFETCH_RA_ASYNC_STORE: | |
| 519 | + case P9_SRR1_MC_IFETCH_LINK_ASYNC_STORE_TIMEOUT: | |
| 520 | + mce_err->severity = MCE_SEV_FATAL; | |
| 521 | + break; | |
| 522 | + default: | |
| 523 | + mce_err->severity = MCE_SEV_ERROR_SYNC; | |
| 524 | + break; | |
| 525 | + } | |
| 526 | + | |
| 527 | + mce_err->initiator = MCE_INITIATOR_CPU; | |
| 528 | + | |
| 529 | + *addr = regs->nip; | |
| 530 | + | |
| 531 | + switch (P9_SRR1_MC_IFETCH(srr1)) { | |
| 532 | + case P9_SRR1_MC_IFETCH_UE: | |
| 533 | + mce_err->error_type = MCE_ERROR_TYPE_UE; | |
| 534 | + mce_err->u.ue_error_type = MCE_UE_ERROR_IFETCH; | |
| 535 | + break; | |
| 536 | + case P9_SRR1_MC_IFETCH_SLB_PARITY: | |
| 537 | + mce_err->error_type = MCE_ERROR_TYPE_SLB; | |
| 538 | + mce_err->u.slb_error_type = MCE_SLB_ERROR_PARITY; | |
| 539 | + break; | |
| 540 | + case P9_SRR1_MC_IFETCH_SLB_MULTIHIT: | |
| 541 | + mce_err->error_type = MCE_ERROR_TYPE_SLB; | |
| 542 | + mce_err->u.slb_error_type = MCE_SLB_ERROR_MULTIHIT; | |
| 543 | + break; | |
| 544 | + case P9_SRR1_MC_IFETCH_ERAT_MULTIHIT: | |
| 545 | + mce_err->error_type = MCE_ERROR_TYPE_ERAT; | |
| 546 | + mce_err->u.erat_error_type = MCE_ERAT_ERROR_MULTIHIT; | |
| 547 | + break; | |
| 548 | + case P9_SRR1_MC_IFETCH_TLB_MULTIHIT: | |
| 549 | + mce_err->error_type = MCE_ERROR_TYPE_TLB; | |
| 550 | + mce_err->u.tlb_error_type = MCE_TLB_ERROR_MULTIHIT; | |
| 551 | + break; | |
| 552 | + case P9_SRR1_MC_IFETCH_UE_TLB_RELOAD: | |
| 553 | + mce_err->error_type = MCE_ERROR_TYPE_UE; | |
| 554 | + mce_err->u.ue_error_type = MCE_UE_ERROR_PAGE_TABLE_WALK_IFETCH; | |
| 555 | + break; | |
| 556 | + case P9_SRR1_MC_IFETCH_LINK_TIMEOUT: | |
| 557 | + mce_err->error_type = MCE_ERROR_TYPE_LINK; | |
| 558 | + mce_err->u.link_error_type = MCE_LINK_ERROR_IFETCH_TIMEOUT; | |
| 559 | + break; | |
| 560 | + case P9_SRR1_MC_IFETCH_LINK_TABLEWALK_TIMEOUT: | |
| 561 | + mce_err->error_type = MCE_ERROR_TYPE_LINK; | |
| 562 | + mce_err->u.link_error_type = MCE_LINK_ERROR_PAGE_TABLE_WALK_IFETCH_TIMEOUT; | |
| 563 | + break; | |
| 564 | + case P9_SRR1_MC_IFETCH_RA: | |
| 565 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 566 | + mce_err->u.ra_error_type = MCE_RA_ERROR_IFETCH; | |
| 567 | + break; | |
| 568 | + case P9_SRR1_MC_IFETCH_RA_TABLEWALK: | |
| 569 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 570 | + mce_err->u.ra_error_type = MCE_RA_ERROR_PAGE_TABLE_WALK_IFETCH; | |
| 571 | + break; | |
| 572 | + case P9_SRR1_MC_IFETCH_RA_ASYNC_STORE: | |
| 573 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 574 | + mce_err->u.ra_error_type = MCE_RA_ERROR_STORE; | |
| 575 | + break; | |
| 576 | + case P9_SRR1_MC_IFETCH_LINK_ASYNC_STORE_TIMEOUT: | |
| 577 | + mce_err->error_type = MCE_ERROR_TYPE_LINK; | |
| 578 | + mce_err->u.link_error_type = MCE_LINK_ERROR_STORE_TIMEOUT; | |
| 579 | + break; | |
| 580 | + case P9_SRR1_MC_IFETCH_RA_TABLEWALK_FOREIGN: | |
| 581 | + mce_err->error_type = MCE_ERROR_TYPE_RA; | |
| 582 | + mce_err->u.ra_error_type = MCE_RA_ERROR_PAGE_TABLE_WALK_IFETCH_FOREIGN; | |
| 583 | + break; | |
| 584 | + default: | |
| 585 | + break; | |
| 586 | + } | |
| 587 | +} | |
| 588 | + | |
| 589 | +long __machine_check_early_realmode_p9(struct pt_regs *regs) | |
| 590 | +{ | |
| 591 | + uint64_t nip, addr; | |
| 592 | + long handled; | |
| 593 | + struct mce_error_info mce_error_info = { 0 }; | |
| 594 | + | |
| 595 | + nip = regs->nip; | |
| 596 | + | |
| 597 | + if (P9_SRR1_MC_LOADSTORE(regs->msr)) { | |
| 598 | + handled = mce_handle_derror_p9(regs); | |
| 599 | + mce_get_derror_p9(regs, &mce_error_info, &addr); | |
| 600 | + } else { | |
| 601 | + handled = mce_handle_ierror_p9(regs); | |
| 602 | + mce_get_ierror_p9(regs, &mce_error_info, &addr); | |
| 603 | + } | |
| 604 | + | |
| 605 | + /* Handle UE error. */ | |
| 606 | + if (mce_error_info.error_type == MCE_ERROR_TYPE_UE) | |
| 607 | + handled = mce_handle_ue_error(regs); | |
| 608 | + | |
| 609 | + save_mce_event(regs, handled, &mce_error_info, nip, addr); | |
| 610 | + return handled; | |
| 611 | +} | |
| @@ -168,6 +168,14 @@ static unsigned long __initdata prom_tce_alloc_start; · 167 unmodified lines | ||
| 168 | 168 | static unsigned long __initdata prom_tce_alloc_end; |
| 169 | 169 | #endif |
| 170 | 170 | |
| 171 | +static bool __initdata prom_radix_disable; | |
| 172 | + | |
| 173 | +struct platform_support { | |
| 174 | + bool hash_mmu; | |
| 175 | + bool radix_mmu; | |
| 176 | + bool radix_gtse; | |
| 177 | +}; | |
| 178 | + | |
| 171 | 179 | /* Platforms codes are now obsolete in the kernel. Now only used within this |
| 172 | 180 | * file and ultimately gone too. Feel free to change them if you need, they |
| 173 | 181 | * are not shared with anything outside of this file anymore |
| @@ -626,6 +634,12 @@ static void __init early_cmdline_parse(void) · 452 unmodified lines | ||
| 626 | 634 | prom_memory_limit = ALIGN(prom_memory_limit, 0x1000000); |
| 627 | 635 | #endif |
| 628 | 636 | } |
| 637 | + | |
| 638 | + opt = strstr(prom_cmd_line, "disable_radix"); | |
| 639 | + if (opt) { | |
| 640 | + prom_debug("Radix disabled from cmdline\n"); | |
| 641 | + prom_radix_disable = true; | |
| 642 | + } | |
| 629 | 643 | } |
| 630 | 644 | |
| 631 | 645 | #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV) |
| @@ -695,6 +709,8 @@ struct option_vector5 { · 63 unmodified lines | ||
| 695 | 709 | u8 byte22; |
| 696 | 710 | u8 intarch; |
| 697 | 711 | u8 mmu; |
| 712 | + u8 hash_ext; | |
| 713 | + u8 radix_ext; | |
| 698 | 714 | } __packed; |
| 699 | 715 | |
| 700 | 716 | struct option_vector6 { |
| @@ -850,8 +866,9 @@ struct ibm_arch_vec __cacheline_aligned ibm_architecture_vec = { · 149 unmodified lines | ||
| 850 | 866 | .reserved3 = 0, |
| 851 | 867 | .subprocessors = 1, |
| 852 | 868 | .intarch = 0, |
| 853 | − .mmu = OV5_FEAT(OV5_MMU_RADIX_300) | OV5_FEAT(OV5_MMU_HASH_300) | | |
| 854 | − OV5_FEAT(OV5_MMU_PROC_TBL) | OV5_FEAT(OV5_MMU_GTSE), | |
| 869 | + .mmu = 0, | |
| 870 | + .hash_ext = 0, | |
| 871 | + .radix_ext = 0, | |
| 855 | 872 | }, |
| 856 | 873 | |
| 857 | 874 | /* option vector 6: IBM PAPR hints */ |
| @@ -990,6 +1007,92 @@ static int __init prom_count_smt_threads(void) · 132 unmodified lines | ||
| 990 | 1007 | |
| 991 | 1008 | } |
| 992 | 1009 | |
| 1010 | +static void __init prom_parse_mmu_model(u8 val, | |
| 1011 | + struct platform_support *support) | |
| 1012 | +{ | |
| 1013 | + switch (val) { | |
| 1014 | + case OV5_FEAT(OV5_MMU_DYNAMIC): | |
| 1015 | + case OV5_FEAT(OV5_MMU_EITHER): /* Either Available */ | |
| 1016 | + prom_debug("MMU - either supported\n"); | |
| 1017 | + support->radix_mmu = !prom_radix_disable; | |
| 1018 | + support->hash_mmu = true; | |
| 1019 | + break; | |
| 1020 | + case OV5_FEAT(OV5_MMU_RADIX): /* Only Radix */ | |
| 1021 | + prom_debug("MMU - radix only\n"); | |
| 1022 | + if (prom_radix_disable) { | |
| 1023 | + /* | |
| 1024 | + * If we __have__ to do radix, we're better off ignoring | |
| 1025 | + * the command line rather than not booting. | |
| 1026 | + */ | |
| 1027 | + prom_printf("WARNING: Ignoring cmdline option disable_radix\n"); | |
| 1028 | + } | |
| 1029 | + support->radix_mmu = true; | |
| 1030 | + break; | |
| 1031 | + case OV5_FEAT(OV5_MMU_HASH): | |
| 1032 | + prom_debug("MMU - hash only\n"); | |
| 1033 | + support->hash_mmu = true; | |
| 1034 | + break; | |
| 1035 | + default: | |
| 1036 | + prom_debug("Unknown mmu support option: 0x%x\n", val); | |
| 1037 | + break; | |
| 1038 | + } | |
| 1039 | +} | |
| 1040 | + | |
| 1041 | +static void __init prom_parse_platform_support(u8 index, u8 val, | |
| 1042 | + struct platform_support *support) | |
| 1043 | +{ | |
| 1044 | + switch (index) { | |
| 1045 | + case OV5_INDX(OV5_MMU_SUPPORT): /* MMU Model */ | |
| 1046 | + prom_parse_mmu_model(val & OV5_FEAT(OV5_MMU_SUPPORT), support); | |
| 1047 | + break; | |
| 1048 | + case OV5_INDX(OV5_RADIX_GTSE): /* Radix Extensions */ | |
| 1049 | + if (val & OV5_FEAT(OV5_RADIX_GTSE)) { | |
| 1050 | + prom_debug("Radix - GTSE supported\n"); | |
| 1051 | + support->radix_gtse = true; | |
| 1052 | + } | |
| 1053 | + break; | |
| 1054 | + } | |
| 1055 | +} | |
| 1056 | + | |
| 1057 | +static void __init prom_check_platform_support(void) | |
| 1058 | +{ | |
| 1059 | + struct platform_support supported = { | |
| 1060 | + .hash_mmu = false, | |
| 1061 | + .radix_mmu = false, | |
| 1062 | + .radix_gtse = false | |
| 1063 | + }; | |
| 1064 | + int prop_len = prom_getproplen(prom.chosen, | |
| 1065 | + "ibm,arch-vec-5-platform-support"); | |
| 1066 | + if (prop_len > 1) { | |
| 1067 | + int i; | |
| 1068 | + u8 vec[prop_len]; | |
| 1069 | + prom_debug("Found ibm,arch-vec-5-platform-support, len: %d\n", | |
| 1070 | + prop_len); | |
| 1071 | + prom_getprop(prom.chosen, "ibm,arch-vec-5-platform-support", | |
| 1072 | + &vec, sizeof(vec)); | |
| 1073 | + for (i = 0; i < prop_len; i += 2) { | |
| 1074 | + prom_debug("%d: index = 0x%x val = 0x%x\n", i / 2 | |
| 1075 | + , vec[i] | |
| 1076 | + , vec[i + 1]); | |
| 1077 | + prom_parse_platform_support(vec[i], vec[i + 1], | |
| 1078 | + &supported); | |
| 1079 | + } | |
| 1080 | + } | |
| 1081 | + | |
| 1082 | + if (supported.radix_mmu && supported.radix_gtse) { | |
| 1083 | + /* Radix preferred - but we require GTSE for now */ | |
| 1084 | + prom_debug("Asking for radix with GTSE\n"); | |
| 1085 | + ibm_architecture_vec.vec5.mmu = OV5_FEAT(OV5_MMU_RADIX); | |
| 1086 | + ibm_architecture_vec.vec5.radix_ext = OV5_FEAT(OV5_RADIX_GTSE); | |
| 1087 | + } else if (supported.hash_mmu) { | |
| 1088 | + /* Default to hash mmu (if we can) */ | |
| 1089 | + prom_debug("Asking for hash\n"); | |
| 1090 | + ibm_architecture_vec.vec5.mmu = OV5_FEAT(OV5_MMU_HASH); | |
| 1091 | + } else { | |
| 1092 | + /* We're probably on a legacy hypervisor */ | |
| 1093 | + prom_debug("Assuming legacy hash support\n"); | |
| 1094 | + } | |
| 1095 | +} | |
| 993 | 1096 | |
| 994 | 1097 | static void __init prom_send_capabilities(void) |
| 995 | 1098 | { |
| @@ -997,6 +1100,9 @@ static void __init prom_send_capabilities(void) · 1 unmodified line | ||
| 997 | 1100 | prom_arg_t ret; |
| 998 | 1101 | u32 cores; |
| 999 | 1102 | |
| 1103 | + /* Check ibm,arch-vec-5-platform-support and fixup vec5 if required */ | |
| 1104 | + prom_check_platform_support(); | |
| 1105 | + | |
| 1000 | 1106 | root = call_prom("open", 1, 1, ADDR("/")); |
| 1001 | 1107 | if (root != 0) { |
| 1002 | 1108 | /* We need to tell the FW about the number of cores we support. |
| @@ -2993,6 +3099,11 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4, · 1990 unmodified lines | ||
| 2993 | 3099 | */ |
| 2994 | 3100 | prom_check_initrd(r3, r4); |
| 2995 | 3101 | |
| 3102 | + /* | |
| 3103 | + * Do early parsing of command line | |
| 3104 | + */ | |
| 3105 | + early_cmdline_parse(); | |
| 3106 | + | |
| 2996 | 3107 | #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV) |
| 2997 | 3108 | /* |
| 2998 | 3109 | * On pSeries, inform the firmware about our capabilities |
| @@ -3008,11 +3119,6 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4, · 9 unmodified lines | ||
| 3008 | 3119 | if (of_platform != PLATFORM_POWERMAC) |
| 3009 | 3120 | copy_and_flush(0, kbase, 0x100, 0); |
| 3010 | 3121 | |
| 3011 | − /* | |
| 3012 | − * Do early parsing of command line | |
| 3013 | − */ | |
| 3014 | − early_cmdline_parse(); | |
| 3015 | − | |
| 3016 | 3122 | /* |
| 3017 | 3123 | * Initialize memory management within prom_init |
| 3018 | 3124 | */ |
| @@ -408,7 +408,10 @@ static void init_cache_info(struct ppc_cache_info *info, u32 size, u32 lsize, · 407 unmodified lines | ||
| 408 | 408 | info->line_size = lsize; |
| 409 | 409 | info->block_size = bsize; |
| 410 | 410 | info->log_block_size = __ilog2(bsize); |
| 411 | − info->blocks_per_page = PAGE_SIZE / bsize; | |
| 411 | + if (bsize) | |
| 412 | + info->blocks_per_page = PAGE_SIZE / bsize; | |
| 413 | + else | |
| 414 | + info->blocks_per_page = 0; | |
| 412 | 415 | |
| 413 | 416 | if (sets == 0) |
| 414 | 417 | info->assoc = 0xffff; |
| @@ -601,7 +601,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, · 600 unmodified lines | ||
| 601 | 601 | hva, NULL, NULL); |
| 602 | 602 | if (ptep) { |
| 603 | 603 | pte = kvmppc_read_update_linux_pte(ptep, 1); |
| 604 | − if (pte_write(pte)) | |
| 604 | + if (__pte_write(pte)) | |
| 605 | 605 | write_ok = 1; |
| 606 | 606 | } |
| 607 | 607 | local_irq_restore(flags); |
| @@ -256,7 +256,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · 255 unmodified lines | ||
| 256 | 256 | } |
| 257 | 257 | pte = kvmppc_read_update_linux_pte(ptep, writing); |
| 258 | 258 | if (pte_present(pte) && !pte_protnone(pte)) { |
| 259 | − if (writing && !pte_write(pte)) | |
| 259 | + if (writing && !__pte_write(pte)) | |
| 260 | 260 | /* make the actual HPTE be read-only */ |
| 261 | 261 | ptel = hpte_make_readonly(ptel); |
| 262 | 262 | is_ci = pte_ci(pte); |
| @@ -20,6 +20,7 @@ obj64-y += copypage_64.o copyuser_64.o usercopy_64.o mem_64.o hweight_64.o \ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | obj64-$(CONFIG_SMP) += locks.o |
| 22 | 22 | obj64-$(CONFIG_ALTIVEC) += vmx-helper.o |
| 23 | +obj64-$(CONFIG_KPROBES_SANITY_TEST) += test_emulate_step.o | |
| 23 | 24 | |
| 24 | 25 | obj-y += checksum_$(BITS).o checksum_wrappers.o |
| 25 | 26 | |
| @@ -1799,8 +1799,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 1798 unmodified lines | ||
| 1799 | 1799 | goto instr_done; |
| 1800 | 1800 | |
| 1801 | 1801 | case LARX: |
| 1802 | − if (regs->msr & MSR_LE) | |
| 1803 | − return 0; | |
| 1804 | 1802 | if (op.ea & (size - 1)) |
| 1805 | 1803 | break; /* can't handle misaligned */ |
| 1806 | 1804 | if (!address_ok(regs, op.ea, size)) |
| @@ -1823,8 +1821,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 16 unmodified lines | ||
| 1823 | 1821 | goto ldst_done; |
| 1824 | 1822 | |
| 1825 | 1823 | case STCX: |
| 1826 | − if (regs->msr & MSR_LE) | |
| 1827 | − return 0; | |
| 1828 | 1824 | if (op.ea & (size - 1)) |
| 1829 | 1825 | break; /* can't handle misaligned */ |
| 1830 | 1826 | if (!address_ok(regs, op.ea, size)) |
| @@ -1849,8 +1845,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 18 unmodified lines | ||
| 1849 | 1845 | goto ldst_done; |
| 1850 | 1846 | |
| 1851 | 1847 | case LOAD: |
| 1852 | − if (regs->msr & MSR_LE) | |
| 1853 | − return 0; | |
| 1854 | 1848 | err = read_mem(®s->gpr[op.reg], op.ea, size, regs); |
| 1855 | 1849 | if (!err) { |
| 1856 | 1850 | if (op.type & SIGNEXT) |
| @@ -1862,8 +1856,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 5 unmodified lines | ||
| 1862 | 1856 | |
| 1863 | 1857 | #ifdef CONFIG_PPC_FPU |
| 1864 | 1858 | case LOAD_FP: |
| 1865 | − if (regs->msr & MSR_LE) | |
| 1866 | − return 0; | |
| 1867 | 1859 | if (size == 4) |
| 1868 | 1860 | err = do_fp_load(op.reg, do_lfs, op.ea, size, regs); |
| 1869 | 1861 | else |
| @@ -1872,15 +1864,11 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 2 unmodified lines | ||
| 1872 | 1864 | #endif |
| 1873 | 1865 | #ifdef CONFIG_ALTIVEC |
| 1874 | 1866 | case LOAD_VMX: |
| 1875 | − if (regs->msr & MSR_LE) | |
| 1876 | − return 0; | |
| 1877 | 1867 | err = do_vec_load(op.reg, do_lvx, op.ea & ~0xfUL, regs); |
| 1878 | 1868 | goto ldst_done; |
| 1879 | 1869 | #endif |
| 1880 | 1870 | #ifdef CONFIG_VSX |
| 1881 | 1871 | case LOAD_VSX: |
| 1882 | − if (regs->msr & MSR_LE) | |
| 1883 | − return 0; | |
| 1884 | 1872 | err = do_vsx_load(op.reg, do_lxvd2x, op.ea, regs); |
| 1885 | 1873 | goto ldst_done; |
| 1886 | 1874 | #endif |
| @@ -1903,8 +1891,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 16 unmodified lines | ||
| 1903 | 1891 | goto instr_done; |
| 1904 | 1892 | |
| 1905 | 1893 | case STORE: |
| 1906 | − if (regs->msr & MSR_LE) | |
| 1907 | − return 0; | |
| 1908 | 1894 | if ((op.type & UPDATE) && size == sizeof(long) && |
| 1909 | 1895 | op.reg == 1 && op.update_reg == 1 && |
| 1910 | 1896 | !(regs->msr & MSR_PR) && |
| @@ -1917,8 +1903,6 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 6 unmodified lines | ||
| 1917 | 1903 | |
| 1918 | 1904 | #ifdef CONFIG_PPC_FPU |
| 1919 | 1905 | case STORE_FP: |
| 1920 | − if (regs->msr & MSR_LE) | |
| 1921 | − return 0; | |
| 1922 | 1906 | if (size == 4) |
| 1923 | 1907 | err = do_fp_store(op.reg, do_stfs, op.ea, size, regs); |
| 1924 | 1908 | else |
| @@ -1927,15 +1911,11 @@ int __kprobes emulate_step(struct pt_regs *regs, unsigned int instr) · 2 unmodified lines | ||
| 1927 | 1911 | #endif |
| 1928 | 1912 | #ifdef CONFIG_ALTIVEC |
| 1929 | 1913 | case STORE_VMX: |
| 1930 | − if (regs->msr & MSR_LE) | |
| 1931 | − return 0; | |
| 1932 | 1914 | err = do_vec_store(op.reg, do_stvx, op.ea & ~0xfUL, regs); |
| 1933 | 1915 | goto ldst_done; |
| 1934 | 1916 | #endif |
| 1935 | 1917 | #ifdef CONFIG_VSX |
| 1936 | 1918 | case STORE_VSX: |
| 1937 | − if (regs->msr & MSR_LE) | |
| 1938 | − return 0; | |
| 1939 | 1919 | err = do_vsx_store(op.reg, do_stxvd2x, op.ea, regs); |
| 1940 | 1920 | goto ldst_done; |
| 1941 | 1921 | #endif |
| @@ -0,0 +1,434 @@ | ||
| 1 | +/* | |
| 2 | + * Simple sanity test for emulate_step load/store instructions. | |
| 3 | + * | |
| 4 | + * Copyright IBM Corp. 2016 | |
| 5 | + * | |
| 6 | + * This program is free software; you can redistribute it and/or modify | |
| 7 | + * it under the terms of the GNU General Public License as published by | |
| 8 | + * the Free Software Foundation; either version 2 of the License, or | |
| 9 | + * (at your option) any later version. | |
| 10 | + */ | |
| 11 | + | |
| 12 | +#define pr_fmt(fmt) "emulate_step_test: " fmt | |
| 13 | + | |
| 14 | +#include <linux/ptrace.h> | |
| 15 | +#include <asm/sstep.h> | |
| 16 | +#include <asm/ppc-opcode.h> | |
| 17 | + | |
| 18 | +#define IMM_L(i) ((uintptr_t)(i) & 0xffff) | |
| 19 | + | |
| 20 | +/* | |
| 21 | + * Defined with TEST_ prefix so it does not conflict with other | |
| 22 | + * definitions. | |
| 23 | + */ | |
| 24 | +#define TEST_LD(r, base, i) (PPC_INST_LD | ___PPC_RT(r) | \ | |
| 25 | + ___PPC_RA(base) | IMM_L(i)) | |
| 26 | +#define TEST_LWZ(r, base, i) (PPC_INST_LWZ | ___PPC_RT(r) | \ | |
| 27 | + ___PPC_RA(base) | IMM_L(i)) | |
| 28 | +#define TEST_LWZX(t, a, b) (PPC_INST_LWZX | ___PPC_RT(t) | \ | |
| 29 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 30 | +#define TEST_STD(r, base, i) (PPC_INST_STD | ___PPC_RS(r) | \ | |
| 31 | + ___PPC_RA(base) | ((i) & 0xfffc)) | |
| 32 | +#define TEST_LDARX(t, a, b, eh) (PPC_INST_LDARX | ___PPC_RT(t) | \ | |
| 33 | + ___PPC_RA(a) | ___PPC_RB(b) | \ | |
| 34 | + __PPC_EH(eh)) | |
| 35 | +#define TEST_STDCX(s, a, b) (PPC_INST_STDCX | ___PPC_RS(s) | \ | |
| 36 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 37 | +#define TEST_LFSX(t, a, b) (PPC_INST_LFSX | ___PPC_RT(t) | \ | |
| 38 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 39 | +#define TEST_STFSX(s, a, b) (PPC_INST_STFSX | ___PPC_RS(s) | \ | |
| 40 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 41 | +#define TEST_LFDX(t, a, b) (PPC_INST_LFDX | ___PPC_RT(t) | \ | |
| 42 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 43 | +#define TEST_STFDX(s, a, b) (PPC_INST_STFDX | ___PPC_RS(s) | \ | |
| 44 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 45 | +#define TEST_LVX(t, a, b) (PPC_INST_LVX | ___PPC_RT(t) | \ | |
| 46 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 47 | +#define TEST_STVX(s, a, b) (PPC_INST_STVX | ___PPC_RS(s) | \ | |
| 48 | + ___PPC_RA(a) | ___PPC_RB(b)) | |
| 49 | +#define TEST_LXVD2X(s, a, b) (PPC_INST_LXVD2X | VSX_XX1((s), R##a, R##b)) | |
| 50 | +#define TEST_STXVD2X(s, a, b) (PPC_INST_STXVD2X | VSX_XX1((s), R##a, R##b)) | |
| 51 | + | |
| 52 | + | |
| 53 | +static void __init init_pt_regs(struct pt_regs *regs) | |
| 54 | +{ | |
| 55 | + static unsigned long msr; | |
| 56 | + static bool msr_cached; | |
| 57 | + | |
| 58 | + memset(regs, 0, sizeof(struct pt_regs)); | |
| 59 | + | |
| 60 | + if (likely(msr_cached)) { | |
| 61 | + regs->msr = msr; | |
| 62 | + return; | |
| 63 | + } | |
| 64 | + | |
| 65 | + asm volatile("mfmsr %0" : "=r"(regs->msr)); | |
| 66 | + | |
| 67 | + regs->msr |= MSR_FP; | |
| 68 | + regs->msr |= MSR_VEC; | |
| 69 | + regs->msr |= MSR_VSX; | |
| 70 | + | |
| 71 | + msr = regs->msr; | |
| 72 | + msr_cached = true; | |
| 73 | +} | |
| 74 | + | |
| 75 | +static void __init show_result(char *ins, char *result) | |
| 76 | +{ | |
| 77 | + pr_info("%-14s : %s\n", ins, result); | |
| 78 | +} | |
| 79 | + | |
| 80 | +static void __init test_ld(void) | |
| 81 | +{ | |
| 82 | + struct pt_regs regs; | |
| 83 | + unsigned long a = 0x23; | |
| 84 | + int stepped = -1; | |
| 85 | + | |
| 86 | + init_pt_regs(®s); | |
| 87 | + regs.gpr[3] = (unsigned long) &a; | |
| 88 | + | |
| 89 | + /* ld r5, 0(r3) */ | |
| 90 | + stepped = emulate_step(®s, TEST_LD(5, 3, 0)); | |
| 91 | + | |
| 92 | + if (stepped == 1 && regs.gpr[5] == a) | |
| 93 | + show_result("ld", "PASS"); | |
| 94 | + else | |
| 95 | + show_result("ld", "FAIL"); | |
| 96 | +} | |
| 97 | + | |
| 98 | +static void __init test_lwz(void) | |
| 99 | +{ | |
| 100 | + struct pt_regs regs; | |
| 101 | + unsigned int a = 0x4545; | |
| 102 | + int stepped = -1; | |
| 103 | + | |
| 104 | + init_pt_regs(®s); | |
| 105 | + regs.gpr[3] = (unsigned long) &a; | |
| 106 | + | |
| 107 | + /* lwz r5, 0(r3) */ | |
| 108 | + stepped = emulate_step(®s, TEST_LWZ(5, 3, 0)); | |
| 109 | + | |
| 110 | + if (stepped == 1 && regs.gpr[5] == a) | |
| 111 | + show_result("lwz", "PASS"); | |
| 112 | + else | |
| 113 | + show_result("lwz", "FAIL"); | |
| 114 | +} | |
| 115 | + | |
| 116 | +static void __init test_lwzx(void) | |
| 117 | +{ | |
| 118 | + struct pt_regs regs; | |
| 119 | + unsigned int a[3] = {0x0, 0x0, 0x1234}; | |
| 120 | + int stepped = -1; | |
| 121 | + | |
| 122 | + init_pt_regs(®s); | |
| 123 | + regs.gpr[3] = (unsigned long) a; | |
| 124 | + regs.gpr[4] = 8; | |
| 125 | + regs.gpr[5] = 0x8765; | |
| 126 | + | |
| 127 | + /* lwzx r5, r3, r4 */ | |
| 128 | + stepped = emulate_step(®s, TEST_LWZX(5, 3, 4)); | |
| 129 | + if (stepped == 1 && regs.gpr[5] == a[2]) | |
| 130 | + show_result("lwzx", "PASS"); | |
| 131 | + else | |
| 132 | + show_result("lwzx", "FAIL"); | |
| 133 | +} | |
| 134 | + | |
| 135 | +static void __init test_std(void) | |
| 136 | +{ | |
| 137 | + struct pt_regs regs; | |
| 138 | + unsigned long a = 0x1234; | |
| 139 | + int stepped = -1; | |
| 140 | + | |
| 141 | + init_pt_regs(®s); | |
| 142 | + regs.gpr[3] = (unsigned long) &a; | |
| 143 | + regs.gpr[5] = 0x5678; | |
| 144 | + | |
| 145 | + /* std r5, 0(r3) */ | |
| 146 | + stepped = emulate_step(®s, TEST_STD(5, 3, 0)); | |
| 147 | + if (stepped == 1 || regs.gpr[5] == a) | |
| 148 | + show_result("std", "PASS"); | |
| 149 | + else | |
| 150 | + show_result("std", "FAIL"); | |
| 151 | +} | |
| 152 | + | |
| 153 | +static void __init test_ldarx_stdcx(void) | |
| 154 | +{ | |
| 155 | + struct pt_regs regs; | |
| 156 | + unsigned long a = 0x1234; | |
| 157 | + int stepped = -1; | |
| 158 | + unsigned long cr0_eq = 0x1 << 29; /* eq bit of CR0 */ | |
| 159 | + | |
| 160 | + init_pt_regs(®s); | |
| 161 | + asm volatile("mfcr %0" : "=r"(regs.ccr)); | |
| 162 | + | |
| 163 | + | |
| 164 | + /*** ldarx ***/ | |
| 165 | + | |
| 166 | + regs.gpr[3] = (unsigned long) &a; | |
| 167 | + regs.gpr[4] = 0; | |
| 168 | + regs.gpr[5] = 0x5678; | |
| 169 | + | |
| 170 | + /* ldarx r5, r3, r4, 0 */ | |
| 171 | + stepped = emulate_step(®s, TEST_LDARX(5, 3, 4, 0)); | |
| 172 | + | |
| 173 | + /* | |
| 174 | + * Don't touch 'a' here. Touching 'a' can do Load/store | |
| 175 | + * of 'a' which result in failure of subsequent stdcx. | |
| 176 | + * Instead, use hardcoded value for comparison. | |
| 177 | + */ | |
| 178 | + if (stepped <= 0 || regs.gpr[5] != 0x1234) { | |
| 179 | + show_result("ldarx / stdcx.", "FAIL (ldarx)"); | |
| 180 | + return; | |
| 181 | + } | |
| 182 | + | |
| 183 | + | |
| 184 | + /*** stdcx. ***/ | |
| 185 | + | |
| 186 | + regs.gpr[5] = 0x9ABC; | |
| 187 | + | |
| 188 | + /* stdcx. r5, r3, r4 */ | |
| 189 | + stepped = emulate_step(®s, TEST_STDCX(5, 3, 4)); | |
| 190 | + | |
| 191 | + /* | |
| 192 | + * Two possible scenarios that indicates successful emulation | |
| 193 | + * of stdcx. : | |
| 194 | + * 1. Reservation is active and store is performed. In this | |
| 195 | + * case cr0.eq bit will be set to 1. | |
| 196 | + * 2. Reservation is not active and store is not performed. | |
| 197 | + * In this case cr0.eq bit will be set to 0. | |
| 198 | + */ | |
| 199 | + if (stepped == 1 && ((regs.gpr[5] == a && (regs.ccr & cr0_eq)) | |
| 200 | + || (regs.gpr[5] != a && !(regs.ccr & cr0_eq)))) | |
| 201 | + show_result("ldarx / stdcx.", "PASS"); | |
| 202 | + else | |
| 203 | + show_result("ldarx / stdcx.", "FAIL (stdcx.)"); | |
| 204 | +} | |
| 205 | + | |
| 206 | +#ifdef CONFIG_PPC_FPU | |
| 207 | +static void __init test_lfsx_stfsx(void) | |
| 208 | +{ | |
| 209 | + struct pt_regs regs; | |
| 210 | + union { | |
| 211 | + float a; | |
| 212 | + int b; | |
| 213 | + } c; | |
| 214 | + int cached_b; | |
| 215 | + int stepped = -1; | |
| 216 | + | |
| 217 | + init_pt_regs(®s); | |
| 218 | + | |
| 219 | + | |
| 220 | + /*** lfsx ***/ | |
| 221 | + | |
| 222 | + c.a = 123.45; | |
| 223 | + cached_b = c.b; | |
| 224 | + | |
| 225 | + regs.gpr[3] = (unsigned long) &c.a; | |
| 226 | + regs.gpr[4] = 0; | |
| 227 | + | |
| 228 | + /* lfsx frt10, r3, r4 */ | |
| 229 | + stepped = emulate_step(®s, TEST_LFSX(10, 3, 4)); | |
| 230 | + | |
| 231 | + if (stepped == 1) | |
| 232 | + show_result("lfsx", "PASS"); | |
| 233 | + else | |
| 234 | + show_result("lfsx", "FAIL"); | |
| 235 | + | |
| 236 | + | |
| 237 | + /*** stfsx ***/ | |
| 238 | + | |
| 239 | + c.a = 678.91; | |
| 240 | + | |
| 241 | + /* stfsx frs10, r3, r4 */ | |
| 242 | + stepped = emulate_step(®s, TEST_STFSX(10, 3, 4)); | |
| 243 | + | |
| 244 | + if (stepped == 1 && c.b == cached_b) | |
| 245 | + show_result("stfsx", "PASS"); | |
| 246 | + else | |
| 247 | + show_result("stfsx", "FAIL"); | |
| 248 | +} | |
| 249 | + | |
| 250 | +static void __init test_lfdx_stfdx(void) | |
| 251 | +{ | |
| 252 | + struct pt_regs regs; | |
| 253 | + union { | |
| 254 | + double a; | |
| 255 | + long b; | |
| 256 | + } c; | |
| 257 | + long cached_b; | |
| 258 | + int stepped = -1; | |
| 259 | + | |
| 260 | + init_pt_regs(®s); | |
| 261 | + | |
| 262 | + | |
| 263 | + /*** lfdx ***/ | |
| 264 | + | |
| 265 | + c.a = 123456.78; | |
| 266 | + cached_b = c.b; | |
| 267 | + | |
| 268 | + regs.gpr[3] = (unsigned long) &c.a; | |
| 269 | + regs.gpr[4] = 0; | |
| 270 | + | |
| 271 | + /* lfdx frt10, r3, r4 */ | |
| 272 | + stepped = emulate_step(®s, TEST_LFDX(10, 3, 4)); | |
| 273 | + | |
| 274 | + if (stepped == 1) | |
| 275 | + show_result("lfdx", "PASS"); | |
| 276 | + else | |
| 277 | + show_result("lfdx", "FAIL"); | |
| 278 | + | |
| 279 | + | |
| 280 | + /*** stfdx ***/ | |
| 281 | + | |
| 282 | + c.a = 987654.32; | |
| 283 | + | |
| 284 | + /* stfdx frs10, r3, r4 */ | |
| 285 | + stepped = emulate_step(®s, TEST_STFDX(10, 3, 4)); | |
| 286 | + | |
| 287 | + if (stepped == 1 && c.b == cached_b) | |
| 288 | + show_result("stfdx", "PASS"); | |
| 289 | + else | |
| 290 | + show_result("stfdx", "FAIL"); | |
| 291 | +} | |
| 292 | +#else | |
| 293 | +static void __init test_lfsx_stfsx(void) | |
| 294 | +{ | |
| 295 | + show_result("lfsx", "SKIP (CONFIG_PPC_FPU is not set)"); | |
| 296 | + show_result("stfsx", "SKIP (CONFIG_PPC_FPU is not set)"); | |
| 297 | +} | |
| 298 | + | |
| 299 | +static void __init test_lfdx_stfdx(void) | |
| 300 | +{ | |
| 301 | + show_result("lfdx", "SKIP (CONFIG_PPC_FPU is not set)"); | |
| 302 | + show_result("stfdx", "SKIP (CONFIG_PPC_FPU is not set)"); | |
| 303 | +} | |
| 304 | +#endif /* CONFIG_PPC_FPU */ | |
| 305 | + | |
| 306 | +#ifdef CONFIG_ALTIVEC | |
| 307 | +static void __init test_lvx_stvx(void) | |
| 308 | +{ | |
| 309 | + struct pt_regs regs; | |
| 310 | + union { | |
| 311 | + vector128 a; | |
| 312 | + u32 b[4]; | |
| 313 | + } c; | |
| 314 | + u32 cached_b[4]; | |
| 315 | + int stepped = -1; | |
| 316 | + | |
| 317 | + init_pt_regs(®s); | |
| 318 | + | |
| 319 | + | |
| 320 | + /*** lvx ***/ | |
| 321 | + | |
| 322 | + cached_b[0] = c.b[0] = 923745; | |
| 323 | + cached_b[1] = c.b[1] = 2139478; | |
| 324 | + cached_b[2] = c.b[2] = 9012; | |
| 325 | + cached_b[3] = c.b[3] = 982134; | |
| 326 | + | |
| 327 | + regs.gpr[3] = (unsigned long) &c.a; | |
| 328 | + regs.gpr[4] = 0; | |
| 329 | + | |
| 330 | + /* lvx vrt10, r3, r4 */ | |
| 331 | + stepped = emulate_step(®s, TEST_LVX(10, 3, 4)); | |
| 332 | + | |
| 333 | + if (stepped == 1) | |
| 334 | + show_result("lvx", "PASS"); | |
| 335 | + else | |
| 336 | + show_result("lvx", "FAIL"); | |
| 337 | + | |
| 338 | + | |
| 339 | + /*** stvx ***/ | |
| 340 | + | |
| 341 | + c.b[0] = 4987513; | |
| 342 | + c.b[1] = 84313948; | |
| 343 | + c.b[2] = 71; | |
| 344 | + c.b[3] = 498532; | |
| 345 | + | |
| 346 | + /* stvx vrs10, r3, r4 */ | |
| 347 | + stepped = emulate_step(®s, TEST_STVX(10, 3, 4)); | |
| 348 | + | |
| 349 | + if (stepped == 1 && cached_b[0] == c.b[0] && cached_b[1] == c.b[1] && | |
| 350 | + cached_b[2] == c.b[2] && cached_b[3] == c.b[3]) | |
| 351 | + show_result("stvx", "PASS"); | |
| 352 | + else | |
| 353 | + show_result("stvx", "FAIL"); | |
| 354 | +} | |
| 355 | +#else | |
| 356 | +static void __init test_lvx_stvx(void) | |
| 357 | +{ | |
| 358 | + show_result("lvx", "SKIP (CONFIG_ALTIVEC is not set)"); | |
| 359 | + show_result("stvx", "SKIP (CONFIG_ALTIVEC is not set)"); | |
| 360 | +} | |
| 361 | +#endif /* CONFIG_ALTIVEC */ | |
| 362 | + | |
| 363 | +#ifdef CONFIG_VSX | |
| 364 | +static void __init test_lxvd2x_stxvd2x(void) | |
| 365 | +{ | |
| 366 | + struct pt_regs regs; | |
| 367 | + union { | |
| 368 | + vector128 a; | |
| 369 | + u32 b[4]; | |
| 370 | + } c; | |
| 371 | + u32 cached_b[4]; | |
| 372 | + int stepped = -1; | |
| 373 | + | |
| 374 | + init_pt_regs(®s); | |
| 375 | + | |
| 376 | + | |
| 377 | + /*** lxvd2x ***/ | |
| 378 | + | |
| 379 | + cached_b[0] = c.b[0] = 18233; | |
| 380 | + cached_b[1] = c.b[1] = 34863571; | |
| 381 | + cached_b[2] = c.b[2] = 834; | |
| 382 | + cached_b[3] = c.b[3] = 6138911; | |
| 383 | + | |
| 384 | + regs.gpr[3] = (unsigned long) &c.a; | |
| 385 | + regs.gpr[4] = 0; | |
| 386 | + | |
| 387 | + /* lxvd2x vsr39, r3, r4 */ | |
| 388 | + stepped = emulate_step(®s, TEST_LXVD2X(39, 3, 4)); | |
| 389 | + | |
| 390 | + if (stepped == 1) | |
| 391 | + show_result("lxvd2x", "PASS"); | |
| 392 | + else | |
| 393 | + show_result("lxvd2x", "FAIL"); | |
| 394 | + | |
| 395 | + | |
| 396 | + /*** stxvd2x ***/ | |
| 397 | + | |
| 398 | + c.b[0] = 21379463; | |
| 399 | + c.b[1] = 87; | |
| 400 | + c.b[2] = 374234; | |
| 401 | + c.b[3] = 4; | |
| 402 | + | |
| 403 | + /* stxvd2x vsr39, r3, r4 */ | |
| 404 | + stepped = emulate_step(®s, TEST_STXVD2X(39, 3, 4)); | |
| 405 | + | |
| 406 | + if (stepped == 1 && cached_b[0] == c.b[0] && cached_b[1] == c.b[1] && | |
| 407 | + cached_b[2] == c.b[2] && cached_b[3] == c.b[3]) | |
| 408 | + show_result("stxvd2x", "PASS"); | |
| 409 | + else | |
| 410 | + show_result("stxvd2x", "FAIL"); | |
| 411 | +} | |
| 412 | +#else | |
| 413 | +static void __init test_lxvd2x_stxvd2x(void) | |
| 414 | +{ | |
| 415 | + show_result("lxvd2x", "SKIP (CONFIG_VSX is not set)"); | |
| 416 | + show_result("stxvd2x", "SKIP (CONFIG_VSX is not set)"); | |
| 417 | +} | |
| 418 | +#endif /* CONFIG_VSX */ | |
| 419 | + | |
| 420 | +static int __init test_emulate_step(void) | |
| 421 | +{ | |
| 422 | + test_ld(); | |
| 423 | + test_lwz(); | |
| 424 | + test_lwzx(); | |
| 425 | + test_std(); | |
| 426 | + test_ldarx_stdcx(); | |
| 427 | + test_lfsx_stfsx(); | |
| 428 | + test_lfdx_stfdx(); | |
| 429 | + test_lvx_stvx(); | |
| 430 | + test_lxvd2x_stxvd2x(); | |
| 431 | + | |
| 432 | + return 0; | |
| 433 | +} | |
| 434 | +late_initcall(test_emulate_step); | |
| @@ -356,18 +356,42 @@ static void early_check_vec5(void) · 355 unmodified lines | ||
| 356 | 356 | unsigned long root, chosen; |
| 357 | 357 | int size; |
| 358 | 358 | const u8 *vec5; |
| 359 | + u8 mmu_supported; | |
| 359 | 360 | |
| 360 | 361 | root = of_get_flat_dt_root(); |
| 361 | 362 | chosen = of_get_flat_dt_subnode_by_name(root, "chosen"); |
| 362 | − if (chosen == -FDT_ERR_NOTFOUND) | |
| 363 | + if (chosen == -FDT_ERR_NOTFOUND) { | |
| 364 | + cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX; | |
| 363 | 365 | return; |
| 366 | + } | |
| 364 | 367 | vec5 = of_get_flat_dt_prop(chosen, "ibm,architecture-vec-5", &size); |
| 365 | − if (!vec5) | |
| 368 | + if (!vec5) { | |
| 369 | + cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX; | |
| 366 | 370 | return; |
| 367 | − if (size <= OV5_INDX(OV5_MMU_RADIX_300) || | |
| 368 | − !(vec5[OV5_INDX(OV5_MMU_RADIX_300)] & OV5_FEAT(OV5_MMU_RADIX_300))) | |
| 369 | − /* Hypervisor doesn't support radix */ | |
| 371 | + } | |
| 372 | + if (size <= OV5_INDX(OV5_MMU_SUPPORT)) { | |
| 370 | 373 | cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX; |
| 374 | + return; | |
| 375 | + } | |
| 376 | + | |
| 377 | + /* Check for supported configuration */ | |
| 378 | + mmu_supported = vec5[OV5_INDX(OV5_MMU_SUPPORT)] & | |
| 379 | + OV5_FEAT(OV5_MMU_SUPPORT); | |
| 380 | + if (mmu_supported == OV5_FEAT(OV5_MMU_RADIX)) { | |
| 381 | + /* Hypervisor only supports radix - check enabled && GTSE */ | |
| 382 | + if (!early_radix_enabled()) { | |
| 383 | + pr_warn("WARNING: Ignoring cmdline option disable_radix\n"); | |
| 384 | + } | |
| 385 | + if (!(vec5[OV5_INDX(OV5_RADIX_GTSE)] & | |
| 386 | + OV5_FEAT(OV5_RADIX_GTSE))) { | |
| 387 | + pr_warn("WARNING: Hypervisor doesn't support RADIX with GTSE\n"); | |
| 388 | + } | |
| 389 | + /* Do radix anyway - the hypervisor said we had to */ | |
| 390 | + cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX; | |
| 391 | + } else if (mmu_supported == OV5_FEAT(OV5_MMU_HASH)) { | |
| 392 | + /* Hypervisor only supports hash - disable radix */ | |
| 393 | + cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX; | |
| 394 | + } | |
| 371 | 395 | } |
| 372 | 396 | |
| 373 | 397 | void __init mmu_early_init_devtree(void) |
| @@ -383,7 +407,7 @@ void __init mmu_early_init_devtree(void) · 9 unmodified lines | ||
| 383 | 407 | * even though the ibm,architecture-vec-5 property created by |
| 384 | 408 | * skiboot doesn't have the necessary bits set. |
| 385 | 409 | */ |
| 386 | − if (early_radix_enabled() && !(mfmsr() & MSR_HV)) | |
| 410 | + if (!(mfmsr() & MSR_HV)) | |
| 387 | 411 | early_check_vec5(); |
| 388 | 412 | |
| 389 | 413 | if (early_radix_enabled()) |
| @@ -186,6 +186,10 @@ static void __init radix_init_pgtable(void) · 185 unmodified lines | ||
| 186 | 186 | */ |
| 187 | 187 | register_process_table(__pa(process_tb), 0, PRTB_SIZE_SHIFT - 12); |
| 188 | 188 | pr_info("Process table %p and radix root for kernel: %p\n", process_tb, init_mm.pgd); |
| 189 | + asm volatile("ptesync" : : : "memory"); | |
| 190 | + asm volatile(PPC_TLBIE_5(%0,%1,2,1,1) : : | |
| 191 | + "r" (TLBIEL_INVAL_SET_LPID), "r" (0)); | |
| 192 | + asm volatile("eieio; tlbsync; ptesync" : : : "memory"); | |
| 189 | 193 | } |
| 190 | 194 | |
| 191 | 195 | static void __init radix_init_partition_table(void) |
| @@ -188,6 +188,8 @@ static inline void perf_get_data_addr(struct pt_regs *regs, u64 *addrp) · 187 unmodified lines | ||
| 188 | 188 | sdsync = POWER7P_MMCRA_SDAR_VALID; |
| 189 | 189 | else if (ppmu->flags & PPMU_ALT_SIPR) |
| 190 | 190 | sdsync = POWER6_MMCRA_SDSYNC; |
| 191 | + else if (ppmu->flags & PPMU_NO_SIAR) | |
| 192 | + sdsync = MMCRA_SAMPLE_ENABLE; | |
| 191 | 193 | else |
| 192 | 194 | sdsync = MMCRA_SDSYNC; |
| 193 | 195 | |
| @@ -65,12 +65,41 @@ static bool is_event_valid(u64 event) · 64 unmodified lines | ||
| 65 | 65 | return !(event & ~valid_mask); |
| 66 | 66 | } |
| 67 | 67 | |
| 68 | −static u64 mmcra_sdar_mode(u64 event) | |
| 68 | +static inline bool is_event_marked(u64 event) | |
| 69 | 69 | { |
| 70 | − if (cpu_has_feature(CPU_FTR_ARCH_300) && !cpu_has_feature(CPU_FTR_POWER9_DD1)) | |
| 71 | − return p9_SDAR_MODE(event) << MMCRA_SDAR_MODE_SHIFT; | |
| 70 | + if (event & EVENT_IS_MARKED) | |
| 71 | + return true; | |
| 72 | + | |
| 73 | + return false; | |
| 74 | +} | |
| 72 | 75 | |
| 73 | − return MMCRA_SDAR_MODE_TLB; | |
| 76 | +static void mmcra_sdar_mode(u64 event, unsigned long *mmcra) | |
| 77 | +{ | |
| 78 | + /* | |
| 79 | + * MMCRA[SDAR_MODE] specifices how the SDAR should be updated in | |
| 80 | + * continous sampling mode. | |
| 81 | + * | |
| 82 | + * Incase of Power8: | |
| 83 | + * MMCRA[SDAR_MODE] will be programmed as "0b01" for continous sampling | |
| 84 | + * mode and will be un-changed when setting MMCRA[63] (Marked events). | |
| 85 | + * | |
| 86 | + * Incase of Power9: | |
| 87 | + * Marked event: MMCRA[SDAR_MODE] will be set to 0b00 ('No Updates'), | |
| 88 | + * or if group already have any marked events. | |
| 89 | + * Non-Marked events (for DD1): | |
| 90 | + * MMCRA[SDAR_MODE] will be set to 0b01 | |
| 91 | + * For rest | |
| 92 | + * MMCRA[SDAR_MODE] will be set from event code. | |
| 93 | + */ | |
| 94 | + if (cpu_has_feature(CPU_FTR_ARCH_300)) { | |
| 95 | + if (is_event_marked(event) || (*mmcra & MMCRA_SAMPLE_ENABLE)) | |
| 96 | + *mmcra &= MMCRA_SDAR_MODE_NO_UPDATES; | |
| 97 | + else if (!cpu_has_feature(CPU_FTR_POWER9_DD1)) | |
| 98 | + *mmcra |= p9_SDAR_MODE(event) << MMCRA_SDAR_MODE_SHIFT; | |
| 99 | + else if (cpu_has_feature(CPU_FTR_POWER9_DD1)) | |
| 100 | + *mmcra |= MMCRA_SDAR_MODE_TLB; | |
| 101 | + } else | |
| 102 | + *mmcra |= MMCRA_SDAR_MODE_TLB; | |
| 74 | 103 | } |
| 75 | 104 | |
| 76 | 105 | static u64 thresh_cmp_val(u64 value) |
| @@ -180,7 +209,7 @@ int isa207_get_constraint(u64 event, unsigned long *maskp, unsigned long *valp) · 103 unmodified lines | ||
| 180 | 209 | value |= CNST_L1_QUAL_VAL(cache); |
| 181 | 210 | } |
| 182 | 211 | |
| 183 | − if (event & EVENT_IS_MARKED) { | |
| 212 | + if (is_event_marked(event)) { | |
| 184 | 213 | mask |= CNST_SAMPLE_MASK; |
| 185 | 214 | value |= CNST_SAMPLE_VAL(event >> EVENT_SAMPLE_SHIFT); |
| 186 | 215 | } |
| @@ -276,7 +305,7 @@ int isa207_compute_mmcr(u64 event[], int n_ev, · 89 unmodified lines | ||
| 276 | 305 | } |
| 277 | 306 | |
| 278 | 307 | /* In continuous sampling mode, update SDAR on TLB miss */ |
| 279 | − mmcra |= mmcra_sdar_mode(event[i]); | |
| 308 | + mmcra_sdar_mode(event[i], &mmcra); | |
| 280 | 309 | |
| 281 | 310 | if (event[i] & EVENT_IS_L1) { |
| 282 | 311 | cache = event[i] >> EVENT_CACHE_SEL_SHIFT; |
| @@ -285,7 +314,7 @@ int isa207_compute_mmcr(u64 event[], int n_ev, · 2 unmodified lines | ||
| 285 | 314 | mmcr1 |= (cache & 1) << MMCR1_DC_QUAL_SHIFT; |
| 286 | 315 | } |
| 287 | 316 | |
| 288 | − if (event[i] & EVENT_IS_MARKED) { | |
| 317 | + if (is_event_marked(event[i])) { | |
| 289 | 318 | mmcra |= MMCRA_SAMPLE_ENABLE; |
| 290 | 319 | |
| 291 | 320 | val = (event[i] >> EVENT_SAMPLE_SHIFT) & EVENT_SAMPLE_MASK; |
| @@ -246,6 +246,7 @@ · 245 unmodified lines | ||
| 246 | 246 | #define MMCRA_THR_CMP_SHIFT 32 |
| 247 | 247 | #define MMCRA_SDAR_MODE_SHIFT 42 |
| 248 | 248 | #define MMCRA_SDAR_MODE_TLB (1ull << MMCRA_SDAR_MODE_SHIFT) |
| 249 | +#define MMCRA_SDAR_MODE_NO_UPDATES ~(0x3ull << MMCRA_SDAR_MODE_SHIFT) | |
| 249 | 250 | #define MMCRA_IFM_SHIFT 30 |
| 250 | 251 | |
| 251 | 252 | /* MMCR1 Threshold Compare bit constant for power9 */ |
| @@ -39,8 +39,8 @@ opal_tracepoint_refcount: · 38 unmodified lines | ||
| 39 | 39 | BEGIN_FTR_SECTION; \ |
| 40 | 40 | b 1f; \ |
| 41 | 41 | END_FTR_SECTION(0, 1); \ |
| 42 | − ld r12,opal_tracepoint_refcount@toc(r2); \ | |
| 43 | − cmpdi r12,0; \ | |
| 42 | + ld r11,opal_tracepoint_refcount@toc(r2); \ | |
| 43 | + cmpdi r11,0; \ | |
| 44 | 44 | bne- LABEL; \ |
| 45 | 45 | 1: |
| 46 | 46 | |
| @@ -395,7 +395,6 @@ static int opal_recover_mce(struct pt_regs *regs, · 394 unmodified lines | ||
| 395 | 395 | struct machine_check_event *evt) |
| 396 | 396 | { |
| 397 | 397 | int recovered = 0; |
| 398 | − uint64_t ea = get_mce_fault_addr(evt); | |
| 399 | 398 | |
| 400 | 399 | if (!(regs->msr & MSR_RI)) { |
| 401 | 400 | /* If MSR_RI isn't set, we cannot recover */ |
| @@ -404,26 +403,18 @@ static int opal_recover_mce(struct pt_regs *regs, · 2 unmodified lines | ||
| 404 | 403 | } else if (evt->disposition == MCE_DISPOSITION_RECOVERED) { |
| 405 | 404 | /* Platform corrected itself */ |
| 406 | 405 | recovered = 1; |
| 407 | − } else if (ea && !is_kernel_addr(ea)) { | |
| 406 | + } else if (evt->severity == MCE_SEV_FATAL) { | |
| 407 | + /* Fatal machine check */ | |
| 408 | + pr_err("Machine check interrupt is fatal\n"); | |
| 409 | + recovered = 0; | |
| 410 | + } else if ((evt->severity == MCE_SEV_ERROR_SYNC) && | |
| 411 | + (user_mode(regs) && !is_global_init(current))) { | |
| 408 | 412 | /* |
| 409 | − * Faulting address is not in kernel text. We should be fine. | |
| 410 | − * We need to find which process uses this address. | |
| 411 | 413 | * For now, kill the task if we have received exception when |
| 412 | 414 | * in userspace. |
| 413 | 415 | * |
| 414 | 416 | * TODO: Queue up this address for hwpoisioning later. |
| 415 | 417 | */ |
| 416 | − if (user_mode(regs) && !is_global_init(current)) { | |
| 417 | − _exception(SIGBUS, regs, BUS_MCEERR_AR, regs->nip); | |
| 418 | − recovered = 1; | |
| 419 | − } else | |
| 420 | − recovered = 0; | |
| 421 | − } else if (user_mode(regs) && !is_global_init(current) && | |
| 422 | − evt->severity == MCE_SEV_ERROR_SYNC) { | |
| 423 | − /* | |
| 424 | − * If we have received a synchronous error when in userspace | |
| 425 | − * kill the task. | |
| 426 | − */ | |
| 427 | 418 | _exception(SIGBUS, regs, BUS_MCEERR_AR, regs->nip); |
| 428 | 419 | recovered = 1; |
| 429 | 420 | } |
| @@ -1775,17 +1775,20 @@ static u64 pnv_pci_ioda_dma_get_required_mask(struct pci_dev *pdev) · 1774 unmodified lines | ||
| 1775 | 1775 | } |
| 1776 | 1776 | |
| 1777 | 1777 | static void pnv_ioda_setup_bus_dma(struct pnv_ioda_pe *pe, |
| 1778 | − struct pci_bus *bus) | |
| 1778 | + struct pci_bus *bus, | |
| 1779 | + bool add_to_group) | |
| 1779 | 1780 | { |
| 1780 | 1781 | struct pci_dev *dev; |
| 1781 | 1782 | |
| 1782 | 1783 | list_for_each_entry(dev, &bus->devices, bus_list) { |
| 1783 | 1784 | set_iommu_table_base(&dev->dev, pe->table_group.tables[0]); |
| 1784 | 1785 | set_dma_offset(&dev->dev, pe->tce_bypass_base); |
| 1785 | − iommu_add_device(&dev->dev); | |
| 1786 | + if (add_to_group) | |
| 1787 | + iommu_add_device(&dev->dev); | |
| 1786 | 1788 | |
| 1787 | 1789 | if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate) |
| 1788 | − pnv_ioda_setup_bus_dma(pe, dev->subordinate); | |
| 1790 | + pnv_ioda_setup_bus_dma(pe, dev->subordinate, | |
| 1791 | + add_to_group); | |
| 1789 | 1792 | } |
| 1790 | 1793 | } |
| 1791 | 1794 | |
| @@ -2191,7 +2194,7 @@ found: · 399 unmodified lines | ||
| 2191 | 2194 | set_iommu_table_base(&pe->pdev->dev, tbl); |
| 2192 | 2195 | iommu_add_device(&pe->pdev->dev); |
| 2193 | 2196 | } else if (pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)) |
| 2194 | − pnv_ioda_setup_bus_dma(pe, pe->pbus); | |
| 2197 | + pnv_ioda_setup_bus_dma(pe, pe->pbus, true); | |
| 2195 | 2198 | |
| 2196 | 2199 | return; |
| 2197 | 2200 | fail: |
| @@ -2426,6 +2429,8 @@ static void pnv_ioda2_take_ownership(struct iommu_table_group *table_group) · 228 unmodified lines | ||
| 2426 | 2429 | |
| 2427 | 2430 | pnv_pci_ioda2_set_bypass(pe, false); |
| 2428 | 2431 | pnv_pci_ioda2_unset_window(&pe->table_group, 0); |
| 2432 | + if (pe->pbus) | |
| 2433 | + pnv_ioda_setup_bus_dma(pe, pe->pbus, false); | |
| 2429 | 2434 | pnv_ioda2_table_free(tbl); |
| 2430 | 2435 | } |
| 2431 | 2436 | |
| @@ -2435,6 +2440,8 @@ static void pnv_ioda2_release_ownership(struct iommu_table_group *table_group) · 3 unmodified lines | ||
| 2435 | 2440 | table_group); |
| 2436 | 2441 | |
| 2437 | 2442 | pnv_pci_ioda2_setup_default_config(pe); |
| 2443 | + if (pe->pbus) | |
| 2444 | + pnv_ioda_setup_bus_dma(pe, pe->pbus, false); | |
| 2438 | 2445 | } |
| 2439 | 2446 | |
| 2440 | 2447 | static struct iommu_table_group_ops pnv_pci_ioda2_ops = { |
| @@ -2624,6 +2631,9 @@ static long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset, · 183 unmodified lines | ||
| 2624 | 2631 | level_shift = entries_shift + 3; |
| 2625 | 2632 | level_shift = max_t(unsigned, level_shift, PAGE_SHIFT); |
| 2626 | 2633 | |
| 2634 | + if ((level_shift - 3) * levels + page_shift >= 60) | |
| 2635 | + return -EINVAL; | |
| 2636 | + | |
| 2627 | 2637 | /* Allocate TCE table */ |
| 2628 | 2638 | addr = pnv_pci_ioda2_table_do_alloc_pages(nid, level_shift, |
| 2629 | 2639 | levels, tce_table_size, &offset, &total_allocated); |
| @@ -2728,7 +2738,7 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb, · 98 unmodified lines | ||
| 2728 | 2738 | if (pe->flags & PNV_IODA_PE_DEV) |
| 2729 | 2739 | iommu_add_device(&pe->pdev->dev); |
| 2730 | 2740 | else if (pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)) |
| 2731 | − pnv_ioda_setup_bus_dma(pe, pe->pbus); | |
| 2741 | + pnv_ioda_setup_bus_dma(pe, pe->pbus, true); | |
| 2732 | 2742 | } |
| 2733 | 2743 | |
| 2734 | 2744 | #ifdef CONFIG_PCI_MSI |
| @@ -116,13 +116,13 @@ dt_offset: · 115 unmodified lines | ||
| 116 | 116 | |
| 117 | 117 | .data |
| 118 | 118 | .balign 8 |
| 119 | −.globl sha256_digest | |
| 120 | −sha256_digest: | |
| 119 | +.globl purgatory_sha256_digest | |
| 120 | +purgatory_sha256_digest: | |
| 121 | 121 | .skip 32 |
| 122 | − .size sha256_digest, . - sha256_digest | |
| 122 | + .size purgatory_sha256_digest, . - purgatory_sha256_digest | |
| 123 | 123 | |
| 124 | 124 | .balign 8 |
| 125 | −.globl sha_regions | |
| 126 | −sha_regions: | |
| 125 | +.globl purgatory_sha_regions | |
| 126 | +purgatory_sha_regions: | |
| 127 | 127 | .skip 8 * 2 * 16 |
| 128 | − .size sha_regions, . - sha_regions | |
| 128 | + .size purgatory_sha_regions, . - purgatory_sha_regions | |
| @@ -274,7 +274,9 @@ failed: · 273 unmodified lines | ||
| 274 | 274 | if (bank->disk->major > 0) |
| 275 | 275 | unregister_blkdev(bank->disk->major, |
| 276 | 276 | bank->disk->disk_name); |
| 277 | − del_gendisk(bank->disk); | |
| 277 | + if (bank->disk->flags & GENHD_FL_UP) | |
| 278 | + del_gendisk(bank->disk); | |
| 279 | + put_disk(bank->disk); | |
| 278 | 280 | } |
| 279 | 281 | device->dev.platform_data = NULL; |
| 280 | 282 | if (bank->io_addr != 0) |
| @@ -299,6 +301,7 @@ axon_ram_remove(struct platform_device *device) · 18 unmodified lines | ||
| 299 | 301 | device_remove_file(&device->dev, &dev_attr_ecc); |
| 300 | 302 | free_irq(bank->irq_id, device); |
| 301 | 303 | del_gendisk(bank->disk); |
| 304 | + put_disk(bank->disk); | |
| 302 | 305 | iounmap((void __iomem *) bank->io_addr); |
| 303 | 306 | kfree(bank); |
| 304 | 307 | |
| @@ -91,6 +91,16 @@ static unsigned int icp_opal_get_irq(void) · 90 unmodified lines | ||
| 91 | 91 | |
| 92 | 92 | static void icp_opal_set_cpu_priority(unsigned char cppr) |
| 93 | 93 | { |
| 94 | + /* | |
| 95 | + * Here be dragons. The caller has asked to allow only IPI's and not | |
| 96 | + * external interrupts. But OPAL XIVE doesn't support that. So instead | |
| 97 | + * of allowing no interrupts allow all. That's still not right, but | |
| 98 | + * currently the only caller who does this is xics_migrate_irqs_away() | |
| 99 | + * and it works in that case. | |
| 100 | + */ | |
| 101 | + if (cppr >= DEFAULT_PRIORITY) | |
| 102 | + cppr = LOWEST_PRIORITY; | |
| 103 | + | |
| 94 | 104 | xics_set_base_cppr(cppr); |
| 95 | 105 | opal_int_set_cppr(cppr); |
| 96 | 106 | iosync(); |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | #include <linux/of.h> |
| 21 | 21 | #include <linux/slab.h> |
| 22 | 22 | #include <linux/spinlock.h> |
| 23 | +#include <linux/delay.h> | |
| 23 | 24 | |
| 24 | 25 | #include <asm/prom.h> |
| 25 | 26 | #include <asm/io.h> |
| @@ -198,9 +199,6 @@ void xics_migrate_irqs_away(void) · 172 unmodified lines | ||
| 198 | 199 | /* Remove ourselves from the global interrupt queue */ |
| 199 | 200 | xics_set_cpu_giq(xics_default_distrib_server, 0); |
| 200 | 201 | |
| 201 | − /* Allow IPIs again... */ | |
| 202 | − icp_ops->set_priority(DEFAULT_PRIORITY); | |
| 203 | − | |
| 204 | 202 | for_each_irq_desc(virq, desc) { |
| 205 | 203 | struct irq_chip *chip; |
| 206 | 204 | long server; |
| @@ -255,6 +253,19 @@ void xics_migrate_irqs_away(void) · 48 unmodified lines | ||
| 255 | 253 | unlock: |
| 256 | 254 | raw_spin_unlock_irqrestore(&desc->lock, flags); |
| 257 | 255 | } |
| 256 | + | |
| 257 | + /* Allow "sufficient" time to drop any inflight IRQ's */ | |
| 258 | + mdelay(5); | |
| 259 | + | |
| 260 | + /* | |
| 261 | + * Allow IPIs again. This is done at the very end, after migrating all | |
| 262 | + * interrupts, the expectation is that we'll only get woken up by an IPI | |
| 263 | + * interrupt beyond this point, but leave externals masked just to be | |
| 264 | + * safe. If we're using icp-opal this may actually allow all | |
| 265 | + * interrupts anyway, but that should be OK. | |
| 266 | + */ | |
| 267 | + icp_ops->set_priority(DEFAULT_PRIORITY); | |
| 268 | + | |
| 258 | 269 | } |
| 259 | 270 | #endif /* CONFIG_HOTPLUG_CPU */ |
| 260 | 271 | |
| @@ -474,8 +474,11 @@ static int ctr_paes_crypt(struct blkcipher_desc *desc, unsigned long modifier, · 473 unmodified lines | ||
| 474 | 474 | ret = blkcipher_walk_done(desc, walk, nbytes - n); |
| 475 | 475 | } |
| 476 | 476 | if (k < n) { |
| 477 | − if (__ctr_paes_set_key(ctx) != 0) | |
| 477 | + if (__ctr_paes_set_key(ctx) != 0) { | |
| 478 | + if (locked) | |
| 479 | + spin_unlock(&ctrblk_lock); | |
| 478 | 480 | return blkcipher_walk_done(desc, walk, -EIO); |
| 481 | + } | |
| 479 | 482 | } |
| 480 | 483 | } |
| 481 | 484 | if (locked) |
| @@ -8,31 +8,27 @@ · 7 unmodified lines | ||
| 8 | 8 | #define _S390_CPUTIME_H |
| 9 | 9 | |
| 10 | 10 | #include <linux/types.h> |
| 11 | −#include <asm/div64.h> | |
| 11 | +#include <asm/timex.h> | |
| 12 | 12 | |
| 13 | 13 | #define CPUTIME_PER_USEC 4096ULL |
| 14 | 14 | #define CPUTIME_PER_SEC (CPUTIME_PER_USEC * USEC_PER_SEC) |
| 15 | 15 | |
| 16 | 16 | /* We want to use full resolution of the CPU timer: 2**-12 micro-seconds. */ |
| 17 | 17 | |
| 18 | −typedef unsigned long long __nocast cputime_t; | |
| 19 | −typedef unsigned long long __nocast cputime64_t; | |
| 20 | − | |
| 21 | 18 | #define cmpxchg_cputime(ptr, old, new) cmpxchg64(ptr, old, new) |
| 22 | 19 | |
| 23 | −static inline unsigned long __div(unsigned long long n, unsigned long base) | |
| 24 | −{ | |
| 25 | − return n / base; | |
| 26 | −} | |
| 27 | − | |
| 28 | 20 | /* |
| 29 | − * Convert cputime to microseconds and back. | |
| 21 | + * Convert cputime to microseconds. | |
| 30 | 22 | */ |
| 31 | −static inline unsigned int cputime_to_usecs(const cputime_t cputime) | |
| 23 | +static inline u64 cputime_to_usecs(const u64 cputime) | |
| 32 | 24 | { |
| 33 | − return (__force unsigned long long) cputime >> 12; | |
| 25 | + return cputime >> 12; | |
| 34 | 26 | } |
| 35 | 27 | |
| 28 | +/* | |
| 29 | + * Convert cputime to nanoseconds. | |
| 30 | + */ | |
| 31 | +#define cputime_to_nsecs(cputime) tod_to_ns(cputime) | |
| 36 | 32 | |
| 37 | 33 | u64 arch_cpu_idle_time(int cpu); |
| 38 | 34 | |
| @@ -24,6 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | * the S390 page table tree. |
| 25 | 25 | */ |
| 26 | 26 | #ifndef __ASSEMBLY__ |
| 27 | +#include <asm-generic/5level-fixup.h> | |
| 27 | 28 | #include <linux/sched.h> |
| 28 | 29 | #include <linux/mm_types.h> |
| 29 | 30 | #include <linux/page-flags.h> |
| @@ -206,20 +206,16 @@ static inline unsigned long long get_tod_clock_monotonic(void) · 205 unmodified lines | ||
| 206 | 206 | * ns = (todval * 125) >> 9; |
| 207 | 207 | * |
| 208 | 208 | * In order to avoid an overflow with the multiplication we can rewrite this. |
| 209 | − * With a split todval == 2^32 * th + tl (th upper 32 bits, tl lower 32 bits) | |
| 209 | + * With a split todval == 2^9 * th + tl (th upper 55 bits, tl lower 9 bits) | |
| 210 | 210 | * we end up with |
| 211 | 211 | * |
| 212 | − * ns = ((2^32 * th + tl) * 125 ) >> 9; | |
| 213 | − * -> ns = (2^23 * th * 125) + ((tl * 125) >> 9); | |
| 212 | + * ns = ((2^9 * th + tl) * 125 ) >> 9; | |
| 213 | + * -> ns = (th * 125) + ((tl * 125) >> 9); | |
| 214 | 214 | * |
| 215 | 215 | */ |
| 216 | 216 | static inline unsigned long long tod_to_ns(unsigned long long todval) |
| 217 | 217 | { |
| 218 | − unsigned long long ns; | |
| 219 | − | |
| 220 | − ns = ((todval >> 32) << 23) * 125; | |
| 221 | − ns += ((todval & 0xffffffff) * 125) >> 9; | |
| 222 | − return ns; | |
| 218 | + return ((todval >> 9) * 125) + (((todval & 0x1ff) * 125) >> 9); | |
| 223 | 219 | } |
| 224 | 220 | |
| 225 | 221 | #endif |
| @@ -313,7 +313,9 @@ · 312 unmodified lines | ||
| 313 | 313 | #define __NR_copy_file_range 375 |
| 314 | 314 | #define __NR_preadv2 376 |
| 315 | 315 | #define __NR_pwritev2 377 |
| 316 | −#define NR_syscalls 378 | |
| 316 | +/* Number 378 is reserved for guarded storage */ | |
| 317 | +#define __NR_statx 379 | |
| 318 | +#define NR_syscalls 380 | |
| 317 | 319 | |
| 318 | 320 | /* |
| 319 | 321 | * There are some system calls that are not present on 64 bit, some |
| @@ -178,3 +178,4 @@ COMPAT_SYSCALL_WRAP3(getpeername, int, fd, struct sockaddr __user *, usockaddr, · 177 unmodified lines | ||
| 178 | 178 | COMPAT_SYSCALL_WRAP6(sendto, int, fd, void __user *, buff, size_t, len, unsigned int, flags, struct sockaddr __user *, addr, int, addr_len); |
| 179 | 179 | COMPAT_SYSCALL_WRAP3(mlock2, unsigned long, start, size_t, len, int, flags); |
| 180 | 180 | COMPAT_SYSCALL_WRAP6(copy_file_range, int, fd_in, loff_t __user *, off_in, int, fd_out, loff_t __user *, off_out, size_t, len, unsigned int, flags); |
| 181 | +COMPAT_SYSCALL_WRAP5(statx, int, dfd, const char __user *, path, unsigned, flags, unsigned, mask, struct statx __user *, buffer); | |
| @@ -490,7 +490,7 @@ ENTRY(pgm_check_handler) · 489 unmodified lines | ||
| 490 | 490 | jnz .Lpgm_svcper # -> single stepped svc |
| 491 | 491 | 1: CHECK_STACK STACK_SIZE,__LC_SAVE_AREA_SYNC |
| 492 | 492 | aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE) |
| 493 | − j 3f | |
| 493 | + j 4f | |
| 494 | 494 | 2: UPDATE_VTIME %r14,%r15,__LC_SYNC_ENTER_TIMER |
| 495 | 495 | lg %r15,__LC_KERNEL_STACK |
| 496 | 496 | lgr %r14,%r12 |
| @@ -499,8 +499,8 @@ ENTRY(pgm_check_handler) · 2 unmodified lines | ||
| 499 | 499 | tm __LC_PGM_ILC+2,0x02 # check for transaction abort |
| 500 | 500 | jz 3f |
| 501 | 501 | mvc __THREAD_trap_tdb(256,%r14),0(%r13) |
| 502 | −3: la %r11,STACK_FRAME_OVERHEAD(%r15) | |
| 503 | − stg %r10,__THREAD_last_break(%r14) | |
| 502 | +3: stg %r10,__THREAD_last_break(%r14) | |
| 503 | +4: la %r11,STACK_FRAME_OVERHEAD(%r15) | |
| 504 | 504 | stmg %r0,%r7,__PT_R0(%r11) |
| 505 | 505 | mvc __PT_R8(64,%r11),__LC_SAVE_AREA_SYNC |
| 506 | 506 | stmg %r8,%r9,__PT_PSW(%r11) |
| @@ -509,14 +509,14 @@ ENTRY(pgm_check_handler) · 2 unmodified lines | ||
| 509 | 509 | xc __PT_FLAGS(8,%r11),__PT_FLAGS(%r11) |
| 510 | 510 | stg %r10,__PT_ARGS(%r11) |
| 511 | 511 | tm __LC_PGM_ILC+3,0x80 # check for per exception |
| 512 | − jz 4f | |
| 512 | + jz 5f | |
| 513 | 513 | tmhh %r8,0x0001 # kernel per event ? |
| 514 | 514 | jz .Lpgm_kprobe |
| 515 | 515 | oi __PT_FLAGS+7(%r11),_PIF_PER_TRAP |
| 516 | 516 | mvc __THREAD_per_address(8,%r14),__LC_PER_ADDRESS |
| 517 | 517 | mvc __THREAD_per_cause(2,%r14),__LC_PER_CODE |
| 518 | 518 | mvc __THREAD_per_paid(1,%r14),__LC_PER_ACCESS_ID |
| 519 | −4: REENABLE_IRQS | |
| 519 | +5: REENABLE_IRQS | |
| 520 | 520 | xc __SF_BACKCHAIN(8,%r15),__SF_BACKCHAIN(%r15) |
| 521 | 521 | larl %r1,pgm_check_table |
| 522 | 522 | llgh %r10,__PT_INT_CODE+2(%r11) |
| @@ -564,6 +564,8 @@ static struct kset *ipl_kset; · 563 unmodified lines | ||
| 564 | 564 | |
| 565 | 565 | static void __ipl_run(void *unused) |
| 566 | 566 | { |
| 567 | + if (MACHINE_IS_LPAR && ipl_info.type == IPL_TYPE_CCW) | |
| 568 | + diag308(DIAG308_LOAD_NORMAL_DUMP, NULL); | |
| 567 | 569 | diag308(DIAG308_LOAD_CLEAR, NULL); |
| 568 | 570 | if (MACHINE_IS_VM) |
| 569 | 571 | __cpcmd("IPL", NULL, 0, NULL); |
| @@ -124,7 +124,10 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long new_stackp, · 123 unmodified lines | ||
| 124 | 124 | clear_tsk_thread_flag(p, TIF_SINGLE_STEP); |
| 125 | 125 | /* Initialize per thread user and system timer values */ |
| 126 | 126 | p->thread.user_timer = 0; |
| 127 | + p->thread.guest_timer = 0; | |
| 127 | 128 | p->thread.system_timer = 0; |
| 129 | + p->thread.hardirq_timer = 0; | |
| 130 | + p->thread.softirq_timer = 0; | |
| 128 | 131 | |
| 129 | 132 | frame->sf.back_chain = 0; |
| 130 | 133 | /* new return point is ret_from_fork */ |
| @@ -386,3 +386,5 @@ SYSCALL(sys_mlock2,compat_sys_mlock2) · 385 unmodified lines | ||
| 386 | 386 | SYSCALL(sys_copy_file_range,compat_sys_copy_file_range) /* 375 */ |
| 387 | 387 | SYSCALL(sys_preadv2,compat_sys_preadv2) |
| 388 | 388 | SYSCALL(sys_pwritev2,compat_sys_pwritev2) |
| 389 | +NI_SYSCALL | |
| 390 | +SYSCALL(sys_statx,compat_sys_statx) | |
| @@ -111,7 +111,7 @@ static inline u64 scale_vtime(u64 vtime) · 110 unmodified lines | ||
| 111 | 111 | } |
| 112 | 112 | |
| 113 | 113 | static void account_system_index_scaled(struct task_struct *p, |
| 114 | − cputime_t cputime, cputime_t scaled, | |
| 114 | + u64 cputime, u64 scaled, | |
| 115 | 115 | enum cpu_usage_stat index) |
| 116 | 116 | { |
| 117 | 117 | p->stimescaled += cputime_to_nsecs(scaled); |
| @@ -608,12 +608,29 @@ void ptep_zap_key(struct mm_struct *mm, unsigned long addr, pte_t *ptep) · 607 unmodified lines | ||
| 608 | 608 | bool test_and_clear_guest_dirty(struct mm_struct *mm, unsigned long addr) |
| 609 | 609 | { |
| 610 | 610 | spinlock_t *ptl; |
| 611 | + pgd_t *pgd; | |
| 612 | + pud_t *pud; | |
| 613 | + pmd_t *pmd; | |
| 611 | 614 | pgste_t pgste; |
| 612 | 615 | pte_t *ptep; |
| 613 | 616 | pte_t pte; |
| 614 | 617 | bool dirty; |
| 615 | 618 | |
| 616 | − ptep = get_locked_pte(mm, addr, &ptl); | |
| 619 | + pgd = pgd_offset(mm, addr); | |
| 620 | + pud = pud_alloc(mm, pgd, addr); | |
| 621 | + if (!pud) | |
| 622 | + return false; | |
| 623 | + pmd = pmd_alloc(mm, pud, addr); | |
| 624 | + if (!pmd) | |
| 625 | + return false; | |
| 626 | + /* We can't run guests backed by huge pages, but userspace can | |
| 627 | + * still set them up and then try to migrate them without any | |
| 628 | + * migration support. | |
| 629 | + */ | |
| 630 | + if (pmd_large(*pmd)) | |
| 631 | + return true; | |
| 632 | + | |
| 633 | + ptep = pte_alloc_map_lock(mm, pmd, addr, &ptl); | |
| 617 | 634 | if (unlikely(!ptep)) |
| 618 | 635 | return false; |
| 619 | 636 | |
| @@ -2,6 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | #define _ASM_SCORE_PGTABLE_H |
| 3 | 3 | |
| 4 | 4 | #include <linux/const.h> |
| 5 | +#define __ARCH_USE_5LEVEL_HACK | |
| 5 | 6 | #include <asm-generic/pgtable-nopmd.h> |
| 6 | 7 | |
| 7 | 8 | #include <asm/fixmap.h> |
| @@ -24,6 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | */ |
| 25 | 25 | |
| 26 | 26 | #include <linux/extable.h> |
| 27 | +#include <linux/ptrace.h> | |
| 27 | 28 | #include <linux/sched/mm.h> |
| 28 | 29 | #include <linux/sched/signal.h> |
| 29 | 30 | #include <linux/sched/debug.h> |
| @@ -24,6 +24,8 @@ · 23 unmodified lines | ||
| 24 | 24 | */ |
| 25 | 25 | |
| 26 | 26 | #include <linux/extable.h> |
| 27 | +#include <linux/ptrace.h> | |
| 28 | +#include <asm/extable.h> | |
| 27 | 29 | |
| 28 | 30 | int fixup_exception(struct pt_regs *regs) |
| 29 | 31 | { |
| @@ -128,7 +128,6 @@ static int __init smsc_superio_setup(void) · 127 unmodified lines | ||
| 128 | 128 | SMSC_SUPERIO_WRITE_INDEXED(1, SMSC_PRIMARY_INT_INDEX); |
| 129 | 129 | SMSC_SUPERIO_WRITE_INDEXED(12, SMSC_SECONDARY_INT_INDEX); |
| 130 | 130 | |
| 131 | −#ifdef CONFIG_IDE | |
| 132 | 131 | /* |
| 133 | 132 | * Only IDE1 exists on the Cayman |
| 134 | 133 | */ |
| @@ -158,7 +157,6 @@ static int __init smsc_superio_setup(void) · 23 unmodified lines | ||
| 158 | 157 | SMSC_SUPERIO_WRITE_INDEXED(0x01, 0xc5); /* GP45 = IDE1_IRQ */ |
| 159 | 158 | SMSC_SUPERIO_WRITE_INDEXED(0x00, 0xc6); /* GP46 = nIOROP */ |
| 160 | 159 | SMSC_SUPERIO_WRITE_INDEXED(0x00, 0xc7); /* GP47 = nIOWOP */ |
| 161 | −#endif | |
| 162 | 160 | |
| 163 | 161 | /* Exit the configuration state */ |
| 164 | 162 | outb(SMSC_EXIT_CONFIG_KEY, SMSC_CONFIG_PORT_ADDR); |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | #ifndef __ASM_SH_PGTABLE_2LEVEL_H |
| 2 | 2 | #define __ASM_SH_PGTABLE_2LEVEL_H |
| 3 | 3 | |
| 4 | +#define __ARCH_USE_5LEVEL_HACK | |
| 4 | 5 | #include <asm-generic/pgtable-nopmd.h> |
| 5 | 6 | |
| 6 | 7 | /* |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | #ifndef __ASM_SH_PGTABLE_3LEVEL_H |
| 2 | 2 | #define __ASM_SH_PGTABLE_3LEVEL_H |
| 3 | 3 | |
| 4 | +#define __ARCH_USE_5LEVEL_HACK | |
| 4 | 5 | #include <asm-generic/pgtable-nopud.h> |
| 5 | 6 | |
| 6 | 7 | /* |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | * the SpitFire page tables. |
| 13 | 13 | */ |
| 14 | 14 | |
| 15 | +#include <asm-generic/5level-fixup.h> | |
| 15 | 16 | #include <linux/compiler.h> |
| 16 | 17 | #include <linux/const.h> |
| 17 | 18 | #include <asm/types.h> |
| @@ -74,6 +74,7 @@ extern unsigned long VMALLOC_RESERVE /* = CONFIG_VMALLOC_RESERVE */; · 73 unmodified lines | ||
| 74 | 74 | #define MAXMEM (_VMALLOC_START - PAGE_OFFSET) |
| 75 | 75 | |
| 76 | 76 | /* We have no pmd or pud since we are strictly a two-level page table */ |
| 77 | +#define __ARCH_USE_5LEVEL_HACK | |
| 77 | 78 | #include <asm-generic/pgtable-nopmd.h> |
| 78 | 79 | |
| 79 | 80 | static inline int pud_huge_page(pud_t pud) { return 0; } |
| @@ -59,6 +59,7 @@ · 58 unmodified lines | ||
| 59 | 59 | #ifndef __ASSEMBLY__ |
| 60 | 60 | |
| 61 | 61 | /* We have no pud since we are a three-level page table. */ |
| 62 | +#define __ARCH_USE_5LEVEL_HACK | |
| 62 | 63 | #include <asm-generic/pgtable-nopud.h> |
| 63 | 64 | |
| 64 | 65 | /* |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #ifndef __UM_PGTABLE_2LEVEL_H |
| 9 | 9 | #define __UM_PGTABLE_2LEVEL_H |
| 10 | 10 | |
| 11 | +#define __ARCH_USE_5LEVEL_HACK | |
| 11 | 12 | #include <asm-generic/pgtable-nopmd.h> |
| 12 | 13 | |
| 13 | 14 | /* PGDIR_SHIFT determines what a third-level page table entry can map */ |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #ifndef __UM_PGTABLE_3LEVEL_H |
| 8 | 8 | #define __UM_PGTABLE_3LEVEL_H |
| 9 | 9 | |
| 10 | +#define __ARCH_USE_5LEVEL_HACK | |
| 10 | 11 | #include <asm-generic/pgtable-nopud.h> |
| 11 | 12 | |
| 12 | 13 | /* PGDIR_SHIFT determines what a third-level page table entry can map */ |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #ifndef __UNICORE_PGTABLE_H__ |
| 13 | 13 | #define __UNICORE_PGTABLE_H__ |
| 14 | 14 | |
| 15 | +#define __ARCH_USE_5LEVEL_HACK | |
| 15 | 16 | #include <asm-generic/pgtable-nopmd.h> |
| 16 | 17 | #include <asm/cpu-single.h> |
| 17 | 18 | |
| @@ -176,6 +176,7 @@ CONFIG_E1000E=y · 175 unmodified lines | ||
| 176 | 176 | CONFIG_SKY2=y |
| 177 | 177 | CONFIG_FORCEDETH=y |
| 178 | 178 | CONFIG_8139TOO=y |
| 179 | +CONFIG_R8169=y | |
| 179 | 180 | CONFIG_FDDI=y |
| 180 | 181 | CONFIG_INPUT_POLLDEV=y |
| 181 | 182 | # CONFIG_INPUT_MOUSEDEV_PSAUX is not set |
| @@ -604,7 +604,7 @@ amd_get_event_constraints_f15h(struct cpu_hw_events *cpuc, int idx, · 603 unmodified lines | ||
| 604 | 604 | return &amd_f15_PMC20; |
| 605 | 605 | } |
| 606 | 606 | case AMD_EVENT_NB: |
| 607 | − /* moved to perf_event_amd_uncore.c */ | |
| 607 | + /* moved to uncore.c */ | |
| 608 | 608 | return &emptyconstraint; |
| 609 | 609 | default: |
| 610 | 610 | return &emptyconstraint; |
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | /* |
| 2 | − * perf_event_intel_cstate.c: support cstate residency counters | |
| 2 | + * Support cstate residency counters | |
| 3 | 3 | * |
| 4 | 4 | * Copyright (C) 2015, Intel Corp. |
| 5 | 5 | * Author: Kan Liang (kan.liang@intel.com) |
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | /* |
| 2 | − * perf_event_intel_rapl.c: support Intel RAPL energy consumption counters | |
| 2 | + * Support Intel RAPL energy consumption counters | |
| 3 | 3 | * Copyright (C) 2013 Google, Inc., Stephane Eranian |
| 4 | 4 | * |
| 5 | 5 | * Intel RAPL interface is specified in the IA-32 Manual Vol3b |
| @@ -360,7 +360,7 @@ extern struct list_head pci2phy_map_head; · 359 unmodified lines | ||
| 360 | 360 | extern struct pci_extra_dev *uncore_extra_pci_dev; |
| 361 | 361 | extern struct event_constraint uncore_constraint_empty; |
| 362 | 362 | |
| 363 | −/* perf_event_intel_uncore_snb.c */ | |
| 363 | +/* uncore_snb.c */ | |
| 364 | 364 | int snb_uncore_pci_init(void); |
| 365 | 365 | int ivb_uncore_pci_init(void); |
| 366 | 366 | int hsw_uncore_pci_init(void); |
| @@ -371,7 +371,7 @@ void nhm_uncore_cpu_init(void); · 4 unmodified lines | ||
| 371 | 371 | void skl_uncore_cpu_init(void); |
| 372 | 372 | int snb_pci2phy_map_init(int devid); |
| 373 | 373 | |
| 374 | −/* perf_event_intel_uncore_snbep.c */ | |
| 374 | +/* uncore_snbep.c */ | |
| 375 | 375 | int snbep_uncore_pci_init(void); |
| 376 | 376 | void snbep_uncore_cpu_init(void); |
| 377 | 377 | int ivbep_uncore_pci_init(void); |
| @@ -385,5 +385,5 @@ void knl_uncore_cpu_init(void); · 7 unmodified lines | ||
| 385 | 385 | int skx_uncore_pci_init(void); |
| 386 | 386 | void skx_uncore_cpu_init(void); |
| 387 | 387 | |
| 388 | −/* perf_event_intel_uncore_nhmex.c */ | |
| 388 | +/* uncore_nhmex.c */ | |
| 389 | 389 | void nhmex_uncore_cpu_init(void); |
| @@ -158,13 +158,13 @@ void hyperv_init(void) · 157 unmodified lines | ||
| 158 | 158 | clocksource_register_hz(&hyperv_cs_tsc, NSEC_PER_SEC/100); |
| 159 | 159 | return; |
| 160 | 160 | } |
| 161 | +register_msr_cs: | |
| 161 | 162 | #endif |
| 162 | 163 | /* |
| 163 | 164 | * For 32 bit guests just use the MSR based mechanism for reading |
| 164 | 165 | * the partition counter. |
| 165 | 166 | */ |
| 166 | 167 | |
| 167 | −register_msr_cs: | |
| 168 | 168 | hyperv_cs = &hyperv_cs_msr; |
| 169 | 169 | if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE) |
| 170 | 170 | clocksource_register_hz(&hyperv_cs_msr, NSEC_PER_SEC/100); |
| @@ -289,7 +289,8 @@ · 288 unmodified lines | ||
| 289 | 289 | #define X86_FEATURE_PKU (16*32+ 3) /* Protection Keys for Userspace */ |
| 290 | 290 | #define X86_FEATURE_OSPKE (16*32+ 4) /* OS Protection Keys Enable */ |
| 291 | 291 | #define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* POPCNT for vectors of DW/QW */ |
| 292 | −#define X86_FEATURE_RDPID (16*32+ 22) /* RDPID instruction */ | |
| 292 | +#define X86_FEATURE_LA57 (16*32+16) /* 5-level page tables */ | |
| 293 | +#define X86_FEATURE_RDPID (16*32+22) /* RDPID instruction */ | |
| 293 | 294 | |
| 294 | 295 | /* AMD-defined CPU features, CPUID level 0x80000007 (ebx), word 17 */ |
| 295 | 296 | #define X86_FEATURE_OVERFLOW_RECOV (17*32+0) /* MCA overflow recovery support */ |
| @@ -273,6 +273,8 @@ static inline pgdval_t pgd_flags(pgd_t pgd) · 272 unmodified lines | ||
| 273 | 273 | } |
| 274 | 274 | |
| 275 | 275 | #if CONFIG_PGTABLE_LEVELS > 3 |
| 276 | +#include <asm-generic/5level-fixup.h> | |
| 277 | + | |
| 276 | 278 | typedef struct { pudval_t pud; } pud_t; |
| 277 | 279 | |
| 278 | 280 | static inline pud_t native_make_pud(pmdval_t val) |
| @@ -285,6 +287,7 @@ static inline pudval_t native_pud_val(pud_t pud) · 6 unmodified lines | ||
| 285 | 287 | return pud.pud; |
| 286 | 288 | } |
| 287 | 289 | #else |
| 290 | +#define __ARCH_USE_5LEVEL_HACK | |
| 288 | 291 | #include <asm-generic/pgtable-nopud.h> |
| 289 | 292 | |
| 290 | 293 | static inline pudval_t native_pud_val(pud_t pud) |
| @@ -306,6 +309,7 @@ static inline pmdval_t native_pmd_val(pmd_t pmd) · 15 unmodified lines | ||
| 306 | 309 | return pmd.pmd; |
| 307 | 310 | } |
| 308 | 311 | #else |
| 312 | +#define __ARCH_USE_5LEVEL_HACK | |
| 309 | 313 | #include <asm-generic/pgtable-nopmd.h> |
| 310 | 314 | |
| 311 | 315 | static inline pmdval_t native_pmd_val(pmd_t pmd) |
| @@ -46,6 +46,15 @@ extern int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey, · 45 unmodified lines | ||
| 46 | 46 | static inline |
| 47 | 47 | bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey) |
| 48 | 48 | { |
| 49 | + /* | |
| 50 | + * "Allocated" pkeys are those that have been returned | |
| 51 | + * from pkey_alloc(). pkey 0 is special, and never | |
| 52 | + * returned from pkey_alloc(). | |
| 53 | + */ | |
| 54 | + if (pkey <= 0) | |
| 55 | + return false; | |
| 56 | + if (pkey >= arch_max_pkey()) | |
| 57 | + return false; | |
| 49 | 58 | return mm_pkey_allocation_map(mm) & (1U << pkey); |
| 50 | 59 | } |
| 51 | 60 | |
| @@ -82,12 +91,6 @@ int mm_pkey_alloc(struct mm_struct *mm) · 30 unmodified lines | ||
| 82 | 91 | static inline |
| 83 | 92 | int mm_pkey_free(struct mm_struct *mm, int pkey) |
| 84 | 93 | { |
| 85 | − /* | |
| 86 | − * pkey 0 is special, always allocated and can never | |
| 87 | − * be freed. | |
| 88 | − */ | |
| 89 | − if (!pkey) | |
| 90 | − return -EINVAL; | |
| 91 | 94 | if (!mm_pkey_is_allocated(mm, pkey)) |
| 92 | 95 | return -EINVAL; |
| 93 | 96 | |
| @@ -0,0 +1,20 @@ | ||
| 1 | +#ifndef _ASM_X86_PURGATORY_H | |
| 2 | +#define _ASM_X86_PURGATORY_H | |
| 3 | + | |
| 4 | +#ifndef __ASSEMBLY__ | |
| 5 | +#include <linux/purgatory.h> | |
| 6 | + | |
| 7 | +extern void purgatory(void); | |
| 8 | +/* | |
| 9 | + * These forward declarations serve two purposes: | |
| 10 | + * | |
| 11 | + * 1) Make sparse happy when checking arch/purgatory | |
| 12 | + * 2) Document that these are required to be global so the symbol | |
| 13 | + * lookup in kexec works | |
| 14 | + */ | |
| 15 | +extern unsigned long purgatory_backup_dest; | |
| 16 | +extern unsigned long purgatory_backup_src; | |
| 17 | +extern unsigned long purgatory_backup_sz; | |
| 18 | +#endif /* __ASSEMBLY__ */ | |
| 19 | + | |
| 20 | +#endif /* _ASM_PURGATORY_H */ | |
| @@ -188,7 +188,7 @@ static inline void __native_flush_tlb_single(unsigned long addr) · 187 unmodified lines | ||
| 188 | 188 | |
| 189 | 189 | static inline void __flush_tlb_all(void) |
| 190 | 190 | { |
| 191 | − if (static_cpu_has(X86_FEATURE_PGE)) | |
| 191 | + if (boot_cpu_has(X86_FEATURE_PGE)) | |
| 192 | 192 | __flush_tlb_global(); |
| 193 | 193 | else |
| 194 | 194 | __flush_tlb(); |
| @@ -58,7 +58,7 @@ struct setup_header { · 57 unmodified lines | ||
| 58 | 58 | __u32 header; |
| 59 | 59 | __u16 version; |
| 60 | 60 | __u32 realmode_swtch; |
| 61 | − __u16 start_sys; | |
| 61 | + __u16 start_sys_seg; | |
| 62 | 62 | __u16 kernel_version; |
| 63 | 63 | __u8 type_of_loader; |
| 64 | 64 | __u8 loadflags; |
| @@ -1610,24 +1610,15 @@ static inline void try_to_enable_x2apic(int remap_mode) { } · 1609 unmodified lines | ||
| 1610 | 1610 | static inline void __x2apic_enable(void) { } |
| 1611 | 1611 | #endif /* !CONFIG_X86_X2APIC */ |
| 1612 | 1612 | |
| 1613 | −static int __init try_to_enable_IR(void) | |
| 1614 | −{ | |
| 1615 | −#ifdef CONFIG_X86_IO_APIC | |
| 1616 | − if (!x2apic_enabled() && skip_ioapic_setup) { | |
| 1617 | − pr_info("Not enabling interrupt remapping due to skipped IO-APIC setup\n"); | |
| 1618 | − return -1; | |
| 1619 | − } | |
| 1620 | −#endif | |
| 1621 | − return irq_remapping_enable(); | |
| 1622 | −} | |
| 1623 | − | |
| 1624 | 1613 | void __init enable_IR_x2apic(void) |
| 1625 | 1614 | { |
| 1626 | 1615 | unsigned long flags; |
| 1627 | 1616 | int ret, ir_stat; |
| 1628 | 1617 | |
| 1629 | − if (skip_ioapic_setup) | |
| 1618 | + if (skip_ioapic_setup) { | |
| 1619 | + pr_info("Not enabling interrupt remapping due to skipped IO-APIC setup\n"); | |
| 1630 | 1620 | return; |
| 1621 | + } | |
| 1631 | 1622 | |
| 1632 | 1623 | ir_stat = irq_remapping_prepare(); |
| 1633 | 1624 | if (ir_stat < 0 && !x2apic_supported()) |
| @@ -1645,7 +1636,7 @@ void __init enable_IR_x2apic(void) · 11 unmodified lines | ||
| 1645 | 1636 | |
| 1646 | 1637 | /* If irq_remapping_prepare() succeeded, try to enable it */ |
| 1647 | 1638 | if (ir_stat >= 0) |
| 1648 | − ir_stat = try_to_enable_IR(); | |
| 1639 | + ir_stat = irq_remapping_enable(); | |
| 1649 | 1640 | /* ir_stat contains the remap mode or an error code */ |
| 1650 | 1641 | try_to_enable_x2apic(ir_stat); |
| 1651 | 1642 | |
| @@ -2062,10 +2053,10 @@ static int allocate_logical_cpuid(int apicid) · 410 unmodified lines | ||
| 2062 | 2053 | |
| 2063 | 2054 | /* Allocate a new cpuid. */ |
| 2064 | 2055 | if (nr_logical_cpuids >= nr_cpu_ids) { |
| 2065 | − WARN_ONCE(1, "Only %d processors supported." | |
| 2056 | + WARN_ONCE(1, "APIC: NR_CPUS/possible_cpus limit of %i reached. " | |
| 2066 | 2057 | "Processor %d/0x%x and the rest are ignored.\n", |
| 2067 | − nr_cpu_ids - 1, nr_logical_cpuids, apicid); | |
| 2068 | − return -1; | |
| 2058 | + nr_cpu_ids, nr_logical_cpuids, apicid); | |
| 2059 | + return -EINVAL; | |
| 2069 | 2060 | } |
| 2070 | 2061 | |
| 2071 | 2062 | cpuid_to_apicid[nr_logical_cpuids] = apicid; |
| @@ -556,10 +556,6 @@ static void early_init_amd(struct cpuinfo_x86 *c) · 555 unmodified lines | ||
| 556 | 556 | if (c->x86_power & (1 << 8)) { |
| 557 | 557 | set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC); |
| 558 | 558 | set_cpu_cap(c, X86_FEATURE_NONSTOP_TSC); |
| 559 | − if (check_tsc_unstable()) | |
| 560 | − clear_sched_clock_stable(); | |
| 561 | − } else { | |
| 562 | − clear_sched_clock_stable(); | |
| 563 | 559 | } |
| 564 | 560 | |
| 565 | 561 | /* Bit 12 of 8000_0007 edx is accumulated power mechanism. */ |
| @@ -105,8 +105,6 @@ static void early_init_centaur(struct cpuinfo_x86 *c) · 104 unmodified lines | ||
| 105 | 105 | #ifdef CONFIG_X86_64 |
| 106 | 106 | set_cpu_cap(c, X86_FEATURE_SYSENTER32); |
| 107 | 107 | #endif |
| 108 | − | |
| 109 | − clear_sched_clock_stable(); | |
| 110 | 108 | } |
| 111 | 109 | |
| 112 | 110 | static void init_centaur(struct cpuinfo_x86 *c) |
| @@ -88,7 +88,6 @@ static void default_init(struct cpuinfo_x86 *c) · 87 unmodified lines | ||
| 88 | 88 | strcpy(c->x86_model_id, "386"); |
| 89 | 89 | } |
| 90 | 90 | #endif |
| 91 | − clear_sched_clock_stable(); | |
| 92 | 91 | } |
| 93 | 92 | |
| 94 | 93 | static const struct cpu_dev default_cpu = { |
| @@ -1077,8 +1076,6 @@ static void identify_cpu(struct cpuinfo_x86 *c) · 982 unmodified lines | ||
| 1077 | 1076 | */ |
| 1078 | 1077 | if (this_cpu->c_init) |
| 1079 | 1078 | this_cpu->c_init(c); |
| 1080 | − else | |
| 1081 | − clear_sched_clock_stable(); | |
| 1082 | 1079 | |
| 1083 | 1080 | /* Disable the PN if appropriate */ |
| 1084 | 1081 | squash_the_stupid_serial_number(c); |
| @@ -185,7 +185,6 @@ static void early_init_cyrix(struct cpuinfo_x86 *c) · 184 unmodified lines | ||
| 185 | 185 | set_cpu_cap(c, X86_FEATURE_CYRIX_ARR); |
| 186 | 186 | break; |
| 187 | 187 | } |
| 188 | − clear_sched_clock_stable(); | |
| 189 | 188 | } |
| 190 | 189 | |
| 191 | 190 | static void init_cyrix(struct cpuinfo_x86 *c) |
| @@ -162,10 +162,6 @@ static void early_init_intel(struct cpuinfo_x86 *c) · 161 unmodified lines | ||
| 162 | 162 | if (c->x86_power & (1 << 8)) { |
| 163 | 163 | set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC); |
| 164 | 164 | set_cpu_cap(c, X86_FEATURE_NONSTOP_TSC); |
| 165 | − if (check_tsc_unstable()) | |
| 166 | − clear_sched_clock_stable(); | |
| 167 | − } else { | |
| 168 | − clear_sched_clock_stable(); | |
| 169 | 165 | } |
| 170 | 166 | |
| 171 | 167 | /* Penwell and Cloverview have the TSC which doesn't sleep on S3 */ |
| @@ -28,7 +28,6 @@ · 27 unmodified lines | ||
| 28 | 28 | #include <linux/sched/signal.h> |
| 29 | 29 | #include <linux/sched/task.h> |
| 30 | 30 | #include <linux/slab.h> |
| 31 | −#include <linux/cpu.h> | |
| 32 | 31 | #include <linux/task_work.h> |
| 33 | 32 | |
| 34 | 33 | #include <uapi/linux/magic.h> |
| @@ -16,8 +16,6 @@ static void early_init_transmeta(struct cpuinfo_x86 *c) · 15 unmodified lines | ||
| 16 | 16 | if (xlvl >= 0x80860001) |
| 17 | 17 | c->x86_capability[CPUID_8086_0001_EDX] = cpuid_edx(0x80860001); |
| 18 | 18 | } |
| 19 | − | |
| 20 | − clear_sched_clock_stable(); | |
| 21 | 19 | } |
| 22 | 20 | |
| 23 | 21 | static void init_transmeta(struct cpuinfo_x86 *c) |
| @@ -30,7 +30,6 @@ · 29 unmodified lines | ||
| 30 | 30 | #include <asm/hypervisor.h> |
| 31 | 31 | #include <asm/timer.h> |
| 32 | 32 | #include <asm/apic.h> |
| 33 | −#include <asm/timer.h> | |
| 34 | 33 | |
| 35 | 34 | #undef pr_fmt |
| 36 | 35 | #define pr_fmt(fmt) "vmware: " fmt |
| @@ -535,7 +535,7 @@ static void run_sync(void) · 534 unmodified lines | ||
| 535 | 535 | { |
| 536 | 536 | int enable_irqs = irqs_disabled(); |
| 537 | 537 | |
| 538 | − /* We may be called with interrupts disbled (on bootup). */ | |
| 538 | + /* We may be called with interrupts disabled (on bootup). */ | |
| 539 | 539 | if (enable_irqs) |
| 540 | 540 | local_irq_enable(); |
| 541 | 541 | on_each_cpu(do_sync_core, NULL, 1); |
| @@ -354,7 +354,7 @@ static int hpet_resume(struct clock_event_device *evt, int timer) · 353 unmodified lines | ||
| 354 | 354 | |
| 355 | 355 | irq_domain_deactivate_irq(irq_get_irq_data(hdev->irq)); |
| 356 | 356 | irq_domain_activate_irq(irq_get_irq_data(hdev->irq)); |
| 357 | − disable_irq(hdev->irq); | |
| 357 | + disable_hardirq(hdev->irq); | |
| 358 | 358 | irq_set_affinity(hdev->irq, cpumask_of(hdev->cpu)); |
| 359 | 359 | enable_irq(hdev->irq); |
| 360 | 360 | } |
| @@ -167,7 +167,7 @@ static int __init boot_params_kdebugfs_init(void) · 166 unmodified lines | ||
| 167 | 167 | struct dentry *dbp, *version, *data; |
| 168 | 168 | int error = -ENOMEM; |
| 169 | 169 | |
| 170 | − dbp = debugfs_create_dir("boot_params", NULL); | |
| 170 | + dbp = debugfs_create_dir("boot_params", arch_debugfs_dir); | |
| 171 | 171 | if (!dbp) |
| 172 | 172 | return -ENOMEM; |
| 173 | 173 | |
| @@ -194,19 +194,22 @@ static int arch_update_purgatory(struct kimage *image) · 193 unmodified lines | ||
| 194 | 194 | |
| 195 | 195 | /* Setup copying of backup region */ |
| 196 | 196 | if (image->type == KEXEC_TYPE_CRASH) { |
| 197 | − ret = kexec_purgatory_get_set_symbol(image, "backup_dest", | |
| 197 | + ret = kexec_purgatory_get_set_symbol(image, | |
| 198 | + "purgatory_backup_dest", | |
| 198 | 199 | &image->arch.backup_load_addr, |
| 199 | 200 | sizeof(image->arch.backup_load_addr), 0); |
| 200 | 201 | if (ret) |
| 201 | 202 | return ret; |
| 202 | 203 | |
| 203 | − ret = kexec_purgatory_get_set_symbol(image, "backup_src", | |
| 204 | + ret = kexec_purgatory_get_set_symbol(image, | |
| 205 | + "purgatory_backup_src", | |
| 204 | 206 | &image->arch.backup_src_start, |
| 205 | 207 | sizeof(image->arch.backup_src_start), 0); |
| 206 | 208 | if (ret) |
| 207 | 209 | return ret; |
| 208 | 210 | |
| 209 | − ret = kexec_purgatory_get_set_symbol(image, "backup_sz", | |
| 211 | + ret = kexec_purgatory_get_set_symbol(image, | |
| 212 | + "purgatory_backup_sz", | |
| 210 | 213 | &image->arch.backup_src_sz, |
| 211 | 214 | sizeof(image->arch.backup_src_sz), 0); |
| 212 | 215 | if (ret) |
| @@ -223,6 +223,22 @@ static struct dmi_system_id __initdata reboot_dmi_table[] = { · 222 unmodified lines | ||
| 223 | 223 | DMI_MATCH(DMI_BOARD_NAME, "P4S800"), |
| 224 | 224 | }, |
| 225 | 225 | }, |
| 226 | + { /* Handle problems with rebooting on ASUS EeeBook X205TA */ | |
| 227 | + .callback = set_acpi_reboot, | |
| 228 | + .ident = "ASUS EeeBook X205TA", | |
| 229 | + .matches = { | |
| 230 | + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 231 | + DMI_MATCH(DMI_PRODUCT_NAME, "X205TA"), | |
| 232 | + }, | |
| 233 | + }, | |
| 234 | + { /* Handle problems with rebooting on ASUS EeeBook X205TAW */ | |
| 235 | + .callback = set_acpi_reboot, | |
| 236 | + .ident = "ASUS EeeBook X205TAW", | |
| 237 | + .matches = { | |
| 238 | + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 239 | + DMI_MATCH(DMI_PRODUCT_NAME, "X205TAW"), | |
| 240 | + }, | |
| 241 | + }, | |
| 226 | 242 | |
| 227 | 243 | /* Certec */ |
| 228 | 244 | { /* Handle problems with rebooting on Certec BPC600 */ |
| @@ -327,9 +327,16 @@ unsigned long long sched_clock(void) · 326 unmodified lines | ||
| 327 | 327 | { |
| 328 | 328 | return paravirt_sched_clock(); |
| 329 | 329 | } |
| 330 | + | |
| 331 | +static inline bool using_native_sched_clock(void) | |
| 332 | +{ | |
| 333 | + return pv_time_ops.sched_clock == native_sched_clock; | |
| 334 | +} | |
| 330 | 335 | #else |
| 331 | 336 | unsigned long long |
| 332 | 337 | sched_clock(void) __attribute__((alias("native_sched_clock"))); |
| 338 | + | |
| 339 | +static inline bool using_native_sched_clock(void) { return true; } | |
| 333 | 340 | #endif |
| 334 | 341 | |
| 335 | 342 | int check_tsc_unstable(void) |
| @@ -1112,8 +1119,10 @@ static void tsc_cs_mark_unstable(struct clocksource *cs) · 776 unmodified lines | ||
| 1112 | 1119 | { |
| 1113 | 1120 | if (tsc_unstable) |
| 1114 | 1121 | return; |
| 1122 | + | |
| 1115 | 1123 | tsc_unstable = 1; |
| 1116 | − clear_sched_clock_stable(); | |
| 1124 | + if (using_native_sched_clock()) | |
| 1125 | + clear_sched_clock_stable(); | |
| 1117 | 1126 | disable_sched_clock_irqtime(); |
| 1118 | 1127 | pr_info("Marking TSC unstable due to clocksource watchdog\n"); |
| 1119 | 1128 | } |
| @@ -1135,18 +1144,20 @@ static struct clocksource clocksource_tsc = { · 15 unmodified lines | ||
| 1135 | 1144 | |
| 1136 | 1145 | void mark_tsc_unstable(char *reason) |
| 1137 | 1146 | { |
| 1138 | − if (!tsc_unstable) { | |
| 1139 | − tsc_unstable = 1; | |
| 1147 | + if (tsc_unstable) | |
| 1148 | + return; | |
| 1149 | + | |
| 1150 | + tsc_unstable = 1; | |
| 1151 | + if (using_native_sched_clock()) | |
| 1140 | 1152 | clear_sched_clock_stable(); |
| 1141 | − disable_sched_clock_irqtime(); | |
| 1142 | − pr_info("Marking TSC unstable due to %s\n", reason); | |
| 1143 | − /* Change only the rating, when not registered */ | |
| 1144 | − if (clocksource_tsc.mult) | |
| 1145 | − clocksource_mark_unstable(&clocksource_tsc); | |
| 1146 | − else { | |
| 1147 | − clocksource_tsc.flags |= CLOCK_SOURCE_UNSTABLE; | |
| 1148 | − clocksource_tsc.rating = 0; | |
| 1149 | − } | |
| 1153 | + disable_sched_clock_irqtime(); | |
| 1154 | + pr_info("Marking TSC unstable due to %s\n", reason); | |
| 1155 | + /* Change only the rating, when not registered */ | |
| 1156 | + if (clocksource_tsc.mult) { | |
| 1157 | + clocksource_mark_unstable(&clocksource_tsc); | |
| 1158 | + } else { | |
| 1159 | + clocksource_tsc.flags |= CLOCK_SOURCE_UNSTABLE; | |
| 1160 | + clocksource_tsc.rating = 0; | |
| 1150 | 1161 | } |
| 1151 | 1162 | } |
| 1152 | 1163 | |
| @@ -7258,9 +7258,8 @@ static int handle_vmoff(struct kvm_vcpu *vcpu) · 7257 unmodified lines | ||
| 7258 | 7258 | static int handle_vmclear(struct kvm_vcpu *vcpu) |
| 7259 | 7259 | { |
| 7260 | 7260 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| 7261 | + u32 zero = 0; | |
| 7261 | 7262 | gpa_t vmptr; |
| 7262 | − struct vmcs12 *vmcs12; | |
| 7263 | − struct page *page; | |
| 7264 | 7263 | |
| 7265 | 7264 | if (!nested_vmx_check_permission(vcpu)) |
| 7266 | 7265 | return 1; |
| @@ -7271,22 +7270,9 @@ static int handle_vmclear(struct kvm_vcpu *vcpu) · 4 unmodified lines | ||
| 7271 | 7270 | if (vmptr == vmx->nested.current_vmptr) |
| 7272 | 7271 | nested_release_vmcs12(vmx); |
| 7273 | 7272 | |
| 7274 | − page = nested_get_page(vcpu, vmptr); | |
| 7275 | − if (page == NULL) { | |
| 7276 | − /* | |
| 7277 | − * For accurate processor emulation, VMCLEAR beyond available | |
| 7278 | − * physical memory should do nothing at all. However, it is | |
| 7279 | − * possible that a nested vmx bug, not a guest hypervisor bug, | |
| 7280 | − * resulted in this case, so let's shut down before doing any | |
| 7281 | − * more damage: | |
| 7282 | − */ | |
| 7283 | − kvm_make_request(KVM_REQ_TRIPLE_FAULT, vcpu); | |
| 7284 | − return 1; | |
| 7285 | − } | |
| 7286 | − vmcs12 = kmap(page); | |
| 7287 | − vmcs12->launch_state = 0; | |
| 7288 | − kunmap(page); | |
| 7289 | − nested_release_page(page); | |
| 7273 | + kvm_vcpu_write_guest(vcpu, | |
| 7274 | + vmptr + offsetof(struct vmcs12, launch_state), | |
| 7275 | + &zero, sizeof(zero)); | |
| 7290 | 7276 | |
| 7291 | 7277 | nested_free_vmcs02(vmx, vmptr); |
| 7292 | 7278 | |
| @@ -9694,10 +9680,8 @@ static inline bool nested_vmx_merge_msr_bitmap(struct kvm_vcpu *vcpu, · 2401 unmodified lines | ||
| 9694 | 9680 | return false; |
| 9695 | 9681 | |
| 9696 | 9682 | page = nested_get_page(vcpu, vmcs12->msr_bitmap); |
| 9697 | − if (!page) { | |
| 9698 | − WARN_ON(1); | |
| 9683 | + if (!page) | |
| 9699 | 9684 | return false; |
| 9700 | − } | |
| 9701 | 9685 | msr_bitmap_l1 = (unsigned long *)kmap(page); |
| 9702 | 9686 | |
| 9703 | 9687 | memset(msr_bitmap_l0, 0xff, PAGE_SIZE); |
| @@ -11121,8 +11105,10 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason, · 1417 unmodified lines | ||
| 11121 | 11105 | */ |
| 11122 | 11106 | static void vmx_leave_nested(struct kvm_vcpu *vcpu) |
| 11123 | 11107 | { |
| 11124 | − if (is_guest_mode(vcpu)) | |
| 11108 | + if (is_guest_mode(vcpu)) { | |
| 11109 | + to_vmx(vcpu)->nested.nested_run_pending = 0; | |
| 11125 | 11110 | nested_vmx_vmexit(vcpu, -1, 0, 0); |
| 11111 | + } | |
| 11126 | 11112 | free_nested(to_vmx(vcpu)); |
| 11127 | 11113 | } |
| 11128 | 11114 | |
| @@ -106,32 +106,35 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr, · 105 unmodified lines | ||
| 106 | 106 | unsigned long end, int write, struct page **pages, int *nr) |
| 107 | 107 | { |
| 108 | 108 | struct dev_pagemap *pgmap = NULL; |
| 109 | − int nr_start = *nr; | |
| 110 | − pte_t *ptep; | |
| 109 | + int nr_start = *nr, ret = 0; | |
| 110 | + pte_t *ptep, *ptem; | |
| 111 | 111 | |
| 112 | − ptep = pte_offset_map(&pmd, addr); | |
| 112 | + /* | |
| 113 | + * Keep the original mapped PTE value (ptem) around since we | |
| 114 | + * might increment ptep off the end of the page when finishing | |
| 115 | + * our loop iteration. | |
| 116 | + */ | |
| 117 | + ptem = ptep = pte_offset_map(&pmd, addr); | |
| 113 | 118 | do { |
| 114 | 119 | pte_t pte = gup_get_pte(ptep); |
| 115 | 120 | struct page *page; |
| 116 | 121 | |
| 117 | 122 | /* Similar to the PMD case, NUMA hinting must take slow path */ |
| 118 | − if (pte_protnone(pte)) { | |
| 119 | − pte_unmap(ptep); | |
| 120 | − return 0; | |
| 121 | − } | |
| 123 | + if (pte_protnone(pte)) | |
| 124 | + break; | |
| 125 | + | |
| 126 | + if (!pte_allows_gup(pte_val(pte), write)) | |
| 127 | + break; | |
| 122 | 128 | |
| 123 | 129 | if (pte_devmap(pte)) { |
| 124 | 130 | pgmap = get_dev_pagemap(pte_pfn(pte), pgmap); |
| 125 | 131 | if (unlikely(!pgmap)) { |
| 126 | 132 | undo_dev_pagemap(nr, nr_start, pages); |
| 127 | − pte_unmap(ptep); | |
| 128 | − return 0; | |
| 133 | + break; | |
| 129 | 134 | } |
| 130 | − } else if (!pte_allows_gup(pte_val(pte), write) || | |
| 131 | − pte_special(pte)) { | |
| 132 | − pte_unmap(ptep); | |
| 133 | − return 0; | |
| 134 | − } | |
| 135 | + } else if (pte_special(pte)) | |
| 136 | + break; | |
| 137 | + | |
| 135 | 138 | VM_BUG_ON(!pfn_valid(pte_pfn(pte))); |
| 136 | 139 | page = pte_page(pte); |
| 137 | 140 | get_page(page); |
| @@ -141,9 +144,11 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr, · 3 unmodified lines | ||
| 141 | 144 | (*nr)++; |
| 142 | 145 | |
| 143 | 146 | } while (ptep++, addr += PAGE_SIZE, addr != end); |
| 144 | − pte_unmap(ptep - 1); | |
| 147 | + if (addr == end) | |
| 148 | + ret = 1; | |
| 149 | + pte_unmap(ptem); | |
| 145 | 150 | |
| 146 | − return 1; | |
| 151 | + return ret; | |
| 147 | 152 | } |
| 148 | 153 | |
| 149 | 154 | static inline void get_head_page_multiple(struct page *page, int nr) |
| @@ -735,6 +735,15 @@ void pcibios_disable_device (struct pci_dev *dev) · 734 unmodified lines | ||
| 735 | 735 | pcibios_disable_irq(dev); |
| 736 | 736 | } |
| 737 | 737 | |
| 738 | +#ifdef CONFIG_ACPI_HOTPLUG_IOAPIC | |
| 739 | +void pcibios_release_device(struct pci_dev *dev) | |
| 740 | +{ | |
| 741 | + if (atomic_dec_return(&dev->enable_cnt) >= 0) | |
| 742 | + pcibios_disable_device(dev); | |
| 743 | + | |
| 744 | +} | |
| 745 | +#endif | |
| 746 | + | |
| 738 | 747 | int pci_ext_cfg_avail(void) |
| 739 | 748 | { |
| 740 | 749 | if (raw_pci_ext_ops) |
| @@ -234,23 +234,14 @@ static int xen_hvm_setup_msi_irqs(struct pci_dev *dev, int nvec, int type) · 233 unmodified lines | ||
| 234 | 234 | return 1; |
| 235 | 235 | |
| 236 | 236 | for_each_pci_msi_entry(msidesc, dev) { |
| 237 | − __pci_read_msi_msg(msidesc, &msg); | |
| 238 | − pirq = MSI_ADDR_EXT_DEST_ID(msg.address_hi) | | |
| 239 | − ((msg.address_lo >> MSI_ADDR_DEST_ID_SHIFT) & 0xff); | |
| 240 | − if (msg.data != XEN_PIRQ_MSI_DATA || | |
| 241 | − xen_irq_from_pirq(pirq) < 0) { | |
| 242 | − pirq = xen_allocate_pirq_msi(dev, msidesc); | |
| 243 | − if (pirq < 0) { | |
| 244 | − irq = -ENODEV; | |
| 245 | − goto error; | |
| 246 | − } | |
| 247 | − xen_msi_compose_msg(dev, pirq, &msg); | |
| 248 | − __pci_write_msi_msg(msidesc, &msg); | |
| 249 | − dev_dbg(&dev->dev, "xen: msi bound to pirq=%d\n", pirq); | |
| 250 | − } else { | |
| 251 | − dev_dbg(&dev->dev, | |
| 252 | − "xen: msi already bound to pirq=%d\n", pirq); | |
| 237 | + pirq = xen_allocate_pirq_msi(dev, msidesc); | |
| 238 | + if (pirq < 0) { | |
| 239 | + irq = -ENODEV; | |
| 240 | + goto error; | |
| 253 | 241 | } |
| 242 | + xen_msi_compose_msg(dev, pirq, &msg); | |
| 243 | + __pci_write_msi_msg(msidesc, &msg); | |
| 244 | + dev_dbg(&dev->dev, "xen: msi bound to pirq=%d\n", pirq); | |
| 254 | 245 | irq = xen_bind_pirq_msi_to_irq(dev, msidesc, pirq, |
| 255 | 246 | (type == PCI_CAP_ID_MSI) ? nvec : 1, |
| 256 | 247 | (type == PCI_CAP_ID_MSIX) ? |
| @@ -1847,7 +1847,6 @@ static void pq_init(int node, int pnode) · 1846 unmodified lines | ||
| 1847 | 1847 | |
| 1848 | 1848 | ops.write_payload_first(pnode, first); |
| 1849 | 1849 | ops.write_payload_last(pnode, last); |
| 1850 | − ops.write_g_sw_ack(pnode, 0xffffUL); | |
| 1851 | 1850 | |
| 1852 | 1851 | /* in effect, all msg_type's are set to MSG_NOOP */ |
| 1853 | 1852 | memset(pqp, 0, sizeof(struct bau_pq_entry) * DEST_Q_SIZE); |
| @@ -10,21 +10,19 @@ · 9 unmodified lines | ||
| 10 | 10 | * Version 2. See the file COPYING for more details. |
| 11 | 11 | */ |
| 12 | 12 | |
| 13 | +#include <linux/bug.h> | |
| 14 | +#include <asm/purgatory.h> | |
| 15 | + | |
| 13 | 16 | #include "sha256.h" |
| 14 | 17 | #include "../boot/string.h" |
| 15 | 18 | |
| 16 | −struct sha_region { | |
| 17 | − unsigned long start; | |
| 18 | − unsigned long len; | |
| 19 | −}; | |
| 20 | − | |
| 21 | −unsigned long backup_dest = 0; | |
| 22 | −unsigned long backup_src = 0; | |
| 23 | −unsigned long backup_sz = 0; | |
| 19 | +unsigned long purgatory_backup_dest __section(.kexec-purgatory); | |
| 20 | +unsigned long purgatory_backup_src __section(.kexec-purgatory); | |
| 21 | +unsigned long purgatory_backup_sz __section(.kexec-purgatory); | |
| 24 | 22 | |
| 25 | −u8 sha256_digest[SHA256_DIGEST_SIZE] = { 0 }; | |
| 23 | +u8 purgatory_sha256_digest[SHA256_DIGEST_SIZE] __section(.kexec-purgatory); | |
| 26 | 24 | |
| 27 | −struct sha_region sha_regions[16] = {}; | |
| 25 | +struct kexec_sha_region purgatory_sha_regions[KEXEC_SEGMENT_MAX] __section(.kexec-purgatory); | |
| 28 | 26 | |
| 29 | 27 | /* |
| 30 | 28 | * On x86, second kernel requries first 640K of memory to boot. Copy |
| @@ -33,26 +31,28 @@ struct sha_region sha_regions[16] = {}; · 2 unmodified lines | ||
| 33 | 31 | */ |
| 34 | 32 | static int copy_backup_region(void) |
| 35 | 33 | { |
| 36 | − if (backup_dest) | |
| 37 | − memcpy((void *)backup_dest, (void *)backup_src, backup_sz); | |
| 38 | − | |
| 34 | + if (purgatory_backup_dest) { | |
| 35 | + memcpy((void *)purgatory_backup_dest, | |
| 36 | + (void *)purgatory_backup_src, purgatory_backup_sz); | |
| 37 | + } | |
| 39 | 38 | return 0; |
| 40 | 39 | } |
| 41 | 40 | |
| 42 | −int verify_sha256_digest(void) | |
| 41 | +static int verify_sha256_digest(void) | |
| 43 | 42 | { |
| 44 | − struct sha_region *ptr, *end; | |
| 43 | + struct kexec_sha_region *ptr, *end; | |
| 45 | 44 | u8 digest[SHA256_DIGEST_SIZE]; |
| 46 | 45 | struct sha256_state sctx; |
| 47 | 46 | |
| 48 | 47 | sha256_init(&sctx); |
| 49 | − end = &sha_regions[sizeof(sha_regions)/sizeof(sha_regions[0])]; | |
| 50 | − for (ptr = sha_regions; ptr < end; ptr++) | |
| 48 | + end = purgatory_sha_regions + ARRAY_SIZE(purgatory_sha_regions); | |
| 49 | + | |
| 50 | + for (ptr = purgatory_sha_regions; ptr < end; ptr++) | |
| 51 | 51 | sha256_update(&sctx, (uint8_t *)(ptr->start), ptr->len); |
| 52 | 52 | |
| 53 | 53 | sha256_final(&sctx, digest); |
| 54 | 54 | |
| 55 | − if (memcmp(digest, sha256_digest, sizeof(digest))) | |
| 55 | + if (memcmp(digest, purgatory_sha256_digest, sizeof(digest))) | |
| 56 | 56 | return 1; |
| 57 | 57 | |
| 58 | 58 | return 0; |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | * This source code is licensed under the GNU General Public License, |
| 10 | 10 | * Version 2. See the file COPYING for more details. |
| 11 | 11 | */ |
| 12 | +#include <asm/purgatory.h> | |
| 12 | 13 | |
| 13 | 14 | .text |
| 14 | 15 | .globl purgatory_start |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | #ifndef SHA256_H |
| 11 | 11 | #define SHA256_H |
| 12 | 12 | |
| 13 | − | |
| 14 | 13 | #include <linux/types.h> |
| 15 | 14 | #include <crypto/sha.h> |
| 16 | 15 | |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #ifndef _XTENSA_PGTABLE_H |
| 12 | 12 | #define _XTENSA_PGTABLE_H |
| 13 | 13 | |
| 14 | +#define __ARCH_USE_5LEVEL_HACK | |
| 14 | 15 | #include <asm-generic/pgtable-nopmd.h> |
| 15 | 16 | #include <asm/page.h> |
| 16 | 17 | #include <asm/kmem_layout.h> |
| @@ -376,10 +376,14 @@ static void punt_bios_to_rescuer(struct bio_set *bs) · 375 unmodified lines | ||
| 376 | 376 | bio_list_init(&punt); |
| 377 | 377 | bio_list_init(&nopunt); |
| 378 | 378 | |
| 379 | − while ((bio = bio_list_pop(current->bio_list))) | |
| 379 | + while ((bio = bio_list_pop(¤t->bio_list[0]))) | |
| 380 | 380 | bio_list_add(bio->bi_pool == bs ? &punt : &nopunt, bio); |
| 381 | + current->bio_list[0] = nopunt; | |
| 381 | 382 | |
| 382 | − *current->bio_list = nopunt; | |
| 383 | + bio_list_init(&nopunt); | |
| 384 | + while ((bio = bio_list_pop(¤t->bio_list[1]))) | |
| 385 | + bio_list_add(bio->bi_pool == bs ? &punt : &nopunt, bio); | |
| 386 | + current->bio_list[1] = nopunt; | |
| 383 | 387 | |
| 384 | 388 | spin_lock(&bs->rescue_lock); |
| 385 | 389 | bio_list_merge(&bs->rescue_list, &punt); |
| @@ -466,7 +470,9 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, int nr_iovecs, struct bio_set *bs) · 80 unmodified lines | ||
| 466 | 470 | * we retry with the original gfp_flags. |
| 467 | 471 | */ |
| 468 | 472 | |
| 469 | − if (current->bio_list && !bio_list_empty(current->bio_list)) | |
| 473 | + if (current->bio_list && | |
| 474 | + (!bio_list_empty(¤t->bio_list[0]) || | |
| 475 | + !bio_list_empty(¤t->bio_list[1]))) | |
| 470 | 476 | gfp_mask &= ~__GFP_DIRECT_RECLAIM; |
| 471 | 477 | |
| 472 | 478 | p = mempool_alloc(bs->bio_pool, gfp_mask); |
| @@ -578,8 +578,6 @@ void blk_cleanup_queue(struct request_queue *q) · 577 unmodified lines | ||
| 578 | 578 | q->queue_lock = &q->__queue_lock; |
| 579 | 579 | spin_unlock_irq(lock); |
| 580 | 580 | |
| 581 | − put_disk_devt(q->disk_devt); | |
| 582 | − | |
| 583 | 581 | /* @q is and will stay empty, shutdown and put */ |
| 584 | 582 | blk_put_queue(q); |
| 585 | 583 | } |
| @@ -1975,7 +1973,14 @@ end_io: · 1389 unmodified lines | ||
| 1975 | 1973 | */ |
| 1976 | 1974 | blk_qc_t generic_make_request(struct bio *bio) |
| 1977 | 1975 | { |
| 1978 | − struct bio_list bio_list_on_stack; | |
| 1976 | + /* | |
| 1977 | + * bio_list_on_stack[0] contains bios submitted by the current | |
| 1978 | + * make_request_fn. | |
| 1979 | + * bio_list_on_stack[1] contains bios that were submitted before | |
| 1980 | + * the current make_request_fn, but that haven't been processed | |
| 1981 | + * yet. | |
| 1982 | + */ | |
| 1983 | + struct bio_list bio_list_on_stack[2]; | |
| 1979 | 1984 | blk_qc_t ret = BLK_QC_T_NONE; |
| 1980 | 1985 | |
| 1981 | 1986 | if (!generic_make_request_checks(bio)) |
| @@ -1992,7 +1997,7 @@ blk_qc_t generic_make_request(struct bio *bio) · 10 unmodified lines | ||
| 1992 | 1997 | * should be added at the tail |
| 1993 | 1998 | */ |
| 1994 | 1999 | if (current->bio_list) { |
| 1995 | − bio_list_add(current->bio_list, bio); | |
| 2000 | + bio_list_add(¤t->bio_list[0], bio); | |
| 1996 | 2001 | goto out; |
| 1997 | 2002 | } |
| 1998 | 2003 | |
| @@ -2011,23 +2016,39 @@ blk_qc_t generic_make_request(struct bio *bio) · 12 unmodified lines | ||
| 2011 | 2016 | * bio_list, and call into ->make_request() again. |
| 2012 | 2017 | */ |
| 2013 | 2018 | BUG_ON(bio->bi_next); |
| 2014 | − bio_list_init(&bio_list_on_stack); | |
| 2015 | − current->bio_list = &bio_list_on_stack; | |
| 2019 | + bio_list_init(&bio_list_on_stack[0]); | |
| 2020 | + current->bio_list = bio_list_on_stack; | |
| 2016 | 2021 | do { |
| 2017 | 2022 | struct request_queue *q = bdev_get_queue(bio->bi_bdev); |
| 2018 | 2023 | |
| 2019 | 2024 | if (likely(blk_queue_enter(q, false) == 0)) { |
| 2025 | + struct bio_list lower, same; | |
| 2026 | + | |
| 2027 | + /* Create a fresh bio_list for all subordinate requests */ | |
| 2028 | + bio_list_on_stack[1] = bio_list_on_stack[0]; | |
| 2029 | + bio_list_init(&bio_list_on_stack[0]); | |
| 2020 | 2030 | ret = q->make_request_fn(q, bio); |
| 2021 | 2031 | |
| 2022 | 2032 | blk_queue_exit(q); |
| 2023 | 2033 | |
| 2024 | − bio = bio_list_pop(current->bio_list); | |
| 2034 | + /* sort new bios into those for a lower level | |
| 2035 | + * and those for the same level | |
| 2036 | + */ | |
| 2037 | + bio_list_init(&lower); | |
| 2038 | + bio_list_init(&same); | |
| 2039 | + while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL) | |
| 2040 | + if (q == bdev_get_queue(bio->bi_bdev)) | |
| 2041 | + bio_list_add(&same, bio); | |
| 2042 | + else | |
| 2043 | + bio_list_add(&lower, bio); | |
| 2044 | + /* now assemble so we handle the lowest level first */ | |
| 2045 | + bio_list_merge(&bio_list_on_stack[0], &lower); | |
| 2046 | + bio_list_merge(&bio_list_on_stack[0], &same); | |
| 2047 | + bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]); | |
| 2025 | 2048 | } else { |
| 2026 | − struct bio *bio_next = bio_list_pop(current->bio_list); | |
| 2027 | − | |
| 2028 | 2049 | bio_io_error(bio); |
| 2029 | − bio = bio_next; | |
| 2030 | 2050 | } |
| 2051 | + bio = bio_list_pop(&bio_list_on_stack[0]); | |
| 2031 | 2052 | } while (bio); |
| 2032 | 2053 | current->bio_list = NULL; /* deactivate */ |
| 2033 | 2054 | |
| @@ -17,6 +17,15 @@ static void blk_mq_sysfs_release(struct kobject *kobj) · 16 unmodified lines | ||
| 17 | 17 | { |
| 18 | 18 | } |
| 19 | 19 | |
| 20 | +static void blk_mq_hw_sysfs_release(struct kobject *kobj) | |
| 21 | +{ | |
| 22 | + struct blk_mq_hw_ctx *hctx = container_of(kobj, struct blk_mq_hw_ctx, | |
| 23 | + kobj); | |
| 24 | + free_cpumask_var(hctx->cpumask); | |
| 25 | + kfree(hctx->ctxs); | |
| 26 | + kfree(hctx); | |
| 27 | +} | |
| 28 | + | |
| 20 | 29 | struct blk_mq_ctx_sysfs_entry { |
| 21 | 30 | struct attribute attr; |
| 22 | 31 | ssize_t (*show)(struct blk_mq_ctx *, char *); |
| @@ -200,7 +209,7 @@ static struct kobj_type blk_mq_ctx_ktype = { · 177 unmodified lines | ||
| 200 | 209 | static struct kobj_type blk_mq_hw_ktype = { |
| 201 | 210 | .sysfs_ops = &blk_mq_hw_sysfs_ops, |
| 202 | 211 | .default_attrs = default_hw_ctx_attrs, |
| 203 | − .release = blk_mq_sysfs_release, | |
| 212 | + .release = blk_mq_hw_sysfs_release, | |
| 204 | 213 | }; |
| 205 | 214 | |
| 206 | 215 | static void blk_mq_unregister_hctx(struct blk_mq_hw_ctx *hctx) |
| @@ -242,24 +251,15 @@ static int blk_mq_register_hctx(struct blk_mq_hw_ctx *hctx) · 35 unmodified lines | ||
| 242 | 251 | static void __blk_mq_unregister_dev(struct device *dev, struct request_queue *q) |
| 243 | 252 | { |
| 244 | 253 | struct blk_mq_hw_ctx *hctx; |
| 245 | − struct blk_mq_ctx *ctx; | |
| 246 | − int i, j; | |
| 254 | + int i; | |
| 247 | 255 | |
| 248 | − queue_for_each_hw_ctx(q, hctx, i) { | |
| 256 | + queue_for_each_hw_ctx(q, hctx, i) | |
| 249 | 257 | blk_mq_unregister_hctx(hctx); |
| 250 | 258 | |
| 251 | − hctx_for_each_ctx(hctx, ctx, j) | |
| 252 | − kobject_put(&ctx->kobj); | |
| 253 | − | |
| 254 | − kobject_put(&hctx->kobj); | |
| 255 | − } | |
| 256 | − | |
| 257 | 259 | blk_mq_debugfs_unregister_hctxs(q); |
| 258 | 260 | |
| 259 | 261 | kobject_uevent(&q->mq_kobj, KOBJ_REMOVE); |
| 260 | 262 | kobject_del(&q->mq_kobj); |
| 261 | − kobject_put(&q->mq_kobj); | |
| 262 | − | |
| 263 | 263 | kobject_put(&dev->kobj); |
| 264 | 264 | |
| 265 | 265 | q->mq_sysfs_init_done = false; |
| @@ -277,7 +277,19 @@ void blk_mq_hctx_kobj_init(struct blk_mq_hw_ctx *hctx) · 11 unmodified lines | ||
| 277 | 277 | kobject_init(&hctx->kobj, &blk_mq_hw_ktype); |
| 278 | 278 | } |
| 279 | 279 | |
| 280 | −static void blk_mq_sysfs_init(struct request_queue *q) | |
| 280 | +void blk_mq_sysfs_deinit(struct request_queue *q) | |
| 281 | +{ | |
| 282 | + struct blk_mq_ctx *ctx; | |
| 283 | + int cpu; | |
| 284 | + | |
| 285 | + for_each_possible_cpu(cpu) { | |
| 286 | + ctx = per_cpu_ptr(q->queue_ctx, cpu); | |
| 287 | + kobject_put(&ctx->kobj); | |
| 288 | + } | |
| 289 | + kobject_put(&q->mq_kobj); | |
| 290 | +} | |
| 291 | + | |
| 292 | +void blk_mq_sysfs_init(struct request_queue *q) | |
| 281 | 293 | { |
| 282 | 294 | struct blk_mq_ctx *ctx; |
| 283 | 295 | int cpu; |
| @@ -297,8 +309,6 @@ int blk_mq_register_dev(struct device *dev, struct request_queue *q) · 13 unmodified lines | ||
| 297 | 309 | |
| 298 | 310 | blk_mq_disable_hotplug(); |
| 299 | 311 | |
| 300 | − blk_mq_sysfs_init(q); | |
| 301 | − | |
| 302 | 312 | ret = kobject_add(&q->mq_kobj, kobject_get(&dev->kobj), "%s", "mq"); |
| 303 | 313 | if (ret < 0) |
| 304 | 314 | goto out; |
| @@ -295,6 +295,9 @@ int blk_mq_reinit_tagset(struct blk_mq_tag_set *set) · 294 unmodified lines | ||
| 295 | 295 | for (i = 0; i < set->nr_hw_queues; i++) { |
| 296 | 296 | struct blk_mq_tags *tags = set->tags[i]; |
| 297 | 297 | |
| 298 | + if (!tags) | |
| 299 | + continue; | |
| 300 | + | |
| 298 | 301 | for (j = 0; j < tags->nr_tags; j++) { |
| 299 | 302 | if (!tags->static_rqs[j]) |
| 300 | 303 | continue; |
| @@ -1434,7 +1434,8 @@ static blk_qc_t request_to_qc_t(struct blk_mq_hw_ctx *hctx, struct request *rq) · 1433 unmodified lines | ||
| 1434 | 1434 | return blk_tag_to_qc_t(rq->internal_tag, hctx->queue_num, true); |
| 1435 | 1435 | } |
| 1436 | 1436 | |
| 1437 | −static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) | |
| 1437 | +static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie, | |
| 1438 | + bool may_sleep) | |
| 1438 | 1439 | { |
| 1439 | 1440 | struct request_queue *q = rq->q; |
| 1440 | 1441 | struct blk_mq_queue_data bd = { |
| @@ -1475,7 +1476,7 @@ static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) · 34 unmodified lines | ||
| 1475 | 1476 | } |
| 1476 | 1477 | |
| 1477 | 1478 | insert: |
| 1478 | − blk_mq_sched_insert_request(rq, false, true, true, false); | |
| 1479 | + blk_mq_sched_insert_request(rq, false, true, false, may_sleep); | |
| 1479 | 1480 | } |
| 1480 | 1481 | |
| 1481 | 1482 | /* |
| @@ -1569,11 +1570,11 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · 87 unmodified lines | ||
| 1569 | 1570 | |
| 1570 | 1571 | if (!(data.hctx->flags & BLK_MQ_F_BLOCKING)) { |
| 1571 | 1572 | rcu_read_lock(); |
| 1572 | − blk_mq_try_issue_directly(old_rq, &cookie); | |
| 1573 | + blk_mq_try_issue_directly(old_rq, &cookie, false); | |
| 1573 | 1574 | rcu_read_unlock(); |
| 1574 | 1575 | } else { |
| 1575 | 1576 | srcu_idx = srcu_read_lock(&data.hctx->queue_rq_srcu); |
| 1576 | − blk_mq_try_issue_directly(old_rq, &cookie); | |
| 1577 | + blk_mq_try_issue_directly(old_rq, &cookie, true); | |
| 1577 | 1578 | srcu_read_unlock(&data.hctx->queue_rq_srcu, srcu_idx); |
| 1578 | 1579 | } |
| 1579 | 1580 | goto done; |
| @@ -1955,16 +1956,6 @@ static void blk_mq_exit_hw_queues(struct request_queue *q, · 375 unmodified lines | ||
| 1955 | 1956 | } |
| 1956 | 1957 | } |
| 1957 | 1958 | |
| 1958 | −static void blk_mq_free_hw_queues(struct request_queue *q, | |
| 1959 | − struct blk_mq_tag_set *set) | |
| 1960 | −{ | |
| 1961 | − struct blk_mq_hw_ctx *hctx; | |
| 1962 | − unsigned int i; | |
| 1963 | − | |
| 1964 | − queue_for_each_hw_ctx(q, hctx, i) | |
| 1965 | − free_cpumask_var(hctx->cpumask); | |
| 1966 | −} | |
| 1967 | − | |
| 1968 | 1959 | static int blk_mq_init_hctx(struct request_queue *q, |
| 1969 | 1960 | struct blk_mq_tag_set *set, |
| 1970 | 1961 | struct blk_mq_hw_ctx *hctx, unsigned hctx_idx) |
| @@ -2045,7 +2036,6 @@ static void blk_mq_init_cpu_queues(struct request_queue *q, · 74 unmodified lines | ||
| 2045 | 2036 | struct blk_mq_ctx *__ctx = per_cpu_ptr(q->queue_ctx, i); |
| 2046 | 2037 | struct blk_mq_hw_ctx *hctx; |
| 2047 | 2038 | |
| 2048 | − memset(__ctx, 0, sizeof(*__ctx)); | |
| 2049 | 2039 | __ctx->cpu = i; |
| 2050 | 2040 | spin_lock_init(&__ctx->lock); |
| 2051 | 2041 | INIT_LIST_HEAD(&__ctx->rq_list); |
| @@ -2257,15 +2247,19 @@ void blk_mq_release(struct request_queue *q) · 205 unmodified lines | ||
| 2257 | 2247 | queue_for_each_hw_ctx(q, hctx, i) { |
| 2258 | 2248 | if (!hctx) |
| 2259 | 2249 | continue; |
| 2260 | − kfree(hctx->ctxs); | |
| 2261 | − kfree(hctx); | |
| 2250 | + kobject_put(&hctx->kobj); | |
| 2262 | 2251 | } |
| 2263 | 2252 | |
| 2264 | 2253 | q->mq_map = NULL; |
| 2265 | 2254 | |
| 2266 | 2255 | kfree(q->queue_hw_ctx); |
| 2267 | 2256 | |
| 2268 | − /* ctx kobj stays in queue_ctx */ | |
| 2257 | + /* | |
| 2258 | + * release .mq_kobj and sw queue's kobject now because | |
| 2259 | + * both share lifetime with request queue. | |
| 2260 | + */ | |
| 2261 | + blk_mq_sysfs_deinit(q); | |
| 2262 | + | |
| 2269 | 2263 | free_percpu(q->queue_ctx); |
| 2270 | 2264 | } |
| 2271 | 2265 | |
| @@ -2330,10 +2324,7 @@ static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set, · 58 unmodified lines | ||
| 2330 | 2324 | if (hctx->tags) |
| 2331 | 2325 | blk_mq_free_map_and_requests(set, j); |
| 2332 | 2326 | blk_mq_exit_hctx(q, set, hctx, j); |
| 2333 | − free_cpumask_var(hctx->cpumask); | |
| 2334 | 2327 | kobject_put(&hctx->kobj); |
| 2335 | − kfree(hctx->ctxs); | |
| 2336 | − kfree(hctx); | |
| 2337 | 2328 | hctxs[j] = NULL; |
| 2338 | 2329 | |
| 2339 | 2330 | } |
| @@ -2352,6 +2343,9 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set, · 12 unmodified lines | ||
| 2352 | 2343 | if (!q->queue_ctx) |
| 2353 | 2344 | goto err_exit; |
| 2354 | 2345 | |
| 2346 | + /* init q->mq_kobj and sw queues' kobjects */ | |
| 2347 | + blk_mq_sysfs_init(q); | |
| 2348 | + | |
| 2355 | 2349 | q->queue_hw_ctx = kzalloc_node(nr_cpu_ids * sizeof(*(q->queue_hw_ctx)), |
| 2356 | 2350 | GFP_KERNEL, set->numa_node); |
| 2357 | 2351 | if (!q->queue_hw_ctx) |
| @@ -2442,7 +2436,6 @@ void blk_mq_free_queue(struct request_queue *q) · 84 unmodified lines | ||
| 2442 | 2436 | blk_mq_del_queue_tag_set(q); |
| 2443 | 2437 | |
| 2444 | 2438 | blk_mq_exit_hw_queues(q, set, set->nr_hw_queues); |
| 2445 | − blk_mq_free_hw_queues(q, set); | |
| 2446 | 2439 | } |
| 2447 | 2440 | |
| 2448 | 2441 | /* Basically redo blk_mq_init_queue with queue frozen */ |
| @@ -77,6 +77,8 @@ static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q, · 76 unmodified lines | ||
| 77 | 77 | /* |
| 78 | 78 | * sysfs helpers |
| 79 | 79 | */ |
| 80 | +extern void blk_mq_sysfs_init(struct request_queue *q); | |
| 81 | +extern void blk_mq_sysfs_deinit(struct request_queue *q); | |
| 80 | 82 | extern int blk_mq_sysfs_register(struct request_queue *q); |
| 81 | 83 | extern void blk_mq_sysfs_unregister(struct request_queue *q); |
| 82 | 84 | extern void blk_mq_hctx_kobj_init(struct blk_mq_hw_ctx *hctx); |
| @@ -572,20 +572,6 @@ exit: · 571 unmodified lines | ||
| 572 | 572 | disk_part_iter_exit(&piter); |
| 573 | 573 | } |
| 574 | 574 | |
| 575 | −void put_disk_devt(struct disk_devt *disk_devt) | |
| 576 | −{ | |
| 577 | − if (disk_devt && atomic_dec_and_test(&disk_devt->count)) | |
| 578 | − disk_devt->release(disk_devt); | |
| 579 | −} | |
| 580 | −EXPORT_SYMBOL(put_disk_devt); | |
| 581 | − | |
| 582 | −void get_disk_devt(struct disk_devt *disk_devt) | |
| 583 | −{ | |
| 584 | − if (disk_devt) | |
| 585 | − atomic_inc(&disk_devt->count); | |
| 586 | −} | |
| 587 | −EXPORT_SYMBOL(get_disk_devt); | |
| 588 | − | |
| 589 | 575 | /** |
| 590 | 576 | * device_add_disk - add partitioning information to kernel list |
| 591 | 577 | * @parent: parent device for the disk |
| @@ -626,13 +612,6 @@ void device_add_disk(struct device *parent, struct gendisk *disk) · 34 unmodified lines | ||
| 626 | 612 | |
| 627 | 613 | disk_alloc_events(disk); |
| 628 | 614 | |
| 629 | − /* | |
| 630 | − * Take a reference on the devt and assign it to queue since it | |
| 631 | − * must not be reallocated while the bdi is registered | |
| 632 | − */ | |
| 633 | − disk->queue->disk_devt = disk->disk_devt; | |
| 634 | − get_disk_devt(disk->disk_devt); | |
| 635 | − | |
| 636 | 615 | /* Register BDI before referencing it from bdev */ |
| 637 | 616 | bdi = disk->queue->backing_dev_info; |
| 638 | 617 | bdi_register_owner(bdi, disk_to_dev(disk)); |
| @@ -681,12 +660,16 @@ void del_gendisk(struct gendisk *disk) · 42 unmodified lines | ||
| 681 | 660 | disk->flags &= ~GENHD_FL_UP; |
| 682 | 661 | |
| 683 | 662 | sysfs_remove_link(&disk_to_dev(disk)->kobj, "bdi"); |
| 684 | − /* | |
| 685 | − * Unregister bdi before releasing device numbers (as they can get | |
| 686 | − * reused and we'd get clashes in sysfs). | |
| 687 | − */ | |
| 688 | − bdi_unregister(disk->queue->backing_dev_info); | |
| 689 | − blk_unregister_queue(disk); | |
| 663 | + if (disk->queue) { | |
| 664 | + /* | |
| 665 | + * Unregister bdi before releasing device numbers (as they can | |
| 666 | + * get reused and we'd get clashes in sysfs). | |
| 667 | + */ | |
| 668 | + bdi_unregister(disk->queue->backing_dev_info); | |
| 669 | + blk_unregister_queue(disk); | |
| 670 | + } else { | |
| 671 | + WARN_ON(1); | |
| 672 | + } | |
| 690 | 673 | blk_unregister_region(disk_devt(disk), disk->minors); |
| 691 | 674 | |
| 692 | 675 | part_stat_set_all(&disk->part0, 0); |
| @@ -1023,7 +1023,6 @@ static int finalize_and_send(struct opal_dev *dev, cont_fn cont) · 1022 unmodified lines | ||
| 1023 | 1023 | |
| 1024 | 1024 | static int gen_key(struct opal_dev *dev, void *data) |
| 1025 | 1025 | { |
| 1026 | − const u8 *method; | |
| 1027 | 1026 | u8 uid[OPAL_UID_LENGTH]; |
| 1028 | 1027 | int err = 0; |
| 1029 | 1028 | |
| @@ -1031,7 +1030,6 @@ static int gen_key(struct opal_dev *dev, void *data) · 1 unmodified line | ||
| 1031 | 1030 | set_comid(dev, dev->comid); |
| 1032 | 1031 | |
| 1033 | 1032 | memcpy(uid, dev->prev_data, min(sizeof(uid), dev->prev_d_len)); |
| 1034 | − method = opalmethod[OPAL_GENKEY]; | |
| 1035 | 1033 | kfree(dev->prev_data); |
| 1036 | 1034 | dev->prev_data = NULL; |
| 1037 | 1035 | |
| @@ -1669,7 +1667,6 @@ static int add_user_to_lr(struct opal_dev *dev, void *data) · 631 unmodified lines | ||
| 1669 | 1667 | static int lock_unlock_locking_range(struct opal_dev *dev, void *data) |
| 1670 | 1668 | { |
| 1671 | 1669 | u8 lr_buffer[OPAL_UID_LENGTH]; |
| 1672 | − const u8 *method; | |
| 1673 | 1670 | struct opal_lock_unlock *lkul = data; |
| 1674 | 1671 | u8 read_locked = 1, write_locked = 1; |
| 1675 | 1672 | int err = 0; |
| @@ -1677,7 +1674,6 @@ static int lock_unlock_locking_range(struct opal_dev *dev, void *data) · 1 unmodified line | ||
| 1677 | 1674 | clear_opal_cmd(dev); |
| 1678 | 1675 | set_comid(dev, dev->comid); |
| 1679 | 1676 | |
| 1680 | − method = opalmethod[OPAL_SET]; | |
| 1681 | 1677 | if (build_locking_range(lr_buffer, sizeof(lr_buffer), |
| 1682 | 1678 | lkul->session.opal_key.lr) < 0) |
| 1683 | 1679 | return -ERANGE; |
| @@ -1733,14 +1729,12 @@ static int lock_unlock_locking_range_sum(struct opal_dev *dev, void *data) · 49 unmodified lines | ||
| 1733 | 1729 | { |
| 1734 | 1730 | u8 lr_buffer[OPAL_UID_LENGTH]; |
| 1735 | 1731 | u8 read_locked = 1, write_locked = 1; |
| 1736 | − const u8 *method; | |
| 1737 | 1732 | struct opal_lock_unlock *lkul = data; |
| 1738 | 1733 | int ret; |
| 1739 | 1734 | |
| 1740 | 1735 | clear_opal_cmd(dev); |
| 1741 | 1736 | set_comid(dev, dev->comid); |
| 1742 | 1737 | |
| 1743 | − method = opalmethod[OPAL_SET]; | |
| 1744 | 1738 | if (build_locking_range(lr_buffer, sizeof(lr_buffer), |
| 1745 | 1739 | lkul->session.opal_key.lr) < 0) |
| 1746 | 1740 | return -ERANGE; |
| @@ -2133,7 +2127,7 @@ static int opal_add_user_to_lr(struct opal_dev *dev, · 386 unmodified lines | ||
| 2133 | 2127 | pr_err("Locking state was not RO or RW\n"); |
| 2134 | 2128 | return -EINVAL; |
| 2135 | 2129 | } |
| 2136 | − if (lk_unlk->session.who < OPAL_USER1 && | |
| 2130 | + if (lk_unlk->session.who < OPAL_USER1 || | |
| 2137 | 2131 | lk_unlk->session.who > OPAL_USER9) { |
| 2138 | 2132 | pr_err("Authority was not within the range of users: %d\n", |
| 2139 | 2133 | lk_unlk->session.who); |
| @@ -2316,7 +2310,7 @@ static int opal_activate_user(struct opal_dev *dev, · 176 unmodified lines | ||
| 2316 | 2310 | int ret; |
| 2317 | 2311 | |
| 2318 | 2312 | /* We can't activate Admin1 it's active as manufactured */ |
| 2319 | − if (opal_session->who < OPAL_USER1 && | |
| 2313 | + if (opal_session->who < OPAL_USER1 || | |
| 2320 | 2314 | opal_session->who > OPAL_USER9) { |
| 2321 | 2315 | pr_err("Who was not a valid user: %d\n", opal_session->who); |
| 2322 | 2316 | return -EINVAL; |
| @@ -266,7 +266,7 @@ unlock: · 265 unmodified lines | ||
| 266 | 266 | return err; |
| 267 | 267 | } |
| 268 | 268 | |
| 269 | −int af_alg_accept(struct sock *sk, struct socket *newsock) | |
| 269 | +int af_alg_accept(struct sock *sk, struct socket *newsock, bool kern) | |
| 270 | 270 | { |
| 271 | 271 | struct alg_sock *ask = alg_sk(sk); |
| 272 | 272 | const struct af_alg_type *type; |
| @@ -281,7 +281,7 @@ int af_alg_accept(struct sock *sk, struct socket *newsock) · 8 unmodified lines | ||
| 281 | 281 | if (!type) |
| 282 | 282 | goto unlock; |
| 283 | 283 | |
| 284 | − sk2 = sk_alloc(sock_net(sk), PF_ALG, GFP_KERNEL, &alg_proto, 0); | |
| 284 | + sk2 = sk_alloc(sock_net(sk), PF_ALG, GFP_KERNEL, &alg_proto, kern); | |
| 285 | 285 | err = -ENOMEM; |
| 286 | 286 | if (!sk2) |
| 287 | 287 | goto unlock; |
| @@ -323,9 +323,10 @@ unlock: · 35 unmodified lines | ||
| 323 | 323 | } |
| 324 | 324 | EXPORT_SYMBOL_GPL(af_alg_accept); |
| 325 | 325 | |
| 326 | −static int alg_accept(struct socket *sock, struct socket *newsock, int flags) | |
| 326 | +static int alg_accept(struct socket *sock, struct socket *newsock, int flags, | |
| 327 | + bool kern) | |
| 327 | 328 | { |
| 328 | − return af_alg_accept(sock->sk, newsock); | |
| 329 | + return af_alg_accept(sock->sk, newsock, kern); | |
| 329 | 330 | } |
| 330 | 331 | |
| 331 | 332 | static const struct proto_ops alg_proto_ops = { |
| @@ -239,7 +239,8 @@ unlock: · 238 unmodified lines | ||
| 239 | 239 | return err ?: len; |
| 240 | 240 | } |
| 241 | 241 | |
| 242 | −static int hash_accept(struct socket *sock, struct socket *newsock, int flags) | |
| 242 | +static int hash_accept(struct socket *sock, struct socket *newsock, int flags, | |
| 243 | + bool kern) | |
| 243 | 244 | { |
| 244 | 245 | struct sock *sk = sock->sk; |
| 245 | 246 | struct alg_sock *ask = alg_sk(sk); |
| @@ -260,7 +261,7 @@ static int hash_accept(struct socket *sock, struct socket *newsock, int flags) · 14 unmodified lines | ||
| 260 | 261 | if (err) |
| 261 | 262 | return err; |
| 262 | 263 | |
| 263 | − err = af_alg_accept(ask->parent, newsock); | |
| 264 | + err = af_alg_accept(ask->parent, newsock, kern); | |
| 264 | 265 | if (err) |
| 265 | 266 | return err; |
| 266 | 267 | |
| @@ -378,7 +379,7 @@ static int hash_recvmsg_nokey(struct socket *sock, struct msghdr *msg, · 111 unmodified lines | ||
| 378 | 379 | } |
| 379 | 380 | |
| 380 | 381 | static int hash_accept_nokey(struct socket *sock, struct socket *newsock, |
| 381 | − int flags) | |
| 382 | + int flags, bool kern) | |
| 382 | 383 | { |
| 383 | 384 | int err; |
| 384 | 385 | |
| @@ -386,7 +387,7 @@ static int hash_accept_nokey(struct socket *sock, struct socket *newsock, · 1 unmodified line | ||
| 386 | 387 | if (err) |
| 387 | 388 | return err; |
| 388 | 389 | |
| 389 | − return hash_accept(sock, newsock, flags); | |
| 390 | + return hash_accept(sock, newsock, flags, kern); | |
| 390 | 391 | } |
| 391 | 392 | |
| 392 | 393 | static struct proto_ops algif_hash_ops_nokey = { |
| @@ -41,8 +41,10 @@ void acpi_gpe_apply_masked_gpes(void); · 40 unmodified lines | ||
| 41 | 41 | void acpi_container_init(void); |
| 42 | 42 | void acpi_memory_hotplug_init(void); |
| 43 | 43 | #ifdef CONFIG_ACPI_HOTPLUG_IOAPIC |
| 44 | +void pci_ioapic_remove(struct acpi_pci_root *root); | |
| 44 | 45 | int acpi_ioapic_remove(struct acpi_pci_root *root); |
| 45 | 46 | #else |
| 47 | +static inline void pci_ioapic_remove(struct acpi_pci_root *root) { return; } | |
| 46 | 48 | static inline int acpi_ioapic_remove(struct acpi_pci_root *root) { return 0; } |
| 47 | 49 | #endif |
| 48 | 50 | #ifdef CONFIG_ACPI_DOCK |
| @@ -206,24 +206,34 @@ int acpi_ioapic_add(acpi_handle root_handle) · 205 unmodified lines | ||
| 206 | 206 | return ACPI_SUCCESS(status) && ACPI_SUCCESS(retval) ? 0 : -ENODEV; |
| 207 | 207 | } |
| 208 | 208 | |
| 209 | −int acpi_ioapic_remove(struct acpi_pci_root *root) | |
| 209 | +void pci_ioapic_remove(struct acpi_pci_root *root) | |
| 210 | 210 | { |
| 211 | − int retval = 0; | |
| 212 | 211 | struct acpi_pci_ioapic *ioapic, *tmp; |
| 213 | 212 | |
| 214 | 213 | mutex_lock(&ioapic_list_lock); |
| 215 | 214 | list_for_each_entry_safe(ioapic, tmp, &ioapic_list, list) { |
| 216 | 215 | if (root->device->handle != ioapic->root_handle) |
| 217 | 216 | continue; |
| 218 | − | |
| 219 | − if (acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base)) | |
| 220 | − retval = -EBUSY; | |
| 221 | − | |
| 222 | 217 | if (ioapic->pdev) { |
| 223 | 218 | pci_release_region(ioapic->pdev, 0); |
| 224 | 219 | pci_disable_device(ioapic->pdev); |
| 225 | 220 | pci_dev_put(ioapic->pdev); |
| 226 | 221 | } |
| 222 | + } | |
| 223 | + mutex_unlock(&ioapic_list_lock); | |
| 224 | +} | |
| 225 | + | |
| 226 | +int acpi_ioapic_remove(struct acpi_pci_root *root) | |
| 227 | +{ | |
| 228 | + int retval = 0; | |
| 229 | + struct acpi_pci_ioapic *ioapic, *tmp; | |
| 230 | + | |
| 231 | + mutex_lock(&ioapic_list_lock); | |
| 232 | + list_for_each_entry_safe(ioapic, tmp, &ioapic_list, list) { | |
| 233 | + if (root->device->handle != ioapic->root_handle) | |
| 234 | + continue; | |
| 235 | + if (acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base)) | |
| 236 | + retval = -EBUSY; | |
| 227 | 237 | if (ioapic->res.flags && ioapic->res.parent) |
| 228 | 238 | release_resource(&ioapic->res); |
| 229 | 239 | list_del(&ioapic->list); |
| @@ -648,12 +648,12 @@ static void acpi_pci_root_remove(struct acpi_device *device) · 647 unmodified lines | ||
| 648 | 648 | |
| 649 | 649 | pci_stop_root_bus(root->bus); |
| 650 | 650 | |
| 651 | − WARN_ON(acpi_ioapic_remove(root)); | |
| 652 | − | |
| 651 | + pci_ioapic_remove(root); | |
| 653 | 652 | device_set_run_wake(root->bus->bridge, false); |
| 654 | 653 | pci_acpi_remove_bus_pm_notifier(device); |
| 655 | 654 | |
| 656 | 655 | pci_remove_root_bus(root->bus); |
| 656 | + WARN_ON(acpi_ioapic_remove(root)); | |
| 657 | 657 | |
| 658 | 658 | dmar_device_remove(device->handle); |
| 659 | 659 | |
| @@ -177,7 +177,8 @@ static int ahci_qoriq_phy_init(struct ahci_host_priv *hpriv) · 176 unmodified lines | ||
| 177 | 177 | case AHCI_LS1043A: |
| 178 | 178 | if (!qpriv->ecc_addr) |
| 179 | 179 | return -EINVAL; |
| 180 | − writel(ECC_DIS_ARMV8_CH2, qpriv->ecc_addr); | |
| 180 | + writel(readl(qpriv->ecc_addr) | ECC_DIS_ARMV8_CH2, | |
| 181 | + qpriv->ecc_addr); | |
| 181 | 182 | writel(AHCI_PORT_PHY_1_CFG, reg_base + PORT_PHY1); |
| 182 | 183 | writel(AHCI_PORT_TRANS_CFG, reg_base + PORT_TRANS); |
| 183 | 184 | if (qpriv->is_dmacoherent) |
| @@ -194,7 +195,8 @@ static int ahci_qoriq_phy_init(struct ahci_host_priv *hpriv) · 10 unmodified lines | ||
| 194 | 195 | case AHCI_LS1046A: |
| 195 | 196 | if (!qpriv->ecc_addr) |
| 196 | 197 | return -EINVAL; |
| 197 | − writel(ECC_DIS_ARMV8_CH2, qpriv->ecc_addr); | |
| 198 | + writel(readl(qpriv->ecc_addr) | ECC_DIS_ARMV8_CH2, | |
| 199 | + qpriv->ecc_addr); | |
| 198 | 200 | writel(AHCI_PORT_PHY_1_CFG, reg_base + PORT_PHY1); |
| 199 | 201 | writel(AHCI_PORT_TRANS_CFG, reg_base + PORT_TRANS); |
| 200 | 202 | if (qpriv->is_dmacoherent) |
| @@ -1482,7 +1482,6 @@ unsigned int ata_sff_qc_issue(struct ata_queued_cmd *qc) · 1481 unmodified lines | ||
| 1482 | 1482 | break; |
| 1483 | 1483 | |
| 1484 | 1484 | default: |
| 1485 | − WARN_ON_ONCE(1); | |
| 1486 | 1485 | return AC_ERR_SYSTEM; |
| 1487 | 1486 | } |
| 1488 | 1487 | |
| @@ -224,7 +224,6 @@ static DECLARE_TRANSPORT_CLASS(ata_port_class, · 223 unmodified lines | ||
| 224 | 224 | |
| 225 | 225 | static void ata_tport_release(struct device *dev) |
| 226 | 226 | { |
| 227 | − put_device(dev->parent); | |
| 228 | 227 | } |
| 229 | 228 | |
| 230 | 229 | /** |
| @@ -284,7 +283,7 @@ int ata_tport_add(struct device *parent, · 53 unmodified lines | ||
| 284 | 283 | device_initialize(dev); |
| 285 | 284 | dev->type = &ata_port_type; |
| 286 | 285 | |
| 287 | − dev->parent = get_device(parent); | |
| 286 | + dev->parent = parent; | |
| 288 | 287 | dev->release = ata_tport_release; |
| 289 | 288 | dev_set_name(dev, "ata%d", ap->print_id); |
| 290 | 289 | transport_setup_device(dev); |
| @@ -348,7 +347,6 @@ static DECLARE_TRANSPORT_CLASS(ata_link_class, · 57 unmodified lines | ||
| 348 | 347 | |
| 349 | 348 | static void ata_tlink_release(struct device *dev) |
| 350 | 349 | { |
| 351 | − put_device(dev->parent); | |
| 352 | 350 | } |
| 353 | 351 | |
| 354 | 352 | /** |
| @@ -410,7 +408,7 @@ int ata_tlink_add(struct ata_link *link) · 55 unmodified lines | ||
| 410 | 408 | int error; |
| 411 | 409 | |
| 412 | 410 | device_initialize(dev); |
| 413 | − dev->parent = get_device(&ap->tdev); | |
| 411 | + dev->parent = &ap->tdev; | |
| 414 | 412 | dev->release = ata_tlink_release; |
| 415 | 413 | if (ata_is_host_link(link)) |
| 416 | 414 | dev_set_name(dev, "link%d", ap->print_id); |
| @@ -589,7 +587,6 @@ static DECLARE_TRANSPORT_CLASS(ata_dev_class, · 172 unmodified lines | ||
| 589 | 587 | |
| 590 | 588 | static void ata_tdev_release(struct device *dev) |
| 591 | 589 | { |
| 592 | − put_device(dev->parent); | |
| 593 | 590 | } |
| 594 | 591 | |
| 595 | 592 | /** |
| @@ -662,7 +659,7 @@ static int ata_tdev_add(struct ata_device *ata_dev) · 66 unmodified lines | ||
| 662 | 659 | int error; |
| 663 | 660 | |
| 664 | 661 | device_initialize(dev); |
| 665 | − dev->parent = get_device(&link->tdev); | |
| 662 | + dev->parent = &link->tdev; | |
| 666 | 663 | dev->release = ata_tdev_release; |
| 667 | 664 | if (ata_is_host_link(link)) |
| 668 | 665 | dev_set_name(dev, "dev%d.%d", ap->print_id,ata_dev->devno); |
| @@ -50,7 +50,7 @@ · 49 unmodified lines | ||
| 50 | 50 | the slower the port i/o. In some cases, setting |
| 51 | 51 | this to zero will speed up the device. (default -1) |
| 52 | 52 | |
| 53 | − major You may use this parameter to overide the | |
| 53 | + major You may use this parameter to override the | |
| 54 | 54 | default major number (46) that this driver |
| 55 | 55 | will use. Be sure to change the device |
| 56 | 56 | name as well. |
| @@ -61,7 +61,7 @@ · 60 unmodified lines | ||
| 61 | 61 | first drive found. |
| 62 | 62 | |
| 63 | 63 | |
| 64 | − major You may use this parameter to overide the | |
| 64 | + major You may use this parameter to override the | |
| 65 | 65 | default major number (45) that this driver |
| 66 | 66 | will use. Be sure to change the device |
| 67 | 67 | name as well. |
| @@ -59,7 +59,7 @@ · 58 unmodified lines | ||
| 59 | 59 | the slower the port i/o. In some cases, setting |
| 60 | 60 | this to zero will speed up the device. (default -1) |
| 61 | 61 | |
| 62 | − major You may use this parameter to overide the | |
| 62 | + major You may use this parameter to override the | |
| 63 | 63 | default major number (47) that this driver |
| 64 | 64 | will use. Be sure to change the device |
| 65 | 65 | name as well. |
| @@ -84,7 +84,7 @@ · 83 unmodified lines | ||
| 84 | 84 | the slower the port i/o. In some cases, setting |
| 85 | 85 | this to zero will speed up the device. (default -1) |
| 86 | 86 | |
| 87 | − major You may use this parameter to overide the | |
| 87 | + major You may use this parameter to override the | |
| 88 | 88 | default major number (97) that this driver |
| 89 | 89 | will use. Be sure to change the device |
| 90 | 90 | name as well. |
| @@ -61,7 +61,7 @@ · 60 unmodified lines | ||
| 61 | 61 | the slower the port i/o. In some cases, setting |
| 62 | 62 | this to zero will speed up the device. (default -1) |
| 63 | 63 | |
| 64 | − major You may use this parameter to overide the | |
| 64 | + major You may use this parameter to override the | |
| 65 | 65 | default major number (96) that this driver |
| 66 | 66 | will use. Be sure to change the device |
| 67 | 67 | name as well. |
| @@ -120,10 +120,11 @@ static int atomic_dec_return_safe(atomic_t *v) · 119 unmodified lines | ||
| 120 | 120 | |
| 121 | 121 | /* Feature bits */ |
| 122 | 122 | |
| 123 | −#define RBD_FEATURE_LAYERING (1<<0) | |
| 124 | −#define RBD_FEATURE_STRIPINGV2 (1<<1) | |
| 125 | −#define RBD_FEATURE_EXCLUSIVE_LOCK (1<<2) | |
| 126 | −#define RBD_FEATURE_DATA_POOL (1<<7) | |
| 123 | +#define RBD_FEATURE_LAYERING (1ULL<<0) | |
| 124 | +#define RBD_FEATURE_STRIPINGV2 (1ULL<<1) | |
| 125 | +#define RBD_FEATURE_EXCLUSIVE_LOCK (1ULL<<2) | |
| 126 | +#define RBD_FEATURE_DATA_POOL (1ULL<<7) | |
| 127 | + | |
| 127 | 128 | #define RBD_FEATURES_ALL (RBD_FEATURE_LAYERING | \ |
| 128 | 129 | RBD_FEATURE_STRIPINGV2 | \ |
| 129 | 130 | RBD_FEATURE_EXCLUSIVE_LOCK | \ |
| @@ -499,16 +500,23 @@ static bool rbd_is_lock_owner(struct rbd_device *rbd_dev) · 369 unmodified lines | ||
| 499 | 500 | return is_lock_owner; |
| 500 | 501 | } |
| 501 | 502 | |
| 503 | +static ssize_t rbd_supported_features_show(struct bus_type *bus, char *buf) | |
| 504 | +{ | |
| 505 | + return sprintf(buf, "0x%llx\n", RBD_FEATURES_SUPPORTED); | |
| 506 | +} | |
| 507 | + | |
| 502 | 508 | static BUS_ATTR(add, S_IWUSR, NULL, rbd_add); |
| 503 | 509 | static BUS_ATTR(remove, S_IWUSR, NULL, rbd_remove); |
| 504 | 510 | static BUS_ATTR(add_single_major, S_IWUSR, NULL, rbd_add_single_major); |
| 505 | 511 | static BUS_ATTR(remove_single_major, S_IWUSR, NULL, rbd_remove_single_major); |
| 512 | +static BUS_ATTR(supported_features, S_IRUGO, rbd_supported_features_show, NULL); | |
| 506 | 513 | |
| 507 | 514 | static struct attribute *rbd_bus_attrs[] = { |
| 508 | 515 | &bus_attr_add.attr, |
| 509 | 516 | &bus_attr_remove.attr, |
| 510 | 517 | &bus_attr_add_single_major.attr, |
| 511 | 518 | &bus_attr_remove_single_major.attr, |
| 519 | + &bus_attr_supported_features.attr, | |
| 512 | 520 | NULL, |
| 513 | 521 | }; |
| 514 | 522 | |
| @@ -1189,6 +1189,8 @@ static int zram_add(void) · 1188 unmodified lines | ||
| 1189 | 1189 | blk_queue_io_min(zram->disk->queue, PAGE_SIZE); |
| 1190 | 1190 | blk_queue_io_opt(zram->disk->queue, PAGE_SIZE); |
| 1191 | 1191 | zram->disk->queue->limits.discard_granularity = PAGE_SIZE; |
| 1192 | + zram->disk->queue->limits.max_sectors = SECTORS_PER_PAGE; | |
| 1193 | + zram->disk->queue->limits.chunk_sectors = 0; | |
| 1192 | 1194 | blk_queue_max_discard_sectors(zram->disk->queue, UINT_MAX); |
| 1193 | 1195 | /* |
| 1194 | 1196 | * zram_bio_discard() will clear all logical blocks if logical block |
| @@ -397,9 +397,8 @@ static int of_get_omap_rng_device_details(struct omap_rng_dev *priv, · 396 unmodified lines | ||
| 397 | 397 | irq, err); |
| 398 | 398 | return err; |
| 399 | 399 | } |
| 400 | − omap_rng_write(priv, RNG_INTMASK_REG, RNG_SHUTDOWN_OFLO_MASK); | |
| 401 | 400 | |
| 402 | − priv->clk = of_clk_get(pdev->dev.of_node, 0); | |
| 401 | + priv->clk = devm_clk_get(&pdev->dev, NULL); | |
| 403 | 402 | if (IS_ERR(priv->clk) && PTR_ERR(priv->clk) == -EPROBE_DEFER) |
| 404 | 403 | return -EPROBE_DEFER; |
| 405 | 404 | if (!IS_ERR(priv->clk)) { |
| @@ -408,6 +407,19 @@ static int of_get_omap_rng_device_details(struct omap_rng_dev *priv, · 2 unmodified lines | ||
| 408 | 407 | dev_err(&pdev->dev, "unable to enable the clk, " |
| 409 | 408 | "err = %d\n", err); |
| 410 | 409 | } |
| 410 | + | |
| 411 | + /* | |
| 412 | + * On OMAP4, enabling the shutdown_oflo interrupt is | |
| 413 | + * done in the interrupt mask register. There is no | |
| 414 | + * such register on EIP76, and it's enabled by the | |
| 415 | + * same bit in the control register | |
| 416 | + */ | |
| 417 | + if (priv->pdata->regs[RNG_INTMASK_REG]) | |
| 418 | + omap_rng_write(priv, RNG_INTMASK_REG, | |
| 419 | + RNG_SHUTDOWN_OFLO_MASK); | |
| 420 | + else | |
| 421 | + omap_rng_write(priv, RNG_CONTROL_REG, | |
| 422 | + RNG_SHUTDOWN_OFLO_MASK); | |
| 411 | 423 | } |
| 412 | 424 | return 0; |
| 413 | 425 | } |
| @@ -6,7 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | #include <linux/module.h> |
| 8 | 8 | #include <linux/kernel.h> |
| 9 | −#include <linux/sched.h> | |
| 9 | +#include <linux/sched/signal.h> | |
| 10 | 10 | #include <linux/interrupt.h> |
| 11 | 11 | #include <linux/time.h> |
| 12 | 12 | #include <linux/timer.h> |
| @@ -312,13 +312,6 @@ static int random_read_wakeup_bits = 64; · 311 unmodified lines | ||
| 312 | 312 | */ |
| 313 | 313 | static int random_write_wakeup_bits = 28 * OUTPUT_POOL_WORDS; |
| 314 | 314 | |
| 315 | −/* | |
| 316 | − * The minimum number of seconds between urandom pool reseeding. We | |
| 317 | − * do this to limit the amount of entropy that can be drained from the | |
| 318 | − * input pool even if there are heavy demands on /dev/urandom. | |
| 319 | − */ | |
| 320 | −static int random_min_urandom_seed = 60; | |
| 321 | − | |
| 322 | 315 | /* |
| 323 | 316 | * Originally, we used a primitive polynomial of degree .poolwords |
| 324 | 317 | * over GF(2). The taps for various sizes are defined below. They |
| @@ -409,7 +402,6 @@ static struct poolinfo { · 84 unmodified lines | ||
| 409 | 402 | */ |
| 410 | 403 | static DECLARE_WAIT_QUEUE_HEAD(random_read_wait); |
| 411 | 404 | static DECLARE_WAIT_QUEUE_HEAD(random_write_wait); |
| 412 | −static DECLARE_WAIT_QUEUE_HEAD(urandom_init_wait); | |
| 413 | 405 | static struct fasync_struct *fasync; |
| 414 | 406 | |
| 415 | 407 | static DEFINE_SPINLOCK(random_ready_list_lock); |
| @@ -467,7 +459,6 @@ struct entropy_store { · 51 unmodified lines | ||
| 467 | 459 | int entropy_count; |
| 468 | 460 | int entropy_total; |
| 469 | 461 | unsigned int initialized:1; |
| 470 | − unsigned int limit:1; | |
| 471 | 462 | unsigned int last_data_init:1; |
| 472 | 463 | __u8 last_data[EXTRACT_SIZE]; |
| 473 | 464 | }; |
| @@ -485,7 +476,6 @@ static __u32 blocking_pool_data[OUTPUT_POOL_WORDS] __latent_entropy; · 11 unmodified lines | ||
| 485 | 476 | static struct entropy_store input_pool = { |
| 486 | 477 | .poolinfo = &poolinfo_table[0], |
| 487 | 478 | .name = "input", |
| 488 | − .limit = 1, | |
| 489 | 479 | .lock = __SPIN_LOCK_UNLOCKED(input_pool.lock), |
| 490 | 480 | .pool = input_pool_data |
| 491 | 481 | }; |
| @@ -493,7 +483,6 @@ static struct entropy_store input_pool = { · 1 unmodified line | ||
| 493 | 483 | static struct entropy_store blocking_pool = { |
| 494 | 484 | .poolinfo = &poolinfo_table[1], |
| 495 | 485 | .name = "blocking", |
| 496 | − .limit = 1, | |
| 497 | 486 | .pull = &input_pool, |
| 498 | 487 | .lock = __SPIN_LOCK_UNLOCKED(blocking_pool.lock), |
| 499 | 488 | .pool = blocking_pool_data, |
| @@ -855,13 +844,6 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r) · 355 unmodified lines | ||
| 855 | 844 | spin_unlock_irqrestore(&primary_crng.lock, flags); |
| 856 | 845 | } |
| 857 | 846 | |
| 858 | −static inline void maybe_reseed_primary_crng(void) | |
| 859 | −{ | |
| 860 | − if (crng_init > 2 && | |
| 861 | − time_after(jiffies, primary_crng.init_time + CRNG_RESEED_INTERVAL)) | |
| 862 | − crng_reseed(&primary_crng, &input_pool); | |
| 863 | −} | |
| 864 | − | |
| 865 | 847 | static inline void crng_wait_ready(void) |
| 866 | 848 | { |
| 867 | 849 | wait_event_interruptible(crng_init_wait, crng_ready()); |
| @@ -1220,15 +1202,6 @@ static void xfer_secondary_pool(struct entropy_store *r, size_t nbytes) · 352 unmodified lines | ||
| 1220 | 1202 | r->entropy_count > r->poolinfo->poolfracbits) |
| 1221 | 1203 | return; |
| 1222 | 1204 | |
| 1223 | − if (r->limit == 0 && random_min_urandom_seed) { | |
| 1224 | − unsigned long now = jiffies; | |
| 1225 | − | |
| 1226 | − if (time_before(now, | |
| 1227 | − r->last_pulled + random_min_urandom_seed * HZ)) | |
| 1228 | − return; | |
| 1229 | − r->last_pulled = now; | |
| 1230 | − } | |
| 1231 | − | |
| 1232 | 1205 | _xfer_secondary_pool(r, nbytes); |
| 1233 | 1206 | } |
| 1234 | 1207 | |
| @@ -1236,8 +1209,6 @@ static void _xfer_secondary_pool(struct entropy_store *r, size_t nbytes) · 1 unmodified line | ||
| 1236 | 1209 | { |
| 1237 | 1210 | __u32 tmp[OUTPUT_POOL_WORDS]; |
| 1238 | 1211 | |
| 1239 | − /* For /dev/random's pool, always leave two wakeups' worth */ | |
| 1240 | − int rsvd_bytes = r->limit ? 0 : random_read_wakeup_bits / 4; | |
| 1241 | 1212 | int bytes = nbytes; |
| 1242 | 1213 | |
| 1243 | 1214 | /* pull at least as much as a wakeup */ |
| @@ -1248,7 +1219,7 @@ static void _xfer_secondary_pool(struct entropy_store *r, size_t nbytes) · 4 unmodified lines | ||
| 1248 | 1219 | trace_xfer_secondary_pool(r->name, bytes * 8, nbytes * 8, |
| 1249 | 1220 | ENTROPY_BITS(r), ENTROPY_BITS(r->pull)); |
| 1250 | 1221 | bytes = extract_entropy(r->pull, tmp, bytes, |
| 1251 | − random_read_wakeup_bits / 8, rsvd_bytes); | |
| 1222 | + random_read_wakeup_bits / 8, 0); | |
| 1252 | 1223 | mix_pool_bytes(r, tmp, bytes); |
| 1253 | 1224 | credit_entropy_bits(r, bytes*8); |
| 1254 | 1225 | } |
| @@ -1276,7 +1247,7 @@ static void push_to_pool(struct work_struct *work) · 21 unmodified lines | ||
| 1276 | 1247 | static size_t account(struct entropy_store *r, size_t nbytes, int min, |
| 1277 | 1248 | int reserved) |
| 1278 | 1249 | { |
| 1279 | − int entropy_count, orig; | |
| 1250 | + int entropy_count, orig, have_bytes; | |
| 1280 | 1251 | size_t ibytes, nfrac; |
| 1281 | 1252 | |
| 1282 | 1253 | BUG_ON(r->entropy_count > r->poolinfo->poolfracbits); |
| @@ -1285,14 +1256,12 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min, · 2 unmodified lines | ||
| 1285 | 1256 | retry: |
| 1286 | 1257 | entropy_count = orig = ACCESS_ONCE(r->entropy_count); |
| 1287 | 1258 | ibytes = nbytes; |
| 1288 | − /* If limited, never pull more than available */ | |
| 1289 | − if (r->limit) { | |
| 1290 | − int have_bytes = entropy_count >> (ENTROPY_SHIFT + 3); | |
| 1259 | + /* never pull more than available */ | |
| 1260 | + have_bytes = entropy_count >> (ENTROPY_SHIFT + 3); | |
| 1291 | 1261 | |
| 1292 | − if ((have_bytes -= reserved) < 0) | |
| 1293 | − have_bytes = 0; | |
| 1294 | − ibytes = min_t(size_t, ibytes, have_bytes); | |
| 1295 | − } | |
| 1262 | + if ((have_bytes -= reserved) < 0) | |
| 1263 | + have_bytes = 0; | |
| 1264 | + ibytes = min_t(size_t, ibytes, have_bytes); | |
| 1296 | 1265 | if (ibytes < min) |
| 1297 | 1266 | ibytes = 0; |
| 1298 | 1267 | |
| @@ -1912,6 +1881,7 @@ SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, · 613 unmodified lines | ||
| 1912 | 1881 | static int min_read_thresh = 8, min_write_thresh; |
| 1913 | 1882 | static int max_read_thresh = OUTPUT_POOL_WORDS * 32; |
| 1914 | 1883 | static int max_write_thresh = INPUT_POOL_WORDS * 32; |
| 1884 | +static int random_min_urandom_seed = 60; | |
| 1915 | 1885 | static char sysctl_bootid[16]; |
| 1916 | 1886 | |
| 1917 | 1887 | /* |
| @@ -2042,63 +2012,64 @@ struct ctl_table random_table[] = { · 124 unmodified lines | ||
| 2042 | 2012 | }; |
| 2043 | 2013 | #endif /* CONFIG_SYSCTL */ |
| 2044 | 2014 | |
| 2045 | −static u32 random_int_secret[MD5_MESSAGE_BYTES / 4] ____cacheline_aligned; | |
| 2046 | − | |
| 2047 | −int random_int_secret_init(void) | |
| 2048 | −{ | |
| 2049 | − get_random_bytes(random_int_secret, sizeof(random_int_secret)); | |
| 2050 | − return 0; | |
| 2051 | −} | |
| 2052 | − | |
| 2053 | −static DEFINE_PER_CPU(__u32 [MD5_DIGEST_WORDS], get_random_int_hash) | |
| 2054 | − __aligned(sizeof(unsigned long)); | |
| 2015 | +struct batched_entropy { | |
| 2016 | + union { | |
| 2017 | + u64 entropy_u64[CHACHA20_BLOCK_SIZE / sizeof(u64)]; | |
| 2018 | + u32 entropy_u32[CHACHA20_BLOCK_SIZE / sizeof(u32)]; | |
| 2019 | + }; | |
| 2020 | + unsigned int position; | |
| 2021 | +}; | |
| 2055 | 2022 | |
| 2056 | 2023 | /* |
| 2057 | − * Get a random word for internal kernel use only. Similar to urandom but | |
| 2058 | − * with the goal of minimal entropy pool depletion. As a result, the random | |
| 2059 | − * value is not cryptographically secure but for several uses the cost of | |
| 2060 | − * depleting entropy is too high | |
| 2024 | + * Get a random word for internal kernel use only. The quality of the random | |
| 2025 | + * number is either as good as RDRAND or as good as /dev/urandom, with the | |
| 2026 | + * goal of being quite fast and not depleting entropy. | |
| 2061 | 2027 | */ |
| 2062 | −unsigned int get_random_int(void) | |
| 2028 | +static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64); | |
| 2029 | +u64 get_random_u64(void) | |
| 2063 | 2030 | { |
| 2064 | − __u32 *hash; | |
| 2065 | − unsigned int ret; | |
| 2031 | + u64 ret; | |
| 2032 | + struct batched_entropy *batch; | |
| 2066 | 2033 | |
| 2067 | − if (arch_get_random_int(&ret)) | |
| 2034 | +#if BITS_PER_LONG == 64 | |
| 2035 | + if (arch_get_random_long((unsigned long *)&ret)) | |
| 2068 | 2036 | return ret; |
| 2037 | +#else | |
| 2038 | + if (arch_get_random_long((unsigned long *)&ret) && | |
| 2039 | + arch_get_random_long((unsigned long *)&ret + 1)) | |
| 2040 | + return ret; | |
| 2041 | +#endif | |
| 2069 | 2042 | |
| 2070 | − hash = get_cpu_var(get_random_int_hash); | |
| 2071 | − | |
| 2072 | − hash[0] += current->pid + jiffies + random_get_entropy(); | |
| 2073 | − md5_transform(hash, random_int_secret); | |
| 2074 | − ret = hash[0]; | |
| 2075 | − put_cpu_var(get_random_int_hash); | |
| 2076 | − | |
| 2043 | + batch = &get_cpu_var(batched_entropy_u64); | |
| 2044 | + if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) { | |
| 2045 | + extract_crng((u8 *)batch->entropy_u64); | |
| 2046 | + batch->position = 0; | |
| 2047 | + } | |
| 2048 | + ret = batch->entropy_u64[batch->position++]; | |
| 2049 | + put_cpu_var(batched_entropy_u64); | |
| 2077 | 2050 | return ret; |
| 2078 | 2051 | } |
| 2079 | −EXPORT_SYMBOL(get_random_int); | |
| 2052 | +EXPORT_SYMBOL(get_random_u64); | |
| 2080 | 2053 | |
| 2081 | −/* | |
| 2082 | − * Same as get_random_int(), but returns unsigned long. | |
| 2083 | − */ | |
| 2084 | −unsigned long get_random_long(void) | |
| 2054 | +static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32); | |
| 2055 | +u32 get_random_u32(void) | |
| 2085 | 2056 | { |
| 2086 | − __u32 *hash; | |
| 2087 | − unsigned long ret; | |
| 2057 | + u32 ret; | |
| 2058 | + struct batched_entropy *batch; | |
| 2088 | 2059 | |
| 2089 | − if (arch_get_random_long(&ret)) | |
| 2060 | + if (arch_get_random_int(&ret)) | |
| 2090 | 2061 | return ret; |
| 2091 | 2062 | |
| 2092 | − hash = get_cpu_var(get_random_int_hash); | |
| 2093 | − | |
| 2094 | − hash[0] += current->pid + jiffies + random_get_entropy(); | |
| 2095 | − md5_transform(hash, random_int_secret); | |
| 2096 | − ret = *(unsigned long *)hash; | |
| 2097 | − put_cpu_var(get_random_int_hash); | |
| 2098 | − | |
| 2063 | + batch = &get_cpu_var(batched_entropy_u32); | |
| 2064 | + if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) { | |
| 2065 | + extract_crng((u8 *)batch->entropy_u32); | |
| 2066 | + batch->position = 0; | |
| 2067 | + } | |
| 2068 | + ret = batch->entropy_u32[batch->position++]; | |
| 2069 | + put_cpu_var(batched_entropy_u32); | |
| 2099 | 2070 | return ret; |
| 2100 | 2071 | } |
| 2101 | −EXPORT_SYMBOL(get_random_long); | |
| 2072 | +EXPORT_SYMBOL(get_random_u32); | |
| 2102 | 2073 | |
| 2103 | 2074 | /** |
| 2104 | 2075 | * randomize_page - Generate a random, page aligned address |
| @@ -2532,4 +2532,5 @@ static int __init cpufreq_core_init(void) · 2531 unmodified lines | ||
| 2532 | 2532 | |
| 2533 | 2533 | return 0; |
| 2534 | 2534 | } |
| 2535 | +module_param(off, int, 0444); | |
| 2535 | 2536 | core_initcall(cpufreq_core_init); |
| @@ -377,6 +377,7 @@ static void intel_pstate_set_performance_limits(struct perf_limits *limits) · 376 unmodified lines | ||
| 377 | 377 | intel_pstate_init_limits(limits); |
| 378 | 378 | limits->min_perf_pct = 100; |
| 379 | 379 | limits->min_perf = int_ext_tofp(1); |
| 380 | + limits->min_sysfs_pct = 100; | |
| 380 | 381 | } |
| 381 | 382 | |
| 382 | 383 | static DEFINE_MUTEX(intel_pstate_driver_lock); |
| @@ -968,11 +969,20 @@ static int intel_pstate_resume(struct cpufreq_policy *policy) · 585 unmodified lines | ||
| 968 | 969 | } |
| 969 | 970 | |
| 970 | 971 | static void intel_pstate_update_policies(void) |
| 972 | + __releases(&intel_pstate_limits_lock) | |
| 973 | + __acquires(&intel_pstate_limits_lock) | |
| 971 | 974 | { |
| 975 | + struct perf_limits *saved_limits = limits; | |
| 972 | 976 | int cpu; |
| 973 | 977 | |
| 978 | + mutex_unlock(&intel_pstate_limits_lock); | |
| 979 | + | |
| 974 | 980 | for_each_possible_cpu(cpu) |
| 975 | 981 | cpufreq_update_policy(cpu); |
| 982 | + | |
| 983 | + mutex_lock(&intel_pstate_limits_lock); | |
| 984 | + | |
| 985 | + limits = saved_limits; | |
| 976 | 986 | } |
| 977 | 987 | |
| 978 | 988 | /************************** debugfs begin ************************/ |
| @@ -1180,10 +1190,10 @@ static ssize_t store_no_turbo(struct kobject *a, struct attribute *b, · 201 unmodified lines | ||
| 1180 | 1190 | |
| 1181 | 1191 | limits->no_turbo = clamp_t(int, input, 0, 1); |
| 1182 | 1192 | |
| 1183 | − mutex_unlock(&intel_pstate_limits_lock); | |
| 1184 | − | |
| 1185 | 1193 | intel_pstate_update_policies(); |
| 1186 | 1194 | |
| 1195 | + mutex_unlock(&intel_pstate_limits_lock); | |
| 1196 | + | |
| 1187 | 1197 | mutex_unlock(&intel_pstate_driver_lock); |
| 1188 | 1198 | |
| 1189 | 1199 | return count; |
| @@ -1217,10 +1227,10 @@ static ssize_t store_max_perf_pct(struct kobject *a, struct attribute *b, · 27 unmodified lines | ||
| 1217 | 1227 | limits->max_perf_pct); |
| 1218 | 1228 | limits->max_perf = div_ext_fp(limits->max_perf_pct, 100); |
| 1219 | 1229 | |
| 1220 | − mutex_unlock(&intel_pstate_limits_lock); | |
| 1221 | − | |
| 1222 | 1230 | intel_pstate_update_policies(); |
| 1223 | 1231 | |
| 1232 | + mutex_unlock(&intel_pstate_limits_lock); | |
| 1233 | + | |
| 1224 | 1234 | mutex_unlock(&intel_pstate_driver_lock); |
| 1225 | 1235 | |
| 1226 | 1236 | return count; |
| @@ -1254,10 +1264,10 @@ static ssize_t store_min_perf_pct(struct kobject *a, struct attribute *b, · 27 unmodified lines | ||
| 1254 | 1264 | limits->min_perf_pct); |
| 1255 | 1265 | limits->min_perf = div_ext_fp(limits->min_perf_pct, 100); |
| 1256 | 1266 | |
| 1257 | − mutex_unlock(&intel_pstate_limits_lock); | |
| 1258 | − | |
| 1259 | 1267 | intel_pstate_update_policies(); |
| 1260 | 1268 | |
| 1269 | + mutex_unlock(&intel_pstate_limits_lock); | |
| 1270 | + | |
| 1261 | 1271 | mutex_unlock(&intel_pstate_driver_lock); |
| 1262 | 1272 | |
| 1263 | 1273 | return count; |
| @@ -1874,13 +1884,11 @@ static int intel_pstate_prepare_request(struct cpudata *cpu, int pstate) · 610 unmodified lines | ||
| 1874 | 1884 | |
| 1875 | 1885 | intel_pstate_get_min_max(cpu, &min_perf, &max_perf); |
| 1876 | 1886 | pstate = clamp_t(int, pstate, min_perf, max_perf); |
| 1877 | − trace_cpu_frequency(pstate * cpu->pstate.scaling, cpu->cpu); | |
| 1878 | 1887 | return pstate; |
| 1879 | 1888 | } |
| 1880 | 1889 | |
| 1881 | 1890 | static void intel_pstate_update_pstate(struct cpudata *cpu, int pstate) |
| 1882 | 1891 | { |
| 1883 | − pstate = intel_pstate_prepare_request(cpu, pstate); | |
| 1884 | 1892 | if (pstate == cpu->pstate.current_pstate) |
| 1885 | 1893 | return; |
| 1886 | 1894 | |
| @@ -1900,6 +1908,8 @@ static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu) · 13 unmodified lines | ||
| 1900 | 1908 | |
| 1901 | 1909 | update_turbo_state(); |
| 1902 | 1910 | |
| 1911 | + target_pstate = intel_pstate_prepare_request(cpu, target_pstate); | |
| 1912 | + trace_cpu_frequency(target_pstate * cpu->pstate.scaling, cpu->cpu); | |
| 1903 | 1913 | intel_pstate_update_pstate(cpu, target_pstate); |
| 1904 | 1914 | |
| 1905 | 1915 | sample = &cpu->sample; |
| @@ -2132,16 +2142,11 @@ static int intel_pstate_set_policy(struct cpufreq_policy *policy) · 226 unmodified lines | ||
| 2132 | 2142 | mutex_lock(&intel_pstate_limits_lock); |
| 2133 | 2143 | |
| 2134 | 2144 | if (policy->policy == CPUFREQ_POLICY_PERFORMANCE) { |
| 2145 | + pr_debug("set performance\n"); | |
| 2135 | 2146 | if (!perf_limits) { |
| 2136 | 2147 | limits = &performance_limits; |
| 2137 | 2148 | perf_limits = limits; |
| 2138 | 2149 | } |
| 2139 | − if (policy->max >= policy->cpuinfo.max_freq && | |
| 2140 | − !limits->no_turbo) { | |
| 2141 | − pr_debug("set performance\n"); | |
| 2142 | − intel_pstate_set_performance_limits(perf_limits); | |
| 2143 | − goto out; | |
| 2144 | − } | |
| 2145 | 2150 | } else { |
| 2146 | 2151 | pr_debug("set powersave\n"); |
| 2147 | 2152 | if (!perf_limits) { |
| @@ -2152,7 +2157,7 @@ static int intel_pstate_set_policy(struct cpufreq_policy *policy) · 4 unmodified lines | ||
| 2152 | 2157 | } |
| 2153 | 2158 | |
| 2154 | 2159 | intel_pstate_update_perf_limits(policy, perf_limits); |
| 2155 | − out: | |
| 2160 | + | |
| 2156 | 2161 | if (cpu->policy == CPUFREQ_POLICY_PERFORMANCE) { |
| 2157 | 2162 | /* |
| 2158 | 2163 | * NOHZ_FULL CPUs need this as the governor callback may not |
| @@ -2198,9 +2203,9 @@ static int intel_pstate_verify_policy(struct cpufreq_policy *policy) · 39 unmodified lines | ||
| 2198 | 2203 | unsigned int max_freq, min_freq; |
| 2199 | 2204 | |
| 2200 | 2205 | max_freq = policy->cpuinfo.max_freq * |
| 2201 | − limits->max_sysfs_pct / 100; | |
| 2206 | + perf_limits->max_sysfs_pct / 100; | |
| 2202 | 2207 | min_freq = policy->cpuinfo.max_freq * |
| 2203 | − limits->min_sysfs_pct / 100; | |
| 2208 | + perf_limits->min_sysfs_pct / 100; | |
| 2204 | 2209 | cpufreq_verify_within_limits(policy, min_freq, max_freq); |
| 2205 | 2210 | } |
| 2206 | 2211 | |
| @@ -2243,13 +2248,8 @@ static int __intel_pstate_cpu_init(struct cpufreq_policy *policy) · 36 unmodified lines | ||
| 2243 | 2248 | |
| 2244 | 2249 | cpu = all_cpu_data[policy->cpu]; |
| 2245 | 2250 | |
| 2246 | − /* | |
| 2247 | − * We need sane value in the cpu->perf_limits, so inherit from global | |
| 2248 | − * perf_limits limits, which are seeded with values based on the | |
| 2249 | − * CONFIG_CPU_FREQ_DEFAULT_GOV_*, during boot up. | |
| 2250 | − */ | |
| 2251 | 2251 | if (per_cpu_limits) |
| 2252 | − memcpy(cpu->perf_limits, limits, sizeof(struct perf_limits)); | |
| 2252 | + intel_pstate_init_limits(cpu->perf_limits); | |
| 2253 | 2253 | |
| 2254 | 2254 | policy->min = cpu->pstate.min_pstate * cpu->pstate.scaling; |
| 2255 | 2255 | policy->max = cpu->pstate.turbo_pstate * cpu->pstate.scaling; |
| @@ -2301,7 +2301,6 @@ static struct cpufreq_driver intel_pstate = { · 45 unmodified lines | ||
| 2301 | 2301 | static int intel_cpufreq_verify_policy(struct cpufreq_policy *policy) |
| 2302 | 2302 | { |
| 2303 | 2303 | struct cpudata *cpu = all_cpu_data[policy->cpu]; |
| 2304 | − struct perf_limits *perf_limits = limits; | |
| 2305 | 2304 | |
| 2306 | 2305 | update_turbo_state(); |
| 2307 | 2306 | policy->cpuinfo.max_freq = limits->turbo_disabled ? |
| @@ -2309,15 +2308,6 @@ static int intel_cpufreq_verify_policy(struct cpufreq_policy *policy) · 1 unmodified line | ||
| 2309 | 2308 | |
| 2310 | 2309 | cpufreq_verify_within_cpu_limits(policy); |
| 2311 | 2310 | |
| 2312 | − if (per_cpu_limits) | |
| 2313 | − perf_limits = cpu->perf_limits; | |
| 2314 | − | |
| 2315 | − mutex_lock(&intel_pstate_limits_lock); | |
| 2316 | − | |
| 2317 | − intel_pstate_update_perf_limits(policy, perf_limits); | |
| 2318 | − | |
| 2319 | − mutex_unlock(&intel_pstate_limits_lock); | |
| 2320 | − | |
| 2321 | 2311 | return 0; |
| 2322 | 2312 | } |
| 2323 | 2313 | |
| @@ -2370,6 +2360,7 @@ static int intel_cpufreq_target(struct cpufreq_policy *policy, · 46 unmodified lines | ||
| 2370 | 2360 | wrmsrl_on_cpu(policy->cpu, MSR_IA32_PERF_CTL, |
| 2371 | 2361 | pstate_funcs.get_val(cpu, target_pstate)); |
| 2372 | 2362 | } |
| 2363 | + freqs.new = target_pstate * cpu->pstate.scaling; | |
| 2373 | 2364 | cpufreq_freq_transition_end(policy, &freqs, false); |
| 2374 | 2365 | |
| 2375 | 2366 | return 0; |
| @@ -2383,8 +2374,9 @@ static unsigned int intel_cpufreq_fast_switch(struct cpufreq_policy *policy, · 7 unmodified lines | ||
| 2383 | 2374 | |
| 2384 | 2375 | target_freq = intel_cpufreq_turbo_update(cpu, policy, target_freq); |
| 2385 | 2376 | target_pstate = DIV_ROUND_UP(target_freq, cpu->pstate.scaling); |
| 2377 | + target_pstate = intel_pstate_prepare_request(cpu, target_pstate); | |
| 2386 | 2378 | intel_pstate_update_pstate(cpu, target_pstate); |
| 2387 | − return target_freq; | |
| 2379 | + return target_pstate * cpu->pstate.scaling; | |
| 2388 | 2380 | } |
| 2389 | 2381 | |
| 2390 | 2382 | static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) |
| @@ -2437,8 +2429,11 @@ static int intel_pstate_register_driver(void) · 46 unmodified lines | ||
| 2437 | 2429 | |
| 2438 | 2430 | intel_pstate_init_limits(&powersave_limits); |
| 2439 | 2431 | intel_pstate_set_performance_limits(&performance_limits); |
| 2440 | − limits = IS_ENABLED(CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE) ? | |
| 2441 | − &performance_limits : &powersave_limits; | |
| 2432 | + if (IS_ENABLED(CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE) && | |
| 2433 | + intel_pstate_driver == &intel_pstate) | |
| 2434 | + limits = &performance_limits; | |
| 2435 | + else | |
| 2436 | + limits = &powersave_limits; | |
| 2442 | 2437 | |
| 2443 | 2438 | ret = cpufreq_register_driver(intel_pstate_driver); |
| 2444 | 2439 | if (ret) { |
| @@ -270,7 +270,7 @@ static void s5p_sg_copy_buf(void *buf, struct scatterlist *sg, · 269 unmodified lines | ||
| 270 | 270 | scatterwalk_done(&walk, out, 0); |
| 271 | 271 | } |
| 272 | 272 | |
| 273 | −static void s5p_aes_complete(struct s5p_aes_dev *dev, int err) | |
| 273 | +static void s5p_sg_done(struct s5p_aes_dev *dev) | |
| 274 | 274 | { |
| 275 | 275 | if (dev->sg_dst_cpy) { |
| 276 | 276 | dev_dbg(dev->dev, |
| @@ -281,8 +281,11 @@ static void s5p_aes_complete(struct s5p_aes_dev *dev, int err) · 4 unmodified lines | ||
| 281 | 281 | } |
| 282 | 282 | s5p_free_sg_cpy(dev, &dev->sg_src_cpy); |
| 283 | 283 | s5p_free_sg_cpy(dev, &dev->sg_dst_cpy); |
| 284 | +} | |
| 284 | 285 | |
| 285 | − /* holding a lock outside */ | |
| 286 | +/* Calls the completion. Cannot be called with dev->lock hold. */ | |
| 287 | +static void s5p_aes_complete(struct s5p_aes_dev *dev, int err) | |
| 288 | +{ | |
| 286 | 289 | dev->req->base.complete(&dev->req->base, err); |
| 287 | 290 | dev->busy = false; |
| 288 | 291 | } |
| @@ -368,51 +371,44 @@ exit: · 79 unmodified lines | ||
| 368 | 371 | } |
| 369 | 372 | |
| 370 | 373 | /* |
| 371 | − * Returns true if new transmitting (output) data is ready and its | |
| 372 | − * address+length have to be written to device (by calling | |
| 373 | − * s5p_set_dma_outdata()). False otherwise. | |
| 374 | + * Returns -ERRNO on error (mapping of new data failed). | |
| 375 | + * On success returns: | |
| 376 | + * - 0 if there is no more data, | |
| 377 | + * - 1 if new transmitting (output) data is ready and its address+length | |
| 378 | + * have to be written to device (by calling s5p_set_dma_outdata()). | |
| 374 | 379 | */ |
| 375 | −static bool s5p_aes_tx(struct s5p_aes_dev *dev) | |
| 380 | +static int s5p_aes_tx(struct s5p_aes_dev *dev) | |
| 376 | 381 | { |
| 377 | − int err = 0; | |
| 378 | − bool ret = false; | |
| 382 | + int ret = 0; | |
| 379 | 383 | |
| 380 | 384 | s5p_unset_outdata(dev); |
| 381 | 385 | |
| 382 | 386 | if (!sg_is_last(dev->sg_dst)) { |
| 383 | − err = s5p_set_outdata(dev, sg_next(dev->sg_dst)); | |
| 384 | − if (err) | |
| 385 | − s5p_aes_complete(dev, err); | |
| 386 | − else | |
| 387 | − ret = true; | |
| 388 | − } else { | |
| 389 | − s5p_aes_complete(dev, err); | |
| 390 | − | |
| 391 | − dev->busy = true; | |
| 392 | − tasklet_schedule(&dev->tasklet); | |
| 387 | + ret = s5p_set_outdata(dev, sg_next(dev->sg_dst)); | |
| 388 | + if (!ret) | |
| 389 | + ret = 1; | |
| 393 | 390 | } |
| 394 | 391 | |
| 395 | 392 | return ret; |
| 396 | 393 | } |
| 397 | 394 | |
| 398 | 395 | /* |
| 399 | − * Returns true if new receiving (input) data is ready and its | |
| 400 | − * address+length have to be written to device (by calling | |
| 401 | − * s5p_set_dma_indata()). False otherwise. | |
| 396 | + * Returns -ERRNO on error (mapping of new data failed). | |
| 397 | + * On success returns: | |
| 398 | + * - 0 if there is no more data, | |
| 399 | + * - 1 if new receiving (input) data is ready and its address+length | |
| 400 | + * have to be written to device (by calling s5p_set_dma_indata()). | |
| 402 | 401 | */ |
| 403 | −static bool s5p_aes_rx(struct s5p_aes_dev *dev) | |
| 402 | +static int s5p_aes_rx(struct s5p_aes_dev *dev/*, bool *set_dma*/) | |
| 404 | 403 | { |
| 405 | − int err; | |
| 406 | − bool ret = false; | |
| 404 | + int ret = 0; | |
| 407 | 405 | |
| 408 | 406 | s5p_unset_indata(dev); |
| 409 | 407 | |
| 410 | 408 | if (!sg_is_last(dev->sg_src)) { |
| 411 | − err = s5p_set_indata(dev, sg_next(dev->sg_src)); | |
| 412 | − if (err) | |
| 413 | − s5p_aes_complete(dev, err); | |
| 414 | − else | |
| 415 | − ret = true; | |
| 409 | + ret = s5p_set_indata(dev, sg_next(dev->sg_src)); | |
| 410 | + if (!ret) | |
| 411 | + ret = 1; | |
| 416 | 412 | } |
| 417 | 413 | |
| 418 | 414 | return ret; |
| @@ -422,33 +418,73 @@ static irqreturn_t s5p_aes_interrupt(int irq, void *dev_id) · 3 unmodified lines | ||
| 422 | 418 | { |
| 423 | 419 | struct platform_device *pdev = dev_id; |
| 424 | 420 | struct s5p_aes_dev *dev = platform_get_drvdata(pdev); |
| 425 | − bool set_dma_tx = false; | |
| 426 | − bool set_dma_rx = false; | |
| 421 | + int err_dma_tx = 0; | |
| 422 | + int err_dma_rx = 0; | |
| 423 | + bool tx_end = false; | |
| 427 | 424 | unsigned long flags; |
| 428 | 425 | uint32_t status; |
| 426 | + int err; | |
| 429 | 427 | |
| 430 | 428 | spin_lock_irqsave(&dev->lock, flags); |
| 431 | 429 | |
| 430 | + /* | |
| 431 | + * Handle rx or tx interrupt. If there is still data (scatterlist did not | |
| 432 | + * reach end), then map next scatterlist entry. | |
| 433 | + * In case of such mapping error, s5p_aes_complete() should be called. | |
| 434 | + * | |
| 435 | + * If there is no more data in tx scatter list, call s5p_aes_complete() | |
| 436 | + * and schedule new tasklet. | |
| 437 | + */ | |
| 432 | 438 | status = SSS_READ(dev, FCINTSTAT); |
| 433 | 439 | if (status & SSS_FCINTSTAT_BRDMAINT) |
| 434 | − set_dma_rx = s5p_aes_rx(dev); | |
| 435 | − if (status & SSS_FCINTSTAT_BTDMAINT) | |
| 436 | − set_dma_tx = s5p_aes_tx(dev); | |
| 440 | + err_dma_rx = s5p_aes_rx(dev); | |
| 441 | + | |
| 442 | + if (status & SSS_FCINTSTAT_BTDMAINT) { | |
| 443 | + if (sg_is_last(dev->sg_dst)) | |
| 444 | + tx_end = true; | |
| 445 | + err_dma_tx = s5p_aes_tx(dev); | |
| 446 | + } | |
| 437 | 447 | |
| 438 | 448 | SSS_WRITE(dev, FCINTPEND, status); |
| 439 | 449 | |
| 440 | − /* | |
| 441 | − * Writing length of DMA block (either receiving or transmitting) | |
| 442 | − * will start the operation immediately, so this should be done | |
| 443 | − * at the end (even after clearing pending interrupts to not miss the | |
| 444 | − * interrupt). | |
| 445 | − */ | |
| 446 | − if (set_dma_tx) | |
| 447 | − s5p_set_dma_outdata(dev, dev->sg_dst); | |
| 448 | − if (set_dma_rx) | |
| 449 | − s5p_set_dma_indata(dev, dev->sg_src); | |
| 450 | + if (err_dma_rx < 0) { | |
| 451 | + err = err_dma_rx; | |
| 452 | + goto error; | |
| 453 | + } | |
| 454 | + if (err_dma_tx < 0) { | |
| 455 | + err = err_dma_tx; | |
| 456 | + goto error; | |
| 457 | + } | |
| 458 | + | |
| 459 | + if (tx_end) { | |
| 460 | + s5p_sg_done(dev); | |
| 461 | + | |
| 462 | + spin_unlock_irqrestore(&dev->lock, flags); | |
| 463 | + | |
| 464 | + s5p_aes_complete(dev, 0); | |
| 465 | + dev->busy = true; | |
| 466 | + tasklet_schedule(&dev->tasklet); | |
| 467 | + } else { | |
| 468 | + /* | |
| 469 | + * Writing length of DMA block (either receiving or | |
| 470 | + * transmitting) will start the operation immediately, so this | |
| 471 | + * should be done at the end (even after clearing pending | |
| 472 | + * interrupts to not miss the interrupt). | |
| 473 | + */ | |
| 474 | + if (err_dma_tx == 1) | |
| 475 | + s5p_set_dma_outdata(dev, dev->sg_dst); | |
| 476 | + if (err_dma_rx == 1) | |
| 477 | + s5p_set_dma_indata(dev, dev->sg_src); | |
| 450 | 478 | |
| 479 | + spin_unlock_irqrestore(&dev->lock, flags); | |
| 480 | + } | |
| 481 | + | |
| 482 | + return IRQ_HANDLED; | |
| 483 | + | |
| 484 | +error: | |
| 485 | + s5p_sg_done(dev); | |
| 451 | 486 | spin_unlock_irqrestore(&dev->lock, flags); |
| 487 | + s5p_aes_complete(dev, err); | |
| 452 | 488 | |
| 453 | 489 | return IRQ_HANDLED; |
| 454 | 490 | } |
| @@ -597,8 +633,9 @@ outdata_error: · 142 unmodified lines | ||
| 597 | 633 | s5p_unset_indata(dev); |
| 598 | 634 | |
| 599 | 635 | indata_error: |
| 600 | − s5p_aes_complete(dev, err); | |
| 636 | + s5p_sg_done(dev); | |
| 601 | 637 | spin_unlock_irqrestore(&dev->lock, flags); |
| 638 | + s5p_aes_complete(dev, err); | |
| 602 | 639 | } |
| 603 | 640 | |
| 604 | 641 | static void s5p_tasklet_cb(unsigned long data) |
| @@ -805,8 +842,9 @@ static int s5p_aes_probe(struct platform_device *pdev) · 200 unmodified lines | ||
| 805 | 842 | dev_warn(dev, "feed control interrupt is not available.\n"); |
| 806 | 843 | goto err_irq; |
| 807 | 844 | } |
| 808 | − err = devm_request_irq(dev, pdata->irq_fc, s5p_aes_interrupt, | |
| 809 | − IRQF_SHARED, pdev->name, pdev); | |
| 845 | + err = devm_request_threaded_irq(dev, pdata->irq_fc, NULL, | |
| 846 | + s5p_aes_interrupt, IRQF_ONESHOT, | |
| 847 | + pdev->name, pdev); | |
| 810 | 848 | if (err < 0) { |
| 811 | 849 | dev_warn(dev, "feed control interrupt is not available.\n"); |
| 812 | 850 | goto err_irq; |
| @@ -82,7 +82,7 @@ void cryp_activity(struct cryp_device_data *device_data, · 81 unmodified lines | ||
| 82 | 82 | void cryp_flush_inoutfifo(struct cryp_device_data *device_data) |
| 83 | 83 | { |
| 84 | 84 | /* |
| 85 | − * We always need to disble the hardware before trying to flush the | |
| 85 | + * We always need to disable the hardware before trying to flush the | |
| 86 | 86 | * FIFO. This is something that isn't written in the design |
| 87 | 87 | * specification, but we have been informed by the hardware designers |
| 88 | 88 | * that this must be done. |
| @@ -65,6 +65,7 @@ static bool __init efi_virtmap_init(void) · 64 unmodified lines | ||
| 65 | 65 | bool systab_found; |
| 66 | 66 | |
| 67 | 67 | efi_mm.pgd = pgd_alloc(&efi_mm); |
| 68 | + mm_init_cpumask(&efi_mm); | |
| 68 | 69 | init_new_context(NULL, &efi_mm); |
| 69 | 70 | |
| 70 | 71 | systab_found = false; |
| @@ -45,6 +45,8 @@ enum efi_secureboot_mode efi_get_secureboot(efi_system_table_t *sys_table_arg) · 44 unmodified lines | ||
| 45 | 45 | size = sizeof(secboot); |
| 46 | 46 | status = get_efi_var(efi_SecureBoot_name, &efi_variable_guid, |
| 47 | 47 | NULL, &size, &secboot); |
| 48 | + if (status == EFI_NOT_FOUND) | |
| 49 | + return efi_secureboot_mode_disabled; | |
| 48 | 50 | if (status != EFI_SUCCESS) |
| 49 | 51 | goto out_efi_err; |
| 50 | 52 | |
| @@ -78,7 +80,5 @@ secure_boot_enabled: · 27 unmodified lines | ||
| 78 | 80 | |
| 79 | 81 | out_efi_err: |
| 80 | 82 | pr_efi_err(sys_table_arg, "Could not determine UEFI Secure Boot status.\n"); |
| 81 | − if (status == EFI_NOT_FOUND) | |
| 82 | − return efi_secureboot_mode_disabled; | |
| 83 | 83 | return efi_secureboot_mode_unknown; |
| 84 | 84 | } |
| @@ -2094,8 +2094,11 @@ int amdgpu_device_resume(struct drm_device *dev, bool resume, bool fbcon) · 2093 unmodified lines | ||
| 2094 | 2094 | } |
| 2095 | 2095 | |
| 2096 | 2096 | r = amdgpu_late_init(adev); |
| 2097 | − if (r) | |
| 2097 | + if (r) { | |
| 2098 | + if (fbcon) | |
| 2099 | + console_unlock(); | |
| 2098 | 2100 | return r; |
| 2101 | + } | |
| 2099 | 2102 | |
| 2100 | 2103 | /* pin cursors */ |
| 2101 | 2104 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { |
| @@ -59,9 +59,10 @@ · 58 unmodified lines | ||
| 59 | 59 | * - 3.7.0 - Add support for VCE clock list packet |
| 60 | 60 | * - 3.8.0 - Add support raster config init in the kernel |
| 61 | 61 | * - 3.9.0 - Add support for memory query info about VRAM and GTT. |
| 62 | + * - 3.10.0 - Add support for new fences ioctl, new gem ioctl flags | |
| 62 | 63 | */ |
| 63 | 64 | #define KMS_DRIVER_MAJOR 3 |
| 64 | −#define KMS_DRIVER_MINOR 9 | |
| 65 | +#define KMS_DRIVER_MINOR 10 | |
| 65 | 66 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 66 | 67 | |
| 67 | 68 | int amdgpu_vram_limit = 0; |
| @@ -202,6 +202,27 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, · 201 unmodified lines | ||
| 202 | 202 | bool kernel = false; |
| 203 | 203 | int r; |
| 204 | 204 | |
| 205 | + /* reject invalid gem flags */ | |
| 206 | + if (args->in.domain_flags & ~(AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | | |
| 207 | + AMDGPU_GEM_CREATE_NO_CPU_ACCESS | | |
| 208 | + AMDGPU_GEM_CREATE_CPU_GTT_USWC | | |
| 209 | + AMDGPU_GEM_CREATE_VRAM_CLEARED| | |
| 210 | + AMDGPU_GEM_CREATE_SHADOW | | |
| 211 | + AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS)) { | |
| 212 | + r = -EINVAL; | |
| 213 | + goto error_unlock; | |
| 214 | + } | |
| 215 | + /* reject invalid gem domains */ | |
| 216 | + if (args->in.domains & ~(AMDGPU_GEM_DOMAIN_CPU | | |
| 217 | + AMDGPU_GEM_DOMAIN_GTT | | |
| 218 | + AMDGPU_GEM_DOMAIN_VRAM | | |
| 219 | + AMDGPU_GEM_DOMAIN_GDS | | |
| 220 | + AMDGPU_GEM_DOMAIN_GWS | | |
| 221 | + AMDGPU_GEM_DOMAIN_OA)) { | |
| 222 | + r = -EINVAL; | |
| 223 | + goto error_unlock; | |
| 224 | + } | |
| 225 | + | |
| 205 | 226 | /* create a gem object to contain this object in */ |
| 206 | 227 | if (args->in.domains & (AMDGPU_GEM_DOMAIN_GDS | |
| 207 | 228 | AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA)) { |
| @@ -788,7 +788,7 @@ static int sdma_v3_0_start(struct amdgpu_device *adev) · 787 unmodified lines | ||
| 788 | 788 | } |
| 789 | 789 | } |
| 790 | 790 | |
| 791 | − /* disble sdma engine before programing it */ | |
| 791 | + /* disable sdma engine before programing it */ | |
| 792 | 792 | sdma_v3_0_ctx_switch_enable(adev, false); |
| 793 | 793 | sdma_v3_0_enable(adev, false); |
| 794 | 794 | |
| @@ -148,6 +148,9 @@ static const struct edid_quirk { · 147 unmodified lines | ||
| 148 | 148 | |
| 149 | 149 | /* Panel in Samsung NP700G7A-S01PL notebook reports 6bpc */ |
| 150 | 150 | { "SEC", 0xd033, EDID_QUIRK_FORCE_8BPC }, |
| 151 | + | |
| 152 | + /* Rotel RSX-1058 forwards sink's EDID but only does HDMI 1.1*/ | |
| 153 | + { "ETR", 13896, EDID_QUIRK_FORCE_8BPC }, | |
| 151 | 154 | }; |
| 152 | 155 | |
| 153 | 156 | /* |
| @@ -41,6 +41,54 @@ enum { · 40 unmodified lines | ||
| 41 | 41 | INTEL_GVT_PCI_BAR_MAX, |
| 42 | 42 | }; |
| 43 | 43 | |
| 44 | +/* bitmap for writable bits (RW or RW1C bits, but cannot co-exist in one | |
| 45 | + * byte) byte by byte in standard pci configuration space. (not the full | |
| 46 | + * 256 bytes.) | |
| 47 | + */ | |
| 48 | +static const u8 pci_cfg_space_rw_bmp[PCI_INTERRUPT_LINE + 4] = { | |
| 49 | + [PCI_COMMAND] = 0xff, 0x07, | |
| 50 | + [PCI_STATUS] = 0x00, 0xf9, /* the only one RW1C byte */ | |
| 51 | + [PCI_CACHE_LINE_SIZE] = 0xff, | |
| 52 | + [PCI_BASE_ADDRESS_0 ... PCI_CARDBUS_CIS - 1] = 0xff, | |
| 53 | + [PCI_ROM_ADDRESS] = 0x01, 0xf8, 0xff, 0xff, | |
| 54 | + [PCI_INTERRUPT_LINE] = 0xff, | |
| 55 | +}; | |
| 56 | + | |
| 57 | +/** | |
| 58 | + * vgpu_pci_cfg_mem_write - write virtual cfg space memory | |
| 59 | + * | |
| 60 | + * Use this function to write virtual cfg space memory. | |
| 61 | + * For standard cfg space, only RW bits can be changed, | |
| 62 | + * and we emulates the RW1C behavior of PCI_STATUS register. | |
| 63 | + */ | |
| 64 | +static void vgpu_pci_cfg_mem_write(struct intel_vgpu *vgpu, unsigned int off, | |
| 65 | + u8 *src, unsigned int bytes) | |
| 66 | +{ | |
| 67 | + u8 *cfg_base = vgpu_cfg_space(vgpu); | |
| 68 | + u8 mask, new, old; | |
| 69 | + int i = 0; | |
| 70 | + | |
| 71 | + for (; i < bytes && (off + i < sizeof(pci_cfg_space_rw_bmp)); i++) { | |
| 72 | + mask = pci_cfg_space_rw_bmp[off + i]; | |
| 73 | + old = cfg_base[off + i]; | |
| 74 | + new = src[i] & mask; | |
| 75 | + | |
| 76 | + /** | |
| 77 | + * The PCI_STATUS high byte has RW1C bits, here | |
| 78 | + * emulates clear by writing 1 for these bits. | |
| 79 | + * Writing a 0b to RW1C bits has no effect. | |
| 80 | + */ | |
| 81 | + if (off + i == PCI_STATUS + 1) | |
| 82 | + new = (~new & old) & mask; | |
| 83 | + | |
| 84 | + cfg_base[off + i] = (old & ~mask) | new; | |
| 85 | + } | |
| 86 | + | |
| 87 | + /* For other configuration space directly copy as it is. */ | |
| 88 | + if (i < bytes) | |
| 89 | + memcpy(cfg_base + off + i, src + i, bytes - i); | |
| 90 | +} | |
| 91 | + | |
| 44 | 92 | /** |
| 45 | 93 | * intel_vgpu_emulate_cfg_read - emulate vGPU configuration space read |
| 46 | 94 | * |
| @@ -123,7 +171,7 @@ static int emulate_pci_command_write(struct intel_vgpu *vgpu, · 76 unmodified lines | ||
| 123 | 171 | u8 changed = old ^ new; |
| 124 | 172 | int ret; |
| 125 | 173 | |
| 126 | − memcpy(vgpu_cfg_space(vgpu) + offset, p_data, bytes); | |
| 174 | + vgpu_pci_cfg_mem_write(vgpu, offset, p_data, bytes); | |
| 127 | 175 | if (!(changed & PCI_COMMAND_MEMORY)) |
| 128 | 176 | return 0; |
| 129 | 177 | |
| @@ -237,6 +285,9 @@ int intel_vgpu_emulate_cfg_write(struct intel_vgpu *vgpu, unsigned int offset, · 107 unmodified lines | ||
| 237 | 285 | { |
| 238 | 286 | int ret; |
| 239 | 287 | |
| 288 | + if (vgpu->failsafe) | |
| 289 | + return 0; | |
| 290 | + | |
| 240 | 291 | if (WARN_ON(bytes > 4)) |
| 241 | 292 | return -EINVAL; |
| 242 | 293 | |
| @@ -274,10 +325,10 @@ int intel_vgpu_emulate_cfg_write(struct intel_vgpu *vgpu, unsigned int offset, · 31 unmodified lines | ||
| 274 | 325 | if (ret) |
| 275 | 326 | return ret; |
| 276 | 327 | |
| 277 | − memcpy(vgpu_cfg_space(vgpu) + offset, p_data, bytes); | |
| 328 | + vgpu_pci_cfg_mem_write(vgpu, offset, p_data, bytes); | |
| 278 | 329 | break; |
| 279 | 330 | default: |
| 280 | − memcpy(vgpu_cfg_space(vgpu) + offset, p_data, bytes); | |
| 331 | + vgpu_pci_cfg_mem_write(vgpu, offset, p_data, bytes); | |
| 281 | 332 | break; |
| 282 | 333 | } |
| 283 | 334 | return 0; |
| @@ -668,7 +668,7 @@ static inline void print_opcode(u32 cmd, int ring_id) · 667 unmodified lines | ||
| 668 | 668 | if (d_info == NULL) |
| 669 | 669 | return; |
| 670 | 670 | |
| 671 | − gvt_err("opcode=0x%x %s sub_ops:", | |
| 671 | + gvt_dbg_cmd("opcode=0x%x %s sub_ops:", | |
| 672 | 672 | cmd >> (32 - d_info->op_len), d_info->name); |
| 673 | 673 | |
| 674 | 674 | for (i = 0; i < d_info->nr_sub_op; i++) |
| @@ -693,23 +693,23 @@ static void parser_exec_state_dump(struct parser_exec_state *s) · 18 unmodified lines | ||
| 693 | 693 | int cnt = 0; |
| 694 | 694 | int i; |
| 695 | 695 | |
| 696 | − gvt_err(" vgpu%d RING%d: ring_start(%08lx) ring_end(%08lx)" | |
| 696 | + gvt_dbg_cmd(" vgpu%d RING%d: ring_start(%08lx) ring_end(%08lx)" | |
| 697 | 697 | " ring_head(%08lx) ring_tail(%08lx)\n", s->vgpu->id, |
| 698 | 698 | s->ring_id, s->ring_start, s->ring_start + s->ring_size, |
| 699 | 699 | s->ring_head, s->ring_tail); |
| 700 | 700 | |
| 701 | − gvt_err(" %s %s ip_gma(%08lx) ", | |
| 701 | + gvt_dbg_cmd(" %s %s ip_gma(%08lx) ", | |
| 702 | 702 | s->buf_type == RING_BUFFER_INSTRUCTION ? |
| 703 | 703 | "RING_BUFFER" : "BATCH_BUFFER", |
| 704 | 704 | s->buf_addr_type == GTT_BUFFER ? |
| 705 | 705 | "GTT" : "PPGTT", s->ip_gma); |
| 706 | 706 | |
| 707 | 707 | if (s->ip_va == NULL) { |
| 708 | − gvt_err(" ip_va(NULL)"); | |
| 708 | + gvt_dbg_cmd(" ip_va(NULL)"); | |
| 709 | 709 | return; |
| 710 | 710 | } |
| 711 | 711 | |
| 712 | − gvt_err(" ip_va=%p: %08x %08x %08x %08x\n", | |
| 712 | + gvt_dbg_cmd(" ip_va=%p: %08x %08x %08x %08x\n", | |
| 713 | 713 | s->ip_va, cmd_val(s, 0), cmd_val(s, 1), |
| 714 | 714 | cmd_val(s, 2), cmd_val(s, 3)); |
| 715 | 715 | |
| @@ -83,44 +83,80 @@ static int pipe_is_enabled(struct intel_vgpu *vgpu, int pipe) · 82 unmodified lines | ||
| 83 | 83 | return 0; |
| 84 | 84 | } |
| 85 | 85 | |
| 86 | +static unsigned char virtual_dp_monitor_edid[GVT_EDID_NUM][EDID_SIZE] = { | |
| 87 | + { | |
| 88 | +/* EDID with 1024x768 as its resolution */ | |
| 89 | + /*Header*/ | |
| 90 | + 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, | |
| 91 | + /* Vendor & Product Identification */ | |
| 92 | + 0x22, 0xf0, 0x54, 0x29, 0x00, 0x00, 0x00, 0x00, 0x04, 0x17, | |
| 93 | + /* Version & Revision */ | |
| 94 | + 0x01, 0x04, | |
| 95 | + /* Basic Display Parameters & Features */ | |
| 96 | + 0xa5, 0x34, 0x20, 0x78, 0x23, | |
| 97 | + /* Color Characteristics */ | |
| 98 | + 0xfc, 0x81, 0xa4, 0x55, 0x4d, 0x9d, 0x25, 0x12, 0x50, 0x54, | |
| 99 | + /* Established Timings: maximum resolution is 1024x768 */ | |
| 100 | + 0x21, 0x08, 0x00, | |
| 101 | + /* Standard Timings. All invalid */ | |
| 102 | + 0x00, 0xc0, 0x00, 0xc0, 0x00, 0x40, 0x00, 0x80, 0x00, 0x00, | |
| 103 | + 0x00, 0x40, 0x00, 0x00, 0x00, 0x01, | |
| 104 | + /* 18 Byte Data Blocks 1: invalid */ | |
| 105 | + 0x00, 0x00, 0x80, 0xa0, 0x70, 0xb0, | |
| 106 | + 0x23, 0x40, 0x30, 0x20, 0x36, 0x00, 0x06, 0x44, 0x21, 0x00, 0x00, 0x1a, | |
| 107 | + /* 18 Byte Data Blocks 2: invalid */ | |
| 108 | + 0x00, 0x00, 0x00, 0xfd, 0x00, 0x18, 0x3c, 0x18, 0x50, 0x11, 0x00, 0x0a, | |
| 109 | + 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, | |
| 110 | + /* 18 Byte Data Blocks 3: invalid */ | |
| 111 | + 0x00, 0x00, 0x00, 0xfc, 0x00, 0x48, | |
| 112 | + 0x50, 0x20, 0x5a, 0x52, 0x32, 0x34, 0x34, 0x30, 0x77, 0x0a, 0x20, 0x20, | |
| 113 | + /* 18 Byte Data Blocks 4: invalid */ | |
| 114 | + 0x00, 0x00, 0x00, 0xff, 0x00, 0x43, 0x4e, 0x34, 0x33, 0x30, 0x34, 0x30, | |
| 115 | + 0x44, 0x58, 0x51, 0x0a, 0x20, 0x20, | |
| 116 | + /* Extension Block Count */ | |
| 117 | + 0x00, | |
| 118 | + /* Checksum */ | |
| 119 | + 0xef, | |
| 120 | + }, | |
| 121 | + { | |
| 86 | 122 | /* EDID with 1920x1200 as its resolution */ |
| 87 | −static unsigned char virtual_dp_monitor_edid[] = { | |
| 88 | − /*Header*/ | |
| 89 | − 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, | |
| 90 | − /* Vendor & Product Identification */ | |
| 91 | − 0x22, 0xf0, 0x54, 0x29, 0x00, 0x00, 0x00, 0x00, 0x04, 0x17, | |
| 92 | − /* Version & Revision */ | |
| 93 | − 0x01, 0x04, | |
| 94 | − /* Basic Display Parameters & Features */ | |
| 95 | − 0xa5, 0x34, 0x20, 0x78, 0x23, | |
| 96 | − /* Color Characteristics */ | |
| 97 | − 0xfc, 0x81, 0xa4, 0x55, 0x4d, 0x9d, 0x25, 0x12, 0x50, 0x54, | |
| 98 | − /* Established Timings: maximum resolution is 1024x768 */ | |
| 99 | − 0x21, 0x08, 0x00, | |
| 100 | − /* | |
| 101 | − * Standard Timings. | |
| 102 | − * below new resolutions can be supported: | |
| 103 | − * 1920x1080, 1280x720, 1280x960, 1280x1024, | |
| 104 | − * 1440x900, 1600x1200, 1680x1050 | |
| 105 | − */ | |
| 106 | − 0xd1, 0xc0, 0x81, 0xc0, 0x81, 0x40, 0x81, 0x80, 0x95, 0x00, | |
| 107 | − 0xa9, 0x40, 0xb3, 0x00, 0x01, 0x01, | |
| 108 | − /* 18 Byte Data Blocks 1: max resolution is 1920x1200 */ | |
| 109 | − 0x28, 0x3c, 0x80, 0xa0, 0x70, 0xb0, | |
| 110 | − 0x23, 0x40, 0x30, 0x20, 0x36, 0x00, 0x06, 0x44, 0x21, 0x00, 0x00, 0x1a, | |
| 111 | − /* 18 Byte Data Blocks 2: invalid */ | |
| 112 | − 0x00, 0x00, 0x00, 0xfd, 0x00, 0x18, 0x3c, 0x18, 0x50, 0x11, 0x00, 0x0a, | |
| 113 | − 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, | |
| 114 | − /* 18 Byte Data Blocks 3: invalid */ | |
| 115 | − 0x00, 0x00, 0x00, 0xfc, 0x00, 0x48, | |
| 116 | − 0x50, 0x20, 0x5a, 0x52, 0x32, 0x34, 0x34, 0x30, 0x77, 0x0a, 0x20, 0x20, | |
| 117 | − /* 18 Byte Data Blocks 4: invalid */ | |
| 118 | − 0x00, 0x00, 0x00, 0xff, 0x00, 0x43, 0x4e, 0x34, 0x33, 0x30, 0x34, 0x30, | |
| 119 | − 0x44, 0x58, 0x51, 0x0a, 0x20, 0x20, | |
| 120 | − /* Extension Block Count */ | |
| 121 | − 0x00, | |
| 122 | − /* Checksum */ | |
| 123 | − 0x45, | |
| 123 | + /*Header*/ | |
| 124 | + 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, | |
| 125 | + /* Vendor & Product Identification */ | |
| 126 | + 0x22, 0xf0, 0x54, 0x29, 0x00, 0x00, 0x00, 0x00, 0x04, 0x17, | |
| 127 | + /* Version & Revision */ | |
| 128 | + 0x01, 0x04, | |
| 129 | + /* Basic Display Parameters & Features */ | |
| 130 | + 0xa5, 0x34, 0x20, 0x78, 0x23, | |
| 131 | + /* Color Characteristics */ | |
| 132 | + 0xfc, 0x81, 0xa4, 0x55, 0x4d, 0x9d, 0x25, 0x12, 0x50, 0x54, | |
| 133 | + /* Established Timings: maximum resolution is 1024x768 */ | |
| 134 | + 0x21, 0x08, 0x00, | |
| 135 | + /* | |
| 136 | + * Standard Timings. | |
| 137 | + * below new resolutions can be supported: | |
| 138 | + * 1920x1080, 1280x720, 1280x960, 1280x1024, | |
| 139 | + * 1440x900, 1600x1200, 1680x1050 | |
| 140 | + */ | |
| 141 | + 0xd1, 0xc0, 0x81, 0xc0, 0x81, 0x40, 0x81, 0x80, 0x95, 0x00, | |
| 142 | + 0xa9, 0x40, 0xb3, 0x00, 0x01, 0x01, | |
| 143 | + /* 18 Byte Data Blocks 1: max resolution is 1920x1200 */ | |
| 144 | + 0x28, 0x3c, 0x80, 0xa0, 0x70, 0xb0, | |
| 145 | + 0x23, 0x40, 0x30, 0x20, 0x36, 0x00, 0x06, 0x44, 0x21, 0x00, 0x00, 0x1a, | |
| 146 | + /* 18 Byte Data Blocks 2: invalid */ | |
| 147 | + 0x00, 0x00, 0x00, 0xfd, 0x00, 0x18, 0x3c, 0x18, 0x50, 0x11, 0x00, 0x0a, | |
| 148 | + 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, | |
| 149 | + /* 18 Byte Data Blocks 3: invalid */ | |
| 150 | + 0x00, 0x00, 0x00, 0xfc, 0x00, 0x48, | |
| 151 | + 0x50, 0x20, 0x5a, 0x52, 0x32, 0x34, 0x34, 0x30, 0x77, 0x0a, 0x20, 0x20, | |
| 152 | + /* 18 Byte Data Blocks 4: invalid */ | |
| 153 | + 0x00, 0x00, 0x00, 0xff, 0x00, 0x43, 0x4e, 0x34, 0x33, 0x30, 0x34, 0x30, | |
| 154 | + 0x44, 0x58, 0x51, 0x0a, 0x20, 0x20, | |
| 155 | + /* Extension Block Count */ | |
| 156 | + 0x00, | |
| 157 | + /* Checksum */ | |
| 158 | + 0x45, | |
| 159 | + }, | |
| 124 | 160 | }; |
| 125 | 161 | |
| 126 | 162 | #define DPCD_HEADER_SIZE 0xb |
| @@ -140,14 +176,20 @@ static void emulate_monitor_status_change(struct intel_vgpu *vgpu) · 13 unmodified lines | ||
| 140 | 176 | vgpu_vreg(vgpu, SDEISR) &= ~(SDE_PORTA_HOTPLUG_SPT | |
| 141 | 177 | SDE_PORTE_HOTPLUG_SPT); |
| 142 | 178 | |
| 143 | − if (intel_vgpu_has_monitor_on_port(vgpu, PORT_B)) | |
| 179 | + if (intel_vgpu_has_monitor_on_port(vgpu, PORT_B)) { | |
| 144 | 180 | vgpu_vreg(vgpu, SDEISR) |= SDE_PORTB_HOTPLUG_CPT; |
| 181 | + vgpu_vreg(vgpu, SFUSE_STRAP) |= SFUSE_STRAP_DDIB_DETECTED; | |
| 182 | + } | |
| 145 | 183 | |
| 146 | − if (intel_vgpu_has_monitor_on_port(vgpu, PORT_C)) | |
| 184 | + if (intel_vgpu_has_monitor_on_port(vgpu, PORT_C)) { | |
| 147 | 185 | vgpu_vreg(vgpu, SDEISR) |= SDE_PORTC_HOTPLUG_CPT; |
| 186 | + vgpu_vreg(vgpu, SFUSE_STRAP) |= SFUSE_STRAP_DDIC_DETECTED; | |
| 187 | + } | |
| 148 | 188 | |
| 149 | − if (intel_vgpu_has_monitor_on_port(vgpu, PORT_D)) | |
| 189 | + if (intel_vgpu_has_monitor_on_port(vgpu, PORT_D)) { | |
| 150 | 190 | vgpu_vreg(vgpu, SDEISR) |= SDE_PORTD_HOTPLUG_CPT; |
| 191 | + vgpu_vreg(vgpu, SFUSE_STRAP) |= SFUSE_STRAP_DDID_DETECTED; | |
| 192 | + } | |
| 151 | 193 | |
| 152 | 194 | if (IS_SKYLAKE(dev_priv) && |
| 153 | 195 | intel_vgpu_has_monitor_on_port(vgpu, PORT_E)) { |
| @@ -160,6 +202,8 @@ static void emulate_monitor_status_change(struct intel_vgpu *vgpu) · 6 unmodified lines | ||
| 160 | 202 | GEN8_PORT_DP_A_HOTPLUG; |
| 161 | 203 | else |
| 162 | 204 | vgpu_vreg(vgpu, SDEISR) |= SDE_PORTA_HOTPLUG_SPT; |
| 205 | + | |
| 206 | + vgpu_vreg(vgpu, DDI_BUF_CTL(PORT_A)) |= DDI_INIT_DISPLAY_DETECTED; | |
| 163 | 207 | } |
| 164 | 208 | } |
| 165 | 209 | |
| @@ -175,10 +219,13 @@ static void clean_virtual_dp_monitor(struct intel_vgpu *vgpu, int port_num) · 9 unmodified lines | ||
| 175 | 219 | } |
| 176 | 220 | |
| 177 | 221 | static int setup_virtual_dp_monitor(struct intel_vgpu *vgpu, int port_num, |
| 178 | − int type) | |
| 222 | + int type, unsigned int resolution) | |
| 179 | 223 | { |
| 180 | 224 | struct intel_vgpu_port *port = intel_vgpu_port(vgpu, port_num); |
| 181 | 225 | |
| 226 | + if (WARN_ON(resolution >= GVT_EDID_NUM)) | |
| 227 | + return -EINVAL; | |
| 228 | + | |
| 182 | 229 | port->edid = kzalloc(sizeof(*(port->edid)), GFP_KERNEL); |
| 183 | 230 | if (!port->edid) |
| 184 | 231 | return -ENOMEM; |
| @@ -189,7 +236,7 @@ static int setup_virtual_dp_monitor(struct intel_vgpu *vgpu, int port_num, · 4 unmodified lines | ||
| 189 | 236 | return -ENOMEM; |
| 190 | 237 | } |
| 191 | 238 | |
| 192 | − memcpy(port->edid->edid_block, virtual_dp_monitor_edid, | |
| 239 | + memcpy(port->edid->edid_block, virtual_dp_monitor_edid[resolution], | |
| 193 | 240 | EDID_SIZE); |
| 194 | 241 | port->edid->data_valid = true; |
| 195 | 242 | |
| @@ -322,16 +369,18 @@ void intel_vgpu_clean_display(struct intel_vgpu *vgpu) · 126 unmodified lines | ||
| 322 | 369 | * Zero on success, negative error code if failed. |
| 323 | 370 | * |
| 324 | 371 | */ |
| 325 | −int intel_vgpu_init_display(struct intel_vgpu *vgpu) | |
| 372 | +int intel_vgpu_init_display(struct intel_vgpu *vgpu, u64 resolution) | |
| 326 | 373 | { |
| 327 | 374 | struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv; |
| 328 | 375 | |
| 329 | 376 | intel_vgpu_init_i2c_edid(vgpu); |
| 330 | 377 | |
| 331 | 378 | if (IS_SKYLAKE(dev_priv)) |
| 332 | − return setup_virtual_dp_monitor(vgpu, PORT_D, GVT_DP_D); | |
| 379 | + return setup_virtual_dp_monitor(vgpu, PORT_D, GVT_DP_D, | |
| 380 | + resolution); | |
| 333 | 381 | else |
| 334 | − return setup_virtual_dp_monitor(vgpu, PORT_B, GVT_DP_B); | |
| 382 | + return setup_virtual_dp_monitor(vgpu, PORT_B, GVT_DP_B, | |
| 383 | + resolution); | |
| 335 | 384 | } |
| 336 | 385 | |
| 337 | 386 | /** |
| @@ -154,10 +154,28 @@ struct intel_vgpu_port { · 153 unmodified lines | ||
| 154 | 154 | int type; |
| 155 | 155 | }; |
| 156 | 156 | |
| 157 | +enum intel_vgpu_edid { | |
| 158 | + GVT_EDID_1024_768, | |
| 159 | + GVT_EDID_1920_1200, | |
| 160 | + GVT_EDID_NUM, | |
| 161 | +}; | |
| 162 | + | |
| 163 | +static inline char *vgpu_edid_str(enum intel_vgpu_edid id) | |
| 164 | +{ | |
| 165 | + switch (id) { | |
| 166 | + case GVT_EDID_1024_768: | |
| 167 | + return "1024x768"; | |
| 168 | + case GVT_EDID_1920_1200: | |
| 169 | + return "1920x1200"; | |
| 170 | + default: | |
| 171 | + return ""; | |
| 172 | + } | |
| 173 | +} | |
| 174 | + | |
| 157 | 175 | void intel_gvt_emulate_vblank(struct intel_gvt *gvt); |
| 158 | 176 | void intel_gvt_check_vblank_emulation(struct intel_gvt *gvt); |
| 159 | 177 | |
| 160 | −int intel_vgpu_init_display(struct intel_vgpu *vgpu); | |
| 178 | +int intel_vgpu_init_display(struct intel_vgpu *vgpu, u64 resolution); | |
| 161 | 179 | void intel_vgpu_reset_display(struct intel_vgpu *vgpu); |
| 162 | 180 | void intel_vgpu_clean_display(struct intel_vgpu *vgpu); |
| 163 | 181 | |
| @@ -80,7 +80,7 @@ static int expose_firmware_sysfs(struct intel_gvt *gvt) · 79 unmodified lines | ||
| 80 | 80 | int ret; |
| 81 | 81 | |
| 82 | 82 | size = sizeof(*h) + info->mmio_size + info->cfg_space_size - 1; |
| 83 | − firmware = vmalloc(size); | |
| 83 | + firmware = vzalloc(size); | |
| 84 | 84 | if (!firmware) |
| 85 | 85 | return -ENOMEM; |
| 86 | 86 | |
| @@ -1825,11 +1825,8 @@ static int emulate_gtt_mmio_write(struct intel_vgpu *vgpu, unsigned int off, · 1824 unmodified lines | ||
| 1825 | 1825 | gma = g_gtt_index << GTT_PAGE_SHIFT; |
| 1826 | 1826 | |
| 1827 | 1827 | /* the VM may configure the whole GM space when ballooning is used */ |
| 1828 | − if (WARN_ONCE(!vgpu_gmadr_is_valid(vgpu, gma), | |
| 1829 | − "vgpu%d: found oob ggtt write, offset %x\n", | |
| 1830 | − vgpu->id, off)) { | |
| 1828 | + if (!vgpu_gmadr_is_valid(vgpu, gma)) | |
| 1831 | 1829 | return 0; |
| 1832 | − } | |
| 1833 | 1830 | |
| 1834 | 1831 | ggtt_get_guest_entry(ggtt_mm, &e, g_gtt_index); |
| 1835 | 1832 | |
| @@ -2015,6 +2012,22 @@ int intel_vgpu_init_gtt(struct intel_vgpu *vgpu) · 179 unmodified lines | ||
| 2015 | 2012 | return create_scratch_page_tree(vgpu); |
| 2016 | 2013 | } |
| 2017 | 2014 | |
| 2015 | +static void intel_vgpu_free_mm(struct intel_vgpu *vgpu, int type) | |
| 2016 | +{ | |
| 2017 | + struct list_head *pos, *n; | |
| 2018 | + struct intel_vgpu_mm *mm; | |
| 2019 | + | |
| 2020 | + list_for_each_safe(pos, n, &vgpu->gtt.mm_list_head) { | |
| 2021 | + mm = container_of(pos, struct intel_vgpu_mm, list); | |
| 2022 | + if (mm->type == type) { | |
| 2023 | + vgpu->gvt->gtt.mm_free_page_table(mm); | |
| 2024 | + list_del(&mm->list); | |
| 2025 | + list_del(&mm->lru_list); | |
| 2026 | + kfree(mm); | |
| 2027 | + } | |
| 2028 | + } | |
| 2029 | +} | |
| 2030 | + | |
| 2018 | 2031 | /** |
| 2019 | 2032 | * intel_vgpu_clean_gtt - clean up per-vGPU graphics memory virulization |
| 2020 | 2033 | * @vgpu: a vGPU |
| @@ -2027,19 +2040,11 @@ int intel_vgpu_init_gtt(struct intel_vgpu *vgpu) · 6 unmodified lines | ||
| 2027 | 2040 | */ |
| 2028 | 2041 | void intel_vgpu_clean_gtt(struct intel_vgpu *vgpu) |
| 2029 | 2042 | { |
| 2030 | − struct list_head *pos, *n; | |
| 2031 | − struct intel_vgpu_mm *mm; | |
| 2032 | − | |
| 2033 | 2043 | ppgtt_free_all_shadow_page(vgpu); |
| 2034 | 2044 | release_scratch_page_tree(vgpu); |
| 2035 | 2045 | |
| 2036 | − list_for_each_safe(pos, n, &vgpu->gtt.mm_list_head) { | |
| 2037 | − mm = container_of(pos, struct intel_vgpu_mm, list); | |
| 2038 | − vgpu->gvt->gtt.mm_free_page_table(mm); | |
| 2039 | − list_del(&mm->list); | |
| 2040 | − list_del(&mm->lru_list); | |
| 2041 | − kfree(mm); | |
| 2042 | − } | |
| 2046 | + intel_vgpu_free_mm(vgpu, INTEL_GVT_MM_PPGTT); | |
| 2047 | + intel_vgpu_free_mm(vgpu, INTEL_GVT_MM_GGTT); | |
| 2043 | 2048 | } |
| 2044 | 2049 | |
| 2045 | 2050 | static void clean_spt_oos(struct intel_gvt *gvt) |
| @@ -2322,6 +2327,13 @@ void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr) · 276 unmodified lines | ||
| 2322 | 2327 | int i; |
| 2323 | 2328 | |
| 2324 | 2329 | ppgtt_free_all_shadow_page(vgpu); |
| 2330 | + | |
| 2331 | + /* Shadow pages are only created when there is no page | |
| 2332 | + * table tracking data, so remove page tracking data after | |
| 2333 | + * removing the shadow pages. | |
| 2334 | + */ | |
| 2335 | + intel_vgpu_free_mm(vgpu, INTEL_GVT_MM_PPGTT); | |
| 2336 | + | |
| 2325 | 2337 | if (!dmlr) |
| 2326 | 2338 | return; |
| 2327 | 2339 | |
| @@ -143,6 +143,8 @@ struct intel_vgpu { · 142 unmodified lines | ||
| 143 | 143 | int id; |
| 144 | 144 | unsigned long handle; /* vGPU handle used by hypervisor MPT modules */ |
| 145 | 145 | bool active; |
| 146 | + bool pv_notified; | |
| 147 | + bool failsafe; | |
| 146 | 148 | bool resetting; |
| 147 | 149 | void *sched_data; |
| 148 | 150 | |
| @@ -203,18 +205,18 @@ struct intel_gvt_firmware { · 54 unmodified lines | ||
| 203 | 205 | }; |
| 204 | 206 | |
| 205 | 207 | struct intel_gvt_opregion { |
| 206 | − void __iomem *opregion_va; | |
| 208 | + void *opregion_va; | |
| 207 | 209 | u32 opregion_pa; |
| 208 | 210 | }; |
| 209 | 211 | |
| 210 | 212 | #define NR_MAX_INTEL_VGPU_TYPES 20 |
| 211 | 213 | struct intel_vgpu_type { |
| 212 | 214 | char name[16]; |
| 213 | − unsigned int max_instance; | |
| 214 | 215 | unsigned int avail_instance; |
| 215 | 216 | unsigned int low_gm_size; |
| 216 | 217 | unsigned int high_gm_size; |
| 217 | 218 | unsigned int fence; |
| 219 | + enum intel_vgpu_edid resolution; | |
| 218 | 220 | }; |
| 219 | 221 | |
| 220 | 222 | struct intel_gvt { |
| @@ -317,6 +319,7 @@ struct intel_vgpu_creation_params { · 96 unmodified lines | ||
| 317 | 319 | __u64 low_gm_sz; /* in MB */ |
| 318 | 320 | __u64 high_gm_sz; /* in MB */ |
| 319 | 321 | __u64 fence_sz; |
| 322 | + __u64 resolution; | |
| 320 | 323 | __s32 primary; |
| 321 | 324 | __u64 vgpu_id; |
| 322 | 325 | }; |
| @@ -449,6 +452,11 @@ struct intel_gvt_ops { · 126 unmodified lines | ||
| 449 | 452 | }; |
| 450 | 453 | |
| 451 | 454 | |
| 455 | +enum { | |
| 456 | + GVT_FAILSAFE_UNSUPPORTED_GUEST, | |
| 457 | + GVT_FAILSAFE_INSUFFICIENT_RESOURCE, | |
| 458 | +}; | |
| 459 | + | |
| 452 | 460 | #include "mpt.h" |
| 453 | 461 | |
| 454 | 462 | #endif |
| @@ -121,6 +121,7 @@ static int new_mmio_info(struct intel_gvt *gvt, · 120 unmodified lines | ||
| 121 | 121 | info->size = size; |
| 122 | 122 | info->length = (i + 4) < end ? 4 : (end - i); |
| 123 | 123 | info->addr_mask = addr_mask; |
| 124 | + info->ro_mask = ro_mask; | |
| 124 | 125 | info->device = device; |
| 125 | 126 | info->read = read ? read : intel_vgpu_default_mmio_read; |
| 126 | 127 | info->write = write ? write : intel_vgpu_default_mmio_write; |
| @@ -150,15 +151,44 @@ static int render_mmio_to_ring_id(struct intel_gvt *gvt, unsigned int reg) · 23 unmodified lines | ||
| 150 | 151 | #define fence_num_to_offset(num) \ |
| 151 | 152 | (num * 8 + i915_mmio_reg_offset(FENCE_REG_GEN6_LO(0))) |
| 152 | 153 | |
| 154 | + | |
| 155 | +static void enter_failsafe_mode(struct intel_vgpu *vgpu, int reason) | |
| 156 | +{ | |
| 157 | + switch (reason) { | |
| 158 | + case GVT_FAILSAFE_UNSUPPORTED_GUEST: | |
| 159 | + pr_err("Detected your guest driver doesn't support GVT-g.\n"); | |
| 160 | + break; | |
| 161 | + case GVT_FAILSAFE_INSUFFICIENT_RESOURCE: | |
| 162 | + pr_err("Graphics resource is not enough for the guest\n"); | |
| 163 | + default: | |
| 164 | + break; | |
| 165 | + } | |
| 166 | + pr_err("Now vgpu %d will enter failsafe mode.\n", vgpu->id); | |
| 167 | + vgpu->failsafe = true; | |
| 168 | +} | |
| 169 | + | |
| 153 | 170 | static int sanitize_fence_mmio_access(struct intel_vgpu *vgpu, |
| 154 | 171 | unsigned int fence_num, void *p_data, unsigned int bytes) |
| 155 | 172 | { |
| 156 | 173 | if (fence_num >= vgpu_fence_sz(vgpu)) { |
| 157 | − gvt_err("vgpu%d: found oob fence register access\n", | |
| 158 | − vgpu->id); | |
| 159 | − gvt_err("vgpu%d: total fence num %d access fence num %d\n", | |
| 160 | − vgpu->id, vgpu_fence_sz(vgpu), fence_num); | |
| 174 | + | |
| 175 | + /* When guest access oob fence regs without access | |
| 176 | + * pv_info first, we treat guest not supporting GVT, | |
| 177 | + * and we will let vgpu enter failsafe mode. | |
| 178 | + */ | |
| 179 | + if (!vgpu->pv_notified) | |
| 180 | + enter_failsafe_mode(vgpu, | |
| 181 | + GVT_FAILSAFE_UNSUPPORTED_GUEST); | |
| 182 | + | |
| 183 | + if (!vgpu->mmio.disable_warn_untrack) { | |
| 184 | + gvt_err("vgpu%d: found oob fence register access\n", | |
| 185 | + vgpu->id); | |
| 186 | + gvt_err("vgpu%d: total fence %d, access fence %d\n", | |
| 187 | + vgpu->id, vgpu_fence_sz(vgpu), | |
| 188 | + fence_num); | |
| 189 | + } | |
| 161 | 190 | memset(p_data, 0, bytes); |
| 191 | + return -EINVAL; | |
| 162 | 192 | } |
| 163 | 193 | return 0; |
| 164 | 194 | } |
| @@ -369,6 +399,74 @@ static int pipeconf_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 204 unmodified lines | ||
| 369 | 399 | return 0; |
| 370 | 400 | } |
| 371 | 401 | |
| 402 | +/* ascendingly sorted */ | |
| 403 | +static i915_reg_t force_nonpriv_white_list[] = { | |
| 404 | + GEN9_CS_DEBUG_MODE1, //_MMIO(0x20ec) | |
| 405 | + GEN9_CTX_PREEMPT_REG,//_MMIO(0x2248) | |
| 406 | + GEN8_CS_CHICKEN1,//_MMIO(0x2580) | |
| 407 | + _MMIO(0x2690), | |
| 408 | + _MMIO(0x2694), | |
| 409 | + _MMIO(0x2698), | |
| 410 | + _MMIO(0x4de0), | |
| 411 | + _MMIO(0x4de4), | |
| 412 | + _MMIO(0x4dfc), | |
| 413 | + GEN7_COMMON_SLICE_CHICKEN1,//_MMIO(0x7010) | |
| 414 | + _MMIO(0x7014), | |
| 415 | + HDC_CHICKEN0,//_MMIO(0x7300) | |
| 416 | + GEN8_HDC_CHICKEN1,//_MMIO(0x7304) | |
| 417 | + _MMIO(0x7700), | |
| 418 | + _MMIO(0x7704), | |
| 419 | + _MMIO(0x7708), | |
| 420 | + _MMIO(0x770c), | |
| 421 | + _MMIO(0xb110), | |
| 422 | + GEN8_L3SQCREG4,//_MMIO(0xb118) | |
| 423 | + _MMIO(0xe100), | |
| 424 | + _MMIO(0xe18c), | |
| 425 | + _MMIO(0xe48c), | |
| 426 | + _MMIO(0xe5f4), | |
| 427 | +}; | |
| 428 | + | |
| 429 | +/* a simple bsearch */ | |
| 430 | +static inline bool in_whitelist(unsigned int reg) | |
| 431 | +{ | |
| 432 | + int left = 0, right = ARRAY_SIZE(force_nonpriv_white_list); | |
| 433 | + i915_reg_t *array = force_nonpriv_white_list; | |
| 434 | + | |
| 435 | + while (left < right) { | |
| 436 | + int mid = (left + right)/2; | |
| 437 | + | |
| 438 | + if (reg > array[mid].reg) | |
| 439 | + left = mid + 1; | |
| 440 | + else if (reg < array[mid].reg) | |
| 441 | + right = mid; | |
| 442 | + else | |
| 443 | + return true; | |
| 444 | + } | |
| 445 | + return false; | |
| 446 | +} | |
| 447 | + | |
| 448 | +static int force_nonpriv_write(struct intel_vgpu *vgpu, | |
| 449 | + unsigned int offset, void *p_data, unsigned int bytes) | |
| 450 | +{ | |
| 451 | + u32 reg_nonpriv = *(u32 *)p_data; | |
| 452 | + int ret = -EINVAL; | |
| 453 | + | |
| 454 | + if ((bytes != 4) || ((offset & (bytes - 1)) != 0)) { | |
| 455 | + gvt_err("vgpu(%d) Invalid FORCE_NONPRIV offset %x(%dB)\n", | |
| 456 | + vgpu->id, offset, bytes); | |
| 457 | + return ret; | |
| 458 | + } | |
| 459 | + | |
| 460 | + if (in_whitelist(reg_nonpriv)) { | |
| 461 | + ret = intel_vgpu_default_mmio_write(vgpu, offset, p_data, | |
| 462 | + bytes); | |
| 463 | + } else { | |
| 464 | + gvt_err("vgpu(%d) Invalid FORCE_NONPRIV write %x\n", | |
| 465 | + vgpu->id, reg_nonpriv); | |
| 466 | + } | |
| 467 | + return ret; | |
| 468 | +} | |
| 469 | + | |
| 372 | 470 | static int ddi_buf_ctl_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, |
| 373 | 471 | void *p_data, unsigned int bytes) |
| 374 | 472 | { |
| @@ -1001,6 +1099,7 @@ static int pvinfo_mmio_read(struct intel_vgpu *vgpu, unsigned int offset, · 626 unmodified lines | ||
| 1001 | 1099 | if (invalid_read) |
| 1002 | 1100 | gvt_err("invalid pvinfo read: [%x:%x] = %x\n", |
| 1003 | 1101 | offset, bytes, *(u32 *)p_data); |
| 1102 | + vgpu->pv_notified = true; | |
| 1004 | 1103 | return 0; |
| 1005 | 1104 | } |
| 1006 | 1105 | |
| @@ -1039,7 +1138,7 @@ static int send_display_ready_uevent(struct intel_vgpu *vgpu, int ready) · 32 unmodified lines | ||
| 1039 | 1138 | char vmid_str[20]; |
| 1040 | 1139 | char display_ready_str[20]; |
| 1041 | 1140 | |
| 1042 | − snprintf(display_ready_str, 20, "GVT_DISPLAY_READY=%d\n", ready); | |
| 1141 | + snprintf(display_ready_str, 20, "GVT_DISPLAY_READY=%d", ready); | |
| 1043 | 1142 | env[0] = display_ready_str; |
| 1044 | 1143 | |
| 1045 | 1144 | snprintf(vmid_str, 20, "VMID=%d", vgpu->id); |
| @@ -1078,6 +1177,9 @@ static int pvinfo_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 32 unmodified lines | ||
| 1078 | 1177 | case _vgtif_reg(execlist_context_descriptor_lo): |
| 1079 | 1178 | case _vgtif_reg(execlist_context_descriptor_hi): |
| 1080 | 1179 | break; |
| 1180 | + case _vgtif_reg(rsv5[0])..._vgtif_reg(rsv5[3]): | |
| 1181 | + enter_failsafe_mode(vgpu, GVT_FAILSAFE_INSUFFICIENT_RESOURCE); | |
| 1182 | + break; | |
| 1081 | 1183 | default: |
| 1082 | 1184 | gvt_err("invalid pvinfo write offset %x bytes %x data %x\n", |
| 1083 | 1185 | offset, bytes, data); |
| @@ -1203,26 +1305,37 @@ static int mailbox_write(struct intel_vgpu *vgpu, unsigned int offset, · 119 unmodified lines | ||
| 1203 | 1305 | u32 *data0 = &vgpu_vreg(vgpu, GEN6_PCODE_DATA); |
| 1204 | 1306 | |
| 1205 | 1307 | switch (cmd) { |
| 1206 | − case 0x6: | |
| 1207 | − /** | |
| 1208 | − * "Read memory latency" command on gen9. | |
| 1209 | − * Below memory latency values are read | |
| 1210 | − * from skylake platform. | |
| 1211 | − */ | |
| 1212 | − if (!*data0) | |
| 1213 | − *data0 = 0x1e1a1100; | |
| 1214 | − else | |
| 1215 | − *data0 = 0x61514b3d; | |
| 1308 | + case GEN9_PCODE_READ_MEM_LATENCY: | |
| 1309 | + if (IS_SKYLAKE(vgpu->gvt->dev_priv)) { | |
| 1310 | + /** | |
| 1311 | + * "Read memory latency" command on gen9. | |
| 1312 | + * Below memory latency values are read | |
| 1313 | + * from skylake platform. | |
| 1314 | + */ | |
| 1315 | + if (!*data0) | |
| 1316 | + *data0 = 0x1e1a1100; | |
| 1317 | + else | |
| 1318 | + *data0 = 0x61514b3d; | |
| 1319 | + } | |
| 1320 | + break; | |
| 1321 | + case SKL_PCODE_CDCLK_CONTROL: | |
| 1322 | + if (IS_SKYLAKE(vgpu->gvt->dev_priv)) | |
| 1323 | + *data0 = SKL_CDCLK_READY_FOR_CHANGE; | |
| 1216 | 1324 | break; |
| 1217 | − case 0x5: | |
| 1325 | + case GEN6_PCODE_READ_RC6VIDS: | |
| 1218 | 1326 | *data0 |= 0x1; |
| 1219 | 1327 | break; |
| 1220 | 1328 | } |
| 1221 | 1329 | |
| 1222 | 1330 | gvt_dbg_core("VM(%d) write %x to mailbox, return data0 %x\n", |
| 1223 | 1331 | vgpu->id, value, *data0); |
| 1224 | − | |
| 1225 | − value &= ~(1 << 31); | |
| 1332 | + /** | |
| 1333 | + * PCODE_READY clear means ready for pcode read/write, | |
| 1334 | + * PCODE_ERROR_MASK clear means no error happened. In GVT-g we | |
| 1335 | + * always emulate as pcode read/write success and ready for access | |
| 1336 | + * anytime, since we don't touch real physical registers here. | |
| 1337 | + */ | |
| 1338 | + value &= ~(GEN6_PCODE_READY | GEN6_PCODE_ERROR_MASK); | |
| 1226 | 1339 | return intel_vgpu_default_mmio_write(vgpu, offset, &value, bytes); |
| 1227 | 1340 | } |
| 1228 | 1341 | |
| @@ -1318,6 +1431,17 @@ static int ring_mode_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 89 unmodified lines | ||
| 1318 | 1431 | bool enable_execlist; |
| 1319 | 1432 | |
| 1320 | 1433 | write_vreg(vgpu, offset, p_data, bytes); |
| 1434 | + | |
| 1435 | + /* when PPGTT mode enabled, we will check if guest has called | |
| 1436 | + * pvinfo, if not, we will treat this guest as non-gvtg-aware | |
| 1437 | + * guest, and stop emulating its cfg space, mmio, gtt, etc. | |
| 1438 | + */ | |
| 1439 | + if (((data & _MASKED_BIT_ENABLE(GFX_PPGTT_ENABLE)) || | |
| 1440 | + (data & _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE))) | |
| 1441 | + && !vgpu->pv_notified) { | |
| 1442 | + enter_failsafe_mode(vgpu, GVT_FAILSAFE_UNSUPPORTED_GUEST); | |
| 1443 | + return 0; | |
| 1444 | + } | |
| 1321 | 1445 | if ((data & _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE)) |
| 1322 | 1446 | || (data & _MASKED_BIT_DISABLE(GFX_RUN_LIST_ENABLE))) { |
| 1323 | 1447 | enable_execlist = !!(data & GFX_RUN_LIST_ENABLE); |
| @@ -1400,6 +1524,9 @@ static int ring_reset_ctl_write(struct intel_vgpu *vgpu, · 76 unmodified lines | ||
| 1400 | 1524 | #define MMIO_GM(reg, d, r, w) \ |
| 1401 | 1525 | MMIO_F(reg, 4, F_GMADR, 0xFFFFF000, 0, d, r, w) |
| 1402 | 1526 | |
| 1527 | +#define MMIO_GM_RDR(reg, d, r, w) \ | |
| 1528 | + MMIO_F(reg, 4, F_GMADR | F_CMD_ACCESS, 0xFFFFF000, 0, d, r, w) | |
| 1529 | + | |
| 1403 | 1530 | #define MMIO_RO(reg, d, f, rm, r, w) \ |
| 1404 | 1531 | MMIO_F(reg, 4, F_RO | f, 0, rm, d, r, w) |
| 1405 | 1532 | |
| @@ -1419,6 +1546,9 @@ static int ring_reset_ctl_write(struct intel_vgpu *vgpu, · 13 unmodified lines | ||
| 1419 | 1546 | #define MMIO_RING_GM(prefix, d, r, w) \ |
| 1420 | 1547 | MMIO_RING_F(prefix, 4, F_GMADR, 0xFFFF0000, 0, d, r, w) |
| 1421 | 1548 | |
| 1549 | +#define MMIO_RING_GM_RDR(prefix, d, r, w) \ | |
| 1550 | + MMIO_RING_F(prefix, 4, F_GMADR | F_CMD_ACCESS, 0xFFFF0000, 0, d, r, w) | |
| 1551 | + | |
| 1422 | 1552 | #define MMIO_RING_RO(prefix, d, f, rm, r, w) \ |
| 1423 | 1553 | MMIO_RING_F(prefix, 4, F_RO | f, 0, rm, d, r, w) |
| 1424 | 1554 | |
| @@ -1427,73 +1557,81 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 2 unmodified lines | ||
| 1427 | 1557 | struct drm_i915_private *dev_priv = gvt->dev_priv; |
| 1428 | 1558 | int ret; |
| 1429 | 1559 | |
| 1430 | − MMIO_RING_DFH(RING_IMR, D_ALL, 0, NULL, intel_vgpu_reg_imr_handler); | |
| 1560 | + MMIO_RING_DFH(RING_IMR, D_ALL, F_CMD_ACCESS, NULL, | |
| 1561 | + intel_vgpu_reg_imr_handler); | |
| 1431 | 1562 | |
| 1432 | 1563 | MMIO_DFH(SDEIMR, D_ALL, 0, NULL, intel_vgpu_reg_imr_handler); |
| 1433 | 1564 | MMIO_DFH(SDEIER, D_ALL, 0, NULL, intel_vgpu_reg_ier_handler); |
| 1434 | 1565 | MMIO_DFH(SDEIIR, D_ALL, 0, NULL, intel_vgpu_reg_iir_handler); |
| 1435 | 1566 | MMIO_D(SDEISR, D_ALL); |
| 1436 | 1567 | |
| 1437 | − MMIO_RING_D(RING_HWSTAM, D_ALL); | |
| 1568 | + MMIO_RING_DFH(RING_HWSTAM, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1438 | 1569 | |
| 1439 | − MMIO_GM(RENDER_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1440 | − MMIO_GM(BSD_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1441 | − MMIO_GM(BLT_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1442 | − MMIO_GM(VEBOX_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1570 | + MMIO_GM_RDR(RENDER_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1571 | + MMIO_GM_RDR(BSD_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1572 | + MMIO_GM_RDR(BLT_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1573 | + MMIO_GM_RDR(VEBOX_HWS_PGA_GEN7, D_ALL, NULL, NULL); | |
| 1443 | 1574 | |
| 1444 | 1575 | #define RING_REG(base) (base + 0x28) |
| 1445 | − MMIO_RING_D(RING_REG, D_ALL); | |
| 1576 | + MMIO_RING_DFH(RING_REG, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1446 | 1577 | #undef RING_REG |
| 1447 | 1578 | |
| 1448 | 1579 | #define RING_REG(base) (base + 0x134) |
| 1449 | − MMIO_RING_D(RING_REG, D_ALL); | |
| 1580 | + MMIO_RING_DFH(RING_REG, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1450 | 1581 | #undef RING_REG |
| 1451 | 1582 | |
| 1452 | − MMIO_GM(0x2148, D_ALL, NULL, NULL); | |
| 1453 | − MMIO_GM(CCID, D_ALL, NULL, NULL); | |
| 1454 | − MMIO_GM(0x12198, D_ALL, NULL, NULL); | |
| 1583 | + MMIO_GM_RDR(0x2148, D_ALL, NULL, NULL); | |
| 1584 | + MMIO_GM_RDR(CCID, D_ALL, NULL, NULL); | |
| 1585 | + MMIO_GM_RDR(0x12198, D_ALL, NULL, NULL); | |
| 1455 | 1586 | MMIO_D(GEN7_CXT_SIZE, D_ALL); |
| 1456 | 1587 | |
| 1457 | − MMIO_RING_D(RING_TAIL, D_ALL); | |
| 1458 | − MMIO_RING_D(RING_HEAD, D_ALL); | |
| 1459 | − MMIO_RING_D(RING_CTL, D_ALL); | |
| 1460 | − MMIO_RING_D(RING_ACTHD, D_ALL); | |
| 1461 | − MMIO_RING_GM(RING_START, D_ALL, NULL, NULL); | |
| 1588 | + MMIO_RING_DFH(RING_TAIL, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1589 | + MMIO_RING_DFH(RING_HEAD, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1590 | + MMIO_RING_DFH(RING_CTL, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1591 | + MMIO_RING_DFH(RING_ACTHD, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1592 | + MMIO_RING_GM_RDR(RING_START, D_ALL, NULL, NULL); | |
| 1462 | 1593 | |
| 1463 | 1594 | /* RING MODE */ |
| 1464 | 1595 | #define RING_REG(base) (base + 0x29c) |
| 1465 | − MMIO_RING_DFH(RING_REG, D_ALL, F_MODE_MASK, NULL, ring_mode_mmio_write); | |
| 1596 | + MMIO_RING_DFH(RING_REG, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, | |
| 1597 | + ring_mode_mmio_write); | |
| 1466 | 1598 | #undef RING_REG |
| 1467 | 1599 | |
| 1468 | − MMIO_RING_DFH(RING_MI_MODE, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1469 | − MMIO_RING_DFH(RING_INSTPM, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1600 | + MMIO_RING_DFH(RING_MI_MODE, D_ALL, F_MODE_MASK | F_CMD_ACCESS, | |
| 1601 | + NULL, NULL); | |
| 1602 | + MMIO_RING_DFH(RING_INSTPM, D_ALL, F_MODE_MASK | F_CMD_ACCESS, | |
| 1603 | + NULL, NULL); | |
| 1470 | 1604 | MMIO_RING_DFH(RING_TIMESTAMP, D_ALL, F_CMD_ACCESS, |
| 1471 | 1605 | ring_timestamp_mmio_read, NULL); |
| 1472 | 1606 | MMIO_RING_DFH(RING_TIMESTAMP_UDW, D_ALL, F_CMD_ACCESS, |
| 1473 | 1607 | ring_timestamp_mmio_read, NULL); |
| 1474 | 1608 | |
| 1475 | − MMIO_DFH(GEN7_GT_MODE, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1476 | − MMIO_DFH(CACHE_MODE_0_GEN7, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1609 | + MMIO_DFH(GEN7_GT_MODE, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1610 | + MMIO_DFH(CACHE_MODE_0_GEN7, D_ALL, F_MODE_MASK | F_CMD_ACCESS, | |
| 1611 | + NULL, NULL); | |
| 1477 | 1612 | MMIO_DFH(CACHE_MODE_1, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 1478 | − | |
| 1479 | − MMIO_DFH(0x20dc, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1480 | − MMIO_DFH(_3D_CHICKEN3, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1481 | − MMIO_DFH(0x2088, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1482 | − MMIO_DFH(0x20e4, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1483 | − MMIO_DFH(0x2470, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1484 | − MMIO_D(GAM_ECOCHK, D_ALL); | |
| 1485 | − MMIO_DFH(GEN7_COMMON_SLICE_CHICKEN1, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1613 | + MMIO_DFH(CACHE_MODE_0, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1614 | + MMIO_DFH(0x2124, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1615 | + | |
| 1616 | + MMIO_DFH(0x20dc, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1617 | + MMIO_DFH(_3D_CHICKEN3, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1618 | + MMIO_DFH(0x2088, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1619 | + MMIO_DFH(0x20e4, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1620 | + MMIO_DFH(0x2470, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1621 | + MMIO_DFH(GAM_ECOCHK, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1622 | + MMIO_DFH(GEN7_COMMON_SLICE_CHICKEN1, D_ALL, F_MODE_MASK | F_CMD_ACCESS, | |
| 1623 | + NULL, NULL); | |
| 1486 | 1624 | MMIO_DFH(COMMON_SLICE_CHICKEN2, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 1487 | − MMIO_D(0x9030, D_ALL); | |
| 1488 | − MMIO_D(0x20a0, D_ALL); | |
| 1489 | − MMIO_D(0x2420, D_ALL); | |
| 1490 | − MMIO_D(0x2430, D_ALL); | |
| 1491 | − MMIO_D(0x2434, D_ALL); | |
| 1492 | − MMIO_D(0x2438, D_ALL); | |
| 1493 | − MMIO_D(0x243c, D_ALL); | |
| 1494 | − MMIO_DFH(0x7018, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1625 | + MMIO_DFH(0x9030, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1626 | + MMIO_DFH(0x20a0, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1627 | + MMIO_DFH(0x2420, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1628 | + MMIO_DFH(0x2430, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1629 | + MMIO_DFH(0x2434, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1630 | + MMIO_DFH(0x2438, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1631 | + MMIO_DFH(0x243c, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 1632 | + MMIO_DFH(0x7018, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1495 | 1633 | MMIO_DFH(HALF_SLICE_CHICKEN3, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 1496 | − MMIO_DFH(0xe100, D_ALL, F_MODE_MASK, NULL, NULL); | |
| 1634 | + MMIO_DFH(GEN7_HALF_SLICE_CHICKEN1, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 1497 | 1635 | |
| 1498 | 1636 | /* display */ |
| 1499 | 1637 | MMIO_F(0x60220, 0x20, 0, 0, 0, D_ALL, NULL, NULL); |
| @@ -2022,8 +2160,8 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 522 unmodified lines | ||
| 2022 | 2160 | MMIO_D(FORCEWAKE_ACK, D_ALL); |
| 2023 | 2161 | MMIO_D(GEN6_GT_CORE_STATUS, D_ALL); |
| 2024 | 2162 | MMIO_D(GEN6_GT_THREAD_STATUS_REG, D_ALL); |
| 2025 | − MMIO_D(GTFIFODBG, D_ALL); | |
| 2026 | − MMIO_D(GTFIFOCTL, D_ALL); | |
| 2163 | + MMIO_DFH(GTFIFODBG, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2164 | + MMIO_DFH(GTFIFOCTL, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2027 | 2165 | MMIO_DH(FORCEWAKE_MT, D_PRE_SKL, NULL, mul_force_wake_write); |
| 2028 | 2166 | MMIO_DH(FORCEWAKE_ACK_HSW, D_HSW | D_BDW, NULL, NULL); |
| 2029 | 2167 | MMIO_D(ECOBUS, D_ALL); |
| @@ -2080,7 +2218,7 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 50 unmodified lines | ||
| 2080 | 2218 | |
| 2081 | 2219 | MMIO_F(0x4f000, 0x90, 0, 0, 0, D_ALL, NULL, NULL); |
| 2082 | 2220 | |
| 2083 | − MMIO_D(GEN6_PCODE_MAILBOX, D_PRE_SKL); | |
| 2221 | + MMIO_D(GEN6_PCODE_MAILBOX, D_PRE_BDW); | |
| 2084 | 2222 | MMIO_D(GEN6_PCODE_DATA, D_ALL); |
| 2085 | 2223 | MMIO_D(0x13812c, D_ALL); |
| 2086 | 2224 | MMIO_DH(GEN7_ERR_INT, D_ALL, NULL, NULL); |
| @@ -2159,36 +2297,35 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 72 unmodified lines | ||
| 2159 | 2297 | MMIO_D(0x1a054, D_ALL); |
| 2160 | 2298 | |
| 2161 | 2299 | MMIO_D(0x44070, D_ALL); |
| 2162 | − | |
| 2163 | − MMIO_D(0x215c, D_HSW_PLUS); | |
| 2300 | + MMIO_DFH(0x215c, D_HSW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2164 | 2301 | MMIO_DFH(0x2178, D_ALL, F_CMD_ACCESS, NULL, NULL); |
| 2165 | 2302 | MMIO_DFH(0x217c, D_ALL, F_CMD_ACCESS, NULL, NULL); |
| 2166 | 2303 | MMIO_DFH(0x12178, D_ALL, F_CMD_ACCESS, NULL, NULL); |
| 2167 | 2304 | MMIO_DFH(0x1217c, D_ALL, F_CMD_ACCESS, NULL, NULL); |
| 2168 | 2305 | |
| 2169 | − MMIO_F(0x2290, 8, 0, 0, 0, D_HSW_PLUS, NULL, NULL); | |
| 2170 | − MMIO_D(GEN7_OACONTROL, D_HSW); | |
| 2306 | + MMIO_F(0x2290, 8, F_CMD_ACCESS, 0, 0, D_HSW_PLUS, NULL, NULL); | |
| 2307 | + MMIO_DFH(GEN7_OACONTROL, D_HSW, F_CMD_ACCESS, NULL, NULL); | |
| 2171 | 2308 | MMIO_D(0x2b00, D_BDW_PLUS); |
| 2172 | 2309 | MMIO_D(0x2360, D_BDW_PLUS); |
| 2173 | − MMIO_F(0x5200, 32, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2174 | − MMIO_F(0x5240, 32, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2175 | − MMIO_F(0x5280, 16, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2310 | + MMIO_F(0x5200, 32, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2311 | + MMIO_F(0x5240, 32, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2312 | + MMIO_F(0x5280, 16, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2176 | 2313 | |
| 2177 | 2314 | MMIO_DFH(0x1c17c, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); |
| 2178 | 2315 | MMIO_DFH(0x1c178, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); |
| 2179 | − MMIO_D(BCS_SWCTRL, D_ALL); | |
| 2180 | − | |
| 2181 | − MMIO_F(HS_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2182 | − MMIO_F(DS_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2183 | − MMIO_F(IA_VERTICES_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2184 | − MMIO_F(IA_PRIMITIVES_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2185 | − MMIO_F(VS_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2186 | − MMIO_F(GS_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2187 | − MMIO_F(GS_PRIMITIVES_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2188 | − MMIO_F(CL_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2189 | − MMIO_F(CL_PRIMITIVES_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2190 | − MMIO_F(PS_INVOCATION_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2191 | − MMIO_F(PS_DEPTH_COUNT, 8, 0, 0, 0, D_ALL, NULL, NULL); | |
| 2316 | + MMIO_DFH(BCS_SWCTRL, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2317 | + | |
| 2318 | + MMIO_F(HS_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2319 | + MMIO_F(DS_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2320 | + MMIO_F(IA_VERTICES_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2321 | + MMIO_F(IA_PRIMITIVES_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2322 | + MMIO_F(VS_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2323 | + MMIO_F(GS_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2324 | + MMIO_F(GS_PRIMITIVES_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2325 | + MMIO_F(CL_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2326 | + MMIO_F(CL_PRIMITIVES_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2327 | + MMIO_F(PS_INVOCATION_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2328 | + MMIO_F(PS_DEPTH_COUNT, 8, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL); | |
| 2192 | 2329 | MMIO_DH(0x4260, D_BDW_PLUS, NULL, gvt_reg_tlb_control_handler); |
| 2193 | 2330 | MMIO_DH(0x4264, D_BDW_PLUS, NULL, gvt_reg_tlb_control_handler); |
| 2194 | 2331 | MMIO_DH(0x4268, D_BDW_PLUS, NULL, gvt_reg_tlb_control_handler); |
| @@ -2196,6 +2333,17 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 1 unmodified line | ||
| 2196 | 2333 | MMIO_DH(0x4270, D_BDW_PLUS, NULL, gvt_reg_tlb_control_handler); |
| 2197 | 2334 | MMIO_DFH(0x4094, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); |
| 2198 | 2335 | |
| 2336 | + MMIO_DFH(ARB_MODE, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 2337 | + MMIO_RING_GM_RDR(RING_BBADDR, D_ALL, NULL, NULL); | |
| 2338 | + MMIO_DFH(0x2220, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2339 | + MMIO_DFH(0x12220, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2340 | + MMIO_DFH(0x22220, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2341 | + MMIO_RING_DFH(RING_SYNC_1, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2342 | + MMIO_RING_DFH(RING_SYNC_0, D_ALL, F_CMD_ACCESS, NULL, NULL); | |
| 2343 | + MMIO_DFH(0x22178, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2344 | + MMIO_DFH(0x1a178, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2345 | + MMIO_DFH(0x1a17c, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2346 | + MMIO_DFH(0x2217c, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2199 | 2347 | return 0; |
| 2200 | 2348 | } |
| 2201 | 2349 | |
| @@ -2204,7 +2352,7 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 2 unmodified lines | ||
| 2204 | 2352 | struct drm_i915_private *dev_priv = gvt->dev_priv; |
| 2205 | 2353 | int ret; |
| 2206 | 2354 | |
| 2207 | − MMIO_DH(RING_IMR(GEN8_BSD2_RING_BASE), D_BDW_PLUS, NULL, | |
| 2355 | + MMIO_DFH(RING_IMR(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_CMD_ACCESS, NULL, | |
| 2208 | 2356 | intel_vgpu_reg_imr_handler); |
| 2209 | 2357 | |
| 2210 | 2358 | MMIO_DH(GEN8_GT_IMR(0), D_BDW_PLUS, NULL, intel_vgpu_reg_imr_handler); |
| @@ -2269,24 +2417,31 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 58 unmodified lines | ||
| 2269 | 2417 | MMIO_DH(GEN8_MASTER_IRQ, D_BDW_PLUS, NULL, |
| 2270 | 2418 | intel_vgpu_reg_master_irq_handler); |
| 2271 | 2419 | |
| 2272 | − MMIO_D(RING_HWSTAM(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2273 | − MMIO_D(0x1c134, D_BDW_PLUS); | |
| 2274 | − | |
| 2275 | − MMIO_D(RING_TAIL(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2276 | − MMIO_D(RING_HEAD(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2277 | − MMIO_GM(RING_START(GEN8_BSD2_RING_BASE), D_BDW_PLUS, NULL, NULL); | |
| 2278 | − MMIO_D(RING_CTL(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2279 | − MMIO_D(RING_ACTHD(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2280 | − MMIO_D(RING_ACTHD_UDW(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2281 | − MMIO_DFH(0x1c29c, D_BDW_PLUS, F_MODE_MASK, NULL, ring_mode_mmio_write); | |
| 2282 | − MMIO_DFH(RING_MI_MODE(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_MODE_MASK, | |
| 2283 | − NULL, NULL); | |
| 2284 | − MMIO_DFH(RING_INSTPM(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_MODE_MASK, | |
| 2285 | − NULL, NULL); | |
| 2420 | + MMIO_DFH(RING_HWSTAM(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2421 | + F_CMD_ACCESS, NULL, NULL); | |
| 2422 | + MMIO_DFH(0x1c134, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2423 | + | |
| 2424 | + MMIO_DFH(RING_TAIL(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_CMD_ACCESS, | |
| 2425 | + NULL, NULL); | |
| 2426 | + MMIO_DFH(RING_HEAD(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2427 | + F_CMD_ACCESS, NULL, NULL); | |
| 2428 | + MMIO_GM_RDR(RING_START(GEN8_BSD2_RING_BASE), D_BDW_PLUS, NULL, NULL); | |
| 2429 | + MMIO_DFH(RING_CTL(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_CMD_ACCESS, | |
| 2430 | + NULL, NULL); | |
| 2431 | + MMIO_DFH(RING_ACTHD(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2432 | + F_CMD_ACCESS, NULL, NULL); | |
| 2433 | + MMIO_DFH(RING_ACTHD_UDW(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2434 | + F_CMD_ACCESS, NULL, NULL); | |
| 2435 | + MMIO_DFH(0x1c29c, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, | |
| 2436 | + ring_mode_mmio_write); | |
| 2437 | + MMIO_DFH(RING_MI_MODE(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2438 | + F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 2439 | + MMIO_DFH(RING_INSTPM(GEN8_BSD2_RING_BASE), D_BDW_PLUS, | |
| 2440 | + F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 2286 | 2441 | MMIO_DFH(RING_TIMESTAMP(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_CMD_ACCESS, |
| 2287 | 2442 | ring_timestamp_mmio_read, NULL); |
| 2288 | 2443 | |
| 2289 | − MMIO_RING_D(RING_ACTHD_UDW, D_BDW_PLUS); | |
| 2444 | + MMIO_RING_DFH(RING_ACTHD_UDW, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2290 | 2445 | |
| 2291 | 2446 | #define RING_REG(base) (base + 0xd0) |
| 2292 | 2447 | MMIO_RING_F(RING_REG, 4, F_RO, 0, |
| @@ -2303,13 +2458,16 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 10 unmodified lines | ||
| 2303 | 2458 | #undef RING_REG |
| 2304 | 2459 | |
| 2305 | 2460 | #define RING_REG(base) (base + 0x234) |
| 2306 | − MMIO_RING_F(RING_REG, 8, F_RO, 0, ~0, D_BDW_PLUS, NULL, NULL); | |
| 2307 | − MMIO_F(RING_REG(GEN8_BSD2_RING_BASE), 4, F_RO, 0, ~0LL, D_BDW_PLUS, NULL, NULL); | |
| 2461 | + MMIO_RING_F(RING_REG, 8, F_RO | F_CMD_ACCESS, 0, ~0, D_BDW_PLUS, | |
| 2462 | + NULL, NULL); | |
| 2463 | + MMIO_F(RING_REG(GEN8_BSD2_RING_BASE), 4, F_RO | F_CMD_ACCESS, 0, | |
| 2464 | + ~0LL, D_BDW_PLUS, NULL, NULL); | |
| 2308 | 2465 | #undef RING_REG |
| 2309 | 2466 | |
| 2310 | 2467 | #define RING_REG(base) (base + 0x244) |
| 2311 | − MMIO_RING_D(RING_REG, D_BDW_PLUS); | |
| 2312 | − MMIO_D(RING_REG(GEN8_BSD2_RING_BASE), D_BDW_PLUS); | |
| 2468 | + MMIO_RING_DFH(RING_REG, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2469 | + MMIO_DFH(RING_REG(GEN8_BSD2_RING_BASE), D_BDW_PLUS, F_CMD_ACCESS, | |
| 2470 | + NULL, NULL); | |
| 2313 | 2471 | #undef RING_REG |
| 2314 | 2472 | |
| 2315 | 2473 | #define RING_REG(base) (base + 0x370) |
| @@ -2331,6 +2489,8 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 15 unmodified lines | ||
| 2331 | 2489 | MMIO_D(GEN7_MISCCPCTL, D_BDW_PLUS); |
| 2332 | 2490 | MMIO_D(0x1c054, D_BDW_PLUS); |
| 2333 | 2491 | |
| 2492 | + MMIO_DH(GEN6_PCODE_MAILBOX, D_BDW_PLUS, NULL, mailbox_write); | |
| 2493 | + | |
| 2334 | 2494 | MMIO_D(GEN8_PRIVATE_PAT_LO, D_BDW_PLUS); |
| 2335 | 2495 | MMIO_D(GEN8_PRIVATE_PAT_HI, D_BDW_PLUS); |
| 2336 | 2496 | |
| @@ -2341,14 +2501,14 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 4 unmodified lines | ||
| 2341 | 2501 | MMIO_F(RING_REG(GEN8_BSD2_RING_BASE), 32, 0, 0, 0, D_BDW_PLUS, NULL, NULL); |
| 2342 | 2502 | #undef RING_REG |
| 2343 | 2503 | |
| 2344 | − MMIO_RING_GM(RING_HWS_PGA, D_BDW_PLUS, NULL, NULL); | |
| 2345 | − MMIO_GM(0x1c080, D_BDW_PLUS, NULL, NULL); | |
| 2504 | + MMIO_RING_GM_RDR(RING_HWS_PGA, D_BDW_PLUS, NULL, NULL); | |
| 2505 | + MMIO_GM_RDR(RING_HWS_PGA(GEN8_BSD2_RING_BASE), D_BDW_PLUS, NULL, NULL); | |
| 2346 | 2506 | |
| 2347 | 2507 | MMIO_DFH(HDC_CHICKEN0, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 2348 | 2508 | |
| 2349 | − MMIO_D(CHICKEN_PIPESL_1(PIPE_A), D_BDW); | |
| 2350 | − MMIO_D(CHICKEN_PIPESL_1(PIPE_B), D_BDW); | |
| 2351 | − MMIO_D(CHICKEN_PIPESL_1(PIPE_C), D_BDW); | |
| 2509 | + MMIO_D(CHICKEN_PIPESL_1(PIPE_A), D_BDW_PLUS); | |
| 2510 | + MMIO_D(CHICKEN_PIPESL_1(PIPE_B), D_BDW_PLUS); | |
| 2511 | + MMIO_D(CHICKEN_PIPESL_1(PIPE_C), D_BDW_PLUS); | |
| 2352 | 2512 | |
| 2353 | 2513 | MMIO_D(WM_MISC, D_BDW); |
| 2354 | 2514 | MMIO_D(BDW_EDP_PSR_BASE, D_BDW); |
| @@ -2362,27 +2522,31 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 7 unmodified lines | ||
| 2362 | 2522 | MMIO_D(GEN8_EU_DISABLE1, D_BDW_PLUS); |
| 2363 | 2523 | MMIO_D(GEN8_EU_DISABLE2, D_BDW_PLUS); |
| 2364 | 2524 | |
| 2365 | − MMIO_D(0xfdc, D_BDW); | |
| 2366 | − MMIO_DFH(GEN8_ROW_CHICKEN, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2367 | − MMIO_D(GEN7_ROW_CHICKEN2, D_BDW_PLUS); | |
| 2368 | − MMIO_D(GEN8_UCGCTL6, D_BDW_PLUS); | |
| 2525 | + MMIO_D(0xfdc, D_BDW_PLUS); | |
| 2526 | + MMIO_DFH(GEN8_ROW_CHICKEN, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, | |
| 2527 | + NULL, NULL); | |
| 2528 | + MMIO_DFH(GEN7_ROW_CHICKEN2, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, | |
| 2529 | + NULL, NULL); | |
| 2530 | + MMIO_DFH(GEN8_UCGCTL6, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2369 | 2531 | |
| 2370 | − MMIO_D(0xb1f0, D_BDW); | |
| 2371 | − MMIO_D(0xb1c0, D_BDW); | |
| 2532 | + MMIO_DFH(0xb1f0, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2533 | + MMIO_DFH(0xb1c0, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2372 | 2534 | MMIO_DFH(GEN8_L3SQCREG4, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); |
| 2373 | − MMIO_D(0xb100, D_BDW); | |
| 2374 | − MMIO_D(0xb10c, D_BDW); | |
| 2535 | + MMIO_DFH(0xb100, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2536 | + MMIO_DFH(0xb10c, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2375 | 2537 | MMIO_D(0xb110, D_BDW); |
| 2376 | 2538 | |
| 2377 | − MMIO_DFH(0x24d0, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2378 | − MMIO_DFH(0x24d4, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2379 | − MMIO_DFH(0x24d8, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2380 | − MMIO_DFH(0x24dc, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2539 | + MMIO_F(0x24d0, 48, F_CMD_ACCESS, 0, 0, D_BDW_PLUS, | |
| 2540 | + NULL, force_nonpriv_write); | |
| 2541 | + | |
| 2542 | + MMIO_D(0x22040, D_BDW_PLUS); | |
| 2543 | + MMIO_D(0x44484, D_BDW_PLUS); | |
| 2544 | + MMIO_D(0x4448c, D_BDW_PLUS); | |
| 2381 | 2545 | |
| 2382 | − MMIO_D(0x83a4, D_BDW); | |
| 2546 | + MMIO_DFH(0x83a4, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2383 | 2547 | MMIO_D(GEN8_L3_LRA_1_GPGPU, D_BDW_PLUS); |
| 2384 | 2548 | |
| 2385 | − MMIO_D(0x8430, D_BDW); | |
| 2549 | + MMIO_DFH(0x8430, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2386 | 2550 | |
| 2387 | 2551 | MMIO_D(0x110000, D_BDW_PLUS); |
| 2388 | 2552 | |
| @@ -2394,10 +2558,19 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt) · 5 unmodified lines | ||
| 2394 | 2558 | MMIO_DFH(0xe194, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 2395 | 2559 | MMIO_DFH(0xe188, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 2396 | 2560 | MMIO_DFH(HALF_SLICE_CHICKEN2, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); |
| 2397 | − MMIO_DFH(0x2580, D_BDW_PLUS, F_MODE_MASK, NULL, NULL); | |
| 2398 | − | |
| 2399 | − MMIO_D(0x2248, D_BDW); | |
| 2400 | − | |
| 2561 | + MMIO_DFH(0x2580, D_BDW_PLUS, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 2562 | + | |
| 2563 | + MMIO_DFH(0x2248, D_BDW, F_CMD_ACCESS, NULL, NULL); | |
| 2564 | + | |
| 2565 | + MMIO_DFH(0xe220, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2566 | + MMIO_DFH(0xe230, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2567 | + MMIO_DFH(0xe240, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2568 | + MMIO_DFH(0xe260, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2569 | + MMIO_DFH(0xe270, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2570 | + MMIO_DFH(0xe280, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2571 | + MMIO_DFH(0xe2a0, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2572 | + MMIO_DFH(0xe2b0, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2573 | + MMIO_DFH(0xe2c0, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2401 | 2574 | return 0; |
| 2402 | 2575 | } |
| 2403 | 2576 | |
| @@ -2420,7 +2593,6 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 16 unmodified lines | ||
| 2420 | 2593 | MMIO_D(HSW_PWR_WELL_BIOS, D_SKL); |
| 2421 | 2594 | MMIO_DH(HSW_PWR_WELL_DRIVER, D_SKL, NULL, skl_power_well_ctl_write); |
| 2422 | 2595 | |
| 2423 | − MMIO_DH(GEN6_PCODE_MAILBOX, D_SKL, NULL, mailbox_write); | |
| 2424 | 2596 | MMIO_D(0xa210, D_SKL_PLUS); |
| 2425 | 2597 | MMIO_D(GEN9_MEDIA_PG_IDLE_HYSTERESIS, D_SKL_PLUS); |
| 2426 | 2598 | MMIO_D(GEN9_RENDER_PG_IDLE_HYSTERESIS, D_SKL_PLUS); |
| @@ -2578,16 +2750,16 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 151 unmodified lines | ||
| 2578 | 2750 | MMIO_F(0xb020, 0x80, F_CMD_ACCESS, 0, 0, D_SKL, NULL, NULL); |
| 2579 | 2751 | |
| 2580 | 2752 | MMIO_D(0xd08, D_SKL); |
| 2581 | − MMIO_D(0x20e0, D_SKL); | |
| 2582 | − MMIO_D(0x20ec, D_SKL); | |
| 2753 | + MMIO_DFH(0x20e0, D_SKL, F_MODE_MASK, NULL, NULL); | |
| 2754 | + MMIO_DFH(0x20ec, D_SKL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL); | |
| 2583 | 2755 | |
| 2584 | 2756 | /* TRTT */ |
| 2585 | − MMIO_D(0x4de0, D_SKL); | |
| 2586 | − MMIO_D(0x4de4, D_SKL); | |
| 2587 | − MMIO_D(0x4de8, D_SKL); | |
| 2588 | − MMIO_D(0x4dec, D_SKL); | |
| 2589 | − MMIO_D(0x4df0, D_SKL); | |
| 2590 | − MMIO_DH(0x4df4, D_SKL, NULL, gen9_trtte_write); | |
| 2757 | + MMIO_DFH(0x4de0, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2758 | + MMIO_DFH(0x4de4, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2759 | + MMIO_DFH(0x4de8, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2760 | + MMIO_DFH(0x4dec, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2761 | + MMIO_DFH(0x4df0, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2762 | + MMIO_DFH(0x4df4, D_SKL, F_CMD_ACCESS, NULL, gen9_trtte_write); | |
| 2591 | 2763 | MMIO_DH(0x4dfc, D_SKL, NULL, gen9_trtt_chicken_write); |
| 2592 | 2764 | |
| 2593 | 2765 | MMIO_D(0x45008, D_SKL); |
| @@ -2611,7 +2783,7 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 17 unmodified lines | ||
| 2611 | 2783 | MMIO_D(0x65f08, D_SKL); |
| 2612 | 2784 | MMIO_D(0x320f0, D_SKL); |
| 2613 | 2785 | |
| 2614 | − MMIO_D(_REG_VCS2_EXCC, D_SKL); | |
| 2786 | + MMIO_DFH(_REG_VCS2_EXCC, D_SKL, F_CMD_ACCESS, NULL, NULL); | |
| 2615 | 2787 | MMIO_D(0x70034, D_SKL); |
| 2616 | 2788 | MMIO_D(0x71034, D_SKL); |
| 2617 | 2789 | MMIO_D(0x72034, D_SKL); |
| @@ -2624,6 +2796,9 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 6 unmodified lines | ||
| 2624 | 2796 | MMIO_D(_PLANE_KEYMSK_1(PIPE_C), D_SKL); |
| 2625 | 2797 | |
| 2626 | 2798 | MMIO_D(0x44500, D_SKL); |
| 2799 | + MMIO_DFH(GEN9_CSFE_CHICKEN1_RCS, D_SKL_PLUS, F_CMD_ACCESS, NULL, NULL); | |
| 2800 | + MMIO_DFH(GEN8_HDC_CHICKEN1, D_SKL, F_MODE_MASK | F_CMD_ACCESS, | |
| 2801 | + NULL, NULL); | |
| 2627 | 2802 | return 0; |
| 2628 | 2803 | } |
| 2629 | 2804 | |
| @@ -96,10 +96,10 @@ static int gvt_dma_map_iova(struct intel_vgpu *vgpu, kvm_pfn_t pfn, · 95 unmodified lines | ||
| 96 | 96 | struct device *dev = &vgpu->gvt->dev_priv->drm.pdev->dev; |
| 97 | 97 | dma_addr_t daddr; |
| 98 | 98 | |
| 99 | − page = pfn_to_page(pfn); | |
| 100 | − if (is_error_page(page)) | |
| 99 | + if (unlikely(!pfn_valid(pfn))) | |
| 101 | 100 | return -EFAULT; |
| 102 | 101 | |
| 102 | + page = pfn_to_page(pfn); | |
| 103 | 103 | daddr = dma_map_page(dev, page, 0, PAGE_SIZE, |
| 104 | 104 | PCI_DMA_BIDIRECTIONAL); |
| 105 | 105 | if (dma_mapping_error(dev, daddr)) |
| @@ -295,10 +295,10 @@ static ssize_t description_show(struct kobject *kobj, struct device *dev, · 189 unmodified lines | ||
| 295 | 295 | return 0; |
| 296 | 296 | |
| 297 | 297 | return sprintf(buf, "low_gm_size: %dMB\nhigh_gm_size: %dMB\n" |
| 298 | − "fence: %d\n", | |
| 299 | − BYTES_TO_MB(type->low_gm_size), | |
| 300 | − BYTES_TO_MB(type->high_gm_size), | |
| 301 | − type->fence); | |
| 298 | + "fence: %d\nresolution: %s\n", | |
| 299 | + BYTES_TO_MB(type->low_gm_size), | |
| 300 | + BYTES_TO_MB(type->high_gm_size), | |
| 301 | + type->fence, vgpu_edid_str(type->resolution)); | |
| 302 | 302 | } |
| 303 | 303 | |
| 304 | 304 | static MDEV_TYPE_ATTR_RO(available_instances); |
| @@ -57,6 +57,58 @@ int intel_vgpu_gpa_to_mmio_offset(struct intel_vgpu *vgpu, u64 gpa) · 56 unmodified lines | ||
| 57 | 57 | (reg >= gvt->device_info.gtt_start_offset \ |
| 58 | 58 | && reg < gvt->device_info.gtt_start_offset + gvt_ggtt_sz(gvt)) |
| 59 | 59 | |
| 60 | +static void failsafe_emulate_mmio_rw(struct intel_vgpu *vgpu, uint64_t pa, | |
| 61 | + void *p_data, unsigned int bytes, bool read) | |
| 62 | +{ | |
| 63 | + struct intel_gvt *gvt = NULL; | |
| 64 | + void *pt = NULL; | |
| 65 | + unsigned int offset = 0; | |
| 66 | + | |
| 67 | + if (!vgpu || !p_data) | |
| 68 | + return; | |
| 69 | + | |
| 70 | + gvt = vgpu->gvt; | |
| 71 | + mutex_lock(&gvt->lock); | |
| 72 | + offset = intel_vgpu_gpa_to_mmio_offset(vgpu, pa); | |
| 73 | + if (reg_is_mmio(gvt, offset)) { | |
| 74 | + if (read) | |
| 75 | + intel_vgpu_default_mmio_read(vgpu, offset, p_data, | |
| 76 | + bytes); | |
| 77 | + else | |
| 78 | + intel_vgpu_default_mmio_write(vgpu, offset, p_data, | |
| 79 | + bytes); | |
| 80 | + } else if (reg_is_gtt(gvt, offset) && | |
| 81 | + vgpu->gtt.ggtt_mm->virtual_page_table) { | |
| 82 | + offset -= gvt->device_info.gtt_start_offset; | |
| 83 | + pt = vgpu->gtt.ggtt_mm->virtual_page_table + offset; | |
| 84 | + if (read) | |
| 85 | + memcpy(p_data, pt, bytes); | |
| 86 | + else | |
| 87 | + memcpy(pt, p_data, bytes); | |
| 88 | + | |
| 89 | + } else if (atomic_read(&vgpu->gtt.n_write_protected_guest_page)) { | |
| 90 | + struct intel_vgpu_guest_page *gp; | |
| 91 | + | |
| 92 | + /* Since we enter the failsafe mode early during guest boot, | |
| 93 | + * guest may not have chance to set up its ppgtt table, so | |
| 94 | + * there should not be any wp pages for guest. Keep the wp | |
| 95 | + * related code here in case we need to handle it in furture. | |
| 96 | + */ | |
| 97 | + gp = intel_vgpu_find_guest_page(vgpu, pa >> PAGE_SHIFT); | |
| 98 | + if (gp) { | |
| 99 | + /* remove write protection to prevent furture traps */ | |
| 100 | + intel_vgpu_clean_guest_page(vgpu, gp); | |
| 101 | + if (read) | |
| 102 | + intel_gvt_hypervisor_read_gpa(vgpu, pa, | |
| 103 | + p_data, bytes); | |
| 104 | + else | |
| 105 | + intel_gvt_hypervisor_write_gpa(vgpu, pa, | |
| 106 | + p_data, bytes); | |
| 107 | + } | |
| 108 | + } | |
| 109 | + mutex_unlock(&gvt->lock); | |
| 110 | +} | |
| 111 | + | |
| 60 | 112 | /** |
| 61 | 113 | * intel_vgpu_emulate_mmio_read - emulate MMIO read |
| 62 | 114 | * @vgpu: a vGPU |
| @@ -75,6 +127,11 @@ int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, uint64_t pa, · 12 unmodified lines | ||
| 75 | 127 | unsigned int offset = 0; |
| 76 | 128 | int ret = -EINVAL; |
| 77 | 129 | |
| 130 | + | |
| 131 | + if (vgpu->failsafe) { | |
| 132 | + failsafe_emulate_mmio_rw(vgpu, pa, p_data, bytes, true); | |
| 133 | + return 0; | |
| 134 | + } | |
| 78 | 135 | mutex_lock(&gvt->lock); |
| 79 | 136 | |
| 80 | 137 | if (atomic_read(&vgpu->gtt.n_write_protected_guest_page)) { |
| @@ -188,6 +245,11 @@ int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, uint64_t pa, · 107 unmodified lines | ||
| 188 | 245 | u32 old_vreg = 0, old_sreg = 0; |
| 189 | 246 | int ret = -EINVAL; |
| 190 | 247 | |
| 248 | + if (vgpu->failsafe) { | |
| 249 | + failsafe_emulate_mmio_rw(vgpu, pa, p_data, bytes, false); | |
| 250 | + return 0; | |
| 251 | + } | |
| 252 | + | |
| 191 | 253 | mutex_lock(&gvt->lock); |
| 192 | 254 | |
| 193 | 255 | if (atomic_read(&vgpu->gtt.n_write_protected_guest_page)) { |
| @@ -236,7 +298,7 @@ int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, uint64_t pa, · 42 unmodified lines | ||
| 236 | 298 | |
| 237 | 299 | mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4)); |
| 238 | 300 | if (!mmio && !vgpu->mmio.disable_warn_untrack) |
| 239 | − gvt_err("vgpu%d: write untracked MMIO %x len %d val %x\n", | |
| 301 | + gvt_dbg_mmio("vgpu%d: write untracked MMIO %x len %d val %x\n", | |
| 240 | 302 | vgpu->id, offset, bytes, *(u32 *)p_data); |
| 241 | 303 | |
| 242 | 304 | if (!intel_gvt_mmio_is_unalign(gvt, offset)) { |
| @@ -322,6 +384,8 @@ void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu) · 79 unmodified lines | ||
| 322 | 384 | |
| 323 | 385 | /* set the bit 0:2(Core C-State ) to C0 */ |
| 324 | 386 | vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0; |
| 387 | + | |
| 388 | + vgpu->mmio.disable_warn_untrack = false; | |
| 325 | 389 | } |
| 326 | 390 | |
| 327 | 391 | /** |
| @@ -27,7 +27,6 @@ · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | static int init_vgpu_opregion(struct intel_vgpu *vgpu, u32 gpa) |
| 29 | 29 | { |
| 30 | − void __iomem *host_va = vgpu->gvt->opregion.opregion_va; | |
| 31 | 30 | u8 *buf; |
| 32 | 31 | int i; |
| 33 | 32 | |
| @@ -43,8 +42,8 @@ static int init_vgpu_opregion(struct intel_vgpu *vgpu, u32 gpa) · 9 unmodified lines | ||
| 43 | 42 | if (!vgpu_opregion(vgpu)->va) |
| 44 | 43 | return -ENOMEM; |
| 45 | 44 | |
| 46 | − memcpy_fromio(vgpu_opregion(vgpu)->va, host_va, | |
| 47 | − INTEL_GVT_OPREGION_SIZE); | |
| 45 | + memcpy(vgpu_opregion(vgpu)->va, vgpu->gvt->opregion.opregion_va, | |
| 46 | + INTEL_GVT_OPREGION_SIZE); | |
| 48 | 47 | |
| 49 | 48 | for (i = 0; i < INTEL_GVT_OPREGION_PAGES; i++) |
| 50 | 49 | vgpu_opregion(vgpu)->gfn[i] = (gpa >> PAGE_SHIFT) + i; |
| @@ -53,6 +53,14 @@ static struct render_mmio gen8_render_mmio_list[] = { · 52 unmodified lines | ||
| 53 | 53 | {RCS, _MMIO(0x24d4), 0, false}, |
| 54 | 54 | {RCS, _MMIO(0x24d8), 0, false}, |
| 55 | 55 | {RCS, _MMIO(0x24dc), 0, false}, |
| 56 | + {RCS, _MMIO(0x24e0), 0, false}, | |
| 57 | + {RCS, _MMIO(0x24e4), 0, false}, | |
| 58 | + {RCS, _MMIO(0x24e8), 0, false}, | |
| 59 | + {RCS, _MMIO(0x24ec), 0, false}, | |
| 60 | + {RCS, _MMIO(0x24f0), 0, false}, | |
| 61 | + {RCS, _MMIO(0x24f4), 0, false}, | |
| 62 | + {RCS, _MMIO(0x24f8), 0, false}, | |
| 63 | + {RCS, _MMIO(0x24fc), 0, false}, | |
| 56 | 64 | {RCS, _MMIO(0x7004), 0xffff, true}, |
| 57 | 65 | {RCS, _MMIO(0x7008), 0xffff, true}, |
| 58 | 66 | {RCS, _MMIO(0x7000), 0xffff, true}, |
| @@ -76,6 +84,14 @@ static struct render_mmio gen9_render_mmio_list[] = { · 17 unmodified lines | ||
| 76 | 84 | {RCS, _MMIO(0x24d4), 0, false}, |
| 77 | 85 | {RCS, _MMIO(0x24d8), 0, false}, |
| 78 | 86 | {RCS, _MMIO(0x24dc), 0, false}, |
| 87 | + {RCS, _MMIO(0x24e0), 0, false}, | |
| 88 | + {RCS, _MMIO(0x24e4), 0, false}, | |
| 89 | + {RCS, _MMIO(0x24e8), 0, false}, | |
| 90 | + {RCS, _MMIO(0x24ec), 0, false}, | |
| 91 | + {RCS, _MMIO(0x24f0), 0, false}, | |
| 92 | + {RCS, _MMIO(0x24f4), 0, false}, | |
| 93 | + {RCS, _MMIO(0x24f8), 0, false}, | |
| 94 | + {RCS, _MMIO(0x24fc), 0, false}, | |
| 79 | 95 | {RCS, _MMIO(0x7004), 0xffff, true}, |
| 80 | 96 | {RCS, _MMIO(0x7008), 0xffff, true}, |
| 81 | 97 | {RCS, _MMIO(0x7000), 0xffff, true}, |
| @@ -139,6 +139,9 @@ static int shadow_context_status_change(struct notifier_block *nb, · 138 unmodified lines | ||
| 139 | 139 | struct intel_vgpu_workload *workload = |
| 140 | 140 | scheduler->current_workload[req->engine->id]; |
| 141 | 141 | |
| 142 | + if (unlikely(!workload)) | |
| 143 | + return NOTIFY_OK; | |
| 144 | + | |
| 142 | 145 | switch (action) { |
| 143 | 146 | case INTEL_CONTEXT_SCHEDULE_IN: |
| 144 | 147 | intel_gvt_load_render_mmio(workload->vgpu, |
| @@ -148,6 +151,15 @@ static int shadow_context_status_change(struct notifier_block *nb, · 3 unmodified lines | ||
| 148 | 151 | case INTEL_CONTEXT_SCHEDULE_OUT: |
| 149 | 152 | intel_gvt_restore_render_mmio(workload->vgpu, |
| 150 | 153 | workload->ring_id); |
| 154 | + /* If the status is -EINPROGRESS means this workload | |
| 155 | + * doesn't meet any issue during dispatching so when | |
| 156 | + * get the SCHEDULE_OUT set the status to be zero for | |
| 157 | + * good. If the status is NOT -EINPROGRESS means there | |
| 158 | + * is something wrong happened during dispatching and | |
| 159 | + * the status should not be set to zero | |
| 160 | + */ | |
| 161 | + if (workload->status == -EINPROGRESS) | |
| 162 | + workload->status = 0; | |
| 151 | 163 | atomic_set(&workload->shadow_ctx_active, 0); |
| 152 | 164 | break; |
| 153 | 165 | default: |
| @@ -359,15 +371,23 @@ static void complete_current_workload(struct intel_gvt *gvt, int ring_id) · 205 unmodified lines | ||
| 359 | 371 | workload = scheduler->current_workload[ring_id]; |
| 360 | 372 | vgpu = workload->vgpu; |
| 361 | 373 | |
| 362 | − if (!workload->status && !vgpu->resetting) { | |
| 374 | + /* For the workload w/ request, needs to wait for the context | |
| 375 | + * switch to make sure request is completed. | |
| 376 | + * For the workload w/o request, directly complete the workload. | |
| 377 | + */ | |
| 378 | + if (workload->req) { | |
| 363 | 379 | wait_event(workload->shadow_ctx_status_wq, |
| 364 | 380 | !atomic_read(&workload->shadow_ctx_active)); |
| 365 | 381 | |
| 366 | − update_guest_context(workload); | |
| 382 | + i915_gem_request_put(fetch_and_zero(&workload->req)); | |
| 383 | + | |
| 384 | + if (!workload->status && !vgpu->resetting) { | |
| 385 | + update_guest_context(workload); | |
| 367 | 386 | |
| 368 | − for_each_set_bit(event, workload->pending_events, | |
| 369 | − INTEL_GVT_EVENT_MAX) | |
| 370 | − intel_vgpu_trigger_virtual_event(vgpu, event); | |
| 387 | + for_each_set_bit(event, workload->pending_events, | |
| 388 | + INTEL_GVT_EVENT_MAX) | |
| 389 | + intel_vgpu_trigger_virtual_event(vgpu, event); | |
| 390 | + } | |
| 371 | 391 | } |
| 372 | 392 | |
| 373 | 393 | gvt_dbg_sched("ring id %d complete workload %p status %d\n", |
| @@ -397,7 +417,6 @@ static int workload_thread(void *priv) · 23 unmodified lines | ||
| 397 | 417 | int ring_id = p->ring_id; |
| 398 | 418 | struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler; |
| 399 | 419 | struct intel_vgpu_workload *workload = NULL; |
| 400 | − long lret; | |
| 401 | 420 | int ret; |
| 402 | 421 | bool need_force_wake = IS_SKYLAKE(gvt->dev_priv); |
| 403 | 422 | DEFINE_WAIT_FUNC(wait, woken_wake_function); |
| @@ -446,23 +465,24 @@ static int workload_thread(void *priv) · 42 unmodified lines | ||
| 446 | 465 | |
| 447 | 466 | gvt_dbg_sched("ring id %d wait workload %p\n", |
| 448 | 467 | workload->ring_id, workload); |
| 449 | − | |
| 450 | − lret = i915_wait_request(workload->req, | |
| 468 | +retry: | |
| 469 | + i915_wait_request(workload->req, | |
| 451 | 470 | 0, MAX_SCHEDULE_TIMEOUT); |
| 452 | − if (lret < 0) { | |
| 453 | − workload->status = lret; | |
| 454 | − gvt_err("fail to wait workload, skip\n"); | |
| 455 | − } else { | |
| 456 | − workload->status = 0; | |
| 471 | + /* I915 has replay mechanism and a request will be replayed | |
| 472 | + * if there is i915 reset. So the seqno will be updated anyway. | |
| 473 | + * If the seqno is not updated yet after waiting, which means | |
| 474 | + * the replay may still be in progress and we can wait again. | |
| 475 | + */ | |
| 476 | + if (!i915_gem_request_completed(workload->req)) { | |
| 477 | + gvt_dbg_sched("workload %p not completed, wait again\n", | |
| 478 | + workload); | |
| 479 | + goto retry; | |
| 457 | 480 | } |
| 458 | 481 | |
| 459 | 482 | complete: |
| 460 | 483 | gvt_dbg_sched("will complete workload %p, status: %d\n", |
| 461 | 484 | workload, workload->status); |
| 462 | 485 | |
| 463 | − if (workload->req) | |
| 464 | − i915_gem_request_put(fetch_and_zero(&workload->req)); | |
| 465 | − | |
| 466 | 486 | complete_current_workload(gvt, ring_id); |
| 467 | 487 | |
| 468 | 488 | if (need_force_wake) |
| @@ -64,6 +64,20 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu) · 63 unmodified lines | ||
| 64 | 64 | WARN_ON(sizeof(struct vgt_if) != VGT_PVINFO_SIZE); |
| 65 | 65 | } |
| 66 | 66 | |
| 67 | +static struct { | |
| 68 | + unsigned int low_mm; | |
| 69 | + unsigned int high_mm; | |
| 70 | + unsigned int fence; | |
| 71 | + enum intel_vgpu_edid edid; | |
| 72 | + char *name; | |
| 73 | +} vgpu_types[] = { | |
| 74 | +/* Fixed vGPU type table */ | |
| 75 | + { MB_TO_BYTES(64), MB_TO_BYTES(512), 4, GVT_EDID_1024_768, "8" }, | |
| 76 | + { MB_TO_BYTES(128), MB_TO_BYTES(512), 4, GVT_EDID_1920_1200, "4" }, | |
| 77 | + { MB_TO_BYTES(256), MB_TO_BYTES(1024), 4, GVT_EDID_1920_1200, "2" }, | |
| 78 | + { MB_TO_BYTES(512), MB_TO_BYTES(2048), 4, GVT_EDID_1920_1200, "1" }, | |
| 79 | +}; | |
| 80 | + | |
| 67 | 81 | /** |
| 68 | 82 | * intel_gvt_init_vgpu_types - initialize vGPU type list |
| 69 | 83 | * @gvt : GVT device |
| @@ -78,9 +92,8 @@ int intel_gvt_init_vgpu_types(struct intel_gvt *gvt) · 8 unmodified lines | ||
| 78 | 92 | unsigned int min_low; |
| 79 | 93 | |
| 80 | 94 | /* vGPU type name is defined as GVTg_Vx_y which contains |
| 81 | − * physical GPU generation type and 'y' means maximum vGPU | |
| 82 | − * instances user can create on one physical GPU for this | |
| 83 | − * type. | |
| 95 | + * physical GPU generation type (e.g V4 as BDW server, V5 as | |
| 96 | + * SKL server). | |
| 84 | 97 | * |
| 85 | 98 | * Depend on physical SKU resource, might see vGPU types like |
| 86 | 99 | * GVTg_V4_8, GVTg_V4_4, GVTg_V4_2, etc. We can create |
| @@ -92,7 +105,7 @@ int intel_gvt_init_vgpu_types(struct intel_gvt *gvt) · 5 unmodified lines | ||
| 92 | 105 | */ |
| 93 | 106 | low_avail = gvt_aperture_sz(gvt) - HOST_LOW_GM_SIZE; |
| 94 | 107 | high_avail = gvt_hidden_sz(gvt) - HOST_HIGH_GM_SIZE; |
| 95 | − num_types = 4; | |
| 108 | + num_types = sizeof(vgpu_types) / sizeof(vgpu_types[0]); | |
| 96 | 109 | |
| 97 | 110 | gvt->types = kzalloc(num_types * sizeof(struct intel_vgpu_type), |
| 98 | 111 | GFP_KERNEL); |
| @@ -101,28 +114,29 @@ int intel_gvt_init_vgpu_types(struct intel_gvt *gvt) · 2 unmodified lines | ||
| 101 | 114 | |
| 102 | 115 | min_low = MB_TO_BYTES(32); |
| 103 | 116 | for (i = 0; i < num_types; ++i) { |
| 104 | − if (low_avail / min_low == 0) | |
| 117 | + if (low_avail / vgpu_types[i].low_mm == 0) | |
| 105 | 118 | break; |
| 106 | − gvt->types[i].low_gm_size = min_low; | |
| 107 | − gvt->types[i].high_gm_size = max((min_low<<3), MB_TO_BYTES(384U)); | |
| 108 | − gvt->types[i].fence = 4; | |
| 109 | − gvt->types[i].max_instance = min(low_avail / min_low, | |
| 110 | − high_avail / gvt->types[i].high_gm_size); | |
| 111 | − gvt->types[i].avail_instance = gvt->types[i].max_instance; | |
| 119 | + | |
| 120 | + gvt->types[i].low_gm_size = vgpu_types[i].low_mm; | |
| 121 | + gvt->types[i].high_gm_size = vgpu_types[i].high_mm; | |
| 122 | + gvt->types[i].fence = vgpu_types[i].fence; | |
| 123 | + gvt->types[i].resolution = vgpu_types[i].edid; | |
| 124 | + gvt->types[i].avail_instance = min(low_avail / vgpu_types[i].low_mm, | |
| 125 | + high_avail / vgpu_types[i].high_mm); | |
| 112 | 126 | |
| 113 | 127 | if (IS_GEN8(gvt->dev_priv)) |
| 114 | − sprintf(gvt->types[i].name, "GVTg_V4_%u", | |
| 115 | − gvt->types[i].max_instance); | |
| 128 | + sprintf(gvt->types[i].name, "GVTg_V4_%s", | |
| 129 | + vgpu_types[i].name); | |
| 116 | 130 | else if (IS_GEN9(gvt->dev_priv)) |
| 117 | − sprintf(gvt->types[i].name, "GVTg_V5_%u", | |
| 118 | − gvt->types[i].max_instance); | |
| 131 | + sprintf(gvt->types[i].name, "GVTg_V5_%s", | |
| 132 | + vgpu_types[i].name); | |
| 119 | 133 | |
| 120 | − min_low <<= 1; | |
| 121 | − gvt_dbg_core("type[%d]: %s max %u avail %u low %u high %u fence %u\n", | |
| 122 | − i, gvt->types[i].name, gvt->types[i].max_instance, | |
| 134 | + gvt_dbg_core("type[%d]: %s avail %u low %u high %u fence %u res %s\n", | |
| 135 | + i, gvt->types[i].name, | |
| 123 | 136 | gvt->types[i].avail_instance, |
| 124 | 137 | gvt->types[i].low_gm_size, |
| 125 | − gvt->types[i].high_gm_size, gvt->types[i].fence); | |
| 138 | + gvt->types[i].high_gm_size, gvt->types[i].fence, | |
| 139 | + vgpu_edid_str(gvt->types[i].resolution)); | |
| 126 | 140 | } |
| 127 | 141 | |
| 128 | 142 | gvt->num_types = i; |
| @@ -138,7 +152,7 @@ static void intel_gvt_update_vgpu_types(struct intel_gvt *gvt) · 9 unmodified lines | ||
| 138 | 152 | { |
| 139 | 153 | int i; |
| 140 | 154 | unsigned int low_gm_avail, high_gm_avail, fence_avail; |
| 141 | − unsigned int low_gm_min, high_gm_min, fence_min, total_min; | |
| 155 | + unsigned int low_gm_min, high_gm_min, fence_min; | |
| 142 | 156 | |
| 143 | 157 | /* Need to depend on maxium hw resource size but keep on |
| 144 | 158 | * static config for now. |
| @@ -154,12 +168,11 @@ static void intel_gvt_update_vgpu_types(struct intel_gvt *gvt) · 9 unmodified lines | ||
| 154 | 168 | low_gm_min = low_gm_avail / gvt->types[i].low_gm_size; |
| 155 | 169 | high_gm_min = high_gm_avail / gvt->types[i].high_gm_size; |
| 156 | 170 | fence_min = fence_avail / gvt->types[i].fence; |
| 157 | − total_min = min(min(low_gm_min, high_gm_min), fence_min); | |
| 158 | − gvt->types[i].avail_instance = min(gvt->types[i].max_instance, | |
| 159 | − total_min); | |
| 171 | + gvt->types[i].avail_instance = min(min(low_gm_min, high_gm_min), | |
| 172 | + fence_min); | |
| 160 | 173 | |
| 161 | − gvt_dbg_core("update type[%d]: %s max %u avail %u low %u high %u fence %u\n", | |
| 162 | − i, gvt->types[i].name, gvt->types[i].max_instance, | |
| 174 | + gvt_dbg_core("update type[%d]: %s avail %u low %u high %u fence %u\n", | |
| 175 | + i, gvt->types[i].name, | |
| 163 | 176 | gvt->types[i].avail_instance, gvt->types[i].low_gm_size, |
| 164 | 177 | gvt->types[i].high_gm_size, gvt->types[i].fence); |
| 165 | 178 | } |
| @@ -248,7 +261,7 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt, · 82 unmodified lines | ||
| 248 | 261 | if (ret) |
| 249 | 262 | goto out_detach_hypervisor_vgpu; |
| 250 | 263 | |
| 251 | − ret = intel_vgpu_init_display(vgpu); | |
| 264 | + ret = intel_vgpu_init_display(vgpu, param->resolution); | |
| 252 | 265 | if (ret) |
| 253 | 266 | goto out_clean_gtt; |
| 254 | 267 | |
| @@ -312,6 +325,7 @@ struct intel_vgpu *intel_gvt_create_vgpu(struct intel_gvt *gvt, · 57 unmodified lines | ||
| 312 | 325 | param.low_gm_sz = type->low_gm_size; |
| 313 | 326 | param.high_gm_sz = type->high_gm_size; |
| 314 | 327 | param.fence_sz = type->fence; |
| 328 | + param.resolution = type->resolution; | |
| 315 | 329 | |
| 316 | 330 | /* XXX current param based on MB */ |
| 317 | 331 | param.low_gm_sz = BYTES_TO_MB(param.low_gm_sz); |
| @@ -387,8 +401,12 @@ void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr, · 69 unmodified lines | ||
| 387 | 401 | populate_pvinfo_page(vgpu); |
| 388 | 402 | intel_vgpu_reset_display(vgpu); |
| 389 | 403 | |
| 390 | − if (dmlr) | |
| 404 | + if (dmlr) { | |
| 391 | 405 | intel_vgpu_reset_cfg_space(vgpu); |
| 406 | + /* only reset the failsafe mode when dmlr reset */ | |
| 407 | + vgpu->failsafe = false; | |
| 408 | + vgpu->pv_notified = false; | |
| 409 | + } | |
| 392 | 410 | } |
| 393 | 411 | |
| 394 | 412 | vgpu->resetting = false; |
| @@ -65,13 +65,11 @@ static int mxsfb_set_pixel_fmt(struct mxsfb_drm_private *mxsfb) · 64 unmodified lines | ||
| 65 | 65 | switch (format) { |
| 66 | 66 | case DRM_FORMAT_RGB565: |
| 67 | 67 | dev_dbg(drm->dev, "Setting up RGB565 mode\n"); |
| 68 | − ctrl |= CTRL_SET_BUS_WIDTH(STMLCDIF_16BIT); | |
| 69 | 68 | ctrl |= CTRL_SET_WORD_LENGTH(0); |
| 70 | 69 | ctrl1 |= CTRL1_SET_BYTE_PACKAGING(0xf); |
| 71 | 70 | break; |
| 72 | 71 | case DRM_FORMAT_XRGB8888: |
| 73 | 72 | dev_dbg(drm->dev, "Setting up XRGB8888 mode\n"); |
| 74 | − ctrl |= CTRL_SET_BUS_WIDTH(STMLCDIF_24BIT); | |
| 75 | 73 | ctrl |= CTRL_SET_WORD_LENGTH(3); |
| 76 | 74 | /* Do not use packed pixels = one pixel per word instead. */ |
| 77 | 75 | ctrl1 |= CTRL1_SET_BYTE_PACKAGING(0x7); |
| @@ -87,6 +85,36 @@ static int mxsfb_set_pixel_fmt(struct mxsfb_drm_private *mxsfb) · 9 unmodified lines | ||
| 87 | 85 | return 0; |
| 88 | 86 | } |
| 89 | 87 | |
| 88 | +static void mxsfb_set_bus_fmt(struct mxsfb_drm_private *mxsfb) | |
| 89 | +{ | |
| 90 | + struct drm_crtc *crtc = &mxsfb->pipe.crtc; | |
| 91 | + struct drm_device *drm = crtc->dev; | |
| 92 | + u32 bus_format = MEDIA_BUS_FMT_RGB888_1X24; | |
| 93 | + u32 reg; | |
| 94 | + | |
| 95 | + reg = readl(mxsfb->base + LCDC_CTRL); | |
| 96 | + | |
| 97 | + if (mxsfb->connector.display_info.num_bus_formats) | |
| 98 | + bus_format = mxsfb->connector.display_info.bus_formats[0]; | |
| 99 | + | |
| 100 | + reg &= ~CTRL_BUS_WIDTH_MASK; | |
| 101 | + switch (bus_format) { | |
| 102 | + case MEDIA_BUS_FMT_RGB565_1X16: | |
| 103 | + reg |= CTRL_SET_BUS_WIDTH(STMLCDIF_16BIT); | |
| 104 | + break; | |
| 105 | + case MEDIA_BUS_FMT_RGB666_1X18: | |
| 106 | + reg |= CTRL_SET_BUS_WIDTH(STMLCDIF_18BIT); | |
| 107 | + break; | |
| 108 | + case MEDIA_BUS_FMT_RGB888_1X24: | |
| 109 | + reg |= CTRL_SET_BUS_WIDTH(STMLCDIF_24BIT); | |
| 110 | + break; | |
| 111 | + default: | |
| 112 | + dev_err(drm->dev, "Unknown media bus format %d\n", bus_format); | |
| 113 | + break; | |
| 114 | + } | |
| 115 | + writel(reg, mxsfb->base + LCDC_CTRL); | |
| 116 | +} | |
| 117 | + | |
| 90 | 118 | static void mxsfb_enable_controller(struct mxsfb_drm_private *mxsfb) |
| 91 | 119 | { |
| 92 | 120 | u32 reg; |
| @@ -168,13 +196,22 @@ static void mxsfb_crtc_mode_set_nofb(struct mxsfb_drm_private *mxsfb) · 75 unmodified lines | ||
| 168 | 196 | vdctrl0 |= VDCTRL0_HSYNC_ACT_HIGH; |
| 169 | 197 | if (m->flags & DRM_MODE_FLAG_PVSYNC) |
| 170 | 198 | vdctrl0 |= VDCTRL0_VSYNC_ACT_HIGH; |
| 171 | − if (bus_flags & DRM_BUS_FLAG_DE_HIGH) | |
| 199 | + /* Make sure Data Enable is high active by default */ | |
| 200 | + if (!(bus_flags & DRM_BUS_FLAG_DE_LOW)) | |
| 172 | 201 | vdctrl0 |= VDCTRL0_ENABLE_ACT_HIGH; |
| 173 | − if (bus_flags & DRM_BUS_FLAG_PIXDATA_NEGEDGE) | |
| 202 | + /* | |
| 203 | + * DRM_BUS_FLAG_PIXDATA_ defines are controller centric, | |
| 204 | + * controllers VDCTRL0_DOTCLK is display centric. | |
| 205 | + * Drive on positive edge -> display samples on falling edge | |
| 206 | + * DRM_BUS_FLAG_PIXDATA_POSEDGE -> VDCTRL0_DOTCLK_ACT_FALLING | |
| 207 | + */ | |
| 208 | + if (bus_flags & DRM_BUS_FLAG_PIXDATA_POSEDGE) | |
| 174 | 209 | vdctrl0 |= VDCTRL0_DOTCLK_ACT_FALLING; |
| 175 | 210 | |
| 176 | 211 | writel(vdctrl0, mxsfb->base + LCDC_VDCTRL0); |
| 177 | 212 | |
| 213 | + mxsfb_set_bus_fmt(mxsfb); | |
| 214 | + | |
| 178 | 215 | /* Frame length in lines. */ |
| 179 | 216 | writel(m->crtc_vtotal, mxsfb->base + LCDC_VDCTRL1); |
| 180 | 217 | |
| @@ -184,8 +221,8 @@ static void mxsfb_crtc_mode_set_nofb(struct mxsfb_drm_private *mxsfb) · 3 unmodified lines | ||
| 184 | 221 | VDCTRL2_SET_HSYNC_PERIOD(m->crtc_htotal), |
| 185 | 222 | mxsfb->base + LCDC_VDCTRL2); |
| 186 | 223 | |
| 187 | − writel(SET_HOR_WAIT_CNT(m->crtc_hblank_end - m->crtc_hsync_end) | | |
| 188 | − SET_VERT_WAIT_CNT(m->crtc_vblank_end - m->crtc_vsync_end), | |
| 224 | + writel(SET_HOR_WAIT_CNT(m->crtc_htotal - m->crtc_hsync_start) | | |
| 225 | + SET_VERT_WAIT_CNT(m->crtc_vtotal - m->crtc_vsync_start), | |
| 189 | 226 | mxsfb->base + LCDC_VDCTRL3); |
| 190 | 227 | |
| 191 | 228 | writel(SET_DOTCLK_H_VALID_DATA_CNT(m->hdisplay), |
| @@ -102,14 +102,18 @@ static void mxsfb_pipe_enable(struct drm_simple_display_pipe *pipe, · 101 unmodified lines | ||
| 102 | 102 | { |
| 103 | 103 | struct mxsfb_drm_private *mxsfb = drm_pipe_to_mxsfb_drm_private(pipe); |
| 104 | 104 | |
| 105 | + drm_panel_prepare(mxsfb->panel); | |
| 105 | 106 | mxsfb_crtc_enable(mxsfb); |
| 107 | + drm_panel_enable(mxsfb->panel); | |
| 106 | 108 | } |
| 107 | 109 | |
| 108 | 110 | static void mxsfb_pipe_disable(struct drm_simple_display_pipe *pipe) |
| 109 | 111 | { |
| 110 | 112 | struct mxsfb_drm_private *mxsfb = drm_pipe_to_mxsfb_drm_private(pipe); |
| 111 | 113 | |
| 114 | + drm_panel_disable(mxsfb->panel); | |
| 112 | 115 | mxsfb_crtc_disable(mxsfb); |
| 116 | + drm_panel_unprepare(mxsfb->panel); | |
| 113 | 117 | } |
| 114 | 118 | |
| 115 | 119 | static void mxsfb_pipe_update(struct drm_simple_display_pipe *pipe, |
| @@ -112,6 +112,7 @@ static int mxsfb_attach_endpoint(struct drm_device *drm, · 111 unmodified lines | ||
| 112 | 112 | |
| 113 | 113 | int mxsfb_create_output(struct drm_device *drm) |
| 114 | 114 | { |
| 115 | + struct mxsfb_drm_private *mxsfb = drm->dev_private; | |
| 115 | 116 | struct device_node *ep_np = NULL; |
| 116 | 117 | struct of_endpoint ep; |
| 117 | 118 | int ret; |
| @@ -127,5 +128,8 @@ int mxsfb_create_output(struct drm_device *drm) · 9 unmodified lines | ||
| 127 | 128 | } |
| 128 | 129 | } |
| 129 | 130 | |
| 131 | + if (!mxsfb->panel) | |
| 132 | + return -EPROBE_DEFER; | |
| 133 | + | |
| 130 | 134 | return 0; |
| 131 | 135 | } |
| @@ -44,6 +44,7 @@ · 43 unmodified lines | ||
| 44 | 44 | #define CTRL_DATA_SELECT (1 << 16) |
| 45 | 45 | #define CTRL_SET_BUS_WIDTH(x) (((x) & 0x3) << 10) |
| 46 | 46 | #define CTRL_GET_BUS_WIDTH(x) (((x) >> 10) & 0x3) |
| 47 | +#define CTRL_BUS_WIDTH_MASK (0x3 << 10) | |
| 47 | 48 | #define CTRL_SET_WORD_LENGTH(x) (((x) & 0x3) << 8) |
| 48 | 49 | #define CTRL_GET_WORD_LENGTH(x) (((x) >> 8) & 0x3) |
| 49 | 50 | #define CTRL_MASTER (1 << 5) |
| @@ -32,6 +32,10 @@ void rcar_du_vsp_enable(struct rcar_du_crtc *crtc) · 31 unmodified lines | ||
| 32 | 32 | { |
| 33 | 33 | const struct drm_display_mode *mode = &crtc->crtc.state->adjusted_mode; |
| 34 | 34 | struct rcar_du_device *rcdu = crtc->group->dev; |
| 35 | + struct vsp1_du_lif_config cfg = { | |
| 36 | + .width = mode->hdisplay, | |
| 37 | + .height = mode->vdisplay, | |
| 38 | + }; | |
| 35 | 39 | struct rcar_du_plane_state state = { |
| 36 | 40 | .state = { |
| 37 | 41 | .crtc = &crtc->crtc, |
| @@ -66,12 +70,12 @@ void rcar_du_vsp_enable(struct rcar_du_crtc *crtc) · 28 unmodified lines | ||
| 66 | 70 | */ |
| 67 | 71 | crtc->group->need_restart = true; |
| 68 | 72 | |
| 69 | − vsp1_du_setup_lif(crtc->vsp->vsp, mode->hdisplay, mode->vdisplay); | |
| 73 | + vsp1_du_setup_lif(crtc->vsp->vsp, &cfg); | |
| 70 | 74 | } |
| 71 | 75 | |
| 72 | 76 | void rcar_du_vsp_disable(struct rcar_du_crtc *crtc) |
| 73 | 77 | { |
| 74 | − vsp1_du_setup_lif(crtc->vsp->vsp, 0, 0); | |
| 78 | + vsp1_du_setup_lif(crtc->vsp->vsp, NULL); | |
| 75 | 79 | } |
| 76 | 80 | |
| 77 | 81 | void rcar_du_vsp_atomic_begin(struct rcar_du_crtc *crtc) |
| @@ -543,7 +543,7 @@ static int vmbus_close_internal(struct vmbus_channel *channel) · 542 unmodified lines | ||
| 543 | 543 | /* |
| 544 | 544 | * In case a device driver's probe() fails (e.g., |
| 545 | 545 | * util_probe() -> vmbus_open() returns -ENOMEM) and the device is |
| 546 | − * rescinded later (e.g., we dynamically disble an Integrated Service | |
| 546 | + * rescinded later (e.g., we dynamically disable an Integrated Service | |
| 547 | 547 | * in Hyper-V Manager), the driver's remove() invokes vmbus_close(): |
| 548 | 548 | * here we should skip most of the below cleanup work. |
| 549 | 549 | */ |
| @@ -465,6 +465,7 @@ static int brcmstb_i2c_xfer(struct i2c_adapter *adapter, · 464 unmodified lines | ||
| 465 | 465 | u8 *tmp_buf; |
| 466 | 466 | int len = 0; |
| 467 | 467 | int xfersz = brcmstb_i2c_get_xfersz(dev); |
| 468 | + u32 cond, cond_per_msg; | |
| 468 | 469 | |
| 469 | 470 | if (dev->is_suspended) |
| 470 | 471 | return -EBUSY; |
| @@ -481,10 +482,11 @@ static int brcmstb_i2c_xfer(struct i2c_adapter *adapter, · 10 unmodified lines | ||
| 481 | 482 | pmsg->buf ? pmsg->buf[0] : '0', pmsg->len); |
| 482 | 483 | |
| 483 | 484 | if (i < (num - 1) && (msgs[i + 1].flags & I2C_M_NOSTART)) |
| 484 | − brcmstb_set_i2c_start_stop(dev, ~(COND_START_STOP)); | |
| 485 | + cond = ~COND_START_STOP; | |
| 485 | 486 | else |
| 486 | − brcmstb_set_i2c_start_stop(dev, | |
| 487 | − COND_RESTART | COND_NOSTOP); | |
| 487 | + cond = COND_RESTART | COND_NOSTOP; | |
| 488 | + | |
| 489 | + brcmstb_set_i2c_start_stop(dev, cond); | |
| 488 | 490 | |
| 489 | 491 | /* Send slave address */ |
| 490 | 492 | if (!(pmsg->flags & I2C_M_NOSTART)) { |
| @@ -497,13 +499,24 @@ static int brcmstb_i2c_xfer(struct i2c_adapter *adapter, · 6 unmodified lines | ||
| 497 | 499 | } |
| 498 | 500 | } |
| 499 | 501 | |
| 502 | + cond_per_msg = cond; | |
| 503 | + | |
| 500 | 504 | /* Perform data transfer */ |
| 501 | 505 | while (len) { |
| 502 | 506 | bytes_to_xfer = min(len, xfersz); |
| 503 | 507 | |
| 504 | − if (len <= xfersz && i == (num - 1)) | |
| 505 | − brcmstb_set_i2c_start_stop(dev, | |
| 506 | − ~(COND_START_STOP)); | |
| 508 | + if (len <= xfersz) { | |
| 509 | + if (i == (num - 1)) | |
| 510 | + cond_per_msg = cond_per_msg & | |
| 511 | + ~(COND_RESTART | COND_NOSTOP); | |
| 512 | + else | |
| 513 | + cond_per_msg = cond; | |
| 514 | + } else { | |
| 515 | + cond_per_msg = (cond_per_msg & ~COND_RESTART) | | |
| 516 | + COND_NOSTOP; | |
| 517 | + } | |
| 518 | + | |
| 519 | + brcmstb_set_i2c_start_stop(dev, cond_per_msg); | |
| 507 | 520 | |
| 508 | 521 | rc = brcmstb_i2c_xfer_bsc_data(dev, tmp_buf, |
| 509 | 522 | bytes_to_xfer, pmsg); |
| @@ -512,6 +525,8 @@ static int brcmstb_i2c_xfer(struct i2c_adapter *adapter, · 2 unmodified lines | ||
| 512 | 525 | |
| 513 | 526 | len -= bytes_to_xfer; |
| 514 | 527 | tmp_buf += bytes_to_xfer; |
| 528 | + | |
| 529 | + cond_per_msg = COND_NOSTART | COND_NOSTOP; | |
| 515 | 530 | } |
| 516 | 531 | } |
| 517 | 532 | |
| @@ -88,6 +88,7 @@ struct dw_i2c_dev { · 87 unmodified lines | ||
| 88 | 88 | void __iomem *base; |
| 89 | 89 | struct completion cmd_complete; |
| 90 | 90 | struct clk *clk; |
| 91 | + struct reset_control *rst; | |
| 91 | 92 | u32 (*get_clk_rate_khz) (struct dw_i2c_dev *dev); |
| 92 | 93 | struct dw_pci_controller *controller; |
| 93 | 94 | int cmd_err; |
| @@ -38,6 +38,7 @@ · 37 unmodified lines | ||
| 38 | 38 | #include <linux/pm_runtime.h> |
| 39 | 39 | #include <linux/property.h> |
| 40 | 40 | #include <linux/io.h> |
| 41 | +#include <linux/reset.h> | |
| 41 | 42 | #include <linux/slab.h> |
| 42 | 43 | #include <linux/acpi.h> |
| 43 | 44 | #include <linux/platform_data/i2c-designware.h> |
| @@ -199,6 +200,14 @@ static int dw_i2c_plat_probe(struct platform_device *pdev) · 155 unmodified lines | ||
| 199 | 200 | dev->irq = irq; |
| 200 | 201 | platform_set_drvdata(pdev, dev); |
| 201 | 202 | |
| 203 | + dev->rst = devm_reset_control_get_optional_exclusive(&pdev->dev, NULL); | |
| 204 | + if (IS_ERR(dev->rst)) { | |
| 205 | + if (PTR_ERR(dev->rst) == -EPROBE_DEFER) | |
| 206 | + return -EPROBE_DEFER; | |
| 207 | + } else { | |
| 208 | + reset_control_deassert(dev->rst); | |
| 209 | + } | |
| 210 | + | |
| 202 | 211 | if (pdata) { |
| 203 | 212 | dev->clk_freq = pdata->i2c_scl_freq; |
| 204 | 213 | } else { |
| @@ -235,12 +244,13 @@ static int dw_i2c_plat_probe(struct platform_device *pdev) · 30 unmodified lines | ||
| 235 | 244 | && dev->clk_freq != 1000000 && dev->clk_freq != 3400000) { |
| 236 | 245 | dev_err(&pdev->dev, |
| 237 | 246 | "Only 100kHz, 400kHz, 1MHz and 3.4MHz supported"); |
| 238 | − return -EINVAL; | |
| 247 | + r = -EINVAL; | |
| 248 | + goto exit_reset; | |
| 239 | 249 | } |
| 240 | 250 | |
| 241 | 251 | r = i2c_dw_eval_lock_support(dev); |
| 242 | 252 | if (r) |
| 243 | − return r; | |
| 253 | + goto exit_reset; | |
| 244 | 254 | |
| 245 | 255 | dev->functionality = I2C_FUNC_10BIT_ADDR | DW_IC_DEFAULT_FUNCTIONALITY; |
| 246 | 256 | |
| @@ -286,10 +296,18 @@ static int dw_i2c_plat_probe(struct platform_device *pdev) · 39 unmodified lines | ||
| 286 | 296 | } |
| 287 | 297 | |
| 288 | 298 | r = i2c_dw_probe(dev); |
| 289 | − if (r && !dev->pm_runtime_disabled) | |
| 290 | − pm_runtime_disable(&pdev->dev); | |
| 299 | + if (r) | |
| 300 | + goto exit_probe; | |
| 291 | 301 | |
| 292 | 302 | return r; |
| 303 | + | |
| 304 | +exit_probe: | |
| 305 | + if (!dev->pm_runtime_disabled) | |
| 306 | + pm_runtime_disable(&pdev->dev); | |
| 307 | +exit_reset: | |
| 308 | + if (!IS_ERR_OR_NULL(dev->rst)) | |
| 309 | + reset_control_assert(dev->rst); | |
| 310 | + return r; | |
| 293 | 311 | } |
| 294 | 312 | |
| 295 | 313 | static int dw_i2c_plat_remove(struct platform_device *pdev) |
| @@ -306,6 +324,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev) · 10 unmodified lines | ||
| 306 | 324 | pm_runtime_put_sync(&pdev->dev); |
| 307 | 325 | if (!dev->pm_runtime_disabled) |
| 308 | 326 | pm_runtime_disable(&pdev->dev); |
| 327 | + if (!IS_ERR_OR_NULL(dev->rst)) | |
| 328 | + reset_control_assert(dev->rst); | |
| 309 | 329 | |
| 310 | 330 | return 0; |
| 311 | 331 | } |
| @@ -457,7 +457,6 @@ static irqreturn_t exynos5_i2c_irq(int irqno, void *dev_id) · 456 unmodified lines | ||
| 457 | 457 | |
| 458 | 458 | int_status = readl(i2c->regs + HSI2C_INT_STATUS); |
| 459 | 459 | writel(int_status, i2c->regs + HSI2C_INT_STATUS); |
| 460 | − trans_status = readl(i2c->regs + HSI2C_TRANS_STATUS); | |
| 461 | 460 | |
| 462 | 461 | /* handle interrupt related to the transfer status */ |
| 463 | 462 | if (i2c->variant->hw == HSI2C_EXYNOS7) { |
| @@ -482,11 +481,13 @@ static irqreturn_t exynos5_i2c_irq(int irqno, void *dev_id) · 18 unmodified lines | ||
| 482 | 481 | goto stop; |
| 483 | 482 | } |
| 484 | 483 | |
| 484 | + trans_status = readl(i2c->regs + HSI2C_TRANS_STATUS); | |
| 485 | 485 | if ((trans_status & HSI2C_MASTER_ST_MASK) == HSI2C_MASTER_ST_LOSE) { |
| 486 | 486 | i2c->state = -EAGAIN; |
| 487 | 487 | goto stop; |
| 488 | 488 | } |
| 489 | 489 | } else if (int_status & HSI2C_INT_I2C) { |
| 490 | + trans_status = readl(i2c->regs + HSI2C_TRANS_STATUS); | |
| 490 | 491 | if (trans_status & HSI2C_NO_DEV_ACK) { |
| 491 | 492 | dev_dbg(i2c->dev, "No ACK from device\n"); |
| 492 | 493 | i2c->state = -ENXIO; |
| @@ -175,7 +175,7 @@ static void meson_i2c_put_data(struct meson_i2c *i2c, char *buf, int len) · 174 unmodified lines | ||
| 175 | 175 | wdata1 |= *buf++ << ((i - 4) * 8); |
| 176 | 176 | |
| 177 | 177 | writel(wdata0, i2c->regs + REG_TOK_WDATA0); |
| 178 | − writel(wdata0, i2c->regs + REG_TOK_WDATA1); | |
| 178 | + writel(wdata1, i2c->regs + REG_TOK_WDATA1); | |
| 179 | 179 | |
| 180 | 180 | dev_dbg(i2c->dev, "%s: data %08x %08x len %d\n", __func__, |
| 181 | 181 | wdata0, wdata1, len); |
| @@ -172,14 +172,6 @@ static const struct i2c_adapter_quirks mt6577_i2c_quirks = { · 171 unmodified lines | ||
| 172 | 172 | .max_comb_2nd_msg_len = 31, |
| 173 | 173 | }; |
| 174 | 174 | |
| 175 | −static const struct i2c_adapter_quirks mt8173_i2c_quirks = { | |
| 176 | − .max_num_msgs = 65535, | |
| 177 | − .max_write_len = 65535, | |
| 178 | − .max_read_len = 65535, | |
| 179 | − .max_comb_1st_msg_len = 65535, | |
| 180 | − .max_comb_2nd_msg_len = 65535, | |
| 181 | −}; | |
| 182 | − | |
| 183 | 175 | static const struct mtk_i2c_compatible mt6577_compat = { |
| 184 | 176 | .quirks = &mt6577_i2c_quirks, |
| 185 | 177 | .pmic_i2c = 0, |
| @@ -199,7 +191,6 @@ static const struct mtk_i2c_compatible mt6589_compat = { · 13 unmodified lines | ||
| 199 | 191 | }; |
| 200 | 192 | |
| 201 | 193 | static const struct mtk_i2c_compatible mt8173_compat = { |
| 202 | − .quirks = &mt8173_i2c_quirks, | |
| 203 | 194 | .pmic_i2c = 0, |
| 204 | 195 | .dcm = 1, |
| 205 | 196 | .auto_restart = 1, |
| @@ -218,8 +218,12 @@ static irqreturn_t riic_tend_isr(int irq, void *data) · 217 unmodified lines | ||
| 218 | 218 | } |
| 219 | 219 | |
| 220 | 220 | if (riic->is_last || riic->err) { |
| 221 | − riic_clear_set_bit(riic, 0, ICIER_SPIE, RIIC_ICIER); | |
| 221 | + riic_clear_set_bit(riic, ICIER_TEIE, ICIER_SPIE, RIIC_ICIER); | |
| 222 | 222 | writeb(ICCR2_SP, riic->base + RIIC_ICCR2); |
| 223 | + } else { | |
| 224 | + /* Transfer is complete, but do not send STOP */ | |
| 225 | + riic_clear_set_bit(riic, ICIER_TEIE, 0, RIIC_ICIER); | |
| 226 | + complete(&riic->msg_done); | |
| 223 | 227 | } |
| 224 | 228 | |
| 225 | 229 | return IRQ_HANDLED; |
| @@ -429,6 +429,7 @@ void i2c_mux_del_adapters(struct i2c_mux_core *muxc) · 428 unmodified lines | ||
| 429 | 429 | while (muxc->num_adapters) { |
| 430 | 430 | struct i2c_adapter *adap = muxc->adapter[--muxc->num_adapters]; |
| 431 | 431 | struct i2c_mux_priv *priv = adap->algo_data; |
| 432 | + struct device_node *np = adap->dev.of_node; | |
| 432 | 433 | |
| 433 | 434 | muxc->adapter[muxc->num_adapters] = NULL; |
| 434 | 435 | |
| @@ -438,6 +439,7 @@ void i2c_mux_del_adapters(struct i2c_mux_core *muxc) · 3 unmodified lines | ||
| 438 | 439 | |
| 439 | 440 | sysfs_remove_link(&priv->adap.dev.kobj, "mux_device"); |
| 440 | 441 | i2c_del_adapter(adap); |
| 442 | + of_node_put(np); | |
| 441 | 443 | kfree(priv); |
| 442 | 444 | } |
| 443 | 445 | } |
| @@ -198,7 +198,8 @@ static const struct irq_domain_ops crossbar_domain_ops = { · 197 unmodified lines | ||
| 198 | 198 | |
| 199 | 199 | static int __init crossbar_of_init(struct device_node *node) |
| 200 | 200 | { |
| 201 | − int i, size, max = 0, reserved = 0, entry; | |
| 201 | + u32 max = 0, entry, reg_size; | |
| 202 | + int i, size, reserved = 0; | |
| 202 | 203 | const __be32 *irqsr; |
| 203 | 204 | int ret = -ENOMEM; |
| 204 | 205 | |
| @@ -275,9 +276,9 @@ static int __init crossbar_of_init(struct device_node *node) · 70 unmodified lines | ||
| 275 | 276 | if (!cb->register_offsets) |
| 276 | 277 | goto err_irq_map; |
| 277 | 278 | |
| 278 | − of_property_read_u32(node, "ti,reg-size", &size); | |
| 279 | + of_property_read_u32(node, "ti,reg-size", ®_size); | |
| 279 | 280 | |
| 280 | − switch (size) { | |
| 281 | + switch (reg_size) { | |
| 281 | 282 | case 1: |
| 282 | 283 | cb->write = crossbar_writeb; |
| 283 | 284 | break; |
| @@ -303,7 +304,7 @@ static int __init crossbar_of_init(struct device_node *node) · 19 unmodified lines | ||
| 303 | 304 | continue; |
| 304 | 305 | |
| 305 | 306 | cb->register_offsets[i] = reserved; |
| 306 | − reserved += size; | |
| 307 | + reserved += reg_size; | |
| 307 | 308 | } |
| 308 | 309 | |
| 309 | 310 | of_property_read_u32(node, "ti,irqs-safe-map", &cb->safe_map); |
| @@ -1601,6 +1601,14 @@ static void __maybe_unused its_enable_quirk_cavium_23144(void *data) · 1600 unmodified lines | ||
| 1601 | 1601 | its->flags |= ITS_FLAGS_WORKAROUND_CAVIUM_23144; |
| 1602 | 1602 | } |
| 1603 | 1603 | |
| 1604 | +static void __maybe_unused its_enable_quirk_qdf2400_e0065(void *data) | |
| 1605 | +{ | |
| 1606 | + struct its_node *its = data; | |
| 1607 | + | |
| 1608 | + /* On QDF2400, the size of the ITE is 16Bytes */ | |
| 1609 | + its->ite_size = 16; | |
| 1610 | +} | |
| 1611 | + | |
| 1604 | 1612 | static const struct gic_quirk its_quirks[] = { |
| 1605 | 1613 | #ifdef CONFIG_CAVIUM_ERRATUM_22375 |
| 1606 | 1614 | { |
| @@ -1617,6 +1625,14 @@ static const struct gic_quirk its_quirks[] = { · 10 unmodified lines | ||
| 1617 | 1625 | .mask = 0xffff0fff, |
| 1618 | 1626 | .init = its_enable_quirk_cavium_23144, |
| 1619 | 1627 | }, |
| 1628 | +#endif | |
| 1629 | +#ifdef CONFIG_QCOM_QDF2400_ERRATUM_0065 | |
| 1630 | + { | |
| 1631 | + .desc = "ITS: QDF2400 erratum 0065", | |
| 1632 | + .iidr = 0x00001070, /* QDF2400 ITS rev 1.x */ | |
| 1633 | + .mask = 0xffffffff, | |
| 1634 | + .init = its_enable_quirk_qdf2400_e0065, | |
| 1635 | + }, | |
| 1620 | 1636 | #endif |
| 1621 | 1637 | { |
| 1622 | 1638 | } |
| @@ -2317,6 +2317,9 @@ static int gigaset_probe(struct usb_interface *interface, · 2316 unmodified lines | ||
| 2317 | 2317 | return -ENODEV; |
| 2318 | 2318 | } |
| 2319 | 2319 | |
| 2320 | + if (hostif->desc.bNumEndpoints < 1) | |
| 2321 | + return -ENODEV; | |
| 2322 | + | |
| 2320 | 2323 | dev_info(&udev->dev, |
| 2321 | 2324 | "%s: Device matched (Vendor: 0x%x, Product: 0x%x)\n", |
| 2322 | 2325 | __func__, le16_to_cpu(udev->descriptor.idVendor), |
| @@ -239,7 +239,7 @@ static void st5481B_mode(struct st5481_bcs *bcs, int mode) · 238 unmodified lines | ||
| 239 | 239 | } |
| 240 | 240 | } |
| 241 | 241 | } else { |
| 242 | − // Disble B channel interrupts | |
| 242 | + // Disable B channel interrupts | |
| 243 | 243 | st5481_usb_device_ctrl_msg(adapter, FFMSK_B1+(bcs->channel * 2), 0, NULL, NULL); |
| 244 | 244 | |
| 245 | 245 | // Disable B channel FIFOs |
| @@ -392,6 +392,7 @@ static struct macio_dev * macio_add_one_device(struct macio_chip *chip, · 391 unmodified lines | ||
| 392 | 392 | * To get all the fields, copy all archdata |
| 393 | 393 | */ |
| 394 | 394 | dev->ofdev.dev.archdata = chip->lbus.pdev->dev.archdata; |
| 395 | + dev->ofdev.dev.dma_ops = chip->lbus.pdev->dev.dma_ops; | |
| 395 | 396 | #endif /* CONFIG_PCI */ |
| 396 | 397 | |
| 397 | 398 | #ifdef DEBUG |
| @@ -4,7 +4,6 @@ · 3 unmodified lines | ||
| 4 | 4 | |
| 5 | 5 | #include <linux/blkdev.h> |
| 6 | 6 | #include <linux/errno.h> |
| 7 | −#include <linux/blkdev.h> | |
| 8 | 7 | #include <linux/kernel.h> |
| 9 | 8 | #include <linux/sched/clock.h> |
| 10 | 9 | #include <linux/llist.h> |
| @@ -989,26 +989,29 @@ static void flush_current_bio_list(struct blk_plug_cb *cb, bool from_schedule) · 988 unmodified lines | ||
| 989 | 989 | struct dm_offload *o = container_of(cb, struct dm_offload, cb); |
| 990 | 990 | struct bio_list list; |
| 991 | 991 | struct bio *bio; |
| 992 | + int i; | |
| 992 | 993 | |
| 993 | 994 | INIT_LIST_HEAD(&o->cb.list); |
| 994 | 995 | |
| 995 | 996 | if (unlikely(!current->bio_list)) |
| 996 | 997 | return; |
| 997 | 998 | |
| 998 | − list = *current->bio_list; | |
| 999 | − bio_list_init(current->bio_list); | |
| 1000 | − | |
| 1001 | − while ((bio = bio_list_pop(&list))) { | |
| 1002 | − struct bio_set *bs = bio->bi_pool; | |
| 1003 | − if (unlikely(!bs) || bs == fs_bio_set) { | |
| 1004 | − bio_list_add(current->bio_list, bio); | |
| 1005 | − continue; | |
| 999 | + for (i = 0; i < 2; i++) { | |
| 1000 | + list = current->bio_list[i]; | |
| 1001 | + bio_list_init(¤t->bio_list[i]); | |
| 1002 | + | |
| 1003 | + while ((bio = bio_list_pop(&list))) { | |
| 1004 | + struct bio_set *bs = bio->bi_pool; | |
| 1005 | + if (unlikely(!bs) || bs == fs_bio_set) { | |
| 1006 | + bio_list_add(¤t->bio_list[i], bio); | |
| 1007 | + continue; | |
| 1008 | + } | |
| 1009 | + | |
| 1010 | + spin_lock(&bs->rescue_lock); | |
| 1011 | + bio_list_add(&bs->rescue_list, bio); | |
| 1012 | + queue_work(bs->rescue_workqueue, &bs->rescue_work); | |
| 1013 | + spin_unlock(&bs->rescue_lock); | |
| 1006 | 1014 | } |
| 1007 | − | |
| 1008 | − spin_lock(&bs->rescue_lock); | |
| 1009 | − bio_list_add(&bs->rescue_list, bio); | |
| 1010 | − queue_work(bs->rescue_workqueue, &bs->rescue_work); | |
| 1011 | − spin_unlock(&bs->rescue_lock); | |
| 1012 | 1015 | } |
| 1013 | 1016 | } |
| 1014 | 1017 | |
| @@ -974,7 +974,8 @@ static void wait_barrier(struct r10conf *conf) · 973 unmodified lines | ||
| 974 | 974 | !conf->barrier || |
| 975 | 975 | (atomic_read(&conf->nr_pending) && |
| 976 | 976 | current->bio_list && |
| 977 | − !bio_list_empty(current->bio_list)), | |
| 977 | + (!bio_list_empty(¤t->bio_list[0]) || | |
| 978 | + !bio_list_empty(¤t->bio_list[1]))), | |
| 978 | 979 | conf->resync_lock); |
| 979 | 980 | conf->nr_waiting--; |
| 980 | 981 | if (!conf->nr_waiting) |
| @@ -256,8 +256,7 @@ int drxbsp_tuner_default_i2c_write_read(struct tuner_instance *tuner, · 255 unmodified lines | ||
| 256 | 256 | * |
| 257 | 257 | * The actual DAP implementation may be restricted to only one of the modes. |
| 258 | 258 | * A compiler warning or error will be generated if the DAP implementation |
| 259 | −* overides or cannot handle the mode defined below. | |
| 260 | −* | |
| 259 | +* overrides or cannot handle the mode defined below. | |
| 261 | 260 | */ |
| 262 | 261 | #ifndef DRXDAP_SINGLE_MASTER |
| 263 | 262 | #define DRXDAP_SINGLE_MASTER 1 |
| @@ -272,7 +271,7 @@ int drxbsp_tuner_default_i2c_write_read(struct tuner_instance *tuner, · 8 unmodified lines | ||
| 272 | 271 | * |
| 273 | 272 | * This maximum size may be restricted by the actual DAP implementation. |
| 274 | 273 | * A compiler warning or error will be generated if the DAP implementation |
| 275 | −* overides or cannot handle the chunksize defined below. | |
| 274 | +* overrides or cannot handle the chunksize defined below. | |
| 276 | 275 | * |
| 277 | 276 | * Beware that the DAP uses DRXDAP_MAX_WCHUNKSIZE to create a temporary data |
| 278 | 277 | * buffer. Do not undefine or choose too large, unless your system is able to |
| @@ -292,8 +291,7 @@ int drxbsp_tuner_default_i2c_write_read(struct tuner_instance *tuner, · 13 unmodified lines | ||
| 292 | 291 | * |
| 293 | 292 | * This maximum size may be restricted by the actual DAP implementation. |
| 294 | 293 | * A compiler warning or error will be generated if the DAP implementation |
| 295 | −* overides or cannot handle the chunksize defined below. | |
| 296 | −* | |
| 294 | +* overrides or cannot handle the chunksize defined below. | |
| 297 | 295 | */ |
| 298 | 296 | #ifndef DRXDAP_MAX_RCHUNKSIZE |
| 299 | 297 | #define DRXDAP_MAX_RCHUNKSIZE 60 |
| @@ -54,12 +54,11 @@ EXPORT_SYMBOL_GPL(vsp1_du_init); · 53 unmodified lines | ||
| 54 | 54 | /** |
| 55 | 55 | * vsp1_du_setup_lif - Setup the output part of the VSP pipeline |
| 56 | 56 | * @dev: the VSP device |
| 57 | − * @width: output frame width in pixels | |
| 58 | − * @height: output frame height in pixels | |
| 57 | + * @cfg: the LIF configuration | |
| 59 | 58 | * |
| 60 | − * Configure the output part of VSP DRM pipeline for the given frame @width and | |
| 61 | − * @height. This sets up formats on the BRU source pad, the WPF0 sink and source | |
| 62 | − * pads, and the LIF sink pad. | |
| 59 | + * Configure the output part of VSP DRM pipeline for the given frame @cfg.width | |
| 60 | + * and @cfg.height. This sets up formats on the BRU source pad, the WPF0 sink | |
| 61 | + * and source pads, and the LIF sink pad. | |
| 63 | 62 | * |
| 64 | 63 | * As the media bus code on the BRU source pad is conditioned by the |
| 65 | 64 | * configuration of the BRU sink 0 pad, we also set up the formats on all BRU |
| @@ -69,8 +68,7 @@ EXPORT_SYMBOL_GPL(vsp1_du_init); · 3 unmodified lines | ||
| 69 | 68 | * |
| 70 | 69 | * Return 0 on success or a negative error code on failure. |
| 71 | 70 | */ |
| 72 | −int vsp1_du_setup_lif(struct device *dev, unsigned int width, | |
| 73 | − unsigned int height) | |
| 71 | +int vsp1_du_setup_lif(struct device *dev, const struct vsp1_du_lif_config *cfg) | |
| 74 | 72 | { |
| 75 | 73 | struct vsp1_device *vsp1 = dev_get_drvdata(dev); |
| 76 | 74 | struct vsp1_pipeline *pipe = &vsp1->drm->pipe; |
| @@ -79,11 +77,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width, · 2 unmodified lines | ||
| 79 | 77 | unsigned int i; |
| 80 | 78 | int ret; |
| 81 | 79 | |
| 82 | − dev_dbg(vsp1->dev, "%s: configuring LIF with format %ux%u\n", | |
| 83 | − __func__, width, height); | |
| 84 | − | |
| 85 | − if (width == 0 || height == 0) { | |
| 86 | − /* Zero width or height means the CRTC is being disabled, stop | |
| 80 | + if (!cfg) { | |
| 81 | + /* NULL configuration means the CRTC is being disabled, stop | |
| 87 | 82 | * the pipeline and turn the light off. |
| 88 | 83 | */ |
| 89 | 84 | ret = vsp1_pipeline_stop(pipe); |
| @@ -108,6 +103,9 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width, · 18 unmodified lines | ||
| 108 | 103 | return 0; |
| 109 | 104 | } |
| 110 | 105 | |
| 106 | + dev_dbg(vsp1->dev, "%s: configuring LIF with format %ux%u\n", | |
| 107 | + __func__, cfg->width, cfg->height); | |
| 108 | + | |
| 111 | 109 | /* Configure the format at the BRU sinks and propagate it through the |
| 112 | 110 | * pipeline. |
| 113 | 111 | */ |
| @@ -117,8 +115,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width, · 3 unmodified lines | ||
| 117 | 115 | for (i = 0; i < bru->entity.source_pad; ++i) { |
| 118 | 116 | format.pad = i; |
| 119 | 117 | |
| 120 | − format.format.width = width; | |
| 121 | − format.format.height = height; | |
| 118 | + format.format.width = cfg->width; | |
| 119 | + format.format.height = cfg->height; | |
| 122 | 120 | format.format.code = MEDIA_BUS_FMT_ARGB8888_1X32; |
| 123 | 121 | format.format.field = V4L2_FIELD_NONE; |
| 124 | 122 | |
| @@ -133,8 +131,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width, · 8 unmodified lines | ||
| 133 | 131 | } |
| 134 | 132 | |
| 135 | 133 | format.pad = bru->entity.source_pad; |
| 136 | − format.format.width = width; | |
| 137 | − format.format.height = height; | |
| 134 | + format.format.width = cfg->width; | |
| 135 | + format.format.height = cfg->height; | |
| 138 | 136 | format.format.code = MEDIA_BUS_FMT_ARGB8888_1X32; |
| 139 | 137 | format.format.field = V4L2_FIELD_NONE; |
| 140 | 138 | |
| @@ -180,7 +178,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width, · 39 unmodified lines | ||
| 180 | 178 | /* Verify that the format at the output of the pipeline matches the |
| 181 | 179 | * requested frame size and media bus code. |
| 182 | 180 | */ |
| 183 | − if (format.format.width != width || format.format.height != height || | |
| 181 | + if (format.format.width != cfg->width || | |
| 182 | + format.format.height != cfg->height || | |
| 184 | 183 | format.format.code != MEDIA_BUS_FMT_ARGB8888_1X32) { |
| 185 | 184 | dev_dbg(vsp1->dev, "%s: format mismatch\n", __func__); |
| 186 | 185 | return -EPIPE; |
| @@ -436,6 +436,8 @@ int lirc_dev_fop_open(struct inode *inode, struct file *file) · 435 unmodified lines | ||
| 436 | 436 | return -ERESTARTSYS; |
| 437 | 437 | |
| 438 | 438 | ir = irctls[iminor(inode)]; |
| 439 | + mutex_unlock(&lirc_dev_lock); | |
| 440 | + | |
| 439 | 441 | if (!ir) { |
| 440 | 442 | retval = -ENODEV; |
| 441 | 443 | goto error; |
| @@ -476,8 +478,6 @@ int lirc_dev_fop_open(struct inode *inode, struct file *file) · 34 unmodified lines | ||
| 476 | 478 | } |
| 477 | 479 | |
| 478 | 480 | error: |
| 479 | − mutex_unlock(&lirc_dev_lock); | |
| 480 | − | |
| 481 | 481 | nonseekable_open(inode, file); |
| 482 | 482 | |
| 483 | 483 | return retval; |
| @@ -176,12 +176,13 @@ static void nvt_write_wakeup_codes(struct rc_dev *dev, · 175 unmodified lines | ||
| 176 | 176 | { |
| 177 | 177 | u8 tolerance, config; |
| 178 | 178 | struct nvt_dev *nvt = dev->priv; |
| 179 | + unsigned long flags; | |
| 179 | 180 | int i; |
| 180 | 181 | |
| 181 | 182 | /* hardcode the tolerance to 10% */ |
| 182 | 183 | tolerance = DIV_ROUND_UP(count, 10); |
| 183 | 184 | |
| 184 | − spin_lock(&nvt->lock); | |
| 185 | + spin_lock_irqsave(&nvt->lock, flags); | |
| 185 | 186 | |
| 186 | 187 | nvt_clear_cir_wake_fifo(nvt); |
| 187 | 188 | nvt_cir_wake_reg_write(nvt, count, CIR_WAKE_FIFO_CMP_DEEP); |
| @@ -203,7 +204,7 @@ static void nvt_write_wakeup_codes(struct rc_dev *dev, · 15 unmodified lines | ||
| 203 | 204 | |
| 204 | 205 | nvt_cir_wake_reg_write(nvt, config, CIR_WAKE_IRCON); |
| 205 | 206 | |
| 206 | − spin_unlock(&nvt->lock); | |
| 207 | + spin_unlock_irqrestore(&nvt->lock, flags); | |
| 207 | 208 | } |
| 208 | 209 | |
| 209 | 210 | static ssize_t wakeup_data_show(struct device *dev, |
| @@ -1663,6 +1663,7 @@ static int rc_setup_rx_device(struct rc_dev *dev) · 1662 unmodified lines | ||
| 1663 | 1663 | { |
| 1664 | 1664 | int rc; |
| 1665 | 1665 | struct rc_map *rc_map; |
| 1666 | + u64 rc_type; | |
| 1666 | 1667 | |
| 1667 | 1668 | if (!dev->map_name) |
| 1668 | 1669 | return -EINVAL; |
| @@ -1677,15 +1678,18 @@ static int rc_setup_rx_device(struct rc_dev *dev) · 8 unmodified lines | ||
| 1677 | 1678 | if (rc) |
| 1678 | 1679 | return rc; |
| 1679 | 1680 | |
| 1680 | − if (dev->change_protocol) { | |
| 1681 | − u64 rc_type = (1ll << rc_map->rc_type); | |
| 1681 | + rc_type = BIT_ULL(rc_map->rc_type); | |
| 1682 | 1682 | |
| 1683 | + if (dev->change_protocol) { | |
| 1683 | 1684 | rc = dev->change_protocol(dev, &rc_type); |
| 1684 | 1685 | if (rc < 0) |
| 1685 | 1686 | goto out_table; |
| 1686 | 1687 | dev->enabled_protocols = rc_type; |
| 1687 | 1688 | } |
| 1688 | 1689 | |
| 1690 | + if (dev->driver_type == RC_DRIVER_IR_RAW) | |
| 1691 | + ir_raw_load_modules(&rc_type); | |
| 1692 | + | |
| 1689 | 1693 | set_bit(EV_KEY, dev->input_dev->evbit); |
| 1690 | 1694 | set_bit(EV_REP, dev->input_dev->evbit); |
| 1691 | 1695 | set_bit(EV_MSC, dev->input_dev->evbit); |
| @@ -1777,12 +1781,6 @@ int rc_register_device(struct rc_dev *dev) · 85 unmodified lines | ||
| 1777 | 1781 | dev->input_name ?: "Unspecified device", path ?: "N/A"); |
| 1778 | 1782 | kfree(path); |
| 1779 | 1783 | |
| 1780 | − if (dev->driver_type != RC_DRIVER_IR_RAW_TX) { | |
| 1781 | − rc = rc_setup_rx_device(dev); | |
| 1782 | − if (rc) | |
| 1783 | − goto out_dev; | |
| 1784 | − } | |
| 1785 | − | |
| 1786 | 1784 | if (dev->driver_type == RC_DRIVER_IR_RAW || |
| 1787 | 1785 | dev->driver_type == RC_DRIVER_IR_RAW_TX) { |
| 1788 | 1786 | if (!raw_init) { |
| @@ -1791,7 +1789,13 @@ int rc_register_device(struct rc_dev *dev) · 2 unmodified lines | ||
| 1791 | 1789 | } |
| 1792 | 1790 | rc = ir_raw_event_register(dev); |
| 1793 | 1791 | if (rc < 0) |
| 1794 | − goto out_rx; | |
| 1792 | + goto out_dev; | |
| 1793 | + } | |
| 1794 | + | |
| 1795 | + if (dev->driver_type != RC_DRIVER_IR_RAW_TX) { | |
| 1796 | + rc = rc_setup_rx_device(dev); | |
| 1797 | + if (rc) | |
| 1798 | + goto out_raw; | |
| 1795 | 1799 | } |
| 1796 | 1800 | |
| 1797 | 1801 | /* Allow the RC sysfs nodes to be accessible */ |
| @@ -1803,8 +1807,8 @@ int rc_register_device(struct rc_dev *dev) · 5 unmodified lines | ||
| 1803 | 1807 | |
| 1804 | 1808 | return 0; |
| 1805 | 1809 | |
| 1806 | −out_rx: | |
| 1807 | − rc_free_rx_device(dev); | |
| 1810 | +out_raw: | |
| 1811 | + ir_raw_event_unregister(dev); | |
| 1808 | 1812 | out_dev: |
| 1809 | 1813 | device_del(&dev->dev); |
| 1810 | 1814 | out_unlock: |
| @@ -487,10 +487,69 @@ static void serial_ir_timeout(unsigned long arg) · 486 unmodified lines | ||
| 487 | 487 | ir_raw_event_handle(serial_ir.rcdev); |
| 488 | 488 | } |
| 489 | 489 | |
| 490 | +/* Needed by serial_ir_probe() */ | |
| 491 | +static int serial_ir_tx(struct rc_dev *dev, unsigned int *txbuf, | |
| 492 | + unsigned int count); | |
| 493 | +static int serial_ir_tx_duty_cycle(struct rc_dev *dev, u32 cycle); | |
| 494 | +static int serial_ir_tx_carrier(struct rc_dev *dev, u32 carrier); | |
| 495 | +static int serial_ir_open(struct rc_dev *rcdev); | |
| 496 | +static void serial_ir_close(struct rc_dev *rcdev); | |
| 497 | + | |
| 490 | 498 | static int serial_ir_probe(struct platform_device *dev) |
| 491 | 499 | { |
| 500 | + struct rc_dev *rcdev; | |
| 492 | 501 | int i, nlow, nhigh, result; |
| 493 | 502 | |
| 503 | + rcdev = devm_rc_allocate_device(&dev->dev, RC_DRIVER_IR_RAW); | |
| 504 | + if (!rcdev) | |
| 505 | + return -ENOMEM; | |
| 506 | + | |
| 507 | + if (hardware[type].send_pulse && hardware[type].send_space) | |
| 508 | + rcdev->tx_ir = serial_ir_tx; | |
| 509 | + if (hardware[type].set_send_carrier) | |
| 510 | + rcdev->s_tx_carrier = serial_ir_tx_carrier; | |
| 511 | + if (hardware[type].set_duty_cycle) | |
| 512 | + rcdev->s_tx_duty_cycle = serial_ir_tx_duty_cycle; | |
| 513 | + | |
| 514 | + switch (type) { | |
| 515 | + case IR_HOMEBREW: | |
| 516 | + rcdev->input_name = "Serial IR type home-brew"; | |
| 517 | + break; | |
| 518 | + case IR_IRDEO: | |
| 519 | + rcdev->input_name = "Serial IR type IRdeo"; | |
| 520 | + break; | |
| 521 | + case IR_IRDEO_REMOTE: | |
| 522 | + rcdev->input_name = "Serial IR type IRdeo remote"; | |
| 523 | + break; | |
| 524 | + case IR_ANIMAX: | |
| 525 | + rcdev->input_name = "Serial IR type AnimaX"; | |
| 526 | + break; | |
| 527 | + case IR_IGOR: | |
| 528 | + rcdev->input_name = "Serial IR type IgorPlug"; | |
| 529 | + break; | |
| 530 | + } | |
| 531 | + | |
| 532 | + rcdev->input_phys = KBUILD_MODNAME "/input0"; | |
| 533 | + rcdev->input_id.bustype = BUS_HOST; | |
| 534 | + rcdev->input_id.vendor = 0x0001; | |
| 535 | + rcdev->input_id.product = 0x0001; | |
| 536 | + rcdev->input_id.version = 0x0100; | |
| 537 | + rcdev->open = serial_ir_open; | |
| 538 | + rcdev->close = serial_ir_close; | |
| 539 | + rcdev->dev.parent = &serial_ir.pdev->dev; | |
| 540 | + rcdev->allowed_protocols = RC_BIT_ALL_IR_DECODER; | |
| 541 | + rcdev->driver_name = KBUILD_MODNAME; | |
| 542 | + rcdev->map_name = RC_MAP_RC6_MCE; | |
| 543 | + rcdev->min_timeout = 1; | |
| 544 | + rcdev->timeout = IR_DEFAULT_TIMEOUT; | |
| 545 | + rcdev->max_timeout = 10 * IR_DEFAULT_TIMEOUT; | |
| 546 | + rcdev->rx_resolution = 250000; | |
| 547 | + | |
| 548 | + serial_ir.rcdev = rcdev; | |
| 549 | + | |
| 550 | + setup_timer(&serial_ir.timeout_timer, serial_ir_timeout, | |
| 551 | + (unsigned long)&serial_ir); | |
| 552 | + | |
| 494 | 553 | result = devm_request_irq(&dev->dev, irq, serial_ir_irq_handler, |
| 495 | 554 | share_irq ? IRQF_SHARED : 0, |
| 496 | 555 | KBUILD_MODNAME, &hardware); |
| @@ -516,9 +575,6 @@ static int serial_ir_probe(struct platform_device *dev) · 19 unmodified lines | ||
| 516 | 575 | return -EBUSY; |
| 517 | 576 | } |
| 518 | 577 | |
| 519 | − setup_timer(&serial_ir.timeout_timer, serial_ir_timeout, | |
| 520 | − (unsigned long)&serial_ir); | |
| 521 | − | |
| 522 | 578 | result = hardware_init_port(); |
| 523 | 579 | if (result < 0) |
| 524 | 580 | return result; |
| @@ -552,7 +608,8 @@ static int serial_ir_probe(struct platform_device *dev) · 27 unmodified lines | ||
| 552 | 608 | sense ? "low" : "high"); |
| 553 | 609 | |
| 554 | 610 | dev_dbg(&dev->dev, "Interrupt %d, port %04x obtained\n", irq, io); |
| 555 | − return 0; | |
| 611 | + | |
| 612 | + return devm_rc_register_device(&dev->dev, rcdev); | |
| 556 | 613 | } |
| 557 | 614 | |
| 558 | 615 | static int serial_ir_open(struct rc_dev *rcdev) |
| @@ -723,7 +780,6 @@ static void serial_ir_exit(void) · 164 unmodified lines | ||
| 723 | 780 | |
| 724 | 781 | static int __init serial_ir_init_module(void) |
| 725 | 782 | { |
| 726 | − struct rc_dev *rcdev; | |
| 727 | 783 | int result; |
| 728 | 784 | |
| 729 | 785 | switch (type) { |
| @@ -754,63 +810,9 @@ static int __init serial_ir_init_module(void) · 24 unmodified lines | ||
| 754 | 810 | sense = !!sense; |
| 755 | 811 | |
| 756 | 812 | result = serial_ir_init(); |
| 757 | − if (result) | |
| 758 | − return result; | |
| 759 | − | |
| 760 | − rcdev = devm_rc_allocate_device(&serial_ir.pdev->dev, RC_DRIVER_IR_RAW); | |
| 761 | − if (!rcdev) { | |
| 762 | − result = -ENOMEM; | |
| 763 | − goto serial_cleanup; | |
| 764 | − } | |
| 765 | − | |
| 766 | − if (hardware[type].send_pulse && hardware[type].send_space) | |
| 767 | − rcdev->tx_ir = serial_ir_tx; | |
| 768 | − if (hardware[type].set_send_carrier) | |
| 769 | − rcdev->s_tx_carrier = serial_ir_tx_carrier; | |
| 770 | − if (hardware[type].set_duty_cycle) | |
| 771 | − rcdev->s_tx_duty_cycle = serial_ir_tx_duty_cycle; | |
| 772 | − | |
| 773 | − switch (type) { | |
| 774 | − case IR_HOMEBREW: | |
| 775 | − rcdev->input_name = "Serial IR type home-brew"; | |
| 776 | − break; | |
| 777 | − case IR_IRDEO: | |
| 778 | − rcdev->input_name = "Serial IR type IRdeo"; | |
| 779 | − break; | |
| 780 | − case IR_IRDEO_REMOTE: | |
| 781 | − rcdev->input_name = "Serial IR type IRdeo remote"; | |
| 782 | − break; | |
| 783 | − case IR_ANIMAX: | |
| 784 | − rcdev->input_name = "Serial IR type AnimaX"; | |
| 785 | − break; | |
| 786 | − case IR_IGOR: | |
| 787 | − rcdev->input_name = "Serial IR type IgorPlug"; | |
| 788 | − break; | |
| 789 | − } | |
| 790 | − | |
| 791 | − rcdev->input_phys = KBUILD_MODNAME "/input0"; | |
| 792 | − rcdev->input_id.bustype = BUS_HOST; | |
| 793 | − rcdev->input_id.vendor = 0x0001; | |
| 794 | − rcdev->input_id.product = 0x0001; | |
| 795 | − rcdev->input_id.version = 0x0100; | |
| 796 | − rcdev->open = serial_ir_open; | |
| 797 | − rcdev->close = serial_ir_close; | |
| 798 | − rcdev->dev.parent = &serial_ir.pdev->dev; | |
| 799 | − rcdev->allowed_protocols = RC_BIT_ALL_IR_DECODER; | |
| 800 | − rcdev->driver_name = KBUILD_MODNAME; | |
| 801 | − rcdev->map_name = RC_MAP_RC6_MCE; | |
| 802 | − rcdev->min_timeout = 1; | |
| 803 | − rcdev->timeout = IR_DEFAULT_TIMEOUT; | |
| 804 | − rcdev->max_timeout = 10 * IR_DEFAULT_TIMEOUT; | |
| 805 | − rcdev->rx_resolution = 250000; | |
| 806 | − | |
| 807 | − serial_ir.rcdev = rcdev; | |
| 808 | − | |
| 809 | − result = rc_register_device(rcdev); | |
| 810 | − | |
| 811 | 813 | if (!result) |
| 812 | 814 | return 0; |
| 813 | −serial_cleanup: | |
| 815 | + | |
| 814 | 816 | serial_ir_exit(); |
| 815 | 817 | return result; |
| 816 | 818 | } |
| @@ -818,7 +820,6 @@ serial_cleanup: · 1 unmodified line | ||
| 818 | 820 | static void __exit serial_ir_exit_module(void) |
| 819 | 821 | { |
| 820 | 822 | del_timer_sync(&serial_ir.timeout_timer); |
| 821 | − rc_unregister_device(serial_ir.rcdev); | |
| 822 | 823 | serial_ir_exit(); |
| 823 | 824 | } |
| 824 | 825 | |
| @@ -68,6 +68,7 @@ · 67 unmodified lines | ||
| 68 | 68 | struct dw2102_state { |
| 69 | 69 | u8 initialized; |
| 70 | 70 | u8 last_lock; |
| 71 | + u8 data[MAX_XFER_SIZE + 4]; | |
| 71 | 72 | struct i2c_client *i2c_client_demod; |
| 72 | 73 | struct i2c_client *i2c_client_tuner; |
| 73 | 74 | |
| @@ -661,62 +662,72 @@ static int su3000_i2c_transfer(struct i2c_adapter *adap, struct i2c_msg msg[], · 587 unmodified lines | ||
| 661 | 662 | int num) |
| 662 | 663 | { |
| 663 | 664 | struct dvb_usb_device *d = i2c_get_adapdata(adap); |
| 664 | − u8 obuf[0x40], ibuf[0x40]; | |
| 665 | + struct dw2102_state *state; | |
| 665 | 666 | |
| 666 | 667 | if (!d) |
| 667 | 668 | return -ENODEV; |
| 669 | + | |
| 670 | + state = d->priv; | |
| 671 | + | |
| 668 | 672 | if (mutex_lock_interruptible(&d->i2c_mutex) < 0) |
| 669 | 673 | return -EAGAIN; |
| 674 | + if (mutex_lock_interruptible(&d->data_mutex) < 0) { | |
| 675 | + mutex_unlock(&d->i2c_mutex); | |
| 676 | + return -EAGAIN; | |
| 677 | + } | |
| 670 | 678 | |
| 671 | 679 | switch (num) { |
| 672 | 680 | case 1: |
| 673 | 681 | switch (msg[0].addr) { |
| 674 | 682 | case SU3000_STREAM_CTRL: |
| 675 | − obuf[0] = msg[0].buf[0] + 0x36; | |
| 676 | − obuf[1] = 3; | |
| 677 | − obuf[2] = 0; | |
| 678 | − if (dvb_usb_generic_rw(d, obuf, 3, ibuf, 0, 0) < 0) | |
| 683 | + state->data[0] = msg[0].buf[0] + 0x36; | |
| 684 | + state->data[1] = 3; | |
| 685 | + state->data[2] = 0; | |
| 686 | + if (dvb_usb_generic_rw(d, state->data, 3, | |
| 687 | + state->data, 0, 0) < 0) | |
| 679 | 688 | err("i2c transfer failed."); |
| 680 | 689 | break; |
| 681 | 690 | case DW2102_RC_QUERY: |
| 682 | − obuf[0] = 0x10; | |
| 683 | − if (dvb_usb_generic_rw(d, obuf, 1, ibuf, 2, 0) < 0) | |
| 691 | + state->data[0] = 0x10; | |
| 692 | + if (dvb_usb_generic_rw(d, state->data, 1, | |
| 693 | + state->data, 2, 0) < 0) | |
| 684 | 694 | err("i2c transfer failed."); |
| 685 | − msg[0].buf[1] = ibuf[0]; | |
| 686 | − msg[0].buf[0] = ibuf[1]; | |
| 695 | + msg[0].buf[1] = state->data[0]; | |
| 696 | + msg[0].buf[0] = state->data[1]; | |
| 687 | 697 | break; |
| 688 | 698 | default: |
| 689 | 699 | /* always i2c write*/ |
| 690 | − obuf[0] = 0x08; | |
| 691 | − obuf[1] = msg[0].addr; | |
| 692 | − obuf[2] = msg[0].len; | |
| 700 | + state->data[0] = 0x08; | |
| 701 | + state->data[1] = msg[0].addr; | |
| 702 | + state->data[2] = msg[0].len; | |
| 693 | 703 | |
| 694 | − memcpy(&obuf[3], msg[0].buf, msg[0].len); | |
| 704 | + memcpy(&state->data[3], msg[0].buf, msg[0].len); | |
| 695 | 705 | |
| 696 | − if (dvb_usb_generic_rw(d, obuf, msg[0].len + 3, | |
| 697 | − ibuf, 1, 0) < 0) | |
| 706 | + if (dvb_usb_generic_rw(d, state->data, msg[0].len + 3, | |
| 707 | + state->data, 1, 0) < 0) | |
| 698 | 708 | err("i2c transfer failed."); |
| 699 | 709 | |
| 700 | 710 | } |
| 701 | 711 | break; |
| 702 | 712 | case 2: |
| 703 | 713 | /* always i2c read */ |
| 704 | − obuf[0] = 0x09; | |
| 705 | − obuf[1] = msg[0].len; | |
| 706 | − obuf[2] = msg[1].len; | |
| 707 | − obuf[3] = msg[0].addr; | |
| 708 | − memcpy(&obuf[4], msg[0].buf, msg[0].len); | |
| 709 | − | |
| 710 | − if (dvb_usb_generic_rw(d, obuf, msg[0].len + 4, | |
| 711 | − ibuf, msg[1].len + 1, 0) < 0) | |
| 714 | + state->data[0] = 0x09; | |
| 715 | + state->data[1] = msg[0].len; | |
| 716 | + state->data[2] = msg[1].len; | |
| 717 | + state->data[3] = msg[0].addr; | |
| 718 | + memcpy(&state->data[4], msg[0].buf, msg[0].len); | |
| 719 | + | |
| 720 | + if (dvb_usb_generic_rw(d, state->data, msg[0].len + 4, | |
| 721 | + state->data, msg[1].len + 1, 0) < 0) | |
| 712 | 722 | err("i2c transfer failed."); |
| 713 | 723 | |
| 714 | − memcpy(msg[1].buf, &ibuf[1], msg[1].len); | |
| 724 | + memcpy(msg[1].buf, &state->data[1], msg[1].len); | |
| 715 | 725 | break; |
| 716 | 726 | default: |
| 717 | 727 | warn("more than 2 i2c messages at a time is not handled yet."); |
| 718 | 728 | break; |
| 719 | 729 | } |
| 730 | + mutex_unlock(&d->data_mutex); | |
| 720 | 731 | mutex_unlock(&d->i2c_mutex); |
| 721 | 732 | return num; |
| 722 | 733 | } |
| @@ -844,17 +855,23 @@ static int su3000_streaming_ctrl(struct dvb_usb_adapter *adap, int onoff) · 121 unmodified lines | ||
| 844 | 855 | static int su3000_power_ctrl(struct dvb_usb_device *d, int i) |
| 845 | 856 | { |
| 846 | 857 | struct dw2102_state *state = (struct dw2102_state *)d->priv; |
| 847 | − u8 obuf[] = {0xde, 0}; | |
| 858 | + int ret = 0; | |
| 848 | 859 | |
| 849 | 860 | info("%s: %d, initialized %d", __func__, i, state->initialized); |
| 850 | 861 | |
| 851 | 862 | if (i && !state->initialized) { |
| 863 | + mutex_lock(&d->data_mutex); | |
| 864 | + | |
| 865 | + state->data[0] = 0xde; | |
| 866 | + state->data[1] = 0; | |
| 867 | + | |
| 852 | 868 | state->initialized = 1; |
| 853 | 869 | /* reset board */ |
| 854 | − return dvb_usb_generic_rw(d, obuf, 2, NULL, 0, 0); | |
| 870 | + ret = dvb_usb_generic_rw(d, state->data, 2, NULL, 0, 0); | |
| 871 | + mutex_unlock(&d->data_mutex); | |
| 855 | 872 | } |
| 856 | 873 | |
| 857 | − return 0; | |
| 874 | + return ret; | |
| 858 | 875 | } |
| 859 | 876 | |
| 860 | 877 | static int su3000_read_mac_address(struct dvb_usb_device *d, u8 mac[6]) |
| @@ -1309,49 +1326,57 @@ static int prof_7500_frontend_attach(struct dvb_usb_adapter *d) · 448 unmodified lines | ||
| 1309 | 1326 | return 0; |
| 1310 | 1327 | } |
| 1311 | 1328 | |
| 1312 | −static int su3000_frontend_attach(struct dvb_usb_adapter *d) | |
| 1329 | +static int su3000_frontend_attach(struct dvb_usb_adapter *adap) | |
| 1313 | 1330 | { |
| 1314 | − u8 obuf[3] = { 0xe, 0x80, 0 }; | |
| 1315 | − u8 ibuf[] = { 0 }; | |
| 1331 | + struct dvb_usb_device *d = adap->dev; | |
| 1332 | + struct dw2102_state *state = d->priv; | |
| 1333 | + | |
| 1334 | + mutex_lock(&d->data_mutex); | |
| 1335 | + | |
| 1336 | + state->data[0] = 0xe; | |
| 1337 | + state->data[1] = 0x80; | |
| 1338 | + state->data[2] = 0; | |
| 1316 | 1339 | |
| 1317 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1340 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1318 | 1341 | err("command 0x0e transfer failed."); |
| 1319 | 1342 | |
| 1320 | − obuf[0] = 0xe; | |
| 1321 | − obuf[1] = 0x02; | |
| 1322 | − obuf[2] = 1; | |
| 1343 | + state->data[0] = 0xe; | |
| 1344 | + state->data[1] = 0x02; | |
| 1345 | + state->data[2] = 1; | |
| 1323 | 1346 | |
| 1324 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1347 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1325 | 1348 | err("command 0x0e transfer failed."); |
| 1326 | 1349 | msleep(300); |
| 1327 | 1350 | |
| 1328 | − obuf[0] = 0xe; | |
| 1329 | − obuf[1] = 0x83; | |
| 1330 | − obuf[2] = 0; | |
| 1351 | + state->data[0] = 0xe; | |
| 1352 | + state->data[1] = 0x83; | |
| 1353 | + state->data[2] = 0; | |
| 1331 | 1354 | |
| 1332 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1355 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1333 | 1356 | err("command 0x0e transfer failed."); |
| 1334 | 1357 | |
| 1335 | − obuf[0] = 0xe; | |
| 1336 | − obuf[1] = 0x83; | |
| 1337 | − obuf[2] = 1; | |
| 1358 | + state->data[0] = 0xe; | |
| 1359 | + state->data[1] = 0x83; | |
| 1360 | + state->data[2] = 1; | |
| 1338 | 1361 | |
| 1339 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1362 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1340 | 1363 | err("command 0x0e transfer failed."); |
| 1341 | 1364 | |
| 1342 | − obuf[0] = 0x51; | |
| 1365 | + state->data[0] = 0x51; | |
| 1343 | 1366 | |
| 1344 | − if (dvb_usb_generic_rw(d->dev, obuf, 1, ibuf, 1, 0) < 0) | |
| 1367 | + if (dvb_usb_generic_rw(d, state->data, 1, state->data, 1, 0) < 0) | |
| 1345 | 1368 | err("command 0x51 transfer failed."); |
| 1346 | 1369 | |
| 1347 | − d->fe_adap[0].fe = dvb_attach(ds3000_attach, &su3000_ds3000_config, | |
| 1348 | − &d->dev->i2c_adap); | |
| 1349 | − if (d->fe_adap[0].fe == NULL) | |
| 1370 | + mutex_unlock(&d->data_mutex); | |
| 1371 | + | |
| 1372 | + adap->fe_adap[0].fe = dvb_attach(ds3000_attach, &su3000_ds3000_config, | |
| 1373 | + &d->i2c_adap); | |
| 1374 | + if (adap->fe_adap[0].fe == NULL) | |
| 1350 | 1375 | return -EIO; |
| 1351 | 1376 | |
| 1352 | − if (dvb_attach(ts2020_attach, d->fe_adap[0].fe, | |
| 1377 | + if (dvb_attach(ts2020_attach, adap->fe_adap[0].fe, | |
| 1353 | 1378 | &dw2104_ts2020_config, |
| 1354 | − &d->dev->i2c_adap)) { | |
| 1379 | + &d->i2c_adap)) { | |
| 1355 | 1380 | info("Attached DS3000/TS2020!"); |
| 1356 | 1381 | return 0; |
| 1357 | 1382 | } |
| @@ -1360,47 +1385,55 @@ static int su3000_frontend_attach(struct dvb_usb_adapter *d) · 2 unmodified lines | ||
| 1360 | 1385 | return -EIO; |
| 1361 | 1386 | } |
| 1362 | 1387 | |
| 1363 | −static int t220_frontend_attach(struct dvb_usb_adapter *d) | |
| 1388 | +static int t220_frontend_attach(struct dvb_usb_adapter *adap) | |
| 1364 | 1389 | { |
| 1365 | − u8 obuf[3] = { 0xe, 0x87, 0 }; | |
| 1366 | − u8 ibuf[] = { 0 }; | |
| 1390 | + struct dvb_usb_device *d = adap->dev; | |
| 1391 | + struct dw2102_state *state = d->priv; | |
| 1392 | + | |
| 1393 | + mutex_lock(&d->data_mutex); | |
| 1367 | 1394 | |
| 1368 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1395 | + state->data[0] = 0xe; | |
| 1396 | + state->data[1] = 0x87; | |
| 1397 | + state->data[2] = 0x0; | |
| 1398 | + | |
| 1399 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1369 | 1400 | err("command 0x0e transfer failed."); |
| 1370 | 1401 | |
| 1371 | − obuf[0] = 0xe; | |
| 1372 | − obuf[1] = 0x86; | |
| 1373 | − obuf[2] = 1; | |
| 1402 | + state->data[0] = 0xe; | |
| 1403 | + state->data[1] = 0x86; | |
| 1404 | + state->data[2] = 1; | |
| 1374 | 1405 | |
| 1375 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1406 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1376 | 1407 | err("command 0x0e transfer failed."); |
| 1377 | 1408 | |
| 1378 | − obuf[0] = 0xe; | |
| 1379 | − obuf[1] = 0x80; | |
| 1380 | − obuf[2] = 0; | |
| 1409 | + state->data[0] = 0xe; | |
| 1410 | + state->data[1] = 0x80; | |
| 1411 | + state->data[2] = 0; | |
| 1381 | 1412 | |
| 1382 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1413 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1383 | 1414 | err("command 0x0e transfer failed."); |
| 1384 | 1415 | |
| 1385 | 1416 | msleep(50); |
| 1386 | 1417 | |
| 1387 | − obuf[0] = 0xe; | |
| 1388 | − obuf[1] = 0x80; | |
| 1389 | − obuf[2] = 1; | |
| 1418 | + state->data[0] = 0xe; | |
| 1419 | + state->data[1] = 0x80; | |
| 1420 | + state->data[2] = 1; | |
| 1390 | 1421 | |
| 1391 | − if (dvb_usb_generic_rw(d->dev, obuf, 3, ibuf, 1, 0) < 0) | |
| 1422 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1392 | 1423 | err("command 0x0e transfer failed."); |
| 1393 | 1424 | |
| 1394 | − obuf[0] = 0x51; | |
| 1425 | + state->data[0] = 0x51; | |
| 1395 | 1426 | |
| 1396 | − if (dvb_usb_generic_rw(d->dev, obuf, 1, ibuf, 1, 0) < 0) | |
| 1427 | + if (dvb_usb_generic_rw(d, state->data, 1, state->data, 1, 0) < 0) | |
| 1397 | 1428 | err("command 0x51 transfer failed."); |
| 1398 | 1429 | |
| 1399 | − d->fe_adap[0].fe = dvb_attach(cxd2820r_attach, &cxd2820r_config, | |
| 1400 | − &d->dev->i2c_adap, NULL); | |
| 1401 | − if (d->fe_adap[0].fe != NULL) { | |
| 1402 | − if (dvb_attach(tda18271_attach, d->fe_adap[0].fe, 0x60, | |
| 1403 | − &d->dev->i2c_adap, &tda18271_config)) { | |
| 1430 | + mutex_unlock(&d->data_mutex); | |
| 1431 | + | |
| 1432 | + adap->fe_adap[0].fe = dvb_attach(cxd2820r_attach, &cxd2820r_config, | |
| 1433 | + &d->i2c_adap, NULL); | |
| 1434 | + if (adap->fe_adap[0].fe != NULL) { | |
| 1435 | + if (dvb_attach(tda18271_attach, adap->fe_adap[0].fe, 0x60, | |
| 1436 | + &d->i2c_adap, &tda18271_config)) { | |
| 1404 | 1437 | info("Attached TDA18271HD/CXD2820R!"); |
| 1405 | 1438 | return 0; |
| 1406 | 1439 | } |
| @@ -1410,23 +1443,30 @@ static int t220_frontend_attach(struct dvb_usb_adapter *d) · 3 unmodified lines | ||
| 1410 | 1443 | return -EIO; |
| 1411 | 1444 | } |
| 1412 | 1445 | |
| 1413 | −static int m88rs2000_frontend_attach(struct dvb_usb_adapter *d) | |
| 1446 | +static int m88rs2000_frontend_attach(struct dvb_usb_adapter *adap) | |
| 1414 | 1447 | { |
| 1415 | − u8 obuf[] = { 0x51 }; | |
| 1416 | − u8 ibuf[] = { 0 }; | |
| 1448 | + struct dvb_usb_device *d = adap->dev; | |
| 1449 | + struct dw2102_state *state = d->priv; | |
| 1450 | + | |
| 1451 | + mutex_lock(&d->data_mutex); | |
| 1417 | 1452 | |
| 1418 | − if (dvb_usb_generic_rw(d->dev, obuf, 1, ibuf, 1, 0) < 0) | |
| 1453 | + state->data[0] = 0x51; | |
| 1454 | + | |
| 1455 | + if (dvb_usb_generic_rw(d, state->data, 1, state->data, 1, 0) < 0) | |
| 1419 | 1456 | err("command 0x51 transfer failed."); |
| 1420 | 1457 | |
| 1421 | − d->fe_adap[0].fe = dvb_attach(m88rs2000_attach, &s421_m88rs2000_config, | |
| 1422 | − &d->dev->i2c_adap); | |
| 1458 | + mutex_unlock(&d->data_mutex); | |
| 1423 | 1459 | |
| 1424 | − if (d->fe_adap[0].fe == NULL) | |
| 1460 | + adap->fe_adap[0].fe = dvb_attach(m88rs2000_attach, | |
| 1461 | + &s421_m88rs2000_config, | |
| 1462 | + &d->i2c_adap); | |
| 1463 | + | |
| 1464 | + if (adap->fe_adap[0].fe == NULL) | |
| 1425 | 1465 | return -EIO; |
| 1426 | 1466 | |
| 1427 | − if (dvb_attach(ts2020_attach, d->fe_adap[0].fe, | |
| 1467 | + if (dvb_attach(ts2020_attach, adap->fe_adap[0].fe, | |
| 1428 | 1468 | &dw2104_ts2020_config, |
| 1429 | − &d->dev->i2c_adap)) { | |
| 1469 | + &d->i2c_adap)) { | |
| 1430 | 1470 | info("Attached RS2000/TS2020!"); |
| 1431 | 1471 | return 0; |
| 1432 | 1472 | } |
| @@ -1439,44 +1479,50 @@ static int tt_s2_4600_frontend_attach(struct dvb_usb_adapter *adap) · 6 unmodified lines | ||
| 1439 | 1479 | { |
| 1440 | 1480 | struct dvb_usb_device *d = adap->dev; |
| 1441 | 1481 | struct dw2102_state *state = d->priv; |
| 1442 | − u8 obuf[3] = { 0xe, 0x80, 0 }; | |
| 1443 | − u8 ibuf[] = { 0 }; | |
| 1444 | 1482 | struct i2c_adapter *i2c_adapter; |
| 1445 | 1483 | struct i2c_client *client; |
| 1446 | 1484 | struct i2c_board_info board_info; |
| 1447 | 1485 | struct m88ds3103_platform_data m88ds3103_pdata = {}; |
| 1448 | 1486 | struct ts2020_config ts2020_config = {}; |
| 1449 | 1487 | |
| 1450 | − if (dvb_usb_generic_rw(d, obuf, 3, ibuf, 1, 0) < 0) | |
| 1488 | + mutex_lock(&d->data_mutex); | |
| 1489 | + | |
| 1490 | + state->data[0] = 0xe; | |
| 1491 | + state->data[1] = 0x80; | |
| 1492 | + state->data[2] = 0x0; | |
| 1493 | + | |
| 1494 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1451 | 1495 | err("command 0x0e transfer failed."); |
| 1452 | 1496 | |
| 1453 | − obuf[0] = 0xe; | |
| 1454 | − obuf[1] = 0x02; | |
| 1455 | − obuf[2] = 1; | |
| 1497 | + state->data[0] = 0xe; | |
| 1498 | + state->data[1] = 0x02; | |
| 1499 | + state->data[2] = 1; | |
| 1456 | 1500 | |
| 1457 | − if (dvb_usb_generic_rw(d, obuf, 3, ibuf, 1, 0) < 0) | |
| 1501 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1458 | 1502 | err("command 0x0e transfer failed."); |
| 1459 | 1503 | msleep(300); |
| 1460 | 1504 | |
| 1461 | − obuf[0] = 0xe; | |
| 1462 | − obuf[1] = 0x83; | |
| 1463 | − obuf[2] = 0; | |
| 1505 | + state->data[0] = 0xe; | |
| 1506 | + state->data[1] = 0x83; | |
| 1507 | + state->data[2] = 0; | |
| 1464 | 1508 | |
| 1465 | − if (dvb_usb_generic_rw(d, obuf, 3, ibuf, 1, 0) < 0) | |
| 1509 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1466 | 1510 | err("command 0x0e transfer failed."); |
| 1467 | 1511 | |
| 1468 | − obuf[0] = 0xe; | |
| 1469 | − obuf[1] = 0x83; | |
| 1470 | − obuf[2] = 1; | |
| 1512 | + state->data[0] = 0xe; | |
| 1513 | + state->data[1] = 0x83; | |
| 1514 | + state->data[2] = 1; | |
| 1471 | 1515 | |
| 1472 | − if (dvb_usb_generic_rw(d, obuf, 3, ibuf, 1, 0) < 0) | |
| 1516 | + if (dvb_usb_generic_rw(d, state->data, 3, state->data, 1, 0) < 0) | |
| 1473 | 1517 | err("command 0x0e transfer failed."); |
| 1474 | 1518 | |
| 1475 | − obuf[0] = 0x51; | |
| 1519 | + state->data[0] = 0x51; | |
| 1476 | 1520 | |
| 1477 | − if (dvb_usb_generic_rw(d, obuf, 1, ibuf, 1, 0) < 0) | |
| 1521 | + if (dvb_usb_generic_rw(d, state->data, 1, state->data, 1, 0) < 0) | |
| 1478 | 1522 | err("command 0x51 transfer failed."); |
| 1479 | 1523 | |
| 1524 | + mutex_unlock(&d->data_mutex); | |
| 1525 | + | |
| 1480 | 1526 | /* attach demod */ |
| 1481 | 1527 | m88ds3103_pdata.clk = 27000000; |
| 1482 | 1528 | m88ds3103_pdata.i2c_wr_max = 33; |
| @@ -219,15 +219,20 @@ static int atomic_pte_lookup(struct vm_area_struct *vma, unsigned long vaddr, · 218 unmodified lines | ||
| 219 | 219 | int write, unsigned long *paddr, int *pageshift) |
| 220 | 220 | { |
| 221 | 221 | pgd_t *pgdp; |
| 222 | − pmd_t *pmdp; | |
| 222 | + p4d_t *p4dp; | |
| 223 | 223 | pud_t *pudp; |
| 224 | + pmd_t *pmdp; | |
| 224 | 225 | pte_t pte; |
| 225 | 226 | |
| 226 | 227 | pgdp = pgd_offset(vma->vm_mm, vaddr); |
| 227 | 228 | if (unlikely(pgd_none(*pgdp))) |
| 228 | 229 | goto err; |
| 229 | 230 | |
| 230 | − pudp = pud_offset(pgdp, vaddr); | |
| 231 | + p4dp = p4d_offset(pgdp, vaddr); | |
| 232 | + if (unlikely(p4d_none(*p4dp))) | |
| 233 | + goto err; | |
| 234 | + | |
| 235 | + pudp = pud_offset(p4dp, vaddr); | |
| 231 | 236 | if (unlikely(pud_none(*pudp))) |
| 232 | 237 | goto err; |
| 233 | 238 | |
| @@ -186,7 +186,7 @@ static inline int write_enable(struct spi_nor *nor) · 185 unmodified lines | ||
| 186 | 186 | } |
| 187 | 187 | |
| 188 | 188 | /* |
| 189 | − * Send write disble instruction to the chip. | |
| 189 | + * Send write disable instruction to the chip. | |
| 190 | 190 | */ |
| 191 | 191 | static inline int write_disable(struct spi_nor *nor) |
| 192 | 192 | { |
| @@ -2272,10 +2272,7 @@ static int xgbe_one_poll(struct napi_struct *napi, int budget) · 2271 unmodified lines | ||
| 2272 | 2272 | processed = xgbe_rx_poll(channel, budget); |
| 2273 | 2273 | |
| 2274 | 2274 | /* If we processed everything, we are done */ |
| 2275 | − if (processed < budget) { | |
| 2276 | − /* Turn off polling */ | |
| 2277 | − napi_complete_done(napi, processed); | |
| 2278 | − | |
| 2275 | + if ((processed < budget) && napi_complete_done(napi, processed)) { | |
| 2279 | 2276 | /* Enable Tx and Rx interrupts */ |
| 2280 | 2277 | if (pdata->channel_irq_mode) |
| 2281 | 2278 | xgbe_enable_rx_tx_int(pdata, channel); |
| @@ -2317,10 +2314,7 @@ static int xgbe_all_poll(struct napi_struct *napi, int budget) · 35 unmodified lines | ||
| 2317 | 2314 | } while ((processed < budget) && (processed != last_processed)); |
| 2318 | 2315 | |
| 2319 | 2316 | /* If we processed everything, we are done */ |
| 2320 | − if (processed < budget) { | |
| 2321 | − /* Turn off polling */ | |
| 2322 | − napi_complete_done(napi, processed); | |
| 2323 | − | |
| 2317 | + if ((processed < budget) && napi_complete_done(napi, processed)) { | |
| 2324 | 2318 | /* Enable Tx and Rx interrupts */ |
| 2325 | 2319 | xgbe_enable_rx_tx_ints(pdata); |
| 2326 | 2320 | } |
| @@ -213,9 +213,9 @@ void aq_pci_func_free_irqs(struct aq_pci_func_s *self) · 212 unmodified lines | ||
| 213 | 213 | if (!((1U << i) & self->msix_entry_mask)) |
| 214 | 214 | continue; |
| 215 | 215 | |
| 216 | − free_irq(pci_irq_vector(pdev, i), self->aq_vec[i]); | |
| 217 | 216 | if (pdev->msix_enabled) |
| 218 | 217 | irq_set_affinity_hint(pci_irq_vector(pdev, i), NULL); |
| 218 | + free_irq(pci_irq_vector(pdev, i), self->aq_vec[i]); | |
| 219 | 219 | self->msix_entry_mask &= ~(1U << i); |
| 220 | 220 | } |
| 221 | 221 | } |
| @@ -13292,17 +13292,15 @@ static int bnx2x_init_dev(struct bnx2x *bp, struct pci_dev *pdev, · 13291 unmodified lines | ||
| 13292 | 13292 | dev->vlan_features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | |
| 13293 | 13293 | NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6 | NETIF_F_HIGHDMA; |
| 13294 | 13294 | |
| 13295 | − /* VF with OLD Hypervisor or old PF do not support filtering */ | |
| 13296 | 13295 | if (IS_PF(bp)) { |
| 13297 | 13296 | if (chip_is_e1x) |
| 13298 | 13297 | bp->accept_any_vlan = true; |
| 13299 | 13298 | else |
| 13300 | 13299 | dev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; |
| 13301 | −#ifdef CONFIG_BNX2X_SRIOV | |
| 13302 | − } else if (bp->acquire_resp.pfdev_info.pf_cap & PFVF_CAP_VLAN_FILTER) { | |
| 13303 | − dev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; | |
| 13304 | −#endif | |
| 13305 | 13300 | } |
| 13301 | + /* For VF we'll know whether to enable VLAN filtering after | |
| 13302 | + * getting a response to CHANNEL_TLV_ACQUIRE from PF. | |
| 13303 | + */ | |
| 13306 | 13304 | |
| 13307 | 13305 | dev->features |= dev->hw_features | NETIF_F_HW_VLAN_CTAG_RX; |
| 13308 | 13306 | dev->features |= NETIF_F_HIGHDMA; |
| @@ -13738,7 +13736,7 @@ static int bnx2x_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) · 429 unmodified lines | ||
| 13738 | 13736 | if (!netif_running(bp->dev)) { |
| 13739 | 13737 | DP(BNX2X_MSG_PTP, |
| 13740 | 13738 | "PTP adjfreq called while the interface is down\n"); |
| 13741 | − return -EFAULT; | |
| 13739 | + return -ENETDOWN; | |
| 13742 | 13740 | } |
| 13743 | 13741 | |
| 13744 | 13742 | if (ppb < 0) { |
| @@ -13797,6 +13795,12 @@ static int bnx2x_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) · 52 unmodified lines | ||
| 13797 | 13795 | { |
| 13798 | 13796 | struct bnx2x *bp = container_of(ptp, struct bnx2x, ptp_clock_info); |
| 13799 | 13797 | |
| 13798 | + if (!netif_running(bp->dev)) { | |
| 13799 | + DP(BNX2X_MSG_PTP, | |
| 13800 | + "PTP adjtime called while the interface is down\n"); | |
| 13801 | + return -ENETDOWN; | |
| 13802 | + } | |
| 13803 | + | |
| 13800 | 13804 | DP(BNX2X_MSG_PTP, "PTP adjtime called, delta = %llx\n", delta); |
| 13801 | 13805 | |
| 13802 | 13806 | timecounter_adjtime(&bp->timecounter, delta); |
| @@ -13809,6 +13813,12 @@ static int bnx2x_ptp_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts) · 6 unmodified lines | ||
| 13809 | 13813 | struct bnx2x *bp = container_of(ptp, struct bnx2x, ptp_clock_info); |
| 13810 | 13814 | u64 ns; |
| 13811 | 13815 | |
| 13816 | + if (!netif_running(bp->dev)) { | |
| 13817 | + DP(BNX2X_MSG_PTP, | |
| 13818 | + "PTP gettime called while the interface is down\n"); | |
| 13819 | + return -ENETDOWN; | |
| 13820 | + } | |
| 13821 | + | |
| 13812 | 13822 | ns = timecounter_read(&bp->timecounter); |
| 13813 | 13823 | |
| 13814 | 13824 | DP(BNX2X_MSG_PTP, "PTP gettime called, ns = %llu\n", ns); |
| @@ -13824,6 +13834,12 @@ static int bnx2x_ptp_settime(struct ptp_clock_info *ptp, · 9 unmodified lines | ||
| 13824 | 13834 | struct bnx2x *bp = container_of(ptp, struct bnx2x, ptp_clock_info); |
| 13825 | 13835 | u64 ns; |
| 13826 | 13836 | |
| 13837 | + if (!netif_running(bp->dev)) { | |
| 13838 | + DP(BNX2X_MSG_PTP, | |
| 13839 | + "PTP settime called while the interface is down\n"); | |
| 13840 | + return -ENETDOWN; | |
| 13841 | + } | |
| 13842 | + | |
| 13827 | 13843 | ns = timespec64_to_ns(ts); |
| 13828 | 13844 | |
| 13829 | 13845 | DP(BNX2X_MSG_PTP, "PTP settime called, ns = %llu\n", ns); |
| @@ -13991,6 +14007,14 @@ static int bnx2x_init_one(struct pci_dev *pdev, · 161 unmodified lines | ||
| 13991 | 14007 | rc = bnx2x_vfpf_acquire(bp, tx_count, rx_count); |
| 13992 | 14008 | if (rc) |
| 13993 | 14009 | goto init_one_freemem; |
| 14010 | + | |
| 14011 | +#ifdef CONFIG_BNX2X_SRIOV | |
| 14012 | + /* VF with OLD Hypervisor or old PF do not support filtering */ | |
| 14013 | + if (bp->acquire_resp.pfdev_info.pf_cap & PFVF_CAP_VLAN_FILTER) { | |
| 14014 | + dev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; | |
| 14015 | + dev->features |= NETIF_F_HW_VLAN_CTAG_FILTER; | |
| 14016 | + } | |
| 14017 | +#endif | |
| 13994 | 14018 | } |
| 13995 | 14019 | |
| 13996 | 14020 | /* Enable SRIOV if capability found in configuration space */ |
| @@ -434,7 +434,9 @@ static int bnx2x_vf_mac_vlan_config(struct bnx2x *bp, · 433 unmodified lines | ||
| 434 | 434 | |
| 435 | 435 | /* Add/Remove the filter */ |
| 436 | 436 | rc = bnx2x_config_vlan_mac(bp, &ramrod); |
| 437 | − if (rc && rc != -EEXIST) { | |
| 437 | + if (rc == -EEXIST) | |
| 438 | + return 0; | |
| 439 | + if (rc) { | |
| 438 | 440 | BNX2X_ERR("Failed to %s %s\n", |
| 439 | 441 | filter->add ? "add" : "delete", |
| 440 | 442 | (filter->type == BNX2X_VF_FILTER_VLAN_MAC) ? |
| @@ -444,6 +446,8 @@ static int bnx2x_vf_mac_vlan_config(struct bnx2x *bp, · 3 unmodified lines | ||
| 444 | 446 | return rc; |
| 445 | 447 | } |
| 446 | 448 | |
| 449 | + filter->applied = true; | |
| 450 | + | |
| 447 | 451 | return 0; |
| 448 | 452 | } |
| 449 | 453 | |
| @@ -469,8 +473,10 @@ int bnx2x_vf_mac_vlan_config_list(struct bnx2x *bp, struct bnx2x_virtf *vf, · 19 unmodified lines | ||
| 469 | 473 | /* Rollback if needed */ |
| 470 | 474 | if (i != filters->count) { |
| 471 | 475 | BNX2X_ERR("Managed only %d/%d filters - rolling back\n", |
| 472 | − i, filters->count + 1); | |
| 476 | + i, filters->count); | |
| 473 | 477 | while (--i >= 0) { |
| 478 | + if (!filters->filters[i].applied) | |
| 479 | + continue; | |
| 474 | 480 | filters->filters[i].add = !filters->filters[i].add; |
| 475 | 481 | bnx2x_vf_mac_vlan_config(bp, vf, qid, |
| 476 | 482 | &filters->filters[i], |
| @@ -1899,7 +1905,8 @@ void bnx2x_iov_adjust_stats_req(struct bnx2x *bp) · 1422 unmodified lines | ||
| 1899 | 1905 | continue; |
| 1900 | 1906 | } |
| 1901 | 1907 | |
| 1902 | − DP(BNX2X_MSG_IOV, "add addresses for vf %d\n", vf->abs_vfid); | |
| 1908 | + DP_AND((BNX2X_MSG_IOV | BNX2X_MSG_STATS), | |
| 1909 | + "add addresses for vf %d\n", vf->abs_vfid); | |
| 1903 | 1910 | for_each_vfq(vf, j) { |
| 1904 | 1911 | struct bnx2x_vf_queue *rxq = vfq_get(vf, j); |
| 1905 | 1912 | |
| @@ -1920,11 +1927,12 @@ void bnx2x_iov_adjust_stats_req(struct bnx2x *bp) · 14 unmodified lines | ||
| 1920 | 1927 | cpu_to_le32(U64_HI(q_stats_addr)); |
| 1921 | 1928 | cur_query_entry->address.lo = |
| 1922 | 1929 | cpu_to_le32(U64_LO(q_stats_addr)); |
| 1923 | − DP(BNX2X_MSG_IOV, | |
| 1924 | − "added address %x %x for vf %d queue %d client %d\n", | |
| 1925 | − cur_query_entry->address.hi, | |
| 1926 | − cur_query_entry->address.lo, cur_query_entry->funcID, | |
| 1927 | − j, cur_query_entry->index); | |
| 1930 | + DP_AND((BNX2X_MSG_IOV | BNX2X_MSG_STATS), | |
| 1931 | + "added address %x %x for vf %d queue %d client %d\n", | |
| 1932 | + cur_query_entry->address.hi, | |
| 1933 | + cur_query_entry->address.lo, | |
| 1934 | + cur_query_entry->funcID, | |
| 1935 | + j, cur_query_entry->index); | |
| 1928 | 1936 | cur_query_entry++; |
| 1929 | 1937 | cur_data_offset += sizeof(struct per_queue_stats); |
| 1930 | 1938 | stats_count++; |
| @@ -114,6 +114,7 @@ struct bnx2x_vf_mac_vlan_filter { · 113 unmodified lines | ||
| 114 | 114 | (BNX2X_VF_FILTER_MAC | BNX2X_VF_FILTER_VLAN) /*shortcut*/ |
| 115 | 115 | |
| 116 | 116 | bool add; |
| 117 | + bool applied; | |
| 117 | 118 | u8 *mac; |
| 118 | 119 | u16 vid; |
| 119 | 120 | }; |
| @@ -868,7 +868,7 @@ int bnx2x_vfpf_set_mcast(struct net_device *dev) · 867 unmodified lines | ||
| 868 | 868 | struct bnx2x *bp = netdev_priv(dev); |
| 869 | 869 | struct vfpf_set_q_filters_tlv *req = &bp->vf2pf_mbox->req.set_q_filters; |
| 870 | 870 | struct pfvf_general_resp_tlv *resp = &bp->vf2pf_mbox->resp.general_resp; |
| 871 | − int rc, i = 0; | |
| 871 | + int rc = 0, i = 0; | |
| 872 | 872 | struct netdev_hw_addr *ha; |
| 873 | 873 | |
| 874 | 874 | if (bp->state != BNX2X_STATE_OPEN) { |
| @@ -883,6 +883,15 @@ int bnx2x_vfpf_set_mcast(struct net_device *dev) · 8 unmodified lines | ||
| 883 | 883 | /* Get Rx mode requested */ |
| 884 | 884 | DP(NETIF_MSG_IFUP, "dev->flags = %x\n", dev->flags); |
| 885 | 885 | |
| 886 | + /* We support PFVF_MAX_MULTICAST_PER_VF mcast addresses tops */ | |
| 887 | + if (netdev_mc_count(dev) > PFVF_MAX_MULTICAST_PER_VF) { | |
| 888 | + DP(NETIF_MSG_IFUP, | |
| 889 | + "VF supports not more than %d multicast MAC addresses\n", | |
| 890 | + PFVF_MAX_MULTICAST_PER_VF); | |
| 891 | + rc = -EINVAL; | |
| 892 | + goto out; | |
| 893 | + } | |
| 894 | + | |
| 886 | 895 | netdev_for_each_mc_addr(ha, dev) { |
| 887 | 896 | DP(NETIF_MSG_IFUP, "Adding mcast MAC: %pM\n", |
| 888 | 897 | bnx2x_mc_addr(ha)); |
| @@ -890,16 +899,6 @@ int bnx2x_vfpf_set_mcast(struct net_device *dev) · 1 unmodified line | ||
| 890 | 899 | i++; |
| 891 | 900 | } |
| 892 | 901 | |
| 893 | − /* We support four PFVF_MAX_MULTICAST_PER_VF mcast | |
| 894 | − * addresses tops | |
| 895 | − */ | |
| 896 | − if (i >= PFVF_MAX_MULTICAST_PER_VF) { | |
| 897 | − DP(NETIF_MSG_IFUP, | |
| 898 | − "VF supports not more than %d multicast MAC addresses\n", | |
| 899 | − PFVF_MAX_MULTICAST_PER_VF); | |
| 900 | − return -EINVAL; | |
| 901 | − } | |
| 902 | − | |
| 903 | 902 | req->n_multicast = i; |
| 904 | 903 | req->flags |= VFPF_SET_Q_FILTERS_MULTICAST_CHANGED; |
| 905 | 904 | req->vf_qid = 0; |
| @@ -924,7 +923,7 @@ int bnx2x_vfpf_set_mcast(struct net_device *dev) · 18 unmodified lines | ||
| 924 | 923 | out: |
| 925 | 924 | bnx2x_vfpf_finalize(bp, &req->first_tlv); |
| 926 | 925 | |
| 927 | − return 0; | |
| 926 | + return rc; | |
| 928 | 927 | } |
| 929 | 928 | |
| 930 | 929 | /* request pf to add a vlan for the vf */ |
| @@ -1778,6 +1777,23 @@ static int bnx2x_vf_mbx_qfilters(struct bnx2x *bp, struct bnx2x_virtf *vf) · 847 unmodified lines | ||
| 1778 | 1777 | goto op_err; |
| 1779 | 1778 | } |
| 1780 | 1779 | |
| 1780 | + /* build vlan list */ | |
| 1781 | + fl = NULL; | |
| 1782 | + | |
| 1783 | + rc = bnx2x_vf_mbx_macvlan_list(bp, vf, msg, &fl, | |
| 1784 | + VFPF_VLAN_FILTER); | |
| 1785 | + if (rc) | |
| 1786 | + goto op_err; | |
| 1787 | + | |
| 1788 | + if (fl) { | |
| 1789 | + /* set vlan list */ | |
| 1790 | + rc = bnx2x_vf_mac_vlan_config_list(bp, vf, fl, | |
| 1791 | + msg->vf_qid, | |
| 1792 | + false); | |
| 1793 | + if (rc) | |
| 1794 | + goto op_err; | |
| 1795 | + } | |
| 1796 | + | |
| 1781 | 1797 | } |
| 1782 | 1798 | |
| 1783 | 1799 | if (msg->flags & VFPF_SET_Q_FILTERS_RX_MASK_CHANGED) { |
| @@ -4465,6 +4465,10 @@ static int bnxt_hwrm_func_qcfg(struct bnxt *bp) · 4464 unmodified lines | ||
| 4465 | 4465 | vf->vlan = le16_to_cpu(resp->vlan) & VLAN_VID_MASK; |
| 4466 | 4466 | } |
| 4467 | 4467 | #endif |
| 4468 | + if (BNXT_PF(bp) && (le16_to_cpu(resp->flags) & | |
| 4469 | + FUNC_QCFG_RESP_FLAGS_FW_DCBX_AGENT_ENABLED)) | |
| 4470 | + bp->flags |= BNXT_FLAG_FW_LLDP_AGENT; | |
| 4471 | + | |
| 4468 | 4472 | switch (resp->port_partition_type) { |
| 4469 | 4473 | case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_0: |
| 4470 | 4474 | case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_5: |
| @@ -5507,8 +5511,9 @@ static int bnxt_hwrm_phy_qcaps(struct bnxt *bp) · 1036 unmodified lines | ||
| 5507 | 5511 | bp->lpi_tmr_hi = le32_to_cpu(resp->valid_tx_lpi_timer_high) & |
| 5508 | 5512 | PORT_PHY_QCAPS_RESP_TX_LPI_TIMER_HIGH_MASK; |
| 5509 | 5513 | } |
| 5510 | − link_info->support_auto_speeds = | |
| 5511 | − le16_to_cpu(resp->supported_speeds_auto_mode); | |
| 5514 | + if (resp->supported_speeds_auto_mode) | |
| 5515 | + link_info->support_auto_speeds = | |
| 5516 | + le16_to_cpu(resp->supported_speeds_auto_mode); | |
| 5512 | 5517 | |
| 5513 | 5518 | hwrm_phy_qcaps_exit: |
| 5514 | 5519 | mutex_unlock(&bp->hwrm_cmd_lock); |
| @@ -6495,8 +6500,14 @@ static void bnxt_reset_task(struct bnxt *bp, bool silent) · 980 unmodified lines | ||
| 6495 | 6500 | if (!silent) |
| 6496 | 6501 | bnxt_dbg_dump_states(bp); |
| 6497 | 6502 | if (netif_running(bp->dev)) { |
| 6503 | + int rc; | |
| 6504 | + | |
| 6505 | + if (!silent) | |
| 6506 | + bnxt_ulp_stop(bp); | |
| 6498 | 6507 | bnxt_close_nic(bp, false, false); |
| 6499 | − bnxt_open_nic(bp, false, false); | |
| 6508 | + rc = bnxt_open_nic(bp, false, false); | |
| 6509 | + if (!silent && !rc) | |
| 6510 | + bnxt_ulp_start(bp); | |
| 6500 | 6511 | } |
| 6501 | 6512 | } |
| 6502 | 6513 | |
| @@ -7444,6 +7455,10 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) · 941 unmodified lines | ||
| 7444 | 7455 | if (rc) |
| 7445 | 7456 | goto init_err_pci_clean; |
| 7446 | 7457 | |
| 7458 | + rc = bnxt_hwrm_func_reset(bp); | |
| 7459 | + if (rc) | |
| 7460 | + goto init_err_pci_clean; | |
| 7461 | + | |
| 7447 | 7462 | bnxt_hwrm_fw_set_time(bp); |
| 7448 | 7463 | |
| 7449 | 7464 | dev->hw_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG | |
| @@ -7554,10 +7569,6 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) · 104 unmodified lines | ||
| 7554 | 7569 | if (rc) |
| 7555 | 7570 | goto init_err_pci_clean; |
| 7556 | 7571 | |
| 7557 | − rc = bnxt_hwrm_func_reset(bp); | |
| 7558 | − if (rc) | |
| 7559 | − goto init_err_pci_clean; | |
| 7560 | − | |
| 7561 | 7572 | rc = bnxt_init_int_mode(bp); |
| 7562 | 7573 | if (rc) |
| 7563 | 7574 | goto init_err_pci_clean; |
| @@ -993,6 +993,7 @@ struct bnxt { · 992 unmodified lines | ||
| 993 | 993 | BNXT_FLAG_ROCEV2_CAP) |
| 994 | 994 | #define BNXT_FLAG_NO_AGG_RINGS 0x20000 |
| 995 | 995 | #define BNXT_FLAG_RX_PAGE_MODE 0x40000 |
| 996 | + #define BNXT_FLAG_FW_LLDP_AGENT 0x80000 | |
| 996 | 997 | #define BNXT_FLAG_CHIP_NITRO_A0 0x1000000 |
| 997 | 998 | |
| 998 | 999 | #define BNXT_FLAG_ALL_CONFIG_FEATS (BNXT_FLAG_TPA | \ |
| @@ -474,7 +474,7 @@ void bnxt_dcb_init(struct bnxt *bp) · 473 unmodified lines | ||
| 474 | 474 | return; |
| 475 | 475 | |
| 476 | 476 | bp->dcbx_cap = DCB_CAP_DCBX_VER_IEEE; |
| 477 | − if (BNXT_PF(bp)) | |
| 477 | + if (BNXT_PF(bp) && !(bp->flags & BNXT_FLAG_FW_LLDP_AGENT)) | |
| 478 | 478 | bp->dcbx_cap |= DCB_CAP_DCBX_HOST; |
| 479 | 479 | else |
| 480 | 480 | bp->dcbx_cap |= DCB_CAP_DCBX_LLD_MANAGED; |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | /* |
| 2 | 2 | * Broadcom GENET (Gigabit Ethernet) controller driver |
| 3 | 3 | * |
| 4 | − * Copyright (c) 2014 Broadcom Corporation | |
| 4 | + * Copyright (c) 2014-2017 Broadcom | |
| 5 | 5 | * |
| 6 | 6 | * This program is free software; you can redistribute it and/or modify |
| 7 | 7 | * it under the terms of the GNU General Public License version 2 as |
| @@ -450,6 +450,22 @@ static inline void bcmgenet_rdma_ring_writel(struct bcmgenet_priv *priv, · 442 unmodified lines | ||
| 450 | 450 | genet_dma_ring_regs[r]); |
| 451 | 451 | } |
| 452 | 452 | |
| 453 | +static int bcmgenet_begin(struct net_device *dev) | |
| 454 | +{ | |
| 455 | + struct bcmgenet_priv *priv = netdev_priv(dev); | |
| 456 | + | |
| 457 | + /* Turn on the clock */ | |
| 458 | + return clk_prepare_enable(priv->clk); | |
| 459 | +} | |
| 460 | + | |
| 461 | +static void bcmgenet_complete(struct net_device *dev) | |
| 462 | +{ | |
| 463 | + struct bcmgenet_priv *priv = netdev_priv(dev); | |
| 464 | + | |
| 465 | + /* Turn off the clock */ | |
| 466 | + clk_disable_unprepare(priv->clk); | |
| 467 | +} | |
| 468 | + | |
| 453 | 469 | static int bcmgenet_get_link_ksettings(struct net_device *dev, |
| 454 | 470 | struct ethtool_link_ksettings *cmd) |
| 455 | 471 | { |
| @@ -778,8 +794,9 @@ static const struct bcmgenet_stats bcmgenet_gstrings_stats[] = { · 322 unmodified lines | ||
| 778 | 794 | STAT_GENET_RUNT("rx_runt_bytes", mib.rx_runt_bytes), |
| 779 | 795 | /* Misc UniMAC counters */ |
| 780 | 796 | STAT_GENET_MISC("rbuf_ovflow_cnt", mib.rbuf_ovflow_cnt, |
| 781 | − UMAC_RBUF_OVFL_CNT), | |
| 782 | − STAT_GENET_MISC("rbuf_err_cnt", mib.rbuf_err_cnt, UMAC_RBUF_ERR_CNT), | |
| 797 | + UMAC_RBUF_OVFL_CNT_V1), | |
| 798 | + STAT_GENET_MISC("rbuf_err_cnt", mib.rbuf_err_cnt, | |
| 799 | + UMAC_RBUF_ERR_CNT_V1), | |
| 783 | 800 | STAT_GENET_MISC("mdf_err_cnt", mib.mdf_err_cnt, UMAC_MDF_ERR_CNT), |
| 784 | 801 | STAT_GENET_SOFT_MIB("alloc_rx_buff_failed", mib.alloc_rx_buff_failed), |
| 785 | 802 | STAT_GENET_SOFT_MIB("rx_dma_failed", mib.rx_dma_failed), |
| @@ -821,6 +838,45 @@ static void bcmgenet_get_strings(struct net_device *dev, u32 stringset, · 35 unmodified lines | ||
| 821 | 838 | } |
| 822 | 839 | } |
| 823 | 840 | |
| 841 | +static u32 bcmgenet_update_stat_misc(struct bcmgenet_priv *priv, u16 offset) | |
| 842 | +{ | |
| 843 | + u16 new_offset; | |
| 844 | + u32 val; | |
| 845 | + | |
| 846 | + switch (offset) { | |
| 847 | + case UMAC_RBUF_OVFL_CNT_V1: | |
| 848 | + if (GENET_IS_V2(priv)) | |
| 849 | + new_offset = RBUF_OVFL_CNT_V2; | |
| 850 | + else | |
| 851 | + new_offset = RBUF_OVFL_CNT_V3PLUS; | |
| 852 | + | |
| 853 | + val = bcmgenet_rbuf_readl(priv, new_offset); | |
| 854 | + /* clear if overflowed */ | |
| 855 | + if (val == ~0) | |
| 856 | + bcmgenet_rbuf_writel(priv, 0, new_offset); | |
| 857 | + break; | |
| 858 | + case UMAC_RBUF_ERR_CNT_V1: | |
| 859 | + if (GENET_IS_V2(priv)) | |
| 860 | + new_offset = RBUF_ERR_CNT_V2; | |
| 861 | + else | |
| 862 | + new_offset = RBUF_ERR_CNT_V3PLUS; | |
| 863 | + | |
| 864 | + val = bcmgenet_rbuf_readl(priv, new_offset); | |
| 865 | + /* clear if overflowed */ | |
| 866 | + if (val == ~0) | |
| 867 | + bcmgenet_rbuf_writel(priv, 0, new_offset); | |
| 868 | + break; | |
| 869 | + default: | |
| 870 | + val = bcmgenet_umac_readl(priv, offset); | |
| 871 | + /* clear if overflowed */ | |
| 872 | + if (val == ~0) | |
| 873 | + bcmgenet_umac_writel(priv, 0, offset); | |
| 874 | + break; | |
| 875 | + } | |
| 876 | + | |
| 877 | + return val; | |
| 878 | +} | |
| 879 | + | |
| 824 | 880 | static void bcmgenet_update_mib_counters(struct bcmgenet_priv *priv) |
| 825 | 881 | { |
| 826 | 882 | int i, j = 0; |
| @@ -836,19 +892,28 @@ static void bcmgenet_update_mib_counters(struct bcmgenet_priv *priv) · 9 unmodified lines | ||
| 836 | 892 | case BCMGENET_STAT_NETDEV: |
| 837 | 893 | case BCMGENET_STAT_SOFT: |
| 838 | 894 | continue; |
| 839 | − case BCMGENET_STAT_MIB_RX: | |
| 840 | − case BCMGENET_STAT_MIB_TX: | |
| 841 | 895 | case BCMGENET_STAT_RUNT: |
| 842 | − if (s->type != BCMGENET_STAT_MIB_RX) | |
| 843 | − offset = BCMGENET_STAT_OFFSET; | |
| 896 | + offset += BCMGENET_STAT_OFFSET; | |
| 897 | + /* fall through */ | |
| 898 | + case BCMGENET_STAT_MIB_TX: | |
| 899 | + offset += BCMGENET_STAT_OFFSET; | |
| 900 | + /* fall through */ | |
| 901 | + case BCMGENET_STAT_MIB_RX: | |
| 844 | 902 | val = bcmgenet_umac_readl(priv, |
| 845 | 903 | UMAC_MIB_START + j + offset); |
| 904 | + offset = 0; /* Reset Offset */ | |
| 846 | 905 | break; |
| 847 | 906 | case BCMGENET_STAT_MISC: |
| 848 | − val = bcmgenet_umac_readl(priv, s->reg_offset); | |
| 849 | − /* clear if overflowed */ | |
| 850 | − if (val == ~0) | |
| 851 | − bcmgenet_umac_writel(priv, 0, s->reg_offset); | |
| 907 | + if (GENET_IS_V1(priv)) { | |
| 908 | + val = bcmgenet_umac_readl(priv, s->reg_offset); | |
| 909 | + /* clear if overflowed */ | |
| 910 | + if (val == ~0) | |
| 911 | + bcmgenet_umac_writel(priv, 0, | |
| 912 | + s->reg_offset); | |
| 913 | + } else { | |
| 914 | + val = bcmgenet_update_stat_misc(priv, | |
| 915 | + s->reg_offset); | |
| 916 | + } | |
| 852 | 917 | break; |
| 853 | 918 | } |
| 854 | 919 | |
| @@ -973,6 +1038,8 @@ static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_eee *e) · 118 unmodified lines | ||
| 973 | 1038 | |
| 974 | 1039 | /* standard ethtool support functions. */ |
| 975 | 1040 | static const struct ethtool_ops bcmgenet_ethtool_ops = { |
| 1041 | + .begin = bcmgenet_begin, | |
| 1042 | + .complete = bcmgenet_complete, | |
| 976 | 1043 | .get_strings = bcmgenet_get_strings, |
| 977 | 1044 | .get_sset_count = bcmgenet_get_sset_count, |
| 978 | 1045 | .get_ethtool_stats = bcmgenet_get_ethtool_stats, |
| @@ -1167,7 +1234,6 @@ static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev, · 188 unmodified lines | ||
| 1167 | 1234 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1168 | 1235 | struct device *kdev = &priv->pdev->dev; |
| 1169 | 1236 | struct enet_cb *tx_cb_ptr; |
| 1170 | − struct netdev_queue *txq; | |
| 1171 | 1237 | unsigned int pkts_compl = 0; |
| 1172 | 1238 | unsigned int bytes_compl = 0; |
| 1173 | 1239 | unsigned int c_index; |
| @@ -1219,13 +1285,8 @@ static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev, · 45 unmodified lines | ||
| 1219 | 1285 | dev->stats.tx_packets += pkts_compl; |
| 1220 | 1286 | dev->stats.tx_bytes += bytes_compl; |
| 1221 | 1287 | |
| 1222 | − txq = netdev_get_tx_queue(dev, ring->queue); | |
| 1223 | − netdev_tx_completed_queue(txq, pkts_compl, bytes_compl); | |
| 1224 | − | |
| 1225 | − if (ring->free_bds > (MAX_SKB_FRAGS + 1)) { | |
| 1226 | − if (netif_tx_queue_stopped(txq)) | |
| 1227 | − netif_tx_wake_queue(txq); | |
| 1228 | − } | |
| 1288 | + netdev_tx_completed_queue(netdev_get_tx_queue(dev, ring->queue), | |
| 1289 | + pkts_compl, bytes_compl); | |
| 1229 | 1290 | |
| 1230 | 1291 | return pkts_compl; |
| 1231 | 1292 | } |
| @@ -1248,8 +1309,16 @@ static int bcmgenet_tx_poll(struct napi_struct *napi, int budget) · 16 unmodified lines | ||
| 1248 | 1309 | struct bcmgenet_tx_ring *ring = |
| 1249 | 1310 | container_of(napi, struct bcmgenet_tx_ring, napi); |
| 1250 | 1311 | unsigned int work_done = 0; |
| 1312 | + struct netdev_queue *txq; | |
| 1313 | + unsigned long flags; | |
| 1251 | 1314 | |
| 1252 | − work_done = bcmgenet_tx_reclaim(ring->priv->dev, ring); | |
| 1315 | + spin_lock_irqsave(&ring->lock, flags); | |
| 1316 | + work_done = __bcmgenet_tx_reclaim(ring->priv->dev, ring); | |
| 1317 | + if (ring->free_bds > (MAX_SKB_FRAGS + 1)) { | |
| 1318 | + txq = netdev_get_tx_queue(ring->priv->dev, ring->queue); | |
| 1319 | + netif_tx_wake_queue(txq); | |
| 1320 | + } | |
| 1321 | + spin_unlock_irqrestore(&ring->lock, flags); | |
| 1253 | 1322 | |
| 1254 | 1323 | if (work_done == 0) { |
| 1255 | 1324 | napi_complete(napi); |
| @@ -2457,24 +2526,28 @@ static int bcmgenet_init_dma(struct bcmgenet_priv *priv) · 1201 unmodified lines | ||
| 2457 | 2526 | /* Interrupt bottom half */ |
| 2458 | 2527 | static void bcmgenet_irq_task(struct work_struct *work) |
| 2459 | 2528 | { |
| 2529 | + unsigned long flags; | |
| 2530 | + unsigned int status; | |
| 2460 | 2531 | struct bcmgenet_priv *priv = container_of( |
| 2461 | 2532 | work, struct bcmgenet_priv, bcmgenet_irq_work); |
| 2462 | 2533 | |
| 2463 | 2534 | netif_dbg(priv, intr, priv->dev, "%s\n", __func__); |
| 2464 | 2535 | |
| 2465 | − if (priv->irq0_stat & UMAC_IRQ_MPD_R) { | |
| 2466 | − priv->irq0_stat &= ~UMAC_IRQ_MPD_R; | |
| 2536 | + spin_lock_irqsave(&priv->lock, flags); | |
| 2537 | + status = priv->irq0_stat; | |
| 2538 | + priv->irq0_stat = 0; | |
| 2539 | + spin_unlock_irqrestore(&priv->lock, flags); | |
| 2540 | + | |
| 2541 | + if (status & UMAC_IRQ_MPD_R) { | |
| 2467 | 2542 | netif_dbg(priv, wol, priv->dev, |
| 2468 | 2543 | "magic packet detected, waking up\n"); |
| 2469 | 2544 | bcmgenet_power_up(priv, GENET_POWER_WOL_MAGIC); |
| 2470 | 2545 | } |
| 2471 | 2546 | |
| 2472 | 2547 | /* Link UP/DOWN event */ |
| 2473 | − if (priv->irq0_stat & UMAC_IRQ_LINK_EVENT) { | |
| 2548 | + if (status & UMAC_IRQ_LINK_EVENT) | |
| 2474 | 2549 | phy_mac_interrupt(priv->phydev, |
| 2475 | − !!(priv->irq0_stat & UMAC_IRQ_LINK_UP)); | |
| 2476 | − priv->irq0_stat &= ~UMAC_IRQ_LINK_EVENT; | |
| 2477 | − } | |
| 2550 | + !!(status & UMAC_IRQ_LINK_UP)); | |
| 2478 | 2551 | } |
| 2479 | 2552 | |
| 2480 | 2553 | /* bcmgenet_isr1: handle Rx and Tx priority queues */ |
| @@ -2483,22 +2556,21 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id) · 2 unmodified lines | ||
| 2483 | 2556 | struct bcmgenet_priv *priv = dev_id; |
| 2484 | 2557 | struct bcmgenet_rx_ring *rx_ring; |
| 2485 | 2558 | struct bcmgenet_tx_ring *tx_ring; |
| 2486 | − unsigned int index; | |
| 2559 | + unsigned int index, status; | |
| 2487 | 2560 | |
| 2488 | − /* Save irq status for bottom-half processing. */ | |
| 2489 | − priv->irq1_stat = | |
| 2490 | − bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_STAT) & | |
| 2561 | + /* Read irq status */ | |
| 2562 | + status = bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_STAT) & | |
| 2491 | 2563 | ~bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_MASK_STATUS); |
| 2492 | 2564 | |
| 2493 | 2565 | /* clear interrupts */ |
| 2494 | − bcmgenet_intrl2_1_writel(priv, priv->irq1_stat, INTRL2_CPU_CLEAR); | |
| 2566 | + bcmgenet_intrl2_1_writel(priv, status, INTRL2_CPU_CLEAR); | |
| 2495 | 2567 | |
| 2496 | 2568 | netif_dbg(priv, intr, priv->dev, |
| 2497 | − "%s: IRQ=0x%x\n", __func__, priv->irq1_stat); | |
| 2569 | + "%s: IRQ=0x%x\n", __func__, status); | |
| 2498 | 2570 | |
| 2499 | 2571 | /* Check Rx priority queue interrupts */ |
| 2500 | 2572 | for (index = 0; index < priv->hw_params->rx_queues; index++) { |
| 2501 | − if (!(priv->irq1_stat & BIT(UMAC_IRQ1_RX_INTR_SHIFT + index))) | |
| 2573 | + if (!(status & BIT(UMAC_IRQ1_RX_INTR_SHIFT + index))) | |
| 2502 | 2574 | continue; |
| 2503 | 2575 | |
| 2504 | 2576 | rx_ring = &priv->rx_rings[index]; |
| @@ -2511,7 +2583,7 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id) · 6 unmodified lines | ||
| 2511 | 2583 | |
| 2512 | 2584 | /* Check Tx priority queue interrupts */ |
| 2513 | 2585 | for (index = 0; index < priv->hw_params->tx_queues; index++) { |
| 2514 | − if (!(priv->irq1_stat & BIT(index))) | |
| 2586 | + if (!(status & BIT(index))) | |
| 2515 | 2587 | continue; |
| 2516 | 2588 | |
| 2517 | 2589 | tx_ring = &priv->tx_rings[index]; |
| @@ -2531,19 +2603,20 @@ static irqreturn_t bcmgenet_isr0(int irq, void *dev_id) · 13 unmodified lines | ||
| 2531 | 2603 | struct bcmgenet_priv *priv = dev_id; |
| 2532 | 2604 | struct bcmgenet_rx_ring *rx_ring; |
| 2533 | 2605 | struct bcmgenet_tx_ring *tx_ring; |
| 2606 | + unsigned int status; | |
| 2607 | + unsigned long flags; | |
| 2534 | 2608 | |
| 2535 | − /* Save irq status for bottom-half processing. */ | |
| 2536 | − priv->irq0_stat = | |
| 2537 | − bcmgenet_intrl2_0_readl(priv, INTRL2_CPU_STAT) & | |
| 2609 | + /* Read irq status */ | |
| 2610 | + status = bcmgenet_intrl2_0_readl(priv, INTRL2_CPU_STAT) & | |
| 2538 | 2611 | ~bcmgenet_intrl2_0_readl(priv, INTRL2_CPU_MASK_STATUS); |
| 2539 | 2612 | |
| 2540 | 2613 | /* clear interrupts */ |
| 2541 | − bcmgenet_intrl2_0_writel(priv, priv->irq0_stat, INTRL2_CPU_CLEAR); | |
| 2614 | + bcmgenet_intrl2_0_writel(priv, status, INTRL2_CPU_CLEAR); | |
| 2542 | 2615 | |
| 2543 | 2616 | netif_dbg(priv, intr, priv->dev, |
| 2544 | − "IRQ=0x%x\n", priv->irq0_stat); | |
| 2617 | + "IRQ=0x%x\n", status); | |
| 2545 | 2618 | |
| 2546 | − if (priv->irq0_stat & UMAC_IRQ_RXDMA_DONE) { | |
| 2619 | + if (status & UMAC_IRQ_RXDMA_DONE) { | |
| 2547 | 2620 | rx_ring = &priv->rx_rings[DESC_INDEX]; |
| 2548 | 2621 | |
| 2549 | 2622 | if (likely(napi_schedule_prep(&rx_ring->napi))) { |
| @@ -2552,7 +2625,7 @@ static irqreturn_t bcmgenet_isr0(int irq, void *dev_id) · 2 unmodified lines | ||
| 2552 | 2625 | } |
| 2553 | 2626 | } |
| 2554 | 2627 | |
| 2555 | − if (priv->irq0_stat & UMAC_IRQ_TXDMA_DONE) { | |
| 2628 | + if (status & UMAC_IRQ_TXDMA_DONE) { | |
| 2556 | 2629 | tx_ring = &priv->tx_rings[DESC_INDEX]; |
| 2557 | 2630 | |
| 2558 | 2631 | if (likely(napi_schedule_prep(&tx_ring->napi))) { |
| @@ -2561,22 +2634,23 @@ static irqreturn_t bcmgenet_isr0(int irq, void *dev_id) · 2 unmodified lines | ||
| 2561 | 2634 | } |
| 2562 | 2635 | } |
| 2563 | 2636 | |
| 2564 | − if (priv->irq0_stat & (UMAC_IRQ_PHY_DET_R | | |
| 2565 | − UMAC_IRQ_PHY_DET_F | | |
| 2566 | − UMAC_IRQ_LINK_EVENT | | |
| 2567 | − UMAC_IRQ_HFB_SM | | |
| 2568 | − UMAC_IRQ_HFB_MM | | |
| 2569 | − UMAC_IRQ_MPD_R)) { | |
| 2570 | − /* all other interested interrupts handled in bottom half */ | |
| 2571 | − schedule_work(&priv->bcmgenet_irq_work); | |
| 2572 | − } | |
| 2573 | − | |
| 2574 | 2637 | if ((priv->hw_params->flags & GENET_HAS_MDIO_INTR) && |
| 2575 | − priv->irq0_stat & (UMAC_IRQ_MDIO_DONE | UMAC_IRQ_MDIO_ERROR)) { | |
| 2576 | − priv->irq0_stat &= ~(UMAC_IRQ_MDIO_DONE | UMAC_IRQ_MDIO_ERROR); | |
| 2638 | + status & (UMAC_IRQ_MDIO_DONE | UMAC_IRQ_MDIO_ERROR)) { | |
| 2577 | 2639 | wake_up(&priv->wq); |
| 2578 | 2640 | } |
| 2579 | 2641 | |
| 2642 | + /* all other interested interrupts handled in bottom half */ | |
| 2643 | + status &= (UMAC_IRQ_LINK_EVENT | | |
| 2644 | + UMAC_IRQ_MPD_R); | |
| 2645 | + if (status) { | |
| 2646 | + /* Save irq status for bottom-half processing. */ | |
| 2647 | + spin_lock_irqsave(&priv->lock, flags); | |
| 2648 | + priv->irq0_stat |= status; | |
| 2649 | + spin_unlock_irqrestore(&priv->lock, flags); | |
| 2650 | + | |
| 2651 | + schedule_work(&priv->bcmgenet_irq_work); | |
| 2652 | + } | |
| 2653 | + | |
| 2580 | 2654 | return IRQ_HANDLED; |
| 2581 | 2655 | } |
| 2582 | 2656 | |
| @@ -2801,6 +2875,8 @@ err_irq0: · 218 unmodified lines | ||
| 2801 | 2875 | err_fini_dma: |
| 2802 | 2876 | bcmgenet_fini_dma(priv); |
| 2803 | 2877 | err_clk_disable: |
| 2878 | + if (priv->internal_phy) | |
| 2879 | + bcmgenet_power_down(priv, GENET_POWER_PASSIVE); | |
| 2804 | 2880 | clk_disable_unprepare(priv->clk); |
| 2805 | 2881 | return ret; |
| 2806 | 2882 | } |
| @@ -3177,6 +3253,12 @@ static void bcmgenet_set_hw_params(struct bcmgenet_priv *priv) · 370 unmodified lines | ||
| 3177 | 3253 | */ |
| 3178 | 3254 | gphy_rev = reg & 0xffff; |
| 3179 | 3255 | |
| 3256 | + /* This is reserved so should require special treatment */ | |
| 3257 | + if (gphy_rev == 0 || gphy_rev == 0x01ff) { | |
| 3258 | + pr_warn("Invalid GPHY revision detected: 0x%04x\n", gphy_rev); | |
| 3259 | + return; | |
| 3260 | + } | |
| 3261 | + | |
| 3180 | 3262 | /* This is the good old scheme, just GPHY major, no minor nor patch */ |
| 3181 | 3263 | if ((gphy_rev & 0xf0) != 0) |
| 3182 | 3264 | priv->gphy_rev = gphy_rev << 8; |
| @@ -3185,12 +3267,6 @@ static void bcmgenet_set_hw_params(struct bcmgenet_priv *priv) · 2 unmodified lines | ||
| 3185 | 3267 | else if ((gphy_rev & 0xff00) != 0) |
| 3186 | 3268 | priv->gphy_rev = gphy_rev; |
| 3187 | 3269 | |
| 3188 | − /* This is reserved so should require special treatment */ | |
| 3189 | − else if (gphy_rev == 0 || gphy_rev == 0x01ff) { | |
| 3190 | − pr_warn("Invalid GPHY revision detected: 0x%04x\n", gphy_rev); | |
| 3191 | − return; | |
| 3192 | − } | |
| 3193 | − | |
| 3194 | 3270 | #ifdef CONFIG_PHYS_ADDR_T_64BIT |
| 3195 | 3271 | if (!(params->flags & GENET_HAS_40BITS)) |
| 3196 | 3272 | pr_warn("GENET does not support 40-bits PA\n"); |
| @@ -3233,6 +3309,7 @@ static int bcmgenet_probe(struct platform_device *pdev) · 36 unmodified lines | ||
| 3233 | 3309 | const void *macaddr; |
| 3234 | 3310 | struct resource *r; |
| 3235 | 3311 | int err = -EIO; |
| 3312 | + const char *phy_mode_str; | |
| 3236 | 3313 | |
| 3237 | 3314 | /* Up to GENET_MAX_MQ_CNT + 1 TX queues and RX queues */ |
| 3238 | 3315 | dev = alloc_etherdev_mqs(sizeof(*priv), GENET_MAX_MQ_CNT + 1, |
| @@ -3276,6 +3353,8 @@ static int bcmgenet_probe(struct platform_device *pdev) · 37 unmodified lines | ||
| 3276 | 3353 | goto err; |
| 3277 | 3354 | } |
| 3278 | 3355 | |
| 3356 | + spin_lock_init(&priv->lock); | |
| 3357 | + | |
| 3279 | 3358 | SET_NETDEV_DEV(dev, &pdev->dev); |
| 3280 | 3359 | dev_set_drvdata(&pdev->dev, dev); |
| 3281 | 3360 | ether_addr_copy(dev->dev_addr, macaddr); |
| @@ -3338,6 +3417,13 @@ static int bcmgenet_probe(struct platform_device *pdev) · 56 unmodified lines | ||
| 3338 | 3417 | priv->clk_eee = NULL; |
| 3339 | 3418 | } |
| 3340 | 3419 | |
| 3420 | + /* If this is an internal GPHY, power it on now, before UniMAC is | |
| 3421 | + * brought out of reset as absolutely no UniMAC activity is allowed | |
| 3422 | + */ | |
| 3423 | + if (dn && !of_property_read_string(dn, "phy-mode", &phy_mode_str) && | |
| 3424 | + !strcasecmp(phy_mode_str, "internal")) | |
| 3425 | + bcmgenet_power_up(priv, GENET_POWER_PASSIVE); | |
| 3426 | + | |
| 3341 | 3427 | err = reset_umac(priv); |
| 3342 | 3428 | if (err) |
| 3343 | 3429 | goto err_clk_disable; |
| @@ -3502,6 +3588,8 @@ static int bcmgenet_resume(struct device *d) · 158 unmodified lines | ||
| 3502 | 3588 | return 0; |
| 3503 | 3589 | |
| 3504 | 3590 | out_clk_disable: |
| 3591 | + if (priv->internal_phy) | |
| 3592 | + bcmgenet_power_down(priv, GENET_POWER_PASSIVE); | |
| 3505 | 3593 | clk_disable_unprepare(priv->clk); |
| 3506 | 3594 | return ret; |
| 3507 | 3595 | } |
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | /* |
| 2 | − * Copyright (c) 2014 Broadcom Corporation | |
| 2 | + * Copyright (c) 2014-2017 Broadcom | |
| 3 | 3 | * |
| 4 | 4 | * This program is free software; you can redistribute it and/or modify |
| 5 | 5 | * it under the terms of the GNU General Public License version 2 as |
| @@ -214,7 +214,9 @@ struct bcmgenet_mib_counters { · 208 unmodified lines | ||
| 214 | 214 | #define MDIO_REG_SHIFT 16 |
| 215 | 215 | #define MDIO_REG_MASK 0x1F |
| 216 | 216 | |
| 217 | −#define UMAC_RBUF_OVFL_CNT 0x61C | |
| 217 | +#define UMAC_RBUF_OVFL_CNT_V1 0x61C | |
| 218 | +#define RBUF_OVFL_CNT_V2 0x80 | |
| 219 | +#define RBUF_OVFL_CNT_V3PLUS 0x94 | |
| 218 | 220 | |
| 219 | 221 | #define UMAC_MPD_CTRL 0x620 |
| 220 | 222 | #define MPD_EN (1 << 0) |
| @@ -224,7 +226,9 @@ struct bcmgenet_mib_counters { · 3 unmodified lines | ||
| 224 | 226 | |
| 225 | 227 | #define UMAC_MPD_PW_MS 0x624 |
| 226 | 228 | #define UMAC_MPD_PW_LS 0x628 |
| 227 | −#define UMAC_RBUF_ERR_CNT 0x634 | |
| 229 | +#define UMAC_RBUF_ERR_CNT_V1 0x634 | |
| 230 | +#define RBUF_ERR_CNT_V2 0x84 | |
| 231 | +#define RBUF_ERR_CNT_V3PLUS 0x98 | |
| 228 | 232 | #define UMAC_MDF_ERR_CNT 0x638 |
| 229 | 233 | #define UMAC_MDF_CTRL 0x650 |
| 230 | 234 | #define UMAC_MDF_ADDR 0x654 |
| @@ -619,11 +623,13 @@ struct bcmgenet_priv { · 388 unmodified lines | ||
| 619 | 623 | struct work_struct bcmgenet_irq_work; |
| 620 | 624 | int irq0; |
| 621 | 625 | int irq1; |
| 622 | − unsigned int irq0_stat; | |
| 623 | − unsigned int irq1_stat; | |
| 624 | 626 | int wol_irq; |
| 625 | 627 | bool wol_irq_disabled; |
| 626 | 628 | |
| 629 | + /* shared status */ | |
| 630 | + spinlock_t lock; | |
| 631 | + unsigned int irq0_stat; | |
| 632 | + | |
| 627 | 633 | /* HW descriptors/checksum variables */ |
| 628 | 634 | bool desc_64b_en; |
| 629 | 635 | bool desc_rxchk_en; |
| @@ -152,7 +152,7 @@ struct octnic_gather { · 151 unmodified lines | ||
| 152 | 152 | */ |
| 153 | 153 | struct octeon_sg_entry *sg; |
| 154 | 154 | |
| 155 | − u64 sg_dma_ptr; | |
| 155 | + dma_addr_t sg_dma_ptr; | |
| 156 | 156 | }; |
| 157 | 157 | |
| 158 | 158 | struct handshake { |
| @@ -734,6 +734,9 @@ static void delete_glists(struct lio *lio) · 575 unmodified lines | ||
| 734 | 734 | struct octnic_gather *g; |
| 735 | 735 | int i; |
| 736 | 736 | |
| 737 | + kfree(lio->glist_lock); | |
| 738 | + lio->glist_lock = NULL; | |
| 739 | + | |
| 737 | 740 | if (!lio->glist) |
| 738 | 741 | return; |
| 739 | 742 | |
| @@ -741,23 +744,26 @@ static void delete_glists(struct lio *lio) · 1 unmodified line | ||
| 741 | 744 | do { |
| 742 | 745 | g = (struct octnic_gather *) |
| 743 | 746 | list_delete_head(&lio->glist[i]); |
| 744 | − if (g) { | |
| 745 | − if (g->sg) { | |
| 746 | − dma_unmap_single(&lio->oct_dev-> | |
| 747 | − pci_dev->dev, | |
| 748 | − g->sg_dma_ptr, | |
| 749 | − g->sg_size, | |
| 750 | − DMA_TO_DEVICE); | |
| 751 | − kfree((void *)((unsigned long)g->sg - | |
| 752 | − g->adjust)); | |
| 753 | − } | |
| 747 | + if (g) | |
| 754 | 748 | kfree(g); |
| 755 | − } | |
| 756 | 749 | } while (g); |
| 750 | + | |
| 751 | + if (lio->glists_virt_base && lio->glists_virt_base[i]) { | |
| 752 | + lio_dma_free(lio->oct_dev, | |
| 753 | + lio->glist_entry_size * lio->tx_qsize, | |
| 754 | + lio->glists_virt_base[i], | |
| 755 | + lio->glists_dma_base[i]); | |
| 756 | + } | |
| 757 | 757 | } |
| 758 | 758 | |
| 759 | − kfree((void *)lio->glist); | |
| 760 | − kfree((void *)lio->glist_lock); | |
| 759 | + kfree(lio->glists_virt_base); | |
| 760 | + lio->glists_virt_base = NULL; | |
| 761 | + | |
| 762 | + kfree(lio->glists_dma_base); | |
| 763 | + lio->glists_dma_base = NULL; | |
| 764 | + | |
| 765 | + kfree(lio->glist); | |
| 766 | + lio->glist = NULL; | |
| 761 | 767 | } |
| 762 | 768 | |
| 763 | 769 | /** |
| @@ -772,13 +778,30 @@ static int setup_glists(struct octeon_device *oct, struct lio *lio, int num_iqs) · 8 unmodified lines | ||
| 772 | 778 | lio->glist_lock = kcalloc(num_iqs, sizeof(*lio->glist_lock), |
| 773 | 779 | GFP_KERNEL); |
| 774 | 780 | if (!lio->glist_lock) |
| 775 | − return 1; | |
| 781 | + return -ENOMEM; | |
| 776 | 782 | |
| 777 | 783 | lio->glist = kcalloc(num_iqs, sizeof(*lio->glist), |
| 778 | 784 | GFP_KERNEL); |
| 779 | 785 | if (!lio->glist) { |
| 780 | − kfree((void *)lio->glist_lock); | |
| 781 | − return 1; | |
| 786 | + kfree(lio->glist_lock); | |
| 787 | + lio->glist_lock = NULL; | |
| 788 | + return -ENOMEM; | |
| 789 | + } | |
| 790 | + | |
| 791 | + lio->glist_entry_size = | |
| 792 | + ROUNDUP8((ROUNDUP4(OCTNIC_MAX_SG) >> 2) * OCT_SG_ENTRY_SIZE); | |
| 793 | + | |
| 794 | + /* allocate memory to store virtual and dma base address of | |
| 795 | + * per glist consistent memory | |
| 796 | + */ | |
| 797 | + lio->glists_virt_base = kcalloc(num_iqs, sizeof(*lio->glists_virt_base), | |
| 798 | + GFP_KERNEL); | |
| 799 | + lio->glists_dma_base = kcalloc(num_iqs, sizeof(*lio->glists_dma_base), | |
| 800 | + GFP_KERNEL); | |
| 801 | + | |
| 802 | + if (!lio->glists_virt_base || !lio->glists_dma_base) { | |
| 803 | + delete_glists(lio); | |
| 804 | + return -ENOMEM; | |
| 782 | 805 | } |
| 783 | 806 | |
| 784 | 807 | for (i = 0; i < num_iqs; i++) { |
| @@ -788,6 +811,16 @@ static int setup_glists(struct octeon_device *oct, struct lio *lio, int num_iqs) · 3 unmodified lines | ||
| 788 | 811 | |
| 789 | 812 | INIT_LIST_HEAD(&lio->glist[i]); |
| 790 | 813 | |
| 814 | + lio->glists_virt_base[i] = | |
| 815 | + lio_dma_alloc(oct, | |
| 816 | + lio->glist_entry_size * lio->tx_qsize, | |
| 817 | + &lio->glists_dma_base[i]); | |
| 818 | + | |
| 819 | + if (!lio->glists_virt_base[i]) { | |
| 820 | + delete_glists(lio); | |
| 821 | + return -ENOMEM; | |
| 822 | + } | |
| 823 | + | |
| 791 | 824 | for (j = 0; j < lio->tx_qsize; j++) { |
| 792 | 825 | g = kzalloc_node(sizeof(*g), GFP_KERNEL, |
| 793 | 826 | numa_node); |
| @@ -796,43 +829,18 @@ static int setup_glists(struct octeon_device *oct, struct lio *lio, int num_iqs) · 2 unmodified lines | ||
| 796 | 829 | if (!g) |
| 797 | 830 | break; |
| 798 | 831 | |
| 799 | − g->sg_size = ((ROUNDUP4(OCTNIC_MAX_SG) >> 2) * | |
| 800 | − OCT_SG_ENTRY_SIZE); | |
| 832 | + g->sg = lio->glists_virt_base[i] + | |
| 833 | + (j * lio->glist_entry_size); | |
| 801 | 834 | |
| 802 | − g->sg = kmalloc_node(g->sg_size + 8, | |
| 803 | − GFP_KERNEL, numa_node); | |
| 804 | − if (!g->sg) | |
| 805 | − g->sg = kmalloc(g->sg_size + 8, GFP_KERNEL); | |
| 806 | − if (!g->sg) { | |
| 807 | − kfree(g); | |
| 808 | − break; | |
| 809 | − } | |
| 810 | − | |
| 811 | − /* The gather component should be aligned on 64-bit | |
| 812 | − * boundary | |
| 813 | − */ | |
| 814 | − if (((unsigned long)g->sg) & 7) { | |
| 815 | − g->adjust = 8 - (((unsigned long)g->sg) & 7); | |
| 816 | − g->sg = (struct octeon_sg_entry *) | |
| 817 | − ((unsigned long)g->sg + g->adjust); | |
| 818 | − } | |
| 819 | − g->sg_dma_ptr = dma_map_single(&oct->pci_dev->dev, | |
| 820 | − g->sg, g->sg_size, | |
| 821 | − DMA_TO_DEVICE); | |
| 822 | − if (dma_mapping_error(&oct->pci_dev->dev, | |
| 823 | − g->sg_dma_ptr)) { | |
| 824 | − kfree((void *)((unsigned long)g->sg - | |
| 825 | − g->adjust)); | |
| 826 | − kfree(g); | |
| 827 | − break; | |
| 828 | − } | |
| 835 | + g->sg_dma_ptr = lio->glists_dma_base[i] + | |
| 836 | + (j * lio->glist_entry_size); | |
| 829 | 837 | |
| 830 | 838 | list_add_tail(&g->list, &lio->glist[i]); |
| 831 | 839 | } |
| 832 | 840 | |
| 833 | 841 | if (j != lio->tx_qsize) { |
| 834 | 842 | delete_glists(lio); |
| 835 | − return 1; | |
| 843 | + return -ENOMEM; | |
| 836 | 844 | } |
| 837 | 845 | } |
| 838 | 846 | |
| @@ -1885,9 +1893,6 @@ static void free_netsgbuf(void *buf) · 1046 unmodified lines | ||
| 1885 | 1893 | i++; |
| 1886 | 1894 | } |
| 1887 | 1895 | |
| 1888 | − dma_sync_single_for_cpu(&lio->oct_dev->pci_dev->dev, | |
| 1889 | − g->sg_dma_ptr, g->sg_size, DMA_TO_DEVICE); | |
| 1890 | − | |
| 1891 | 1896 | iq = skb_iq(lio, skb); |
| 1892 | 1897 | spin_lock(&lio->glist_lock[iq]); |
| 1893 | 1898 | list_add_tail(&g->list, &lio->glist[iq]); |
| @@ -1933,9 +1938,6 @@ static void free_netsgbuf_with_resp(void *buf) · 39 unmodified lines | ||
| 1933 | 1938 | i++; |
| 1934 | 1939 | } |
| 1935 | 1940 | |
| 1936 | − dma_sync_single_for_cpu(&lio->oct_dev->pci_dev->dev, | |
| 1937 | − g->sg_dma_ptr, g->sg_size, DMA_TO_DEVICE); | |
| 1938 | − | |
| 1939 | 1941 | iq = skb_iq(lio, skb); |
| 1940 | 1942 | |
| 1941 | 1943 | spin_lock(&lio->glist_lock[iq]); |
| @@ -3273,8 +3275,6 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) · 1331 unmodified lines | ||
| 3273 | 3275 | i++; |
| 3274 | 3276 | } |
| 3275 | 3277 | |
| 3276 | − dma_sync_single_for_device(&oct->pci_dev->dev, g->sg_dma_ptr, | |
| 3277 | − g->sg_size, DMA_TO_DEVICE); | |
| 3278 | 3278 | dptr = g->sg_dma_ptr; |
| 3279 | 3279 | |
| 3280 | 3280 | if (OCTEON_CN23XX_PF(oct)) |
| @@ -108,6 +108,8 @@ struct octnic_gather { · 107 unmodified lines | ||
| 108 | 108 | * received from the IP layer. |
| 109 | 109 | */ |
| 110 | 110 | struct octeon_sg_entry *sg; |
| 111 | + | |
| 112 | + dma_addr_t sg_dma_ptr; | |
| 111 | 113 | }; |
| 112 | 114 | |
| 113 | 115 | struct octeon_device_priv { |
| @@ -490,6 +492,9 @@ static void delete_glists(struct lio *lio) · 376 unmodified lines | ||
| 490 | 492 | struct octnic_gather *g; |
| 491 | 493 | int i; |
| 492 | 494 | |
| 495 | + kfree(lio->glist_lock); | |
| 496 | + lio->glist_lock = NULL; | |
| 497 | + | |
| 493 | 498 | if (!lio->glist) |
| 494 | 499 | return; |
| 495 | 500 | |
| @@ -497,17 +502,26 @@ static void delete_glists(struct lio *lio) · 1 unmodified line | ||
| 497 | 502 | do { |
| 498 | 503 | g = (struct octnic_gather *) |
| 499 | 504 | list_delete_head(&lio->glist[i]); |
| 500 | − if (g) { | |
| 501 | − if (g->sg) | |
| 502 | − kfree((void *)((unsigned long)g->sg - | |
| 503 | − g->adjust)); | |
| 505 | + if (g) | |
| 504 | 506 | kfree(g); |
| 505 | − } | |
| 506 | 507 | } while (g); |
| 508 | + | |
| 509 | + if (lio->glists_virt_base && lio->glists_virt_base[i]) { | |
| 510 | + lio_dma_free(lio->oct_dev, | |
| 511 | + lio->glist_entry_size * lio->tx_qsize, | |
| 512 | + lio->glists_virt_base[i], | |
| 513 | + lio->glists_dma_base[i]); | |
| 514 | + } | |
| 507 | 515 | } |
| 508 | 516 | |
| 517 | + kfree(lio->glists_virt_base); | |
| 518 | + lio->glists_virt_base = NULL; | |
| 519 | + | |
| 520 | + kfree(lio->glists_dma_base); | |
| 521 | + lio->glists_dma_base = NULL; | |
| 522 | + | |
| 509 | 523 | kfree(lio->glist); |
| 510 | − kfree(lio->glist_lock); | |
| 524 | + lio->glist = NULL; | |
| 511 | 525 | } |
| 512 | 526 | |
| 513 | 527 | /** |
| @@ -522,13 +536,30 @@ static int setup_glists(struct lio *lio, int num_iqs) · 8 unmodified lines | ||
| 522 | 536 | lio->glist_lock = |
| 523 | 537 | kzalloc(sizeof(*lio->glist_lock) * num_iqs, GFP_KERNEL); |
| 524 | 538 | if (!lio->glist_lock) |
| 525 | − return 1; | |
| 539 | + return -ENOMEM; | |
| 526 | 540 | |
| 527 | 541 | lio->glist = |
| 528 | 542 | kzalloc(sizeof(*lio->glist) * num_iqs, GFP_KERNEL); |
| 529 | 543 | if (!lio->glist) { |
| 530 | 544 | kfree(lio->glist_lock); |
| 531 | − return 1; | |
| 545 | + lio->glist_lock = NULL; | |
| 546 | + return -ENOMEM; | |
| 547 | + } | |
| 548 | + | |
| 549 | + lio->glist_entry_size = | |
| 550 | + ROUNDUP8((ROUNDUP4(OCTNIC_MAX_SG) >> 2) * OCT_SG_ENTRY_SIZE); | |
| 551 | + | |
| 552 | + /* allocate memory to store virtual and dma base address of | |
| 553 | + * per glist consistent memory | |
| 554 | + */ | |
| 555 | + lio->glists_virt_base = kcalloc(num_iqs, sizeof(*lio->glists_virt_base), | |
| 556 | + GFP_KERNEL); | |
| 557 | + lio->glists_dma_base = kcalloc(num_iqs, sizeof(*lio->glists_dma_base), | |
| 558 | + GFP_KERNEL); | |
| 559 | + | |
| 560 | + if (!lio->glists_virt_base || !lio->glists_dma_base) { | |
| 561 | + delete_glists(lio); | |
| 562 | + return -ENOMEM; | |
| 532 | 563 | } |
| 533 | 564 | |
| 534 | 565 | for (i = 0; i < num_iqs; i++) { |
| @@ -536,34 +567,33 @@ static int setup_glists(struct lio *lio, int num_iqs) · 1 unmodified line | ||
| 536 | 567 | |
| 537 | 568 | INIT_LIST_HEAD(&lio->glist[i]); |
| 538 | 569 | |
| 570 | + lio->glists_virt_base[i] = | |
| 571 | + lio_dma_alloc(lio->oct_dev, | |
| 572 | + lio->glist_entry_size * lio->tx_qsize, | |
| 573 | + &lio->glists_dma_base[i]); | |
| 574 | + | |
| 575 | + if (!lio->glists_virt_base[i]) { | |
| 576 | + delete_glists(lio); | |
| 577 | + return -ENOMEM; | |
| 578 | + } | |
| 579 | + | |
| 539 | 580 | for (j = 0; j < lio->tx_qsize; j++) { |
| 540 | 581 | g = kzalloc(sizeof(*g), GFP_KERNEL); |
| 541 | 582 | if (!g) |
| 542 | 583 | break; |
| 543 | 584 | |
| 544 | − g->sg_size = ((ROUNDUP4(OCTNIC_MAX_SG) >> 2) * | |
| 545 | − OCT_SG_ENTRY_SIZE); | |
| 585 | + g->sg = lio->glists_virt_base[i] + | |
| 586 | + (j * lio->glist_entry_size); | |
| 546 | 587 | |
| 547 | − g->sg = kmalloc(g->sg_size + 8, GFP_KERNEL); | |
| 548 | − if (!g->sg) { | |
| 549 | − kfree(g); | |
| 550 | − break; | |
| 551 | − } | |
| 588 | + g->sg_dma_ptr = lio->glists_dma_base[i] + | |
| 589 | + (j * lio->glist_entry_size); | |
| 552 | 590 | |
| 553 | − /* The gather component should be aligned on 64-bit | |
| 554 | − * boundary | |
| 555 | − */ | |
| 556 | − if (((unsigned long)g->sg) & 7) { | |
| 557 | − g->adjust = 8 - (((unsigned long)g->sg) & 7); | |
| 558 | − g->sg = (struct octeon_sg_entry *) | |
| 559 | − ((unsigned long)g->sg + g->adjust); | |
| 560 | − } | |
| 561 | 591 | list_add_tail(&g->list, &lio->glist[i]); |
| 562 | 592 | } |
| 563 | 593 | |
| 564 | 594 | if (j != lio->tx_qsize) { |
| 565 | 595 | delete_glists(lio); |
| 566 | − return 1; | |
| 596 | + return -ENOMEM; | |
| 567 | 597 | } |
| 568 | 598 | } |
| 569 | 599 | |
| @@ -1324,10 +1354,6 @@ static void free_netsgbuf(void *buf) · 754 unmodified lines | ||
| 1324 | 1354 | i++; |
| 1325 | 1355 | } |
| 1326 | 1356 | |
| 1327 | − dma_unmap_single(&lio->oct_dev->pci_dev->dev, | |
| 1328 | − finfo->dptr, g->sg_size, | |
| 1329 | − DMA_TO_DEVICE); | |
| 1330 | − | |
| 1331 | 1357 | iq = skb_iq(lio, skb); |
| 1332 | 1358 | |
| 1333 | 1359 | spin_lock(&lio->glist_lock[iq]); |
| @@ -1374,10 +1400,6 @@ static void free_netsgbuf_with_resp(void *buf) · 40 unmodified lines | ||
| 1374 | 1400 | i++; |
| 1375 | 1401 | } |
| 1376 | 1402 | |
| 1377 | − dma_unmap_single(&lio->oct_dev->pci_dev->dev, | |
| 1378 | − finfo->dptr, g->sg_size, | |
| 1379 | − DMA_TO_DEVICE); | |
| 1380 | − | |
| 1381 | 1403 | iq = skb_iq(lio, skb); |
| 1382 | 1404 | |
| 1383 | 1405 | spin_lock(&lio->glist_lock[iq]); |
| @@ -2382,23 +2404,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) · 998 unmodified lines | ||
| 2382 | 2404 | i++; |
| 2383 | 2405 | } |
| 2384 | 2406 | |
| 2385 | − dptr = dma_map_single(&oct->pci_dev->dev, | |
| 2386 | − g->sg, g->sg_size, | |
| 2387 | − DMA_TO_DEVICE); | |
| 2388 | − if (dma_mapping_error(&oct->pci_dev->dev, dptr)) { | |
| 2389 | − dev_err(&oct->pci_dev->dev, "%s DMA mapping error 4\n", | |
| 2390 | − __func__); | |
| 2391 | − dma_unmap_single(&oct->pci_dev->dev, g->sg[0].ptr[0], | |
| 2392 | − skb->len - skb->data_len, | |
| 2393 | − DMA_TO_DEVICE); | |
| 2394 | − for (j = 1; j <= frags; j++) { | |
| 2395 | − frag = &skb_shinfo(skb)->frags[j - 1]; | |
| 2396 | − dma_unmap_page(&oct->pci_dev->dev, | |
| 2397 | − g->sg[j >> 2].ptr[j & 3], | |
| 2398 | − frag->size, DMA_TO_DEVICE); | |
| 2399 | − } | |
| 2400 | − return NETDEV_TX_BUSY; | |
| 2401 | − } | |
| 2407 | + dptr = g->sg_dma_ptr; | |
| 2402 | 2408 | |
| 2403 | 2409 | ndata.cmd.cmd3.dptr = dptr; |
| 2404 | 2410 | finfo->dptr = dptr; |
| @@ -71,17 +71,17 @@ · 70 unmodified lines | ||
| 71 | 71 | #define CN23XX_MAX_RINGS_PER_VF 8 |
| 72 | 72 | |
| 73 | 73 | #define CN23XX_MAX_INPUT_QUEUES CN23XX_MAX_RINGS_PER_PF |
| 74 | −#define CN23XX_MAX_IQ_DESCRIPTORS 2048 | |
| 74 | +#define CN23XX_MAX_IQ_DESCRIPTORS 512 | |
| 75 | 75 | #define CN23XX_DB_MIN 1 |
| 76 | 76 | #define CN23XX_DB_MAX 8 |
| 77 | 77 | #define CN23XX_DB_TIMEOUT 1 |
| 78 | 78 | |
| 79 | 79 | #define CN23XX_MAX_OUTPUT_QUEUES CN23XX_MAX_RINGS_PER_PF |
| 80 | −#define CN23XX_MAX_OQ_DESCRIPTORS 2048 | |
| 80 | +#define CN23XX_MAX_OQ_DESCRIPTORS 512 | |
| 81 | 81 | #define CN23XX_OQ_BUF_SIZE 1536 |
| 82 | 82 | #define CN23XX_OQ_PKTSPER_INTR 128 |
| 83 | 83 | /*#define CAVIUM_ONLY_CN23XX_RX_PERF*/ |
| 84 | −#define CN23XX_OQ_REFIL_THRESHOLD 128 | |
| 84 | +#define CN23XX_OQ_REFIL_THRESHOLD 16 | |
| 85 | 85 | |
| 86 | 86 | #define CN23XX_OQ_INTR_PKT 64 |
| 87 | 87 | #define CN23XX_OQ_INTR_TIME 100 |
| @@ -155,11 +155,6 @@ octeon_droq_destroy_ring_buffers(struct octeon_device *oct, · 154 unmodified lines | ||
| 155 | 155 | recv_buffer_destroy(droq->recv_buf_list[i].buffer, |
| 156 | 156 | pg_info); |
| 157 | 157 | |
| 158 | − if (droq->desc_ring && droq->desc_ring[i].info_ptr) | |
| 159 | − lio_unmap_ring_info(oct->pci_dev, | |
| 160 | − (u64)droq-> | |
| 161 | − desc_ring[i].info_ptr, | |
| 162 | − OCT_DROQ_INFO_SIZE); | |
| 163 | 158 | droq->recv_buf_list[i].buffer = NULL; |
| 164 | 159 | } |
| 165 | 160 | |
| @@ -211,10 +206,7 @@ int octeon_delete_droq(struct octeon_device *oct, u32 q_no) · 45 unmodified lines | ||
| 211 | 206 | vfree(droq->recv_buf_list); |
| 212 | 207 | |
| 213 | 208 | if (droq->info_base_addr) |
| 214 | − cnnic_free_aligned_dma(oct->pci_dev, droq->info_list, | |
| 215 | − droq->info_alloc_size, | |
| 216 | − droq->info_base_addr, | |
| 217 | − droq->info_list_dma); | |
| 209 | + lio_free_info_buffer(oct, droq); | |
| 218 | 210 | |
| 219 | 211 | if (droq->desc_ring) |
| 220 | 212 | lio_dma_free(oct, (droq->max_count * OCT_DROQ_DESC_SIZE), |
| @@ -294,12 +286,7 @@ int octeon_init_droq(struct octeon_device *oct, · 73 unmodified lines | ||
| 294 | 286 | dev_dbg(&oct->pci_dev->dev, "droq[%d]: num_desc: %d\n", q_no, |
| 295 | 287 | droq->max_count); |
| 296 | 288 | |
| 297 | − droq->info_list = | |
| 298 | − cnnic_numa_alloc_aligned_dma((droq->max_count * | |
| 299 | − OCT_DROQ_INFO_SIZE), | |
| 300 | − &droq->info_alloc_size, | |
| 301 | − &droq->info_base_addr, | |
| 302 | − numa_node); | |
| 289 | + droq->info_list = lio_alloc_info_buffer(oct, droq); | |
| 303 | 290 | if (!droq->info_list) { |
| 304 | 291 | dev_err(&oct->pci_dev->dev, "Cannot allocate memory for info list.\n"); |
| 305 | 292 | lio_dma_free(oct, (droq->max_count * OCT_DROQ_DESC_SIZE), |
| @@ -325,10 +325,10 @@ struct octeon_droq { · 324 unmodified lines | ||
| 325 | 325 | size_t desc_ring_dma; |
| 326 | 326 | |
| 327 | 327 | /** Info ptr list are allocated at this virtual address. */ |
| 328 | − size_t info_base_addr; | |
| 328 | + void *info_base_addr; | |
| 329 | 329 | |
| 330 | 330 | /** DMA mapped address of the info list */ |
| 331 | − size_t info_list_dma; | |
| 331 | + dma_addr_t info_list_dma; | |
| 332 | 332 | |
| 333 | 333 | /** Allocated size of info list. */ |
| 334 | 334 | u32 info_alloc_size; |
| @@ -140,48 +140,6 @@ err_release_region: · 139 unmodified lines | ||
| 140 | 140 | return 1; |
| 141 | 141 | } |
| 142 | 142 | |
| 143 | −static inline void * | |
| 144 | −cnnic_numa_alloc_aligned_dma(u32 size, | |
| 145 | − u32 *alloc_size, | |
| 146 | − size_t *orig_ptr, | |
| 147 | − int numa_node) | |
| 148 | −{ | |
| 149 | − int retries = 0; | |
| 150 | − void *ptr = NULL; | |
| 151 | − | |
| 152 | −#define OCTEON_MAX_ALLOC_RETRIES 1 | |
| 153 | − do { | |
| 154 | − struct page *page = NULL; | |
| 155 | − | |
| 156 | − page = alloc_pages_node(numa_node, | |
| 157 | − GFP_KERNEL, | |
| 158 | − get_order(size)); | |
| 159 | − if (!page) | |
| 160 | − page = alloc_pages(GFP_KERNEL, | |
| 161 | − get_order(size)); | |
| 162 | − ptr = (void *)page_address(page); | |
| 163 | − if ((unsigned long)ptr & 0x07) { | |
| 164 | − __free_pages(page, get_order(size)); | |
| 165 | − ptr = NULL; | |
| 166 | − /* Increment the size required if the first | |
| 167 | − * attempt failed. | |
| 168 | − */ | |
| 169 | − if (!retries) | |
| 170 | − size += 7; | |
| 171 | − } | |
| 172 | − retries++; | |
| 173 | − } while ((retries <= OCTEON_MAX_ALLOC_RETRIES) && !ptr); | |
| 174 | − | |
| 175 | − *alloc_size = size; | |
| 176 | − *orig_ptr = (unsigned long)ptr; | |
| 177 | − if ((unsigned long)ptr & 0x07) | |
| 178 | − ptr = (void *)(((unsigned long)ptr + 7) & ~(7UL)); | |
| 179 | − return ptr; | |
| 180 | −} | |
| 181 | − | |
| 182 | −#define cnnic_free_aligned_dma(pci_dev, ptr, size, orig_ptr, dma_addr) \ | |
| 183 | − free_pages(orig_ptr, get_order(size)) | |
| 184 | − | |
| 185 | 143 | static inline int |
| 186 | 144 | sleep_cond(wait_queue_head_t *wait_queue, int *condition) |
| 187 | 145 | { |
| @@ -62,6 +62,9 @@ struct lio { · 61 unmodified lines | ||
| 62 | 62 | |
| 63 | 63 | /** Array of gather component linked lists */ |
| 64 | 64 | struct list_head *glist; |
| 65 | + void **glists_virt_base; | |
| 66 | + dma_addr_t *glists_dma_base; | |
| 67 | + u32 glist_entry_size; | |
| 65 | 68 | |
| 66 | 69 | /** Pointer to the NIC properties for the Octeon device this network |
| 67 | 70 | * interface is associated with. |
| @@ -344,6 +347,29 @@ static inline void tx_buffer_free(void *buffer) · 276 unmodified lines | ||
| 344 | 347 | #define lio_dma_free(oct, size, virt_addr, dma_addr) \ |
| 345 | 348 | dma_free_coherent(&(oct)->pci_dev->dev, size, virt_addr, dma_addr) |
| 346 | 349 | |
| 350 | +static inline void * | |
| 351 | +lio_alloc_info_buffer(struct octeon_device *oct, | |
| 352 | + struct octeon_droq *droq) | |
| 353 | +{ | |
| 354 | + void *virt_ptr; | |
| 355 | + | |
| 356 | + virt_ptr = lio_dma_alloc(oct, (droq->max_count * OCT_DROQ_INFO_SIZE), | |
| 357 | + &droq->info_list_dma); | |
| 358 | + if (virt_ptr) { | |
| 359 | + droq->info_alloc_size = droq->max_count * OCT_DROQ_INFO_SIZE; | |
| 360 | + droq->info_base_addr = virt_ptr; | |
| 361 | + } | |
| 362 | + | |
| 363 | + return virt_ptr; | |
| 364 | +} | |
| 365 | + | |
| 366 | +static inline void lio_free_info_buffer(struct octeon_device *oct, | |
| 367 | + struct octeon_droq *droq) | |
| 368 | +{ | |
| 369 | + lio_dma_free(oct, droq->info_alloc_size, droq->info_base_addr, | |
| 370 | + droq->info_list_dma); | |
| 371 | +} | |
| 372 | + | |
| 347 | 373 | static inline |
| 348 | 374 | void *get_rbd(struct sk_buff *skb) |
| 349 | 375 | { |
| @@ -359,22 +385,7 @@ void *get_rbd(struct sk_buff *skb) · 9 unmodified lines | ||
| 359 | 385 | static inline u64 |
| 360 | 386 | lio_map_ring_info(struct octeon_droq *droq, u32 i) |
| 361 | 387 | { |
| 362 | − dma_addr_t dma_addr; | |
| 363 | − struct octeon_device *oct = droq->oct_dev; | |
| 364 | − | |
| 365 | − dma_addr = dma_map_single(&oct->pci_dev->dev, &droq->info_list[i], | |
| 366 | − OCT_DROQ_INFO_SIZE, DMA_FROM_DEVICE); | |
| 367 | − | |
| 368 | − WARN_ON(dma_mapping_error(&oct->pci_dev->dev, dma_addr)); | |
| 369 | − | |
| 370 | − return (u64)dma_addr; | |
| 371 | −} | |
| 372 | − | |
| 373 | −static inline void | |
| 374 | −lio_unmap_ring_info(struct pci_dev *pci_dev, | |
| 375 | − u64 info_ptr, u32 size) | |
| 376 | −{ | |
| 377 | − dma_unmap_single(&pci_dev->dev, info_ptr, size, DMA_FROM_DEVICE); | |
| 388 | + return droq->info_list_dma + (i * sizeof(struct octeon_droq_info)); | |
| 378 | 389 | } |
| 379 | 390 | |
| 380 | 391 | static inline u64 |
| @@ -269,6 +269,7 @@ struct nicvf { · 268 unmodified lines | ||
| 269 | 269 | #define MAX_QUEUES_PER_QSET 8 |
| 270 | 270 | struct queue_set *qs; |
| 271 | 271 | struct nicvf_cq_poll *napi[8]; |
| 272 | + void *iommu_domain; | |
| 272 | 273 | u8 vf_id; |
| 273 | 274 | u8 sqs_id; |
| 274 | 275 | bool sqs_mode; |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <linux/log2.h> |
| 17 | 17 | #include <linux/prefetch.h> |
| 18 | 18 | #include <linux/irq.h> |
| 19 | +#include <linux/iommu.h> | |
| 19 | 20 | |
| 20 | 21 | #include "nic_reg.h" |
| 21 | 22 | #include "nic.h" |
| @@ -525,7 +526,12 @@ static void nicvf_snd_pkt_handler(struct net_device *netdev, · 503 unmodified lines | ||
| 525 | 526 | /* Get actual TSO descriptors and free them */ |
| 526 | 527 | tso_sqe = |
| 527 | 528 | (struct sq_hdr_subdesc *)GET_SQ_DESC(sq, hdr->rsvd2); |
| 529 | + nicvf_unmap_sndq_buffers(nic, sq, hdr->rsvd2, | |
| 530 | + tso_sqe->subdesc_cnt); | |
| 528 | 531 | nicvf_put_sq_desc(sq, tso_sqe->subdesc_cnt + 1); |
| 532 | + } else { | |
| 533 | + nicvf_unmap_sndq_buffers(nic, sq, cqe_tx->sqe_ptr, | |
| 534 | + hdr->subdesc_cnt); | |
| 529 | 535 | } |
| 530 | 536 | nicvf_put_sq_desc(sq, hdr->subdesc_cnt + 1); |
| 531 | 537 | prefetch(skb); |
| @@ -576,6 +582,7 @@ static void nicvf_rcv_pkt_handler(struct net_device *netdev, · 44 unmodified lines | ||
| 576 | 582 | { |
| 577 | 583 | struct sk_buff *skb; |
| 578 | 584 | struct nicvf *nic = netdev_priv(netdev); |
| 585 | + struct nicvf *snic = nic; | |
| 579 | 586 | int err = 0; |
| 580 | 587 | int rq_idx; |
| 581 | 588 | |
| @@ -592,7 +599,7 @@ static void nicvf_rcv_pkt_handler(struct net_device *netdev, · 10 unmodified lines | ||
| 592 | 599 | if (err && !cqe_rx->rb_cnt) |
| 593 | 600 | return; |
| 594 | 601 | |
| 595 | − skb = nicvf_get_rcv_skb(nic, cqe_rx); | |
| 602 | + skb = nicvf_get_rcv_skb(snic, cqe_rx); | |
| 596 | 603 | if (!skb) { |
| 597 | 604 | netdev_dbg(nic->netdev, "Packet not received\n"); |
| 598 | 605 | return; |
| @@ -1643,6 +1650,9 @@ static int nicvf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 1044 unmodified lines | ||
| 1643 | 1650 | if (!pass1_silicon(nic->pdev)) |
| 1644 | 1651 | nic->hw_tso = true; |
| 1645 | 1652 | |
| 1653 | + /* Get iommu domain for iova to physical addr conversion */ | |
| 1654 | + nic->iommu_domain = iommu_get_domain_for_dev(dev); | |
| 1655 | + | |
| 1646 | 1656 | pci_read_config_word(nic->pdev, PCI_SUBSYSTEM_ID, &sdevid); |
| 1647 | 1657 | if (sdevid == 0xA134) |
| 1648 | 1658 | nic->t88 = true; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/netdevice.h> |
| 11 | 11 | #include <linux/ip.h> |
| 12 | 12 | #include <linux/etherdevice.h> |
| 13 | +#include <linux/iommu.h> | |
| 13 | 14 | #include <net/ip.h> |
| 14 | 15 | #include <net/tso.h> |
| 15 | 16 | |
| @@ -18,6 +19,16 @@ · 2 unmodified lines | ||
| 18 | 19 | #include "q_struct.h" |
| 19 | 20 | #include "nicvf_queues.h" |
| 20 | 21 | |
| 22 | +#define NICVF_PAGE_ORDER ((PAGE_SIZE <= 4096) ? PAGE_ALLOC_COSTLY_ORDER : 0) | |
| 23 | + | |
| 24 | +static inline u64 nicvf_iova_to_phys(struct nicvf *nic, dma_addr_t dma_addr) | |
| 25 | +{ | |
| 26 | + /* Translation is installed only when IOMMU is present */ | |
| 27 | + if (nic->iommu_domain) | |
| 28 | + return iommu_iova_to_phys(nic->iommu_domain, dma_addr); | |
| 29 | + return dma_addr; | |
| 30 | +} | |
| 31 | + | |
| 21 | 32 | static void nicvf_get_page(struct nicvf *nic) |
| 22 | 33 | { |
| 23 | 34 | if (!nic->rb_pageref || !nic->rb_page) |
| @@ -87,7 +98,7 @@ static void nicvf_free_q_desc_mem(struct nicvf *nic, struct q_desc_mem *dmem) · 63 unmodified lines | ||
| 87 | 98 | static inline int nicvf_alloc_rcv_buffer(struct nicvf *nic, gfp_t gfp, |
| 88 | 99 | u32 buf_len, u64 **rbuf) |
| 89 | 100 | { |
| 90 | − int order = (PAGE_SIZE <= 4096) ? PAGE_ALLOC_COSTLY_ORDER : 0; | |
| 101 | + int order = NICVF_PAGE_ORDER; | |
| 91 | 102 | |
| 92 | 103 | /* Check if request can be accomodated in previous allocated page */ |
| 93 | 104 | if (nic->rb_page && |
| @@ -97,22 +108,27 @@ static inline int nicvf_alloc_rcv_buffer(struct nicvf *nic, gfp_t gfp, · 3 unmodified lines | ||
| 97 | 108 | } |
| 98 | 109 | |
| 99 | 110 | nicvf_get_page(nic); |
| 100 | − nic->rb_page = NULL; | |
| 101 | 111 | |
| 102 | 112 | /* Allocate a new page */ |
| 113 | + nic->rb_page = alloc_pages(gfp | __GFP_COMP | __GFP_NOWARN, | |
| 114 | + order); | |
| 103 | 115 | if (!nic->rb_page) { |
| 104 | − nic->rb_page = alloc_pages(gfp | __GFP_COMP | __GFP_NOWARN, | |
| 105 | − order); | |
| 106 | − if (!nic->rb_page) { | |
| 107 | − this_cpu_inc(nic->pnicvf->drv_stats-> | |
| 108 | − rcv_buffer_alloc_failures); | |
| 109 | − return -ENOMEM; | |
| 110 | − } | |
| 111 | − nic->rb_page_offset = 0; | |
| 116 | + this_cpu_inc(nic->pnicvf->drv_stats->rcv_buffer_alloc_failures); | |
| 117 | + return -ENOMEM; | |
| 112 | 118 | } |
| 113 | − | |
| 119 | + nic->rb_page_offset = 0; | |
| 114 | 120 | ret: |
| 115 | − *rbuf = (u64 *)((u64)page_address(nic->rb_page) + nic->rb_page_offset); | |
| 121 | + /* HW will ensure data coherency, CPU sync not required */ | |
| 122 | + *rbuf = (u64 *)((u64)dma_map_page_attrs(&nic->pdev->dev, nic->rb_page, | |
| 123 | + nic->rb_page_offset, buf_len, | |
| 124 | + DMA_FROM_DEVICE, | |
| 125 | + DMA_ATTR_SKIP_CPU_SYNC)); | |
| 126 | + if (dma_mapping_error(&nic->pdev->dev, (dma_addr_t)*rbuf)) { | |
| 127 | + if (!nic->rb_page_offset) | |
| 128 | + __free_pages(nic->rb_page, order); | |
| 129 | + nic->rb_page = NULL; | |
| 130 | + return -ENOMEM; | |
| 131 | + } | |
| 116 | 132 | nic->rb_page_offset += buf_len; |
| 117 | 133 | |
| 118 | 134 | return 0; |
| @@ -158,16 +174,21 @@ static int nicvf_init_rbdr(struct nicvf *nic, struct rbdr *rbdr, · 39 unmodified lines | ||
| 158 | 174 | rbdr->dma_size = buf_size; |
| 159 | 175 | rbdr->enable = true; |
| 160 | 176 | rbdr->thresh = RBDR_THRESH; |
| 177 | + rbdr->head = 0; | |
| 178 | + rbdr->tail = 0; | |
| 161 | 179 | |
| 162 | 180 | nic->rb_page = NULL; |
| 163 | 181 | for (idx = 0; idx < ring_len; idx++) { |
| 164 | 182 | err = nicvf_alloc_rcv_buffer(nic, GFP_KERNEL, RCV_FRAG_LEN, |
| 165 | 183 | &rbuf); |
| 166 | − if (err) | |
| 184 | + if (err) { | |
| 185 | + /* To free already allocated and mapped ones */ | |
| 186 | + rbdr->tail = idx - 1; | |
| 167 | 187 | return err; |
| 188 | + } | |
| 168 | 189 | |
| 169 | 190 | desc = GET_RBDR_DESC(rbdr, idx); |
| 170 | − desc->buf_addr = virt_to_phys(rbuf) >> NICVF_RCV_BUF_ALIGN; | |
| 191 | + desc->buf_addr = (u64)rbuf >> NICVF_RCV_BUF_ALIGN; | |
| 171 | 192 | } |
| 172 | 193 | |
| 173 | 194 | nicvf_get_page(nic); |
| @@ -179,7 +200,7 @@ static int nicvf_init_rbdr(struct nicvf *nic, struct rbdr *rbdr, · 5 unmodified lines | ||
| 179 | 200 | static void nicvf_free_rbdr(struct nicvf *nic, struct rbdr *rbdr) |
| 180 | 201 | { |
| 181 | 202 | int head, tail; |
| 182 | − u64 buf_addr; | |
| 203 | + u64 buf_addr, phys_addr; | |
| 183 | 204 | struct rbdr_entry_t *desc; |
| 184 | 205 | |
| 185 | 206 | if (!rbdr) |
| @@ -192,18 +213,26 @@ static void nicvf_free_rbdr(struct nicvf *nic, struct rbdr *rbdr) · 6 unmodified lines | ||
| 192 | 213 | head = rbdr->head; |
| 193 | 214 | tail = rbdr->tail; |
| 194 | 215 | |
| 195 | − /* Free SKBs */ | |
| 216 | + /* Release page references */ | |
| 196 | 217 | while (head != tail) { |
| 197 | 218 | desc = GET_RBDR_DESC(rbdr, head); |
| 198 | − buf_addr = desc->buf_addr << NICVF_RCV_BUF_ALIGN; | |
| 199 | − put_page(virt_to_page(phys_to_virt(buf_addr))); | |
| 219 | + buf_addr = ((u64)desc->buf_addr) << NICVF_RCV_BUF_ALIGN; | |
| 220 | + phys_addr = nicvf_iova_to_phys(nic, buf_addr); | |
| 221 | + dma_unmap_page_attrs(&nic->pdev->dev, buf_addr, RCV_FRAG_LEN, | |
| 222 | + DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); | |
| 223 | + if (phys_addr) | |
| 224 | + put_page(virt_to_page(phys_to_virt(phys_addr))); | |
| 200 | 225 | head++; |
| 201 | 226 | head &= (rbdr->dmem.q_len - 1); |
| 202 | 227 | } |
| 203 | − /* Free SKB of tail desc */ | |
| 228 | + /* Release buffer of tail desc */ | |
| 204 | 229 | desc = GET_RBDR_DESC(rbdr, tail); |
| 205 | − buf_addr = desc->buf_addr << NICVF_RCV_BUF_ALIGN; | |
| 206 | − put_page(virt_to_page(phys_to_virt(buf_addr))); | |
| 230 | + buf_addr = ((u64)desc->buf_addr) << NICVF_RCV_BUF_ALIGN; | |
| 231 | + phys_addr = nicvf_iova_to_phys(nic, buf_addr); | |
| 232 | + dma_unmap_page_attrs(&nic->pdev->dev, buf_addr, RCV_FRAG_LEN, | |
| 233 | + DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); | |
| 234 | + if (phys_addr) | |
| 235 | + put_page(virt_to_page(phys_to_virt(phys_addr))); | |
| 207 | 236 | |
| 208 | 237 | /* Free RBDR ring */ |
| 209 | 238 | nicvf_free_q_desc_mem(nic, &rbdr->dmem); |
| @@ -250,7 +279,7 @@ refill: · 40 unmodified lines | ||
| 250 | 279 | break; |
| 251 | 280 | |
| 252 | 281 | desc = GET_RBDR_DESC(rbdr, tail); |
| 253 | − desc->buf_addr = virt_to_phys(rbuf) >> NICVF_RCV_BUF_ALIGN; | |
| 282 | + desc->buf_addr = (u64)rbuf >> NICVF_RCV_BUF_ALIGN; | |
| 254 | 283 | refill_rb_cnt--; |
| 255 | 284 | new_rb++; |
| 256 | 285 | } |
| @@ -361,9 +390,29 @@ static int nicvf_init_snd_queue(struct nicvf *nic, · 104 unmodified lines | ||
| 361 | 390 | return 0; |
| 362 | 391 | } |
| 363 | 392 | |
| 393 | +void nicvf_unmap_sndq_buffers(struct nicvf *nic, struct snd_queue *sq, | |
| 394 | + int hdr_sqe, u8 subdesc_cnt) | |
| 395 | +{ | |
| 396 | + u8 idx; | |
| 397 | + struct sq_gather_subdesc *gather; | |
| 398 | + | |
| 399 | + /* Unmap DMA mapped skb data buffers */ | |
| 400 | + for (idx = 0; idx < subdesc_cnt; idx++) { | |
| 401 | + hdr_sqe++; | |
| 402 | + hdr_sqe &= (sq->dmem.q_len - 1); | |
| 403 | + gather = (struct sq_gather_subdesc *)GET_SQ_DESC(sq, hdr_sqe); | |
| 404 | + /* HW will ensure data coherency, CPU sync not required */ | |
| 405 | + dma_unmap_page_attrs(&nic->pdev->dev, gather->addr, | |
| 406 | + gather->size, DMA_TO_DEVICE, | |
| 407 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 408 | + } | |
| 409 | +} | |
| 410 | + | |
| 364 | 411 | static void nicvf_free_snd_queue(struct nicvf *nic, struct snd_queue *sq) |
| 365 | 412 | { |
| 366 | 413 | struct sk_buff *skb; |
| 414 | + struct sq_hdr_subdesc *hdr; | |
| 415 | + struct sq_hdr_subdesc *tso_sqe; | |
| 367 | 416 | |
| 368 | 417 | if (!sq) |
| 369 | 418 | return; |
| @@ -379,8 +428,22 @@ static void nicvf_free_snd_queue(struct nicvf *nic, struct snd_queue *sq) · 9 unmodified lines | ||
| 379 | 428 | smp_rmb(); |
| 380 | 429 | while (sq->head != sq->tail) { |
| 381 | 430 | skb = (struct sk_buff *)sq->skbuff[sq->head]; |
| 382 | − if (skb) | |
| 383 | − dev_kfree_skb_any(skb); | |
| 431 | + if (!skb) | |
| 432 | + goto next; | |
| 433 | + hdr = (struct sq_hdr_subdesc *)GET_SQ_DESC(sq, sq->head); | |
| 434 | + /* Check for dummy descriptor used for HW TSO offload on 88xx */ | |
| 435 | + if (hdr->dont_send) { | |
| 436 | + /* Get actual TSO descriptors and unmap them */ | |
| 437 | + tso_sqe = | |
| 438 | + (struct sq_hdr_subdesc *)GET_SQ_DESC(sq, hdr->rsvd2); | |
| 439 | + nicvf_unmap_sndq_buffers(nic, sq, hdr->rsvd2, | |
| 440 | + tso_sqe->subdesc_cnt); | |
| 441 | + } else { | |
| 442 | + nicvf_unmap_sndq_buffers(nic, sq, sq->head, | |
| 443 | + hdr->subdesc_cnt); | |
| 444 | + } | |
| 445 | + dev_kfree_skb_any(skb); | |
| 446 | +next: | |
| 384 | 447 | sq->head++; |
| 385 | 448 | sq->head &= (sq->dmem.q_len - 1); |
| 386 | 449 | } |
| @@ -559,9 +622,11 @@ static void nicvf_rcv_queue_config(struct nicvf *nic, struct queue_set *qs, · 172 unmodified lines | ||
| 559 | 622 | nicvf_send_msg_to_pf(nic, &mbx); |
| 560 | 623 | |
| 561 | 624 | if (!nic->sqs_mode && (qidx == 0)) { |
| 562 | − /* Enable checking L3/L4 length and TCP/UDP checksums */ | |
| 625 | + /* Enable checking L3/L4 length and TCP/UDP checksums | |
| 626 | + * Also allow IPv6 pkts with zero UDP checksum. | |
| 627 | + */ | |
| 563 | 628 | nicvf_queue_reg_write(nic, NIC_QSET_RQ_GEN_CFG, 0, |
| 564 | − (BIT(24) | BIT(23) | BIT(21))); | |
| 629 | + (BIT(24) | BIT(23) | BIT(21) | BIT(20))); | |
| 565 | 630 | nicvf_config_vlan_stripping(nic, nic->netdev->features); |
| 566 | 631 | } |
| 567 | 632 | |
| @@ -882,6 +947,14 @@ static inline int nicvf_get_sq_desc(struct snd_queue *sq, int desc_cnt) · 314 unmodified lines | ||
| 882 | 947 | return qentry; |
| 883 | 948 | } |
| 884 | 949 | |
| 950 | +/* Rollback to previous tail pointer when descriptors not used */ | |
| 951 | +static inline void nicvf_rollback_sq_desc(struct snd_queue *sq, | |
| 952 | + int qentry, int desc_cnt) | |
| 953 | +{ | |
| 954 | + sq->tail = qentry; | |
| 955 | + atomic_add(desc_cnt, &sq->free_cnt); | |
| 956 | +} | |
| 957 | + | |
| 885 | 958 | /* Free descriptor back to SQ for future use */ |
| 886 | 959 | void nicvf_put_sq_desc(struct snd_queue *sq, int desc_cnt) |
| 887 | 960 | { |
| @@ -1207,8 +1280,9 @@ int nicvf_sq_append_skb(struct nicvf *nic, struct snd_queue *sq, · 319 unmodified lines | ||
| 1207 | 1280 | struct sk_buff *skb, u8 sq_num) |
| 1208 | 1281 | { |
| 1209 | 1282 | int i, size; |
| 1210 | − int subdesc_cnt, tso_sqe = 0; | |
| 1283 | + int subdesc_cnt, hdr_sqe = 0; | |
| 1211 | 1284 | int qentry; |
| 1285 | + u64 dma_addr; | |
| 1212 | 1286 | |
| 1213 | 1287 | subdesc_cnt = nicvf_sq_subdesc_required(nic, skb); |
| 1214 | 1288 | if (subdesc_cnt > atomic_read(&sq->free_cnt)) |
| @@ -1223,12 +1297,21 @@ int nicvf_sq_append_skb(struct nicvf *nic, struct snd_queue *sq, · 8 unmodified lines | ||
| 1223 | 1297 | /* Add SQ header subdesc */ |
| 1224 | 1298 | nicvf_sq_add_hdr_subdesc(nic, sq, qentry, subdesc_cnt - 1, |
| 1225 | 1299 | skb, skb->len); |
| 1226 | − tso_sqe = qentry; | |
| 1300 | + hdr_sqe = qentry; | |
| 1227 | 1301 | |
| 1228 | 1302 | /* Add SQ gather subdescs */ |
| 1229 | 1303 | qentry = nicvf_get_nxt_sqentry(sq, qentry); |
| 1230 | 1304 | size = skb_is_nonlinear(skb) ? skb_headlen(skb) : skb->len; |
| 1231 | − nicvf_sq_add_gather_subdesc(sq, qentry, size, virt_to_phys(skb->data)); | |
| 1305 | + /* HW will ensure data coherency, CPU sync not required */ | |
| 1306 | + dma_addr = dma_map_page_attrs(&nic->pdev->dev, virt_to_page(skb->data), | |
| 1307 | + offset_in_page(skb->data), size, | |
| 1308 | + DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); | |
| 1309 | + if (dma_mapping_error(&nic->pdev->dev, dma_addr)) { | |
| 1310 | + nicvf_rollback_sq_desc(sq, qentry, subdesc_cnt); | |
| 1311 | + return 0; | |
| 1312 | + } | |
| 1313 | + | |
| 1314 | + nicvf_sq_add_gather_subdesc(sq, qentry, size, dma_addr); | |
| 1232 | 1315 | |
| 1233 | 1316 | /* Check for scattered buffer */ |
| 1234 | 1317 | if (!skb_is_nonlinear(skb)) |
| @@ -1241,15 +1324,26 @@ int nicvf_sq_append_skb(struct nicvf *nic, struct snd_queue *sq, · 6 unmodified lines | ||
| 1241 | 1324 | |
| 1242 | 1325 | qentry = nicvf_get_nxt_sqentry(sq, qentry); |
| 1243 | 1326 | size = skb_frag_size(frag); |
| 1244 | − nicvf_sq_add_gather_subdesc(sq, qentry, size, | |
| 1245 | − virt_to_phys( | |
| 1246 | − skb_frag_address(frag))); | |
| 1327 | + dma_addr = dma_map_page_attrs(&nic->pdev->dev, | |
| 1328 | + skb_frag_page(frag), | |
| 1329 | + frag->page_offset, size, | |
| 1330 | + DMA_TO_DEVICE, | |
| 1331 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 1332 | + if (dma_mapping_error(&nic->pdev->dev, dma_addr)) { | |
| 1333 | + /* Free entire chain of mapped buffers | |
| 1334 | + * here 'i' = frags mapped + above mapped skb->data | |
| 1335 | + */ | |
| 1336 | + nicvf_unmap_sndq_buffers(nic, sq, hdr_sqe, i); | |
| 1337 | + nicvf_rollback_sq_desc(sq, qentry, subdesc_cnt); | |
| 1338 | + return 0; | |
| 1339 | + } | |
| 1340 | + nicvf_sq_add_gather_subdesc(sq, qentry, size, dma_addr); | |
| 1247 | 1341 | } |
| 1248 | 1342 | |
| 1249 | 1343 | doorbell: |
| 1250 | 1344 | if (nic->t88 && skb_shinfo(skb)->gso_size) { |
| 1251 | 1345 | qentry = nicvf_get_nxt_sqentry(sq, qentry); |
| 1252 | − nicvf_sq_add_cqe_subdesc(sq, qentry, tso_sqe, skb); | |
| 1346 | + nicvf_sq_add_cqe_subdesc(sq, qentry, hdr_sqe, skb); | |
| 1253 | 1347 | } |
| 1254 | 1348 | |
| 1255 | 1349 | nicvf_sq_doorbell(nic, skb, sq_num, subdesc_cnt); |
| @@ -1282,6 +1376,7 @@ struct sk_buff *nicvf_get_rcv_skb(struct nicvf *nic, struct cqe_rx_t *cqe_rx) · 26 unmodified lines | ||
| 1282 | 1376 | int offset; |
| 1283 | 1377 | u16 *rb_lens = NULL; |
| 1284 | 1378 | u64 *rb_ptrs = NULL; |
| 1379 | + u64 phys_addr; | |
| 1285 | 1380 | |
| 1286 | 1381 | rb_lens = (void *)cqe_rx + (3 * sizeof(u64)); |
| 1287 | 1382 | /* Except 88xx pass1 on all other chips CQE_RX2_S is added to |
| @@ -1296,15 +1391,23 @@ struct sk_buff *nicvf_get_rcv_skb(struct nicvf *nic, struct cqe_rx_t *cqe_rx) · 8 unmodified lines | ||
| 1296 | 1391 | else |
| 1297 | 1392 | rb_ptrs = (void *)cqe_rx + (7 * sizeof(u64)); |
| 1298 | 1393 | |
| 1299 | − netdev_dbg(nic->netdev, "%s rb_cnt %d rb0_ptr %llx rb0_sz %d\n", | |
| 1300 | − __func__, cqe_rx->rb_cnt, cqe_rx->rb0_ptr, cqe_rx->rb0_sz); | |
| 1301 | − | |
| 1302 | 1394 | for (frag = 0; frag < cqe_rx->rb_cnt; frag++) { |
| 1303 | 1395 | payload_len = rb_lens[frag_num(frag)]; |
| 1396 | + phys_addr = nicvf_iova_to_phys(nic, *rb_ptrs); | |
| 1397 | + if (!phys_addr) { | |
| 1398 | + if (skb) | |
| 1399 | + dev_kfree_skb_any(skb); | |
| 1400 | + return NULL; | |
| 1401 | + } | |
| 1402 | + | |
| 1304 | 1403 | if (!frag) { |
| 1305 | 1404 | /* First fragment */ |
| 1405 | + dma_unmap_page_attrs(&nic->pdev->dev, | |
| 1406 | + *rb_ptrs - cqe_rx->align_pad, | |
| 1407 | + RCV_FRAG_LEN, DMA_FROM_DEVICE, | |
| 1408 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 1306 | 1409 | skb = nicvf_rb_ptr_to_skb(nic, |
| 1307 | − *rb_ptrs - cqe_rx->align_pad, | |
| 1410 | + phys_addr - cqe_rx->align_pad, | |
| 1308 | 1411 | payload_len); |
| 1309 | 1412 | if (!skb) |
| 1310 | 1413 | return NULL; |
| @@ -1312,8 +1415,11 @@ struct sk_buff *nicvf_get_rcv_skb(struct nicvf *nic, struct cqe_rx_t *cqe_rx) · 1 unmodified line | ||
| 1312 | 1415 | skb_put(skb, payload_len); |
| 1313 | 1416 | } else { |
| 1314 | 1417 | /* Add fragments */ |
| 1315 | − page = virt_to_page(phys_to_virt(*rb_ptrs)); | |
| 1316 | − offset = phys_to_virt(*rb_ptrs) - page_address(page); | |
| 1418 | + dma_unmap_page_attrs(&nic->pdev->dev, *rb_ptrs, | |
| 1419 | + RCV_FRAG_LEN, DMA_FROM_DEVICE, | |
| 1420 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 1421 | + page = virt_to_page(phys_to_virt(phys_addr)); | |
| 1422 | + offset = phys_to_virt(phys_addr) - page_address(page); | |
| 1317 | 1423 | skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, |
| 1318 | 1424 | offset, payload_len, RCV_FRAG_LEN); |
| 1319 | 1425 | } |
| @@ -87,7 +87,7 @@ · 86 unmodified lines | ||
| 87 | 87 | #define RCV_BUF_COUNT (1ULL << (RBDR_SIZE + 13)) |
| 88 | 88 | #define MAX_RCV_BUF_COUNT (1ULL << (RBDR_SIZE6 + 13)) |
| 89 | 89 | #define RBDR_THRESH (RCV_BUF_COUNT / 2) |
| 90 | −#define DMA_BUFFER_LEN 2048 /* In multiples of 128bytes */ | |
| 90 | +#define DMA_BUFFER_LEN 1536 /* In multiples of 128bytes */ | |
| 91 | 91 | #define RCV_FRAG_LEN (SKB_DATA_ALIGN(DMA_BUFFER_LEN + NET_SKB_PAD) + \ |
| 92 | 92 | SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) |
| 93 | 93 | |
| @@ -301,6 +301,8 @@ struct queue_set { · 207 unmodified lines | ||
| 301 | 301 | |
| 302 | 302 | #define CQ_ERR_MASK (CQ_WR_FULL | CQ_WR_DISABLE | CQ_WR_FAULT) |
| 303 | 303 | |
| 304 | +void nicvf_unmap_sndq_buffers(struct nicvf *nic, struct snd_queue *sq, | |
| 305 | + int hdr_sqe, u8 subdesc_cnt); | |
| 304 | 306 | void nicvf_config_vlan_stripping(struct nicvf *nic, |
| 305 | 307 | netdev_features_t features); |
| 306 | 308 | int nicvf_set_qset_resources(struct nicvf *nic); |
| @@ -123,14 +123,44 @@ static int bgx_poll_reg(struct bgx *bgx, u8 lmac, u64 reg, u64 mask, bool zero) · 122 unmodified lines | ||
| 123 | 123 | return 1; |
| 124 | 124 | } |
| 125 | 125 | |
| 126 | +static int max_bgx_per_node; | |
| 127 | +static void set_max_bgx_per_node(struct pci_dev *pdev) | |
| 128 | +{ | |
| 129 | + u16 sdevid; | |
| 130 | + | |
| 131 | + if (max_bgx_per_node) | |
| 132 | + return; | |
| 133 | + | |
| 134 | + pci_read_config_word(pdev, PCI_SUBSYSTEM_ID, &sdevid); | |
| 135 | + switch (sdevid) { | |
| 136 | + case PCI_SUBSYS_DEVID_81XX_BGX: | |
| 137 | + max_bgx_per_node = MAX_BGX_PER_CN81XX; | |
| 138 | + break; | |
| 139 | + case PCI_SUBSYS_DEVID_83XX_BGX: | |
| 140 | + max_bgx_per_node = MAX_BGX_PER_CN83XX; | |
| 141 | + break; | |
| 142 | + case PCI_SUBSYS_DEVID_88XX_BGX: | |
| 143 | + default: | |
| 144 | + max_bgx_per_node = MAX_BGX_PER_CN88XX; | |
| 145 | + break; | |
| 146 | + } | |
| 147 | +} | |
| 148 | + | |
| 149 | +static struct bgx *get_bgx(int node, int bgx_idx) | |
| 150 | +{ | |
| 151 | + int idx = (node * max_bgx_per_node) + bgx_idx; | |
| 152 | + | |
| 153 | + return bgx_vnic[idx]; | |
| 154 | +} | |
| 155 | + | |
| 126 | 156 | /* Return number of BGX present in HW */ |
| 127 | 157 | unsigned bgx_get_map(int node) |
| 128 | 158 | { |
| 129 | 159 | int i; |
| 130 | 160 | unsigned map = 0; |
| 131 | 161 | |
| 132 | − for (i = 0; i < MAX_BGX_PER_NODE; i++) { | |
| 133 | − if (bgx_vnic[(node * MAX_BGX_PER_NODE) + i]) | |
| 162 | + for (i = 0; i < max_bgx_per_node; i++) { | |
| 163 | + if (bgx_vnic[(node * max_bgx_per_node) + i]) | |
| 134 | 164 | map |= (1 << i); |
| 135 | 165 | } |
| 136 | 166 | |
| @@ -143,7 +173,7 @@ int bgx_get_lmac_count(int node, int bgx_idx) · 6 unmodified lines | ||
| 143 | 173 | { |
| 144 | 174 | struct bgx *bgx; |
| 145 | 175 | |
| 146 | − bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 176 | + bgx = get_bgx(node, bgx_idx); | |
| 147 | 177 | if (bgx) |
| 148 | 178 | return bgx->lmac_count; |
| 149 | 179 | |
| @@ -158,7 +188,7 @@ void bgx_get_lmac_link_state(int node, int bgx_idx, int lmacid, void *status) · 8 unmodified lines | ||
| 158 | 188 | struct bgx *bgx; |
| 159 | 189 | struct lmac *lmac; |
| 160 | 190 | |
| 161 | − bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 191 | + bgx = get_bgx(node, bgx_idx); | |
| 162 | 192 | if (!bgx) |
| 163 | 193 | return; |
| 164 | 194 | |
| @@ -172,7 +202,7 @@ EXPORT_SYMBOL(bgx_get_lmac_link_state); · 7 unmodified lines | ||
| 172 | 202 | |
| 173 | 203 | const u8 *bgx_get_lmac_mac(int node, int bgx_idx, int lmacid) |
| 174 | 204 | { |
| 175 | − struct bgx *bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 205 | + struct bgx *bgx = get_bgx(node, bgx_idx); | |
| 176 | 206 | |
| 177 | 207 | if (bgx) |
| 178 | 208 | return bgx->lmac[lmacid].mac; |
| @@ -183,7 +213,7 @@ EXPORT_SYMBOL(bgx_get_lmac_mac); · 4 unmodified lines | ||
| 183 | 213 | |
| 184 | 214 | void bgx_set_lmac_mac(int node, int bgx_idx, int lmacid, const u8 *mac) |
| 185 | 215 | { |
| 186 | − struct bgx *bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 216 | + struct bgx *bgx = get_bgx(node, bgx_idx); | |
| 187 | 217 | |
| 188 | 218 | if (!bgx) |
| 189 | 219 | return; |
| @@ -194,7 +224,7 @@ EXPORT_SYMBOL(bgx_set_lmac_mac); · 4 unmodified lines | ||
| 194 | 224 | |
| 195 | 225 | void bgx_lmac_rx_tx_enable(int node, int bgx_idx, int lmacid, bool enable) |
| 196 | 226 | { |
| 197 | − struct bgx *bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 227 | + struct bgx *bgx = get_bgx(node, bgx_idx); | |
| 198 | 228 | struct lmac *lmac; |
| 199 | 229 | u64 cfg; |
| 200 | 230 | |
| @@ -217,7 +247,7 @@ EXPORT_SYMBOL(bgx_lmac_rx_tx_enable); · 16 unmodified lines | ||
| 217 | 247 | void bgx_lmac_get_pfc(int node, int bgx_idx, int lmacid, void *pause) |
| 218 | 248 | { |
| 219 | 249 | struct pfc *pfc = (struct pfc *)pause; |
| 220 | − struct bgx *bgx = bgx_vnic[(node * MAX_BGX_PER_CN88XX) + bgx_idx]; | |
| 250 | + struct bgx *bgx = get_bgx(node, bgx_idx); | |
| 221 | 251 | struct lmac *lmac; |
| 222 | 252 | u64 cfg; |
| 223 | 253 | |
| @@ -237,7 +267,7 @@ EXPORT_SYMBOL(bgx_lmac_get_pfc); · 13 unmodified lines | ||
| 237 | 267 | void bgx_lmac_set_pfc(int node, int bgx_idx, int lmacid, void *pause) |
| 238 | 268 | { |
| 239 | 269 | struct pfc *pfc = (struct pfc *)pause; |
| 240 | − struct bgx *bgx = bgx_vnic[(node * MAX_BGX_PER_CN88XX) + bgx_idx]; | |
| 270 | + struct bgx *bgx = get_bgx(node, bgx_idx); | |
| 241 | 271 | struct lmac *lmac; |
| 242 | 272 | u64 cfg; |
| 243 | 273 | |
| @@ -369,7 +399,7 @@ u64 bgx_get_rx_stats(int node, int bgx_idx, int lmac, int idx) · 125 unmodified lines | ||
| 369 | 399 | { |
| 370 | 400 | struct bgx *bgx; |
| 371 | 401 | |
| 372 | − bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 402 | + bgx = get_bgx(node, bgx_idx); | |
| 373 | 403 | if (!bgx) |
| 374 | 404 | return 0; |
| 375 | 405 | |
| @@ -383,7 +413,7 @@ u64 bgx_get_tx_stats(int node, int bgx_idx, int lmac, int idx) · 7 unmodified lines | ||
| 383 | 413 | { |
| 384 | 414 | struct bgx *bgx; |
| 385 | 415 | |
| 386 | − bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 416 | + bgx = get_bgx(node, bgx_idx); | |
| 387 | 417 | if (!bgx) |
| 388 | 418 | return 0; |
| 389 | 419 | |
| @@ -411,7 +441,7 @@ void bgx_lmac_internal_loopback(int node, int bgx_idx, · 21 unmodified lines | ||
| 411 | 441 | struct lmac *lmac; |
| 412 | 442 | u64 cfg; |
| 413 | 443 | |
| 414 | − bgx = bgx_vnic[(node * MAX_BGX_PER_NODE) + bgx_idx]; | |
| 444 | + bgx = get_bgx(node, bgx_idx); | |
| 415 | 445 | if (!bgx) |
| 416 | 446 | return; |
| 417 | 447 | |
| @@ -1011,12 +1041,6 @@ static void bgx_print_qlm_mode(struct bgx *bgx, u8 lmacid) · 593 unmodified lines | ||
| 1011 | 1041 | dev_info(dev, "%s: 40G_KR4\n", (char *)str); |
| 1012 | 1042 | break; |
| 1013 | 1043 | case BGX_MODE_QSGMII: |
| 1014 | − if ((lmacid == 0) && | |
| 1015 | − (bgx_get_lane2sds_cfg(bgx, lmac) != lmacid)) | |
| 1016 | − return; | |
| 1017 | − if ((lmacid == 2) && | |
| 1018 | − (bgx_get_lane2sds_cfg(bgx, lmac) == lmacid)) | |
| 1019 | − return; | |
| 1020 | 1044 | dev_info(dev, "%s: QSGMII\n", (char *)str); |
| 1021 | 1045 | break; |
| 1022 | 1046 | case BGX_MODE_RGMII: |
| @@ -1334,11 +1358,13 @@ static int bgx_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 311 unmodified lines | ||
| 1334 | 1358 | goto err_release_regions; |
| 1335 | 1359 | } |
| 1336 | 1360 | |
| 1361 | + set_max_bgx_per_node(pdev); | |
| 1362 | + | |
| 1337 | 1363 | pci_read_config_word(pdev, PCI_DEVICE_ID, &sdevid); |
| 1338 | 1364 | if (sdevid != PCI_DEVICE_ID_THUNDER_RGX) { |
| 1339 | 1365 | bgx->bgx_id = (pci_resource_start(pdev, |
| 1340 | 1366 | PCI_CFG_REG_BAR_NUM) >> 24) & BGX_ID_MASK; |
| 1341 | − bgx->bgx_id += nic_get_node_id(pdev) * MAX_BGX_PER_NODE; | |
| 1367 | + bgx->bgx_id += nic_get_node_id(pdev) * max_bgx_per_node; | |
| 1342 | 1368 | bgx->max_lmac = MAX_LMAC_PER_BGX; |
| 1343 | 1369 | bgx_vnic[bgx->bgx_id] = bgx; |
| 1344 | 1370 | } else { |
| @@ -22,7 +22,6 @@ · 21 unmodified lines | ||
| 22 | 22 | #define MAX_BGX_PER_CN88XX 2 |
| 23 | 23 | #define MAX_BGX_PER_CN81XX 3 /* 2 BGXs + 1 RGX */ |
| 24 | 24 | #define MAX_BGX_PER_CN83XX 4 |
| 25 | −#define MAX_BGX_PER_NODE 4 | |
| 26 | 25 | #define MAX_LMAC_PER_BGX 4 |
| 27 | 26 | #define MAX_BGX_CHANS_PER_LMAC 16 |
| 28 | 27 | #define MAX_DMAC_PER_LMAC 8 |
| @@ -2589,8 +2589,6 @@ static int emac_dt_mdio_probe(struct emac_instance *dev) · 2588 unmodified lines | ||
| 2589 | 2589 | static int emac_dt_phy_connect(struct emac_instance *dev, |
| 2590 | 2590 | struct device_node *phy_handle) |
| 2591 | 2591 | { |
| 2592 | − int res; | |
| 2593 | − | |
| 2594 | 2592 | dev->phy.def = devm_kzalloc(&dev->ofdev->dev, sizeof(*dev->phy.def), |
| 2595 | 2593 | GFP_KERNEL); |
| 2596 | 2594 | if (!dev->phy.def) |
| @@ -2617,7 +2615,7 @@ static int emac_dt_phy_probe(struct emac_instance *dev) · 20 unmodified lines | ||
| 2617 | 2615 | { |
| 2618 | 2616 | struct device_node *np = dev->ofdev->dev.of_node; |
| 2619 | 2617 | struct device_node *phy_handle; |
| 2620 | − int res = 0; | |
| 2618 | + int res = 1; | |
| 2621 | 2619 | |
| 2622 | 2620 | phy_handle = of_parse_phandle(np, "phy-handle", 0); |
| 2623 | 2621 | |
| @@ -2714,13 +2712,24 @@ static int emac_init_phy(struct emac_instance *dev) · 90 unmodified lines | ||
| 2714 | 2712 | if (emac_has_feature(dev, EMAC_FTR_HAS_RGMII)) { |
| 2715 | 2713 | int res = emac_dt_phy_probe(dev); |
| 2716 | 2714 | |
| 2717 | − mutex_unlock(&emac_phy_map_lock); | |
| 2718 | − if (!res) | |
| 2715 | + switch (res) { | |
| 2716 | + case 1: | |
| 2717 | + /* No phy-handle property configured. | |
| 2718 | + * Continue with the existing phy probe | |
| 2719 | + * and setup code. | |
| 2720 | + */ | |
| 2721 | + break; | |
| 2722 | + | |
| 2723 | + case 0: | |
| 2724 | + mutex_unlock(&emac_phy_map_lock); | |
| 2719 | 2725 | goto init_phy; |
| 2720 | 2726 | |
| 2721 | − dev_err(&dev->ofdev->dev, "failed to attach dt phy (%d).\n", | |
| 2722 | − res); | |
| 2723 | − return res; | |
| 2727 | + default: | |
| 2728 | + mutex_unlock(&emac_phy_map_lock); | |
| 2729 | + dev_err(&dev->ofdev->dev, "failed to attach dt phy (%d).\n", | |
| 2730 | + res); | |
| 2731 | + return res; | |
| 2732 | + } | |
| 2724 | 2733 | } |
| 2725 | 2734 | |
| 2726 | 2735 | if (dev->phy_address != 0xffffffff) |
| @@ -404,7 +404,7 @@ static int ibmvnic_open(struct net_device *netdev) · 403 unmodified lines | ||
| 404 | 404 | send_map_query(adapter); |
| 405 | 405 | for (i = 0; i < rxadd_subcrqs; i++) { |
| 406 | 406 | init_rx_pool(adapter, &adapter->rx_pool[i], |
| 407 | − IBMVNIC_BUFFS_PER_POOL, i, | |
| 407 | + adapter->req_rx_add_entries_per_subcrq, i, | |
| 408 | 408 | be64_to_cpu(size_array[i]), 1); |
| 409 | 409 | if (alloc_rx_pool(adapter, &adapter->rx_pool[i])) { |
| 410 | 410 | dev_err(dev, "Couldn't alloc rx pool\n"); |
| @@ -419,23 +419,23 @@ static int ibmvnic_open(struct net_device *netdev) · 8 unmodified lines | ||
| 419 | 419 | for (i = 0; i < tx_subcrqs; i++) { |
| 420 | 420 | tx_pool = &adapter->tx_pool[i]; |
| 421 | 421 | tx_pool->tx_buff = |
| 422 | − kcalloc(adapter->max_tx_entries_per_subcrq, | |
| 422 | + kcalloc(adapter->req_tx_entries_per_subcrq, | |
| 423 | 423 | sizeof(struct ibmvnic_tx_buff), GFP_KERNEL); |
| 424 | 424 | if (!tx_pool->tx_buff) |
| 425 | 425 | goto tx_pool_alloc_failed; |
| 426 | 426 | |
| 427 | 427 | if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, |
| 428 | − adapter->max_tx_entries_per_subcrq * | |
| 428 | + adapter->req_tx_entries_per_subcrq * | |
| 429 | 429 | adapter->req_mtu)) |
| 430 | 430 | goto tx_ltb_alloc_failed; |
| 431 | 431 | |
| 432 | 432 | tx_pool->free_map = |
| 433 | − kcalloc(adapter->max_tx_entries_per_subcrq, | |
| 433 | + kcalloc(adapter->req_tx_entries_per_subcrq, | |
| 434 | 434 | sizeof(int), GFP_KERNEL); |
| 435 | 435 | if (!tx_pool->free_map) |
| 436 | 436 | goto tx_fm_alloc_failed; |
| 437 | 437 | |
| 438 | − for (j = 0; j < adapter->max_tx_entries_per_subcrq; j++) | |
| 438 | + for (j = 0; j < adapter->req_tx_entries_per_subcrq; j++) | |
| 439 | 439 | tx_pool->free_map[j] = j; |
| 440 | 440 | |
| 441 | 441 | tx_pool->consumer_index = 0; |
| @@ -705,6 +705,7 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) · 263 unmodified lines | ||
| 705 | 705 | u8 *hdrs = (u8 *)&adapter->tx_rx_desc_req; |
| 706 | 706 | struct device *dev = &adapter->vdev->dev; |
| 707 | 707 | struct ibmvnic_tx_buff *tx_buff = NULL; |
| 708 | + struct ibmvnic_sub_crq_queue *tx_scrq; | |
| 708 | 709 | struct ibmvnic_tx_pool *tx_pool; |
| 709 | 710 | unsigned int tx_send_failed = 0; |
| 710 | 711 | unsigned int tx_map_failed = 0; |
| @@ -724,6 +725,7 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) · 13 unmodified lines | ||
| 724 | 725 | int ret = 0; |
| 725 | 726 | |
| 726 | 727 | tx_pool = &adapter->tx_pool[queue_num]; |
| 728 | + tx_scrq = adapter->tx_scrq[queue_num]; | |
| 727 | 729 | txq = netdev_get_tx_queue(netdev, skb_get_queue_mapping(skb)); |
| 728 | 730 | handle_array = (u64 *)((u8 *)(adapter->login_rsp_buf) + |
| 729 | 731 | be32_to_cpu(adapter->login_rsp_buf-> |
| @@ -744,7 +746,7 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) · 14 unmodified lines | ||
| 744 | 746 | |
| 745 | 747 | tx_pool->consumer_index = |
| 746 | 748 | (tx_pool->consumer_index + 1) % |
| 747 | − adapter->max_tx_entries_per_subcrq; | |
| 749 | + adapter->req_tx_entries_per_subcrq; | |
| 748 | 750 | |
| 749 | 751 | tx_buff = &tx_pool->tx_buff[index]; |
| 750 | 752 | tx_buff->skb = skb; |
| @@ -817,7 +819,7 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) · 66 unmodified lines | ||
| 817 | 819 | |
| 818 | 820 | if (tx_pool->consumer_index == 0) |
| 819 | 821 | tx_pool->consumer_index = |
| 820 | − adapter->max_tx_entries_per_subcrq - 1; | |
| 822 | + adapter->req_tx_entries_per_subcrq - 1; | |
| 821 | 823 | else |
| 822 | 824 | tx_pool->consumer_index--; |
| 823 | 825 | |
| @@ -826,6 +828,14 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) · 2 unmodified lines | ||
| 826 | 828 | ret = NETDEV_TX_BUSY; |
| 827 | 829 | goto out; |
| 828 | 830 | } |
| 831 | + | |
| 832 | + atomic_inc(&tx_scrq->used); | |
| 833 | + | |
| 834 | + if (atomic_read(&tx_scrq->used) >= adapter->req_tx_entries_per_subcrq) { | |
| 835 | + netdev_info(netdev, "Stopping queue %d\n", queue_num); | |
| 836 | + netif_stop_subqueue(netdev, queue_num); | |
| 837 | + } | |
| 838 | + | |
| 829 | 839 | tx_packets++; |
| 830 | 840 | tx_bytes += skb->len; |
| 831 | 841 | txq->trans_start = jiffies; |
| @@ -1213,6 +1223,7 @@ static struct ibmvnic_sub_crq_queue *init_sub_crq_queue(struct ibmvnic_adapter · 381 unmodified lines | ||
| 1213 | 1223 | scrq->adapter = adapter; |
| 1214 | 1224 | scrq->size = 4 * PAGE_SIZE / sizeof(*scrq->msgs); |
| 1215 | 1225 | scrq->cur = 0; |
| 1226 | + atomic_set(&scrq->used, 0); | |
| 1216 | 1227 | scrq->rx_skb_top = NULL; |
| 1217 | 1228 | spin_lock_init(&scrq->lock); |
| 1218 | 1229 | |
| @@ -1355,14 +1366,28 @@ restart_loop: · 136 unmodified lines | ||
| 1355 | 1366 | DMA_TO_DEVICE); |
| 1356 | 1367 | } |
| 1357 | 1368 | |
| 1358 | − if (txbuff->last_frag) | |
| 1369 | + if (txbuff->last_frag) { | |
| 1370 | + atomic_dec(&scrq->used); | |
| 1371 | + | |
| 1372 | + if (atomic_read(&scrq->used) <= | |
| 1373 | + (adapter->req_tx_entries_per_subcrq / 2) && | |
| 1374 | + netif_subqueue_stopped(adapter->netdev, | |
| 1375 | + txbuff->skb)) { | |
| 1376 | + netif_wake_subqueue(adapter->netdev, | |
| 1377 | + scrq->pool_index); | |
| 1378 | + netdev_dbg(adapter->netdev, | |
| 1379 | + "Started queue %d\n", | |
| 1380 | + scrq->pool_index); | |
| 1381 | + } | |
| 1382 | + | |
| 1359 | 1383 | dev_kfree_skb_any(txbuff->skb); |
| 1384 | + } | |
| 1360 | 1385 | |
| 1361 | 1386 | adapter->tx_pool[pool].free_map[adapter->tx_pool[pool]. |
| 1362 | 1387 | producer_index] = index; |
| 1363 | 1388 | adapter->tx_pool[pool].producer_index = |
| 1364 | 1389 | (adapter->tx_pool[pool].producer_index + 1) % |
| 1365 | − adapter->max_tx_entries_per_subcrq; | |
| 1390 | + adapter->req_tx_entries_per_subcrq; | |
| 1366 | 1391 | } |
| 1367 | 1392 | /* remove tx_comp scrq*/ |
| 1368 | 1393 | next->tx_comp.first = 0; |
| @@ -863,6 +863,7 @@ struct ibmvnic_sub_crq_queue { · 862 unmodified lines | ||
| 863 | 863 | spinlock_t lock; |
| 864 | 864 | struct sk_buff *rx_skb_top; |
| 865 | 865 | struct ibmvnic_adapter *adapter; |
| 866 | + atomic_t used; | |
| 866 | 867 | }; |
| 867 | 868 | |
| 868 | 869 | struct ibmvnic_long_term_buff { |
| @@ -14,6 +14,7 @@ config MLX5_CORE · 13 unmodified lines | ||
| 14 | 14 | config MLX5_CORE_EN |
| 15 | 15 | bool "Mellanox Technologies ConnectX-4 Ethernet support" |
| 16 | 16 | depends on NETDEVICES && ETHERNET && PCI && MLX5_CORE |
| 17 | + depends on IPV6=y || IPV6=n || MLX5_CORE=m | |
| 17 | 18 | imply PTP_1588_CLOCK |
| 18 | 19 | default n |
| 19 | 20 | ---help--- |
| @@ -302,6 +302,9 @@ static u8 mlx5e_dcbnl_setdcbx(struct net_device *dev, u8 mode) · 301 unmodified lines | ||
| 302 | 302 | struct mlx5e_priv *priv = netdev_priv(dev); |
| 303 | 303 | struct mlx5e_dcbx *dcbx = &priv->dcbx; |
| 304 | 304 | |
| 305 | + if (mode & DCB_CAP_DCBX_LLD_MANAGED) | |
| 306 | + return 1; | |
| 307 | + | |
| 305 | 308 | if ((!mode) && MLX5_CAP_GEN(priv->mdev, dcbx)) { |
| 306 | 309 | if (dcbx->mode == MLX5E_DCBX_PARAM_VER_OPER_AUTO) |
| 307 | 310 | return 0; |
| @@ -315,13 +318,10 @@ static u8 mlx5e_dcbnl_setdcbx(struct net_device *dev, u8 mode) · 7 unmodified lines | ||
| 315 | 318 | return 1; |
| 316 | 319 | } |
| 317 | 320 | |
| 318 | − if (mlx5e_dcbnl_switch_to_host_mode(netdev_priv(dev))) | |
| 321 | + if (!(mode & DCB_CAP_DCBX_HOST)) | |
| 319 | 322 | return 1; |
| 320 | 323 | |
| 321 | − if ((mode & DCB_CAP_DCBX_LLD_MANAGED) || | |
| 322 | − !(mode & DCB_CAP_DCBX_VER_CEE) || | |
| 323 | − !(mode & DCB_CAP_DCBX_VER_IEEE) || | |
| 324 | − !(mode & DCB_CAP_DCBX_HOST)) | |
| 324 | + if (mlx5e_dcbnl_switch_to_host_mode(netdev_priv(dev))) | |
| 325 | 325 | return 1; |
| 326 | 326 | |
| 327 | 327 | return 0; |
| @@ -204,9 +204,6 @@ mlx5e_test_loopback_validate(struct sk_buff *skb, · 203 unmodified lines | ||
| 204 | 204 | struct iphdr *iph; |
| 205 | 205 | |
| 206 | 206 | /* We are only going to peek, no need to clone the SKB */ |
| 207 | − if (skb->protocol != htons(ETH_P_IP)) | |
| 208 | − goto out; | |
| 209 | − | |
| 210 | 207 | if (MLX5E_TEST_PKT_SIZE - ETH_HLEN > skb_headlen(skb)) |
| 211 | 208 | goto out; |
| 212 | 209 | |
| @@ -249,7 +246,7 @@ static int mlx5e_test_loopback_setup(struct mlx5e_priv *priv, · 36 unmodified lines | ||
| 249 | 246 | lbtp->loopback_ok = false; |
| 250 | 247 | init_completion(&lbtp->comp); |
| 251 | 248 | |
| 252 | − lbtp->pt.type = htons(ETH_P_ALL); | |
| 249 | + lbtp->pt.type = htons(ETH_P_IP); | |
| 253 | 250 | lbtp->pt.func = mlx5e_test_loopback_validate; |
| 254 | 251 | lbtp->pt.dev = priv->netdev; |
| 255 | 252 | lbtp->pt.af_packet_priv = lbtp; |
| @@ -48,9 +48,14 @@ · 47 unmodified lines | ||
| 48 | 48 | #include "eswitch.h" |
| 49 | 49 | #include "vxlan.h" |
| 50 | 50 | |
| 51 | +enum { | |
| 52 | + MLX5E_TC_FLOW_ESWITCH = BIT(0), | |
| 53 | +}; | |
| 54 | + | |
| 51 | 55 | struct mlx5e_tc_flow { |
| 52 | 56 | struct rhash_head node; |
| 53 | 57 | u64 cookie; |
| 58 | + u8 flags; | |
| 54 | 59 | struct mlx5_flow_handle *rule; |
| 55 | 60 | struct list_head encap; /* flows sharing the same encap */ |
| 56 | 61 | struct mlx5_esw_flow_attr *attr; |
| @@ -177,7 +182,7 @@ static void mlx5e_tc_del_flow(struct mlx5e_priv *priv, · 120 unmodified lines | ||
| 177 | 182 | mlx5_fc_destroy(priv->mdev, counter); |
| 178 | 183 | } |
| 179 | 184 | |
| 180 | − if (esw && esw->mode == SRIOV_OFFLOADS) { | |
| 185 | + if (flow->flags & MLX5E_TC_FLOW_ESWITCH) { | |
| 181 | 186 | mlx5_eswitch_del_vlan_action(esw, flow->attr); |
| 182 | 187 | if (flow->attr->action & MLX5_FLOW_CONTEXT_ACTION_ENCAP) |
| 183 | 188 | mlx5e_detach_encap(priv, flow); |
| @@ -598,6 +603,7 @@ static int __parse_cls_flower(struct mlx5e_priv *priv, · 414 unmodified lines | ||
| 598 | 603 | } |
| 599 | 604 | |
| 600 | 605 | static int parse_cls_flower(struct mlx5e_priv *priv, |
| 606 | + struct mlx5e_tc_flow *flow, | |
| 601 | 607 | struct mlx5_flow_spec *spec, |
| 602 | 608 | struct tc_cls_flower_offload *f) |
| 603 | 609 | { |
| @@ -609,7 +615,7 @@ static int parse_cls_flower(struct mlx5e_priv *priv, · 5 unmodified lines | ||
| 609 | 615 | |
| 610 | 616 | err = __parse_cls_flower(priv, spec, f, &min_inline); |
| 611 | 617 | |
| 612 | − if (!err && esw->mode == SRIOV_OFFLOADS && | |
| 618 | + if (!err && (flow->flags & MLX5E_TC_FLOW_ESWITCH) && | |
| 613 | 619 | rep->vport != FDB_UPLINK_VPORT) { |
| 614 | 620 | if (min_inline > esw->offloads.inline_mode) { |
| 615 | 621 | netdev_warn(priv->netdev, |
| @@ -1132,23 +1138,19 @@ int mlx5e_configure_flower(struct mlx5e_priv *priv, __be16 protocol, · 516 unmodified lines | ||
| 1132 | 1138 | struct tc_cls_flower_offload *f) |
| 1133 | 1139 | { |
| 1134 | 1140 | struct mlx5e_tc_table *tc = &priv->fs.tc; |
| 1135 | − int err = 0; | |
| 1136 | − bool fdb_flow = false; | |
| 1141 | + int err, attr_size = 0; | |
| 1137 | 1142 | u32 flow_tag, action; |
| 1138 | 1143 | struct mlx5e_tc_flow *flow; |
| 1139 | 1144 | struct mlx5_flow_spec *spec; |
| 1140 | 1145 | struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; |
| 1146 | + u8 flow_flags = 0; | |
| 1141 | 1147 | |
| 1142 | − if (esw && esw->mode == SRIOV_OFFLOADS) | |
| 1143 | − fdb_flow = true; | |
| 1144 | − | |
| 1145 | − if (fdb_flow) | |
| 1146 | − flow = kzalloc(sizeof(*flow) + | |
| 1147 | − sizeof(struct mlx5_esw_flow_attr), | |
| 1148 | − GFP_KERNEL); | |
| 1149 | − else | |
| 1150 | − flow = kzalloc(sizeof(*flow), GFP_KERNEL); | |
| 1148 | + if (esw && esw->mode == SRIOV_OFFLOADS) { | |
| 1149 | + flow_flags = MLX5E_TC_FLOW_ESWITCH; | |
| 1150 | + attr_size = sizeof(struct mlx5_esw_flow_attr); | |
| 1151 | + } | |
| 1151 | 1152 | |
| 1153 | + flow = kzalloc(sizeof(*flow) + attr_size, GFP_KERNEL); | |
| 1152 | 1154 | spec = mlx5_vzalloc(sizeof(*spec)); |
| 1153 | 1155 | if (!spec || !flow) { |
| 1154 | 1156 | err = -ENOMEM; |
| @@ -1156,12 +1158,13 @@ int mlx5e_configure_flower(struct mlx5e_priv *priv, __be16 protocol, · 1 unmodified line | ||
| 1156 | 1158 | } |
| 1157 | 1159 | |
| 1158 | 1160 | flow->cookie = f->cookie; |
| 1161 | + flow->flags = flow_flags; | |
| 1159 | 1162 | |
| 1160 | − err = parse_cls_flower(priv, spec, f); | |
| 1163 | + err = parse_cls_flower(priv, flow, spec, f); | |
| 1161 | 1164 | if (err < 0) |
| 1162 | 1165 | goto err_free; |
| 1163 | 1166 | |
| 1164 | − if (fdb_flow) { | |
| 1167 | + if (flow->flags & MLX5E_TC_FLOW_ESWITCH) { | |
| 1165 | 1168 | flow->attr = (struct mlx5_esw_flow_attr *)(flow + 1); |
| 1166 | 1169 | err = parse_tc_fdb_actions(priv, f->exts, flow); |
| 1167 | 1170 | if (err < 0) |
| @@ -1136,7 +1136,7 @@ static struct mlx5_flow_group *create_autogroup(struct mlx5_flow_table *ft, · 1135 unmodified lines | ||
| 1136 | 1136 | u32 *match_criteria) |
| 1137 | 1137 | { |
| 1138 | 1138 | int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); |
| 1139 | − struct list_head *prev = ft->node.children.prev; | |
| 1139 | + struct list_head *prev = &ft->node.children; | |
| 1140 | 1140 | unsigned int candidate_index = 0; |
| 1141 | 1141 | struct mlx5_flow_group *fg; |
| 1142 | 1142 | void *match_criteria_addr; |
| @@ -1352,6 +1352,7 @@ static int init_one(struct pci_dev *pdev, · 1351 unmodified lines | ||
| 1352 | 1352 | if (err) |
| 1353 | 1353 | goto clean_load; |
| 1354 | 1354 | |
| 1355 | + pci_save_state(pdev); | |
| 1355 | 1356 | return 0; |
| 1356 | 1357 | |
| 1357 | 1358 | clean_load: |
| @@ -1407,9 +1408,8 @@ static pci_ers_result_t mlx5_pci_err_detected(struct pci_dev *pdev, · 49 unmodified lines | ||
| 1407 | 1408 | |
| 1408 | 1409 | mlx5_enter_error_state(dev); |
| 1409 | 1410 | mlx5_unload_one(dev, priv, false); |
| 1410 | − /* In case of kernel call save the pci state and drain the health wq */ | |
| 1411 | + /* In case of kernel call drain the health wq */ | |
| 1411 | 1412 | if (state) { |
| 1412 | − pci_save_state(pdev); | |
| 1413 | 1413 | mlx5_drain_health_wq(dev); |
| 1414 | 1414 | mlx5_pci_disable_device(dev); |
| 1415 | 1415 | } |
| @@ -1461,6 +1461,7 @@ static pci_ers_result_t mlx5_pci_slot_reset(struct pci_dev *pdev) · 45 unmodified lines | ||
| 1461 | 1461 | |
| 1462 | 1462 | pci_set_master(pdev); |
| 1463 | 1463 | pci_restore_state(pdev); |
| 1464 | + pci_save_state(pdev); | |
| 1464 | 1465 | |
| 1465 | 1466 | if (wait_vital(pdev)) { |
| 1466 | 1467 | dev_err(&pdev->dev, "%s: wait_vital timed out\n", __func__); |
| @@ -769,7 +769,7 @@ static inline void mlxsw_reg_spvid_pack(char *payload, u8 local_port, u16 pvid) · 768 unmodified lines | ||
| 769 | 769 | #define MLXSW_REG_SPVM_ID 0x200F |
| 770 | 770 | #define MLXSW_REG_SPVM_BASE_LEN 0x04 /* base length, without records */ |
| 771 | 771 | #define MLXSW_REG_SPVM_REC_LEN 0x04 /* record length */ |
| 772 | −#define MLXSW_REG_SPVM_REC_MAX_COUNT 256 | |
| 772 | +#define MLXSW_REG_SPVM_REC_MAX_COUNT 255 | |
| 773 | 773 | #define MLXSW_REG_SPVM_LEN (MLXSW_REG_SPVM_BASE_LEN + \ |
| 774 | 774 | MLXSW_REG_SPVM_REC_LEN * MLXSW_REG_SPVM_REC_MAX_COUNT) |
| 775 | 775 | |
| @@ -1702,7 +1702,7 @@ static inline void mlxsw_reg_sfmr_pack(char *payload, · 926 unmodified lines | ||
| 1702 | 1702 | #define MLXSW_REG_SPVMLR_ID 0x2020 |
| 1703 | 1703 | #define MLXSW_REG_SPVMLR_BASE_LEN 0x04 /* base length, without records */ |
| 1704 | 1704 | #define MLXSW_REG_SPVMLR_REC_LEN 0x04 /* record length */ |
| 1705 | −#define MLXSW_REG_SPVMLR_REC_MAX_COUNT 256 | |
| 1705 | +#define MLXSW_REG_SPVMLR_REC_MAX_COUNT 255 | |
| 1706 | 1706 | #define MLXSW_REG_SPVMLR_LEN (MLXSW_REG_SPVMLR_BASE_LEN + \ |
| 1707 | 1707 | MLXSW_REG_SPVMLR_REC_LEN * \ |
| 1708 | 1708 | MLXSW_REG_SPVMLR_REC_MAX_COUNT) |
| @@ -303,11 +303,11 @@ void mlxsw_sp_flower_destroy(struct mlxsw_sp_port *mlxsw_sp_port, bool ingress, · 302 unmodified lines | ||
| 303 | 303 | ruleset = mlxsw_sp_acl_ruleset_get(mlxsw_sp, mlxsw_sp_port->dev, |
| 304 | 304 | ingress, |
| 305 | 305 | MLXSW_SP_ACL_PROFILE_FLOWER); |
| 306 | − if (WARN_ON(IS_ERR(ruleset))) | |
| 306 | + if (IS_ERR(ruleset)) | |
| 307 | 307 | return; |
| 308 | 308 | |
| 309 | 309 | rule = mlxsw_sp_acl_rule_lookup(mlxsw_sp, ruleset, f->cookie); |
| 310 | − if (!WARN_ON(!rule)) { | |
| 310 | + if (rule) { | |
| 311 | 311 | mlxsw_sp_acl_rule_del(mlxsw_sp, rule); |
| 312 | 312 | mlxsw_sp_acl_rule_destroy(mlxsw_sp, rule); |
| 313 | 313 | } |
| @@ -422,8 +422,9 @@ static void qed_cxt_set_proto_cid_count(struct qed_hwfn *p_hwfn, · 421 unmodified lines | ||
| 422 | 422 | u32 page_sz = p_mgr->clients[ILT_CLI_CDUC].p_size.val; |
| 423 | 423 | u32 cxt_size = CONN_CXT_SIZE(p_hwfn); |
| 424 | 424 | u32 elems_per_page = ILT_PAGE_IN_BYTES(page_sz) / cxt_size; |
| 425 | + u32 align = elems_per_page * DQ_RANGE_ALIGN; | |
| 425 | 426 | |
| 426 | − p_conn->cid_count = roundup(p_conn->cid_count, elems_per_page); | |
| 427 | + p_conn->cid_count = roundup(p_conn->cid_count, align); | |
| 427 | 428 | } |
| 428 | 429 | } |
| 429 | 430 | |
| @@ -2389,9 +2389,8 @@ qed_chain_alloc_sanity_check(struct qed_dev *cdev, · 2388 unmodified lines | ||
| 2389 | 2389 | * size/capacity fields are of a u32 type. |
| 2390 | 2390 | */ |
| 2391 | 2391 | if ((cnt_type == QED_CHAIN_CNT_TYPE_U16 && |
| 2392 | − chain_size > 0x10000) || | |
| 2393 | − (cnt_type == QED_CHAIN_CNT_TYPE_U32 && | |
| 2394 | − chain_size > 0x100000000ULL)) { | |
| 2392 | + chain_size > ((u32)U16_MAX + 1)) || | |
| 2393 | + (cnt_type == QED_CHAIN_CNT_TYPE_U32 && chain_size > U32_MAX)) { | |
| 2395 | 2394 | DP_NOTICE(cdev, |
| 2396 | 2395 | "The actual chain size (0x%llx) is larger than the maximal possible value\n", |
| 2397 | 2396 | chain_size); |
| @@ -190,6 +190,9 @@ qed_sp_iscsi_func_start(struct qed_hwfn *p_hwfn, · 189 unmodified lines | ||
| 190 | 190 | p_init->num_sq_pages_in_ring = p_params->num_sq_pages_in_ring; |
| 191 | 191 | p_init->num_r2tq_pages_in_ring = p_params->num_r2tq_pages_in_ring; |
| 192 | 192 | p_init->num_uhq_pages_in_ring = p_params->num_uhq_pages_in_ring; |
| 193 | + p_init->ooo_enable = p_params->ooo_enable; | |
| 194 | + p_init->ll2_rx_queue_id = p_hwfn->hw_info.resc_start[QED_LL2_QUEUE] + | |
| 195 | + p_params->ll2_ooo_queue_id; | |
| 193 | 196 | p_init->func_params.log_page_size = p_params->log_page_size; |
| 194 | 197 | val = p_params->num_tasks; |
| 195 | 198 | p_init->func_params.num_tasks = cpu_to_le16(val); |
| @@ -786,6 +789,23 @@ static void qed_iscsi_release_connection(struct qed_hwfn *p_hwfn, · 590 unmodified lines | ||
| 786 | 789 | spin_unlock_bh(&p_hwfn->p_iscsi_info->lock); |
| 787 | 790 | } |
| 788 | 791 | |
| 792 | +void qed_iscsi_free_connection(struct qed_hwfn *p_hwfn, | |
| 793 | + struct qed_iscsi_conn *p_conn) | |
| 794 | +{ | |
| 795 | + qed_chain_free(p_hwfn->cdev, &p_conn->xhq); | |
| 796 | + qed_chain_free(p_hwfn->cdev, &p_conn->uhq); | |
| 797 | + qed_chain_free(p_hwfn->cdev, &p_conn->r2tq); | |
| 798 | + dma_free_coherent(&p_hwfn->cdev->pdev->dev, | |
| 799 | + sizeof(struct tcp_upload_params), | |
| 800 | + p_conn->tcp_upload_params_virt_addr, | |
| 801 | + p_conn->tcp_upload_params_phys_addr); | |
| 802 | + dma_free_coherent(&p_hwfn->cdev->pdev->dev, | |
| 803 | + sizeof(struct scsi_terminate_extra_params), | |
| 804 | + p_conn->queue_cnts_virt_addr, | |
| 805 | + p_conn->queue_cnts_phys_addr); | |
| 806 | + kfree(p_conn); | |
| 807 | +} | |
| 808 | + | |
| 789 | 809 | struct qed_iscsi_info *qed_iscsi_alloc(struct qed_hwfn *p_hwfn) |
| 790 | 810 | { |
| 791 | 811 | struct qed_iscsi_info *p_iscsi_info; |
| @@ -807,6 +827,17 @@ void qed_iscsi_setup(struct qed_hwfn *p_hwfn, · 15 unmodified lines | ||
| 807 | 827 | void qed_iscsi_free(struct qed_hwfn *p_hwfn, |
| 808 | 828 | struct qed_iscsi_info *p_iscsi_info) |
| 809 | 829 | { |
| 830 | + struct qed_iscsi_conn *p_conn = NULL; | |
| 831 | + | |
| 832 | + while (!list_empty(&p_hwfn->p_iscsi_info->free_list)) { | |
| 833 | + p_conn = list_first_entry(&p_hwfn->p_iscsi_info->free_list, | |
| 834 | + struct qed_iscsi_conn, list_entry); | |
| 835 | + if (p_conn) { | |
| 836 | + list_del(&p_conn->list_entry); | |
| 837 | + qed_iscsi_free_connection(p_hwfn, p_conn); | |
| 838 | + } | |
| 839 | + } | |
| 840 | + | |
| 810 | 841 | kfree(p_iscsi_info); |
| 811 | 842 | } |
| 812 | 843 | |
| @@ -211,6 +211,8 @@ static void qed_ll2b_complete_rx_packet(struct qed_hwfn *p_hwfn, · 210 unmodified lines | ||
| 211 | 211 | /* If need to reuse or there's no replacement buffer, repost this */ |
| 212 | 212 | if (rc) |
| 213 | 213 | goto out_post; |
| 214 | + dma_unmap_single(&cdev->pdev->dev, buffer->phys_addr, | |
| 215 | + cdev->ll2->rx_size, DMA_FROM_DEVICE); | |
| 214 | 216 | |
| 215 | 217 | skb = build_skb(buffer->data, 0); |
| 216 | 218 | if (!skb) { |
| @@ -474,7 +476,7 @@ qed_ll2_rxq_completion_gsi(struct qed_hwfn *p_hwfn, · 257 unmodified lines | ||
| 474 | 476 | static int qed_ll2_rxq_completion_reg(struct qed_hwfn *p_hwfn, |
| 475 | 477 | struct qed_ll2_info *p_ll2_conn, |
| 476 | 478 | union core_rx_cqe_union *p_cqe, |
| 477 | − unsigned long lock_flags, | |
| 479 | + unsigned long *p_lock_flags, | |
| 478 | 480 | bool b_last_cqe) |
| 479 | 481 | { |
| 480 | 482 | struct qed_ll2_rx_queue *p_rx = &p_ll2_conn->rx_queue; |
| @@ -495,10 +497,10 @@ static int qed_ll2_rxq_completion_reg(struct qed_hwfn *p_hwfn, · 14 unmodified lines | ||
| 495 | 497 | "Mismatch between active_descq and the LL2 Rx chain\n"); |
| 496 | 498 | list_add_tail(&p_pkt->list_entry, &p_rx->free_descq); |
| 497 | 499 | |
| 498 | − spin_unlock_irqrestore(&p_rx->lock, lock_flags); | |
| 500 | + spin_unlock_irqrestore(&p_rx->lock, *p_lock_flags); | |
| 499 | 501 | qed_ll2b_complete_rx_packet(p_hwfn, p_ll2_conn->my_id, |
| 500 | 502 | p_pkt, &p_cqe->rx_cqe_fp, b_last_cqe); |
| 501 | − spin_lock_irqsave(&p_rx->lock, lock_flags); | |
| 503 | + spin_lock_irqsave(&p_rx->lock, *p_lock_flags); | |
| 502 | 504 | |
| 503 | 505 | return 0; |
| 504 | 506 | } |
| @@ -538,7 +540,8 @@ static int qed_ll2_rxq_completion(struct qed_hwfn *p_hwfn, void *cookie) · 33 unmodified lines | ||
| 538 | 540 | break; |
| 539 | 541 | case CORE_RX_CQE_TYPE_REGULAR: |
| 540 | 542 | rc = qed_ll2_rxq_completion_reg(p_hwfn, p_ll2_conn, |
| 541 | − cqe, flags, b_last_cqe); | |
| 543 | + cqe, &flags, | |
| 544 | + b_last_cqe); | |
| 542 | 545 | break; |
| 543 | 546 | default: |
| 544 | 547 | rc = -EIO; |
| @@ -968,7 +971,7 @@ static int qed_ll2_start_ooo(struct qed_dev *cdev, · 423 unmodified lines | ||
| 968 | 971 | { |
| 969 | 972 | struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev); |
| 970 | 973 | u8 *handle = &hwfn->pf_params.iscsi_pf_params.ll2_ooo_queue_id; |
| 971 | − struct qed_ll2_conn ll2_info; | |
| 974 | + struct qed_ll2_conn ll2_info = { 0 }; | |
| 972 | 975 | int rc; |
| 973 | 976 | |
| 974 | 977 | ll2_info.conn_type = QED_LL2_TYPE_ISCSI_OOO; |
| @@ -159,6 +159,8 @@ struct qed_ooo_info *qed_ooo_alloc(struct qed_hwfn *p_hwfn) · 158 unmodified lines | ||
| 159 | 159 | if (!p_ooo_info->ooo_history.p_cqes) |
| 160 | 160 | goto no_history_mem; |
| 161 | 161 | |
| 162 | + p_ooo_info->ooo_history.num_of_cqes = QED_MAX_NUM_OOO_HISTORY_ENTRIES; | |
| 163 | + | |
| 162 | 164 | return p_ooo_info; |
| 163 | 165 | |
| 164 | 166 | no_history_mem: |
| @@ -1162,8 +1162,8 @@ struct ob_mac_tso_iocb_rsp { · 1161 unmodified lines | ||
| 1162 | 1162 | struct ib_mac_iocb_rsp { |
| 1163 | 1163 | u8 opcode; /* 0x20 */ |
| 1164 | 1164 | u8 flags1; |
| 1165 | −#define IB_MAC_IOCB_RSP_OI 0x01 /* Overide intr delay */ | |
| 1166 | −#define IB_MAC_IOCB_RSP_I 0x02 /* Disble Intr Generation */ | |
| 1165 | +#define IB_MAC_IOCB_RSP_OI 0x01 /* Override intr delay */ | |
| 1166 | +#define IB_MAC_IOCB_RSP_I 0x02 /* Disable Intr Generation */ | |
| 1167 | 1167 | #define IB_MAC_CSUM_ERR_MASK 0x1c /* A mask to use for csum errs */ |
| 1168 | 1168 | #define IB_MAC_IOCB_RSP_TE 0x04 /* Checksum error */ |
| 1169 | 1169 | #define IB_MAC_IOCB_RSP_NU 0x08 /* No checksum rcvd */ |
| @@ -1535,32 +1535,33 @@ static int smc_close(struct net_device *dev) · 1534 unmodified lines | ||
| 1535 | 1535 | * Ethtool support |
| 1536 | 1536 | */ |
| 1537 | 1537 | static int |
| 1538 | −smc_ethtool_getsettings(struct net_device *dev, struct ethtool_cmd *cmd) | |
| 1538 | +smc_ethtool_get_link_ksettings(struct net_device *dev, | |
| 1539 | + struct ethtool_link_ksettings *cmd) | |
| 1539 | 1540 | { |
| 1540 | 1541 | struct smc_local *lp = netdev_priv(dev); |
| 1541 | 1542 | int ret; |
| 1542 | 1543 | |
| 1543 | − cmd->maxtxpkt = 1; | |
| 1544 | − cmd->maxrxpkt = 1; | |
| 1545 | − | |
| 1546 | 1544 | if (lp->phy_type != 0) { |
| 1547 | 1545 | spin_lock_irq(&lp->lock); |
| 1548 | − ret = mii_ethtool_gset(&lp->mii, cmd); | |
| 1546 | + ret = mii_ethtool_get_link_ksettings(&lp->mii, cmd); | |
| 1549 | 1547 | spin_unlock_irq(&lp->lock); |
| 1550 | 1548 | } else { |
| 1551 | − cmd->supported = SUPPORTED_10baseT_Half | | |
| 1549 | + u32 supported = SUPPORTED_10baseT_Half | | |
| 1552 | 1550 | SUPPORTED_10baseT_Full | |
| 1553 | 1551 | SUPPORTED_TP | SUPPORTED_AUI; |
| 1554 | 1552 | |
| 1555 | 1553 | if (lp->ctl_rspeed == 10) |
| 1556 | − ethtool_cmd_speed_set(cmd, SPEED_10); | |
| 1554 | + cmd->base.speed = SPEED_10; | |
| 1557 | 1555 | else if (lp->ctl_rspeed == 100) |
| 1558 | − ethtool_cmd_speed_set(cmd, SPEED_100); | |
| 1556 | + cmd->base.speed = SPEED_100; | |
| 1557 | + | |
| 1558 | + cmd->base.autoneg = AUTONEG_DISABLE; | |
| 1559 | + cmd->base.port = 0; | |
| 1560 | + cmd->base.duplex = lp->tcr_cur_mode & TCR_SWFDUP ? | |
| 1561 | + DUPLEX_FULL : DUPLEX_HALF; | |
| 1559 | 1562 | |
| 1560 | − cmd->autoneg = AUTONEG_DISABLE; | |
| 1561 | − cmd->transceiver = XCVR_INTERNAL; | |
| 1562 | − cmd->port = 0; | |
| 1563 | − cmd->duplex = lp->tcr_cur_mode & TCR_SWFDUP ? DUPLEX_FULL : DUPLEX_HALF; | |
| 1563 | + ethtool_convert_legacy_u32_to_link_mode( | |
| 1564 | + cmd->link_modes.supported, supported); | |
| 1564 | 1565 | |
| 1565 | 1566 | ret = 0; |
| 1566 | 1567 | } |
| @@ -1569,24 +1570,26 @@ smc_ethtool_getsettings(struct net_device *dev, struct ethtool_cmd *cmd) · 2 unmodified lines | ||
| 1569 | 1570 | } |
| 1570 | 1571 | |
| 1571 | 1572 | static int |
| 1572 | −smc_ethtool_setsettings(struct net_device *dev, struct ethtool_cmd *cmd) | |
| 1573 | +smc_ethtool_set_link_ksettings(struct net_device *dev, | |
| 1574 | + const struct ethtool_link_ksettings *cmd) | |
| 1573 | 1575 | { |
| 1574 | 1576 | struct smc_local *lp = netdev_priv(dev); |
| 1575 | 1577 | int ret; |
| 1576 | 1578 | |
| 1577 | 1579 | if (lp->phy_type != 0) { |
| 1578 | 1580 | spin_lock_irq(&lp->lock); |
| 1579 | − ret = mii_ethtool_sset(&lp->mii, cmd); | |
| 1581 | + ret = mii_ethtool_set_link_ksettings(&lp->mii, cmd); | |
| 1580 | 1582 | spin_unlock_irq(&lp->lock); |
| 1581 | 1583 | } else { |
| 1582 | − if (cmd->autoneg != AUTONEG_DISABLE || | |
| 1583 | − cmd->speed != SPEED_10 || | |
| 1584 | − (cmd->duplex != DUPLEX_HALF && cmd->duplex != DUPLEX_FULL) || | |
| 1585 | − (cmd->port != PORT_TP && cmd->port != PORT_AUI)) | |
| 1584 | + if (cmd->base.autoneg != AUTONEG_DISABLE || | |
| 1585 | + cmd->base.speed != SPEED_10 || | |
| 1586 | + (cmd->base.duplex != DUPLEX_HALF && | |
| 1587 | + cmd->base.duplex != DUPLEX_FULL) || | |
| 1588 | + (cmd->base.port != PORT_TP && cmd->base.port != PORT_AUI)) | |
| 1586 | 1589 | return -EINVAL; |
| 1587 | 1590 | |
| 1588 | −// lp->port = cmd->port; | |
| 1589 | − lp->ctl_rfduplx = cmd->duplex == DUPLEX_FULL; | |
| 1591 | +// lp->port = cmd->base.port; | |
| 1592 | + lp->ctl_rfduplx = cmd->base.duplex == DUPLEX_FULL; | |
| 1590 | 1593 | |
| 1591 | 1594 | // if (netif_running(dev)) |
| 1592 | 1595 | // smc_set_port(dev); |
| @@ -1744,8 +1747,6 @@ static int smc_ethtool_seteeprom(struct net_device *dev, · 151 unmodified lines | ||
| 1744 | 1747 | |
| 1745 | 1748 | |
| 1746 | 1749 | static const struct ethtool_ops smc_ethtool_ops = { |
| 1747 | − .get_settings = smc_ethtool_getsettings, | |
| 1748 | − .set_settings = smc_ethtool_setsettings, | |
| 1749 | 1750 | .get_drvinfo = smc_ethtool_getdrvinfo, |
| 1750 | 1751 | |
| 1751 | 1752 | .get_msglevel = smc_ethtool_getmsglevel, |
| @@ -1755,6 +1756,8 @@ static const struct ethtool_ops smc_ethtool_ops = { · 3 unmodified lines | ||
| 1755 | 1756 | .get_eeprom_len = smc_ethtool_geteeprom_len, |
| 1756 | 1757 | .get_eeprom = smc_ethtool_geteeprom, |
| 1757 | 1758 | .set_eeprom = smc_ethtool_seteeprom, |
| 1759 | + .get_link_ksettings = smc_ethtool_get_link_ksettings, | |
| 1760 | + .set_link_ksettings = smc_ethtool_set_link_ksettings, | |
| 1758 | 1761 | }; |
| 1759 | 1762 | |
| 1760 | 1763 | static const struct net_device_ops smc_netdev_ops = { |
| @@ -700,6 +700,8 @@ struct net_device_context { · 699 unmodified lines | ||
| 700 | 700 | |
| 701 | 701 | u32 tx_checksum_mask; |
| 702 | 702 | |
| 703 | + u32 tx_send_table[VRSS_SEND_TAB_SIZE]; | |
| 704 | + | |
| 703 | 705 | /* Ethtool settings */ |
| 704 | 706 | u8 duplex; |
| 705 | 707 | u32 speed; |
| @@ -757,7 +759,6 @@ struct netvsc_device { · 51 unmodified lines | ||
| 757 | 759 | |
| 758 | 760 | struct nvsp_message revoke_packet; |
| 759 | 761 | |
| 760 | − u32 send_table[VRSS_SEND_TAB_SIZE]; | |
| 761 | 762 | u32 max_chn; |
| 762 | 763 | u32 num_chn; |
| 763 | 764 | spinlock_t sc_lock; /* Protects num_sc_offered variable */ |
| @@ -1136,15 +1136,11 @@ static void netvsc_receive(struct net_device *ndev, · 1135 unmodified lines | ||
| 1136 | 1136 | static void netvsc_send_table(struct hv_device *hdev, |
| 1137 | 1137 | struct nvsp_message *nvmsg) |
| 1138 | 1138 | { |
| 1139 | − struct netvsc_device *nvscdev; | |
| 1140 | 1139 | struct net_device *ndev = hv_get_drvdata(hdev); |
| 1140 | + struct net_device_context *net_device_ctx = netdev_priv(ndev); | |
| 1141 | 1141 | int i; |
| 1142 | 1142 | u32 count, *tab; |
| 1143 | 1143 | |
| 1144 | − nvscdev = get_outbound_net_device(hdev); | |
| 1145 | − if (!nvscdev) | |
| 1146 | − return; | |
| 1147 | − | |
| 1148 | 1144 | count = nvmsg->msg.v5_msg.send_table.count; |
| 1149 | 1145 | if (count != VRSS_SEND_TAB_SIZE) { |
| 1150 | 1146 | netdev_err(ndev, "Received wrong send-table size:%u\n", count); |
| @@ -1155,7 +1151,7 @@ static void netvsc_send_table(struct hv_device *hdev, · 4 unmodified lines | ||
| 1155 | 1151 | nvmsg->msg.v5_msg.send_table.offset); |
| 1156 | 1152 | |
| 1157 | 1153 | for (i = 0; i < count; i++) |
| 1158 | − nvscdev->send_table[i] = tab[i]; | |
| 1154 | + net_device_ctx->tx_send_table[i] = tab[i]; | |
| 1159 | 1155 | } |
| 1160 | 1156 | |
| 1161 | 1157 | static void netvsc_send_vf(struct net_device_context *net_device_ctx, |
| @@ -206,17 +206,15 @@ static u16 netvsc_select_queue(struct net_device *ndev, struct sk_buff *skb, · 205 unmodified lines | ||
| 206 | 206 | void *accel_priv, select_queue_fallback_t fallback) |
| 207 | 207 | { |
| 208 | 208 | struct net_device_context *net_device_ctx = netdev_priv(ndev); |
| 209 | − struct netvsc_device *nvsc_dev = net_device_ctx->nvdev; | |
| 209 | + unsigned int num_tx_queues = ndev->real_num_tx_queues; | |
| 210 | 210 | struct sock *sk = skb->sk; |
| 211 | 211 | int q_idx = sk_tx_queue_get(sk); |
| 212 | 212 | |
| 213 | − if (q_idx < 0 || skb->ooo_okay || | |
| 214 | − q_idx >= ndev->real_num_tx_queues) { | |
| 213 | + if (q_idx < 0 || skb->ooo_okay || q_idx >= num_tx_queues) { | |
| 215 | 214 | u16 hash = __skb_tx_hash(ndev, skb, VRSS_SEND_TAB_SIZE); |
| 216 | 215 | int new_idx; |
| 217 | 216 | |
| 218 | − new_idx = nvsc_dev->send_table[hash] | |
| 219 | − % nvsc_dev->num_chn; | |
| 217 | + new_idx = net_device_ctx->tx_send_table[hash] % num_tx_queues; | |
| 220 | 218 | |
| 221 | 219 | if (q_idx != new_idx && sk && |
| 222 | 220 | sk_fullsock(sk) && rcu_access_pointer(sk->sk_dst_cache)) |
| @@ -225,9 +223,6 @@ static u16 netvsc_select_queue(struct net_device *ndev, struct sk_buff *skb, · 2 unmodified lines | ||
| 225 | 223 | q_idx = new_idx; |
| 226 | 224 | } |
| 227 | 225 | |
| 228 | − if (unlikely(!nvsc_dev->chan_table[q_idx].channel)) | |
| 229 | − q_idx = 0; | |
| 230 | − | |
| 231 | 226 | return q_idx; |
| 232 | 227 | } |
| 233 | 228 | |
| @@ -1883,17 +1883,6 @@ static int m88e1510_probe(struct phy_device *phydev) · 1882 unmodified lines | ||
| 1883 | 1883 | return m88e1510_hwmon_probe(phydev); |
| 1884 | 1884 | } |
| 1885 | 1885 | |
| 1886 | −static void marvell_remove(struct phy_device *phydev) | |
| 1887 | −{ | |
| 1888 | −#ifdef CONFIG_HWMON | |
| 1889 | − | |
| 1890 | − struct marvell_priv *priv = phydev->priv; | |
| 1891 | − | |
| 1892 | − if (priv && priv->hwmon_dev) | |
| 1893 | − hwmon_device_unregister(priv->hwmon_dev); | |
| 1894 | −#endif | |
| 1895 | −} | |
| 1896 | − | |
| 1897 | 1886 | static struct phy_driver marvell_drivers[] = { |
| 1898 | 1887 | { |
| 1899 | 1888 | .phy_id = MARVELL_PHY_ID_88E1101, |
| @@ -1974,7 +1963,6 @@ static struct phy_driver marvell_drivers[] = { · 74 unmodified lines | ||
| 1974 | 1963 | .features = PHY_GBIT_FEATURES, |
| 1975 | 1964 | .flags = PHY_HAS_INTERRUPT, |
| 1976 | 1965 | .probe = &m88e1121_probe, |
| 1977 | − .remove = &marvell_remove, | |
| 1978 | 1966 | .config_init = &m88e1121_config_init, |
| 1979 | 1967 | .config_aneg = &m88e1121_config_aneg, |
| 1980 | 1968 | .read_status = &marvell_read_status, |
| @@ -2087,7 +2075,6 @@ static struct phy_driver marvell_drivers[] = { · 106 unmodified lines | ||
| 2087 | 2075 | .features = PHY_GBIT_FEATURES | SUPPORTED_FIBRE, |
| 2088 | 2076 | .flags = PHY_HAS_INTERRUPT, |
| 2089 | 2077 | .probe = &m88e1510_probe, |
| 2090 | − .remove = &marvell_remove, | |
| 2091 | 2078 | .config_init = &m88e1510_config_init, |
| 2092 | 2079 | .config_aneg = &m88e1510_config_aneg, |
| 2093 | 2080 | .read_status = &marvell_read_status, |
| @@ -2109,7 +2096,6 @@ static struct phy_driver marvell_drivers[] = { · 15 unmodified lines | ||
| 2109 | 2096 | .features = PHY_GBIT_FEATURES, |
| 2110 | 2097 | .flags = PHY_HAS_INTERRUPT, |
| 2111 | 2098 | .probe = m88e1510_probe, |
| 2112 | − .remove = &marvell_remove, | |
| 2113 | 2099 | .config_init = &marvell_config_init, |
| 2114 | 2100 | .config_aneg = &m88e1510_config_aneg, |
| 2115 | 2101 | .read_status = &marvell_read_status, |
| @@ -2127,7 +2113,6 @@ static struct phy_driver marvell_drivers[] = { · 11 unmodified lines | ||
| 2127 | 2113 | .phy_id_mask = MARVELL_PHY_ID_MASK, |
| 2128 | 2114 | .name = "Marvell 88E1545", |
| 2129 | 2115 | .probe = m88e1510_probe, |
| 2130 | − .remove = &marvell_remove, | |
| 2131 | 2116 | .features = PHY_GBIT_FEATURES, |
| 2132 | 2117 | .flags = PHY_HAS_INTERRUPT, |
| 2133 | 2118 | .config_init = &marvell_config_init, |
| @@ -1864,7 +1864,7 @@ static struct phy_driver genphy_driver[] = { · 1863 unmodified lines | ||
| 1864 | 1864 | .phy_id = 0xffffffff, |
| 1865 | 1865 | .phy_id_mask = 0xffffffff, |
| 1866 | 1866 | .name = "Generic PHY", |
| 1867 | − .soft_reset = genphy_soft_reset, | |
| 1867 | + .soft_reset = genphy_no_soft_reset, | |
| 1868 | 1868 | .config_init = genphy_config_init, |
| 1869 | 1869 | .features = PHY_GBIT_FEATURES | SUPPORTED_MII | |
| 1870 | 1870 | SUPPORTED_AUI | SUPPORTED_FIBRE | |
| @@ -491,13 +491,14 @@ static int ks8995_probe(struct spi_device *spi) · 490 unmodified lines | ||
| 491 | 491 | if (err) |
| 492 | 492 | return err; |
| 493 | 493 | |
| 494 | − ks->regs_attr.size = ks->chip->regs_size; | |
| 495 | 494 | memcpy(&ks->regs_attr, &ks8995_registers_attr, sizeof(ks->regs_attr)); |
| 495 | + ks->regs_attr.size = ks->chip->regs_size; | |
| 496 | 496 | |
| 497 | 497 | err = ks8995_reset(ks); |
| 498 | 498 | if (err) |
| 499 | 499 | return err; |
| 500 | 500 | |
| 501 | + sysfs_attr_init(&ks->regs_attr.attr); | |
| 501 | 502 | err = sysfs_create_bin_file(&spi->dev.kobj, &ks->regs_attr); |
| 502 | 503 | if (err) { |
| 503 | 504 | dev_err(&spi->dev, "unable to create sysfs file, err=%d\n", |
| @@ -2072,6 +2072,7 @@ static int team_dev_type_check_change(struct net_device *dev, · 2071 unmodified lines | ||
| 2072 | 2072 | static void team_setup(struct net_device *dev) |
| 2073 | 2073 | { |
| 2074 | 2074 | ether_setup(dev); |
| 2075 | + dev->max_mtu = ETH_MAX_MTU; | |
| 2075 | 2076 | |
| 2076 | 2077 | dev->netdev_ops = &team_netdev_ops; |
| 2077 | 2078 | dev->ethtool_ops = &team_ethtool_ops; |
| @@ -822,7 +822,18 @@ static void tun_net_uninit(struct net_device *dev) · 821 unmodified lines | ||
| 822 | 822 | /* Net device open. */ |
| 823 | 823 | static int tun_net_open(struct net_device *dev) |
| 824 | 824 | { |
| 825 | + struct tun_struct *tun = netdev_priv(dev); | |
| 826 | + int i; | |
| 827 | + | |
| 825 | 828 | netif_tx_start_all_queues(dev); |
| 829 | + | |
| 830 | + for (i = 0; i < tun->numqueues; i++) { | |
| 831 | + struct tun_file *tfile; | |
| 832 | + | |
| 833 | + tfile = rtnl_dereference(tun->tfiles[i]); | |
| 834 | + tfile->socket.sk->sk_write_space(tfile->socket.sk); | |
| 835 | + } | |
| 836 | + | |
| 826 | 837 | return 0; |
| 827 | 838 | } |
| 828 | 839 | |
| @@ -1103,9 +1114,10 @@ static unsigned int tun_chr_poll(struct file *file, poll_table *wait) · 274 unmodified lines | ||
| 1103 | 1114 | if (!skb_array_empty(&tfile->tx_array)) |
| 1104 | 1115 | mask |= POLLIN | POLLRDNORM; |
| 1105 | 1116 | |
| 1106 | − if (sock_writeable(sk) || | |
| 1107 | − (!test_and_set_bit(SOCKWQ_ASYNC_NOSPACE, &sk->sk_socket->flags) && | |
| 1108 | − sock_writeable(sk))) | |
| 1117 | + if (tun->dev->flags & IFF_UP && | |
| 1118 | + (sock_writeable(sk) || | |
| 1119 | + (!test_and_set_bit(SOCKWQ_ASYNC_NOSPACE, &sk->sk_socket->flags) && | |
| 1120 | + sock_writeable(sk)))) | |
| 1109 | 1121 | mask |= POLLOUT | POLLWRNORM; |
| 1110 | 1122 | |
| 1111 | 1123 | if (tun->dev->reg_state != NETREG_REGISTERED) |
| @@ -2570,7 +2582,6 @@ static int __init tun_init(void) · 1458 unmodified lines | ||
| 2570 | 2582 | int ret = 0; |
| 2571 | 2583 | |
| 2572 | 2584 | pr_info("%s, %s\n", DRV_DESCRIPTION, DRV_VERSION); |
| 2573 | − pr_info("%s\n", DRV_COPYRIGHT); | |
| 2574 | 2585 | |
| 2575 | 2586 | ret = rtnl_link_register(&tun_link_ops); |
| 2576 | 2587 | if (ret) { |
| @@ -340,6 +340,7 @@ static netdev_tx_t is_ip_tx_frame(struct sk_buff *skb, struct net_device *dev) · 339 unmodified lines | ||
| 340 | 340 | |
| 341 | 341 | static netdev_tx_t vrf_xmit(struct sk_buff *skb, struct net_device *dev) |
| 342 | 342 | { |
| 343 | + int len = skb->len; | |
| 343 | 344 | netdev_tx_t ret = is_ip_tx_frame(skb, dev); |
| 344 | 345 | |
| 345 | 346 | if (likely(ret == NET_XMIT_SUCCESS || ret == NET_XMIT_CN)) { |
| @@ -347,7 +348,7 @@ static netdev_tx_t vrf_xmit(struct sk_buff *skb, struct net_device *dev) · 1 unmodified line | ||
| 347 | 348 | |
| 348 | 349 | u64_stats_update_begin(&dstats->syncp); |
| 349 | 350 | dstats->tx_pkts++; |
| 350 | − dstats->tx_bytes += skb->len; | |
| 351 | + dstats->tx_bytes += len; | |
| 351 | 352 | u64_stats_update_end(&dstats->syncp); |
| 352 | 353 | } else { |
| 353 | 354 | this_cpu_inc(dev->dstats->tx_drps); |
| @@ -2976,6 +2976,44 @@ static int vxlan_dev_configure(struct net *src_net, struct net_device *dev, · 2975 unmodified lines | ||
| 2976 | 2976 | return 0; |
| 2977 | 2977 | } |
| 2978 | 2978 | |
| 2979 | +static int __vxlan_dev_create(struct net *net, struct net_device *dev, | |
| 2980 | + struct vxlan_config *conf) | |
| 2981 | +{ | |
| 2982 | + struct vxlan_net *vn = net_generic(net, vxlan_net_id); | |
| 2983 | + struct vxlan_dev *vxlan = netdev_priv(dev); | |
| 2984 | + int err; | |
| 2985 | + | |
| 2986 | + err = vxlan_dev_configure(net, dev, conf, false); | |
| 2987 | + if (err) | |
| 2988 | + return err; | |
| 2989 | + | |
| 2990 | + dev->ethtool_ops = &vxlan_ethtool_ops; | |
| 2991 | + | |
| 2992 | + /* create an fdb entry for a valid default destination */ | |
| 2993 | + if (!vxlan_addr_any(&vxlan->default_dst.remote_ip)) { | |
| 2994 | + err = vxlan_fdb_create(vxlan, all_zeros_mac, | |
| 2995 | + &vxlan->default_dst.remote_ip, | |
| 2996 | + NUD_REACHABLE | NUD_PERMANENT, | |
| 2997 | + NLM_F_EXCL | NLM_F_CREATE, | |
| 2998 | + vxlan->cfg.dst_port, | |
| 2999 | + vxlan->default_dst.remote_vni, | |
| 3000 | + vxlan->default_dst.remote_vni, | |
| 3001 | + vxlan->default_dst.remote_ifindex, | |
| 3002 | + NTF_SELF); | |
| 3003 | + if (err) | |
| 3004 | + return err; | |
| 3005 | + } | |
| 3006 | + | |
| 3007 | + err = register_netdevice(dev); | |
| 3008 | + if (err) { | |
| 3009 | + vxlan_fdb_delete_default(vxlan, vxlan->default_dst.remote_vni); | |
| 3010 | + return err; | |
| 3011 | + } | |
| 3012 | + | |
| 3013 | + list_add(&vxlan->next, &vn->vxlan_list); | |
| 3014 | + return 0; | |
| 3015 | +} | |
| 3016 | + | |
| 2979 | 3017 | static int vxlan_nl2conf(struct nlattr *tb[], struct nlattr *data[], |
| 2980 | 3018 | struct net_device *dev, struct vxlan_config *conf, |
| 2981 | 3019 | bool changelink) |
| @@ -3172,8 +3210,6 @@ static int vxlan_nl2conf(struct nlattr *tb[], struct nlattr *data[], · 190 unmodified lines | ||
| 3172 | 3210 | static int vxlan_newlink(struct net *src_net, struct net_device *dev, |
| 3173 | 3211 | struct nlattr *tb[], struct nlattr *data[]) |
| 3174 | 3212 | { |
| 3175 | − struct vxlan_net *vn = net_generic(src_net, vxlan_net_id); | |
| 3176 | − struct vxlan_dev *vxlan = netdev_priv(dev); | |
| 3177 | 3213 | struct vxlan_config conf; |
| 3178 | 3214 | int err; |
| 3179 | 3215 | |
| @@ -3181,36 +3217,7 @@ static int vxlan_newlink(struct net *src_net, struct net_device *dev, · 1 unmodified line | ||
| 3181 | 3217 | if (err) |
| 3182 | 3218 | return err; |
| 3183 | 3219 | |
| 3184 | − err = vxlan_dev_configure(src_net, dev, &conf, false); | |
| 3185 | − if (err) | |
| 3186 | − return err; | |
| 3187 | − | |
| 3188 | − dev->ethtool_ops = &vxlan_ethtool_ops; | |
| 3189 | − | |
| 3190 | − /* create an fdb entry for a valid default destination */ | |
| 3191 | − if (!vxlan_addr_any(&vxlan->default_dst.remote_ip)) { | |
| 3192 | − err = vxlan_fdb_create(vxlan, all_zeros_mac, | |
| 3193 | − &vxlan->default_dst.remote_ip, | |
| 3194 | − NUD_REACHABLE | NUD_PERMANENT, | |
| 3195 | − NLM_F_EXCL | NLM_F_CREATE, | |
| 3196 | − vxlan->cfg.dst_port, | |
| 3197 | − vxlan->default_dst.remote_vni, | |
| 3198 | − vxlan->default_dst.remote_vni, | |
| 3199 | − vxlan->default_dst.remote_ifindex, | |
| 3200 | − NTF_SELF); | |
| 3201 | − if (err) | |
| 3202 | − return err; | |
| 3203 | − } | |
| 3204 | − | |
| 3205 | − err = register_netdevice(dev); | |
| 3206 | − if (err) { | |
| 3207 | − vxlan_fdb_delete_default(vxlan, vxlan->default_dst.remote_vni); | |
| 3208 | − return err; | |
| 3209 | − } | |
| 3210 | − | |
| 3211 | − list_add(&vxlan->next, &vn->vxlan_list); | |
| 3212 | − | |
| 3213 | − return 0; | |
| 3220 | + return __vxlan_dev_create(src_net, dev, &conf); | |
| 3214 | 3221 | } |
| 3215 | 3222 | |
| 3216 | 3223 | static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], |
| @@ -3440,7 +3447,7 @@ struct net_device *vxlan_dev_create(struct net *net, const char *name, · 223 unmodified lines | ||
| 3440 | 3447 | if (IS_ERR(dev)) |
| 3441 | 3448 | return dev; |
| 3442 | 3449 | |
| 3443 | − err = vxlan_dev_configure(net, dev, conf, false); | |
| 3450 | + err = __vxlan_dev_create(net, dev, conf); | |
| 3444 | 3451 | if (err < 0) { |
| 3445 | 3452 | free_netdev(dev); |
| 3446 | 3453 | return ERR_PTR(err); |
| @@ -381,8 +381,8 @@ static netdev_tx_t ucc_hdlc_tx(struct sk_buff *skb, struct net_device *dev) · 380 unmodified lines | ||
| 381 | 381 | /* set bd status and length */ |
| 382 | 382 | bd_status = (bd_status & T_W_S) | T_R_S | T_I_S | T_L_S | T_TC_S; |
| 383 | 383 | |
| 384 | − iowrite16be(bd_status, &bd->status); | |
| 385 | 384 | iowrite16be(skb->len, &bd->length); |
| 385 | + iowrite16be(bd_status, &bd->status); | |
| 386 | 386 | |
| 387 | 387 | /* Move to next BD in the ring */ |
| 388 | 388 | if (!(bd_status & T_W_S)) |
| @@ -457,7 +457,7 @@ static int hdlc_rx_done(struct ucc_hdlc_private *priv, int rx_work_limit) · 68 unmodified lines | ||
| 457 | 457 | struct sk_buff *skb; |
| 458 | 458 | hdlc_device *hdlc = dev_to_hdlc(dev); |
| 459 | 459 | struct qe_bd *bd; |
| 460 | − u32 bd_status; | |
| 460 | + u16 bd_status; | |
| 461 | 461 | u16 length, howmany = 0; |
| 462 | 462 | u8 *bdbuffer; |
| 463 | 463 | int i; |
| @@ -467,6 +467,9 @@ int i2400mu_probe(struct usb_interface *iface, · 466 unmodified lines | ||
| 467 | 467 | struct i2400mu *i2400mu; |
| 468 | 468 | struct usb_device *usb_dev = interface_to_usbdev(iface); |
| 469 | 469 | |
| 470 | + if (iface->cur_altsetting->desc.bNumEndpoints < 4) | |
| 471 | + return -ENODEV; | |
| 472 | + | |
| 470 | 473 | if (usb_dev->speed != USB_SPEED_HIGH) |
| 471 | 474 | dev_err(dev, "device not connected as high speed\n"); |
| 472 | 475 | |
| @@ -165,13 +165,17 @@ static int xenvif_start_xmit(struct sk_buff *skb, struct net_device *dev) · 164 unmodified lines | ||
| 165 | 165 | { |
| 166 | 166 | struct xenvif *vif = netdev_priv(dev); |
| 167 | 167 | struct xenvif_queue *queue = NULL; |
| 168 | − unsigned int num_queues = vif->num_queues; | |
| 168 | + unsigned int num_queues; | |
| 169 | 169 | u16 index; |
| 170 | 170 | struct xenvif_rx_cb *cb; |
| 171 | 171 | |
| 172 | 172 | BUG_ON(skb->dev != dev); |
| 173 | 173 | |
| 174 | − /* Drop the packet if queues are not set up */ | |
| 174 | + /* Drop the packet if queues are not set up. | |
| 175 | + * This handler should be called inside an RCU read section | |
| 176 | + * so we don't need to enter it here explicitly. | |
| 177 | + */ | |
| 178 | + num_queues = READ_ONCE(vif->num_queues); | |
| 175 | 179 | if (num_queues < 1) |
| 176 | 180 | goto drop; |
| 177 | 181 | |
| @@ -222,18 +226,18 @@ static struct net_device_stats *xenvif_get_stats(struct net_device *dev) · 44 unmodified lines | ||
| 222 | 226 | { |
| 223 | 227 | struct xenvif *vif = netdev_priv(dev); |
| 224 | 228 | struct xenvif_queue *queue = NULL; |
| 229 | + unsigned int num_queues; | |
| 225 | 230 | u64 rx_bytes = 0; |
| 226 | 231 | u64 rx_packets = 0; |
| 227 | 232 | u64 tx_bytes = 0; |
| 228 | 233 | u64 tx_packets = 0; |
| 229 | 234 | unsigned int index; |
| 230 | 235 | |
| 231 | − spin_lock(&vif->lock); | |
| 232 | − if (vif->queues == NULL) | |
| 233 | − goto out; | |
| 236 | + rcu_read_lock(); | |
| 237 | + num_queues = READ_ONCE(vif->num_queues); | |
| 234 | 238 | |
| 235 | 239 | /* Aggregate tx and rx stats from each queue */ |
| 236 | − for (index = 0; index < vif->num_queues; ++index) { | |
| 240 | + for (index = 0; index < num_queues; ++index) { | |
| 237 | 241 | queue = &vif->queues[index]; |
| 238 | 242 | rx_bytes += queue->stats.rx_bytes; |
| 239 | 243 | rx_packets += queue->stats.rx_packets; |
| @@ -241,8 +245,7 @@ static struct net_device_stats *xenvif_get_stats(struct net_device *dev) · 1 unmodified line | ||
| 241 | 245 | tx_packets += queue->stats.tx_packets; |
| 242 | 246 | } |
| 243 | 247 | |
| 244 | −out: | |
| 245 | − spin_unlock(&vif->lock); | |
| 248 | + rcu_read_unlock(); | |
| 246 | 249 | |
| 247 | 250 | vif->dev->stats.rx_bytes = rx_bytes; |
| 248 | 251 | vif->dev->stats.rx_packets = rx_packets; |
| @@ -378,10 +381,13 @@ static void xenvif_get_ethtool_stats(struct net_device *dev, · 129 unmodified lines | ||
| 378 | 381 | struct ethtool_stats *stats, u64 * data) |
| 379 | 382 | { |
| 380 | 383 | struct xenvif *vif = netdev_priv(dev); |
| 381 | − unsigned int num_queues = vif->num_queues; | |
| 384 | + unsigned int num_queues; | |
| 382 | 385 | int i; |
| 383 | 386 | unsigned int queue_index; |
| 384 | 387 | |
| 388 | + rcu_read_lock(); | |
| 389 | + num_queues = READ_ONCE(vif->num_queues); | |
| 390 | + | |
| 385 | 391 | for (i = 0; i < ARRAY_SIZE(xenvif_stats); i++) { |
| 386 | 392 | unsigned long accum = 0; |
| 387 | 393 | for (queue_index = 0; queue_index < num_queues; ++queue_index) { |
| @@ -390,6 +396,8 @@ static void xenvif_get_ethtool_stats(struct net_device *dev, · 2 unmodified lines | ||
| 390 | 396 | } |
| 391 | 397 | data[i] = accum; |
| 392 | 398 | } |
| 399 | + | |
| 400 | + rcu_read_unlock(); | |
| 393 | 401 | } |
| 394 | 402 | |
| 395 | 403 | static void xenvif_get_strings(struct net_device *dev, u32 stringset, u8 * data) |
| @@ -214,7 +214,7 @@ static void xenvif_fatal_tx_err(struct xenvif *vif) · 213 unmodified lines | ||
| 214 | 214 | netdev_err(vif->dev, "fatal error; disabling device\n"); |
| 215 | 215 | vif->disabled = true; |
| 216 | 216 | /* Disable the vif from queue 0's kthread */ |
| 217 | − if (vif->queues) | |
| 217 | + if (vif->num_queues) | |
| 218 | 218 | xenvif_kick_thread(&vif->queues[0]); |
| 219 | 219 | } |
| 220 | 220 | |
| @@ -495,26 +495,26 @@ static void backend_disconnect(struct backend_info *be) · 494 unmodified lines | ||
| 495 | 495 | struct xenvif *vif = be->vif; |
| 496 | 496 | |
| 497 | 497 | if (vif) { |
| 498 | + unsigned int num_queues = vif->num_queues; | |
| 498 | 499 | unsigned int queue_index; |
| 499 | − struct xenvif_queue *queues; | |
| 500 | 500 | |
| 501 | 501 | xen_unregister_watchers(vif); |
| 502 | 502 | #ifdef CONFIG_DEBUG_FS |
| 503 | 503 | xenvif_debugfs_delif(vif); |
| 504 | 504 | #endif /* CONFIG_DEBUG_FS */ |
| 505 | 505 | xenvif_disconnect_data(vif); |
| 506 | − for (queue_index = 0; | |
| 507 | − queue_index < vif->num_queues; | |
| 508 | − ++queue_index) | |
| 509 | − xenvif_deinit_queue(&vif->queues[queue_index]); | |
| 510 | 506 | |
| 511 | − spin_lock(&vif->lock); | |
| 512 | − queues = vif->queues; | |
| 507 | + /* At this point some of the handlers may still be active | |
| 508 | + * so we need to have additional synchronization here. | |
| 509 | + */ | |
| 513 | 510 | vif->num_queues = 0; |
| 514 | − vif->queues = NULL; | |
| 515 | − spin_unlock(&vif->lock); | |
| 511 | + synchronize_net(); | |
| 516 | 512 | |
| 517 | − vfree(queues); | |
| 513 | + for (queue_index = 0; queue_index < num_queues; ++queue_index) | |
| 514 | + xenvif_deinit_queue(&vif->queues[queue_index]); | |
| 515 | + | |
| 516 | + vfree(vif->queues); | |
| 517 | + vif->queues = NULL; | |
| 518 | 518 | |
| 519 | 519 | xenvif_disconnect_ctrl(vif); |
| 520 | 520 | } |
| @@ -132,10 +132,6 @@ static int exynos5440_pcie_get_mem_resources(struct platform_device *pdev, · 131 unmodified lines | ||
| 132 | 132 | struct device *dev = pci->dev; |
| 133 | 133 | struct resource *res; |
| 134 | 134 | |
| 135 | − /* If using the PHY framework, doesn't need to get other resource */ | |
| 136 | − if (ep->using_phy) | |
| 137 | − return 0; | |
| 138 | − | |
| 139 | 135 | ep->mem_res = devm_kzalloc(dev, sizeof(*ep->mem_res), GFP_KERNEL); |
| 140 | 136 | if (!ep->mem_res) |
| 141 | 137 | return -ENOMEM; |
| @@ -145,6 +141,10 @@ static int exynos5440_pcie_get_mem_resources(struct platform_device *pdev, · 3 unmodified lines | ||
| 145 | 141 | if (IS_ERR(ep->mem_res->elbi_base)) |
| 146 | 142 | return PTR_ERR(ep->mem_res->elbi_base); |
| 147 | 143 | |
| 144 | + /* If using the PHY framework, doesn't need to get other resource */ | |
| 145 | + if (ep->using_phy) | |
| 146 | + return 0; | |
| 147 | + | |
| 148 | 148 | res = platform_get_resource(pdev, IORESOURCE_MEM, 1); |
| 149 | 149 | ep->mem_res->phy_base = devm_ioremap_resource(dev, res); |
| 150 | 150 | if (IS_ERR(ep->mem_res->phy_base)) |
| @@ -478,7 +478,7 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link, · 477 unmodified lines | ||
| 478 | 478 | |
| 479 | 479 | static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist) |
| 480 | 480 | { |
| 481 | − struct pci_dev *child, *parent = link->pdev; | |
| 481 | + struct pci_dev *child = link->downstream, *parent = link->pdev; | |
| 482 | 482 | struct pci_bus *linkbus = parent->subordinate; |
| 483 | 483 | struct aspm_register_info upreg, dwreg; |
| 484 | 484 | |
| @@ -491,9 +491,7 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist) · 6 unmodified lines | ||
| 491 | 491 | |
| 492 | 492 | /* Get upstream/downstream components' register state */ |
| 493 | 493 | pcie_get_aspm_reg(parent, &upreg); |
| 494 | − child = pci_function_0(linkbus); | |
| 495 | 494 | pcie_get_aspm_reg(child, &dwreg); |
| 496 | − link->downstream = child; | |
| 497 | 495 | |
| 498 | 496 | /* |
| 499 | 497 | * If ASPM not supported, don't mess with the clocks and link, |
| @@ -800,6 +798,7 @@ static struct pcie_link_state *alloc_pcie_link_state(struct pci_dev *pdev) · 300 unmodified lines | ||
| 800 | 798 | INIT_LIST_HEAD(&link->children); |
| 801 | 799 | INIT_LIST_HEAD(&link->link); |
| 802 | 800 | link->pdev = pdev; |
| 801 | + link->downstream = pci_function_0(pdev->subordinate); | |
| 803 | 802 | |
| 804 | 803 | /* |
| 805 | 804 | * Root Ports and PCI/PCI-X to PCIe Bridges are roots of PCIe |
| @@ -2174,6 +2174,7 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_LSI_LOGIC, 0x005d, quirk_blacklist_vpd); · 2173 unmodified lines | ||
| 2174 | 2174 | DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_LSI_LOGIC, 0x005f, quirk_blacklist_vpd); |
| 2175 | 2175 | DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATTANSIC, PCI_ANY_ID, |
| 2176 | 2176 | quirk_blacklist_vpd); |
| 2177 | +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_QLOGIC, 0x2261, quirk_blacklist_vpd); | |
| 2177 | 2178 | |
| 2178 | 2179 | /* |
| 2179 | 2180 | * For Broadcom 5706, 5708, 5709 rev. A nics, any read beyond the |
| @@ -422,6 +422,20 @@ static int msm_gpio_direction_output(struct gpio_chip *chip, unsigned offset, in · 421 unmodified lines | ||
| 422 | 422 | return 0; |
| 423 | 423 | } |
| 424 | 424 | |
| 425 | +static int msm_gpio_get_direction(struct gpio_chip *chip, unsigned int offset) | |
| 426 | +{ | |
| 427 | + struct msm_pinctrl *pctrl = gpiochip_get_data(chip); | |
| 428 | + const struct msm_pingroup *g; | |
| 429 | + u32 val; | |
| 430 | + | |
| 431 | + g = &pctrl->soc->groups[offset]; | |
| 432 | + | |
| 433 | + val = readl(pctrl->regs + g->ctl_reg); | |
| 434 | + | |
| 435 | + /* 0 = output, 1 = input */ | |
| 436 | + return val & BIT(g->oe_bit) ? 0 : 1; | |
| 437 | +} | |
| 438 | + | |
| 425 | 439 | static int msm_gpio_get(struct gpio_chip *chip, unsigned offset) |
| 426 | 440 | { |
| 427 | 441 | const struct msm_pingroup *g; |
| @@ -510,6 +524,7 @@ static void msm_gpio_dbg_show(struct seq_file *s, struct gpio_chip *chip) · 82 unmodified lines | ||
| 510 | 524 | static struct gpio_chip msm_gpio_template = { |
| 511 | 525 | .direction_input = msm_gpio_direction_input, |
| 512 | 526 | .direction_output = msm_gpio_direction_output, |
| 527 | + .get_direction = msm_gpio_get_direction, | |
| 513 | 528 | .get = msm_gpio_get, |
| 514 | 529 | .set = msm_gpio_set, |
| 515 | 530 | .request = gpiochip_generic_request, |
| @@ -390,22 +390,22 @@ static const struct pinctrl_pin_desc uniphier_ld11_pins[] = { · 389 unmodified lines | ||
| 390 | 390 | UNIPHIER_PINCTRL_PIN(140, "AO1D0", 140, |
| 391 | 391 | 140, UNIPHIER_PIN_DRV_1BIT, |
| 392 | 392 | 140, UNIPHIER_PIN_PULL_DOWN), |
| 393 | − UNIPHIER_PINCTRL_PIN(141, "TCON0", 141, | |
| 393 | + UNIPHIER_PINCTRL_PIN(141, "AO1D1", 141, | |
| 394 | 394 | 141, UNIPHIER_PIN_DRV_1BIT, |
| 395 | 395 | 141, UNIPHIER_PIN_PULL_DOWN), |
| 396 | − UNIPHIER_PINCTRL_PIN(142, "TCON1", 142, | |
| 396 | + UNIPHIER_PINCTRL_PIN(142, "AO1D2", 142, | |
| 397 | 397 | 142, UNIPHIER_PIN_DRV_1BIT, |
| 398 | 398 | 142, UNIPHIER_PIN_PULL_DOWN), |
| 399 | − UNIPHIER_PINCTRL_PIN(143, "TCON2", 143, | |
| 399 | + UNIPHIER_PINCTRL_PIN(143, "XIRQ9", 143, | |
| 400 | 400 | 143, UNIPHIER_PIN_DRV_1BIT, |
| 401 | 401 | 143, UNIPHIER_PIN_PULL_DOWN), |
| 402 | − UNIPHIER_PINCTRL_PIN(144, "TCON3", 144, | |
| 402 | + UNIPHIER_PINCTRL_PIN(144, "XIRQ10", 144, | |
| 403 | 403 | 144, UNIPHIER_PIN_DRV_1BIT, |
| 404 | 404 | 144, UNIPHIER_PIN_PULL_DOWN), |
| 405 | − UNIPHIER_PINCTRL_PIN(145, "TCON4", 145, | |
| 405 | + UNIPHIER_PINCTRL_PIN(145, "XIRQ11", 145, | |
| 406 | 406 | 145, UNIPHIER_PIN_DRV_1BIT, |
| 407 | 407 | 145, UNIPHIER_PIN_PULL_DOWN), |
| 408 | − UNIPHIER_PINCTRL_PIN(146, "TCON5", 146, | |
| 408 | + UNIPHIER_PINCTRL_PIN(146, "XIRQ13", 146, | |
| 409 | 409 | 146, UNIPHIER_PIN_DRV_1BIT, |
| 410 | 410 | 146, UNIPHIER_PIN_PULL_DOWN), |
| 411 | 411 | UNIPHIER_PINCTRL_PIN(147, "PWMA", 147, |
| @@ -103,15 +103,6 @@ static struct quirk_entry quirk_asus_x200ca = { · 102 unmodified lines | ||
| 103 | 103 | .wapf = 2, |
| 104 | 104 | }; |
| 105 | 105 | |
| 106 | −static struct quirk_entry quirk_no_rfkill = { | |
| 107 | − .no_rfkill = true, | |
| 108 | −}; | |
| 109 | − | |
| 110 | −static struct quirk_entry quirk_no_rfkill_wapf4 = { | |
| 111 | − .wapf = 4, | |
| 112 | − .no_rfkill = true, | |
| 113 | −}; | |
| 114 | − | |
| 115 | 106 | static struct quirk_entry quirk_asus_ux303ub = { |
| 116 | 107 | .wmi_backlight_native = true, |
| 117 | 108 | }; |
| @@ -194,7 +185,7 @@ static const struct dmi_system_id asus_quirks[] = { · 76 unmodified lines | ||
| 194 | 185 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), |
| 195 | 186 | DMI_MATCH(DMI_PRODUCT_NAME, "X456UA"), |
| 196 | 187 | }, |
| 197 | − .driver_data = &quirk_no_rfkill_wapf4, | |
| 188 | + .driver_data = &quirk_asus_wapf4, | |
| 198 | 189 | }, |
| 199 | 190 | { |
| 200 | 191 | .callback = dmi_matched, |
| @@ -203,7 +194,7 @@ static const struct dmi_system_id asus_quirks[] = { · 2 unmodified lines | ||
| 203 | 194 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), |
| 204 | 195 | DMI_MATCH(DMI_PRODUCT_NAME, "X456UF"), |
| 205 | 196 | }, |
| 206 | − .driver_data = &quirk_no_rfkill_wapf4, | |
| 197 | + .driver_data = &quirk_asus_wapf4, | |
| 207 | 198 | }, |
| 208 | 199 | { |
| 209 | 200 | .callback = dmi_matched, |
| @@ -367,42 +358,6 @@ static const struct dmi_system_id asus_quirks[] = { · 157 unmodified lines | ||
| 367 | 358 | }, |
| 368 | 359 | .driver_data = &quirk_asus_x200ca, |
| 369 | 360 | }, |
| 370 | − { | |
| 371 | − .callback = dmi_matched, | |
| 372 | − .ident = "ASUSTeK COMPUTER INC. X555UB", | |
| 373 | − .matches = { | |
| 374 | − DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 375 | − DMI_MATCH(DMI_PRODUCT_NAME, "X555UB"), | |
| 376 | − }, | |
| 377 | − .driver_data = &quirk_no_rfkill, | |
| 378 | − }, | |
| 379 | − { | |
| 380 | − .callback = dmi_matched, | |
| 381 | − .ident = "ASUSTeK COMPUTER INC. N552VW", | |
| 382 | − .matches = { | |
| 383 | − DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 384 | − DMI_MATCH(DMI_PRODUCT_NAME, "N552VW"), | |
| 385 | − }, | |
| 386 | − .driver_data = &quirk_no_rfkill, | |
| 387 | − }, | |
| 388 | − { | |
| 389 | − .callback = dmi_matched, | |
| 390 | − .ident = "ASUSTeK COMPUTER INC. U303LB", | |
| 391 | − .matches = { | |
| 392 | − DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 393 | − DMI_MATCH(DMI_PRODUCT_NAME, "U303LB"), | |
| 394 | − }, | |
| 395 | − .driver_data = &quirk_no_rfkill, | |
| 396 | − }, | |
| 397 | − { | |
| 398 | − .callback = dmi_matched, | |
| 399 | − .ident = "ASUSTeK COMPUTER INC. Z550MA", | |
| 400 | − .matches = { | |
| 401 | − DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 402 | − DMI_MATCH(DMI_PRODUCT_NAME, "Z550MA"), | |
| 403 | − }, | |
| 404 | − .driver_data = &quirk_no_rfkill, | |
| 405 | − }, | |
| 406 | 361 | { |
| 407 | 362 | .callback = dmi_matched, |
| 408 | 363 | .ident = "ASUSTeK COMPUTER INC. UX303UB", |
| @@ -159,6 +159,8 @@ MODULE_LICENSE("GPL"); · 158 unmodified lines | ||
| 159 | 159 | #define USB_INTEL_XUSB2PR 0xD0 |
| 160 | 160 | #define PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_XHCI 0x9c31 |
| 161 | 161 | |
| 162 | +static const char * const ashs_ids[] = { "ATK4001", "ATK4002", NULL }; | |
| 163 | + | |
| 162 | 164 | struct bios_args { |
| 163 | 165 | u32 arg0; |
| 164 | 166 | u32 arg1; |
| @@ -2051,6 +2053,16 @@ static int asus_wmi_fan_init(struct asus_wmi *asus) · 1886 unmodified lines | ||
| 2051 | 2053 | return 0; |
| 2052 | 2054 | } |
| 2053 | 2055 | |
| 2056 | +static bool ashs_present(void) | |
| 2057 | +{ | |
| 2058 | + int i = 0; | |
| 2059 | + while (ashs_ids[i]) { | |
| 2060 | + if (acpi_dev_found(ashs_ids[i++])) | |
| 2061 | + return true; | |
| 2062 | + } | |
| 2063 | + return false; | |
| 2064 | +} | |
| 2065 | + | |
| 2054 | 2066 | /* |
| 2055 | 2067 | * WMI Driver |
| 2056 | 2068 | */ |
| @@ -2095,7 +2107,11 @@ static int asus_wmi_add(struct platform_device *pdev) · 38 unmodified lines | ||
| 2095 | 2107 | if (err) |
| 2096 | 2108 | goto fail_leds; |
| 2097 | 2109 | |
| 2098 | − if (!asus->driver->quirks->no_rfkill) { | |
| 2110 | + asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_WLAN, &result); | |
| 2111 | + if (result & (ASUS_WMI_DSTS_PRESENCE_BIT | ASUS_WMI_DSTS_USER_BIT)) | |
| 2112 | + asus->driver->wlan_ctrl_by_user = 1; | |
| 2113 | + | |
| 2114 | + if (!(asus->driver->wlan_ctrl_by_user && ashs_present())) { | |
| 2099 | 2115 | err = asus_wmi_rfkill_init(asus); |
| 2100 | 2116 | if (err) |
| 2101 | 2117 | goto fail_rfkill; |
| @@ -2134,10 +2150,6 @@ static int asus_wmi_add(struct platform_device *pdev) · 32 unmodified lines | ||
| 2134 | 2150 | if (err) |
| 2135 | 2151 | goto fail_debugfs; |
| 2136 | 2152 | |
| 2137 | − asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_WLAN, &result); | |
| 2138 | − if (result & (ASUS_WMI_DSTS_PRESENCE_BIT | ASUS_WMI_DSTS_USER_BIT)) | |
| 2139 | − asus->driver->wlan_ctrl_by_user = 1; | |
| 2140 | − | |
| 2141 | 2153 | return 0; |
| 2142 | 2154 | |
| 2143 | 2155 | fail_debugfs: |
| @@ -39,7 +39,6 @@ struct key_entry; · 38 unmodified lines | ||
| 39 | 39 | struct asus_wmi; |
| 40 | 40 | |
| 41 | 41 | struct quirk_entry { |
| 42 | − bool no_rfkill; | |
| 43 | 42 | bool hotplug_wireless; |
| 44 | 43 | bool scalar_panel_brightness; |
| 45 | 44 | bool store_backlight_power; |
| @@ -78,18 +78,18 @@ · 77 unmodified lines | ||
| 78 | 78 | |
| 79 | 79 | #define FUJITSU_LCD_N_LEVELS 8 |
| 80 | 80 | |
| 81 | −#define ACPI_FUJITSU_CLASS "fujitsu" | |
| 82 | −#define ACPI_FUJITSU_HID "FUJ02B1" | |
| 83 | −#define ACPI_FUJITSU_DRIVER_NAME "Fujitsu laptop FUJ02B1 ACPI brightness driver" | |
| 84 | −#define ACPI_FUJITSU_DEVICE_NAME "Fujitsu FUJ02B1" | |
| 85 | −#define ACPI_FUJITSU_HOTKEY_HID "FUJ02E3" | |
| 86 | −#define ACPI_FUJITSU_HOTKEY_DRIVER_NAME "Fujitsu laptop FUJ02E3 ACPI hotkeys driver" | |
| 87 | −#define ACPI_FUJITSU_HOTKEY_DEVICE_NAME "Fujitsu FUJ02E3" | |
| 81 | +#define ACPI_FUJITSU_CLASS "fujitsu" | |
| 82 | +#define ACPI_FUJITSU_BL_HID "FUJ02B1" | |
| 83 | +#define ACPI_FUJITSU_BL_DRIVER_NAME "Fujitsu laptop FUJ02B1 ACPI brightness driver" | |
| 84 | +#define ACPI_FUJITSU_BL_DEVICE_NAME "Fujitsu FUJ02B1" | |
| 85 | +#define ACPI_FUJITSU_LAPTOP_HID "FUJ02E3" | |
| 86 | +#define ACPI_FUJITSU_LAPTOP_DRIVER_NAME "Fujitsu laptop FUJ02E3 ACPI hotkeys driver" | |
| 87 | +#define ACPI_FUJITSU_LAPTOP_DEVICE_NAME "Fujitsu FUJ02E3" | |
| 88 | 88 | |
| 89 | 89 | #define ACPI_FUJITSU_NOTIFY_CODE1 0x80 |
| 90 | 90 | |
| 91 | 91 | /* FUNC interface - command values */ |
| 92 | −#define FUNC_RFKILL 0x1000 | |
| 92 | +#define FUNC_FLAGS 0x1000 | |
| 93 | 93 | #define FUNC_LEDS 0x1001 |
| 94 | 94 | #define FUNC_BUTTONS 0x1002 |
| 95 | 95 | #define FUNC_BACKLIGHT 0x1004 |
| @@ -97,6 +97,11 @@ · 1 unmodified line | ||
| 97 | 97 | /* FUNC interface - responses */ |
| 98 | 98 | #define UNSUPPORTED_CMD 0x80000000 |
| 99 | 99 | |
| 100 | +/* FUNC interface - status flags */ | |
| 101 | +#define FLAG_RFKILL 0x020 | |
| 102 | +#define FLAG_LID 0x100 | |
| 103 | +#define FLAG_DOCK 0x200 | |
| 104 | + | |
| 100 | 105 | #if IS_ENABLED(CONFIG_LEDS_CLASS) |
| 101 | 106 | /* FUNC interface - LED control */ |
| 102 | 107 | #define FUNC_LED_OFF 0x1 |
| @@ -136,7 +141,7 @@ · 33 unmodified lines | ||
| 136 | 141 | #endif |
| 137 | 142 | |
| 138 | 143 | /* Device controlling the backlight and associated keys */ |
| 139 | −struct fujitsu_t { | |
| 144 | +struct fujitsu_bl { | |
| 140 | 145 | acpi_handle acpi_handle; |
| 141 | 146 | struct acpi_device *dev; |
| 142 | 147 | struct input_dev *input; |
| @@ -150,12 +155,12 @@ struct fujitsu_t { · 7 unmodified lines | ||
| 150 | 155 | unsigned int brightness_level; |
| 151 | 156 | }; |
| 152 | 157 | |
| 153 | −static struct fujitsu_t *fujitsu; | |
| 158 | +static struct fujitsu_bl *fujitsu_bl; | |
| 154 | 159 | static int use_alt_lcd_levels = -1; |
| 155 | 160 | static int disable_brightness_adjust = -1; |
| 156 | 161 | |
| 157 | −/* Device used to access other hotkeys on the laptop */ | |
| 158 | −struct fujitsu_hotkey_t { | |
| 162 | +/* Device used to access hotkeys and other features on the laptop */ | |
| 163 | +struct fujitsu_laptop { | |
| 159 | 164 | acpi_handle acpi_handle; |
| 160 | 165 | struct acpi_device *dev; |
| 161 | 166 | struct input_dev *input; |
| @@ -163,17 +168,15 @@ struct fujitsu_hotkey_t { · 1 unmodified line | ||
| 163 | 168 | struct platform_device *pf_device; |
| 164 | 169 | struct kfifo fifo; |
| 165 | 170 | spinlock_t fifo_lock; |
| 166 | − int rfkill_supported; | |
| 167 | − int rfkill_state; | |
| 171 | + int flags_supported; | |
| 172 | + int flags_state; | |
| 168 | 173 | int logolamp_registered; |
| 169 | 174 | int kblamps_registered; |
| 170 | 175 | int radio_led_registered; |
| 171 | 176 | int eco_led_registered; |
| 172 | 177 | }; |
| 173 | 178 | |
| 174 | −static struct fujitsu_hotkey_t *fujitsu_hotkey; | |
| 175 | − | |
| 176 | −static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event); | |
| 179 | +static struct fujitsu_laptop *fujitsu_laptop; | |
| 177 | 180 | |
| 178 | 181 | #if IS_ENABLED(CONFIG_LEDS_CLASS) |
| 179 | 182 | static enum led_brightness logolamp_get(struct led_classdev *cdev); |
| @@ -222,8 +225,6 @@ static struct led_classdev eco_led = { · 42 unmodified lines | ||
| 222 | 225 | static u32 dbg_level = 0x03; |
| 223 | 226 | #endif |
| 224 | 227 | |
| 225 | −static void acpi_fujitsu_notify(struct acpi_device *device, u32 event); | |
| 226 | − | |
| 227 | 228 | /* Fujitsu ACPI interface function */ |
| 228 | 229 | |
| 229 | 230 | static int call_fext_func(int cmd, int arg0, int arg1, int arg2) |
| @@ -239,7 +240,7 @@ static int call_fext_func(int cmd, int arg0, int arg1, int arg2) · 9 unmodified lines | ||
| 239 | 240 | unsigned long long value; |
| 240 | 241 | acpi_handle handle = NULL; |
| 241 | 242 | |
| 242 | − status = acpi_get_handle(fujitsu_hotkey->acpi_handle, "FUNC", &handle); | |
| 243 | + status = acpi_get_handle(fujitsu_laptop->acpi_handle, "FUNC", &handle); | |
| 243 | 244 | if (ACPI_FAILURE(status)) { |
| 244 | 245 | vdbg_printk(FUJLAPTOP_DBG_ERROR, |
| 245 | 246 | "FUNC interface is not present\n"); |
| @@ -300,9 +301,9 @@ static int radio_led_set(struct led_classdev *cdev, · 54 unmodified lines | ||
| 300 | 301 | enum led_brightness brightness) |
| 301 | 302 | { |
| 302 | 303 | if (brightness >= LED_FULL) |
| 303 | − return call_fext_func(FUNC_RFKILL, 0x5, RADIO_LED_ON, RADIO_LED_ON); | |
| 304 | + return call_fext_func(FUNC_FLAGS, 0x5, RADIO_LED_ON, RADIO_LED_ON); | |
| 304 | 305 | else |
| 305 | − return call_fext_func(FUNC_RFKILL, 0x5, RADIO_LED_ON, 0x0); | |
| 306 | + return call_fext_func(FUNC_FLAGS, 0x5, RADIO_LED_ON, 0x0); | |
| 306 | 307 | } |
| 307 | 308 | |
| 308 | 309 | static int eco_led_set(struct led_classdev *cdev, |
| @@ -346,7 +347,7 @@ static enum led_brightness radio_led_get(struct led_classdev *cdev) · 37 unmodified lines | ||
| 346 | 347 | { |
| 347 | 348 | enum led_brightness brightness = LED_OFF; |
| 348 | 349 | |
| 349 | − if (call_fext_func(FUNC_RFKILL, 0x4, 0x0, 0x0) & RADIO_LED_ON) | |
| 350 | + if (call_fext_func(FUNC_FLAGS, 0x4, 0x0, 0x0) & RADIO_LED_ON) | |
| 350 | 351 | brightness = LED_FULL; |
| 351 | 352 | |
| 352 | 353 | return brightness; |
| @@ -373,10 +374,10 @@ static int set_lcd_level(int level) · 20 unmodified lines | ||
| 373 | 374 | vdbg_printk(FUJLAPTOP_DBG_TRACE, "set lcd level via SBLL [%d]\n", |
| 374 | 375 | level); |
| 375 | 376 | |
| 376 | − if (level < 0 || level >= fujitsu->max_brightness) | |
| 377 | + if (level < 0 || level >= fujitsu_bl->max_brightness) | |
| 377 | 378 | return -EINVAL; |
| 378 | 379 | |
| 379 | − status = acpi_get_handle(fujitsu->acpi_handle, "SBLL", &handle); | |
| 380 | + status = acpi_get_handle(fujitsu_bl->acpi_handle, "SBLL", &handle); | |
| 380 | 381 | if (ACPI_FAILURE(status)) { |
| 381 | 382 | vdbg_printk(FUJLAPTOP_DBG_ERROR, "SBLL not present\n"); |
| 382 | 383 | return -ENODEV; |
| @@ -398,10 +399,10 @@ static int set_lcd_level_alt(int level) · 15 unmodified lines | ||
| 398 | 399 | vdbg_printk(FUJLAPTOP_DBG_TRACE, "set lcd level via SBL2 [%d]\n", |
| 399 | 400 | level); |
| 400 | 401 | |
| 401 | − if (level < 0 || level >= fujitsu->max_brightness) | |
| 402 | + if (level < 0 || level >= fujitsu_bl->max_brightness) | |
| 402 | 403 | return -EINVAL; |
| 403 | 404 | |
| 404 | − status = acpi_get_handle(fujitsu->acpi_handle, "SBL2", &handle); | |
| 405 | + status = acpi_get_handle(fujitsu_bl->acpi_handle, "SBL2", &handle); | |
| 405 | 406 | if (ACPI_FAILURE(status)) { |
| 406 | 407 | vdbg_printk(FUJLAPTOP_DBG_ERROR, "SBL2 not present\n"); |
| 407 | 408 | return -ENODEV; |
| @@ -421,19 +422,19 @@ static int get_lcd_level(void) · 13 unmodified lines | ||
| 421 | 422 | |
| 422 | 423 | vdbg_printk(FUJLAPTOP_DBG_TRACE, "get lcd level via GBLL\n"); |
| 423 | 424 | |
| 424 | − status = | |
| 425 | − acpi_evaluate_integer(fujitsu->acpi_handle, "GBLL", NULL, &state); | |
| 425 | + status = acpi_evaluate_integer(fujitsu_bl->acpi_handle, "GBLL", NULL, | |
| 426 | + &state); | |
| 426 | 427 | if (ACPI_FAILURE(status)) |
| 427 | 428 | return 0; |
| 428 | 429 | |
| 429 | − fujitsu->brightness_level = state & 0x0fffffff; | |
| 430 | + fujitsu_bl->brightness_level = state & 0x0fffffff; | |
| 430 | 431 | |
| 431 | 432 | if (state & 0x80000000) |
| 432 | − fujitsu->brightness_changed = 1; | |
| 433 | + fujitsu_bl->brightness_changed = 1; | |
| 433 | 434 | else |
| 434 | − fujitsu->brightness_changed = 0; | |
| 435 | + fujitsu_bl->brightness_changed = 0; | |
| 435 | 436 | |
| 436 | − return fujitsu->brightness_level; | |
| 437 | + return fujitsu_bl->brightness_level; | |
| 437 | 438 | } |
| 438 | 439 | |
| 439 | 440 | static int get_max_brightness(void) |
| @@ -443,14 +444,14 @@ static int get_max_brightness(void) · 3 unmodified lines | ||
| 443 | 444 | |
| 444 | 445 | vdbg_printk(FUJLAPTOP_DBG_TRACE, "get max lcd level via RBLL\n"); |
| 445 | 446 | |
| 446 | − status = | |
| 447 | − acpi_evaluate_integer(fujitsu->acpi_handle, "RBLL", NULL, &state); | |
| 447 | + status = acpi_evaluate_integer(fujitsu_bl->acpi_handle, "RBLL", NULL, | |
| 448 | + &state); | |
| 448 | 449 | if (ACPI_FAILURE(status)) |
| 449 | 450 | return -1; |
| 450 | 451 | |
| 451 | − fujitsu->max_brightness = state; | |
| 452 | + fujitsu_bl->max_brightness = state; | |
| 452 | 453 | |
| 453 | − return fujitsu->max_brightness; | |
| 454 | + return fujitsu_bl->max_brightness; | |
| 454 | 455 | } |
| 455 | 456 | |
| 456 | 457 | /* Backlight device stuff */ |
| @@ -483,7 +484,7 @@ static int bl_update_status(struct backlight_device *b) · 26 unmodified lines | ||
| 483 | 484 | return ret; |
| 484 | 485 | } |
| 485 | 486 | |
| 486 | −static const struct backlight_ops fujitsubl_ops = { | |
| 487 | +static const struct backlight_ops fujitsu_bl_ops = { | |
| 487 | 488 | .get_brightness = bl_get_brightness, |
| 488 | 489 | .update_status = bl_update_status, |
| 489 | 490 | }; |
| @@ -511,7 +512,7 @@ show_brightness_changed(struct device *dev, · 21 unmodified lines | ||
| 511 | 512 | |
| 512 | 513 | int ret; |
| 513 | 514 | |
| 514 | − ret = fujitsu->brightness_changed; | |
| 515 | + ret = fujitsu_bl->brightness_changed; | |
| 515 | 516 | if (ret < 0) |
| 516 | 517 | return ret; |
| 517 | 518 | |
| @@ -539,7 +540,7 @@ static ssize_t store_lcd_level(struct device *dev, · 21 unmodified lines | ||
| 539 | 540 | int level, ret; |
| 540 | 541 | |
| 541 | 542 | if (sscanf(buf, "%i", &level) != 1 |
| 542 | − || (level < 0 || level >= fujitsu->max_brightness)) | |
| 543 | + || (level < 0 || level >= fujitsu_bl->max_brightness)) | |
| 543 | 544 | return -EINVAL; |
| 544 | 545 | |
| 545 | 546 | if (use_alt_lcd_levels) |
| @@ -567,9 +568,9 @@ static ssize_t · 21 unmodified lines | ||
| 567 | 568 | show_lid_state(struct device *dev, |
| 568 | 569 | struct device_attribute *attr, char *buf) |
| 569 | 570 | { |
| 570 | − if (!(fujitsu_hotkey->rfkill_supported & 0x100)) | |
| 571 | + if (!(fujitsu_laptop->flags_supported & FLAG_LID)) | |
| 571 | 572 | return sprintf(buf, "unknown\n"); |
| 572 | − if (fujitsu_hotkey->rfkill_state & 0x100) | |
| 573 | + if (fujitsu_laptop->flags_state & FLAG_LID) | |
| 573 | 574 | return sprintf(buf, "open\n"); |
| 574 | 575 | else |
| 575 | 576 | return sprintf(buf, "closed\n"); |
| @@ -579,9 +580,9 @@ static ssize_t · 3 unmodified lines | ||
| 579 | 580 | show_dock_state(struct device *dev, |
| 580 | 581 | struct device_attribute *attr, char *buf) |
| 581 | 582 | { |
| 582 | − if (!(fujitsu_hotkey->rfkill_supported & 0x200)) | |
| 583 | + if (!(fujitsu_laptop->flags_supported & FLAG_DOCK)) | |
| 583 | 584 | return sprintf(buf, "unknown\n"); |
| 584 | − if (fujitsu_hotkey->rfkill_state & 0x200) | |
| 585 | + if (fujitsu_laptop->flags_state & FLAG_DOCK) | |
| 585 | 586 | return sprintf(buf, "docked\n"); |
| 586 | 587 | else |
| 587 | 588 | return sprintf(buf, "undocked\n"); |
| @@ -591,9 +592,9 @@ static ssize_t · 3 unmodified lines | ||
| 591 | 592 | show_radios_state(struct device *dev, |
| 592 | 593 | struct device_attribute *attr, char *buf) |
| 593 | 594 | { |
| 594 | − if (!(fujitsu_hotkey->rfkill_supported & 0x20)) | |
| 595 | + if (!(fujitsu_laptop->flags_supported & FLAG_RFKILL)) | |
| 595 | 596 | return sprintf(buf, "unknown\n"); |
| 596 | − if (fujitsu_hotkey->rfkill_state & 0x20) | |
| 597 | + if (fujitsu_laptop->flags_state & FLAG_RFKILL) | |
| 597 | 598 | return sprintf(buf, "on\n"); |
| 598 | 599 | else |
| 599 | 600 | return sprintf(buf, "killed\n"); |
| @@ -607,7 +608,7 @@ static DEVICE_ATTR(lid, 0444, show_lid_state, ignore_store); · 7 unmodified lines | ||
| 607 | 608 | static DEVICE_ATTR(dock, 0444, show_dock_state, ignore_store); |
| 608 | 609 | static DEVICE_ATTR(radios, 0444, show_radios_state, ignore_store); |
| 609 | 610 | |
| 610 | −static struct attribute *fujitsupf_attributes[] = { | |
| 611 | +static struct attribute *fujitsu_pf_attributes[] = { | |
| 611 | 612 | &dev_attr_brightness_changed.attr, |
| 612 | 613 | &dev_attr_max_brightness.attr, |
| 613 | 614 | &dev_attr_lcd_level.attr, |
| @@ -617,11 +618,11 @@ static struct attribute *fujitsupf_attributes[] = { · 3 unmodified lines | ||
| 617 | 618 | NULL |
| 618 | 619 | }; |
| 619 | 620 | |
| 620 | −static struct attribute_group fujitsupf_attribute_group = { | |
| 621 | − .attrs = fujitsupf_attributes | |
| 621 | +static struct attribute_group fujitsu_pf_attribute_group = { | |
| 622 | + .attrs = fujitsu_pf_attributes | |
| 622 | 623 | }; |
| 623 | 624 | |
| 624 | −static struct platform_driver fujitsupf_driver = { | |
| 625 | +static struct platform_driver fujitsu_pf_driver = { | |
| 625 | 626 | .driver = { |
| 626 | 627 | .name = "fujitsu-laptop", |
| 627 | 628 | } |
| @@ -630,39 +631,30 @@ static struct platform_driver fujitsupf_driver = { · 2 unmodified lines | ||
| 630 | 631 | static void __init dmi_check_cb_common(const struct dmi_system_id *id) |
| 631 | 632 | { |
| 632 | 633 | pr_info("Identified laptop model '%s'\n", id->ident); |
| 633 | − if (use_alt_lcd_levels == -1) { | |
| 634 | − if (acpi_has_method(NULL, | |
| 635 | − "\\_SB.PCI0.LPCB.FJEX.SBL2")) | |
| 636 | − use_alt_lcd_levels = 1; | |
| 637 | − else | |
| 638 | − use_alt_lcd_levels = 0; | |
| 639 | − vdbg_printk(FUJLAPTOP_DBG_TRACE, "auto-detected usealt as " | |
| 640 | − "%i\n", use_alt_lcd_levels); | |
| 641 | − } | |
| 642 | 634 | } |
| 643 | 635 | |
| 644 | 636 | static int __init dmi_check_cb_s6410(const struct dmi_system_id *id) |
| 645 | 637 | { |
| 646 | 638 | dmi_check_cb_common(id); |
| 647 | − fujitsu->keycode1 = KEY_SCREENLOCK; /* "Lock" */ | |
| 648 | − fujitsu->keycode2 = KEY_HELP; /* "Mobility Center" */ | |
| 639 | + fujitsu_bl->keycode1 = KEY_SCREENLOCK; /* "Lock" */ | |
| 640 | + fujitsu_bl->keycode2 = KEY_HELP; /* "Mobility Center" */ | |
| 649 | 641 | return 1; |
| 650 | 642 | } |
| 651 | 643 | |
| 652 | 644 | static int __init dmi_check_cb_s6420(const struct dmi_system_id *id) |
| 653 | 645 | { |
| 654 | 646 | dmi_check_cb_common(id); |
| 655 | − fujitsu->keycode1 = KEY_SCREENLOCK; /* "Lock" */ | |
| 656 | − fujitsu->keycode2 = KEY_HELP; /* "Mobility Center" */ | |
| 647 | + fujitsu_bl->keycode1 = KEY_SCREENLOCK; /* "Lock" */ | |
| 648 | + fujitsu_bl->keycode2 = KEY_HELP; /* "Mobility Center" */ | |
| 657 | 649 | return 1; |
| 658 | 650 | } |
| 659 | 651 | |
| 660 | 652 | static int __init dmi_check_cb_p8010(const struct dmi_system_id *id) |
| 661 | 653 | { |
| 662 | 654 | dmi_check_cb_common(id); |
| 663 | − fujitsu->keycode1 = KEY_HELP; /* "Support" */ | |
| 664 | − fujitsu->keycode3 = KEY_SWITCHVIDEOMODE; /* "Presentation" */ | |
| 665 | − fujitsu->keycode4 = KEY_WWW; /* "Internet" */ | |
| 655 | + fujitsu_bl->keycode1 = KEY_HELP; /* "Support" */ | |
| 656 | + fujitsu_bl->keycode3 = KEY_SWITCHVIDEOMODE; /* "Presentation" */ | |
| 657 | + fujitsu_bl->keycode4 = KEY_WWW; /* "Internet" */ | |
| 666 | 658 | return 1; |
| 667 | 659 | } |
| 668 | 660 | |
| @@ -693,7 +685,7 @@ static const struct dmi_system_id fujitsu_dmi_table[] __initconst = { · 24 unmodified lines | ||
| 693 | 685 | |
| 694 | 686 | /* ACPI device for LCD brightness control */ |
| 695 | 687 | |
| 696 | −static int acpi_fujitsu_add(struct acpi_device *device) | |
| 688 | +static int acpi_fujitsu_bl_add(struct acpi_device *device) | |
| 697 | 689 | { |
| 698 | 690 | int state = 0; |
| 699 | 691 | struct input_dev *input; |
| @@ -702,22 +694,22 @@ static int acpi_fujitsu_add(struct acpi_device *device) · 2 unmodified lines | ||
| 702 | 694 | if (!device) |
| 703 | 695 | return -EINVAL; |
| 704 | 696 | |
| 705 | − fujitsu->acpi_handle = device->handle; | |
| 706 | − sprintf(acpi_device_name(device), "%s", ACPI_FUJITSU_DEVICE_NAME); | |
| 697 | + fujitsu_bl->acpi_handle = device->handle; | |
| 698 | + sprintf(acpi_device_name(device), "%s", ACPI_FUJITSU_BL_DEVICE_NAME); | |
| 707 | 699 | sprintf(acpi_device_class(device), "%s", ACPI_FUJITSU_CLASS); |
| 708 | − device->driver_data = fujitsu; | |
| 700 | + device->driver_data = fujitsu_bl; | |
| 709 | 701 | |
| 710 | − fujitsu->input = input = input_allocate_device(); | |
| 702 | + fujitsu_bl->input = input = input_allocate_device(); | |
| 711 | 703 | if (!input) { |
| 712 | 704 | error = -ENOMEM; |
| 713 | 705 | goto err_stop; |
| 714 | 706 | } |
| 715 | 707 | |
| 716 | − snprintf(fujitsu->phys, sizeof(fujitsu->phys), | |
| 708 | + snprintf(fujitsu_bl->phys, sizeof(fujitsu_bl->phys), | |
| 717 | 709 | "%s/video/input0", acpi_device_hid(device)); |
| 718 | 710 | |
| 719 | 711 | input->name = acpi_device_name(device); |
| 720 | − input->phys = fujitsu->phys; | |
| 712 | + input->phys = fujitsu_bl->phys; | |
| 721 | 713 | input->id.bustype = BUS_HOST; |
| 722 | 714 | input->id.product = 0x06; |
| 723 | 715 | input->dev.parent = &device->dev; |
| @@ -730,7 +722,7 @@ static int acpi_fujitsu_add(struct acpi_device *device) · 6 unmodified lines | ||
| 730 | 722 | if (error) |
| 731 | 723 | goto err_free_input_dev; |
| 732 | 724 | |
| 733 | − error = acpi_bus_update_power(fujitsu->acpi_handle, &state); | |
| 725 | + error = acpi_bus_update_power(fujitsu_bl->acpi_handle, &state); | |
| 734 | 726 | if (error) { |
| 735 | 727 | pr_err("Error reading power state\n"); |
| 736 | 728 | goto err_unregister_input_dev; |
| @@ -740,7 +732,7 @@ static int acpi_fujitsu_add(struct acpi_device *device) · 3 unmodified lines | ||
| 740 | 732 | acpi_device_name(device), acpi_device_bid(device), |
| 741 | 733 | !device->power.state ? "on" : "off"); |
| 742 | 734 | |
| 743 | − fujitsu->dev = device; | |
| 735 | + fujitsu_bl->dev = device; | |
| 744 | 736 | |
| 745 | 737 | if (acpi_has_method(device->handle, METHOD_NAME__INI)) { |
| 746 | 738 | vdbg_printk(FUJLAPTOP_DBG_INFO, "Invoking _INI\n"); |
| @@ -750,6 +742,15 @@ static int acpi_fujitsu_add(struct acpi_device *device) · 3 unmodified lines | ||
| 750 | 742 | pr_err("_INI Method failed\n"); |
| 751 | 743 | } |
| 752 | 744 | |
| 745 | + if (use_alt_lcd_levels == -1) { | |
| 746 | + if (acpi_has_method(NULL, "\\_SB.PCI0.LPCB.FJEX.SBL2")) | |
| 747 | + use_alt_lcd_levels = 1; | |
| 748 | + else | |
| 749 | + use_alt_lcd_levels = 0; | |
| 750 | + vdbg_printk(FUJLAPTOP_DBG_TRACE, "auto-detected usealt as %i\n", | |
| 751 | + use_alt_lcd_levels); | |
| 752 | + } | |
| 753 | + | |
| 753 | 754 | /* do config (detect defaults) */ |
| 754 | 755 | use_alt_lcd_levels = use_alt_lcd_levels == 1 ? 1 : 0; |
| 755 | 756 | disable_brightness_adjust = disable_brightness_adjust == 1 ? 1 : 0; |
| @@ -758,7 +759,7 @@ static int acpi_fujitsu_add(struct acpi_device *device) · 2 unmodified lines | ||
| 758 | 759 | use_alt_lcd_levels, disable_brightness_adjust); |
| 759 | 760 | |
| 760 | 761 | if (get_max_brightness() <= 0) |
| 761 | − fujitsu->max_brightness = FUJITSU_LCD_N_LEVELS; | |
| 762 | + fujitsu_bl->max_brightness = FUJITSU_LCD_N_LEVELS; | |
| 762 | 763 | get_lcd_level(); |
| 763 | 764 | |
| 764 | 765 | return 0; |
| @@ -772,38 +773,38 @@ err_stop: · 7 unmodified lines | ||
| 772 | 773 | return error; |
| 773 | 774 | } |
| 774 | 775 | |
| 775 | −static int acpi_fujitsu_remove(struct acpi_device *device) | |
| 776 | +static int acpi_fujitsu_bl_remove(struct acpi_device *device) | |
| 776 | 777 | { |
| 777 | − struct fujitsu_t *fujitsu = acpi_driver_data(device); | |
| 778 | − struct input_dev *input = fujitsu->input; | |
| 778 | + struct fujitsu_bl *fujitsu_bl = acpi_driver_data(device); | |
| 779 | + struct input_dev *input = fujitsu_bl->input; | |
| 779 | 780 | |
| 780 | 781 | input_unregister_device(input); |
| 781 | 782 | |
| 782 | − fujitsu->acpi_handle = NULL; | |
| 783 | + fujitsu_bl->acpi_handle = NULL; | |
| 783 | 784 | |
| 784 | 785 | return 0; |
| 785 | 786 | } |
| 786 | 787 | |
| 787 | 788 | /* Brightness notify */ |
| 788 | 789 | |
| 789 | −static void acpi_fujitsu_notify(struct acpi_device *device, u32 event) | |
| 790 | +static void acpi_fujitsu_bl_notify(struct acpi_device *device, u32 event) | |
| 790 | 791 | { |
| 791 | 792 | struct input_dev *input; |
| 792 | 793 | int keycode; |
| 793 | 794 | int oldb, newb; |
| 794 | 795 | |
| 795 | − input = fujitsu->input; | |
| 796 | + input = fujitsu_bl->input; | |
| 796 | 797 | |
| 797 | 798 | switch (event) { |
| 798 | 799 | case ACPI_FUJITSU_NOTIFY_CODE1: |
| 799 | 800 | keycode = 0; |
| 800 | − oldb = fujitsu->brightness_level; | |
| 801 | + oldb = fujitsu_bl->brightness_level; | |
| 801 | 802 | get_lcd_level(); |
| 802 | − newb = fujitsu->brightness_level; | |
| 803 | + newb = fujitsu_bl->brightness_level; | |
| 803 | 804 | |
| 804 | 805 | vdbg_printk(FUJLAPTOP_DBG_TRACE, |
| 805 | 806 | "brightness button event [%i -> %i (%i)]\n", |
| 806 | − oldb, newb, fujitsu->brightness_changed); | |
| 807 | + oldb, newb, fujitsu_bl->brightness_changed); | |
| 807 | 808 | |
| 808 | 809 | if (oldb < newb) { |
| 809 | 810 | if (disable_brightness_adjust != 1) { |
| @@ -840,7 +841,7 @@ static void acpi_fujitsu_notify(struct acpi_device *device, u32 event) · 30 unmodified lines | ||
| 840 | 841 | |
| 841 | 842 | /* ACPI device for hotkey handling */ |
| 842 | 843 | |
| 843 | −static int acpi_fujitsu_hotkey_add(struct acpi_device *device) | |
| 844 | +static int acpi_fujitsu_laptop_add(struct acpi_device *device) | |
| 844 | 845 | { |
| 845 | 846 | int result = 0; |
| 846 | 847 | int state = 0; |
| @@ -851,42 +852,42 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 4 unmodified lines | ||
| 851 | 852 | if (!device) |
| 852 | 853 | return -EINVAL; |
| 853 | 854 | |
| 854 | − fujitsu_hotkey->acpi_handle = device->handle; | |
| 855 | + fujitsu_laptop->acpi_handle = device->handle; | |
| 855 | 856 | sprintf(acpi_device_name(device), "%s", |
| 856 | − ACPI_FUJITSU_HOTKEY_DEVICE_NAME); | |
| 857 | + ACPI_FUJITSU_LAPTOP_DEVICE_NAME); | |
| 857 | 858 | sprintf(acpi_device_class(device), "%s", ACPI_FUJITSU_CLASS); |
| 858 | − device->driver_data = fujitsu_hotkey; | |
| 859 | + device->driver_data = fujitsu_laptop; | |
| 859 | 860 | |
| 860 | 861 | /* kfifo */ |
| 861 | − spin_lock_init(&fujitsu_hotkey->fifo_lock); | |
| 862 | − error = kfifo_alloc(&fujitsu_hotkey->fifo, RINGBUFFERSIZE * sizeof(int), | |
| 862 | + spin_lock_init(&fujitsu_laptop->fifo_lock); | |
| 863 | + error = kfifo_alloc(&fujitsu_laptop->fifo, RINGBUFFERSIZE * sizeof(int), | |
| 863 | 864 | GFP_KERNEL); |
| 864 | 865 | if (error) { |
| 865 | 866 | pr_err("kfifo_alloc failed\n"); |
| 866 | 867 | goto err_stop; |
| 867 | 868 | } |
| 868 | 869 | |
| 869 | − fujitsu_hotkey->input = input = input_allocate_device(); | |
| 870 | + fujitsu_laptop->input = input = input_allocate_device(); | |
| 870 | 871 | if (!input) { |
| 871 | 872 | error = -ENOMEM; |
| 872 | 873 | goto err_free_fifo; |
| 873 | 874 | } |
| 874 | 875 | |
| 875 | − snprintf(fujitsu_hotkey->phys, sizeof(fujitsu_hotkey->phys), | |
| 876 | + snprintf(fujitsu_laptop->phys, sizeof(fujitsu_laptop->phys), | |
| 876 | 877 | "%s/video/input0", acpi_device_hid(device)); |
| 877 | 878 | |
| 878 | 879 | input->name = acpi_device_name(device); |
| 879 | − input->phys = fujitsu_hotkey->phys; | |
| 880 | + input->phys = fujitsu_laptop->phys; | |
| 880 | 881 | input->id.bustype = BUS_HOST; |
| 881 | 882 | input->id.product = 0x06; |
| 882 | 883 | input->dev.parent = &device->dev; |
| 883 | 884 | |
| 884 | 885 | set_bit(EV_KEY, input->evbit); |
| 885 | − set_bit(fujitsu->keycode1, input->keybit); | |
| 886 | − set_bit(fujitsu->keycode2, input->keybit); | |
| 887 | − set_bit(fujitsu->keycode3, input->keybit); | |
| 888 | − set_bit(fujitsu->keycode4, input->keybit); | |
| 889 | − set_bit(fujitsu->keycode5, input->keybit); | |
| 886 | + set_bit(fujitsu_bl->keycode1, input->keybit); | |
| 887 | + set_bit(fujitsu_bl->keycode2, input->keybit); | |
| 888 | + set_bit(fujitsu_bl->keycode3, input->keybit); | |
| 889 | + set_bit(fujitsu_bl->keycode4, input->keybit); | |
| 890 | + set_bit(fujitsu_bl->keycode5, input->keybit); | |
| 890 | 891 | set_bit(KEY_TOUCHPAD_TOGGLE, input->keybit); |
| 891 | 892 | set_bit(KEY_UNKNOWN, input->keybit); |
| 892 | 893 | |
| @@ -894,7 +895,7 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 1 unmodified line | ||
| 894 | 895 | if (error) |
| 895 | 896 | goto err_free_input_dev; |
| 896 | 897 | |
| 897 | − error = acpi_bus_update_power(fujitsu_hotkey->acpi_handle, &state); | |
| 898 | + error = acpi_bus_update_power(fujitsu_laptop->acpi_handle, &state); | |
| 898 | 899 | if (error) { |
| 899 | 900 | pr_err("Error reading power state\n"); |
| 900 | 901 | goto err_unregister_input_dev; |
| @@ -904,7 +905,7 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 3 unmodified lines | ||
| 904 | 905 | acpi_device_name(device), acpi_device_bid(device), |
| 905 | 906 | !device->power.state ? "on" : "off"); |
| 906 | 907 | |
| 907 | − fujitsu_hotkey->dev = device; | |
| 908 | + fujitsu_laptop->dev = device; | |
| 908 | 909 | |
| 909 | 910 | if (acpi_has_method(device->handle, METHOD_NAME__INI)) { |
| 910 | 911 | vdbg_printk(FUJLAPTOP_DBG_INFO, "Invoking _INI\n"); |
| @@ -920,27 +921,27 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 9 unmodified lines | ||
| 920 | 921 | ; /* No action, result is discarded */ |
| 921 | 922 | vdbg_printk(FUJLAPTOP_DBG_INFO, "Discarded %i ringbuffer entries\n", i); |
| 922 | 923 | |
| 923 | − fujitsu_hotkey->rfkill_supported = | |
| 924 | − call_fext_func(FUNC_RFKILL, 0x0, 0x0, 0x0); | |
| 924 | + fujitsu_laptop->flags_supported = | |
| 925 | + call_fext_func(FUNC_FLAGS, 0x0, 0x0, 0x0); | |
| 925 | 926 | |
| 926 | 927 | /* Make sure our bitmask of supported functions is cleared if the |
| 927 | 928 | RFKILL function block is not implemented, like on the S7020. */ |
| 928 | − if (fujitsu_hotkey->rfkill_supported == UNSUPPORTED_CMD) | |
| 929 | − fujitsu_hotkey->rfkill_supported = 0; | |
| 929 | + if (fujitsu_laptop->flags_supported == UNSUPPORTED_CMD) | |
| 930 | + fujitsu_laptop->flags_supported = 0; | |
| 930 | 931 | |
| 931 | − if (fujitsu_hotkey->rfkill_supported) | |
| 932 | − fujitsu_hotkey->rfkill_state = | |
| 933 | − call_fext_func(FUNC_RFKILL, 0x4, 0x0, 0x0); | |
| 932 | + if (fujitsu_laptop->flags_supported) | |
| 933 | + fujitsu_laptop->flags_state = | |
| 934 | + call_fext_func(FUNC_FLAGS, 0x4, 0x0, 0x0); | |
| 934 | 935 | |
| 935 | 936 | /* Suspect this is a keymap of the application panel, print it */ |
| 936 | 937 | pr_info("BTNI: [0x%x]\n", call_fext_func(FUNC_BUTTONS, 0x0, 0x0, 0x0)); |
| 937 | 938 | |
| 938 | 939 | #if IS_ENABLED(CONFIG_LEDS_CLASS) |
| 939 | 940 | if (call_fext_func(FUNC_LEDS, 0x0, 0x0, 0x0) & LOGOLAMP_POWERON) { |
| 940 | − result = led_classdev_register(&fujitsu->pf_device->dev, | |
| 941 | + result = led_classdev_register(&fujitsu_bl->pf_device->dev, | |
| 941 | 942 | &logolamp_led); |
| 942 | 943 | if (result == 0) { |
| 943 | − fujitsu_hotkey->logolamp_registered = 1; | |
| 944 | + fujitsu_laptop->logolamp_registered = 1; | |
| 944 | 945 | } else { |
| 945 | 946 | pr_err("Could not register LED handler for logo lamp, error %i\n", |
| 946 | 947 | result); |
| @@ -949,10 +950,10 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 2 unmodified lines | ||
| 949 | 950 | |
| 950 | 951 | if ((call_fext_func(FUNC_LEDS, 0x0, 0x0, 0x0) & KEYBOARD_LAMPS) && |
| 951 | 952 | (call_fext_func(FUNC_BUTTONS, 0x0, 0x0, 0x0) == 0x0)) { |
| 952 | − result = led_classdev_register(&fujitsu->pf_device->dev, | |
| 953 | + result = led_classdev_register(&fujitsu_bl->pf_device->dev, | |
| 953 | 954 | &kblamps_led); |
| 954 | 955 | if (result == 0) { |
| 955 | − fujitsu_hotkey->kblamps_registered = 1; | |
| 956 | + fujitsu_laptop->kblamps_registered = 1; | |
| 956 | 957 | } else { |
| 957 | 958 | pr_err("Could not register LED handler for keyboard lamps, error %i\n", |
| 958 | 959 | result); |
| @@ -966,10 +967,10 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 7 unmodified lines | ||
| 966 | 967 | * that an RF LED is present. |
| 967 | 968 | */ |
| 968 | 969 | if (call_fext_func(FUNC_BUTTONS, 0x0, 0x0, 0x0) & BIT(24)) { |
| 969 | − result = led_classdev_register(&fujitsu->pf_device->dev, | |
| 970 | + result = led_classdev_register(&fujitsu_bl->pf_device->dev, | |
| 970 | 971 | &radio_led); |
| 971 | 972 | if (result == 0) { |
| 972 | − fujitsu_hotkey->radio_led_registered = 1; | |
| 973 | + fujitsu_laptop->radio_led_registered = 1; | |
| 973 | 974 | } else { |
| 974 | 975 | pr_err("Could not register LED handler for radio LED, error %i\n", |
| 975 | 976 | result); |
| @@ -983,10 +984,10 @@ static int acpi_fujitsu_hotkey_add(struct acpi_device *device) · 7 unmodified lines | ||
| 983 | 984 | */ |
| 984 | 985 | if ((call_fext_func(FUNC_LEDS, 0x0, 0x0, 0x0) & BIT(14)) && |
| 985 | 986 | (call_fext_func(FUNC_LEDS, 0x2, ECO_LED, 0x0) != UNSUPPORTED_CMD)) { |
| 986 | − result = led_classdev_register(&fujitsu->pf_device->dev, | |
| 987 | + result = led_classdev_register(&fujitsu_bl->pf_device->dev, | |
| 987 | 988 | &eco_led); |
| 988 | 989 | if (result == 0) { |
| 989 | − fujitsu_hotkey->eco_led_registered = 1; | |
| 990 | + fujitsu_laptop->eco_led_registered = 1; | |
| 990 | 991 | } else { |
| 991 | 992 | pr_err("Could not register LED handler for eco LED, error %i\n", |
| 992 | 993 | result); |
| @@ -1002,47 +1003,47 @@ err_unregister_input_dev: · 9 unmodified lines | ||
| 1002 | 1003 | err_free_input_dev: |
| 1003 | 1004 | input_free_device(input); |
| 1004 | 1005 | err_free_fifo: |
| 1005 | − kfifo_free(&fujitsu_hotkey->fifo); | |
| 1006 | + kfifo_free(&fujitsu_laptop->fifo); | |
| 1006 | 1007 | err_stop: |
| 1007 | 1008 | return error; |
| 1008 | 1009 | } |
| 1009 | 1010 | |
| 1010 | −static int acpi_fujitsu_hotkey_remove(struct acpi_device *device) | |
| 1011 | +static int acpi_fujitsu_laptop_remove(struct acpi_device *device) | |
| 1011 | 1012 | { |
| 1012 | − struct fujitsu_hotkey_t *fujitsu_hotkey = acpi_driver_data(device); | |
| 1013 | − struct input_dev *input = fujitsu_hotkey->input; | |
| 1013 | + struct fujitsu_laptop *fujitsu_laptop = acpi_driver_data(device); | |
| 1014 | + struct input_dev *input = fujitsu_laptop->input; | |
| 1014 | 1015 | |
| 1015 | 1016 | #if IS_ENABLED(CONFIG_LEDS_CLASS) |
| 1016 | − if (fujitsu_hotkey->logolamp_registered) | |
| 1017 | + if (fujitsu_laptop->logolamp_registered) | |
| 1017 | 1018 | led_classdev_unregister(&logolamp_led); |
| 1018 | 1019 | |
| 1019 | − if (fujitsu_hotkey->kblamps_registered) | |
| 1020 | + if (fujitsu_laptop->kblamps_registered) | |
| 1020 | 1021 | led_classdev_unregister(&kblamps_led); |
| 1021 | 1022 | |
| 1022 | − if (fujitsu_hotkey->radio_led_registered) | |
| 1023 | + if (fujitsu_laptop->radio_led_registered) | |
| 1023 | 1024 | led_classdev_unregister(&radio_led); |
| 1024 | 1025 | |
| 1025 | − if (fujitsu_hotkey->eco_led_registered) | |
| 1026 | + if (fujitsu_laptop->eco_led_registered) | |
| 1026 | 1027 | led_classdev_unregister(&eco_led); |
| 1027 | 1028 | #endif |
| 1028 | 1029 | |
| 1029 | 1030 | input_unregister_device(input); |
| 1030 | 1031 | |
| 1031 | − kfifo_free(&fujitsu_hotkey->fifo); | |
| 1032 | + kfifo_free(&fujitsu_laptop->fifo); | |
| 1032 | 1033 | |
| 1033 | − fujitsu_hotkey->acpi_handle = NULL; | |
| 1034 | + fujitsu_laptop->acpi_handle = NULL; | |
| 1034 | 1035 | |
| 1035 | 1036 | return 0; |
| 1036 | 1037 | } |
| 1037 | 1038 | |
| 1038 | −static void acpi_fujitsu_hotkey_press(int keycode) | |
| 1039 | +static void acpi_fujitsu_laptop_press(int keycode) | |
| 1039 | 1040 | { |
| 1040 | − struct input_dev *input = fujitsu_hotkey->input; | |
| 1041 | + struct input_dev *input = fujitsu_laptop->input; | |
| 1041 | 1042 | int status; |
| 1042 | 1043 | |
| 1043 | − status = kfifo_in_locked(&fujitsu_hotkey->fifo, | |
| 1044 | + status = kfifo_in_locked(&fujitsu_laptop->fifo, | |
| 1044 | 1045 | (unsigned char *)&keycode, sizeof(keycode), |
| 1045 | − &fujitsu_hotkey->fifo_lock); | |
| 1046 | + &fujitsu_laptop->fifo_lock); | |
| 1046 | 1047 | if (status != sizeof(keycode)) { |
| 1047 | 1048 | vdbg_printk(FUJLAPTOP_DBG_WARN, |
| 1048 | 1049 | "Could not push keycode [0x%x]\n", keycode); |
| @@ -1054,16 +1055,16 @@ static void acpi_fujitsu_hotkey_press(int keycode) · 5 unmodified lines | ||
| 1054 | 1055 | "Push keycode into ringbuffer [%d]\n", keycode); |
| 1055 | 1056 | } |
| 1056 | 1057 | |
| 1057 | −static void acpi_fujitsu_hotkey_release(void) | |
| 1058 | +static void acpi_fujitsu_laptop_release(void) | |
| 1058 | 1059 | { |
| 1059 | − struct input_dev *input = fujitsu_hotkey->input; | |
| 1060 | + struct input_dev *input = fujitsu_laptop->input; | |
| 1060 | 1061 | int keycode, status; |
| 1061 | 1062 | |
| 1062 | 1063 | while (true) { |
| 1063 | − status = kfifo_out_locked(&fujitsu_hotkey->fifo, | |
| 1064 | + status = kfifo_out_locked(&fujitsu_laptop->fifo, | |
| 1064 | 1065 | (unsigned char *)&keycode, |
| 1065 | 1066 | sizeof(keycode), |
| 1066 | − &fujitsu_hotkey->fifo_lock); | |
| 1067 | + &fujitsu_laptop->fifo_lock); | |
| 1067 | 1068 | if (status != sizeof(keycode)) |
| 1068 | 1069 | return; |
| 1069 | 1070 | input_report_key(input, keycode, 0); |
| @@ -1073,14 +1074,14 @@ static void acpi_fujitsu_hotkey_release(void) · 3 unmodified lines | ||
| 1073 | 1074 | } |
| 1074 | 1075 | } |
| 1075 | 1076 | |
| 1076 | −static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event) | |
| 1077 | +static void acpi_fujitsu_laptop_notify(struct acpi_device *device, u32 event) | |
| 1077 | 1078 | { |
| 1078 | 1079 | struct input_dev *input; |
| 1079 | 1080 | int keycode; |
| 1080 | 1081 | unsigned int irb = 1; |
| 1081 | 1082 | int i; |
| 1082 | 1083 | |
| 1083 | − input = fujitsu_hotkey->input; | |
| 1084 | + input = fujitsu_laptop->input; | |
| 1084 | 1085 | |
| 1085 | 1086 | if (event != ACPI_FUJITSU_NOTIFY_CODE1) { |
| 1086 | 1087 | keycode = KEY_UNKNOWN; |
| @@ -1093,9 +1094,9 @@ static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event) · 6 unmodified lines | ||
| 1093 | 1094 | return; |
| 1094 | 1095 | } |
| 1095 | 1096 | |
| 1096 | − if (fujitsu_hotkey->rfkill_supported) | |
| 1097 | − fujitsu_hotkey->rfkill_state = | |
| 1098 | − call_fext_func(FUNC_RFKILL, 0x4, 0x0, 0x0); | |
| 1097 | + if (fujitsu_laptop->flags_supported) | |
| 1098 | + fujitsu_laptop->flags_state = | |
| 1099 | + call_fext_func(FUNC_FLAGS, 0x4, 0x0, 0x0); | |
| 1099 | 1100 | |
| 1100 | 1101 | i = 0; |
| 1101 | 1102 | while ((irb = |
| @@ -1103,19 +1104,19 @@ static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event) · 1 unmodified line | ||
| 1103 | 1104 | && (i++) < MAX_HOTKEY_RINGBUFFER_SIZE) { |
| 1104 | 1105 | switch (irb & 0x4ff) { |
| 1105 | 1106 | case KEY1_CODE: |
| 1106 | − keycode = fujitsu->keycode1; | |
| 1107 | + keycode = fujitsu_bl->keycode1; | |
| 1107 | 1108 | break; |
| 1108 | 1109 | case KEY2_CODE: |
| 1109 | − keycode = fujitsu->keycode2; | |
| 1110 | + keycode = fujitsu_bl->keycode2; | |
| 1110 | 1111 | break; |
| 1111 | 1112 | case KEY3_CODE: |
| 1112 | − keycode = fujitsu->keycode3; | |
| 1113 | + keycode = fujitsu_bl->keycode3; | |
| 1113 | 1114 | break; |
| 1114 | 1115 | case KEY4_CODE: |
| 1115 | − keycode = fujitsu->keycode4; | |
| 1116 | + keycode = fujitsu_bl->keycode4; | |
| 1116 | 1117 | break; |
| 1117 | 1118 | case KEY5_CODE: |
| 1118 | − keycode = fujitsu->keycode5; | |
| 1119 | + keycode = fujitsu_bl->keycode5; | |
| 1119 | 1120 | break; |
| 1120 | 1121 | case 0: |
| 1121 | 1122 | keycode = 0; |
| @@ -1128,17 +1129,17 @@ static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event) · 6 unmodified lines | ||
| 1128 | 1129 | } |
| 1129 | 1130 | |
| 1130 | 1131 | if (keycode > 0) |
| 1131 | − acpi_fujitsu_hotkey_press(keycode); | |
| 1132 | + acpi_fujitsu_laptop_press(keycode); | |
| 1132 | 1133 | else if (keycode == 0) |
| 1133 | − acpi_fujitsu_hotkey_release(); | |
| 1134 | + acpi_fujitsu_laptop_release(); | |
| 1134 | 1135 | } |
| 1135 | 1136 | |
| 1136 | 1137 | /* On some models (first seen on the Skylake-based Lifebook |
| 1137 | 1138 | * E736/E746/E756), the touchpad toggle hotkey (Fn+F4) is |
| 1138 | − * handled in software; its state is queried using FUNC_RFKILL | |
| 1139 | + * handled in software; its state is queried using FUNC_FLAGS | |
| 1139 | 1140 | */ |
| 1140 | − if ((fujitsu_hotkey->rfkill_supported & BIT(26)) && | |
| 1141 | − (call_fext_func(FUNC_RFKILL, 0x1, 0x0, 0x0) & BIT(26))) { | |
| 1141 | + if ((fujitsu_laptop->flags_supported & BIT(26)) && | |
| 1142 | + (call_fext_func(FUNC_FLAGS, 0x1, 0x0, 0x0) & BIT(26))) { | |
| 1142 | 1143 | keycode = KEY_TOUCHPAD_TOGGLE; |
| 1143 | 1144 | input_report_key(input, keycode, 1); |
| 1144 | 1145 | input_sync(input); |
| @@ -1150,83 +1151,81 @@ static void acpi_fujitsu_hotkey_notify(struct acpi_device *device, u32 event) · 5 unmodified lines | ||
| 1150 | 1151 | |
| 1151 | 1152 | /* Initialization */ |
| 1152 | 1153 | |
| 1153 | −static const struct acpi_device_id fujitsu_device_ids[] = { | |
| 1154 | − {ACPI_FUJITSU_HID, 0}, | |
| 1154 | +static const struct acpi_device_id fujitsu_bl_device_ids[] = { | |
| 1155 | + {ACPI_FUJITSU_BL_HID, 0}, | |
| 1155 | 1156 | {"", 0}, |
| 1156 | 1157 | }; |
| 1157 | 1158 | |
| 1158 | −static struct acpi_driver acpi_fujitsu_driver = { | |
| 1159 | − .name = ACPI_FUJITSU_DRIVER_NAME, | |
| 1159 | +static struct acpi_driver acpi_fujitsu_bl_driver = { | |
| 1160 | + .name = ACPI_FUJITSU_BL_DRIVER_NAME, | |
| 1160 | 1161 | .class = ACPI_FUJITSU_CLASS, |
| 1161 | − .ids = fujitsu_device_ids, | |
| 1162 | + .ids = fujitsu_bl_device_ids, | |
| 1162 | 1163 | .ops = { |
| 1163 | − .add = acpi_fujitsu_add, | |
| 1164 | − .remove = acpi_fujitsu_remove, | |
| 1165 | − .notify = acpi_fujitsu_notify, | |
| 1164 | + .add = acpi_fujitsu_bl_add, | |
| 1165 | + .remove = acpi_fujitsu_bl_remove, | |
| 1166 | + .notify = acpi_fujitsu_bl_notify, | |
| 1166 | 1167 | }, |
| 1167 | 1168 | }; |
| 1168 | 1169 | |
| 1169 | −static const struct acpi_device_id fujitsu_hotkey_device_ids[] = { | |
| 1170 | − {ACPI_FUJITSU_HOTKEY_HID, 0}, | |
| 1170 | +static const struct acpi_device_id fujitsu_laptop_device_ids[] = { | |
| 1171 | + {ACPI_FUJITSU_LAPTOP_HID, 0}, | |
| 1171 | 1172 | {"", 0}, |
| 1172 | 1173 | }; |
| 1173 | 1174 | |
| 1174 | −static struct acpi_driver acpi_fujitsu_hotkey_driver = { | |
| 1175 | − .name = ACPI_FUJITSU_HOTKEY_DRIVER_NAME, | |
| 1175 | +static struct acpi_driver acpi_fujitsu_laptop_driver = { | |
| 1176 | + .name = ACPI_FUJITSU_LAPTOP_DRIVER_NAME, | |
| 1176 | 1177 | .class = ACPI_FUJITSU_CLASS, |
| 1177 | − .ids = fujitsu_hotkey_device_ids, | |
| 1178 | + .ids = fujitsu_laptop_device_ids, | |
| 1178 | 1179 | .ops = { |
| 1179 | − .add = acpi_fujitsu_hotkey_add, | |
| 1180 | − .remove = acpi_fujitsu_hotkey_remove, | |
| 1181 | − .notify = acpi_fujitsu_hotkey_notify, | |
| 1180 | + .add = acpi_fujitsu_laptop_add, | |
| 1181 | + .remove = acpi_fujitsu_laptop_remove, | |
| 1182 | + .notify = acpi_fujitsu_laptop_notify, | |
| 1182 | 1183 | }, |
| 1183 | 1184 | }; |
| 1184 | 1185 | |
| 1185 | 1186 | static const struct acpi_device_id fujitsu_ids[] __used = { |
| 1186 | − {ACPI_FUJITSU_HID, 0}, | |
| 1187 | − {ACPI_FUJITSU_HOTKEY_HID, 0}, | |
| 1187 | + {ACPI_FUJITSU_BL_HID, 0}, | |
| 1188 | + {ACPI_FUJITSU_LAPTOP_HID, 0}, | |
| 1188 | 1189 | {"", 0} |
| 1189 | 1190 | }; |
| 1190 | 1191 | MODULE_DEVICE_TABLE(acpi, fujitsu_ids); |
| 1191 | 1192 | |
| 1192 | 1193 | static int __init fujitsu_init(void) |
| 1193 | 1194 | { |
| 1194 | − int ret, result, max_brightness; | |
| 1195 | + int ret, max_brightness; | |
| 1195 | 1196 | |
| 1196 | 1197 | if (acpi_disabled) |
| 1197 | 1198 | return -ENODEV; |
| 1198 | 1199 | |
| 1199 | − fujitsu = kzalloc(sizeof(struct fujitsu_t), GFP_KERNEL); | |
| 1200 | − if (!fujitsu) | |
| 1200 | + fujitsu_bl = kzalloc(sizeof(struct fujitsu_bl), GFP_KERNEL); | |
| 1201 | + if (!fujitsu_bl) | |
| 1201 | 1202 | return -ENOMEM; |
| 1202 | − fujitsu->keycode1 = KEY_PROG1; | |
| 1203 | − fujitsu->keycode2 = KEY_PROG2; | |
| 1204 | − fujitsu->keycode3 = KEY_PROG3; | |
| 1205 | − fujitsu->keycode4 = KEY_PROG4; | |
| 1206 | − fujitsu->keycode5 = KEY_RFKILL; | |
| 1203 | + fujitsu_bl->keycode1 = KEY_PROG1; | |
| 1204 | + fujitsu_bl->keycode2 = KEY_PROG2; | |
| 1205 | + fujitsu_bl->keycode3 = KEY_PROG3; | |
| 1206 | + fujitsu_bl->keycode4 = KEY_PROG4; | |
| 1207 | + fujitsu_bl->keycode5 = KEY_RFKILL; | |
| 1207 | 1208 | dmi_check_system(fujitsu_dmi_table); |
| 1208 | 1209 | |
| 1209 | − result = acpi_bus_register_driver(&acpi_fujitsu_driver); | |
| 1210 | − if (result < 0) { | |
| 1211 | − ret = -ENODEV; | |
| 1210 | + ret = acpi_bus_register_driver(&acpi_fujitsu_bl_driver); | |
| 1211 | + if (ret) | |
| 1212 | 1212 | goto fail_acpi; |
| 1213 | − } | |
| 1214 | 1213 | |
| 1215 | 1214 | /* Register platform stuff */ |
| 1216 | 1215 | |
| 1217 | − fujitsu->pf_device = platform_device_alloc("fujitsu-laptop", -1); | |
| 1218 | − if (!fujitsu->pf_device) { | |
| 1216 | + fujitsu_bl->pf_device = platform_device_alloc("fujitsu-laptop", -1); | |
| 1217 | + if (!fujitsu_bl->pf_device) { | |
| 1219 | 1218 | ret = -ENOMEM; |
| 1220 | 1219 | goto fail_platform_driver; |
| 1221 | 1220 | } |
| 1222 | 1221 | |
| 1223 | − ret = platform_device_add(fujitsu->pf_device); | |
| 1222 | + ret = platform_device_add(fujitsu_bl->pf_device); | |
| 1224 | 1223 | if (ret) |
| 1225 | 1224 | goto fail_platform_device1; |
| 1226 | 1225 | |
| 1227 | 1226 | ret = |
| 1228 | − sysfs_create_group(&fujitsu->pf_device->dev.kobj, | |
| 1229 | − &fujitsupf_attribute_group); | |
| 1227 | + sysfs_create_group(&fujitsu_bl->pf_device->dev.kobj, | |
| 1228 | + &fujitsu_pf_attribute_group); | |
| 1230 | 1229 | if (ret) |
| 1231 | 1230 | goto fail_platform_device2; |
| 1232 | 1231 | |
| @@ -1236,90 +1235,88 @@ static int __init fujitsu_init(void) · 3 unmodified lines | ||
| 1236 | 1235 | struct backlight_properties props; |
| 1237 | 1236 | |
| 1238 | 1237 | memset(&props, 0, sizeof(struct backlight_properties)); |
| 1239 | − max_brightness = fujitsu->max_brightness; | |
| 1238 | + max_brightness = fujitsu_bl->max_brightness; | |
| 1240 | 1239 | props.type = BACKLIGHT_PLATFORM; |
| 1241 | 1240 | props.max_brightness = max_brightness - 1; |
| 1242 | − fujitsu->bl_device = backlight_device_register("fujitsu-laptop", | |
| 1243 | − NULL, NULL, | |
| 1244 | − &fujitsubl_ops, | |
| 1245 | − &props); | |
| 1246 | − if (IS_ERR(fujitsu->bl_device)) { | |
| 1247 | − ret = PTR_ERR(fujitsu->bl_device); | |
| 1248 | − fujitsu->bl_device = NULL; | |
| 1241 | + fujitsu_bl->bl_device = backlight_device_register("fujitsu-laptop", | |
| 1242 | + NULL, NULL, | |
| 1243 | + &fujitsu_bl_ops, | |
| 1244 | + &props); | |
| 1245 | + if (IS_ERR(fujitsu_bl->bl_device)) { | |
| 1246 | + ret = PTR_ERR(fujitsu_bl->bl_device); | |
| 1247 | + fujitsu_bl->bl_device = NULL; | |
| 1249 | 1248 | goto fail_sysfs_group; |
| 1250 | 1249 | } |
| 1251 | − fujitsu->bl_device->props.brightness = fujitsu->brightness_level; | |
| 1250 | + fujitsu_bl->bl_device->props.brightness = fujitsu_bl->brightness_level; | |
| 1252 | 1251 | } |
| 1253 | 1252 | |
| 1254 | − ret = platform_driver_register(&fujitsupf_driver); | |
| 1253 | + ret = platform_driver_register(&fujitsu_pf_driver); | |
| 1255 | 1254 | if (ret) |
| 1256 | 1255 | goto fail_backlight; |
| 1257 | 1256 | |
| 1258 | − /* Register hotkey driver */ | |
| 1257 | + /* Register laptop driver */ | |
| 1259 | 1258 | |
| 1260 | − fujitsu_hotkey = kzalloc(sizeof(struct fujitsu_hotkey_t), GFP_KERNEL); | |
| 1261 | − if (!fujitsu_hotkey) { | |
| 1259 | + fujitsu_laptop = kzalloc(sizeof(struct fujitsu_laptop), GFP_KERNEL); | |
| 1260 | + if (!fujitsu_laptop) { | |
| 1262 | 1261 | ret = -ENOMEM; |
| 1263 | − goto fail_hotkey; | |
| 1262 | + goto fail_laptop; | |
| 1264 | 1263 | } |
| 1265 | 1264 | |
| 1266 | − result = acpi_bus_register_driver(&acpi_fujitsu_hotkey_driver); | |
| 1267 | − if (result < 0) { | |
| 1268 | − ret = -ENODEV; | |
| 1269 | − goto fail_hotkey1; | |
| 1270 | − } | |
| 1265 | + ret = acpi_bus_register_driver(&acpi_fujitsu_laptop_driver); | |
| 1266 | + if (ret) | |
| 1267 | + goto fail_laptop1; | |
| 1271 | 1268 | |
| 1272 | 1269 | /* Sync backlight power status (needs FUJ02E3 device, hence deferred) */ |
| 1273 | 1270 | if (acpi_video_get_backlight_type() == acpi_backlight_vendor) { |
| 1274 | 1271 | if (call_fext_func(FUNC_BACKLIGHT, 0x2, 0x4, 0x0) == 3) |
| 1275 | − fujitsu->bl_device->props.power = FB_BLANK_POWERDOWN; | |
| 1272 | + fujitsu_bl->bl_device->props.power = FB_BLANK_POWERDOWN; | |
| 1276 | 1273 | else |
| 1277 | − fujitsu->bl_device->props.power = FB_BLANK_UNBLANK; | |
| 1274 | + fujitsu_bl->bl_device->props.power = FB_BLANK_UNBLANK; | |
| 1278 | 1275 | } |
| 1279 | 1276 | |
| 1280 | 1277 | pr_info("driver " FUJITSU_DRIVER_VERSION " successfully loaded\n"); |
| 1281 | 1278 | |
| 1282 | 1279 | return 0; |
| 1283 | 1280 | |
| 1284 | −fail_hotkey1: | |
| 1285 | − kfree(fujitsu_hotkey); | |
| 1286 | −fail_hotkey: | |
| 1287 | − platform_driver_unregister(&fujitsupf_driver); | |
| 1281 | +fail_laptop1: | |
| 1282 | + kfree(fujitsu_laptop); | |
| 1283 | +fail_laptop: | |
| 1284 | + platform_driver_unregister(&fujitsu_pf_driver); | |
| 1288 | 1285 | fail_backlight: |
| 1289 | − backlight_device_unregister(fujitsu->bl_device); | |
| 1286 | + backlight_device_unregister(fujitsu_bl->bl_device); | |
| 1290 | 1287 | fail_sysfs_group: |
| 1291 | − sysfs_remove_group(&fujitsu->pf_device->dev.kobj, | |
| 1292 | − &fujitsupf_attribute_group); | |
| 1288 | + sysfs_remove_group(&fujitsu_bl->pf_device->dev.kobj, | |
| 1289 | + &fujitsu_pf_attribute_group); | |
| 1293 | 1290 | fail_platform_device2: |
| 1294 | − platform_device_del(fujitsu->pf_device); | |
| 1291 | + platform_device_del(fujitsu_bl->pf_device); | |
| 1295 | 1292 | fail_platform_device1: |
| 1296 | − platform_device_put(fujitsu->pf_device); | |
| 1293 | + platform_device_put(fujitsu_bl->pf_device); | |
| 1297 | 1294 | fail_platform_driver: |
| 1298 | − acpi_bus_unregister_driver(&acpi_fujitsu_driver); | |
| 1295 | + acpi_bus_unregister_driver(&acpi_fujitsu_bl_driver); | |
| 1299 | 1296 | fail_acpi: |
| 1300 | − kfree(fujitsu); | |
| 1297 | + kfree(fujitsu_bl); | |
| 1301 | 1298 | |
| 1302 | 1299 | return ret; |
| 1303 | 1300 | } |
| 1304 | 1301 | |
| 1305 | 1302 | static void __exit fujitsu_cleanup(void) |
| 1306 | 1303 | { |
| 1307 | − acpi_bus_unregister_driver(&acpi_fujitsu_hotkey_driver); | |
| 1304 | + acpi_bus_unregister_driver(&acpi_fujitsu_laptop_driver); | |
| 1308 | 1305 | |
| 1309 | − kfree(fujitsu_hotkey); | |
| 1306 | + kfree(fujitsu_laptop); | |
| 1310 | 1307 | |
| 1311 | − platform_driver_unregister(&fujitsupf_driver); | |
| 1308 | + platform_driver_unregister(&fujitsu_pf_driver); | |
| 1312 | 1309 | |
| 1313 | − backlight_device_unregister(fujitsu->bl_device); | |
| 1310 | + backlight_device_unregister(fujitsu_bl->bl_device); | |
| 1314 | 1311 | |
| 1315 | − sysfs_remove_group(&fujitsu->pf_device->dev.kobj, | |
| 1316 | − &fujitsupf_attribute_group); | |
| 1312 | + sysfs_remove_group(&fujitsu_bl->pf_device->dev.kobj, | |
| 1313 | + &fujitsu_pf_attribute_group); | |
| 1317 | 1314 | |
| 1318 | − platform_device_unregister(fujitsu->pf_device); | |
| 1315 | + platform_device_unregister(fujitsu_bl->pf_device); | |
| 1319 | 1316 | |
| 1320 | − acpi_bus_unregister_driver(&acpi_fujitsu_driver); | |
| 1317 | + acpi_bus_unregister_driver(&acpi_fujitsu_bl_driver); | |
| 1321 | 1318 | |
| 1322 | − kfree(fujitsu); | |
| 1319 | + kfree(fujitsu_bl); | |
| 1323 | 1320 | |
| 1324 | 1321 | pr_info("driver unloaded\n"); |
| 1325 | 1322 | } |
| @@ -1341,7 +1338,3 @@ MODULE_AUTHOR("Jonathan Woithe, Peter Gruber, Tony Vroon"); · 15 unmodified lines | ||
| 1341 | 1338 | MODULE_DESCRIPTION("Fujitsu laptop extras support"); |
| 1342 | 1339 | MODULE_VERSION(FUJITSU_DRIVER_VERSION); |
| 1343 | 1340 | MODULE_LICENSE("GPL"); |
| 1344 | − | |
| 1345 | −MODULE_ALIAS("dmi:*:svnFUJITSUSIEMENS:*:pvr:rvnFUJITSU:rnFJNB1D3:*:cvrS6410:*"); | |
| 1346 | −MODULE_ALIAS("dmi:*:svnFUJITSUSIEMENS:*:pvr:rvnFUJITSU:rnFJNB1E6:*:cvrS6420:*"); | |
| 1347 | −MODULE_ALIAS("dmi:*:svnFUJITSU:*:pvr:rvnFUJITSU:rnFJNB19C:*:cvrS7020:*"); | |
| @@ -1241,19 +1241,32 @@ config SCSI_LPFC · 1240 unmodified lines | ||
| 1241 | 1241 | tristate "Emulex LightPulse Fibre Channel Support" |
| 1242 | 1242 | depends on PCI && SCSI |
| 1243 | 1243 | depends on SCSI_FC_ATTRS |
| 1244 | − depends on NVME_FC && NVME_TARGET_FC | |
| 1245 | 1244 | select CRC_T10DIF |
| 1246 | − help | |
| 1245 | + ---help--- | |
| 1247 | 1246 | This lpfc driver supports the Emulex LightPulse |
| 1248 | 1247 | Family of Fibre Channel PCI host adapters. |
| 1249 | 1248 | |
| 1250 | 1249 | config SCSI_LPFC_DEBUG_FS |
| 1251 | 1250 | bool "Emulex LightPulse Fibre Channel debugfs Support" |
| 1252 | 1251 | depends on SCSI_LPFC && DEBUG_FS |
| 1253 | − help | |
| 1252 | + ---help--- | |
| 1254 | 1253 | This makes debugging information from the lpfc driver |
| 1255 | 1254 | available via the debugfs filesystem. |
| 1256 | 1255 | |
| 1256 | +config LPFC_NVME_INITIATOR | |
| 1257 | + bool "Emulex LightPulse Fibre Channel NVME Initiator Support" | |
| 1258 | + depends on SCSI_LPFC && NVME_FC | |
| 1259 | + ---help--- | |
| 1260 | + This enables NVME Initiator support in the Emulex lpfc driver. | |
| 1261 | + | |
| 1262 | +config LPFC_NVME_TARGET | |
| 1263 | + bool "Emulex LightPulse Fibre Channel NVME Initiator Support" | |
| 1264 | + depends on SCSI_LPFC && NVME_TARGET_FC | |
| 1265 | + ---help--- | |
| 1266 | + This enables NVME Target support in the Emulex lpfc driver. | |
| 1267 | + Target enablement must still be enabled on a per adapter | |
| 1268 | + basis by module parameters. | |
| 1269 | + | |
| 1257 | 1270 | config SCSI_SIM710 |
| 1258 | 1271 | tristate "Simple 53c710 SCSI support (Compaq, NCR machines)" |
| 1259 | 1272 | depends on (EISA || MCA) && SCSI |
| @@ -468,7 +468,7 @@ err_out: · 467 unmodified lines | ||
| 468 | 468 | return -1; |
| 469 | 469 | |
| 470 | 470 | err_blink: |
| 471 | − return (status > 16) & 0xFF; | |
| 471 | + return (status >> 16) & 0xFF; | |
| 472 | 472 | } |
| 473 | 473 | |
| 474 | 474 | static inline u32 aac_get_vector(struct aac_dev *dev) |
| @@ -6278,7 +6278,7 @@ ahd_reset(struct ahd_softc *ahd, int reinit) · 6277 unmodified lines | ||
| 6278 | 6278 | * does not disable its parity logic prior to |
| 6279 | 6279 | * the start of the reset. This may cause a |
| 6280 | 6280 | * parity error to be detected and thus a |
| 6281 | − * spurious SERR or PERR assertion. Disble | |
| 6281 | + * spurious SERR or PERR assertion. Disable | |
| 6282 | 6282 | * PERR and SERR responses during the CHIPRST. |
| 6283 | 6283 | */ |
| 6284 | 6284 | mod_cmd = cmd & ~(PCIM_CMD_PERRESPEN|PCIM_CMD_SERRESPEN); |
| @@ -561,8 +561,12 @@ static void iscsi_complete_task(struct iscsi_task *task, int state) · 560 unmodified lines | ||
| 561 | 561 | WARN_ON_ONCE(task->state == ISCSI_TASK_FREE); |
| 562 | 562 | task->state = state; |
| 563 | 563 | |
| 564 | − if (!list_empty(&task->running)) | |
| 564 | + spin_lock_bh(&conn->taskqueuelock); | |
| 565 | + if (!list_empty(&task->running)) { | |
| 566 | + pr_debug_once("%s while task on list", __func__); | |
| 565 | 567 | list_del_init(&task->running); |
| 568 | + } | |
| 569 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 566 | 570 | |
| 567 | 571 | if (conn->task == task) |
| 568 | 572 | conn->task = NULL; |
| @@ -784,7 +788,9 @@ __iscsi_conn_send_pdu(struct iscsi_conn *conn, struct iscsi_hdr *hdr, · 215 unmodified lines | ||
| 784 | 788 | if (session->tt->xmit_task(task)) |
| 785 | 789 | goto free_task; |
| 786 | 790 | } else { |
| 791 | + spin_lock_bh(&conn->taskqueuelock); | |
| 787 | 792 | list_add_tail(&task->running, &conn->mgmtqueue); |
| 793 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 788 | 794 | iscsi_conn_queue_work(conn); |
| 789 | 795 | } |
| 790 | 796 | |
| @@ -1475,8 +1481,10 @@ void iscsi_requeue_task(struct iscsi_task *task) · 684 unmodified lines | ||
| 1475 | 1481 | * this may be on the requeue list already if the xmit_task callout |
| 1476 | 1482 | * is handling the r2ts while we are adding new ones |
| 1477 | 1483 | */ |
| 1484 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1478 | 1485 | if (list_empty(&task->running)) |
| 1479 | 1486 | list_add_tail(&task->running, &conn->requeue); |
| 1487 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1480 | 1488 | iscsi_conn_queue_work(conn); |
| 1481 | 1489 | } |
| 1482 | 1490 | EXPORT_SYMBOL_GPL(iscsi_requeue_task); |
| @@ -1513,22 +1521,26 @@ static int iscsi_data_xmit(struct iscsi_conn *conn) · 30 unmodified lines | ||
| 1513 | 1521 | * only have one nop-out as a ping from us and targets should not |
| 1514 | 1522 | * overflow us with nop-ins |
| 1515 | 1523 | */ |
| 1524 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1516 | 1525 | check_mgmt: |
| 1517 | 1526 | while (!list_empty(&conn->mgmtqueue)) { |
| 1518 | 1527 | conn->task = list_entry(conn->mgmtqueue.next, |
| 1519 | 1528 | struct iscsi_task, running); |
| 1520 | 1529 | list_del_init(&conn->task->running); |
| 1530 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1521 | 1531 | if (iscsi_prep_mgmt_task(conn, conn->task)) { |
| 1522 | 1532 | /* regular RX path uses back_lock */ |
| 1523 | 1533 | spin_lock_bh(&conn->session->back_lock); |
| 1524 | 1534 | __iscsi_put_task(conn->task); |
| 1525 | 1535 | spin_unlock_bh(&conn->session->back_lock); |
| 1526 | 1536 | conn->task = NULL; |
| 1537 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1527 | 1538 | continue; |
| 1528 | 1539 | } |
| 1529 | 1540 | rc = iscsi_xmit_task(conn); |
| 1530 | 1541 | if (rc) |
| 1531 | 1542 | goto done; |
| 1543 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1532 | 1544 | } |
| 1533 | 1545 | |
| 1534 | 1546 | /* process pending command queue */ |
| @@ -1536,19 +1548,24 @@ check_mgmt: · 1 unmodified line | ||
| 1536 | 1548 | conn->task = list_entry(conn->cmdqueue.next, struct iscsi_task, |
| 1537 | 1549 | running); |
| 1538 | 1550 | list_del_init(&conn->task->running); |
| 1551 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1539 | 1552 | if (conn->session->state == ISCSI_STATE_LOGGING_OUT) { |
| 1540 | 1553 | fail_scsi_task(conn->task, DID_IMM_RETRY); |
| 1554 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1541 | 1555 | continue; |
| 1542 | 1556 | } |
| 1543 | 1557 | rc = iscsi_prep_scsi_cmd_pdu(conn->task); |
| 1544 | 1558 | if (rc) { |
| 1545 | 1559 | if (rc == -ENOMEM || rc == -EACCES) { |
| 1560 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1546 | 1561 | list_add_tail(&conn->task->running, |
| 1547 | 1562 | &conn->cmdqueue); |
| 1548 | 1563 | conn->task = NULL; |
| 1564 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1549 | 1565 | goto done; |
| 1550 | 1566 | } else |
| 1551 | 1567 | fail_scsi_task(conn->task, DID_ABORT); |
| 1568 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1552 | 1569 | continue; |
| 1553 | 1570 | } |
| 1554 | 1571 | rc = iscsi_xmit_task(conn); |
| @@ -1559,6 +1576,7 @@ check_mgmt: · 4 unmodified lines | ||
| 1559 | 1576 | * we need to check the mgmt queue for nops that need to |
| 1560 | 1577 | * be sent to aviod starvation |
| 1561 | 1578 | */ |
| 1579 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1562 | 1580 | if (!list_empty(&conn->mgmtqueue)) |
| 1563 | 1581 | goto check_mgmt; |
| 1564 | 1582 | } |
| @@ -1578,12 +1596,15 @@ check_mgmt: · 13 unmodified lines | ||
| 1578 | 1596 | conn->task = task; |
| 1579 | 1597 | list_del_init(&conn->task->running); |
| 1580 | 1598 | conn->task->state = ISCSI_TASK_RUNNING; |
| 1599 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1581 | 1600 | rc = iscsi_xmit_task(conn); |
| 1582 | 1601 | if (rc) |
| 1583 | 1602 | goto done; |
| 1603 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1584 | 1604 | if (!list_empty(&conn->mgmtqueue)) |
| 1585 | 1605 | goto check_mgmt; |
| 1586 | 1606 | } |
| 1607 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1587 | 1608 | spin_unlock_bh(&conn->session->frwd_lock); |
| 1588 | 1609 | return -ENODATA; |
| 1589 | 1610 | |
| @@ -1739,7 +1760,9 @@ int iscsi_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *sc) · 149 unmodified lines | ||
| 1739 | 1760 | goto prepd_reject; |
| 1740 | 1761 | } |
| 1741 | 1762 | } else { |
| 1763 | + spin_lock_bh(&conn->taskqueuelock); | |
| 1742 | 1764 | list_add_tail(&task->running, &conn->cmdqueue); |
| 1765 | + spin_unlock_bh(&conn->taskqueuelock); | |
| 1743 | 1766 | iscsi_conn_queue_work(conn); |
| 1744 | 1767 | } |
| 1745 | 1768 | |
| @@ -2897,6 +2920,7 @@ iscsi_conn_setup(struct iscsi_cls_session *cls_session, int dd_size, · 1151 unmodified lines | ||
| 2897 | 2920 | INIT_LIST_HEAD(&conn->mgmtqueue); |
| 2898 | 2921 | INIT_LIST_HEAD(&conn->cmdqueue); |
| 2899 | 2922 | INIT_LIST_HEAD(&conn->requeue); |
| 2923 | + spin_lock_init(&conn->taskqueuelock); | |
| 2900 | 2924 | INIT_WORK(&conn->xmitwork, iscsi_xmitworker); |
| 2901 | 2925 | |
| 2902 | 2926 | /* allocate login_task used for the login/text sequences */ |
| @@ -99,12 +99,13 @@ struct lpfc_sli2_slim; · 98 unmodified lines | ||
| 99 | 99 | #define FC_MAX_ADPTMSG 64 |
| 100 | 100 | |
| 101 | 101 | #define MAX_HBAEVT 32 |
| 102 | +#define MAX_HBAS_NO_RESET 16 | |
| 102 | 103 | |
| 103 | 104 | /* Number of MSI-X vectors the driver uses */ |
| 104 | 105 | #define LPFC_MSIX_VECTORS 2 |
| 105 | 106 | |
| 106 | 107 | /* lpfc wait event data ready flag */ |
| 107 | −#define LPFC_DATA_READY (1<<0) | |
| 108 | +#define LPFC_DATA_READY 0 /* bit 0 */ | |
| 108 | 109 | |
| 109 | 110 | /* queue dump line buffer size */ |
| 110 | 111 | #define LPFC_LBUF_SZ 128 |
| @@ -692,6 +693,7 @@ struct lpfc_hba { · 581 unmodified lines | ||
| 692 | 693 | * capability |
| 693 | 694 | */ |
| 694 | 695 | #define HBA_NVME_IOQ_FLUSH 0x80000 /* NVME IO queues flushed. */ |
| 696 | +#define NVME_XRI_ABORT_EVENT 0x100000 | |
| 695 | 697 | |
| 696 | 698 | uint32_t fcp_ring_in_use; /* When polling test if intr-hndlr active*/ |
| 697 | 699 | struct lpfc_dmabuf slim2p; |
| @@ -3010,6 +3010,12 @@ MODULE_PARM_DESC(lpfc_poll, "FCP ring polling mode control:" · 3009 unmodified lines | ||
| 3010 | 3010 | static DEVICE_ATTR(lpfc_poll, S_IRUGO | S_IWUSR, |
| 3011 | 3011 | lpfc_poll_show, lpfc_poll_store); |
| 3012 | 3012 | |
| 3013 | +int lpfc_no_hba_reset_cnt; | |
| 3014 | +unsigned long lpfc_no_hba_reset[MAX_HBAS_NO_RESET] = { | |
| 3015 | + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}; | |
| 3016 | +module_param_array(lpfc_no_hba_reset, ulong, &lpfc_no_hba_reset_cnt, 0444); | |
| 3017 | +MODULE_PARM_DESC(lpfc_no_hba_reset, "WWPN of HBAs that should not be reset"); | |
| 3018 | + | |
| 3013 | 3019 | LPFC_ATTR(sli_mode, 0, 0, 3, |
| 3014 | 3020 | "SLI mode selector:" |
| 3015 | 3021 | " 0 - auto (SLI-3 if supported)," |
| @@ -4451,7 +4457,8 @@ lpfc_fcp_imax_store(struct device *dev, struct device_attribute *attr, · 1435 unmodified lines | ||
| 4451 | 4457 | return -EINVAL; |
| 4452 | 4458 | |
| 4453 | 4459 | phba->cfg_fcp_imax = (uint32_t)val; |
| 4454 | − for (i = 0; i < phba->io_channel_irqs; i++) | |
| 4460 | + | |
| 4461 | + for (i = 0; i < phba->io_channel_irqs; i += LPFC_MAX_EQ_DELAY_EQID_CNT) | |
| 4455 | 4462 | lpfc_modify_hba_eq_delay(phba, i); |
| 4456 | 4463 | |
| 4457 | 4464 | return strlen(buf); |
| @@ -384,7 +384,7 @@ void lpfc_free_sysfs_attr(struct lpfc_vport *); · 383 unmodified lines | ||
| 384 | 384 | extern struct device_attribute *lpfc_hba_attrs[]; |
| 385 | 385 | extern struct device_attribute *lpfc_vport_attrs[]; |
| 386 | 386 | extern struct scsi_host_template lpfc_template; |
| 387 | −extern struct scsi_host_template lpfc_template_s3; | |
| 387 | +extern struct scsi_host_template lpfc_template_no_hr; | |
| 388 | 388 | extern struct scsi_host_template lpfc_template_nvme; |
| 389 | 389 | extern struct scsi_host_template lpfc_vport_template; |
| 390 | 390 | extern struct fc_function_template lpfc_transport_functions; |
| @@ -554,3 +554,5 @@ void lpfc_nvme_abort_fcreq_cmpl(struct lpfc_hba *phba, · 163 unmodified lines | ||
| 554 | 554 | struct lpfc_wcqe_complete *abts_cmpl); |
| 555 | 555 | extern int lpfc_enable_nvmet_cnt; |
| 556 | 556 | extern unsigned long long lpfc_enable_nvmet[]; |
| 557 | +extern int lpfc_no_hba_reset_cnt; | |
| 558 | +extern unsigned long lpfc_no_hba_reset[]; | |
| @@ -939,8 +939,8 @@ lpfc_cmpl_ct_cmd_gft_id(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, · 938 unmodified lines | ||
| 939 | 939 | "FC4 x%08x, Data: x%08x x%08x\n", |
| 940 | 940 | ndlp, did, ndlp->nlp_fc4_type, |
| 941 | 941 | FC_TYPE_FCP, FC_TYPE_NVME); |
| 942 | + ndlp->nlp_prev_state = NLP_STE_REG_LOGIN_ISSUE; | |
| 942 | 943 | } |
| 943 | − ndlp->nlp_prev_state = NLP_STE_REG_LOGIN_ISSUE; | |
| 944 | 944 | lpfc_nlp_set_state(vport, ndlp, NLP_STE_PRLI_ISSUE); |
| 945 | 945 | lpfc_issue_els_prli(vport, ndlp, 0); |
| 946 | 946 | } else |
| @@ -3653,17 +3653,6 @@ lpfc_idiag_queacc_write(struct file *file, const char __user *buf, · 3652 unmodified lines | ||
| 3653 | 3653 | idiag.ptr_private = phba->sli4_hba.nvmels_cq; |
| 3654 | 3654 | goto pass_check; |
| 3655 | 3655 | } |
| 3656 | − /* NVME LS complete queue */ | |
| 3657 | − if (phba->sli4_hba.nvmels_cq && | |
| 3658 | − phba->sli4_hba.nvmels_cq->queue_id == queid) { | |
| 3659 | − /* Sanity check */ | |
| 3660 | − rc = lpfc_idiag_que_param_check( | |
| 3661 | − phba->sli4_hba.nvmels_cq, index, count); | |
| 3662 | − if (rc) | |
| 3663 | − goto error_out; | |
| 3664 | − idiag.ptr_private = phba->sli4_hba.nvmels_cq; | |
| 3665 | − goto pass_check; | |
| 3666 | − } | |
| 3667 | 3656 | /* FCP complete queue */ |
| 3668 | 3657 | if (phba->sli4_hba.fcp_cq) { |
| 3669 | 3658 | for (qidx = 0; qidx < phba->cfg_fcp_io_channel; |
| @@ -3738,17 +3727,6 @@ lpfc_idiag_queacc_write(struct file *file, const char __user *buf, · 68 unmodified lines | ||
| 3738 | 3727 | idiag.ptr_private = phba->sli4_hba.nvmels_wq; |
| 3739 | 3728 | goto pass_check; |
| 3740 | 3729 | } |
| 3741 | − /* NVME LS work queue */ | |
| 3742 | − if (phba->sli4_hba.nvmels_wq && | |
| 3743 | − phba->sli4_hba.nvmels_wq->queue_id == queid) { | |
| 3744 | − /* Sanity check */ | |
| 3745 | − rc = lpfc_idiag_que_param_check( | |
| 3746 | − phba->sli4_hba.nvmels_wq, index, count); | |
| 3747 | − if (rc) | |
| 3748 | − goto error_out; | |
| 3749 | − idiag.ptr_private = phba->sli4_hba.nvmels_wq; | |
| 3750 | − goto pass_check; | |
| 3751 | − } | |
| 3752 | 3730 | /* FCP work queue */ |
| 3753 | 3731 | if (phba->sli4_hba.fcp_wq) { |
| 3754 | 3732 | for (qidx = 0; qidx < phba->cfg_fcp_io_channel; |
| @@ -5177,15 +5177,15 @@ lpfc_rdp_res_speed(struct fc_rdp_port_speed_desc *desc, struct lpfc_hba *phba) · 5176 unmodified lines | ||
| 5177 | 5177 | |
| 5178 | 5178 | static uint32_t |
| 5179 | 5179 | lpfc_rdp_res_diag_port_names(struct fc_rdp_port_name_desc *desc, |
| 5180 | − struct lpfc_hba *phba) | |
| 5180 | + struct lpfc_vport *vport) | |
| 5181 | 5181 | { |
| 5182 | 5182 | |
| 5183 | 5183 | desc->tag = cpu_to_be32(RDP_PORT_NAMES_DESC_TAG); |
| 5184 | 5184 | |
| 5185 | − memcpy(desc->port_names.wwnn, phba->wwnn, | |
| 5185 | + memcpy(desc->port_names.wwnn, &vport->fc_nodename, | |
| 5186 | 5186 | sizeof(desc->port_names.wwnn)); |
| 5187 | 5187 | |
| 5188 | − memcpy(desc->port_names.wwpn, phba->wwpn, | |
| 5188 | + memcpy(desc->port_names.wwpn, &vport->fc_portname, | |
| 5189 | 5189 | sizeof(desc->port_names.wwpn)); |
| 5190 | 5190 | |
| 5191 | 5191 | desc->length = cpu_to_be32(sizeof(desc->port_names)); |
| @@ -5279,7 +5279,7 @@ lpfc_els_rdp_cmpl(struct lpfc_hba *phba, struct lpfc_rdp_context *rdp_context, · 87 unmodified lines | ||
| 5279 | 5279 | len += lpfc_rdp_res_link_error((struct fc_rdp_link_error_status_desc *) |
| 5280 | 5280 | (len + pcmd), &rdp_context->link_stat); |
| 5281 | 5281 | len += lpfc_rdp_res_diag_port_names((struct fc_rdp_port_name_desc *) |
| 5282 | − (len + pcmd), phba); | |
| 5282 | + (len + pcmd), vport); | |
| 5283 | 5283 | len += lpfc_rdp_res_attach_port_names((struct fc_rdp_port_name_desc *) |
| 5284 | 5284 | (len + pcmd), vport, ndlp); |
| 5285 | 5285 | len += lpfc_rdp_res_fec_desc((struct fc_fec_rdp_desc *)(len + pcmd), |
| @@ -8371,11 +8371,17 @@ lpfc_cmpl_reg_new_vport(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) · 3085 unmodified lines | ||
| 8371 | 8371 | spin_lock_irq(shost->host_lock); |
| 8372 | 8372 | vport->fc_flag |= FC_VPORT_NEEDS_REG_VPI; |
| 8373 | 8373 | spin_unlock_irq(shost->host_lock); |
| 8374 | − if (vport->port_type == LPFC_PHYSICAL_PORT | |
| 8375 | − && !(vport->fc_flag & FC_LOGO_RCVD_DID_CHNG)) | |
| 8376 | − lpfc_issue_init_vfi(vport); | |
| 8377 | − else | |
| 8374 | + if (mb->mbxStatus == MBX_NOT_FINISHED) | |
| 8375 | + break; | |
| 8376 | + if ((vport->port_type == LPFC_PHYSICAL_PORT) && | |
| 8377 | + !(vport->fc_flag & FC_LOGO_RCVD_DID_CHNG)) { | |
| 8378 | + if (phba->sli_rev == LPFC_SLI_REV4) | |
| 8379 | + lpfc_issue_init_vfi(vport); | |
| 8380 | + else | |
| 8381 | + lpfc_initial_flogi(vport); | |
| 8382 | + } else { | |
| 8378 | 8383 | lpfc_initial_fdisc(vport); |
| 8384 | + } | |
| 8379 | 8385 | break; |
| 8380 | 8386 | } |
| 8381 | 8387 | } else { |
| @@ -313,8 +313,7 @@ lpfc_dev_loss_tmo_handler(struct lpfc_nodelist *ndlp) · 312 unmodified lines | ||
| 313 | 313 | ndlp->nlp_state, ndlp->nlp_rpi); |
| 314 | 314 | } |
| 315 | 315 | |
| 316 | − if (!(vport->load_flag & FC_UNLOADING) && | |
| 317 | − !(ndlp->nlp_flag & NLP_DELAY_TMO) && | |
| 316 | + if (!(ndlp->nlp_flag & NLP_DELAY_TMO) && | |
| 318 | 317 | !(ndlp->nlp_flag & NLP_NPR_2B_DISC) && |
| 319 | 318 | (ndlp->nlp_state != NLP_STE_UNMAPPED_NODE) && |
| 320 | 319 | (ndlp->nlp_state != NLP_STE_REG_LOGIN_ISSUE) && |
| @@ -641,6 +640,8 @@ lpfc_work_done(struct lpfc_hba *phba) · 320 unmodified lines | ||
| 641 | 640 | lpfc_handle_rrq_active(phba); |
| 642 | 641 | if (phba->hba_flag & FCP_XRI_ABORT_EVENT) |
| 643 | 642 | lpfc_sli4_fcp_xri_abort_event_proc(phba); |
| 643 | + if (phba->hba_flag & NVME_XRI_ABORT_EVENT) | |
| 644 | + lpfc_sli4_nvme_xri_abort_event_proc(phba); | |
| 644 | 645 | if (phba->hba_flag & ELS_XRI_ABORT_EVENT) |
| 645 | 646 | lpfc_sli4_els_xri_abort_event_proc(phba); |
| 646 | 647 | if (phba->hba_flag & ASYNC_EVENT) |
| @@ -2173,7 +2174,7 @@ lpfc_mbx_cmpl_fcf_scan_read_fcf_rec(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq) · 1526 unmodified lines | ||
| 2173 | 2174 | uint32_t boot_flag, addr_mode; |
| 2174 | 2175 | uint16_t fcf_index, next_fcf_index; |
| 2175 | 2176 | struct lpfc_fcf_rec *fcf_rec = NULL; |
| 2176 | − uint16_t vlan_id; | |
| 2177 | + uint16_t vlan_id = LPFC_FCOE_NULL_VID; | |
| 2177 | 2178 | bool select_new_fcf; |
| 2178 | 2179 | int rc; |
| 2179 | 2180 | |
| @@ -4020,9 +4021,11 @@ lpfc_register_remote_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) · 1840 unmodified lines | ||
| 4020 | 4021 | rdata = rport->dd_data; |
| 4021 | 4022 | /* break the link before dropping the ref */ |
| 4022 | 4023 | ndlp->rport = NULL; |
| 4023 | − if (rdata && rdata->pnode == ndlp) | |
| 4024 | − lpfc_nlp_put(ndlp); | |
| 4025 | − rdata->pnode = NULL; | |
| 4024 | + if (rdata) { | |
| 4025 | + if (rdata->pnode == ndlp) | |
| 4026 | + lpfc_nlp_put(ndlp); | |
| 4027 | + rdata->pnode = NULL; | |
| 4028 | + } | |
| 4026 | 4029 | /* drop reference for earlier registeration */ |
| 4027 | 4030 | put_device(&rport->dev); |
| 4028 | 4031 | } |
| @@ -4344,9 +4347,8 @@ lpfc_initialize_node(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp, · 315 unmodified lines | ||
| 4344 | 4347 | { |
| 4345 | 4348 | INIT_LIST_HEAD(&ndlp->els_retry_evt.evt_listp); |
| 4346 | 4349 | INIT_LIST_HEAD(&ndlp->dev_loss_evt.evt_listp); |
| 4347 | − init_timer(&ndlp->nlp_delayfunc); | |
| 4348 | − ndlp->nlp_delayfunc.function = lpfc_els_retry_delay; | |
| 4349 | − ndlp->nlp_delayfunc.data = (unsigned long)ndlp; | |
| 4350 | + setup_timer(&ndlp->nlp_delayfunc, lpfc_els_retry_delay, | |
| 4351 | + (unsigned long)ndlp); | |
| 4350 | 4352 | ndlp->nlp_DID = did; |
| 4351 | 4353 | ndlp->vport = vport; |
| 4352 | 4354 | ndlp->phba = vport->phba; |
| @@ -4606,9 +4608,9 @@ lpfc_sli4_dequeue_nport_iocbs(struct lpfc_hba *phba, · 253 unmodified lines | ||
| 4606 | 4608 | pring = qp->pring; |
| 4607 | 4609 | if (!pring) |
| 4608 | 4610 | continue; |
| 4609 | − spin_lock_irq(&pring->ring_lock); | |
| 4611 | + spin_lock(&pring->ring_lock); | |
| 4610 | 4612 | __lpfc_dequeue_nport_iocbs(phba, ndlp, pring, dequeue_list); |
| 4611 | − spin_unlock_irq(&pring->ring_lock); | |
| 4613 | + spin_unlock(&pring->ring_lock); | |
| 4612 | 4614 | } |
| 4613 | 4615 | spin_unlock_irq(&phba->hbalock); |
| 4614 | 4616 | } |
| @@ -1001,7 +1001,7 @@ struct eq_delay_info { · 1000 unmodified lines | ||
| 1001 | 1001 | uint32_t phase; |
| 1002 | 1002 | uint32_t delay_multi; |
| 1003 | 1003 | }; |
| 1004 | −#define LPFC_MAX_EQ_DELAY 8 | |
| 1004 | +#define LPFC_MAX_EQ_DELAY_EQID_CNT 8 | |
| 1005 | 1005 | |
| 1006 | 1006 | struct sgl_page_pairs { |
| 1007 | 1007 | uint32_t sgl_pg0_addr_lo; |
| @@ -1070,7 +1070,7 @@ struct lpfc_mbx_modify_eq_delay { · 62 unmodified lines | ||
| 1070 | 1070 | union { |
| 1071 | 1071 | struct { |
| 1072 | 1072 | uint32_t num_eq; |
| 1073 | − struct eq_delay_info eq[LPFC_MAX_EQ_DELAY]; | |
| 1073 | + struct eq_delay_info eq[LPFC_MAX_EQ_DELAY_EQID_CNT]; | |
| 1074 | 1074 | } request; |
| 1075 | 1075 | struct { |
| 1076 | 1076 | uint32_t word0; |
| @@ -3555,6 +3555,44 @@ out_free_mem: · 3554 unmodified lines | ||
| 3555 | 3555 | return rc; |
| 3556 | 3556 | } |
| 3557 | 3557 | |
| 3558 | +static uint64_t | |
| 3559 | +lpfc_get_wwpn(struct lpfc_hba *phba) | |
| 3560 | +{ | |
| 3561 | + uint64_t wwn; | |
| 3562 | + int rc; | |
| 3563 | + LPFC_MBOXQ_t *mboxq; | |
| 3564 | + MAILBOX_t *mb; | |
| 3565 | + | |
| 3566 | + | |
| 3567 | + mboxq = (LPFC_MBOXQ_t *) mempool_alloc(phba->mbox_mem_pool, | |
| 3568 | + GFP_KERNEL); | |
| 3569 | + if (!mboxq) | |
| 3570 | + return (uint64_t)-1; | |
| 3571 | + | |
| 3572 | + /* First get WWN of HBA instance */ | |
| 3573 | + lpfc_read_nv(phba, mboxq); | |
| 3574 | + rc = lpfc_sli_issue_mbox(phba, mboxq, MBX_POLL); | |
| 3575 | + if (rc != MBX_SUCCESS) { | |
| 3576 | + lpfc_printf_log(phba, KERN_ERR, LOG_SLI, | |
| 3577 | + "6019 Mailbox failed , mbxCmd x%x " | |
| 3578 | + "READ_NV, mbxStatus x%x\n", | |
| 3579 | + bf_get(lpfc_mqe_command, &mboxq->u.mqe), | |
| 3580 | + bf_get(lpfc_mqe_status, &mboxq->u.mqe)); | |
| 3581 | + mempool_free(mboxq, phba->mbox_mem_pool); | |
| 3582 | + return (uint64_t) -1; | |
| 3583 | + } | |
| 3584 | + mb = &mboxq->u.mb; | |
| 3585 | + memcpy(&wwn, (char *)mb->un.varRDnvp.portname, sizeof(uint64_t)); | |
| 3586 | + /* wwn is WWPN of HBA instance */ | |
| 3587 | + mempool_free(mboxq, phba->mbox_mem_pool); | |
| 3588 | + if (phba->sli_rev == LPFC_SLI_REV4) | |
| 3589 | + return be64_to_cpu(wwn); | |
| 3590 | + else | |
| 3591 | + return (((wwn & 0xffffffff00000000) >> 32) | | |
| 3592 | + ((wwn & 0x00000000ffffffff) << 32)); | |
| 3593 | + | |
| 3594 | +} | |
| 3595 | + | |
| 3558 | 3596 | /** |
| 3559 | 3597 | * lpfc_sli4_nvme_sgl_update - update xri-sgl sizing and mapping |
| 3560 | 3598 | * @phba: pointer to lpfc hba data structure. |
| @@ -3676,17 +3714,32 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev) · 115 unmodified lines | ||
| 3676 | 3714 | struct lpfc_vport *vport; |
| 3677 | 3715 | struct Scsi_Host *shost = NULL; |
| 3678 | 3716 | int error = 0; |
| 3717 | + int i; | |
| 3718 | + uint64_t wwn; | |
| 3719 | + bool use_no_reset_hba = false; | |
| 3720 | + | |
| 3721 | + wwn = lpfc_get_wwpn(phba); | |
| 3722 | + | |
| 3723 | + for (i = 0; i < lpfc_no_hba_reset_cnt; i++) { | |
| 3724 | + if (wwn == lpfc_no_hba_reset[i]) { | |
| 3725 | + lpfc_printf_log(phba, KERN_ERR, LOG_SLI, | |
| 3726 | + "6020 Setting use_no_reset port=%llx\n", | |
| 3727 | + wwn); | |
| 3728 | + use_no_reset_hba = true; | |
| 3729 | + break; | |
| 3730 | + } | |
| 3731 | + } | |
| 3679 | 3732 | |
| 3680 | 3733 | if (phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP) { |
| 3681 | 3734 | if (dev != &phba->pcidev->dev) { |
| 3682 | 3735 | shost = scsi_host_alloc(&lpfc_vport_template, |
| 3683 | 3736 | sizeof(struct lpfc_vport)); |
| 3684 | 3737 | } else { |
| 3685 | − if (phba->sli_rev == LPFC_SLI_REV4) | |
| 3738 | + if (!use_no_reset_hba) | |
| 3686 | 3739 | shost = scsi_host_alloc(&lpfc_template, |
| 3687 | 3740 | sizeof(struct lpfc_vport)); |
| 3688 | 3741 | else |
| 3689 | − shost = scsi_host_alloc(&lpfc_template_s3, | |
| 3742 | + shost = scsi_host_alloc(&lpfc_template_no_hr, | |
| 3690 | 3743 | sizeof(struct lpfc_vport)); |
| 3691 | 3744 | } |
| 3692 | 3745 | } else if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) { |
| @@ -3734,17 +3787,14 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev) · 41 unmodified lines | ||
| 3734 | 3787 | INIT_LIST_HEAD(&vport->rcv_buffer_list); |
| 3735 | 3788 | spin_lock_init(&vport->work_port_lock); |
| 3736 | 3789 | |
| 3737 | − init_timer(&vport->fc_disctmo); | |
| 3738 | − vport->fc_disctmo.function = lpfc_disc_timeout; | |
| 3739 | − vport->fc_disctmo.data = (unsigned long)vport; | |
| 3790 | + setup_timer(&vport->fc_disctmo, lpfc_disc_timeout, | |
| 3791 | + (unsigned long)vport); | |
| 3740 | 3792 | |
| 3741 | − init_timer(&vport->els_tmofunc); | |
| 3742 | − vport->els_tmofunc.function = lpfc_els_timeout; | |
| 3743 | − vport->els_tmofunc.data = (unsigned long)vport; | |
| 3793 | + setup_timer(&vport->els_tmofunc, lpfc_els_timeout, | |
| 3794 | + (unsigned long)vport); | |
| 3744 | 3795 | |
| 3745 | − init_timer(&vport->delayed_disc_tmo); | |
| 3746 | − vport->delayed_disc_tmo.function = lpfc_delayed_disc_tmo; | |
| 3747 | − vport->delayed_disc_tmo.data = (unsigned long)vport; | |
| 3796 | + setup_timer(&vport->delayed_disc_tmo, lpfc_delayed_disc_tmo, | |
| 3797 | + (unsigned long)vport); | |
| 3748 | 3798 | |
| 3749 | 3799 | error = scsi_add_host_with_dma(shost, dev, &phba->pcidev->dev); |
| 3750 | 3800 | if (error) |
| @@ -5406,21 +5456,15 @@ lpfc_setup_driver_resource_phase1(struct lpfc_hba *phba) · 1655 unmodified lines | ||
| 5406 | 5456 | INIT_LIST_HEAD(&phba->luns); |
| 5407 | 5457 | |
| 5408 | 5458 | /* MBOX heartbeat timer */ |
| 5409 | − init_timer(&psli->mbox_tmo); | |
| 5410 | − psli->mbox_tmo.function = lpfc_mbox_timeout; | |
| 5411 | − psli->mbox_tmo.data = (unsigned long) phba; | |
| 5459 | + setup_timer(&psli->mbox_tmo, lpfc_mbox_timeout, (unsigned long)phba); | |
| 5412 | 5460 | /* Fabric block timer */ |
| 5413 | − init_timer(&phba->fabric_block_timer); | |
| 5414 | − phba->fabric_block_timer.function = lpfc_fabric_block_timeout; | |
| 5415 | − phba->fabric_block_timer.data = (unsigned long) phba; | |
| 5461 | + setup_timer(&phba->fabric_block_timer, lpfc_fabric_block_timeout, | |
| 5462 | + (unsigned long)phba); | |
| 5416 | 5463 | /* EA polling mode timer */ |
| 5417 | − init_timer(&phba->eratt_poll); | |
| 5418 | − phba->eratt_poll.function = lpfc_poll_eratt; | |
| 5419 | − phba->eratt_poll.data = (unsigned long) phba; | |
| 5464 | + setup_timer(&phba->eratt_poll, lpfc_poll_eratt, | |
| 5465 | + (unsigned long)phba); | |
| 5420 | 5466 | /* Heartbeat timer */ |
| 5421 | − init_timer(&phba->hb_tmofunc); | |
| 5422 | − phba->hb_tmofunc.function = lpfc_hb_timeout; | |
| 5423 | − phba->hb_tmofunc.data = (unsigned long)phba; | |
| 5467 | + setup_timer(&phba->hb_tmofunc, lpfc_hb_timeout, (unsigned long)phba); | |
| 5424 | 5468 | |
| 5425 | 5469 | return 0; |
| 5426 | 5470 | } |
| @@ -5446,9 +5490,8 @@ lpfc_sli_driver_resource_setup(struct lpfc_hba *phba) · 19 unmodified lines | ||
| 5446 | 5490 | */ |
| 5447 | 5491 | |
| 5448 | 5492 | /* FCP polling mode timer */ |
| 5449 | − init_timer(&phba->fcp_poll_timer); | |
| 5450 | − phba->fcp_poll_timer.function = lpfc_poll_timeout; | |
| 5451 | − phba->fcp_poll_timer.data = (unsigned long) phba; | |
| 5493 | + setup_timer(&phba->fcp_poll_timer, lpfc_poll_timeout, | |
| 5494 | + (unsigned long)phba); | |
| 5452 | 5495 | |
| 5453 | 5496 | /* Host attention work mask setup */ |
| 5454 | 5497 | phba->work_ha_mask = (HA_ERATT | HA_MBATT | HA_LATT); |
| @@ -5482,7 +5525,8 @@ lpfc_sli_driver_resource_setup(struct lpfc_hba *phba) · 27 unmodified lines | ||
| 5482 | 5525 | |
| 5483 | 5526 | /* Initialize the host templates the configured values. */ |
| 5484 | 5527 | lpfc_vport_template.sg_tablesize = phba->cfg_sg_seg_cnt; |
| 5485 | − lpfc_template_s3.sg_tablesize = phba->cfg_sg_seg_cnt; | |
| 5528 | + lpfc_template_no_hr.sg_tablesize = phba->cfg_sg_seg_cnt; | |
| 5529 | + lpfc_template.sg_tablesize = phba->cfg_sg_seg_cnt; | |
| 5486 | 5530 | |
| 5487 | 5531 | /* There are going to be 2 reserved BDEs: 1 FCP cmnd + 1 FCP rsp */ |
| 5488 | 5532 | if (phba->cfg_enable_bg) { |
| @@ -5617,14 +5661,11 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) · 128 unmodified lines | ||
| 5617 | 5661 | * Initialize timers used by driver |
| 5618 | 5662 | */ |
| 5619 | 5663 | |
| 5620 | − init_timer(&phba->rrq_tmr); | |
| 5621 | − phba->rrq_tmr.function = lpfc_rrq_timeout; | |
| 5622 | − phba->rrq_tmr.data = (unsigned long)phba; | |
| 5664 | + setup_timer(&phba->rrq_tmr, lpfc_rrq_timeout, (unsigned long)phba); | |
| 5623 | 5665 | |
| 5624 | 5666 | /* FCF rediscover timer */ |
| 5625 | − init_timer(&phba->fcf.redisc_wait); | |
| 5626 | − phba->fcf.redisc_wait.function = lpfc_sli4_fcf_redisc_wait_tmo; | |
| 5627 | − phba->fcf.redisc_wait.data = (unsigned long)phba; | |
| 5667 | + setup_timer(&phba->fcf.redisc_wait, lpfc_sli4_fcf_redisc_wait_tmo, | |
| 5668 | + (unsigned long)phba); | |
| 5628 | 5669 | |
| 5629 | 5670 | /* |
| 5630 | 5671 | * Control structure for handling external multi-buffer mailbox |
| @@ -5706,6 +5747,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) · 75 unmodified lines | ||
| 5706 | 5747 | /* Initialize the host templates with the updated values. */ |
| 5707 | 5748 | lpfc_vport_template.sg_tablesize = phba->cfg_sg_seg_cnt; |
| 5708 | 5749 | lpfc_template.sg_tablesize = phba->cfg_sg_seg_cnt; |
| 5750 | + lpfc_template_no_hr.sg_tablesize = phba->cfg_sg_seg_cnt; | |
| 5709 | 5751 | |
| 5710 | 5752 | if (phba->cfg_sg_dma_buf_size <= LPFC_MIN_SG_SLI4_BUF_SZ) |
| 5711 | 5753 | phba->cfg_sg_dma_buf_size = LPFC_MIN_SG_SLI4_BUF_SZ; |
| @@ -5736,6 +5778,8 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) · 24 unmodified lines | ||
| 5736 | 5778 | /* Initialize the Abort nvme buffer list used by driver */ |
| 5737 | 5779 | spin_lock_init(&phba->sli4_hba.abts_nvme_buf_list_lock); |
| 5738 | 5780 | INIT_LIST_HEAD(&phba->sli4_hba.lpfc_abts_nvme_buf_list); |
| 5781 | + /* Fast-path XRI aborted CQ Event work queue list */ | |
| 5782 | + INIT_LIST_HEAD(&phba->sli4_hba.sp_nvme_xri_aborted_work_queue); | |
| 5739 | 5783 | } |
| 5740 | 5784 | |
| 5741 | 5785 | /* This abort list used by worker thread */ |
| @@ -8712,12 +8756,9 @@ lpfc_sli4_queue_setup(struct lpfc_hba *phba) · 2970 unmodified lines | ||
| 8712 | 8756 | } |
| 8713 | 8757 | } |
| 8714 | 8758 | |
| 8715 | − /* | |
| 8716 | − * Configure EQ delay multipier for interrupt coalescing using | |
| 8717 | − * MODIFY_EQ_DELAY for all EQs created, LPFC_MAX_EQ_DELAY at a time. | |
| 8718 | − */ | |
| 8719 | − for (qidx = 0; qidx < io_channel; qidx += LPFC_MAX_EQ_DELAY) | |
| 8759 | + for (qidx = 0; qidx < io_channel; qidx += LPFC_MAX_EQ_DELAY_EQID_CNT) | |
| 8720 | 8760 | lpfc_modify_hba_eq_delay(phba, qidx); |
| 8761 | + | |
| 8721 | 8762 | return 0; |
| 8722 | 8763 | |
| 8723 | 8764 | out_destroy: |
| @@ -8973,6 +9014,11 @@ lpfc_sli4_cq_event_release_all(struct lpfc_hba *phba) · 249 unmodified lines | ||
| 8973 | 9014 | /* Pending ELS XRI abort events */ |
| 8974 | 9015 | list_splice_init(&phba->sli4_hba.sp_els_xri_aborted_work_queue, |
| 8975 | 9016 | &cqelist); |
| 9017 | + if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) { | |
| 9018 | + /* Pending NVME XRI abort events */ | |
| 9019 | + list_splice_init(&phba->sli4_hba.sp_nvme_xri_aborted_work_queue, | |
| 9020 | + &cqelist); | |
| 9021 | + } | |
| 8976 | 9022 | /* Pending asynnc events */ |
| 8977 | 9023 | list_splice_init(&phba->sli4_hba.sp_asynce_work_queue, |
| 8978 | 9024 | &cqelist); |
| @@ -10400,12 +10446,7 @@ lpfc_pci_remove_one_s3(struct pci_dev *pdev) · 1421 unmodified lines | ||
| 10400 | 10446 | fc_remove_host(shost); |
| 10401 | 10447 | scsi_remove_host(shost); |
| 10402 | 10448 | |
| 10403 | − /* Perform ndlp cleanup on the physical port. The nvme and nvmet | |
| 10404 | − * localports are destroyed after to cleanup all transport memory. | |
| 10405 | − */ | |
| 10406 | 10449 | lpfc_cleanup(vport); |
| 10407 | − lpfc_nvmet_destroy_targetport(phba); | |
| 10408 | − lpfc_nvme_destroy_localport(vport); | |
| 10409 | 10450 | |
| 10410 | 10451 | /* |
| 10411 | 10452 | * Bring down the SLI Layer. This step disable all interrupts, |
| @@ -12018,6 +12059,7 @@ static struct pci_driver lpfc_driver = { · 1606 unmodified lines | ||
| 12018 | 12059 | .id_table = lpfc_id_table, |
| 12019 | 12060 | .probe = lpfc_pci_probe_one, |
| 12020 | 12061 | .remove = lpfc_pci_remove_one, |
| 12062 | + .shutdown = lpfc_pci_remove_one, | |
| 12021 | 12063 | .suspend = lpfc_pci_suspend_one, |
| 12022 | 12064 | .resume = lpfc_pci_resume_one, |
| 12023 | 12065 | .err_handler = &lpfc_err_handler, |
| @@ -646,7 +646,6 @@ lpfc_sli4_nvmet_alloc(struct lpfc_hba *phba) · 645 unmodified lines | ||
| 646 | 646 | } |
| 647 | 647 | |
| 648 | 648 | dma_buf->iocbq = lpfc_sli_get_iocbq(phba); |
| 649 | − dma_buf->iocbq->iocb_flag = LPFC_IO_NVMET; | |
| 650 | 649 | if (!dma_buf->iocbq) { |
| 651 | 650 | kfree(dma_buf->context); |
| 652 | 651 | pci_pool_free(phba->lpfc_drb_pool, dma_buf->dbuf.virt, |
| @@ -658,6 +657,7 @@ lpfc_sli4_nvmet_alloc(struct lpfc_hba *phba) · 5 unmodified lines | ||
| 658 | 657 | "2621 Ran out of nvmet iocb/WQEs\n"); |
| 659 | 658 | return NULL; |
| 660 | 659 | } |
| 660 | + dma_buf->iocbq->iocb_flag = LPFC_IO_NVMET; | |
| 661 | 661 | nvmewqe = dma_buf->iocbq; |
| 662 | 662 | wqe = (union lpfc_wqe128 *)&nvmewqe->wqe; |
| 663 | 663 | /* Initialize WQE */ |
| @@ -316,7 +316,7 @@ lpfc_nvme_gen_req(struct lpfc_vport *vport, struct lpfc_dmabuf *bmp, · 315 unmodified lines | ||
| 316 | 316 | bf_set(wqe_dfctl, &wqe->gen_req.wge_ctl, 0); |
| 317 | 317 | bf_set(wqe_si, &wqe->gen_req.wge_ctl, 1); |
| 318 | 318 | bf_set(wqe_la, &wqe->gen_req.wge_ctl, 1); |
| 319 | − bf_set(wqe_rctl, &wqe->gen_req.wge_ctl, FC_RCTL_DD_UNSOL_CTL); | |
| 319 | + bf_set(wqe_rctl, &wqe->gen_req.wge_ctl, FC_RCTL_ELS4_REQ); | |
| 320 | 320 | bf_set(wqe_type, &wqe->gen_req.wge_ctl, FC_TYPE_NVME); |
| 321 | 321 | |
| 322 | 322 | /* Word 6 */ |
| @@ -620,15 +620,15 @@ lpfc_nvme_adj_fcp_sgls(struct lpfc_vport *vport, · 297 unmodified lines | ||
| 620 | 620 | * Embed the payload in the last half of the WQE |
| 621 | 621 | * WQE words 16-30 get the NVME CMD IU payload |
| 622 | 622 | * |
| 623 | − * WQE Word 16 is already setup with flags | |
| 624 | − * WQE words 17-19 get payload Words 2-4 | |
| 623 | + * WQE words 16-19 get payload Words 1-4 | |
| 625 | 624 | * WQE words 20-21 get payload Words 6-7 |
| 626 | 625 | * WQE words 22-29 get payload Words 16-23 |
| 627 | 626 | */ |
| 628 | − wptr = &wqe->words[17]; /* WQE ptr */ | |
| 627 | + wptr = &wqe->words[16]; /* WQE ptr */ | |
| 629 | 628 | dptr = (uint32_t *)nCmd->cmdaddr; /* payload ptr */ |
| 630 | − dptr += 2; /* Skip Words 0-1 in payload */ | |
| 629 | + dptr++; /* Skip Word 0 in payload */ | |
| 631 | 630 | |
| 631 | + *wptr++ = *dptr++; /* Word 1 */ | |
| 632 | 632 | *wptr++ = *dptr++; /* Word 2 */ |
| 633 | 633 | *wptr++ = *dptr++; /* Word 3 */ |
| 634 | 634 | *wptr++ = *dptr++; /* Word 4 */ |
| @@ -978,9 +978,6 @@ lpfc_nvme_prep_io_cmd(struct lpfc_vport *vport, · 343 unmodified lines | ||
| 978 | 978 | bf_set(wqe_cmd_type, &wqe->generic.wqe_com, |
| 979 | 979 | NVME_WRITE_CMD); |
| 980 | 980 | |
| 981 | − /* Word 16 */ | |
| 982 | − wqe->words[16] = LPFC_NVME_EMBED_WRITE; | |
| 983 | − | |
| 984 | 981 | phba->fc4NvmeOutputRequests++; |
| 985 | 982 | } else { |
| 986 | 983 | /* Word 7 */ |
| @@ -1002,9 +999,6 @@ lpfc_nvme_prep_io_cmd(struct lpfc_vport *vport, · 15 unmodified lines | ||
| 1002 | 999 | bf_set(wqe_cmd_type, &wqe->generic.wqe_com, |
| 1003 | 1000 | NVME_READ_CMD); |
| 1004 | 1001 | |
| 1005 | − /* Word 16 */ | |
| 1006 | − wqe->words[16] = LPFC_NVME_EMBED_READ; | |
| 1007 | − | |
| 1008 | 1002 | phba->fc4NvmeInputRequests++; |
| 1009 | 1003 | } |
| 1010 | 1004 | } else { |
| @@ -1026,9 +1020,6 @@ lpfc_nvme_prep_io_cmd(struct lpfc_vport *vport, · 15 unmodified lines | ||
| 1026 | 1020 | /* Word 11 */ |
| 1027 | 1021 | bf_set(wqe_cmd_type, &wqe->generic.wqe_com, NVME_READ_CMD); |
| 1028 | 1022 | |
| 1029 | − /* Word 16 */ | |
| 1030 | − wqe->words[16] = LPFC_NVME_EMBED_CMD; | |
| 1031 | − | |
| 1032 | 1023 | phba->fc4NvmeControlRequests++; |
| 1033 | 1024 | } |
| 1034 | 1025 | /* |
| @@ -1286,6 +1277,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, · 251 unmodified lines | ||
| 1286 | 1277 | pnvme_fcreq->private = (void *)lpfc_ncmd; |
| 1287 | 1278 | lpfc_ncmd->nvmeCmd = pnvme_fcreq; |
| 1288 | 1279 | lpfc_ncmd->nrport = rport; |
| 1280 | + lpfc_ncmd->ndlp = ndlp; | |
| 1289 | 1281 | lpfc_ncmd->start_time = jiffies; |
| 1290 | 1282 | |
| 1291 | 1283 | lpfc_nvme_prep_io_cmd(vport, lpfc_ncmd, ndlp); |
| @@ -1319,7 +1311,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, · 27 unmodified lines | ||
| 1319 | 1311 | "sid: x%x did: x%x oxid: x%x\n", |
| 1320 | 1312 | ret, vport->fc_myDID, ndlp->nlp_DID, |
| 1321 | 1313 | lpfc_ncmd->cur_iocbq.sli4_xritag); |
| 1322 | − ret = -EINVAL; | |
| 1314 | + ret = -EBUSY; | |
| 1323 | 1315 | goto out_free_nvme_buf; |
| 1324 | 1316 | } |
| 1325 | 1317 | |
| @@ -1821,10 +1813,10 @@ lpfc_post_nvme_sgl_list(struct lpfc_hba *phba, · 495 unmodified lines | ||
| 1821 | 1813 | pdma_phys_sgl1, cur_xritag); |
| 1822 | 1814 | if (status) { |
| 1823 | 1815 | /* failure, put on abort nvme list */ |
| 1824 | − lpfc_ncmd->exch_busy = 1; | |
| 1816 | + lpfc_ncmd->flags |= LPFC_SBUF_XBUSY; | |
| 1825 | 1817 | } else { |
| 1826 | 1818 | /* success, put on NVME buffer list */ |
| 1827 | − lpfc_ncmd->exch_busy = 0; | |
| 1819 | + lpfc_ncmd->flags &= ~LPFC_SBUF_XBUSY; | |
| 1828 | 1820 | lpfc_ncmd->status = IOSTAT_SUCCESS; |
| 1829 | 1821 | num_posted++; |
| 1830 | 1822 | } |
| @@ -1854,10 +1846,10 @@ lpfc_post_nvme_sgl_list(struct lpfc_hba *phba, · 23 unmodified lines | ||
| 1854 | 1846 | struct lpfc_nvme_buf, list); |
| 1855 | 1847 | if (status) { |
| 1856 | 1848 | /* failure, put on abort nvme list */ |
| 1857 | − lpfc_ncmd->exch_busy = 1; | |
| 1849 | + lpfc_ncmd->flags |= LPFC_SBUF_XBUSY; | |
| 1858 | 1850 | } else { |
| 1859 | 1851 | /* success, put on NVME buffer list */ |
| 1860 | − lpfc_ncmd->exch_busy = 0; | |
| 1852 | + lpfc_ncmd->flags &= ~LPFC_SBUF_XBUSY; | |
| 1861 | 1853 | lpfc_ncmd->status = IOSTAT_SUCCESS; |
| 1862 | 1854 | num_posted++; |
| 1863 | 1855 | } |
| @@ -2099,7 +2091,7 @@ lpfc_release_nvme_buf(struct lpfc_hba *phba, struct lpfc_nvme_buf *lpfc_ncmd) · 235 unmodified lines | ||
| 2099 | 2091 | unsigned long iflag = 0; |
| 2100 | 2092 | |
| 2101 | 2093 | lpfc_ncmd->nonsg_phys = 0; |
| 2102 | − if (lpfc_ncmd->exch_busy) { | |
| 2094 | + if (lpfc_ncmd->flags & LPFC_SBUF_XBUSY) { | |
| 2103 | 2095 | spin_lock_irqsave(&phba->sli4_hba.abts_nvme_buf_list_lock, |
| 2104 | 2096 | iflag); |
| 2105 | 2097 | lpfc_ncmd->nvmeCmd = NULL; |
| @@ -2135,11 +2127,12 @@ lpfc_release_nvme_buf(struct lpfc_hba *phba, struct lpfc_nvme_buf *lpfc_ncmd) · 29 unmodified lines | ||
| 2135 | 2127 | int |
| 2136 | 2128 | lpfc_nvme_create_localport(struct lpfc_vport *vport) |
| 2137 | 2129 | { |
| 2130 | + int ret = 0; | |
| 2138 | 2131 | struct lpfc_hba *phba = vport->phba; |
| 2139 | 2132 | struct nvme_fc_port_info nfcp_info; |
| 2140 | 2133 | struct nvme_fc_local_port *localport; |
| 2141 | 2134 | struct lpfc_nvme_lport *lport; |
| 2142 | − int len, ret = 0; | |
| 2135 | + int len; | |
| 2143 | 2136 | |
| 2144 | 2137 | /* Initialize this localport instance. The vport wwn usage ensures |
| 2145 | 2138 | * that NPIV is accounted for. |
| @@ -2156,8 +2149,12 @@ lpfc_nvme_create_localport(struct lpfc_vport *vport) · 10 unmodified lines | ||
| 2156 | 2149 | /* localport is allocated from the stack, but the registration |
| 2157 | 2150 | * call allocates heap memory as well as the private area. |
| 2158 | 2151 | */ |
| 2152 | +#ifdef CONFIG_LPFC_NVME_INITIATOR | |
| 2159 | 2153 | ret = nvme_fc_register_localport(&nfcp_info, &lpfc_nvme_template, |
| 2160 | 2154 | &vport->phba->pcidev->dev, &localport); |
| 2155 | +#else | |
| 2156 | + ret = -ENOMEM; | |
| 2157 | +#endif | |
| 2161 | 2158 | if (!ret) { |
| 2162 | 2159 | lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME | LOG_NVME_DISC, |
| 2163 | 2160 | "6005 Successfully registered local " |
| @@ -2173,10 +2170,10 @@ lpfc_nvme_create_localport(struct lpfc_vport *vport) · 9 unmodified lines | ||
| 2173 | 2170 | lport->vport = vport; |
| 2174 | 2171 | INIT_LIST_HEAD(&lport->rport_list); |
| 2175 | 2172 | vport->nvmei_support = 1; |
| 2173 | + len = lpfc_new_nvme_buf(vport, phba->sli4_hba.nvme_xri_max); | |
| 2174 | + vport->phba->total_nvme_bufs += len; | |
| 2176 | 2175 | } |
| 2177 | 2176 | |
| 2178 | − len = lpfc_new_nvme_buf(vport, phba->sli4_hba.nvme_xri_max); | |
| 2179 | − vport->phba->total_nvme_bufs += len; | |
| 2180 | 2177 | return ret; |
| 2181 | 2178 | } |
| 2182 | 2179 | |
| @@ -2193,6 +2190,7 @@ lpfc_nvme_create_localport(struct lpfc_vport *vport) · 10 unmodified lines | ||
| 2193 | 2190 | void |
| 2194 | 2191 | lpfc_nvme_destroy_localport(struct lpfc_vport *vport) |
| 2195 | 2192 | { |
| 2193 | +#ifdef CONFIG_LPFC_NVME_INITIATOR | |
| 2196 | 2194 | struct nvme_fc_local_port *localport; |
| 2197 | 2195 | struct lpfc_nvme_lport *lport; |
| 2198 | 2196 | struct lpfc_nvme_rport *rport = NULL, *rport_next = NULL; |
| @@ -2208,7 +2206,6 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) · 9 unmodified lines | ||
| 2208 | 2206 | lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME, |
| 2209 | 2207 | "6011 Destroying NVME localport %p\n", |
| 2210 | 2208 | localport); |
| 2211 | − | |
| 2212 | 2209 | list_for_each_entry_safe(rport, rport_next, &lport->rport_list, list) { |
| 2213 | 2210 | /* The last node ref has to get released now before the rport |
| 2214 | 2211 | * private memory area is released by the transport. |
| @@ -2222,6 +2219,7 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) · 7 unmodified lines | ||
| 2222 | 2219 | "6008 rport fail destroy %x\n", ret); |
| 2223 | 2220 | wait_for_completion_timeout(&rport->rport_unreg_done, 5); |
| 2224 | 2221 | } |
| 2222 | + | |
| 2225 | 2223 | /* lport's rport list is clear. Unregister |
| 2226 | 2224 | * lport and release resources. |
| 2227 | 2225 | */ |
| @@ -2245,6 +2243,7 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) · 17 unmodified lines | ||
| 2245 | 2243 | "Failed, status x%x\n", |
| 2246 | 2244 | ret); |
| 2247 | 2245 | } |
| 2246 | +#endif | |
| 2248 | 2247 | } |
| 2249 | 2248 | |
| 2250 | 2249 | void |
| @@ -2275,6 +2274,7 @@ lpfc_nvme_update_localport(struct lpfc_vport *vport) · 24 unmodified lines | ||
| 2275 | 2274 | int |
| 2276 | 2275 | lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) |
| 2277 | 2276 | { |
| 2277 | +#ifdef CONFIG_LPFC_NVME_INITIATOR | |
| 2278 | 2278 | int ret = 0; |
| 2279 | 2279 | struct nvme_fc_local_port *localport; |
| 2280 | 2280 | struct lpfc_nvme_lport *lport; |
| @@ -2348,7 +2348,6 @@ lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) · 67 unmodified lines | ||
| 2348 | 2348 | rpinfo.port_role |= FC_PORT_ROLE_NVME_INITIATOR; |
| 2349 | 2349 | rpinfo.port_name = wwn_to_u64(ndlp->nlp_portname.u.wwn); |
| 2350 | 2350 | rpinfo.node_name = wwn_to_u64(ndlp->nlp_nodename.u.wwn); |
| 2351 | − | |
| 2352 | 2351 | ret = nvme_fc_register_remoteport(localport, &rpinfo, |
| 2353 | 2352 | &remote_port); |
| 2354 | 2353 | if (!ret) { |
| @@ -2384,6 +2383,9 @@ lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) · 29 unmodified lines | ||
| 2384 | 2383 | ndlp->nlp_type, ndlp->nlp_DID, ndlp); |
| 2385 | 2384 | } |
| 2386 | 2385 | return ret; |
| 2386 | +#else | |
| 2387 | + return 0; | |
| 2388 | +#endif | |
| 2387 | 2389 | } |
| 2388 | 2390 | |
| 2389 | 2391 | /* lpfc_nvme_unregister_port - unbind the DID and port_role from this rport. |
| @@ -2401,6 +2403,7 @@ lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) · 11 unmodified lines | ||
| 2401 | 2403 | void |
| 2402 | 2404 | lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) |
| 2403 | 2405 | { |
| 2406 | +#ifdef CONFIG_LPFC_NVME_INITIATOR | |
| 2404 | 2407 | int ret; |
| 2405 | 2408 | struct nvme_fc_local_port *localport; |
| 2406 | 2409 | struct lpfc_nvme_lport *lport; |
| @@ -2458,7 +2461,61 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) · 51 unmodified lines | ||
| 2458 | 2461 | return; |
| 2459 | 2462 | |
| 2460 | 2463 | input_err: |
| 2464 | +#endif | |
| 2461 | 2465 | lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME_DISC, |
| 2462 | 2466 | "6168: State error: lport %p, rport%p FCID x%06x\n", |
| 2463 | 2467 | vport->localport, ndlp->rport, ndlp->nlp_DID); |
| 2464 | 2468 | } |
| 2469 | + | |
| 2470 | +/** | |
| 2471 | + * lpfc_sli4_nvme_xri_aborted - Fast-path process of NVME xri abort | |
| 2472 | + * @phba: pointer to lpfc hba data structure. | |
| 2473 | + * @axri: pointer to the fcp xri abort wcqe structure. | |
| 2474 | + * | |
| 2475 | + * This routine is invoked by the worker thread to process a SLI4 fast-path | |
| 2476 | + * FCP aborted xri. | |
| 2477 | + **/ | |
| 2478 | +void | |
| 2479 | +lpfc_sli4_nvme_xri_aborted(struct lpfc_hba *phba, | |
| 2480 | + struct sli4_wcqe_xri_aborted *axri) | |
| 2481 | +{ | |
| 2482 | + uint16_t xri = bf_get(lpfc_wcqe_xa_xri, axri); | |
| 2483 | + uint16_t rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); | |
| 2484 | + struct lpfc_nvme_buf *lpfc_ncmd, *next_lpfc_ncmd; | |
| 2485 | + struct lpfc_nodelist *ndlp; | |
| 2486 | + unsigned long iflag = 0; | |
| 2487 | + int rrq_empty = 0; | |
| 2488 | + | |
| 2489 | + if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME)) | |
| 2490 | + return; | |
| 2491 | + spin_lock_irqsave(&phba->hbalock, iflag); | |
| 2492 | + spin_lock(&phba->sli4_hba.abts_nvme_buf_list_lock); | |
| 2493 | + list_for_each_entry_safe(lpfc_ncmd, next_lpfc_ncmd, | |
| 2494 | + &phba->sli4_hba.lpfc_abts_nvme_buf_list, | |
| 2495 | + list) { | |
| 2496 | + if (lpfc_ncmd->cur_iocbq.sli4_xritag == xri) { | |
| 2497 | + list_del(&lpfc_ncmd->list); | |
| 2498 | + lpfc_ncmd->flags &= ~LPFC_SBUF_XBUSY; | |
| 2499 | + lpfc_ncmd->status = IOSTAT_SUCCESS; | |
| 2500 | + spin_unlock( | |
| 2501 | + &phba->sli4_hba.abts_nvme_buf_list_lock); | |
| 2502 | + | |
| 2503 | + rrq_empty = list_empty(&phba->active_rrq_list); | |
| 2504 | + spin_unlock_irqrestore(&phba->hbalock, iflag); | |
| 2505 | + ndlp = lpfc_ncmd->ndlp; | |
| 2506 | + if (ndlp) { | |
| 2507 | + lpfc_set_rrq_active( | |
| 2508 | + phba, ndlp, | |
| 2509 | + lpfc_ncmd->cur_iocbq.sli4_lxritag, | |
| 2510 | + rxid, 1); | |
| 2511 | + lpfc_sli4_abts_err_handler(phba, ndlp, axri); | |
| 2512 | + } | |
| 2513 | + lpfc_release_nvme_buf(phba, lpfc_ncmd); | |
| 2514 | + if (rrq_empty) | |
| 2515 | + lpfc_worker_wake_up(phba); | |
| 2516 | + return; | |
| 2517 | + } | |
| 2518 | + } | |
| 2519 | + spin_unlock(&phba->sli4_hba.abts_nvme_buf_list_lock); | |
| 2520 | + spin_unlock_irqrestore(&phba->hbalock, iflag); | |
| 2521 | +} | |
| @@ -57,6 +57,7 @@ struct lpfc_nvme_buf { · 56 unmodified lines | ||
| 57 | 57 | struct list_head list; |
| 58 | 58 | struct nvmefc_fcp_req *nvmeCmd; |
| 59 | 59 | struct lpfc_nvme_rport *nrport; |
| 60 | + struct lpfc_nodelist *ndlp; | |
| 60 | 61 | |
| 61 | 62 | uint32_t timeout; |
| 62 | 63 | |
| @@ -571,6 +571,7 @@ lpfc_nvmet_xmt_fcp_op(struct nvmet_fc_target_port *tgtport, · 570 unmodified lines | ||
| 571 | 571 | lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, |
| 572 | 572 | "6102 Bad state IO x%x aborted\n", |
| 573 | 573 | ctxp->oxid); |
| 574 | + rc = -ENXIO; | |
| 574 | 575 | goto aerr; |
| 575 | 576 | } |
| 576 | 577 | |
| @@ -580,6 +581,7 @@ lpfc_nvmet_xmt_fcp_op(struct nvmet_fc_target_port *tgtport, · 3 unmodified lines | ||
| 580 | 581 | lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, |
| 581 | 582 | "6152 FCP Drop IO x%x: Prep\n", |
| 582 | 583 | ctxp->oxid); |
| 584 | + rc = -ENXIO; | |
| 583 | 585 | goto aerr; |
| 584 | 586 | } |
| 585 | 587 | |
| @@ -618,8 +620,9 @@ lpfc_nvmet_xmt_fcp_op(struct nvmet_fc_target_port *tgtport, · 32 unmodified lines | ||
| 618 | 620 | ctxp->wqeq->hba_wqidx = 0; |
| 619 | 621 | nvmewqeq->context2 = NULL; |
| 620 | 622 | nvmewqeq->context3 = NULL; |
| 623 | + rc = -EBUSY; | |
| 621 | 624 | aerr: |
| 622 | − return -ENXIO; | |
| 625 | + return rc; | |
| 623 | 626 | } |
| 624 | 627 | |
| 625 | 628 | static void |
| @@ -668,9 +671,13 @@ lpfc_nvmet_create_targetport(struct lpfc_hba *phba) · 42 unmodified lines | ||
| 668 | 671 | lpfc_tgttemplate.target_features = NVMET_FCTGTFEAT_READDATA_RSP | |
| 669 | 672 | NVMET_FCTGTFEAT_NEEDS_CMD_CPUSCHED; |
| 670 | 673 | |
| 674 | +#ifdef CONFIG_LPFC_NVME_TARGET | |
| 671 | 675 | error = nvmet_fc_register_targetport(&pinfo, &lpfc_tgttemplate, |
| 672 | 676 | &phba->pcidev->dev, |
| 673 | 677 | &phba->targetport); |
| 678 | +#else | |
| 679 | + error = -ENOMEM; | |
| 680 | +#endif | |
| 674 | 681 | if (error) { |
| 675 | 682 | lpfc_printf_log(phba, KERN_ERR, LOG_NVME_DISC, |
| 676 | 683 | "6025 Cannot register NVME targetport " |
| @@ -731,9 +738,25 @@ lpfc_nvmet_update_targetport(struct lpfc_hba *phba) · 54 unmodified lines | ||
| 731 | 738 | return 0; |
| 732 | 739 | } |
| 733 | 740 | |
| 741 | +/** | |
| 742 | + * lpfc_sli4_nvmet_xri_aborted - Fast-path process of nvmet xri abort | |
| 743 | + * @phba: pointer to lpfc hba data structure. | |
| 744 | + * @axri: pointer to the nvmet xri abort wcqe structure. | |
| 745 | + * | |
| 746 | + * This routine is invoked by the worker thread to process a SLI4 fast-path | |
| 747 | + * NVMET aborted xri. | |
| 748 | + **/ | |
| 749 | +void | |
| 750 | +lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, | |
| 751 | + struct sli4_wcqe_xri_aborted *axri) | |
| 752 | +{ | |
| 753 | + /* TODO: work in progress */ | |
| 754 | +} | |
| 755 | + | |
| 734 | 756 | void |
| 735 | 757 | lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba) |
| 736 | 758 | { |
| 759 | +#ifdef CONFIG_LPFC_NVME_TARGET | |
| 737 | 760 | struct lpfc_nvmet_tgtport *tgtp; |
| 738 | 761 | |
| 739 | 762 | if (phba->nvmet_support == 0) |
| @@ -745,6 +768,7 @@ lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba) · 5 unmodified lines | ||
| 745 | 768 | wait_for_completion_timeout(&tgtp->tport_unreg_done, 5); |
| 746 | 769 | } |
| 747 | 770 | phba->targetport = NULL; |
| 771 | +#endif | |
| 748 | 772 | } |
| 749 | 773 | |
| 750 | 774 | /** |
| @@ -764,6 +788,7 @@ static void · 13 unmodified lines | ||
| 764 | 788 | lpfc_nvmet_unsol_ls_buffer(struct lpfc_hba *phba, struct lpfc_sli_ring *pring, |
| 765 | 789 | struct hbq_dmabuf *nvmebuf) |
| 766 | 790 | { |
| 791 | +#ifdef CONFIG_LPFC_NVME_TARGET | |
| 767 | 792 | struct lpfc_nvmet_tgtport *tgtp; |
| 768 | 793 | struct fc_frame_header *fc_hdr; |
| 769 | 794 | struct lpfc_nvmet_rcv_ctx *ctxp; |
| @@ -844,6 +869,7 @@ dropit: · 74 unmodified lines | ||
| 844 | 869 | |
| 845 | 870 | atomic_inc(&tgtp->xmt_ls_abort); |
| 846 | 871 | lpfc_nvmet_unsol_ls_issue_abort(phba, ctxp, sid, oxid); |
| 872 | +#endif | |
| 847 | 873 | } |
| 848 | 874 | |
| 849 | 875 | /** |
| @@ -865,6 +891,7 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, · 15 unmodified lines | ||
| 865 | 891 | struct rqb_dmabuf *nvmebuf, |
| 866 | 892 | uint64_t isr_timestamp) |
| 867 | 893 | { |
| 894 | +#ifdef CONFIG_LPFC_NVME_TARGET | |
| 868 | 895 | struct lpfc_nvmet_rcv_ctx *ctxp; |
| 869 | 896 | struct lpfc_nvmet_tgtport *tgtp; |
| 870 | 897 | struct fc_frame_header *fc_hdr; |
| @@ -955,7 +982,7 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, · 84 unmodified lines | ||
| 955 | 982 | |
| 956 | 983 | atomic_inc(&tgtp->rcv_fcp_cmd_drop); |
| 957 | 984 | lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, |
| 958 | − "6159 FCP Drop IO x%x: nvmet_fc_rcv_fcp_req x%x\n", | |
| 985 | + "6159 FCP Drop IO x%x: err x%x\n", | |
| 959 | 986 | ctxp->oxid, rc); |
| 960 | 987 | dropit: |
| 961 | 988 | lpfc_nvmeio_data(phba, "NVMET FCP DROP: xri x%x sz %d from %06x\n", |
| @@ -970,6 +997,7 @@ dropit: · 8 unmodified lines | ||
| 970 | 997 | /* We assume a rcv'ed cmd ALWAYs fits into 1 buffer */ |
| 971 | 998 | lpfc_nvmet_rq_post(phba, NULL, &nvmebuf->hbuf); |
| 972 | 999 | } |
| 1000 | +#endif | |
| 973 | 1001 | } |
| 974 | 1002 | |
| 975 | 1003 | /** |
| @@ -1114,7 +1142,7 @@ lpfc_nvmet_prep_ls_wqe(struct lpfc_hba *phba, · 138 unmodified lines | ||
| 1114 | 1142 | bf_set(wqe_dfctl, &wqe->xmit_sequence.wge_ctl, 0); |
| 1115 | 1143 | bf_set(wqe_ls, &wqe->xmit_sequence.wge_ctl, 1); |
| 1116 | 1144 | bf_set(wqe_la, &wqe->xmit_sequence.wge_ctl, 0); |
| 1117 | − bf_set(wqe_rctl, &wqe->xmit_sequence.wge_ctl, FC_RCTL_DD_SOL_CTL); | |
| 1145 | + bf_set(wqe_rctl, &wqe->xmit_sequence.wge_ctl, FC_RCTL_ELS4_REP); | |
| 1118 | 1146 | bf_set(wqe_type, &wqe->xmit_sequence.wge_ctl, FC_TYPE_NVME); |
| 1119 | 1147 | |
| 1120 | 1148 | /* Word 6 */ |
| @@ -1445,7 +1473,6 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, · 324 unmodified lines | ||
| 1445 | 1473 | |
| 1446 | 1474 | case NVMET_FCOP_RSP: |
| 1447 | 1475 | /* Words 0 - 2 */ |
| 1448 | − sgel = &rsp->sg[0]; | |
| 1449 | 1476 | physaddr = rsp->rspdma; |
| 1450 | 1477 | wqe->fcp_trsp.bde.tus.f.bdeFlags = BUFF_TYPE_BDE_64; |
| 1451 | 1478 | wqe->fcp_trsp.bde.tus.f.bdeSize = rsp->rsplen; |
| @@ -1681,8 +1708,8 @@ lpfc_nvmet_unsol_issue_abort(struct lpfc_hba *phba, · 229 unmodified lines | ||
| 1681 | 1708 | struct lpfc_nodelist *ndlp; |
| 1682 | 1709 | |
| 1683 | 1710 | lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, |
| 1684 | − "6067 %s: Entrypoint: sid %x xri %x\n", __func__, | |
| 1685 | − sid, xri); | |
| 1711 | + "6067 Abort: sid %x xri x%x/x%x\n", | |
| 1712 | + sid, xri, ctxp->wqeq->sli4_xritag); | |
| 1686 | 1713 | |
| 1687 | 1714 | tgtp = (struct lpfc_nvmet_tgtport *)phba->targetport->private; |
| 1688 | 1715 | |
| @@ -1693,7 +1720,7 @@ lpfc_nvmet_unsol_issue_abort(struct lpfc_hba *phba, · 4 unmodified lines | ||
| 1693 | 1720 | atomic_inc(&tgtp->xmt_abort_rsp_error); |
| 1694 | 1721 | lpfc_printf_log(phba, KERN_WARNING, LOG_NVME_ABTS, |
| 1695 | 1722 | "6134 Drop ABTS - wrong NDLP state x%x.\n", |
| 1696 | − ndlp->nlp_state); | |
| 1723 | + (ndlp) ? ndlp->nlp_state : NLP_STE_MAX_STATE); | |
| 1697 | 1724 | |
| 1698 | 1725 | /* No failure to an ABTS request. */ |
| 1699 | 1726 | return 0; |
| @@ -1791,7 +1818,7 @@ lpfc_nvmet_sol_fcp_issue_abort(struct lpfc_hba *phba, · 91 unmodified lines | ||
| 1791 | 1818 | atomic_inc(&tgtp->xmt_abort_rsp_error); |
| 1792 | 1819 | lpfc_printf_log(phba, KERN_WARNING, LOG_NVME_ABTS, |
| 1793 | 1820 | "6160 Drop ABTS - wrong NDLP state x%x.\n", |
| 1794 | − ndlp->nlp_state); | |
| 1821 | + (ndlp) ? ndlp->nlp_state : NLP_STE_MAX_STATE); | |
| 1795 | 1822 | |
| 1796 | 1823 | /* No failure to an ABTS request. */ |
| 1797 | 1824 | return 0; |
| @@ -5953,12 +5953,13 @@ struct scsi_host_template lpfc_template_nvme = { · 5952 unmodified lines | ||
| 5953 | 5953 | .track_queue_depth = 0, |
| 5954 | 5954 | }; |
| 5955 | 5955 | |
| 5956 | −struct scsi_host_template lpfc_template_s3 = { | |
| 5956 | +struct scsi_host_template lpfc_template_no_hr = { | |
| 5957 | 5957 | .module = THIS_MODULE, |
| 5958 | 5958 | .name = LPFC_DRIVER_NAME, |
| 5959 | 5959 | .proc_name = LPFC_DRIVER_NAME, |
| 5960 | 5960 | .info = lpfc_info, |
| 5961 | 5961 | .queuecommand = lpfc_queuecommand, |
| 5962 | + .eh_timed_out = fc_eh_timed_out, | |
| 5962 | 5963 | .eh_abort_handler = lpfc_abort_handler, |
| 5963 | 5964 | .eh_device_reset_handler = lpfc_device_reset_handler, |
| 5964 | 5965 | .eh_target_reset_handler = lpfc_target_reset_handler, |
| @@ -6015,7 +6016,6 @@ struct scsi_host_template lpfc_vport_template = { · 50 unmodified lines | ||
| 6015 | 6016 | .eh_abort_handler = lpfc_abort_handler, |
| 6016 | 6017 | .eh_device_reset_handler = lpfc_device_reset_handler, |
| 6017 | 6018 | .eh_target_reset_handler = lpfc_target_reset_handler, |
| 6018 | − .eh_bus_reset_handler = lpfc_bus_reset_handler, | |
| 6019 | 6019 | .slave_alloc = lpfc_slave_alloc, |
| 6020 | 6020 | .slave_configure = lpfc_slave_configure, |
| 6021 | 6021 | .slave_destroy = lpfc_slave_destroy, |
| @@ -1,3 +1,4 @@ | ||
| 1 | + | |
| 1 | 2 | /******************************************************************* |
| 2 | 3 | * This file is part of the Emulex Linux Device Driver for * |
| 3 | 4 | * Fibre Channel Host Bus Adapters. * |
| @@ -952,7 +953,7 @@ __lpfc_sli_get_els_sglq(struct lpfc_hba *phba, struct lpfc_iocbq *piocbq) · 948 unmodified lines | ||
| 952 | 953 | start_sglq = sglq; |
| 953 | 954 | while (!found) { |
| 954 | 955 | if (!sglq) |
| 955 | − return NULL; | |
| 956 | + break; | |
| 956 | 957 | if (ndlp && ndlp->active_rrqs_xri_bitmap && |
| 957 | 958 | test_bit(sglq->sli4_lxritag, |
| 958 | 959 | ndlp->active_rrqs_xri_bitmap)) { |
| @@ -12212,6 +12213,41 @@ void lpfc_sli4_fcp_xri_abort_event_proc(struct lpfc_hba *phba) · 11253 unmodified lines | ||
| 12212 | 12213 | } |
| 12213 | 12214 | } |
| 12214 | 12215 | |
| 12216 | +/** | |
| 12217 | + * lpfc_sli4_nvme_xri_abort_event_proc - Process nvme xri abort event | |
| 12218 | + * @phba: pointer to lpfc hba data structure. | |
| 12219 | + * | |
| 12220 | + * This routine is invoked by the worker thread to process all the pending | |
| 12221 | + * SLI4 NVME abort XRI events. | |
| 12222 | + **/ | |
| 12223 | +void lpfc_sli4_nvme_xri_abort_event_proc(struct lpfc_hba *phba) | |
| 12224 | +{ | |
| 12225 | + struct lpfc_cq_event *cq_event; | |
| 12226 | + | |
| 12227 | + /* First, declare the fcp xri abort event has been handled */ | |
| 12228 | + spin_lock_irq(&phba->hbalock); | |
| 12229 | + phba->hba_flag &= ~NVME_XRI_ABORT_EVENT; | |
| 12230 | + spin_unlock_irq(&phba->hbalock); | |
| 12231 | + /* Now, handle all the fcp xri abort events */ | |
| 12232 | + while (!list_empty(&phba->sli4_hba.sp_nvme_xri_aborted_work_queue)) { | |
| 12233 | + /* Get the first event from the head of the event queue */ | |
| 12234 | + spin_lock_irq(&phba->hbalock); | |
| 12235 | + list_remove_head(&phba->sli4_hba.sp_nvme_xri_aborted_work_queue, | |
| 12236 | + cq_event, struct lpfc_cq_event, list); | |
| 12237 | + spin_unlock_irq(&phba->hbalock); | |
| 12238 | + /* Notify aborted XRI for NVME work queue */ | |
| 12239 | + if (phba->nvmet_support) { | |
| 12240 | + lpfc_sli4_nvmet_xri_aborted(phba, | |
| 12241 | + &cq_event->cqe.wcqe_axri); | |
| 12242 | + } else { | |
| 12243 | + lpfc_sli4_nvme_xri_aborted(phba, | |
| 12244 | + &cq_event->cqe.wcqe_axri); | |
| 12245 | + } | |
| 12246 | + /* Free the event processed back to the free pool */ | |
| 12247 | + lpfc_sli4_cq_event_release(phba, cq_event); | |
| 12248 | + } | |
| 12249 | +} | |
| 12250 | + | |
| 12215 | 12251 | /** |
| 12216 | 12252 | * lpfc_sli4_els_xri_abort_event_proc - Process els xri abort event |
| 12217 | 12253 | * @phba: pointer to lpfc hba data structure. |
| @@ -12709,10 +12745,22 @@ lpfc_sli4_sp_handle_abort_xri_wcqe(struct lpfc_hba *phba, · 491 unmodified lines | ||
| 12709 | 12745 | spin_unlock_irqrestore(&phba->hbalock, iflags); |
| 12710 | 12746 | workposted = true; |
| 12711 | 12747 | break; |
| 12748 | + case LPFC_NVME: | |
| 12749 | + spin_lock_irqsave(&phba->hbalock, iflags); | |
| 12750 | + list_add_tail(&cq_event->list, | |
| 12751 | + &phba->sli4_hba.sp_nvme_xri_aborted_work_queue); | |
| 12752 | + /* Set the nvme xri abort event flag */ | |
| 12753 | + phba->hba_flag |= NVME_XRI_ABORT_EVENT; | |
| 12754 | + spin_unlock_irqrestore(&phba->hbalock, iflags); | |
| 12755 | + workposted = true; | |
| 12756 | + break; | |
| 12712 | 12757 | default: |
| 12713 | 12758 | lpfc_printf_log(phba, KERN_ERR, LOG_SLI, |
| 12714 | − "0603 Invalid work queue CQE subtype (x%x)\n", | |
| 12715 | − cq->subtype); | |
| 12759 | + "0603 Invalid CQ subtype %d: " | |
| 12760 | + "%08x %08x %08x %08x\n", | |
| 12761 | + cq->subtype, wcqe->word0, wcqe->parameter, | |
| 12762 | + wcqe->word2, wcqe->word3); | |
| 12763 | + lpfc_sli4_cq_event_release(phba, cq_event); | |
| 12716 | 12764 | workposted = false; |
| 12717 | 12765 | break; |
| 12718 | 12766 | } |
| @@ -13827,6 +13875,8 @@ lpfc_dual_chute_pci_bar_map(struct lpfc_hba *phba, uint16_t pci_barset) · 1108 unmodified lines | ||
| 13827 | 13875 | * @startq: The starting FCP EQ to modify |
| 13828 | 13876 | * |
| 13829 | 13877 | * This function sends an MODIFY_EQ_DELAY mailbox command to the HBA. |
| 13878 | + * The command allows up to LPFC_MAX_EQ_DELAY_EQID_CNT EQ ID's to be | |
| 13879 | + * updated in one mailbox command. | |
| 13830 | 13880 | * |
| 13831 | 13881 | * The @phba struct is used to send mailbox command to HBA. The @startq |
| 13832 | 13882 | * is used to get the starting FCP EQ to change. |
| @@ -13879,7 +13929,7 @@ lpfc_modify_hba_eq_delay(struct lpfc_hba *phba, uint32_t startq) · 46 unmodified lines | ||
| 13879 | 13929 | eq_delay->u.request.eq[cnt].phase = 0; |
| 13880 | 13930 | eq_delay->u.request.eq[cnt].delay_multi = dmult; |
| 13881 | 13931 | cnt++; |
| 13882 | − if (cnt >= LPFC_MAX_EQ_DELAY) | |
| 13932 | + if (cnt >= LPFC_MAX_EQ_DELAY_EQID_CNT) | |
| 13883 | 13933 | break; |
| 13884 | 13934 | } |
| 13885 | 13935 | eq_delay->u.request.num_eq = cnt; |
| @@ -15185,17 +15235,17 @@ lpfc_mrq_create(struct lpfc_hba *phba, struct lpfc_queue **hrqp, · 1299 unmodified lines | ||
| 15185 | 15235 | drq = drqp[idx]; |
| 15186 | 15236 | cq = cqp[idx]; |
| 15187 | 15237 | |
| 15188 | − if (hrq->entry_count != drq->entry_count) { | |
| 15189 | − status = -EINVAL; | |
| 15190 | − goto out; | |
| 15191 | − } | |
| 15192 | − | |
| 15193 | 15238 | /* sanity check on queue memory */ |
| 15194 | 15239 | if (!hrq || !drq || !cq) { |
| 15195 | 15240 | status = -ENODEV; |
| 15196 | 15241 | goto out; |
| 15197 | 15242 | } |
| 15198 | 15243 | |
| 15244 | + if (hrq->entry_count != drq->entry_count) { | |
| 15245 | + status = -EINVAL; | |
| 15246 | + goto out; | |
| 15247 | + } | |
| 15248 | + | |
| 15199 | 15249 | if (idx == 0) { |
| 15200 | 15250 | bf_set(lpfc_mbx_rq_create_num_pages, |
| 15201 | 15251 | &rq_create->u.request, |
| @@ -642,6 +642,7 @@ struct lpfc_sli4_hba { · 641 unmodified lines | ||
| 642 | 642 | struct list_head sp_asynce_work_queue; |
| 643 | 643 | struct list_head sp_fcp_xri_aborted_work_queue; |
| 644 | 644 | struct list_head sp_els_xri_aborted_work_queue; |
| 645 | + struct list_head sp_nvme_xri_aborted_work_queue; | |
| 645 | 646 | struct list_head sp_unsol_work_queue; |
| 646 | 647 | struct lpfc_sli4_link link_state; |
| 647 | 648 | struct lpfc_sli4_lnk_info lnk_info; |
| @@ -794,9 +795,14 @@ void lpfc_sli4_fcf_redisc_event_proc(struct lpfc_hba *); · 146 unmodified lines | ||
| 794 | 795 | int lpfc_sli4_resume_rpi(struct lpfc_nodelist *, |
| 795 | 796 | void (*)(struct lpfc_hba *, LPFC_MBOXQ_t *), void *); |
| 796 | 797 | void lpfc_sli4_fcp_xri_abort_event_proc(struct lpfc_hba *); |
| 798 | +void lpfc_sli4_nvme_xri_abort_event_proc(struct lpfc_hba *phba); | |
| 797 | 799 | void lpfc_sli4_els_xri_abort_event_proc(struct lpfc_hba *); |
| 798 | 800 | void lpfc_sli4_fcp_xri_aborted(struct lpfc_hba *, |
| 799 | 801 | struct sli4_wcqe_xri_aborted *); |
| 802 | +void lpfc_sli4_nvme_xri_aborted(struct lpfc_hba *phba, | |
| 803 | + struct sli4_wcqe_xri_aborted *axri); | |
| 804 | +void lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, | |
| 805 | + struct sli4_wcqe_xri_aborted *axri); | |
| 800 | 806 | void lpfc_sli4_els_xri_aborted(struct lpfc_hba *, |
| 801 | 807 | struct sli4_wcqe_xri_aborted *); |
| 802 | 808 | void lpfc_sli4_vport_delete_els_xri_aborted(struct lpfc_vport *); |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | * included with this package. * |
| 21 | 21 | *******************************************************************/ |
| 22 | 22 | |
| 23 | −#define LPFC_DRIVER_VERSION "11.2.0.7" | |
| 23 | +#define LPFC_DRIVER_VERSION "11.2.0.10" | |
| 24 | 24 | #define LPFC_DRIVER_NAME "lpfc" |
| 25 | 25 | |
| 26 | 26 | /* Used for SLI 2/3 */ |
| @@ -1442,9 +1442,6 @@ void mpt3sas_transport_update_links(struct MPT3SAS_ADAPTER *ioc, · 1441 unmodified lines | ||
| 1442 | 1442 | u64 sas_address, u16 handle, u8 phy_number, u8 link_rate); |
| 1443 | 1443 | extern struct sas_function_template mpt3sas_transport_functions; |
| 1444 | 1444 | extern struct scsi_transport_template *mpt3sas_transport_template; |
| 1445 | −extern int scsi_internal_device_block(struct scsi_device *sdev); | |
| 1446 | −extern int scsi_internal_device_unblock(struct scsi_device *sdev, | |
| 1447 | − enum scsi_device_state new_state); | |
| 1448 | 1445 | /* trigger data externs */ |
| 1449 | 1446 | void mpt3sas_send_trigger_data_event(struct MPT3SAS_ADAPTER *ioc, |
| 1450 | 1447 | struct SL_WH_TRIGGERS_EVENT_DATA_T *event_data); |
| @@ -2859,7 +2859,7 @@ _scsih_internal_device_block(struct scsi_device *sdev, · 2858 unmodified lines | ||
| 2859 | 2859 | sas_device_priv_data->sas_target->handle); |
| 2860 | 2860 | sas_device_priv_data->block = 1; |
| 2861 | 2861 | |
| 2862 | − r = scsi_internal_device_block(sdev); | |
| 2862 | + r = scsi_internal_device_block(sdev, false); | |
| 2863 | 2863 | if (r == -EINVAL) |
| 2864 | 2864 | sdev_printk(KERN_WARNING, sdev, |
| 2865 | 2865 | "device_block failed with return(%d) for handle(0x%04x)\n", |
| @@ -2895,7 +2895,7 @@ _scsih_internal_device_unblock(struct scsi_device *sdev, · 29 unmodified lines | ||
| 2895 | 2895 | "performing a block followed by an unblock\n", |
| 2896 | 2896 | r, sas_device_priv_data->sas_target->handle); |
| 2897 | 2897 | sas_device_priv_data->block = 1; |
| 2898 | − r = scsi_internal_device_block(sdev); | |
| 2898 | + r = scsi_internal_device_block(sdev, false); | |
| 2899 | 2899 | if (r) |
| 2900 | 2900 | sdev_printk(KERN_WARNING, sdev, "retried device_block " |
| 2901 | 2901 | "failed with return(%d) for handle(0x%04x)\n", |
| @@ -4677,7 +4677,6 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) · 1775 unmodified lines | ||
| 4677 | 4677 | struct MPT3SAS_DEVICE *sas_device_priv_data; |
| 4678 | 4678 | u32 response_code = 0; |
| 4679 | 4679 | unsigned long flags; |
| 4680 | − unsigned int sector_sz; | |
| 4681 | 4680 | |
| 4682 | 4681 | mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply); |
| 4683 | 4682 | |
| @@ -4742,20 +4741,6 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) · 58 unmodified lines | ||
| 4742 | 4741 | } |
| 4743 | 4742 | |
| 4744 | 4743 | xfer_cnt = le32_to_cpu(mpi_reply->TransferCount); |
| 4745 | − | |
| 4746 | − /* In case of bogus fw or device, we could end up having | |
| 4747 | − * unaligned partial completion. We can force alignment here, | |
| 4748 | − * then scsi-ml does not need to handle this misbehavior. | |
| 4749 | − */ | |
| 4750 | − sector_sz = scmd->device->sector_size; | |
| 4751 | − if (unlikely(!blk_rq_is_passthrough(scmd->request) && sector_sz && | |
| 4752 | − xfer_cnt % sector_sz)) { | |
| 4753 | − sdev_printk(KERN_INFO, scmd->device, | |
| 4754 | − "unaligned partial completion avoided (xfer_cnt=%u, sector_sz=%u)\n", | |
| 4755 | − xfer_cnt, sector_sz); | |
| 4756 | − xfer_cnt = round_down(xfer_cnt, sector_sz); | |
| 4757 | − } | |
| 4758 | − | |
| 4759 | 4744 | scsi_set_resid(scmd, scsi_bufflen(scmd) - xfer_cnt); |
| 4760 | 4745 | if (ioc_status & MPI2_IOCSTATUS_FLAG_LOG_INFO_AVAILABLE) |
| 4761 | 4746 | log_info = le32_to_cpu(mpi_reply->IOCLogInfo); |
| @@ -81,14 +81,17 @@ struct qedf_dbg_ctx { · 80 unmodified lines | ||
| 81 | 81 | #define QEDF_INFO(pdev, level, fmt, ...) \ |
| 82 | 82 | qedf_dbg_info(pdev, __func__, __LINE__, level, fmt, \ |
| 83 | 83 | ## __VA_ARGS__) |
| 84 | − | |
| 85 | −extern void qedf_dbg_err(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 84 | +__printf(4, 5) | |
| 85 | +void qedf_dbg_err(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 86 | 86 | const char *fmt, ...); |
| 87 | −extern void qedf_dbg_warn(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 87 | +__printf(4, 5) | |
| 88 | +void qedf_dbg_warn(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 88 | 89 | const char *, ...); |
| 89 | −extern void qedf_dbg_notice(struct qedf_dbg_ctx *qedf, const char *func, | |
| 90 | +__printf(4, 5) | |
| 91 | +void qedf_dbg_notice(struct qedf_dbg_ctx *qedf, const char *func, | |
| 90 | 92 | u32 line, const char *, ...); |
| 91 | −extern void qedf_dbg_info(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 93 | +__printf(5, 6) | |
| 94 | +void qedf_dbg_info(struct qedf_dbg_ctx *qedf, const char *func, u32 line, | |
| 92 | 95 | u32 info, const char *fmt, ...); |
| 93 | 96 | |
| 94 | 97 | /* GRC Dump related defines */ |
| @@ -203,7 +203,7 @@ void qedf_fip_recv(struct qedf_ctx *qedf, struct sk_buff *skb) · 202 unmodified lines | ||
| 203 | 203 | case FIP_DT_MAC: |
| 204 | 204 | mp = (struct fip_mac_desc *)desc; |
| 205 | 205 | QEDF_INFO(&(qedf->dbg_ctx), QEDF_LOG_LL2, |
| 206 | − "fd_mac=%pM.\n", __func__, mp->fd_mac); | |
| 206 | + "fd_mac=%pM\n", mp->fd_mac); | |
| 207 | 207 | ether_addr_copy(cvl_mac, mp->fd_mac); |
| 208 | 208 | break; |
| 209 | 209 | case FIP_DT_NAME: |
| @@ -1342,7 +1342,7 @@ void qedf_scsi_completion(struct qedf_ctx *qedf, struct fcoe_cqe *cqe, · 1341 unmodified lines | ||
| 1342 | 1342 | } else { |
| 1343 | 1343 | refcount = kref_read(&io_req->refcount); |
| 1344 | 1344 | QEDF_INFO(&(qedf->dbg_ctx), QEDF_LOG_IO, |
| 1345 | − "%d:0:%d:%d xid=0x%0x op=0x%02x " | |
| 1345 | + "%d:0:%d:%lld xid=0x%0x op=0x%02x " | |
| 1346 | 1346 | "lba=%02x%02x%02x%02x cdb_status=%d " |
| 1347 | 1347 | "fcp_resid=0x%x refcount=%d.\n", |
| 1348 | 1348 | qedf->lport->host->host_no, sc_cmd->device->id, |
| @@ -1426,7 +1426,7 @@ void qedf_scsi_done(struct qedf_ctx *qedf, struct qedf_ioreq *io_req, · 77 unmodified lines | ||
| 1426 | 1426 | |
| 1427 | 1427 | sc_cmd->result = result << 16; |
| 1428 | 1428 | refcount = kref_read(&io_req->refcount); |
| 1429 | − QEDF_INFO(&(qedf->dbg_ctx), QEDF_LOG_IO, "%d:0:%d:%d: Completing " | |
| 1429 | + QEDF_INFO(&(qedf->dbg_ctx), QEDF_LOG_IO, "%d:0:%d:%lld: Completing " | |
| 1430 | 1430 | "sc_cmd=%p result=0x%08x op=0x%02x lba=0x%02x%02x%02x%02x, " |
| 1431 | 1431 | "allowed=%d retries=%d refcount=%d.\n", |
| 1432 | 1432 | qedf->lport->host->host_no, sc_cmd->device->id, |
| @@ -2456,8 +2456,8 @@ static int qedf_alloc_bdq(struct qedf_ctx *qedf) · 2455 unmodified lines | ||
| 2456 | 2456 | } |
| 2457 | 2457 | |
| 2458 | 2458 | QEDF_INFO(&(qedf->dbg_ctx), QEDF_LOG_DISC, |
| 2459 | − "BDQ PBL addr=0x%p dma=0x%llx.\n", qedf->bdq_pbl, | |
| 2460 | − qedf->bdq_pbl_dma); | |
| 2459 | + "BDQ PBL addr=0x%p dma=%pad\n", | |
| 2460 | + qedf->bdq_pbl, &qedf->bdq_pbl_dma); | |
| 2461 | 2461 | |
| 2462 | 2462 | /* |
| 2463 | 2463 | * Populate BDQ PBL with physical and virtual address of individual |
| @@ -14,7 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/debugfs.h> |
| 15 | 15 | #include <linux/module.h> |
| 16 | 16 | |
| 17 | −int do_not_recover; | |
| 17 | +int qedi_do_not_recover; | |
| 18 | 18 | static struct dentry *qedi_dbg_root; |
| 19 | 19 | |
| 20 | 20 | void |
| @@ -74,22 +74,22 @@ qedi_dbg_exit(void) · 53 unmodified lines | ||
| 74 | 74 | static ssize_t |
| 75 | 75 | qedi_dbg_do_not_recover_enable(struct qedi_dbg_ctx *qedi_dbg) |
| 76 | 76 | { |
| 77 | − if (!do_not_recover) | |
| 78 | − do_not_recover = 1; | |
| 77 | + if (!qedi_do_not_recover) | |
| 78 | + qedi_do_not_recover = 1; | |
| 79 | 79 | |
| 80 | 80 | QEDI_INFO(qedi_dbg, QEDI_LOG_DEBUGFS, "do_not_recover=%d\n", |
| 81 | − do_not_recover); | |
| 81 | + qedi_do_not_recover); | |
| 82 | 82 | return 0; |
| 83 | 83 | } |
| 84 | 84 | |
| 85 | 85 | static ssize_t |
| 86 | 86 | qedi_dbg_do_not_recover_disable(struct qedi_dbg_ctx *qedi_dbg) |
| 87 | 87 | { |
| 88 | − if (do_not_recover) | |
| 89 | − do_not_recover = 0; | |
| 88 | + if (qedi_do_not_recover) | |
| 89 | + qedi_do_not_recover = 0; | |
| 90 | 90 | |
| 91 | 91 | QEDI_INFO(qedi_dbg, QEDI_LOG_DEBUGFS, "do_not_recover=%d\n", |
| 92 | − do_not_recover); | |
| 92 | + qedi_do_not_recover); | |
| 93 | 93 | return 0; |
| 94 | 94 | } |
| 95 | 95 | |
| @@ -141,7 +141,7 @@ qedi_dbg_do_not_recover_cmd_read(struct file *filp, char __user *buffer, · 45 unmodified lines | ||
| 141 | 141 | if (*ppos) |
| 142 | 142 | return 0; |
| 143 | 143 | |
| 144 | − cnt = sprintf(buffer, "do_not_recover=%d\n", do_not_recover); | |
| 144 | + cnt = sprintf(buffer, "do_not_recover=%d\n", qedi_do_not_recover); | |
| 145 | 145 | cnt = min_t(int, count, cnt - *ppos); |
| 146 | 146 | *ppos += cnt; |
| 147 | 147 | return cnt; |
| @@ -1461,9 +1461,9 @@ static void qedi_tmf_work(struct work_struct *work) · 1460 unmodified lines | ||
| 1461 | 1461 | get_itt(tmf_hdr->rtt), get_itt(ctask->itt), cmd->task_id, |
| 1462 | 1462 | qedi_conn->iscsi_conn_id); |
| 1463 | 1463 | |
| 1464 | − if (do_not_recover) { | |
| 1464 | + if (qedi_do_not_recover) { | |
| 1465 | 1465 | QEDI_ERR(&qedi->dbg_ctx, "DONT SEND CLEANUP/ABORT %d\n", |
| 1466 | − do_not_recover); | |
| 1466 | + qedi_do_not_recover); | |
| 1467 | 1467 | goto abort_ret; |
| 1468 | 1468 | } |
| 1469 | 1469 | |
| @@ -12,8 +12,14 @@ · 11 unmodified lines | ||
| 12 | 12 | |
| 13 | 13 | #include "qedi_iscsi.h" |
| 14 | 14 | |
| 15 | +#ifdef CONFIG_DEBUG_FS | |
| 16 | +extern int qedi_do_not_recover; | |
| 17 | +#else | |
| 18 | +#define qedi_do_not_recover (0) | |
| 19 | +#endif | |
| 20 | + | |
| 15 | 21 | extern uint qedi_io_tracing; |
| 16 | −extern int do_not_recover; | |
| 22 | + | |
| 17 | 23 | extern struct scsi_host_template qedi_host_template; |
| 18 | 24 | extern struct iscsi_transport qedi_iscsi_transport; |
| 19 | 25 | extern const struct qed_iscsi_ops *qedi_ops; |
| @@ -833,7 +833,7 @@ qedi_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr, · 832 unmodified lines | ||
| 833 | 833 | return ERR_PTR(ret); |
| 834 | 834 | } |
| 835 | 835 | |
| 836 | − if (do_not_recover) { | |
| 836 | + if (qedi_do_not_recover) { | |
| 837 | 837 | ret = -ENOMEM; |
| 838 | 838 | return ERR_PTR(ret); |
| 839 | 839 | } |
| @@ -957,7 +957,7 @@ static int qedi_ep_poll(struct iscsi_endpoint *ep, int timeout_ms) · 117 unmodified lines | ||
| 957 | 957 | struct qedi_endpoint *qedi_ep; |
| 958 | 958 | int ret = 0; |
| 959 | 959 | |
| 960 | − if (do_not_recover) | |
| 960 | + if (qedi_do_not_recover) | |
| 961 | 961 | return 1; |
| 962 | 962 | |
| 963 | 963 | qedi_ep = ep->dd_data; |
| @@ -1025,7 +1025,7 @@ static void qedi_ep_disconnect(struct iscsi_endpoint *ep) · 61 unmodified lines | ||
| 1025 | 1025 | } |
| 1026 | 1026 | |
| 1027 | 1027 | if (test_bit(QEDI_IN_RECOVERY, &qedi->flags)) { |
| 1028 | − if (do_not_recover) { | |
| 1028 | + if (qedi_do_not_recover) { | |
| 1029 | 1029 | QEDI_INFO(&qedi->dbg_ctx, QEDI_LOG_INFO, |
| 1030 | 1030 | "Do not recover cid=0x%x\n", |
| 1031 | 1031 | qedi_ep->iscsi_cid); |
| @@ -1039,7 +1039,7 @@ static void qedi_ep_disconnect(struct iscsi_endpoint *ep) · 7 unmodified lines | ||
| 1039 | 1039 | } |
| 1040 | 1040 | } |
| 1041 | 1041 | |
| 1042 | − if (do_not_recover) | |
| 1042 | + if (qedi_do_not_recover) | |
| 1043 | 1043 | goto ep_exit_recover; |
| 1044 | 1044 | |
| 1045 | 1045 | switch (qedi_ep->state) { |
| @@ -1805,7 +1805,7 @@ static int __qedi_probe(struct pci_dev *pdev, int mode) · 1804 unmodified lines | ||
| 1805 | 1805 | */ |
| 1806 | 1806 | qedi_ops->common->update_pf_params(qedi->cdev, &qedi->pf_params); |
| 1807 | 1807 | |
| 1808 | − qedi_setup_int(qedi); | |
| 1808 | + rc = qedi_setup_int(qedi); | |
| 1809 | 1809 | if (rc) |
| 1810 | 1810 | goto stop_iscsi_func; |
| 1811 | 1811 | |
| @@ -2707,13 +2707,9 @@ ql_dump_buffer(uint32_t level, scsi_qla_host_t *vha, int32_t id, · 2706 unmodified lines | ||
| 2707 | 2707 | "%-+5d 0 1 2 3 4 5 6 7 8 9 A B C D E F\n", size); |
| 2708 | 2708 | ql_dbg(level, vha, id, |
| 2709 | 2709 | "----- -----------------------------------------------\n"); |
| 2710 | − for (cnt = 0; cnt < size; cnt++, buf++) { | |
| 2711 | − if (cnt % 16 == 0) | |
| 2712 | − ql_dbg(level, vha, id, "%04x:", cnt & ~0xFU); | |
| 2713 | − printk(" %02x", *buf); | |
| 2714 | − if (cnt % 16 == 15) | |
| 2715 | − printk("\n"); | |
| 2710 | + for (cnt = 0; cnt < size; cnt += 16) { | |
| 2711 | + ql_dbg(level, vha, id, "%04x: ", cnt); | |
| 2712 | + print_hex_dump(KERN_CONT, "", DUMP_PREFIX_NONE, 16, 1, | |
| 2713 | + buf + cnt, min(16U, size - cnt), false); | |
| 2716 | 2714 | } |
| 2717 | − if (cnt % 16 != 0) | |
| 2718 | − printk("\n"); | |
| 2719 | 2715 | } |
| @@ -2932,6 +2932,8 @@ EXPORT_SYMBOL(scsi_target_resume); · 2931 unmodified lines | ||
| 2932 | 2932 | /** |
| 2933 | 2933 | * scsi_internal_device_block - internal function to put a device temporarily into the SDEV_BLOCK state |
| 2934 | 2934 | * @sdev: device to block |
| 2935 | + * @wait: Whether or not to wait until ongoing .queuecommand() / | |
| 2936 | + * .queue_rq() calls have finished. | |
| 2935 | 2937 | * |
| 2936 | 2938 | * Block request made by scsi lld's to temporarily stop all |
| 2937 | 2939 | * scsi commands on the specified device. May sleep. |
| @@ -2949,7 +2951,7 @@ EXPORT_SYMBOL(scsi_target_resume); · 11 unmodified lines | ||
| 2949 | 2951 | * remove the rport mutex lock and unlock calls from srp_queuecommand(). |
| 2950 | 2952 | */ |
| 2951 | 2953 | int |
| 2952 | −scsi_internal_device_block(struct scsi_device *sdev) | |
| 2954 | +scsi_internal_device_block(struct scsi_device *sdev, bool wait) | |
| 2953 | 2955 | { |
| 2954 | 2956 | struct request_queue *q = sdev->request_queue; |
| 2955 | 2957 | unsigned long flags; |
| @@ -2969,12 +2971,16 @@ scsi_internal_device_block(struct scsi_device *sdev) · 13 unmodified lines | ||
| 2969 | 2971 | * request queue. |
| 2970 | 2972 | */ |
| 2971 | 2973 | if (q->mq_ops) { |
| 2972 | − blk_mq_quiesce_queue(q); | |
| 2974 | + if (wait) | |
| 2975 | + blk_mq_quiesce_queue(q); | |
| 2976 | + else | |
| 2977 | + blk_mq_stop_hw_queues(q); | |
| 2973 | 2978 | } else { |
| 2974 | 2979 | spin_lock_irqsave(q->queue_lock, flags); |
| 2975 | 2980 | blk_stop_queue(q); |
| 2976 | 2981 | spin_unlock_irqrestore(q->queue_lock, flags); |
| 2977 | − scsi_wait_for_queuecommand(sdev); | |
| 2982 | + if (wait) | |
| 2983 | + scsi_wait_for_queuecommand(sdev); | |
| 2978 | 2984 | } |
| 2979 | 2985 | |
| 2980 | 2986 | return 0; |
| @@ -3036,7 +3042,7 @@ EXPORT_SYMBOL_GPL(scsi_internal_device_unblock); · 55 unmodified lines | ||
| 3036 | 3042 | static void |
| 3037 | 3043 | device_block(struct scsi_device *sdev, void *data) |
| 3038 | 3044 | { |
| 3039 | − scsi_internal_device_block(sdev); | |
| 3045 | + scsi_internal_device_block(sdev, true); | |
| 3040 | 3046 | } |
| 3041 | 3047 | |
| 3042 | 3048 | static int |
| @@ -188,8 +188,5 @@ static inline void scsi_dh_remove_device(struct scsi_device *sdev) { } · 187 unmodified lines | ||
| 188 | 188 | */ |
| 189 | 189 | |
| 190 | 190 | #define SCSI_DEVICE_BLOCK_MAX_TIMEOUT 600 /* units in seconds */ |
| 191 | −extern int scsi_internal_device_block(struct scsi_device *sdev); | |
| 192 | −extern int scsi_internal_device_unblock(struct scsi_device *sdev, | |
| 193 | − enum scsi_device_state new_state); | |
| 194 | 191 | |
| 195 | 192 | #endif /* _SCSI_PRIV_H */ |
| @@ -1783,6 +1783,8 @@ static int sd_done(struct scsi_cmnd *SCpnt) · 1782 unmodified lines | ||
| 1783 | 1783 | { |
| 1784 | 1784 | int result = SCpnt->result; |
| 1785 | 1785 | unsigned int good_bytes = result ? 0 : scsi_bufflen(SCpnt); |
| 1786 | + unsigned int sector_size = SCpnt->device->sector_size; | |
| 1787 | + unsigned int resid; | |
| 1786 | 1788 | struct scsi_sense_hdr sshdr; |
| 1787 | 1789 | struct scsi_disk *sdkp = scsi_disk(SCpnt->request->rq_disk); |
| 1788 | 1790 | struct request *req = SCpnt->request; |
| @@ -1813,6 +1815,21 @@ static int sd_done(struct scsi_cmnd *SCpnt) · 24 unmodified lines | ||
| 1813 | 1815 | scsi_set_resid(SCpnt, blk_rq_bytes(req)); |
| 1814 | 1816 | } |
| 1815 | 1817 | break; |
| 1818 | + default: | |
| 1819 | + /* | |
| 1820 | + * In case of bogus fw or device, we could end up having | |
| 1821 | + * an unaligned partial completion. Check this here and force | |
| 1822 | + * alignment. | |
| 1823 | + */ | |
| 1824 | + resid = scsi_get_resid(SCpnt); | |
| 1825 | + if (resid & (sector_size - 1)) { | |
| 1826 | + sd_printk(KERN_INFO, sdkp, | |
| 1827 | + "Unaligned partial completion (resid=%u, sector_sz=%u)\n", | |
| 1828 | + resid, sector_size); | |
| 1829 | + resid = min(scsi_bufflen(SCpnt), | |
| 1830 | + round_up(resid, sector_size)); | |
| 1831 | + scsi_set_resid(SCpnt, resid); | |
| 1832 | + } | |
| 1816 | 1833 | } |
| 1817 | 1834 | |
| 1818 | 1835 | if (result) { |
| @@ -3075,23 +3092,6 @@ static void sd_probe_async(void *data, async_cookie_t cookie) · 1256 unmodified lines | ||
| 3075 | 3092 | put_device(&sdkp->dev); |
| 3076 | 3093 | } |
| 3077 | 3094 | |
| 3078 | −struct sd_devt { | |
| 3079 | − int idx; | |
| 3080 | − struct disk_devt disk_devt; | |
| 3081 | −}; | |
| 3082 | − | |
| 3083 | −static void sd_devt_release(struct disk_devt *disk_devt) | |
| 3084 | −{ | |
| 3085 | − struct sd_devt *sd_devt = container_of(disk_devt, struct sd_devt, | |
| 3086 | − disk_devt); | |
| 3087 | − | |
| 3088 | − spin_lock(&sd_index_lock); | |
| 3089 | − ida_remove(&sd_index_ida, sd_devt->idx); | |
| 3090 | − spin_unlock(&sd_index_lock); | |
| 3091 | − | |
| 3092 | − kfree(sd_devt); | |
| 3093 | −} | |
| 3094 | − | |
| 3095 | 3095 | /** |
| 3096 | 3096 | * sd_probe - called during driver initialization and whenever a |
| 3097 | 3097 | * new scsi device is attached to the system. It is called once |
| @@ -3113,7 +3113,6 @@ static void sd_devt_release(struct disk_devt *disk_devt) · 15 unmodified lines | ||
| 3113 | 3113 | static int sd_probe(struct device *dev) |
| 3114 | 3114 | { |
| 3115 | 3115 | struct scsi_device *sdp = to_scsi_device(dev); |
| 3116 | − struct sd_devt *sd_devt; | |
| 3117 | 3116 | struct scsi_disk *sdkp; |
| 3118 | 3117 | struct gendisk *gd; |
| 3119 | 3118 | int index; |
| @@ -3139,13 +3138,9 @@ static int sd_probe(struct device *dev) · 19 unmodified lines | ||
| 3139 | 3138 | if (!sdkp) |
| 3140 | 3139 | goto out; |
| 3141 | 3140 | |
| 3142 | − sd_devt = kzalloc(sizeof(*sd_devt), GFP_KERNEL); | |
| 3143 | − if (!sd_devt) | |
| 3144 | − goto out_free; | |
| 3145 | − | |
| 3146 | 3141 | gd = alloc_disk(SD_MINORS); |
| 3147 | 3142 | if (!gd) |
| 3148 | − goto out_free_devt; | |
| 3143 | + goto out_free; | |
| 3149 | 3144 | |
| 3150 | 3145 | do { |
| 3151 | 3146 | if (!ida_pre_get(&sd_index_ida, GFP_KERNEL)) |
| @@ -3161,11 +3156,6 @@ static int sd_probe(struct device *dev) · 9 unmodified lines | ||
| 3161 | 3156 | goto out_put; |
| 3162 | 3157 | } |
| 3163 | 3158 | |
| 3164 | − atomic_set(&sd_devt->disk_devt.count, 1); | |
| 3165 | − sd_devt->disk_devt.release = sd_devt_release; | |
| 3166 | − sd_devt->idx = index; | |
| 3167 | − gd->disk_devt = &sd_devt->disk_devt; | |
| 3168 | − | |
| 3169 | 3159 | error = sd_format_disk_name("sd", index, gd->disk_name, DISK_NAME_LEN); |
| 3170 | 3160 | if (error) { |
| 3171 | 3161 | sdev_printk(KERN_WARNING, sdp, "SCSI disk (sd) name length exceeded.\n"); |
| @@ -3205,12 +3195,11 @@ static int sd_probe(struct device *dev) · 33 unmodified lines | ||
| 3205 | 3195 | return 0; |
| 3206 | 3196 | |
| 3207 | 3197 | out_free_index: |
| 3208 | − put_disk_devt(&sd_devt->disk_devt); | |
| 3209 | − sd_devt = NULL; | |
| 3198 | + spin_lock(&sd_index_lock); | |
| 3199 | + ida_remove(&sd_index_ida, index); | |
| 3200 | + spin_unlock(&sd_index_lock); | |
| 3210 | 3201 | out_put: |
| 3211 | 3202 | put_disk(gd); |
| 3212 | − out_free_devt: | |
| 3213 | − kfree(sd_devt); | |
| 3214 | 3203 | out_free: |
| 3215 | 3204 | kfree(sdkp); |
| 3216 | 3205 | out: |
| @@ -3271,7 +3260,10 @@ static void scsi_disk_release(struct device *dev) · 54 unmodified lines | ||
| 3271 | 3260 | struct scsi_disk *sdkp = to_scsi_disk(dev); |
| 3272 | 3261 | struct gendisk *disk = sdkp->disk; |
| 3273 | 3262 | |
| 3274 | − put_disk_devt(disk->disk_devt); | |
| 3263 | + spin_lock(&sd_index_lock); | |
| 3264 | + ida_remove(&sd_index_ida, sdkp->index); | |
| 3265 | + spin_unlock(&sd_index_lock); | |
| 3266 | + | |
| 3275 | 3267 | disk->private_data = NULL; |
| 3276 | 3268 | put_disk(disk); |
| 3277 | 3269 | put_device(&sdkp->device->sdev_gendev); |
| @@ -400,8 +400,6 @@ MODULE_PARM_DESC(storvsc_vcpus_per_sub_channel, "Ratio of VCPUs to subchannels") · 399 unmodified lines | ||
| 400 | 400 | */ |
| 401 | 401 | static int storvsc_timeout = 180; |
| 402 | 402 | |
| 403 | −static int msft_blist_flags = BLIST_TRY_VPD_PAGES; | |
| 404 | − | |
| 405 | 403 | #if IS_ENABLED(CONFIG_SCSI_FC_ATTRS) |
| 406 | 404 | static struct scsi_transport_template *fc_transport_template; |
| 407 | 405 | #endif |
| @@ -1383,6 +1381,22 @@ static int storvsc_do_io(struct hv_device *device, · 975 unmodified lines | ||
| 1383 | 1381 | return ret; |
| 1384 | 1382 | } |
| 1385 | 1383 | |
| 1384 | +static int storvsc_device_alloc(struct scsi_device *sdevice) | |
| 1385 | +{ | |
| 1386 | + /* | |
| 1387 | + * Set blist flag to permit the reading of the VPD pages even when | |
| 1388 | + * the target may claim SPC-2 compliance. MSFT targets currently | |
| 1389 | + * claim SPC-2 compliance while they implement post SPC-2 features. | |
| 1390 | + * With this flag we can correctly handle WRITE_SAME_16 issues. | |
| 1391 | + * | |
| 1392 | + * Hypervisor reports SCSI_UNKNOWN type for DVD ROM device but | |
| 1393 | + * still supports REPORT LUN. | |
| 1394 | + */ | |
| 1395 | + sdevice->sdev_bflags = BLIST_REPORTLUN2 | BLIST_TRY_VPD_PAGES; | |
| 1396 | + | |
| 1397 | + return 0; | |
| 1398 | +} | |
| 1399 | + | |
| 1386 | 1400 | static int storvsc_device_configure(struct scsi_device *sdevice) |
| 1387 | 1401 | { |
| 1388 | 1402 | |
| @@ -1395,14 +1409,6 @@ static int storvsc_device_configure(struct scsi_device *sdevice) · 6 unmodified lines | ||
| 1395 | 1409 | |
| 1396 | 1410 | sdevice->no_write_same = 1; |
| 1397 | 1411 | |
| 1398 | − /* | |
| 1399 | − * Add blist flags to permit the reading of the VPD pages even when | |
| 1400 | − * the target may claim SPC-2 compliance. MSFT targets currently | |
| 1401 | − * claim SPC-2 compliance while they implement post SPC-2 features. | |
| 1402 | − * With this patch we can correctly handle WRITE_SAME_16 issues. | |
| 1403 | − */ | |
| 1404 | − sdevice->sdev_bflags |= msft_blist_flags; | |
| 1405 | − | |
| 1406 | 1412 | /* |
| 1407 | 1413 | * If the host is WIN8 or WIN8 R2, claim conformance to SPC-3 |
| 1408 | 1414 | * if the device is a MSFT virtual device. If the host is |
| @@ -1661,6 +1667,7 @@ static struct scsi_host_template scsi_driver = { · 252 unmodified lines | ||
| 1661 | 1667 | .eh_host_reset_handler = storvsc_host_reset_handler, |
| 1662 | 1668 | .proc_name = "storvsc_host", |
| 1663 | 1669 | .eh_timed_out = storvsc_eh_timed_out, |
| 1670 | + .slave_alloc = storvsc_device_alloc, | |
| 1664 | 1671 | .slave_configure = storvsc_device_configure, |
| 1665 | 1672 | .cmd_per_lun = 255, |
| 1666 | 1673 | .this_id = -1, |
| @@ -146,7 +146,7 @@ enum attr_idn { · 145 unmodified lines | ||
| 146 | 146 | /* Descriptor idn for Query requests */ |
| 147 | 147 | enum desc_idn { |
| 148 | 148 | QUERY_DESC_IDN_DEVICE = 0x0, |
| 149 | − QUERY_DESC_IDN_CONFIGURAION = 0x1, | |
| 149 | + QUERY_DESC_IDN_CONFIGURATION = 0x1, | |
| 150 | 150 | QUERY_DESC_IDN_UNIT = 0x2, |
| 151 | 151 | QUERY_DESC_IDN_RFU_0 = 0x3, |
| 152 | 152 | QUERY_DESC_IDN_INTERCONNECT = 0x4, |
| @@ -162,19 +162,13 @@ enum desc_header_offset { · 9 unmodified lines | ||
| 162 | 162 | QUERY_DESC_DESC_TYPE_OFFSET = 0x01, |
| 163 | 163 | }; |
| 164 | 164 | |
| 165 | −enum ufs_desc_max_size { | |
| 166 | − QUERY_DESC_DEVICE_MAX_SIZE = 0x40, | |
| 167 | − QUERY_DESC_CONFIGURAION_MAX_SIZE = 0x90, | |
| 168 | − QUERY_DESC_UNIT_MAX_SIZE = 0x23, | |
| 169 | − QUERY_DESC_INTERCONNECT_MAX_SIZE = 0x06, | |
| 170 | − /* | |
| 171 | − * Max. 126 UNICODE characters (2 bytes per character) plus 2 bytes | |
| 172 | − * of descriptor header. | |
| 173 | − */ | |
| 174 | − QUERY_DESC_STRING_MAX_SIZE = 0xFE, | |
| 175 | − QUERY_DESC_GEOMETRY_MAX_SIZE = 0x44, | |
| 176 | − QUERY_DESC_POWER_MAX_SIZE = 0x62, | |
| 177 | − QUERY_DESC_RFU_MAX_SIZE = 0x00, | |
| 165 | +enum ufs_desc_def_size { | |
| 166 | + QUERY_DESC_DEVICE_DEF_SIZE = 0x40, | |
| 167 | + QUERY_DESC_CONFIGURATION_DEF_SIZE = 0x90, | |
| 168 | + QUERY_DESC_UNIT_DEF_SIZE = 0x23, | |
| 169 | + QUERY_DESC_INTERCONNECT_DEF_SIZE = 0x06, | |
| 170 | + QUERY_DESC_GEOMETRY_DEF_SIZE = 0x44, | |
| 171 | + QUERY_DESC_POWER_DEF_SIZE = 0x62, | |
| 178 | 172 | }; |
| 179 | 173 | |
| 180 | 174 | /* Unit descriptor parameters offsets in bytes*/ |
| @@ -100,19 +100,6 @@ · 99 unmodified lines | ||
| 100 | 100 | #define ufshcd_hex_dump(prefix_str, buf, len) \ |
| 101 | 101 | print_hex_dump(KERN_ERR, prefix_str, DUMP_PREFIX_OFFSET, 16, 4, buf, len, false) |
| 102 | 102 | |
| 103 | −static u32 ufs_query_desc_max_size[] = { | |
| 104 | − QUERY_DESC_DEVICE_MAX_SIZE, | |
| 105 | − QUERY_DESC_CONFIGURAION_MAX_SIZE, | |
| 106 | − QUERY_DESC_UNIT_MAX_SIZE, | |
| 107 | − QUERY_DESC_RFU_MAX_SIZE, | |
| 108 | − QUERY_DESC_INTERCONNECT_MAX_SIZE, | |
| 109 | − QUERY_DESC_STRING_MAX_SIZE, | |
| 110 | − QUERY_DESC_RFU_MAX_SIZE, | |
| 111 | − QUERY_DESC_GEOMETRY_MAX_SIZE, | |
| 112 | − QUERY_DESC_POWER_MAX_SIZE, | |
| 113 | − QUERY_DESC_RFU_MAX_SIZE, | |
| 114 | −}; | |
| 115 | − | |
| 116 | 103 | enum { |
| 117 | 104 | UFSHCD_MAX_CHANNEL = 0, |
| 118 | 105 | UFSHCD_MAX_ID = 1, |
| @@ -2857,7 +2844,7 @@ static int __ufshcd_query_descriptor(struct ufs_hba *hba, · 2738 unmodified lines | ||
| 2857 | 2844 | goto out; |
| 2858 | 2845 | } |
| 2859 | 2846 | |
| 2860 | − if (*buf_len <= QUERY_DESC_MIN_SIZE || *buf_len > QUERY_DESC_MAX_SIZE) { | |
| 2847 | + if (*buf_len < QUERY_DESC_MIN_SIZE || *buf_len > QUERY_DESC_MAX_SIZE) { | |
| 2861 | 2848 | dev_err(hba->dev, "%s: descriptor buffer size (%d) is out of range\n", |
| 2862 | 2849 | __func__, *buf_len); |
| 2863 | 2850 | err = -EINVAL; |
| @@ -2937,6 +2924,92 @@ static int ufshcd_query_descriptor_retry(struct ufs_hba *hba, · 73 unmodified lines | ||
| 2937 | 2924 | return err; |
| 2938 | 2925 | } |
| 2939 | 2926 | |
| 2927 | +/** | |
| 2928 | + * ufshcd_read_desc_length - read the specified descriptor length from header | |
| 2929 | + * @hba: Pointer to adapter instance | |
| 2930 | + * @desc_id: descriptor idn value | |
| 2931 | + * @desc_index: descriptor index | |
| 2932 | + * @desc_length: pointer to variable to read the length of descriptor | |
| 2933 | + * | |
| 2934 | + * Return 0 in case of success, non-zero otherwise | |
| 2935 | + */ | |
| 2936 | +static int ufshcd_read_desc_length(struct ufs_hba *hba, | |
| 2937 | + enum desc_idn desc_id, | |
| 2938 | + int desc_index, | |
| 2939 | + int *desc_length) | |
| 2940 | +{ | |
| 2941 | + int ret; | |
| 2942 | + u8 header[QUERY_DESC_HDR_SIZE]; | |
| 2943 | + int header_len = QUERY_DESC_HDR_SIZE; | |
| 2944 | + | |
| 2945 | + if (desc_id >= QUERY_DESC_IDN_MAX) | |
| 2946 | + return -EINVAL; | |
| 2947 | + | |
| 2948 | + ret = ufshcd_query_descriptor_retry(hba, UPIU_QUERY_OPCODE_READ_DESC, | |
| 2949 | + desc_id, desc_index, 0, header, | |
| 2950 | + &header_len); | |
| 2951 | + | |
| 2952 | + if (ret) { | |
| 2953 | + dev_err(hba->dev, "%s: Failed to get descriptor header id %d", | |
| 2954 | + __func__, desc_id); | |
| 2955 | + return ret; | |
| 2956 | + } else if (desc_id != header[QUERY_DESC_DESC_TYPE_OFFSET]) { | |
| 2957 | + dev_warn(hba->dev, "%s: descriptor header id %d and desc_id %d mismatch", | |
| 2958 | + __func__, header[QUERY_DESC_DESC_TYPE_OFFSET], | |
| 2959 | + desc_id); | |
| 2960 | + ret = -EINVAL; | |
| 2961 | + } | |
| 2962 | + | |
| 2963 | + *desc_length = header[QUERY_DESC_LENGTH_OFFSET]; | |
| 2964 | + return ret; | |
| 2965 | + | |
| 2966 | +} | |
| 2967 | + | |
| 2968 | +/** | |
| 2969 | + * ufshcd_map_desc_id_to_length - map descriptor IDN to its length | |
| 2970 | + * @hba: Pointer to adapter instance | |
| 2971 | + * @desc_id: descriptor idn value | |
| 2972 | + * @desc_len: mapped desc length (out) | |
| 2973 | + * | |
| 2974 | + * Return 0 in case of success, non-zero otherwise | |
| 2975 | + */ | |
| 2976 | +int ufshcd_map_desc_id_to_length(struct ufs_hba *hba, | |
| 2977 | + enum desc_idn desc_id, int *desc_len) | |
| 2978 | +{ | |
| 2979 | + switch (desc_id) { | |
| 2980 | + case QUERY_DESC_IDN_DEVICE: | |
| 2981 | + *desc_len = hba->desc_size.dev_desc; | |
| 2982 | + break; | |
| 2983 | + case QUERY_DESC_IDN_POWER: | |
| 2984 | + *desc_len = hba->desc_size.pwr_desc; | |
| 2985 | + break; | |
| 2986 | + case QUERY_DESC_IDN_GEOMETRY: | |
| 2987 | + *desc_len = hba->desc_size.geom_desc; | |
| 2988 | + break; | |
| 2989 | + case QUERY_DESC_IDN_CONFIGURATION: | |
| 2990 | + *desc_len = hba->desc_size.conf_desc; | |
| 2991 | + break; | |
| 2992 | + case QUERY_DESC_IDN_UNIT: | |
| 2993 | + *desc_len = hba->desc_size.unit_desc; | |
| 2994 | + break; | |
| 2995 | + case QUERY_DESC_IDN_INTERCONNECT: | |
| 2996 | + *desc_len = hba->desc_size.interc_desc; | |
| 2997 | + break; | |
| 2998 | + case QUERY_DESC_IDN_STRING: | |
| 2999 | + *desc_len = QUERY_DESC_MAX_SIZE; | |
| 3000 | + break; | |
| 3001 | + case QUERY_DESC_IDN_RFU_0: | |
| 3002 | + case QUERY_DESC_IDN_RFU_1: | |
| 3003 | + *desc_len = 0; | |
| 3004 | + break; | |
| 3005 | + default: | |
| 3006 | + *desc_len = 0; | |
| 3007 | + return -EINVAL; | |
| 3008 | + } | |
| 3009 | + return 0; | |
| 3010 | +} | |
| 3011 | +EXPORT_SYMBOL(ufshcd_map_desc_id_to_length); | |
| 3012 | + | |
| 2940 | 3013 | /** |
| 2941 | 3014 | * ufshcd_read_desc_param - read the specified descriptor parameter |
| 2942 | 3015 | * @hba: Pointer to adapter instance |
| @@ -2951,42 +3024,49 @@ static int ufshcd_query_descriptor_retry(struct ufs_hba *hba, · 8 unmodified lines | ||
| 2951 | 3024 | static int ufshcd_read_desc_param(struct ufs_hba *hba, |
| 2952 | 3025 | enum desc_idn desc_id, |
| 2953 | 3026 | int desc_index, |
| 2954 | − u32 param_offset, | |
| 3027 | + u8 param_offset, | |
| 2955 | 3028 | u8 *param_read_buf, |
| 2956 | − u32 param_size) | |
| 3029 | + u8 param_size) | |
| 2957 | 3030 | { |
| 2958 | 3031 | int ret; |
| 2959 | 3032 | u8 *desc_buf; |
| 2960 | − u32 buff_len; | |
| 3033 | + int buff_len; | |
| 2961 | 3034 | bool is_kmalloc = true; |
| 2962 | 3035 | |
| 2963 | − /* safety checks */ | |
| 2964 | − if (desc_id >= QUERY_DESC_IDN_MAX) | |
| 3036 | + /* Safety check */ | |
| 3037 | + if (desc_id >= QUERY_DESC_IDN_MAX || !param_size) | |
| 2965 | 3038 | return -EINVAL; |
| 2966 | 3039 | |
| 2967 | − buff_len = ufs_query_desc_max_size[desc_id]; | |
| 2968 | − if ((param_offset + param_size) > buff_len) | |
| 2969 | − return -EINVAL; | |
| 3040 | + /* Get the max length of descriptor from structure filled up at probe | |
| 3041 | + * time. | |
| 3042 | + */ | |
| 3043 | + ret = ufshcd_map_desc_id_to_length(hba, desc_id, &buff_len); | |
| 2970 | 3044 | |
| 2971 | − if (!param_offset && (param_size == buff_len)) { | |
| 2972 | − /* memory space already available to hold full descriptor */ | |
| 2973 | − desc_buf = param_read_buf; | |
| 2974 | − is_kmalloc = false; | |
| 2975 | − } else { | |
| 2976 | − /* allocate memory to hold full descriptor */ | |
| 3045 | + /* Sanity checks */ | |
| 3046 | + if (ret || !buff_len) { | |
| 3047 | + dev_err(hba->dev, "%s: Failed to get full descriptor length", | |
| 3048 | + __func__); | |
| 3049 | + return ret; | |
| 3050 | + } | |
| 3051 | + | |
| 3052 | + /* Check whether we need temp memory */ | |
| 3053 | + if (param_offset != 0 || param_size < buff_len) { | |
| 2977 | 3054 | desc_buf = kmalloc(buff_len, GFP_KERNEL); |
| 2978 | 3055 | if (!desc_buf) |
| 2979 | 3056 | return -ENOMEM; |
| 3057 | + } else { | |
| 3058 | + desc_buf = param_read_buf; | |
| 3059 | + is_kmalloc = false; | |
| 2980 | 3060 | } |
| 2981 | 3061 | |
| 3062 | + /* Request for full descriptor */ | |
| 2982 | 3063 | ret = ufshcd_query_descriptor_retry(hba, UPIU_QUERY_OPCODE_READ_DESC, |
| 2983 | − desc_id, desc_index, 0, desc_buf, | |
| 2984 | − &buff_len); | |
| 3064 | + desc_id, desc_index, 0, | |
| 3065 | + desc_buf, &buff_len); | |
| 2985 | 3066 | |
| 2986 | 3067 | if (ret) { |
| 2987 | 3068 | dev_err(hba->dev, "%s: Failed reading descriptor. desc_id %d, desc_index %d, param_offset %d, ret %d", |
| 2988 | 3069 | __func__, desc_id, desc_index, param_offset, ret); |
| 2989 | − | |
| 2990 | 3070 | goto out; |
| 2991 | 3071 | } |
| 2992 | 3072 | |
| @@ -2998,25 +3078,9 @@ static int ufshcd_read_desc_param(struct ufs_hba *hba, · 5 unmodified lines | ||
| 2998 | 3078 | goto out; |
| 2999 | 3079 | } |
| 3000 | 3080 | |
| 3001 | − /* | |
| 3002 | − * While reading variable size descriptors (like string descriptor), | |
| 3003 | − * some UFS devices may report the "LENGTH" (field in "Transaction | |
| 3004 | − * Specific fields" of Query Response UPIU) same as what was requested | |
| 3005 | − * in Query Request UPIU instead of reporting the actual size of the | |
| 3006 | − * variable size descriptor. | |
| 3007 | − * Although it's safe to ignore the "LENGTH" field for variable size | |
| 3008 | − * descriptors as we can always derive the length of the descriptor from | |
| 3009 | − * the descriptor header fields. Hence this change impose the length | |
| 3010 | − * match check only for fixed size descriptors (for which we always | |
| 3011 | − * request the correct size as part of Query Request UPIU). | |
| 3012 | − */ | |
| 3013 | − if ((desc_id != QUERY_DESC_IDN_STRING) && | |
| 3014 | − (buff_len != desc_buf[QUERY_DESC_LENGTH_OFFSET])) { | |
| 3015 | − dev_err(hba->dev, "%s: desc_buf length mismatch: buff_len %d, buff_len(desc_header) %d", | |
| 3016 | − __func__, buff_len, desc_buf[QUERY_DESC_LENGTH_OFFSET]); | |
| 3017 | − ret = -EINVAL; | |
| 3018 | − goto out; | |
| 3019 | − } | |
| 3081 | + /* Check wherher we will not copy more data, than available */ | |
| 3082 | + if (is_kmalloc && param_size > buff_len) | |
| 3083 | + param_size = buff_len; | |
| 3020 | 3084 | |
| 3021 | 3085 | if (is_kmalloc) |
| 3022 | 3086 | memcpy(param_read_buf, &desc_buf[param_offset], param_size); |
| @@ -5919,8 +5983,8 @@ static int ufshcd_set_icc_levels_attr(struct ufs_hba *hba, u32 icc_level) · 2896 unmodified lines | ||
| 5919 | 5983 | static void ufshcd_init_icc_levels(struct ufs_hba *hba) |
| 5920 | 5984 | { |
| 5921 | 5985 | int ret; |
| 5922 | − int buff_len = QUERY_DESC_POWER_MAX_SIZE; | |
| 5923 | − u8 desc_buf[QUERY_DESC_POWER_MAX_SIZE]; | |
| 5986 | + int buff_len = hba->desc_size.pwr_desc; | |
| 5987 | + u8 desc_buf[hba->desc_size.pwr_desc]; | |
| 5924 | 5988 | |
| 5925 | 5989 | ret = ufshcd_read_power_desc(hba, desc_buf, buff_len); |
| 5926 | 5990 | if (ret) { |
| @@ -6017,11 +6081,10 @@ static int ufs_get_device_desc(struct ufs_hba *hba, · 90 unmodified lines | ||
| 6017 | 6081 | { |
| 6018 | 6082 | int err; |
| 6019 | 6083 | u8 model_index; |
| 6020 | − u8 str_desc_buf[QUERY_DESC_STRING_MAX_SIZE + 1] = {0}; | |
| 6021 | − u8 desc_buf[QUERY_DESC_DEVICE_MAX_SIZE]; | |
| 6084 | + u8 str_desc_buf[QUERY_DESC_MAX_SIZE + 1] = {0}; | |
| 6085 | + u8 desc_buf[hba->desc_size.dev_desc]; | |
| 6022 | 6086 | |
| 6023 | − err = ufshcd_read_device_desc(hba, desc_buf, | |
| 6024 | − QUERY_DESC_DEVICE_MAX_SIZE); | |
| 6087 | + err = ufshcd_read_device_desc(hba, desc_buf, hba->desc_size.dev_desc); | |
| 6025 | 6088 | if (err) { |
| 6026 | 6089 | dev_err(hba->dev, "%s: Failed reading Device Desc. err = %d\n", |
| 6027 | 6090 | __func__, err); |
| @@ -6038,14 +6101,14 @@ static int ufs_get_device_desc(struct ufs_hba *hba, · 10 unmodified lines | ||
| 6038 | 6101 | model_index = desc_buf[DEVICE_DESC_PARAM_PRDCT_NAME]; |
| 6039 | 6102 | |
| 6040 | 6103 | err = ufshcd_read_string_desc(hba, model_index, str_desc_buf, |
| 6041 | − QUERY_DESC_STRING_MAX_SIZE, ASCII_STD); | |
| 6104 | + QUERY_DESC_MAX_SIZE, ASCII_STD); | |
| 6042 | 6105 | if (err) { |
| 6043 | 6106 | dev_err(hba->dev, "%s: Failed reading Product Name. err = %d\n", |
| 6044 | 6107 | __func__, err); |
| 6045 | 6108 | goto out; |
| 6046 | 6109 | } |
| 6047 | 6110 | |
| 6048 | − str_desc_buf[QUERY_DESC_STRING_MAX_SIZE] = '\0'; | |
| 6111 | + str_desc_buf[QUERY_DESC_MAX_SIZE] = '\0'; | |
| 6049 | 6112 | strlcpy(dev_desc->model, (str_desc_buf + QUERY_DESC_HDR_SIZE), |
| 6050 | 6113 | min_t(u8, str_desc_buf[QUERY_DESC_LENGTH_OFFSET], |
| 6051 | 6114 | MAX_MODEL_LEN)); |
| @@ -6251,6 +6314,51 @@ static void ufshcd_clear_dbg_ufs_stats(struct ufs_hba *hba) · 199 unmodified lines | ||
| 6251 | 6314 | hba->req_abort_count = 0; |
| 6252 | 6315 | } |
| 6253 | 6316 | |
| 6317 | +static void ufshcd_init_desc_sizes(struct ufs_hba *hba) | |
| 6318 | +{ | |
| 6319 | + int err; | |
| 6320 | + | |
| 6321 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_DEVICE, 0, | |
| 6322 | + &hba->desc_size.dev_desc); | |
| 6323 | + if (err) | |
| 6324 | + hba->desc_size.dev_desc = QUERY_DESC_DEVICE_DEF_SIZE; | |
| 6325 | + | |
| 6326 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_POWER, 0, | |
| 6327 | + &hba->desc_size.pwr_desc); | |
| 6328 | + if (err) | |
| 6329 | + hba->desc_size.pwr_desc = QUERY_DESC_POWER_DEF_SIZE; | |
| 6330 | + | |
| 6331 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_INTERCONNECT, 0, | |
| 6332 | + &hba->desc_size.interc_desc); | |
| 6333 | + if (err) | |
| 6334 | + hba->desc_size.interc_desc = QUERY_DESC_INTERCONNECT_DEF_SIZE; | |
| 6335 | + | |
| 6336 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_CONFIGURATION, 0, | |
| 6337 | + &hba->desc_size.conf_desc); | |
| 6338 | + if (err) | |
| 6339 | + hba->desc_size.conf_desc = QUERY_DESC_CONFIGURATION_DEF_SIZE; | |
| 6340 | + | |
| 6341 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_UNIT, 0, | |
| 6342 | + &hba->desc_size.unit_desc); | |
| 6343 | + if (err) | |
| 6344 | + hba->desc_size.unit_desc = QUERY_DESC_UNIT_DEF_SIZE; | |
| 6345 | + | |
| 6346 | + err = ufshcd_read_desc_length(hba, QUERY_DESC_IDN_GEOMETRY, 0, | |
| 6347 | + &hba->desc_size.geom_desc); | |
| 6348 | + if (err) | |
| 6349 | + hba->desc_size.geom_desc = QUERY_DESC_GEOMETRY_DEF_SIZE; | |
| 6350 | +} | |
| 6351 | + | |
| 6352 | +static void ufshcd_def_desc_sizes(struct ufs_hba *hba) | |
| 6353 | +{ | |
| 6354 | + hba->desc_size.dev_desc = QUERY_DESC_DEVICE_DEF_SIZE; | |
| 6355 | + hba->desc_size.pwr_desc = QUERY_DESC_POWER_DEF_SIZE; | |
| 6356 | + hba->desc_size.interc_desc = QUERY_DESC_INTERCONNECT_DEF_SIZE; | |
| 6357 | + hba->desc_size.conf_desc = QUERY_DESC_CONFIGURATION_DEF_SIZE; | |
| 6358 | + hba->desc_size.unit_desc = QUERY_DESC_UNIT_DEF_SIZE; | |
| 6359 | + hba->desc_size.geom_desc = QUERY_DESC_GEOMETRY_DEF_SIZE; | |
| 6360 | +} | |
| 6361 | + | |
| 6254 | 6362 | /** |
| 6255 | 6363 | * ufshcd_probe_hba - probe hba to detect device and initialize |
| 6256 | 6364 | * @hba: per-adapter instance |
| @@ -6285,6 +6393,9 @@ static int ufshcd_probe_hba(struct ufs_hba *hba) · 28 unmodified lines | ||
| 6285 | 6393 | if (ret) |
| 6286 | 6394 | goto out; |
| 6287 | 6395 | |
| 6396 | + /* Init check for device descriptor sizes */ | |
| 6397 | + ufshcd_init_desc_sizes(hba); | |
| 6398 | + | |
| 6288 | 6399 | ret = ufs_get_device_desc(hba, &card); |
| 6289 | 6400 | if (ret) { |
| 6290 | 6401 | dev_err(hba->dev, "%s: Failed getting device info. err = %d\n", |
| @@ -6320,6 +6431,7 @@ static int ufshcd_probe_hba(struct ufs_hba *hba) · 29 unmodified lines | ||
| 6320 | 6431 | |
| 6321 | 6432 | /* set the state as operational after switching to desired gear */ |
| 6322 | 6433 | hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL; |
| 6434 | + | |
| 6323 | 6435 | /* |
| 6324 | 6436 | * If we are in error handling context or in power management callbacks |
| 6325 | 6437 | * context, no need to scan the host |
| @@ -7774,6 +7886,9 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) · 1448 unmodified lines | ||
| 7774 | 7886 | hba->mmio_base = mmio_base; |
| 7775 | 7887 | hba->irq = irq; |
| 7776 | 7888 | |
| 7889 | + /* Set descriptor lengths to specification defaults */ | |
| 7890 | + ufshcd_def_desc_sizes(hba); | |
| 7891 | + | |
| 7777 | 7892 | err = ufshcd_hba_init(hba); |
| 7778 | 7893 | if (err) |
| 7779 | 7894 | goto out_error; |
| @@ -220,6 +220,15 @@ struct ufs_dev_cmd { · 219 unmodified lines | ||
| 220 | 220 | struct ufs_query query; |
| 221 | 221 | }; |
| 222 | 222 | |
| 223 | +struct ufs_desc_size { | |
| 224 | + int dev_desc; | |
| 225 | + int pwr_desc; | |
| 226 | + int geom_desc; | |
| 227 | + int interc_desc; | |
| 228 | + int unit_desc; | |
| 229 | + int conf_desc; | |
| 230 | +}; | |
| 231 | + | |
| 223 | 232 | /** |
| 224 | 233 | * struct ufs_clk_info - UFS clock related info |
| 225 | 234 | * @list: list headed by hba->clk_list_head |
| @@ -483,6 +492,7 @@ struct ufs_stats { · 257 unmodified lines | ||
| 483 | 492 | * @clk_list_head: UFS host controller clocks list node head |
| 484 | 493 | * @pwr_info: holds current power mode |
| 485 | 494 | * @max_pwr_info: keeps the device max valid pwm |
| 495 | + * @desc_size: descriptor sizes reported by device | |
| 486 | 496 | * @urgent_bkops_lvl: keeps track of urgent bkops level for device |
| 487 | 497 | * @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for |
| 488 | 498 | * device is known or not. |
| @@ -666,6 +676,7 @@ struct ufs_hba { · 177 unmodified lines | ||
| 666 | 676 | bool is_urgent_bkops_lvl_checked; |
| 667 | 677 | |
| 668 | 678 | struct rw_semaphore clk_scaling_lock; |
| 679 | + struct ufs_desc_size desc_size; | |
| 669 | 680 | }; |
| 670 | 681 | |
| 671 | 682 | /* Returns true if clocks can be gated. Otherwise false */ |
| @@ -832,6 +843,10 @@ int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, · 160 unmodified lines | ||
| 832 | 843 | enum flag_idn idn, bool *flag_res); |
| 833 | 844 | int ufshcd_hold(struct ufs_hba *hba, bool async); |
| 834 | 845 | void ufshcd_release(struct ufs_hba *hba); |
| 846 | + | |
| 847 | +int ufshcd_map_desc_id_to_length(struct ufs_hba *hba, enum desc_idn desc_id, | |
| 848 | + int *desc_length); | |
| 849 | + | |
| 835 | 850 | u32 ufshcd_get_local_unipro_ver(struct ufs_hba *hba); |
| 836 | 851 | |
| 837 | 852 | /* Wrapper functions for safely calling variant operations */ |
| @@ -1487,7 +1487,7 @@ static int pvscsi_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 1486 unmodified lines | ||
| 1487 | 1487 | irq_flag &= ~PCI_IRQ_MSI; |
| 1488 | 1488 | |
| 1489 | 1489 | error = pci_alloc_irq_vectors(adapter->dev, 1, 1, irq_flag); |
| 1490 | − if (error) | |
| 1490 | + if (error < 0) | |
| 1491 | 1491 | goto out_reset_adapter; |
| 1492 | 1492 | |
| 1493 | 1493 | adapter->use_req_threshold = pvscsi_setup_req_threshold(adapter, true); |
| @@ -532,7 +532,7 @@ lnet_sock_accept(struct socket **newsockp, struct socket *sock) · 531 unmodified lines | ||
| 532 | 532 | |
| 533 | 533 | newsock->ops = sock->ops; |
| 534 | 534 | |
| 535 | − rc = sock->ops->accept(sock, newsock, O_NONBLOCK); | |
| 535 | + rc = sock->ops->accept(sock, newsock, O_NONBLOCK, false); | |
| 536 | 536 | if (rc == -EAGAIN) { |
| 537 | 537 | /* Nothing ready, so wait for activity */ |
| 538 | 538 | init_waitqueue_entry(&wait, current); |
| @@ -540,7 +540,7 @@ lnet_sock_accept(struct socket **newsockp, struct socket *sock) · 1 unmodified line | ||
| 540 | 540 | set_current_state(TASK_INTERRUPTIBLE); |
| 541 | 541 | schedule(); |
| 542 | 542 | remove_wait_queue(sk_sleep(sock->sk), &wait); |
| 543 | − rc = sock->ops->accept(sock, newsock, O_NONBLOCK); | |
| 543 | + rc = sock->ops->accept(sock, newsock, O_NONBLOCK, false); | |
| 544 | 544 | } |
| 545 | 545 | |
| 546 | 546 | if (rc) |
| @@ -336,7 +336,6 @@ static int cvm_oct_poll(struct oct_rx_group *rx_group, int budget) · 335 unmodified lines | ||
| 336 | 336 | if (likely((port < TOTAL_NUMBER_OF_PORTS) && |
| 337 | 337 | cvm_oct_device[port])) { |
| 338 | 338 | struct net_device *dev = cvm_oct_device[port]; |
| 339 | − struct octeon_ethernet *priv = netdev_priv(dev); | |
| 340 | 339 | |
| 341 | 340 | /* |
| 342 | 341 | * Only accept packets for devices that are |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | config BCM2835_VCHIQ |
| 2 | 2 | tristate "Videocore VCHIQ" |
| 3 | 3 | depends on HAS_DMA |
| 4 | + depends on OF | |
| 4 | 5 | depends on RASPBERRYPI_FIRMWARE || (COMPILE_TEST && !RASPBERRYPI_FIRMWARE) |
| 5 | 6 | default y |
| 6 | 7 | help |
| @@ -114,7 +114,7 @@ · 113 unmodified lines | ||
| 114 | 114 | #define DEFAULT_TX_BUF_COUNT 3 |
| 115 | 115 | |
| 116 | 116 | struct n_hdlc_buf { |
| 117 | − struct n_hdlc_buf *link; | |
| 117 | + struct list_head list_item; | |
| 118 | 118 | int count; |
| 119 | 119 | char buf[1]; |
| 120 | 120 | }; |
| @@ -122,8 +122,7 @@ struct n_hdlc_buf { · 1 unmodified line | ||
| 122 | 122 | #define N_HDLC_BUF_SIZE (sizeof(struct n_hdlc_buf) + maxframe) |
| 123 | 123 | |
| 124 | 124 | struct n_hdlc_buf_list { |
| 125 | − struct n_hdlc_buf *head; | |
| 126 | − struct n_hdlc_buf *tail; | |
| 125 | + struct list_head list; | |
| 127 | 126 | int count; |
| 128 | 127 | spinlock_t spinlock; |
| 129 | 128 | }; |
| @@ -136,7 +135,6 @@ struct n_hdlc_buf_list { · 6 unmodified lines | ||
| 136 | 135 | * @backup_tty - TTY to use if tty gets closed |
| 137 | 136 | * @tbusy - reentrancy flag for tx wakeup code |
| 138 | 137 | * @woke_up - FIXME: describe this field |
| 139 | − * @tbuf - currently transmitting tx buffer | |
| 140 | 138 | * @tx_buf_list - list of pending transmit frame buffers |
| 141 | 139 | * @rx_buf_list - list of received frame buffers |
| 142 | 140 | * @tx_free_buf_list - list unused transmit frame buffers |
| @@ -149,7 +147,6 @@ struct n_hdlc { · 6 unmodified lines | ||
| 149 | 147 | struct tty_struct *backup_tty; |
| 150 | 148 | int tbusy; |
| 151 | 149 | int woke_up; |
| 152 | − struct n_hdlc_buf *tbuf; | |
| 153 | 150 | struct n_hdlc_buf_list tx_buf_list; |
| 154 | 151 | struct n_hdlc_buf_list rx_buf_list; |
| 155 | 152 | struct n_hdlc_buf_list tx_free_buf_list; |
| @@ -159,6 +156,8 @@ struct n_hdlc { · 3 unmodified lines | ||
| 159 | 156 | /* |
| 160 | 157 | * HDLC buffer list manipulation functions |
| 161 | 158 | */ |
| 159 | +static void n_hdlc_buf_return(struct n_hdlc_buf_list *buf_list, | |
| 160 | + struct n_hdlc_buf *buf); | |
| 162 | 161 | static void n_hdlc_buf_put(struct n_hdlc_buf_list *list, |
| 163 | 162 | struct n_hdlc_buf *buf); |
| 164 | 163 | static struct n_hdlc_buf *n_hdlc_buf_get(struct n_hdlc_buf_list *list); |
| @@ -208,16 +207,9 @@ static void flush_tx_queue(struct tty_struct *tty) · 43 unmodified lines | ||
| 208 | 207 | { |
| 209 | 208 | struct n_hdlc *n_hdlc = tty2n_hdlc(tty); |
| 210 | 209 | struct n_hdlc_buf *buf; |
| 211 | − unsigned long flags; | |
| 212 | 210 | |
| 213 | 211 | while ((buf = n_hdlc_buf_get(&n_hdlc->tx_buf_list))) |
| 214 | 212 | n_hdlc_buf_put(&n_hdlc->tx_free_buf_list, buf); |
| 215 | − spin_lock_irqsave(&n_hdlc->tx_buf_list.spinlock, flags); | |
| 216 | − if (n_hdlc->tbuf) { | |
| 217 | − n_hdlc_buf_put(&n_hdlc->tx_free_buf_list, n_hdlc->tbuf); | |
| 218 | − n_hdlc->tbuf = NULL; | |
| 219 | − } | |
| 220 | − spin_unlock_irqrestore(&n_hdlc->tx_buf_list.spinlock, flags); | |
| 221 | 213 | } |
| 222 | 214 | |
| 223 | 215 | static struct tty_ldisc_ops n_hdlc_ldisc = { |
| @@ -283,7 +275,6 @@ static void n_hdlc_release(struct n_hdlc *n_hdlc) · 59 unmodified lines | ||
| 283 | 275 | } else |
| 284 | 276 | break; |
| 285 | 277 | } |
| 286 | − kfree(n_hdlc->tbuf); | |
| 287 | 278 | kfree(n_hdlc); |
| 288 | 279 | |
| 289 | 280 | } /* end of n_hdlc_release() */ |
| @@ -402,13 +393,7 @@ static void n_hdlc_send_frames(struct n_hdlc *n_hdlc, struct tty_struct *tty) · 112 unmodified lines | ||
| 402 | 393 | n_hdlc->woke_up = 0; |
| 403 | 394 | spin_unlock_irqrestore(&n_hdlc->tx_buf_list.spinlock, flags); |
| 404 | 395 | |
| 405 | − /* get current transmit buffer or get new transmit */ | |
| 406 | − /* buffer from list of pending transmit buffers */ | |
| 407 | − | |
| 408 | − tbuf = n_hdlc->tbuf; | |
| 409 | − if (!tbuf) | |
| 410 | − tbuf = n_hdlc_buf_get(&n_hdlc->tx_buf_list); | |
| 411 | − | |
| 396 | + tbuf = n_hdlc_buf_get(&n_hdlc->tx_buf_list); | |
| 412 | 397 | while (tbuf) { |
| 413 | 398 | if (debuglevel >= DEBUG_LEVEL_INFO) |
| 414 | 399 | printk("%s(%d)sending frame %p, count=%d\n", |
| @@ -420,7 +405,7 @@ static void n_hdlc_send_frames(struct n_hdlc *n_hdlc, struct tty_struct *tty) · 5 unmodified lines | ||
| 420 | 405 | |
| 421 | 406 | /* rollback was possible and has been done */ |
| 422 | 407 | if (actual == -ERESTARTSYS) { |
| 423 | − n_hdlc->tbuf = tbuf; | |
| 408 | + n_hdlc_buf_return(&n_hdlc->tx_buf_list, tbuf); | |
| 424 | 409 | break; |
| 425 | 410 | } |
| 426 | 411 | /* if transmit error, throw frame away by */ |
| @@ -435,10 +420,7 @@ static void n_hdlc_send_frames(struct n_hdlc *n_hdlc, struct tty_struct *tty) · 8 unmodified lines | ||
| 435 | 420 | |
| 436 | 421 | /* free current transmit buffer */ |
| 437 | 422 | n_hdlc_buf_put(&n_hdlc->tx_free_buf_list, tbuf); |
| 438 | − | |
| 439 | − /* this tx buffer is done */ | |
| 440 | − n_hdlc->tbuf = NULL; | |
| 441 | − | |
| 423 | + | |
| 442 | 424 | /* wait up sleeping writers */ |
| 443 | 425 | wake_up_interruptible(&tty->write_wait); |
| 444 | 426 | |
| @@ -448,10 +430,12 @@ static void n_hdlc_send_frames(struct n_hdlc *n_hdlc, struct tty_struct *tty) · 3 unmodified lines | ||
| 448 | 430 | if (debuglevel >= DEBUG_LEVEL_INFO) |
| 449 | 431 | printk("%s(%d)frame %p pending\n", |
| 450 | 432 | __FILE__,__LINE__,tbuf); |
| 451 | − | |
| 452 | − /* buffer not accepted by driver */ | |
| 453 | − /* set this buffer as pending buffer */ | |
| 454 | − n_hdlc->tbuf = tbuf; | |
| 433 | + | |
| 434 | + /* | |
| 435 | + * the buffer was not accepted by driver, | |
| 436 | + * return it back into tx queue | |
| 437 | + */ | |
| 438 | + n_hdlc_buf_return(&n_hdlc->tx_buf_list, tbuf); | |
| 455 | 439 | break; |
| 456 | 440 | } |
| 457 | 441 | } |
| @@ -749,7 +733,8 @@ static int n_hdlc_tty_ioctl(struct tty_struct *tty, struct file *file, · 291 unmodified lines | ||
| 749 | 733 | int error = 0; |
| 750 | 734 | int count; |
| 751 | 735 | unsigned long flags; |
| 752 | − | |
| 736 | + struct n_hdlc_buf *buf = NULL; | |
| 737 | + | |
| 753 | 738 | if (debuglevel >= DEBUG_LEVEL_INFO) |
| 754 | 739 | printk("%s(%d)n_hdlc_tty_ioctl() called %d\n", |
| 755 | 740 | __FILE__,__LINE__,cmd); |
| @@ -763,8 +748,10 @@ static int n_hdlc_tty_ioctl(struct tty_struct *tty, struct file *file, · 7 unmodified lines | ||
| 763 | 748 | /* report count of read data available */ |
| 764 | 749 | /* in next available frame (if any) */ |
| 765 | 750 | spin_lock_irqsave(&n_hdlc->rx_buf_list.spinlock,flags); |
| 766 | − if (n_hdlc->rx_buf_list.head) | |
| 767 | − count = n_hdlc->rx_buf_list.head->count; | |
| 751 | + buf = list_first_entry_or_null(&n_hdlc->rx_buf_list.list, | |
| 752 | + struct n_hdlc_buf, list_item); | |
| 753 | + if (buf) | |
| 754 | + count = buf->count; | |
| 768 | 755 | else |
| 769 | 756 | count = 0; |
| 770 | 757 | spin_unlock_irqrestore(&n_hdlc->rx_buf_list.spinlock,flags); |
| @@ -776,8 +763,10 @@ static int n_hdlc_tty_ioctl(struct tty_struct *tty, struct file *file, · 5 unmodified lines | ||
| 776 | 763 | count = tty_chars_in_buffer(tty); |
| 777 | 764 | /* add size of next output frame in queue */ |
| 778 | 765 | spin_lock_irqsave(&n_hdlc->tx_buf_list.spinlock,flags); |
| 779 | − if (n_hdlc->tx_buf_list.head) | |
| 780 | − count += n_hdlc->tx_buf_list.head->count; | |
| 766 | + buf = list_first_entry_or_null(&n_hdlc->tx_buf_list.list, | |
| 767 | + struct n_hdlc_buf, list_item); | |
| 768 | + if (buf) | |
| 769 | + count += buf->count; | |
| 781 | 770 | spin_unlock_irqrestore(&n_hdlc->tx_buf_list.spinlock,flags); |
| 782 | 771 | error = put_user(count, (int __user *)arg); |
| 783 | 772 | break; |
| @@ -825,14 +814,14 @@ static unsigned int n_hdlc_tty_poll(struct tty_struct *tty, struct file *filp, · 41 unmodified lines | ||
| 825 | 814 | poll_wait(filp, &tty->write_wait, wait); |
| 826 | 815 | |
| 827 | 816 | /* set bits for operations that won't block */ |
| 828 | − if (n_hdlc->rx_buf_list.head) | |
| 817 | + if (!list_empty(&n_hdlc->rx_buf_list.list)) | |
| 829 | 818 | mask |= POLLIN | POLLRDNORM; /* readable */ |
| 830 | 819 | if (test_bit(TTY_OTHER_CLOSED, &tty->flags)) |
| 831 | 820 | mask |= POLLHUP; |
| 832 | 821 | if (tty_hung_up_p(filp)) |
| 833 | 822 | mask |= POLLHUP; |
| 834 | 823 | if (!tty_is_writelocked(tty) && |
| 835 | − n_hdlc->tx_free_buf_list.head) | |
| 824 | + !list_empty(&n_hdlc->tx_free_buf_list.list)) | |
| 836 | 825 | mask |= POLLOUT | POLLWRNORM; /* writable */ |
| 837 | 826 | } |
| 838 | 827 | return mask; |
| @@ -856,7 +845,12 @@ static struct n_hdlc *n_hdlc_alloc(void) · 17 unmodified lines | ||
| 856 | 845 | spin_lock_init(&n_hdlc->tx_free_buf_list.spinlock); |
| 857 | 846 | spin_lock_init(&n_hdlc->rx_buf_list.spinlock); |
| 858 | 847 | spin_lock_init(&n_hdlc->tx_buf_list.spinlock); |
| 859 | − | |
| 848 | + | |
| 849 | + INIT_LIST_HEAD(&n_hdlc->rx_free_buf_list.list); | |
| 850 | + INIT_LIST_HEAD(&n_hdlc->tx_free_buf_list.list); | |
| 851 | + INIT_LIST_HEAD(&n_hdlc->rx_buf_list.list); | |
| 852 | + INIT_LIST_HEAD(&n_hdlc->tx_buf_list.list); | |
| 853 | + | |
| 860 | 854 | /* allocate free rx buffer list */ |
| 861 | 855 | for(i=0;i<DEFAULT_RX_BUF_COUNT;i++) { |
| 862 | 856 | buf = kmalloc(N_HDLC_BUF_SIZE, GFP_KERNEL); |
| @@ -883,54 +877,66 @@ static struct n_hdlc *n_hdlc_alloc(void) · 20 unmodified lines | ||
| 883 | 877 | |
| 884 | 878 | } /* end of n_hdlc_alloc() */ |
| 885 | 879 | |
| 880 | +/** | |
| 881 | + * n_hdlc_buf_return - put the HDLC buffer after the head of the specified list | |
| 882 | + * @buf_list - pointer to the buffer list | |
| 883 | + * @buf - pointer to the buffer | |
| 884 | + */ | |
| 885 | +static void n_hdlc_buf_return(struct n_hdlc_buf_list *buf_list, | |
| 886 | + struct n_hdlc_buf *buf) | |
| 887 | +{ | |
| 888 | + unsigned long flags; | |
| 889 | + | |
| 890 | + spin_lock_irqsave(&buf_list->spinlock, flags); | |
| 891 | + | |
| 892 | + list_add(&buf->list_item, &buf_list->list); | |
| 893 | + buf_list->count++; | |
| 894 | + | |
| 895 | + spin_unlock_irqrestore(&buf_list->spinlock, flags); | |
| 896 | +} | |
| 897 | + | |
| 886 | 898 | /** |
| 887 | 899 | * n_hdlc_buf_put - add specified HDLC buffer to tail of specified list |
| 888 | − * @list - pointer to buffer list | |
| 900 | + * @buf_list - pointer to buffer list | |
| 889 | 901 | * @buf - pointer to buffer |
| 890 | 902 | */ |
| 891 | −static void n_hdlc_buf_put(struct n_hdlc_buf_list *list, | |
| 903 | +static void n_hdlc_buf_put(struct n_hdlc_buf_list *buf_list, | |
| 892 | 904 | struct n_hdlc_buf *buf) |
| 893 | 905 | { |
| 894 | 906 | unsigned long flags; |
| 895 | − spin_lock_irqsave(&list->spinlock,flags); | |
| 896 | − | |
| 897 | − buf->link=NULL; | |
| 898 | − if (list->tail) | |
| 899 | − list->tail->link = buf; | |
| 900 | − else | |
| 901 | − list->head = buf; | |
| 902 | − list->tail = buf; | |
| 903 | − (list->count)++; | |
| 904 | − | |
| 905 | − spin_unlock_irqrestore(&list->spinlock,flags); | |
| 906 | − | |
| 907 | + | |
| 908 | + spin_lock_irqsave(&buf_list->spinlock, flags); | |
| 909 | + | |
| 910 | + list_add_tail(&buf->list_item, &buf_list->list); | |
| 911 | + buf_list->count++; | |
| 912 | + | |
| 913 | + spin_unlock_irqrestore(&buf_list->spinlock, flags); | |
| 907 | 914 | } /* end of n_hdlc_buf_put() */ |
| 908 | 915 | |
| 909 | 916 | /** |
| 910 | 917 | * n_hdlc_buf_get - remove and return an HDLC buffer from list |
| 911 | − * @list - pointer to HDLC buffer list | |
| 918 | + * @buf_list - pointer to HDLC buffer list | |
| 912 | 919 | * |
| 913 | 920 | * Remove and return an HDLC buffer from the head of the specified HDLC buffer |
| 914 | 921 | * list. |
| 915 | 922 | * Returns a pointer to HDLC buffer if available, otherwise %NULL. |
| 916 | 923 | */ |
| 917 | −static struct n_hdlc_buf* n_hdlc_buf_get(struct n_hdlc_buf_list *list) | |
| 924 | +static struct n_hdlc_buf *n_hdlc_buf_get(struct n_hdlc_buf_list *buf_list) | |
| 918 | 925 | { |
| 919 | 926 | unsigned long flags; |
| 920 | 927 | struct n_hdlc_buf *buf; |
| 921 | − spin_lock_irqsave(&list->spinlock,flags); | |
| 922 | − | |
| 923 | − buf = list->head; | |
| 928 | + | |
| 929 | + spin_lock_irqsave(&buf_list->spinlock, flags); | |
| 930 | + | |
| 931 | + buf = list_first_entry_or_null(&buf_list->list, | |
| 932 | + struct n_hdlc_buf, list_item); | |
| 924 | 933 | if (buf) { |
| 925 | − list->head = buf->link; | |
| 926 | − (list->count)--; | |
| 934 | + list_del(&buf->list_item); | |
| 935 | + buf_list->count--; | |
| 927 | 936 | } |
| 928 | − if (!list->head) | |
| 929 | − list->tail = NULL; | |
| 930 | − | |
| 931 | − spin_unlock_irqrestore(&list->spinlock,flags); | |
| 937 | + | |
| 938 | + spin_unlock_irqrestore(&buf_list->spinlock, flags); | |
| 932 | 939 | return buf; |
| 933 | − | |
| 934 | 940 | } /* end of n_hdlc_buf_get() */ |
| 935 | 941 | |
| 936 | 942 | static char hdlc_banner[] __initdata = |
| @@ -1031,8 +1031,10 @@ static int s3c64xx_serial_startup(struct uart_port *port) · 1030 unmodified lines | ||
| 1031 | 1031 | if (ourport->dma) { |
| 1032 | 1032 | ret = s3c24xx_serial_request_dma(ourport); |
| 1033 | 1033 | if (ret < 0) { |
| 1034 | − dev_warn(port->dev, "DMA request failed\n"); | |
| 1035 | − return ret; | |
| 1034 | + dev_warn(port->dev, | |
| 1035 | + "DMA request failed, DMA will not be used\n"); | |
| 1036 | + devm_kfree(port->dev, ourport->dma); | |
| 1037 | + ourport->dma = NULL; | |
| 1036 | 1038 | } |
| 1037 | 1039 | } |
| 1038 | 1040 | |
| @@ -250,6 +250,7 @@ static void dwc3_omap_set_mailbox(struct dwc3_omap *omap, · 249 unmodified lines | ||
| 250 | 250 | val = dwc3_omap_read_utmi_ctrl(omap); |
| 251 | 251 | val |= USBOTGSS_UTMI_OTG_CTRL_IDDIG; |
| 252 | 252 | dwc3_omap_write_utmi_ctrl(omap, val); |
| 253 | + break; | |
| 253 | 254 | |
| 254 | 255 | case OMAP_DWC3_VBUS_OFF: |
| 255 | 256 | val = dwc3_omap_read_utmi_ctrl(omap); |
| @@ -392,7 +393,7 @@ static void dwc3_omap_set_utmi_mode(struct dwc3_omap *omap) · 136 unmodified lines | ||
| 392 | 393 | { |
| 393 | 394 | u32 reg; |
| 394 | 395 | struct device_node *node = omap->dev->of_node; |
| 395 | − int utmi_mode = 0; | |
| 396 | + u32 utmi_mode = 0; | |
| 396 | 397 | |
| 397 | 398 | reg = dwc3_omap_read_utmi_ctrl(omap); |
| 398 | 399 | |
| @@ -1342,6 +1342,68 @@ static int dwc3_gadget_ep_dequeue(struct usb_ep *ep, · 1341 unmodified lines | ||
| 1342 | 1342 | if (r == req) { |
| 1343 | 1343 | /* wait until it is processed */ |
| 1344 | 1344 | dwc3_stop_active_transfer(dwc, dep->number, true); |
| 1345 | + | |
| 1346 | + /* | |
| 1347 | + * If request was already started, this means we had to | |
| 1348 | + * stop the transfer. With that we also need to ignore | |
| 1349 | + * all TRBs used by the request, however TRBs can only | |
| 1350 | + * be modified after completion of END_TRANSFER | |
| 1351 | + * command. So what we do here is that we wait for | |
| 1352 | + * END_TRANSFER completion and only after that, we jump | |
| 1353 | + * over TRBs by clearing HWO and incrementing dequeue | |
| 1354 | + * pointer. | |
| 1355 | + * | |
| 1356 | + * Note that we have 2 possible types of transfers here: | |
| 1357 | + * | |
| 1358 | + * i) Linear buffer request | |
| 1359 | + * ii) SG-list based request | |
| 1360 | + * | |
| 1361 | + * SG-list based requests will have r->num_pending_sgs | |
| 1362 | + * set to a valid number (> 0). Linear requests, | |
| 1363 | + * normally use a single TRB. | |
| 1364 | + * | |
| 1365 | + * For each of these two cases, if r->unaligned flag is | |
| 1366 | + * set, one extra TRB has been used to align transfer | |
| 1367 | + * size to wMaxPacketSize. | |
| 1368 | + * | |
| 1369 | + * All of these cases need to be taken into | |
| 1370 | + * consideration so we don't mess up our TRB ring | |
| 1371 | + * pointers. | |
| 1372 | + */ | |
| 1373 | + wait_event_lock_irq(dep->wait_end_transfer, | |
| 1374 | + !(dep->flags & DWC3_EP_END_TRANSFER_PENDING), | |
| 1375 | + dwc->lock); | |
| 1376 | + | |
| 1377 | + if (!r->trb) | |
| 1378 | + goto out1; | |
| 1379 | + | |
| 1380 | + if (r->num_pending_sgs) { | |
| 1381 | + struct dwc3_trb *trb; | |
| 1382 | + int i = 0; | |
| 1383 | + | |
| 1384 | + for (i = 0; i < r->num_pending_sgs; i++) { | |
| 1385 | + trb = r->trb + i; | |
| 1386 | + trb->ctrl &= ~DWC3_TRB_CTRL_HWO; | |
| 1387 | + dwc3_ep_inc_deq(dep); | |
| 1388 | + } | |
| 1389 | + | |
| 1390 | + if (r->unaligned) { | |
| 1391 | + trb = r->trb + r->num_pending_sgs + 1; | |
| 1392 | + trb->ctrl &= ~DWC3_TRB_CTRL_HWO; | |
| 1393 | + dwc3_ep_inc_deq(dep); | |
| 1394 | + } | |
| 1395 | + } else { | |
| 1396 | + struct dwc3_trb *trb = r->trb; | |
| 1397 | + | |
| 1398 | + trb->ctrl &= ~DWC3_TRB_CTRL_HWO; | |
| 1399 | + dwc3_ep_inc_deq(dep); | |
| 1400 | + | |
| 1401 | + if (r->unaligned) { | |
| 1402 | + trb = r->trb + 1; | |
| 1403 | + trb->ctrl &= ~DWC3_TRB_CTRL_HWO; | |
| 1404 | + dwc3_ep_inc_deq(dep); | |
| 1405 | + } | |
| 1406 | + } | |
| 1345 | 1407 | goto out1; |
| 1346 | 1408 | } |
| 1347 | 1409 | dev_err(dwc->dev, "request %p was not queued to %s\n", |
| @@ -1352,6 +1414,7 @@ static int dwc3_gadget_ep_dequeue(struct usb_ep *ep, · 4 unmodified lines | ||
| 1352 | 1414 | |
| 1353 | 1415 | out1: |
| 1354 | 1416 | /* giveback the request */ |
| 1417 | + dep->queued_requests--; | |
| 1355 | 1418 | dwc3_gadget_giveback(dep, req, -ECONNRESET); |
| 1356 | 1419 | |
| 1357 | 1420 | out0: |
| @@ -2126,12 +2189,12 @@ static int __dwc3_cleanup_done_trbs(struct dwc3 *dwc, struct dwc3_ep *dep, · 768 unmodified lines | ||
| 2126 | 2189 | return 1; |
| 2127 | 2190 | } |
| 2128 | 2191 | |
| 2129 | − if ((trb->ctrl & DWC3_TRB_CTRL_HWO) && status != -ESHUTDOWN) | |
| 2130 | − return 1; | |
| 2131 | − | |
| 2132 | 2192 | count = trb->size & DWC3_TRB_SIZE_MASK; |
| 2133 | 2193 | req->remaining += count; |
| 2134 | 2194 | |
| 2195 | + if ((trb->ctrl & DWC3_TRB_CTRL_HWO) && status != -ESHUTDOWN) | |
| 2196 | + return 1; | |
| 2197 | + | |
| 2135 | 2198 | if (dep->direction) { |
| 2136 | 2199 | if (count) { |
| 2137 | 2200 | trb_status = DWC3_TRB_SIZE_TRBSTS(trb->size); |
| @@ -3228,15 +3291,10 @@ void dwc3_gadget_exit(struct dwc3 *dwc) · 1090 unmodified lines | ||
| 3228 | 3291 | |
| 3229 | 3292 | int dwc3_gadget_suspend(struct dwc3 *dwc) |
| 3230 | 3293 | { |
| 3231 | − int ret; | |
| 3232 | − | |
| 3233 | 3294 | if (!dwc->gadget_driver) |
| 3234 | 3295 | return 0; |
| 3235 | 3296 | |
| 3236 | − ret = dwc3_gadget_run_stop(dwc, false, false); | |
| 3237 | − if (ret < 0) | |
| 3238 | − return ret; | |
| 3239 | − | |
| 3297 | + dwc3_gadget_run_stop(dwc, false, false); | |
| 3240 | 3298 | dwc3_disconnect_gadget(dwc); |
| 3241 | 3299 | __dwc3_gadget_stop(dwc); |
| 3242 | 3300 | |
| @@ -28,23 +28,23 @@ struct dwc3; · 27 unmodified lines | ||
| 28 | 28 | #define gadget_to_dwc(g) (container_of(g, struct dwc3, gadget)) |
| 29 | 29 | |
| 30 | 30 | /* DEPCFG parameter 1 */ |
| 31 | −#define DWC3_DEPCFG_INT_NUM(n) ((n) << 0) | |
| 31 | +#define DWC3_DEPCFG_INT_NUM(n) (((n) & 0x1f) << 0) | |
| 32 | 32 | #define DWC3_DEPCFG_XFER_COMPLETE_EN (1 << 8) |
| 33 | 33 | #define DWC3_DEPCFG_XFER_IN_PROGRESS_EN (1 << 9) |
| 34 | 34 | #define DWC3_DEPCFG_XFER_NOT_READY_EN (1 << 10) |
| 35 | 35 | #define DWC3_DEPCFG_FIFO_ERROR_EN (1 << 11) |
| 36 | 36 | #define DWC3_DEPCFG_STREAM_EVENT_EN (1 << 13) |
| 37 | −#define DWC3_DEPCFG_BINTERVAL_M1(n) ((n) << 16) | |
| 37 | +#define DWC3_DEPCFG_BINTERVAL_M1(n) (((n) & 0xff) << 16) | |
| 38 | 38 | #define DWC3_DEPCFG_STREAM_CAPABLE (1 << 24) |
| 39 | −#define DWC3_DEPCFG_EP_NUMBER(n) ((n) << 25) | |
| 39 | +#define DWC3_DEPCFG_EP_NUMBER(n) (((n) & 0x1f) << 25) | |
| 40 | 40 | #define DWC3_DEPCFG_BULK_BASED (1 << 30) |
| 41 | 41 | #define DWC3_DEPCFG_FIFO_BASED (1 << 31) |
| 42 | 42 | |
| 43 | 43 | /* DEPCFG parameter 0 */ |
| 44 | −#define DWC3_DEPCFG_EP_TYPE(n) ((n) << 1) | |
| 45 | −#define DWC3_DEPCFG_MAX_PACKET_SIZE(n) ((n) << 3) | |
| 46 | −#define DWC3_DEPCFG_FIFO_NUMBER(n) ((n) << 17) | |
| 47 | −#define DWC3_DEPCFG_BURST_SIZE(n) ((n) << 22) | |
| 44 | +#define DWC3_DEPCFG_EP_TYPE(n) (((n) & 0x3) << 1) | |
| 45 | +#define DWC3_DEPCFG_MAX_PACKET_SIZE(n) (((n) & 0x7ff) << 3) | |
| 46 | +#define DWC3_DEPCFG_FIFO_NUMBER(n) (((n) & 0x1f) << 17) | |
| 47 | +#define DWC3_DEPCFG_BURST_SIZE(n) (((n) & 0xf) << 22) | |
| 48 | 48 | #define DWC3_DEPCFG_DATA_SEQ_NUM(n) ((n) << 26) |
| 49 | 49 | /* This applies for core versions earlier than 1.94a */ |
| 50 | 50 | #define DWC3_DEPCFG_IGN_SEQ_NUM (1 << 31) |
| @@ -269,6 +269,7 @@ static ssize_t gadget_dev_desc_UDC_store(struct config_item *item, · 268 unmodified lines | ||
| 269 | 269 | ret = unregister_gadget(gi); |
| 270 | 270 | if (ret) |
| 271 | 271 | goto err; |
| 272 | + kfree(name); | |
| 272 | 273 | } else { |
| 273 | 274 | if (gi->composite.gadget_driver.udc_name) { |
| 274 | 275 | ret = -EBUSY; |
| @@ -1834,11 +1834,14 @@ static int ffs_func_eps_enable(struct ffs_function *func) · 1833 unmodified lines | ||
| 1834 | 1834 | spin_lock_irqsave(&func->ffs->eps_lock, flags); |
| 1835 | 1835 | while(count--) { |
| 1836 | 1836 | struct usb_endpoint_descriptor *ds; |
| 1837 | + struct usb_ss_ep_comp_descriptor *comp_desc = NULL; | |
| 1838 | + int needs_comp_desc = false; | |
| 1837 | 1839 | int desc_idx; |
| 1838 | 1840 | |
| 1839 | − if (ffs->gadget->speed == USB_SPEED_SUPER) | |
| 1841 | + if (ffs->gadget->speed == USB_SPEED_SUPER) { | |
| 1840 | 1842 | desc_idx = 2; |
| 1841 | − else if (ffs->gadget->speed == USB_SPEED_HIGH) | |
| 1843 | + needs_comp_desc = true; | |
| 1844 | + } else if (ffs->gadget->speed == USB_SPEED_HIGH) | |
| 1842 | 1845 | desc_idx = 1; |
| 1843 | 1846 | else |
| 1844 | 1847 | desc_idx = 0; |
| @@ -1855,6 +1858,14 @@ static int ffs_func_eps_enable(struct ffs_function *func) · 10 unmodified lines | ||
| 1855 | 1858 | |
| 1856 | 1859 | ep->ep->driver_data = ep; |
| 1857 | 1860 | ep->ep->desc = ds; |
| 1861 | + | |
| 1862 | + comp_desc = (struct usb_ss_ep_comp_descriptor *)(ds + | |
| 1863 | + USB_DT_ENDPOINT_SIZE); | |
| 1864 | + ep->ep->maxburst = comp_desc->bMaxBurst + 1; | |
| 1865 | + | |
| 1866 | + if (needs_comp_desc) | |
| 1867 | + ep->ep->comp_desc = comp_desc; | |
| 1868 | + | |
| 1858 | 1869 | ret = usb_ep_enable(ep->ep); |
| 1859 | 1870 | if (likely(!ret)) { |
| 1860 | 1871 | epfile->ep = ep; |
| @@ -2253,7 +2264,7 @@ static int __ffs_data_do_os_desc(enum ffs_os_desc_type type, · 392 unmodified lines | ||
| 2253 | 2264 | |
| 2254 | 2265 | if (len < sizeof(*d) || |
| 2255 | 2266 | d->bFirstInterfaceNumber >= ffs->interfaces_count || |
| 2256 | − d->Reserved1) | |
| 2267 | + !d->Reserved1) | |
| 2257 | 2268 | return -EINVAL; |
| 2258 | 2269 | for (i = 0; i < ARRAY_SIZE(d->Reserved2); ++i) |
| 2259 | 2270 | if (d->Reserved2[i]) |
| @@ -258,13 +258,6 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl) · 257 unmodified lines | ||
| 258 | 258 | memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req)); |
| 259 | 259 | v4l2_event_queue(&uvc->vdev, &v4l2_event); |
| 260 | 260 | |
| 261 | − /* Pass additional setup data to userspace */ | |
| 262 | − if (uvc->event_setup_out && uvc->event_length) { | |
| 263 | − uvc->control_req->length = uvc->event_length; | |
| 264 | − return usb_ep_queue(uvc->func.config->cdev->gadget->ep0, | |
| 265 | − uvc->control_req, GFP_ATOMIC); | |
| 266 | − } | |
| 267 | − | |
| 268 | 261 | return 0; |
| 269 | 262 | } |
| 270 | 263 | |
| @@ -84,8 +84,7 @@ static int ep_open(struct inode *, struct file *); · 83 unmodified lines | ||
| 84 | 84 | |
| 85 | 85 | /* /dev/gadget/$CHIP represents ep0 and the whole device */ |
| 86 | 86 | enum ep0_state { |
| 87 | − /* DISBLED is the initial state. | |
| 88 | − */ | |
| 87 | + /* DISABLED is the initial state. */ | |
| 89 | 88 | STATE_DEV_DISABLED = 0, |
| 90 | 89 | |
| 91 | 90 | /* Only one open() of /dev/gadget/$CHIP; only one file tracks |
| @@ -1782,8 +1781,10 @@ dev_config (struct file *fd, const char __user *buf, size_t len, loff_t *ptr) · 1690 unmodified lines | ||
| 1782 | 1781 | |
| 1783 | 1782 | spin_lock_irq (&dev->lock); |
| 1784 | 1783 | value = -EINVAL; |
| 1785 | − if (dev->buf) | |
| 1784 | + if (dev->buf) { | |
| 1785 | + kfree(kbuf); | |
| 1786 | 1786 | goto fail; |
| 1787 | + } | |
| 1787 | 1788 | dev->buf = kbuf; |
| 1788 | 1789 | |
| 1789 | 1790 | /* full or low speed config */ |
| @@ -610,7 +610,7 @@ usba_ep_enable(struct usb_ep *_ep, const struct usb_endpoint_descriptor *desc) · 609 unmodified lines | ||
| 610 | 610 | { |
| 611 | 611 | struct usba_ep *ep = to_usba_ep(_ep); |
| 612 | 612 | struct usba_udc *udc = ep->udc; |
| 613 | − unsigned long flags, ept_cfg, maxpacket; | |
| 613 | + unsigned long flags, maxpacket; | |
| 614 | 614 | unsigned int nr_trans; |
| 615 | 615 | |
| 616 | 616 | DBG(DBG_GADGET, "%s: ep_enable: desc=%p\n", ep->ep.name, desc); |
| @@ -630,7 +630,7 @@ usba_ep_enable(struct usb_ep *_ep, const struct usb_endpoint_descriptor *desc) · 13 unmodified lines | ||
| 630 | 630 | ep->is_in = 0; |
| 631 | 631 | |
| 632 | 632 | DBG(DBG_ERR, "%s: EPT_CFG = 0x%lx (maxpacket = %lu)\n", |
| 633 | − ep->ep.name, ept_cfg, maxpacket); | |
| 633 | + ep->ep.name, ep->ept_cfg, maxpacket); | |
| 634 | 634 | |
| 635 | 635 | if (usb_endpoint_dir_in(desc)) { |
| 636 | 636 | ep->is_in = 1; |
| @@ -1031,6 +1031,8 @@ static int dummy_udc_probe(struct platform_device *pdev) · 1030 unmodified lines | ||
| 1031 | 1031 | int rc; |
| 1032 | 1032 | |
| 1033 | 1033 | dum = *((void **)dev_get_platdata(&pdev->dev)); |
| 1034 | + /* Clear usb_gadget region for new registration to udc-core */ | |
| 1035 | + memzero_explicit(&dum->gadget, sizeof(struct usb_gadget)); | |
| 1034 | 1036 | dum->gadget.name = gadget_name; |
| 1035 | 1037 | dum->gadget.ops = &dummy_ops; |
| 1036 | 1038 | dum->gadget.max_speed = USB_SPEED_SUPER; |
| @@ -1146,15 +1146,15 @@ static int scan_dma_completions(struct net2280_ep *ep) · 1145 unmodified lines | ||
| 1146 | 1146 | */ |
| 1147 | 1147 | while (!list_empty(&ep->queue)) { |
| 1148 | 1148 | struct net2280_request *req; |
| 1149 | − u32 tmp; | |
| 1149 | + u32 req_dma_count; | |
| 1150 | 1150 | |
| 1151 | 1151 | req = list_entry(ep->queue.next, |
| 1152 | 1152 | struct net2280_request, queue); |
| 1153 | 1153 | if (!req->valid) |
| 1154 | 1154 | break; |
| 1155 | 1155 | rmb(); |
| 1156 | − tmp = le32_to_cpup(&req->td->dmacount); | |
| 1157 | − if ((tmp & BIT(VALID_BIT)) != 0) | |
| 1156 | + req_dma_count = le32_to_cpup(&req->td->dmacount); | |
| 1157 | + if ((req_dma_count & BIT(VALID_BIT)) != 0) | |
| 1158 | 1158 | break; |
| 1159 | 1159 | |
| 1160 | 1160 | /* SHORT_PACKET_TRANSFERRED_INTERRUPT handles "usb-short" |
| @@ -1163,40 +1163,41 @@ static int scan_dma_completions(struct net2280_ep *ep) · 2 unmodified lines | ||
| 1163 | 1163 | */ |
| 1164 | 1164 | if (unlikely(req->td->dmadesc == 0)) { |
| 1165 | 1165 | /* paranoia */ |
| 1166 | − tmp = readl(&ep->dma->dmacount); | |
| 1167 | − if (tmp & DMA_BYTE_COUNT_MASK) | |
| 1166 | + u32 const ep_dmacount = readl(&ep->dma->dmacount); | |
| 1167 | + | |
| 1168 | + if (ep_dmacount & DMA_BYTE_COUNT_MASK) | |
| 1168 | 1169 | break; |
| 1169 | 1170 | /* single transfer mode */ |
| 1170 | − dma_done(ep, req, tmp, 0); | |
| 1171 | + dma_done(ep, req, req_dma_count, 0); | |
| 1171 | 1172 | num_completed++; |
| 1172 | 1173 | break; |
| 1173 | 1174 | } else if (!ep->is_in && |
| 1174 | 1175 | (req->req.length % ep->ep.maxpacket) && |
| 1175 | 1176 | !(ep->dev->quirks & PLX_PCIE)) { |
| 1176 | 1177 | |
| 1177 | − tmp = readl(&ep->regs->ep_stat); | |
| 1178 | + u32 const ep_stat = readl(&ep->regs->ep_stat); | |
| 1178 | 1179 | /* AVOID TROUBLE HERE by not issuing short reads from |
| 1179 | 1180 | * your gadget driver. That helps avoids errata 0121, |
| 1180 | 1181 | * 0122, and 0124; not all cases trigger the warning. |
| 1181 | 1182 | */ |
| 1182 | − if ((tmp & BIT(NAK_OUT_PACKETS)) == 0) { | |
| 1183 | + if ((ep_stat & BIT(NAK_OUT_PACKETS)) == 0) { | |
| 1183 | 1184 | ep_warn(ep->dev, "%s lost packet sync!\n", |
| 1184 | 1185 | ep->ep.name); |
| 1185 | 1186 | req->req.status = -EOVERFLOW; |
| 1186 | 1187 | } else { |
| 1187 | − tmp = readl(&ep->regs->ep_avail); | |
| 1188 | − if (tmp) { | |
| 1188 | + u32 const ep_avail = readl(&ep->regs->ep_avail); | |
| 1189 | + if (ep_avail) { | |
| 1189 | 1190 | /* fifo gets flushed later */ |
| 1190 | 1191 | ep->out_overflow = 1; |
| 1191 | 1192 | ep_dbg(ep->dev, |
| 1192 | 1193 | "%s dma, discard %d len %d\n", |
| 1193 | − ep->ep.name, tmp, | |
| 1194 | + ep->ep.name, ep_avail, | |
| 1194 | 1195 | req->req.length); |
| 1195 | 1196 | req->req.status = -EOVERFLOW; |
| 1196 | 1197 | } |
| 1197 | 1198 | } |
| 1198 | 1199 | } |
| 1199 | − dma_done(ep, req, tmp, 0); | |
| 1200 | + dma_done(ep, req, req_dma_count, 0); | |
| 1200 | 1201 | num_completed++; |
| 1201 | 1202 | } |
| 1202 | 1203 | |
| @@ -2534,9 +2534,10 @@ static int pxa_udc_remove(struct platform_device *_dev) · 2533 unmodified lines | ||
| 2534 | 2534 | usb_del_gadget_udc(&udc->gadget); |
| 2535 | 2535 | pxa_cleanup_debugfs(udc); |
| 2536 | 2536 | |
| 2537 | − if (!IS_ERR_OR_NULL(udc->transceiver)) | |
| 2537 | + if (!IS_ERR_OR_NULL(udc->transceiver)) { | |
| 2538 | 2538 | usb_unregister_notifier(udc->transceiver, &pxa27x_udc_phy); |
| 2539 | − usb_put_phy(udc->transceiver); | |
| 2539 | + usb_put_phy(udc->transceiver); | |
| 2540 | + } | |
| 2540 | 2541 | |
| 2541 | 2542 | udc->transceiver = NULL; |
| 2542 | 2543 | the_controller = NULL; |
| @@ -350,7 +350,7 @@ static int ohci_at91_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue, · 349 unmodified lines | ||
| 350 | 350 | |
| 351 | 351 | case USB_PORT_FEAT_SUSPEND: |
| 352 | 352 | dev_dbg(hcd->self.controller, "SetPortFeat: SUSPEND\n"); |
| 353 | − if (valid_port(wIndex)) { | |
| 353 | + if (valid_port(wIndex) && ohci_at91->sfr_regmap) { | |
| 354 | 354 | ohci_at91_port_suspend(ohci_at91->sfr_regmap, |
| 355 | 355 | 1); |
| 356 | 356 | return 0; |
| @@ -393,7 +393,7 @@ static int ohci_at91_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue, · 36 unmodified lines | ||
| 393 | 393 | |
| 394 | 394 | case USB_PORT_FEAT_SUSPEND: |
| 395 | 395 | dev_dbg(hcd->self.controller, "ClearPortFeature: SUSPEND\n"); |
| 396 | − if (valid_port(wIndex)) { | |
| 396 | + if (valid_port(wIndex) && ohci_at91->sfr_regmap) { | |
| 397 | 397 | ohci_at91_port_suspend(ohci_at91->sfr_regmap, |
| 398 | 398 | 0); |
| 399 | 399 | return 0; |
| @@ -109,7 +109,7 @@ static void xhci_print_cap_regs(struct xhci_hcd *xhci) · 108 unmodified lines | ||
| 109 | 109 | xhci_dbg(xhci, "RTSOFF 0x%x:\n", temp & RTSOFF_MASK); |
| 110 | 110 | |
| 111 | 111 | /* xhci 1.1 controllers have the HCCPARAMS2 register */ |
| 112 | − if (hci_version > 100) { | |
| 112 | + if (hci_version > 0x100) { | |
| 113 | 113 | temp = readl(&xhci->cap_regs->hcc_params2); |
| 114 | 114 | xhci_dbg(xhci, "HCC PARAMS2 0x%x:\n", (unsigned int) temp); |
| 115 | 115 | xhci_dbg(xhci, " HC %s Force save context capability", |
| @@ -382,7 +382,6 @@ static int usb_wakeup_of_property_parse(struct xhci_hcd_mtk *mtk, · 381 unmodified lines | ||
| 382 | 382 | |
| 383 | 383 | static int xhci_mtk_setup(struct usb_hcd *hcd); |
| 384 | 384 | static const struct xhci_driver_overrides xhci_mtk_overrides __initconst = { |
| 385 | − .extra_priv_size = sizeof(struct xhci_hcd), | |
| 386 | 385 | .reset = xhci_mtk_setup, |
| 387 | 386 | }; |
| 388 | 387 | |
| @@ -678,13 +677,13 @@ static int xhci_mtk_probe(struct platform_device *pdev) · 289 unmodified lines | ||
| 678 | 677 | goto power_off_phys; |
| 679 | 678 | } |
| 680 | 679 | |
| 681 | − if (HCC_MAX_PSA(xhci->hcc_params) >= 4) | |
| 682 | − xhci->shared_hcd->can_do_streams = 1; | |
| 683 | − | |
| 684 | 680 | ret = usb_add_hcd(hcd, irq, IRQF_SHARED); |
| 685 | 681 | if (ret) |
| 686 | 682 | goto put_usb3_hcd; |
| 687 | 683 | |
| 684 | + if (HCC_MAX_PSA(xhci->hcc_params) >= 4) | |
| 685 | + xhci->shared_hcd->can_do_streams = 1; | |
| 686 | + | |
| 688 | 687 | ret = usb_add_hcd(xhci->shared_hcd, irq, IRQF_SHARED); |
| 689 | 688 | if (ret) |
| 690 | 689 | goto dealloc_usb2_hcd; |
| @@ -286,6 +286,8 @@ static int xhci_plat_remove(struct platform_device *dev) · 285 unmodified lines | ||
| 286 | 286 | struct xhci_hcd *xhci = hcd_to_xhci(hcd); |
| 287 | 287 | struct clk *clk = xhci->clk; |
| 288 | 288 | |
| 289 | + xhci->xhc_state |= XHCI_STATE_REMOVING; | |
| 290 | + | |
| 289 | 291 | usb_remove_hcd(xhci->shared_hcd); |
| 290 | 292 | usb_phy_shutdown(hcd->usb_phy); |
| 291 | 293 | |
| @@ -1308,7 +1308,6 @@ static int tegra_xhci_setup(struct usb_hcd *hcd) · 1307 unmodified lines | ||
| 1308 | 1308 | } |
| 1309 | 1309 | |
| 1310 | 1310 | static const struct xhci_driver_overrides tegra_xhci_overrides __initconst = { |
| 1311 | − .extra_priv_size = sizeof(struct xhci_hcd), | |
| 1312 | 1311 | .reset = tegra_xhci_setup, |
| 1313 | 1312 | }; |
| 1314 | 1313 | |
| @@ -868,7 +868,7 @@ static void xhci_disable_port_wake_on_bits(struct xhci_hcd *xhci) · 867 unmodified lines | ||
| 868 | 868 | |
| 869 | 869 | spin_lock_irqsave(&xhci->lock, flags); |
| 870 | 870 | |
| 871 | − /* disble usb3 ports Wake bits*/ | |
| 871 | + /* disable usb3 ports Wake bits */ | |
| 872 | 872 | port_index = xhci->num_usb3_ports; |
| 873 | 873 | port_array = xhci->usb3_ports; |
| 874 | 874 | while (port_index--) { |
| @@ -879,7 +879,7 @@ static void xhci_disable_port_wake_on_bits(struct xhci_hcd *xhci) · 4 unmodified lines | ||
| 879 | 879 | writel(t2, port_array[port_index]); |
| 880 | 880 | } |
| 881 | 881 | |
| 882 | − /* disble usb2 ports Wake bits*/ | |
| 882 | + /* disable usb2 ports Wake bits */ | |
| 883 | 883 | port_index = xhci->num_usb2_ports; |
| 884 | 884 | port_array = xhci->usb2_ports; |
| 885 | 885 | while (port_index--) { |
| @@ -781,12 +781,6 @@ static int iowarrior_probe(struct usb_interface *interface, · 780 unmodified lines | ||
| 781 | 781 | iface_desc = interface->cur_altsetting; |
| 782 | 782 | dev->product_id = le16_to_cpu(udev->descriptor.idProduct); |
| 783 | 783 | |
| 784 | − if (iface_desc->desc.bNumEndpoints < 1) { | |
| 785 | − dev_err(&interface->dev, "Invalid number of endpoints\n"); | |
| 786 | − retval = -EINVAL; | |
| 787 | − goto error; | |
| 788 | − } | |
| 789 | − | |
| 790 | 784 | /* set up the endpoint information */ |
| 791 | 785 | for (i = 0; i < iface_desc->desc.bNumEndpoints; ++i) { |
| 792 | 786 | endpoint = &iface_desc->endpoint[i].desc; |
| @@ -797,6 +791,21 @@ static int iowarrior_probe(struct usb_interface *interface, · 4 unmodified lines | ||
| 797 | 791 | /* this one will match for the IOWarrior56 only */ |
| 798 | 792 | dev->int_out_endpoint = endpoint; |
| 799 | 793 | } |
| 794 | + | |
| 795 | + if (!dev->int_in_endpoint) { | |
| 796 | + dev_err(&interface->dev, "no interrupt-in endpoint found\n"); | |
| 797 | + retval = -ENODEV; | |
| 798 | + goto error; | |
| 799 | + } | |
| 800 | + | |
| 801 | + if (dev->product_id == USB_DEVICE_ID_CODEMERCS_IOW56) { | |
| 802 | + if (!dev->int_out_endpoint) { | |
| 803 | + dev_err(&interface->dev, "no interrupt-out endpoint found\n"); | |
| 804 | + retval = -ENODEV; | |
| 805 | + goto error; | |
| 806 | + } | |
| 807 | + } | |
| 808 | + | |
| 800 | 809 | /* we have to check the report_size often, so remember it in the endianness suitable for our machine */ |
| 801 | 810 | dev->report_size = usb_endpoint_maxp(dev->int_in_endpoint); |
| 802 | 811 | if ((dev->interface->cur_altsetting->desc.bInterfaceNumber == 0) && |
| @@ -375,18 +375,24 @@ static int usb251xb_get_ofdata(struct usb251xb *hub, · 374 unmodified lines | ||
| 375 | 375 | if (of_get_property(np, "dynamic-power-switching", NULL)) |
| 376 | 376 | hub->conf_data2 |= BIT(7); |
| 377 | 377 | |
| 378 | − if (of_get_property(np, "oc-delay-100us", NULL)) { | |
| 379 | − hub->conf_data2 &= ~BIT(5); | |
| 380 | − hub->conf_data2 &= ~BIT(4); | |
| 381 | − } else if (of_get_property(np, "oc-delay-4ms", NULL)) { | |
| 382 | − hub->conf_data2 &= ~BIT(5); | |
| 383 | − hub->conf_data2 |= BIT(4); | |
| 384 | − } else if (of_get_property(np, "oc-delay-8ms", NULL)) { | |
| 385 | − hub->conf_data2 |= BIT(5); | |
| 386 | − hub->conf_data2 &= ~BIT(4); | |
| 387 | − } else if (of_get_property(np, "oc-delay-16ms", NULL)) { | |
| 388 | − hub->conf_data2 |= BIT(5); | |
| 389 | − hub->conf_data2 |= BIT(4); | |
| 378 | + if (!of_property_read_u32(np, "oc-delay-us", property_u32)) { | |
| 379 | + if (*property_u32 == 100) { | |
| 380 | + /* 100 us*/ | |
| 381 | + hub->conf_data2 &= ~BIT(5); | |
| 382 | + hub->conf_data2 &= ~BIT(4); | |
| 383 | + } else if (*property_u32 == 4000) { | |
| 384 | + /* 4 ms */ | |
| 385 | + hub->conf_data2 &= ~BIT(5); | |
| 386 | + hub->conf_data2 |= BIT(4); | |
| 387 | + } else if (*property_u32 == 16000) { | |
| 388 | + /* 16 ms */ | |
| 389 | + hub->conf_data2 |= BIT(5); | |
| 390 | + hub->conf_data2 |= BIT(4); | |
| 391 | + } else { | |
| 392 | + /* 8 ms (DEFAULT) */ | |
| 393 | + hub->conf_data2 |= BIT(5); | |
| 394 | + hub->conf_data2 &= ~BIT(4); | |
| 395 | + } | |
| 390 | 396 | } |
| 391 | 397 | |
| 392 | 398 | if (of_get_property(np, "compound-device", NULL)) |
| @@ -432,30 +438,9 @@ static int usb251xb_get_ofdata(struct usb251xb *hub, · 39 unmodified lines | ||
| 432 | 438 | } |
| 433 | 439 | } |
| 434 | 440 | |
| 435 | − hub->max_power_sp = USB251XB_DEF_MAX_POWER_SELF; | |
| 436 | − if (!of_property_read_u32(np, "max-sp-power", property_u32)) | |
| 437 | − hub->max_power_sp = min_t(u8, be32_to_cpu(*property_u32) / 2, | |
| 438 | − 250); | |
| 439 | − | |
| 440 | − hub->max_power_bp = USB251XB_DEF_MAX_POWER_BUS; | |
| 441 | − if (!of_property_read_u32(np, "max-bp-power", property_u32)) | |
| 442 | − hub->max_power_bp = min_t(u8, be32_to_cpu(*property_u32) / 2, | |
| 443 | − 250); | |
| 444 | − | |
| 445 | − hub->max_current_sp = USB251XB_DEF_MAX_CURRENT_SELF; | |
| 446 | − if (!of_property_read_u32(np, "max-sp-current", property_u32)) | |
| 447 | − hub->max_current_sp = min_t(u8, be32_to_cpu(*property_u32) / 2, | |
| 448 | − 250); | |
| 449 | − | |
| 450 | − hub->max_current_bp = USB251XB_DEF_MAX_CURRENT_BUS; | |
| 451 | − if (!of_property_read_u32(np, "max-bp-current", property_u32)) | |
| 452 | − hub->max_current_bp = min_t(u8, be32_to_cpu(*property_u32) / 2, | |
| 453 | − 250); | |
| 454 | − | |
| 455 | 441 | hub->power_on_time = USB251XB_DEF_POWER_ON_TIME; |
| 456 | − if (!of_property_read_u32(np, "power-on-time", property_u32)) | |
| 457 | − hub->power_on_time = min_t(u8, be32_to_cpu(*property_u32) / 2, | |
| 458 | − 255); | |
| 442 | + if (!of_property_read_u32(np, "power-on-time-ms", property_u32)) | |
| 443 | + hub->power_on_time = min_t(u8, *property_u32 / 2, 255); | |
| 459 | 444 | |
| 460 | 445 | if (of_property_read_u16_array(np, "language-id", &hub->lang_id, 1)) |
| 461 | 446 | hub->lang_id = USB251XB_DEF_LANGUAGE_ID; |
| @@ -492,6 +477,10 @@ static int usb251xb_get_ofdata(struct usb251xb *hub, · 30 unmodified lines | ||
| 492 | 477 | /* The following parameters are currently not exposed to devicetree, but |
| 493 | 478 | * may be as soon as needed. |
| 494 | 479 | */ |
| 480 | + hub->max_power_sp = USB251XB_DEF_MAX_POWER_SELF; | |
| 481 | + hub->max_power_bp = USB251XB_DEF_MAX_POWER_BUS; | |
| 482 | + hub->max_current_sp = USB251XB_DEF_MAX_CURRENT_SELF; | |
| 483 | + hub->max_current_bp = USB251XB_DEF_MAX_CURRENT_BUS; | |
| 495 | 484 | hub->bat_charge_en = USB251XB_DEF_BATTERY_CHARGING_ENABLE; |
| 496 | 485 | hub->boost_up = USB251XB_DEF_BOOST_UP; |
| 497 | 486 | hub->boost_x = USB251XB_DEF_BOOST_X; |
| @@ -33,6 +33,12 @@ static const struct i2c_device_id isp1301_id[] = { · 32 unmodified lines | ||
| 33 | 33 | }; |
| 34 | 34 | MODULE_DEVICE_TABLE(i2c, isp1301_id); |
| 35 | 35 | |
| 36 | +static const struct of_device_id isp1301_of_match[] = { | |
| 37 | + {.compatible = "nxp,isp1301" }, | |
| 38 | + { }, | |
| 39 | +}; | |
| 40 | +MODULE_DEVICE_TABLE(of, isp1301_of_match); | |
| 41 | + | |
| 36 | 42 | static struct i2c_client *isp1301_i2c_client; |
| 37 | 43 | |
| 38 | 44 | static int __isp1301_write(struct isp1301 *isp, u8 reg, u8 value, u8 clear) |
| @@ -130,6 +136,7 @@ static int isp1301_remove(struct i2c_client *client) · 91 unmodified lines | ||
| 130 | 136 | static struct i2c_driver isp1301_driver = { |
| 131 | 137 | .driver = { |
| 132 | 138 | .name = DRV_NAME, |
| 139 | + .of_match_table = of_match_ptr(isp1301_of_match), | |
| 133 | 140 | }, |
| 134 | 141 | .probe = isp1301_probe, |
| 135 | 142 | .remove = isp1301_remove, |
| @@ -1500,7 +1500,7 @@ static int digi_read_oob_callback(struct urb *urb) · 1499 unmodified lines | ||
| 1500 | 1500 | return -1; |
| 1501 | 1501 | |
| 1502 | 1502 | /* handle each oob command */ |
| 1503 | − for (i = 0; i < urb->actual_length - 4; i += 4) { | |
| 1503 | + for (i = 0; i < urb->actual_length - 3; i += 4) { | |
| 1504 | 1504 | opcode = buf[i]; |
| 1505 | 1505 | line = buf[i + 1]; |
| 1506 | 1506 | status = buf[i + 2]; |
| @@ -1674,6 +1674,12 @@ static void edge_interrupt_callback(struct urb *urb) · 1673 unmodified lines | ||
| 1674 | 1674 | function = TIUMP_GET_FUNC_FROM_CODE(data[0]); |
| 1675 | 1675 | dev_dbg(dev, "%s - port_number %d, function %d, info 0x%x\n", __func__, |
| 1676 | 1676 | port_number, function, data[1]); |
| 1677 | + | |
| 1678 | + if (port_number >= edge_serial->serial->num_ports) { | |
| 1679 | + dev_err(dev, "bad port number %d\n", port_number); | |
| 1680 | + goto exit; | |
| 1681 | + } | |
| 1682 | + | |
| 1677 | 1683 | port = edge_serial->serial->port[port_number]; |
| 1678 | 1684 | edge_port = usb_get_serial_port_data(port); |
| 1679 | 1685 | if (!edge_port) { |
| @@ -1755,7 +1761,7 @@ static void edge_bulk_in_callback(struct urb *urb) · 75 unmodified lines | ||
| 1755 | 1761 | |
| 1756 | 1762 | port_number = edge_port->port->port_number; |
| 1757 | 1763 | |
| 1758 | − if (edge_port->lsr_event) { | |
| 1764 | + if (urb->actual_length > 0 && edge_port->lsr_event) { | |
| 1759 | 1765 | edge_port->lsr_event = 0; |
| 1760 | 1766 | dev_dbg(dev, "%s ===== Port %u LSR Status = %02x, Data = %02x ======\n", |
| 1761 | 1767 | __func__, port_number, edge_port->lsr_mask, *data); |
| @@ -31,7 +31,6 @@ · 30 unmodified lines | ||
| 31 | 31 | #define BT_IGNITIONPRO_ID 0x2000 |
| 32 | 32 | |
| 33 | 33 | /* function prototypes */ |
| 34 | −static int omninet_open(struct tty_struct *tty, struct usb_serial_port *port); | |
| 35 | 34 | static void omninet_process_read_urb(struct urb *urb); |
| 36 | 35 | static void omninet_write_bulk_callback(struct urb *urb); |
| 37 | 36 | static int omninet_write(struct tty_struct *tty, struct usb_serial_port *port, |
| @@ -60,7 +59,6 @@ static struct usb_serial_driver zyxel_omninet_device = { · 22 unmodified lines | ||
| 60 | 59 | .attach = omninet_attach, |
| 61 | 60 | .port_probe = omninet_port_probe, |
| 62 | 61 | .port_remove = omninet_port_remove, |
| 63 | − .open = omninet_open, | |
| 64 | 62 | .write = omninet_write, |
| 65 | 63 | .write_room = omninet_write_room, |
| 66 | 64 | .write_bulk_callback = omninet_write_bulk_callback, |
| @@ -140,17 +138,6 @@ static int omninet_port_remove(struct usb_serial_port *port) · 73 unmodified lines | ||
| 140 | 138 | return 0; |
| 141 | 139 | } |
| 142 | 140 | |
| 143 | −static int omninet_open(struct tty_struct *tty, struct usb_serial_port *port) | |
| 144 | −{ | |
| 145 | − struct usb_serial *serial = port->serial; | |
| 146 | − struct usb_serial_port *wport; | |
| 147 | − | |
| 148 | − wport = serial->port[1]; | |
| 149 | − tty_port_tty_set(&wport->port, tty); | |
| 150 | − | |
| 151 | − return usb_serial_generic_open(tty, port); | |
| 152 | −} | |
| 153 | − | |
| 154 | 141 | #define OMNINET_HEADERLEN 4 |
| 155 | 142 | #define OMNINET_BULKOUTSIZE 64 |
| 156 | 143 | #define OMNINET_PAYLOADSIZE (OMNINET_BULKOUTSIZE - OMNINET_HEADERLEN) |
| @@ -200,6 +200,11 @@ static void safe_process_read_urb(struct urb *urb) · 199 unmodified lines | ||
| 200 | 200 | if (!safe) |
| 201 | 201 | goto out; |
| 202 | 202 | |
| 203 | + if (length < 2) { | |
| 204 | + dev_err(&port->dev, "malformed packet\n"); | |
| 205 | + return; | |
| 206 | + } | |
| 207 | + | |
| 203 | 208 | fcs = fcs_compute10(data, length, CRC10_INITFCS); |
| 204 | 209 | if (fcs) { |
| 205 | 210 | dev_err(&port->dev, "%s - bad CRC %x\n", __func__, fcs); |
| @@ -2071,6 +2071,20 @@ UNUSUAL_DEV( 0x1370, 0x6828, 0x0110, 0x0110, · 2070 unmodified lines | ||
| 2071 | 2071 | USB_SC_DEVICE, USB_PR_DEVICE, NULL, |
| 2072 | 2072 | US_FL_IGNORE_RESIDUE ), |
| 2073 | 2073 | |
| 2074 | +/* | |
| 2075 | + * Reported by Tobias Jakobi <tjakobi@math.uni-bielefeld.de> | |
| 2076 | + * The INIC-3619 bridge is used in the StarTech SLSODDU33B | |
| 2077 | + * SATA-USB enclosure for slimline optical drives. | |
| 2078 | + * | |
| 2079 | + * The quirk enables MakeMKV to properly exchange keys with | |
| 2080 | + * an installed BD drive. | |
| 2081 | + */ | |
| 2082 | +UNUSUAL_DEV( 0x13fd, 0x3609, 0x0209, 0x0209, | |
| 2083 | + "Initio Corporation", | |
| 2084 | + "INIC-3619", | |
| 2085 | + USB_SC_DEVICE, USB_PR_DEVICE, NULL, | |
| 2086 | + US_FL_IGNORE_RESIDUE ), | |
| 2087 | + | |
| 2074 | 2088 | /* Reported by Qinglin Ye <yestyle@gmail.com> */ |
| 2075 | 2089 | UNUSUAL_DEV( 0x13fe, 0x3600, 0x0100, 0x0100, |
| 2076 | 2090 | "Kingston", |
| @@ -681,3 +681,50 @@ xen_swiotlb_set_dma_mask(struct device *dev, u64 dma_mask) · 680 unmodified lines | ||
| 681 | 681 | return 0; |
| 682 | 682 | } |
| 683 | 683 | EXPORT_SYMBOL_GPL(xen_swiotlb_set_dma_mask); |
| 684 | + | |
| 685 | +/* | |
| 686 | + * Create userspace mapping for the DMA-coherent memory. | |
| 687 | + * This function should be called with the pages from the current domain only, | |
| 688 | + * passing pages mapped from other domains would lead to memory corruption. | |
| 689 | + */ | |
| 690 | +int | |
| 691 | +xen_swiotlb_dma_mmap(struct device *dev, struct vm_area_struct *vma, | |
| 692 | + void *cpu_addr, dma_addr_t dma_addr, size_t size, | |
| 693 | + unsigned long attrs) | |
| 694 | +{ | |
| 695 | +#if defined(CONFIG_ARM) || defined(CONFIG_ARM64) | |
| 696 | + if (__generic_dma_ops(dev)->mmap) | |
| 697 | + return __generic_dma_ops(dev)->mmap(dev, vma, cpu_addr, | |
| 698 | + dma_addr, size, attrs); | |
| 699 | +#endif | |
| 700 | + return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size); | |
| 701 | +} | |
| 702 | +EXPORT_SYMBOL_GPL(xen_swiotlb_dma_mmap); | |
| 703 | + | |
| 704 | +/* | |
| 705 | + * This function should be called with the pages from the current domain only, | |
| 706 | + * passing pages mapped from other domains would lead to memory corruption. | |
| 707 | + */ | |
| 708 | +int | |
| 709 | +xen_swiotlb_get_sgtable(struct device *dev, struct sg_table *sgt, | |
| 710 | + void *cpu_addr, dma_addr_t handle, size_t size, | |
| 711 | + unsigned long attrs) | |
| 712 | +{ | |
| 713 | +#if defined(CONFIG_ARM) || defined(CONFIG_ARM64) | |
| 714 | + if (__generic_dma_ops(dev)->get_sgtable) { | |
| 715 | +#if 0 | |
| 716 | + /* | |
| 717 | + * This check verifies that the page belongs to the current domain and | |
| 718 | + * is not one mapped from another domain. | |
| 719 | + * This check is for debug only, and should not go to production build | |
| 720 | + */ | |
| 721 | + unsigned long bfn = PHYS_PFN(dma_to_phys(dev, handle)); | |
| 722 | + BUG_ON (!page_is_ram(bfn)); | |
| 723 | +#endif | |
| 724 | + return __generic_dma_ops(dev)->get_sgtable(dev, sgt, cpu_addr, | |
| 725 | + handle, size, attrs); | |
| 726 | + } | |
| 727 | +#endif | |
| 728 | + return dma_common_get_sgtable(dev, sgt, cpu_addr, handle, size); | |
| 729 | +} | |
| 730 | +EXPORT_SYMBOL_GPL(xen_swiotlb_get_sgtable); | |
| @@ -55,7 +55,6 @@ · 54 unmodified lines | ||
| 55 | 55 | #include <linux/string.h> |
| 56 | 56 | #include <linux/slab.h> |
| 57 | 57 | #include <linux/miscdevice.h> |
| 58 | −#include <linux/init.h> | |
| 59 | 58 | |
| 60 | 59 | #include <xen/xenbus.h> |
| 61 | 60 | #include <xen/xen.h> |
| @@ -743,7 +743,7 @@ static int tcp_accept_from_sock(struct connection *con) · 742 unmodified lines | ||
| 743 | 743 | newsock->type = con->sock->type; |
| 744 | 744 | newsock->ops = con->sock->ops; |
| 745 | 745 | |
| 746 | − result = con->sock->ops->accept(con->sock, newsock, O_NONBLOCK); | |
| 746 | + result = con->sock->ops->accept(con->sock, newsock, O_NONBLOCK, true); | |
| 747 | 747 | if (result < 0) |
| 748 | 748 | goto accept_err; |
| 749 | 749 | |
| @@ -1359,6 +1359,16 @@ out: · 1358 unmodified lines | ||
| 1359 | 1359 | return 0; |
| 1360 | 1360 | } |
| 1361 | 1361 | |
| 1362 | +static void fat_dummy_inode_init(struct inode *inode) | |
| 1363 | +{ | |
| 1364 | + /* Initialize this dummy inode to work as no-op. */ | |
| 1365 | + MSDOS_I(inode)->mmu_private = 0; | |
| 1366 | + MSDOS_I(inode)->i_start = 0; | |
| 1367 | + MSDOS_I(inode)->i_logstart = 0; | |
| 1368 | + MSDOS_I(inode)->i_attrs = 0; | |
| 1369 | + MSDOS_I(inode)->i_pos = 0; | |
| 1370 | +} | |
| 1371 | + | |
| 1362 | 1372 | static int fat_read_root(struct inode *inode) |
| 1363 | 1373 | { |
| 1364 | 1374 | struct msdos_sb_info *sbi = MSDOS_SB(inode->i_sb); |
| @@ -1803,12 +1813,13 @@ int fat_fill_super(struct super_block *sb, void *data, int silent, int isvfat, · 438 unmodified lines | ||
| 1803 | 1813 | fat_inode = new_inode(sb); |
| 1804 | 1814 | if (!fat_inode) |
| 1805 | 1815 | goto out_fail; |
| 1806 | − MSDOS_I(fat_inode)->i_pos = 0; | |
| 1816 | + fat_dummy_inode_init(fat_inode); | |
| 1807 | 1817 | sbi->fat_inode = fat_inode; |
| 1808 | 1818 | |
| 1809 | 1819 | fsinfo_inode = new_inode(sb); |
| 1810 | 1820 | if (!fsinfo_inode) |
| 1811 | 1821 | goto out_fail; |
| 1822 | + fat_dummy_inode_init(fsinfo_inode); | |
| 1812 | 1823 | fsinfo_inode->i_ino = MSDOS_FSINFO_INO; |
| 1813 | 1824 | sbi->fsinfo_inode = fsinfo_inode; |
| 1814 | 1825 | insert_inode_hash(fsinfo_inode); |
| @@ -173,19 +173,33 @@ static void wb_wakeup(struct bdi_writeback *wb) · 172 unmodified lines | ||
| 173 | 173 | spin_unlock_bh(&wb->work_lock); |
| 174 | 174 | } |
| 175 | 175 | |
| 176 | +static void finish_writeback_work(struct bdi_writeback *wb, | |
| 177 | + struct wb_writeback_work *work) | |
| 178 | +{ | |
| 179 | + struct wb_completion *done = work->done; | |
| 180 | + | |
| 181 | + if (work->auto_free) | |
| 182 | + kfree(work); | |
| 183 | + if (done && atomic_dec_and_test(&done->cnt)) | |
| 184 | + wake_up_all(&wb->bdi->wb_waitq); | |
| 185 | +} | |
| 186 | + | |
| 176 | 187 | static void wb_queue_work(struct bdi_writeback *wb, |
| 177 | 188 | struct wb_writeback_work *work) |
| 178 | 189 | { |
| 179 | 190 | trace_writeback_queue(wb, work); |
| 180 | 191 | |
| 181 | − spin_lock_bh(&wb->work_lock); | |
| 182 | − if (!test_bit(WB_registered, &wb->state)) | |
| 183 | − goto out_unlock; | |
| 184 | 192 | if (work->done) |
| 185 | 193 | atomic_inc(&work->done->cnt); |
| 186 | − list_add_tail(&work->list, &wb->work_list); | |
| 187 | − mod_delayed_work(bdi_wq, &wb->dwork, 0); | |
| 188 | −out_unlock: | |
| 194 | + | |
| 195 | + spin_lock_bh(&wb->work_lock); | |
| 196 | + | |
| 197 | + if (test_bit(WB_registered, &wb->state)) { | |
| 198 | + list_add_tail(&work->list, &wb->work_list); | |
| 199 | + mod_delayed_work(bdi_wq, &wb->dwork, 0); | |
| 200 | + } else | |
| 201 | + finish_writeback_work(wb, work); | |
| 202 | + | |
| 189 | 203 | spin_unlock_bh(&wb->work_lock); |
| 190 | 204 | } |
| 191 | 205 | |
| @@ -1873,16 +1887,9 @@ static long wb_do_writeback(struct bdi_writeback *wb) · 1681 unmodified lines | ||
| 1873 | 1887 | |
| 1874 | 1888 | set_bit(WB_writeback_running, &wb->state); |
| 1875 | 1889 | while ((work = get_next_work_item(wb)) != NULL) { |
| 1876 | − struct wb_completion *done = work->done; | |
| 1877 | − | |
| 1878 | 1890 | trace_writeback_exec(wb, work); |
| 1879 | − | |
| 1880 | 1891 | wrote += wb_writeback(wb, work); |
| 1881 | − | |
| 1882 | − if (work->auto_free) | |
| 1883 | − kfree(work); | |
| 1884 | − if (done && atomic_dec_and_test(&done->cnt)) | |
| 1885 | − wake_up_all(&wb->bdi->wb_waitq); | |
| 1892 | + finish_writeback_work(wb, work); | |
| 1886 | 1893 | } |
| 1887 | 1894 | |
| 1888 | 1895 | /* |
| @@ -207,7 +207,7 @@ struct lm_lockname { · 206 unmodified lines | ||
| 207 | 207 | struct gfs2_sbd *ln_sbd; |
| 208 | 208 | u64 ln_number; |
| 209 | 209 | unsigned int ln_type; |
| 210 | −}; | |
| 210 | +} __packed __aligned(sizeof(int)); | |
| 211 | 211 | |
| 212 | 212 | #define lm_name_equal(name1, name2) \ |
| 213 | 213 | (((name1)->ln_number == (name2)->ln_number) && \ |
| @@ -846,7 +846,8 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, · 845 unmodified lines | ||
| 846 | 846 | struct address_space *mapping = iocb->ki_filp->f_mapping; |
| 847 | 847 | struct inode *inode = file_inode(iocb->ki_filp); |
| 848 | 848 | size_t count = iov_iter_count(iter); |
| 849 | − loff_t pos = iocb->ki_pos, end = iocb->ki_pos + count - 1, ret = 0; | |
| 849 | + loff_t pos = iocb->ki_pos, start = pos; | |
| 850 | + loff_t end = iocb->ki_pos + count - 1, ret = 0; | |
| 850 | 851 | unsigned int flags = IOMAP_DIRECT; |
| 851 | 852 | struct blk_plug plug; |
| 852 | 853 | struct iomap_dio *dio; |
| @@ -887,12 +888,12 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, · 34 unmodified lines | ||
| 887 | 888 | } |
| 888 | 889 | |
| 889 | 890 | if (mapping->nrpages) { |
| 890 | − ret = filemap_write_and_wait_range(mapping, iocb->ki_pos, end); | |
| 891 | + ret = filemap_write_and_wait_range(mapping, start, end); | |
| 891 | 892 | if (ret) |
| 892 | 893 | goto out_free_dio; |
| 893 | 894 | |
| 894 | 895 | ret = invalidate_inode_pages2_range(mapping, |
| 895 | − iocb->ki_pos >> PAGE_SHIFT, end >> PAGE_SHIFT); | |
| 896 | + start >> PAGE_SHIFT, end >> PAGE_SHIFT); | |
| 896 | 897 | WARN_ON_ONCE(ret); |
| 897 | 898 | ret = 0; |
| 898 | 899 | } |
| @@ -941,6 +942,8 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, · 42 unmodified lines | ||
| 941 | 942 | __set_current_state(TASK_RUNNING); |
| 942 | 943 | } |
| 943 | 944 | |
| 945 | + ret = iomap_dio_complete(dio); | |
| 946 | + | |
| 944 | 947 | /* |
| 945 | 948 | * Try again to invalidate clean pages which might have been cached by |
| 946 | 949 | * non-direct readahead, or faulted in by get_user_pages() if the source |
| @@ -949,12 +952,12 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, · 2 unmodified lines | ||
| 949 | 952 | * this invalidation fails, tough, the write still worked... |
| 950 | 953 | */ |
| 951 | 954 | if (iov_iter_rw(iter) == WRITE && mapping->nrpages) { |
| 952 | − ret = invalidate_inode_pages2_range(mapping, | |
| 953 | − iocb->ki_pos >> PAGE_SHIFT, end >> PAGE_SHIFT); | |
| 954 | − WARN_ON_ONCE(ret); | |
| 955 | + int err = invalidate_inode_pages2_range(mapping, | |
| 956 | + start >> PAGE_SHIFT, end >> PAGE_SHIFT); | |
| 957 | + WARN_ON_ONCE(err); | |
| 955 | 958 | } |
| 956 | 959 | |
| 957 | − return iomap_dio_complete(dio); | |
| 960 | + return ret; | |
| 958 | 961 | |
| 959 | 962 | out_free_dio: |
| 960 | 963 | kfree(dio); |
| @@ -1863,7 +1863,7 @@ static int o2net_accept_one(struct socket *sock, int *more) · 1862 unmodified lines | ||
| 1863 | 1863 | |
| 1864 | 1864 | new_sock->type = sock->type; |
| 1865 | 1865 | new_sock->ops = sock->ops; |
| 1866 | − ret = sock->ops->accept(sock, new_sock, O_NONBLOCK); | |
| 1866 | + ret = sock->ops->accept(sock, new_sock, O_NONBLOCK, false); | |
| 1867 | 1867 | if (ret < 0) |
| 1868 | 1868 | goto out; |
| 1869 | 1869 | |
| @@ -12,7 +12,6 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/slab.h> |
| 13 | 13 | #include <linux/cred.h> |
| 14 | 14 | #include <linux/xattr.h> |
| 15 | −#include <linux/sched/signal.h> | |
| 16 | 15 | #include "overlayfs.h" |
| 17 | 16 | #include "ovl_entry.h" |
| 18 | 17 | |
| @@ -400,9 +400,9 @@ SYSCALL_DEFINE2(timerfd_create, int, clockid, int, flags) · 399 unmodified lines | ||
| 400 | 400 | clockid != CLOCK_BOOTTIME_ALARM)) |
| 401 | 401 | return -EINVAL; |
| 402 | 402 | |
| 403 | − if (!capable(CAP_WAKE_ALARM) && | |
| 404 | − (clockid == CLOCK_REALTIME_ALARM || | |
| 405 | − clockid == CLOCK_BOOTTIME_ALARM)) | |
| 403 | + if ((clockid == CLOCK_REALTIME_ALARM || | |
| 404 | + clockid == CLOCK_BOOTTIME_ALARM) && | |
| 405 | + !capable(CAP_WAKE_ALARM)) | |
| 406 | 406 | return -EPERM; |
| 407 | 407 | |
| 408 | 408 | ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); |
| @@ -449,7 +449,7 @@ static int do_timerfd_settime(int ufd, int flags, · 40 unmodified lines | ||
| 449 | 449 | return ret; |
| 450 | 450 | ctx = f.file->private_data; |
| 451 | 451 | |
| 452 | − if (!capable(CAP_WAKE_ALARM) && isalarm(ctx)) { | |
| 452 | + if (isalarm(ctx) && !capable(CAP_WAKE_ALARM)) { | |
| 453 | 453 | fdput(f); |
| 454 | 454 | return -EPERM; |
| 455 | 455 | } |
| @@ -138,8 +138,6 @@ out: · 137 unmodified lines | ||
| 138 | 138 | * userfaultfd_ctx_get - Acquires a reference to the internal userfaultfd |
| 139 | 139 | * context. |
| 140 | 140 | * @ctx: [in] Pointer to the userfaultfd context. |
| 141 | − * | |
| 142 | − * Returns: In case of success, returns not zero. | |
| 143 | 141 | */ |
| 144 | 142 | static void userfaultfd_ctx_get(struct userfaultfd_ctx *ctx) |
| 145 | 143 | { |
| @@ -267,6 +265,7 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, · 121 unmodified lines | ||
| 267 | 265 | { |
| 268 | 266 | struct mm_struct *mm = ctx->mm; |
| 269 | 267 | pgd_t *pgd; |
| 268 | + p4d_t *p4d; | |
| 270 | 269 | pud_t *pud; |
| 271 | 270 | pmd_t *pmd, _pmd; |
| 272 | 271 | pte_t *pte; |
| @@ -277,7 +276,10 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, · 4 unmodified lines | ||
| 277 | 276 | pgd = pgd_offset(mm, address); |
| 278 | 277 | if (!pgd_present(*pgd)) |
| 279 | 278 | goto out; |
| 280 | − pud = pud_offset(pgd, address); | |
| 279 | + p4d = p4d_offset(pgd, address); | |
| 280 | + if (!p4d_present(*p4d)) | |
| 281 | + goto out; | |
| 282 | + pud = pud_offset(p4d, address); | |
| 281 | 283 | if (!pud_present(*pud)) |
| 282 | 284 | goto out; |
| 283 | 285 | pmd = pmd_offset(pud, address); |
| @@ -490,7 +492,7 @@ int handle_userfault(struct vm_fault *vmf, unsigned long reason) · 206 unmodified lines | ||
| 490 | 492 | * in such case. |
| 491 | 493 | */ |
| 492 | 494 | down_read(&mm->mmap_sem); |
| 493 | − ret = 0; | |
| 495 | + ret = VM_FAULT_NOPAGE; | |
| 494 | 496 | } |
| 495 | 497 | } |
| 496 | 498 | |
| @@ -527,10 +529,11 @@ out: · 30 unmodified lines | ||
| 527 | 529 | return ret; |
| 528 | 530 | } |
| 529 | 531 | |
| 530 | −static int userfaultfd_event_wait_completion(struct userfaultfd_ctx *ctx, | |
| 531 | − struct userfaultfd_wait_queue *ewq) | |
| 532 | +static void userfaultfd_event_wait_completion(struct userfaultfd_ctx *ctx, | |
| 533 | + struct userfaultfd_wait_queue *ewq) | |
| 532 | 534 | { |
| 533 | − int ret = 0; | |
| 535 | + if (WARN_ON_ONCE(current->flags & PF_EXITING)) | |
| 536 | + goto out; | |
| 534 | 537 | |
| 535 | 538 | ewq->ctx = ctx; |
| 536 | 539 | init_waitqueue_entry(&ewq->wq, current); |
| @@ -547,8 +550,16 @@ static int userfaultfd_event_wait_completion(struct userfaultfd_ctx *ctx, · 10 unmodified lines | ||
| 547 | 550 | break; |
| 548 | 551 | if (ACCESS_ONCE(ctx->released) || |
| 549 | 552 | fatal_signal_pending(current)) { |
| 550 | − ret = -1; | |
| 551 | 553 | __remove_wait_queue(&ctx->event_wqh, &ewq->wq); |
| 554 | + if (ewq->msg.event == UFFD_EVENT_FORK) { | |
| 555 | + struct userfaultfd_ctx *new; | |
| 556 | + | |
| 557 | + new = (struct userfaultfd_ctx *) | |
| 558 | + (unsigned long) | |
| 559 | + ewq->msg.arg.reserved.reserved1; | |
| 560 | + | |
| 561 | + userfaultfd_ctx_put(new); | |
| 562 | + } | |
| 552 | 563 | break; |
| 553 | 564 | } |
| 554 | 565 | |
| @@ -566,9 +577,8 @@ static int userfaultfd_event_wait_completion(struct userfaultfd_ctx *ctx, · 11 unmodified lines | ||
| 566 | 577 | * ctx may go away after this if the userfault pseudo fd is |
| 567 | 578 | * already released. |
| 568 | 579 | */ |
| 569 | − | |
| 580 | +out: | |
| 570 | 581 | userfaultfd_ctx_put(ctx); |
| 571 | − return ret; | |
| 572 | 582 | } |
| 573 | 583 | |
| 574 | 584 | static void userfaultfd_event_complete(struct userfaultfd_ctx *ctx, |
| @@ -626,7 +636,7 @@ int dup_userfaultfd(struct vm_area_struct *vma, struct list_head *fcs) · 51 unmodified lines | ||
| 626 | 636 | return 0; |
| 627 | 637 | } |
| 628 | 638 | |
| 629 | −static int dup_fctx(struct userfaultfd_fork_ctx *fctx) | |
| 639 | +static void dup_fctx(struct userfaultfd_fork_ctx *fctx) | |
| 630 | 640 | { |
| 631 | 641 | struct userfaultfd_ctx *ctx = fctx->orig; |
| 632 | 642 | struct userfaultfd_wait_queue ewq; |
| @@ -636,17 +646,15 @@ static int dup_fctx(struct userfaultfd_fork_ctx *fctx) · 3 unmodified lines | ||
| 636 | 646 | ewq.msg.event = UFFD_EVENT_FORK; |
| 637 | 647 | ewq.msg.arg.reserved.reserved1 = (unsigned long)fctx->new; |
| 638 | 648 | |
| 639 | − return userfaultfd_event_wait_completion(ctx, &ewq); | |
| 649 | + userfaultfd_event_wait_completion(ctx, &ewq); | |
| 640 | 650 | } |
| 641 | 651 | |
| 642 | 652 | void dup_userfaultfd_complete(struct list_head *fcs) |
| 643 | 653 | { |
| 644 | − int ret = 0; | |
| 645 | 654 | struct userfaultfd_fork_ctx *fctx, *n; |
| 646 | 655 | |
| 647 | 656 | list_for_each_entry_safe(fctx, n, fcs, list) { |
| 648 | − if (!ret) | |
| 649 | − ret = dup_fctx(fctx); | |
| 657 | + dup_fctx(fctx); | |
| 650 | 658 | list_del(&fctx->list); |
| 651 | 659 | kfree(fctx); |
| 652 | 660 | } |
| @@ -689,8 +697,7 @@ void mremap_userfaultfd_complete(struct vm_userfaultfd_ctx *vm_ctx, · 36 unmodified lines | ||
| 689 | 697 | userfaultfd_event_wait_completion(ctx, &ewq); |
| 690 | 698 | } |
| 691 | 699 | |
| 692 | −void userfaultfd_remove(struct vm_area_struct *vma, | |
| 693 | − struct vm_area_struct **prev, | |
| 700 | +bool userfaultfd_remove(struct vm_area_struct *vma, | |
| 694 | 701 | unsigned long start, unsigned long end) |
| 695 | 702 | { |
| 696 | 703 | struct mm_struct *mm = vma->vm_mm; |
| @@ -699,13 +706,11 @@ void userfaultfd_remove(struct vm_area_struct *vma, · 2 unmodified lines | ||
| 699 | 706 | |
| 700 | 707 | ctx = vma->vm_userfaultfd_ctx.ctx; |
| 701 | 708 | if (!ctx || !(ctx->features & UFFD_FEATURE_EVENT_REMOVE)) |
| 702 | − return; | |
| 709 | + return true; | |
| 703 | 710 | |
| 704 | 711 | userfaultfd_ctx_get(ctx); |
| 705 | 712 | up_read(&mm->mmap_sem); |
| 706 | 713 | |
| 707 | − *prev = NULL; /* We wait for ACK w/o the mmap semaphore */ | |
| 708 | − | |
| 709 | 714 | msg_init(&ewq.msg); |
| 710 | 715 | |
| 711 | 716 | ewq.msg.event = UFFD_EVENT_REMOVE; |
| @@ -714,7 +719,7 @@ void userfaultfd_remove(struct vm_area_struct *vma, · 2 unmodified lines | ||
| 714 | 719 | |
| 715 | 720 | userfaultfd_event_wait_completion(ctx, &ewq); |
| 716 | 721 | |
| 717 | − down_read(&mm->mmap_sem); | |
| 722 | + return false; | |
| 718 | 723 | } |
| 719 | 724 | |
| 720 | 725 | static bool has_unmap_ctx(struct userfaultfd_ctx *ctx, struct list_head *unmaps, |
| @@ -775,34 +780,6 @@ void userfaultfd_unmap_complete(struct mm_struct *mm, struct list_head *uf) · 54 unmodified lines | ||
| 775 | 780 | } |
| 776 | 781 | } |
| 777 | 782 | |
| 778 | −void userfaultfd_exit(struct mm_struct *mm) | |
| 779 | −{ | |
| 780 | − struct vm_area_struct *vma = mm->mmap; | |
| 781 | − | |
| 782 | − /* | |
| 783 | − * We can do the vma walk without locking because the caller | |
| 784 | − * (exit_mm) knows it now has exclusive access | |
| 785 | − */ | |
| 786 | − while (vma) { | |
| 787 | − struct userfaultfd_ctx *ctx = vma->vm_userfaultfd_ctx.ctx; | |
| 788 | − | |
| 789 | − if (ctx && (ctx->features & UFFD_FEATURE_EVENT_EXIT)) { | |
| 790 | − struct userfaultfd_wait_queue ewq; | |
| 791 | − | |
| 792 | − userfaultfd_ctx_get(ctx); | |
| 793 | − | |
| 794 | − msg_init(&ewq.msg); | |
| 795 | − ewq.msg.event = UFFD_EVENT_EXIT; | |
| 796 | − | |
| 797 | − userfaultfd_event_wait_completion(ctx, &ewq); | |
| 798 | − | |
| 799 | − ctx->features &= ~UFFD_FEATURE_EVENT_EXIT; | |
| 800 | − } | |
| 801 | − | |
| 802 | − vma = vma->vm_next; | |
| 803 | − } | |
| 804 | −} | |
| 805 | − | |
| 806 | 783 | static int userfaultfd_release(struct inode *inode, struct file *file) |
| 807 | 784 | { |
| 808 | 785 | struct userfaultfd_ctx *ctx = file->private_data; |
| @@ -25,24 +25,6 @@ · 24 unmodified lines | ||
| 25 | 25 | #include "kmem.h" |
| 26 | 26 | #include "xfs_message.h" |
| 27 | 27 | |
| 28 | −/* | |
| 29 | − * Greedy allocation. May fail and may return vmalloced memory. | |
| 30 | − */ | |
| 31 | −void * | |
| 32 | −kmem_zalloc_greedy(size_t *size, size_t minsize, size_t maxsize) | |
| 33 | −{ | |
| 34 | − void *ptr; | |
| 35 | − size_t kmsize = maxsize; | |
| 36 | − | |
| 37 | − while (!(ptr = vzalloc(kmsize))) { | |
| 38 | − if ((kmsize >>= 1) <= minsize) | |
| 39 | − kmsize = minsize; | |
| 40 | − } | |
| 41 | − if (ptr) | |
| 42 | − *size = kmsize; | |
| 43 | − return ptr; | |
| 44 | −} | |
| 45 | − | |
| 46 | 28 | void * |
| 47 | 29 | kmem_alloc(size_t size, xfs_km_flags_t flags) |
| 48 | 30 | { |
| @@ -69,8 +69,6 @@ static inline void kmem_free(const void *ptr) · 68 unmodified lines | ||
| 69 | 69 | } |
| 70 | 70 | |
| 71 | 71 | |
| 72 | −extern void *kmem_zalloc_greedy(size_t *, size_t, size_t); | |
| 73 | − | |
| 74 | 72 | static inline void * |
| 75 | 73 | kmem_zalloc(size_t size, xfs_km_flags_t flags) |
| 76 | 74 | { |
| @@ -763,8 +763,8 @@ xfs_bmap_extents_to_btree( · 762 unmodified lines | ||
| 763 | 763 | args.type = XFS_ALLOCTYPE_START_BNO; |
| 764 | 764 | args.fsbno = XFS_INO_TO_FSB(mp, ip->i_ino); |
| 765 | 765 | } else if (dfops->dop_low) { |
| 766 | −try_another_ag: | |
| 767 | 766 | args.type = XFS_ALLOCTYPE_START_BNO; |
| 767 | +try_another_ag: | |
| 768 | 768 | args.fsbno = *firstblock; |
| 769 | 769 | } else { |
| 770 | 770 | args.type = XFS_ALLOCTYPE_NEAR_BNO; |
| @@ -790,13 +790,17 @@ try_another_ag: · 19 unmodified lines | ||
| 790 | 790 | if (xfs_sb_version_hasreflink(&cur->bc_mp->m_sb) && |
| 791 | 791 | args.fsbno == NULLFSBLOCK && |
| 792 | 792 | args.type == XFS_ALLOCTYPE_NEAR_BNO) { |
| 793 | − dfops->dop_low = true; | |
| 793 | + args.type = XFS_ALLOCTYPE_FIRST_AG; | |
| 794 | 794 | goto try_another_ag; |
| 795 | 795 | } |
| 796 | + if (WARN_ON_ONCE(args.fsbno == NULLFSBLOCK)) { | |
| 797 | + xfs_iroot_realloc(ip, -1, whichfork); | |
| 798 | + xfs_btree_del_cursor(cur, XFS_BTREE_ERROR); | |
| 799 | + return -ENOSPC; | |
| 800 | + } | |
| 796 | 801 | /* |
| 797 | 802 | * Allocation can't fail, the space was reserved. |
| 798 | 803 | */ |
| 799 | − ASSERT(args.fsbno != NULLFSBLOCK); | |
| 800 | 804 | ASSERT(*firstblock == NULLFSBLOCK || |
| 801 | 805 | args.agno >= XFS_FSB_TO_AGNO(mp, *firstblock)); |
| 802 | 806 | *firstblock = cur->bc_private.b.firstblock = args.fsbno; |
| @@ -4150,6 +4154,19 @@ xfs_bmapi_read( · 3347 unmodified lines | ||
| 4150 | 4154 | return 0; |
| 4151 | 4155 | } |
| 4152 | 4156 | |
| 4157 | +/* | |
| 4158 | + * Add a delayed allocation extent to an inode. Blocks are reserved from the | |
| 4159 | + * global pool and the extent inserted into the inode in-core extent tree. | |
| 4160 | + * | |
| 4161 | + * On entry, got refers to the first extent beyond the offset of the extent to | |
| 4162 | + * allocate or eof is specified if no such extent exists. On return, got refers | |
| 4163 | + * to the extent record that was inserted to the inode fork. | |
| 4164 | + * | |
| 4165 | + * Note that the allocated extent may have been merged with contiguous extents | |
| 4166 | + * during insertion into the inode fork. Thus, got does not reflect the current | |
| 4167 | + * state of the inode fork on return. If necessary, the caller can use lastx to | |
| 4168 | + * look up the updated record in the inode fork. | |
| 4169 | + */ | |
| 4153 | 4170 | int |
| 4154 | 4171 | xfs_bmapi_reserve_delalloc( |
| 4155 | 4172 | struct xfs_inode *ip, |
| @@ -4236,13 +4253,8 @@ xfs_bmapi_reserve_delalloc( · 80 unmodified lines | ||
| 4236 | 4253 | got->br_startblock = nullstartblock(indlen); |
| 4237 | 4254 | got->br_blockcount = alen; |
| 4238 | 4255 | got->br_state = XFS_EXT_NORM; |
| 4239 | − xfs_bmap_add_extent_hole_delay(ip, whichfork, lastx, got); | |
| 4240 | 4256 | |
| 4241 | − /* | |
| 4242 | − * Update our extent pointer, given that xfs_bmap_add_extent_hole_delay | |
| 4243 | − * might have merged it into one of the neighbouring ones. | |
| 4244 | − */ | |
| 4245 | − xfs_bmbt_get_all(xfs_iext_get_ext(ifp, *lastx), got); | |
| 4257 | + xfs_bmap_add_extent_hole_delay(ip, whichfork, lastx, got); | |
| 4246 | 4258 | |
| 4247 | 4259 | /* |
| 4248 | 4260 | * Tag the inode if blocks were preallocated. Note that COW fork |
| @@ -4254,10 +4266,6 @@ xfs_bmapi_reserve_delalloc( · 5 unmodified lines | ||
| 4254 | 4266 | if (whichfork == XFS_COW_FORK && (prealloc || aoff < off || alen > len)) |
| 4255 | 4267 | xfs_inode_set_cowblocks_tag(ip); |
| 4256 | 4268 | |
| 4257 | − ASSERT(got->br_startoff <= aoff); | |
| 4258 | − ASSERT(got->br_startoff + got->br_blockcount >= aoff + alen); | |
| 4259 | − ASSERT(isnullstartblock(got->br_startblock)); | |
| 4260 | − ASSERT(got->br_state == XFS_EXT_NORM); | |
| 4261 | 4269 | return 0; |
| 4262 | 4270 | |
| 4263 | 4271 | out_unreserve_blocks: |
| @@ -447,8 +447,8 @@ xfs_bmbt_alloc_block( · 446 unmodified lines | ||
| 447 | 447 | |
| 448 | 448 | if (args.fsbno == NULLFSBLOCK) { |
| 449 | 449 | args.fsbno = be64_to_cpu(start->l); |
| 450 | −try_another_ag: | |
| 451 | 450 | args.type = XFS_ALLOCTYPE_START_BNO; |
| 451 | +try_another_ag: | |
| 452 | 452 | /* |
| 453 | 453 | * Make sure there is sufficient room left in the AG to |
| 454 | 454 | * complete a full tree split for an extent insert. If |
| @@ -488,8 +488,8 @@ try_another_ag: · 33 unmodified lines | ||
| 488 | 488 | if (xfs_sb_version_hasreflink(&cur->bc_mp->m_sb) && |
| 489 | 489 | args.fsbno == NULLFSBLOCK && |
| 490 | 490 | args.type == XFS_ALLOCTYPE_NEAR_BNO) { |
| 491 | − cur->bc_private.b.dfops->dop_low = true; | |
| 492 | 491 | args.fsbno = cur->bc_private.b.firstblock; |
| 492 | + args.type = XFS_ALLOCTYPE_FIRST_AG; | |
| 493 | 493 | goto try_another_ag; |
| 494 | 494 | } |
| 495 | 495 | |
| @@ -506,7 +506,7 @@ try_another_ag: · 10 unmodified lines | ||
| 506 | 506 | goto error0; |
| 507 | 507 | cur->bc_private.b.dfops->dop_low = true; |
| 508 | 508 | } |
| 509 | − if (args.fsbno == NULLFSBLOCK) { | |
| 509 | + if (WARN_ON_ONCE(args.fsbno == NULLFSBLOCK)) { | |
| 510 | 510 | XFS_BTREE_TRACE_CURSOR(cur, XBT_EXIT); |
| 511 | 511 | *stat = 0; |
| 512 | 512 | return 0; |
| @@ -274,54 +274,49 @@ xfs_end_io( · 273 unmodified lines | ||
| 274 | 274 | struct xfs_ioend *ioend = |
| 275 | 275 | container_of(work, struct xfs_ioend, io_work); |
| 276 | 276 | struct xfs_inode *ip = XFS_I(ioend->io_inode); |
| 277 | + xfs_off_t offset = ioend->io_offset; | |
| 278 | + size_t size = ioend->io_size; | |
| 277 | 279 | int error = ioend->io_bio->bi_error; |
| 278 | 280 | |
| 279 | 281 | /* |
| 280 | − * Set an error if the mount has shut down and proceed with end I/O | |
| 281 | − * processing so it can perform whatever cleanups are necessary. | |
| 282 | + * Just clean up the in-memory strutures if the fs has been shut down. | |
| 282 | 283 | */ |
| 283 | − if (XFS_FORCED_SHUTDOWN(ip->i_mount)) | |
| 284 | + if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { | |
| 284 | 285 | error = -EIO; |
| 286 | + goto done; | |
| 287 | + } | |
| 285 | 288 | |
| 286 | 289 | /* |
| 287 | − * For a CoW extent, we need to move the mapping from the CoW fork | |
| 288 | − * to the data fork. If instead an error happened, just dump the | |
| 289 | − * new blocks. | |
| 290 | + * Clean up any COW blocks on an I/O error. | |
| 290 | 291 | */ |
| 291 | − if (ioend->io_type == XFS_IO_COW) { | |
| 292 | − if (error) | |
| 293 | − goto done; | |
| 294 | − if (ioend->io_bio->bi_error) { | |
| 295 | − error = xfs_reflink_cancel_cow_range(ip, | |
| 296 | − ioend->io_offset, ioend->io_size); | |
| 297 | − goto done; | |
| 292 | + if (unlikely(error)) { | |
| 293 | + switch (ioend->io_type) { | |
| 294 | + case XFS_IO_COW: | |
| 295 | + xfs_reflink_cancel_cow_range(ip, offset, size, true); | |
| 296 | + break; | |
| 298 | 297 | } |
| 299 | − error = xfs_reflink_end_cow(ip, ioend->io_offset, | |
| 300 | − ioend->io_size); | |
| 301 | − if (error) | |
| 302 | − goto done; | |
| 298 | + | |
| 299 | + goto done; | |
| 303 | 300 | } |
| 304 | 301 | |
| 305 | 302 | /* |
| 306 | − * For unwritten extents we need to issue transactions to convert a | |
| 307 | − * range to normal written extens after the data I/O has finished. | |
| 308 | − * Detecting and handling completion IO errors is done individually | |
| 309 | − * for each case as different cleanup operations need to be performed | |
| 310 | − * on error. | |
| 303 | + * Success: commit the COW or unwritten blocks if needed. | |
| 311 | 304 | */ |
| 312 | − if (ioend->io_type == XFS_IO_UNWRITTEN) { | |
| 313 | − if (error) | |
| 314 | − goto done; | |
| 315 | − error = xfs_iomap_write_unwritten(ip, ioend->io_offset, | |
| 316 | − ioend->io_size); | |
| 317 | − } else if (ioend->io_append_trans) { | |
| 318 | − error = xfs_setfilesize_ioend(ioend, error); | |
| 319 | − } else { | |
| 320 | − ASSERT(!xfs_ioend_is_append(ioend) || | |
| 321 | − ioend->io_type == XFS_IO_COW); | |
| 305 | + switch (ioend->io_type) { | |
| 306 | + case XFS_IO_COW: | |
| 307 | + error = xfs_reflink_end_cow(ip, offset, size); | |
| 308 | + break; | |
| 309 | + case XFS_IO_UNWRITTEN: | |
| 310 | + error = xfs_iomap_write_unwritten(ip, offset, size); | |
| 311 | + break; | |
| 312 | + default: | |
| 313 | + ASSERT(!xfs_ioend_is_append(ioend) || ioend->io_append_trans); | |
| 314 | + break; | |
| 322 | 315 | } |
| 323 | 316 | |
| 324 | 317 | done: |
| 318 | + if (ioend->io_append_trans) | |
| 319 | + error = xfs_setfilesize_ioend(ioend, error); | |
| 325 | 320 | xfs_destroy_ioend(ioend, error); |
| 326 | 321 | } |
| 327 | 322 | |
| @@ -1608,7 +1608,7 @@ xfs_inode_free_cowblocks( · 1607 unmodified lines | ||
| 1608 | 1608 | xfs_ilock(ip, XFS_IOLOCK_EXCL); |
| 1609 | 1609 | xfs_ilock(ip, XFS_MMAPLOCK_EXCL); |
| 1610 | 1610 | |
| 1611 | − ret = xfs_reflink_cancel_cow_range(ip, 0, NULLFILEOFF); | |
| 1611 | + ret = xfs_reflink_cancel_cow_range(ip, 0, NULLFILEOFF, false); | |
| 1612 | 1612 | |
| 1613 | 1613 | xfs_iunlock(ip, XFS_MMAPLOCK_EXCL); |
| 1614 | 1614 | xfs_iunlock(ip, XFS_IOLOCK_EXCL); |
| @@ -1615,7 +1615,7 @@ xfs_itruncate_extents( · 1614 unmodified lines | ||
| 1615 | 1615 | |
| 1616 | 1616 | /* Remove all pending CoW reservations. */ |
| 1617 | 1617 | error = xfs_reflink_cancel_cow_blocks(ip, &tp, first_unmap_block, |
| 1618 | − last_block); | |
| 1618 | + last_block, true); | |
| 1619 | 1619 | if (error) |
| 1620 | 1620 | goto out; |
| 1621 | 1621 | |
| @@ -630,6 +630,11 @@ retry: · 629 unmodified lines | ||
| 630 | 630 | goto out_unlock; |
| 631 | 631 | } |
| 632 | 632 | |
| 633 | + /* | |
| 634 | + * Flag newly allocated delalloc blocks with IOMAP_F_NEW so we punch | |
| 635 | + * them out if the write happens to fail. | |
| 636 | + */ | |
| 637 | + iomap->flags = IOMAP_F_NEW; | |
| 633 | 638 | trace_xfs_iomap_alloc(ip, offset, count, 0, &got); |
| 634 | 639 | done: |
| 635 | 640 | if (isnullstartblock(got.br_startblock)) |
| @@ -1071,16 +1076,22 @@ xfs_file_iomap_end_delalloc( · 435 unmodified lines | ||
| 1071 | 1076 | struct xfs_inode *ip, |
| 1072 | 1077 | loff_t offset, |
| 1073 | 1078 | loff_t length, |
| 1074 | − ssize_t written) | |
| 1079 | + ssize_t written, | |
| 1080 | + struct iomap *iomap) | |
| 1075 | 1081 | { |
| 1076 | 1082 | struct xfs_mount *mp = ip->i_mount; |
| 1077 | 1083 | xfs_fileoff_t start_fsb; |
| 1078 | 1084 | xfs_fileoff_t end_fsb; |
| 1079 | 1085 | int error = 0; |
| 1080 | 1086 | |
| 1081 | − /* behave as if the write failed if drop writes is enabled */ | |
| 1082 | − if (xfs_mp_drop_writes(mp)) | |
| 1087 | + /* | |
| 1088 | + * Behave as if the write failed if drop writes is enabled. Set the NEW | |
| 1089 | + * flag to force delalloc cleanup. | |
| 1090 | + */ | |
| 1091 | + if (xfs_mp_drop_writes(mp)) { | |
| 1092 | + iomap->flags |= IOMAP_F_NEW; | |
| 1083 | 1093 | written = 0; |
| 1094 | + } | |
| 1084 | 1095 | |
| 1085 | 1096 | /* |
| 1086 | 1097 | * start_fsb refers to the first unused block after a short write. If |
| @@ -1094,14 +1105,14 @@ xfs_file_iomap_end_delalloc( · 7 unmodified lines | ||
| 1094 | 1105 | end_fsb = XFS_B_TO_FSB(mp, offset + length); |
| 1095 | 1106 | |
| 1096 | 1107 | /* |
| 1097 | − * Trim back delalloc blocks if we didn't manage to write the whole | |
| 1098 | − * range reserved. | |
| 1108 | + * Trim delalloc blocks if they were allocated by this write and we | |
| 1109 | + * didn't manage to write the whole range. | |
| 1099 | 1110 | * |
| 1100 | 1111 | * We don't need to care about racing delalloc as we hold i_mutex |
| 1101 | 1112 | * across the reserve/allocate/unreserve calls. If there are delalloc |
| 1102 | 1113 | * blocks in the range, they are ours. |
| 1103 | 1114 | */ |
| 1104 | − if (start_fsb < end_fsb) { | |
| 1115 | + if ((iomap->flags & IOMAP_F_NEW) && start_fsb < end_fsb) { | |
| 1105 | 1116 | truncate_pagecache_range(VFS_I(ip), XFS_FSB_TO_B(mp, start_fsb), |
| 1106 | 1117 | XFS_FSB_TO_B(mp, end_fsb) - 1); |
| 1107 | 1118 | |
| @@ -1131,7 +1142,7 @@ xfs_file_iomap_end( · 23 unmodified lines | ||
| 1131 | 1142 | { |
| 1132 | 1143 | if ((flags & IOMAP_WRITE) && iomap->type == IOMAP_DELALLOC) |
| 1133 | 1144 | return xfs_file_iomap_end_delalloc(XFS_I(inode), offset, |
| 1134 | − length, written); | |
| 1145 | + length, written, iomap); | |
| 1135 | 1146 | return 0; |
| 1136 | 1147 | } |
| 1137 | 1148 | |
| @@ -361,7 +361,6 @@ xfs_bulkstat( · 360 unmodified lines | ||
| 361 | 361 | xfs_agino_t agino; /* inode # in allocation group */ |
| 362 | 362 | xfs_agnumber_t agno; /* allocation group number */ |
| 363 | 363 | xfs_btree_cur_t *cur; /* btree cursor for ialloc btree */ |
| 364 | − size_t irbsize; /* size of irec buffer in bytes */ | |
| 365 | 364 | xfs_inobt_rec_incore_t *irbuf; /* start of irec buffer */ |
| 366 | 365 | int nirbuf; /* size of irbuf */ |
| 367 | 366 | int ubcount; /* size of user's buffer */ |
| @@ -388,11 +387,10 @@ xfs_bulkstat( · 20 unmodified lines | ||
| 388 | 387 | *ubcountp = 0; |
| 389 | 388 | *done = 0; |
| 390 | 389 | |
| 391 | − irbuf = kmem_zalloc_greedy(&irbsize, PAGE_SIZE, PAGE_SIZE * 4); | |
| 390 | + irbuf = kmem_zalloc_large(PAGE_SIZE * 4, KM_SLEEP); | |
| 392 | 391 | if (!irbuf) |
| 393 | 392 | return -ENOMEM; |
| 394 | − | |
| 395 | − nirbuf = irbsize / sizeof(*irbuf); | |
| 393 | + nirbuf = (PAGE_SIZE * 4) / sizeof(*irbuf); | |
| 396 | 394 | |
| 397 | 395 | /* |
| 398 | 396 | * Loop over the allocation groups, starting from the last |
| @@ -513,8 +513,7 @@ STATIC void · 512 unmodified lines | ||
| 513 | 513 | xfs_set_inoalignment(xfs_mount_t *mp) |
| 514 | 514 | { |
| 515 | 515 | if (xfs_sb_version_hasalign(&mp->m_sb) && |
| 516 | − mp->m_sb.sb_inoalignmt >= | |
| 517 | − XFS_B_TO_FSBT(mp, mp->m_inode_cluster_size)) | |
| 516 | + mp->m_sb.sb_inoalignmt >= xfs_icluster_size_fsb(mp)) | |
| 518 | 517 | mp->m_inoalign_mask = mp->m_sb.sb_inoalignmt - 1; |
| 519 | 518 | else |
| 520 | 519 | mp->m_inoalign_mask = 0; |
| @@ -548,14 +548,18 @@ xfs_reflink_trim_irec_to_next_cow( · 547 unmodified lines | ||
| 548 | 548 | } |
| 549 | 549 | |
| 550 | 550 | /* |
| 551 | − * Cancel all pending CoW reservations for some block range of an inode. | |
| 551 | + * Cancel CoW reservations for some block range of an inode. | |
| 552 | + * | |
| 553 | + * If cancel_real is true this function cancels all COW fork extents for the | |
| 554 | + * inode; if cancel_real is false, real extents are not cleared. | |
| 552 | 555 | */ |
| 553 | 556 | int |
| 554 | 557 | xfs_reflink_cancel_cow_blocks( |
| 555 | 558 | struct xfs_inode *ip, |
| 556 | 559 | struct xfs_trans **tpp, |
| 557 | 560 | xfs_fileoff_t offset_fsb, |
| 558 | − xfs_fileoff_t end_fsb) | |
| 561 | + xfs_fileoff_t end_fsb, | |
| 562 | + bool cancel_real) | |
| 559 | 563 | { |
| 560 | 564 | struct xfs_ifork *ifp = XFS_IFORK_PTR(ip, XFS_COW_FORK); |
| 561 | 565 | struct xfs_bmbt_irec got, del; |
| @@ -579,7 +583,7 @@ xfs_reflink_cancel_cow_blocks( · 17 unmodified lines | ||
| 579 | 583 | &idx, &got, &del); |
| 580 | 584 | if (error) |
| 581 | 585 | break; |
| 582 | − } else { | |
| 586 | + } else if (del.br_state == XFS_EXT_UNWRITTEN || cancel_real) { | |
| 583 | 587 | xfs_trans_ijoin(*tpp, ip, 0); |
| 584 | 588 | xfs_defer_init(&dfops, &firstfsb); |
| 585 | 589 | |
| @@ -621,13 +625,17 @@ xfs_reflink_cancel_cow_blocks( · 35 unmodified lines | ||
| 621 | 625 | } |
| 622 | 626 | |
| 623 | 627 | /* |
| 624 | − * Cancel all pending CoW reservations for some byte range of an inode. | |
| 628 | + * Cancel CoW reservations for some byte range of an inode. | |
| 629 | + * | |
| 630 | + * If cancel_real is true this function cancels all COW fork extents for the | |
| 631 | + * inode; if cancel_real is false, real extents are not cleared. | |
| 625 | 632 | */ |
| 626 | 633 | int |
| 627 | 634 | xfs_reflink_cancel_cow_range( |
| 628 | 635 | struct xfs_inode *ip, |
| 629 | 636 | xfs_off_t offset, |
| 630 | − xfs_off_t count) | |
| 637 | + xfs_off_t count, | |
| 638 | + bool cancel_real) | |
| 631 | 639 | { |
| 632 | 640 | struct xfs_trans *tp; |
| 633 | 641 | xfs_fileoff_t offset_fsb; |
| @@ -653,7 +661,8 @@ xfs_reflink_cancel_cow_range( · 19 unmodified lines | ||
| 653 | 661 | xfs_trans_ijoin(tp, ip, 0); |
| 654 | 662 | |
| 655 | 663 | /* Scrape out the old CoW reservations */ |
| 656 | − error = xfs_reflink_cancel_cow_blocks(ip, &tp, offset_fsb, end_fsb); | |
| 664 | + error = xfs_reflink_cancel_cow_blocks(ip, &tp, offset_fsb, end_fsb, | |
| 665 | + cancel_real); | |
| 657 | 666 | if (error) |
| 658 | 667 | goto out_cancel; |
| 659 | 668 | |
| @@ -1450,7 +1459,7 @@ next: · 790 unmodified lines | ||
| 1450 | 1459 | * We didn't find any shared blocks so turn off the reflink flag. |
| 1451 | 1460 | * First, get rid of any leftover CoW mappings. |
| 1452 | 1461 | */ |
| 1453 | − error = xfs_reflink_cancel_cow_blocks(ip, tpp, 0, NULLFILEOFF); | |
| 1462 | + error = xfs_reflink_cancel_cow_blocks(ip, tpp, 0, NULLFILEOFF, true); | |
| 1454 | 1463 | if (error) |
| 1455 | 1464 | return error; |
| 1456 | 1465 | |
| @@ -39,9 +39,9 @@ extern void xfs_reflink_trim_irec_to_next_cow(struct xfs_inode *ip, · 38 unmodified lines | ||
| 39 | 39 | |
| 40 | 40 | extern int xfs_reflink_cancel_cow_blocks(struct xfs_inode *ip, |
| 41 | 41 | struct xfs_trans **tpp, xfs_fileoff_t offset_fsb, |
| 42 | − xfs_fileoff_t end_fsb); | |
| 42 | + xfs_fileoff_t end_fsb, bool cancel_real); | |
| 43 | 43 | extern int xfs_reflink_cancel_cow_range(struct xfs_inode *ip, xfs_off_t offset, |
| 44 | − xfs_off_t count); | |
| 44 | + xfs_off_t count, bool cancel_real); | |
| 45 | 45 | extern int xfs_reflink_end_cow(struct xfs_inode *ip, xfs_off_t offset, |
| 46 | 46 | xfs_off_t count); |
| 47 | 47 | extern int xfs_reflink_recover_cow(struct xfs_mount *mp); |
| @@ -953,7 +953,7 @@ xfs_fs_destroy_inode( · 952 unmodified lines | ||
| 953 | 953 | XFS_STATS_INC(ip->i_mount, vn_remove); |
| 954 | 954 | |
| 955 | 955 | if (xfs_is_reflink_inode(ip)) { |
| 956 | − error = xfs_reflink_cancel_cow_range(ip, 0, NULLFILEOFF); | |
| 956 | + error = xfs_reflink_cancel_cow_range(ip, 0, NULLFILEOFF, true); | |
| 957 | 957 | if (error && !XFS_FORCED_SHUTDOWN(ip->i_mount)) |
| 958 | 958 | xfs_warn(ip->i_mount, |
| 959 | 959 | "Error %d while evicting CoW blocks for inode %llu.", |
| @@ -15,7 +15,6 @@ · 14 unmodified lines | ||
| 15 | 15 | ((unlikely(pgd_none(*(pud))) && __pmd_alloc(mm, pud, address))? \ |
| 16 | 16 | NULL: pmd_offset(pud, address)) |
| 17 | 17 | |
| 18 | −#define pud_alloc(mm, pgd, address) (pgd) | |
| 19 | 18 | #define pud_offset(pgd, start) (pgd) |
| 20 | 19 | #define pud_none(pud) 0 |
| 21 | 20 | #define pud_bad(pud) 0 |
| @@ -35,4 +34,6 @@ · 13 unmodified lines | ||
| 35 | 34 | #undef pud_addr_end |
| 36 | 35 | #define pud_addr_end(addr, end) (end) |
| 37 | 36 | |
| 37 | +#include <asm-generic/5level-fixup.h> | |
| 38 | + | |
| 38 | 39 | #endif |
| @@ -0,0 +1,41 @@ | ||
| 1 | +#ifndef _5LEVEL_FIXUP_H | |
| 2 | +#define _5LEVEL_FIXUP_H | |
| 3 | + | |
| 4 | +#define __ARCH_HAS_5LEVEL_HACK | |
| 5 | +#define __PAGETABLE_P4D_FOLDED | |
| 6 | + | |
| 7 | +#define P4D_SHIFT PGDIR_SHIFT | |
| 8 | +#define P4D_SIZE PGDIR_SIZE | |
| 9 | +#define P4D_MASK PGDIR_MASK | |
| 10 | +#define PTRS_PER_P4D 1 | |
| 11 | + | |
| 12 | +#define p4d_t pgd_t | |
| 13 | + | |
| 14 | +#define pud_alloc(mm, p4d, address) \ | |
| 15 | + ((unlikely(pgd_none(*(p4d))) && __pud_alloc(mm, p4d, address)) ? \ | |
| 16 | + NULL : pud_offset(p4d, address)) | |
| 17 | + | |
| 18 | +#define p4d_alloc(mm, pgd, address) (pgd) | |
| 19 | +#define p4d_offset(pgd, start) (pgd) | |
| 20 | +#define p4d_none(p4d) 0 | |
| 21 | +#define p4d_bad(p4d) 0 | |
| 22 | +#define p4d_present(p4d) 1 | |
| 23 | +#define p4d_ERROR(p4d) do { } while (0) | |
| 24 | +#define p4d_clear(p4d) pgd_clear(p4d) | |
| 25 | +#define p4d_val(p4d) pgd_val(p4d) | |
| 26 | +#define p4d_populate(mm, p4d, pud) pgd_populate(mm, p4d, pud) | |
| 27 | +#define p4d_page(p4d) pgd_page(p4d) | |
| 28 | +#define p4d_page_vaddr(p4d) pgd_page_vaddr(p4d) | |
| 29 | + | |
| 30 | +#define __p4d(x) __pgd(x) | |
| 31 | +#define set_p4d(p4dp, p4d) set_pgd(p4dp, p4d) | |
| 32 | + | |
| 33 | +#undef p4d_free_tlb | |
| 34 | +#define p4d_free_tlb(tlb, x, addr) do { } while (0) | |
| 35 | +#define p4d_free(mm, x) do { } while (0) | |
| 36 | +#define __p4d_free_tlb(tlb, x, addr) do { } while (0) | |
| 37 | + | |
| 38 | +#undef p4d_addr_end | |
| 39 | +#define p4d_addr_end(addr, end) (end) | |
| 40 | + | |
| 41 | +#endif | |
| @@ -0,0 +1,62 @@ | ||
| 1 | +#ifndef _PGTABLE_NOP4D_HACK_H | |
| 2 | +#define _PGTABLE_NOP4D_HACK_H | |
| 3 | + | |
| 4 | +#ifndef __ASSEMBLY__ | |
| 5 | +#include <asm-generic/5level-fixup.h> | |
| 6 | + | |
| 7 | +#define __PAGETABLE_PUD_FOLDED | |
| 8 | + | |
| 9 | +/* | |
| 10 | + * Having the pud type consist of a pgd gets the size right, and allows | |
| 11 | + * us to conceptually access the pgd entry that this pud is folded into | |
| 12 | + * without casting. | |
| 13 | + */ | |
| 14 | +typedef struct { pgd_t pgd; } pud_t; | |
| 15 | + | |
| 16 | +#define PUD_SHIFT PGDIR_SHIFT | |
| 17 | +#define PTRS_PER_PUD 1 | |
| 18 | +#define PUD_SIZE (1UL << PUD_SHIFT) | |
| 19 | +#define PUD_MASK (~(PUD_SIZE-1)) | |
| 20 | + | |
| 21 | +/* | |
| 22 | + * The "pgd_xxx()" functions here are trivial for a folded two-level | |
| 23 | + * setup: the pud is never bad, and a pud always exists (as it's folded | |
| 24 | + * into the pgd entry) | |
| 25 | + */ | |
| 26 | +static inline int pgd_none(pgd_t pgd) { return 0; } | |
| 27 | +static inline int pgd_bad(pgd_t pgd) { return 0; } | |
| 28 | +static inline int pgd_present(pgd_t pgd) { return 1; } | |
| 29 | +static inline void pgd_clear(pgd_t *pgd) { } | |
| 30 | +#define pud_ERROR(pud) (pgd_ERROR((pud).pgd)) | |
| 31 | + | |
| 32 | +#define pgd_populate(mm, pgd, pud) do { } while (0) | |
| 33 | +/* | |
| 34 | + * (puds are folded into pgds so this doesn't get actually called, | |
| 35 | + * but the define is needed for a generic inline function.) | |
| 36 | + */ | |
| 37 | +#define set_pgd(pgdptr, pgdval) set_pud((pud_t *)(pgdptr), (pud_t) { pgdval }) | |
| 38 | + | |
| 39 | +static inline pud_t *pud_offset(pgd_t *pgd, unsigned long address) | |
| 40 | +{ | |
| 41 | + return (pud_t *)pgd; | |
| 42 | +} | |
| 43 | + | |
| 44 | +#define pud_val(x) (pgd_val((x).pgd)) | |
| 45 | +#define __pud(x) ((pud_t) { __pgd(x) }) | |
| 46 | + | |
| 47 | +#define pgd_page(pgd) (pud_page((pud_t){ pgd })) | |
| 48 | +#define pgd_page_vaddr(pgd) (pud_page_vaddr((pud_t){ pgd })) | |
| 49 | + | |
| 50 | +/* | |
| 51 | + * allocating and freeing a pud is trivial: the 1-entry pud is | |
| 52 | + * inside the pgd, so has no extra memory associated with it. | |
| 53 | + */ | |
| 54 | +#define pud_alloc_one(mm, address) NULL | |
| 55 | +#define pud_free(mm, x) do { } while (0) | |
| 56 | +#define __pud_free_tlb(tlb, x, a) do { } while (0) | |
| 57 | + | |
| 58 | +#undef pud_addr_end | |
| 59 | +#define pud_addr_end(addr, end) (end) | |
| 60 | + | |
| 61 | +#endif /* __ASSEMBLY__ */ | |
| 62 | +#endif /* _PGTABLE_NOP4D_HACK_H */ | |
| @@ -0,0 +1,56 @@ | ||
| 1 | +#ifndef _PGTABLE_NOP4D_H | |
| 2 | +#define _PGTABLE_NOP4D_H | |
| 3 | + | |
| 4 | +#ifndef __ASSEMBLY__ | |
| 5 | + | |
| 6 | +#define __PAGETABLE_P4D_FOLDED | |
| 7 | + | |
| 8 | +typedef struct { pgd_t pgd; } p4d_t; | |
| 9 | + | |
| 10 | +#define P4D_SHIFT PGDIR_SHIFT | |
| 11 | +#define PTRS_PER_P4D 1 | |
| 12 | +#define P4D_SIZE (1UL << P4D_SHIFT) | |
| 13 | +#define P4D_MASK (~(P4D_SIZE-1)) | |
| 14 | + | |
| 15 | +/* | |
| 16 | + * The "pgd_xxx()" functions here are trivial for a folded two-level | |
| 17 | + * setup: the p4d is never bad, and a p4d always exists (as it's folded | |
| 18 | + * into the pgd entry) | |
| 19 | + */ | |
| 20 | +static inline int pgd_none(pgd_t pgd) { return 0; } | |
| 21 | +static inline int pgd_bad(pgd_t pgd) { return 0; } | |
| 22 | +static inline int pgd_present(pgd_t pgd) { return 1; } | |
| 23 | +static inline void pgd_clear(pgd_t *pgd) { } | |
| 24 | +#define p4d_ERROR(p4d) (pgd_ERROR((p4d).pgd)) | |
| 25 | + | |
| 26 | +#define pgd_populate(mm, pgd, p4d) do { } while (0) | |
| 27 | +/* | |
| 28 | + * (p4ds are folded into pgds so this doesn't get actually called, | |
| 29 | + * but the define is needed for a generic inline function.) | |
| 30 | + */ | |
| 31 | +#define set_pgd(pgdptr, pgdval) set_p4d((p4d_t *)(pgdptr), (p4d_t) { pgdval }) | |
| 32 | + | |
| 33 | +static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address) | |
| 34 | +{ | |
| 35 | + return (p4d_t *)pgd; | |
| 36 | +} | |
| 37 | + | |
| 38 | +#define p4d_val(x) (pgd_val((x).pgd)) | |
| 39 | +#define __p4d(x) ((p4d_t) { __pgd(x) }) | |
| 40 | + | |
| 41 | +#define pgd_page(pgd) (p4d_page((p4d_t){ pgd })) | |
| 42 | +#define pgd_page_vaddr(pgd) (p4d_page_vaddr((p4d_t){ pgd })) | |
| 43 | + | |
| 44 | +/* | |
| 45 | + * allocating and freeing a p4d is trivial: the 1-entry p4d is | |
| 46 | + * inside the pgd, so has no extra memory associated with it. | |
| 47 | + */ | |
| 48 | +#define p4d_alloc_one(mm, address) NULL | |
| 49 | +#define p4d_free(mm, x) do { } while (0) | |
| 50 | +#define __p4d_free_tlb(tlb, x, a) do { } while (0) | |
| 51 | + | |
| 52 | +#undef p4d_addr_end | |
| 53 | +#define p4d_addr_end(addr, end) (end) | |
| 54 | + | |
| 55 | +#endif /* __ASSEMBLY__ */ | |
| 56 | +#endif /* _PGTABLE_NOP4D_H */ | |
| @@ -3,52 +3,57 @@ · 2 unmodified lines | ||
| 3 | 3 | |
| 4 | 4 | #ifndef __ASSEMBLY__ |
| 5 | 5 | |
| 6 | +#ifdef __ARCH_USE_5LEVEL_HACK | |
| 7 | +#include <asm-generic/pgtable-nop4d-hack.h> | |
| 8 | +#else | |
| 9 | +#include <asm-generic/pgtable-nop4d.h> | |
| 10 | + | |
| 6 | 11 | #define __PAGETABLE_PUD_FOLDED |
| 7 | 12 | |
| 8 | 13 | /* |
| 9 | − * Having the pud type consist of a pgd gets the size right, and allows | |
| 10 | − * us to conceptually access the pgd entry that this pud is folded into | |
| 14 | + * Having the pud type consist of a p4d gets the size right, and allows | |
| 15 | + * us to conceptually access the p4d entry that this pud is folded into | |
| 11 | 16 | * without casting. |
| 12 | 17 | */ |
| 13 | −typedef struct { pgd_t pgd; } pud_t; | |
| 18 | +typedef struct { p4d_t p4d; } pud_t; | |
| 14 | 19 | |
| 15 | −#define PUD_SHIFT PGDIR_SHIFT | |
| 20 | +#define PUD_SHIFT P4D_SHIFT | |
| 16 | 21 | #define PTRS_PER_PUD 1 |
| 17 | 22 | #define PUD_SIZE (1UL << PUD_SHIFT) |
| 18 | 23 | #define PUD_MASK (~(PUD_SIZE-1)) |
| 19 | 24 | |
| 20 | 25 | /* |
| 21 | − * The "pgd_xxx()" functions here are trivial for a folded two-level | |
| 26 | + * The "p4d_xxx()" functions here are trivial for a folded two-level | |
| 22 | 27 | * setup: the pud is never bad, and a pud always exists (as it's folded |
| 23 | − * into the pgd entry) | |
| 28 | + * into the p4d entry) | |
| 24 | 29 | */ |
| 25 | −static inline int pgd_none(pgd_t pgd) { return 0; } | |
| 26 | −static inline int pgd_bad(pgd_t pgd) { return 0; } | |
| 27 | −static inline int pgd_present(pgd_t pgd) { return 1; } | |
| 28 | −static inline void pgd_clear(pgd_t *pgd) { } | |
| 29 | −#define pud_ERROR(pud) (pgd_ERROR((pud).pgd)) | |
| 30 | +static inline int p4d_none(p4d_t p4d) { return 0; } | |
| 31 | +static inline int p4d_bad(p4d_t p4d) { return 0; } | |
| 32 | +static inline int p4d_present(p4d_t p4d) { return 1; } | |
| 33 | +static inline void p4d_clear(p4d_t *p4d) { } | |
| 34 | +#define pud_ERROR(pud) (p4d_ERROR((pud).p4d)) | |
| 30 | 35 | |
| 31 | −#define pgd_populate(mm, pgd, pud) do { } while (0) | |
| 36 | +#define p4d_populate(mm, p4d, pud) do { } while (0) | |
| 32 | 37 | /* |
| 33 | − * (puds are folded into pgds so this doesn't get actually called, | |
| 38 | + * (puds are folded into p4ds so this doesn't get actually called, | |
| 34 | 39 | * but the define is needed for a generic inline function.) |
| 35 | 40 | */ |
| 36 | −#define set_pgd(pgdptr, pgdval) set_pud((pud_t *)(pgdptr), (pud_t) { pgdval }) | |
| 41 | +#define set_p4d(p4dptr, p4dval) set_pud((pud_t *)(p4dptr), (pud_t) { p4dval }) | |
| 37 | 42 | |
| 38 | −static inline pud_t * pud_offset(pgd_t * pgd, unsigned long address) | |
| 43 | +static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address) | |
| 39 | 44 | { |
| 40 | − return (pud_t *)pgd; | |
| 45 | + return (pud_t *)p4d; | |
| 41 | 46 | } |
| 42 | 47 | |
| 43 | −#define pud_val(x) (pgd_val((x).pgd)) | |
| 44 | −#define __pud(x) ((pud_t) { __pgd(x) } ) | |
| 48 | +#define pud_val(x) (p4d_val((x).p4d)) | |
| 49 | +#define __pud(x) ((pud_t) { __p4d(x) }) | |
| 45 | 50 | |
| 46 | −#define pgd_page(pgd) (pud_page((pud_t){ pgd })) | |
| 47 | −#define pgd_page_vaddr(pgd) (pud_page_vaddr((pud_t){ pgd })) | |
| 51 | +#define p4d_page(p4d) (pud_page((pud_t){ p4d })) | |
| 52 | +#define p4d_page_vaddr(p4d) (pud_page_vaddr((pud_t){ p4d })) | |
| 48 | 53 | |
| 49 | 54 | /* |
| 50 | 55 | * allocating and freeing a pud is trivial: the 1-entry pud is |
| 51 | − * inside the pgd, so has no extra memory associated with it. | |
| 56 | + * inside the p4d, so has no extra memory associated with it. | |
| 52 | 57 | */ |
| 53 | 58 | #define pud_alloc_one(mm, address) NULL |
| 54 | 59 | #define pud_free(mm, x) do { } while (0) |
| @@ -58,4 +63,5 @@ static inline pud_t * pud_offset(pgd_t * pgd, unsigned long address) · 3 unmodified lines | ||
| 58 | 63 | #define pud_addr_end(addr, end) (end) |
| 59 | 64 | |
| 60 | 65 | #endif /* __ASSEMBLY__ */ |
| 66 | +#endif /* !__ARCH_USE_5LEVEL_HACK */ | |
| 61 | 67 | #endif /* _PGTABLE_NOPUD_H */ |
| @@ -10,9 +10,9 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/bug.h> |
| 11 | 11 | #include <linux/errno.h> |
| 12 | 12 | |
| 13 | −#if 4 - defined(__PAGETABLE_PUD_FOLDED) - defined(__PAGETABLE_PMD_FOLDED) != \ | |
| 14 | − CONFIG_PGTABLE_LEVELS | |
| 15 | −#error CONFIG_PGTABLE_LEVELS is not consistent with __PAGETABLE_{PUD,PMD}_FOLDED | |
| 13 | +#if 5 - defined(__PAGETABLE_P4D_FOLDED) - defined(__PAGETABLE_PUD_FOLDED) - \ | |
| 14 | + defined(__PAGETABLE_PMD_FOLDED) != CONFIG_PGTABLE_LEVELS | |
| 15 | +#error CONFIG_PGTABLE_LEVELS is not consistent with __PAGETABLE_{P4D,PUD,PMD}_FOLDED | |
| 16 | 16 | #endif |
| 17 | 17 | |
| 18 | 18 | /* |
| @@ -424,6 +424,13 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot) · 405 unmodified lines | ||
| 424 | 424 | (__boundary - 1 < (end) - 1)? __boundary: (end); \ |
| 425 | 425 | }) |
| 426 | 426 | |
| 427 | +#ifndef p4d_addr_end | |
| 428 | +#define p4d_addr_end(addr, end) \ | |
| 429 | +({ unsigned long __boundary = ((addr) + P4D_SIZE) & P4D_MASK; \ | |
| 430 | + (__boundary - 1 < (end) - 1)? __boundary: (end); \ | |
| 431 | +}) | |
| 432 | +#endif | |
| 433 | + | |
| 427 | 434 | #ifndef pud_addr_end |
| 428 | 435 | #define pud_addr_end(addr, end) \ |
| 429 | 436 | ({ unsigned long __boundary = ((addr) + PUD_SIZE) & PUD_MASK; \ |
| @@ -444,6 +451,7 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot) · 14 unmodified lines | ||
| 444 | 451 | * Do the tests inline, but report and clear the bad entry in mm/memory.c. |
| 445 | 452 | */ |
| 446 | 453 | void pgd_clear_bad(pgd_t *); |
| 454 | +void p4d_clear_bad(p4d_t *); | |
| 447 | 455 | void pud_clear_bad(pud_t *); |
| 448 | 456 | void pmd_clear_bad(pmd_t *); |
| 449 | 457 | |
| @@ -458,6 +466,17 @@ static inline int pgd_none_or_clear_bad(pgd_t *pgd) · 8 unmodified lines | ||
| 458 | 466 | return 0; |
| 459 | 467 | } |
| 460 | 468 | |
| 469 | +static inline int p4d_none_or_clear_bad(p4d_t *p4d) | |
| 470 | +{ | |
| 471 | + if (p4d_none(*p4d)) | |
| 472 | + return 1; | |
| 473 | + if (unlikely(p4d_bad(*p4d))) { | |
| 474 | + p4d_clear_bad(p4d); | |
| 475 | + return 1; | |
| 476 | + } | |
| 477 | + return 0; | |
| 478 | +} | |
| 479 | + | |
| 461 | 480 | static inline int pud_none_or_clear_bad(pud_t *pud) |
| 462 | 481 | { |
| 463 | 482 | if (pud_none(*pud)) |
| @@ -844,11 +863,30 @@ static inline int pmd_protnone(pmd_t pmd) · 380 unmodified lines | ||
| 844 | 863 | #endif /* CONFIG_MMU */ |
| 845 | 864 | |
| 846 | 865 | #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP |
| 866 | + | |
| 867 | +#ifndef __PAGETABLE_P4D_FOLDED | |
| 868 | +int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot); | |
| 869 | +int p4d_clear_huge(p4d_t *p4d); | |
| 870 | +#else | |
| 871 | +static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot) | |
| 872 | +{ | |
| 873 | + return 0; | |
| 874 | +} | |
| 875 | +static inline int p4d_clear_huge(p4d_t *p4d) | |
| 876 | +{ | |
| 877 | + return 0; | |
| 878 | +} | |
| 879 | +#endif /* !__PAGETABLE_P4D_FOLDED */ | |
| 880 | + | |
| 847 | 881 | int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); |
| 848 | 882 | int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); |
| 849 | 883 | int pud_clear_huge(pud_t *pud); |
| 850 | 884 | int pmd_clear_huge(pmd_t *pmd); |
| 851 | 885 | #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ |
| 886 | +static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot) | |
| 887 | +{ | |
| 888 | + return 0; | |
| 889 | +} | |
| 852 | 890 | static inline int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) |
| 853 | 891 | { |
| 854 | 892 | return 0; |
| @@ -857,6 +895,10 @@ static inline int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) · 2 unmodified lines | ||
| 857 | 895 | { |
| 858 | 896 | return 0; |
| 859 | 897 | } |
| 898 | +static inline int p4d_clear_huge(p4d_t *p4d) | |
| 899 | +{ | |
| 900 | + return 0; | |
| 901 | +} | |
| 860 | 902 | static inline int pud_clear_huge(pud_t *pud) |
| 861 | 903 | { |
| 862 | 904 | return 0; |
| @@ -270,6 +270,12 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb, · 269 unmodified lines | ||
| 270 | 270 | __pte_free_tlb(tlb, ptep, address); \ |
| 271 | 271 | } while (0) |
| 272 | 272 | |
| 273 | +#define pmd_free_tlb(tlb, pmdp, address) \ | |
| 274 | + do { \ | |
| 275 | + __tlb_adjust_range(tlb, address, PAGE_SIZE); \ | |
| 276 | + __pmd_free_tlb(tlb, pmdp, address); \ | |
| 277 | + } while (0) | |
| 278 | + | |
| 273 | 279 | #ifndef __ARCH_HAS_4LEVEL_HACK |
| 274 | 280 | #define pud_free_tlb(tlb, pudp, address) \ |
| 275 | 281 | do { \ |
| @@ -278,11 +284,13 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb, · 2 unmodified lines | ||
| 278 | 284 | } while (0) |
| 279 | 285 | #endif |
| 280 | 286 | |
| 281 | −#define pmd_free_tlb(tlb, pmdp, address) \ | |
| 287 | +#ifndef __ARCH_HAS_5LEVEL_HACK | |
| 288 | +#define p4d_free_tlb(tlb, pudp, address) \ | |
| 282 | 289 | do { \ |
| 283 | − __tlb_adjust_range(tlb, address, PAGE_SIZE); \ | |
| 284 | − __pmd_free_tlb(tlb, pmdp, address); \ | |
| 290 | + __tlb_adjust_range(tlb, address, PAGE_SIZE); \ | |
| 291 | + __p4d_free_tlb(tlb, pudp, address); \ | |
| 285 | 292 | } while (0) |
| 293 | +#endif | |
| 286 | 294 | |
| 287 | 295 | #define tlb_migrate_finish(mm) do {} while (0) |
| 288 | 296 | |
| @@ -73,7 +73,7 @@ int af_alg_unregister_type(const struct af_alg_type *type); · 72 unmodified lines | ||
| 73 | 73 | |
| 74 | 74 | int af_alg_release(struct socket *sock); |
| 75 | 75 | void af_alg_release_parent(struct sock *sk); |
| 76 | −int af_alg_accept(struct sock *sk, struct socket *newsock); | |
| 76 | +int af_alg_accept(struct sock *sk, struct socket *newsock, bool kern); | |
| 77 | 77 | |
| 78 | 78 | int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len); |
| 79 | 79 | void af_alg_free_sg(struct af_alg_sgl *sgl); |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | #define CS42L42_HPOUT_LOAD_1NF 0 |
| 21 | 21 | #define CS42L42_HPOUT_LOAD_10NF 1 |
| 22 | 22 | |
| 23 | −/* HPOUT Clamp to GND Overide */ | |
| 23 | +/* HPOUT Clamp to GND Override */ | |
| 24 | 24 | #define CS42L42_HPOUT_CLAMP_EN 0 |
| 25 | 25 | #define CS42L42_HPOUT_CLAMP_DIS 1 |
| 26 | 26 | |
| @@ -435,7 +435,6 @@ struct request_queue { · 434 unmodified lines | ||
| 435 | 435 | struct delayed_work delay_work; |
| 436 | 436 | |
| 437 | 437 | struct backing_dev_info *backing_dev_info; |
| 438 | − struct disk_devt *disk_devt; | |
| 439 | 438 | |
| 440 | 439 | /* |
| 441 | 440 | * The queue owner gets to use this for whatever they like. |
| @@ -48,6 +48,7 @@ struct ceph_options { · 47 unmodified lines | ||
| 48 | 48 | unsigned long mount_timeout; /* jiffies */ |
| 49 | 49 | unsigned long osd_idle_ttl; /* jiffies */ |
| 50 | 50 | unsigned long osd_keepalive_timeout; /* jiffies */ |
| 51 | + unsigned long osd_request_timeout; /* jiffies */ | |
| 51 | 52 | |
| 52 | 53 | /* |
| 53 | 54 | * any type that can't be simply compared or doesn't need need |
| @@ -68,6 +69,7 @@ struct ceph_options { · 14 unmodified lines | ||
| 68 | 69 | #define CEPH_MOUNT_TIMEOUT_DEFAULT msecs_to_jiffies(60 * 1000) |
| 69 | 70 | #define CEPH_OSD_KEEPALIVE_DEFAULT msecs_to_jiffies(5 * 1000) |
| 70 | 71 | #define CEPH_OSD_IDLE_TTL_DEFAULT msecs_to_jiffies(60 * 1000) |
| 72 | +#define CEPH_OSD_REQUEST_TIMEOUT_DEFAULT 0 /* no timeout */ | |
| 71 | 73 | |
| 72 | 74 | #define CEPH_MONC_HUNT_INTERVAL msecs_to_jiffies(3 * 1000) |
| 73 | 75 | #define CEPH_MONC_PING_INTERVAL msecs_to_jiffies(10 * 1000) |
| @@ -189,6 +189,7 @@ struct ceph_osd_request { · 188 unmodified lines | ||
| 189 | 189 | |
| 190 | 190 | /* internal */ |
| 191 | 191 | unsigned long r_stamp; /* jiffies, send or check time */ |
| 192 | + unsigned long r_start_stamp; /* jiffies */ | |
| 192 | 193 | int r_attempts; |
| 193 | 194 | struct ceph_eversion r_replay_version; /* aka reassert_version */ |
| 194 | 195 | u32 r_last_force_resend; |
| @@ -163,6 +163,7 @@ struct dccp_request_sock { · 162 unmodified lines | ||
| 163 | 163 | __u64 dreq_isr; |
| 164 | 164 | __u64 dreq_gsr; |
| 165 | 165 | __be32 dreq_service; |
| 166 | + spinlock_t dreq_lock; | |
| 166 | 167 | struct list_head dreq_featneg; |
| 167 | 168 | __u32 dreq_timestamp_echo; |
| 168 | 169 | __u32 dreq_timestamp_time; |
| @@ -409,6 +409,7 @@ struct bpf_prog { · 408 unmodified lines | ||
| 409 | 409 | u16 pages; /* Number of allocated pages */ |
| 410 | 410 | kmemcheck_bitfield_begin(meta); |
| 411 | 411 | u16 jited:1, /* Is our filter JIT'ed? */ |
| 412 | + locked:1, /* Program image locked? */ | |
| 412 | 413 | gpl_compatible:1, /* Is filter GPL compatible? */ |
| 413 | 414 | cb_access:1, /* Is control block accessed? */ |
| 414 | 415 | dst_needed:1, /* Do we need dst entry? */ |
| @@ -554,22 +555,29 @@ static inline bool bpf_prog_was_classic(const struct bpf_prog *prog) · 139 unmodified lines | ||
| 554 | 555 | #ifdef CONFIG_ARCH_HAS_SET_MEMORY |
| 555 | 556 | static inline void bpf_prog_lock_ro(struct bpf_prog *fp) |
| 556 | 557 | { |
| 557 | − set_memory_ro((unsigned long)fp, fp->pages); | |
| 558 | + fp->locked = 1; | |
| 559 | + WARN_ON_ONCE(set_memory_ro((unsigned long)fp, fp->pages)); | |
| 558 | 560 | } |
| 559 | 561 | |
| 560 | 562 | static inline void bpf_prog_unlock_ro(struct bpf_prog *fp) |
| 561 | 563 | { |
| 562 | − set_memory_rw((unsigned long)fp, fp->pages); | |
| 564 | + if (fp->locked) { | |
| 565 | + WARN_ON_ONCE(set_memory_rw((unsigned long)fp, fp->pages)); | |
| 566 | + /* In case set_memory_rw() fails, we want to be the first | |
| 567 | + * to crash here instead of some random place later on. | |
| 568 | + */ | |
| 569 | + fp->locked = 0; | |
| 570 | + } | |
| 563 | 571 | } |
| 564 | 572 | |
| 565 | 573 | static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr) |
| 566 | 574 | { |
| 567 | − set_memory_ro((unsigned long)hdr, hdr->pages); | |
| 575 | + WARN_ON_ONCE(set_memory_ro((unsigned long)hdr, hdr->pages)); | |
| 568 | 576 | } |
| 569 | 577 | |
| 570 | 578 | static inline void bpf_jit_binary_unlock_ro(struct bpf_binary_header *hdr) |
| 571 | 579 | { |
| 572 | − set_memory_rw((unsigned long)hdr, hdr->pages); | |
| 580 | + WARN_ON_ONCE(set_memory_rw((unsigned long)hdr, hdr->pages)); | |
| 573 | 581 | } |
| 574 | 582 | #else |
| 575 | 583 | static inline void bpf_prog_lock_ro(struct bpf_prog *fp) |
| @@ -2678,7 +2678,7 @@ static const char * const kernel_read_file_str[] = { · 2677 unmodified lines | ||
| 2678 | 2678 | |
| 2679 | 2679 | static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) |
| 2680 | 2680 | { |
| 2681 | − if (id < 0 || id >= READING_MAX_ID) | |
| 2681 | + if ((unsigned)id >= READING_MAX_ID) | |
| 2682 | 2682 | return kernel_read_file_str[READING_UNKNOWN]; |
| 2683 | 2683 | |
| 2684 | 2684 | return kernel_read_file_str[id]; |
| @@ -167,13 +167,6 @@ struct blk_integrity { · 166 unmodified lines | ||
| 167 | 167 | }; |
| 168 | 168 | |
| 169 | 169 | #endif /* CONFIG_BLK_DEV_INTEGRITY */ |
| 170 | −struct disk_devt { | |
| 171 | − atomic_t count; | |
| 172 | − void (*release)(struct disk_devt *disk_devt); | |
| 173 | −}; | |
| 174 | − | |
| 175 | −void put_disk_devt(struct disk_devt *disk_devt); | |
| 176 | −void get_disk_devt(struct disk_devt *disk_devt); | |
| 177 | 170 | |
| 178 | 171 | struct gendisk { |
| 179 | 172 | /* major, first_minor and minors are input parameters only, |
| @@ -183,7 +176,6 @@ struct gendisk { · 3 unmodified lines | ||
| 183 | 176 | int first_minor; |
| 184 | 177 | int minors; /* maximum number of minors, =1 for |
| 185 | 178 | * disks that can't be partitioned. */ |
| 186 | − struct disk_devt *disk_devt; | |
| 187 | 179 | |
| 188 | 180 | char disk_name[DISK_NAME_LEN]; /* name of major driver */ |
| 189 | 181 | char *(*devnode)(struct gendisk *gd, umode_t *mode); |
| @@ -122,7 +122,7 @@ struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address, · 121 unmodified lines | ||
| 122 | 122 | struct page *follow_huge_pud(struct mm_struct *mm, unsigned long address, |
| 123 | 123 | pud_t *pud, int flags); |
| 124 | 124 | int pmd_huge(pmd_t pmd); |
| 125 | −int pud_huge(pud_t pmd); | |
| 125 | +int pud_huge(pud_t pud); | |
| 126 | 126 | unsigned long hugetlb_change_protection(struct vm_area_struct *vma, |
| 127 | 127 | unsigned long address, unsigned long end, pgprot_t newprot); |
| 128 | 128 | |
| @@ -197,6 +197,9 @@ static inline void __unmap_hugepage_range(struct mmu_gather *tlb, · 68 unmodified lines | ||
| 197 | 197 | #ifndef pgd_huge |
| 198 | 198 | #define pgd_huge(x) 0 |
| 199 | 199 | #endif |
| 200 | +#ifndef p4d_huge | |
| 201 | +#define p4d_huge(x) 0 | |
| 202 | +#endif | |
| 200 | 203 | |
| 201 | 204 | #ifndef pgd_write |
| 202 | 205 | static inline int pgd_write(pgd_t pgd) |
| @@ -373,6 +373,8 @@ · 372 unmodified lines | ||
| 373 | 373 | #define ICC_IGRPEN0_EL1_MASK (1 << ICC_IGRPEN0_EL1_SHIFT) |
| 374 | 374 | #define ICC_IGRPEN1_EL1_SHIFT 0 |
| 375 | 375 | #define ICC_IGRPEN1_EL1_MASK (1 << ICC_IGRPEN1_EL1_SHIFT) |
| 376 | +#define ICC_SRE_EL1_DIB (1U << 2) | |
| 377 | +#define ICC_SRE_EL1_DFB (1U << 1) | |
| 376 | 378 | #define ICC_SRE_EL1_SRE (1U << 0) |
| 377 | 379 | |
| 378 | 380 | /* |
| @@ -524,6 +524,10 @@ static inline struct irq_domain *irq_find_matching_fwnode( · 523 unmodified lines | ||
| 524 | 524 | { |
| 525 | 525 | return NULL; |
| 526 | 526 | } |
| 527 | +static inline bool irq_domain_check_msi_remap(void) | |
| 528 | +{ | |
| 529 | + return false; | |
| 530 | +} | |
| 527 | 531 | #endif /* !CONFIG_IRQ_DOMAIN */ |
| 528 | 532 | |
| 529 | 533 | #endif /* _LINUX_IRQDOMAIN_H */ |
| @@ -90,6 +90,13 @@ extern bool static_key_initialized; · 89 unmodified lines | ||
| 90 | 90 | struct static_key { |
| 91 | 91 | atomic_t enabled; |
| 92 | 92 | /* |
| 93 | + * Note: | |
| 94 | + * To make anonymous unions work with old compilers, the static | |
| 95 | + * initialization of them requires brackets. This creates a dependency | |
| 96 | + * on the order of the struct with the initializers. If any fields | |
| 97 | + * are added, STATIC_KEY_INIT_TRUE and STATIC_KEY_INIT_FALSE may need | |
| 98 | + * to be modified. | |
| 99 | + * | |
| 93 | 100 | * bit 0 => 1 if key is initially true |
| 94 | 101 | * 0 if initially false |
| 95 | 102 | * bit 1 => 1 if points to struct static_key_mod |
| @@ -166,10 +173,10 @@ extern void static_key_disable(struct static_key *key); · 70 unmodified lines | ||
| 166 | 173 | */ |
| 167 | 174 | #define STATIC_KEY_INIT_TRUE \ |
| 168 | 175 | { .enabled = { 1 }, \ |
| 169 | − .entries = (void *)JUMP_TYPE_TRUE } | |
| 176 | + { .entries = (void *)JUMP_TYPE_TRUE } } | |
| 170 | 177 | #define STATIC_KEY_INIT_FALSE \ |
| 171 | 178 | { .enabled = { 0 }, \ |
| 172 | − .entries = (void *)JUMP_TYPE_FALSE } | |
| 179 | + { .entries = (void *)JUMP_TYPE_FALSE } } | |
| 173 | 180 | |
| 174 | 181 | #else /* !HAVE_JUMP_LABEL */ |
| 175 | 182 | |
| @@ -18,6 +18,7 @@ extern unsigned char kasan_zero_page[PAGE_SIZE]; · 17 unmodified lines | ||
| 18 | 18 | extern pte_t kasan_zero_pte[PTRS_PER_PTE]; |
| 19 | 19 | extern pmd_t kasan_zero_pmd[PTRS_PER_PMD]; |
| 20 | 20 | extern pud_t kasan_zero_pud[PTRS_PER_PUD]; |
| 21 | +extern p4d_t kasan_zero_p4d[PTRS_PER_P4D]; | |
| 21 | 22 | |
| 22 | 23 | void kasan_populate_zero_shadow(const void *shadow_start, |
| 23 | 24 | const void *shadow_end); |
| @@ -29,6 +29,11 @@ struct hlist_nulls_node { · 28 unmodified lines | ||
| 29 | 29 | ((ptr)->first = (struct hlist_nulls_node *) NULLS_MARKER(nulls)) |
| 30 | 30 | |
| 31 | 31 | #define hlist_nulls_entry(ptr, type, member) container_of(ptr,type,member) |
| 32 | + | |
| 33 | +#define hlist_nulls_entry_safe(ptr, type, member) \ | |
| 34 | + ({ typeof(ptr) ____ptr = (ptr); \ | |
| 35 | + !is_a_nulls(____ptr) ? hlist_nulls_entry(____ptr, type, member) : NULL; \ | |
| 36 | + }) | |
| 32 | 37 | /** |
| 33 | 38 | * ptr_is_a_nulls - Test if a ptr is a nulls |
| 34 | 39 | * @ptr: ptr to be tested |
| @@ -1560,14 +1560,24 @@ static inline pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr, · 1559 unmodified lines | ||
| 1560 | 1560 | return ptep; |
| 1561 | 1561 | } |
| 1562 | 1562 | |
| 1563 | +#ifdef __PAGETABLE_P4D_FOLDED | |
| 1564 | +static inline int __p4d_alloc(struct mm_struct *mm, pgd_t *pgd, | |
| 1565 | + unsigned long address) | |
| 1566 | +{ | |
| 1567 | + return 0; | |
| 1568 | +} | |
| 1569 | +#else | |
| 1570 | +int __p4d_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address); | |
| 1571 | +#endif | |
| 1572 | + | |
| 1563 | 1573 | #ifdef __PAGETABLE_PUD_FOLDED |
| 1564 | −static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, | |
| 1574 | +static inline int __pud_alloc(struct mm_struct *mm, p4d_t *p4d, | |
| 1565 | 1575 | unsigned long address) |
| 1566 | 1576 | { |
| 1567 | 1577 | return 0; |
| 1568 | 1578 | } |
| 1569 | 1579 | #else |
| 1570 | −int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address); | |
| 1580 | +int __pud_alloc(struct mm_struct *mm, p4d_t *p4d, unsigned long address); | |
| 1571 | 1581 | #endif |
| 1572 | 1582 | |
| 1573 | 1583 | #if defined(__PAGETABLE_PMD_FOLDED) || !defined(CONFIG_MMU) |
| @@ -1619,11 +1629,22 @@ int __pte_alloc_kernel(pmd_t *pmd, unsigned long address); · 45 unmodified lines | ||
| 1619 | 1629 | * Remove it when 4level-fixup.h has been removed. |
| 1620 | 1630 | */ |
| 1621 | 1631 | #if defined(CONFIG_MMU) && !defined(__ARCH_HAS_4LEVEL_HACK) |
| 1622 | −static inline pud_t *pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address) | |
| 1632 | + | |
| 1633 | +#ifndef __ARCH_HAS_5LEVEL_HACK | |
| 1634 | +static inline p4d_t *p4d_alloc(struct mm_struct *mm, pgd_t *pgd, | |
| 1635 | + unsigned long address) | |
| 1636 | +{ | |
| 1637 | + return (unlikely(pgd_none(*pgd)) && __p4d_alloc(mm, pgd, address)) ? | |
| 1638 | + NULL : p4d_offset(pgd, address); | |
| 1639 | +} | |
| 1640 | + | |
| 1641 | +static inline pud_t *pud_alloc(struct mm_struct *mm, p4d_t *p4d, | |
| 1642 | + unsigned long address) | |
| 1623 | 1643 | { |
| 1624 | − return (unlikely(pgd_none(*pgd)) && __pud_alloc(mm, pgd, address))? | |
| 1625 | − NULL: pud_offset(pgd, address); | |
| 1644 | + return (unlikely(p4d_none(*p4d)) && __pud_alloc(mm, p4d, address)) ? | |
| 1645 | + NULL : pud_offset(p4d, address); | |
| 1626 | 1646 | } |
| 1647 | +#endif /* !__ARCH_HAS_5LEVEL_HACK */ | |
| 1627 | 1648 | |
| 1628 | 1649 | static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address) |
| 1629 | 1650 | { |
| @@ -2385,7 +2406,8 @@ void sparse_mem_maps_populate_node(struct page **map_map, · 755 unmodified lines | ||
| 2385 | 2406 | |
| 2386 | 2407 | struct page *sparse_mem_map_populate(unsigned long pnum, int nid); |
| 2387 | 2408 | pgd_t *vmemmap_pgd_populate(unsigned long addr, int node); |
| 2388 | −pud_t *vmemmap_pud_populate(pgd_t *pgd, unsigned long addr, int node); | |
| 2409 | +p4d_t *vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node); | |
| 2410 | +pud_t *vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node); | |
| 2389 | 2411 | pmd_t *vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node); |
| 2390 | 2412 | pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node); |
| 2391 | 2413 | void *vmemmap_alloc_block(unsigned long size, int node); |
| @@ -146,7 +146,7 @@ struct proto_ops { · 145 unmodified lines | ||
| 146 | 146 | int (*socketpair)(struct socket *sock1, |
| 147 | 147 | struct socket *sock2); |
| 148 | 148 | int (*accept) (struct socket *sock, |
| 149 | − struct socket *newsock, int flags); | |
| 149 | + struct socket *newsock, int flags, bool kern); | |
| 150 | 150 | int (*getname) (struct socket *sock, |
| 151 | 151 | struct sockaddr *addr, |
| 152 | 152 | int *sockaddr_len, int peer); |
| @@ -837,6 +837,10 @@ int genphy_read_status(struct phy_device *phydev); · 836 unmodified lines | ||
| 837 | 837 | int genphy_suspend(struct phy_device *phydev); |
| 838 | 838 | int genphy_resume(struct phy_device *phydev); |
| 839 | 839 | int genphy_soft_reset(struct phy_device *phydev); |
| 840 | +static inline int genphy_no_soft_reset(struct phy_device *phydev) | |
| 841 | +{ | |
| 842 | + return 0; | |
| 843 | +} | |
| 840 | 844 | void phy_driver_unregister(struct phy_driver *drv); |
| 841 | 845 | void phy_drivers_unregister(struct phy_driver *drv, int n); |
| 842 | 846 | int phy_driver_register(struct phy_driver *new_driver, struct module *owner); |
| @@ -0,0 +1,23 @@ | ||
| 1 | +#ifndef _LINUX_PURGATORY_H | |
| 2 | +#define _LINUX_PURGATORY_H | |
| 3 | + | |
| 4 | +#include <linux/types.h> | |
| 5 | +#include <crypto/sha.h> | |
| 6 | +#include <uapi/linux/kexec.h> | |
| 7 | + | |
| 8 | +struct kexec_sha_region { | |
| 9 | + unsigned long start; | |
| 10 | + unsigned long len; | |
| 11 | +}; | |
| 12 | + | |
| 13 | +/* | |
| 14 | + * These forward declarations serve two purposes: | |
| 15 | + * | |
| 16 | + * 1) Make sparse happy when checking arch/purgatory | |
| 17 | + * 2) Document that these are required to be global so the symbol | |
| 18 | + * lookup in kexec works | |
| 19 | + */ | |
| 20 | +extern struct kexec_sha_region purgatory_sha_regions[KEXEC_SEGMENT_MAX]; | |
| 21 | +extern u8 purgatory_sha256_digest[SHA256_DIGEST_SIZE]; | |
| 22 | + | |
| 23 | +#endif | |
| @@ -37,14 +37,26 @@ extern void get_random_bytes(void *buf, int nbytes); · 36 unmodified lines | ||
| 37 | 37 | extern int add_random_ready_callback(struct random_ready_callback *rdy); |
| 38 | 38 | extern void del_random_ready_callback(struct random_ready_callback *rdy); |
| 39 | 39 | extern void get_random_bytes_arch(void *buf, int nbytes); |
| 40 | −extern int random_int_secret_init(void); | |
| 41 | 40 | |
| 42 | 41 | #ifndef MODULE |
| 43 | 42 | extern const struct file_operations random_fops, urandom_fops; |
| 44 | 43 | #endif |
| 45 | 44 | |
| 46 | −unsigned int get_random_int(void); | |
| 47 | −unsigned long get_random_long(void); | |
| 45 | +u32 get_random_u32(void); | |
| 46 | +u64 get_random_u64(void); | |
| 47 | +static inline unsigned int get_random_int(void) | |
| 48 | +{ | |
| 49 | + return get_random_u32(); | |
| 50 | +} | |
| 51 | +static inline unsigned long get_random_long(void) | |
| 52 | +{ | |
| 53 | +#if BITS_PER_LONG == 64 | |
| 54 | + return get_random_u64(); | |
| 55 | +#else | |
| 56 | + return get_random_u32(); | |
| 57 | +#endif | |
| 58 | +} | |
| 59 | + | |
| 48 | 60 | unsigned long randomize_page(unsigned long start, unsigned long range); |
| 49 | 61 | |
| 50 | 62 | u32 prandom_u32(void); |
| @@ -156,5 +156,19 @@ static inline void hlist_nulls_add_tail_rcu(struct hlist_nulls_node *n, · 155 unmodified lines | ||
| 156 | 156 | ({ tpos = hlist_nulls_entry(pos, typeof(*tpos), member); 1; }); \ |
| 157 | 157 | pos = rcu_dereference_raw(hlist_nulls_next_rcu(pos))) |
| 158 | 158 | |
| 159 | +/** | |
| 160 | + * hlist_nulls_for_each_entry_safe - | |
| 161 | + * iterate over list of given type safe against removal of list entry | |
| 162 | + * @tpos: the type * to use as a loop cursor. | |
| 163 | + * @pos: the &struct hlist_nulls_node to use as a loop cursor. | |
| 164 | + * @head: the head for your list. | |
| 165 | + * @member: the name of the hlist_nulls_node within the struct. | |
| 166 | + */ | |
| 167 | +#define hlist_nulls_for_each_entry_safe(tpos, pos, head, member) \ | |
| 168 | + for (({barrier();}), \ | |
| 169 | + pos = rcu_dereference_raw(hlist_nulls_first_rcu(head)); \ | |
| 170 | + (!is_a_nulls(pos)) && \ | |
| 171 | + ({ tpos = hlist_nulls_entry(pos, typeof(*tpos), member); \ | |
| 172 | + pos = rcu_dereference_raw(hlist_nulls_next_rcu(pos)); 1; });) | |
| 159 | 173 | #endif |
| 160 | 174 | #endif |
| @@ -65,7 +65,7 @@ struct regulator_state { · 64 unmodified lines | ||
| 65 | 65 | int uV; /* suspend voltage */ |
| 66 | 66 | unsigned int mode; /* suspend regulator operating mode */ |
| 67 | 67 | int enabled; /* is regulator enabled in this suspend state */ |
| 68 | − int disabled; /* is the regulator disbled in this suspend state */ | |
| 68 | + int disabled; /* is the regulator disabled in this suspend state */ | |
| 69 | 69 | }; |
| 70 | 70 | |
| 71 | 71 | /** |
| @@ -72,7 +72,7 @@ struct ucounts { · 71 unmodified lines | ||
| 72 | 72 | struct hlist_node node; |
| 73 | 73 | struct user_namespace *ns; |
| 74 | 74 | kuid_t uid; |
| 75 | − atomic_t count; | |
| 75 | + int count; | |
| 76 | 76 | atomic_t ucount[UCOUNT_COUNTS]; |
| 77 | 77 | }; |
| 78 | 78 | |
| @@ -61,8 +61,7 @@ extern void mremap_userfaultfd_complete(struct vm_userfaultfd_ctx *, · 60 unmodified lines | ||
| 61 | 61 | unsigned long from, unsigned long to, |
| 62 | 62 | unsigned long len); |
| 63 | 63 | |
| 64 | −extern void userfaultfd_remove(struct vm_area_struct *vma, | |
| 65 | − struct vm_area_struct **prev, | |
| 64 | +extern bool userfaultfd_remove(struct vm_area_struct *vma, | |
| 66 | 65 | unsigned long start, |
| 67 | 66 | unsigned long end); |
| 68 | 67 | |
| @@ -72,8 +71,6 @@ extern int userfaultfd_unmap_prep(struct vm_area_struct *vma, · 3 unmodified lines | ||
| 72 | 71 | extern void userfaultfd_unmap_complete(struct mm_struct *mm, |
| 73 | 72 | struct list_head *uf); |
| 74 | 73 | |
| 75 | −extern void userfaultfd_exit(struct mm_struct *mm); | |
| 76 | − | |
| 77 | 74 | #else /* CONFIG_USERFAULTFD */ |
| 78 | 75 | |
| 79 | 76 | /* mm helpers */ |
| @@ -120,11 +117,11 @@ static inline void mremap_userfaultfd_complete(struct vm_userfaultfd_ctx *ctx, · 40 unmodified lines | ||
| 120 | 117 | { |
| 121 | 118 | } |
| 122 | 119 | |
| 123 | −static inline void userfaultfd_remove(struct vm_area_struct *vma, | |
| 124 | − struct vm_area_struct **prev, | |
| 120 | +static inline bool userfaultfd_remove(struct vm_area_struct *vma, | |
| 125 | 121 | unsigned long start, |
| 126 | 122 | unsigned long end) |
| 127 | 123 | { |
| 124 | + return true; | |
| 128 | 125 | } |
| 129 | 126 | |
| 130 | 127 | static inline int userfaultfd_unmap_prep(struct vm_area_struct *vma, |
| @@ -139,10 +136,6 @@ static inline void userfaultfd_unmap_complete(struct mm_struct *mm, · 8 unmodified lines | ||
| 139 | 136 | { |
| 140 | 137 | } |
| 141 | 138 | |
| 142 | −static inline void userfaultfd_exit(struct mm_struct *mm) | |
| 143 | −{ | |
| 144 | −} | |
| 145 | − | |
| 146 | 139 | #endif /* CONFIG_USERFAULTFD */ |
| 147 | 140 | |
| 148 | 141 | #endif /* _LINUX_USERFAULTFD_K_H */ |
| @@ -79,6 +79,9 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, · 78 unmodified lines | ||
| 79 | 79 | THP_SPLIT_PAGE_FAILED, |
| 80 | 80 | THP_DEFERRED_SPLIT_PAGE, |
| 81 | 81 | THP_SPLIT_PMD, |
| 82 | +#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD | |
| 83 | + THP_SPLIT_PUD, | |
| 84 | +#endif | |
| 82 | 85 | THP_ZERO_PAGE_ALLOC, |
| 83 | 86 | THP_ZERO_PAGE_ALLOC_FAILED, |
| 84 | 87 | #endif |
| @@ -620,30 +620,19 @@ do { \ · 619 unmodified lines | ||
| 620 | 620 | __ret; \ |
| 621 | 621 | }) |
| 622 | 622 | |
| 623 | +extern int do_wait_intr(wait_queue_head_t *, wait_queue_t *); | |
| 624 | +extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_t *); | |
| 623 | 625 | |
| 624 | −#define __wait_event_interruptible_locked(wq, condition, exclusive, irq) \ | |
| 626 | +#define __wait_event_interruptible_locked(wq, condition, exclusive, fn) \ | |
| 625 | 627 | ({ \ |
| 626 | − int __ret = 0; \ | |
| 628 | + int __ret; \ | |
| 627 | 629 | DEFINE_WAIT(__wait); \ |
| 628 | 630 | if (exclusive) \ |
| 629 | 631 | __wait.flags |= WQ_FLAG_EXCLUSIVE; \ |
| 630 | 632 | do { \ |
| 631 | − if (likely(list_empty(&__wait.task_list))) \ | |
| 632 | − __add_wait_queue_tail(&(wq), &__wait); \ | |
| 633 | − set_current_state(TASK_INTERRUPTIBLE); \ | |
| 634 | − if (signal_pending(current)) { \ | |
| 635 | − __ret = -ERESTARTSYS; \ | |
| 633 | + __ret = fn(&(wq), &__wait); \ | |
| 634 | + if (__ret) \ | |
| 636 | 635 | break; \ |
| 637 | − } \ | |
| 638 | − if (irq) \ | |
| 639 | − spin_unlock_irq(&(wq).lock); \ | |
| 640 | − else \ | |
| 641 | − spin_unlock(&(wq).lock); \ | |
| 642 | − schedule(); \ | |
| 643 | − if (irq) \ | |
| 644 | − spin_lock_irq(&(wq).lock); \ | |
| 645 | − else \ | |
| 646 | − spin_lock(&(wq).lock); \ | |
| 647 | 636 | } while (!(condition)); \ |
| 648 | 637 | __remove_wait_queue(&(wq), &__wait); \ |
| 649 | 638 | __set_current_state(TASK_RUNNING); \ |
| @@ -676,7 +665,7 @@ do { \ · 26 unmodified lines | ||
| 676 | 665 | */ |
| 677 | 666 | #define wait_event_interruptible_locked(wq, condition) \ |
| 678 | 667 | ((condition) \ |
| 679 | − ? 0 : __wait_event_interruptible_locked(wq, condition, 0, 0)) | |
| 668 | + ? 0 : __wait_event_interruptible_locked(wq, condition, 0, do_wait_intr)) | |
| 680 | 669 | |
| 681 | 670 | /** |
| 682 | 671 | * wait_event_interruptible_locked_irq - sleep until a condition gets true |
| @@ -703,7 +692,7 @@ do { \ · 20 unmodified lines | ||
| 703 | 692 | */ |
| 704 | 693 | #define wait_event_interruptible_locked_irq(wq, condition) \ |
| 705 | 694 | ((condition) \ |
| 706 | − ? 0 : __wait_event_interruptible_locked(wq, condition, 0, 1)) | |
| 695 | + ? 0 : __wait_event_interruptible_locked(wq, condition, 0, do_wait_intr_irq)) | |
| 707 | 696 | |
| 708 | 697 | /** |
| 709 | 698 | * wait_event_interruptible_exclusive_locked - sleep exclusively until a condition gets true |
| @@ -734,7 +723,7 @@ do { \ · 24 unmodified lines | ||
| 734 | 723 | */ |
| 735 | 724 | #define wait_event_interruptible_exclusive_locked(wq, condition) \ |
| 736 | 725 | ((condition) \ |
| 737 | − ? 0 : __wait_event_interruptible_locked(wq, condition, 1, 0)) | |
| 726 | + ? 0 : __wait_event_interruptible_locked(wq, condition, 1, do_wait_intr)) | |
| 738 | 727 | |
| 739 | 728 | /** |
| 740 | 729 | * wait_event_interruptible_exclusive_locked_irq - sleep until a condition gets true |
| @@ -765,7 +754,7 @@ do { \ · 24 unmodified lines | ||
| 765 | 754 | */ |
| 766 | 755 | #define wait_event_interruptible_exclusive_locked_irq(wq, condition) \ |
| 767 | 756 | ((condition) \ |
| 768 | − ? 0 : __wait_event_interruptible_locked(wq, condition, 1, 1)) | |
| 757 | + ? 0 : __wait_event_interruptible_locked(wq, condition, 1, do_wait_intr_irq)) | |
| 769 | 758 | |
| 770 | 759 | |
| 771 | 760 | #define __wait_event_killable(wq, condition) \ |
| @@ -20,8 +20,17 @@ struct device; · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | int vsp1_du_init(struct device *dev); |
| 22 | 22 | |
| 23 | −int vsp1_du_setup_lif(struct device *dev, unsigned int width, | |
| 24 | − unsigned int height); | |
| 23 | +/** | |
| 24 | + * struct vsp1_du_lif_config - VSP LIF configuration | |
| 25 | + * @width: output frame width | |
| 26 | + * @height: output frame height | |
| 27 | + */ | |
| 28 | +struct vsp1_du_lif_config { | |
| 29 | + unsigned int width; | |
| 30 | + unsigned int height; | |
| 31 | +}; | |
| 32 | + | |
| 33 | +int vsp1_du_setup_lif(struct device *dev, const struct vsp1_du_lif_config *cfg); | |
| 25 | 34 | |
| 26 | 35 | struct vsp1_du_atomic_config { |
| 27 | 36 | u32 pixelformat; |
| @@ -20,7 +20,8 @@ int __inet_stream_connect(struct socket *sock, struct sockaddr *uaddr, · 19 unmodified lines | ||
| 20 | 20 | int addr_len, int flags, int is_sendmsg); |
| 21 | 21 | int inet_dgram_connect(struct socket *sock, struct sockaddr *uaddr, |
| 22 | 22 | int addr_len, int flags); |
| 23 | −int inet_accept(struct socket *sock, struct socket *newsock, int flags); | |
| 23 | +int inet_accept(struct socket *sock, struct socket *newsock, int flags, | |
| 24 | + bool kern); | |
| 24 | 25 | int inet_sendmsg(struct socket *sock, struct msghdr *msg, size_t size); |
| 25 | 26 | ssize_t inet_sendpage(struct socket *sock, struct page *page, int offset, |
| 26 | 27 | size_t size, int flags); |
| @@ -258,7 +258,7 @@ inet_csk_rto_backoff(const struct inet_connection_sock *icsk, · 257 unmodified lines | ||
| 258 | 258 | return (unsigned long)min_t(u64, when, max_when); |
| 259 | 259 | } |
| 260 | 260 | |
| 261 | −struct sock *inet_csk_accept(struct sock *sk, int flags, int *err); | |
| 261 | +struct sock *inet_csk_accept(struct sock *sk, int flags, int *err, bool kern); | |
| 262 | 262 | |
| 263 | 263 | int inet_csk_get_port(struct sock *sk, unsigned short snum); |
| 264 | 264 | |
| @@ -59,7 +59,7 @@ struct lap_cb; · 58 unmodified lines | ||
| 59 | 59 | * Slot timer must never exceed 85 ms, and must always be at least 25 ms, |
| 60 | 60 | * suggested to 75-85 msec by IrDA lite. This doesn't work with a lot of |
| 61 | 61 | * devices, and other stackes uses a lot more, so it's best we do it as well |
| 62 | − * (Note : this is the default value and sysctl overides it - Jean II) | |
| 62 | + * (Note : this is the default value and sysctl overrides it - Jean II) | |
| 63 | 63 | */ |
| 64 | 64 | #define SLOT_TIMEOUT (90*HZ/1000) |
| 65 | 65 | |
| @@ -476,7 +476,8 @@ struct sctp_pf { · 475 unmodified lines | ||
| 476 | 476 | int (*send_verify) (struct sctp_sock *, union sctp_addr *); |
| 477 | 477 | int (*supported_addrs)(const struct sctp_sock *, __be16 *); |
| 478 | 478 | struct sock *(*create_accept_sk) (struct sock *sk, |
| 479 | − struct sctp_association *asoc); | |
| 479 | + struct sctp_association *asoc, | |
| 480 | + bool kern); | |
| 480 | 481 | int (*addr_to_user)(struct sctp_sock *sk, union sctp_addr *addr); |
| 481 | 482 | void (*to_sk_saddr)(union sctp_addr *, struct sock *sk); |
| 482 | 483 | void (*to_sk_daddr)(union sctp_addr *, struct sock *sk); |
| @@ -236,6 +236,7 @@ struct sock_common { · 235 unmodified lines | ||
| 236 | 236 | * @sk_shutdown: mask of %SEND_SHUTDOWN and/or %RCV_SHUTDOWN |
| 237 | 237 | * @sk_userlocks: %SO_SNDBUF and %SO_RCVBUF settings |
| 238 | 238 | * @sk_lock: synchronizer |
| 239 | + * @sk_kern_sock: True if sock is using kernel lock classes | |
| 239 | 240 | * @sk_rcvbuf: size of receive buffer in bytes |
| 240 | 241 | * @sk_wq: sock wait queue and async head |
| 241 | 242 | * @sk_rx_dst: receive input route used by early demux |
| @@ -430,7 +431,8 @@ struct sock { · 188 unmodified lines | ||
| 430 | 431 | #endif |
| 431 | 432 | |
| 432 | 433 | kmemcheck_bitfield_begin(flags); |
| 433 | − unsigned int sk_padding : 2, | |
| 434 | + unsigned int sk_padding : 1, | |
| 435 | + sk_kern_sock : 1, | |
| 434 | 436 | sk_no_check_tx : 1, |
| 435 | 437 | sk_no_check_rx : 1, |
| 436 | 438 | sk_userlocks : 4, |
| @@ -1015,7 +1017,8 @@ struct proto { · 578 unmodified lines | ||
| 1015 | 1017 | int addr_len); |
| 1016 | 1018 | int (*disconnect)(struct sock *sk, int flags); |
| 1017 | 1019 | |
| 1018 | − struct sock * (*accept)(struct sock *sk, int flags, int *err); | |
| 1020 | + struct sock * (*accept)(struct sock *sk, int flags, int *err, | |
| 1021 | + bool kern); | |
| 1019 | 1022 | |
| 1020 | 1023 | int (*ioctl)(struct sock *sk, int cmd, |
| 1021 | 1024 | unsigned long arg); |
| @@ -1573,7 +1576,7 @@ int sock_cmsg_send(struct sock *sk, struct msghdr *msg, · 551 unmodified lines | ||
| 1573 | 1576 | int sock_no_bind(struct socket *, struct sockaddr *, int); |
| 1574 | 1577 | int sock_no_connect(struct socket *, struct sockaddr *, int, int); |
| 1575 | 1578 | int sock_no_socketpair(struct socket *, struct socket *); |
| 1576 | −int sock_no_accept(struct socket *, struct socket *, int); | |
| 1579 | +int sock_no_accept(struct socket *, struct socket *, int, bool); | |
| 1577 | 1580 | int sock_no_getname(struct socket *, struct sockaddr *, int *, int); |
| 1578 | 1581 | unsigned int sock_no_poll(struct file *, struct socket *, |
| 1579 | 1582 | struct poll_table_struct *); |
| @@ -196,6 +196,7 @@ struct iscsi_conn { · 195 unmodified lines | ||
| 196 | 196 | struct iscsi_task *task; /* xmit task in progress */ |
| 197 | 197 | |
| 198 | 198 | /* xmit */ |
| 199 | + spinlock_t taskqueuelock; /* protects the next three lists */ | |
| 199 | 200 | struct list_head mgmtqueue; /* mgmt (control) xmit queue */ |
| 200 | 201 | struct list_head cmdqueue; /* data-path cmd queue */ |
| 201 | 202 | struct list_head requeue; /* tasks needing another run */ |
| @@ -472,6 +472,10 @@ static inline int scsi_device_created(struct scsi_device *sdev) · 471 unmodified lines | ||
| 472 | 472 | sdev->sdev_state == SDEV_CREATED_BLOCK; |
| 473 | 473 | } |
| 474 | 474 | |
| 475 | +int scsi_internal_device_block(struct scsi_device *sdev, bool wait); | |
| 476 | +int scsi_internal_device_unblock(struct scsi_device *sdev, | |
| 477 | + enum scsi_device_state new_state); | |
| 478 | + | |
| 475 | 479 | /* accessor functions for the SCSI parameters */ |
| 476 | 480 | static inline int scsi_device_sync(struct scsi_device *sdev) |
| 477 | 481 | { |
| @@ -1,5 +1,6 @@ | ||
| 1 | 1 | #undef TRACE_SYSTEM |
| 2 | 2 | #define TRACE_SYSTEM raw_syscalls |
| 3 | +#undef TRACE_INCLUDE_FILE | |
| 3 | 4 | #define TRACE_INCLUDE_FILE syscalls |
| 4 | 5 | |
| 5 | 6 | #if !defined(_TRACE_EVENTS_SYSCALLS_H) || defined(TRACE_HEADER_MULTI_READ) |
| @@ -64,7 +64,7 @@ struct packet_diag_mclist { · 63 unmodified lines | ||
| 64 | 64 | __u32 pdmc_count; |
| 65 | 65 | __u16 pdmc_type; |
| 66 | 66 | __u16 pdmc_alen; |
| 67 | − __u8 pdmc_addr[MAX_ADDR_LEN]; | |
| 67 | + __u8 pdmc_addr[32]; /* MAX_ADDR_LEN */ | |
| 68 | 68 | }; |
| 69 | 69 | |
| 70 | 70 | struct packet_diag_ring { |
| @@ -18,8 +18,7 @@ · 17 unmodified lines | ||
| 18 | 18 | * means the userland is reading). |
| 19 | 19 | */ |
| 20 | 20 | #define UFFD_API ((__u64)0xAA) |
| 21 | −#define UFFD_API_FEATURES (UFFD_FEATURE_EVENT_EXIT | \ | |
| 22 | − UFFD_FEATURE_EVENT_FORK | \ | |
| 21 | +#define UFFD_API_FEATURES (UFFD_FEATURE_EVENT_FORK | \ | |
| 23 | 22 | UFFD_FEATURE_EVENT_REMAP | \ |
| 24 | 23 | UFFD_FEATURE_EVENT_REMOVE | \ |
| 25 | 24 | UFFD_FEATURE_EVENT_UNMAP | \ |
| @@ -113,7 +112,6 @@ struct uffd_msg { · 87 unmodified lines | ||
| 113 | 112 | #define UFFD_EVENT_REMAP 0x14 |
| 114 | 113 | #define UFFD_EVENT_REMOVE 0x15 |
| 115 | 114 | #define UFFD_EVENT_UNMAP 0x16 |
| 116 | −#define UFFD_EVENT_EXIT 0x17 | |
| 117 | 115 | |
| 118 | 116 | /* flags for UFFD_EVENT_PAGEFAULT */ |
| 119 | 117 | #define UFFD_PAGEFAULT_FLAG_WRITE (1<<0) /* If this was a write fault */ |
| @@ -163,7 +161,6 @@ struct uffdio_api { · 43 unmodified lines | ||
| 163 | 161 | #define UFFD_FEATURE_MISSING_HUGETLBFS (1<<4) |
| 164 | 162 | #define UFFD_FEATURE_MISSING_SHMEM (1<<5) |
| 165 | 163 | #define UFFD_FEATURE_EVENT_UNMAP (1<<6) |
| 166 | −#define UFFD_FEATURE_EVENT_EXIT (1<<7) | |
| 167 | 164 | __u64 features; |
| 168 | 165 | |
| 169 | 166 | __u64 ioctls; |
| @@ -2,6 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | #define __LINUX_SWIOTLB_XEN_H |
| 3 | 3 | |
| 4 | 4 | #include <linux/dma-direction.h> |
| 5 | +#include <linux/scatterlist.h> | |
| 5 | 6 | #include <linux/swiotlb.h> |
| 6 | 7 | |
| 7 | 8 | extern int xen_swiotlb_init(int verbose, bool early); |
| @@ -55,4 +56,14 @@ xen_swiotlb_dma_supported(struct device *hwdev, u64 mask); · 47 unmodified lines | ||
| 55 | 56 | |
| 56 | 57 | extern int |
| 57 | 58 | xen_swiotlb_set_dma_mask(struct device *dev, u64 dma_mask); |
| 59 | + | |
| 60 | +extern int | |
| 61 | +xen_swiotlb_dma_mmap(struct device *dev, struct vm_area_struct *vma, | |
| 62 | + void *cpu_addr, dma_addr_t dma_addr, size_t size, | |
| 63 | + unsigned long attrs); | |
| 64 | + | |
| 65 | +extern int | |
| 66 | +xen_swiotlb_get_sgtable(struct device *dev, struct sg_table *sgt, | |
| 67 | + void *cpu_addr, dma_addr_t handle, size_t size, | |
| 68 | + unsigned long attrs); | |
| 58 | 69 | #endif /* __LINUX_SWIOTLB_XEN_H */ |
| @@ -882,7 +882,6 @@ static void __init do_basic_setup(void) · 881 unmodified lines | ||
| 882 | 882 | do_ctors(); |
| 883 | 883 | usermodehelper_enable(); |
| 884 | 884 | do_initcalls(); |
| 885 | − random_int_secret_init(); | |
| 886 | 885 | } |
| 887 | 886 | |
| 888 | 887 | static void __init do_pre_smp_initcalls(void) |
| @@ -13,11 +13,12 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/bpf.h> |
| 14 | 14 | #include <linux/jhash.h> |
| 15 | 15 | #include <linux/filter.h> |
| 16 | +#include <linux/rculist_nulls.h> | |
| 16 | 17 | #include "percpu_freelist.h" |
| 17 | 18 | #include "bpf_lru_list.h" |
| 18 | 19 | |
| 19 | 20 | struct bucket { |
| 20 | − struct hlist_head head; | |
| 21 | + struct hlist_nulls_head head; | |
| 21 | 22 | raw_spinlock_t lock; |
| 22 | 23 | }; |
| 23 | 24 | |
| @@ -44,9 +45,14 @@ enum extra_elem_state { · 20 unmodified lines | ||
| 44 | 45 | /* each htab element is struct htab_elem + key + value */ |
| 45 | 46 | struct htab_elem { |
| 46 | 47 | union { |
| 47 | − struct hlist_node hash_node; | |
| 48 | − struct bpf_htab *htab; | |
| 49 | − struct pcpu_freelist_node fnode; | |
| 48 | + struct hlist_nulls_node hash_node; | |
| 49 | + struct { | |
| 50 | + void *padding; | |
| 51 | + union { | |
| 52 | + struct bpf_htab *htab; | |
| 53 | + struct pcpu_freelist_node fnode; | |
| 54 | + }; | |
| 55 | + }; | |
| 50 | 56 | }; |
| 51 | 57 | union { |
| 52 | 58 | struct rcu_head rcu; |
| @@ -162,7 +168,8 @@ skip_percpu_elems: · 109 unmodified lines | ||
| 162 | 168 | offsetof(struct htab_elem, lru_node), |
| 163 | 169 | htab->elem_size, htab->map.max_entries); |
| 164 | 170 | else |
| 165 | − pcpu_freelist_populate(&htab->freelist, htab->elems, | |
| 171 | + pcpu_freelist_populate(&htab->freelist, | |
| 172 | + htab->elems + offsetof(struct htab_elem, fnode), | |
| 166 | 173 | htab->elem_size, htab->map.max_entries); |
| 167 | 174 | |
| 168 | 175 | return 0; |
| @@ -217,6 +224,11 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) · 48 unmodified lines | ||
| 217 | 224 | int err, i; |
| 218 | 225 | u64 cost; |
| 219 | 226 | |
| 227 | + BUILD_BUG_ON(offsetof(struct htab_elem, htab) != | |
| 228 | + offsetof(struct htab_elem, hash_node.pprev)); | |
| 229 | + BUILD_BUG_ON(offsetof(struct htab_elem, fnode.next) != | |
| 230 | + offsetof(struct htab_elem, hash_node.pprev)); | |
| 231 | + | |
| 220 | 232 | if (lru && !capable(CAP_SYS_ADMIN)) |
| 221 | 233 | /* LRU implementation is much complicated than other |
| 222 | 234 | * maps. Hence, limit to CAP_SYS_ADMIN for now. |
| @@ -326,7 +338,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) · 103 unmodified lines | ||
| 326 | 338 | goto free_htab; |
| 327 | 339 | |
| 328 | 340 | for (i = 0; i < htab->n_buckets; i++) { |
| 329 | − INIT_HLIST_HEAD(&htab->buckets[i].head); | |
| 341 | + INIT_HLIST_NULLS_HEAD(&htab->buckets[i].head, i); | |
| 330 | 342 | raw_spin_lock_init(&htab->buckets[i].lock); |
| 331 | 343 | } |
| 332 | 344 | |
| @@ -366,20 +378,44 @@ static inline struct bucket *__select_bucket(struct bpf_htab *htab, u32 hash) · 33 unmodified lines | ||
| 366 | 378 | return &htab->buckets[hash & (htab->n_buckets - 1)]; |
| 367 | 379 | } |
| 368 | 380 | |
| 369 | −static inline struct hlist_head *select_bucket(struct bpf_htab *htab, u32 hash) | |
| 381 | +static inline struct hlist_nulls_head *select_bucket(struct bpf_htab *htab, u32 hash) | |
| 370 | 382 | { |
| 371 | 383 | return &__select_bucket(htab, hash)->head; |
| 372 | 384 | } |
| 373 | 385 | |
| 374 | −static struct htab_elem *lookup_elem_raw(struct hlist_head *head, u32 hash, | |
| 386 | +/* this lookup function can only be called with bucket lock taken */ | |
| 387 | +static struct htab_elem *lookup_elem_raw(struct hlist_nulls_head *head, u32 hash, | |
| 375 | 388 | void *key, u32 key_size) |
| 376 | 389 | { |
| 390 | + struct hlist_nulls_node *n; | |
| 391 | + struct htab_elem *l; | |
| 392 | + | |
| 393 | + hlist_nulls_for_each_entry_rcu(l, n, head, hash_node) | |
| 394 | + if (l->hash == hash && !memcmp(&l->key, key, key_size)) | |
| 395 | + return l; | |
| 396 | + | |
| 397 | + return NULL; | |
| 398 | +} | |
| 399 | + | |
| 400 | +/* can be called without bucket lock. it will repeat the loop in | |
| 401 | + * the unlikely event when elements moved from one bucket into another | |
| 402 | + * while link list is being walked | |
| 403 | + */ | |
| 404 | +static struct htab_elem *lookup_nulls_elem_raw(struct hlist_nulls_head *head, | |
| 405 | + u32 hash, void *key, | |
| 406 | + u32 key_size, u32 n_buckets) | |
| 407 | +{ | |
| 408 | + struct hlist_nulls_node *n; | |
| 377 | 409 | struct htab_elem *l; |
| 378 | 410 | |
| 379 | − hlist_for_each_entry_rcu(l, head, hash_node) | |
| 411 | +again: | |
| 412 | + hlist_nulls_for_each_entry_rcu(l, n, head, hash_node) | |
| 380 | 413 | if (l->hash == hash && !memcmp(&l->key, key, key_size)) |
| 381 | 414 | return l; |
| 382 | 415 | |
| 416 | + if (unlikely(get_nulls_value(n) != (hash & (n_buckets - 1)))) | |
| 417 | + goto again; | |
| 418 | + | |
| 383 | 419 | return NULL; |
| 384 | 420 | } |
| 385 | 421 | |
| @@ -387,7 +423,7 @@ static struct htab_elem *lookup_elem_raw(struct hlist_head *head, u32 hash, · 1 unmodified line | ||
| 387 | 423 | static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) |
| 388 | 424 | { |
| 389 | 425 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 390 | − struct hlist_head *head; | |
| 426 | + struct hlist_nulls_head *head; | |
| 391 | 427 | struct htab_elem *l; |
| 392 | 428 | u32 hash, key_size; |
| 393 | 429 | |
| @@ -400,7 +436,7 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) · 6 unmodified lines | ||
| 400 | 436 | |
| 401 | 437 | head = select_bucket(htab, hash); |
| 402 | 438 | |
| 403 | − l = lookup_elem_raw(head, hash, key, key_size); | |
| 439 | + l = lookup_nulls_elem_raw(head, hash, key, key_size, htab->n_buckets); | |
| 404 | 440 | |
| 405 | 441 | return l; |
| 406 | 442 | } |
| @@ -433,8 +469,9 @@ static void *htab_lru_map_lookup_elem(struct bpf_map *map, void *key) · 26 unmodified lines | ||
| 433 | 469 | static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) |
| 434 | 470 | { |
| 435 | 471 | struct bpf_htab *htab = (struct bpf_htab *)arg; |
| 436 | − struct htab_elem *l, *tgt_l; | |
| 437 | − struct hlist_head *head; | |
| 472 | + struct htab_elem *l = NULL, *tgt_l; | |
| 473 | + struct hlist_nulls_head *head; | |
| 474 | + struct hlist_nulls_node *n; | |
| 438 | 475 | unsigned long flags; |
| 439 | 476 | struct bucket *b; |
| 440 | 477 | |
| @@ -444,9 +481,9 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) · 3 unmodified lines | ||
| 444 | 481 | |
| 445 | 482 | raw_spin_lock_irqsave(&b->lock, flags); |
| 446 | 483 | |
| 447 | − hlist_for_each_entry_rcu(l, head, hash_node) | |
| 484 | + hlist_nulls_for_each_entry_rcu(l, n, head, hash_node) | |
| 448 | 485 | if (l == tgt_l) { |
| 449 | − hlist_del_rcu(&l->hash_node); | |
| 486 | + hlist_nulls_del_rcu(&l->hash_node); | |
| 450 | 487 | break; |
| 451 | 488 | } |
| 452 | 489 | |
| @@ -459,7 +496,7 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) · 6 unmodified lines | ||
| 459 | 496 | static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) |
| 460 | 497 | { |
| 461 | 498 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 462 | − struct hlist_head *head; | |
| 499 | + struct hlist_nulls_head *head; | |
| 463 | 500 | struct htab_elem *l, *next_l; |
| 464 | 501 | u32 hash, key_size; |
| 465 | 502 | int i; |
| @@ -473,7 +510,7 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) · 7 unmodified lines | ||
| 473 | 510 | head = select_bucket(htab, hash); |
| 474 | 511 | |
| 475 | 512 | /* lookup the key */ |
| 476 | − l = lookup_elem_raw(head, hash, key, key_size); | |
| 513 | + l = lookup_nulls_elem_raw(head, hash, key, key_size, htab->n_buckets); | |
| 477 | 514 | |
| 478 | 515 | if (!l) { |
| 479 | 516 | i = 0; |
| @@ -481,7 +518,7 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) · 1 unmodified line | ||
| 481 | 518 | } |
| 482 | 519 | |
| 483 | 520 | /* key was found, get next key in the same bucket */ |
| 484 | − next_l = hlist_entry_safe(rcu_dereference_raw(hlist_next_rcu(&l->hash_node)), | |
| 521 | + next_l = hlist_nulls_entry_safe(rcu_dereference_raw(hlist_nulls_next_rcu(&l->hash_node)), | |
| 485 | 522 | struct htab_elem, hash_node); |
| 486 | 523 | |
| 487 | 524 | if (next_l) { |
| @@ -500,7 +537,7 @@ find_first_elem: · 12 unmodified lines | ||
| 500 | 537 | head = select_bucket(htab, i); |
| 501 | 538 | |
| 502 | 539 | /* pick first element in the bucket */ |
| 503 | − next_l = hlist_entry_safe(rcu_dereference_raw(hlist_first_rcu(head)), | |
| 540 | + next_l = hlist_nulls_entry_safe(rcu_dereference_raw(hlist_nulls_first_rcu(head)), | |
| 504 | 541 | struct htab_elem, hash_node); |
| 505 | 542 | if (next_l) { |
| 506 | 543 | /* if it's not empty, just return it */ |
| @@ -582,9 +619,13 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, · 75 unmodified lines | ||
| 582 | 619 | int err = 0; |
| 583 | 620 | |
| 584 | 621 | if (prealloc) { |
| 585 | − l_new = (struct htab_elem *)pcpu_freelist_pop(&htab->freelist); | |
| 586 | − if (!l_new) | |
| 622 | + struct pcpu_freelist_node *l; | |
| 623 | + | |
| 624 | + l = pcpu_freelist_pop(&htab->freelist); | |
| 625 | + if (!l) | |
| 587 | 626 | err = -E2BIG; |
| 627 | + else | |
| 628 | + l_new = container_of(l, struct htab_elem, fnode); | |
| 588 | 629 | } else { |
| 589 | 630 | if (atomic_inc_return(&htab->count) > htab->map.max_entries) { |
| 590 | 631 | atomic_dec(&htab->count); |
| @@ -661,7 +702,7 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, · 70 unmodified lines | ||
| 661 | 702 | { |
| 662 | 703 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 663 | 704 | struct htab_elem *l_new = NULL, *l_old; |
| 664 | − struct hlist_head *head; | |
| 705 | + struct hlist_nulls_head *head; | |
| 665 | 706 | unsigned long flags; |
| 666 | 707 | struct bucket *b; |
| 667 | 708 | u32 key_size, hash; |
| @@ -700,9 +741,9 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, · 32 unmodified lines | ||
| 700 | 741 | /* add new element to the head of the list, so that |
| 701 | 742 | * concurrent search will find it before old elem |
| 702 | 743 | */ |
| 703 | − hlist_add_head_rcu(&l_new->hash_node, head); | |
| 744 | + hlist_nulls_add_head_rcu(&l_new->hash_node, head); | |
| 704 | 745 | if (l_old) { |
| 705 | − hlist_del_rcu(&l_old->hash_node); | |
| 746 | + hlist_nulls_del_rcu(&l_old->hash_node); | |
| 706 | 747 | free_htab_elem(htab, l_old); |
| 707 | 748 | } |
| 708 | 749 | ret = 0; |
| @@ -716,7 +757,7 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, · 7 unmodified lines | ||
| 716 | 757 | { |
| 717 | 758 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 718 | 759 | struct htab_elem *l_new, *l_old = NULL; |
| 719 | − struct hlist_head *head; | |
| 760 | + struct hlist_nulls_head *head; | |
| 720 | 761 | unsigned long flags; |
| 721 | 762 | struct bucket *b; |
| 722 | 763 | u32 key_size, hash; |
| @@ -757,10 +798,10 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, · 34 unmodified lines | ||
| 757 | 798 | /* add new element to the head of the list, so that |
| 758 | 799 | * concurrent search will find it before old elem |
| 759 | 800 | */ |
| 760 | − hlist_add_head_rcu(&l_new->hash_node, head); | |
| 801 | + hlist_nulls_add_head_rcu(&l_new->hash_node, head); | |
| 761 | 802 | if (l_old) { |
| 762 | 803 | bpf_lru_node_set_ref(&l_new->lru_node); |
| 763 | − hlist_del_rcu(&l_old->hash_node); | |
| 804 | + hlist_nulls_del_rcu(&l_old->hash_node); | |
| 764 | 805 | } |
| 765 | 806 | ret = 0; |
| 766 | 807 | |
| @@ -781,7 +822,7 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, · 14 unmodified lines | ||
| 781 | 822 | { |
| 782 | 823 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 783 | 824 | struct htab_elem *l_new = NULL, *l_old; |
| 784 | − struct hlist_head *head; | |
| 825 | + struct hlist_nulls_head *head; | |
| 785 | 826 | unsigned long flags; |
| 786 | 827 | struct bucket *b; |
| 787 | 828 | u32 key_size, hash; |
| @@ -820,7 +861,7 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, · 32 unmodified lines | ||
| 820 | 861 | ret = PTR_ERR(l_new); |
| 821 | 862 | goto err; |
| 822 | 863 | } |
| 823 | − hlist_add_head_rcu(&l_new->hash_node, head); | |
| 864 | + hlist_nulls_add_head_rcu(&l_new->hash_node, head); | |
| 824 | 865 | } |
| 825 | 866 | ret = 0; |
| 826 | 867 | err: |
| @@ -834,7 +875,7 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, · 7 unmodified lines | ||
| 834 | 875 | { |
| 835 | 876 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 836 | 877 | struct htab_elem *l_new = NULL, *l_old; |
| 837 | − struct hlist_head *head; | |
| 878 | + struct hlist_nulls_head *head; | |
| 838 | 879 | unsigned long flags; |
| 839 | 880 | struct bucket *b; |
| 840 | 881 | u32 key_size, hash; |
| @@ -882,7 +923,7 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, · 41 unmodified lines | ||
| 882 | 923 | } else { |
| 883 | 924 | pcpu_copy_value(htab, htab_elem_get_ptr(l_new, key_size), |
| 884 | 925 | value, onallcpus); |
| 885 | − hlist_add_head_rcu(&l_new->hash_node, head); | |
| 926 | + hlist_nulls_add_head_rcu(&l_new->hash_node, head); | |
| 886 | 927 | l_new = NULL; |
| 887 | 928 | } |
| 888 | 929 | ret = 0; |
| @@ -910,7 +951,7 @@ static int htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, · 21 unmodified lines | ||
| 910 | 951 | static int htab_map_delete_elem(struct bpf_map *map, void *key) |
| 911 | 952 | { |
| 912 | 953 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 913 | − struct hlist_head *head; | |
| 954 | + struct hlist_nulls_head *head; | |
| 914 | 955 | struct bucket *b; |
| 915 | 956 | struct htab_elem *l; |
| 916 | 957 | unsigned long flags; |
| @@ -930,7 +971,7 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) · 13 unmodified lines | ||
| 930 | 971 | l = lookup_elem_raw(head, hash, key, key_size); |
| 931 | 972 | |
| 932 | 973 | if (l) { |
| 933 | − hlist_del_rcu(&l->hash_node); | |
| 974 | + hlist_nulls_del_rcu(&l->hash_node); | |
| 934 | 975 | free_htab_elem(htab, l); |
| 935 | 976 | ret = 0; |
| 936 | 977 | } |
| @@ -942,7 +983,7 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) · 5 unmodified lines | ||
| 942 | 983 | static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) |
| 943 | 984 | { |
| 944 | 985 | struct bpf_htab *htab = container_of(map, struct bpf_htab, map); |
| 945 | − struct hlist_head *head; | |
| 986 | + struct hlist_nulls_head *head; | |
| 946 | 987 | struct bucket *b; |
| 947 | 988 | struct htab_elem *l; |
| 948 | 989 | unsigned long flags; |
| @@ -962,7 +1003,7 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) · 13 unmodified lines | ||
| 962 | 1003 | l = lookup_elem_raw(head, hash, key, key_size); |
| 963 | 1004 | |
| 964 | 1005 | if (l) { |
| 965 | − hlist_del_rcu(&l->hash_node); | |
| 1006 | + hlist_nulls_del_rcu(&l->hash_node); | |
| 966 | 1007 | ret = 0; |
| 967 | 1008 | } |
| 968 | 1009 | |
| @@ -977,12 +1018,12 @@ static void delete_all_elements(struct bpf_htab *htab) · 8 unmodified lines | ||
| 977 | 1018 | int i; |
| 978 | 1019 | |
| 979 | 1020 | for (i = 0; i < htab->n_buckets; i++) { |
| 980 | − struct hlist_head *head = select_bucket(htab, i); | |
| 981 | − struct hlist_node *n; | |
| 1021 | + struct hlist_nulls_head *head = select_bucket(htab, i); | |
| 1022 | + struct hlist_nulls_node *n; | |
| 982 | 1023 | struct htab_elem *l; |
| 983 | 1024 | |
| 984 | − hlist_for_each_entry_safe(l, n, head, hash_node) { | |
| 985 | − hlist_del_rcu(&l->hash_node); | |
| 1025 | + hlist_nulls_for_each_entry_safe(l, n, head, hash_node) { | |
| 1026 | + hlist_nulls_del_rcu(&l->hash_node); | |
| 986 | 1027 | if (l->state != HTAB_EXTRA_ELEM_USED) |
| 987 | 1028 | htab_elem_free(htab, l); |
| 988 | 1029 | } |
| @@ -500,9 +500,15 @@ unlock: · 499 unmodified lines | ||
| 500 | 500 | raw_spin_unlock(&trie->lock); |
| 501 | 501 | } |
| 502 | 502 | |
| 503 | +static int trie_get_next_key(struct bpf_map *map, void *key, void *next_key) | |
| 504 | +{ | |
| 505 | + return -ENOTSUPP; | |
| 506 | +} | |
| 507 | + | |
| 503 | 508 | static const struct bpf_map_ops trie_ops = { |
| 504 | 509 | .map_alloc = trie_alloc, |
| 505 | 510 | .map_free = trie_free, |
| 511 | + .map_get_next_key = trie_get_next_key, | |
| 506 | 512 | .map_lookup_elem = trie_lookup_elem, |
| 507 | 513 | .map_update_elem = trie_update_elem, |
| 508 | 514 | .map_delete_elem = trie_delete_elem, |
| @@ -1329,7 +1329,7 @@ static int cgroup_css_links_read(struct seq_file *seq, void *v) · 1328 unmodified lines | ||
| 1329 | 1329 | struct task_struct *task; |
| 1330 | 1330 | int count = 0; |
| 1331 | 1331 | |
| 1332 | − seq_printf(seq, "css_set %p\n", cset); | |
| 1332 | + seq_printf(seq, "css_set %pK\n", cset); | |
| 1333 | 1333 | |
| 1334 | 1334 | list_for_each_entry(task, &cset->tasks, cg_list) { |
| 1335 | 1335 | if (count++ > MAX_TASKS_SHOWN_PER_CSS) |
| @@ -2669,7 +2669,7 @@ static bool css_visible(struct cgroup_subsys_state *css) · 2668 unmodified lines | ||
| 2669 | 2669 | * |
| 2670 | 2670 | * Returns 0 on success, -errno on failure. On failure, csses which have |
| 2671 | 2671 | * been processed already aren't cleaned up. The caller is responsible for |
| 2672 | − * cleaning up with cgroup_apply_control_disble(). | |
| 2672 | + * cleaning up with cgroup_apply_control_disable(). | |
| 2673 | 2673 | */ |
| 2674 | 2674 | static int cgroup_apply_control_enable(struct cgroup *cgrp) |
| 2675 | 2675 | { |
| @@ -229,7 +229,7 @@ static int pids_can_fork(struct task_struct *task) · 228 unmodified lines | ||
| 229 | 229 | /* Only log the first time events_limit is incremented. */ |
| 230 | 230 | if (atomic64_inc_return(&pids->events_limit) == 1) { |
| 231 | 231 | pr_info("cgroup: fork rejected by pids controller in "); |
| 232 | − pr_cont_cgroup_path(task_cgroup(current, pids_cgrp_id)); | |
| 232 | + pr_cont_cgroup_path(css->cgroup); | |
| 233 | 233 | pr_cont("\n"); |
| 234 | 234 | } |
| 235 | 235 | cgroup_file_notify(&pids->events_file); |
| @@ -1001,7 +1001,7 @@ list_update_cgroup_event(struct perf_event *event, · 1000 unmodified lines | ||
| 1001 | 1001 | */ |
| 1002 | 1002 | #define PERF_CPU_HRTIMER (1000 / HZ) |
| 1003 | 1003 | /* |
| 1004 | − * function must be called with interrupts disbled | |
| 1004 | + * function must be called with interrupts disabled | |
| 1005 | 1005 | */ |
| 1006 | 1006 | static enum hrtimer_restart perf_mux_hrtimer_handler(struct hrtimer *hr) |
| 1007 | 1007 | { |
| @@ -554,7 +554,6 @@ static void exit_mm(void) · 553 unmodified lines | ||
| 554 | 554 | enter_lazy_tlb(mm, current); |
| 555 | 555 | task_unlock(current); |
| 556 | 556 | mm_update_next_owner(mm); |
| 557 | − userfaultfd_exit(mm); | |
| 558 | 557 | mmput(mm); |
| 559 | 558 | if (test_thread_flag(TIF_MEMDIE)) |
| 560 | 559 | exit_oom_victim(); |
| @@ -614,13 +614,13 @@ static int kexec_calculate_store_digests(struct kimage *image) · 613 unmodified lines | ||
| 614 | 614 | ret = crypto_shash_final(desc, digest); |
| 615 | 615 | if (ret) |
| 616 | 616 | goto out_free_digest; |
| 617 | − ret = kexec_purgatory_get_set_symbol(image, "sha_regions", | |
| 618 | − sha_regions, sha_region_sz, 0); | |
| 617 | + ret = kexec_purgatory_get_set_symbol(image, "purgatory_sha_regions", | |
| 618 | + sha_regions, sha_region_sz, 0); | |
| 619 | 619 | if (ret) |
| 620 | 620 | goto out_free_digest; |
| 621 | 621 | |
| 622 | − ret = kexec_purgatory_get_set_symbol(image, "sha256_digest", | |
| 623 | − digest, SHA256_DIGEST_SIZE, 0); | |
| 622 | + ret = kexec_purgatory_get_set_symbol(image, "purgatory_sha256_digest", | |
| 623 | + digest, SHA256_DIGEST_SIZE, 0); | |
| 624 | 624 | if (ret) |
| 625 | 625 | goto out_free_digest; |
| 626 | 626 | } |
| @@ -15,11 +15,7 @@ int kimage_is_destination_range(struct kimage *image, · 14 unmodified lines | ||
| 15 | 15 | extern struct mutex kexec_mutex; |
| 16 | 16 | |
| 17 | 17 | #ifdef CONFIG_KEXEC_FILE |
| 18 | −struct kexec_sha_region { | |
| 19 | − unsigned long start; | |
| 20 | − unsigned long len; | |
| 21 | −}; | |
| 22 | − | |
| 18 | +#include <linux/purgatory.h> | |
| 23 | 19 | void kimage_file_post_load_cleanup(struct kimage *image); |
| 24 | 20 | #else /* CONFIG_KEXEC_FILE */ |
| 25 | 21 | static inline void kimage_file_post_load_cleanup(struct kimage *image) { } |
| @@ -3262,10 +3262,17 @@ static int __lock_acquire(struct lockdep_map *lock, unsigned int subclass, · 3261 unmodified lines | ||
| 3262 | 3262 | if (depth) { |
| 3263 | 3263 | hlock = curr->held_locks + depth - 1; |
| 3264 | 3264 | if (hlock->class_idx == class_idx && nest_lock) { |
| 3265 | − if (hlock->references) | |
| 3265 | + if (hlock->references) { | |
| 3266 | + /* | |
| 3267 | + * Check: unsigned int references:12, overflow. | |
| 3268 | + */ | |
| 3269 | + if (DEBUG_LOCKS_WARN_ON(hlock->references == (1 << 12)-1)) | |
| 3270 | + return 0; | |
| 3271 | + | |
| 3266 | 3272 | hlock->references++; |
| 3267 | − else | |
| 3273 | + } else { | |
| 3268 | 3274 | hlock->references = 2; |
| 3275 | + } | |
| 3269 | 3276 | |
| 3270 | 3277 | return 1; |
| 3271 | 3278 | } |
| @@ -50,7 +50,7 @@ static void test_mutex_work(struct work_struct *work) · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | if (mtx->flags & TEST_MTX_TRY) { |
| 52 | 52 | while (!ww_mutex_trylock(&mtx->mutex)) |
| 53 | − cpu_relax(); | |
| 53 | + cond_resched(); | |
| 54 | 54 | } else { |
| 55 | 55 | ww_mutex_lock(&mtx->mutex, NULL); |
| 56 | 56 | } |
| @@ -88,7 +88,7 @@ static int __test_mutex(unsigned int flags) · 31 unmodified lines | ||
| 88 | 88 | ret = -EINVAL; |
| 89 | 89 | break; |
| 90 | 90 | } |
| 91 | − cpu_relax(); | |
| 91 | + cond_resched(); | |
| 92 | 92 | } while (time_before(jiffies, timeout)); |
| 93 | 93 | } else { |
| 94 | 94 | ret = wait_for_completion_timeout(&mtx.done, TIMEOUT); |
| @@ -627,7 +627,7 @@ static int __init test_ww_mutex_init(void) · 532 unmodified lines | ||
| 627 | 627 | if (ret) |
| 628 | 628 | return ret; |
| 629 | 629 | |
| 630 | − ret = stress(4096, hweight32(STRESS_ALL)*ncpus, 1<<12, STRESS_ALL); | |
| 630 | + ret = stress(4095, hweight32(STRESS_ALL)*ncpus, 1<<12, STRESS_ALL); | |
| 631 | 631 | if (ret) |
| 632 | 632 | return ret; |
| 633 | 633 | |
| @@ -3287,10 +3287,15 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) · 3286 unmodified lines | ||
| 3287 | 3287 | struct task_struct *p; |
| 3288 | 3288 | |
| 3289 | 3289 | /* |
| 3290 | − * Optimization: we know that if all tasks are in | |
| 3291 | − * the fair class we can call that function directly: | |
| 3290 | + * Optimization: we know that if all tasks are in the fair class we can | |
| 3291 | + * call that function directly, but only if the @prev task wasn't of a | |
| 3292 | + * higher scheduling class, because otherwise those loose the | |
| 3293 | + * opportunity to pull in more work from other CPUs. | |
| 3292 | 3294 | */ |
| 3293 | − if (likely(rq->nr_running == rq->cfs.h_nr_running)) { | |
| 3295 | + if (likely((prev->sched_class == &idle_sched_class || | |
| 3296 | + prev->sched_class == &fair_sched_class) && | |
| 3297 | + rq->nr_running == rq->cfs.h_nr_running)) { | |
| 3298 | + | |
| 3294 | 3299 | p = fair_sched_class.pick_next_task(rq, prev, rf); |
| 3295 | 3300 | if (unlikely(p == RETRY_TASK)) |
| 3296 | 3301 | goto again; |
| @@ -36,6 +36,7 @@ struct sugov_policy { · 35 unmodified lines | ||
| 36 | 36 | u64 last_freq_update_time; |
| 37 | 37 | s64 freq_update_delay_ns; |
| 38 | 38 | unsigned int next_freq; |
| 39 | + unsigned int cached_raw_freq; | |
| 39 | 40 | |
| 40 | 41 | /* The next fields are only needed if fast switch cannot be used. */ |
| 41 | 42 | struct irq_work irq_work; |
| @@ -52,7 +53,6 @@ struct sugov_cpu { · 10 unmodified lines | ||
| 52 | 53 | struct update_util_data update_util; |
| 53 | 54 | struct sugov_policy *sg_policy; |
| 54 | 55 | |
| 55 | − unsigned int cached_raw_freq; | |
| 56 | 56 | unsigned long iowait_boost; |
| 57 | 57 | unsigned long iowait_boost_max; |
| 58 | 58 | u64 last_update; |
| @@ -116,7 +116,7 @@ static void sugov_update_commit(struct sugov_policy *sg_policy, u64 time, · 57 unmodified lines | ||
| 116 | 116 | |
| 117 | 117 | /** |
| 118 | 118 | * get_next_freq - Compute a new frequency for a given cpufreq policy. |
| 119 | − * @sg_cpu: schedutil cpu object to compute the new frequency for. | |
| 119 | + * @sg_policy: schedutil policy object to compute the new frequency for. | |
| 120 | 120 | * @util: Current CPU utilization. |
| 121 | 121 | * @max: CPU capacity. |
| 122 | 122 | * |
| @@ -136,19 +136,18 @@ static void sugov_update_commit(struct sugov_policy *sg_policy, u64 time, · 13 unmodified lines | ||
| 136 | 136 | * next_freq (as calculated above) is returned, subject to policy min/max and |
| 137 | 137 | * cpufreq driver limitations. |
| 138 | 138 | */ |
| 139 | −static unsigned int get_next_freq(struct sugov_cpu *sg_cpu, unsigned long util, | |
| 140 | − unsigned long max) | |
| 139 | +static unsigned int get_next_freq(struct sugov_policy *sg_policy, | |
| 140 | + unsigned long util, unsigned long max) | |
| 141 | 141 | { |
| 142 | − struct sugov_policy *sg_policy = sg_cpu->sg_policy; | |
| 143 | 142 | struct cpufreq_policy *policy = sg_policy->policy; |
| 144 | 143 | unsigned int freq = arch_scale_freq_invariant() ? |
| 145 | 144 | policy->cpuinfo.max_freq : policy->cur; |
| 146 | 145 | |
| 147 | 146 | freq = (freq + (freq >> 2)) * util / max; |
| 148 | 147 | |
| 149 | − if (freq == sg_cpu->cached_raw_freq && sg_policy->next_freq != UINT_MAX) | |
| 148 | + if (freq == sg_policy->cached_raw_freq && sg_policy->next_freq != UINT_MAX) | |
| 150 | 149 | return sg_policy->next_freq; |
| 151 | − sg_cpu->cached_raw_freq = freq; | |
| 150 | + sg_policy->cached_raw_freq = freq; | |
| 152 | 151 | return cpufreq_driver_resolve_freq(policy, freq); |
| 153 | 152 | } |
| 154 | 153 | |
| @@ -213,7 +212,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, · 58 unmodified lines | ||
| 213 | 212 | } else { |
| 214 | 213 | sugov_get_util(&util, &max); |
| 215 | 214 | sugov_iowait_boost(sg_cpu, &util, &max); |
| 216 | − next_f = get_next_freq(sg_cpu, util, max); | |
| 215 | + next_f = get_next_freq(sg_policy, util, max); | |
| 217 | 216 | } |
| 218 | 217 | sugov_update_commit(sg_policy, time, next_f); |
| 219 | 218 | } |
| @@ -267,7 +266,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, · 47 unmodified lines | ||
| 267 | 266 | sugov_iowait_boost(j_sg_cpu, &util, &max); |
| 268 | 267 | } |
| 269 | 268 | |
| 270 | − return get_next_freq(sg_cpu, util, max); | |
| 269 | + return get_next_freq(sg_policy, util, max); | |
| 271 | 270 | } |
| 272 | 271 | |
| 273 | 272 | static void sugov_update_shared(struct update_util_data *hook, u64 time, |
| @@ -580,6 +579,7 @@ static int sugov_start(struct cpufreq_policy *policy) · 306 unmodified lines | ||
| 580 | 579 | sg_policy->next_freq = UINT_MAX; |
| 581 | 580 | sg_policy->work_in_progress = false; |
| 582 | 581 | sg_policy->need_freq_update = false; |
| 582 | + sg_policy->cached_raw_freq = 0; | |
| 583 | 583 | |
| 584 | 584 | for_each_cpu(cpu, policy->cpus) { |
| 585 | 585 | struct sugov_cpu *sg_cpu = &per_cpu(sugov_cpu, cpu); |
| @@ -590,7 +590,6 @@ static int sugov_start(struct cpufreq_policy *policy) · 4 unmodified lines | ||
| 590 | 590 | sg_cpu->max = 0; |
| 591 | 591 | sg_cpu->flags = SCHED_CPUFREQ_RT; |
| 592 | 592 | sg_cpu->last_update = 0; |
| 593 | − sg_cpu->cached_raw_freq = 0; | |
| 594 | 593 | sg_cpu->iowait_boost = 0; |
| 595 | 594 | sg_cpu->iowait_boost_max = policy->cpuinfo.max_freq; |
| 596 | 595 | cpufreq_add_update_util_hook(cpu, &sg_cpu->update_util, |
| @@ -5799,7 +5799,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t · 5798 unmodified lines | ||
| 5799 | 5799 | * Due to large variance we need a large fuzz factor; hackbench in |
| 5800 | 5800 | * particularly is sensitive here. |
| 5801 | 5801 | */ |
| 5802 | − if ((avg_idle / 512) < avg_cost) | |
| 5802 | + if (sched_feat(SIS_AVG_CPU) && (avg_idle / 512) < avg_cost) | |
| 5803 | 5803 | return -1; |
| 5804 | 5804 | |
| 5805 | 5805 | time = local_clock(); |
| @@ -51,6 +51,11 @@ SCHED_FEAT(NONTASK_CAPACITY, true) · 50 unmodified lines | ||
| 51 | 51 | */ |
| 52 | 52 | SCHED_FEAT(TTWU_QUEUE, true) |
| 53 | 53 | |
| 54 | +/* | |
| 55 | + * When doing wakeups, attempt to limit superfluous scans of the LLC domain. | |
| 56 | + */ | |
| 57 | +SCHED_FEAT(SIS_AVG_CPU, false) | |
| 58 | + | |
| 54 | 59 | #ifdef HAVE_RT_PUSH_IPI |
| 55 | 60 | /* |
| 56 | 61 | * In order to avoid a thundering herd attack of CPUs that are |
| @@ -242,6 +242,45 @@ long prepare_to_wait_event(wait_queue_head_t *q, wait_queue_t *wait, int state) · 241 unmodified lines | ||
| 242 | 242 | } |
| 243 | 243 | EXPORT_SYMBOL(prepare_to_wait_event); |
| 244 | 244 | |
| 245 | +/* | |
| 246 | + * Note! These two wait functions are entered with the | |
| 247 | + * wait-queue lock held (and interrupts off in the _irq | |
| 248 | + * case), so there is no race with testing the wakeup | |
| 249 | + * condition in the caller before they add the wait | |
| 250 | + * entry to the wake queue. | |
| 251 | + */ | |
| 252 | +int do_wait_intr(wait_queue_head_t *wq, wait_queue_t *wait) | |
| 253 | +{ | |
| 254 | + if (likely(list_empty(&wait->task_list))) | |
| 255 | + __add_wait_queue_tail(wq, wait); | |
| 256 | + | |
| 257 | + set_current_state(TASK_INTERRUPTIBLE); | |
| 258 | + if (signal_pending(current)) | |
| 259 | + return -ERESTARTSYS; | |
| 260 | + | |
| 261 | + spin_unlock(&wq->lock); | |
| 262 | + schedule(); | |
| 263 | + spin_lock(&wq->lock); | |
| 264 | + return 0; | |
| 265 | +} | |
| 266 | +EXPORT_SYMBOL(do_wait_intr); | |
| 267 | + | |
| 268 | +int do_wait_intr_irq(wait_queue_head_t *wq, wait_queue_t *wait) | |
| 269 | +{ | |
| 270 | + if (likely(list_empty(&wait->task_list))) | |
| 271 | + __add_wait_queue_tail(wq, wait); | |
| 272 | + | |
| 273 | + set_current_state(TASK_INTERRUPTIBLE); | |
| 274 | + if (signal_pending(current)) | |
| 275 | + return -ERESTARTSYS; | |
| 276 | + | |
| 277 | + spin_unlock_irq(&wq->lock); | |
| 278 | + schedule(); | |
| 279 | + spin_lock_irq(&wq->lock); | |
| 280 | + return 0; | |
| 281 | +} | |
| 282 | +EXPORT_SYMBOL(do_wait_intr_irq); | |
| 283 | + | |
| 245 | 284 | /** |
| 246 | 285 | * finish_wait - clean up after waiting in a queue |
| 247 | 286 | * @q: waitqueue waited on |
| @@ -125,7 +125,7 @@ int register_refined_jiffies(long cycles_per_second) · 124 unmodified lines | ||
| 125 | 125 | shift_hz += cycles_per_tick/2; |
| 126 | 126 | do_div(shift_hz, cycles_per_tick); |
| 127 | 127 | /* Calculate nsec_per_tick using shift_hz */ |
| 128 | − nsec_per_tick = (u64)TICK_NSEC << 8; | |
| 128 | + nsec_per_tick = (u64)NSEC_PER_SEC << 8; | |
| 129 | 129 | nsec_per_tick += (u32)shift_hz/2; |
| 130 | 130 | do_div(nsec_per_tick, (u32)shift_hz); |
| 131 | 131 | |
| @@ -4416,16 +4416,24 @@ static int __init set_graph_notrace_function(char *str) · 4415 unmodified lines | ||
| 4416 | 4416 | } |
| 4417 | 4417 | __setup("ftrace_graph_notrace=", set_graph_notrace_function); |
| 4418 | 4418 | |
| 4419 | +static int __init set_graph_max_depth_function(char *str) | |
| 4420 | +{ | |
| 4421 | + if (!str) | |
| 4422 | + return 0; | |
| 4423 | + fgraph_max_depth = simple_strtoul(str, NULL, 0); | |
| 4424 | + return 1; | |
| 4425 | +} | |
| 4426 | +__setup("ftrace_graph_max_depth=", set_graph_max_depth_function); | |
| 4427 | + | |
| 4419 | 4428 | static void __init set_ftrace_early_graph(char *buf, int enable) |
| 4420 | 4429 | { |
| 4421 | 4430 | int ret; |
| 4422 | 4431 | char *func; |
| 4423 | 4432 | struct ftrace_hash *hash; |
| 4424 | 4433 | |
| 4425 | − if (enable) | |
| 4426 | − hash = ftrace_graph_hash; | |
| 4427 | − else | |
| 4428 | − hash = ftrace_graph_notrace_hash; | |
| 4434 | + hash = alloc_ftrace_hash(FTRACE_HASH_DEFAULT_BITS); | |
| 4435 | + if (WARN_ON(!hash)) | |
| 4436 | + return; | |
| 4429 | 4437 | |
| 4430 | 4438 | while (buf) { |
| 4431 | 4439 | func = strsep(&buf, ","); |
| @@ -4435,6 +4443,11 @@ static void __init set_ftrace_early_graph(char *buf, int enable) · 3 unmodified lines | ||
| 4435 | 4443 | printk(KERN_DEBUG "ftrace: function %s not " |
| 4436 | 4444 | "traceable\n", func); |
| 4437 | 4445 | } |
| 4446 | + | |
| 4447 | + if (enable) | |
| 4448 | + ftrace_graph_hash = hash; | |
| 4449 | + else | |
| 4450 | + ftrace_graph_notrace_hash = hash; | |
| 4438 | 4451 | } |
| 4439 | 4452 | #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ |
| 4440 | 4453 | |
| @@ -5488,7 +5501,7 @@ static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip, · 1047 unmodified lines | ||
| 5488 | 5501 | * Normally the mcount trampoline will call the ops->func, but there |
| 5489 | 5502 | * are times that it should not. For example, if the ops does not |
| 5490 | 5503 | * have its own recursion protection, then it should call the |
| 5491 | − * ftrace_ops_recurs_func() instead. | |
| 5504 | + * ftrace_ops_assist_func() instead. | |
| 5492 | 5505 | * |
| 5493 | 5506 | * Returns the function that the trampoline should call for @ops. |
| 5494 | 5507 | */ |
| @@ -65,7 +65,7 @@ void stack_trace_print(void) · 64 unmodified lines | ||
| 65 | 65 | } |
| 66 | 66 | |
| 67 | 67 | /* |
| 68 | − * When arch-specific code overides this function, the following | |
| 68 | + * When arch-specific code overrides this function, the following | |
| 69 | 69 | * data should be filled up, assuming stack_trace_max_lock is held to |
| 70 | 70 | * prevent concurrent updates. |
| 71 | 71 | * stack_trace_index[] |
| @@ -144,7 +144,7 @@ static struct ucounts *get_ucounts(struct user_namespace *ns, kuid_t uid) · 143 unmodified lines | ||
| 144 | 144 | |
| 145 | 145 | new->ns = ns; |
| 146 | 146 | new->uid = uid; |
| 147 | − atomic_set(&new->count, 0); | |
| 147 | + new->count = 0; | |
| 148 | 148 | |
| 149 | 149 | spin_lock_irq(&ucounts_lock); |
| 150 | 150 | ucounts = find_ucounts(ns, uid, hashent); |
| @@ -155,8 +155,10 @@ static struct ucounts *get_ucounts(struct user_namespace *ns, kuid_t uid) · 4 unmodified lines | ||
| 155 | 155 | ucounts = new; |
| 156 | 156 | } |
| 157 | 157 | } |
| 158 | − if (!atomic_add_unless(&ucounts->count, 1, INT_MAX)) | |
| 158 | + if (ucounts->count == INT_MAX) | |
| 159 | 159 | ucounts = NULL; |
| 160 | + else | |
| 161 | + ucounts->count += 1; | |
| 160 | 162 | spin_unlock_irq(&ucounts_lock); |
| 161 | 163 | return ucounts; |
| 162 | 164 | } |
| @@ -165,13 +167,15 @@ static void put_ucounts(struct ucounts *ucounts) · 2 unmodified lines | ||
| 165 | 167 | { |
| 166 | 168 | unsigned long flags; |
| 167 | 169 | |
| 168 | − if (atomic_dec_and_test(&ucounts->count)) { | |
| 169 | − spin_lock_irqsave(&ucounts_lock, flags); | |
| 170 | + spin_lock_irqsave(&ucounts_lock, flags); | |
| 171 | + ucounts->count -= 1; | |
| 172 | + if (!ucounts->count) | |
| 170 | 173 | hlist_del_init(&ucounts->node); |
| 171 | − spin_unlock_irqrestore(&ucounts_lock, flags); | |
| 174 | + else | |
| 175 | + ucounts = NULL; | |
| 176 | + spin_unlock_irqrestore(&ucounts_lock, flags); | |
| 172 | 177 | |
| 173 | − kfree(ucounts); | |
| 174 | − } | |
| 178 | + kfree(ucounts); | |
| 175 | 179 | } |
| 176 | 180 | |
| 177 | 181 | static inline bool atomic_inc_below(atomic_t *v, int u) |
| @@ -1507,6 +1507,7 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq, · 1506 unmodified lines | ||
| 1507 | 1507 | struct timer_list *timer = &dwork->timer; |
| 1508 | 1508 | struct work_struct *work = &dwork->work; |
| 1509 | 1509 | |
| 1510 | + WARN_ON_ONCE(!wq); | |
| 1510 | 1511 | WARN_ON_ONCE(timer->function != delayed_work_timer_fn || |
| 1511 | 1512 | timer->data != (unsigned long)dwork); |
| 1512 | 1513 | WARN_ON_ONCE(timer_pending(timer)); |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <asm/pgtable.h> |
| 15 | 15 | |
| 16 | 16 | #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP |
| 17 | +static int __read_mostly ioremap_p4d_capable; | |
| 17 | 18 | static int __read_mostly ioremap_pud_capable; |
| 18 | 19 | static int __read_mostly ioremap_pmd_capable; |
| 19 | 20 | static int __read_mostly ioremap_huge_disabled; |
| @@ -35,6 +36,11 @@ void __init ioremap_huge_init(void) · 15 unmodified lines | ||
| 35 | 36 | } |
| 36 | 37 | } |
| 37 | 38 | |
| 39 | +static inline int ioremap_p4d_enabled(void) | |
| 40 | +{ | |
| 41 | + return ioremap_p4d_capable; | |
| 42 | +} | |
| 43 | + | |
| 38 | 44 | static inline int ioremap_pud_enabled(void) |
| 39 | 45 | { |
| 40 | 46 | return ioremap_pud_capable; |
| @@ -46,6 +52,7 @@ static inline int ioremap_pmd_enabled(void) · 5 unmodified lines | ||
| 46 | 52 | } |
| 47 | 53 | |
| 48 | 54 | #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ |
| 55 | +static inline int ioremap_p4d_enabled(void) { return 0; } | |
| 49 | 56 | static inline int ioremap_pud_enabled(void) { return 0; } |
| 50 | 57 | static inline int ioremap_pmd_enabled(void) { return 0; } |
| 51 | 58 | #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ |
| @@ -94,14 +101,14 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, · 42 unmodified lines | ||
| 94 | 101 | return 0; |
| 95 | 102 | } |
| 96 | 103 | |
| 97 | −static inline int ioremap_pud_range(pgd_t *pgd, unsigned long addr, | |
| 104 | +static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, | |
| 98 | 105 | unsigned long end, phys_addr_t phys_addr, pgprot_t prot) |
| 99 | 106 | { |
| 100 | 107 | pud_t *pud; |
| 101 | 108 | unsigned long next; |
| 102 | 109 | |
| 103 | 110 | phys_addr -= addr; |
| 104 | − pud = pud_alloc(&init_mm, pgd, addr); | |
| 111 | + pud = pud_alloc(&init_mm, p4d, addr); | |
| 105 | 112 | if (!pud) |
| 106 | 113 | return -ENOMEM; |
| 107 | 114 | do { |
| @@ -120,6 +127,32 @@ static inline int ioremap_pud_range(pgd_t *pgd, unsigned long addr, · 12 unmodified lines | ||
| 120 | 127 | return 0; |
| 121 | 128 | } |
| 122 | 129 | |
| 130 | +static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, | |
| 131 | + unsigned long end, phys_addr_t phys_addr, pgprot_t prot) | |
| 132 | +{ | |
| 133 | + p4d_t *p4d; | |
| 134 | + unsigned long next; | |
| 135 | + | |
| 136 | + phys_addr -= addr; | |
| 137 | + p4d = p4d_alloc(&init_mm, pgd, addr); | |
| 138 | + if (!p4d) | |
| 139 | + return -ENOMEM; | |
| 140 | + do { | |
| 141 | + next = p4d_addr_end(addr, end); | |
| 142 | + | |
| 143 | + if (ioremap_p4d_enabled() && | |
| 144 | + ((next - addr) == P4D_SIZE) && | |
| 145 | + IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { | |
| 146 | + if (p4d_set_huge(p4d, phys_addr + addr, prot)) | |
| 147 | + continue; | |
| 148 | + } | |
| 149 | + | |
| 150 | + if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) | |
| 151 | + return -ENOMEM; | |
| 152 | + } while (p4d++, addr = next, addr != end); | |
| 153 | + return 0; | |
| 154 | +} | |
| 155 | + | |
| 123 | 156 | int ioremap_page_range(unsigned long addr, |
| 124 | 157 | unsigned long end, phys_addr_t phys_addr, pgprot_t prot) |
| 125 | 158 | { |
| @@ -135,7 +168,7 @@ int ioremap_page_range(unsigned long addr, · 9 unmodified lines | ||
| 135 | 168 | pgd = pgd_offset_k(addr); |
| 136 | 169 | do { |
| 137 | 170 | next = pgd_addr_end(addr, end); |
| 138 | − err = ioremap_pud_range(pgd, addr, next, phys_addr+addr, prot); | |
| 171 | + err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); | |
| 139 | 172 | if (err) |
| 140 | 173 | break; |
| 141 | 174 | } while (pgd++, addr = next, addr != end); |
| @@ -2129,8 +2129,8 @@ int ida_pre_get(struct ida *ida, gfp_t gfp) · 2128 unmodified lines | ||
| 2129 | 2129 | struct ida_bitmap *bitmap = kmalloc(sizeof(*bitmap), gfp); |
| 2130 | 2130 | if (!bitmap) |
| 2131 | 2131 | return 0; |
| 2132 | − bitmap = this_cpu_cmpxchg(ida_bitmap, NULL, bitmap); | |
| 2133 | − kfree(bitmap); | |
| 2132 | + if (this_cpu_cmpxchg(ida_bitmap, NULL, bitmap)) | |
| 2133 | + kfree(bitmap); | |
| 2134 | 2134 | } |
| 2135 | 2135 | |
| 2136 | 2136 | return 1; |
| @@ -58,7 +58,7 @@ bool refcount_add_not_zero(unsigned int i, refcount_t *r) · 57 unmodified lines | ||
| 58 | 58 | val = old; |
| 59 | 59 | } |
| 60 | 60 | |
| 61 | − WARN(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n"); | |
| 61 | + WARN_ONCE(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n"); | |
| 62 | 62 | |
| 63 | 63 | return true; |
| 64 | 64 | } |
| @@ -66,7 +66,7 @@ EXPORT_SYMBOL_GPL(refcount_add_not_zero); · 1 unmodified line | ||
| 66 | 66 | |
| 67 | 67 | void refcount_add(unsigned int i, refcount_t *r) |
| 68 | 68 | { |
| 69 | − WARN(!refcount_add_not_zero(i, r), "refcount_t: addition on 0; use-after-free.\n"); | |
| 69 | + WARN_ONCE(!refcount_add_not_zero(i, r), "refcount_t: addition on 0; use-after-free.\n"); | |
| 70 | 70 | } |
| 71 | 71 | EXPORT_SYMBOL_GPL(refcount_add); |
| 72 | 72 | |
| @@ -97,7 +97,7 @@ bool refcount_inc_not_zero(refcount_t *r) · 24 unmodified lines | ||
| 97 | 97 | val = old; |
| 98 | 98 | } |
| 99 | 99 | |
| 100 | − WARN(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n"); | |
| 100 | + WARN_ONCE(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n"); | |
| 101 | 101 | |
| 102 | 102 | return true; |
| 103 | 103 | } |
| @@ -111,7 +111,7 @@ EXPORT_SYMBOL_GPL(refcount_inc_not_zero); · 7 unmodified lines | ||
| 111 | 111 | */ |
| 112 | 112 | void refcount_inc(refcount_t *r) |
| 113 | 113 | { |
| 114 | − WARN(!refcount_inc_not_zero(r), "refcount_t: increment on 0; use-after-free.\n"); | |
| 114 | + WARN_ONCE(!refcount_inc_not_zero(r), "refcount_t: increment on 0; use-after-free.\n"); | |
| 115 | 115 | } |
| 116 | 116 | EXPORT_SYMBOL_GPL(refcount_inc); |
| 117 | 117 | |
| @@ -125,7 +125,7 @@ bool refcount_sub_and_test(unsigned int i, refcount_t *r) · 7 unmodified lines | ||
| 125 | 125 | |
| 126 | 126 | new = val - i; |
| 127 | 127 | if (new > val) { |
| 128 | − WARN(new > val, "refcount_t: underflow; use-after-free.\n"); | |
| 128 | + WARN_ONCE(new > val, "refcount_t: underflow; use-after-free.\n"); | |
| 129 | 129 | return false; |
| 130 | 130 | } |
| 131 | 131 | |
| @@ -164,7 +164,7 @@ EXPORT_SYMBOL_GPL(refcount_dec_and_test); · 32 unmodified lines | ||
| 164 | 164 | |
| 165 | 165 | void refcount_dec(refcount_t *r) |
| 166 | 166 | { |
| 167 | − WARN(refcount_dec_and_test(r), "refcount_t: decrement hit 0; leaking memory.\n"); | |
| 167 | + WARN_ONCE(refcount_dec_and_test(r), "refcount_t: decrement hit 0; leaking memory.\n"); | |
| 168 | 168 | } |
| 169 | 169 | EXPORT_SYMBOL_GPL(refcount_dec); |
| 170 | 170 | |
| @@ -204,7 +204,7 @@ bool refcount_dec_not_one(refcount_t *r) · 33 unmodified lines | ||
| 204 | 204 | |
| 205 | 205 | new = val - 1; |
| 206 | 206 | if (new > val) { |
| 207 | − WARN(new > val, "refcount_t: underflow; use-after-free.\n"); | |
| 207 | + WARN_ONCE(new > val, "refcount_t: underflow; use-after-free.\n"); | |
| 208 | 208 | return true; |
| 209 | 209 | } |
| 210 | 210 | |
| @@ -683,33 +683,26 @@ static int cgwb_bdi_init(struct backing_dev_info *bdi) · 682 unmodified lines | ||
| 683 | 683 | static void cgwb_bdi_destroy(struct backing_dev_info *bdi) |
| 684 | 684 | { |
| 685 | 685 | struct radix_tree_iter iter; |
| 686 | − struct rb_node *rbn; | |
| 687 | 686 | void **slot; |
| 688 | 687 | |
| 689 | 688 | WARN_ON(test_bit(WB_registered, &bdi->wb.state)); |
| 690 | 689 | |
| 691 | 690 | spin_lock_irq(&cgwb_lock); |
| 692 | − | |
| 693 | 691 | radix_tree_for_each_slot(slot, &bdi->cgwb_tree, &iter, 0) |
| 694 | 692 | cgwb_kill(*slot); |
| 695 | − | |
| 696 | − while ((rbn = rb_first(&bdi->cgwb_congested_tree))) { | |
| 697 | − struct bdi_writeback_congested *congested = | |
| 698 | − rb_entry(rbn, struct bdi_writeback_congested, rb_node); | |
| 699 | − | |
| 700 | − rb_erase(rbn, &bdi->cgwb_congested_tree); | |
| 701 | − congested->bdi = NULL; /* mark @congested unlinked */ | |
| 702 | − } | |
| 703 | − | |
| 704 | 693 | spin_unlock_irq(&cgwb_lock); |
| 705 | 694 | |
| 706 | 695 | /* |
| 707 | − * All cgwb's and their congested states must be shutdown and | |
| 708 | − * released before returning. Drain the usage counter to wait for | |
| 709 | − * all cgwb's and cgwb_congested's ever created on @bdi. | |
| 696 | + * All cgwb's must be shutdown and released before returning. Drain | |
| 697 | + * the usage counter to wait for all cgwb's ever created on @bdi. | |
| 710 | 698 | */ |
| 711 | 699 | atomic_dec(&bdi->usage_cnt); |
| 712 | 700 | wait_event(cgwb_release_wait, !atomic_read(&bdi->usage_cnt)); |
| 701 | + /* | |
| 702 | + * Grab back our reference so that we hold it when @bdi gets | |
| 703 | + * re-registered. | |
| 704 | + */ | |
| 705 | + atomic_inc(&bdi->usage_cnt); | |
| 713 | 706 | } |
| 714 | 707 | |
| 715 | 708 | /** |
| @@ -749,6 +742,21 @@ void wb_blkcg_offline(struct blkcg *blkcg) · 33 unmodified lines | ||
| 749 | 742 | spin_unlock_irq(&cgwb_lock); |
| 750 | 743 | } |
| 751 | 744 | |
| 745 | +static void cgwb_bdi_exit(struct backing_dev_info *bdi) | |
| 746 | +{ | |
| 747 | + struct rb_node *rbn; | |
| 748 | + | |
| 749 | + spin_lock_irq(&cgwb_lock); | |
| 750 | + while ((rbn = rb_first(&bdi->cgwb_congested_tree))) { | |
| 751 | + struct bdi_writeback_congested *congested = | |
| 752 | + rb_entry(rbn, struct bdi_writeback_congested, rb_node); | |
| 753 | + | |
| 754 | + rb_erase(rbn, &bdi->cgwb_congested_tree); | |
| 755 | + congested->bdi = NULL; /* mark @congested unlinked */ | |
| 756 | + } | |
| 757 | + spin_unlock_irq(&cgwb_lock); | |
| 758 | +} | |
| 759 | + | |
| 752 | 760 | #else /* CONFIG_CGROUP_WRITEBACK */ |
| 753 | 761 | |
| 754 | 762 | static int cgwb_bdi_init(struct backing_dev_info *bdi) |
| @@ -769,7 +777,9 @@ static int cgwb_bdi_init(struct backing_dev_info *bdi) · 14 unmodified lines | ||
| 769 | 777 | return 0; |
| 770 | 778 | } |
| 771 | 779 | |
| 772 | −static void cgwb_bdi_destroy(struct backing_dev_info *bdi) | |
| 780 | +static void cgwb_bdi_destroy(struct backing_dev_info *bdi) { } | |
| 781 | + | |
| 782 | +static void cgwb_bdi_exit(struct backing_dev_info *bdi) | |
| 773 | 783 | { |
| 774 | 784 | wb_congested_put(bdi->wb_congested); |
| 775 | 785 | } |
| @@ -857,6 +867,8 @@ int bdi_register_owner(struct backing_dev_info *bdi, struct device *owner) · 81 unmodified lines | ||
| 857 | 867 | MINOR(owner->devt)); |
| 858 | 868 | if (rc) |
| 859 | 869 | return rc; |
| 870 | + /* Leaking owner reference... */ | |
| 871 | + WARN_ON(bdi->owner); | |
| 860 | 872 | bdi->owner = owner; |
| 861 | 873 | get_device(owner); |
| 862 | 874 | return 0; |
| @@ -898,6 +910,7 @@ static void bdi_exit(struct backing_dev_info *bdi) · 35 unmodified lines | ||
| 898 | 910 | { |
| 899 | 911 | WARN_ON_ONCE(bdi->dev); |
| 900 | 912 | wb_exit(&bdi->wb); |
| 913 | + cgwb_bdi_exit(bdi); | |
| 901 | 914 | } |
| 902 | 915 | |
| 903 | 916 | static void release_bdi(struct kref *ref) |
| @@ -226,6 +226,7 @@ struct page *follow_page_mask(struct vm_area_struct *vma, · 225 unmodified lines | ||
| 226 | 226 | unsigned int *page_mask) |
| 227 | 227 | { |
| 228 | 228 | pgd_t *pgd; |
| 229 | + p4d_t *p4d; | |
| 229 | 230 | pud_t *pud; |
| 230 | 231 | pmd_t *pmd; |
| 231 | 232 | spinlock_t *ptl; |
| @@ -243,8 +244,13 @@ struct page *follow_page_mask(struct vm_area_struct *vma, · 11 unmodified lines | ||
| 243 | 244 | pgd = pgd_offset(mm, address); |
| 244 | 245 | if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd))) |
| 245 | 246 | return no_page_table(vma, flags); |
| 246 | − | |
| 247 | − pud = pud_offset(pgd, address); | |
| 247 | + p4d = p4d_offset(pgd, address); | |
| 248 | + if (p4d_none(*p4d)) | |
| 249 | + return no_page_table(vma, flags); | |
| 250 | + BUILD_BUG_ON(p4d_huge(*p4d)); | |
| 251 | + if (unlikely(p4d_bad(*p4d))) | |
| 252 | + return no_page_table(vma, flags); | |
| 253 | + pud = pud_offset(p4d, address); | |
| 248 | 254 | if (pud_none(*pud)) |
| 249 | 255 | return no_page_table(vma, flags); |
| 250 | 256 | if (pud_huge(*pud) && vma->vm_flags & VM_HUGETLB) { |
| @@ -325,6 +331,7 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address, · 74 unmodified lines | ||
| 325 | 331 | struct page **page) |
| 326 | 332 | { |
| 327 | 333 | pgd_t *pgd; |
| 334 | + p4d_t *p4d; | |
| 328 | 335 | pud_t *pud; |
| 329 | 336 | pmd_t *pmd; |
| 330 | 337 | pte_t *pte; |
| @@ -338,7 +345,9 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address, · 7 unmodified lines | ||
| 338 | 345 | else |
| 339 | 346 | pgd = pgd_offset_gate(mm, address); |
| 340 | 347 | BUG_ON(pgd_none(*pgd)); |
| 341 | − pud = pud_offset(pgd, address); | |
| 348 | + p4d = p4d_offset(pgd, address); | |
| 349 | + BUG_ON(p4d_none(*p4d)); | |
| 350 | + pud = pud_offset(p4d, address); | |
| 342 | 351 | BUG_ON(pud_none(*pud)); |
| 343 | 352 | pmd = pmd_offset(pud, address); |
| 344 | 353 | if (pmd_none(*pmd)) |
| @@ -1400,13 +1409,13 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end, · 1055 unmodified lines | ||
| 1400 | 1409 | return 1; |
| 1401 | 1410 | } |
| 1402 | 1411 | |
| 1403 | −static int gup_pud_range(pgd_t pgd, unsigned long addr, unsigned long end, | |
| 1412 | +static int gup_pud_range(p4d_t p4d, unsigned long addr, unsigned long end, | |
| 1404 | 1413 | int write, struct page **pages, int *nr) |
| 1405 | 1414 | { |
| 1406 | 1415 | unsigned long next; |
| 1407 | 1416 | pud_t *pudp; |
| 1408 | 1417 | |
| 1409 | − pudp = pud_offset(&pgd, addr); | |
| 1418 | + pudp = pud_offset(&p4d, addr); | |
| 1410 | 1419 | do { |
| 1411 | 1420 | pud_t pud = READ_ONCE(*pudp); |
| 1412 | 1421 | |
| @@ -1428,6 +1437,31 @@ static int gup_pud_range(pgd_t pgd, unsigned long addr, unsigned long end, · 15 unmodified lines | ||
| 1428 | 1437 | return 1; |
| 1429 | 1438 | } |
| 1430 | 1439 | |
| 1440 | +static int gup_p4d_range(pgd_t pgd, unsigned long addr, unsigned long end, | |
| 1441 | + int write, struct page **pages, int *nr) | |
| 1442 | +{ | |
| 1443 | + unsigned long next; | |
| 1444 | + p4d_t *p4dp; | |
| 1445 | + | |
| 1446 | + p4dp = p4d_offset(&pgd, addr); | |
| 1447 | + do { | |
| 1448 | + p4d_t p4d = READ_ONCE(*p4dp); | |
| 1449 | + | |
| 1450 | + next = p4d_addr_end(addr, end); | |
| 1451 | + if (p4d_none(p4d)) | |
| 1452 | + return 0; | |
| 1453 | + BUILD_BUG_ON(p4d_huge(p4d)); | |
| 1454 | + if (unlikely(is_hugepd(__hugepd(p4d_val(p4d))))) { | |
| 1455 | + if (!gup_huge_pd(__hugepd(p4d_val(p4d)), addr, | |
| 1456 | + P4D_SHIFT, next, write, pages, nr)) | |
| 1457 | + return 0; | |
| 1458 | + } else if (!gup_pud_range(p4d, addr, next, write, pages, nr)) | |
| 1459 | + return 0; | |
| 1460 | + } while (p4dp++, addr = next, addr != end); | |
| 1461 | + | |
| 1462 | + return 1; | |
| 1463 | +} | |
| 1464 | + | |
| 1431 | 1465 | /* |
| 1432 | 1466 | * Like get_user_pages_fast() except it's IRQ-safe in that it won't fall back to |
| 1433 | 1467 | * the regular GUP. It will only return non-negative values. |
| @@ -1478,7 +1512,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, · 44 unmodified lines | ||
| 1478 | 1512 | if (!gup_huge_pd(__hugepd(pgd_val(pgd)), addr, |
| 1479 | 1513 | PGDIR_SHIFT, next, write, pages, &nr)) |
| 1480 | 1514 | break; |
| 1481 | − } else if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) | |
| 1515 | + } else if (!gup_p4d_range(pgd, addr, next, write, pages, &nr)) | |
| 1482 | 1516 | break; |
| 1483 | 1517 | } while (pgdp++, addr = next, addr != end); |
| 1484 | 1518 | local_irq_restore(flags); |
| @@ -1828,7 +1828,7 @@ static void __split_huge_pud_locked(struct vm_area_struct *vma, pud_t *pud, · 1827 unmodified lines | ||
| 1828 | 1828 | VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PUD_SIZE, vma); |
| 1829 | 1829 | VM_BUG_ON(!pud_trans_huge(*pud) && !pud_devmap(*pud)); |
| 1830 | 1830 | |
| 1831 | − count_vm_event(THP_SPLIT_PMD); | |
| 1831 | + count_vm_event(THP_SPLIT_PUD); | |
| 1832 | 1832 | |
| 1833 | 1833 | pudp_huge_clear_flush_notify(vma, haddr, pud); |
| 1834 | 1834 | } |
| @@ -2048,6 +2048,7 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address, · 213 unmodified lines | ||
| 2048 | 2048 | bool freeze, struct page *page) |
| 2049 | 2049 | { |
| 2050 | 2050 | pgd_t *pgd; |
| 2051 | + p4d_t *p4d; | |
| 2051 | 2052 | pud_t *pud; |
| 2052 | 2053 | pmd_t *pmd; |
| 2053 | 2054 | |
| @@ -2055,7 +2056,11 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address, · 1 unmodified line | ||
| 2055 | 2056 | if (!pgd_present(*pgd)) |
| 2056 | 2057 | return; |
| 2057 | 2058 | |
| 2058 | − pud = pud_offset(pgd, address); | |
| 2059 | + p4d = p4d_offset(pgd, address); | |
| 2060 | + if (!p4d_present(*p4d)) | |
| 2061 | + return; | |
| 2062 | + | |
| 2063 | + pud = pud_offset(p4d, address); | |
| 2059 | 2064 | if (!pud_present(*pud)) |
| 2060 | 2065 | return; |
| 2061 | 2066 | |
| @@ -4555,7 +4555,8 @@ out: · 4554 unmodified lines | ||
| 4555 | 4555 | int huge_pmd_unshare(struct mm_struct *mm, unsigned long *addr, pte_t *ptep) |
| 4556 | 4556 | { |
| 4557 | 4557 | pgd_t *pgd = pgd_offset(mm, *addr); |
| 4558 | − pud_t *pud = pud_offset(pgd, *addr); | |
| 4558 | + p4d_t *p4d = p4d_offset(pgd, *addr); | |
| 4559 | + pud_t *pud = pud_offset(p4d, *addr); | |
| 4559 | 4560 | |
| 4560 | 4561 | BUG_ON(page_count(virt_to_page(ptep)) == 0); |
| 4561 | 4562 | if (page_count(virt_to_page(ptep)) == 1) |
| @@ -4586,11 +4587,13 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, · 24 unmodified lines | ||
| 4586 | 4587 | unsigned long addr, unsigned long sz) |
| 4587 | 4588 | { |
| 4588 | 4589 | pgd_t *pgd; |
| 4590 | + p4d_t *p4d; | |
| 4589 | 4591 | pud_t *pud; |
| 4590 | 4592 | pte_t *pte = NULL; |
| 4591 | 4593 | |
| 4592 | 4594 | pgd = pgd_offset(mm, addr); |
| 4593 | − pud = pud_alloc(mm, pgd, addr); | |
| 4595 | + p4d = p4d_offset(pgd, addr); | |
| 4596 | + pud = pud_alloc(mm, p4d, addr); | |
| 4594 | 4597 | if (pud) { |
| 4595 | 4598 | if (sz == PUD_SIZE) { |
| 4596 | 4599 | pte = (pte_t *)pud; |
| @@ -4610,18 +4613,22 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, · 13 unmodified lines | ||
| 4610 | 4613 | pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr) |
| 4611 | 4614 | { |
| 4612 | 4615 | pgd_t *pgd; |
| 4616 | + p4d_t *p4d; | |
| 4613 | 4617 | pud_t *pud; |
| 4614 | − pmd_t *pmd = NULL; | |
| 4618 | + pmd_t *pmd; | |
| 4615 | 4619 | |
| 4616 | 4620 | pgd = pgd_offset(mm, addr); |
| 4617 | − if (pgd_present(*pgd)) { | |
| 4618 | − pud = pud_offset(pgd, addr); | |
| 4619 | − if (pud_present(*pud)) { | |
| 4620 | − if (pud_huge(*pud)) | |
| 4621 | − return (pte_t *)pud; | |
| 4622 | − pmd = pmd_offset(pud, addr); | |
| 4623 | − } | |
| 4624 | − } | |
| 4621 | + if (!pgd_present(*pgd)) | |
| 4622 | + return NULL; | |
| 4623 | + p4d = p4d_offset(pgd, addr); | |
| 4624 | + if (!p4d_present(*p4d)) | |
| 4625 | + return NULL; | |
| 4626 | + pud = pud_offset(p4d, addr); | |
| 4627 | + if (!pud_present(*pud)) | |
| 4628 | + return NULL; | |
| 4629 | + if (pud_huge(*pud)) | |
| 4630 | + return (pte_t *)pud; | |
| 4631 | + pmd = pmd_offset(pud, addr); | |
| 4625 | 4632 | return (pte_t *) pmd; |
| 4626 | 4633 | } |
| 4627 | 4634 | |
| @@ -30,6 +30,9 @@ · 29 unmodified lines | ||
| 30 | 30 | */ |
| 31 | 31 | unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; |
| 32 | 32 | |
| 33 | +#if CONFIG_PGTABLE_LEVELS > 4 | |
| 34 | +p4d_t kasan_zero_p4d[PTRS_PER_P4D] __page_aligned_bss; | |
| 35 | +#endif | |
| 33 | 36 | #if CONFIG_PGTABLE_LEVELS > 3 |
| 34 | 37 | pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; |
| 35 | 38 | #endif |
| @@ -82,10 +85,10 @@ static void __init zero_pmd_populate(pud_t *pud, unsigned long addr, · 46 unmodified lines | ||
| 82 | 85 | } while (pmd++, addr = next, addr != end); |
| 83 | 86 | } |
| 84 | 87 | |
| 85 | −static void __init zero_pud_populate(pgd_t *pgd, unsigned long addr, | |
| 88 | +static void __init zero_pud_populate(p4d_t *p4d, unsigned long addr, | |
| 86 | 89 | unsigned long end) |
| 87 | 90 | { |
| 88 | − pud_t *pud = pud_offset(pgd, addr); | |
| 91 | + pud_t *pud = pud_offset(p4d, addr); | |
| 89 | 92 | unsigned long next; |
| 90 | 93 | |
| 91 | 94 | do { |
| @@ -107,6 +110,23 @@ static void __init zero_pud_populate(pgd_t *pgd, unsigned long addr, · 15 unmodified lines | ||
| 107 | 110 | } while (pud++, addr = next, addr != end); |
| 108 | 111 | } |
| 109 | 112 | |
| 113 | +static void __init zero_p4d_populate(pgd_t *pgd, unsigned long addr, | |
| 114 | + unsigned long end) | |
| 115 | +{ | |
| 116 | + p4d_t *p4d = p4d_offset(pgd, addr); | |
| 117 | + unsigned long next; | |
| 118 | + | |
| 119 | + do { | |
| 120 | + next = p4d_addr_end(addr, end); | |
| 121 | + | |
| 122 | + if (p4d_none(*p4d)) { | |
| 123 | + p4d_populate(&init_mm, p4d, | |
| 124 | + early_alloc(PAGE_SIZE, NUMA_NO_NODE)); | |
| 125 | + } | |
| 126 | + zero_pud_populate(p4d, addr, next); | |
| 127 | + } while (p4d++, addr = next, addr != end); | |
| 128 | +} | |
| 129 | + | |
| 110 | 130 | /** |
| 111 | 131 | * kasan_populate_zero_shadow - populate shadow memory region with |
| 112 | 132 | * kasan_zero_page |
| @@ -125,6 +145,7 @@ void __init kasan_populate_zero_shadow(const void *shadow_start, · 12 unmodified lines | ||
| 125 | 145 | next = pgd_addr_end(addr, end); |
| 126 | 146 | |
| 127 | 147 | if (IS_ALIGNED(addr, PGDIR_SIZE) && end - addr >= PGDIR_SIZE) { |
| 148 | + p4d_t *p4d; | |
| 128 | 149 | pud_t *pud; |
| 129 | 150 | pmd_t *pmd; |
| 130 | 151 | |
| @@ -135,9 +156,22 @@ void __init kasan_populate_zero_shadow(const void *shadow_start, · 4 unmodified lines | ||
| 135 | 156 | * 3,2 - level page tables where we don't have |
| 136 | 157 | * puds,pmds, so pgd_populate(), pud_populate() |
| 137 | 158 | * is noops. |
| 159 | + * | |
| 160 | + * The ifndef is required to avoid build breakage. | |
| 161 | + * | |
| 162 | + * With 5level-fixup.h, pgd_populate() is not nop and | |
| 163 | + * we reference kasan_zero_p4d. It's not defined | |
| 164 | + * unless 5-level paging enabled. | |
| 165 | + * | |
| 166 | + * The ifndef can be dropped once all KASAN-enabled | |
| 167 | + * architectures will switch to pgtable-nop4d.h. | |
| 138 | 168 | */ |
| 139 | − pgd_populate(&init_mm, pgd, lm_alias(kasan_zero_pud)); | |
| 140 | − pud = pud_offset(pgd, addr); | |
| 169 | +#ifndef __ARCH_HAS_5LEVEL_HACK | |
| 170 | + pgd_populate(&init_mm, pgd, lm_alias(kasan_zero_p4d)); | |
| 171 | +#endif | |
| 172 | + p4d = p4d_offset(pgd, addr); | |
| 173 | + p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); | |
| 174 | + pud = pud_offset(p4d, addr); | |
| 141 | 175 | pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); |
| 142 | 176 | pmd = pmd_offset(pud, addr); |
| 143 | 177 | pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); |
| @@ -148,6 +182,6 @@ void __init kasan_populate_zero_shadow(const void *shadow_start, · 4 unmodified lines | ||
| 148 | 182 | pgd_populate(&init_mm, pgd, |
| 149 | 183 | early_alloc(PAGE_SIZE, NUMA_NO_NODE)); |
| 150 | 184 | } |
| 151 | − zero_pud_populate(pgd, addr, next); | |
| 185 | + zero_p4d_populate(pgd, addr, next); | |
| 152 | 186 | } while (pgd++, addr = next, addr != end); |
| 153 | 187 | } |
| @@ -25,6 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/printk.h> |
| 26 | 26 | #include <linux/shrinker.h> |
| 27 | 27 | #include <linux/slab.h> |
| 28 | +#include <linux/srcu.h> | |
| 28 | 29 | #include <linux/string.h> |
| 29 | 30 | #include <linux/types.h> |
| 30 | 31 | |
| @@ -103,6 +104,7 @@ static int quarantine_tail; · 72 unmodified lines | ||
| 103 | 104 | /* Total size of all objects in global_quarantine across all batches. */ |
| 104 | 105 | static unsigned long quarantine_size; |
| 105 | 106 | static DEFINE_SPINLOCK(quarantine_lock); |
| 107 | +DEFINE_STATIC_SRCU(remove_cache_srcu); | |
| 106 | 108 | |
| 107 | 109 | /* Maximum size of the global queue. */ |
| 108 | 110 | static unsigned long quarantine_max_size; |
| @@ -173,17 +175,22 @@ void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) · 64 unmodified lines | ||
| 173 | 175 | struct qlist_head *q; |
| 174 | 176 | struct qlist_head temp = QLIST_INIT; |
| 175 | 177 | |
| 178 | + /* | |
| 179 | + * Note: irq must be disabled until after we move the batch to the | |
| 180 | + * global quarantine. Otherwise quarantine_remove_cache() can miss | |
| 181 | + * some objects belonging to the cache if they are in our local temp | |
| 182 | + * list. quarantine_remove_cache() executes on_each_cpu() at the | |
| 183 | + * beginning which ensures that it either sees the objects in per-cpu | |
| 184 | + * lists or in the global quarantine. | |
| 185 | + */ | |
| 176 | 186 | local_irq_save(flags); |
| 177 | 187 | |
| 178 | 188 | q = this_cpu_ptr(&cpu_quarantine); |
| 179 | 189 | qlist_put(q, &info->quarantine_link, cache->size); |
| 180 | − if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) | |
| 190 | + if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { | |
| 181 | 191 | qlist_move_all(q, &temp); |
| 182 | 192 | |
| 183 | − local_irq_restore(flags); | |
| 184 | − | |
| 185 | − if (unlikely(!qlist_empty(&temp))) { | |
| 186 | − spin_lock_irqsave(&quarantine_lock, flags); | |
| 193 | + spin_lock(&quarantine_lock); | |
| 187 | 194 | WRITE_ONCE(quarantine_size, quarantine_size + temp.bytes); |
| 188 | 195 | qlist_move_all(&temp, &global_quarantine[quarantine_tail]); |
| 189 | 196 | if (global_quarantine[quarantine_tail].bytes >= |
| @@ -196,20 +203,33 @@ void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) · 6 unmodified lines | ||
| 196 | 203 | if (new_tail != quarantine_head) |
| 197 | 204 | quarantine_tail = new_tail; |
| 198 | 205 | } |
| 199 | − spin_unlock_irqrestore(&quarantine_lock, flags); | |
| 206 | + spin_unlock(&quarantine_lock); | |
| 200 | 207 | } |
| 208 | + | |
| 209 | + local_irq_restore(flags); | |
| 201 | 210 | } |
| 202 | 211 | |
| 203 | 212 | void quarantine_reduce(void) |
| 204 | 213 | { |
| 205 | 214 | size_t total_size, new_quarantine_size, percpu_quarantines; |
| 206 | 215 | unsigned long flags; |
| 216 | + int srcu_idx; | |
| 207 | 217 | struct qlist_head to_free = QLIST_INIT; |
| 208 | 218 | |
| 209 | 219 | if (likely(READ_ONCE(quarantine_size) <= |
| 210 | 220 | READ_ONCE(quarantine_max_size))) |
| 211 | 221 | return; |
| 212 | 222 | |
| 223 | + /* | |
| 224 | + * srcu critical section ensures that quarantine_remove_cache() | |
| 225 | + * will not miss objects belonging to the cache while they are in our | |
| 226 | + * local to_free list. srcu is chosen because (1) it gives us private | |
| 227 | + * grace period domain that does not interfere with anything else, | |
| 228 | + * and (2) it allows synchronize_srcu() to return without waiting | |
| 229 | + * if there are no pending read critical sections (which is the | |
| 230 | + * expected case). | |
| 231 | + */ | |
| 232 | + srcu_idx = srcu_read_lock(&remove_cache_srcu); | |
| 213 | 233 | spin_lock_irqsave(&quarantine_lock, flags); |
| 214 | 234 | |
| 215 | 235 | /* |
| @@ -237,6 +257,7 @@ void quarantine_reduce(void) · 21 unmodified lines | ||
| 237 | 257 | spin_unlock_irqrestore(&quarantine_lock, flags); |
| 238 | 258 | |
| 239 | 259 | qlist_free_all(&to_free, NULL); |
| 260 | + srcu_read_unlock(&remove_cache_srcu, srcu_idx); | |
| 240 | 261 | } |
| 241 | 262 | |
| 242 | 263 | static void qlist_move_cache(struct qlist_head *from, |
| @@ -280,12 +301,28 @@ void quarantine_remove_cache(struct kmem_cache *cache) · 37 unmodified lines | ||
| 280 | 301 | unsigned long flags, i; |
| 281 | 302 | struct qlist_head to_free = QLIST_INIT; |
| 282 | 303 | |
| 304 | + /* | |
| 305 | + * Must be careful to not miss any objects that are being moved from | |
| 306 | + * per-cpu list to the global quarantine in quarantine_put(), | |
| 307 | + * nor objects being freed in quarantine_reduce(). on_each_cpu() | |
| 308 | + * achieves the first goal, while synchronize_srcu() achieves the | |
| 309 | + * second. | |
| 310 | + */ | |
| 283 | 311 | on_each_cpu(per_cpu_remove_cache, cache, 1); |
| 284 | 312 | |
| 285 | 313 | spin_lock_irqsave(&quarantine_lock, flags); |
| 286 | − for (i = 0; i < QUARANTINE_BATCHES; i++) | |
| 314 | + for (i = 0; i < QUARANTINE_BATCHES; i++) { | |
| 315 | + if (qlist_empty(&global_quarantine[i])) | |
| 316 | + continue; | |
| 287 | 317 | qlist_move_cache(&global_quarantine[i], &to_free, cache); |
| 318 | + /* Scanning whole quarantine can take a while. */ | |
| 319 | + spin_unlock_irqrestore(&quarantine_lock, flags); | |
| 320 | + cond_resched(); | |
| 321 | + spin_lock_irqsave(&quarantine_lock, flags); | |
| 322 | + } | |
| 288 | 323 | spin_unlock_irqrestore(&quarantine_lock, flags); |
| 289 | 324 | |
| 290 | 325 | qlist_free_all(&to_free, cache); |
| 326 | + | |
| 327 | + synchronize_srcu(&remove_cache_srcu); | |
| 291 | 328 | } |
| @@ -513,7 +513,43 @@ static long madvise_dontneed(struct vm_area_struct *vma, · 512 unmodified lines | ||
| 513 | 513 | if (!can_madv_dontneed_vma(vma)) |
| 514 | 514 | return -EINVAL; |
| 515 | 515 | |
| 516 | − userfaultfd_remove(vma, prev, start, end); | |
| 516 | + if (!userfaultfd_remove(vma, start, end)) { | |
| 517 | + *prev = NULL; /* mmap_sem has been dropped, prev is stale */ | |
| 518 | + | |
| 519 | + down_read(¤t->mm->mmap_sem); | |
| 520 | + vma = find_vma(current->mm, start); | |
| 521 | + if (!vma) | |
| 522 | + return -ENOMEM; | |
| 523 | + if (start < vma->vm_start) { | |
| 524 | + /* | |
| 525 | + * This "vma" under revalidation is the one | |
| 526 | + * with the lowest vma->vm_start where start | |
| 527 | + * is also < vma->vm_end. If start < | |
| 528 | + * vma->vm_start it means an hole materialized | |
| 529 | + * in the user address space within the | |
| 530 | + * virtual range passed to MADV_DONTNEED. | |
| 531 | + */ | |
| 532 | + return -ENOMEM; | |
| 533 | + } | |
| 534 | + if (!can_madv_dontneed_vma(vma)) | |
| 535 | + return -EINVAL; | |
| 536 | + if (end > vma->vm_end) { | |
| 537 | + /* | |
| 538 | + * Don't fail if end > vma->vm_end. If the old | |
| 539 | + * vma was splitted while the mmap_sem was | |
| 540 | + * released the effect of the concurrent | |
| 541 | + * operation may not cause MADV_DONTNEED to | |
| 542 | + * have an undefined result. There may be an | |
| 543 | + * adjacent next vma that we'll walk | |
| 544 | + * next. userfaultfd_remove() will generate an | |
| 545 | + * UFFD_EVENT_REMOVE repetition on the | |
| 546 | + * end-vma->vm_end range, but the manager can | |
| 547 | + * handle a repetition fine. | |
| 548 | + */ | |
| 549 | + end = vma->vm_end; | |
| 550 | + } | |
| 551 | + VM_WARN_ON(start >= end); | |
| 552 | + } | |
| 517 | 553 | zap_page_range(vma, start, end - start); |
| 518 | 554 | return 0; |
| 519 | 555 | } |
| @@ -554,8 +590,10 @@ static long madvise_remove(struct vm_area_struct *vma, · 34 unmodified lines | ||
| 554 | 590 | * mmap_sem. |
| 555 | 591 | */ |
| 556 | 592 | get_file(f); |
| 557 | − userfaultfd_remove(vma, prev, start, end); | |
| 558 | − up_read(¤t->mm->mmap_sem); | |
| 593 | + if (userfaultfd_remove(vma, start, end)) { | |
| 594 | + /* mmap_sem was not released by userfaultfd_remove() */ | |
| 595 | + up_read(¤t->mm->mmap_sem); | |
| 596 | + } | |
| 559 | 597 | error = vfs_fallocate(f, |
| 560 | 598 | FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, |
| 561 | 599 | offset, end - start); |
| @@ -1118,7 +1118,10 @@ unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn, · 1117 unmodified lines | ||
| 1118 | 1118 | } |
| 1119 | 1119 | } while (left < right); |
| 1120 | 1120 | |
| 1121 | − return min(PHYS_PFN(type->regions[right].base), max_pfn); | |
| 1121 | + if (right == type->cnt) | |
| 1122 | + return max_pfn; | |
| 1123 | + else | |
| 1124 | + return min(PHYS_PFN(type->regions[right].base), max_pfn); | |
| 1122 | 1125 | } |
| 1123 | 1126 | |
| 1124 | 1127 | /** |
| @@ -466,6 +466,8 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page) · 465 unmodified lines | ||
| 466 | 466 | struct mem_cgroup_tree_per_node *mctz; |
| 467 | 467 | |
| 468 | 468 | mctz = soft_limit_tree_from_page(page); |
| 469 | + if (!mctz) | |
| 470 | + return; | |
| 469 | 471 | /* |
| 470 | 472 | * Necessary to update all ancestors when hierarchy is used. |
| 471 | 473 | * because their event counter is not touched. |
| @@ -503,7 +505,8 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg) · 31 unmodified lines | ||
| 503 | 505 | for_each_node(nid) { |
| 504 | 506 | mz = mem_cgroup_nodeinfo(memcg, nid); |
| 505 | 507 | mctz = soft_limit_tree_node(nid); |
| 506 | − mem_cgroup_remove_exceeded(mz, mctz); | |
| 508 | + if (mctz) | |
| 509 | + mem_cgroup_remove_exceeded(mz, mctz); | |
| 507 | 510 | } |
| 508 | 511 | } |
| 509 | 512 | |
| @@ -2558,7 +2561,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, · 2048 unmodified lines | ||
| 2558 | 2561 | * is empty. Do it lockless to prevent lock bouncing. Races |
| 2559 | 2562 | * are acceptable as soft limit is best effort anyway. |
| 2560 | 2563 | */ |
| 2561 | − if (RB_EMPTY_ROOT(&mctz->rb_root)) | |
| 2564 | + if (!mctz || RB_EMPTY_ROOT(&mctz->rb_root)) | |
| 2562 | 2565 | return 0; |
| 2563 | 2566 | |
| 2564 | 2567 | /* |
| @@ -4135,17 +4138,22 @@ static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) · 1570 unmodified lines | ||
| 4135 | 4138 | kfree(memcg->nodeinfo[node]); |
| 4136 | 4139 | } |
| 4137 | 4140 | |
| 4138 | −static void mem_cgroup_free(struct mem_cgroup *memcg) | |
| 4141 | +static void __mem_cgroup_free(struct mem_cgroup *memcg) | |
| 4139 | 4142 | { |
| 4140 | 4143 | int node; |
| 4141 | 4144 | |
| 4142 | − memcg_wb_domain_exit(memcg); | |
| 4143 | 4145 | for_each_node(node) |
| 4144 | 4146 | free_mem_cgroup_per_node_info(memcg, node); |
| 4145 | 4147 | free_percpu(memcg->stat); |
| 4146 | 4148 | kfree(memcg); |
| 4147 | 4149 | } |
| 4148 | 4150 | |
| 4151 | +static void mem_cgroup_free(struct mem_cgroup *memcg) | |
| 4152 | +{ | |
| 4153 | + memcg_wb_domain_exit(memcg); | |
| 4154 | + __mem_cgroup_free(memcg); | |
| 4155 | +} | |
| 4156 | + | |
| 4149 | 4157 | static struct mem_cgroup *mem_cgroup_alloc(void) |
| 4150 | 4158 | { |
| 4151 | 4159 | struct mem_cgroup *memcg; |
| @@ -4196,7 +4204,7 @@ static struct mem_cgroup *mem_cgroup_alloc(void) · 44 unmodified lines | ||
| 4196 | 4204 | fail: |
| 4197 | 4205 | if (memcg->id.id > 0) |
| 4198 | 4206 | idr_remove(&mem_cgroup_idr, memcg->id.id); |
| 4199 | − mem_cgroup_free(memcg); | |
| 4207 | + __mem_cgroup_free(memcg); | |
| 4200 | 4208 | return NULL; |
| 4201 | 4209 | } |
| 4202 | 4210 | |
| @@ -445,7 +445,7 @@ static inline void free_pmd_range(struct mmu_gather *tlb, pud_t *pud, · 444 unmodified lines | ||
| 445 | 445 | mm_dec_nr_pmds(tlb->mm); |
| 446 | 446 | } |
| 447 | 447 | |
| 448 | −static inline void free_pud_range(struct mmu_gather *tlb, pgd_t *pgd, | |
| 448 | +static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d, | |
| 449 | 449 | unsigned long addr, unsigned long end, |
| 450 | 450 | unsigned long floor, unsigned long ceiling) |
| 451 | 451 | { |
| @@ -454,7 +454,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, pgd_t *pgd, · 2 unmodified lines | ||
| 454 | 454 | unsigned long start; |
| 455 | 455 | |
| 456 | 456 | start = addr; |
| 457 | − pud = pud_offset(pgd, addr); | |
| 457 | + pud = pud_offset(p4d, addr); | |
| 458 | 458 | do { |
| 459 | 459 | next = pud_addr_end(addr, end); |
| 460 | 460 | if (pud_none_or_clear_bad(pud)) |
| @@ -462,6 +462,39 @@ static inline void free_pud_range(struct mmu_gather *tlb, pgd_t *pgd, · 1 unmodified line | ||
| 462 | 462 | free_pmd_range(tlb, pud, addr, next, floor, ceiling); |
| 463 | 463 | } while (pud++, addr = next, addr != end); |
| 464 | 464 | |
| 465 | + start &= P4D_MASK; | |
| 466 | + if (start < floor) | |
| 467 | + return; | |
| 468 | + if (ceiling) { | |
| 469 | + ceiling &= P4D_MASK; | |
| 470 | + if (!ceiling) | |
| 471 | + return; | |
| 472 | + } | |
| 473 | + if (end - 1 > ceiling - 1) | |
| 474 | + return; | |
| 475 | + | |
| 476 | + pud = pud_offset(p4d, start); | |
| 477 | + p4d_clear(p4d); | |
| 478 | + pud_free_tlb(tlb, pud, start); | |
| 479 | +} | |
| 480 | + | |
| 481 | +static inline void free_p4d_range(struct mmu_gather *tlb, pgd_t *pgd, | |
| 482 | + unsigned long addr, unsigned long end, | |
| 483 | + unsigned long floor, unsigned long ceiling) | |
| 484 | +{ | |
| 485 | + p4d_t *p4d; | |
| 486 | + unsigned long next; | |
| 487 | + unsigned long start; | |
| 488 | + | |
| 489 | + start = addr; | |
| 490 | + p4d = p4d_offset(pgd, addr); | |
| 491 | + do { | |
| 492 | + next = p4d_addr_end(addr, end); | |
| 493 | + if (p4d_none_or_clear_bad(p4d)) | |
| 494 | + continue; | |
| 495 | + free_pud_range(tlb, p4d, addr, next, floor, ceiling); | |
| 496 | + } while (p4d++, addr = next, addr != end); | |
| 497 | + | |
| 465 | 498 | start &= PGDIR_MASK; |
| 466 | 499 | if (start < floor) |
| 467 | 500 | return; |
| @@ -473,9 +506,9 @@ static inline void free_pud_range(struct mmu_gather *tlb, pgd_t *pgd, · 5 unmodified lines | ||
| 473 | 506 | if (end - 1 > ceiling - 1) |
| 474 | 507 | return; |
| 475 | 508 | |
| 476 | − pud = pud_offset(pgd, start); | |
| 509 | + p4d = p4d_offset(pgd, start); | |
| 477 | 510 | pgd_clear(pgd); |
| 478 | − pud_free_tlb(tlb, pud, start); | |
| 511 | + p4d_free_tlb(tlb, p4d, start); | |
| 479 | 512 | } |
| 480 | 513 | |
| 481 | 514 | /* |
| @@ -539,7 +572,7 @@ void free_pgd_range(struct mmu_gather *tlb, · 57 unmodified lines | ||
| 539 | 572 | next = pgd_addr_end(addr, end); |
| 540 | 573 | if (pgd_none_or_clear_bad(pgd)) |
| 541 | 574 | continue; |
| 542 | − free_pud_range(tlb, pgd, addr, next, floor, ceiling); | |
| 575 | + free_p4d_range(tlb, pgd, addr, next, floor, ceiling); | |
| 543 | 576 | } while (pgd++, addr = next, addr != end); |
| 544 | 577 | } |
| 545 | 578 | |
| @@ -658,7 +691,8 @@ static void print_bad_pte(struct vm_area_struct *vma, unsigned long addr, · 112 unmodified lines | ||
| 658 | 691 | pte_t pte, struct page *page) |
| 659 | 692 | { |
| 660 | 693 | pgd_t *pgd = pgd_offset(vma->vm_mm, addr); |
| 661 | − pud_t *pud = pud_offset(pgd, addr); | |
| 694 | + p4d_t *p4d = p4d_offset(pgd, addr); | |
| 695 | + pud_t *pud = pud_offset(p4d, addr); | |
| 662 | 696 | pmd_t *pmd = pmd_offset(pud, addr); |
| 663 | 697 | struct address_space *mapping; |
| 664 | 698 | pgoff_t index; |
| @@ -1023,16 +1057,16 @@ static inline int copy_pmd_range(struct mm_struct *dst_mm, struct mm_struct *src · 358 unmodified lines | ||
| 1023 | 1057 | } |
| 1024 | 1058 | |
| 1025 | 1059 | static inline int copy_pud_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, |
| 1026 | − pgd_t *dst_pgd, pgd_t *src_pgd, struct vm_area_struct *vma, | |
| 1060 | + p4d_t *dst_p4d, p4d_t *src_p4d, struct vm_area_struct *vma, | |
| 1027 | 1061 | unsigned long addr, unsigned long end) |
| 1028 | 1062 | { |
| 1029 | 1063 | pud_t *src_pud, *dst_pud; |
| 1030 | 1064 | unsigned long next; |
| 1031 | 1065 | |
| 1032 | − dst_pud = pud_alloc(dst_mm, dst_pgd, addr); | |
| 1066 | + dst_pud = pud_alloc(dst_mm, dst_p4d, addr); | |
| 1033 | 1067 | if (!dst_pud) |
| 1034 | 1068 | return -ENOMEM; |
| 1035 | − src_pud = pud_offset(src_pgd, addr); | |
| 1069 | + src_pud = pud_offset(src_p4d, addr); | |
| 1036 | 1070 | do { |
| 1037 | 1071 | next = pud_addr_end(addr, end); |
| 1038 | 1072 | if (pud_trans_huge(*src_pud) || pud_devmap(*src_pud)) { |
| @@ -1056,6 +1090,28 @@ static inline int copy_pud_range(struct mm_struct *dst_mm, struct mm_struct *src · 17 unmodified lines | ||
| 1056 | 1090 | return 0; |
| 1057 | 1091 | } |
| 1058 | 1092 | |
| 1093 | +static inline int copy_p4d_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, | |
| 1094 | + pgd_t *dst_pgd, pgd_t *src_pgd, struct vm_area_struct *vma, | |
| 1095 | + unsigned long addr, unsigned long end) | |
| 1096 | +{ | |
| 1097 | + p4d_t *src_p4d, *dst_p4d; | |
| 1098 | + unsigned long next; | |
| 1099 | + | |
| 1100 | + dst_p4d = p4d_alloc(dst_mm, dst_pgd, addr); | |
| 1101 | + if (!dst_p4d) | |
| 1102 | + return -ENOMEM; | |
| 1103 | + src_p4d = p4d_offset(src_pgd, addr); | |
| 1104 | + do { | |
| 1105 | + next = p4d_addr_end(addr, end); | |
| 1106 | + if (p4d_none_or_clear_bad(src_p4d)) | |
| 1107 | + continue; | |
| 1108 | + if (copy_pud_range(dst_mm, src_mm, dst_p4d, src_p4d, | |
| 1109 | + vma, addr, next)) | |
| 1110 | + return -ENOMEM; | |
| 1111 | + } while (dst_p4d++, src_p4d++, addr = next, addr != end); | |
| 1112 | + return 0; | |
| 1113 | +} | |
| 1114 | + | |
| 1059 | 1115 | int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, |
| 1060 | 1116 | struct vm_area_struct *vma) |
| 1061 | 1117 | { |
| @@ -1111,7 +1167,7 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, · 49 unmodified lines | ||
| 1111 | 1167 | next = pgd_addr_end(addr, end); |
| 1112 | 1168 | if (pgd_none_or_clear_bad(src_pgd)) |
| 1113 | 1169 | continue; |
| 1114 | − if (unlikely(copy_pud_range(dst_mm, src_mm, dst_pgd, src_pgd, | |
| 1170 | + if (unlikely(copy_p4d_range(dst_mm, src_mm, dst_pgd, src_pgd, | |
| 1115 | 1171 | vma, addr, next))) { |
| 1116 | 1172 | ret = -ENOMEM; |
| 1117 | 1173 | break; |
| @@ -1267,14 +1323,14 @@ next: · 149 unmodified lines | ||
| 1267 | 1323 | } |
| 1268 | 1324 | |
| 1269 | 1325 | static inline unsigned long zap_pud_range(struct mmu_gather *tlb, |
| 1270 | − struct vm_area_struct *vma, pgd_t *pgd, | |
| 1326 | + struct vm_area_struct *vma, p4d_t *p4d, | |
| 1271 | 1327 | unsigned long addr, unsigned long end, |
| 1272 | 1328 | struct zap_details *details) |
| 1273 | 1329 | { |
| 1274 | 1330 | pud_t *pud; |
| 1275 | 1331 | unsigned long next; |
| 1276 | 1332 | |
| 1277 | − pud = pud_offset(pgd, addr); | |
| 1333 | + pud = pud_offset(p4d, addr); | |
| 1278 | 1334 | do { |
| 1279 | 1335 | next = pud_addr_end(addr, end); |
| 1280 | 1336 | if (pud_trans_huge(*pud) || pud_devmap(*pud)) { |
| @@ -1295,6 +1351,25 @@ next: · 14 unmodified lines | ||
| 1295 | 1351 | return addr; |
| 1296 | 1352 | } |
| 1297 | 1353 | |
| 1354 | +static inline unsigned long zap_p4d_range(struct mmu_gather *tlb, | |
| 1355 | + struct vm_area_struct *vma, pgd_t *pgd, | |
| 1356 | + unsigned long addr, unsigned long end, | |
| 1357 | + struct zap_details *details) | |
| 1358 | +{ | |
| 1359 | + p4d_t *p4d; | |
| 1360 | + unsigned long next; | |
| 1361 | + | |
| 1362 | + p4d = p4d_offset(pgd, addr); | |
| 1363 | + do { | |
| 1364 | + next = p4d_addr_end(addr, end); | |
| 1365 | + if (p4d_none_or_clear_bad(p4d)) | |
| 1366 | + continue; | |
| 1367 | + next = zap_pud_range(tlb, vma, p4d, addr, next, details); | |
| 1368 | + } while (p4d++, addr = next, addr != end); | |
| 1369 | + | |
| 1370 | + return addr; | |
| 1371 | +} | |
| 1372 | + | |
| 1298 | 1373 | void unmap_page_range(struct mmu_gather *tlb, |
| 1299 | 1374 | struct vm_area_struct *vma, |
| 1300 | 1375 | unsigned long addr, unsigned long end, |
| @@ -1310,7 +1385,7 @@ void unmap_page_range(struct mmu_gather *tlb, · 9 unmodified lines | ||
| 1310 | 1385 | next = pgd_addr_end(addr, end); |
| 1311 | 1386 | if (pgd_none_or_clear_bad(pgd)) |
| 1312 | 1387 | continue; |
| 1313 | − next = zap_pud_range(tlb, vma, pgd, addr, next, details); | |
| 1388 | + next = zap_p4d_range(tlb, vma, pgd, addr, next, details); | |
| 1314 | 1389 | } while (pgd++, addr = next, addr != end); |
| 1315 | 1390 | tlb_end_vma(tlb, vma); |
| 1316 | 1391 | } |
| @@ -1465,16 +1540,24 @@ EXPORT_SYMBOL_GPL(zap_vma_ptes); · 148 unmodified lines | ||
| 1465 | 1540 | pte_t *__get_locked_pte(struct mm_struct *mm, unsigned long addr, |
| 1466 | 1541 | spinlock_t **ptl) |
| 1467 | 1542 | { |
| 1468 | − pgd_t *pgd = pgd_offset(mm, addr); | |
| 1469 | − pud_t *pud = pud_alloc(mm, pgd, addr); | |
| 1470 | − if (pud) { | |
| 1471 | − pmd_t *pmd = pmd_alloc(mm, pud, addr); | |
| 1472 | − if (pmd) { | |
| 1473 | − VM_BUG_ON(pmd_trans_huge(*pmd)); | |
| 1474 | − return pte_alloc_map_lock(mm, pmd, addr, ptl); | |
| 1475 | − } | |
| 1476 | − } | |
| 1477 | − return NULL; | |
| 1543 | + pgd_t *pgd; | |
| 1544 | + p4d_t *p4d; | |
| 1545 | + pud_t *pud; | |
| 1546 | + pmd_t *pmd; | |
| 1547 | + | |
| 1548 | + pgd = pgd_offset(mm, addr); | |
| 1549 | + p4d = p4d_alloc(mm, pgd, addr); | |
| 1550 | + if (!p4d) | |
| 1551 | + return NULL; | |
| 1552 | + pud = pud_alloc(mm, p4d, addr); | |
| 1553 | + if (!pud) | |
| 1554 | + return NULL; | |
| 1555 | + pmd = pmd_alloc(mm, pud, addr); | |
| 1556 | + if (!pmd) | |
| 1557 | + return NULL; | |
| 1558 | + | |
| 1559 | + VM_BUG_ON(pmd_trans_huge(*pmd)); | |
| 1560 | + return pte_alloc_map_lock(mm, pmd, addr, ptl); | |
| 1478 | 1561 | } |
| 1479 | 1562 | |
| 1480 | 1563 | /* |
| @@ -1740,7 +1823,7 @@ static inline int remap_pmd_range(struct mm_struct *mm, pud_t *pud, · 259 unmodified lines | ||
| 1740 | 1823 | return 0; |
| 1741 | 1824 | } |
| 1742 | 1825 | |
| 1743 | −static inline int remap_pud_range(struct mm_struct *mm, pgd_t *pgd, | |
| 1826 | +static inline int remap_pud_range(struct mm_struct *mm, p4d_t *p4d, | |
| 1744 | 1827 | unsigned long addr, unsigned long end, |
| 1745 | 1828 | unsigned long pfn, pgprot_t prot) |
| 1746 | 1829 | { |
| @@ -1748,7 +1831,7 @@ static inline int remap_pud_range(struct mm_struct *mm, pgd_t *pgd, · 1 unmodified line | ||
| 1748 | 1831 | unsigned long next; |
| 1749 | 1832 | |
| 1750 | 1833 | pfn -= addr >> PAGE_SHIFT; |
| 1751 | − pud = pud_alloc(mm, pgd, addr); | |
| 1834 | + pud = pud_alloc(mm, p4d, addr); | |
| 1752 | 1835 | if (!pud) |
| 1753 | 1836 | return -ENOMEM; |
| 1754 | 1837 | do { |
| @@ -1760,6 +1843,26 @@ static inline int remap_pud_range(struct mm_struct *mm, pgd_t *pgd, · 5 unmodified lines | ||
| 1760 | 1843 | return 0; |
| 1761 | 1844 | } |
| 1762 | 1845 | |
| 1846 | +static inline int remap_p4d_range(struct mm_struct *mm, pgd_t *pgd, | |
| 1847 | + unsigned long addr, unsigned long end, | |
| 1848 | + unsigned long pfn, pgprot_t prot) | |
| 1849 | +{ | |
| 1850 | + p4d_t *p4d; | |
| 1851 | + unsigned long next; | |
| 1852 | + | |
| 1853 | + pfn -= addr >> PAGE_SHIFT; | |
| 1854 | + p4d = p4d_alloc(mm, pgd, addr); | |
| 1855 | + if (!p4d) | |
| 1856 | + return -ENOMEM; | |
| 1857 | + do { | |
| 1858 | + next = p4d_addr_end(addr, end); | |
| 1859 | + if (remap_pud_range(mm, p4d, addr, next, | |
| 1860 | + pfn + (addr >> PAGE_SHIFT), prot)) | |
| 1861 | + return -ENOMEM; | |
| 1862 | + } while (p4d++, addr = next, addr != end); | |
| 1863 | + return 0; | |
| 1864 | +} | |
| 1865 | + | |
| 1763 | 1866 | /** |
| 1764 | 1867 | * remap_pfn_range - remap kernel memory to userspace |
| 1765 | 1868 | * @vma: user vma to map to |
| @@ -1816,7 +1919,7 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr, · 50 unmodified lines | ||
| 1816 | 1919 | flush_cache_range(vma, addr, end); |
| 1817 | 1920 | do { |
| 1818 | 1921 | next = pgd_addr_end(addr, end); |
| 1819 | − err = remap_pud_range(mm, pgd, addr, next, | |
| 1922 | + err = remap_p4d_range(mm, pgd, addr, next, | |
| 1820 | 1923 | pfn + (addr >> PAGE_SHIFT), prot); |
| 1821 | 1924 | if (err) |
| 1822 | 1925 | break; |
| @@ -1932,7 +2035,7 @@ static int apply_to_pmd_range(struct mm_struct *mm, pud_t *pud, · 109 unmodified lines | ||
| 1932 | 2035 | return err; |
| 1933 | 2036 | } |
| 1934 | 2037 | |
| 1935 | −static int apply_to_pud_range(struct mm_struct *mm, pgd_t *pgd, | |
| 2038 | +static int apply_to_pud_range(struct mm_struct *mm, p4d_t *p4d, | |
| 1936 | 2039 | unsigned long addr, unsigned long end, |
| 1937 | 2040 | pte_fn_t fn, void *data) |
| 1938 | 2041 | { |
| @@ -1940,7 +2043,7 @@ static int apply_to_pud_range(struct mm_struct *mm, pgd_t *pgd, · 1 unmodified line | ||
| 1940 | 2043 | unsigned long next; |
| 1941 | 2044 | int err; |
| 1942 | 2045 | |
| 1943 | − pud = pud_alloc(mm, pgd, addr); | |
| 2046 | + pud = pud_alloc(mm, p4d, addr); | |
| 1944 | 2047 | if (!pud) |
| 1945 | 2048 | return -ENOMEM; |
| 1946 | 2049 | do { |
| @@ -1952,6 +2055,26 @@ static int apply_to_pud_range(struct mm_struct *mm, pgd_t *pgd, · 5 unmodified lines | ||
| 1952 | 2055 | return err; |
| 1953 | 2056 | } |
| 1954 | 2057 | |
| 2058 | +static int apply_to_p4d_range(struct mm_struct *mm, pgd_t *pgd, | |
| 2059 | + unsigned long addr, unsigned long end, | |
| 2060 | + pte_fn_t fn, void *data) | |
| 2061 | +{ | |
| 2062 | + p4d_t *p4d; | |
| 2063 | + unsigned long next; | |
| 2064 | + int err; | |
| 2065 | + | |
| 2066 | + p4d = p4d_alloc(mm, pgd, addr); | |
| 2067 | + if (!p4d) | |
| 2068 | + return -ENOMEM; | |
| 2069 | + do { | |
| 2070 | + next = p4d_addr_end(addr, end); | |
| 2071 | + err = apply_to_pud_range(mm, p4d, addr, next, fn, data); | |
| 2072 | + if (err) | |
| 2073 | + break; | |
| 2074 | + } while (p4d++, addr = next, addr != end); | |
| 2075 | + return err; | |
| 2076 | +} | |
| 2077 | + | |
| 1955 | 2078 | /* |
| 1956 | 2079 | * Scan a region of virtual memory, filling in page tables as necessary |
| 1957 | 2080 | * and calling a provided function on each leaf page table. |
| @@ -1970,7 +2093,7 @@ int apply_to_page_range(struct mm_struct *mm, unsigned long addr, · 12 unmodified lines | ||
| 1970 | 2093 | pgd = pgd_offset(mm, addr); |
| 1971 | 2094 | do { |
| 1972 | 2095 | next = pgd_addr_end(addr, end); |
| 1973 | − err = apply_to_pud_range(mm, pgd, addr, next, fn, data); | |
| 2096 | + err = apply_to_p4d_range(mm, pgd, addr, next, fn, data); | |
| 1974 | 2097 | if (err) |
| 1975 | 2098 | break; |
| 1976 | 2099 | } while (pgd++, addr = next, addr != end); |
| @@ -3653,11 +3776,15 @@ static int __handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · 1676 unmodified lines | ||
| 3653 | 3776 | }; |
| 3654 | 3777 | struct mm_struct *mm = vma->vm_mm; |
| 3655 | 3778 | pgd_t *pgd; |
| 3779 | + p4d_t *p4d; | |
| 3656 | 3780 | int ret; |
| 3657 | 3781 | |
| 3658 | 3782 | pgd = pgd_offset(mm, address); |
| 3783 | + p4d = p4d_alloc(mm, pgd, address); | |
| 3784 | + if (!p4d) | |
| 3785 | + return VM_FAULT_OOM; | |
| 3659 | 3786 | |
| 3660 | − vmf.pud = pud_alloc(mm, pgd, address); | |
| 3787 | + vmf.pud = pud_alloc(mm, p4d, address); | |
| 3661 | 3788 | if (!vmf.pud) |
| 3662 | 3789 | return VM_FAULT_OOM; |
| 3663 | 3790 | if (pud_none(*vmf.pud) && transparent_hugepage_enabled(vma)) { |
| @@ -3779,12 +3906,35 @@ int handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · 115 unmodified lines | ||
| 3779 | 3906 | } |
| 3780 | 3907 | EXPORT_SYMBOL_GPL(handle_mm_fault); |
| 3781 | 3908 | |
| 3909 | +#ifndef __PAGETABLE_P4D_FOLDED | |
| 3910 | +/* | |
| 3911 | + * Allocate p4d page table. | |
| 3912 | + * We've already handled the fast-path in-line. | |
| 3913 | + */ | |
| 3914 | +int __p4d_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address) | |
| 3915 | +{ | |
| 3916 | + p4d_t *new = p4d_alloc_one(mm, address); | |
| 3917 | + if (!new) | |
| 3918 | + return -ENOMEM; | |
| 3919 | + | |
| 3920 | + smp_wmb(); /* See comment in __pte_alloc */ | |
| 3921 | + | |
| 3922 | + spin_lock(&mm->page_table_lock); | |
| 3923 | + if (pgd_present(*pgd)) /* Another has populated it */ | |
| 3924 | + p4d_free(mm, new); | |
| 3925 | + else | |
| 3926 | + pgd_populate(mm, pgd, new); | |
| 3927 | + spin_unlock(&mm->page_table_lock); | |
| 3928 | + return 0; | |
| 3929 | +} | |
| 3930 | +#endif /* __PAGETABLE_P4D_FOLDED */ | |
| 3931 | + | |
| 3782 | 3932 | #ifndef __PAGETABLE_PUD_FOLDED |
| 3783 | 3933 | /* |
| 3784 | 3934 | * Allocate page upper directory. |
| 3785 | 3935 | * We've already handled the fast-path in-line. |
| 3786 | 3936 | */ |
| 3787 | −int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address) | |
| 3937 | +int __pud_alloc(struct mm_struct *mm, p4d_t *p4d, unsigned long address) | |
| 3788 | 3938 | { |
| 3789 | 3939 | pud_t *new = pud_alloc_one(mm, address); |
| 3790 | 3940 | if (!new) |
| @@ -3793,10 +3943,17 @@ int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address) · 2 unmodified lines | ||
| 3793 | 3943 | smp_wmb(); /* See comment in __pte_alloc */ |
| 3794 | 3944 | |
| 3795 | 3945 | spin_lock(&mm->page_table_lock); |
| 3796 | − if (pgd_present(*pgd)) /* Another has populated it */ | |
| 3946 | +#ifndef __ARCH_HAS_5LEVEL_HACK | |
| 3947 | + if (p4d_present(*p4d)) /* Another has populated it */ | |
| 3797 | 3948 | pud_free(mm, new); |
| 3798 | 3949 | else |
| 3799 | − pgd_populate(mm, pgd, new); | |
| 3950 | + p4d_populate(mm, p4d, new); | |
| 3951 | +#else | |
| 3952 | + if (pgd_present(*p4d)) /* Another has populated it */ | |
| 3953 | + pud_free(mm, new); | |
| 3954 | + else | |
| 3955 | + pgd_populate(mm, p4d, new); | |
| 3956 | +#endif /* __ARCH_HAS_5LEVEL_HACK */ | |
| 3800 | 3957 | spin_unlock(&mm->page_table_lock); |
| 3801 | 3958 | return 0; |
| 3802 | 3959 | } |
| @@ -3839,6 +3996,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, · 36 unmodified lines | ||
| 3839 | 3996 | pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp) |
| 3840 | 3997 | { |
| 3841 | 3998 | pgd_t *pgd; |
| 3999 | + p4d_t *p4d; | |
| 3842 | 4000 | pud_t *pud; |
| 3843 | 4001 | pmd_t *pmd; |
| 3844 | 4002 | pte_t *ptep; |
| @@ -3847,7 +4005,11 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, · 2 unmodified lines | ||
| 3847 | 4005 | if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd))) |
| 3848 | 4006 | goto out; |
| 3849 | 4007 | |
| 3850 | − pud = pud_offset(pgd, address); | |
| 4008 | + p4d = p4d_offset(pgd, address); | |
| 4009 | + if (p4d_none(*p4d) || unlikely(p4d_bad(*p4d))) | |
| 4010 | + goto out; | |
| 4011 | + | |
4 more lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 162 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 143 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.