Pull kvm fixes from Paolo Bonzini: "arm64: - Fix a tiny buglet when propagating the deactivation of an interrupt from a nested guest, which happened to trigger a gold plated CPU bug on a particular implementation - Fix a race between LPI unmapping and mapping, resulting in leaked LPIs - Make LPI mapping more robust on memory allocation failure - Fix the handling of the EL2 tracing clock being disabled - A couple of Sashiko-driven fixes for corner cases in the EL2 tracing code - Add missing sysreg tracepoint for the EL2 code - Tidy-up the mutual exclusion of guest-memfd and MTE - Update Fuad's email address to point to @linux.dev s390: - several fixes for PCI passthru in s390 kvm - fix a 7.2-rc regression in the adapter interrupt mapping code x86: - Add memory clobber to asm for VMX instructions; without one, the compiler could reorder them in troublesome ways because "asm volatile" and "asm goto" only protect against removal of the asm. - Cancel delayed I/O APIC EOI handling before destroying vCPUs - Check all address spaces (normal and SMM) for write tracking and large pages, not just the current one. - Always update x2APIC MSR intercepts for L1 when AVIC is deactivated, even if not running L1. If the deactivation is VM-wide rather than being caused by something in L2's vCPU state, after a nested vmexit L1 will be able to access the host's APIC state" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (21 commits) KVM: SVM: Update x2APIC MSR intercepts if AVIC is inhibited while L2 is active KVM: x86/mmu: Check all address spaces before skipping unsync KVM: x86/mmu: Check write tracking in all address spaces KVM: x86: Cancel delayed I/O APIC EOI handling before destroying vCPUs KVM: VMX: add memory clobber to asm for VMX instructions KVM: s390: Fall back to short-term pinning in MAP ioctl KVM: s390: pci: Validate AIBV and AISB before pinning guest pages KVM: s390: pci: Fix resource leak on IRQ registration failure KVM: s390: pci: Fix NULL dereference on AIBV allocation failure KVM: s390: pci: Fix missing error codes and memory unaccounting KVM: s390: pci: Fix memory accounting for pinned/unpinned pages KVM: s390: pci: Reject adapter interrupt forwarding if already enabled KVM: arm64: Reject guest_memfd memslots when the VM has MTE KVM: arm64: Add missing hyp_enter when trapping sysreg KVM: arm64: Fix hyp_trace_desc allocation size in hyp_trace_load() KVM: arm64: Fix potential leak in hyp_trace_buffer_alloc_bpages_backing KVM: arm64: Fix hyp_trace clock disabling KVM: arm64: vgic: Mitigate potential LPI registration failure KVM: arm64: vgic: Fix race between LPI release and re-registration KVM: arm64: Update Fuad Tabba's email address ...
Unified · Split
| @@ -296,6 +296,7 @@ Frank Rowand <frowand.list@gmail.com> <frank.rowand@sony.com> · 295 unmodified lines | ||
| 296 | 296 | Frank Rowand <frowand.list@gmail.com> <frank.rowand@sonymobile.com> |
| 297 | 297 | Frank Rowand <frowand.list@gmail.com> <frowand@mvista.com> |
| 298 | 298 | Frank Zago <fzago@systemfabricworks.com> |
| 299 | +Fuad Tabba <fuad.tabba@linux.dev> <tabba@google.com> | |
| 299 | 300 | Gao Xiang <xiang@kernel.org> <gaoxiang25@huawei.com> |
| 300 | 301 | Gao Xiang <xiang@kernel.org> <hsiangkao@aol.com> |
| 301 | 302 | Gao Xiang <xiang@kernel.org> <hsiangkao@linux.alibaba.com> |
| @@ -55,10 +55,14 @@ stable kernels. · 54 unmodified lines | ||
| 55 | 55 | +----------------+-----------------+-----------------+-----------------------------+ |
| 56 | 56 | | Ampere | AmpereOne | AC03_CPU_38 | AMPERE_ERRATUM_AC03_CPU_38 | |
| 57 | 57 | +----------------+-----------------+-----------------+-----------------------------+ |
| 58 | +| Ampere | AmpereOne | AC03_CPU_57 | N/A | | |
| 59 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 58 | 60 | | Ampere | AmpereOne AC04 | AC04_CPU_10 | AMPERE_ERRATUM_AC03_CPU_38 | |
| 59 | 61 | +----------------+-----------------+-----------------+-----------------------------+ |
| 60 | 62 | | Ampere | AmpereOne AC04 | AC04_CPU_23 | AMPERE_ERRATUM_AC04_CPU_23 | |
| 61 | 63 | +----------------+-----------------+-----------------+-----------------------------+ |
| 64 | +| Ampere | AmpereOne AC04 | AC04_CPU_29 | N/A | | |
| 65 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 62 | 66 | +----------------+-----------------+-----------------+-----------------------------+ |
| 63 | 67 | | ARM | Cortex-A510 | #2457168 | ARM64_ERRATUM_2457168 | |
| 64 | 68 | +----------------+-----------------+-----------------+-----------------------------+ |
| @@ -8414,6 +8414,12 @@ When this capability is enabled all memory in memslots must be mapped as · 8413 unmodified lines | ||
| 8414 | 8414 | attempts to create a memslot with an invalid mmap will result in an |
| 8415 | 8415 | -EINVAL return. |
| 8416 | 8416 | |
| 8417 | +``guest_memfd``, even though it is an anonymous file, is not supported with MTE. | |
| 8418 | +Attempting to create a memslot backed by ``guest_memfd`` when the MTE capability | |
| 8419 | +is enabled, or attempting to enable the MTE capability after | |
| 8420 | +``guest_memfd``-backed memslots have been created, will result in an -EINVAL | |
| 8421 | +return. | |
| 8422 | + | |
| 8417 | 8423 | When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to |
| 8418 | 8424 | perform a bulk copy of tags to/from the guest. |
| 8419 | 8425 | |
| @@ -112,9 +112,20 @@ Groups: · 111 unmodified lines | ||
| 112 | 112 | mask or unmask the adapter, as specified in mask |
| 113 | 113 | |
| 114 | 114 | KVM_S390_IO_ADAPTER_MAP |
| 115 | − This is now a no-op. The mapping is purely done by the irq route. | |
| 115 | + Map an adapter indicator or summary page for long-term pinning so that | |
| 116 | + interrupt injection can be performed in atomic context. If long-term | |
| 117 | + pinning is not possible (e.g. file-backed memory), the page is verified | |
| 118 | + via a short-term pin and the ioctl returns success; interrupt injection | |
| 119 | + will use the non-atomic irqfd path with short-term pinning on each | |
| 120 | + interrupt. In Secure Execution mode this is a no-op and the ioctl | |
| 121 | + returns success. | |
| 122 | + | |
| 116 | 123 | KVM_S390_IO_ADAPTER_UNMAP |
| 117 | − This is now a no-op. The mapping is purely done by the irq route. | |
| 124 | + Unmap a previously mapped adapter indicator or summary page and release | |
| 125 | + the long-term pin. If the page was not long-term pinned (e.g. file-backed | |
| 126 | + memory), the map entry is removed and success is returned; if no prior | |
| 127 | + map entry exists, -ENOENT is returned. In Secure Execution mode this is | |
| 128 | + a no-op and the ioctl returns success. | |
| 118 | 129 | |
| 119 | 130 | KVM_DEV_FLIC_AISM |
| 120 | 131 | modify the adapter-interruption-suppression mode for a given isc if the |
| @@ -14193,7 +14193,7 @@ F: virt/kvm/* · 14192 unmodified lines | ||
| 14193 | 14193 | KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) |
| 14194 | 14194 | M: Marc Zyngier <maz@kernel.org> |
| 14195 | 14195 | M: Oliver Upton <oupton@kernel.org> |
| 14196 | −R: Fuad Tabba <tabba@google.com> | |
| 14196 | +R: Fuad Tabba <fuad.tabba@linux.dev> | |
| 14197 | 14197 | R: Joey Gouly <joey.gouly@arm.com> |
| 14198 | 14198 | R: Steffen Eiden <seiden@linux.ibm.com> |
| 14199 | 14199 | R: Suzuki K Poulose <suzuki.poulose@arm.com> |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | enum hyp_enter_exit_reason { |
| 13 | 13 | HYP_REASON_SMC, |
| 14 | 14 | HYP_REASON_HVC, |
| 15 | + HYP_REASON_SYS, | |
| 15 | 16 | HYP_REASON_PSCI, |
| 16 | 17 | HYP_REASON_HOST_ABORT, |
| 17 | 18 | HYP_REASON_GUEST_EXIT, |
| @@ -149,14 +149,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · 148 unmodified lines | ||
| 149 | 149 | set_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER, |
| 150 | 150 | &kvm->arch.flags); |
| 151 | 151 | break; |
| 152 | − case KVM_CAP_ARM_MTE: | |
| 153 | − mutex_lock(&kvm->lock); | |
| 154 | − if (system_supports_mte() && !kvm->created_vcpus) { | |
| 155 | − r = 0; | |
| 156 | − set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); | |
| 152 | + case KVM_CAP_ARM_MTE: { | |
| 153 | + struct kvm_memory_slot *memslot; | |
| 154 | + int bkt; | |
| 155 | + | |
| 156 | + guard(mutex)(&kvm->lock); | |
| 157 | + if (!system_supports_mte() || kvm->created_vcpus) | |
| 158 | + break; | |
| 159 | + | |
| 160 | + r = 0; | |
| 161 | + guard(mutex)(&kvm->slots_lock); | |
| 162 | + kvm_for_each_memslot(memslot, bkt, kvm_memslots(kvm)) { | |
| 163 | + if (kvm_slot_has_gmem(memslot)) { | |
| 164 | + r = -EINVAL; | |
| 165 | + break; | |
| 166 | + } | |
| 157 | 167 | } |
| 158 | − mutex_unlock(&kvm->lock); | |
| 168 | + if (r == 0) | |
| 169 | + set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); | |
| 159 | 170 | break; |
| 171 | + | |
| 172 | + } | |
| 160 | 173 | case KVM_CAP_ARM_SYSTEM_SUSPEND: |
| 161 | 174 | r = 0; |
| 162 | 175 | set_bit(KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED, &kvm->arch.flags); |
| @@ -932,6 +932,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) · 931 unmodified lines | ||
| 932 | 932 | handle_host_mem_abort(host_ctxt); |
| 933 | 933 | break; |
| 934 | 934 | case ESR_ELx_EC_SYS64: |
| 935 | + trace_hyp_enter(host_ctxt, HYP_REASON_SYS); | |
| 935 | 936 | if (handle_host_mte(esr)) |
| 936 | 937 | break; |
| 937 | 938 | fallthrough; |
| @@ -37,8 +37,6 @@ static struct hyp_trace_clock { · 36 unmodified lines | ||
| 37 | 37 | u32 shift; |
| 38 | 38 | struct delayed_work work; |
| 39 | 39 | struct completion ready; |
| 40 | − struct mutex lock; | |
| 41 | − bool running; | |
| 42 | 40 | } hyp_clock; |
| 43 | 41 | |
| 44 | 42 | static void __hyp_clock_work(struct work_struct *work) |
| @@ -110,12 +108,9 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 65 unmodified lines | ||
| 110 | 108 | { |
| 111 | 109 | struct system_time_snapshot snap; |
| 112 | 110 | |
| 113 | − if (hyp_clock->running == enable) | |
| 114 | − return; | |
| 115 | − | |
| 116 | 111 | if (!enable) { |
| 117 | 112 | cancel_delayed_work_sync(&hyp_clock->work); |
| 118 | − hyp_clock->running = false; | |
| 113 | + return; | |
| 119 | 114 | } |
| 120 | 115 | |
| 121 | 116 | ktime_get_snapshot_id(CLOCK_BOOTTIME, &snap); |
| @@ -128,7 +123,6 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 6 unmodified lines | ||
| 128 | 123 | INIT_DELAYED_WORK(&hyp_clock->work, __hyp_clock_work); |
| 129 | 124 | schedule_delayed_work(&hyp_clock->work, msecs_to_jiffies(CLOCK_INIT_MS)); |
| 130 | 125 | wait_for_completion(&hyp_clock->ready); |
| 131 | − hyp_clock->running = true; | |
| 132 | 126 | } |
| 133 | 127 | |
| 134 | 128 | /* Access to this struct within the trace_remote_callbacks are protected by the trace_remote lock */ |
| @@ -160,6 +154,7 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 25 unmodified lines | ||
| 160 | 154 | int nr_bpages = (PAGE_ALIGN(size) / PAGE_SIZE) + 1; |
| 161 | 155 | size_t backing_size; |
| 162 | 156 | void *start; |
| 157 | + int ret; | |
| 163 | 158 | |
| 164 | 159 | backing_size = PAGE_ALIGN(sizeof(struct simple_buffer_page) * nr_bpages * |
| 165 | 160 | num_possible_cpus()); |
| @@ -168,10 +163,16 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 2 unmodified lines | ||
| 168 | 163 | if (!start) |
| 169 | 164 | return -ENOMEM; |
| 170 | 165 | |
| 166 | + ret = __map_hyp(start, backing_size); | |
| 167 | + if (ret) { | |
| 168 | + free_pages_exact(start, backing_size); | |
| 169 | + return ret; | |
| 170 | + } | |
| 171 | + | |
| 171 | 172 | trace_buffer->desc->bpages_backing_start = (unsigned long)start; |
| 172 | 173 | trace_buffer->desc->bpages_backing_size = backing_size; |
| 173 | 174 | |
| 174 | − return __map_hyp(start, backing_size); | |
| 175 | + return ret; | |
| 175 | 176 | } |
| 176 | 177 | |
| 177 | 178 | static void hyp_trace_buffer_free_bpages_backing(struct hyp_trace_buffer *trace_buffer) |
| @@ -228,18 +229,22 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) · 50 unmodified lines | ||
| 228 | 229 | static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) |
| 229 | 230 | { |
| 230 | 231 | struct hyp_trace_buffer *trace_buffer = priv; |
| 232 | + size_t desc_size, tb_desc_size; | |
| 231 | 233 | struct hyp_trace_desc *desc; |
| 232 | − size_t desc_size; | |
| 233 | 234 | int ret; |
| 234 | 235 | |
| 235 | 236 | if (WARN_ON(trace_buffer->desc)) |
| 236 | 237 | return ERR_PTR(-EINVAL); |
| 237 | 238 | |
| 238 | − desc_size = trace_buffer_desc_size(size, num_possible_cpus()); | |
| 239 | + tb_desc_size = trace_buffer_desc_size(size, num_possible_cpus()); | |
| 240 | + desc_size = size_add(tb_desc_size, offsetof(struct hyp_trace_desc, trace_buffer_desc)); | |
| 239 | 241 | if (desc_size == SIZE_MAX) |
| 240 | 242 | return ERR_PTR(-E2BIG); |
| 241 | 243 | |
| 242 | 244 | desc_size = PAGE_ALIGN(desc_size); |
| 245 | + if (!desc_size) | |
| 246 | + return ERR_PTR(-E2BIG); | |
| 247 | + | |
| 243 | 248 | desc = (struct hyp_trace_desc *)alloc_pages_exact(desc_size, GFP_KERNEL); |
| 244 | 249 | if (!desc) |
| 245 | 250 | return ERR_PTR(-ENOMEM); |
| @@ -255,7 +260,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) · 9 unmodified lines | ||
| 255 | 260 | if (ret) |
| 256 | 261 | goto err_free_desc; |
| 257 | 262 | |
| 258 | − ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, desc_size, size, | |
| 263 | + ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, tb_desc_size, size, | |
| 259 | 264 | cpu_possible_mask); |
| 260 | 265 | if (ret) |
| 261 | 266 | goto err_free_backing; |
| @@ -304,9 +309,15 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) · 42 unmodified lines | ||
| 304 | 309 | |
| 305 | 310 | static int hyp_trace_enable_tracing(bool enable, void *priv) |
| 306 | 311 | { |
| 312 | + int ret; | |
| 313 | + | |
| 307 | 314 | hyp_trace_clock_enable(&hyp_clock, enable); |
| 308 | 315 | |
| 309 | − return kvm_call_hyp_nvhe(__tracing_enable, enable); | |
| 316 | + ret = kvm_call_hyp_nvhe(__tracing_enable, enable); | |
| 317 | + if (ret) | |
| 318 | + hyp_trace_clock_enable(&hyp_clock, !enable); | |
| 319 | + | |
| 320 | + return ret; | |
| 310 | 321 | } |
| 311 | 322 | |
| 312 | 323 | static int hyp_trace_swap_reader_page(unsigned int cpu, void *priv) |
| @@ -398,6 +409,7 @@ static const char *__hyp_enter_exit_reason_str(u8 reason) · 85 unmodified lines | ||
| 398 | 409 | static const char strs[][12] = { |
| 399 | 410 | "smc", |
| 400 | 411 | "hvc", |
| 412 | + "sys", | |
| 401 | 413 | "psci", |
| 402 | 414 | "host_abort", |
| 403 | 415 | "guest_exit", |
| @@ -2652,6 +2652,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, · 2651 unmodified lines | ||
| 2652 | 2652 | if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new)) |
| 2653 | 2653 | return -EINVAL; |
| 2654 | 2654 | |
| 2655 | + /* guest_memfd is incompatible with MTE. */ | |
| 2656 | + if (kvm_slot_has_gmem(new) && kvm_has_mte(kvm)) | |
| 2657 | + return -EINVAL; | |
| 2658 | + | |
| 2655 | 2659 | hva = new->userspace_addr; |
| 2656 | 2660 | reg_end = hva + (new->npages << PAGE_SHIFT); |
| 2657 | 2661 | |
| @@ -116,18 +116,27 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, · 115 unmodified lines | ||
| 116 | 116 | kfree(irq); |
| 117 | 117 | irq = oldirq; |
| 118 | 118 | } else { |
| 119 | − ret = xa_err(__xa_store(&dist->lpi_xa, intid, irq, 0)); | |
| 119 | + /* | |
| 120 | + * The entry is either empty or contains a dead LPI (refcount=0) | |
| 121 | + * from the deferred release path, pending cleanup by | |
| 122 | + * vgic_release_deleted_lpis(). Evict and free it if present. | |
| 123 | + */ | |
| 124 | + oldirq = __xa_store(&dist->lpi_xa, intid, irq, | |
| 125 | + GFP_NOWAIT | __GFP_ACCOUNT); | |
| 126 | + ret = xa_err(oldirq); | |
| 127 | + if (ret) { | |
| 128 | + xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 129 | + kfree(irq); | |
| 130 | + | |
| 131 | + return ERR_PTR(ret); | |
| 132 | + } | |
| 133 | + | |
| 134 | + if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount))) | |
| 135 | + kfree_rcu(oldirq, rcu); | |
| 120 | 136 | } |
| 121 | 137 | |
| 122 | 138 | xa_unlock_irqrestore(&dist->lpi_xa, flags); |
| 123 | 139 | |
| 124 | − if (ret) { | |
| 125 | − xa_release(&dist->lpi_xa, intid); | |
| 126 | − kfree(irq); | |
| 127 | − | |
| 128 | − return ERR_PTR(ret); | |
| 129 | − } | |
| 130 | − | |
| 131 | 140 | /* |
| 132 | 141 | * We "cache" the configuration table entries in our struct vgic_irq's. |
| 133 | 142 | * However we only have those structs for mapped IRQs, so we read in |
| @@ -275,7 +275,13 @@ void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val) · 274 unmodified lines | ||
| 275 | 275 | lr = vgic_v3_compute_lr(vcpu, irq) & ~ICH_LR_ACTIVE_BIT; |
| 276 | 276 | } |
| 277 | 277 | |
| 278 | − if (lr & ICH_LR_HW) | |
| 278 | + /* | |
| 279 | + * In the nested state, the irq has already been deactivated via the HW | |
| 280 | + * bit in the LR. Deactivating again would be harmless except AmpereOne | |
| 281 | + * errata AC03_CPU_57, AC04_CPU_29 could cause irq delivery to break if | |
| 282 | + * the deactivation hits the highest priority pending irq. | |
| 283 | + */ | |
| 284 | + if ((lr & ICH_LR_HW) && !vgic_state_is_nested(vcpu)) | |
| 279 | 285 | vgic_v3_deactivate_phys(FIELD_GET(ICH_LR_PHYS_ID_MASK, lr)); |
| 280 | 286 | |
| 281 | 287 | vgic_v3_fold_lr(vcpu, lr); |
| @@ -146,11 +146,7 @@ static __must_check bool __vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 145 unmodified lines | ||
| 146 | 146 | |
| 147 | 147 | static __must_check bool vgic_put_irq_norelease(struct kvm *kvm, struct vgic_irq *irq) |
| 148 | 148 | { |
| 149 | − if (!__vgic_put_irq(kvm, irq)) | |
| 150 | − return false; | |
| 151 | − | |
| 152 | − irq->pending_release = true; | |
| 153 | − return true; | |
| 149 | + return __vgic_put_irq(kvm, irq); | |
| 154 | 150 | } |
| 155 | 151 | |
| 156 | 152 | void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) |
| @@ -167,12 +163,14 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 10 unmodified lines | ||
| 167 | 163 | guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock); |
| 168 | 164 | } |
| 169 | 165 | |
| 170 | − if (!__vgic_put_irq(kvm, irq)) | |
| 166 | + if (!irq_is_lpi(kvm, irq->intid)) | |
| 171 | 167 | return; |
| 172 | 168 | |
| 173 | − xa_lock_irqsave(&dist->lpi_xa, flags); | |
| 174 | − vgic_release_lpi_locked(dist, irq); | |
| 175 | − xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 169 | + if (refcount_dec_and_lock_irqsave(&irq->refcount, | |
| 170 | + &dist->lpi_xa.xa_lock, &flags)) { | |
| 171 | + vgic_release_lpi_locked(dist, irq); | |
| 172 | + xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 173 | + } | |
| 176 | 174 | } |
| 177 | 175 | |
| 178 | 176 | static void vgic_release_deleted_lpis(struct kvm *kvm) |
| @@ -184,7 +182,7 @@ static void vgic_release_deleted_lpis(struct kvm *kvm) · 5 unmodified lines | ||
| 184 | 182 | xa_lock_irqsave(&dist->lpi_xa, flags); |
| 185 | 183 | |
| 186 | 184 | xa_for_each(&dist->lpi_xa, intid, irq) { |
| 187 | − if (irq->pending_release) | |
| 185 | + if (!refcount_read(&irq->refcount)) | |
| 188 | 186 | vgic_release_lpi_locked(dist, irq); |
| 189 | 187 | } |
| 190 | 188 | |
| @@ -476,6 +476,11 @@ struct s390_map_info { · 475 unmodified lines | ||
| 476 | 476 | __u64 guest_addr; |
| 477 | 477 | __u64 addr; |
| 478 | 478 | struct page *page; |
| 479 | + /* | |
| 480 | + * True if the page is long-term pinned. False if long-term pinning | |
| 481 | + * failed and this entry exists only to preserve MAP/UNMAP symmetry. | |
| 482 | + */ | |
| 483 | + bool pinned; | |
| 479 | 484 | }; |
| 480 | 485 | |
| 481 | 486 | struct s390_io_adapter { |
| @@ -2520,8 +2520,22 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 2519 unmodified lines | ||
| 2520 | 2520 | map->addr = host_addr; |
| 2521 | 2521 | map->page = pin_map_page(kvm, host_addr, FOLL_LONGTERM); |
| 2522 | 2522 | if (!map->page) { |
| 2523 | − ret = -EINVAL; | |
| 2524 | − goto out; | |
| 2523 | + /* | |
| 2524 | + * Long-term pinning may fail for memory types such as file-backed | |
| 2525 | + * memory. Verify that short-term pinning succeeds so that the | |
| 2526 | + * non-atomic irqfd path can handle interrupt injection. | |
| 2527 | + */ | |
| 2528 | + map->page = pin_map_page(kvm, host_addr, 0); | |
| 2529 | + if (!map->page) { | |
| 2530 | + ret = -EINVAL; | |
| 2531 | + goto out; | |
| 2532 | + } | |
| 2533 | + unpin_user_page(map->page); | |
| 2534 | + map->page = NULL; | |
| 2535 | + map->pinned = false; | |
| 2536 | + /* Add an entry to preserve MAP/UNMAP symmetry. */ | |
| 2537 | + } else { | |
| 2538 | + map->pinned = true; | |
| 2525 | 2539 | } |
| 2526 | 2540 | spin_lock_irqsave(&adapter->maps_lock, flags); |
| 2527 | 2541 | if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) { |
| @@ -2532,7 +2546,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 4 unmodified lines | ||
| 2532 | 2546 | ret = -EINVAL; |
| 2533 | 2547 | } |
| 2534 | 2548 | spin_unlock_irqrestore(&adapter->maps_lock, flags); |
| 2535 | − if (ret) | |
| 2549 | + if (ret && map->page) | |
| 2536 | 2550 | unpin_user_page(map->page); |
| 2537 | 2551 | out: |
| 2538 | 2552 | if (ret) |
| @@ -2546,6 +2560,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 7 unmodified lines | ||
| 2546 | 2560 | struct s390_map_info *map, *tmp, *map_to_free; |
| 2547 | 2561 | struct page *map_page_to_put = NULL; |
| 2548 | 2562 | u64 map_addr_to_mark = 0; |
| 2563 | + bool map_pinned = false; | |
| 2549 | 2564 | unsigned long flags; |
| 2550 | 2565 | int found = 0, idx; |
| 2551 | 2566 | |
| @@ -2560,6 +2575,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 8 unmodified lines | ||
| 2560 | 2575 | list_del(&map->list); |
| 2561 | 2576 | map_page_to_put = map->page; |
| 2562 | 2577 | map_addr_to_mark = map->guest_addr; |
| 2578 | + map_pinned = map->pinned; | |
| 2563 | 2579 | map_to_free = map; |
| 2564 | 2580 | break; |
| 2565 | 2581 | } |
| @@ -2568,11 +2584,18 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 2 unmodified lines | ||
| 2568 | 2584 | |
| 2569 | 2585 | if (found) { |
| 2570 | 2586 | kfree(map_to_free); |
| 2571 | − idx = srcu_read_lock(&kvm->srcu); | |
| 2572 | − mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); | |
| 2573 | − set_page_dirty_lock(map_page_to_put); | |
| 2574 | − srcu_read_unlock(&kvm->srcu, idx); | |
| 2575 | − unpin_user_page(map_page_to_put); | |
| 2587 | + if (map_pinned) { | |
| 2588 | + /* | |
| 2589 | + * Only long-term pinned pages need to be marked dirty | |
| 2590 | + * and released. Fallback entries exist only for | |
| 2591 | + * MAP/UNMAP symmetry. | |
| 2592 | + */ | |
| 2593 | + idx = srcu_read_lock(&kvm->srcu); | |
| 2594 | + mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); | |
| 2595 | + set_page_dirty_lock(map_page_to_put); | |
| 2596 | + srcu_read_unlock(&kvm->srcu, idx); | |
| 2597 | + unpin_user_page(map_page_to_put); | |
| 2598 | + } | |
| 2576 | 2599 | } |
| 2577 | 2600 | |
| 2578 | 2601 | return found ? 0 : -ENOENT; |
| @@ -2598,11 +2621,13 @@ void kvm_s390_unmap_all_adapters(struct kvm *kvm) · 19 unmodified lines | ||
| 2598 | 2621 | |
| 2599 | 2622 | list_for_each_entry_safe(map, tmp, &local_list, list) { |
| 2600 | 2623 | list_del(&map->list); |
| 2601 | − idx = srcu_read_lock(&kvm->srcu); | |
| 2602 | − mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); | |
| 2603 | − set_page_dirty_lock(map->page); | |
| 2604 | − srcu_read_unlock(&kvm->srcu, idx); | |
| 2605 | − unpin_user_page(map->page); | |
| 2624 | + if (map->pinned) { | |
| 2625 | + idx = srcu_read_lock(&kvm->srcu); | |
| 2626 | + mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); | |
| 2627 | + set_page_dirty_lock(map->page); | |
| 2628 | + srcu_read_unlock(&kvm->srcu, idx); | |
| 2629 | + unpin_user_page(map->page); | |
| 2630 | + } | |
| 2606 | 2631 | kfree(map); |
| 2607 | 2632 | } |
| 2608 | 2633 | } |
| @@ -2929,8 +2954,11 @@ static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, · 320 unmodified lines | ||
| 2929 | 2954 | return NULL; |
| 2930 | 2955 | |
| 2931 | 2956 | list_for_each_entry(map, &adapter->maps, list) { |
| 2932 | − if (map->addr == addr) | |
| 2957 | + if (map->addr == addr) { | |
| 2958 | + if (!map->pinned) | |
| 2959 | + return NULL; | |
| 2933 | 2960 | return map; |
| 2961 | + } | |
| 2934 | 2962 | } |
| 2935 | 2963 | return NULL; |
| 2936 | 2964 | } |
| @@ -190,40 +190,61 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev) · 189 unmodified lines | ||
| 190 | 190 | return cc ? -EIO : 0; |
| 191 | 191 | } |
| 192 | 192 | |
| 193 | −static inline void unaccount_mem(unsigned long nr_pages) | |
| 193 | +static inline void unaccount_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) | |
| 194 | 194 | { |
| 195 | − struct user_struct *user = get_uid(current_user()); | |
| 195 | + struct user_struct *user = kzdev->user_account; | |
| 196 | + struct mm_struct *mm_account = kzdev->mm_account; | |
| 196 | 197 | |
| 197 | − if (user) | |
| 198 | + if (user) { | |
| 198 | 199 | atomic_long_sub(nr_pages, &user->locked_vm); |
| 199 | − if (current->mm) | |
| 200 | − atomic64_sub(nr_pages, ¤t->mm->pinned_vm); | |
| 200 | + free_uid(user); | |
| 201 | + kzdev->user_account = NULL; | |
| 202 | + } | |
| 203 | + | |
| 204 | + if (mm_account) { | |
| 205 | + atomic64_sub(nr_pages, &mm_account->pinned_vm); | |
| 206 | + mmdrop(mm_account); | |
| 207 | + kzdev->mm_account = NULL; | |
| 208 | + } | |
| 201 | 209 | } |
| 202 | 210 | |
| 203 | −static inline int account_mem(unsigned long nr_pages) | |
| 211 | +static inline int account_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) | |
| 204 | 212 | { |
| 205 | 213 | struct user_struct *user = get_uid(current_user()); |
| 206 | 214 | unsigned long page_limit, cur_pages, new_pages; |
| 215 | + int rc = 0; | |
| 207 | 216 | |
| 208 | 217 | page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; |
| 209 | 218 | |
| 210 | 219 | cur_pages = atomic_long_read(&user->locked_vm); |
| 211 | 220 | do { |
| 212 | 221 | new_pages = cur_pages + nr_pages; |
| 213 | − if (new_pages > page_limit) | |
| 214 | − return -ENOMEM; | |
| 222 | + if (new_pages > page_limit) { | |
| 223 | + rc = -ENOMEM; | |
| 224 | + goto out; | |
| 225 | + } | |
| 215 | 226 | } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages)); |
| 216 | 227 | |
| 217 | − atomic64_add(nr_pages, ¤t->mm->pinned_vm); | |
| 228 | + if (current->mm) { | |
| 229 | + mmgrab(current->mm); | |
| 230 | + atomic64_add(nr_pages, ¤t->mm->pinned_vm); | |
| 231 | + } | |
| 232 | + | |
| 233 | + kzdev->user_account = user; | |
| 234 | + kzdev->mm_account = current->mm; | |
| 218 | 235 | |
| 219 | 236 | return 0; |
| 237 | + | |
| 238 | +out: | |
| 239 | + free_uid(user); | |
| 240 | + return rc; | |
| 220 | 241 | } |
| 221 | 242 | |
| 222 | 243 | static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, |
| 223 | 244 | bool assist) |
| 224 | 245 | { |
| 225 | 246 | struct page *pages[1], *aibv_page, *aisb_page = NULL; |
| 226 | − unsigned int msi_vecs, idx; | |
| 247 | + unsigned int msi_vecs, idx, size; | |
| 227 | 248 | struct zpci_gaite *gaite; |
| 228 | 249 | unsigned long hva, bit; |
| 229 | 250 | struct kvm *kvm; |
| @@ -237,6 +258,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 7 unmodified lines | ||
| 237 | 258 | if (zdev->gisa == 0) |
| 238 | 259 | return -EINVAL; |
| 239 | 260 | |
| 261 | + /* AIF already enabled for the device */ | |
| 262 | + if (zdev->kzdev->fib.fmt0.aibv != 0) | |
| 263 | + return -EINVAL; | |
| 264 | + | |
| 240 | 265 | kvm = zdev->kzdev->kvm; |
| 241 | 266 | msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi); |
| 242 | 267 | |
| @@ -246,6 +271,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 3 unmodified lines | ||
| 246 | 271 | return gisc; |
| 247 | 272 | |
| 248 | 273 | /* Replace AIBV address */ |
| 274 | + size = BITS_TO_LONGS(msi_vecs + fib->fmt0.aibvo) * sizeof(unsigned long); | |
| 275 | + npages = DIV_ROUND_UP((fib->fmt0.aibv & ~PAGE_MASK) + size, PAGE_SIZE); | |
| 276 | + /* AIBV cannot span more than 1 page */ | |
| 277 | + if (npages > 1) { | |
| 278 | + rc = -EINVAL; | |
| 279 | + goto out; | |
| 280 | + } | |
| 281 | + | |
| 249 | 282 | idx = srcu_read_lock(&kvm->srcu); |
| 250 | 283 | hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv)); |
| 251 | 284 | npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages); |
| @@ -261,6 +294,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 9 unmodified lines | ||
| 261 | 294 | |
| 262 | 295 | /* Pin the guest AISB if one was specified */ |
| 263 | 296 | if (fib->fmt0.sum == 1) { |
| 297 | + /* AISB must be dword aligned */ | |
| 298 | + if (fib->fmt0.aisb & 0x7) { | |
| 299 | + rc = -EINVAL; | |
| 300 | + goto unpin1; | |
| 301 | + } | |
| 302 | + | |
| 264 | 303 | idx = srcu_read_lock(&kvm->srcu); |
| 265 | 304 | hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb)); |
| 266 | 305 | npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, |
| @@ -275,20 +314,28 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 8 unmodified lines | ||
| 275 | 314 | } |
| 276 | 315 | |
| 277 | 316 | /* Account for pinned pages, roll back on failure */ |
| 278 | − if (account_mem(pcount)) | |
| 317 | + rc = account_mem(zdev->kzdev, pcount); | |
| 318 | + if (rc) | |
| 279 | 319 | goto unpin2; |
| 280 | 320 | |
| 281 | 321 | /* AISB must be allocated before we can fill in GAITE */ |
| 282 | 322 | mutex_lock(&aift->aift_lock); |
| 283 | 323 | bit = airq_iv_alloc_bit(aift->sbv); |
| 284 | − if (bit == -1UL) | |
| 324 | + if (bit == -1UL) { | |
| 325 | + rc = -ENOMEM; | |
| 285 | 326 | goto unlock; |
| 327 | + } | |
| 286 | 328 | zdev->aisb = bit; /* store the summary bit number */ |
| 287 | 329 | zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA | |
| 288 | 330 | AIRQ_IV_BITLOCK | |
| 289 | 331 | AIRQ_IV_GUESTVEC, |
| 290 | 332 | phys_to_virt(fib->fmt0.aibv)); |
| 291 | 333 | |
| 334 | + if (!zdev->aibv) { | |
| 335 | + rc = -ENOMEM; | |
| 336 | + goto free_aisb; | |
| 337 | + } | |
| 338 | + | |
| 292 | 339 | spin_lock_irq(&aift->gait_lock); |
| 293 | 340 | gaite = aift->gait + zdev->aisb; |
| 294 | 341 | |
| @@ -311,21 +358,39 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 16 unmodified lines | ||
| 311 | 358 | aift->kzdev[zdev->aisb] = zdev->kzdev; |
| 312 | 359 | spin_unlock_irq(&aift->gait_lock); |
| 313 | 360 | |
| 314 | − /* Update guest FIB for re-issue */ | |
| 315 | − fib->fmt0.aisbo = zdev->aisb & 63; | |
| 316 | − fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; | |
| 317 | − fib->fmt0.isc = gisc; | |
| 318 | − | |
| 319 | 361 | /* Save some guest fib values in the host for later use */ |
| 320 | − zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc; | |
| 362 | + zdev->kzdev->fib.fmt0.isc = gisc; | |
| 321 | 363 | zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv; |
| 322 | − mutex_unlock(&aift->aift_lock); | |
| 323 | 364 | |
| 324 | 365 | /* Issue the clp to setup the irq now */ |
| 325 | 366 | rc = kvm_zpci_set_airq(zdev); |
| 326 | − return rc; | |
| 367 | + if (!rc) { | |
| 368 | + mutex_unlock(&aift->aift_lock); | |
| 369 | + return rc; | |
| 370 | + } | |
| 371 | + | |
| 372 | + /* Start cleanup */ | |
| 373 | + zdev->kzdev->fib.fmt0.isc = 0; | |
| 374 | + zdev->kzdev->fib.fmt0.aibv = 0; | |
| 327 | 375 | |
| 376 | + spin_lock_irq(&aift->gait_lock); | |
| 377 | + gaite->count--; | |
| 378 | + gaite->aisb = 0; | |
| 379 | + gaite->gisc = 0; | |
| 380 | + gaite->aisbo = 0; | |
| 381 | + gaite->gisa = 0; | |
| 382 | + aift->kzdev[zdev->aisb] = NULL; | |
| 383 | + spin_unlock_irq(&aift->gait_lock); | |
| 384 | + | |
| 385 | + airq_iv_release(zdev->aibv); | |
| 386 | + zdev->aibv = NULL; | |
| 387 | + | |
| 388 | +free_aisb: | |
| 389 | + airq_iv_free_bit(aift->sbv, zdev->aisb); | |
| 390 | + zdev->aisb = 0; | |
| 328 | 391 | unlock: |
| 392 | + if (pcount > 0) | |
| 393 | + unaccount_mem(zdev->kzdev, pcount); | |
| 329 | 394 | mutex_unlock(&aift->aift_lock); |
| 330 | 395 | unpin2: |
| 331 | 396 | if (fib->fmt0.sum == 1) |
| @@ -396,7 +461,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) · 64 unmodified lines | ||
| 396 | 461 | pcount++; |
| 397 | 462 | } |
| 398 | 463 | if (pcount > 0) |
| 399 | − unaccount_mem(pcount); | |
| 464 | + unaccount_mem(kzdev, pcount); | |
| 400 | 465 | out: |
| 401 | 466 | mutex_unlock(&aift->aift_lock); |
| 402 | 467 | |
| @@ -22,6 +22,8 @@ struct kvm_zdev { · 21 unmodified lines | ||
| 22 | 22 | struct kvm *kvm; |
| 23 | 23 | struct zpci_fib fib; |
| 24 | 24 | struct list_head entry; |
| 25 | + struct user_struct *user_account; | |
| 26 | + struct mm_struct *mm_account; | |
| 25 | 27 | }; |
| 26 | 28 | |
| 27 | 29 | struct zpci_gaite { |
| @@ -722,6 +722,26 @@ static struct kvm_lpage_info *lpage_info_slot(gfn_t gfn, · 721 unmodified lines | ||
| 722 | 722 | return &slot->arch.lpage_info[level - 2][idx]; |
| 723 | 723 | } |
| 724 | 724 | |
| 725 | +static bool kvm_gfn_is_lpage_allowed(struct kvm *kvm, | |
| 726 | + const struct kvm_memory_slot *slot, | |
| 727 | + gfn_t gfn, int level) | |
| 728 | +{ | |
| 729 | + const struct kvm_memory_slot *other_slot; | |
| 730 | + | |
| 731 | + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); | |
| 732 | + | |
| 733 | + if (lpage_info_slot(gfn, slot, level)->disallow_lpage) | |
| 734 | + return false; | |
| 735 | + | |
| 736 | + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { | |
| 737 | + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); | |
| 738 | + if (other_slot && lpage_info_slot(gfn, other_slot, level)->disallow_lpage) | |
| 739 | + return false; | |
| 740 | + } | |
| 741 | + | |
| 742 | + return true; | |
| 743 | +} | |
| 744 | + | |
| 725 | 745 | /* |
| 726 | 746 | * The most significant bit in disallow_lpage tracks whether or not memory |
| 727 | 747 | * attributes are mixed, i.e. not identical for all gfns at the current level. |
| @@ -2968,7 +2988,7 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, · 2240 unmodified lines | ||
| 2968 | 2988 | * write-protected (see above), thus if the gfn can be mapped with a |
| 2969 | 2989 | * hugepage and isn't write-tracked, it can't have a shadow page. |
| 2970 | 2990 | */ |
| 2971 | − if (!lpage_info_slot(gfn, slot, PG_LEVEL_2M)->disallow_lpage) | |
| 2991 | + if (kvm_gfn_is_lpage_allowed(kvm, slot, gfn, PG_LEVEL_2M)) | |
| 2972 | 2992 | return 0; |
| 2973 | 2993 | |
| 2974 | 2994 | /* |
| @@ -130,13 +130,23 @@ void __kvm_write_track_remove_gfn(struct kvm *kvm, · 129 unmodified lines | ||
| 130 | 130 | kvm_mmu_gfn_allow_lpage(slot, gfn); |
| 131 | 131 | } |
| 132 | 132 | |
| 133 | −/* | |
| 134 | − * check if the corresponding access on the specified guest page is tracked. | |
| 135 | − */ | |
| 133 | +static bool __kvm_gfn_is_write_tracked(const struct kvm_memory_slot *slot, | |
| 134 | + gfn_t gfn) | |
| 135 | +{ | |
| 136 | + int index; | |
| 137 | + | |
| 138 | + if (!slot) | |
| 139 | + return false; | |
| 140 | + | |
| 141 | + index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); | |
| 142 | + return !!READ_ONCE(slot->arch.gfn_write_track[index]); | |
| 143 | +} | |
| 144 | + | |
| 145 | +/* check if write access is tracked on the specified guest page. */ | |
| 136 | 146 | bool kvm_gfn_is_write_tracked(struct kvm *kvm, |
| 137 | 147 | const struct kvm_memory_slot *slot, gfn_t gfn) |
| 138 | 148 | { |
| 139 | − int index; | |
| 149 | + const struct kvm_memory_slot *other_slot; | |
| 140 | 150 | |
| 141 | 151 | if (!slot) |
| 142 | 152 | return false; |
| @@ -144,8 +154,18 @@ bool kvm_gfn_is_write_tracked(struct kvm *kvm, · 1 unmodified line | ||
| 144 | 154 | if (!kvm_page_track_write_tracking_enabled(kvm)) |
| 145 | 155 | return false; |
| 146 | 156 | |
| 147 | − index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); | |
| 148 | − return !!READ_ONCE(slot->arch.gfn_write_track[index]); | |
| 157 | + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); | |
| 158 | + | |
| 159 | + if (__kvm_gfn_is_write_tracked(slot, gfn)) | |
| 160 | + return true; | |
| 161 | + | |
| 162 | + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { | |
| 163 | + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); | |
| 164 | + if (__kvm_gfn_is_write_tracked(other_slot, gfn)) | |
| 165 | + return true; | |
| 166 | + } | |
| 167 | + | |
| 168 | + return false; | |
| 149 | 169 | } |
| 150 | 170 | |
| 151 | 171 | #ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING |
| @@ -240,14 +240,6 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) · 239 unmodified lines | ||
| 240 | 240 | if (!is_sev_es_guest(&svm->vcpu)) |
| 241 | 241 | svm_set_intercept(svm, INTERCEPT_CR8_WRITE); |
| 242 | 242 | |
| 243 | − /* | |
| 244 | − * If running nested and the guest uses its own MSR bitmap, there | |
| 245 | − * is no need to update L0's msr bitmap | |
| 246 | − */ | |
| 247 | − if (is_guest_mode(&svm->vcpu) && | |
| 248 | − vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)) | |
| 249 | − return; | |
| 250 | − | |
| 251 | 243 | /* Enabling MSR intercept for x2APIC registers */ |
| 252 | 244 | avic_set_x2apic_msr_interception(svm, true); |
| 253 | 245 | } |
| @@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) · 100 unmodified lines | ||
| 101 | 101 | |
| 102 | 102 | : [output] "=r" (value) |
| 103 | 103 | : [field] "r" (field) |
| 104 | − : "cc" | |
| 104 | + : "cc", "memory" | |
| 105 | 105 | : do_fail, do_exception); |
| 106 | 106 | |
| 107 | 107 | return value; |
| @@ -145,7 +145,7 @@ do_exception: · 37 unmodified lines | ||
| 145 | 145 | |
| 146 | 146 | : ASM_CALL_CONSTRAINT, [output] "=&r" (value) |
| 147 | 147 | : [field] "r" (field) |
| 148 | − : "cc"); | |
| 148 | + : "cc", "memory"); | |
| 149 | 149 | return value; |
| 150 | 150 | |
| 151 | 151 | #endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ |
| @@ -192,7 +192,7 @@ do { \ · 40 unmodified lines | ||
| 192 | 192 | asm goto("1: " __stringify(insn) " %0\n\t" \ |
| 193 | 193 | "jna %l[error]\n\t" \ |
| 194 | 194 | _ASM_EXTABLE(1b, %l[fault]) \ |
| 195 | − : : op1 : "cc" : error, fault); \ | |
| 195 | + : : op1 : "cc", "memory" : error, fault); \ | |
| 196 | 196 | return; \ |
| 197 | 197 | error: \ |
| 198 | 198 | instrumentation_begin(); \ |
| @@ -208,7 +208,7 @@ do { \ · 9 unmodified lines | ||
| 208 | 208 | asm goto("1: " __stringify(insn) " %1, %0\n\t" \ |
| 209 | 209 | "jna %l[error]\n\t" \ |
| 210 | 210 | _ASM_EXTABLE(1b, %l[fault]) \ |
| 211 | − : : op1, op2 : "cc" : error, fault); \ | |
| 211 | + : : op1, op2 : "cc", "memory" : error, fault);\ | |
| 212 | 212 | return; \ |
| 213 | 213 | error: \ |
| 214 | 214 | instrumentation_begin(); \ |
| @@ -13429,9 +13429,15 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm) · 13428 unmodified lines | ||
| 13429 | 13429 | * iterating over vCPUs in a different task while vCPUs are being freed |
| 13430 | 13430 | * is unsafe, i.e. will lead to use-after-free. The PIT also needs to |
| 13431 | 13431 | * be stopped before IRQ routing is freed. |
| 13432 | + * | |
| 13433 | + * Do NOT free the in-kernel PIC or I/O APIC here (but as above, make | |
| 13434 | + * sure to flush any background work), as KVM expects interrupt routing | |
| 13435 | + * structures to be valid until vCPUs are destroyed. | |
| 13432 | 13436 | */ |
| 13433 | 13437 | #ifdef CONFIG_KVM_IOAPIC |
| 13434 | 13438 | kvm_free_pit(kvm); |
| 13439 | + if (kvm->arch.vioapic) | |
| 13440 | + cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject); | |
| 13435 | 13441 | #endif |
| 13436 | 13442 | |
| 13437 | 13443 | kvm_mmu_pre_destroy_vm(kvm); |
| @@ -247,9 +247,6 @@ struct vgic_irq { · 246 unmodified lines | ||
| 247 | 247 | * affinity reg (v3). |
| 248 | 248 | */ |
| 249 | 249 | |
| 250 | − bool pending_release:1; /* Used for LPIs only, unreferenced IRQ | |
| 251 | − * pending a release */ | |
| 252 | − | |
| 253 | 250 | bool pending_latch:1; /* The pending latch state used to calculate |
| 254 | 251 | * the pending state for both level |
| 255 | 252 | * and edge triggered IRQs. */ |