History

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull kvm fixes from Paolo Bonzini: "arm64: - Fix a tiny buglet when propagating the deactivation of an interrupt from a nested guest, which happened to trigger a gold plated CPU bug on a particular implementation - Fix a race between LPI unmapping and mapping, resulting in leaked LPIs - Make LPI mapping more robust on memory allocation failure - Fix the handling of the EL2 tracing clock being disabled - A couple of Sashiko-driven fixes for corner cases in the EL2 tracing code - Add missing sysreg tracepoint for the EL2 code - Tidy-up the mutual exclusion of guest-memfd and MTE - Update Fuad's email address to point to @linux.dev s390: - several fixes for PCI passthru in s390 kvm - fix a 7.2-rc regression in the adapter interrupt mapping code x86: - Add memory clobber to asm for VMX instructions; without one, the compiler could reorder them in troublesome ways because "asm volatile" and "asm goto" only protect against removal of the asm. - Cancel delayed I/O APIC EOI handling before destroying vCPUs - Check all address spaces (normal and SMM) for write tracking and large pages, not just the current one. - Always update x2APIC MSR intercepts for L1 when AVIC is deactivated, even if not running L1. If the deactivation is VM-wide rather than being caused by something in L2's vCPU state, after a nested vmexit L1 will be able to access the host's APIC state" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (21 commits) KVM: SVM: Update x2APIC MSR intercepts if AVIC is inhibited while L2 is active KVM: x86/mmu: Check all address spaces before skipping unsync KVM: x86/mmu: Check write tracking in all address spaces KVM: x86: Cancel delayed I/O APIC EOI handling before destroying vCPUs KVM: VMX: add memory clobber to asm for VMX instructions KVM: s390: Fall back to short-term pinning in MAP ioctl KVM: s390: pci: Validate AIBV and AISB before pinning guest pages KVM: s390: pci: Fix resource leak on IRQ registration failure KVM: s390: pci: Fix NULL dereference on AIBV allocation failure KVM: s390: pci: Fix missing error codes and memory unaccounting KVM: s390: pci: Fix memory accounting for pinned/unpinned pages KVM: s390: pci: Reject adapter interrupt forwarding if already enabled KVM: arm64: Reject guest_memfd memslots when the VM has MTE KVM: arm64: Add missing hyp_enter when trapping sysreg KVM: arm64: Fix hyp_trace_desc allocation size in hyp_trace_load() KVM: arm64: Fix potential leak in hyp_trace_buffer_alloc_bpages_backing KVM: arm64: Fix hyp_trace clock disabling KVM: arm64: vgic: Mitigate potential LPI registration failure KVM: arm64: vgic: Fix race between LPI release and re-registration KVM: arm64: Update Fuad Tabba's email address ...

Linus Torvalds committed 3 months ago

Commit fc46aed2 parents fc02acf 7d3aae223 files · +298 −97Browse files

Unified · Split

23 files
  1. .mailmap +1 −0
  2. Documentation/arch/arm64/silicon-errata.rst +4 −0
  3. Documentation/virt/kvm/api.rst +6 −0
  4. Documentation/virt/kvm/devices/s390_flic.rst +13 −2
  5. MAINTAINERS +1 −1
  6. arch/arm64/include/asm/kvm_hypevents.h +1 −0
  7. arch/arm64/kvm/arm.c +19 −6
  8. arch/arm64/kvm/hyp/nvhe/hyp-main.c +1 −0
  9. arch/arm64/kvm/hyp_trace.c +24 −12
  10. arch/arm64/kvm/mmu.c +4 −0
  11. arch/arm64/kvm/vgic/vgic-its.c +17 −8
  12. arch/arm64/kvm/vgic/vgic-v3.c +7 −1
  13. arch/arm64/kvm/vgic/vgic.c +8 −10
  14. arch/s390/include/asm/kvm_host.h +5 −0
  15. arch/s390/kvm/interrupt.c +42 −14
  16. arch/s390/kvm/pci.c +86 −21
  17. arch/s390/kvm/pci.h +2 −0
  18. arch/x86/kvm/mmu/mmu.c +21 −1
  19. arch/x86/kvm/mmu/page_track.c +26 −6
  20. arch/x86/kvm/svm/avic.c +0 −8
  21. arch/x86/kvm/vmx/vmx_ops.h +4 −4
  22. arch/x86/kvm/x86.c +6 −0
  23. include/kvm/arm_vgic.h +0 −3
.mailmap+1−0
@@ -296,6 +296,7 @@ Frank Rowand <frowand.list@gmail.com> <frank.rowand@sony.com> · 295 unmodified lines
296Frank Rowand <frowand.list@gmail.com> <frank.rowand@sonymobile.com>296Frank Rowand <frowand.list@gmail.com> <frank.rowand@sonymobile.com>
297Frank Rowand <frowand.list@gmail.com> <frowand@mvista.com>297Frank Rowand <frowand.list@gmail.com> <frowand@mvista.com>
298Frank Zago <fzago@systemfabricworks.com>298Frank Zago <fzago@systemfabricworks.com>
299Fuad Tabba <fuad.tabba@linux.dev> <tabba@google.com>
299Gao Xiang <xiang@kernel.org> <gaoxiang25@huawei.com>300Gao Xiang <xiang@kernel.org> <gaoxiang25@huawei.com>
300Gao Xiang <xiang@kernel.org> <hsiangkao@aol.com>301Gao Xiang <xiang@kernel.org> <hsiangkao@aol.com>
301Gao Xiang <xiang@kernel.org> <hsiangkao@linux.alibaba.com>302Gao Xiang <xiang@kernel.org> <hsiangkao@linux.alibaba.com>
Documentation/arch/arm64/silicon-errata.rst+4−0
@@ -55,10 +55,14 @@ stable kernels. · 54 unmodified lines
55+----------------+-----------------+-----------------+-----------------------------+55+----------------+-----------------+-----------------+-----------------------------+
56| Ampere | AmpereOne | AC03_CPU_38 | AMPERE_ERRATUM_AC03_CPU_38 |56| Ampere | AmpereOne | AC03_CPU_38 | AMPERE_ERRATUM_AC03_CPU_38 |
57+----------------+-----------------+-----------------+-----------------------------+57+----------------+-----------------+-----------------+-----------------------------+
58| Ampere | AmpereOne | AC03_CPU_57 | N/A |
59+----------------+-----------------+-----------------+-----------------------------+
58| Ampere | AmpereOne AC04 | AC04_CPU_10 | AMPERE_ERRATUM_AC03_CPU_38 |60| Ampere | AmpereOne AC04 | AC04_CPU_10 | AMPERE_ERRATUM_AC03_CPU_38 |
59+----------------+-----------------+-----------------+-----------------------------+61+----------------+-----------------+-----------------+-----------------------------+
60| Ampere | AmpereOne AC04 | AC04_CPU_23 | AMPERE_ERRATUM_AC04_CPU_23 |62| Ampere | AmpereOne AC04 | AC04_CPU_23 | AMPERE_ERRATUM_AC04_CPU_23 |
61+----------------+-----------------+-----------------+-----------------------------+63+----------------+-----------------+-----------------+-----------------------------+
64| Ampere | AmpereOne AC04 | AC04_CPU_29 | N/A |
65+----------------+-----------------+-----------------+-----------------------------+
62+----------------+-----------------+-----------------+-----------------------------+66+----------------+-----------------+-----------------+-----------------------------+
63| ARM | Cortex-A510 | #2457168 | ARM64_ERRATUM_2457168 |67| ARM | Cortex-A510 | #2457168 | ARM64_ERRATUM_2457168 |
64+----------------+-----------------+-----------------+-----------------------------+68+----------------+-----------------+-----------------+-----------------------------+
Documentation/virt/kvm/api.rst+6−0
@@ -8414,6 +8414,12 @@ When this capability is enabled all memory in memslots must be mapped as · 8413 unmodified lines
8414attempts to create a memslot with an invalid mmap will result in an8414attempts to create a memslot with an invalid mmap will result in an
8415-EINVAL return.8415-EINVAL return.
8416 8416
8417``guest_memfd``, even though it is an anonymous file, is not supported with MTE.
8418Attempting to create a memslot backed by ``guest_memfd`` when the MTE capability
8419is enabled, or attempting to enable the MTE capability after
8420``guest_memfd``-backed memslots have been created, will result in an -EINVAL
8421return.
8422
8417When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to8423When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to
8418perform a bulk copy of tags to/from the guest.8424perform a bulk copy of tags to/from the guest.
8419 8425
Documentation/virt/kvm/devices/s390_flic.rst+13−2
@@ -112,9 +112,20 @@ Groups: · 111 unmodified lines
112 mask or unmask the adapter, as specified in mask112 mask or unmask the adapter, as specified in mask
113 113
114 KVM_S390_IO_ADAPTER_MAP114 KVM_S390_IO_ADAPTER_MAP
115 This is now a no-op. The mapping is purely done by the irq route.115 Map an adapter indicator or summary page for long-term pinning so that
116 interrupt injection can be performed in atomic context. If long-term
117 pinning is not possible (e.g. file-backed memory), the page is verified
118 via a short-term pin and the ioctl returns success; interrupt injection
119 will use the non-atomic irqfd path with short-term pinning on each
120 interrupt. In Secure Execution mode this is a no-op and the ioctl
121 returns success.
122
116 KVM_S390_IO_ADAPTER_UNMAP123 KVM_S390_IO_ADAPTER_UNMAP
117 This is now a no-op. The mapping is purely done by the irq route.124 Unmap a previously mapped adapter indicator or summary page and release
125 the long-term pin. If the page was not long-term pinned (e.g. file-backed
126 memory), the map entry is removed and success is returned; if no prior
127 map entry exists, -ENOENT is returned. In Secure Execution mode this is
128 a no-op and the ioctl returns success.
118 129
119 KVM_DEV_FLIC_AISM130 KVM_DEV_FLIC_AISM
120 modify the adapter-interruption-suppression mode for a given isc if the131 modify the adapter-interruption-suppression mode for a given isc if the
MAINTAINERS+1−1
@@ -14193,7 +14193,7 @@ F: virt/kvm/* · 14192 unmodified lines
14193KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)14193KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)
14194M: Marc Zyngier <maz@kernel.org>14194M: Marc Zyngier <maz@kernel.org>
14195M: Oliver Upton <oupton@kernel.org>14195M: Oliver Upton <oupton@kernel.org>
14196R: Fuad Tabba <tabba@google.com>14196R: Fuad Tabba <fuad.tabba@linux.dev>
14197R: Joey Gouly <joey.gouly@arm.com>14197R: Joey Gouly <joey.gouly@arm.com>
14198R: Steffen Eiden <seiden@linux.ibm.com>14198R: Steffen Eiden <seiden@linux.ibm.com>
14199R: Suzuki K Poulose <suzuki.poulose@arm.com>14199R: Suzuki K Poulose <suzuki.poulose@arm.com>
arch/arm64/include/asm/kvm_hypevents.h+1−0
@@ -12,6 +12,7 @@ · 11 unmodified lines
12enum hyp_enter_exit_reason {12enum hyp_enter_exit_reason {
13 HYP_REASON_SMC,13 HYP_REASON_SMC,
14 HYP_REASON_HVC,14 HYP_REASON_HVC,
15 HYP_REASON_SYS,
15 HYP_REASON_PSCI,16 HYP_REASON_PSCI,
16 HYP_REASON_HOST_ABORT,17 HYP_REASON_HOST_ABORT,
17 HYP_REASON_GUEST_EXIT,18 HYP_REASON_GUEST_EXIT,
arch/arm64/kvm/arm.c+19−6
@@ -149,14 +149,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · 148 unmodified lines
149 set_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER,149 set_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER,
150 &kvm->arch.flags);150 &kvm->arch.flags);
151 break;151 break;
152 case KVM_CAP_ARM_MTE:152 case KVM_CAP_ARM_MTE: {
153 mutex_lock(&kvm->lock);153 struct kvm_memory_slot *memslot;
154 if (system_supports_mte() && !kvm->created_vcpus) {154 int bkt;
155 r = 0;155
156 set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags);156 guard(mutex)(&kvm->lock);
157 if (!system_supports_mte() || kvm->created_vcpus)
158 break;
159
160 r = 0;
161 guard(mutex)(&kvm->slots_lock);
162 kvm_for_each_memslot(memslot, bkt, kvm_memslots(kvm)) {
163 if (kvm_slot_has_gmem(memslot)) {
164 r = -EINVAL;
165 break;
166 }
157 }167 }
158 mutex_unlock(&kvm->lock);168 if (r == 0)
169 set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags);
159 break;170 break;
171
172 }
160 case KVM_CAP_ARM_SYSTEM_SUSPEND:173 case KVM_CAP_ARM_SYSTEM_SUSPEND:
161 r = 0;174 r = 0;
162 set_bit(KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED, &kvm->arch.flags);175 set_bit(KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED, &kvm->arch.flags);
arch/arm64/kvm/hyp/nvhe/hyp-main.c+1−0
@@ -932,6 +932,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) · 931 unmodified lines
932 handle_host_mem_abort(host_ctxt);932 handle_host_mem_abort(host_ctxt);
933 break;933 break;
934 case ESR_ELx_EC_SYS64:934 case ESR_ELx_EC_SYS64:
935 trace_hyp_enter(host_ctxt, HYP_REASON_SYS);
935 if (handle_host_mte(esr))936 if (handle_host_mte(esr))
936 break;937 break;
937 fallthrough;938 fallthrough;
arch/arm64/kvm/hyp_trace.c+24−12
@@ -37,8 +37,6 @@ static struct hyp_trace_clock { · 36 unmodified lines
37 u32 shift;37 u32 shift;
38 struct delayed_work work;38 struct delayed_work work;
39 struct completion ready;39 struct completion ready;
40 struct mutex lock;
41 bool running;
42} hyp_clock;40} hyp_clock;
43 41
44static void __hyp_clock_work(struct work_struct *work)42static void __hyp_clock_work(struct work_struct *work)
@@ -110,12 +108,9 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 65 unmodified lines
110{108{
111 struct system_time_snapshot snap;109 struct system_time_snapshot snap;
112 110
113 if (hyp_clock->running == enable)
114 return;
115
116 if (!enable) {111 if (!enable) {
117 cancel_delayed_work_sync(&hyp_clock->work);112 cancel_delayed_work_sync(&hyp_clock->work);
118 hyp_clock->running = false;113 return;
119 }114 }
120 115
121 ktime_get_snapshot_id(CLOCK_BOOTTIME, &snap);116 ktime_get_snapshot_id(CLOCK_BOOTTIME, &snap);
@@ -128,7 +123,6 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 6 unmodified lines
128 INIT_DELAYED_WORK(&hyp_clock->work, __hyp_clock_work);123 INIT_DELAYED_WORK(&hyp_clock->work, __hyp_clock_work);
129 schedule_delayed_work(&hyp_clock->work, msecs_to_jiffies(CLOCK_INIT_MS));124 schedule_delayed_work(&hyp_clock->work, msecs_to_jiffies(CLOCK_INIT_MS));
130 wait_for_completion(&hyp_clock->ready);125 wait_for_completion(&hyp_clock->ready);
131 hyp_clock->running = true;
132}126}
133 127
134/* Access to this struct within the trace_remote_callbacks are protected by the trace_remote lock */128/* Access to this struct within the trace_remote_callbacks are protected by the trace_remote lock */
@@ -160,6 +154,7 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 25 unmodified lines
160 int nr_bpages = (PAGE_ALIGN(size) / PAGE_SIZE) + 1;154 int nr_bpages = (PAGE_ALIGN(size) / PAGE_SIZE) + 1;
161 size_t backing_size;155 size_t backing_size;
162 void *start;156 void *start;
157 int ret;
163 158
164 backing_size = PAGE_ALIGN(sizeof(struct simple_buffer_page) * nr_bpages *159 backing_size = PAGE_ALIGN(sizeof(struct simple_buffer_page) * nr_bpages *
165 num_possible_cpus());160 num_possible_cpus());
@@ -168,10 +163,16 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 2 unmodified lines
168 if (!start)163 if (!start)
169 return -ENOMEM;164 return -ENOMEM;
170 165
166 ret = __map_hyp(start, backing_size);
167 if (ret) {
168 free_pages_exact(start, backing_size);
169 return ret;
170 }
171
171 trace_buffer->desc->bpages_backing_start = (unsigned long)start;172 trace_buffer->desc->bpages_backing_start = (unsigned long)start;
172 trace_buffer->desc->bpages_backing_size = backing_size;173 trace_buffer->desc->bpages_backing_size = backing_size;
173 174
174 return __map_hyp(start, backing_size);175 return ret;
175}176}
176 177
177static void hyp_trace_buffer_free_bpages_backing(struct hyp_trace_buffer *trace_buffer)178static void hyp_trace_buffer_free_bpages_backing(struct hyp_trace_buffer *trace_buffer)
@@ -228,18 +229,22 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) · 50 unmodified lines
228static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv)229static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv)
229{230{
230 struct hyp_trace_buffer *trace_buffer = priv;231 struct hyp_trace_buffer *trace_buffer = priv;
232 size_t desc_size, tb_desc_size;
231 struct hyp_trace_desc *desc;233 struct hyp_trace_desc *desc;
232 size_t desc_size;
233 int ret;234 int ret;
234 235
235 if (WARN_ON(trace_buffer->desc))236 if (WARN_ON(trace_buffer->desc))
236 return ERR_PTR(-EINVAL);237 return ERR_PTR(-EINVAL);
237 238
238 desc_size = trace_buffer_desc_size(size, num_possible_cpus());239 tb_desc_size = trace_buffer_desc_size(size, num_possible_cpus());
240 desc_size = size_add(tb_desc_size, offsetof(struct hyp_trace_desc, trace_buffer_desc));
239 if (desc_size == SIZE_MAX)241 if (desc_size == SIZE_MAX)
240 return ERR_PTR(-E2BIG);242 return ERR_PTR(-E2BIG);
241 243
242 desc_size = PAGE_ALIGN(desc_size);244 desc_size = PAGE_ALIGN(desc_size);
245 if (!desc_size)
246 return ERR_PTR(-E2BIG);
247
243 desc = (struct hyp_trace_desc *)alloc_pages_exact(desc_size, GFP_KERNEL);248 desc = (struct hyp_trace_desc *)alloc_pages_exact(desc_size, GFP_KERNEL);
244 if (!desc)249 if (!desc)
245 return ERR_PTR(-ENOMEM);250 return ERR_PTR(-ENOMEM);
@@ -255,7 +260,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) · 9 unmodified lines
255 if (ret)260 if (ret)
256 goto err_free_desc;261 goto err_free_desc;
257 262
258 ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, desc_size, size,263 ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, tb_desc_size, size,
259 cpu_possible_mask);264 cpu_possible_mask);
260 if (ret)265 if (ret)
261 goto err_free_backing;266 goto err_free_backing;
@@ -304,9 +309,15 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) · 42 unmodified lines
304 309
305static int hyp_trace_enable_tracing(bool enable, void *priv)310static int hyp_trace_enable_tracing(bool enable, void *priv)
306{311{
312 int ret;
313
307 hyp_trace_clock_enable(&hyp_clock, enable);314 hyp_trace_clock_enable(&hyp_clock, enable);
308 315
309 return kvm_call_hyp_nvhe(__tracing_enable, enable);316 ret = kvm_call_hyp_nvhe(__tracing_enable, enable);
317 if (ret)
318 hyp_trace_clock_enable(&hyp_clock, !enable);
319
320 return ret;
310}321}
311 322
312static int hyp_trace_swap_reader_page(unsigned int cpu, void *priv)323static int hyp_trace_swap_reader_page(unsigned int cpu, void *priv)
@@ -398,6 +409,7 @@ static const char *__hyp_enter_exit_reason_str(u8 reason) · 85 unmodified lines
398 static const char strs[][12] = {409 static const char strs[][12] = {
399 "smc",410 "smc",
400 "hvc",411 "hvc",
412 "sys",
401 "psci",413 "psci",
402 "host_abort",414 "host_abort",
403 "guest_exit",415 "guest_exit",
arch/arm64/kvm/mmu.c+4−0
@@ -2652,6 +2652,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, · 2651 unmodified lines
2652 if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new))2652 if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new))
2653 return -EINVAL;2653 return -EINVAL;
2654 2654
2655 /* guest_memfd is incompatible with MTE. */
2656 if (kvm_slot_has_gmem(new) && kvm_has_mte(kvm))
2657 return -EINVAL;
2658
2655 hva = new->userspace_addr;2659 hva = new->userspace_addr;
2656 reg_end = hva + (new->npages << PAGE_SHIFT);2660 reg_end = hva + (new->npages << PAGE_SHIFT);
2657 2661
arch/arm64/kvm/vgic/vgic-its.c+17−8
@@ -116,18 +116,27 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, · 115 unmodified lines
116 kfree(irq);116 kfree(irq);
117 irq = oldirq;117 irq = oldirq;
118 } else {118 } else {
119 ret = xa_err(__xa_store(&dist->lpi_xa, intid, irq, 0));119 /*
120 * The entry is either empty or contains a dead LPI (refcount=0)
121 * from the deferred release path, pending cleanup by
122 * vgic_release_deleted_lpis(). Evict and free it if present.
123 */
124 oldirq = __xa_store(&dist->lpi_xa, intid, irq,
125 GFP_NOWAIT | __GFP_ACCOUNT);
126 ret = xa_err(oldirq);
127 if (ret) {
128 xa_unlock_irqrestore(&dist->lpi_xa, flags);
129 kfree(irq);
130
131 return ERR_PTR(ret);
132 }
133
134 if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount)))
135 kfree_rcu(oldirq, rcu);
120 }136 }
121 137
122 xa_unlock_irqrestore(&dist->lpi_xa, flags);138 xa_unlock_irqrestore(&dist->lpi_xa, flags);
123 139
124 if (ret) {
125 xa_release(&dist->lpi_xa, intid);
126 kfree(irq);
127
128 return ERR_PTR(ret);
129 }
130
131 /*140 /*
132 * We "cache" the configuration table entries in our struct vgic_irq's.141 * We "cache" the configuration table entries in our struct vgic_irq's.
133 * However we only have those structs for mapped IRQs, so we read in142 * However we only have those structs for mapped IRQs, so we read in
arch/arm64/kvm/vgic/vgic-v3.c+7−1
@@ -275,7 +275,13 @@ void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val) · 274 unmodified lines
275 lr = vgic_v3_compute_lr(vcpu, irq) & ~ICH_LR_ACTIVE_BIT;275 lr = vgic_v3_compute_lr(vcpu, irq) & ~ICH_LR_ACTIVE_BIT;
276 }276 }
277 277
278 if (lr & ICH_LR_HW)278 /*
279 * In the nested state, the irq has already been deactivated via the HW
280 * bit in the LR. Deactivating again would be harmless except AmpereOne
281 * errata AC03_CPU_57, AC04_CPU_29 could cause irq delivery to break if
282 * the deactivation hits the highest priority pending irq.
283 */
284 if ((lr & ICH_LR_HW) && !vgic_state_is_nested(vcpu))
279 vgic_v3_deactivate_phys(FIELD_GET(ICH_LR_PHYS_ID_MASK, lr));285 vgic_v3_deactivate_phys(FIELD_GET(ICH_LR_PHYS_ID_MASK, lr));
280 286
281 vgic_v3_fold_lr(vcpu, lr);287 vgic_v3_fold_lr(vcpu, lr);
arch/arm64/kvm/vgic/vgic.c+8−10
@@ -146,11 +146,7 @@ static __must_check bool __vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 145 unmodified lines
146 146
147static __must_check bool vgic_put_irq_norelease(struct kvm *kvm, struct vgic_irq *irq)147static __must_check bool vgic_put_irq_norelease(struct kvm *kvm, struct vgic_irq *irq)
148{148{
149 if (!__vgic_put_irq(kvm, irq))149 return __vgic_put_irq(kvm, irq);
150 return false;
151
152 irq->pending_release = true;
153 return true;
154}150}
155 151
156void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq)152void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq)
@@ -167,12 +163,14 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 10 unmodified lines
167 guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock);163 guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock);
168 }164 }
169 165
170 if (!__vgic_put_irq(kvm, irq))166 if (!irq_is_lpi(kvm, irq->intid))
171 return;167 return;
172 168
173 xa_lock_irqsave(&dist->lpi_xa, flags);169 if (refcount_dec_and_lock_irqsave(&irq->refcount,
174 vgic_release_lpi_locked(dist, irq);170 &dist->lpi_xa.xa_lock, &flags)) {
175 xa_unlock_irqrestore(&dist->lpi_xa, flags);171 vgic_release_lpi_locked(dist, irq);
172 xa_unlock_irqrestore(&dist->lpi_xa, flags);
173 }
176}174}
177 175
178static void vgic_release_deleted_lpis(struct kvm *kvm)176static void vgic_release_deleted_lpis(struct kvm *kvm)
@@ -184,7 +182,7 @@ static void vgic_release_deleted_lpis(struct kvm *kvm) · 5 unmodified lines
184 xa_lock_irqsave(&dist->lpi_xa, flags);182 xa_lock_irqsave(&dist->lpi_xa, flags);
185 183
186 xa_for_each(&dist->lpi_xa, intid, irq) {184 xa_for_each(&dist->lpi_xa, intid, irq) {
187 if (irq->pending_release)185 if (!refcount_read(&irq->refcount))
188 vgic_release_lpi_locked(dist, irq);186 vgic_release_lpi_locked(dist, irq);
189 }187 }
190 188
arch/s390/include/asm/kvm_host.h+5−0
@@ -476,6 +476,11 @@ struct s390_map_info { · 475 unmodified lines
476 __u64 guest_addr;476 __u64 guest_addr;
477 __u64 addr;477 __u64 addr;
478 struct page *page;478 struct page *page;
479 /*
480 * True if the page is long-term pinned. False if long-term pinning
481 * failed and this entry exists only to preserve MAP/UNMAP symmetry.
482 */
483 bool pinned;
479};484};
480 485
481struct s390_io_adapter {486struct s390_io_adapter {
arch/s390/kvm/interrupt.c+42−14
@@ -2520,8 +2520,22 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 2519 unmodified lines
2520 map->addr = host_addr;2520 map->addr = host_addr;
2521 map->page = pin_map_page(kvm, host_addr, FOLL_LONGTERM);2521 map->page = pin_map_page(kvm, host_addr, FOLL_LONGTERM);
2522 if (!map->page) {2522 if (!map->page) {
2523 ret = -EINVAL;2523 /*
2524 goto out;2524 * Long-term pinning may fail for memory types such as file-backed
2525 * memory. Verify that short-term pinning succeeds so that the
2526 * non-atomic irqfd path can handle interrupt injection.
2527 */
2528 map->page = pin_map_page(kvm, host_addr, 0);
2529 if (!map->page) {
2530 ret = -EINVAL;
2531 goto out;
2532 }
2533 unpin_user_page(map->page);
2534 map->page = NULL;
2535 map->pinned = false;
2536 /* Add an entry to preserve MAP/UNMAP symmetry. */
2537 } else {
2538 map->pinned = true;
2525 }2539 }
2526 spin_lock_irqsave(&adapter->maps_lock, flags);2540 spin_lock_irqsave(&adapter->maps_lock, flags);
2527 if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) {2541 if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) {
@@ -2532,7 +2546,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 4 unmodified lines
2532 ret = -EINVAL;2546 ret = -EINVAL;
2533 }2547 }
2534 spin_unlock_irqrestore(&adapter->maps_lock, flags);2548 spin_unlock_irqrestore(&adapter->maps_lock, flags);
2535 if (ret)2549 if (ret && map->page)
2536 unpin_user_page(map->page);2550 unpin_user_page(map->page);
2537out:2551out:
2538 if (ret)2552 if (ret)
@@ -2546,6 +2560,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 7 unmodified lines
2546 struct s390_map_info *map, *tmp, *map_to_free;2560 struct s390_map_info *map, *tmp, *map_to_free;
2547 struct page *map_page_to_put = NULL;2561 struct page *map_page_to_put = NULL;
2548 u64 map_addr_to_mark = 0;2562 u64 map_addr_to_mark = 0;
2563 bool map_pinned = false;
2549 unsigned long flags;2564 unsigned long flags;
2550 int found = 0, idx;2565 int found = 0, idx;
2551 2566
@@ -2560,6 +2575,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 8 unmodified lines
2560 list_del(&map->list);2575 list_del(&map->list);
2561 map_page_to_put = map->page;2576 map_page_to_put = map->page;
2562 map_addr_to_mark = map->guest_addr;2577 map_addr_to_mark = map->guest_addr;
2578 map_pinned = map->pinned;
2563 map_to_free = map;2579 map_to_free = map;
2564 break;2580 break;
2565 }2581 }
@@ -2568,11 +2584,18 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 2 unmodified lines
2568 2584
2569 if (found) {2585 if (found) {
2570 kfree(map_to_free);2586 kfree(map_to_free);
2571 idx = srcu_read_lock(&kvm->srcu);2587 if (map_pinned) {
2572 mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT);2588 /*
2573 set_page_dirty_lock(map_page_to_put);2589 * Only long-term pinned pages need to be marked dirty
2574 srcu_read_unlock(&kvm->srcu, idx);2590 * and released. Fallback entries exist only for
2575 unpin_user_page(map_page_to_put);2591 * MAP/UNMAP symmetry.
2592 */
2593 idx = srcu_read_lock(&kvm->srcu);
2594 mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT);
2595 set_page_dirty_lock(map_page_to_put);
2596 srcu_read_unlock(&kvm->srcu, idx);
2597 unpin_user_page(map_page_to_put);
2598 }
2576 }2599 }
2577 2600
2578 return found ? 0 : -ENOENT;2601 return found ? 0 : -ENOENT;
@@ -2598,11 +2621,13 @@ void kvm_s390_unmap_all_adapters(struct kvm *kvm) · 19 unmodified lines
2598 2621
2599 list_for_each_entry_safe(map, tmp, &local_list, list) {2622 list_for_each_entry_safe(map, tmp, &local_list, list) {
2600 list_del(&map->list);2623 list_del(&map->list);
2601 idx = srcu_read_lock(&kvm->srcu);2624 if (map->pinned) {
2602 mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT);2625 idx = srcu_read_lock(&kvm->srcu);
2603 set_page_dirty_lock(map->page);2626 mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT);
2604 srcu_read_unlock(&kvm->srcu, idx);2627 set_page_dirty_lock(map->page);
2605 unpin_user_page(map->page);2628 srcu_read_unlock(&kvm->srcu, idx);
2629 unpin_user_page(map->page);
2630 }
2606 kfree(map);2631 kfree(map);
2607 }2632 }
2608 }2633 }
@@ -2929,8 +2954,11 @@ static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, · 320 unmodified lines
2929 return NULL;2954 return NULL;
2930 2955
2931 list_for_each_entry(map, &adapter->maps, list) {2956 list_for_each_entry(map, &adapter->maps, list) {
2932 if (map->addr == addr)2957 if (map->addr == addr) {
2958 if (!map->pinned)
2959 return NULL;
2933 return map;2960 return map;
2961 }
2934 }2962 }
2935 return NULL;2963 return NULL;
2936}2964}
arch/s390/kvm/pci.c+86−21
@@ -190,40 +190,61 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev) · 189 unmodified lines
190 return cc ? -EIO : 0;190 return cc ? -EIO : 0;
191}191}
192 192
193static inline void unaccount_mem(unsigned long nr_pages)193static inline void unaccount_mem(struct kvm_zdev *kzdev, unsigned long nr_pages)
194{194{
195 struct user_struct *user = get_uid(current_user());195 struct user_struct *user = kzdev->user_account;
196 struct mm_struct *mm_account = kzdev->mm_account;
196 197
197 if (user)198 if (user) {
198 atomic_long_sub(nr_pages, &user->locked_vm);199 atomic_long_sub(nr_pages, &user->locked_vm);
199 if (current->mm)200 free_uid(user);
200 atomic64_sub(nr_pages, &current->mm->pinned_vm);201 kzdev->user_account = NULL;
202 }
203
204 if (mm_account) {
205 atomic64_sub(nr_pages, &mm_account->pinned_vm);
206 mmdrop(mm_account);
207 kzdev->mm_account = NULL;
208 }
201}209}
202 210
203static inline int account_mem(unsigned long nr_pages)211static inline int account_mem(struct kvm_zdev *kzdev, unsigned long nr_pages)
204{212{
205 struct user_struct *user = get_uid(current_user());213 struct user_struct *user = get_uid(current_user());
206 unsigned long page_limit, cur_pages, new_pages;214 unsigned long page_limit, cur_pages, new_pages;
215 int rc = 0;
207 216
208 page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;217 page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
209 218
210 cur_pages = atomic_long_read(&user->locked_vm);219 cur_pages = atomic_long_read(&user->locked_vm);
211 do {220 do {
212 new_pages = cur_pages + nr_pages;221 new_pages = cur_pages + nr_pages;
213 if (new_pages > page_limit)222 if (new_pages > page_limit) {
214 return -ENOMEM;223 rc = -ENOMEM;
224 goto out;
225 }
215 } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages));226 } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages));
216 227
217 atomic64_add(nr_pages, &current->mm->pinned_vm);228 if (current->mm) {
229 mmgrab(current->mm);
230 atomic64_add(nr_pages, &current->mm->pinned_vm);
231 }
232
233 kzdev->user_account = user;
234 kzdev->mm_account = current->mm;
218 235
219 return 0;236 return 0;
237
238out:
239 free_uid(user);
240 return rc;
220}241}
221 242
222static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,243static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
223 bool assist)244 bool assist)
224{245{
225 struct page *pages[1], *aibv_page, *aisb_page = NULL;246 struct page *pages[1], *aibv_page, *aisb_page = NULL;
226 unsigned int msi_vecs, idx;247 unsigned int msi_vecs, idx, size;
227 struct zpci_gaite *gaite;248 struct zpci_gaite *gaite;
228 unsigned long hva, bit;249 unsigned long hva, bit;
229 struct kvm *kvm;250 struct kvm *kvm;
@@ -237,6 +258,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 7 unmodified lines
237 if (zdev->gisa == 0)258 if (zdev->gisa == 0)
238 return -EINVAL;259 return -EINVAL;
239 260
261 /* AIF already enabled for the device */
262 if (zdev->kzdev->fib.fmt0.aibv != 0)
263 return -EINVAL;
264
240 kvm = zdev->kzdev->kvm;265 kvm = zdev->kzdev->kvm;
241 msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi);266 msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi);
242 267
@@ -246,6 +271,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 3 unmodified lines
246 return gisc;271 return gisc;
247 272
248 /* Replace AIBV address */273 /* Replace AIBV address */
274 size = BITS_TO_LONGS(msi_vecs + fib->fmt0.aibvo) * sizeof(unsigned long);
275 npages = DIV_ROUND_UP((fib->fmt0.aibv & ~PAGE_MASK) + size, PAGE_SIZE);
276 /* AIBV cannot span more than 1 page */
277 if (npages > 1) {
278 rc = -EINVAL;
279 goto out;
280 }
281
249 idx = srcu_read_lock(&kvm->srcu);282 idx = srcu_read_lock(&kvm->srcu);
250 hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv));283 hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv));
251 npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages);284 npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages);
@@ -261,6 +294,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 9 unmodified lines
261 294
262 /* Pin the guest AISB if one was specified */295 /* Pin the guest AISB if one was specified */
263 if (fib->fmt0.sum == 1) {296 if (fib->fmt0.sum == 1) {
297 /* AISB must be dword aligned */
298 if (fib->fmt0.aisb & 0x7) {
299 rc = -EINVAL;
300 goto unpin1;
301 }
302
264 idx = srcu_read_lock(&kvm->srcu);303 idx = srcu_read_lock(&kvm->srcu);
265 hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb));304 hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb));
266 npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM,305 npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM,
@@ -275,20 +314,28 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 8 unmodified lines
275 }314 }
276 315
277 /* Account for pinned pages, roll back on failure */316 /* Account for pinned pages, roll back on failure */
278 if (account_mem(pcount))317 rc = account_mem(zdev->kzdev, pcount);
318 if (rc)
279 goto unpin2;319 goto unpin2;
280 320
281 /* AISB must be allocated before we can fill in GAITE */321 /* AISB must be allocated before we can fill in GAITE */
282 mutex_lock(&aift->aift_lock);322 mutex_lock(&aift->aift_lock);
283 bit = airq_iv_alloc_bit(aift->sbv);323 bit = airq_iv_alloc_bit(aift->sbv);
284 if (bit == -1UL)324 if (bit == -1UL) {
325 rc = -ENOMEM;
285 goto unlock;326 goto unlock;
327 }
286 zdev->aisb = bit; /* store the summary bit number */328 zdev->aisb = bit; /* store the summary bit number */
287 zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA |329 zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA |
288 AIRQ_IV_BITLOCK |330 AIRQ_IV_BITLOCK |
289 AIRQ_IV_GUESTVEC,331 AIRQ_IV_GUESTVEC,
290 phys_to_virt(fib->fmt0.aibv));332 phys_to_virt(fib->fmt0.aibv));
291 333
334 if (!zdev->aibv) {
335 rc = -ENOMEM;
336 goto free_aisb;
337 }
338
292 spin_lock_irq(&aift->gait_lock);339 spin_lock_irq(&aift->gait_lock);
293 gaite = aift->gait + zdev->aisb;340 gaite = aift->gait + zdev->aisb;
294 341
@@ -311,21 +358,39 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 16 unmodified lines
311 aift->kzdev[zdev->aisb] = zdev->kzdev;358 aift->kzdev[zdev->aisb] = zdev->kzdev;
312 spin_unlock_irq(&aift->gait_lock);359 spin_unlock_irq(&aift->gait_lock);
313 360
314 /* Update guest FIB for re-issue */
315 fib->fmt0.aisbo = zdev->aisb & 63;
316 fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8;
317 fib->fmt0.isc = gisc;
318
319 /* Save some guest fib values in the host for later use */361 /* Save some guest fib values in the host for later use */
320 zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc;362 zdev->kzdev->fib.fmt0.isc = gisc;
321 zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv;363 zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv;
322 mutex_unlock(&aift->aift_lock);
323 364
324 /* Issue the clp to setup the irq now */365 /* Issue the clp to setup the irq now */
325 rc = kvm_zpci_set_airq(zdev);366 rc = kvm_zpci_set_airq(zdev);
326 return rc;367 if (!rc) {
368 mutex_unlock(&aift->aift_lock);
369 return rc;
370 }
371
372 /* Start cleanup */
373 zdev->kzdev->fib.fmt0.isc = 0;
374 zdev->kzdev->fib.fmt0.aibv = 0;
327 375
376 spin_lock_irq(&aift->gait_lock);
377 gaite->count--;
378 gaite->aisb = 0;
379 gaite->gisc = 0;
380 gaite->aisbo = 0;
381 gaite->gisa = 0;
382 aift->kzdev[zdev->aisb] = NULL;
383 spin_unlock_irq(&aift->gait_lock);
384
385 airq_iv_release(zdev->aibv);
386 zdev->aibv = NULL;
387
388free_aisb:
389 airq_iv_free_bit(aift->sbv, zdev->aisb);
390 zdev->aisb = 0;
328unlock:391unlock:
392 if (pcount > 0)
393 unaccount_mem(zdev->kzdev, pcount);
329 mutex_unlock(&aift->aift_lock);394 mutex_unlock(&aift->aift_lock);
330unpin2:395unpin2:
331 if (fib->fmt0.sum == 1)396 if (fib->fmt0.sum == 1)
@@ -396,7 +461,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) · 64 unmodified lines
396 pcount++;461 pcount++;
397 }462 }
398 if (pcount > 0)463 if (pcount > 0)
399 unaccount_mem(pcount);464 unaccount_mem(kzdev, pcount);
400out:465out:
401 mutex_unlock(&aift->aift_lock);466 mutex_unlock(&aift->aift_lock);
402 467
arch/s390/kvm/pci.h+2−0
@@ -22,6 +22,8 @@ struct kvm_zdev { · 21 unmodified lines
22 struct kvm *kvm;22 struct kvm *kvm;
23 struct zpci_fib fib;23 struct zpci_fib fib;
24 struct list_head entry;24 struct list_head entry;
25 struct user_struct *user_account;
26 struct mm_struct *mm_account;
25};27};
26 28
27struct zpci_gaite {29struct zpci_gaite {
arch/x86/kvm/mmu/mmu.c+21−1
@@ -722,6 +722,26 @@ static struct kvm_lpage_info *lpage_info_slot(gfn_t gfn, · 721 unmodified lines
722 return &slot->arch.lpage_info[level - 2][idx];722 return &slot->arch.lpage_info[level - 2][idx];
723}723}
724 724
725static bool kvm_gfn_is_lpage_allowed(struct kvm *kvm,
726 const struct kvm_memory_slot *slot,
727 gfn_t gfn, int level)
728{
729 const struct kvm_memory_slot *other_slot;
730
731 BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2);
732
733 if (lpage_info_slot(gfn, slot, level)->disallow_lpage)
734 return false;
735
736 if (kvm_arch_nr_memslot_as_ids(kvm) > 1) {
737 other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn);
738 if (other_slot && lpage_info_slot(gfn, other_slot, level)->disallow_lpage)
739 return false;
740 }
741
742 return true;
743}
744
725/*745/*
726 * The most significant bit in disallow_lpage tracks whether or not memory746 * The most significant bit in disallow_lpage tracks whether or not memory
727 * attributes are mixed, i.e. not identical for all gfns at the current level.747 * attributes are mixed, i.e. not identical for all gfns at the current level.
@@ -2968,7 +2988,7 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, · 2240 unmodified lines
2968 * write-protected (see above), thus if the gfn can be mapped with a2988 * write-protected (see above), thus if the gfn can be mapped with a
2969 * hugepage and isn't write-tracked, it can't have a shadow page.2989 * hugepage and isn't write-tracked, it can't have a shadow page.
2970 */2990 */
2971 if (!lpage_info_slot(gfn, slot, PG_LEVEL_2M)->disallow_lpage)2991 if (kvm_gfn_is_lpage_allowed(kvm, slot, gfn, PG_LEVEL_2M))
2972 return 0;2992 return 0;
2973 2993
2974 /*2994 /*
arch/x86/kvm/mmu/page_track.c+26−6
@@ -130,13 +130,23 @@ void __kvm_write_track_remove_gfn(struct kvm *kvm, · 129 unmodified lines
130 kvm_mmu_gfn_allow_lpage(slot, gfn);130 kvm_mmu_gfn_allow_lpage(slot, gfn);
131}131}
132 132
133/*133static bool __kvm_gfn_is_write_tracked(const struct kvm_memory_slot *slot,
134 * check if the corresponding access on the specified guest page is tracked.134 gfn_t gfn)
135 */135{
136 int index;
137
138 if (!slot)
139 return false;
140
141 index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K);
142 return !!READ_ONCE(slot->arch.gfn_write_track[index]);
143}
144
145/* check if write access is tracked on the specified guest page. */
136bool kvm_gfn_is_write_tracked(struct kvm *kvm,146bool kvm_gfn_is_write_tracked(struct kvm *kvm,
137 const struct kvm_memory_slot *slot, gfn_t gfn)147 const struct kvm_memory_slot *slot, gfn_t gfn)
138{148{
139 int index;149 const struct kvm_memory_slot *other_slot;
140 150
141 if (!slot)151 if (!slot)
142 return false;152 return false;
@@ -144,8 +154,18 @@ bool kvm_gfn_is_write_tracked(struct kvm *kvm, · 1 unmodified line
144 if (!kvm_page_track_write_tracking_enabled(kvm))154 if (!kvm_page_track_write_tracking_enabled(kvm))
145 return false;155 return false;
146 156
147 index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K);157 BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2);
148 return !!READ_ONCE(slot->arch.gfn_write_track[index]);158
159 if (__kvm_gfn_is_write_tracked(slot, gfn))
160 return true;
161
162 if (kvm_arch_nr_memslot_as_ids(kvm) > 1) {
163 other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn);
164 if (__kvm_gfn_is_write_tracked(other_slot, gfn))
165 return true;
166 }
167
168 return false;
149}169}
150 170
151#ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING171#ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING
arch/x86/kvm/svm/avic.c+0−8
@@ -240,14 +240,6 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) · 239 unmodified lines
240 if (!is_sev_es_guest(&svm->vcpu))240 if (!is_sev_es_guest(&svm->vcpu))
241 svm_set_intercept(svm, INTERCEPT_CR8_WRITE);241 svm_set_intercept(svm, INTERCEPT_CR8_WRITE);
242 242
243 /*
244 * If running nested and the guest uses its own MSR bitmap, there
245 * is no need to update L0's msr bitmap
246 */
247 if (is_guest_mode(&svm->vcpu) &&
248 vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT))
249 return;
250
251 /* Enabling MSR intercept for x2APIC registers */243 /* Enabling MSR intercept for x2APIC registers */
252 avic_set_x2apic_msr_interception(svm, true);244 avic_set_x2apic_msr_interception(svm, true);
253}245}
arch/x86/kvm/vmx/vmx_ops.h+4−4
@@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) · 100 unmodified lines
101 101
102 : [output] "=r" (value)102 : [output] "=r" (value)
103 : [field] "r" (field)103 : [field] "r" (field)
104 : "cc"104 : "cc", "memory"
105 : do_fail, do_exception);105 : do_fail, do_exception);
106 106
107 return value;107 return value;
@@ -145,7 +145,7 @@ do_exception: · 37 unmodified lines
145 145
146 : ASM_CALL_CONSTRAINT, [output] "=&r" (value)146 : ASM_CALL_CONSTRAINT, [output] "=&r" (value)
147 : [field] "r" (field)147 : [field] "r" (field)
148 : "cc");148 : "cc", "memory");
149 return value;149 return value;
150 150
151#endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */151#endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
@@ -192,7 +192,7 @@ do { \ · 40 unmodified lines
192 asm goto("1: " __stringify(insn) " %0\n\t" \192 asm goto("1: " __stringify(insn) " %0\n\t" \
193 "jna %l[error]\n\t" \193 "jna %l[error]\n\t" \
194 _ASM_EXTABLE(1b, %l[fault]) \194 _ASM_EXTABLE(1b, %l[fault]) \
195 : : op1 : "cc" : error, fault); \195 : : op1 : "cc", "memory" : error, fault); \
196 return; \196 return; \
197error: \197error: \
198 instrumentation_begin(); \198 instrumentation_begin(); \
@@ -208,7 +208,7 @@ do { \ · 9 unmodified lines
208 asm goto("1: " __stringify(insn) " %1, %0\n\t" \208 asm goto("1: " __stringify(insn) " %1, %0\n\t" \
209 "jna %l[error]\n\t" \209 "jna %l[error]\n\t" \
210 _ASM_EXTABLE(1b, %l[fault]) \210 _ASM_EXTABLE(1b, %l[fault]) \
211 : : op1, op2 : "cc" : error, fault); \211 : : op1, op2 : "cc", "memory" : error, fault);\
212 return; \212 return; \
213error: \213error: \
214 instrumentation_begin(); \214 instrumentation_begin(); \
arch/x86/kvm/x86.c+6−0
@@ -13429,9 +13429,15 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm) · 13428 unmodified lines
13429 * iterating over vCPUs in a different task while vCPUs are being freed13429 * iterating over vCPUs in a different task while vCPUs are being freed
13430 * is unsafe, i.e. will lead to use-after-free. The PIT also needs to13430 * is unsafe, i.e. will lead to use-after-free. The PIT also needs to
13431 * be stopped before IRQ routing is freed.13431 * be stopped before IRQ routing is freed.
13432 *
13433 * Do NOT free the in-kernel PIC or I/O APIC here (but as above, make
13434 * sure to flush any background work), as KVM expects interrupt routing
13435 * structures to be valid until vCPUs are destroyed.
13432 */13436 */
13433#ifdef CONFIG_KVM_IOAPIC13437#ifdef CONFIG_KVM_IOAPIC
13434 kvm_free_pit(kvm);13438 kvm_free_pit(kvm);
13439 if (kvm->arch.vioapic)
13440 cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject);
13435#endif13441#endif
13436 13442
13437 kvm_mmu_pre_destroy_vm(kvm);13443 kvm_mmu_pre_destroy_vm(kvm);
include/kvm/arm_vgic.h+0−3
@@ -247,9 +247,6 @@ struct vgic_irq { · 246 unmodified lines
247 * affinity reg (v3).247 * affinity reg (v3).
248 */248 */
249 249
250 bool pending_release:1; /* Used for LPIs only, unreferenced IRQ
251 * pending a release */
252
253 bool pending_latch:1; /* The pending latch state used to calculate250 bool pending_latch:1; /* The pending latch state used to calculate
254 * the pending state for both level251 * the pending state for both level
255 * and edge triggered IRQs. */252 * and edge triggered IRQs. */

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t