Merge branches 'fixes', 'misc', 'mmu', 'pvclock', 'selftests', 'selftests_6.14',...

* fixes:
  KVM: x86: Load DR6 with guest value only before entering .vcpu_run() loop
  KVM: nSVM: Enter guest mode before initializing nested NPT MMU
  KVM: selftests: Add CPUID tests for Hyper-V features that need in-kernel APIC
  KVM: selftests: Manage CPUID array in Hyper-V CPUID test's core helper
  KVM: selftests: Mark test_hv_cpuid_e2big() static in Hyper-V CPUID test
  KVM: x86: Reject Hyper-V's SEND_IPI hypercalls if local APIC isn't in-kernel

* misc:
  KVM: x86: Defer runtime updates of dynamic CPUID bits until CPUID emulation
  KVM: x86: Query X86_FEATURE_MWAIT iff userspace owns the CPUID feature bit
  KVM: x86: Apply TSX_CTRL_CPUID_CLEAR if and only if the vCPU has RTM or HLE
  KVM: x86: Use for-loop to iterate over XSTATE size entries
  KVM: x86/cpuid: add type suffix to decimal const 48 fix building warning
  KVM: x86: Clear pv_unhalted on all transitions to KVM_MP_STATE_RUNNABLE
  KVM: x86: Introduce kvm_set_mp_state()
  KVM: x86: Use kvfree_rcu() to free old optimized APIC map
  KVM: x86: Wake vCPU for PIC interrupt injection iff a valid IRQ was found
  KVM: x86: Remove unused iommu_domain and iommu_noncoherent from kvm_arch

* mmu:
  KVM: x86/mmu: Walk rmaps (shadow MMU) without holding mmu_lock when aging gfns
  KVM: x86/mmu: Add support for lockless walks of rmap SPTEs
  KVM: x86/mmu: Add infrastructure to allow walking rmaps outside of mmu_lock
  KVM: x86/mmu: Refactor low level rmap helpers to prep for walking w/o mmu_lock
  KVM: x86/mmu: Only check gfn age in shadow MMU if indirect_shadow_pages > 0
  KVM: x86/mmu: Skip shadow MMU test_young if TDP MMU reports page as young
  KVM: x86/mmu: Age TDP MMU SPTEs without holding mmu_lock
  KVM: x86/mmu: Always update A/D-disabled SPTEs atomically
  KVM: x86/mmu: Don't force atomic update if only the Accessed bit is volatile
  KVM: x86/mmu: Factor out spte atomic bit clearing routine
  KVM: Allow lockless walk of SPTEs when handing aging mmu_notifier event
  KVM: Rename kvm_handle_hva_range()

* pvclock:
  KVM: x86: Override TSC_STABLE flag for Xen PV clocks in kvm_guest_time_update()
  KVM: x86: Setup Hyper-V TSC page before Xen PV clocks (during clock update)
  KVM: x86: Remove per-vCPU "cache" of its reference pvclock
  KVM: x86: Pass reference pvclock as a param to kvm_setup_guest_pvclock()
  KVM: x86: Set PVCLOCK_GUEST_STOPPED only for kvmclock, not for Xen PV clock
  KVM: x86: Don't bleed PVCLOCK_GUEST_STOPPED across PV clocks
  KVM: x86/xen: Use guest's copy of pvclock when starting timer
  KVM: x86: Process "guest stopped request" once per guest time update
  KVM: x86: Drop local pvclock_flags variable in kvm_guest_time_update()
  KVM: x86: Eliminate "handling" of impossible errors during SUSPEND
  KVM: x86: Don't take kvm->lock when iterating over vCPUs in suspend notifier

* selftests: (29 commits)
  KVM: selftests: Add infrastructure for getting vCPU binary stats
  KVM: selftests: Adjust number of files rlimit for all "standard" VMs
  KVM: selftests: Get VM's binary stats FD when opening VM
  KVM: selftests: Add struct and helpers to wrap binary stats cache
  KVM: selftests: Macrofy vm_get_stat() to auto-generate stat name string
  KVM: selftests: Assert that __vm_get_stat() actually finds a stat
  KVM: selftests: Close VM's binary stats FD when releasing VM
  KVM: selftests: Fix mostly theoretical leak of VM's binary stats FD
  KVM: selftests: Allow running a single iteration of dirty_log_test
  KVM: selftests: Fix an off-by-one in the number of dirty_log_test iterations
  KVM: selftests: Set per-iteration variables at the start of each iteration
  KVM: selftests: Tighten checks around prev iter's last dirty page in ring
  KVM: selftests: Ensure guest writes min number of pages in dirty_log_test
  KVM: sefltests: Verify value of dirty_log_test last page isn't bogus
  KVM: selftests: Collect *all* dirty entries in each dirty_log_test iteration
  KVM: selftests: Print (previous) last_page on dirty page value mismatch
  KVM: selftests: Use continue to handle all "pass" scenarios in dirty_log_test
  KVM: selftests: Post to sem_vcpu_stop if and only if vcpu_stop is true
  KVM: selftests: Keep dirty_log_test vCPU in guest until it needs to stop
  KVM: selftests: Honor "stop" request in dirty ring test
  ...

* selftests_6.14:
  KVM: selftests: Print out the actual Top-Down Slots count on failure
  KVM: selftests: Drop the "feature event" param from guest test helpers
  KVM: selftests: Remove dead code in Intel PMU counters test
  KVM: selftests: Only validate counts for hardware-supported arch events
  KVM: selftests: Make Intel arch events globally available in PMU counters test
  KVM: selftests: Add helpers for locally (un)blocking IRQs on x86
  KVM: selftests: Use data load to trigger LLC references/misses in Intel PMU
  KVM: selftests: Add printf attribute to _no_printf()
  KVM: selftests: Remove unneeded semicolon
  KVM: selftests: Add defines for AMD PMU CPUID features and properties
  KVM: selftests: Fix typos in x86's PMU counter test's macro variable use

* svm:
  KVM: SEV: Use long-term pin when registering encrypted memory regions
  KVM: SVM: Ensure PSP module is initialized if KVM module is built-in
  crypto: ccp: Add external API interface for PSP module initialization
  KVM: SEV: Use to_kvm_sev_info() for fetching kvm_sev_info struct

* xen:
  KVM: x86/xen: Only write Xen hypercall page for guest writes to MSR

diff --cc arch/x86/include/asm/kvm_host.h
index b15cde0a9b5ca7ae2c90b15fed4a4debd34f825c,0b7af5902ff757238dd8a91fa21979203a88b191,94b14b3a3b8d2b71f8dc3520d7c87618c093c743,51188aa31008f9cb620f6a051e56e54248f268a3,14b2d033925e6b48698d65460c7b8b02ada38638,b15cde0a9b5ca7ae2c90b15fed4a4debd34f825c,e159e44a6a1b61dba500b5f4709031c93828794d,b15cde0a9b5ca7ae2c90b15fed4a4debd34f825c,b15cde0a9b5ca7ae2c90b15fed4a4debd34f825c..3506f497741bf2b5bd429cfe5c8c9e7f912bab4a
--- 1/arch/x86/include/asm/kvm_host.h
--- 2/arch/x86/include/asm/kvm_host.h
--- 3/arch/x86/include/asm/kvm_host.h
--- 4/arch/x86/include/asm/kvm_host.h
--- 5/arch/x86/include/asm/kvm_host.h
--- 6/arch/x86/include/asm/kvm_host.h
--- 7/arch/x86/include/asm/kvm_host.h
--- 8/arch/x86/include/asm/kvm_host.h
--- 9/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@@@@@@@@@ -24,9 -24,9 -24,9 -24,10 -24,9 -24,9 -24,9 -24,9 -24,9 +24,10 @@@@@@@@@@
         #include <linux/pvclock_gtod.h>
         #include <linux/clocksource.h>
         #include <linux/irqbypass.h>
      -  #include <linux/hyperv.h>
         #include <linux/kfifo.h>
         #include <linux/sched/vhost_task.h>
      +  #include <linux/call_once.h>
+++ +++++#include <linux/atomic.h>
         
         #include <asm/apic.h>
         #include <asm/pvclock-abi.h>
@@@@@@@@@@ -879,6 -879,6 -879,7 -880,6 -879,6 -879,6 -854,7 -879,6 -879,6 +880,7 @@@@@@@@@@ struct kvm_vcpu_arch 
         
                int cpuid_nent;
                struct kvm_cpuid_entry2 *cpuid_entries;
      -         struct kvm_hypervisor_cpuid kvm_cpuid;
++ ++++++       bool cpuid_dynamic_bits_dirty;
                bool is_amd_compatible;
         
                /*

diff --cc arch/x86/kvm/mmu/mmu.c
Simple merge

diff --cc arch/x86/kvm/svm/nested.c
Simple merge

diff --cc arch/x86/kvm/svm/sev.c
Simple merge

diff --cc arch/x86/kvm/svm/svm.c
Simple merge

diff --cc arch/x86/kvm/vmx/vmx.c
Simple merge

diff --cc arch/x86/kvm/x86.c
index 8e77e61d4fbd4dc0de7bfd900270e887ff81296c,02159c967d29e5f9134a9ceba8f9ebffd9fa96f6,263ab2ec6f712394cc5e16ac282eaf1df6ef5506,8e77e61d4fbd4dc0de7bfd900270e887ff81296c,3e9bead6a65fe69a0c177bb40e9a4a801f3f5855,8e77e61d4fbd4dc0de7bfd900270e887ff81296c,1b04092ec76aa4be07bad196eef7848f1396d638,8e77e61d4fbd4dc0de7bfd900270e887ff81296c,462a5cd6ac4a8621c30824f81144dca6afd90050..58b82d6fd77c1d56c6f07c07cbcd6adb9e5c7fb3
--- 1/arch/x86/kvm/x86.c
--- 2/arch/x86/kvm/x86.c
--- 3/arch/x86/kvm/x86.c
--- 4/arch/x86/kvm/x86.c
--- 5/arch/x86/kvm/x86.c
--- 6/arch/x86/kvm/x86.c
--- 7/arch/x86/kvm/x86.c
--- 8/arch/x86/kvm/x86.c
--- 9/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@@@@@@@@@ -2080,10 -2080,10 -2080,20 -2080,10 -2080,10 -2080,10 -2094,10 -2080,10 -2080,10 +2080,20 @@@@@@@@@@ EXPORT_SYMBOL_GPL(kvm_handle_invalid_op
         
         static int kvm_emulate_monitor_mwait(struct kvm_vcpu *vcpu, const char *insn)
         {
-- ------       if (!kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS) &&
-- --- --           !guest_cpu_cap_has(vcpu, X86_FEATURE_MWAIT))
      -             !guest_cpuid_has(vcpu, X86_FEATURE_MWAIT))
++ ++++++       bool enabled;
++ ++++++
++ ++++++       if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS))
++ ++++++               goto emulate_as_nop;
++ ++++++
++ ++++++       if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT))
++ ++++++               enabled = guest_cpu_cap_has(vcpu, X86_FEATURE_MWAIT);
++ ++++++       else
++ ++++++               enabled = vcpu->arch.ia32_misc_enable_msr & MSR_IA32_MISC_ENABLE_MWAIT;
++ ++++++
++ ++++++       if (!enabled)
                        return kvm_handle_invalid_op(vcpu);
         
++ ++++++emulate_as_nop:
                pr_warn_once("%s instruction emulated as NOP!\n", insn);
                return kvm_emulate_as_nop(vcpu);
         }
@@@@@@@@@@ -3148,17 -3148,17 -3158,17 -3148,17 -3150,9 -3148,17 -3162,17 -3148,17 -3148,17 +3160,9 @@@@@@@@@@ static void kvm_setup_guest_pvclock(str
                smp_wmb();
         
                /* retain PVCLOCK_GUEST_STOPPED if set in guest copy */
---- ----       vcpu->hv_clock.flags |= (guest_hv_clock->flags & PVCLOCK_GUEST_STOPPED);
---- ----
---- ----       if (vcpu->pvclock_set_guest_stopped_request) {
---- ----               vcpu->hv_clock.flags |= PVCLOCK_GUEST_STOPPED;
---- ----               vcpu->pvclock_set_guest_stopped_request = false;
---- ----       }
  -   -  
  -   -         memcpy(guest_hv_clock, &vcpu->hv_clock, sizeof(*guest_hv_clock));
++++ ++++       hv_clock.flags |= (guest_hv_clock->flags & PVCLOCK_GUEST_STOPPED);
         
-- - - --       memcpy(guest_hv_clock, &vcpu->hv_clock, sizeof(*guest_hv_clock));
-- - - --
---- ----       if (force_tsc_unstable)
---- ----               guest_hv_clock->flags &= ~PVCLOCK_TSC_STABLE_BIT;
++++ ++++       memcpy(guest_hv_clock, &hv_clock, sizeof(*guest_hv_clock));
         
                smp_wmb();
         
@@@@@@@@@@ -3261,24 -3261,24 -3271,24 -3261,24 -3247,45 -3261,24 -3275,24 -3261,24 -3261,24 +3257,45 @@@@@@@@@@ static int kvm_guest_time_update(struc
                vcpu->last_guest_tsc = tsc_timestamp;
         
                /* If the host uses TSC clocksource, then it is stable */
---- ----       pvclock_flags = 0;
++++ ++++       hv_clock.flags = 0;
                if (use_master_clock)
---- ----               pvclock_flags |= PVCLOCK_TSC_STABLE_BIT;
++++ ++++               hv_clock.flags |= PVCLOCK_TSC_STABLE_BIT;
++++ + ++
      -         vcpu->hv_clock.flags = pvclock_flags;
++++ ++++       if (vcpu->pv_time.active) {
++++ ++++               /*
++++ ++++                * GUEST_STOPPED is only supported by kvmclock, and KVM's
++++ ++++                * historic behavior is to only process the request if kvmclock
++++ ++++                * is active/enabled.
++++ ++++                */
++++ ++++               if (vcpu->pvclock_set_guest_stopped_request) {
++++ ++++                       hv_clock.flags |= PVCLOCK_GUEST_STOPPED;
++++ ++++                       vcpu->pvclock_set_guest_stopped_request = false;
++++ ++++               }
++++ ++++               kvm_setup_guest_pvclock(&hv_clock, v, &vcpu->pv_time, 0);
++ + +++ 
  -     -       vcpu->hv_clock.flags = pvclock_flags;
++++ ++++               hv_clock.flags &= ~PVCLOCK_GUEST_STOPPED;
++++ ++++       }
  +   + +
-- - - -        vcpu->hv_clock.flags = pvclock_flags;
++++ ++++       kvm_hv_setup_tsc_page(v->kvm, &hv_clock);
         
---- ----       if (vcpu->pv_time.active)
---- ----               kvm_setup_guest_pvclock(v, &vcpu->pv_time, 0, false);
         #ifdef CONFIG_KVM_XEN
++++ ++++       /*
++++ ++++        * For Xen guests we may need to override PVCLOCK_TSC_STABLE_BIT as unless
++++ ++++        * explicitly told to use TSC as its clocksource Xen will not set this bit.
++++ ++++        * This default behaviour led to bugs in some guest kernels which cause
++++ ++++        * problems if they observe PVCLOCK_TSC_STABLE_BIT in the pvclock flags.
++++ ++++        *
++++ ++++        * Note!  Clear TSC_STABLE only for Xen clocks, i.e. the order matters!
++++ ++++        */
++++ ++++       if (ka->xen_hvm_config.flags & KVM_XEN_HVM_CONFIG_PVCLOCK_TSC_UNSTABLE)
++++ ++++               hv_clock.flags &= ~PVCLOCK_TSC_STABLE_BIT;
++++ ++++
                if (vcpu->xen.vcpu_info_cache.active)
---- ----               kvm_setup_guest_pvclock(v, &vcpu->xen.vcpu_info_cache,
---- ----                                       offsetof(struct compat_vcpu_info, time),
---- ----                                       xen_pvclock_tsc_unstable);
++++ ++++               kvm_setup_guest_pvclock(&hv_clock, v, &vcpu->xen.vcpu_info_cache,
++++ ++++                                       offsetof(struct compat_vcpu_info, time));
                if (vcpu->xen.vcpu_time_info_cache.active)
---- ----               kvm_setup_guest_pvclock(v, &vcpu->xen.vcpu_time_info_cache, 0,
---- ----                                       xen_pvclock_tsc_unstable);
++++ ++++               kvm_setup_guest_pvclock(&hv_clock, v, &vcpu->xen.vcpu_time_info_cache, 0);
         #endif
---- ----       kvm_hv_setup_tsc_page(v->kvm, &vcpu->hv_clock);
                return 0;
         }
         
@@@@@@@@@@ -3886,10 -3886,10 -3896,10 -3886,10 -3893,10 -3886,10 -3900,10 -3886,10 -3892,10 +3909,10 @@@@@@@@@@ int kvm_set_msr_common(struct kvm_vcpu 
         
                        if (!kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT) &&
                            ((old_val ^ data)  & MSR_IA32_MISC_ENABLE_MWAIT)) {
      -                         if (!guest_cpuid_has(vcpu, X86_FEATURE_XMM3))
      +                         if (!guest_cpu_cap_has(vcpu, X86_FEATURE_XMM3))
                                        return 1;
                                vcpu->arch.ia32_misc_enable_msr = data;
-- ------                       kvm_update_cpuid_runtime(vcpu);
++ ++++++                       vcpu->arch.cpuid_dynamic_bits_dirty = true;
                        } else {
                                vcpu->arch.ia32_misc_enable_msr = data;
                        }
@@@@@@@@@@ -6905,23 -6905,23 -6915,23 -6905,23 -6912,15 -6905,23 -6914,23 -6905,23 -6911,23 +6928,15 @@@@@@@@@@ static int kvm_arch_suspend_notifier(st
         {
                struct kvm_vcpu *vcpu;
                unsigned long i;
---- ----       int ret = 0;
      - -
      - -       mutex_lock(&kvm->lock);
      - -       kvm_for_each_vcpu(i, vcpu, kvm) {
      - -               if (!vcpu->arch.pv_time.active)
      - -                       continue;
         
---- - -        mutex_lock(&kvm->lock);
---- - -        kvm_for_each_vcpu(i, vcpu, kvm) {
---- - -                if (!vcpu->arch.pv_time.active)
---- - -                        continue;
---- - - 
---- ----               ret = kvm_set_guest_paused(vcpu);
---- ----               if (ret) {
---- ----                       kvm_err("Failed to pause guest VCPU%d: %d\n",
---- ----                               vcpu->vcpu_id, ret);
---- ----                       break;
---- ----               }
---- ----       }
---- ----       mutex_unlock(&kvm->lock);
++++ ++++       /*
++++ ++++        * Ignore the return, marking the guest paused only "fails" if the vCPU
++++ ++++        * isn't using kvmclock; continuing on is correct and desirable.
++++ ++++        */
++++ ++++       kvm_for_each_vcpu(i, vcpu, kvm)
++++ ++++               (void)kvm_set_guest_paused(vcpu);
         
---- ----       return ret ? NOTIFY_BAD : NOTIFY_DONE;
++++ ++++       return NOTIFY_DONE;
         }
         
         int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state)

diff --cc arch/x86/kvm/xen.c
Simple merge

diff --cc tools/testing/selftests/kvm/include/x86/processor.h
Simple merge

diff --cc tools/testing/selftests/kvm/x86/xapic_ipi_test.c
Simple merge