lisa/android_kernel_xiaomi_sm8350
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
a0e90ad8ce
android_kernel_xiaomi_sm8350/arch/mips/kvm/tlb.c

661 lines
16 KiB
C
Raw Normal View History

KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
/*
MIPS: KVM: Reformat code and comments No logic changes inside. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:34 -04:00
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* KVM/MIPS TLB handling, this file is part of the Linux host kernel so that
* TLB handlers run from KSEG0
*
* Copyright (C) 2012 MIPS Technologies, Inc. All rights reserved.
* Authors: Sanjay Lal <sanjayl@kymasys.com>
*/
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
#include <linux/sched.h>
#include <linux/smp.h>
#include <linux/mm.h>
#include <linux/delay.h>
MIPS: KVM: Move non-TLB handling code out of tlb.c Various functions in tlb.c perform higher level MMU handling, but don't strictly need to be statically built into the kernel as they don't directly manipulate TLB entries. Move these functions out into a separate mmu.c which will be built into the KVM kernel module. This allows them to directly reference KVM functions in the KVM kernel module in future. Module exports of these functions have been removed, since they aren't needed outside of KVM. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:10 -04:00
#include <linux/export.h>
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
#include <linux/kvm_host.h>
KVM/MIPS32: Wrap calls to gfn_to_pfn() with srcu_read_lock/unlock() - As suggested by Gleb, wrap calls to gfn_to_pfn() with srcu_read_lock/unlock(). Memory slots should be acccessed from a SRCU read section. - kvm_mips_map_page() now returns an error code to it's callers, instead of calling panic() if it cannot find a mapping for a particular gfn. Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Signed-off-by: Gleb Natapov <gleb@redhat.com>
2013-05-18 09:54:24 -04:00
#include <linux/srcu.h>
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
#include <asm/cpu.h>
#include <asm/bootinfo.h>
#include <asm/mmu_context.h>
#include <asm/pgtable.h>
#include <asm/cacheflush.h>
MIPS: KVM: use common EHINV aware UNIQUE_ENTRYHI When KVM is enabled and TLB invalidation is supported, kvm_mips_flush_host_tlb() can cause a machine check exception due to multiple matching TLB entries. This can occur on shutdown even when KVM hasn't been actively used. Commit adb78de9eae8 (MIPS: mm: Move UNIQUE_ENTRYHI macro to a header file) created a common UNIQUE_ENTRYHI in asm/tlb.h but it didn't update the copy of UNIQUE_ENTRYHI in kvm_tlb.c to use it. Commit 36b175451399 (MIPS: tlb: Set the EHINV bit for TLBINVF cores when invalidating the TLB) later added TLB invalidation (EHINV) support to the common UNIQUE_ENTRYHI. Therefore make kvm_tlb.c use the EHINV aware UNIQUE_ENTRYHI implementation in asm/tlb.h too. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: Gleb Natapov <gleb@redhat.com> Cc: kvm@vger.kernel.org Cc: Sanjay Lal <sanjayl@kymasys.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: John Crispin <blogic@openwrt.org> Patchwork: http://patchwork.linux-mips.org/patch/6383/
2014-01-17 07:01:30 -05:00
#include <asm/tlb.h>
MIPS: KVM: Use dump_tlb_all() for kvm_mips_dump_host_tlbs() KVM implements its own routine for dumping the host TLB entries, but we already have dump_tlb_all() which does something very similar (although it only prints out TLB entries which match the current ASID or are global). Make KVM use dump_tlb_all() along with dump_tlb_regs() to avoid the duplication and inevitable bitrot, allowing TLB dumping enhancements (e.g. for VZ and GuestIDs) to be made in a single place. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:16 -04:00
#include <asm/tlbdebug.h>
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
#undef CONFIG_MIPS_MT
#include <asm/r4kcache.h>
#define CONFIG_MIPS_MT
#define KVM_GUEST_PC_TLB 0
#define KVM_GUEST_SP_TLB 1
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
#ifdef CONFIG_KVM_MIPS_VZ
KVM: MIPS: Implement VZ support Add the main support for the MIPS Virtualization ASE (A.K.A. VZ) to MIPS KVM. The bulk of this work is in vz.c, with various new state and definitions elsewhere. Enough is implemented to be able to run on a minimal VZ core. Further patches will fill out support for guest features which are optional or can be disabled. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Cc: linux-doc@vger.kernel.org
2017-03-14 06:15:31 -04:00
unsigned long GUESTID_MASK;
EXPORT_SYMBOL_GPL(GUESTID_MASK);
unsigned long GUESTID_FIRST_VERSION;
EXPORT_SYMBOL_GPL(GUESTID_FIRST_VERSION);
unsigned long GUESTID_VERSION_MASK;
EXPORT_SYMBOL_GPL(GUESTID_VERSION_MASK);
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
static u32 kvm_mips_get_root_asid(struct kvm_vcpu *vcpu)
{
struct mm_struct *gpa_mm = &vcpu->kvm->arch.gpa_mm;
if (cpu_has_guestid)
return 0;
else
return cpu_asid(smp_processor_id(), gpa_mm);
}
#endif
MIPS: KVM: Move non-TLB handling code out of tlb.c Various functions in tlb.c perform higher level MMU handling, but don't strictly need to be statically built into the kernel as they don't directly manipulate TLB entries. Move these functions out into a separate mmu.c which will be built into the KVM kernel module. This allows them to directly reference KVM functions in the KVM kernel module in future. Module exports of these functions have been removed, since they aren't needed outside of KVM. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:10 -04:00
static u32 kvm_mips_get_kernel_asid(struct kvm_vcpu *vcpu)
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
{
KVM: MIPS: Remove duplicated ASIDs from vcpu The kvm_vcpu_arch structure contains both mm_structs for allocating MMU contexts (primarily the ASID) but it also copies the resulting ASIDs into guest_{user,kernel}_asid[] arrays which are referenced from uasm generated code. This duplication doesn't seem to serve any purpose, and it gets in the way of generalising the ASID handling across guest kernel/user modes, so lets just extract the ASID straight out of the mm_struct on demand, and in fact there are convenient cpu_context() and cpu_asid() macros for doing so. To reduce the verbosity of this code we do also add kern_mm and user_mm local variables where the kernel and user mm_structs are used. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-11 18:14:39 -04:00
struct mm_struct *kern_mm = &vcpu->arch.guest_kernel_mm;
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 09:36:23 -04:00
int cpu = smp_processor_id();
KVM: MIPS: Remove duplicated ASIDs from vcpu The kvm_vcpu_arch structure contains both mm_structs for allocating MMU contexts (primarily the ASID) but it also copies the resulting ASIDs into guest_{user,kernel}_asid[] arrays which are referenced from uasm generated code. This duplication doesn't seem to serve any purpose, and it gets in the way of generalising the ASID handling across guest kernel/user modes, so lets just extract the ASID straight out of the mm_struct on demand, and in fact there are convenient cpu_context() and cpu_asid() macros for doing so. To reduce the verbosity of this code we do also add kern_mm and user_mm local variables where the kernel and user mm_structs are used. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-11 18:14:39 -04:00
return cpu_asid(cpu, kern_mm);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
}
MIPS: KVM: Move non-TLB handling code out of tlb.c Various functions in tlb.c perform higher level MMU handling, but don't strictly need to be statically built into the kernel as they don't directly manipulate TLB entries. Move these functions out into a separate mmu.c which will be built into the KVM kernel module. This allows them to directly reference KVM functions in the KVM kernel module in future. Module exports of these functions have been removed, since they aren't needed outside of KVM. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:10 -04:00
static u32 kvm_mips_get_user_asid(struct kvm_vcpu *vcpu)
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
{
KVM: MIPS: Remove duplicated ASIDs from vcpu The kvm_vcpu_arch structure contains both mm_structs for allocating MMU contexts (primarily the ASID) but it also copies the resulting ASIDs into guest_{user,kernel}_asid[] arrays which are referenced from uasm generated code. This duplication doesn't seem to serve any purpose, and it gets in the way of generalising the ASID handling across guest kernel/user modes, so lets just extract the ASID straight out of the mm_struct on demand, and in fact there are convenient cpu_context() and cpu_asid() macros for doing so. To reduce the verbosity of this code we do also add kern_mm and user_mm local variables where the kernel and user mm_structs are used. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-11 18:14:39 -04:00
struct mm_struct *user_mm = &vcpu->arch.guest_user_mm;
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 09:36:23 -04:00
int cpu = smp_processor_id();
KVM: MIPS: Remove duplicated ASIDs from vcpu The kvm_vcpu_arch structure contains both mm_structs for allocating MMU contexts (primarily the ASID) but it also copies the resulting ASIDs into guest_{user,kernel}_asid[] arrays which are referenced from uasm generated code. This duplication doesn't seem to serve any purpose, and it gets in the way of generalising the ASID handling across guest kernel/user modes, so lets just extract the ASID straight out of the mm_struct on demand, and in fact there are convenient cpu_context() and cpu_asid() macros for doing so. To reduce the verbosity of this code we do also add kern_mm and user_mm local variables where the kernel and user mm_structs are used. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-11 18:14:39 -04:00
return cpu_asid(cpu, user_mm);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
}
MIPS: KVM: Reformat code and comments No logic changes inside. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:34 -04:00
/* Structure defining an tlb entry data set. */
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
void kvm_mips_dump_host_tlbs(void)
{
unsigned long flags;
local_irq_save(flags);
MIPS: KVM: Use KVM internal logger Replace printks with kvm_[err|info|debug]. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:35 -04:00
kvm_info("HOST TLBs:\n");
MIPS: KVM: Use dump_tlb_all() for kvm_mips_dump_host_tlbs() KVM implements its own routine for dumping the host TLB entries, but we already have dump_tlb_all() which does something very similar (although it only prints out TLB entries which match the current ASID or are global). Make KVM use dump_tlb_all() along with dump_tlb_regs() to avoid the duplication and inevitable bitrot, allowing TLB dumping enhancements (e.g. for VZ and GuestIDs) to be made in a single place. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:16 -04:00
dump_tlb_regs();
pr_info("\n");
dump_tlb_all();
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
local_irq_restore(flags);
}
MIPS: KVM: Convert EXPORT_SYMBOL to _GPL Export symbols only to GPL modules to match other KVM symbols in virt/kvm/ and arch/*/kvm/. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Gleb Natapov <gleb@kernel.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11887/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-12-16 18:49:30 -05:00
EXPORT_SYMBOL_GPL(kvm_mips_dump_host_tlbs);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
void kvm_mips_dump_guest_tlbs(struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_mips_tlb tlb;
int i;
MIPS: KVM: Use KVM internal logger Replace printks with kvm_[err|info|debug]. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:35 -04:00
kvm_info("Guest TLBs:\n");
kvm_info("Guest EntryHi: %#lx\n", kvm_read_c0_guest_entryhi(cop0));
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
for (i = 0; i < KVM_MIPS_GUEST_TLB_SIZE; i++) {
tlb = vcpu->arch.guest_tlb[i];
MIPS: KVM: Use KVM internal logger Replace printks with kvm_[err|info|debug]. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:35 -04:00
kvm_info("TLB%c%3d Hi 0x%08lx ",
MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h Convert KVM to use the MIPS_ENTRYLO_* definitions from <asm/mipsregs.h> rather than custom definitions in kvm_host.h Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:19 -04:00
(tlb.tlb_lo[0] | tlb.tlb_lo[1]) & ENTRYLO_V
MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo* The values of the EntryLo0 and EntryLo1 registers for a TLB entry are stored in separate members of struct kvm_mips_tlb called tlb_lo0 and tlb_lo1 respectively. To allow future code which needs to manipulate arbitrary EntryLo data in the TLB entry to be simpler and less conditional, replace these members with an array of two elements. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:17 -04:00
? ' ' : '*',
MIPS: KVM: Use KVM internal logger Replace printks with kvm_[err|info|debug]. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:35 -04:00
i, tlb.tlb_hi);
MIPS: KVM: Convert code to kernel sized types Convert the MIPS KVM C code to use standard kernel sized types (e.g. u32) instead of inttypes.h style ones (e.g. uint32_t) or other types as appropriate. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:08 -04:00
kvm_info("Lo0=0x%09llx %c%c attr %lx ",
MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo* The values of the EntryLo0 and EntryLo1 registers for a TLB entry are stored in separate members of struct kvm_mips_tlb called tlb_lo0 and tlb_lo1 respectively. To allow future code which needs to manipulate arbitrary EntryLo data in the TLB entry to be simpler and less conditional, replace these members with an array of two elements. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:17 -04:00
(u64) mips3_tlbpfn_to_paddr(tlb.tlb_lo[0]),
MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h Convert KVM to use the MIPS_ENTRYLO_* definitions from <asm/mipsregs.h> rather than custom definitions in kvm_host.h Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:19 -04:00
(tlb.tlb_lo[0] & ENTRYLO_D) ? 'D' : ' ',
(tlb.tlb_lo[0] & ENTRYLO_G) ? 'G' : ' ',
(tlb.tlb_lo[0] & ENTRYLO_C) >> ENTRYLO_C_SHIFT);
MIPS: KVM: Convert code to kernel sized types Convert the MIPS KVM C code to use standard kernel sized types (e.g. u32) instead of inttypes.h style ones (e.g. uint32_t) or other types as appropriate. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:08 -04:00
kvm_info("Lo1=0x%09llx %c%c attr %lx sz=%lx\n",
MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo* The values of the EntryLo0 and EntryLo1 registers for a TLB entry are stored in separate members of struct kvm_mips_tlb called tlb_lo0 and tlb_lo1 respectively. To allow future code which needs to manipulate arbitrary EntryLo data in the TLB entry to be simpler and less conditional, replace these members with an array of two elements. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:17 -04:00
(u64) mips3_tlbpfn_to_paddr(tlb.tlb_lo[1]),
MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h Convert KVM to use the MIPS_ENTRYLO_* definitions from <asm/mipsregs.h> rather than custom definitions in kvm_host.h Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:19 -04:00
(tlb.tlb_lo[1] & ENTRYLO_D) ? 'D' : ' ',
(tlb.tlb_lo[1] & ENTRYLO_G) ? 'G' : ' ',
(tlb.tlb_lo[1] & ENTRYLO_C) >> ENTRYLO_C_SHIFT,
tlb.tlb_mask);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
}
}
MIPS: KVM: Convert EXPORT_SYMBOL to _GPL Export symbols only to GPL modules to match other KVM symbols in virt/kvm/ and arch/*/kvm/. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Gleb Natapov <gleb@kernel.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11887/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-12-16 18:49:30 -05:00
EXPORT_SYMBOL_GPL(kvm_mips_dump_guest_tlbs);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
int kvm_mips_guest_tlb_lookup(struct kvm_vcpu *vcpu, unsigned long entryhi)
{
int i;
int index = -1;
struct kvm_mips_tlb *tlb = vcpu->arch.guest_tlb;
for (i = 0; i < KVM_MIPS_GUEST_TLB_SIZE; i++) {
MIPS: KVM: Reformat code and comments No logic changes inside. Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Reviewed-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-06-26 15:11:34 -04:00
if (TLB_HI_VPN2_HIT(tlb[i], entryhi) &&
TLB_HI_ASID_HIT(tlb[i], entryhi)) {
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
index = i;
break;
}
}
kvm_debug("%s: entryhi: %#lx, index: %d lo0: %#lx, lo1: %#lx\n",
MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo* The values of the EntryLo0 and EntryLo1 registers for a TLB entry are stored in separate members of struct kvm_mips_tlb called tlb_lo0 and tlb_lo1 respectively. To allow future code which needs to manipulate arbitrary EntryLo data in the TLB entry to be simpler and less conditional, replace these members with an array of two elements. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:17 -04:00
__func__, entryhi, index, tlb[i].tlb_lo[0], tlb[i].tlb_lo[1]);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
return index;
}
MIPS: KVM: Convert EXPORT_SYMBOL to _GPL Export symbols only to GPL modules to match other KVM symbols in virt/kvm/ and arch/*/kvm/. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Gleb Natapov <gleb@kernel.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11887/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-12-16 18:49:30 -05:00
EXPORT_SYMBOL_GPL(kvm_mips_guest_tlb_lookup);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID Refactor kvm_mips_host_tlb_inv() to also be able to invalidate any matching TLB entry in the kernel ASID rather than assuming only the TLB entries in the user ASID can change. Two new bool user/kernel arguments allow the caller to indicate whether the mapping should affect each of the ASIDs for guest user/kernel mode. - kvm_mips_invalidate_guest_tlb() (used by TLBWI/TLBWR emulation) can now invalidate any corresponding TLB entry in both the kernel ASID (guest kernel may have accessed any guest mapping), and the user ASID if the entry being replaced is in guest USeg (where guest user may also have accessed it). - The tlbmod fault handler (and the KSeg0 / TLB mapped / commpage fault handlers in later patches) can now invalidate the corresponding TLB entry in whichever ASID is currently active, since only a single page table will have been updated anyway. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 19:15:52 -04:00
static int _kvm_mips_host_tlb_inv(unsigned long entryhi)
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
{
int idx;
KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID Refactor kvm_mips_host_tlb_inv() to also be able to invalidate any matching TLB entry in the kernel ASID rather than assuming only the TLB entries in the user ASID can change. Two new bool user/kernel arguments allow the caller to indicate whether the mapping should affect each of the ASIDs for guest user/kernel mode. - kvm_mips_invalidate_guest_tlb() (used by TLBWI/TLBWR emulation) can now invalidate any corresponding TLB entry in both the kernel ASID (guest kernel may have accessed any guest mapping), and the user ASID if the entry being replaced is in guest USeg (where guest user may also have accessed it). - The tlbmod fault handler (and the KSeg0 / TLB mapped / commpage fault handlers in later patches) can now invalidate the corresponding TLB entry in whichever ASID is currently active, since only a single page table will have been updated anyway. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 19:15:52 -04:00
write_c0_entryhi(entryhi);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
mtc0_tlbw_hazard();
tlb_probe();
tlb_probe_hazard();
idx = read_c0_index();
if (idx >= current_cpu_data.tlbsize)
BUG();
KVM: MIPS/TLB: Fix off-by-one in TLB invalidate kvm_mips_host_tlb_inv() uses the TLBP instruction to probe the host TLB for an entry matching the given guest virtual address, and determines whether a match was found based on whether CP0_Index > 0. This is technically incorrect as an index of 0 (with the high bit clear) is a perfectly valid TLB index. This is harmless at the moment due to the use of at least 1 wired TLB entry for the KVM commpage, however we will soon be ridding ourselves of that particular wired entry so lets fix the condition in case the entry needing invalidation does land at TLB index 0. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 17:01:05 -04:00
if (idx >= 0) {
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
write_c0_entryhi(UNIQUE_ENTRYHI(idx));
write_c0_entrylo0(0);
write_c0_entrylo1(0);
mtc0_tlbw_hazard();
tlb_write_indexed();
MIPS: KVM: Clean up TLB management hazards KVM's host TLB handling routines were using tlbw hazard barrier macros around tlb_read(). Now that hazard barrier macros exist for tlbr, update this case to use them. Also fix various other unnecessary hazard barriers in this code. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-09 09:19:15 -04:00
tlbw_use_hazard();
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
}
KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID Refactor kvm_mips_host_tlb_inv() to also be able to invalidate any matching TLB entry in the kernel ASID rather than assuming only the TLB entries in the user ASID can change. Two new bool user/kernel arguments allow the caller to indicate whether the mapping should affect each of the ASIDs for guest user/kernel mode. - kvm_mips_invalidate_guest_tlb() (used by TLBWI/TLBWR emulation) can now invalidate any corresponding TLB entry in both the kernel ASID (guest kernel may have accessed any guest mapping), and the user ASID if the entry being replaced is in guest USeg (where guest user may also have accessed it). - The tlbmod fault handler (and the KSeg0 / TLB mapped / commpage fault handlers in later patches) can now invalidate the corresponding TLB entry in whichever ASID is currently active, since only a single page table will have been updated anyway. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 19:15:52 -04:00
return idx;
}
int kvm_mips_host_tlb_inv(struct kvm_vcpu *vcpu, unsigned long va,
bool user, bool kernel)
{
KVM: MIPS: Fix maybe-uninitialized build failure This commit fixes a "maybe-uninitialized" build failure in arch/mips/kvm/tlb.c when KVM, DYNAMIC_DEBUG and JUMP_LABEL are all enabled. The failure is: In file included from ./include/linux/printk.h:329:0, from ./include/linux/kernel.h:13, from ./include/asm-generic/bug.h:15, from ./arch/mips/include/asm/bug.h:41, from ./include/linux/bug.h:4, from ./include/linux/thread_info.h:11, from ./include/asm-generic/current.h:4, from ./arch/mips/include/generated/asm/current.h:1, from ./include/linux/sched.h:11, from arch/mips/kvm/tlb.c:13: arch/mips/kvm/tlb.c: In function ‘kvm_mips_host_tlb_inv’: ./include/linux/dynamic_debug.h:126:3: error: ‘idx_kernel’ may be used uninitialized in this function [-Werror=maybe-uninitialized] __dynamic_pr_debug(&descriptor, pr_fmt(fmt), \ ^~~~~~~~~~~~~~~~~~ arch/mips/kvm/tlb.c:169:16: note: ‘idx_kernel’ was declared here int idx_user, idx_kernel; ^~~~~~~~~~ There is a similar error relating to "idx_user". Both errors were observed with GCC 6. As far as I can tell, it is impossible for either idx_user or idx_kernel to be uninitialized when they are later read in the calls to kvm_debug, but to satisfy the compiler, add zero initializers to both variables. Signed-off-by: James Cowgill <James.Cowgill@imgtec.com> Fixes: 57e3869cfaae ("KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID") Cc: <stable@vger.kernel.org> # 4.11+ Acked-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2017-06-20 05:57:51 -04:00
/*
* Initialize idx_user and idx_kernel to workaround bogus
* maybe-initialized warning when using GCC 6.
*/
int idx_user = 0, idx_kernel = 0;
KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID Refactor kvm_mips_host_tlb_inv() to also be able to invalidate any matching TLB entry in the kernel ASID rather than assuming only the TLB entries in the user ASID can change. Two new bool user/kernel arguments allow the caller to indicate whether the mapping should affect each of the ASIDs for guest user/kernel mode. - kvm_mips_invalidate_guest_tlb() (used by TLBWI/TLBWR emulation) can now invalidate any corresponding TLB entry in both the kernel ASID (guest kernel may have accessed any guest mapping), and the user ASID if the entry being replaced is in guest USeg (where guest user may also have accessed it). - The tlbmod fault handler (and the KSeg0 / TLB mapped / commpage fault handlers in later patches) can now invalidate the corresponding TLB entry in whichever ASID is currently active, since only a single page table will have been updated anyway. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 19:15:52 -04:00
unsigned long flags, old_entryhi;
local_irq_save(flags);
old_entryhi = read_c0_entryhi();
if (user)
idx_user = _kvm_mips_host_tlb_inv((va & VPN2_MASK) |
kvm_mips_get_user_asid(vcpu));
if (kernel)
idx_kernel = _kvm_mips_host_tlb_inv((va & VPN2_MASK) |
kvm_mips_get_kernel_asid(vcpu));
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
write_c0_entryhi(old_entryhi);
mtc0_tlbw_hazard();
local_irq_restore(flags);
KVM: MIPS/TLB: Handle virtually tagged icaches When TLB entries are invalidated in the presence of a virtually tagged icache, such as that found on Octeon CPUs, flush the icache so that we don't get a reserved instruction exception even though the TLB mapping is removed. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: David Daney <david.daney@cavium.com> Cc: Andreas Herrmann <andreas.herrmann@caviumnetworks.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:25:46 -04:00
/*
* We don't want to get reserved instruction exceptions for missing tlb
* entries.
*/
if (cpu_has_vtag_icache)
flush_icache_all();
KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID Refactor kvm_mips_host_tlb_inv() to also be able to invalidate any matching TLB entry in the kernel ASID rather than assuming only the TLB entries in the user ASID can change. Two new bool user/kernel arguments allow the caller to indicate whether the mapping should affect each of the ASIDs for guest user/kernel mode. - kvm_mips_invalidate_guest_tlb() (used by TLBWI/TLBWR emulation) can now invalidate any corresponding TLB entry in both the kernel ASID (guest kernel may have accessed any guest mapping), and the user ASID if the entry being replaced is in guest USeg (where guest user may also have accessed it). - The tlbmod fault handler (and the KSeg0 / TLB mapped / commpage fault handlers in later patches) can now invalidate the corresponding TLB entry in whichever ASID is currently active, since only a single page table will have been updated anyway. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-07 19:15:52 -04:00
if (user && idx_user >= 0)
kvm_debug("%s: Invalidated guest user entryhi %#lx @ idx %d\n",
__func__, (va & VPN2_MASK) |
kvm_mips_get_user_asid(vcpu), idx_user);
if (kernel && idx_kernel >= 0)
kvm_debug("%s: Invalidated guest kernel entryhi %#lx @ idx %d\n",
__func__, (va & VPN2_MASK) |
kvm_mips_get_kernel_asid(vcpu), idx_kernel);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
return 0;
}
MIPS: KVM: Convert EXPORT_SYMBOL to _GPL Export symbols only to GPL modules to match other KVM symbols in virt/kvm/ and arch/*/kvm/. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Gleb Natapov <gleb@kernel.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11887/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-12-16 18:49:30 -05:00
EXPORT_SYMBOL_GPL(kvm_mips_host_tlb_inv);
KVM/MIPS32: MMU/TLB operations for the Guest. - Note that this file is statically linked with the rest of the host kernel (KSEG0). This is because kernel modules are loaded into mapped space on MIPS and we want to make sure that we don't get any host kernel TLB faults while manipulating TLBs. - Virtual Guest TLBs are implemented as 64 entry array regardless of the number of host TLB entries. - Shadow TLBs map Guest virtual addresses to Host physical addresses. - TLB miss handling details: Guest KSEG0 TLBMISS (0x40000000 – 0x60000000): Transparent to the Guest. Guest KSEG2/3 (0x60000000 – 0x80000000) & Guest UM TLBMISS (0x00000000 – 0x40000000) Lookup in Guest/Virtual TLB If an entry doesn’t match deliver appropriate TLBMISS LD/ST exception to the guest If entry does exist in the Guest TLB and is NOT Valid Deliver TLB invalid exception to the guest If entry does exist in the Guest TLB and is VALID Inject the TLB entry into the Shadow TLB Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-21 21:34:05 -05:00
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
#ifdef CONFIG_KVM_MIPS_VZ
/* GuestID management */
/**
* clear_root_gid() - Set GuestCtl1.RID for normal root operation.
*/
static inline void clear_root_gid(void)
{
if (cpu_has_guestid) {
clear_c0_guestctl1(MIPS_GCTL1_RID);
mtc0_tlbw_hazard();
}
}
/**
* set_root_gid_to_guest_gid() - Set GuestCtl1.RID to match GuestCtl1.ID.
*
* Sets the root GuestID to match the current guest GuestID, for TLB operation
* on the GPA->RPA mappings in the root TLB.
*
* The caller must be sure to disable HTW while the root GID is set, and
* possibly longer if TLB registers are modified.
*/
static inline void set_root_gid_to_guest_gid(void)
{
unsigned int guestctl1;
if (cpu_has_guestid) {
back_to_back_c0_hazard();
guestctl1 = read_c0_guestctl1();
guestctl1 = (guestctl1 & ~MIPS_GCTL1_RID) |
((guestctl1 & MIPS_GCTL1_ID) >> MIPS_GCTL1_ID_SHIFT)
<< MIPS_GCTL1_RID_SHIFT;
write_c0_guestctl1(guestctl1);
mtc0_tlbw_hazard();
}
}
int kvm_vz_host_tlb_inv(struct kvm_vcpu *vcpu, unsigned long va)
{
int idx;
unsigned long flags, old_entryhi;
local_irq_save(flags);
htw_stop();
/* Set root GuestID for root probe and write of guest TLB entry */
set_root_gid_to_guest_gid();
old_entryhi = read_c0_entryhi();
idx = _kvm_mips_host_tlb_inv((va & VPN2_MASK) |
kvm_mips_get_root_asid(vcpu));
write_c0_entryhi(old_entryhi);
clear_root_gid();
mtc0_tlbw_hazard();
htw_start();
local_irq_restore(flags);
KVM: MIPS/TLB: Handle virtually tagged icaches When TLB entries are invalidated in the presence of a virtually tagged icache, such as that found on Octeon CPUs, flush the icache so that we don't get a reserved instruction exception even though the TLB mapping is removed. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: David Daney <david.daney@cavium.com> Cc: Andreas Herrmann <andreas.herrmann@caviumnetworks.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:25:46 -04:00
/*
* We don't want to get reserved instruction exceptions for missing tlb
* entries.
*/
if (cpu_has_vtag_icache)
flush_icache_all();
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
if (idx > 0)
kvm_debug("%s: Invalidated root entryhi %#lx @ idx %d\n",
__func__, (va & VPN2_MASK) |
kvm_mips_get_root_asid(vcpu), idx);
return 0;
}
EXPORT_SYMBOL_GPL(kvm_vz_host_tlb_inv);
/**
* kvm_vz_guest_tlb_lookup() - Lookup a guest VZ TLB mapping.
* @vcpu: KVM VCPU pointer.
* @gpa: Guest virtual address in a TLB mapped guest segment.
* @gpa: Ponter to output guest physical address it maps to.
*
* Converts a guest virtual address in a guest TLB mapped segment to a guest
* physical address, by probing the guest TLB.
*
* Returns: 0 if guest TLB mapping exists for @gva. *@gpa will have been
* written.
* -EFAULT if no guest TLB mapping exists for @gva. *@gpa may not
* have been written.
*/
int kvm_vz_guest_tlb_lookup(struct kvm_vcpu *vcpu, unsigned long gva,
unsigned long *gpa)
{
unsigned long o_entryhi, o_entrylo[2], o_pagemask;
unsigned int o_index;
unsigned long entrylo[2], pagemask, pagemaskbit, pa;
unsigned long flags;
int index;
/* Probe the guest TLB for a mapping */
local_irq_save(flags);
/* Set root GuestID for root probe of guest TLB entry */
htw_stop();
set_root_gid_to_guest_gid();
o_entryhi = read_gc0_entryhi();
o_index = read_gc0_index();
write_gc0_entryhi((o_entryhi & 0x3ff) | (gva & ~0xfffl));
mtc0_tlbw_hazard();
guest_tlb_probe();
tlb_probe_hazard();
index = read_gc0_index();
if (index < 0) {
/* No match, fail */
write_gc0_entryhi(o_entryhi);
write_gc0_index(o_index);
clear_root_gid();
htw_start();
local_irq_restore(flags);
return -EFAULT;
}
/* Match! read the TLB entry */
o_entrylo[0] = read_gc0_entrylo0();
o_entrylo[1] = read_gc0_entrylo1();
o_pagemask = read_gc0_pagemask();
mtc0_tlbr_hazard();
guest_tlb_read();
tlb_read_hazard();
entrylo[0] = read_gc0_entrylo0();
entrylo[1] = read_gc0_entrylo1();
pagemask = ~read_gc0_pagemask() & ~0x1fffl;
write_gc0_entryhi(o_entryhi);
write_gc0_index(o_index);
write_gc0_entrylo0(o_entrylo[0]);
write_gc0_entrylo1(o_entrylo[1]);
write_gc0_pagemask(o_pagemask);
clear_root_gid();
htw_start();
local_irq_restore(flags);
/* Select one of the EntryLo values and interpret the GPA */
pagemaskbit = (pagemask ^ (pagemask & (pagemask - 1))) >> 1;
pa = entrylo[!!(gva & pagemaskbit)];
/*
* TLB entry may have become invalid since TLB probe if physical FTLB
* entries are shared between threads (e.g. I6400).
*/
if (!(pa & ENTRYLO_V))
return -EFAULT;
/*
* Note, this doesn't take guest MIPS32 XPA into account, where PFN is
* split with XI/RI in the middle.
*/
pa = (pa << 6) & ~0xfffl;
pa |= gva & ~(pagemask | pagemaskbit);
*gpa = pa;
return 0;
}
EXPORT_SYMBOL_GPL(kvm_vz_guest_tlb_lookup);
/**
* kvm_vz_local_flush_roottlb_all_guests() - Flush all root TLB entries for
* guests.
*
* Invalidate all entries in root tlb which are GPA mappings.
*/
void kvm_vz_local_flush_roottlb_all_guests(void)
{
unsigned long flags;
unsigned long old_entryhi, old_pagemask, old_guestctl1;
int entry;
if (WARN_ON(!cpu_has_guestid))
return;
local_irq_save(flags);
htw_stop();
/* TLBR may clobber EntryHi.ASID, PageMask, and GuestCtl1.RID */
old_entryhi = read_c0_entryhi();
old_pagemask = read_c0_pagemask();
old_guestctl1 = read_c0_guestctl1();
/*
* Invalidate guest entries in root TLB while leaving root entries
* intact when possible.
*/
for (entry = 0; entry < current_cpu_data.tlbsize; entry++) {
write_c0_index(entry);
mtc0_tlbw_hazard();
tlb_read();
tlb_read_hazard();
/* Don't invalidate non-guest (RVA) mappings in the root TLB */
if (!(read_c0_guestctl1() & MIPS_GCTL1_RID))
continue;
/* Make sure all entries differ. */
write_c0_entryhi(UNIQUE_ENTRYHI(entry));
write_c0_entrylo0(0);
write_c0_entrylo1(0);
write_c0_guestctl1(0);
mtc0_tlbw_hazard();
tlb_write_indexed();
}
write_c0_entryhi(old_entryhi);
write_c0_pagemask(old_pagemask);
write_c0_guestctl1(old_guestctl1);
tlbw_use_hazard();
htw_start();
local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(kvm_vz_local_flush_roottlb_all_guests);
/**
* kvm_vz_local_flush_guesttlb_all() - Flush all guest TLB entries.
*
* Invalidate all entries in guest tlb irrespective of guestid.
*/
void kvm_vz_local_flush_guesttlb_all(void)
{
unsigned long flags;
unsigned long old_index;
unsigned long old_entryhi;
unsigned long old_entrylo[2];
unsigned long old_pagemask;
int entry;
KVM: MIPS/VZ: VZ hardware setup for Octeon III Set up hardware virtualisation on Octeon III cores, configuring guest interrupt routing and carving out half of the root TLB for guest use, restoring it back again afterwards. We need to be careful to inhibit TLB shutdown machine check exceptions while invalidating guest TLB entries, since TLB invalidation is not available so guest entries must be invalidated by setting them to unique unmapped addresses, which could conflict with mappings set by the guest or root if recently repartitioned. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: David Daney <david.daney@cavium.com> Cc: Andreas Herrmann <andreas.herrmann@caviumnetworks.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:25:48 -04:00
u64 cvmmemctl2 = 0;
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
local_irq_save(flags);
/* Preserve all clobbered guest registers */
old_index = read_gc0_index();
old_entryhi = read_gc0_entryhi();
old_entrylo[0] = read_gc0_entrylo0();
old_entrylo[1] = read_gc0_entrylo1();
old_pagemask = read_gc0_pagemask();
KVM: MIPS/VZ: VZ hardware setup for Octeon III Set up hardware virtualisation on Octeon III cores, configuring guest interrupt routing and carving out half of the root TLB for guest use, restoring it back again afterwards. We need to be careful to inhibit TLB shutdown machine check exceptions while invalidating guest TLB entries, since TLB invalidation is not available so guest entries must be invalidated by setting them to unique unmapped addresses, which could conflict with mappings set by the guest or root if recently repartitioned. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: David Daney <david.daney@cavium.com> Cc: Andreas Herrmann <andreas.herrmann@caviumnetworks.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:25:48 -04:00
switch (current_cpu_type()) {
case CPU_CAVIUM_OCTEON3:
/* Inhibit machine check due to multiple matching TLB entries */
cvmmemctl2 = read_c0_cvmmemctl2();
cvmmemctl2 |= CVMMEMCTL2_INHIBITTS;
write_c0_cvmmemctl2(cvmmemctl2);
break;
};
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
/* Invalidate guest entries in guest TLB */
write_gc0_entrylo0(0);
write_gc0_entrylo1(0);
write_gc0_pagemask(0);
for (entry = 0; entry < current_cpu_data.guest.tlbsize; entry++) {
/* Make sure all entries differ. */
write_gc0_index(entry);
write_gc0_entryhi(UNIQUE_GUEST_ENTRYHI(entry));
mtc0_tlbw_hazard();
guest_tlb_write_indexed();
}
KVM: MIPS/VZ: VZ hardware setup for Octeon III Set up hardware virtualisation on Octeon III cores, configuring guest interrupt routing and carving out half of the root TLB for guest use, restoring it back again afterwards. We need to be careful to inhibit TLB shutdown machine check exceptions while invalidating guest TLB entries, since TLB invalidation is not available so guest entries must be invalidated by setting them to unique unmapped addresses, which could conflict with mappings set by the guest or root if recently repartitioned. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: David Daney <david.daney@cavium.com> Cc: Andreas Herrmann <andreas.herrmann@caviumnetworks.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:25:48 -04:00
if (cvmmemctl2) {
cvmmemctl2 &= ~CVMMEMCTL2_INHIBITTS;
write_c0_cvmmemctl2(cvmmemctl2);
};
KVM: MIPS/TLB: Add VZ TLB management Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by: James Hogan <james.hogan@imgtec.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2017-03-14 06:15:27 -04:00
write_gc0_index(old_index);
write_gc0_entryhi(old_entryhi);
write_gc0_entrylo0(old_entrylo[0]);
write_gc0_entrylo1(old_entrylo[1]);
write_gc0_pagemask(old_pagemask);
tlbw_use_hazard();
local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(kvm_vz_local_flush_guesttlb_all);
/**
* kvm_vz_save_guesttlb() - Save a range of guest TLB entries.
* @buf: Buffer to write TLB entries into.
* @index: Start index.
* @count: Number of entries to save.
*
* Save a range of guest TLB entries. The caller must ensure interrupts are
* disabled.
*/
void kvm_vz_save_guesttlb(struct kvm_mips_tlb *buf, unsigned int index,
unsigned int count)
{
unsigned int end = index + count;
unsigned long old_entryhi, old_entrylo0, old_entrylo1, old_pagemask;
unsigned int guestctl1 = 0;
int old_index, i;
/* Save registers we're about to clobber */
old_index = read_gc0_index();
old_entryhi = read_gc0_entryhi();
old_entrylo0 = read_gc0_entrylo0();
old_entrylo1 = read_gc0_entrylo1();
old_pagemask = read_gc0_pagemask();
/* Set root GuestID for root probe */
htw_stop();
set_root_gid_to_guest_gid();
if (cpu_has_guestid)
guestctl1 = read_c0_guestctl1();
/* Read each entry from guest TLB */
for (i = index; i < end; ++i, ++buf) {
write_gc0_index(i);
mtc0_tlbr_hazard();
guest_tlb_read();
tlb_read_hazard();
if (cpu_has_guestid &&
(read_c0_guestctl1() ^ guestctl1) & MIPS_GCTL1_RID) {
/* Entry invalid or belongs to another guest */
buf->tlb_hi = UNIQUE_GUEST_ENTRYHI(i);
buf->tlb_lo[0] = 0;
buf->tlb_lo[1] = 0;
buf->tlb_mask = 0;
} else {
/* Entry belongs to the right guest */
buf->tlb_hi = read_gc0_entryhi();
buf->tlb_lo[0] = read_gc0_entrylo0();
buf->tlb_lo[1] = read_gc0_entrylo1();
buf->tlb_mask = read_gc0_pagemask();
}
}
/* Clear root GuestID again */
clear_root_gid();
htw_start();
/* Restore clobbered registers */
write_gc0_index(old_index);
write_gc0_entryhi(old_entryhi);
write_gc0_entrylo0(old_entrylo0);
write_gc0_entrylo1(old_entrylo1);
write_gc0_pagemask(old_pagemask);
tlbw_use_hazard();
}
EXPORT_SYMBOL_GPL(kvm_vz_save_guesttlb);
/**
* kvm_vz_load_guesttlb() - Save a range of guest TLB entries.
* @buf: Buffer to read TLB entries from.
* @index: Start index.
* @count: Number of entries to load.
*
* Load a range of guest TLB entries. The caller must ensure interrupts are
* disabled.
*/
void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
unsigned int count)
{
unsigned int end = index + count;
unsigned long old_entryhi, old_entrylo0, old_entrylo1, old_pagemask;
int old_index, i;
/* Save registers we're about to clobber */
old_index = read_gc0_index();
old_entryhi = read_gc0_entryhi();
old_entrylo0 = read_gc0_entrylo0();
old_entrylo1 = read_gc0_entrylo1();
old_pagemask = read_gc0_pagemask();
/* Set root GuestID for root probe */
htw_stop();
set_root_gid_to_guest_gid();
/* Write each entry to guest TLB */
for (i = index; i < end; ++i, ++buf) {
write_gc0_index(i);
write_gc0_entryhi(buf->tlb_hi);
write_gc0_entrylo0(buf->tlb_lo[0]);
write_gc0_entrylo1(buf->tlb_lo[1]);
write_gc0_pagemask(buf->tlb_mask);
mtc0_tlbw_hazard();
guest_tlb_write_indexed();
}
/* Clear root GuestID again */
clear_root_gid();
htw_start();
/* Restore clobbered registers */
write_gc0_index(old_index);
write_gc0_entryhi(old_entryhi);
write_gc0_entrylo0(old_entrylo0);
write_gc0_entrylo1(old_entrylo1);
write_gc0_pagemask(old_pagemask);
tlbw_use_hazard();
}
EXPORT_SYMBOL_GPL(kvm_vz_load_guesttlb);
#endif
KVM: MIPS/T&E: active_mm = init_mm in guest context Set init_mm as the active_mm and update mm_cpumask(current->mm) to reflect that it isn't active when in guest context. This prevents cache management code from attempting cache flushes on host virtual addresses while in guest context, for example due to a cache management IPIs or later when writing of dynamically translated code hits copy on write. We do this using helpers in static kernel code to avoid having to export init_mm to modules. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-11-14 19:06:05 -05:00
/**
* kvm_mips_suspend_mm() - Suspend the active mm.
* @cpu The CPU we're running on.
*
* Suspend the active_mm, ready for a switch to a KVM guest virtual address
* space. This is left active for the duration of guest context, including time
* with interrupts enabled, so we need to be careful not to confuse e.g. cache
* management IPIs.
*
* kvm_mips_resume_mm() should be called before context switching to a different
* process so we don't need to worry about reference counting.
*
* This needs to be in static kernel code to avoid exporting init_mm.
*/
void kvm_mips_suspend_mm(int cpu)
{
cpumask_clear_cpu(cpu, mm_cpumask(current->active_mm));
current->active_mm = &init_mm;
}
EXPORT_SYMBOL_GPL(kvm_mips_suspend_mm);
/**
* kvm_mips_resume_mm() - Resume the current process mm.
* @cpu The CPU we're running on.
*
* Resume the mm of the current process, after a switch back from a KVM guest
* virtual address space (see kvm_mips_suspend_mm()).
*/
void kvm_mips_resume_mm(int cpu)
{
cpumask_set_cpu(cpu, mm_cpumask(current->mm));
current->active_mm = current->mm;
}
EXPORT_SYMBOL_GPL(kvm_mips_resume_mm);
Reference in New Issue Copy Permalink