AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2025-01-19 22:46:15 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity
7faa6eec69
linux_dsm_epyc7002/arch/mips/include/asm/mmu_context.h

228 lines
5.6 KiB
C
Raw Normal View History

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/*
* Switch a MMU context.
*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* Copyright (C) 1996, 1997, 1998, 1999 by Ralf Baechle
* Copyright (C) 1999 Silicon Graphics, Inc.
*/
#ifndef _ASM_MMU_CONTEXT_H
#define _ASM_MMU_CONTEXT_H
#include <linux/errno.h>
#include <linux/sched.h>
MIPS: Build fix - include <linux/smp.h> into all smp_processor_id() users. Some of the were relying into smp.h being dragged in by another header which of course is fragile. <asm/cpu-info.h> uses smp_processor_id() only in macros and including smp.h there leads to an include loop, so don't change cpu-info.h. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-06-19 20:05:26 +07:00
#include <linux/smp.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#include <linux/slab.h>
#include <asm/cacheflush.h>
MIPS: Use per-mm page to execute branch delay slot instructions In some cases the kernel needs to execute an instruction from the delay slot of an emulated branch instruction. These cases include: - Emulated floating point branch instructions (bc1[ft]l?) for systems which don't include an FPU, or upon which the kernel is run with the "nofpu" parameter. - MIPSr6 systems running binaries targeting older revisions of the architecture, which may include branch instructions whose encodings are no longer valid in MIPSr6. Executing instructions from such delay slots is done by writing the instruction to memory followed by a trap, as part of an "emuframe", and executing it. This avoids the requirement of an emulator for the entire MIPS instruction set. Prior to this patch such emuframes are written to the user stack and executed from there. This patch moves FP branch delay emuframes off of the user stack and into a per-mm page. Allocating a page per-mm leaves userland with access to only what it had access to previously, and compared to other solutions is relatively simple. When a thread requires a delay slot emulation, it is allocated a frame. A thread may only have one frame allocated at any one time, since it may only ever be executing one instruction at any one time. In order to ensure that we can free up allocated frame later, its index is recorded in struct thread_struct. In the typical case, after executing the delay slot instruction we'll execute a break instruction with the BRK_MEMU code. This traps back to the kernel & leads to a call to do_dsemulret which frees the allocated frame & moves the user PC back to the instruction that would have executed following the emulated branch. In some cases the delay slot instruction may be invalid, such as a branch, or may trigger an exception. In these cases the BRK_MEMU break instruction will not be hit. In order to ensure that frames are freed this patch introduces dsemul_thread_cleanup() and calls it to free any allocated frame upon thread exit. If the instruction generated an exception & leads to a signal being delivered to the thread, or indeed if a signal simply happens to be delivered to the thread whilst it is executing from the struct emuframe, then we need to take care to exit the frame appropriately. This is done by either rolling back the user PC to the branch or advancing it to the continuation PC prior to signal delivery, using dsemul_thread_rollback(). If this were not done then a sigreturn would return to the struct emuframe, and if that frame had meanwhile been used in response to an emulated branch instruction within the signal handler then we would execute the wrong user code. Whilst a user could theoretically place something like a compact branch to self in a delay slot and cause their thread to become stuck in an infinite loop with the frame never being deallocated, this would: - Only affect the users single process. - Be architecturally invalid since there would be a branch in the delay slot, which is forbidden. - Be extremely unlikely to happen by mistake, and provide a program with no more ability to harm the system than a simple infinite loop would. If a thread requires a delay slot emulation & no frame is available to it (ie. the process has enough other threads that all frames are currently in use) then the thread joins a waitqueue. It will sleep until a frame is freed by another thread in the process. Since we now know whether a thread has an allocated frame due to our tracking of its index, the cookie field of struct emuframe is removed as we can be more certain whether we have a valid frame. Since a thread may only ever have a single frame at any given time, the epc field of struct emuframe is also removed & the PC to continue from is instead stored in struct thread_struct. Together these changes simplify & shrink struct emuframe somewhat, allowing twice as many frames to fit into the page allocated for them. The primary benefit of this patch is that we are now free to mark the user stack non-executable where that is possible. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: Maciej Rozycki <maciej.rozycki@imgtec.com> Cc: Faraz Shahbazker <faraz.shahbazker@imgtec.com> Cc: Raghu Gandham <raghu.gandham@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13764/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-07-08 17:06:19 +07:00
#include <asm/dsemul.h>
MIPS: Avoid potential hazard on Context register set_saved_sp reads Context register. Avoid reading stale value from earlier incomplete write. Issue found and fixed for head.S by Chris Dearman <chris@mips.com>. Signed-off-by: Chris Dearman <chris@mips.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-14 04:23:28 +07:00
#include <asm/hazards.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#include <asm/tlbflush.h>
[PATCH] x86: PARAVIRT: add hooks to intercept mm creation and destruction Add hooks to allow a paravirt implementation to track the lifetime of an mm. Paravirtualization requires three hooks, but only two are needed in common code. They are: arch_dup_mmap, which is called when a new mmap is created at fork arch_exit_mmap, which is called when the last process reference to an mm is dropped, which typically happens on exit and exec. The third hook is activate_mm, which is called from the arch-specific activate_mm() macro/function, and so doesn't need stub versions for other architectures. It's called when an mm is first used. Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com> Signed-off-by: Andi Kleen <ak@suse.de> Cc: linux-arch@vger.kernel.org Cc: James Bottomley <James.Bottomley@SteelEye.com> Acked-by: Ingo Molnar <mingo@elte.hu>
2007-05-03 00:27:14 +07:00
#include <asm-generic/mm_hooks.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: mm: Use the Hardware Page Table Walker if the core supports it The Hardware Page Table Walker aims to speed up TLB refill exceptions by handling them in the hardware level instead of having a software TLB refill handler. However, a TLB refill exception can still be thrown in certain cases such as, synchronus exceptions, or address translation or memory errors during the HTW operation. As a result of which, HTW must not be considered a complete replacement for the TLB refill software handler, but rather a fast-path for it. For HTW to work, the PWBase register must contain the task's page global directory address so the HTW will kick in on TLB refill exceptions. Due to HTW being a separate engine embedded deep in the CPU pipeline, we need to restart the HTW everytime a PTE changes to avoid HTW fetching a old entry from the page tables. It's also necessary to restart the HTW on context switches to prevent it from fetching a page from the previous process. Finally, since HTW is using the entryhi register to write the translations to the TLB, it's necessary to stop the HTW whenever the entryhi changes (eg for tlb probe perations) and enable it back afterwards. == Performance == The following trivial test was used to measure the performance of the HTW. Using the same root filesystem, the following command was used to measure the number of tlb refill handler executions with and without (using 'nohtw' kernel parameter) HTW support. The kernel was modified to use a scratch register as a counter for the TLB refill exceptions. find /usr -type f -exec ls -lh {} \; HTW Enabled: TLB refill exceptions: 12306 HTW Disabled: TLB refill exceptions: 17805 Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/7336/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-07-14 18:47:09 +07:00
#define htw_set_pwbase(pgd) \
do { \
if (cpu_has_htw) { \
write_c0_pwbase(pgd); \
back_to_back_c0_hazard(); \
} \
} while (0)
KVM: MIPS/T&E: Activate GVA page tables in guest context Activate the GVA page tables when in guest context. This will allow the normal Linux TLB refill handler to fill from it when guest memory is read, as well as preventing accidental reading from user memory. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
2016-10-08 05:58:53 +07:00
extern void tlbmiss_handler_setup_pgd(unsigned long);
/* Note: This is also implemented with uasm in arch/mips/kvm/entry.c */
MIPS: Make declarations and definitions of tlbmiss_handler_setup_pgd match. tlbmiss_handler_setup_pgd is run-time generated code and it was convenient to pretend the symbol was an array in the generator but a function for the users. LTO gcc won't tolerate this kind of lie anymore so solve the problem through a cast and function pointer instead. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-03-21 17:28:10 +07:00
#define TLBMISS_HANDLER_SETUP_PGD(pgd) \
do { \
tlbmiss_handler_setup_pgd((unsigned long)(pgd)); \
MIPS: mm: Use the Hardware Page Table Walker if the core supports it The Hardware Page Table Walker aims to speed up TLB refill exceptions by handling them in the hardware level instead of having a software TLB refill handler. However, a TLB refill exception can still be thrown in certain cases such as, synchronus exceptions, or address translation or memory errors during the HTW operation. As a result of which, HTW must not be considered a complete replacement for the TLB refill software handler, but rather a fast-path for it. For HTW to work, the PWBase register must contain the task's page global directory address so the HTW will kick in on TLB refill exceptions. Due to HTW being a separate engine embedded deep in the CPU pipeline, we need to restart the HTW everytime a PTE changes to avoid HTW fetching a old entry from the page tables. It's also necessary to restart the HTW on context switches to prevent it from fetching a page from the previous process. Finally, since HTW is using the entryhi register to write the translations to the TLB, it's necessary to stop the HTW whenever the entryhi changes (eg for tlb probe perations) and enable it back afterwards. == Performance == The following trivial test was used to measure the performance of the HTW. Using the same root filesystem, the following command was used to measure the number of tlb refill handler executions with and without (using 'nohtw' kernel parameter) HTW support. The kernel was modified to use a scratch register as a counter for the TLB refill exceptions. find /usr -type f -exec ls -lh {} \; HTW Enabled: TLB refill exceptions: 12306 HTW Disabled: TLB refill exceptions: 17805 Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/7336/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-07-14 18:47:09 +07:00
htw_set_pwbase((unsigned long)pgd); \
MIPS: Make declarations and definitions of tlbmiss_handler_setup_pgd match. tlbmiss_handler_setup_pgd is run-time generated code and it was convenient to pretend the symbol was an array in the generator but a function for the users. LTO gcc won't tolerate this kind of lie anymore so solve the problem through a cast and function pointer instead. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-03-21 17:28:10 +07:00
} while (0)
MIPS: Put PGD in C0_CONTEXT for 64-bit R2 processors. Processors that support the mips64r2 ISA can in four instructions convert a shifted PGD pointer stored in the upper bits of c0_context into a usable pointer. By doing this we save a memory load and associated potential cache miss in the TLB exception handlers. Since the upper bits of c0_context were holding the CPU number, we move this to the upper bits of c0_xcontext which doesn't have enough bits to hold the PGD pointer, but has plenty for the CPU number. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-15 02:16:56 +07:00
MIPS: mm: Use scratch for PGD when !CONFIG_MIPS_PGD_C0_CONTEXT Allow usage of scratch register for current pgd even when MIPS_PGD_C0_CONTEXT is not configured. MIPS_PGD_C0_CONTEXT is set for 64r2 platforms to indicate availability of Xcontext for saving cpuid, thus freeing Context to be used for saving PGD. This option was also tied to using a scratch register for storing PGD. This commit will allow usage of scratch register to store the current pgd if one can be allocated for the platform, even when MIPS_PGD_C0_CONTEXT is not set. The cpuid will be kept in the CP0 Context register in this case. The code to store the current pgd for the TLB miss handler is now generated in all cases. When scratch register is available, the PGD is also stored in the scratch register. Signed-off-by: Jayachandran C <jchandra@broadcom.com> Cc: linux-mips@linux-mips.org Cc: Hauke Mehrtens <hauke@hauke-m.de> Patchwork: https://patchwork.linux-mips.org/patch/5906/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-09-25 17:58:04 +07:00
#ifdef CONFIG_MIPS_PGD_C0_CONTEXT
MIPS: traps: Add CPU PM callback for trap configuration Implement a CPU power management callback for restoring trap related CPU configuration after CPU power up from a low power state. The following state is restored: - Status register - HWREna register - Exception vector configuration registers - Context/XContext register Signed-off-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-03-04 17:20:43 +07:00
#define TLBMISS_HANDLER_RESTORE() \
write_c0_xcontext((unsigned long) smp_processor_id() << \
SMP_CPUID_REGSHIFT)
MIPS: Put PGD in C0_CONTEXT for 64-bit R2 processors. Processors that support the mips64r2 ISA can in four instructions convert a shifted PGD pointer stored in the upper bits of c0_context into a usable pointer. By doing this we save a memory load and associated potential cache miss in the TLB exception handlers. Since the upper bits of c0_context were holding the CPU number, we move this to the upper bits of c0_xcontext which doesn't have enough bits to hold the PGD pointer, but has plenty for the CPU number. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-15 02:16:56 +07:00
#define TLBMISS_HANDLER_SETUP() \
do { \
TLBMISS_HANDLER_SETUP_PGD(swapper_pg_dir); \
MIPS: traps: Add CPU PM callback for trap configuration Implement a CPU power management callback for restoring trap related CPU configuration after CPU power up from a low power state. The following state is restored: - Status register - HWREna register - Exception vector configuration registers - Context/XContext register Signed-off-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-03-04 17:20:43 +07:00
TLBMISS_HANDLER_RESTORE(); \
MIPS: Put PGD in C0_CONTEXT for 64-bit R2 processors. Processors that support the mips64r2 ISA can in four instructions convert a shifted PGD pointer stored in the upper bits of c0_context into a usable pointer. By doing this we save a memory load and associated potential cache miss in the TLB exception handlers. Since the upper bits of c0_context were holding the CPU number, we move this to the upper bits of c0_xcontext which doesn't have enough bits to hold the PGD pointer, but has plenty for the CPU number. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-15 02:16:56 +07:00
} while (0)
MIPS: Move definition of SMP processor id register to header file The definition of the CP0 register used to save the smp processor id is repicated in many files, move them all to thread_info.h. Signed-off-by: Jayachandran C <jchandra@broadcom.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/5708/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-08-11 18:40:16 +07:00
#else /* !CONFIG_MIPS_PGD_C0_CONTEXT: using pgd_current*/
MIPS: Put PGD in C0_CONTEXT for 64-bit R2 processors. Processors that support the mips64r2 ISA can in four instructions convert a shifted PGD pointer stored in the upper bits of c0_context into a usable pointer. By doing this we save a memory load and associated potential cache miss in the TLB exception handlers. Since the upper bits of c0_context were holding the CPU number, we move this to the upper bits of c0_xcontext which doesn't have enough bits to hold the PGD pointer, but has plenty for the CPU number. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-15 02:16:56 +07:00
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/*
* For the fast tlb miss handlers, we keep a per cpu array of pointers
* to the current pgd for each processor. Also, the proc. id is stuffed
* into the context register.
*/
extern unsigned long pgd_current[];
MIPS: traps: Add CPU PM callback for trap configuration Implement a CPU power management callback for restoring trap related CPU configuration after CPU power up from a low power state. The following state is restored: - Status register - HWREna register - Exception vector configuration registers - Context/XContext register Signed-off-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-03-04 17:20:43 +07:00
#define TLBMISS_HANDLER_RESTORE() \
MIPS: Move definition of SMP processor id register to header file The definition of the CP0 register used to save the smp processor id is repicated in many files, move them all to thread_info.h. Signed-off-by: Jayachandran C <jchandra@broadcom.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/5708/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-08-11 18:40:16 +07:00
write_c0_context((unsigned long) smp_processor_id() << \
MIPS: traps: Add CPU PM callback for trap configuration Implement a CPU power management callback for restoring trap related CPU configuration after CPU power up from a low power state. The following state is restored: - Status register - HWREna register - Exception vector configuration registers - Context/XContext register Signed-off-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-03-04 17:20:43 +07:00
SMP_CPUID_REGSHIFT)
#define TLBMISS_HANDLER_SETUP() \
TLBMISS_HANDLER_RESTORE(); \
MIPS: Avoid potential hazard on Context register set_saved_sp reads Context register. Avoid reading stale value from earlier incomplete write. Issue found and fixed for head.S by Chris Dearman <chris@mips.com>. Signed-off-by: Chris Dearman <chris@mips.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-14 04:23:28 +07:00
back_to_back_c0_hazard(); \
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
TLBMISS_HANDLER_SETUP_PGD(swapper_pg_dir)
MIPS: Put PGD in C0_CONTEXT for 64-bit R2 processors. Processors that support the mips64r2 ISA can in four instructions convert a shifted PGD pointer stored in the upper bits of c0_context into a usable pointer. By doing this we save a memory load and associated potential cache miss in the TLB exception handlers. Since the upper bits of c0_context were holding the CPU number, we move this to the upper bits of c0_xcontext which doesn't have enough bits to hold the PGD pointer, but has plenty for the CPU number. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-10-15 02:16:56 +07:00
#endif /* CONFIG_MIPS_PGD_C0_CONTEXT*/
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
/*
* All unused by hardware upper bits will be considered
* as a software asid extension.
*/
static unsigned long asid_version_mask(unsigned int cpu)
{
unsigned long asid_mask = cpu_asid_mask(&cpu_data[cpu]);
Revert "MIPS: Allow ASID size to be determined at boot time." This reverts commit d532f3d26716a39dfd4b88d687bd344fbe77e390. The original commit has several problems: 1) Doesn't work with 64-bit kernels. 2) Calls TLBMISS_HANDLER_SETUP() before the code is generated. 3) Calls TLBMISS_HANDLER_SETUP() twice in per_cpu_trap_init() when only one call is needed. [ralf@linux-mips.org: Also revert the bits of the ASID patch which were hidden in the KVM merge.] Signed-off-by: David Daney <david.daney@cavium.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: "Steven J. Hill" <Steven.Hill@imgtec.com> Cc: David Daney <david.daney@cavium.com> Patchwork: https://patchwork.linux-mips.org/patch/5242/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-05-14 03:56:44 +07:00
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
return ~(asid_mask | (asid_mask - 1));
}
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
static unsigned long asid_first_version(unsigned int cpu)
{
return ~asid_version_mask(cpu) + 1;
}
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: Preliminary VDSO This is a preliminary patch to add a vdso to all user processes. Still missing are ELF headers and .eh_frame information. But it is enough to allow us to move signal trampolines off of the stack. Note that emulation of branch delay slots in the FPU emulator still requires the stack. We allocate a single page (the vdso) and write all possible signal trampolines into it. The stack is moved down by one page and the vdso is mapped into this space. Signed-off-by: David Daney <ddaney@caviumnetworks.com> To: linux-mips@linux-mips.org Patchwork: http://patchwork.linux-mips.org/patch/975/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2010-02-19 07:13:04 +07:00
#define cpu_context(cpu, mm) ((mm)->context.asid[cpu])
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#define asid_cache(cpu) (cpu_data[cpu].asid_cache)
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
#define cpu_asid(cpu, mm) \
(cpu_context((cpu), (mm)) & cpu_asid_mask(&cpu_data[cpu]))
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
{
}
Revert "MIPS: Allow ASID size to be determined at boot time." This reverts commit d532f3d26716a39dfd4b88d687bd344fbe77e390. The original commit has several problems: 1) Doesn't work with 64-bit kernels. 2) Calls TLBMISS_HANDLER_SETUP() before the code is generated. 3) Calls TLBMISS_HANDLER_SETUP() twice in per_cpu_trap_init() when only one call is needed. [ralf@linux-mips.org: Also revert the bits of the ASID patch which were hidden in the KVM merge.] Signed-off-by: David Daney <david.daney@cavium.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: "Steven J. Hill" <Steven.Hill@imgtec.com> Cc: David Daney <david.daney@cavium.com> Patchwork: https://patchwork.linux-mips.org/patch/5242/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-05-14 03:56:44 +07:00
[MIPS] MT: Improved multithreading support. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-04-05 15:45:45 +07:00
/* Normal, classic MIPS get_new_mmu_context */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
static inline void
get_new_mmu_context(struct mm_struct *mm, unsigned long cpu)
{
MIPS: If KVM is enabled then use the KVM specific routine to flush the TLBs on a ASID wrap. Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-22 09:34:11 +07:00
extern void kvm_local_flush_tlb_all(void);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
unsigned long asid = asid_cache(cpu);
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
if (!((asid += cpu_asid_inc()) & cpu_asid_mask(&cpu_data[cpu]))) {
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
if (cpu_has_vtag_icache)
flush_icache_all();
MIPS: include: mmu_context.h: Replace VIRTUALIZATION with KVM The kvm_* symbols are only available if KVM is selected. Fixes the following linking problem on a randconfig: arch/mips/built-in.o: In function `local_flush_tlb_mm': (.text+0x18a94): undefined reference to `kvm_local_flush_tlb_all' arch/mips/built-in.o: In function `local_flush_tlb_range': (.text+0x18d0c): undefined reference to `kvm_local_flush_tlb_all' kernel/built-in.o: In function `__schedule': core.c:(.sched.text+0x2a00): undefined reference to `kvm_local_flush_tlb_all' mm/built-in.o: In function `use_mm': (.text+0x30214): undefined reference to `kvm_local_flush_tlb_all' fs/built-in.o: In function `flush_old_exec': (.text+0xf0a0): undefined reference to `kvm_local_flush_tlb_all' make: *** [vmlinux] Error 1 Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Acked-by: Steven J. Hill <Steven.Hill@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/5437/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-06-10 19:16:16 +07:00
#ifdef CONFIG_KVM
MIPS: If KVM is enabled then use the KVM specific routine to flush the TLBs on a ASID wrap. Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-22 09:34:11 +07:00
kvm_local_flush_tlb_all(); /* start new asid cycle */
#else
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
local_flush_tlb_all(); /* start new asid cycle */
MIPS: If KVM is enabled then use the KVM specific routine to flush the TLBs on a ASID wrap. Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-22 09:34:11 +07:00
#endif
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
if (!asid) /* fix version if needed */
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
asid = asid_first_version(cpu);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
}
MIPS: If KVM is enabled then use the KVM specific routine to flush the TLBs on a ASID wrap. Signed-off-by: Sanjay Lal <sanjayl@kymasys.com> Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-11-22 09:34:11 +07:00
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
cpu_context(cpu, mm) = asid_cache(cpu) = asid;
}
/*
* Initialize the context related info for a new mm_struct
* instance.
*/
static inline int
init_new_context(struct task_struct *tsk, struct mm_struct *mm)
{
int i;
MIPS: Init new mmu_context for each possible CPU to avoid memory corruption Currently, init_new_context() only for each online CPU, this may cause memory corruption when CPU hotplug and fork() happens at the same time. To avoid this, we make init_new_context() cover each possible CPU. Scenario: 1, CPU#1 is being offline; 2, On CPU#0, do_fork() call dup_mm() and copy a mm_struct to the child; 3, On CPU#0, dup_mm() call init_new_context(), since CPU#1 is offline and init_new_context() only covers the online CPUs, child has the same asid as its parent on CPU#1 (however, child's asid should be 0); 4, CPU#1 is being online; 5, Now, if both parent and child run on CPU#1, memory corruption (e.g. segfault, bus error, etc.) will occur. Signed-off-by: Huacai Chen <chenhc@lemote.com> Acked-by: David Daney <david.daney@cavium.com> Patchwork: http://patchwork.linux-mips.org/patch/4995/ Acked-by: John Crispin <blogic@openwrt.org>
2013-03-17 18:50:14 +07:00
for_each_possible_cpu(i)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
cpu_context(i, mm) = 0;
MIPS,prctl: add PR_[GS]ET_FP_MODE prctl options for MIPS Userland code may be built using an ABI which permits linking to objects that have more restrictive floating point requirements. For example, userland code may be built to target the O32 FPXX ABI. Such code may be linked with other FPXX code, or code built for either one of the more restrictive FP32 or FP64. When linking with more restrictive code, the overall requirement of the process becomes that of the more restrictive code. The kernel has no way to know in advance which mode the process will need to be executed in, and indeed it may need to change during execution. The dynamic loader is the only code which will know the overall required mode, and so it needs to have a means to instruct the kernel to switch the FP mode of the process. This patch introduces 2 new options to the prctl syscall which provide such a capability. The FP mode of the process is represented as a simple bitmask combining a number of mode bits mirroring those present in the hardware. Userland can either retrieve the current FP mode of the process: mode = prctl(PR_GET_FP_MODE); or modify the current FP mode of the process: err = prctl(PR_SET_FP_MODE, new_mode); Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/8899/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-08 19:17:37 +07:00
atomic_set(&mm->context.fp_mode_switching, 0);
MIPS: Use per-mm page to execute branch delay slot instructions In some cases the kernel needs to execute an instruction from the delay slot of an emulated branch instruction. These cases include: - Emulated floating point branch instructions (bc1[ft]l?) for systems which don't include an FPU, or upon which the kernel is run with the "nofpu" parameter. - MIPSr6 systems running binaries targeting older revisions of the architecture, which may include branch instructions whose encodings are no longer valid in MIPSr6. Executing instructions from such delay slots is done by writing the instruction to memory followed by a trap, as part of an "emuframe", and executing it. This avoids the requirement of an emulator for the entire MIPS instruction set. Prior to this patch such emuframes are written to the user stack and executed from there. This patch moves FP branch delay emuframes off of the user stack and into a per-mm page. Allocating a page per-mm leaves userland with access to only what it had access to previously, and compared to other solutions is relatively simple. When a thread requires a delay slot emulation, it is allocated a frame. A thread may only have one frame allocated at any one time, since it may only ever be executing one instruction at any one time. In order to ensure that we can free up allocated frame later, its index is recorded in struct thread_struct. In the typical case, after executing the delay slot instruction we'll execute a break instruction with the BRK_MEMU code. This traps back to the kernel & leads to a call to do_dsemulret which frees the allocated frame & moves the user PC back to the instruction that would have executed following the emulated branch. In some cases the delay slot instruction may be invalid, such as a branch, or may trigger an exception. In these cases the BRK_MEMU break instruction will not be hit. In order to ensure that frames are freed this patch introduces dsemul_thread_cleanup() and calls it to free any allocated frame upon thread exit. If the instruction generated an exception & leads to a signal being delivered to the thread, or indeed if a signal simply happens to be delivered to the thread whilst it is executing from the struct emuframe, then we need to take care to exit the frame appropriately. This is done by either rolling back the user PC to the branch or advancing it to the continuation PC prior to signal delivery, using dsemul_thread_rollback(). If this were not done then a sigreturn would return to the struct emuframe, and if that frame had meanwhile been used in response to an emulated branch instruction within the signal handler then we would execute the wrong user code. Whilst a user could theoretically place something like a compact branch to self in a delay slot and cause their thread to become stuck in an infinite loop with the frame never being deallocated, this would: - Only affect the users single process. - Be architecturally invalid since there would be a branch in the delay slot, which is forbidden. - Be extremely unlikely to happen by mistake, and provide a program with no more ability to harm the system than a simple infinite loop would. If a thread requires a delay slot emulation & no frame is available to it (ie. the process has enough other threads that all frames are currently in use) then the thread joins a waitqueue. It will sleep until a frame is freed by another thread in the process. Since we now know whether a thread has an allocated frame due to our tracking of its index, the cookie field of struct emuframe is removed as we can be more certain whether we have a valid frame. Since a thread may only ever have a single frame at any given time, the epc field of struct emuframe is also removed & the PC to continue from is instead stored in struct thread_struct. Together these changes simplify & shrink struct emuframe somewhat, allowing twice as many frames to fit into the page allocated for them. The primary benefit of this patch is that we are now free to mark the user stack non-executable where that is possible. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: Maciej Rozycki <maciej.rozycki@imgtec.com> Cc: Faraz Shahbazker <faraz.shahbazker@imgtec.com> Cc: Raghu Gandham <raghu.gandham@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13764/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-07-08 17:06:19 +07:00
mm->context.bd_emupage_allocmap = NULL;
spin_lock_init(&mm->context.bd_emupage_lock);
init_waitqueue_head(&mm->context.bd_emupage_queue);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
return 0;
}
static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
MIPS: Whitespace cleanup. Having received another series of whitespace patches I decided to do this once and for all rather than dealing with this kind of patches trickling in forever. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2013-01-22 18:59:30 +07:00
struct task_struct *tsk)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
{
unsigned int cpu = smp_processor_id();
unsigned long flags;
[MIPS] MT: Improved multithreading support. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-04-05 15:45:45 +07:00
local_irq_save(flags);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_stop();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/* Check if our ASID is of an older version and thus invalid */
MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mips In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-05-06 20:36:23 +07:00
if ((cpu_context(cpu, next) ^ asid_cache(cpu)) & asid_version_mask(cpu))
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
get_new_mmu_context(next, cpu);
MIPS: Don't write ones to reserved entryhi bits. We've silently been relying on the hardware chopping off excess, reserved ASID bits for no better reason that it saving an instruction. Because we already have: #define cpu_asid(cpu, mm) (cpu_context((cpu), (mm)) & ASID_MASK) in <asm/mmu_context.h>. We can use a cleanup to avoid writing non-zero bits into the reserved entryhi bits. This avoid triggering some debugging assertion in the Cavium simulator. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-05-27 23:29:37 +07:00
write_c0_entryhi(cpu_asid(cpu, next));
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
TLBMISS_HANDLER_SETUP_PGD(next->pgd);
/*
* Mark current->active_mm as not "active" anymore.
* We don't want to mislead possible IPI tlb flush routines.
*/
cpumask: use mm_cpumask() wrapper: mips Makes code futureproof against the impending change to mm->cpu_vm_mask. It's also a chance to use the new cpumask_ ops which take a pointer (the older ones are deprecated, but there's no hurry for arch code). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2009-09-24 22:34:50 +07:00
cpumask_clear_cpu(cpu, mm_cpumask(prev));
cpumask_set_cpu(cpu, mm_cpumask(next));
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_start();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
local_irq_restore(flags);
}
/*
* Destroy context related info for an mm_struct that is about
* to be put to rest.
*/
static inline void destroy_context(struct mm_struct *mm)
{
MIPS: Use per-mm page to execute branch delay slot instructions In some cases the kernel needs to execute an instruction from the delay slot of an emulated branch instruction. These cases include: - Emulated floating point branch instructions (bc1[ft]l?) for systems which don't include an FPU, or upon which the kernel is run with the "nofpu" parameter. - MIPSr6 systems running binaries targeting older revisions of the architecture, which may include branch instructions whose encodings are no longer valid in MIPSr6. Executing instructions from such delay slots is done by writing the instruction to memory followed by a trap, as part of an "emuframe", and executing it. This avoids the requirement of an emulator for the entire MIPS instruction set. Prior to this patch such emuframes are written to the user stack and executed from there. This patch moves FP branch delay emuframes off of the user stack and into a per-mm page. Allocating a page per-mm leaves userland with access to only what it had access to previously, and compared to other solutions is relatively simple. When a thread requires a delay slot emulation, it is allocated a frame. A thread may only have one frame allocated at any one time, since it may only ever be executing one instruction at any one time. In order to ensure that we can free up allocated frame later, its index is recorded in struct thread_struct. In the typical case, after executing the delay slot instruction we'll execute a break instruction with the BRK_MEMU code. This traps back to the kernel & leads to a call to do_dsemulret which frees the allocated frame & moves the user PC back to the instruction that would have executed following the emulated branch. In some cases the delay slot instruction may be invalid, such as a branch, or may trigger an exception. In these cases the BRK_MEMU break instruction will not be hit. In order to ensure that frames are freed this patch introduces dsemul_thread_cleanup() and calls it to free any allocated frame upon thread exit. If the instruction generated an exception & leads to a signal being delivered to the thread, or indeed if a signal simply happens to be delivered to the thread whilst it is executing from the struct emuframe, then we need to take care to exit the frame appropriately. This is done by either rolling back the user PC to the branch or advancing it to the continuation PC prior to signal delivery, using dsemul_thread_rollback(). If this were not done then a sigreturn would return to the struct emuframe, and if that frame had meanwhile been used in response to an emulated branch instruction within the signal handler then we would execute the wrong user code. Whilst a user could theoretically place something like a compact branch to self in a delay slot and cause their thread to become stuck in an infinite loop with the frame never being deallocated, this would: - Only affect the users single process. - Be architecturally invalid since there would be a branch in the delay slot, which is forbidden. - Be extremely unlikely to happen by mistake, and provide a program with no more ability to harm the system than a simple infinite loop would. If a thread requires a delay slot emulation & no frame is available to it (ie. the process has enough other threads that all frames are currently in use) then the thread joins a waitqueue. It will sleep until a frame is freed by another thread in the process. Since we now know whether a thread has an allocated frame due to our tracking of its index, the cookie field of struct emuframe is removed as we can be more certain whether we have a valid frame. Since a thread may only ever have a single frame at any given time, the epc field of struct emuframe is also removed & the PC to continue from is instead stored in struct thread_struct. Together these changes simplify & shrink struct emuframe somewhat, allowing twice as many frames to fit into the page allocated for them. The primary benefit of this patch is that we are now free to mark the user stack non-executable where that is possible. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: Maciej Rozycki <maciej.rozycki@imgtec.com> Cc: Faraz Shahbazker <faraz.shahbazker@imgtec.com> Cc: Raghu Gandham <raghu.gandham@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13764/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-07-08 17:06:19 +07:00
dsemul_mm_cleanup(mm);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
}
[MIPS] checkfiles: Fix "need space after that ','" errors. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-10-12 05:46:15 +07:00
#define deactivate_mm(tsk, mm) do { } while (0)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/*
* After we have set current->mm to a new value, this activates
* the context for the new mm so we see the new mappings.
*/
static inline void
activate_mm(struct mm_struct *prev, struct mm_struct *next)
{
unsigned long flags;
unsigned int cpu = smp_processor_id();
local_irq_save(flags);
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_stop();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/* Unconditionally get a new ASID. */
get_new_mmu_context(next, cpu);
MIPS: Don't write ones to reserved entryhi bits. We've silently been relying on the hardware chopping off excess, reserved ASID bits for no better reason that it saving an instruction. Because we already have: #define cpu_asid(cpu, mm) (cpu_context((cpu), (mm)) & ASID_MASK) in <asm/mmu_context.h>. We can use a cleanup to avoid writing non-zero bits into the reserved entryhi bits. This avoid triggering some debugging assertion in the Cavium simulator. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2009-05-27 23:29:37 +07:00
write_c0_entryhi(cpu_asid(cpu, next));
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
TLBMISS_HANDLER_SETUP_PGD(next->pgd);
/* mark mmu ownership change */
cpumask: use mm_cpumask() wrapper: mips Makes code futureproof against the impending change to mm->cpu_vm_mask. It's also a chance to use the new cpumask_ ops which take a pointer (the older ones are deprecated, but there's no hurry for arch code). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2009-09-24 22:34:50 +07:00
cpumask_clear_cpu(cpu, mm_cpumask(prev));
cpumask_set_cpu(cpu, mm_cpumask(next));
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_start();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
local_irq_restore(flags);
}
/*
* If mm is currently active_mm, we can't really drop it. Instead,
* we will get a new one for it.
*/
static inline void
drop_mmu_context(struct mm_struct *mm, unsigned cpu)
{
unsigned long flags;
local_irq_save(flags);
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_stop();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
cpumask: use mm_cpumask() wrapper: mips Makes code futureproof against the impending change to mm->cpu_vm_mask. It's also a chance to use the new cpumask_ ops which take a pointer (the older ones are deprecated, but there's no hurry for arch code). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2009-09-24 22:34:50 +07:00
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) {
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
get_new_mmu_context(mm, cpu);
write_c0_entryhi(cpu_asid(cpu, mm));
} else {
/* will get a new context next time */
cpu_context(cpu, mm) = 0;
}
MIPS: HTW: Prevent accidental HTW start due to nested htw_{start, stop} activate_mm() and switch_mm() call get_new_mmu_context() which in turn can enable the HTW before the entryhi is changed with the new ASID. Since the latter will enable the HTW in local_flush_tlb_all(), then there is a small timing window where the HTW is running with the new ASID but with an old pgd since the TLBMISS_HANDLER_SETUP_PGD hasn't assigned a new one yet. In order to prevent that, we introduce a simple htw counter to avoid starting HTW accidentally due to nested htw_{start,stop}() sequences. Moreover, since various IPI calls can enforce TLB flushing operations on a different core, such an operation may interrupt another htw_{stop,start} in progress leading inconsistent updates of the htw_seq variable. In order to avoid that, we disable the interrupts whenever we update that variable. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # 3.17+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9118/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-01-26 20:04:33 +07:00
htw_start();
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
local_irq_restore(flags);
}
#endif /* _ASM_MMU_CONTEXT_H */
Reference in New Issue Copy Permalink