AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2024-12-26 04:45:36 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity
bb176f6709
linux_dsm_epyc7002/arch/sparc/kernel/etrap_64.S

259 lines
6.0 KiB
ArmAsm
Raw Normal View History

sparc64: remove CVS keywords This patch removes the CVS keywords that weren't updated for a long time from comments. Signed-off-by: Adrian Bunk <bunk@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2008-05-20 06:52:27 +07:00
/*
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
* etrap.S: Preparing for entry into the kernel on Sparc V9.
*
* Copyright (C) 1996, 1997 David S. Miller (davem@caip.rutgers.edu)
* Copyright (C) 1997, 1998, 1999 Jakub Jelinek (jj@ultra.linux.cz)
*/
#include <asm/asi.h>
#include <asm/pstate.h>
#include <asm/ptrace.h>
#include <asm/page.h>
#include <asm/spitfire.h>
#include <asm/head.h>
#include <asm/processor.h>
#include <asm/mmu.h>
#define TASK_REGOFF (THREAD_SIZE-TRACEREG_SZ-STACKFRAME_SZ)
sparc64: Run the kernel always in the TSO memory model. The fact of the matter is, all UltraSPARC-III and later chips only implement TSO. They don't implement PSO and RMO memory models at all. Only the Ultra-I and Ultra-II family chips implement RMO and they are only helped marginally by using this setting when executing kernel code. The big plus to doing this is that we can eliminate all of the non-Sync memory barriers in the kernel except for the ones used in the optimized memcpy/memset code (these use block load and store operations which have their own memory ordering rules). Signed-off-by: David S. Miller <davem@davemloft.net>
2008-10-30 11:25:00 +07:00
#define ETRAP_PSTATE1 (PSTATE_TSO | PSTATE_PRIV)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#define ETRAP_PSTATE2 \
sparc64: Run the kernel always in the TSO memory model. The fact of the matter is, all UltraSPARC-III and later chips only implement TSO. They don't implement PSO and RMO memory models at all. Only the Ultra-I and Ultra-II family chips implement RMO and they are only helped marginally by using this setting when executing kernel code. The big plus to doing this is that we can eliminate all of the non-Sync memory barriers in the kernel except for the ones used in the optimized memcpy/memset code (these use block load and store operations which have their own memory ordering rules). Signed-off-by: David S. Miller <davem@davemloft.net>
2008-10-30 11:25:00 +07:00
(PSTATE_TSO | PSTATE_PEF | PSTATE_PRIV | PSTATE_IE)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/*
* On entry, %g7 is return address - 0x4.
* %g4 and %g5 will be preserved %l4 and %l5 respectively.
*/
.text
.align 64
sparc: Fix debugger syscall restart interactions. So, forever, we've had this ptrace_signal_deliver implementation which tries to handle all of the nasties that can occur when the debugger looks at a process about to take a signal. It's meant to address all of these issues inside of the kernel so that the debugger need not be mindful of such things. Problem is, this doesn't work. The idea was that we should do the syscall restart business first, so that the debugger captures that state. Otherwise, if the debugger for example saves the child's state, makes the child execute something else, then restores the saved state, we won't handle the syscall restart properly because we lose the "we're in a syscall" state. The code here worked for most cases, but if the debugger actually passes the signal through to the child unaltered, it's possible that we would do a syscall restart when we shouldn't have. In particular this breaks the case of debugging a process under a gdb which is being debugged by yet another gdb. gdb uses sigsuspend to wait for SIGCHLD of the inferior, but if gdb itself is being debugged by a top-level gdb we get a ptrace_stop(). The top-level gdb does a PTRACE_CONT with SIGCHLD to let the inferior gdb see the signal. But ptrace_signal_deliver() assumed the debugger would cancel out the signal and therefore did a syscall restart, because the return error was ERESTARTNOHAND. Fix this by simply making ptrace_signal_deliver() a nop, and providing a way for the debugger to control system call restarting properly: 1) Report a "in syscall" software bit in regs->{tstate,psr}. It is set early on in trap entry to a system call and is fully visible to the debugger via ptrace() and regsets. 2) Test this bit right before doing a syscall restart. We have to do a final recheck right after get_signal_to_deliver() in case the debugger cleared the bit during ptrace_stop(). 3) Clear the bit in trap return so we don't accidently try to set that bit in the real register. As a result we also get a ptrace_{is,clear}_syscall() for sparc32 just like sparc64 has. M68K has this same exact bug, and is now the only other user of the ptrace_signal_deliver hook. It needs to be fixed in the same exact way as sparc. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-05-11 16:07:19 +07:00
.globl etrap_syscall, etrap, etrap_irq, etraptl1
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
etrap: rdpr %pil, %g2
sparc: Fix debugger syscall restart interactions. So, forever, we've had this ptrace_signal_deliver implementation which tries to handle all of the nasties that can occur when the debugger looks at a process about to take a signal. It's meant to address all of these issues inside of the kernel so that the debugger need not be mindful of such things. Problem is, this doesn't work. The idea was that we should do the syscall restart business first, so that the debugger captures that state. Otherwise, if the debugger for example saves the child's state, makes the child execute something else, then restores the saved state, we won't handle the syscall restart properly because we lose the "we're in a syscall" state. The code here worked for most cases, but if the debugger actually passes the signal through to the child unaltered, it's possible that we would do a syscall restart when we shouldn't have. In particular this breaks the case of debugging a process under a gdb which is being debugged by yet another gdb. gdb uses sigsuspend to wait for SIGCHLD of the inferior, but if gdb itself is being debugged by a top-level gdb we get a ptrace_stop(). The top-level gdb does a PTRACE_CONT with SIGCHLD to let the inferior gdb see the signal. But ptrace_signal_deliver() assumed the debugger would cancel out the signal and therefore did a syscall restart, because the return error was ERESTARTNOHAND. Fix this by simply making ptrace_signal_deliver() a nop, and providing a way for the debugger to control system call restarting properly: 1) Report a "in syscall" software bit in regs->{tstate,psr}. It is set early on in trap entry to a system call and is fully visible to the debugger via ptrace() and regsets. 2) Test this bit right before doing a syscall restart. We have to do a final recheck right after get_signal_to_deliver() in case the debugger cleared the bit during ptrace_stop(). 3) Clear the bit in trap return so we don't accidently try to set that bit in the real register. As a result we also get a ptrace_{is,clear}_syscall() for sparc32 just like sparc64 has. M68K has this same exact bug, and is now the only other user of the ptrace_signal_deliver hook. It needs to be fixed in the same exact way as sparc. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-05-11 16:07:19 +07:00
etrap_irq: clr %g3
etrap_syscall: TRAP_LOAD_THREAD_REG(%g6, %g1)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
rdpr %tstate, %g1
sparc: Fix debugger syscall restart interactions. So, forever, we've had this ptrace_signal_deliver implementation which tries to handle all of the nasties that can occur when the debugger looks at a process about to take a signal. It's meant to address all of these issues inside of the kernel so that the debugger need not be mindful of such things. Problem is, this doesn't work. The idea was that we should do the syscall restart business first, so that the debugger captures that state. Otherwise, if the debugger for example saves the child's state, makes the child execute something else, then restores the saved state, we won't handle the syscall restart properly because we lose the "we're in a syscall" state. The code here worked for most cases, but if the debugger actually passes the signal through to the child unaltered, it's possible that we would do a syscall restart when we shouldn't have. In particular this breaks the case of debugging a process under a gdb which is being debugged by yet another gdb. gdb uses sigsuspend to wait for SIGCHLD of the inferior, but if gdb itself is being debugged by a top-level gdb we get a ptrace_stop(). The top-level gdb does a PTRACE_CONT with SIGCHLD to let the inferior gdb see the signal. But ptrace_signal_deliver() assumed the debugger would cancel out the signal and therefore did a syscall restart, because the return error was ERESTARTNOHAND. Fix this by simply making ptrace_signal_deliver() a nop, and providing a way for the debugger to control system call restarting properly: 1) Report a "in syscall" software bit in regs->{tstate,psr}. It is set early on in trap entry to a system call and is fully visible to the debugger via ptrace() and regsets. 2) Test this bit right before doing a syscall restart. We have to do a final recheck right after get_signal_to_deliver() in case the debugger cleared the bit during ptrace_stop(). 3) Clear the bit in trap return so we don't accidently try to set that bit in the real register. As a result we also get a ptrace_{is,clear}_syscall() for sparc32 just like sparc64 has. M68K has this same exact bug, and is now the only other user of the ptrace_signal_deliver hook. It needs to be fixed in the same exact way as sparc. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-05-11 16:07:19 +07:00
or %g1, %g3, %g1
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
sllx %g2, 20, %g3
andcc %g1, TSTATE_PRIV, %g0
or %g1, %g3, %g1
bne,pn %xcc, 1f
sub %sp, STACKFRAME_SZ+TRACEREG_SZ-STACK_BIAS, %g2
sparc64: speed up etrap/rtrap on NG2 and later processors For many sun4v processor types, reading or writing a privileged register has a latency of 40 to 70 cycles. Use a combination of the low-latency allclean, otherw, normalw, and nop instructions in etrap and rtrap to replace 2 rdpr and 5 wrpr instructions and improve etrap/rtrap performance. allclean, otherw, and normalw are available on NG2 and later processors. The average ticks to execute the flush windows trap ("ta 0x3") with and without this patch on select platforms: CPU Not patched Patched % Latency Reduction NG2 1762 1558 -11.58 NG4 3619 3204 -11.47 M7 3015 2624 -12.97 SPARC64-X 829 770 -7.12 Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-08-19 02:40:36 +07:00
661: wrpr %g0, 7, %cleanwin
.section .fast_win_ctrl_1insn_patch, "ax"
.word 661b
.word 0x85880000 ! allclean
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
sethi %hi(TASK_REGOFF), %g2
sethi %hi(TSTATE_PEF), %g3
or %g2, %lo(TASK_REGOFF), %g2
and %g1, %g3, %g3
brnz,pn %g3, 1f
add %g6, %g2, %g2
wr %g0, 0, %fprs
1: rdpr %tpc, %g3
stx %g1, [%g2 + STACKFRAME_SZ + PT_V9_TSTATE]
rdpr %tnpc, %g1
stx %g3, [%g2 + STACKFRAME_SZ + PT_V9_TPC]
rd %y, %g3
stx %g1, [%g2 + STACKFRAME_SZ + PT_V9_TNPC]
[SPARC64]: Store magic cookie and trap type in pt_regs. This sets us up for several simplifications and facilities: 1) The magic cookie lets us identify trap frames more accurately in stack backtraces. 2) The trap type lets us simplify all of the "are we in a syscall" state management and checks. 3) We can now see if a task off the cpu is sleeping in a system call or not. In fact, we can see what trap it is sleeping in whatever the type. The utrace guys will use this. Based upon some discussions with Roland McGrath. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-04-24 12:52:13 +07:00
rdpr %tt, %g1
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
st %g3, [%g2 + STACKFRAME_SZ + PT_V9_Y]
[SPARC64]: Store magic cookie and trap type in pt_regs. This sets us up for several simplifications and facilities: 1) The magic cookie lets us identify trap frames more accurately in stack backtraces. 2) The trap type lets us simplify all of the "are we in a syscall" state management and checks. 3) We can now see if a task off the cpu is sleeping in a system call or not. In fact, we can see what trap it is sleeping in whatever the type. The utrace guys will use this. Based upon some discussions with Roland McGrath. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-04-24 12:52:13 +07:00
sethi %hi(PT_REGS_MAGIC), %g3
or %g3, %g1, %g1
st %g1, [%g2 + STACKFRAME_SZ + PT_V9_MAGIC]
[SPARC64]: Refine register window trap handling. When saving and restoing trap state, do the window spill/fill handling inline so that we never trap deeper than 2 trap levels. This is important for chips like Niagara. The window fixup code is massively simplified, and many more improvements are now possible. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-04 15:10:01 +07:00
rdpr %cansave, %g1
brnz,pt %g1, etrap_save
nop
rdpr %cwp, %g1
add %g1, 2, %g1
wrpr %g1, %cwp
be,pt %xcc, etrap_user_spill
mov ASI_AIUP, %g3
rdpr %otherwin, %g3
brz %g3, etrap_kernel_spill
mov ASI_AIUS, %g3
etrap_user_spill:
wr %g3, 0x0, %asi
ldx [%g6 + TI_FLAGS], %g3
and %g3, _TIF_32BIT, %g3
brnz,pt %g3, etrap_user_spill_32bit
nop
ba,a,pt %xcc, etrap_user_spill_64bit
etrap_save: save %g2, -STACK_BIAS, %sp
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
mov %g6, %l6
bne,pn %xcc, 3f
mov PRIMARY_CONTEXT, %l4
sparc64: speed up etrap/rtrap on NG2 and later processors For many sun4v processor types, reading or writing a privileged register has a latency of 40 to 70 cycles. Use a combination of the low-latency allclean, otherw, normalw, and nop instructions in etrap and rtrap to replace 2 rdpr and 5 wrpr instructions and improve etrap/rtrap performance. allclean, otherw, and normalw are available on NG2 and later processors. The average ticks to execute the flush windows trap ("ta 0x3") with and without this patch on select platforms: CPU Not patched Patched % Latency Reduction NG2 1762 1558 -11.58 NG4 3619 3204 -11.47 M7 3015 2624 -12.97 SPARC64-X 829 770 -7.12 Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-08-19 02:40:36 +07:00
661: rdpr %canrestore, %g3
.section .fast_win_ctrl_1insn_patch, "ax"
.word 661b
nop
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
rdpr %wstate, %g2
sparc64: speed up etrap/rtrap on NG2 and later processors For many sun4v processor types, reading or writing a privileged register has a latency of 40 to 70 cycles. Use a combination of the low-latency allclean, otherw, normalw, and nop instructions in etrap and rtrap to replace 2 rdpr and 5 wrpr instructions and improve etrap/rtrap performance. allclean, otherw, and normalw are available on NG2 and later processors. The average ticks to execute the flush windows trap ("ta 0x3") with and without this patch on select platforms: CPU Not patched Patched % Latency Reduction NG2 1762 1558 -11.58 NG4 3619 3204 -11.47 M7 3015 2624 -12.97 SPARC64-X 829 770 -7.12 Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-08-19 02:40:36 +07:00
661: wrpr %g0, 0, %canrestore
.section .fast_win_ctrl_1insn_patch, "ax"
.word 661b
nop
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
sll %g2, 3, %g2
sparc64: clear syscall_noerror on the entry to syscall, not on the exit Move that sucker to just before TI_FPDEPTH and replace stb with sth in etrap_save(). Take current_ds to its old place, so that we don't push wsaved into TI_... flags. That allows to lose clearing syscall_noerror on return from syscall. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-09-26 12:21:14 +07:00
/* Set TI_SYS_FPDEPTH to 1 and clear TI_SYS_NOERROR. */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
mov 1, %l5
sparc64: clear syscall_noerror on the entry to syscall, not on the exit Move that sucker to just before TI_FPDEPTH and replace stb with sth in etrap_save(). Take current_ds to its old place, so that we don't push wsaved into TI_... flags. That allows to lose clearing syscall_noerror on return from syscall. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-09-26 12:21:14 +07:00
sth %l5, [%l6 + TI_SYS_NOERROR]
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
sparc64: speed up etrap/rtrap on NG2 and later processors For many sun4v processor types, reading or writing a privileged register has a latency of 40 to 70 cycles. Use a combination of the low-latency allclean, otherw, normalw, and nop instructions in etrap and rtrap to replace 2 rdpr and 5 wrpr instructions and improve etrap/rtrap performance. allclean, otherw, and normalw are available on NG2 and later processors. The average ticks to execute the flush windows trap ("ta 0x3") with and without this patch on select platforms: CPU Not patched Patched % Latency Reduction NG2 1762 1558 -11.58 NG4 3619 3204 -11.47 M7 3015 2624 -12.97 SPARC64-X 829 770 -7.12 Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-08-19 02:40:36 +07:00
661: wrpr %g3, 0, %otherwin
.section .fast_win_ctrl_1insn_patch, "ax"
.word 661b
.word 0x87880000 ! otherw
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
wrpr %g2, 0, %wstate
[SPARC64]: Replace cheetah+ code patching with variables. Instead of code patching to handle the page size fields in the context registers, just use variables from which we get the proper values. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-05 05:23:20 +07:00
sethi %hi(sparc64_kern_pri_context), %g2
ldx [%g2 + %lo(sparc64_kern_pri_context)], %g3
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 13:13:05 +07:00
661: stxa %g3, [%l4] ASI_DMMU
.section .sun4v_1insn_patch, "ax"
.word 661b
stxa %g3, [%l4] ASI_MMU
.previous
[SPARC64]: Fix bogus flush instruction usage. Some of the trap code was still assuming that alternate global %g6 was hard coded with current_thread_info(). Let's just consistently flush at KERNBASE when we need a pipeline synchronization. That's locked into the TLB and will always work. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-01 09:33:00 +07:00
sethi %hi(KERNBASE), %l4
flush %l4
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
mov ASI_AIUS, %l7
2: mov %g4, %l4
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
mov %g5, %l5
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
add %g7, 4, %l2
[SPARC64]: Add initial code to twiddle %gl on trap entry/exit. Instead of setting/clearing PSTATE_AG we have to change the %gl register value on sun4v. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 12:29:28 +07:00
/* Go to trap time globals so we can save them. */
661: wrpr %g0, ETRAP_PSTATE1, %pstate
[SPARC64]: Rename gl_{1,2}insn_patch --> sun4v_{1,2}insn_patch Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 15:00:16 +07:00
.section .sun4v_1insn_patch, "ax"
[SPARC64]: Add initial code to twiddle %gl on trap entry/exit. Instead of setting/clearing PSTATE_AG we have to change the %gl register value on sun4v. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 12:29:28 +07:00
.word 661b
SET_GL(0)
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
stx %g1, [%sp + PTREGS_OFF + PT_V9_G1]
stx %g2, [%sp + PTREGS_OFF + PT_V9_G2]
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
sllx %l7, 24, %l7
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
stx %g3, [%sp + PTREGS_OFF + PT_V9_G3]
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
rdpr %cwp, %l0
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
stx %g4, [%sp + PTREGS_OFF + PT_V9_G4]
stx %g5, [%sp + PTREGS_OFF + PT_V9_G5]
stx %g6, [%sp + PTREGS_OFF + PT_V9_G6]
stx %g7, [%sp + PTREGS_OFF + PT_V9_G7]
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
or %l7, %l0, %l7
sparc64: Run the kernel always in the TSO memory model. The fact of the matter is, all UltraSPARC-III and later chips only implement TSO. They don't implement PSO and RMO memory models at all. Only the Ultra-I and Ultra-II family chips implement RMO and they are only helped marginally by using this setting when executing kernel code. The big plus to doing this is that we can eliminate all of the non-Sync memory barriers in the kernel except for the ones used in the optimized memcpy/memset code (these use block load and store operations which have their own memory ordering rules). Signed-off-by: David S. Miller <davem@davemloft.net>
2008-10-30 11:25:00 +07:00
sethi %hi(TSTATE_TSO | TSTATE_PEF), %l0
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
or %l7, %l0, %l7
wrpr %l2, %tnpc
wrpr %l7, (TSTATE_PRIV | TSTATE_IE), %tstate
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
stx %i0, [%sp + PTREGS_OFF + PT_V9_I0]
stx %i1, [%sp + PTREGS_OFF + PT_V9_I1]
stx %i2, [%sp + PTREGS_OFF + PT_V9_I2]
stx %i3, [%sp + PTREGS_OFF + PT_V9_I3]
stx %i4, [%sp + PTREGS_OFF + PT_V9_I4]
stx %i5, [%sp + PTREGS_OFF + PT_V9_I5]
stx %i6, [%sp + PTREGS_OFF + PT_V9_I6]
mov %l6, %g6
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
stx %i7, [%sp + PTREGS_OFF + PT_V9_I7]
[SPARC64]: Add explicit register args to trap state loading macros. This, as well as making the code cleaner, allows a simplification in the TSB miss handling path. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-03 12:55:10 +07:00
LOAD_PER_CPU_BASE(%g5, %g6, %g4, %g3, %l1)
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
ldx [%g6 + TI_TASK], %g4
done
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
[SPARC64]: Fill dead cycles on trap entry with real work. As we save trap state onto the stack, the store buffer fills up mid-way through and we stall for several cycles as the store buffer trickles out to the L2 cache. Meanwhile we can do some privileged register reads and other calculations, essentially for free. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 11:47:26 +07:00
3: mov ASI_P, %l7
ldub [%l6 + TI_FPDEPTH], %l5
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
add %l6, TI_FPSAVED + 1, %l4
srl %l5, 1, %l3
add %l5, 2, %l5
sparc64: clear syscall_noerror on the entry to syscall, not on the exit Move that sucker to just before TI_FPDEPTH and replace stb with sth in etrap_save(). Take current_ds to its old place, so that we don't push wsaved into TI_... flags. That allows to lose clearing syscall_noerror on return from syscall. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-09-26 12:21:14 +07:00
/* Set TI_SYS_FPDEPTH to %l5 and clear TI_SYS_NOERROR. */
sth %l5, [%l6 + TI_SYS_NOERROR]
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
ba,pt %xcc, 2b
stb %g0, [%l4 + %l3]
nop
etraptl1: /* Save tstate/tpc/tnpc of TL 1-->4 and the tl register itself.
* We place this right after pt_regs on the trap stack.
* The layout is:
* 0x00 TL1's TSTATE
* 0x08 TL1's TPC
* 0x10 TL1's TNPC
* 0x18 TL1's TT
* ...
* 0x58 TL4's TT
* 0x60 TL
*/
[SPARC64]: Add explicit register args to trap state loading macros. This, as well as making the code cleaner, allows a simplification in the TSB miss handling path. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-03 12:55:10 +07:00
TRAP_LOAD_THREAD_REG(%g6, %g1)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
sub %sp, ((4 * 8) * 4) + 8, %g2
rdpr %tl, %g1
wrpr %g0, 1, %tl
rdpr %tstate, %g3
stx %g3, [%g2 + STACK_BIAS + 0x00]
rdpr %tpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x08]
rdpr %tnpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x10]
rdpr %tt, %g3
stx %g3, [%g2 + STACK_BIAS + 0x18]
wrpr %g0, 2, %tl
rdpr %tstate, %g3
stx %g3, [%g2 + STACK_BIAS + 0x20]
rdpr %tpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x28]
rdpr %tnpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x30]
rdpr %tt, %g3
stx %g3, [%g2 + STACK_BIAS + 0x38]
[SPARC64]: Fix tl1 trap state capture/dump on SUN4V. No trap levels above 2 in privileged mode on SUN4V. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-16 16:41:41 +07:00
sethi %hi(is_sun4v), %g3
lduw [%g3 + %lo(is_sun4v)], %g3
brnz,pn %g3, finish_tl1_capture
nop
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
wrpr %g0, 3, %tl
rdpr %tstate, %g3
stx %g3, [%g2 + STACK_BIAS + 0x40]
rdpr %tpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x48]
rdpr %tnpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x50]
rdpr %tt, %g3
stx %g3, [%g2 + STACK_BIAS + 0x58]
wrpr %g0, 4, %tl
rdpr %tstate, %g3
stx %g3, [%g2 + STACK_BIAS + 0x60]
rdpr %tpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x68]
rdpr %tnpc, %g3
stx %g3, [%g2 + STACK_BIAS + 0x70]
rdpr %tt, %g3
stx %g3, [%g2 + STACK_BIAS + 0x78]
stx %g1, [%g2 + STACK_BIAS + 0x80]
[SPARC64]: Fix tl1 trap state capture/dump on SUN4V. No trap levels above 2 in privileged mode on SUN4V. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-16 16:41:41 +07:00
finish_tl1_capture:
[SPARC64]: Add initial code to twiddle %gl on trap entry/exit. Instead of setting/clearing PSTATE_AG we have to change the %gl register value on sun4v. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 12:29:28 +07:00
wrpr %g0, 1, %tl
661: nop
[SPARC64]: Rename gl_{1,2}insn_patch --> sun4v_{1,2}insn_patch Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-07 15:00:16 +07:00
.section .sun4v_1insn_patch, "ax"
[SPARC64]: Add initial code to twiddle %gl on trap entry/exit. Instead of setting/clearing PSTATE_AG we have to change the %gl register value on sun4v. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-06 12:29:28 +07:00
.word 661b
SET_GL(1)
.previous
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
rdpr %tstate, %g1
sub %g2, STACKFRAME_SZ + TRACEREG_SZ - STACK_BIAS, %g2
ba,pt %xcc, 1b
andcc %g1, TSTATE_PRIV, %g0
#undef TASK_REGOFF
#undef ETRAP_PSTATE1
Reference in New Issue Copy Permalink