AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2024-12-20 17:36:45 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity
b4b6aa8343
linux_dsm_epyc7002/arch/mips/kernel/entry.S

188 lines
4.3 KiB
ArmAsm
Raw Normal View History

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
/*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* Copyright (C) 1994 - 2000, 2001, 2003 Ralf Baechle
* Copyright (C) 1999, 2000 Silicon Graphics, Inc.
* Copyright (C) 2001 MIPS Technologies, Inc.
*/
#include <asm/asm.h>
#include <asm/asmmacro.h>
MIPS: kernel: entry.S: Set correct ISA level for mips_ihb Commit 6ebb496ffc7e("MIPS: kernel: entry.S: Add MIPS R6 related definitions") added the MIPSR6 definition but it did not update the ISA level of the actual assembly code so a pre-MIPSR6 jr.hb instruction was generated instead. Fix this by using the MISP_ISA_LEVEL_RAW macro. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Fixes: 6ebb496ffc7e("MIPS: kernel: entry.S: Add MIPS R6 related definitions") Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9386/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-03-04 01:48:49 +07:00
#include <asm/compiler.h>
MIPS: Fix IRQ tracing & lockdep when rescheduling When the scheduler sets TIF_NEED_RESCHED & we call into the scheduler from arch/mips/kernel/entry.S we disable interrupts. This is true regardless of whether we reach work_resched from syscall_exit_work, resume_userspace or by looping after calling schedule(). Although we disable interrupts in these paths we don't call trace_hardirqs_off() before calling into C code which may acquire locks, and we therefore leave lockdep with an inconsistent view of whether interrupts are disabled or not when CONFIG_PROVE_LOCKING & CONFIG_DEBUG_LOCKDEP are both enabled. Without tracing this interrupt state lockdep will print warnings such as the following once a task returns from a syscall via syscall_exit_partial with TIF_NEED_RESCHED set: [ 49.927678] ------------[ cut here ]------------ [ 49.934445] WARNING: CPU: 0 PID: 1 at kernel/locking/lockdep.c:3687 check_flags.part.41+0x1dc/0x1e8 [ 49.946031] DEBUG_LOCKS_WARN_ON(current->hardirqs_enabled) [ 49.946355] CPU: 0 PID: 1 Comm: init Not tainted 4.10.0-00439-gc9fd5d362289-dirty #197 [ 49.963505] Stack : 0000000000000000 ffffffff81bb5d6a 0000000000000006 ffffffff801ce9c4 [ 49.974431] 0000000000000000 0000000000000000 0000000000000000 000000000000004a [ 49.985300] ffffffff80b7e487 ffffffff80a24498 a8000000ff160000 ffffffff80ede8b8 [ 49.996194] 0000000000000001 0000000000000000 0000000000000000 0000000077c8030c [ 50.007063] 000000007fd8a510 ffffffff801cd45c 0000000000000000 a8000000ff127c88 [ 50.017945] 0000000000000000 ffffffff801cf928 0000000000000001 ffffffff80a24498 [ 50.028827] 0000000000000000 0000000000000001 0000000000000000 0000000000000000 [ 50.039688] 0000000000000000 a8000000ff127bd0 0000000000000000 ffffffff805509bc [ 50.050575] 00000000140084e0 0000000000000000 0000000000000000 0000000000040a00 [ 50.061448] 0000000000000000 ffffffff8010e1b0 0000000000000000 ffffffff805509bc [ 50.072327] ... [ 50.076087] Call Trace: [ 50.079869] [<ffffffff8010e1b0>] show_stack+0x80/0xa8 [ 50.086577] [<ffffffff805509bc>] dump_stack+0x10c/0x190 [ 50.093498] [<ffffffff8015dde0>] __warn+0xf0/0x108 [ 50.099889] [<ffffffff8015de34>] warn_slowpath_fmt+0x3c/0x48 [ 50.107241] [<ffffffff801c15b4>] check_flags.part.41+0x1dc/0x1e8 [ 50.114961] [<ffffffff801c239c>] lock_is_held_type+0x8c/0xb0 [ 50.122291] [<ffffffff809461b8>] __schedule+0x8c0/0x10f8 [ 50.129221] [<ffffffff80946a60>] schedule+0x30/0x98 [ 50.135659] [<ffffffff80106278>] work_resched+0x8/0x34 [ 50.142397] ---[ end trace 0cb4f6ef5b99fe21 ]--- [ 50.148405] possible reason: unannotated irqs-off. [ 50.154600] irq event stamp: 400463 [ 50.159566] hardirqs last enabled at (400463): [<ffffffff8094edc8>] _raw_spin_unlock_irqrestore+0x40/0xa8 [ 50.171981] hardirqs last disabled at (400462): [<ffffffff8094eb98>] _raw_spin_lock_irqsave+0x30/0xb0 [ 50.183897] softirqs last enabled at (400450): [<ffffffff8016580c>] __do_softirq+0x4ac/0x6a8 [ 50.195015] softirqs last disabled at (400425): [<ffffffff80165e78>] irq_exit+0x110/0x128 Fix this by using the TRACE_IRQS_OFF macro to call trace_hardirqs_off() when CONFIG_TRACE_IRQFLAGS is enabled. This is done before invoking schedule() following the work_resched label because: 1) Interrupts are disabled regardless of the path we take to reach work_resched() & schedule(). 2) Performing the tracing here avoids the need to do it in paths which disable interrupts but don't call out to C code before hitting a path which uses the RESTORE_SOME macro that will call trace_hardirqs_on() or trace_hardirqs_off() as appropriate. We call trace_hardirqs_on() using the TRACE_IRQS_ON macro before calling syscall_trace_leave() for similar reasons, ensuring that lockdep has a consistent view of state after we re-enable interrupts. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Cc: linux-mips@linux-mips.org Cc: stable <stable@vger.kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/15385/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-03-04 06:26:05 +07:00
#include <asm/irqflags.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#include <asm/regdef.h>
#include <asm/mipsregs.h>
#include <asm/stackframe.h>
#include <asm/isadep.h>
#include <asm/thread_info.h>
#include <asm/war.h>
[MIPS] ret_from_irq adjustment Make sure that RA on top of interrupt stack is an address of ret_from_irq, so that dump_stack etc. can trace info interrupted context. Also this patch fixes except_vec_vi_handler and __smtc_ipi_vector which seems broken. Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-10-08 23:24:23 +07:00
#ifndef CONFIG_PREEMPT
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#define resume_kernel restore_all
[MIPS] Make entry.S a little more readable. When CONFIG_PREEMPT is not set, it also moves one branch instruction from ret_from_irq() to ret_from_exception(). Therefore we favour the return from irq case which should be more common than the other one. Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com> Acked-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-02-13 20:50:18 +07:00
#else
#define __ret_from_irq ret_from_exception
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#endif
.text
.align 5
[MIPS] Make entry.S a little more readable. When CONFIG_PREEMPT is not set, it also moves one branch instruction from ret_from_irq() to ret_from_exception(). Therefore we favour the return from irq case which should be more common than the other one. Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com> Acked-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-02-13 20:50:18 +07:00
#ifndef CONFIG_PREEMPT
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
FEXPORT(ret_from_exception)
[MIPS] Make entry.S a little more readable. When CONFIG_PREEMPT is not set, it also moves one branch instruction from ret_from_irq() to ret_from_exception(). Therefore we favour the return from irq case which should be more common than the other one. Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com> Acked-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-02-13 20:50:18 +07:00
local_irq_disable # preempt stop
b __ret_from_irq
[MIPS] ret_from_irq adjustment Make sure that RA on top of interrupt stack is an address of ret_from_irq, so that dump_stack etc. can trace info interrupted context. Also this patch fixes except_vec_vi_handler and __smtc_ipi_vector which seems broken. Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-10-08 23:24:23 +07:00
#endif
[MIPS] Make entry.S a little more readable. When CONFIG_PREEMPT is not set, it also moves one branch instruction from ret_from_irq() to ret_from_exception(). Therefore we favour the return from irq case which should be more common than the other one. Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com> Acked-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-02-13 20:50:18 +07:00
FEXPORT(ret_from_irq)
LONG_S s0, TI_REGS($28)
FEXPORT(__ret_from_irq)
MIPS: Fix endless loop when processing signals for kernel tasks The problem occurs [1] when a kernel-mode task returns from a system call with a pending signal. A real-life scenario is a child of 'khelper' returning from a failed kernel_execve() in ____call_usermodehelper() [ kernel/kmod.c ]. kernel_execve() fails due to a pending SIGKILL, which is the result of "kill -9 -1" (at least, busybox's init does it upon reboot). The loop is as follows: * syscall_exit_work: - work_pending: // start_of_the_loop - work_notifysig: - do_notify_resume() - do_signal() - if (!user_mode(regs)) return; - resume_userspace // TIF_SIGPENDING is still set - work_pending // so we call work_pending => goto // start_of_the_loop More information can be found in another LKML thread: http://www.serverphorums.com/read.php?12,457826 [1] The problem was also reproduced on !CONFIG_VM86 x86, and the following fix was accepted. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29a2e2836ff9ea65a603c89df217f4198973a74f Signed-off-by: Dmitry Adamushko <dmitry.adamushko@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/3571/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-04-06 01:37:34 +07:00
/*
* We can be coming here from a syscall done in the kernel space,
* e.g. a failed kernel_execve().
*/
resume_userspace_check:
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
LONG_L t0, PT_STATUS(sp) # returning to kernel mode?
andi t0, t0, KU_USER
beqz t0, resume_kernel
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
resume_userspace:
local_irq_disable # make sure we dont miss an
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
# interrupt setting need_resched
# between sampling and return
LONG_L a2, TI_FLAGS($28) # current->work
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
andi t0, a2, _TIF_WORK_MASK # (ignoring syscall_trace)
bnez t0, work_pending
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
j restore_all
#ifdef CONFIG_PREEMPT
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
resume_kernel:
Use preempt_schedule_irq. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-02-07 09:54:29 +07:00
local_irq_disable
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
lw t0, TI_PRE_COUNT($28)
bnez t0, restore_all
need_resched:
LONG_L t0, TI_FLAGS($28)
andi t1, t0, _TIF_NEED_RESCHED
beqz t1, restore_all
LONG_L t0, PT_STATUS(sp) # Interrupts off?
andi t0, 1
beqz t0, restore_all
Use preempt_schedule_irq. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-02-07 09:54:29 +07:00
jal preempt_schedule_irq
Fix preemption bug. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-03-04 19:35:42 +07:00
b need_resched
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
#endif
mips: switch to generic kernel_thread() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-10-10 03:27:45 +07:00
FEXPORT(ret_from_kernel_thread)
jal schedule_tail # a0 = struct task_struct *prev
move a0, s1
jal s0
mips: switch to saner kernel_execve() semantics Acked-by: Ralf Baechle <ralf@linux-mips.org> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-10-12 03:01:20 +07:00
j syscall_exit
mips: switch to generic kernel_thread() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2012-10-10 03:27:45 +07:00
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
FEXPORT(ret_from_fork)
[PATCH] sched: cleanup, remove task_t, convert to struct task_struct cleanup: remove task_t and convert all the uses to struct task_struct. I introduced it for the scheduler anno and it was a mistake. Conversion was mostly scripted, the result was reviewed and all secondary whitespace and style impact (if any) was fixed up by hand. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-07-03 14:25:41 +07:00
jal schedule_tail # a0 = struct task_struct *prev
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
FEXPORT(syscall_exit)
MIPS: Add syscall detection for restartable sequences Syscalls are not allowed inside restartable sequences, so add a call to rseq_syscall() at the very beginning of the system call exit path when CONFIG_DEBUG_RSEQ=y. This will help us to detect whether there is a syscall issued erroneously inside a restartable sequence. Signed-off-by: Paul Burton <paul.burton@mips.com> Reviewed-by: James Hogan <jhogan@kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/19522/ Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org
2018-06-15 00:20:54 +07:00
#ifdef CONFIG_DEBUG_RSEQ
move a0, sp
jal rseq_syscall
#endif
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
local_irq_disable # make sure need_resched and
# signals dont change between
# sampling and return
LONG_L a2, TI_FLAGS($28) # current->work
li t0, _TIF_ALLWORK_MASK
and t0, a2, t0
bnez t0, syscall_exit_work
MIPS: Merge the identical "return from syscall" per-ABI code No need to keep 4 copies of that stuff; merged and taken to entry.S, unused public symbols there killed off. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-05-06 03:11:35 +07:00
restore_all: # restore full frame
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
.set noat
RESTORE_TEMP
RESTORE_AT
RESTORE_STATIC
MIPS: Merge the identical "return from syscall" per-ABI code No need to keep 4 copies of that stuff; merged and taken to entry.S, unused public symbols there killed off. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-05-06 03:11:35 +07:00
restore_partial: # restore partial frame
[MIPS] TRACE_IRQFLAGS_SUPPORT support. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-07 20:07:18 +07:00
#ifdef CONFIG_TRACE_IRQFLAGS
SAVE_STATIC
SAVE_AT
SAVE_TEMP
LONG_L v0, PT_STATUS(sp)
[MIPS] lockdep: Handle interrupts in R3000 style c0_status register. Check the IEP bit for R3000 style processors when checking to see if interrupts will be reenabled in restore_all. Signed-off-by: Chris Dearman <chris@mips.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2007-03-26 20:47:06 +07:00
#if defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
and v0, ST0_IEP
#else
and v0, ST0_IE
#endif
[MIPS] TRACE_IRQFLAGS_SUPPORT support. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-07-07 20:07:18 +07:00
beqz v0, 1f
jal trace_hardirqs_on
b 2f
1: jal trace_hardirqs_off
2:
RESTORE_TEMP
RESTORE_AT
RESTORE_STATIC
#endif
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
RESTORE_SOME
RESTORE_SP_AND_RET
.set at
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
work_pending:
andi t0, a2, _TIF_NEED_RESCHED # a2 is preloaded with TI_FLAGS
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
beqz t0, work_notifysig
work_resched:
MIPS: Fix IRQ tracing & lockdep when rescheduling When the scheduler sets TIF_NEED_RESCHED & we call into the scheduler from arch/mips/kernel/entry.S we disable interrupts. This is true regardless of whether we reach work_resched from syscall_exit_work, resume_userspace or by looping after calling schedule(). Although we disable interrupts in these paths we don't call trace_hardirqs_off() before calling into C code which may acquire locks, and we therefore leave lockdep with an inconsistent view of whether interrupts are disabled or not when CONFIG_PROVE_LOCKING & CONFIG_DEBUG_LOCKDEP are both enabled. Without tracing this interrupt state lockdep will print warnings such as the following once a task returns from a syscall via syscall_exit_partial with TIF_NEED_RESCHED set: [ 49.927678] ------------[ cut here ]------------ [ 49.934445] WARNING: CPU: 0 PID: 1 at kernel/locking/lockdep.c:3687 check_flags.part.41+0x1dc/0x1e8 [ 49.946031] DEBUG_LOCKS_WARN_ON(current->hardirqs_enabled) [ 49.946355] CPU: 0 PID: 1 Comm: init Not tainted 4.10.0-00439-gc9fd5d362289-dirty #197 [ 49.963505] Stack : 0000000000000000 ffffffff81bb5d6a 0000000000000006 ffffffff801ce9c4 [ 49.974431] 0000000000000000 0000000000000000 0000000000000000 000000000000004a [ 49.985300] ffffffff80b7e487 ffffffff80a24498 a8000000ff160000 ffffffff80ede8b8 [ 49.996194] 0000000000000001 0000000000000000 0000000000000000 0000000077c8030c [ 50.007063] 000000007fd8a510 ffffffff801cd45c 0000000000000000 a8000000ff127c88 [ 50.017945] 0000000000000000 ffffffff801cf928 0000000000000001 ffffffff80a24498 [ 50.028827] 0000000000000000 0000000000000001 0000000000000000 0000000000000000 [ 50.039688] 0000000000000000 a8000000ff127bd0 0000000000000000 ffffffff805509bc [ 50.050575] 00000000140084e0 0000000000000000 0000000000000000 0000000000040a00 [ 50.061448] 0000000000000000 ffffffff8010e1b0 0000000000000000 ffffffff805509bc [ 50.072327] ... [ 50.076087] Call Trace: [ 50.079869] [<ffffffff8010e1b0>] show_stack+0x80/0xa8 [ 50.086577] [<ffffffff805509bc>] dump_stack+0x10c/0x190 [ 50.093498] [<ffffffff8015dde0>] __warn+0xf0/0x108 [ 50.099889] [<ffffffff8015de34>] warn_slowpath_fmt+0x3c/0x48 [ 50.107241] [<ffffffff801c15b4>] check_flags.part.41+0x1dc/0x1e8 [ 50.114961] [<ffffffff801c239c>] lock_is_held_type+0x8c/0xb0 [ 50.122291] [<ffffffff809461b8>] __schedule+0x8c0/0x10f8 [ 50.129221] [<ffffffff80946a60>] schedule+0x30/0x98 [ 50.135659] [<ffffffff80106278>] work_resched+0x8/0x34 [ 50.142397] ---[ end trace 0cb4f6ef5b99fe21 ]--- [ 50.148405] possible reason: unannotated irqs-off. [ 50.154600] irq event stamp: 400463 [ 50.159566] hardirqs last enabled at (400463): [<ffffffff8094edc8>] _raw_spin_unlock_irqrestore+0x40/0xa8 [ 50.171981] hardirqs last disabled at (400462): [<ffffffff8094eb98>] _raw_spin_lock_irqsave+0x30/0xb0 [ 50.183897] softirqs last enabled at (400450): [<ffffffff8016580c>] __do_softirq+0x4ac/0x6a8 [ 50.195015] softirqs last disabled at (400425): [<ffffffff80165e78>] irq_exit+0x110/0x128 Fix this by using the TRACE_IRQS_OFF macro to call trace_hardirqs_off() when CONFIG_TRACE_IRQFLAGS is enabled. This is done before invoking schedule() following the work_resched label because: 1) Interrupts are disabled regardless of the path we take to reach work_resched() & schedule(). 2) Performing the tracing here avoids the need to do it in paths which disable interrupts but don't call out to C code before hitting a path which uses the RESTORE_SOME macro that will call trace_hardirqs_on() or trace_hardirqs_off() as appropriate. We call trace_hardirqs_on() using the TRACE_IRQS_ON macro before calling syscall_trace_leave() for similar reasons, ensuring that lockdep has a consistent view of state after we re-enable interrupts. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Cc: linux-mips@linux-mips.org Cc: stable <stable@vger.kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/15385/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-03-04 06:26:05 +07:00
TRACE_IRQS_OFF
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
jal schedule
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
local_irq_disable # make sure need_resched and
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
# signals dont change between
# sampling and return
LONG_L a2, TI_FLAGS($28)
andi t0, a2, _TIF_WORK_MASK # is there any work to be done
# other than syscall tracing?
beqz t0, restore_all
andi t0, a2, _TIF_NEED_RESCHED
bnez t0, work_resched
work_notifysig: # deal with pending signals and
# notify-resume requests
move a0, sp
li a1, 0
jal do_notify_resume # a2 already loaded
MIPS: Fix endless loop when processing signals for kernel tasks The problem occurs [1] when a kernel-mode task returns from a system call with a pending signal. A real-life scenario is a child of 'khelper' returning from a failed kernel_execve() in ____call_usermodehelper() [ kernel/kmod.c ]. kernel_execve() fails due to a pending SIGKILL, which is the result of "kill -9 -1" (at least, busybox's init does it upon reboot). The loop is as follows: * syscall_exit_work: - work_pending: // start_of_the_loop - work_notifysig: - do_notify_resume() - do_signal() - if (!user_mode(regs)) return; - resume_userspace // TIF_SIGPENDING is still set - work_pending // so we call work_pending => goto // start_of_the_loop More information can be found in another LKML thread: http://www.serverphorums.com/read.php?12,457826 [1] The problem was also reproduced on !CONFIG_VM86 x86, and the following fix was accepted. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29a2e2836ff9ea65a603c89df217f4198973a74f Signed-off-by: Dmitry Adamushko <dmitry.adamushko@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/3571/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-04-06 01:37:34 +07:00
j resume_userspace_check
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
MIPS: Merge the identical "return from syscall" per-ABI code No need to keep 4 copies of that stuff; merged and taken to entry.S, unused public symbols there killed off. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-05-06 03:11:35 +07:00
FEXPORT(syscall_exit_partial)
MIPS: Add syscall detection for restartable sequences Syscalls are not allowed inside restartable sequences, so add a call to rseq_syscall() at the very beginning of the system call exit path when CONFIG_DEBUG_RSEQ=y. This will help us to detect whether there is a syscall issued erroneously inside a restartable sequence. Signed-off-by: Paul Burton <paul.burton@mips.com> Reviewed-by: James Hogan <jhogan@kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/19522/ Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org
2018-06-15 00:20:54 +07:00
#ifdef CONFIG_DEBUG_RSEQ
move a0, sp
jal rseq_syscall
#endif
MIPS: Merge the identical "return from syscall" per-ABI code No need to keep 4 copies of that stuff; merged and taken to entry.S, unused public symbols there killed off. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-05-06 03:11:35 +07:00
local_irq_disable # make sure need_resched doesn't
# change between and return
LONG_L a2, TI_FLAGS($28) # current->work
li t0, _TIF_ALLWORK_MASK
and t0, a2
beqz t0, restore_partial
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
SAVE_STATIC
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
syscall_exit_work:
MIPS: Prevent hitting do_notify_resume() with !user_mode(regs). Too late to do anything there... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2012-05-03 08:45:12 +07:00
LONG_L t0, PT_STATUS(sp) # returning to kernel mode?
andi t0, t0, KU_USER
beqz t0, resume_kernel
MIPS: Use single define for pending work on syscall exit Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-05-19 15:21:28 +07:00
li t0, _TIF_WORK_SYSCALL_EXIT
Remove unused arguments from preempt_{start,stop}/local_irq_{en,dis}able. Don't clobber the preloaded TI_FLAGS in a2 needlessly. Unexport local functions. Signed-off-by: Thiemo Seufer <ths@networkno.de> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2004-12-10 19:56:33 +07:00
and t0, a2 # a2 is preloaded with TI_FLAGS
beqz t0, work_pending # trace bit set?
MIPS: Split do_syscall_trace into two functions. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-05-19 15:21:29 +07:00
local_irq_enable # could let syscall_trace_leave()
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
# call schedule() instead
MIPS: Fix IRQ tracing & lockdep when rescheduling When the scheduler sets TIF_NEED_RESCHED & we call into the scheduler from arch/mips/kernel/entry.S we disable interrupts. This is true regardless of whether we reach work_resched from syscall_exit_work, resume_userspace or by looping after calling schedule(). Although we disable interrupts in these paths we don't call trace_hardirqs_off() before calling into C code which may acquire locks, and we therefore leave lockdep with an inconsistent view of whether interrupts are disabled or not when CONFIG_PROVE_LOCKING & CONFIG_DEBUG_LOCKDEP are both enabled. Without tracing this interrupt state lockdep will print warnings such as the following once a task returns from a syscall via syscall_exit_partial with TIF_NEED_RESCHED set: [ 49.927678] ------------[ cut here ]------------ [ 49.934445] WARNING: CPU: 0 PID: 1 at kernel/locking/lockdep.c:3687 check_flags.part.41+0x1dc/0x1e8 [ 49.946031] DEBUG_LOCKS_WARN_ON(current->hardirqs_enabled) [ 49.946355] CPU: 0 PID: 1 Comm: init Not tainted 4.10.0-00439-gc9fd5d362289-dirty #197 [ 49.963505] Stack : 0000000000000000 ffffffff81bb5d6a 0000000000000006 ffffffff801ce9c4 [ 49.974431] 0000000000000000 0000000000000000 0000000000000000 000000000000004a [ 49.985300] ffffffff80b7e487 ffffffff80a24498 a8000000ff160000 ffffffff80ede8b8 [ 49.996194] 0000000000000001 0000000000000000 0000000000000000 0000000077c8030c [ 50.007063] 000000007fd8a510 ffffffff801cd45c 0000000000000000 a8000000ff127c88 [ 50.017945] 0000000000000000 ffffffff801cf928 0000000000000001 ffffffff80a24498 [ 50.028827] 0000000000000000 0000000000000001 0000000000000000 0000000000000000 [ 50.039688] 0000000000000000 a8000000ff127bd0 0000000000000000 ffffffff805509bc [ 50.050575] 00000000140084e0 0000000000000000 0000000000000000 0000000000040a00 [ 50.061448] 0000000000000000 ffffffff8010e1b0 0000000000000000 ffffffff805509bc [ 50.072327] ... [ 50.076087] Call Trace: [ 50.079869] [<ffffffff8010e1b0>] show_stack+0x80/0xa8 [ 50.086577] [<ffffffff805509bc>] dump_stack+0x10c/0x190 [ 50.093498] [<ffffffff8015dde0>] __warn+0xf0/0x108 [ 50.099889] [<ffffffff8015de34>] warn_slowpath_fmt+0x3c/0x48 [ 50.107241] [<ffffffff801c15b4>] check_flags.part.41+0x1dc/0x1e8 [ 50.114961] [<ffffffff801c239c>] lock_is_held_type+0x8c/0xb0 [ 50.122291] [<ffffffff809461b8>] __schedule+0x8c0/0x10f8 [ 50.129221] [<ffffffff80946a60>] schedule+0x30/0x98 [ 50.135659] [<ffffffff80106278>] work_resched+0x8/0x34 [ 50.142397] ---[ end trace 0cb4f6ef5b99fe21 ]--- [ 50.148405] possible reason: unannotated irqs-off. [ 50.154600] irq event stamp: 400463 [ 50.159566] hardirqs last enabled at (400463): [<ffffffff8094edc8>] _raw_spin_unlock_irqrestore+0x40/0xa8 [ 50.171981] hardirqs last disabled at (400462): [<ffffffff8094eb98>] _raw_spin_lock_irqsave+0x30/0xb0 [ 50.183897] softirqs last enabled at (400450): [<ffffffff8016580c>] __do_softirq+0x4ac/0x6a8 [ 50.195015] softirqs last disabled at (400425): [<ffffffff80165e78>] irq_exit+0x110/0x128 Fix this by using the TRACE_IRQS_OFF macro to call trace_hardirqs_off() when CONFIG_TRACE_IRQFLAGS is enabled. This is done before invoking schedule() following the work_resched label because: 1) Interrupts are disabled regardless of the path we take to reach work_resched() & schedule(). 2) Performing the tracing here avoids the need to do it in paths which disable interrupts but don't call out to C code before hitting a path which uses the RESTORE_SOME macro that will call trace_hardirqs_on() or trace_hardirqs_off() as appropriate. We call trace_hardirqs_on() using the TRACE_IRQS_ON macro before calling syscall_trace_leave() for similar reasons, ensuring that lockdep has a consistent view of state after we re-enable interrupts. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Cc: linux-mips@linux-mips.org Cc: stable <stable@vger.kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/15385/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-03-04 06:26:05 +07:00
TRACE_IRQS_ON
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
move a0, sp
MIPS: Split do_syscall_trace into two functions. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2011-05-19 15:21:29 +07:00
jal syscall_trace_leave
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 05:20:36 +07:00
b resume_userspace
[MIPS] R2: Instruction hazard barrier. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-04-05 15:45:45 +07:00
MIPS: kernel: entry.S: Add MIPS R6 related definitions The instruction hazard barrier in the form of: jr.hb ra nop is valid on MIPS R6 as well. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
2014-11-14 17:05:41 +07:00
#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) || \
defined(CONFIG_MIPS_MT)
[MIPS] R2: Instruction hazard barrier. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-04-05 15:45:45 +07:00
/*
* MIPS32R2 Instruction Hazard Barrier - must be called
*
* For C code use the inline version named instruction_hazard().
*/
LEAF(mips_ihb)
MIPS: kernel: entry.S: Set correct ISA level for mips_ihb Commit 6ebb496ffc7e("MIPS: kernel: entry.S: Add MIPS R6 related definitions") added the MIPSR6 definition but it did not update the ISA level of the actual assembly code so a pre-MIPSR6 jr.hb instruction was generated instead. Fix this by using the MISP_ISA_LEVEL_RAW macro. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Fixes: 6ebb496ffc7e("MIPS: kernel: entry.S: Add MIPS R6 related definitions") Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9386/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-03-04 01:48:49 +07:00
.set MIPS_ISA_LEVEL_RAW
[MIPS] R2: Instruction hazard barrier. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2006-04-05 15:45:45 +07:00
jr.hb ra
nop
END(mips_ihb)
MIPS: kernel: entry.S: Add MIPS R6 related definitions The instruction hazard barrier in the form of: jr.hb ra nop is valid on MIPS R6 as well. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
2014-11-14 17:05:41 +07:00
#endif /* CONFIG_CPU_MIPSR2 or CONFIG_CPU_MIPSR6 or CONFIG_MIPS_MT */
Reference in New Issue Copy Permalink