The FPU should not be left enabled after a task context switch. This
isn't usually a problem as the FPU enable bit is updated before
returning to userland, however it can potentially mask kernel bugs, and
in fact KVM assumes it won't happen and won't clear the FPU enable bit
before returning to the guest, which allows the guest to use stale FPU
context.
Interrupts and exceptions save and restore most bits of the CP0 Status
register which contains the FPU enable bit (CU1). When the kernel needs
to enable or disable the FPU (for example due to attempted FPU use by
userland, or the scheduler being invoked) both the actual Status
register and the saved value in the userland context are updated.
However this doesn't work correctly with full kernel preemption enabled,
since the FPU enable bit can be cleared from within an interrupt when
the scheduler is invoked, and only the userland context is updated, not
the interrupt context.
For example:
1) Enter kernel with FPU already enabled, TIF_USEDFPU=1, Status.CU1=1
saved.
2) Take a timer interrupt while in kernel mode, Status.CU1=1 saved.
3) Timer interrupt invokes scheduler to preempt the task, which clears
TIF_USEDFPU, disables the FPU in Status register (Status.CU1=0), and
the value stored in user context from step (1), but not the interrupt
context from step (2).
4) When the process is scheduled back in again Status.CU1=0.
5) The interrupt context from step (2) is restored, which sets
Status.CU1=1. So from user context point of view, preemption has
re-enabled FPU!
6) If the scheduler is invoked again (via preemption or voluntarily)
before returning to userland, TIF_USEDFPU=0 so the FPU is not
disabled before the task context switch.
7) The next task resumes from the context switch with FPU enabled!
The restoring of the Status register on return from interrupt/exception
is already selective about which bits to restore, leaving the interrupt
mask bits alone so enabling/disabling of CPU interrupt lines can
persist. Extend this to also leave both the CU1 bit (FPU enable) and the
FR bit (which specifies the FPU mode and gets changed with CU1). This
prevents a stale Status value being restored in step (5) above and
persisting through subsequent context switches.
Also switch to the use of definitions from asm/mipsregs.h while we're at
it.
Since this change also affects the restoration of Status register on the
path back to userland, it increases the sensitivity of the kernel to the
problem of the FPU being left enabled, allowing it to propagate to
userland, therefore a warning is also added to lose_fpu_inatomic() to
point out any future reoccurances before they do any damage.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12303/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Rather than saving the scalar FP or vector context in the assembly
resume function, reuse the existing C code we have in fpu.h to do
exactly that. This reduces duplication, results in a much easier to read
resume function & should allow the compiler to optimise out more MSA
code due to is_msa_enabled()/cpu_has_msa being known-zero at compile
time for kernels without MSA support.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Cc: Maciej W. Rozycki <macro@linux-mips.org>
Cc: linux-kernel@vger.kernel.org
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Markos Chandras <markos.chandras@imgtec.com>
Cc: Manuel Lauss <manuel.lauss@gmail.com>
Patchwork: https://patchwork.linux-mips.org/patch/10830/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Commit 6134d94923 ("MIPS: asm: fpu: Allow 64-bit FPU on MIPS32 R6")
added support for 64-bit FPU on a 32-bit MIPS R6 processor but it missed
the 64-bit CPU case leading to FPU failures when requesting FR=1 mode
(which is always the case for MIPS R6 userland) when running a 32-bit
kernel on a 64-bit CPU. We also fix the MIPS R2 case.
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Fixes: 6134d94923 ("MIPS: asm: fpu: Allow 64-bit FPU on MIPS32 R6")
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.0+
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10734/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Define the central place the default FCSR value is set from, initialised
in `cpu_probe'. Determine the FCSR mask applied to values written to
the register with CTC1 in the full emulation mode and via ptrace(2),
according to the ISA level of processor hardware or the writability of
bits 31:18 if actual FPU hardware is used.
Software may rely on FCSR bits whose functions our emulator does not
implement, so it should not allow them to be set or software may get
confused. For ptrace(2) it's just sanity.
[ralf@linux-mips.org: Fixed double inclusion of <asm/current.h>.]
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9711/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The expected semantics of __enable_fpu are for the FPU to be enabled
in the given mode if possible, otherwise for the FPU to be left
disabled and SIGFPE returned. The FPU was incorrectly being left
enabled in cases where the desired value for FR was unavailable.
Without ensuring the FPU is disabled in this case, it would be
possible for userland to go on to execute further FP instructions
natively in the incorrect mode, rather than those instructions being
trapped & emulated as they need to be.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9167/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The lose_fpu() function only disables the FPU in CP0_Status.CU1 if the
FPU is in use and MSA isn't enabled.
This isn't necessarily a problem because KSTK_STATUS(current), the
version of CP0_Status stored on the kernel stack on entry from user
mode, does always get updated and gets restored when returning to user
mode, but I don't think it was intended, and it is inconsistent with the
case of only the FPU being in use. Sometimes leaving the FPU enabled may
also mask kernel bugs where FPU operations are executed when the FPU
might not be enabled.
So lets disable the FPU in the MSA case too.
Fixes: 33c771ba5c ("MIPS: save/disable MSA in lose_fpu")
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9323/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Pull MIPS updates from Ralf Baechle:
"This is the main pull request for MIPS:
- a number of fixes that didn't make the 3.19 release.
- a number of cleanups.
- preliminary support for Cavium's Octeon 3 SOCs which feature up to
48 MIPS64 R3 cores with FPU and hardware virtualization.
- support for MIPS R6 processors.
Revision 6 of the MIPS architecture is a major revision of the MIPS
architecture which does away with many of original sins of the
architecture such as branch delay slots. This and other changes in
R6 require major changes throughout the entire MIPS core
architecture code and make up for the lion share of this pull
request.
- finally some preparatory work for eXtendend Physical Address
support, which allows support of up to 40 bit of physical address
space on 32 bit processors"
[ Ahh, MIPS can't leave the PAE brain damage alone. It's like
every CPU architect has to make that mistake, but pee in the snow
by changing the TLA. But whether it's called PAE, LPAE or XPA,
it's horrid crud - Linus ]
* 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (114 commits)
MIPS: sead3: Corrected get_c0_perfcount_int
MIPS: mm: Remove dead macro definitions
MIPS: OCTEON: irq: add CIB and other fixes
MIPS: OCTEON: Don't do acknowledge operations for level triggered irqs.
MIPS: OCTEON: More OCTEONIII support
MIPS: OCTEON: Remove setting of processor specific CVMCTL icache bits.
MIPS: OCTEON: Core-15169 Workaround and general CVMSEG cleanup.
MIPS: OCTEON: Update octeon-model.h code for new SoCs.
MIPS: OCTEON: Implement DCache errata workaround for all CN6XXX
MIPS: OCTEON: Add little-endian support to asm/octeon/octeon.h
MIPS: OCTEON: Implement the core-16057 workaround
MIPS: OCTEON: Delete unused COP2 saving code
MIPS: OCTEON: Use correct instruction to read 64-bit COP0 register
MIPS: OCTEON: Save and restore CP2 SHA3 state
MIPS: OCTEON: Fix FP context save.
MIPS: OCTEON: Save/Restore wider multiply registers in OCTEON III CPUs
MIPS: boot: Provide more uImage options
MIPS: Remove unneeded #ifdef __KERNEL__ from asm/processor.h
MIPS: ip22-gio: Remove legacy suspend/resume support
mips: pci: Add ifdef around pci_proc_domain
...
Commit 4227a2d4ef (MIPS: Support for hybrid
FPRs) changes the kernel to execute read_c0_config5() even on processors
that don't have a Config5 register. According to the arch spec the
behaviour of trying to read or write this register is UNDEFINED where this
register doesn't exist, that is merely looking at this register is
already cruel because that might kill a kitten.
In case of Qemu older than v2.2 Qemu has elected to implement this
UNDEFINED behaviour by taking a RI exception - which then fries the
kernel:
[...]
Freeing YAMON memory: 956k freed
Freeing unused kernel memory: 240K (80674000 - 806b0000)
Reserved instruction in kernel code[#1]:
CPU: 0 PID: 1 Comm: init Not tainted 3.18.0-rc6-00058-g4227a2d #26
task: 86047588 ti: 86048000 task.ti: 86048000
$ 0 : 00000000 77a638cc 00000000 00000000
[...]
For qemu v2.2.0 commit f31b035a9f10dc9b57f01c426110af845d453ce2
(target-mips: correctly handle access to unimplemented CP0 register)
changed the behaviour to returning zero on read and ignoring writes
which more matches how typical hardware implementations actually behave.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Hybrid FPRs is a scheme where scalar FP registers are 64b wide, but
accesses to odd indexed single registers use bits 63:32 of the
preceeding even indexed 64b register. In this mode all FP code
except that built for the plain FP64 ABI can execute correctly. Most
notably a combination of FP64A & FP32 code can execute correctly,
allowing for existing FP32 binaries to be linked with new FP64A binaries
that can make use of 64 bit FP & MSA.
Hybrid FPRs are implemented by setting both the FR & FRE bits, trapping
& emulating single precision FP instructions (via Reserved Instruction
exceptions) whilst allowing others to execute natively. It therefore has
a penalty in terms of execution speed, and should only be used when no
fully native mode can be. As more binaries are recompiled to use either
the FPXX or FP64(A) ABIs, the need for hybrid FPRs should diminish.
However in the short to mid term it allows for a gradual transition
towards that world, rather than a complete ABI break which is not
feasible for some users & not desirable for many.
A task will be executed using the hybrid FPR scheme when its
TIF_HYBRID_FPREGS flag is set & TIF_32BIT_FPREGS is clear. A further
patch will set the flags as necessary, this patch simply adds the
infrastructure necessary for the hybrid FPR mode to work.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/7683/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Starting with version 2.24.51.20140728 MIPS binutils complain loudly
about mixing soft-float and hard-float object files, leading to this
build failure since GCC is invoked with "-msoft-float" on MIPS:
{standard input}: Warning: .gnu_attribute 4,3 requires `softfloat'
LD arch/mips/alchemy/common/built-in.o
mipsel-softfloat-linux-gnu-ld: Warning: arch/mips/alchemy/common/built-in.o
uses -msoft-float (set by arch/mips/alchemy/common/prom.o),
arch/mips/alchemy/common/sleeper.o uses -mhard-float
To fix this, we detect if GAS is new enough to support "-msoft-float" command
option, and if it does, we can let GCC pass it to GAS; but then we also need
to sprinkle the files which make use of floating point registers with the
necessary ".set hardfloat" directives.
Signed-off-by: Manuel Lauss <manuel.lauss@gmail.com>
Cc: Linux-MIPS <linux-mips@linux-mips.org>
Cc: Matthew Fortune <Matthew.Fortune@imgtec.com>
Cc: Markos Chandras <Markos.Chandras@imgtec.com>
Cc: Maciej W. Rozycki <macro@linux-mips.org>
Patchwork: https://patchwork.linux-mips.org/patch/8355/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Preemption must be disabled throughout the process of enabling the FPU,
enabling MSA & initialising the vector registers. Without doing so it
is possible to lose the FPU or MSA whilst initialising them causing
that initialisation to fail.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7307/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The kernel depends upon MSA never being enabled when the FPU is not, a
condition which is currently violated in a few places (whilst saving
sigcontext, following mips_cpu_save). Catch all the problem cases by
disabling MSA in lose_fpu, after saving context if necessary.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7302/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This patch replaces the fpureg_t typedef with a "union fpureg" enabling
easier access to 32 & 64 bit values. This allows the access macros used
in cp1emu.c to be simplified somewhat. It will also make it easier to
expand the width of the FP registers as will be done in a future
patch in order to support the 128 bit registers introduced with MSA.
No behavioural change is intended by this patch.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Qais Yousef <qais.yousef@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/6532/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Commit 597ce1723e ("MIPS: Support for 64-bit FP with O32 binaries")
introduced references to two undefined Kconfig macros. CONFIG_MIPS32_R2
should clearly be replaced with CONFIG_CPU_MIPS32_R2. And CONFIG_MIPS64
should be replaced with CONFIG_64BIT.
Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/6522/
Tested-by: Aaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
__enable_fpu produces a build failure when CONFIG_BUG is not set:
In file included from arch/mips/kernel/cpu-probe.c:24:0:
arch/mips/include/asm/fpu.h: In function '__enable_fpu':
arch/mips/include/asm/fpu.h:77:1: error: control reaches end of non-void function [-Werror=return-type]
This is regression introduced in 3.14-rc1. Fix that.
Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi>
Acked-by: Paul Burton <paul.burton@imgtec.com>
Cc: John Crispin <blogic@openwrt.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/6504/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
CPUs implementing MIPS32 R2 may include a 64-bit FPU, just as MIPS64 CPUs
do. In order to preserve backwards compatibility a 64-bit FPU will act
like a 32-bit FPU (by accessing doubles from the least significant 32
bits of an even-odd pair of FP registers) when the Status.FR bit is
zero, again just like a mips64 CPU. The standard O32 ABI is defined
expecting a 32-bit FPU, however recent toolchains support use of a
64-bit FPU from an O32 MIPS32 executable. When an ELF executable is
built to use a 64-bit FPU a new flag (EF_MIPS_FP64) is set in the ELF
header.
With this patch the kernel will check the EF_MIPS_FP64 flag when
executing an O32 binary, and set Status.FR accordingly. The addition
of O32 64-bit FP support lessens the opportunity for optimisation in
the FPU emulator, so a CONFIG_MIPS_O32_FP64_SUPPORT Kconfig option is
introduced to allow this support to be disabled for those that don't
require it.
Inspired by an earlier patch by Leonid Yegoshin, but implemented more
cleanly & correctly.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Paul Burton <paul.burton@imgtec.com>
Patchwork: https://patchwork.linux-mips.org/patch/6154/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Having received another series of whitespace patches I decided to do this
once and for all rather than dealing with this kind of patches trickling
in forever.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
