linux_dsm_epyc7002/arch/arm64/lib
Will Deacon 223e23e8aa arm64: lib: improve copy_page to deal with 128 bytes at a time
We want to avoid lots of different copy_page implementations, settling
for something that is "good enough" everywhere and hopefully easy to
understand and maintain whilst we're at it.

This patch reworks our copy_page implementation based on discussions
with Cavium on the list and benchmarking on Cortex-A processors so that:

  - The loop is unrolled to copy 128 bytes per iteration

  - The reads are offset so that we read from the next 128-byte block
    in the same iteration that we store the previous block

  - Explicit prefetch instructions are removed for now, since they hurt
    performance on CPUs with hardware prefetching

  - The loop exit condition is calculated at the start of the loop

Signed-off-by: Will Deacon <will.deacon@arm.com>
Tested-by: Andrew Pinski <apinski@cavium.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2016-02-16 15:12:33 +00:00
..
atomic_ll_sc.c arm64: introduce CONFIG_ARM64_LSE_ATOMICS as fallback to ll/sc atomics 2015-07-27 15:28:50 +01:00
bitops.S arm64: atomics: prefetch the destination word for write prior to stxr 2015-07-27 15:28:53 +01:00
clear_page.S arm64: Miscellaneous library functions 2012-09-17 13:42:18 +01:00
clear_user.S arm64: kernel: Add support for Privileged Access Never 2015-07-27 11:08:41 +01:00
copy_from_user.S arm64: copy_to-from-in_user optimization using copy template 2015-10-07 11:34:44 +01:00
copy_in_user.S arm64: copy_to-from-in_user optimization using copy template 2015-10-07 11:34:44 +01:00
copy_page.S arm64: lib: improve copy_page to deal with 128 bytes at a time 2016-02-16 15:12:33 +00:00
copy_template.S arm64: Change memcpy in kernel to use the copy template file 2015-10-07 11:34:43 +01:00
copy_to_user.S arm64: copy_to-from-in_user optimization using copy template 2015-10-07 11:34:44 +01:00
delay.c arm64: Miscellaneous library functions 2012-09-17 13:42:18 +01:00
Makefile arm64: introduce CONFIG_ARM64_LSE_ATOMICS as fallback to ll/sc atomics 2015-07-27 15:28:50 +01:00
memchr.S arm64: use ENDPIPROC() to annotate position independent assembler routines 2015-10-12 16:19:45 +01:00
memcmp.S arm64: use ENDPIPROC() to annotate position independent assembler routines 2015-10-12 16:19:45 +01:00
memcpy.S arm64: add KASAN support 2015-10-12 17:46:36 +01:00
memmove.S arm64: add KASAN support 2015-10-12 17:46:36 +01:00
memset.S arm64: add KASAN support 2015-10-12 17:46:36 +01:00
strchr.S arm64: klib: Optimised string functions 2013-03-21 17:39:30 +00:00
strcmp.S arm64: use ENDPIPROC() to annotate position independent assembler routines 2015-10-12 16:19:45 +01:00
strlen.S arm64: use ENDPIPROC() to annotate position independent assembler routines 2015-10-12 16:19:45 +01:00
strncmp.S arm64: use ENDPIPROC() to annotate position independent assembler routines 2015-10-12 16:19:45 +01:00
strnlen.S arm64: lib: Implement optimized string length routines 2014-05-23 15:17:12 +01:00
strrchr.S arm64: klib: Optimised string functions 2013-03-21 17:39:30 +00:00