linux_dsm_epyc7002/arch/powerpc/mm
Mathieu Desnoyers 3ccfebedd8 powerpc, membarrier: Skip memory barrier in switch_mm()
Allow PowerPC to skip the full memory barrier in switch_mm(), and
only issue the barrier when scheduling into a task belonging to a
process that has registered to use expedited private.

Threads targeting the same VM but which belong to different thread
groups is a tricky case. It has a few consequences:

It turns out that we cannot rely on get_nr_threads(p) to count the
number of threads using a VM. We can use
(atomic_read(&mm->mm_users) == 1 && get_nr_threads(p) == 1)
instead to skip the synchronize_sched() for cases where the VM only has
a single user, and that user only has a single thread.

It also turns out that we cannot use for_each_thread() to set
thread flags in all threads using a VM, as it only iterates on the
thread group.

Therefore, test the membarrier state variable directly rather than
relying on thread flags. This means
membarrier_register_private_expedited() needs to set the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag, issue synchronize_sched(), and
only then set MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY which allows
private expedited membarrier commands to succeed.
membarrier_arch_switch_mm() now tests for the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Andrea Parri <parri.andrea@gmail.com>
Cc: Andrew Hunter <ahh@google.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Avi Kivity <avi@scylladb.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Dave Watson <davejwatson@fb.com>
Cc: David Sehr <sehr@google.com>
Cc: Greg Hackmann <ghackmann@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Maged Michael <maged.michael@gmail.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/20180129202020.8515-3-mathieu.desnoyers@efficios.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-02-05 21:34:02 +01:00
..
8xx_mmu.c powerpc/8xx: mark init functions with __init 2017-08-15 22:55:54 +10:00
40x_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
44x_mmu.c
copro_fault.c powerpc/mm: Update PROTFAULT handling in the page fault path 2017-02-15 20:02:39 +11:00
dma-noncoherent.c powerpc/mm: Rename map_page() to map_kernel_page() on 32-bit 2017-06-05 19:59:03 +10:00
dump_hashpagetable.c powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 2017-11-06 16:48:14 +11:00
dump_linuxpagetables.c powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 2017-11-06 16:48:14 +11:00
fault.c powerpc/mm: Fix SEGV on mapped region to return SEGV_ACCERR 2018-01-02 21:12:33 +11:00
fsl_booke_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
hash64_4k.c powerpc/mm: Fix lazy icache flush on pre-POWER5 2016-11-29 23:59:40 +11:00
hash64_64k.c powerpc/mm: Fix lazy icache flush on pre-POWER5 2016-11-29 23:59:40 +11:00
hash_low_32.S powerpc: fix location of two EXPORT_SYMBOL 2017-09-01 16:42:45 +10:00
hash_native_64.c powerpc/powernv: Fix kexec crashes caused by tlbie tracing 2017-11-23 23:10:14 +11:00
hash_utils_64.c powerpc/mm/hash: Add pr_fmt() to hash_utils64.c 2017-11-06 16:48:13 +11:00
highmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugepage-hash64.c powerpc/mm: Move hash table ops to a separate structure 2016-07-21 18:59:09 +10:00
hugetlbpage-book3e.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbpage-hash64.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbpage-radix.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
hugetlbpage.c mm: account pud page tables 2017-11-15 18:21:04 -08:00
init_32.c powerpc/mm/hugetlb: Add support for reserving gigantic huge pages via kernel command line 2017-08-16 14:56:12 +10:00
init_64.c powerpc/mm: Add a CONFIG option to choose if radix is used by default 2017-11-06 16:48:15 +11:00
init-common.c Merge branch 'topic/ppc-kvm' into next 2017-02-14 17:18:29 +11:00
Makefile powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
mem.c powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
mmap.c powerpc/64s: mm_context.addr_limit is only used on hash 2017-11-13 23:35:43 +11:00
mmu_context_book3s64.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
mmu_context_hash32.c
mmu_context_iommu.c Merge branch 'topic/ppc-kvm' into next 2017-04-28 20:19:37 +10:00
mmu_context_nohash.c powerpc/mm/nohash: MM_SLICE is only used by book3s 64 2017-03-31 23:09:47 +11:00
mmu_context.c powerpc, membarrier: Skip memory barrier in switch_mm() 2018-02-05 21:34:02 +01:00
mmu_decl.h powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
numa.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
pgtable_32.c powerpc/mm: Call flush_tlb_kernel_range with interrupts enabled 2017-10-04 22:15:30 +11:00
pgtable_64.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
pgtable-book3e.c powerpc/mm: Make page table size a variable 2016-05-01 18:32:48 +10:00
pgtable-book3s64.c powerpc/mm/cxl: Add the fault handling cpu to mm cpumask 2017-08-17 23:31:52 +10:00
pgtable-hash64.c powerpc/mm: Use mm_is_thread_local() instread of open-coding 2017-08-23 22:27:45 +10:00
pgtable-radix.c powerpc/mm/radix: Fix crashes on Power9 DD1 with radix MMU and STRICT_RWX 2017-11-12 23:25:48 +11:00
pgtable.c powerpc/mm: Fix typo in set_pte_at() 2017-02-17 22:16:25 +11:00
ppc_mmu_32.c powerpc32: refactor x_mapped_by_bats() and x_mapped_by_tlbcam() together 2016-03-11 17:18:02 -06:00
slb_low.S powerpc/64s: mm_context.addr_limit is only used on hash 2017-11-13 23:35:43 +11:00
slb.c powerpc/64s: Rename slb_allocate_realmode() to slb_allocate() 2017-06-21 16:18:33 +10:00
slice.c powerpc/64s/slice: Use addr limit when computing slice mask 2017-11-20 19:28:25 +11:00
subpage-prot.c powerpc/mm/hash: Free the subpage_prot_table correctly 2017-07-27 13:05:50 +10:00
tlb_hash32.c powerpc/mm: remove flush_tlb_page_nohash 2016-08-01 11:15:13 +10:00
tlb_hash64.c powerpc/mm: Use mm_is_thread_local() instread of open-coding 2017-08-23 22:27:45 +10:00
tlb_low_64e.S
tlb_nohash_low.S powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
tlb_nohash.c powerpc/nohash: Fix use of mmu_has_feature() in setup_initial_memory_limit() 2017-04-11 07:46:04 +10:00
tlb-radix.c powerpc/64s/radix: Improve TLB flushing for page table freeing 2017-11-10 21:33:35 +11:00
vphn.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vphn.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00