linux_dsm_epyc7002/mm
Jann Horn c8c01da728 mm, slub: consider rest of partial list if acquire_slab() fails
commit 8ff60eb052eeba95cfb3efe16b08c9199f8121cf upstream.

acquire_slab() fails if there is contention on the freelist of the page
(probably because some other CPU is concurrently freeing an object from
the page).  In that case, it might make sense to look for a different page
(since there might be more remote frees to the page from other CPUs, and
we don't want contention on struct page).

However, the current code accidentally stops looking at the partial list
completely in that case.  Especially on kernels without CONFIG_NUMA set,
this means that get_partial() fails and new_slab_objects() falls back to
new_slab(), allocating new pages.  This could lead to an unnecessary
increase in memory fragmentation.

Link: https://lkml.kernel.org/r/20201228130853.1871516-1-jannh@google.com
Fixes: 7ced371971 ("slub: Acquire_slab() avoid loop")
Signed-off-by: Jann Horn <jannh@google.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-01-19 18:27:32 +01:00
..
kasan kasan: fix object remaining in offline per-cpu quarantine 2020-12-11 14:02:14 -08:00
backing-dev.c
balloon_compaction.c
cleancache.c
cma_debug.c
cma.c
cma.h
compaction.c
debug_page_ref.c
debug_vm_pgtable.c
debug.c
dmapool.c
early_ioremap.c
fadvise.c
failslab.c
filemap.c revert "mm/filemap: add static for function __add_to_page_cache_locked" 2020-12-11 14:02:14 -08:00
frame_vector.c
frontswap.c
gup_benchmark.c
gup.c mm/gup: combine put_compound_head() and unpin_user_page() 2020-12-30 11:53:54 +01:00
highmem.c
hmm.c
huge_memory.c mm/rmap: always do TTU_IGNORE_ACCESS 2020-12-30 11:53:55 +01:00
hugetlb_cgroup.c hugetlb_cgroup: fix offline of hugetlb cgroup with reservations 2020-12-06 10:19:07 -08:00
hugetlb.c mm/hugetlb: fix potential missing huge page size info 2021-01-19 18:27:21 +01:00
hwpoison-inject.c
init-mm.c mm/gup: prevent gup_fast from racing with COW during fork 2020-12-30 11:53:54 +01:00
internal.h
interval_tree.c
ioremap.c
Kconfig mm/zsmalloc.c: drop ZSMALLOC_PGTABLE_MAPPING 2020-12-06 10:19:07 -08:00
Kconfig.debug
khugepaged.c
kmemleak.c
ksm.c
list_lru.c
maccess.c
madvise.c mm,memory_failure: always pin the page in madvise_inject_error 2020-12-30 11:53:55 +01:00
Makefile
mapping_dirty_helpers.c
memblock.c
memcontrol.c mm: memcg/slab: fix use after free in obj_cgroup_charge 2020-12-30 11:53:54 +01:00
memfd.c
memory_hotplug.c mm: memmap defer init doesn't work as expected 2021-01-06 14:56:50 +01:00
memory-failure.c mm,memory_failure: always pin the page in madvise_inject_error 2020-12-30 11:53:55 +01:00
memory.c mm/gup: prevent gup_fast from racing with COW during fork 2020-12-30 11:53:54 +01:00
mempolicy.c
mempool.c
memremap.c
memtest.c
migrate.c mm/rmap: always do TTU_IGNORE_ACCESS 2020-12-30 11:53:55 +01:00
mincore.c
mlock.c
mm_init.c
mmap.c mm/mmap.c: fix mmap return value when vma is merged after call_mmap() 2020-12-06 10:19:07 -08:00
mmu_gather.c
mmu_notifier.c
mmzone.c
mprotect.c
mremap.c
msync.c
nommu.c
oom_kill.c
page_alloc.c mm: memmap defer init doesn't work as expected 2021-01-06 14:56:50 +01:00
page_counter.c
page_ext.c
page_idle.c
page_io.c
page_isolation.c
page_owner.c
page_poison.c
page_reporting.c
page_reporting.h
page_vma_mapped.c
page-writeback.c mm: make wait_on_page_writeback() wait for multiple pending writebacks 2021-01-12 20:18:22 +01:00
pagewalk.c
percpu-internal.h
percpu-km.c
percpu-stats.c
percpu-vm.c
percpu.c
pgalloc-track.h
pgtable-generic.c
process_vm_access.c mm/process_vm_access.c: include compat.h 2021-01-19 18:27:21 +01:00
ptdump.c
readahead.c
rmap.c mm/rmap: always do TTU_IGNORE_ACCESS 2020-12-30 11:53:55 +01:00
rodata_test.c
shmem.c
shuffle.c
shuffle.h
slab_common.c
slab.c
slab.h
slob.c
slub.c mm, slub: consider rest of partial list if acquire_slab() fails 2021-01-19 18:27:32 +01:00
sparse-vmemmap.c
sparse.c
swap_cgroup.c
swap_slots.c
swap_state.c
swap.c
swapfile.c mm/swapfile: do not sleep with a spin lock held 2020-12-06 10:19:07 -08:00
truncate.c
usercopy.c
userfaultfd.c
util.c
vmacache.c
vmalloc.c mm/vmalloc.c: fix potential memory leak 2021-01-19 18:27:21 +01:00
vmpressure.c
vmscan.c mm: don't put pinned pages into the swap cache 2021-01-19 18:27:29 +01:00
vmstat.c
workingset.c
z3fold.c z3fold: stricter locking and more careful reclaim 2020-12-30 11:54:10 +01:00
zbud.c
zpool.c
zsmalloc.c mm/zsmalloc.c: drop ZSMALLOC_PGTABLE_MAPPING 2020-12-06 10:19:07 -08:00
zswap.c