linux_dsm_epyc7002/mm
Vladimir Davydov 3f97b16320 list_lru: add helpers to isolate items
Currently, the isolate callback passed to the list_lru_walk family of
functions is supposed to just delete an item from the list upon returning
LRU_REMOVED or LRU_REMOVED_RETRY, while nr_items counter is fixed by
__list_lru_walk_one after the callback returns.  Since the callback is
allowed to drop the lock after removing an item (it has to return
LRU_REMOVED_RETRY then), the nr_items can be less than the actual number
of elements on the list even if we check them under the lock.  This makes
it difficult to move items from one list_lru_one to another, which is
required for per-memcg list_lru reparenting - we can't just splice the
lists, we have to move entries one by one.

This patch therefore introduces helpers that must be used by callback
functions to isolate items instead of raw list_del/list_move.  These are
list_lru_isolate and list_lru_isolate_move.  They not only remove the
entry from the list, but also fix the nr_items counter, making sure
nr_items always reflects the actual number of elements on the list if
checked under the appropriate lock.

Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Dave Chinner <david@fromorbit.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-12 18:54:10 -08:00
..
backing-dev.c
balloon_compaction.c
bootmem.c
cleancache.c mm: fix cleancache debugfs directory path 2015-01-20 14:08:31 +01:00
cma.c mm: cma: fix totalcma_pages to include DT defined CMA regions 2015-02-11 17:06:03 -08:00
compaction.c mm/compaction: add tracepoint to observe behaviour of compaction defer 2015-02-11 17:06:04 -08:00
debug-pagealloc.c
debug.c mm: account pmd page tables to the process 2015-02-11 17:06:04 -08:00
dmapool.c
early_ioremap.c
fadvise.c
failslab.c
filemap_xip.c mm: drop vm_ops->remap_pages and generic_file_remap_pages() stub 2015-02-10 14:30:30 -08:00
filemap.c mm: drop vm_ops->remap_pages and generic_file_remap_pages() stub 2015-02-10 14:30:30 -08:00
frontswap.c
gup.c mm: convert p[te|md]_numa users to p[te|md]_protnone_numa 2015-02-12 18:54:08 -08:00
highmem.c
huge_memory.c mm: numa: avoid unnecessary TLB flushes when setting NUMA hinting entries 2015-02-12 18:54:08 -08:00
hugetlb_cgroup.c mm: page_counter: pull "-1" handling out of page_counter_memparse() 2015-02-11 17:06:02 -08:00
hugetlb.c mm: account pmd page tables to the process 2015-02-11 17:06:04 -08:00
hwpoison-inject.c
init-mm.c
internal.h mm: reduce try_to_compact_pages parameters 2015-02-11 17:06:02 -08:00
interval_tree.c mm: replace vma->sharead.linear with vma->shared 2015-02-10 14:30:31 -08:00
iov_iter.c
Kconfig
Kconfig.debug mm/debug_pagealloc: remove obsolete Kconfig options 2015-01-08 15:10:52 -08:00
kmemcheck.c
kmemleak-test.c
kmemleak.c
ksm.c mm: remove rest usage of VM_NONLINEAR and pte_file() 2015-02-10 14:30:31 -08:00
list_lru.c list_lru: add helpers to isolate items 2015-02-12 18:54:10 -08:00
maccess.c
madvise.c mm: remove rest usage of VM_NONLINEAR and pte_file() 2015-02-10 14:30:31 -08:00
Makefile mm: replace remap_file_pages() syscall with emulation 2015-02-10 14:30:30 -08:00
memblock.c
memcontrol.c memcg: free memcg_caches slot on css offline 2015-02-12 18:54:10 -08:00
memory_hotplug.c
memory-failure.c vmscan: per memory cgroup slab shrinkers 2015-02-12 18:54:09 -08:00
memory.c mm: numa: add paranoid check around pte_protnone_numa 2015-02-12 18:54:08 -08:00
mempolicy.c mm: convert p[te|md]_mknonnuma and remaining page table manipulations 2015-02-12 18:54:08 -08:00
mempool.c
migrate.c mm: convert p[te|md]_mknonnuma and remaining page table manipulations 2015-02-12 18:54:08 -08:00
mincore.c mincore: apply page table walker on do_mincore() 2015-02-11 17:06:06 -08:00
mlock.c
mm_init.c
mmap.c mm/mmap.c: fix arithmetic overflow in __vm_enough_memory() 2015-02-11 17:06:07 -08:00
mmu_context.c
mmu_notifier.c
mmzone.c mm: microoptimize zonelist operations 2015-02-11 17:06:02 -08:00
mprotect.c mm: numa: avoid unnecessary TLB flushes when setting NUMA hinting entries 2015-02-12 18:54:08 -08:00
mremap.c mm: remove rest usage of VM_NONLINEAR and pte_file() 2015-02-10 14:30:31 -08:00
msync.c mm: remove rest usage of VM_NONLINEAR and pte_file() 2015-02-10 14:30:31 -08:00
nobootmem.c
nommu.c mm/nommu.c: fix arithmetic overflow in __vm_enough_memory() 2015-02-11 17:06:07 -08:00
oom_kill.c mm: account pmd page tables to the process 2015-02-11 17:06:04 -08:00
page_alloc.c mm: more aggressive page stealing for UNMOVABLE allocations 2015-02-11 17:06:06 -08:00
page_counter.c mm: page_counter: pull "-1" handling out of page_counter_memparse() 2015-02-11 17:06:02 -08:00
page_ext.c
page_io.c
page_isolation.c
page_owner.c mm/page_owner.c: remove unnecessary stack_trace field 2015-02-11 17:06:07 -08:00
page-writeback.c page_writeback: put account_page_redirty() after set_page_dirty() 2015-02-11 17:06:04 -08:00
pagewalk.c mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP) 2015-02-11 17:06:06 -08:00
percpu-km.c
percpu-vm.c
percpu.c
pgtable-generic.c mm: convert p[te|md]_mknonnuma and remaining page table manipulations 2015-02-12 18:54:08 -08:00
process_vm_access.c mm: gup: use get_user_pages_unlocked 2015-02-11 17:06:05 -08:00
quicklist.c
readahead.c
rmap.c mm: memcontrol: track move_lock state internally 2015-02-11 17:06:00 -08:00
shmem.c swap: remove unused mem_cgroup_uncharge_swapcache declaration 2015-02-11 17:06:00 -08:00
slab_common.c memcg: free memcg_caches slot on css offline 2015-02-12 18:54:10 -08:00
slab.c slab: link memcg caches of the same kind into a list 2015-02-12 18:54:09 -08:00
slab.h slab: link memcg caches of the same kind into a list 2015-02-12 18:54:09 -08:00
slob.c
slub.c slab: link memcg caches of the same kind into a list 2015-02-12 18:54:09 -08:00
sparse-vmemmap.c
sparse.c
swap_cgroup.c
swap_state.c
swap.c rmap: drop support of non-linear mappings 2015-02-10 14:30:31 -08:00
swapfile.c
truncate.c
util.c mm: gup: use get_user_pages_unlocked within get_user_pages_fast 2015-02-11 17:06:05 -08:00
vmacache.c
vmalloc.c
vmpressure.c
vmscan.c vmscan: per memory cgroup slab shrinkers 2015-02-12 18:54:09 -08:00
vmstat.c vmstat: Reduce time interval to stat update on idle cpu 2015-02-11 17:06:07 -08:00
workingset.c list_lru: add helpers to isolate items 2015-02-12 18:54:10 -08:00
zbud.c
zpool.c
zsmalloc.c
zswap.c