linux_dsm_epyc7002/kernel
Gregory Haskins 73fe6aae84 sched: add RT-balance cpu-weight
Some RT tasks (particularly kthreads) are bound to one specific CPU.
It is fairly common for two or more bound tasks to get queued up at the
same time.  Consider, for instance, softirq_timer and softirq_sched.  A
timer goes off in an ISR which schedules softirq_thread to run at RT50.
Then the timer handler determines that it's time to smp-rebalance the
system so it schedules softirq_sched to run.  So we are in a situation
where we have two RT50 tasks queued, and the system will go into
rt-overload condition to request other CPUs for help.

This causes two problems in the current code:

1) If a high-priority bound task and a low-priority unbounded task queue
   up behind the running task, we will fail to ever relocate the unbounded
   task because we terminate the search on the first unmovable task.

2) We spend precious futile cycles in the fast-path trying to pull
   overloaded tasks over.  It is therefore optimial to strive to avoid the
   overhead all together if we can cheaply detect the condition before
   overload even occurs.

This patch tries to achieve this optimization by utilizing the hamming
weight of the task->cpus_allowed mask.  A weight of 1 indicates that
the task cannot be migrated.  We will then utilize this information to
skip non-migratable tasks and to eliminate uncessary rebalance attempts.

We introduce a per-rq variable to count the number of migratable tasks
that are currently running.  We only go into overload if we have more
than one rt task, AND at least one of them is migratable.

In addition, we introduce a per-task variable to cache the cpus_allowed
weight, since the hamming calculation is probably relatively expensive.
We only update the cached value when the mask is updated which should be
relatively infrequent, especially compared to scheduling frequency
in the fast path.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-01-25 21:08:07 +01:00
..
irq genirq: revert lazy irq disable for simple irqs 2007-12-18 18:05:58 +01:00
power driver core: make /sys/power a kobject 2008-01-24 20:40:25 -08:00
time Driver core: change sysdev classes to use dynamic kobject names 2008-01-24 20:40:40 -08:00
.gitignore
acct.c acct: real_parent ppid 2008-01-07 14:55:37 -08:00
audit_tree.c [PATCH] audit: watching subtrees 2007-10-21 02:37:45 -04:00
audit.c [PATCH] audit: watching subtrees 2007-10-21 02:37:45 -04:00
audit.h [PATCH] audit: watching subtrees 2007-10-21 02:37:45 -04:00
auditfilter.c [PATCH] audit: watching subtrees 2007-10-21 02:37:45 -04:00
auditsc.c auditsc: fix kernel-doc param warnings 2007-10-22 19:40:02 -07:00
capability.c
cgroup_debug.c
cgroup.c Improve cgroup printks 2007-11-14 18:45:37 -08:00
compat.c
configs.c
cpu.c cpu-hotplug: replace per-subsystem mutexes with get_online_cpus() 2008-01-25 21:08:02 +01:00
cpuset.c cpu-hotplug: replace lock_cpu_hotplug() with get_online_cpus() 2008-01-25 21:08:02 +01:00
delayacct.c
dma.c
exec_domain.c
exit.c wait_task_stopped(): pass correct exit_code to wait_noreap_copyout() 2007-11-29 09:24:55 -08:00
extable.c
fork.c sched: add RT-balance cpu-weight 2008-01-25 21:08:07 +01:00
futex_compat.c [FUTEX] Fix address computation in compat code. 2007-11-09 16:13:08 -08:00
futex.c futex: Prevent stale futex owner when interrupted/timeout 2008-01-08 16:21:39 -08:00
hrtimer.c hrtimer: fix section mismatch 2008-01-21 19:39:41 -08:00
itimer.c
kallsyms.c FRV: fix the extern declaration of kallsyms_num_syms 2007-11-29 09:24:54 -08:00
Kconfig.hz
Kconfig.instrumentation Tiny clean-up of OPROFILE/KPROBES configuration 2007-12-06 09:41:12 -08:00
Kconfig.preempt
kexec.c vmcoreinfo: add the array length of "free_list" for filtering free pages 2008-01-08 16:10:36 -08:00
kfifo.c
kmod.c Fix unbalanced helper_lock in kernel/kmod.c 2008-01-17 15:38:59 -08:00
kprobes.c
ksysfs.c Kobject: convert remaining kobject_unregister() to kobject_put() 2008-01-24 20:40:40 -08:00
kthread.c
latency.c
lockdep_internals.h
lockdep_proc.c
lockdep.c softlockup: automatically detect hung TASK_UNINTERRUPTIBLE tasks 2008-01-25 21:08:02 +01:00
Makefile revert "Task Control Groups: example CPU accounting subsystem" 2007-11-14 18:45:40 -08:00
marker.c Linux Kernel Markers: fix marker mutex not taken upon module load 2007-11-14 18:45:40 -08:00
module.c Kobject: convert remaining kobject_unregister() to kobject_put() 2008-01-24 20:40:40 -08:00
mutex-debug.c
mutex-debug.h
mutex.c
mutex.h
notifier.c
ns_cgroup.c
nsproxy.c
panic.c debug: add end-of-oops marker 2007-12-20 15:01:17 +01:00
params.c Modules: remove unneeded release function 2008-01-24 20:40:39 -08:00
pid.c pidns: Place under CONFIG_EXPERIMENTAL 2007-11-14 18:45:43 -08:00
posix-cpu-timers.c
posix-timers.c
printk.c sched: remove printk_clock() 2008-01-25 21:07:59 +01:00
profile.c sched: document profile=sleep requiring CONFIG_SCHEDSTATS 2007-10-24 18:23:50 +02:00
ptrace.c ptrace: Call arch_ptrace_attach() when request=PTRACE_TRACEME 2008-01-25 08:31:39 +01:00
rcupdate.c rcu: fix section mismatch 2008-01-22 09:17:48 -08:00
rcutorture.c cpu-hotplug: replace lock_cpu_hotplug() with get_online_cpus() 2008-01-25 21:08:02 +01:00
relay.c
resource.c Add IORESOUCE_BUSY flag for System RAM 2007-11-14 18:45:39 -08:00
rtmutex_common.h
rtmutex-debug.c
rtmutex-debug.h
rtmutex-tester.c Driver core: change sysdev classes to use dynamic kobject names 2008-01-24 20:40:40 -08:00
rtmutex.c
rtmutex.h
rwsem.c sched: mark rwsem functions as __sched for wchan/profiling 2007-12-18 15:21:13 +01:00
sched_debug.c sched: fix gcc warnings 2007-12-30 17:24:35 +01:00
sched_fair.c sched: group scheduler, fix fairness of cpu bandwidth allocation for task groups 2008-01-25 21:08:00 +01:00
sched_idletask.c sched: isolate SMP balancing code a bit more 2007-10-24 18:23:51 +02:00
sched_rt.c sched: add RT-balance cpu-weight 2008-01-25 21:08:07 +01:00
sched_stats.h sched: clean up kernel/sched_stat.h 2007-11-28 15:52:56 +01:00
sched.c sched: add RT-balance cpu-weight 2008-01-25 21:08:07 +01:00
seccomp.c
signal.c sigwait eats blocked default-ignore signals 2007-11-12 16:05:23 -08:00
softirq.c
softlockup.c softlockup: automatically detect hung TASK_UNINTERRUPTIBLE tasks 2008-01-25 21:08:02 +01:00
spinlock.c
srcu.c
stacktrace.c
stop_machine.c cpu-hotplug: replace lock_cpu_hotplug() with get_online_cpus() 2008-01-25 21:08:02 +01:00
sys_ni.c [COMPAT]: Fix build on COMPAT platforms when CONFIG_NET is disabled. 2007-10-30 21:29:56 -07:00
sys.c x86: ignore the sys_getcpu() tcache parameter 2007-11-17 16:27:00 +01:00
sysctl_check.c sysctl: fix ax25 checks 2007-12-17 19:28:17 -08:00
sysctl.c softlockup: automatically detect hung TASK_UNINTERRUPTIBLE tasks 2008-01-25 21:08:02 +01:00
taskstats.c kernel/taskstats.c: fix bogus nlmsg_free() 2007-11-14 18:45:44 -08:00
time.c
timer.c timer: fix section mismatch 2008-01-21 19:39:41 -08:00
tsacct.c
uid16.c
user_namespace.c
user.c Kobject: convert kernel/user.c to use kobject_init/add_ng() 2008-01-24 20:40:31 -08:00
utsname_sysctl.c Isolate the UTS namespace's domainname and hostname back 2007-11-29 09:24:53 -08:00
utsname.c
wait.c
workqueue.c cpu-hotplug: replace per-subsystem mutexes with get_online_cpus() 2008-01-25 21:08:02 +01:00