Peter Zijlstra 4793241be4 sched: backward looking buddy ... Impact: improve/change/fix wakeup-buddy scheduling Currently we only have a forward looking buddy, that is, we prefer to schedule to the task we last woke up, under the presumption that its going to consume the data we just produced, and therefore will have cache hot benefits. This allows co-waking producer/consumer task pairs to run ahead of the pack for a little while, keeping their cache warm. Without this, we would interleave all pairs, utterly trashing the cache. This patch introduces a backward looking buddy, that is, suppose that in the above scenario, the consumer preempts the producer before it can go to sleep, we will therefore miss the wakeup from consumer to producer (its already running, after all), breaking the cycle and reverting to the cache-trashing interleaved schedule pattern. The backward buddy will try to schedule back to the task that woke us up in case the forward buddy is not available, under the assumption that the last task will be the one with the most cache hot task around barring current. This will basically allow a task to continue after it got preempted. In order to avoid starvation, we allow either buddy to get wakeup_gran ahead of the pack. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Acked-by: Mike Galbraith <efault@gmx.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>		2008-11-05 10:30:14 +01:00
..
irq	irq: make variable static	2008-10-22 07:37:17 +02:00
power	PM_TEST_SUSPEND should depend on RTC_CLASS, not RTC_LIB	2008-11-01 12:40:38 -07:00
time	Merge branch 'timers/range-hrtimers' into v28-range-hrtimers-for-linus-v2	2008-10-22 09:48:06 +02:00
trace	tracing, ring-buffer: add paranoid checks for loops	2008-11-03 11:10:04 +01:00
.gitignore	Update kernel/.gitignore with new auto-generated files	2008-02-09 23:27:01 -08:00
acct.c	tty: Fix abusers of current->sighand->tty	2008-10-13 09:51:42 -07:00
audit_tree.c	[PATCH] get rid of nameidata in audit_tree	2008-10-23 05:12:53 -04:00
audit.c	[PATCH] Fix the bug of using AUDIT_STATUS_RATE_LIMIT when set fail, no error output.	2008-08-01 12:15:16 -04:00
audit.h	[PATCH 1/2] audit: move extern declarations to audit.h	2008-04-28 06:28:04 -04:00
auditfilter.c	Re: [PATCH] the loginuid field should be output in all AUDIT_CONFIG_CHANGE audit messages	2008-08-01 12:15:03 -04:00
auditsc.c	tty: Fix abusers of current->sighand->tty	2008-10-13 09:51:42 -07:00
backtracetest.c	backtrace: replace timer with tasklet + completions	2008-06-27 18:09:16 +02:00
bounds.c	Add kbuild.h that contains common definitions for kbuild users	2008-04-29 08:06:29 -07:00
capability.c	security: Fix setting of PF_SUPERPRIV by __capable()	2008-08-14 22:59:43 +10:00
cgroup_debug.c	cgroups: fix probable race with put_css_set[_taskexit] and find_css_set	2008-10-20 08:52:38 -07:00
cgroup_freezer.c	freezer_cg: simplify freezer_change_state()	2008-10-30 11:38:45 -07:00
cgroup.c	cgroup: remove unused variable	2008-10-26 09:38:17 -07:00
compat.c	Merge branches 'timers/clocksource', 'timers/hrtimers', 'timers/nohz', 'timers/ntp', 'timers/posixtimers' and 'timers/debug' into v28-timers-for-linus	2008-10-20 13:14:06 +02:00
configs.c	kernel/configs.c: remove useless comments	2008-10-20 08:52:34 -07:00
cpu.c	Merge branches 'sched/devel', 'sched/cpu-hotplug', 'sched/cpusets' and 'sched/urgent' into sched/core	2008-10-08 11:31:02 +02:00
cpuset.c	cpuset: use seq_*mask_* to print masks	2008-10-20 08:52:39 -07:00
delayacct.c	per-task-delay-accounting: update taskstats for memory reclaim delay	2008-07-25 10:53:47 -07:00
dma-coherent.c	dma-coherent: export dma_[alloc|release]_from_coherent methods	2008-08-22 08:34:53 +02:00
dma.c	kernel/dma.c: remove a CVS keyword	2008-10-16 11:21:30 -07:00
exec_domain.c	proc: move /proc/execdomains to kernel/exec_domain.c	2008-10-23 14:30:41 +04:00
exit.c	Merge branch 'tracing-v28-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2008-10-20 13:35:07 -07:00
extable.c	module: Don't report discarded init pages as kernel text.	2008-01-29 17:13:18 +11:00
fork.c	Merge branch 'timers/range-hrtimers' into v28-range-hrtimers-for-linus-v2	2008-10-22 09:48:06 +02:00
freezer.c	freezer_cg: use thaw_process() in unfreeze_cgroup()	2008-10-30 11:38:45 -07:00
futex_compat.c	futex_compat __user annotation	2008-03-30 14:18:41 -07:00
futex.c	hrtimer: make the futex() system call use the per process slack value	2008-09-11 07:17:00 -07:00
hrtimer.c	Merge branch 'timers/range-hrtimers' into v28-range-hrtimers-for-linus-v2	2008-10-22 09:48:06 +02:00
itimer.c	timers: fix itimer/many thread hang	2008-09-14 16:25:35 +02:00
kallsyms.c	kernel/kallsyms.c: fix double return	2008-10-16 11:21:32 -07:00
Kconfig.freezer	container freezer: implement freezer cgroup subsystem	2008-10-20 08:52:34 -07:00
Kconfig.hz	sched: fix SCHED_HRTICK dependency	2008-07-28 14:37:38 +02:00
Kconfig.preempt	rcu: move PREEMPT_RCU config option back under PREEMPT	2008-03-10 18:01:20 -07:00
kexec.c	kexec: fix crash_save_vmcoreinfo_init build problem	2008-10-20 15:28:50 -07:00
kfifo.c
kgdb.c	kgdb: call touch_softlockup_watchdog on resume	2008-10-06 13:50:59 -05:00
kmod.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux-2.6-for-linus	2008-10-16 12:38:34 -07:00
kprobes.c	make kprobes.c:kretprobe_table_lock() static	2008-10-16 11:21:52 -07:00
ksysfs.c	profiling: dynamically enable readprofile at runtime	2008-10-16 11:21:31 -07:00
kthread.c	Merge branch 'tracing-v28-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2008-10-20 13:35:07 -07:00
latencytop.c	kernel: use non-racy method for proc entries creation	2008-04-29 08:06:22 -07:00
lockdep_internals.h	lockdep: build fix	2008-08-13 12:55:10 +02:00
lockdep_proc.c	lockstat: fix numerical output rounding error	2008-08-26 10:37:46 +02:00
lockdep.c	lockdep: fix irqs on/off ip tracing	2008-10-28 11:19:07 +01:00
Makefile	Merge branch 'tracing/ftrace' into tracing/urgent	2008-10-22 09:08:14 +02:00
marker.c	markers: bit-field is not thread-safe nor smp-safe	2008-10-14 10:38:45 +02:00
module.c	Merge branch 'proc' of git://git.kernel.org/pub/scm/linux/kernel/git/adobriyan/proc	2008-10-23 12:04:37 -07:00
mutex-debug.c	mutex-debug: check mutex magic before owner	2008-05-16 16:53:35 +02:00
mutex-debug.h
mutex.c	locking: fix mutex @key parameter kernel-doc notation	2008-07-28 18:12:36 +02:00
mutex.h
notifier.c	ftrace: ignore functions that cannot be kprobe-ed	2008-10-14 10:34:22 +02:00
ns_cgroup.c	cgroup_clone: use pid of newly created task for new cgroup	2008-07-25 10:53:37 -07:00
nsproxy.c	removed unused #include <linux/version.h>'s	2008-08-23 12:14:12 -07:00
panic.c	Make panic= and panic_on_oops into core_params	2008-10-22 10:00:25 +11:00
params.c	Fix compile warning in kernel/params.c	2008-10-23 12:09:00 -07:00
pid_namespace.c	pid_ns: (BUG 11391) change ->child_reaper when init->group_leader exits	2008-09-02 19:21:38 -07:00
pid.c	pidns: remove now unused find_pid function.	2008-07-25 10:53:45 -07:00
pm_qos_params.c	pm_qos_requirement might sleep	2008-09-02 19:21:40 -07:00
posix-cpu-timers.c	timers: fix itimer/many thread hang, v2	2008-09-23 13:38:44 +02:00
posix-timers.c	Merge branch 'timers/range-hrtimers' into v28-range-hrtimers-for-linus-v2	2008-10-22 09:48:06 +02:00
printk.c	printk: remove unused code from kernel/printk.c	2008-10-23 21:54:29 +02:00
profile.c	kernel/profile: fix profile_init() section mismatch	2008-10-30 11:38:46 -07:00
ptrace.c	make ptrace_untrace() static	2008-10-20 08:52:39 -07:00
rcuclassic.c	rcu: RCU-based detection of stalled CPUs for Classic RCU, fix	2008-10-03 10:41:00 +02:00
rcupdate.c	rcupdate: fix bug of rcu_barrier*()	2008-10-21 15:59:53 +02:00
rcupreempt_trace.c	rcu: trace fix possible mem-leak	2008-08-15 17:54:40 +02:00
rcupreempt.c	byteorder: remove direct includes of linux/byteorder/swab[b].h	2008-10-20 08:52:40 -07:00
rcutorture.c	byteorder: remove direct includes of linux/byteorder/swab[b].h	2008-10-20 12:51:53 -07:00
relay.c	relay: fix "full buffer with exactly full last subbuffer" accounting problem	2008-08-05 14:33:46 -07:00
res_counter.c	cgroup files: convert res_counter_write() to be a cgroups write_string() handler	2008-07-25 10:53:36 -07:00
resource.c	reserve_region_with_split: Fix GFP_KERNEL usage under spinlock	2008-11-01 09:53:58 -07:00
rtmutex_common.h	Don't operate with pid_t in rtmutex tester	2008-02-08 09:22:41 -08:00
rtmutex-debug.c	Don't operate with pid_t in rtmutex tester	2008-02-08 09:22:41 -08:00
rtmutex-debug.h
rtmutex-tester.c	sysdev: Pass the attribute to the low level sysdev show/store function	2008-07-21 21:55:02 -07:00
rtmutex.c	hrtimer: convert kernel/* to the new hrtimer apis	2008-09-05 21:35:13 -07:00
rtmutex.h
rwsem.c	sched: mark rwsem functions as __sched for wchan/profiling	2007-12-18 15:21:13 +01:00
sched_clock.c	sched_clock: prevent scd->clock from moving backwards	2008-10-10 11:17:04 +02:00
sched_cpupri.c	sched: use a 2-d bitmap for searching lowest-pri CPU	2008-06-06 15:19:28 +02:00
sched_cpupri.h	sched: fix the cpuprio count really	2008-06-06 15:19:44 +02:00
sched_debug.c	sched: change sched_debug's mode to 0444	2008-10-30 11:37:57 +01:00
sched_fair.c	sched: backward looking buddy	2008-11-05 10:30:14 +01:00
sched_features.h	sched: backward looking buddy	2008-11-05 10:30:14 +01:00
sched_idletask.c	sched: add CONFIG_SMP consistency	2008-10-22 10:01:52 +02:00
sched_rt.c	Merge commit 'v2.6.28-rc1' into sched/urgent	2008-10-24 12:48:46 +02:00
sched_stats.h	Merge branch 'proc' of git://git.kernel.org/pub/scm/linux/kernel/git/adobriyan/proc	2008-10-23 12:04:37 -07:00
sched.c	sched: backward looking buddy	2008-11-05 10:30:14 +01:00
seccomp.c
semaphore.c	semaphore: __down_common: use signal_pending_state()	2008-08-05 14:33:47 -07:00
signal.c	'kill sig -1' must only apply to caller's namespace	2008-10-30 11:38:46 -07:00
smp.c	smp: have smp_call_function_single() detect invalid CPUs	2008-08-25 17:45:48 -07:00
softirq.c	Merge branches 'timers/clocksource', 'timers/hrtimers', 'timers/nohz', 'timers/ntp', 'timers/posixtimers' and 'timers/debug' into v28-timers-for-linus	2008-10-20 13:14:06 +02:00
softlockup.c	Make the taint flags reliable	2008-10-16 11:21:31 -07:00
spinlock.c	lockdep: spin_lock_nest_lock(), checkpatch fixes	2008-08-13 13:56:51 +02:00
srcu.c	make srcu_readers_active() static	2008-02-06 10:41:02 -08:00
stacktrace.c	stacktrace: fix modular build, export print_stack_trace and save_stack_trace	2008-06-30 09:20:55 +02:00
stop_machine.c	Revert "Call init_workqueues before pre smp initcalls."	2008-10-25 19:53:38 -07:00
sys_ni.c	Configure out AIO support	2008-10-16 11:21:51 -07:00
sys.c	Merge branch 'timers/range-hrtimers' into v28-range-hrtimers-for-linus-v2	2008-10-22 09:48:06 +02:00
sysctl_check.c	sysctl: check for bogus modes	2008-07-25 10:53:45 -07:00
sysctl.c	Merge commit 'v2.6.28-rc2' into tracing/urgent	2008-10-27 10:50:54 +01:00
taskstats.c	taskstats: remove initialization of static per-cpu variable	2008-07-25 10:53:47 -07:00
test_kprobes.c	kprobes: kretprobe user entry-handler	2008-02-06 10:41:11 -08:00
time.c	select: add a timespec_add_safe() function	2008-09-05 21:34:57 -07:00
timeconst.pl	Make constants in kernel/timeconst.h fixed 64 bits	2008-05-02 16:18:42 -07:00
timer.c	Merge branches 'timers/clocksource', 'timers/hrtimers', 'timers/nohz', 'timers/ntp', 'timers/posixtimers' and 'timers/debug' into v28-timers-for-linus	2008-10-20 13:14:06 +02:00
tracepoint.c	tracepoint: check if the probe has been registered	2008-10-27 16:45:46 +01:00
tsacct.c	task IO accounting: move all IO statistics in struct task_io_accounting	2008-07-27 16:12:28 -07:00
uid16.c	asmlinkage_protect replaces prevent_tail_call	2008-04-10 17:28:26 -07:00
user_namespace.c	removed unused #include <linux/version.h>'s	2008-08-23 12:14:12 -07:00
user.c	sched: rt-bandwidth for user grouping interface	2008-08-19 13:10:09 +02:00
utsname_sysctl.c	sysctl: simplify ->strategy	2008-10-16 11:21:47 -07:00
utsname.c	removed unused #include <linux/version.h>'s	2008-08-23 12:14:12 -07:00
wait.c	wait: kill is_sync_wait()	2008-10-16 11:21:31 -07:00
workqueue.c	workqueue: introduce create_rt_workqueue	2008-10-22 10:00:25 +11:00