linux_dsm_epyc7002/net/sched
Eric Dumazet 55a93b3ea7 qdisc: validate skb without holding lock
Validation of skb can be pretty expensive :

GSO segmentation and/or checksum computations.

We can do this without holding qdisc lock, so that other cpus
can queue additional packets.

Trick is that requeued packets were already validated, so we carry
a boolean so that sch_direct_xmit() can validate a fresh skb list,
or directly use an old one.

Tested on 40Gb NIC (8 TX queues) and 200 concurrent flows, 48 threads
host.

Turning TSO on or off had no effect on throughput, only few more cpu
cycles. Lock contention on qdisc lock disappeared.

Same if disabling TX checksum offload.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-10-03 15:36:11 -07:00
..
act_api.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
act_csum.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_gact.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_ipt.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_mirred.c net_sched: hold tcf_lock in netdevice notifier 2014-07-20 20:31:42 -07:00
act_nat.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_pedit.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_police.c net: sched: make bstats per cpu and estimator RCU safe 2014-09-30 01:02:26 -04:00
act_simple.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_skbedit.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
cls_api.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_basic.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_bpf.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_cgroup.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_flow.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_fw.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_route.c net_sched: remove the first parameter from tcf_exts_destroy() 2014-09-28 17:29:01 -04:00
cls_rsvp6.c [NET_SCHED]: Remove unnecessary includes 2007-07-10 22:16:41 -07:00
cls_rsvp.c [NET_SCHED]: Remove unnecessary includes 2007-07-10 22:16:41 -07:00
cls_rsvp.h net: sched: cls_rcvp, complete rcu conversion 2014-09-29 00:04:55 -04:00
cls_tcindex.c net_sched: fix another crash in cls_tcindex 2014-10-01 22:00:42 -04:00
cls_u32.c net_sched: avoid calling tcf_unbind_filter() in call_rcu callback 2014-10-01 22:00:42 -04:00
em_canid.c net: em_canid: remove useless statements from em_canid_change 2014-06-21 15:40:22 -07:00
em_cmp.c net_sched: cleanups 2011-01-19 23:31:12 -08:00
em_ipset.c em_ipset: use dev_net() accessor 2013-10-18 16:23:06 -04:00
em_meta.c net: Change skb_get_rxhash to skb_get_hash 2013-12-17 16:36:21 -05:00
em_nbyte.c net_sched: cleanups 2011-01-19 23:31:12 -08:00
em_text.c net_sched: cleanups 2011-01-19 23:31:12 -08:00
em_u32.c net_sched: cleanups 2011-01-19 23:31:12 -08:00
ematch.c ematch: Fix matching of inverted containers. 2014-09-29 15:31:29 -04:00
Kconfig net: pkt_sched: PIE AQM scheme 2014-01-06 15:13:01 -05:00
Makefile net: pkt_sched: PIE AQM scheme 2014-01-06 15:13:01 -05:00
sch_api.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_atm.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_blackhole.c pkt_sched: Add qdisc->ops->peek() implementation. 2008-10-31 00:45:55 -07:00
sch_cbq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_choke.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_codel.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_drr.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_dsmark.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_fifo.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_fq_codel.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_fq.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_generic.c qdisc: validate skb without holding lock 2014-10-03 15:36:11 -07:00
sch_gred.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_hfsc.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_hhf.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_htb.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_ingress.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_mq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_mqprio.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_multiq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_netem.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_pie.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_plug.c net_sched: sch_plug: plug_qdisc_ops is static 2012-02-13 16:04:40 -05:00
sch_prio.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_qfq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_red.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_sfb.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_sfq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_tbf.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_teql.c net: qdisc: use rcu prefix and silence sparse warnings 2014-09-13 12:30:25 -04:00