linux_dsm_epyc7002/net/sched
Eric Dumazet 46b1c18f9d net: sched: put back q.qlen into a single location
In the series fc8b81a598 ("Merge branch 'lockless-qdisc-series'")
John made the assumption that the data path had no need to read
the qdisc qlen (number of packets in the qdisc).

It is true when pfifo_fast is used as the root qdisc, or as direct MQ/MQPRIO
children.

But pfifo_fast can be used as leaf in class full qdiscs, and existing
logic needs to access the child qlen in an efficient way.

HTB breaks badly, since it uses cl->leaf.q->q.qlen in :
  htb_activate() -> WARN_ON()
  htb_dequeue_tree() to decide if a class can be htb_deactivated
  when it has no more packets.

HFSC, DRR, CBQ, QFQ have similar issues, and some calls to
qdisc_tree_reduce_backlog() also read q.qlen directly.

Using qdisc_qlen_sum() (which iterates over all possible cpus)
in the data path is a non starter.

It seems we have to put back qlen in a central location,
at least for stable kernels.

For all qdisc but pfifo_fast, qlen is guarded by the qdisc lock,
so the existing q.qlen{++|--} are correct.

For 'lockless' qdisc (pfifo_fast so far), we need to use atomic_{inc|dec}()
because the spinlock might be not held (for example from
pfifo_fast_enqueue() and pfifo_fast_dequeue())

This patch adds atomic_qlen (in the same location than qlen)
and renames the following helpers, since we want to express
they can be used without qdisc lock, and that qlen is no longer percpu.

- qdisc_qstats_cpu_qlen_dec -> qdisc_qstats_atomic_qlen_dec()
- qdisc_qstats_cpu_qlen_inc -> qdisc_qstats_atomic_qlen_inc()

Later (net-next) we might revert this patch by tracking all these
qlen uses and replace them by a more efficient method (not having
to access a precise qlen, but an empty/non_empty status that might
be less expensive to maintain/track).

Another possibility is to have a legacy pfifo_fast version that would
be used when used a a child qdisc, since the parent qdisc needs
a spinlock anyway. But then, future lockless qdiscs would also
have the same problem.

Fixes: 7e66016f2c ("net: sched: helpers to sum qlen and qlen for per cpu logic")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: Cong Wang <xiyou.wangcong@gmail.com>
Cc: Jiri Pirko <jiri@resnulli.us>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-02 14:10:18 -08:00
..
act_api.c net/sched: Remove egdev mechanism 2018-12-10 15:54:34 -08:00
act_bpf.c Revert "net: sched: act: add extack for lookup callback" 2018-08-31 22:50:15 -07:00
act_connmark.c net_sched: add missing tcf_lock for act_connmark 2018-08-31 22:57:43 -07:00
act_csum.c Revert "net: sched: act: add extack for lookup callback" 2018-08-31 22:50:15 -07:00
act_gact.c net/sched: act_gact: disallow 'goto chain' on fallback control action 2018-10-22 19:40:55 -07:00
act_ife.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2018-09-04 21:33:03 -07:00
act_ipt.c net/sched: act_ipt: fix refcount leak when replace fails 2019-02-24 17:28:55 -08:00
act_meta_mark.c net: remove duplicate includes 2017-12-13 13:18:46 -05:00
act_meta_skbprio.c net sched actions: change IFE modules alias names 2017-10-12 22:13:20 -07:00
act_meta_skbtcindex.c net: remove duplicate includes 2017-12-13 13:18:46 -05:00
act_mirred.c act_mirred: clear skb->tstamp on redirect 2018-11-11 10:21:31 -08:00
act_nat.c net: sched: act_nat: remove dependency on rtnl lock 2018-09-08 10:18:25 -07:00
act_pedit.c net/sched: act_pedit: fix memory leak when IDR allocation fails 2018-11-16 19:53:45 -08:00
act_police.c net/sched: act_police: fix memory leak in case of invalid control action 2018-11-30 17:14:06 -08:00
act_sample.c Merge ra.kernel.org:/pub/scm/linux/kernel/git/davem/net 2018-09-18 09:33:27 -07:00
act_simple.c Revert "net: sched: act: add extack for lookup callback" 2018-08-31 22:50:15 -07:00
act_skbedit.c net/sched: act_skbedit: fix refcount leak when replace fails 2019-02-24 17:31:43 -08:00
act_skbmod.c Revert "net: sched: act: add extack for lookup callback" 2018-08-31 22:50:15 -07:00
act_tunnel_key.c net: sched: act_tunnel_key: fix NULL pointer dereference during init 2019-02-25 10:13:38 -08:00
act_vlan.c net/core: use __vlan_hwaccel helpers 2018-11-08 20:45:04 -08:00
cls_api.c net_sched: refetch skb protocol for each filter 2019-01-16 13:25:11 -08:00
cls_basic.c sched: fix trailing whitespace 2018-07-24 14:10:42 -07:00
cls_bpf.c net_sched: fold tcf_block_cb_call() into tc_setup_cb_call() 2018-12-14 15:32:19 -08:00
cls_cgroup.c net_sched: switch to rcu_work 2018-05-24 22:56:15 -04:00
cls_flow.c net_sched: switch to rcu_work 2018-05-24 22:56:15 -04:00
cls_flower.c net: cls_flower: Remove filter from mask before freeing it 2019-02-04 09:19:14 -08:00
cls_fw.c net_sched: switch to rcu_work 2018-05-24 22:56:15 -04:00
cls_matchall.c net_sched: fold tcf_block_cb_call() into tc_setup_cb_call() 2018-12-14 15:32:19 -08:00
cls_route.c net_sched: switch to rcu_work 2018-05-24 22:56:15 -04:00
cls_rsvp6.c
cls_rsvp.c
cls_rsvp.h net_sched: switch to rcu_work 2018-05-24 22:56:15 -04:00
cls_tcindex.c net_sched: fix two more memory leaks in cls_tcindex 2019-02-12 14:10:56 -05:00
cls_u32.c net_sched: fold tcf_block_cb_call() into tc_setup_cb_call() 2018-12-14 15:32:19 -08:00
em_canid.c net: sched: remove tcf_proto from ematch calls 2014-10-06 18:02:32 -04:00
em_cmp.c
em_ipset.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
em_ipt.c net: sched: add em_ipt ematch for calling xtables matches 2018-02-21 13:15:33 -05:00
em_meta.c net: convert sock.sk_refcnt from atomic_t to refcount_t 2017-07-01 07:39:08 -07:00
em_nbyte.c net: sched: em_nbyte: don't add the data offset twice 2018-01-24 14:52:40 -05:00
em_text.c net: Remove state argument from skb_find_text() 2015-02-22 15:59:54 -05:00
em_u32.c
ematch.c net: sched: ematch: obtain net pointer from blocks 2017-10-16 21:00:40 +01:00
Kconfig tc: Add support for configuring the taprio scheduler 2018-10-04 13:52:23 -07:00
Makefile tc: Add support for configuring the taprio scheduler 2018-10-04 13:52:23 -07:00
sch_api.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2018-12-27 13:04:52 -08:00
sch_atm.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_blackhole.c net_sched: blackhole: tell upper qdisc about dropped packets 2018-06-17 08:42:33 +09:00
sch_cake.c sch_cake: Correctly update parent qlen when splitting GSO packets 2019-01-15 20:12:01 -08:00
sch_cbq.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_cbs.c sched: Avoid dereferencing skb pointer after child enqueue 2019-01-15 20:12:00 -08:00
sch_choke.c net: sched: sch: add extack for change qdisc ops 2017-12-21 12:32:50 -05:00
sch_codel.c net: sched: sch: add extack for change qdisc ops 2017-12-21 12:32:50 -05:00
sch_drr.c sched: Fix detection of empty queues in child qdiscs 2019-01-15 20:12:00 -08:00
sch_dsmark.c sched: Avoid dereferencing skb pointer after child enqueue 2019-01-15 20:12:00 -08:00
sch_etf.c etf: Drop all expired packets 2018-11-16 20:39:34 -08:00
sch_fifo.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_fq_codel.c net: Add and use skb_mark_not_on_list(). 2018-09-10 10:06:54 -07:00
sch_fq.c net_sched: sch_fq: avoid calling ktime_get_ns() if not needed 2018-11-20 09:51:32 -08:00
sch_generic.c net: sched: put back q.qlen into a single location 2019-03-02 14:10:18 -08:00
sch_gred.c net: sched: gred: support reporting stats from offloads 2018-11-19 18:53:46 -08:00
sch_hfsc.c sched: Fix detection of empty queues in child qdiscs 2019-01-15 20:12:00 -08:00
sch_hhf.c net: Add and use skb_mark_not_on_list(). 2018-09-10 10:06:54 -07:00
sch_htb.c sched: Avoid dereferencing skb pointer after child enqueue 2019-01-15 20:12:00 -08:00
sch_ingress.c net: sched: allow ingress and clsact qdiscs to share filter blocks 2018-01-17 14:53:57 -05:00
sch_mq.c net: sched: mq: offload a graft notification 2018-11-14 08:51:28 -08:00
sch_mqprio.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_multiq.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_netem.c net: netem: fix skb length BUG_ON in __skb_to_sgvec 2019-02-28 10:31:31 -08:00
sch_pie.c net: sched: pie: fix coding style issues 2018-10-07 20:39:01 -07:00
sch_plug.c net: sched: sch: add extack for change qdisc ops 2017-12-21 12:32:50 -05:00
sch_prio.c sched: Avoid dereferencing skb pointer after child enqueue 2019-01-15 20:12:00 -08:00
sch_qfq.c sched: Fix detection of empty queues in child qdiscs 2019-01-15 20:12:00 -08:00
sch_red.c net: sched: red: notify drivers about RED's limit parameter 2018-11-14 08:51:28 -08:00
sch_sfb.c net: sched: rename qdisc_destroy() to qdisc_put() 2018-09-25 20:17:35 -07:00
sch_sfq.c net: sch: api: add extack support in tcf_block_get 2017-12-21 12:32:51 -05:00
sch_skbprio.c net/sched: add skbprio scheduler 2018-07-24 14:44:00 -07:00
sch_taprio.c tc: Add support for configuring the taprio scheduler 2018-10-04 13:52:23 -07:00
sch_tbf.c sched: Avoid dereferencing skb pointer after child enqueue 2019-01-15 20:12:00 -08:00
sch_teql.c net: sched: sch: add extack for init callback 2017-12-21 12:32:50 -05:00