linux_dsm_epyc7002/include/net
Magnus Karlsson 9ad0375ed2 xsk: Fix race in SKB mode transmit with shared cq
commit f09ced4053bc0a2094a12b60b646114c966ef4c6 upstream.

Fix a race when multiple sockets are simultaneously calling sendto()
when the completion ring is shared in the SKB case. This is the case
when you share the same netdev and queue id through the
XDP_SHARED_UMEM bind flag. The problem is that multiple processes can
be in xsk_generic_xmit() and call the backpressure mechanism in
xskq_prod_reserve(xs->pool->cq). As this is a shared resource in this
specific scenario, a race might occur since the rings are
single-producer single-consumer.

Fix this by moving the tx_completion_lock from the socket to the pool
as the pool is shared between the sockets that share the completion
ring. (The pool is not shared when this is not the case.) And then
protect the accesses to xskq_prod_reserve() with this lock. The
tx_completion_lock is renamed cq_lock to better reflect that it
protects accesses to the potentially shared completion ring.

Fixes: 35fcde7f8d ("xsk: support for Tx")
Reported-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Björn Töpel <bjorn.topel@intel.com>
Link: https://lore.kernel.org/bpf/20201218134525.13119-2-magnus.karlsson@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-01-17 14:17:05 +01:00
..
9p net: 9p: drop duplicate word in comment 2020-07-15 20:34:11 -07:00
bluetooth Bluetooth: L2CAP: Fix calling sk_filter on non-socket based channel 2020-09-25 20:21:55 +02:00
caif net: caif: Remove unused caif SPI driver 2020-09-29 14:02:53 -07:00
iucv net/af_iucv: clean up function prototypes 2020-05-19 12:50:14 -07:00
netfilter netfilter: nft_dynset: fix timeouts later than 23 days 2020-12-08 20:42:11 +01:00
netns can: remove obsolete version strings 2020-10-12 10:06:39 +02:00
nfc NFC: Replace zero-length array with flexible-array member 2020-02-27 12:06:20 -08:00
phonet treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 336 2019-06-05 17:37:07 +02:00
sctp net: sctp: Fix IPv6 ancestor_size calc in sctp_copy_descendant 2020-09-20 14:15:12 -07:00
tc_act Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-10-15 12:43:21 -07:00
6lowpan.h 6lowpan: Replace zero-length array with flexible-array member 2020-02-28 14:51:30 +01:00
act_api.h net_sched: defer tcf_idr_insert() in tcf_action_init_1() 2020-09-24 19:46:21 -07:00
addrconf.h ipv6: some fixes for ipv6_dev_find() 2020-08-18 15:58:53 -07:00
af_ieee802154.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
af_rxrpc.h rxrpc: Make rxrpc_kernel_get_srtt() indicate validity 2020-08-20 18:21:28 +01:00
af_unix.h unix: uses an atomic type for scm files accounting 2020-02-28 12:12:53 -08:00
af_vsock.h vsock: add local transport support in the vsock core 2019-12-11 15:01:23 -08:00
ah.h
arp.h net: avoid potential false sharing in neighbor related code 2019-11-06 16:14:48 -08:00
atmclip.h
ax25.h ax25: fix possible use-after-free 2019-01-23 11:18:00 -08:00
ax88796.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
bareudp.h bareudp: Reverted support to enable & disable rx metadata collection 2020-07-21 18:30:47 -07:00
bond_3ad.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 90 2019-05-24 17:37:53 +02:00
bond_alb.h bonding/alb: Add helper functions to get the xmit slave 2020-05-01 12:15:37 -07:00
bond_options.h bonding: add an option to specify a delay between peer notifications 2019-07-04 12:30:48 -07:00
bonding.h bonding: fix feature flag setting at init time 2020-12-08 11:26:08 -08:00
bpf_sk_storage.h bpf: Change bpf_sk_storage_*() to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON 2020-09-25 13:58:01 -07:00
busy_poll.h net: Avoid overwriting valid skb->napi_id 2020-06-20 17:30:59 -07:00
calipso.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13 2019-05-21 11:28:45 +02:00
cfg80211-wext.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cfg80211.h mac80211: fix kernel-doc markups 2020-10-30 10:06:09 +01:00
cfg802154.h cfg802154: Replace zero-length array with flexible-array member 2020-02-29 14:39:08 +01:00
checksum.h saner calling conventions for csum_and_copy_..._user() 2020-08-20 15:45:15 -04:00
cipso_ipv4.h cipso: Remove unused inline functions 2020-07-15 07:45:24 -07:00
cls_cgroup.h bpf: Allow to retrieve cgroup v1 classid from v2 hooks 2020-03-27 19:40:38 -07:00
codel_impl.h
codel_qdisc.h
codel.h
compat.h net: simplify cBPF setsockopt compat handling 2020-07-19 18:16:40 -07:00
datalink.h
dcbevent.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 201 2019-05-30 11:29:52 -07:00
dcbnl.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 201 2019-05-30 11:29:52 -07:00
devlink.h devlink: Add enable_remote_dev_reset generic parameter 2020-10-09 12:06:53 -07:00
dn_dev.h
dn_fib.h net: dn_fib: Replace zero-length array with flexible-array member 2020-02-29 21:52:20 -08:00
dn_neigh.h
dn_nsp.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 24 2019-05-21 11:52:39 +02:00
dn_route.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 24 2019-05-21 11:52:39 +02:00
dn.h
dsa.h net: dsa: propagate switchdev vlan_filtering prepare phase to drivers 2020-10-05 05:56:48 -07:00
dsfield.h ipv6: Annotate bitwise IPv6 dsfield pointer cast 2019-12-16 16:09:44 -08:00
dst_cache.h
dst_metadata.h
dst_ops.h net/dst: use a smaller percpu_counter batch for dst entries accounting 2020-05-08 21:33:33 -07:00
dst.h net: clean up codestyle 2020-08-31 12:33:34 -07:00
erspan.h erspan: Add type I version 0 support. 2020-05-05 13:23:29 -07:00
esp.h ESP: Export esp_output_fill_trailer function 2020-02-19 13:52:32 +01:00
espintcp.h xfrm: espintcp: save and call old ->sk_destruct 2020-04-20 07:34:16 +02:00
ethoc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
failover.h
fib_notifier.h ipv6: Remove old route notifications and convert listeners 2019-12-24 22:37:30 -08:00
fib_rules.h fib: use indirect call wrappers in the most common fib_rules_ops 2020-07-28 17:42:31 -07:00
firewire.h
flow_dissector.h net/flow_dissector: add packet hash dissection 2020-07-24 15:23:31 -07:00
flow_offload.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next 2020-08-05 20:13:21 -07:00
flow.h ipv4: Initialize flowi4_multipath_hash in data path 2020-09-14 14:54:56 -07:00
fou.h
fq_impl.h net/fq_impl: use skb_get_hash instead of skb_get_hash_perturb 2020-07-31 09:24:24 +02:00
fq.h net/fq_impl: use skb_get_hash instead of skb_get_hash_perturb 2020-07-31 09:24:24 +02:00
garp.h treewide: Use sizeof_field() macro 2019-12-09 10:36:44 -08:00
gen_stats.h net_sched: extend packet counter to 64bit 2019-11-05 18:20:55 -08:00
genetlink.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-10-05 18:40:01 -07:00
geneve.h net: Move the definition of the default Geneve udp port to public header file 2019-03-22 12:09:31 -07:00
gre.h net: Add netif_is_gretap()/netif_is_ip6gretap() 2018-12-10 15:53:04 -08:00
gro_cells.h
gtp.h
gue.h GUE: Fix a typo 2020-06-22 21:12:44 -07:00
hwbm.h net: hwbm: if CONFIG_NET_HWBM unset, make stub functions static 2019-10-25 16:24:32 -07:00
icmp.h icmp: introduce helper for nat'd source address in network device context 2020-02-13 14:19:00 -08:00
ieee80211_radiotap.h mac80211: add radiotap flag to prevent sequence number overwrite 2020-07-31 09:27:00 +02:00
ieee802154_netdev.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
if_inet6.h ipv6: Replace zero-length array with flexible-array 2020-05-11 13:18:54 -07:00
ife.h net: ife: drop include of module.h from net/ife.h 2019-04-22 21:50:53 -07:00
ila.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
inet6_connection_sock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
inet6_hashtables.h net: Track socket refcounts in skb_steal_sock() 2020-03-30 13:45:04 -07:00
inet_common.h bpf: Allow any port in bpf_bind helper 2020-05-09 00:48:20 +02:00
inet_connection_sock.h tcp: add exponential backoff in __tcp_send_ack() 2020-09-30 14:21:30 -07:00
inet_ecn.h inet_ecn: Fix endianness of checksum update when setting ECT(1) 2020-12-01 17:16:54 -08:00
inet_frag.h inet: frags: re-introduce skb coalescing for local delivery 2019-08-08 15:55:10 -07:00
inet_hashtables.h tcp: fix race condition when creating child sockets from syncookies 2020-11-23 16:32:33 -08:00
inet_sock.h inet: remove inet_sk_copy_descendant() 2020-08-26 07:33:19 -07:00
inet_timewait_sock.h tcp: honor SO_PRIORITY in TIME_WAIT state 2019-09-27 12:05:02 +02:00
inetpeer.h net: ipv4: use a dedicated counter for icmp_v4 redirect packets 2019-02-08 21:50:15 -08:00
ip6_checksum.h tcp: remove indirect calls for icsk->icsk_af_ops->send_check 2020-06-20 17:47:53 -07:00
ip6_fib.h net: ip6_fib.h: drop duplicate word in comment 2020-07-15 20:34:11 -07:00
ip6_route.h ipv6: lift copy_from_user out of ipv6_route_ioctl 2020-05-18 17:35:02 -07:00
ip6_tunnel.h ip6_tunnel: allow not to count pkts on tstats by passing dev as NULL 2019-06-18 20:48:45 -04:00
ip_fib.h ipv4: nexthop version of fib_info_nh_uses_dev 2020-05-26 16:06:07 -07:00
ip_tunnels.h ip_tunnels: Set tunnel option flag when tunnel metadata is present 2020-11-13 16:58:10 -08:00
ip_vs.h ipvs: remove dependency on ip6_tables 2020-08-31 23:06:51 +02:00
ip.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-10-05 18:40:01 -07:00
ipcomp.h
ipconfig.h
ipv6_frag.h ipv6: Remove dependency of ipv6_frag_thdr_truncated on ipv6 module 2020-11-19 10:49:50 -08:00
ipv6_stubs.h ipv6: add ipv6_fragment hook in ipv6_stub 2020-08-31 12:26:39 -07:00
ipv6.h ipv6: Remove dependency of ipv6_frag_thdr_truncated on ipv6 module 2020-11-19 10:49:50 -08:00
ipx.h bonding/alb: properly access headers in bond_alb_xmit() 2020-02-05 14:28:09 +01:00
iw_handler.h
kcm.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
l3mdev.h l3mdev: add infrastructure for table to VRF mapping 2020-06-20 17:22:22 -07:00
lag.h
lapb.h
lib80211.h
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h llc: fix sk_buff leak in llc_conn_service() 2019-10-08 13:23:05 -07:00
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h
lwtunnel.h net: add net available in build_state 2020-03-29 22:30:57 -07:00
mac80211.h mac80211: fix kernel-doc markups 2020-10-30 10:06:09 +01:00
mac802154.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
macsec.h net: macsec: add support for getting offloaded stats 2020-03-26 20:17:36 -07:00
mip6.h net: mip6: Replace zero-length array with flexible-array member 2020-03-02 11:16:27 -08:00
mld.h net: ipv6: mld: Replace zero-length array with flexible-array member 2020-02-29 21:52:20 -08:00
mpls_iptunnel.h net: mpls: Replace zero-length array with flexible-array member 2020-02-28 12:08:37 -08:00
mpls.h net: Make mpls_entry_encode() available for generic users 2020-05-29 21:20:20 -07:00
mptcp.h net: tcp: drop unused function argument from mptcp_incoming_options 2020-09-24 20:17:01 -07:00
mrp.h treewide: Use sizeof_field() macro 2019-12-09 10:36:44 -08:00
ncsi.h
ndisc.h ipv6: ndisc: adjust ndisc_ifinfo_sysctl_change prototype 2020-08-24 06:40:07 -07:00
neighbour.h net: Exempt multicast addresses from five-second neighbor lifetime 2020-11-13 14:24:39 -08:00
net_failover.h
net_namespace.h bpf, net: Rework cookie generator as per-cpu one 2020-09-30 11:50:35 -07:00
net_ratelimit.h
netevent.h
netlabel.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13 2019-05-21 11:28:45 +02:00
netlink.h netlink: export policy in extended ACK 2020-10-09 20:22:32 -07:00
netprio_cgroup.h netprio: use css ID instead of cgroup ID 2019-11-12 08:18:03 -08:00
netrom.h net: netrom: Fix error cleanup path of nr_proto_init 2019-04-11 13:59:49 -07:00
nexthop.h nexthop: Remove NEXTHOP_EVENT_ADD 2020-09-15 16:31:11 -07:00
nl802154.h
nsh.h
p8022.h
page_pool.h net: page_pool: API cleanup and comments 2020-02-20 10:09:25 -08:00
pie.h pie: realign comment 2020-03-04 13:25:55 -08:00
ping.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
pkt_cls.h net: sched: Do not drop root lock in tcf_qevent_handle() 2020-07-16 16:48:34 -07:00
pkt_sched.h net/sched: get rid of qdisc->padded 2020-10-09 08:08:08 -07:00
pptp.h
protocol.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
psample.h net: sched: take reference to psample group in flow_action infra 2019-09-16 09:18:03 +02:00
psnap.h
raw.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
rawv6.h
red.h net: sched: prevent invalid Scell_log shift count 2021-01-12 20:18:12 +01:00
regulatory.h net/wireless: regulatory.h: drop duplicate word in comment 2020-07-31 09:24:23 +02:00
request_sock.h tcp: bpf: Optionally store mac header in TCP_SAVE_SYN 2020-08-24 14:35:00 -07:00
rose.h
route.h Remove DST_HOST 2020-03-23 21:57:44 -07:00
rpl.h net: ipv6: Use struct_size() helper and kcalloc() 2020-06-23 20:27:09 -07:00
rsi_91x.h
rtnetlink.h
rtnh.h net: Rename net/nexthop.h net/rtnh.h 2019-04-22 21:47:25 -07:00
sch_generic.h net/sched: get rid of qdisc->padded 2020-10-09 08:08:08 -07:00
scm.h fs: Move __scm_install_fd() to __receive_fd() 2020-07-13 11:03:44 -07:00
secure_seq.h
seg6_hmac.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
seg6_local.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
seg6.h seg6: fix seg6_validate_srh() to avoid slab-out-of-bounds 2020-06-04 15:39:32 -07:00
slhc_vj.h
smc.h net/smc: introduce CHID callback for ISM devices 2020-09-28 15:19:03 -07:00
snmp.h net/tls: add skeleton of MIB statistics 2019-10-05 16:29:00 -07:00
sock_reuseport.h net: sock_reuseport: Replace zero-length array with flexible-array member 2020-02-29 21:52:19 -08:00
sock.h mptcp: add sk_stop_timer_sync helper 2020-09-24 19:58:34 -07:00
Space.h
stp.h
strparser.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
switchdev.h bridge: Add SWITCHDEV_FDB_FLUSH_TO_BRIDGE notifier 2020-09-15 13:21:47 -07:00
tcp_states.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
tcp.h bpf: tcp: Do not limit cb_flags when creating child sk from listen sk 2020-10-02 11:34:48 -07:00
timewait_sock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
tipc.h
tls_toe.h net/tls: rename tls_hw_* functions tls_toe_* 2019-10-04 14:07:07 -07:00
tls.h net/tls: Protect from calling tls_dev_del for TLS RX twice 2020-11-25 17:31:06 -08:00
transp_v6.h tcp: move ipv4_specific to tcp include file 2020-06-23 20:10:15 -07:00
tso.h net: tso: cache transport header length 2020-06-18 20:46:23 -07:00
tun_proto.h
udp_tunnel.h udp_tunnel: add the ability to share port tables 2020-09-28 12:50:12 -07:00
udp.h net/udp: switch udp_lib_setsockopt to sockptr_t 2020-07-24 15:41:54 -07:00
udplite.h
vsock_addr.h vsock: remove include/linux/vm_sockets.h file 2019-11-14 18:12:17 -08:00
vxlan.h net: sched: only keep the available bits when setting vxlan md->gbp 2020-09-14 16:49:39 -07:00
wext.h
wimax.h net: wimax: fix duplicate words in comments 2020-07-15 20:34:02 -07:00
x25.h net/x25: add new state X25_STATE_5 2019-12-09 10:28:43 -08:00
x25device.h
xdp_priv.h page_pool: do not release pool until inflight == 0. 2019-11-16 12:39:10 -08:00
xdp_sock_drv.h xsk: i40e: ice: ixgbe: mlx5: Test for dma_need_sync earlier for better performance 2020-08-31 21:15:04 +02:00
xdp_sock.h xsk: Fix race in SKB mode transmit with shared cq 2021-01-17 14:17:05 +01:00
xdp.h xdp: Remove the xdp_attachment_flags_ok() callback 2020-12-09 16:27:42 +01:00
xfrm.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-10-05 18:40:01 -07:00
xsk_buff_pool.h xsk: Fix race in SKB mode transmit with shared cq 2021-01-17 14:17:05 +01:00