linux_dsm_epyc7002/include/uapi/linux
Alexei Starovoitov 969bf05eb3 bpf: direct packet access
Extended BPF carried over two instructions from classic to access
packet data: LD_ABS and LD_IND. They're highly optimized in JITs,
but due to their design they have to do length check for every access.
When BPF is processing 20M packets per second single LD_ABS after JIT
is consuming 3% cpu. Hence the need to optimize it further by amortizing
the cost of 'off < skb_headlen' over multiple packet accesses.
One option is to introduce two new eBPF instructions LD_ABS_DW and LD_IND_DW
with similar usage as skb_header_pointer().
The kernel part for interpreter and x64 JIT was implemented in [1], but such
new insns behave like old ld_abs and abort the program with 'return 0' if
access is beyond linear data. Such hidden control flow is hard to workaround
plus changing JITs and rolling out new llvm is incovenient.

Therefore allow cls_bpf/act_bpf program access skb->data directly:
int bpf_prog(struct __sk_buff *skb)
{
  struct iphdr *ip;

  if (skb->data + sizeof(struct iphdr) + ETH_HLEN > skb->data_end)
      /* packet too small */
      return 0;

  ip = skb->data + ETH_HLEN;

  /* access IP header fields with direct loads */
  if (ip->version != 4 || ip->saddr == 0x7f000001)
      return 1;
  [...]
}

This solution avoids introduction of new instructions. llvm stays
the same and all JITs stay the same, but verifier has to work extra hard
to prove safety of the above program.

For XDP the direct store instructions can be allowed as well.

The skb->data is NET_IP_ALIGNED, so for common cases the verifier can check
the alignment. The complex packet parsers where packet pointer is adjusted
incrementally cannot be tracked for alignment, so allow byte access in such cases
and misaligned access on architectures that define efficient_unaligned_access

[1] https://git.kernel.org/cgit/linux/kernel/git/ast/bpf.git/?h=ld_abs_dw

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-05-06 16:01:53 -04:00
..
android
byteorder include/uapi/linux/byteorder, swab: force inlining of some byteswap operations 2016-03-17 15:09:34 -07:00
caif
can can: avoid using timeval for uapi 2015-10-13 17:42:34 +02:00
cifs
dvb [media] include/uapi/linux/dvb/video.h: remove stdint.h include 2015-11-19 08:18:38 -02:00
genwqe
hdlc
hsi
iio iio: ph: add IIO_PH channel type 2016-01-30 16:27:17 +00:00
isdn
mmc mmc: block: Add new ioctl to send multi commands 2015-10-26 16:00:00 +01:00
netfilter libnl: nla_put_net64(): align on a 64-bit area 2016-04-23 20:13:24 -04:00
netfilter_arp netfilter: fix include files for compilation 2015-11-23 17:54:38 +01:00
netfilter_bridge netfilter: fix include files for compilation 2015-11-23 17:54:38 +01:00
netfilter_ipv4 netfilter: fix include files for compilation 2015-11-23 17:54:38 +01:00
netfilter_ipv6 netfilter: fix include files for compilation 2015-11-23 17:54:38 +01:00
nfsd
raid drivers: md: use ktime_get_real_seconds() 2016-01-06 11:39:53 +11:00
spi
sunrpc
tc_act sched: align nlattr properly when needed 2016-04-26 12:00:49 -04:00
tc_ematch
usb usb: ch9: Fix SSP Device Cap wFunctionalitySupport type 2016-03-29 13:26:04 +03:00
wimax
a.out.h
acct.h
adb.h
adfs_fs.h
affs_hardblocks.h
agpgart.h include/uapi/linux/agpgart.h: include stdlib.h in userspace 2015-12-10 12:33:23 +01:00
aio_abi.h
am437x-vpfe.h
apm_bios.h
arcfb.h
atalk.h
atm_eni.h
atm_he.h
atm_idt77105.h
atm_nicstar.h
atm_tcp.h
atm_zatm.h atm: remove 'struct zatm_t_hist' 2015-10-05 03:16:46 -07:00
atm.h
atmapi.h
atmarp.h
atmbr2684.h
atmclip.h
atmdev.h
atmioc.h
atmlec.h
atmmpc.h
atmppp.h
atmsap.h
atmsvc.h
audit.h audit: stop an old auditd being starved out by a new auditd 2016-01-25 18:04:15 -05:00
auto_fs4.h autofs4: fix some white space errors 2016-03-15 16:55:16 -07:00
auto_fs.h autofs4: fix some white space errors 2016-03-15 16:55:16 -07:00
auxvec.h
ax25.h
b1lli.h
baycom.h
bcache.h
bcm933xx_hcs.h
bfs_fs.h
binfmts.h
blkpg.h mtd: provide proper 32/64-bit compat_ioctl() support for BLKPG 2015-09-29 13:37:04 -07:00
blktrace_api.h
bpf_common.h
bpf.h bpf: direct packet access 2016-05-06 16:01:53 -04:00
bpqether.h
bsg.h
btrfs.h btrfs: extend balance filter usage to take minimum and maximum 2015-10-26 19:38:30 -07:00
can.h
capability.h
capi.h
cciss_defs.h
cciss_ioctl.h
cdrom.h
cgroupstats.h
chio.h
cm4000_cs.h
cn_proc.h
coda_psdev.h
coda.h
coff.h
connector.h
const.h
cramfs_fs.h
cryptouser.h
cuda.h
cyclades.h
cycx_cfm.h
dcbnl.h
dccp.h
devlink.h devlink: implement shared buffer occupancy monitoring interface 2016-04-14 16:22:03 -04:00
dlm_device.h dlm: fix lvb copy for user locks 2015-08-25 14:41:50 -05:00
dlm_netlink.h
dlm_plock.h
dlm.h
dlmconstants.h
dm-ioctl.h dm: add support for passing through persistent reservations 2015-10-31 19:05:59 -04:00
dm-log-userspace.h
dma-buf.h dma-buf: Add ioctls to allow userspace to flush 2016-02-12 16:01:32 +01:00
dn.h
dqblk_xfs.h quota: add new quotactl Q_XGETNEXTQUOTA 2016-02-08 11:21:50 +11:00
edd.h
efs_fs_sb.h
elf-em.h include/uapi/linux/elf-em.h: remove v850 2016-03-17 15:09:34 -07:00
elf-fdpic.h
elf.h
elfcore.h
errno.h
errqueue.h
ethtool.h ethtool: minor doc update 2016-03-22 15:45:44 -04:00
eventpoll.h epoll: add EPOLLEXCLUSIVE flag 2016-01-20 17:09:18 -08:00
fadvise.h
falloc.h
fanotify.h
fb.h
fcntl.h
fd.h
fdreg.h
fib_rules.h libnl: nla_put_be64(): align on a 64-bit area 2016-04-23 20:13:24 -04:00
fiemap.h
filter.h
firewire-cdev.h
firewire-constants.h
flat.h
fou.h
fs.h Merge tag 'for-f2fs-4.6' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs 2016-03-21 11:03:02 -07:00
fsl_hypervisor.h
fuse.h fuse: add support for SEEK_HOLE and SEEK_DATA in lseek 2015-11-10 10:32:37 +01:00
futex.h
gameport.h
gen_stats.h sched: align nlattr properly when needed 2016-04-26 12:00:49 -04:00
genetlink.h openvswitch: allow management from inside user namespaces 2016-02-11 09:53:19 -05:00
gfs2_ondisk.h gfs2: change gfs2 readdir cookie 2015-12-14 12:19:37 -06:00
gigaset_dev.h
gpio.h gpio: uapi: use 0xB4 as ioctl() major 2016-03-10 16:02:52 +07:00
gsmmux.h
hash_info.h keys, trusted: select hash algorithm for TPM2 chips 2015-12-20 15:27:12 +02:00
hdlc.h
hdlcdrv.h
hdreg.h
hid.h
hiddev.h
hidraw.h
hpet.h
hsr_netlink.h
hw_breakpoint.h
hyperv.h tools: hv: report ENOSPC errors in hv_fcopy_daemon 2015-12-14 19:12:21 -08:00
hysdn_if.h
i2c-dev.h i2c-dev: Fix typo in ioctl name reference 2015-10-23 23:26:43 +02:00
i2c.h
i2o-dev.h
i8k.h
icmp.h
icmpv6.h
if_addr.h
if_addrlabel.h
if_alg.h
if_arcnet.h arcnet: fix indentation of if_arcnet.h 2015-09-23 08:44:22 +02:00
if_arp.h
if_bonding.h
if_bridge.h bridge: netlink: export per-vlan stats 2016-05-02 22:27:06 -04:00
if_cablemodem.h
if_eql.h
if_ether.h net/hsr: Added support for HSR v1 2016-04-15 17:06:48 -04:00
if_fc.h
if_fddi.h
if_frad.h
if_hippi.h
if_infiniband.h
if_link.h bridge: netlink: export per-vlan stats 2016-05-02 22:27:06 -04:00
if_ltalk.h
if_macsec.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-04-27 15:43:10 -04:00
if_packet.h
if_phonet.h
if_plip.h
if_ppp.h
if_pppol2tp.h
if_pppox.h
if_slip.h
if_team.h
if_tun.h
if_tunnel.h
if_vlan.h
if_x25.h
if.h net: fix a comment typo 2016-03-18 19:40:27 -04:00
igmp.h
ila.h ila: add checksum neutral ILA translations 2016-04-26 01:27:07 -04:00
in6.h ipv6: add IPV6_HDRINCL option for raw sockets 2015-12-17 15:12:28 -05:00
in_route.h
in.h
inet_diag.h sock_diag: align nlattr properly when needed 2016-04-26 12:00:48 -04:00
inotify.h
input-event-codes.h Input: add input-event-codes header file 2015-10-16 15:32:05 -07:00
input.h Input: synaptics-rmi4 - add support for Synaptics RMI4 devices 2016-03-10 16:02:39 -08:00
ioctl.h
ip6_tunnel.h
ip_vs.h netfilter/ipvs: use nla_put_u64_64bit() 2016-04-25 15:09:11 -04:00
ip.h ipv4: add option to drop gratuitous ARP packets 2016-02-11 04:27:35 -05:00
ipc.h
ipmi_msgdefs.h
ipmi.h
ipsec.h
ipv6_route.h
ipv6.h Revert "ipv6: Revert optional address flusing on ifdown." 2016-04-26 11:47:41 -04:00
ipx.h
irda.h
irqnr.h
isdn_divertif.h
isdn_ppp.h
isdn.h
isdnif.h
iso_fs.h
ivtv.h
ivtvfb.h
ixjuser.h
jffs2.h
joystick.h
Kbuild devlink: add missing install of header 2016-04-11 21:33:44 -04:00
kcm.h kcm: Kernel Connection Multiplexor module 2016-03-09 16:36:14 -05:00
kcmp.h
kcov.h kernel: add kcov code coverage 2016-03-22 15:36:02 -07:00
kd.h
kdev_t.h
kernel-page-flags.h proc: export idle flag via kpageflags 2015-09-10 13:29:01 -07:00
kernel.h uapi: define DIV_ROUND_UP for userland 2016-03-04 16:10:36 -05:00
kernelcapi.h
kexec.h
keyboard.h
keyctl.h
kfd_ioctl.h
kvm_para.h
kvm.h KVM/ARM updates for 4.6 2016-03-09 11:50:42 +01:00
l2tp.h l2tp: use nla_put_u64_64bit() 2016-04-25 15:09:10 -04:00
libc-compat.h
lightnvm.h lightnvm: introduce factory reset 2016-01-12 08:21:18 -07:00
limits.h
lirc.h [media] bz#75751: Move internal header file lirc.h to uapi/ 2015-11-17 06:47:43 -02:00
llc.h
loop.h block: loop: introduce ioctl command of LOOP_SET_DIRECT_IO 2015-09-23 11:01:16 -06:00
lp.h
lwtunnel.h libnl: nla_put_be64(): align on a 64-bit area 2016-04-23 20:13:24 -04:00
magic.h Merge branch 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs 2016-01-21 12:20:46 -08:00
major.h
map_to_7segment.h
matroxfb.h
mdio.h
media-bus-format.h
media.h Merge commit '840f5b0572ea' into v4l_for_linus 2016-03-15 07:48:28 -03:00
mei.h
membarrier.h sys_membarrier(): system-wide memory barrier (generic, x86) 2015-09-11 15:21:34 -07:00
memfd.h
mempolicy.h
meye.h
mic_common.h misc: mic: Update MIC host daemon with COSM changes 2015-10-04 12:54:54 +01:00
mic_ioctl.h
mii.h
minix_fs.h
mman.h
mmtimer.h
module.h
mpls_iptunnel.h
mpls.h
mqueue.h
mroute6.h uapi: define DIV_ROUND_UP for userland 2016-03-04 16:10:36 -05:00
mroute.h net: ipmr: fix code and comment style 2015-11-23 15:06:38 -05:00
msdos_fs.h
msg.h
mtio.h
n_r3964.h
nbd.h
ncp_fs.h
ncp_mount.h
ncp_no.h
ncp.h
ndctl.h nfit, libnvdimm: clear poison command support 2016-03-05 18:06:14 -08:00
neighbour.h libnl: nla_put_msecs(): align on a 64-bit area 2016-04-23 20:13:24 -04:00
net_dropmon.h
net_namespace.h
net_tstamp.h sock: accept SO_TIMESTAMPING flags in socket cmsg 2016-04-04 15:50:30 -04:00
net.h
netconf.h netconf: add macro to represent all attributes 2016-03-13 21:54:44 -04:00
netdevice.h
netfilter_arp.h
netfilter_bridge.h netfilter: fix include files for compilation 2015-11-23 17:54:38 +01:00
netfilter_decnet.h
netfilter_ipv4.h
netfilter_ipv6.h
netfilter.h
netlink_diag.h netlink: remove mmapped netlink support 2016-02-18 11:42:18 -05:00
netlink.h netlink: remove mmapped netlink support 2016-02-18 11:42:18 -05:00
netrom.h
nfc.h NFC: netlink: Add missing NFC_ATTR comments 2015-10-27 03:55:10 +01:00
nfs2.h
nfs3.h
nfs4_mount.h
nfs4.h NFS: Update NFS4_BITMAP_SIZE 2015-08-27 19:44:53 -04:00
nfs_fs.h
nfs_idmap.h
nfs_mount.h
nfs.h nfs: use btrfs ioctl defintions for clone 2015-11-23 21:53:08 -05:00
nfsacl.h
nl80211.h wireless: use nla_put_u64_64bit() 2016-04-25 15:09:11 -04:00
nubus.h
nvme_ioctl.h nvme: move hardware structures out of the uapi version of nvme.h 2015-10-09 10:40:37 -06:00
nvram.h
omap3isp.h
omapfb.h
oom.h
openvswitch.h ovs: align nlattr properly when needed 2016-04-26 12:00:48 -04:00
packet_diag.h
param.h
parport.h
patchkey.h
pci_regs.h PCI: Make Enhanced Allocation bitmasks more obvious 2015-10-29 17:35:40 -05:00
pci.h
perf_event.h perf/x86: Add option to disable reading branch flags/cycles 2015-11-23 09:58:25 +01:00
personality.h
pfkeyv2.h
pg.h
phantom.h
phonet.h
pkt_cls.h sched: align nlattr properly when needed 2016-04-26 12:00:49 -04:00
pkt_sched.h fq_codel: add batch ability to fq_codel_drop() 2016-05-03 12:47:09 -04:00
pktcdvd.h
pmu.h
poll.h
posix_types.h
ppdev.h
ppp_defs.h
ppp-comp.h
ppp-ioctl.h
pps.h
pr.h block: add an API for Persistent Reservations 2015-10-21 14:46:56 -06:00
prctl.h capabilities: ambient capabilities 2015-09-04 16:54:41 -07:00
psci.h drivers: firmware: psci: add system suspend support 2015-10-02 14:35:17 +01:00
ptp_clock.h ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping 2016-03-03 14:23:43 -08:00
ptrace.h seccomp, ptrace: add support for dumping seccomp filters 2015-10-27 19:55:13 -07:00
qnx4_fs.h
qnxtypes.h
quota.h fs/quota: use nla_put_u64_64bit() 2016-04-26 12:00:48 -04:00
radeonfb.h
random.h
raw.h
rds.h
reboot.h
reiserfs_fs.h
reiserfs_xattr.h
resource.h
rfkill.h rfkill: Update userspace API documentation 2016-02-24 09:04:25 +01:00
romfs_fs.h
rose.h
route.h
rtc.h
rtnetlink.h sched: align nlattr properly when needed 2016-04-26 12:00:49 -04:00
scc.h
sched.h sched: new clone flag CLONE_NEWCGROUP for cgroup namespace 2016-02-16 13:04:58 -05:00
scif_ioctl.h misc: mic: SCIF RMA header file and IOCTL changes 2015-10-04 12:54:54 +01:00
screen_info.h efifb: Add support for 64-bit frame buffer addresses 2015-10-12 14:20:06 +01:00
sctp.h
sdla.h
seccomp.h
securebits.h capabilities: add a securebit to disable PR_CAP_AMBIENT_RAISE 2015-09-04 16:54:41 -07:00
selinux_netlink.h
sem.h
serial_core.h serial: mvebu-uart: initial support for Armada-3700 serial port 2016-03-07 16:11:14 -08:00
serial_reg.h
serial.h serial: support 16-bit register interface for console 2015-12-13 19:59:48 -08:00
serio.h Input: add eGalaxTouch serial touchscreen driver 2015-12-16 11:31:33 -08:00
shm.h
signal.h
signalfd.h
smiapp.h
snmp.h
sock_diag.h sock_diag: add SK_MEMINFO_DROPS 2016-04-04 22:11:20 -04:00
socket.h
sockios.h include/uapi/linux/sockios.h: mark SIOCRTMSG unused 2016-01-05 16:44:06 -05:00
sonet.h
sonypi.h
sound.h
soundcard.h
stat.h
stddef.h uapi/linux/stddef.h: Provide __always_inline to userspace headers 2016-03-30 12:50:17 +02:00
stm.h stm class: Introduce an abstraction for System Trace Module devices 2015-10-04 20:28:58 +01:00
string.h
suspend_ioctls.h
swab.h include/uapi/linux/byteorder, swab: force inlining of some byteswap operations 2016-03-17 15:09:34 -07:00
synclink.h
sysctl.h
sysinfo.h
target_core_user.h target/user: Report capability of handling out-of-order completions to userspace 2016-03-10 21:49:09 -08:00
taskstats.h
tcp_metrics.h libnl: nla_put_msecs(): align on a 64-bit area 2016-04-23 20:13:24 -04:00
tcp.h tcp: Add RFC4898 tcpEStatsPerfDataSegsOut/In 2016-03-14 14:55:26 -04:00
telephony.h
termios.h
thermal.h
time.h
times.h
timex.h
tiocl.h
tipc_config.h
tipc_netlink.h
tipc.h
toshiba.h
tty_flags.h
tty.h
types.h
udf_fs_i.h
udp.h
uhid.h
uinput.h Input: uinput - rework ABS validation 2015-12-18 17:48:51 -08:00
uio.h
ultrasound.h
un.h
unistd.h
unix_diag.h
usbdevice_fs.h usb: devio: Add ioctl to disallow detaching kernel USB drivers. 2016-03-05 12:05:01 -08:00
usbip.h
userfaultfd.h userfaultfd: remove kernel header include from uapi header 2015-10-01 21:42:35 -04:00
userio.h Input: add userio module 2015-10-27 18:55:31 -07:00
utime.h
utsname.h
uuid.h
uvcvideo.h
v4l2-common.h [media] media: v4l: Dual license v4l2-common.h under GPL v2 and BSD licenses 2016-02-01 08:47:05 -02:00
v4l2-controls.h [media] v4l: add V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME 2016-02-19 08:10:35 -02:00
v4l2-dv-timings.h [media] v4l2-dv-timings.h: fix polarity for 4k formats 2016-04-25 10:19:56 -03:00
v4l2-mediabus.h
v4l2-subdev.h
veth.h
vfio.h vfio/pci: Intel IGD host and LCP bridge config space access 2016-02-22 16:10:09 -07:00
vhost.h vhost_net: basic polling support 2016-03-11 02:18:53 +02:00
videodev2.h [media] UVC: Add support for R200 depth camera 2016-03-03 06:49:20 -03:00
virtio_9p.h
virtio_balloon.h virtio_balloon: export 'available' memory to balloon statistics 2016-03-17 15:09:34 -07:00
virtio_blk.h virtio_blk: VIRTIO_BLK_F_WCE->VIRTIO_BLK_F_FLUSH 2016-03-02 17:01:59 +02:00
virtio_config.h virtio: add VIRTIO_CONFIG_S_NEEDS_RESET device status bit 2016-04-07 15:16:41 +03:00
virtio_console.h
virtio_gpu.h include/uapi/linux/virtio_gpu.h: use __u8 from <linux/types.h> 2015-12-10 12:33:23 +01:00
virtio_ids.h Revert "Merge branch 'vsock-virtio'" 2015-12-08 21:55:49 -05:00
virtio_input.h
virtio_net.h
virtio_pci.h
virtio_ring.h
virtio_rng.h
virtio_scsi.h
virtio_types.h
vm_sockets.h
vsp1.h
vt.h
wait.h
wanrouter.h
watchdog.h
wil6210_uapi.h
wimax.h
wireless.h
x25.h
xattr.h
xfrm.h xfrm: align nlattr properly when needed 2016-04-23 20:13:25 -04:00
xilinx-v4l2-controls.h
zorro_ids.h
zorro.h