linux_dsm_epyc7002/tools/lib/bpf
Andrii Nakryiko abd29c9314 libbpf: allow specifying map definitions using BTF
This patch adds support for a new way to define BPF maps. It relies on
BTF to describe mandatory and optional attributes of a map, as well as
captures type information of key and value naturally. This eliminates
the need for BPF_ANNOTATE_KV_PAIR hack and ensures key/value sizes are
always in sync with the key/value type.

Relying on BTF, this approach allows for both forward and backward
compatibility w.r.t. extending supported map definition features. By
default, any unrecognized attributes are treated as an error, but it's
possible relax this using MAPS_RELAX_COMPAT flag. New attributes, added
in the future will need to be optional.

The outline of the new map definition (short, BTF-defined maps) is as follows:
1. All the maps should be defined in .maps ELF section. It's possible to
   have both "legacy" map definitions in `maps` sections and BTF-defined
   maps in .maps sections. Everything will still work transparently.
2. The map declaration and initialization is done through
   a global/static variable of a struct type with few mandatory and
   extra optional fields:
   - type field is mandatory and specified type of BPF map;
   - key/value fields are mandatory and capture key/value type/size information;
   - max_entries attribute is optional; if max_entries is not specified or
     initialized, it has to be provided in runtime through libbpf API
     before loading bpf_object;
   - map_flags is optional and if not defined, will be assumed to be 0.
3. Key/value fields should be **a pointer** to a type describing
   key/value. The pointee type is assumed (and will be recorded as such
   and used for size determination) to be a type describing key/value of
   the map. This is done to save excessive amounts of space allocated in
   corresponding ELF sections for key/value of big size.
4. As some maps disallow having BTF type ID associated with key/value,
   it's possible to specify key/value size explicitly without
   associating BTF type ID with it. Use key_size and value_size fields
   to do that (see example below).

Here's an example of simple ARRAY map defintion:

struct my_value { int x, y, z; };

struct {
	int type;
	int max_entries;
	int *key;
	struct my_value *value;
} btf_map SEC(".maps") = {
	.type = BPF_MAP_TYPE_ARRAY,
	.max_entries = 16,
};

This will define BPF ARRAY map 'btf_map' with 16 elements. The key will
be of type int and thus key size will be 4 bytes. The value is struct
my_value of size 12 bytes. This map can be used from C code exactly the
same as with existing maps defined through struct bpf_map_def.

Here's an example of STACKMAP definition (which currently disallows BTF type
IDs for key/value):

struct {
	__u32 type;
	__u32 max_entries;
	__u32 map_flags;
	__u32 key_size;
	__u32 value_size;
} stackmap SEC(".maps") = {
	.type = BPF_MAP_TYPE_STACK_TRACE,
	.max_entries = 128,
	.map_flags = BPF_F_STACK_BUILD_ID,
	.key_size = sizeof(__u32),
	.value_size = PERF_MAX_STACK_DEPTH * sizeof(struct bpf_stack_build_id),
};

This approach is naturally extended to support map-in-map, by making a value
field to be another struct that describes inner map. This feature is not
implemented yet. It's also possible to incrementally add features like pinning
with full backwards and forward compatibility. Support for static
initialization of BPF_MAP_TYPE_PROG_ARRAY using pointers to BPF programs
is also on the roadmap.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-06-18 00:10:41 +02:00
..
.gitignore Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-05-02 22:14:21 -04:00
bpf_prog_linfo.c libbpf: add common min/max macro to libbpf_internal.h 2019-06-18 00:08:54 +02:00
bpf.c libbpf: add common min/max macro to libbpf_internal.h 2019-06-18 00:08:54 +02:00
bpf.h libbpf: add "prog_flags" to bpf_program/bpf_prog_load_attr/bpf_load_program_attr 2019-05-24 18:58:37 -07:00
btf_dump.c libbpf: add common min/max macro to libbpf_internal.h 2019-06-18 00:08:54 +02:00
btf.c libbpf: add common min/max macro to libbpf_internal.h 2019-06-18 00:08:54 +02:00
btf.h libbpf: allow specifying map definitions using BTF 2019-06-18 00:10:41 +02:00
Build libbpf: add btf_dump API for BTF-to-C conversion 2019-05-24 14:05:58 -07:00
hashmap.c libbpf: add resizable non-thread safe internal hashmap 2019-05-24 14:05:57 -07:00
hashmap.h libbpf: add resizable non-thread safe internal hashmap 2019-05-24 14:05:57 -07:00
libbpf_errno.c tools/bpf: make libbpf _GNU_SOURCE friendly 2018-11-30 02:41:02 +01:00
libbpf_internal.h libbpf: add common min/max macro to libbpf_internal.h 2019-06-18 00:08:54 +02:00
libbpf_probes.c libbpf: detect supported kernel BTF features and sanitize BTF 2019-05-13 01:31:20 +02:00
libbpf_util.h libbpf: move logging helpers into libbpf_internal.h 2019-05-16 12:47:47 -07:00
libbpf.c libbpf: allow specifying map definitions using BTF 2019-06-18 00:10:41 +02:00
libbpf.h bpf: add a new API libbpf_num_possible_cpus() 2019-06-11 10:36:02 +02:00
libbpf.map bpf: add a new API libbpf_num_possible_cpus() 2019-06-11 10:36:02 +02:00
libbpf.pc.template tools/bpf: generate pkg-config file for libbpf 2019-03-28 17:06:03 +01:00
Makefile libbpf: add bpf_object__load_xattr() API function to pass log_level 2019-05-28 11:03:26 +02:00
netlink.c libbpf: Add a support for getting xdp prog id on ifindex 2019-02-01 23:37:51 +01:00
nlattr.c libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
nlattr.h libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
README.rst bpf, libbpf: clarify bump in libbpf version info 2019-03-24 19:49:04 -07:00
str_error.c libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
str_error.h libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
test_libbpf.cpp tools/bpf: simplify libbpf API function libbpf_set_print() 2019-02-04 09:40:59 -08:00
xsk.c libbpf: remove qidconf and better support external bpf programs. 2019-06-10 23:31:26 -07:00
xsk.h libbpf: optimize barrier for XDP socket rings 2019-04-16 20:13:10 -07:00

.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)

libbpf API naming convention
============================

libbpf API provides access to a few logically separated groups of
functions and types. Every group has its own naming convention
described here. It's recommended to follow these conventions whenever a
new function or type is added to keep libbpf API clean and consistent.

All types and functions provided by libbpf API should have one of the
following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``.

System call wrappers
--------------------

System call wrappers are simple wrappers for commands supported by
sys_bpf system call. These wrappers should go to ``bpf.h`` header file
and map one-on-one to corresponding commands.

For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM``
command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc.

Objects
-------

Another class of types and functions provided by libbpf API is "objects"
and functions to work with them. Objects are high-level abstractions
such as BPF program or BPF map. They're represented by corresponding
structures such as ``struct bpf_object``, ``struct bpf_program``,
``struct bpf_map``, etc.

Structures are forward declared and access to their fields should be
provided via corresponding getters and setters rather than directly.

These objects are associated with corresponding parts of ELF object that
contains compiled BPF programs.

For example ``struct bpf_object`` represents ELF object itself created
from an ELF file or from a buffer, ``struct bpf_program`` represents a
program in ELF object and ``struct bpf_map`` is a map.

Functions that work with an object have names built from object name,
double underscore and part that describes function purpose.

For example ``bpf_object__open`` consists of the name of corresponding
object, ``bpf_object``, double underscore and ``open`` that defines the
purpose of the function to open ELF file and create ``bpf_object`` from
it.

Another example: ``bpf_program__load`` is named for corresponding
object, ``bpf_program``, that is separated from other part of the name
by double underscore.

All objects and corresponding functions other than BTF related should go
to ``libbpf.h``. BTF types and functions should go to ``btf.h``.

Auxiliary functions
-------------------

Auxiliary functions and types that don't fit well in any of categories
described above should have ``libbpf_`` prefix, e.g.
``libbpf_get_error`` or ``libbpf_prog_type_by_name``.

AF_XDP functions
-------------------

AF_XDP functions should have an ``xsk_`` prefix, e.g.
``xsk_umem__get_data`` or ``xsk_umem__create``. The interface consists
of both low-level ring access functions and high-level configuration
functions. These can be mixed and matched. Note that these functions
are not reentrant for performance reasons.

Please take a look at Documentation/networking/af_xdp.rst in the Linux
kernel source tree on how to use XDP sockets and for some common
mistakes in case you do not get any traffic up to user space.

libbpf ABI
==========

libbpf can be both linked statically or used as DSO. To avoid possible
conflicts with other libraries an application is linked with, all
non-static libbpf symbols should have one of the prefixes mentioned in
API documentation above. See API naming convention to choose the right
name for a new symbol.

Symbol visibility
-----------------

libbpf follow the model when all global symbols have visibility "hidden"
by default and to make a symbol visible it has to be explicitly
attributed with ``LIBBPF_API`` macro. For example:

.. code-block:: c

        LIBBPF_API int bpf_prog_get_fd_by_id(__u32 id);

This prevents from accidentally exporting a symbol, that is not supposed
to be a part of ABI what, in turn, improves both libbpf developer- and
user-experiences.

ABI versionning
---------------

To make future ABI extensions possible libbpf ABI is versioned.
Versioning is implemented by ``libbpf.map`` version script that is
passed to linker.

Version name is ``LIBBPF_`` prefix + three-component numeric version,
starting from ``0.0.1``.

Every time ABI is being changed, e.g. because a new symbol is added or
semantic of existing symbol is changed, ABI version should be bumped.
This bump in ABI version is at most once per kernel development cycle.

For example, if current state of ``libbpf.map`` is:

.. code-block::
        LIBBPF_0.0.1 {
        	global:
                        bpf_func_a;
                        bpf_func_b;
        	local:
        		\*;
        };

, and a new symbol ``bpf_func_c`` is being introduced, then
``libbpf.map`` should be changed like this:

.. code-block::
        LIBBPF_0.0.1 {
        	global:
                        bpf_func_a;
                        bpf_func_b;
        	local:
        		\*;
        };
        LIBBPF_0.0.2 {
                global:
                        bpf_func_c;
        } LIBBPF_0.0.1;

, where new version ``LIBBPF_0.0.2`` depends on the previous
``LIBBPF_0.0.1``.

Format of version script and ways to handle ABI changes, including
incompatible ones, described in details in [1].

Stand-alone build
=================

Under https://github.com/libbpf/libbpf there is a (semi-)automated
mirror of the mainline's version of libbpf for a stand-alone build.

However, all changes to libbpf's code base must be upstreamed through
the mainline kernel tree.

License
=======

libbpf is dual-licensed under LGPL 2.1 and BSD 2-Clause.

Links
=====

[1] https://www.akkadia.org/drepper/dsohowto.pdf
    (Chapter 3. Maintaining APIs and ABIs).