Przemek suggests that I shouldn't accuse GCC of witchcraft,
there is a simpler explanation for why we need manual define.
scripts/headers_install.sh modifies the guard, removing _UAPI.
That's why including a kernel header from the tree and from
/usr leads to duplicate definitions.
This also solves the mystery of why I needed to include
the header conditionally. I had the wrong guards for most
cases but ethtool.
Suggested-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Link: https://lore.kernel.org/r/20230621231719.2728928-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The inability to include the uAPI headers directly in tools/
is one of the bigger annoyances of compiling user space code.
Most projects trade the pain for smaller inconvenience of having
to copy the headers under tools/include.
In case of netlink headers I think that we can avoid both.
Netlink family headers are simple and should be self-contained.
We can try to twiddle the Makefile a little to force-include
just the family header, and use system headers for the rest.
This works fairly well. There are two warts - for some reason
if we specify -include $path/family.h as a compilation flag,
the #ifdef header guard does not seem to work. So we need
to throw the guard in on the command line as well. Seems like
GCC detects that the header is different and tries to include
both. Second problem is that make wants hash sign to be escaped
or not depending on the version. Sigh.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Instead of reimplementing policies in MutliAttr for every
underlying type forward the calls to the base type.
This will be needed for DPLL which uses a multi-attr nest,
and currently gets an invalid NLA_NEST policy generated.
Reviewed-by: Jacob Keller <Jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Scalar range validation assumes enums start at 0.
Teach it to properly calculate the value range.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Configuring / reading ring sizes and counts is a fairly common
operation for ethtool netlink. Present a sample doing that with
YNL:
$ ./ethtool
Channels:
enp1s0: combined 1
eni1np1: combined 1
eni2np1: combined 1
Rings:
enp1s0: rx 256 tx 256
eni1np1: rx 0 tx 0
eni2np1: rx 0 tx 0
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Generate the protocol code for ethtool. Skip the stats
for now, they are the only outlier in terms of complexity.
Stats are a sort-of semi-polymorphic (attr space of a nest
depends on value of another attr) or a type-value-scalar,
depending on how one wants to look at it...
A challenge for another time.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Ethtool has an attribute set called stringset, from which
we'll generate struct ethtool_stringset. Unfortunately,
the old ethtool header declares enum ethtool_stringset
(the same name), to which compilers object.
This seems unavoidable. Check struct names against known
constants and append an underscore if conflict is detected.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
If attr set or enum has empty enum name we need to use u32 or int
as function arguments and struct members.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Ethtool's PSE PoDL has a attr nest with different prefixes:
/* Power Sourcing Equipment */
enum {
ETHTOOL_A_PSE_UNSPEC,
ETHTOOL_A_PSE_HEADER, /* nest - _A_HEADER_* */
ETHTOOL_A_PODL_PSE_ADMIN_STATE, /* u32 */
ETHTOOL_A_PODL_PSE_ADMIN_CONTROL, /* u32 */
ETHTOOL_A_PODL_PSE_PW_D_STATUS, /* u32 */
Header has a prefix of ETHTOOL_A_PSE_ and other attrs prefix of
ETHTOOL_A_PODL_PSE_ we can't cover them uniformly.
If PODL was after PSE life would be easy.
Now we either need to add prefixes to attr names which is yucky
or support setting prefix name per attr.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
ynl-regen needs to know the arguments used to generate a file.
Record excluded ops and, while at it, user headers.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The ethtool family has a small handful of quite tricky ops
and a lot of simple very useful ops. Teach ynl-gen to skip
ops so that we can bypass the tricky ones.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Netlink specs support both events and notifications (former can
define their own message contents). Plug in missing code to
generate types, parsers and include events into notification
tables.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Don't modify the raw dicts (as loaded from YAML) to pretend
that the notify attributes also exist on the ops. This makes
the code easier to follow.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Common notification handler was supposed to be a way for the user
to parse the notifications from a socket synchronously.
I don't think we'll end up using it, ynl_ntf_check() works for
all known use cases.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reading attr type with mnl_attr_get_type() for each condition
leads to most conditions being longer than 80 chars.
Avoid this by reading the type to a variable on the stack.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Code gen currently prints:
}
else if (...
This is really ugly. Fix it by delaying printing of closing
brackets in anticipation of else coming along.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
C keywords need to be avoided when naming things.
Complete the list (ethtool has at least one thing called "auto").
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This reverts commit e7c5433c5a.
Commit e7c5433c5a ("tools: ynl: Remove duplicated include
in handshake-user.c") was applied too hastily. It changes
an auto-generated file, and there's already a proper fix
on the list.
Link: https://lore.kernel.org/all/ZIMPLYi%2FxRih+DlC@nanopsycho/
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
./tools/net/ynl/generated/handshake-user.c: stdlib.h is included more than once.
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5464
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Add a sample to show off how to issue basic devlink requests.
For added testing issue get requests while walking a dump.
$ ./devlink
netdevsim/netdevsim1:
driver: netdevsim
running fw:
fw.mgmt: 10.20.30
...
netdevsim/netdevsim2:
driver: netdevsim
running fw:
fw.mgmt: 10.20.30
...
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Admittedly the devlink.yaml spec is fairly limitted,
it only covers basic device get and info-get ops.
That's sufficient to be useful (monitoring FW versions
in the fleet). Plus it gives us a chance to exercise
deep nesting and directional messaging in YNL.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Now that all nested types have structs and are sorted topologically
there should be no need to generate forward declarations for policies.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
So far we had only created structures for nested types nested
directly in messages (second level of attrs so to speak).
Walk types in depth to support deeper nesting.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We only render parse and netlink generation helpers as needed,
to avoid generating dead code. Propagate the information from
first- and second-layer attribute sets onto all children.
Otherwise devlink won't work, it has a lot more levels of nesting.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We need to sort the structures to avoid the need for forward
declarations. While at it remove the sort of structs when
rendering, it doesn't do anything.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
In preparation for supporting families which use different msg
ids to and from the kernel - make sure the ids in op strmap
are correct. The map is expected to be used mostly for notifications,
don't generate a separate map for the "to kernel" direction.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When parsing multi-attr we count the objects and then allocate
an array to hold the parsed objects. If an attr space has multiple
multi-attr objects, however, if parsing the first array fails
we'll leave the object count for the second even tho the second
array was never allocated.
This may cause crashes when freeing objects on error.
Count attributes to a variable on the stack and only set the count
in the object once the memory was allocated.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The handshake family needs support for MultiAttr scalars.
Right now we only support code gen for MultiAttr nested
types.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a sample application using the C library.
My main goal is to make writing selftests easier but until
I have some of those ready I think it's useful to show off
the functionality and let people poke and tinker.
Sample outputs - dump:
$ ./netdev
Select ifc ($ifindex; or 0 = dump; or -2 ntf check): 0
lo[1] 0:
enp1s0[2] 23: basic redirect rx-sg
Notifications (watching veth pair getting added and deleted):
$ ./netdev
Select ifc ($ifindex; or 0 = dump; or -2 ntf check): -2
[53] 0: (ntf: dev-add-ntf)
[54] 0: (ntf: dev-add-ntf)
[54] 23: basic redirect rx-sg (ntf: dev-change-ntf)
[53] 23: basic redirect rx-sg (ntf: dev-change-ntf)
[53] 23: basic redirect rx-sg (ntf: dev-del-ntf)
[54] 23: basic redirect rx-sg (ntf: dev-del-ntf)
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Generate the code for netdev and fou families. They are simple
and already supported by the code gen.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add "fixed" part of the user space Netlink Spec-based library.
This will get linked with the protocol implementations to form
a full API.
Acked-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Notifications may come in at any time. The family must be always
ready to parse a random incoming notification. Generate notification
table for parsing and tell YNL which request we're processing
to distinguish responses from notifications.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We'll want to store static info about the family soon.
Generate a struct. This changes creation from, e.g.:
ys = ynl_sock_create("netdev", &yerr);
to:
ys = ynl_sock_create(&ynl_netdev_family, &yerr);
on user's side.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We expect user to allocate requests with calloc(),
make things a bit more consistent and provide helpers.
Generate free calls, too.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We generate send() and recv() calls and all msg handling for
each operation. It's a lot of repeated code and will only grow
with notification handling. Call back to a helper YNL lib instead.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
It's sometimes useful to print the name of an enum value,
flag or name of the op. Python can do it, add C helper
code gen for getting names of things.
Example:
static const char * const netdev_xdp_act_strmap[] = {
[0] = "basic",
[1] = "redirect",
[2] = "ndo-xmit",
[3] = "xsk-zerocopy",
[4] = "hw-offload",
[5] = "rx-sg",
[6] = "ndo-xmit-sg",
};
const char *netdev_xdp_act_str(enum netdev_xdp_act value)
{
value = ffs(value) - 1;
if (value < 0 || value >= (int)MNL_ARRAY_SIZE(netdev_xdp_act_strmap))
return NULL;
return netdev_xdp_act_strmap[value];
}
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Parsing nested types may return an error, propagate it.
Not marking as a fix, because nothing uses YNL upstream.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Both event and notify types are always consistent. Rewrite
the condition checking if we can reuse reply types to be
less picky and let notify thru.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
For pure structs (parsed nested attributes) we track what
forms of the struct exist in request and reply directions.
Make sure we don't overwrite the recorded struct each time,
otherwise the information is lost.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Unused and Pad attributes don't carry information.
Unused should never exist, and be rejected.
Pad should be silently skipped.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Make sure all relevant headers are included, we allocate memory,
use memcpy() and Linux types without including the headers.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Support decoding scalars as enums in struct members for genetlink-legacy
specs.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This eliminates the need for e.g. --json '{"dp-ifindex":0}' which is not
too big a deal for ovs but will get tiresome for fixed header structs that
have many members.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
To keep things simple we used to include the uAPI header
in the kernel in the #include <linux/$family.h> format.
This works well enough, most of the genl families should
have headers in include/net/ so linux/$family.h ends up
referring to the uAPI header, anyway. And if it doesn't
no big deal, we'll just include more info than we need.
Unless that is there is a naming conflict. Someone recently
created include/linux/psp.h which will be a problem when
supporting the PSP protocol. (I'm talking about
work-in-progress patches, but it's just a proof that assuming
lack of name conflicts was overly optimistic.)
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Python 3.9.0 or newer supports combining dicts() with |,
but older versions of Python are still used in the wild
(e.g. on CentOS 8, which goes EoL May 31, 2024).
With Python 3.6.8 we get:
TypeError: unsupported operand type(s) for |: 'dict' and 'dict'
Use older syntax. Tested with non-legacy families only.
Fixes: f036d936ca ("tools: ynl: Add fixed-header support to ynl")
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Tested-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230524170712.2036128-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for byte-order in struct members in the genetlink-legacy
spec.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use a dict of predefined Struct() objects to decode scalar types in native,
big or little endian format. This removes the repetitive code for the
scalar variants and ensures all the signed variants are supported.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Make it explicit that this tool is not a drop-in replacement for ethtool.
This tool is intended for testing ethtool functionality implemented in the
kernel and should use a name that differentiates it from the ethtool
utility.
Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Link: https://lore.kernel.org/r/20230413012252.184434-2-rrameshbabu@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Absolute paths for the spec and schema files make the ethtool testing tool
unusable with freshly checked-out source trees. Replace absolute paths with
relative paths for both files in the Documentation/ directory.
Issue seen before the change
Traceback (most recent call last):
File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/./ethtool", line 424, in <module>
main()
File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/./ethtool", line 158, in main
ynl = YnlFamily(spec, schema)
File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/lib/ynl.py", line 342, in __init__
super().__init__(def_path, schema)
File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/lib/nlspec.py", line 333, in __init__
with open(spec_path, "r") as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/google/home/sdf/src/linux/Documentation/netlink/specs/ethtool.yaml'
Fixes: f3d07b02b2 ("tools: ynl: ethtool testing tool")
Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Link: https://lore.kernel.org/r/20230413012252.184434-1-rrameshbabu@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
cli.py currently throws a pure KeyError if kernel doesn't support
a netlink family. Users who did not write ynl (hah) may waste
their time investigating what's wrong with the Python code.
Improve the error message:
Traceback (most recent call last):
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 362, in __init__
self.family = GenlFamily(self.yaml['name'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 331, in __init__
self.genl_family = genl_family_name_to_id[family_name]
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'netdev'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kicinski/devel/linux/./tools/net/ynl/cli.py", line 52, in <module>
main()
File "/home/kicinski/devel/linux/./tools/net/ynl/cli.py", line 31, in main
ynl = YnlFamily(args.spec, args.schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 364, in __init__
raise Exception(f"Family '{self.yaml['name']}' not supported by the kernel")
Exception: Family 'netdev' not supported by the kernel
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230407145609.297525-1-kuba@kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This is what I've been using to see whether the spec makes sense.
A small subset of getters (mostly the unprivileged ones) is implemented.
Some setters (channels) also work.
Setters for messages with bitmasks are not implemented.
Initially I was trying to make this tool look 1:1 like real ethtool,
but eventually gave up :-)
Sample output:
$ ./tools/net/ynl/ethtool enp0s31f6
Settings for enp0s31f6:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half
100baseT/Full 1000baseT/Full
Supported pause frame use: no
Supports auto-negotiation: yes
Supported FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 2
Transceiver: Internal
MDI-X: Unknown (auto)
Current message level: drv probe link
Link detected: no
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of dumping the error on the stdout, make the callee and
opportunity to decide what to do with it. This is mostly for the
ethtool testing.
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for netlink families that add an optional fixed header structure
after the genetlink header and before any attributes. The fixed-header can be
specified on a per op basis, or once for all operations, which serves as a
default value that can be overridden.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for decoding attributes that contain C structs.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for decoding C arrays from binay blobs in genetlink-legacy
messages.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add python classes for struct definitions to nlspec
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I was a bit too optimistic in commit bf51d27704 ("tools: ynl: fix
get_mask utility routine"), not every mask we use is necessarily
coming from an enum of type "flags". We also allow flipping an
enum into flags on per-attribute basis. That's done by
the 'enum-as-flags' property of an attribute.
Restore this functionality, it's not currently used by any in-tree
family.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
While testing the tool I noticed we miss the u16 type on payload create.
On the code inspection it turned out we miss also u64 - add them.
We also miss the decoding of u16 despite the fact `NlAttr` class
supports it - add it.
Signed-off-by: Michal Michalik <michal.michalik@intel.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
It is a good practice to state explicitly which are the required Python
packages needed in a particular project to run it. The most commonly
used way is to store them in the `requirements.txt` file*.
*URL: https://pip.pypa.io/en/stable/reference/requirements-file-format/
Currently user needs to figure out himself that Python needs `PyYAML`
and `jsonschema` (and theirs requirements) packages to use the tool.
Add the `requirements.txt` for user convenience.
How to use it:
1) (optional) Create and activate empty virtual environment:
python3.X -m venv venv3X
source ./venv3X/bin/activate
2) Install all the required packages:
pip install -r requirements.txt
or
python -m pip install -r requirements.txt
3) Run the script!
The `requirements.txt` file was tested for:
* Python 3.6
* Python 3.8
* Python 3.10
Signed-off-by: Michal Michalik <michal.michalik@intel.com>
Link: https://lore.kernel.org/r/20230323190802.32206-1-michal.michalik@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Playing with dpll netlink, I came across following issue:
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/dpll.yaml --do pin-set --json '{"id": 0, "pin-idx": 1, "pin-state": 1}'
Traceback (most recent call last):
File "tools/net/ynl/cli.py", line 52, in <module>
main()
File "tools/net/ynl/cli.py", line 40, in main
reply = ynl.do(args.do, attrs)
File "tools/net/ynl/lib/ynl.py", line 520, in do
return self._op(method, vals)
File "tools/net/ynl/lib/ynl.py", line 476, in _op
msg += self._add_attr(op.attr_set.name, name, value)
File "tools/net/ynl/lib/ynl.py", line 344, in _add_attr
raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}')
Exception: Unknown type at dpll pin-state 1 u8
I'm not that familiar with ynl code, but from a quick peek, I suspect
that couple other types are missing for both encoding and decoding.
Ignoring those here as I'm scratching my local itch only.
Fix the issue by adding u8 attr packing.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20230322154242.1739136-1-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The pack strings use 'b' signed char for cmd and version but struct
genlmsghdr defines them as unsigned char. Use 'B' instead.
Fixes: 4e4480e89c ("tools: ynl: move the cli and netlink code around")
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230319193803.97453-1-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Jiri suggests it reads more naturally to skip the explicit
array size when possible. When we export the symbol we want
to make sure that the size is right but for statics its
not needed.
Link: https://lore.kernel.org/r/20230321044159.1031040-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The (only recently documented) expectation is that all specs
are under a certain license, but we don't actually enforce it.
What's worse we then go ahead and assume the license was right,
outputting the expected license into generated files.
Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I relicensed Netlink spec code to GPL-2.0 OR BSD-3-Clause but
we still put a slightly different license on the uAPI header
than the rest of the code. Use the Linux-syscall-note on all
the specs and all generated code. It's moot for kernel code,
but should not hurt. This way the licenses match everywhere.
Cc: Chuck Lever <chuck.lever@oracle.com>
Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
definitions are optional, commit in question breaks cli for ethtool.
Fixes: 6517a60b03 ("tools: ynl: move the enum classes to shared code")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Fix get_mask utility routine in order to take into account possible gaps
in the elements list.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Properly manage render-max property for flags definition type
introducing mask value and setting it to (last_element << 1) - 1
instead of adding max value set to last_element + 1
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Lorenzo points out that the generic CLI is broken for the netdev
family. When I added the support for documentation of enums
(and sparse enums) the client script was not updated.
It expects the values in enum to be a list of names,
now it can also be a dict (YAML object).
Reported-by: Lorenzo Bianconi <lorenzo@kernel.org>
Fixes: e4b48ed460 ("tools: ynl: add a completely generic client")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I was intending to make all the Netlink Spec code BSD-3-Clause
to ease the adoption but it appears that:
- I fumbled the uAPI and used "GPL WITH uAPI note" there
- it gives people pause as they expect GPL in the kernel
As suggested by Chuck re-license under dual. This gives us benefit
of full BSD freedom while fulfilling the broad "kernel is under GPL"
expectations.
Link: https://lore.kernel.org/all/20230304120108.05dd44c5@kernel.org/
Link: https://lore.kernel.org/r/20230306200457.3903854-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Pretty much all families use value: 1 or reserve as unspec
the first entry in attribute set and the first operation.
Make this the default. Update documentation (the doc for
values of operations just refers back to doc for attrs
so updating only attrs).
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
To avoid having to repeat the entire definition of an attribute
(including the value) use the Attr object from the original set.
In fact this is already the documented expectation.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Python will generate its customary cache when running ynl scripts:
?? tools/net/ynl/lib/__pycache__/
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
traceback.print_exception() seems tricky to call, we're missing
some argument, so re-raise instead.
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: 3aacf82813 ("tools: ynl: add an object hierarchy to represent parsed spec")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Chuck run into an issue with a single-element attr-set which
only has an attr with value of 0. The search for max attr in
a struct records attrs with value larger than 0 only (max_val
is set to 0 at the start). Adjust the comparison, alternatively
max_val could be init'ed to -1. Somehow picking the last attr
of a value seems like a good idea in general.
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The scripts require Python 3 and some distros are dropping
Python 2 support.
Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The CLI script tries to validate jsonschema by default.
It's seems better to validate too many times than too few.
However, when copying the scripts to random servers having
to install jsonschema is tedious. Load jsonschema via
importlib, and let the user opt out.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When I wrote the first version of the Python code I was quite
excited that we can generate class methods directly from the
spec. Unfortunately we need to use valid identifiers for method
names (specifically no dashes are allowed). Don't reuse those
names on the CLI, it's much more natural to use the operation
names exactly as listed in the spec.
Instead of:
./cli --do rings_get
use:
./cli --do rings-get
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
One of my favorite features of the Netlink specs is that they
make decoding structured extack a ton easier.
Implement pretty printing bad attribute names in YNL.
For example it will now say:
'bad-attr': '.header.flags'
rather than the useless:
'bad-attr-offs': 32
Proof:
$ ./cli.py --spec ethtool.yaml --do rings_get \
--json '{"header":{"dev-index":1, "flags":4}}'
Netlink error: Invalid argument
nl_len = 68 (52) nl_flags = 0x300 nl_type = 2
error: -22 extack: {'msg': 'reserved bit set',
'bad-attr': '.header.flags'}
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
There's a lot of copy and pasting going on between the "cli"
and code gen when it comes to representing the parsed spec.
Create a library which both can use.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Move the CLI code out of samples/ and the library part
of it into tools/net/ynl/lib/. This way we can start
sharing some code with the code gen.
Initially I thought that code gen is too C-specific to
share anything but basic stuff like calculating values
for enums can easily be shared.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
An earlier fix tried to address generated code jumping around
one code-gen run to another. Turns out dict()s are already
ordered since Python 3.7, the problem is that we iterate over
operation modes using a set(). Sets are unordered in Python.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When rendering code we should walk the ops in the order in which
they are declared in the spec. This is both more intuitive and
prevents code from jumping around when hashing in the dict changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ops_list contains all the operations, but the main iteration use
case is to walk only ops which define attrs. Rename ops_list to
msg_list, because now it looks like the contents are the same,
just the format is different. While at it convert from tuple
to just keys, none of the users care about the name of the op.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Lorenzo reports that after switching from enum to flags netdev
family lost ability to render kdoc (and the enum contents got
generally garbled).
Combine the flags and enum handling in uAPI handling.
Reported-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a CLI sample which can take in arbitrary request
in JSON format, convert it to Netlink and do the inverse
for output.
It's meant as a development tool primarily and perhaps
for selftests which need to tickle netlink in a special way.
Acked-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Code generators to turn Netlink specs into C code.
I'm definitely not proud of it.
The main generator is in Python, there's a bash script
to regen all code-gen'ed files in tree after making
spec changes.
Acked-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
We currently only have BPF tools in the tools/net directory.
We are about to add more BPF tools there, not necessarily
networking related, rename the directory and related Makefile
targets to bpf.
Suggested-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Simon Horman <simon.horman@netronome.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Dynamically allocate memory so that JIT images larger than the size of
the statically allocated array can be handled.
Signed-off-by: David Daney <david.daney@cavium.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
When debugging the JIT on an embedded platform or cross build
environment, libbfd may not be available, making it impossible to run
bpf_jit_disasm natively.
Add an option to emit a binary image of the JIT code to a file. This
file can then be disassembled off line. Typical usage in this case
might be (pasting mips64 dmesg output to cat command):
$ cat > jit.raw
$ bpf_jit_disasm -f jit.raw -O jit.bin
$ mips64-linux-gnu-objdump -D -b binary -m mips:isa64r2 -EB jit.bin
Signed-off-by: David Daney <david.daney@cavium.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
klogctl can fail and return -ve len, so check for this and
return NULL to avoid passing a (size_t)-1 to malloc.
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
We can already use yylval in the lexer for encoding the BPF extension
number, so that the parser rules can be further reduced to a single one
for each B/H/W case.
Signed-off-by: Ray Bellis <ray@isc.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch fixes the checkpatch.pl error to bpf_dbg.c:
ERROR: do not initialise statics to 0
Signed-off-by: Wei Tang <tangwei@cmss.chinamobile.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use the local uapi headers to keep in sync with "recently" added #define's
(e.g. SKF_AD_VLAN_TPID). Refactored CFLAGS, and bpf_asm doesn't need -I.
Fixes: 3f356385e8 ("filter: bpf_asm: add minimal bpf asm tool")
Signed-off-by: Kamal Mostafa <kamal@canonical.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
The function returns always non-negative values.
The problem has been detected using proposed semantic patch
scripts/coccinelle/tests/assign_signed_to_unsigned.cocci [1].
[1]: http://permalink.gmane.org/gmane.linux.kernel/2046107
Signed-off-by: Andrzej Hajda <a.hajda@samsung.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support to read the dmesg BPF JIT dump also from a
file instead of the klog buffer. I found this quite useful when going
through some 'before/after patch' logs. It also fixes a regex leak
found by valgrind when no image dump was found.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
With recent debugging, I noticed that bpf_jit_disasm segfaults when
there's no debugging output from the JIT compiler to the kernel log.
Reason is that when regexec(3) doesn't match on anything, start/end
offsets are not being filled out and contain some uninitialized garbage
from stack. Thus, we need zero out offsets first.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
We now have K_VLANT, K_VLANP and K_VLANTPID. Clean them up into more
descriptive token, namely K_VLAN_TCI, K_VLAN_AVAIL and K_VLAN_TPID.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
If vlan offloading takes place then vlan header is removed from frame
and its contents, both vlan_tci and vlan_proto, is available to user
space via TPACKET interface. However, only vlan_tci can be used in BPF
filters.
This commit introduces a new BPF extension. It makes possible to load
the value of vlan_proto (vlan TPID) to register A. Support for classic
BPF and eBPF is being added, analogous to skb->protocol.
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Jiri Pirko <jpirko@redhat.com>
Signed-off-by: Michal Sekletar <msekleta@redhat.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Reviewed-by: Jiri Pirko <jiri@resnulli.us>
Signed-off-by: David S. Miller <davem@davemloft.net>
JITed seccomp filters can be quite large if they check a lot of syscalls
Simply increase buffer size
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
Acked-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Conflicts:
drivers/net/ethernet/altera/altera_sgdma.c
net/netlink/af_netlink.c
net/sched/cls_api.c
net/sched/sch_api.c
The netlink conflict dealt with moving to netlink_capable() and
netlink_ns_capable() in the 'net' tree vs. supporting 'tc' operations
in non-init namespaces. These were simple transformations from
netlink_capable to netlink_ns_capable.
The Altera driver conflict was simply code removal overlapping some
void pointer cast cleanups in net-next.
Signed-off-by: David S. Miller <davem@davemloft.net>
The AND instruction is erroneously using the X register instead
of the K register.
Signed-off-by: Brendan Hickey <bhickey@google.com>
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Added a new ancillary load (bpf call in eBPF parlance) that produces
a 32-bit random number. We are implementing it as an ancillary load
(instead of an ISA opcode) because (a) it is simpler, (b) allows easy
JITing, and (c) seems more in line with generic ISAs that do not have
"get a random number" as a instruction, but as an OS call.
The main use for this ancillary load is to perform random packet sampling.
Signed-off-by: Chema Gonzalez <chema@google.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Acked-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Conflicts:
drivers/net/usb/r8152.c
drivers/net/xen-netback/netback.c
Both the r8152 and netback conflicts were simple overlapping
changes.
Signed-off-by: David S. Miller <davem@davemloft.net>
Fixes the following build problem with binutils-2.24
gcc -Wall -O2 -c -o bpf_jit_disasm.o bpf_jit_disasm.c
In file included from bpf_jit_disasm.c:25:0:
/usr/include/bfd.h:35:2: error: #error config.h must be included
before this header
#error config.h must be included before this header
This is similar to commit 3ce711a6ab
"perf tools: bfd.h/libbfd detection fails with recent binutils"
See: https://sourceware.org/bugzilla/show_bug.cgi?id=14243
CC: David S. Miller <davem@davemloft.net>
CC: Daniel Borkmann <dborkman@redhat.com>
CC: netdev@vger.kernel.org
Acked-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Lets clean up bpf_dbg a bit and improve its code slightly
in various areas: i) Get rid of some macros as there's no
good reason for keeping them, ii) remove one unused variable
and reduce scope of various variables found by cppcheck,
iii) Close non-default file descriptors when exiting the shell.
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Valgrind found that extracted labels that are passed from the lexer
weren't freed upon exit. Therefore, add a small helper function that
walks label tables and frees them. Since also NULL can be passed to
free(3), we do not need to take care of that here. While at it, fix
up a spacing error in bpf_set_curr_label().
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
We must not leave the socket intact in bpf_runnable(). The socket
is used to test if the filter code is being accepted by the kernel
or not. So right after we do the setsockopt(2), we need to close
it again.
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
There are a couple of valid use cases for a minimal low-level bpf asm
like tool, for example, using/linking to libpcap is not an option, the
required BPF filters use Linux extensions that are not supported by
libpcap's compiler, a filter might be more complex and not cleanly
implementable with libpcap's compiler, particular filter codes should
be optimized differently than libpcap's internal BPF compiler does,
or for security audits of emitted BPF JIT code for prepared set of BPF
instructions resp. BPF JIT compiler development in general.
Then, in such cases writing such a filter in low-level syntax can be
an good alternative, for example, xt_bpf and cls_bpf users might have
requirements that could result in more complex filter code, or one that
cannot be expressed with libpcap (e.g. different return codes in
cls_bpf for flowids on various BPF code paths).
Moreover, BPF JIT implementors may wish to manually write test cases
in order to verify the resulting JIT image, and thus need low-level
access to BPF code generation as well. Therefore, complete the available
toolchain for BPF with this small bpf_asm helper tool for the tools/net/
directory. These 3 complementary minimal helper tools round up and
facilitate BPF development.
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds a minimal BPF debugger that "emulates" the kernel's
BPF engine (w/o extensions) and allows for single stepping (forwards
and backwards through BPF code) or running with >=1 breakpoints through
selected or all packets from a pcap file with a provided user filter
in order to facilitate verification of a BPF program. When a breakpoint
is being hit, it dumps all register contents, decoded instructions and
in case of branches both decoded branch targets as well as other useful
information.
Having this facility is in particular useful to verify BPF programs
against given test traffic *before* attaching to a live system.
With the general availability of cls_bpf, xt_bpf, socket filters,
team driver and e.g. PTP code, all BPF users, quite often a single
more complex BPF program is being used. Reasons for a more complex
BPF program are primarily to optimize execution time for making a
verdict when multiple simple BPF programs are combined into one in
order to prevent parsing same headers multiple times. In particular,
for cls_bpf that can have various return paths for encoding flowids,
and xt_bpf to come to a fw verdict this can be the case.
Therefore, as this can result in more complex and harder to debug
code, it would be very useful to have this minimal tool for testing
purposes. It can also be of help for BPF JIT developers as filters
are "test attached" to the kernel on a temporary socket thus
triggering a JIT image dump when enabled. The tool uses an interactive
libreadline shell with auto-completion and history support.
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This is a minimal stand-alone user space helper, that allows for debugging or
verification of emitted BPF JIT images. This is in particular useful for
emitted opcode debugging, since minor bugs in the JIT compiler can be fatal.
The disassembler is architecture generic and uses libopcodes and libbfd.
How to get to the disassembly, example:
1) `echo 2 > /proc/sys/net/core/bpf_jit_enable`
2) Load a BPF filter (e.g. `tcpdump -p -n -s 0 -i eth1 host 192.168.20.0/24`)
3) Run e.g. `bpf_jit_disasm -o` to disassemble the most recent JIT code output
`bpf_jit_disasm -o` will display the related opcodes to a particular instruction
as well. Example for x86_64:
$ ./bpf_jit_disasm
94 bytes emitted from JIT compiler (pass:3, flen:9)
ffffffffa0356000 + <x>:
0: push %rbp
1: mov %rsp,%rbp
4: sub $0x60,%rsp
8: mov %rbx,-0x8(%rbp)
c: mov 0x68(%rdi),%r9d
10: sub 0x6c(%rdi),%r9d
14: mov 0xe0(%rdi),%r8
1b: mov $0xc,%esi
20: callq 0xffffffffe0d01b71
25: cmp $0x86dd,%eax
2a: jne 0x000000000000003d
2c: mov $0x14,%esi
31: callq 0xffffffffe0d01b8d
36: cmp $0x6,%eax
[...]
5c: leaveq
5d: retq
$ ./bpf_jit_disasm -o
94 bytes emitted from JIT compiler (pass:3, flen:9)
ffffffffa0356000 + <x>:
0: push %rbp
55
1: mov %rsp,%rbp
48 89 e5
4: sub $0x60,%rsp
48 83 ec 60
8: mov %rbx,-0x8(%rbp)
48 89 5d f8
c: mov 0x68(%rdi),%r9d
44 8b 4f 68
10: sub 0x6c(%rdi),%r9d
44 2b 4f 6c
[...]
5c: leaveq
c9
5d: retq
c3
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
