cli.py currently throws a pure KeyError if kernel doesn't support
a netlink family. Users who did not write ynl (hah) may waste
their time investigating what's wrong with the Python code.
Improve the error message:
Traceback (most recent call last):
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 362, in __init__
self.family = GenlFamily(self.yaml['name'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 331, in __init__
self.genl_family = genl_family_name_to_id[family_name]
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'netdev'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kicinski/devel/linux/./tools/net/ynl/cli.py", line 52, in <module>
main()
File "/home/kicinski/devel/linux/./tools/net/ynl/cli.py", line 31, in main
ynl = YnlFamily(args.spec, args.schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/kicinski/devel/linux/tools/net/ynl/lib/ynl.py", line 364, in __init__
raise Exception(f"Family '{self.yaml['name']}' not supported by the kernel")
Exception: Family 'netdev' not supported by the kernel
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230407145609.297525-1-kuba@kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This is what I've been using to see whether the spec makes sense.
A small subset of getters (mostly the unprivileged ones) is implemented.
Some setters (channels) also work.
Setters for messages with bitmasks are not implemented.
Initially I was trying to make this tool look 1:1 like real ethtool,
but eventually gave up :-)
Sample output:
$ ./tools/net/ynl/ethtool enp0s31f6
Settings for enp0s31f6:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half
100baseT/Full 1000baseT/Full
Supported pause frame use: no
Supports auto-negotiation: yes
Supported FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 2
Transceiver: Internal
MDI-X: Unknown (auto)
Current message level: drv probe link
Link detected: no
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of dumping the error on the stdout, make the callee and
opportunity to decide what to do with it. This is mostly for the
ethtool testing.
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for netlink families that add an optional fixed header structure
after the genetlink header and before any attributes. The fixed-header can be
specified on a per op basis, or once for all operations, which serves as a
default value that can be overridden.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for decoding attributes that contain C structs.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for decoding C arrays from binay blobs in genetlink-legacy
messages.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add python classes for struct definitions to nlspec
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I was a bit too optimistic in commit bf51d27704 ("tools: ynl: fix
get_mask utility routine"), not every mask we use is necessarily
coming from an enum of type "flags". We also allow flipping an
enum into flags on per-attribute basis. That's done by
the 'enum-as-flags' property of an attribute.
Restore this functionality, it's not currently used by any in-tree
family.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
While testing the tool I noticed we miss the u16 type on payload create.
On the code inspection it turned out we miss also u64 - add them.
We also miss the decoding of u16 despite the fact `NlAttr` class
supports it - add it.
Signed-off-by: Michal Michalik <michal.michalik@intel.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
It is a good practice to state explicitly which are the required Python
packages needed in a particular project to run it. The most commonly
used way is to store them in the `requirements.txt` file*.
*URL: https://pip.pypa.io/en/stable/reference/requirements-file-format/
Currently user needs to figure out himself that Python needs `PyYAML`
and `jsonschema` (and theirs requirements) packages to use the tool.
Add the `requirements.txt` for user convenience.
How to use it:
1) (optional) Create and activate empty virtual environment:
python3.X -m venv venv3X
source ./venv3X/bin/activate
2) Install all the required packages:
pip install -r requirements.txt
or
python -m pip install -r requirements.txt
3) Run the script!
The `requirements.txt` file was tested for:
* Python 3.6
* Python 3.8
* Python 3.10
Signed-off-by: Michal Michalik <michal.michalik@intel.com>
Link: https://lore.kernel.org/r/20230323190802.32206-1-michal.michalik@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Playing with dpll netlink, I came across following issue:
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/dpll.yaml --do pin-set --json '{"id": 0, "pin-idx": 1, "pin-state": 1}'
Traceback (most recent call last):
File "tools/net/ynl/cli.py", line 52, in <module>
main()
File "tools/net/ynl/cli.py", line 40, in main
reply = ynl.do(args.do, attrs)
File "tools/net/ynl/lib/ynl.py", line 520, in do
return self._op(method, vals)
File "tools/net/ynl/lib/ynl.py", line 476, in _op
msg += self._add_attr(op.attr_set.name, name, value)
File "tools/net/ynl/lib/ynl.py", line 344, in _add_attr
raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}')
Exception: Unknown type at dpll pin-state 1 u8
I'm not that familiar with ynl code, but from a quick peek, I suspect
that couple other types are missing for both encoding and decoding.
Ignoring those here as I'm scratching my local itch only.
Fix the issue by adding u8 attr packing.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20230322154242.1739136-1-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The pack strings use 'b' signed char for cmd and version but struct
genlmsghdr defines them as unsigned char. Use 'B' instead.
Fixes: 4e4480e89c ("tools: ynl: move the cli and netlink code around")
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230319193803.97453-1-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Jiri suggests it reads more naturally to skip the explicit
array size when possible. When we export the symbol we want
to make sure that the size is right but for statics its
not needed.
Link: https://lore.kernel.org/r/20230321044159.1031040-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The (only recently documented) expectation is that all specs
are under a certain license, but we don't actually enforce it.
What's worse we then go ahead and assume the license was right,
outputting the expected license into generated files.
Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I relicensed Netlink spec code to GPL-2.0 OR BSD-3-Clause but
we still put a slightly different license on the uAPI header
than the rest of the code. Use the Linux-syscall-note on all
the specs and all generated code. It's moot for kernel code,
but should not hurt. This way the licenses match everywhere.
Cc: Chuck Lever <chuck.lever@oracle.com>
Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
definitions are optional, commit in question breaks cli for ethtool.
Fixes: 6517a60b03 ("tools: ynl: move the enum classes to shared code")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Fix get_mask utility routine in order to take into account possible gaps
in the elements list.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Properly manage render-max property for flags definition type
introducing mask value and setting it to (last_element << 1) - 1
instead of adding max value set to last_element + 1
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Lorenzo points out that the generic CLI is broken for the netdev
family. When I added the support for documentation of enums
(and sparse enums) the client script was not updated.
It expects the values in enum to be a list of names,
now it can also be a dict (YAML object).
Reported-by: Lorenzo Bianconi <lorenzo@kernel.org>
Fixes: e4b48ed460 ("tools: ynl: add a completely generic client")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I was intending to make all the Netlink Spec code BSD-3-Clause
to ease the adoption but it appears that:
- I fumbled the uAPI and used "GPL WITH uAPI note" there
- it gives people pause as they expect GPL in the kernel
As suggested by Chuck re-license under dual. This gives us benefit
of full BSD freedom while fulfilling the broad "kernel is under GPL"
expectations.
Link: https://lore.kernel.org/all/20230304120108.05dd44c5@kernel.org/
Link: https://lore.kernel.org/r/20230306200457.3903854-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Pretty much all families use value: 1 or reserve as unspec
the first entry in attribute set and the first operation.
Make this the default. Update documentation (the doc for
values of operations just refers back to doc for attrs
so updating only attrs).
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
To avoid having to repeat the entire definition of an attribute
(including the value) use the Attr object from the original set.
In fact this is already the documented expectation.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Python will generate its customary cache when running ynl scripts:
?? tools/net/ynl/lib/__pycache__/
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
traceback.print_exception() seems tricky to call, we're missing
some argument, so re-raise instead.
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: 3aacf82813 ("tools: ynl: add an object hierarchy to represent parsed spec")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Chuck run into an issue with a single-element attr-set which
only has an attr with value of 0. The search for max attr in
a struct records attrs with value larger than 0 only (max_val
is set to 0 at the start). Adjust the comparison, alternatively
max_val could be init'ed to -1. Somehow picking the last attr
of a value seems like a good idea in general.
Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The scripts require Python 3 and some distros are dropping
Python 2 support.
Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The CLI script tries to validate jsonschema by default.
It's seems better to validate too many times than too few.
However, when copying the scripts to random servers having
to install jsonschema is tedious. Load jsonschema via
importlib, and let the user opt out.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When I wrote the first version of the Python code I was quite
excited that we can generate class methods directly from the
spec. Unfortunately we need to use valid identifiers for method
names (specifically no dashes are allowed). Don't reuse those
names on the CLI, it's much more natural to use the operation
names exactly as listed in the spec.
Instead of:
./cli --do rings_get
use:
./cli --do rings-get
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
One of my favorite features of the Netlink specs is that they
make decoding structured extack a ton easier.
Implement pretty printing bad attribute names in YNL.
For example it will now say:
'bad-attr': '.header.flags'
rather than the useless:
'bad-attr-offs': 32
Proof:
$ ./cli.py --spec ethtool.yaml --do rings_get \
--json '{"header":{"dev-index":1, "flags":4}}'
Netlink error: Invalid argument
nl_len = 68 (52) nl_flags = 0x300 nl_type = 2
error: -22 extack: {'msg': 'reserved bit set',
'bad-attr': '.header.flags'}
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
There's a lot of copy and pasting going on between the "cli"
and code gen when it comes to representing the parsed spec.
Create a library which both can use.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Move the CLI code out of samples/ and the library part
of it into tools/net/ynl/lib/. This way we can start
sharing some code with the code gen.
Initially I thought that code gen is too C-specific to
share anything but basic stuff like calculating values
for enums can easily be shared.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
An earlier fix tried to address generated code jumping around
one code-gen run to another. Turns out dict()s are already
ordered since Python 3.7, the problem is that we iterate over
operation modes using a set(). Sets are unordered in Python.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When rendering code we should walk the ops in the order in which
they are declared in the spec. This is both more intuitive and
prevents code from jumping around when hashing in the dict changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ops_list contains all the operations, but the main iteration use
case is to walk only ops which define attrs. Rename ops_list to
msg_list, because now it looks like the contents are the same,
just the format is different. While at it convert from tuple
to just keys, none of the users care about the name of the op.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Lorenzo reports that after switching from enum to flags netdev
family lost ability to render kdoc (and the enum contents got
generally garbled).
Combine the flags and enum handling in uAPI handling.
Reported-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a CLI sample which can take in arbitrary request
in JSON format, convert it to Netlink and do the inverse
for output.
It's meant as a development tool primarily and perhaps
for selftests which need to tickle netlink in a special way.
Acked-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Code generators to turn Netlink specs into C code.
I'm definitely not proud of it.
The main generator is in Python, there's a bash script
to regen all code-gen'ed files in tree after making
spec changes.
Acked-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
