st 6. 5. 2026 v 5:33 odesílatel Stephen Hemminger <
stephen@networkplumber.org> napsal:

Background
----------

Multiple efforts over the past few cycles have tried to make
testpmd's flow rule grammar reusable from outside testpmd.
External applications that need rte_flow want a documented way
to turn human-written rules into the rte_flow_attr/item/action
arrays accepted by rte_flow_create().

The most recent attempt is Lukas Sismis's series, currently at
v12:

  http://patches.dpdk.org/project/dpdk/list/?series=37384  (or
  most recent thread on dev@dpdk.org)

That series factors testpmd's existing cmdline_flow.c into a
library and updates testpmd to consume it.  It works, but
inherits two properties of cmdline_flow.c that I think are worth
avoiding in a reusable library:

  - Coupling to librte_cmdline.  Even after the v12 split into
    a "simple" part and a "cmdline" part, the parser is still
    organized around testpmd's command interpreter, and v12 has
    cmdline depending on ethdev to break a previous circular
    dependency.  A library used by daemons, control planes, or
    unit tests should not need that.

  - Ad-hoc grammar.  cmdline_flow.c implements parsing per-token
    in long dispatch logic; the grammar emerges from the code
    rather than being stated, and adding a new flow item
    requires touching the parser.

This RFC explores a different shape and is posted to ask the
list which one is preferred before more work goes into either.

So I started a new green field library for parsing flow rules
(with the help of AI assistance). It is only a few hours old,
but passes tests and passes review.

This series
-----------

lib/flow_compile -- a small new library providing the same
service via a pcap_compile()-style API:

    char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE];
    struct rte_flow_compile *fc = rte_flow_compile(rule, errbuf);
    if (fc == NULL)
            fail(errbuf);            /* "line:col: message" */

    rte_flow_compile_create(port_id, fc, &flow_error);
    rte_flow_compile_free(fc);

Design properties:

  - Hand-rolled lexer + recursive descent parser.  No flex/bison.
  - Parser is driven entirely by descriptor tables of items and
    actions.  Adding a new flow item is a table edit, not a
    parser change.  A custom-setter hook on each field covers
    the layouts that do not fit a plain byte range (bitfields,
    indirect arrays).
  - Dependencies: rte_ethdev and rte_net only.  No librte_cmdline,
    no flex/bison.  Builds clean on Linux, FreeBSD, and Windows.
  - Per-allocation rte_zmalloc for spec/mask/last/conf payloads;
    rte_flow_compile_free() walks the pattern and action arrays.
    ASan/LSan run clean on the autotest.

The grammar follows testpmd's syntax closely so familiar rules
carry over:

    ingress pattern eth / ipv4 src is 10.0.0.1 / end
    actions queue index 3 / count / end

and is documented as a formal BNF in the programmer's guide
chapter (patch 2).

Initial coverage: eth, vlan, ipv4, ipv6, tcp, udp, vxlan,
port_id, port_representor, represented_port items; drop,
passthru, queue, mark, jump, count, port_id and representor
variants, of_pop_vlan, vxlan_decap actions.  Variable-conf
items and actions (RSS, RAW) require custom setters and are
deferred to a follow-up.

What this RFC is *not*
----------------------

Not a replacement for cmdline_flow.c in testpmd.  If the shape
here is acceptable, the next step is a separate series adding a
"flow compile <port> <rule>" command in testpmd alongside the
existing parser, so users can adopt the library incrementally
without breaking scripts that depend on the current syntax.

What I'd like feedback on
-------------------------

Hi Stephen,

Thank you for looking into this. I genuinely appreciate it.
I found this almost by accident, so here are some of my answers:

1. API shape.  pcap_compile-style (one string -> opaque object ->
   arrays) versus the three-call attr/pattern/actions form
   Sismis's v12 exposes.  What does your application actually
   want?

Generally, I am fine with pcap_compile style as per your suggestions.
The original intent was to only use traffic patterns because everything
else was set by the application itself. You can see an example of the use
here:
https://github.com/OISF/suricata/pull/15290/changes#diff-4d5f5239a7124ed4b4d45501788bddffe803911f19c157e3c4ac7676a84d68b1R301


That is why I divided the simple API into 3 calls -- to let the application
decide which part of the flow expression to parse. But a single entry point
to the library is fine too, as the application could just fill it into the
desired rule.

2. Library placement.  Stand-alone at lib/flow_compile/ versus

   addition to lib/ethdev.  This series treats it as a
   control-path parser layered on top of ethdev rather than
   part of ethdev itself; v12 places its parser inside ethdev.

My general take on this is that it can be so useful for developers, so it
should be part of the lib/ethdev to make it more visible. As highlighted by
Thomas, it heavily interacts with the rte_flow objects.
But, as it probes for the flex/bison tools and skips the library if they
are unavailable, wouldn't it be a problem if it is part of lib/ethdev
directly? This can be a very strong argument for a separate library, then.

3. Table-driven extension model.  Is "to add a new flow item,
   add a row to the descriptor table" the right contract?
   Should the tables live alongside each rte_flow_item_*
   definition in rte_flow.h, or in their own file as here?

4. Convergence.  If this design is preferred, I'm happy to
   coordinate with Lukas to fold in the testpmd-side changes
   from his series.

(My) original idea was to reuse one parser for both the testpmd and the
library to avoid multiple implementations of essentially the same thing.

How would "flow compile" be useful if testpmd already has the "flow create"
command? Perhaps as a library demo only? I would expect testpmd users to
like the `flow create` endpoint more for its more comprehensive support.
Similarly, at this moment, since testpmd supports interactivity, I cannot
see how the `_compile` API would replace functions in testpmd. But perhaps
it would be part of a greater effort to migrate to flex/bison solution.
Since this is also a simple, single-rule, stateless parser, it is not fully
compatible with all of TestPMD's commands (e.g., "set raw_encap"). (Ok,
noticed, you highlighted this)
Perhaps it is fine, just wanted to highlight this.

From the user-as-an-engineer's perspective, I would be generally happy for
this parser to exist. The drop-filter feature in Suricata, in my view,
requires just a simple network pattern specification so looking at the
supported patterns already ticks off most of the items I've had in mind,
except, e.g., MPLS. I can also see this as a viable way for Suricata
user-specified decap options for better applicability of RSS.

5. API code. Is it readable enough. Lots of code here
   but doing lexer/parser is prime template territory
   and AI has a good chance here.

Stephen Hemminger (3):
  flow_compile: introduce textual flow rule compiler
  doc: add programmer's guide for flow rule compiler
  test/flow_compile: add unit tests for flow rule compiler

 MAINTAINERS                                    |    8 +
 app/test/meson.build                           |    2 +
 app/test/test_flow_compile.c                   |  230 +++
 doc/guides/prog_guide/flow_compile_lib.rst     |  170 ++
 doc/guides/prog_guide/index.rst                |    1 +
 doc/guides/rel_notes/release_26_07.rst         |    5 +
 lib/flow_compile/flow_compile_lex.c            |  488 +++++
 lib/flow_compile/flow_compile_parse.c          |  634 ++++++
 lib/flow_compile/flow_compile_priv.h           |  181 ++
 lib/flow_compile/flow_compile_tables.c         |  245 +++
 lib/flow_compile/meson.build                   |   15 +
 lib/flow_compile/rte_flow_compile.h            |  158 ++
 lib/flow_compile/rte_flow_compile_api.c        |  132 ++
 lib/flow_compile/version.map                   |   13 +
 lib/meson.build                                |    1 +
 15 files changed, 2283 insertions(+)
 create mode 100644 app/test/test_flow_compile.c
 create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst
 create mode 100644 lib/flow_compile/flow_compile_lex.c
 create mode 100644 lib/flow_compile/flow_compile_parse.c
 create mode 100644 lib/flow_compile/flow_compile_priv.h
 create mode 100644 lib/flow_compile/flow_compile_tables.c
 create mode 100644 lib/flow_compile/meson.build
 create mode 100644 lib/flow_compile/rte_flow_compile.h
 create mode 100644 lib/flow_compile/rte_flow_compile_api.c
 create mode 100644 lib/flow_compile/version.map

--
2.43.7


Stephen Hemminger (3):
  flow_compile: introduce textual flow rule compiler
  doc: add programmer's guide for flow rule compiler
  test/flow_compile: add unit tests for flow rule compiler

 MAINTAINERS                                |   7 +
 app/test/meson.build                       |   1 +
 app/test/test_flow_compile.c               | 234 ++++++++
 doc/guides/prog_guide/flow_compile_lib.rst | 272 +++++++++
 doc/guides/prog_guide/index.rst            |   1 +
 doc/guides/rel_notes/release_26_07.rst     |   6 +
 lib/flow_compile/flow_compile_lex.c        | 510 +++++++++++++++++
 lib/flow_compile/flow_compile_parse.c      | 634 +++++++++++++++++++++
 lib/flow_compile/flow_compile_priv.h       | 181 ++++++
 lib/flow_compile/flow_compile_tables.c     | 245 ++++++++
 lib/flow_compile/meson.build               |  15 +
 lib/flow_compile/rte_flow_compile.h        | 158 +++++
 lib/flow_compile/rte_flow_compile_api.c    | 132 +++++
 lib/meson.build                            |   1 +
 14 files changed, 2397 insertions(+)
 create mode 100644 app/test/test_flow_compile.c
 create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst
 create mode 100644 lib/flow_compile/flow_compile_lex.c
 create mode 100644 lib/flow_compile/flow_compile_parse.c
 create mode 100644 lib/flow_compile/flow_compile_priv.h
 create mode 100644 lib/flow_compile/flow_compile_tables.c
 create mode 100644 lib/flow_compile/meson.build
 create mode 100644 lib/flow_compile/rte_flow_compile.h
 create mode 100644 lib/flow_compile/rte_flow_compile_api.c

--
2.53.0