Re: Port mirroring (RFC)

On 12/15/21 11:48 AM, Florian Fainelli wrote:

On 12/14/21 6:47 AM, Alex Elder wrote:

quoted

I am implementing what amounts to port mirroring functionality
for the IPA driver.

The IPA hardware isn't exactly a network switch (it's sort of
more than that), but it has the ability to supply replicas of
packets transferred within it to a special (read only) interface.

My plan is to implement this using a new "ipa_mirror" network
device, so it could be used with a raw socket to capture the
arriving packets.  There currently exists one other netdev,
which represents access through a modem to a WWAN network.

I would like some advice on how to proceed with this.  I want
the result to match "best practice" upstream, and would like
this to be as well integrated possible with existing network
tools.

A few details about the stream of packets that arrive on
this hardware interface:
- Packet data is truncated if it's larger than a certain size
- Each packet is preceded by a fixed-size header describing it
- Packets (and their headers) are aggregated into a buffer; i.e.
   a single receive might carry a dozen (truncated) packets

Andrew has already responded, but there are currently sort of two ways
that mirroring is implemented in upstream supported drivers:

- for "classic" Ethernet switches mirroring is typically from one port,
or a set of ports, to another with a choice in whether you want to
capture ingress, egress, or both towards that port. Because each port is
a netdev already you just run tcpdump/pcap the way you normally do on
your Ethernet NIC and get the captured packets. Configuration is via
offloading the tc_mired action.

I read about switch port mirroring, and it sounded along the
lines of what I'm trying to implement, which is why I suggested
this might be a case of "port mirroring."  That said, it is
not configurable in the same way.  Here, we really have a
dedicated port through which replicated packets arrive.  It is
not possible to just use one of the existing switch ports.  So
it isn't really the same.

- mlxsw implements devlink traps which is not exaclty designed for
capturing mirrored packets but more like management events such as why
the packet was trapped etc. You could however imagine using devlink
traps for that purpose of capturing mirrored packets in the absence of a
suitable network device

Just your description tells me this approach is probably not right.
But I'll look into it some more.

Not sure if this helps, but that is the situation.

Right now I find it useful to hear about anything, even things
I might not ultimately use.

quoted

Here are a few specific questions, but I would love to get
*any* feedback about what I'm doing.
- Is representing this as a separate netdev a reasonable
   thing to do?

I would think so, given this allows standard tools to work.

It's reassuring knowing there isn't disagreement.

quoted

- Is there anything wrong with making a netdev read-only?
   (Any packets supplied for transmit would be dropped)

This looks reasonable.

quoted

- Are there things I should do so it's clear this interface
   does not carry IP traffic (or even UDP, etc.)?

There were various patches in the past to prevent a network device from
getting any IP stack to be configured:

https://yhbt.net/lore/all/20150825232021.GA8482@Alexeis-MacBook-Pro-2.local/t/

I forgot whether as a driver you can just refuse to have an IP address
assigned to your network device, AFAIR it requires changes to the
network stack as proposed in the patch set.

Thanks, I'll follow up with that as well.

quoted

- Should the driver de-aggregate the received packets, i.e.
   separating each into a separate SKB for reading?

If this is truly a mirroring device, you would expect it to "render" the
mirrored packets exactly the way they are, maybe add an ethtool private
option to de-aggregate packets if this helps debugging?

I have some details to work out before I can implement such a
thing, but this would be helpful.

Thanks a lot for your input.

					-Alex

quoted

I might have *many* more questions, but I'd just like to make
sure I'm on the right track, and would like both specific and
general suggestions about how to do this the right way.

Thanks.
                     -Alex