quoted

This is still overhead for every packet on the switches we support. The
number of ports can go close to 128
(40G ports can be broken into 4x10G ports).

Agreed. Given that, and since dev_get_by_index_rcu uses a hash to find the
device pointer, it may actually be (much) faster (and the above "iterate
over all ports" is a bit misleading).

I tested the above approach with DSA and a Marvell switch chip. It works,
but I am a bit concerned about the per-packet overhead, especially
in larger networks. I would prefer if there would be a means to 'catch'
duplicate packets earlier - before they are even created, if that is
possible.

Seems like we're running into issues that would be encountered by stacked
switches.  Should we consider similar solutions, i.e. a stacking protocol
like HiGig?  In this situation each packet would be passed between switches
along with a stacking data structure that indicates relevant information
such as where the packet was first received (i.e. an ingress module ID) 
and a bit indicating that it's a broadcast packet.  The receiving switch
then knows which ports the broadcast frame has already egressed and can
then flood remaining ports if necessary.  You can also create a CPU bit
if you want to forward the frame specifically to the host CPU.

It requires more upfront work but pays off later in flexibility.

Dave