Re: DPDK - TX from lcore in packet distributor configuration

21.01.2015, 20:16, "deco33000 Jog" [off-list ref]:

21.01.2015, 17:43, "Bruce Richardson" [off-list ref]:

quoted

 On Wed, Jan 21, 2015 at 03:00:50PM +0100, deco33000 Jog wrote:

quoted

  Hello,

  -- PROBLEM
  I have a AF_PACKET socket which is in promiscuous mode to get all the NIC traffic and let my apps do the whole stuff.

  So I have one receiver and need to communicate the packet to different threads/processes (lcore) so that they can process the rest of the packet (tcp/udp...)

  -- QUESTION
  I read about the packet distributor architecture which seems to answer that need.
  http://dpdk.org/doc/guides/prog_guide/packet_distrib_lib.html

  BUT i fear that it be slow at resending the packet to the distributor which may already be overloaded by inputs from the net. Why pass back the answer to the distributor if the lcore could send to the wire directly ?

  My problem is going back to the distributor after the packet processing. i would a direct send to the tx ring.

  Is it possible ? How ?  By passing the TX pointer to the lcore ?

 If you sent up the NIC so that it has multiple queues for each thread you can
 have each worker send directly to the NIC. Without multiple queues, they could
 still send directly, they will just have to use locking or some other access
 mechanism to mediate access to the common TX queue.
 The other problem with this approach is that sending packets individually is
 almost always slower than sending them in bursts. To mitigate against this,
 you could look to buffer packets inside the workers before transmitting them
 back out, but that could lead to packets being sent out of order - not sure if
 that is a problem for you or not.

 The reason the distributor sample app sends the packets back to the distributor
 after worker processing is to overcome these limitations. There is no additional
 cross-core round trip involved in sending a packet back to the distributor along
 with the request, and having the distributor re-gather the packets ensures ordering
 within flows/tags is maintained. Thereafter packets can be burst-sent out an
 ethernet port.

 Out of interest, given you are using the AF_PACKET driver, what rate of packets
 per second are you looking at?

 /Bruce

Bruce, thanks,

1)
I am looking for saturation of a 1 Gb/s link not a 10 Gb/s yet.
I want to feel at home before going higher.

I work with raw packets for security purposes.
Do you think AF_PACKET is not good enough ? Do DPDK enables another (more efficient) solution (I may have missed it)?

2)
I tried Netmap and PF_RING too; i am currently seeking for my "weapon" :)
I want to use it in a socket context for a custom webserver.

At first i thought the packet return would incur a cross core operation.
But the problem remains :
the distributor has to deal with Rx data and packets returned from the workers as well. Even if it is fast, I think I prefer to buffer + send directly form the worker to release the distributor from doing it.
i decided to avoid the Nagle algo as much as possible because i want the least latency as possible for my app. I modulate to to balance the trade-off though.