Re: the side effect of using copy skb instead of skb_clone in cdc ncm/mbim driver

joey ming [off-list ref] writes:

Hi, I was hoping you can help me with some questions.

some qualcomm LTE modem has two mode: ECM and QC_NCM.

Yes, and worse than that: You can even mix and match these on UL and DL.

QC_NCM is the
protocol which qualcomm defined,the  difference between qc-ncm and cdc-mbim
are:
data pipe:     NDP sig vary
                   qc-ncm  packets are composed of  NTH+datagram+NTB  but
mbim/ncm are NTH+NTB+datagram

Both these schemes are allowed in NCM (and therefore MBIM). There is no
required datagram nd NTB order. Proper alignment is the only requirement.
So the only real difference is the signature, which AFAIK is
configurable on the QC_NCM devices.

control pipe: the biggest difference is control pipe. QC-NCM uses QMI cmd
to control device but mbim/ncm uses CDC cmd .

Yes, but this usage is actually also within the NCM spec.  The only
issues making a typical QC_NCM capable device fail as a CDC NCM class
device are
 - the default signature
 - the USB descriptors

Both are of course configurable by the modem vendors using these
chips/firmware toolkits, so it's possible to create standard CDC NCM
class devices out of these.  But you probably don't want to do that,
because there aren't any NCM class drivers which will let you access the
QMI control channel (except the one you have made :-), and you need QMI
for modem management.

so for a QC-NCM driver, we can borrow the code of data pipe from
cdc-mbim/ncm. for a real LTE modem(DL speed:150Mbps) the qc-ncm driver
speed which used skb_clone in qc_ncm_rxfixup  can reach 126Mbps,but at the
same time and same envirmont ecm driver can reach 144Mbps for the same
modem.
Thanks for google, I found  Bjorn Mork's discussion about buffering
restrictions of cdc-ncm host driver.
 the qc-ncm driver speed can reach 145Mbps  if  I used copy skb instead of
skb_clone.
as we know, the aggregation protocol(such as cdc-ncm,cdc-mbim,qc-ncm )
major purpose are improving performance of CPU by reducing interrupts.
Compared ecm and qc-ncm test result, I thought we didn't reach the goal.
the interrupts decreased 20% BUT the driver load was no change.
I used Intel Core Duo laptop and oprofiled in 3.12 kernel:
  QC-NCM           0.3146%(CPU0)    0.2899%(CPU1)  vmlinux
 vmlinux                  memcpy
                          0.2198%(CPU0)    0.1864%(CPU1)  usbnet
        usbnet                   usbnet
                          0.0423 %(CPU0)   0.0385%(CPU1)  qmi_wwan
     qmi_wwan              /qmi_wwan

  CDC-ECM         0.2470%(CPU0)     0.1735%(CPU1)  usbnet
usbnet                   /usbnet
                          0.1341%(CPU0)     0.0132%(CPU1)  vmlinux
         vmlinux                  memcpy
                          0.0335%(CPU0)      0.0387%(CPU1)  qc_ncm
        qc_ncm                  /qc_ncm

for a high speed modem(DL >100Mbps), Is it a appropriate solution use copy
skb? as above test result, memcpy load was higher than usbnet.

copy is the only solution.  With aggregation on the USB link we can
choose either
 - USB buffers which are much bigger than one IP packet, or
 - splitting some IP packets over two USB buffers

Either way we'll have to copy the IP packets before handing them over to
the upper layers.   Cloning oversized skbs is not a good idea.

The problem you found is that there is no real value in this USB layer
aggregation on Linux.  It's just pointless complexity. A plain ECM
driver transmitting ethernet frames back to back will achieve the exact
same utilisation of the USB link, without all the aggregation complexity
and overhead.

I don't know why the USB-IF found it necessary to create these
aggregation protocols, but I assume there are other OS's than Linux out
there (running on either hosts or devices) which are unable to use the
full USB bandwith with ECM.

But the results you show above is the primary reason why I chose to
completely ignore the QC_NCM bastard protocol.  AFAIK all devices
capable of using this protocol can also do the plain ECM, and will do so
by default.  So they are supported by the qmi_wwan driver with no more
fuzz, doing the maximum speed the devices are capable of.

Thanks a lot for testing this and providing the results!


Bjørn