Re: [RFC PATCH net-next 4/4] net: wwan: Add Qualcomm BAM-DMUX WWAN network driver

Hello Stephan,

On Mon, Jul 19, 2021 at 6:01 PM Stephan Gerhold [off-list ref] wrote:

The BAM Data Multiplexer provides access to the network data channels of
modems integrated into many older Qualcomm SoCs, e.g. Qualcomm MSM8916 or
MSM8974. It is built using a simple protocol layer on top of a DMA engine
(Qualcomm BAM) and bidirectional interrupts to coordinate power control.

The modem announces a fixed set of channels by sending an OPEN command.
The driver exports each channel as separate network interface so that
a connection can be established via QMI from userspace. The network
interface can work either in Ethernet or Raw-IP mode (configurable via
QMI). However, Ethernet mode seems to be broken with most firmwares
(network packets are actually received as Raw-IP), therefore the driver
only supports Raw-IP mode.

The driver uses runtime PM to coordinate power control with the modem.
TX/RX buffers are put in a kind of "ring queue" and submitted via
the bam_dma driver of the DMAEngine subsystem.

The basic architecture looks roughly like this:

                   +------------+                +-------+
         [IPv4/6]  |  BAM-DMUX  |                |       |
         [Data...] |            |                |       |
        ---------->|rmnet0      | [DMUX chan: x] |       |
         [IPv4/6]  | (chan: 0)  | [IPv4/6]       |       |
         [Data...] |            | [Data...]      |       |
        ---------->|rmnet1      |--------------->| Modem |
                   | (chan: 1)  |      BAM       |       |
         [IPv4/6]  | ...        |  (DMA Engine)  |       |
         [Data...] |            |                |       |
        ---------->|rmnet7      |                |       |
                   | (chan: 7)  |                |       |
                   +------------+                +-------+

However, on newer SoCs/firmware versions Qualcomm began gradually moving
to QMAP (rmnet driver) as backend-independent protocol for multiplexing
and data aggegration. Some firmware versions allow using QMAP on top of
BAM-DMUX (effectively resulting in a second multiplexing layer plus data
aggregation). The architecture with QMAP would look roughly like this:

           +-------------+           +------------+                  +-------+
 [IPv4/6]  |    RMNET    |           |  BAM-DMUX  |                  |       |
 [Data...] |             |           |            | [DMUX chan: 0]   |       |
---------->|rmnet_data1  |     ----->|rmnet0      | [QMAP mux-id: x] |       |
           | (mux-id: 1) |     |     | (chan: 0)  | [IPv4/6]         |       |
           |             |     |     |            | [Data...]        |       |
 [IPv4/6]  | ...         |------     |            |----------------->| Modem |
 [Data...] |             |           |            |       BAM        |       |
---------->|rmnet_data42 | [QMAP: x] |[rmnet1]    |   (DMA Engine)   |       |
           | (mux-id: 42)| [IPv4/6]  |... unused! |                  |       |
           |             | [Data...] |[rmnet7]    |                  |       |
           |             |           |            |                  |       |
           +-------------+           +------------+                  +-------+

In this case, rmnet1-7 would remain unused. The firmware used on the most
recent SoCs with BAM-DMUX even seems to announce only a single BAM-DMUX
channel (rmnet0), which makes QMAP the only option for multiplexing there.

So far the driver is mainly tested on various smartphones/tablets based on
Qualcomm MSM8916/MSM8974 without QMAP. It looks like QMAP depends on a MTU
negotiation feature in BAM-DMUX which is not yet supported by the driver.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
---
Note that this is my first network driver, so I apologize in advance
if I made some obvious mistakes. :)

I'm not sure how to integrate the driver with the WWAN subsystem yet.
At the moment the driver creates network interfaces for all channels
announced by the modem, it does not make use of the WWAN link management
yet. Unfortunately, this is a bit complicated:

Both QMAP and the built-in multiplexing layer might be needed at some point.
There are firmware versions that do not support QMAP and the other way around
(the built-in multiplexing was disabled on very recent firmware versions).
Only userspace can check if QMAP is supported in the firmware (via QMI).

I am not very familiar with the Qualcomm protocols and am just curious
whether BAM-DMUX has any control (management) channels or only IPv4/v6
data channels?

The WWAN subsystem began as a framework for exporting management
interfaces (MBIM, AT, etc.) to user space. And then the network
interfaces (data channels) management interface was added to
facilitate management of devices with multiple data channels. That is
why I am curious about the BAM-DMUX device management interface or in
other words, how a user space application could control the modem
work?

I could ignore QMAP completely for now but I think someone will show up
who will need this eventually. And if there is going to be common code for
QMAP/rmnet link management it would be nice if BAM-DMUX could also make
use of it.

But the question is, how could this look like? How do we know if we should
create a link for QMAP or a BAM-DMUX channel? Does it even make sense
to manage the 1-8 channels via the WWAN link management?

Another problem is that the WWAN subsystem currently creates all network
interfaces below the common WWAN device. This means that userspace like
ModemManager has no way to check which driver provides them. This is
necessary though to decide how to set it up via QMI (ModemManager uses it).

For reference, example of the channels announced by firmwares on various SoCs:
  - Qualcomm MSM8974: channel 0-7, QMAP not supported
  - Qualcomm MSM8916: channel 0-7, QMAP usually supported, but not always
                                   (depends on firmware version)
  - Qualcomm MSM8937: channel 0 only, QMAP required for multiplexing(?)
     (Note: This one is theoretic based on logs, this was not tested so far...)

-- 
Sergey