At 2025-12-19 00:20:28, "Leon Romanovsky" [off-list ref] wrote:

On Wed, Dec 17, 2025 at 04:49:47PM +0800, Xiong Weimin wrote:

quoted

Hi all,

This testing instructions aims to introduce an emulating a soft ROCE 
device with normal NIC(no RDMA).

What is it? We already have one soft RoCE device implemented in the
kernel (drivers/infiniband/sw/rxe), which doesn't require any QEMU
changes at all.

Thanks

The framwork of vhost_user_rdma(dpdk)/virtio-rdma driver(kernel) is actually 
a userspace RDMA backend optimized for virtualization, while rxe (Soft-RoCE) 
is a kernel-based software RDMA implementation. Key advantages include:
1. Zero-Copy Architecture: vhost_user_rdma uses shared memory between VMs and 
host processes, eliminating data copies.rxe requires kernel-mediated data copies, 
adding latency.


2. Polling Mode: Avoids VM-Exit interrupts by using busy-wait polling, reducing 
CPU context switches.


3. QEMU/KVM Native Support: vhost_user_rdma integrates directly with hypervisors 
via vhost-user protocol.rxe requires PCI device passthrough ( e.g., VFIO), 
complicating deployment.


4. Features Support: vhost_user_rdma enables live migration, multi-queue virtio, 
and NUMA-aware I/O processing.


5. Userspace Processing: Operates entirely in userspace ( e.g., with SPDK), bypassing
the kernel network stack. rxe relies on the Linux kernel network stack, consuming more 
CPU resources.


6. Resource Efficiency: Achieves lower latency in benchmarks for VM-to-VM communication. 


7. vhost-user Backend: DPDK provides a vhost-user library that implements the vhost-user 
protocol in userspace. This library enables efficient communication between the hypervisor 
(QEMU) and the userspace networking stack (like a DPDK-based application). For RDMA, this 
means that the vhost-user backend can directly handle RDMA operations without going through 
the kernel.


Thanks