Re: [Qemu-devel] macvlan/macvtap: guest/host cannot communicate when network cable is unplugged

On Tue, Aug 28, 2012 at 11:38 PM, ching [off-list ref] wrote:

Hi all,

I try to setup guest network with macvlan, as guest cannot communicate with host directly, i try to workaround by configuring host to using a macvlan interface

host: Gentoo x64, kernel 3.5.2, qemu-kvm 1.1.1-r1, libvirt 0.9.13
guest: Ubuntu 12.04, kernel 3.4.9, virtio-net

The host is using macvlan (interface name: znet0, ip: 192.168.1.2, bridge mode)

The guest is using macvtap managed by libvirt (interface name: macvtap0, ip: 192.168.1.184, bridge mode, with vhost)


I am facing a strange problem that the guest/host can communicate only when the network cable of the ethernet adapter is connected.




The network config of the host

    $ifconfig

    eth0      Link encap:Ethernet  HWaddr f4:6d:xx:xx:xx:xx
              inet6 addr: fe80::xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:86507 errors:0 dropped:0 overruns:0 frame:0
              TX packets:55940 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:126005746 (120.1 MiB)  TX bytes:4394225 (4.1 MiB)

    macvtap0  Link encap:Ethernet  HWaddr 52:54:xx:xx:xx:xx
              inet6 addr: fe80::xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:70 errors:0 dropped:0 overruns:0 frame:0
              TX packets:84 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500
              RX bytes:9036 (8.8 KiB)  TX bytes:14734 (14.3 KiB)

    znet0     Link encap:Ethernet  HWaddr 00:60:xx:xx:xx:xx
              inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: 2002:xx:xx:xx:xx/64 Scope:Global
              inet6 addr: fe80:xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4463190 errors:0 dropped:0 overruns:0 frame:0
              TX packets:12527522 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:3959213697 (3.6 GiB)  TX bytes:18590336476 (17.3 GiB)

$ip -d link show

    10: znet0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
        link/ether 00:60:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
        macvlan  mode bridge
    17: macvtap0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 500
        link/ether 52:54:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
        macvtap  mode bridge

libvirt config of the guest network

    <interface type='direct'>
      <mac address='52:54:xx:xx:xx:xx'/>
      <source dev='eth0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>

The network config of the guest

eth0      Link encap:Ethernet  HWaddr 52:54:xx:xx:xx:xx
          inet addr:192.168.1.184  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: 2002:xx:xx:xx:xxx/64 Scope:Global
          inet6 addr: 2002:xx:xx:xx:xx/64 Scope:Global
          inet6 addr: fe80:xx:xx:xx:xx/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:25 errors:0 dropped:0 overruns:0 frame:0
          TX packets:66 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3108 (3.1 KB)  TX bytes:13066 (13.0 KB)

when network cable of host is plugged in

$ ping 192.168.1.184
PING 192.168.1.184 (192.168.1.184) 56(84) bytes of data.
64 bytes from 192.168.1.184: icmp_req=1 ttl=64 time=0.314 ms
64 bytes from 192.168.1.184: icmp_req=2 ttl=64 time=0.253 ms
64 bytes from 192.168.1.184: icmp_req=3 ttl=64 time=0.230 ms


Afterward, i unplugged the network cable to ensure that packet is not routed via external router, but the test seems failed.

$ ping 192.168.1.184
PING 192.168.1.184 (192.168.1.184) 56(84) bytes of data.
From 192.168.1.2: icmp_seq=10 Destination Host Unreachable
From 192.168.1.2: icmp_seq=11 Destination Host Unreachable


Do anyone have similar problem? Am i missing something?

Can you try the same test with two macvlan interfaces on the host (no
macvtap)?  You may need to use the ping -I <interface-address>
argument to force the ping source address to a specific macvlan
interface.

If you see the same problem, it may just be the macvlan design - it is
stacked on top of eth0 and might not work when eth0 is down.  CCing
macvlan/macvtap folks.

Stefan

Can you try the same test with two macvlan interfaces on the host (no
macvtap)?  You may need to use the ping -I <interface-address>
argument to force the ping source address to a specific macvlan
interface.

If you see the same problem, it may just be the macvlan design - it is
stacked on top of eth0 and might not work when eth0 is down.  CCing
macvlan/macvtap folks.

Stefan

tested as below

$ifconfig

    eth0      Link encap:Ethernet  HWaddr f4:6d:xx:xx:xx:xx
              inet6 addr: fe80::xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:86507 errors:0 dropped:0 overruns:0 frame:0
              TX packets:55940 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:126005746 (120.1 MiB)  TX bytes:4394225 (4.1 MiB)

    macvtap0  Link encap:Ethernet  HWaddr 52:54:xx:xx:xx:xx
              inet6 addr: fe80::xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:70 errors:0 dropped:0 overruns:0 frame:0
              TX packets:84 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500
              RX bytes:9036 (8.8 KiB)  TX bytes:14734 (14.3 KiB)

    znet0     Link encap:Ethernet  HWaddr 00:60:xx:xx:xx:92
              inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: 2002:xx:xx:xx:xx/64 Scope:Global
              inet6 addr: fe80:xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4463190 errors:0 dropped:0 overruns:0 frame:0
              TX packets:12527522 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:3959213697 (3.6 GiB)  TX bytes:18590336476 (17.3 GiB)
             
   znet1     Link encap:Ethernet  HWaddr 00:60:xx:xx:xx:99 
              inet addr:192.168.1.177  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: 2002:xx:xx:xx:xx64 Scope:Global
              inet6 addr: fe80:xx:xx:xx:xx/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:8 errors:0 dropped:0 overruns:0 frame:0
              TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:1399 (1.3 KiB)  TX bytes:1522 (1.4 KiB)

$ ip -d link show
         
    10: znet0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
        link/ether 00:60:xx:xx:xx:92 brd ff:ff:ff:ff:ff:ff
        macvlan  mode bridge
    15: znet1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT
        link/ether 00:60:xx:xx:xx:99 brd ff:ff:ff:ff:ff:ff
        macvlan  mode bridge


the macvlan interface cannot ping each other no matter network cable is plugged or not

$ ping -I 192.168.1.2 192.168.1.177
PING 192.168.1.177 (192.168.1.177) from 192.168.1.2 : 56(84) bytes of data.

--- 192.168.1.177 ping statistics ---

6 packets transmitted, 0 received, 100% packet loss, time 4999ms


I also perform an additional test: the guests (macvtap bridge mode) CAN communicate each other no matter network cable is plugged or not.

On Thu, Aug 30, 2012 at 1:13 PM, ching [off-list ref] wrote:

In bridge mode I expected them to be able to communicate.

I also perform an additional test: the guests (macvtap bridge mode) CAN communicate each other no matter network cable is plugged or not.

Strange.  I thought the original problem was that the macvtap guests
cannot communicate with each other when the network cable is
unplugged?

Hopefully someone else can help you, I'm not familiar enough with
macvlan/macvtap.

Stefan

quoted

I also perform an additional test: the guests (macvtap bridge mode) CAN communicate each other no matter network cable is plugged or not.

Strange.  I thought the original problem was that the macvtap guests
cannot communicate with each other when the network cable is
unplugged?

Hopefully someone else can help you, I'm not familiar enough with
macvlan/macvtap.

Stefan

My word may be a little bit confusing, it should be

I also perform an additional test: I created two qemu guests (macvtap bridge mode on eth0), they CAN communicate each other no matter network cable of the host is unplugged or not.

and my problem is specified in the subject already: guest/host cannot communicate when network cable is unplugged


if my configuration seems correct, i may file a kernel bugzilla and seek for their advice.

ching

On Thu, Aug 30, 2012 at 11:32 PM, ching [off-list ref] wrote:

quoted

quoted

I also perform an additional test: the guests (macvtap bridge mode) CAN communicate each other no matter network cable is plugged or not.

Strange.  I thought the original problem was that the macvtap guests
cannot communicate with each other when the network cable is
unplugged?

Hopefully someone else can help you, I'm not familiar enough with
macvlan/macvtap.

Stefan

My word may be a little bit confusing, it should be

I also perform an additional test: I created two qemu guests (macvtap bridge mode on eth0), they CAN communicate each other no matter network cable of the host is unplugged or not.

and my problem is specified in the subject already: guest/host cannot communicate when network cable is unplugged


if my configuration seems correct, i may file a kernel bugzilla and seek for their advice.

Yes.  If no one else responds then it's worth filing a bug.

Stefan