Fedora, udev and nic renaming - vdsm-devel - Fedora mailing-lists

4 Dec 2012


      Hi,
We are currently working on stabilizing the networking part of vdsm in Fedora
18 and, to achieve that purpose, we decided to test in in both physical hosts
and, for extra convenience and better support, also in VMs.
Due to the move of Fedora 17 and 18 to systemd and newer udev versions, we
encountered some issues that should be noted and worked on to provide our users
with a hassle-free experience.
First of all, let me state what happens (in renaming) in RHEL-6.x when a new
ethernet device is handled by udev:
a) One or more udev rules match the characteristics of the interface: The last
matching rule is applied.
b) No rule is matching: /lib/udev/write-net-rules writes a permanent
rule using the MAC address of the interface in a udev rules file, so the
interface name will be permanent and in the ethX namespace.
In Fedora 17 (but even more so in F18), with the move to a newer version of
udev and, specially, with the change from sysV init to systemd, the mechanism
changed. Since systemd is making the boot happen in a parallelized way, some
changes had to be enforced in udev to keep the renaming working:
- To avoid naming collisions, it was decided to use Dell's biosdevname software
  to retrieve a device name for the network interfaces. Typically emX for
  onboard nics and pXpY for pci connected nics.
- For devices which biosdevname could not provide any information, it was
  agreed to assign them a name in the ethX space in a first-come, first-served
  fashion.
- Optionally, one could define the interace MAC addr in an ifcfg file
  and /lib/udev/rename-device would look into the ifcfg file and assign
  the device name there set (I have not yet succeeded in that part, I have to
  investigate more, I guess).
As you can see, biosdevname, never reports names in the eth space to avoid
collision with a potential parallel discovery of an interface not recognizable
by it, to which the kernel could have assigned already a bios reported name.
For physical machines this approach works fine. However, for Virtual machines
with more than one nic, the automatic process described above presents some
issues. Biosdevname, due to the different ways the virtualization hypervisors
report the vnics, dropped support for VMs in 0.3.7 (F18 uses 0.4.1-2) and
decided that on VMs, it would just return "4" to indicate to udev to use
kernel first-come, first-served for those interfaces (ethX namespace).
The issue with using first-come first-served, is that due to the highly
parallelized boot there is now, it is very common to encounter that the names
of your devices (as identified by MAC address) suffer a permutation upon each
reboot. Here you can see an example:
NOTE: The libvirt dump of the VM reports the same PCI address for each
interface across reboots.
Boot 0 (Nov 13th 14:59)
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:54:85:57  txqueuelen 1000  (Ethernet)
    eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:77:45:6b  txqueuelen 1000  (Ethernet)
    eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:ca:41:c7  txqueuelen 1000  (Ethernet)
    eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:f5:3d:c8  txqueuelen 1000  (Ethernet)
    eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:5e:10:76  txqueuelen 1000  (Ethernet)
    eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:95:00:93  txqueuelen 1000  (Ethernet)
Boot 1 (Nov 13th 15:01)
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:ca:41:c7  txqueuelen 1000  (Ethernet)
    eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:54:85:57  txqueuelen 1000  (Ethernet)
    eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:77:45:6b  txqueuelen 1000  (Ethernet)
    eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:f5:3d:c8  txqueuelen 1000  (Ethernet)
    eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:5e:10:76  txqueuelen 1000  (Ethernet)
    eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:54:00:95:00:93  txqueuelen 1000  (Ethernet)
As you can see, after rebooting:
eth0 -> eth1
eth1 -> eth2
eth2 -> eth0
This is an issue if different vnics are connected to different networks or for
whichever reason require distinct configuration. To solve this issue, on the
guest there are three options:
- Assign somebody with BIOS knowledge to add KVM guest support to biosdevname
  so we can use the emX/pXpY namespace and maintain a native like experience in
  the VMs. My intuition is that it could just report pXpY where X is bus and Y
  is slot. This is the preferred option.
- Use libguestfs for setting udev rules using the MAC addresses we know from
  the VM definition in the netX namespace (I have been told that it is not
  entirely safe to use emX namespace for this, but hopefully for VMs this could
  be done safely or, at least, pXpY).
- Use libguestfs to provide ifcfg files that specify the vnics' MAC addresses
  so they are picked up by udev's rename process (Have to find out how to make
  that work, as I have not managed to do it yet).
Best,
Toni
PS: For physical machines, beware that some usb network devices don't have very
original MAC addresses or don't have a mac address burned in (randomized at
each boot) and, for this reason, they can create quite a lot of issues and more
complicated (as in using bus and device and/or ID) udev rules should be used.