After updating world of a Gentoo server to recent software (kernel 3.8.13, udev-204), I got the new "predictable networking names". These appear to interfere with booting over pxe and having an nfs-root. The system insists on keeping the name eth0 for the interface it uses for pxe/nfs, only the second networking interface gets the new persistent name: enp3s0: flags=4098<BROADCAST,MULTICAST> mtu 9000 ether 00:25:90:a8:f7:09 txqueuelen 1000 (Ethernet) RX packets 5005 bytes 510777 (498.8 KiB) RX errors 0 dropped 97 overruns 0 frame 0 TX packets 6 bytes 492 (492.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 18 memory 0xdf900000-df920000 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.32.99 netmask 255.255.224.0 broadcast 192.168.63.255 inet6 fe80::225:90ff:fea8:f708 prefixlen 64 scopeid 0x20<link> ether 00:25:90:a8:f7:08 txqueuelen 1000 (Ethernet) RX packets 4475952 bytes 1566172198 (1.4 GiB) RX errors 0 dropped 97831 overruns 0 frame 0 TX packets 4422320 bytes 3882851392 (3.6 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 16 memory 0xdfa00000-dfa20000 /etc/udev/rules.d is empty as recommended by the udev page. The system appears to run fine in principle, but at least the network scripts in init.d somehow mix up the interfaces: pt-nds client # /etc/init.d/net.enp3s0 stop * Bringing down interface enp3s0 * root filesystem is network mounted -- can't stop enp3s0 * ERROR: net.enp3s0 failed to stop So it thinks that the second interface (enp3s0) is the one that is used for the nds root - which is not true: the nfs root is on eth0, the other interface does not even have an ip address. Why do I still see eth0? This should (imo) be enp2s0. Why does it mix up eth0 and enp3s0?
1. look inside /sys/class/net/ for the interface(s) 2. run, for example, udevadm test-builtin net_id /sys/class/net/eth0 2> /dev/null and do that for every interface you have, and post the output to this bug 3. do you have files in /etc/udev/rules.d/ and if you do, what they are and what do they contain?
in addition to Comment #1, edit also /etc/conf.d/udev to enable both logging to 'YES' and then /etc/udev/udev.conf remove the # to make it start logging then after booting you should get more verbose 'dmesg' and logs in /run/udev*.log attach them here too reopen the bug once requested data has been provided.
pt-nds client # ll /sys/class/net/ total 0 drwxr-xr-x 2 root root 0 Jul 30 13:35 . drwxr-xr-x 45 root root 0 Jul 30 13:35 .. lrwxrwxrwx 1 root root 0 Jul 30 13:35 bond0 -> ../../devices/virtual/net/bond0 -rw-r--r-- 1 root root 4096 Jul 30 13:35 bonding_masters lrwxrwxrwx 1 root root 0 Jul 30 13:35 dummy0 -> ../../devices/virtual/net/dummy0 lrwxrwxrwx 1 root root 0 Jul 30 13:35 enp3s0 -> ../../devices/pci0000:00/0000:00:1c.6/0000:03:00.0/net/enp3s0 lrwxrwxrwx 1 root root 0 Jul 30 13:35 eth0 -> ../../devices/pci0000:00/0000:00:1c.4/0000:02:00.0/net/eth0 lrwxrwxrwx 1 root root 0 Jul 30 13:35 lo -> ../../devices/virtual/net/lo lrwxrwxrwx 1 root root 0 Jul 30 13:35 sit0 -> ../../devices/virtual/net/sit0 pt-nds client # udevadm test-builtin net_id /sys/class/net/eth0 2> /dev/null ID_NET_NAME_MAC=enx002590a8f708 ID_OUI_FROM_DATABASE=Super Micro Computer, Inc. ID_NET_NAME_PATH=enp2s0 pt-nds client # udevadm test-builtin net_id /sys/class/net/enp3s0 2> /dev/null ID_NET_NAME_MAC=enx002590a8f709 ID_OUI_FROM_DATABASE=Super Micro Computer, Inc. ID_NET_NAME_PATH=enp3s0 pt-nds client # ll /etc/udev/rules.d/ total 6 drwxr-xr-x 2 root root 3 Jul 19 16:52 . drwxr-xr-x 4 root root 6 Jul 19 19:01 .. -rw-r--r-- 1 root root 813 Nov 6 2012 10-open-mx.rules pt-nds client # cat //etc/udev/rules.d/10-open-mx.rules # Open-MX # Copyright © inria 2007-2010 (see AUTHORS file) # # The development of this software has been funded by Myricom, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or (at # your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See the GNU General Public License in COPYING.GPL for more details. KERNEL=="open-mx", NAME="open-mx", GROUP="root", MODE="0666", OPTIONS="last_rule" KERNEL=="open-mx-raw", NAME="open-mx-raw", GROUP="root", MODE="0666", OPTIONS="last_rule"
(In reply to Samuli Suominen from comment #2) > in addition to Comment #1, edit also /etc/conf.d/udev to enable both logging > to 'YES' and then /etc/udev/udev.conf remove the # to make it start logging > then after booting you should get more verbose 'dmesg' and logs in > /run/udev*.log > attach them here too > > reopen the bug once requested data has been provided. waiting for these.
Do you have sys-apps/biosdevname installed? If you do, uninstall it.
(In reply to Gerrit Kühn from comment #0) > After updating world of a Gentoo server to recent software (kernel 3.8.13, > udev-204), I got the new "predictable networking names". These appear to > interfere with booting over pxe and having an nfs-root. The system insists > on keeping the name eth0 for the interface it uses for pxe/nfs, only the > second networking interface gets the new persistent name: Freenode, #systemd: 11:06 < ohsix> The system insists on keeping the name eth0 for the interface it uses for pxe/nfs, only the second networking interface gets the new persistent name 11:06 < ohsix> not being able to rename those probably has something to do with it, also, you know which one is eth0 11:07 < ohsix> i dunno if udev would even see an event for that device
Can you rename the interface using iproute2? Like this: # ip link set eth0 name butt Does the eth0 change to butt? If not, then this is not udev issue but your setup has eth0 hardlocked otherwise.
11:11 < ohsix> SIOCSIFNAME 11:11 < ohsix> Changes the name of the interface specified in ifr_name to ifr_newname. This is a privileged operation. ***It is only allowed when the interface is not up***
(In reply to Samuli Suominen from comment #8) > 11:11 < ohsix> SIOCSIFNAME > 11:11 < ohsix> Changes the name of the interface specified in ifr_name to > ifr_newname. This is a privileged operation. ***It is only allowed when the > interface is not up*** 11:13 < ohsix> it's obvious to me that you wouldn't be able to change the one you boot from, but i'm checking the code to make sure that's happening 11:15 < ohsix> it might be just because the device is already up 11:17 < ohsix> yep
(In reply to Samuli Suominen from comment #4) > (In reply to Samuli Suominen from comment #2) > > in addition to Comment #1, edit also /etc/conf.d/udev to enable both logging > > to 'YES' and then /etc/udev/udev.conf remove the # to make it start logging > > then after booting you should get more verbose 'dmesg' and logs in > > /run/udev*.log > > attach them here too > > > > reopen the bug once requested data has been provided. > > waiting for these. Yes, I know. The system is in semi-production, I could not yet reboot it. Will do soon.
(In reply to Samuli Suominen from comment #5) > Do you have sys-apps/biosdevname installed? If you do, uninstall it. It is not installed: pt-nds client # emerge -s biosdevname Searching... [ Results for search key : biosdevname ] [ Applications found : 1 ] * sys-apps/biosdevname Latest version available: 0.4.1 Latest version installed: [ Not Installed ] Size of files: 183 kB Homepage: http://linux.dell.com/biosdevname/ Description: Sets BIOS-given device names instead of kernel eth* names License: GPL-2
(In reply to Samuli Suominen from comment #5) > Do you have sys-apps/biosdevname installed? If you do, uninstall it. It is not installed: pt-nds client # emerge -s biosdevname Searching... [ Results for search key : biosdevname ] [ Applications found : 1 ] * sys-apps/biosdevname Latest version available: 0.4.1 Latest version installed: [ Not Installed ] Size of files: 183 kB Homepage: http://linux.dell.com/biosdevname/ Description: Sets BIOS-given device names instead of kernel eth* names License: GPL-2 (In reply to Samuli Suominen from comment #9) > (In reply to Samuli Suominen from comment #8) > > 11:11 < ohsix> SIOCSIFNAME > > 11:11 < ohsix> Changes the name of the interface specified in ifr_name to > > ifr_newname. This is a privileged operation. ***It is only allowed when the > > interface is not up*** > > 11:13 < ohsix> it's obvious to me that you wouldn't be able to change the > one you boot from, but i'm checking the code to make > sure that's happening > 11:15 < ohsix> it might be just because the device is already up > 11:17 < ohsix> yep I would be fine with eth0 as name for the first interface. But why does the init.d/net script not let me stop the other interface and complains that the nfs root would be mounted over it?
(In reply to Samuli Suominen from comment #7) > Can you rename the interface using iproute2? Like this: > > # ip link set eth0 name butt > > Does the eth0 change to butt? If not, then this is not udev issue but your > setup has eth0 hardlocked otherwise. Had to install iproute2 first, sorry for the delay. No, this does not work: pt-nds client # ip link set eth0 name butt RTNETLINK answers: Device or resource busy
(In reply to Gerrit Kühn from comment #13) > (In reply to Samuli Suominen from comment #7) > > Can you rename the interface using iproute2? Like this: > > > > # ip link set eth0 name butt > > > > Does the eth0 change to butt? If not, then this is not udev issue but your > > setup has eth0 hardlocked otherwise. > > Had to install iproute2 first, sorry for the delay. > No, this does not work: > > pt-nds client # ip link set eth0 name butt > RTNETLINK answers: Device or resource busy Right, and udevd does this: if (dev->flags & IFF_UP) return -EBUSY; As in, neither lets it to be renamed. So it really is correctly eth0 because of the PXE setup. OK, moving on. What did you mean by "But why does the init.d/net script not let me stop the other interface and complains that the nfs root would be mounted over it?" What happens if you "/etc/init.d/enp3s0 stop"?
(In reply to Samuli Suominen from comment #14) > What happens if you "/etc/init.d/enp3s0 stop"? I meant '/etc/init.d/net.enp3s0 stop' And just an out-of-topic suggestion: If you have these 2 interfaces and the another is always eth0, I think you might be good by setting net.ifnames=0 to kernel cmdline, and the second interface would be eth1. I think the PXE setup would keep them sticky.
(In reply to Samuli Suominen from comment #15) > (In reply to Samuli Suominen from comment #14) > > What happens if you "/etc/init.d/enp3s0 stop"? > > I meant '/etc/init.d/net.enp3s0 stop' > > And just an out-of-topic suggestion: > > If you have these 2 interfaces and the another is always eth0, I think you > might be good by setting net.ifnames=0 to kernel cmdline, and the second > interface would be eth1. I think the PXE setup would keep them sticky. Like I wrote in the bug report above: pt-nds init.d # /etc/init.d/net.enp3s0 stop * Bringing down interface enp3s0 * root filesystem is network mounted -- can't stop enp3s0 * ERROR: net.enp3s0 failed to stop It looks like it thinks that this is eth0. Yes, I already thought about going back to classical names. However, I wanted to sort out first if this is expected behaviour or something else is going wrong. Do you know if net.ifnames=0 will be future-proof, or will it go away soon (and only new names be supported)?
net.ifnames=0 is future proof
(In reply to Gerrit Kühn from comment #16) > Like I wrote in the bug report above: > > pt-nds init.d # /etc/init.d/net.enp3s0 stop > * Bringing down interface enp3s0 > * root filesystem is network mounted -- can't stop enp3s0 > * ERROR: net.enp3s0 failed to stop > > It looks like it thinks that this is eth0. That doesn't look like udev related at all. If you could do some more digging to find the actual culprit, so we can assign the bug to correct people.
(In reply to Samuli Suominen from comment #18) > (In reply to Gerrit Kühn from comment #16) > > Like I wrote in the bug report above: > > > > pt-nds init.d # /etc/init.d/net.enp3s0 stop > > * Bringing down interface enp3s0 > > * root filesystem is network mounted -- can't stop enp3s0 > > * ERROR: net.enp3s0 failed to stop > > > > It looks like it thinks that this is eth0. > > That doesn't look like udev related at all. If you could do some more > digging to find the actual culprit, so we can assign the bug to correct > people. I already looked into the script, the message is produced here: --- if [ "$(command -v predown)" = "predown" ]; then ebegin "Running predown" eindent predown || return 1 eoutdent else if is_net_fs /; then eerror "root filesystem is network mounted -- can't stop ${IFACE}" return 1 fi fi --- However, I could not find is_net_fs anywhere in /etc, where does it come from? Is this stuff belonging to OpenRC people?
(In reply to Gerrit Kühn from comment #19) > I already looked into the script, the message is produced here: > > --- > if [ "$(command -v predown)" = "predown" ]; then > ebegin "Running predown" > eindent > predown || return 1 > eoutdent > else > if is_net_fs /; then > eerror "root filesystem is network mounted -- can't > stop ${IFACE}" > return 1 > fi > fi > --- > > > However, I could not find is_net_fs anywhere in /etc, where does it come > from? Is this stuff belonging to OpenRC people? I found the function in /lib64/rc/sh/rc-functions.sh, belonging to OpenRC. It reports correctly that "/" is nfs-mounted. However, net.lo should not care in the first place, because we are not dealing with the interface the mount is running over. Will need to look into net.lo again...
(In reply to Gerrit Kühn from comment #20) > Will need to look into net.lo again... --- stop() { local IFACE=${RC_SVCNAME#*.} module= local IFVAR=$(shell_var "${IFACE}") opts= einfo "Bringing down interface ${IFACE}" eindent if [ -z "${MODULES}" ]; then local MODULES= _load_modules false fi if [ "$(command -v predown)" = "predown" ]; then ebegin "Running predown" eindent predown || return 1 eoutdent else if is_net_fs /; then eerror "root filesystem is network mounted -- can't stop ${IFACE}" return 1 fi fi --- Um, to me this looks like net.lo cannot take down *any* interface when root is on nfs?!
(In reply to Gerrit Kühn from comment #21) > Um, to me this looks like net.lo cannot take down *any* interface when root > is on nfs?! Ok, for the record: I went around this now by defining my own predown-function like this in conf.d/net: --- predown() { if is_net_fs /; then if [ "${IFACE}" = "eth0" ]; then eerror "root filesystem is network mounted -- can't stop ${IFACE}" return 1 fi fi return 0 } --- This limits a check for nfs-root to eth0, and looks like it is working for me now as expected. If this is the way this is supposed to be set up, maybe this should be added to the documentation for pxe/nfs booting (http://www.gentoo.org/doc/en/diskless-howto.xml)?! Anyway, thank you for your excellent support!
