Bugzilla Bug 215167

I recently upgraded an openvz VPS from baselayout-vserver to baselayout-2.  The
system is otherwise fully stable, with no non-standard or unusual tweaks.  On
reboot, the system never gets past 'rc sysinit'.  I tracked the fault down to
/lib/rc/sh/init.sh, which is attempting to call single_user as a result of
failing to mount /sys.

/sbin/rc contains code that detects virtual systems.  Openvz systems are
detected by the presence of an envId in /proc/self/status.  Unfortunately, when
/sbin/rc first runs, /proc is not mounted, and so the openvz test fails.  The
/lib/rc/sh/init.sh script runs without RC_SYS set to VPS, and the
initialisation proceeds as though the system were real.


Reproducible: Always

Steps to Reproduce:
1. Create a plain VPS, based on baselayout-2.
2. Configure VPS from openvz host, so that /sys access is denied.
3. Start VPS

Actual Results:  
/lib/rc/sh/init.sh executes as though the system were a real system.  The
script attempts to mount /sys, which is not supported in my openvz
configuration.  The script calls sulogin, which fails.  The script finally
calls reboot.  This sequence repeats indefinitely.


Expected Results:  
The openvz test performed by rc-misc.c in /sbin/rc should set RC_SYS to "VPS". 
This variable would then control the operation of /lib/rc/sh/init.sh.  The
script would then not attempt to mount /sys.


Openvz apparently differs from other virtualisation servers in that /proc is
not mounted when a VPS is started.

I modified /etc/inittab in my VPS to mount /proc before '/sbin/rc sysinit' is
called, and this fixes the problem.

The bug only affects the first call of /rc/sbin (the sysinit call), as this is
the call that is intended to mount /proc.  Subsequent calls of /sbin/rc
correctly identify the system as openvz and correctly set the RC_SYS
environment variable.

If an openvz system is configured to support VPS use of /sys, and if RC_DEVICES
in /etc/conf.d/rc is set to "static", then this bug will go unnoticed and the
VPS will apparently boot correctly.

upgrade to openrc and retest please

Upgraded to openrc.

The bug is not fixed, although its affect will now go unnoticed in many
systems.

The version of /lib/rc/sh/init.sh in openrc does not try to mount /sys unless
there is kernel support, as indicated in /proc/filesystems.  This prevents the
endless rebooting that I was experiencing with baselayout 2.0.0_rc6.

However, the /lib/rc/sh/init.sh script still has logic that depends on the
setting of the RC_SYS environment variable, set by /sbin/rc.  In the case of
openvz, this variable will never be set, as /proc is not mounted when /sbin/rc
is first run.

With openrc, the worst outcome of this bug is that udev may be started in
error.  The init.sh code is meant to suppress the loading of udev (or any other
device manager) for virtual operating systems:

# Try to figure out how the user wants /dev handled
if [ "${rc_devices}" = "static" \
        -o "${RC_SYS}" = "VSERVER" \
        -o "${RC_SYS}" = "OPENVZ" \
        -o "${RC_UNAME}" = "GNU/kFreeBSD" ]
then
        ebegin "Using existing device nodes in /dev"
        eend 0
else...

For an out of the box installation, running on an openvz VPS, rc_devices will
be unset and RC_SYS will also be unset.  The script will try to load udev, and
then devfs, and then mdev.  On the system that hosts my VPS, this prevented my
virtual system from booting, as the host kernel is not compatible with the
latest udev.

As loading a device manager can cause problems (for example, mounting a tmpfs
over /dev thereby hiding a set of perfectly good devices), I suggest that the
virtual operating system detection code that exists in /sbin/rc be re-run
immediately after init.sh has mounted /proc.  This would cause only a minimal
delay to non-virtual systems, but would ensure that openrc would work on
vanilla installations running in VPS environments.  This change would impact
only on init.sh.

To remove redundant checking, /sbin/rc could be modified to suppress the RC_SYS
check before calling init.sh, and comments in init.sh added to reflect the fact
that RC_SYS is not set on entry.

Hollow: Aren't you doing this successfully with the in tree openrc version?

i'm not using openvz myself, but openrc should work with openvz

As far as I know, vserver boots virtual servers with /proc mounted, whereas
openvz boots virtual servers without /proc mounted.  This is a crucial
difference.

/sbin/rc uses /proc/self/status to detect virtual systems (librc.c):

#ifdef __linux__
        if (exists("/proc/xen")) {
                if (file_regex("/proc/xen/capabilities", "control_d"))
                        return RC_SYS_XEN0;
                return RC_SYS_XENU;
        } else if (file_regex("/proc/cpuinfo", "UML"))
                return RC_SYS_UML;
        else if (file_regex("/proc/self/status",
                            "(s_context|VxID):[[:space:]]*[1-9]"))
                return RC_SYS_VSERVER;
        else if (file_regex("/proc/self/status",
                            "envID:[[:space:]]*[1-9]"))
                return RC_SYS_OPENVZ;
#endif

The first time /sbin/rc is run in an openvz boot, /proc is not mounted and so
openvz is not detected.  In a vserver system, /proc is mounted and vserver is
correctly detected.

The first run of /sbin/rc is the sysinit part of the boot process.  /sbin/rc
runs the three init scripts in /lib/rc/sh.  These scripts contain logic that
relies on the correct detection of virtual systems through the RC_SYS
environment variable.  This logic will work for vserver systems, but will be
compromised for openvz systems.

Whether an openvz system boots or not will depend on settings in /etc/rc.conf,
and how the VPS is configured in the openvz host.

Created an attachment (id=148624) [details]
Re-examine RC_SYS after mounting /proc if empty.

Try this patch against openrc-0.2.1, post back if it works or not.

Reopening as seems that we really have this bug in VE detection. I missed this
bug as running udevd inside VE is not an error - it just works ) and there are
different suggestions how to run udevd inside VE:

http://wiki.openvz.org/Special:Search?search=udev&go=Go

Well, the patch really fixes detection but contains a small typo:

-+[ -z "{RC_SYS}" ] && export RC_SYS="$(rc --sys)"
++[ -z "${RC_SYS}" ] && export RC_SYS="$(rc --sys)"

But is it possible to udevd inside VE?

You tell us. You're the Gentoo OpenVZ dev... ;)

Patch works (with typo fixed as per comment 7).

On my VPS, udev fails because the kernel version of the host node is too old
for the version of udev that gentoo needs to install for openrc.

(In reply to comment #8)
> You tell us. You're the Gentoo OpenVZ dev... ;)

He-h, the question was is it possible to enable udevd with openrc when openrc
detects VE correctly... reading init.sh seems that impossible. But well I
remember that Roy does not use vservers but accept patches. Not sure what I can
do here (as my time is sorta limited at the moment), but we'll see.

Roy, in any case it's worth to apply patch.

udev working or not working in a VPS isn't my concern as udev is a seperate
package ;)

All we can do is try to start it or not start it.

Patch applied with the alteration to the git repo.

Fixed in openrc-0.2.1-r2. Thanks for testing.