Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 265528 - sys-apps/coreutils: "/usr/bin/who -b" broken -- probably by sys-apps/openrc
Summary: sys-apps/coreutils: "/usr/bin/who -b" broken -- probably by sys-apps/openrc
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL: N/A
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-04-09 10:25 UTC by Robert Bradbury
Modified: 2011-05-03 10:06 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Bradbury 2009-04-09 10:25:28 UTC
It appears that openrc may have broken "who -b".  I believe that the init files are supposed to update /var/run/utmpx or more likely /var/run/utmp (since those are the files that "who -b" attempts to open) and the init process no longer does this.

Reproducible: Always

Steps to Reproduce:
1. "who -b" to see what the old boot date was.
2. Reboot Linux
3. "who -b" to see if boot time has been updated.

Actual Results:  
On my system "who -b" says Dec 30 yet I know it was booted in March (/var/log/dmesg is dated Mar 14). 

Expected Results:  
Init files should update /var/run/utmp with the correct boot date record.
Comment 1 Doug Goldstein (RETIRED) gentoo-dev 2009-04-10 15:31:28 UTC
$ who -b
         system boot  2009-04-10 09:08

Works properly for me.
Comment 2 Andrei Slavoiu 2009-04-10 17:30:58 UTC
(In reply to comment #1)
> $ who -b
>          system boot  2009-04-10 09:08
> 
> Works properly for me.
> 

Are you using baselayout-2? It works fine on my stable amd64 computer too, but on my ~amd64 I get the same problem as the reporter of the bug.
Comment 3 Doug Goldstein (RETIRED) gentoo-dev 2009-04-10 18:08:05 UTC
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.4.3-r1
Comment 4 Andrei Slavoiu 2009-04-10 19:37:13 UTC
emerge --info for the affected system

Portage 2.2_rc28 (default/linux/amd64/2008.0/desktop, gcc-4.3.3, glibc-2.9_p20081201-r2, 2.6.29-gentoo-r1 x86_64)
=================================================================                                                
System uname: Linux-2.6.29-gentoo-r1-x86_64-AMD_Phenom-tm-_9350e_Quad-Core_Processor-with-gentoo-2.0.0
Timestamp of tree: Fri, 10 Apr 2009 19:00:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p17
dev-java/java-config: 2.1.7
dev-lang/python:     2.5.4-r2, 2.6.1-r1
dev-util/ccache:     2.4-r8
dev-util/cmake:      2.6.3-r1
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.4.3-r1
sys-apps/sandbox:    1.9
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.28-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -ggdb -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=native -ggdb -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="buildpkg ccache collision-protect distlocks fixpackages parallel-fetch preserve-libs preserved-rebuild protect-owned sandbox sfperms splitdebug strict unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://mirrors.evolva.ro/gentoo http://ftp.roedu.net/pub/mirrors/gentoo.org http://mirrors.xservers.ro/gentoo http://distfiles.gentoo.org"
LANG="ro_RO.UTF-8"
LC_ALL="ro_RO.UTF-8"
LDFLAGS="-Wl,-O1,--enable-new-dtags,--sort-common,-z,combreloc,--as-needed"
LINGUAS="ro en eo"
MAKEOPTS="-j5 -l8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/portage/local/layman/sunrise"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext 3ds 7zip X a52 aac aalib accessibility acl acpi aiglx akode alsa amd64 amr amrnb amrwb apache2 asf aspell audiofile bash-completion berkdb bonjour branding bzip2 caps captury cddb cdinstall cdparanoia cdr cisco clamav cracklib crypt css cups curl dba dbus dga djvu dmi doc dpms dri dts dv dvd dvdr dvdread eds emboss encode erandom evo examples exif expat faac faad fam fbcon fbcondecor ffmpeg firefox flac fontconfig fortran ftp gcj gd gdbm geoip gif git glib glut gmp gnokii gnutls gpm gps graphviz gsm hal havekernel hddtemp hou htmlhandbook iconv id3tag idn imagemagick imlib inotify ipv6 isdnlog jadetex java javascript jingle jpeg jpeg2k justify kde kdeenablefinal kdehiddenvisibility kerberos kig-scripting kipi kqemu lame lcms ldap libcaca libclamav libgcrypt libnotify libsamplerate libssh2 libwww live lm_sensors lua lzma lzo mad maildir matroska mbrola metric midi mikmod mjpeg mmap mmx mmxext mng mp3 mp4 mplayer mppe-mppc mudflap multilib musepack musicbrainz mysql mythtv ncurses network nls nodrm nowin nptl nptlonly offensive ofx ogg openal openexr opengl openmp pam pch pcre pdf perl phonon php physfs plasma plugins png posix povray ppds pppd pth python qt qt3 qt3support qt4 quicktime rdesktop readline reflectionreiserfs rss rtc ruby samba sasl schroedinger sdl session slang sms sockets sou sound sox speex spell spl sqlite sqlite3 sse sse2 ssl startup-notification subversion svg symlink sysfs syslog sysvipc tcpd theora threads thumbnail tidy tiff timidity truetype tunepimp unicode usb vhook videos visualization vnc voice vorbis webkit wifi wmf x264 xattr xcb xcomposite xine xml xml2 xmlrpc xmp xorg xpm xsl xulrunner xv xvid xvmc yahoo zeroconf zip zlib zvbi" ALSA_CARDS="bt87x via82xx" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="roen eo" LIRC_DEVICES="kworld" USERLAND="GNU" VIDEO_CARDS="radeon radeonhd vesa"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 5 SpanKY gentoo-dev 2009-04-10 19:48:13 UTC
`who -b` does not work on my unstable amd64 or my stable x86 (but both run baselayout 2 and openrc) ... and ive rebooted them a couple times today ;)

it does work on my stable amd64 which is running baselayout 1
Comment 6 Roy Marples 2009-04-12 13:23:55 UTC
who -b fails on my amd64

however, utmpdump /var/run/utmp shows this

Utmp dump of /var/run/utmp
[8] [01680] [rc  ] [        ] [            ] [                    ] [0.0.0.0        ] [Sun Apr 12 11:57:01 2009 BST]
[1] [20019] [~~  ] [runlevel] [~           ] [                    ] [0.0.0.0        ] [Sun Apr 12 11:57:01 2009 BST]
[8] [02343] [l3  ] [        ] [            ] [                    ] [0.0.0.0        ] [Sun Apr 12 11:57:05 2009 BST]
[6] [03236] [c1  ] [LOGIN   ] [tty1        ] [                    ] [0.0.0.0        ] [Sun Apr 12 11:57:05 2009 BST]

Can someone do utmpdump /var/run/utmp on a working system?
I'm pretty sure that any error lies in sysvinit and not OpenRC
Comment 7 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-04-14 02:26:45 UTC
(In reply to comment #6)
> who -b fails on my amd64
> 
> however, utmpdump /var/run/utmp shows this
> 
> Utmp dump of /var/run/utmp
> [8] [01680] [rc  ] [        ] [            ] [                    ] [0.0.0.0   
>     ] [Sun Apr 12 11:57:01 2009 BST]
> [1] [20019] [~~  ] [runlevel] [~           ] [                    ] [0.0.0.0   
>     ] [Sun Apr 12 11:57:01 2009 BST]
> [8] [02343] [l3  ] [        ] [            ] [                    ] [0.0.0.0   
>     ] [Sun Apr 12 11:57:05 2009 BST]
> [6] [03236] [c1  ] [LOGIN   ] [tty1        ] [                    ] [0.0.0.0   
>     ] [Sun Apr 12 11:57:05 2009 BST]
> 
> Can someone do utmpdump /var/run/utmp on a working system?
> I'm pretty sure that any error lies in sysvinit and not OpenRC
> 

utmpdump on a working host:

$ utmpdump /var/run/utmp
Utmp dump of /var/run/utmp
[8] [01126] [si  ] [        ] [            ] [                    ] [0.0.0.0        ] [Thu Apr 09 18:19:24 2009 CDT]
[2] [00000] [~~  ] [reboot  ] [~           ] [                    ] [0.0.0.0        ] [Thu Apr 09 18:19:24 2009 CDT]
[8] [02492] [rc  ] [        ] [            ] [                    ] [0.0.0.0        ] [Thu Apr 09 18:19:31 2009 CDT]
[1] [20019] [~~  ] [runlevel] [~           ] [                    ] [0.0.0.0        ] [Thu Apr 09 18:19:31 2009 CDT]
<snip after the time that 'who -b' outputs>

$ who -b
         system boot  Apr  9 18:19
Comment 8 Roy Marples 2009-06-13 20:07:50 UTC
If /var/run/utmp is removed and the system is rebooted the who -b works correctly for the next boot only. I am at a loss to explain why this is so.
Comment 9 Robert Bradbury 2009-06-14 10:34:06 UTC
Roy, what kind of system are you running -- one with mountable system partitions (i.e. /root, /usr, /var & /tmp all on separate devices) or one with a "unified" system file system (all "system" directories on a single partition)?

I run mountable system partitions and ran into problems during the openrc changeover (a nightmare in my case because I just assumed the openrc upgrade was like any other "user-package" upgrade, didn't bother to back the root up, had to deal with backing out init/conf files from old root backups, etc.)  Gentoo should *really* warn about upgrades that can *really* break the system boot process.

At any rate, in the process I ran into a number of cases where the system was trying to do things which required /usr (e.g. gnustep (/usr/GNUstep/System), X11 (/usr/lib), etc.) or /var (all the daemon managed programs, syslogd, and whomever (?!?) sets the boot time into /var/run/utmp.  When those file systems were not mounted (for whatever reasons) the init scripts tended to break in very strange ways.  This only showed up when I booted single user and found that a normal boot was mounting /usr and /var on "partially" built trees.  Apparently some init processes will create directories and files that don't exist and some will not.  I don't recall exactly how I ran into this situation but it pointed out that the entire init process needs to be tested completely on both unified and mountable system partitions (trees).

My current bootmisc seems to want a writeable /var/run and seems bypass playing with /var/run/utmp if it doesn't exist.  So the sequencing of bootmisc and file system mounting or failures to mount (esp. /var) could be involved.
Comment 10 Roy Marples 2009-10-16 23:26:05 UTC
I seem to have missed this comment - my appologies.

My laptop has a setup of LVM + many volumes, all mounted correctly.
From memory it's like so

/
/usr
/usr/portage
/var
/var/portage
/home
/usr/src
/tmp

And I've not had many problems over the years, even with development code.
Comment 11 Trevor Summers Smith 2010-02-19 14:02:38 UTC
I am running OpenRC and see the same problem. The boot entry in /var/run/utmp is not being created. One more strange thing: the entry in /var/run/wtmp IS created. Also, I see the same behavior that Roy described: deleting /var/run/utmp BEFORE shutdown correctly creates the boot utmp entry on next boot.

I looked briefly at the sysvinit code: init checks to see if it can write to utmp by making a call to access(2), whereas wtmp calls open(2). However that may all be a worthless train of thought -- I'm just becoming familiar with init and OpenRC.

Using:
sys-apps/openrc-0.6.0-r1
sys-apps/sysvinit-2.87-r3
sys-kernel/gentoo-sources-2.6.31-r6

Thoughts?

Trevor
Comment 12 Robert Bradbury 2010-02-19 17:36:12 UTC
The question is precisely *when* during the init sequence is it calling the access/open calls?  On my system /var is a mounted file system, so /var/run isn't going to exist if the access/open calls are used *before* /var is successfully mounted.  I suppose the trick would be to create a /var/run on the root drive, then see if booting single user create the proper files.  If so, then the proper solution might be to have the init.d mount script check to see if the files exist and do something like save them in /tmp, mount /var, then move them to their final location.

Ideally, init should not attempt to create these files until *after* any mounts of /var or /var/run are completed.  Its a very tricky business when final file locations are on mounted file systems.  I hate to see what would happen if somebody was using mounted file systems with /sbin or /etc.
Comment 13 Trevor Summers Smith 2010-02-19 18:10:28 UTC
I think that starting in single user mode should not work, because that would not load the boot level init scripts, and the "root" script that mounts /var/run as rw is in the boot runlevel in OpenRC.

I tested with single user mode anyway though, just in case I had missed something. I see the same failure. 
Comment 14 Robert Bradbury 2010-02-20 01:09:35 UTC
Ok, I think I found it.  The code in init is a little convoluted because it
seems to try and take into account that /var may not be mounted.  But since  
But if I check rc.log it looks like the mounts (/etc/init.d/localmount) are
done before bootmisc (which creates the utmp/wtmp files in /var/run).  So it
"should work", unfortunately the code which is cleaning up /var/run (around
lines 91-106 in my version) only excludes utmp (and a few other files) from being deleted from /var/run.  Since the just created wtmp is deleted init can't open it later on when it wants to write the boot record.

The easiest way to fix it is to simply add wtmp to the list of "allowed" files
in /var/run (in the "find /var/run ..." command).  Tricky to find but easy to fix.

So this is actually a openrc bug (unless bootmisc is part of the sysvinit
package).  I haven't verified this will work but it looks like it should.
Comment 15 Roy Marples 2010-02-20 22:21:15 UTC
Nice idea, but wtmp lives in /var/log and not /var/run
Comment 16 Robert Bradbury 2010-02-21 00:17:52 UTC
Sorry, I was not looking closely.  Scratch the easy fix. Here is what I know so far.
1) It looks like the logic in init may be correct and it is compiled correctly.
2) /var/log/wtmp does contain the correct "reboot" record that init writes.
3) /var/run/utmp does not contain the correct "reboot" record that init writes at the same time it writes the wtmp record.
4) Writing records into utmp actually uses the function "pututline()" which is documented ("man pututline" works but the function isn't documented well if at all).
5) Looking at the glibc source code its clear that the pututline() function is designed to perform "locked" atomic writes to the utmp file (which makes sense since multiple logins/logouts could be occurring at the same time and they could overwrite each other's records).
6) I straced "who -b" and it is looking at utmp (if it looked at wtmp it would probably work).
7) The initial contents of utmp, as dumped by by the utmpdump program seems strange in that it contains the string "rc" but it isn't really in the correct location in the record.  Now the "/sbin/rc" program is part of openrc package, not part of the sysvinit package.

Which would suggest that /sbin/rc is playing fast and loose with utmp, either not realizing that there should be a "reboot" record there, or not using the pututline() function (and writing to the file directly).  I'll move onto openrc next, but the above would suggest that sysvinit and openrc *have* to be kept in sync with each other (and maybe glibc as well).  If someone changes the structure of utmp (utmp.h which is part of glibc) probably both sysvinit and openrc are going to need to be recompiled.

And just to make this even more fun, "who" is part of the coreutils package, probably meaning that if utmp.h were to change then the coreutils package would need to be rebuilt as well. :-)
Comment 17 Robert Bradbury 2010-02-21 03:43:35 UTC
Now I think I've really got it and it wasn't easy.  It seems to be a consequence of how inittab, init, and the openrc localmount & bootmisc scripts collectively operate.

1) init starts out with wrote_utmp_reboot = 1 (which would be valid in cases where one is doing a resume);
2) init also creates and closes UTMP_FILE (init.c:2443)
3) init starts with state 'si'/'#' (Notation here is inittab/init.) then goes into the SYSINIT -> BOOT transition -- the 'rc'/'*' this code (init.c:2197-2200) resets wrote_utmp_reboot = 0 and tries to write the BOOT_TIME record (presumably works for both utmp & wtmp) *if* /var is mounted;
3) init starts to run /sbin/rc boot which creates a fork to run all of the scripts in /etc/init.d, presumably executing localmount (mounting /var) and bootmisc (which clears utmp but leaves wtmp);
4) the "boot" sequence finishes up and comes back to init which goes off to deal with whatever needs to be done for the 'id' (initdefault) state -- but;
5) the code in utmp.c:write_utmp:127-141 which is supposed to catch cases where the BOOT_TIME record wasn't written will not work because its aleady bumped wrote_utmp_reboot.

I think the solutions perhaps involve:
1) Don't have bootmisc clear utmp (since init creates it); and/or
2) Move the line "if( type == BOOT_TIME) wrote_utmp_reboot++;" after the lines which try to fudge the BOOT_TIME record in utmp for cases when /var wasn't mounted.

A third possibility is a "delayed" creation of utmp by init if the first creation failed because /var wasn't mounted.  I'm fairly sure bootmisc should not be playing with utmp/wtmp at all since init/agetty/login are the primary players with the files.

So the problem involves both the sysvinit and openrc packages.  The util-linux package appears to behave properly with regard to using pututline() and related functions.

Interestingly, the "last" command will properly provide the system boot time and the uptime though not in as nice a format as "who -b" -- a consequence of the fact that it gets the information from wtmp rather than utmp.

I'm not sure, but it might be better for openrc to zero (not remove) /var/run/utmp during a shutdown (just before the localmount unmounts occur).