after emerging udev, emerge/portage hangs for no apparent reason. I first ran across this issue when doing an emerge -e world, and it has since repeated when emerging it alone, as part of a group of packages (updating), and with 124-r1, -r2, and 133. Reproducible: Always Steps to Reproduce: 1.emerge udev 2.??? 3.profit? Actual Results: Hangs Expected Results: the return of my prompt Portage 2.1.6_rc3 (selinux/2007.0/amd64, gcc-4.1.2, glibc-2.6.1-r0, 2.6.25-gentoo-r7 x86_64) ================================================================= System uname: Linux-2.6.25-gentoo-r7-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4800+-with-glibc2.2.5 Timestamp of tree: Sat, 06 Dec 2008 05:01:01 +0000 app-shells/bash: 3.2_p33 dev-lang/python: 2.5.2-r7 dev-util/cmake: 2.4.6-r1 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r2 sys-devel/automake: 1.4_p6, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=athlon64 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=athlon64 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks fixpackages loadpolicy parallel-fetch protect-owned sandbox selinux sesandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://www.gtlib.gatech.edu/pub/gentoo/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LDFLAGS="-Wl,-O1 -Wl,--sort-common -s" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="bzip2" PORTAGE_COMPRESS_FLAGS="-9" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acpi aim alsa amd64 apm bash-completion cli cracklib cscope dri gif gtk hou iconv ipv6 jpeg mbox mmx mmxext mp3 msn mudflap ncurses nodrm nptl nptlonly nvidia openal opengl openmp pam pcre png ruby selinux session sse sse2 ssl tcpd unicode vim-syntax xcb xorg xv xvmc yahoo zlib" ALSA_CARDS="i8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="hd44780" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Hangs how? Can you kill it with Ctrl-C? Is your system hard-locked? Did it catch on fire or kill your dog? Have you examined the state of the emerge process with `ps` in another term while it's "hanging"?
Use `pstree` to check for subprocesses of the emerge command.
The state ps aux | grep "emerge" gives S+ pstree gives the following: bash - emerge - ebuild.sh It can be ended with CTRL+C, and the install does complete, but hangs at the end of the ebuild (after the message stating to go to the udev guide), and seemingly never ends. I timed it once (to see if it was just taking a long time), and ended up CTRL+Cing it, returning a time near 120 minutes.
The only thing that makes the udev ebuild special is that it restarts udevd in pkg_postinst. But if ebuild.sh hangs not right after the message "restarting udevd now." is printed, I think this is unrelated.
I checked and I am getting the "restarting udev" message (I never saw it before, so I doubt-checked). Could this be related to SELinux?
It sounds like maybe udevd (or some other process spawned by the ebuild) is inheriting stdin/stdout from the ebuild process and holding that device open for some reason. It's normal for emerge to keep waiting in cases like this, since otherwise some output might get lost and it's not normal for an ebuild to start a process in the background that holds open stdio like that.
(In reply to comment #6) > It sounds like maybe udevd (or some other process spawned by the ebuild) is > inheriting stdin/stdout from the ebuild process and holding that device open > for some reason. Sounds logical, still I doubt udevd will do this under normal circumstances. udevd does normally close all fds and dups /dev/null onto stdin/stdout/stderr. For me the udevd process has no children and looks like this: # lsof |grep udevd COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME udevd 9063 root cwd DIR 254,0 576 2 / udevd 9063 root rtd DIR 254,0 576 2 / udevd 9063 root txt REG 254,0 95900 129493 /sbin/udevd udevd 9063 root mem REG 254,0 1298796 156978 /lib/libc-2.8.so udevd 9063 root mem REG 254,0 38416 68138 /lib/libnss_files-2.8.so udevd 9063 root mem REG 254,0 34356 148448 /lib/libnss_nis-2.8.so udevd 9063 root mem REG 254,0 79612 124803 /lib/libnsl-2.8.so udevd 9063 root mem REG 254,0 30440 153305 /lib/libnss_compat-2.8.so udevd 9063 root mem REG 254,0 113172 156963 /lib/ld-2.8.so udevd 9063 root 0u CHR 1,3 0t0 930 /dev/null udevd 9063 root 1u CHR 1,3 0t0 930 /dev/null udevd 9063 root 2u CHR 1,3 0t0 930 /dev/null udevd 9063 root 3r DIR 0,8 0 1 inotify udevd 9063 root 4u unix 0xc8f78ea0 0t0 2245935 socket udevd 9063 root 5u sock 0,4 0t0 2245936 can't identify protocol udevd 9063 root 6r FIFO 0,5 0t0 2245937 pipe udevd 9063 root 7w FIFO 0,5 0t0 2245937 pipe
phrexianreaper, use lsof to check which files the ebuild.sh process has open. Normally the stdio for ebuild.sh is connected to a pseudo-terminal device, located in /dev/pts/. Also, use lsof to see if another process is holding open the same pseudo-terminal device as ebuild.sh.
Actually, you might not be able to directly see the pseudo-terminal device that was connected to ebuild.sh after it's defunct. However, maybe you might be able to account for all of your open devices in /dev/pts/ and deduce which one it was by process of elimination.
I reran emerge udev, and lsof (had to use ps aux to track down the pid). Here is the output of lsof | grep emerge: emerge 16516 root cwd DIR 8,8 12288 54564833 /home/wrr emerge 16516 root rtd DIR 8,3 4096 2 / emerge 16516 root txt REG 8,3 6416 70696 /usr/bin/python2.5 emerge 16516 root mem REG 8,3 10600 86354 /usr/lib64/python2.5/lib-dynload/syslog.so emerge 16516 root mem REG 8,3 24744 86349 /usr/lib64/python2.5/lib-dynload/termios.so emerge 16516 root mem REG 8,3 435784 224148 /lib64/libncursesw.so.5.6 emerge 16516 root mem REG 8,3 78248 86343 /usr/lib64/python2.5/lib-dynload/_curses.so emerge 16516 root mem REG 8,3 36520 86339 /usr/lib64/python2.5/lib-dynload/operator.so emerge 16516 root mem REG 8,3 21160 86345 /usr/lib64/python2.5/lib-dynload/_locale.so emerge 16516 root mem REG 8,3 71440 134000 /usr/lib64/python2.5/site-packages/selinux_aux.so emerge 16516 root mem REG 8,3 249896 224146 /lib64/libsepol.so.1 emerge 16516 root mem REG 8,3 94624 224111 /lib64/libselinux.so.1 emerge 16516 root mem REG 8,3 117104 134889 /usr/lib64/python2.5/site-packages/_selinux.so emerge 16516 root mem REG 8,3 11192 86299 /usr/lib64/python2.5/lib-dynload/resource.so emerge 16516 root mem REG 8,3 44056 224209 /lib64/libnss_files-2.6.1.so emerge 16516 root mem REG 8,3 44184 224206 /lib64/libnss_nis-2.6.1.so emerge 16516 root mem REG 8,3 86544 224204 /lib64/libnsl-2.6.1.so emerge 16516 root mem REG 8,3 31840 224202 /lib64/libnss_compat-2.6.1.so emerge 16516 root mem REG 8,3 11448 86346 /usr/lib64/python2.5/lib-dynload/grp.so emerge 16516 root mem REG 8,3 12528 86335 /usr/lib64/python2.5/lib-dynload/_bisect.so emerge 16516 root mem REG 8,3 84952 224149 /lib64/libz.so.1.2.3 emerge 16516 root mem REG 8,3 20616 86348 /usr/lib64/python2.5/lib-dynload/_ssl.so emerge 16516 root mem REG 8,3 60240 86350 /usr/lib64/python2.5/lib-dynload/_socket.so emerge 16516 root mem REG 8,3 1565864 70581 /usr/lib64/libcrypto.so.0.9.8 emerge 16516 root mem REG 8,3 319696 70580 /usr/lib64/libssl.so.0.9.8 emerge 16516 root mem REG 8,3 16584 86352 /usr/lib64/python2.5/lib-dynload/_hashlib.so emerge 16516 root mem REG 8,3 31048 86292 /usr/lib64/python2.5/lib-dynload/_struct.so emerge 16516 root mem REG 8,3 41808 86336 /usr/lib64/python2.5/lib-dynload/itertools.so emerge 16516 root mem REG 8,3 15400 86293 /usr/lib64/python2.5/lib-dynload/_random.so emerge 16516 root mem REG 8,3 24688 86290 /usr/lib64/python2.5/lib-dynload/binascii.so emerge 16516 root mem REG 8,3 17512 86338 /usr/lib64/python2.5/lib-dynload/math.so emerge 16516 root mem REG 8,3 74968 86347 /usr/lib64/python2.5/lib-dynload/cPickle.so emerge 16516 root mem REG 8,3 10888 86341 /usr/lib64/python2.5/lib-dynload/_weakref.so emerge 16516 root mem REG 8,3 17160 86355 /usr/lib64/python2.5/lib-dynload/select.so emerge 16516 root mem REG 8,3 22728 86344 /usr/lib64/python2.5/lib-dynload/cStringIO.so emerge 16516 root mem REG 8,3 27624 86353 /usr/lib64/python2.5/lib-dynload/strop.so emerge 16516 root mem REG 8,3 23944 86340 /usr/lib64/python2.5/lib-dynload/time.so emerge 16516 root mem REG 8,3 18280 86351 /usr/lib64/python2.5/lib-dynload/fcntl.so emerge 16516 root mem REG 8,3 26968 86342 /usr/lib64/python2.5/lib-dynload/collections.so emerge 16516 root mem REG 8,3 42384 86291 /usr/lib64/python2.5/lib-dynload/array.so emerge 16516 root mem REG 8,3 1337272 224210 /lib64/libc-2.6.1.so emerge 16516 root mem REG 8,3 540712 224208 /lib64/libm-2.6.1.so emerge 16516 root mem REG 8,3 10976 224203 /lib64/libutil-2.6.1.so emerge 16516 root mem REG 8,3 15208 224199 /lib64/libdl-2.6.1.so emerge 16516 root mem REG 8,3 99344 224205 /lib64/libpthread-2.6.1.so emerge 16516 root mem REG 8,3 1394336 70697 /usr/lib64/libpython2.5.so.1.0 emerge 16516 root mem REG 8,3 116696 224201 /lib64/ld-2.6.1.so emerge 16516 root 0u CHR 136,4 6 /dev/pts/4 emerge 16516 root 1u CHR 136,4 6 /dev/pts/4 emerge 16516 root 2u CHR 136,4 6 /dev/pts/4 emerge 16516 root 3rW REG 8,3 0 20140 /var/db/.pkg.portage_lockfile emerge 16516 root 4uW REG 8,6 0 40242 /var/tmp/portage/sys-fs/.udev-124-r1.portage_lockfile emerge 16516 root 5u CHR 5,2 3726 /dev/ptmx I grepped through lsof and couldn't find anything refering to ebuild.sh, so I hope this output will work.
(In reply to comment #10) That doesn't help because all we can see is that emerge has the master end of the pseudo-terminal open (/dev/ptmx). We can use /etc/portage/bashrc to echo the pseudo-terminal device that ebuild.sh is using, and then you can use lsof to find other processes that have it open. Create /etc/portage/bashrc if it doesn't exist and add the following line: echo EBUILD_PHASE=\"${EBUILD_PHASE}\" tty=$(tty) When the udev ebuild hangs at the end of pkg_postinst, look for something like this in the ebuild output: EBUILD_PHASE=postinst tty=/dev/pts/5 Instead of /dev/pts/5 the number will probably be something different. Whatever device that is, you need to grep for it in the lsof output, like this: lsof | grep /dev/pts/5
(In reply to comment #11) > Create /etc/portage/bashrc if it > doesn't exist and add the following line: > > echo EBUILD_PHASE=\"${EBUILD_PHASE}\" tty=$(tty) Actually, the above code doesn't work properly. Use these two lines instead: echo -n EBUILD_PHASE=\"${EBUILD_PHASE}\" tty= python -c 'import os, sys; print os.ttyname(sys.stdout.fileno())'
I tried the python line, and I was getting EBUILD_PHASE="portinst" tty=/dev/pts/4 Traceback: (most recent call last): File "<string>", line 1, in <module> AttributeError: 'module' object has no attribute 'stout' That is running: echo -n EBUILD_PHASE=\"${EBUILD_PHASE}\" tty=$(tty) python -c 'import os, sys; print os.ttyname(sys.stout.fileno())' lsof | grep /dev/pts/4 output: bash 9173 wrr 0u CHR 136,4 6 /dev/pts/4 bash 9173 wrr 1u CHR 136,4 6 /dev/pts/4 bash 9173 wrr 2u CHR 136,4 6 /dev/pts/4 bash 9173 wrr 255u CHR 136,4 6 /dev/pts/4
(In reply to comment #13) > EBUILD_PHASE="portinst" tty=/dev/pts/4 > Traceback: (most recent call last): > File "<string>", line 1, in <module> > AttributeError: 'module' object has no attribute 'stout' It seems like you spelled it wrong and it didn't work right. You have 'stout' where it should be 'stdout'. Also, you had $(tty) which gave you the incorrect device. Again, the correct lines are as follows: echo -n EBUILD_PHASE=\"${EBUILD_PHASE}\" tty= python -c 'import os, sys; print os.ttyname(sys.stdout.fileno())'
I checked that four times for typos, oh well. Here is the output of lsof | grep /dev/pts/2 emerge 12901 root 0u CHR 136,2 4 /dev/pts/2 emerge 12901 root 1u CHR 136,2 4 /dev/pts/2 emerge 12901 root 2u CHR 136,2 4 /dev/pts/2 bash 18390 wrr 0u CHR 136,2 4 /dev/pts/2 bash 18390 wrr 1u CHR 136,2 4 /dev/pts/2 bash 18390 wrr 2u CHR 136,2 4 /dev/pts/2 bash 18390 wrr 255u CHR 136,2 4 /dev/pts/2 su 24692 root 0u CHR 136,2 4 /dev/pts/2 su 24692 root 1u CHR 136,2 4 /dev/pts/2 su 24692 root 2u CHR 136,2 4 /dev/pts/2 bash 24702 root 0u CHR 136,2 4 /dev/pts/2 bash 24702 root 1u CHR 136,2 4 /dev/pts/2 bash 24702 root 2u CHR 136,2 4 /dev/pts/2 bash 24702 root 255u CHR 136,2 4 /dev/pts/2
(In reply to comment #15) It looks like /dev/pts/2 is the one from your currently running shell. It shouldn't be possible for that one to be allocated for ebuild.sh since it's already in use. Can you attach your /etc/portage/bashrc file so that I can verify it for correctness?
echo -n EBUILD_PHASE=\"${EBUILD_PHASE}\" tty= python -c 'import os, sys; print os.ttyname(sys.stdout.fileno())' thats the /etc/portage/bashrc as it stands. The output of the line of python was the same as $(tty), fwiw.
It seems that file descriptor 9 (which emerge uses to monitor ebuild.sh) is being inherited by udevd. This problem is only noticeable on selinux systems when PORT_LOGDIR is not enabled. You can verify this by setting PORT_LOGDIR in /etc/make.conf (most people use /var/log/portage). The reason for the different behavior on selinux is the security policy which makes it impossible to pipe output from ebuild.sh (bug #162404). Hopefully this issue will not affect you, since the output will go should go through a pseudo-terminal (instead of a normal pipe) and hopefully the security policy will allow it.
Created attachment 174816 [details, diff] prevent ebuild.sh subprocesses from inheriting file descriptor 9 If this patch is saved as /tmp/close_pipe.patch, then it can be applied as follows: patch /usr/lib/portage/bin/ebuild.sh /tmp/close_pipe.patch
It works now w/ the patch. Thanks.
Thanks for testing. This is fixed in 2.1.6.1.
