emerge seems to hang but in fact it's launching many instances of install-xattr. # pgrep install-xattr | wc -l 25556
Mon 29 Sep 10:57:09 CEST 2014 emerge --info sys-apps/portage Portage 2.2.14_rc1 (python 3.3.5-final-0, default/linux/hppa/13.0, gcc-4.8.3, glibc-2.19-r1, 3.14.15-gentoo-JeR parisc64) ================================================================= System Settings ================================================================= System uname: Linux-3.14.15-gentoo-JeR-parisc64-PA8800_-Mako-with-gentoo-2.2 KiB Mem: 8231872 total, 6632116 free KiB Swap: 2101676 total, 2059640 free Timestamp of tree: Mon, 29 Sep 2014 03:15:01 +0000 ld GNU ld (Gentoo 2.24 p1.4) 2.24 distcc 3.1 hppa2.0-unknown-linux-gnu [enabled] ccache version 3.1.9 [disabled] app-shells/bash: 3.1_p20, 3.2_p54, 4.0_p41, 4.1_p12-r1, 4.2_p49 dev-lang/python: 2.7.7, 3.2.5-r6, 3.3.5-r1, 3.4.1 dev-util/ccache: 3.1.9-r3 dev-util/cmake: 2.8.12.2-r1 dev-util/pkgconfig: 0.28-r1 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.11.6, 1.12.6, 1.13.4 sys-devel/binutils: 2.22-r1, 2.23.1, 2.23.2, 2.24-r3 sys-devel/gcc: 4.6.3, 4.6.4, 4.7.2-r1, 4.7.3-r1, 4.8.1-r1, 4.8.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2-r1 sys-devel/make: 4.0-r1 sys-kernel/linux-headers: 3.16 (virtual/os-headers) sys-libs/glibc: 2.19-r1 Repositories: gentoo JeR ACCEPT_KEYWORDS="hppa" ACCEPT_LICENSE="*" CBUILD="hppa2.0-unknown-linux-gnu" CFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe -Wno-comment" CHOST="hppa2.0-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/lib/distcc/bin /usr/lib/python2.7/site-packages/buildbot/status/web /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /var/bind /var/spool/torque /var/www/localhost/htdocs/wordpress/wp-config.php" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe" DISTDIR="/world/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build=n" FCFLAGS="" FEATURES="assume-digests binpkg-logs buildpkg cgroup compressdebug config-protect-if-modified distcc distlocks ebuild-locks fixlafiles merge-sync metadata-transfer news notitles parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict test test-fail-continue unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe" GENTOO_MIRRORS="http://de-mirror.org/gentoo/ http://gentoo.mneisen.org/ http://mirror.netcologne.de/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.halifax.rwth-aachen.de/gentoo/ http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://mirror.leaseweb.com/gentoo/ http://gentoo.tiscali.nl/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo" LANG="C" LC_ALL="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" MAKEOPTS="-j6" PKGDIR="/keeps/gentoo/packages/karsten" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/dev/shm" PORTDIR="/world/gentoo/portage" PORTDIR_OVERLAY="/keeps/gentoo/local" SYNC="rsync://rsync.nl.gentoo.org/gentoo-portage" USE="256-color X acl alsa amr anacron apng audit bash-completion berkdb bfio bittorrent bzip2 cairo canberra cli consolekit coverage cracklib crossdev crypt cups custom-cflags cxx dane dbus declarative dirac directfb distinct-l dlz doc dot dri emacs exceptions exif expat fbcon filecaps firefox firmware foomaticdb fortran frontend-optional gbm gdbm geoip geoloc glep gles1 gles2 gost gpm graphite gsm gstreamer gtk gudev guile hddtemp hppa hwdb iconv idn imlib inotify ipv6 jadetex javascript jingle jit jpeg jpeg2k json kdrive kmod ladspa lasi launch-frontend libcaca libcanberra libevent libopts libsoxr libusb libwww lua lzma lzo mad mng mod modplug modules mp3 multislot multitarget munge nagios ncurses netlink nfs nls nptl objc objc++ objc-gc offensive openexr opengl openmp openrc openvg opus osmesa oss pam pcre php pic pkcs11 png pnm policykit python qt qt3support qt4 readline realtime regression-test rtmp samba schroedinger session smi socks5 speex spell spidermonkey sqlite ssl tcl tcpd test tga theora threads tiff tk tokyocabinet truetype unicode unwind v4l vim-syntax vorbis vpx wayland wcwidth x264 xa xattr xcb xft xhtml xkb xml2 xvfb xvid zlib" ALSA_CARDS="ad1889 usb-audio" APACHE2_MODULES="authz_host dir mime alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CURL_SSL="nss" DRACUT_MODULES="nfs syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev joystick keyboard mouse acecad aiptek calcomp citron digitaledge dmc dynapro elo2300 elographics fpit hyperpen jamstudio magellan microtouch mutouch palmax penmount spaceorb summa tek4957 tslib ur98 void" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en nl he" OFFICE_IMPLEMENTATION="libreoffice" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="stifb dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.3" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= sys-apps/portage-2.2.14_rc1 was built with the following: USE="doc epydoc (ipc) xattr -build (-selinux)" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 python3_4 (-pypy) -python3_2"
All these install-xattr calls look like this: portage 17052 0.7 0.0 2088 384 pts/4 SN+ 10:59 0:00 /usr/bin/install-xattr -m0700 -d /dev/shm/portage/sys-apps/portage-2.2.8-r2/work
When I kill all install-xattr instances, I get: 0 top [ebuild UD ] sys-apps/portage-2.2.8-r2 [2.2.14_rc1] USE="doc epydoc (ipc) xattr -build (-pypy2_0) -python2% -python3% (-selinux) " LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 python3_4 (-pypy2_0) (-python2_6) (-python3_2) (-pypy%)" 0 KiB Total: 1 package (1 downgrade), Size of downloads: 0 KiB >>> Verifying ebuild manifests >>> Emerging (1 of 1) sys-apps/portage-2.2.8-r2::gentoo * portage-2.2.8.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] /var/tmp/portage/portage/._portage_reinstall_.kkh3sl/bin/phase-functions.sh: line 234: 7578 Killed install -m${PORTAGE_WORKDIR_MODE:-0700} -d "${WORKDIR}" * ERROR: sys-apps/portage-2.2.8-r2::gentoo failed (unpack phase): * Failed to create dir '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/work' * * Call stack: * ebuild.sh, line 719: Called __ebuild_main 'unpack' * phase-functions.sh, line 964: Called __dyn_unpack * phase-functions.sh, line 240: Called die * The specific snippet of code: * install -m${PORTAGE_WORKDIR_MODE:-0700} -d "${WORKDIR}" || die "Failed to create dir '${WORKDIR}'" * * If you need support, post the output of `emerge --info '=sys-apps/portage-2.2.8-r2::gentoo'`, * the complete build log and the output of `emerge -pqv '=sys-apps/portage-2.2.8-r2::gentoo'`. * The complete build log is located at '/keeps/gentoo/emergelogs/karsten/sys-apps:portage-2.2.8-r2:20140929-091317.log'. * For convenience, a symlink to the build log is located at '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/temp/environment'. * Working directory: '/var/tmp/portage/portage/._portage_reinstall_.kkh3sl/pym' * S: '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/work/portage-2.2.8' * The ebuild phase 'unpack' with pid 7522 appears to have left an orphan
(In reply to Jeroen Roovers from comment #0) > emerge seems to hang but in fact it's launching many instances of > install-xattr. > > # pgrep install-xattr | wc -l > 25556 It looks like possibly a recursive fork bomb in sys-apps/install-xattr. What version of sys-apps/install-xattr do you have? If you use pstree or `ps axf` to look at the processes, does it look like a single tree of recursive processes?
(In reply to Zac Medico from comment #4) > (In reply to Jeroen Roovers from comment #0) > > emerge seems to hang but in fact it's launching many instances of > > install-xattr. > > > > # pgrep install-xattr | wc -l > > 25556 > > It looks like possibly a recursive fork bomb in sys-apps/install-xattr. What > version of sys-apps/install-xattr do you have? If you use pstree or `ps axf` > to look at the processes, does it look like a single tree of recursive > processes? =sys-apps/install-xattr-0.3 2740 pts/4 Ss 0:56 \_ -/bin/bash 3335 pts/4 SN+ 0:14 | \_ /usr/bin/python3.3 -b /usr/lib/python-exec/python3.3/emerge --nodeps -v1 app-portage/euses 3372 pts/4 SN+ 0:00 | \_ [app-portage/euses-2.5.9] sandbox /usr/lib/portage/python3.3/ebuild.sh unpack 3374 pts/4 SN+ 0:00 | \_ /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack 3391 pts/4 SN+ 0:00 | \_ /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack 3395 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3396 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3399 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3405 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3406 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3407 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3408 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3409 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3410 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3411 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3412 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work 3413 pts/4 SN+ 0:00 | \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.
(In reply to Jeroen Roovers from comment #5) \_ > /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack > 3395 pts/4 SN+ 0:00 | > \_ /usr/bin/install-xattr -m0700 -d > /var/tmp/portage/portage/app-portage/euses-2.5.9/work > 3396 pts/4 SN+ 0:00 | > \_ /usr/bin/install-xattr -m0700 -d > /var/tmp/portage/portage/app-portage/euses-2.5.9/work > 3399 pts/4 SN+ 0:00 | > \_ /usr/bin/install-xattr -m0700 -d That's a recursive fork bomb in install-xattr.
It could be due to install-xattr calling portage's install wrapper instead of /usr/bin/install.
(In reply to Zac Medico from comment #8) > It could be due to install-xattr calling portage's install wrapper instead > of /usr/bin/install. My guess is that the changed installation location of portage's install wrapper in portage-2.2.14_rc1 has broken some assumption inside install-xattr.
Looking at the which() function in install-xattr, it compares portage_helper_path, to canpath and skips it when appropriate: if (portage_helper_path) if (!strcmp(portage_helper_path, canpath)) goto skip; However, portage_helper_path has not been canonicalized with the realpath function, so strcmp can return false even though the paths are equivalent. This probably occurs because /usr/lib is a symlink to /usr/lib64.
Created attachment 385912 [details, diff] install-xattr: canonicalize portage_helper_path This patch compiles, but I haven't actually tested it to see if it fixes the fork bomb.
(In reply to Jeroen Roovers from comment #6) > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem. Can you apply the attached install-xattr patch and test if that fixes it for you?
(In reply to Zac Medico from comment #12) > (In reply to Jeroen Roovers from comment #6) > > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem. > > Can you apply the attached install-xattr patch and test if that fixes it for > you? Wow, I just got a bunch of emails from 9-29. This is the first I'm seeing of this bug. Zac's patch makes sense and I've applied it. Can you test install-xattr-9999. I'll push out release 0.4 in a few days.
(In reply to Anthony Basile from comment #13) > (In reply to Zac Medico from comment #12) > > (In reply to Jeroen Roovers from comment #6) > > > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem. > > > > Can you apply the attached install-xattr patch and test if that fixes it for > > you? > > Wow, I just got a bunch of emails from 9-29. This is the first I'm seeing > of this bug. > > Zac's patch makes sense and I've applied it. Can you test > install-xattr-9999. I'll push out release 0.4 in a few days. I wasn't able to reproduce the fork bomb. I've also noticed one other thing, /usr/lib/portage/python3.3/ebuild-helpers/xattr/install uses PORTAGE_PYM_PATH=${PORTAGE_PYM_PATH:-/usr/lib/portage/pym} but portage-2.2.14_rc1 doesn't have a pym directory. Could this have soemthing to do with it?
(In reply to Anthony Basile from comment #14) > I wasn't able to reproduce the fork bomb. I've also noticed one other > thing, /usr/lib/portage/python3.3/ebuild-helpers/xattr/install uses > PORTAGE_PYM_PATH=${PORTAGE_PYM_PATH:-/usr/lib/portage/pym} but > portage-2.2.14_rc1 doesn't have a pym directory. Could this have soemthing > to do with it? No, because portage always exports PORTAGE_PYM_PATH, so the /usr/lib/portage/pym fallback is never used these days.
(In reply to Anthony Basile from comment #13) > (In reply to Zac Medico from comment #12) > > (In reply to Jeroen Roovers from comment #6) > > > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem. > > > > Can you apply the attached install-xattr patch and test if that fixes it for > > you? > > Wow, I just got a bunch of emails from 9-29. This is the first I'm seeing > of this bug. > > Zac's patch makes sense and I've applied it. Can you test > install-xattr-9999. I'll push out release 0.4 in a few days. @jer, did you have a chance to test -9999? I'd like to be sure this works before I push out -0.4.
(In reply to Anthony Basile from comment #16) > @jer, did you have a chance to test -9999? I'd like to be sure this works > before I push out -0.4. It seems to fix this problem.
(In reply to Jeroen Roovers from comment #17) > (In reply to Anthony Basile from comment #16) > > > @jer, did you have a chance to test -9999? I'd like to be sure this works > > before I push out -0.4. > > It seems to fix this problem. Okay I pushed out install-xattr-0.4 with this fix. Thanks guys!
