Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 523994

Summary: sys-apps/install-xattr-0.3: recursive fork bomb with =sys-apps/portage-2.2.14_rc1 and FEATURES=xattr when /usr/lib is a symlink
Product: Portage Development Reporter: Jeroen Roovers (RETIRED) <jer>
Component: UnclassifiedAssignee: Anthony Basile <blueness>
Status: RESOLVED FIXED    
Severity: normal CC: dev-portage, pacho
Priority: Normal Keywords: PATCH, REGRESSION
Version: 2.2   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 484436    
Attachments: install-xattr: canonicalize portage_helper_path

Description Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 08:56:54 UTC
emerge seems to hang but in fact it's launching many instances of install-xattr.

# pgrep install-xattr | wc -l
25556
Comment 1 Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 08:57:37 UTC
Mon 29 Sep 10:57:09 CEST 2014
emerge --info sys-apps/portage
Portage 2.2.14_rc1 (python 3.3.5-final-0, default/linux/hppa/13.0, gcc-4.8.3, glibc-2.19-r1, 3.14.15-gentoo-JeR parisc64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-3.14.15-gentoo-JeR-parisc64-PA8800_-Mako-with-gentoo-2.2
KiB Mem:     8231872 total,   6632116 free
KiB Swap:    2101676 total,   2059640 free
Timestamp of tree: Mon, 29 Sep 2014 03:15:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
distcc 3.1 hppa2.0-unknown-linux-gnu [enabled]
ccache version 3.1.9 [disabled]
app-shells/bash:          3.1_p20, 3.2_p54, 4.0_p41, 4.1_p12-r1, 4.2_p49
dev-lang/python:          2.7.7, 3.2.5-r6, 3.3.5-r1, 3.4.1
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.9.6-r3, 1.11.6, 1.12.6, 1.13.4
sys-devel/binutils:       2.22-r1, 2.23.1, 2.23.2, 2.24-r3
sys-devel/gcc:            4.6.3, 4.6.4, 4.7.2-r1, 4.7.3-r1, 4.8.1-r1, 4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2-r1
sys-devel/make:           4.0-r1
sys-kernel/linux-headers: 3.16 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo JeR
ACCEPT_KEYWORDS="hppa"
ACCEPT_LICENSE="*"
CBUILD="hppa2.0-unknown-linux-gnu"
CFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe -Wno-comment"
CHOST="hppa2.0-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib/distcc/bin /usr/lib/python2.7/site-packages/buildbot/status/web /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /var/bind /var/spool/torque /var/www/localhost/htdocs/wordpress/wp-config.php"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe"
DISTDIR="/world/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=n"
FCFLAGS=""
FEATURES="assume-digests binpkg-logs buildpkg cgroup compressdebug config-protect-if-modified distcc distlocks ebuild-locks fixlafiles merge-sync metadata-transfer news notitles parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict test test-fail-continue unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-mschedule=8000 -march=2.0 -ggdb -Wall -O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/ http://gentoo.mneisen.org/ http://mirror.netcologne.de/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.halifax.rwth-aachen.de/gentoo/ http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://mirror.leaseweb.com/gentoo/ http://gentoo.tiscali.nl/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo"
LANG="C"
LC_ALL="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
MAKEOPTS="-j6"
PKGDIR="/keeps/gentoo/packages/karsten"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/dev/shm"
PORTDIR="/world/gentoo/portage"
PORTDIR_OVERLAY="/keeps/gentoo/local"
SYNC="rsync://rsync.nl.gentoo.org/gentoo-portage"
USE="256-color X acl alsa amr anacron apng audit bash-completion berkdb bfio bittorrent bzip2 cairo canberra cli consolekit coverage cracklib crossdev crypt cups custom-cflags cxx dane dbus declarative dirac directfb distinct-l dlz doc dot dri emacs exceptions exif expat fbcon filecaps firefox firmware foomaticdb fortran frontend-optional gbm gdbm geoip geoloc glep gles1 gles2 gost gpm graphite gsm gstreamer gtk gudev guile hddtemp hppa hwdb iconv idn imlib inotify ipv6 jadetex javascript jingle jit jpeg jpeg2k json kdrive kmod ladspa lasi launch-frontend libcaca libcanberra libevent libopts libsoxr libusb libwww lua lzma lzo mad mng mod modplug modules mp3 multislot multitarget munge nagios ncurses netlink nfs nls nptl objc objc++ objc-gc offensive openexr opengl openmp openrc openvg opus osmesa oss pam pcre php pic pkcs11 png pnm policykit python qt qt3support qt4 readline realtime regression-test rtmp samba schroedinger session smi socks5 speex spell spidermonkey sqlite ssl tcl tcpd test tga theora threads tiff tk tokyocabinet truetype unicode unwind v4l vim-syntax vorbis vpx wayland wcwidth x264 xa xattr xcb xft xhtml xkb xml2 xvfb xvid zlib" ALSA_CARDS="ad1889 usb-audio" APACHE2_MODULES="authz_host dir mime alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CURL_SSL="nss" DRACUT_MODULES="nfs syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev joystick keyboard mouse acecad aiptek calcomp citron digitaledge dmc dynapro elo2300 elographics fpit hyperpen jamstudio magellan microtouch mutouch palmax penmount spaceorb summa tek4957 tslib ur98 void" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en nl he" OFFICE_IMPLEMENTATION="libreoffice" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="stifb dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7 3.3"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

sys-apps/portage-2.2.14_rc1 was built with the following:
USE="doc epydoc (ipc) xattr -build (-selinux)" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 python3_4 (-pypy) -python3_2"
Comment 2 Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 09:00:25 UTC
All these install-xattr calls look like this:
portage  17052  0.7  0.0   2088   384 pts/4    SN+  10:59   0:00 /usr/bin/install-xattr -m0700 -d /dev/shm/portage/sys-apps/portage-2.2.8-r2/work
Comment 3 Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 09:16:30 UTC
When I kill all install-xattr instances, I get:

  0 top                                                                                                                             
             
[ebuild     UD ] sys-apps/portage-2.2.8-r2 [2.2.14_rc1] USE="doc epydoc (ipc) xattr -build (-pypy2_0) -python2% -python3% (-selinux)
" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 python3_4 (-pypy2_0) (-python2_6) (-python3_2) (-pypy%)" 0 KiB
            
Total: 1 package (1 downgrade), Size of downloads: 0 KiB
            
            
>>> Verifying ebuild manifests
            
>>> Emerging (1 of 1) sys-apps/portage-2.2.8-r2::gentoo
 * portage-2.2.8.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                [ ok ]
/var/tmp/portage/portage/._portage_reinstall_.kkh3sl/bin/phase-functions.sh: line 234:  7578 Killed                  install -m${PORTAGE_WORKDIR_MODE:-0700} -d "${WORKDIR}"
 * ERROR: sys-apps/portage-2.2.8-r2::gentoo failed (unpack phase):
 *   Failed to create dir '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/work'
 *          
 * Call stack:
 *            ebuild.sh, line 719:  Called __ebuild_main 'unpack'
 *   phase-functions.sh, line 964:  Called __dyn_unpack
 *   phase-functions.sh, line 240:  Called die
 * The specific snippet of code:
 *              install -m${PORTAGE_WORKDIR_MODE:-0700} -d "${WORKDIR}" || die "Failed to create dir '${WORKDIR}'"
 *          
 * If you need support, post the output of `emerge --info '=sys-apps/portage-2.2.8-r2::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-apps/portage-2.2.8-r2::gentoo'`.
 * The complete build log is located at '/keeps/gentoo/emergelogs/karsten/sys-apps:portage-2.2.8-r2:20140929-091317.log'.
 * For convenience, a symlink to the build log is located at '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/temp/environment'.
 * Working directory: '/var/tmp/portage/portage/._portage_reinstall_.kkh3sl/pym'
 * S: '/var/tmp/portage/portage/sys-apps/portage-2.2.8-r2/work/portage-2.2.8'
 * The ebuild phase 'unpack' with pid 7522 appears to have left an orphan
Comment 4 Zac Medico gentoo-dev 2014-09-29 14:58:24 UTC
(In reply to Jeroen Roovers from comment #0)
> emerge seems to hang but in fact it's launching many instances of
> install-xattr.
> 
> # pgrep install-xattr | wc -l
> 25556

It looks like possibly a recursive fork bomb in sys-apps/install-xattr. What version of sys-apps/install-xattr do you have? If you use pstree or `ps axf` to look at the processes, does it look like a single tree of recursive processes?
Comment 5 Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 15:26:38 UTC
(In reply to Zac Medico from comment #4)
> (In reply to Jeroen Roovers from comment #0)
> > emerge seems to hang but in fact it's launching many instances of
> > install-xattr.
> > 
> > # pgrep install-xattr | wc -l
> > 25556
> 
> It looks like possibly a recursive fork bomb in sys-apps/install-xattr. What
> version of sys-apps/install-xattr do you have? If you use pstree or `ps axf`
> to look at the processes, does it look like a single tree of recursive
> processes?

=sys-apps/install-xattr-0.3


 2740 pts/4    Ss     0:56                              \_ -/bin/bash
 3335 pts/4    SN+    0:14                              |   \_ /usr/bin/python3.3 -b /usr/lib/python-exec/python3.3/emerge --nodeps -v1 app-portage/euses
 3372 pts/4    SN+    0:00                              |       \_ [app-portage/euses-2.5.9] sandbox /usr/lib/portage/python3.3/ebuild.sh unpack
 3374 pts/4    SN+    0:00                              |           \_ /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack
 3391 pts/4    SN+    0:00                              |               \_ /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack
 3395 pts/4    SN+    0:00                              |                   \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3396 pts/4    SN+    0:00                              |                       \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3399 pts/4    SN+    0:00                              |                           \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3405 pts/4    SN+    0:00                              |                               \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3406 pts/4    SN+    0:00                              |                                   \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3407 pts/4    SN+    0:00                              |                                       \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3408 pts/4    SN+    0:00                              |                                           \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3409 pts/4    SN+    0:00                              |                                               \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3410 pts/4    SN+    0:00                              |                                                   \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3411 pts/4    SN+    0:00                              |                                                       \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3412 pts/4    SN+    0:00                              |                                                           \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
 3413 pts/4    SN+    0:00                              |                                                               \_ /usr/bin/install-xattr -m0700 -d /var/tmp/portage/portage/app-portage/euses-2.5.9/work
Comment 6 Jeroen Roovers (RETIRED) gentoo-dev 2014-09-29 15:42:28 UTC
PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.
Comment 7 Zac Medico gentoo-dev 2014-09-29 17:54:19 UTC
(In reply to Jeroen Roovers from comment #5)              \_
> /bin/bash /usr/lib/portage/python3.3/ebuild.sh unpack
>  3395 pts/4    SN+    0:00                              |                  
> \_ /usr/bin/install-xattr -m0700 -d
> /var/tmp/portage/portage/app-portage/euses-2.5.9/work
>  3396 pts/4    SN+    0:00                              |                   
> \_ /usr/bin/install-xattr -m0700 -d
> /var/tmp/portage/portage/app-portage/euses-2.5.9/work
>  3399 pts/4    SN+    0:00                              |                   
> \_ /usr/bin/install-xattr -m0700 -d

That's a recursive fork bomb in install-xattr.
Comment 8 Zac Medico gentoo-dev 2014-09-29 17:58:04 UTC
It could be due to install-xattr calling portage's install wrapper instead of /usr/bin/install.
Comment 9 Zac Medico gentoo-dev 2014-09-29 18:00:23 UTC
(In reply to Zac Medico from comment #8)
> It could be due to install-xattr calling portage's install wrapper instead
> of /usr/bin/install.

My guess is that the changed installation location of portage's install wrapper in  portage-2.2.14_rc1 has broken some assumption inside install-xattr.
Comment 10 Zac Medico gentoo-dev 2014-10-01 18:18:31 UTC
Looking at the which() function in install-xattr, it compares portage_helper_path, to canpath and skips it when appropriate:

	if (portage_helper_path)
		if (!strcmp(portage_helper_path, canpath))
			goto skip;

However, portage_helper_path has not been canonicalized with the realpath function, so strcmp can return false even though the paths are equivalent. This probably occurs because /usr/lib is a symlink to /usr/lib64.
Comment 11 Zac Medico gentoo-dev 2014-10-01 18:40:20 UTC
Created attachment 385912 [details, diff]
install-xattr: canonicalize portage_helper_path

This patch compiles, but I haven't actually tested it to see if it fixes the fork bomb.
Comment 12 Zac Medico gentoo-dev 2014-10-06 21:35:40 UTC
(In reply to Jeroen Roovers from comment #6)
> PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.

Can you apply the attached install-xattr patch and test if that fixes it for you?
Comment 13 Anthony Basile gentoo-dev 2014-10-09 02:42:28 UTC
(In reply to Zac Medico from comment #12)
> (In reply to Jeroen Roovers from comment #6)
> > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.
> 
> Can you apply the attached install-xattr patch and test if that fixes it for
> you?

Wow, I just got a bunch of emails from 9-29.  This is the first I'm seeing of this bug.

Zac's patch makes sense and I've applied it.  Can you test install-xattr-9999.  I'll push out release 0.4 in a few days.
Comment 14 Anthony Basile gentoo-dev 2014-10-09 20:49:30 UTC
(In reply to Anthony Basile from comment #13)
> (In reply to Zac Medico from comment #12)
> > (In reply to Jeroen Roovers from comment #6)
> > > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.
> > 
> > Can you apply the attached install-xattr patch and test if that fixes it for
> > you?
> 
> Wow, I just got a bunch of emails from 9-29.  This is the first I'm seeing
> of this bug.
> 
> Zac's patch makes sense and I've applied it.  Can you test
> install-xattr-9999.  I'll push out release 0.4 in a few days.

I wasn't able to reproduce the fork bomb.  I've also noticed one other thing, /usr/lib/portage/python3.3/ebuild-helpers/xattr/install uses PORTAGE_PYM_PATH=${PORTAGE_PYM_PATH:-/usr/lib/portage/pym} but portage-2.2.14_rc1 doesn't have a pym directory.  Could this have soemthing to do with it?
Comment 15 Zac Medico gentoo-dev 2014-10-09 21:27:47 UTC
(In reply to Anthony Basile from comment #14)
> I wasn't able to reproduce the fork bomb.  I've also noticed one other
> thing, /usr/lib/portage/python3.3/ebuild-helpers/xattr/install uses
> PORTAGE_PYM_PATH=${PORTAGE_PYM_PATH:-/usr/lib/portage/pym} but
> portage-2.2.14_rc1 doesn't have a pym directory.  Could this have soemthing
> to do with it?

No, because portage always exports PORTAGE_PYM_PATH, so the /usr/lib/portage/pym fallback is never used these days.
Comment 16 Anthony Basile gentoo-dev 2014-10-12 17:37:47 UTC
(In reply to Anthony Basile from comment #13)
> (In reply to Zac Medico from comment #12)
> > (In reply to Jeroen Roovers from comment #6)
> > > PORTAGE_INSTALL_XATTR_IMPLEMENTATION="python" works around the problem.
> > 
> > Can you apply the attached install-xattr patch and test if that fixes it for
> > you?
> 
> Wow, I just got a bunch of emails from 9-29.  This is the first I'm seeing
> of this bug.
> 
> Zac's patch makes sense and I've applied it.  Can you test
> install-xattr-9999.  I'll push out release 0.4 in a few days.

@jer, did you have a chance to test -9999?  I'd like to be sure this works before I push out -0.4.
Comment 17 Jeroen Roovers (RETIRED) gentoo-dev 2014-10-12 22:13:08 UTC
(In reply to Anthony Basile from comment #16)

> @jer, did you have a chance to test -9999?  I'd like to be sure this works
> before I push out -0.4.

It seems to fix this problem.
Comment 18 Anthony Basile gentoo-dev 2014-10-13 02:30:55 UTC
(In reply to Jeroen Roovers from comment #17)
> (In reply to Anthony Basile from comment #16)
> 
> > @jer, did you have a chance to test -9999?  I'd like to be sure this works
> > before I push out -0.4.
> 
> It seems to fix this problem.

Okay I pushed out install-xattr-0.4 with this fix.  Thanks guys!