Parallel build of dev-lang/R-2.14.0 is broken on amd64 - most of the time. Occasionally the build will succeed and I've not been able to get the build to fail on x86. The failure seems to be associated with building the "Recommended" package DB provided with R. These packages are lazy loaded and the DB for each package has its own Makefile. From the build log the failure seems to be associated with: make[4]: Entering directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD' ( cd Lib ; make ) make[4]: Entering directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD' ( cd Lib ; make clean ) make[5]: Entering directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD/Lib' x86_64-pc- ../Core/cholmod_aat.c -o cholmod_aat.o x86_64-pc- ../Core/cholmod_add.c -o cholmod_add.o make[5]: Entering directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD/Lib' x86_64-pc- ../Core/cholmod_band.c -o cholmod_band.o make[5]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD/Lib' make[4]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD' which results in the failure: make[5]: *** [../../CHOLMOD.a] Error 1 make[5]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD/Lib' make[4]: *** [library] Error 2 make[4]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD' make[3]: *** [sublibraries] Error 1 make[3]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src' ERROR: compilation failed for package Matrix * removing /var/tmp/portage/dev-lang/R-2.14.0/work/R-2.14.0/library/Matrix make[2]: *** [Matrix.ts] Error 1 The log of the failed build will be attached. I've also attached a patch that seems to allow building of these "Recommended" packages in a more "orderly" fashion. The patch may cover up something more subtle but, when applied, R does build. The log of the patched/successful build will also be attached. Reproducible: Sometimes Steps to Reproduce: 1.emerge =dev-lang/R-2.14.0 2. 3. My emerge --info dev-lang/R: Portage 2.1.10.11 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.13-r4, 2.6.39-gentoo-r3 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-2.6.39-gentoo-r3-x86_64-Dual-Core_AMD_Opteron-tm-_Processor_2216-with-gentoo-2.0.3 Timestamp of tree: Mon, 12 Dec 2011 03:45:01 +0000 ccache version 3.1.6 [enabled] app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.5.4-r4, 2.6.6-r2, 2.7.2-r3::sage-on-gentoo, 3.1.4-r3 dev-util/ccache: 3.1.6 dev-util/cmake: 2.8.4-r1 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.1.2, 4.3.4, 4.4.5, 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 2.6.39 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo desktop-effects science sunrise x11 sage-on-gentoo bicatali local-overlay ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA googleearth AdobeFlash-10 AdobeFlash-10.1 Oracle-BCLA-JavaSE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=opteron -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=opteron -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages multilib-strict news parallel-fetch protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--delete-after --timeout=500" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/desktop-effects /var/lib/layman/science /var/lib/layman/sunrise /var/lib/layman/x11 /var/lib/layman/sage-on-gentoo /var/lib/layman/bicatali /usr/local/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="3dnow X Xaw3d a52 aac aalib acl acpi alsa amd64 apache2 audiofile bash-completion berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cups cxx dbus dri dts dvd dvdr emboss encode exif fam firefox flac fortran gdbm gdu gif gphoto2 gpm gstreamer gtk iconv ieee1394 imlib ipv6 java jpeg lcms ldap lesstif libnotify lm_sensors mad midi mmx mng modules motif mozilla mp3 mp4 mpeg mudflap multilib ncurses nls nptl nptlonly nsplugin ogg opengl openmp oss pam pango pcre pdf png policykit ppds pppd qt3support qt4 radius readline sdl session spell sse sse2 ssl startup-notification svg sysfs tcl tcpd tiff tk truetype udev unicode usb vorbis x264 xcb xml xorg xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia fbdev vesa vga nv" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS ================================================================= Package Settings ================================================================= dev-lang/R-2.14.0 was built with the following: USE="X bash-completion cairo java jpeg (multilib) nls openmp png readline tk -doc -lapack -minimal -perl -profile -static-libs"
Created attachment 296473 [details, diff] patch allow Recommended packages to build on amd64
Created attachment 296477 [details] log of failed build of R-2.14.0
Created attachment 296479 [details] log of successful build of R-2.14.0 with the attached patch
Comment on attachment 296473 [details, diff] patch allow Recommended packages to build on amd64 If you're going to fix make files, then why don't you fix them properly (by setting proper prerequisites)?
I see I missed one important first line when I listed the failure. Here it is again with the missing line: ar: cholmod_aat.o: No such file or directory make[5]: *** [../../CHOLMOD.a] Error 1 make[5]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD/Lib' make[4]: *** [library] Error 2 make[4]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src/CHOLMOD' make[3]: *** [sublibraries] Error 1 make[3]: Leaving directory `/var/tmp/portage/dev-lang/R-2.14.0/temp/RtmpdTJrs4/R.INSTALL6ecfe941/Matrix/src' ERROR: compilation failed for package Matrix * removing /var/tmp/portage/dev-lang/R-2.14.0/work/R-2.14.0/library/Matrix make[2]: *** [Matrix.ts] Error 1
I can confirm, although it only happened once in some 10 builds. It looks like a race condition between the objects and the library archive, but I can not find the problem on the CHOLMOD Makefile of the Matrix tar ball included in the R package. Could you report upstream [1]? Thanks. [1] http://r-forge.r-project.org/projects/matrix/
done. Please see https://r-forge.r-project.org/tracker/index.php?func=detail&aid=1723&group_id=61&atid=294
R-2.14.0 is now out of the main tree and has been replace with R-2.14.1. This version of R still occasionally fails to build on 64bit machines with the same race condition. This has also been reported upstream.
I also experienced the same race condition, while emerging dev-lang/R-2.14.1 today. However, by temporarily modifying /etc/make.conf to: MAKEOPTS="-j1" I was able to emerge dev-lang/R successfully and workaround the problem. Afterward, I reverted back to a more efficient value (-j12, in my case, FWIW).
*** Bug 401279 has been marked as a duplicate of this bug. ***
*** Bug 407409 has been marked as a duplicate of this bug. ***
This issue still exists with dev-lang/R-2.15.0. Emerge fails with MAKEOPTS="-j12 --load-average=15.2" but is OK with MAKEOPTS="-j1" in /etc/make.conf
I too can confirm this issue on my ~amd64 laptop. The build fails with anything other than MAKEOPTS="-j1".
*** Bug 420967 has been marked as a duplicate of this bug. ***
*** Bug 430438 has been marked as a duplicate of this bug. ***
Upstream has commented: https://r-forge.r-project.org/tracker/index.php?func=detail&aid=1723&group_id=61&atid=294 They have not seen the failure and before attempting to fix things they need to verify the failure for a version of Matrix >=1.0-9. Presently R-2.15.1, which occasionally presents the failure, comes shipped with Matrix_1.0-6. So one can guess when this will be resolved?
This issue still exists with dev-lang/R-2.15.1
Is it time to force -j1 via the ebuild yet? Upstream doesn't seem to be making progress on this, and it's getting in the way of default kde-meta installs (on ~arch only, of course).
Every parallel failure I've seen is associated with building the "Recommended" packages. I could be wrong here. But if this is correct it seems to me that disabling the building of those packages in parallel will solve the issue. An attempt was made to do this previously. Of course building the entire package with MAKEOPTS="-j1" will work as well, but may be overkill.
I just hit this failure too. Bug has been open for over a year - please force MAKEOPTS in the ebuild until upstream figures it out properly :)
there was a race condition between a clean target and the build target in the Makevars file of the Matrix recommended package. this was especially true for high number of jobs. since the Matrix package is a tar ball within the R tar ball, I pushed a new R-2.15.2 tar ball on the gentoo mirrors. for completeness, the fix is the following: --- src/Makevars (revision 2863) +++ src/Makevars (working copy) @@ -23,7 +23,7 @@ ## INSTALL only cleans src/*.o src/*$(SHLIB_EXT) for each arch sublibs: subclean sublibraries -sublibraries: +sublibraries: subclean @for d in $(SUBDIRS); do \ (cd $${d} && CFLAGS="$(CFLAGS)" CXXFLAGS="$(CXXFLAGS)" MkInclude="$(MkInclude)" $(MAKE) library) || exit 1; \ hopefully the issue should be fixed. re-open if not. + 29 Jan 2013; Sébastien Fabbro <bicatali@gentoo.org> R-2.15.2-r1.ebuild: + Fixed bash completion spaces (bug #417951, thanks Marco Poletti) and parallel + build (bug #395403) +
