Bug 266739 – >=media-video/dvdrip-0.98.10-r2 has parallel make issues

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Lars Wendler (Polynomial-C) (RETIRED) 2009-04-19 09:08:54 UTC]

Hi,

I don't know how hard it is to trigger this bug when you're not on a decent multi-core system. I was never able to trigger this bug on my dual-core Athlon 64 FX (K8/Windsor core) machine but when I set up my shiny new dual quad-core Opteron 2380 (K10/shanghai cores) system, most but not all emerges trigger this problem.
The error message is always the same when this problem gets triggered:

mv: cannot stat `dvdrip-splitpipe': No such file or directory
make[1]: *** [../bin/dvdrip-splitpipe] Error 1

I will attach the build.log file of a failed emerge.



Reproducible: Sometimes


# emerge --info
Portage 2.2_rc30 (default/linux/amd64/2008.0/desktop, gcc-4.3.3, glibc-2.8_p20080602-r1, 2.6.27.21 x86_64)
=================================================================
System uname: Linux-2.6.27.21-x86_64-Quad-Core_AMD_Opteron-tm-_Processor_2380-with-gentoo-2.0.0
Timestamp of tree: Sun, 19 Apr 2009 06:45:01 +0000
app-shells/bash:     4.0_p17-r1
dev-java/java-config: 2.1.7
dev-lang/python:     2.6.2
dev-util/cmake:      2.6.3-r1
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.4.3-r2
sys-apps/sandbox:    1.9
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.5, 1.7.9-r1, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.28-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=barcelona -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/init.d /etc/logrotate.d /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-march=barcelona -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--alphabetical --with-bdeps=y"
FEATURES="collision-protect distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LDFLAGS="-Wl,-O1,--hash-style=gnu,--sort-common,--as-needed"
LINGUAS="de en"
MAKEOPTS="-j7"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac acpi alsa amd64 berkdb branding bzip2 cairo cdda cdr cli cracklib crypt cups dri dvd dvdr dvdread emboss encode evo fam ffmpeg firefox flac gdbm gif gnutls gtk iconv idn isdnlog jpeg jpeg2k kde lame libnotify mad midi mikmod mmap mmx mmxext mp3 mp4 mpeg mudflap multilib ncurses nls nptl nptlonly nsplugin ogg opengl openmp pam pcre pdf png ppds pppd qt3 qt3support quicktime readline reflection sdl session slang spell spl sse sse2 ssl startup-notification svg sysfs theora threads tiff truetype usb vcd vorbis x264 xml xorg xulrunner xv xvid zlib" ALSA_CARDS="emu10k1 hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LINGUAS="de en" USERLAND="GNU" VIDEO_CARDS="nv nvidia"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 1 Lars Wendler (Polynomial-C) (RETIRED) 2009-04-19 09:09:46 UTC]

Created attachment 188854 [details]
build.log

[Comment 2 Matthias Langer 2009-08-24 09:19:28 UTC]

Although I cannot reproduce it anymore, I've just triggered exactly this bug on a dual core system:

Portage 2.1.6.13 (default/linux/amd64/2008.0/desktop, gcc-4.3.2, glibc-2.9_p20081201-r2, 2.6.30-gentoo-r5 x86_64)
=================================================================
System uname: Linux-2.6.30-gentoo-r5-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E6550_@_2.33GHz-with-glibc2.2.5
Timestamp of tree: Sun, 23 Aug 2009 21:45:02 +0000
app-shells/bash:     3.2_p39
dev-java/java-config: 2.1.8-r1
dev-lang/python:     2.5.4-r3
dev-util/cmake:      2.6.4
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.63
sys-devel/automake:  1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=core2 -O2 -pipe"
DISTDIR="/var/portage/distfiles"
FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://gd.tuwien.ac.at/opsys/linux/gentoo/ http://mirror.uni-c.dk/pub/gentoo/ http://trumpetti.atm.tut.fi/gentoo/ "
LANG="en_US.utf8"
LC_ALL="en_US.utf8"
LDFLAGS="-Wl,-O1"
LINGUAS="en de"
MAKEOPTS="-j3"
PKGDIR="/var/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/var/portage/repos/gentoo"
PORTDIR_OVERLAY="/var/portage/repos/layman/java-overlay /var/portage/repos/layman/games /var/portage/repos/private"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac acpi alsa amd64 berkdb bluetooth branding bzip2 cairo caps cdda cddb cdparanoia cdr cli cracklib crypt cups dbus djvu dri dts dvd dvdr eds emboss encode evo exif fam ffmpeg firefox flac fortran gd gdbm gif gimp gnome gnome-keyring gphoto2 gpm gstreamer gtk hal hddtemp iconv icu ipv6 isdnlog java java6 jpeg jpeg2k kpathsea lcms libnotify lm_sensors log4j mad matroska mikmod mmap mmx mmxext mono mp3 mp4 mpeg mudflap multilib musicbrainz ncurses nls nptl nptlonly nvidia ogg opengl openmp pam pcre pdf perl plotutils png ppds pppd pulseaudio python qt3support quicktime readline reflection ruby sdl session spell spl sse sse2 ssl ssse3 startup-notification svg sysfs tcpd tetex theora threads thunar tiff truetype unicode usb vcd vim-syntax vorbis wavpack x264 xattr xine xinerama xml xorg xulrunner xv xvid zlib" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="canon konica ptp2 kodak" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LINGUAS="en de" USERLAND="GNU" VIDEO_CARDS="nvidia nv"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 3 Jorge Manuel B. S. Vicetto (RETIRED) 2009-09-01 23:44:09 UTC]

I've hit this bug while building the livedvd-stage1.spec for the 10.0 release with catalyst. I have a very similar build log to the one provided by Lars Wendler. This is a Quad-Core box with MAKEOPTS="-j5".

[Comment 4 Samuli Suominen (RETIRED) 2009-12-22 13:52:46 UTC]

Added MAKEOPTS to -j1 as a workaround

[Comment 5 Dane Smith (RETIRED) 2011-05-12 13:08:45 UTC]

Created attachment 272935 [details]
Patch to dvdrip-0.98.11.ebuild

I have attached a patch to the ebuild, and a patch for the ebuild to apply. The patch to be applied by the ebuild should remove the issue for parallel-compilation, removing the need for the -j1 workaround.

The patch to the ebuild removes the -j1 workaround, applies the patch, and also fixes a lack of respect for the toolchain that I happened to spot while working on this.

media-video, I will revbump this and apply my patches on 5/17 unless you guys beat me to it, or unless I hear that you don't want me to.

[Comment 6 Dane Smith (RETIRED) 2011-05-12 13:09:21 UTC]

Created attachment 272937 [details]
Patch to be applied by the ebuild

[Comment 7 Dane Smith (RETIRED) 2011-05-18 13:32:37 UTC]

Created attachment 273835 [details]
Diff of -r1 from 0.98.11-r1

First, my earlier fix only partially fixed the issue at hand. It removed the issue with compilation (meaning it wouldn't fail) but it still didn't respect the jobserver. Also, my src_compile was over riding the perl-module_src_compile which was where OTHERLDFLAGS was defined. I dug into it further and found a slightly better way to handle it.

New patch to be applied by the ebuild to follow.

@media-video - I don't mind applying this, but I'd love an ack just so that I know you're ok with it. Having said that, if I don't hear something soon, I'm just going to go ahead with the revbump.

Thanks!

[Comment 8 Dane Smith (RETIRED) 2011-05-18 13:34:39 UTC]

Created attachment 273837 [details]
Patch to be applied

This one will fix everything in one go. It fixes src/Makefile to output to ../bin/${prog}, which will remove the concurrent need problems. It also patches Makefile.PL to generate cd src && ( $(MAKE) CC=$(CC) ) so that both the jobserver is respected, and that the submake also uses the correct CC.

[Comment 9 Dane Smith (RETIRED) 2011-05-19 17:12:41 UTC]

Acked by Samuli.

+*dvdrip-0.98.11-r1 (19 May 2011)
+
+  19 May 2011; Dane Smith <[e-mail]> +dvdrip-0.98.11-r1.ebuild,
+  +files/parallelmakefix.diff:
+  Non-maintainer commit: Revbump. Add new patch to address parallel make issues
+  wrt bug 266738. Also address issues with respect to the toolchain.
+