Bugzilla Bug 130668

My default MAKEOPTS is -j4, and with that setting, emerge libpcre fails on one
of the scan unit tests involving the linking of libpcreposix.la.  Here's some
of the output:

x86_64-pc-linux-gnu-ar cru .libs/libpcrecpp.a  pcrecpp.o pcre_scanner.o
pcre_stringpiece.o
x86_64-pc-linux-gnu-ranlib .libs/libpcrecpp.a
creating libpcrecpp.la
(cd .libs && rm -f libpcrecpp.la && ln -s ../libpcrecpp.la libpcrecpp.la)
/bin/sh ./libtool --mode=link x86_64-pc-linux-gnu-g++  -I. -I.   -o
pcre_scanner_unittest  pcre_scanner_unittest.o \
        libpcrecpp.la libpcreposix.la
/bin/sh ./libtool --mode=link x86_64-pc-linux-gnu-g++  -I. -I.   -o
pcre_stringpiece_unittest   pcre_stringpiece_unittest.o \
        libpcrecpp.la libpcreposix.la
make: `libpcrecpp.la' is up to date.
libtool: link: cannot find the library `libpcreposix.la'
make: *** [pcre_stringpiece_unittest] Error 1
make: *** Waiting for unfinished jobs....
libtool: link: cannot find the library `libpcreposix.la'
make: *** [pcre_scanner_unittest] Error 1
generating symbol list for `libpcreposix.la'
/usr/bin/x86_64-pc-linux-gnu-nm -B  .libs/pcreposix.o  | sed -n -e 's/^.*[     
]\([ABCDGIRSTW][ABCDGIRSTW]*\)[         ][     
]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p' | /bin/sed 's/.* //' | sort | uniq >
.libs/libpcreposix.exp
grep -E -e "^[^_]|__?pcre_.*utf8|__?pcre_printint" ".libs/libpcreposix.exp" >
".libs/libpcreposix.expT"
mv -f ".libs/libpcreposix.expT" ".libs/libpcreposix.exp"
echo "{ global:" > .libs/libpcreposix.ver
 cat .libs/libpcreposix.exp | sed -e "s/\(.*\)/\1;/" >> .libs/libpcreposix.ver
 echo "local: *; };" >> .libs/libpcreposix.ver
 x86_64-pc-linux-gnu-gcc -shared  .libs/pcreposix.o  -Wl,--rpath
-Wl,/var/tmp/portage/libpcre-6.3/work/pcre-6.3/.libs
-L/var/tmp/portage/libpcre-6.3/work/pcre-6.3
-L/var/tmp/portage/libpcre-6.3/work/pcre-6.3/.libs ./.libs/libpcre.so 
-Wl,-soname -Wl,libpcreposix.so.0 -Wl,-version-script
-Wl,.libs/libpcreposix.ver -o .libs/libpcreposix.so.0.0.0
(cd .libs && rm -f libpcreposix.so.0 && ln -s libpcreposix.so.0.0.0
libpcreposix.so.0)
(cd .libs && rm -f libpcreposix.so && ln -s libpcreposix.so.0.0.0
libpcreposix.so)
x86_64-pc-linux-gnu-ar cru .libs/libpcreposix.a  pcreposix.o
x86_64-pc-linux-gnu-ranlib .libs/libpcreposix.a
creating libpcreposix.la
(cd .libs && rm -f libpcreposix.la && ln -s ../libpcreposix.la libpcreposix.la)

!!! ERROR: dev-libs/libpcre-6.3 failed.



I was able to get past this problem by doing MAKEOPTS=-j1 emerge libpcre.

Here's my emerge info:

Portage 2.0.54 (hardened/amd64, gcc-3.4.5, glibc-2.3.5-r2, 2.6.14-hardened-r6
x86_64)
=================================================================
System uname: 2.6.14-hardened-r6 x86_64 AMD Athlon(tm) 64 X2 Dual Core
Processor 3800+
Gentoo Base System version 1.6.14
dev-lang/python:     2.4.2
sys-apps/sandbox:    1.2.12
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -fomit-frame-pointer -march=k8"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -pipe -fomit-frame-pointer -march=k8"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo/
http://gentoo.mirrors.pair.com/"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 berkdb bzlib crypt doc hardened imap iproute2 maildir nls nptl
nptlonly pam pcre pic posix postgres prelude readline sasl skey ssl tcpd vhosts
userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS,
PORTDIR_OVERLAY

Created an attachment (id=85078) [details]
Patch makefile for llel builds.

Can't reproduce the problem on my X2 system with same makeopts - can you (or
anyone else) please apply the attached patch - and see if it fixes it for you?

(ebuild blah unpack ; cd /var/tmp/portage/libpcre-6.3/work/pcre-6.3 ; patch -p0
</path/to/patch ; ebuild blah compile)

Thanks.

It's a little finicky:

It will bomb out like 50% of the time now, the rest of the time it will build
fine.  I noticed that while the processors were in use (by other builds), it
would bomb out more often.

I'll try your patch and post a reply...

I decided to do a little testing when using your patch (which applies
successfully):

I tested the build 5 times without the patch using this line:

ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild clean && 
ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild unpack && 
ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild compile

Failed:     3
Succeeded:  2

Here's the test build with your patch using this line:

ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild clean && 
ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild unpack && 
cd /var/tmp/portage/libpcre-6.3/work/pcre-6.3 && 
patch -p0 < ~/patch && 
ebuild /usr/portage/dev-libs/libpcre/libpcre-6.3.ebuild compile

Failed:     0
Succeeded:  5


So in conclusion, your patch seems to work.

the libpcre-6.6 still fails randomly with -jN (N>1) and thus needs patching

This affects the 2006.1 release.  I didn't find this bug in my searching, so I
cheated and changed the emake to use -j1 in the 2006.1 snapshot.  This won't
affect anything in the live tree, but I just wanted to let the maintainer know
that this is definitely something I can verify.

I can also confirm this bug, on a new Core Duo machine, latest ~x86 branch,
MAKEOPTS=-j4 by default.

Compiling with MAKEOPTS="-j1" emerge -av1 libpcre succeeds.

confirmed for me, running AMD 64 X2, with Makeopts = -j3

Relevant output:
/bin/sh ./libtool --mode=link x86_64-pc-linux-gnu-g++  -I. -I.   -o
pcre_scanner_unittest  pcre_scanner_unittest.o \
        libpcrecpp.la libpcreposix.la
libtool: link: cannot find the library `libpcreposix.la'
make: *** [pcre_scanner_unittest] Error 1
make: *** Waiting for unfinished jobs....
(cd .libs && rm -f libpcreposix.so.0 && ln -s libpcreposix.so.0.0.0
libpcreposix.so.0)
(cd .libs && rm -f libpcreposix.so && ln -s libpcreposix.so.0.0.0
libpcreposix.so)
x86_64-pc-linux-gnu-ar cru .libs/libpcreposix.a  pcreposix.o
x86_64-pc-linux-gnu-ranlib .libs/libpcreposix.a
creating libpcreposix.la
(cd .libs && rm -f libpcreposix.la && ln -s ../libpcreposix.la libpcreposix.la)

!!! ERROR: dev-libs/libpcre-6.3 failed.
Call stack:
  ebuild.sh, line 1543:   Called dyn_compile
  ebuild.sh, line 938:   Called src_compile
  libpcre-6.3.ebuild, line 38:   Called die

- looks like the file libpcreposix.la gets created AFTER it gets looked for
(with -j>1)

Emerge --info:
Gentoo Base System version 1.12.4
Portage 2.1.1_pre5 (default-linux/amd64/2006.0, gcc-4.1.0/amd64-vanilla,
glibc-2.4-r3, 2.6.17-gentoo-r4 x86_64)
=================================================================
System uname: 2.6.17-gentoo-r4 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor
4400+
Last Sync: Tue, 15 Aug 2006 08:50:01 +0000
ccache version 2.3 [disabled]
app-admin/eselect-compiler: 2.0.0_rc2-r1
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.3
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.17
sys-devel/gcc-config: 2.0.0_rc1
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe -msse3"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/eselect/compiler /etc/gconf
/etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=athlon64 -O2 -pipe -msse3"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer parallel-fetch sandbox sfperms
userpriv usersandbox"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LANG="en_CA.UTF-8"
LC_ALL="en_CA.UTF-8"
LINGUAS="en_CA en_US en_GB en"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="amd64 X a52 aac alsa apache2 arts avi bash-completion berkdb bitmap-fonts
bzip2 cairo cdparanoia cdr cjk cli crypt css cups directfb dlloader doc dri dv
dvd dvdr dvdread eds elibc_glibc emboss emul-linux-x86 encode examples fbcon
ffmpeg firefox foomaticdb fortran gif glut gmp gnome gpm gstreamer gtk gtk2
ieee1394 imlib input_devices_evdev input_devices_keyboard input_devices_mouse
ipod isdnlog java javascript jikes jpeg kde kdeenablefinal kdehiddenvisibility
kernel_linux lame ldap libwww linguas_en linguas_en_CA linguas_en_GB
linguas_en_US lm_sensors logrotate lzw lzw-tiff mad motif mp3 mpeg msn musepack
ncurses nls nptl nptlonly nsplugin nvidia ofx ogg oggvorbis opengl pam pcre
pdflib perl png ppds pppd python qt3 qt4 quicktime readline reflection sdl
session slang spell spl ssl svg tcltk tcpd threads tidy tiff tk truetype
truetype-fonts type1-fonts unicode usb userland_GNU video_cards_nv
video_cards_nvidia video_cards_vesa vorbis wma wmf xml xmms xorg xpm xprint xv
xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS,
PORTAGE_RSYNC_EXTRA_OPTS

Ping! Still broken...

ping reply -- yep, still broken :)

Hit this one on a x86_64 with -j4

Installed correctly with with `MAKEOPTS=-j1 emerge --resume'

I just added the fix for this into libpcre-6.6. No revbump since it's just a
compile fix. It's been around a while so can probably be considered for stable
rather than patching two versions of the package.