Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 543522

Summary: app-admin/syslog-ng does not build with MAKEOPTS="-j17"
Product: Gentoo Linux Reporter: j.habenicht
Component: Current packagesAssignee: Mr. Bones. (RETIRED) <mr_bones_>
Status: RESOLVED FIXED    
Severity: normal CC: chainsaw, k_f, nikoli, zorry
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: https://fossies.org/diffs/syslog-ng/3.6.1_vs_3.6.2/syslog-ng-ctl/Makefile.am-diff.html
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Logfile from failed build
build.log
syslog-ng-3.6.2-parallel.patch
ebuild.diff

Description j.habenicht 2015-03-16 17:03:40 UTC
Excessive use of parallel build is not possible with app-admin/syslog-ng.
'MAKEOPTS="-j17" emerge app-admin/syslog-ng' crashes during build.

Some builds are fine though. This looks like some race condition during the build

Reducing the number of parallel builds stabilizes the build, too (e.g. MAKEOPTS="-j1")

regards


Reproducible: Sometimes
Comment 1 j.habenicht 2015-03-16 17:04:15 UTC
Created attachment 399042 [details]
Logfile from failed build
Comment 2 j.habenicht 2015-03-16 17:04:52 UTC
Portage 2.2.14 (python 2.7.9-final-0, default/linux/amd64/13.0, gcc-4.8.3, glibc-2.19-r1, 3.18.7-gentoo x86_64)
=================================================================
System uname: Linux-3.18.7-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5-2623_v3_@_3.00GHz-with-gentoo-2.2
KiB Mem:   131937056 total,   2267628 free
KiB Swap:  146800632 total, 146799128 free
Timestamp of tree: Sun, 15 Mar 2015 03:45:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.2_p53
dev-java/java-config:     2.2.0
dev-lang/perl:            5.20.1-r4
dev-lang/python:          2.7.9-r1, 3.3.5-r1
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.11
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.13.4
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.4
sys-devel/make:           4.1-r1
sys-kernel/linux-headers: 3.18 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo rion fritteli x-portage
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /var/lib/redmine/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /var/lib/redmine/config/locales /var/lib/redmine/config/settings.yml"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/"
LANG="de_DE.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j17"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/rion /var/lib/layman/fritteli /usr/local/portage"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="acl amd64 avx avx2 berkdb bindist bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 libav mmx modules multilib ncurses nls nptl openmp pam pcre readline session sse sse2 sse4_1 sse4_2 ssl ssse3 tcpd unicode vhosts zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias proxy proxy_http" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 3 Josh G 2015-03-16 20:29:36 UTC
I see this intermittently with: 

MAKEOPTS="-j15"
Comment 4 Kristian Fiskerstrand (RETIRED) gentoo-dev 2015-03-17 09:50:07 UTC
I can confirm this issue, running into the issue on my binhost with MAKEOPTS="-j16":

libtool: link: cannot find the library `./lib/libsyslog-ng-crypto.la' or unhandled argument `./lib/libsyslog-ng-crypto.la'
Makefile:5899: recipe for target 'syslog-ng-ctl/syslog-ng-ctl' failed
make[2]: *** [syslog-ng-ctl/syslog-ng-ctl] Error 1
make[2]: *** Waiting for unfinished jobs....
Comment 5 Mr. Bones. (RETIRED) gentoo-dev 2015-03-17 17:16:36 UTC
if you add --with-embedded-crypto to the configure line does it make it more reliable?
Comment 6 Kristian Fiskerstrand (RETIRED) gentoo-dev 2015-03-17 22:57:13 UTC
(In reply to Mr. Bones. from comment #5)
> if you add --with-embedded-crypto to the configure line does it make it more
> reliable?

yes, just done 5 consecutive builds without the issue
Comment 7 j.habenicht 2015-03-18 09:49:52 UTC
Yes it does.
5 builds with MAKEOPTS="-j17" and 3 builds with MAKEOPTS="-j32" did well.

But the real fix should be upstream. (Judging from './configure --help') I think the build should wait for finish of "separate libsyslog-ng-crypto" before going on.

Anyways with this workaround the build is ok. Please close it.

regards
Comment 8 Adam Randall 2015-03-20 08:30:00 UTC
I had this issue on three servers tonight out of 13. Happened every time on the affected servers with -jN>9 until I set them to -j1 for syslog-ng. Specifically, the affected servers used -j25 and -j33. The other servers were varyingly set to -j3, -j5 and -j9.
Comment 9 Tony Vroon (RETIRED) gentoo-dev 2015-03-23 09:35:49 UTC
Created attachment 399534 [details]
build.log

Portage 2.2.14 (python 2.7.9-final-0, hardened/linux/amd64/no-multilib, gcc-4.8.3, glibc-2.20-r2, 3.17.7-hardened-r1 x86_64)
=================================================================
System uname: Linux-3.17.7-hardened-r1-x86_64-AMD_Opteron-tm-_Processor_6380-with-gentoo-2.2
KiB Mem:   131923124 total, 127860508 free
KiB Swap:          0 total,         0 free
Timestamp of tree: Mon, 23 Mar 2015 00:30:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.2_p53
dev-lang/perl:            5.20.1-r4
dev-lang/python:          2.7.9-r1
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.11
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.10.3-r1, 1.11.6-r1, 1.12.6, 1.13.4
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.7.3-r1, 4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.4
sys-devel/make:           4.1-r1
sys-kernel/linux-headers: 3.18 (virtual/os-headers)
sys-libs/glibc:           2.20-r2
Repositories: gentoo x-portage
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=bdver1 -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mfma4 -mxop -mavx -msse4.2 -msse4.1 -mlzcnt -mprfchw -mfxsr -mxsave --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver1 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.4/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=bdver1 -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mfma4 -mxop -mavx -msse4.2 -msse4.1 -mlzcnt -mprfchw -mfxsr -mxsave --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver1 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles lmirror merge-sync mirror news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_GB.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j32"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
USE="64bit amd64 apache2 bash-completion berkdb blksha1 bzip2 calendar cgi clamdtop cli cracklib crypt ctype curl cxx dahdi diskio dri elf exif expat filter gd gdbm geoip hardened hash hipe hwdb iconv idn ilbc imap innodb intl introspection ipv6 ithreads jabber jpeg json justify keymap kmod kpoll libav lzma lzo mhash minizip mmx mmxext modules mongodb mpm-prefork mudflap mysql mysqli ncurses network-cron no-old-linux nptl nptlonly openmp pam pax_kernel pcre pdo perl pic pie png posix ptpax python rar readline rle sasl session sieve simplexml snmp sockets spl sse sse2 ssl syslog threadsafe tokenizer truetype tty-helpers unicode urandom vhosts vim-syntax vpx xml xml2 xmlreader xmlrpc xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias cgi cgid" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog cpu disk ethstat lvm network nginx statsd vmem" CPU_FLAGS_X86="aes avx fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" CURL_SSL="gnutls" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGINX_MODULES_HTTP="access auth_basic auth_request autoindex browser cache_purge charset degradation empty_gif fastcgi geo gzip limit_conn limit_req map memcached metrics perl proxy referer rewrite scgi secure_link split_clients spdy ssi sticky stub_status upstream_ip_hash upload_progress userid uwsgi" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="none" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON
Comment 10 Tony Vroon (RETIRED) gentoo-dev 2015-03-23 09:36:36 UTC
This is hitting me on the build server, where -j32 is not excessive. The blade has 32 cores, after all.
Comment 11 Tony Vroon (RETIRED) gentoo-dev 2015-03-23 12:38:04 UTC
Created attachment 399542 [details, diff]
syslog-ng-3.6.2-parallel.patch

Backing out 3.6.1 -> 3.6.2 change.
Comment 12 Tony Vroon (RETIRED) gentoo-dev 2015-03-23 12:38:43 UTC
Created attachment 399544 [details, diff]
ebuild.diff

autoreconf and apply patch.
Comment 13 Tony Vroon (RETIRED) gentoo-dev 2015-03-23 12:39:20 UTC
I believe it to be a single change introduced in 3.6.1 -> 3.6.2 that's problematic, backing it out and running eautoreconf allows it to build for me.
Comment 14 Mr. Bones. (RETIRED) gentoo-dev 2015-03-23 18:00:29 UTC
well, I can't see a downside to using the embedded code there so I committed my suggested work-around.