Bugzilla Bug 239101

Sometimes net-analyzer/rrdtool parallel make install failed with ruby bindings,
but emerge doesn't failed.

# USE=ruby emerge rrdtool
(snip)
make[4]: Entering directory
`/var/tmp/portage/net-analyzer/rrdtool-1.3.3/work/rrdtool-1.3.3/bindings/ruby'
mkdir -p
/var/tmp/portage/net-analyzer/rrdtool-1.3.3/image//usr/lib64/ruby/site_ruby/1.8/x86_64-linux
/usr/bin/install -c -m 0755 RRD.so
/var/tmp/portage/net-analyzer/rrdtool-1.3.3/image//usr/lib64/ruby/site_ruby/1.8/x86_64-linux
/usr/bin/install: cannot create regular file
`/var/tmp/portage/net-analyzer/rrdtool-1.3.3/image//usr/lib64/ruby/site_ruby/1.8/x86_64-linux':
File exists
make[4]: ***
[/var/tmp/portage/net-analyzer/rrdtool-1.3.3/image//usr/lib64/ruby/site_ruby/1.8/x86_64-linux/RRD.so]
Error 1
make[4]: Leaving directory
`/var/tmp/portage/net-analyzer/rrdtool-1.3.3/work/rrdtool-1.3.3/bindings/ruby'


I think that it should add "-j1" to "emake install".


Portage 2.2_rc11 (!../usr/portage/profiles/default-linux/amd64/2007.0/desktop,
gcc-4.3.1, glibc-2.8_p20080602-r0, 2.6.26-gentoo-r1 x86_64)
=================================================================
System uname:
Linux-2.6.26-gentoo-r1-x86_64-Intel-R-_Core-TM-2_Duo_CPU_L7300_@_1.40GHz-with-glibc2.2.5
Timestamp of tree: Unknown
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[enabled]
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p39
dev-java/java-config: 1.3.7, 2.1.6-r1
dev-lang/python:     2.5.2-r7
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.2.5
sys-apps/sandbox:    1.2.18.1-r3
sys-devel/autoconf:  2.13, 2.63
sys-devel/automake:  1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   2.2.4
virtual/os-headers:  2.6.26
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/
/etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release
/etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -march=nocona -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache collision-protect cvs distcc distlocks
parallel-fetch preserve-libs protect-owned sandbox sfperms strict test
unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="ja_JP.UTF-8"
LDFLAGS=""
LINGUAS="ja"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --stats --timeout=180 --exclude=/distfiles
--exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/home/matsuu/gentoo-x86"
PORTDIR_OVERLAY="/usr/local/portage /home/matsuu/work/matsuu_overlay/secondlife
/home/matsuu/work/matsuu_overlay/misc /home/matsuu/work/matsuu_overlay/awesome
/home/matsuu/work/matsuu_overlay/processing"
SYNC="cvs://matsuu@cvs.gentoo.org:/var/cvsroot"
USE="X a52 aac acl acpi alsa amd64 anthy apache2 arts avahi bash-completion
berkdb bitmap-fonts cairo ccache cdr cjk cli cracklib crypt cups curl daap dbus
dga dmx dri dvd dvdr dvdread eds emboss encode esd evo fam firefox fortran gdbm
gif glitz gnome gpm gstreamer gtk hal iconv imap immqt-bc ipv6 ipw4965 isdnlog
java jpeg kde kerberos logrotate mad mdnsresponder-compat midi mikmod mmx
mozdevelop mp3 mpeg mudflap ncurses nls nptl nptlonly nsplugin ogg opengl
openmp oss pam pcre pdf perl png pppd python qt3 qt3support qt4 quicktime
readline reflection sdl session spell spl sse sse2 ssl ssse3 svg tcpd tiff
truetype truetype-fonts type1-fonts unicode vorbis wireshark x264 xcb
xcomposite xml xorg xosd xulrunner xv xvid zlib" ALSA_CARDS="hda-intel"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file
hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route
share shm softvol" ELIBC="glibc" INPUT_DEVICES="mouse keyboard synaptics evdev"
KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001
mtxorb ncurses text" LINGUAS="ja" USERLAND="GNU" VIDEO_CARDS="i810 vesa vga"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL,
PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Created an attachment (id=166787) [details]
rrdtool-1.3.3.ebuild-parallel.patch

Please, attach full build output. -j1 is only workaround which we should avoid
as much as possible...

Created an attachment (id=166948) [details]
rrdtool-1.3.3-with-ruby.log

no ruby binding are installed.
it seems that 'mkmf' ruby module has something problem.

Created an attachment (id=166950) [details]
rrdtool emerge log with MAKEOPTS=-j1

Please, provide full build output of broken build. I need to see how it
fails...

rrdtool-1.3.3-with-ruby.log is broken build. it has no emerge error, but
/usr/lib64/python2.5/site-packages/rrdtoolmodule.so is not provided.

please check `diff -u rrdtool-1.3.3-with-ruby.log
rrdtool-1.3.3-with-ruby-j1.log`.

This is ruby build problem so this is impossible to fix in rrdtool.

Makefile for ruby bindings is created with 

ruby extconf.rb

where extconf.rb is really short file:
========================================
require 'mkmf'

if /linux/ =~ RUBY_PLATFORM
   $LDFLAGS += '-Wl,--rpath -Wl,$(EPREFIX)/lib'
elsif /solaris/ =~ RUBY_PLATFORM
   $LDFLAGS += '-R$(EPREFIX)/lib'
elsif /hpux/ =~ RUBY_PLATFORM
   $LDFLAGS += '+b$(EPREFIX)/lib'
elsif /aix/ =~ RUBY_PLATFORM
   $LDFLAGS += '-Wl,-blibpath:$(EPREFIX)/lib'
end

dir_config("rrd","../../src","../../src/.libs")
have_library("rrd", "rrd_create")
create_makefile("RRD")
========================================

Resulted Makefile is not parallel safe...
================================================================
install: install-so ## Not affects install-rb

install-so: $(RUBYARCHDIR)
install-so: $(RUBYARCHDIR)/$(DLLIB)
$(RUBYARCHDIR)/$(DLLIB): $(DLLIB)
        $(INSTALL_PROG) $(DLLIB) $(RUBYARCHDIR)

$(RUBYARCHDIR):
        $(MAKEDIRS) $@

================================================================

So it calls $(RUBYARCHDIR) and $(RUBYARCHDIR)/$(DLLIB) in parallel. From make
file info:
"
One file can be the target of several rules.  All the prerequisites
mentioned in all the rules are merged into one list of prerequisites for
the target.
"

So reasons found but this is definitely not rrdtool issue. This is the issue
with ruby 1.8.6_p287-r1.

Ruby is this know issue?

It wasnt' a known issue to the ruby herd. It's not clear to me where the real
problem is. Perhaps ruby's mkmf should write out parallel make safe Makefiles,
but it can only do that if it understands the dependencies.

In any case I think some investigation into the ruby side of things is needed
first.

So, rather than forcing -j1 on everybody, can we "fix" rrdtool's ebuild to only
use -j1 if USE=ruby, or, even better... simply patch the generated Makefile
after it's been created?

I found the issue and reported to upstream.
http://redmine.ruby-lang.org/issues/show/1337

I think it would be good idea to backport the patch to
ruby-1.8.7_p72-r10.ebuild.
http://redmine.ruby-lang.org/attachments/download/338

(In reply to comment #9)
> So, rather than forcing -j1 on everybody, can we "fix" rrdtool's ebuild to
> only use -j1 if USE=ruby

Since this issue affects only 'make install' do you think it's worth additional
logic in ebuild?

(In reply to comment #10)
> http://redmine.ruby-lang.org/attachments/download/338

MATSUU, these are really great news. Thank you for localizing and fixing this
issue.


@ruby: Could you backport patch, please? :)

(In reply to comment #11)
> @ruby: Could you backport patch, please? :)
> 

Fixed in ruby-1.8.6_p287-r13.ebuild, ruby-1.8.7_p72-r11.ebuild and
ruby-1.9.1-r1.ebuild. Built rrdtool successfully against it multiple times.

Thanks everyone!