Bug 489280 – git-r3.eclass breaks paralell install for packages sharing repository

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Jakub Caban 2013-10-24 17:54:11 UTC]

I have lots of git versions of packages (All KDE for example, from KDE overlay) and after recent switch to git-r3 eclass I stumble upon many failures like this:

 * Package:    kde-base/kdepim-icons-9999
 * Repository: kde
 * Maintainer: [e-mail]
 * USE:        amd64 elibc_glibc kernel_linux userland_GNU
 * FEATURES:   ccache compressdebug preserve-libs sandbox userpriv usersandbox
 * Fetching HEAD from git://anongit.kde.org/kdepim ...
_git-r3_smart_fetch --no-tags git://anongit.kde.org/kdepim -f HEAD:kde-base/kdepim-icons/4/__main__
git fetch --progress --no-tags git://anongit.kde.org/kdepim -f HEAD:kde-base/kdepim-icons/4/__main__
 * Checking out git://anongit.kde.org/kdepim to /var/tmp/portage/kde-base/kdepim-icons-9999/work/kdepim-icons-9999 ...
git checkout -f kde-base/kdepim-icons/4/__main__ .
fatal: Unable to create '/var/tmp/portage/kde-base/kdepim-icons-9999/work/kdepim-icons-9999/../../../../../../../usr/portage/distfiles/git3-src/kdepim.git/index.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.
 * ERROR: kde-base/kdepim-icons-9999::kde failed (unpack phase):
 *   git checkout kde-base/kdepim-icons/4/__main__ failed
 *
 * Call stack:
 *     ebuild.sh, line   93:  Called src_unpack
 *   environment, line 4703:  Called kde4-meta_src_unpack
 *   environment, line 3901:  Called git-r3_src_unpack
 *   environment, line 3072:  Called git-r3_checkout
 *   environment, line 2886:  Called die
 * The specific snippet of code:
 *       "${@}" || die "git checkout ${local_id}/__main__ failed";
 *
 * If you need support, post the output of `emerge --info '=kde-base/kdepim-icons-9999::kde'`,
 * the complete build log and the output of `emerge -pqv '=kde-base/kdepim-icons-9999::kde'`.
 * The complete build log is located at '/var/tmp/portage/kde-base/kdepim-icons-9999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/kde-base/kdepim-icons-9999/temp/environment'.
 * Working directory: '/var/tmp/portage/kde-base/kdepim-icons-9999/work'
 * S: '/var/tmp/portage/kde-base/kdepim-icons-9999/work/kdepim-icons-9999'

They happens when I run:

smart-live-rebuild -- -aqv1 -j4 --keep-going	


It seems to me that no two packages can build from the same source repository, which is the case for kdepim, kde-workspace etc. 

Reproducible: Always

Steps to Reproduce:
1. Try to install for example kde-meta-9999 from KDE overlay with paralell emerge (for example emerge -j4)

Actual Results:  
Random packages fails due to shared repository

Expected Results:  
Succesful emerge

Portage 2.2.7 (default/linux/amd64/13.0/desktop, gcc-4.8.1, glibc-2.17, 3.11.6-gentoo x86_64)
=================================================================
System uname: Linux-3.11.6-gentoo-x86_64-Intel-R-_Core-TM-_i7-2670QM_CPU_@_2.20GHz-with-gentoo-2.2
KiB Mem:     8141400 total,   1041612 free
KiB Swap:    4194300 total,   4194244 free
Timestamp of tree: Thu, 24 Oct 2013 15:30:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
ccache version 3.1.9 [enabled]
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r3, 3.3.2-r2
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.12::kde
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.3
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.14
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.8.1-r1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.11 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo kde mozilla qt x11 mysql my
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-mtune=native -march=native -O2 -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-mtune=native -march=native -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs ccache clean-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O2 -Wl,--sort-common"
MAKEOPTS="-j8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/kde /var/lib/layman/mozilla /var/lib/layman/qt /var/lib/layman/x11 /var/lib/layman/mysql /var/lib/layman/my"
USE="3dnow 3dnowext X a52 aac acl acpi addressbook alsa amd64 attica avi bash-completion berkdb bluetooth bluray branding bzip2 cairo cdda cdio cdr cli cracklib crypt cups cxx dbus dirac divx djvu dri dts dv dvd dvdr egl emboss encode eps exif facebook fam fame ffmpeg firefox firmware flac fortran g3dvl gdbm gif git gles gpm iconv icu id3tag ipv6 jpeg kde kdenlive kdepim kipi kontact lcms libass libkms lm_sensors lzma mad mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib mysql ncurses nls nptl ogg opengl openmp optimized-qmake opus pam pango pch pcre pdf php plasma png policykit ppds pulseaudio python3 qt3support qt4 readline rss samba scanner schroedinger sdl semantic-desktop session sftp spell sse sse2 sse2_4way ssl ssse3 ssse4 svg system-cairo system-ffmpeg system-icu system-jpeg system-sqlite systemd tcpd theora threads tidy tiff truetype udev udisks unicode upower usb v4l v4l2 vdpau vim-syntax vorbis wayland wicd wifi wxwidgets x264 xcb xcomposite xinerama xml xv xvid xvmc zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="access_compat authn_core vhost_alias {authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias}" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="mouse keyboard synaptics evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python3_3 python2_7" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7 3.3"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC

[Comment 1 Michał Górny 2013-10-24 18:22:53 UTC]

Hmm, it looks like a race condition creating the lock file to me...

[Comment 2 Jakub Caban 2013-10-26 11:41:38 UTC]

Is there some logic behind keeping this lock file as it is?

Maybe it could work the way that the package would wait for the lock instead of dying immediately?

[Comment 3 Michał Górny 2013-10-26 14:38:16 UTC]

The lock file is handled by git itself and the eclass code isn't aware of it at all. However, I don't think git upstream really considered using the same git dir for multiple checkouts. I have to investigate this closer but I currently lack the time to do so.

I guess we could work-around the issue through using an additional lock file. I'll try to wrap up a patch in a few minutes since I have a proper locking snippet somewhere already.

[Comment 4 Michał Górny 2013-10-26 14:51:43 UTC]

Created attachment 361978 [details, diff]
Temporary work-around patch

Please test this patch.

[Comment 5 Jakub Caban 2013-10-26 17:02:33 UTC]

I tried it once right now - 71 packages from KDE overlay merged with no problem.

Although one test is not conclusive here i think we have good direction :)

[Comment 6 Jakub Caban 2013-10-30 16:47:24 UTC]

A few days later and hundreds of packages recompiled by smart-live rebuild -- -j4 I can say that this is bulletproof.

There was not even single failure due to this git lock problem.

[Comment 7 Michał Górny 2013-10-30 19:21:25 UTC]

+  30 Oct 2013; Michał Górny <[e-mail]> git-r3.eclass:
+  Fix parallel checkout race conditions, bug #489280.