Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 679640 - sys-fs/e2fsprogs-1.44.6 parallel install failure
Summary: sys-fs/e2fsprogs-1.44.6 parallel install failure
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords: PullRequest
: 679704 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-03-06 21:23 UTC by Maciej S. Szmigiero
Modified: 2020-03-27 08:36 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build install log (build-log.txt,3.50 KB, text/plain)
2019-03-06 21:24 UTC, Maciej S. Szmigiero
Details
workaround ebuild patch (e2fsprogs-1.44.6.patch,526 bytes, patch)
2019-03-06 21:26 UTC, Maciej S. Szmigiero
Details | Diff
e2fsprogs-1.44.6:20190307-035957.log (e2fsprogs-1.44.6:20190307-035957.log,160.72 KB, text/plain)
2019-03-07 04:44 UTC, Arfrever Frehtes Taifersar Arahesis
Details
parallel install failure on e2fsprogs-1.45.5 (stdout.txt,242.69 KB, text/plain)
2020-03-27 00:33 UTC, gwendal grignou
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Maciej S. Szmigiero 2019-03-06 21:23:33 UTC
sys-fs/e2fsprogs-1.44.6 fails to install to a temporary install directory
when using MAKEOPTS="-j2" (and possibly others, too).

It looks like it is trying to compile util/symlinks.c in parallel with linking
the resulting util/symlinks.o file into util/symlinks at the install time
(the relevant portion of the build log is attached).

A simple workaround is to run src_install()'s emake with "-j1" - that is,
to serialize only the installation step, while leaving src_compile()
still potentially parallel (ebuild patch attached).

See also bug #675346 comment 1 - it might be the same issue.
Comment 1 Maciej S. Szmigiero 2019-03-06 21:24:43 UTC
Created attachment 568046 [details]
build install log
Comment 2 Maciej S. Szmigiero 2019-03-06 21:26:34 UTC
Created attachment 568048 [details, diff]
workaround ebuild patch
Comment 3 Arfrever Frehtes Taifersar Arahesis 2019-03-07 04:41:18 UTC
It fails differently for me.
Some interesting parts:
        CC symlinks.c
        CC symlinks.c
...
        LD symlinks
        LD symlinks
...
        INSTALL-ELF-LIB /usr/lib64/libe2p.so.2.3
        INSTALL-ELF-LIB /usr/lib64/libe2p.so.2.3
        INSTALL //sbin/tune2fs
/usr/bin/install: cannot create regular file '/var/tmp/portage/sys-fs/e2fsprogs-1.44.6/image/usr/lib64/libe2p.so.2.3': File exists
make[1]: *** [Makefile:433: install-shlibs] Error 1
Comment 4 Arfrever Frehtes Taifersar Arahesis 2019-03-07 04:44:24 UTC
Created attachment 568080 [details]
e2fsprogs-1.44.6:20190307-035957.log
Comment 5 Larry the Git Cow gentoo-dev 2019-03-07 06:34:09 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02762e0987dc086a8f66a33fd13cac57ac9a7d29

commit 02762e0987dc086a8f66a33fd13cac57ac9a7d29
Author:     Lars Wendler <polynomial-c@gentoo.org>
AuthorDate: 2019-03-07 06:33:49 +0000
Commit:     Lars Wendler <polynomial-c@gentoo.org>
CommitDate: 2019-03-07 06:33:49 +0000

    sys-fs/e2fsprogs: Attempt to fix parallel install issue.
    
    Bug: https://bugs.gentoo.org/679640
    Package-Manager: Portage-2.3.62, Repoman-2.3.12
    Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>

 sys-fs/e2fsprogs/e2fsprogs-1.44.6.ebuild                       |  2 +-
 sys-fs/e2fsprogs/files/e2fsprogs-1.44.6-parallel_install.patch | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)
Comment 6 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2019-03-07 06:36:02 UTC
Please re-sync, give the updated ebuild a try and report back success/failure.
To make sure you have the updated ebuild watch for the 

  e2fsprogs-1.44.6-parallel_install.patch

getting applied.
Comment 7 Arfrever Frehtes Taifersar Arahesis 2019-03-07 08:58:11 UTC
I have successfully rebuilt sys-fs/e2fsprogs-1.44.6 10 times with -j10.
But maybe I was just lucky...
Comment 8 Maciej S. Szmigiero 2019-03-07 12:19:20 UTC
Both sys-fs/e2fsprogs-1.44.6 and sys-fs/e2fsprogs-1.45.0 now build and install
fine on the system that previously had this bug - thanks.
Comment 9 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2019-03-07 20:07:07 UTC
*** Bug 679704 has been marked as a duplicate of this bug. ***
Comment 10 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2019-03-08 06:24:20 UTC
*** Bug 679704 has been marked as a duplicate of this bug. ***
Comment 11 gwendal grignou 2020-03-27 00:33:35 UTC
Created attachment 626364 [details]
parallel install failure on e2fsprogs-1.45.5

It is much harder to reproduce, we still see parallel execution related error:

e2fsprogs-1.45.5: make[1]: Entering directory '/var/tmp/portage/sys-fs/e2fsprogs-1.45.5/work/e2fsprogs-1.45.5/lib/ext2fs'
e2fsprogs-1.45.5: 	MKDIR_P /usr/lib64 /usr/lib64
e2fsprogs-1.45.5: make[2]: Entering directory '/var/tmp/portage/sys-fs/e2fsprogs-1.45.5/work/e2fsprogs-1.45.5/lib/ext2fs'
e2fsprogs-1.45.5: 	INSTALL-ELF-LIB /usr/lib64/libext2fs.so.2.4
e2fsprogs-1.45.5: 	MKDIR_P /usr/lib64 /usr/lib64
e2fsprogs-1.45.5: 	INSTALL-ELF-LIB /usr/lib64/libext2fs.so.2.4
e2fsprogs-1.45.5: 	SYMLINK /usr/lib64/libext2fs.so.2
e2fsprogs-1.45.5: 	SYMLINK /usr/lib64/libext2fs.so.2
e2fsprogs-1.45.5: 	SYMLINK /usr/lib64/libext2fs.so
e2fsprogs-1.45.5: ln: failed to create symbolic link '/var/tmp/portage/sys-fs/e2fsprogs-1.45.5/image/usr/lib64/libext2fs.so.2': File exists
e2fsprogs-1.45.5: 	SYMLINK /usr/lib64/libext2fs.so
e2fsprogs-1.45.5: 	LDCONFIG
e2fsprogs-1.45.5: ln: failed to create symbolic link '/var/tmp/portage/sys-fs/e2fsprogs-1.45.5/image/usr/lib64/libext2fs.so': File exists
e2fsprogs-1.45.5: 	MKDIR_P /usr/lib64 /usr/include/ext2fs
e2fsprogs-1.45.5: 	LDCONFIG
e2fsprogs-1.45.5: 	INSTALL_DATA /usr/lib64/libext2fs.a
e2fsprogs-1.45.5: 	MKDIR_P /usr/lib64 /usr/include/ext2fs


Looking at the makefile, without the patch in #5, we have

emerge calls make for rules install and install-libs.
make install triggered a forked make install-libs, so 2 instances of make install-libs are working in parallel.

The patch in #5 force rule install-libs-recursive (called by install-libs) to wait for install_shill_libs_recursive to complete to proceed.,

There is still a problem if the forked instance is triggered just when install_shill_libs_recursive completes.

Instead of patch #5, I propose in the ebuild to just call emake ... install, no install-libs.
Comment 12 Larry the Git Cow gentoo-dev 2020-03-27 08:36:58 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0e2716a8a51d96c6c05c1c5b721c11d8933dff34

commit 0e2716a8a51d96c6c05c1c5b721c11d8933dff34
Author:     Gwendal Grignou <gwendal@chromium.org>
AuthorDate: 2020-03-27 04:01:56 +0000
Commit:     Lars Wendler <polynomial-c@gentoo.org>
CommitDate: 2020-03-27 08:33:40 +0000

    sysdevel/e2fsprogs: install library through install
    
    Given e2fsprogs forks a make instance to install libraries,
    do not specify the rule install-libs when installing e2fsprogs.
    make install will do it once other dependencies are in place.
    
    Closes: https://bugs.gentoo.org/679640
    Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
    Closes: https://github.com/gentoo/gentoo/pull/15132
    Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>

 sys-fs/e2fsprogs/e2fsprogs-1.45.4.ebuild                       |  3 +--
 sys-fs/e2fsprogs/e2fsprogs-1.45.5.ebuild                       |  3 +--
 sys-fs/e2fsprogs/e2fsprogs-1.45.6.ebuild                       |  3 +--
 sys-fs/e2fsprogs/files/e2fsprogs-1.44.6-parallel_install.patch | 10 ----------
 4 files changed, 3 insertions(+), 16 deletions(-)