Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 660754

Summary: >=dev-python/numpy-1.10.4: parallel build issue
Product: Gentoo Linux Reporter: Thomas Deutschmann <whissi>
Component: Current packagesAssignee: Gentoo Science Related Packages <sci>
Status: CONFIRMED ---    
Severity: normal CC: alexander, alonbl, frp.bissey, hydrapolic, klaus818, python, viklevin2
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: https://github.com/numpy/numpy/issues/13080
See Also: https://bugs.gentoo.org/show_bug.cgi?id=679298
https://bugs.gentoo.org/show_bug.cgi?id=623790
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: build.log

Description Thomas Deutschmann gentoo-dev Security 2018-07-08 21:45:26 UTC
Created attachment 538880 [details]
build.log

See build.log for details.


Portage 2.3.40 (python 3.6.5-final-0, default/linux/x86/17.0, gcc-7.3.0, glibc-2.26-r7, 4.14.52-gentoo i686)
=================================================================
System uname: Linux-4.14.52-gentoo-i686-Intel-R-_Core-TM-_i7-3770K_CPU_@_3.50GHz-with-gentoo-2.4.1
KiB Mem:     3107600 total,    778568 free
KiB Swap:     488276 total,    488276 free
Timestamp of repository gentoo: Sun, 08 Jul 2018 19:44:24 +0000
Head commit of repository gentoo: 3e71460c2f988d8821b6d4c9efae2cc5999a4820

sh bash 4.4_p12
ld GNU ld (Gentoo 2.30 p2) 2.30.0
app-shells/bash:          4.4_p12::gentoo
dev-lang/perl:            5.24.3-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.6.5::gentoo
dev-util/cmake:           3.9.6::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.15.1-r2::gentoo
sys-devel/binutils:       2.30-r2::gentoo
sys-devel/gcc:            7.3.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26-r7::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000

ABI="x86"
ABI_X86="32"
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
ACCEPT_PROPERTIES="*"
ACCEPT_RESTRICT="*"
ARCH="x86"
BROOT=""
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
CHOST="i686-pc-linux-gnu"
CHOST_x86="i686-pc-linux-gnu"
COLLISION_IGNORE="/lib/modules/* *.py[co] *$py.class */dropin.cache"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CPU_FLAGS_X86="mmx mmxext sse sse2"
CXXFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
DEFAULT_ABI="x86"
EDITOR="/usr/bin/mcedit"
ELIBC="glibc"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
EPREFIX=""
EROOT="/"
ESYSROOT="/"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs cgroup config-protect-if-modified distlocks downgrade-backup ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -march=i686 -pipe"
GCC_SPECS=""
GRUB_PLATFORMS="efi-32 pc"
GSETTINGS_BACKEND="dconf"
HOME="/root"
INFOPATH="/usr/share/gcc-data/i686-pc-linux-gnu/7.3.0/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.30/info:/usr/share/info"
INPUT_DEVICES="libinput keyboard mouse"
IUSE_IMPLICIT="abi_x86_32 prefix prefix-chain prefix-guest"
KERNEL="linux"
L10N="en en-US de de-DE"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_MESSAGES="C"
LC_PAPER="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LIBDIR_x86="lib"
LINGUAS="en de"
LOGNAME="root"
MAIL="/var/mail/root"
MAKEOPTS="--jobs 5 --load-average 7.95"
MULTILIB_ABIS="x86"
NOCOLOR="true"
OFFICE_IMPLEMENTATION="libreoffice"
OLDPWD="/root"
OPENCL_PROFILE="mesa"
OPENGL_PROFILE="xorg-x11"
PAGER="/usr/bin/less"
PATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
PHP_TARGETS="php5-6 php7-0"
POSTGRES_TARGETS="postgres9_5 postgres10"
PWD="/tmp/stable"
PYTHONDONTWRITEBYTECODE="1"
PYTHON_SINGLE_TARGET="python3_6"
PYTHON_TARGETS="python2_7 python3_6"
QT_GRAPHICSSYSTEM="raster"
ROOT="/"
ROOTPATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
RUBY_TARGETS="ruby22 ruby23"
SHELL="/bin/bash"
SHLVL="2"
SSH_TTY="/dev/pts/0"
SYSROOT="/"
TERM="tmux-256color"
TMUX="/tmp/tmux-0/default,5402,0"
TMUX_PANE="%2"
TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE="1"
USER="root"
USERLAND="GNU"
VIDEO_CARDS="vmware"
Comment 1 François Bissey 2018-07-08 21:48:43 UTC
Is it after a recent upgrade of gcc?
Comment 2 Thomas Deutschmann gentoo-dev Security 2018-07-08 21:55:21 UTC
Mh, don't think so:

     Mon Jun 25 17:41:35 2018 >>> dev-python/numpy-1.10.4
     Sun Jul  1 21:25:20 2018 >>> dev-python/numpy-1.10.4
     Wed Jul  4 01:34:20 2018 >>> dev-python/numpy-1.10.4
     Wed Jul  4 01:42:25 2018 >>> dev-python/numpy-1.10.4
     Fri Jul  6 23:17:27 2018 >>> dev-python/numpy-1.10.4


     Wed Dec 20 23:40:20 2017 >>> sys-devel/gcc-6.4.0
     Fri Jan 12 16:51:32 2018 >>> sys-devel/gcc-6.4.0-r1
     Sat Mar  3 23:53:15 2018 >>> sys-devel/gcc-6.4.0-r1
     Tue Jun 19 13:34:22 2018 >>> sys-devel/gcc-7.3.0-r3
Comment 3 François Bissey 2018-07-08 22:01:17 UTC
(In reply to Thomas Deutschmann from comment #2)
> Mh, don't think so:
> 
>      Mon Jun 25 17:41:35 2018 >>> dev-python/numpy-1.10.4
>      Sun Jul  1 21:25:20 2018 >>> dev-python/numpy-1.10.4
>      Wed Jul  4 01:34:20 2018 >>> dev-python/numpy-1.10.4
>      Wed Jul  4 01:42:25 2018 >>> dev-python/numpy-1.10.4
>      Fri Jul  6 23:17:27 2018 >>> dev-python/numpy-1.10.4
> 
> 
>      Wed Dec 20 23:40:20 2017 >>> sys-devel/gcc-6.4.0
>      Fri Jan 12 16:51:32 2018 >>> sys-devel/gcc-6.4.0-r1
>      Sat Mar  3 23:53:15 2018 >>> sys-devel/gcc-6.4.0-r1
>      Tue Jun 19 13:34:22 2018 >>> sys-devel/gcc-7.3.0-r3

OK so that's not change of fortran soname. That would have been so easy. Instead it looks more like the parallel building bug that has been plaguing scipy for a while
/var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-i686-3.6/numpy/linalg/lapack_lite/python_xerbla.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
/var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-i686-3.6/numpy/linalg/lapack_lite/python_xerbla.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status

Try again at "-j1".
Comment 4 Thomas Deutschmann gentoo-dev Security 2018-07-08 22:47:51 UTC
Yup, looks like a parallel build issue: Building with MAKEOPTS=-j2 successful.
Comment 5 Alexander Tsoy 2018-08-22 11:23:13 UTC
(In reply to Thomas Deutschmann from comment #4)
> Building with MAKEOPTS=-j2 successful.

I can build only with MAKEOPTS=-j1. Got the same error as you with MAKEOPTS=-j2:

/usr/lib64/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
/usr/lib64/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)
error: Command "x86_64-pc-linux-gnu-gcc -O0 -fno-omit-frame-pointer -ggdb -pipe -shared -Wl,-O1 -Wl,--as-needed -shared -O0 -fno-omit-frame-pointer -ggdb -pipe -fno-strict-aliasing /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_litemodule.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/python_xerbla.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/zlapack_lite.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/dlapack_lite.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/blas_lite.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/dlamch.o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/f2c_lite.o -L/usr/lib64 -L/var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/temp.linux-x86_64-3.6 -lpython3.6m -o /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/lib/numpy/linalg/lapack_lite.cpython-36m-x86_64-linux-gnu.so" failed with exit status 1
Comment 6 Tomáš Mózes 2018-09-20 12:43:06 UTC
(In reply to Alexander Tsoy from comment #5)
> (In reply to Thomas Deutschmann from comment #4)
> > Building with MAKEOPTS=-j2 successful.
> 
> I can build only with MAKEOPTS=-j1. Got the same error as you with
> MAKEOPTS=-j2:
> 
> /usr/lib64/python3.6/distutils/dist.py:261: UserWarning: Unknown
> distribution option: 'test_suite'
>   warnings.warn(msg)
> /usr/lib64/python3.6/distutils/dist.py:261: UserWarning: Unknown
> distribution option: 'define_macros'
>   warnings.warn(msg)
> error: Command "x86_64-pc-linux-gnu-gcc -O0 -fno-omit-frame-pointer -ggdb
> -pipe -shared -Wl,-O1 -Wl,--as-needed -shared -O0 -fno-omit-frame-pointer
> -ggdb -pipe -fno-strict-aliasing
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_litemodule.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/python_xerbla.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/zlapack_lite.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/dlapack_lite.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/blas_lite.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/dlamch.o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6/numpy/linalg/lapack_lite/f2c_lite.o -L/usr/lib64
> -L/var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> temp.linux-x86_64-3.6 -lpython3.6m -o
> /var/tmp/portage/dev-python/numpy-1.10.4/work/numpy-1.10.4-python3_6/build/
> lib/numpy/linalg/lapack_lite.cpython-36m-x86_64-linux-gnu.so" failed with
> exit status 1

Got the same error on ~amd64 with numpy 1.14.5. MAKEOPTS="-j1" emerge numpy works.
Comment 7 Larry the Git Cow gentoo-dev 2018-09-20 12:58:57 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0b5b4b4c8770a61a9eb28ca5fc45ceca65a7ee6d

commit 0b5b4b4c8770a61a9eb28ca5fc45ceca65a7ee6d
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2018-09-20 12:50:41 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2018-09-20 12:58:48 +0000

    dev-python/numpy: force MAKEOPTS=-j1
    
    Looks like nobody cares. To allow users to build that package in
    the meanwhile, let's force MAKEOPTS=-j1.
    
    Bug: https://bugs.gentoo.org/660754
    Package-Manager: Portage-2.3.49, Repoman-2.3.10

 dev-python/numpy/numpy-1.10.4.ebuild | 2 ++
 dev-python/numpy/numpy-1.13.3.ebuild | 2 ++
 dev-python/numpy/numpy-1.14.5.ebuild | 2 ++
 dev-python/numpy/numpy-9999.ebuild   | 2 ++
 4 files changed, 8 insertions(+)
Comment 8 Larry the Git Cow gentoo-dev 2019-03-02 22:57:39 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=885d5f81e25739a1b088e0d69369d2db992454cc

commit 885d5f81e25739a1b088e0d69369d2db992454cc
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-03-02 22:57:21 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-03-02 22:57:21 +0000

    dev-python/numpy: drop parallel build workaround
    
    Cannot reproduce with >numpy-1.13.3 anymore.
    
    Bug: https://bugs.gentoo.org/660754
    Package-Manager: Portage-2.3.62, Repoman-2.3.12
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 dev-python/numpy/numpy-1.14.5.ebuild | 4 +---
 dev-python/numpy/numpy-1.15.4.ebuild | 2 --
 dev-python/numpy/numpy-9999.ebuild   | 2 --
 3 files changed, 1 insertion(+), 7 deletions(-)
Comment 9 Thomas Deutschmann gentoo-dev Security 2019-03-03 16:15:48 UTC
*** Bug 679298 has been marked as a duplicate of this bug. ***
Comment 10 Thomas Deutschmann gentoo-dev Security 2019-03-03 16:22:21 UTC
Looks like this is still happening (bug 679298). But it looks like it is only happening for USE=-lapack. I'll push an updated ebuild shortly which will force MAKEOPTS=-j1 only for USE=-lapack... please test and report.
Comment 11 Larry the Git Cow gentoo-dev 2019-03-03 16:25:40 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=09f3824f30a37092664909b512043b0321e3d14d

commit 09f3824f30a37092664909b512043b0321e3d14d
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-03-03 16:23:05 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-03-03 16:25:31 +0000

    dev-python/numpy: re-add parallel make workaround for USE=-lapack
    
    Bug: https://bugs.gentoo.org/660754
    Package-Manager: Portage-2.3.62, Repoman-2.3.12
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 dev-python/numpy/numpy-1.13.3.ebuild | 4 ++--
 dev-python/numpy/numpy-1.14.5.ebuild | 2 ++
 dev-python/numpy/numpy-1.15.4.ebuild | 2 ++
 dev-python/numpy/numpy-1.16.1.ebuild | 2 ++
 dev-python/numpy/numpy-9999.ebuild   | 2 ++
 5 files changed, 10 insertions(+), 2 deletions(-)
Comment 12 Larry the Git Cow gentoo-dev 2019-03-03 16:29:31 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d5337dedc68adc4dfcbfe8bb045c96554512aabf

commit d5337dedc68adc4dfcbfe8bb045c96554512aabf
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-03-03 16:29:22 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-03-03 16:29:22 +0000

    dev-python/numpy: force MAKEOPTS=-j1 for all USE configurations
    
    ...even failing for USE=lapack on some systems.
    
    Bug: https://bugs.gentoo.org/660754
    Package-Manager: Portage-2.3.62, Repoman-2.3.12
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 dev-python/numpy/numpy-1.13.3.ebuild | 2 +-
 dev-python/numpy/numpy-1.14.5.ebuild | 2 +-
 dev-python/numpy/numpy-1.15.4.ebuild | 2 +-
 dev-python/numpy/numpy-1.16.1.ebuild | 2 +-
 dev-python/numpy/numpy-9999.ebuild   | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)