Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 717828

Summary: dev-python/numexpr-2.6.9 compilation fail: fatal error: _numpyconfig.h: No such file or directory
Product: Gentoo Linux Reporter: MeatShooter <universe.government>
Component: Current packagesAssignee: Gentoo Linux bug wranglers <bug-wranglers>
Status: RESOLVED DUPLICATE    
Severity: normal CC: idarktemplar, me, mgorny, vivo75
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: build.log

Description MeatShooter 2020-04-17 07:37:29 UTC
So, I want to emerge dev-python/pandas, and it requires dev-python/numexpr, which cannot be emerged.
It fails with fatal error: _numpyconfig.h: No such file or directory


Reproducible: Always





# emerge pandas --verbose

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] dev-python/numexpr-2.6.9::gentoo  USE="-mkl" PYTHON_TARGETS="python3_6 -python3_7" 0 KiB
[ebuild  N     ] sci-libs/scipy-1.1.0::gentoo  USE="-doc -sparse -test" PYTHON_TARGETS="python2_7 python3_6 -python3_7" 0 KiB
[ebuild  N     ] dev-python/bottleneck-1.2.1-r1::gentoo  USE="-test" PYTHON_TARGETS="python3_6 -python3_7" 0 KiB
[ebuild  N     ] dev-python/pandas-0.24.2::gentoo  USE="X (-doc) -full-support -minimal -test" PYTHON_TARGETS="python3_6 -python3_7" 0 KiB

Total: 4 packages (4 new), Size of downloads: 0 KiB


>>> Verifying ebuild manifests

>>> Emerging (1 of 4) dev-python/numexpr-2.6.9::gentoo
 * Fetching files in the background.
 * To view fetch progress, run in another terminal:
 * tail -f /var/log/emerge-fetch.log
 * numexpr-2.6.9.tar.gz BLAKE2B SHA512 size ;-) ...                                                                      [ ok ]
>>> Unpacking source...
>>> Unpacking numexpr-2.6.9.tar.gz to /var/tmp/portage/dev-python/numexpr-2.6.9/work
>>> Source unpacked in /var/tmp/portage/dev-python/numexpr-2.6.9/work
>>> Preparing source in /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9 ...
 * python3_6: running distutils-r1_run_phase python_compile
python3.6 setup.py build
system_info:
  NOT AVAILABLE

running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
creating /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/src.linux-x86_64-3.6
creating /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/src.linux-x86_64-3.6/numexpr
building extension "numexpr.interpreter" sources
build_src: building npy-pkg config files
running build_py
creating /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/necompiler.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/expressions.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/version.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/cpuinfo.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/utils.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying numexpr/__init__.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
copying /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/src.linux-x86_64-3.6/numexpr/__config__.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr
creating /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr/tests
copying ./numexpr/tests/test_numexpr.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr/tests
copying ./numexpr/tests/__init__.py -> /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/lib/numexpr/tests
Skipping optional fixer: buffer
Skipping optional fixer: idioms
Skipping optional fixer: set_literal
Skipping optional fixer: ws_comma
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'numexpr.interpreter' extension
compiling C++ sources
C compiler: x86_64-pc-linux-gnu-g++ -O2 -pipe -fPIC

creating /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/temp.linux-x86_64-3.6/numexpr
compile options: '-I/usr/lib64/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m -c'
extra options: '-funroll-all-loops'
x86_64-pc-linux-gnu-g++: numexpr/interpreter.cpp
x86_64-pc-linux-gnu-g++: numexpr/module.cpp
x86_64-pc-linux-gnu-g++: numexpr/numexpr_object.cpp
In file included from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/npy_common.h:5,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:4,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from numexpr/module.hpp:14,
                 from numexpr/interpreter.cpp:10:
/usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h:4:10: fatal error: _numpyconfig.h: No such file or directory
    4 | #include "_numpyconfig.h"
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/npy_common.h:5,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:4,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from numexpr/module.hpp:14,
                 from numexpr/numexpr_object.cpp:10:
/usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h:4:10: fatal error: _numpyconfig.h: No such file or directory
    4 | #include "_numpyconfig.h"
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/npy_common.h:5,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:4,
                 from /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from numexpr/module.hpp:14,
                 from numexpr/module.cpp:12:
/usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h:4:10: fatal error: _numpyconfig.h: No such file or directory
    4 | #include "_numpyconfig.h"
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
Warning: Assuming default configuration (numexpr/tests/{setup_tests,setup}.py was not found)error: Command "x86_64-pc-linux-gnu-g++ -O2 -pipe -fPIC -I/usr/lib64/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m -c numexpr/interpreter.cpp -o /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/temp.linux-x86_64-3.6/numexpr/interpreter.o -MMD -MF /var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9-python3_6/temp.linux-x86_64-3.6/numexpr/interpreter.o.d -funroll-all-loops" failed with exit status 1
 * ERROR: dev-python/numexpr-2.6.9::gentoo failed (compile phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_compile
 *   environment, line 4082:  Called distutils-r1_src_compile
 *   environment, line 1320:  Called _distutils-r1_run_foreach_impl 'python_compile'
 *   environment, line  377:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 3615:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 2924:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 2922:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_compile'
 *   environment, line  878:  Called distutils-r1_run_phase 'python_compile'
 *   environment, line 1313:  Called python_compile
 *   environment, line 3360:  Called distutils-r1_python_compile
 *   environment, line 1183:  Called esetup.py 'build'
 *   environment, line 1979:  Called die
 * The specific snippet of code:
 *       "${@}" || die "${die_args[@]}";
 * 
 * If you need support, post the output of `emerge --info '=dev-python/numexpr-2.6.9::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-python/numexpr-2.6.9::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/dev-python/numexpr-2.6.9/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-python/numexpr-2.6.9/temp/environment'.
 * Working directory: '/var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9'
 * S: '/var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9'

>>> Failed to emerge dev-python/numexpr-2.6.9, Log file:

>>>  '/var/tmp/portage/dev-python/numexpr-2.6.9/temp/build.log'

 * Messages for package dev-python/numexpr-2.6.9:

 * ERROR: dev-python/numexpr-2.6.9::gentoo failed (compile phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_compile
 *   environment, line 4082:  Called distutils-r1_src_compile
 *   environment, line 1320:  Called _distutils-r1_run_foreach_impl 'python_compile'
 *   environment, line  377:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 3615:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 2924:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 2922:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_compile'
 *   environment, line  878:  Called distutils-r1_run_phase 'python_compile'
 *   environment, line 1313:  Called python_compile
 *   environment, line 3360:  Called distutils-r1_python_compile
 *   environment, line 1183:  Called esetup.py 'build'
 *   environment, line 1979:  Called die
 * The specific snippet of code:
 *       "${@}" || die "${die_args[@]}";
 * 
 * If you need support, post the output of `emerge --info '=dev-python/numexpr-2.6.9::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-python/numexpr-2.6.9::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/dev-python/numexpr-2.6.9/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-python/numexpr-2.6.9/temp/environment'.
 * Working directory: '/var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9'
 * S: '/var/tmp/portage/dev-python/numexpr-2.6.9/work/numexpr-2.6.9'












# emerge --info


Portage 2.3.89 (python 3.6.10-final-0, default/linux/amd64/17.1/desktop, gcc-9.2.0, glibc-2.29-r7, 5.4.28-gentoo x86_64)
=================================================================
System uname: Linux-5.4.28-gentoo-x86_64-Intel-R-_Core-TM-_i7-4720HQ_CPU_@_2.60GHz-with-gentoo-2.6
KiB Mem:    16306220 total,   3536380 free
KiB Swap:   16777212 total,  16777212 free
Timestamp of repository gentoo: Fri, 17 Apr 2020 07:00:01 +0000
Head commit of repository gentoo: b3135cb530048af58697cfb823af1f878d38da05
sh bash 4.4_p23-r1
ld GNU ld (Gentoo 2.33.1 p2) 2.33.1
app-shells/bash:          4.4_p23-r1::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.30.1::gentoo
dev-lang/python:          2.7.17-r2::gentoo, 3.6.10-r1::gentoo, 3.7.7-r1::gentoo
dev-util/cmake:           3.16.5::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.33.1-r1::gentoo
sys-devel/gcc:            9.2.0-r2::gentoo
sys-devel/gcc-config:     2.2.1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.29-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-extra-opts: 
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-metamanifest: yes

localrepo
    location: /var/db/repos/localrepo
    masters: gentoo

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="ru_RU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus dri dts dvd dvdr elogind emboss encode exif flac fortran gdbm gif gpm gtk iconv icu ipv6 jpeg lcms ldap libnotify libtirpc mad mng mp3 mp4 mpeg mtp multilib ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds qt5 readline sdl seccomp spell split-usr ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vorbis wxwidgets x264 xattr xcb xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="synaptics libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 MeatShooter 2020-04-17 07:39:33 UTC
Created attachment 633368 [details]
build.log
Comment 2 John Bowler 2020-04-17 15:04:24 UTC
Seems to be the same as 717804, I encountered this problem with a vigra build, but it looks like an issue with numpy itself:

$ equery belongs /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h
 * Searching for /usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h ... 
dev-python/numpy-1.17.4-r2 (/usr/lib64/python3.6/site-packages/numpy/core/include/numpy/numpyconfig.h)
Comment 3 i.Dark_Templar 2020-04-17 19:06:54 UTC
Happens to me as well. It's definitely same as 717804, and it's a bug in build of numpy.

Reproduction is easy and takes 2 steps:
1. re-emerge numpy;
# emerge -1 dev-python/numpy
2. emerge something depending on numpy
# emerge -1 dev-python/matplotlib

Result:
Build fails for numpy dependency (dev-python/matplotlib)

Expected result:
dev-python/matplotlib should build successfully.

Note: before duing such experiment I'd recommend to use quickpkg to save current copy of numpy if it's working.

I have only python3.7 enabled for numpy.

Files list before numpy re-emerge:

$ equery f numpy | grep numpyconfig
/usr/lib/python3.7/site-packages/numpy/core/include/numpy/_numpyconfig.h
/usr/lib/python3.7/site-packages/numpy/core/include/numpy/numpyconfig.h

After numpy re-emerge _numpyconfig.h and a few more files disappear.

I guess it might be caused by following patch, because working numpy version was built without this patch, but just removing patch makes numpy build fail for me:

$ diff -u /var/db/pkg/dev-python/numpy-1.16.5/numpy-1.16.5.ebuild /usr/portage/dev-python/numpy/numpy-1.16.5.ebuild 
--- /var/db/pkg/dev-python/numpy-1.16.5/numpy-1.16.5.ebuild     2020-01-30 21:49:44.006938297 +0300
+++ /usr/portage/dev-python/numpy/numpy-1.16.5.ebuild   2020-04-16 21:09:40.000000000 +0300
@@ -22,7 +22,7 @@
        )"
 LICENSE="BSD"
 SLOT="0"
-KEYWORDS="~alpha amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris ~x64-solaris ~x86-solaris"
+KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~mips ppc ppc64 s390 sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris ~x64-solaris ~x86-solaris"
 IUSE="doc lapack test"
 RESTRICT="!test? ( test )"
 
@@ -41,6 +41,7 @@
 
 PATCHES=(
        "${FILESDIR}"/${PN}-1.15.4-no-hardcode-blas.patch
+       "${FILESDIR}"/numpy-1.16.5-setup.py-install-skip-build-fails.patch
 )
 
 src_unpack() {
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-04-18 05:29:38 UTC

*** This bug has been marked as a duplicate of bug 717502 ***
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-04-18 05:30:22 UTC

*** This bug has been marked as a duplicate of bug 717502 ***