Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 659804 - dev-python/numba-0.37.0: numba.tests.test_linalg.TestLinalgLstsq crashes on x86
Summary: dev-python/numba-0.37.0: numba.tests.test_linalg.TestLinalgLstsq crashes on x86
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 658884
  Show dependency tree
 
Reported: 2018-07-01 22:34 UTC by Thomas Deutschmann (RETIRED)
Modified: 2018-07-09 01:24 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,182.78 KB, text/plain)
2018-07-01 22:34 UTC, Thomas Deutschmann (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Deutschmann (RETIRED) gentoo-dev 2018-07-01 22:34:58 UTC
Created attachment 538058 [details]
build.log

Please see build.log for details.


Portage 2.3.40 (python 3.6.5-final-0, default/linux/x86/17.0, gcc-7.3.0, glibc-2.26-r7, 4.9.95-gentoo i686)
=================================================================
System uname: Linux-4.9.95-gentoo-i686-Intel-R-_Core-TM-_i7-3770K_CPU_@_3.50GHz-with-gentoo-2.4.1
KiB Mem:     3108308 total,   1934432 free
KiB Swap:     488276 total,    474896 free
Timestamp of repository gentoo: Sun, 01 Jul 2018 17:46:15 +0000
Head commit of repository gentoo: 8976c97207c582dcdc87e474f3d435fcb6929037

sh bash 4.4_p12
ld GNU ld (Gentoo 2.30 p2) 2.30.0
app-shells/bash:          4.4_p12::gentoo
dev-lang/perl:            5.24.3-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.6.5::gentoo
dev-util/cmake:           3.9.6::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.15.1-r2::gentoo
sys-devel/binutils:       2.30-r2::gentoo
sys-devel/gcc:            7.3.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26-r7::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000

ABI="x86"
ABI_X86="32"
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
ACCEPT_PROPERTIES="*"
ACCEPT_RESTRICT="*"
ARCH="x86"
BROOT=""
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
CHOST="i686-pc-linux-gnu"
CHOST_x86="i686-pc-linux-gnu"
COLLISION_IGNORE="/lib/modules/* *.py[co] *$py.class */dropin.cache"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CPU_FLAGS_X86="mmx mmxext sse sse2"
CXXFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
DEFAULT_ABI="x86"
EDITOR="/usr/bin/mcedit"
ELIBC="glibc"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
EPREFIX=""
EROOT="/"
ESYSROOT="/"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs cgroup config-protect-if-modified distlocks downgrade-backup ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -march=i686 -pipe"
GCC_SPECS=""
GRUB_PLATFORMS="efi-32 pc"
GSETTINGS_BACKEND="dconf"
HOME="/root"
INFOPATH="/usr/share/gcc-data/i686-pc-linux-gnu/7.3.0/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.30/info:/usr/share/info"
INPUT_DEVICES="libinput keyboard mouse"
IUSE_IMPLICIT="abi_x86_32 prefix prefix-chain prefix-guest"
KERNEL="linux"
L10N="en en-US de de-DE"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_MESSAGES="C"
LC_PAPER="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LIBDIR_x86="lib"
LINGUAS="en de"
LOGNAME="root"
MAIL="/var/mail/root"
MAKEOPTS="--jobs 5 --load-average 7.95"
MULTILIB_ABIS="x86"
NOCOLOR="true"
OFFICE_IMPLEMENTATION="libreoffice"
OPENCL_PROFILE="mesa"
OPENGL_PROFILE="xorg-x11"
PAGER="/usr/bin/less"
PATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
PHP_TARGETS="php5-6 php7-0"
POSTGRES_TARGETS="postgres9_5 postgres10"
PWD="/root"
PYTHONDONTWRITEBYTECODE="1"
PYTHON_SINGLE_TARGET="python3_6"
PYTHON_TARGETS="python2_7 python3_6"
QT_GRAPHICSSYSTEM="raster"
ROOT="/"
ROOTPATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
RUBY_TARGETS="ruby22 ruby23"
SHELL="/bin/bash"
SHLVL="2"
SSH_TTY="/dev/pts/0"
SYSROOT="/"
TERM="tmux-256color"
TMUX="/tmp/tmux-0/default,6917,0"
TMUX_PANE="%3"
TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE="1"
USER="root"
USERLAND="GNU"
VIDEO_CARDS="vmware"
Comment 1 Pacho Ramos gentoo-dev 2018-07-02 08:13:17 UTC
It seems tests want more memory... but I don't know how much they need:
..............................sss.s...........MemoryError: Insufficient memory for buffer allocation                             required by LAPACK.
Comment 2 Mike Gilbert gentoo-dev 2018-07-03 17:34:13 UTC
I am unable to reproduce this in a stable x86 container with a 1 GB RSS limit (ulimit -m 1G).
Comment 3 Mike Gilbert gentoo-dev 2018-07-03 17:35:50 UTC
Oh, I have dev-python/numpy[-lapack]. I will try enabling lapack.
Comment 4 Mike Gilbert gentoo-dev 2018-07-03 20:41:45 UTC
Nope, still can't reproduce this, even with lapack enabled.

Whissi: Are you able to reproduce this consistently?
Comment 5 Thomas Deutschmann (RETIRED) gentoo-dev 2018-07-03 23:07:34 UTC
Yes, it is reproducible.

Due to 32-bit, this system has only 2.96GB memory available.

I just re-run test while watching memory usage:
It looks like it didn't even hit that 2.96GB limit. Memory peak usage was ~1.2GB when tests ended with the error shown. Not even swapping or something like this.

I also noticed that the test process (/usr/bin/python2.7 -c import numba; numba.test()) which will get started isn't releasing any memory. I.e. starting with ~200MB and increasing memory until test suite is failing.
Comment 6 Mike Gilbert gentoo-dev 2018-07-06 20:17:52 UTC
Is anyone else able to reproduce this on x86?
Comment 7 Mike Gilbert gentoo-dev 2018-07-06 20:31:05 UTC
I modified the python_test function to run the tests in verbose mode. If you can re-run the tests, we should be able to tell which one is triggering the memory error.
Comment 8 Mike Gilbert gentoo-dev 2018-07-06 22:24:11 UTC
I was able to reproduce the error with dev-python/scipy installed. I suspect the test is skipped without it.
Comment 9 Mike Gilbert gentoo-dev 2018-07-06 22:57:44 UTC
The failing test is numba.tests.test_linalg.TestLinalgLstsq.

strace shows it trying to allocate ~3.5G of memory.

mmap2(NULL, 3752775680, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Comment 10 Thomas Deutschmann (RETIRED) gentoo-dev 2018-07-06 23:11:19 UTC
It passed for me in verbose mode?!
Comment 11 Thomas Deutschmann (RETIRED) gentoo-dev 2018-07-06 23:16:21 UTC
Ah, at least during the last test run, sci-libs/scipy wasn't installed:

> test_linalg_lstsq (numba.tests.test_linalg.TestLinalgLstsq) ... skipped 'LAPACK needs Scipy 0.16+'
Comment 12 Larry the Git Cow gentoo-dev 2018-07-09 01:24:47 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ef879f0851014f832170d6104ca4479c67b46b5b

commit ef879f0851014f832170d6104ca4479c67b46b5b
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2018-07-09 01:13:09 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2018-07-09 01:24:40 +0000

    dev-python/numba: skip memory-intensive test on 32-bit systems
    
    Closes: https://bugs.gentoo.org/659804
    Package-Manager: Portage-2.3.40_p15, Repoman-2.3.9_p247

 dev-python/numba/files/numba-skip-tests.patch | 14 ++++++++++++++
 dev-python/numba/numba-0.37.0.ebuild          |  4 ++++
 2 files changed, 18 insertions(+)