Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 393539 - dev-python/numpy-{1.5.1 -> 1.6.1} fail test phase
Summary: dev-python/numpy-{1.5.1 -> 1.6.1} fail test phase
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: TESTFAILURE
Depends on:
Blocks: 418291
  Show dependency tree
 
Reported: 2011-12-07 18:34 UTC by Ian Delaney (RETIRED)
Modified: 2012-06-01 14:10 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
a cutdown of build log for numpy-1.5.1 (cutdown_numpy-1.5.1.build.log,313.37 KB, text/plain)
2011-12-07 18:34 UTC, Ian Delaney (RETIRED)
Details
a cutdown of build log for numpy-1.6.1 (cutdown_numpy-1.6.1.build.log,75.37 KB, text/plain)
2011-12-07 18:34 UTC, Ian Delaney (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ian Delaney (RETIRED) gentoo-dev 2011-12-07 18:34:06 UTC
Created attachment 295113 [details]
a cutdown of build log for numpy-1.5.1

Testing initially cython test phase lead to finding that numpy itself suffers a failure at test phase, which was curiously not picked up on the last stabilisation of 1.6.0.  Checked and tested all numpy versions and they all trip over.

The failure is likely fixable but it's one I can't figure.

The file that is reported to trip over is _configtest.c.  This however must be generated during configure or compile because it's not present at unpack so I can't pin down what initial file to change.

_configtest.c:5:16: error: size of array 'test_array' is negative
_configtest.c:7:16: error: size of array 'test_array' is negative
_configtest.c:7:16: error: size of array 'test_array' is negative
_configtest.c:8:12: error: 'HAVE_DECL_SIGNBIT' undeclared (first use in this function)
_configtest.c:8:12: error: 'HAVE_DECL_SIGNBIT' undeclared (first use in this function)

about sums up what goes wrong.  grepping for HAVE_DECL_SIGNBIT gives many hits, too many.  It takes someone with more background in this to pin down what needs a sed statement.  It's feasible that the missing header that has HAVE_DECL_SIGNBIT for the generated _configtest.c just may rectify the size of array 'test_array' is negative.  Anyway, this one I cqan't find a resolution for.
Comment 1 Ian Delaney (RETIRED) gentoo-dev 2011-12-07 18:34:44 UTC
Created attachment 295115 [details]
a cutdown of build log for numpy-1.6.1
Comment 2 Ian Delaney (RETIRED) gentoo-dev 2011-12-23 13:27:21 UTC

*** This bug has been marked as a duplicate of bug 351057 ***
Comment 3 Ian Delaney (RETIRED) gentoo-dev 2011-12-23 13:30:17 UTC
*** Bug 351057 has been marked as a duplicate of this bug. ***
Comment 4 Ian Delaney (RETIRED) gentoo-dev 2012-05-31 05:43:11 UTC
you realise a test fail doesn't block stabilisation?
Comment 5 Dirkjan Ochtman (RETIRED) gentoo-dev 2012-05-31 07:03:07 UTC
A test failure that we understand and know to be harmless doesn't have to block stabilization. A test failure we don't understand or one that exposes an actual problem should certainly block stabilization. The comments here don't make me confident that this one falls in the former category.
Comment 6 Mike Gilbert gentoo-dev 2012-05-31 15:01:16 UTC
If I read those build logs correctly, the test failure in 1.5.1 is this:

ERROR: Failure: OSError (/mnt/gen2/tmpdir/portage/dev-python/numpy-1.5.1/work/numpy-1.5.1/test-3.2/lib/python/numpy/core/multiarray.pyd: cannot open shared object file: No such file or directory)


Your log for 1.6.1 shows a successful test run, and I have confirmed that with 1.6.1-r1.

I think the errors coming from _configtest.c are a different issue. In fact, I suspect it might not even be an issue at all; a name like "_configtest" indicates that it is probably just probing to see what works on a given system.
Comment 7 Ian Delaney (RETIRED) gentoo-dev 2012-06-01 13:38:35 UTC
(In reply to comment #6)
> If I read those build logs correctly, the test failure in 1.5.1 is this:
> 
> ERROR: Failure: OSError
> (/mnt/gen2/tmpdir/portage/dev-python/numpy-1.5.1/work/numpy-1.5.1/test-3.2/
> lib/python/numpy/core/multiarray.pyd: cannot open shared object file: No
> such file or directory)
> 
> 
> I think the errors coming from _configtest.c are a different issue. In fact,
> I suspect it might not even be an issue at all; a name like "_configtest"
> indicates that it is probably just probing to see what works on a given
> system.

nm about _configtest.c, they are merely warnings.  Where to start

archtester numpy # USE_PYTHON="2.6 3.2" ebuild  numpy-1.6.0.ebuild clean test

(numpy-1.6.0/test-2.6/lib/python/numpy/polynomial/tests/)

copying build-2.6/temp.linux-x86_64-2.6/libnpymath.a -> /mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0/test-2.6/lib/python/numpy/core/lib
.....................................................................................................................................................----------------------------------------------------------------------
Ran 3529 tests in 16.213s
________________________________________________________________________-

/mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0/build/py3k/build-3.2/lib.linux-x86_64-3.2/numpy /mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0

....................................................................E........
======================================================================
ERROR: Failure: ImportError (cannot import name Random)

Ran 2312 tests in 10.523s

FAILED (KNOWNFAIL=5, SKIP=3, errors=10, failures=1)

So;
     sed -e "s:'%s.pyd' % libname:'%s.cpython-32.so' % libname:" 
     -i numpy/ctypeslib.py

fixes python2.  As for python3; where to begin.

# ls  /mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0/

we have after the build;

build, build-2.6, numpy test-3.2  plus other original files & folders.
remember this was made with USE_PYTHON="2.6 3.2".
Where then may I ask is test-2.6?  Was never made.
What is build?  As it does in python packages it's the folder where the sources are copied and 2to3 refactors the content.  However, the ebuild simply point the PYTHONPATH to

       pushd "${S}/test-${PYTHON_ABI}/"lib* > /dev/null
       PYTHONPATH=python "$(PYTHON)" -c "import numpy; numpy.test()"

So a python2.6 test appears to be being run in test-${PYTHON_ABI}
which is in fact test-3.2  and it passed.  well wonders never cease.
I think it is in fact a python2 compliant folder named test-3.2.  Oh my.

So, next;

   if [[ $(python_get_version --major) == "3" ]]; then
      pushd "${S}"/$(find build/py3build-${PYTHON_ABIlib.linux-*-${PYTHON_ABI}/ -name ${PN}) || die
      PYTHONPATH=. "$(PYTHON)" -c "import numpy; numpy.test()" 2>&1 | tee test.log
      grep -Eq "^(ERROR|FAIL):" test.log && return 1
      popd > /dev/null
      [ -d test-${PYTHON_ABI} ] && rm -fr test-${PYTHON_ABI} || exit_status="1"

finally sees PYTHONPATH pointed to a py3k compliant folder.  It took 2 attempts to get the right one because we are spoilt for choice.

ls  /mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0/build/py3k/

build-3.2  numpy  site.cfg

hmm, a build-3.2 folder finally appears 3 deep from ${S}, and it has a folder numpy at the same level.

archtester numpy # ls  /mnt/gen2/TmpDir/portage/dev-python/numpy-1.6.0/work/numpy-1.6.0/build/py3k/build-3.2/lib.linux-x86_64-3.2/numpy

and we finally find the essential extension built modules for python3.
hence pushd "${S}"/$(find build/py3build-${PYTHON_ABIlib.linux-*-${PYTHON_ABI}/ -name ${PN}) || die.

Now that's a mouthful.  Having pointed PYTHONPATH -> the py3k compliant folder; as a reward;

FAILED (KNOWNFAIL=5, SKIP=3, errors=10, failures=1

Well we only had 1 before.  We are going backwards.

  File "/usr/lib64/python3.2/tempfile.py", line 37, in <module>
    from random import Random as _Random
ImportError: cannot import name Random
not random, Random.

times many times, around 8.  

archtester numpy # qfile Random
dev-python/pycrypto (/usr/lib64/python2.5/site-packages/Crypto/Random)
dev-python/pycrypto (/usr/lib64/python2.5/site-packages/Crypto/SelfTest/Random)
dev-python/pycrypto (/usr/lib64/python2.6/site-packages/Crypto/Random)
dev-python/pycrypto (/usr/lib64/python2.6/site-packages/Crypto/SelfTest/Random)
dev-python/pycrypto (/usr/lib64/python2.7/site-packages/Crypto/Random)
dev-python/pycrypto (/usr/lib64/python2.7/site-packages/Crypto/SelfTest/Random)

archtester numpy # grep RESTRICT_PYTHON_ABIS  ../pycrypto/pycrypto-2.6.ebuild
RESTRICT_PYTHON_ABIS="3.* *-jython 2.7-pypy-*"

Well, it appears that pycrypto is a dep for the tests AND IT DOESN'T support python3

archtester numpy # grep pycrypto numpy-1.6.0.ebuild
yields blank.  No mention of it in the ebuild anywhere.

Might I suggest PYTHON_TESTS_RESTRICTED_ABIS="3*", as somewhere to start?
Oh, but numpy-1.6.1 passes tests.
Is this the ebuild contemplated as put forward for making stable?