Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 181653

Summary: Please sunset dev-python/numeric
Product: Gentoo Linux Reporter: Albert Hopkins (RETIRED) <marduk>
Component: New packagesAssignee: Python Gentoo Team <python>
Status: RESOLVED FIXED    
Severity: normal CC: coldwind, danost, daviebdawg, djc, endgame.dos, mmokrejs, rmay31, sci, togge.gentoo
Priority: High Keywords: PMASKED
Version: 2007.0   
Hardware: All   
OS: Linux   
URL: http://numpy.scipy.org/
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 181888, 185561, 185649, 185692, 271019, 289682, 313263    
Bug Blocks:    
Attachments: list of ebuilds that depend on dev-python/numeric

Description Albert Hopkins (RETIRED) gentoo-dev 2007-06-11 14:54:03 UTC
Per the developers' web site, numeric's maintenance has ceased and user should transition to numpy as quickly as possible.  Numpy (and scipy) are already in portage.

The motivation for this bug report is that numeric seems to be one of the things holding back the unmasking of python 2.5.  But since numeric has been dead for almost 2 years I see no justification for it having block an active project whose   release was 7 months ago.
Comment 1 Jakub Moc (RETIRED) gentoo-dev 2007-06-11 15:22:07 UTC
Easier said than done: there's *loads* of stuff depending on dev-python/numeric, absolutely a no go ATM.

Comment 2 Jakub Moc (RETIRED) gentoo-dev 2007-06-11 15:25:31 UTC
Created attachment 121744 [details]
list of ebuilds that depend on dev-python/numeric
Comment 3 Sébastien Fabbro (RETIRED) gentoo-dev 2007-06-11 15:40:13 UTC
Hi

Yes, Jakub is right, many packages still depend on numeric. It would be faster to help on bug #176914 rather than waiting for upstream to switch from numeric to numpy. I will try to see if some of packages from the posted list could benefit a version bump or an ebuild cleaning to get rid of the numeric dependency.

Sébastien
Comment 4 Albert Hopkins (RETIRED) gentoo-dev 2007-06-11 16:03:51 UTC
Can you just instead have numeric depend on python 2.4?
Comment 5 Markus Dittrich (RETIRED) gentoo-dev 2007-06-12 13:19:55 UTC
That's quite a mess! I am amazed how many well maintained
packages (pymol comes to mind) still depend on numeric more
than one year after numeric has been deprecated.

I have the feeling that making numeric + python-2.5 64bit safe
will be very tricky without good knowledge of the code base and
probably break again soon.

Maybe we should rather contact upstream for all packages that
still need numeric asking about the status of transitioning
to numpy and masking packages if there's no reasonable 
timeline.

Markus
Comment 6 Albert Hopkins (RETIRED) gentoo-dev 2007-06-12 13:37:30 UTC
I just checked my Fedora 7 install and it has both python 2.5 and numeric installed by default.  Now I don't know how they do it... maybe they don't have the same package dependencies that we do, but they do have numeric/python 2.5 and it does apparently run on x64.  I'd be more than happy if someone could assist me in to looking into how it's done on Fedora.
Comment 7 Sébastien Fabbro (RETIRED) gentoo-dev 2007-06-12 14:23:09 UTC

> and it does apparently run on x64.  I'd be more than happy if someone could
> assist me in to looking into how it's done on Fedora.

http://cvs.fedora.redhat.com/viewcvs/devel/python-numeric/?root=extras
shows you the spec file and patches. Looking at it, they do less patching than us, 
it might compile, but should not pass the tests.
Could you try the test Tests/test.py from the targz as in bug #176914 on your fedora amd64 box?
Comment 8 Sébastien Fabbro (RETIRED) gentoo-dev 2007-06-12 14:35:53 UTC
(In reply to comment #4)
> Can you just instead have numeric depend on python 2.4?

I'm currently looking at it. I am also looking at the source code to have it worked on amd64. It is not so straight forward.

Meanwhile, here is a list of the packages that are guilty and possible alternative/solution to the numeric dep:

dev-python/f2py-*               needs numpy stable, then sunset it
dev-python/fonttools            sunset it?
dev-python/gnuplot-py           needs patch
dev-python/matplotlib           lives well without numeric
dev-python/pycairo-1.2.*        needs pycairo-1.4.* stable
dev-python/pycdf                needs bump
dev-python/pyclimate-*          does not work without numeric
dev-python/pygame-*             ??
dev-python/pygtk-*              ??
dev-python/pyqwt                needs bump
dev-python/python-biggles       does not work without numeric
dev-python/rpy                  needs patch
dev-python/scientificpython     next stable version with numpy
dev-python/ttfquery             sunset it?
dev-python/visual-*             ??
games-arcade/pydance            ??
games-rpg/galaxymage            ??
gnome-extra/music-applet        ??
sci-biology/biopython           ??
sci-chemistry/pymol             ??
sci-libs/plplot                 ??
sci-libs/pymmlib                needs bump
sci-physics/camfr               needs bump

Some of those (??) I have looked at them, or I simply don't know. pygtk and pygame need extra care since lots of other packages depend on it.
Anyone could fill the ??
Here is a decent page for possible patches:

http://www.scipy.org/Porting_to_NumPy
Comment 9 Albert Hopkins (RETIRED) gentoo-dev 2007-06-12 14:47:51 UTC
Un(In reply to comment #7)
> 
> Could you try the test Tests/test.py from the targz as in bug #176914 on your
> fedora amd64 box?

Unfortunately my Fedora 7 box is not running on x64.  I was just mentioning that it apparently *does* run on x64.

What I could do is d/l the x64 and run it from the Live CD.  Give me some time to do that.

Comment 10 Albert Hopkins (RETIRED) gentoo-dev 2007-06-12 17:46:05 UTC
(In reply to comment #7)

Apparently the Fedora folks neglected to check to see if their Live CD actually fits on a CD.  The x64 ISO is 780.... about 80MB too big to burn on a standard CDR.  I ended up burning it to a DVDR. They also apparently do not run or ignore the tests. From the Fedora 7 Live x64 CD:

[fedora@localhost ~]$ cat /etc/redhat-release ; uname -a ; python -V ; rpm -q python-numeric
Fedora release 7 (Moonshine)
Linux localhost.localdomain 2.6.21-1.3194.fc7 #1 SMP Wed May 23 22:47:07 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
Python 2.5
python-numeric-24.2-4.fc7
[fedora@localhost disk]$ cd /media/disk/Test/
[fedora@localhost Test]$ ls
llstest.py  test.py  testwin.bat
[fedora@localhost Test]$ python test.py 
......E.E...........E.........E..............
======================================================================
ERROR: test concatenate
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 433, in testConcatenate
    assert_eq(Numeric.concatenate((self.a[:3], self.a[3:])), [0,1,2,3,4,5])
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(3,)=array([0, 1, 2])
b(6,)=[0, 1, 2, 3, 4, 5]

======================================================================
ERROR: Test the diagonal function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 584, in testDiagonal
    assert_eq(Numeric.diagonal(c,1), [[2,7,4], [2,7,4]])
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(4, 2)=array([[5, 1],
       [6, 2],
       [7, 3],
       [8, 4]])
b(2, 3)=[[2, 7, 4], [2, 7, 4]]

======================================================================
ERROR: Test of average function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 297, in testAverage
    c.shape=(3,2)
ValueError: total size of new array must be unchanged

======================================================================
ERROR: Test slicing, like x[1:3]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 141, in testSlice
    assert_eq(a[0:], a)
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(0,)=zeros((0,), 'l')
b(4,)=array([1, 2, 3, 4])

----------------------------------------------------------------------
Ran 45 tests in 0.146s

FAILED (errors=4)
[fedora@localhost Test]$ cd
[fedora@localhost ~]$ cd /media/disk/Test/
[fedora@localhost Test]$ python test.py 
......E.E...........E.........E..............
======================================================================
ERROR: test concatenate
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 433, in testConcatenate
    assert_eq(Numeric.concatenate((self.a[:3], self.a[3:])), [0,1,2,3,4,5])
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(3,)=array([0, 1, 2])
b(6,)=[0, 1, 2, 3, 4, 5]

======================================================================
ERROR: Test the diagonal function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 584, in testDiagonal
    assert_eq(Numeric.diagonal(c,1), [[2,7,4], [2,7,4]])
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(4, 2)=array([[5, 1],
       [6, 2],
       [7, 3],
       [8, 4]])
b(2, 3)=[[2, 7, 4], [2, 7, 4]]

======================================================================
ERROR: Test of average function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 297, in testAverage
    c.shape=(3,2)
ValueError: total size of new array must be unchanged

======================================================================
ERROR: Test slicing, like x[1:3]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 141, in testSlice
    assert_eq(a[0:], a)
  File "test.py", line 28, in assert_eq
    assert eq(a,b)
  File "test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(0,)=zeros((0,), 'l')
b(4,)=array([1, 2, 3, 4])

----------------------------------------------------------------------
Ran 45 tests in 0.119s

FAILED (errors=4)





Comment 11 Markus Dittrich (RETIRED) gentoo-dev 2007-06-13 13:15:04 UTC
I am pretty sure that quite a few upstream devs are not
aware of the fact that their packages may break on 64bit arches 
with python-2.5 due to numeric. Maybe this will convince
them to finally move to numpy ;)
Comment 12 Markus Dittrich (RETIRED) gentoo-dev 2007-06-14 13:06:12 UTC
I'll ping the pymol developer and inquire about numpy.

Comment 13 Albert Hopkins (RETIRED) gentoo-dev 2007-06-14 13:27:20 UTC
You may already be aware, this "Porting to Numpy" page specifically mentions pygame and pygtk... even including patches.  It's done by the SciPy people... why doesn't upstream do this themselves? The page, however, doesn't seem to have been modified since February:

http://www.scipy.org/Porting_to_NumPy

pymol, at least recent versions, seem to require numpy.

biopython still requires Numeric as of March 2007.  Their web site implies they will be porting to numpy.

Comment 14 Markus Dittrich (RETIRED) gentoo-dev 2007-06-14 14:58:24 UTC
I just talked to the pymol maintainer and he pointed out to me
that numeric is not strictly required for pymol. As a matter of
fact, the default in setup.py is to not use numeric (via commenting
out _PYMOL_NUMPY); since our ebuilds don't seem to change this
behavior (je_fro might want to correct me here) it looks to me as
if numeric is not actually used by pymol at the moment and can
be removed from DEPEND.

Markus
Comment 15 Jeffrey Gardner (RETIRED) gentoo-dev 2007-06-14 17:38:23 UTC
Yup, you're absolutely right.
I'm removing the numeric dependency in pymol-0.99_rc10.
Comment 16 Albert Hopkins (RETIRED) gentoo-dev 2007-06-14 17:40:11 UTC
pycdf has been cleaned up per bug #181778.
Comment 17 Jeffrey Gardner (RETIRED) gentoo-dev 2007-07-17 19:25:50 UTC
From the biopython site: 
"Note that BioPython has not (yet) switched to the 'new' numpy library. You need the 'old' Numeric library, version 24.2 is recommended"

Hopefully the transition is in the works...
Comment 18 Santiago M. Mola (RETIRED) gentoo-dev 2007-07-17 20:28:34 UTC
Status update:

dev-python/f2py-*               stable numpy on ia64 and x86, then sunset it or update it from numpy
dev-python/fonttools            sunset it? alternative packages?
dev-python/gnuplot-py           needs patch
dev-python/matplotlib           lives well without numeric
dev-python/pyclimate-*          does not work without numeric, dead upstream, no revdeps
dev-python/pygame-*             needs patch: http://www.scipy.org/Porting_to_NumPy / pygame-1.8 will support numpy
dev-python/pygtk-*              needs patch: http://www.scipy.org/Porting_to_NumPy http://bugzilla.gnome.org/show_bug.cgi?id=397544
dev-python/pyqwt                needs bump
dev-python/python-biggles       ported in CVS
dev-python/rpy                  needs patch: http://www.scipy.org/Porting_to_NumPy
dev-python/scientificpython     next stable version with numpy
dev-python/ttfquery             sunset it? alternative packages?
dev-python/visual-*             numeric is optional, alternatively it can use numarray
games-rpg/galaxymage            reported upstream https://gna.org/bugs/index.php?9528
gnome-extra/music-applet        low use of numeric, contacted upstream
sci-biology/biopython           seems that port in its way upstream, not ready
sci-libs/plplot                 reported upstream http://sourceforge.net/tracker/index.php?func=detail&aid=1755542&group_id=2915&atid=202915
sci-libs/pymmlib                needs bump
sci-physics/camfr               ported version is coming soon!
Comment 19 Santiago M. Mola (RETIRED) gentoo-dev 2007-07-18 17:27:12 UTC
dev-python/pyopengl is on the list too, but it's missing dev-python/numeric dependency.
Comment 20 Ali Polatel (RETIRED) gentoo-dev 2007-07-19 11:09:11 UTC
(In reply to comment #18)
> Status update:
> 
> dev-python/fonttools            sunset it? alternative packages?
The new version in the tree uses numpy.
> dev-python/gnuplot-py           needs patch
I've backported upstream's patches for this one. I'm waiting numpy to have enough keywords to do a revbump.
> dev-python/pygtk-*              needs patch:
There's a working patch and an ebuild on bug 176914. It'll need testing before we can commit it to the tree.
> dev-python/python-biggles       ported in CVS
I've contacted upstream to learn when they're planning to release a new version. If I don't hear from them soon, I'll backport the changes in CVS.
> dev-python/ttfquery             sunset it? alternative packages?
The new version in the tree uses numpy.
Comment 21 Santiago M. Mola (RETIRED) gentoo-dev 2007-08-21 20:44:02 UTC
pbienst just commited a new camfr bump. That makes one less package to port ;-)
Comment 22 Peter Bienstman (RETIRED) gentoo-dev 2007-08-21 20:44:21 UTC
camfr fixed.
Comment 23 Jakub Moc (RETIRED) gentoo-dev 2008-02-12 08:09:37 UTC
*** Bug 209767 has been marked as a duplicate of this bug. ***
Comment 24 Martin Mokrejš 2008-06-11 06:50:04 UTC
Regarding biopython, see http://portal.open-bio.org/pipermail/biopython-dev/2007-March/002722.html for some details.

But, I see numpy referenced in installation docs ... http://www.biopython.org/DIST/docs/install/Installation.html ... so it seems biopython ebuild could get changed.

BTW, numpy-1.1.0 is out.
Comment 25 Martin Mokrejš 2008-06-11 08:57:39 UTC
For the biopython issue, watch http://bugzilla.open-bio.org/show_bug.cgi?id=2516
and http://bugzilla.open-bio.org/show_bug.cgi?id=2251. The install docs I have mentioned in previous comment were not up-to-date and will get fixed in a while.
Comment 26 Dustin Polke 2009-07-16 11:47:36 UTC
For the record, numeric fails with python-2.6 which is scheduled to go stable soon.

>>> Source compiled.
......E.............E......E..E..............
======================================================================
ERROR: test concatenate
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 433, in testConcatenate
    assert_eq(Numeric.concatenate((self.a[:3], self.a[3:])), [0,1,2,3,4,5])
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 28, in assert_eq
    assert eq(a,b)
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(3,)=array([0, 1, 2])
b(6,)=[0, 1, 2, 3, 4, 5]

======================================================================
ERROR: Test of average function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 297, in testAverage
    c.shape=(3,2)
ValueError: total size of new array must be unchanged

======================================================================
ERROR: Test various object arrays
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 334, in testObject
    a = Numeric.array([f,w,b],'O')
ValueError: invalid input sequence

======================================================================
ERROR: Test slicing, like x[1:3]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 141, in testSlice
    assert_eq(a[0:], a)
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 28, in assert_eq
    assert eq(a,b)
  File "/var/tmp/portage/dev-python/numeric-24.2-r6/work/Numeric-24.2/Test/test.py", line 24, in eq
    (aa.shape, a, ab.shape, b))
ValueError: sequences have different shapes:
a(0,)=zeros((0,), 'l')
b(4,)=array([1, 2, 3, 4])

----------------------------------------------------------------------
Ran 45 tests in 0.153s

FAILED (errors=4)
Comment 27 Dirkjan Ochtman (RETIRED) gentoo-dev 2009-10-17 09:00:45 UTC
It appears nothing depends on numeric anymore. The last to maintain it was Fabian Groffen, though I'm not sure why. Fabian, is there any reason to keep numeric around?
Comment 28 Fabian Groffen gentoo-dev 2009-10-17 09:34:03 UTC
I can't imagine me maintaining any piece of python software.  I did add patches/changes from the Prefix overlay, though.  I think you can just proceed.
Comment 29 Justin Lecher (RETIRED) gentoo-dev 2009-10-17 10:32:19 UTC
(In reply to comment #27)
> It appears nothing depends on numeric anymore. 

That's wrong. There are still a couple of ebuild which contain it as *DEPEND and are not masked.
Comment 30 Dirkjan Ochtman (RETIRED) gentoo-dev 2009-10-17 13:45:40 UTC
Okay, I used equery to check. What did you use?
Comment 31 Justin Lecher (RETIRED) gentoo-dev 2009-10-17 22:24:51 UTC
qgrep -v dev-python/numeric|sed -e "s|:.*$||g"

which gave following packages:
dev-python/pyclimate-1.2.1
dev-python/pyclimate-1.2.1-r1

    version 1.2.2 has no numeric dep

dev-python/pygtk-2.12.1-r2
dev-python/pygtk-2.14.1

    has versions w/o numeric dep. Interestingly 2 of 6 have.

dev-python/visual-3.2.9-r2

    version 5* which is in ~arch doesn't depend on numeric    

games-rpg/galaxymage-0.3.0

    is the only version

sci-chemistry/mead-2.2.7

    is latest version and has optional dep of numeric for python support.

sci-libs/plplot-5.5.2

sci-libs/pymmlib-0.9.7
Comment 32 Justin Lecher (RETIRED) gentoo-dev 2009-10-17 22:29:11 UTC
sci-libs/plplot-5.5.2
    All newer versions are clean

sci-libs/pymmlib-0.9.7
    Version 1* is clean
Comment 33 Justin Lecher (RETIRED) gentoo-dev 2009-10-17 22:29:44 UTC
Sorry my fault.
Comment 34 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-01-01 22:04:34 UTC
Update:

dev-python/pygtk has numeric-less 2.16 waiting in unstable
dev-python/visual is waiting for stabilization of dev-libs/boost-1.41.0
games-strategy/castle-combat seems to have a rather dead upstream
sci-chemistry/mead homepage is 404
Comment 35 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-04-18 21:58:44 UTC
If there are no objections, then I will mask dev-python/numeric in next week.
Comment 36 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-04-19 06:53:41 UTC
So you're also sunsetting castle-combat and mead?
Comment 37 Justin Lecher (RETIRED) gentoo-dev 2010-04-19 06:55:41 UTC
(In reply to comment #36)
> So you're also sunsetting castle-combat and mead?
> 

I will fix mead in ten minutes.
Comment 38 Justin Lecher (RETIRED) gentoo-dev 2010-04-19 07:02:40 UTC
mead is fixed.
Comment 39 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-04-19 16:38:21 UTC
(In reply to comment #36)

games-strategy/castle-combat is already masked.
Comment 40 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-04-25 20:33:38 UTC
dev-python/numeric has been masked for deletion.