See http://www.scipy.org for information about SciPy. SciPy is a "Scientific Libraries Collection for Python" The license is "BSD" I'll be attaching it very shortly. Just testing it now. Reproducible: Always Steps to Reproduce:
Sorry, I forgot to mention it's a Python-dev package, it that wasn't obvious from the name.
Created attachment 14552 [details] new ebuild SciPy-0.2.0.200.4161a.ebuild
Created attachment 14553 [details] slightly fixed ebuild
hmm .. very scary version number ;) with distutils, you can remove the src_compile() function and the src_install() function all together as it will inherit from those from the eclass. put the dodoc installed in a variable like: DOCS="README ...." otherwise, the ebuild looks good, could you fix those issues and attach it? thanks
Hey, sorry for the late response, what exactly do I do about the src_install() section? I have this: src_install() { distutils_src_install dodoc FORMAT_GUIDELINES.TXT LICENSE.TXT PKG-INFO THANKS.txt } So if I remove this function all together, do I just need to do DOCS = "FORMAT_GUIDELINES.TXT LICENSE.TXT PKG-INFO THANKS.txt" and that will take care of everything? Is there any information about distutils anywhere? Secondly, is it possible to just create any old USE flags that I want for this package? I'd like to create USE flags for wxWindows (needed only for the plt module) and fftw (needed only for doing FFTs).
Basically the routine should look like this: src_install() { DOCS="FORMAT_GUIDELINES.TXT LICENSE.TXT PKG-INFO THANKS.txt" distutils_src_install } On the question of use flags: you should try to re-use as much as possible existing use flags but if you can't flags can be added if they make sense
Created attachment 16237 [details] newest ebuild newest ebuild. ready for portage I think
It's kind of hard to open the 'newest ebuild' with Konqueror. Please don't upload ebuilds as plain text, not compressed.
i guess you mean, please upload them as plain text. text/plain is the way to go for uploading ebuilds to bugzilla, otherwise it is rather annoying to open them. thanks.
I get this error when I try to emerge scipy: /var/tmp/portage/scipy-cvs-20030611/work/SciPy-0.2.0_alpha_200.4161/Lib/interpolate/fitpack/concur.f: In subroutine `concur': /var/tmp/portage/scipy-cvs-20030611/work/SciPy-0.2.0_alpha_200.4161/Lib/interpolate/fitpack/concur.f:287: warning: unused variable `dist' /var/tmp/portage/scipy-cvs-20030611/temp/ccNKQcwf.s: Assembler messages: /var/tmp/portage/scipy-cvs-20030611/temp/ccNKQcwf.s:467: Error: value of ffffffffffffff78 too large for field of 1 bytes at 0000000000000515 error: failure during compile (exit status = 256) !!! ERROR: app-sci/scipy-cvs-20030611 failed. !!! Function distutils_src_compile, Line 36, Exitcode 1 !!! compilation failed I think a good name for the ebuild would be scipy-cvs-20030611.ebuild because the downloads available are all CVS snapshots, except version 0.1, which I can't get to compile either. The mailing list archive says 0.1.0 has known problems, use CVS. emerge info Portage 2.0.49-r3 (default-x86-1.4, gcc-3.2.2, glibc-2.3.1-r4, 2.4.21) ================================================================= System uname: 2.4.21 i686 AMD Athlon(tm) Processor ccache version 2.2 [enabled] ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-mcpu=athlon -O2 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config /usr/kde/3/share/config /usr/X11R6/lib/X11/xkb /usr/kde/3.1/share/config /usr/share/config" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-O2 -mcpu=i686 -pipe" DISTDIR="/mnt/b2/distfiles" FEATURES="sandbox autoaddcvs ccache noauto" GENTOO_MIRRORS="http://gentoo.mirrors.pair.com/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 oss apm crypt cups foomaticdb gif libg++ mad mikmod ncurses spell truetype xmms xv zlib directfb gtkhtml alsa gdbm berkdb slang readline arts nas bonobo ggi tcltk java guile ruby gpm tcpd pam libwww ssl esd imlib qt kde motif -nls apache2 opengl -svga mysql quicktime X gtk gtk2 -pdflib -gnome python perl oggvorbis sdl -postgres jpeg png xml xml2 avi aalib mpeg encode mmx"
Please upload ebuilds as plain text.
Created attachment 18321 [details] ebuild, little fixes for previous, and plain text ;) The DEPENDs were incorrect. I did s/dev-lang/dev-python and s/Numeric/numeric
We'll need some more testing reports from this...it worked for me and it didn't for Rob Cakebread, but what about anyone else?
I found what the problem was for me. The compiler optimizations are hard-coded in build_flib.py (It tries to determine cpu info itself and has -O3 and funroll-loops). Removing the funroll-loops options solved it for me. I'm making a patch to grab the CFLAGS instead.
After finding a few more issues with this, I'll hold off on the patch until Tues, Sept. 30 when they are scheduled to release version 0.2. This package should also depend on gnuplot, in case any of you had problems with it, btw.
Can this be
Created attachment 29602 [details] Ebuild for SciPy (fetching from CVS) scipy-cvs-0.3.ebuild I have attached an ebuild for SciPy that fetches from its CVS repository; uses eclasses, should have correct deps, etc.
I'd rather like to take a snapshot. Live CVS ebuilds are a maintainance nightmare.
If you read the docs, the authors strongly prefer that packagers build off CVS, not snapshots. Perhaps the docs are out of date :)
Well, we don't have to take their snapshots, we could make or own ones. I just would like to avoid the situation that every user who emerges the package gets something different.
*** Bug 31271 has been marked as a duplicate of this bug. ***
Re: snapshots Your concerns are well noted. But, snapshotting in a coherent way would have to be done from your end, a la xfree-drm. src_install() would still be rather similar; the "PACKAGERS.txt" file has instruction to aid with what you propose.
Created attachment 34889 [details] scipy-0.3.ebuild Since there isn't much enthusiasm for scipy-cvs, here's an ebuild for the scipy-3.0 release (Apr 15 2004). The ebuild features the hearty goodness of distutils inheritance and unit testing.
Created attachment 34896 [details] scipy-0.3.ebuild Fix typo: wxPython -> wxpython in deps
Let's nail down the dependancies that should be in this ebuild. In the other scipy bug, it has this: > >=dev-python/imaging-1.1.4 > >=app-sci/lapack-3.0-r1 > >=dev-python/f2py-2.39.235.1693 and in this one, we have this: < >=dev-libs/atlas-3.2.1 < >=app-sci/lapack-atlas-3.2.1 < >=dev-python/f2py-2.35.229.1492 What should it be? Is imaging required? Should atlas optimized lapack be used? What version of f2py is needed?
Wow, it's been a while; IIRC, all I did was look at the building docs, the setup scripts and made sure the unit tests passed. For the current 0.3.2 release, this doc is likely most relevant: http://www.scipy.org/documentation/buildscipy.txt * It strongly recommends using ATLAS with a complete LAPACK interface, as opposed to LAPACK alone * They seem to have bumped the f2py required rev. to 2.39.235-1644 * imaging hasn't been required for a while, AFAIK, since the package has its own plotting modules: http://www.scipy.org/documentation/plottutorial.html X, gnuplot and wx are required in succession for these plotting modules to function, but the rest of scipy doesn't need them, so I don't have them in the deps .... Hope this is helpful.
Created attachment 42756 [details] scipy-0.3.2.ebuild Updated ebuild 0.3.2
*** Bug 63949 has been marked as a duplicate of this bug. ***
Regarding comments 25 through 28: yes, ATLAS is preferred over vanilla LAPACK and BLAS. However the ATLAS library does not yet build on 64 bit AMD port and so the newly posted scipy-0.3.2.ebuild fails to work for this platform. The duplicate bug report 63949 (ref comments 25 & 28) was for Hardware=amd64 while this report is for Hardware=x86. I'm new to this so it wasn't clear to me how the priorities work--ebuid over hardware or vice versa? Makes more sense to me to have a single ebuild that works on all hardware. In that case is it possible to add an if statement in the ebuild that says "if this is for AMD64 then use just app-sci/lapack-3.0-r1 by itself, otherwise use both dev-libs/atlas-3.2.1 and app-sci/lapack-atlas-3.2.1" ?
Hey Al. Yeah I was thinking all the same things. But I knew that it was possible to put "if"-like statements in the file. We can make amd64 demend on vanilla lapack and the rest use lapack-atlas. I didn't do this yet, I wanted to focus on the x86 dependancies, get those set, now let's add in the amd64 (prefably someone with an amd64) as "if"s.
Hi guys. First I'd like to thank everybody involved for the nice progress on this bug and let you know that we follow it ;). Second, please do not use app-sci/lapack and dev-libs/atlas. These are deprecated and will be removed as soon as fortran.eclass matures and replacement packages (app-sci/blas-{config,reference,atlas} and lapack-*) are marked stable. Use these instead, besides they work on all these arches. You should actually have virtual/bas and virtual/lapack in dependencies and allow user to select the appropriate implementation (or the combination thereof). Yes, you can mix and match with these :). See #30453 for (more than you would want to know) details. George
I've been using the virtual/blas and virtual/lapack ebuilds that George mentioned on my workstation here and have edited the depends for blas and lapack in various ebuilds for software I have used, including scipy. There is one problem with using them at this point, which I have placed in bug 51926. In short, it has to do with scipy assuming that /usr/lib/libblas.so won't have any threading or atlaslike features. I've mentioned this on the scipy list but nobody there seems concerned about it. This is probably something that bears examination before unmasking scipy for people to play with by default. Andy
If I make scipy just depend on >=app-sci/lapack-atlas-3.6.0, which in turn depends on =app-sci/blas-atlas-3.6.0, scipy can't detect lapack during build. lapack_opt_info: atlas_threads_info: scipy_distutils.system_info.atlas_threads_info NOT AVAILABLE atlas_info: scipy_distutils.system_info.atlas_info NOT AVAILABLE scipy_core/scipy_distutils/system_info.py:909: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the scipy_distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) lapack_info: NOT AVAILABLE scipy_core/scipy_distutils/system_info.py:920: UserWarning: Lapack (http://www.netlib.org/lapack/) libraries not found. Directories to search for the libraries can be specified in the scipy_distutils/site.cfg file (section [lapack]) or by setting the LAPACK environment variable. warnings.warn(LapackNotFoundError.__doc__) lapack_src_info: NOT AVAILABLE scipy_core/scipy_distutils/system_info.py:923: UserWarning: Lapack (http://www.netlib.org/lapack/) sources not found. Directories to search for the sources can be specified in the scipy_distutils/site.cfg file (section [lapack_src]) or by setting the LAPACK_SRC environment variable. warnings.warn(LapackSrcNotFoundError.__doc__) NOT AVAILABLE
What are the outstanding issues with this ebuild? Are will still waiting for the new atlas ebuilds to work perfectly? Can someone summarize what work needs to be done concerning this ebuild? Also if someone can clarify once again the dependancies that would be great. If you can tell me the most stable working dependancies, then that's fine. I don't need atlas necessarily, I just need something that works, hopefully by today.
Created attachment 47445 [details] scipy-0.3.2-r1.ebuild a fix of 0.3.2 Modified to accept the new location of atlas and lapack in the portage tree.
sci-libs/atlas is deprecated See comments 66 and 67: http://bugs.gentoo.org/show_bug.cgi?id=30453 Suggest using sci-libs/blas-atlas and in fact this is the only way I was able to install scipy on an amd64. thanks
In reference to comment 33: I built scipy by first installing lapack-atlas, which depended on blas-atlas. As in comment 31, we dont want to use sci-libs/atlas, because it will overwrite files from lapack-atlas. Secondly, I installed from a scipy from a tarball. scipy_core/scipy_distutils/site.cfg needed to be edited to find libblas instead of libf77blas. I ended up installing the scipy_distutils first, and dont know if this is a necessary step. But running system_info.py did find blas threading and atlas features. Hope this is helpful.
regarding comment 33: I just finished building scipy from a tarball. As recommended here, we want to use lapack-atlas and not sci-libs/atlas. sci-libs/atlas will overwrite files from lapack-atlas. Additionally, I had to edit scipy_core/scipy_distutils/site.cfg so atlas_libs reflects blas instead of f77blas. I also installed scipy_distutils before scipy, but dont know if this is a necessary step. When I ran scipy_core/scipy_distutils/system_info.py, it DID find lapack and blas with threading and atlas behavior.
Created attachment 50559 [details] build error scipy-0.3.2-r1 doesn't build for me on ia-32. Good try, though. root # emerge info Portage 2.0.51-r15 (default-linux/x86/2004.2, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.10-hardened-r3 i686) ================================================================= System uname: 2.6.10-hardened-r3 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz Gentoo Base System version 1.4.16 Python: dev-lang/python-2.2.3-r1,dev-lang/python-2.3.4 [2.3.4 (#1, Oct 21 2004, 00:57:41)] ccache version 2.3 [enabled] dev-lang/python: 2.2.3-r1, 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.9.4, 1.8.5-r3, 1.5, 1.6.3, 1.4_p6, 1.7.9-r1 sys-devel/binutils: 2.15.92.0.2-r1 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.6.8.1-r1, 2.6.8.1-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O3 -march=pentium4 -funroll-loops -fprefetch-loop-arrays -pipe -fomit-frame-pointer -ftracer" CHOST="i686-pc-linux-gnu"
re: #39 Ben, you might want to try less agressive CFLAGS. This package failed for me with anything but minimal CFLAGS i.e. CFLAGS="-O2 -march=pentium4 -pipe -fomit-frame-pointer"
All builds fail in exactly the same way, even if I use LDFLAGS="" and minimalist CFLAGS. I imagine that gcc with +hardened in USE might be the problem but I don't really know.
Created attachment 50925 [details, diff] site.cfg patch See my next post for details
Created attachment 50928 [details] proposed change for scipy-0.3.2-r2.ebuild There is a conflict between sci-libs/atlas and lapack-atlas (see previous comments). I believe that atlas is not needed, so this patch removes it from the DEPENDS list. The full lapack implementation is supplied by lapack-atlas and its dependency on blas-atlas. blas-atlas provides libraries called libblas instead of libf77blas. So I supplied a patch for site.cfg, removing f77blas and adding blas to the library search list. However, scipy_core/scipy_distutils/sample_site.cfg has to be copied to scipy_core/scipy_distutils/site.cfg before applying the patch. I dont know how to add this step to the ebuild.
Sorry, the ebuild was not a patch, just a text file.
I just submitted a patch and proposed an ebuild for Numeric-23.7 (-r1?). If USE="atlas", lapack-atlas is required and the patched setup.py will find the libraries. I hope this is useful for the SciPy folks, since Numeric is a requirement. The link is below. http://bugs.gentoo.org/show_bug.cgi?id=81520
The bug report I mentioned in comment 45 appears to have been resolved. atlas is being phased out as a USE flag, they suggest using "lapack" instead. The lapack dependency in the Numeric ebuild reads lapack? ( virtual/lapack ) in order to play nice with lapack-config. I think this new Numeric ebuild will show up in the tree soon. I'll work this week or weekend on the scipy ebuild. But there is an issue that may need to be addressed. lapack-config and blas-config allow you to switch between different implementations (say, MKL or ATLAS). Will this be a problem for Scipy, say if the user switches to a version that was compiled with the Intel FORTAN Compiler?
I've just installed and (quickly) tested the scipy-0.3.2-r2 ebuild; seems to work (I just ran some SciPy functions in the Python interpreter, like the small stuff listed at http://www.scipy.org/documentation/FAQ.html).
I successfully installed SciPy and my first tests went fine. Thanks for this ebuild!
re: comment #43 I don't think we should have a hard dependancy on lapack-atlas like that. Why can't we do as Numeric does (re: comment #46), using virtual/lapack?
Devs, please mark the old ebuilds as obsolete to avoid confusion. Thanks.
Re comment 50, You are correct. As soon as the Numeric ebuild is accepted into the tree, I plan on updating the scipy ebuild to reflect the appropriate BLAS/LAPACK dependencies (sci-libs/atlas-3.6.0 should not be in there). The numeric ebuild seems to be in limbo. I dont think they have worked through all the issues related to this virtual dependency (I dont understand how they deal with atlas).
Created attachment 58619 [details, diff] patch to create site.cfg This patch should create site.cfg so Scipy will know what libraries are available and where to look for them. This will need a minor tweak probably, when I figure out how to deal with the atlas libraries.
Created attachment 58620 [details, diff] bug fix for system_info.py This bugfix will allow SciPy to actually read site.cfg during the installation process. The problem has been corrected in the CVS version of SciPy, but still exists in the available tarballs.
scipy is now in ~x86 in CVS.
Ebuild for new scipy core can be found here: http://bugs.gentoo.org/show_bug.cgi?id=117676
