There is a pretty serious memory leak in sci-libs/scipy's griddata module. I reported it to SciPy's bug tracker but they claim it is a problem with the Gentoo build. Here is the bug report and the suggested fix: https://github.com/scipy/scipy/issues/2388 http://stackoverflow.com/questions/17438239/memory-leak-in-scipy-griddata-ver-0-12-0 If i understand this correctly, some SciPy code must be recompiled with a bug-corrected version of cython to fix the problem... I can reproduce the memory leak with both stable and masked versions of cython and scipy, i.e. (sci-libs/scipy-0.12.1 & dev-python/cython-0.19.1) and (sci-libs/scipy-0.13.0 & dev-python/cython-0.19.2), where i was sure to build cython first and scipy afterwards to make sure that the latest cython would be used (if needed) to compile scipy... I don't know whether it's useful, but the memory leak happens only for 'linear' and 'cubic' griddata methods, not for 'nearest'. Thanks! PS Transfered here from the forum thread: https://forums.gentoo.org/viewtopic-p-7449808.html
Can you track down the patch to cpython that fixes this issue? Then we can see if the fix is already in cpython-0.19.x or it needs to be applied by the ebuild.
According to the stackoverflow.com link in my first comment above, cython 0.19.2 should work: http://stackoverflow.com/a/19362976 "First updating cython to the most recent version 0.19.2, then recompiling scipy from sources (but the ones from github, not from .tar.gz)" Cython 0.19.2 did not work for me. The stackoverflow comment was made on October 14 when the latest scipy version on sourceforge was 0.13.0. Gentoo's latest masked scipy is still 0.13.0, and as far as i can tell it uses the tarball. As i said in my first comment, it did not work. So my suggestion would be to bump scipy to 0.13.1, the latest version on sourceforge. I suspect that *that* tarball incorporates the github fix, and therefore should resolve the bug.
Please try scipy-0.13.1.
Justin, OK, so i did the following: 1. I unmasked dev-python/cython and dev-python/numpy and updated my Gentoo. Now i am on dev-python/cython-0.19.2, dev-python/numpy-1.8.0, and dev-lang/python-2.7.5-r3. 2. I downloaded the SciPy 0.13.1 tarball from http://sourceforge.net/projects/scipy/files/scipy/ 3. I installed SciPy 0.13.1 in a local directory via "python setup.py install --prefix=$HOME/foo/" 4. I ran the problematic python script mentioned in http://stackoverflow.com/questions/17438239/memory-leak-in-scipy-griddata-ver-0-12-0 as well as my own script that led me to notice the problem in the first place. I included a "print scipy.interpolate.__file__ " statement to make sure that the updated SciPy was being used. Result: Unfortunately, in both cases the memory leak persists.
(In reply to orionbelt2 from comment #4) > Justin, > > OK, so i did the following: > > 1. I unmasked dev-python/cython and dev-python/numpy and updated my Gentoo. > Now i am on dev-python/cython-0.19.2, dev-python/numpy-1.8.0, and > dev-lang/python-2.7.5-r3. > > 2. I downloaded the SciPy 0.13.1 tarball from > http://sourceforge.net/projects/scipy/files/scipy/ I bumped it in the tree so you can use it there. I will look into that tonight.
I emerged scipy-0.13.1 and tried again the tests with this configuration: sci-libs/scipy-0.13.1 dev-python/numpy-1.8.0 dev-lang/python-2.7.5-r3 The memory leak is always present.
same here. I will contact upstream.
Thanks. A reminder that i had already contacted SciPy before bothering Gentoo: https://github.com/scipy/scipy/issues/2388 Just in case you want to follow up on that thread.
+*scipy-0.13.1-r1 (29 Nov 2013) + + 29 Nov 2013; Justin Lecher <jlec@gentoo.org> -scipy-0.13.1.ebuild, + +scipy-0.13.1-r1.ebuild: + Force rebuild of .c files to fix memory leak, #492760 +
I just ran "eix-sync" and 0.13.1-r1 is not yet available. Is this just a question of propagating the change to the mirrors? Any idea how long it takes for this to happen? Just to avoid abusing the repositories with repeated attempts... :)
(In reply to orionbelt2 from comment #10) > I just ran "eix-sync" and 0.13.1-r1 is not yet available. Is this just a > question of propagating the change to the mirrors? Any idea how long it > takes for this to happen? Just to avoid abusing the repositories with > repeated attempts... :) takes around 1 hour. if you want to have it faster, edit the ebuild like this python_compile() { ${EPYTHON} tools/cythonize.py || die distutils-r1_python_compile ${SCIPY_FCONFIG} } and run repoman fix then you can emerge scipy again.
Justin, Have you tried scipy-0.13.1-r1 and confirmed that it works? Unfortunately, it still does not work for me. Again, for reference, i am using: sci-libs/scipy-0.13.1-r1 dev-python/numpy-1.8.0 dev-python/cython-0.19.2 dev-lang/python-2.7.5-r3 Could someone else please give it a try?
(In reply to orionbelt2 from comment #12) > Justin, > > Have you tried scipy-0.13.1-r1 and confirmed that it works? > > Unfortunately, it still does not work for me. Again, for reference, i am > using: > > sci-libs/scipy-0.13.1-r1 > dev-python/numpy-1.8.0 > dev-python/cython-0.19.2 > dev-lang/python-2.7.5-r3 > > Could someone else please give it a try? That should work. Please attach a build.log.
+*scipy-0.13.1-r2 (01 Dec 2013) + + 01 Dec 2013; Justin Lecher <jlec@gentoo.org> -scipy-0.13.1-r1.ebuild, + +scipy-0.13.1-r2.ebuild: + Make realy sure cython is run always, #492760 + r2 is really fixed now.
I can confirm that i see no memory leak with scipy-0.13.1-r2 (there was still with r1). Thanks for taking care of this! I will now have to remember to remove cython, numpy, scipy and tempita from my package.keywords when scipy-0.13.1-r2 becomes stable...
(In reply to orionbelt2 from comment #15) > I can confirm that i see no memory leak with scipy-0.13.1-r2 (there was > still with r1). Thanks for taking care of this! > > I will now have to remember to remove cython, numpy, scipy and tempita from > my package.keywords when scipy-0.13.1-r2 becomes stable... my idea it to wait for 0.13.2 and give that a quick stabilization.
