Bugzilla Bug 186913

Emerging numpy without an installed fortran compiler fails even if the numpy
upstream package does not have such a hard dependency.

Reproducible: Always

Steps to Reproduce:
1. USE="-fortran" emerge gcc
2. emerge numpy

Actual Results:  
the emerge fails with:

 * None of the needed Fortran Compilers (g77 gfortran) is installed.
 * To install one of these, choose one of the following steps:
 * [1] USE="fortran" emerge =sys-devel/gcc-3*
 * [2] USE="fortran" emerge =sys-devel/gcc-4*
 * 
 * ERROR: dev-python/numpy-1.0.2 failed.
 * Call stack:
 *   ebuild.sh, line 1664:   Called dyn_setup
 *   ebuild.sh, line 719:   Called qa_call 'pkg_setup'
 *   ebuild.sh, line 44:   Called pkg_setup
 *   ebuild.sh, line 1328:   Called fortran_pkg_setup
 *   fortran.eclass, line 181:   Called need_fortran 'g77' 'gfortran'
 *   fortran.eclass, line 109:   Called die
 * 
 * Install a Fortran Compiler !
 * If you need support, post the topmost build error, and the call stack if
relevant.
 * A complete build log is located at
'/var/tmp/portage/dev-python/numpy-1.0.2/temp/build.log'.

P.S.: This 1.0.2 test is because I didn't want to uninstall my overlayed &
hacked 1.0.3, which is independent of fortran, and wanted to report the actual
error message, which is the same with 1.0.3 ebuild.

Expected Results:  
emerge numpy should succeed, as the numpy package does not depend on the actual
presence of a fortran compiler.

As far as I know this is only used for the internal & optional f2py thing.

Maybe check the fortran USE flag and do as was done for numpy-0.9.8.ebuild or
something like that.

I tried to conditionalize "inherits fortran" eclass with a fortran USE flag for
numpy ebuild but that's not allowed by portage.

I tried to change the way to use the fortran eclass in numpy ebuild but it
fails right at the "inherits" because fortran_pkg_setup() is called by
ebuild.sh, and that calls need_fortran()...

Then I just ripped all fortran-related things in numpy.ebuild and it emerged
properly. But this is not a clean solution, I think...

Can't you simply override fortran_pkg_setup() by adding your own pkg_setup() to
your ebuild, so we can keep the USE=fortran option?

Something as horrible as

pkg_setup() {
    true
}

should do the trick. :)

A note: I use numpy with lapack, which in turn requires fortran. If changes are
to be made, maybe this needs to be considered also.

(In reply to comment #2)
> A note: I use numpy with lapack, which in turn requires fortran. If changes are
> to be made, maybe this needs to be considered also.

I tried Jeroen's idea, and it worked, so could you try if the attached patch
works for you with:
  USE=fortran emerge numpy
(no need for the USE=fortran if it is in your global USE flags)
So maybe the patch can be considered for inclusion...

Created an attachment (id=126326) [details]
Add "fortran" USE flag

This is the solution proposed by Jeroen, it works for me with:
  USE=-fortran emerge numpy

A quick test with the mentioned patch in the comment #4:

Calculating dependencies... done!
[ebuild   R   ] dev-python/numpy-1.0.3  USE="fortran%* lapack -debug" 0 kB [1]

Total: 1 package (1 reinstall), Size of downloads: 0 kB
Portage overlays:
 [1] /usr/local/portage

Would you like to merge these packages? [Yes/No]

>>> Emerging (1 of 1) dev-python/numpy-1.0.3 to /
 * numpy-1.0.3.tar.gz RMD160 ;-) ...                                           
                                                       [ ok ]
 * numpy-1.0.3.tar.gz SHA1 ;-) ...                                             
                                                       [ ok ]
 * numpy-1.0.3.tar.gz SHA256 ;-) ...                                           
                                                       [ ok ]
 * numpy-1.0.3.tar.gz size ;-) ...                                             
                                                       [ ok ]
>>> Unpacking source...
>>> Unpacking numpy-1.0.3.tar.gz to /var/tmp/portage/dev-python/numpy-1.0.3/work
 * Applying numpy-1.0.1-f2py.patch ...                                         
                                                       [ ok ]
>>> Source unpacked.
>>> Compiling source in /var/tmp/portage/dev-python/numpy-1.0.3/work/numpy-1.0.3 ...
 * Invalid Fortran compiler \'\'

!!! ERROR: dev-python/numpy-1.0.3 failed.
Call stack:
  ebuild.sh, line 1621:   Called dyn_compile
  ebuild.sh, line 973:   Called qa_call 'src_compile'
  ebuild.sh, line 44:   Called src_compile
  numpy-1.0.3.ebuild, line 128:   Called numpy_configure
  numpy-1.0.3.ebuild, line 101:   Called die

Created an attachment (id=126329) [details]
Further cleanups

This is an additional cleanup, while I'm in there,
could someone have a look at it...
Works for me 
USE=-fortran emerge numpy -> OK
USE=fortran emerge numpy -> Fails, as expected with no fortran compiler
installed
someone, please test those 2 cases on a host with a fortran compiler

(In reply to comment #5)
> A quick test with the mentioned patch in the comment #4:
[...]
> !!! ERROR: dev-python/numpy-1.0.3 failed.

OK I think see now what's going on here, try the new patch and ignore the
cleanup ebuild, it will probably need fixing too.

Created an attachment (id=126331) [details]
Fixed patch for "USE=fortran" failure

The patch in the comment #8 applies cleanly.

Although I do not have a say in this matter as an end-user, I am still a little
sceptical why all this is needed, given that Numpy and Scipy are like Siamese
twins - and the latter requires fortran.

And should not the optional fortran USE flag then exclude the whole f2py
interface generator and all files related to it in order to avoid confusion
among the users; and please then deal with the blocker to the dev-python/f2py?

(In reply to comment #9)
> The patch in the comment #8 applies cleanly.

Does that mean it works for you ?
Assuming a yes, and with USE=fortran, could you test USE=-fortran too ?
Just to know it work in all 4 cases, this won't take a lot of your time,
I think...

> Although I do not have a say in this matter as an end-user,

I'm too

> I am still a little
> sceptical why all this is needed, given that Numpy and Scipy
> are like Siamese twins - and the latter requires fortran.

But all uses of numpy don't need the f2py fortran part, that's my point

> And should not the optional fortran USE flag then exclude the whole f2py
> interface generator and all files related to it in order to avoid confusion
> among the users;

Oh certainly, but that would be for the upstream package, as far as I know, it
always install f2py, even if no fortran compiler has been detected at install
time,. Maybe in case one is installed later, I don't know...

> and please then deal with the blocker to the dev-python/f2py?

Hey I'm not gentoo dev/maintainer, just that it annoyed me to be forced a
fortran compiler upon me, that I wouldn't even use...

And I just provide my hack^Wfix for that tiny issue, if a maintainer would like
to do the proper fix, fine by me. I'll test proposed alternatives.

I could even keep it my local portage overlay too, if that is not accepted.

Just willing to share something back...

Vincent,

both USE="fortran" and USE="-fortran" compile fine with your patch.

Note that it is naturally impossible to build a system with USE="-fortran"
emerge gcc && USE="lapack" emerge numpy.

I assume that USE=-fortran should exclude everything related to fortran: this
means that every single function in the whole package is proved to work without
a fotran compiler; otherwise expect more bug reports in the future. You could
try to exclude the f2py (do we know that this is the only one?) in the sandbox
environment but this gets probably too ugly.

Now, let us leave this to the developers.

(In reply to comment #11)
> Vincent,
> 
> both USE="fortran" and USE="-fortran" compile fine with your patch.

Good, thanks for testing

> Note that it is naturally impossible to build a system with USE="-fortran"
> emerge gcc && USE="lapack" emerge numpy.

That does not look good, maybe I should conditionalize on lapack USE flag
instead of adding & using the fortran one then...

> I assume that USE=-fortran should exclude everything related to fortran: this
> means that every single function in the whole package is proved to work without
> a fortran compiler; otherwise expect more bug reports in the future. You could
> try to exclude the f2py (do we know that this is the only one?) in the sandbox
> environment but this gets probably too ugly.

But that is not a gentoo bug, since the upstream package has the same problem,
it will happily install when no fortran compiler is there.

Maybe we should bug them about that, but it seems unrelated to the gentoo issue
at hand.

I added a warning for that matter in the fixed & cleaned-up ebuild. Maybe that
will do the trick until upstream numpy configuration is more finegrained...

> Now, let us leave this to the developers.

OK, I think I'll need input to go further on that subject...

Created an attachment (id=126345) [details]
Fixed the cleanup ebuild & additional modifs

Some trivialities, maybe too much, but why not ?

- Extracted linker_configure(), lapack_configure() & fortran_configure()
  from numpy_configure() and conditionalize their call on respective USE flags
- Avoid fortran.eclass::fortran_pkg_setup() replace by new require_fortran()
- Enhance "Invalid Fortran compiler" dying message
- Warn user about custom LDFLAG ignoring
- Extract patch_atlas() from src_unpack()
- Harden numpy's autotests checking (I ran them, no failures with USE=-fortran)
- Warn about f2py being unusable with USE=-fortran

Hi,

I'm in the process of updating numpy-1.0.3.1. Do we really need an extra
USE=fortran? fortran is needed for lapack and for f2py. What is described in
comment #0 happen in all packages which "inherit fortran", and is the expected
unfortunate behaviour.

dev-python/f2py is to be removed and replaced by numpy. If we add a
USE=fortran, any deps on f2py and probably some others on numpy (such as scipy)
will need to set an undesirable "built_with_use dev-python/numpy fortran ||
die" thingy.

You need to tell me what is the argument in favor of having a numpy without
f2py and not lapack enabled.

As for cleanups in comment #13, I'll take a look.

Sébastien

> You need to tell me what is the argument in favor of having a numpy without
> f2py and not lapack enabled.

I found out that numpy does not actually need fortran to build. Many packages
dependent on numpy don't use f2py, so I simply warn the user if no fortran
compiler is found. For scipy, lapack libs are needed anyway, so it will pull
fortran.
The newly commited numpy-1.0.3.1 implements the changes.
Thanks all.

(In reply to comment #15)
> I found out that numpy does not actually need fortran to build.

Yes, as I said in comment #12

> The newly commited numpy-1.0.3.1 implements the changes.

I tested emerging it with -fortran, it worked.
The warning is OK too, thanks.

Let me know if you look at the "cleanups"

> I tested emerging it with -fortran, it worked.

You don't actually need a fortran use flag.

> Let me know if you look at the "cleanups"

I did, and I added some of mine. You can look at the numpy-1.0.3.1.ebuild I
commited.
Thanks for your work.