Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 648988 - dev-python/numpy-1.13.3 - src_compile(): don't know how to compile Fortran code on platform 'posix'
Summary: dev-python/numpy-1.13.3 - src_compile(): don't know how to compile Fortran co...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo Science Related Packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-02-27 20:27 UTC by coKe
Modified: 2019-05-10 07:51 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,9.40 KB, text/plain)
2018-02-27 20:27 UTC, coKe
Details
emerge --info (emerge --info.txt,5.97 KB, text/plain)
2018-02-27 20:27 UTC, coKe
Details
emerge --info for numpy-1.14.5 (emerge-info,6.42 KB, text/plain)
2018-08-31 02:20 UTC, Kenton Groombridge
Details
build.log for numpy-1.14.5 (build.log,11.28 KB, text/x-log)
2018-08-31 02:21 UTC, Kenton Groombridge
Details

Note You need to log in before you can comment on or make changes to this bug.
Description coKe 2018-02-27 20:27:26 UTC
Created attachment 521336 [details]
build.log

Can't update numpy to 1.13.3 because compiling error which doesn't state anything apart from failure. Doing # MAKEOPTS="-j1" emerge -av1 numpy doesn't solve it as stated in the 623790 ticket.
Comment 1 coKe 2018-02-27 20:27:48 UTC
Created attachment 521338 [details]
emerge --info
Comment 2 François Bissey 2018-03-01 06:12:40 UTC
Question: Did you recently upgrade to gcc-7? When you moved to gcc-7 did you rebuild packages depending on fortran?
Comment 3 coKe 2018-03-18 17:38:06 UTC
Yes Im making a system upgrade and changed gcc from 6.4.0 to 7.3.0 but cant really tell if it rebuilt the fortran packages... How do I make a specific update of those packages?
Comment 4 coKe 2018-03-18 19:26:42 UTC
(In reply to François Bissey from comment #2)
> Question: Did you recently upgrade to gcc-7? When you moved to gcc-7 did you
> rebuild packages depending on fortran?

I recently moved to gcc-7 but I didn't rebuild packages depending on fortran (at least manually).. how do I rebuild them?
Comment 5 François Bissey 2018-03-18 19:44:31 UTC
try:
revdep-rebuild --library 'libgfortran.so.3' -- --exclude gcc
Comment 6 coKe 2018-03-18 22:44:20 UTC
(In reply to François Bissey from comment #5)
> try:
> revdep-rebuild --library 'libgfortran.so.3' -- --exclude gcc

It says system is consistent.
Comment 7 François Bissey 2018-03-18 22:50:24 UTC
(In reply to coKe from comment #6)
> (In reply to François Bissey from comment #5)
> > try:
> > revdep-rebuild --library 'libgfortran.so.3' -- --exclude gcc
> 
> It says system is consistent.

Hum... you require numpy[-lapack] so unless some dependency is not actually conditional anymore it should be fine and blas/lapack being wrong may be a false lead.
For safety you should rebuild whatever implementation of blas and then lapack if any before moving to the next possible culprit, setuptools being too old.
Comment 8 coKe 2018-03-18 23:35:44 UTC
(In reply to François Bissey from comment #7)
> (In reply to coKe from comment #6)
> > (In reply to François Bissey from comment #5)
> > > try:
> > > revdep-rebuild --library 'libgfortran.so.3' -- --exclude gcc
> > 
> > It says system is consistent.
> 
> Hum... you require numpy[-lapack] so unless some dependency is not actually
> conditional anymore it should be fine and blas/lapack being wrong may be a
> false lead.
> For safety you should rebuild whatever implementation of blas and then
> lapack if any before moving to the next possible culprit, setuptools being
> too old.

set up lapack useflag for numpy, blas and lapack and setuptools emerged but numpy buid still crashes.
Comment 9 Kenton Groombridge 2018-08-31 02:20:51 UTC
Created attachment 545570 [details]
emerge --info for numpy-1.14.5
Comment 10 Kenton Groombridge 2018-08-31 02:21:22 UTC
Created attachment 545572 [details]
build.log for numpy-1.14.5
Comment 11 Kenton Groombridge 2018-08-31 02:21:59 UTC
I am sure I have the exact same issue, but I am not upgrading, I am performing a fresh install (although a ~amd64). I have three other systems (all ~amd64) that are fine, but been working with this one for four days now and can't get past this error. It is a laptop nearly identical to another laptop I have.

numpy-1.10.4 installs fine, but later versions fail with the same issue.

Trying to install kde-plasma/plasma-meta
Comment 12 François Bissey 2018-08-31 02:37:04 UTC
Hum... This is not how it should look right after `blas_info:`. What blas do you have? And is it properly setup by eselect?
Comment 13 Kenton Groombridge 2018-08-31 11:12:32 UTC
(In reply to François Bissey from comment #12)
> Hum... This is not how it should look right after `blas_info:`. What blas do
> you have? And is it properly setup by eselect?

No blas or or lapack installed. The other system I have where numpy builds ok also do not have blas or lapack installed.

In comparing the build output with known good system, the breakage appears to be in the prep trying to configure the intel icc compiler when none is present on the system. See messages:

customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc

Those don't appear on the known good system. Just a guess.
Comment 14 François Bissey 2018-08-31 12:24:51 UTC
Could update your build log with the output of `emerge -1v numpy`. I must say I haven't done a build of numpy without blas/lapack for a while. But your mention of intel compilers tells me I need the extra info as well.
Comment 15 Kenton Groombridge 2018-08-31 13:16:08 UTC
(In reply to François Bissey from comment #14)
> Could update your build log with the output of `emerge -1v numpy`. I must
> say I haven't done a build of numpy without blas/lapack for a while. But
> your mention of intel compilers tells me I need the extra info as well.

I performed the build with the '-1v' and '-1 --verbose', but the build.log is the same as without the -v or --verbose (verified with diff).

I should add, I installed blas/lapack in an earlier attempt to fix the issue, but the results were the same.

Here is the point in the build where breakage appears to occur (trying to configure the Intel compiler):

get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']'
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
Could not locate executable x86_64-pc-linux-gnu-gcc -O2 -pipe -march=native -mtune=native -shared
Found executable /usr/bin/x86_64-pc-linux-gnu-ar
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc


Here is what it looks like on one of my other systems where it builds OK:

get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']'
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
Could not locate executable x86_64-pc-linux-gnu-gcc -O2 -pipe -march=native -mtune=native -shared
Found executable /usr/bin/x86_64-pc-linux-gnu-ar
customize Gnu95FCompiler
customize Gnu95FCompiler using config
Comment 16 François Bissey 2018-09-02 23:59:54 UTC
Yes, got it. So far I haven't been able to reproduce the problem. It is unfortunate that python install procedure doesn't leave any traces behind when something goes horribly wrong. 

The only thing I can think of is that it is a manifestation of #335908. It was fixed but only if you were using the lapack useflag. Which i think is a bit of an oversight since in the absence of the flag you will compile a minimal, internal, blas/lapack.

I think having `NUMPY_FCONFIG` properly defined the same way as when lapack is in use may help.
Can you try emerging with
NUMPY_FCONFIG="config_fc --noopt --noarch --fcompiler=gnu95" emerge -1v numpy
Comment 17 François Bissey 2018-09-03 00:53:58 UTC
Scrap my last comment, I don't think it is useful. The question is why does numpy's setup.py think your gfortran is broken. Which is hard to evaluate here.
Comment 18 François Bissey 2018-09-03 01:14:44 UTC
OK let's see if we can get more info from python
PYTHONVERBOSE="2" emerge -1v numpy
please.
Comment 19 François Bissey 2018-09-03 01:21:05 UTC
PYTHONVERBOSE=1 rather than 2 should be plenty for now.
Comment 20 Kenton Groombridge 2018-09-05 00:18:23 UTC
I am not sure what caused it, but I masked >dev-python/numpy-1.10.4 so that I could continue to install. Once plasma-meta was installed (and rebooting), I unmasked numpy to try to do more troubleshooting. It installed with no issue after that. I am dumbfounded.
Comment 21 François Bissey 2018-09-05 00:22:39 UTC
(In reply to Kenton Groombridge from comment #20)
> I am not sure what caused it, but I masked >dev-python/numpy-1.10.4 so that
> I could continue to install. Once plasma-meta was installed (and rebooting),
> I unmasked numpy to try to do more troubleshooting. It installed with no
> issue after that. I am dumbfounded.

Question:
When you were getting the problem, did you always use the same shell (as in a continuous session), or did you have it from several sessions?
Comment 22 Maxime Lombard 2019-01-06 21:16:57 UTC
(In reply to François Bissey from comment #21)
> (In reply to Kenton Groombridge from comment #20)
> > I am not sure what caused it, but I masked >dev-python/numpy-1.10.4 so that
> > I could continue to install. Once plasma-meta was installed (and rebooting),
> > I unmasked numpy to try to do more troubleshooting. It installed with no
> > issue after that. I am dumbfounded.
> 
> Question:
> When you were getting the problem, did you always use the same shell (as in
> a continuous session), or did you have it from several sessions?

Same problem for me except that i install Gentoo ~amd64 on my laptop and i use a SSH session with PuTTY on Win10 from my desktop.

If i install Gentoo with Xubuntu LiveCD, this issue does not appears. 
mask >dev-python/numpy-1.10.4 fix the issue too for me :-)
Comment 23 Michael Haubenwallner (RETIRED) gentoo-dev 2019-01-14 21:41:29 UTC
Same here while setting up a fresh stable amd64 using system-rescue-cd (which is Gentoo based), performing the emerge inside the 'chroot /mnt/gentoo /bin/bash'.

For me, it does help to set SHELL=/bin/bash before emerging numpy:
$ SHELL=/bin/bash emerge -1 dev-python/numpy

Analysis for my situation:
* sysresccd provides /bin/zsh as default shell, hence
* SHELL=/bin/zsh environment variable is set.
* /mnt/gentoo does not have app-shells/zsh installed.
* 'chroot /mnt/gentoo /bin/bash' does preserve SHELL environment variable.

How analysed:
$ strace -f -e trace=execve emerge -1 dev-python/numpy
shows it does want (but fails) to execute /bin/zsh.
Comment 24 Yury Gavrilovich 2019-01-20 16:27:23 UTC
(In reply to Michael Haubenwallner from comment #23)
> Same here while setting up a fresh stable amd64 using system-rescue-cd
> (which is Gentoo based), performing the emerge inside the 'chroot
> /mnt/gentoo /bin/bash'.
> 
> For me, it does help to set SHELL=/bin/bash before emerging numpy:
> $ SHELL=/bin/bash emerge -1 dev-python/numpy
> 
> Analysis for my situation:
> * sysresccd provides /bin/zsh as default shell, hence
> * SHELL=/bin/zsh environment variable is set.
> * /mnt/gentoo does not have app-shells/zsh installed.
> * 'chroot /mnt/gentoo /bin/bash' does preserve SHELL environment variable.
> 
> How analysed:
> $ strace -f -e trace=execve emerge -1 dev-python/numpy
> shows it does want (but fails) to execute /bin/zsh.

Installing Gentoo multilib with systemd on the new PC and encountered the same problem with SystemRescueCD and missing zsh in chroot, while installing wicd

Running emerge with bash instead of zsh solves my problem
$ SHELL=/bin/bash emerge -1 dev-python/numpy

Thanks for resolving the problem!