Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 699650

Summary: sci-libs/fftw-3.3.8 : configure: error: could not find mpi library for --enable-mpi
Product: Gentoo Linux Reporter: Toralf Förster <toralf>
Component: Current packagesAssignee: Gentoo Science Related Packages <sci>
Status: RESOLVED TEST-REQUEST    
Severity: normal CC: jorge.ramos, jstein, preed, zlogene
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 733576    
Bug Blocks:    
Attachments: emerge-info.txt
emerge-history.txt
environment
etc.portage.tbz2
logs.tbz2
sci-libs:fftw-3.3.8:20191109-070937.log
temp.tbz2
Fixes fftw-3.3.6_p2 ebuild for systems with no openmpi.

Description Toralf Förster gentoo-dev 2019-11-09 09:59:47 UTC
checking for MPI_Init in -lmpi... no
checking for MPI_Init in -lmpich... no
configure: error: could not find mpi library for --enable-mpi

!!! Please attach the following file when seeking support:
!!! /var/tmp/portage/sci-libs/fftw-3.3.8/work/fftw-3.3.8-single-abi_x86_64.amd64/config.log

  -------------------------------------------------------------------

  This is an unstable amd64 chroot image at a tinderbox (==build bot)
  name: 17.1_desktop_gnome-libressl-20191105-031619

  -------------------------------------------------------------------

gcc-config -l:
 [1] x86_64-pc-linux-gnu-9.2.0 *

clang:
clang version 9.0.0 (tags/RELEASE_900/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm/9/bin
llvm:
9.0.0
Available Python interpreters, in order of preference:
  [1]   python3.6
  [2]   python2.7 (fallback)
  [3]   pypy (fallback)
Available Ruby profiles:
  [1]   ruby24 (with Rubygems)
  [2]   ruby25 (with Rubygems) *
Available Rust versions:
  [1]   rust-1.38.0 *
java-config:
The following VMs are available for generation-2:
*)	IcedTea JDK 3.13.0 [icedtea-bin-8]
Available Java Virtual Machines:
  [1]   icedtea-bin-8  system-vm

repository:
==> /var/db/repos/gentoo/metadata/timestamp.chk <==
Sat, 09 Nov 2019 06:06:17 +0000

emerge -qpvO sci-libs/fftw
[ebuild   R   ] sci-libs/fftw-3.3.8  USE="fortran mpi openmp (-altivec) -doc (-neon) -quad -static-libs -test -threads (-zbus)" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="avx avx2 fma3 sse sse2 -fma4"
Comment 1 Toralf Förster gentoo-dev 2019-11-09 09:59:50 UTC
Created attachment 595522 [details]
emerge-info.txt
Comment 2 Toralf Förster gentoo-dev 2019-11-09 09:59:53 UTC
Created attachment 595524 [details]
emerge-history.txt
Comment 3 Toralf Förster gentoo-dev 2019-11-09 09:59:55 UTC
Created attachment 595526 [details]
environment
Comment 4 Toralf Förster gentoo-dev 2019-11-09 09:59:58 UTC
Created attachment 595528 [details]
etc.portage.tbz2
Comment 5 Toralf Förster gentoo-dev 2019-11-09 10:00:01 UTC
Created attachment 595530 [details]
logs.tbz2
Comment 6 Toralf Förster gentoo-dev 2019-11-09 10:00:04 UTC
Created attachment 595532 [details]
sci-libs:fftw-3.3.8:20191109-070937.log
Comment 7 Toralf Förster gentoo-dev 2019-11-09 10:00:07 UTC
Created attachment 595534 [details]
temp.tbz2
Comment 8 Antti Mäkelä 2020-03-23 17:45:13 UTC
I started getting this with latest emerge --sync. I guess something must have stabilized.
Comment 9 J. Paul Reed 2020-03-29 00:04:05 UTC
I ran into this problem starting ~last week myself; it's due to an assumption in the fftw 3.3.6_p2 and 3.3.8 ebuilds.

For a reason I don't understand (and I'm sure someone will tell me), MPI functionality which was provided by the openmpi library was replaced by the mpich2 library last week; not sure if this is because the virtual changed, or what, but I have:

> 1584881864:  >>> emerge (23 of 23) sys-cluster/mpich2-1.5 to /
> 1584881864:  === (23 of 23) Cleaning (sys-cluster/mpich2-1.5::/usr/portage/sys-cluster/mpich2/mpich2-1.5.ebuild)
> 1584881864:  === (23 of 23) Compiling/Merging (sys-cluster/mpich2-1.5::/usr/portage/sys-cluster/mpich2/mpich2-1.5.ebuild)
> 1584882013:  === (23 of 23) Merging (sys-cluster/mpich2-1.5::/usr/portage/sys-cluster/mpich2/mpich2-1.5.ebuild)
> 1584882015:  >>> AUTOCLEAN: sys-cluster/mpich2:0
> 1584882019:  === (23 of 23) Post-Build Cleaning (sys-cluster/mpich2-1.5::/usr/portage/sys-cluster/mpich2/mpich2-1.5.ebuild)
> 1584882019:  ::: completed emerge (23 of 23) sys-cluster/mpich2-1.5 to /
> 1584882019: === Unmerging... (sys-cluster/openmpi-2.0.2)
> 1584882025:  >>> unmerge success: sys-cluster/openmpi-2.0.2

Anyway, fftw's configure will check for MPI_Init() in both -lmpi (openmpi's implementation) and -lmpich (mpich2's), and seems to happily use either.

The problem is: the ebuild calls ./configure with -lmpi, which means it will _always_ try to link with libmpi, which means if you don't have openmpi on your system, then even though the link against -lmpich will succeed, the ./configure (and thus the build) will fail.

Attaching a patch for 3.3.6 shortly, but building the patched ebuild locally gives:

> make[1]: Leaving directory '/var/tmp/portage/sci-libs/fftw-3.3.6_p2-r1/work/fftw-3.3.6-pl2-longdouble-abi_x86_64.amd64/m4'
> >>> Completed installing sci-libs/fftw-3.3.6_p2-r1 into /var/tmp/portage/sci-libs/fftw-3.3.6_p2-r1/image/
>
> * Final size of build directory: 104832 KiB (102.3 MiB)
> * Final size of installed tree:    9872 KiB (  9.6 MiB)
>
> strip: x86_64-pc-linux-gnu-strip --strip-unneeded -N __gentoo_check_ldflags__ -R .comment -R .GCC.command.line -R .note.gnu.gold-version
>   /usr/lib64/libfftw3f_threads.so.3.5.6
>   /usr/lib64/libfftw3f_omp.so.3.5.6
>   /usr/lib64/libfftw3f.so.3.5.6
>   /usr/lib64/libfftw3f_mpi.so.3.5.6
>   /usr/bin/fftwf-wisdom
>   /usr/lib64/libfftw3.so.3.5.6
>   /usr/lib64/libfftw3_threads.so.3.5.6
>   /usr/lib64/libfftw3_omp.so.3.5.6
>   /usr/lib64/libfftw3_mpi.so.3.5.6
>   /usr/bin/fftw-wisdom
>   /usr/lib64/libfftw3l.so.3.5.6
>   /usr/lib64/libfftw3l_threads.so.3.5.6
>   /usr/lib64/libfftw3l_omp.so.3.5.6
>   /usr/lib64/libfftw3l_mpi.so.3.5.6
>   /usr/bin/fftwl-wisdom
>
> >>> Installing (1 of 1) sci-libs/fftw-3.3.6_p2-r1::localrepo
> <<< !needed  sym /usr/lib64/libmpi.so.20
> <<< !needed  obj /usr/lib64/libmpi.so.20.0.2
> <<< !needed  sym /usr/lib64/libopen-pal.so.20
> <<< !needed  obj /usr/lib64/libopen-pal.so.20.2.0
> <<< !needed  sym /usr/lib64/libopen-rte.so.20
> <<< !needed  obj /usr/lib64/libopen-rte.so.20.1.0
>
> >>> Recording sci-libs/fftw:3.0 in "world" favorites file...
> >>> Auto-cleaning packages...

The same patch should work for fftw-3.3.8, but I didn't try it.

NOTE: I don't know if it's actually intentional that openmpi got blown away; perhaps that's actually the core issue here?
Comment 10 J. Paul Reed 2020-03-29 00:07:15 UTC
Created attachment 626622 [details, diff]
Fixes fftw-3.3.6_p2 ebuild for systems with no openmpi.

Patch to fftw-3.3.6_p2 ebuild to fix the build on systems that only have mpich2 (and no openmpi) on them.
Comment 11 Christoph Junghans (RETIRED) gentoo-dev 2020-03-29 00:21:14 UTC
If I remember correctly that "-lmpi" had something to do with multilib. Have you tried that patch with ABI_X86="32 64"?
Comment 12 J. Paul Reed 2020-03-29 00:57:16 UTC
(In reply to Christoph Junghans from comment #11)
> If I remember correctly that "-lmpi" had something to do with multilib. Have
> you tried that patch with ABI_X86="32 64"?

I have not, and probably won't because doing so causes portage to want to rebuild a bunch of libraries that I'm not super-interested in dealing with.

Interestingly enough, asking portage to do this causes a conflict between openmpi and mpich2 in the other opposite direction.

> Calculating dependencies... done!
> [ebuild   R    ] dev-libs/libltdl-2.4.6  ABI_X86="32*" 
> [ebuild   R    ] dev-libs/libevent-2.1.8  ABI_X86="32*" 
> [ebuild   R    ] sys-process/numactl-2.0.12  ABI_X86="32*" 
> [ebuild   R    ] sys-apps/pciutils-3.5.6-r1  ABI_X86="32*" 
> [ebuild   R    ] x11-libs/libpciaccess-0.16  ABI_X86="32*" 
> [ebuild   R    ] sys-apps/hwloc-1.11.2-r1  ABI_X86="32*" 
> [ebuild  N     ] sys-cluster/openmpi-2.0.4  USE="fortran ipv6 threads -cma (-cuda) -cxx -heterogeneous -java -mpi-threads -numa -romio" ABI_X86="32 (64) (-x32)" OPENMPI_FABRICS="(-knem) (-ofed) (-psm)" OPENMPI_OFED_FEATURES="(-connectx-xrc) (-control-hdr-padding) (-dynamic-sl) (-failover) (-rdmacm) (-udcm)" OPENMPI_RM="(-pbs) (-slurm)" 
> [ebuild   R    ] virtual/mpi-2.0-r4  USE="fortran* -cxx*" ABI_X86="32*" 
> [ebuild   R    ] sci-libs/fftw-3.3.6_p2-r1  ABI_X86="32*" 
> [blocks B      ] sys-cluster/mpich2 ("sys-cluster/mpich2" is blocking sys-cluster/openmpi-2.0.4)
> [blocks B      ] sys-cluster/openmpi ("sys-cluster/openmpi" is blocking sys-cluster/mpich2-1.5)

This _looks_ like it may be related to: https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-cluster?id=b287f5f8078ecfc7b222267aa1d590d994bd87b9

There's also weirdness in that portage choose mpich2, as opposed to mpich (see bug 547208) when resolving the upgrade path.

Cc'ing the commit author.

Finally, it wouldn't surprise me too much if the -lmpi was put in there to solve multilib issues, but it seems like fftw does the right thing when looking for MPI_Init(), so I'm not sure it'd be necessary anymore anyway? Perhaps I'm missing why a force-link to a library would be needed when the ./configure seems to check for it. But even if it is requires, perhaps protecting it by a check for multilib makes sense...
Comment 13 Larry the Git Cow gentoo-dev 2020-03-29 16:57:37 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=67a3fff18cedaf357e1a8cff0eae5f8aaf4824eb

commit 67a3fff18cedaf357e1a8cff0eae5f8aaf4824eb
Author:     Christoph Junghans <junghans@gentoo.org>
AuthorDate: 2020-03-29 16:57:10 +0000
Commit:     Christoph Junghans <junghans@gentoo.org>
CommitDate: 2020-03-29 16:57:28 +0000

    sci-libs/fftw: fix build with mpich
    
    Closes: https://bugs.gentoo.org/699650
    Package-Manager: Portage-2.3.89, Repoman-2.3.20
    Signed-off-by: Christoph Junghans <junghans@gentoo.org>

 sci-libs/fftw/fftw-3.3.8-r1.ebuild | 182 +++++++++++++++++++++++++++++++++++++
 sci-libs/fftw/fftw-9999.ebuild     |  34 +++----
 2 files changed, 196 insertions(+), 20 deletions(-)
Comment 14 Jonas Stein gentoo-dev 2020-04-12 22:04:58 UTC
*** Bug 717164 has been marked as a duplicate of this bug. ***
Comment 15 Jonas Stein gentoo-dev 2020-04-12 22:06:17 UTC
Regarding to #717164 the bug does still exist. Reopening.
Comment 16 Christoph Junghans (RETIRED) gentoo-dev 2020-04-12 22:10:49 UTC
It was only fixed in fftw-3.3.8-r1, not fftw-3.3.6_p2. I just don't like to touch stable packages.
Comment 17 jorge 2020-04-13 00:36:32 UTC
(In reply to Christoph Junghans from comment #16)
> It was only fixed in fftw-3.3.8-r1, not fftw-3.3.6_p2. I just don't like to
> touch stable packages.

Oh, that's why. Let me know if you need anything.
Comment 18 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-02-25 03:06:57 UTC
Please check 3.3.9.
Comment 19 jorge 2021-02-25 21:02:39 UTC
fftw-3.3.9 Compiles for me