Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 652432 - sys-cluster/pmix-2.1.1 with sys-cluster/openmpi-3.0.0-r1 - file collisions in /usr/lib64/pmix/* /usr/share/pmix/*
Summary: sys-cluster/pmix-2.1.1 with sys-cluster/openmpi-3.0.0-r1 - file collisions in...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Cluster Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 656080
  Show dependency tree
 
Reported: 2018-04-04 17:12 UTC by Toralf Förster
Modified: 2024-07-12 05:55 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge-info.txt (emerge-info.txt,15.98 KB, text/plain)
2018-04-04 17:12 UTC, Toralf Förster
Details
emerge-history.txt (emerge-history.txt,144.10 KB, text/plain)
2018-04-04 17:12 UTC, Toralf Förster
Details
etc.portage.tbz2 (etc.portage.tbz2,12.03 KB, application/x-bzip)
2018-04-04 17:12 UTC, Toralf Förster
Details
logs.tbz2 (logs.tbz2,37.75 KB, application/x-bzip)
2018-04-04 17:12 UTC, Toralf Förster
Details
sys-cluster:pmix-2.1.1:20180404-013731.log (sys-cluster:pmix-2.1.1:20180404-013731.log,262.43 KB, text/plain)
2018-04-04 17:12 UTC, Toralf Förster
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toralf Förster gentoo-dev 2018-04-04 17:12:22 UTC
  -------------------------------------------------------------------

  This is an unstable amd64 chroot image at a tinderbox (==build bot)
  name: 17.0-desktop-plasma-systemd_20180328-181735

  -------------------------------------------------------------------

gcc-config -l:
 [1] x86_64-pc-linux-gnu-7.3.0 *

Available Python interpreters, in order of preference:
  [1]   python3.5
  [2]   python2.7 (fallback)
  [3]   jython2.7 (fallback)
Available Ruby profiles:
  [1]   ruby22 (with Rubygems)
  [2]   ruby23 (with Rubygems) *
java-config:
The following VMs are available for generation-2:
*)	IcedTea JDK 3.7.0 [icedtea-bin-8]
2)	JamVM JDK 2.0.0 [jamvm]
Available Java Virtual Machines:
  [1]   icedtea-bin-8  system-vm
  [2]   jamvm 

emerge -qpv sys-cluster/pmix
[ebuild  N    ] sys-cluster/pmix-2.1.1  USE="munge -debug -pmi"
Comment 1 Toralf Förster gentoo-dev 2018-04-04 17:12:25 UTC
Created attachment 526550 [details]
emerge-info.txt
Comment 2 Toralf Förster gentoo-dev 2018-04-04 17:12:28 UTC
Created attachment 526552 [details]
emerge-history.txt
Comment 3 Toralf Förster gentoo-dev 2018-04-04 17:12:31 UTC
Created attachment 526554 [details]
etc.portage.tbz2
Comment 4 Toralf Förster gentoo-dev 2018-04-04 17:12:34 UTC
Created attachment 526556 [details]
logs.tbz2
Comment 5 Toralf Förster gentoo-dev 2018-04-04 17:12:37 UTC
Created attachment 526558 [details]
sys-cluster:pmix-2.1.1:20180404-013731.log
Comment 6 Justin Bronder (RETIRED) gentoo-dev 2018-04-04 17:47:12 UTC
Looks like jlec just recently added this.  It's been internally packaged in openmpi since at least the 2.0.0 release.

With 3.0.0, openmpi updated it's internal copy to 2.1.0, so I can just update the dependency there and have openmpi use the new package (assuming 2.1.1 is just a patch release).

For older openmpi versions, I suggest we just create blockers (they all used <=pmix-1.2.5) in order to avoid complexity.

jlec, that work for you?
Comment 7 silver_ghost 2018-07-08 22:57:05 UTC
Open MPI configure has --with-pmix=external option. I've managed to build sys-cluster/openmpi-3.1.1 for ABI_X86="64 -32 -x32" (pmix isn't multilib) with this change:
--- /usr/portage/sys-cluster/openmpi/openmpi-3.1.1.ebuild       2018-07-02 20:09:21.000000000 +0300
+++ /usr/portage/sys-cluster/openmpi/openmpi-3.1.1.ebuild  2018-07-09 01:26:47.037477263 +0300
@@ -48,7 +48,7 @@
        !sys-cluster/mpich2
        !sys-cluster/nullmpi
        !sys-cluster/mpiexec
-       !sys-cluster/pmix
+       sys-cluster/pmix
        >=dev-libs/libevent-2.0.22[${MULTILIB_USEDEP},threads]
        dev-libs/libltdl:0[${MULTILIB_USEDEP}]
        >=sys-apps/hwloc-1.11.2[${MULTILIB_USEDEP},numa?]
@@ -109,6 +109,7 @@
                --with-libltdl="${EPREFIX}/usr" \
                --with-libevent="${EPREFIX}/usr" \
                --enable-mpi-fortran=$(usex fortran all no) \
+               $(multilib_native_with pmix external) \
                $(use_enable cxx mpi-cxx) \
                $(use_with cma) \
                $(use_with cuda cuda "${EPREFIX}"/opt/cuda) \
Comment 8 silver_ghost 2018-07-20 16:22:21 UTC
How is it going? Did this info helped?:
(In reply to silver_ghost from comment #7)
> Open MPI configure has --with-pmix=external option. I've managed to build
> .....
Comment 9 silver_ghost 2018-07-22 19:11:33 UTC
Am I right that pmix should have multilib support if openmpi does?
Comment 10 Justin Bronder (RETIRED) gentoo-dev 2018-07-23 14:10:07 UTC
(In reply to silver_ghost from comment #8)
> How is it going? Did this info helped?:
> (In reply to silver_ghost from comment #7)
> > Open MPI configure has --with-pmix=external option. I've managed to build
> > .....

That doesn't help as we need multilib support all the way down the pmix deptree.  Which means, pending interest from others to do some of that work, this blocker is probably going to stay around for a while.
Comment 11 silver_ghost 2018-07-23 14:23:03 UTC
(In reply to Justin Bronder from comment #10)
> That doesn't help as we need multilib support all the way down the pmix
> deptree.  Which means, pending interest from others to do some of that work,
> this blocker is probably going to stay around for a while.

I'm trying to do this. I've managed to build sys-cluster/pmix[-debug -munge -pmi] for now. ABI_X86="32 64".
Comment 12 Larry the Git Cow gentoo-dev 2024-06-14 06:45:58 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b462e21d01293f5ac38417ff9cb4785b19d46385

commit b462e21d01293f5ac38417ff9cb4785b19d46385
Author:     Eli Schwartz <eschwartz93@gmail.com>
AuthorDate: 2024-06-10 06:00:28 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-06-14 06:44:43 +0000

    sys-cluster/openmpi: force disable building against system pmix
    
    It cannot currently work, since it lacks multilib support plus keywords.
    Avoid automagically depending on it, at least.
    
    Bug: https://bugs.gentoo.org/652432
    Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-cluster/openmpi/openmpi-4.1.6.ebuild | 2 ++
 1 file changed, 2 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7e492f3130ba7d79ef13fecc3a676956bbc0b8c5

commit 7e492f3130ba7d79ef13fecc3a676956bbc0b8c5
Author:     Eli Schwartz <eschwartz93@gmail.com>
AuthorDate: 2024-06-11 13:57:11 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-06-14 06:44:41 +0000

    sys-cluster/openmpi: prevent parallel installation with pmix
    
    It is vendored in openmpi, so installing openmpi gets you the contents
    of sys-cluster/pmix. This is bad and wrong, but also work to resolve. (It
    is also work that *needs* to be resolved for openmpi 5.x.)
    
    Things that pmix needs before becoming a proper dependency of openmpi:
    - multilib
    - keywording on various arches
    
    The stopgap measure is to mark the two as incompatible while we work on
    the rest.
    
    Bug: https://bugs.gentoo.org/652432
    Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-cluster/openmpi/openmpi-4.1.6.ebuild | 1 +
 1 file changed, 1 insertion(+)
Comment 13 Larry the Git Cow gentoo-dev 2024-07-12 05:55:15 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=496a4f0ce86f43da3fe77ffd6c9bef2e41cf3852

commit 496a4f0ce86f43da3fe77ffd6c9bef2e41cf3852
Author:     Eli Schwartz <eschwartz93@gmail.com>
AuthorDate: 2024-06-10 04:05:03 +0000
Commit:     Eli Schwartz <eschwartz@gentoo.org>
CommitDate: 2024-07-12 05:54:15 +0000

    sys-cluster/openmpi: add 5.0.3
    
    A bunch of upstream changes occurred. In particular:
    
    - openmpi drops ALL support for 32-bit, and errors out in ./configure if
      you try. This follows pmix. Rip out all the multilib-minimal
      scaffolding.
    
    - libompitrace "was incomplete and unmaintained" and is now removed from
      the sources
    
    - upstream now defaults to --disable-dlopen, and configuring with
      libltdl enabled externally returns errors saying a non libltdl header
      doesn't exist. Unclear if it actually supports this
    
    - a couple dependencies can now be configured --with-*=external instead
      of passing paths
    
    - libibverbs handling is gone upstream and no longer makes sense to
      configure via USE flags (or at all):
      https://github.com/open-mpi/ompi/commit/59c8ab6da4276ff398453a54910c6c0fb67a153c
    
    Delayed:
    - heterogeneous was broken in older versions, and its USE flag is
      supposed to be restored. But the upstream docs still suggest it is
      broken.
    
    Independent of upstream rework of pmix, we take the opportunity of a
    version bump to build against the system pmix, resolving a longstanding
    bug due to openmpi publicly shipping its own pmix installation that
    stomps all over the global system namespace. Temporarily drop keywords
    which the pmix package lacks.
    
    Bug: https://bugs.gentoo.org/828123
    Closes: https://bugs.gentoo.org/652432
    Closes: https://bugs.gentoo.org/927828
    Closes: https://bugs.gentoo.org/930362
    Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
    Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>

 sys-cluster/openmpi/Manifest             |   1 +
 sys-cluster/openmpi/openmpi-5.0.3.ebuild | 141 +++++++++++++++++++++++++++++++
 2 files changed, 142 insertions(+)