Hi. Recently, in bug #512884 dev-libs/boost was ported to use multilib features. But neigher sys-cluster/openmpi nor sys-cluster/mpich2 ebuilds have them - those two packages are dependendices for Boost, builded with USE="mpi". I have added dirty hack to workaround this issue for now - MPI is used only for native ABI. Please provide multilib-aware ebuilds for those two packages or explain if it can not be done, thanks in advance
Please, we had similar problem for fftw in bug #519094 and bug #483758.
sys-cluster/mpich deps: - dev-libs/libaio (already multilib) - sys-apps/hwloc (not multilib, but bug #456984) - net-fs/nfs-utils (not multilib, but only USE=romio) - sys-devel/libtool (already multilib) sys-cluster/openmpi: - dev-libs/libevent (already multilib) - >=sys-apps/hwloc-1.7.2 (not multilib) - sys-libs/zlib (already multilib) - dev-util/nvidia-cuda-toolkit (not multilib, but only USE=cuda) - ... many more.
Ok, I implemented a version for testing in the science overlay.
@pinkbyte: I have added =dev-libs/boost-1.56.0-r2::science with multilib mpi for testing. However, currently there are two major showstoppers 1.) need to export multilib FC/F77 in toolchain.eclass (bug #540294) 2.) wrong path in gcc's 32bit libtool files (bug #540392)
(In reply to Christoph Junghans from comment #3) > Ok, I implemented a version for testing in the science overlay. Please continue to use mpi.eclass in the overlay.
https://github.com/gentoo-science/sci/issues/372
(In reply to Justin Bronder from comment #5) > Please continue to use mpi.eclass in the overlay. Sorry, I overlooked that, is mpi.eclass already multilib capable?
(In reply to Christoph Junghans from comment #7) > (In reply to Justin Bronder from comment #5) > > Please continue to use mpi.eclass in the overlay. > Sorry, I overlooked that, is mpi.eclass already multilib capable? I have no idea :) I haven't really looked at multilib too closely as the deps were not multilib. Then you guys all got motivated to work on this so who was I to complain... until I noticed mpi.eclass got left out. But people are depending on being able to use empi to install multiple MPI implementations from the overlay. It's not a huge issue right now as empi will ignore stuff that doesn't use mpi.eclass. So no one is 'broken', but eventually they will want upgrades. As I try to keep things somewhat in sync between the main tree and the overlay, not being able to use mpi.eclass with multilib will be a blocker for merging this stuff to gx86 until the two can play nicely together. I can look closer at multilib later, but if it's also playing games with the configure arguments like mpi.eclass then there's probably going to be some work involved. Hopefully we can pass stuff like mpi_root() and mpi_class() to multilib to come up with paths both eclasses are happy with.
(In reply to Justin Bronder from comment #8) > (In reply to Christoph Junghans from comment #7) > > (In reply to Justin Bronder from comment #5) > > > Please continue to use mpi.eclass in the overlay. > > Sorry, I overlooked that, is mpi.eclass already multilib capable? > > I have no idea :) I haven't really looked at multilib too closely as the > deps were not multilib. Then you guys all got motivated to work on this so > who was I to complain... until I noticed mpi.eclass got left out. Sorry to have broken your stuff! Is there any reason to not port mpi.eclass to gx86? Maybe this would be the right opportunity to at least add support for multiple simultaneously installed mpi versions.
(In reply to Christoph Junghans from comment #9) > (In reply to Justin Bronder from comment #8) > > (In reply to Christoph Junghans from comment #7) > > > (In reply to Justin Bronder from comment #5) > > > > Please continue to use mpi.eclass in the overlay. > > > Sorry, I overlooked that, is mpi.eclass already multilib capable? > > > > I have no idea :) I haven't really looked at multilib too closely as the > > deps were not multilib. Then you guys all got motivated to work on this so > > who was I to complain... until I noticed mpi.eclass got left out. > Sorry to have broken your stuff! Is there any reason to not port mpi.eclass > to gx86? Maybe this would be the right opportunity to at least add support > for multiple simultaneously installed mpi versions. Not really. Way back when mpi.eclass was originally written posting an eclass to -dev was just asking for flames and bikeshedding so I didn't bother. People using it typically install the overlay anyway and seemed happy enough with the current setup. It might be worth making sure that people still want this functionality. When it was under development I got a lot of feedback but I haven't heard much recently especially from the larger sites.
Created attachment 442882 [details, diff] patch to add multilib support to openmpi
Created attachment 442884 [details, diff] patch to add multilib support to mpich
Stuff now in the main tree, let's wait a couple of weeks and then re-add keywords.
(In reply to Christoph Junghans from comment #13) > Stuff now in the main tree, let's wait a couple of weeks and then re-add > keywords. Assuming no calls to keep empi/mpi.eclass around from the science mailing list that is ;)
(In reply to Justin Bronder from comment #14) > (In reply to Christoph Junghans from comment #13) > > Stuff now in the main tree, let's wait a couple of weeks and then re-add > > keywords. > > Assuming no calls to keep empi/mpi.eclass around from the science mailing > list that is ;) Two weeks later and only one response (by a gentoo dev) later, what should we do?
(In reply to Christoph Junghans from comment #15) > (In reply to Justin Bronder from comment #14) > > (In reply to Christoph Junghans from comment #13) > > > Stuff now in the main tree, let's wait a couple of weeks and then re-add > > > keywords. > > > > Assuming no calls to keep empi/mpi.eclass around from the science mailing > > list that is ;) > Two weeks later and only one response (by a gentoo dev) later, what should > we do? The problem with unkeyworded ebuilds is that don't really get any exposure. I would opt for keywording them all and unleashing them on unstable users.
Well that's two production HPC clusters with some number of users depending on mpi.eclass which I believe are exactly the sort of people we should be supporting with openmpi and mpich. I'm still not sure what the advantage of multilib.eclass is for that target audience. I still have no problem with it being added but I'm very hesitant to break current functionality that anyone is using. I'd ask that you guys look at getting it working in the overlay with mpi.eclass first [1] or let me know why that's not a workable solution. Really, all that eclass does is install to a different prefix. 1. If I don't get to it first, but let's be honest, I'm not all that motivated to spend the limited amount of time I have on this.
I don't want to hinder development in the main tree due to some eclass in the science overlay. Why aren't we just masking "sys-cluster/{openmpi,mpich2}::gentoo abi_x86_32" and "sci-libs/fftw::gentoo abi_x86_32" in science overlay and put "!multilib_is_native_abi" guard around the extra stuff we added to multilib sys-cluster/{openmpi,mpich2} and merge these changes back in the science overlay.
The actual users of these packages are using the science overlay. But, if I understand your proposal correctly, I think it satisfies my concerns. 1. Preserve functionality people are currently using 2. Make it easy to move changes back and forth from the overlay (the diff between ebuilds should be small and very obvious as to what can be left out). So, if the patch ends up looking like something similar to the difference between pkg-9999.ebuild and pkg-1.2.3.ebuild where the former pulls from a vcs and the latter is using tarballs, I like it. Please post something for review.
Ok, package.use{,.mask} is a per-overlay thing, so we cannot mask the abi_x86_32 in gx86 from the science overlay, but we could explicitly block against sys-cluster/openmpi[abi_x86_32] in mpi.eclass on amd64 and the same in virtual/mpi. Concerning the changes, I would add a variable to empi.eclass e.g. IS_EMPI=yes, which we can use differentiate the overlays. The advantage of a variable over a functions that the default is empty and hence gx86 doesn't need to know about mpi.eclass.
Work in progress on the multilib_mpi branch of the science overlay: https://github.com/gentoo-science/sci/compare/multilib_mpi I think I will port sys-cluster/nullmpi to gx86 and hence remove virtual/mpi from the science overlay. @soap: Can you back-port your clean-up to mpich and openmpi to the science overlay and put the multilib addition in [[ ${EMPI} ]] guards? On a related though mpi.eclass should maybe export MULTILIB_COMPAT="abi_x86_64", too?
I found a much simpler solution using REQUIRED_USE and ported the changes from gx86 back into the science overlay, see: https://github.com/gentoo-science/sci/pull/664 I didn't tested mpich yet!
Multilib support merge in science overlay. Re-added keywords to ebuild in gx86, let's see what breaks!
@toralf: Could please you do a tinderbox run using multilib mpi?
(In reply to Christoph Junghans from comment #24) I think it is already in place : tinderbox@mr-fox ~ $ grep -i ' mpi' amd64-*/etc/portage/make.conf amd64-hardened-libressl-unstable_20160821-203105/etc/portage/make.conf: libinput mad midi mikmod mod modplug mpeg2 mpg123 mpi multitarget amd64-hardened-unstable_20160821-190619/etc/portage/make.conf: libvirtd midi mod mpeg mpi multimedia multitarget nss opencv -pam pdo
