Hi, (1) please change the dependency for compilation of sci-libs/caffe2 with MPI support from sys-cluster/openmpi to virtual/mpi. Only one MPI implementation can be installed at the time (to my knowledge) and sys-cluster/openmpi forces the system to use OpenMPI. I am happy to confirm that sci-libs/caffe2 compiles OK using the sys-cluster/mpich MPI implementation. (2) as a side note: sci-libs/caffe2 wants the TORCH_CUDA_ARCH_LIST space separated, not comma separated, e.g. TORCH_CUDA_ARCH_LIST='7.5 3.5' Here is a patch: --- /var/db/repos/gentoo/sci-libs/caffe2/caffe2-2.0.1-r1.ebuild 2023-05-15 01:40:34.000000000 -0500 +++ caffe2-2.0.1-r2.ebuild 2023-06-13 15:07:27.276181886 -0500 @@ -3,7 +3,7 @@ EAPI=8 -PYTHON_COMPAT=( python3_{9..11} ) +PYTHON_COMPAT=( python3_{9..12} ) inherit python-single-r1 cmake cuda flag-o-matic MYPN=pytorch @@ -49,7 +49,7 @@ fbgemm? ( dev-libs/FBGEMM ) ffmpeg? ( media-video/ffmpeg:= ) gloo? ( sci-libs/gloo[cuda?] ) - mpi? ( sys-cluster/openmpi ) + mpi? ( virtual/mpi ) nnpack? ( sci-libs/NNPACK ) numpy? ( $(python_gen_cond_dep ' dev-python/numpy[${PYTHON_USEDEP}] @@ -106,7 +106,7 @@ ewarn "" ewarn "To configure caffe2 with the CUDA compute capability that is optimal for your GPU," ewarn "set TORCH_CUDA_ARCH_LIST in your make.conf, and re-emerge caffe2." - ewarn "For example, to use CUDA capability 7.5 & 3.5, add: TORCH_CUDA_ARCH_LIST=7.5,3.5" + ewarn "For example, to use CUDA capability 7.5 & 3.5, add: TORCH_CUDA_ARCH_LIST='7.5 3.5'" ewarn "For a Maxwell model GPU, an example value would be: TORCH_CUDA_ARCH_LIST=Maxwell" ewarn "" ewarn "You can look up your GPU's CUDA compute capability at https://developer.nvidia.com/cuda-gpus" Reproducible: Always Steps to Reproduce: 1. emerge --unmerge sys-cluster/openmpi 2. emerge -1 sys-cluster/mpich 3. emerge virtual/mpi 4. emerge -1av caffe2 Actual Results: Calculating dependencies... done! Dependency resolution took 12.56 s. [ebuild N ] sys-cluster/openmpi-4.1.4-r1 USE="cxx fortran ipv6 java romio -cma (-cuda) -libompitrace -peruse -valgrind" ABI_X86="(64) -32 (-x32)" OPENMPI_FABRICS="(-knem) (-ofed)" OPENMPI_OFED_FEATURES="(-control-hdr-padding) (-dynamic-sl) (-rdmacm) (-udcm)" OPENMPI_RM="(-pbs) (-slurm)" [ebuild UD~] sci-libs/caffe2-2.0.1-r1 [2.0.1-r2] [blocks B ] sys-cluster/openmpi ("sys-cluster/openmpi" is soft blocking sys-cluster/mpich-3.4.3-r1) [blocks B ] sys-cluster/mpich ("sys-cluster/mpich" is soft blocking sys-cluster/openmpi-4.1.4-r1) * Error: The above package list contains packages which cannot be * installed at the same time on the same system. (sys-cluster/openmpi-4.1.4-r1:0/0::gentoo, ebuild scheduled for merge) pulled in by >=sys-cluster/openmpi-4.1.2[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_s390_32(-)?,abi_s390_64(-)?,cxx?,fortran?,romio?,threads(+)?] (>=sys-cluster/openmpi-4.1.2[abi_x86_64(-),cxx,fortran,romio,threads(+)]) required by (virtual/mpi-2.0-r7:0/0::gentoo, installed) USE="cxx fortran romio threads userland_GNU -nullmpi" ABI_X86="(64) -32 (-x32)" sys-cluster/openmpi required by (sci-libs/caffe2-2.0.1-r1:0/0::gentoo, ebuild scheduled for merge) USE="cuda distributed ffmpeg mpi numpy opencl opencv openmp tensorpipe -fbgemm -gloo -nnpack -qnnpack -xnnpack" ABI_X86="(64)" PYTHON_SINGLE_TARGET="python3_11 -python3_10" (sys-cluster/mpich-3.4.3-r1:0/0::myRepo, installed) pulled in by sys-cluster/mpich required by (sci-libs/mkl-2023.0.0.25398:0/0::gentoo, installed) USE="" ABI_X86="(64)" For more information about Blocked Packages, please refer to the following section of the Gentoo Linux x86 Handbook (architecture is irrelevant): https://wiki.gentoo.org/wiki/Handbook:X86/Working/Portage#Blocked_packages Expected Results: These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 11.35 s. [ebuild R ~] sci-libs/caffe2-2.0.1-r2::myRepo USE="cuda distributed ffmpeg mpi numpy opencl opencv openmp tensorpipe -fbgemm -gloo -nnpack -qnnpack -xnnpack" PYTHON_SINGLE_TARGET="python3_11 -python3_10 -python3_12%" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No] Portage 3.0.45.3 (python 3.11.3-final-0, default/linux/amd64/17.1/systemd/merged-usr, gcc-12, glibc-2.36-r8, 5.10.181-gentoo x86_64) ================================================================= System uname: Linux-5.10.181-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5-2680_v3_@_2.50GHz-with-glibc2.36 KiB Mem: 197999764 total, 88293836 free KiB Swap: 199229436 total, 192372732 free Timestamp of repository gentoo: Sun, 04 Jun 2023 16:00:01 +0000 Head commit of repository gentoo: dc822b01fbf03a429b1b9d3457134695bcb6a790 sh bash 5.1_p16-r4 ld GNU ld (Gentoo 2.39 p6) 2.39.0 app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r4::gentoo dev-java/java-config: 2.3.1-r1::gentoo dev-lang/perl: 5.36.0-r2::gentoo dev-lang/python: 2.7.18_p16-r1::gentoo, 3.9.16-r1::gentoo, 3.10.11::gentoo, 3.11.3::gentoo dev-lang/rust: 1.69.0-r1::gentoo dev-util/cmake: 3.26.3::gentoo dev-util/meson: 1.1.1::gentoo sys-apps/baselayout: 2.13-r1::gentoo sys-apps/sandbox: 2.29::gentoo sys-apps/systemd: 253.3-r1::gentoo sys-devel/autoconf: 2.13-r7::gentoo, 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 15.0.7-r1::gentoo sys-devel/gcc: 11.3.1_p20230427::gentoo, 12.2.1_p20230428-r1::gentoo sys-devel/gcc-config: 2.10::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/lld: 15.0.7::gentoo sys-devel/llvm: 14.0.6-r2::gentoo, 15.0.7::gentoo sys-devel/make: 4.4.1-r1::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r8::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 volatile: False sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: sync-rsync-verify-max-age: 24 sync-rsync-verify-metamanifest: yes myRepo location: /var/db/repos/myRepo masters: gentoo volatile: False ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE @FREE @BINARY-REDISTRIBUTABLE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mfpmath=sse -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mfpmath=sse -O2 -pipe" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://mirror.csclub.uwaterloo.ca/gentoo-distfiles/" LANG="en_CA.UTF-8" LC_ALL="" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" LINGUAS="en en_CA en_GB de de_CH fr fr_CA fr_FR" MAKEOPTS="-j12" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="X a52 aac aalib acl acpi aim alsa amd64 amr amrnb amrwb aom apache2 audio avahi bidi blas bluray branding bzip2 cairo cdda cddb cdparanoia cdr cjk cli crypt cuda cups curl curlwrappers dav1d dbus dmi dri dts dv dvb dvd dvdr dvdread emacs emboss emf encode exif faac faad fam ffmpeg fftw firefox flac fontconfig foomaticdb fortran ftp gcj gd gdbm gif gimp ginac glade glut gmp gnome-keyring gnutls gphoto2 gpm graphviz gstreamer gtk gtk3 gtkhtml guile heif iconv icq icu idn ieee1394 imagemagick imap imlib ipod ipv6 jabber java javascript jbig jpeg jpeg2k jpegxl lapack lcms ldap lesstif libaom libcaca libedit libnotify libtirpc lm-sensors lto lz4 lzip lzma lzo m17n-lib mad matroska mikmod mime mmap mng modplug mono mozilla mp3 mpeg mpi mplayer mtp multilib musepack ncurses netcdf nls nntp nptl nptlonly ogg opencl opengl openh264 openmp opus osc oscar pam pcre pdf perl php plotutils png policykit posix postscript povray ppds pulseaudio python qml qt5 quicktime rar rav1e readline recode rtmp samba sdl seccomp sharedmem smp sndfile sound sox speex spell sqlite ssl startup-notification svg svt-av1 syslog systemd taglib tcl test-rust theora threads threadsafe thunar tiff tk truetype udev udisks unicode upnp usb v4l vaapi vcd vdpau video vnc vorbis vpx vulkan wav webp win32codecs wmf wxwindows x264 x265 xattr xcb xcomposite xft xine xinerama xml xmp xosd xpm xv xvid zlib zstd" ABI_X86="64 32" ADA_TARGET="gnat_2021" ALSA_CARDS="hda-intel" APACHE2_MODULES="access_compat actions alias asis auth_basic auth_digest authn_anon authn_core authn_dbd authn_dbm authn_default authn_file authz_core authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgid dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio lbmethod_byrequests lbmethod_bytraffic lbmethod_bybusyness lbmethod_heartbeat \ mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif slotmem_shm so socache_shmcb speling status unique_id unixd userdir usertrack vhost_alias" APACHE2_MPMS="worker" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en en-CA en-GB de de-CH fr fr-CA fr-FR" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_9 python3_10 python3_11" RUBY_TARGETS="ruby30" SANE_BACKENDS="epson epson2" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Created attachment 863795 [details, diff] MPI patch for sci-libs/caffe2 to use virtual/mpi I have attached the patch as a file. Thank you very much! With best regards, Markus
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=715a287b389218206948f60a4e848300b56364e3 commit 715a287b389218206948f60a4e848300b56364e3 Author: Alfredo Tupone <tupone@gentoo.org> AuthorDate: 2023-06-17 14:02:05 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2023-06-17 14:02:05 +0000 sci-libs/caffe2: depends on virtual/mpi Closes: https://bugs.gentoo.org/908462 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> sci-libs/caffe2/{caffe2-2.0.1-r1.ebuild => caffe2-2.0.1-r2.ebuild} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
