Building with rocm use flag [0mhipblas VERSION: 1.1.0[0m [31mCMake Error at cmake/public/LoadHIP.cmake:36 (find_package): By not providing "Findhipblaslt.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "hipblaslt", but CMake did not find one. Could not find a package configuration file provided by "hipblaslt" with any of the following names: hipblasltConfig.cmake hipblaslt-config.cmake Add the installation prefix of "hipblaslt" to CMAKE_PREFIX_PATH or set "hipblaslt_DIR" to a directory containing one of the above files. If "hipblaslt" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): cmake/public/LoadHIP.cmake:159 (find_package_and_print_version) cmake/Dependencies.cmake:1258 (include) CMakeLists.txt:754 (include) [0m -- Configuring incomplete, errors occurred! Reproducible: Always Steps to Reproduce: 1. Set rocm use flag 2. emerge caffe2-2.3.0 Actual Results: Can't find hipblaslt and refuses to compile Expected Results: Either adds a dependency for hipblaslt (which does not seem to be the same thing as just hipBLAS) or configure around it Portage 3.0.64 (python 3.11.9-final-0, default/linux/amd64/23.0, gcc-13, glibc-2.39-r4, 6.8.0-gentoo-x86_64 x86_64) ================================================================= System uname: Linux-6.8.0-gentoo-x86_64-x86_64-Intel-R-_Xeon-R-_CPU_X5690_@_3.47GHz-with-glibc2.39 KiB Mem: 49333516 total, 5987772 free KiB Swap: 16777212 total, 14564092 free Head commit of repository gentoo: e601cb60a768c484a08a141f1385b7251357cfc0 Timestamp of repository haarp: Mon, 29 Apr 2024 12:22:50 +0000 Head commit of repository haarp: f96d5fc350861e3bcb3a8fda300431db4d614a7e Timestamp of repository science: Mon, 29 Apr 2024 11:35:36 +0000 Head commit of repository science: 77e5907b0fc244ad4f7240a2869015189c7160cb Timestamp of repository steam-overlay: Fri, 19 Apr 2024 23:18:33 +0000 Head commit of repository steam-overlay: 69fc49999f96346c4cc4692a9a781b757a11c22a Timestamp of repository thegreatmcpain: Tue, 30 Apr 2024 06:03:30 +0000 Head commit of repository thegreatmcpain: d16f7ffe55a9fa12e0ce4e8952b043e51a228828 sh bash 5.2_p26-r3 ld GNU ld (Gentoo 2.42 p3) 2.42.0 ccache version 4.9.1 [enabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p26-r3::gentoo dev-build/autoconf: 2.72-r1::gentoo dev-build/automake: 1.16.5-r2::gentoo dev-build/cmake: 3.29.2::gentoo dev-build/libtool: 2.4.7-r4::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.4.0-r1::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r2::gentoo dev-lang/python: 3.10.14_p1::gentoo, 3.11.9::gentoo, 3.12.3::gentoo dev-lang/rust: 1.77.1::gentoo dev-util/ccache: 4.9.1-r1::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/openrc: 0.54::gentoo sys-apps/sandbox: 2.38::gentoo sys-devel/binutils: 2.42-r1::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 17.0.6::gentoo, 18.1.4::gentoo sys-devel/gcc: 13.2.1_p20240210::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 17.0.6::gentoo, 18.1.4::gentoo sys-devel/llvm: 17.0.6::gentoo, 18.1.4::gentoo sys-kernel/linux-headers: 6.8-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.39-r4::gentoo Repositories: gentoo location: /usr/portage sync-type: git sync-uri: git://git.gentoo.org/repo/gentoo.git priority: -1000 volatile: True haarp location: /var/db/repos/haarp sync-type: git sync-uri: https://github.com/gentoo-mirror/haarp.git masters: gentoo volatile: False science location: /var/db/repos/science sync-type: git sync-uri: https://github.com/gentoo-mirror/science.git masters: gentoo volatile: False steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo volatile: False thegreatmcpain location: /var/db/repos/thegreatmcpain sync-type: git sync-uri: https://github.com/gentoo-mirror/thegreatmcpain.git masters: gentoo volatile: False ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc/apache2 /etc/bash /etc/bitcoin /etc/chromium /etc/conf.d /etc/cups /etc/default /etc/dnsmasq.conf /etc/fstab /etc/genkernel.conf /etc/hosts /etc/inittab /etc/locale.gen /etc/locale.nopurge /etc/modprobe.d/nvidia.conf /etc/modules.autoload.d /etc/mongodb.conf /etc/monitrc /etc/mysql /etc/nginx/nginx.conf /etc/nzbget.conf /etc/php /etc/rc.conf /etc/runit /etc/sabnzbd /etc/samba /etc/security/limits.conf /etc/squid /etc/ssh /etc/sv /etc/sysctl.conf /etc/syslog.conf /etc/vim /etc/vpnc /var/lib/mysql" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.2/ext-active/ /etc/php/cgi-php8.2/ext-active/ /etc/php/cli-php8.2/ext-active/ /etc/php/fpm-php8.2/ext-active/ /etc/php/phpdbg-php8.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d" CXXFLAGS="-O2 -march=native -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build=n --with-bdeps=y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -march=native -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news nodoc noinfo noman parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sfperms strict unknown-features-warn unmerge-logs unmerge-orphans usersandbox usersync xattr" FFLAGS="-O2 -march=native -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" INSTALL_MASK="/usr/share/locale -/usr/share/locale/en -/usr/share/locale/en_GB" LANG="C" LDFLAGS="-Wl,--as-needed -Wl,-O1" LEX="flex" LINGUAS="en_GB en_US en" MAKEOPTS="-j6" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="X acl aes-ni aio alsa amd64 amdgpu bzip2 cet crypt cups custom-cflags encode flac gdbm gif glamor graphite iconv ipv6 jemalloc jit jpeg kdeprefix libtirpc lto lzma lzo mad midi mp3 multilib ncurses nls nptl ogg opencl opengl openmp openssl pam pcre png qt3support readline rocm run-as-root seccomp spell ssl static static-libs svg test-rust theora threads truetype udev unicode vaapi vdpau vorbis vulkan x264 xattr xfs xinerama xvid zlib zstd" ABI_X86="32 64" ADA_TARGET="gcc_12" AMDGPU_TARGETS="gfx906" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en-GB en" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" QEMU_SOFTMMU_TARGETS="x86_64" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Created attachment 892081 [details] build.log
This patch seems to fix the hipblaslt errors by making it optional, since hipblaslt doesn't even work for newer cards https://github.com/pytorch/pytorch/commit/8edc7b9854150185d52f0c1b2905913f5ed09442.patch This package still has some errors though. It tries to clone a git repo for aotriton and gets stuck due to the network-sandbox
Can you attach the log with the remaining errors?
Created attachment 892543 [details] build.log network-sandbox errors due to trying to clone a git repo
Can you pass MYCMAKEARGS="-DUSE_FLASH_ATTENTION=no" via env file?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=27e4c06d52a66978ec41a325468bdd3b756236ba commit 27e4c06d52a66978ec41a325468bdd3b756236ba Author: Alfredo Tupone <tupone@gentoo.org> AuthorDate: 2024-05-09 18:44:58 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-05-09 18:45:45 +0000 sci-libs/caffe2: filtering FLASH ATTENTION by USE flash Bug: https://bugs.gentoo.org/931046 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> .../caffe2/{caffe2-2.3.0-r2.ebuild => caffe2-2.3.0-r3.ebuild} | 8 ++++++-- sci-libs/caffe2/metadata.xml | 1 + 2 files changed, 7 insertions(+), 2 deletions(-)
Created attachment 892644 [details] -r3 build without flash I tried -DUSE_FLASH_ATTENTION=no on -r2 and it still eventually reaches a point where it tries to pull in aotriton again. I think they didn't wrap it correctly upstream. This log is an -r3 compile where the ebuild already has flash_attention turned off, it progresses further but eventually gets stuck again on the same dependency.
I have masked rocm on 2.3.0
hipblaslt is available for a while in ::gentoo sci-libs/hipBLASLt provides it.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02c9f5d0ade25b3117d965a070f172e52f76e7a5 commit 02c9f5d0ade25b3117d965a070f172e52f76e7a5 Author: Alfredo Tupone <tupone@gentoo.org> AuthorDate: 2024-07-22 16:49:11 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-07-22 16:50:53 +0000 profiles/base: enable rocm on caffe2 Bug: https://bugs.gentoo.org/931046 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> profiles/base/package.use.mask | 4 ---- 1 file changed, 4 deletions(-)
Yes, it was added on 2024-06-26. The build portion of this bug regarding hipblaslt is resolved. It still tries to pull in aotriton though, so this ebuild is still currently broken with rocm.
Created attachment 898149 [details] caffe2-2.3.0-r3 build log with sci-libs/hipBLASLt installed fatal: unable to access 'https://github.com/ROCm/aotriton.git/': Could not resolve host: github.com
Created attachment 898150 [details] MYCMAKEARGS="-DUSE_FLASH_ATTENTION=no" build.log sci-libs/hipBLASLt installed with MYCMAKEARGS="-DUSE_FLASH_ATTENTION=no"
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ca2e68ab5527af67bf95684068758b4efbd5b5a5 commit ca2e68ab5527af67bf95684068758b4efbd5b5a5 Author: Sv. Lockal <lockalsash@gmail.com> AuthorDate: 2024-07-25 09:27:07 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-07-27 19:08:42 +0000 sci-libs/caffe2: update dependencies to fix rocm flag pytorch 2.3.0 introduced 2 new direct dependencies: hipBLASLt and aotriton. pytorch uses hipBLASLt to perform gemm operation on datacenter AMD Instinct GPUs. For other GPUs pytorch fallbacks to hipBLAS. caffe2-2.3.x ebuilds now contain a patch to optionally disable this dependency, when none AMDGPU_TARGETS="gfx90a gfx940 gfx941 gfx942" is used. pytorch uses aotriton to perform FlashAttention operation. caffe2-2.3.x ebuilds now contain a patch which fully disables aotriton dependency, as there is no such package yet. Technically aotriton can be compiled (with minor patches), but I suggest to wait for next releases. It is a massive burden, as it depends on forked triton and forked clang (merge with upstream is not expected anytime soon). aotriton is usually distributed as a huge static (!) library (but in next release library will be shared). Minor fixes added for compatibility with libc++ (used in experimental llvm Gentoo profile), however other ebuilds also require minor patches (in other words: right now ROCm ecosystem can be compiled with libc++, but only by people with experience in C++). Closes: https://bugs.gentoo.org/931046 Signed-off-by: Sv. Lockal <lockalsash@gmail.com> Signed-off-by: Alfredo Tupone <tupone@gentoo.org> sci-libs/caffe2/caffe2-2.3.0-r3.ebuild | 45 ++-- sci-libs/caffe2/caffe2-2.3.1.ebuild | 46 ++-- .../files/caffe2-2.3.0-exclude-aotriton.patch | 35 +++ .../caffe2-2.3.0-fix-gcc-clang-abi-compat.patch | 17 ++ .../caffe2/files/caffe2-2.3.0-fix-libcpp.patch | 24 +++ .../files/caffe2-2.3.0-fix-rocm-gcc14-clamp.patch | 18 ++ .../files/caffe2-2.3.0-optional-hipblaslt.patch | 235 +++++++++++++++++++++ 7 files changed, 393 insertions(+), 27 deletions(-) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bc7200889449eff6eaa076219f1e3b6e5a130d95 commit bc7200889449eff6eaa076219f1e3b6e5a130d95 Author: Sv. Lockal <lockalsash@gmail.com> AuthorDate: 2024-07-27 18:44:26 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-07-27 19:08:47 +0000 profiles/base: remove mask for ">=sci-libs/caffe2-2.3.0 rocm" due to fix Bug: https://bugs.gentoo.org/931046 Signed-off-by: Sv. Lockal <lockalsash@gmail.com> Closes: https://github.com/gentoo/gentoo/pull/37751 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> profiles/base/package.use.mask | 4 ---- 1 file changed, 4 deletions(-)
