Summary: | sci-libs/vtk-9.0.1: Error detecting architecture flags for CUDA | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Adrian <agrigo2001> |
Component: | Current packages | Assignee: | Gentoo Science Related Packages <sci> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | ionen, proxy-maint, waebbl-gentoo |
Priority: | Normal | Keywords: | PullRequest |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://github.com/gentoo/gentoo/pull/21855 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
Allow the user to specify their CUDA architecture
Build log for vtk-9.0.1 CMakeCache.txt |
Description
Adrian
2021-07-24 11:23:09 UTC
Created attachment 726430 [details, diff]
Allow the user to specify their CUDA architecture
If VTKm_CUDA_Architecture is set in the environment, pass it to cmake, where it will force compilation for the given architecture. Otherwise use the default value native which autodetects the capabilities of the build system.
Please post both emerge --info and full build log Portage 3.0.20 (python 3.9.5-final-0, default/linux/amd64/17.1/desktop/gnome/systemd, gcc-10.3.0, glibc-2.33-r1, 5.10.27-gentoo x86_64) ================================================================= System uname: Linux-5.10.27-gentoo-x86_64-AMD_Ryzen_7_1800X_Eight-Core_Processor-with-glibc2.33 KiB Mem: 16349500 total, 5624672 free KiB Swap: 20971516 total, 15744988 free Timestamp of repository gentoo: Fri, 16 Jul 2021 08:15:01 +0000 Head commit of repository gentoo: 81d144f27fe0064edc2cc5778cd00843e3588425 Head commit of repository dotnet: 6e75dacad0e99680e84049b1c7f6b40887cc8e29 Head commit of repository ebuild-overlay: c462c95cb8c62354f8536fb94edb74aa1dec4924 Head commit of repository sakaki-tools: 571e34870d665e98677caecbd56cf0d399ee4b3c sh bash 5.1_p8 ld GNU ld (Gentoo 2.35.2 p1) 2.35.2 app-shells/bash: 5.1_p8::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.32.1::gentoo dev-lang/python: 2.7.18_p10::gentoo, 3.8.10_p2::gentoo, 3.9.5_p2::gentoo dev-lang/rust: 1.52.1::gentoo dev-util/cmake: 3.18.5::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/sandbox: 2.24::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.16.3-r1::gentoo sys-devel/binutils: 2.35.2::gentoo sys-devel/gcc: 10.3.0-r1::gentoo sys-devel/gcc-config: 2.4::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers) sys-libs/glibc: 2.33-r1::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.au.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 sync-rsync-verify-metamanifest: yes crossdev location: /usr/local/portage/crossdev masters: gentoo priority: 10 dotnet location: /usr/local/portage/dotnet sync-type: git sync-uri: https://github.com/gentoo/dotnet.git masters: gentoo priority: 50 ebuild-overlay location: /usr/local/portage/ebuild-overlay sync-type: git sync-uri: https://github.com/redchillipadi/ebuild-overlay.git masters: gentoo priority: 50 sakaki-tools location: /usr/local/portage/sakaki-tools sync-type: git sync-uri: https://github.com/sakaki-/sakaki-tools.git masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE CC-Sampling-Plus-1.0" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=znver1 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=znver1 -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--jobs 8 --load-average 8.0" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.swin.edu.au/gentoo" LANG="en_AU.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/home/portage" USE="X a52 aac acl acpi alsa amd64 bluetooth branding bzip2 cairo cdda cdr cli colord crypt cuda cups dbus dri dts dvd dvdr eds emboss encode evo exif flac fortran gdbm gif gnome gnome-keyring gnome-online-accounts gpm gstreamer gtk gui iconv icu introspection ipv6 jpeg lcms libglvnd libnotify libsecret libtirpc mad mng mp3 mp4 mpeg multilib nautilus ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds pulseaudio qt5 readline sdl seccomp spell split-usr ssl startup-notification svg sysprof systemd tcpd tiff truetype udev udisks unicode upower usb vorbis wayland wxwidgets x264 xattr xcb xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB en-AU" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9 python3_8" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS Created attachment 726664 [details]
Build log for vtk-9.0.1
emerge -pqv '=sci-libs/vtk-9.0.1::gentoo' [ebuild R ] sci-libs/vtk-9.0.1 [9.0.1] USE="X cuda json openmp qt5 rendering -all-modules -boost -doc -examples -ffmpeg -gdal -imaging -java -kits -mpi -mysql -odbc -offscreen -pegtl -postgres -python -tbb -test -theora -tk -views -web" PYTHON_SINGLE_TARGET="python3_9 -python3_8" VIDEO_CARDS="nvidia" Hi Adrian, could you please attach the CMakeCache.txt file as well? I had the issue of not finding the native GPU as well one or two times during my tests, but couldn't reproduce it to get to root of its cause. Using an env variable seems a little fragile to me, as you have already pointed out. I'm unsure whether it is a good move for solving this issue. IMO a more stable approach would be, to improve the cuda.eclass to deliver such configuration options to the ebuilds and further down to the build system used. Also note, there's a NVCCFLAGS variable defined in cuda.eclass which we could use for this purpose, instead of defining a new env var. Created attachment 727017 [details]
CMakeCache.txt
My NVCCFLAGS are -O2 -v --compiler-bindir "/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0" --compiler-options "-march=znver1 -O2 -pipe" --linker-options "-O1,--as-needed" -v --compiler-bindir "/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0" --compiler-options "-march=znver1 -O2 -pipe" --linker-options "-O1,--as-needed" When I run at the command line: nvcc --run ThirdParty/vtkm/vtkvtkm/vtk-m/CMake/VTKmDetectCUDAVersion.cu it returns the appropriate --generate-code=arch=compute_61,code=sm_61; however the command in VTKmDeviceAdapters.cmake returns a non zero (failure) code the cmake file is trying to run /opt/cuda/bin/nvcc --run;/home/portage/portage/sci-libs/vtk-9.0.1/work/VTK-9.0.1/ThirdParty/vtkm/vtkvtkm/vtk-m/CMake/VTKmDetectCUDAVersion.cu I think the problem is the extra ; after --run But to allow emerging on a build system, working towards allowing the ebuild to specify the desired GPU architecure to cmake is still the best fix. (In reply to Adrian from comment #11) > the cmake file is trying to run /opt/cuda/bin/nvcc > --run;/home/portage/portage/sci-libs/vtk-9.0.1/work/VTK-9.0.1/ThirdParty/ > vtkm/vtkvtkm/vtk-m/CMake/VTKmDetectCUDAVersion.cu > > I think the problem is the extra ; after --run > > But to allow emerging on a build system, working towards allowing the ebuild > to specify the desired GPU architecure to cmake is still the best fix. This looked like a good catch, but replacing the ';' with a space didn't solve it, but instead gives a fatal unknown parameter error when calling nvcc. I addressed the issue in the ebuild, but still think, it should be solved within the cuda.eclass, possibly by adding a USE_EXPAND value. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=582031cbf749c6a79ba6b0b6f09b6754857c5915 commit 582031cbf749c6a79ba6b0b6f09b6754857c5915 Author: Bernd Waibel <waebbl-gentoo@posteo.net> AuthorDate: 2021-08-01 14:56:15 +0000 Commit: Ionen Wolkens <ionen@gentoo.org> CommitDate: 2021-08-02 20:49:46 +0000 sci-libs/vtk: fix auto-detection of CUDA arch In some cases, the CUDA architecture is not successfully detected, when 'native' is passed to the build system. For this, check for the environment variable VTK_CUDA_ARCH to be exported and pass that value from the environment instead, if it's a valid CUDA architecture name. This ignores the 'fermi' architecture, as it's no longer supported by current dev-util/nvidia-cuda-toolkit-11* versions. Closes: https://bugs.gentoo.org/803818 Package-Manager: Portage-3.0.20, Repoman-3.0.3 Signed-off-by: Bernd Waibel <waebbl-gentoo@posteo.net> Closes: https://github.com/gentoo/gentoo/pull/21855 Signed-off-by: Ionen Wolkens <ionen@gentoo.org> sci-libs/vtk/vtk-9.0.1.ebuild | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) |