Bug 907965 – media-libs/mesa-23.1.1: chooses mismatched LLVM version, fails due to wrong version of dev-util/spirv-llvm-translator (Dependency LLVMSPIRVLib found: NO found 15.0.0.0 but need: '>= 16.0' ; matched: '>= 8.0.1.3', '< 16.1')

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Markus Giese 2023-06-07 00:03:11 UTC]

media-libs/mesa-23.1.1 fails due to wrong version of spirv-llvm-translator-15.0.0-r1, installing 16.x does not fix this

Reproducible: Always

Steps to Reproduce:
1.emerge sync
2.emerge mesa
3.
Actual Results:  
Dependency LLVMSPIRVLib found: NO found 15.0.0.0 but need: '>= 16.0' ; matched: '>= 8.0.1.3', '< 16.1'
Run-time dependency llvmspirvlib found: NO (tried cmake)

Expected Results:  
should compile fine

I uninstalled spirv-llvm-translator-15.0.0-r1 and installed 16.x but the error remains

[Comment 1 Markus Giese 2023-06-07 00:04:18 UTC]

Created attachment 863435 [details]
emerge --info

[Comment 2 Markus Giese 2023-06-07 00:09:36 UTC]

Created attachment 863436 [details]
build.log

[Comment 3 Markus Giese 2023-06-09 08:27:09 UTC]

I tried uninstalling dev-util/spirv-llvm-translator-15.0.0-r1 but as soon as I do this, when trying to compile mesa it pulls in spirv-llvm-translator-15.0.0-r1 (although 16.0.0-r1 is installed) again, so my guess is that this is a simple ebuild dependecy issue

[Comment 4 Matt Turner 2023-06-10 17:42:48 UTC]

Hm. What versions of llvm and spirv-llvm-translator do you have on your system?

The ebuild does this:

> PKG_CONFIG_PATH="$(get_llvm_prefix)/$(get_libdir)/pkgconfig"

which should be directing the build system to use only the version of LLVM that llvm_check_deps() says is okay.

Your build log shows

> --build.pkg-config-path /usr/lib/llvm/15/lib64/pkgconfig:/usr/share/pkgconfig
>       --pkg-config-path /usr/lib/llvm/15/lib64/pkgconfig:/usr/share/pkgconfig

So it looks like it's choosing llvm:15 and spirv-llvm-translator:15.

But then the build system chooses llvm 16...

> Run-time dependency LLVM (modules: LLVM) found: YES 16.0.5

[Comment 5 Matt Turner 2023-06-10 17:51:09 UTC]

Oh, I bet Mesa's build system detects llvm:16 and prefers it over llvm:15 because llvm is detected via `llvm-config` and not pkgconfig /o\

[Comment 6 Matt Turner 2023-06-11 02:31:12 UTC]

llvm@: I think we need to export LLVM_CONFIG or something in llvm.eclass to handle this?

[Comment 7 Markus Giese 2023-06-11 11:56:05 UTC]

ok, how do I set llvm-16.x as default ? I found out that on my system I only got sys-devel/llvm-toolchain-symlinks-15-r1 installed. Also there is only sys-devel/llvm-common-15.x installed. Do you think it would help to emerge their respective 16.x versions in this case?

[Comment 8 cyrillic 2023-06-11 23:31:39 UTC]

As long as you don't need multiple slots of llvm for some other reason, the easiest solution is to unmerge llvm:15 so that the build system will have no choice but to use llvm:16 for everything.

[Comment 9 Markus Giese 2023-06-11 23:49:11 UTC]

I tried that:

# emerge -a mesa

[ebuild  NS   ~] sys-devel/llvm-15.0.7-r3 [16.0.5] LLVM_TARGETS="(-LoongArch*)" 
[ebuild  N     ] sys-devel/clang-15.0.7-r1  USE="extra (pie) static-analyzer xml -debug -doc (-ieee-long-double) -test -verify-sig" ABI_X86="32 (64) (-x32)" LLVM_TARGETS="(AArch64) (AMDGPU) (ARM) (AVR) (BPF) (Hexagon) (Lanai) (MSP430) (Mips) (NVPTX) (PowerPC) (RISCV) (Sparc) (SystemZ) (VE) (WebAssembly) (X86) (XCore) (-ARC) (-CSKY) (-DirectX) (-LoongArch) (-M68k) (-SPIRV)" PYTHON_SINGLE_TARGET="python3_11 -python3_10" 
[ebuild  NS    ] dev-util/spirv-llvm-translator-15.0.0-r1 [16.0.0-r1]
[ebuild     U ~] media-libs/mesa-23.1.2 [23.0.3-r1]



Ideas ? I think I try package.mask for all those <16 packages

[Comment 10 Sam James 2023-06-11 23:54:34 UTC]

(In reply to Matt Turner from comment #6)
> llvm@: I think we need to export LLVM_CONFIG or something in llvm.eclass to
> handle this?

Yeah, I'm.. surprised we don't?

[Comment 11 cyrillic 2023-06-12 02:01:15 UTC]

(In reply to Markus Giese from comment #9)

> [ebuild  N     ] sys-devel/clang-15.0.7-r1

It looks like you are missing clang:16

[Comment 12 Tomasz Golinski 2023-06-12 09:05:26 UTC]

I have the same problem and I can't get rid of :15 packages. In my case there are a few packages requiring clang:15 explicitly, like thunderbird or ROCM framework.

[Comment 13 Sam James 2023-06-12 09:25:16 UTC]

Yes, that's a workaround, not a real fix. We're discussing a real fix (exporting LLVM_CONFIG in llvm.eclass).

[Comment 14 Markus Giese 2023-06-12 11:39:40 UTC]

use thunderbird-bin then. I masked <sys-devel/llvm-16 and <sys-devel/clang-16, removed all 15 packages and clion and goland which needed 15.x Now at least my world update run through... Thanks to all involved. And yeah it sure is a workaround and no fix. Nonetheless I am happy for now

[Comment 15 Sam James 2023-06-14 00:38:50 UTC]

Created attachment 863803 [details, diff]
llvm-eclass-export-LLVM_CONFIG.patch

[Comment 16 Larry the Git Cow 2023-06-15 21:47:28 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8e9c05ddfcd74be3b53c7c4c2fbc799a9ab7fa61

commit 8e9c05ddfcd74be3b53c7c4c2fbc799a9ab7fa61
Author:     Sam James <[e-mail]>
AuthorDate: 2023-06-14 00:39:07 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2023-06-15 21:47:19 +0000

    llvm.eclass: export LLVM_CONFIG if not cross-compiling
    
    Set LLVM_CONFIG to help Meson (bug #907965) but only do it for empty ESYSROOT
    (as a proxy for "are we cross-compiling?").
    
    Closes: https://bugs.gentoo.org/907965
    Signed-off-by: Sam James <[e-mail]>
    Closes: https://github.com/gentoo/gentoo/pull/31429
    Signed-off-by: Sam James <[e-mail]>

 eclass/llvm.eclass | 6 ++++++
 1 file changed, 6 insertions(+)

[Comment 17 Tomasz Golinski 2023-06-16 15:53:27 UTC]

The build still fails if there is no dev-util/spirv-llvm-translator matching the highest installed llvm version. 

I have llvm 15 and 16, but only spirv 15. Ebuild dependency check probably finds 15 as the correct slot, but meson doesn't know about it beforehand and first choses llvm 16 and fails to find spirv. It's obviously partially my fault for not adding that package to accepted_keywords, but maybe ebuild should try to check for this and warn before starting the configure stage.

Run-time dependency LLVM (modules: LLVM) found: YES 16.0.6
Run-time dependency spirv-tools found: YES 2023.2.1
Dependency LLVMSPIRVLib found: NO found 15.0.0.0 but need: '>= 16.0' ; matched: '>= 8.0.1.3', '< 16.1'
Run-time dependency llvmspirvlib found: NO (tried cmake)

../mesa-23.1.2/meson.build:1740:21: ERROR: Dependency lookup for LLVMSPIRVLib with method 'pkgconfig' failed: Invalid version, need 'LLVMSPIRVLib' ['>= 16.0'] found '15.0.0.0'.

 equery l llvm*
 * Searching for llvm* ...
[IP-] [  ] sys-devel/llvm-15.0.7-r3:15/15
[IP-] [  ] sys-devel/llvm-16.0.6:16/16
[...]

 * Searching for spir* ...
[IP-] [  ] dev-util/spirv-llvm-translator-15.0.0-r1:15
[...]

[Comment 18 Sam James 2023-06-16 20:31:16 UTC]

(In reply to Tomasz Golinski from comment #17)
> The build still fails if there is no dev-util/spirv-llvm-translator matching
> the highest installed llvm version. 
> 

Can you upload the full set of new logs then please?

[Comment 19 Tomasz Golinski 2023-06-16 22:11:46 UTC]

Created attachment 863983 [details]
build log

[Comment 20 Tomasz Golinski 2023-06-16 22:12:00 UTC]

Created attachment 863984 [details]
meson log

[Comment 21 Tomasz Golinski 2023-06-16 22:12:59 UTC]

Created attachment 863985 [details]
emerge --info

[Comment 22 Tomasz Golinski 2023-06-17 13:19:27 UTC]

I don't know if it is related but rustcl crashes for me and backtrace shows that mixed LLVM version libraries are in use, like here from Gimp:

[Switching to Thread 0x7fffacfc86c0 (LWP 20294)]
0x00007fff995d2130 in llvm::MachinePointerInfo::getAddrSpace() const ()
   from /usr/lib/llvm/16/lib64/libLLVM-16.so
(gdb) bt
#0  0x00007fff995d2130 in llvm::MachinePointerInfo::getAddrSpace() const ()
    at /usr/lib/llvm/16/lib64/libLLVM-16.so
#1  0x00007fff99a88d49 in  () at /usr/lib/llvm/16/lib64/libLLVM-16.so
#2  0x00007fff99a89234 in  () at /usr/lib/llvm/16/lib64/libLLVM-16.so
#3  0x00007ffe7da2f87f in llvm::FoldingSetBase::FindNodeOrInsertPos(llvm::FoldingSetNodeID const&, void*&, llvm::FoldingSetBase::FoldingSetInfo const&) ()
    at /usr/lib/llvm/15/lib64/libLLVM-15.so
#4  0x00007ffe7e3683c1 in llvm::SelectionDAG::FindNodeOrInsertPos(llvm::FoldingSetNodeID const&, llvm::SDLoc const&, void*&) () at /usr/lib/llvm/15/lib64/libLLVM-15.so
[...]

[Comment 23 Alan McKinnon 2023-09-26 11:40:41 UTC]

This bug is still occurring, I hit it today with mesa-23.2.0_rc4. But with updates etc it now finds v16 instead of v17. Are there any workarounds? This is a recent install, about a month old, and mesa has been successfully built 4 times already.





# emerge --info '=media-libs/mesa-23.2.0_rc4::gentoo'
Portage 3.0.51 (python 3.11.5-final-0, default/linux/amd64/17.1/desktop/plasma/systemd/merged-usr, gcc-13, glibc-2.38-r2, 6.1.46-gentoo-dist x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-6.1.46-gentoo-dist-x86_64-Intel-R-_Core-TM-_i7-10510U_CPU_@_1.80GHz-with-glibc2.38
KiB Mem:    16142908 total,   4962096 free
KiB Swap:    8388604 total,   7975164 free
Timestamp of repository gentoo: Tue, 26 Sep 2023 08:30:01 +0000
Head commit of repository gentoo: c6429f4ff13444ba2c9f09e42e5ad9e7e8a4a4d0
sh bash 5.2_p15-r6
ld GNU ld (Gentoo 2.41 p2) 2.41.0
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p15-r6::gentoo
dev-lang/perl:             5.38.0-r1::gentoo
dev-lang/python:           3.11.5::gentoo, 3.12.0_rc3::gentoo
dev-lang/rust-bin:         1.72.0::gentoo
dev-util/cmake:            3.27.6-r1::gentoo
dev-util/meson:            1.2.1-r2::gentoo
sys-apps/baselayout:       2.14::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-apps/systemd:          254.4::gentoo
sys-devel/autoconf:        2.13-r8::gentoo, 2.71-r7::gentoo
sys-devel/automake:        1.16.5-r1::gentoo
sys-devel/binutils:        2.41-r1::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           16.0.6::gentoo, 17.0.1::gentoo
sys-devel/gcc:             13.2.1_p20230826::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/lld:             16.0.6::gentoo
sys-devel/llvm:            16.0.6::gentoo, 17.0.1::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.5::gentoo (virtual/os-headers)
sys-libs/glibc:            2.38-r2::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    volatile: False
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-jobs: 1

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE @BINARY-REDISTRIBUTABLE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-march=native -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -O2 -pipe"
GENTOO_MIRRORS="https://gentoo.osuosl.org/     http://gentoo.osuosl.org/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi activities alsa amd64 bash-completion blas bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus declarative dri dts dvd dvdr encode exif ffmpeg flac fontconfig fortran gdbm gif git gpm graphicsmagick graphviz gui gzip iconv icu inotify ipv6 jpeg kde kwallet lame lapack lcms libcaca libnotify libtirpc lm-sensors lua lz4 lzma lzo mad magic matroska mng modules mp3 mp4 mpeg mplayer mtp multilib musicbrainz ncurses networkmanager nls nptl offensive ogg opengl openmp pam pango pcre pdf pipewire plasma png policykit ppds pulseaudio qml qt5 rar rdp readline screencast seccomp slang smp snmp sound spell ssl startup-notification svg systemd szip test-rust tiff truetype udev udisks unicode upower usb vdpau vim-syntax vorbis vulkan wayland webkit widgets wxwidgets x264 xattr xcb xcomposite xft xml xv xvid zip zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt rdrand sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

[Comment 24 Alan McKinnon 2023-09-26 11:42:50 UTC]

# emerge -pqv '=media-libs/mesa-23.2.0_rc4::gentoo'
[ebuild     U ] media-libs/mesa-23.2.0_rc4 [23.2.0_rc3] USE="X gles2 llvm lm-sensors proprietary-codecs vdpau vulkan wayland zstd -d3d9 -debug -gles1 -opencl -osmesa (-selinux) -test -unwind -vaapi -valgrind -vulkan-overlay -xa -zink" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="sse2" VIDEO_CARDS="intel nouveau radeon radeonsi -d3d12 (-freedreno) -lavapipe (-lima) (-panfrost) -r300 -r600 (-v3d) (-vc4) -virgl (-vivante) -vmware" 

 * IMPORTANT: 4 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.

[Comment 25 Alan McKinnon 2023-09-26 11:46:56 UTC]

Created attachment 871477 [details]
build log

[Comment 26 Alan McKinnon 2023-09-26 11:49:40 UTC]

Created attachment 871478 [details]
ebuild environment

[Comment 27 Sam James 2023-09-26 18:32:49 UTC]

*** Bug 914632 has been marked as a duplicate of this bug. ***

[Comment 28 Sam James 2023-09-26 18:33:04 UTC]

*** Bug 914742 has been marked as a duplicate of this bug. ***

[Comment 29 Sam James 2023-09-26 18:34:00 UTC]

Okay, this is still happening with newer Mesa, despite the fixes in bug 914632.

We need to stop Meson from picking a mismatched version of LLVM, see Eli's comment at https://bugs.gentoo.org/914632#c30.

[Comment 30 Sam James 2023-09-26 18:42:21 UTC]

*** Bug 914742 has been marked as a duplicate of this bug. ***

[Comment 31 Gary E. Miller 2023-09-26 18:49:08 UTC]

My issue mutated to end up here.  So: #MeToo

[Comment 32 Sam James 2023-09-27 01:40:29 UTC]

*** Bug 914775 has been marked as a duplicate of this bug. ***

[Comment 33 Larry the Git Cow 2023-09-27 01:52:19 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=90da97e78ade65753dfb2e0fb30ec9067c5aa766

commit 90da97e78ade65753dfb2e0fb30ec9067c5aa766
Author:     Sam James <[e-mail]>
AuthorDate: 2023-09-27 01:49:30 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2023-09-27 01:51:44 +0000

    media-libs/mesa: fix LLVM detection
    
    Meson will prefer looking up LLVM via CMake to llvm-config and it's non-trivial
    to try coerce CMake into looking at the right thing, so we end up trying to use
    the wrong LLVM and exploding - fortunately just at configure time.
    
    Add a patch to force using llvm-config - which llvm.eclass controls the availability
    of - and avoid the CMake problem entirely.
    
    Of course, it'd be far better if LLVM just finally adopted pkg-config.
    
    Bug: https://bugs.gentoo.org/914632
    Bug: https://bugs.gentoo.org/914742
    Bug: https://bugs.gentoo.org/914775
    Closes: https://bugs.gentoo.org/907965
    Thanks-to: Eli Schwartz <[e-mail]>
    Signed-off-by: Sam James <[e-mail]>

 media-libs/mesa/files/clang_config_tool.patch | 13 +++++++++++++
 media-libs/mesa/mesa-23.1.8.ebuild            |  3 +++
 media-libs/mesa/mesa-23.2.0_rc4.ebuild        |  5 +++++
 media-libs/mesa/mesa-9999.ebuild              |  5 +++++
 4 files changed, 26 insertions(+)

[Comment 34 Sam James 2023-09-29 05:35:42 UTC]

*** Bug 914903 has been marked as a duplicate of this bug. ***