Anki application crashes on start here: ================================================================================= peter@x1 ~ $ anki Preparing to run... Qt warning: QGuiApplication::setDesktopFileName: the specified desktop file name ends with .desktop. For compatibility reasons, the .desktop suffix will be removed. Please specify a desktop file name without .desktop suffix Qt warning: Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8. Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead. If this causes problems, reconfigure your locale. See the locale(1) manual for more information. 2024-07-02 13:39:44,783:INFO:aqt.mediasrv: Serving on http://127.0.0.1:34755 Segmentation fault (core dumped) ================================================================================= I've managed to gather backtrace and latest function calls are the following: (gdb) bt #0 0x00007ffff41606c0 in ZSTD_freeDDict () at /usr/lib/python3.10/site-packages/zstandard/_cffi.cpython-310-x86_64-linux-gnu.so #1 0x00007ffff4166bbb in ZSTD_decompressDCtx () at /usr/lib/python3.10/site-packages/zstandard/_cffi.cpython-310-x86_64-linux-gnu.so #2 0x00007ffff404513b in ZSTD_decompress () at /usr/lib64/libzstd.so.1 (Full backtrace is here: https://github.com/indygreg/python-zstandard/issues/232) What makes me wonder, why from ZSTD_decompress () from zstd library (/usr/lib64/libzstd.so.1) uses ZSTD_decompressDCtx() from zstandard python bindings? I expect it to use ZSTD_decompressDCtx() from the same library that is defined in libzstd.so. So this looks like an error. I've started to investigate what may cause this and found the following sed in zstandard-0.22.0.ebuild: # the C backend is repeatedly broken, so force CFFI instead sed -e '/PYTHON_ZSTANDARD_IMPORT_POLICY/s:default:cffi:' \ -i zstandard/__init__.py || die I've checked zstandard code and found, that I can use CPython backend. This allowed me to find workaround: if I export PYTHON_ZSTANDARD_IMPORT_POLICY="CPython" anki binary starts to work. So now the problem here is that we should fix cffi import policy somehow. Or may be easiest solution will be to switch back to default PYTHON_ZSTANDARD_IMPORT_POLICY? ================================================================================= # emerge --info zstandard Portage 3.0.65 (python 3.11.9-final-0, default/linux/amd64/23.0/split-usr/desktop, gcc-14, glibc-2.39-r6, 6.9.6-gentoo-dist x86_64) ================================================================= System Settings ================================================================= System uname: Linux-6.9.6-gentoo-dist-x86_64-Intel-R-_Core-TM-_i7-10510U_CPU_@_1.80GHz-with-glibc2.39 KiB Mem: 16021032 total, 1455932 free KiB Swap: 24786936 total, 21355860 free Timestamp of repository gentoo: Mon, 01 Jul 2024 15:45:00 +0000 Head commit of repository gentoo: e0a589377c4b46eb8f4941e4075f40044a0634b3 Timestamp of repository guru: Mon, 01 Jul 2024 15:33:56 +0000 Head commit of repository guru: 4a0cedb6c31274186574bb7d6e69d9df1cb0d22e Head commit of repository pouch: a7e7210b9de87efcb1b999db73d771231d99388e Head commit of repository sheep-box: 1cbafd32048d51977767b73af2fc6b5d8ad3494f sh bash 5.2_p26-r6 ld GNU ld (Gentoo 2.42 p6) 2.42.0 ccache version 4.10.1 [disabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p26-r6::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.71-r7::gentoo dev-build/automake: 1.16.5-r2::gentoo dev-build/cmake: 3.28.5::gentoo dev-build/libtool: 2.4.7-r4::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.4.1::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r3::gentoo dev-lang/python: 3.10.14_p1-r1::gentoo, 3.11.9-r1::gentoo, 3.12.3-r1::gentoo dev-lang/rust: 1.79.0::gentoo dev-util/ccache: 4.10.1::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/sandbox: 2.38::gentoo sys-apps/systemd: 254.13::gentoo sys-devel/binutils: 2.42-r2::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 17.0.6::gentoo, 18.1.8::gentoo sys-devel/gcc: 14.1.1_p20240622::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 17.0.6::gentoo, 18.1.8::gentoo sys-devel/llvm: 17.0.6::gentoo, 18.1.8::gentoo sys-kernel/linux-headers: 6.9::gentoo (virtual/os-headers) sys-libs/glibc: 2.39-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: ssh://185.198.112.84/var/db/repos/gentoo priority: -1000 volatile: True sync-rsync-verify-max-age: 3 sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes guru location: /var/db/repos/guru sync-type: git sync-uri: https://github.com/gentoo-mirror/guru.git masters: gentoo volatile: False pouch location: /portage/overlays/pouch sync-type: git sync-uri: https://github.com/pva/pouch masters: gentoo volatile: True sheep-box location: /usr/local/sheep-box sync-type: git sync-uri: ssh://git@dev.tightvideo.com:22222/sheep-box.git masters: gentoo volatile: True ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -pipe -march=native -mtune=native -ggdb" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /opt/cxoffice/etc/cxoffice.conf /usr/lib64/libreoffice/program/sofficerc /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.9/conf" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -pipe -march=native -mtune=native -ggdb" DISTDIR="/usr/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --jobs=10 --rebuilt-binaries=y --autounmask=n -gk --binpkg-changed-deps=n" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg buildpkg-live candy compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms sign splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="ru_RU.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs" LEX="flex" LINGUAS="ru en en_GB en_US" MAKEOPTS="-j1" PKGDIR="/var/cache/binpkgs" PORTAGE_BINHOST="ssh://binpkguser@185.198.112.84:22203/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp/" SHELL="/bin/bash" USE="X a52 aac acl acpi alsa amd64 amr archive audit avif blake2 bluetooth bluray bpf branding btrfs bzip2 cairo caps cdda cddb cdr cet chromaprint colord crypt cryptsetup cups curl dav1d dbus device-mapper dga dist-kernel djvu dnssec dri dri3 drm dts dv dvd dvdr dvi eds egl enchant encode evdev exif fam fdk ffmpeg fftw flac fluidsynth fontconfig fprint gdbm gdk-pixbuf gif gimp glib gnome-keyring gnome-online-accounts google googledrive gpg gphoto2 graphviz gsm gstreamer gtk gtk3 gui harfbuzz heif highlight http2 iconv icu id3tag idn imagemagick introspection ios ipmi iproute2 ipv6 jbig jit jpeg jpeg2k kf6compat kms latex lcms libass libnotify libtirpc lm-sensors lto lvm lzma lzo mad matroska midi mng modemmanager modplug mp3 mp4 mpeg mtp multilib musepack nautilus ncurses netlink networkauth networkmanager nls nsplugin ocr ofx ogg opengl openmp opus osc pam pango pcap pcre pdf pgo pipewire plotutils png policykit postscript ppds pulseaudio python qml qt6 rav1e raw rdp readline scanner sdl seccomp smi snappy sound speex spell spice split-usr sqlite srt ssl startup-notification stemmer svg svt-av1 systemd t1lib taglib test-rust theora threads tiff tracker truetype twolame udev udisks unicode upower urandom usb user-session vaapi vcd vim-syntax visio vlc vorbis vpx vulkan wavpack wayland webchannel webkit webp widgets wifi wmf wxwidgets x264 x265 xattr xcb xetex xft xkb xml xmp xpm xps xv xvid zlib zstd" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="evdev" KERNEL="linux" L10N="ru en en_GB en_US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LIBREOFFICE_EXTENSIONS="scripting-javascript wiki-publisher" LLVM_SLOT="18" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10 python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS ================================================================= Package Settings ================================================================= dev-python/zstandard-0.22.0::gentoo was built with the following: USE="-debug -test" ABI_X86="(64)" PYTHON_TARGETS="python3_10 python3_11 -pypy3 -python3_12 -python3_13" CFLAGS="-O2 -pipe -march=haswell -mtune=skylake" CXXFLAGS="-O2 -pipe -march=haswell -mtune=skylake" =================================================================================
Can you make an independent reproducer? Presumably it happens when calling zstandard from pure Python, correct?
I've tried, but unfortunately, I failed to find a way to call ZSTD_decompress() in a way it is done from anki bin. Note, that this function is called from /usr/lib64/libQt6Core.so.6, #3 0x00007ffff32f1af2 in QResourcePrivate::decompress(char*, long long) const () at /usr/lib64/libQt6Core.so.6 Note, there is no zstd_decompress function in exported in dev-python/zstandard. So, to reproduce this issue I need somehow to: 1. build binary that will calls ZSTD_decompress() (that's easy, zstd has examples) 2. to link this binary somehow with cffi. And here I don't understand how to do that... BTW, I found that if I build mesa with "zstd" backtrace will be different: 0x00007ffff41606c0 in ZSTD_freeDDict () from /usr/lib/python3.10/site-packages/zstandard/_cffi.cpython-310-x86_64-linux-gnu.so (gdb) bt #0 0x00007ffff41606c0 in ZSTD_freeDDict () at /usr/lib/python3.10/site-packages/zstandard/_cffi.cpython-310-x86_64-linux-gnu.so #1 0x00007ffff4166bbb in ZSTD_decompressDCtx () at /usr/lib/python3.10/site-packages/zstandard/_cffi.cpython-310-x86_64-linux-gnu.so #2 0x00007ffff404513b in ZSTD_decompress () at /usr/lib64/libzstd.so.1 #3 0x00007fffdda83f2f in util_compress_inflate () at /usr/lib64/dri/iris_dri.so #4 0x00007fffdda5ce71 in parse_and_validate_cache_item () at /usr/lib64/dri/iris_dri.so #5 0x00007fffdda5d5de in disk_cache_load_item () at /usr/lib64/dri/iris_dri.so #6 0x00007fffdda5c6ae in disk_cache_get () at /usr/lib64/dri/iris_dri.so #7 0x00007fffde1838cb in iris_disk_cache_retrieve () at /usr/lib64/dri/iris_dri.so #8 0x00007fffde18ebf4 in iris_create_shader_state () at /usr/lib64/dri/iris_dri.so #9 0x00007fffddb089be in st_create_fp_variant () at /usr/lib64/dri/iris_dri.so #10 0x00007fffddb090a6 in st_get_fp_variant () at /usr/lib64/dri/iris_dri.so #11 0x00007fffddb0959b in st_finalize_program () at /usr/lib64/dri/iris_dri.so #12 0x00007fffddb09772 in st_program_string_notify () at /usr/lib64/dri/iris_dri.so #13 0x00007fffddc871bc in _mesa_get_fixed_func_fragment_program () at /usr/lib64/dri/iris_dri.so #14 0x00007fffddaa1938 in update_program () at /usr/lib64/dri/iris_dri.so #15 0x00007fffddaa1f50 in _mesa_update_state_locked () at /usr/lib64/dri/iris_dri.so #16 0x00007fffddaa21b5 in _mesa_update_state () at /usr/lib64/dri/iris_dri.so #17 0x00007fffddce1eec in find_value () at /usr/lib64/dri/iris_dri.so #18 0x00007fffddce3665 in _mesa_GetBooleanv () at /usr/lib64/dri/iris_dri.so #19 0x00007ffff0255d10 in qt_gl_resolve_extensions() () at /usr/lib64/libQt6Gui.so.6 #20 0x00007ffff0256739 in QOpenGLExtensions::hasOpenGLExtension(QOpenGLExtensions::OpenGLExtension) const () at /usr/lib64/libQt6Gui.so.6 #21 0x00007ffff026b45e in QRhiGles2::create(QFlags<QRhi::Flag>) () at /usr/lib64/libQt6Gui.so.6 #22 0x00007ffff01144b7 in QRhi::create(QRhi::Implementation, QRhiInitParams*, QFlags<QRhi::Flag>, QRhiNativeHandles*) () at /usr/lib64/libQt6Gui.so.6 #23 0x00007fffe3c1ebf0 in QtWebEngineCore::GPUInfo::GPUInfo() () at /usr/lib64/libQt6WebEngineCore.so.6 #24 0x00007fffffffa710 in ??? () #25 0xffffffffffffff88 in ??? () #26 0x0000000000000002 in ??? () #27 0x0000555555e93f40 in ??? () #28 0x00005555568190a0 in ??? () #29 0x0000000000000000 in ??? () Here ZSTD_decompress() is called from some function in /usr/lib64/dri/iris_dri.so. But still crash is the same.
Ah, so the problem is that dev-python/zstandard exports symbols colliding with symbols from libzstd itself?
As far as I understand yes. zstandard exports symbols from zstd using CFFI, and somehow it exports them with higher priority. Thus, we have two ZSTD_decompressDCtx() functions - from CFFI and zstd library. Now, instead of calling ZSTD_decompressDCtx() function from zstd, ZSTD_decompress()-call inside zstd library calls ZSTD_decompressDCtx() from CFFI and crashs there.
