Created attachment 859616 [details] cryptography-40.0.1:20230406-115120.log.gz >>> Test phase: dev-python/cryptography-40.0.1 * python3_10: running distutils-r1_run_phase python_test python3.10 -m pytest -vv -ra -l -Wdefault --color=yes -o console_output_style=count -p no:cov -p no:flake8 -p no:flakes -p no:pylint -p no:markdown -p no:sugar -p no:xvfb -p no:tavern --ignore tests/bench -n 16 ImportError while loading conftest '/var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1/tests/conftest.py'. tests/conftest.py:9: in <module> from cryptography.hazmat.backends.openssl import backend as openssl_backend ../cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/__init__.py:6: in <module> from cryptography.hazmat.backends.openssl.backend import backend ../cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py:13: in <module> from cryptography import utils, x509 ../cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/x509/__init__.py:6: in <module> from cryptography.x509 import certificate_transparency ../cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/x509/certificate_transparency.py:10: in <module> from cryptography.hazmat.bindings._rust import x509 as rust_x509 E ImportError: /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/bindings/_rust.abi3.so: undefined symbol: PyInit__openssl * ERROR: dev-python/cryptography-40.0.1::gentoo failed (test phase) # emerge --info =dev-python/cryptography-40.0.1 Portage 3.0.44 (python 3.10.10-final-0, default/linux/amd64/17.1/desktop/plasma/systemd/merged-usr, gcc-12, glibc-2.36-r7, 6.1.19-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-6.1.19-gentoo-x86_64-AMD_Ryzen_7_5700G_with_Radeon_Graphics-with-glibc2.36 KiB Mem: 65125968 total, 49175164 free KiB Swap: 67108860 total, 66598652 free Timestamp of repository gentoo: Thu, 06 Apr 2023 08:45:01 +0000 Head commit of repository gentoo: 07bfee0fdcf63a416031055e30785a16475f9e89 sh dash 0.5.12 ld GNU ld (Gentoo 2.39 p5) 2.39.0 distcc 3.4 x86_64-pc-linux-gnu [disabled] ccache version 4.7.4 [disabled] app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r2::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.36.0-r2::gentoo dev-lang/python: 3.10.10_p3::gentoo, 3.11.2_p2::gentoo dev-lang/rust: 1.66.1::gentoo dev-util/ccache: 4.7.4::gentoo dev-util/cmake: 3.25.3::gentoo dev-util/meson: 1.0.1::gentoo sys-apps/baselayout: 2.13-r1::gentoo sys-apps/sandbox: 2.29::gentoo sys-apps/systemd: 252.7::gentoo sys-devel/autoconf: 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/clang: 15.0.7-r1::gentoo sys-devel/gcc: 12.2.1_p20230121-r1::gentoo sys-devel/gcc-config: 2.8::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/lld: 15.0.7::gentoo sys-devel/llvm: 15.0.7::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r7::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage priority: -1000 volatile: True sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes x-portage location: /usr/local/portage masters: gentoo priority: 0 volatile: True ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" AR="/usr/bin/gcc-ar" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -frecord-gcc-switches -march=znver3 -ftree-vectorize -flto=8" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d" CXXFLAGS="-O2 -pipe -frecord-gcc-switches -march=znver3 -ftree-vectorize -flto=8" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--keep-going y --with-bdeps y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe -frecord-gcc-switches -march=znver3 -ftree-vectorize -flto=8" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live cgroup compress-build-logs config-protect-if-modified distlocks ebuild-locks ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms sign split-elog split-log strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -frecord-gcc-switches -march=znver3 -ftree-vectorize -flto=8" GENTOO_MIRRORS="http://mirror.init7.net/gentoo/ https://mirror.init7.net/gentoo/ http://ftp.uni-hannover.de/gentoo/ http://mirror.eu.oneandone.net/linux/distributions/gentoo/gentoo/" LANG="it_IT.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--sort-common -ftree-vectorize -flto=8" LEX="flex" LINGUAS="it it_IT" MAKEOPTS="-j16 -l16" NM="/usr/bin/gcc-nm" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" RANLIB="/usr/bin/gcc-ranlib" SHELL="/bin/bash" USE="X a52 aac aalib acl acpi activities adns alsa amd64 ao audiofile bash-completion bluetooth branding brotli bzip2 cairo caps cdda cddb cdparanoia cdr cli crypt css cups curl dbus declarative dga djvu dri dts dvd dvdr encode exif expat fbcon ffmpeg fftw flac fontconfig foomaticdb fortran ftp gd gdbm geoip gif gimp gmp gnutls gphoto2 gpm graphviz gstreamer gtk gui guile handbook iconv icu idn imagemagick imlib introspection ipv6 java javascript jbig jemalloc jpeg jpeg2k kde kwallet lame lcms libass libglvnd libnotify libsamplerate libtirpc lm-sensors lua lz4 lzma lzo mad magic mhash mmap mng mp3 mp4 mpeg multilib musicbrainz ncurses nls nptl offensive ogg openal opengl openmp opus pam pango pcre pdf plasma png policykit postscript ppds pulseaudio qml qt5 rdesktop readline recode samba screencast sctp sdl seccomp sndfile sockets sound speex spell sqlite ssl startup-notification svg symlink syslog systemd sysvipc taglib telemetry test test-rust theora threads tidy tiff truetype udev udisks unicode upower usb v4l vaapi vala verify-sig vim-syntax vnc vorbis wavpack wayland webp widgets win32codecs wmf wxwidgets x264 xattr xcb xft xinerama xml xpm xscreensaver xv xvid yahoo zip zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2021" ALSA_CARDS="hda-intel virmidi" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" CURL_SSL="gnutls" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="it en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AMDGPU" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby31" USERLAND="GNU" VIDEO_CARDS="amdgpu radeon radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS ================================================================= Package Settings ================================================================= dev-python/cryptography-40.0.1::gentoo was built with the following: USE="-debug -test" ABI_X86="(64)" PYTHON_TARGETS="python3_10 (-pypy3) -python3_11 -python3_9" FEATURES="split-elog parallel-fetch unmerge-logs unknown-features-warn strict xattr multilib-strict binpkg-logs unmerge-orphans buildpkg-live protect-owned userfetch config-protect-if-modified compress-build-logs usersync binpkg-docompress split-log binpkg-dostrip network-sandbox preserve-libs news sfperms sign usersandbox userpriv qa-unresolved-soname-deps cgroup distlocks merge-sync assume-digests pid-sandbox ipc-sandbox ebuild-locks sandbox" # emerge -1pqv =dev-python/cryptography-40.0.1 [ebuild R ] dev-python/cryptography-40.0.1 USE="test* -debug" PYTHON_TARGETS="python3_10 (-pypy3) -python3_9 -python3_11"
*** Bug 903915 has been marked as a duplicate of this bug. ***
readelf -a /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/bindings/_rust.abi3.so | grep PyInit
Could it be due to this: /usr/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'cryptography.hazmat.bindings._rust' as data is deprecated, please list it in `packages`. !! ############################ # Package would be ignored # ############################ Python recognizes 'cryptography.hazmat.bindings._rust' as an importable package, but it is not listed in the `packages` configuration of setuptools. 'cryptography.hazmat.bindings._rust' has been automatically added to the distribution only because it may contain data files, but this behavior is likely to change in future versions of setuptools (and therefore is considered deprecated). Please make sure that 'cryptography.hazmat.bindings._rust' is included as a package by using the `packages` configuration field or the proper discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" and "data files" on setuptools documentation page. !! check.warn(importable) /usr/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'cryptography.hazmat.bindings._rust.openssl' as data is deprecated, please list it in `packages`. !! ############################ # Package would be ignored # ############################ Python recognizes 'cryptography.hazmat.bindings._rust.openssl' as an importable package, but it is not listed in the `packages` configuration of setuptools. 'cryptography.hazmat.bindings._rust.openssl' has been automatically added to the distribution only because it may contain data files, but this behavior is likely to change in future versions of setuptools (and therefore is considered deprecated). Please make sure that 'cryptography.hazmat.bindings._rust.openssl' is included as a package by using the `packages` configuration field or the proper discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" and "data files" on setuptools documentation page. !! check.warn(importable) Now I have plenty of python modules failing with /usr/lib/python3.10/site-packages/cryptography/hazmat/bindings/_rust.abi3.so: undefined symbol: PyInit__openssl
You didn't answer my question.
(In reply to Michał Górny from comment #4) > You didn't answer my question. Which question?
Oh, I see. Sorry, I didn't undestand it was a question. Here it is: # readelf -a /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/bindings/_rust.abi3.so | grep PyInit 0000001d3780 003e00000006 R_X86_64_GLOB_DAT 0000000000000000 PyInit__openssl + 0 62: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND PyInit__openssl 168: 00000000000b43a0 526 FUNC GLOBAL DEFAULT 15 PyInit__rust 5461: 00000000000b43a0 526 FUNC GLOBAL DEFAULT 15 PyInit__rust 5519: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND PyInit__openssl
Now this is weird. I only have: 732: 0000000000098ae0 526 FUNC GLOBAL DEFAULT 11 PyInit__rust Feel like reporting it upstream? I'm afraid it's beyond my skills to figure out what Rust is doing here.
(In reply to Michał Górny from comment #7) > Feel like reporting it upstream? I'm afraid it's beyond my skills to figure > out what Rust is doing here. If it's beyond your skill, it's certainly beyond mine as well, but I'll try...
(In reply to Michał Górny from comment #7) > Now this is weird. I only have: > > 732: 0000000000098ae0 526 FUNC GLOBAL DEFAULT 11 PyInit__rust > > Feel like reporting it upstream? I'm afraid it's beyond my skills to figure > out what Rust is doing here. Which version of openssl do you have? I'm trying openssl-3 ATM. Could it be the culprit?
Let me rephrase: it's beyond my skill to help you, and I can't reproduce it myself. However, there's a good chance that upstream (or setuptools-rust people, or PyO3 people) will be able to figure it out — it's just a matter that it would probably be more efficient if I wouldn't have to proxy between them and you ;-).
(In reply to Paolo Pedroni from comment #9) > Which version of openssl do you have? I'm trying openssl-3 ATM. Could it be > the culprit? Using openssl-3 for as long as I remember ;-). Did you rebuild everything else against it?
(In reply to Michał Górny from comment #11) > (In reply to Paolo Pedroni from comment #9) > > Which version of openssl do you have? I'm trying openssl-3 ATM. Could it be > > the culprit? > > Using openssl-3 for as long as I remember ;-). Did you rebuild everything > else against it? I'm quite sure I made at least an 'emerge -1e @world' since switching to openssl-3.
(In reply to Michał Górny from comment #10) > Let me rephrase: it's beyond my skill to help you, and I can't reproduce it > myself. However, there's a good chance that upstream (or setuptools-rust > people, or PyO3 people) will be able to figure it out — it's just a matter > that it would probably be more efficient if I wouldn't have to proxy between > them and you ;-). It's perfectly clear. I'm working on that.
There's a chance this is related to LTO. Could you rebuild the relevant packages here just to rule that out?
(In reply to Sam James from comment #14) > There's a chance this is related to LTO. Could you rebuild the relevant > packages here just to rule that out? Good idea. Which packages do you mean?
I tried compiling dev-lang/rust and dev-libs/openssl without LTO, without any discernible effect. If I try compiling dev-python/cryptography itself without LTO, the error message changes and becomes: >>> Test phase: dev-python/cryptography-40.0.1 * python3_10: running distutils-r1_run_phase python_test python3.10 -m pytest -vv -ra -l -Wdefault --color=yes -o console_output_style=count -p no:cov -p no:flake8 -p no:flakes -p no:pylint -p no:markdown -p no:sugar -p no:xvfb -p no:tavern --ignore tests/bench -n 16 ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...] __main__.py: error: unrecognized arguments: --no-subtests-shortletter inifile: /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1/pyproject.toml rootdir: /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1 (same thing if I compile cryptography with clang instead of gcc) And now: # readelf -a /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1-python3_10/install/usr/lib/python3.10/site-packages/cryptography/hazmat/bindings/_rust.abi3.so | grep PyInit Password: 731: 00000000000e9560 526 FUNC GLOBAL DEFAULT 15 PyInit__rust 7572: 00000000002639d0 239 FUNC LOCAL DEFAULT 15 PyInit__openssl 7595: 00000000000e9560 526 FUNC GLOBAL DEFAULT 15 PyInit__rust
(In reply to Paolo Pedroni from comment #16) > __main__.py: error: unrecognized arguments: --no-subtests-shortletter > inifile: > /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1/ > pyproject.toml > rootdir: > /var/tmp/portage/dev-python/cryptography-40.0.1/work/cryptography-40.0.1 That was because I was missing dev-python/pytest-subtests. After installing it tests ran fine. The culprit was then LTO with gcc. Shall I close this bug as invalid, or put it to block "the" LTO bug?
This is one of those things where I don't actually have a grasp of what the problem is, other than knowing Rust needs LTO handling specially. I think it requires thin LTO because it's all LLVM based and hence needs Clang if doing LTO.
Had this problem as well, used the compiler-clang environment file in https://wiki.gentoo.org/wiki/Clang and now it works.
I also ran into this. For me it was sufficient to just build dev-python/cryptography without LTO - no clang necessary (on ppc64). A filter-lto should do the trick.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d92f1991a0f3461bf226de3e239b5588b63305b0 commit d92f1991a0f3461bf226de3e239b5588b63305b0 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-07-10 02:00:49 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-07-10 02:01:07 +0000 dev-python/cryptography: filter-lto Closes: https://bugs.gentoo.org/903908 Signed-off-by: Sam James <sam@gentoo.org> .../cryptography/cryptography-40.0.2-r1.ebuild | 4 + .../cryptography/cryptography-41.0.1-r1.ebuild | 149 +++++++++++++++++++++ dev-python/cryptography/cryptography-41.0.1.ebuild | 4 + 3 files changed, 157 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f24547d15dfd8e2ec62c474fea0a05d860241679 commit f24547d15dfd8e2ec62c474fea0a05d860241679 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-07-10 02:44:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-07-10 02:45:28 +0000 dev-python/cryptography: revbump for earlier filter-lto change In d92f1991a0f3461bf226de3e239b5588b63305b0, I revbumped for ~arch but not stable, but on reflection, it's not really worth the distinction and it's more likely to cause confusion given LTO is widespread nowadays. Bug: https://bugs.gentoo.org/903908 Signed-off-by: Sam James <sam@gentoo.org> ...0.2-r1.ebuild => cryptography-40.0.2-r2.ebuild} | 0 .../cryptography/cryptography-41.0.1-r1.ebuild | 2 +- dev-python/cryptography/cryptography-41.0.1.ebuild | 149 --------------------- 3 files changed, 1 insertion(+), 150 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=dc51935f7aae5f89d1ffecabef322680979952b8 commit dc51935f7aae5f89d1ffecabef322680979952b8 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2023-02-09 00:49:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-07-12 06:24:42 +0000 cargo.eclass: filter out lto flags for C/CXX compilers we do it in src_compile to avoid excessive flag stripping in projects using cargo.eclass just to fetch crates. Bug: https://bugs.gentoo.org/903908 Closes: https://bugs.gentoo.org/893658 Closes: https://bugs.gentoo.org/910220 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> Signed-off-by: Sam James <sam@gentoo.org> eclass/cargo.eclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
