My current portage version has suddenly started segfaulting on all merges, including binary packages. It also segfaults when trying to run emerge --info, so system info is going to be a bit limited. I am currently trying to figure out what changes were made to the system that could have caused this - the only recent one I can think of is an upgrade to =dev-libs/isl-0.21.0 and the subsequent rebuild of sys-devel/gcc. However the segfaults occur even on operations where gcc is not involved, such as emerging binary packages. When unmerging packages, portage gets through the step of "No package files given... Grabbing a set." and lists the set, then segfaults. The files *are* removed from the filesystem, but the package is still listed as installed in the database. Since I can't emerge binary packages, I tried unpacking the an older version of isl, =dev-libs/isl-0.19.0 over my root filesystem to no effect. I cannot emerge sys-devel/gdb, so I downloaded a statically-linked version to get the following backtrace. Unfortunately I did not have debug info enabled previously, so its usefulness will be limited. No other programs are segfaulting other than portage. The python interpreter starts up and performs simple math just fine. I would also note that I also have two arm machines also running Gentoo unstable on the same version of gcc and isl with no issues - only this amd64 box has observed the issue. Below as as much relevant info as I can think of. Reproducible: Always $ uname -a Linux 5.0.7-gentoo #1 SMP Sun Apr 7 11:13:42 EDT 2019 x86_64 Intel(R) Core(TM)2 Duo CPU T5850 @ 2.16GHz GenuineIntel GNU/Linux $ gcc --version gcc (Gentoo Hardened 8.3.0-r1 p1.1) 8.3.0 $ cat /etc/portage/make.conf PORTDIR="/usr/portage" # PORTDIR_OVERLAY="/usr/local/portage" # deprecated DISTDIR="${PORTDIR}/distfiles" PKGDIR="${PORTDIR}/packages" FEATURES="parallel-fetch buildpkg" ACCEPT_KEYWORDS="amd64 ~amd64" PORTAGE_NICENESS="19" EMERGE_DEFAULT_OPTS="--usepkg" USE="-X -alsa -avahi -cgi -cups -dbus -gnome -gtk -introspection -kde -opengl -perl -pulseaudio -python -qt4 -qt5 -systemd -wayland -xinerama -zeroconf graphite pgo threads vhosts" CFLAGS="-O3 -pipe -march=native -funroll-loops -flto=8 -floop-block -floop-interchange -floop-strip-mine -ftree-loop-distribution" CXXFLAGS="${CFLAGS}" LC_MESSAGES=C PHP_TARGETS="php7-3" RUBY_TARGETS="ruby25" PYTHON_TARGETS="python3_6" PYTHON_SINGLE_TARGET="python3_6" CPU_FLAGS_X86="mmx mmxext sse sse2 sse3 ssse3" $ cat /var/lib/portage/world app-admin/filebeat app-admin/graylog2 app-admin/logstash-bin app-admin/sudo app-admin/syslog-ng app-arch/atool app-arch/unzip app-editors/vim app-misc/elasticsearch app-misc/jq app-misc/neofetch app-misc/tmux app-portage/gentoolkit app-portage/layman cross-armv6j-unknown-linux-gnueabihf/binutils cross-armv6j-unknown-linux-gnueabihf/gcc cross-armv6j-unknown-linux-gnueabihf/glibc cross-armv6j-unknown-linux-gnueabihf/linux-headers dev-db/mongodb dev-php/pecl-apcu_bc dev-util/strace dev-vcs/git media-sound/airsonic-bin net-analyzer/moloch net-analyzer/netcat net-analyzer/nethogs net-analyzer/nmap net-analyzer/ossec-hids net-analyzer/tcpdump net-analyzer/zabbix net-dns/dnscrypt-proxy net-dns/ldns-utils net-fs/samba net-misc/dhcpcd net-misc/mosh net-misc/ntp net-p2p/transmission net-vpn/openvpn net-vpn/tinc net-wireless/broadcom-sta net-wireless/wpa_supplicant sys-boot/grub sys-devel/bc sys-devel/bison sys-devel/crossdev sys-devel/flex sys-fs/ncdu sys-kernel/genkernel sys-process/htop sys-process/lsof virtual/linux-sources www-apps/kibana-bin www-apps/nextcloud www-servers/nginx
Created attachment 572600 [details] backtrace
You can try this to see if it also affects the python2 interpreter: python2 /usr/bin/emerge [args]
I have python2 installed, but do not have portage built for it. So I only get the following message: /usr/lib/python-exec/python2.7/emerge: this Python implementation (python2.7) is not supported by the script.
Since portage is not built for python2, you can `git clone https://anongit.gentoo.org/git/proj/portage.git` or unpack http://distfiles.gentoo.org/distfiles/portage-2.3.62.tar.bz2, and then use the PYTHONPATH/PATH approach described here: https://wiki.gentoo.org/wiki/Project:Portage#Testing_Portage Use: python2 /checkouts/portage/bin/emerge [arg]...
Thank you! That was able to rescue my portage build. I rebuild both python and portage, though the most likely suspect is the native-extensions module. Unsure of how it got in this state - should dev-libs/isl bumps force a sys-apps/portage rebuild in the future?
From the backtrace it looks like an issue with library loading via ctypes, so maybe just a python rebuild would have been enough?
@zmedico: The issue popped up again for me, and now that I had a way to fix it I was able to do some more tests. Surprisingly, it actually seems to be openssl-related. I found that I had been downgraded to a 1.1.0x version of openssl due to some dependency, as opposed to the 1.1.1 version the system was built against. Using python2 to reinstall the 1.1.1 binary package makes the installed portage functional again, no rebuild necessary. I am not quite sure of the answers to a number of questions, namely: - Why does python2 work but python3 does not, when they are both built against openssl-1.1.1? - Why does the segfault only occur when merging a package, and not immediately upon startup of the python interpreter? Is this because it is related to a native module, and python loads them dynamically? - If that's the case, what does openssl have to do with the portage native module? Isn't it only used for filesystem operations? I don't know if any of these questions will ever get answered, but for now I've hopefully put it to rest by tightening up my version masks on dev-libs/openssl and enabling splitdebug features on this system. Thank you for the help.
The openssl-1.1 libraries should have different sonames from the openssl-1.0 libraries, so FEATURES=preserve-libs should preserve the openssl-1.1 libraries when necessary, and things should keep working.
