I have two virtually identical machines where glibc was updated and now portage cannot compile/install anything. CPU_FLAGS_X86 is unset. The error is the same for all packages: After >>> Preparing source in...: /usr/lib/portage/python3.11/ebuild.sh: line 789: 21 Illegal instruction ( if [[ -n ${PORTAGE_PIPE_FD} ]]; then eval "exec ${PORTAGE_PIPE_FD}>&-"; unset PORTAGE_PIPE_FD; fi; __ebuild_main ${EBUILD_SH_ARGS}; exit 0 ) Re-installing glibc from a binhost is successful but still produces the same error when attempting to compile anything. There is additional information in a thread under Portage and Programming on the Gentoo forums. The bug system will not let me link to it. # emerge --info Portage 3.0.59 (python 3.11.7-final-0, default/linux/x86/17.0, gcc-12, glibc-2.38-r9, 6.5.3 i686) ================================================================= System uname: Linux-6.5.3-i686-Celeron_-Mendocino-with-glibc2.38 KiB Mem: 369472 total, 147072 free KiB Swap: 3145720 total, 3145720 free Timestamp of repository gentoo: Wed, 10 Jan 2024 17:30:02 +0000 Head commit of repository gentoo: e6381523c8cc1f827edbb0386adf058bfd89a4b2 sh bash 5.1_p16-r6 ld GNU ld (Gentoo 2.40 p5) 2.40.0 app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.1_p16-r6::gentoo dev-lang/perl: 5.38.2-r1::gentoo dev-lang/python: 3.11.7::gentoo, 3.12.1::gentoo dev-util/meson: 1.3.0-r2::gentoo sys-apps/baselayout: 2.14-r1::gentoo sys-apps/openrc: 0.48::gentoo sys-apps/sandbox: 2.38::gentoo sys-devel/autoconf: 2.71-r6::gentoo sys-devel/automake: 1.16.5-r1::gentoo sys-devel/binutils: 2.40-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/gcc: 12.3.1_p20230526::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/make: 4.4.1-r1::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.38-r9::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 volatile: True sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 3 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="@FREE" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=i686 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/usr/portage/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -march=i686 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -march=i686 -pipe" GENTOO_MIRRORS="REMOVED SO I CAN POST THIS" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl bzip2 cli crypt dri fortran gdbm iconv ipv6 libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl test-rust unicode x86 xattr zlib" ABI_X86="32" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Is there any chance you can get gdb installed? From https://forums.gentoo.org/viewtopic-p-8813438.html#8813438: ``` [...] 186eb1: 88 95 64 fb ff ff mov %dl,-0x49c(%ebp) 186eb7: 0f 84 75 0b 00 00 je 187a32 <glob@GLIBC_2.0+0xca2> [...] ``` Illegal instruction doesn't make sense here by itself (the things in the region are fine); what is perhaps possible is it's jumping to the wrong place.
Sam, I have gotten gdb copied onto one of the effected systems but it is something I have rarely used... "./gdb --args /usr/bin/emerge -1v traceroute" then "run" complains: "/usr/bin/emerge": not in executable format: file format not recognized # file /usr/bin/emerge /usr/bin/emerge: symbolic link to ../lib/python-exec/python-exec2 # /usr/lib/python-exec/python-exec2 -1vp traceroute /usr/lib/python-exec/python-exec2: python-exec2 is a wrapper, it must not be run directly. Can you give me a hint as to how to correctly run this with gdb? Thanks.
Hi drumgod, I will try get back to you with some more instructions soon. Also, immolo has kindly arranged with a volunteer to get me SSH access to a machine where it is reproducible, so I will try that as well, just not yet had a chance.
Created attachment 883331 [details] 2.39 log and failed test out files Testing the latest version in the live ebuilds I have found 3 failed test results with misc/tst-dirname looking like the one of interest: Didn't expect signal from child: got `Illegal instruction' I will test this with 2.38-r9 just to confirm and report back in the next couple of days when finished. emerge --info: Portage 3.0.61 (python 3.11.5-final-0, default/linux/x86/17.0, gcc-13, glibc-2.37-r7, 6.6.13-gentoo-dist i686) ================================================================= System uname: Linux-6.6.13-gentoo-dist-i686-Pentium_II_-Deschutes-with-glibc2.37 KiB Mem: 393536 total, 64536 free KiB Swap: 8388604 total, 8381892 free Timestamp of repository gentoo: Thu, 25 Jan 2024 12:10:01 +0000 Head commit of repository gentoo: 708167085bbacc7172b0045364883e958f9cef4a sh bash 5.1_p16-r6 ld GNU ld (Gentoo 2.40 p5) 2.40.0 app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.1_p16-r6::gentoo dev-build/autoconf: 2.71-r6::gentoo dev-build/automake: 1.16.5-r1::gentoo dev-build/libtool: 2.4.7-r1::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.3.0-r2::gentoo dev-lang/perl: 5.38.0-r1::gentoo dev-lang/python: 3.11.5::gentoo sys-apps/baselayout: 2.14::gentoo sys-apps/openrc: 0.48::gentoo sys-apps/sandbox: 2.38::gentoo sys-devel/binutils: 2.40-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/gcc: 13.2.1_p20230826::gentoo sys-devel/gcc-config: 2.11::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.37-r7::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 volatile: False sync-rsync-verify-max-age: 3 sync-rsync-verify-metamanifest: yes sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: Binary Repositories: gentoobinhost priority: 1 sync-uri: https://gentoo.osuosl.org/releases/x86/binpackages/17.0/i686 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="@FREE" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=i686 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -march=i686 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -march=i686 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="C.UTF8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl bzip2 cli crypt dri fortran gdbm iconv ipv6 libtirpc ncurses nls openmp pam pcre readline seccomp split-usr ssl test-rust unicode x86 xattr zlib" ABI_X86="32" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
I have also confirmed the test errors are the same on 2.38 and an upstream bug has now been reported. For extra information I am also including the dmesg out here: [106071.719000] process 'ld-linux.so.2' launched '/var/tmp/portage/sys-libs/glibc-9999/temp/testscriptRsIrAs' with NULL argv: empty string added [110136.637049] traps: ld-linux.so.2[9275] trap invalid opcode ip:b7e4fc8f sp:bfa360cc error:0 in libc.so[b7cc7000+194000] [110326.703936] ld-linux.so.2[9796]: memfd_create() called without MFD_EXEC or MFD_NOEXEC_SEAL set [240275.802537] ld-linux.so.2[16706]: segfault at 9393b150 ip b7d814e3 sp bfc36c40 error 4 in libc.so[b7d02000+193000] likely on CPU 0 (core 0, socket 0) [240275.802721] Code: 8d 70 08 f7 c6 0f 00 00 00 0f 85 20 0a 00 00 8d 5c 97 04 65 a1 0c 00 00 00 85 c0 0f 85 46 04 00 00 8b 4c 24 18 89 f0 c1 e8 0c <8b> 79 08 8b 4c 24 14 31 f8 89 41 04 8b 44 24 18 8b 40 04 89 44 24 [240275.807617] ld-linux.so.2[16707]: segfault at 151537d0 ip b7d814e3 sp bfc36c40 error 4 in libc.so[b7d02000+193000] likely on CPU 0 (core 0, socket 0) [240275.807770] Code: 8d 70 08 f7 c6 0f 00 00 00 0f 85 20 0a 00 00 8d 5c 97 04 65 a1 0c 00 00 00 85 c0 0f 85 46 04 00 00 8b 4c 24 18 89 f0 c1 e8 0c <8b> 79 08 8b 4c 24 14 31 f8 89 41 04 8b 44 24 18 8b 40 04 89 44 24 [264420.901533] traps: ld-linux.so.2[18454] trap invalid opcode ip:b7f0258f sp:bf80e39c error:0 in libc.so[b7d7b000+193000] [363201.403362] traps: bash[1205] trap invalid opcode ip:b7dcceb5 sp:bf90298c error:0 in libc.so.6[b7c46000+193000] [363357.204421] traps: bash[2431] trap invalid opcode ip:b7e9aeb5 sp:bfb088ac error:0 in libc.so.6[b7d14000+193000]
Fixed in 2.38-r11 and 2.39-r2
This bug still causes issues on new stage3 installs and stable profiles until sys-libs/glibc-2.38-r11 is stable on x86. I've tested using 2.38-r11 on a stable install and can find no issue to not recommend we at least bump the stable version to -r11 for x86 to make sure any users doesn't hit this. In the meantime any user of a CPU of this era (a x86 CPU without SSE and SSE2) can use the following workaround: 1. Use the stage3 at https://mirror.reenigne.net/gentoo-snapshots/2023-12-25-0600/releases/x86/autobuilds/current-stage3-i686-openrc/stage3-i686-openrc-20231127T164703Z.tar.xz 2. Add the following to /etc/portage/package.accept_keywords/glibc "=sys-libs/glibc-2/38-r11" 3. Complete the install as normal Step 2 is only required for a system upgrading from 2.37 to 2.38 however if you have upgraded to a 2.38 pre r11 or 2.39 pre 2 then your system will require a reinstall for the best way to fix the state the system will be left in.
