Bug 933385 – Ugly traceback on binpkg creation failure

• Product: Portage Development
• Status: RESOLVED FIXED

[Description Sam James 2024-06-02 02:23:24 UTC]

Created attachment 894894 [details]
llvm-17.0.6:20240602-002452.log.xz

Note that the weird objcopy crashes are related to bug 932749 (where we don't handle errors properly). I've not looked into why they're crashing yet as the reproducers so far are huge and everything seems OK otherwise.

Anyway..

```
[...]
/usr/bin/dwz: /var/tmp/portage.notmp/portage/sys-devel/llvm-17.0.6/image/usr/lib/llvm/17/lib/libLLVM-17.so: Too many DIEs, not optimizing
/usr/bin/dwz: /var/tmp/portage.notmp/portage/sys-devel/llvm-17.0.6/image/usr/lib/llvm/17/bin/llvm-exegesis: Too many DIEs, not optimizing
/usr/bin/dwz: /var/tmp/portage.notmp/portage/sys-devel/llvm-17.0.6/image/usr/lib/llvm/17/lib64/libLLVM-17.so: Too many DIEs, not optimizing
/var/tmp/portage/._portage_reinstall_.5nywj7ss/bin/estrip: line 282: 1744892 Segmentation fault      (core dumped) ${OBJCOPY} ${objcopy_flags} "${src}" "${dst}"
/var/tmp/portage/._portage_reinstall_.5nywj7ss/bin/estrip: line 282: 1744064 Segmentation fault      (core dumped) ${OBJCOPY} ${objcopy_flags} "${src}" "${dst}"
installsources: rsyncing source files
Traceback (most recent call last):
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/bin/gpkg-helper.py", line 75, in <module>
    rval = main(sys.argv[:])
           ^^^^^^^^^^^^^^^^^
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/bin/gpkg-helper.py", line 67, in main
    rval = command_compose(args[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/bin/gpkg-helper.py", line 39, in command_compose
    gpkg_file.compress(image_dir, metadata)
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/lib/portage/gpkg.py", line 952, in compress
    with tar_stream_writer(
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/lib/portage/gpkg.py", line 147, in __exit__
    self.close()
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/lib/portage/gpkg.py", line 221, in close
    if not (self.killed or self.error):
                           ^^^^^^^^^^
AttributeError: 'tar_stream_writer' object has no attribute 'error'
Exception ignored in: <function tar_stream_writer.__del__ at 0x7f3145b6b920>
Traceback (most recent call last):
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/lib/portage/gpkg.py", line 141, in __del__
  File "/var/tmp/portage/._portage_reinstall_.5nywj7ss/lib/portage/gpkg.py", line 221, in close
AttributeError: 'tar_stream_writer' object has no attribute 'error'
 * ERROR: sys-devel/llvm-17.0.6::gentoo failed (package phase):
 *   Failed to create binpkg file
 *
 * Call stack:
 *   misc-functions.sh, line 682:  Called __dyn_package
 *   misc-functions.sh, line 569:  Called die
 * The specific snippet of code:
 *                      die "Failed to create binpkg file"
[...]
```

--

$ emerge --info
Portage 3.0.64-29+g120b2ec98-dirty (python 3.12.3-final-0, default/linux/amd64/23.0/hardened, gcc-15, glibc-2.39-r7, 6.6.32 x86_64)
=================================================================
System uname: Linux-6.6.32-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.39
KiB Mem:    65740432 total,   8488232 free
KiB Swap:   16662520 total,       244 free
Timestamp of repository gentoo: Sat, 01 Jun 2024 02:49:02 +0000
Head commit of repository gentoo: 5d8e8365e7841131ecf606588798611af143b5b4

Timestamp of repository kde: Thu, 30 May 2024 17:33:47 +0000
Head commit of repository kde: f10c67775d120f33ca716c8e8086b1150c5f3d04

Timestamp of repository qt: Thu, 30 May 2024 13:03:51 +0000
Head commit of repository qt: 003dee833214a358a9cfa818e39ad37d7eeee8a6

Timestamp of repository sam_c: Fri, 17 May 2024 23:20:36 +0000
Head commit of repository sam_c: 18fc34a19e49bd9285d25831c6eaa195a08f54b8

sh dash 0.5.12
ld GNU ld (Gentoo 9999 p1) 2.42.50.20240601
ccache version 4.10 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.3_alpha-r2::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.69-r9::gentoo, 2.72-r1::gentoo, 9999::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.29.3::gentoo
dev-build/libtool:         2.5.0::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.4.1::gentoo
dev-java/java-config:      2.3.3-r1::gentoo
dev-lang/perl:             5.40.0_rc1-r1::gentoo
dev-lang/python:           3.10.14_p1-r1::gentoo, 3.11.9-r1::gentoo, 3.12.3-r1::gentoo, 3.13.0_beta1_p3::gentoo
dev-lang/rust:             1.78.0::gentoo
dev-util/ccache:           4.10-r1::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-apps/systemd:          255.7::gentoo
sys-devel/binutils:        2.38-r2::gentoo, 2.39-r5::gentoo, 2.40-r9::gentoo, 2.41-r5::gentoo, 2.42-r1::gentoo, 9999::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           17.0.6::gentoo, 18.1.6::gentoo
sys-devel/gcc:             10.5.0::gentoo, 11.4.1_p20240529::gentoo, 12.3.1_p20240530::gentoo, 13.3.1_p20240531::gentoo, 14.1.1_p20240525::gentoo, 15.0.9999::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             17.0.6::gentoo, 18.1.6::gentoo
sys-devel/llvm:            17.0.6::gentoo, 18.1.6::gentoo
sys-kernel/linux-headers:  6.9::gentoo (virtual/os-headers)
sys-libs/glibc:            2.39-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000
    eclass-overrides: sam_c
    volatile: False
    sync-git-clone-extra-opts: -b stable -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now
    sync-git-verify-commit-signature: yes

crossdev
    location: /var/db/repos/crossdev
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

kde
    location: /var/db/repos/kde
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/kde.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

local
    location: /var/db/repos/local
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

qt
    location: /var/db/repos/qt
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/qt.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

sam_c
    location: /var/db/repos/sam_c
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/sam_c.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

Installed sets: @sam-emacs, @sam-fonts, @sam-fortune
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr -Wbuiltin-declaration-mismatch"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --complete-graph --deep --dynamic-deps=n --changed-deps=n --usepkg=y --implicit-system-deps=n --usepkg-exclude sys-fs/zfs --usepkg-exclude sys-fs/zfs-kmod --jobs=4 --load-average 32"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg clean-logs config-protect-if-modified distlocks fixlafiles ipc-sandbox merge-wait mount-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
GENTOO_MIRRORS="https://www.mirrorservice.org/sites/distfiles.gentoo.org/ https://mirror.bytemark.co.uk/gentoo/ https://mirrors.soeasyto.com/distfiles.gentoo.org/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -fdiagnostics-color=always -flto=jobserver -Wl,--defsym=__gentoo_check_ldflags__=0"
LEX="reflex"
LINGUAS="en en_GB"
MAKEOPTS="-j32 -l32"
PKGDIR="/var/cache/binpkgs"
PORTAGE_BUNZIP2_COMMAND="lbunzip2"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RUSTFLAGS="-C target-cpu=native -C link-arg=-Wl,-z,pack-relative-relocs"
SHELL="/bin/bash"
USE="X a52 aac acl acpi activities aes alsa amd64 avx avx2 bash-completion bluetooth branding bzip2 cairo caps cdda cdr clang crypt dbus declarative dist-kernel dri dts dvd dvdr editorconfig emacs encode exif f16c filecaps firewalld flac fma3 freetype gdbm gif gmp gpm graphite gtk gui hardened harfbuzz hunspell iconv icu ipv6 jit jpeg kde kdesu kf6compat kwallet lcms libnotify libtirpc lto mad mmx mmxext mng modules-compress mp3 mp4 mpeg multilib ncurses nftables nls ogg opengl openmp pam pango pclmul pcre pdf pgo pic pie pipewire plasma png policykit popcnt ppds pulseaudio qml qt5 rdrand readline screencast sdl seccomp semantic-desktop sha sound spell sse sse2 sse3 sse4_1 sse4_2 sse4a ssl ssp ssse3 startup-notification svg system-av1 system-binutils system-boost system-bootstrap system-cairo system-clang system-digest system-ffmpeg system-harfbuzz system-heimdal system-icu system-jpeg system-leveldb system-libevent system-libs system-libvpx system-libyaml system-llvm system-lz4 system-mitkrb5 system-sqlite system-ssl system-tbb system-uulib system-webp system-zlib systemd test-rust tiff truetype udev udisks unicode upower usb valgrind verify-sig vorbis vulkan wayland widgets x264 xattr xcb xft xml xtpax xv xvid zlib zsh-completion" ABI_X86="32 64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX PowerPC RISCV Sparc SystemZ WebAssembly X86 XCore" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_11 pypy3 python3_10 python3_12 python3_13" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="aarch64 aarch64_be alpha armeb hppa i386 m68k ppc ppc64 ppc64le sparc sparc64 x86_64" RUBY_TARGETS="ruby31 ruby32" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
YACC="byacc"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, SIZE, STRINGS, STRIP, YFLAGS

[Comment 1 Sam James 2024-06-02 02:24:19 UTC]

Note that we *did* handle the failure okay:
"""
>>> Completed (92 of 892) dev-lang/python-3.12.3-r1::gentoo
>>> Emerging (93 of 892) sys-devel/llvm-17.0.6::gentoo
>>> Jobs: 92 of 892 complete, 1 running             Load avg: 19.5, 18.9, 15.6

>>> Jobs: 92 of 892 complete, 1 running             Load avg: 19.4, 18.9, 15.7
>>> Failed to emerge sys-devel/llvm-17.0.6, Log file:
>>>  '/var/log/portage/build/sys-devel/llvm-17.0.6:20240602-002452.log'
>>> Jobs: 92 of 892 complete, 1 failed              Load avg: 43.4, 40.9, 49.4
*** Resuming merge...
[...]
"""

I just filed because the traceback looks like we may need to give it attention anyway.

[Comment 2 Larry the Git Cow 2024-06-02 18:28:44 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=eb855b8cd1248f49649003dcfb9bf009b70e88cb

commit eb855b8cd1248f49649003dcfb9bf009b70e88cb
Author:     Zac Medico <[e-mail]>
AuthorDate: 2024-06-02 17:56:44 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2024-06-02 18:05:35 +0000

    tar_stream_writer: Add missing error attribute
    
    This attribute was previously initialized only
    in an exception handler.
    
    Fixes: b8c3f38ec5ee ("Add more error handling for binpkgs")
    Bug: https://bugs.gentoo.org/933385
    Signed-off-by: Zac Medico <[e-mail]>

 lib/portage/gpkg.py | 1 +
 1 file changed, 1 insertion(+)

[Comment 3 Larry the Git Cow 2024-06-04 01:31:39 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1dd08f05355b14c7594d0e0ccf46cdb2b1486cbe

commit 1dd08f05355b14c7594d0e0ccf46cdb2b1486cbe
Author:     Sam James <[e-mail]>
AuthorDate: 2024-06-04 01:31:07 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2024-06-04 01:31:07 +0000

    sys-apps/portage: add 3.0.65
    
    Closes: https://bugs.gentoo.org/930802
    Closes: https://bugs.gentoo.org/787623
    Closes: https://bugs.gentoo.org/922629
    Closes: https://bugs.gentoo.org/927241
    Closes: https://bugs.gentoo.org/930992
    Closes: https://bugs.gentoo.org/932739
    Closes: https://bugs.gentoo.org/932804
    Closes: https://bugs.gentoo.org/933297
    Closes: https://bugs.gentoo.org/933385
    Closes: https://bugs.gentoo.org/933442
    Signed-off-by: Sam James <[e-mail]>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.65.ebuild | 242 +++++++++++++++++++++++++++++++++
 2 files changed, 243 insertions(+)