Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 916317 - app-arch/patool-1.12_p20230424: failed (test phase): Unable to allocate enough memory for operation (x86) with processors that have more than 10 threads
Summary: app-arch/patool-1.12_p20230424: failed (test phase): Unable to allocate enoug...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Alfred Wingate
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2023-10-27 05:43 UTC by Joonas Niilola
Modified: 2024-03-25 14:52 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patool-1.12_p20230424:20231027-053615.log (patool-1.12_p20230424:20231027-053615.log,86.99 KB, text/plain)
2023-10-27 05:43 UTC, Joonas Niilola
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joonas Niilola gentoo-dev 2023-10-27 05:43:30 UTC
Portage 3.0.49 (python 3.11.5-final-0, default/linux/x86/17.0, gcc-13, glibc-2.37-r7, 6.1.57-gentoo i686)
=================================================================
System uname: Linux-6.1.57-gentoo-i686-AMD_Ryzen_9_5950X_16-Core_Processor-with-glibc2.37
KiB Mem:   131792192 total, 131536800 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Fri, 27 Oct 2023 04:47:46 +0000
Head commit of repository gentoo: c37d2fd28559917c83967a4d3cdbaeefe13ec2d5

sh bash 5.1_p16-r6
ld GNU ld (Gentoo 2.40 p5) 2.40.0
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.1_p16-r6::gentoo
dev-lang/perl:             5.38.0-r1::gentoo
dev-lang/python:           3.11.5::gentoo
sys-apps/baselayout:       2.14::gentoo
sys-apps/openrc:           0.48::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-devel/autoconf:        2.71-r6::gentoo
sys-devel/automake:        1.16.5-r1::gentoo
sys-devel/binutils:        2.40-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/gcc:             13.2.1_p20230826::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.37-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo
    priority: -1000
    volatile: False

ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=i686 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=i686 -O2 -pipe"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=y --autounmask-continue --autounmask-use=y --autounmask-write --binpkg-changed-deps=y --buildpkg-exclude virtual/* --buildpkg-exclude acct-* --buildpkg-exclude sys-kernel/* --complete-graph --nospinner --jobs=12 --usepkg"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-march=i686 -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=i686 -O2 -pipe"
GENTOO_MIRRORS="https://distfiles.gentoo.org"
LANG="C.UTF8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
LINGUAS=" en en_US en-US fi sv "
MAKEOPTS="-j32"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl bzip2 cli crypt dbus dri elogind fortran gdbm iconv ipv6 libtirpc ncurses nptl openmp openrc pam pcre readline seccomp split-usr ssl test-rust udev unicode x86 xattr zlib" ABI_X86="32" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en en_US en-US fi sv" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="BPF NVPTX" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SHELL, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 1 Joonas Niilola gentoo-dev 2023-10-27 05:43:44 UTC
Created attachment 873510 [details]
patool-1.12_p20230424:20231027-053615.log
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-10-28 04:39:46 UTC
We should be able to skip some tests selectively as it's going to depend on the input.
Comment 3 Alfred Wingate 2023-10-28 14:44:27 UTC
Some output from an x86 chroot.

gentoo-x86-test ~ # lrzip --version
lrzip version 0.651
gentoo-x86-test ~ # cat t.txt 
42gentoo-x86-test ~ # lrzip -o t.txt.lrz t.txt
Warning, low memory for chosen compression settings
Unable to allocate enough memory for operation
Failed to open streams in rzip_chunk
Fatal error - exiting
Comment 4 Alfred Wingate 2023-10-28 14:55:37 UTC
This seems to be case where lrzip autodetects max threads and then uses that. This then has issues where using that many threads uses too much memory for x86 environment.

gentoo-x86-test ~ # lrzip -p 1 -o t.txt.lrz t.txt
Warning, low memory for chosen compression settings
t.txt - Compression Ratio: 0.028. Average Compression Speed:  0.000MB/s.
Total time: 00:00:00.01
Comment 5 Alfred Wingate 2023-10-28 15:01:10 UTC
This would be a bug for lrzip as it should limit threads to avoid this.

https://github.com/ckolivas/lrzip/commit/cdec0951d7a7a67a331f82d021ea2ae3039f99af
Comment 6 Larry the Git Cow gentoo-dev 2023-11-13 11:36:16 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b2c166c7b506ae2bc30a9765b58ef370f7a2151c

commit b2c166c7b506ae2bc30a9765b58ef370f7a2151c
Author:     Alfred Wingate <parona@protonmail.com>
AuthorDate: 2023-11-13 07:21:32 +0000
Commit:     Yixun Lan <dlan@gentoo.org>
CommitDate: 2023-11-13 11:32:29 +0000

    app-arch/patool: exclude lrzip tests on x86
    
    Bug: https://bugs.gentoo.org/916317
    Signed-off-by: Alfred Wingate <parona@protonmail.com>
    Signed-off-by: Yixun Lan <dlan@gentoo.org>

 app-arch/patool/patool-1.12_p20230424.ebuild | 2 ++
 app-arch/patool/patool-1.14.1.ebuild         | 2 ++
 2 files changed, 4 insertions(+)