dev-lang/spidermonkey-78.4.0 ebuild checks for what it thinks sufficient free disk space and then proceeds to die during unpacking due to 'No space left on device' Reproducible: Always Steps to Reproduce: 1. mount /var/tmp/portage as tmpfs with 'size=6G' option 2. emerge dev-lang/spidermonkey-78.4.0 Actual Results: fail due to multiple 'open: No space left on device' errors Expected Results: installed package it looks like ebuild has wrong disk space number Portage 3.0.8 (python 3.7.9-final-0, default/linux/x86/17.0/desktop, gcc-9.3.0, glibc-2.32-r2, 4.19.152-gentoo-ns i686) ================================================================= System uname: Linux-4.19.152-gentoo-ns-i686-AMD_Phenom-tm-_9550_Quad-Core_Processor-with-gentoo-2.7 KiB Mem: 8236044 total, 503080 free KiB Swap: 1048572 total, 1048572 free Timestamp of repository gentoo: Sat, 07 Nov 2020 00:00:01 +0000 Head commit of repository gentoo: 17752fd503ad39121680fd9e219d471c26eddb3a sh bash 5.0_p18 ld GNU ld (Gentoo 2.34 p6) 2.34.0 ccache version 3.7.11 [enabled] app-shells/bash: 5.0_p18::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.30.3::gentoo dev-lang/python: 2.7.18-r4::gentoo, 3.7.9::gentoo, 3.8.5::gentoo dev-util/ccache: 3.7.11::gentoo dev-util/cmake: 3.17.4-r1::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.20::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.34-r2::gentoo sys-devel/gcc: 9.3.0-r1::gentoo sys-devel/gcc-config: 2.3.2::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.32-r2::gentoo Repositories: gentoo location: /var/lib/portage/tree sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-jobs: 1 local location: /var/lib/portage/local masters: gentoo priority: 0 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="*" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib/libreoffice/program/sofficerc /usr/share/applications/chromium-browser-chromium.desktop /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.4/ext-active/ /etc/php/cgi-php7.4/ext-active/ /etc/php/cli-php7.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=native -pipe" DISTDIR="/var/lib/portage/distfiles" EMERGE_DEFAULT_OPTS="--jobs 2 --load-average 5 --ask y --ask-enter-invalid --alert y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -march=i686 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs ccache clean-logs compress-build-logs config-protect-if-modified distlocks downgrade-backup ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-elog split-log strict suidctl unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -march=i686 -pipe" GENTOO_MIRRORS="https://ftp.snt.utwente.nl/pub/os/linux/gentoo ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo rsync://ftp.snt.utwente.nl/gentoo" LANG="en_CA.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en_CA en" MAKEOPTS="-j 4 -l 6" PKGDIR="/var/lib/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="3dnow 3dnowext X a52 aac acl acpi alsa amr apache2 asf audiofile bash-completion berkdb binary-drivers bittorrent blas branding bs2b bzip2 cairo caps ccache cdda cddb cdparanoia cdr cli colordiff crypt css cups curl dbus device-mapper dga djvu dmx dnssec dri dts dvb dvd dvdr elogind enca encode exif fbcon ffmpeg fftw flac foomaticdb fortran ftp gd gdbm geoip gif gimp gphoto2 gpm gs gtk gui iconv icu id3 id3tag idn javascript jpeg kipi latex lcms lensfun libglvnd libnotify libtirpc lilo live lm_sensors logrotate logwatch lzo mad matroska mbox md5sum memlimit mime mjpeg mmx mmxext mng mozilla mp3 mp4 mpeg mpeg2 mplayer mtp multiuser musicbrainz mysql na_dd na_icons ncurses network network-cron nls nptl nvidia offensive ogg openal opengl openmp opus pam pango pch pcre pdf png pnm policykit posix ppds pvr qt5 quicktime radio rar readline reiserfs replaygain rrdcgi rss rtsp sdl seccomp secure-delete sharedmem shorten smp sox spell split-usr srt srtp sse2 ssl startup-notification svg system-icu system-jpeg system-sqlite tex4ht theora threads threadsafe tiff truetype tv_check udev udisks umfpack unicode upower usb v4l v4l2 vdpau vim vim-pager vim-syntax vorbis wma wmf wmp wordperfect wxwidgets x264 x265 x86 xanim xattr xcb xinerama xml xmp xscreensaver xv xvid xvmc zlib" ABI_X86="32" ADA_TARGET="gnat_2018" ALSA_CARDS="usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a" ELIBC="glibc" INPUT_DEVICES="evdev libinput" KERNEL="linux" L10N="en-CA en" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby25 ruby26" USERLAND="GNU" VIDEO_CARDS="vesa nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
build log is too big to attach even compressed, but the gist is: * Package: dev-lang/spidermonkey-78.4.0 * Repository: gentoo * Maintainer: mozilla@gentoo.org * USE: abi_x86_32 elibc_glibc jit kernel_linux userland_GNU x86 * FEATURES: ccache network-sandbox preserve-libs sandbox suidctl userpriv usersandbox * Checking for at least 5600 MiB disk space at "/var/tmp/portage/dev-lang/spidermonkey-78.4.0/temp" ... [ ok ] * Using python3.7 to build >>> Unpacking source... >>> Unpacking firefox-78.4.0esr.source.tar.xz to /var/tmp/portage/dev-lang/spidermonkey-78.4.0/work tar: firefox-78.4.0/js/src/jit-test/tests/binast/lazy/basic/testCondSwitch3.binjs: Cannot open: No space left on device tar: firefox-78.4.0/js/src/jit-test/tests/binast/lazy/basic/bug913445.binjs: Cannot open: No space left on device etc, etc...
df /var/tmp/portage/ < Filesystem 1K-blocks Used Available Use% Mounted on none 6291456 0 6291456 0% /var/tmp/portage
How is it possible that 0 is being used when the build has quit partway through? It won’t clean up by itself. Also, because you have a long build log, this means some work was done. So it’s not a check-reqs issue. Please upload a tarball of the log.
Created attachment 670289 [details] build log there was no work done. just 118908 'No space left on device' error messages.
Are you able to extract the firefox-78.4.0esr.source.tar.xz archive yourself over that tmpfs? What does `df -i /var/tmp/portage/` say? Curious about inode count given it's a 32bit 4.19.x kernel (not sure but may default to low values on tmpfs), and this archive has 307602 files. But the question as to why it's empty remain, was it not able to write anything at all? Or did you cleanup?
(In reply to Ionen Wolkens from comment #5) > Are you able to extract the firefox-78.4.0esr.source.tar.xz archive yourself > over that tmpfs? no, same thing - borks halfway through with 'no space left on device' > What does `df -i /var/tmp/portage/` say? Curious about inode count given > it's a 32bit 4.19.x kernel (not sure but may default to low values on > tmpfs), and this archive has 307602 files. hmmm, this may be the culprit! Filesystem Inodes IUsed IFree IUse% Mounted on188723 none 188724 1 188723 1% /var/tmp/portage and 188723+118908=307631, so the math almost checks out. after manual unpack: Filesystem 1K-blocks Used Available Use% Mounted on none 6291456 1336784 4954672 22% /var/tmp/portage Filesystem Inodes IUsed IFree IUse% Mounted on none 188724 188724 0 100% /var/tmp/portage from tmpfs(5): " nr_inodes=inodes The maximum number of inodes for this instance. The default is half of the number of your physical RAM pages, or (on a machine with highmem) the number of lowmem RAM pages, whichever is smaller." remounting /var/tmp/portage with 'nr_inodes=400000' allowed me to successfully unpack manually. total unpacked size is Filesystem 1K-blocks Used Available Use% Mounted on none 6291456 2695868 3595588 43% /var/tmp/portage Filesystem Inodes IUsed IFree IUse% Mounted on none 400000 307603 92397 77% /var/tmp/portage so it looks like ebuild is not checking available inodes at all and tar gives misleading error messages. > But the question as to why it's empty remain, was it not able to write > anything at all? Or did you cleanup? don't read too much into it, it was just to illustrate that there's plenty of free space available. i was trying different things to find a workaround and it's quite possible one of them did a cleanup.
Not a problem in ebuild, reassigning to eclass maintainer for further investigation.
The only option I see for check-reqs.eclass would be adding new eclass variables for the required number of inodes. Not sure if that's feasible; it would put additional burden on package maintainers. OTOH, it's the responsibility of the system maintainer to create or mount file systems with parameters (like nr_inodes) appropriate for their intended use. An FS intended entirely for package builds can be expected to contain many small files.
(In reply to Ulrich Müller from comment #8) > OTOH, it's the responsibility of the system maintainer to create or mount > file systems with parameters (like nr_inodes) appropriate for their intended > use. An FS intended entirely for package builds can be expected to contain > many small files. no argument here. just give said system maintainers an idea of what to expect, like you already do for disk space. the particular peril of tmpfs is that by default number of inodes does not scale with the size of filesystem, but the size of RAM, and tar emits misleading error messages. even though inodes exhaustion is mentioned in wiki https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs, the proposed remedy is to use undocumented tmpfs feature (nr_inodes=0 -- a value not mentioned in tmpfs(5) nor mount(8)).
It's mentioned in `man tmpfs`, > Mount options > The tmpfs filesystem supports the following mount options: > > size=bytes > Specify an upper limit on the size of the filesystem. The size is given in bytes, and rounded up to > entire pages. > > The size may have a k, m, or g suffix for Ki, Mi, Gi (binary kilo (kibi), binary mega (mebi) and binary > giga (gibi)). > > The size may also have a % suffix to limit this instance to a percentage of physical RAM. > > The default, when neither size nor nr_blocks is specified, is size=50%. > > nr_blocks=blocks > The same as size, but in blocks of PAGE_CACHE_SIZE. > > Blocks may be specified with k, m, or g suffixes like size, but not a % suffix. > > nr_inodes=inodes > The maximum number of inodes for this instance. The default is half of the number of your physical RAM > pages, or (on a machine with highmem) the number of lowmem RAM pages, whichever is smaller. > > Inodes may be specified with k, m, or g suffixes like size, but not a % suffix. > >
the parameter is mentioned, but the specific value recommended in gentoo wiki "To workaround - append nr_inodes=0 to the list of your options for the tmpfs mount in the /etc/fstab file. For additional information refer to 'tmpfs' section in man mount." is undocumented
(In reply to Nick Soveiko from comment #9) > the particular peril of tmpfs is that by default number of inodes does not > scale with the size of filesystem, but the size of RAM, and tar emits > misleading error messages. even though inodes exhaustion is mentioned in > wiki https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs, the proposed > remedy is to use undocumented tmpfs feature (nr_inodes=0 -- a value not > mentioned in tmpfs(5) nor mount(8)). Actually, I was wondering why the number of inodes would be so low (188724) in your case. According to tmpfs(5): "The default is half of the number of your physical RAM pages, or (on a machine with highmem) the number of lowmem RAM pages, whichever is smaller." Presumably on x86 the latter is smaller. 188724 pages would be 737 MiB, could that possibly be correct? It should be larger even with a 1G/3G lowmem/highmem split. > no argument here. just give said system maintainers an idea of what to > expect, like you already do for disk space. That number of inodes is simply too small for a 6 GiB file system used for PORTAGE_TMPDIR. Basically it says that you expect files to have an average size of 33 KiB which is an unreasonable assumption for that use case (actual average is about 8 KiB for spidermonkey). So I stay with what I said in comment #8: > OTOH, it's the responsibility of the system maintainer to create or mount > file systems with parameters (like nr_inodes) appropriate for their intended > use. An FS intended entirely for package builds can be expected to contain > many small files. I am aware that the small number of inodes is due to the tmpfs default on x86. Then again, this is not the most common configuration, and it is easy to work around the problem by specifying the mount option.
(In reply to Ulrich Müller from comment #12) > Actually, I was wondering why the number of inodes would be so low (188724) > in your case. > > According to tmpfs(5): "The default is half of the number of your physical > RAM pages, or (on a machine with highmem) the number of lowmem RAM pages, > whichever is smaller." > > Presumably on x86 the latter is smaller. 188724 pages would be 737 MiB, > could that possibly be correct? It should be larger even with a 1G/3G > lowmem/highmem split. no, that machine has 8GB of memory. the only reason it runs a 32-bit system is that this install dates back to 2003-ish and switching to 64 bits requires pretty much reinstall from scratch.
This bug is about running out of INODES. Removing bug 757276 which is about wrong CHECKREQS_DISK_BUILD value.
*** Bug 761334 has been marked as a duplicate of this bug. ***
