I am not sure if is a bug, but at least I ran into it. When emerging net-misc/openssh-9.2_p1-r2::gentoo, configure aborts with the error message ``` [...] configure: creating ./config.status cat: -: No such file or directory configure: error: could not make ./config.status !!! Please attach the following file when seeking support: !!! /var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1/config.log * ERROR: net-misc/openssh-9.2_p1-r2::gentoo failed (configure phase): * econf failed * * Call stack: * ebuild.sh, line 136: Called src_configure * environment, line 2615: Called econf '--with-ldflags=-Wl,-O1 -Wl,--as-needed' '--disable-strip' '--with-pid-dir=/run' '--sysconfdir=/etc/ssh' '--libexecdir=/usr/lib64/misc' '--datadir=/usr/share/openssh' '--with-privsep-path=/var/empty' '--with-privsep-user=sshd' '--without-audit' '--without-kerberos5' '--without-ldns' '--without-libedit' '--with-pam' '--with-pie' '--without-selinux' '--without-security-key-builtin' '--with-openssl' '--with-ssl-engine' '--with-hardening' * phase-helpers.sh, line 730: Called __helpers_die 'econf failed' * isolated-functions.sh, line 112: Called die * The specific snippet of code: * die "$@" * * If you need support, post the output of `emerge --info '=net-misc/openssh-9.2_p1-r2::gentoo'`, * the complete build log and the output of `emerge -pqv '=net-misc/openssh-9.2_p1-r2::gentoo'`. * The complete build log is located at '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/temp/environment'. * Working directory: '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1' * S: '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1' * Messages for package sys-apps/portage-3.0.44: * Unable to find kernel sources at /usr/src/linux * Unable to calculate Linux Kernel version for build, attempting to use running version * Unable to check for the following kernel config options due * to absence of any configured kernel sources or compiled * config: * - IPC_NS * - PID_NS * - NET_NS * - UTS_NS * You're on your own to make sure they are set if needed. * Messages for package net-misc/openssh-9.2_p1-r2: * ERROR: net-misc/openssh-9.2_p1-r2::gentoo failed (configure phase): * econf failed * * Call stack: * ebuild.sh, line 136: Called src_configure * environment, line 2615: Called econf '--with-ldflags=-Wl,-O1 -Wl,--as-needed' '--disable-strip' '--with-pid-dir=/run' '--sysconfdir=/etc/ssh' '--libexecdir=/usr/lib64/misc' '--datadir=/usr/share/openssh' '--with-privsep-path=/var/empty' '--with-privsep-user=sshd' '--without-audit' '--without-kerberos5' '--without-ldns' '--without-libedit' '--with-pam' '--with-pie' '--without-selinux' '--without-security-key-builtin' '--with-openssl' '--with-ssl-engine' '--with-hardening' * phase-helpers.sh, line 730: Called __helpers_die 'econf failed' * isolated-functions.sh, line 112: Called die * The specific snippet of code: * die "$@" * * If you need support, post the output of `emerge --info '=net-misc/openssh-9.2_p1-r2::gentoo'`, * the complete build log and the output of `emerge -pqv '=net-misc/openssh-9.2_p1-r2::gentoo'`. * The complete build log is located at '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/temp/environment'. * Working directory: '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1' * S: '/var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1' * Messages for package sys-apps/portage-3.0.44: * Unable to find kernel sources at /usr/src/linux * Unable to calculate Linux Kernel version for build, attempting to use running version * Unable to check for the following kernel config options due * to absence of any configured kernel sources or compiled * config: * - IPC_NS * - PID_NS * - NET_NS * - UTS_NS * You're on your own to make sure they are set if needed. * Messages for package sys-apps/portage-3.0.44: * Unable to find kernel sources at /usr/src/linux * Unable to calculate Linux Kernel version for build, attempting to use running version * GNU info directory index is up-to-date. * IMPORTANT: 12 news items need reading for repository 'gentoo'. * Use eselect news read to view new items. * After world updates, it is important to remove obsolete packages with * emerge --depclean. Refer to `man emerge` for more information. subprocess exited with status 1 subprocess exited with status 1 [...] ``` I am running the gentoo stage3:latest docker image from docker.io on Gitlab with Buildah. My Dockerfile: ``` FROM docker.io/gentoo/stage3:latest MAINTAINER Ronny Forberger <ronnyforberger@ronnyforberger.de> COPY entrypoint.sh / RUN mkdir /etc/portage/repos.conf RUN cat /usr/share/portage/config/repos.conf | grep -v sync-uri > /etc/portage/repos.conf/gentoo.conf RUN echo "sync-uri = rsync://app01.thesrv.de/gentoo-portage" >> /etc/portage/repos.conf/gentoo.conf RUN emerge --sync --quiet RUN emerge -uD --keep-going=y --quiet-build=y @world RUN emerge --keep-going=y --quiet-build=y distcc EXPOSE 3632 ENTRYPOINT ["/entrypoint.sh"] ``` I have already tried to prefix TMPDIR=$(mktemp -d /tmp/emerge.XXXXXX) to the emerge command, since I thought the issue is related to this: https://timesinker.blogspot.com/2019/08/cat-no-such-file-or-directory.html But it seems not. Could anyone have a look? Thanks in advance. Reproducible: Always Steps to Reproduce: 1. emerge -uD --keep-going=y --quiet-build=y @world 2. 3. Actual Results: configure error on openssh ebuild with cat: -: No such file or directory Expected Results: successful emerge of @world
Please provide emerge --info. Also, attach the following files: /var/tmp/portage/net-misc/openssh-9.2_p1-r2/temp/build.log /var/tmp/portage/net-misc/openssh-9.2_p1-r2/work/openssh-9.2p1/config.log Change the bug status back to UNCONFIRMED after the above has been completed.
Created attachment 852302 [details] build.log xz complressed
Created attachment 852304 [details] config.log xz compressed
OK, I am attaching the log files and here is emerge --info: Portage 3.0.43 (python 3.10.9-final-0, default/linux/amd64/17.1, gcc-12, glibc-2.36-r5, 5.10.0-20-amd64 x86_64) ================================================================= System uname: Linux-5.10.0-20-amd64-x86_64-QEMU_Virtual_CPU_version_2.5+-with-glibc2.36 KiB Mem: 8147428 total, 751796 free KiB Swap: 8387580 total, 4354448 free Timestamp of repository gentoo: Fri, 17 Feb 2023 01:00:01 +0000 Head commit of repository gentoo: 9578cc8931cd01ee9702e521556ae235640c078c sh bash 5.1_p16-r2 ld GNU ld (Gentoo 2.39 p5) 2.39.0 app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r2::gentoo dev-lang/perl: 5.36.0-r1::gentoo dev-lang/python: 3.10.9-r1::gentoo, 3.11.1-r1::gentoo dev-util/meson: 0.64.1::gentoo sys-apps/baselayout: 2.9::gentoo sys-apps/openrc: 0.45.2-r2::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/gcc: 12.2.1_p20230121-r1::gentoo sys-devel/gcc-config: 2.8::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.15-r3::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r5::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://app01.thesrv.de/gentoo-portage priority: -1000 volatile: True sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="C.UTF8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 bzip2 cli crypt dri fortran gdbm iconv ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl test-rust unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27 ruby30" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SHELL, SIZE, STRINGS, STRIP, YACC, YFLAGS
It's possible that it's to do with: >Unable to unshare: EPERM (for FEATURES="pid-sandbox") >Unable to unshare: EPERM (for FEATURES="ipc-sandbox network-sandbox pid-sandbox") as we've had issues before when something tries to ptrace and gets blocked (even if the EPERM isn't for that specific sandbox).
OK, thanks, I have tried to set --cap-add=CAP_SYS_PTRACE on the buildah bud command, but I am still getting the error cat - No such file or directory on emerging openssh. Any other capability I need to add? If unsure, I will open an issue on buildah (podman). Thanks.
For my image builds I add all of CAP_SYS_ADMIN,CAP_NET_ADMIN,CAP_SYS_PTRACE: https://github.com/ajakk/podman-gentoo/blob/main/podman-gentoo.sh#L32 Would be curious to see another build log without those EPERM issues
what's ``cat --version''? cat - shouldn't even make attempts to open anything, at least on GNU cat, so I'm confused about where sandbox would even inject a failure.. I'll try to replicate this in docker, similarly without privileges.
no such luck - getting podman
OK, I have tried to add the caps CAP_SYS_ADMIN,CAP_NET_ADMIN,CAP_SYS_PTRACE to buildah bud, but still the same error. cat version is cat (GNU coreutils) 9.1 Packaged by Gentoo (9.1-r2 (p0)) Also your script uses podman build, not buildah bud, maybe that is why the caps are not taken into account? Not sure what causes the problem, I am using a buildah container from quay.io/buildah/stable:latest. Maybe I should try podman.
The issue seems to be related to a heredoc that configure uses to generate config.status. For large heredocs, bash writes the data to a temp file, and attaches the temp file to stdin before executing cat. My hunch is that something funky is happening to the temp file, probably caused by the container manager. I do note that GNU cat is smart enough to detect when both stdin and stdout are files (not pipes), and uses copy_file_range() to optimize the data copy in that case. We have seen cases where copy_file_range() returns ENOENT (No such file or directory) on weird filesystems (bug 885793), so I wonder if that might be related.
Yes, I think the error happens in the heredoc in the configure script. The filesystem is fuse-overlayfs where the container manager seems to mount the root filesystem of the image it builds and on which portage has its tempdir. fuse-overlayfs on /var/lib/containers/storage/overlay/57e7c4440c6cc490c600dff7d1fb84d52fe01a8f4f55a3face00c5b9de5300fe/merged type fuse.fuse-overlayfs (rw,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other)
Maybe try mounting a tmpfs on /tmp and /var/tmp within the container to see if that improves things.
Mounting tmpfs on /tmp and /var/tmp solves the problem. Maybe it is a bug in fuse overlayfs?`
(In reply to Ronny Forberger from comment #14) Yeah, that seems likely to me.
I believe bash unlinks the temp file before executing cat. Maybe fuse-overlayfs does not handle unlinking of open files correctly -- it is supposed to keep the file contents around until all processes have closed any open file descriptors.
I can reproduce the problem by running the script below from a fuse-overlayfs directory. > #!/bin/bash > echo hello world > tempfile > exec <tempfile > rm tempfile > cat > stdout
OK, I have checked fuse-overlay for already known issues, but couldn't find anything. Should I open an issue there? Thanks for your all support.
Created attachment 852762 [details] strace log As I suspected, the culprit is copy_file_range(). 799913 copy_file_range(0, NULL, 1, NULL, 9223372035781033984, 0) = -1 ENOENT (No such file or directory) If I don't redirect stdout to a file, cat calls read/write instead, and the error does not occur.
(In reply to Ronny Forberger from comment #18) > OK, I have checked fuse-overlay for already known issues, but couldn't find > anything. > > Should I open an issue there? Yes, this is a bug in fuse-overlayfs and should be reported there. We should also raise an issue with GNU coreutils: cat should fall back to read/write if copy_file_range() fails.
I'm not sure that'd help. If the FS fails to bookkeep properly for copy_file_range, I see no reason it'd work better with read/write.
(In reply to Arsen Arsenovic from comment #21) > I'm not sure that'd help. If the FS fails to bookkeep properly for > copy_file_range, I see no reason it'd work better with read/write. When I force cat to use read/write by not redirecting stdout, everything works as expected.
(In reply to Mike Gilbert from comment #22) > (In reply to Arsen Arsenovic from comment #21) > > I'm not sure that'd help. If the FS fails to bookkeep properly for > > copy_file_range, I see no reason it'd work better with read/write. > > When I force cat to use read/write by not redirecting stdout, everything > works as expected. Interesting... https://github.com/containers/fuse-overlayfs/blob/25db5be78a4cbe4d17116b95299dac2e34f2740d/NEWS#L7 Maybe we just need a bump.
OK, thanks, I guess it fixed in fuse-overlayfs-1.10. My container gitlab-runner is using fuse-overlayfs-1.9.
Thanks, we still need to fix it in Gentoo though.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=78b30c144db7b3a83869cd99c8245b43eaeb1e3a commit 78b30c144db7b3a83869cd99c8245b43eaeb1e3a Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-02-22 01:08:42 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-02-22 01:10:06 +0000 sys-fs/fuse-overlayfs: add 1.10 Closes: https://bugs.gentoo.org/895144 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-fs/fuse-overlayfs/Manifest | 1 + sys-fs/fuse-overlayfs/fuse-overlayfs-1.10.ebuild | 26 ++++++++++++++++++++++++ 2 files changed, 27 insertions(+)
