Bug 673900 – =sys-apps/portage-2.3.53 with FEATURES="pid-sandbox": Unable to unshare: EINVAL - sandbox:setup_sandbox could not read fd path: /proc/self/fd: No such file or directory

• Status: RESOLVED FIXED
• Product: Portage Development

[Description Lars Wendler (Polynomial-C) (RETIRED) 2018-12-28 16:54:42 UTC]

# emerge -1Ov =http-parser-2.8.1_p20181224

These are the packages that would be merged, in order:

[ebuild   R    ] net-libs/http-parser-2.8.1_p20181224:0/2.8.0::gentoo  USE="-static-libs" ABI_X86="-32 (64) (-x32)" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB


>>> Verifying ebuild manifests

>>> Emerging (1 of 1) net-libs/http-parser-2.8.1_p20181224::gentoo
Unable to unshare: EINVAL
 * http-parser-2.8.1_p20181224.tar.gz BLAKE2B SHA512 size ;-) ...                                                                                                                                                                     [ ok ]
Unable to unshare: EINVAL
sandbox:setup_sandbox  could not read fd path: /proc/self/fd: No such file or directory

/usr/lib/portage/python3.6/ebuild.sh: line 631: /var/portage/net-libs/http-parser/http-parser-2.8.1_p20181224.ebuild: No such file or directory
 * ERROR: net-libs/http-parser-2.8.1_p20181224::gentoo failed (unpack phase):
 *   error sourcing ebuild
 * 
 * Call stack:
 *   ebuild.sh, line 631:  Called die
 * The specific snippet of code:
 *                      source "$EBUILD" || die "error sourcing ebuild"
 * 
 * If you need support, post the output of `emerge --info '=net-libs/http-parser-2.8.1_p20181224::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=net-libs/http-parser-2.8.1_p20181224::gentoo'`.
/usr/lib/portage/python3.6/isolated-functions.sh: line 214: /var/tmp/portage/net-libs/http-parser-2.8.1_p20181224/.die_hooks: No such file or directory
 * The complete build log is located at '/var/tmp/portage/net-libs/http-parser-2.8.1_p20181224/temp/build.log'.
 * Working directory: '/usr/lib64/python3.6/site-packages'
 * S: '/var/tmp/portage/net-libs/http-parser-2.8.1_p20181224/work/http-parser-2.8.1_p20181224'
/bin/sh: 1: cannot create /dev/null: Permission denied
Traceback (most recent call last):
  File "/usr/lib/portage/python3.6/ebuild-ipc.py", line 277, in <module>
    sys.exit(ebuild_ipc_main(sys.argv[1:]))
  File "/usr/lib/portage/python3.6/ebuild-ipc.py", line 273, in ebuild_ipc_main
    return ebuild_ipc.communicate(args)
  File "/usr/lib/portage/python3.6/ebuild-ipc.py", line 130, in communicate
    lock_obj = portage.locks.lockfile(self.ipc_lock_file, unlinkfile=True)
  File "/usr/lib64/python3.6/site-packages/portage/locks.py", line 147, in lockfile
    raise DirectoryNotFound(os.path.dirname(mypath))
portage.exception.DirectoryNotFound: /var/tmp/portage/net-libs/http-parser-2.8.1_p20181224
 * The ebuild phase 'unpack' has exited unexpectedly. This type of behavior
 * is known to be triggered by things such as failed variable assignments
 * (bug #190128) or bad substitution errors (bug #200313). Normally, before
 * exiting, bash should have displayed an error message above. If bash did
 * not produce an error message above, it's possible that the ebuild has
 * called `exit` when it should have called `die` instead. This behavior
 * may also be triggered by a corrupt bash binary or a hardware problem
 * such as memory or cpu malfunction. If the problem is not reproducible or
 * it appears to occur randomly, then it is likely to be triggered by a
 * hardware problem. If you suspect a hardware problem then you should try
 * some basic hardware diagnostics such as memtest. Please do not report
 * this as a bug unless it is consistently reproducible and you are sure
 * that your bash binary and hardware are functioning properly.

>>> Failed to emerge net-libs/http-parser-2.8.1_p20181224, Log file:

>>>  '/var/tmp/portage/net-libs/http-parser-2.8.1_p20181224/temp/build.log'

 * Messages for package net-libs/http-parser-2.8.1_p20181224:

 * The ebuild phase 'unpack' has exited unexpectedly. This type of behavior
 * is known to be triggered by things such as failed variable assignments
 * (bug #190128) or bad substitution errors (bug #200313). Normally, before
 * exiting, bash should have displayed an error message above. If bash did
 * not produce an error message above, it's possible that the ebuild has
 * called `exit` when it should have called `die` instead. This behavior
 * may also be triggered by a corrupt bash binary or a hardware problem
 * such as memory or cpu malfunction. If the problem is not reproducible or
 * it appears to occur randomly, then it is likely to be triggered by a
 * hardware problem. If you suspect a hardware problem then you should try
 * some basic hardware diagnostics such as memtest. Please do not report
 * this as a bug unless it is consistently reproducible and you are sure
 * that your bash binary and hardware are functioning properly.
 * 
 * The following package has failed to build, install, or execute postinst:
 * 
 *  (net-libs/http-parser-2.8.1_p20181224:0/2.8.0::gentoo, ebuild scheduled for merge), Log file:
 *   '/var/tmp/portage/net-libs/http-parser-2.8.1_p20181224/temp/build.log'
 * 


This is quite severe... downgrading portage is impossible unless I manually extract a binpkg of a previous portage version and then re-emerge that same portage version.

[Comment 1 Lars Wendler (Polynomial-C) (RETIRED) 2018-12-28 16:55:31 UTC]

# emerge --info portage
Portage 2.3.53 (python 3.6.8-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-8.2.0, glibc-2.27-r6, 4.19.12 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.19.12-x86_64-Intel-R-_Core-TM-_i7-3740QM_CPU_@_2.70GHz-with-gentoo-2.6
KiB Mem:    32941108 total,  21897264 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Fri, 28 Dec 2018 07:00:01 +0000
Head commit of repository gentoo: 45d6f1b6e9a2aa0b3b231f10a8b22548ad1c17e9
Timestamp of repository poly-c: Thu, 27 Dec 2018 23:35:04 +0000
sh dash 0.5.10.2
ld GNU ld (Gentoo 2.31.1 p5) 2.31.1
app-shells/bash:          4.4_p23::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.26.2::gentoo
dev-lang/python:          2.7.15::gentoo, 3.6.8_pre::poly-c, 3.7.2_pre::poly-c
dev-util/cmake:           3.13.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::poly-c
sys-apps/openrc:          0.40.3::gentoo
sys-apps/sandbox:         2.14::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.13.4-r2::gentoo, 1.16.1-r1::gentoo
sys-devel/binutils:       2.31.1-r3::gentoo
sys-devel/gcc:            6.5.0::gentoo, 7.3.0-r6::gentoo, 8.2.0-r6::gentoo
sys-devel/gcc-config:     2.0::gentoo
sys-devel/libtool:        2.4.6-r5::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 4.20::gentoo (virtual/os-headers)
sys-libs/glibc:           2.27-r6::gentoo
Repositories:

gentoo
    location: /var/portage
    sync-type: rsync
    sync-uri: rsync://192.168.0.254/gentoo-portage
    priority: -1000
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: --new-compress
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-jobs: 1

thabairne
    location: /var/lib/layman/thabairne
    masters: gentoo
    priority: 0

poly-c
    location: /var/lib/layman/poly-c
    masters: gentoo
    priority: 1

Installed sets: @system
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -mtune=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ConsoleKit /etc/X11/Sessions /etc/X11/app-defaults /etc/X11/xinit /etc/bonobo-activation /etc/ca-certificates /etc/ca-certificates.conf /etc/dbus-1 /etc/dconf /etc/env.d /etc/fonts /etc/fonts/fonts.conf /etc/foomatic /etc/games /etc/gconf /etc/gentoo-release /etc/gimp /etc/gnome-vfs-2.0 /etc/gre.d /etc/gtk-2.0 /etc/hp /etc/hsqldb /etc/htdig /etc/init.d /etc/iproute2 /etc/java-config-2 /etc/lftp /etc/logrotate.d /etc/ntop /etc/opt /etc/pango /etc/polkit-1 /etc/profile.d /etc/qt4 /etc/revdep-rebuild /etc/sandbox.d /etc/sasl2 /etc/sensors.d /etc/sgml /etc/sound /etc/ssl /etc/ssmtp /etc/t1lib /etc/terminfo /etc/usb_modeswitch.d /etc/vbox /etc/xdg /etc/xinetd.d /etc/xml"
CXXFLAGS="-march=native -mtune=native -O2 -pipe"
DISTDIR="/var/tmp/distfiles"
EMERGE_DEFAULT_OPTS="--alphabetical --with-bdeps=y --misspell-suggestions=n --autounmask=n --quiet-build=n --binpkg-respect-use=n --verbose-slot-rebuilds=n --keep-going --fuzzy-search=n --dynamic-deps=y"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms sign strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo"
INSTALL_MASK="/etc/systemd /lib/systemd /lib32/systemd /lib64/systemd /usr/lib/systemd /usr/lib32/systemd /usr/lib64/systemd /etc/init.d/functions.sh /usr/lib64/firefox/browser/features/[e-mail]"
LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common -Wl,--as-needed"
LINGUAS="de de@1901 en"
MAKEOPTS="-j4"
PKGDIR="/opt/portage/packages"
PORTAGE_BZIP2_COMMAND="lbzip2 -q"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--new-compress"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi activities alsa amd64 branding brotli bzip2 cairo caps cdda cdr cli consolekit crypt cups cxx dbus declarative dts dvd dvdr emboss encode exif fam ffmpeg flac gcrypt gdbm gif glamor gmp gnutls gtk iconv icu idn inotify ipv6 jpeg jpeg2k kde kipi kwallet lame lcms libinput libnotify libtirpc lzma mad matroska midi mmap mmx mmxext mng mp3 mp4 mpeg mpfr multilib ncurses nls nptl nsplugin ogg opengl openmp opus pam pango pcre pdf phonon plasma png policykit ppds pulseaudio qml qt5 quicktime readline rtmp sdl seccomp slang spell split-usr sse sse2 ssl startup-notification svg theora threads tiff tinfo truetype twolame udev udisks unicode upower usb vaapi vcd vdpau vorbis vpx wayland webp widgets wxwidgets x264 x265 xattr xcb xcomposite xinerama xml xrandr xv xvid zlib zstd" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="gnutls" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="libinput" KERNEL="linux" L10N="de de-1901 en" LIBREOFFICE_EXTENSIONS="pdfimport presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6 python3_7" QEMU_SOFTMMU_TARGETS="arm i386 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="radeon radeonsi"
Unset:  CC, CPPFLAGS, CTARGET, CXX, LANG, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS

=================================================================
                        Package Settings
=================================================================

sys-apps/portage-2.3.53::gentoo was built with the following:
USE="-build -doc -epydoc -gentoo-dev (ipc) native-extensions rsync-verify (-selinux) xattr" ABI_X86="(64)" PYTHON_TARGETS="-pypy python2_7 -python3_4 -python3_5 python3_6 python3_7"

[Comment 2 Arfrever Frehtes Taifersar Arahesis 2018-12-28 17:34:28 UTC]

Probably due to FEATURES="pid-sandbox", which has been enabled by default in this release:
https://gitweb.gentoo.org/proj/portage.git/commit/?id=55a9d4ccc5ac90b454638f9205f8a5d20ca8b47a

Please check if locally disabling it helps.

[Comment 3 Lars Wendler (Polynomial-C) (RETIRED) 2018-12-28 17:53:14 UTC]

FEATURES="-pid-sandbox" gives me back a working portage. But I have CONFIG_PID_NS enabled in my kernels so I wonder what's the issue here...

[Comment 4 Mike Gilbert 2018-12-28 18:14:47 UTC]

Do the following commands succeed when run as root on your system?

unshare -p
unshare -p -m

[Comment 5 Arfrever Frehtes Taifersar Arahesis 2018-12-28 18:19:14 UTC]

For me, both these commands result in:
-bash: fork: Cannot allocate memory

[Comment 6 Larry the Git Cow 2018-12-28 18:23:42 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=97e55f75067a87c7dfd46d1a139cbd4e01bc70b9

commit 97e55f75067a87c7dfd46d1a139cbd4e01bc70b9
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-12-28 18:18:34 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-12-28 18:19:09 +0000

    sys-apps/portage: 2.3.53-r1 revbump for bug 673900
    
    Temporarily disable new FEATURES that require unshare, since
    they may not fail gracefully in some cases.
    
    Bug: https://bugs.gentoo.org/673900
    Package-Manager: Portage-2.3.53, Repoman-2.3.12
    Signed-off-by: Zac Medico <[e-mail]>

 .../portage/{portage-2.3.53.ebuild => portage-2.3.53-r1.ebuild}    | 7 +++++++
 1 file changed, 7 insertions(+)

[Comment 7 Zac Medico 2018-12-29 08:00:11 UTC]

The behavior shown in comment #0 indicates that the "failed" unshare call has some side-effect(s) despite the EINVAL error. These side-effects make it impossible to safely continue.

In order to prevent side-effects from affecting an essential process, we can test the relevant unshare call in a short-lived multiprocessing.Process instance, and use the exitcode to indicate success or failure. If the unshare call is successful in the short-lived process, then we can consider it safe to perform the same call within an essential process.

[Comment 8 Arfrever Frehtes Taifersar Arahesis 2018-12-29 08:44:08 UTC]

(In reply to Mike Gilbert from comment #4)
> Do the following commands succeed when run as root on your system?
> 
> unshare -p
> unshare -p -m

It seems that these commands require additionally -f option to succeed.
`unshare -f -p` and `unshare -f -p -m` work for me.

[Comment 9 Zac Medico 2018-12-29 13:06:48 UTC]

Patch posted for review:

https://archives.gentoo.org/gentoo-portage-dev/message/785387c73fc49b941c0e113f43980e0f
https://github.com/gentoo/portage/pull/390

[Comment 10 Larry the Git Cow 2018-12-30 07:40:46 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b737b77e6e0f8802c172d9424ed1f8942ea40d66

commit b737b77e6e0f8802c172d9424ed1f8942ea40d66
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-12-30 07:34:56 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-12-30 07:38:51 +0000

    sys-apps/portage: add linux-info check for namespace support
    
    Bug: https://bugs.gentoo.org/673900
    Package-Manager: Portage-2.3.53, Repoman-2.3.12
    Signed-off-by: Zac Medico <[e-mail]>

 sys-apps/portage/portage-9999.ebuild | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

[Comment 11 Jeroen Roovers (RETIRED) 2019-01-01 23:34:05 UTC]

*** Bug 674288 has been marked as a duplicate of this bug. ***

[Comment 12 Larry the Git Cow 2019-01-04 03:49:46 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=c2a9850a25b2f32a25b43ef30189cd6657f397ad

commit c2a9850a25b2f32a25b43ef30189cd6657f397ad
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-12-29 06:56:40 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2019-01-04 03:04:49 +0000

    process.spawn: validate unshare calls (bug 673900)
    
    In order to prevent failed unshare calls from corrupting the state
    of an essential process, validate the relevant unshare call in a
    short-lived subprocess. An unshare call is considered valid if it
    successfully executes in a short-lived subprocess.
    
    Bug: https://bugs.gentoo.org/673900
    Signed-off-by: Zac Medico <[e-mail]>

 lib/portage/process.py | 159 +++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 133 insertions(+), 26 deletions(-)

[Comment 13 Larry the Git Cow 2019-01-04 06:03:38 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=549e77312735f9f19863daafaef0382ab1a2157f

commit 549e77312735f9f19863daafaef0382ab1a2157f
Author:     Zac Medico <[e-mail]>
AuthorDate: 2019-01-04 04:01:57 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2019-01-04 06:03:28 +0000

    sys-apps/portage: version bump to 2.3.54
    
     #671808 rsync: fix usersync timestamp file permission issue
     #673738 fix PORTAGE_TMPDIR=/ edge case
     #673900 validate unshare calls
    
    Bug: https://bugs.gentoo.org/671808
    Bug: https://bugs.gentoo.org/673738
    Bug: https://bugs.gentoo.org/673900
    Package-Manager: Portage-2.3.54, Repoman-2.3.12
    Signed-off-by: Zac Medico <[e-mail]>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.54.ebuild | 270 +++++++++++++++++++++++++++++++++
 2 files changed, 271 insertions(+)