Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 581880 - sys-apps/portage: Incorrect distcc permissions if compiler is called first in pkg_setup()
Summary: sys-apps/portage: Incorrect distcc permissions if compiler is called first in...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal critical with 1 vote (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-02 21:05 UTC by Michał Górny
Modified: 2023-01-11 17:45 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
media-libs/libmypaint-1.6.1-r2 build log (media-libs_libmypaint-1.6.1-r2_20230111-165421.log,25.69 KB, text/plain)
2023-01-11 17:45 UTC, gen2dev
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-05-02 21:05:12 UTC
How to reproduce:

1. enable FEATURES='distcc userpriv',

2. remove $PORTAGE_TMPDIR,

3. emerge -1v dev-libs/glib.


In other words, if dev-libs/glib is the first package merged and distcc directory does not exist, Portage creates it with wrong permissions:

$ tree -ugp /tmp/portage/.distcc
/tmp/portage/.distcc
├── [drwxrwsr-x root     portage ]  lock
│   └── [-rw-r--r-- root     portage ]  cpu_localhost_0
└── [drwxrwsr-x root     portage ]  state
    ├── [-rw-r--r-- portage  portage ]  binstate_23135
    ├── [-rw-r--r-- portage  portage ]  binstate_23139
    ├── [-rw-r--r-- portage  portage ]  binstate_23152
    └── [-rw-r--r-- portage  portage ]  binstate_23192

As a result, distcc process running as portage:portage can't access cpu_localhost_0 lock file and distcc does not work.


The 'normal' permissions are:

$ tree -ugp /tmp/portage/.distcc
/tmp/portage/.distcc
├── [drwxrwsr-x root     portage ]  lock
│   ├── [-rw-r--r-- portage  portage ]  cpu_localhost_0
│   ├── [-rw-r--r-- portage  portage ]  cpu_localhost_1
│   └── [-rw-r--r-- portage  portage ]  cpu_tcp_pomiocik.local_3632_0
└── [drwxrwsr-x root     portage ]  state
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-05-02 21:09:14 UTC
And I guess this happens because the compiler is called during pkg_setup()...
Comment 2 Mateusz Lenik 2018-06-15 08:38:38 UTC
Are there any plans to fix this? As I'm using tmpfs to store my PORTAGE_TMPDIR, this issue is quite annoying :)
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-06-19 05:44:54 UTC
Indeed it is.  It also causes some configure checks to fail silently, causing ambiguous failures later on or miscompilation.
Comment 4 Larry the Git Cow gentoo-dev 2020-05-06 06:37:37 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=61f76236aa10c297f79c2fee123a79d26bfa9327

commit 61f76236aa10c297f79c2fee123a79d26bfa9327
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2020-05-06 04:54:03 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2020-05-06 06:37:12 +0000

    doebuild: Use ccache/distcc/icecream only in src_* phases
    
    Enable ccache/distcc/icecream only when src_* phases are executed.
    There is generally little value from them in pkg_*, as these phases
    use CC only to do a few tests at most.  These compilations generally
    are not run in parallel and the programs are tiny, so usually
    the overhead exceeds the gain.
    
    On the other hand, running them in pkg_* phases results in files being
    created with root ownership that breaks them afterwards when using
    userpriv.  As a result, user not only loses the benefit of, say, distcc
    but also frequently hits error messages that confuse some configure
    scripts and result in miscompiled packages.
    
    Bug: https://bugs.gentoo.org/581880
    Reviewed-by: Zac Medico <zmedico@gentoo.org>
    Closes: https://github.com/gentoo/portage/pull/547
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
Comment 5 Zac Medico gentoo-dev 2020-05-24 23:52:49 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=878eddfea083eda5aea60510aa1c4e9145f26b99

commit 878eddfea083eda5aea60510aa1c4e9145f26b99
Author: Michał Górny <mgorny@gentoo.org>
Date:   2020-05-18 21:22:17 +0200

    Revert "doebuild: Use ccache/distcc/icecream only in src_* phases"
    
    This breaks ccache/etc. entirely as they are no longer present in PATH
    in later phases.
    
    Reverts: 61f76236aa10c297f79c2fee123a79d26bfa9327
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-05-31 08:44:05 UTC
Ok, lemme explain what happened here.

Since Portage does environment saving, PATH was saved in earlier phase (when distcc was not yet in PATH), and restored later overriding this.

I think the preferred solution here is to create the shadow directories somewhere in PORTAGE_BUILDDIR, add them to PATH immediately but populate in the first phase allowing ccache/distcc/etc.  As a bonus, we can make Portage create all the correct symlink itself and make it independent of system shadow directory updates.
Comment 7 gen2dev 2023-01-11 17:45:27 UTC
Created attachment 848231 [details]
media-libs/libmypaint-1.6.1-r2 build log

I ran into this failure today updating a system that has been working fine forever. Emerging media-libs/libmypaint-1.6.1-r2 fails consistently. That package has been installed a long time and built okay before. Glib (suggested by the OP) does not fail for me. Fwiw, distcc is set up to run 8 jobs locally and a bunch on other hosts, not in pump mode, but the build fails even restricting 'emerge' and 'make' to a single job.

Build log is attached.

*** emerge --info:
Portage 3.0.41 (python 3.10.9-final-0, default/linux/amd64/17.1, gcc-11, glibc-2.36-r5, 5.15.80-gentoo-x86_64 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.15.80-gentoo-x86_64-x86_64-Intel-R-_Xeon-R-_CPU_E3-1275_V2_@_3.50GHz-with-glibc2.36
KiB Mem:    16342680 total,  13639424 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Wed, 11 Jan 2023 04:45:02 +0000
Head commit of repository gentoo: 648aea5f63c5ae0de0bdbf3d63cba2648022781b
sh bash 5.1_p16-r2
ld GNU ld (Gentoo 2.39 p5) 2.39.0
distcc 3.4 x86_64-pc-linux-gnu [enabled]
app-misc/pax-utils:        1.3.5::gentoo
app-shells/bash:           5.1_p16-r2::gentoo
dev-java/java-config:      2.3.1::gentoo
dev-lang/perl:             5.36.0-r1::gentoo
dev-lang/python:           3.10.9::gentoo
dev-util/cmake:            3.24.3::gentoo
dev-util/meson:            0.64.1::gentoo
sys-apps/baselayout:       2.9::gentoo
sys-apps/openrc:           0.45.2-r2::gentoo
sys-apps/sandbox:          2.29::gentoo
sys-devel/autoconf:        2.71-r5::gentoo
sys-devel/automake:        1.16.5::gentoo
sys-devel/binutils:        2.39-r4::gentoo
sys-devel/binutils-config: 5.4.1::gentoo
sys-devel/gcc:             11.3.1_p20221209::gentoo
sys-devel/gcc-config:      2.8::gentoo
sys-devel/libtool:         2.4.7::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.15-r3::gentoo (virtual/os-headers)
sys-libs/glibc:            2.36-r5::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.us.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts: 
    sync-rsync-verify-jobs: 1

local
    location: /etc/portage/overlay
    masters: gentoo

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distcc distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="C"
LC_ALL="C"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp/tmpfs"
SHELL="/bin/bash"
USE="acl amd64 bzip2 cli crypt dri fortran gdbm iconv ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl test-rust unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

=================================================================
                        Package Settings
=================================================================

media-libs/libmypaint-1.6.1-r2::gentoo was built with the following:
USE="gegl nls openmp -introspection" ABI_X86="(64)"
CFLAGS="-fstack-protector -O2 -ftree-vectorize -pipe -march=ivybridge -maes --param l2-cache-size=8192"
CXXFLAGS="-fstack-protector -O2 -ftree-vectorize -pipe -march=ivybridge -maes --param l2-cache-size=8192"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distcc distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"