Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 601252 - sys-apps/portage: "No write access" to DISTDIR as root on root_squash NFS share when userfetch is set
Summary: sys-apps/portage: "No write access" to DISTDIR as root on root_squash NFS sha...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 709606 (view as bug list)
Depends on:
Blocks: 377365 711148
  Show dependency tree
 
Reported: 2016-11-30 04:07 UTC by Paul B. Henson
Modified: 2024-02-12 00:20 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul B. Henson 2016-11-30 04:07:49 UTC
I have /usr/portage/distfiles mounted via nfs without root privs on the client. This used to work fine. After an update a while back now emerge fails if the distfile is not already present with an error:

dns ~ # emerge bind  
Calculating dependencies... done!

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) net-dns/bind-9.10.4_p4::gentoo
!!! No write access to '/usr/portage/distfiles'
!!! File bind-9.10.4-P4.tar.gz isn't fetched but unable to get it.

Indeed, root has no privs:

dns ~ # touch /usr/portage/distfiles/test
touch: cannot touch ‘/usr/portage/distfiles/test’: Permission denied

However, the portage user does:

dns ~ # su -s /bin/bash - portage -c 'touch /usr/portage/distfiles/test'
dns ~ # ls -l /usr/portage/distfiles/test
-rw-r--r-- 1 portage portage 0 Nov 29 20:02 /usr/portage/distfiles/test

and in the past emerge had no issues downloading to it:

dns ~ # ls -l /usr/portage/distfiles
[...]
-rw-rw-r-- 1 portage portage    503012 Mar 26  2014 yaml-0.1.6.tar.gz
-rw-rw-r-- 1 portage portage      9656 Jan 15  2013 yasm-1.2.0-x32.patch.xz
-rw-rw-r-- 1 portage portage   1436502 Oct 31  2011 yasm-1.2.0.tar.gz
-rw-rw-r-- 1 portage portage   1287223 Jul  8  2008 zip30.zip

At some point it seems to have started explicitly testing for write access as root before setting uid to the portage user for the download and failing.

When the userfetch feature is enabled root does not need write access to the distfiles directory so this failure seems buggy.

dns ~ # emerge --info
Portage 2.3.0 (python 3.4.3-final-0, default/linux/amd64/13.0, gcc-4.9.3, glibc-2.22-r4, 3.14.77-gen
too x86_64)
=================================================================
System uname: Linux-3.14.77-gentoo-x86_64-Intel_Xeon_E312xx_-Sandy_Bridge-with-gentoo-2.2
KiB Mem:     1020704 total,    193048 free
KiB Swap:    1048572 total,   1048572 free
Timestamp of repository gentoo: Mon, 28 Nov 2016 04:45:01 +0000
sh bash 4.3_p46-r1
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
app-shells/bash:          4.3_p46-r1::gentoo
dev-lang/perl:            5.20.2::gentoo
dev-lang/python:          2.7.10-r1::gentoo, 3.4.3-r1::gentoo
dev-util/pkgconfig:       0.28-r2::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.21.3::gentoo
sys-apps/sandbox:         2.10-r1::gentoo
sys-devel/autoconf:       2.69::gentoo
sys-devel/automake:       1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo
sys-devel/gcc-config:     1.7.3::gentoo
sys-devel/libtool:        2.4.6::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers)
sys-libs/glibc:           2.22-r4::gentoo
Repositories:
gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.namerica.gentoo.org/gentoo-portage
    priority: -1000

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -mtune=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep
-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -O2 -pipe -march=native -mtune=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg collision-protect config-protect-if-modified distlocks
 ebuild-locks fixlafiles merge-sync news notitles parallel-fetch preserve-libs protect-owned sandbox
 sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox us
ersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.ucsb.edu/pub/mirrors/linux/gentoo/ http://gentoo.mirrors.easynews.com/lin
ux/gentoo/ http://gentoo.osuosl.org/ http://mirror.usu.edu/mirrors/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --f
orce --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/lo
cal --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/lib/portage/tmp"
USE="acl acpi amd64 berkdb bzip2 cli crypt cxx dri gdbm iconv kerberos lzma mmx mmxext modules multi
lib ncurses nls nptl openmp pam pcre readline seccomp session sse sse2 ssl unicode urandom xattr zip
 zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens
1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via
82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_bas
ic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile
 authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cac
he env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mim
e_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGR
A_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" C
OLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext
 popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate eve
rmore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtc
m104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse e
vdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb
 ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION=
"libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python
3_4" RUBY_TARGETS="ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon rade
onsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface g
eoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMA
ND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Sven E. 2018-11-09 00:51:07 UTC
Problem still exists, just ran into this.

Is there possibly any workaround?
Comment 2 Paul B. Henson 2018-11-09 01:27:29 UTC
Download it by hand before you emerge 8-/?

Would sure be nice to get this fixed.
Comment 3 Sven E. 2018-11-09 02:36:17 UTC
(In reply to Paul B. Henson from comment #2)
> Download it by hand before you emerge 8-/?
> 
> Would sure be nice to get this fixed.

True, looking at the code, seems awfully broken. Trying to figure otu the logic and it just doesn't add up.
Comment 4 Larry the Git Cow gentoo-dev 2018-11-09 03:22:07 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=4264ac4b07c4555eb5db2ab21288dba05a7f4b6d

commit 4264ac4b07c4555eb5db2ab21288dba05a7f4b6d
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-11-09 03:15:49 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-11-09 03:21:55 +0000

    fetch: check DISTDIR write access as fetch user (bug 601252)
    
    Test DISTDIR write access as the fetch user, for correct results
    on NFS shares with root_squash enabled.
    
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 5 Zac Medico gentoo-dev 2018-11-09 04:37:08 UTC
If you'd like to test the patch, save it in a directory named /etc/portage/patches/sys-apps/portage-2.3.51/ an then `emerge -1 =portage-2.3.51`:

https://gitweb.gentoo.org/proj/portage.git/patch/?id=4264ac4b07c4555eb5db2ab21288dba05a7f4b6d
Comment 6 Sven E. 2018-11-09 20:04:04 UTC
(In reply to Zac Medico from comment #5)
> If you'd like to test the patch, save it in a directory named
> /etc/portage/patches/sys-apps/portage-2.3.51/ an then `emerge -1
> =portage-2.3.51`:
> 
> https://gitweb.gentoo.org/proj/portage.git/patch/
> ?id=4264ac4b07c4555eb5db2ab21288dba05a7f4b6d

Fixes the first problem, next one is this:
PermissionDenied: open('/usr/portage/distfiles/.ldc-1.4.0-src.tar.gz.portage_lockfile')

I added some output to fetch.py and UID: 0 EUID: 0 right before the lock is acquired, so, we'd want to set(e)uid before doing the actual work, I assume.
Comment 7 Zac Medico gentoo-dev 2018-11-09 21:00:14 UTC
(In reply to Sven E. from comment #6)
> (In reply to Zac Medico from comment #5)
> > If you'd like to test the patch, save it in a directory named
> > /etc/portage/patches/sys-apps/portage-2.3.51/ an then `emerge -1
> > =portage-2.3.51`:
> > 
> > https://gitweb.gentoo.org/proj/portage.git/patch/
> > ?id=4264ac4b07c4555eb5db2ab21288dba05a7f4b6d
> 
> Fixes the first problem, next one is this:
> PermissionDenied:
> open('/usr/portage/distfiles/.ldc-1.4.0-src.tar.gz.portage_lockfile')

You might use FEATURES="-distlocks" until we have a better solution.

> I added some output to fetch.py and UID: 0 EUID: 0 right before the lock is
> acquired, so, we'd want to set(e)uid before doing the actual work, I assume.

Yeah, we have a _LockProcess class that we could use to drop privileges here. Alternatively, we could have EbuildFetcher drop privileges for the whole fetch function call which is executed in a subprocess, as long as root privileges are not needed anywhere in the function.
Comment 8 Sven E. 2018-11-09 21:10:40 UTC
(In reply to Zac Medico from comment #7)
> (In reply to Sven E. from comment #6)
> > (In reply to Zac Medico from comment #5)
> > I added some output to fetch.py and UID: 0 EUID: 0 right before the lock is
> > acquired, so, we'd want to set(e)uid before doing the actual work, I assume.
> 
> Yeah, we have a _LockProcess class that we could use to drop privileges
> here. Alternatively, we could have EbuildFetcher drop privileges for the
> whole fetch function call which is executed in a subprocess, as long as root
> privileges are not needed anywhere in the function.

My POV: Drop privs as soon as possible. But, if you look at fetch.py around lines 524 and 533, root privs are needed to fix permission and create distdir.

The code looks pretty ugly to me anyway. I don't know about portage utils and helpers and stuff, so I can hardly advice on what way to go.

If I am getting you right EbuildFetcher means the whole fetch.py script. Doing that unpriv'ed doesn't seem to be doable right now (Path and File Permission Cleanups would need to move out of place completely).
Comment 9 Sven E. 2018-11-09 21:17:29 UTC
Addition:
FEATURES="-distlocks"

Doesn't work out, there's some os.unlink() call in fetch.py which again seems to be done with UID=0. (And yes, the unlink is triggered by a download problem, but that's another place where userfetch needs to be treated accordingly.)

Another thing I was wondering looking at the source: Does userpriv actually imply userfetch? Looking at the code I'm under the impression it does.
Comment 10 Zac Medico gentoo-dev 2018-11-10 01:31:56 UTC
(In reply to Sven E. from comment #9)
> Addition:
> FEATURES="-distlocks"
> 
> Doesn't work out, there's some os.unlink() call in fetch.py which again
> seems to be done with UID=0. (And yes, the unlink is triggered by a download
> problem, but that's another place where userfetch needs to be treated
> accordingly.)

Thanks for testing. We can easily add a function to drop privileges for each unlink call. If we try to drop privileges for the whole fetch function, then we'll have to test on selinux to check if the selinux PORTAGE_FETCH_T context switch works after we've dropped privileges.

> Another thing I was wondering looking at the source: Does userpriv actually
> imply userfetch? Looking at the code I'm under the impression it does.

It doesn't actually drop privileges here unless userfetch is enabled.
Comment 11 Zac Medico gentoo-dev 2018-11-19 06:37:28 UTC
I've tested on selinux with this patch, and it showed that fetch is possible after calling setgid and setuid prior to the portage_fetch_t domain transition:

> --- a/package/ebuild/fetch.py
> +++ b/package/ebuild/fetch.py
> @@ -77,8 +77,8 @@
>         if "userfetch" in settings.features and \
>                 os.getuid() == 0 and portage_gid and portage_uid and \
>                 hasattr(os, "setgroups"):
> -               kwargs.update(_userpriv_spawn_kwargs)
> -               logname = portage.data._portage_username
> +               os.setgid(int(portage_gid))
> +               os.setuid(int(portage_uid))
>  
>         spawn_func = spawn
Comment 12 Larry the Git Cow gentoo-dev 2018-11-19 06:43:20 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=58eb01022e604b0a538b9d67bd64daa44da9f94a

commit 58eb01022e604b0a538b9d67bd64daa44da9f94a
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-11-19 06:38:21 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-11-19 06:42:06 +0000

    Revert "fetch: check DISTDIR write access as fetch user (bug 601252)"
    
    This reverts commit 4264ac4b07c4555eb5db2ab21288dba05a7f4b6d.
    Dropping privileges for the access call is not useful by itself,
    since lock and unlink calls do not drop privileges yet. We might
    decide to drop privileges for the entire fetch function call.
    
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 13 Zac Medico gentoo-dev 2020-02-14 17:33:26 UTC
*** Bug 709606 has been marked as a duplicate of this bug. ***
Comment 14 Larry the Git Cow gentoo-dev 2020-02-15 20:28:57 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b07545003fed649b1a0a8a9e5cb69d5e2fa4951

commit 9b07545003fed649b1a0a8a9e5cb69d5e2fa4951
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-02-15 20:23:23 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-02-15 20:25:34 +0000

    fetch: split out _ensure_distdir function (bug 601252)
    
    Split out an _ensure_distdir function which will have to be
    called earlier if the fetch function is called with dropped
    privileges as discussed in bug 601252.
    
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/fetch.py | 102 +++++++++++++++++++-----------------
 1 file changed, 55 insertions(+), 47 deletions(-)
Comment 15 Larry the Git Cow gentoo-dev 2020-02-15 20:44:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d8d02bd82f11921db5140941cef4077359b0211d

commit d8d02bd82f11921db5140941cef4077359b0211d
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-02-15 20:37:39 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-02-15 20:42:24 +0000

    fetch: remove secpass import (bug 601252)
    
    Refer to the portage.secpass.data attribute directly rather than via
    import value, since the value will change after fork and os.setuid
    calls when the fetch function is called with dropped privileges as
    discussed in bug 601252.
    
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/fetch.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Comment 17 Larry the Git Cow gentoo-dev 2020-03-01 06:21:21 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=ec654122c0eb191c90ffb2c191403d342dbc361e

commit ec654122c0eb191c90ffb2c191403d342dbc361e
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-01 05:58:00 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-03-01 05:58:46 +0000

    fetch: drop privileges early for NFS root_squash (bug 601252)
    
    Drop privileges prior to fetch function calls, so that
    all necessary operations can succeed when DISTDIR is
    on NFS with root_squash enabled.
    
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/_emerge/EbuildFetcher.py           | 12 +++++++++++-
 lib/portage/package/ebuild/doebuild.py | 22 +++++++++++++++++-----
 lib/portage/package/ebuild/fetch.py    | 31 +++++++++++++++++++++++++++++++
 3 files changed, 59 insertions(+), 6 deletions(-)
Comment 18 Larry the Git Cow gentoo-dev 2020-03-01 07:02:03 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=196e51a0010cf17a3733fe6bc2516cf9e01a4a8a

commit 196e51a0010cf17a3733fe6bc2516cf9e01a4a8a
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-01 06:51:49 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-03-01 06:58:10 +0000

    sys-apps/portage: Bump to version 2.3.90
    
     #601252 DISTDIR NFS root_squash support
     #709746 new PORTAGE_LOG_FILTER_FILE variable specifies a
             command that filters build log output to a log file
     #710076 einstalldocs: Fix test for DOCS being unset
    
    Bug: https://bugs.gentoo.org/711148
    Bug: https://bugs.gentoo.org/601252
    Bug: https://bugs.gentoo.org/709746
    Bug: https://bugs.gentoo.org/710076
    Package-Manager: Portage-2.3.90, Repoman-2.3.20
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.90.ebuild | 271 +++++++++++++++++++++++++++++++++
 2 files changed, 272 insertions(+)
Comment 19 Larry the Git Cow gentoo-dev 2020-03-02 16:53:01 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=0bf72716cacc7f79abb61ad3ce33332c743809c9

commit 0bf72716cacc7f79abb61ad3ce33332c743809c9
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-02 16:49:33 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-03-02 16:51:15 +0000

    doebuild: avoid emerge --pretend --fetchonly event loop recursion
    
    Fixes: ec654122c0eb ("fetch: drop privileges early for NFS root_squash (bug 601252)")
    Bug: https://bugs.gentoo.org/601252
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
Comment 20 Pavol Cizmarik 2020-05-05 22:56:34 UTC
Fetch is not working with current version: sys-apps/portage-2.3.99-r2:0 with message:
!!! No write access to '/usr/portage/distfiles'

I have /usr/portage/distfiles mounted via NFS without root privs as the original bug from 2016. My permissions are set to portage:portage and 0775 for whole directory. I don't use userfetch feature. 

What I found is that os.access('/usr/portage/distfiles', os.W_OK) returns False if I try it. I can create dirs and files with portage user on NFS mount.
Older portage version was working without issues as it probably used root privs for fetch.

emerge --info:

Portage 2.3.99 (python 3.6.10-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-9.3.0, glibc-2.30-r8, 5.5.19-gentoo-x86_64 x86_64)
=================================================================
System uname: Linux-5.5.19-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i5-6600K_CPU_@_3.50GHz-with-gentoo-2.7
KiB Mem:    16336860 total,   7844476 free
KiB Swap:          0 total,         0 free
Head commit of repository gentoo: 72d7d2e8b3e7e83e13fe0d6cbb28349c7b6fd875
                                                               
sh bash 5.0_p17          
ld GNU ld (Gentoo 2.34 p1) 2.34.0                           
distcc 3.3.3 x86_64-pc-linux-gnu [disabled]
app-shells/bash:          5.0_p17::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.30.1::gentoo
dev-lang/python:          2.7.18::gentoo, 3.6.10-r2::gentoo, 3.7.7-r2::gentoo, 3.8.2-r2::gentoo
dev-util/cmake:           3.17.1::gentoo
dev-util/pkgconfig:       0.29.2::gentoo        
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.18::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.11.6-r3::gentoo, 1.13.4-r2::gentoo, 1.16.2::gentoo
sys-devel/binutils:       2.34::gentoo
sys-devel/gcc:            9.3.0::gentoo
sys-devel/gcc-config:     2.2.1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.2.1-r4::gentoo                                                                                                                                                                                                                   
sys-kernel/linux-headers: 5.6::gentoo (virtual/os-headers)                                                                                                                                                                                                   sys-libs/glibc:           2.30-r8::gentoo                                                                                                                                                                                                                    
Repositories:                     
                                                               
gentoo                                                                                                                                                                                                                                                       
    location: /usr/portage
    sync-type: git                                                                                                                                                                                                                                               sync-uri: git://anongit.gentoo.org/repo/gentoo.git                                                                        
    priority: -1000
                                                                                                                                                                                                                                                             
crossdev         
    location: /usr/local/portage/crossdev
    masters: gentoo
    priority: 10
                                                               
gcpan                 
    location: /usr/local/portage/gcpan                                                                                                                                                                                                                       
    masters: gentoo      
    priority: 10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          ace                                                                                                                                                                                                                                                              location: /var/lib/layman/ace                                                                                                                                                                                                                                sync-type: laymansync                                                                                                                                                                                                                                        sync-uri: https://github.com/ananace/overlay.git                                                                                                                                                                                                             masters: gentoo                                                                                                                                                                                                                                              priority: 50                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          cizo2000                                                                                                                                                                                                                                                         location: /var/lib/layman/cizo2000                                                                                                                                                                                                                           sync-type: laymansync                                                                                                                                                                                                                                        sync-uri: git://github.com/cizo2000/gentoo-overlay.git                                                                                                                                                                                                       masters: gentoo                                                                                                                                                                                                                                              priority: 50              
                                                                                                                                                                                                                                                             
emc                                                                                                                                                                                                                                                          
    location: /var/lib/layman/emc
    sync-type: laymansync
    sync-uri: https://github.com/emcek/gentoo.mipl.git
    masters: gentoo
    priority: 50

perl-experimental
    location: /var/lib/layman/perl-experimental
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/perl-overlay.git
    masters: gentoo
    priority: 50

raiagent
    location: /var/lib/layman/raiagent
    sync-type: laymansync
    sync-uri: https://github.com/leycec/raiagent
    masters: gentoo
    priority: 50

Installed sets: @system
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /etc/stunnel/stunnel.conf /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.3/ext-active/ /etc/php/apache2-php7.4/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/cgi-php7.4/ext-ac$
ive/ /etc/php/cli-php7.3/ext-active/ /etc/php/cli-php7.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=native -pipe"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs qa-unresolved-soname$
deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="rsync://ftp6.linux.cz/pub/linux/gentoo/ ftp://ftp.fi.muni.cz/pub/linux/gentoo/ rsync://ftp.fi.muni.cz/pub/linux/gentoo/ ftp://tux.rainside.sk/gentoo/ ftp://gentoo.wheel.sk/pub/linux/gentoo/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en sk"
MAKEOPTS="-j5"
PKGDIR="/var/cache/binpkgs"       
PORTAGE_CONFIGROOT="/"          
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="64bit X a52 aac aalib accessibility acct acl acpi activities alaw alisp alsa amd64 amrr ao apache2 apm apng autoipd autoreplace avahi bash-completion bazaar berkdb bittorrent bluetooth branding bzip2 cairo cdda cddb cdio cdr cdrom cgi cleartype cli
 client-libs clucene contactnotes crossdev crypt css ctype curl curlwrappers cvs cvsgraph dbus declarative device-mapper dhclient djvu dri dssi dts dv dvd dvdr egl eigen elogind emboss encode equalizer evdev exif expat extensions extras faac faad fbcon 
fbcondecor ffmpeg flac fontconfig fortran ftdi ftp fts3 fuse gbm gcrypt gd gdbm gif git gles glitz gmp gpl gpm gsm gtalk gtk gtkstyle gudev hddtemp hfs highlight history hwdb iconv icq icu id3tag idn imagemagick imlib initramfs injection inotify introsp
ection iproute2 ipv6 irc irda jabber jack java javascript jce jingle jpeg jpeg2k kde kdrive kipi kwallet l7filter ladspa lame lapack lcms ldap libcaca libextractor libkms libnotify libtirpc llvm lm_sensors logrotate mad mdadm mercurial midi minizip mjpe
g mng motif mozilla mp3 mp4 mpeg mpi mplayer mtp multilib musicbrainz mysql ncurses nls nptl nsplugin ntfs ntfsprogs obex ogg onoe openexr opengl openmp openssl oscar otr pam pango pcf pcmcia pcntl pcre pdf perl phonon php plasma png policykit postgres 
ppds pulseaudio python q32 q8 qml qt3support qt5 rar raw rdesktop rdp readline redeyes resid resolvconf romio rpc rsync rtsp samba sasl script sdl sdl-image seccomp secure-delete semantic-desktop shout sift sip skins slang slp smi sms sndfile snmp socke
ts sox spell split-usr sql sqlite sqlite3 sse3 sse4a ssl ssse3 startup-notification statistics stream subversion svg sysfs syslog taglib tcl tcpd texteffect threads thumbnail tiff timidity tk transparent-proxy truetype udev udisks ulaw unicode unsupport
ed upnp upower usb userlocales utempter v4l v4l2 vcd vhosts video vim-syntax visualization vlm vnc vorbis wav wayland widgets wifi winpopup wxwidgets wxwindows x264 xattr xcb xcomposite xephyr xforms xine xinerama xinetd xml xosd xrandr xulrunner xv xvf
b xvid yahoo zlib zsh-completion" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel usb-audio mp401 virmidi seq-dummy" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_core authn_dbm authn_file authz_core authz_dbm authz_gr
oupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif socache_shmcb speling status u
nique_id unixd userdir usertrack vhost_alias proxy proxy_ajp proxy_connect proxy_fcgi proxy_ftp proxy_html proxy_http proxy_scgi proxy_wstunnel xml2enc" APACHE2_MPMS="itk" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load m
emory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oc
eanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="pr
esenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby26 ruby27" SANE_BACKEND
S="hp" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"                    
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 21 Zac Medico gentoo-dev 2020-05-06 02:00:30 UTC
(In reply to Pavol Cizmarik from comment #20)
> Fetch is not working with current version: sys-apps/portage-2.3.99-r2:0 with
> message:
> !!! No write access to '/usr/portage/distfiles'

If it worked with portage-2.3.89-r3, with otherwise same configuration, then I'm not sure how. It should run as root if you have FEATURES=userfetch disabled, regardless of portage version.

> I have /usr/portage/distfiles mounted via NFS without root privs as the
> original bug from 2016. My permissions are set to portage:portage and 0775
> for whole directory. I don't use userfetch feature. 

If you enable FEATURES=userfetch then I think it should work for you.

> What I found is that os.access('/usr/portage/distfiles', os.W_OK) returns
> False if I try it. I can create dirs and files with portage user on NFS
> mount.

Yes, that's why I think FEATURES=userfetch should work for you.

> Older portage version was working without issues as it probably used root
> privs for fetch.

That's not supposed to work for people with root_squash enabled, which is the rationale for the change in portage-2.3.99-r2.
Comment 22 Pavol Cizmarik 2020-05-06 18:20:29 UTC
(In reply to Zac Medico from comment #21)
> (In reply to Pavol Cizmarik from comment #20)
> > Fetch is not working with current version: sys-apps/portage-2.3.99-r2:0 with
> > message:
> > !!! No write access to '/usr/portage/distfiles'
> 
> If it worked with portage-2.3.89-r3, with otherwise same configuration, then
> I'm not sure how. It should run as root if you have FEATURES=userfetch
> disabled, regardless of portage version.
> 
> > I have /usr/portage/distfiles mounted via NFS without root privs as the
> > original bug from 2016. My permissions are set to portage:portage and 0775
> > for whole directory. I don't use userfetch feature. 
> 
> If you enable FEATURES=userfetch then I think it should work for you.

My fault I checked FEATURES again and userfetch was enabled by default.

> > What I found is that os.access('/usr/portage/distfiles', os.W_OK) returns
> > False if I try it. I can create dirs and files with portage user on NFS
> > mount.
> 
> Yes, that's why I think FEATURES=userfetch should work for you.
> 
> > Older portage version was working without issues as it probably used root
> > privs for fetch.
> 
> That's not supposed to work for people with root_squash enabled, which is
> the rationale for the change in portage-2.3.99-r2.

I have set FEATURES to -userfetch and fetch is working.

Current FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userpriv usersandbox usersync xattr"

Settings for distfiles export are (I use Synology DS):

/volume1/gentoo-distfiles       192.168.252.0/24(rw,sync,no_wdelay,insecure,no_root_squash,insecure_locks,sec=sys,anonuid=1025,anongid=100)

Mounted as:

192.168.252.9:/volume1/gentoo-distfiles on /usr/portage/distfiles type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.252.10,local_lock=none,addr=192.168.252.9,_netdev)

Do I have something set incorrectly?
Comment 23 Zac Medico gentoo-dev 2020-05-08 19:11:35 UTC
(In reply to Pavol Cizmarik from comment #22)
> Mounted as:
> 
> 192.168.252.9:/volume1/gentoo-distfiles on /usr/portage/distfiles type nfs4
> (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,
> timeo=600,retrans=2,sec=sys,clientaddr=192.168.252.10,local_lock=none,
> addr=192.168.252.9,_netdev)
> 
> Do I have something set incorrectly?

Please test if a shell running as the portage user has write access:

su -s /bin/bash portage
[[ -w /usr/portage/distfiles ]] && echo yes || echo no
Comment 24 Pavol Cizmarik 2020-05-10 20:18:13 UTC
(In reply to Zac Medico from comment #23)
> (In reply to Pavol Cizmarik from comment #22)
> > Mounted as:
> > 
> > 192.168.252.9:/volume1/gentoo-distfiles on /usr/portage/distfiles type nfs4
> > (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,
> > timeo=600,retrans=2,sec=sys,clientaddr=192.168.252.10,local_lock=none,
> > addr=192.168.252.9,_netdev)
> > 
> > Do I have something set incorrectly?
> 
> Please test if a shell running as the portage user has write access:
> 
> su -s /bin/bash portage
> [[ -w /usr/portage/distfiles ]] && echo yes || echo no

Test result is "no".
It's strange, because I can create files, directories with "portage" user in distfiles directory without issues. It seems I have something wrong.