I have a central server with NFS shares---one being portage and another being a data share where distfiles are stored---and all other machines mount server:/usr/portage read-only and server:/usr/local/share/data read/write. All machines are configured to store distfiles in /usr/local/share/data/pub/linux/gentoo/distfiles and binary packages in /usr/local/share/data/pub/linux/gentoo/packages/${ARCH}. I have sudo set up to run portage as root only for certain users and those users are a member of the portage group, plus, that group has rwx permissions on $DISTDIR. I added root to the portage group because all NFS shares have the option root_squash defined and this was the only way to get distfiles downloaded. Running 'sudo emerge somepackage' or 'sudo emerge -f somepackage' on NFS client machines causes the error "No write access to $DISTDIR" and stops the emerge if the source files have not been downloaded. If I run 'emerge -f somepackage' as a portage group member, it downloads the packages just fine so I know portage members have write access to the NFS mounted $DISTDIR. If I then run 'sudo emerge -b somepackage' as a portage group member, it builds the package but errors out with "OSError: [Errno 13] Permission denied: '$PKGDIR/${CATEGORY}'" or "mv: cannot create regular file `$PKGDIR/All/${PACKAGE}.tbz2': Permission denied" when trying to copy the binary package to $PKGDIR. I emerge portage-2.0.51.22-r1 to see if it had a fix for the problem but apparently it does not. This used to work as expected on client machines but recently emerge started returning the errors above. These errors prevent client machines from emerging packages unattented and require that I baby sit them for anywhere from 1 to 16 hours when trying to build binary packages. The only thing major I have done is update the /etc/make.profile link to a newer profile as instructed by emerge. The /etc/make.profile link now points to ../usr/portage/profiles/default-linux/x86/2005.0/. Reproducible: Always Steps to Reproduce: 1. NFS mount /usr/portage on a client machine ro. 2. Point $DISTDIR to path on another NFS share with root_squash and mount that share on the client machine rw. 3. Point $PKGDIR to a path on the same NFS share as in step 2. 4. Run 'emerge somepackage', 'emerge -f somepackage', 'emerge -b somepackage', 'emerge -Du world', 'emerge -Duf world', or 'emerge -Dub world' on the NFS client machine as root or set up sudo to run them under an unprivileged user. Actual Results: emerge reports one of the following errors and exits. "No write access to $DISTDIR" "OSError: [Errno 13] Permission denied: '$PKGDIR/${CATEGORY}'" "mv: cannot create regular file `$PKGDIR/All/${PACKAGE}.tbz2': Permission denied" Expected Results: emerge should download, compile, install and/or create a binary package unattented for the selected package or all packages when updating world. Here is the output of 'emerge info': Portage 2.0.51.22-r1 (default-linux/x86/2005.0, gcc-3.3.5-20050130, glibc-2.3.4.20041102-r1, 2.6.11-gentoo-r6 i686) ================================================================= System uname: 2.6.11-gentoo-r6 i686 Unknown CPU Type Gentoo Base System version 1.4.16 distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.3 [enabled] dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.8 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.4 sys-devel/binutils: 2.15.92.0.2-r7 sys-devel/libtool: 1.5.16 virtual/os-headers: 2.6.8.1-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=athlon-xp -m3dnow -mmmx -msse -O2 -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/X11/gdm /etc/X11/rstart/rstartd.real /etc/X11/serverconfig /etc/X11/starthere /etc/X11/sysconfig /etc/X11/xdm/chooser /etc/X11/xorg.conf.example /etc/gconf /etc/gnome-vfs-2.0/modules /etc/hotplug /etc/init.d /etc/openldap/schema /etc/sound/event /etc/terminfo /usr/X11R6/lib/X11/xkb /etc/env.d" CXXFLAGS="-march=athlon-xp -m3dnow -mmmx -msse -O2 -fomit-frame-pointer" DISTDIR="/usr/local/share/data/pub/linux/gentoo/distfiles" FEATURES="autoconfig buildpkg ccache distcc distlocks sandbox sfperms strict userpriv usersandbox" GENTOO_MIRRORS="http://gentoo.oregonstate.edu/ http://www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="en_US" LINGUAS="en" MAKEOPTS="-j4" PKGDIR="/usr/local/share/data/pub/linux/gentoo/packages/athlon-xp" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/share/data/pub/linux/gentoo/portage" SYNC="rsync://charge.electrostatic.org/gentoo-portage" USE="x86 3dnow 3dnowext X Xaw3d acl alsa apm arts audiofile avi berkdb bitmap-fonts bonobo cdparanoia cdr crypt cups curl dga divx4linux dv dvd dvdr dvdread eds emboss encode esd evms2 evo fam flac foomaticdb gd gdbm gif gnome2 gpm gstreamer gtk gtk2 gtkhtml guile imagemagick imlib ipv6 java jpeg kde kdeenablefinal ldap libg++ libwww live mad mmx mmx2 mmxext motif mozilla mp3 mpeg mysql nas ncurses network nvidia offensive ofx ogg oggvorbis opengl pam pdflib perl png python qt quicktime quotes readline samba scanner sdl spell sse ssl svga tcltk tcpd tetex tiff truetype truetype-fonts type1-fonts usb userlocales v4l v4l2 vorbis xine xml xml2 xmms xscreensaver xv xvid xvmc zlib linguas_en userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LC_ALL, LDFLAGS
Oh, I reported this on bug 69763 (http://bugs.gentoo.org/show_bug.cgi?id=69763) but was told my issue is completely unrelated.
What happens when you do this stuff as root?
I get the same errors when running as root on the NFS client machines. host1 root # emerge chickens Calculating dependencies ...done! >>> emerge (1 of 2) media-libs/allegro-4.0.3 to / !!! No write access to /usr/local/share/data/pub/linux/gentoo/distfiles/ !!! File allegro-4.0.3.tar.gz isn't fetched but unable to get it. ----- host1 root # emerge -f chickens Calculating dependencies ...done! >>> emerge (1 of 2) media-libs/allegro-4.0.3 to / !!! No write access to /usr/local/share/data/pub/linux/gentoo/distfiles/ !!! File allegro-4.0.3.tar.gz isn't fetched but unable to get it. !!! Fetch for /usr/portage/media-libs/allegro/allegro-4.0.3.ebuild failed, continuing... >>> emerge (2 of 2) games-action/chickens-0.2.4 to / !!! No write access to /usr/local/share/data/pub/linux/gentoo/distfiles/ !!! File ChickensForLinux-Linux-0.2.4.tar.gz isn't fetched but unable to get it. !!! Fetch for /usr/portage/games-action/chickens/chickens-0.2.4.ebuild failed, continuing... !!! Some fetch errors were encountered. Please see above for details. ----- host1 root # ls -ld /usr/local/share/data/pub/linux/gentoo/distfiles drwxrwsr-x 4 user portage 61544 May 27 00:21 /usr/local/share/data/pub/linux/gentoo/distfiles/ host1 root # touch /usr/local/share/data/pub/linux/gentoo/distfiles/bogusfile touch: cannot touch `/usr/local/share/data/pub/linux/gentoo/distfiles/bogusfile': Permission denied Interesting. I seem to remember this used to work if root was at least a member of the portage group and even if root's primary group was not portage. ----- host1 root # sg - portage host1 root # id uid=0(root) gid=250(portage) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),18(audio),20(dialout),26(tape),27(video),250(portage) host1 root # touch /usr/local/share/data/pub/linux/gentoo/distfiles/bogusfile host1 root # ls -l /usr/local/share/data/pub/linux/gentoo/distfiles/bogusfile -rw-r--r-- 1 4294967294 portage 0 May 27 00:27 /usr/local/share/data/pub/linux/gentoo/distfiles/bogusfile Interesting to note the file's owner was set to 4294967294 instead of nobody because of the root_squash. What the hell? Did older versions of portage set the group to portage when emerging a package and now newer version do not? ----- I did 'emerge -f chickens' as a unprivileged user, which is a member of the portage group, to fetch all the sources. ----- host1 root # id uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),18(audio),20(dialout),26(tape),27(video),250(portage) host1 root # emerge -b chickens Calculating dependencies ...done! >>> emerge (1 of 2) media-libs/allegro-4.0.3 to / !!! No write access to /usr/local/share/data/pub/linux/gentoo/distfiles/ >>> md5 files ;-) allegro-4.0.3.ebuild >>> md5 files ;-) allegro-4.1.14.ebuild >>> md5 files ;-) allegro-4.1.18.ebuild >>> md5 files ;-) files/digest-allegro-4.0.3 >>> md5 files ;-) files/digest-allegro-4.1.14 >>> md5 files ;-) files/digest-allegro-4.1.18 >>> md5 src_uri ;-) allegro-4.0.3.tar.gz >>> Unpacking source... >>> Unpacking allegro-4.0.3.tar.gz to /var/tmp/portage/allegro-4.0.3/work >>> Source unpacked. ... >>> Install allegro-4.0.3 into /var/tmp/portage/allegro-4.0.3/image/ category media-libs ... >>> Completed installing allegro-4.0.3 into /var/tmp/portage/allegro-4.0.3/image/ ... mv: cannot create regular file `/usr/local/share/data/pub/linux/gentoo/packages/athlon-xp/All/allegro-4.0.3.tbz2': Permission denied !!! ERROR: media-libs/allegro-4.0.3 failed. !!! Function dyn_package, Line 954, Exitcode 1 !!! Failed to move tbz2 to /usr/local/share/data/pub/linux/gentoo/packages/athlon-xp/All !!! If you need support, post the topmost build error, NOT this status message. ----- At this point I would have to manually copy /var/tmp/portage/allegro-4.0.3/allegro-4.0.3.tbz2 to /usr/local/share/data/pub/linux/gentoo/packages/athlon-xp/All, run 'emerge -k chickens', wait for portage to fail when trying to move the binary package for chickens, and rinse then repeat. You can imagine how frustrating this gets when there are many packages involved. I migrated my server to Linux 2.6 on May 11 but I seem to remember I did not have problems like this after the migration. If I get the time, I might just pull out one of the older Gentoo install CDs and install it on a machine to test whether the problems lie with the server or portage on the client machines.
I believe older versions didn't touch the permissions at all. Portage started changing the group of fetched files once fetching as a user became supported, but I don't believe it has ever touched the owner. However, after re-checking the code the user is specified as -1 which is meant to imply that the user is left as is. Apparently something somewhere is interpreting that as being the actual user id. As for binary packages, nothing has changed in a long time.
I just remembered something while submitting another bug. When managing my DISTDIR and PKGDIR, I seem to remember files used to be owned by portage:portage. Did portage used to fetch files as the portage user but was changed recently?
IIRC, if you have FEATURES="userpriv" ( set by default on most arches ) then in newer versions of portage, it will fetch as the portage user, if you have that off, it will fetch as root.
(In reply to comment #6) > IIRC, if you have FEATURES="userpriv" ( set by default on most arches ) then in > newer versions of portage, it will fetch as the portage user, if you have that > off, it will fetch as root. Apparently not in my case because I have userpriv in my features and portage continues to report "No write access to $DISTDIR". The make.conf man page states: userpriv Allow portage to drop root privledges and compile packages as portage:portage without a sandbox (unless usersandbox is also used). ----- But, does not mention anything about fetching files and saving binary packages as portage. I don't know python but I can understand programming and from what I can tell the spawn function in /usr/lib/portage/pym/portage_exec.py eventually gets called by spawn_bash or spawn_sandbox and spawn is where the fetch command is run with the defined uid, gid, groups and umask. Now, I see the spawn function in /usr/lib/portage/pym/portage.py checks if droppriv, portage_gid, portage_uid are defined; uid is not defined; updates uid, gid, groups and umask; then calls spawn_bash or spawn_sandbox in portage_exec.py. From what I can tell, this indicates that userpriv should indeed run fetch commands as portage, however, in my case this does not appear to happen. ... I think I found the problem. I'm looking at portage.py and on lines 1768 - 1771 the fetch function appears to check the current user for write access to DISTDIR and if the user does not have write access sets can_fetch to False. Further down on lines 1871 - 1877, can_fetch is tested then one of two messages is written to the console and return is called if can_fetch is false. One of those messages---"!!! File %s isn't fetched but unable to get it."---is one of the last errors reported by emerge before it quits. Shouldn't the fetch function in portage.py check to see if the portage user has write access to DISTDIR when userpriv is set instead of always checking the current user? Actually, shouldn't all checks for read and/or write access be performed on the portage user when using userpriv and writing files as portage? After all, root may not have write access to DISTDIR when DISTDIR is a central nfs share with the root_squash option or other network file system that can limit root access.
Correction: Shouldn't the fetch function in portage.py check to see if the portage user has write access to DISTDIR when userpriv is set **and emerge is being run as root** instead of always checking the current user? Actually, shouldn't all checks for read and/or write access be performed on the portage user when using userpriv and writing files as portage **and emerge is being run as root**?
Any of the developers have input on comments #7 and #8?
Seems like an awful lot of trouble for nfs being it's usual friendly self ;) Bit ambivalent on this one. If ran as root, it should attempt to fix perms, but in your case it's not possible, except under userpriv. This is a bit of an inversion of the normal case; I'd wonder how wide spread this is, and why you're not collapsing root down to portage user on the nfs mount...
FEATURES="userfetch", how about closing this?
(In reply to comment #11) > FEATURES="userfetch", how about closing this? Yeah, userfetch has been enabled by default for some time now and that probably solves it. Please reopen if there is still a problem with DISTDIR, or file a new bug if there is a separate PKGDIR issue.
This is reoccurring here using NFS4 >>> Emerging (1 of 1) sys-apps/hal-0.5.11-r1 to / Traceback (most recent call last): File "/usr/bin/emerge", line 6971, in <module> retval = emerge_main() File "/usr/bin/emerge", line 6965, in emerge_main myopts, myaction, myfiles, spinner) File "/usr/bin/emerge", line 6395, in action_build retval = mergetask.merge(pkglist, favorites, mtimedb) File "/usr/bin/emerge", line 3981, in merge return self._merge(mylist, favorites, mtimedb) File "/usr/bin/emerge", line 4259, in _merge prev_mtimes=ldpath_mtimes) File "/usr/lib/portage/pym/portage.py", line 4676, in doebuild fetchme, mysettings, listonly=listonly, fetchonly=fetchonly): File "/usr/lib/portage/pym/portage.py", line 3107, in fetch if portage_util.ensure_dirs(mydir, gid=portage_gid, mode=dirmode, mask=modemask): File "/usr/lib/portage/pym/portage_util.py", line 872, in ensure_dirs perms_modified = apply_permissions(dir_path, *args, **kwargs) File "/usr/lib/portage/pym/portage_util.py", line 589, in apply_permissions os.chown(filename, uid, gid) OSError: [Errno 22] Invalid argument: '/usr/portage/distfiles/' /etc/make.conf: FEATURES="nostrip -distlocks -fixpackages ccache sandbox userfetch userpriv usersandbox loadpolicy"
Created attachment 161800 [details, diff] adjust permissions only when necessary If this patch is saved as /tmp/distdir-perms.patch, then it can be applied as follows: patch /usr/lib/portage/pym/portage/__init__.py /tmp/distdir-perms.patch
This is fixed in 2.2_rc6.
How about back porting the fix to portage 2.1 or is that not possible.
Yes, I will include this in the 2.1.5.7 release that I'm working on.
