>>> Emerging (1 of 35) app-text/recode-3.6-r1 to / >>> Downloading torpage://pug.lan/gentoo/distfiles/recode-3.6.tar.gz Torpage is initiating fetch for recode-3.6.tar.gz (app-text/recode-3.6-r1) This is torpage on pug (protocol level 0.2) using DATADIR=/usr/portage/app-text/recode Locking for recode-3.6.tar.gz. Obtained lock, proceeding to download Successfully fetched files for app-text/recode-3.6-r1 Traceback (most recent call last): File "/usr/bin/emerge", line 3524, in ? mydepgraph.merge(pkglist) File "/usr/bin/emerge", line 2056, in merge retval=portage.doebuild(y,"merge",myroot,self.pkgsettings,edebug,tree="porttree") File "/usr/lib/portage/pym/portage.py", line 2944, in doebuild if need_distfiles and not fetch(fetchme, mysettings, listonly=listonly, fetchonly=fetchonly): File "/usr/lib/portage/pym/portage.py", line 2250, in fetch os.chmod(mysettings["DISTDIR"]+"/"+myfile,0664) OSError: [Errno 30] Read-only file system: '/usr/portage/distfiles/recode-3.6.tar.gz' xacatecas input # ls -l /usr/portage/distfiles/recode-3.6.tar.gz -rw-rw-r-- 1 root portage 1751886 Mar 24 2003 /usr/portage/distfiles/recode-3.6.tar.gz xacatecas input # emerge --info Portage 2.1-r1 (default-linux/x86/2006.0, gcc-3.3.6, glibc-2.3.6-r4, 2.6.15.4 i686) ================================================================= System uname: 2.6.15.4 i686 Pentium III (Katmai) Gentoo Base System version 1.6.15 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] app-admin/eselect-compiler: [Not Present] dev-lang/python: 2.3.5-r2, 2.4.3-r1 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: [Not Present] dev-util/confcache: [Not Present] sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.13-r3 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r5 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=pentium3 -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-O2 -march=pentium3 -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig metadata-transfer sandbox sfperms skiprocheck strict" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://pug.lan/gentoo-portage" USE="x86 apache2 apm avi berkdb bitmap-fonts bzip2 cli crypt cups dlloader dri eds emboss encode foomaticdb gdbm gif gpm gstreamer imlib isdnlog jpeg kde ldap libg++ libwww logrotate mad mikmod mp3 mpeg mysql ncurses nptl nptlonly ogg pam pcre pdflib png pppd readline reflection samba session spl ssl truetype truetype-fonts type1-fonts udev userlocales v4l vorbis xml xmms xorg zlib zvbi elibc_glibc input_devices_keyboard input_devices_mouse input_devices_evdev kernel_linux userland_GNU" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS, PORTAGE_RSYNC_EXTRA_OPTS xacatecas input # Should be trivial to make the process fixing the ownership and permissions on DISTFILES simply not happen if we are on a read-only filesystem. Yes, I am running systems with read-only nfs mounted /usr/portage/distfiles directories. I won't go into the reasoning behind this here but I would appreciate it if someone can fix this issue.
Fix what? That you are trying to download files to read-only mounted-filesystem and wondering why it fails? Joking?
Nope. From my /etc/make.conf: FETCHCOMMAND="/bin/false" RESUMECOMMAND="/bin/false" FETCHCOMMAND_TORPAGE="/usr/sbin/torpage_fetch \${URI} \${DISTDIR}" RESUMECOMMAND_TORPAGE="/usr/sbin/torpage_fetch \${URI} \${DISTDIR}" FEATURES="skiprocheck -distlocks" And then in /etc/portage/mirrors: local torpage://pug.lan/gentoo :). I cooked up torpage a long, long time ago before distlocks even existed in order to allow multiple machines to download onto a single nfs mounted distfiles directory. And it turns out that torpage is still a handy tool as it allows you to restrict files that are going to be downloaded to those actually in portage. And yes, this actually used to work :p
NFC what's torpage; and no, you can't fetch anything to readonly filesystem and I really fail to see why is this a bug.
Basically the bug with this, is what was stated: "Should be trivial to make the process fixing the ownership and permissions on DISTFILES simply not happen if we are on a read-only filesystem." Using some really ugly portage hacks (on the fetch command to use cp instead of ftp and mounting the filesystem rw and ro) the latest portage-2.1.1_pre4-r2 this doesn't seem to break anymore And i understand the idea behind torpage, since i wrote a similar system basically one distfile repository for an entire network, then you grab the files off the local mirror before trying any other mirror, even with fetch restrictions A better way of doing this would be to hook the portage.fetch function in portage, although i'm not sure how the developers would like to add hooks into the fetch system, because it does away with a lot of the checks in it
My understanding was that the FETCHCOMMAND_* config options was the method to hook into the fetch system. There is imho no reason to not perform the check for a read-only fs and only update permissions/ownership if we are on a read-write filesystem. We could _only_ have gotten to that point if the skiprocheck FEATURE was enabled since when it's disabled (as is the default) the fetch command would bail out with "Trying to fetch to a read-only filesystem" long before it even gathers a list of possible mirrors.
Created attachment 93533 [details, diff] DISTLOCKS_SUBDIR.patch This patch allows portage to use a DISTLOCKS_SUBDIR feature from /etc/make.conf ... not sure where or how to set a default of ".lock". It also modifies the fetch() function to interpret this only as a relative path if it does not start with a slash (in which case it's relative to DISTDIR). Whilst it doesn't solve this bug it's required to make use of parallel-fetch which relies on distlocks, which in this case of torpage will get locked on the server so it doesn't really matter but this is a simpler solution.
Created attachment 93534 [details, diff] patch against portage-2.1-r1 This patch also includes the default of .locks since the default of settings['DISTLOCKS_DIR'] == '' And it fixes the issue involved as well
Closing, don't seem to have this issue any more.
