| Summary: | portage should create locks for downloading in DISTDIR | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | emdee_is <emdee_is> |
| Component: | Core | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED WORKSFORME | ||
| Severity: | normal | CC: | jstein |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
See also: https://bugs.gentoo.org/884071 As I understand it, one of tne of the purposes of this kind of locking is to prevent multiple Portages from writing the same file, yielding a corrupted distfile. If the lockfile isn't written to the same directory as the distfile, how can the locking possibly be useful?
Also, let's not forget the context here: you seem to have symlinks from DISTDIR to locations outside DISTDIR which aren't actually writeable. Thus, why not use PORTAGE_RO_DISTDIRS for DISTDIRs which shouldn't be written to?
>>> Emerging (1 of 4) media-libs/virglrenderer-0.10.1::gentoo
Process Process-11:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/portage/locks.py", line 248, in _lockfile_iteration
myfd = os.open(lockfilename, os.O_CREAT | os.O_RDWR, 0o660)
File "/usr/lib/python3.10/site-packages/portage/__init__.py", line 281, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs)
PermissionError: [Errno 13] Permission denied: b'/usr/portage/distfiles/.virglrenderer-0.10.1.tar.gz.portage_lockfile'
# ls -ld /usr/portage/distfiles/
drwxr-xr-x 4 portage portage 1445888 Dec 2 16:55 /usr/portage/distfiles/
# ls -l /usr/portage/distfiles/.virglrenderer-0.10.1.tar.gz.portage_lockfile
lrwxrwxrwx 1 root root 90 Oct 24 16:23 /usr/portage/distfiles/.virglrenderer-0.10.1.tar.gz.portage_lockfile -> /i/net/Http/distfiles.gentoo.org/distfiles/./.virglrenderer-0.10.1.tar.gz.portage_lockfile
# ls -l /usr/portage/distfiles/.virglrenderer-0.10.1.tar.gz
ls: cannot access '/usr/portage/distfiles/.virglrenderer-0.10.1.tar.gz': No such file or directory
# ls -l /usr/portage/distfiles/virglrenderer-0.10.1.tar.gz
lrwxrwxrwx 1 root root 73 Nov 4 02:40 /usr/portage/distfiles/virglrenderer-0.10.1.tar.gz -> ../../net/Http/distfiles.gentoo.org/distfiles/virglrenderer-0.10.1.tar.gz
As I understand it, the locking in DISTDIR prevents multiple Portages from writing the same file, yielding a corrupted distfile. The lockfile written to DISTDIR will prevent multiple downloading of the distfile, and is thus very useful. Portage will be downloading to DISTDIR so the lock should be in DISTDIR. With the portage code as-is it fails on the lockfile generation as I think it generates an illegal filename for an NTFS filesystem, where the symlink points to. With the code fixed to lock in DISTDIR, it renames the symlink linking to a broken download to /usr/portage/distfiles/virglrenderer-0.10.1.tar.gz._checksum_failure_.typrg33e and redownloads to DISTDIR as intended. I don't think PORTAGE_RO_DISTDIRS is appropriate here, and I'm not sure it doesn't have issues of it's own https://bugs.gentoo.org/752984 Sorry, but I don't really follow what you are proposing we change here. Given an input of mypath = "/usr/portage/distfiles/virglrenderer-0.10.1.tar", both the original code and your modified code produce the same result: lockfilename = "/usr/portage/distfiles/.virglrenderer-0.10.1.tar.portage_lockfile" The added dirname and basename calls are entirely pointless. |
If a file in /usr/portage/distfiles is a symlink, portage currently tries to write a lockfile in the directory where the symlink points to. L234 of portage/locks.py: ``` base, tail = os.path.split(mypath) lockfilename = os.path.join(base, "." + tail + ".portage_lockfile") ``` The lockfile should be on DISTDIR where the symlink is: ``` dirname = os.path.dirname(mypath) filename = os.path.basename(mypath) base, tail = os.path.split(filename) lockfilename = os.path.join(dirname, base, "." + tail + ".portage_lockf\ ile") ``` I'm not sure if a filename like ```.virglrenderer-0.10.1.tar.gz.portage_lockfile``` is legal on all filesystems that the symlink may point to, and portage may not have write permissions there. The filename in /usr/portage/distfiles is unique so the lockfilename is unique.