This involving soft-blocker file collision resolution for bind-tools-9.10.3_p2 and net-dns/bind-9.10.3_p2 was reported in bug 576786, comment #2: >>> Emerging binary (278 of 410) net-dns/bind-tools-9.10.3_p2::gentoo >>> Emerging binary (279 of 410) net-dns/bind-9.10.3_p2::gentoo >>> Installing (279 of 410) net-dns/bind-9.10.3_p2::gentoo >>> Installing (278 of 410) net-dns/bind-tools-9.10.3_p2::gentoo ..... >>> Jobs: 284 of 410 complete, 1 failed Load avg: 2.90, 2.90, 1.97 >>> Extracting info * Package: net-dns/bind-tools-9.10.3_p2 * Repository: gentoo * USE: userland_GNU elibc_glibc readline amd64 ssl abi_x86_64 seccomp urandom kernel_linux * FEATURES: preserve-libs sandbox userpriv usersandbox >>> Extracting net-dns/bind-tools-9.10.3_p2 * checking 29 files for package collisions >>> Merging net-dns/bind-tools-9.10.3_p2 to / --- /usr/ --- /usr/share/ --- /usr/share/man/ --- /usr/share/man/man8/ >>> /usr/share/man/man8/dnssec-revoke.8.bz2 >>> /usr/share/man/man8/dnssec-verify.8.bz2 >>> /usr/share/man/man8/dnssec-settime.8.bz2 >>> /usr/share/man/man8/dnssec-keygen.8.bz2 >>> /usr/share/man/man8/dnssec-keyfromlabel.8.bz2 >>> /usr/share/man/man8/dnssec-importkey.8.bz2 >>> /usr/share/man/man8/dnssec-dsfromkey.8.bz2 >>> /usr/share/man/man8/dnssec-signzone.8.bz2 --- /usr/share/man/man1/ >>> /usr/share/man/man1/host.1.bz2 >>> /usr/share/man/man1/nsupdate.1.bz2 >>> /usr/share/man/man1/dig.1.bz2 >>> /usr/share/man/man1/delv.1.bz2 >>> /usr/share/man/man1/nslookup.1.bz2 --- /usr/share/doc/ >>> /usr/share/doc/bind-tools-9.10.3_p2/ >>> /usr/share/doc/bind-tools-9.10.3_p2/CHANGES.bz2 >>> /usr/share/doc/bind-tools-9.10.3_p2/README.bz2 >>> /usr/share/doc/bind-tools-9.10.3_p2/FAQ.bz2 --- /usr/bin/ >>> /usr/bin/dnssec-keyfromlabel >>> /usr/bin/nsupdate >>> /usr/bin/dnssec-signzone >>> /usr/bin/nslookup >>> /usr/bin/dnssec-dsfromkey >>> /usr/bin/dnssec-settime >>> /usr/bin/dnssec-verify >>> /usr/bin/dnssec-revoke >>> /usr/bin/dnssec-keygen >>> /usr/bin/dnssec-importkey >>> /usr/bin/host >>> /usr/bin/delv >>> /usr/bin/dig >>> Safely unmerging already-installed instance... >>> Original instance of package unmerged safely. Traceback (most recent call last): File "/usr/lib64/python3.4/site-packages/portage/dbapi/_MergeProcess.py", line 235, in _spawn prev_mtimes=self.prev_mtimes, counter=counter) File "/usr/lib64/python3.4/site-packages/portage/dbapi/vartree.py", line 5013, in merge counter=counter) File "/usr/lib64/python3.4/site-packages/portage/dbapi/vartree.py", line 4297, in treewalk relative_paths=False) File "/usr/lib64/python3.4/site-packages/portage/dbapi/vartree.py", line 1063, in removeFromContents self.writeContentsToContentsFile(pkg, new_contents) File "/usr/lib64/python3.4/site-packages/portage/dbapi/vartree.py", line 1075, in writeContentsToContentsFile f = atomic_ofstream(os.path.join(pkg.dbdir, "CONTENTS")) File "/usr/lib64/python3.4/site-packages/portage/util/__init__.py", line 1288, in __init__ mode=mode, **portage._native_kwargs(kargs))) FileNotFoundError: [Errno 2] No such file or directory: b'/var/db/pkg/net-dns/bind-9.9.5-r3/CONTENTS.12168'
The list of blocked packages is evaluated inside the MergeProcess._spawn method. With parallel-install, the list of blocked packages can be stale by the time dblink.treewalk operates on the list with removeFromContents. This can be fixed by making dblink.treewalk detect stale items in the list of blocked packages, and skip removeFromContents calls for these items.
(In reply to Zac Medico from comment #1) > This can be fixed by making dblink.treewalk detect stale items in the list > of blocked packages, and skip removeFromContents calls for these items. The blocked packages are also used in the _collision_protect call before the files are merged, so we should acquire a lock for _collision_protect, but only when there are blocked packages to account for.
There's a patch in the following branch: https://github.com/zmedico/portage/tree/bug_576888
(In reply to Zac Medico from comment #3) > There's a patch in the following branch: > > https://github.com/zmedico/portage/tree/bug_576888 Withdrawing this patch for now, since I'm afraid that concurrent uninstall of one of the blocked packages could unmerge files that were just installed by the current package.
We can make the Scheduler avoid these kinds of problems by placing conflicting packages in a queue to be processed serially.
(In reply to Zac Medico from comment #5) > We can make the Scheduler avoid these kinds of problems by placing > conflicting packages in a queue to be processed serially. The scheduler won't be able to account for concurrent emerge/ebuild invocations, whereas locks can handle that very well. If we introduce locking at the package-slot level, then a package with blockers can trigger locking of all the blocked package-slots. This should minimize the amount of time that packages spend waiting on one another, and will allow for cooperation with concurrent emerge/ebuild invocations.
(In reply to Zac Medico from comment #3) > There's a patch in the following branch: > > https://github.com/zmedico/portage/tree/bug_576888 There's a new patch in that branch, posted for review here: https://archives.gentoo.org/gentoo-portage-dev/message/8687ecc13cc170a7b75c1da34a15d91d
This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=dccd4687999aed1788dfe5c3461523f2b23ca79f
Can i include the patch in sys-apps/portage-2.2.28 ? wget -O /etc/portage/patches/sys-apps/portage/bug_576888.patch https://gitweb.gentoo.org/proj/portage.git/patch/?id=dccd4687999aed1788dfe5c3461523f2b23ca79f and use it ?
(In reply to marco from comment #9) > Can i include the patch in sys-apps/portage-2.2.28 ? > > wget -O /etc/portage/patches/sys-apps/portage/bug_576888.patch > https://gitweb.gentoo.org/proj/portage.git/patch/ > ?id=dccd4687999aed1788dfe5c3461523f2b23ca79f > > and use it ? Yes, that's fine.
