I've seen this race triggered when dev-python/setuptools-git or dev-python/pbr is merged while a package is being built with setuptools: > Traceback (most recent call last): > File "setup.py", line 114, in <module> > include_package_data=False, > File "/usr/lib64/python3.6/site-packages/setuptools/__init__.py", line 129, in setup > return distutils.core.setup(**attrs) > File "/usr/lib64/python3.6/distutils/core.py", line 148, in setup > dist.run_commands() > File "/usr/lib64/python3.6/distutils/dist.py", line 955, in run_commands > self.run_command(cmd) > File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.6/site-packages/setuptools/command/install.py", line 61, in run > return orig.install.run(self) > File "/usr/lib64/python3.6/distutils/command/install.py", line 557, in run > self.run_command(cmd_name) > File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command > self.distribution.run_command(command) > File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.6/site-packages/setuptools/command/install_egg_info.py", line 34, in run > self.run_command('egg_info') > File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command > self.distribution.run_command(command) > File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.6/site-packages/setuptools/command/egg_info.py", line 272, in run > writer = ep.resolve() > File "/usr/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2428, in resolve > module = __import__(self.module_name, fromlist=['__name__'], level=0) > ModuleNotFoundError: No module named 'pbr' > > Traceback (most recent call last): > File "setup.py", line 142, in <module> > keywords=['interface', 'components', 'plugins'], > File "/usr/lib64/python3.4/site-packages/setuptools/__init__.py", line 129, in setup > return distutils.core.setup(**attrs) > File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup > dist.run_commands() > File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands > self.run_command(cmd) > File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.4/site-packages/setuptools/command/install.py", line 61, in run > return orig.install.run(self) > File "/usr/lib64/python3.4/distutils/command/install.py", line 539, in run > self.run_command('build') > File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command > self.distribution.run_command(command) > File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.4/distutils/command/build.py", line 126, in run > self.run_command(cmd_name) > File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command > self.distribution.run_command(command) > File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.4/site-packages/setuptools/command/build_py.py", line 53, in run > self.build_package_data() > File "/usr/lib64/python3.4/site-packages/setuptools/command/build_py.py", line 118, in build_package_data > for package, src_dir, build_dir, filenames in self.data_files: > File "/usr/lib64/python3.4/site-packages/setuptools/command/build_py.py", line 66, in __getattr__ > self.data_files = self._get_data_files() > File "/usr/lib64/python3.4/site-packages/setuptools/command/build_py.py", line 82, in _get_data_files > self.analyze_manifest() > File "/usr/lib64/python3.4/site-packages/setuptools/command/build_py.py", line 138, in analyze_manifest > self.run_command('egg_info') > File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command > self.distribution.run_command(command) > File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "/usr/lib64/python3.4/site-packages/setuptools/command/egg_info.py", line 280, in run > self.find_sources() > File "/usr/lib64/python3.4/site-packages/setuptools/command/egg_info.py", line 295, in find_sources > mm.run() > File "/usr/lib64/python3.4/site-packages/setuptools/command/egg_info.py", line 526, in run > self.add_defaults() > File "/usr/lib64/python3.4/site-packages/setuptools/command/egg_info.py", line 565, in add_defaults > rcfiles = list(walk_revctrl()) > File "/usr/lib64/python3.4/site-packages/setuptools/command/sdist.py", line 20, in walk_revctrl > for item in ep.load()(dirname): > File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 2422, in load > return self.resolve() > File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 2428, in resolve > module = __import__(self.module_name, fromlist=['__name__'], level=0) > ImportError: No module named 'setuptools_git'
A potential way to model this relationship would be to introduce a new kind of blocker that indicates that merging a package will temporarily interfere with the operation of another package. In the case described in comment #0, merging dev-python/setuptools-git or dev-python/pbr temporarily interferes with the operation of dev-python/setuptools.
A "more atomic" package merge approach would minimize interference with setuptools, for example, these libraries use directory renames when possible so that the new package files are exposed as atomically as possible: https://github.com/mgorny/atomic-install https://github.com/mgorny/atomic-install-py
Also, it might be possible to make setuptools do a retry loop, using timestamp comparisons to detect file/directory changes that have occurred during the last few seconds.
If the package manager has access to snapshot capabilities for the root filesystem (bug 40127), it can run each build in a chroot environment with an immutable snapshot of the root filesystem, effectively implementing read-copy-update (RCU) for the root filesystem. I imagine that many libraries with plugin systems might be vulnerable to the sort of race condition that setuptools / pkg_resources has. It seems like it's practically impossible to handle this sort of thing at the individual library level, since there could be a vast number of consumers of the library at any given time, so there might never be a safe time to install a plugin on a running system.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b3e9f8fd458a77e067ccb50f8a7f0cd29587c18c commit b3e9f8fd458a77e067ccb50f8a7f0cd29587c18c Author: Sam James <sam@gentoo.org> AuthorDate: 2023-09-01 00:55:20 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-01 00:55:20 +0000 dev-python/setuptools: restore comment + add bug ref for setuptools_scm PDEPEND See e96c1d113f47ac076ee71b951afebe3515850918. Eli noticed that the comment had got lost here & it wasn't particularly descriptive, so restore the comment and add a bug reference. Bug: https://bugs.gentoo.org/663324 Signed-off-by: Sam James <sam@gentoo.org> dev-python/setuptools/setuptools-68.0.0-r1.ebuild | 2 ++ dev-python/setuptools/setuptools-68.1.2.ebuild | 2 ++ 2 files changed, 4 insertions(+)
(In reply to Zac Medico from comment #1) > A potential way to model this relationship would be to introduce a new kind > of blocker that indicates that merging a package will temporarily interfere > with the operation of another package. In the case described in comment #0, > merging dev-python/setuptools-git or dev-python/pbr temporarily interferes > with the operation of dev-python/setuptools. See also: * bug 906809 (bit different, but related) * Exherbo's https://www.exherbolinux.org/docs/eapi/exheres-for-smarties.html#package_dependencies and https://www.exherbolinux.org/docs/eapi/exheres-for-smarties.html#annotations which has """ resolution, for blockers only: manual uninstall-blocked-after suggests uninstalling the blocked package after we are merged (you’re allowed to file collide if you do this). uninstall-blocked-before suggests uninstalling the blocked package before we are merged (only use if you really really have to, and most definitely not just for collisions). upgrade-blocked-before suggests upgrading the blocked package before we are merged """
This is a bit of a funny issue and an unpleasant solution to have to feel compelled into using. I think it deserves a more in depth rethink. In general, (deleting one version of a package and/or) installing a package is an unsafe time of transition. Any program started in the interim period may manifest random issues, but the best that all us package managers can hope for is that the user won't start new programs while running installations in another shell, or cross our fingers and hope the timing works out. The problem here is just another example of that, but one that portage specifically is prone to triggering by use of "good" timing, since it starts a bunch of programs around the same time. I think the intuitive solution here is that portage should be able to merge images in parallel, and it should be able to perform compiles in parallel, but it shouldn't try to merge an image in parallel with doing a compile -- this is inherently unsafe. Portage could/should add locking to pause one package after creating the image but before merging it, if another compile is running. This sacrifices a bit of parallelism but IMHO people will be thankful for that.
(In reply to Eli Schwartz from comment #7) > Portage could/should add locking to pause one package after creating the > image but before merging it, if another compile is running. This sacrifices > a bit of parallelism but IMHO people will be thankful for that. Yeah, we have a _merge_wait_queue that implements similar behavior to handle system packages since bug 256616: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e589ece0600c449238d70f543eaba2f32a93ef5f commit e589ece0600c449238d70f543eaba2f32a93ef5f Author: Zac Medico <zmedico@gentoo.org> Date: 2009-02-02 06:11:06 +0000 Bug #256616 - Since dependencies on system packages are frequently unspecified, merge them only when no builds are executing. When a system package finishes building, it's added to a wait queue that is only processed when the number of running builds drops to zero. All pending merges are then processed before any new builds are allowed to start. svn path=/main/trunk/; revision=12569
That bug has a commenter making a bit of a stink about always using binpkgs, which I think is unnecessary. But I do think that using the system set for this may have been optimistic. :)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=825db01b91a37dcd9890ee5bf9f462ea524ac5cc commit 825db01b91a37dcd9890ee5bf9f462ea524ac5cc Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-01-16 05:06:36 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-01-16 05:20:37 +0000 Add merge-wait FEATURES setting enabled by default Add a new merge-wait FEATURES setting, enabled by default, which prevents packages from being merged while other packages are building. Previously, this behavior was already implemented for packages that satisfy direct or indirect dependencies of the system set. This feature can be disabled only for other packages which do not satisfy direct or indirect dependencies of the system set, in order to trade the possibility of random build failures for greater parallelism. Currently, it is known that having merge-wait disabled can cause "random" build failures for builds using setuptools when setuptools plugins are merged during the build. Bug: https://bugs.gentoo.org/663324 Signed-off-by: Zac Medico <zmedico@gentoo.org> cnf/make.globals | 4 ++-- lib/_emerge/Scheduler.py | 14 ++++++++------ lib/portage/const.py | 3 ++- man/make.conf.5 | 16 +++++++++++++--- 4 files changed, 25 insertions(+), 12 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77c44c46194922509bc4f2b5cfc099412a560a69 commit 77c44c46194922509bc4f2b5cfc099412a560a69 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-02-22 07:23:40 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-02-22 07:23:50 +0000 sys-apps/portage: add 3.0.62 Closes: https://bugs.gentoo.org/663324 Closes: https://bugs.gentoo.org/728046 Closes: https://bugs.gentoo.org/891137 Closes: https://bugs.gentoo.org/906368 Closes: https://bugs.gentoo.org/916566 Closes: https://bugs.gentoo.org/921170 Closes: https://bugs.gentoo.org/921208 Closes: https://bugs.gentoo.org/921400 Closes: https://bugs.gentoo.org/922038 Closes: https://bugs.gentoo.org/922142 Closes: https://bugs.gentoo.org/923368 Closes: https://bugs.gentoo.org/923750 Closes: https://bugs.gentoo.org/923841 Closes: https://bugs.gentoo.org/923852 Closes: https://bugs.gentoo.org/923854 Closes: https://bugs.gentoo.org/924192 Closes: https://bugs.gentoo.org/924273 Closes: https://bugs.gentoo.org/924585 Closes: https://bugs.gentoo.org/921380 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.62.ebuild | 246 +++++++++++++++++++++++++++++++++ 2 files changed, 247 insertions(+)
