When attempting to build a stage1, catalyst fails. Previous versions of catalyst worked but they are no longer in Portage. /etc/portage/make.profile link and package files are fine. They haven't changed since the last working version of catalyst. Reproducible: Always Steps to Reproduce: 1. eix-sync 2. catalyst -s latest 3. catalyst -f stage1.spec Actual Results: 12 Dec 2020 19:42:48 -05: NOTICE : Loading configuration file: /etc/catalyst/catalyst.conf 12 Dec 2020 19:42:48 -05: NOTICE : conf_values[options] = {'sticky-config'} 12 Dec 2020 19:42:48 -05: NOTICE : Processing spec file: stage1.spec 12 Dec 2020 19:42:48 -05: NOTICE : Using target: stage1 12 Dec 2020 19:42:48 -05: NOTICE : Source file specification matching setting is: strict 12 Dec 2020 19:42:48 -05: NOTICE : Accepted source file extensions search order: ['tar.xz', 'tpxz', 'xz', 'tar.bz2', 'bz2', 'tbz2', 'squashfs', 'sfs', 'tar.gz', 'gz', 'tar'] 12 Dec 2020 19:42:48 -05: NOTICE : Source path set to /home/audiodef/Documents/gentoostudio/catalyst/builds/stage3-amd64-seed.tar.xz 12 Dec 2020 19:42:48 -05: NOTICE : stage1 stage path is /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/tmp/stage1root 12 Dec 2020 19:42:48 -05: NOTICE : --- Running action sequence: unpack 12 Dec 2020 19:42:48 -05: NOTICE : Clearing the chroot path ... 12 Dec 2020 19:42:48 -05: NOTICE : Emptying directory: /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest 12 Dec 2020 19:42:48 -05: NOTICE : Starting auto from /home/audiodef/Documents/gentoostudio/catalyst/builds/stage3-amd64-seed.tar.xz 12 Dec 2020 19:42:48 -05: NOTICE : to /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest (this may take some time) .. 12 Dec 2020 19:43:02 -05: NOTICE : --- Running action sequence: unpack_snapshot 12 Dec 2020 19:43:02 -05: NOTICE : Unpacking portage tree (this can take a long time) ... 12 Dec 2020 19:43:05 -05: NOTICE : --- Running action sequence: setup_confdir 12 Dec 2020 19:43:05 -05: NOTICE : --- Running action sequence: portage_overlay 12 Dec 2020 19:43:05 -05: NOTICE : --- Running action sequence: base_dirs 12 Dec 2020 19:43:05 -05: NOTICE : --- Running action sequence: bind 12 Dec 2020 19:43:06 -05: NOTICE : --- Running action sequence: chroot_setup 12 Dec 2020 19:43:06 -05: NOTICE : Setting up chroot... NOTICE:catalyst:Setting up chroot... 12 Dec 2020 19:43:06 -05: WARNING : Overriding certain env variables may cause catastrophic failure. WARNING:catalyst:Overriding certain env variables may cause catastrophic failure. 12 Dec 2020 19:43:06 -05: WARNING : If your build fails look here first as the possible problem. WARNING:catalyst:If your build fails look here first as the possible problem. 12 Dec 2020 19:43:06 -05: WARNING : Catalyst assumes you know what you are doing when setting these variables. WARNING:catalyst:Catalyst assumes you know what you are doing when setting these variables. 12 Dec 2020 19:43:06 -05: WARNING : Catalyst Maintainers use VERY minimal envscripts, if used at all. WARNING:catalyst:Catalyst Maintainers use VERY minimal envscripts, if used at all. 12 Dec 2020 19:43:06 -05: WARNING : You have been warned. WARNING:catalyst:You have been warned. 12 Dec 2020 19:43:06 -05: NOTICE : Writing the stage make.conf to: /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/etc/portage/make.conf NOTICE:catalyst:Writing the stage make.conf to: /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/etc/portage/make.conf 12 Dec 2020 19:43:06 -05: NOTICE : STICKY-CONFIG is enabled NOTICE:catalyst:STICKY-CONFIG is enabled 12 Dec 2020 19:43:06 -05: NOTICE : --- Running action sequence: setup_environment NOTICE:catalyst:--- Running action sequence: setup_environment 12 Dec 2020 19:43:06 -05: NOTICE : --- Running action sequence: run_local NOTICE:catalyst:--- Running action sequence: run_local copying make.conf to /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/tmp/stage1root/etc/portage Copying stage1-chroot.sh to /tmp copying stage1-chroot.sh to /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/tmp copying chroot-functions.sh to /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/tmp Ensure the file has the executable bit set Running stage1-chroot.sh in chroot: chroot /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest /tmp/stage1-chroot.sh !!! Section 'GentooStudio' in repos.conf has location attribute set to nonexistent directory: '/var/lib/GentooStudio' !!! Section 'audio-overlay' in repos.conf has location attribute set to nonexistent directory: '/var/lib/audio-overlay' !!! Section 'GentooStudio' in repos.conf has location attribute set to nonexistent directory: '/var/lib/GentooStudio' !!! Section 'audio-overlay' in repos.conf has location attribute set to nonexistent directory: '/var/lib/audio-overlay' >>> Regenerating /etc/ld.so.cache... Traceback (most recent call last): File "/tmp/build.py", line 5, in <module> import portage ModuleNotFoundError: No module named 'portage' !!! Section 'GentooStudio' in repos.conf has location attribute set to nonexistent directory: '/var/lib/GentooStudio' !!! Section 'audio-overlay' in repos.conf has location attribute set to nonexistent directory: '/var/lib/audio-overlay' Your profile seems to be broken. Could not build a list of build packages. Double check your /etc/portage/make.profile link and the 'packages' files. 12 Dec 2020 19:43:09 -05: ERROR : CatalystError: cmd(['/usr/share/catalyst/targets/stage1/stage1-controller.sh', 'run']) exited 1 ERROR:catalyst:CatalystError: cmd(['/usr/share/catalyst/targets/stage1/stage1-controller.sh', 'run']) exited 1 12 Dec 2020 19:43:09 -05: NOTICE : /home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/var/tmp/portage is not a mount point. Skipping NOTICE:catalyst:/home/audiodef/Documents/gentoostudio/catalyst/tmp/default/stage1-amd64-latest/var/tmp/portage is not a mount point. Skipping 12 Dec 2020 19:43:09 -05: ERROR : CatalystError: Stage build aborting due to error. ERROR:catalyst:CatalystError: Stage build aborting due to error. 12 Dec 2020 19:43:09 -05: ERROR : Exception running action sequence run_local Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1409, in run_local env=self.env) File "/usr/lib/python3.7/site-packages/catalyst/support.py", line 54, in cmd print_traceback=False) catalyst.support.CatalystError: cmd(['/usr/share/catalyst/targets/stage1/stage1-controller.sh', 'run']) exited 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1514, in run getattr(self, x)() File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1418, in run_local print_traceback=False) catalyst.support.CatalystError: Stage build aborting due to error. ERROR:catalyst:Exception running action sequence run_local Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1409, in run_local env=self.env) File "/usr/lib/python3.7/site-packages/catalyst/support.py", line 54, in cmd print_traceback=False) catalyst.support.CatalystError: cmd(['/usr/share/catalyst/targets/stage1/stage1-controller.sh', 'run']) exited 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1514, in run getattr(self, x)() File "/usr/lib/python3.7/site-packages/catalyst/base/stagebase.py", line 1418, in run_local print_traceback=False) catalyst.support.CatalystError: Stage build aborting due to error. Expected Results: Completed stage1 build. My stage1.spec: subarch: amd64 target: stage1 version_stamp: latest rel_type: default profile: default/linux/amd64/17.1 snapshot: latest source_subpath: stage3-amd64-seed update_seed: yes update_seed_command: --update --deep --newuse --with-bdeps=y --backtrack=250 @system @world portage_confdir: /etc/portage catalyst.conf: digests="sha512 whirlpool" contents="auto" distdir="/usr/portage/distfiles" envscript="/etc/catalyst/catalystrc" hash_function="crc32" options="sticky-config" portdir="/usr/portage" repo_basedir="/usr" repo_name="portage" target_distdir="/usr/portage/distfiles" target_pkgdir="/usr/portage/packages" sharedir="/usr/share/catalyst" shdir="%(sharedir)s/targets" snapshot_cache="/home/audiodef/Documents/gentoostudio/catalyst/snapshot_cache" storedir="/home/audiodef/Documents/gentoostudio/catalyst" source_matching="strict"
is this a duplicate of 519656?
No. I don't even use snapcache, and the issue I've reported has nothing to do with snapcache or running more than one instance.
I had a couple of previous versions in distfiles, so I tossed in ebuilds for them, and the result is the same. The issue might be with python, not with catalyst, or perhaps with how catalyst handles python 3.7 or higher.
Please verify that the version of Python being used with Catalyst is aligned with PYTHON_TARGETS and such of Portage.
I think the build.py in question is /root/catalyst/tmp/default/stage1-amd64-latest/tmp/build.py. Apparently there is no module named "portage" and maybe it's this that needs to be addressed. Not sure how that would happen, though. #!/usr/bin/python import os import sys import portage # this loads files from the profiles ... # wrap it here to take care of the different # ways portage handles stacked profiles # last case is for portage-2.1_pre* def scan_profile(path): if "grab_stacked" in dir(portage): return portage.grab_stacked(path, portage.settings.profiles, portage.grabfile, incremental_lines=1) else: if "grab_multiple" in dir(portage): return portage.stack_lists(portage.grab_multiple(path, portage.settings.profiles, portage.grabfile), incremental=1) else: return portage.stack_lists([portage.grabfile_package(os.path.join(x, path)) for x in portage.settings.profiles], incremental=1) # loaded the stacked packages / packages.build files pkgs = scan_profile("packages") buildpkgs = scan_profile("packages.build") # go through the packages list and strip off all the # crap to get just the <category>/<package> ... then # search the buildpkg list for it ... if it's found, # we replace the buildpkg item with the one in the # system profile (it may have <,>,=,etc... operators # and version numbers) for idx in range(0, len(pkgs)): try: bidx = buildpkgs.index(portage.dep_getkey(pkgs[idx])) buildpkgs[bidx] = pkgs[idx] if buildpkgs[bidx][0:1] == "*": buildpkgs[bidx] = buildpkgs[bidx][1:] except Exception: pass for b in buildpkgs: sys.stdout.write(b + " ")
(In reply to Sam James from comment #4) > Please verify that the version of Python being used with Catalyst is aligned > with PYTHON_TARGETS and such of Portage. What do I need to do to verify this?
This bug has been hitting our releng builds the last couple of weeks. The problem is that the stage1/build.py has a generic /usr/bin/python in its shebang, and is not wrapped by python-exec. It gets executed under whatever python interpreter version is first in "eselect python list" (/etc/python-exec/python-exec.conf), even if that version is not enabled in sys-apps/portage's PYTHON_TARGETS. Most likely what has hapened is that python3.9 got newly installed, and it put itself at the top of the priority list. When build.py gets executed under 3.9, it cannot import the portage module because sys-apps/portage has been built with PYTHON_TARGETS="python2_7 python3_8". I'm still thinking about what the best fix might be. A rough workaround would be to package.mask dev-lang/python:3.9 for now via portage_confdir.
Thanks for confirming. On my system here, this happens regardless of which python is at the top of my eselect python list.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=37cabf5c536fe8eef16532f18790c1da566aae15 commit 37cabf5c536fe8eef16532f18790c1da566aae15 Author: Matt Turner <mattst88@gentoo.org> AuthorDate: 2020-12-15 03:29:56 +0000 Commit: Matt Turner <mattst88@gentoo.org> CommitDate: 2020-12-16 03:01:23 +0000 targets: Use interpreter with portage module for build.py Closes: https://bugs.gentoo.org/759685 Signed-off-by: Matt Turner <mattst88@gentoo.org> targets/stage1/chroot.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
I cherry-picked these patches to the stable branch. They're included in catalyst-3.0.15.
Thank you. I can confirm that catalyst now proceeds to build stage1.
