It looks like after a recent stage1 run by catalyst on a mips3/n32 target, it created a rogue SHA256 hash digest file and put it in a separate *.sha256 file instead of in the DIGESTS file. I am unsure if this is intended behavior or not. This trips up an error clause in catalyst because it uses a glob to verify the filepath to the source_subpath file in the next stage's spec file, and that glob is now matching >1 files, so it bails out. E.g., it creates <filename>.tar.xz.<digest type>. So for a SHA256 digest of a mips3_n32 stage2 tarball, you get something like "builds/n32/stage1-mips3_n32-20221112.tar.xz.sha256". There's a bit of code in catalyst's support.py that doesn't handle this: > def file_check(filepath, extensions=None, strict=True): > '''Check for the files existence and that only one exists > if others are found with various extensions > ''' > if os.path.isfile(filepath): > return filepath > # it didn't exist > # so check if there are files of that name with an extension > files = glob.glob("%s.*" % filepath) > # remove any false positive files > files = [x for x in files if not x.endswith(".CONTENTS") and not > x.endswith(".CONTENTS.gz") and not x.endswith(".DIGESTS")] > > if len(files) is 1: > return files[0] > elif len(files) > 1 and strict: > msg = "Ambiguos Filename: %s\nPlease specify the correct extension as well" % filepath > raise CatalystError(msg, print_traceback=False) > else: > target_file = None > for ext in extensions: > target = filepath + "." + ext > if target in files: > target_file = target > break > if target_file: > return target_file > raise CatalystError("File Not Found: %s" % filepath) The list expression that builds out "files" is picking up that extraneous *.sha256 file that has a single SHA256 hash in it, because it's not excluding it. This falls into the "elif len(files) > 1 and strict" clause, because strict=True, which causes catalyst to abort with this error: > # time catalyst -v -f specs/n32/stage2-mips3_n32-20221112.spec -a > 17 Nov 2022 17:58:16 EST: INFO : Catalyst 3.0.22 > 17 Nov 2022 17:58:16 EST: INFO : extended version info unavailable > 17 Nov 2022 17:58:16 EST: INFO : Copyright 2003-2022 Gentoo Foundation > 17 Nov 2022 17:58:16 EST: INFO : Copyright 2008-2012 various authors > 17 Nov 2022 17:58:16 EST: INFO : Distributed under the GNU General Public License version 2.1 > 17 Nov 2022 17:58:16 EST: NOTICE : Loading configuration file: /etc/catalyst/catalyst.conf > 17 Nov 2022 17:58:16 EST: INFO : Autoresuming support enabled. > 17 Nov 2022 17:58:16 EST: INFO : Package cache support enabled. > 17 Nov 2022 17:58:16 EST: INFO : Snapshot cache support enabled. > 17 Nov 2022 17:58:16 EST: INFO : Kernel cache support enabled. > 17 Nov 2022 17:58:16 EST: INFO : Seed cache support enabled. > 17 Nov 2022 17:58:16 EST: INFO : Envscript support enabled. > 17 Nov 2022 17:58:16 EST: NOTICE : conf_values[options] = {'autoresume', 'pkgcache', 'snapcache', 'bindist', 'kerncache', 'clear-autoresume', 'seedcache'} > 17 Nov 2022 17:58:16 EST: NOTICE : Processing spec file: specs/n32/stage2-mips3_n32-20221112.spec > 17 Nov 2022 17:58:16 EST: NOTICE : Using target: stage2 > 17 Nov 2022 17:58:16 EST: INFO : Cross-compiling on amd64 for different machine type mips > 17 Nov 2022 17:58:16 EST: NOTICE : Source file specification matching setting is: strict > 17 Nov 2022 17:58:16 EST: NOTICE : Accepted source file extensions search order: ['tar.xz', 'xz', 'tar.bz2', 'bz2', 'tbz2'] > 17 Nov 2022 17:58:16 EST: INFO : SNAPSHOT_PATH set to: /nas/catalyst/snapshots/portage-20221112.tar.bz2 > 17 Nov 2022 17:58:16 EST: INFO : # CRC32 HASH (portage-20221112.tar.bz2) = # CRC32 HASH > 17 Nov 2022 17:58:16 EST: INFO : fc0dd119 portage-20221112.tar.bz2 > 17 Nov 2022 17:58:16 EST: ERROR : CatalystError: Ambiguos Filename: /nas/catalyst/builds/n32/stage1-mips3_n32-20221112 > 17 Nov 2022 17:58:16 EST: ERROR : Please specify the correct extension as well And if I list the files matching "builds/n32/stage1-mips3_n32-20221112*": > # ls -l builds/n32/stage1-mips3_n32-20221112.tar.xz* > -rw-r--r-- 1 root root 86M Nov 17 07:03 builds/n32/stage1-mips3_n32-20221112.tar.xz > -rw-r--r-- 1 root root 204K Nov 17 07:03 builds/n32/stage1-mips3_n32-20221112.tar.xz.CONTENTS.gz > -rw-r--r-- 1 root root 798 Nov 17 07:03 builds/n32/stage1-mips3_n32-20221112.tar.xz.DIGESTS > -rw-r--r-- 1 root root 113 Nov 17 07:03 builds/n32/stage1-mips3_n32-20221112.tar.xz.sha256 So I am not sure if this is just old code that wasn't update for new behavior (generating digests into separate files), something changed with how digests are actually generated with Python (I am running 3.11 as primary while catalyst still invokes 3.10's interpreter), or if something else went sideways. I checked things out on my NAS machine and there aren't any notable issues there, so I don't think it has anything to do with the catalyst directory being mounted over NFSv4. The weird thing is, per my /etc/catalyst/catalyst.conf file, I should only be generating SHA512 and BLAKE2B hashes, not SHA256 hashes as well. So I am leaning towards something changing in my userland somewhere that causes rogue *.sha256 hashes to get created, but I have no idea where to even look to figure that out. Filing a bug just in case it's something in catalyst acting up. If not, feel free to resolve with no action.
https://gitweb.gentoo.org/proj/catalyst.git/commit/?h=catalyst-3.0-stable&id=804d9f20534d0bc504f873a893596f79c8837f46 Looks like we need to update > files = [x for x in files if not x.endswith(".CONTENTS") and not > x.endswith(".CONTENTS.gz") and not x.endswith(".DIGESTS")]
Fixed by commit 0601ad81033983b5aca31bbc3f75f2f995c40be4 (origin/catalyst-3.0-stable, catalyst-3.0-stable) Author: Matt Turner <mattst88@gentoo.org> Date: Wed Feb 16 14:33:17 2022 -0800 catalyst: Add .sha256 to list of ignored extensions Otherwise the presence of a .sha256 file would cause this error: ERROR:catalyst:CatalystError: Ambiguous Filename: /home/catalyst/builds/default/stage3-ia64-openrc-20220216T032203Z ERROR:catalyst:Please specify the correct extension as well Fixes: 7457cd3b ("catalyst: generate .sha256 file if any digest is enabled") Closes: https://bugs.gentoo.org/881705 Signed-off-by: Matt Turner <mattst88@gentoo.org> (cherry picked from commit 1a8d6cd3d31f960755c3237679539d9cab175bc6)
