The following patch series generates .CONTENTS.gz files, and includes them in the .DIGESTS file.
Created attachment 141973 [details, diff] 0001-Add-.CONTENTS-generation-functions-using-the-same-m.patch
Created attachment 141974 [details, diff] 0002-Add-generic-hook-for-generating-.CONTENTS-files.patch
Created attachment 141975 [details, diff] 0003-Whenever-a-.DIGESTS-file-is-generated-consider-a-.C.patch
Created attachment 141976 [details, diff] 0004-When-generating-.DIGESTS-files-also-generate-a-hash.patch
Created attachment 141978 [details, diff] 0005-Wire-up-.CONTENTS-stuff-to-the-config-file.patch
Created attachment 141980 [details, diff] 0006-Add-ChangeLog-entry-and-AUTHORS-listing.patch
Created attachment 141983 [details, diff] 0007-Fix-whitespace.patch Fixes a minor whitespace glitch in a previous patch.
Created attachment 141985 [details, diff] 0009-Generate-.CONTENTS.gz-instead-of-plain-.CONTENTS-be.patch This last one is optional, but recommended. It generates .CONTENTS.gz instead of a raw .CONTENTS. Since it's pure text, it tends to be 90% smaller.
Still TODO: Come up with a sane way to include the squashfs internal contents if we want to include it at all. Testing: I tested the snapshot target and it worked fine. I haven't tested the other targets however.
OK. I've gone and added these to catalyst *except* for 0009, since I'd rather wait until we have the compression options before utilizing this one. That, and I'd prefer uncompressed files on the mirrors (for now), so people can read them directly.
Robin: I hit a problem. Can you look into this? Total translation table size: 2048 Total rockridge attributes bytes: 2034 Total directory bytes: 4096 Path table size(bytes): 26 Max brk space used 0 803862 extents written (1570 MB) Traceback (most recent call last): File "/usr/lib64/catalyst/modules/catalyst_support.py", line 79, in generate_contents _ = contents_map[contents_function] KeyError: 'auto' None !!! catalyst: Error generating contents, is appropriate utility (auto) installed on your system? !!! catalyst: Error encountered during run of target livecd-stage2 Catalyst aborting.... lockfile does not exist '/var/tmp/catalyst/tmp/default/livecd-stage2-amd64-2008.0/.catalyst_lock'
wolf31o2: haven't had time to run the catalyst stuff, but here's a debug patch that should generate some useful output: diff --git a/modules/catalyst_support.py b/modules/catalyst_support.py index a9054f9..4afde74 100644 --- a/modules/catalyst_support.py +++ b/modules/catalyst_support.py @@ -75,6 +75,9 @@ def generate_contents(file,contents_function="auto",verbose=False): elif file.endswith('.tar'): _ = 'tar-tv' + if _ is 'auto': + die('File %r has unknown type for automatic detection.' % (file, )) + contents_function = _ _ = contents_map[contents_function] return _[0](file,_[1],verbose)
I took care of this. Apparently, the "_ is 'auto'" combined with the 'and' produced some "odd" results. Both clauses in that if evaluated to True individually, but the whole thing was False. I changed 'is' to == and it works fine.
Created attachment 145786 [details, diff] 0010-If-we-do-not-know-what-type-the-file-is-for-content.patch If we are processing an unknown file, just give a warning and avoid CONTENTS generation safely, without stopping the entire process (untested for functionality).
Created attachment 145787 [details, diff] 0011-Do-not-generate-.CONTENTS-for-GRP-srcset.patch For GRP srcsets, do not generate .CONTENTS files. This patch was tested by rangerpb after he hit it trying to generate .CONTENTS for files it did not know how to handle.
agaffney/wolf31o2: Please apply 0010/0011 ASAP to the tree, to help avoid catalyst exiting when it is almost completed it's full run.
Created attachment 145788 [details, diff] 0010-If-we-do-not-know-what-type-the-file-is-for-content-respin.patch Respin for the 'is' vs. '==' bug that agaffney taught me about.
Latest 2 patches are in SVN
This is in catalyst 2.0.6, which is in the tree and stable. If there's still a problem with this, please REOPEN this bug. Thanks!
