I've been fooling around with Jigdo--a program that scans a bunch of files, finds their exact data in a bigger file, and makes a template of all the data that isn't in those files plus a description of where to find the files, and then allows you to use this to rebuild the original file--and trying to make it work for the Gentoo basic install CDs. I've gotten a template to build a CD here. Debian uses Jigdo to reduce space usage on their mirrors. They put the files on the CDs on the mirrors, and use Jigdo to build the CD images on the user end. Many debian mirrors don't have ISOs for the CDs or DVDs; but the Jigdo files can be used to build both the CD and DVD images from the same data on the server. On Gentoo, you would in theory be able to supply a Jigdo to build a basic live CD or a Universal Live CD for any architecture using compatible files and having the jigdo point at the stages. For example, all 2004.2 x86 Live CDs could use basic i386 programs on the CD, and use either no stages (basic), x86 stages, Athlon stages, P3 stages, or P4 stages. The stages would all be on the mirrors, and the other files on the CD would be on the mirrors exactly once. The individual jigdo and template files would be used to construct various CDs whose contents is identical, except for the stages. This would also be good for the GRP package CDs. Jigdos for DVD and CD forms of these could easily be supplied. The actual packages would be in the grp/arch/ directory on the mirrors; and the CDs would be built by downloading these files and assembling them into the original CD. Furthermore, a user could just grab the package without downloading the CD; this would be extremely useful if Portage became able to handle binary packages. Here are instructions for creating a test jigdo image from an ISO file. These instructions assume that you will be working in /data/genjig, and that the image is install-x86-minimal-2004.1.iso. This is all shown done as root; there are files on the CD that are only readable by root. 1. Mount the CD image with the nocompress option: genjig # mount -o loop,nocompress /data/genjig/install-x86-minimal-2004.1.mounted.iso /mnt/cdrom/ 'nocompress' prevents zisofs compresesd files from being automatically decompressed. If you don't use nocompress, and your kernel has zisofs capabilities, then you will wind up with a huge template file; the cd image contains the files compressed, and jigdo needs to scan them in their compressed form to locate them. This is a copy of install-x86-minimal-2004.1.iso; later we'll be destroying the original ISO, but we want this one to still be mounted. 2. Generate the template: genjig # find /mnt/cdrom -type f | jigdo-file mt --image=install-x86-minimal-2004.1.iso -T - The `find` command is used to find files (actual files) to search for in the image. this list of files is passed to jigdo-file. The "mt" option is "make template," and causes jigdo-file to output a .jigdo and a .template, used to reassemble the image. The .jigdo needs to be redistributed; the .template will be downloaded by jigdo-lite in a real-world use. 3. Put the path to the files in the .jigdo. The section will look like this: [Servers] A=file:/ Because `find` returned paths such as "/mnt/cdrom/foo", jigdo will look for file://mnt/cdrom/foo when searching for this file. Ideally, you'll want to do the job so that you get a relative path from find (like ./stages/ and ./cddata/2004.1/) 4. Reassemble the image genjig # rm install-x86-minimal-2004.1.iso genjig # jigdo-lite install-x86-minimal-2004.1.jigdo The first thing that it will ask you for is "Files to scan:". Normally, you could point it at a local copy of the files (i.e. /mnt/cdrom); however, in this case, you want to test jigdo out; leave this blank, it'll just copy the files off the CD. 5. Test the new image If all goes well, unmount /mnt/cdrom and mount the new image there, then take a look. Or heck, check the md5 sums against the new image and .mounted.iso
Also, on amd64 at least, I could only make jigdo 0.7.1 work, and only with USE="-gtk". You'll need to update the ebuilds to get 0.7.1 and build without GTK+ to do this most likely. See the dependencies here.
>Ideally, you'll want to do the job so that you get a relative > path from find (like ./stages/ and ./cddata/2004.1/) You could easy do it by command find /mnt/cdrom// -type f | \ jigdo-file mt --image=install-x86-minimal-2004.1.iso -T - because jigdo-file interprets '//' as root. If I understand you right...
releng: mind checking into this bug, and probably just closing it outright. users wanting this sort of functionality can just use catalyst.
Yeah, I'm marking this as WONTFIX for now. That doesn't mean that we might not revisit it, but unless the software works on all architectures supported by Gentoo, I don't see us ever using it. That being said, it doesn't mean that we won't revisit this as some point in the future, just right now it isn't even on Release Engineering's radar.
