bzip compresses files to a greater extent than does gzip. By default, rsync uses
gzip, while ISO images aren't compressed at all. This is a distinct disadvantage
to all dialup users, and to metered broadband users. Gentoo would be more
attractive to these large subsets of the Linux user community if the download
sizes were smaller.
Steps to Reproduce:
I'll volunteer to make bzip versions of the packages as they're released.
As was explained on IRC, our ISO images have the bulk of the filesystem as a compressed file (cloop). This file can't be compressed further with any real benefit. The only space gains result from three uncompressed initrd files. This results in a 10-15mb space gain, which is trivial and not worth the time required to bzip2 each ISO image (in my opinion). That space gain will not increase as the ISO size increases, it's pretty static.
If it can be shown that there is a significant desire from users for this, it may be worthwhile.
The gentoo iso images (at least newer x86 and ppc) are using some compression (ucl or zlib depending on situation) internally, the gain you have today is just because some part (mostly the initrd for bootsplash) aren't compressed at all, the result is that you may now have some gain in having them compressed with bz2 now.
But you have at least 2 problems:
1 bz2 is quite cpu hunger, making that quite BAD for things like rsync
2 every compression algorithm increase the size of a block of data if it can't compress it, and that is more likely if you have already compressed data, the result is that you may even get bz2 bigger than iso.
We were thinking about other algorithm like some ppm ones but we didn't find anything mature/free/available easly yet.
Your point, having iso and rsync stream compressed better, is good, your proposed solution, sadly, isn't.
If you can find a compression algorithm, already implemented and usable, faster than bz2 and better than zlib please tell us and we'll be happy to implement it.
see comment #2 and comment #3