In livecd stage2 busybox is configured, compiling and copied to cache. On following runs busybox is compiled again, the cache is not used. Cache for other components of the kernel build are pulled from the cache such as the kernel and module-init-tools. Reproducible: Always Steps to Reproduce: 1. Run livecd-stage2 build, notice busybox is compiled and copied to cache. 2. Run livecd-stage2 again, notice busybox is compiled again. Expected Results: busybox package should be used from the cache instead of recompiled.
This isn't catalyst. Genkernel is the one that builds busybox.
Looking at compile_busybox() in gen_compile.sh, it looks like there's some basic logic for using the cached busybox, but it's not complete. It will remove the cache if the busybox config is newer than what the cached version was built with. It also checks to see if the cache does not exist before compiling. However, there is no actual code to use the cache. Since this isn't a bug that actually prevents you from building a kernel, this probably isn't a priority. You're welcome to whip up a patch and submit it, though. This will make it a lot more likely that this problem gets fixed.
Okay, what I said was completely wrong. There doesn't need to be any more logic to use an existing busybox bincache. The code that compiles just creates the bincache, which is what's used to create the initramfs. So, the problem lies elsewhere.
It looks like this may be catalyst's fault (indirectly). The first thing catalyst does when it goes to build the kernel is (re-)emerge genkernel. When portage installs a package, it creates the files with a [cm]time of *now*. Because of this, the busy-config from genkernel is *always* newer than the busybox bincache of the previous run, which causes the bincache check in gen_compile.sh's compile_busybox() to fail and remove the bincache. I'm not sure there is a way to fix this.
Could a hash be used (MD5, etc) instead of time?
Perhaps, but where would it be stored? An option is to package the busy-config file with the bzip2'd busybox binary (currently, the busybox binary is bzip2'd directly without tar) and then comparing the md5 of the stored busy-config to the "new" one. It doesn't look like it would require modification of very much code. I can probably whip up a patch for this if I get bored at work this afternoon :)
Created attachment 120626 [details, diff] patch Okay, so I got bored at work quicker than I thought. This patch implements saving the busy-config along with the compiled busybox binary and then comparing the MD5 of the old and new when checking for bincache "staleness". It is completely untested. As far as I can tell, this patch should not break with busybox bincache files from "older" versions of genkernel, since the filename is changing as well. It will just ignore the old file.
* ERROR: Could not extract busybox bincache! If I have time I may try to figure this out.
Created attachment 120644 [details, diff] genkernel-busybox.patch Modifications: 1. md5sum only the .config file in the cache, not the entire cache. 2. Added -C "..." path to tar extract so that busybox gets put into the right directory. Please note that this still does not work because the ${BUSYBOX_CONFIG} file is copied and modified before building, but the original is used in the comparison. The cache check may need to be moved later or the original ${BUSYBOX_CONFIG} stored in the cache instead of the modified config. However, if genkernel changes what is modified using the original config would cause problems.
Created attachment 120649 [details] updated patch with old stale cache checking code actually removed I forgot to actually remove the old code for checking for the stale cache in my original patch. This one has that removed and your changes.
Created attachment 120651 [details, diff] updated patch with old stale cache checking code actually removed I forgot to actually remove the old code for checking for the stale cache in my original patch. This one has that removed and your changes.
The latest patch has the tar -C option referencing a file, it should be a directory: -C "${TEMP}/initramfs-busybox-temp/bin/busybox.tar.bz2" should be -C "${TEMP}/initramfs-busybox-temp/bin"
Created attachment 120657 [details, diff] another patch Whoops, let's try that again...this time paying attention
Created attachment 120661 [details, diff] patch using modified config Sorry that the format of this patch isn't the same as yours, I don't have the SVN checkout. This changes the config that is compared by using the config created just before compiling. Works for me!
At first glance, the patch looks fine. Also, you can access anon SVN via <http://anonsvn.gentoo.org>. The repo name is 'catalyst'.
Thanks. This is in SVN.
Please test genkernel 3.4.9_prer1 or better. This should be fixed now.
Works for me. Thanks.
