Bugzilla Bug 180211

In livecd stage2 busybox is configured, compiling and copied to cache. On
following runs busybox is compiled again, the cache is not used. Cache for
other components of the kernel build are pulled from the cache such as the
kernel and module-init-tools.

Reproducible: Always

Steps to Reproduce:
1. Run livecd-stage2 build, notice busybox is compiled and copied to cache.
2. Run livecd-stage2 again, notice busybox is compiled again.


Expected Results:  
busybox package should be used from the cache instead of recompiled.

This isn't catalyst. Genkernel is the one that builds busybox.

Looking at compile_busybox() in gen_compile.sh, it looks like there's some
basic logic for using the cached busybox, but it's not complete. It will remove
the cache if the busybox config is newer than what the cached version was built
with. It also checks to see if the cache does not exist before compiling.
However, there is no actual code to use the cache.

Since this isn't a bug that actually prevents you from building a kernel, this
probably isn't a priority. You're welcome to whip up a patch and submit it,
though. This will make it a lot more likely that this problem gets fixed.

Okay, what I said was completely wrong. There doesn't need to be any more logic
to use an existing busybox bincache. The code that compiles just creates the
bincache, which is what's used to create the initramfs. So, the problem lies
elsewhere.

It looks like this may be catalyst's fault (indirectly).

The first thing catalyst does when it goes to build the kernel is (re-)emerge
genkernel. When portage installs a package, it creates the files with a
[cm]time of *now*. Because of this, the busy-config from genkernel is *always*
newer than the busybox bincache of the previous run, which causes the bincache
check in gen_compile.sh's compile_busybox() to fail and remove the bincache.

I'm not sure there is a way to fix this.

Could a hash be used (MD5, etc) instead of time?

Perhaps, but where would it be stored?

An option is to package the busy-config file with the bzip2'd busybox binary
(currently, the busybox binary is bzip2'd directly without tar) and then
comparing the md5 of the stored busy-config to the "new" one. It doesn't look
like it would require modification of very much code. I can probably whip up a
patch for this if I get bored at work this afternoon :)

Created an attachment (id=120626) [details]
patch

Okay, so I got bored at work quicker than I thought. This patch implements
saving the busy-config along with the compiled busybox binary and then
comparing the MD5 of the old and new when checking for bincache "staleness". It
is completely untested.

As far as I can tell, this patch should not break with busybox bincache files
from "older" versions of genkernel, since the filename is changing as well. It
will just ignore the old file.

* ERROR: Could not extract busybox bincache!

If I have time I may try to figure this out.

Created an attachment (id=120644) [details]
genkernel-busybox.patch

Modifications:
1. md5sum only the .config file in the cache, not the entire cache.
2. Added -C "..." path to tar extract so that busybox gets put into the right
directory.

Please note that this still does not work because the ${BUSYBOX_CONFIG} file is
copied and modified before building, but the original is used in the
comparison. The cache check may need to be moved later or the original
${BUSYBOX_CONFIG} stored in the cache instead of the modified config. However,
if genkernel changes what is modified using the original config would cause
problems.

Created an attachment (id=120649) [details]
updated patch with old stale cache checking code actually removed

I forgot to actually remove the old code for checking for the stale cache in my
original patch. This one has that removed and your changes.

Created an attachment (id=120651) [details]
updated patch with old stale cache checking code actually removed

I forgot to actually remove the old code for checking for the stale cache in my
original patch. This one has that removed and your changes.

The latest patch has the tar -C option referencing a file, it should be a
directory:

-C "${TEMP}/initramfs-busybox-temp/bin/busybox.tar.bz2" 

should be

-C "${TEMP}/initramfs-busybox-temp/bin" 

Created an attachment (id=120657) [details]
another patch

Whoops, let's try that again...this time paying attention

Created an attachment (id=120661) [details]
patch using modified config

Sorry that the format of this patch isn't the same as yours, I don't have the
SVN checkout. This changes the config that is compared by using the config
created just before compiling. Works for me!

At first glance, the patch looks fine. Also, you can access anon SVN via
<http://anonsvn.gentoo.org>. The repo name is 'catalyst'.

Thanks. This is in SVN.

Please test genkernel 3.4.9_prer1 or better.  This should be fixed now.

Works for me. Thanks.