Script to rebuild packages that compile/install kernel modules. Since I have seen several questions on the gentoo-user ML and have been bitten by not recompiling packages after installing/recompiling a kernel, I have created kernelmod-rebuild. The script is based upon revdep-rebuild, but instead of looking for broken dependencies, it looks for packages that have installed kernel modules. I personally think that this would be a good addition to the Gentoo toolkit. Sample Session: garath root # kernelmod-rebuild --quiet --pretend Finding packages which install kernel modules... Packages will be recompiled. Finding ebuilds... done. Evaluating package order... done. All prepared. Starting rebuild... These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild R ] media-sound/alsa-driver-0.9.8 [ebuild R ] media-video/nvidia-kernel-1.0.4496-r3 Now you can remove -p (or --pretend) from arguments and re-run kernelmod-rebuild. Both the source and ebuild tarballs may be downloaded from http://varnerfamily.org/pvarner/gentoo Reproducible: Always Steps to Reproduce: 1. 2. 3.
Created attachment 21598 [details] kernelmod-rebuild-0.1.ebuild Ebuild for kernelmod-rebuild.0.1
the idea looks interesting, you'll have to give me a couple of days to look at the implementation, I'm a bit busy at the moment
i thought drobbins changes makes this not needed with latest portage see Bug #1477
Looking at bug 1477, that appears to fix the issue with having different kernels installed and only being able to have alsa-driver (for example) installed for one of them. It doesn't solve the issue that I was looking at which is I recompile my kernel with a different config (I've had that break 3rd party modules) or installing a new kernel and remembering all of the 3rd party packages that install modules and rebuilding them to support the new kernel. I built this not so much to fix something broken, but to aid in the different kernel install and kernel recompilation process. So I still see this as a useful utility, however, I will do some testing with the current portage and different kernels to see what happens and what doesn't happen. Now I'm off to look at the Portage-ng roadmap to see how it is planning to handle this type of thing :)
true, i thought the purpose of the pkg was to cover 1477, not a new/rebuilt kernel
I installed portage 2.0.49-r18 and did some testing. I feel that there is definitely a need for this utility. Additionally, on my machine it doesn't appear that the bug fix to 1477 worked. (I have added a comment to that case)
Created attachment 21640 [details] Kernelmod-rebuild test results
Comment on attachment 21640 [details] Kernelmod-rebuild test results >Current kernel - 2.4.22-gentoo-test-r1 (only kernel installed) >Note: I'm using genkernel to build/rebuild kernels due to it being recommended in the install guide. > >Kernel Module Packages Installed: >media-sound/alsa-driver-0.9.8 >media-video/nvidia-kernel-1.0.4496-r3 >net-fs/shfs-0.31-r1 >net-fs/ftpfs-0.6.2-r3 > >Testing - >1. Installed Portage 2.0.49-r18 >2. run genkernel to rebuild current kernel > >Results: >alsa-driver: modules removed - need to reinstall >nvidia-kernel: modules not removed - no need to reinstall >shfs: modules removed - need to reinstall >ftpfs: modules removed - need to reinstall > >fix: run kernelmod-rebuild --quiet -- --usepkg --verbose > >Results: >alsa-driver: modules installed >nvidia-kernel: modules installed >shfs: modules installed >ftpfs: modules installed > >3. Installed gentoo-sources kernel (2.4.20-gentoo-r9) >4. Copied config from gentoo-test kernel to gentoo kernel >5. changed /usr/src/linux to point to /usr/src/linux-2.4.20-gentoo-r9 >6. Run genkernel to build kernel > >Results: >gentoo-test-sources >alsa-driver: modules installed >nvidia-kernel: modules installed >shfs: modules installed >ftpfs: modules installed > >gentoo-sources >alsa-driver: no modules - need to install >nvidia-kernel: no modules - need to install >shfs: no modules - need to install >ftpfs: no modules - need to install > >Fix: env CONFIG_PROTECT="/lib/modules/2.4.22-gentoo-test-r1" run kernelmod-rebuild --quiet -- --verbose > >Results: >gentoo-test-sources >alsa-driver: modules installed >nvidia-kernel: modules installed >shfs: modules installed >ftpfs: modules installed > >gentoo-sources >alsa-driver: modules installed >nvidia-kernel: modules installed >shfs: modules installed >ftpfs: modules installed
Discovered that latest portage does not have the fix from bug 1477 included. Using teh CONFIG_PROTECT variable to manually protect the modules, I reran the test and have updated the results. Overall, my extended testing has shown this to be useful and once the fix for 1477 makes it into portage should help reduce a lot of issues with kernel module rebuilds and installation.
Created attachment 21657 [details] Kernelmod-rebuild test results
After this was featured in GWN, I can say that there is definitely interest in this. Anyhow based on the comments I have received, I'm in the process of writing the 0.2 release. I will attach the ebuild when finished.
I have finished the 0.2 version and it is currently being tested, If no problems are found, I wll attach the 0.2 ebuild tommorrow.
I have completed the 0.2 version. The complete ebuild is located at: http://varnerfamily.org/pvarner/gentoo/kernelmod-rebuild-0.2.ebuild.tar.gz The source is at: http://varnerfamily.org/pvarner/gentoo/kernelmod-rebuild-0.2.tar.gz Changes are: Smart rebuilding switch - don't re-emerge the package if it finds that the module already exists. Configuration file - addition of a configuration file in /etc to allow you to blacklist/mask packages against certain kernels and add directories other than /lib/modules to look for modules. Ignore switch - tell it to ignore temporary files that already exist Aged temporary files - don't use the temporary files if they are older than one day. No color switch - Turn off colored output, also uses the NOCOLOR variable from portage Ordered Rebuild option. Without the option, I don't run the package ordering logic. I really don't think that it is needed for what this script does. Not ordering the packages, significantly speeds up the script.
Created attachment 22547 [details] kernelmod-rebuild-0.2 ebuild
Created attachment 23526 [details] kernelmod-rebuild-0.2.1.ebuild Updated package. I broke the --package-names option with the 0.2 release. This version fixes the bug. Available from http://varnerfamily.org/pvarner/gentoo
Is this a kernel issue or a feature that should be added to portage/gentoolkit? I am personally voting for a portage/gentoolkit feature add. Perhaps we could bounce this back to the portage team and see what they have to say.
I like the script, so please add it to the gentoolkit. :')
Created attachment 44805 [details] kernelmod-rebuild-0.2.2.ebuild Updated ebuild. It removes the dependency from the deprecated qpkg and uses equery instead. Full ebuild for a portage overlay can be found at http://varnerfamily.org/pvarner/gentoo
Created attachment 46508 [details] kernelmod-rebuild-0.2.3.ebuild Updated for equery in gentoolkit-0.2.0
I am using this script quite some time now without any problems. Would be nice to see it in the tree.
I've used this for months on six systems spanning 2.4 and 2.6 kernels. The latest version has run consistently and well on all of them. I'd be really happy to see this in the package database.
Opinion of kernel group?
This, all in all is a nice idea. Currently however this is in conflict with what the kernel team have in mind, however our attention to this problem has been lacking as of late. Can I please suggest to the author, to look into utilising linux-mod to install, update, maintain some kind of module cache (something like ${CATEGORY}/${PN} would be nice) and then look at using the external tool to utilise this cache instead. not only will be a dramatic speed increase over your implementation, it will look at helping us move forward with this. Another idea, which was partially implemented, was to optionall install module source (and required build params) to /usr/src/kernel-modules/${P}/ which was then processed by an external tool. This is nice when we think about the size of userland tools for some of these modules - however the idea now is to try split it out into module-driver module-userland and module-firmware. Anyways, interested in hearing back you comments.
I just want to make sure that I understand. 1. Make changes to the linux-mod eclass to create a cache for all external modules installed via portage. 2. Have the kernelmod-rebuild script use the cache to determine if an external module needs to be rebuilt. This is instead of using equery to find the modules Additionally, would you be willing to elaborate on how the current script conflicts with the kernel team's goals.
Sure. wrt your first question. Correct. linux-mod will be used for every single module installed with portage, and therefore should be quite accurate. postrm should remove the entry from the cache, postinst should add it to/check for it in the cache. Basically, the overall goal is to be able to automate kernel installs (if configured to do so of course) and all of this will be focussed through the eclectic inteface. eclectic is a kind of central configuration tool, currently under development. this tool will be used to interact in numerous ways with the end user experience. ie: auto symlink, auto-build (genkernel), auto-install, auto-rebuild-modules. as a few ideas. Your script will be, more than anything, inspiration for the final implementation and a stop-gap for those wishing to utilise this behaviour now. It isnt in conflict with our goals assuming the implementation were to incorporate linux-mod, and I am sure we can cater for such a tool within the tree. One of the problems we potentially face, is rebuilding userland applications unneccessarily. this is why we throught of installing module sources for this, although all in all, I think the trade of speed for managability is worth it.
*** Bug 88938 has been marked as a duplicate of this bug. ***
please find module-rebuild in the tree, in a couple of hours time. This should hopefully solve your problems. Any questions, just ask :) Thanks very much.
*** Bug 95331 has been marked as a duplicate of this bug. ***
*** Bug 89444 has been marked as a duplicate of this bug. ***
*** Bug 29634 has been marked as a duplicate of this bug. ***
