Among the myriad of other improvement in the 2.6 series Linux kernel, a new infrastructure, "kbuild", has been developed to create a highly configurable and versatile build system. The full scope of this change is not the focus of this document. Of importance is the new recommended method for compiling external kernel modules against a 2.6 kernel tree. From Documentation/kbuild/modules.txt:

Compiling modules outside the official kernel
---------------------------------------------
Often modules are developed outside the official kernel.
To keep up with changes in the build system the most portable way
to compile a module outside the kernel is to use the following command-line:

make -C path/to/kernel/src SUBDIRS=$PWD modules

By overriding the SUBDIRS variable, the latest kbuild infrastructure can easily be used to compile external modules. This in itself is a great boon, but the particulars are where things become problematic.

Having configured and compiled a 2.6 series kernel residing in /usr/src/linux, it's not unreasonable to want to compile some external modules against this kernel source tree. Beyond that, one would hope that this could be done by an unprivileged user. Trying to do so shows that this is not the case. Such an attempt fails as make attempt to update files in /usr/src/linux, which an unprivileged user has no access to write to.

Modules being built within the portage system suffer from this exact same problem, dying with sandbox violations when such updates are attempted. The added features of kbuild have also added complexity of a sort which is very problematic for those wishing to follow safe practices of least privilege.

Searching Gentoo's bugzilla will find many bugs opened because of this very problem, and many forums posts have arisen suggesting sandbox disabling as the only solution. Indeed, with the current state of affairs, it seemed inevitable that some writing or updating to /usr/src/linux would be inevitable for modules built against 2.6 kernels.

One solution exists for this problem which does not require much change to the current way modules are handled. By creating a build directory in the sandbox and symlinking the important files from /usr/src/linux into this build path, the access violations can be made to disappear. Although this fix works, it is a bit more of a "hack" than a full-fledged solution. For backwards compatibility, support for this has been added to the new kernel-mod.eclass. However, users should be strongly urged to use the new koutput method which will be detailed below.

The cleaner solution has arisen out of a realization that fighting the new features of kbuild would only result in further and further problems in the future. In fact, kbuild's new ability to send all output files to a separate directory would prove to be the solution. Keeping the kernel source tree completely clean and allowing external modules to build against a clean source tree while they themselves sent their output files to a temporary directory is the key.

The details of how these elements interplay is somewhat complex. Hopefully the following section will break the concepts into more easily digestable pieces, such that developers and users wishing to enhance or use this new system will have the knowledge needed to do so.

kbuild provides two variables for dictating where the kernel should output its files.

The combination of these two variables is the key to successfully using kbuild and portage to install kernel modules.

To make use of the kernel output features, the new kernel-2.eclass written by John Mylchreest has been patched to add a default KBUILD_OUTPUT path in the top level kernel makefile (/usr/src/linux/Makefile). By default, this path will be set to /var/tmp/kernel-output/${KV} as /var/tmp is a suitable place for temporary files that need to remain between reboots.

Once this variable is set, all make commands issued in the kernel source tree send their outputs to this new directory. No further work is required on the users part and the change is most (see the next section for exceptions) seamless. Once a kernel is installed, a user can still just do:

# make menuconfig

and they will be able to configure their kernel and then build it.

As all generated files are now placed in a separate directory, a few key files will end up in a place unexpected by the user. In particular, a users .config and bzImage will no longer be in /usr/src/linux as they were before. Peter Johanson has written a new 2.6 kernel guide oriented at end users which emphasises the kbuild changes and in particular the new locations of these files.

Now that kernels are outputting their files to a different location, writing ebuilds that adhere to the new system is the next step. Updating ebuilds to look for certain generated files and headers in the proper location and getting them using the correct Makefile magic is key.

The general idea is to patch Makefiles in packages so that when compiling against 2.6 kernels, they use the O variable to output to a temporary directory within our sandboxed environment. Adding a few small pieces to make this all succeed and we've got another successfully merging external module package.

Changes have been added to the kernel-mod.eclass to include some variables and tools that ebuild authors may use to more easily affect these changes on the ebuild level.

The new koutput setup adds certain necessities to the Makefiles of kernel module packages. In particular, they shall need to use the kbuild variable O to output the built files into some subdirectory of WORKDIR. Usually this will be something like ${S}/tmp. When using this system, the output directory must have the kernel's .config file present in that directory. The copying of the config can be done in either the makefile or in the ebuild, but doing so in the makefile in a sensible way might lead to an upstream patch aleviating Gentoo from doing so.

All kernel module ebuilds should now use the kernel-mod.eclass. Here's a quick rundown of its use. The kernel-mod_src_unpack() function handles unpacking things, as well as setting a group of extremely useful variables which can be used elsewhere. Three variables can be set by the ebuild which affect how kernel-mod_src-unpack() functions.

kernel-mod_src_unpack() works very hard to figure out how the driver being built should be handled. In particular, it handles the two different 2.6 build methods very well. In particular, if the old style of building modules is detected, the directory KV_BUILD will be created with necessary symlinks to /usr/src/linux such that that directory can be used as the "kernel source directory" and avoid sandbox errors. If the new method using koutput is detected, this will not be done, and the patch in KERNEL_MOD_KOUTPUT_PATCH will be applied if it exists. After kernel-mod_src_unpack() has been called, there are a plethora of variables available for use elsewhere in the ebuild.

What KV_OUTPUT and KV_BUILD end up being set to is ultimately the factor dictating which kernel setup we've found, and which method of building will be used. Here's a table showing the three different kernel configurations, and what these two variables will be set to after kernel-mod_src_unpack() is called.

Most ebuilds will generally need one patch to the makefiles to enabled outputing to a different directory, and then do some sedding based on what is found in KV_OUTPUT and KV_BUILD after kernel-mod_src_unpack() is called. The example ebuild below will show how that is handled. For those ebuilds needing to more work by hand (nvidia-kernel is one), there are a few functions that may be called individually for more find grained control.

The following is a real world example of how the hostap-driver package was fixed to create a fully functioning ebuild for both 2.4 and 2.6 series kernels. It uses a patch to the hostap-driver's top level Makefile (which is generalized enough to be sent upstream and allow future ebuilds to be simplified) as well as some changes to the ebuild to account for some file location changes.

Below are some excerpts from the original Makefile and some new and modified sections which enable proper building. First, we need to fix a line which included the .config expected to be in the path with the kernel sources.

include $(KERNEL_PATH)/.config

Below is the fixed version which lets KERNEL_OUTPUT_PATH be set if not already (for 2.4 backwards compatibility) and looks for the .config in that location.

ifndef KERNEL_OUTPUT_PATH
KERNEL_OUTPUT_PATH=$(KERNEL_PATH)
endif

include $(KERNEL_OUTPUT_PATH)/.config

Since we now have a KERNEL_OUTPUT_PATH variable at our disposal, fixing the following variable declaration involving the generated version.h is simple. The original:

VERFILE := $(KERNEL_PATH)/include/linux/version.h

And now the edited fixed version:

VERFILE := $(KERNEL_OUTPUT_PATH)/include/linux/version.h

Finally, our patch fixes the line which invokes the 2.6 kbuild system to include the use of the O variable, setting the system to output to a directory tmp/ in the current directory.

Here is the original:

$(MAKE) -C $(KERNEL_PATH) SUBDIRS=$(PWD)/driver/modules \
	MODVERDIR=$(PWD)/driver/modules modules

And here is the edited version:

mkdir -p $(PWD)/tmp
-cp $(KERNEL_OUTPUT_PATH)/.config $(PWD)/tmp
$(MAKE) -C $(KERNEL_PATH) O=$(PWD)/tmp \
	SUBDIRS=$(PWD)/driver/modules \
	MODVERDIR=$(PWD)/driver/modules  modules

The changes to the ebuild are relatively straightforward. Things are made particularly easy by the kernel-mod eclass. Here are the important pieces of the src_unpack section and some variables we set in the ebuild:

KERNEL_MOD_SOURCES="${P}.tar.gz"
KERNEL_MOD_KOUTPUT_PATCH="${PN}-koutput.diff.gz"

src_unpack() {
    # Unpack and set some variables
    kernel-mod_src_unpack
	
    ## unpack the pcmcia-cs sources if needed
    pcmcia_src_unpack

    epatch "${FILESDIR}/${P}.firmware.diff.bz2"
    
    # Handle two possible 2.6 kernel build methods
    if [ "${KV_MINOR}" -gt "4" ]
    then
	# If outputting to the source directory, use KV_BUILD as the kernel
	# else we've patched for koutput and we can set the variable we added
	# in the patch

        if [ "${KV_OUTPUT}" = "/usr/src/linux" ]; then
            sed -i -e s:^KERNEL_PATH=.*:KERNEL_PATH=${KV_BUILD}:" \
                ${S}/Makefile
        else
            sed -i -e \
	        "s:^# KERNEL_OUTPUT_PATH=.*:KERNEL_OUTPUT_PATH=${KV_OUTPUT}:" \
                ${S}/Makefile
	fi
    fi
}

Notice the use of the two variables, KERNEL_MOD_SOURCES and KERNEL_MOD_KOUTPUT_PATCH. These can be set as needed to specify which sources should be unpacked by kernel-mod_src_unpack() and a file to patch if we find that koutput is being used. KERNEL_MOD_KOUTPUT_PATCH will not be applied if it is discovered that the 2.6 kernel is still outputing to /usr/src/linux in the old fasion. Here is a trimmed down version of the src_compile() function:

src_compile() {
    # Configure the pcmcia-cs sources as needed.
    pcmcia_configure

    einfo "Building hostap-driver for kernel version: ${KV}"
    case ${KV_MINOR} in
        [34])
            local mydrivers

            use pcmcia && mydrivers="${mydrivers} pccard"
            use hostap-nopci || mydrivers="${mydrivers} pci"
            use hostap-noplx || mydrivers="${mydrivers} plx"

            einfo "Building the following drivers: ${mydrivers}"
            emake ${mydrivers} || die "make failed"
            ;;
        [56])
            unset ARCH
            emake all || die "make failed"
            ;;
        *)
            eerror "Unsupported kernel version: ${KV}"
	    die
	    ;;
    esac
}

Notice in particular that the ARCH variable needs to be unset. The new kernel makefiles use ARCH to determine what to build for and use different syntax for i386, etc.

This ebuild's src_install() function will not be shown, as it basically just installs all the modules into /lib/modules/${KV}/. Of note it handles the change in kernel module extensions from .o to .ko in the 2.6 kernel. kernel-mod.eclass sets the variable KV_OBJ to either o or ko as appropriate to make things easier on you.