Go to:
Gentoo Home
Documentation
Forums
Lists
Bugs
Planet
Store
Wiki
Get Gentoo!
Gentoo's Bugzilla – Attachment 25229 Details for
Bug 40933
New Developer Document on proposed kernel output changes for 2.6
Home
|
New
–
[Ex]
|
Browse
|
Search
|
Privacy Policy
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
Developer Document detailing the proposed koutput changes
2.6-koutput.xml (text/plain), 14.40 KB, created by
Peter Johanson (RETIRED)
on 2004-02-08 20:54:07 UTC
(
hide
)
Description:
Developer Document detailing the proposed koutput changes
Filename:
MIME Type:
Creator:
Peter Johanson (RETIRED)
Created:
2004-02-08 20:54:07 UTC
Size:
14.40 KB
patch
obsolete
><?xml version='1.0' encoding="UTF-8"?> ><!-- $Header: $ --> ><!DOCTYPE guide SYSTEM "/dtd/guide.dtd"> > ><guide link="/doc/en/2.6-koutput.xml"> > ><title>2.6 Kernel Output</title> ><author title="Developer"> > <mail link="latexer@gentoo.org">Peter Johanson</mail> ></author> > ><license/> > ><abstract> >This guide is aimed at developers and covers the details of the new kernel output changes in Gentoo. ></abstract> > ><version>1.0</version> ><date>February 8, 2004</date> > ><chapter> ><title>Preface</title> > ><section> ><title>The New System</title> ><body> ><p>Among the myriad of other improvement in the 2.6 series Linux kernel, a new infrastructure, "kbuild," has been developed to create a highly configurable and versatile build system. The full scope of this change is not the focus of this document. Of importance is the new recommended method for compiling external kernel modules against a 2.6 kernel tree. From <path>Documentation/kbuild/modules.txt</path>:</p> ><pre> >Compiling modules outside the official kernel >--------------------------------------------- >Often modules are developed outside the official kernel. >To keep up with changes in the build system the most portable way >to compile a module outside the kernel is to use the following command-line: > >make -C path/to/kernel/src SUBDIRS=$PWD modules ></pre> ><p>By overriding the SUBDIRS variable, the latest kbuild infrastructure can easily be used to compile external modules. This in itself is a <b>great</b> boon, but the particulars are where things become problematic.</p> ></body> ></section> > ><section> ><title>The Problem</title> ><body> ><p>Having configured and compiled a 2.6 series kernel residing in <path>/usr/src/linux</path>, it's not unreasonable to want to compile some external modules against this kernel source tree. Beyond that, one would hope that this could be done by an unpriveleged user. Trying to do so shows that this is <b>not</b> the case. Such an attempt fails as <i>make</i> attempt to update files in <path>/usr/src/linux</path>, which an unpriveleged user has no access to write to.</p> ><p>Modules being built within the portage system suffer from this exact same problem, dying with sandbox violations when such updates are attempted. The added features of kbuild have also added complexity, of a sort which is very problematic for those wishing to follow safe practices of least privelege.</p> ><p>Searching Gentoo's bugzilla will find many bugs openned because of this very problem, and many forums posts have arisen suggesting sandbox disables as the only solution. Indeed, with the current state of affairs, it seemed inevitable that some writing or updating to <path>/usr/src/linux</path> would be inevitable for modules built against 2.6 kernels.</p> ></body> ></section> > ><section> ><title>Embracing Change</title> ><body> ><p>The solution has arisen out of a realization that fighting the new features of kbuild would only result and further and further problems in the future. In fact, kbuild's new ability to send all output files to a seperate directory would prove to be the solution. Keeping the kernel source tree completely clean, and allowing external modules to build against a clean source tree while they themselves sent their output files to a temporary directory is the key.</p> ><p>The details of how these elements interplay is somewhat complex. Hopefully the following section will break the concepts into more easily digestable pieces, such that developers and users wishing to enhance or use this new system will have the knowledge needed to do so.</p> ></body> ></section> > ></chapter> > ><chapter> ><title>Kernel Output</title> > ><section> ><title>KBUILD_OUTPUT and O variables</title> ><body> ><p>kbuild provides two variables for dictating where the kernel should output its files. These variables and their roles and differences are listed in Figure 2.1.</p> ><table> > <tr> > <th>Variable</th> > <th>Usage</th> > </tr> > <tr> > <ti>KBUILD_OUTPUT</ti> > <ti>This variables can be defined in the top-level kernel Makefile. This can be used if you want to set a default output other than in the source tree itself.</ti> > </tr> > <tr> > <ti>O</ti> > <ti>This variable is to be used on the command line to override any other settings and indicate where the output for the current command should be placed.</ti> > </tr> ></table> ><p>The combination of these two variables is the key to successfully using kbuild and portage to install kernel modules.</p> ></body> ></section> > ><section> ><title>Kernel Installation Changes</title> ><body> ><p>To make use of the kernel output features, the new kernel-2.eclass written by <mail link="johnm@gentoo.org">johnm</mail> has been patched to add a default KBUILD_OUTPUT path in the top level kernel makefile (<path>/usr/src/linux/Makefile</path>). By default, this path will be set to <path>/var/tmp/kernel-output/${KV}</path> as <path>/var/tmp</path> is a suitable place for temporary files that need to remain between reboots.</p> ><p>Once this variable is set, all make commands issued in the kernel source tree send their outputs to this new directory. No further work is required on the users part, and the change is most (see the next section for exceptions) seamless. Once a kernel is installed, a user can still just do:</p> ><pre> ># <i>make menuconfig</i> ></pre> ><p>and they will be able to configure their kernel and then build it.</p> ></body> ></section> > ><section> ><title>Important File Location Changes</title> ><body> ><p>As all generated files are now placed in a seperate directory, a few key files will end up in a place unexpected by the user. In particular, a users <path>.config</path> and <path>bzImage</path> will no longer be in <path>/usr/src/linux</path> as they were before. <mail link="latexer@gentoo.org">Latexer</mail> has written a new 2.6 kernel guide oriented at end users which emphasises the kbuild changes, and in particular the new locations of these files.</p> ></body> ></section> ></chapter> > ><chapter> ><title>External Module Ebuilds</title> > ><section> ><title>General Approach</title> ><body> ><p>Now that kernels are outputing their files to a different location, writing ebuilds that cohere to the new system is the next step. Updating ebuilds to look for certain generated files and headers in the proper location, and getting them using the correct Makefile magic is key.</p> ><p>The general idea is to patch Makefiles in packages so that when compiling against 2.6 kernels, they use the O variable to output to a temporary directory within our sandboxed environment. Adding a few small pieces to make this all succeed, and we've got another successfully merging external module package.</p> ><note>The actual usability and functionality of modules after they are built and installed is <b>not</b> the goal of this document. That is a whole seperate issue that upstream driver authors should be handling.</note> ><p>Changes have been added to the <i>kernel-mod.eclass</i> to include some variables and tools that ebuild authors may use to more easily affect these changes on the ebuild level.</p> ></body> ></section> > ><section> ><title>Makefile requirements</title> ><body> ><p>The new koutput setup adds certain necessities to the Makefiles of kernel module packages. In particular, the shall need to use the kbuild variable <i>O</i> to output the built files into some subdirectory of <i>WORKDIR</i>. Usually this will be something like <path>${S}/tmp</path>. When using this system, the output directory must have the kernel's <path>.config</path> file present in that directory. The copying of the config can be done in either the makefile or in the ebuild, but doing so in the makefile in a sensible way might lead to an upstream patch aleviating Gentoo from doing so.</p> ></body> ></section> > ><section> ><title>kernel-mod.eclass Features</title> ><body> ><p>All kernel module ebuilds should now use the kernel-mod.eclass. Here's a quick rundown of its use. The kernel-mod_src_unpack() function handles unpacking tihngs, as well as setting a group of extremely useful variables which can be used elsewhere. Two variables can be set by the ebuild which affect how kernel-mod_src-unpack().</p> ><table> > <tr> > <th>Variable</th> > <th>Usage</th> > </tr> > <tr> > <ti>KERNEL_MOD_SOURCES</ti> > <ti>If set, these files (tarballs usually) will be unpacked into WORKDIR. Otherwise ${A} will be unpacked.</ti> > </tr> > <tr> > <ti>KERNEL_MOD_KOUTPUT_PATCH</ti> > <ti>If set, this patch will be applied using epatch in ${S}. This could be done by hand of course, but this might be useful to avoid needing to define a src_unpack() at all in an ebuild.</ti> > </tr> ></table> ><p>After kernel-mod_src_unpack() has been called, there are a plethora of variables available for use elsewhere in the ebuild.</p> ><table> > <tr> > <th>Variable</th> > <th>Meaning</th> > </tr> > <tr> > <ti>KV_OUTPUT</ti> > <ti>The full path where the kernel is outputting its files. FOr 2.4 kernels this will always be <path>/usr/src/linux</path>. For 2.6 this will hopefully be a different directory (otherwise things will fail during module compilation).</ti> > </tr> > <tr> > <ti>KV_OJB</ti> > <ti>The extension for kernel modules for this kernel version. Either "ko" or "o" depending.</ti> > </tr> > <tr> > <ti>KV_VERSION_FULL</ti> > <ti>The full kernel version.</ti> > </tr> > <tr> > <ti>KV_MAJOR</ti> > <ti>The major number of the kernel.</ti> > </tr> > <ti>KV_MINOR</ti> > <ti>The minor number of the kernel.</ti> > <tr> > <ti>KV_PATCH</ti> > <ti>The patch number of the kernel.</ti> > </tr> > <tr> > <ti>KV_TYPE</ti> > <ti>The type of kernel (e.g. "-gentoo-r1" in "2.4.23-gentoo-r1")</ti> > </tr> ></table> > ></body> ></section> > ><section> ><title>Example ebuild and Makefile fix</title> ><body> ><p>The following is a real world example of how the hostap-driver package was fixed to create a fully functioning ebuild for both 2.4 and 2.6 series kernels. It uses a patch to the hostap-driver's top level Makefile (which is generalized enough to be sent upstream and allow future ebuilds to be simplified) as well as some changes to the ebuild to account for some file location changes.</p> ><p>Below are some excerpts from the original Makefile and some new and modified sections which enable proper building. First, we need to fix a line which included the <path>.config</path> expected to be in the path with the kernel sources. </p> ><note>The KERNEL_OUTPUT_PATH variable is added earlier in the build to complement the KERNEL_PATH variable in the original.</note> ><pre caption="Original include line"> >include $(KERNEL_PATH)/.config ></pre> ><p>Below is the fixed version which lets KERNEL_OUTPUT_PATH be set if not already (for 2.4 backwards compatibility) and looks for the <path>.config</path> in that location</p> ><pre caption="Fixed include line"> >ifndef KERNEL_OUTPUT_PATH >KERNEL_OUTPUT_PATH=$(KERNEL_PATH) >endif > >include $(KERNEL_OUTPUT_PATH)/.config ></pre> ><p>Since we know have a the KERNEL_OUTPUT_PATH variable at our disposal, fixing the following variable declaration involving the generated <path>version.h</path> is simple. The original:</p> ><pre> >VERFILE := $(KERNEL_PATH)/include/linux/version.h ></pre> ><p>And now the edited fixed version:</p> ><pre> >VERFILE := $(KERNEL_OUTPUT_PATH)/include/linux/version.h ></pre> ><p>Finally, our patch fixes the line which invokes the 2.6 kbuild system to include the use of the O variable, setting the system to output to a directory <path>tmp</path> in the current directory.</p> ><warn>kbuild expects to find a valid <path>.config</path> in the output directory specified by <i>O=foo</i>. Either the ebuild or the makefile should be edited to copy the <path>.config</path> in ${KV_OUTPUT} (set by kernel-mod.eclass) into the output directory.</warn> ><p>Here is the original:</p> ><pre> >$(MAKE) -C $(KERNEL_PATH) SUBDIRS=$(PWD)/driver/modules \ > MODVERDIR=$(PWD)/driver/modules modules ></pre> ><p>And here is the edited version:</p> ><pre> >mkdir -p $(PWD)/tmp >-cp $(KERNEL_OUTPUT_PATH)/.config $(PWD)/tmp >$(MAKE) -C $(KERNEL_PATH) O=$(PWD)/tmp \ > SUBDIRS=$(PWD)/driver/modules \ > MODVERDIR=$(PWD)/driver/modules modules ></pre> ><p>The changes to the ebuild are relatively straight forward. Things are made particularly easy by the kernel-mod eclass. Here are the important pieces of the src_unpack section and some variables we set in the ebuild:</p> ><pre caption="src_unpack() function"> >KERNEL_MOD_SOURCES="${P}.tar.gz" >KERNEL_MOD_KOUTPUT_PATCH="${PN}-koutput.diff.gz" > >src_unpack() { > # Unpack and set some variables > kernel-mod_src_unpack > > ## unpack the pcmcia-cs sources if needed > pcmcia_src_unpack > > epatch "${FILESDIR}/${P}.firmware.diff.bz2" > > # If on 2.5 or 2.6, set the kernel output stuff > if [ "${KV_MINOR}" -gt "4" ] > then > sed -i -e "s:^# KERNEL_OUTPUT_PATH=.*:KERNEL_OUTPUT_PATH=${KV_OUTPUT}:" \ > ${S}/Makefile > fi >} ></pre> ><p>Notice the use of the two variables, <i>KERNEL_MOD_SOURCES</i> and <i>KERNEL_MOD_KOUTPUT_PATCH</i>. These can be set as needed to specify which sources should be unpacked by kernel-mod_src_unpack(), and a file to patch if needed to fix the kernel output in the package's Makefiles. Here is a trimmed down version of the src_compile() function:</p> ><pre caption="src_compile() function"> >src_compile() { > # Configure the pcmcia-cs sources as needed. > pcmcia_configure > > einfo "Building hostap-driver for kernel version: ${KV}" > case ${KV_MINOR} in > [34]) > local mydrivers > > use pcmcia && mydrivers="${mydrivers} pccard" > use hostap-nopci || mydrivers="${mydrivers} pci" > use hostap-noplx || mydrivers="${mydrivers} plx" > > einfo "Building the following drivers: ${mydrivers}" > emake EXTRA_CFLAGS="-DPRISM2_DOWNLOAD_SUPPORT" ${mydrivers} || die "make failed" > ;; > [56]) > unset ARCH > emake EXTRA_CFLAGS="-DPRISM2_DOWNLOAD_SUPPORT" all || die "make failed" > ;; > *) > eerror "Unsupported kernel version: ${KV}" > die > ;; > esac >} ></pre> ><p>Notice in particular that the <i>ARCH</i> variable needs to be unset. The new kernel makefiles use <i>ARCH</i> to determine what to build for, and use different syntax for i386, etc.</p> ><p>This ebuild's src_install() function will not be shown, as it basically just installs all the modules into <path>/lib/modules/${KV}/</path>. Of note however is handling of the change in kernel module extensions from <i>.o</i> to <i>.ko</i> in the 2.6 kernel. kernel-mod.eclass sets the variable <i>KV_OBJ</i> to either <i>o</i> or <i>ko</i> as appropriate to make things easier on you.</p> ></body> ></section> > ></chapter> > ></guide>
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 40933
:
25229
|
25276
|
25314
|
25339
|
25457
|
25461
|
25496
|
27437
|
28025
|
28081
|
28194