Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 32737

Summary: Compiling external kernel modules with 2.6 kernels
Product: Gentoo Linux Reporter: Peter Johanson (RETIRED) <latexer>
Component: [OLD] Core systemAssignee: Peter Johanson (RETIRED) <latexer>
Status: RESOLVED REMIND    
Severity: normal CC: aalmenar, chrb, chutz+bugs.gentoo.org, dberkholz, dragonheart, egore, esigra, ferret-bgo, fvdpol, henrik, humpback, jay, jbilbo, kanaka, radek, rmalheiro, scott, swift, wschlich, x86-kernel
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 40099    
Bug Blocks: 33163, 34399, 35501, 35564, 35980, 37229, 42858, 44410, 44807, 45785, 61789    
Attachments: patch kernel-mod.eclass
Patch to get KV set using /usr/src/linux/Makefile
Patch to johnm's kernel-2.eclass to set KBUILD_OUTPUT in /usr/src/linux/Makefile
new hostap-driver ebuild that uses O= foo to compile against clean 2.6 kernel source
pcmcia-cs-3.2.7 ebuild that has a fix for finding .config
More robust ExtractKernelVersion
kernel-mod.eclass changes to make ebuilds easier with new system
Path to linux-wlan-ng ebuild (see details!)
Cleaner hostap-driver ebuild using new kernel-mod.eclass
Cleaner updated pcmcia-cs ebuild
Change pcmcia.eclass to use kernel-mod.eclass stuff
Newest version of kernel-mod.eclass patch
New linux-wlan-ng ebuild using latest {pcmcia,kernel-mod}.eclass changes
Patch to hostap-0.1.3.ebuild to use latest eclasses
Patch to kernel-mod.eclass dubbed "Have My Cake and Eat it Too"
hostap-driver-0.1.3-r1 ebuild that works with latest no matter what.
New prism54 ebuild using "cake and eat it too" kernel-mod.eclass
"Cake and eat it too" fixed to add a symlinks for .tmp_versions
Latest linux-wlan-ng
Updated 3.2.7-r1 ebuild to fix patch for monitor mode!
"Cake and eat it too 2" - 2nd version of the "Cake and Eat it Too" style kernel-mod.eclass patch
nvidia-kernel ebuild using latest foo!
ati-drivers ebuild using new foo
omnibook ebuild
New patch implementing things last discussed
New patch to kernel-2.eclass
New version of kernel-2 eclass patch to make --set-extraversion useful
Ebuild for the ORiNOCO wireless driver
vmmon-4.5.1.7568-koutput.patch
vmware-config-4.5.1.7568-koutput.patch
vmware-workstation-4.5.1.7568.patch
vmware-workstation-4.5.1.7568.ebuild.diff (kalin)
vmmon-4.5.1.7568-koutput.patch (kalin)
vmnet-4.5.1.7568-koutput.patch (kalin)
Patch for net-dialup/slmodem-2.9.7
vmware-workstation-4.5.2.8848.ebuild.patch
vmware-config-4.5.2.8848-koutput.patch
vmnet-4.5.2.8848-koutput.patch
vmmon-4.5.2.8848-koutput.patch
vmware-config-4.5.2.8848-koutput.patch (using POD for neater patch)

Description Peter Johanson (RETIRED) gentoo-dev 2003-11-04 13:10:44 UTC
Openning a new meta-bug for dealing with gentoo's handling of external modules
compiling against 2.6 kernels. In particular, the suggested method in makefiles
of doing
make -C /usr/src/linux SUBDIRS=/some/path/to/external/modules/ modules

fails to succeed since it ends up need to write to /usr/src/linux at some point,
and sandbox kills it.

For now, any ebuilds which need to do this can just have sandbox disabled, but a
real fix needs to be worked on. Elementary support for outputing to a different
directory via KBUILD_OUTPUT and O options to the make line seem to be the right
way to go, but that still has issues.

Ebuilds so far that i know have issues:
hostap (old)/ hostap-driver

Please use this ebuild to discuess any particular ebuilds that suffer from this,
or more generally a fix that will work for all ebuilds that need to add external
modules (that use the above make method)

Reproducible: Always
Steps to Reproduce:
1.
2.
3.
Comment 1 John Mylchreest (RETIRED) gentoo-dev 2003-11-04 14:59:30 UTC
could we perhaps start pushing these kind of ebuilds through an eclass?
am I misunderstood or is kernel-mod.eclass designed for this?
it will of course need work for 2.6 no doubt, an my appologies for not looking
at the eclass properly before posting this.

however regardless, I do feel this would be the more correct solution.
Comment 2 Peter Johanson (RETIRED) gentoo-dev 2003-11-05 07:10:04 UTC
Yeah, I was definitely thinking an appropriate eclass would be the best solution
for this. Just need to find out what foo we need to exert on the kbuild stuff
from 2.6 to get around these sandbox errors.
Comment 3 Donnie Berkholz (RETIRED) gentoo-dev 2003-11-14 12:32:23 UTC
xfree-drm, ati-drivers, ati-gatos are candidates. I'm not sure about nvidia-kernel.
Comment 4 SpanKY gentoo-dev 2003-11-19 18:15:01 UTC
you could look at the script that the cisco-vpnclient-3des client uses
Comment 5 Peter Johanson (RETIRED) gentoo-dev 2004-01-16 08:23:07 UTC
Created attachment 23927 [details, diff]
patch kernel-mod.eclass

It's not a solution, but until we find something good, i think an addition to
kernel-mod.eclass like this patch would be handy. Adds the "check_module_safe"
function which will die with verbose errors if someone is attemtping to install
kernel modules with sandbox or usersandbox enabled.

Anybody using this in ebuilds will have to be *very* sure that the ebuild only
touches /usr/src/linux, cause we *really* don't want to tell users to disable
sandbox to emerge something then have it do Bad Things (tm).

What do people think of this to at least inform users of what's going on?
Comment 6 Peter Johanson (RETIRED) gentoo-dev 2004-01-18 11:06:06 UTC
UPDATE: Okay, after pounding my head against this issue for at least a month, i think i've finally found a viable, moderately sexy solution. I'm gonna post the general concept here now, and patch to a few eclasses and ebuilds to demonstrate will be forthcoming to both the gentoo-dev ML and this bug.

PROBLEM: sandbox issues!

FURTHER PROBLEM: setting O=/some/tmp/path just makes the kbuild stuff complain that you need to clean out your /usr/src/linux by doing "make mrproper:

REAL PROBLEM: fighting the kbuild stuff instead of "embracing and extending"

PROPOSED SOLUTION: the new 2.6 stuff has two different variables used in deciding where output files end up, KBUILD_OUTPUT and the O that has been dicsussed before. O is used for changing the output on the command line, etc. KBUILD_OUTPUT can be defined in /usr/src/linux/Makefile and sets the default output for *all* files built. that means .config, include/linux/version.h, etc.

I'm suggesting that we add to johnm's new kernel-2.eclass to add some sedding to set KBUILD_OUTPUT=/some/to/be/decided/standard/path for all new 2.6 kernels.

REPURCUSIONS:

1) We need to decide on where things would go (i was thinking /var/lib/kernel-output/${KV} maybe?)

2) DOCS!!!!  This is a pretty major change, so we'll need to keep users informed of what's changed, why it's used, etc.

3) check_KV would need fixing cause it keeps thinking include/linux/version.h should be in /usr/src/linux/ (it's generated, so it ends up in KBUILD_OUTPUT)

4) Driver ebuilds will need to use the "make -C /usr/src/linux SUBDIRS=some/path O=some/tmp/dir modules" (most use -C, and SUBDIRS already, but not necessarily the O= stuff). Some also need some makefile editing to look for .config, etc in a different location (this may be the hard part) I'm hoping these types of changes can be generalized and sent upstream so we'll stop having to worry about lots of patching, etc. I've got hostap-driver working by hand with some makefile changes, i'll be getting my local ebuild using it jiving soon.

5) Anybody can compile a kernel from the same code base! joeuser can just do "make O=/home/joeuser/kernel-output/ menuconfig", configure his kernel, then do "make O=/home/joeuser/kernel-output/" and he's compiled a kernel.

6) Unmerging a kernel actually would remove the whole directory (and you'd be able to safely zap one directory to clean things out in KBUILD_OUTPUT, etc.


Okay, if you've made it this far... Comments? It's a big change, but i think we're going to have to embrace this stuff or it's just going to get uglier and uglier trying to get things working. Please post your ideas here, i'll have example patches, ebuilds, etc coming shortly.

p.s. *phew*

Comment 7 Donnie Berkholz (RETIRED) gentoo-dev 2004-01-18 11:16:00 UTC
I like the idea, but I'd suggest /var/tmp or /var/cache based on FHS.

/var/lib:
This hierarchy holds state information pertaining to an application or the system. State information is data that programs modify while they run, and that pertains to one specific host. Users must never need to modify files in /var/lib to configure a package's operation.

/var/tmp:
The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots.

/var/cache:
/var/cache is intended for cached data from applications. Such data is locally generated as a result of time-consuming I/O or calculation. The application must be able to regenerate or restore the data. Unlike /var/spool, the cached files can be deleted without data loss. The data must remain valid between invocations of the application and rebooting the system.
Comment 8 Brian Jackson (RETIRED) gentoo-dev 2004-01-18 11:44:41 UTC
Good job tracking down a solution on this latexer. I'd suggest we send an emissary to LKML to get some input on paths and everything. There may be some standard already in use for other distros that makes sense to follow (although I've not heard of anyone else using this in standard practice). It would be nice if we could be the initators of this standard rather than having to follow some corporate lead set in a boardroom.
Comment 9 Peter Johanson (RETIRED) gentoo-dev 2004-01-18 13:39:10 UTC
donnie: okay, yeah /var/tmp or /var/cache does sound better. /var/cache sounds good except for maybe the "The data must remain valid between invocations of the application and rebooting the system" part which isn't necessarily true even for a single boot as changing something like SMP causes pretty all the object files to be recreated. /var/tmp seems more in line with gentoo's current setup as /var/tmp/portage is our standard location. Personally i think i prefer /var/tmp/<something> (i think either /var/tmp/kernel-output/${KV} or /var/tmp/kernel-objs/${KV})

iggy: I'm not subscribed to LKML, but i can send an email briefly detailing things if you want, asking for feedback.
Comment 10 Peter Johanson (RETIRED) gentoo-dev 2004-01-19 13:13:52 UTC
Created attachment 24094 [details, diff]
Patch to get KV set using /usr/src/linux/Makefile

This is a patch to /usr/lib/portage/pym/portage.py which makes portage use
/usr/src/linux/Makefile to set KV, and not
/usr/src/linux/include/linux/version.h

This is needed since version.h is generated (and ends up in KBUILD_OUTPUT)

This is 2.4 backwards compatible, since the makefiles use the same layout for
the first 4 lines (thank god/thor/someone for that)
Comment 11 Peter Johanson (RETIRED) gentoo-dev 2004-01-19 13:17:27 UTC
Created attachment 24095 [details, diff]
Patch to johnm's kernel-2.eclass to set KBUILD_OUTPUT in /usr/src/linux/Makefile

This is the small bit that makes things get output to
/var/tmp/kernel-output/${KV}

It tells the user what it's doing, makes the directory, etc. Not much here, but
this ends up making the whole shebang interesting/messy/great/horrible

Patch then emerge mm-sources-2.6.1-mm4 to get a taste if you dare.
You can run "make menuconfig" as usual, but all the things output end up in
/var/tmp/kernel-output/${KV}
Comment 12 Peter Johanson (RETIRED) gentoo-dev 2004-01-19 13:25:31 UTC
Created attachment 24096 [details]
new hostap-driver ebuild that uses O= foo to compile against clean 2.6 kernel source

Here's a revised hostap-driver ebuild that now will work if you have a clean
2.6 kernel tree.

I've got linux-wlan-ng-0.2.1_pre16 *almost* working with this same method.
Comment 13 Peter Johanson (RETIRED) gentoo-dev 2004-01-20 07:43:25 UTC
Created attachment 24131 [details]
pcmcia-cs-3.2.7 ebuild that has a fix for finding .config

Here's an ebuild for pcmcia-cs-3.2.7 that has a fix for finding a users .config


The patch is up at http://dev.gentoo.org/~latexer/files/patches/

It's unintrusive and general enough that i may send the patch upstream once i
make it a little prettier.
Comment 14 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 09:37:24 UTC
Created attachment 24646 [details, diff]
More robust ExtractKernelVersion

Here's a more robust patch to portage.py which doesn't traceback if somebody
messes with the variables and doesn't have spaces between the "=" sign. (First
version was a biproduct of my naive familiarity with python)
Comment 15 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 09:45:03 UTC
Created attachment 24647 [details, diff]
kernel-mod.eclass changes to make ebuilds easier with new system

This is a patch against kernel-mod.eclass which does a few things:

1) Set some new variable at peoples disposal -
   KV_OUTPUT - location of kernel output, can be used for both 2.4 and 2.6
kernels as it will be set to /usr/src/linux if KBUILD_OUTPUT isn't set in the
toplevel kernel Makefile.
   KV_OBJ - the extension for kernel objects. If the packages installation make
foo is too annoying, modules can be installed by hand using doinst
foo.${KV_OJB} so you don't have to worry about .o vs .ko

2) Add a kernel-mod_src_unpack for a new better default. If an ebuild needs to
do any patching, makefile editing, etc, they should still call this function in
order to set needed variables, check for a sane environment, etc. 

3) Along with the kernel-mod_src_unpack, a new variable KERNEL_MOD_SOURCES can
be set to all the sources that should be unpacked. This may be different from
${A} if you have patches, or conditionally unpacked things.


Comments? Additions? I'll be adding new versions of the hostap-driver and
pcmcia-cs ebuilds, as well as my working linux-wlan-ng ebuilds soon that use
the items in this new eclass
Comment 16 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 10:57:05 UTC
Created attachment 24649 [details, diff]
Path to linux-wlan-ng ebuild (see details!)

Here is a patch to the 0.2.1_pre16 in portage that uses the new koutput stuff
with the latest revisions in kernel-mod.eclass.

NOTE: This will only work with recent -mm* series kernels. There was/is a bug
with includes in EXTRA_CFLAGS. Sam Ravnborg posted a patch to LKML a while ago
with the patch but it has not made its way into the vanilla kernel yet. That
patch is found here: http://lkml.org/lkml/2004/1/16/247
Comment 17 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 10:59:59 UTC
Created attachment 24650 [details]
Cleaner hostap-driver ebuild using new kernel-mod.eclass

Updated cleaner hostap-driver ebuild using the new features added to the
kernel-mod.eclass
Comment 18 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 11:01:38 UTC
Created attachment 24651 [details]
Cleaner updated pcmcia-cs ebuild

Same cleanups using new kernel-mod.eclass
Comment 19 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 11:06:23 UTC
Okay, I'm going to stop spamming the attachments with fixed ebuilds. Both the linux-wlan-ng and hostap-driver ebuilds along with the patches they pull should be suffient for people to look at to get a sense of what needs to be done on the ebuild end of things.

I have ebuilds for *new* drivers prism54 and at76c503a up at:
http://dev.gentoo.org/~latexer/files/ebuilds/

On the documentation side, I've started work on a general 2.6 kernel guide that will hopefully sufficiently cover the KBUILD stuff so that we can refer users there when the change occurs.

If there is sufficient desire, I will also make up a guide for *developers* detailing how the new system affects them, and how they can write ebuilds to effectively use this new system. Would that be appreciated?
Comment 20 Olivier Crete (RETIRED) gentoo-dev 2004-01-30 16:39:21 UTC
If I understand correctly you want to add a hole in sandbox to a /var/??/kernel-output directory. Why is this any better than making that hole /usr/src/linux? We do need a shared kernel tree in somewhere in any case. Or am I missing something?
Comment 21 Peter Johanson (RETIRED) gentoo-dev 2004-01-30 18:02:09 UTC
No no. this is actually a solution so we exactly *don't* have to do something like that.

What is going on in a nutshell (i'll be writing an extensive doc aimed at developers this weekend detailing all the pieces involved):

Kernel compiles output their stuff to a different directory (/var/tmp/kernel-output/${KV})

External Modules use the kernel source in /usr/src/linux (and sometimes the .config in the output directory) to compile but *OUTPUT* to a subdirectory of their temp directory (this is done with the O=$(PWD)/tmp stuff in the makefiles, see the patches downloaded from any of these ebuilds i've posted) This allows us to build against a clean source tree, and not have sandbox errors. Currently we get this error since it's trying to build using the kernel sources AND the files already output to /usr/src/linux, it thinks it needs to update things (make is a blessing and a curse) and BOOM! sandbox error. Having a clean sources directory, and having external modules output to their own location is the *key* to all this working.

All of the extra work is making sure packages that need to find .config, version.h, etc don't die from these things being in a new location.

The document will hopefully be more detailed, but does this make sense for now?
Comment 22 Olivier Crete (RETIRED) gentoo-dev 2004-01-31 00:27:22 UTC
ahh that's great.. didnt realise you had two separate output directories... But I'm still worried about the un-vanillaness of the solution.. but I guess we dont really have a choice..
Comment 23 Peter Johanson (RETIRED) gentoo-dev 2004-01-31 13:54:20 UTC
If anybody has a more "vanilla" solution that gets things done cleanly, i'm all for it. As this is just taking advantage of a new kernel build feature, i don't see this as being *that* "unvanilla." If users feel like building to the same directory, and disabling sandbox for their modules, all they need to do is edit one line in a Makefile (i'll have that clearly described as an option in the doc i've got going) and voila. The kernel-mod class stuff is flexible enough that this is all it takes.
Comment 24 Peter Johanson (RETIRED) gentoo-dev 2004-02-01 13:17:52 UTC
Added a dep on a new bug i opened specifically for the portage.py changes and assigned to dev-portage. see bug #40099
Comment 25 Ray Russell Reese III (RETIRED) gentoo-dev 2004-02-02 12:17:33 UTC
Correct me if I'm wrong, but as I understand this solution now, for it to work, essentially we have to insure that all new kernels are built using O=/var/tmp/kernel-output/${KV}. That way the kernel source remains clean so ebuilds containing kernel modules can also then output to their own tmp directory inside the sandbox.

Do we plan on modifying the ebuilds for all the sys-kernel/*-sources to patch the Makefile to make that the default output directory? Or modify genkernel? How are we going to handle all the legacy kernel builds where people have manually built their kernels.  Just add checks inside the ebuild and notify the user of the proper Gentoo way of building a linux kernel?

Don't get me wrong, I think this is the best long term solution, but I want to make sure it doesn't create too much of a hassle for users who built their own kernels.
Comment 26 Peter Johanson (RETIRED) gentoo-dev 2004-02-02 12:25:27 UTC
anti: All 2.6 kernels (there's not too many yet, fortunately) would need to use johnm's new kernel-2.eclass (some/most already do). The changed eclass adds a define for KBUILD_OUTPUT which is a variable for use in the Makefile, which can be overriden by the O variable. Those ebuilds would not need to be changed in any way. I was already planning on adding a check in the kernel-mod eclass changes to spit out warnings if a 2.6 kernel is found that is built in the old method. In theory we could do lots of extra work to *not* add the O= stuff in the ebuild and give the user an option to disable sandbox to merge, but since this is not guaranteed to work plus is a security problem i don't see any strong reason for doing this.

To make this less of a hassle, i think strong docs and even an announcement on www.gentoo.org about changes will help ease (at least somewhat) the transition once we're ready.
Comment 27 Peter Johanson (RETIRED) gentoo-dev 2004-02-08 21:00:18 UTC
Opened bug #40933 for documentation on this change. Already attached a developer oriented document so hopefully i can get some more testing from people. I'll be posting some revised patches soon for a few things, including small changes to kernel-mod.eclass and a small change to the new pcmcia.eclass and drivers that use it to make things even easier.
Comment 28 Peter Johanson (RETIRED) gentoo-dev 2004-02-08 22:39:33 UTC
Created attachment 25235 [details, diff]
Change pcmcia.eclass to use kernel-mod.eclass stuff

Patch to pcmcia.eclass to enabled using koutput by using variables from the
kernel-mod.eclass

Versions of linux-wlan-ng and hostap-driver using this eclass and updated
kernel-mod.eclass to follow soon.
Comment 29 Peter Johanson (RETIRED) gentoo-dev 2004-02-08 22:44:21 UTC
Created attachment 25237 [details, diff]
Newest version of kernel-mod.eclass patch

Added some checks to see if on 2.6 kernel but not using koutput stuff. Also
added use of variable KERNEL_MOD_KOUTPUT_PATCH which can be set if someone
needs just to patch the sources for koutput compatibility. (Potentially letting
someone not have to write their own src_unpack(), just set
KERNEL_MOD_KOUTPUT_PATCH and KERNEL_MOD_SOURCES)
Comment 30 Peter Johanson (RETIRED) gentoo-dev 2004-02-08 22:46:03 UTC
Created attachment 25238 [details]
New linux-wlan-ng ebuild using latest {pcmcia,kernel-mod}.eclass changes

New version of the linux-wlan-ng ebuild using latest eclass changes. (I swear,
i'm almost done spamming this bug for the night)
Comment 31 Peter Johanson (RETIRED) gentoo-dev 2004-02-08 23:13:20 UTC
Created attachment 25239 [details, diff]
Patch to hostap-0.1.3.ebuild to use latest eclasses

Last one for the night. Just keeping these two up-to-date so people have
something to test with the latest work (*nudge* *nudge*)
Comment 32 Chris Bainbridge (RETIRED) gentoo-dev 2004-02-09 04:27:06 UTC
Hi, I recently submitted an ebuild for prism54 that builds fine as an external module in a sandbox. I use a build directory with contents symlinked to the current kernel sources:

src_unpack() {
        check_KV
        unpack ${A}
        # Make our own build directory symlinked to the kernel sources
        mkdir ${WORKDIR}/build
        cd ${WORKDIR}/build
        SRC=/lib/modules/${KV}/build
        for f in ${SRC}/.config ${SRC}/.version ${SRC}/*; do ln -s $f .; done
}


src_compile() {
        cd ${S}
        make clean
        # hmm portage exports ARCH as x86?
        emake KDIR=${WORKDIR}/build ARCH=i386 V=1 modules || die
}

I expect this will work for other external modules as well. 
Comment 33 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 06:28:38 UTC
Interesting, i'd noticed this behavior in the ebuild, but hadn't had time to investigate it fully. This might end up being easier in the long wrong (as much as i hate to admit it after all this work i've done)

If this turns out to be the easy way out, do we want to do this, or use this new method which gives us lots of other goodies but more complexity changing the tree and educating users?
Comment 34 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 06:51:04 UTC
Okay, after taking a shower and thinking on it, this is *great*

Why? We can offer *both* so that users have maximum flexibility. Consider the koutput method a feature! kernel-mod.eclass can be relatively trivially modified to use this newly discovered method (if it turns out to work with other packages) if it sees ${KV_OUTPUT}=/usr/src/linux when on a 2.6 kernel. This would allow seemless use by people using the old method, and anyone looking for advanced stuff like:
/usr mounted readonly
Building by arbitrary users into their $HOME
Multiarch building with ease

CAN do that, just by using KBUILD_OUTPUT in their kernel makefile. I think i will also create some variable a user can set (mabye just use "O") so they can on the fly emerge things against a kernel config that is in an arbitrary directory they happened to build their kernel too.

If people don't wanna do it, they just use the old method with this symlinks magic (i'm not convinced it's magic yet, it still kinda "hacks" its way around the kbuild stuff)!

What do people think of *this*?
Comment 35 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 11:32:20 UTC
Created attachment 25279 [details, diff]
Patch to kernel-mod.eclass dubbed "Have My Cake and Eat it Too"

Okay, after thinking that we can make *everyone* happy by allowing any of the
three options, 2.4, 2.6 normally, or 2.6 with files output elsewhere, i've
hacked up a new version of the kernel-mod.eclass patch.

*simpler* ebuild for prism54 and updated ebuild for hostap-0.1.3-r3 to follow!
linux-wlan-ng is mostly working, but when using the "make symlinks" version,
some funkiness happens with .tmp_versions
Comment 36 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 11:33:50 UTC
Created attachment 25280 [details]
hostap-driver-0.1.3-r1 ebuild that works with latest no matter what.

Using latest "Cake and eat it too" kernel-mod.eclass changes.
Comment 37 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 11:35:50 UTC
Created attachment 25281 [details]
New prism54 ebuild using "cake and eat it too" kernel-mod.eclass

Simpler prism54 using new kernel-mod.eclass stuff
Comment 38 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 11:48:26 UTC
Created attachment 25282 [details, diff]
"Cake and eat it too" fixed to add a symlinks for .tmp_versions

Okay, only mildly modified to include a symlink for .tmp_versions when using
the symlinks style building.

linux-wlan-ng ebuild to follow.
Comment 39 Peter Johanson (RETIRED) gentoo-dev 2004-02-09 11:49:42 UTC
Created attachment 25283 [details]
Latest linux-wlan-ng

Oh yeah.
Comment 40 Peter Johanson (RETIRED) gentoo-dev 2004-02-10 14:55:56 UTC
Created attachment 25363 [details]
Updated 3.2.7-r1 ebuild to fix patch for monitor mode!

Just a small fix for the monitor mode patch
Comment 41 Peter Johanson (RETIRED) gentoo-dev 2004-02-11 21:33:14 UTC
Created attachment 25453 [details, diff]
"Cake and eat it too 2" - 2nd version of the "Cake and Eat it Too" style kernel-mod.eclass patch

Okay, Here's an updated patch to kernel-mod.eclass. Below is s summary of *new*
changes (beyond what was added/altered in last patch)

1) KV_BUILD now puts the symlinks fake silliness in ${T}/kernel-build instead
of ${S}/kernel-build. Thankfully the abstraction of that layer means all
ebuilds from before should still work like a charm. This was done for packages
like nvidia-kernel which must do some unpacking by hand (they have an annoying
bash script) since ${S} won't be around. ${T} is really a better spot for this
temporary fake kernel build tree anyway.

2) Moved some of the stuff out of kernel-mod_src_unpack into seperate
functions. Ebuilds like nvidia-kernel which need to do more by hand now can use
these and not duplicate the work in the ebuilds themselves.

3) Fixed kernel-mod_getmakefilevar() so it removes anything returned with a
":="  in it. This is needed so we get "" when KBUILD_OUTPUT isn't actually
defined in the 2.6 makefiles.

4) Can now have KERNEL_MOD_SOURCES="none" if you don't want *anything* unpacked
by kernel-mod_src_unpack


In other news:
I searched for which ebuilds where using the old kernel-mod so I can see what
might break with these changes. So far looks like the few other old functions
that were lying around should be uneffected. Found a few ebuilds like
nvidia-kernel which were doing things like looking for /proc/mtrr (tsk tsk!)
but otherwise things should be mostly hunky dory on that front.

nvidia-kernel ebuild using latest stuff to follow

For those looking for an easy way to test things, here a quick and dirty
rundown:
Apply latest kernel-mod.eclass patch
Apply pcmcia.eclass patch
Download any of recent driver ebuilds or patches, or try on
http://dev.gentoo.org/~latexer/files/ebuilds for one or two not here, and
digest and try building!

Testing on all combinations, 2.4, 2.6 without any kernel output mucking, and
2.6 with kernel output location changed is all appreciated!

To change the kernel output by hand, just add
"KBUILD_OUTPUT=/var/tmp/kernel-output/<kernel version" in your toplevel
Makefile, then make sure you create that directory.

I'm more than willing to help via email or irc in #gentoo-laptop on freenode if
people want help or info on testing this!
Comment 42 Peter Johanson (RETIRED) gentoo-dev 2004-02-11 21:36:25 UTC
Created attachment 25454 [details]
nvidia-kernel ebuild using latest foo!

Yes, it does work for drivers other than wireless ones! (:
Comment 43 Peter Johanson (RETIRED) gentoo-dev 2004-02-12 14:17:35 UTC
Created attachment 25497 [details]
ati-drivers ebuild using new foo

Chalk up ati-drivers as another one working. Next is the beast that is
xfree-drm... this might get ugly.
Comment 44 Donnie Berkholz (RETIRED) gentoo-dev 2004-02-12 17:03:38 UTC
Pete,
Keep in mind you'll have to essentially do it three times for xfree-drm: once with any video card but mach64, once with mach64, and once with USE=gatos and video cards radeon or r128. Watch the patches to see where the Makefile patch is re-used, if it is.
Comment 45 Chris Bainbridge (RETIRED) gentoo-dev 2004-02-23 06:05:05 UTC
I applied the kernel-mod.eclass patch and tried the new prism54 ebuild. Firstly the ebuild fails because the firmware doesn't exist on prism54.org anymore :-( After adding it to distfiles manually I get:

>>> Unpacking prism54-cvs20040208.tar.bz2 to /var/tmp/portage/prism54-20040208-r1/work
 * Creating temporary build directory:
 *      /var/tmp/portage/prism54-20040208-r1/temp/kernel-build
 * using files in:
 *      /usr/src/linux
>>> Source unpacked.
make -C ksrc/ modules
make[1]: Entering directory `/var/tmp/portage/prism54-20040208-r1/work/prism54-cvs20040208/ksrc'
make -C /var/tmp/portage/prism54-20040208-r1/temp/kernel-build SUBDIRS=/var/tmp/portage/prism54-20040208-r1/work/prism54-cvs20040208/ksrc V=1 modules
make[2]: Entering directory `/var/tmp/portage/prism54-20040208-r1/temp/kernel-build'
make -f scripts/Makefile.build obj=scripts
  gcc -Wp,-MD,scripts/.empty.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude  -D__KERNEL__ -Iinclude  -Wall -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=pentium3 -Iinclude/asm-i386/mach-default -O2 -fomit-frame-pointer -Wdeclaration-after-statement    -DKBUILD_BASENAME=empty -DKBUILD_MODNAME=empty -c -o scripts/empty.o scripts/empty.c
cc1: Permission denied: opening dependency file scripts/.empty.o.d
Assembler messages:
FATAL: can't create scripts/empty.o: Permission denied
make[3]: *** [scripts/empty.o] Error 1
make[2]: *** [scripts] Error 2
make[2]: Leaving directory `/var/tmp/portage/prism54-20040208-r1/temp/kernel-build'
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/var/tmp/portage/prism54-20040208-r1/work/prism54-cvs20040208/ksrc'
make: *** [modules-all] Error 2

I have userpriv and usersandbox set. I also see a message "By not using the kernel's ability to output to an alternative directory, some external module builds may fail."
Comment 46 Peter Johanson (RETIRED) gentoo-dev 2004-02-24 18:33:04 UTC
Hrm. odd. Check in /usr/src/linux/scripts, and see if you have all the needed scripts. If those aren't built, this might be the issue. Is this with a compiled kernel tree?
Comment 47 Chris Bainbridge (RETIRED) gentoo-dev 2004-02-25 05:45:55 UTC
I fixed that problem with 'make all' in the kernel src, which seemed to want to rebuild postmod (no idea why, this is from a freshly built kernel). Now I get:

make[1]: Entering directory `/var/tmp/portage/prism54-20040208-r1/work/prism54-cvs20040208/ksrc'
make -C /var/tmp/portage/prism54-20040208-r1/temp/kernel-build SUBDIRS=/var/tmp/portage/prism54-20040208-r1/work/prism54-cvs20040208/ksrc V=1 modules
make[2]: Entering directory `/var/tmp/portage/prism54-20040208-r1/temp/kernel-build'
make -f scripts/Makefile.build obj=scripts
*** Warning: Overriding SUBDIRS on the command line can cause
***          inconsistencies
mkdir -p .tmp_versions
make -f scripts/Makefile.build obj=arch/i386/kernel arch/i386/kernel/asm-offsets.s
make[3]: `arch/i386/kernel/asm-offsets.s' is up to date.
  CHK     include/asm-i386/asm_offsets.h
/bin/sh: line 1: include/asm-i386/asm_offsets.h.tmp: Permission denied
  UPD     include/asm-i386/asm_offsets.h
mv: cannot stat `include/asm-i386/asm_offsets.h.tmp': No such file or directory
make[2]: *** [include/asm-i386/asm_offsets.h] Error 1

This is with development-sources-2.6.3. From the kernel Makefile it looks as though it does:

       echo '  CHK     $@';                    \
        mkdir -p $(dir $@);                     \
        $(filechk_$(1)) < $< > $@.tmp;          \
        if [ -r $@ ] && cmp -s $@ $@.tmp; then  \

which outputs stuff to a .tmp file. It may be necessary to create a more complete directory structure in the kernel build directory and only symlink the actual files (allowing the CHK to run in the build dir but using the original kernel source files)
Comment 48 Tim Yamin (RETIRED) gentoo-dev 2004-02-29 03:53:18 UTC
Pete: If you could add omnibook-20040120 to the list as well, that'd be great. Thanks in advance.
Comment 49 Peter Johanson (RETIRED) gentoo-dev 2004-02-29 06:51:44 UTC
Created attachment 26588 [details]
omnibook ebuild

Your wish is my command. (:


In other news:
After talking to johnm about a few things, it seems things will be behaving a
little differently.
1) Using the "symlink" method ultimately is no better than doing "addwrite
${ROOT}/usr/src/linux". However, this is a security no-no. I *really* don't
like doing this whatsoever. BUT, we need to have a relatively smooth upgrade
path for users (sigh, the users). We can't just one day have a user suddenly
unable to emerge any of the kernel module ebuilds they know and love. As a
temporary solution til we can get people using koutput, using kernel-mod.eclass
to uniformly do this addwrite, warn users about it, and most importantly FORCE
THEM to accept it initially is a Good Thing(tm). This can be done via a
"ALLOW_LINUX_WRITABLE" or somesuch variable found in /etc/env.d/20kernel (or
somewhere in env.d). To facilitate this, and a few other things, we'll come to
point 2.

2) kernel-config tool. *-config tools seem to be the latest rage. Why not have
on for some kernel configuration stuff. One use would be the above, e.g
# kernel-config allow-writable yes
Or somesuch. 
Other things would include configuring the prefix for koutput stuff (i'll be
submitting a new kernel-2.eclass patch which shows how this would be used.
All the info on what i've been mulling around with john about is found here:
http://dev.gentoo.org/~latexer/kernel-config-proposal

3) I'm really excited for this. No change here though.
Comment 50 Tim Yamin (RETIRED) gentoo-dev 2004-02-29 10:21:55 UTC
Looks good, great!

I also agree that the kernel-config would help everyone, as copying your .config and also changing symlinks when doing testing gets very tiring. Regarding the security issue.

However, I think the best transparent solution would be to have a "pass-through" sandbox, which logs all write events in big bold red and writes straight to /usr/src/linux if it's needed but doesn't prohibit the write action: i.e. the user knows what the ebuild touched while at the same time the user's ebuild that he loved still works.
Comment 51 Donnie Berkholz (RETIRED) gentoo-dev 2004-02-29 11:44:04 UTC
Pete, join the latest craze and make config-* tools. That way people can actually find them with tab completion.
Comment 52 Peter Johanson (RETIRED) gentoo-dev 2004-02-29 17:18:38 UTC
Created attachment 26635 [details, diff]
New patch implementing things last discussed

Okay, here's a new patch for my latest work on the kernel-mod.eclass stuff.
This behaves pretty differently in a few areas, so old ebuilds posted here
probably won't work. I've got the changes for those mostly ready. From now on,
i'll be posting the latest ebuilds, patches, etc at:
http://dev.gentoo.org/~latexer/files/koutput/
so i don't keep spamming this bug with tons of ebuilds. Things that are
different in this version.

1) No longer does the symlink stuff. For simplicity's sake, and since it's a
mess no matter how we do it, this version adds kernel-mod_make_linux_writable()
which is used to intelligently make /usr/src/linux-${KV} writable. Forces the
variable LINUX_PORTAGE_WRITABLE=yes in order to do it. For now i still like the
user being made aware of this *once* before we do it automatically for them.

2) added is_koutput() command to more easily tell if we're on the new setup.
This is prefered over forcing ebuilds to check what KV_OUTPUT equals, etc.

3) Added "kernel-mod_getconfigvar() FOO" which prints the config value for
CONFIG_FOO from the current kernel config when called. 

4) Syntax fixes from spyderous. he's the man

5) Probably more i'm forgetting.

I'll be posting a new version of my kernel-2.eclass changes here, then putting
all the new ebuilds up at the above URL. I've got acx100 working now too. (:
Comment 53 Peter Johanson (RETIRED) gentoo-dev 2004-02-29 20:25:42 UTC
Created attachment 26643 [details, diff]
New patch to kernel-2.eclass

Okay, here's a new patch to kernel-2.eclass. This now checks for the
KBUILD_OUTPUT_PREFIX variable to decide whether or not to set KBUILD_OUTPUT in
the newly installed kernel's toplevel Makefile. This variable would be one of
the one's controlled by a new kernel-config/config-kernel tool.
Comment 54 Andreas Mohr 2004-03-04 15:04:38 UTC
I just committed your acx100 2.6.x out-of-kernel build patch (just when I intended to migrate to 2.6 and do this work myself :).
Thanks a LOT for this very useful work!
Comment 55 Chris Bainbridge (RETIRED) gentoo-dev 2004-03-05 04:46:47 UTC
I tested the latest patches with prism54; it seems to work fine! I get a warning about the name of my source dir (I have linux-beta symlinked to linux-2.6.3) which should probably not happen:

* The kernel Makefile says that this is a 2.6.3 kernel
 * but the source is in a directory for a beta kernel.
 *
 * This goes against the recommended Gentoo naming convention.
 * Please rename your source directory to 'linux-2.6.3'

For anyone else interested this was my list of things that I had to do to test:

patch /usr/portage/eclass/kernel-mod.eclass kernel-mod-KV_OUTPUT-v6.diff
patch /usr/portage/eclass/kernel-2.eclass kernel-2-koutput-v2.diff
cd /usr/src/linux
make mrproper
edit Makefile, add KBUILD_OUTPUT=/var/tmp/kernel-output/2.6.3 to top line
mv .config /var/tmp/kernel-output/2.6.3
make all modules_install
emerge the new kernel module
Comment 56 Chris Bainbridge (RETIRED) gentoo-dev 2004-03-05 04:48:16 UTC
Just a thought, would it be possible to use this method to also build vmware modules?
Comment 57 Peter Johanson (RETIRED) gentoo-dev 2004-03-05 06:14:09 UTC
Excellent! glad it's working for other people. One thing, you should really put KBUILD_OUTPUT *below* the first four lines of the toplevel Makefile (after the EXTRAVERSION line) because new portage uses those first four lines to determine KV

A few other updates:
1) I may redo this as a new seperate eclass kmod-2.eclass, in order to aboid having to retain some of the cruft from the original kernel-mod.eclass

2) I've started work on the new config-kernel (decided on config-kernel instead of kernel-config) in python. I've got most of the framework, and some of the functions implemented, as soon as it's in quasi-usable state i'll be adding it to the "gentoo" cvs module.

As for vmware, i'll have to check it out, that's one i hadn't looked at yet. Thanks for testing!
Comment 58 Chris Bainbridge (RETIRED) gentoo-dev 2004-03-05 14:10:20 UTC
The prism54 firmware is back at http://prism54.org/~mcgrof/firmware/. The file 1.0.4.3.arm is the same as the old isl3890 (in fact, it needs renaming to this before it will work).
Comment 59 Radek Podgorny 2004-03-08 14:27:03 UTC
This is a great piece of work you're doing here and I don't want to add any new tasks but will it work with lirc?
Comment 60 Peter Johanson (RETIRED) gentoo-dev 2004-03-15 23:14:17 UTC
Okay, so here's the latest update:

1) DOCS - the developer oriented doc is much better, still at bug #40933, I've just submitted a user oriented doc at bug #44807 which i will hopefully be fleshing out more soon

2) UTILITY - I added config-kernel into CVS today, so that people can take a look at it. I'm not a python coder, so anybody who wants to submit patches to clean/improve that, please do. All of the functionality for that is working though. Waiting on johnm or i to add the corresponding support to kernel-2.eclass (not mission critical)

3) EBUILD/ECLASSES - I've decided to take over the kmod.eclass name, instead of patching the kernel-mod.eclass. First, because the name is nicer, and second, so i don't accidentally break anything currently using kernel-mod.eclass. ALL the latest ebuilds to reflect this, along with eclasses, etc is up at: http://dev.gentoo.org/~latexer/files/koutput/
so get your test on.

Where do we go now?
I need eyes on the user oriented doc, we really can't commit any of this til that is ready. config-kernel needs some pounding on beyond what i've done. I'll hopefully have an ebuild and current tarball up at the above location soon for people. I need to convert more ebuilds, like lufs, xfree-drm (battousai and i mostly have this done), and a few others.

WORKING:
hostap-driver, linux-wlan-ng, ndiswrapper, prism54, at76c503a, nvidia-kernel, ati-drivers, qc-usb (with CVS snapshot), omnibook, ipw2100 (new Intell 2100 drivers), acx100

NEED FIXING:
lufs, xfree-drm, mtx-drivers-pro, kyro-kernel, lirc (THIS is going to hurt), slmodem, madwifi (i've got this working with a LOT of makefile changes, i've submitted them upstream, hopefully they will merge those), esm (some random thing max wanted me to do)

What this boils down to. I WANT TO GET THIS MERGED!!!!! i'm dying to have this come to fruition. My goal is to clean up a few things in config-kernel, expand that user doc a bit more, then ROLL WITH THIS. hopefully wednesday and thursday will be merge days.

Anyone who wants to speak out against this, do so now! If people want, i will send a final email to gentoo-dev requesting eyes/testing/info.
Comment 61 Peter Johanson (RETIRED) gentoo-dev 2004-03-17 23:36:26 UTC
Created attachment 27538 [details, diff]
New version of kernel-2 eclass patch to make --set-extraversion useful

Okay, to make life easier, I'm going to not be posting any more of the ebuilds
here, instead, they will be available in an overlay tarball at:
http://dev.gentoo.org/~latexer/files/koutput

This should make testing easier, as it includes ebuilds in the right spot,
eclasses, etc, along with a README.

config-kernel in CVS (i forgot before, its in gentoo/src/kernel/config-kernel)
now has the added --set-extraversion as discusson gentoo-dev. An ebuild for
installing a snapshot of config-kernel is available in the kmod-overlay tarball
as well. this should let people more easily test things.

Attached is the latest patch to kernel-2.eclass. This implements setting
KBUILD_OUTPUT as dicussed in that same thread. Now uses
${KBUILD_OUTPUT_PREFIX}, but then sets it such that the makefile itself
determines the rest. This is the bit that makes the --set-extraversion so
handy. (: Still waiting on johnm for the auto symlink and auto config code in
the eclass, but that's more icing than core functionality.

Hopefully the steps requied to test this are clear/easy enough that this can
get wider attention. Thanks all who've given me feedback, specially Georgi for
poking me today to make this even more flexible. Nice to get these things
ironned out *now* and not later when we will have to worry about breaking the
previous setup. Any other wrinkles people want to point out? (:
Comment 62 Chris Bainbridge (RETIRED) gentoo-dev 2004-04-02 13:54:05 UTC
A word of wanring - the nvidia-kernel ebuilds now check for the built include files in a hardcoded path so they no longer work if you have your kernel compiled with output in a different directory. It looks like the downloaded source from nvidia uses the hardcoded paths as well.
Comment 63 Peter Johanson (RETIRED) gentoo-dev 2004-04-02 14:06:12 UTC
Yes, the version up in the overlay works (tested by johnm and i) It uses the nice kbuild makefile nvidia ships, plus some fixes for paths in various locations. See that ebuild or bug #46592 for the particulars of that specific ebuild. Feel free to test. info as always at: http://dev.gentoo.org/~latexer/files/koutput/

The nvidia-kernel ebulid in that overlay uses the proposed changes. thanks.

Comment 64 Chris Bainbridge (RETIRED) gentoo-dev 2004-04-03 06:49:59 UTC
I tested the new nvidia-kernel. Works fine :-)

The new prism54 failed because the cvs tarball isnt mirrored in gentoo mirrors. I went to prism54.org and found they have released a 1.1 stable version.. just minor changes to the ebuild - rename, add SRC_URI="http://prism54.org/pub/linux/stable/tars/2004-03/prism54-1.1.tar.bz2..
 and fix MY_P=prism54-1.1. After that it also emerged fine.
Comment 65 Georgi Georgiev 2004-04-04 18:51:36 UTC
I get an error every time I try to "config-kernel --set-extraversion -s1":

# config-kernel --set-extraversion -s1
 * Backing up original makefile to /usr/src/linux/Makefile.orig
 * Setting EXTRAVERSION to '-s1'
Traceback (most recent call last):
  File "/usr/bin/config-kernel", line 166, in ?
    main()
  File "/usr/bin/config-kernel", line 78, in main
    ck_actions.setextraversion(variables['extraversion'])
  File "/usr/lib/python2.3/site-packages/config_kernel/ck_actions.py", line 156, in setextraversion
    os.mkdir(os.path.join(kbuildprefix,newKV))
OSError: [Errno 2] No such file or directory: ' /var/tmp/linux-build/2.6.5-s1'

Problem is in ck_actions.py
    # FIXME: Nasty bit of bash hiding in a python program. shame on me.
    tochild , fromchild , childerror = os.popen3("grep '^KBUILD_OUTPUT.*=.*' /usr/src/linux/Makefile | grep VERSION | cut -d'=' -f2  | sed 's:$(VERSION.*::'")

because I somehow ended up with:
# head -5 /usr/src/linux/Makefile
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 5
EXTRAVERSION = -s1
KBUILD_OUTPUT = /var/tmp/linux-build/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)

Note the space before /var/tmp/linux-build (which may not be visible with bugzilla).
Comment 66 Georgi Georgiev 2004-04-04 19:00:13 UTC
What about config-kernel not giving s fatal error if KBUILD_OUTPUT already exists? It seems that it tries to create the directory, but python gives an error if it is already there.
Comment 67 Georgi Georgiev 2004-04-04 20:31:36 UTC
config-kernel should probably provide some query functions that would return the config-kernel variables that are in effect in a kernel source tree (KBUILD_OUTPUT, the prefix, etc).

I think it is better to parse a Makefile using make, so I'll just drop the following very ugly but failure proof suggestion:

echo -e 'include /usr/src/linux/Makefile\ne:\n\t@echo $(KBUILD_OUTPUT)\n' | make -f - srctree=/usr/src/linux  e 2>/dev/null
Comment 68 Georgi Georgiev 2004-04-04 20:48:46 UTC
I keep posting, but questions come one by one.

I am trying to work on the vmware modules.

The vmware-configure.pl script has a whole lot of error checks about whether the kernel tree the user is trying to use is the same as the kernel they are running. This might be unnecessary on a gentoo system, because users are required to have /usr/src/linux pointing to the source tree of their running kernel. I am thinking of maybe automating the script so that it uses gcc and /usr/src/linux by default (instead of asking the user). It might be a great idea to also build the kernel modules for vmware during the emerge phase, so it will be more consistent with the portage system. mplayer for example installs the mga_vid module if necessary.

It may also be a good idea to take building the modules to another ebuild -- vmware-workstation-modules for example. vmware-workstation would RDEPEND on "${PN}-modules-${PV}" then. This idea, however, is the same as bug #28846, and that one didn't get much appreciation and it might be just unnecessary.
Comment 69 Chris Bainbridge (RETIRED) gentoo-dev 2004-04-05 06:15:21 UTC
I upgraded to 2.6.5 and let config-kernel change the Makefile. Now emerging nvidia-kernel-1.0.5336-r2 gives me:

>>> emerge (1 of 1) media-video/nvidia-kernel-1.0.5336-r2 to /
>>> md5 src_uri ;-) NVIDIA-Linux-x86-1.0-5336-pkg1.run
>>> md5 src_uri ;-) nvidia-kernel-koutput.diff.gz
>>> Unpacking source...
 * /usr/src/linux is a symbolic link
 * Determining the real directory of the Linux kernel source code
 * Building for Linux 2.6.5 found in /usr/src/linux
 * which outputs to /var/tmp/kernel-output/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)
grep: ///var/tmp/kernel-output/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)/.config: No such file or directory
 * This version needs MTRR support for most chipsets!
 * Please enable MTRR support in your kernel config, found at:
 *
 *   Processor type and features -> [*] MTRR (Memory Type Range Register) support
 *
 * and recompile your kernel ...

It looks as though using variables in the path doesn't work.
Comment 70 Chris Bainbridge (RETIRED) gentoo-dev 2004-04-05 06:17:05 UTC
btw prism54 is now in the main kernel. Maybe the external module should be split into firmware and driver ebuilds so people who use the kernel sources can get the firmware using portage but needn't rebuild the whole driver.
Comment 71 Henrik Brix Andersen 2004-04-16 18:08:42 UTC
Seems sys-kernel/config-kernel-0.3.3 doesn't create the /var/tmp/kernel-output/ directory if it doesn't exists:

config-kernel --make-koutput=current
 * Changing kernel found in /usr/src/linux
 * to output to /var/tmp/kernel-output/2.6.6-rc1
 * You will lose all of your compiled files in /usr/src/linux
 * Hit Control C to cancel this now.
5 4 3 2 1
 * Running 'make mrproper to clean your kernel tree (This make take a while)
Traceback (most recent call last):
  File "/usr/bin/config-kernel", line 166, in ?
    main()
  File "/usr/bin/config-kernel", line 81, in main
    ck_actions.makekoutput(variables['makeoutput'])
  File "/usr/lib/python2.3/site-packages/config_kernel/ck_actions.py", line 238, in makekoutput
    os.mkdir(outputpath)
OSError: [Errno 2] No such file or directory: '/var/tmp/kernel-output/2.6.6-rc1'
Comment 72 Eric Brown 2004-04-22 13:56:35 UTC
I finally understand why config-kernel is around :)

but why not just now allow sandbox to apply on ebuilds that build modules? Seems like a much simpler system than a whole new eclass.
Comment 73 Henrik Brix Andersen 2004-04-25 11:58:31 UTC
Created attachment 30030 [details]
Ebuild for the ORiNOCO wireless driver

I've created an ebuild for the orinoco drivers found on
http://www.nongnu.org/orinoco/

Feedback is most welcome.
Comment 74 Henrik Brix Andersen 2004-05-07 04:20:06 UTC
media-libs/svgalib also need to be updated to work with the new koutput stuff.

kernel/svgalib_helper/Makefile determines the location of the autoconf.h file using INCLUDEDIR = /lib/modules/$(shell uname -r)/build/include - which obviously wont work with koutput. A simple patch is required.
Comment 75 Chris Bainbridge (RETIRED) gentoo-dev 2004-05-07 09:32:57 UTC
Theres an interesting thread on LKML about what exactly is required to build external modules, and some work on refining the build system.. http://www.uwsg.iu.edu/hypermail/linux/kernel/0405.0/1140.html 
Comment 76 Georgi Georgiev 2004-05-13 01:15:36 UTC
As a reply to my comment #68, I have a vmware-workstation-4.5.1.7568.ebuild in my PORTDIR_OVERLAY that works fine with KOUTPUT. It's been here for a while, but I didn't post it because it is not very pretty. The changes I had to do are:

Patch the vmware-config.pl file, to remove most questions about the locations of files. Gentoo has a nomenclature, so /usr/src/linux is assumed as most ebuilds already do.

Add a line to /etc/vmware/locations:
answer HEADER_DIR /usr/src/linux/include

Patch the vmmon.tar tarball, to compile fine against a kbuild-output-enabled kernel.

The original Makefile.kernel in the tarball is doing some grepping through kernel headers and doing some -Dfoo depending on the result of the grep. I removed those checks, because it was grepping the asm/fixmap.h header, which goes in KBUILD_OUTPUT if the feature is enabled. At the moment, there is no easy way to query where KBUILD_OUTPUT is. If config-kernel is made to support it, the checks can be put back in.
As it is, the patch should work fine with 2.6.x kernels.

I'll proceed to attach the files now.
Comment 77 Georgi Georgiev 2004-05-13 01:17:01 UTC
Created attachment 31314 [details, diff]
vmmon-4.5.1.7568-koutput.patch

Here goes the patch for the vmmon module to make sure it compiles on
KBUILD_OUTPUT enabled sources.
Comment 78 Georgi Georgiev 2004-05-13 01:17:40 UTC
Created attachment 31315 [details, diff]
vmware-config-4.5.1.7568-koutput.patch

And this is the patch for vmware-config.pl
Comment 79 Georgi Georgiev 2004-05-13 01:20:32 UTC
Created attachment 31317 [details, diff]
vmware-workstation-4.5.1.7568.patch

A patch for the vmware-workstation ebuild.

All of these are also available from
rsync://rsync.gg3.net/gentoo-portage-chutz/app-emulation/vmware-workstation/
Comment 80 Kalin KOZHUHAROV 2004-05-13 10:06:55 UTC
Ok, firs a little "history" of what I am trying to achieve.

I have a few boxes, all running different flavours of Gentoo (server, WS, note, etc.) on quite different hardware (PIII,Crusoe,AthlonXP) and of course with different kernel configs.

I usually am running the latest stable 2.6 kernel and update everything immediately after release (exept the server).

When the KBUILD_OUTPUT came along I was really happy imagining how I will be able to live just with one source tree, all my kernels build on one machine (with distcc)...

It still is not a reality, but I am trying to do it. The config-kernel is helping a bit, but I don't like the idea of changing my make file (in /usr/src/linux-2.6.6). So I came up with the solution of putting
   EXTRAVERSION=my_string
in /etc/env.d/05kernel and
   KBUILD_OUTPUT =/var/tmp/kernels/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)
in /usr/src/linux-2.6.6/Makefile. Now trying to get without the Makefile change.

So I had to patch a few things to work, starting with VMware-workstation.
The patched versions are available from:
   rsync://rsync.tar.bz/gentoo-portage-kalin/app-emulation/vmware-workstation/
and I will post them here as well.

They are mainly based on the patches proposed by Georgi and as such they should replace them fully.
Comment 81 Kalin KOZHUHAROV 2004-05-13 10:08:03 UTC
Created attachment 31345 [details, diff]
vmware-workstation-4.5.1.7568.ebuild.diff   (kalin)

diff against the one in portage...
Comment 82 Kalin KOZHUHAROV 2004-05-13 10:09:03 UTC
Created attachment 31346 [details, diff]
vmmon-4.5.1.7568-koutput.patch  (kalin)
Comment 83 Kalin KOZHUHAROV 2004-05-13 10:11:09 UTC
Created attachment 31347 [details, diff]
vmnet-4.5.1.7568-koutput.patch (kalin)

vmware-config-4.5.1.7568-koutput.patch is the same as in Georgi's version
Comment 84 Scott Cytacki 2004-05-16 07:58:40 UTC
What is the status on merging this?  I see kmod.eclass and config-kernel are in portage but not pcmcia.eclass, and all the new ebuilds.

I'm curious because I started using kmod in a new version of the ipw2100 driver.  But then I found it wouldn't work unless the new pcmcia.eclass and the new hostap-driver were added portage.
Comment 85 Peter Johanson (RETIRED) gentoo-dev 2004-05-16 08:55:02 UTC
Hey everyone,


sorry for the delay, but the two weekends i had set aside as times i could merge this stuff turned out to be weekends that Optimum Online decided to fsck around with my cable connection. After that, the whole "finals, i need to actually graduate from college" stuff kicked in. I'm back to normal starting monday, at which time i will be dusting of a few things, then getting this ready for inclusion. Stupid meatspace always screwing with important things.

As for ipw2100, it doesn't use the pcmcia bus at all, so you should be able to write an ebuild using kmod.eclass for this easily. There's a slightly outdated version in the kmod overlay, which mainly should just need updating to work with this.
Comment 86 Scott Cytacki 2004-05-16 18:04:10 UTC
Actually ipw2100 requires hostap-driver (for wep) which uses pcmcia.  You can see my comments about this and the ebuild here: bug #50694
Comment 87 Gilberto Ficara 2004-05-27 07:32:44 UTC
Seems that linux-wlan-ng-0.2.1_pre20 has the same problem (SANDBOX VIOLATION). Should I submit another bug? Here, a small summary:

  CC [M]  /var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211wext.o
  CC [M]  /var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211netdev.o
/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211wext.c:172:2: warning: #warning "make a smarter sharedkey/opensystem auth decision"
/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211wext.c:438:2: warning: #warning "get rid of p2mib here"
/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211wext.c:748:2: warning: #warning "Get rid of p2mib here!"
/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211netdev.c: In function `register_wlandev':
/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211netdev.c:1051: warning: `dev_get' is deprecated (declared at include/linux/netdevice.h:531)
  LD [M]  /var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211/p80211.o
  Building modules, stage 2.
  MODPOST
ACCESS DENIED  open_wr:   /usr/src/linux-2.6.6/lib/zlib_inflate/zlib_inflate.mod.c
lib/zlib_inflate/zlib_inflate.mod.c: Permission denied
make[4]: *** [__modpost] Error 1
make[3]: *** [modules] Error 2
make[3]: Leaving directory `/usr/src/linux-2.6.6'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src/p80211'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/var/tmp/portage/linux-wlan-ng-0.2.1_pre20/work/linux-wlan-ng-0.2.1-pre20/src'
make: *** [all] Error 2

!!! ERROR: net-wireless/linux-wlan-ng-0.2.1_pre20 failed.
!!! Function src_compile, Line 92, Exitcode 2
!!! failed compiling

--------------------------- ACCESS VIOLATION SUMMARY ---------------------------
LOG FILE = "/tmp/sandbox-net-wireless_-_linux-wlan-ng-0.2.1_pre20-7972.log"

open_wr:   /usr/src/linux-2.6.6/lib/zlib_inflate/zlib_inflate.mod.c
--------------------------------------------------------------------------------


I'm _NOT_ using an emerged kernel sources (just the old way tar jxvf of vanilla sources from kernel.org), but I think that this isn't an issue.
Comment 88 Henrik Brix Andersen 2004-06-08 12:24:08 UTC
Created attachment 32935 [details, diff]
Patch for net-dialup/slmodem-2.9.7

The attached patch changes the net-dialup/slmodem-2.9.7 ebuild to work with the
new M= build environment of kernel 2.6.6.
Comment 89 Kalin KOZHUHAROV 2004-06-11 07:39:55 UTC
Sometimes it is funny when you find your own mistakes... sometimes it is not.

So, as I was playing with different ways of setting EXTRAVERSION for the kernel, I came up with the idea to comment out the 'EXTRAVERSION =' line in /usr/src/linux/Makefile ... and everything worked better than ever.

So at the moment:

$ head -n 6 /usr/src/linux/Makefile 
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 6
#EXTRAVERSION = 
KBUILD_OUTPUT =/var/tmp/kernels/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)
NAME=Zonked Quokka


As far as vmware patches (attachments id=31345,id=31346,id=31347) go, they may not be needed, i.e. Georgi's version should probably do. Will test later as vmware-workstation-4.5.1.7568-r1 seems too badly broken at the moment.
Comment 90 Georgi Georgiev 2004-06-12 00:54:06 UTC
Created attachment 33123 [details, diff]
vmware-workstation-4.5.2.8848.ebuild.patch

vmware-workstation-4.5.2.8848 is out. This is a patchs against the 4.5.1-r1
ebuild.
Comment 91 Georgi Georgiev 2004-06-12 00:55:27 UTC
Created attachment 33124 [details, diff]
vmware-config-4.5.2.8848-koutput.patch

This file goes in /usr/portage/app-emulation/vmware-workstation/files
Needed by the previously attached ebuild.
Comment 92 Georgi Georgiev 2004-06-12 00:57:37 UTC
Created attachment 33125 [details, diff]
vmnet-4.5.2.8848-koutput.patch

This file goes in /usr/portage/app-emulation/vmware-workstation/files
Needed by the ebuild above.

Again, checks for some compatibility structs are removed, and a recent kernel
is assumed. Since gentoo does not provide 2.5.x kernels, this is a safe
assumption.
Comment 93 Georgi Georgiev 2004-06-15 21:20:57 UTC
Created attachment 33356 [details, diff]
vmmon-4.5.2.8848-koutput.patch

I forgot to submit the patch for the vmmon module. Here it is.

It goes in $PORTDIR/app-emulation/vmware-workstation/files/
Comment 94 Kalin KOZHUHAROV 2004-08-01 20:02:30 UTC
Created attachment 36615 [details, diff]
vmware-config-4.5.2.8848-koutput.patch (using POD for neater patch)

This is a rework for Gerogi's patch (id=33124) by using perl POD (=Plain Old
Documentation) instead of just removing a few hundred lines.

What does the Perl interpereter see is exactly the same in both cases, however
with my patch the old code remains (commented out).
Comment 95 Daniel Black (RETIRED) gentoo-dev 2004-08-03 16:45:19 UTC
Comment on attachment 32935 [details, diff]
Patch for net-dialup/slmodem-2.9.7

slmodem-2.9.9-r1 commited - if someone could check tis for me I'd appreciate
it.
Comment 96 Daniel Black (RETIRED) gentoo-dev 2004-08-03 16:50:09 UTC
bug #59121 is also after a kernel module ebuild if someone gets time - l7-filter (Layer 7 iptables) http://l7-filter.sourceforge.net.
Comment 97 Eddi 2004-08-13 06:46:04 UTC
With latest portage:

slmodem ok
vmware ko
hostap-driver ko
ipw2100 ko

with ebuild and patches from this bugs:
hostap-driver ko
other untested

Eddi
Comment 98 Christoph Brill (egore) (RESIGNED) 2004-08-31 06:35:26 UTC
I tried to use the ebuild for acx100, but I cannot find the firmware.eclass (which isn't available on your page either). Any help where to get this?
Comment 99 John Mylchreest (RETIRED) gentoo-dev 2004-11-10 14:48:19 UTC
is this still valid?

there is a current re-write of how we handle fixing up kernel modules.
Hopefully we can start migrating kernel modules in the coming few weeks.
I will close this for now as REMIND since this is work in progress.
Comment 100 Donnie Berkholz (RETIRED) gentoo-dev 2004-12-01 23:57:23 UTC
Anyone hitting bugs with things looking for /usr/src/linux/arch/x86/Makefile needs to use set_arch_to_kernel() and set_arch_to_portage() from eutils.eclass.