linux_info can't find my kernel version - SUBLEVEL and EXTRAVERSION not present in ${KBUILD_OUTPUT}/Makefile, but linux_info search for them there Reproducible: Always Steps to Reproduce: 1. compile kernel using KBUILD_OUTPUT 2. use it 3. emerge nvidia-kernel Actual Results: >>> emerge (1 of 1) media-video/nvidia-kernel-1.0.6629-r1 to / >>> md5 src_uri ;-) NVIDIA-Linux-x86-1.0-6629-pkg1.run * Determining the location of the kernel source code * Found kernel source directory: * /usr/src/linux * Could not detect kernel version. * Please ensure that /usr/src/linux points to a complete set of Linux sources. !!! ERROR: media-video/nvidia-kernel-1.0.6629-r1 failed. !!! Function linux-info_pkg_setup, Line 534, Exitcode 1 !!! Unable to calculate Linux Kernel version !!! If you need support, post the topmost build error, NOT this status message. Expected Results: Compile that module of course :-) I assume correct upgrade is get KERNELSRC and read SUBLEVEL and EXTRAVERSION from ${KERNELSRC}/Makefile. Portage 2.0.51.19 (default-linux/x86/2004.0, gcc-3.3.2, glibc-2.3.2-r9, 2.6.10-gentoo-r6-32 i686) ================================================================= System uname: 2.6.10-gentoo-r6-32 i686 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.4.16 Python: dev-lang/python-2.3.3 [2.3.3 (#1, May 5 2004, 21:35:16)] distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] dev-lang/python: 2.3.3 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.8.5-r3, 1.5, 1.6.3, 1.7.9-r1, 1.4_p6, 1.9.4 sys-devel/binutils: 2.14.90.0.8-r1 sys-devel/libtool: 1.4.3-r3, 1.5.2-r7 virtual/os-headers: 2.4.21 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O2 -mcpu=athlon -march=i686 -fomit-frame-pointer -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -mcpu=athlon -march=i686 -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms" GENTOO_MIRRORS="http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/ http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/mnt/Gentoo64/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="3dnow 3dnowex X Xaw3d aalib alsa apache2 apm arts avi berkdb bitmap-fonts caps cdr crypt cups curl dga directfb doc dvd encode esd f77 fbcon flac font-server foomaticdb fortran gd gdbm ggi gif gnome gpm gtk gtk2 imagemagick imlib innodb ipv6 java jpeg lcms lesstif libcaca libg++ libwww lirc mad mailwrapper mbox mcal memlimit mikmod mmx mmx2 mng motif mozilla mpeg multislot mysql ncurses nls oggvorbis opengl oss pam pdflib perl png python qt quicktime readline samba sdl slang snmp spell sqlite sse ssl svga tcpd tetex theora tiff truetype truetype-fonts type1-fonts unicode usb v4l v4l2 vhosts videos wmf x86 xml xml2 xmms xosd xv xvid zlib" Unset: LDFLAGS :~# cat `readlink /usr/src/linux`/Makefile # Automatically generated by /unix/usr/src/linux+kernel/linux-2.6.10-gentoo-r6/scripts/mkmakefile: don't edit VERSION = 2 PATCHLEVEL = 6 KERNELSRC := /unix/usr/src/linux+kernel/linux-2.6.10-gentoo-r6 KERNELOUTPUT := /unix/mnt/sketch/hda45/-src MAKEFLAGS += --no-print-directory all: $(MAKE) -C $(KERNELSRC) O=$(KERNELOUTPUT) %:: $(MAKE) -C $(KERNELSRC) O=$(KERNELOUTPUT) $@
are you passing KBUILD_OUTPUT=/whatever on the command line, or is KBUILD_OUTPUT set in the kernel Makefile? It cant pick it up if it doesnt know about KBUILD_OUTPUT
I use "make O=/whatever menuconfig", which creates /whatever/Makefile using this rule: .PHONY: outputmakefile # outputmakefile generate a Makefile to be placed in output directory, if # using a seperate output directory. This allows convinient use # of make in output directory outputmakefile: $(Q)if test ! $(srctree) -ef $(objtree); then \ $(CONFIG_SHELL) $(srctree)/scripts/mkmakefile \ $(srctree) $(objtree) $(VERSION) $(PATCHLEVEL) \ > $(objtree)/Makefile; \ echo ' GEN $(objtree)/Makefile'; \ fi Then I call all other make (menuconfig, bzImage, modules, modules_install) from /whatever and it works. Also, /usr/src/linux is pointing to /whatever (I did it that way) and /lib/modules/`uname -r`/build points to /whatever (make modules did it that way). Finally, linux_info DID find /whatever right. Problem is, it search for all kernel version variables in /whatever/Makefile. But there are only half of them. Remainder (SUBLEVEL and EXTRAVERSION) can be found in only in Makefile in kernel source directory, which can be found at /lib/modules/`uname -r`/source and also variable KERNELSRC in /whatever/Makefile is set to this path. BTW, in my case /unix/usr/src/linux+kernel/linux-2.6.10-gentoo-r6 is kernel source and /unix/mnt/sketch/hda45/-src is /whatever. Am I finally clear enough ? Note: I'm using CONFIG_LOCALVERSION from .config to discriminate between kernel builds. Note for others with same problem: I workarounded this bug by manually adding SUBLEVEL and EXTRAVERSION to /whatever/Makefile. But that's not right solution.
Im not sure if you still ecxperience these problems, however Ive never known the makefiles to differ in such a way, Looking through the kbuild code I cant see how. Can you please try adding KBUILD_OUTPUT="whatever" to the Makefile in /usr/src/linux/ and see if it still occurs? This is a most unusual setup.
I think we have some basic misunderstanding. On which "kernel source" you suppose /usr/src/linux will point to ? Real source or output directory ? Your responses seems like you expect /usr/src/linux will point to source (/lib/modules/`uname -r`/source). In this case, adding KBUILD_OUTPUT to enviroment solve problem (I didn't tested adding it to Makefile - I want it to be read-only.) On the other hand, my propossed solution is point /usr/src/linux to output directory (/lib/modules/`uname -r`/build) and find source by parsing Makefile in that directory. I have still same problems with this case and I just updated /usr/portage/eclass. (I think my workaround was removed by make menuconfig).
The idea of /usr/src/linux is that you can compile for a kernel that you aren't running. Using `uname -r` always indicates the current booted kernel which is precisely not what we designed our system for. Does this clear things up for you?
I see you still don't understand me. Is my english so bad ? I'm talking about difference between /lib/modules/`uname -r`/source and /lib/modules/`uname -r`/build. I know /usr/src/linux can point to other kernel, lets suppose its kernel whose uname -r is $UNAMER. Then, should /usr/src/linux point to /lib/modules/$UNAMER/source or lib/modules/$UNAMER/build ?
No, I just know very little about kbuild. I'll reassign to the man in the know.
basically, to summarize... Lets assume /usr/src/linux/ contains 2.6.11.7 /lib/modules/2.6.11.7/source should point to /usr/src/linux (or, to be exact $srctree) /lib/modules/2.6.11.7/build should be $objtree, which is whatever KBUILD_OUTPUT is set to. The lines you might want to look at are (in, srctree/Makefile): 104, 133-161 Your solution would possibly work, but there is on big problem. How can you reliably detect the directory to look in, in /lib/modules/ without knowing the source-directory to look in, and also when trying to take into account all the arising problems you might have with LOCALVERSION etc?
I will not parse /lib/modules/* ... I have 'ln -sf linux+kernel/linux-`uname -r` /usr/src/linux' in startup scripts and I have (I mean I will have) all kernels in linux+kernel with proper name, that is linux-$VERSION.$PATCHLEVEL.$SUBLEVEL.$EXTRAVERSION for source trees and linux-$VERSION.$PATCHLEVEL.$SUBLEVEL.$EXTRAVERSION$CONFIG_LOCALVERSION for object trees (don't using other localversion variants). I used /lib/modules/`uname -r`/* only because I did't know terms $srctree and $objtree is assumed to be known. If I need to compile for other kernel (not so often), I change /usr/src/linux manually. Only thing which I need from you (I mean gentoo) is test, if /usr/src/linux is objtree and if so, find srctree using KERNELSRC from /usr/src/linux/Makefile. It seems to me as simple change which couldn't damage other variants, but of course I don't really understand linux_info script so I might be wrong.
Sorry, I still dont really understand the benefit here. Why can't you use KERNEL_DIR="something" on the command line. KERNEL_DIR="something" makes something the srctree, rather than the default, which is /usr/src/linux
You mean you can't understand the benefit in fact I must not remember which ebuild needs kernel when upgrading ? Not speaking about length of that KERNEL_DIR typed on command line ...
KERNEL_DIR can of course be set in env.d or some such if thats how you prefer it. What I am asking is what benefit does this give you? basically, we do not know how to find objtree, without checking srctree/Makefile first. and srctree is subject to whats in KERNEL_DIR. I need to see an example, as I dont understand you very well.
I already said it: you know how to find objtree on my computer. It's on end of /usr/src/linux symlink chain. Do you mean you don't know how to identify objtree ? What about (SUBLEVEL is missing) && (KERNELSRC is defined) in Makefile ? BTW, you can't find objtree using srctree only. You can't find it even using srctree and /lib/modules, because of localversions ... you must use KBUILD_OUTPUT. So, you are basically telling me I should set both KBUILD_OUTPUT and KERNEL_DIR in env.d.
Sorry but after reviewing this, I cannot see the problem. If you can re-itterate your point and setup, I may be able ot help further but as KBUILD_OUTPUT is the objtree KERNEL_DIR is the srctree As far as finding directories go, no of course we cannot fully find objtree due to localversion* etc, although we do try and take a good guess at it. I can elaborate on this more, although I'll wait to see the outcome first.
I already said it (literally): you know how to find objtree on my computer. It's on end of /usr/src/linux symlink chain. Problem is, you (your code) found objtree, but you're thinking you found srctree, and then you fail because it's NOT srctree. Solution you are offering is to ignore that your heuristic find objtree in my setup and solve both problems (finding objtree and finding srctree) using other ways, while I recomend to take advantage of that "luck". I don't know if I'm describing it so badly or if it's your problem, but it seems I'll must make patch myself just for the purpose of showing you what I need to be done ...
Created attachment 66083 [details, diff] Proposed patch to eclass/linux-info.eclass OK, so this is patch which solves my problem and shouldn't broke anything else, because if it's starting condition is true unpatched linux-info.eclass fail. I also hope you finally get what I was trying to tell you ... (You can add this patch to portage in either case).
Why I can't reopen bug by adding patch ?
I patched my linux-info.eclass with Honza's patch, this works fine to detect kernel version when kernel is built using make O=somewhere! Why didn't that patch make it into the eclass after such a long time? PS: If some module ebuild fails on external build (after patch to linux-info.eclass) it's often because of incorrect use of KV_DIR instead of KV_OUT_DIR in the module's ebuild!
the reason this patch never made it into the eclass was because /usr/src/linux should not point to objtree. objtree is calculated based on the srctree and environment conditionals, and as such /usr/src/linux should point to the appropriate srctree. This patch would overcome the shortfalls of this, but this is not the policy which we work to. Aside from this, its things like broken makefiles outside of portage which expect kernel headers to exist in /usr/src/linux still, which although we do not support it will give some perculiar behaviour when /usr/src/linux is objtree and srctree = subject to some variable. I'm not going to close this bug just yet because I would like to hear technical arguments as to why to allow this behaviour or not, further to my own.
(In reply to comment #19) > the reason this patch never made it into the eclass was because > /usr/src/linux should not point to objtree. > objtree is calculated based on the srctree and environment conditionals, and > as such /usr/src/linux should point to the appropriate srctree. In that case it may be useful to introduce a /usr/src/build (or similar) that points to the object tree (which in turn has all information to find the matching source tree). Defining the build path via environment is quite contra-productive (would happen very often to forget setting env or having /usr/src/linux and obj trees not matching) The important point in my eyes is to have 1 persistent pointer to obj tree being used by kernel eclasses (a symbolic link under /usr/src would be best) Eventually deprecating /usr/src/linux symlink would also prevent the issues with some external makefiles. Kernel obj and src trees should possibly be auto-detected using some alternative checks (taking the first one working): 1) Look for /usr/src/build 2) Look for /usr/src/linux 3) Look for /lib/modules/`uname -r`/{build,source} Warning user and giving him some time to interrupt process before using results of 3 can be useful. > This patch would overcome the shortfalls of this, but this is not the policy > which we work to. What is the policy Gentoo is working towards in this context? > Aside from this, its things like broken makefiles outside of > portage which expect kernel headers to exist in /usr/src/linux still, which > although we do not support it will give some perculiar behaviour when > /usr/src/linux is objtree and srctree = subject to some variable. As said above, deprecating /usr/src/linux will avoid such perculiar behavior.
Not /usr/src/build, please. I have other thinks in /usr/src that kernel ... why not /usr/src/linux-build ?
(In reply to comment #21) > Not /usr/src/build, please. I have other thinks in /usr/src that kernel ... > why not /usr/src/linux-build ? For me linux-build is fine too.
(In reply to comment #20) > (In reply to comment #19) > > the reason this patch never made it into the eclass was because > > /usr/src/linux should not point to objtree. > > objtree is calculated based on the srctree and environment conditionals, and > > as such /usr/src/linux should point to the appropriate srctree. > In that case it may be useful to introduce a /usr/src/build (or similar) that > points to the object tree (which in turn has all information to find the > matching source tree). Defining the build path via environment is quite > contra-productive (would happen very often to forget setting env or having > /usr/src/linux and obj trees not matching) > The important point in my eyes is to have 1 persistent pointer to obj tree > being used by kernel eclasses (a symbolic link under /usr/src would be best) > Eventually deprecating /usr/src/linux symlink would also prevent the issues > with some external makefiles. I can see the argument here, but I'm finding I find it very difficult to justify even more bloat into /usr/src unneccessarily. Also, defining the build-path via environment variables is actually the desired behaviour both within gentoo and upstream which is why the approach has been taken. IMO the only time you need the objtree is when you want to reference the current running kernel, outside of that then we should identify the kernel sources we point to with the /usr/src/linux symlink. In which case would $(readlink /lib/modules/`uname -r/build) not suffice for your tasks? > Kernel obj and src trees should possibly be auto-detected using some > alternative checks (taking the first one working): > 1) Look for /usr/src/build > 2) Look for /usr/src/linux > 3) Look for /lib/modules/`uname -r`/{build,source} although its supported in the eclass, we never use `uname -r` through portage without a very strong argument to do so. > > This patch would overcome the shortfalls of this, but this is not the policy > > which we work to. > What is the policy Gentoo is working towards in this context? I admit to the documentation on this subject being somewhat sparse. However: http://www.gentoo.org/doc/en/kernel-upgrade.xml google: site:gentoo.org /usr/src/linux > > Aside from this, its things like broken makefiles outside of > > portage which expect kernel headers to exist in /usr/src/linux still, which > > although we do not support it will give some perculiar behaviour when > > /usr/src/linux is objtree and srctree = subject to some variable. > As said above, deprecating /usr/src/linux will avoid such perculiar behavior.
(In reply to comment #23) > (In reply to comment #20) > > (In reply to comment #19) > > > the reason this patch never made it into the eclass was because > > > /usr/src/linux should not point to objtree. > > > objtree is calculated based on the srctree and environment conditionals, > > > and as such /usr/src/linux should point to the appropriate srctree. > > In that case it may be useful to introduce a /usr/src/build (or similar) > > that points to the object tree (which in turn has all information to find > > the matching source tree). Defining the build path via environment is > > quite contra-productive (would happen very often to forget setting env or > > having /usr/src/linux and obj trees not matching) > > The important point in my eyes is to have 1 persistent pointer to obj tree > > being used by kernel eclasses (a symbolic link under /usr/src would be > > best) Eventually deprecating /usr/src/linux symlink would also prevent the > > issues with some external makefiles. > > I can see the argument here, but I'm finding I find it very difficult to > justify even more bloat into /usr/src unneccessarily. > Also, defining the build-path via environment variables is actually the > desired behaviour both within gentoo and upstream which is why the approach > has been taken. Even specifying the kernel build dir (ojbtree) via environment does not work correctly (the kernel source tree behind /usr/src/linux is read which can leed to incorrect version checks if KBUILD_OUTPUT's source tree and /usr/src/linux are not the same) Depending on the ebuild (if ebuild is using KV_OUT_DIR or KV_DIR) it will also build against different source trees. Setting KERNEL_DIR in addition to KBUILD_OUTPUT gets error prone (as with /usr/src/linux) because it will happen too often that both trees don't match. To avoid such uncertainty and possible build failure of modules because they need configured tree (= objtree) the search should start from most deterministic place! - objtree is associated to exactly 1 source tree (prefer, detect src tree in objtree makefile!, no matter what /usr/src/linux says. Warning/Error if both don't match is ok) - source tree can have many objtrees (see below -> impossible to find the right one; if srctree == objtree one can assume there is only 1 objtree) > IMO the only time you need the objtree is when you want to reference the > current running kernel, outside of that then we should identify the kernel > sources we point to with the /usr/src/linux symlink. In which case would > $(readlink /lib/modules/`uname -r/build) not suffice for your tasks? To build some external module the source tree is often unsufficient (such modules often need a configured tree which is located in the objtree). In addition, when building a new kernel, external modules may need to be compiled before being able to reboot using new kernel (e.g. drivers for hardware needed at early boot stage) For a single source tree you could also have multiple objtrees, one with a very complete kernel (lots of modules, may things built in) and one with very conservative configuration. (e.g. for a laptop running a minimal kernel on the road and bloated one on the docking station) > > Kernel obj and src trees should possibly be auto-detected using some > > alternative checks (taking the first one working): > > 1) Look for /usr/src/build > > 2) Look for /usr/src/linux > > 3) Look for /lib/modules/`uname -r`/{build,source} > > although its supported in the eclass, we never use `uname -r` through > portage without a very strong argument to do so. This is quite reasonable.
> Even specifying the kernel build dir (ojbtree) via environment does not work > correctly (the kernel source tree behind /usr/src/linux is read which can leed > to incorrect version checks if KBUILD_OUTPUT's source tree and /usr/src/linux > are not the same) Depending on the ebuild (if ebuild is using KV_OUT_DIR or > KV_DIR) it will also build against different source trees. Setting KERNEL_DIR > in addition to KBUILD_OUTPUT gets error prone (as with /usr/src/linux) because > it will happen too often that both trees don't match. I think you've answered my questions further on, but why would an objtree built from a srctree, not match srctree? or more to the point, why would you specify an objtree which isnt in sync with a srctree? afterall, /usr/src/linux (or more accurately, KERNEL_DIR) doesnt fluctuate. > - source tree can have many objtrees (see below -> impossible to find the right > one; if srctree == objtree one can assume there is only 1 objtree) Correct, it is impossible to find the correct one without specifying it. Thats kind of the idea though :) a universal place to find srctree, with an objective/conditional pointer to the specific objtree you want to build for. > For a single source tree you could also have multiple objtrees, one with a very > complete kernel (lots of modules, may things built in) and one with very > conservative configuration. (e.g. for a laptop running a minimal kernel on the > road and bloated one on the docking station) in which case how would portage reliably detect the objtree? This is a chicken & egg situation, and I much prefer to stem from the egg which hatches several chickens than trace it backwards. In honesty, this is continuing to confuse me as I see no practical benefits, nor any real world differences. What problems are you facing setting KERNEL_DIR and KBUILD_OUTPUT? I also cannot see the "out-of-sync" issues unless objtree was not built off srctree, in which case we have no hope anyways?
Created attachment 117659 [details, diff] My current patch for linux-info eclass The issue I have with setting both KERNEL_DIR and KBUILD_OUTPUT is especially that KERNEL_DIR is redundant information. /usr/src/linux is too static and should not be required. It may often be useful to emerge extra kernel modules for a few kernels (module updates for the most recent kernel as well as for the older but more stable kernel) and here it's way easier and quicker to just update one env variable instead of two. I have improved the patch suggested by Honza to be able to detect correct kernel information with just KBUILD_OUTPUT env variable, optionaly without /usr/src/linux symlink or KERNEL_DIR env var (values detected from KBUILD_OUTPUT/Makefile taking precedence over KERNEL_DIR or /usr/src/linux). It works for a few module ebuilds I'm using (out of the tree or from my own overlay)
Just tested the Bruno version of patch and it works for me (and I'm mentioning it here mainly for hope it will wake this bug :-)).
*** Bug 206701 has been marked as a duplicate of this bug. ***
*** Bug 207234 has been marked as a duplicate of this bug. ***
*** Bug 209027 has been marked as a duplicate of this bug. ***
*** Bug 211319 has been marked as a duplicate of this bug. ***
Created attachment 153403 [details, diff] Updated patch so it also works with >=2.5.25 kernels Since 2.6.25 the external build (make O=...) generates a Makefile with a different format, not providing KERNELOUTPUT and KERNELSRC variables anymore but a "magic" MAKEARGS variable instead. The updated patch looks for this new variable and decodes it if present. Note: multiple external modules do not consider the case of distinct locations for KERNEL_DIR and KBUILD_OUTPUT and just fail in this case (they should call make from KBUILD_OUTPUT instead of KERNEL_DIR)
What happened? I've been using patched linux-info.eclass in /usr/local/portage/eclass for some time now and now it stopped working. And compiling ebuild (virtualbox-modules) became unable to find my kernel again. (Compiling with manually set KBUILD_OUTPUT and KERNEL_DIR worked but I really feel stupid when I must manually show portage both of these location only because it's not able to notice that /usr/src/linux is KBUILD_OUTPUT instead of KERNEL_DIR). Is there some progress with this bug?
Created attachment 203372 [details, diff] Patch against linux-info.eclass as of 2009-09-05
(In reply to comment #34) > Created an attachment (id=203372) [edit] > Patch against linux-info.eclass as of 2009-09-05 This version re-introduces breakage if the sources are not configured. Please: - Test and modify that unconfigured sources do not break it (getfilevar requires them to be configured).
(In reply to comment #35) > (In reply to comment #34) > > Created an attachment (id=203372) [edit] > > Patch against linux-info.eclass as of 2009-09-05 > This version re-introduces breakage if the sources are not configured. > Please: > - Test and modify that unconfigured sources do not break it (getfilevar > requires them to be configured). > Huh, passes for me (just some not so nice eerror output from getfilevar): # No /usr/src/linux, no KERNEL_DIR and no KBUILD_OUTPUT: emerge --oneshot hal ... * hal-0.5.11.tar.bz2 RMD160 SHA1 SHA256 size ;-) [ ok ] * checking ebuild checksums ;-) [ ok ] * checking auxfile checksums ;-) [ ok ] * checking miscfile checksums ;-) [ ok ] * Determining the location of the kernel source code * getfilevar_noexec requires 2 variables, with the second a valid file. * getfilevar_noexec <VARIABLE> <CONFIGFILE> * Unable to find kernel sources at /usr/src/linux * This package requires Linux sources. * Please make sure that /usr/src/linux points at your running kernel, * (or the kernel you wish to build against). * Alternatively, set the KERNEL_DIR environment variable to the kernel sources location * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - HOTPLUG * - NET * You're on your own to make sure they are set if needed. * Determining the location of the kernel source code * getfilevar_noexec requires 2 variables, with the second a valid file. * getfilevar_noexec <VARIABLE> <CONFIGFILE> * Unable to find kernel sources at /usr/src/linux * This package requires Linux sources. * Please make sure that /usr/src/linux points at your running kernel, * (or the kernel you wish to build against). * Alternatively, set the KERNEL_DIR environment variable to the kernel sources location * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - INOTIFY_USER * You're on your own to make sure they are set if needed. >>> Unpacking source... # KERNEL_DIR points to raw kernel source tree as untared from kernel.org: KERNEL_DIR=/usr/src/linux-2.6.30.4 emerge --oneshot hal * hal-0.5.11.tar.bz2 RMD160 SHA1 SHA256 size ;-) [ ok ] * checking ebuild checksums ;-) [ ok ] * checking auxfile checksums ;-) [ ok ] * checking miscfile checksums ;-) [ ok ] * Determining the location of the kernel source code * Found kernel source directory: * /usr/src/linux-2.6.30.4 * Found sources for kernel version: * 2.6.30.4 * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - HOTPLUG * - NET * You're on your own to make sure they are set if needed. * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - INOTIFY_USER * You're on your own to make sure they are set if needed. >>> Unpacking source... # /usr/src/build-2.6.30.4 contains just Makefile for out-of-tree build KBUILD_OUTPUT=/usr/src/build-2.6.30.4 emerge --oneshot hal * hal-0.5.11.tar.bz2 RMD160 SHA1 SHA256 size ;-) [ ok ] * checking ebuild checksums ;-) [ ok ] * checking auxfile checksums ;-) [ ok ] * checking miscfile checksums ;-) [ ok ] * Determining the location of the kernel source code * This KBUILD_OUTPUT makefile looks like 2.6.25 or more recent format * Found kernel source directory: * /usr/src/linux-2.6.30.4 * Found kernel object directory: * /usr/src/build-2.6.30.4 * Found sources for kernel version: * 2.6.30.4 * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - HOTPLUG * - NET * You're on your own to make sure they are set if needed. * Unable to check for the following kernel config options due * to absence of any configured kernel sources: * - INOTIFY_USER * You're on your own to make sure they are set if needed. >>> Unpacking source... Note: patch is against linux-info.eclass with the following header: # Copyright 1999-2006 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 # $Header: /var/cvsroot/gentoo-x86/eclass/linux-info.eclass,v 1.61 2009/08/30 22:37:06 robbat2 Exp $ # # Original author: John Mylchreest <johnm@gentoo.org> # Maintainer: kernel-misc@gentoo.org # # Please direct your bugs to the current eclass maintainer :) What am I missing? Do you have a failing test-case I could check on? I admit there have been a few more revisions to linux-info.eclass since 1.61 (e.g. since I synced on Saturday) I will have a look at forward-porting the patch to current revision (as of this writing 1.71) and see I above sample case still work (and getting rid of the complaint from getfilevar_noexec). What the patch is missing is proper fall-back if KERNEL_DIR points to KBUILD_OUTPUT for recent kernel. (it currently only support fallback for older kernels).
Please update to the 1.70 version, and retest. There's definite changes that are needed between 1.61 and 1.70. Here's the list of cases, I think it should be all of them. Each of these needs to be completed with /proc/config.gz existing AND not existing (build it as a module, makes it easiest). Empty: ------ - No /usr/src/linux, no KERNEL_DIR and no KBUILD_OUTPUT KERNEL_DIR='' KBUILD_OUTPUT='' !indirect, !.config, !prep -------------------------- - KERNEL_DIR points to raw kernel source tree - no .config exists - "make prepare" NOT done KERNEL_DIR=/usr/src/linux-2.6.30.4 !indirect, .config, !prep ------------------------- - KERNEL_DIR points to raw kernel source tree - .config exists - "make prepare" NOT done KERNEL_DIR=/usr/src/linux-2.6.30.4 !indirect, .config, prep ------------------------ - KERNEL_DIR points to raw kernel source tree, - .config exists - "make prepare" done KERNEL_DIR=/usr/src/linux-2.6.30.4 indirect, !.config, !prep ------------------------- - /usr/src/build-2.6.30.4 contains just Makefile for out-of-tree build - no .config - "make prepare" NOT done. KBUILD_OUTPUT=/usr/src/build-2.6.30.4 indirect, .config, !prep ------------------------ - /usr/src/build-2.6.30.4 contains just Makefile for out-of-tree build - .config exists - "make prepare" NOT done. KBUILD_OUTPUT=/usr/src/build-2.6.30.4 indirect, .config, prep ----------------------- - /usr/src/build-2.6.30.4 contains just Makefile for out-of-tree build - .config exists - "make prepare" done. KBUILD_OUTPUT=/usr/src/build-2.6.30.4
Tested on sys-kernel/gentoo-sources-2.6.30-r5 ... didn't worked until I added the KERNELOUTPUT and KERNELSRC variables. Seems like it doesn't look for >=2.6.25 possibility ...
Created attachment 203706 [details, diff] Patch against linux-info.eclass v1.71 (as of 2009-09-09) This updated patch should match behavior of v1.71 eclass but properly handle cases where just KBUILD_OUTPUT is provided instead of mixing data from runtime kernel and build tree. Test logs follow in separate attachment for readability.
Created attachment 203709 [details] Test case logs for v1.71 with(out) patch in attachment #203706 [details, diff] Test case log with comment on each test-case result. Please correct me if my expectations are wrong.
(In reply to comment #38) > Tested on sys-kernel/gentoo-sources-2.6.30-r5 ... didn't worked until I added > the KERNELOUTPUT and KERNELSRC variables. Seems like it doesn't look for > >=2.6.25 possibility ... > With my patch it's sufficient to indicate KBUILD_OUTPUT, KERNEL_DIR is auto-detected if KBUILD_OUTPUT is valid. What is your setup looking like? e.g. emerge cmd-line, values of KBUILD_OUTPUT, KERNEL_DIR and if /usr/src/linux exists what it refers to. If KBUILD_OUTPUT points to unconfigured kernel tree it's worth noting as well. With latest patch (attachment #203706 [details, diff]) it should work just as good for older as well as newer kernels (though I didn't check with pre-2.6.25 kernels yet)
Close on the patch, the testsuite made it nice to see. > # KERNEL_DIR= KBUILD_OUTPUT= emerge --oneshot hal > * Found sources for kernel version: > * 2.6.31 This might be a side-effect of the other recent changes, but by empty I meant that there was NO sources installed at all. You just identified an additional testcase: sources present, variables set to empty (to override anything in make.conf) > # KERNEL_DIR= KBUILD_OUTPUT=/usr/src/build-2.6.30.4 emerge --oneshot hal I'm not sure that's a valid commandline. Short of probing the output location for it's source location, we have no way of knowing. Should we even support this combination? I'm against it. Just do: [ -z "${KERNEL_DIR}" -a -n "${KBUILD_OUTPUT}" ] && KERNEL_DIR="${KBUILD_OUTPUT}" As I don't think it worked before either.
(In reply to comment #42) > Close on the patch, the testsuite made it nice to see. > > > # KERNEL_DIR= KBUILD_OUTPUT= emerge --oneshot hal > > * Found sources for kernel version: > > * 2.6.31 > This might be a side-effect of the other recent changes, but by empty I > meant that there was NO sources installed at all. > You just identified an additional testcase: sources present, variables set > to empty (to override anything in make.conf) This detection was based on runtime kernel as I don't have and /usr/src/linux symlink, but I can also run checks after hiding /usr/src. When it did find something it is either by looking for /usr/src/linux-$(uname -r) or going via /lib/modules/$(uname -r)/{source,build}/ > > # KERNEL_DIR= KBUILD_OUTPUT=/usr/src/build-2.6.30.4 emerge --oneshot hal > I'm not sure that's a valid commandline. Short of probing the output > location for it's source location, we have no way of knowing. Should we > even support this combination? I'm against it. Just do: > [ -z "${KERNEL_DIR}" -a -n "${KBUILD_OUTPUT}" ] && > KERNEL_DIR="${KBUILD_OUTPUT}" > As I don't think it worked before either. Well this is exactly the case I do want to get working as providing both KERNEL_DIR and KBUILD_OUTPUT does not make sense. KBUILD_OUTPUT does know where its sources are located (if they exist). It makes no sense to request the user to tell us where it is (this avoids having KBUILD_OUTPUT and KERNEL_DIR refer to unrelated trees) KERNEL_DIR never knows where its matching KBUILD_OUTPUT resides (as there may be multiple build trees for a single source tree) except when both are the same (and a .config exists) as then there is a 1-1 mapping.
I meant adding KERNELOUTPUT and KERNELSRC into kernel objtree Makefile thus emulating pre-2.6.25 setup. My configuration is no variables set and /usr/src/linux pointing to objtree (KBUILD_OUTPUT) which is different from KERNEL_DIR (out-of-tree build). Note that it was with linux-info.eclass-20090905.patch ... I didn't tested linux-info.eclass-20090909.patch yet.
linux-info.eclass-20090909.patch is not working either (with my configuration and linux-2.6.30-gentoo-r8).
The 20090909 version is useless. Now it doesn't work even when emulating pre-2.6.25 setup. And how it's supposed to work anyway - it only modified the case when KBUILD_OUTPUT is set ... should I repeat my configuration again? Note that the /lib/modules/`uname -n`/source is not working either and when I remove /lib/modules/`uname -n`/source to run /lib/modules/`uname -n`/build version (why can't it try both?) it doesn't work too.
Created attachment 212561 [details, diff] Patch against linux-info.eclass v1.73 (as of 2009-12-09) (In reply to comment #46) > The 20090909 version is useless. Now it doesn't work even when emulating > pre-2.6.25 setup. And how it's supposed to work anyway - it only modified the > case when KBUILD_OUTPUT is set ... should I repeat my configuration again? There was an incorrect '!' in there... Here is a newer one It fixes <2.6.25 kernel build system not being properly recognized as getfilevar_noexec() fails to extract vars that are set with ':=' instead of just '='. It should accept KBUILD_OUTPUT alone, with KERNEL_DIR and also KERNEL_DIR pointing to kernel object dir instead of source dir. When looking up runtime kernel KBUILD_OUTPUT is set so result is correct for out-of-tree build (if both source and object dirs are still present) Results look fine when testing with sys-fs/udev-146-r1
Created attachment 212562 [details, diff] Patch against linux-info.eclass v1.73 (as of 2009-12-09) Oops, previous patch ( attachment #212561 [details, diff] ) was incremental above my '!' fix I had locally instead of being against v1.73. This new one is against v1.73
Great - works! (on openafs-kernel). (On a partially related note, any idea why /usr/local/portage/eclass is not working anymore for me? I must now copy the patched linux-info.eclass to /usr/portage/eclass ...)
Can you please test and then respin against rev 1.81 as needed. Additional changes have been made to many parts of the eclass per the now-completed work on non-fatal config checks without kernel sources. The logic for SUBLEVEL, EXTRAVERSION, OUTPUT_DIR are all changed somewhat.
Created attachment 216766 [details, diff] Patch against v1.81 Ported patch for v1.81, I've done limited testing with udev against 2.6.24 and 2.6.32 kernels, both built out of tree. In all cases it also determines running kernel but seems to call make on the kernel tree at some time as there is a long wait period when kernel tree is not in cache. I've not checked yet how v1.81 alone behaves, so I don't know if that hit is caused by my patch or not. Feedback welcome! A part that should probably be applied to eclass independently of the rest is hunk 2 which makes the getfilevar_noexec() function properly extract variables in cases when in Makefile they are assigned with ':=' instead of just '='. What was the idea behind using the variable KERNEL_MAKEFILE to store path to Makefile? Is it just to avoid some string processing or does it have some additional use?
(In reply to comment #51) > Created an attachment (id=216766) [details] > Patch against v1.81 > > Ported patch for v1.81, I've done limited testing with udev against 2.6.24 and > 2.6.32 kernels, both built out of tree. You dropped an important part of 1.81. The block starting with: # Check if the Makefile is valid for direct parsing. that sets the mkfunc variable. It's important for a couple of reasons: 1. It turned out that there are kernel Makefiles out there that append to EXTRAVERSION via using makefile string appending, so we HAVE to execute the makefile to get the correct value. 2. If the makefile is NOT directly parsable by make, using the regular getfilevar function results in a lot of error output, which gets captured by and foo="$(getfilevar ...)" construct, and leads to weird behavior. We'll need to use a similar check on the $KBUILD_OUTPUT/Makefile location. I've just refactored it to make a new function, get_makefile_extract_function commited in v1.83. > In all cases it also determines running kernel but seems to call make on the > kernel tree at some time as there is a long wait period when kernel tree is not in cache. > > I've not checked yet how v1.81 alone behaves, so I don't know if that hit is > caused by my patch or not. Can you please test without your patch? We have to hit the kernel Makefiles at some point. > A part that should probably be applied to eclass independently of the rest is > hunk 2 which makes the getfilevar_noexec() function properly extract variables > in cases when in Makefile they are assigned with ':=' instead of just '='. Applied as v1.82, thanks. > What was the idea behind using the variable KERNEL_MAKEFILE to store path to > Makefile? Is it just to avoid some string processing or does it have some > additional use? Just avoiding string processing and duplicate constants. More questions: in get_running_version, should we explicitly prevent it from checking KBUILD_OUTPUT/OUTPUT_DIR?
> > Ported patch for v1.81, I've done limited testing with udev against 2.6.24 > > and 2.6.32 kernels, both built out of tree. > You dropped an important part of 1.81. > The block starting with: > # Check if the Makefile is valid for direct parsing. > that sets the mkfunc variable. > > It's important for a couple of reasons: > 1. It turned out that there are kernel Makefiles out there that append to > EXTRAVERSION via using makefile string appending, so we HAVE to execute the > makefile to get the correct value. > 2. If the makefile is NOT directly parsable by make, using the regular > getfilevar function results in a lot of error output, which gets captured by > and foo="$(getfilevar ...)" construct, and leads to weird behavior. We'll need > to use a similar check on the $KBUILD_OUTPUT/Makefile location. > > I've just refactored it to make a new function, get_makefile_extract_function > commited in v1.83. Thanks for the details. Yep I dropped that part as I was too lazy to redo the check if the target Makefile changed (because of switching from build tree to source tree) and thought that failing cases would be those caused by ':=' versus '=' assignments. > > In all cases it also determines running kernel but seems to call make on > > the kernel tree at some time as there is a long wait period when kernel > > tree is not in cache. > > > > I've not checked yet how v1.81 alone behaves, so I don't know if that hit is > > caused by my patch or not. > Can you please test without your patch? We have to hit the kernel Makefiles at > some point. I will check (one wild guess is that it's related to my current tree being a GIT tree with CONFIG_LOCALVERSION_AUTO=y) > More questions: > in get_running_version, should we explicitly prevent it from checking > KBUILD_OUTPUT/OUTPUT_DIR? Not sure what you mean with 'prevent it from checking KBUILD_OUTPUT/OUTPUT_DIR', it shall give the right results when kernel is being built in-tree as well as out of tree. What *might* not be needed is to check if build and source symlinks under ${ROOT}/lib/modules/${KV_FULL}/ are consistent though what definitely makes sense is to add a check that extract version still matches ${KV_FULL}! (as especially with GIT trees that get built the location is still the same but kernel version there probably doesn't match anymore!)
Created attachment 216846 [details, diff] Updated patch against v1.83 This patch should take care of points raised in comment #53 as well as #54 The long think time is git when it calculates extra version for CONFIG_LOCALVERSION_AUTO=y
Created attachment 216847 [details] Test log for v1.83 without and with patch applied
Comment on attachment 216846 [details, diff] Updated patch against v1.83 i've updated linux-info.eclass so it has basic KBUILD_OUTPUT detection support (by way of bug 454294). it's enough that i have /usr/src/linux set as my output and /usr/src/linux-3.7 is the current kernel source tree. i don't set any kernel variables, so it's auto-detecting everything by way of KERNEL_DIR=/usr/src/linux >+ get_version_warning_done= > if [[ -f ${ROOT}/lib/modules/${KV_FULL}/source/Makefile ]]; then > KERNEL_DIR=$(readlink -f ${ROOT}/lib/modules/${KV_FULL}/source) >+ if [[ -f ${ROOT}/lib/modules/${KV_FULL}/build/Makefile ]]; then >+ KBUILD_OUTPUT=$(readlink -f ${ROOT}/lib/modules/${KV_FULL}/build) >+ fi > unset KV_FULL >- get_version >- return $? >+ if get_version; then >+ if [[ "$(uname -r)" = "${KV_FULL}" ]]; then >+ return 0 >+ else >+ qewarn "Kernel sources have changed since kernel was built" >+ qewarn "git tree updated or patch applied?" >+ return 0 >+ fi >+ else >+ return $? >+ fi > elif [[ -f ${ROOT}/lib/modules/${KV_FULL}/build/Makefile ]]; then > KERNEL_DIR=$(readlink -f ${ROOT}/lib/modules/${KV_FULL}/build) > unset KV_FULL >- get_version >- return $? >+ if get_version; then >+ if [[ "$(uname -r)" = "${KV_FULL}" ]]; then >+ return 0 >+ else >+ qewarn "Kernel sources have changed since kernel was built" >+ qewarn "git tree updated or patch applied?" >+ return 0 >+ fi >+ else >+ return $? >+ fi this section is wrong and looks overly complicated. you shouldn't be checking `uname -r` here as that only tests the currently booted kernel and often times has no bearing at all on the source you're compiling. i'd just drop it all.
If this is not fixed with Vapier's change and there's something more here to do, please re-open.
*** Bug 454294 has been marked as a duplicate of this bug. ***
