This bug is common for all kernel sources ebuilds: sys-kernel/xen-sources sys-kernel/xbox-sources sys-kernel/vserver-sources sys-kernel/vanilla-sources sys-kernel/usermode-sources sys-kernel/openvz-sources sys-kernel/mm-sources sys-kernel/hardened-sources sys-kernel/gentoo-sources and maybe more. I personally use only sys-kernel/gentoo-sources . Source of problem: esync && emerge -DaNuv world condition: perform an update where both the kernel source and a module are updated. Most often, I see in the build list that the linux tree is to be updated first, and the module at last. key condition: have the flag "symlink" turned on. Problem is that, in an ideal world, I would dream of emerge to be able to emerge from first to last ebuild in annouced order without failure. Fact is that, when conditions are met, it is possible to assume that emerge will fail. This is because when updateing sources, the /usr/src/linux symlink is redirected to the new fresh tree; when some driver are update a few minutes later, they complain tree is uncompiled, what is very easy to understand. The module complaining is most often lirc; but svgalib and usbirboy may also complain. I would like to turn "build" flag on, but euse says not to do so: # euse -i build global use flags (searching: build) ************************************************************ [- c ] build - !!internal use only!! DO NOT SET THIS FLAG YOURSELF!, used for creating build images and the first half of bootstrapping [make stage1] local use flags (searching: build) ************************************************************ no matching entries found proposed solution: all source ebuilds should be sensible to a new use flag that would compile automatically the kernel during emerge. As explained in http://forums.gentoo.org/viewtopic.php?p=3847543#3847543 , many people do not want this; thus, the default setting should be to have the flag OFF. I am not to list here why people dont want auto-build doring emerge. I am to explain why other people would need it: - to me, emerge is the place to compile things; compile should not rely on any manual things, nor third party; and at the moment, apart from binary apckages, the kernel is the only package that is installed via sources, and that is not compiled during emerge - when emerge terminates successfully, I expect system to be "up to date"; when it returns, sometimes it says to run etc-update; that IS a mention to manual intervention, but at least emerge says about it; whereas after a kernel source update, emerge does NOT remind to compile kernel. So, if update updated kernel source, did not fail on lirc, and user did not remember there is a kernel update on way, user will miss the kernel update, and system is to reboot on deprecated version - why would emerge download and compile any source *except* kernel ? turning symlink is not a solution for me: - with symlink off, lirc will be updated against the old kernel - one more manual operation will be required when user will update the kernel If user wants to be notified of kernel update, ebuild can use enotices to do so. Any way, only users getting aware of the flag will turn it on; so, when they turn it on, they should read the documentation about it. Reasons not to use -bin package for kernel are too long to be explained here. The feature of the flag should be: - default state = OFF - when ON, it should read a configuration setting in /etc (either new file, new folder, or a new line in /etc/make.conf) - propose to run one or more actions throughout (inclusive OR): genkernel (with specific options), user script, anything else ... Operations to perform should be possibly cumulative, and have optionnal order; a critical case should allow for example these chronological steps: - user script1 - genkernel - user script 2 - module-rebuild rebuild - user script 3 Maybe a simplified version of the flag should just run /etc/gentoo-sources_autobuild.sh and report everything in ewarns. User is responsible for writing apropriate scripts, or customise genkernel to match his needs.
Having optional portage-controlled kernel upgrades would be a nice features, but not for the reasons you describe. The whole USE=symlink thing is broken and will be removed. The reason behind this is that the symlink should point to the built kernel you want the system to run against (or to run against very soon) and this is clearly not the case immediately after it has been emerged. In the example you give, lirc SHOULD be updated against the "old" (read: currently targetted/running) kernel (as merely merging kernel sources does NOT imply you will ever run them), then when you come to manually compile the new one later, you will run modules-update after updating the symlink and all will be well. I'll leave this bug open for a while (for the automatic build functionality) but while kernel resources are really thin it really needs someone to contribute the functionality before things can go further.
Key point for me is that when emerge terminates successfully, all downloaded sources should be compiled, and no other command than reboot should be required to get kernel working. Lirc part just helped me understanding that emerge did not build the kernel. Most important for me is that successfull "emerge gentoo-source" should result in "ready to use binary program", whatever the kernel is aimed for host or any other box ... If team is that thin, I may propose a basic ebuild if you want me to introduce a proof-of-concept ...
(In reply to comment #2) > Key point for me is that when emerge terminates successfully, all downloaded > sources should be compiled, and no other command than reboot should be required > to get kernel working. This is not the way the system is designed, and I don't think this will change. Note that the package names all end in SOURCES. If the system was changed to revolve around actually building the kernel then we would lose lots of flexibility. However, adding something on the top to allow automatic compilation would certainly be nice for other classes of users, so yes, if you can implement something and implement it nicely then I'll be glad to consider it.
(In reply to comment #1) > Having optional portage-controlled kernel upgrades would be a nice features, > but not for the reasons you describe. > > The whole USE=symlink thing is broken and will be removed. The reason behind > this is that the symlink should point to the built kernel you want the system > to run against (or to run against very soon) and this is clearly not the case > immediately after it has been emerged. I dissagree with portage-controlled kernel upgrades. It's good that USE=symlink is getting punted. (In reply to comment #3) > (In reply to comment #2) > > Key point for me is that when emerge terminates successfully, all downloaded > > sources should be compiled, and no other command than reboot should be required > > to get kernel working. > > This is not the way the system is designed, and I don't think this will change. > Note that the package names all end in SOURCES. If the system was changed to > revolve around actually building the kernel then we would lose lots of > flexibility. > > However, adding something on the top to allow automatic compilation would > certainly be nice for other classes of users, so yes, if you can implement > something and implement it nicely then I'll be glad to consider it. > If anything it should be added to eselects kernel module, some option to automaticaly copy the old config and run "make oldconfig && mount /boot ; make && make modules modules_install install && module-rebuild -X rebuild". You get the idea.
(In reply to comment #1) > The whole USE=symlink thing is broken and will be removed. The reason behind > this is that the symlink should point to the built kernel you want the system > to run against (or to run against very soon) and this is clearly not the case > immediately after it has been emerged. It is for me. (In reply to comment #3) > This is not the way the system is designed, and I don't think this will change. > Note that the package names all end in SOURCES. Then, should we create a sys-kernel/gentoo-kbuild ebuild that would dep on sys-kernel/gentoo-sources ? Maybe this would calm down those who do not understand that an OFF defaulted option is harmless ! but why not ... I dislike this because it will require to update twice ebuilds on every kernel release ... it would be more explicit for users, but would put more load for maints and server (I am not only talking about my prefered sys-kernel/gentoo-sources but also about the dozen of other kernel sources, possibly even BSD and Hurd if they are still alive around in Gentoo ...). Furthermore, I prefer an option that is visible through "emerge -vp foobar" over any other alternative. > If the system was changed to > revolve around actually building the kernel then we would lose lots of > flexibility. For end-user desktop, I cant see what could be the problem. All kernel in the same branch should be conf compaitble. If kernel does not work, user can always reboot back to the old one ! It is implicit to me this option should NEVER be turned on for servers ! (but still be available for ALL ebuilds ! no segregation ! ) *My use of computing is to script and automate repetitive tasks*.
(In reply to comment #5) > (In reply to comment #1) > > The whole USE=symlink thing is broken and will be removed. The reason behind > > this is that the symlink should point to the built kernel you want the system > > to run against (or to run against very soon) and this is clearly not the case > > immediately after it has been emerged. > > It is for me. No, it can't be. I think you misunderstood what I was saying. My point is that when you do USE=symlink, the symlink is updated immediately after source installation, whereas the symlink should only ever be pointing at a configured & compiled set of kernel sources. With the current system there is simply no way you can have configured and compiled the kernel before the USE=symlink code updates the symlink, since it does it immediately. > Then, should we create a sys-kernel/gentoo-kbuild ebuild that would dep on > sys-kernel/gentoo-sources ? Maybe this would calm down those who do not > understand that an OFF defaulted option is harmless ! but why not ... I dislike > this because it will require to update twice ebuilds on every kernel release > ... it would be more explicit for users, but would put more load for maints > and server (I am not only talking about my prefered sys-kernel/gentoo-sources > but also about the dozen of other kernel sources, possibly even BSD and Hurd if > they are still alive around in Gentoo ...). Furthermore, I prefer an option > that is visible through "emerge -vp foobar" over any other alternative. I dislike the idea of a separate ebuild too. My point was just many of your 'complaints' about the way things currently work are not viewed as complaints by myself -- the system is like it is BY DESIGN. So, if you attempt to redesign the whole thing, we'll almost certainly reject it. On the other hand, if you add some kind of layer on top which generalises much of the existing system and allows users to automatically build kernels in some way, then this is something we'd be much more open to. > For end-user desktop, I cant see what could be the problem. All kernel in the > same branch should be conf compaitble. If kernel does not work, user can always > reboot back to the old one ! Agreed. However, Gentoo serves *a lot* more than end-user desktops. We need to maintain the flexibility of the way the sources currently work (which makes a lot of sense for servers and other systems, as you acknowledge), but it would also be nice to offer more automated (less choice) functionality. But the only way this can work effectively is if you design the automated functionality around the source flexibility, not the other way around. I think we both understand what the other is saying though. I believe this is a harder problem than it sounds, but I look forward to seeing your efforts in code :)
Yup; I went in forums for chat; I came here for action. But I admit I am not very good at coding in a flexible way. I ll try to provide a modified ebuild soon. Cant promise it will be quick. But this problem is getting bigger and bigger in my mind; so, my brain affords more and more ennergy to fix it.
I'm going to close this while we don't have anything to review. If you do follow up with this, please reopen when an initial implementation is ready.
I thhink I discovered an other problem in the eclass; I am actualy patching kernel-2.eclass to add my use flag, and see what emerge proposes me: [ebuild NS ] sys-kernel/gentoo-sources-2.6.21-r2 USE="symlink -autobuild -build" 197 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.21-r1 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.20-r3 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.20-r4 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.20-r5 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.20-r6 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.20 USE="symlink -autobuild% -build" 0 kB [ebuild R ] sys-kernel/gentoo-sources-2.6.21 USE="symlink -autobuild% -build" 0 kB My warning is that when you will want to remove symlink (see comment #1 ), you are likely to force users to recompile all their kernels ... Re-opening bug. Patch is written, but I need a few hours testing before publishing them here.
my workaround about previous comment is to fill /etc/portage/package.mask with <sys-kernel/gentoo-sources-2.6.21-r2
Usually, when I alter an ebuild, emerge requires me to update manifest. Today, I have updated an eclass, and I am not asked to update anything. Thus I conclude that eclasses do not have manifest; is there a possible security issue in this ? I wonder if adding a manifest would improove security against file alteration (even accidental one).
Created attachment 120707 [details] /etc/conf.d/kernel-autobuild put this file under /etc/conf.d/kernel-autobuild put the last part in the apropriate ebuild, then remove it from the file.
It seems like genkernel can not be called from within an ebuild, due to sandbox. Thus, I giveup. I have no idea how to workaround this problem. I want to build kernel from /usr/src without copying it, and this point seems impossible to me. I just give state of art, if any one want to see my attempts. An answer to comments #9 and #11 would be apreciated. Initial implementation is not efficient. WONTFIX.
Thanks for spending time looking into this. (In reply to comment #9) > I thhink I discovered an other problem in the eclass; I am actualy patching > kernel-2.eclass to add my use flag, and see what emerge proposes me: This is not related to the eclass, it's a portage thing. (In reply to comment #11) > Usually, when I alter an ebuild, emerge requires me to update manifest. > > Today, I have updated an eclass, and I am not asked to update anything. Thus I > conclude that eclasses do not have manifest; is there a possible security issue > in this ? I wonder if adding a manifest would improove security against file > alteration (even accidental one). Again, this is unrelated to this bug, it's a portage thing. There is a bug open for this.
