current ebuild for grub-1.96 stripps binaries and grub cant load its modules according to http://savannah.gnu.org/bugs/?22750 grub MUST be unstripped Reproducible: Always
This issue can be resolved adding RESTRICT="strip" to grub ebuild
Created attachment 160988 [details] grub-1.96.ebuild ebuild for grub-1.96 works on x86 and x86_64 no need multilib toolchain setup anymore =)
Hey how about fixing this bug? It seems that vapier dont test grub-1.96 at all when he add it. While this bootloader works great in contrast of grub-legacy
grub's module loader looks for symbol names in .symtab table which is dropped by strip. See grub_dl_resolve_symbols() in kern/dl.c for more information on how this works... So yes, strip kills grub. If there will be no objections I'll commit this ebuild at the end of this week.
And while we are here and not to forgive to fix this during commit. seems that license is also changed. From NEWS: - The license term is changed to GNU General Public License Version 3.
Uh, and why netboot is still in IUSE. Seems that grub2 still does not have netboot features...
of course i never tested it. i dont care about the bleeding edge and users requested the ebuild. plus the thing is hard masked and marked as "unsupported". i could always just delete it if it makes you feel better. i guess people dont understand the concepts in play here.
Comment on attachment 160988 [details] grub-1.96.ebuild committing an ebuild because it works does not mean it's correct. simply look at the build and the answer is obvious: grub2 modules are relocatable objects which means only those need to be filtered. so use STRIP_MASK="/lib*/grub/*/*.mod".
Thanks for suggestion. I'll check and will do this way.
Created attachment 176735 [details] grub-1.96.ebuild While I'm working on grub-9999.ebuild, here's this one too. Adds STRIP_MASK="/lib*/grub/*/*.mod" declaration.
roger has fixed this with Bug 252769 http://sources.gentoo.org/sys-boot/grub/grub-1.96.ebuild?r1=1.5&r2=1.6
Created attachment 176811 [details] grub-1.96-r1.ebuild - Version bump for taking Spanky's suggestion for not stripping *.mod files. - Unifies with grub-9999 SVN ebuild - Adds trivial comment for installing (Spanky, I've included if/then's statement per your request -- separating the fetch locations, etc.)
Created attachment 176830 [details] grub-1.96-r1.ebuild - Removed KEYWORD definition from other then 9999 versions.
Its NOT fixed since 1 It shouldnt use multilib setup on amd64 iyt simply not working 2 -9999 version should install grub headers in /usr/include not in /include
Created attachment 176846 [details] grub-1.96.ebuild this ebuild works fine on x86 and x86_64
this bug is only about stripping and said issue is fixed
Have you test your grub2 ebuilds? RESTRICT="strip" still needed since with in portage version I still have problem that grub cannot find its modules -> so i cannot find root partition -> system can't boot
> since with in portage version I still have problem that grub cannot find its > modules -> so i cannot find root partition -> system can't boot > Same here. For a rather long time, btw. At least on x86-64 it can't find modules so you can't use it in real system.
Created attachment 176869 [details] grub-9999.ebuild working ebuild for grub2
Comment on attachment 176869 [details] grub-9999.ebuild and you get the same answer for posting the same "solution": RESTRICT=strip is wrong and not acceptable
besides, current ebuild works just fine for me: # emerge -C =grub-9999 # emerge =grub-9999 # file /lib32/grub/i386-pc/*.mod | head /lib32/grub/i386-pc/acorn.mod: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped .......
(In reply to comment #21) > besides, current ebuild works just fine for me: > # emerge -C =grub-9999 > # emerge =grub-9999 > # file /lib32/grub/i386-pc/*.mod | head > /lib32/grub/i386-pc/acorn.mod: ELF 32-bit LSB relocatable, Intel 80386, > version 1 (SYSV), not stripped > ....... > So you want to say that you can boot using portage version of grub2? right? or you again test only that it builds for you? whats wrong with RESTRICT="strip"? You only say thats wrong solution but you doesnt say why its wrong
the answer is obvious: RESTRICT=strip prevents stripping of all files, not specific ones (the *.mod) if you have a problem with the grub2 ebuild, you can figure it out. it isnt supported (which is why it has KEYWORDS="").
Alexey: Since nobody is posting write-ups on how to install Grub2 or the problems with it, all I can do is build Grub2 and run grub-setup /dev/sda. Booting is still a problem here too. <shrugs> I'll give booting another shot with restrict strip. Ditto the concern of Spanky's. Restrict strip strips everything. As such, it's a hack to fix a broken problem. If you read the change log, you would have seen all I did was employ his suggestion preventing stripping of *.mod files. This bug is going to be an issue as Grub2 is a required dependency of Ext4 which is now in the recently released kernels. Let's not get our panties in a bunch here ladies. ;-) Spanky, I got the file | head post and now see it's not stripping. Thanks for the tip! I've got some things to do now... but will be back later.
I know that grub2 is unsupported software =) but its only a way to boot system in some specific configurations such as efi systems or for example if you using md or dmraid so for example as i describe above if grub installed via curent ebuild that exist in tree then after running grub-install system backame unbootable because of grub kernel cannot load modules -> it cannot find root device if i remove STRIP_MASK and add RESTRICT=strip instead after running grub install system will boot fine BTW multilib setup not needed for grub2 anymore. It compiles and works fine without it on amd64
(In reply to comment #23) > the answer is obvious: RESTRICT=strip prevents stripping of all files, not > specific ones (the *.mod) > > if you have a problem with the grub2 ebuild, you can figure it out. it isnt > supported (which is why it has KEYWORDS=""). > And still cant figure out why have some file unstripped is bad? there many ebuilds in off tree that uses same hack. And again have all grub2 files unstripped leads to working bootable system!
You're right. $ fgrep /usr/portage/* -r -e "RESTRICT" |grep strip Although most packages using this are binary packages (aka closed source/ proprietary). I did find one well known open source package dev-libs/klibc using the same, before stopping the fgrep process. I'm guessing the reason it's frowned on, without stripping file size is increased. (I'm not sure of the specifics of security or performance.) Need to find why STRIP_MASK isn't providing a working grub which was suggested by Spanky and further notes it is working (see Comment #21). What else should be prevented from stripping is the question. Alexey? Let me know what people decide here and I'll be glad submit new ebuilds & will try again later booting.
I've got a grub2 working install finally... (... thanks to the lack of documentation on grub2 ;-) Anyhow, I'm guessing /sbin/grub* binaries are the ones that also need strip masking (aside from *.mod files). I don't know if /bin/grub* files are also used by grub-setup... but at this point, I'd just say screw-it and use RESTRICT="strip".
cr*p. hang on. just rechecked the grub ebuild I thought I emerged and found RESTRICT="strip" commented. I need to retest/recompile to ensure it's working.
Created attachment 176933 [details] grub-1.96-r1.ebuild - Changes license to GNU GPL-3 - Modify/Add RESTRICT="strip"
Created attachment 176935 [details] grub-9999.ebuild - Changes license to GNU GPL-3 - Modify/Add RESTRICT="strip"
If you really want STRIP_MASK, then I'd suggest including all bin/sbin files as I'm not sure which is affected by this issue. /lib/grub/*/*.mod /sbin/grub* /bin/grub* The other files are scripts, includes, docs. <shrugs>
I did some research on the discussion of STRIP_MASK (.. i think i see your point using this instead of RESTRICT). Is the following correct/suitable pattern matching for STRIP_MASK variable? $ ls /{bin,lib,sbin}/{grub*,grub/*/*.mod} ls: cannot access /bin/grub/*/*.mod: No such file or directory ls: cannot access /sbin/grub/*/*.mod: No such file or directory But with this, the following installed files are listed correctly: /bin/grub*, /lib/grub/*/_bsd.*mod, /sbin/grub* I'm guessing STRIP_MASK will ignore the "no such file/dir" errors? As such: STRIP_MASK="/{bin,lib,sbin}/{grub*,grub/*/*.mod}"
Created attachment 176938 [details] grub-1.96-r1.ebuild - Changes license to GNU GPL-3 - Modify STRIP_MASK to include bin & *.mod files
Created attachment 176939 [details] grub-9999.ebuild - Changes license to GNU GPL-3 - Modify STRIP_MASK to include bin & *.mod files (Alexey, please test grub-1.96-r1. Mine seems to boot just fine using grub-9999.)
roger and SparKY src_compile should look like this so ebuild would work on x86_64 =) src_compile() { use custom-cflags || unset CFLAGS CPPFLAGS LDFLAGS use static && append-ldflags -static econf \ --sbindir=/sbin \ --bindir=/bin \ --libdir=/$(get_libdir) \ --includedir=/usr/include \ || die "econf failed" emake -j1 || die "making regular stuff" }
And btw roger's STRIP_MASK is equivalent to RESTRICT=strip in this case So Vapier give an nasver why in this case STRIP_MASK is a good solution while RESTRICT=strip is bad even when they are equivalent ;-)
Comment on attachment 176938 [details] grub-1.96-r1.ebuild merged the LICENSE change and the multilib drop -- note that the multilib code worked just fine, but if it isnt need, then it can be dropped
Comment on attachment 176939 [details] grub-9999.ebuild i'm not adding an ebuild with RESTRICT=strip or STRIP_MASK set like that. figure out what's actually going on.
Created attachment 176946 [details] grub-1.96-r1.ebuild - Changes license to GNU GPL-3 - Modify STRIP_MASK to include bin & *.mod files
Created attachment 176948 [details] grub-9999.ebuild - Changes license to GNU GPL-3 - Modify STRIP_MASK to include bin & *.mod files Hopefully I got this correct. (I forgot multilib though.)
Comment on attachment 176946 [details] grub-1.96-r1.ebuild there's no need to post two ebuilds when they're exactly the same thing
> i'm not adding an ebuild with RESTRICT=strip or STRIP_MASK set like that. > figure out what's actually going on. Is it better to have non-working package in portage then to post working ebuild that doesn't violate any rules? If there is any problems with such RESTRICT or STRIP_MASK - give a link about it.
(In reply to comment #36) Alexey let's just focus on this bug for now. My recommendation is to open a new bug for your x86_64 issue. Besides, it looks more then just a one-liner too. And, I'm poor, wifeless & crippled. As such, I only have x86_32 boxes here. :-/ (In reply to comment #42) But grub-9999 builds from SVN and grub-1.96 builds from a snapshot (granted, a year old snapshot). <shrugs> ... but, granted, the ext4 fs articles do recommend grub-1.96. Is it pushed to the tree & can we close?
DOH! I meant "but, granted, the ext4 fs articles do recommend grub-9999/svn version!
if it's broken, i can always just drop it from the tree everything has been committed but the strip related change
> if it's broken, i can always just drop it from the tree > > everything has been committed but the strip related change > It's not bootable because it need to be striped. Can you imagine bootloader that needs libraries from not mounted partition? Well, grub-2 from this ebuild is. It needs to be striped to work properly (with plugins, etc.) And you didn't answered, why you are against this kind of fix? And will you do when grub-2 will came out? Do same? Non-working grub-2 ebuild without stripe?
you have your answer already. figure out what exactly is going on.
(In reply to comment #48) > you have your answer already. figure out what exactly is going on. > Then just say: I'm too lazy to make working ebuild for grub2. Grub2 binaries needs to be unstriped (not only modules, but binaries too) or it'll fail to load modules and you'll get non-working bootloader. If you dislike RESTRICT="strip" then make properly fix yourself or point us to documentation which will explain why it's so bad and how to avoid it, but make all binaries unstriped.
Guys, can we please stop the "pissing" contest? This is not a discussion forum, but a bug tracker. Shouting against the Gentoo maintainer won't get any issue fixed. SpanKY has explained that anyone interested will have to find out what's going on as this isn't top priority right now.
I think people are wrongly assuming Spanky knows what is causing this bug. What I think really is the problem when a binary usually requires a nostrip in order to work, means the stripped binary has bug(s) in the code causing an exception/segfault/error. You can see already on emerge/compile, especially on the last notes of the emerge, it has "poor programming practices" or implicit declarations. Fix these and it might not require nostrip(?)
Heck. I might just hook up my serial cable to my laptop to debug grub via gdb (ref. GNU GRUB 2 Debugging HOWTO). Any C coders here that can verify, or am I it?
I've added notes to the Grub2 on gentoo-wiki.com how to manually prevent stripping. I've also made notes concerning the compiler warnings and also filed a bug report concerning the warnings to the Grub Bug tracker. (Seems useless without submitting a patch along with the bug. But at least people aren't starting from square one like I am because of people using Grub2 without writing documentation install procedures. :-/) If I am correct here about the bugs being the cause of having to use nostrip functions, and nobody wants to further debug, I'd suggest setting up an overlay. I may spend sometime today reading the Grub2 GDB debugging techniques.
you should really document the USE=multislot approach on the wiki rather than having people create binpkgs and such ...
I'll look into multislot. (I'm guessing with /etc/portage/package.use "sys-boot/grub multislot", the folders/bins are suffixed with version numbers. An excellent tip. Thanks.)
the grub binaries in /bin and /sbin do not need to be unstripped. if you thought that, it means you didnt actually research things. the STRIP_MASK behavior isnt completely sane in current portage (requires matching $D), so ive fixed portage and tweaked the grub ebuild.
Created attachment 176991 [details] grub-9999-ldflags.patch Adds ldflags removing executable stacks. Should fix requiring strip/STRIP_MASK.
I don't think we need STRIP_MASK or RESTRICT="strip" anymore. According to: http://www.gentoo.org/proj/en/hardened/gnu-stack.xml I've successfully been able to use "/sbin/grub-setup /dev/sda" using the following entry into the grub-9999.ebuild: append-ldflags -Wl,-z,noexecstack (append-flags option wouldn't allow passing configure exec tests.) Alexey, please test grub-9999-ldflags.patch against the current grub-9999.patch. Please note, the patch might not apply cleanly at all as I manually edited it. It's really only there to show you where to put the append-ldflags entry. Re-run /sbin/grub-setup /dev/sda (or whatever) & reboot. I seem to be booting just fine here with this. If it works for everybody, I might skip SLOTting and move onto something else. Did somebody close this bug? Or did I accidentlly click close??
One more thing, I'll post an updated ebuild tomorrow.
Created attachment 177031 [details] grub-9999.ebuild - Added ldflags_append option removing executable stack. - Removed (No longer requires) STRIP_MASK or RESTRICT="strip". Once commited, I suggest marking "Resolved|WorksForMe" ... as some people think it's a holiday, while I slave through the holiday night and morning working on a fix. I've tested this several times here including regressing to grub-0.97* stable series, installing to MBR, rebooting, etc. So definitely appears working here now. Again, anybody wanting more information on this bug, see: www.gentoo.org/proj/en/hardened/gnu-stack.xml (This bugs trampoline warnings during compile are associated with this bug) Have a great New Year.
this bug is about stripping and is thus fixed however, i really dont believe your ldflags option is correct. gcc itself is generating the execstack, so forcing the execstack off means it'll break.
Well, not stripping is safer. I'm filing the rest upstream. I'll also file the exec stack as a separate bug along w/ multi slotting.
g'luck with the upstream stuff ... i dont have a lot of faith it'll happen. i seem to recall the usage of nested functions is what's triggering the exec stack in the C files, and upstream was kind of set in its usage.
@#$@#$. I was greeted by the nice "Grub Rescue Mode" prompt again. lsmod only showed a few modules. As I'm no expert in Grub2, it still looks as if /sbin/grub-setup is being stripped, or one of the other bins. (Personally, I like the "append-ldflags -Wl,-z,noexecstack" fix. Obviously hitting a bug/exception with a CPU execution with executable stacks.) Well, one thing is for sure, the gentoo-wiki.com/Grub2 recovery method works a *lot* better with multislot USE Flag enabled!! (Should advise them to make a hard copy if not committed to memory.) Also, according to http://grub.enbug.org/NestedFunctions "The last option is nota real opion. It removes an useful feature to fix things. (Concerning: Removing the nested functions." If their statement is entirely correct, the only feature we are loosing when preventing executable stacks is the ability to fix grub (from what I guess, from a low-level debugger or something). As we can obviously see from this bug, nobody has yet to run GDB through a debugger. By method of elimination -- preventing executable stacks, we are probably preventing the bad code, likely contained within the nested loops from executing and causing Grub to spawn an error - hence - booting into it's rescue mode. So. We have our options. Is there any debugging I can do at this end to verify grub2's binarys are not being stripped prior to reboot? (gdb??)
Another possibility in reference to their "... to fix things." could also be referring to the Grub2 editor during boot? (To be able to fix grub.cfg from within the Grub2 boot menu?)
Spanky, some further reading on the GCC bug filed and referenced on Grub2's site: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27702 Comment #1 states, "This is not a bug in GCC but with your distro changing the rules that are always used." And, the bug further states RedHat has fixed their GCC since June. All I can say is, welcome to low-level coding -- it appears we should be pointing the finger more at Gentoo GCC maintainers for this issue concerning Trampoline (Nested Function) code?
no, there is nothing Gentoo specific about this issue. the fedora thing referenced was a change in the kernel that prevented exec stacks.
Cr*p. Even after adding "append-ldflags -Wl,-z,noexecstack" to the current ebuild within the tree, Grub2 is still going into rescue mode. (I was double checking to insure I didn't change anything in the ebuild per one of your other posts.) I need to repeat my last two builds and reboots to verify all this, since nobody else is testing. :-/ Also note, just ran into a wonderful, "die 'configure is not executable'" error. (Upstream error with svn revision 1937, odd, must've happened overnight as I thought 1937 was working.)
rebooting is painful. why dont you just use qemu ? that's what i tested with. echo 3 > /proc/sys/vm/drop_caches ; qemu -hda /dev/sda
Granted, I did a really cr*ppy job at writing up this bug on Grub Bugs, but think they comprehended my issues within entirety. (In the midst of reviving my system from glibc package event.. will comment on qemu in a day or so...) https://savannah.gnu.org/bugs/?25220 comment #5 by Vesa Jääskeläinen <chaac>: Disucssion of the executable stack issue can be found on mailing list archives and on Wiki page you already found. Issue is as is. Warnings however should be fixed. There are no such reports from elsewhere that there is compilation bug in GRUB 2 so it is somehow related to your system/distribution. Stripping should not be done to binaries as it removes all dynamic loader name references and thus GRUB 2 cannot load modules.
I can confirm this problem still persists on amd64, portage ebuild with STRIP_MASK produces a nonworking grub, which cannot even run itself during boot, not mentioning to load any modules. On the contrary, adding RESTRICT="strip" without mask works fine well. It is quite an expectable result, considering how grub2 operates on itself. Thanks, Alexey! For I really needed that one fix.
(In reply to comment #71) > I can confirm this problem still persists on amd64, portage ebuild with > STRIP_MASK produces a nonworking grub, which cannot even run itself during > boot, not mentioning to load any modules. On the contrary, adding > RESTRICT="strip" without mask works fine well. It is quite an expectable > result, considering how grub2 operates on itself. Thanks, Alexey! For I really > needed that one fix. > thats was fixed upstream. So -9999 compiles and works fine =)
So, shouldn't a new snapshot be yanked from grub-9999 SVN? ...before they break it upstream again? (It's quite common for SVN based ebuilds to go broken, and not to be fixed for many months.) ie. Yank a snapshot of the SVN version and call it grub-2.00-pre1 or something? (If I'm not mistaken, if not every platform, grun-1.96 is probably broken unless users hack in the nostrip option.)
grub-1.97 released 2009-09-04 according to http://grub.enbug.org/FrontPage it has fix to run with stripped binaryes =)
