Home | Docs | Forums | Lists | Bugs | Planet | Store | GMN | Get Gentoo!
Not eligible to see or edit group visibility for this bug.
View Bug Activity | Format For Printing | XML | Clone This Bug
This is a reference bug concerning a TODO item, please assign it to the hardened developers for further reference and documentation purposes. This bug describes the process of the migration and the paperwork for comment 103 of the following bug: http://bugs.gentoo.org/show_bug.cgi?id=94325#c103 My first inspections show this: gcc/gcc.c contains a list of existing defined specs sections. For each feature of comment 103, there needs to be a new definition in this list: #define INIT_STATIC_SPEC(NAME,PTR) \ { NAME, NULL, PTR, (struct spec_list *) 0, sizeof (NAME) - 1, 0 } /* List of statically defined specs. */ static struct spec_list static_specs[] = { INIT_STATIC_SPEC ("asm", &asm_spec), INIT_STATIC_SPEC ("asm_debug", &asm_debug), INIT_STATIC_SPEC ("asm_final", &asm_final_spec), INIT_STATIC_SPEC ("asm_options", &asm_options), INIT_STATIC_SPEC ("invoke_as", &invoke_as), ... Alex
This is from Kevin Quinn on the old bug. It's quoted here for reference. [Quote] We (in particular psm) have a range of ideas for what to do with the gcc specs themselves; including: 1) Doing bind_now/relro in ld (binutils), rather than gcc-specs (or in addition to, for transition since they don't conflict) 2) Factoring the toolchain mods in to easy-to see/maintain chunks, so that you would see for example: *cc1: %(cc1_cpu) %{profile:-p} %(cc1_ssp) %(cc1_pie) *cc1_ssp: %{D__KERNEL__|nostdlib|nodefaultlibs|fno-stack-protector|fstack-protector|fno-stack-protector-all:;:-fstack-protector-all} *cc1_pie: %{D__KERNEL__|static|nostdlib|nostartfiles|fPIC|fpic|fno-PIC|fno-pic|fPIE|fpie|fno-PIE|fno-pie|nopie:;:-fPIE} *link_command: %{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S: %(linker) %l %{pie:-pie;:%(link_command_pie)} %X %{o*} %{A} %{d} %{e*} %{m} %{N} %{n} %{r} %{s} %{t} %{u*} %{x} %{z} %{Z} %{!A:%{!nostdlib:%{!nostartfiles:%S}}} %{static:} %{L*} %(mfwrap) %(link_libgcc) %o %(mflib) %{fprofile-arcs|fprofile-generate|coverage:-lgcov} %{!nostdlib:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}} %{!A:%{!nostdlib:%{!nostartfiles:%E}}} %{T*} }}}}}} *link_command_pie: %{D__KERNEL__|static|A|nostdlib|nostartfiles|fPIC|fpic|fno-PIC|fno-pic|fPIE|fpie|fno-PIE|fno-pie|nopie:;:-pie} (gcc specs still did bind-now/relro it would be done as new "link_command_zrelro" and "link_command_znow" definitions, referenced from the appropriate place in the link_command definition) This means that the switched specs files no longer have to be full sets - they can be just the modifications necessary; i.e. they just need to define cc1_ssp, cc1_pie and link_command_pie. Note that the GCC_SPECS variable can now contain more than one entry; e.g. one could do: GCC_SPECS="default_ssp.specs:default_pie.specs" gcc ... where default_ssp.specs would contain the cc1_ssp definition and default_pie.specs would be the cc1_pie definition. There are pros and cons to having the builtin specs themselves hardened, still undecided on that. [/Quote]
My open/undecided issues are (I have all of this already implemented for gcc-3.4.6/4.1.1 - well, for 4.0.x too, but I wouldnt support it due to additionally needed ssp patches): a. what do we do with fno-stack-protector-all, do we patch gcc to support it or leave it as it is now in gcc-4.1, last option applies, so if -fstack-protector is last, after an -fstack-protector-all, the non-all version wins. Independently of which path we go, I would like to keep gcc-3.x and 4.x in sync, so the hardened profiles work the same way (easier to handle in toolchain.eclass that way) b. should gcc itself be built as "hardened", or we leave it vanilla in any case, adding only the new specs c. do we go with GCC_SPECS supporting more specs files and provide mini-specs for each functionality (pie/ssp/ssp_all/relro/now), adding all the needed ones to the list of GCC_SPECS (havent looked to eselect if supported) or we provide combined "maxi"-specs to have the same profiles as earlier (the latter works probably with eselect) d. the most disturbing change in gcc-4.1 is, that we lost the info about where the stack smashing occured. Are we willing to stay like that, or we patch gcc to provide the function failing as well (that would make gcc-vanilla not quite vanilla) Before I provide the patches, I would like to have the answers to the questions above. Note: due to the changes I did pie/ssp detection used in ebuilds does not work anymore, needs adaption as well
(In reply to comment #2) > a. what do we do with fno-stack-protector-all My preference is to leave gcc-4.1 unchanged (i.e. don't add -fno-stack-protector-all, leave the last-setting-wins in place), just tweak the filter-flags() function in flag-o-matic() to do the right thing according to gcc version (and possibly also append-flags() ). My arguments are: 1) it's unlikely that anyone actually uses '-fno-stack-protector-all', except when trying to undo the hardened specs - which should be done via filter-flags() anyway (any ebuilds that don't use flag-o-matic for this can be fixed). 2) as a general aim, I think it's a good idea to try to keep gcc as close to upstream as possible. Ideally there should be no impact on the vanilla compiler from hardened (but see (b) below). > Independently of which path we go, I would like to keep gcc-3.x and 4.x in > sync, so the hardened profiles work the same way (easier to handle in > toolchain.eclass that way) I'm not worried about that. We probably have to leave the current 3.x management in toolchain.eclass anyway, due to the long-standing issue of never removing functionality from an eclass. My approach to differences between the two major versions would be to handle that in eclasses (i.e. abstract it from ebuilds). If we leave the 3.x stuff as it stands, it makes it easier to detect which type of handling is present - just a check on gcc major version - whereas if we go back and modify the older toolchains, we need to do more run-time detection to work out which way the older toolchain was built. > b. should gcc itself be built as "hardened", or we leave it vanilla in any > case, adding only the new specs The case for having the compiler itself built PIE/SSP is relatively weak, I think (unlike glibc, where a stack overflow could be exploited by any application that links to it, including suid apps). However there are the support libraries, which I think should be SSP and (for the .a's) PIE. Ah; if you're talking about whether the builtin specs should be hardened or not - I would say no (i.e. leave the builtin specs vanilla, apart from the minispec callouts). However others make a case for having the built-in specs hardened on a hardened machine. Not building in the hardened minispecs means the patch to gcc itself becomes much smaller (just adds the call-outs to the minispecs, and default vanilla minispecs for some items), and we no longer have to build xgcc once for each spec combination (see toolchain.eclass). > c. do we go with GCC_SPECS supporting more specs files and provide mini-specs > for each functionality (pie/ssp/ssp_all/relro/now), adding all the needed ones > to the list of GCC_SPECS (havent looked to eselect if supported) or we provide > combined "maxi"-specs to have the same profiles as earlier (the latter works > probably with eselect) eselect-compiler supports multiple specs in GCC_SPECS (that was added a long while back). It's gcc-config that still needs to be updated (bug #125805 - waiting for vapier to get around to it). I prefer the mini-specs approach (that's what I have implemented locally). Note also that for bind-now, relro, I'd like to consider doing that in binutils (by changing the defaults in ld itself) rather than gcc specs. In the end both can co-exist, especially easily with minispecs, so maybe that's something we can leave for later. > d. the most disturbing change in gcc-4.1 is, that we lost the info about where > the stack smashing occured. Are we willing to stay like that, or we patch gcc > to provide the function failing as well (that would make gcc-vanilla not quite > vanilla) My opinion here is that we follow upstream. As pageexec said elsewhere, once a stack smash occurs, it can be tracked down more effectively via gdb (which reminds me - need to rig the stack smash handler to cause a core dump, which it doesn't at the moment). On the (slight) plus side, without the extra parameters to the stack smash handler, there is less further modification of the stack between the smash detection and the abort. > Before I provide the patches, I would like to have the answers to > the questions above. > > Note: > due to the changes I did pie/ssp detection used in ebuilds does not work > anymore, needs adaption as well Yes; I'm expecting that (tweaks to toolchain-funcs.eclass, and possibly ebuilds that try to do things directly although there shouldn't be many of those).
i am following Kevins thoughts on the source building of gcc and glibc. My thoughts on the modularized/single specs: The specs file is the guts and gore of gcc. Nobody likes to mess with it, and most users don't even know about it. And if they know about it, i don't think it would make a big difference to have ONE big file or MANY small files. My opinion: if you are into specs editing, you don't care. For a source building distribution like Gentoo, i would estimate that 20% to 33% of all tasks in the distribution is about compiling new applications and updates for security advisories and feature bugs. This situation makes the compiler an important part of the acceptance of the distribution as a whole thing. Now, i am thinking: for runtime performance of the gcc compiling these applications my concerns are that opening and parsing several small files especially with MAKEOPTS=-j20 and thus having shitloads of gcc instances running in with small shifts of workloads, the modularized specs effort is taking more disk i/o and runtime performance than just sucking in one big file and parsing it internally. This could be totally neutralized by hard disk performance and caches, but it's just my thoughts and i hope i'm not absolutely wrong. Having the PATCHES modularized eases maintenance and is good for devs. Having the specs modularized is unnecessary and can lead to increase the compilers internal overhead and thus have an impact on the workload of the compiler itself in my eyes. Peter: it would be nice if you could attach the modularized gcc-4.1.1 patches here for our users to be able to start testing (nixnut was asking for PPC specifically today and i am refusing to repeat your already finished work by mocking up my preliminary "shots" into the 4.1.1 direction) Thank you and have fun! Alex
my bad, I have attached them to #149292
I don't think the cost of opening a few specs files is significant, even for a machine that did nothing but compilation all day. Either way, merging the minispecs into one file is trivial, and can be done by the ebuild (well, toolchain.eclass). The main reason for the minispecs is not so much the minispecs themselves, but the idea that the specs modifications are implemented with call-outs to new spec names, which makes it easier to override, and to see what the modifications are. For reference/comparison, you can find my toolchain mods in my overlay at http://overlays.gentoo.org/dev/kevquinn. They're not finished so I won't attach them here. Currently gcc itself doesn't get built hardened, but that I'm planning on doing by setting GCC_SPECS during the build. I also haven't split out the startfile/endfile stuff yet. The core difference between what mine (will eventually) do, and what Peter's patches do, is that with my approach the compiler is built with (effectively) vanilla specs built-in for everyone (so hardened users always have GCC_SPECS set to the hardened minispecs), whereas Peter continues the approach of having different builtin specs for hardened and vanilla users. FWIW my patching of gcc itself should end up being somewhat smaller, as it won't need to do the EFAULT_PIE/SSP etc stuff.
kevquinn: I have reread and reread your posting: You can achieve what you want (having vanilla specs builtin by doing it in toolchain.eclass only, without touching the piepatch tarballs, less the needed change to be compliant with the gentoo used uclibc patch in 4.1.1). I had in mind keeping compatibility with current behaviour as much as possible (sspall.specs has to have %{!fno-stack-protector-all:-fstack-protector-all} for gcc-3.4.x), you definitely go another route. If you change the logic of the piepatches, I am asking you politely to change the patch tarballs name and versioning style, document in changelog the fact that it is your version and not mine.
Peter; it's just a discussion point at the moment - whether to integrate the hardened specs into the gcc driver, or whether to have them outside. There are pros and cons to both approaches, but I feel that if we have both actually implemented we can make an informed choice (same goes for building libc with ssp-all throughout, bug #94325, which affects whether we have the _LIBC/_LIBC_REENTRANT guard for ssp-all). I've installed your version into a new directory in my dev overlays area, so it's easier for interested parties to try it out: svn co http://overlays.gentoo.org/kevquinn/hardened/toolchain-psm The only differences are the patch tweak I mentioned before, and I've moved the specs files into a specs subdirectory, to keep the sys-devel/gcc/files directory relatively clean, and obviously I've added the files needed for a gcc-4.1.1-r1 ebuild.
_LIBC* check was added because glibc failed to build, uClibc has an own SSP config option to allow compile all of it at least with -fssp (-all is used for all lib*.so, less libc.so), you can remove those anytime after you get glibc fixed. There is also a patch for kernel, so the kernel check could go too (though it is not proven that it is useful there at all).I personally want to keep building gcc itself fully hardened, because it is a good test to see if something is breaking on the way, before you installed a broken gcc. As an example: I know that something broke between gcc-4.1.1 and -r1, because I do generation of the specs (as it is done in gcc-3.4) SPLIT_SPECS is enabled. I know I do not need to do it anymore, due to the minispecs, but doing it early enough, I know that this version of gcc is expected to misbehave.
forgotten to mention, I should have rather said: not gcc-4.1.1 itself is broken, some change in toolchain.eclass around that time did it
> I've installed your version into a new directory in my dev overlays area, so > it's easier for interested parties to try it out: > > svn co http://overlays.gentoo.org/kevquinn/hardened/toolchain-psm # svn co http://overlays.gentoo.org/kevquinn/hardened/toolchain-psm svn: PROPFIND request failed on '/kevquinn/hardened/toolchain-psm' svn: PROPFIND of '/kevquinn/hardened/toolchain-psm': 405 Method Not Allowed (http://overlays.gentoo.org) Kevin, i will try to fetch your work via http, hope it goes well. Alex
19 svn checkout http://overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain 20 svn checkout http://overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain-psm this one did the trick for both overlays a good introduction how to add and use overlays: http://overlays.gentoo.org/dev/wschlich -> Getting started There you can learn about using layman, but svn is cool for me too... Alex
the move of -z now and relro to binutils is fine with me, it is more logical and will do it all the time (not only if invoked by gcc). binutils-2.16.1-r3 has already the -z lazy patch, norelro was already there, so this could be a first step, updating 2.16.1-r3/2.17[.50.0.5], I wouldnt care about others. The eclass needs the handling of -z lazy. kevquinn: your 2.17 is wrong s/bfd_dma/bfd_vma/ and you missed including the needed header (as in 2.16)
* The current gcc config appears valid, so it will not be * automatically switched for you. If you would like to * switch to the newly installed gcc version, do the * following: * gcc-config x86_64-pc-linux-gnu-4.1.1 * source /etc/profile # gcc-config x86_64-pc-linux-gnu-4.1.1 * /usr/bin/gcc-config: Could not locate 'x86_64-pc-linux-gnu-4.1.1' in '/etc/env.d/gcc/' ! 13:31:01 [/space/chroots/chroot001:19137.pts-5.miranda]miranda /master/svn/overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain/sys-devel/gcc # env-update >>> Regenerating /etc/ld.so.cache... 13:31:07 [/space/chroots/chroot001:19137.pts-5.miranda]miranda /master/svn/overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain/sys-devel/gcc # gcc-config x86_64-pc-linux-gnu-4.1.1 * /usr/bin/gcc-config: Could not locate 'x86_64-pc-linux-gnu-4.1.1' in '/etc/env.d/gcc/' ! 13:31:09 [/space/chroots/chroot001:19137.pts-5.miranda]miranda /master/svn/overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain/sys-devel/gcc # source /etc/profile miranda gcc # gcc-config x86_64-pc-linux-gnu-4.1.1 * /usr/bin/gcc-config: Could not locate 'x86_64-pc-linux-gnu-4.1.1' in '/etc/env.d/gcc/' ! miranda gcc # cd /etc/env.d/gcc miranda gcc # ls config x86_64-pc-linux-gnu-4.1.1-hardened x86_64-pc-linux-gnu-3.4.4 x86_64-pc-linux-gnu-4.1.1-hardenednopie x86_64-pc-linux-gnu-3.4.4-hardenednopie x86_64-pc-linux-gnu-4.1.1-hardenednopiessp x86_64-pc-linux-gnu-3.4.4-hardenednopiessp x86_64-pc-linux-gnu-4.1.1-hardenednossp x86_64-pc-linux-gnu-3.4.4-hardenednossp x86_64-pc-linux-gnu-4.1.1-vanilla x86_64-pc-linux-gnu-3.4.4-vanilla miranda gcc # gcc-config x86_64-pc-linux-gnu-4.1.1-hardened * Switching native-compiler to x86_64-pc-linux-gnu-4.1.1-hardened ... [ ok ] The message at the end of the ebuild is a bit "erroneously" because it points to a profile which does not exist... Maybe someone can change this. Alex
Hello again, specs modularization is working with Kevins ebuilds and patches. Using built-in specs. Reading specs from /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/hardened.specs Reading specs from /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/pie.specs Reading specs from /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/ssp.specs Reading specs from /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/zrelro.specs Reading specs from /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/znow.specs /usr/libexec/gcc/x86_64-pc-linux-gnu/4.1.1/cc1 -quiet -v vuln-stack.c -fstack-protector-all -fPIE -quiet -dumpbase vuln-stack.c -mtune=k8 -auxbase vuln-stack -version -o /tmp/ccPYgID3.s /usr/libexec/gcc/x86_64-pc-linux-gnu/4.1.1/collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -o vuln-stack /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/crtbeginS.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../.. /tmp/ccalyi1V.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/crtn.o The assembler step has been left out and some other debug info too. But there is one "probleM" i have with the current setup. gcc -dumpspecs shows only the internal specs. A good thing however is: when GCC_SPECS is unset, the gcc uses internal specs and skips the undefined portions. Nonetheless the problem with gcc -dumpspecs i vote YES to the modularized specs and hope we can introduce the glibc changes to glibc-2.5 (vapiers mail to gentoo-dev and his private message we should not work on glibc-2.4 any more) and the specs patches to gcc-4.1.1. Peter and Kevin: thanks for your work! Alex
/var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c: In function 'main': /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c:314: error: 'bfd_dma' undeclared (first use in this function) /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c:314: error: (Each undeclared identifier is reported only once /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c:314: error: for each function it appears in.) /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c:314: error: expected ';' before 'DF_BIND_NOW' /var/tmp/portage/binutils-2.17/work/binutils-2.17/ld/ldmain.c:315: error: expected ';' before 'DF_1_NOW' make[4]: *** [ldmain.o] Error 1 make[4]: Leaving directory `/var/tmp/portage/binutils-2.17/work/build/ld' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/var/tmp/portage/binutils-2.17/work/build/ld' make[2]: *** [all] Error 2 make[2]: Leaving directory `/var/tmp/portage/binutils-2.17/work/build/ld' make[1]: *** [all-ld] Error 2 make[1]: Leaving directory `/var/tmp/portage/binutils-2.17/work/build' make: *** [all] Error 2 !!! ERROR: sys-devel/binutils-2.17 failed. Call stack: ebuild.sh, line 1546: Called dyn_compile ebuild.sh, line 937: Called src_compile ebuild.sh, line 1255: Called toolchain-binutils_src_compile toolchain-binutils.eclass, line 215: Called die !!! emake failed !!! If you need support, post the topmost build error, and the call stack if relevant. !!! This ebuild is from an overlay: '/master/svn/overlays.gentoo.org/svn/dev/kevquinn/hardened/toolchain' binutils-2.17 broken as Peter Mazinger already told in a previous comment. -Alex
Created an attachment (id=99045) [edit] portage-hardened-glibc-gcc.patch emerge sync; cd /usr/portage; patch -p1 --dry-run < portage-hardened-glibc-gcc.patch -Alex
(From update of attachment 99045 [edit]) this patch is missing some files from Kevins overlay.
Created an attachment (id=99055) [edit] portage-hardened-glibc-gcc.patch with needed files and ppc tryout
Kevin, is it possible to move the work of your patch from gcc/gcc.c (where the *_SPEC definitions are overriden) completely to the configure logic? As far as i understood it, the "tm_file" in config.gcc merges the architecture specific pieces of specs with the global specs in gcc/gcc.c. So maybe we can get rid of editing the direct gcc files altogether and add some stuff to the configure scripts. Thanks, Alex
Alex - I updated the binutils-2.17 earlier today, after psm noted the errors, although I haven't built it yet. Obviously I put the wrong stuff into svn before... Re. gcc -dumpspecs - it is defined (see gcc info) to show the built-in specs only, and changing that would be a bad idea, IMHO. In toolchain-funcs.eclass you can find a simple bit of shell+awk that can parse specs. I've just updated svn for my gcc style, which now builds hardened as it goes by setting GCC_SPECS for the build process. It also creates the file "specs", which is unconditionally read by the compiler if present, and replaces the built-in specs. At this point the difference for the user between my way and Peter's becomes ever smaller; the only real difference now (besides integration of the uclibc stuff, which I haven't put into my toolchain.eclass) is that Peter's has callouts for startfile/endfile, whereas I just supply completely new ones in my minispecs. My intention there was to supply target-specific versions as necessary in the ebuild's files directory. Lastly, I have not been able to reproduce the stack smash in ix86_split_to_parts() (bug #149292) - the patching to inhibit the stack protector on the code that smashes is commented out.
(In reply to comment #20) > Kevin, is it possible to move the work of your patch from gcc/gcc.c (where the > *_SPEC definitions are overriden) completely to the configure logic? I'm not sure; it would be nice if we could. Assuming we could just create (say) config/gentoo-hardened.h and add it to tm_file, so that it is parsed last in the list of target macro files, it would be possible provided we can append to existing macros. You'll notice in my patch I define a new macro CC1_SPEC_2 to be CC1_SPEC appended with the callouts, and use that on the cc1 definition. I originally tried to do this by doing things like: #define PREV_CC1_SPEC CC1_SPEC #undef CC1_SPEC #define CC1_SPEC PREV_CC1_SPEC " %(cc1_ssp) %(cc1_pie) " but macros don't work like that. I couldn't find a way to do it.
yeah, bad news: it would theoretically possible to manage the configure.in with --enable-autopie and --enable-autossp and then check for it in config.gcc and add the config/hardened/hardened.h to the tm_file macro expansion list. But this would mean a lot of code rework in the gcc source. which results in a larger patch. I will have to think about the whole thing for a bit. But in the meantime i am happy with the patch as it is at the moment. -Alex
the syntax %{pie:-pie;:%(anything)} is not valid on all gcc's, you have to %{pie:-pie} %{!pie:%(anything)}
it is possible, but you will have to work harder to support uclibc as well
Alex, gcc -dumpspecs did return the builtin specs even in your times, it was never returning the really used specs Kevin, I have intentionally created startfile_pie/crtfile_pie/endfile_pie/asm_pie to get rid of the fact that each arch has its own version of these sections, not speaking of the fact, that some archs do not use the generic section names, they have specific linux versions you have to overwrite too. The problem does not occur with ssp, there we can append to CC1_SPEC. Note: On x86 and x86_64 for gcc-4.1.1 my patch is broken (correct only for 4.1.0) I have already created a new one, but I would update 9.0.4 by also removing znow/zrelro from specs. Opinions?
to reproduce the ssp bug, use gcc-3.4.6 hardened profile and build a vanilla or hardened gcc-4.1 (where DEFAULT_SSP & DEFAULT_PIE are set)
Peter: yeah, the ix86_split_to_parts error was also reproduced by me. Tomorrow i will try to put together some "best parts of both worlds" patch which is orientated at Kevins approach of "minispecs" containing the emitters for the commandlines but at the same time taking care of Peters work for uclibc compatibility and multiple platform support (read: sparc and ppc). Pijn, nixnut and Weedy (my favourite guinea pigs for test requests, along with the chroots on tinder and miranda) have been working the last two days testing a preliminary adapted version of Kevins glibc patches and gcc minispecs on x86, amd64 and today ppc, and so far things are looking good. The versions of the ebuilds used for integration and testing are glibc-2.5 and gcc-4.1.1-r1. There are some minor problems with the toolchain eclass regarding the message about switching to the new gcc profile and on ppc the ebuild is not copying over the specs files to the target directory of the emerged gcc-4.1.1-r1. Also, the warning message with the aliased function appears on ppc, so maybe there is an #ifdef ppc missing in the code for the glibc gcc-3.* SSP compatibility patch. Good job and good luck with the next steps of the migration, Alex
(In reply to comment #24) > the syntax %{pie:-pie;:%(anything)} is not valid on all gcc's, you have to > %{pie:-pie} %{!pie:%(anything)} I figured it would be valid for all versions 3.4.6 and above - is that not the case?
I would have used that syntax in my patches as well, cant recall now when it failed
(In reply to comment #26) > Kevin, I have intentionally created > startfile_pie/crtfile_pie/endfile_pie/asm_pie to get > rid of the fact that each arch has its own version of these sections, not > speaking > of the fact, that some archs do not use the generic section names, they have > specific linux versions you have to overwrite too. The problem does not occur > with > ssp, there we can append to CC1_SPEC. Yeah; I see that. I was going to deal with that by having different minispecs for each arch, as necessary, either held in ${FILESDIR}, or as a separate tarball, laid out so: ${FILESDIR}/specs/*.specs - most common minispecs ${FILESDIR}/specs/${PV}/*.specs - common minispecs for a specific GCC version ${FILESDIR}/specs/${ARCH}/*.specs - minispecs for a specific arch ${FILESDIR}/specs/${ARCH}/${PV}/*.specs - minispecs for a specific arch and GCC version Then the installation of specs is simply to copy from each location in order, ensuring that the more specific versions overwrite the more generic versions. The intention was to minimise as much as possible the need to patch gcc itself. Thinking about it, it could all be done this way, and so have no patching of gcc itself at all - however it becomes clear that it would be easier to maintain the patches to gcc as Peter has proposed (not least to support uclibc as well). > Note: On x86 and x86_64 for gcc-4.1.1 my patch is broken (correct only for > 4.1.0) > I have already created a new one, but I would update 9.0.4 by also removing > znow/zrelro from specs. Opinions? I need to think carefully about that. We need to ensure the upgrade path works properly; I think it can be done by having DEPENDs conditional on USE=hardened, but it would involve creating stable rev-bumps of binutils, taking into consideration the various supported hardened arches. My feeling at the moment is that it will be simpler to run with both simultaneously for a while, then drop them from gcc perhaps for 4.2 (by which time binutils with the changed defaults should be stable where necessary).
I am afraid, that if we do not patch gcc, a careless update of gcc won't catch all the needed changes (patching gcc would fail at least maybe, telling that something has to be adapted). Copying minispecs will always work, but won't be compliant with each version, anyone updating gcc will have to carefully read what was updated and we have to keep minispecs for different versions and archs , my approach eliminates this need, we need only one set of minispecs for all (even so 4.1.0->4.1.1 added ENDFILE_SPEC to i386/linux.h and linux64 - this is the failure I have encountered - for 4.1.0 the ENDFILE_SPEC for these 2 configs came from the generic one, as result I ended up using crtend.o instead of crtendS.o for PIE, resulting in fake TEXTREL Kevin: have you tested if providing GCC_SPECS for the stage builds results in building all the stages in gcc with pie/ssp enabled. If that is working (as I stated I want to keep the possibility to compile gcc throughout with hardened) then DEFAULT_PIE & SSP can be removed from the patches.
(In reply to comment #32) > Copying minispecs will always work, but won't be > compliant with each version, anyone updating gcc will have to carefully read > what was updated and we have to keep minispecs for different versions and archs > , my approach eliminates this need, we need only one set of minispecs for all That's a good point. I was already coming to the conclusion that maintaining minispecs for an unpatched gcc would be more work than maintaining your patches - I guess that settles it :) > Kevin: have you tested if providing GCC_SPECS for the stage builds results in > building all the stages in gcc with pie/ssp enabled. If that is working (as I > stated I want to keep the possibility to compile gcc throughout with hardened) > then DEFAULT_PIE & SSP can be removed from the patches. Yes, everything got built with ssp. A sample: # readelf -a ./gcc/stage1/tree-ssa-phiopt.o | grep stack [11] .note.GNU-stack PROGBITS 00000000 0028f8 000000 00 0 0 1 0000048d 00002e04 R_386_PLT32 00000000 __stack_smash_handler 000009d3 00002e04 R_386_PLT32 00000000 __stack_smash_handler 00000aed 00002e04 R_386_PLT32 00000000 __stack_smash_handler 00001122 00002e04 R_386_PLT32 00000000 __stack_smash_handler 00001659 00002e04 R_386_PLT32 00000000 __stack_smash_handler 0000187f 00002e04 R_386_PLT32 00000000 __stack_smash_handler 000021e2 00002e04 R_386_PLT32 00000000 __stack_smash_handler 00002591 00002e04 R_386_PLT32 00000000 __stack_smash_handler 000025d8 00002e04 R_386_PLT32 00000000 __stack_smash_handler 46: 00000000 0 NOTYPE GLOBAL DEFAULT UND __stack_smash_handler # readelf -a ./gcc/stageprofile/tree-ssa-phiopt.o | grep stack [30] .note.GNU-stack PROGBITS 00000000 014dbf 000000 00 0 0 1 00000043 00001e02 R_386_PC32 00000000 __stack_chk_fail_local 00000185 00001e02 R_386_PC32 00000000 __stack_chk_fail_local 000007db 00001e02 R_386_PC32 00000000 __stack_chk_fail_local 00001461 00001e02 R_386_PC32 00000000 __stack_chk_fail_local 00002d41 00001e02 R_386_PC32 00000000 __stack_chk_fail_local 30: 00000000 0 NOTYPE GLOBAL DEFAULT UND __stack_chk_fail_local # readelf -a ./gcc/tree-ssa-phiopt.o | grep stack [30] .note.GNU-stack PROGBITS 00000000 0126c6 000000 00 0 0 1 00000209 00001a02 R_386_PC32 00000000 __stack_chk_fail_local 00000001 00001a02 R_386_PC32 00000000 __stack_chk_fail_local 00000036 00001a02 R_386_PC32 00000000 __stack_chk_fail_local 00000109 00001a02 R_386_PC32 00000000 __stack_chk_fail_local 26: 00000000 0 NOTYPE GLOBAL DEFAULT UND __stack_chk_fail_local (the different number of calls is down to differences in the way the optimisation varied; there are fewer functions in the final 4.1 object - I've checked the object disassembly and all functions are protected, in all cases). In toolchain.eclass, I installed the minispecs into the build area, concatenated them to create "${WORKDIR}"/build.specs which I set in GCC_SPECS: concat_minispecs() { local entry spec while read entry; do spec=${entry/\%include <} if [[ ${spec} == ${entry} ]]; then printf "%s\n" "$entry" >> $2 else spec=${spec/>} cat "${FILESDIR}"/specs/${spec} >> $2 fi done < ${FILESDIR}/specs/${1}.specs } ... gcc_src_compile() { gcc_do_filter_flags einfo "CFLAGS=\"${CFLAGS}\"" einfo "CXXFLAGS=\"${CXXFLAGS}\"" if use hardened ; then if hardened_gcc_works ; then concat_minispecs hardened "${WORKDIR}"/build.specs elif hardened_gcc_works pie ; then concat_minispecs hardenednossp "${WORKDIR}"/build.specs elif hardened_gcc_works ssp ; then concat_minispecs hardenednopie "${WORKDIR}"/build.specs else concat_minispecs hardenednopiessp "${WORKDIR}"/build.specs fi export GCC_SPECS="${WORKDIR}"/build.specs fi ... This was where 'hardened.specs' etc contain just '%include <xxx>' references to the other minispecs. Note; %include is not permitted in built-in specs, nor in the file /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/specs etc, but it's fine in anything set in GCC_SPECS.
okay, i think the best thing would be to take the best from both solutions: 1) gcc patch: adding minispecs for callouts to PIE and SSP logic the only "top level" callout added in gcc/gcc.c should be the callout for SSP in CC1_SPECS the patch for adding the top level callout to CC1_SPECS can be called gcc-ssp-callout.patch and kept minimal intrusive. the rest of the PIE callouts should ONLY be added arch dependent in the respective subdirectories of config/<arch>/<arch>.h 2) gcc patch: branding the version to show a hardened flavour the patch can be named gcc-hardened-branding.patch 3) toolchain eclass: manipulating the GCC_SPECS env var 4) toolchain eclass: adding additional specs to GCC_SPECS list for callouts The toolchain eclass will create corresponding gcc-config profiles and the correct GCC_SPECS environment variables containing a list of specs files in the order of priority. The specs files for resolving the callouts will resemble the behaviour, either nondefault or default hardened PIE SSP. This is managed by setting the appropriate GCC_SPECS env var with the gcc-config profile. The respective patches for the SSP callout in the gcc/ top level dir and PIE callouts in the arch dependent sub directories of the gcc/config/ dir can be triggered depending on - hardened use flag - libc version (do we have a glibc use flag?) - architecture use flag if use hardened && use glibc; then epatch XXX/gcc-4.1.1-ssp-noarch-glibc-callout.patch epatch XXX/gcc-hardened-noarch-glibc-branding.patch if use x86; then epatch XXX/gcc-pie-x86-glibc-callout.patch else if use amd64; then epatch XXX/gcc-pie-x86-glibc-callout.patch fi fi I would like to have a meeting in #gentoo-hardened-dev at the beginning of next week to coordinate the efforts of our team because i am getting the feeling there is a bit of a lack of communication here. I will work out some test patches resembling this scheme now. Thanks, Alex
as i can see from the eclass, branding is done without a special hardened patch applied. but at least these parts of the original toolchain eclass have to be rewritten because the concept is undergoing changes with the minispecs approach: # corrects startfile/endfile selection and shared/static/pie flag usage EPATCH_MULTI_MSG="Applying upstream pie patches ..." \ epatch "${WORKDIR}"/piepatch/upstream # adds non-default pie support (rs6000) EPATCH_MULTI_MSG="Applying non-default pie patches ..." \ epatch "${WORKDIR}"/piepatch/nondef # adds default pie support (rs6000 too) if DEFAULT_PIE[_SSP] is defined EPATCH_MULTI_MSG="Applying default pie patches ..." \ epatch "${WORKDIR}"/piepatch/def
One other issue, is -fno-stack-protector-all. Since it's a '-f' switch, it gets rejected by gcc-4. We could add it to gcc-4, but I would prefer to avoid that. I suggest we do something like this: cc1_ssp: %{!nostdlib:%{!nodefaultlibs:%{!fno-stack-protector:%{!fstack-protector:%{!fstack-protector-all:-fstack-protector %(cc1_ssp_all) }}}}} cc1_ssp_all: %{!fno-stack-protector-all:-fstack-protector-all} The '-fno-stack-protector-all' in cc1_ssp_all is ineffective but causes no problems on gcc-4. The adjustment to cc1_ssp ensure that any requested setting of stack-protector does not get overridden by the specs. With regards to glibc - I intend either to get it building ok with ssp-all, or to adjust the glibc build process to switch off ssp where it needs to. Thus we should be able to drop the D_LIBC/D_LIBC_REENTRANT conditions.
(In reply to comment #34) > I would like to have a meeting in #gentoo-hardened-dev at the beginning of next > week to coordinate the efforts of our team because i am getting the feeling > there is a bit of a lack of communication here. I feel we're progressing just fine. We've explored various possibilities, which has helped us understand things better - discarded some ideas, developed others. We've decided we'll go with Peter's style of patches, hopefully with the GCC_SPECS approach to building if that works out ok. I guess, Alex, you're perhaps at a bit of a disadvantage. Peter and I came up with the minispecs idea some months ago; we've worked at it independently on and off for some time now so we understand a lot about the issues it raises. Perhaps you could put together a list of what you think should be discussed at the meeting (a meeting agenda, if you like). I'd prefer to deal with that here rather than IRC, though. Lastly, I'm not going to rush this. I've mentioned to people before that we hope to have stuff in a decent shape in the tree by the end of the year. The switch from gcc-3/glib-2.3.6 to gcc-4/glibc-2.4 is a time to introduce significant changes, which is what we're considering, and I don't want to be changing things significantly again after that.
vapier says we should work on glibc-2.5 glibc-2.4 is fixed according his plans (see gentoo-dev ML) Alex
Alex, I am not in a hurry either and don't want to rush ... I am using these changes since about May, but I am not quite happy with it uClibc has support for gcc4-style ssp only in svn, the glibc implementation (haven't looked though lately because I do not have any glibc based box running) probably needs also some changes. Kevin: I will do some testing with your GCC_SPECS use in eclass, if that goes fine, I can remove all the DEFAULT_* from my patches and leave the handling exclusively to the eclass. Do you need earlier the changes needed for x86[_64]?
Kevin: I diffed your toolchain.eclass, you missed a '\' after -n ${PIE_VER} I have added -DTARGET_LIBC_PROVIDES_SSP to *_CFLAGS to be sure that I have it the configure check does a simplistic check for glibc-2.4 (checking for GLIBC_MAJOR/MINOR that I didnt liked and I am not sure how good that will be in a cross-compiled env. I have attempted to discuss this on gcc-ml to generalize the check (really doing a check in ld.so for __stack_chk_guard and looking for __stack_chk_fail in libc, haven't gotten any answer from there. I recall having some problem with the sed expression editing LIBGCC2_CFLAGS because the line breaks and -fno-stack-protector was appended after the trailing '\' Maybe someone else could also try to provide the piepatches/upstream stuff to the gcc list, less one of them they are real corrections for the case HAVE_LD_PIE is not defined - true only for really old binutils - , but were discarded by JJ, saying that 1-2 cycles more in processing do not matter. Does anyone know if it is expected to have boundschecking patches for gcc-4.1? by JJ,
I wanted the meeting because i'm lost in your ideas and don't know how we can solve the problems together. My goal is to get 2007.0 hardened stages and a livecd perhaps. But for this we need to get the stuff in late october or at least november to have enough time for the ~arch -> arch move. As far as it goes for my research, changes to the following sections can be outsourced to callouts to specially named sections in external specs files: cc1_specs (needs stuff for SSP and PIE added) asm_specs (needs stuff for PIE added on some arches like sparc and ppc) link_pie_specs (needs stuff for PIE added but leaving the original intact when we carefully craft the trigger list with respect to the existing %{pie:-pie}) because of the complexity of startfile and endfile we need to completely replace startfile and endfile for the given situations: - different arches - different libc types - different model (default PIE, nondefault PIE) Now,my idea is to tackle it this way: adding a header framework to tm_file: gcc/config.gcc:tm_file="gentoo-hardened/hardened.h ${cpu_type}/${cpu_type}.h" then adding the "empty" definition fallbacks to gcc.c and the sections prepared for overwriting gcc/gcc.c: /* Gentoo Hardened appends extra args to the ASM_SPECS */ #ifndef GENTOO_HARDENED_ASM_SPEC #define GENTOO_HARDENED_ASM_SPEC "" #endif /* Gentoo Hardened appends extra args to the CC1_SPECS */ #ifndef GENTOO_HARDENED_CC1_SPEC #define GENTOO_HARDENED_CC1_SPEC "" #endif /* Gentoo Hardened definitions may overwrite the default specs */ #ifdef GENTOO_HARDENED_STARTFILE_SPEC #undef STARTFILE_SPEC #define STARTFILE_SPEC GENTOO_HARDENED_STARTFILE_SPEC #endif /* Gentoo Hardened definitions may overwrite the default specs */ #ifdef GENTOO_HARDENED_ENDFILE_SPEC #undef ENDFILE_SPEC #define ENDFILE_SPEC GENTOO_HARDENED_ENDFILE_SPEC #endif /* Gentoo Hardened definitions appended to LINK_PIE if LD_PIE is available */ #ifndef HAVE_LD_PIE #undef GENTOO_HARDENED_LINK_PIE_SPEC #define GENTOO_HARDENED_LINK_PIE_SPEC "" #else #ifndef GENTOO_HARDENED_LINK_PIE_SPEC #define GENTOO_HARDENED_LINK_PIE_SPEC "" #endif #endif #ifndef LINK_COMMAND_SPEC #define LINK_COMMAND_SPEC "\ %{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S:\ %(linker) %l " LINK_PIE_SPEC GENTOO_HARDENED_LINK_PIE_SPEC " %X %{o*} %{A} %{d} %{e*} %{m} %{N} %{n} %{r}\ %{s} %{t} %{u*} %{x} %{z} %{Z} %{!A:%{!nostdlib:%{!nostartfiles:%S}}}\ %{static:} %{L*} %(mfwrap) %(link_libgcc) %o %(mflib)\ %{fprofile-arcs|fprofile-generate|coverage:-lgcov}\ %{!nostdlib:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}}\ %{!A:%{!nostdlib:%{!nostartfiles:%E}}} %{T*} }}}}}}" #endif static const char *cc1_spec = CC1_SPEC " " GENTOO_HARDENED_CC1_SPEC; static const char *cc1plus_spec = CC1PLUS_SPEC; static const char *link_gcc_c_sequence_spec = LINK_GCC_C_SEQUENCE_SPEC; static const char *link_ssp_spec = LINK_SSP_SPEC; static const char *asm_spec = ASM_SPEC " " GENTOO_HARDENED_ASM_SPEC; The rest of the work is done by either defining the DEFINES or letting it undefined. This is keeping the intrusive parts out of gcc.c and move the logic to a separate directory of header files responsible for setting up the exclude triggers, deciding about the arches and the libc needed. Hope you like the idea and it will help make things clearer for all of us. Alex
Alex, it seems you were not listening ;-) Kevin and me agreed upon having only one set of minispecs for all archs. That means, that gcc.c will get added new sections (minimal possible list: asm_pie_spec, cc1_hardened_spec - covering pie and ssp -, crtfile_pie_spec, endfile_pie_spec, startfile_pie_spec and link_hardened_spec - covering pie, relro and now). If you do not go that route, you have to provide for each arch separate minispecs and probably with each gcc release you will have to regenerate at least some of them
Alex - re glibc-2.5, that's too far in the future for the moment. I'm assuming that vapier is talking about non-hardened use; I don't think he'll object if we put stuff into glibc-2.4 for hardened use, provided we don't break things for anyone else. Worst case is that we won't be able to have ssp-built glibc-2.4 in the tree, since it may involve quite a bit of patching to the build system. Re your ideas about specs; best thing for you to do is to implement it and see how it works in practice, then compare it to what we're doing to see if it's better or not. Probably my efforts were a bit confusing as I didn't get as far as sorting out startfile/endfile properly. As Peter says one of the core concepts is that the minispecs are generic - it's just where and how they get called that can be target-specific. Peter - re x86[_64] changes, I'm happy to wait while you try out the GCC_SPECS thing. I'll focus on glibc in the meantime. I saw the problem with sed'ing LIBGCC2_CFLAGS - later (4.x) gcc's have the definition in gcc/Makefile.in split over more than one line, and as you note the existing sed puts '-fno-stack-protector' on the end of the line - after the '\', which messes things up :/ I fixed it by putting the -fno-stack-protector as the first thing after the ' = ' - I think you've fixed it pretty much the same way. On the missing '\' - FWIW it's not necessary after '&&' (or '||') as a command conjunction; only time you need it after '&&' is if '&&' is part of a [[ ... ]] phrase; so: [[ -n ${PIE_VER} ]] && PIE_CORE=... and if [[ ${GCCMAJOR} == 3 ]] && [[ ${GCCMINOR} -ge 3 ]] then are ok (putting the trailing '\' in doesn't hurt, though), but if [[ -n ${PIE_VER} && -z ${PIE_CORE} ]]; then is broken.
Peter, i'm reading this bug since it was created and all i can read is two sides of one story from which i don't understand either side of it. The only things i am understanding is: - we should minimize the amount of gcc patching - there should be a way to provide for automatic/nonautomatic PIE SSP building - the support for nonglibc versions and arches has to be improved/cleaned - the existing macros cannot be redefined without losing the old information So the question for me remains: where to put the information about the arches. At the moment it's like this: A directory structure in the pie-patches tarball controls information about general patches for all architectures and architecture-specific patches which dive into subdirs of gcc/config and apply to local architecture-specific files. Previously this information has been stored in a complicated "generator" header-file which directly applied the information to the specs definitions. Now you are telling me you want to make a "mini specs" for all arches the same, which makes no sense to me because it just can't work. The reason is that x86 for example doesn't use a modified -K PIC switch in the asm section while sparc and ppc do. While hppa does not use SSP, it makes use of PIE in the usual ways. Simply put: i don't get the idea. Alex
Alex, read my 9.0.3 patches nondef directory, it will show you what I do for ex: #define ASM_SPEC <original blah> %(asm_pie) I create additional asm_pie section (but only the affected archs ASM_SPEC uses asm_pie), so if I make *asm_pie: -K PIC generally, it will only affect those archs where, %(asm_pie) was added. This way I am sure that I get all the original info from ASM_SPEC and add my own one. An affected arch will show you in its *asm: <original blah> %(asm_pie), you only have to set asm_pie in minispecs Why I dont want separate minispecs: - we have 2 sorts of crt parts in startfile, with and without profile - we have 2 sorts of crtbegin* users, those who have crtbeginT.o for static, others using crtbegin.o for static - we have at least 2 sorts of endfile* sections, with and without fastmath - we have archs that do not use startfile/endfile, they have linux specific naming - we have archs that need for PIC/PIE asm section modifications
Note: I have even checked with the gcc devs the possibility of (cant recall exact syntax): if-exist crtbeginT.o else crtbegin.o, they said that this will have heavy performance issues
okay, so if i understand you right: you keep modifying the same locations in gcc sourcecode with patches, but this time you don't enter the direct information, you only put a %(XXX) callout to a minispec file into it and let the real implementation be handled by the minispecs file taking care of the problems. And the minispec is generated with respect to the arch and libc it is running on. Am i misunderstanding something or something missing? Alex
I provide for each newly created sections defaults as vanilla gcc would behave (lets assume for a moment that hardened built of gcc is not used), so unless you do not provide any file to GCC_SPECS, you have a vanilla gcc, but you get the possibility to provide 1-n files in GCC_SPECS, each having some sections redefined: default *link_relro: %{norelro:}, but you can provide a minispec having *link_relro: %{!norelro: -z relro} and that gets added to link_command, because *link_command is defined as: ... %(link_relro) ...
okay, sounds good. Alex
Kevin: does the default specs file support having 2 sections with the same name where the last one wins? wouldnt it be easier to leave the default specs alone and only default GCC_SPECS to the proper one, depending on use hardened or not? I think the procedure setting GCC_SPECS in toolchain.eclass to build.specs will cause some trouble, if we have an old compiler without the new sections, the compiler will either bail out due to unknown section, or it wont do anything
Created an attachment (id=99170) [edit] gcc-hardened-callouts.patch This patch does multiple things. If gcc/config/gentoo-hardened/hardened.h is an empty file, it makes gcc behave like a normal gcc without any callouts and changes. If hardened.h contains the mentioned definitions of the callouts, these are added to the "safe" sections of specs where we can append or insert our callout handler without doing harm to the section itself. The handling of startfile and endfile is fully left to the additional minispecs in GCC_SPECS env variable. If no such env variable is present, the gcc behaves like a normal one, even though compiled with the "hooks" to trigger hardened behaviour on demand. If it finds the env var, the additional specs file is supposed to: a) implement the sections of the callouts embedded in the original sections b) overwrite startfile specs section c) overwrite endfile specs section if the additional specs file only implements SSP, it can skip b) and c) and only implement the cc1_hardened_ssp section for example. if the additonal specs file implements only PIE, it can skip parts of a) but must do b) and c) according to Peters logic he explained previously. Thanks for helping me understand your patches, Alex
oh, and what i forgot. This patch also makes it easy to go both routes: a PIE SSP patch can either - directly patch up the GENTOO_HARDENED_STARTFILE_SPEC inside the hardened.h header in the config/gentoo-hardened directory - or use the new startfile section in the specs file available via GCC_SPECS env var It is also safe to have a patch add a customized GENTOO_HARDENED_STARTFILE_SPEC and a GENTOO_HARDENED_ENDFILE_SPEC via patching hardened.h in the ebuild or toolchain eclass (for example an improved nondefault PIE version for uclibc or glibc) and then add a specs file to the ENV var which does DEFAULT PIE in the startfile and endfile and fills out the handlers for the callouts with the respective action (like triggering -pie automatic). I hope you understand what i mean :) Alex
first testing shows it works: # cat /tmp/ssp.specs *cc1_hardened_ssp: %{!fno-stack-protector: %{!fno-stack-protector-all: %{!fstack-protector: %{!fstack-protector-all: -fstack-protector-all} } } } # GCC_SPECS="/tmp/ssp.specs" ./gcc/xgcc -o vuln-stack vuln-stack.c ; ./vuln-stack AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA *** stack smashing detected ***: vuln-stack terminated Killed 02:41:04 [/space/chroots/chroot001:6690.pts-2.miranda]miranda /var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/build # GCC_SPECS="" ./gcc/xgcc -o vuln-stack vuln-stack.c ; ./vuln-stack AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Segmentation fault 02:41:10 [/space/chroots/chroot001:6690.pts-2.miranda]miranda /var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/build # GCC_SPECS="/tmp/ssp.specs" ./gcc/xgcc -o vuln-stack vuln-stack.c ; ./vuln-stack AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA *** stack smashing detected ***: vuln-stack terminated Killed # GCC_SPECS="/tmp/ssp.specs" ./gcc/xgcc -v -o vuln-stack vuln-stack.c ; ./vuln-stack AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Using built-in specs. Reading specs from /tmp/ssp.specs Target: x86_64-pc-linux-gnu Configured with: /var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/gcc-4.1.1/configure --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.1.1 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.1.1 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.1.1/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.1.1/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/include/g++-v4 --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --disable-altivec --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --disable-libunwind-exceptions --disable-multilib --disable-libmudflap --disable-libssp --disable-libgcj --enable-languages=c,c++ --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu Thread model: posix gcc version 4.1.1 (Gentoo 4.1.1-r1) /usr/libexec/gcc/x86_64-pc-linux-gnu/4.1.1/cc1 -quiet -v -iprefix /var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/build/gcc/../../../lib/gcc/x86_64-pc-linux-gnu/4.1.1/ vuln-stack.c -fstack-protector-all -quiet -dumpbase vuln-stack.c -mtune=k8 -auxbase vuln-stack -version -o /tmp/ccT1OWUN.s ignoring nonexistent directory "/var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/build/gcc/../../../lib/gcc/x86_64-pc-linux-gnu/4.1.1/include" ignoring nonexistent directory "/var/tmp/portage/gcc-4.1.1-r1.TESTBED/work/build/gcc/../../../lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/include" ignoring nonexistent directory "/usr/local/include" ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/include" #include "..." search starts here: #include <...> search starts here: /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/include /usr/include End of search list. GNU C version 4.1.1 (Gentoo 4.1.1) (x86_64-pc-linux-gnu) compiled by GNU C version 4.1.1 (Gentoo 4.1.1). GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 Compiler executable checksum: 54533a42eb6b472ac36fa421b5271c16 /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/bin/as -V -Qy -o /tmp/cc2OGymt.o /tmp/ccT1OWUN.s GNU assembler version 2.16.1 (x86_64-pc-linux-gnu) using BFD version 2.16.1 /usr/libexec/gcc/x86_64-pc-linux-gnu/4.1.1/collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o vuln-stack /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/crtbegin.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../.. /tmp/cc2OGymt.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/crtend.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/crtn.o *** stack smashing detected ***: vuln-stack terminated Killed
Alex, I have no idea what your intention with this patch is, but it seems you should reread the patch piepatch/nondef/00* and explain to me how your proposed changes will improve that one.
(In reply to comment #50) > Kevin: does the default specs file support having 2 sections with the same > name where the last one wins? Yes. Later specs definitions override previous ones. > wouldnt it be easier to leave the default specs alone > and only default GCC_SPECS to the proper one, depending on use hardened or not? > I think the procedure setting GCC_SPECS in toolchain.eclass to build.specs will > cause some trouble, if we have an old compiler without the new sections, the > compiler will either bail out due to unknown section, or it wont do anything Yes; in that case it won't do anything. In my mods to toolchain.eclass, I have this conditional on the new specs arrangement, so that older compiler versions are unaffected. I had done that by setting another variable in the ebuild, but the easiest way would be to do it conditional on the PIE_VER, so that if PIE_VER is >=9.0 we do the new stuff, if it's <9 we do things the old way. Note that if you're building from hardened 3.4.6-r1 (or earlier), the existing default specs file (/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/specs) is always read, so will be hardened even if GCC_SPECS is set to build.specs (in that case the contents of build.specs make no difference). It is also useful to create ${LIBPATH}/specs for the final installation - I do that in my toolchain.eclass at the end of gcc-compiler_src_install(). This means that the compiler is hardened even in an empty environment (i.e. GCC_SPECS unset).
(In reply to comment #54) > Alex, I have no idea what your intention with this patch is, but it seems you > should reread the patch piepatch/nondef/00* and explain to me how your proposed > changes will improve that one. > Hey Peter, the only thing i am intending is that i was working out an idea because i didn't understand the setup of your patch. As i said, to me it makes more sense to change startfile/endfile in a central location instead of in the subdirectories of the arches. And the rest of the patch resembles your and Kevins idea about modularizing the specs into an external file. The only thing i changed was to modifications to gcc.c and adding an own config dir for the hardened work which looks more transparent to me. This whole thing is not meant to replace your work or make you look bad, it's just a way of opening up my mind into another perspective. Thanks, Alex
To me it does not make sense to have a central location to change startfile/endfile due to the variety of them mentioned in another post. You could search the gcc-ml for a proposed patch that would have done the generalization, it was unacceptable
(In reply to comment #39) > Kevin: I will do some testing with your GCC_SPECS use in eclass, if that goes > fine, I can remove all the DEFAULT_* from my patches and leave the handling > exclusively to the eclass. Do you need earlier the changes needed for x86[_64]? Actually, I've changed my mind :) Could you provide the differences for 4.1.1? I've just committed a version of toolchain.eclass to my SVN area that I think should work with the GCC_SPECS method of building gcc hardened and your patchset once you've removed the EFAULT_XXX bits (also includes the uclibc changes). Probably clearer than some of my explanations :) Obviously it's untested as it needs your updated patchset.
