When compiling dev-haskell/haddock-0.6-r1 using ghc-6.2 and a Propolice gcc, the compilation of FastMutInt.hs fails with error messages mentioning __stack_smash_handler. I infer that this is related to using Propolice gcc, either in compiling ghc or haddock. Full messages below. Reproducible: Didn't try Steps to Reproduce: 1. emerge -Dv haddock #(using unstable) Actual Results: make[1]: Entering directory `/var/tmp/portage/haddock-0.6-r1/work/haddock-0.6/haddock' ------------------------------------------------------------------------ ===fptools== Recursively making `all' in src html doc ... PWD = /var/tmp/portage/haddock-0.6-r1/work/haddock-0.6/haddock ------------------------------------------------------------------------ ------------------------------------------------------------------------ ==fptools== make all -wr; in /var/tmp/portage/haddock-0.6-r1/work/haddock-0.6/haddock/src ------------------------------------------------------------------------ make INSTALLING=0 BIN_DIST=0 - --no-print-directory -r all /usr/bin/ghc -H16m -O -package network -fglasgow-exts -cpp -c FastMutInt.hs -o FastMutInt.o -ohi FastMutInt.hi Warning: retaining unknown function `__i686.get_pc_thunk.bx' in output from C compiler Prologue junk?: .globl __stack_smash_handler .globl __stginit_FastMutInt .type __stginit_FastMutInt, @function __stginit_FastMutInt: movl %ebx, 40(%esp) call __i686.get_pc_thunk.bx addl $_GLOBAL_OFFSET_TABLE_, %ebx movl __guard@GOT(%ebx), %eax movl (%eax), %eax movl %eax, 16(%esp) make[3]: *** [FastMutInt.o] Error 255 make[2]: *** [all] Error 2 make[1]: *** [all] Error 1 make[1]: Leaving directory `/var/tmp/portage/haddock-0.6-r1/work/haddock-0.6/haddock' make: *** [build] Error 1 !!! ERROR: dev-haskell/haddock-0.6-r1 failed. !!! Function src_compile, Line 40, Exitcode 2 !!! make failed Expected Results: Should have compiled successfully. Portage 2.0.49-r18 (default-x86-1.4, gcc-3.3.2, glibc-2.3.3_pre20031212-r0, 2.6.0-gentoo) ================================================================= System uname: 2.6.0-gentoo i686 AMD Athlon(tm) XP 2200+ Gentoo Base System version 1.4.3.12 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-mcpu=athlon-xp -O2 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /opt/tomcat/conf /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-mcpu=athlon-xp -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache sandbox usersandbox" GENTOO_MIRRORS="ftp://mirror.iawnet.sandia.gov/pub/gentoo/ http://gentoo.seren.com/gentoo ftp://csociety-ftp.ecn.purdue.edu/pub/gentoo/" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow X Xaw3d acl acpi acpi4linux alsa amd antlr apache2 apm arts artswrappersuid avi berkdb bonobo cdr crypt cups directfb dnd doc dv dvd dvdr encode esd ethereal evo fbcon fbdev foomaticdb foreign-package foreign-sysvinit freetds gb gd gd-external gdbm gif ginac gnome gnomedb gpm gstreamer gtk gtk2 gtkhtml guile imap imlib innodb ipv6 java jikes jpeg junit kde kerberos ldap libg++ libgda libwww mad maildir mbox mdb mikmod mmx motif mozilla moznoirc moznomail mpeg mysql ncurses nls nptl oci8 odbc ofx oggvorbis opengl oss pam pda pdflib perl pg-hier pic plotutils png postgres ppds python qt quicktime radeon readline ruby ruby18 samba sasl sdk sdl slang slp snmp spell sse ssl svga tcltk tcpd tetex tiff truetype type1 usb wmf wxwindows x86 xml xml2 xmms xv zeo zlib"
can you please try. CFLAGS="-fno-stack-protector -fno-stack-protector-all" emerge program
re: Comment #1, solar, I tried your suggestion (now on haddock-0.6-r2), but got the same error compiling the same program. (Note: I am using the hardened gcc, not just the -fstack-protector option. Should I try turning hardened gcc off completely? If so, what commands should I use to test this?)
for the test you will want to try. hcc -R ; emerge pkg ; hcc -a solar@simple solar $ hcc Usage: hcc-3.3.2.0-x86 [single option] Change the current cc/gcc specs file for PIE support and propolice -l, -L, --list list currently active settings -a, -A, --activate activate default transparent settings -r, -R, --restore recover original settings -v, -V, --version display program version -------------------------------------------------------------------- What I think is going on here is the problem itself is in /usr/bin/ghc I however don't know what "ghc" is and trying to come up with a fix other than filtering flags is somewhat tricky. (Is it a compiler of a sort?)
Re: Comment #3, solar, your suggestion to restore the original gcc settings worked successfully, so this appears to be a workaround. I don't know if the problem is that ghc is incompatible with hardened gcc in general or what. This will still need to be investigated. GHC is the Glasgow Haskell Compiler, one of the full-featured Haskell compilers. I'm sure kosmikus could explain it more fully than me. You can find out about it from the Haskell website: http://www.haskell.org . Note: Based on this problem compiling haddock, I infer that GHC uses GCC internally while converting Haskell programs to executables. Perhaps GHC assumes something about the format of the object file produced by GCC that is modified by using the Hardened GCC features. This is merely a hypothesis. I have not studied how GHC works.
Yes, GHC does use GCC internally, and it postprocesses GCCs output using some perl script (called GHC's Evil Mangler). Unfortunately, I am not at all a C expert (and even less an assembler) and cannot tell how this works in more detail. I know that GHC caused some problems in conjunction with hardened-gcc on its own, and that pappy added something to the ebuild (based on experiments done by Peter Simons) to fix this particular issue (contained, for example, in ghc-6.2.ebuild). Maybe this is somehow related? ks
Can somebody that uses ghc verify this bug is/(is not) still a problem.
Re: Comment #6, solar, I still get the same error when using Hardened GCC. However, turning off hardening causes the compile to complete successfully. See comment #4 for what worked before. Based on Andres's comment #5, I believe the fix will require making GHC's Perl postprocessor aware of Propolice, or turning off Propolice when using GCC inside of GHC. I'd refer this to the GHC developers to see if they want to make their Perl postprocessor handle Propolice or not.
Created attachment 24521 [details] ssp.c Lets see I don't know much about ghc but I'm willing to share any insights I may have about ssp and the way it works. First it works on c/ccp code. By default it's only the symbols it provides are via libgcc.a, what we have done at gentoo to avoid problems and the lack of being able to link other things properly is to move those symbols into glibc itself. (you will noticed this when build gcc-3.3.2-rX) I don't know if it will help but I'll provide a ssp.c that can provide the same symbols as the one gcc provides which you should be able to compile into object code and do whatever you need with (I'd hope). As for the "call __i686.get_pc_thunk.bx" <-- I dont know what exactly causes that. It's not ssp as far as I'm aware of. This is pic-ssp for sure however. # -------------------------------------- addl $_GLOBAL_OFFSET_TABLE_, %ebx movl __guard@GOT(%ebx), %eax movl (%eax), %eax movl %eax, 16(%esp) # --------------------------------------- I'll try to help in anyway I can.
I'm still interested to get this fixed, but I'm afraid I can't be of much help, because I don't know much about C, and almost nothing at all about all this hardened/propolice stuff. BTW, solar, can you point me to a good reference that might give me some insight what's going on? I'm not really sure what to do with your ssp.c, though, sorry. Howard, have you done anything more? If I understand correctly, there is a workaround, using the hcc -R and hcc -a combination. Is it a "good" workaround, i.e. would it be suitable to put that into the ebuild under certain circumstances, or does it miss the point completely? Howard, if ghc is the problem, then other Haskell-builds should fail, too. Can you confirm that. What, for example, with alex or happy? ks
Re: Comment #9: Andres, I tried to re-emerge happy, and it failed with the same problem as haddock. My suggestion at this point is the same as my Comment #7. Andres, do you have a channel to the GHC developers? If so, please let them know about this problem and ask if they want to make their "Evil Mangler" aware of the extra code produced by Propolice. Andres, to find out more about what Propolice is, see the Gentoo Hardened GCC Project's Propolice Guide, available at http://www.gentoo.org/proj/en/hardened/propolice.xml .
this problem is still unsolved? what did the ghc upstream say about it? we have two choices: modify the GHC commands given to gcc for hardened-gcc or hardened gcc or file it upstream and close it here giving a reference to it.
sorry, I did not have time to pursue this. I have found a brief discussion on the topic of compiling ghc with propolice enabled on one of the ghc mailing lists, but none of the official developers picked up. I guess I could ask again, more specifically. Anyway, I don't expect this to get solved overnight, so I tend to say that for now it would be best to adopt a solution that disables these features for ghc completely. This is already done in the ghc ebuild (while building ghc, because it bootstraps off itself), but it needs to be built into the compiler in general on a hardened-gcc system -- otherwise, programs that need to be built by ghc such as haddock or alex or other Haskell tools will still fail. I have a patch on my local system which seems to work reasonably well and does this, but I don't like it because it patches the ghc sources at compile time, depending on a check on "has_version sys-devel/hardened-gcc", the same check that is used in the ghc ebuild to determine whether propolice has to be disabled during the build. But as I understand it, this is not optimal. Hardened-gcc could be installed on the system *after* ghc has been emerged, and even with hardened-gcc installed, its features can be en-/and disabled using the hcc tool. So, is there a fast and reliable test that could be used on the fly to determine whether the currently used gcc is propolice-enabled or not? hcc -l produces a lot of output that I cannot interpret. If such a test exists, the already existing wrapper script around ghc could be patched in such a way that it inserts the disabling compiler flags whenever the current compiler is propolice-enabled. ks
Alexander, following up on Comment #12: Is there an environment variable that the Haskell compiler (GHC) could set to turn off the Hardened GCC features for the internal calls to GCC? If so, this seems to me to be a good workaround until GHC's "Evil Mangler" is made ProPolice aware. (It doesn't appear to me that this feature currently exists in Hardened GCC, but I'm not sure.) Here's why I recommend this approach: 1. If Hardened GCC IS in use, it would turn it off for that internal compilation only. 2. If Hardened GCC IS NOT installed, the environment variable wouldn't mean anything. 3. To add the "hcc -R" ... "hcc -a" wrapper around the internal compilation would affect any compiles that occur during the interim, not just for this internal call of GCC. 4. The "hcc -R" ... "hcc -a" wrapper only works for privileged users who can write to /usr/lib/gcc-lib/i686-pc-linux-gnu/3.X.Y/specs (where 3.X.Y is the current GCC version). 5. The wrapper would fail if Hardened GCC isn't installed. I guess this could be wrapped by a test for the existence of Hardened GCC, but it wouldn't solve items 3 and 4 above. I don't know how this feature could be implemented, since it requires changing which "specs" file is used on the fly. If you know how to do this, please let me know. If you don't know how to do it, please tell me and I'll consider the problem some more to see if I can come up with either another workaround or a way to choose the "specs" file on the fly. Thanks.
An environment variable would be one, but not the only solution An efficient check would be sufficient, afaics. If I understand correctly (and this is at least how things appear to me after some very limited local tests), it is enough to pass -optc-yet_exec -optc-yno_propolice to ghc if using a propolice-enabled gcc, and things will work just fine. This could automatically done in the shell script that /bin/ghc is. The only problem is that a normal gcc does not recognize these options, and a call to ghc then always produces a warning message, and I don't find this acceptable. Therefore, I need an efficient check if gcc supports the flag, efficient enough to perform it every time that ghc is called. But maybe I can just use gcc -c -yno_propolice 2>&1 | grep -q propolice as a test? Would that be sufficiently robust? ks
I have just committed what I proposed in #14 to CVS. The ebuild is dev-lang/ghc-6.2-r1 . I have been able to successfully install ghc, and afterwards, I could successfully compile both haddock and alex with it, without any modification in those ebuilds. Please test if it works for you. If so, I will modify also ghc-bin accordingly. ks
we have several choices here i am currently dreaming about another "wily hack" which could make this possible for ebuilds: the gcc wrapper that is set up by gcc-config has to learn about the environment anyway to find out where the gcc executable is which it has to use, correct? so why not teach the gcc wrapper (/usr/bin/gcc) to check for those exported HARDENED env variables during compiling, it will automatically insert the appropriate flags transparently. the exporting would happen in the ebuild using a special class: flag-o-matic. you do only the following in any ebuild: filter-flags "-fstack-protector" or "-fPIC" this will trigger the following then in the eclass: the eclass flag-o-matic will export HARDENED_SSP="-fno-stack-protector" because this works for every supported gcc (can check with has_version) it will also insert HARDENED_PIE="-nopie" or "-yet_exec" when hardened specs are found (gcc --version sed logic or has_version or USE flag logic) i agree that trashing the userland system with superfluous error messages is bad. and with Gentoo we have the choice to deal with that transparently: the ebuild uses filter-flags to trigger the logic in the eclass independent of which gcc version is used when there is a hardened gcc found that understands and needs those flags, the flag-o-matic eclass exports the environment variables which get recognized by the gcc wrapper little executable, which then feeds those options to the real gcc the only drawback would be environments who overwrite the ENV during building but then we can still fall back to sed mangling in the Makefiles directly in the ebuild... but this is always a choice we have and still keep with that ENV approach. good hunting! ;-)
While #16 at first glance seems to be a very elegant solution for ebuilds, it is not necessarily sufficient for ghc. The Glasgow Haskell Compiler calls gcc whenever the -O flag is used, which will happen in some ebuilds -- but of course, the user who installs ghc is allowed to use the compiler outside of ebuilds. I did not really see that to the full extent until recently, because I never had hardened-gcc installed on any of my machines. Now I have, and I think the only option (besides ghc supporting all of this, and that's beyond my sphere of influence right now) is to make sure these features get disabled whenever ghc is called on the system, thus not just in ebuilds. I am not sure if flag-o-matic.eclass, being largely just a shell-script, could be imported into the ghc wrapper script and be used there. Otherwise, I'd prefer a solution where I can disable/control the features from anywhere. Possibilities that I see: (1) via a call to gcc that checks whether it complains; not elegant, but done right now (cf. comments #14 and #15) and works for me (2) via an environment variable; setting DISABLE_HARDENED to something would disable special features and would be checked; if gcc doesn't know about hardened, the envvar does no harm (3) via a different executable: just use gcc-nohardened to get the traditional gcc; the advantage over the current situation: gcc-nohardened could be relied upon to be present whenever hardened-gcc is installed; it could maybe even be installed as a symlink to just gcc by the gcc-config package if hardened-gcc is not installed. At the moment, checking for the presence of hardened-gcc is not enought, because in addition, one has to check if it is *active* or not. (4) via gcc-config (I think I might prefer that). If one could use a call like `gcc-config --traditional` to get a call including flags that does not make use of any hardened features, that could be transparently used and relied upon by any Gentoo package and program. The interface is open to discussion. Essentially, gcc-config could also do the filtering. I could say `gcc-config --call-gcc <all-my-options>`, and gcc-config would automatically filter options that the currently selected gcc doesn't support. I hope that what I said at least partially made sense. ks
Re: Comments #11 through #17: I don't understand how gcc, gcc-config, hardened gcc, etc., interact. Therefore, I'm not able to understand the alternatives fully and assess their pros and cons. Is there some documentation of the interactions of these various pieces that I could read to learn more? TIA.
Re: Comment #15: Andres, I emerged ghc-6.2-r1 and then successfully emerged haddock-0.6-r2. Therefore, your modification to ghc-6.2-r1 to disable Propolice on the internal gcc compilation works for me.
fix and close thanks for your ongoing commitment, Alex (happy pappy now)
