This bug is specifically for adding CET to be enabled by default with USE=hardened. There's a PR with work by xaero here: https://github.com/gentoo/gentoo/pull/21588. Needs a tweak to the specs patch. (The linked PR also adds USE=cet to GCC but that's orthogonal to having CET functionality by default when compiling.) Plan is to let it soak in hardened for a bit and then try to upstream it with a nice configure option, like with -fstack-clash-protection, SSP, etc. Note that Ubuntu have been doing this for quite some time without many issues: https://salsa.debian.org/toolchain-team/gcc/-/blob/master/debian/patches/gcc-distro-specs.diff.
Enabled by default in https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=51a7ace358097005038a0d31350b0c6d3da34e00 for USE=cet && USE=hardened. See also: - https://github.com/gentoo/gentoo/commit/98e481cd2d539c32a7a094d0e5ca6de36f9a0639 (toolchain.eclass: enable CET by default on hardened for >= gcc 11.2.1) - https://github.com/gentoo/gentoo/commit/10bad611687710903699c02b6a0d12280337f96b (profiles/features/hardened/amd64: unmask CET) - https://github.com/gentoo/gentoo/commit/b034debd0ee10095843845b6f1f6e7385188c208 (profiles/base: mask sys-devel/gcc[cet] in base)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=2b36f3ad2ba0114eae1d32bae5e395e098b3714b commit 2b36f3ad2ba0114eae1d32bae5e395e098b3714b Author: Sam James <sam@gentoo.org> AuthorDate: 2021-12-28 03:44:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2021-12-28 03:55:44 +0000 11.3.0: fix CET patch Our patch was causing unhandled state to leak into the LTO metadata writer, it shouldn't have got that far though. Instead of messing about with GCC's option handling, use the macro they provide for purposes like this, which makes things far simpler (and less fragile). Bug: https://bugs.gentoo.org/828400 Bug: https://bugs.gentoo.org/822036 Thanks-to: Sergei Trofimovich <slyich@gmail.com> (debugging help in #gentoo-toolchain) Thanks-to: Georgy Yakovlev <gyakovlev@gentoo.org> (debugging) Reported-by: matoro <matoro@airmail.cc> Signed-off-by: Sam James <sam@gentoo.org> 11.3.0/gentoo/26_all_enable-cet.patch | 65 +++++------------------------------ 1 file changed, 9 insertions(+), 56 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b96fd11e3e5626181c32c38381f814aba21fb9f0 commit b96fd11e3e5626181c32c38381f814aba21fb9f0 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-01-18 13:16:42 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-01-18 13:19:30 +0000 sys-devel/gcc: add 11.2.1_p20220115 Fairly minor changes upstream since the last snapshot of the 11 stable branch. Includes more CET fixes and the upstream cross-compile patch. Also, the PCH ICE fix, although we've since masked PCH globally due to its instability. Bug: https://bugs.gentoo.org/822036 Closes: https://bugs.gentoo.org/803371 Closes: https://bugs.gentoo.org/828400 Closes: https://bugs.gentoo.org/822690 Signed-off-by: Sam James <sam@gentoo.org> profiles/base/package.use.mask | 2 +- sys-devel/gcc/Manifest | 3 +++ sys-devel/gcc/gcc-11.2.1_p20220115.ebuild | 26 ++++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 1 deletion(-)
Multiple people myself included have been running CET enabled toolchain for a while (with a CET capable CPU, of course). Could this maybe be ready for going default? One thing that kinda makes me doubt things is that from what I have read and understood -fcf-protection requires both the binary as well as all of its libraries to be built with -mshstk or the support for CET will be silently disabled by, I believe, the dynamic linker. Is or could there be some kind of sanity check tool to ensure that there's an actual CF enforcement rather than a dud?
(In reply to Niklāvs Koļesņikovs from comment #4) > Multiple people myself included have been running CET enabled toolchain for > a while (with a CET capable CPU, of course). Could this maybe be ready for > going default? Yeah, I don't think we've had any reported problems at all. So, we probably can soon. > > One thing that kinda makes me doubt things is that from what I have read and > understood -fcf-protection requires both the binary as well as all of its > libraries to be built with -mshstk or the support for CET will be silently > disabled by, I believe, the dynamic linker. Is or could there be some kind > of sanity check tool to ensure that there's an actual CF enforcement rather > than a dud? I don't think this is the case for modern GCCs. Please provide some sources if you believe it is and I'll look into it, but when I researched it before (and had to in order to do the patching), I had no such indications.
The best I could immediately find is this: https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc which specifies that Fedora 28 onwards will use `-mcet -fcf-protection` (Fedora 28 hardening notes clear up that it's -fcf-protection=full). As far as I could google, at some point GCC replaced the Intel specific -mcet with the vendor neutral -mshstk which is or will be available for at least AMD64 and some(?) ARM64 ISA. Therefore the current CFLAGS should be `fcf-protection=full -mshstk`. Regarding the statement that not just the binary but all of its libraries must be built with support for CET (to be clear I merely omitted -fcf-protection for brevity, -mshstk may or may be enough for libraries), I tried but could not find the source of that but hopefully someone will have better luck than me. Finally I was under impression that -fcf-protection on its own does nothing if -mshstk is also not specified, however its `man gcc` entry only says that: "The -mshstk option enables shadow stack built-in functions from x86 Control-flow Enforcement Technology (CET)." So I'm now left unsure if -mshstk is merely good and helpful or, because it replaces -mcet, mandatory for -fcf-protection to work. One thing I have not checked but that may be happening is that perhaps -march=native or -fcf-protection either on their own or together enable -mshstk on CET capable or maybe even all CPUs that can support multi-byte NOPs (please don't ask me if multi-byte there means >8 bits or >16 bits or more).
(In reply to Niklāvs Koļesņikovs from comment #6) > The best I could immediately find is this: > https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc > which specifies that Fedora 28 onwards will use `-mcet -fcf-protection` > (Fedora 28 hardening notes clear up that it's -fcf-protection=full). I think the 28 stuff predates the GCC changes to simplify the picture. From building a test binary with -fcf-protection=full, if I use 'readelf -n /tmp/a.out', I see: > Properties: x86 feature: IBT, SHSTK It's identical if I append -mshstk. > > Finally I was under impression that -fcf-protection on its own does nothing > if -mshstk is also not specified, however its `man gcc` entry only says that: > > "The -mshstk option enables shadow stack built-in functions from x86 > Control-flow Enforcement Technology (CET)." > > So I'm now left unsure if -mshstk is merely good and helpful or, because it > replaces -mcet, mandatory for -fcf-protection to work. > I don't think it's necessary at all, as far as I can tell. > One thing I have not checked but that may be happening is that perhaps > -march=native or -fcf-protection either on their own or together enable > -mshstk on CET capable or maybe even all CPUs that can support multi-byte > NOPs (please don't ask me if multi-byte there means >8 bits or >16 bits or > more). Yes, that's possible. I think one is to indicate that your CPU supports it, but -march= might imply it a lot of the time? But indeed I see SHSTK being mentioned via the notes section even without -march, or by picking something older like core2. I'm not sure -mshstk actually matters given that in terms of code emitted, the CMOVs used by CET are completely safe on i686+ (and indeed we only bother enabling it for 64-bit && CMOV being available as a sanity check). All this to say: I don't yet see any difference from adding -mshstk. -fcf-protection=full implies the relevant options (it implies both shstk and ibt, because both are necessary for -fcf-protection=return and -fcf-protection=branch respectively) In gcc-config/i386/gnu-property.cc, we have: ``` file_end_indicate_exec_stack_and_gnu_property(void) { [...] if (flag_cf_protection & CF_BRANCH) /* GNU_PROPERTY_X86_FEATURE_1_IBT. */ feature_1 |= 0x1; if (flag_cf_protection & CF_RETURN) /* GNU_PROPERTY_X86_FEATURE_1_SHSTK. */ feature_1 |= 0x2; [...] } ``` In gcc/config/i386/i386.md, we have: ``` ;; CET instructions (define_insn "@rdssp<mode>" [(set (match_operand:SWI48 0 "register_operand" "=r") (unspec_volatile:SWI48 [(match_operand:SWI48 1 "register_operand" "0")] UNSPECV_NOP_RDSSP))] "TARGET_SHSTK || (flag_cf_protection & CF_RETURN)" "rdssp<mskmodesuffix>\t%0" [(set_attr "length" "6") (set_attr "type" "other")]) [...] ``` What's confusing is, while instructions are emitted for -fcf-protection=full and such, there's a few parts gated by SHSTK only. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85417 has some interesting background here. Ubuntu don't bother either (Debian may be using their patch too): https://salsa.debian.org/toolchain-team/gcc/-/blob/master/debian/patches/gcc-distro-specs.diff.
Wanted to split this part into a new comment. I think I get it now. >This patch changes -fcf-protection to implement indirect branch and >return address tracking with multi-byte NOPs. -mibt and -mshstk are >changed to only enable CET built-in functions. gcc/config/i386/cetintrin.h are the builtins that hj lu refers to in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85417. I think you might need -mibt and -mshstk if you want the whole program to be using CET, *but* it comes at the cost that you cannot disable it for any component that gets linked. GCC's view appears to be that it's unsuitable to set this by default or imply it by -fcf-protection=*: >-fcf-protection -mcet can't be used with IFUNC features, like symbol >multiversioning or target clone, since IBT/SHSTK are applied to the whole >program and they may be disabled in some functions. It's also harmful to compatibility then. (-mcet got renamed, as you noted). The question is how much these builtins really matter. Given that it does seem to actually work (I get e.g. __CET__ defined and Ubuntu had issues w/ say https://news.ycombinator.com/item?id=26060748, https://github.com/ipxe/ipxe/commit/e8393c372), I'm inclined to say we're fine. It seems that -mshstk and -mibt are only appropriate per-program if it's known to work with it. It's unclear to me if it's safe to build your system with it.
(In reply to Sam James from comment #8) > > It seems that -mshstk and -mibt are only appropriate per-program if it's > known to work with it. It's unclear to me if it's safe to build your system > with it. The fact that it breaks IFUNC seems to me to be a dealbreaker for doing that by default: https://sourceware.org/glibc/wiki/GNU_IFUNC.
Thank you for the good explanation. I had found and read https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85417 once or twice but somehow it didn't occur to me that setting -mshstk was a bad idea. The only issue I have observed that may or may not be related is that since the upgrade to Firefox 100 the widevince plugin has been crashing with www-client/firefox{,-bin} but not the Flatpak Firefox. And while I definitely did suspect -fcf-protection -mshstk as being the likely culprits, it has been a very low priority issue for me, so I have not gotten to rebuilding everything without one of them yet (but I'll try removing -mshstk at some point).
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2a542765c79efb7cb7767f1c4ace4abcb45c9420 commit 2a542765c79efb7cb7767f1c4ace4abcb45c9420 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-05-28 23:30:26 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-05-28 23:30:26 +0000 sys-devel/gcc: add 12.1.1_p20220528; patch set 6 New patch set just includes CET default dropped from 11->12 at some point. Not that it matters much given it's all unkw'd anyway, but still. Bug: https://bugs.gentoo.org/822036 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 3 +++ sys-devel/gcc/gcc-12.1.1_p20220528.ebuild | 27 +++++++++++++++++++++++++++ sys-devel/gcc/gcc-12.2.9999.ebuild | 6 +++--- 3 files changed, 33 insertions(+), 3 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=ce1ac49c2c52611e507c4ec853f5eef3770aa723 commit ce1ac49c2c52611e507c4ec853f5eef3770aa723 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-05-28 23:19:08 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-05-28 23:36:28 +0000 12.1.0: restore 26_all_enable-cet.patch Got lost apparently in 11->12 migration. Bug: https://bugs.gentoo.org/822036 Signed-off-by: Sam James <sam@gentoo.org> 12.1.0/gentoo/26_all_enable-cet.patch | 57 +++++++++++++++++++++++++++++++++++ 12.1.0/gentoo/README.history | 4 +++ 2 files changed, 61 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=d67ec824d290c0e678b1d7d71ba47e24e2d892ca commit d67ec824d290c0e678b1d7d71ba47e24e2d892ca Author: Sam James <sam@gentoo.org> AuthorDate: 2022-05-30 06:41:16 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-06-02 07:15:27 +0000 12.1.0: update CET patch Bug: https://bugs.gentoo.org/822036 Signed-off-by: Sam James <sam@gentoo.org> 12.1.0/gentoo/26_all_enable-cet.patch | 225 ++++++++++++++++++++++++++++------ 12.1.0/gentoo/README.history | 4 + 2 files changed, 191 insertions(+), 38 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9eab5244a2b42dba2fac3ecb7be347b841c2ab15 commit 9eab5244a2b42dba2fac3ecb7be347b841c2ab15 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-06-02 06:27:01 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-06-02 07:15:41 +0000 sys-devel/gcc: update CET patch for 12 Let's switch to the Ubuntu-style spec changing given that the old approach was fragile and could fall over w/ bootstrap (at least w/ USE="pgo lto jit"). Didn't seem to happen with different combinations though. Bug: https://bugs.gentoo.org/822036 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 1 + sys-devel/gcc/gcc-12.1.1_p20220528-r1.ebuild | 27 +++++++++++++++++++++++++++ sys-devel/gcc/gcc-12.2.9999.ebuild | 2 +- 3 files changed, 29 insertions(+), 1 deletion(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f69a93f7c1aac2540ea2ecd80a264b7b8ba1094a commit f69a93f7c1aac2540ea2ecd80a264b7b8ba1094a Author: Sam James <sam@gentoo.org> AuthorDate: 2023-01-20 21:18:02 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-01-20 21:18:02 +0000 profiles/features/hardened/amd64: enable USE=cet Closes: https://bugs.gentoo.org/822036 Signed-off-by: Sam James <sam@gentoo.org> profiles/features/hardened/amd64/make.defaults | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=15daf0510a5fab17cd556261d688a6618391a0c1 commit 15daf0510a5fab17cd556261d688a6618391a0c1 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-01-22 08:35:13 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-01-23 00:35:20 +0000 11.4.0: drop 26_all_enable-cet.patch (bootstrapping issues with PGO/LTO?) See https://bugs.gentoo.org/891655#c2 in particular: """ In https://bugs.gentoo.org/822036#c14, I got fed up and changed to the Ubuntu-style specs patching for the time being, because it was too fragile otherwise. We should probably just drop the CET patch for 11.x. We can keep the USE in make.defaults for CET on hardened profiles though, as support for it is still useful, but it'll only be on by default with 12. I do want to revisit the specs patch later on as it's not very maintainable though. """ 11.x is really not the place we want to experiment further at this point, so go for the safe option and either develop the approach further for 12 or maybe even 13. Arsen notes that we might be able to try --enable-cet=yes (rather than default auto) but let's leave that for now given ^ wrt 11. Bug: https://bugs.gentoo.org/822036 Closes: https://bugs.gentoo.org/891655 Signed-off-by: Sam James <sam@gentoo.org> 11.4.0/gentoo/26_all_enable-cet.patch | 48 ----------------------------------- 11.4.0/gentoo/README.history | 3 +++ 2 files changed, 3 insertions(+), 48 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=016184c289f2cc6c6ade496a700a12f135fbae07 commit 016184c289f2cc6c6ade496a700a12f135fbae07 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-01-23 00:25:35 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-01-23 00:42:40 +0000 sys-devel/gcc: backport CET fix for 11; further make jobserver related fixes for all - jobserver: Led to crashes during build in some cases. Note that this is a bug in the driver so could easily happen when using GCC to build other things too. - CET: See https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=15daf0510a5fab17cd556261d688a6618391a0c1 for details, but went for a more conservative approach for GCC 11. Bug: https://bugs.gentoo.org/822036 Closes: https://bugs.gentoo.org/885501 Closes: https://bugs.gentoo.org/891655 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 4 +- ...30119.ebuild => gcc-10.4.1_p20230119-r1.ebuild} | 8 ++-- ...30120.ebuild => gcc-11.3.1_p20230120-r1.ebuild} | 8 ++-- sys-devel/gcc/gcc-12.2.1_p20230121-r1.ebuild | 52 ++++++++++++++++++++++ 4 files changed, 63 insertions(+), 9 deletions(-)
