Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 816024 - linux-mod.eclass: automatically override compiler and linker to support clang built kernels
Summary: linux-mod.eclass: automatically override compiler and linker to support clang...
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2021-10-03 09:41 UTC by Adel KARA SLIMANE
Modified: 2023-08-12 15:45 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adel KARA SLIMANE 2021-10-03 09:41:41 UTC
Hello,

Clang built kernels are becoming mainstream now (including with the now upstream LTO option) and most modules in Gentoo's repository do not work out of the box for them. Although some of them do, like nvidia-drivers, and I started opening PRs to implement the support in the modules I use [1,2]. And this is the first option for such support.

A second option one may think of is to simply pass CC and LD env vars so it works, it does not look like it works with every ebuild [1]. Although I haven't explored all the possibilities to figure out a working one.

I am opening this bug report to suggest a third option : override the compiler and linker in the linux-mod eclass so things just work :TM: 

One potential downside to third option, is to think about how to enable the user to override that override. But, as far as I know, there is no use to let the user decide the compiler and linker for the kernel modules, as they must be exactly the same as the ones used while building the target kernel.

The code snippet that seem to work within ebuilds, taken from nvidia-drivers, is the following

if linux_chkconfig_present CC_IS_CLANG; then
  BUILD_PARAMS+=' CC=${CHOST}-clang'
  if linux_chkconfig_present LD_IS_LLD; then
    BUILD_PARAMS+=' LD=ld.lld'
    if linux_chkconfig_present LTO_CLANG_THIN; then
      # kernel enables cache by default leading to sandbox violations
      BUILD_PARAMS+=' ldflags-y=--thinlto-cache-dir= LDFLAGS_MODULE=--thinlto-cache-dir='
    fi
  fi
fi

I suppose this snippet could be copied to linux-mod.eclass, with more checks for other linkers maybe ? I do not know how things are currently handled when GCC + a different linker are used.


What do you guys think of this proposal ?


[1] https://github.com/gentoo/gentoo/pull/22143
[2] https://github.com/gentoo/gentoo/pull/22456
Comment 1 Ionen Wolkens gentoo-dev 2021-10-03 10:11:47 UTC
I don't intend to work on this myself but, as I mentioned on a PR, just to say that I don't believe some variables I used are safe to enable globally. thinlto workarounds are notably rather specific (may be simpler to not support it for now), and some ebuilds are already be setting CC/LD too.

Albeit as long as enabled in steps like I did, this doesn't set anything unless the option is enabled in the kernel and hopefully shouldn't make things worse.

Idea was also to try to match kernel toolchain /only/ for the modules and not the entire nvidia-drivers ebuild which doesn't play that well with it.

Also haven't looked at this in a while, there may be better ways.
Comment 2 Ionen Wolkens gentoo-dev 2021-10-03 10:17:05 UTC
(In reply to Ionen Wolkens from comment #1)
> Also haven't looked at this in a while, there may be better ways.
nvidia-drivers also ignores clang slot and so on, just assumes current is "same as kernel", which maybe isn't so bad in this case but well. I left an "experimental" notice in the ebuild given I barely test it.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-05-11 15:47:06 UTC
*** Bug 859553 has been marked as a duplicate of this bug. ***
Comment 4 Larry the Git Cow gentoo-dev 2023-05-29 13:05:48 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d9687a4df038382187300d6f44230661ff5bc377

commit d9687a4df038382187300d6f44230661ff5bc377
Author:     Ionen Wolkens <ionen@gentoo.org>
AuthorDate: 2023-05-23 09:25:02 +0000
Commit:     Ionen Wolkens <ionen@gentoo.org>
CommitDate: 2023-05-29 13:03:27 +0000

    linux-mod-r1.eclass: new eclass, rewrite of linux-mod.eclass
    
    Here's a rough overview of -r0 -> -r1 differences with occasional
    rationale behind them if felt relevant (for migrating, refer to
    the eclassdocs instead as this does not really document usage
    changes):
    
    Features that did not exist in previous eclass (not exhaustive):
    * automatic modules signing support, been often requested and
      users would instead use messy bashrc hacks to accomplish this
     (enabled with USE=modules-sign)
    * modules (manual) stripping support to allow stripping *before*
      signing and compression
     (can be disabled with USE=-strip)
    * can auto-select toolchain to match kernel, e.g. if built with
      clang-15 then it won't use gcc nor clang-16 if possible
     (will warn if the matching compiler went missing)
    * helper functions to profit from the above 3 even if not using
      linux-mod-r1_src_compile+install (e.g. for zfs-kmod)
    * generic supported kernel version checks (min/max) which comes with
      an encouragement to use LTS kernels for out-of-tree modules
     (but max is not enforced, just makes a strong suggestion)
    * linux-mod-r1_src_install now does einstalldocs
    * can guess some common build targets rather than just 'module',
      largely removing the need for BUILD_TARGETS
    * user-oriented MODULES_EXTRA_EMAKE among other few variables
    * various additional sanity checks hopefully making issues
      clearer for users and ebuilds a bit harder to write wrong
    
    "Features" that existed but were not kept (not exhaustive):
    * support for <kernel-2.6(!) modules, eclass only tested with >=4.14.x
    * allowing doing all in global scope using variables ran through `eval`
     (this often led to all sort of variable misuse in global scope)
    * MODULESD_* support, originally meant to keep but it is used by only
      5 packages and holds very little meaning that I can see even in these
     (when needed, packages should write their own .conf)
    * moduledb, was being updated for nothing in postinst/postrm
      despite the tool that can use this (sys-kernel/module-rebuild)
      being gone from the tree since Feb 2014
    * convert_to_m(), only 1 in-tree ebuild uses this right now (svgalib)
    * various other functions with no consumers were dropped, some
      were likely meant to be @INTERNAL, get-KERNEL_CC was never used
      either and now there's ${KERNEL_CC}
    * running 'clean' by default, this sometime led to race conditions by
      attempting to clean and build at same time in e.g. nvidia-drivers
     (if an ebuild truly need this, it can be specified manually)
    * INSTALL_MOD_PATH support, this integrates badly with ebuilds that
      use it as DESTDIR with e.g. `make INSTALL_MOD_PATH="${ED}" install`
      or find "${ED}"/lib/modules, etc... requiring extra consideration
     (also feels inconsistent with how ebuilds normally work, the few
      concerned users may be interested by setting ROOT or manually moving
      files instead -- but support was missing until 2021 so it should
      have little impact)
    * BUILD_FIXES support, this is set by linux-info.eclass but has no
      real relevance that I can see (ebuilds have sometime wrongly used it)
    * undocumented feature CONFIG_CHECK="@CONFIG:modname" (or so?) meant
      for automagic based on kernel config is no longer supported, this
      also removes the also undocumented MODULE_IGNORE used by it (found
      0 ebuilds using these in the tree, can be done manually if needed)
    * converting CONFIG_CHECK to non-fatal for running again on binary
      merge when (while *possible*) it's rather unlikely would build
      modules for a different kernel than the one that will be used
    * having preinst and postrm exports, removed
      -> originally wanted to remove pkg_setup too but it complicated
         things with linux-info's own pkg_setup and made the eclass
         feel less convenient and error-prone with environment handling
    
    Dependency changes:
    * virtual/libelf DEPEND removed, building objtool (which uses this) is
      not handled by the eclass and does not seem auto-built by make if
      missing, as such the dependency is not used *here* but rather by
      dist-kernels and source packages which both already request it.
    * sys-apps/kmod[tools] BDEPEND+IDEPEND added, and removed from DEPEND
      (linux-mod-r0 uses it similarly but lacks the eapi7+8 adjustment)
    * modules-sign? ( dev-libs/openssl virtual/pkgconfig ) BDEPEND for
      building sign-file, unlike objtool it may need rebuilds for openssl
      and is handled here
    * dependencies are no longer guarded by "kernel_linux? ( )", it only
      served to silence pkgcheck and then give build failures (linux-only
      ebuilds should be masked on non-Linux profiles or, if mixed, use a
      masked MODULES_OPTIONAL_IUSE which *can* be kernel_linux).
    
    Tentative changes:
    * drop KERNEL_ABI support, (nowadays) kernel seems to append its own
      -m32/-m64 and should be no need for multilib.eclass complications
     (tested to work *at least* with x32[userland]+64bit[kernel])
    * ^ but add hppa2.0->64 kgcc64 switching like kernel-build.eclass
    * drop addpredict wrt bug #653286, assuming no longer relevant given
      unable to reproduce even with kernel-4.14.315+split-debug+some misc
      modules, perhaps would with spl but that (removed) ebuild is too
      broken to try
    
    Misc changes:
    * misc -> extra default install dir, to match the kernel's defaults
     (this is also where zfs-kmod already installs due to that default)
    
    Three bugs were addressed, but not closing given -r0 remains affected:
    * bug #447352: modules signing is supported
    * bug #759238: arguably not an issue anymore in -r0 either due to
      CHECKCONFIG_DONOTHING=1 (bug #862315) now existing, but -r1
      additionally makes it non-fatal if a whitelist exists in the kernel
    * bug #816024: trying to select toolchain better is a -r1 highlight
    
    Additionally, becomes WONTFIX in this version:
    * bug #642240: INSTALL_MOD_PATH support is reverted
    
    Bug: https://bugs.gentoo.org/447352
    Bug: https://bugs.gentoo.org/642240
    Bug: https://bugs.gentoo.org/759238
    Bug: https://bugs.gentoo.org/816024
    Closes: https://github.com/gentoo/gentoo/pull/31154
    Signed-off-by: Ionen Wolkens <ionen@gentoo.org>

 eclass/linux-mod-r1.eclass | 1206 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 1206 insertions(+)
Comment 5 Mike Pagano gentoo-dev 2023-08-12 15:45:49 UTC
linux-mod-r1.eclass now supports clang.  Packages have all been directed to migrate to this new eclass