After upgrading from gentoo-kernel-bin-6.6.51 to 6.6.57 and rebuilding sys-fs/zfs-kmod-2.2.5 I am unable to load the zfs module. modprobe zfs gives the following error message: modprobe: ERROR: could not insert 'zfs': Exec format error And in dmesg I find this message: module: x86/modules: Invalid relocation target, existing value is nonzero for type 1, loc 0000000008e0c66a, val ffffffffc07cea40 Same result if using ~amd64-keyworded sys-fs/zfs-kmod-2.2.6. sys-fs/zfs-kmod-2.2.5 worked fine with 6.6.51. Various web searches suggests re-installing linux headers, but a rebuild of zfs-kmod after reinstalling sys-kernel/linux-headers-6.6-r1 gives the same result. Reproducible: Always Steps to Reproduce: 1. emerge sys-kernel/gentoo-kernel-bin-6.6.57 2. Rebuild sys-fs/zfs-kmod-2.2.5 3. Update bootloader config to load new kernel and initramfs 4. reboot to new kernel Actual Results: zfs module is unable to load. Expected Results: zfs module should load in new kernel Portage 3.0.65 (python 3.12.6-final-0, default/linux/amd64/23.0/hardened/selinux/systemd, gcc-13, glibc-2.39-r6, 6.6.57- gentoo-dist x86_64) ================================================================= System uname: Linux-6.6.57-gentoo-dist-x86_64-Westmere_E56xx-L56xx-X56xx_-IBRS_update-with-glibc2.39 KiB Mem: 12242872 total, 11888524 free KiB Swap: 8388604 total, 8388604 free Timestamp of repository gentoo: Thu, 24 Oct 2024 15:45:00 +0000 Head commit of repository gentoo: 4593af3cb2493af337f0772572ce0b4c9c8cba33 Head commit of repository feffe: 95f22b285994c8fc4f2ddc558cdb03b559888a02 sh bash 5.2_p37 ld GNU ld (Gentoo 2.42 p6) 2.42.0 ccache version 4.10.1 [enabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p37::gentoo dev-build/autoconf: 2.72-r1::gentoo dev-build/automake: 1.16.5-r2::gentoo dev-build/cmake: 3.30.5::gentoo dev-build/libtool: 2.4.7-r4::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.5.2::gentoo dev-lang/perl: 5.40.0::gentoo dev-lang/python: 3.12.6_p4::gentoo dev-util/ccache: 4.10.1::gentoo sec-policy/selinux-base: 2.20240916-r1::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/sandbox: 2.39::gentoo sys-apps/systemd: 255.11::gentoo sys-devel/binutils: 2.42-r2::gentoo sys-devel/binutils-config: 5.5.2::gentoo sys-devel/gcc: 13.3.1_p20240614::gentoo sys-devel/gcc-config: 2.11::gentoo sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.39-r6::gentoo sys-libs/libselinux: 3.6-r1::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage priority: -1000 volatile: True sync-rsync-verify-metamanifest: true sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: feffe location: /var/tmp/repos/feffe sync-type: git sync-uri: https://gitea.fulh.ax/feffe/feffe-portage-overlay.git masters: gentoo priority: 100 volatile: True sync-git-verify-commit-signature: false Binary Repositories: gentoo sync-uri: https://pkg.winterbird.org/gentoo ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -ut2003-demo -GIMPS -ETQW -f.lux -google-chrome -bestcrypt -MikroTik -sfpg -ubiquiti -SPS -PUEL-11 -Fl ightradar24 -Gameplay-Group-EULA -TeamViewer -NVIDIA-CUDA -geekbench -THINKTANKS -AMD-GPU-PRO-EULA -MakeMKV-EULA -2dboy- EULA -microsoft-edge -MTA-0.5 -Ookla -ut2003 -GOG-EULA -Introversion -supermicro -teamspeak5 -zi-labone -Synology -PUEL- 12 -Q3AEULA-20000111 -RAR -Intel-SDP -Vivaldi -CAPYBARA-EULA -OPERA-2018 -NVIDIA-SDK -PICO-8 -Primate-Plunge -PAPERS-PLE ASE -microsoft-azurevpnclient -Steam -Mojang -baudline -LOKI-EULA -JoyPixels -WPS-EULA -PassMark-EULA -FAH-EULA-2014 -ge nymotion -AnyDesk-TOS -worklog-assistant -Coherent-Graphics -Microsemi -Dell-EULA -android -Microsoft-vscode -Aseprite-E ULA -QUAKE4 -teamspeak3 -TIK" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=x86-64-v2 -mtune=intel" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d" CXXFLAGS="-O2 -pipe -march=x86-64-v2 -mtune=intel" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--usepkg --with-bdeps=y --binpkg-respect-use=y --binpkg-changed-deps=y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME " FCFLAGS="-O2 -pipe -march=x86-64-v2 -mtune=intel" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance binpkg-request-signature bin pkg-signing buildpkg buildpkg-live ccache config-protect-if-modified distlocks ebuild-locks fixlafiles getbinpkg gpg-kee palive ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pkgdir-index-trusted preser ve-libs protect-owned qa-unresolved-soname-deps sandbox selinux sesandbox sfperms strict unknown-features-warn unmerge-l ogs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=x86-64-v2 -mtune=intel" GENTOO_MIRRORS="https://ftp.halifax.rwth-aachen.de/gentoo/ https://ftp.snt.utwente.nl/pub/os/linux/gentoo https://www.mi rrorservice.org/sites/distfiles.gentoo.org/" LANG="sv_SE.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs" LEX="flex" LINGUAS="sv en sv-SE en-GB en-US" MAKEOPTS="-j5 -l6" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file -- delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl amd64 audit berkdb bzip2 caps cet cjk crypt dbus device-mapper dist-kernel dnssec dracut fortran gdbm grub hard ened http2 iconv icu idn iproute2 ipset ipv6 lapack libtirpc lm-sensors lvm lz4 lzma multilib ncurses nginx nls nss open mp pam pcre pic pie pkcs11 policykit python readline ruby seccomp selinux spice sqlite ssh ssl ssp systemd test-rust ude v unicode usb usbredir vim-syntax vpx xattr xfs xtpax zfs zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_ core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile a uthz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_ca che filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id u serdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrd tool swap syslog" CPU_FLAGS_X86="aes mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROT OCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm1 04v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64 pc" GUILE_SINGLE_TARGET="3-0" GUI LE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="sv en sv-SE en-GB en-US" LCD_DEVICES="bayrad cfontz glk h d44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PH P_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARG ETS="ruby32" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTR A_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PO RTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Hm. Do you have pahole installed?
(In reply to Sam James from comment #1) > Hm. Do you have pahole installed? Yes, it seems I have dev-util/pahole-1.27-r1 installed. Not anything I'm familiar with, but looks like a dependency for zfs-kmod.
I was thinking of the changes we made in e.g. commit f3bb6bb643e02c3957e2f99cfefa1e2d3fd3c4cc Author: Sam James <sam@gentoo.org> Date: Thu Sep 5 12:50:50 2024 +0100 sys-kernel/gentoo-kernel-bin: BDEPEND on dev-util/pahole Without this, if CONFIG_DEBUG_INFO_BTF_MODULES is enabled, as it is by default since 49e0de80b0d0b9bd5767d438dae773194d838525, while g-k-bin will seem to install fine, it's installed incorrectly. It also leads to external modules later not being loadable with relocation errors from unhandled BTF: ``` [ 1648.352993] module: x86/modules: Invalid relocation target, existing value is nonzero for type 1, loc 00000000bc2721ed, val ffffffffc11e7850 ``` Thanks-to: Ionen Wolkens <ionen@gentoo.org> Reported-by: Sebastian Engel <sighunter@gmx.de> Signed-off-by: Sam James <sam@gentoo.org> but if it's installed, then it doesn't make sense to be hitting that same problem (it only happened if it *wasn't* installed, as BTF couldn't be processed).
Off the top of my head, the only thing I can possibly think of is pahole not being installed on a system where you built a zfs-kmod binpkg or something, but I don't see how that's doable with a fresh binpkg or anything built against a new just-built kernel anyway because of the eclass dependencies.
Well, it *is* on a package builder VM and it does auto-update, so it wouldn't be entirely unlikely that something happened in the wrong order leading to this situation. But during the troubleshooting I disabled binary packages (they were on the unavailable zfs filesystem anyway). Just to be safe I rebuilt the packages again in this order: 1. pahole 2. zfs-kmod 3. gentoo-kernel-bin 4. zfs-kmod No luck after 2 or 3, but after rebuilding zfs-kmod the second time the module loaded successfully! So my best guess is that pahole was needed when gentoo-kernel-bin was originally upgraded but wasn't pulled in until afterwards for some reason?
Some additional info: I'm using this VM to build binpkgs for a few profiles. The packages are build periodically in chroots in the VM. The packages are stored locally and when the nightly auto update for the VM runs it will install binpkg from the local storage if the packages has already been built. I hadn't bothered to exclude binary ebuilds (such as gentoo-kernel-bin) from building binpkg, so when the VM was upgraded it most likely used a binpkg built from within one of the chroots. Since the quoted commit sets pahole as a BDEPEND to gentoo-kernel-bin I guess this is likely to be related. For now I will exclude both gentoo-kernel-bin and zfs-kmod from building and using binpkg.
(In reply to Fredrik Eriksson from comment #5) > So my best guess is that pahole was needed when gentoo-kernel-bin was > originally upgraded but wasn't pulled in until afterwards for some reason? I'd like to say that this shouldn't happen because it BDEPEND on pahole.. but only 6.11.4 and 6.11.5 has it in BDEPEND (6.6.57 does not). So.. that's likely what happened.
Hi, Just to confirm this issue, unrelated to zfs, I need out-of-tree kernel modules for my laptop webcam that I rebuild every kernel upgrade and upgrading to 6.6.57 lead to all modules being built successfully but all refusing to load with the error Exec format error. After installing pahole and re-emerging gentoo-kernel-bin and @module-rebuild, everything works fine again. So yeah, pahole presence seems required to build any module on this version while it wasn't previously.
Thanks. I'll be fixing the dep shortly.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=389519b911cb85e7138045bb202c5109a1d532a8 commit 389519b911cb85e7138045bb202c5109a1d532a8 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-10-25 21:24:14 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-10-25 21:24:14 +0000 sys-kernel/gentoo-kernel-bin: add pahole BDEPEND Same as we did in f3bb6bb643e02c3957e2f99cfefa1e2d3fd3c4cc but we missed this for 6.6.x when we made the change to default to debug there too recently. Closes: https://bugs.gentoo.org/942087 Signed-off-by: Sam James <sam@gentoo.org> ...oo-kernel-bin-6.6.57.ebuild => gentoo-kernel-bin-6.6.57-r1.ebuild} | 3 ++- ...oo-kernel-bin-6.6.58.ebuild => gentoo-kernel-bin-6.6.58-r1.ebuild} | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-)
