Yeah I know, I thought I was mad too. Running genkernel (at least with my config) invokes ccache (as its "cc") and that replaces /dev/null with a file. Using the steps below, I think I've found half of the problem: 151075 execve("/usr/lib/ccache/bin/cc", ["/usr/lib/ccache/bin/cc", "-Werror", "-D__SIZEOF_INT128__=0", "-S", "-x", "c", "/dev/null", "-o", "/dev/null"], 0x55c8400f6000 /* 142 vars */) = 0 That's the last mention of "/dev/null" before the first occurrence of stat returning it as a file, not a char device. I'm just not sure which bits of the packages mentioned are actually at fault. I've set it to major as replacement of /dev/null with these permissions practically cripples everything for users until fixed. Reproducible: Always Steps to Reproduce: 1. ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Mar 10 19:36 /dev/null 2. strace -fo /var/tmp/str genkernel --kernel-outputdir=/var/tmp/kernel/build all * Gentoo Linux Genkernel; Version 70 * Running with options: --kernel-outputdir=/var/tmp/kernel/build all * Using genkernel.conf from /etc/genkernel.conf * Sourcing arch-specific config.sh from /usr/share/genkernel/arch/x86_64/config.sh .. * Sourcing arch-specific modules_load from /usr/share/genkernel/arch/x86_64/modules_load .. * Linux Kernel 5.5.8-gentoo for x86_64... * .. with config file /usr/share/genkernel/arch/x86_64/kernel-config * kernel: --mrproper is disabled; not running 'make mrproper'. * >> Running oldconfig... (this doesn't need to run to completion) 3. ls -l /dev/null -rw-r--r-- 1 root root 0 Mar 10 19:37 /dev/null I'll attach configuration files shortly. Portage 2.3.93 (python 3.6.10-final-0, default/linux/amd64/17.0/desktop/plasma/systemd, gcc-9.2.0, glibc-2.30-r4, 5.5.5-gentoo x86_64) ================================================================= System uname: Linux-5.5.5-gentoo-x86_64-Intel-R-_Core-TM-_i7-3820_CPU_@_3.60GHz-with-gentoo-2.7 KiB Mem: 8134516 total, 1371820 free KiB Swap: 8388604 total, 8388604 free Timestamp of repository gentoo: Tue, 10 Mar 2020 18:57:01 +0000 Head commit of repository gentoo: faeb40151010875e46c45ec83d37eb9577ae24c8 sh bash 5.0_p16 ld GNU ld (Gentoo 2.34 p1) 2.34.0 distcc 3.3.3 x86_64-pc-linux-gnu [disabled] ccache version 3.7.7 [disabled] app-shells/bash: 5.0_p16::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.30.1::gentoo dev-lang/python: 2.7.17-r1::gentoo, 3.6.10::gentoo, 3.7.6::gentoo, 3.8.2::gentoo dev-util/ccache: 3.7.7-r1::gentoo dev-util/cmake: 3.16.5::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.16.1-r2::gentoo sys-devel/binutils: 2.34::gentoo sys-devel/gcc: 7.4.0-r2::gentoo, 9.2.0-r4::gentoo sys-devel/gcc-config: 2.2.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.5::gentoo (virtual/os-headers) sys-libs/glibc: 2.30-r4::gentoo Repositories: gentoo location: /usr/portage sync-type: git sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 Octal location: /home/randomdan/dev/octal/portage masters: gentoo priority: 10 RandomLAN location: /home/randomdan/dev/portage masters: gentoo priority: 10 jorgicio location: /var/lib/layman/jorgicio masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE linux-fw-redistributable no-source-code vim.org MSttfEULA free-noncomm CC-Sampling-Plus-1.0 PUEL NVIDIA-CUDA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O3 -march=native -pipe -fno-stack-protector" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O3 -march=native -pipe -fno-stack-protector" DISTDIR="/usr/portage/distfiles" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://portage.randomdan.homeip.net/ http://mirrorsearch.randomdan.homeip.net/ http://distfiles.gentoo.org/" LANG="en_GB.utf8" LC_ALL="en_GB.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -O3 -march=native -pipe" LINGUAS="en en_GB" MAKEOPTS="-j6 -l6" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 acl acpi activities alsa amd64 apng autoshutdown autoupdate bash-completion berkdb bluetooth branding btrfs bzip2 c++0x cairo caps cdda cddb cdr clang cli cscope css curl custom-cflags custom-optimization cxx dbus declarative device-mapper dot dri dts dv dvb dvd dvdr emboss encode evdev exif fam ffmpeg firmware flac fontconfig forticlient frei0r fuse gallium gcrypt gdbm gif git glib gnome-keyring gnutls gold gpm graphviz grub gtk hpn http2 hwaccel iconv icu imagemagick inspector ipv6 ithreads javascript jemalloc jit jpeg jpeg2k json kipi kwallet lame lcms ldap libglvnd libkms libnotify libtirpc lightning llvm lm-sensors lvm lvmboot lvmroot lzma lzo mad man markdown minizip mng mp3 mp4 mpeg mpm-prefork mtp mysql ncurses network network-cron nls nohourlyupdate nptl nscd nuv offensive offload ogg openexr opengl openmp pam pango pch pcre pcre-jit pcre16 pcre32 pdf phonon plasma plymouth png policykit postgres postproc ppds python qml qt5 readline rtmp savedconfig seccomp secure-delete silc smp sndfile spell split-usr sqlite ssl startup-notification svg symlink syslog system-boost system-cairo system-cmark system-crontab system-ffmpeg system-harfbuzz system-heimdal system-icu system-jpeg system-jsoncpp system-libevent system-libmspack system-libs system-libvpx system-libyaml system-llvm system-lua system-qemu system-seabios system-sqlite system-uulib system-wide systemd taglib tcpd threads thumbnail tiff truetype udev udisks unicode upower usb user-session uuid vaapi vdpau vim-pager vim-syntax virtualbox vlc vorbis wayland widevine widgets wxwidgets x264 xa xattr xcb xcomposite xinerama xml xmp xrandr xv xvid xvmc zip zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="filter authz_host cgid mime_magic mime autoindex vhost_alias dav dir deflate log_config env rewrite cache mem_cache unixd socache authz_core authn_core socache_shmcb http2 alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="none" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="NVPTX X86 WebAssembly" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python3_6" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 617946 [details] genkernel configuration
Created attachment 617948 [details] "ccache -p" as root
Created attachment 617950 [details] Kernel .config
No, it's this... just haven't figured out yet what process that is. 151070 rename("/dev/null", "/dev/null.rm.akira.151070.XXXXXX") = 0 151070 unlink("/dev/null.rm.akira.151070.XXXXXX") = 0
Ah, basically the same. 151070 execve("/usr/lib/ccache/bin/cc", ["/usr/lib/ccache/bin/cc", "-Werror", "-Wmaybe-uninitialized", "-S", "-x", "c", "/dev/null", "-o", "/dev/null"], 0x561819dbd000 /* 142 vars */) = 0
Runing as user command $ strace -y -f /usr/lib/ccache/bin/cc -Werror -Wmaybe-uninitialized -S -x c /dev/null -o /dev/null |& fgrep dev/null |& fgrep -i unli I don't see an attempt to delete /dev/null. Do you observe that command to delete a file for you? If it does please attach full strace output.
Created attachment 619256 [details] strace when run as root See line 450, rename of /dev/null.
Created attachment 619258 [details] strace when run as user It's different by paths and addresses, but vimdiff can match enough to make sense. The rename at 450 is never attempted.
Yeah, it's a ccache's deficiency. genkernel-next should not run builds as root either though.
I guess genkernel itself is also able to trigger that because we don't use sandbox when building kernel.
Looks like it's specific to 'hard_link = true' option.
Filed https://github.com/ccache/ccache/issues/564 for ccache upstream. But even if it gets fixed I would expect more problems like that if you run builds as root.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=650d7b15773aea3eda20f2b870ad3266ddeefa6d commit 650d7b15773aea3eda20f2b870ad3266ddeefa6d Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2020-03-24 22:27:06 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2020-03-24 22:29:55 +0000 dev-util/ccache: backport workaround for /dev/null clobber In bug #712080 'hard_link = true' mode of ccache unlinked /dev/null as ccache was running as root. This workaround allows /dev/null to survive. Other special files still stay affected. Bug: https://bugs.gentoo.org/712080 Package-Manager: Portage-2.3.95, Repoman-2.3.21 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> dev-util/ccache/ccache-3.7.8-r1.ebuild | 71 +++++++++++++++++++++++ dev-util/ccache/files/ccache-3.7.8-dev-null.patch | 57 ++++++++++++++++++ 2 files changed, 128 insertions(+)
Removing toolchain@ from CC. Fell free to readd for further ccache-related questions.
Package removed.
