Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 870001 - sys-devel/clang: doesn't include <stdc-predef.h> (dev-libs/libedit: fails to compile with Clang 15 (and musl?) (error: wchar_t must store ISO 10646 characters))
Summary: sys-devel/clang: doesn't include <stdc-predef.h> (dev-libs/libedit: fails to ...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: LLVM support project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: systemwide-clang musl-porting
  Show dependency tree
 
Reported: 2022-09-13 18:09 UTC by Sam James
Modified: 2022-11-18 08:02 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (file_870001.txt,45.56 KB, text/plain)
2022-09-13 18:09 UTC, Sam James
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-09-13 18:09:45 UTC
Created attachment 804988 [details]
build.log

libtool: compile:  clang -DHAVE_CONFIG_H -I. -I/var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src -I.. -O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches -c /var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/vis.c  -fPIC -DPIC -o .libs/vis.o
In file included from /var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/el.c:60:
In file included from /var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/el.h:50:
/var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/chartype.h:46:3: error: wchar_t must store ISO 10646 characters
        #error wchar_t must store ISO 10646 characters
         ^
In file included from /var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/chared.c:51:
In file included from /var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/el.h:50:
/var/tmp/portage/dev-libs/libedit-20210419.3.1/work/libedit-20210419-3.1/src/chartype.h:46:3: error: wchar_t must store ISO 10646 characters
        #error wchar_t must store ISO 10646 characters
         ^
[...]

----

# emerge --info
!!! No gcc found. You probably need to 'source /etc/profile'
!!! to update the environment of this terminal and possibly
!!! other terminals also.
Portage 3.0.36 (python 3.10.7-final-0, default/linux/amd64/17.0/musl/clang, [unavailable], musl-1.2.3-r3, 5.15.67-gentoo-dist-hardened x86_64)
=================================================================
System uname: Linux-5.15.67-gentoo-dist-hardened-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-libc
KiB Mem:    65765036 total,  31630744 free
KiB Swap:    8290300 total,   6431228 free
Timestamp of repository gentoo: Tue, 13 Sep 2022 18:02:01 +0000
sh dash 0.5.11.5
app-misc/pax-utils:        1.3.5::gentoo
app-shells/bash:           5.1_p16-r2::gentoo
dev-lang/perl:             5.36.0::gentoo
dev-lang/python:           3.10.7::gentoo, 3.11.0_rc2::gentoo
dev-util/cmake:            3.24.1::gentoo
dev-util/meson:            0.63.2-r1::gentoo
sys-apps/baselayout:       2.8-r2::gentoo
sys-apps/openrc:           0.45.2::gentoo
sys-apps/sandbox:          2.29::gentoo
sys-devel/autoconf:        2.71-r2::gentoo
sys-devel/automake:        1.16.5::gentoo
sys-devel/binutils:        2.39-r2::gentoo
sys-devel/binutils-config: 5.4.1::gentoo
sys-devel/clang:           13.0.1::gentoo, 14.0.6-r1::gentoo, 15.0.0::gentoo
sys-devel/gcc-config:      2.5-r1::gentoo
sys-devel/libtool:         2.4.7::gentoo
sys-devel/lld:             13.0.1::gentoo
sys-devel/llvm:            13.0.1::gentoo, 14.0.6-r2::gentoo, 15.0.0::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.19::gentoo (virtual/os-headers)
sys-libs/musl:             1.2.3-r3::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: yes

Installed sets: @sam
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
ADDR2LINE="llvm-addr2line"
AR="llvm-ar"
AS="clang -c"
CBUILD="x86_64-gentoo-linux-musl"
CC="clang"
CFLAGS="-O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches"
CHOST="x86_64-gentoo-linux-musl"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CPP="clang-cpp"
CXX="clang++"
CXXFLAGS="-O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going=y --complete-graph"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
INSTALL_MASK="charset.alias /usr/share/locale/locale.alias"
LANG="C.UTF8"
LD="ld.lld"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind -Wl,--as-needed  -Wl,--defsym=__gentoo_check_ldflags__=0"
NM="llvm-nm"
OBJCOPY="llvm-objcopy"
OBJDUMP="llvm-objdump"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RANLIB="llvm-ranlib"
READELF="llvm-readelf"
STRINGS="llvm-strings"
STRIP="llvm-strip"
USE="acl amd64 bzip2 clang cli crypt dri fortran iconv ipv6 libglvnd libtirpc llvm-libunwind ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl timidity unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ARFLAGS, ASFLAGS, CCLD, CONFIG_SHELL, CPPFLAGS, CTARGET, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS, SHELL, SIZE, YACC, YFLAGS
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-09-13 21:16:36 UTC
Arsen helped dig into this and the issue is that Clang never ends up getting #include <stdc-predef.h> on e.g. musl systems. On glibc systems, it's fine, because /usr/include/features.h contains it.

See:
- https://www.openwall.com/lists/musl/2021/04/16/8
- https://reviews.llvm.org/D34158
- https://reviews.llvm.org/D106577
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2022-09-19 10:22:39 UTC
I'm a bit confused how this works, to be honest.

FWICS on glibc targets, GCC preincludes <stdc-predef.h>.  However, this seems specific to glibc -- so do we patch GCC on musl to do that as well, or does this rule get applied to musl target as well?

Furthermore, glibc has the <features.h> fallback-include for this file.  Since clang doesn't do what GCC does, apparently it relies on this fallback on glibc systems.

Now, FWICS musl also includes such a file and expects the compiler to preinclude it.  However, it does not include the fallback which basically means that it's going to cause issues on every compiler that doesn't preinclude it.

From the linked threads I can only establish the following:

- musl upstream shifts blame to clang (so effectively every compiler that doesn't follow some magic code?)

- the original patch to include <stdc-predef.h> in clang got reverted because of some test failures and because apparently "it broke chromium build", with no info how it was broken and no reply to the questions asked there

- the newer patch to define the macro directly got stuck on some standards-related discussion, and is waiting "to hear back" for a year now

So we're basically stuck between people throwing blame around and not caring at all about user systems working.
Comment 3 Arsen Arsenovic 2022-09-19 14:38:42 UTC
(In reply to Michał Górny from comment #2)
> I'm a bit confused how this works, to be honest.
See gccint link below.

> FWICS on glibc targets, GCC preincludes <stdc-predef.h>.  However, this
> seems specific to glibc -- so do we patch GCC on musl to do that as well, or
> does this rule get applied to musl target as well?
It's fine on Alpine:
/ # gcc -E -x c - </dev/null
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 0 "<command-line>" 2
# 1 "<stdin>"
/ # cat /etc/os-release 
NAME="Alpine Linux"

... and with Crossdev:
[i] ~$ </dev/null x86_64-linux-musl-gcc -E -x c -
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "/usr/x86_64-linux-musl/usr/include/stdc-predef.h" 1 3 4
# 0 "<command-line>" 2
# 1 "<stdin>"
[i] ~$ 

Though, I can't quite discern why (I could give looking at it throughly another try later, if so desired).

> Now, FWICS musl also includes such a file and expects the compiler to
> preinclude it.  However, it does not include the fallback which basically
> means that it's going to cause issues on every compiler that doesn't
> preinclude it.
> 
> From the linked threads I can only establish the following:
> 
> - musl upstream shifts blame to clang (so effectively every compiler that
> doesn't follow some magic code?)
The protocol for this falls into the libc<->compiler category and is pretty simple: put a header somewhere (and tell the compiler about it) so that macros that are supposed to be predefined are predefined. Glibc, as always, has a workaround for misbehaving compilers, but this shouldn't be needed ultimately.

IMO, this is a compiler concern. GCC does this per-target via a hook in gcc/gcc/config (see https://gcc.gnu.org/onlinedocs/gccint/Misc.html#index-TARGET_005fC_005fPREINCLUDE), and Clang could pretty easily do a similar/equivalent thing. A solution we could adopt between now and when Clang solves this issue is via -include in the config file.

PS: I'm not certain, but, if I had to guess, the Chromium build broke because Bionic didn't have the predef header or something, or Chromium relied on a macro in there to do some fixup.