| Summary: | sys-devel/gcc-12.2.0-r1: unexpected behavior change for flag -stdlib=libc++, fatal error: algorithm: No such file or directory | ||
|---|---|---|---|
| Product: | Gentoo/Alt | Reporter: | Tom Li <biergaizi2009> |
| Component: | Prefix Support | Assignee: | Gentoo Prefix <prefix> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | ||
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | OS X | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=905131 https://github.com/iains/gcc-12-branch/issues/21 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 886491 | ||
Confirmed. GCC 12.1.0 is not affected by this bug, it's indeed a regression between 12.1.0 and 12.2.0:
bash-3.2$ cat main.cpp
#include <vector>
int main(void)
{
}
bash-3.2$ g++-12.1.0 main.cpp -o main -stdlib=libc++
g++-12.1.0: error: unrecognized command-line option '-stdlib=libc++'
bash-3.2$ g++-12.2.0 main.cpp -o main -stdlib=libc++
main.cpp:1:10: fatal error: vector: No such file or directory
1 | #include <vector>
| ^~~~~~~~
compilation terminated.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=67b3c82c7943e73f410e089797e4c961bace64a8 commit 67b3c82c7943e73f410e089797e4c961bace64a8 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-04-27 10:24:49 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-04-27 10:27:20 +0000 profiles/prefix/darwin/macos/package.mask: mask gcc-12.2 #905152 odd interaction with -stdlib=libc++ Bug: https://bugs.gentoo.org/905152 Signed-off-by: Fabian Groffen <grobian@gentoo.org> profiles/prefix/darwin/macos/package.mask | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) Just as I suspected, it looks like an unintended consequence of commit 2fd0917c ("configure, Darwin: Always add the stdlib option for modern OS versions "). [1] Previously, the "stdlib" option was disabled. But now stdlib is enabled, but apparently with a broken include path.
According to the source code gcc/cppdefault.cc [2], when --stdlib=libc++ is passed, it attempts to find the include files from GPLUSPLUS_LIBCXX_INCLUDE_DIR and GPLUSPLUS_LIBCXX_INCLUDE_DIR_ADD_SYSROOT.
> #ifdef GPLUSPLUS_LIBCXX_INCLUDE_DIR
> /* Pick up libc++ include files, if we have -stdlib=libc++. */
> { GPLUSPLUS_LIBCXX_INCLUDE_DIR, "G++", 2, 1,
> GPLUSPLUS_LIBCXX_INCLUDE_DIR_ADD_SYSROOT, 0 },
> #endif
So this path must be broken, at least according to the built-time option for Gentoo Prefix.
According to the source code gcc/configure.ac [3], the path for "-stdlib=libc++" is controlled by --with-gxx-libcxx-include-dir.
So it appears that all we need to do is set this build-time option for GCC.
> # Configuration for an alternate set of C++ headers.
> gcc_gxx_libcxx_include_dir=
> # Specify the alternate g++ header file directory
> AC_ARG_WITH(gxx-libcxx-include-dir,
> [AS_HELP_STRING([--with-gxx-libcxx-include-dir=DIR],
> [specifies directory to find libc++ header files])],
> [case "${withval}" in
> yes) AC_MSG_ERROR(bad value ${withval} given for libc++ include directory) ;;
> no) ;;
> *) gcc_gxx_libcxx_include_dir=$with_gxx_libcxx_include_dir ;;
> esac])
>
> # If both --with-sysroot and --with-gxx-libcxx-include-dir are passed, we
> # check to see if the latter starts with the former and, upon success, compute
> # gcc_gxx_libcxx_include_dir as relative to the sysroot.
> gcc_gxx_libcxx_include_dir_add_sysroot=0
>
> if test x${gcc_gxx_libcxx_include_dir} != x; then
> AC_DEFINE(ENABLE_STDLIB_OPTION, 1,
> [Define if the -stdlib= option should be enabled.])
> else
> case $target in
> *-darwin1[[1-9]]* | *-darwin2*)
> # Default this on for Darwin versions which default to libcxx.
> AC_DEFINE(ENABLE_STDLIB_OPTION, 1)
> ;;
> *)
> AC_DEFINE(ENABLE_STDLIB_OPTION, 0)
> ;;
> esac
> fi
[1] https://github.com/iains/gcc-12-branch/commit/2fd0917c
[2] https://github.com/iains/gcc-12-branch/blob/gcc-12-2-darwin/gcc/cppdefault.cc#L58
[3] https://github.com/iains/gcc-12-branch/blob/gcc-12-2-darwin/gcc/configure.ac#L231.
Without --stdlib=libc++, the search path is: gcc version 12.2.0 (Gentoo 12.2.0-r1 p1) COLLECT_GCC_OPTIONS='-E' '-v' '-mmacosx-version-min=13.0.0' '-asm_macosx_version_min=13.0' '-nodefaultexport' '-mtune=core2' /Users/ec2-user/gentoo/tmp/usr/libexec/gcc/x86_64-apple-darwin22/12.2.0/cc1plus -E -quiet -v -D__DYNAMIC__ - -fPIC -mmacosx-version-min=13.0.0 -mtune=core2 -dumpbase - ignoring nonexistent directory "/Users/ec2-user/gentoo/include" ignoring nonexistent directory "/Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/../../../../x86_64-apple-darwin22/include" #include "..." search starts here: #include <...> search starts here: /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12 /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12/x86_64-apple-darwin22 /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12/backward /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include /Users/ec2-user/gentoo/tmp/usr/include /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include-fixed /Users/ec2-user/gentoo/tmp/MacOSX.sdk/usr/include /Users/ec2-user/gentoo/tmp/MacOSX.sdk/System/Library/Frameworks /Users/ec2-user/gentoo/tmp/Frameworks /System/Library/Frameworks /Library/Frameworks End of search list. And with --stdlib=libc++, the search path is: gcc version 12.2.0 (Gentoo 12.2.0-r1 p1) COLLECT_GCC_OPTIONS='-stdlib=libc++' '-E' '-v' '-mmacosx-version-min=13.0.0' '-asm_macosx_version_min=13.0' '-nodefaultexport' '-mtune=core2' /Users/ec2-user/gentoo/tmp/usr/libexec/gcc/x86_64-apple-darwin22/12.2.0/cc1plus -E -quiet -v -D__DYNAMIC__ - -fPIC -mmacosx-version-min=13.0.0 -mtune=core2 -stdlib=libc++ -dumpbase - ignoring nonexistent directory "/Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/ibcxx_incdir" ignoring nonexistent directory "/Users/ec2-user/gentoo/include" ignoring nonexistent directory "/Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/../../../../x86_64-apple-darwin22/include" #include "..." search starts here: #include <...> search starts here: /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include /Users/ec2-user/gentoo/tmp/usr/include /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include-fixed /Users/ec2-user/gentoo/tmp/MacOSX.sdk/usr/include /Users/ec2-user/gentoo/tmp/MacOSX.sdk/System/Library/Frameworks /Users/ec2-user/gentoo/tmp/Frameworks /System/Library/Frameworks /Library/Frameworks End of search list. Note how these three paths have disappeared: /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12 /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12/x86_64-apple-darwin22 /Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12/backward GCC was built with these options: Using built-in specs. COLLECT_GCC=gcc-12.2.0 COLLECT_LTO_WRAPPER=/Users/ec2-user/gentoo/tmp/usr/libexec/gcc/x86_64-apple-darwin22/12.2.0/lto-wrapper Target: x86_64-apple-darwin22 Configured with: /Users/ec2-user/gentoo/tmp/var/tmp/portage/sys-devel/gcc-12.2.0-r1/work/gcc-12-branch-gcc-12.2-darwin-r0/configure --host=x86_64-apple-darwin22 --build=x86_64-apple-darwin22 --prefix=/Users/ec2-user/gentoo/tmp/usr --bindir=/Users/ec2-user/gentoo/tmp/usr/x86_64-apple-darwin22/gcc-bin/12.2.0 --includedir=/Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include --datadir=/Users/ec2-user/gentoo/tmp/usr/share/gcc-data/x86_64-apple-darwin22/12.2.0 --mandir=/Users/ec2-user/gentoo/tmp/usr/share/gcc-data/x86_64-apple-darwin22/12.2.0/man --infodir=/Users/ec2-user/gentoo/tmp/usr/share/gcc-data/x86_64-apple-darwin22/12.2.0/info --with-gxx-include-dir=/Users/ec2-user/gentoo/tmp/usr/lib/gcc/x86_64-apple-darwin22/12.2.0/include/g++-v12 --with-python-dir=/share/gcc-data/x86_64-apple-darwin22/12.2.0/python --enable-languages=c,c++,objc,obj-c++ --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --disable-nls --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 12.2.0-r1 p1' --disable-esp --enable-libstdcxx-time --disable-libstdcxx-pch --enable-shared --enable-threads=posix --with-local-prefix=/Users/ec2-user/gentoo/usr --with-native-system-header-dir=/Users/ec2-user/gentoo/tmp/MacOSX.sdk/usr/include --disable-multilib --with-multilib-list=m64 --disable-fixed-point --enable-version-specific-runtime-libs --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --without-isl --disable-libsanitizer --enable-default-pie --enable-default-ssp --disable-bootstrap --with-local-prefix=/Users/ec2-user/gentoo --disable-darwin-at-rpath Thread model: posix Supported LTO compression algorithms: zlib gcc version 12.2.0 (Gentoo 12.2.0-r1 p1) I just replicated the problem with Homebrew. Time to report the problem at upstream!
homebrew@ip-172-31-28-139 ~ % cat test.cc
#include <vector>
int main()
{
}
homebrew@ip-172-31-28-139 ~ % /usr/local/Cellar/gcc@11/11.3.0/bin/g++-11 test.cc -o test -stdlib=libc++
g++-11: error: unrecognized command-line option '-stdlib=libc++'
homebrew@ip-172-31-28-139 ~ % /usr/local/Cellar/gcc/12.2.0/bin/g++-12 test.cc -o test -stdlib=libc++
test.cc:1:10: fatal error: vector: No such file or directory
1 | #include <vector>
| ^~~~~~~~
compilation terminated.
Reported upstream: https://github.com/iains/gcc-12-branch/issues/21 To rephrase the problem: In the past, Darwin shipped an ancient C++ standard library by default, so most C++ programs needed to be compiled with the option stdlib=libc++ to use the newer library shipped on macOS. This option is only supported by clang on Darwin, and it's invalid for GCC. Thus, most ./configure scripts attempt to check whether -stdlib=libc++ is a valid build-time option using a dummy program - if it's the case, this flag is passed to the compiler. If it's not, it's omitted. Thus, when the program is compiled with (an up-to-date version of) GCC, it automatically links to native C++ runtime for g++. When the program is compiled with clang, it automatically links to Apple's new C++ library. Unfortunately, since GCC Darwin 12.2, now it recognizes the option stdlib=libc++ and attempts linking to Darwin's C++ library instead of GCC's native C++ library. As a result, compilation of many previously successful programs now may fail if C++ if libc++'s header files are not present on the system. Since a dummy program without any #include statement is used to check the existence of -stdlib=libc++, it escapes detection but later fails in the middle of the build. The minimum example is: bash-3.2$ cat main.cpp #include <vector> int main(void) { } bash-3.2$ g++-12.1.0 main.cpp -o main # OKAY bash-3.2$ g++-12.1.0 main.cpp -o main -stdlib=libc++ g++-12.1.0: error: unrecognized command-line option '-stdlib=libc++' bash-3.2$ g++-12.2.0 main.cpp -o main # OKAY bash-3.2$ g++-12.2.0 main.cpp -o main -stdlib=libc++ main.cpp:1:10: fatal error: vector: No such file or directory 1 | #include <vector> | ^~~~~~~~ compilation terminated. In the past, because -stdlib=libc++ was unsupported, the build system would remove -stdlib=libc++ from the CFLAGS and successfully building the program. But now, -stdlib=libc++ is passed to GCC, the compilation of the same program now fails due to missing libc++ system headers. Replicated on both Gentoo Prefix and Homebrew. It's not strictly a bug, but it's noteworthy enough to be reported here. To summarize, the old behaviors in a large number of build systems are:
* If clang is used, stdlib=libc++ is supported, add stdlib=libc++ to use Darwin's libc++. As a result, a clang build always uses libc++.
* If stdlib=libc++ is unsupported, don't add anything, as a result, GCC's native C++ library is used, as a result, a GCC build always uses libstdc++
But now, by supporting stdlib=libc++ in GCC, the behaviors are now:
* stdlib=libc++ is supported by both clang and GCC, as a result, programs that meant to use GCC's libstdc++is now suddenly using libc++, which is often be unintended. And sometimes the necessary headers for libc++ must not even present on the system, causing build failures with missing headers in previously working program.
For now, as a workaround, for Gentoo Prefix, we will use just --disable-stdlib-option to fallback to the old behaviors, which is at least well understood and doesn't create a new compatibility problem.
The old behavior is not perfect, of course, if clang and GCC are mixed when building software libraries, cross-linking libraries built by clang and built by GCC into the same binary can be a complete mess with a ton of conflicts. But the new behavior is equally a headache, since it broke the default assumption of many programs for deciding when to use libstdc++ or libc++. Both the old and new behaviors are equally problematic... Solution in unclear.
I opened a Pull Request at https://github.com/gentoo/prefix/pull/28 to revbump GCC 12.2 and disable "-stdlib=" for now. The commit message contains a detail explanation and analysis of the current situation, why it's needed to be disable, and some possible ways to properly support "-stdlib=" in the future. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=6aa53d70242b7dd91fb04d1dc42da75ca582f033 commit 6aa53d70242b7dd91fb04d1dc42da75ca582f033 Author: Yifeng Li <tomli@tomli.me> AuthorDate: 2023-04-27 18:59:44 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-04-29 07:15:56 +0000 sys-devel/gcc: revbump to disable "stdlib=" support on Darwin in 12.2.0-r2 Historically, Darwin included two C++ libraries, an outdated GCC and its libstdc++, and a new libc++ (from clang). The old library was the default. Thus, when building a modern C++ program using clang, the flag "stdlib=libc++" must be supplied to clang. On the other hand, when an updated GCC is used, it links to an updated GCC libstdc++ by default, no flag is needed. In fact, since "stdlib=libc++" cannot be recognized by GCC, it should not be used. As a result, many build scripts and build systems on macOS check if the flag "stdlib=libc++" is supported. If it's the case, the flag is used. If it's not, no flag is added. Practically, it creates the de-facto behavior of: If the program is built with clang, it's linked to clang's libc++. If the program is built with GCC, it's linked to GCC's native libstdc++. So far so good. This is also the most reasonable behavior, as expected by most users. As time passes, it was realized that using GCC and clang on the same system may create tricky conflicts, it's useful to be able to link against clang's libc++ even if GCC is used (this is useful beyond Darwin, e.g. for FreeBSD). Therefore, GCC now supports "stdlib=libc++ as well. The first immediate (but trivial) problem is a path problem. GCC's Darwin fork (maintained by Ian) enables stdlib= support by default (unlike upstream GCC that only conditionally enables it when an include path "gcc_gxx_libcxx_include_dir" is passed to GCC during build time). However, the default include path is invalid. Building a program with "stdlib=libc++" would fail since GCC cannot find basic C++ headers, like <vector> or <algorithm>. For example: main.cpp:1:10: fatal error: algorithm: No such file or directory 1 | #include <algorithm> | ^~~~~~~~ compilation terminated. In principle, setting the correct clang libc++ path on macOS would fix this problem, but it's far from our only trouble here. Unfortunately, this consequences of this change can be far-reaching and unexpected. In the past, if a program is compiled on clang, it's always linked to libc++, if a program is compiled with gcc, it's always linked to GCC's native libstdc++. But now this assumption has been broken when GCC starts supporting "stdlib=libc++". It means that now programs compiled by GCC will sometimes be linked to libc++, and sometimes be linked to libstdc++, depending on two factors: 1. If a program's build system does not make any attempt to detect the existence of "stdlib=libc++" when it's built with GCC, it will be linked to GCC's native libstdc++. This situation happens if the program was not ported to macOS in the past, or if the program is designed for macOS 10.9.x and newer systems (in which libc++ became clang's default). 2. If a program's build system attempts to detect the existence of "stdlib=libc++", it's now linked to clang's libc++ when it's built by GCC - when previously it would link to GCC's native libstdc++. Thus, when GCC is used, some programs would be linked to libstdc++, but others may suddenly be linked to clang's libc++. depending on the build system of the program. This would create surprising and somewhat unpredictable situations. The solution requires careful thought. There are four possibilities: 1. Disable stdlib= support, so the existing behavior is maintained (more importantly, maintained without the need to patch the build system of countless programs). The disadvantage is that end-users would lose the ability to use stdlib=libc++ to build their own programs when they found it's necessary. 2. Enable stdlib= support. This allows users to enjoy the interoperability benefits of "stdlib=libc++" should it be necessary. But now some programs would suddenly be linked to clang's libc++ when GCC is used, while others would still use GCC. This is unexpected and would be surprising to end-users. And Since Gentoo Prefix currently assumes a consistent GCC environment, it may potentially create compatibility problems as well. To maintain the historical behavior (programs built by GCC always links to GCC's libstdc++), we need to patch the "stdlib=libc++" out of the build system of countless programs... On the other hand, perhaps it's still doable, since many programs are moving away from "stdlib=libc++" as modern macOS defaults to libc++. 3. Enable stdlib= support, and pass stdlib=libstdc++ in the global CXXFLAGS of Portage. 4. Enable stdlib= support, and pass stdlib=libc++ in the global CXXFLAGS of Portage. The last two options sound more reasonable, perhaps in the future, it can be Portage's responsibility to decide which libc++ is used, just like how it currently can decide when to use GCC or clang. Since in GCC, new compiler flags can override previous flag, we can force a "stdlib=" choice to allow well-defined, predictable selection of C++ libraries. Option 2 or 1 is also imaginable, depending on the circumstances. But we clearly need further considerations. For now, we choose the simplest solution, disable support for "stdlib=" to maintain the existing behavior, at least as a stop-gap solution. This may change in the future. Closes: https://bugs.gentoo.org/905152 Signed-off-by: Yifeng Li <tomli@tomli.me> Closes: https://github.com/gentoo/prefix/pull/28 Signed-off-by: Sam James <sam@gentoo.org> ...cc-12.2.0-disable-stdlib-option-on-darwin.patch | 151 +++++++++++++++++++++ .../{gcc-12.2.0-r1.ebuild => gcc-12.2.0-r2.ebuild} | 11 ++ 2 files changed, 162 insertions(+) |
It seems that GCC 12.2 no longer rejects "-stdlib=libc++" as an invalid flag, instead, it prevents GCC from locating the C++ standard library. bash-3.2$ cat main.cpp #include <vector> int main(void) { } bash-3.2$ g++ main.cpp -o main -stdlib=libc++ main.cpp:1:10: fatal error: vector: No such file or directory 1 | #include <vector> | ^~~~~~~~ compilation terminated. The side-effect of this change is significant - most ./configure scripts attempts to check whether "-stdlib=libc++" is a valid option using a dummy program. It's suppose to fail and the build system knows it's GCC. But now it escapes detection. This causes a build failure of sys-devel/binutils-apple-8.2.1-r101, and possibly many other packages that rely on this check. This is strange. I suspect it's a regression GCC 12.2 (or GCC on x64) as an attempt to improve compatibility, but the unintended consequence is instead breaking compatibility. I'm now testing whether GCC 12.1 is also affected. Originally reported in Bug 905131.