bootstrap-prefix.sh fails on sys-devel/gcc-12.1.0 in stage3 (amd64/x64) with linker error: ld: -pie and -no_pie can't be used together collect2: error: ld returned 1 exit status make[3]: *** [Makefile:3121: gcov] Error 1 Reproducible: Always Steps to Reproduce: 1.LATEST_TREE_YES=1 ACCEPT_KEYWORDS="~x64-macos" ~/bootstrap-prefix.sh
Created attachment 855412 [details] gcc-12.1.0 build-logs
I tried to work around the issue and to get rid of the "-pie" linker flag. Found in file work/gcc-12-branch-gcc-12.1-darwin-r0/gcc/Makefile.in: ``` ifneq (@enable_pie_tools@,yes) # Build and link the compilers and tools without PIE. COMPILER += $(NO_PIE_CFLAGS) LINKER += $(NO_PIE_FLAG) else # FIXME these need to be configured. COMPILER += -fPIE LINKER += -pie endif ``` So i changed ebuild file and added "export enable_pie_tools=no" in src_configure... I really don't know if this hack has any __unwanted side effekts__, but it merged gcc successfully and continues to bootstrap now...
app-arch/tar-1.34-r2 failed while installing to ${EPREFIX}/tmp/... Messages for package app-arch/tar-1.34-r2: * FAILED postinst: 1 * ERROR: app-arch/tar-1.34-r2::gentoo_prefix failed (postinst phase): * (no error message) * * Call stack: * ebuild.sh, line 127: Called pkg_postinst * environment, line 453: Called die * The specific snippet of code: * ln -s gtar "${EROOT}/bin/tar" || die; But the mentioned log ${EPREFIX}/tmp/var/tmp/portage/app-arch/tar-1.34-r2/temp/build.log wasn't there when trying to access it. So i just tried LATEST_TREE_YES=1 ACCEPT_KEYWORDS="~x64-macos" ~/bootstrap-prefix.sh again, which is running now and already installed tar (and other stuff) to ${EPREFIX}/...
Failed to emerge sys-devel/binutils-apple-8.2.1-r101, happened in late stage3, after ">>> Syncing repository 'gentoo_prefix'", while running emerge -uDNv system. The log looks strange - many escape sequences embedded there - i think the error message is: /Users/jin/Gentoo/MacOSX.sdk/usr/include/objc/runtime.h:373:29:error: expected '[' before '^' token. bootstrap was unsuccessful. Waiting now for help :-)
Created attachment 855426 [details] sys-devel/binutils-apple build.log
(In reply to jin from comment #3) > app-arch/tar-1.34-r2 failed while installing to ${EPREFIX}/tmp/... > > Messages for package app-arch/tar-1.34-r2: > * FAILED postinst: 1 > * ERROR: app-arch/tar-1.34-r2::gentoo_prefix failed (postinst phase): > * (no error message) > * > * Call stack: > * ebuild.sh, line 127: Called pkg_postinst > * environment, line 453: Called die > * The specific snippet of code: > * ln -s gtar "${EROOT}/bin/tar" || die; > > But the mentioned log > ${EPREFIX}/tmp/var/tmp/portage/app-arch/tar-1.34-r2/temp/build.log wasn't > there when trying to access it. So i just tried > LATEST_TREE_YES=1 ACCEPT_KEYWORDS="~x64-macos" ~/bootstrap-prefix.sh > again, which is running now and already installed tar (and other stuff) to > ${EPREFIX}/... I ran into this myself, but it baffles me, it is a postinst check, therefore the log is removed, which checks first if the file already is a symlink. And if I check it myself, the file indeed is. So it should have never run the ln.
(In reply to Fabian Groffen from comment #6) > (In reply to jin from comment #3) > > app-arch/tar-1.34-r2 failed while installing to ${EPREFIX}/tmp/... > > > > Messages for package app-arch/tar-1.34-r2: > > * FAILED postinst: 1 > > * ERROR: app-arch/tar-1.34-r2::gentoo_prefix failed (postinst phase): > > * (no error message) > > * > > * Call stack: > > * ebuild.sh, line 127: Called pkg_postinst > > * environment, line 453: Called die > > * The specific snippet of code: > > * ln -s gtar "${EROOT}/bin/tar" || die; > > > > But the mentioned log > > ${EPREFIX}/tmp/var/tmp/portage/app-arch/tar-1.34-r2/temp/build.log wasn't > > there when trying to access it. So i just tried > > LATEST_TREE_YES=1 ACCEPT_KEYWORDS="~x64-macos" ~/bootstrap-prefix.sh > > again, which is running now and already installed tar (and other stuff) to > > ${EPREFIX}/... > > I ran into this myself, but it baffles me, it is a postinst check, therefore > the log is removed, which checks first if the file already is a symlink. > And if I check it myself, the file indeed is. So it should have never run > the ln. This is Bug 904887 and now it has been fixed.
(In reply to jin from comment #4) > Failed to emerge sys-devel/binutils-apple-8.2.1-r101, happened in late > stage3, after ">>> Syncing repository 'gentoo_prefix'", while running emerge > -uDNv system. > > The log looks strange - many escape sequences embedded there - i think the > error message is: > > /Users/jin/Gentoo/MacOSX.sdk/usr/include/objc/runtime.h:373:29:error: > expected '[' before '^' token. > > bootstrap was unsuccessful. Waiting now for help :-) This is caused by missing support of the Blocks language extension in GCC. The header file objc/runtime.h contains a Blocks declaration, so everything that uses this header fails. Fortunately it seems that the header is actually not strictly necessary right now, and can be removed from inclusion. But the lack of Blocks support is a ticking time bomb, sooner or later GCC would completely stop functioning as a system toolchain. This is bad new... Right now, try apply my patch to sys-devel/binutils-apple-8.2.1-r101's source code: https://github.com/biergaizi/darwin-xtools/commit/58967cfac9ab9c39d5c6988624bdd6a5c986537e For completeness, these two patches are better to be applied as well. https://github.com/grobian/darwin-xtools/commit/e67225380b32c77cd6ae2e89a746739eb62d921d.patch https://github.com/grobian/darwin-xtools/commit/3127e3387d688d5b98149b038e235762846fe49e.patch
For cross-reference, please add Bug 886491 to the "Blocks" list of the current bug. I don't have the permissions to make this edit.
(In reply to Tom Li from comment #8) > This is caused by missing support of the Blocks language extension in GCC. > The header file objc/runtime.h contains a Blocks declaration, so everything > that uses this header fails. Fortunately it seems that the header is > actually not strictly necessary right now, and can be removed from > inclusion. But the lack of Blocks support is a ticking time bomb, sooner or > later GCC would completely stop functioning as a system toolchain. This is > bad new... > Yeah, you already need Clang to get nice macOS integrations (anything using Frameworks within the Prefix) and that's just going to keep getting worse, so ultimately, Clang bootstrap is the goal and far easier than trying to implement Blocks in GCC.
Bootstrapping with Clang sounds awesome. Is there a ticket for that, or how far along is the work?
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=c9f40c64294b16646560d39ae010fc18c6f28985 commit c9f40c64294b16646560d39ae010fc18c6f28985 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-04-25 15:06:30 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-04-25 15:06:30 +0000 scripts/bootstrap-prefix: bump bootstrap snapshot for Darwin (arm64) fixes Closes: https://bugs.gentoo.org/898610 Bug: https://bugs.gentoo.org/886491 Signed-off-by: Fabian Groffen <grobian@gentoo.org> scripts/bootstrap-prefix.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Bootstrapping with Clang is near impossible because LLVM is a multi-headed beast, that has grown from a cute dragon it was at first, into a bizzarre monster of unmet proportions. Hence, GCC. Let's Ian gets to implementing it.
This bug is NOT FIXED, do NOT close this bug.
You don't need to use capitals.
(In reply to Fabian Groffen from comment #13) > Bootstrapping with Clang is near impossible because LLVM is a multi-headed > beast, that has grown from a cute dragon it was at first, into a bizzarre > monster of unmet proportions. Hence, GCC. Let's Ian gets to implementing > it. on ARM64, bootstrap uses native-cctools so it doesn't have this problem, on x86_64, it tries to build binutils-apple so it fails. I believe I can fix binutils-apple by changing two lines of code. Don't close this bug yet.
(In reply to jin from comment #0) > bootstrap-prefix.sh fails on sys-devel/gcc-12.1.0 in stage3 (amd64/x64) with > linker error: > ld: -pie and -no_pie can't be used together > collect2: error: ld returned 1 exit status > make[3]: *** [Makefile:3121: gcov] Error 1 I suspect this problem was caused by an outdated patch "09_all_nopie-all-flags.patch" in the Gentoo gcc-12.2.0-patches-1 patchset. This patch basically forcefully adds the $(NO_PIE_CFLAGS), regardless of PIE is enabled or disabled at build time. > We need to pass NO_PIE_CFLAGS to ALL_* so gcc don't fail when > we compile it with older gcc and pie. > > --- a/gcc/Makefile.in 2015-06-25 19:18:12.000000000 +0200 > +++ b/gcc/Makefile.in 2016-04-22 00:12:54.029178860 +0200 > @@ -1054,10 +1054,10 @@ ALL_CXXFLAGS = $(T_CFLAGS) $(CFLAGS-$@) > ALL_CPPFLAGS = $(INCLUDES) $(CPPFLAGS) > > # This is the variable to use when using $(COMPILER). > -ALL_COMPILERFLAGS = $(ALL_CXXFLAGS) > +ALL_COMPILERFLAGS = $(NO_PIE_CFLAGS) $(ALL_CXXFLAGS) > > # This is the variable to use when using $(LINKER). > -ALL_LINKERFLAGS = $(ALL_CXXFLAGS) > +ALL_LINKERFLAGS = $(NO_PIE_CFLAGS) $(ALL_CXXFLAGS) I cannot find the original context or the issue that created this patch, it seemed to be an ancient patch, carried over from the era of GCC 6.2 from 7 years ago by originally Magnus Granberg. But my best reasoning was, long time ago, GCC didn't support building itself with PIE, so this flag needs to be passed to forcefully disable it. However, modern GCC supports self-building with PIE. In mainline GCC, this feature is called "--enable-host-pie" [2], and in Darwin GCC it's called "--enable-pie-tools". Furthermore, in today's security-consensus world (as pioneered by Gentoo Hardened 10 years ago), building with PIE is now the standard, and Apple Silicon its use is mandatory and cannot be disabled [3]. Thus, PIE is now a default-on option, but the outdated patch "09_all_nopie-all-flags.patch" is still passing $NO_PIE_CFLAGS unconditionally into GCC's Makefile, creating the error: > ld: -pie and -no_pie can't be used together Apple Silicon is not affected by this problem, possibly because "-no_pie" is silently ignored. In my opinion, the most reasonable fix here is leaving PIE enabled on macOS, and dropping "09_all_nopie-all-flags.patch" from the official Gentoo patchset so it doesn't create contradictory flags. However, the original context of why the patch is needed isn't clear, without a clear description, I'm afraid that it would produce regression on other platforms. As a compromise, let's keep PIE on by default, but add a new patch into Gentoo's prefix tree to revert this original Gentoo Gentoo, but only on Darwin. Until the effect of the original PIE patch is investigated, then the original patch should be either dropped or updated to take effect only if PIE is disabled. [1] https://gitweb.gentoo.org/proj/gcc-patches.git/tree/6.2.0/gentoo/54_all_nopie-all-flags.patch [2] https://www.mail-archive.com/gcc-patches@gcc.gnu.org/msg279973.html [3] https://github.com/iains/gcc-darwin-arm64/blob/d206171d082dfadf9bd0b71b5f29c6b30e18819a/gcc/configure.ac#L7516
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=32708ce1cab1fc98af5eccc8d2f39df761d528bf commit 32708ce1cab1fc98af5eccc8d2f39df761d528bf Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-04-26 17:56:30 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-04-26 17:56:30 +0000 sys-devel/gcc-12.2.0-r1: revbump to drop nopie patch for Apple SI Bug: https://bugs.gentoo.org/898610 Signed-off-by: Fabian Groffen <grobian@gentoo.org> sys-devel/gcc/gcc-12.2.0-r1.ebuild | 131 +++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+)
(In reply to Larry the Git Cow from comment #18) > The bug has been referenced in the following commit(s): > > https://gitweb.gentoo.org/repo/proj/prefix.git/commit/ > ?id=32708ce1cab1fc98af5eccc8d2f39df761d528bf > > commit 32708ce1cab1fc98af5eccc8d2f39df761d528bf > Author: Fabian Groffen <grobian@gentoo.org> > AuthorDate: 2023-04-26 17:56:30 +0000 > Commit: Fabian Groffen <grobian@gentoo.org> > CommitDate: 2023-04-26 17:56:30 +0000 > > sys-devel/gcc-12.2.0-r1: revbump to drop nopie patch for Apple SI > > Bug: https://bugs.gentoo.org/898610 > Signed-off-by: Fabian Groffen <grobian@gentoo.org> > > sys-devel/gcc/gcc-12.2.0-r1.ebuild | 131 > +++++++++++++++++++++++++++++++++++++ > 1 file changed, 131 insertions(+) This commit is wrong, please revert it, it should be applied to Intel, _not_ ARM. Or preferably, applied on both systems simultaneously. To repeat the problem here: On Apple Silicon, PIE is mandatory so the option "-no_pie" ironically appears harmless for some reason (possibly because it's being ignored anyway). It only fails on Intel because it produces the contradictory build option, "-pie -no_pie". In my opinion, the build workflow should be kept the same on both systems if possible, so my suggestion is removing the patch on both systems and allowing them defaulting to PIE. (even if a compatibility problem arise, PIE should be disabled properly via compiler flags, not this outdated patch, or at least the patch should be updated). Eventually, this outdated patch needs to be either updated dropped from Gentoo altogether.
For the proposed patch, I opened a Pull Request at https://github.com/gentoo/prefix/pull/27
There's a serious miscommunication problem in this bug report, first it was closed even when the last comment was still discussing a proposed solution, next an incorrect patch was committed into the tree when the last comment suggested the opposite solution (and in still in the middle of troubleshooting the problem). At this rate, I think it's better for me to refrain from sharing technical details about the problem until a full patch is ready to eliminate the chance of miscommunication.
Perhaps then we shouldn't be re-using a bug for many things? Also, I tried to interpret your messages. I have successfully bootstrapped and compiled gcc-12.2 on x64-macos today. So I assumed this wouldn't be an issue there. Why is it an issue on x64? Is it related to macOS version? Then GCC should take that into account or something? I'm fine to make it *-darwin, afterall that's why I originally shared the keyword with both archs until QA sliced that.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=8257ec94a7fe25205d4770b63c3cecb47ce8eced commit 8257ec94a7fe25205d4770b63c3cecb47ce8eced Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-04-26 19:20:56 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-04-26 19:20:56 +0000 sys-devel/gcc-12.2.0-r1: drop no_pie patch for all of Darwin Bug: https://bugs.gentoo.org/898610 Signed-off-by: Fabian Groffen <grobian@gentoo.org> sys-devel/gcc/gcc-12.2.0-r1.ebuild | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(In reply to Fabian Groffen from comment #22) > Perhaps then we shouldn't be re-using a bug for many things? I agree, I'll create a new bug for the binutils-apple issue. > I have successfully bootstrapped and compiled gcc-12.2 on x64-macos today. I have successfully replicated the bootstrap failure on x64-macos today. > Why is it an issue on x64? Is it related to macOS version? The title of the bug report mentions "macOS 13.2.1 Ventura" > Then GCC should take that into account or something? GCC Darwin always took that into account, but an outdated out-of-tree patch was adding the No PIE flag mindlessly - this is clearly outside of GCC's control.
I just created Bug #905131, please redirect further discussions about the binutils-apple build failure to the new report. Meanwhile, I'm currently testing the committed fix for GCC 12.2. I also plan to test the same fix on GCC 12.1 (GCC 12.1 was the first "fixed" GCC version, so a fallback option is always nice to have). If all things go as planned, this bug can be closed automatically when my GCC 12.1 patch is submitted and merged into the tree.
I have confirmed that GCC 12.1 has been fixed by Fabian Groffen in commit dce4c2d2b08d1d4e1dc6198ce123cd913a53395f [1]. This bug can be closed. It's another step forward on fixing macOS 13 bootstrapping. [1] https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=dce4c2d2b08d1d4e1dc6198ce123cd913a53395f
(In reply to Tom Li from comment #26) > I have confirmed that GCC 12.1 has been fixed by Fabian Groffen in commit > dce4c2d2b08d1d4e1dc6198ce123cd913a53395f [1]. This bug can be closed. It's > another step forward on fixing macOS 13 bootstrapping. > > [1] > https://gitweb.gentoo.org/repo/proj/prefix.git/commit/ > ?id=dce4c2d2b08d1d4e1dc6198ce123cd913a53395f Thank you!
