| Summary: | bootstrap-prefix.sh fails on macOS Big Sur Intel MacBook Pro during installation of app-shells/bash-5.1_rc3 | ||
|---|---|---|---|
| Product: | Gentoo/Alt | Reporter: | Robin Randhawa <robin.randhawa> |
| Component: | Mac OSX | Assignee: | Gentoo Prefix <prefix> |
| Status: | RESOLVED FIXED | ||
| Severity: | blocker | CC: | cognifloyd+gentoobugs, sam |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | AMD64 | ||
| OS: | OS X | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
Partial build log
Build log for bootstrap-bash.sh failure Build log from stage 2 gentoo functions failure bootstrap-prefix.sh modifications for stage1/stage2 using clang bootstrap-prefix.sh modifications for stage1/stage2 using clang bootstrap-prefix.sh modifications for stage1/stage2 using clang |
||
|
Description
Robin Randhawa
2020-11-28 15:33:00 UTC
Created attachment 675601 [details]
Partial build log
The build log only contains the text from the build of readline and on to the actual failure with bash.
>/Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 89: /Users/robin/Gentoo/tmp/var/tmp/portage/app-shells/bash-5.1_rc3/image/Users/robin/Gentoo/tmp/usr/share/doc/bash-5.1_rc3/POSIX.ecompress: Interrupted system call
> * ERROR: app-shells/bash-5.1_rc3::gentoo_prefix failed:
This looks weird. Were you running low on memory or something at the time?
(By the way, please bear with us, we've just managed to get things kind of working again, thanks for trying it out!)
Also, if you run the script again, does the error happen? (it should resume) is this a Clang or GCC bootstrap? (In reply to Sam James from comment #2) > >/Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 89: /Users/robin/Gentoo/tmp/var/tmp/portage/app-shells/bash-5.1_rc3/image/Users/robin/Gentoo/tmp/usr/share/doc/bash-5.1_rc3/POSIX.ecompress: Interrupted system call > > * ERROR: app-shells/bash-5.1_rc3::gentoo_prefix failed: > > This looks weird. Were you running low on memory or something at the time? TBH I'm not sure! This is reproducible though so I'll give it another shot and this time watch out for any indicators of low memory. > (By the way, please bear with us, we've just managed to get things kind of > working again, thanks for trying it out!) You're welcome and godspeed to you folks! :) (In reply to Fabian Groffen from comment #4) > is this a Clang or GCC bootstrap? Erm, I didn't make any specific changes to specify either. Looking at the build log I can only see clang invocations so perhaps it is a clang bootstrap. Let me know if you would like me to try a GCC bootstrap and if so, how to arrange that. Thanks. When you re-run it, it should immediately compile Bash, so does it fail straight away there? That's quite interesting if it does... (In reply to Sam James from comment #7) > When you re-run it, it should immediately compile Bash, so does it fail > straight away there? That's quite interesting if it does... Wow! Did a re-run and tt managed to build and install Bash. But it's now gone and failed kind of similarly for sed! Log snippet: ############################################## >>> Completed installing sys-apps/sed-4.5 into /Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/sed-4.5/image/ * Final size of build directory: 14536 KiB (14.1 MiB) * Final size of installed tree: 612 KiB /Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 65: declare: -A: invalid option declare: usage: declare [-afFirtx] [-p] [name[=value] ...] /Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 89: /Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/sed-4.5/image/Users/robin/Gentoo/tmp/usr/share/doc/sed-4.5/CHANGELOG.ecompress: Interrupted system call * ERROR: sys-apps/sed-4.5::gentoo_prefix failed: * (no error message) ############################################## This time I kept an eye on the memory usage trend with htop and there was about 50% usage. Is that "declare: -A: invalid option" message any indication of what's up ? I just realised that this might be relevant: I'm using macports. As in, the bootstrap-prefix.sh script was run from within a shell environment from macports (and not the default macOS environment). I assume that bootstrap-prefix.sh effectively cleanses the environment early on and therefore there shouldn't be any 'pollution' ? bootstrap-prefix won't cleanse. However, it's meant to be run from a clean env, if you can't guarantee that, you might want to try http://rsync.prefix.bitzolder.nl/scripts/auto-bootstraps/dobootstrap but it is meant to be run without intervention, e.g. from a build-system. In any case, please run from without macports, and please export DARWIN_USE_GCC=1 in your environment whenever you call bootstrap-prefix. This is the only tested configuration right now, I'll probably soon make that default, but we're still trying to finalise things. At this point your bootstrap will get stuck, but you can move forward with it if you set some more "void warranty" variables ;) I hope to be able to resolve that soon. Thanks Fabian. So I effectively disabled macports by removing all relevant references from PATH. Also - given that Big Sur doesn't have bash (to my knowledge) I chose to: wget https://gitweb.gentoo.org/repo/proj/prefix.git/plain/scripts/bootstrap-bash.sh chmod +x bootstrap-bash.sh ./bootstrap-bash.sh /var/tmp/bash Unfortunately this fails with: *********************************************************** * * * GNU bash, version 4.2.0(1)-release (i386-apple-darwin20.1.0) * * *********************************************************** rm -f shell.o gcc -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"i386"' -DCONF_OSTYPE='"darwin20.1.0"' -DCONF_MACHTYPE='"i386-apple-darwin20.1.0"' -DCONF_VENDOR='"apple"' -DLOCALEDIR='"/var/tmp/bash/usr/share/locale"' -DPACKAGE='"bash"' -DSHELL -DHAVE_CONFIG_H -DMACOSX -I. -I. -I./include -I./lib -g -O2 -c shell.c shell.c:739:7: error: implicit declaration of function 'get_tty_state' is invalid in C99 [-Werror,-Wimplicit-function-declaration] get_tty_state (); .. and some other similar errors (I'll attach the complete log). Any pointers appreciated. Thanks. Created attachment 675631 [details]
Build log for bootstrap-bash.sh failure
I've almost got the clang bootstrap working :D Don't kill it on me now. With clang, I have hit an odd build failure that is probably a concurrency bug in the build somewhere saying it couldn't find something that was clearly part of bash sources. To try and debug, I added VERBOSE=1 to MAKEOPTS in make.conf, and then the build succeeded. It also didn't increase the verbosity at all. Now, every time I wipe out my prefix to retry my bootstrap-prefix.sh changes, I consistently hit the same bug in stage2 where portage is building bash. I re-add the MAKEOPTS change in make.conf, and then remove it again once bash is built. Oddly enough, if I start stage2 the build with MAKEOPTS already modified, it still fails the first time and I have to remove it, have another go, and then re-add the change before it succeeds. I haven't dug into it farther, as I was focused on getting to the subsequent issues with clang so I could work on that. I don't know if the issue is related or not. Also, I have macports on my machine, but I'm not using a macports shell. I also open a new bash shell to run bootstrap-prefix.sh in before running it (so it runs in bash instead of zsh). I think using zsh is fine, I just used it on Big Sur, no need to bootstrap a bash I think? Regarding Clang path, do you use an updated clang toolchain (e.g. 11?) or the old copies that once worked in Prefix (e.g. 7)? Another update: - I 'cleansed' my Big Sur zsh shell environment to remove all traces of macports. - Got and ran bootstrap-prefix.sh - Did not specify a GCC build I got this far: Fetching prefix-overlay-20201128.tar.bz2 wget --no-check-certificate http://distfiles.gentoo.org/distfiles/prefix-overlay-20201128.tar.bz2 --2020-11-29 14:56:47-- http://distfiles.gentoo.org/distfiles/prefix-overlay-20201128.tar.bz2 Resolving distfiles.gentoo.org... 2600:3404:200:237::2, 2605:bc80:3010::134, 2600:3402:200:227::2, ... Connecting to distfiles.gentoo.org|2600:3404:200:237::2|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2020-11-29 14:56:48 ERROR 404: Not Found. wget --no-check-certificate http://distfiles.prefix.bitzolder.nl/distfiles/prefix-overlay-20201128.tar.bz2 --2020-11-29 14:56:48-- http://distfiles.prefix.bitzolder.nl/distfiles/prefix-overlay-20201128.tar.bz2 Resolving distfiles.prefix.bitzolder.nl... 2a10:3781:81c::69, 45.95.64.8 Connecting to distfiles.prefix.bitzolder.nl|2a10:3781:81c::69|:80... connected. HTTP request sent, awaiting response... 302 Look Elsewhere Location: http://distfiles.gentoo.org/distfiles/e8/prefix-overlay-20201128.tar.bz2 [following] --2020-11-29 14:56:48-- http://distfiles.gentoo.org/distfiles/e8/prefix-overlay-20201128.tar.bz2 Resolving distfiles.gentoo.org... 2605:bc80:3010::134, 2600:3402:200:227::2, 140.211.166.134, ... Connecting to distfiles.gentoo.org|2605:bc80:3010::134|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2020-11-29 14:56:48 ERROR 404: Not Found. wget --no-check-certificate http://distfiles.prefix.bitzolder.nl/prefix/distfiles/prefix-overlay-20201128.tar.bz2 --2020-11-29 14:56:48-- http://distfiles.prefix.bitzolder.nl/prefix/distfiles/prefix-overlay-20201128.tar.bz2 Resolving distfiles.prefix.bitzolder.nl... 2a10:3781:81c::69, 45.95.64.8 Connecting to distfiles.prefix.bitzolder.nl|2a10:3781:81c::69|:80... connected. HTTP request sent, awaiting response... 302 Look Elsewhere Location: https://distfiles.prefix.bitzolder.nl/prefix/distfiles/e8/prefix-overlay-20201128.tar.bz2 [following] --2020-11-29 14:56:48-- https://distfiles.prefix.bitzolder.nl/prefix/distfiles/e8/prefix-overlay-20201128.tar.bz2 Connecting to distfiles.prefix.bitzolder.nl|2a10:3781:81c::69|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-11-29 14:56:49 ERROR 404: Not Found. wget --no-check-certificate http://dev.gentoo.org/~grobian/distfiles/prefix-overlay-20201128.tar.bz2 --2020-11-29 14:56:49-- http://dev.gentoo.org/~grobian/distfiles/prefix-overlay-20201128.tar.bz2 Resolving dev.gentoo.org... 2001:470:ea4a:1:5054:ff:fec7:86e4, 140.211.166.183 Connecting to dev.gentoo.org|2001:470:ea4a:1:5054:ff:fec7:86e4|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://dev.gentoo.org/~grobian/distfiles/prefix-overlay-20201128.tar.bz2 [following] --2020-11-29 14:56:49-- https://dev.gentoo.org/~grobian/distfiles/prefix-overlay-20201128.tar.bz2 Connecting to dev.gentoo.org|2001:470:ea4a:1:5054:ff:fec7:86e4|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-11-29 14:56:50 ERROR 404: Not Found. !!! downloading http://dev.gentoo.org/~grobian/distfiles/prefix-overlay-20201128.tar.bz2 failed! I tried running bootstrap_stage1_log but that failed :( I have no clue, really. Please find friendly folks in #gentoo-prefix on irc.gentoo.org, gentoo-alt@lists.gentoo.org mailing list, or file a bug at bugs.gentoo.org under Gentoo/Alt, Prefix Support. Sorry that I have failed you master. I shall now return to my humble cave. You can find a log of what happened in /Users/robin/Gentoo/stage1.log Apparently there's a missing 'prefix-overlay-20201128.tar.bz2' tarball ? my mistake, uploaded the file with the wrong name, you can restart it now, and it should be able to download the file sorry No worries and thanks for your prompt responses! So a re-run from scratch got me further but there are still problems. This is some of the build log. I'll attach more separately. * stage1 successfully finished * Bootstrapping Gentoo prefixed portage installation using * host: x86_64-apple-darwin20 * prefix: /Users/robin/Gentoo * ready to bootstrap stage2_log . . . >>> Completed installing sys-apps/gentoo-functions-0.14 into /Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/image * Final size of build directory: 100 KiB * Final size of installed tree: 68 KiB /Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 65: declare: -A: invalid option declare: usage: declare [-afFirtx] [-p] [name[=value] ...] /Users/robin/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 89: /Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/image/Users/robin/Gentoo/tmp/usr/share/man/man1/consoletype.1.ecompress: Interrupted system call * ERROR: sys-apps/gentoo-functions-0.14::gentoo_prefix failed: * (no error message) * * If you need support, post the output of `emerge --info '=sys-apps/gentoo-functions-0.14::gentoo_prefix'`, * the complete build log and the output of `emerge -pqv '=sys-apps/gentoo-functions-0.14::gentoo_prefix'`. * The complete build log is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/temp/build.log'. * The ebuild environment file is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/temp/environment'. * Working directory: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/image' * S: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/work/gentoo-functions-0.14' !!! post install failed; exiting. >>> Failed to emerge sys-apps/gentoo-functions-0.14, Log file: >>> '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/temp/build.log' * Messages for package sys-apps/gentoo-functions-0.14: * ERROR: sys-apps/gentoo-functions-0.14::gentoo_prefix failed: * (no error message) * * If you need support, post the output of `emerge --info '=sys-apps/gentoo-functions-0.14::gentoo_prefix'`, * the complete build log and the output of `emerge -pqv '=sys-apps/gentoo-functions-0.14::gentoo_prefix'`. * The complete build log is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/temp/build.log'. * The ebuild environment file is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/temp/environment'. * Working directory: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/image' * S: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.14/work/gentoo-functions-0.14' Odd! Running /opt/local/bin/bash ./bootstrap-prefix.sh "/Users/robin/Gentoo" stage2 failed Created attachment 675703 [details]
Build log from stage 2 gentoo functions failure
There's that now familiar 'declare -A' error to do withe compress again!
/opt/local/bin/bash ... you can ignore all of those warnings and errors, because it's simply the env not being up2date to Gentoo's standards or tools simply missing Yikes! You're right. I didn't really get macports out of the way there! My turn to apologise! Now I made 100% sure Macports isn't around. Some observations: 1. I've had multiple failures all of the same sort involving a 'Interrupted system call' message in the vicinity of the 'declare: -A' message. 2. That always happens when the machine is stressed out. I mean - the fans are screaming. Memory load looks ok in htop. 3. At each failure, if I let the fans settle and then start the bootstrap-prefix.sh script again, the failure doesn't happen and things move ahead. Until the next similar failure. It sounds like there is some user-space code path where a check for an interrupted system call followed by a retry is missing ? Anyway - by repeating that process I now have things reaching the binutils build where there is now what appears to be a 'proper' compile failure to do with missing 'tapi.h'. Some key lines from the build log: Source prepared. >>> Configuring source in /Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work ... >>> Source configured. >>> Compiling source in /Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work ... * building ld64-530 make SHELL=/Users/robin/Gentoo/tmp/bin/bash -j5 LTO=0 LTO_INCDIR= LTO_LIBDIR= TAPI=1 TAPI_LIBDIR=/Users/robin/Gentoo/tmp/usr/lib clang++ -m64 -march=native -O2 -pipe -Wno-invalid-offsetof -std=c++0x -stdlib=libc++ -I/Users/robin/Gentoo/tmp/usr/include -DNDEBUG -Iinclude -Iabstraction -Ild - Ild/parsers -Iother -I. -DTAPI -c -o other/unwinddump.o other/unwinddump.cpp [snip] clang++ -m64 -march=native -O2 -pipe -Wno-invalid-offsetof -std=c++0x -stdlib=libc++ -I/Users/robin/Gentoo/tmp/usr/include -DNDEBUG -Iinclude -Iabstraction -Ild -Ild/parsers -Iother -I. -DTAPI -c -o ld/parsers/archive_file.o ld/parsers/archive_file.cpp clang++ -m64 -march=native -O2 -pipe -Wno-invalid-offsetof -std=c++0x -stdlib=libc++ -I/Users/robin/Gentoo/tmp/usr/include -DNDEBUG -Iinclude -Iabstraction -Ild -Ild/parsers -Iother -I. -DTAPI -c -o ld/parsers/opaque_section_file.o ld/parsers/opaque_section_file.cpp In file included from ld/parsers/archive_file.cpp:41: In file included from ld/parsers/macho_relocatable_file.h:29: ld/Options.h:31:10: fatal error: 'tapi/tapi.h' file not found #include <tapi/tapi.h> ^~~~~~~~~~~~~ 1 error generated. make: *** [<builtin>: ld/parsers/archive_file.o] Error 1 make: *** Waiting for unfinished jobs.... * ERROR: sys-devel/binutils-apple-11.3.1::gentoo_prefix failed (compile phase): * emake failed * * If you need support, post the output of `emerge --info '=sys-devel/binutils-apple-11.3.1::gentoo_prefix'`, * the complete build log and the output of `emerge -pqv '=sys-devel/binutils-apple-11.3.1::gentoo_prefix'`. * The complete build log is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/temp/build.log'. * The ebuild environment file is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/temp/environment'. * Working directory: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work/ld64-530/src' * S: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work' >>> Failed to emerge sys-devel/binutils-apple-11.3.1, Log file: >>> '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/temp/build.log' * Messages for package sys-devel/binutils-apple-11.3.1: * ERROR: sys-devel/binutils-apple-11.3.1::gentoo_prefix failed (compile phase): * emake failed * * If you need support, post the output of `emerge --info '=sys-devel/binutils-apple-11.3.1::gentoo_prefix'`, * the complete build log and the output of `emerge -pqv '=sys-devel/binutils-apple-11.3.1::gentoo_prefix'`. * The complete build log is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/temp/build.log'. * The ebuild environment file is located at '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/temp/environment'. * Working directory: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work/ld64-530/src' * S: '/Users/robin/Gentoo/tmp/var/tmp/portage/sys-devel/binutils-apple-11.3.1/work' Odd! Running /bin/bash ./bootstrap-prefix.sh "/Users/robin/Gentoo" stage2 I think this is because you're using the Clang path. For recent macOS versions, we AFAIK need to (for now) use the GCC path: DARWIN_USE_GCC=1. This will avoid pulling in binutils-apple. (Regarding that Bash error you saw - you can workaround this with some CFLAGS, but it actually stopped happening for me anyway.) (In reply to Robin Randhawa from comment #22) > Yikes! You're right. I didn't really get macports out of the way there! My > turn to apologise! > > Now I made 100% sure Macports isn't around. > > Some observations: > > 1. I've had multiple failures all of the same sort involving a 'Interrupted > system call' message in the vicinity of the 'declare: -A' message. > > 2. That always happens when the machine is stressed out. I mean - the fans > are screaming. Memory load looks ok in htop. > > 3. At each failure, if I let the fans settle and then start the > bootstrap-prefix.sh script again, the failure doesn't happen and things move > ahead. Until the next similar failure. If you think your machine is close to overheating, then you can try using less parallel processes. I don't know how many cores your system has, but let's say that you have 4 and we'd like to use 2, then export in your environment: $ export USE_CPU_CORES=2 and re-run bootstrap-prefix.sh again. It will respect the value instead of using ceil(cpus / 2), which may be just too much in your case. (In reply to Robin Randhawa from comment #22) > ld/Options.h:31:10: fatal error: 'tapi/tapi.h' file not found > #include <tapi/tapi.h> > ^~~~~~~~~~~~~ I'm following the clang route. It doesn't quite work yet, but I have a fix for this issue. It requires an extra bootstrap step during stage1: bootstrap_libtapi. Here is the bootstrap_libtapi function (it downloads the tapi headers from the macports build, but then binutils-apple actually links with the CommandLineTools libtapi.dylib) https://gist.github.com/cognifloyd/f4ff46e3dcfd48ca9827c32e04a8fe90/46a85e952c02c943f4369b857cfea57c501d8e86#file-bootstrap-prefix-sh-L1383-L1406 Here is where bootstrap_libtapi is called during stage1: https://gist.github.com/cognifloyd/f4ff46e3dcfd48ca9827c32e04a8fe90/46a85e952c02c943f4369b857cfea57c501d8e86#file-bootstrap-prefix-sh-L1530-L1534 (In reply to Jacob Floyd from comment #25) > (In reply to Robin Randhawa from comment #22) > > ld/Options.h:31:10: fatal error: 'tapi/tapi.h' file not found > > #include <tapi/tapi.h> > > ^~~~~~~~~~~~~ > > I'm following the clang route. It doesn't quite work yet, but I have a fix > for this issue. It requires an extra bootstrap step during stage1: > bootstrap_libtapi. > > Here is the bootstrap_libtapi function (it downloads the tapi headers from > the macports build, but then binutils-apple actually links with the > CommandLineTools libtapi.dylib) > > https://gist.github.com/cognifloyd/f4ff46e3dcfd48ca9827c32e04a8fe90/ > 46a85e952c02c943f4369b857cfea57c501d8e86#file-bootstrap-prefix-sh-L1383-L1406 > > Here is where bootstrap_libtapi is called during stage1: > > https://gist.github.com/cognifloyd/f4ff46e3dcfd48ca9827c32e04a8fe90/ > 46a85e952c02c943f4369b857cfea57c501d8e86#file-bootstrap-prefix-sh-L1530-L1534 is it possible to just emerge tapi prior to binutils-apple instead of bootstrapping it? we're in portage territory already, aren't we? something like this (untested)
--- a/scripts/bootstrap-prefix.sh
+++ b/scripts/bootstrap-prefix.sh
@@ -241,7 +241,7 @@ configure_toolchain() {
sys-libs/libcxx"
CC=clang
CXX=clang++
- linker=sys-devel/binutils-apple
+ linker="sys-libs/tapi sys-devel/binutils-apple"
;;
*"Apple LLVM version "*)
vers=${ccvers#*Apple LLVM version }
Created attachment 675736 [details, diff]
bootstrap-prefix.sh modifications for stage1/stage2 using clang
for the clang route, you will also need a few other changes to bootstrap-prefix.sh to get much farther. This patch includes:
- stage1: bootstrap_libtapi
- stage1/stage2: set up ${EPREFIX}/MacOSX.sdk and related include paths to use it for the clang build, not just for GCC.
- stage2: add additional build deps (dev-util/pkgconf and app-misc/pax-utils)
- stage2: drop ssl useflag for app-crypt/rhash (only applies to stage2 build)
- a one char typo fix in a comment
(In reply to Jacob Floyd from comment #28) > Created attachment 675736 [details, diff] [details, diff] > bootstrap-prefix.sh modifications for stage1/stage2 using clang > > for the clang route, you will also need a few other changes to > bootstrap-prefix.sh to get much farther. This patch includes: > > - stage1: bootstrap_libtapi stage1 is for tools only that are necessary to run Portage, tapi is not one of those, so we should be able to just emerge it instead > - stage1/stage2: set up ${EPREFIX}/MacOSX.sdk and related include paths to > use it for the clang build, not just for GCC. ack > - stage2: add additional build deps (dev-util/pkgconf and app-misc/pax-utils) why do we need pax-utils? is clang/llvm insisting on scanmacho somehow? > - stage2: drop ssl useflag for app-crypt/rhash (only applies to stage2 build) hmmm, that's odd, we take great care to actually have ssl available for wget > - a one char typo fix in a comment (In reply to Fabian Groffen from comment #29) > (In reply to Jacob Floyd from comment #28) > > - stage1: bootstrap_libtapi > stage1 is for tools only that are necessary to run Portage, tapi is not one > of those, so we should be able to just emerge it instead We could do that in stage2 then, but building libtapi before building binutils-apple doesn't work, because in order to build libtapi, we need a portage installed llvm and clang. Plus, to build libtapi, you essentially have to re-build llvm and clang to get some private codegen utilities needed to build libtapi. So, I grabbed the headers to escape the circular build requirements (needed whether or not portage is actually checking deps). > > - stage1/stage2: set up ${EPREFIX}/MacOSX.sdk and related include paths to > > use it for the clang build, not just for GCC. > ack > > - stage2: add additional build deps (dev-util/pkgconf and app-misc/pax-utils) > why do we need pax-utils? is clang/llvm insisting on scanmacho somehow? Interesting. I just rebuilt without including pkgconf and pax-utils and I no longer hit the errors. I believe it was an issue somewhere in the eclasses where they expected to use pkg-config and scanmacho, but readline-8.1_rc3 and xz-utils-5.2.5 don't seem to be hitting the issues, so maybe there were some tree updates that fixed this. I'll update the patch to drop these. > > - stage2: drop ssl useflag for app-crypt/rhash (only applies to stage2 build) > hmmm, that's odd, we take great care to actually have ssl available for wget No, this doesn't drop ssl support anywhere that would affect wget. It merely tells rhash to use internal, less-optimized, version of various algorithms. With the ssl useflag, it uses the openssl library for more optimized versions of those same algorithms. > > - a one char typo fix in a comment (In reply to Jacob Floyd from comment #30) > (In reply to Fabian Groffen from comment #29) > > (In reply to Jacob Floyd from comment #28) > > > - stage1: bootstrap_libtapi > > stage1 is for tools only that are necessary to run Portage, tapi is not one > > of those, so we should be able to just emerge it instead > > We could do that in stage2 then, but building libtapi before building > binutils-apple doesn't work, because in order to build libtapi, we need a > portage installed llvm and clang. Plus, to build libtapi, you essentially > have to re-build llvm and clang to get some private codegen utilities needed > to build libtapi. So, I grabbed the headers to escape the circular build > requirements (needed whether or not portage is actually checking deps). > I get they need to build Clang/LLVM to grab the internal stuff, but do we need Clang/LLVM to *build* it? Created attachment 675748 [details, diff]
bootstrap-prefix.sh modifications for stage1/stage2 using clang
updated to drop pkgconf and pax-utils
(In reply to Sam James from comment #31) > (In reply to Jacob Floyd from comment #30) > > (In reply to Fabian Groffen from comment #29) > > > (In reply to Jacob Floyd from comment #28) > > > > - stage1: bootstrap_libtapi > > > stage1 is for tools only that are necessary to run Portage, tapi is not one > > > of those, so we should be able to just emerge it instead > > > > We could do that in stage2 then, but building libtapi before building > > binutils-apple doesn't work, because in order to build libtapi, we need a > > portage installed llvm and clang. Plus, to build libtapi, you essentially > > have to re-build llvm and clang to get some private codegen utilities needed > > to build libtapi. So, I grabbed the headers to escape the circular build > > requirements (needed whether or not portage is actually checking deps). > > > > I get they need to build Clang/LLVM to grab the internal stuff, but do we > need Clang/LLVM to *build* it? I just tried again unsuccessfully. > >>> Emerging (1 of 1) sys-libs/tapi-2.0.0::gentoo_prefix > Warning: File compression unsupported zstd. Missing package: app-arch/zstd > * clang-800.0.42.1.tar.gz BLAKE2B SHA512 size ;-) ... [ ok ] > * llvm-5.0.1.src.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ] > * cfe-5.0.1.src.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ] > * tapi-2.0.0.tar.gz BLAKE2B SHA512 size ;-) ... [ ok ] > * ERROR: sys-libs/tapi-2.0.0::gentoo_prefix failed (setup phase): > * No LLVM slot found installed! This comes from llvm.eclass -> llvm_pkg_setup -> get_llvm_prefix -> has_version. Even if we can somehow get around these issues so that it trys building llvm/clang without llvm/clang being installed, I don't think it will be able to build llvm because we don't have cmake and the other llvm deps yet when building the linker. Plus, I haven't worked on getting llvm/clang building right yet, so I don't know what errors to expect. It might be possible to return to this again once I get llvm/clang building to see if there's a way to work around the eclass to insert sys-libs/tapi before binutils-apple. Created attachment 675772 [details, diff]
bootstrap-prefix.sh modifications for stage1/stage2 using clang
Moved bootstrap_libtapi into stage2 with comments explaining why.
macOS Big Sur bootstraps are working now let's use bug #757513 for the Clang parts Hello, I'm getting what looks like a similar error when building readline, bootstrapping on Big Sur. I set DARWIN_USE_GCC=1 before I ran bootstrap-prefix.sh. Is this the same issue?
make[1]: Leaving directory '/Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/work/readline-8.1-abi_x86_64.amd64/shlib'
>>> Completed installing sys-libs/readline-8.1 into /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/image
* Final size of build directory: 10940 KiB (10.6 MiB)
* Final size of installed tree: 3404 KiB ( 3.3 MiB)
* QA Notice: command not found:
*
* /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/temp/environment: line 2588: pkg-config: command not found
* /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/temp/environment: line 2581: pkg-config: command not found
* /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/temp/environment: line 2288: pkg-config: command not found
* /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/temp/environment: line 2325: pkg-config: command not found
/Users/ash/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 65: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
/Users/ash/Gentoo/tmp/usr/lib/portage/bin/ecompress: line 35: /Users/ash/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.1/temp/.ecompress_skip_files: Interrupted system call
* ERROR: sys-libs/readline-8.1::gentoo_prefix failed:
* (no error message)
(In reply to ash from comment #36) > Hello, I'm getting what looks like a similar error when building readline, > bootstrapping on Big Sur. I set DARWIN_USE_GCC=1 before I ran > bootstrap-prefix.sh. Is this the same issue? > Please file a new bug - but the issue here is: >...temp/.ecompress_skip_files: Interrupted system call The gist seemed to be a overheating/overloaded system. |
