Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 642656 - bootstrap-prefix.sh: Changes necessary to bootstrap directly into LLVM/clang-5.0.1
Summary: bootstrap-prefix.sh: Changes necessary to bootstrap directly into LLVM/clang-...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All OS X
: Normal normal
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
Depends on: 538364 642616 642644
Blocks:
  Show dependency tree
 
Reported: 2017-12-29 14:46 UTC by Michael Weiser
Modified: 2018-01-04 09:48 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Changes to bootstrap-prefix.sh (bootstrap-prefix.sh.patch,2.56 KB, patch)
2017-12-29 14:53 UTC, Michael Weiser
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Weiser 2017-12-29 14:46:16 UTC
Some settings in bootstrap-prefix.sh prevent bootstrap directly into LLVM/clang-5.0.1:

1. libcxx-headers is still requested but does not exist any more for new versions.
2. libcxx* is requested in versions < 3.6 for bootstraps directly into current LLVM/clang. This collides with the compiler's dependencies on current versions of the libs.
3. cmake-3.10 wants to use the system libuv even without use server.
4. llvm/clang slot bin directory needs to be added to PREROOTPATH.

Reproducible: Always
Comment 1 Michael Weiser 2017-12-29 14:53:23 UTC
Created attachment 511970 [details, diff]
Changes to bootstrap-prefix.sh

This is just to show the workarounds that enable bootstrap directly into LLVM/clang-5. No idea how to fix this properly.

Also, bootstrap via intermediate LLVM/clang-3.4 remains broken because libcxx-headers will most likely block upgrade of libcxx{,abi}. libcxx-headers would still need to be eradicated for the old versions as well (https://bugs.gentoo.org/538364) and somehow declared as replaced by libcxx so it's automatically unmerged upon upgrade.
Comment 2 Michael Weiser 2017-12-29 15:01:38 UTC
With these changes and LATEST_TREE_YES=1 I am able to bootstrap directly into LLVM/clang-5.0.1 on 10.12 and 10.13. Additional resources needed:
- ldwrapper with -rpath from https://bugs.gentoo.org/642616
- libcxx* ebuilds from https://bugs.gentoo.org/538364
- clang ebuilds from https://bugs.gentoo.org/642644
- changes to cmake-utils.eclass (bug with details forthcoming)
- package.keywords keywording sys-devel/llvm, sys-devel/llvm-common, sys-devel/clang, sys-devel/clang-runtime, sys-libs/compiler-rt, sys-libs/compiler-rt-sanitizers * ~* for the time being
- package.use: dev-lang/python -aqua
Comment 3 Michael Weiser 2017-12-29 15:33:28 UTC
... and I needed to patch the stage1 m4. Did that by hand (lib/vasnprintf.c) after unpacking being too lazy to add any automation to bootstrap-prefix.sh.

Also, stage 3 fails with portage (rather wget) being unable to download files from HTTPS URLs. Just restarting the exact same command then instantly merges wget against the already installed openssl and portage can continue. I have not looked into the reason for that.
Comment 4 Michael Weiser 2017-12-29 19:33:46 UTC
> - clang ebuilds from https://bugs.gentoo.org/642644
https://bugs.gentoo.org/642668
Comment 5 Michael Weiser 2017-12-29 19:34:28 UTC
> - changes to cmake-utils.eclass (bug with details forthcoming) https://bugs.gentoo.org/642668
Comment 6 Michael Weiser 2017-12-30 16:56:00 UTC
I tried bootstrapping on 10.8. It needs some help to work itself out of llvm-3.4.2 but looks good regarding compilation of 5.0.1 versions of LLVM components. I have noticed two problems that would need fixing for older macOS releases:

1. binutils-config wrapper in ${ROOT}/tmp adds rpaths to ${ROOT}/tmp/{,usr/}lib. This is fine and necessary during stage2 but problematic in stage3 when compiling into ${ROOT}. In particular it makes stage 3 compier binaries in ${ROOT} load ${ROOT}/tmp/usr/lib/libc++.1.dylib, which is an older version than what they were compiled against. This is bound to cause breakage. Maybe at the end of stage2 binutils-config could be installed again into ${ROOT}/tmp but targetting ${ROOT}?

2. Whether clang defaults to -stdlib=libc++ is system not compiler version dependant. On 10.8 clang-3.4.2 as well as stage 3 clang-5.0.1 need intensive help finding the library headers as well as linking against it.
Comment 7 Michael Weiser 2017-12-30 17:15:34 UTC
While the stdlib default can be overridden in clang-5.0.1 (cmake -DCLANG_DEFAULT_CXX_STDLIB=libc++), it's hardcoded in 3.4.2: libstdc++ for <10.10 and libc++ for >=10.10. So bootstrap would need to inject -stdlib=libc++ and maybe -I${ROOT}/usr/include/c++/v1 into CXXFLAGS and LDFLAGS(!).

On the positive side: libcxx-headers is a non-issue since it only ever appears in ${ROOT}/tmp and poses no conflict when merging newer versions without a -headers package into ${ROOT}.
Comment 8 Michael Weiser 2017-12-30 17:25:58 UTC
Just BTW: Bootstrap on 10.5 x86_64 is not a happy camper:

- binutils-apple-3.2 are missing
- gmp-6.1.2 runs into https://bugs.gentoo.org/536894 again
- gcc-apple-4.2.1 fails to compile because binutils-apple-4.3-r1 ld64 segfaults on  configure tests in libstdc++-v3 which makes it fail to configure (with unhelpful configure: error: Link tests are not allowed after GCC_NO_EXECUTABLES.)

I gave up there, so can't say how LLVM/clang and @rpath would have done.
Comment 9 Larry the Git Cow gentoo-dev 2018-01-02 16:27:34 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=dae1cff11a36860ac0ddd1ae18cd02daec894c8a

commit dae1cff11a36860ac0ddd1ae18cd02daec894c8a
Author:     Fabian Groffen <grobian@gentoo.org>
AuthorDate: 2018-01-02 16:26:54 +0000
Commit:     Fabian Groffen <grobian@gentoo.org>
CommitDate: 2018-01-02 16:26:54 +0000

    scripts/bootstrap-prefix.sh: change Darwin bootstrap to use clang5
    
    changes by Michael Weiser
    
    Bug: https://bugs.gentoo.org/642656

 scripts/bootstrap-prefix.sh | 25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)}
Comment 10 Fabian Groffen gentoo-dev 2018-01-02 17:48:45 UTC
this obviously won't work until I bump the snapshot, which I intend to do tomorrow
Comment 11 Michael Weiser 2018-01-02 18:05:43 UTC
The MYCMAKEARGS="-DCMAKE_USE_SYSTEM_LIBRARY_LIBUV=OFF" workaround also causes warnings from cmake packages other than cmake itself because they don't define it. It's just warnings though, everything builds fine.
Comment 12 Fabian Groffen gentoo-dev 2018-01-03 14:58:35 UTC
I didn't manage to add keywords to llvm/clang in time, so the snapshot has to wait until tomorrow.
Comment 13 Fabian Groffen gentoo-dev 2018-01-03 20:53:47 UTC
(In reply to Michael Weiser from comment #8)
> Just BTW: Bootstrap on 10.5 x86_64 is not a happy camper:
> 
> - binutils-apple-3.2 are missing
> - gmp-6.1.2 runs into https://bugs.gentoo.org/536894 again
> - gcc-apple-4.2.1 fails to compile because binutils-apple-4.3-r1 ld64
> segfaults on  configure tests in libstdc++-v3 which makes it fail to
> configure (with unhelpful configure: error: Link tests are not allowed after
> GCC_NO_EXECUTABLES.)
> 
> I gave up there, so can't say how LLVM/clang and @rpath would have done.

I've re-patched gmp-6.1.2 and re-added binutils-apple-3.2
Comment 14 Larry the Git Cow gentoo-dev 2018-01-04 09:46:24 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=bd802a20f3c8d250fd85ed66272df93583edde2a

commit bd802a20f3c8d250fd85ed66272df93583edde2a
Author:     Fabian Groffen <grobian@gentoo.org>
AuthorDate: 2018-01-04 09:46:00 +0000
Commit:     Fabian Groffen <grobian@gentoo.org>
CommitDate: 2018-01-04 09:46:00 +0000

    scripts/bootstrap-prefix: update snapshots
    
    Bug: https://bugs.gentoo.org/642656
    Bug: https://bugs.gentoo.org/618396
    Bug: https://bugs.gentoo.org/632500
    Bug: https://bugs.gentoo.org/638816

 scripts/bootstrap-prefix.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)}
Comment 15 Fabian Groffen gentoo-dev 2018-01-04 09:48:19 UTC
I think we're done here, everything is pushed