As agreed in https://bugs.gentoo.org/show_bug.cgi?id=474748, this bug shall track inclusion of ebuilds for libc++ and libc++abi for prefix on OS X. I'll attach the current versions I've got. One question I'd like to raise immediately: Should we really fork out -apple-Versions of the ebuilds or does standard Gentoo have something we can and should merge with? Last time I looked, there was a libcxx ebuild but that depended on libcxxrt or somesuch and looked very pre-release. Reproducible: Always
Created attachment 395264 [details] ebuilds for libc++ and libc++abi
reason for the -apple versions is/was that they build from apple sources, which are heavily tampered with, so really different (not suitable for anything but apple, I guess)
Okay, that's certainly not the case here: It's vanilla upstream sources just compiled with some hacks for Apple systems. So I'll check current Gentoo portage for libc++abi and libc++ and try to patch whatever's there to build the way we need on a Mac.
Created attachment 395852 [details, diff] updated libcxx-9999 ebuild
Created attachment 395854 [details] ebuilds for libcxx-headers, libcxxabi and libcx-3.5.1 There is no libcxxabi ebuild in the tree yet. So I've just renamed the libcxx-apple-headers to libcxx-headers and libcxxabi-apple to libcxxabi. I've added some minor platform checks but left them otherwise untouched. I especially did not multiabi-ise libcxxabi because we don't need that for Darwin and I'm not sure if it actually runs anywhere else yet. I've added the Darwin stuff to the existing libcxx ebuild. I've tried a minimally invasive approach by adding CHOST-darwin blocks with returns at the ends to keep the diff small. Other changes include: - renamed libcxxrt USE flag to libsupc and inverted meaning because: - I set REQUIRED_USE so that libsupc and static-libs *must not* be enabled on Darwin I stuck with the ./buildit script for actual building because I do not see the benefit in adding support for Darwin to the ebuild's custom Makefile. buildit does all we need and upstream maintains it for us. If those disparate code paths in the ebuild are a concern, we could consider switching to the cmake build system and tweak that so it works for building on all platforms.
Ah, one thing: Both ebuilds (libcxxabi and libcxx) will simply fail if CC and CXX are not pointing at clang. I assume, we can and should use the darwin or macos profile to make that the default for those packages.
Created attachment 395882 [details, diff] update binutils ebuilds to require libcxx instead of libcxx-apple on USE=libcxx
where did you get the 3.5.1 version number from? I prefer to have a tarball-ed version for bootstrap/general use instead of a live ebuild.
llvm.org is doing proper releases of libc++ and libc++abi now: http://llvm.org/releases/download.html#3.5.1 http://llvm.org/releases/3.5.1/libcxx-3.5.1.src.tar.xz http://llvm.org/releases/3.5.1/libcxxabi-3.5.1.src.tar.xz I updated the -9999-ebuild to keep it identical to the -3.5.1-one because I thought it might increase chances of Anthony and Alexis not objecting to our hacking their ebuild if we showed some effort to not completely fork it. ;)
That's appreciated, although the USE-rename will not help for that case. Since most bits are adding conditional stuff, it should be ok. I was about to not commit the USE-rename.
Yep, I don't like the USE rename either. It's IIRC only about the warnings by src_prepare. Alternatives are: - require use libcxxrt being off on Darwin and let pkg_setup always print its warning - make warning of use libcxxrt being off conditional on Darwin Mhmm, I just realised that after all this thought about warnings and renaming I added the CC/CXX checking block with a return at the end anyway, effectively implementing alternative 2. New ebuild forthcoming.
Created attachment 396062 [details, diff] updated patch to libcxx-9999.ebuild
Created attachment 396064 [details] updated ebuilds for libcxx-headers, libcxxabi and libcxx-3.5.1
@aballier: do you agree with adding libcxx-3.5.1 and adding the Darwin support?
(In reply to Fabian Groffen from comment #14) > @aballier: do you agree with adding libcxx-3.5.1 and adding the Darwin > support? yes, give me some time to do the bump; I hadn't noticed they're now doing releases as for the patches, I'd have some comments: +REQUIRED_USE="kernel_Darwin? ( libcxxrt !static-libs )" better use package.use.force / .mask as on amd64-fbsd here also, !static-libs means 'clang++ -stdlib=libc++ -static ...' won't work; this was a problem on fbsd as some core packages were written in c++ and linking statically. + if [[ ${CHOST} == *darwin* ]] ; then + MY_CC=$(tc-getCC) + MY_CXX=$(tc-getCXX) + if [[ ${MY_CC} != *clang* || ${MY_CXX} != *clang++* ]] ; then + eerror "${PN} needs to be built with clang++. Please do not override" + eerror "CC ($MY_CC) and CXX ($MY_CXX)" + eerror "or point them at clang and eerror clang++ respectively." + die + fi + return + fi I remember, when gcc wasn't able to build it, doing something like: DEPEND=clang export CC=clang CXX=clang++ src_configure() { + tc-export AR CC CXX + + # on Darwin we're all set + [[ ${CHOST} == *darwin* ]] && return + ... multilib_src_compile() { cd "${BUILD_DIR}/lib" || die + if [[ ${CHOST} == *darwin* ]] ; then + TRIPLE=-apple- ./buildit || die + return + fi + I'd like to avoid using that buildit script, current src_configure code sets things up to avoid hacking it like you do in src_prepare + if [[ ${CHOST} == *darwin* ]] ; then + dolib.so libc++*dylib + return + fi + use get_libname from multlib.eclass (nfc why it's in this eclass and not some more general one though) you certainly want the ldscripts: these make the '-lc++' (used internally in clang 'specs') work properly, in both static and shared cases
also, I take it you want to split libcxx from its headers to break the circular dep between libcxx & libcxxabi, however, you seem to be patching the headers in both libcxx & libcxx-headers, there must be a better way
> also, !static-libs means 'clang++ -stdlib=libc++ -static ...' won't work; As far as I'm aware, there's no way to produce a fully statically linked binary on OS X. Also Apple does not provide a static version of libc++ with the system or the devel tools. So I seriously doubt that static libc++ is supported or even ever tested on OS X. > I remember, when gcc wasn't able to build it, doing something like: > DEPEND=clang > export CC=clang CXX=clang++ I was trying to avoid a silent, non-optional override. Instead I was thinking, what do I know what compiler the user is actually trying to build me with? Let's just warn them that what they're doing will most likely fail. > I'd like to avoid using that buildit script, current src_configure code sets > things up to avoid hacking it like you do in src_prepare On the other hand I'd like to avoid reinventing build logic that upstream already has implemented and is maintaining for us. Apart from lib/buildit, libcxx-3.6.0 comes with a standard Makefile and cmake support as it seems. From what I gather, there seems to be a move to cmake underway for all of the llvm.org projects. Have you had a look at the libcxx cmake build system already? Can we perhaps drop buildit and a Gentoo-custom Makefile in favour of upstream's cmake system (or their Makefile)? > you certainly want the ldscripts: these make the '-lc++' (used internally in > clang 'specs') work properly, in both static and shared cases OS X has its own linker (ld64). As far as I'm aware, it doesn't support ldscripts. > also, I take it you want to split libcxx from its headers to break the > circular dep between libcxx & libcxxabi, however, you seem to be patching the > headers in both libcxx & libcxx-headers, there must be a better way We could compile libc++ against the installed headers of libcxx-headers. But I suspect if we add a version-specific dependency on libcxx-headers to libcxx, upgrading might get tricky in some cases. And I do not know if it's actually required to use exactly matching headers when compiling/linking against libc++ or if there's some kind of ABI compatibility. If it's actually allowed to compile with headers of version 3.5.0 but link against library of version 3.6.0 then I'd view the packages as clearly separate. Do you know more?
(In reply to Michael Weiser from comment #17) > > also, !static-libs means 'clang++ -stdlib=libc++ -static ...' won't work; > > As far as I'm aware, there's no way to produce a fully statically linked > binary on OS X. Also Apple does not provide a static version of libc++ with > the system or the devel tools. So I seriously doubt that static libc++ is > supported or even ever tested on OS X. Then it should be tested, and if it doesnt work, masked on relevant profiles > > I remember, when gcc wasn't able to build it, doing something like: > > DEPEND=clang > > export CC=clang CXX=clang++ > > I was trying to avoid a silent, non-optional override. Instead I was > thinking, what do I know what compiler the user is actually trying to build > me with? Let's just warn them that what they're doing will most likely fail. Well, I'm not sure, it probably makes sense on OSX since clang should be default (afaik); on linux or fbsd, gcc was/is the default, so having something that doesn't build by default didn't make sense. > > I'd like to avoid using that buildit script, current src_configure code sets > > things up to avoid hacking it like you do in src_prepare > > On the other hand I'd like to avoid reinventing build logic that upstream > already has implemented and is maintaining for us. Did you have a look at that buildit script ? It really is just a makefile ersatz, without parallel job support, and hardcoding a lot of stuff. > Apart from lib/buildit, libcxx-3.6.0 comes with a standard Makefile and > cmake support as it seems. From what I gather, there seems to be a move to > cmake underway for all of the llvm.org projects. Have you had a look at the > libcxx cmake build system already? Can we perhaps drop buildit and a > Gentoo-custom Makefile in favour of upstream's cmake system (or their > Makefile)? This is something that might be worth having a look at. I think I did the makefile stuff because their buildit script sucked. > > you certainly want the ldscripts: these make the '-lc++' (used internally in > > clang 'specs') work properly, in both static and shared cases > > OS X has its own linker (ld64). As far as I'm aware, it doesn't support > ldscripts. Then you might want to file a bug for toolchain-funcs.eclass to drop darwin support in gen_usr_ldscript :) > > also, I take it you want to split libcxx from its headers to break the > > circular dep between libcxx & libcxxabi, however, you seem to be patching the > > headers in both libcxx & libcxx-headers, there must be a better way > > We could compile libc++ against the installed headers of libcxx-headers. But > I suspect if we add a version-specific dependency on libcxx-headers to > libcxx, upgrading might get tricky in some cases. how? > And I do not know if it's > actually required to use exactly matching headers when compiling/linking > against libc++ or if there's some kind of ABI compatibility. If it's > actually allowed to compile with headers of version 3.5.0 but link against > library of version 3.6.0 then I'd view the packages as clearly separate. Do > you know more? I am not sure, but I certainly wouldn't compile 3.6 with 3.5 headers. I'd ~ pin the dep. After all, there must be a reason upstream ships them together :)
(In reply to Alexis Ballier from comment #18) > > > you certainly want the ldscripts: these make the '-lc++' (used internally in > > > clang 'specs') work properly, in both static and shared cases > > > > OS X has its own linker (ld64). As far as I'm aware, it doesn't support > > ldscripts. > > Then you might want to file a bug for toolchain-funcs.eclass to drop darwin > support in gen_usr_ldscript :) It creates symlinks for most non-ELF targets.
Ok, I'll take a stab at updated libcxx{,-abi,-headers}-3.6.0 ebuilds with this new information and have a look at cmake as the build system.
(In reply to Michael Weiser from comment #20) > Ok, I'll take a stab at updated libcxx{,-abi,-headers}-3.6.0 ebuilds with > this new information and have a look at cmake as the build system. heh, now i remember: 03 Jul 2013; Alexis Ballier <aballier@gentoo.org> libcxx-9999.ebuild, +files/Makefile: Use a simple Makefile instead of cmake for building it and drop our patches. It no longer needs to be built with clang. I did this because cmake wasn't really useful, and, most importantly, requires a working c++ stack. Imagine, you have a clang only system, broke libc++ but built cmake against it, then you can't rebuild libcxx...
(In reply to Alexis Ballier from comment #21) > (In reply to Michael Weiser from comment #20) > > Ok, I'll take a stab at updated libcxx{,-abi,-headers}-3.6.0 ebuilds with > > this new information and have a look at cmake as the build system. > > heh, now i remember: > > 03 Jul 2013; Alexis Ballier <aballier@gentoo.org> libcxx-9999.ebuild, > +files/Makefile: > Use a simple Makefile instead of cmake for building it and drop our > patches. > It no longer needs to be built with clang. > > > I did this because cmake wasn't really useful, and, most importantly, > requires a working c++ stack. > Imagine, you have a clang only system, broke libc++ but built cmake against > it, then you can't rebuild libcxx... and thinking more about it, it is probably irrelevant since clang also depends on a c++ stack...
Created attachment 408380 [details] updated ebuild for libcxx-headers-3.6.2 I haven't gotten around to implementing cmake support in the ebuild as promised. For the time being, here are updated ebuilds for versions 3.6.2 of libcxxabi and libcxx.
Created attachment 408382 [details] updated ebuild for libc++abi 3.6.2
Created attachment 408384 [details] updated ebuild for libc++ 3.6.2
Created attachment 450912 [details] updated ebuild for libcxx-headers-3.9.0 Because lib/buildit is not working any more as of 3.9.0 I took the opportunity to port the ebuilds for libcxx-headers, libcxxabi and libcxx over to cmake-utils and cmake-multilib as needed.
Created attachment 450914 [details] updated ebuild for libcxxabi-3.9.0
Created attachment 450916 [details] updated ebuild for libcxx-3.9.0 Notable changes from the old ebuild: - let cmake generate the ldscripts (untested since apparently not needed on OS X) - use libc++abi as ABI library by default since as per upstream documentation it's available and preferred for Linux as well now - expose feature of statically linking the ABI library via use flag (untested since not supported on OS X) - feedback on how the static libs are built appreciated Alexis: Does this ebuild or at least the direction it's traveling jive for you?
I added this in src_prepare: + # make sure we build multilib on OSX, because llvm insists on + # building multilib too + [[ ${CHOST} == *86*-darwin* ]] && append-flags -arch i386 -arch x86_64 llvm-3.9.0 fails to build looking for the i386 variant else.
FWIW, I've done some work on the 3.9.1 to make them much closer to their gx86 variants. The changes are: - deprecation of libcxx-headers: libcxx provides the headers in gx86, and libcxxabi just pulls in the headers in the temporary buildspace, so I thought let's just follow that, doesn't break anything for us -> headers still there afterwards, stuff still compiles. - cmake defines stuff, dropped our stuff mostly, for it seemed (no longer) necessary. The problem we're facing now, is that we need the 3.5.1 versions (like we have in Prefix) in order to bootstrap, and those versions are quite different from the more recent ones.
Created attachment 511952 [details] ebuild for libcxxabi-5.0.1 This ebuild is a straight clone of libcxxabi-3.9.1.ebuild with patches removed because they're now included in the upstream release and a workaround for a test for the c++ standard library leaking in from the LLVM cmake module: Since we are building part of the C++ standard library and that part is self-contained, we can safely disable it. This affects bootstrap because during bootstrap the freshly compiled C++ compiler will not have s C++ standard library when we first compile libcxxabi. @@ -87,6 +78,11 @@ if [[ ${CHOST} == *86*-darwin* ]] ; then append-flags -arch i386 -arch x86_64 append-cxxflags -std=c++11 + local mycmakeargs+=( + # disable test for libstdc++ leaking in from LLVM cmake module - + # apart from libcxx headers, libcxxabi is self-contained + -DLLVM_ENABLE_LIBCXX=ON + ) fi cmake-utils_src_configure
Created attachment 511954 [details] ebuild for libcxx-5.0.1 This is a straight clone of libcxx-3.9.1.ebuild with the static lib patch removed because that was a backport and seems to be in the upstream release now.
Created attachment 511982 [details, diff] Defuse static path references to libc++abi.dylib Not good: # otool -L usr/lib/libc++.dylib usr/lib/libc++.dylib: @rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libc++abi.dylib (compatibility version 1.0.0, current version 400.7.0) @rpath/libc++abi.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0) Attached patch to the ebuild eprefixifies the static path reference in some symbol re-export magic and avoids linking it in twice.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=dfb79c4a7df3b2a3c2f47f46b241e8213c174c2e commit dfb79c4a7df3b2a3c2f47f46b241e8213c174c2e Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2018-01-02 16:21:33 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2018-01-02 16:21:33 +0000 sys-libs/libcxx: fix path references by Michael Weiser, bug #538364 Closes: https://bugs.gentoo.org/538364 Package-Manager: Portage-2.3.18-prefix, Repoman-2.3.6 sys-libs/libcxx/libcxx-5.0.1.ebuild | 9 +++++++++ 1 file changed, 9 insertions(+) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=ce4edc8cdd9d6a2ec822de9ce37febb8b05402ff commit ce4edc8cdd9d6a2ec822de9ce37febb8b05402ff Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2018-01-02 16:18:01 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2018-01-02 16:18:01 +0000 sys-libs/libcxx: bump to 5.0.1 by Michael Weiser, bug #538364 Bug: https://bugs.gentoo.org/538364 Package-Manager: Portage-2.3.18-prefix, Repoman-2.3.6 sys-libs/libcxx/Manifest | 2 +- sys-libs/libcxx/libcxx-3.9.0.ebuild | 157 ------------------------ sys-libs/libcxx/libcxx-5.0.1.ebuild | 230 ++++++++++++++++++++++++++++++++++++ 3 files changed, 231 insertions(+), 158 deletions(-) https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=68b93b4e5c261672b8d053a5adfe10510fa3dde8 commit 68b93b4e5c261672b8d053a5adfe10510fa3dde8 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2018-01-02 16:15:31 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2018-01-02 16:15:31 +0000 sys-libs/libcxxabi: bump to 5.0.1 by Michael Weiser, bug #538364 Bug: https://bugs.gentoo.org/538364 Package-Manager: Portage-2.3.18-prefix, Repoman-2.3.6 sys-libs/libcxxabi/Manifest | 2 + sys-libs/libcxxabi/libcxxabi-5.0.1.ebuild | 103 ++++++++++++++++++++++++++++++ 2 files changed, 105 insertions(+)}
Can confirm that both compile and run fine on 10.13.
