Some fact: 1. sys-devel/gcc-apple stays on 4.2.1 and probably will stay on that version forever 2. sys-devel/gcc is not supported (“Using FSF GCC on Mac OS X is not supported by Gentoo Prefix”). 3. clang is becoming the standard toolchain on OSX So, should we consider using clang (along with apple’s binutils) as the default compiler on OSX prefix? I’m asking from a user’s perspective, and I’m not sure how much effort would be involved in changing the default compiler so forgive me if my question seems too naive. Reproducible: Always
FYI, We have some thought of use clang. http://thread.gmane.org/gmane.linux.gentoo.summer-of-code/1361 But no actually plans for it yet.
accepting clang, as much as fsf-gcc isn't much of an issue, as long as we don't get major pains from packages not compiling. Apple's gcc currently is able to compile everything from system and beyond, that's the real reason for only supporting that one.
i think this needs to be revisited. I spend an awful lot of time looking into it. You can compile clang when fiddling with the ebuild on darwin. I already use it for certain projects, because they fail to compile with gcc, and even if they do the linker will fail. GCC can't compile projects that include nsnotification.h on mavericks. That includes python 2.7. However the linker is outdated for clang. Using the current linker means you have to add -lcrt1.o manually to the linker flags. Using the xcode bundled linker on the other hand works. It also means that I have to add the gentoo library search paths manually. There are new dyld and ld utils, but they require libc++ headers not libstdc++ and clang. (it's like a loop and I'm not entirely sure what component is the first) How exactly does the bootstrap with gcc-apple work? can anyone with some more internal knowledge spare some time for a 5-30 minute chat?
(In reply to Reza Jelveh from comment #3) > i think this needs to be revisited. I spend an awful lot of time looking > into it. welcome to my world ;) > You can compile clang when fiddling with the ebuild on darwin. I already use > it for certain projects, because they fail to compile with gcc, and even if > they do the linker will fail. > > GCC can't compile projects that include nsnotification.h on mavericks. That > includes python 2.7. > > However the linker is outdated for clang. Using the current linker means you > have to add -lcrt1.o manually to the linker flags. Using the xcode bundled > linker on the other hand works. It also means that I have to add the gentoo > library search paths manually. > > There are new dyld and ld utils, but they require libc++ headers not > libstdc++ and clang. (it's like a loop and I'm not entirely sure what > component is the first) Thanks for diving into this. I'm affraid there indeed is a nice circular bootstrapping dependency to get a full clang toolchain installed. > How exactly does the bootstrap with gcc-apple work? can anyone with some > more internal knowledge spare some time for a 5-30 minute chat? Not sure what you actually are after here. The bootstrap is "documented" in the bootstrap-prefix.sh script. It boils down to compiling binutils-apple, followed by gcc-apple. Another option perhaps could be to investigate the FSF GCC line.
*** Bug 493934 has been marked as a duplicate of this bug. ***
https://bugs.gentoo.org/show_bug.cgi?id=493934#c5 gives interesting information on a possible path forwards
(In reply to Fabian Groffen from comment #4) > > Not sure what you actually are after here. The bootstrap is "documented" in > the bootstrap-prefix.sh script. It boils down to compiling binutils-apple, > followed by gcc-apple. > > Another option perhaps could be to investigate the FSF GCC line. thanks for looking into this again fabian! what i meant is that there seems to have been quite a lot of work into making gcc-apple work the way it does. the most basic patches are hardcoding prefix directories in. then there are patches to get lib unwind working and more. some of these are necessary some of them not really(like the libunwind i think), but generally compiling clang and the new osx binutils seems to be less fun. i was just trying to understand how the whole toolchain works, I just assumed that you knew more about this than me already. also, something is odd about the apple linker. when gnu binutils encounter duplicate symbols generated by gcc it will ignore the duplicates. the apple ldtools will instead fail completely. this problem does not occur when you use clang as compiler even though the duplicate symbols still exist afaict
i started working on building the binutils. see my github for reference. ld builds so far, and after replacing the gentoo ld things compile fine now even with -mmacosx-version-min=10.8 https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/binutils-apple/binutils-apple-4.5.ebuild
Created attachment 371990 [details] ld64-136 make file, with snapshot.o
Created attachment 371992 [details, diff] liblto patch for ld64-136 also sets ALL_SUPPORTED_ARCHS
Created attachment 371994 [details] compile_stubs script as header for snapshot.cpp normally generated by pbxproj
Created attachment 372004 [details, diff] eprefix library search path needs to be added, happens with a sed in ebuild
a little caveat, i'm cheating in so far that i'm still using the old cctools. also if you build it without using the -mminosx version set to 10.6 it will start looking for crashreporter, which is in libc. that seems to be a little messy to deal with. so far i can finally compile apps like mpv that require a newer compile than gcc 4.2
Created attachment 372008 [details, diff] lto patch to allow cctools to build
i now have a fully working updated binutils toolchain :) but there's a little cheat happening. since i copied over the ld64 before finishing my ebuild, we need to set mmacos-min-ver at the right places in the ebuild. anyway, it should be mainly cleaning up now
Created attachment 372012 [details, diff] preliminary ebuild patch this works for now but needs to be cleaned up
Created attachment 372020 [details, diff] updated preliminary patch with some cleanups
i've updated the binutils-apple patches to support the latest cctools https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/binutils-apple it requires libc++ https://github.com/fishman/timebomb-gentoo-osx-overlay/tree/master/sys-libs/libcxx-apple
I'm stealing your llvm compile fix patch. Once I got a clang, I'll steal your work for binutils-apple too, as it seems building libcxx is impossible without c++11, so is binutils-apple. I'm hoping that we can go the following route: - after bootstrap gcc-apple-4.2.1 + binutils-apple-4.3 - get llvm/clang with this toolchain - update to binutils-apple-5.1 - make clang default compiler somehow Alternatively get the bootstrap to compile llvm/clang with the host-provided clang/gcc-apple. Seems like that should be possible then too. Will see where this ends. I didn't manage to complete the compile of llvm/clang yet.
Created attachment 383032 [details] libcxx ebuild
(In reply to Fabian Groffen from comment #19) > I'm stealing your llvm compile fix patch. Once I got a clang, I'll steal > your work for binutils-apple too, as it seems building libcxx is impossible > without c++11, so is binutils-apple. > > I'm hoping that we can go the following route: > - after bootstrap gcc-apple-4.2.1 + binutils-apple-4.3 > - get llvm/clang with this toolchain > - update to binutils-apple-5.1 > - make clang default compiler somehow > > Alternatively get the bootstrap to compile llvm/clang with the host-provided > clang/gcc-apple. Seems like that should be possible then too. > > Will see where this ends. I didn't manage to complete the compile of > llvm/clang yet. sounds good, a note on that. clang needs libcxx to build, which needs clang to build. also I use clang env with this package.env https://gist.github.com/85e58ce3e557d350058f CC=clang CXX=clang++ CFLAGS="${CFLAGS} -stdlib=libstdc++" CXXFLAGS="${CXXFLAGS} -stdlib=libstdc++" libcxx is needed sometimes particularly for c++11 but in general compatibility and side effects are reduced with stdlib=libstdc++. if you can mention me in the release notes somewhere ;) you can also contact me directly for questions, here's my overlay: https://github.com/fishman/timebomb-gentoo-osx-overlay
I'm able to compile/install clang. I think it uses system libcxx, but I'm happy to ignore that for a bit. I couldn't compile libcxx with gcc-apple (although the website says it should be). Clang right now is useless as it doesn't search Prefix search paths, which I'm trying to fix now.
(In reply to Fabian Groffen from comment #22) > I'm able to compile/install clang. I think it uses system libcxx, but I'm > happy to ignore that for a bit. I couldn't compile libcxx with gcc-apple > (although the website says it should be). > > Clang right now is useless as it doesn't search Prefix search paths, which > I'm trying to fix now. As I said most of that stuff is already fixed. https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/llvm/llvm-3.4-r2.ebuild https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/binutils-apple/files/ld64-136-librarypath.patch i've been happily using clang for a lot of ebuilds ages now. https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/binutils-apple/binutils-apple-4.5-r1.ebuild
(In reply to Reza Jelveh from comment #23) > https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/ > binutils-apple/files/ld64-136-librarypath.patch This should be handled by the ld wrapper, so I don't quite understand why you need it. (In reply to Reza Jelveh from comment #23) > (In reply to Fabian Groffen from comment #22) > > Clang right now is useless as it doesn't search Prefix search paths, which > > I'm trying to fix now. > > As I said most of that stuff is already fixed. > > https://github.com/fishman/timebomb-gentoo-osx-overlay/blob/master/sys-devel/ > llvm/llvm-3.4-r2.ebuild I'm checking it out, thanks!
(In reply to Fabian Groffen from comment #24) > This should be handled by the ld wrapper, so I don't quite understand why > you need it. are you talking about the stuff that gcc would dump with dumpspec? a lot of builds use ld directly. leaving out will leave you with a bunch of random unexpected failures, because of some files mistakenly link to system libraries instead of prefix ones
ldwrapper is a gnu binutils thing. apples ld is a whole different beast(not even compatible for the most part)
I'm talking about the ld wrapper we wrote for Prefix specifically, to deal with adding the lib paths as needed. This is the program you call when you use 'ld' or 'CHOST-ld' in gentoo. For example: % $EPREFIX/usr/i686-apple-darwin11/binutils-bin/4.3/ld -v -L . @(#)PROGRAM:ld PROJECT:ld64-128.2 (Gentoo binutils-apple-4.3) Library search paths: . /usr/lib /usr/local/lib Framework search paths: /Library/Frameworks/ /System/Library/Frameworks/ ld: warning: -arch not specified ld: warning: -macosx_version_min not specificed, assuming 10.6 ld: no object files specified for inferred architecture i386 % ld -v -L . @(#)PROGRAM:ld PROJECT:ld64-128.2 (Gentoo binutils-apple-4.3) Library search paths: . /Users/fabian/Gentoo/usr/i686-apple-darwin11/lib /Users/fabian/Gentoo/usr/lib /Users/fabian/Gentoo/lib /usr/lib /usr/local/lib Framework search paths: /Library/Frameworks/ /System/Library/Frameworks/ ld: warning: -arch not specified ld: no object files specified for inferred architecture i386 Not using the wrapper is asking for trouble, since Gentoo installs multiple linkers at the same time and switches them using binutils-config. That means linker versions can be removed as well, making any compiler that directly points to that linker break down instantly.
up to you, not something I can influence. I don't know which packages it was. But I know I had problems with this. The wrapper as attached works all nice and cozy, but that didn't fix the build issues I had. It might actually fix some scons packages on osx too, but as I said it's been months. I don't remember which builds were fixed by that patch. Either way, I don't see what the downside of that patch might be. binutils-apple is only ever used on osx. And the double search path won't really make linking that much slower binutils> % ld -v -L . @(#)PROGRAM:ld PROJECT:ld64-136 (Gentoo binutils-apple-4.5-r1) configured to support archs: x86_64 i386 Library search paths: . /Users/timebomb/Gentoo/usr/x86_64-apple-darwin13/lib /Users/timebomb/Gentoo/usr/lib /Users/timebomb/Gentoo/lib /Users/timebomb/Gentoo/usr/lib /Users/timebomb/Gentoo/usr/local/lib /usr/lib /usr/local/lib Framework search paths: /Library/Frameworks/ /System/Library/Frameworks/ ld: warning: -arch not specified ld: no object files specified for inferred architecture x86_64 <binutils> % /Users/timebomb/Gentoo/usr/x86_64-apple-darwin13/binutils-bin/4.5/ld -v -L. @(#)PROGRAM:ld PROJECT:ld64-136 (Gentoo binutils-apple-4.5-r1) configured to support archs: x86_64 i386 Library search paths: . /Users/timebomb/Gentoo/usr/lib /Users/timebomb/Gentoo/usr/local/lib /usr/lib /usr/local/lib Framework search paths: /Library/Frameworks/ /System/Library/Frameworks/ ld: warning: -arch not specified ld: warning: -macosx_version_min not specified, assuming 10.6 ld: no object files specified for inferred architecture x86_64
Any progress on this? I think this could really push forward the macos prefix!
Big problem here: clang/llvm hardcodes all kinds of search paths which need to go, because it should look at the prefix, not at macports and homebrew. I started on a patch there, but time is a limiting factor (as well as an old Core2Dua laptop).
// By default on OS X 10.8 and later, we don't link with a crt1.o // file and the linker knows to use _main as the entry point. Since we couldn't get up with our linker, the one we have doesn't understand this, and hence when MACOSX_DEPLOYMENT_TARGET > 10.7, linking with clang fails because it cannot find _main. % clang -o test test.c Undefined symbols for architecture x86_64: "start", referenced from: -u command line option ld: symbol(s) not found for architecture x86_64
I'm not sure it's a *good* fix, but the ebuild I have here appears to solve this problem: https://github.com/srcshelton/gentoo-ebuilds/tree/master/sys-devel/llvm ... by retaining 'CmdArgs.push_back("-lcrt1.10.6.o");' in ToolChains.cpp even if the deployment target is 10.8 or newer. (https://raw.githubusercontent.com/srcshelton/gentoo-ebuilds/master/sys-devel/llvm/files/clang-3.5-fix_linker.patch) (In reply to Fabian Groffen from comment #31) > // By default on OS X 10.8 and later, we don't link with a crt1.o > // file and the linker knows to use _main as the entry point. > > Since we couldn't get up with our linker, the one we have doesn't understand > this, and hence when MACOSX_DEPLOYMENT_TARGET > 10.7, linking with clang > fails because it cannot find _main. > > % clang -o test test.c > Undefined symbols for architecture x86_64: > "start", referenced from: > -u command line option > ld: symbol(s) not found for architecture x86_64
a working linker has been in my github repo for quite a while now https://github.com/fishman/timebomb-gentoo-osx-overlay/tree/master/sys-devel/binutils-apple
In fact, I based my llvm ebuild on yours! I never could get binutils-apple-4.5(-r1)?.ebuild to build though... Now you've mentioned it, though, I'll give that a try again and let you know the exact error I'm hitting (if it's still a blocker - I've not retried this version in a while). (In reply to Reza Jelveh from comment #33) > a working linker has been in my github repo for quite a while now > > https://github.com/fishman/timebomb-gentoo-osx-overlay/tree/master/sys-devel/ > binutils-apple
Created attachment 394024 [details] current libcxx ebuilds The ebuilds and patches for binutils-apple 5.1 and 6.1 in bug https://bugs.gentoo.org/show_bug.cgi?id=473068 might be useful to you. Using those in a freshly bootstrapped prefix on OS X 10.6 32bit and 10.7 64bit, I was able to upgrade from binutils-apple-4.3 to 5.1 and 6.1 using gcc-apple. Then I could install llvm-3.4.2 (but not 3.5.0 as that requires a compiler that understands -std=c++11). With that I was able to install libcxx-apple with the ebuilds contained in the attached archive. Using the ebuild's libcxx USE flag, I was then able to recompile binutils-apple with clang and libc++ (clang with libstdc++ works just as well, though). LTO also seems to work OOB (clang nowadays always installs lto.h and libLTO so binutils-apple can just go ahead and use it). With this I could then install clang-3.5.0 using CC=clang CXX="clang++ -stdlib=libc++" on OS X 10.7 (on 10.6 llvm-3.5 won't compile just yet - looking into it). 10.6 is a good test case for libcxx, because it's not installed by default. Turns out, libcxx requires libcxxabi and libcxxabi needs libcxx's headers. So we get a libcxx-apple-headers -> libcxxabi-apple -> libcxx-apple dependency tree. If not done right, each of them silently revert to the system-installed versions. Putting binutils-apple and libcxx*-apple* in a mini-overlay and providing it to bootstrap-prefix.sh via PORTDIR_OVERLAY, I was able to bootstrap directly into a binutils-apple-6.1/gcc-apple-4.2.1 prefix on my OS X 10.10 box. Surprisingly no fiddling with MACOSX_DEPLOYMENT_TARGET was required for any of that to work (apart from what prefix intentionally does in profiles/prefix/darwin/macos/10.10/profile.bashrc).
Update: 3.4.2 and 3.5.0 tested successfully all the way down to 10.5 on Intel. Got no PPC box anymore. 32bit 64bit 10.5 fine N/A 10.6 fine fine 10.7 fine fine 10.8 fine fine 10.9 fine fine 10.10 N/A fine 3.5.1 tested on 10.5 32bit and 10.6 64bit. Seems fine as well. ebuilds needed some fettling as documented here: https://bugs.gentoo.org/show_bug.cgi?id=473068#c8 https://bugs.gentoo.org/show_bug.cgi?id=536716#c2 https://bugs.gentoo.org/show_bug.cgi?id=537342#c1
Created attachment 395260 [details] updated/cleaned libcxx ebuilds that work all the way back to 10.4 llvm/clang now work almost OOB on OS X. The remaining niggles and corner cases are filed in separate tickets: https://bugs.gentoo.org/show_bug.cgi?id=536716 https://bugs.gentoo.org/show_bug.cgi?id=537342 https://bugs.gentoo.org/show_bug.cgi?id=537684 https://bugs.gentoo.org/show_bug.cgi?id=537826 Apart from missing libc++ ebuilds, supporting clang IMO is now mostly a bootstrapping issue: The path gcc-apple -> llvm-3.4.2 -> llvm-3.5.1 is pretty long and *will* eventually break. I propose, we close this bug and I open a new one for inclusion of libc++ ebuilds and another for bootstrapping directly into clang. Agreed?
Agreed, I'm working on including your patches as we type.
The new bugs: https://bugs.gentoo.org/show_bug.cgi?id=538364 https://bugs.gentoo.org/show_bug.cgi?id=538366
Hi Fabian, (In reply to Fabian Groffen from comment #38) > Agreed, I'm working on including your patches as we type. Could you kindly tell me where the working ebuilds are commited to?
(In reply to Benda Xu from comment #40) > Hi Fabian, > > (In reply to Fabian Groffen from comment #38) > > Agreed, I'm working on including your patches as we type. > > Could you kindly tell me where the working ebuilds are commited to? they are stuck on libcxx, with gentoo maintainer I was thinking about adding Michaels ebuilds to the prefix tree for the time being, so we can continue without having to wait for maintainer.
