Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 564814 - OS X El Capitan (10.11) support
Summary: OS X El Capitan (10.11) support
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Linux
: Normal normal with 3 votes (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
Depends on: 572790 573176
Blocks:
  Show dependency tree
 
Reported: 2015-11-03 16:54 UTC by yegle
Modified: 2016-04-10 06:24 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
stage1.log (stage1.log.bz2,101.62 KB, application/x-bzip2)
2016-02-17 04:04 UTC, *
Details
stage2.log (stage2.log.bz2,239.57 KB, application/x-bzip2)
2016-02-17 04:04 UTC, *
Details
stage3.log (stage3.log.bz2,948.68 KB, application/x-bzip2)
2016-02-17 04:05 UTC, *
Details
stage3.log failing on 2016-03-30 (stage3.log.bz2,304.07 KB, application/x-bzip2)
2016-03-30 16:05 UTC, *
Details

Note You need to log in before you can comment on or make changes to this bug.
Description yegle 2015-11-03 16:54:53 UTC
Right now at least the bootstrap script doesn't work. Newer versions of clang are still masked.

Reproducible: Always
Comment 1 Fabian Groffen gentoo-dev 2015-11-07 16:18:02 UTC
ok, is this using latest-tree?
Comment 2 Justin Lecher (RETIRED) gentoo-dev 2015-11-07 16:43:10 UTC
I tried a couple of days ago, downloading the latest bootstrapping script. Didn't work.
Comment 3 * 2015-11-08 04:02:18 UTC
When I tried with LATEST_TREE_YES=1 it failed during stage2 when building llvm-3.4.2:
ld: library not found for -lc++
Comment 4 yegle 2015-12-11 06:26:18 UTC
Just tried the latest-tree as of Dec 10, 2015. Failed in installing llvm-3.4.2 and same error as comment 3. Also managed to install llvm-3.5.0 by modify the bootstrap script without any luck.
Comment 5 Qiming Wang 2015-12-25 02:34:28 UTC
I did manage to bootstrap on el capitan with a lot of hacks.
But I have to use system clang to compile my clang even now.
Comment 6 * 2016-01-17 16:13:11 UTC
(In reply to Qiming Wang from comment #5)
> I did manage to bootstrap on el capitan with a lot of hacks.
> But I have to use system clang to compile my clang even now.

Did you happen to record the hacks, or are they easy to describe here?
Comment 7 Fabian Groffen gentoo-dev 2016-01-22 08:17:31 UTC
the profile should be in the snapshot that's used by default now
Comment 8 * 2016-01-23 06:22:55 UTC
Unfortunately it still fails building llvm-3.4.2 as described in comment 3. I'm using the latest bootstrap-prefix.sh (PV="20160121").
Comment 9 * 2016-01-23 21:47:47 UTC
Okay, the llvm issue was resolved thanks to a hint in this thread:
https://forums.gentoo.org/viewtopic-t-1034702.html

Then some issues with xattr were resolved with the latest bootstrap-prefix.sh (now PV="20160122").

However, the bootstrap still fails during stage3 while installing python-3.5.1. I've created a new bug here:
https://bugs.gentoo.org/show_bug.cgi?id=572726
Comment 10 Fabian Groffen gentoo-dev 2016-01-24 18:14:49 UTC
bootstrap currently goes all the way until the final emerge -e system.  There it fails on collisions for ca-certificates, which I don't fully comprehend yet.
Comment 11 Fabian Groffen gentoo-dev 2016-01-24 19:08:33 UTC
need to think of a workaround for bug #572790 so tomorrow a bootstrap can stand a chance to succeed
Comment 12 Fabian Groffen gentoo-dev 2016-01-26 08:58:05 UTC
Workaround is in place, yesterday got as far as almost the end of emerge -e system stage.  Retrying now (10.11).
Comment 13 Fabian Groffen gentoo-dev 2016-01-26 14:22:33 UTC
hmm, odd, it got stuck on the permissions thing again today.
Comment 14 * 2016-01-27 07:51:22 UTC
It's pretty close. I didn't encounter the ca-certificates failure, but it failed here:
>>> Emerging (99 of 107) sys-devel/llvm-3.5.2::gentoo_prefix

I'll either try masking that or file a report.
Comment 15 Fabian Groffen gentoo-dev 2016-01-27 09:32:03 UTC
Masking won't work so well.  Can you try to find the error (and if it is a segmentation fault maybe) in the logs?
Comment 16 * 2016-01-28 06:18:54 UTC
Yes, appears to be a segfault. I went ahead and filed Bug 573176. Also ran into Bug 573038.
Comment 17 Askar Bektassov 2016-02-01 13:04:27 UTC
Great, the problem seems being scaled down only to llvm. Unsure if it is the same bug (https://llvm.org/bugs/show_bug.cgi?id=25943), but someone upstream already noticed that llvm fails compiling llvm on darwin15. I also confirm that in my case, only llvm and glib are missing to complete bootstrap.
Comment 18 Askar Bektassov 2016-02-02 22:53:07 UTC
Bootstrap still fails during compilation of llvm at stage3. Was able to work around by compiling llvm-3.7.1 using host compiler.

CC=/usr/bin/clang CXX=/usr/bin/clang++ emerge llvm

Afterwards, went back to the regular bootstrap process and re-emerged the system (this time, llvm compiles llvm successfully).

bootstrap-script.sh $EPREFIX stage3
emerge system -e
Comment 19 Fabian Groffen gentoo-dev 2016-02-03 09:13:08 UTC
yeah, there is an annoying problem here, 3.5.2 is unable to compile itself it seems when compiled from 3.4.2 (like we do).

3.4.2 is the one version convenient for us since its deps are minimal (which is necessary during bootstrap).

3.5.2 is pulled in because it provides proper aliases: ${CHOST}-clang, 3.4.2 doesn't do this.  For this reason I couldn't easily try if 3.4.2 can actually compile 3.7.1, which should be the result of a compile now.

3.4.2 understands the system headers from newer OSX worse than 3.5.2, 3.7.1 seems to understand them sufficiently to compile everything.

I'd like to skip 3.6 at this point since it's not ported/tested at all.

Any ideas on things to try (other than 3.4.2 -> 3.7.1)?
Comment 20 Askar Bektassov 2016-02-03 20:36:01 UTC
Any other alternative may require downgrading the compiler used during stage1 and/or stage2 (e.g. llvm 3.3 instead of 3.4.2, and/or llvm 3.5.0 instead of 3.5.2), which has the risk of becoming a painful and worthless solution.
Comment 21 Fabian Groffen gentoo-dev 2016-02-04 11:22:25 UTC
Ok, llvm-3.5.0 has the same problem (segfaults compiling 3.5.2)

llvm perhaps is better dealt with in bug #573176
Comment 22 * 2016-02-13 23:11:18 UTC
Bug #573176 is fixed, but now it's segfaulting with llvm-3.7.1. I went ahead and filed bug #574656, though it may be related.

Only llvm-3.7.1 and clang-3.7.1 prevent the bootstrap from completing. I'm actually able to get a working prefix using comment #18, which is *fantastic*.

Thanks for all your work on this. Lots of tricky issues this time.
Comment 23 Fabian Groffen gentoo-dev 2016-02-14 08:36:33 UTC
That's cool.  Same issue for me yesterday.  It puzzled me.  Today continuing on that ... :)
Comment 24 Stuart Shelton 2016-02-14 13:08:12 UTC
Using system compilers like this allows me to compile llvm (3.7.1) successfully - but when I try to use *this* llvm to rebuild itself, I get errors along the lines of:

building for iOS simulator,
    but linking in object file built for OSX, for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

... which the post at [1] suggests can be fixed by adding '-mmacosx-version-min=XX ' (should this be set from the portage prefix/macos profile?)

Now, however, I'm getting:

ld: library not found for -lSystem
x86_64-apple-darwin15-clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

... and haven't yet figured out what I need to do to fix this.  Adding 'LDFLAGS="-L/usr/lib"' (which I'd have guessed would be implicit) doesn't appear to have helped...


[1] https://karp.id.au/post/xcode_7_linker_rules/


(In reply to Askar Bektassov from comment #18)
> Bootstrap still fails during compilation of llvm at stage3. Was able to work
> around by compiling llvm-3.7.1 using host compiler.
> 
> CC=/usr/bin/clang CXX=/usr/bin/clang++ emerge llvm
> 
> Afterwards, went back to the regular bootstrap process and re-emerged the
> system (this time, llvm compiles llvm successfully).
> 
> bootstrap-script.sh $EPREFIX stage3
> emerge system -e
Comment 25 Fabian Groffen gentoo-dev 2016-02-14 16:55:35 UTC
please try the latest bootstrap-prefix.sh, I managed to get into the emerge -e phase (last one), thus installing llvm-3.7.1
Comment 26 Justin Lecher (RETIRED) gentoo-dev 2016-02-15 10:40:46 UTC
(In reply to Fabian Groffen from comment #25)
> please try the latest bootstrap-prefix.sh, I managed to get into the emerge
> -e phase (last one), thus installing llvm-3.7.1

Still happen here. When doing the stage2 build of llvm-3.4.2.
Comment 27 Justin Lecher (RETIRED) gentoo-dev 2016-02-15 10:41:25 UTC
Could it be an issue that I have homebrews gcc installed?
Comment 28 Fabian Groffen gentoo-dev 2016-02-15 10:42:21 UTC
if it's being used somehow, then yes.
Comment 29 Justin Lecher (RETIRED) gentoo-dev 2016-02-15 10:49:59 UTC
(In reply to Fabian Groffen from comment #28)
> if it's being used somehow, then yes.

The logs show quite a number of usages of /usr/local/{include,lib}. I just removed gcc and see if I can build llvm now.
Comment 30 * 2016-02-15 17:22:47 UTC
(In reply to Stuart Shelton from comment #24)
> building for iOS simulator,
>     but linking in object file built for OSX, for architecture x86_64
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
> 

There are a lot of hits for this on Google. My guess would be it's some kind of issue with Xcode. I assume you're using the latest version (or tried reinstalling) and switched over to use the command line tools?
xcode-select -s /Library/Developer/CommandLineTools
Comment 31 * 2016-02-15 17:23:45 UTC
The latest bootstrap works for me.
Comment 32 Guilherme Amadio gentoo-dev 2016-02-15 17:45:34 UTC
(In reply to * from comment #31)
> The latest bootstrap works for me.

Can you please post the output of the two commands below, and the exact version of OS X you have (e.g. 10.11.3)?

xcode-select -p
xcode-select -v

Thanks!
Comment 33 * 2016-02-15 20:28:33 UTC
$ xcode-select -p
/Library/Developer/CommandLineTools
$ xcode-select -v
xcode-select version 2343.
$ sw_vers -productVersion
10.11.3
Comment 34 Askar Bektassov 2016-02-15 20:54:14 UTC
(In reply to Stuart Shelton from comment #24)
> Using system compilers like this allows me to compile llvm (3.7.1)
> successfully - but when I try to use *this* llvm to rebuild itself, I get
> errors along the lines of:
> 
> building for iOS simulator,
>     but linking in object file built for OSX, for architecture x86_64
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
> 
> ... which the post at [1] suggests can be fixed by adding
> '-mmacosx-version-min=XX ' (should this be set from the portage prefix/macos
> profile?)
> 
> Now, however, I'm getting:
> 
> ld: library not found for -lSystem
> x86_64-apple-darwin15-clang-3.7: error: linker command failed with exit code
> 1 (use -v to see invocation)
> ninja: build stopped: subcommand failed.
> 
> ... and haven't yet figured out what I need to do to fix this.  Adding
> 'LDFLAGS="-L/usr/lib"' (which I'd have guessed would be implicit) doesn't
> appear to have helped...
> 
> 
> [1] https://karp.id.au/post/xcode_7_linker_rules/

Stuart, at certain point in time, I had a problem similar to yours, which was fully resolved by the new bootstrap-prefix.sh.

Please check if you have more than one directory:

la $EPREFIX/usr/x86_64-apple-darwin1*

If you have both, x86_64-apple-darwin14 AND x86_64-apple-darwin15, then the linker may not work unless you create a symlink to x86_64-apple-darwin15 within the x86_64-apple-darwin14 directory, i.e.:

cd $EPREFIX/usr/x86_64-apple-darwin14
ln -s ../x86_64-apple-darwin15

At which point the linker worked, at least in my case.
Comment 35 Askar Bektassov 2016-02-15 20:55:54 UTC
(In reply to Askar Bektassov from comment #34)
> (In reply to Stuart Shelton from comment #24)

> la $EPREFIX/usr/x86_64-apple-darwin1*

Sorry, ls instead of la

ls $EPREFIX/usr/x86_64-apple-darwin1*
Comment 36 Fabian Groffen gentoo-dev 2016-02-16 08:08:06 UTC
just for the record:

[Priscilla:var/tmp/bootstrap-20160215] fgroffen% xcode-select -p
/Library/Developer/CommandLineTools
[Priscilla:var/tmp/bootstrap-20160215] fgroffen% xcode-select -v
xcode-select version 2339.
[Priscilla:var/tmp/bootstrap-20160215] fgroffen% sw_vers 
ProductName:    Mac OS X
ProductVersion: 10.10.5
BuildVersion:   14F1605

this bootstrap succeed (NOT El Capitan)

comment #31 mentions success also.  My bootstrap on El Capitan failed in the emerge -e system phase on a collision for openssl certs.  That means the llvm shizzle all succeed (I verified it did).

So, it definitely is related to xcode versions and stuff.

I used this version of clang on Yosemite:

Apple LLVM version 7.0.2 (clang-700.1.81)

(I don't know what version that is in our world, 3.7-ish?)
Comment 37 Askar Bektassov 2016-02-16 20:02:10 UTC
Yesterday night bootstrap was still failing during llvm compilation (stage3) on my system.

$ xcode-select -p
/Library/Developer/CommandLineTools

$ xcode-select -v
xcode-select version 2343.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.11.3
BuildVersion:	15D21
Comment 38 * 2016-02-17 04:03:48 UTC
You're sure you're using the latest bootstrap?
$ shasum bootstrap-prefix.sh
8b60fdd6b7640bc095c8edf49df61faa782580b9  bootstrap-prefix.sh

I'll upload my stage logs if you want to diff them. If they don't belong here, someone should feel free to delete.
Comment 39 * 2016-02-17 04:04:24 UTC
Created attachment 425688 [details]
stage1.log
Comment 40 * 2016-02-17 04:04:48 UTC
Created attachment 425690 [details]
stage2.log
Comment 41 * 2016-02-17 04:05:15 UTC
Created attachment 425692 [details]
stage3.log
Comment 42 Victor Orozco 2016-02-17 15:09:36 UTC
Yesterday, I tested the last bootstrap script.

Stage 1, 2 and 3 succeeded, however emerge -e system failed while Installing app-misc/ca-certificates-20160104.3.21 due file collisions.

$ xcode-select -p
/Library/Developer/CommandLineTools

$ xcode-select -v
xcode-select version 2343.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.11.3
BuildVersion:	15D21

$ shasum bootstrap-prefix.sh
8b60fdd6b7640bc095c8edf49df61faa782580b9  bootstrap-prefix.sh
Comment 43 Fabian Groffen gentoo-dev 2016-02-17 15:17:19 UTC
(In reply to Victor Orozco from comment #42)
> Yesterday, I tested the last bootstrap script.
> 
> Stage 1, 2 and 3 succeeded, however emerge -e system failed while Installing
> app-misc/ca-certificates-20160104.3.21 due file collisions.

Yes, that should be bug #572790

You can basically recover from this manually if you like.  Your system is already installed.

> $ xcode-select -p
> /Library/Developer/CommandLineTools
> 
> $ xcode-select -v
> xcode-select version 2343.
> 
> $ sw_vers
> ProductName:	Mac OS X
> ProductVersion:	10.11.3
> BuildVersion:	15D21
> 
> $ shasum bootstrap-prefix.sh
> 8b60fdd6b7640bc095c8edf49df61faa782580b9  bootstrap-prefix.sh
Comment 44 Askar Bektassov 2016-02-18 19:45:01 UTC
My mistake, the bootstrap was not the last version. Now I can successfully bootstrap, provided that the ca-certificates is fixed by disabling colision-protect FEATURE.

Below are the three commands to complete the entire process.

$ bootstrap-prefix.sh
$ FEATURES="-collision-protect" ebuild Gentoo/usr/portage/app-misc/ca-certificates/ca-certificates-20160104.3.21.ebuild install qmerge clean
$ emerge -r

My system is as follows

askarbektassov@Askars-iMac ~ $ xcode-select -p
/Library/Developer/CommandLineTools
askarbektassov@Askars-iMac ~ $ xcode-select -v
xcode-select version 2343.
askarbektassov@Askars-iMac ~ $ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.11.3
BuildVersion:	15D21
askarbektassov@Askars-iMac ~ $ shasum Downloads/bootstrap-prefix.sh
8b60fdd6b7640bc095c8edf49df61faa782580b9  Downloads/bootstrap-prefix.sh
Comment 45 Stuart Shelton 2016-02-18 23:18:06 UTC
I'd forgotten to switch the command-line tools - this fixed things for me!


(In reply to * from comment #30)
> (In reply to Stuart Shelton from comment #24)
> > building for iOS simulator,
> >     but linking in object file built for OSX, for architecture x86_64
> > clang: error: linker command failed with exit code 1 (use -v to see
> > invocation)
> > 
> 
> There are a lot of hits for this on Google. My guess would be it's some kind
> of issue with Xcode. I assume you're using the latest version (or tried
> reinstalling) and switched over to use the command line tools?
> xcode-select -s /Library/Developer/CommandLineTools
Comment 46 Stuart Shelton 2016-02-18 23:34:19 UTC
(... although building with the 'lldb' USE-flag enabled still fails.)
Comment 47 * 2016-02-19 05:42:56 UTC
Not sure I understand how you're modifying the bootstrap to use the "lldb" USE flag?
Comment 48 Stuart Shelton 2016-03-07 19:04:49 UTC
(In reply to * from comment #47)
> Not sure I understand how you're modifying the bootstrap to use the "lldb"
> USE flag?

I'm just trying to compile LLVM on an existing install, rather than re-bootstrap.
Comment 49 Elizabeth Myers 2016-03-24 19:49:10 UTC
Hi,

After adjusting xcode correctly, I had an issue with python-exec being unable to be fetched. I had to manually resync the tree, but that resulted in problems coming up due to newer pythons requiring the ssl flag to proceed. After I got past that, I got this problem:

setlocale: unsupported locale setting 
Failed to validate a sane '/dev'. 
bash process substitution doesn't work; this may be an indication of a broken '/dev/fd'. 

Hmmmm, I was already afraid of this to happen.  Running 
  /Users/elizabeth/Gentoo/bin/bash ./bootstrap-prefix.sh "/Users/elizabeth/Gentoo" stage3 
somewhere failed :(  Details might be found in the build log: 
  (no build logs found?!?)
Comment 50 * 2016-03-27 15:27:38 UTC
No idea but personally I would start over. Move /Users/elizabeth/Gentoo out of the way, make sure you have the latest bootstrap-prefix.sh, and try again. If the first problem you encounter is reproducible, then report back. If the problem is due to the bootstrap failing to build a certain package, then consider filing a bug for that package and reference this bug.
Comment 51 * 2016-03-30 16:04:42 UTC
I can reproduce the problem with python requiring USE=ssl doing a fresh bootstrap. I think it's failing on line 1280 while emerging portage and dependencies. I'm not sure if this is related to bug 572790.
Comment 52 * 2016-03-30 16:05:51 UTC
Created attachment 429306 [details]
stage3.log failing on 2016-03-30
Comment 53 Fabian Groffen gentoo-dev 2016-04-06 15:29:45 UTC
This seems fixed after my latest round of adjustments.  I could bootstrap on El Capitan.
Comment 54 * 2016-04-07 06:06:56 UTC
For the record, it also works for me on 10.11.4.
Comment 55 Tim Schafer 2016-04-08 15:56:04 UTC
fails at llvm-3.4.2 when:
xcode-select -p
/Applications/Xcode.app/Contents/Developer

If Xcode.app is installed need to:
sudo xcode-select -s /Library/Developer/CommandLineTools

See bug 562800
Comment 56 Fabian Groffen gentoo-dev 2016-04-10 06:24:25 UTC
I've added that bit to the bootstrap script to check.