On Mac OS, some projects, such as LLVM, will set an install_name on their dylibs that is relative to the executable. (This way, the dylib and executable can be moved around and it will still work.) In order to do this, they use the special variable @executable_path, which expands at runtime to the directory containing the executable. Other Mach-O special variables include @loader_path, which expands to the directory containing the binary whose references are being resolved, and @rpath, which is expanded once for each entry in the rpath list. Unfortunately, when install_qa_check_macho is attempting to verify that the library names are correct, it rejects any names starting with a special variable. As a result, Portage will refuse to merge any ebuild that uses the special variables in its dylibs. A patch is attached that fixes the problem. Reproducible: Always Steps to Reproduce: 1. Add 'sys-devel/llvm **' to the package.keywords file (it's masked on Mac OS) 2. emerge llvm Actual Results: LLVM fails to merge because it uses the special variable '@executable_path' in its dylibs. Expected Results: LLVM should merge successfully (but llc should trap at runtime because of a bug in the LLVM build system). Portage 2.2.01.19120-prefix (prefix/darwin/macos/10.6/x86, gcc-4.2.1, unavailable, 10.8.0 i386) ================================================================= System Settings ================================================================= System uname: Darwin-10.8.0-i386-32bit Timestamp of tree: Sat, 03 Sep 2011 21:09:59 +0000 distcc 3.1-toolwhip.1 i386-apple-darwin10.0 [disabled] app-shells/bash: 4.2_p10 dev-lang/python: 2.7.2 dev-util/cmake: 2.8.4-r1 dev-util/pkgconfig: 0.25-r2 sys-devel/autoconf: 2.68 sys-devel/automake: 1.10.3, 1.11.1 sys-devel/gcc-config: 1.4.1-r00.2 sys-devel/libtool: 2.4-r01.1 sys-devel/make: 3.82 Repositories: gentoo_prefix Installed sets: ACCEPT_KEYWORDS="~x86-macos" ACCEPT_LICENSE="* -@EULA" CBUILD="i686-apple-darwin10" CFLAGS="-O2 -pipe -march=core2" CHOST="i686-apple-darwin10" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/portage /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-O2 -pipe -march=core2" DISTDIR="/Users/chip/Gentoo/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs collision-protect distlocks ebuild-locks fixlafiles fixpackages news nostrip parallel-fetch preserve-libs protect-owned sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-dead_strip_dylibs" MAKEOPTS="-j4" PKGDIR="/Users/chip/Gentoo/usr/portage/packages" PORTAGE_CONFIGROOT="/Users/chip/Gentoo/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/Users/chip/Gentoo/var/tmp" PORTDIR="/Users/chip/Gentoo/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.prefix.freens.org/gentoo-portage-prefix" USE="X aqua bash-completion berkdb bzip2 cairo coreaudio cracklib crypt curl cxx dbus doc exceptions expat extensions fontconfig gdbm gmp gnutls gpg gzip iconv icu ipv6 jbig jpeg libssh2 lzma lzo mmx mmxext mng modules mysql ncurses nls objc objc++ opengl pch perl png prefix python qt3support readline ruby sasl sql sqlite3 sse sse2 ssl subversion tcl threads tiff tk truetype unicode vim-syntax x86-macos xinerama xml xpm xv zlib" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="Darwin" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse" KERNEL="Darwin" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" SANE_BACKENDS="apple" USERLAND="GNU" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 285543 [details, diff] Patch to fix this bug
odd, I'm sure @executable_path is already ignored
(In reply to comment #2) > odd, I'm sure @executable_path is already ignored It is, but by that time the function has already produced the '.install_name_check_failed' file, which eventually causes Portage to bail (c.f. misc-functions.sh, line 940).
ehm... an install_name can be relative? install_name references can be @whatever now, but the install_name itself (id) not.
llvm has special code in the ebuild to kill the @executable_path crap, so I don't understand why this is an issue (it has ~ppc-macos keyword).
(In reply to comment #4) > ehm... an install_name can be relative? Yeah. Where do you think the install_name references come from? > > install_name references can be @whatever now, but the install_name itself (id) > not. Why not? It seems to work fine for LLVM. (By the way, Wine wants to do this too, but they fail at it because they use install_name_tool to change it after linking, but there's not enough room in the load command to change it.) Most of the LLVM devs work for Apple, so I think they know a thing or two about how Mach-O works :). (In reply to comment #5) > llvm has special code in the ebuild to kill the @executable_path crap, so I > don't understand why this is an issue (it has ~ppc-macos keyword). The svn (version 9999) ebuild doesn't work quite right because the version on the dylib is 3.0svn, not 9999. The code actually skips over it because libLLVM-9999.dylib doesn't exist.
(In reply to comment #6) > (In reply to comment #4) > > ehm... an install_name can be relative? > Yeah. Where do you think the install_name references come from? Set from a build-system that knows exactly what it's doing. You can't just make a relative install name and expect every application/library to obey that. > > install_name references can be @whatever now, but the install_name itself (id) > > not. > Why not? It seems to work fine for LLVM. (By the way, Wine wants to do this > too, but they fail at it because they use install_name_tool to change it after > linking, but there's not enough room in the load command to change it.) -max_pad_header_names or something, I thought we emitted that one by default. On libraries, it can never be the right thing, unless you're in a bundle, perhaps. Since Gentoo Prefix is not about bundles, but traditional UNIX filesystem layout, these refs are bad/broken by design. > Most of the LLVM devs work for Apple, so I think they know a thing or two about > how Mach-O works :). Yes, and they know how what they're creating is packaged. Which is absolutely not like Gentoo Prefix. I doubt they even know what it is. > (In reply to comment #5) > > llvm has special code in the ebuild to kill the @executable_path crap, so I > > don't understand why this is an issue (it has ~ppc-macos keyword). > The svn (version 9999) ebuild doesn't work quite right because the version on > the dylib is 3.0svn, not 9999. The code actually skips over it because > libLLVM-9999.dylib doesn't exist. Ohw. Well don't use those versions anyway. 2.9 is broken as hell as well, so you best stick to 2.8. LLVM is another typical example of an Apple-style project, which happens to hardcodedly only work on vanilla Apple systems in the first place.
(In reply to comment #7) > (In reply to comment #6) > > (In reply to comment #4) > > > ehm... an install_name can be relative? > > Yeah. Where do you think the install_name references come from? > > Set from a build-system that knows exactly what it's doing. You can't just > make a relative install name and expect every application/library to obey that. > > > > install_name references can be @whatever now, but the install_name itself (id) > > > not. > > Why not? It seems to work fine for LLVM. (By the way, Wine wants to do this > > too, but they fail at it because they use install_name_tool to change it after > > linking, but there's not enough room in the load command to change it.) > > -max_pad_header_names or something, I thought we emitted that one by default. It's -headermax_pad_install_names. And we do? Huh. I did not know that. I'm familiar with Wine outside of Gentoo because I work on it. (Go ahead, look at the Git history. Grep for my name.) And I know for a fact that they don't pass it on their own. That's why I brought that up. > > On libraries, it can never be the right thing, unless you're in a bundle, > perhaps. Since Gentoo Prefix is not about bundles, but traditional UNIX > filesystem layout, these refs are bad/broken by design. So the real problem is that LLVM is setting this at all. I'll see what I can do; I'm also an LLVM developer, you know :). A related problem is that LLVM doesn't obey the various fine-grained directory settings from the configure script. It assumes they're all relative to the prefix that it was given. > > > Most of the LLVM devs work for Apple, so I think they know a thing or two about > > how Mach-O works :). > > Yes, and they know how what they're creating is packaged. Which is absolutely > not like Gentoo Prefix. I doubt they even know what it is. I filed a bug on LLVM Bugzilla about Clang's behavior inside of a Gentoo prefix (in particular, how it barfs on Gentoo's ld64 version string). I have gotten absolutely no response from them about that. So I see what you mean. > > > > (In reply to comment #5) > > > llvm has special code in the ebuild to kill the @executable_path crap, so I > > > don't understand why this is an issue (it has ~ppc-macos keyword). > > The svn (version 9999) ebuild doesn't work quite right because the version on > > the dylib is 3.0svn, not 9999. The code actually skips over it because > > libLLVM-9999.dylib doesn't exist. > > Ohw. Well don't use those versions anyway. 2.9 is broken as hell as well, so > you best stick to 2.8. In what ways? Maybe we can get them fixed for 3.0. (By the way, I use trunk myself, because I'm an LLVM developer.)
> > -max_pad_header_names or something, I thought we emitted that one by default. > It's -headermax_pad_install_names. And we do? Huh. I did not know that. I'm > familiar with Wine outside of Gentoo because I work on it. (Go ahead, look at > the Git history. Grep for my name.) And I know for a fact that they don't pass > it on their own. That's why I brought that up. We as in Gentoo Prefix. However, I lied. We don't. http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-alt/trunk/toolchain-prefix-wrapper/ld/darwinplugin.c?revision=1638&view=markup > > On libraries, it can never be the right thing, unless you're in a bundle, > > perhaps. Since Gentoo Prefix is not about bundles, but traditional UNIX > > filesystem layout, these refs are bad/broken by design. > So the real problem is that LLVM is setting this at all. I'll see what I can > do; I'm also an LLVM developer, you know :). > > A related problem is that LLVM doesn't obey the various fine-grained directory > settings from the configure script. It assumes they're all relative to the > prefix that it was given. I made a remark about the complex build-system in the ebuild where I install_name_tool the libs and bins. I noticed it's complex, and hence chose to work around, rather than fix it ;) Still, Gentoo Prefix is not the regular target for any LLVM/Apple developer, I'd suppose. > > > Most of the LLVM devs work for Apple, so I think they know a thing or two about > > > how Mach-O works :). > > > > Yes, and they know how what they're creating is packaged. Which is absolutely > > not like Gentoo Prefix. I doubt they even know what it is. > I filed a bug on LLVM Bugzilla about Clang's behavior inside of a Gentoo prefix > (in particular, how it barfs on Gentoo's ld64 version string). I have gotten > absolutely no response from them about that. So I see what you mean. Ohw, do you have a bug-ref? It takes a lot more to get llvm/clang to "behave" as there's even a file that hardcodes what to do with CHOSTs and include paths (it takes the Xcode SDKs) and more. I appreciate the efforts spent on it! > > Ohw. Well don't use those versions anyway. 2.9 is broken as hell as well, so > > you best stick to 2.8. > In what ways? Maybe we can get them fixed for 3.0. (By the way, I use trunk > myself, because I'm an LLVM developer.) I see. Well, on Linux linking is totally fubared because gcc is no longer used to perform the link. LLVM devs assumed that gcc just calls ld without any extra arguments or something, which of course isn't true. Chances are big they have solved this in the meantime.
(In reply to comment #9) > I made a remark about the complex build-system in the ebuild where I > install_name_tool the libs and bins. I noticed it's complex, and hence chose > to work around, rather than fix it ;) > Still, Gentoo Prefix is not the regular target for any LLVM/Apple developer, > I'd suppose. Yeah. I don't build LLVM inside of my Prefix because linking is horribly broken (due to the bug I linked below). > > Ohw, do you have a bug-ref? http://llvm.org/PR8339 > It takes a lot more to get llvm/clang to "behave" as there's even a file that > hardcodes what to do with CHOSTs and include paths (it takes the Xcode SDKs) > and more. I appreciate the efforts spent on it! [...] > > > > Ohw. Well don't use those versions anyway. 2.9 is broken as hell as well, so > > > you best stick to 2.8. > > In what ways? Maybe we can get them fixed for 3.0. (By the way, I use trunk > > myself, because I'm an LLVM developer.) > > I see. Well, on Linux linking is totally fubared because gcc is no longer used > to perform the link. LLVM devs assumed that gcc just calls ld without any > extra arguments or something, which of course isn't true. Chances are big they > have solved this in the meantime. Yeah. As you've seen, the Clang driver is a huge mess. It's just hack after hack after hack to make the whole thing work out of the box on every Linux distro under the sun--at least, every one they know about. And it gets worse all the time. They don't even know what switches they need to pass to ld(1) on Gentoo. Hell, I'm not even sure they know that Gentoo exists! (It's not listed as a 'LinuxDistro' in that named enum.) In the short term, I can add Gentoo to this mess. But long term, we really need something better. This can't go on. They all know this, too, but nobody's willing to do the work to fix this (c.f. http://clang.llvm.org/UniversalDriver.html).
