Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 924839

Summary: [clang darwin-prefix] sys-libs/zlib-1.3.1-r1: fails to build with unknown attribute kind error
Product: Gentoo/Alt Reporter: Chris Pritchard <chris>
Component: Prefix SupportAssignee: Gentoo Prefix <prefix>
Status: RESOLVED FIXED    
Severity: normal CC: alexey+gentoo, chris
Priority: Normal Keywords: PullRequest
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://github.com/gentoo/gentoo/pull/35613
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 758167    
Attachments: build.log
zlib-test-ar-darwin.patch
bootstrap-prefix.sh

Description Chris Pritchard 2024-02-17 19:45:22 UTC
zlib fails to build, this is due to the configure script overriding the AR environment variable and setting it to libtool due to an underlying OS X issue.



Reproducible: Always

Steps to Reproduce:
Try to build zlib in the emerge -e @system stage of the darwin prefix build
Actual Results:  
see log file

Expected Results:  
package builds
Comment 1 Chris Pritchard 2024-02-17 19:46:36 UTC
Created attachment 885255 [details]
build.log

first error:

/usr/bin/libtool -o libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o 
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: Unknown attribute kind (86) (Producer: 'LLVM17.0.6+libcxx' Reader: 'LLVM APPLE_1_1500.1.0.2.5_0')
Comment 2 Chris Pritchard 2024-02-17 19:47:55 UTC
Created attachment 885256 [details, diff]
zlib-test-ar-darwin.patch

Patch to the configure file which fixes the error
Comment 3 Fabian Groffen gentoo-dev 2024-02-17 20:39:26 UTC
is this during bootstrap?
Comment 4 Chris Pritchard 2024-02-17 20:41:14 UTC
After bootstrap stages 1, 2, and 3, but during the emerge -e @system stage.

I used a slightly modified version of the bootstrap script posted by Alexey (removed the lines copying from their patched repository).
Comment 5 Chris Pritchard 2024-02-17 20:42:03 UTC
Created attachment 885258 [details]
bootstrap-prefix.sh

This is the bootstrap script I used
Comment 6 Fabian Groffen gentoo-dev 2024-02-17 20:43:42 UTC
Feels like the clang profile should pull in native-cctools + binutils-config then to get <chost>-libtool available in the prefix.
Comment 7 Fabian Groffen gentoo-dev 2024-02-17 20:44:58 UTC
does ld64.lld provide libtool?  If so, binutils-config could probably wrap it.
Comment 8 Alexey 2024-02-17 22:45:58 UTC
Near the end of stage2 I'm setting up this symlink:

ln -s ${ROOT}/tmp/usr/lib/llvm/*/bin/llvm-libtool-darwin ${ROOT}/usr/local/bin/libtool

This makes `libtool` to be available for stage3. I don't know why I don't encounter this error. Might be related to usage of LATEST_TREE vs snapshot?
Comment 9 Alexey 2024-02-18 00:03:17 UTC
I don't see how binutils-config can work because it needs to depend on which LLVM version is installed, unless it creates a generic CHOST-libtool which uses PATH to find the correct llvm-libtool-darwin
Comment 10 Fabian Groffen gentoo-dev 2024-02-18 08:34:35 UTC
binutils-config manages the symlinks in EPREFIX/usr/bin for things like ld, ar, libtool, and their CHOST-variants.

Now, I understand you don't use that, because you export LD to be what you need, but that means a part of the system isn't any more like we'd expect it to be.  Most interesting/importantly, the ld-wrapper is no longer in effect that is used to add things like -search_paths_first and -LEPREFIX/usr/lib to each linker invocation.

Now I'm sure llvm will have a tight integration with the linker, perhaps making the wrapper not necessary.  However, perhaps just for maintaining the same set of binaries in the same locations, we should install binutils-config to make at least the symlinks for the above mentioned tools based on llvm/lld.ld64.
Comment 11 Alexey 2024-02-18 08:55:17 UTC
> Most interesting/importantly, the ld-wrapper is no longer in effect that is used to add things like -search_paths_first and -LEPREFIX/usr/lib to each linker invocation.

Here it is done using sys-devel/clang-common:

https://github.com/gentoo/gentoo/blob/05d45de3a13578648028ece5658bc3a2c6126486/sys-devel/clang-common/clang-common-17.0.6-r1.ebuild#L261

> Now I'm sure llvm will have a tight integration with the linker, perhaps making the wrapper not necessary.  However, perhaps just for maintaining the same set of binaries in the same locations, we should install binutils-config to make at least the symlinks for the above mentioned tools based on llvm/lld.ld64.

That might be true. But it won't help in this specific case of zlib anyway, because llvm-libtool-darwin -V doesn't have 'Apple' in its output, so the configure check is going to use /usr/bin/libtool anyway, so it has to be patched.

I've just checked why zlib doesn't fail for me, and it uses 'libtool', but somehow /usr/bin ends up in PATH in ${T}/environment, so it ends up using /usr/bin/libtool anyway for such programs which want to see 'Apple' in the version. I don't know why my /usr/bin/libtool happened to not fail though. Its version is 1009.3
Comment 12 Chris Pritchard 2024-02-18 09:50:09 UTC
Figured out why Alexey's doens't fail. It seems that apple's libtool is OK with non-LTO'd output from llvm 17.0.6 (apple's clang is version 15), but with LTO apple's libtool throws an error. My apologies, it slipped my mind that I'd copied that cflag across!

Either way, the prefixed packages should ideally be using the llvm toolchain, including libtool and/or ar, instead of the xcode ones (which really should only be used during the bootstrap). The hard coded check in the configure file, without any way to override it without a patch, isn't what we want as we're not using apple's toolchain to build the package, and the issue is with apple's linker (see https://patchwork.kernel.org/project/qemu-devel/patch/1462236478-61645-2-git-send-email-chrisfriedt@gmail.com/ for some background)
Comment 13 Alexey 2024-03-03 21:27:36 UTC
I can reproduce this, but only AFTER bootstrap, if I change in the successfully installed prefix's make.conf to have CFLAGS="$CFLAGS -flto=thin"

I don't know how to make the bootstrap script itself pick up that flag. But for bootstrap it shouldn't use LTO anyway.
Comment 14 Larry the Git Cow gentoo-dev 2024-03-04 02:24:30 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=743b110bffe25c0501ae5473ad8ad68253a14f29

commit 743b110bffe25c0501ae5473ad8ad68253a14f29
Author:     Alexey Sokolov <alexey+gentoo@asokolov.org>
AuthorDate: 2024-03-03 20:35:02 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-03-04 02:21:09 +0000

    sys-libs/zlib: respect AR on darwin
    
    Closes: https://bugs.gentoo.org/924839
    Signed-off-by: Alexey Sokolov <alexey+gentoo@asokolov.org>
    Signed-off-by: Sam James <sam@gentoo.org>

 ...-1.3.1-configure-fix-AR-libtool-on-darwin.patch | 22 ++++++++++++++++++++++
 sys-libs/zlib/zlib-1.3.1-r1.ebuild                 |  3 +++
 2 files changed, 25 insertions(+)

Additionally, it has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=afcd5aa47776265486ddaa6ca2adcd8480b32574

commit afcd5aa47776265486ddaa6ca2adcd8480b32574
Author:     Alexey Sokolov <alexey+gentoo@asokolov.org>
AuthorDate: 2024-03-03 21:20:54 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-03-04 02:21:09 +0000

    sys-devel/llvm: use correct libtool on macos prefix
    
    Bug: https://bugs.gentoo.org/924839
    Signed-off-by: Alexey Sokolov <alexey+gentoo@asokolov.org>
    Closes: https://github.com/gentoo/gentoo/pull/35613
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/llvm/llvm-17.0.6.ebuild             | 8 +++++---
 sys-devel/llvm/llvm-18.1.0_rc3.ebuild         | 8 +++++---
 sys-devel/llvm/llvm-18.1.0_rc4.ebuild         | 8 +++++---
 sys-devel/llvm/llvm-19.0.0.9999.ebuild        | 8 +++++---
 sys-devel/llvm/llvm-19.0.0_pre20240224.ebuild | 8 +++++---
 sys-devel/llvm/llvm-19.0.0_pre20240302.ebuild | 8 +++++---
 6 files changed, 30 insertions(+), 18 deletions(-)