Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 456322 - sys-devel/llvm: conversion to use cmake
Summary: sys-devel/llvm: conversion to use cmake
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Bernard Cafarelli
URL:
Whiteboard:
Keywords:
: 478090 (view as bug list)
Depends on: 461024 461270
Blocks: 425844
  Show dependency tree
 
Reported: 2013-02-09 15:17 UTC by Michał Górny
Modified: 2015-06-09 16:55 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
LLVM test failures with new ebuild (llvm-failures.log,3.62 KB, text/plain)
2013-02-09 15:59 UTC, Michał Górny
Details
LLVM ebuild, rev1 (llvm-3.2-r1.ebuild,4.11 KB, text/plain)
2013-02-09 16:01 UTC, Michał Górny
Details
mesa build failure log (20130209-160408.log,340.96 KB, text/plain)
2013-02-09 17:31 UTC, Michał Górny
Details
LLVM ebuild, rev2 (llvm-3.2-r1.ebuild,4.53 KB, text/plain)
2013-02-09 21:40 UTC, Michał Górny
Details
LLVM ebuild, rev3 (llvm-3.2-r1.ebuild,4.66 KB, text/plain)
2013-02-09 23:40 UTC, Michał Górny
Details
clang ebuild, rev1 (clang-3.2-r1.ebuild,4.67 KB, text/plain)
2013-02-10 11:18 UTC, Michał Górny
Details
add soversion cmake patch (0001-cmake-Add-soversion.patch,1.23 KB, patch)
2013-03-09 20:40 UTC, Alexey Shvetsov
Details | Diff
LLVM ebuild, rev4 (llvm-3.2-r1.ebuild,5.22 KB, text/plain)
2013-03-09 22:28 UTC, Michał Górny
Details
clang ebuild, rev2 (clang-3.2-r1.ebuild,4.30 KB, text/plain)
2013-03-09 22:30 UTC, Michał Górny
Details
LLVM, rev5 (llvm-3.2-r1.ebuild,5.57 KB, text/plain)
2013-03-10 19:27 UTC, Michał Górny
Details
clang, rev3 (clang-3.2-r1.ebuild,4.30 KB, text/plain)
2013-03-10 19:32 UTC, Michał Górny
Details
Build log for tests (test.log,456.60 KB, text/plain)
2013-03-10 20:07 UTC, Michał Górny
Details
LLVM second round, r0 (llvm-9999-r1.ebuild,6.09 KB, text/plain)
2013-12-28 12:04 UTC, Michał Górny
Details
second round, r1 (llvm-9999-r1.ebuild,8.60 KB, text/plain)
2015-06-06 10:12 UTC, Michał Górny
Details
second round, r2 (llvm-9999-r1.ebuild,10.94 KB, text/plain)
2015-06-07 21:55 UTC, Michał Górny
Details
second round, r3 (llvm-9999-r1.ebuild,10.69 KB, text/plain)
2015-06-08 19:10 UTC, Michał Górny
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 15:17:31 UTC
I'm slowly attempting at converting llvm & cmake ebuilds to use cmake.

So far, I've noticed the following:

1) there's no versioned libLLVM-3.2svn.so,

2) if I enable shared libs, the whole bunch of llvm libs is installed as .so,

3) libs are installed to /usr/lib64, (not /llvm),

4) udis86 is not supported for some reason [1],

5) AFAICS gold plugin is to be built separately,

6) ocaml bindings i still have to check. not sure if anybody wants that, though.

So far, I was able to build LLVM and run the test suite. I am attempting a second build with final ebuild and a test suite run. Then I will check if mesa still works and proceed to converting clang.

[1]:http://llvm.org/bugs/show_bug.cgi?id=15221
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 15:59:30 UTC
Created attachment 338408 [details]
LLVM test failures with new ebuild

There's 3 new test failures, PaX issues (but that's probably me failing) and gtest file collisions.

For the last one, I think we should unbundle it.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 16:01:57 UTC
Created attachment 338410 [details]
LLVM ebuild, rev1
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 17:31:53 UTC
Created attachment 338422 [details]
mesa build failure log

mesa fails to build, although I'm not sure if that's strictly LLVM-related or just randomly triggereed.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 18:12:47 UTC
Ok, PaX now works and that leaves us with 3 test failures which happen only with CMake.

I'm also wondering about installing all the libraries to /usr/$(get_libdir)/${P}, so that lack of SONAMEs wouldn't cause problems. But it'd be probably necessary to inject -Wl,--rpath to the build system.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 21:40:05 UTC
Created attachment 338440 [details]
LLVM ebuild, rev2

This one installs to /usr/lib*/${P} and has proper llvm-config. It also builds the gold plugin, though I haven't tested it.
Comment 6 Matt Turner gentoo-dev 2013-02-09 22:21:56 UTC
(In reply to comment #3)
> Created attachment 338422 [details]
> mesa build failure log
> 
> mesa fails to build, although I'm not sure if that's strictly LLVM-related
> or just randomly triggereed.

I completed the static Make -> automake transition for Mesa 9.1 which will be released in February. The first RC should appear on Monday. It builds against this LLVM ebuild cleanly.

The build failure in your log is regarding some developer-only tests programs that shouldn't even be built. We could remove them from the build, but..

Mesa 9.0.1 still has a large static Makefile build system which seems to assume that some test programs and libraries don't need to link with LLVM since it's already statically linked into another convenience library. But now that LLVM is a collection of DSOs instead of static archives it fails.

I don't really care to support shared LLVM on the previous Mesa build system. I'd be perfectly fine to require >=llvm-3.2 (built with CMake) for the upcoming Mesa 9.1 and require <llvm-3.2 (built with non-CMake) for older Mesa.

If we can do this, it will probably allow us to move forward with the multilib transition and include Mesa. That'd allow us to get rid of emul-linux-x86-opengl quite easily.

Cc'ing x11@ to get acknowledgement from chithead.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 22:48:30 UTC
Now I need to figure out how to build compiler-rt. It will probably end up in llvm ebuild, I guess.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-09 23:40:51 UTC
Created attachment 338444 [details]
LLVM ebuild, rev3

1) made PIC unconditional -- this is upstream default and it is required for gold plugin in cmake,

2) added USE=compiler-rt to build compiler-rt libraries. A bit ugly but easiest way around.

It shall be noted that due to the libdir-trick used to sit llvm in /usr/lib*/llvm-3.2, clang files are installed to /usr/lib*/llvm-3.2/clang/3.2/lib/linux. But that's probably no problem, just a bit ugly.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-10 11:18:57 UTC
Created attachment 338486 [details]
clang ebuild, rev1

With this ebuild, clang builds fine against external LLVM, and is installed in proper locations.

However, it can't find its own files so the toolchain finding code needs a lot of patching.
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-11 13:09:01 UTC
(In reply to comment #9)
> However, it can't find its own files so the toolchain finding code needs a
> lot of patching.

@ryao, could you please take a look at this? You were mangling the toolchain code so you should know best what to do.
Comment 11 José Romildo Malaquias 2013-02-20 01:45:21 UTC
(In reply to comment #0)

> 6) ocaml bindings i still have to check. not sure if anybody wants that,
> though.

Please, keep the ocaml bindings as I use them on my compiler construction classes.
Comment 12 José Romildo Malaquias 2013-02-20 01:56:41 UTC
(In reply to comment #8)

> It shall be noted that due to the libdir-trick used to sit llvm in
> /usr/lib*/llvm-3.2, clang files are installed to
> /usr/lib*/llvm-3.2/clang/3.2/lib/linux. But that's probably no problem, just
> a bit ugly.

In a previous ebuild I had worked out myself I have used 

  $(usex amd64 "-DLLVM_LIBDIR_SUFFIX=64")
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-21 08:54:33 UTC
(In reply to comment #11)
> (In reply to comment #0)
> 
> > 6) ocaml bindings i still have to check. not sure if anybody wants that,
> > though.
> 
> Please, keep the ocaml bindings as I use them on my compiler construction
> classes.

Hm, how did you handle them in your cmake ebuild?
Comment 14 José Romildo Malaquias 2013-03-01 13:16:59 UTC
(In reply to comment #13)
> (In reply to comment #11)
> > (In reply to comment #0)
> > 
> > > 6) ocaml bindings i still have to check. not sure if anybody wants that,
> > > though.
> > 
> > Please, keep the ocaml bindings as I use them on my compiler construction
> > classes.
> 
> Hm, how did you handle them in your cmake ebuild?

I didn't.

I asked on the llvmdev mailing list and I was told that in fact the CMake build does not contain the OCaml bindings.

I entered a new bug on llvm bug tracker asking to add them: http://llvm.org/bugs/show_bug.cgi?id=15325

I am using llvm bindings to Haskell, OCaml and Java.

The jllvm Java bindings used to require that llvm be installed with CMake, as jllvm itself uses CMake. But the latest version of jllvm has relaxed this requirement, and now can be build even when llvm is installed with the autotools.
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-04 18:57:06 UTC
Well, the binding part I can handle myself. But I need someone to hack the clang sources to fix libdir there, and possibly find and report all the libdir issues upstream.
Comment 16 Alexey Shvetsov archtester gentoo-dev 2013-03-09 20:40:27 UTC
Created attachment 341464 [details, diff]
add soversion cmake patch

Patch that adds soversions to clang and llvm libs
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-09 22:28:37 UTC
Created attachment 341478 [details]
LLVM ebuild, rev4

Changes:

- added patch from alexxy and moved libs back to /usr/lib*,
- added asan_*.py install (from compiler-rt),
- restricted tests if compiler-rt is enabled (fails to build and requires clang...).
Comment 18 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-09 22:30:05 UTC
Created attachment 341480 [details]
clang ebuild, rev2

Adjusted paths back to the original expectations and clang now works. Sadly, the build system is broken (what else would you expect of cmake?) and build clang-tblgen in $WORKDIR of llvm build. Bug 461024 for that.
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-10 19:27:20 UTC
Created attachment 341590 [details]
LLVM, rev5

Changes:

- FileCheck is now installed (though it has insecure RUNPATH),
- lit Python modules are installed.
Comment 20 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-10 19:32:34 UTC
Created attachment 341594 [details]
clang, rev3

Now clang runs tests. Many of them fail.
Comment 21 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-03-10 20:07:13 UTC
Created attachment 341596 [details]
Build log for tests

(I've stripped the irrelevant parts and verbose lit output)
Comment 22 Bernard Cafarelli gentoo-dev 2013-04-09 08:10:30 UTC
I had the same kind of failures (not,count, ... command not found) when I tried to fix dragonegg tests (#445680). Now why do these tests work with the old build system...
Comment 23 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-25 12:35:35 UTC
*** Bug 478090 has been marked as a duplicate of this bug. ***
Comment 24 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-12-26 20:22:18 UTC
From what I got from the mailing list, it seems that autoconf bs is basically unmaintained right now and I guess it's unlikely that the issues are going to end up being fixed. I think we should start sending cmake build fixes upstream.
Comment 25 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-12-28 12:04:47 UTC
Created attachment 366362 [details]
LLVM second round, r0

Plain LLVM without any other components so far. Tests work!

We will need to think how to build it all. The problematic thing is that compiler-rt uses system compiler to build the runtime, so we would need something like:

1. install LLVM,

2. install clang,

3. build compiler-rt using just-installed clang and install it.

If we do that in three ebuilds, it involves having broken/incomplete clang for some time. And compiler-rt is built from llvm's cmake, so we'll need to hack on it a fair bit.
Comment 26 Luca Barbato gentoo-dev 2014-02-10 11:47:56 UTC
llvm-3.4 multilib fails on cmake, maybe is better to reconsider this.
Comment 27 Bernard Cafarelli gentoo-dev 2015-02-19 12:18:08 UTC
Looks like upstream position nowadays is official cmake, and autotools slowly going the way of the dodo. I'd love to find some time to get this going again

4) udis86 was removed in 3.5 :)
6) bindings can now be built via cmake (but only with autodetection at the moment...).

New notes:
* Go bindings apparently can only be built with cmake
* cmake now support "host" as a target
* some features are (or will be) cmake-only: http://article.gmane.org/gmane.comp.compilers.llvm.devel/82213
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-06 10:12:53 UTC
Created attachment 404668 [details]
second round, r1

Here's the last ebuild updated a little bit. Still needs a lot of work.
Comment 29 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-07 21:55:57 UTC
Created attachment 404766 [details]
second round, r2

I'm getting closer. Known issues so far:

1. OCaml doc build has race conditions, patch submitted upstream,

2. sphinx has some mkdir race conditions, gotta figure it out,

3. OCaml bindings fail to build with -9999, probably upstream fault, gotta check release,

4. libclang_rt isn't installed,

5. clang resources end up in /usr/$(get_libdir)/clang rather than /usr/lib/clang,

6. compiler-rt build uses host CFLAGS/CXXFLAGS -- and passing some gcc flags to clang makes it unhappy. For now, I'm just stripping those flags globally.

Probably more issues, gotta get some sleep now.
Comment 30 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-08 12:43:20 UTC
If everything goes fine, I'll commit something p.masked today.
Comment 31 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-08 16:57:36 UTC
(In reply to Michał Górny from comment #29)
> Created attachment 404766 [details]
> second round, r2
> 
> I'm getting closer. Known issues so far:
> 
> 1. OCaml doc build has race conditions, patch submitted upstream,

Upstream has applied the patch.

> 2. sphinx has some mkdir race conditions, gotta figure it out,

Patch submitted upstream.

> 
> 3. OCaml bindings fail to build with -9999, probably upstream fault, gotta
> check release,

Release fails the same way :).

> 4. libclang_rt isn't installed,

Ok, have some more details on this one. LLVM has two options of building compiler-rt:

- with host compiler,

- with just-built clang.

If compiler-rt is built with host compiler, it installs normally. The build is also faster, can benefit from ninja and distcc. If it's built with clang, it lacks install target.

Now, I wonder if building with host gcc is 'good enough' for clang.
Comment 32 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-08 19:10:34 UTC
Created attachment 404780 [details]
second round, r3

Major issues left:

1. libgtest is installed,

2. clang runtime is installed into lib32/64.
Comment 33 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-06-09 16:55:50 UTC
+  09 Jun 2015; Michał Górny <mgorny@gentoo.org>
+  +files/cmake/0002-cmake-Support-overriding-Sphinx-HTML-doc-install-dir.patch,
+  +files/cmake/0003-cmake-Add-an-ordering-dep-between-HTML-man-Sphinx-ta.patch,
+  +files/cmake/0004-cmake-Do-not-install-libgtest.patch, +files/cmake/clang-0001
+  -Install-clang-runtime-into-usr-lib-without-suffix.patch,
+  +files/cmake/clang-0002-cmake-Make-CLANG_LIBDIR_SUFFIX-overridable.patch, +fil
+  es/cmake/compiler-rt-0001-cmake-Install-compiler-rt-into-usr-lib-without-suffi
+  .patch, llvm-9999.ebuild:
+  Switch the live ebuild to CMake, and few more improvements. Covered by bug
+  #456322.

Enjoy the new issues!

Just to be clear, I made it quite work. So far bfd plugin seems not to work for some reason but I don't know if it's cmake- or just 9999-related.