Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 477432 - sys-devel/llvm improvements from ::mgorny overlay
Summary: sys-devel/llvm improvements from ::mgorny overlay
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Bernard Cafarelli
URL:
Whiteboard:
Keywords: PMASKED
Depends on:
Blocks: 417349 422121 434200 453788 473690 474450 477250
  Show dependency tree
 
Reported: 2013-07-19 21:24 UTC by Michał Górny
Modified: 2013-07-31 23:00 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
fix some stuff (file_477432.txt,3.19 KB, patch)
2013-07-20 08:07 UTC, Oleg Ageev
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-19 21:24:26 UTC
Since my work in ::mgorny overlay has diverged pretty far from what we have in the tree and it addressed multiple issues, I decided to open a separate bug and describe all the changes.


1. Clang has been merged into sys-devel/llvm under USE=clang. Now sys-devel/clang serve the purpose of virtual/meta-package (but a meaningful, versioned one).

This solves the issue of LLVM libraries being rebuilt in sys-devel/clang, and also gives more guarantees about the sync between the LLVM library clang was built against and the library it will use. As a downside, build time and space consumption has increased dangerously.

I still believe that we should work on having clang built separately and therefore I believe we should keep other ebuilds depending on proper sys-devel/clang versions.


2. Multilib support has been added.

All LLVM and clang libraries are installed multilib now, and all multilib variants are tested in src_test(). As a downside, with USE=test we're building all LLVM and clang tools for all ABIs. With USE=-test, we're building only most important LLVM utils but the whole clang (build system limitation).

Bad thing is that on my amd64, both ABIs with USE=clang consume up to 2.3 GiB during build time.


3. RPATHs have been fixed and added to 'llvm-config --ldflags'.

Shortly saying, all issues related to RPATHs should now be fixed. I removed the $ORIGIN RPATHs completely. Now the executables are built with final RPATH only, and they are found during build time with a help of LD_LIBRARY_PATH set in the ebuild.

Additionally, 'llvm-config --ldflags' now gives RPATH switch in addition to the usual '-L'. This fixes things built against LLVM such as mesa.


4. Sed substitutions were replaced by patches with post-seds.

This means that a change in code would not cause silent breakage due to non-applied sed substitution. Instead, a patch will fail and have to be updated and figuring out bugs such as RPATH one would be easier.


5. clang installs its runtime to /usr/lib/clang (rather than libdir).

I think this is the only breaking change here. It follows suit with gcc, and improves multilib. The clang runtime contains only ABI-agnostic headers and ABI-versioned runtime libraries which are reused by the 32-bit and 64-bit compilers.


6. Live ebuild enforces use of the same SVN revision in all the repos.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-19 21:27:56 UTC
I'll allow myself to block all the bugs I've fixed so far.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-20 07:36:19 UTC
Good news, everyone! I've finally got working 3.3 build in my overlay.

Last issue that remains is the USE=debug build failing. However, it is horribly space-consuming so it will take a while before I can investigate it further. And this is low-priority to me, to be honest.
Comment 3 Oleg Ageev 2013-07-20 08:07:14 UTC
Created attachment 353696 [details, diff]
fix some stuff

I do not have time to catch up with you
once again rewrite your patch
please see
and if you have any questions can be discussed?
Comment 4 Oleg Ageev 2013-07-20 08:58:21 UTC
And some comments 
We should check broken gcc version only if we use gcc
I don't know how truth way to determine which compiler we use
Comment 5 Oleg Ageev 2013-07-20 09:12:32 UTC
(In reply to Michał Górny from comment #0) 
> I still believe that we should work on having clang built separately and
> therefore I believe we should keep other ebuilds depending on proper
> sys-devel/clang versions.
We cannot fix #417349 if we build clang separately and use clang as compiler
Comment 6 Nikoli 2013-07-20 10:22:44 UTC
Now ebuilds in overlay are broken, seems someone did not run 'repoman -pxv full':
  IUSE.missing                  2
   sys-devel/llvm/llvm-3.3-r1.ebuild: RDEPEND: USE flag 'vim-syntax' referenced in conditional 'vim-syntax?' is not in IUSE
   sys-devel/llvm/llvm-9999-r1.ebuild: RDEPEND: USE flag 'vim-syntax' referenced in conditional 'vim-syntax?' is not in IUSE

See attached patch.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-20 10:28:17 UTC
(In reply to Nikoli from comment #6)
> Now ebuilds in overlay are broken, seems someone did not run 'repoman -pxv
> full':
>   IUSE.missing                  2
>    sys-devel/llvm/llvm-3.3-r1.ebuild: RDEPEND: USE flag 'vim-syntax'
> referenced in conditional 'vim-syntax?' is not in IUSE
>    sys-devel/llvm/llvm-9999-r1.ebuild: RDEPEND: USE flag 'vim-syntax'
> referenced in conditional 'vim-syntax?' is not in IUSE

Fixed now.
Comment 8 Nikoli 2013-07-20 12:00:24 UTC
sys-devel/llvm-3.3-r1 builds fine, all tests work fine too, building vacuum package with clang works fine, scan-build also works fine.

But there are QA warnings:
 * QA Notice: The following files contain writable and executable sections
 *  Files with such sections will not work properly (or at all!) on some
 *  architectures/operating systems.  A bug should be filed at
 *  http://bugs.gentoo.org/ to make sure the issue is fixed.
 *  For more information, see http://hardened.gentoo.org/gnu-stack.xml
 *  Please include the following list of files in your report:
 *  Note: Bugs should be filed for the respective maintainers
 *  of the package in question and not hardened@g.o.
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundidf.o
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundisf.o
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundixf.o

Also there is QA warning about a lot of files:
 * QA Notice: The following files contain insecure RUNPATHs
 *  Please file a bug about this at http://bugs.gentoo.org/
 *  with the maintaining herd of the package.
 *   /var/tmp/portage/sys-devel/llvm-3.3-r1/image/usr/bin/clang
 *   /var/tmp/portage/sys-devel/llvm-3.3-r1/image/usr/bin/c-index-test
...
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-20 12:36:52 UTC
(In reply to Nikoli from comment #8)
> Also there is QA warning about a lot of files:
>  * QA Notice: The following files contain insecure RUNPATHs
>  *  Please file a bug about this at http://bugs.gentoo.org/
>  *  with the maintaining herd of the package.
>  *   /var/tmp/portage/sys-devel/llvm-3.3-r1/image/usr/bin/clang
>  *   /var/tmp/portage/sys-devel/llvm-3.3-r1/image/usr/bin/c-index-test
> ...

Are you sure you've used the newest version? I think I've fixed this.already. But please wait a few minutes, my next batch of LLVM builds shall finish and I'll have a fresh result and push next changes to the overlay.
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-20 15:51:52 UTC
Two new commits in the overlay. One fixes the runpath issue Nikoli mentioned. The other adds out-of-source build support. Using it, I was able to get multilib llvm+clang down to ~2G of disk space.
Comment 11 Nikoli 2013-07-20 16:38:56 UTC
'insecure RUNPATHs' are fixed, but these warnings are still here:
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundidf.o
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundisf.o
 * !WX --- --- usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundixf.o
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-21 09:16:48 UTC
(In reply to Nikoli from comment #11)
> 'insecure RUNPATHs' are fixed, but these warnings are still here:
>  * !WX --- ---
> usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundidf.o
>  * !WX --- ---
> usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundisf.o
>  * !WX --- ---
> usr/lib/clang/3.3/lib/linux/libclang_rt.full-x86_64.a:floatundixf.o

I'm afraid this one is outside my abilities and shall be taken upstream.
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-21 10:05:52 UTC
Committed to the tree and added to multilib p.mask.
Comment 14 Oleg Ageev 2013-07-22 11:52:05 UTC
(In reply to Michał Górny from comment #13)
> Committed to the tree and added to multilib p.mask.
in tree missing llvm-3.3-insecure-rpath.patch
which require for llvm-3.3-r1.ebuild

and why we check broken gcc version in pkg_setup
it can be checked in pkg_pretend or no?
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-22 12:03:17 UTC
(In reply to Oleg Ageev from comment #14)
> (In reply to Michał Górny from comment #13)
> > Committed to the tree and added to multilib p.mask.
> in tree missing llvm-3.3-insecure-rpath.patch
> which require for llvm-3.3-r1.ebuild

Missing patch added.

> and why we check broken gcc version in pkg_setup
> it can be checked in pkg_pretend or no?

That's something for a separate bug and careful consideration. I'm not sure if we can safely do that pkg_pretend() (gcc may be updated during the 'emerge @world' run).
Comment 16 Oleg Ageev 2013-07-22 12:11:08 UTC
(In reply to Michał Górny from comment #15)
package masked because have dependency which also masked?
but if I have no-multilib profile, I should unmask manually?
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-22 12:12:29 UTC
(In reply to Oleg Ageev from comment #16)
> (In reply to Michał Górny from comment #15)
> package masked because have dependency which also masked?
> but if I have no-multilib profile, I should unmask manually?

Yes, because of deps and because it waits for voyageur's review.
Comment 18 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-31 23:00:04 UTC
Committed and unmasked as -3.3-r1 and 9999.