Is possible to unbundle it?
Unfortunately, it's not possible to unbundle llvm. It is used within ROOT in a heavily patched form to provide a C++ interpreter (https://root.cern.ch/cling). This is a new interpreter to substitute the old one (https://root.cern.ch/cint). ROOT began to be developed before C++ and Python as we know them existed. At the time, the developers decided to create an interpreter for C++ to allow scripting of data analysis, plotting figures, etc. It came to substitute PAW, which had a Fortran interpreter for the same purpose. We can't fix this now, but according to the current maintainer of ROOT, Philippe Canal, unbundling is on their plans: On Thu, Oct 29, 2015 at 04:44:03PM -0500, Philippe Canal wrote: > Hi Guilherme, > > We want to eventually get to a point where it would be possible but > this requires a lot of work (working the patch to be acceptable for > the general case, testing them, etc) *and* we would have to be in a > point in time when we need patches rarely enough that there is a > chance that a release of llvm/clang has all we need. > > Cheers, Philippe Cheers, —Guilherme
This is still an issue, and a major one.
Yes, this is still an issue, and it is still impossible to fix, because ROOT requires patched version of LLVM, consider it an another fork project. By the way I disagree that this issue is a major one: we have hundreds of packages with bundled stuff due to one reason or another.
The bug is not resolved, and please do not close it until you resolve it properly. Other bad packages are not an excuse. Especially when you start introducing more broken packages on the excuse of one, and then end up excusing the one with other packages committed afterwards. Also, bundling LLVM is a *major* problem because LLVM is *huge*.
(and considering how much effort proper LLVM maintenance takes, I seriously doubt they're doing it right)
(In reply to Michał Górny from comment #4) > The bug is not resolved, and please do not close it until you resolve it > properly. This bug is not possible to resolve. CANTFIX resolution exists exactly for such cases. Please use bugzilla properly. > Other bad packages are not an excuse. Especially when you start introducing > more broken packages on the excuse of one, and then end up excusing the one > with other packages committed afterwards. > > Also, bundling LLVM is a *major* problem because LLVM is *huge*. Have you read what I wrote earlier in the comment #3? > Yes, this is still an issue, and it is still impossible to fix, because ROOT > requires patched version of LLVM, consider it an another fork project. Unbundling will fix nothing. Unbundled version will be not LLVM, but another project based on LLVM. Normal LLVM consumers may not be able to use it, ROOT will not be use system LLVM as well because it lacks required patches. Unbundled ROOT version of LLVM will have to be updated strictly with each ROOT update. Technically this is no different than taking a part of some large project and forcefully packaging it separately. Please explain me what benefits we will have from unbundling ROOT's LLVM version considering it is not fully compatible with upstream LLVM and can't be used outside ROOT as well as ROOT can't use system LLVM.
(In reply to Michał Górny from comment #4) > The bug is not resolved, and please do not close it until you resolve it > properly. > > Other bad packages are not an excuse. Especially when you start introducing > more broken packages on the excuse of one, and then end up excusing the one > with other packages committed afterwards. > > Also, bundling LLVM is a *major* problem because LLVM is *huge*. Yes, we all agree with this, but this is not a problem that can or should be solved by Gentoo. ROOT uses LLVM to create a C++ interpreter, and it unfortunately requires changes (and maybe also a few hacks) on the compiler side to make it work. Last year I started working at CERN on the ROOT development team, so I can assure you that the issue is being addressed upstream, by pushing patches to LLVM and removing them from ROOT. In fact, just last week it has become possible to build ROOT using an external version of LLVM. However, ROOT still needs patches in clang to make the interpreter work, and which are API-incompatible with vanilla clang. In the long term, the goal is to be able to use external clang as well. When the next release of ROOT comes out, I will package it in Gentoo using an external LLVM already, and when it becomes possible to compile ROOT against external clang, that will be done as well. By all means leave the bug open and assigned to me, although closing it as upstream is probably more appropriate.
I see =sci-physics/root-6.14.02 still uses "-Dbuiltin_llvm=ON". For clang, I fully understand this is still being worked on, but for LLVM, is this really still needed?
(In reply to Oliver Freyermuth from comment #8) > I see =sci-physics/root-6.14.02 still uses "-Dbuiltin_llvm=ON". > For clang, I fully understand this is still being worked on, but for LLVM, > is this really still needed? Unfortunately, yes. Basically, if LLVM isn't statically linked into libCore.so, but picked from outside as an external library, ROOT crashes at runtime when I try to run one of the demos (e.g. hsimple). I've had LLVM as external, but reverted later when I realized it was broken. If you want, the live ebuild uses external LLVM by default. I am keeping track of the progress of support for external LLVM, but for the moment we still have to live with this.
Any progress?
Hi Benda. Yes! ROOT 6.16/00 has just been released, and I think that now using external LLVM in Gentoo will be ok. However, Clang still needs ~80 patches so it will take longer.
(In reply to Guilherme Amadio from comment #11) > Hi Benda. Yes! ROOT 6.16/00 has just been released, and I think that now > using external LLVM in Gentoo will be ok. However, Clang still needs ~80 > patches so it will take longer. That's a good news! Thank you :D
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77fb886a0bf351673ebf371d6f87d50cf5f0dd87 commit 77fb886a0bf351673ebf371d6f87d50cf5f0dd87 Author: Guilherme Amadio <amadio@gentoo.org> AuthorDate: 2019-02-04 07:37:36 +0000 Commit: Guilherme Amadio <amadio@gentoo.org> CommitDate: 2019-02-04 07:37:57 +0000 sci-physics/root: version bump to 6.16/00 Now using external LLVM. Bug: https://bugs.gentoo.org/564302 Package-Manager: Portage-2.3.59, Repoman-2.3.12 Signed-off-by: Guilherme Amadio <amadio@gentoo.org> sci-physics/root/Manifest | 1 + sci-physics/root/root-6.16.00.ebuild | 322 +++++++++++++++++++++++++++++++++++ 2 files changed, 323 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=175836287f5f0b91f09445dcdcd2f0b340cc7699 commit 175836287f5f0b91f09445dcdcd2f0b340cc7699 Author: Guilherme Amadio <amadio@gentoo.org> AuthorDate: 2019-09-06 11:56:33 +0000 Commit: Guilherme Amadio <amadio@gentoo.org> CommitDate: 2019-09-09 06:10:27 +0000 sci-physics/root: switch back to builtin LLVM Using external LLVM never really worked and can cause crashes if ROOT is used with anything else that dynamically links against LLVM, such as media-libs/mesa[llvm]. The bundled version of clang is also API incompatible with vanilla clang and doesn't work as a compiler with the applied patches that are required to make the cling C/C++ interpreter work. More information can be found in the links below: Bug: https://bugs.gentoo.org/564302 Bug: https://bugs.gentoo.org/687212 Bug: https://sft.its.cern.ch/jira/browse/ROOT-9965 Bug: https://sft.its.cern.ch/jira/browse/ROOT-10287 Bug: https://cern.service-now.com/service-portal/view-incident.do?n=INC1242141 (requires authentication) Bug: https://bugs.freedesktop.org/show_bug.cgi?id=93103 Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1441957 (restricted) Package-Manager: Portage-2.3.71, Repoman-2.3.17 Signed-off-by: Guilherme Amadio <amadio@gentoo.org> .../root/{root-6.16.00-r3.ebuild => root-6.16.00-r4.ebuild} | 8 ++------ .../root/{root-6.18.00-r2.ebuild => root-6.18.00-r3.ebuild} | 8 ++------ sci-physics/root/root-9999.ebuild | 8 ++------ 3 files changed, 6 insertions(+), 18 deletions(-)
I really think we should just close this one as an upstream problem. I explain in the comments above why it's not possible to ubundle LLVM. Upstream (ROOT) is working on upstreaming most stuff to LLVM/clang as part of clang-repl to eventually use vanilla llvm/clang as much as possible (see https://github.com/compiler-research/CppInterOp), but due to the nature of ROOT being a C++ interpreter, that may never happen. This is not the usual type of vendoring people do, it involves a *lot* of work and is definitely not something we (Gentoo) can fix on our own. Even more recently upstream has issues filed for problems with external llvm, it's usually a clash with mesa or some other project that links to a different version of llvm and that's why it needs to be statically linked with hidden symbols. See https://github.com/root-project/root/issues/14778 I think there's no point in keeping this bug open. What do you think?
A bundled library is a bundled library. What upstream thinks doesn't change the fact that it's a security liability.
Keeping the bug open will not change the fact that we cannot fix this. And I don't think this is a security issue, there's constant work in LLVM (including upstreaming patches) that is done by people working on ROOT. I suggest you chat with Vassil about this. In any case, if you want this to stay open, it's fine by me, I just would personally close this as can't fix.
