Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 715612 - dev-util/llvm-polly: Polly - Polyhedral optimizations for LLVM
Summary: dev-util/llvm-polly: Polly - Polyhedral optimizations for LLVM
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal with 6 votes (vote)
Assignee: Default Assignee for New Packages
URL:
Whiteboard:
Keywords:
: 771138 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-03-31 12:28 UTC by Constantin Runge
Modified: 2021-12-04 09:08 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
LLVM ebuild (llvm-11.1.0-r2.ebuild,12.56 KB, text/plain)
2021-04-04 09:11 UTC, David Carlos Manuelda
Details
Polly 12.0.0 (polly-12.0.0.ebuild,1.74 KB, text/plain)
2021-04-22 05:48 UTC, Leonardo Neumann
Details
Polly 13.0.0 (polly-13.0.0.ebuild,1.45 KB, text/plain)
2021-12-04 07:38 UTC, Leonardo Neumann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Constantin Runge 2020-03-31 12:28:16 UTC
I'd like to request adding polly [1].

polly is a part of the llvm projects. It's their 'Framework for High-Level Loop and Data-Locality Optimizations'. As such, it is quite similar to gcc's Graphite.

As it is part of the llvm-project [2], I guess it should be buildable using the llvm.org eclass as well.

  [1] polly.llvm.org
  [2] https://github.com/llvm/llvm-project/tree/master/polly
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-02-17 20:03:23 UTC
*** Bug 771138 has been marked as a duplicate of this bug. ***
Comment 2 David Carlos Manuelda 2021-04-04 09:10:48 UTC
As a playground I modified the LLVM ebuild with an use flag to include polly support.

For now, due my limited knowledge about the LLVM build system, it is for the moment built statically.

Someone could take a look for building it as a library? I tried to modify distribution_components functions to add it, but somehow its libraries are not installed.

Anyway this static version works in my tests and can be a good place to start?
Comment 3 David Carlos Manuelda 2021-04-04 09:11:43 UTC
Created attachment 697146 [details]
LLVM ebuild

LLVM ebuild with static polly support via USE flag
Comment 4 David Carlos Manuelda 2021-04-04 12:40:02 UTC
Note that attached ebuild will make polly to work in currently installed Clang but will break every LLVM dependant install due to not installing Polly LLVMPolly and PollyISL libraries
Comment 5 Leonardo Neumann 2021-04-21 00:56:20 UTC
(In reply to David Carlos Manuelda from comment #4)
> Note that attached ebuild will make polly to work in currently installed
> Clang but will break every LLVM dependant install due to not installing
> Polly LLVMPolly and PollyISL libraries

Could you please describe the steps to reproduce the problem with your ebuild?
I ported your modifications to LLVM 12.0.0, but it seems to work both using Clang and rustc. I will use it for a few more days to see if something comes up.
Comment 6 David Carlos Manuelda 2021-04-21 17:14:53 UTC
(In reply to Leonardo Neumann from comment #5)
> (In reply to David Carlos Manuelda from comment #4)
> > Note that attached ebuild will make polly to work in currently installed
> > Clang but will break every LLVM dependant install due to not installing
> > Polly LLVMPolly and PollyISL libraries
> 
> Could you please describe the steps to reproduce the problem with your
> ebuild?
> I ported your modifications to LLVM 12.0.0, but it seems to work both using
> Clang and rustc. I will use it for a few more days to see if something comes
> up.

The ebuild I posted will work, however, once installed/merged, other LLVM dependant software will fail at configure stage due to missing files.

To reproduce, install LLVM with this changes and then try to emerge clang, at Cmake's configure stage it will complain about some .a files not installed by the ebuild.
Comment 7 Leonardo Neumann 2021-04-22 05:48:03 UTC
Created attachment 701571 [details]
Polly 12.0.0

Well, I managed to get polly as a standalone package and it seems to work fine and pass on all the supported tests. However, it has the drawback that you have to load it as an external optimization pass, which I think that can be solved by properly linking llvm against polly. To build a program using clang with standalone polly, you can do something like this:

clang -Xclang -load -Xclang libPolly.so -O2 -mllvm -polly test.c
Comment 8 David Carlos Manuelda 2021-04-22 10:58:06 UTC
(In reply to Leonardo Neumann from comment #7)
> Created attachment 701571 [details]
> Polly 12.0.0
> 
> Well, I managed to get polly as a standalone package and it seems to work
> fine and pass on all the supported tests. However, it has the drawback that
> you have to load it as an external optimization pass, which I think that can
> be solved by properly linking llvm against polly. To build a program using
> clang with standalone polly, you can do something like this:
> 
> clang -Xclang -load -Xclang libPolly.so -O2 -mllvm -polly test.c

I also saw that ebuild and tested that way.

Unfortunatelly using it this way breaks many packages, specially some autotools based.

The best would be to build LLVM with support for it but done in the proper way otherwise the build process would be a nightmare:

Build first LLVM+clang
Build polly
Rebuild LLVM (+clang?) to link against it
Comment 9 Perfect Gentleman 2021-04-22 11:53:45 UTC
I've implemented Polly several months ago without ebuilds modifications.
Comment 10 Leonardo Neumann 2021-04-22 12:19:22 UTC
(In reply to David Carlos Manuelda from comment #8)

> Unfortunatelly using it this way breaks many packages, specially some
> autotools based.

It shouldn't break anything, as long as the user can provide custom compiler flags. But I agree that linking against LLVM should be way more practical and we are not too far from achieving that.

(In reply to Perfect Gentleman from comment #9)
> I've implemented Polly several months ago without ebuilds modifications.

I stumbled upon your implementation, but unfortunately I didn't manage to get the tests running. Does it fix the linking problems with clang and other LLVM-dependent programs?
Comment 11 Perfect Gentleman 2021-06-17 17:57:02 UTC
(In reply to Leonardo Neumann from comment #10)
> I stumbled upon your implementation, but unfortunately I didn't manage to
> get the tests running. Does it fix the linking problems with clang and other
> LLVM-dependent programs?

Hmm... It's needed to add some extra LDFLAGS for LLVM staff.
Comment 12 unhappy-ending 2021-10-02 05:13:21 UTC
(In reply to Perfect Gentleman from comment #9)
> I've implemented Polly several months ago without ebuilds modifications.

Hi PG, how can I do this? I'd like to be able to test Polly. Thank you.
Comment 13 Leandro Hoshiko 2021-10-15 01:25:36 UTC
(In reply to unhappy-ending from comment #12)
> (In reply to Perfect Gentleman from comment #9)
> > I've implemented Polly several months ago without ebuilds modifications.
> 
> Hi PG, how can I do this? I'd like to be able to test Polly. Thank you.

I think you need to rebuild LLVM adding the Polly libraries -lPolly -lPollyISL in linker.

It works up to LLVM 12. In LLVM 13 I'm having linking error because lld is picking the 32-bit library instead of the 64-bit in the 64-bit compile phase, throwing an error of being incompatible with elf64-x86-64 (I've adapted the ebuild to be multilib).
Comment 14 Leonardo Neumann 2021-12-04 07:38:47 UTC
Created attachment 757373 [details]
Polly 13.0.0

The linker flags are the only missing thing to get Polly to work without the annoying plugin loading flags. I've been experimenting with Perfect Gentleman's ebuild for a while, and did some slight modifications, as both LLVM_POLLY_LINK_INTO_TOOLS and LLVM_CMAKE_PATH are not required for the ebuild to work. Also, the LDFLAGS are only required on the LLVM package, as both clang and rustc forwards polly optimizations to LLVM. Please test this ebuild and report whether it works for you.
Comment 15 Leonardo Neumann 2021-12-04 07:52:34 UTC
The only step that I think it would be important now is to add a polly useflag into LLVM ebuild so it pulls Polly as a dependency and also adds the required LDFLAGS for it to work.
Comment 16 Perfect Gentleman 2021-12-04 08:42:38 UTC
(In reply to Leonardo Neumann from comment #14)
> Created attachment 757373 [details]
> Polly 13.0.0

Tried your ebuild, built polly and tried to rebuild llvm
---
    ld.lld: error: unable to find library -lPolly
    ld.lld: error: unable to find library -lPollyISL
---
Comment 17 Leonardo Neumann 2021-12-04 08:57:36 UTC
(In reply to Perfect Gentleman from comment #16)
> (In reply to Leonardo Neumann from comment #14)
> > Created attachment 757373 [details]
> > Polly 13.0.0
> 
> Tried your ebuild, built polly and tried to rebuild llvm
> ---
>     ld.lld: error: unable to find library -lPolly
>     ld.lld: error: unable to find library -lPollyISL
> ---

Those libraries should be available under /usr/lib/llvm/13/lib/libPolly{ISL}.a, just like in your ebuild. I only removed LLVM_POLLY_LINK_INTO_TOOLS that should add the LDFLAGS in the LLVM tools if you build both LLVM and Polly together. Can you please check the installation image if these files are available?
Comment 18 Perfect Gentleman 2021-12-04 09:00:27 UTC
(In reply to Leonardo Neumann from comment #17)
> Those libraries should be available under
> /usr/lib/llvm/13/lib/libPolly{ISL}.a, just like in your ebuild. I only
> removed LLVM_POLLY_LINK_INTO_TOOLS that should add the LDFLAGS in the LLVM
> tools if you build both LLVM and Polly together. Can you please check the
> installation image if these files are available?

No, my ebuild installs them into /usr/lib64
-------
/usr/lib64/libPolly.a
/usr/lib64/libPollyISL.a
-------
Comment 19 Perfect Gentleman 2021-12-04 09:08:13 UTC
(In reply to Perfect Gentleman from comment #18)
> (In reply to Leonardo Neumann from comment #17)
> > Those libraries should be available under
> > /usr/lib/llvm/13/lib/libPolly{ISL}.a, just like in your ebuild. I only
> > removed LLVM_POLLY_LINK_INTO_TOOLS that should add the LDFLAGS in the LLVM
> > tools if you build both LLVM and Polly together. Can you please check the
> > installation image if these files are available?
> 
> No, my ebuild installs them into /usr/lib64
> -------
> /usr/lib64/libPolly.a
> /usr/lib64/libPollyISL.a
> -------

ebuild for polly-12.0 installed there