In attempting to compile the parallel programming language Chapel ( https://chapel-lang.org/ https://github.com/chapel-lang/chapel ) and possibly later make an initial .ebuild file for such, I tried building it in my own (regular) user directory as follows: wget https://github.com/chapel-lang/chapel/releases/download/1.25.0/chapel-1.25.0.tar.gz tar xvf chapel-1.25.0.tar.gz cd chapel-1.25.0 export CHPL_LLVM=bundled source util/setchplenv.bash make -j which builds Chapel successfully --by building and compiling a bundled LLVM/Clang. (ASIDE: The v1.25.0 release of Chapel will only build with v11.) Unfortunately, if one already has installed or would prefer to use sys-devel/clang:11 installed in the system one should be able to do this: # Run the next line if clang:11 is not already installed... emerge -av sys-devel/clang:11 wget https://github.com/chapel-lang/chapel/releases/download/1.25.0/chapel-1.25.0.tar.gz tar xvf chapel-1.25.0.tar.gz cd chapel-1.25.0 export CHPL_LLVM=system export CHPL_LLVM_CONFIG=/usr/lib/llvm/11/bin/llvm-config export CHPL_HOST_COMPILER=llvm export CHPL_TARGET_COMPILER=llvm source util/setchplenv.bash make -j except this will fail due to these errors: ld.lld: error: unable to find library -lclangFrontend ld.lld: error: unable to find library -lclangSerialization ld.lld: error: unable to find library -lclangDriver ld.lld: error: unable to find library -lclangCodeGen ld.lld: error: unable to find library -lclangParse ld.lld: error: unable to find library -lclangSema ld.lld: error: unable to find library -lclangAnalysis ld.lld: error: unable to find library -lclangEdit ld.lld: error: unable to find library -lclangASTMatchers ld.lld: error: unable to find library -lclangAST ld.lld: error: unable to find library -lclangLex ld.lld: error: unable to find library -lclangBasic clang-11: error: linker command failed with exit code 1 (use -v to see invocation) I checked for these libraries in sys-devel/clang with: $ equery f sys-devel/clang:11 | grep libclang /usr/lib/llvm/11/lib64/libclang-cpp.so /usr/lib/llvm/11/lib64/libclang-cpp.so.11.1 /usr/lib/llvm/11/lib64/libclang.so /usr/lib/llvm/11/lib64/libclang.so.11.1 $ and clearly they are missing. I installed Debian stable in a chroot and determined that the Debian package libclang-11-dev (which must be installed as an "extra") includes a number of libraries that are not installed in Gentoo, i.e., ones that Chapel needs to link to (i.e. the ones in the above error message). With libclang-11-dev using "export CHPL_LLVM_CONFIG=/usr/lib/llvm-11/bin/llvm-config" under Debian, Chapel builds using the system-installed LLVM/Clang. I examined the ebuild for =sys-devel/clang-11.1.0 and noted this line in src_prepare(): mv ../clang-tools-extra tools/extra || die and looking at the build logs it appears that effectively this line appears to prevent these libraries from being installed. If I comment out that line, the ebuild fails --but there is a lot going on in the .ebuild and I haven't explored such. In the end, the consequence is it is not possible to build Chapel (v1.25.0) under Gentoo using a system-installed LLVM/Clang even if such is already installed. So... Would it be easily possible (perhaps via a USE flag) to add the clang-tools-extra components to sys-devel/clang:11 (and other versions too)? This would allow tools such as Chapel to successfully build using system-installed components and not downloaded/bundled components and would assist creating ebuilds that require such components. (Using a USE flag would allow one to not install such if it is not needed.) Or is there a larger technical set of reasons these library components are not installed with dev-lang/clang under Gentoo?
https://archives.gentoo.org/gentoo-dev/message/6d3cf88d858fcf3fbb11818ce5d6ea42 So build system of Chapel should try to use -lclang-cpp (or less likely -lclang) instead of those many -lclang* options (-lclangFrontend, -lclangSerialization, ...).
I'm sorry but your analysis is wrong. The ebuild does install clang-tools-extra -- e.g. clangd is definitely installed. The ebuild does not install static libraries by design. All public API should be included in libclang-cpp.so. Chapel needs to be fixed to support linking to the single shared library (also called the 'dylib' in upstream LLVM terms). If you hit any problems after replacing the list of linked libraries with '-lclang-cpp', feel free to reopen the bug.
Changes needed probably in this file: https://github.com/chapel-lang/chapel/blob/main/third-party/llvm/Makefile.include-system
Thanks for the heads-up on libraries to link to: I fixed the Chapel source to link to -lclang-cpp. Such still failed so I also ensured third-party/llvm/Makefile.include-system had the following: LLVM_CLANG_LIBS=-lclang-cpp LLVM_CXXFLAGS=$(LLVM_CONFIG_CXXFLAGS) $(LLVM_MY_CXXFLAGS) -DHAVE_LLVM -fPIC LLVM_CFLAGS=$(LLVM_CONFIG_CFLAGS) -DHAVE_LLVM -fPIC LLVM_LIBS=-dynamic -shared -L$(LLVM_CONFIG_LIB_DIR) $(LLVM_CLANG_LIBS) $(LLVM_LLVM_LIBS) So by starting a new shell and running: wget https://github.com/chapel-lang/chapel/releases/download/1.25.0/chapel-1.25.0.tar.gz tar xvf chapel-1.25.0.tar.gz cd chapel-1.25.0 # update third-party/llvm/Makefile.include-system per above export CHPL_LLVM=system export CHPL_LLVM_CONFIG=/usr/lib/llvm/11/bin/llvm-config export CHPL_HOST_COMPILER=llvm export CHPL_TARGET_COMPILER=llvm source util/setchplenv.bash make -j it successfully compiles but when the chpl program is run (even with no options), it immediately segfaults. (Using other options export CHPL_LIB_PIC=pic option in the above doesn't help either. The -fPIC is needed and shared library options are needed in the LLVM Makefile for the code to compile and link. If either is omitted, it will fail to link.)
My current hypothesis is the segfault may be the result of the Chapel code using include files from an older Clang v11 (i.e., what is bundled) than what is installed in the system. I will be investigating this.
Created attachment 745935 [details] Singularity definition file that builds a Gentoo container + some tools. If you don't already have Singularity installed, "emerge sys-cluster/singularity". See the comment with the demo.sh attachment for additional details.
Created attachment 745938 [details] BASH script that automates building a Gentoo Linux and test case containers. I made a discovery: if sys-devel/clang is built using default options (i.e., the USE flags default-compiler-rt, default-libcxx, and default-lld are not set) then Chapel v1.25.0 builds and works. In various other cases it does not. I made this discovery because I was asked in my Chapel ticket concerning this issue how to install Gentoo Linux so such could be re-created, etc. I thought it would be best to use a container for such and Singularity (i.e., sys-cluster/singularity for those running Gentoo Linux, or, https://singularity.hpcng.org/ https://github.com/hpcng/singularity (or commercial: https://sylabs.io )), thus... I have attached two files: * attachment demo.sh * attachment singularity-gentoo-amd64-chapel-basis.def that completely automate the following: 1. Building a pristine Gentoo Linux (stage3) Singularity container image + some needed packages. This is placed in a directory called gentoo.dir/ by demo.sh. 2. Copying gentoo.dir/ to caseNUM.dir/ and then installing sys-devel/clang:11 + Chapel in the caseNUM.dir/ Singularity container image. Item 1 and each case in item 2 are logged, i.e., gentoo.log and caseNUM.log files. Since each of these are in their own separate directories and are containers one can explore after demo.sh finishes any container by running: * singularity shell -C -w caseNUM.dir (per demo.sh the Chapel stuff was built and installed in /testing/chapel and the rest of the image is Gentoo Linux). Adding more cases (whatever with USE flags, etc.) to demo.sh is straight-forward and will sanely facilitate determining which issues are Chapel issues and/or Gentoo Linux sys-devel/clang-related issues. (I do set MAKEOPTS="-j$(nproc) -l$(nproc)" in the Gentoo Linux build to speed up builds. If such is undesired, it is easily found and edited in both files.) Running demo.sh on a 16-core AMD Ryzen Threadripper PRO 3955WX in RAM takes approximately 30 minutes or so for each case after building the base Gentoo Linux container used by all cases. Each case's log file is about 20M and each case directory is about 3.3G. (gentoo.dir is about 1.9G and gentoo.log about 6.8M.) Chapel is built and successfully runs in case01 and case04. In case01, sys-devel/clang is built using the default USE flags. In case04, sys-devel/clang is built with the default-lld USE flag set. The remaining cases fail as follows: * case02: sys-devel/clang was successfully built with the default-compiler-rt USE flag. Unfortunately, Chapel failed to link at the end with "/usr/bin/x86_64-pc-linux-gnu-ld: ./../build/compiler/linux64/llvm/x86_64/llvm-system/none/AST/AggregateType.o: undefined reference to symbol '_Unwind_Resume@@GCC_3.0' /usr/bin/x86_64-pc-linux-gnu-ld: /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/libgcc_s.so.1: error adding symbols: DSO missing from command line" * case03: sys-devel/clang was successfully built with the default-libcxx USE flag. Unfortunately, Chapel failed to link with the errors that originally triggered me opening this ticket. * case05: sys-devel/clang was successfully built with the default-compiler-rt and default-lld USE flags. Unfortunately, Chapel failed to link with a (different) _Unwind_Resume (undefined symbol) error than case01. * case06: sys-devel/clang was NOT successfully built with the default-compiler-rt, default-libcxx, and default-lld USE flags set due to a block, i.e., "[blocks B ] sys-libs/libunwind ("sys-libs/libunwind" is blocking sys-libs/llvm-libunwind-12.0.1)". This is essentially a stage3 Gentoo Linux install + a couple of tools + attempting to install clang. NOTE: For case06 and this bug report... This sys-devel/clang block is desired behaviour, fine but if not, then something needs to be fixed. (Obviously building Chapel was not done in case06.) NOTE: The above cases write to /etc/portage/package.use/llvm to specify the USE flags for sys-devel/clang. The demo.sh script emerges sys-devel/clang:11 --and somehow that pulls in sys-devel/clang:12. Again, if this is not desired, then perhaps it is something that should be fixed? For completeness (although it is easily seen in demo.sh) building Chapel was done in each case using the following commands: source /etc/profile cd /testing/chapel/ export CHPL_LLVM=system export CHPL_LLVM_CONFIG=/usr/lib/llvm/11/bin/llvm-config export CHPL_HOST_COMPILER=llvm export CHPL_HOST_CC=clang-11 export CHPL_HOST_CXX=clang++-11 export CHPL_TARGET_COMPILER=llvm export CHPL_TARGET_CC=clang-11 export CHPL_TARGET_CXX=clang++-11 sed -i -e 's/LLVM_CLANG_LIBS=.*/LLVM_CLANG_LIBS=-lclang-cpp/' third-party/llvm/Makefile.include-system source util/setchplenv.bash make -j\$(nproc) chpl --version chpl --help The reason I used CHPL_HOST_* and CHPL_TARGET_* was to try to ensure Chapel used Clang version 11 since more than one version of Clang was pulled in through the emerging of sys-devel/clang:11 to try to keep things clean. Finally, in the demo.sh script one should start at the bottom, e.g., build_singularity_base_gentoo_linux_container |& tee -a "gentoo.log" build_working_chapel ${CHAPEL_VERSION} case01 build_working_chapel ${CHAPEL_VERSION} case02 default-compiler-rt build_working_chapel ${CHAPEL_VERSION} case03 default-libcxx build_working_chapel ${CHAPEL_VERSION} case04 default-lld build_working_chapel ${CHAPEL_VERSION} case05 default-compiler-rt default-lld build_working_chapel ${CHAPEL_VERSION} case06 default-compiler-rt default-libcxx default-lld If one wants to re-create a case, simply "sudo rm -rf caseNUM.dir" and then re-run the script (or comment out those lines). Obviously, to run demo.sh one has to have app-admin/sudo installed and configured + sys-cluster/singularity installed first. Many continued thanks for all help, comments, suggestions, etc. :-) Paul P.S. It took a while to create all of this and with the script + the .def file and I hope it is useful for this bug (and perhaps maybe to help explore other bugs/tickets/issues). P.P.S. I opened a Github issue with Chapel concerning building it with a system-installed LLVM/Clang. It is here: https://github.com/chapel-lang/chapel/issues/18589 . I will be appropriately posting there about the demo.sh script, etc. very soon as well.
