Created attachment 894488 [details] build.log While compiling a new i586 system I have found it seems to no long be able to compile clang-18.1.6 with -flto enable as it leads to an memory exhausted error. I'm using a faster amd64 machine for the compile in a chroot, testing the following: -flto -j32 - fail -flto -j1 - fail -flto -Wl,--no-keep-memory -Wl,--reduce-memory-overheads -j1 - fail No -flto and -j32 - Pass Further testing is required to see if this issue exists while compiling clang with itself as it has been known RAM usage has increased since -fno-lifetime-dse was added to clang when being compiled with GCC.
I'm also running out of memory during link of clang on x86 (i686) even without LTO, Seems to be an address space limit as there's plenty of swap unused. Tried native build (2GiB RAM, i686) and a chroot build (12GiB, amd64 with 32-bit emulation). Is there a recipe the devs are using that is consistently passing on i686 and other 32-bit systems? Maybe use some other linker? Or somehow reduce the image size?
There isn't a lot Gentoo can do here. Normal linking memory reducing tricks do not work for this issue and the changes in binutils-2.43 didn't help either. Personally I think you should not be natively compiling LTO as the time cost to benefit is very low so the best options moving forward are: 1. Disable LTO for llvm and clang on x86 (possibly all 32bit arches) 2. Look into using a proper cross compiler for enabling LTO on llvm. Option 2 becomes moot though as you won't be running on the host as the compiling has been offloaded to a faster machine. Note: this is a paraphrased discussion I've had with Sam on the topic, so thanks need to mainly go to him.
(In reply to Ben from comment #1) > I'm also running out of memory during link of clang on x86 (i686) even > without LTO, Seems to be an address space limit as there's plenty of swap > unused. > Is bfd OOMing or GCC? If it's bfd, does lld or mold fare any better? Also, what version of binutils, and are you building debug info?
I normally used bfd linker. I just tried lld and mold, last attempt was with mold: mold: fatal: lib/libclang-cpp.so.18.1: mmap failed: Cannot allocate memory collect2: error: ld returned 1 exit status last attempt of bfd: binutils-2.40 ... need to test 2.42 which is installed last attempt of lld: llvm-16 ... need to somehow use llvm-18.1 which was just installed last attempt of mold: mold-2.33.0 All three crashed with something mentioning running out of memory.
*** Bug 940228 has been marked as a duplicate of this bug. ***
Yes, please try the latest stable and then ~arch version of binutils (making sure to select it with binutils-config).
How can I disable lto for clang? This is my binutils config: [1] i686-pc-linux-gnu-2.40 * [2] i686-pc-linux-gnu-2.41 [3] i686-pc-linux-gnu-2.42 Will try 2.42, best without LTO if you tell me how to.
You're only using LTO if you're passing -flto. I'll retitle the bug as it's confusing.
I think I erroneously hijacked this bug as it initially was with LTO enabled. It seems many people are running into OOM with LTO disabled now too. I do wonder how immolo was able to build without LTO? I suspect using ROOT= and cross compile it may work but finding a recipe where the machine can finish build on its own ("self host"), with plenty of swap, would be the ideal case.
LLVM continues to grow which will be a big part of it. Please try the following things: 1) latest binutils in stable (2.42) 2) latest binutils in ~arch (2.43) 3) -Wl,--no-keep-memory -Wl,--reduce-memory-overheads in LDFLAGS 4) if none of the above work, try unforcing all the targets (I can tell you how to do this later but it's not risk-free)
For others out there and as an intermediate data point -- still testing this -- I was able to get it to build using lld-18.1 through a chroot on an amd64 using kernel IA32_EMULATION. (a dual core i3-4160 is much much faster than an Atom-N270) I'm currently redoing this on the N270 by itself (with distcc) to see if it will replicate. Unfortunately I think even if it does work, it's probably on the borderline of pass/fail and not a long term solution -- unless I somehow figure out how close it is to failing.
(In reply to Sam James from comment #10) > LLVM continues to grow which will be a big part of it. > > Please try the following things: > 1) latest binutils in stable (2.42) > 2) latest binutils in ~arch (2.43) > 3) -Wl,--no-keep-memory -Wl,--reduce-memory-overheads in LDFLAGS > 4) if none of the above work, try unforcing all the targets (I can tell you > how to do this later but it's not risk-free) I added as 3) (didn't know reduce-memory-overheads) in /etc/portage/make.conf and 1) switched from 2.40 to 2.42 and after 10hrs 11min according to genlop clang 18 emerged on my ThinkPad. Clang 5 was 4 hours... So for now a workaround on this which a "beefy" ThinkPad, on lesser 32bit it will be an issue.
Created attachment 903813 [details] Collated test results I think this report has become a little muddied with different things going on, so I have collated all the test info into one tarballl to allow us to see what is going on. The overview of the information is that llvm/clang now both require at least binutils 2.42 to be able to compile and 4GB of RAM (I used 3GB+1GB swap.)
Need to watch out -- there's more things going on here than seen. The Atom native failed sys-devel/clang using lld-18.1 where the same setup on the amd64 chroot. I also had native build fail with binutils-2.42 as well. IIRC the chroot/emulation on the amd64 is a hugely different environment than native, as well as QEMU. IIRC using amd64-ia32emu-chroot you get a larger user space as the kernel is doing funny stuff with the user/kernel mode split but there's still a 4GiB limit factored in. On pure 32-bit kernel the split is that typical 3G user/1G kernel or worse 2G user/ 2G kernel and it's easier to OOM with only 2G. That being said more clarification is necessary: failing sys-devel/clang or sys-devel/llvm. I did not have a "real" issue with sys-devel/llvm even on the Atom/native 32-bit. The main problem I had building llvm on the Atom is running too many llvm-tblgen at the same time due to running -j for distcc on the other portions. If I had -j1 the whole task it will always finish; given enough swap space larger -j will finish. sys-devel/clang on the other hand, is being stubborn and no amount of swap space helps sometimes. So far I've only been able to successfully build using amd64-ia32emu-chroot using lld-18.1. There's another distinction I need to bring up here: OOM because of OOM (usually overcommit) or OOM because of virtual memory exhaustion (out of address space). The former can be rectified with more swap - usually OOMkiller is involved here. The latter cannot be solved with swap. In fact OOMkiller is not even involved - check your dmesg to tell the difference, as well as the build log - when something is killed by OOMkiller the software has no chance of giving any last words. Voluntarily giving up because malloc() failed you can give some last words before giving up. sys-devel/clang is falling into this category where all three of ld.bfd, ld.mold, and ld.llvm are voluntarily giving up with OOM while trying to link. lld link failure on sys-devel/clang: FAILED: bin/clang-tidy ld.lld: error: cannot open lib/libclang-cpp.so.18.1: Cannot allocate memory collect2: error: ld returned 1 exit status bfd binutils link failure on sys-devel/clang: /usr/lib/gcc/i686-pc-linux-gnu/13/../../../../i686-pc-linux-gnu/bin/ld.bfd: can not read symbols: memory exhausted /usr/lib/gcc/i686-pc-linux-gnu/13/../../../../i686-pc-linux-gnu/bin/ld.bfd: out of memory allocating 1000 bytes after a total of 2425733120 bytes collect2: error: ld returned 1 exit status A table sort of needs to be drawn, this currently is my observations for sys-devel/clang so far: binutils-2.40 binutils-2.42 lld-16 lld-18.1 mold-2.33 native FAIL FAIL FAIL FAIL _ chroot FAIL _ FAIL PASS FAIL
This is interesting, I'll run more tests as well.
