Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 712052 - sys-devel/clang fails to compile on x86, gets over 4GB in RAM
Summary: sys-devel/clang fails to compile on x86, gets over 4GB in RAM
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: LLVM support project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-03-10 11:21 UTC by Agostino Sarubbo
Modified: 2020-06-29 15:05 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
clang-9.0.1:20200310-104054.log.bz2 (clang-9.0.1:20200310-104054.log.bz2,62.17 KB, application/x-bzip2)
2020-03-10 11:21 UTC, Agostino Sarubbo
Details
emerge --info (.emergeinfo,5.26 KB, text/plain)
2020-03-10 11:21 UTC, Agostino Sarubbo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Agostino Sarubbo gentoo-dev 2020-03-10 11:21:29 UTC
This is an auto-filled bug because sys-devel/clang fails to compile on x86. Feel free to adjust the summary to clarify the exact issue. Attached build log and emerge --info
Comment 1 Agostino Sarubbo gentoo-dev 2020-03-10 11:21:35 UTC
Created attachment 617888 [details]
clang-9.0.1:20200310-104054.log.bz2

build log
Comment 2 Agostino Sarubbo gentoo-dev 2020-03-10 11:21:38 UTC
Created attachment 617890 [details]
emerge --info

emerge --info
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-03-15 14:46:05 UTC
/usr/lib/gcc/i686-pc-linux-gnu/9.2.0/../../../../i686-pc-linux-gnu/bin/ld: bfd_new_link_order failed
collect2: error: ld returned 1 exit status


Looks like binutils problem.
Comment 4 Sergei Trofimovich (RETIRED) gentoo-dev 2020-03-17 00:35:17 UTC
> CFLAGS="-O2 -pipe -march=native -g -ggdb"
> CXXFLAGS="-O2 -pipe -march=native -g -ggdb"
> LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu"

I would guess that debugging information enabled by -ggdb overflows 4GB limit on disk. Let's see if I can reproduce it. Should be easy to make linker error less obscure.
Comment 5 Sergei Trofimovich (RETIRED) gentoo-dev 2020-03-17 18:02:07 UTC
On my x86 system final linking fails with:

  /usr/lib/gcc/i686-pc-linux-gnu/9.2.0/../../../../i686-pc-linux-gnu/bin/ld: failed to set dynamic section sizes: memory exhausted

Which sounds about right if you are close to the limit.
Comment 6 Sergei Trofimovich (RETIRED) gentoo-dev 2020-03-17 19:27:54 UTC
LDFLAGS+=-Wl,--no-keep-memory allows to contain memory usage down to ~2.1GB RSS RAM.
Comment 7 Sergei Trofimovich (RETIRED) gentoo-dev 2020-03-17 21:48:54 UTC
You might want to add more conservative RAM behaviour with:
    use x86 && append-ldflags "-Wl,--no-keep-memory"

I'll sort out binutils error reporting separately.
Comment 8 Sergei Trofimovich (RETIRED) gentoo-dev 2020-03-17 22:44:09 UTC
As for where RAM usage comes from 'ld.gold ... --stat' renders the following stats:

```
...
ld.gold: output file size: 1506527692 bytes
...
ld.gold: .debug_str merged strings input bytes: 1861265804
ld.gold: .debug_str merged strings input strings: 25865501
ld.gold: .debug_str merged strings entries: 849387; buckets: 1447153
ld.gold: .debug_str merged strings Stringdata structures: 85222
...
```

1.8GB of debug strings alone which required in-memory deduplication. ld.bfd has a similar profile with 1.5x smaller RAM usage.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-06-29 15:04:42 UTC
I wonder if this is something we should do on all systems where linker executable uses ABI with 32-bit pointers (something for toolchain-funcs or flag-o-matic?) but I guess this is good enough for now.
Comment 10 Larry the Git Cow gentoo-dev 2020-06-29 15:05:02 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ec2b13a11a6b5712817b20dab096a668f4afd06f

commit ec2b13a11a6b5712817b20dab096a668f4afd06f
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2020-06-29 15:03:37 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2020-06-29 15:04:58 +0000

    sys-devel/clang: Pass -Wl,--no-keep-memory to linker on x86
    
    Closes: https://bugs.gentoo.org/712052
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 sys-devel/clang/clang-10.0.0.ebuild      | 4 ++++
 sys-devel/clang/clang-10.0.1.9999.ebuild | 4 ++++
 sys-devel/clang/clang-10.0.1_rc1.ebuild  | 4 ++++
 sys-devel/clang/clang-11.0.0.9999.ebuild | 4 ++++
 4 files changed, 16 insertions(+)