>> collect2: ld terminated with signal 9 [Killed] Portage 2.2.0_alpha177 (default/linux/amd64/13.0, gcc-4.6.3, glibc-2.15-r3, 3.8.13-gnu-ck1 x86_64) ================================================================= System uname: Linux-3.8.13-gnu-ck1-x86_64-Intel-R-_Core-TM-2_CPU_6400_@_2.13GHz-with-gentoo-2.2 KiB Mem: 4051392 total, 3135524 free KiB Swap: 2047996 total, 1591828 free Timestamp of tree: Tue, 28 May 2013 14:00:01 +0000 ld GNU ld (GNU Binutils) 2.22 distcc 3.1 x86_64-pc-linux-gnu [disabled] ccache version 3.1.9 [enabled] app-shells/bash: 4.2_p45 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.5.4-r5, 2.6.8-r1, 2.7.3-r3, 3.1.5-r1, 3.2.3-r2, 3.3.2 dev-util/ccache: 3.1.9 dev-util/cmake: 2.8.10.2-r2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.10.3, 1.11.6, 1.12.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.4.7, 4.5.4, 4.6.3, 4.7.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo gentoo-zh Neurogeek sunrise arx-libertatis maggu2810-overlay hasufell crossdev hasufell-overlay science Installed sets: @bleh, @development, @games, @optional, @steam, @test, @xfce ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -O2 -pipe -Wall -g" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/applications /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=core2 -O2 -pipe -Wall -g" DISTDIR="/home/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs ccache collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch protect-owned sandbox sfperms sign split-log splitdebug strict test-fail-continue unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://mirror.netcologne.de/gentoo/ ftp://gentoo.imj.fr/pub/gentoo/ ftp://de-mirror.org/gentoo/" INSTALL_MASK="/usr/lib/systemd/*" LANG="de_DE.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu" MAKEOPTS="-j2" PKGDIR="/home/packages" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="-z -9" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/gentoo-zh /var/lib/layman/neurogeek /var/lib/layman/sunrise /var/lib/layman/arx-libertatis /var/lib/layman/maggu2810-overlay /var/lib/layman/hasufell /usr/local/crossdev /usr/local/portage /usr/local/portage-science" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 apng bash-completion berkdb bzip2 cairo cdr cli consolekit cracklib crypt cscope css cxx dbus dri dv dvd dvdr exif fat ffmpeg flac fontconfig fortran gdbm gif gpm gstreamer hddtemp iconv icq icu ipv6 jabber java jpeg jpeg2k lame libnotify lm_sensors matroska mmx modules mp3 mp4 mpeg mudflap multilib musepack ncurses nls nptl nsplugin ntfs ogg opengl openmp oscar pam pcre pdf png policykit python raw readline sdl session sound sse sse2 ssl ssse3 svg tcpd threads tiff timidity truetype udev unicode usb v4l vcd vdpau vim-syntax vnc vorbis wavpack wmf x264 xv xvid xvmc zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="pdfimport presenter-console presenter-minimizer wiki-publisher" LINGUAS="en de" NETBEANS_MODULES="*" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_5 python2_6 python2_7 python3_1 python3_2 python3_3" QEMU_SOFTMMU_TARGETS="i386 ppc ppc64 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.5 2.6 2.7 3.1 3.2" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 349652 [details] chromium-27.0.1453.93:20130529-144438.log.xz
Any messages in the kernel log?
[138174.687107] Out of memory: Kill process 23076 (ld) score 737 or sacrifice child [138174.687109] Killed process 23076 (ld) total-vm:4684904kB, anon-rss:3804424kB, file-rss:68kB That is with 4GB ram and no big applications running. Webkit, firefox, libreoffice and other bloat stuff never failed like that for me. I wonder how chromium can be stable that way. Can reproduce.
Remove "-g" from your CFLAGS and CXXFLAGS; the debug information makes the linker consume a much larger amount of memory. You can use package.env if you want to do it only for chromium.
I wonder why the death hook did not warn you of this... do you have ewarn messages disabled or something? See chromium_pkg_die in chromium.eclass.
pkg_pretend() { if is-flagq '-g?(gdb)?([1-9])'; then if ! is-flagq '-g?(gdb)0' ; then CHECKREQS_MEMORY=4G nonfatal check-reqs_pkg_pretend fi fi } this could be used either in the ebuild or the eclass
since nonfatal on eclass functions only works because of a portage bug we could do: pkg_pretend() { if is-flagq '-g?(gdb)?([1-9])'; then if ! is-flagq '-g?(gdb)0' ; then CHECKREQS_MEMORY=4G I_KNOW_WHAT_I_AM_DOING=1 check-reqs_pkg_pretend fi fi }
That seems ok to me. We may want to see what the memory requirement for x86 is, or if it is even possible to build with -g on that arch; it may hit the 2G per-process limit there.
(In reply to Julian Ospald (hasufell) from comment #7) > since nonfatal on eclass functions only works because of a portage bug we > could do: > > pkg_pretend() { > if is-flagq '-g?(gdb)?([1-9])'; then > if ! is-flagq '-g?(gdb)0' ; then > CHECKREQS_MEMORY=4G I_KNOW_WHAT_I_AM_DOING=1 check-reqs_pkg_pretend > fi > fi > } Actually let's remove the I_KNOW_WHAT_I_AM_DOING=1 part. Let's also figure out the requirements without -g flag. I'll be doing my measurements at some point, so I'm fine with checking in something that we could tweak later.
(In reply to Paweł Hajdan, Jr. from comment #9) > (In reply to Julian Ospald (hasufell) from comment #7) > > since nonfatal on eclass functions only works because of a portage bug we > > could do: > > > > pkg_pretend() { > > if is-flagq '-g?(gdb)?([1-9])'; then > > if ! is-flagq '-g?(gdb)0' ; then > > CHECKREQS_MEMORY=4G I_KNOW_WHAT_I_AM_DOING=1 check-reqs_pkg_pretend > > fi > > fi > > } > > Actually let's remove the I_KNOW_WHAT_I_AM_DOING=1 part. I would not trust check-reqs to reliably figure that out and abort the emerge process. What if a user uses zram and is able to compile it that way? There are probably other cases where our calculation might not be accurate enough.
(In reply to Julian Ospald (hasufell) from comment #10) > I would not trust check-reqs to reliably figure that out and abort the > emerge process. What if a user uses zram and is able to compile it that way? > There are probably other cases where our calculation might not be accurate > enough. That's perfect case for the user setting I_KNOW_WHAT_I_AM_DOING=1. Note that we already have a _warning_ if compile fails (the die hook), and it prints info about memory and -g. If that's not enough for you (and I respect that opinion), another _warning_ doesn't seem to add much to me.
(In reply to Paweł Hajdan, Jr. from comment #11) > (In reply to Julian Ospald (hasufell) from comment #10) > > I would not trust check-reqs to reliably figure that out and abort the > > emerge process. What if a user uses zram and is able to compile it that way? > > There are probably other cases where our calculation might not be accurate > > enough. > > That's perfect case for the user setting I_KNOW_WHAT_I_AM_DOING=1. > > Note that we already have a _warning_ if compile fails (the die hook), and > it prints info about memory and -g. > > If that's not enough for you (and I respect that opinion), another _warning_ > doesn't seem to add much to me. The warning is simply misplaced. The user should know about possible complications with his setup BEFORE he compiles chromium for 1.5h which I did, just to realize it was all for nothing. Having it in both places however is not bad.
(In reply to Paweł Hajdan, Jr. from comment #11) > (In reply to Julian Ospald (hasufell) from comment #10) > > I would not trust check-reqs to reliably figure that out and abort the > > emerge process. What if a user uses zram and is able to compile it that way? > > There are probably other cases where our calculation might not be accurate > > enough. > > That's perfect case for the user setting I_KNOW_WHAT_I_AM_DOING=1. > That's an assumption.
/usr/lib/gcc/i686-pc-linux-gnu/4.6.3/../../../../i686-pc-linux-gnu/bin/ld: failed to set dynamic section sizes: Memory exhausted This is a MAJOR issue guys. In the original report we have a system with 4GB of RAM simply not having enough ram to build with -g or -ggdb, however, the system I'm building it on has 64GB of ram but building for i686 runs into a per-process limit of 4GB. We need to filter flags -g -ggdb here on x86, there is a ZERO percent chance of successful build with those flags enabled on x86. Please take this issue seriously.
This does not affect most users, and the workaround is simple: remove -ggdb from CFLAGS. I think filtering it on x86 is not a bad idea. It's a pain in the ass for you, but that does not make this a critically important issue. Turn it down a notch please.
(In reply to Mike Gilbert from comment #15) > This does not affect most users, and the workaround is simple: remove -ggdb > from CFLAGS. > > I think filtering it on x86 is not a bad idea. > > It's a pain in the ass for you, but that does not make this a critically > important issue. Turn it down a notch please. I didn't mean to make it sound like I was angry. However, when it is not possible to build a package on a certain arch with certain _very_ common cflags, gentoo was kind enough to provide us filter-flags to fix it for the users. I wasn't trying to be a drama queen, but if you want to know the truth, the fact that such a common cflag can't be used and isn't filtered out is embarassing, and just plain bad ebuild writing. I will wait for comment from Paweł as we agreed on irc, however, this is honestly an issue for a QA mask. Users should not spend hours waiting for an ebuild with no chance to complete just because someone doesn't care enough to add filter flags for flags which afaict can't possibly work. So please, take it up a notch.
(In reply to Mike Gilbert from comment #15) > I think filtering it on x86 is not a bad idea. How about this snippet (inside the !custom-cflags "if"): if use x86; then filter-flags "-g*" fi Julian (hasufell), Rick (zerochaos), does it fix the build for you? Note that if this gets really severe I'll consider dropping stable x86 keywords for this package. Upstream is also hitting various issues with 32-bit builds, but it can use tricks like 64-bit kernel with 32-bit userland that are not really standard on Gentoo x86.
(In reply to Paweł Hajdan, Jr. from comment #17) > (In reply to Mike Gilbert from comment #15) > > I think filtering it on x86 is not a bad idea. > > How about this snippet (inside the !custom-cflags "if"): > > if use x86; then > filter-flags "-g*" > fi > > Julian (hasufell), Rick (zerochaos), does it fix the build for you? > > Note that if this gets really severe I'll consider dropping stable x86 > keywords for this package. Upstream is also hitting various issues with > 32-bit builds, but it can use tricks like 64-bit kernel with 32-bit userland > that are not really standard on Gentoo x86. Yes I believe this should solve the issue. I am not sure if there are other -g* flags besides -g and -ggdb but we may want to be careful with that glob and specify the specific flags we don't like.
(In reply to Paweł Hajdan, Jr. from comment #17) > Julian (hasufell), Rick (zerochaos), does it fix the build for you? > hasufell's problem is slightly different -- he's on amd64, but only has 4 GB of system memory. We could possibly filter flags based on check-reqs results, but I think dying in pkg_pretend is more appropriate.
(In reply to Rick Farina (Zero_Chaos) from comment #18) > Yes I believe this should solve the issue. I am not sure if there are other > -g* flags besides -g and -ggdb but we may want to be careful with that glob > and specify the specific flags we don't like. Committed now for 28+ . Given this is on the same code path that calls strip-flags, I think this is predictably safe (also checked man gcc). Now the remaining issue is good pkg_pretend.
> Committed now for 28+ . Given this is on the same code path that calls > strip-flags, I think this is predictably safe (also checked man gcc). I'm sure I'm the only moron running "stable" but, would you mind fixing 27 as well please? :-)
(In reply to Rick Farina (Zero_Chaos) from comment #21) > I'm sure I'm the only moron running "stable" but, would you mind fixing 27 > as well please? :-) I generally avoid modifying stable ebuilds to avoid regressions even on seemingly simple changes. Feel free to use the solution from http://wiki.gentoo.org/wiki/Knowledge_Base:Overriding_environment_variables_per_package locally, I think it's just as good especially that you're now fully aware of what's going on as opposed to people who may be surprised by it.
(In reply to Paweł Hajdan, Jr. from comment #22) > I generally avoid modifying stable ebuilds to avoid regressions even on > seemingly simple changes. As it is not currently possible to build with those flags on x86, anyone affected by this has not been able to build the package which is currently marked stable. In my opinion, having a simple -g or -ggdb in CFLAGS breaking the build of a "stable" package means either that package should be masked or the stable keyword should be removed for the arch. Please don't leave user visible breakage in tree and marked STABLE.
(In reply to Julian Ospald (hasufell) from comment #3) > [138174.687107] Out of memory: Kill process 23076 (ld) score 737 or > sacrifice child > [138174.687109] Killed process 23076 (ld) total-vm:4684904kB, > anon-rss:3804424kB, file-rss:68kB > > That is with 4GB ram and no big applications running. Webkit, firefox, > libreoffice and other bloat stuff never failed like that for me. I wonder > how chromium can be stable that way. Can reproduce. I've no issues here (amd64, 4 GB RAM, -ggdb). Maybe your virtual memory is exhausted? 2 GB of swap space seems to be a bit on the low side.
(In reply to Ulrich Müller from comment #24) > (In reply to Julian Ospald (hasufell) from comment #3) > > [138174.687107] Out of memory: Kill process 23076 (ld) score 737 or > > sacrifice child > > [138174.687109] Killed process 23076 (ld) total-vm:4684904kB, > > anon-rss:3804424kB, file-rss:68kB > > > > That is with 4GB ram and no big applications running. Webkit, firefox, > > libreoffice and other bloat stuff never failed like that for me. I wonder > > how chromium can be stable that way. Can reproduce. > > I've no issues here (amd64, 4 GB RAM, -ggdb). > > Maybe your virtual memory is exhausted? 2 GB of swap space seems to be a bit > on the low side. I can reproduce this OOM on a system with 64GB of ram because x86 can only address 4GB of ram per process and the linker needs more than that.
(In reply to Rick Farina (Zero_Chaos) from comment #25) > > I've no issues here (amd64, 4 GB RAM, -ggdb). > > > > Maybe your virtual memory is exhausted? 2 GB of swap space seems to be > > a bit on the low side. > > I can reproduce this OOM on a system with 64GB of ram because x86 can only > address 4GB of ram per process and the linker needs more than that. I was talking about the configuration visible in emerge --info which is amd64. (In reply to Julian Ospald (hasufell) from comment #0) > Linux-3.8.13-gnu-ck1-x86_64-Intel-R-_Core-TM-2_CPU_6400_@_2.13GHz-with- > gentoo-2.2 > KiB Mem: 4051392 total, 3135524 free > KiB Swap: 2047996 total, 1591828 free
All versions in portage now filter -g* on x86, so this is resolved, right?
I'm okay with closing this at this time. Maintainer, on you.
(In reply to Rick Farina (Zero_Chaos) from comment #28) > I'm okay with closing this at this time. Maintainer, on you. I'd rather not miss an opportunity to close a bug. :)
(In reply to Paweł Hajdan, Jr. from comment #29) > (In reply to Rick Farina (Zero_Chaos) from comment #28301) > > I'm okay with closing this at this time. Maintainer, on you. > > I'd rather not miss an opportunity to close a bug. :) The initial bug report is not fixed.
I don't think this is going to be fixed unless/until somebody comes up with a way to somewhat reliably check the amount of virtual memory (including swap) available. Does anyone plan to work on that?
I don't think that running out of memory while compiling with debug options on is a bug. https://bugs.gentoo.org/buglist.cgi?quicksearch=ALL%20killed Look at all of those INVALID/CANTFIX/etc. Or is this turning into a major portage feature request bug?
(In reply to Mike Gilbert from comment #31) > I don't think this is going to be fixed unless/until somebody comes up with > a way to somewhat reliably check the amount of virtual memory (including > swap) available. Does anyone plan to work on that? Then it has to be WONTFIX or something similar, not FIXED.
I know how to operate bugzilla, thanks. My question still remains. If nobody answers affirmatively, then I agree we should just WONTFIX this and move on.
(In reply to Mike Gilbert from comment #34) > I know how to operate bugzilla, thanks. My question still remains. If nobody > answers affirmatively, then I agree we should just WONTFIX this and move on. Wasn't meant as an offense. I have upgraded my machine, so I can't even reproduce it anymore. I don't currently plan to work on this, maybe when I have more time or feel bored. I don't think there is any need to close this, it does not block anything. Just leave it open in case someone wants to improve it.
Honestly, since -g is filtered from cflags at this point the build won't fail on a 4GB machine and as such, I would consider this fixed...
(In reply to Rick Farina (Zero_Chaos) from comment #36) > Honestly, since -g is filtered from cflags at this point the build won't > fail on a 4GB machine and as such, I would consider this fixed... That is incorrect. It is not filtered for amd64.
As I understood the change, it was filtered on x86 because the memory addressing limitations there prevent it from working on ANY x86. On amd64, you just need to add some more memory and it will work. Get more memory or turn off debugging on massive packages like chromium. If you are turning on debugging globally, you take on some responsibility here, imho.
*** Bug 547288 has been marked as a duplicate of this bug. ***
I added a basic check, see bug #547288 : 23 Apr 2015; Pawel Hajdan jr chromium-44.0.2376.0.ebuild: Add available memory check to pkg_pretend, bug #471810 by hasufell. For now it just checks for 3GB of memory - I checked using cgroups that it's enough to build without -g* . With -g* I get 32GB from cgroups - and given that it includes filesystem cache and check-reqs doesn't check swap, I'd rather not use 32GB as the limit. Further refinements are welcome. Maybe I could e.g. try different memory cgroup limits and -g* in flags, and see which is the lowest one that still makes compile possible.
There should be a way to bypass this check through an use flag or something as you can't determine that it will fail based on installed RAM alone. I've successfully built about every release since the check was added with 2.5GB+4GB swap.
*** Bug 555386 has been marked as a duplicate of this bug. ***
(In reply to Fernando Rodriguez from comment #41) You may set I_KNOW_WHAT_I_AM_DOING=1 in the environment to bypass the check.
*** Bug 597144 has been marked as a duplicate of this bug. ***
By the way, even with regular flags (no -g) 4G RAM on amd64 is not enough anymore, with the bfd linker (I recall gold was more efficient here but haven't tested it for a while). It swaps like hell.
I have 16 gigs of RAM but some of it is used for the Intel integrated graphics so "free" shows slightly less than 16 gig and therefore this test fails. It would be nice if you changed the requirement check to 15 gig instead to fix this.
In a recent test I performed, the linker consumed more than 15 G of memory with -g in CXXFLAGS. Once again, if you want to risk it and bypass the check, simply set I_KNOW_WHAT_I_AM_DOING=1.
