Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 538028 - as: out of memory allocating 288230376151711728 bytes after a total of 272400 bytes
Summary: as: out of memory allocating 288230376151711728 bytes after a total of 272400...
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Hardened (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: The Gentoo Linux Hardened Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-28 07:52 UTC by Jocelyn Mayer
Modified: 2015-08-20 08:04 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jocelyn Mayer 2015-01-28 07:52:06 UTC
as from binutils 2.23.2 and 2.24 runs out of memory while assembling amd64 arch dependant linux kernel cryptographic helpers.
This also happens when trying to compile dev-libs/nss package.

Reproducible: Always

Steps to Reproduce:
Build a kennel:
# make V=1 bzImage

Actual Results:  
make -f scripts/Makefile.build obj=arch/x86/crypto
  gcc -Wp,-MD,arch/x86/crypto/.glue_helper.o.d  -nostdinc -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/include -I./arch/x86/include -Iarch/x86/include/generated  -Iinclude -I./arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -m64 -mno-mmx -mno-sse -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -march=k8 -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -ffreestanding -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -fno-delete-null-pointer-checks -O2 --param=allow-store-data-races=0 -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fomit-frame-pointer -fno-var-tracking-assignments -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -DCC_HAVE_ASM_GOTO -fplugin=./tools/gcc/constify_plugin.so -DCONSTIFY_PLUGIN -fplugin=./tools/gcc/stackleak_plugin.so -DSTACKLEAK_PLUGIN -fplugin-arg-stackleak_plugin-track-lowest-sp=100 -fplugin=./tools/gcc/kernexec_plugin.so -fplugin-arg-kernexec_plugin-method="bts" -DKERNEXEC_PLUGIN -fplugin=./tools/gcc/colorize_plugin.so -fplugin=./tools/gcc/size_overflow_plugin/size_overflow_plugin.so -DSIZE_OVERFLOW_PLUGIN -fplugin=./tools/gcc/latent_entropy_plugin.so -DLATENT_ENTROPY_PLUGIN -fplugin=./tools/gcc/structleak_plugin.so -DSTRUCTLEAK_PLUGIN -fplugin=./tools/gcc/randomize_layout_plugin.so -DRANDSTRUCT_PLUGIN -fplugin-arg-randomize_layout_plugin-performance-mode    -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(glue_helper)"  -D"KBUILD_MODNAME=KBUILD_STR(glue_helper)" -c -o arch/x86/crypto/glue_helper.o arch/x86/crypto/glue_helper.c
  gcc -Wp,-MD,arch/x86/crypto/.aes-x86_64-asm_64.o.d  -nostdinc -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/include -I./arch/x86/include -Iarch/x86/include/generated  -Iinclude -I./arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__  -D__ASSEMBLY__ -m64 -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1   -DKERNEXEC_PLUGIN         -c -o arch/x86/crypto/aes-x86_64-asm_64.o arch/x86/crypto/aes-x86_64-asm_64.S

/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/bin/as: out of memory allocating 288230376151711728 bytes after a total of 272400 bytes
scripts/Makefile.build:293: recipe for target 'arch/x86/crypto/aes-x86_64-asm_64.o' failed
make[2]: *** [arch/x86/crypto/aes-x86_64-asm_64.o] Error 1
scripts/Makefile.build:404: recipe for target 'arch/x86/crypto' failed
make[1]: *** [arch/x86/crypto] Error 2
Makefile:998: recipe for target 'arch/x86' failed
make: *** [arch/x86] Error 2


Expected Results:  
build a kernel image

l-0, hardened/linux/amd64/no-multilib, gcc-4.8.3, glibc-2.19-r1, 3.15.5-hardened-r1 x86_64)
=================================================================
System uname: Linux-3.15.5-hardened-r1-x86_64-AMD_Athlon-tm-_X2_Dual_Core_Processor_BE-2300-with-gentoo-2.2
KiB Mem:     1984812 total,    524440 free
KiB Swap:    2008088 total,   1944812 free
Timestamp of tree: Mon, 26 Jan 2015 04:45:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.2_p53
dev-lang/perl:            5.18.2-r2
dev-lang/python:          2.7.9-r1, 3.2.5-r6, 3.3.5-r1, 3.4.1
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.8
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6-r1, 1.12.6, 1.13.4
sys-devel/binutils:       2.22-r1, 2.23.2, 2.24-r3
sys-devel/gcc:            3.4.6-r2, 4.3.6-r1, 4.4.7, 4.5.4, 4.6.4, 4.7.3-r1, 4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.4
sys-devel/make:           4.0-r1
sys-kernel/linux-headers: 3.16 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo x-portage
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=amdfam10 -O2 -funroll-loops -pipe -fexpensive-optimizations -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=amdfam10 -O2 -funroll-loops -pipe -fexpensive-optimizations -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildsyspkg config-protect-if-modified ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://gentoo.inode.at/ http://mirror.datapipe.net/gentoo http://gentoo.ynet.sk/pub http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.easynet.nl/mirror/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/"
LANG="POSIX"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j 1"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext acl acpi adns amd64 apache2 apm bcmath berkdb branding bzip2 caps cgi cli cracklib crypt ctype cups curl curlwrappers dbus encode exif fam filecaps ftp gd gif gnutls gzip hardened hddtemp imap ipv6 jpeg jpeg2k justify lm_sensors lzma lzo maildir mbox memlimit mhash mmap mmx mmxext modules nas ncurses nntp nptl offensive pam pax_kernel pcntl pcre png posix postgres readline sasl session sharedmem skey slang smp sse sse2 sse3 ssl svg syslog sysvipc szip tcpd threads tiff truetype udev unicode urandom vhosts wifi wmf xattr xinetd xml xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

# binutils-config -l
 [1] x86_64-pc-linux-gnu-2.22
 [2] x86_64-pc-linux-gnu-2.23.2
 [3] x86_64-pc-linux-gnu-2.24 *

Note 1: this does not happen when compiling gentoo-sources on non-hardened profile
Note 2: this does not happen when compiling using binutils version 2.22
Note 3: this does not seem to depend on kernel version: tested with linux-3.15.10-hardened-r1, linux-3.15.5-hardened-r1 and linux-3.17.7-hardened-r1


* The bug also shows up when trying to merge dev-libs/nss package:

# emerge --verbose dev-libs/nss
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-libs/nss-3.17.3  USE="cacert nss-pem utils" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

[ ... ]

x86_64-pc-linux-gnu-gcc -o Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/intel-gcm.o   -fPIC -DLINUX2_1 -m64 -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -ffunction-sections -fdata-sections -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -UDEBUG -DNDEBUG -D_REENTRANT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -I./dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include -I./dist/public/ -I./dist/private/ -I./dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include/dbm -march=amdfam10 -O2 -pipe  -I/usr/include/nspr   -fPIC -DLINUX2_1 -m64 -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -ffunction-sections -fdata-sections -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -UDEBUG -DNDEBUG -D_REENTRANT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -I../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include -I../dist/public/ -I../dist/private/ -I../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include/dbm -march=amdfam10 -O2 -pipe  -I/usr/include/nspr   -fPIC -DLINUX2_1 -m64 -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -ffunction-sections -fdata-sections -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -DSHLIB_SUFFIX=\"so\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -D_REENTRANT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DNSS_USE_64 -DFREEBL_NO_DEPEND -DNSS_X86_OR_X64 -DNSS_X64 -DNSS_BEVAND_ARCFOUR -DMPI_AMD64 -DMP_ASSEMBLY_MULTIPLY -DNSS_USE_COMBA -DMP_CHAR_STORE_SLOW -DMP_IS_LITTLE_ENDIAN -DUSE_HW_AES -DINTEL_GCM -DMP_API_COMPATIBLE -I../../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include -I../../dist/public/nss -I../../dist/private/nss -I../../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include/dbm -Impi -Iecl -march=amdfam10 -O2 -pipe  -I/usr/include/nspr  -fPIC -Wa,--noexecstack  -fPIC -DLINUX2_1 -m64 -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -ffunction-sections -fdata-sections -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -DSHLIB_SUFFIX=\"so\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -D_REENTRANT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DNSS_USE_64 -DFREEBL_NO_DEPEND -DNSS_X86_OR_X64 -DNSS_X64 -DNSS_BEVAND_ARCFOUR -DMPI_AMD64 -DMP_ASSEMBLY_MULTIPLY -DNSS_USE_COMBA -DMP_CHAR_STORE_SLOW -DMP_IS_LITTLE_ENDIAN -DUSE_HW_AES -DINTEL_GCM -DMP_API_COMPATIBLE -I../../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include -I../../dist/public/nss -I../../dist/private/nss -I../../dist/Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/include/dbm -Impi -Iecl -march=amdfam10 -O2 -pipe  -I/usr/include/nspr  -fPIC -Wa,--noexecstack -c intel-gcm.s

/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/bin/as: out of memory allocating 288230376151711728 bytes after a total of 405920 bytes
../../coreconf/rules.mk:408: recipe for target 'Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/intel-gcm.o' failed
make[3]: *** [Linux2.6_x86_64_x86_64-pc-linux-gnu-gcc_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/intel-gcm.o] Error 1
make[3]: Leaving directory '/var/tmp/portage/dev-libs/nss-3.17.3/work/nss-3.17.3/nss-abi_x86_64.amd64/lib/freebl'
Makefile:565: recipe for target 'libs' failed
make[2]: *** [libs] Error 2
make[2]: Leaving directory '/var/tmp/portage/dev-libs/nss-3.17.3/work/nss-3.17.3/nss-abi_x86_64.amd64/lib/freebl'
../coreconf/rules.mk:101: recipe for target 'libs' failed
make[1]: *** [libs] Error 2
make[1]: Leaving directory '/var/tmp/portage/dev-libs/nss-3.17.3/work/nss-3.17.3/nss-abi_x86_64.amd64/lib'
coreconf/rules.mk:101: recipe for target 'libs' failed
make: *** [libs] Error 2
make: Leaving directory '/var/tmp/portage/dev-libs/nss-3.17.3/work/nss-3.17.3/nss-abi_x86_64.amd64'
 * ERROR: dev-libs/nss-3.17.3::gentoo failed (compile phase):
 *   emake failed
 *
Comment 1 Jeroen Roovers (RETIRED) gentoo-dev 2015-01-28 08:29:41 UTC
Usually you install more memory to fix problems with programs that complain about memory problems. But maybe there is a huge memory leak here - who knows?
Comment 2 Jocelyn Mayer 2015-01-28 09:46:41 UTC
this does not look like a memory leak: as tells us it tries to allocate about 288000 TB of memory at once. And the failed allocation size do not seem to be an "innocent" one, feels like a rounded negative size (-16 ?):
> printf "%016x\n" 288230376151711728
03fffffffffffff0

IMHO, for the Gentoo distribution, a bug in a stable build toolchain is a major one as the whole distribution is to be built from sources.
Comment 3 Anthony Basile gentoo-dev 2015-01-28 22:39:29 UTC
(In reply to Jocelyn Mayer from comment #2)
> this does not look like a memory leak: as tells us it tries to allocate
> about 288000 TB of memory at once. And the failed allocation size do not
> seem to be an "innocent" one, feels like a rounded negative size (-16 ?):
> > printf "%016x\n" 288230376151711728
> 03fffffffffffff0
> 
> IMHO, for the Gentoo distribution, a bug in a stable build toolchain is a
> major one as the whole distribution is to be built from sources.

I'm not sure its a bug yet.  This appears related to your bug #538026.  I'm suspecting something is seriously broken on your system.  Are you hitting both bugs on the same box?
Comment 4 Jocelyn Mayer 2015-01-29 06:56:24 UTC
(In reply to Anthony Basile from comment #3)
> (In reply to Jocelyn Mayer from comment #2)
> > this does not look like a memory leak: as tells us it tries to allocate
> > about 288000 TB of memory at once. And the failed allocation size do not
> > seem to be an "innocent" one, feels like a rounded negative size (-16 ?):
> > > printf "%016x\n" 288230376151711728
> > 03fffffffffffff0
> > 
> > IMHO, for the Gentoo distribution, a bug in a stable build toolchain is a
> > major one as the whole distribution is to be built from sources.
> 
> I'm not sure its a bug yet.  This appears related to your bug #538026.  I'm
> suspecting something is seriously broken on your system.  Are you hitting
> both bugs on the same box?

Yes, the 2 problems appear on the same box.
Comment 5 Jocelyn Mayer 2015-01-31 05:37:57 UTC
(In reply to Jocelyn Mayer from comment #4)
> (In reply to Anthony Basile from comment #3)
> > (In reply to Jocelyn Mayer from comment #2)
> > > this does not look like a memory leak: as tells us it tries to allocate
> > > about 288000 TB of memory at once. And the failed allocation size do not
> > > seem to be an "innocent" one, feels like a rounded negative size (-16 ?):
> > > > printf "%016x\n" 288230376151711728
> > > 03fffffffffffff0
> > > 
> > > IMHO, for the Gentoo distribution, a bug in a stable build toolchain is a
> > > major one as the whole distribution is to be built from sources.
> > 
> > I'm not sure its a bug yet.  This appears related to your bug #538026.  I'm
> > suspecting something is seriously broken on your system.  Are you hitting
> > both bugs on the same box?
> 
> Yes, the 2 problems appear on the same box.

Having made more investigations, it happens that bug #538026 can be reproduced on other boxes with different configuration (not hardened gentoo) while this one only shows up on my hardened gentoo server.
Then, I'm quite sure now this bug is not related to #538026.
Comment 6 Jocelyn Mayer 2015-01-31 05:50:59 UTC
In case it's useful, here are USE flags differences between the faulty box and the one(s) which do not show the bug:
Faulty box, running hardened gentoo configuration:
# equery uses sys-devel/binutils
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/binutils-2.24-r3:
 U I
 - - cxx         : Build support for C++ (bindings, extra libraries, code
                   generation, ...)
 - - multitarget : Adds support to binutils for cross compiling (does not work
                   with gas)
 - - nls         : Add Native Language Support (using gettext - GNU locale
                   utilities)
 - - static-libs : Build static versions of dynamic libraries as well
 - - test        : Workaround to pull in packages needed to run with
                   FEATURES=test. Portage-2.1.2 handles this internally, so
                   don't set it in make.conf/package.use anymore
 - - vanilla     : Do not add extra patches which change default behaviour; DO
                   NOT USE THIS ON A GLOBAL SCALE as the severity of the
                   meaning changes drastically
 + + zlib        : Add support for zlib (de)compression
# equery uses sys-devel/gcc     
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/gcc-4.8.3:
 U I
 - - awt             : Useful only when building GCJ, this enables Abstract
                       Window Toolkit (AWT) peer support on top of GTK+
 + + cxx             : Build support for C++ (bindings, extra libraries, code
                       generation, ...)
 - - doc             : Add extra documentation (API, Javadoc, etc). It is
                       recommended to enable per package instead of globally
 - - fortran         : Add support for fortran
 - - gcj             : Enable building with gcj (The GNU Compiler for the
                       Javatm Programming Language)
 - - go              : Build the GCC Go language frontend.
 - - graphite        : Add support for the framework for loop optimizations
                       based on a polyhedral intermediate representation
 + + hardened        : Activate default security enhancements for toolchain
                       (gcc, glibc, binutils)
 - - mudflap         : Add support for mudflap, a pointer use checking library
 - - nls             : Add Native Language Support (using gettext - GNU locale
                       utilities)
 - - nopie           : Disable PIE support (NOT FOR GENERAL USE)
 - - nossp           : Disable SSP support (NOT FOR GENERAL USE)
 + + nptl            : Enable support for Native POSIX Threads Library, the new
                       threading module (requires linux-2.6 or better usually)
 - - objc            : Build support for the Objective C code language
 - - objc++          : Build support for the Objective C++ language
 - - objc-gc         : Build support for the Objective C code language Garbage
                       Collector
 - - openmp          : Build support for the OpenMP (support parallel
                       computing), requires >=sys-devel/gcc-4.2 built with
                       USE="openmp"
 - - regression-test : Run the testsuite and install the results (requires
                       FEATURES=test)
 - - sanitize        : Build support various sanitizer functions
                       (ASAN/TSAN/etc...)
 - - vanilla         : Do not add extra patches which change default behaviour;
                       DO NOT USE THIS ON A GLOBAL SCALE as the severity of the
                       meaning changes drastically
# equery uses sys-libs/glibc
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-libs/glibc-2.19-r1:
 U I
 - - debug     : When USE=hardened, allow fortify/stack violations to dump core
                 (SIGABRT) and not kill self (SIGKILL)
 + + gd        : build memusage and memusagestat tools
 + + hardened  : Activate default security enhancements for toolchain (gcc,
                 glibc, binutils)
 - - nscd      : Build, and enable support for, the Name Service Cache Daemon
 - - profile   : Add support for software performance analysis (will likely
                 vary from ebuild to ebuild)
 - - suid      : Make internal pt_chown helper setuid -- not needed if using
                 Linux and have /dev/pts mounted with gid=5
 - - systemtap : enable systemtap static probe points
 - - vanilla   : Do not add extra patches which change default behaviour; DO
                 NOT USE THIS ON A GLOBAL SCALE as the severity of the meaning
                 changes drastically


Box where the kernel and nss library build without any problem:
# equery uses sys-devel/binutils
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/binutils-2.24-r3:
 U I
 + + cxx         : Build support for C++ (bindings, extra libraries, code
                   generation, ...)
 + + multitarget : Adds support to binutils for cross compiling (does not work
                   with gas)
 + + nls         : Add Native Language Support (using gettext - GNU locale
                   utilities)
 + + static-libs : Build static versions of dynamic libraries as well
 - - test        : Workaround to pull in packages needed to run with
                   FEATURES=test. Portage-2.1.2 handles this internally, so
                   don't set it in make.conf/package.use anymore
 - - vanilla     : Do not add extra patches which change default behaviour; DO
                   NOT USE THIS ON A GLOBAL SCALE as the severity of the
                   meaning changes drastically
 + + zlib        : Add support for zlib (de)compression
# equery uses sys-devel/gcc     
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/gcc-4.8.3:
 U I
 - - awt             : Useful only when building GCJ, this enables Abstract
                       Window Toolkit (AWT) peer support on top of GTK+
 + + cxx             : Build support for C++ (bindings, extra libraries, code
                       generation, ...)
 + + doc             : Add extra documentation (API, Javadoc, etc). It is
                       recommended to enable per package instead of globally
 + + fortran         : Add support for fortran
 - - gcj             : Enable building with gcj (The GNU Compiler for the
                       Javatm Programming Language)
 - - go              : Build the GCC Go language frontend.
 + + graphite        : Add support for the framework for loop optimizations
                       based on a polyhedral intermediate representation
 - - mudflap         : Add support for mudflap, a pointer use checking library
 + + nls             : Add Native Language Support (using gettext - GNU locale
                       utilities)
 - - nopie           : Disable PIE support (NOT FOR GENERAL USE)
 - - nossp           : Disable SSP support (NOT FOR GENERAL USE)
 + + nptl            : Enable support for Native POSIX Threads Library, the new
                       threading module (requires linux-2.6 or better usually)
 + + objc            : Build support for the Objective C code language
 + + objc++          : Build support for the Objective C++ language
 - - objc-gc         : Build support for the Objective C code language Garbage
                       Collector
 + + openmp          : Build support for the OpenMP (support parallel
                       computing), requires >=sys-devel/gcc-4.2 built with
                       USE="openmp"
 - - regression-test : Run the testsuite and install the results (requires
                       FEATURES=test)
 + + sanitize        : Build support various sanitizer functions
                       (ASAN/TSAN/etc...)
 - - vanilla         : Do not add extra patches which change default behaviour;
                       DO NOT USE THIS ON A GLOBAL SCALE as the severity of the
                       meaning changes drastically
# equery uses sys-libs/glibc
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-libs/glibc-2.19-r1:
 U I
 - - debug     : When USE=hardened, allow fortify/stack violations to dump core
                 (SIGABRT) and not kill self (SIGKILL)
 + + gd        : build memusage and memusagestat tools
 - - nscd      : Build, and enable support for, the Name Service Cache Daemon
 - - profile   : Add support for software performance analysis (will likely
                 vary from ebuild to ebuild)
 - - suid      : Make internal pt_chown helper setuid -- not needed if using
                 Linux and have /dev/pts mounted with gid=5
 - - systemtap : enable systemtap static probe points
 - - vanilla   : Do not add extra patches which change default behaviour; DO
                 NOT USE THIS ON A GLOBAL SCALE as the severity of the meaning
                 changes drastically
Comment 7 Anthony Basile gentoo-dev 2015-02-01 02:07:39 UTC
(In reply to Jocelyn Mayer from comment #0)
> as from binutils 2.23.2 and 2.24 runs out of memory while assembling amd64
> arch dependant linux kernel cryptographic helpers.
> This also happens when trying to compile dev-libs/nss package.
> 
> Reproducible: Always

This one, I can't reproduce.  I'm not sure how you're getting into this state.  I gather from your previous comments that this is the only box where you're getting the strange memory allocation.
Comment 8 Anthony Basile gentoo-dev 2015-02-02 11:32:06 UTC
(In reply to Anthony Basile from comment #7)
> (In reply to Jocelyn Mayer from comment #0)
> > as from binutils 2.23.2 and 2.24 runs out of memory while assembling amd64
> > arch dependant linux kernel cryptographic helpers.
> > This also happens when trying to compile dev-libs/nss package.
> > 
> > Reproducible: Always
> 
> This one, I can't reproduce.  I'm not sure how you're getting into this
> state.  I gather from your previous comments that this is the only box where
> you're getting the strange memory allocation.

Still trying to reproduce --- can I have your kernel config file so I'm sure I'm doing the same thing here as you are there.
Comment 9 Anthony Basile gentoo-dev 2015-02-06 13:10:45 UTC
(In reply to Anthony Basile from comment #8)
> (In reply to Anthony Basile from comment #7)
> > (In reply to Jocelyn Mayer from comment #0)
> > > as from binutils 2.23.2 and 2.24 runs out of memory while assembling amd64
> > > arch dependant linux kernel cryptographic helpers.
> > > This also happens when trying to compile dev-libs/nss package.
> > > 
> > > Reproducible: Always
> > 
> > This one, I can't reproduce.  I'm not sure how you're getting into this
> > state.  I gather from your previous comments that this is the only box where
> > you're getting the strange memory allocation.
> 
> Still trying to reproduce --- can I have your kernel config file so I'm sure
> I'm doing the same thing here as you are there.

Last ping before closing this need info.
Comment 10 Anthony Basile gentoo-dev 2015-02-07 18:49:46 UTC
Reopen when you get more info.
Comment 11 Jocelyn Mayer 2015-08-20 08:04:12 UTC
I finally found and solved this issue.
The point was that CFLAGS with -march=amdfam10 is not correct anymore for this specific CPU (_X2_Dual_Core_Processor_BE-2300). It happens to be -march=k8-sse3 with recent gcc, missing SSE4a and ABM instruction sets; the difference in CPU options tuning is then -mno-popcnt -mno-abm -mno-lzcnt.
It seems like this CPU implements (some of) the missing instructions but not giving the correct result.
As a consequence, there was a problem in zlib which made as do weird things while compiling kernel and dev-libs/nss package. Recompiling zlib with -march=native solved the issue.
Then I did recompile the whole system changing my CFLAGS from
CFLAGS="-march=amdfam10 -O2 -funroll-loops -pipe -fexpensive-optimizations -fomit-frame-pointer"
to
CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
in order to be sure no other package is affected by this issue (doing emerge --verbose --empty @system ; emerge --verbose --empty @world)

Thanks for your help.