Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 34705 - emerging perl (during emerge system) eats up all main memory and is killed by an out-of-memory exception during make tests phase of perl build
Summary: emerging perl (during emerge system) eats up all main memory and is killed by...
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: x86 Linux
: High critical (vote)
Assignee: Gentoo Perl team
URL:
Whiteboard:
Keywords:
: 54835 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-11-29 09:16 UTC by Heiko Wundram
Modified: 2004-10-28 08:41 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Heiko Wundram 2003-11-29 09:16:39 UTC
When emerging perl-5.8.0-r??, the system slows down continuously while testing 
and lib/Config eats up more and more memory, resulting in the killing of user 
processes, and finally in the killing of the actual lib/Config test. 
 
lib/Config isn't the only test which exhibits this behaviour. If make tests is allowed to 
run further (isn't killed by the kernel which tries to handle OOM), the compilation 
freezes the computer on one of the Extutils/* tests. 
 
This error occurs while emerging perl during an emerge system (but also if emerged 
seperately) when running the emerge process inside a chroot environment on a 
working Gentoo system (Live-CD or working installation) when starting from a stage1 
build. 
 
This error occurs once in a while, depending somehow on the underlying stage1 
build, as rerunning the build from stage1 and then doing another stage2 emerge 
system solves the problem sometimes (without doing anything differently during the 
emerging of stage1). 
 
Hardware errors of this computer have been checked more than once in result to 
this error, and nothing seems to be obviously broken (at least as reported by 
memtest86). Total memory of this computer (including swap) is 1.5GB. 
 
Output of the actual emerge cannot be pasted, as the computer freezes in the 
process of the next test (Extutils/*). 

Reproducible: Sometimes
Steps to Reproduce:
1. Start a stage1 build on a fresh stage1 tarball (x86) 
2. Start scripts/bootstrap.sh 
3. Start emerge system. 
Actual Results:  
The kernel starts killing user-processes to free memory for the lib/Config test, which 
is invariably killed itself when memory requirements can't be satisfied. 

Expected Results:  
lib/Config test succeeds and the perl build completes successfully. 

Portage 2.0.49-r15 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r3, 2.4.22-hw1) 
================================================================= 
System uname: 2.4.22-hw1 i686 AMD Athlon(tm) XP 2400+ 
Gentoo Base System version 1.4.3.10 
ccache version 2.3 [enabled] 
ACCEPT_KEYWORDS="x86" 
AUTOCLEAN="yes" 
CFLAGS="-O3 -march=athlon -funroll-loops -pipe" 
CHOST="i686-pc-linux-gnu" 
COMPILER="gcc3" 
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/
kde/3.1/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/tex/
generic/config/ /usr/share/texmf/tex/platex/config/ /var/bind /var/qmail/control" 
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" 
CXXFLAGS="-O3 -march=athlon -funroll-loops -pipe" 
DISTDIR="/usr/portage/distfiles" 
FEATURES="autoaddcvs ccache sandbox" 
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror 
http://mirrors.sec.informatik.tu-darmstadt.de/gentoo http://ftp.uni-erlangen.de/pub/
mirrors/gentoo" 
MAKEOPTS="-j2" 
PKGDIR="/usr/portage/packages" 
PORTAGE_TMPDIR="/var/tmp" 
PORTDIR="/usr/portage" 
PORTDIR_OVERLAY="/usr/local/portage" 
SYNC="rsync://rsync.gentoo.org/gentoo-portage" 
USE="3dnow X Xaw3d aalib acl acpi alsa apache2 apm arts avi berkdb bonobo 
cdr crypt cups curl dvd dvdr emacs emacs-w3 encode esd ethereal evo fastcgi 
foomaticdb freetds gd gdbm gif gnome gpm gtk gtk2 gtkhtml guile imap imlib innodb 
ipv6 java joystick jpeg kde lcms ldap leim libg++ libwww lirc mad maildir mbox mcal 
mikmod mmx motif mozilla mpeg mule mysql ncurses nls oggvorbis opengl oss pam 
pda pdflib perl plotutils png postgres prelude python qt quicktime readline ruby 
samba sasl scanner sdl slang spell sse ssl svga tcltk tcpd tetex tiff truetype usb 
videos wmf wxwindows x86 xml2 xmms xv zlib" 
 
If compiled in chroot on a running system, the host portage gives the following 
output: 
 
Portage 2.0.49-r18 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r3, 2.4.22-hw1) 
================================================================= 
System uname: 2.4.22-hw1 i686 AMD Athlon(tm) XP 2400+ 
Gentoo Base System version 1.4.3.10 
ccache version 2.3 [enabled] 
ACCEPT_KEYWORDS="x86" 
AUTOCLEAN="yes" 
CFLAGS="-O3 -march=athlon-xp -funroll-loops -pipe" 
CHOST="i686-pc-linux-gnu" 
COMPILER="gcc3" 
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/
kde/3.1/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/tex/
generic/config/ /usr/share/texmf/tex/platex/config/ /var/bind /var/qmail/control" 
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" 
CXXFLAGS="-O3 -march=athlon-xp -funroll-loops -pipe" 
DISTDIR="/usr/portage/distfiles" 
FEATURES="autoaddcvs ccache sandbox" 
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror 
http://mirrors.sec.informatik.tu-darmstadt.de/gentoo http://ftp.uni-erlangen.de/pub/
mirrors/gentoo" 
MAKEOPTS="-j2" 
PKGDIR="/usr/portage/packages" 
PORTAGE_TMPDIR="/var/tmp" 
PORTDIR="/usr/portage" 
PORTDIR_OVERLAY="/usr/local/portage" 
SYNC="rsync://rsync.gentoo.org/gentoo-portage" 
USE="3dnow X Xaw3d aalib acl acpi alsa apache2 apm arts avi berkdb bonobo 
cdr crypt cups curl dvd dvdr emacs emacs-w3 encode esd ethereal evo fastcgi 
foomaticdb freetds gd gdbm gif gnome gpm gtk gtk2 gtkhtml guile imap imlib innodb 
ipv6 java joystick jpeg kde lcms ldap leim libg++ libwww lirc mad maildir mbox mcal 
mikmod mmx motif mozilla mpeg mule mysql ncurses nls oggvorbis opengl oss pam 
pda pdflib perl plotutils png postgres prelude python qt quicktime readline ruby 
samba sasl scanner sdl slang spell sse ssl svga tcltk tcpd tetex tiff truetype usb 
videos wmf wxwindows x86 xml2 xmms xv zlib" 
 
Computer: 
AMD Athlon XP 2400+ (or 1900+, a different machine exhibiting the same problems) 
512 MB RAM 
1 GB swap 
WD 180GB harddisk 
NVIDIA GeForce4 Ti 4200 
Asrock K7VT2, Bios 1.30 mainboard
Comment 1 Robert Coie (RETIRED) gentoo-dev 2003-11-29 22:25:25 UTC
I have never seen or heard of this before.  If you want to disable the tests so
that you can continue, it should be possible to comment that section out of the
ebuild.
Comment 2 Heiko Wundram 2003-11-30 02:50:01 UTC
This bug seems so strange to me too, that I invariably thought that this bug must be due to a hardware failure, but I reran tests on all hardware again tonight, and there are none. I can reproduce this bug on pretty much every machine that I have access to.

I've yet to try and see whether the bug goes away if changing the optimization settings of perl (-O3 shouldn't be the problem, but -march=athlon-xp or athlon maybe is, as several other programs start to segfault on exit too if built using a compiler which was optimized using -march=athlon-xp).
Comment 3 Heiko Wundram 2003-11-30 09:13:31 UTC
Updated info, equal behaviour if the following emerge base system is used. Equal results:

Portage 2.0.49-r15 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r3, 2.4.22-hw1)
=================================================================
System uname: 2.4.22-hw1 i686 AMD Athlon(tm) XP 2400+
Gentoo Base System version 1.4.3.10
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-Os -mcpu=athlon-xp -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER="gcc3"
CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config /usr/kde/3/share/config /var/bind /usr/X11R6/lib/X11/xkb /usr/kde/3.1/share/config /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/config"
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"
CXXFLAGS="-Os -mcpu=athlon-xp -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs sandbox ccache"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 oss apm arts avi crypt cups encode foomaticdb gif gpm gtk imlib jpeg kde gnome libg++ libwww mad mikmod motif mpeg ncurses oggvorbis opengl pam pdflib png qt quicktime sdl slang ssl svga tcpd truetype X xml2 xmms xv zlib gdbm berkdb readline 3dnow acl acpi apache2 cdr dvd dvdr emacs emacs-w3 ethereal fam imap ipv6 java ldap maildir mbox memlimit mmx mozilla mule nls perl postgres python ruby samba sasl spell sse tcltk tetex wxwindows"
Comment 4 Robert Coie (RETIRED) gentoo-dev 2003-12-16 15:17:06 UTC
For the record, I've built perl literally hundreds of times on systems with
either -march=athlon or -march=athlon-xp, and never seen anything like this.
Comment 5 Aled Gest 2003-12-21 04:20:40 UTC
Hi, i experienced simular problems while doing an emerge -u system on my gentoo box.

While building libperl 5.8.2 it would spawn 1000s of make processes which would result in all memory being eaten up.

After some investigation it turns out that my system date had somehow been set back to a few months previous and because of this the files in the perl tar archive were dated in the future (compared to my system date).

Once i had set the system date to the correct value it fixed the problem and perl emerged succesfully.

Regards,
Aled
Comment 6 Robert Coie (RETIRED) gentoo-dev 2004-01-26 10:52:45 UTC
Is it possible that the system date was the problem?  Still unable to reproduce
this.
Comment 7 Michael Cummings (RETIRED) gentoo-dev 2004-02-27 08:22:32 UTC
Please reopen this bug if it persists with perl 5.8.2 (inactive for a month now)
Comment 8 Heiko Wundram 2004-02-28 04:00:16 UTC
Problem has disappeared with Perl 5.8.2, I can confirm this.
Comment 9 Robert Coie (RETIRED) gentoo-dev 2004-06-22 23:44:37 UTC
*** Bug 54835 has been marked as a duplicate of this bug. ***
Comment 10 Ryan Earl 2004-06-23 00:36:27 UTC
Well, hell, this is an old bug?  Well, it's back.  If you're gonna close my bug, please reopen this one!  It's back!

My question is: Why now?  I don't know what changed to 'cause this.
Comment 11 Ryan Earl 2004-06-23 00:38:29 UTC
Portage 2.0.50-r8 (gcc34-amd64-2004.1, gcc-3.4.0, glibc-2.3.4.20040605-r0, 2.6.5-gentoo-r1)
=================================================================
System uname: 2.6.5-gentoo-r1 x86_64 4
Gentoo Base System version 1.4.16
Autoconf: sys-devel/autoconf-2.59-r3
Automake: sys-devel/automake-1.8.3
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CFLAGS="-march=k8 -O3 -pipe"
CHOST="x86_64-pc-linux-gnu"
COMPILER="gcc3"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=k8 -O3 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache sandbox"
GENTOO_MIRRORS="http://gentoo.oregonstate.edu http://distro.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl acpi amd64 apache2 apm arts avi crypt dedicated divx4linux doc dvb encode flash foomaticdb gd gdbm geoip gif gmp gpm gtk2 imap imlib innodb jpeg ldap libwww maildir mailwrapper mikmod motif mpeg mysql ncurses nls oggvorbis oss pam pdflib perl pic pie png python quicktime readline samba sasl sdl slang snmp speex spell ssl tcpd truetype vda vhosts xml2 xv zlib"

How do you create a list of all packages you have installed on your system and their versions?
Comment 12 Kai Ruhnau 2004-08-06 07:31:12 UTC
Portage 2.0.50-r9 (default-x86-1.4, gcc-3.3.3, glibc-2.3.3.20040420-r0, 2.6.1-gentoo-r1)
=================================================================
System uname: 2.6.1-gentoo-r1 i686 Intel(R) Xeon(TM) CPU 2.40GHz
Gentoo Base System version 1.4.16
distcc 2.13 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled]
Autoconf: sys-devel/autoconf-2.59-r3
Automake: sys-devel/automake-1.8.3
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O3 -march=pentium4 -funroll-loops -pipe -fomit-frame-pointer -ffast-math"
CHOST="i686-pc-linux-gnu"
COMPILER="gcc3"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O3 -march=pentium4 -funroll-loops -pipe -fomit-frame-pointer -ffast-math"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache distcc sandbox"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="apache apache2 apm arts avi berkdb bindist crypt cups curl encode flash foomaticdb gd gdbm gif gtk2 imap imlib innodb java jpeg lcms ldap libg++ libwww mad mikmod motif mpeg mysql ncurses nls oggvorbis opengl oss pam pdflib perl php png postfix python quicktime readline samba sdl slang slp spell ssl tcpd tetex tiff truetype wmf x86 xml xml2 xmms xv zlib"

Same Problem here
Comment 13 Kai Ruhnau 2004-08-27 05:58:53 UTC
I just tested to reemerge the already installed perl-5.8.2-r1.
It didn't work neither.

Also disabling all CFLAGS had no effect on the test-process.
Comment 14 Ryan Earl 2004-09-01 05:12:34 UTC
I still get this problem as well.  Can you at least add a use flag to not do the Perl tests?
Comment 15 Alec Shaner 2004-10-07 11:26:39 UTC
I am also getting this bug trying to emerge perl-5.8.4-r1 - lib/Config test eats all memory and eventually kills the system. This has not happened on any previous versions of Perl

My emerge info:

Portage 2.0.50-r11 (default-x86-1.4, gcc-3.3.4, glibc-2.3.3.20040420-r1, 2.6.8-ck8)
=================================================================
System uname: 2.6.8-ck8 i686 Intel(R) Pentium(R) 4 CPU 2.40GHz
Gentoo Base System version 1.4.16
distcc 2.13 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled]
Autoconf: sys-devel/autoconf-2.59-r4
Automake: sys-devel/automake-1.8.5-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium4 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium4 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache distcc sandbox"
GENTOO_MIRRORS="http://mirrors.tds.net/gentoo"
MAKEOPTS="-j6"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X aalib alsa apache2 apm arts avi berkdb bitmap-fonts cdr crypt cups encode esd foomaticdb gdbm gif gpm gtk gtk2 imlib java jpeg kde libwww motif mozilla mpeg mysql ncurses nls oggvorbis opengl oss pam pdflib perl png python qt quicktime readline sdl slang spell ssl svga tcpd truetype vhosts x86 xml2 xmmsxprint xv zlib"
Comment 16 Robert Coie (RETIRED) gentoo-dev 2004-10-07 15:14:35 UTC
Can you make sure /dev/null exists and is writable?
Comment 17 Ryan Earl 2004-10-08 02:14:42 UTC
I can confirm, /dev/null does exist and is writable by all.  It always memleaks on me in the tests.
Comment 18 Alec Shaner 2004-10-08 14:21:56 UTC
I have also confirmed that /dev/null exists and is writable

bash-2.05b$ ls -l /dev/null
crwxrwxrwx  1 root root 1, 3 Dec 31  1969 /dev/null

bash-2.05b$ echo test > /dev/null
Comment 19 Kai Ruhnau 2004-10-28 08:41:28 UTC
Compiling with gcc 3.4.2 solved this problem. At least for me.
Since gcc 3.4.2 is installed in an own slot it sounds safe to emerge it, change to it and then emerge / update perl.

HTH