Bug 224031 - Building net-p2p/rtorrent-0.8.2 with gcc-3.4.6 result in immediate memory saturation
|
Bug#:
224031
|
Product: Gentoo Linux
|
Version: unspecified
|
Platform: All
|
|
OS/Version: Linux
|
Status: RESOLVED
|
Severity: critical
|
Priority: P2
|
|
Resolution: FIXED
|
Assigned To: hardened@gentoo.org
|
Reported By: kerframil@gmail.com
|
|
Component: Hardened
|
|
|
URL:
|
|
Summary: Building net-p2p/rtorrent-0.8.2 with gcc-3.4.6 result in immediate memory saturation
|
|
Keywords: Bug
|
|
Status Whiteboard:
|
|
Opened: 2008-05-28 23:24 0000
|
As per summary:
1) emerge =rtorrent-0.8.2 (which is a C++ application)
2) Watch in horror as one's available RAM and swap are saturated within
mere seconds and the OOM starts shooting down everything in sight.
Confirmed reproducible using gcc-3.4.6-r2 (with any available specs) on x86 and
amd64. gcc-4.1.2 confirmed as working in a standard 2008.0_beta2 chroot on the
same system.
Strictly speaking, this probably isn't a hardened bug at all but I think it
belongs with the herd as, let's face it, no-one else cares much about gcc-3*
anymore.
Same here:
Portage 2.1.4.4 (hardened/amd64/multilib, gcc-3.4.6, glibc-2.6.1-r0,
2.6.23-hardened-r12 x86_64)
=================================================================
System uname: 2.6.23-hardened-r12 x86_64 AMD Athlon(tm) 64 X2 Dual Core
Processor 5600+
Timestamp of tree: Wed, 28 May 2008 23:45:01 +0000
app-shells/bash: 3.2_p33
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python: 2.4.4-r13
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox: 1.2.18.1-r2
sys-devel/autoconf: 2.61-r1
sys-devel/automake: 1.5, 1.7.9-r1, 1.9.6-r2, 1.10.1
sys-devel/binutils: 2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool: 1.5.26
virtual/os-headers: 2.6.23-r3
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf
/etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/
/etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo
/etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer noinfo parallel-fetch sandbox
sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://mirror.q-mex.net/gentoo/
http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/
http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/"
LC_ALL="en_US.UTF-8"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --stats --timeout=180 --exclude=/distfiles
--exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/webapps-experimental
/usr/portage/local/local"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="7zip aim amd64 authdaemond bash-completion berkdb bitmap-fonts bzip2 caps
cgi chroot cli cracklib crypt cscope ctype curl curlwrappers dynamicplugin
encode expat fam fastcgi fasttrack flatfile ftp gd gdbm geoip gif gnutella
gnutls gpgme hardened iconv idn imagemagick imap iproute2 ithreads jpeg jpeg2k
kqemu libg++ libwww lighttpd logrotate maildir mailwrapper mime mng mudflap
multilib mysql ncurses nls nptl nptlonly ntlm offensive ogg pam pcre pdf perl
php pic png pop python readline reflection rrdtool ruby sasl session slang
smime smtp snmp socks5 spell spl sse sse2 ssl svg sysfs tcpd theora threads
tidy tiff tordns truetype truetype-fonts type1-fonts unicode userlocales vhosts
vim x264 xml xvid zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem
bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel
intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem
ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug
file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate
route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias
authn_anon authn_dbm authn_default authn_file authz_dbm authz_default
authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs
dav_lock deflate dir disk_cache env expires ext_filter file_cache filter
headers include info log_config logio mem_cache mime mime_magic negotiation
rewrite setenvif speling status unique_id userdir usertrack vhost_alias"
ELIBC="glibc" INPUT_DEVICES="mouse keyboard" KERNEL="linux" LCD_DEVICES="bayrad
cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU"
VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i810 mach64
mga neomagic nv r128 radeon rendition s3 s3virge savage siliconmotion sis
sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LDFLAGS,
LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
CCing Timothy Redaelli here as he is listed as a maintainer in metadata.xml.
I think package.masking net-p2p/rtorrent-0.8.2 at least on hardened profiles
would be appropriate in the mean time. Will save users from this unhappy
surprise.
Yes, either that or check gcc-major-version in the ebuild itself until we
figure out just what to do. I don't believe that the problem is connected with
the hardened toolchain; I'll seek to confirm one way or the other at some point
today.
*** This bug has been marked as a duplicate of bug 216118 ***
This is not a duplicate of bug 216118 (which is really solved), re-opening.
Ok, my mistake. Using MAKEOPTS="-j1" The build does appear to be failing on
dht_manager.cc as in bug 216118. However, that bug is against an SVN version,
not against a version in portage as this bug is. Also compiling the SVN
version stated there results in immediate segfault without the memory
saturation problem, so the failure is different. For these reasons that bug
should be marked a dupe of this one IMO.
make[3]: Entering directory
`/var/tmp/portage/net-p2p/rtorrent-0.8.2/work/rtorrent-0.8.2/src/core'
i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../.. -I. -I./.. -I../..
-march=athlon64 -msse3 -O2 -fweb -fno-ident -pipe -fno-strict-aliasing -DNDEBUG
-I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/include -c -o
curl_get.o curl_get.cc
i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../.. -I. -I./.. -I../..
-march=athlon64 -msse3 -O2 -fweb -fno-ident -pipe -fno-strict-aliasing -DNDEBUG
-I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/include -c -o
curl_stack.o curl_stack.cc
i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../.. -I. -I./.. -I../..
-march=athlon64 -msse3 -O2 -fweb -fno-ident -pipe -fno-strict-aliasing -DNDEBUG
-I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/include -c -o
dht_manager.o dht_manager.cc
In file included from ./../rpc/command_map.h:45,
from ./../rpc/parse_commands.h:43,
from dht_manager.cc:46:
./../rpc/command.h: In function `rpc::target_type rpc::get_target_left(const
rpc::target_type&)':
./../rpc/command.h:153: internal compiler error: in type_unification_real, at
cp/pt.c:9134
Please submit a full bug report,
...
*** Bug 216118 has been marked as a duplicate of this bug. ***
(In reply to comment #6)
> [...] Also compiling the SVN version stated there results in immediate
> segfault without the memory saturation problem, so the failure is different.
> For these reasons that bug should be marked a dupe of this one IMO.
once again:
when using the custom ebuild or the make command (even on an svn source), it
does not throw segfault, it starts eating up all the memory and the swapfile,
until all of it is gone [...]
so i still think, it's the same bug, and has nothing to do where do i download
the source from (svn, tar.gz from the official libtorrent page, or a gentoo
mirror)
Well in that case I apologize. That was not the behavior I observed when I
tried it so perhaps my observations were wrong. Regardless this bug has more
useful build output, more complete CC list, possible temporary courses of
action and since it is actually IN portage seems to be now getting attention
where as the last one received almost none outside myself who reproduced it
shortly after you reported it.
This is p.masked in the hardened profile for now.
libtorrent 0.12.0 should also be masked on hardened, since the good old
rtorrent 0.7.9 won't compile if this one is installed.
Was rtorrent-0.8.0 removed for this reason as well? I went to rebuild rtorrent,
and after masking 0.8.2*, it's going back to 0.7.9. I had 0.8.0 working fine,
but I have to go back to 0.7.9 now because it's not available anymore, and I'm
still getting 0.8.2-r2 in my list, which doesn't build still.
With today's sync updates brings libtorrent-0.12.2-r3, curl-7.18.2 and
rtorrent-0.8.2-r3.
An emerge --update --deep world results in rtorrent's 0.8.2-r3 a failed build.
emerge --info:
Portage 2.1.4.4 (hardened/linux/x86, gcc-3.4.6, glibc-2.6.1-r0,
2.6.24-hardened-r3 i686)
=================================================================
System uname: 2.6.24-hardened-r3 i686 Intel(R) Pentium(R) M processor 1300MHz
Timestamp of tree: Wed, 30 Jul 2008 07:45:03 +0000
app-shells/bash: 3.2_p33
dev-lang/python: 2.5.2-r5
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox: 1.2.18.1-r2
sys-devel/autoconf: 2.61-r2
sys-devel/automake: 1.10.1
sys-devel/binutils: 2.18-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool: 1.5.26
virtual/os-headers: 2.6.23-r3
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium-m -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo
/etc/udev/rules.d"
CXXFLAGS="-O2 -march=pentium-m -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache collision-protect distlocks metadata-transfer parallel-fetch
sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://cesium.di.uminho.pt/pub/gentoo/
ftp://cesium.di.uminho.pt/pub/gentoo/ http://ftp.dei.uc.pt/pub/linux/gentoo/
ftp://ftp.dei.uc.pt/pub/linux/gentoo/ "
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --stats --timeout=180 --exclude=/distfiles
--exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/sunrise"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="acpi bash-completion berkdb cli cracklib crypt dbus dri gpm hal hardened
iconv midi mudflap ncurses nptl nptlonly openmp pam pcre perl pic pppd python
readline reflection samba session spl ssl symlink tcpd thread threads truetype
type1 unicode urandom x86 xorg zlib" ALSA_CARDS="intel8x0 intel8x0m"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file
hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route
share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias
authn_anon authn_dbm authn_default authn_file authz_dbm authz_default
authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs
dav_lock deflate dir disk_cache env expires ext_filter file_cache filter
headers include info log_config logio mem_cache mime mime_magic negotiation
rewrite setenvif speling status unique_id userdir usertrack vhost_alias"
ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev synaptics" KERNEL="linux"
LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses
text" USERLAND="GNU" VIDEO_CARDS="radeon"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LINGUAS,
PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
>=net-p2p/rtorrent-0.8.2 is package.masked on hardened profiles until someone adds a check for >=gcc-4 such as that in media-video/cinelerra. If you unmask it, as it appears you have, that is NOP.
My system is _not_ hardened (amd64, gcc 3.4.6) and I get this saturation issue.
Daniel
Thanks Daniel. We are aware that this is a bug in the affected version of gcc
and that it doesn't specifically apply to the hardened toolchain. However, as I
alluded to in my original report, it is unrealistic to expect anyone else
outside of the hardened herd to be at all concerned. The silence from the
toolchain herd and the package maintainer is unlikely to be broken.
Therefore, unless a better solution can be found, and as gcc-3.4.6-r2 is still
the "supported" compiler in Hardened Gentoo, it seems that the only way to get
this bug truly fixed is for the herd to either fix the existing compiler or
provide a supported gcc-4.x release (whichever comes first).
(In reply to comment #16)
> >=net-p2p/rtorrent-0.8.2 is package.masked on hardened profiles until someone adds a check for >=gcc-4 such as that in media-video/cinelerra. If you unmask it, as it appears you have, that is NOP.
>
The package is not masked on hardened profiles. That is why I had to mask it
alongside with libtorrent-0.12.2-r3 to able to emerge rtorrent.
(In reply to comment #19)
> (In reply to comment #16)
> > >=net-p2p/rtorrent-0.8.2 is package.masked on hardened profiles until someone adds a check for >=gcc-4 such as that in media-video/cinelerra. If you unmask it, as it appears you have, that is NOP.
> >
>
> The package is not masked on hardened profiles. That is why I had to mask it
> alongside with libtorrent-0.12.2-r3 to able to emerge rtorrent.
>
I should clarify.
# eselect profile show
Current make.profile symlink:
/usr/portage/profiles/hardened/linux/x86
# cat /etc/portage/package.mask
>=net-p2p/rtorrent-0.8.2-r3
>=net-libs/libtorrent-0.12.2-r3
>=net-misc/curl-7.18.2
By masking as above I compile the packages without problems.
Here's the default eix output for rtorrent:
# eix rtorrent
[U] net-p2p/rtorrent
Available versions: 0.7.9 0.8.2-r3 {debug ipv6 xmlrpc}
Installed versions: 0.7.9(01:36:46 PM 07/30/2008)(-debug -xmlrpc)
Homepage: http://libtorrent.rakshasa.no/
Description: BitTorrent Client using libtorrent
Hope that helps.
Thanks Gordon. :)
Wasn't aware of the profile issue.
+ 01 Aug 2008; Peter Alfredsen <loki_val@gentoo.org>
+ +files/rtorrent-0.8.2-gcc34.patch, rtorrent-0.8.2-r3.ebuild:
+ Add patch for gcc-3.4, bug #224031. Thanks goes to Herbie Hopkins
+ <herbie@hopkins.net>. Upstream http://libtorrent.rakshasa.no/ticket/1271 .
+ Verified with Josef Drexler via IRC that this is the correct solution.
+
It should work for you guys now too. It works here on ~x86 w/ gcc-4.3.1-r1.
Please verify and unmask as appropriate. Leaving status as it is for the same
purpose.
To both Peter and Herbie, many thanks indeed. The patch in Comment 21 does
indeed solve the problem (tested on an Opteron system here). I just wish I knew
how this could be addressed in the compiler itself.
It sounds like this is fixed then? Reopen if not.
Works for me on hardened. Somebody needs to unmask it (I had to add it to
package.unmask today).
(In reply to comment #27)
> Works for me on hardened. Somebody needs to unmask it..
Done.
Bug is back. Looks like ${FILESDIR}/${PN}-0.8.2-gcc34.patch was dropped from
rtorrent-0.8.5 revbump.
Adding patrick to CC, 0.8.5 was his revbump.
+ 27 Oct 2009; Patrick Lauer <patrick@gentoo.org> rtorrent-0.8.5.ebuild:
+ Readding 0.8.2-gcc34.patch for #224031
Sorry, since the gcc44 patch didn't apply I removed both.
And since that worked quite well for me ... oops.