When using USE="strong-optimization" eix gets a huge memroy leak. eix-update and eix will fill 8GB ram in 2 seconds. $ einfo =app-portage/eix-0.22.8 Portage 2.2.0_alpha30 (default/linux/amd64/10.0, gcc-4.6.0-asneeded, glibc-2.13-r2, 2.6.38-lh-r2 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-2.6.38-lh-r2-x86_64-Intel-R-_Core-TM-_i7_CPU_860_@_2.80GHz-with-gentoo-2.0.2 Timestamp of tree: Wed, 20 Apr 2011 18:00:01 +0000 ccache version 3.1.4 [enabled] app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.6.6-r2, 2.7.1-r1, 3.1.3-r1 dev-util/ccache: 3.1.4 dev-util/cmake: 2.8.4 sys-apps/baselayout: 2.0.2 sys-apps/openrc: 0.8.2 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.21 sys-devel/gcc: 4.5.2, 4.6.0 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.4-r1::last-hope sys-devel/make: 3.81-r2 sys-kernel/linux-headers: 2.6.38 virtual/os-headers: 0 Repositories: gentoo sunrise bicatali betagarden dummy science last-hope g-ctan Installed sets: ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g -Wdisabled-optimization -Wimplicit-function-declaration" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/nano/ /usr/share/openvpn/easy-rsa /var/lib/hsqldb /var/spool/torque" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g -Wdisabled-optimization -Wenum-compare" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="-t --jobs=12 --load-average=12 --keep-going" FEATURES="assume-digests binpkg-logs buildsyspkg ccache collision-protect distlocks fixlafiles fixpackages metadata-transfer multilib-strict news noinfo parallel-fetch preserve-libs protect-owned sandbox sfperms sign split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g -Wdisabled-optimization" GENTOO_MIRRORS=" http://gentoo.j-schmitz.net/mirror/ ftp://ftp.gentoo.mesh-solutions.com/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common" LINGUAS="en" MAKEOPTS="-j16 -l12" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="-z -v" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/data/local/sunrise/reviewed /data/local/bicatali /data/local/betagarden /data/local/dummy /data/local/sci /data/local/lh/ebuilds /data/local/g-ctan" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="64bit 7zip X acpi additions alsa amd64 arpwarp atlas bash-completion berkdb blas branding bzip2 c++ cairo caps cblas ccache cleartype cli command-args consolekit corefonts cracklib cups cupsddk custom-optimization cxx dbus dri dts exif fbcondecor fortran gdbm gif glibc-omitfp gmp gnome gnome-keyring gpm graphics gstreamer gtk hddtemp iconv icu ios ipod iproute2 iptables ipv6 ipython javascript jpeg kqemu lapack largefile lcms ldap libnotify libsexy lm_sensors lzma mailtrain md5sum mmx mng modules mp3 mudflap multilib multiuser nagios-dns nagios-ntp nagios-ping nagios-ssh nano-syntax ncurses network-cron nis nls nptl nptlonly nsplugin objc objc++ opengl openmp openntpd pam pcre pdf perl png policykit pppd pymol python qt-static qt3support readline rrdcgi rrdtool science sensord session smp sqlite sqlite3 sse sse2 ssl startup-notification svg sysfs system-sqlite tcpd threads tiff truetype type1 udev unicode vaapi vdpau x264 xcb xcomposite xinerama xorg xulrunner zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= app-portage/eix-0.22.8 was built with the following: USE="bzip2 (multilib) nls optimization sqlite strong-optimization tools -debug -doc -hardened -zsh-completion" CFLAGS="-march=core2 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=core2 -frecord-gcc-switches -g" CXXFLAGS="-march=core2 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=core2 -frecord-gcc-switches -g"
And some hardened problems come up * QA Notice: The following files contain writable and executable sections * Files with such sections will not work properly (or at all!) on some * architectures/operating systems. A bug should be filed at * http://bugs.gentoo.org/ to make sure the issue is fixed. * For more information, see http://hardened.gentoo.org/gnu-stack.xml * Please include the following list of files in your report: * Note: Bugs should be filed for the respective maintainers * of the package in question and not hardened@g.o. * RWX --- --- usr/bin/eix * RWX --- --- usr/bin/versionsort
If you cant reproduce it, Icann test more. It also might be related to the sqlite metacahes.
need a build log.
Created attachment 270753 [details] build.log Build is fine, only runtime is eval.
Yes, but I need to know what crazyass flags strong-optimization enabled. We don't support -flto.
I have no access these days to my main system where gcc:4.6 is installed, but I had no such memory leak with USE=strong-optimization. I didn't measure, but since the machine has only 2GB RAM and 4GB swap, I guess that I would have realized if eix would have needed 8GB :) Maybe it is the combination with some of your other CXXFLAGS which breaks? (In particular, I cannot test of course you -march and -mtune setttings). I will retry on that system in some days. Currently, I have only a small machine here with gcc:4.5 on which I could not reproduce the problem, even with your CXXFLAGS, and gcc:4.6 will take one day or two to compile (if it will compile at all). The sqlite is probably not related with this, since this should involve only update-eix and not eix itself. Well, not really: It might be that just loading the sqlite library (with -Wl,-z,now) needs so much memory, i.e. that actually sqlite needs the memory. You might try to compile with EXTRA_ECONF="--enable-separate-binaries" to get an eix binary which does not link against sqlite. To the other comments: eix does support -lto, and it profits enormously from it (about 1/3 of generated code size), so I will try hard to keep it working. The writable sections in code segment might be a problem of broken lto, but it didn't occur here with gcc:4.5 nor with gcc:4.6 on the other system.
CFLAGS="-O2 -pipe -march=native" CXXFLAGS="-O2 -pipe -march=native" LDFLAGS="-Wl,-O1" MAKEOPTS="V=1" EXTRA_ECONF="--enable-separate-binaries" emerge -1 eix was what I tested last, but it still leaks. And I don't thing 8GB is the end, it is just the physical limit here.
Back to my main machine... Now I can produce a problem (a segfault) which is always caused by -flto. The main change to my previous setting when everything was working fine is that I recompiled gcc-4.6 after the silent bump of April 13. It seems that this bump has broken -flto completely. Always segfault with -flto, no problems without it (even with graphite etc). eix dies at a harmless array.push_back, so I am rather sure that gcc just produces broken code with -flto. Unfortunately, I was not able to produce a minimal code example which breaks.
Hmm, the changes in that bump were backported from upstream, except for the joined-cpp-defs patch which shouldn't affect anything other than specs.
(In reply to comment #9) > Hmm, the changes in that bump were backported from upstream You are right: I re-fetched the previous gcc-4.6 and recompiled it and now have the same segfault. No idea why it had worked previously. I can only guess that it might have something to do with ccache or with a recompilation of mpc,mpfr,gmp,glib: When I tested, they were compiled with gcc-4.6; however, I recompiled them for testing now again with gcc-4.6 and always the same result: With -flto and gcc-4.6 segfault and the segment problems, without -flto or with gcc-4.5 everything works as it should. I will wait whether upstream will fix LTO with gcc-4.6.1 (I guess it is important also to them) or whether I find at least a reliable test program for configure.ac - I would not like to check the gcc version. For the moment, until I know more, I will leave eix as it is now.
Then include an ewarn in the ebuild. On system with smaller ram as my 8GB, there is hardly time to react.
I am still unable to produce a minimal example for the -flto bug, so I cannot work around the problem in ./configure. In eix' svn trunk (>=eix-0.22.9), I test now explicitly for gcc-4.6.0. Of course, this is a bad hack and makes no sense if e.g. gcc-4.6.1 will produce broken code with -flto, too. So, I will probably wait with the release until either I find a better solution or gcc-4.6.1 comes out.
It seems the segfault is fixed with gcc-4.6.1, so I plan to release eix-0.22.9 rather soon which only tests for gcc-4.6.0. Since the executable+writable sections problem now seems to be independent, I move the latter to bug 369795.
app-portage/eix-0.22.9 is in the tree
