Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 171367 - [gcc-4.1 regression?] extreme memory consumption with safe optimizations
Summary: [gcc-4.1 regression?] extreme memory consumption with safe optimizations
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-03-18 18:10 UTC by Jeremy Huddleston (RETIRED)
Modified: 2009-04-20 00:49 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
my emerge info (ein,3.52 KB, text/plain)
2007-09-19 14:42 UTC, Michał Kiedrowicz
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jeremy Huddleston (RETIRED) gentoo-dev 2007-03-18 18:10:59 UTC
You can reference bug #160020 and #153650 for more background on this issue.

The more I investigate this issue, the more I believe it to be a non-issue, but I am filing this bug report with my findings and will allow those experiencing the above bugs to comment on them as I cannot verify the extremity with which they are experiencing problems.

##### This section shows expected results using gcc-4.0.4
# i686-pc-linux-gnu-cc --version                                             
i686-pc-linux-gnu-gcc (GCC) 4.0.4 (Gentoo 4.0.4, pie-8.7.8)
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc40.O0.s -O0                                                                                           
  - completed in 13s using ~160M of memory
# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc40.O1.s -O1
  - completed in 1m 15s using ~600M of memory                                                                                         
# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc40.O2.s -O2                                                                                           
  - completed in 1m 50s using ~475M of memory

###### This next section shows results using gcc-4.1.2 ######

# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc41.O0.s -O0
 - completed in 9s using ~145M of memory
# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc41.O1.s -O1
 - completed in 23m (4m CPU time) using ~1400M of memory

 - Others (CCd) have reported that this consumes over 3G on their system.

The gcc.info file included in gcc-4.1.2 indicates that -O1 is equivalent to the options used below.  -fipa-pure-const -fipa-reference -ftree-copy-prop -ftree-sink -ftree-salias are not mentioned in the gcc.info file, but looking at opts.c:~515 reveals that they should be included in that list.

# i686-pc-linux-gnu-cc -x c++ -S execute.cpp.gcc4.1.2.pre -o execute.gcc41.s -fdefer-pop -fomit-frame-pointer -fguess-branch-probability -fcprop-registers -floop-optimize -fif-conversion -fif-conversion2 -ftree-ccp -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-ter -ftree-lrs -ftree-sra -ftree-copyrename -ftree-fre -ftree-ch -funit-at-a-time -fmerge-constants -fipa-pure-const -fipa-reference -ftree-copy-prop -ftree-sink -ftree-salias
  - completed in 12s using 160M of memory

Therefore it has something to do with the optimize flag in general and not with any of the options it enables.
Comment 1 Jeremy Huddleston (RETIRED) gentoo-dev 2007-03-18 18:28:32 UTC
The preprocessed file is too big to be attached to bugzilla, so I put it here:
http://dev.gentoo.org/~eradicator/execute.cpp.gcc4.1.2.pre
Comment 2 Christian Faulhammer (RETIRED) gentoo-dev 2007-03-19 18:34:43 UTC
The provided file and the given flags don't consume all memory on my system, even adding all additional flags of -O2 (from man page) don't show that leak.
Comment 3 Jeremy Huddleston (RETIRED) gentoo-dev 2007-03-20 16:33:13 UTC
Interesting... Christian, can you try preprocessing the file.  I just did a s/ -c / -E / on the compile command during emerge

i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../.. -I.. -O2 -march=athlon64 -fomit-frame-pointer -pipe -MT execute.lo -MD -MP -MF .deps/execute.Tpo -E execute.cpp -fPIC -DPIC -o execute.pre

which is actually equivalent to

i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../.. -I.. -O2 -E execute.cpp -DPIC -o execute.pre
Comment 4 Christian Faulhammer (RETIRED) gentoo-dev 2007-08-21 06:08:17 UTC
pixie 2.0.2-r1 is not in the tree anymore and I had no problems with it.  Is there still a need to investigate further?
Comment 5 Jeremy Huddleston (RETIRED) gentoo-dev 2007-08-24 02:17:06 UTC
Yes, the current version of pixie shows the same memory consumption (as did the 1.x versions of pixie).  It's a problem with gcc, I think... not pixie.  
Comment 6 Michał Kiedrowicz 2007-09-19 14:41:30 UTC
Hi

I also experience problems with huge memory usage during compilation. It seems it is connected with artithmetic operations, because gcc eats enormous amounts of RAM when compiling for example libqalculate, boost.

Example 'ps axu' output:

portage  27360  0.0  0.1   7528  1064 pts/2    SN+  16:35   0:00 /usr/x86_64-pc-linux-gnu/gcc-bin/4.2.0/x86_64-pc-linux-gnu-g++ -ftemplate-depth-128 -march=nocona -O2 -pipe -finline-functions -Wno-inline -Wall -fPIC -DBOOST_ALL_NO_LIB=1 -DNDEBUG -I.. -c -o ../bin.v2/libs/multi_index/test/test_update.test/gcc-4.2/release/debug-symbols-none/optimization-none/test_update.o ../libs/multi_index/test/test_update.cpp
portage  27361 89.7 31.5 334808 320364 pts/2   RN+  16:35   0:06 /usr/libexec/gcc/x86_64-pc-linux-gnu/4.2.0/cc1plus -quiet -I.. -D_GNU_SOURCE -DBOOST_ALL_NO_LIB=1 -DNDEBUG ../libs/multi_index/test/test_update.cpp -quiet -dumpbase test_update.cpp -march=nocona -auxbase-strip ../bin.v2/libs/multi_index/test/test_update.test/gcc-4.2/release/debug-symbols-none/optimization-none/test_update.o -O2 -Wno-inline -Wall -ftemplate-depth-128 -finline-functions -fPIC -o -
portage  27362  0.4  0.6  17604  6476 pts/2    SN+  16:35   0:00 /usr/lib/gcc/x86_64-pc-linux-gnu/4.2.0/../../../../x86_64-pc-linux-gnu/bin/as -Qy -o ../bin.v2/libs/multi_index/test/test_update.test/gcc-4.2/release/debug-symbols-none/optimization-none/test_update.o -

g++ is using 320MB!
I have core2duo and I set MAKEOPTS to "-j3", that gives 1GB RAM for compilation. I didn't investigate it much, but I can note most memory consuming packages and list them here.
Comment 7 Michał Kiedrowicz 2007-09-19 14:42:07 UTC
Created attachment 131296 [details]
my emerge info
Comment 8 Mark Loeser (RETIRED) gentoo-dev 2009-04-20 00:49:17 UTC
File a new bug if this is a problem with gcc-4.3 with an example attached.