Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 224611

Summary: [4.3] media-gfx/digikam-0.9.4_beta5 - compile takes longer than it should while building greycstoration
Product: Gentoo Linux Reporter: Chris Smith <chris>
Component: Current packagesAssignee: Gentoo Toolchain Maintainers <toolchain>
Status: RESOLVED UPSTREAM    
Severity: normal CC: alexxy, andriy155, bbutscheidt, dliana, flameeyes, gapon, gentoo, hoertp, jarausch, jlp.bugs, kde, kevin.bowling, konsti, maekke, patrick.holthaus, paulo, rhill, zeekec
Priority: High    
Version: 2006.1   
Hardware: AMD64   
OS: Linux   
URL: http://gcc.gnu.org/PR36439
Whiteboard:
Package list:
Runtime testing required: ---

Description Chris Smith 2008-06-02 15:10:29 UTC
compilation hangs with high cpu usage

Reproducible: Always

Steps to Reproduce:
1.emerge =media-gfx/digikam-0.9.4_beta5
2.
3.

Actual Results:  
Compilation gets to the following point and hangs with ~100% cpu usage on one core:
==========================================================
/usr/qt/3/bin/moc ./greycstorationwidget.h -o greycstorationwidget.moc
creating libgreycstoration_la.all_cpp.cpp ...
/bin/sh ../../../libtool --silent --tag=CXX   --mode=compile x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../digikam/libs/dimg -I../../../digikam/libs/dimg/filters -I../../../digikam/digikam  -I/usr/kde/3.5/include -I/usr/qt/3/include -I.  -I/usr/kde/3.5/include  -DQT_THREAD_SUPPORT  -D_REENTRANT  -Wno-long-long -Wundef -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wchar-subscripts -Wall -W -Wpointer-arith -DNDEBUG -DNO_DEBUG -O2 -march=native -O2 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION -DQT_CLEAN_NAMESPACE -fexceptions -w  -c -o libgreycstoration_la.all_cpp.lo libgreycstoration_la.all_cpp.cpp
==========================================================


Expected Results:  
Compilation should have completed.

emerge --info
Portage 2.1.5.3 (default/linux/amd64/2008.0, gcc-4.3.0, glibc-2.7-r2, 2.6.25-gentoo-r4 x86_64)
=================================================================
System uname: 2.6.25-gentoo-r4 x86_64 Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
Timestamp of tree: Mon, 02 Jun 2008 01:45:01 +0000
app-shells/bash:     3.2_p39
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python:     2.5.2-r4
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.2.5
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.62
sys-devel/automake:  1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1
sys-devel/binutils:  2.18.50.0.7
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   2.2.4
virtual/os-headers:  2.6.25-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -fomit-frame-pointer -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/4.0/env /usr/kde/4.0/share/config /usr/kde/4.0/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=native -O2 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="distlocks parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,--as-needed"
LINGUAS="en en_US"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/local/overlays/xeffects/trunk"
SYNC="rsync://davinci.realcomputerguy.soho/gentoo-portage"
USE="X a52 aac aalib acl acpi alsa amd64 amr apache2 audiofile avi bash-completion berkdb bzip2 cairo caps cdparanoia cdr cli cracklib crypt cups curl dbus djvu dri dts dv dvb dvd dvdr dvdread encode exif fam ffmpeg flac fortran gdbm gif glitz gphoto2 gpm gstreamer gtk2 hal iconv imagemagick imap imlib ipv6 isdnlog jack java jpeg jpeg2k kde kdeenablefinal kerberos lame lcd lcms ldap mad midi mmap mmx mozcalendar mozilla mp3 mudflap multilib musicbrainz ncurses nls nodrm nptl nptlonly nsplugin nvidia ofx ogg oggvorbis openexr opengl openmp pam pcre pda pdf perl png pppd pulseaudio python qt3 qt4 quicktime readline reflection samba sasl scanner sdl session slp sndfile spell spl sse sse2 ssl subversion svg tcltk theora threads tiff tk transparency truetype unicode usb utf8 v4l v4l2 vcd vorbis wav wavpack wmf x264 xattr xcomposite xine xml xorg xscreensaver xv xvid xvmc zlib" ALSA_CARDS="hda-intel hdsp" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_US" USERLAND="GNU" VIDEO_CARDS="nvidia nv vesa fbdev vmware v4l"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-02 21:23:27 UTC
Does it compile for you with gcc 4.2?
Comment 2 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-02 21:26:27 UTC
Probably an upstream bug... it happens the same on ppc: http://www.nabble.com/Bug-483137:-digikam-fails-to-build-on-powerpc---libgreycstoration_la.all_cpp.lo-td17489973.html
Comment 3 Chris Smith 2008-06-02 21:49:05 UTC
(In reply to comment #1)
> Does it compile for you with gcc 4.2?
> 

No clue. I don't have gcc-4.2 installed.
There is no problem compiling beta4.
Comment 4 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-02 21:51:24 UTC
I have to ask you to try with gcc 4.2 so that I can locate the source of the problem. Please, be so kind to confirm whether you can compile with gcc 4.2 or not.
Comment 5 Chris Smith 2008-06-02 22:01:53 UTC
(In reply to comment #4)
> I have to ask you to try with gcc 4.2 so that I can locate the source of the
> problem. Please, be so kind to confirm whether you can compile with gcc 4.2 or
> not.
> 

That'll take awhile.
I did have the same problem with beta4 and gcc-4.2.3 with a self edited ebuild (before there was one in portage), that didn't use the lcms patch.
Comment 6 Chris Smith 2008-06-02 22:58:43 UTC
Yes, it compiles with gcc-4.2.4.
Comment 7 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-03 07:49:03 UTC
Then we need a patch to make it compile on GCC 4.3.

I've sent this upstream: http://bugs.kde.org/show_bug.cgi?id=163118
Comment 8 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-03 08:03:34 UTC
Reported to CImg maintainer: http://sourceforge.net/forum/message.php?msg_id=5004166

Comment 9 Ryan Hill (RETIRED) gentoo-dev 2008-06-04 01:05:50 UTC
toolchain: this is an infinite loop with cpu pinned compiling C++ code with GCC 4.3.  source package is CImg.  appears with optimization >=O2 (has anyone tried -O1?).

current 4.3 branch svn shows the same problem.  cimg upstream is aware of the problem but doesn't know how to go about reporting it.

halo ~ # emerge --info
Portage 2.2_pre7-r1 (default/linux/amd64/2008.0/developer, gcc-4.3.1-pre20080528, glibc-2.7-r2, 2.6.25-gentoo-r3 x86_64)
=================================================================
System uname: 2.6.25-gentoo-r3 x86_64 Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz
Timestamp of tree: Tue, 03 Jun 2008 04:00:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p39
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python:     2.5.2-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.2.5
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.62
sys-devel/automake:  1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1
sys-devel/binutils:  2.18.50.0.7
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.25-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O1 -march=core2 -fomit-frame-pointer -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O1 -march=core2 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache collision-protect cvs digest distlocks fixpackages multilib-strict parallel-fetch preserve-libs sandbox sfperms sign splitdebug strict test unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://gentoo.mirrors.tera-byte.com/ http://gentoo.osuosl.org http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1"
LINGUAS="en"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/layman/mozilla /home/dirtyepic/overlay /home/dirtyepic/svn/dirtyepic /home/dirtyepic/svn/gcc-porting"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac acpi akode alsa amd64 audiofile avahi bash-completion berkdb bluetooth bzip2 cairo cdr cli consolekit cpudetection cracklib crypt css curl cvs dbus dri dts dvd dvdr dvdread eds emboss encode evo exif expat fam ffmpeg fftw flac fontconfig gdbm gif gimp gmp gpgme gphoto2 gstreamer gtk hal hdaps howl-compat hpn iconv id3tag ieee1394 imap imlib ipw4965 isdnlog jpeg kde kdeenablefinal kdehiddenvisibility lame laptop libffi libwww mad mdnsresponder-compat midi mikmod mmap mmx mozilla mp3 mpeg multilib ncurses network-cron nntp nptl nptlonly nsplugin ogg opengl openmp pam pch pcre pdf perl pic png pop ppds pppd pth python qt3 qt3support quicktime readline reflection rtc sasl session smapi smime smp smtp sndfile spell spl sql sse sse2 ssl ssse3 startup-notification subversion svg tcpd theora threads tiff truetype unicode urandom usb utempter vim-syntax vorbis wifi x264 xcb xcomposite xine xml xorg xpm xulrunner xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="mouse keyboard synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" USERLAND="GNU" VIDEO_CARDS="i810"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 10 Ryan Hill (RETIRED) gentoo-dev 2008-06-04 01:15:03 UTC
-O1 worked
Comment 11 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-04 06:25:26 UTC
If I understand correctly, one of the flags implied by -O2 is the causer of this. 

Are we to use a patch on CImg or should we force flag removal compilation on it? Does that imply compiling digikam with that restriction as well?
Comment 12 SpanKY gentoo-dev 2008-06-04 08:36:57 UTC
flag filtering really only should be done for current stable gcc.  filtering flags for a version of gcc that is *masked* is not ok.

what is the reproduction ?  build digikam with gcc-4.3 with -O2 ?
Comment 13 Ryan Hill (RETIRED) gentoo-dev 2008-06-04 13:27:42 UTC
yep, that'll do it.  i have a testcase i'm running through delta (it's around 3000 lines right now).  it'll take a while since each run is 60 seconds long to check for hang, but i should have something tonite.
Comment 14 Ryan Hill (RETIRED) gentoo-dev 2008-06-05 07:16:08 UTC
i'm having trouble coming up with a test script that can reliably tell whether gcc has hung or not.  right now i'm doing

  #!/bin/sh
  ( ulimit -t 60; g++ -c -O2 $1 2>&1 | grep 'Internal error: Killed (program cc1plus)' )
  if ! test "$?" == "0"; then
    exit 1
  fi
  exit 0


but this doesn't always work, because depending on what g++ is doing at the moment it gets killed by ulimit, it can spit out a variety of generic error messages.  any ideas how to tell the diffence between a hanging run from that fails right away for some other reason?

precompiled source is @ http://dev.gentoo.org/~dirtyepic/gcc/bug224611.x.i (1.7 MiB) if anyone's interested.
Comment 15 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-05 08:04:49 UTC
Don't worry guys, there is a better solution. The guys in digikam have removed optimizations only for that specific file and everything seems to work correctly now. I'll try to apply the patch tonight.

SVN commit 816533 by cgilles:

disable final options to compile CImg.h version 1.2.8.
This is a temporally solution util a fix is apply to CImg
CCBUGS: 163118


 M  +5 -0      Makefile.am  


WebSVN link: http://websvn.kde.org/?view=rev&revision=816533
Comment 16 SpanKY gentoo-dev 2008-06-05 14:53:05 UTC
that is not a real solution.  if gcc is broken, it needs to get fixed, not ignored.
Comment 17 SpanKY gentoo-dev 2008-06-05 15:05:35 UTC
Ryan: it seems that cc1plus is the thing getting hung while generating the assembly output ... so this should reproduce the problem as well:
g++ -O2 -S bug224611.ii -o bug224611.s

rather than using ulimit, monitor the filesize of the .s file ... once it stops increasing after ~60 seconds and g++ is still running, it's prob hung
Comment 18 SpanKY gentoo-dev 2008-06-05 15:10:14 UTC
may also be worth noting that when things seem to take off, the only system calls (as can be seen via strace) cc1plus is making is to brk() ... which means all it's doing is allocating more and more memory
Comment 19 Ryan Hill (RETIRED) gentoo-dev 2008-06-07 14:29:55 UTC
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36439
Comment 20 Diego Elio Pettenò (RETIRED) gentoo-dev 2008-06-09 11:48:45 UTC
Err I doubt it's a total hang sincerely:

digikam: Tue Apr 22 06:16:52 2008: 20 minutes, 55 seconds
digikam: Thu May  1 05:52:41 2008: 5 hours, 53 minutes, 20 seconds

The problem is that CImg.h is huge by itself, and it's highly template-based, so it expands exponentially during compile. It takes _eons_ to compile, but it compiles just fine.
Comment 21 Diego Elio Pettenò (RETIRED) gentoo-dev 2008-06-09 11:49:16 UTC
And forgot to say, it used something along the lines of 2GB of memory.
Comment 22 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-10 07:52:08 UTC
(In reply to comment #20)
> Err I doubt it's a total hang sincerely:
> 
> digikam: Tue Apr 22 06:16:52 2008: 20 minutes, 55 seconds
> digikam: Thu May  1 05:52:41 2008: 5 hours, 53 minutes, 20 seconds
> 
> The problem is that CImg.h is huge by itself, and it's highly template-based,
> so it expands exponentially during compile. It takes _eons_ to compile, but it
> compiles just fine.

The problem that it requires more time than expected remains. It should have the same behavior as gcc4.2 or gcc4.3 with low optimization flags.
Comment 23 Diego Elio Pettenò (RETIRED) gentoo-dev 2008-06-10 08:58:39 UTC
I don't disagree on that but it doesn't mean it's infinite either :)
Comment 24 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-10 22:46:37 UTC
You're right.
Comment 25 Carsten Lohrke (RETIRED) gentoo-dev 2008-06-13 10:52:22 UTC
*** Bug 226201 has been marked as a duplicate of this bug. ***
Comment 26 Andrés Becerra Sandoval 2008-06-26 12:42:25 UTC
Upstream has released digikam-0.9.4_rc1.

Copying the ebuild for digikam-0.9.4_beta5 to digikam-0.9.4_rc1.ebuild solves the compilation problem for me in ~x86.

emerge took arount 21 minutes compiling digikam succesfully.
Comment 27 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-06-27 23:34:55 UTC
(In reply to comment #26)
> Upstream has released digikam-0.9.4_rc1.
> Copying the ebuild for digikam-0.9.4_beta5 to digikam-0.9.4_rc1.ebuild solves
> the compilation problem for me in ~x86.
> emerge took arount 21 minutes compiling digikam succesfully.

See comment #16
Comment 28 Ioannis Aslanidis (RETIRED) gentoo-dev 2008-07-21 20:13:20 UTC
Just for the record, the latest release of digikam 0.9.5 has a patch to work around this issue, so this bug is somewhat unnecessary and could be closed as wontfix.
Comment 29 Chris Smith 2008-07-21 20:30:05 UTC
(In reply to comment #28)
> Just for the record, the latest release of digikam 0.9.5 has a patch to work
> around this issue, so this bug is somewhat unnecessary and could be closed as
> wontfix.
> 

Fine with me.
Comment 30 Mark Loeser (RETIRED) gentoo-dev 2008-07-21 20:39:26 UTC
Still a compiler bug that we need to fix.  They are just hiding that its a problem.
Comment 31 Volker Hemmann 2008-10-07 16:01:01 UTC
Hi,
I just want to add, I have the problem with gcc 4.3.2 and digikam-0.10.0_beta3
Comment 32 Epicanis 2008-10-19 17:29:08 UTC
(In reply to comment #31)
> Hi,
> I just want to add, I have the problem with gcc 4.3.2 and digikam-0.10.0_beta3
> 

Same here with the media-gfx/digikam-0.10.0_beta2-r2 in the kde-testing overlay.
(gcc (Gentoo 4.3.2 p1.0) 4.3.2 on AMD64)
Comment 33 Phill 2008-11-18 19:38:04 UTC
I had the same cc1plus hang problem with gcc-4.3.2.  It worked using gcc-4.2.4 though.

Comment 34 Mark Loeser (RETIRED) gentoo-dev 2009-05-10 02:06:59 UTC
It looks like upstream isn't going to be fixing this for 4.3, and it works fine in 4.4.  I'm going off one of the comments upstream where they said "probably not much to do here ..."