Bugzilla Bug 164677

There is a new version of enblend which seems to add considerable improvements.
I would like to ask for it to be added to the portage tree (masked).

Reproducible: Always

Created an attachment (id=109026) [edit]
enblend-3.0.ebuild

The new enblend has a few new dependencies including media-libs/lcms,
virtual/glut, media-libs/glew, and media-libs/libxmi. Currently,
media-libs/libxmi is not in portage.

I also included two patches that can be found in the sf.net tracker for
enblend.

Created an attachment (id=109027) [edit]
libxmi-1.2.ebuild

Created an attachment (id=109028) [edit]
endless_loop_anneal.patch

Created an attachment (id=109029) [edit]
amd64_compilation.patch

It compiles fine for me on x86. Can someone test it on amd64? 

Someone on the hugin-ptx mailing list needed the changes found in the
amd64_compilation.patch to make it work for an AMD64 Ubuntu machine. I am
wondering how it does on Gentoo.

libxmi is part of GNU plotutils which is in portage.  I had just finished
getting my own ebuild working and we had a "mid-air collision" with our posts. 
I think the correct thing to do is to remove the the dependency on libxmi and
use media-libs/plotutils instead. This will remove the need for a new package. 

With the amd64 patch it built fine on my machine.  

Created an attachment (id=109034) [edit]
enblend-3.0.ebuild

Thanks, changed the dep to media-libs/plotutils.

enblend-3.0.ebuild compiles fine on amd64 with amd64_compilation.patch (didn't
tested without)
it is serveral times faster than 2.x *WOW 

Thanks guys, just bumped it in cvs!

By the way, if any of you could provide me with some pictures, it would help me
better test enblend, hugin and others.

Cheers,
Marcelo

Well, it don't compil here on my amd64 :
make[3]: Entering directory
`/var/tmp/portage/media-gfx/enblend-3.0/work/enblend-3.0/src'
if x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I. -I..    -g -O3 -ffast-math
-Wall -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DENBLEND_CACHE_IMAGES -DNDEBUG
-I../include -O2 -march=k8 -pipe -MT enblend-enblend.o -MD -MP -MF
".deps/enblend-enblend.Tpo" -c -o enblend-enblend.o `test -f 'enblend.cc' ||
echo './'`enblend.cc; \
        then mv -f ".deps/enblend-enblend.Tpo" ".deps/enblend-enblend.Po"; else
rm -f ".deps/enblend-enblend.Tpo"; exit 1; fi
if x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I. -I..    -g -O3 -ffast-math
-Wall -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DENBLEND_CACHE_IMAGES -DNDEBUG
-I../include -O2 -march=k8 -pipe -MT enblend-gpu.o -MD -MP -MF
".deps/enblend-gpu.Tpo" -c -o enblend-gpu.o `test -f 'gpu.cc' || echo
'./'`gpu.cc; \
        then mv -f ".deps/enblend-gpu.Tpo" ".deps/enblend-gpu.Po"; else rm -f
".deps/enblend-gpu.Tpo"; exit 1; fi

And it stays here for ages without going further...

emerge --info
--- Invalid atom in /usr/portage/profiles/package.mask:
=dev-ada/booch_components-20030815.ebuild
--- Invalid atom in /usr/portage/profiles/package.mask:
<=dev-ada/adasockets-1.8.4.4.ebuild
Portage 2.1.2-r7 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.5-r0, 2.6.20
x86_64)
=================================================================
System uname: 2.6.20 x86_64 AMD Athlon(tm) 64 Processor 3000+
Gentoo Base System version 1.12.9
Timestamp of tree: Mon, 05 Feb 2007 12:00:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[disabled]
ccache version 2.4 [disabled]
dev-java/java-config: 1.3.7, 2.0.31-r3
dev-lang/python:     2.4.4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r6
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.17
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.20
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=k8 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.3/env /usr/kde/3.3/share/config
/usr/kde/3.3/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf
/etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c"
CXXFLAGS="-O2 -march=k8 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks fixpackages metadata-transfer parallel-fetch
sandbox sfperms strict userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://mirror.switch.ch/mirror/gentoo/
http://www.gigaload.org/gentoo.org/ ftp://mirror.switch.ch/mirror/gentoo/"
LC_ALL="en_GB.iso8859-1"
LDFLAGS="-Wl,-O1 "
LINGUAS="en ko"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/science
/usr/portage/local/layman/gentoo-de /usr/portage/local/layman/vdr-testing
/usr/portage/local/layman/voip /usr/local/portage
/usr/portage/local/layman/portage-xgl"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnowex 64bit 7zip X Xaw3d a52 aac aalib acl acpi aim aio alsa amd64 amr
ansi aotuv apache2 apm asf async atlas auctex audacious audiofile avahi
bash-completion batch berkdb bidi bitmap-fonts blas bonjour bonobo
browserplugin buffysize buttons bzip2 bzlib cairo caps cardclient cdda cddb
cdio cdparanoia cdr cg chm chroot cli conax console constcw corba cracklib
crypt cryptoworks cscope css csv ctype cups curl curlwrappers daap dar64 dbus
dga dio directfb divx divx4linux djbfft djvu dlloader doc dri dts dv dvb dvd
dvdr dvdread dvi dynagraph ecc effects encode exif expat exscalibar ext-doc
ext-dvi ext-gnumeric ext-koffice ext-kpresenter ext-kspread ext-kword ext-lyx
ext-man ext-openoffice ext-pdf ext-ppt ext-ps ext-rpm ext-rtf ext-tex ext-xls
fame fasttrack fat fbcon ffmpeg fftw firefox flac flood fltk fontconfig
foomaticdb fortran fping fpx freetext freetype2 gb gcl gd gdal gdbm general gg
ggi gif gimp gimpprint ginac glibc-omitfp glitz glut glx gmedia gmp gmtfull
gmthigh gmtsuppl gmttria gnome gnutella gnutls gopher gpgme gpm grammar
graphviz grass gs gstreamer gtalk gtk gtk2 gtkhtml guile h323 hal hdf5
howl-compat httpd iconv icq id3 idn ieee1394 imagemagick imlib ipv6 irc irdeto
isdnlog ithreads jabber java javascript jbig jikes jp2 jpeg jpeg2k
kig-scripting kqemu lame latex lcms ldap libcaca libedit libg++ libnotify
libsamplerate libtommath libvisual libwww live lj lm_sensors lnbsharing
logitech-mouse logrotate lzo mad matroska mbox midi mikmod mime mimencode mjpeg
mng mod mono motif moznocompose moznoirc moznomail moznoroaming mozsvg
mozxmlterm mp3 mp4 mp4live mpeg mpeg2 mplayer msn msnextras musepack mysql
mythtv nagra nautilus ncurses nethack netpbm network nfs nls nntp normalizemime
nptl nptlonly nsplugin nvidia nvram nvtv offensive ogg oggvorbis ole
on-the-fly-crypt openexr opengl openssl oscar pam pcre pda pdf perl player
plotutils png posix ppds pppd preview-latex pulseaudio python pyzor qhull qt3
qt4 quicktime rar readline recode reflection reiser4 reiserfs rss rtc rtsp sasl
screen sdl seca session shl shout sid silc slang slp smime sms sndfile sox
speex spell spl srt sse-filters ssl stream subtitles svg svgz szip tcl tcltk
tcpd tetex theora thesaurus threads tiff tk tomsfastmath tools transcode
truetype truetype-fonts type1-fonts udev unicode usb userlocales utf8 uudeview
v4l v4l2 vcd vcdimager vdr viaccess vim vim-pager vim-with-x vlm vorbis webdav
wma wma123 wmf wmp wordperfect wv wxgtk1 wxwindows x264 xanim xattr xext xfs
xine xinerama xinetd xml xml2 xorg xpm xprint xv xvid xvmc yahoo zlib"
ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x
ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3
trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw
asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa
lfloat linear meter mulaw multi null plug rate route share shm softvol"
ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux"
LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses
text" LINGUAS="en ko" USERLAND="GNU" VIDEO_CARDS="ati fbdev nv v4l vesa vga
nvidia"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG

(In reply to comment #10)
> Well, it don't compil here on my amd64 :
> make[3]: Entering directory

...

> './'`gpu.cc; \
>         then mv -f ".deps/enblend-gpu.Tpo" ".deps/enblend-gpu.Po"; else rm -f
> ".deps/enblend-gpu.Tpo"; exit 1; fi
> 
> And it stays here for ages without going further...

How, long have you waited before canceling the compile? On my machine it
usually takes about 5 minutes to compile just the gpu.cc file. Also, the
compiler uses about 500-600 MBs of memory during that phase of the compile. If
you don't have very much free memory it will take a lot longer as it will have
to swap things in and out of memory.

I would try letting the compile run longer, especially if your machine is low
on memory. Start the process and go do something else for awhile.

For people with only 1Gb of RAM and only (amd64@3000) compilation take 3 hours
???

But it compil :)

spurious compile warning ... then error on amd64 (march=nocona) & gcc-4.1.1



Portage 2.1.2-r9 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.5-r0,
2.6.18-gentoo-r6 x86_64)
=================================================================
System uname: 2.6.18-gentoo-r6 x86_64 Intel(R) Core(TM)2 CPU         T7200  @
2.00GHz
Gentoo Base System version 1.12.6
Timestamp of tree: Fri, 02 Mar 2007 18:30:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[disabled]
ccache version 2.4 [disabled]
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r6
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -march=nocona -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O3 -march=nocona -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LINGUAS="en_GB fr"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X aac acpi alsa amd64 berkdb bitmap-fonts bzip2 cairo cli cracklib crypt
cups dbus dri dvd dvdread exif fam ffmpeg firefox fortran gcj gdbm glut gnome
gpm gtk hal iconv ipv6 isdnlog jpeg jpeg2k ldap libg++ matroska midi mp3
ncurses network nls nptl nptlonly oss pam pcre perl png ppds pppd python
readline reflection samba sdl session spl sse3 ssl ssse3 svg tcpd threads tiff
truetype-fonts type1-fonts unicode usb vorbis xorg xvidzlib zlib"
ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare
dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw
multi null plug rate route share shm softvol" ELIBC="glibc"
INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz
cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_GB fr"
USERLAND="GNU" VIDEO_CARDS="nv nvidia vesa"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS,
PORTAGE_RSYNC_EXTRA_OPTS



../include/vigra/impex.hxx: In function 'void
vigra::detail::mapVectorImageToLowerPixelType(SrcIterator, SrcIterator,
SrcAccessor, MArray&) [with SrcIterator = vigra::Diff2D, SrcAccessor =
vigra::MultiImageVectorMaskAccessor4<vigra::ConstCachedFileImageIterator<vigra::RGBValue<short
int, 0u, 1u, 2u> >, vigra::RGBAccessor<vigra::RGBValue<short int, 0u, 1u, 2u>
>, vigra::CachedFileImageIterator<unsigned char>,
vigra_ext::ReadFunctorAccessor<vigra::Threshold<unsigned char, short int>,
vigra::StandardValueAccessor<unsigned char> > >, MArray = vigra::MultiArray<3u,
int, std::allocator<int> >]':
../include/vigra/impex.hxx:641: warning: 'minmax$min' may be used uninitialized
in this function
../include/vigra/impex.hxx:641: warning: 'minmax$max' may be used uninitialized
in this function
../include/vigra/impex.hxx: In function 'void
vigra::detail::mapVectorImageToLowerPixelType(SrcIterator, SrcIterator,
SrcAccessor, MArray&) [with SrcIterator = vigra::Diff2D, SrcAccessor =
vigra::MultiImageVectorMaskAccessor4<vigra::ConstCachedFileImageIterator<vigra::RGBValue<short
unsigned int, 0u, 1u, 2u> >, vigra::RGBAccessor<vigra::RGBValue<short unsigned
int, 0u, 1u, 2u> >, vigra::CachedFileImageIterator<unsigned char>,
vigra_ext::ReadFunctorAccessor<vigra::Threshold<unsigned char, short unsigned
int>, vigra::StandardValueAccessor<unsigned char> > >, MArray =
vigra::MultiArray<3u, int, std::allocator<int> >]':
../include/vigra/impex.hxx:641: warning: 'minmax$min' may be used uninitialized
in this function
../include/vigra/impex.hxx:641: warning: 'minmax$max' may be used uninitialized
in this function
../include/vigra/impex.hxx: In function 'void
vigra::detail::mapVectorImageToLowerPixelType(SrcIterator, SrcIterator,
SrcAccessor, MArray&) [with SrcIterator = vigra::Diff2D, SrcAccessor =
vigra::MultiImageMaskAccessor2<vigra::ConstCachedFileImageIterator<int>,
vigra::StandardConstValueAccessor<int>, vigra::CachedFileImageIterator<unsigned
char>, vigra_ext::ReadFunctorAccessor<vigra::Threshold<unsigned char, int>,
vigra::StandardValueAccessor<unsigned char> > >, MArray = vigra::MultiArray<3u,
int, std::allocator<int> >]':
../include/vigra/impex.hxx:641: warning: 'minmax$min' may be used uninitialized
in this function
../include/vigra/impex.hxx:641: warning: 'minmax$max' may be used uninitialized
in this function
{standard input}: Assembler messages:
{standard input}:74707: Warning: end of file not at end of a line; newline
inserted
x86_64-pc-linux-gnu-g++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <URL:http://bugs.gentoo.org/> for instructions.
make[3]: *** [enblend-enblend.o] Error 1
make[3]: Leaving directory
`/var/tmp/portage/media-gfx/enblend-3.0/work/enblend-3.0/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory
`/var/tmp/portage/media-gfx/enblend-3.0/work/enblend-3.0/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/var/tmp/portage/media-gfx/enblend-3.0/work/enblend-3.0'
make: *** [all] Error 2

!!! ERROR: media-gfx/enblend-3.0 failed.
Call stack:
  ebuild.sh, line 1614:   Called dyn_compile
  ebuild.sh, line 971:   Called qa_call 'src_compile'
  environment, line 3025:   Called src_compile
  ebuild.sh, line 645:   Called die

I can confirm the huge memory need for compiling this package. I have 768MB of
RAM plus 730MB of swap. It stopped at:

if i686-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I. -I..    -g -O3 -ffast-math
-Wall -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DENBLEND_CACHE_IMAGES -DNDEBUG
-I../include -O2 -pipe -march=i686 -mtune=pentium3 -fomit-frame-pointer -MT
enblend-enblend.o -MD -MP -MF ".deps/enblend-enblend.Tpo" -c -o
enblend-enblend.o `test -f 'enblend.cc' || echo './'`enblend.cc; \
        then mv -f ".deps/enblend-enblend.Tpo" ".deps/enblend-enblend.Po"; else
rm -f ".deps/enblend-enblend.Tpo"; exit 1; fi

for some long minutes... Used all my RAM. It was swapping a lot and my computer
got too unresponsive, so I pressed Ctrl+C at emerge.

I don't understand why this huge memory requirement. Not even KDE requires
this. However, I can notice -O3 and -O2 on same line (look above). Maybe this
is culprit?

re: Comment #14: indeed, optimizations are at fault here.  However, changing
CFLAGS in make.conf to remove -O is not sufficient because
enblend-3.0/src/Makefile contains:

enblend_CXXFLAGS = -g -O3 -ffast-math -Wall ...

If I manually remove the -O3, the compilation uses less memory (about 300MiB
from my estimates watching top).  Even -O seems to result in the huge memory
usage noted earlier.

I suppose we could sed the -O3 out... but maybe people with 4GiB of RAM want
the optimizations?  Seems to me, at the very least we need a warning about
extreme RAM usage (c.f., eclipse-sdk ebuild which IIRC uses a eclass for this
purpose).

(In reply to comment #15)
> re: Comment #14: indeed, optimizations are at fault here.  However, changing
> CFLAGS in make.conf to remove -O is not sufficient because
> enblend-3.0/src/Makefile contains:
> 
> enblend_CXXFLAGS = -g -O3 -ffast-math -Wall ...
> 
> If I manually remove the -O3, the compilation uses less memory (about 300MiB
> from my estimates watching top).  Even -O seems to result in the huge memory
> usage noted earlier.
> 
> I suppose we could sed the -O3 out... but maybe people with 4GiB of RAM want
> the optimizations?  Seems to me, at the very least we need a warning about
> extreme RAM usage (c.f., eclipse-sdk ebuild which IIRC uses a eclass for this
> purpose).
> 

I'm in the same situation: neither enblend or hugin compile! It fills my ram
and then fails. (512+512swap).
Shouldn't this be handled with a use flag or something like that?