Bugzilla Bug 155804

cp Magick.bs blib/arch/auto/Image/Magick/Magick.bs
chmod 644 blib/arch/auto/Image/Magick/Magick.bs
rm -f blib/arch/auto/Image/Magick/Magick.so
x86_64-pc-linux-gnu-gcc  -L../magick/.libs -lMagick -shared -L/usr/local/lib64
Magick.o  -o blib/arch/auto/Image/Magick/Magick.so       \
           -L/usr/lib64 -L/usr/lib64 -lfreetype -lz -L/usr/lib -ltiff
-lfreetype -ljpeg -lfontconfig -lXext -lSM -lICE -lX11 -lXt -lbz2 -lz -lpthread
-lm -lpthread     \

/usr/lib/gcc/x86_64-pc-linux-gnu/3.4.6/../../../../x86_64-pc-linux-gnu/bin/ld:
cannot find -lMagick
collect2: ld returned 1 exit status
make[1]: *** [blib/arch/auto/Image/Magick/Magick.so] Error 1
make[1]: Leaving directory
`/var/tmp/portage/media-gfx/imagemagick-6.3.0.5/work/ImageMagick-6.3.0/PerlMagick'
make: *** [all-perl] Error 2
make: *** Waiting for unfinished jobs....

!!! ERROR: media-gfx/imagemagick-6.3.0.5 failed.
Call stack:
  ebuild.sh, line 1568:   Called dyn_compile
  ebuild.sh, line 937:   Called src_compile
  imagemagick-6.3.0.5.ebuild, line 86:   Called die

!!! compile problem
!!! If you need support, post the topmost build error, and the call stack if
relevant.

Created an attachment (id=102428) [details]
emerge --info

The emerge --info output

Created an attachment (id=102429) [details]
Other warnings before fail message

This are addiditonal warnings I see before the ld error is displayed.

The same thing happens on hppa. Can't seem to reproduce it on an x86. The
PerlMagick build does not fail if imagemagick is already installed on the
system. Also, it does not fail when I build it on an x86 system with no
imagemagick installed.

Created an attachment (id=102493) [details]
imagemagick-6.3.0.5.ebuild.patch

This should remove the offending -lMagick.

I experience the same problem. I think it's because of my MAKEOPTS="-j5",
because imagemagick-6.3.0.5 emerged fine when I set MAKEOPTS="-j1".

(In reply to comment #5)
> I experience the same problem. I think it's because of my MAKEOPTS="-j5",
> because imagemagick-6.3.0.5 emerged fine when I set MAKEOPTS="-j1".
> 

I also experienced this bug on PPC and when the ebuild failed (multiple times)
I did a manual make in the /var/tmp/portage/imagemagick...../ touched the
.compiled file and did a 
#ebuild /usr/portage/media-gfx/imagemagick/imagemagick-6.3.0.5.ebuild install

and from then on everything worked OK...
So I guess the -j1 "hack" works.

The patch from comment #4 works.
And also using MAKEOPTS="-j1" works (for me).

(In reply to comment #7)
> The patch from comment #4 works.
> And also using MAKEOPTS="-j1" works (for me).

The patch works because that causes the build to fail if -lMagick is not found.
That library would only be available on the live system to link against, which
would usually be a stale library you do not want to use anyway because you are
installing a new library, hence the patch.

I don't know about the other problem mentioned here: imagemagick builds fine
for me on several systems with MAKEOPTS=-j2 (but with the patch, or else it
will fail on a fresh install, or it will build against the old lib).

(In reply to comment #8)
> I don't know about the other problem mentioned here: imagemagick builds fine
> for me on several systems with MAKEOPTS=-j2 (but with the patch, or else it
> will fail on a fresh install, or it will build against the old lib).

I have tested to build (without the patch) with MAKEOPTS=-j2, this fails.
I have tested to build (without the patch) with MAKEOPTS=-j1, this succeeds.
I have tested to build (with the patch) with MAKEOPTS=-j2, this succeeds.

So there is still something strange around when using MAKEOPTS=-j2 (or higher).

(In reply to comment #3)
> The same thing happens on hppa. Can't seem to reproduce it on an x86. The
> PerlMagick build does not fail if imagemagick is already installed on the
> system. Also, it does not fail when I build it on an x86 system with no
> imagemagick installed.
> 

It fails on me (exact same error) on x86, so I won't bother pasting the exact
same output.  As some of the other commenters have mentioned, running with
MAKEOPTS="-j1" works for me (as opposed to the MAKEOPTS="-j5" in make.conf --
this is a 4-way box).

I'm also seeing this problem on an amd64 machine. I'm running with
MAKEOPTS="-j2".

I'll use the -j1 hack for the moment.

*** Bug 156477 has been marked as a duplicate of this bug. ***

*** Bug 157039 has been marked as a duplicate of this bug. ***

I can confirm that the MAKEOPTS="-j1" solution works for x86.  Thanks for the
hint!

*** Bug 157747 has been marked as a duplicate of this bug. ***

x86, 1 processor available, athlon-xp

-j3 fails with same error
-j2 as well
-j1 succeeds

maybe the ebuild should force -j1 until this is resolved.

save everyone wasting time banging into this. 

*** Bug 158761 has been marked as a duplicate of this bug. ***

I was having the same problem and did the -j1 fix, which worked.  Here is some
background:

SirTux ~ # emerge --info
Portage 2.1.1-r2 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.4-r4,
2.6.18-gentoo-r4 x86_64)
=================================================================
System uname: 2.6.18-gentoo-r4 x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.12.6
Last Sync: Sat, 23 Dec 2006 06:30:01 +0000
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.30
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.60
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf
/etc/java-config/vms/ /etc/terminfo /etc/texmf/web2c"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X acpi aim alsa alsa_pcm_plugins_adpcm alsa_pcm_plugins_alaw
alsa_pcm_plugins_asym alsa_pcm_plugins_copy alsa_pcm_plugins_dmix
alsa_pcm_plugins_dshare alsa_pcm_plugins_dsnoop alsa_pcm_plugins_empty
alsa_pcm_plugins_extplug alsa_pcm_plugins_file alsa_pcm_plugins_hooks
alsa_pcm_plugins_iec958 alsa_pcm_plugins_ioplug alsa_pcm_plugins_ladspa
alsa_pcm_plugins_lfloat alsa_pcm_plugins_linear alsa_pcm_plugins_meter
alsa_pcm_plugins_mulaw alsa_pcm_plugins_multi alsa_pcm_plugins_null
alsa_pcm_plugins_plug alsa_pcm_plugins_rate alsa_pcm_plugins_route
alsa_pcm_plugins_share alsa_pcm_plugins_shm alsa_pcm_plugins_softvol amd64
apache2 automount avi bash-completion berkdb binary-drivers bitmap-fonts
bonjour bootsplash branding bzip2 cdinstall cdr clamav cli cracklib crypt cups
curl curlwrappers dbus dbx dio directfb dlloader doc dri dvd dvdr eds
elibc_glibc emboss encode esd ethereal fam fbcon firefox font-server fortran
ftp gdbm gif gmail gmailtimestamps gnome gpm gstreamer gtalk gtk gtk2 hal
hardened hardenedphp hfs howl html iconv imap input_devices_evdev
input_devices_keyboard input_devices_mouse java javascript jikes jpeg
kernel_linux krb4 ldap libg++ mad madwifi mikmod mime mono mp3 mpeg msn mysql
mysqli ncurses nls nocd nptl nptlonly nsplugin ntfs ogg opengl oss pam pcre pda
pdf pdflib perl php png posix ppds pppd python qt3 qt4 quicktime rdesktop
readline recode reflection samba screen sdl seamonkey session slp snmp spell
spl ssl swat symlink tcpd tiff tokenizer truetype truetype-fonts type1-fonts
udev unicode usb userland_GNU video_cards_apm video_cards_ark video_cards_ati
video_cards_chips video_cards_cirrus video_cards_cyrix video_cards_dummy
video_cards_fbdev video_cards_glint video_cards_i128 video_cards_i810
video_cards_mga video_cards_neomagic video_cards_nv video_cards_rendition
video_cards_s3 video_cards_s3virge video_cards_savage video_cards_siliconmotion
video_cards_sis video_cards_sisusb video_cards_tdfx video_cards_tga
video_cards_trident video_cards_tseng video_cards_v4l video_cards_vesa
video_cards_vga video_cards_via video_cards_vmware video_cards_voodoo videos
vorbis wifi winbind wxwindows xine xml xorg xscreensaver xv xvid yahoo zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS,
LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

Changing MAKEOPTS="-j1" worked for me as well

Also experienced this problem on x86/2006.1.

Turning off parallel building also worked for me.

Managed to duplicate this in a stable ppc system. Did not try the patch, just
the -j1. It was fresh instalation.

*** Bug 160655 has been marked as a duplicate of this bug. ***

From #gentoo:

10:25 <Munger> kojiro, Patch [comment #4] worked great. My bugzilla account is
in a state of confusion right now. Can you pass that info on to the appropriate
channels?

(In reply to comment #24)
> From #gentoo:
> 
> 10:25 <Munger> kojiro, Patch [comment #4] worked great. My bugzilla account is
> in a state of confusion right now. Can you pass that info on to the appropriate
> channels?
> 

Yep. I have got bugzilla back and can confirm that the patch does the trick.

--
Munger

I also have the bug (-j2 on ~x86); dont think you need a detailed report from
me; when will the patch be merged to portage ? already 70d old available fix,
and still not merged ? is maint dead ?

I ll test the patch tomorow.

Could somebody change hardware to All and Version to 2006.1, to reflect
reality? And yes, it would really be nice, if the patch could be applied, as
building still fails... :(

Created an attachment (id=110287) [details]
imagemagick-6.3.0.5-nolMagick.patch

reJ's patch from Comment  #4 works fine. Just in case it hasn't been adopted
because of qualms with using bare GNU sed in ebuilds, here's a Makefile patch
that does the same thing.

Confirmed that this problem still exists on 32-bit PPC and x86 on 2006.1, so I
changed the Hardware to All and the Version to unspecified. There are at least
two easy workarounds, so I changed the Severity to minor, too.

PerlMagick should be linked against whatever is in ../magick/.libs but not to
plain -lMagick because:

1) we already know that no Magick library is available in the cwd, but there
might be one in /usr/lib which it should *not* link against, ever.
2) there ought to be one in -L../magick/.libs. If not, something is horribly
wrong, and if there is, there is no need at all to specify -lMagick as well.

This bugs seems to be linked to bug #121142 in many ways. I hope the issue is
now resolved, by applying patch #110287. The fix is in the tree. Please test it
and reopen the bug as needed.

I've recompiled imagemagick-6.3.0.5 with the MAKEOPTS="-j2" and this works
indeed.
Thank you for fixing it.

At the moment, imagemagick may compile.

But the missing "-lMagick" causes, that the Magick.so (the perl-library)
contains a lot of uresolved symbols.

perl -MImage::Magick -e "new Image::Magick();" does not work anymore.
The perl-library is not linked against -lMagick, and therefor libMagick.so is
not loaded and therefor there are unresolved symbols.

*** Bug 168030 has been marked as a duplicate of this bug. ***

reopen wrt Comment #32

Created an attachment (id=111005) [details]
imagemagick-6.3.0.5-parallel-build.patch

patch, that adds dependency on $(LIBMAGICK) to perl-all.

I attached a patch.

I still have to modify the ebuild. After the patch, you have to run automake,
autoconf, etc. But unfortunatly, eautoreconf from autotool.eclass aborts with
an error. If i run things by hand, it works.


Anyway: after you do things by hand, you can run "make all-perl" and the
build-process with first build magick/.libs/libMagick.so and will then build
the perl-library. So after all i know about make, this should tell make to wait
until magick/.libs/libMagick.so is built before building the perl-library in
case of parallel build.

Created an attachment (id=111008) [details]
ebuild.patch

patch for the ebuild.

The ebuild simply runs autoreconf, since eautoreconf doesn't work :-(

In addition:
- removed a "einfo $S" command
- added a "-e config.sub"-test, because "chmod +x config.sub" caused a "file
not found"-error on my machine

Please my 2 patches! I'm pretty optimistic, that it fixed the parallel-build
problem without breaking the perl-module.

I've tested the 2 patches sven provided (as I originally alerted him to the
problem) and that they do fix imagemagick's perl module, and it built with -j2
for me.

*** Bug 168164 has been marked as a duplicate of this bug. ***

To avoid any further confusion: this never had anything to do with parallel
make.

If not parallel make: what else did cause libMagick.so not to exist in
../magick/.libs ?

In former versions of ImageMagick, Makefile.PL has always linked PerlMagick
with
  -L/usr/lib -L../magick/.libs -lMagick

The result was, that libMagick.so was searched in /usr/lib first, and then
found there, if ImageMagick was already installed. PerlMagick was not linked
against the newly compiled libMagick.so, but the old one in /usr/lib.
I worked on a patch for quite some time. I finally resolved the issue by
changing the order of the statements to this one:
  -L../magick/.libs -lMagick -L/usr/lib

In other words:

The -lMagick has _always_ been there. Just in case of an already installed
ImageMagick, the order of the -L parameters made a difference.


So as far as i can tell, "-L../magick/.libs -lMagick" is correct. It's a MUST
HAVE. Without it, unresolved symbols will occur in PerlMagick. And in some
comment, i read that "-j1" seems to work fine. Just in case of "-j2" or "-j3"
we get this strange error, that libMagick.so cannot be found.

For me, this indicates a parallel make problem.
But instead of fixing this parallel make problem, the -lMagick was simply
removed. That results in unresolved symbols.


So please correct me, if i'm wrong.
And if it still occurs with my patches, please provide the full output of the
make-process (emerge imagemagick 2>&1 | tee output.txt).

Well, someone revert the patch please and revbump the ebuild, keeping the
stable keywords there. 

As noted numerous time here and on other duplicates, this *was* a parallel make
problem, because it worked just fine w/ -j1 for *everyone*. Now we have a
broken app instead; so either fix it properly or force -j1 there meanwhile, but
the current patch is wrong.

Sven's patches worked for me. Thanks!

*** Bug 168544 has been marked as a duplicate of this bug. ***

(In reply to comment #44)
> Sven's patches worked for me. Thanks!

I contacted him by mail. He used -j3 to build imagemagick.

Created an attachment (id=111503) [details]
imagemagick-6.3.0.5-parallel-build.patch

New patch. Added $(LIBMAGICK) dependency to another point in
PerlMagick/Makefile.PL.am. It actually moved up one level in the dependency
hierarchie.

libMagick.so is also required for running "perl Makefile.PL". With the old
patch, it was not required. I was not satisfied with that.

Please: can somebody take care of this?

nolMagick-patch is still present in stable ebuild. imagemagick is still broken.

I'd like to add to the clamoring voices asking for the patch to be added to the
stable codebase.  I've been beating my head on this off and on for a couple
months now because I couldn't locate the bug here or a solution on the forums
(see http://forums.gentoo.org/viewtopic.php?p=3943402#3943402 for
cross-reference).

For reference, I have this problem on a P4 2GHz with -j3 and also on a dual P4
2GHz with -j5, both running latest stable gentoo.

I had the problem with the perl lib also. I hacked my ebuild to use the
parallel-build and ebuild patches from Sven and that fixed it. 

I compiled on a dual core Pentium D with MAKEOPTS=-j3 without error and "perl
-e ' use Image::Magick'" doesn't complain about a missing symbol anymore.

(In reply to comment #42)
> If not parallel make: what else did cause libMagick.so not to exist in
> ../magick/.libs ?

The fact that imagemagick's developers expect you to

1) unpack, configure, build install imagemagick
2) cd to the perlmagick subdir and build and install there too.

For upstream, this issue simply does not exist. For Gentoo, this problem has
existed ever since one or more Gentoo devs thought it would be great to have
one ebuild with USE=perl instead of an imagemagick and a perlmagick ebuild.
Doing the above two steps in one ebuild is impossible without patching the
source.

> In former versions of ImageMagick, Makefile.PL has always linked PerlMagick
> with
>   -L/usr/lib -L../magick/.libs -lMagick
> 
> The result was, that libMagick.so was searched in /usr/lib first, and then
> found there, if ImageMagick was already installed. PerlMagick was not linked
> against the newly compiled libMagick.so, but the old one in /usr/lib.
> I worked on a patch for quite some time. I finally resolved the issue by
> changing the order of the statements to this one:
>   -L../magick/.libs -lMagick -L/usr/lib

-L/usr/lib should not be in there. By the time perlmagick is built, the fresh
libMagick is not installed yet and you would be linking against the stale
libMagick that happens to be installed.

> In other words:
> 
> The -lMagick has _always_ been there. Just in case of an already installed
> ImageMagick, the order of the -L parameters made a difference.

Indeed. For upstream this makes sense, but not for Gentoo's recent imagemagick
ebuilds. I vote to split up imagemagick and perlmagick into two separate
packages again with RDEPEND="perl? ( ~media-gfx/perlmagick-${PV} )" or
something even better.

> 
> So as far as i can tell, "-L../magick/.libs -lMagick" is correct. It's a MUST
> HAVE. Without it, unresolved symbols will occur in PerlMagick. And in some
> comment, i read that "-j1" seems to work fine. Just in case of "-j2" or "-j3"
> we get this strange error, that libMagick.so cannot be found.

Very good of you to notice.

> For me, this indicates a parallel make problem.

For Gentoo, you might conclude that is the problem.

> But instead of fixing this parallel make problem, the -lMagick was simply
> removed. That results in unresolved symbols.

Sorry.

> So please correct me, if i'm wrong.

Part wrong, part right. I think you are right about the correct fix to this
bug.

I am going to commit your changes as they don't seem to kill puppies and I am
going to follow jakub's suggestion to revbump the current ebuild to force the
upgrade.

And then I am going to refrain from touching imagemagick and step back to just
wait until the graphics herd regains control. I hope they soon find some active
developers out there to help out with this bug and other media-gfx bugs.

(In reply to comment #51)
> (In reply to comment #42)
> > If not parallel make: what else did cause libMagick.so not to exist in
> > ../magick/.libs ?
> 
> The fact that imagemagick's developers expect you to
> 
> 1) unpack, configure, build install imagemagick
> 2) cd to the perlmagick subdir and build and install there too.
> 
> For upstream, this issue simply does not exist. For Gentoo, this problem has
> existed ever since one or more Gentoo devs thought it would be great to have
> one ebuild with USE=perl instead of an imagemagick and a perlmagick ebuild.
> Doing the above two steps in one ebuild is impossible without patching the
> source.

Some versions ago, i sent a patch upstream, that fixed the issue.
Before my patch, it was the way you said. Now it's different.
Here's an overview:

Old situation:
  make all => compiles imagemagick
  make install => installs imagemagick + compiles/installs perlmagick
  (linked against libMagick.so in /usr/lib)
Current situation:
  make all => compiles imagemagick + perlmagick
  make install => install imagemagick + perlmagick
  (linked against freshly compiled libMagick.so)


> > In former versions of ImageMagick, Makefile.PL has always linked PerlMagick
> > with
> >   -L/usr/lib -L../magick/.libs -lMagick
> > 
> > The result was, that libMagick.so was searched in /usr/lib first, and then
> > found there, if ImageMagick was already installed. PerlMagick was not linked
> > against the newly compiled libMagick.so, but the old one in /usr/lib.
> > I worked on a patch for quite some time. I finally resolved the issue by
> > changing the order of the statements to this one:
> >   -L../magick/.libs -lMagick -L/usr/lib
> 
> -L/usr/lib should not be in there. By the time perlmagick is built, the fresh
> libMagick is not installed yet and you would be linking against the stale
> libMagick that happens to be installed.

I never tried to remove -L/usr/lib. I assume, that the linker would complain
about other missing libraries. AFAIK, the linker searches the -L options from
left to right. So searching ../magick/.libs before /usr/lib is sufficient.

My parallel-build.patch has been applied upstream. It will be in version
6.3.3-1 released tomorrow.

I've verified the fix after compiling with -j3 on my single P4 system.

So what version do I need here, exactly (for the release) ?

(In reply to comment #55)
> So what version do I need here, exactly (for the release) ?

What do you mean?

(In reply to comment #55)
> So what version do I need here, exactly (for the release) ?

6.3.0.5-r1 is unbroken again...

Created an attachment (id=112657) [details]
imagemagick-6.3.0.5-parallel-build.patch (no need for autoreconf)

I've uploaded another version of the patch. It simply also patches Makefile.in,
so that we don't need to run autoreconf.

Well, the current ebuild is fine. Just in case that autoreconf causes problems
...

parallel-build.patch have been upstreamed and there's no longer any need for
autoreconf in 6.3.3. Please reopen this bug if it still applies to 6.3.3.