When converting PDF files to JPEG, imagemagick's convert will crash in libjpeg.so.8. Reproducible: Always Steps to Reproduce: 1. convert Problem.pdf Problem.jpg Actual Results: SIGSEGV, stacktrace with media-libs/jpeg-8d: #0 0x0000000000000000 in ?? () #1 0x00007ffff3a5772e in jpeg_alloc (cinfo=0x7fffffff9f10, size=168, info=0x7ffff3ceee50 "JPEG small internal data allocation") at ./base/sjpegc.c:196 #2 0x00007ffff34894f7 in jinit_memory_mgr (cinfo=0x7fffffff9f10) at jmemmgr.c:1060 #3 0x00007ffff345d4ee in jpeg_CreateCompress (cinfo=0x7fffffff9f10, version=<optimized out>, structsize=584) at jcapimin.c:59 #4 0x00007ffff0611e02 in WriteJPEGImage (image_info=0x635d10, image=0x678960) at coders/jpeg.c:2047 #5 0x00007ffff79bdbfc in WriteImage (image_info=<optimized out>, image=0x678960) at magick/constitute.c:1229 #6 0x00007ffff79be170 in WriteImages (image_info=<optimized out>, images=<optimized out>, filename=<optimized out>, exception=0x605610) at magick/constitute.c:1378 #7 0x00007ffff76558ba in ConvertImageCommand (image_info=0x60a510, argc=<optimized out>, argv=0x6022c0, metadata=0x0, exception=0x605610) at wand/convert.c:3049 #8 0x00007ffff76c1b9f in MagickCommandGenesis (image_info=0x605890, command=0x400840 <ConvertImageCommand@plt>, argc=5, argv=0x7fffffffd968, metadata=0x0, exception=<optimized out>) at wand/mogrify.c:161 #9 0x00000000004008b7 in ConvertMain (argv=0x7fffffffd968, argc=5) at utilities/convert.c:81 #10 main (argc=<optimized out>, argv=0x7fffffffd968) at utilities/convert.c:92 Expected Results: Produce a correct JPEG file from the PDF input. - This happens with all PDF files I've tried so far. - It happens with both media-libs/jpeg and media-libs/libjpeg-turbo - It only appears to happen with direct .pdf -> .jpg conversion. I.e., the following sequence works without crash: convert Problem.pdf Problem.png && convert Problem.png Problem.jpg - The same crash also occurs when performing the same conversion using libMagickWand from custom software. - As far as I can reconstruct, the erroneous behaviour started between March 20 and today, with no upgrades/installs in between (nothing in emerge.log). On March 20, I did a number of PDF -> JPEG conversions without any problems; I've checked some of the files that used to work on the 20th, those break today. I'll attach one problematic PDF file. Just now I'm running some upgrades, which also switches my system back to libjpeg-turbo. I'll post the stacktrace against libjpeg-turbo once the required builds have finished.
Created attachment 343441 [details] PDF file crashing for me
Please post your `emerge --info' output in a comment.
I'll add the 'emerge --info' once the updates to current stable are finished, and I reproduced the issue against libjpeg-turbo again. In the meantime, there's an interesting observation which corroborates the "only crashes when converting from PDF in one step" point: The last symbol shown in the stacktrace, 0x00007ffff3a5772e in jpeg_alloc at ./base/sjpegc.c:196 actually comes from /usr/lib64/libgs.so.9: 0x00007ffff3a09c20 - 0x00007ffff3ce8fe8 is .text in /usr/lib64/libgs.so.9 My current hypothesis is that once ImageMagick uses ghostscript (via libgs) to accomplish something, gs will install its own memory handling for the instance of libjpeg that gs, in turn, uses. This memory handling setup survives after gs has finished its task. When - then - IM uses libjpeg, it still expects ghostscript's memory management to be available - which no longer is the case. Reminds me of the issue described here; the outer circumstances certainly fit: http://blog.flameeyes.eu/2011/01/hide-those-symbols
Stacktrace with libjpeg-turbo, same problem: #0 0x0000000000000000 in ?? () #1 0x00007ffff3a5772e in jpeg_alloc (cinfo=0x7fffffff9f30, size=168, info=0x7ffff3ceee50 "JPEG small internal data allocation") at ./base/sjpegc.c:196 #2 0x00007ffff346cb77 in jinit_memory_mgr (cinfo=0x7fffffff9f30) at jmemmgr.c:1092 #3 0x00007ffff34477de in jpeg_CreateCompress (cinfo=0x7fffffff9f30, version=<optimized out>, structsize=584) at jcapimin.c:59 #4 0x00007ffff0c97e02 in WriteJPEGImage (image_info=0x60e790, image=0x676780) at coders/jpeg.c:2047 #5 0x00007ffff79bdbfc in WriteImage (image_info=<optimized out>, image=0x676780) at magick/constitute.c:1229 #6 0x00007ffff79be170 in WriteImages (image_info=<optimized out>, images=<optimized out>, filename=<optimized out>, exception=0x605610) at magick/constitute.c:1378 #7 0x00007ffff76558ba in ConvertImageCommand (image_info=0x60a510, argc=<optimized out>, argv=0x605540, metadata=0x0, exception=0x605610) at wand/convert.c:3049 #8 0x00007ffff76c1b9f in MagickCommandGenesis (image_info=0x605890, command=0x400840 <ConvertImageCommand@plt>, argc=3, argv=0x7fffffffd988, metadata=0x0, exception=<optimized out>) at wand/mogrify.c:161 #9 0x00000000004008b7 in ConvertMain (argv=0x7fffffffd988, argc=3) at utilities/convert.c:81 #10 main (argc=<optimized out>, argv=0x7fffffffd988) at utilities/convert.c:92 Again, 0x00007ffff3a5772e in jpeg_alloc at ./base/sjpegc.c:196 points to: 0x00007ffff3a09c20 - 0x00007ffff3ce8fe8 is .text in /usr/lib64/libgs.so.9
# emerge --info Portage 2.1.11.55 (default/linux/amd64/13.0/desktop/kde, gcc-4.6.3, glibc-2.15-r3, 3.7.10-gentoo x86_64) ================================================================= System uname: Linux-3.7.10-gentoo-x86_64-Intel-R-_Core-TM-_i7_CPU_970_@_3.20GHz-with-gentoo-2.1 KiB Mem: 12300564 total, 8012672 free KiB Swap: 16777212 total, 16295584 free Timestamp of tree: Wed, 27 Mar 2013 13:45:01 +0000 ld GNU ld (GNU Binutils) 2.22 distcc 3.1 x86_64-pc-linux-gnu [enabled] ccache version 3.1.9 [disabled] app-shells/bash: 4.2_p37 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.7.3-r3, 3.2.3-r2 dev-util/ccache: 3.1.9 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo LOCAL jmbreuer steam-overlay ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=corei7 -mtune=corei7 -O2 -pipe -momit-leaf-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb" CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /usr/share/X11/xkb" CXXFLAGS="-march=corei7 -mtune=corei7 -O2 -pipe -momit-leaf-frame-pointer" DISTDIR="/storage/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps y --jobs=64 --load-average=16" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg collision-protect config-protect-if-modified distcc distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/ http://gentoo.mneisen.org/ http://gentoo.wheel.sk/ http://gentoo.mirror.pw.edu.pl/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="--jobs=64 --load-average=18" PKGDIR="/storage/portage/packages/x64cd" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/jmbreuer /var/lib/layman/steam" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="X Xaw3d a52 aac acl acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr chardet cjk cli consolekit cracklib crypt css cups curl cxx dbus declarative dga dri dts dv dvd dvdr embedded emboss encode exif fam fame fastcgi ffmpeg firefox flac flash fluidsynth fontconfig foomaticdb fortran gcj gd gdbm gif gimp glitz glut gnutls gphoto2 gpm gstreamer gtk iconv icu ieee1394 imap imlib innodb iphone ipod ipv6 java jce jpeg kde kerberos kipi kpathsea lame lastfm lcms ldap leim libgda libnotify libsamplerate lm_sensors lzo mad mbox mikmod mjpeg mmx mmxext mng modplug modules mono mp3 mp4 mpeg mtp mudflap mule multilib ncurses network network-cron nis nls nptl nsplugin nspluginwrapper nvidia offensive ogg openexr opengl openmp pam pango pcap pcre pda pdf phonon plasma plotutils png policykit postscript ppds qt3support qt4 rdesktop readline rtc samba sasl scanner sdk sdl semantic-desktop session shout slang slp smp sndfile socks5 sound sox spell sqlite sse sse2 ssl ssse3 startup-notification subversion svg tcpd theora threads tiff tk truetype trusted udev udisks unicode upower usb vdpau vim-syntax vnc vorbis webdav wma wmf wxwidgets x264 xanim xcb xcomposite xiph xml xscreensaver xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse wacom evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en de" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="net" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
I am experiencing a problem that seems to be suspiciously too similar. I would risk saying that the problem really lies in ghostscript-gpl, not in imagemagick or libjpeg-turbo. I usually do have two PDF viewers around: Evince and zathura. I have several zathura plugins built, including Adobe PostScript (via ghostscript) and Adobe PDF (via libpoppler). When doing a new install on another machine, I decided to largely mirror my current install. So, except for a few single packages I removed, I just did what could be said to be an "emerge -e world" of one of my older computers. But, in the end, zathura crashes a *lot* in the new system. I did not confirm if this was indeed the cause, but today all crashes were caused while allocating memory on request from libjpeg.so. I will attach a backtrace (lacking debugging symbols, but apparently good enough to see the problem). Interestingly, Evince does not crash. Take, for example, the PDF paper at http://www.klofas.com/papers/CommSurvey-Bryan_Klofas.pdf Reading this in zathura succeeds until you hit the first page with an image (which, at this point, I can surely say to be a JPEG stream, without needing to inspect the PDF file), at which point zathura crashes. The crash happens after a call to jpeg_alloc() in libgs.so. Functions before this in the call stack are from libjpeg.so. Now, after finding this bug and reading Diego's blogpost, I presumed this was a clash between libjpeg-turbo and ghostscript-gpl. I unmerged app-text/zathura-ps, the postscript handler for zathura. And, after that, it didn't crash anymore. More: I mentioned I had mostly mirrored the installed packages from another machine (just by copying the world file). The interesting thing is that my other Gentoo installs do not have this issue. The only difference between the installs, other than that I removed some of the packages and that I've still not configured any overlays (I will try to check if anything from the overlays is causing issues), is the CPU manufacturer: all CPUs are amd64 (as well as the installs), but the installs where I can't observe this issue are AMD CPUs, the one where I get the crash is an Intel Core i7. If I manage to get time for that, I may try to run emerge -e world on one of my older installs -- it will take time, but it *should* point out whether the problem was introduced by a newer release of GCC or any other toolchain component (in fact, I've already tried recompiling ghostscript, libjpeg-turbo and libpoppler, but I still don't get any crash). I'll also try to run convert on that machine.
Created attachment 350674 [details] Backtrace of zathura crash caused by ghostscript when unpacking JPEG stream in libpoppler. backtrace lacking some of the debugging information (I'll post a more accurate one later) -- but #0 *is* jpeg_alloc() from libgs.so
There seem to have been several problems over time with Ghostscript bundling its own version of libjpeg, with its own memory management system: - Ubuntu: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/182623 - Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653061 - (a very old one) from Fedora Core: https://www.redhat.com/archives/fedora-maintainers/2006-November/msg00194.html And there is at least one report at GS' own bug tracker: - http://bugs.ghostscript.com/show_bug.cgi?id=692891 Which mentions a hack, introduced more than one year ago: http://git.ghostscript.com/?p=ghostpdl.git;a=commitdiff;h=ceef32 but which does not seem to be in the amd64 stable version of ghostscript. If possible, I will try to 1) Reproduce the crash with imagemagick, just to, well, be a bit more on-topic; 2) Upgrade ghostscript-gpl and check whether a newer version prevents the crash.
Created attachment 350746 [details] Backtrace of zathura crash caused by ghostscript when unpacking JPEG stream in libpoppler. Backtrace with libjpeg and libgs debug symbols.
I can reproduce the crash with imagemagick, converting the same PDF that crashes zathura to JPEG. USE flags for imagemagick are as follows: (X bzip2 cxx djvu jpeg openmp png postscript raw svg tiff wmf zlib -autotrace -corefonts -fftw -fontconfig -fpx -graphviz -hdri -jbig -jpeg2k -lcms -lqr -lzma -opencl -openexr -pango -perl -q32 -q64 -q8 -static-libs -test -truetype -webp -xml) (Please note that USE=postscript is, of course, necessary to observe this crash.) I can also confirm that =ghostscript-gpl-9.06 avoids the crash (I'd guess it is the change from upstream I linked earlier).
# emerge --info Portage 2.1.11.62 (default/linux/amd64/13.0, gcc-4.6.3, glibc-2.15-r3, 3.8.13-gentoo x86_64) ================================================================= System uname: Linux-3.8.13-gentoo-x86_64-Intel-R-_Core-TM-_i7_CPU_Q_720_@_1.60GHz-with-gentoo-2.2 KiB Mem: 8167940 total, 5664092 free KiB Swap: 4094972 total, 4094972 free Timestamp of tree: Mon, 10 Jun 2013 13:00:01 +0000 ld GNU ld (GNU Binutils) 2.22 distcc 3.1 x86_64-pc-linux-gnu [disabled] app-shells/bash: 4.2_p45 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.7.3-r3, 3.2.3-r2 dev-util/cmake: 2.8.10.2-r2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.12.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.81 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo x-portage ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=native" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://trumpetti.atm.tut.fi/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9 --load-average=9" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acl alsa amd64 berkdb bindist bzip2 cli cracklib crypt cups cxx djvu dri dvi emacs fortran gdbm gpm iconv ipv6 mmx modules mudflap multilib ncurses nls nptl opengl openmp pam pcre ppp readline session sse sse2 ssl tcpd unicode zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse synaptics evdev aiptek" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en pt fi" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="vesa nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Obsolete version, please reopen if still exists with current versions
