Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 510292 - =media-libs/{libjpeg-turbo-1.3.1,jpeg-8d-r1}: CFLAGS="-ftree-vectorize" causing segfault, possible toolchain problem(?)
Summary: =media-libs/{libjpeg-turbo-1.3.1,jpeg-8d-r1}: CFLAGS="-ftree-vectorize" causi...
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo Toolchain Maintainers
URL: http://sourceforge.net/p/libjpeg-turb...
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2014-05-14 08:30 UTC by Marco Ziebell
Modified: 2019-08-20 22:39 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
novectorize patch (novectorize.patch,1.13 KB, patch)
2014-05-14 13:33 UTC, Marco Ziebell
Details | Diff
media-libs/jpeg-8d no vectorize patch (media_libs_jpeg_8d_novectorize.patch,1.38 KB, patch)
2014-05-14 20:15 UTC, Marco Ziebell
Details | Diff
emerge --info (emerge.info,16.02 KB, application/x-info)
2014-05-15 14:03 UTC, Marco Ziebell
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marco Ziebell 2014-05-14 08:30:46 UTC
If I compile libjpeg-turbo with specific CFLAGS, evince fails to open an PDF and causes an Segmentation fault.
Changing or removing any of the CFLAG

the CFLAGS="-march=native -O2 -ftree-vectorize"
the PDF= "http://glenngreenwald.net/pdf/NoPlaceToHide-Documents-Compressed.pdf"
evince version: 3.10.3

backtrace

#0  0x00007fffe821fbe7 in ?? () from /usr/lib64/libjpeg.so.62
#1  0x00007fffe822b0e7 in ?? () from /usr/lib64/libjpeg.so.62
#2  0x00007fffe82232b4 in ?? () from /usr/lib64/libjpeg.so.62
#3  0x00007fffe821c552 in jpeg_read_scanlines () from /usr/lib64/libjpeg.so.62
#4  0x00007fffd9b0f579 in DCTStream::getChars(int, unsigned char*) () from /usr/lib64/libpoppler.so.44
#5  0x00007fffd9aa593b in ImageStream::getLine() () from /usr/lib64/libpoppler.so.44
#6  0x00007fffe849c609 in RescaleDrawImage::getRow(int, unsigned int*) () from /usr/lib64/libpoppler-glib.so.8
#7  0x00007fffe849cedc in CairoRescaleBox::downScaleImage(unsigned int, unsigned int, int, int, unsigned short, unsigned short, unsigned short, unsigned short, _cairo_surface*) ()
   from /usr/lib64/libpoppler-glib.so.8
#8  0x00007fffe849aac1 in CairoOutputDev::drawImage(GfxState*, Object*, Stream*, int, int, GfxImageColorMap*, bool, int*, bool) () from /usr/lib64/libpoppler-glib.so.8
#9  0x00007fffd9a52a72 in Gfx::doImage(Object*, Stream*, bool) () from /usr/lib64/libpoppler.so.44
#10 0x00007fffd9a53726 in Gfx::opXObject(Object*, int) () from /usr/lib64/libpoppler.so.44
#11 0x00007fffd9a4ea0e in Gfx::go(bool) () from /usr/lib64/libpoppler.so.44
#12 0x00007fffd9a4eea8 in Gfx::display(Object*, bool) () from /usr/lib64/libpoppler.so.44
#13 0x00007fffd9a9bb16 in Page::displaySlice(OutputDev*, double, double, int, bool, bool, int, int, int, int, bool, bool (*)(void*), void*, bool (*)(Annot*, void*), void*, bool) ()
   from /usr/lib64/libpoppler.so.44
#14 0x00007fffe8488b9f in ?? () from /usr/lib64/libpoppler-glib.so.8
#15 0x00007fffe86b7b1c in ?? () from /usr/lib64/evince/4/backends/libpdfdocument.so
#16 0x00007fffe86b8734 in ?? () from /usr/lib64/evince/4/backends/libpdfdocument.so
#17 0x00007ffff7742d69 in ?? () from /usr/lib64/libevview3.so.3
#18 0x00007ffff775b35a in ?? () from /usr/lib64/libevview3.so.3
#19 0x0000003f35879ef5 in ?? () from /usr/lib64/libglib-2.0.so.0
#20 0x0000003f32c080ba in start_thread () from /lib64/libpthread.so.0
#21 0x0000003f324ea3ed in clone () from /lib64/libc.so.6


I'm not sure if this is a fault in the library, a very unlinkely compiler bug or just an very stupid attitude of mine.

Reproducible: Always
Comment 1 Samuli Suominen (RETIRED) gentoo-dev 2014-05-14 09:00:44 UTC
-ftree-vectorize for example on x86 (32bit) breaks zlib with certain compiler versions, i've never considered it a safe flag to use

does it go away if 'inherit flag-o-matic' and 'append-cflags -fno-tree-vectorize' is added to libjpeg-turbo's ebuild (before econf)?

also, post emerge --info
Comment 2 Marco Ziebell 2014-05-14 13:33:48 UTC
Created attachment 376900 [details, diff]
novectorize patch

your suggestion works fine so i propose a patch for the ebuild.
Comment 3 Marco Ziebell 2014-05-14 13:38:00 UTC
What's still strage is that removing "-march=native" or "-O2" also solves the problem. 
Could be the way "where and when" gcc applies vectoriztion. 
Even if, patch works and thanks for your quick response.
Comment 4 Marco Ziebell 2014-05-14 20:15:26 UTC
Created attachment 376932 [details, diff]
media-libs/jpeg-8d no vectorize patch

After a upstream bug report and some further research suggested by them i can commit an other patch for media-libs/jpeg-8d.

https://sourceforge.net/p/libjpeg-turbo/bugs/65/

both patches should work, should I open an other bugreport?
Comment 5 Samuli Suominen (RETIRED) gentoo-dev 2014-05-14 21:10:47 UTC
are you really seeing same problem with libjpeg.so.8 from jpeg-8d? that'd be just weird, it's very different code and without asm of anykind

this is starting to sound like a compiler bug more and more

please just drop -ftree-vectorize for now

CCing toolchain@g.o for comment howto proceed with the debugging
Comment 6 Samuli Suominen (RETIRED) gentoo-dev 2014-05-14 21:12:05 UTC
(In reply to Samuli Suominen from comment #1)
> also, post emerge --info

also you missed this, to see what compiler you are using

post `emerge -pv sys-devel/gcc --nodeps` too
Comment 7 Samuli Suominen (RETIRED) gentoo-dev 2014-05-14 21:15:09 UTC
does upgrading to gcc latest from 4.8* and if that doesn't work, to binutils latest from series of 2.24*

using gcc-config to switch to 4.8*
using possible binutils-config to switch to ld.gold from ld.bfd

-march=native with certain processors doesn't play well, the solution is often gcc upgrade
Comment 8 Samuli Suominen (RETIRED) gentoo-dev 2014-05-14 21:15:39 UTC
(In reply to Samuli Suominen from comment #5)
> are you really seeing same problem with libjpeg.so.8 from jpeg-8d?

i see now from the upstream bug you filed that they could indeed be same.
Comment 9 Ryan Hill (RETIRED) gentoo-dev 2014-05-14 23:34:20 UTC
Need emerge --info.  

Also, `echo "" | gcc -march=native -v -E - 2>&1 | grep cc1`
Comment 10 Marco Ziebell 2014-05-15 14:02:21 UTC
the compiler information:

sys-devel/gcc-4.7.3-r1:4.7  USE="cxx fortran gcj graphite mudflap (multilib) nls nptl openmp (-altivec) -awt -doc (-fixed-point) -go (-hardened) (-libssp) -multislot -nopie -nossp -objc -objc++ -objc-gc -regression-test -vanilla"


/usr/libexec/gcc/x86_64-pc-linux-gnu/4.7.3/cc1 -E -quiet -v - -march=bdver2 -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mno-bmi2 -mtbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mlzcnt -mno-rdrnd -mf16c -mno-fsgsbase --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=1024 -mtune=bdver2


Tests with higher toolchain version is in progress
Comment 11 Marco Ziebell 2014-05-15 14:03:02 UTC
Created attachment 377008 [details]
emerge --info
Comment 12 Marco Ziebell 2014-05-15 17:03:18 UTC
Switching to sys-devel/gcc-4.8.2 , sys-devel/binutils-2.24-r2 and ld.gold did solve the problem.
Comment 13 Ryan Hill (RETIRED) gentoo-dev 2014-05-16 03:37:32 UTC
Cool.  Can you switch back to gcc 4.7 and see if it still fails.  I seem to remember some bdver2 bugs but I thought they were fixed in 4.7...
Comment 14 Samuli Suominen (RETIRED) gentoo-dev 2014-05-16 10:41:42 UTC
(In reply to MJJoker from comment #12)
> Switching to sys-devel/gcc-4.8.2 , sys-devel/binutils-2.24-r2 and ld.gold
> did solve the problem.

(In reply to Ryan Hill from comment #13)
> Cool.  Can you switch back to gcc 4.7 and see if it still fails.  I seem to
> remember some bdver2 bugs but I thought they were fixed in 4.7...

^ This is what I meant too, test one by one, not all at once to narrow the problem down.
Was it gcc upgrade from 4.7 to 4.8, or binutils from 2.23 to 2.24, or binutils ld.bfd to ld.gold, or what, specifically, made the problem go away?
Comment 15 Marco Ziebell 2014-05-19 16:45:16 UTC
My Mistake. 
switching to the next version of GCC and binutils and ld.gold DID NOT SOLVE the problem.

I made some further test over the weekend, switching back to gcc 4.7.3 or ld.bfd didn't change it.

The segmentation fault is still occurring!
Comment 16 Samuli Suominen (RETIRED) gentoo-dev 2014-06-04 13:58:10 UTC
toolchain: 

should I by adding 'append-cflags -fno-tree-vectorize' to jpeg and libjpeg-turbo ebuilds to prevent users from breaking it like evident in this bug?

my position currently is that if gcc doesn't enable tree vectorize by default for given arch, it's likely not safe, and it should most definately not be used like this globally like the user is using in this bug

i feel inclined to close this as resolved, worksforme, solved by removing user defined -ftree-vectorize from make.conf
Comment 17 Samuli Suominen (RETIRED) gentoo-dev 2014-06-04 13:59:44 UTC
i mean, i don't like the idea of adding any append-flags -fno-tree-vectorize since that would actually disable tree vectorize for arches/gcc versions where it's enabled by default
Comment 18 Ryan Hill (RETIRED) gentoo-dev 2014-06-05 05:33:47 UTC
4.7.4 is about to be released and then the branch will be closed.  So I think the best thing to do is check if it's fixed there, and if not add

  [[ $(gcc-version) -eq 4.7 ]] && append-flags -fno-tree-vectorize

The reason being that -ftree-vectorize is enabled by -O3 in 4.7, which is something we should try to support when we can.
Comment 19 Marco Ziebell 2014-06-05 10:23:08 UTC
looking at the fixes in 4.7.4 I see a lot of vectorize/O3/instruction error ... waiting for 4.7.4 seems like a reasonable thing to do.

As soon as 4.7.4 arrives in the tree I will continue testing!

https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED&resolution=FIXED&target_milestone=4.7.4

thanks so far.
Comment 20 Ryan Hill (RETIRED) gentoo-dev 2014-06-17 04:57:08 UTC
4.7.4 is now in the tree.
Comment 21 Marco Ziebell 2014-06-23 11:19:46 UTC
I'm facing serious issues with the latest stable version of gentoo-sources which makes it impossible for me to compile gcc ...
It will take a few more days, but I'm still on it


Thanks in advancedIT will
Comment 22 Marco Ziebell 2014-07-01 12:39:03 UTC
The segfault still appears with gcc-4.7.4 and gcc-4.8.3 in combination with binutils-2.32.2.
I'm not sure how to continue.
- the pdf behind the link which segfaults  was replaced with a new version. The new version still fails to open with -ftree-vectorize
- the segfault appears in an codesegment which is ancient ... libjpeg-turbo took it from libjpeg



... the bug still seems to be fishy ... I will try some further research but can't promise anything.
Comment 23 Sergei Trofimovich (RETIRED) gentoo-dev 2019-02-13 22:45:41 UTC
Does it still happen on modern toolchain?
Comment 24 Sergei Trofimovich (RETIRED) gentoo-dev 2019-08-20 22:39:48 UTC
I was not able to reproduce locally. Assuming fixed.