The bmptopnm that results from emerging media-libs/netpbm-10.39.0 gives a floating point exception when used on certain bmps. When I built netpbm-10.39.0 from scratch, the resulting bmptopnm did not have the problem on the same test bmp. Reproducible: Always Steps to Reproduce: 1. Emerge media-libs/netpbm-10.39.0 2. bmptopnm broken.bmp Actual Results: bmptopnm: Windows BMP, 262x128x16 bmptopnm: WRITING PPM IMAGE P6 262 128 255 Floating point exception Expected Results: A correct pnm on stdout. On the one I built from scratch, I only set "CFLAGS_SHLIB = -fPIC" in Makefile.config, after running config with all default answers.
Created attachment 126198 [details] Test bmp that causes the problem.
Created attachment 133100 [details, diff] Fixes dealing with 16-bit color depth bmp's The problem occurs when extractBitFields is called with a pixelformat that has a trn.mask of zero (it divides by the masks without checking them first). This seems like it will always happen when extractBitFields is called since trn.mask is defaulted to zero for pixelformat's, but it is only called for non-16-bit depths when the bitmap doesn't use conventional BGR format, which I guess is rare. It is always called for 16-bit depth bitmaps, though, so it looks like 16-bit bitmaps will always break. This patch fixes the problem by adding some checks so that extractBitFields won't do the divide by zero and also fixes the default rgb shifts and masks to what I think they should be for 16-bit color depth.
I found the difference between building from scratch and emerging: the default config when building from scratch used -O3 while I have -O2 in my make.conf. Any thoughts on why compiling with a higher optimization level would cause it to avoid a divide by zero? I am having trouble debugging it with optimizations so it is tough for me to see whether it might be no longer calling extractBitFields after compiling with -O3 for some reason.
(In reply to comment #3) > I found the difference between building from scratch and emerging: the default > config when building from scratch used -O3 while I have -O2 in my make.conf. > Any thoughts on why compiling with a higher optimization level would cause it > to avoid a divide by zero? NFC; CCing toolchain. Reproducible here w/ gcc-4.2.2.
before or after the proposed patch ... i'm not going to bother looking at a divide by 0 bug when it's already been fixed. whether or not it crashes based on compiling doesnt really matter -- it's dividing by 0 and it shouldnt be doing that.
latest ebuild is fixed
