Bug 172421 – media-libs/lib3ds-1.2.0 violates strict aliasing rules

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Anssi 2007-03-27 15:01:05 UTC]

The gcc-4.1 uses optimizations which break the version 1.2.0 of lib3ds
when compiled with higher than -O0. 

Reproducible: Always

Steps to Reproduce:
1. add -O2 to CFLAGS in /etc/make.conf
2. emerge lib3ds 
3. 3dsdump ./path/to/validfile.3ds
4. values (vertices) after each "point list:" are zeroes.
eg. 
....
 point list:
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000
....
Actual Results:  
Values in the dump were:
---clip---
 point list:
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000
    0.000000 0.000000 0.000000

----clip----

Expected Results:  
1. add -O2 to CFLAGS in /etc/make.conf
2. emerge lib3ds 
3. 3dsdump ./path/to/validfile.3ds
4. values (vertices) after each "point list:" contain also other values than zeroes. 
eg.
---clip---
point list:
    0.518930 -4.385780 3.989722
    0.418113 -4.384385 3.964804
    0.409068 -4.404268 3.972737
    0.447289 -4.366141 3.958578
    0.490817 -4.353148 3.955289
---clip---

The work-around should at least revert -O{123} optimization flag to -O0.

[Comment 1 SpanKY 2007-03-30 06:04:06 UTC]

post a valid .3ds file for us to test with

[Comment 2 Anssi 2007-03-30 07:21:52 UTC]

Created attachment 114913 [details]
A valid 3ds file representing a regular, yet very uninteresting, box.

[Comment 3 SpanKY 2007-03-31 03:11:11 UTC]

not a bug in the toolchain, lib3ds violates strict aliasing rules

building it with '-O3 -fno-strict-aliasing' "fixes" things for me

[Comment 4 SpanKY 2007-03-31 03:20:15 UTC]

looking at the code showed this should be pretty easy to fix so i did:
...
-  return(*((Lib3dsFloat*)&d));
+  memcpy(&ret, &d, sizeof(ret));
+  return ret;
...
-  d=*((Lib3dsDword*)&l);
+  memcpy(&d, &l, sizeof(d));
...

[Comment 5 SpanKY 2007-03-31 03:25:50 UTC]

looks like upstream cvs has fixed the read function but not the write ... filed bug report upstream