The accelerated atan function not only computes the wrong value on 32-bit platforms, but crashes on "mixed" bit machines (those with 64-bit pointers but 32-bit integers). Reproducible: Always Steps to Reproduce: Run toolame -e -s 48 -p 2 -b 224 shortfile.audio output.mp2 with the supplied audio file (assuming I can figure out how to attach it). Observe the segfault. Actual Results: Segfault. Expected Results: Not a segfault.
Created attachment 56727 [details] Audio file that causes problems with toolame
Created attachment 56728 [details, diff] The patch to solve the problem and other problems
The patch I included solves several problems: 1. it fixes a operand size issue with an error message. It casts to (int) what is normally (long). That is the fix in audio_read.c 2. It fixes the seg fault in atan_table by using 'unsigned int' instead of 'int'. The problem is that on 64-bit machines, index would get cast as an int and if x==0, then index=MIN_INT; That is a hugely negative number. On 32-bit machines, &atan_table[index] is the same as ((char *)atan_table)+(index*4). Of course index*4 of MIN_INT == 0, thus atan_table[index]==atan_table[0]. On a 64-bit machine, that does not happen. Instead invalid memory is accessed. Thus instead of checking if index<0, I chose to use unsigned integers. Because... 3. It fixes a bug in atan_table where if x==0, then index=0. That means atan would return 0 instead of 1.57, as it should have. By using an unsigned integer for index, you are assured that (unsigned int)(ATANSCALE * fabs(y/0)) will be MAX_INT instead of MIN_INT. The code on the next line will cap index where it should be (ATANSIZE-1), instead of 0.
in portage. thanks
http://sourceforge.net/tracker/?func=detail&aid=2798811&group_id=19022&atid=119022
