Bug 376867 – www-client/firefox fails when build with gcc 4.4.5 p1.3 ( patch included )

• Product: Gentoo Linux
• Status: RESOLVED WONTFIX

[Description Sergei Golubchik 2011-07-29 08:27:15 UTC]

firefox, when built with gcc 4.4.5 p1.3, doesn't work. Most often it silently exist when started. Sometimes it does start and crashes when clicking on a link.

Reproducible: Always

Steps to Reproduce:
1. make sure that gcc --version says "4.4.5 p1.3". Note that using gcc 4.4.5. p1.2 will NOT show the bug. Ebuild version is gcc-4.4.5 for BOTH.

2. emerge firefox (3.6.17) and xulrunner.

3. start it

Actual Results:  
firefox behaves as if it was /bin/false:

$ firefox
$

Expected Results:  
firefox should open a window

it all gets down to the function NS_InvokeByIndex_P() which invokes another function by emulating normal C ABI call. It puts arguments on the stack and in registers and then calls the function by pointer. First six arguments go into registers, the rest is pushed on the stack. Like that (xptcinvoke_x86_64_unix.cpp):

141     // Stack, if used, must be 16-bytes aligned
142     if (nr_stack)
143         nr_stack = (nr_stack + 1) & ~1;
...
145     // Load parameters to stack, if necessary
146     PRUint64 *stack = (PRUint64 *) __builtin_alloca(nr_stack * 8);
...
149     invoke_copy_to_stack(stack, paramCount, params, gpregs, fpregs);

Method nsComponentManagerImpl::RegisterFactoryLocation has seven arguments, thus one is pushed on the stack.

The code above doesn't work in gcc version mentioned above. nr_stack is 1, in line 142 it becomes 2. __builtin_alloca allocates two words. But RegisterFactoryLocation looks for the seventh argument in the wrong location, in the stack but two words too early. As a result, RegisterFactoryLocation gets garbage as mime type, fails to recognize it, and fails with most unhelpful NS_ERROR_OUT_OF_MEMORY (in nsComponentManager.cpp:2644). Even that error is only visible when compiled with USE=debug, otherwise firefox just fails silently.

as a workaround I emerged gcc 4.3.4 (different slot) and build firefox with that. That worked.

[Comment 1 Sergei Golubchik 2011-07-29 08:29:44 UTC]

Created attachment 281401 [details, diff]
builf firefox with USE=debug

After applying this patch one can build firefox with USE=debug

[Comment 2 Agostino Sarubbo 2011-07-29 18:20:51 UTC]

where is the build log of fail?

[Comment 3 Jory A. Pratt 2012-01-22 23:21:37 UTC]

amd64/x86 only have support for tb/fx-7.0 and newer, please feel free to reopen and update summary if you are still having issues.