Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 516548 (PR61538) - [4.8/4.9] gcc-4.8+ __atomic_* builtins broken on R10000-based MIPS systems.
Summary: [4.8/4.9] gcc-4.8+ __atomic_* builtins broken on R10000-based MIPS systems.
Status: RESOLVED FIXED
Alias: PR61538
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: MIPS Linux
: Normal critical (vote)
Assignee: Gentoo Toolchain Maintainers
URL: https://gcc.gnu.org/PR61538
Whiteboard:
Keywords: REGRESSION, UPSTREAM
Depends on:
Blocks: gcc-4.8-stable 536874
  Show dependency tree
 
Reported: 2014-07-06 20:52 UTC by Joshua Kinard
Modified: 2015-02-22 18:37 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
WinMerge comparison of objdump disassembly of static 'sln' binary, __lll_lock_wait_private() function (gcc-futex-bug.png,59.08 KB, image/png)
2014-07-11 07:05 UTC, Joshua Kinard
Details
Backport of commit 0d18e650 to gcc-4.8.4 for PR61538 (gcc-4.8.4-mips-fix-pr61538.patch,2.23 KB, patch)
2015-02-18 12:29 UTC, Joshua Kinard
Details | Diff
Backport of commit 0d18e650 to gcc-4.9.2 for PR61538 (gcc-4.9.2-mips-fix-pr61538.patch,2.23 KB, patch)
2015-02-18 12:30 UTC, Joshua Kinard
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Kinard gentoo-dev 2014-07-06 20:52:56 UTC
I am finally creating a Gentoo bug to track the upstream gcc bug for this issue.

In a nutshell, in gcc-4.8.0 and greater, compiling any code with g++ and linking it to libpthread on an SGI R1x000-based system (SGI Octane, SGI Origin 200/2000, SGI Onyx2) will hang in a futex syscall.  I *think* it's some kind of double-locking bug or it's exposing some unknown erratum or quirk of the R10000 processor family.  I've verified this at least on both R12000 and R14000 CPUs, so it probably affects R10000 as well, as they're all pretty much the same.

One workaround I have found so far is to disable Linux futex support in gcc.  This allows a gcc-4.9.0 build to compile a simple conftest.c testcase with g++, link to libpthread, and not hang.  The other testcase I need to verify is in the latter phase of glibc's install, where it builds a static binary, 'sln', that sets up a couple of symlinks.  That binary also hangs if built with 4.8 or 4.9.  That test is running now, but will take a few hours to complete.

If that works, then the solution I've got for now, until someone from gcc upstream can look into the problem, is to pass --disable-linux-futex to our gcc if USE contains any of 'ip27', 'ip28', or 'ip30', which are already defined for sys-kernel/mips-sources.  This will cover our supported R10000-based SGI systems.
Comment 1 Joshua Kinard gentoo-dev 2014-07-09 10:35:16 UTC
The test failed.  gcc w/o linux-futex support can compile C++ with pthreads apps, but whatever it does to glibc still creates a static 'sln' binary that hangs when run, even without arguments, which prevents glibc's install phase from finishing.

I think I will have to bisect the gcc tree somehow, because I am getting absolutely no help or feedback from the gcc upstream staff.
Comment 2 Ryan Hill (RETIRED) gentoo-dev 2014-07-11 06:00:03 UTC
Have you tried comparing with the output of 4.7?  Kacper, you've done git bisects of gcc before.  Do you have any scripts you could share?

Though with a 13hr run cycle that might be your last resort. :p  If you're not doing it already you can cut that down to 1/3 by disabling bootstrap.
Comment 3 Joshua Kinard gentoo-dev 2014-07-11 07:03:07 UTC
I have, and using the statically-compiled 'sln' binary, I'd found some minor differences in the resulting ASM output (objdumped-asm output), but I have no idea why it's being caused.

Already started attempting to git bisect.  Not fun.  It took me a while to figure out how to force gcc's two remote branches of gcc-4_7-branch and gcc-4_8-branch to become local branches so that bisecting would actually work.  Got that first run built, and have been trying to recompile glibc as an unprivileged user w/ debugging, which is taking forever.

It looks like when you compile glibc outside of Portage, you trigger a compile failure that might be patched by this patch:
https://sourceware.org/ml/libc-alpha/2014-02/msg00507.html

But I don't know just yet.  80% of that patch applies on top of glibc-2.19, but until I complete a full build and can launch the install phase, where sln is built, I won't know for certain.  If this works, and 'sln' hangs, I can finally 'git bisect bad'.

Dunno about disabling bootstrap.  As long as a given bisection actually compiles, I am happy.  Since I don't know what leads to the cause, it's probably best to go through the full build (though I am only enabling the 'C' language, since I'll bet if I find the problem, it'll fix the C++ side of things, too).

I'll attach a PNG file showing the differences in the asm from glibc's __lll_lock_wait_private() function between gcc-4.7 and gcc-4.8.  I checked gcc-4.8 to gcc-4.9, and there is virtually no discernible change, so the problem was introduced somewhere between 4.7 and 4.8.

More interesting, is in gcc-4.7, we never even enter the __lll_lock_wait_private() function!  Only in 4.8, does the code path change such that we call into this function.  Not sure about that just yet...
Comment 4 Joshua Kinard gentoo-dev 2014-07-11 07:05:39 UTC
Created attachment 380558 [details]
WinMerge comparison of objdump disassembly of static 'sln' binary, __lll_lock_wait_private() function

Screenshot of WinMerge comparing the objdump disassembly from a static 'sln' binary disassembled on MIPS, one compiled by gcc-4.7 and the other by gcc-4.8.
Comment 5 Joshua Kinard gentoo-dev 2014-07-11 07:20:06 UTC
And, here's the replay of that function in gcc-4.8 in GDB, insn-by-insn, with a full register dump _after_ the execution of each instruction:

<START>
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x2            v1: 0x1              ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424428                                               ¦

                                                      (0x1)
B+>¦0x424428 <__lll_lock_wait_private+56>   ll     v0,0(s0)
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x1              ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424434                                               ¦


  >¦0x424434 <__lll_lock_wait_private+68>   beqzl  at,0x424428 <__lll_lock_wait_private+56>
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x1              ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x42443c                                               ¦
¦                                                                              ¦


  >¦0x42443c <__lll_lock_wait_private+76>   sync
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x1              ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424440                                               ¦


  >¦0x424440 <__lll_lock_wait_private+80>   bnez   v0,0x424410 <__lll_lock_wait_private+32>
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x1              ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424410                                               ¦


  >¦0x424410 <__lll_lock_wait_private+32>   0x7c03e83b  (rdhwr t,$cs)
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x25           a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424414                                               ¦


   ¦0x424414 <__lll_lock_wait_private+36>   li     a2,2
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0xd0d0dada     a2: 0x2            a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424418                                               ¦


   ¦0x424418 <__lll_lock_wait_private+40>   lw     a1,-29832(v1)
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0x0            a2: 0x2            a3: 0x8908feff       ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x42441c                                               ¦


   ¦0x42441c <__lll_lock_wait_private+44>   move   a3,zero
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x1            v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0x0            a2: 0x2            a3: 0x0              ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424420                                               ¦


   ¦0x424420 <__lll_lock_wait_private+48>   li     v0,4238
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x108e         v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0x0            a2: 0x2            a3: 0x0              ¦
¦t0: 0x475990       t1: 0x7efefeff     t2: 0x25252525     t3: 0x81010100       ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0xfff          k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x24        pc: 0x424424                                               ¦
¦                                                                              ¦


   ¦0x424424 <__lll_lock_wait_private+52>   syscall
+--Register group: general-----------------------------------------------------+
¦zero: 0x0          at: 0x1            v0: 0x108e         v1: 0x4ac490         ¦
¦a0: 0x4a416c       a1: 0x0            a2: 0x2            a3: 0x0              ¦
¦t0: 0x0            t1: 0x0            t2: 0x0            t3: 0x0              ¦
¦t4: 0x9            t5: 0x4a0000       t6: 0x1            t7: 0x4a3ad0         ¦
¦s0: 0x4a416c       s1: 0xffff9010     s2: 0x7fff5ddc     s3: 0x47598e         ¦
¦s4: 0x0            s5: 0x475974       s6: 0x4df6b8       s7: 0x4a2248         ¦
¦t8: 0x1f           t9: 0x4243f0       k0: 0x0            k1: 0x0              ¦
¦gp: 0x4ab030       sp: 0x7fff5870     s8: 0x7fff5870     ra: 0x43380c         ¦
¦status: 0x8004fdf3 lo: 0x999999a3     hi: 0x25           badvaddr: 0x43364c   ¦
¦cause: 0x20        pc: 0x424428                                               ¦


B+>¦0x424428 <__lll_lock_wait_private+56>   ll     v0,0(s0)
<HANGS HERE>


The syscall actually completes, it looks, as 'cause' is set to 0x20, which my "See MIPS Run" book says is correct for a syscall.  But I have to assume that there is something funny about the resulting registers that the 'll' insn dislikes, which hangs the app up.  What, I don't know.

There's an old erratum regarding the use of ll/sc and branch insns, where R10000-based processors require branch-likely insns instead (beqzl instead of beqz, bnel instead of bne, etc..), but that should be fully covered.  I wrote the original fix patch for gcc years ago, and Matt Turner got the glibc-side taken care of a few years ago as well.  Plus, my CPU's silicon revision should be corrected and not impacted by it.  But, I'm now not so sure if that's haunting me again or if I've stumbled into a new erratum.
Comment 6 Kacper Kowalik (Xarthisius) (RETIRED) gentoo-dev 2014-07-11 08:26:55 UTC
(In reply to Ryan Hill from comment #2)
> Have you tried comparing with the output of 4.7?  Kacper, you've done git
> bisects of gcc before.  Do you have any scripts you could share?
> 
> Though with a 13hr run cycle that might be your last resort. :p  If you're
> not doing it already you can cut that down to 1/3 by disabling bootstrap.

I was using something I've found here[1]. Since the original page doesn't open for me, I've dpasted my local copy of that blog entry[2]. I guess it will be hard to do automatic bisect if the symptom is the hanging...

[1] http://moxielogic.org/blog/?p=482
[2] https://paste.lugons.org/show/z2X4BkaxGvT7HnAvqXwz/
Comment 7 Joshua Kinard gentoo-dev 2014-07-11 08:39:55 UTC
(In reply to Kacper Kowalik (Xarthisius) from comment #6)
> I was using something I've found here[1]. Since the original page doesn't
> open for me, I've dpasted my local copy of that blog entry[2]. I guess it
> will be hard to do automatic bisect if the symptom is the hanging...
> 
> [1] http://moxielogic.org/blog/?p=482
> [2] https://paste.lugons.org/show/z2X4BkaxGvT7HnAvqXwz/

Funny, I was at that exact site a few days ago, reading up on bisecting gcc.  Site was working then, so must be maintenance problems.

And yeah, since the program hangs, that's harder to script around.  I suppose if I really wanted to script it, I'd rig up some kind of timer to wait a few seconds before deciding to kill it and flag the bisection as bad or not.

But it's probably better to handle this manually, since I am dealing with large, complex packages.
Comment 8 Joshua Kinard gentoo-dev 2014-07-11 11:23:12 UTC
(In reply to Joshua Kinard from comment #3)
> 
> It looks like when you compile glibc outside of Portage, you trigger a
> compile failure that might be patched by this patch:
> https://sourceware.org/ml/libc-alpha/2014-02/msg00507.html

Of course glibc has two copies of dns-hosts.c, one in resolv/, the other in glibc-compat/, and the second one fails with too few args to functions __libc_res_nsearch and __libc_res_nquery -_-

Does anyone know where glibc-compat comes from?  It appears to be separate from the main glibc source tree (i.e., not in glibc git), and the copy I found on our sources.g.o server is 6 years old.  Not a clue why it fails when built outside of Portage -- could be I missed a configure flag.  But it's cost me a day of trying to bisect gcc.
Comment 9 Joshua Kinard gentoo-dev 2014-07-14 00:35:57 UTC
5 bisects in, and I've narrowed it down somewhat.  gcc-4.8 from 20120626 doesn't work, while gcc-4.8 from 20120612 does work.  So some change there seems to be the trigger.  Already gone through a git log between those two date ranges, but nothing is sticking out as a possible cause, so I'll have to keep bisecting.
Comment 10 Joshua Kinard gentoo-dev 2014-07-21 07:21:00 UTC
I've isolated the problem down to these four commits on 2012-06-20:

1. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=39a8c5eaded1e5771a941c56a49ca0a5e9c5eca0
2. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=974f0a74e2116143b88d8cea8e1dd5a9c18ef96c
3. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=0f8e46b16a53c02d7255dcd6b6e9b5bc7f8ec953
4. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=30c3c4427521f96fb58b6e1debb86da4f113f06f

Reversing them solves the issue, but that's still not a fix.  I changed the component on the upstream GCC bug to a regression, so maybe that'll get some more eyeballs.
Comment 11 Joshua Kinard gentoo-dev 2014-08-12 09:04:21 UTC
(In reply to Joshua Kinard from comment #10)
> I've isolated the problem down to these four commits on 2012-06-20:
> 
> 1. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=39a8c5eaded1e5771a941c56a49ca0a5e9c5eca0
> 2. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=974f0a74e2116143b88d8cea8e1dd5a9c18ef96c
> 3. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=0f8e46b16a53c02d7255dcd6b6e9b5bc7f8ec953
> 4. https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=30c3c4427521f96fb58b6e1debb86da4f113f06f

I've eliminated #2 as the problem.  #4 depends on #1, as it basically reverts it a mere seven minutes later, going by the commit times.  And #3 is linked to #1 and #4.  Reversing all three makes gcc-4.8+ work right again, and is a much easier revert-patch to create since #2 added new asm constraints, but since the reverts aren't limited to just MIPS, I don't know if that's a good approach for us, meaning I still have to await an answer from someone upstream.

I assume this doesn't affect newer MIPS systems, but  don't have anything other than SGIs to test out.  So, not sure the best course of action here.  Reverting only for SGI systems would be hackish and ugly.  Reverting for all MIPS systems may harm efforts on hardware like Netlogic XLP, some of the code which is touched by said reversions.  I can't rule out someone is running Gentoo on those systems.
Comment 12 Joshua Kinard gentoo-dev 2014-09-30 03:16:33 UTC
It appears that the __atomic_* builtins added for MIPS in gcc-4.8 are broken, or don't work 100% on R10000-based MIPS systems (SGIs).  Specifically, atomic_exchange_acq() is used in the glibc locking code in __lll_lock_wait_private for futex support.  In gcc-4.7 and earlier, this function was a define that equated to a __sync_lock_test_and_set() call, which is an "acquire" memory operation.  So the equivalent __atomic_* builtin added in gcc-4.8 *should* have worked fine, but they don't.

On the kernel side, it appears that a call to freezable_schedule() is made, and never returns until the running test app is killed by ctrl+c.  Only then, does it exit back out of the kernel properly.  Uncertain why the program is either frozen permanently or not scheduled correctly.  It does point back to the way the user program is compiled, though, as a kernel built with gcc-4.7 (using the old __sync_* builtins) has the same problem as a 4.8+-built kernel.  Whereas a 4.8+-built kernel has no problems with a user program built with gcc-4.7 and earlier.

References:
https://gcc.gnu.org/wiki/Atomic/GCCMM
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/_005f_005fatomic-Builtins.html#_005f_005fatomic-Builtins
https://gcc.gnu.org/onlinedocs/gcc-4.6.3/gcc/Atomic-Builtins.html
http://stackoverflow.com/questions/13941385/using-gcc-atomic-builtins
Comment 13 Joshua Kinard gentoo-dev 2015-02-18 08:25:50 UTC
Running tests now, but it looks like this might have been fixed in upstream w/o anyone noticing the open PR.  This patch:
https://gcc.gnu.org/ml/gcc-patches/2014-11/msg02282.html

Which is commit 0d18e650 in the upstream master git was able to compile Python-3.3 fine.  I am now testing the glibc case to see if elf/sln runs w/o issue or not.  If so, we'll need to backport that patch to 4.9.2 and 4.8.4.  It looks like 4.9.3 will include the fix, and I haven't checked 4.8.x yet.
Comment 14 Joshua Kinard gentoo-dev 2015-02-18 12:14:21 UTC
(In reply to Joshua Kinard from comment #13)
> Running tests now, but it looks like this might have been fixed in upstream
> w/o anyone noticing the open PR.  This patch:
> https://gcc.gnu.org/ml/gcc-patches/2014-11/msg02282.html
> 
> Which is commit 0d18e650 in the upstream master git was able to compile
> Python-3.3 fine.  I am now testing the glibc case to see if elf/sln runs w/o
> issue or not.  If so, we'll need to backport that patch to 4.9.2 and 4.8.4. 
> It looks like 4.9.3 will include the fix, and I haven't checked 4.8.x yet.

Well, upstream closed the PR once I pointed out the patch.  They feel that is indeed the solution, and I can confirm that both of my observed testcases, Python's configure and glibc's elf/sln both work fine now under 4.9.2.

I'll attach patches shortly for 4.8.4 and 4.9.2.  I've only build-tested the 4.9.2 patch, but I can't imagine the 4.8.4 patch failing...
Comment 15 Joshua Kinard gentoo-dev 2015-02-18 12:29:26 UTC
Created attachment 396826 [details, diff]
Backport of commit 0d18e650 to gcc-4.8.4 for PR61538

This is the patch for our gcc-4.8.4.  Can this get pushed into a patchball soon?  This does not require a revbump, since the number of affected systems is so small.  I don't know if this will be fixed in the forthcoming gcc-4.8.5 or not.
Comment 16 Joshua Kinard gentoo-dev 2015-02-18 12:30:34 UTC
Created attachment 396828 [details, diff]
Backport of commit 0d18e650 to gcc-4.9.2 for PR61538

And this is the patch for our gcc-4.9.2.  This does not require a revbump, since the number of affected systems is so small.  This patch won't be needed in 4.9.3, since I know it is fixed there.
Comment 17 Anthony Basile gentoo-dev 2015-02-18 12:40:33 UTC
(In reply to Joshua Kinard from comment #16)
> Created attachment 396828 [details, diff] [details, diff]
> Backport of commit 0d18e650 to gcc-4.9.2 for PR61538
> 
> And this is the patch for our gcc-4.9.2.  This does not require a revbump,
> since the number of affected systems is so small.  This patch won't be
> needed in 4.9.3, since I know it is fixed there.

We can add these to 4.8.4 and 4.9.2 without rev bump --- just bump the patch version.  I can look at it and push it out if everything is okay.
Comment 18 Magnus Granberg gentoo-dev 2015-02-19 21:13:41 UTC
Gcc 4.8.4 and 4.9.2 gentoo patchset bumped to 1.2 with your fix
Please test.
Comment 19 Joshua Kinard gentoo-dev 2015-02-22 18:37:22 UTC
4.9.2 Cross-toolchain works for kernels, completed several stage runs on catalyst on two separate systems with no ill effects thus far.  Marking FIXED.