Bug 732206 – sys-libs/newlib-3.3.0-r1 won't compile for spu-elf target due to thread errors

• Product: Gentoo Linux
• Status: RESOLVED OBSOLETE

[Description Michał Dec 2020-07-11 14:00:08 UTC]

Compiling the toolchain for spu-elf with crossdev will produce an incomplete toolchain due to failure to compile newlib. Further investigation into the error strongly suggests there is something wrong going on with how newlib handles pthreads. Despite disabling USE=threads, the problem persists.

Reproducible: Always

Steps to Reproduce:
1. Run `crossdev -S --g 9.3.0 --l 3.3.0-r1 -oO /usr/portage -t spu-elf` on any host. Just remember /usr/portage has to be valid or change it to where you keep the Gentoo overlay locally.
2. Depending on the hardware prepare to wait. On PS3, the platform where this compiler is a must-have, it will take some 3 hours. On a high-end amd64 it shouldn't take more than 10 minutes.
3. Read the logs and despair.
Actual Results:  
I got binutils and gcc but no libc for SPUs.

Expected Results:  
Crossdev should produce a full working toolchain for SPUs.

This has been reproduced by the reporter on ppc64 (PS3) and amd64 with the same effect.

[Comment 1 Michał Dec 2020-07-11 14:02:07 UTC]

Created attachment 648906 [details]
Compilation log of newlib for spu-elf on amd64

[Comment 2 Michał Dec 2020-07-11 14:03:08 UTC]

Created attachment 648910 [details]
Compilation log of newlib for spu-elf on ppc64 (PS3)

[Comment 3 Michał Dec 2020-07-11 15:02:19 UTC]

I have reconstructed newlib-2.5.0.ebuild from https://gitweb.gentoo.org/repo/gentoo.git/diff/sys-libs/newlib/newlib-2.5.0.ebuild?id=ae8fe4521b047d71437f98218ff595715f41cfc9 and used it as a toolchain ingredient.

This seems to have mitigated the compilation failure.

[Comment 4 Michał Dec 2020-07-11 15:59:13 UTC]

This bug is also reproducible with:
- newlib-2.5.0.20171222
- newlib-2.5.0.20170922
- newlib-2.5.0.20170818
- newlib-2.5.0.20170720
- newlib-2.5.0.20170623

The latest newlib-2.5.0 that seems to be working is newlib-2.5.0.20170519.

[Comment 5 Michał Dec 2020-07-11 23:57:43 UTC]

I can't remember how much time I spent hacking the 3.3.0-r1 ebuild but for what it's worth here's the diff between last working and first broken version https://github.com/grepwood/newlib/commit/831b6666af1f83ac173b491a5c191246203ddc37

Maybe if more eyes are set on this, we can find the offending code.

[Comment 6 Sergei Trofimovich (RETIRED) 2020-07-14 22:20:45 UTC]

Let's post build error for completeness:

"""
spu-elf-gcc -B/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/build/spu-elf/mea64/newlib/ -isystem /var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/build/spu-elf/mea64/newlib/targ-include -isystem /var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/include -B/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/build/spu-elf/mea64/libgloss/spu -L/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/build/spu-elf/mea64/libgloss/libnosys -L/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/libgloss/spu  -mea64 -DPACKAGE_NAME=\"newlib\" -DPACKAGE_TARNAME=\"newlib\" -DPACKAGE_VERSION=\"3.3.0\" -DPACKAGE_STRING=\"newlib\ 3.3.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I. -I/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/argz -fno-builtin -D_POSIX_MODE -DREENTRANT_SYSCALLS_PROVIDED -DMISSING_SYSCALL_NAMES -ffunction-sections -fdata-sections  -D__NO_SYSCALLS__ -DMISSING_SYSCALL_NAMES -DHAVE_INIT_FINI      -O2 -pipe -ggdb -ffunction-sections -fdata-sections -U_FORTIFY_SOURCE  -mea64 -c -o lib_a-argz_add_sep.o `test -f 'argz_add_sep.c' || echo '/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/argz/'`argz_add_sep.c
,,,
In file included from /var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/include/sys/types.h:223,
                 from /var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/argz/argz_add_sep.c:8:
/var/tmp/portage/cross-spu-elf/newlib-3.3.0-r1/work/newlib-3.3.0/newlib/libc/include/sys/_pthreadtypes.h:68:22: error: field ‘schedparam’ has incomplete type
   68 |   struct sched_param schedparam;
      |                      ^~~~~~~~~~
"""

[Comment 7 Sergei Trofimovich (RETIRED) 2020-07-14 22:31:36 UTC]

Note that spu-elf target was removed in gcc-10 release. Expect the rest of ecosystem to degrade quickly without a fresh compiler.

I suspect that in patch you have posted a few type and function declataions changed. Probably due to guard change like:

--- a/newlib/libc/include/sys/_pthreadtypes.h
+++ b/newlib/libc/include/sys/_pthreadtypes.h
@@ -18,7 +18,7 @@
 #ifndef _SYS__PTHREADTYPES_H_
 #define	_SYS__PTHREADTYPES_H_
 
-#if defined(_POSIX_THREADS)
+#if defined(_POSIX_THREADS) || __POSIX_VISIBLE >= 199506

[Comment 8 Andreas K. Hüttel 2023-08-22 21:58:16 UTC]

(In reply to Sergei Trofimovich (RETIRED) from comment #7)
> Note that spu-elf target was removed in gcc-10 release. Expect the rest of
> ecosystem to degrade quickly without a fresh compiler.
> 

Degrading this bug now too.