Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 778014 - Prefix: Big Sur ARM (M1 MacBook Pro MYD82LL/A) build failure due to missing symbols
Summary: Prefix: Big Sur ARM (M1 MacBook Pro MYD82LL/A) build failure due to missing s...
Status: CONFIRMED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: ARM64 OS X
: Highest blocker with 5 votes (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
: 832037 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-03-24 19:49 UTC by Kenneth G. Strawn
Modified: 2022-01-26 10:02 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,395.52 KB, text/plain)
2021-03-24 19:50 UTC, Kenneth G. Strawn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kenneth G. Strawn 2021-03-24 19:49:44 UTC
bootstrap_prefix.sh at least appears to do fine up until the attempt to compile GCC, at which point numerous "undefined symbol" errors arise

Reproducible: Always

Steps to Reproduce:
1. wget https://gitweb.gentoo.org/repo/proj/prefix.git/plain/scripts/bootstrap-prefix.sh
2. chmod +x bootstrap-prefix.sh
3. ./bootstrap-prefix.sh
Actual Results:  
Undefined symbols for architecture arm64:
  "__Z5yyendv", referenced from:
      __Z10parse_filePKc in gengtype-parse.o
  "__Z5yylexPPKc", referenced from:
      __ZL5tokenv in gengtype-parse.o
  "__Z7yybeginPKc", referenced from:
      __Z10parse_filePKc in gengtype-parse.o
  "_lexer_line", referenced from:
      __ZL22create_optional_field_P4pairP4typePKcS4_i in gengtype.o
      __ZL20adjust_field_rtx_defP4typeP7options in gengtype.o
      __ZL21adjust_field_tree_expP4typeP7options in gengtype.o
      __Z17adjust_field_typeP4typeP7options in gengtype.o
      __ZL11parse_errorPKcz in gengtype-parse.o
      __ZL16struct_field_seqv in gengtype-parse.o
      __ZL4typePP7optionsb in gengtype-parse.o
      ...
  "_lexer_toplevel_done", referenced from:
      __Z10parse_filePKc in gengtype-parse.o
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:3004: build/gengtype] Error 1
make[3]: *** Waiting for unfinished jobs....
/opt/gentoo/tmp/bin/bash /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/../move-if-change tmp-options.h options.h
echo timestamp > s-options-h
/opt/gentoo/tmp/bin/bash /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/../move-if-change tmp-c-target-hooks-def.h \
				     c-family/c-target-hooks-def.h
/opt/gentoo/tmp/bin/bash /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/../move-if-change tmp-target-hooks-def.h \
				     target-hooks-def.h
/opt/gentoo/tmp/bin/bash /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/../move-if-change tmp-d-target-hooks-def.h \
				     d/d-target-hooks-def.h
/opt/gentoo/tmp/bin/bash /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/../move-if-change tmp-common-target-hooks-def.h \
				     common/common-target-hooks-def.h
echo timestamp > s-c-target-hooks-def-h
echo timestamp > s-target-hooks-def-h
echo timestamp > s-d-target-hooks-def-h
echo timestamp > s-common-target-hooks-def-h
rm gcov.pod lto-dump.pod gfdl.pod gpl.pod cpp.pod gcc.pod gcov-dump.pod gcov-tool.pod fsf-funding.pod
make[3]: Leaving directory '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/build/gcc'
make[2]: *** [Makefile:4786: all-stage1-gcc] Error 2
make[2]: Leaving directory '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/build'
make[1]: *** [Makefile:22346: stage1-bubble] Error 2
make[1]: Leaving directory '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/build'
make: *** [Makefile:1009: all] Error 2
 * ERROR: sys-devel/gcc-11_pre20200206::gentoo_prefix failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=sys-devel/gcc-11_pre20200206::gentoo_prefix'`,
 * the complete build log and the output of `emerge -pqv '=sys-devel/gcc-11_pre20200206::gentoo_prefix'`.
 * The complete build log is located at '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/temp/build.log'.
 * The ebuild environment file is located at '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/temp/environment'.
 * Working directory: '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/build'
 * S: '/opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si'
 * 
 * Please include /opt/gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-build-logs.tar.bz2 in your bug report.
 * 

Expected Results:  
Successful build

Attaching full build.log file after publishing this
Comment 1 Kenneth G. Strawn 2021-03-24 19:50:30 UTC
Created attachment 693357 [details]
build.log

Log file for failed GCC build
Comment 2 Fabian Groffen gentoo-dev 2021-03-25 07:16:05 UTC
Yes, the problem is m4 calling flex.  For some reason execv() call fails, leaving an empty file, which is then ignored.  Obviously, from there you get these missing symbols.

There is nothing we/I can find as to why the exec call fails, nor why the version provided by Apple /does/ work.  Compiling their code results in the same non-functional binary.
Comment 3 Kenneth G. Strawn 2021-03-25 17:47:28 UTC
I wonder therefore if there's a way to patch M4 to make it use Apple's exec() call instead of M4's on arm64-macos-big-sur as a temporary workaround.
Comment 4 Kenneth G. Strawn 2021-03-25 17:48:32 UTC
Oops, bad wording: I wonder therefore if there's a way to patch M4 to use Apple's exec() call instead of Flex's on arm64-macos11 as a temporary workaround.
Comment 5 Fabian Groffen gentoo-dev 2021-03-25 18:18:09 UTC
perhaps that's an option for the time being, to see if there's other stuff popping up.

Sidenote, you can just bootstrap x86_64 mode if you need a working system: /usr/bin/arch -x86_64 ./bootstrap-prefix.
Comment 6 Steve Gilberd 2021-05-27 14:47:03 UTC
> the problem is m4 calling flex...

Shouldn't this read "flex calling m4"? That appears to be the issue on my system, and the symptoms are otherwise identical to this bug report.

Interestingly, while the exec call fails when flex is run via make, running exactly the same flex command from a shell instead completes without any errors, and results in a gengtype-lex.c which works correctly. Injecting this file into the build process, replacing the make-generated one, allows the gcc build to proceed properly, without encountering the missing symbol error.

```
TARGET_CPU_DEFAULT="" \                                            
HEADERS="auto-host.h ansidecl.h" DEFINES="" \
/Volumes/Gentoo/tmp/bin/bash /Volumes/Gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/mkconfig.sh config.h
TARGET_CPU_DEFAULT="" \
HEADERS="options.h insn-constants.h config/aarch64/biarchlp64.h config/aarch64/aarch64.h config/darwin.h config/aarch64/darwin.h config/aarch64/aarch64-errata.h config/initfini-array.h defaults.h" DEFINES="LIBC_GLIBC=1 LIBC_UCLIBC=2 LIBC_BIONIC=3 LIBC_MUSL=4 DEF_MIN_OSX_VERSION=\"11.0\" DEF_LD64=\"609.0\" TARGET_DEFAULT_ASYNC_UNWIND_TABLES=1" \
/Volumes/Gentoo/tmp/bin/bash /Volumes/Gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/mkconfig.sh tm.h
TARGET_CPU_DEFAULT="" \
HEADERS="config/aarch64/aarch64-protos.h config/arm/aarch-common-protos.h config/darwin-protos.h tm-preds.h" DEFINES="" \
/Volumes/Gentoo/tmp/bin/bash /Volumes/Gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/mkconfig.sh tm_p.h
TARGET_CPU_DEFAULT="" \
HEADERS="auto-host.h ansidecl.h" DEFINES="" \
/Volumes/Gentoo/tmp/bin/bash /Volumes/Gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/mkconfig.sh bconfig.h
flex  -ogengtype-lex.c /Volumes/Gentoo/var/tmp/portage/sys-devel/gcc-11_pre20200206/work/gcc-darwin-arm64-master-wip-apple-si/gcc/gengtype-lex.l && { \
  echo '#ifdef HOST_GENERATOR_FILE' > gengtype-lex.c.tmp; \
  echo '#include "config.h"'       >> gengtype-lex.c.tmp; \
  echo '#else'                     >> gengtype-lex.c.tmp; \
  echo '#include "bconfig.h"'      >> gengtype-lex.c.tmp; \
  echo '#endif'                    >> gengtype-lex.c.tmp; \
  cat gengtype-lex.c >> gengtype-lex.c.tmp; \
  mv gengtype-lex.c.tmp gengtype-lex.c; \
}
```
Comment 7 Fabian Groffen gentoo-dev 2021-05-27 18:01:52 UTC
yes, and the mystery remains why the call fails
Comment 8 Steve Gilberd 2021-06-03 08:35:38 UTC
The precise location this error ultimately seems to originate from is here:
https://github.com/westes/flex/blob/04c5b7c9209801aa1bdbf279ccdcde0d57874a55/src/filter.c#L182

Of particular note is the comment beginning on line 156 - it looks like there are hacky assumptions being made, which may (given that Mac OS is a BSD derivative) be applicable here, and not working as the author of that particular bit of code intended.
Comment 9 Fabian Groffen gentoo-dev 2021-06-03 08:41:25 UTC
Indeed, but even when you compile Apple's version of flex (from their opensource page), which actually does the same, it fails as well.

So, at least in my attempts, compiling Apple's version of flex using Apple's Clang, doesn't result in a working flex, whereas Apple's provided binary for flex works.

So it appears there's something we do not have here.  Just to be complete, the same OS version on x64, the whole problem isn't present, so it appears to be something entirely related to the CPU architecture.  Even using Rosetta this problem doesn't appear.
Comment 10 Steve Gilberd 2021-06-03 08:47:12 UTC
Interesting.

A bit of wild speculation here - do we know whether Apple's flex binary was cross-compiled, or compiled natively?

Do we know which version of Clang Apple used to build it?
Comment 11 Fabian Groffen gentoo-dev 2021-06-03 08:51:33 UTC
I don't think we can see any of that from the binary.  We know which version of Clang there was at the time of the introduction of the OS, but there's no guarantee that that's the version they used to build the OS.
Comment 12 Sylwester Arabas 2021-10-15 17:52:34 UTC
FWIW, just confirming that the problem has not disappeared, same symptoms here. Any hints? Thanks
Comment 13 john 2022-01-26 09:52:01 UTC
*** Bug 832037 has been marked as a duplicate of this bug. ***
Comment 14 john 2022-01-26 10:02:50 UTC
Also experiencing the same issue and erroneously filed 832037 before I stumbled on this one :)