Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 755839 - sys-libs/readline: Prefix stage2 fails to build readline with "symbol(s) not found" error
Summary: sys-libs/readline: Prefix stage2 fails to build readline with "symbol(s) not ...
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: AMD64 OS X
: Normal normal (vote)
Assignee: Gentoo Prefix
Depends on:
Blocks: 755644
  Show dependency tree
Reported: 2020-11-20 16:32 UTC by Jacob Floyd
Modified: 2020-12-01 10:23 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---

build.log for sys-libs/readline-8.0::gentoo_prefix (build.log,69.53 KB, text/plain)
2020-11-20 16:36 UTC, Jacob Floyd
environment for sys-libs/readline-8.0::gentoo_prefix (environment,125.59 KB, text/plain)
2020-11-20 16:38 UTC, Jacob Floyd

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Floyd 2020-11-20 16:32:39 UTC
I'm on Mac OS X Catalina

I got through stage1 using a modified

In stage2, I got gentoo-functions-0.14 installed, and the both elt-patches and ncurses emerged correctly.

However readline is failing during the compile phase with error:

ld: symbol(s) not found for architecture x86_64

Reproducible: Always

Steps to Reproduce:
1. finish stage1 on Mac OSX Catalina
2. run stage2
3. boom! readline fails to build
Actual Results:  
Undefined symbols for architecture x86_64:
  "_BC", referenced from:
      __rl_init_terminal_io in terminal.o
  "_PC", referenced from:
      __rl_init_terminal_io in terminal.o
  "_UP", referenced from:
      __rl_init_terminal_io in terminal.o
  "_tgetent", referenced from:
      __rl_init_terminal_io in terminal.o
  "_tgetflag", referenced from:
      __rl_init_terminal_io in terminal.o
  "_tgetnum", referenced from:
      __rl_get_screen_size in terminal.o
  "_tgetstr", referenced from:
      __rl_init_terminal_io in terminal.o
  "_tgoto", referenced from:
      _update_line in display.o
  "_tputs", referenced from:
      _rl_redisplay in display.o
      _update_line in display.o
      __rl_clear_to_eol in display.o
      __rl_move_vert in display.o
      __rl_move_cursor_relative in display.o
      _rl_clear_visible_line in display.o
      __rl_clear_screen in display.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [Makefile:219:] Error 1
make[1]: *** Waiting for unfinished jobs....
libtool: link: ( cd ".libs" && rm -f "" && ln -s "../" "" )
make[1]: Leaving directory '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work/readline-8.0-abi_x86_64.amd64/shlib'
make: *** [Makefile:219: shared] Error 2
 * ERROR: sys-libs/readline-8.0::gentoo_prefix failed (compile phase):
 *   emake failed
 * If you need support, post the output of `emerge --info '=sys-libs/readline-8.0::gentoo_prefix'`,
 * the complete build log and the output of `emerge -pqv '=sys-libs/readline-8.0::gentoo_prefix'`.
 * The complete build log is located at '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/temp/build.log'.
 * The ebuild environment file is located at '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/temp/environment'.
 * Working directory: '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work/readline-8.0-abi_x86_64.amd64'
 * S: '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work/readline-8.0'

>>> Failed to emerge sys-libs/readline-8.0, Log file:

>>>  '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/temp/build.log'
 * Messages for package sys-libs/readline-8.0:
 * ERROR: sys-libs/readline-8.0::gentoo_prefix failed (compile phase):
 *   emake failed
 * If you need support, post the output of `emerge --info '=sys-libs/readline-8.0::gentoo_prefix'`,
 * the complete build log and the output of `emerge -pqv '=sys-libs/readline-8.0::gentoo_prefix'`.
 * The complete build log is located at '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/temp/build.log'.
 * The ebuild environment file is located at '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/temp/environment'.
 * Working directory: '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work/readline-8.0-abi_x86_64.amd64'
 * S: '/Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work/readline-8.0'

Expected Results:  
>>> sys-libs/readline-8.0 merged.

I will attach the build.log
Comment 1 Jacob Floyd 2020-11-20 16:36:16 UTC
Created attachment 673930 [details]
build.log for sys-libs/readline-8.0::gentoo_prefix
Comment 2 Jacob Floyd 2020-11-20 16:38:00 UTC
Created attachment 673933 [details]
environment for sys-libs/readline-8.0::gentoo_prefix
Comment 3 Jacob Floyd 2020-11-20 16:44:59 UTC
Looking at the patches, it looks like a darwin patch failed to apply to host-libtool:
>>> Unpacking host-libtool-0.1.0.tar.gz to /Users/jafloyd/Gentoo/tmp/var/tmp/portage/sys-libs/readline-8.0/work
 * Running elibtoolize in: host-libtool-0.1.0/
 *   Applying portage/1.2.0 patch ...
 *   Applying sed/1.5.6 patch ...
 *   Applying as-needed/2.4.2 patch ...
 *   Darwin patch set 'darwin-conf' failed to apply!
 *   Applying target-nm/2.4.2 patch ...
 *   Applying ppc64le/2.4.2 patch ...
 *   Applying darwin-conf/2.2.6 patch ...

Not sure if that's a red herring
Comment 4 Jacob Floyd 2020-11-20 18:44:22 UTC
Here are some possible problems in build.log

> configure: WARNING: you should use --build, --host, --target

configure is getting called with --host=x86_64-apple-darwin19 and --build=x86_64-apple-darwin19 but it is not getting --target.

> configure: WARNING: invalid host type: +

Not sure why "+" is being seen as a host type. configure is getting called "--with-curses +". maybe that's the reason for this message?

> checking for x86_64-apple-darwin19-gcc... clang -m64

I saw a similar message when bootstrapping python. I switched that one with CC=${CC/clang/gcc}. I'm not sure how to do the same for an individual package, but that would probably fix this.

Ah and based on a few random bug reports for people trying to build with readline, they needed to add -ltinfo. So, I looked at, (provided by =sys-libs/ncurses-6.2-r1), and (provided by sys-libs/ncurses-compat-5.9) on my gentoo box. It provides all of the missing symbols, so the linker is not finding one or more these on my Gentoo Prefix install for some reason (all installed by the recently built =sys-libs/ncurses-6.1-r3):
/Users/jafloyd/Gentoo/tmp/usr/lib/libtinfo.dylib -> libtinfo.6.dylib
/Users/jafloyd/Gentoo/tmp/usr/lib/libtinfow.dylib -> libtinfow.6.dylib
Comment 5 Fabian Groffen gentoo-dev 2020-11-20 18:48:40 UTC
This may be due to an incorrect stage1, or toolchain configuration.
Comment 6 Jacob Floyd 2020-11-20 23:16:57 UTC
OK. I'm trying to find the misconfiguration then.

From config.log:

> configure:6091: checking which library has the termcap functions
> configure:6316: result: using ncurses

however, it should be using tinfo, not ncurses or curses.

bash-3.2$ for dylib in libcurses.dylib libncurses.dylib libncursesw.dylib libtinfo.dylib libtinfow.dylib; do echo ~/Gentoo/tmp/lib/${dylib}; for symbol in BC PC UP tgetent tgetflag tgetnum tgetstr tgoto tputs; do nm -gU  ~/Gentoo/tmp/lib/${dylib} | grep ${symbol}; done; done
000000000002bc00 S _BC
000000000002bc0c S _PC
000000000002bbf8 S _UP
000000000000a530 T _tgetent
0000000000009f40 T _tgetent_sp
000000000000a670 T _tgetflag
000000000000a550 T _tgetflag_sp
000000000000a7b0 T _tgetnum
000000000000a690 T _tgetnum_sp
000000000000a990 T _tgetstr
000000000000a7d0 T _tgetstr_sp
000000000000aa70 T _tgoto
000000000000d820 T _tputs
000000000000d440 T _tputs_sp
000000000002bc10 S _BC
000000000002bc1c S _PC
000000000002bc08 S _UP
0000000000009a40 T _tgetent
0000000000009400 T _tgetent_sp
0000000000009b90 T _tgetflag
0000000000009a60 T _tgetflag_sp
0000000000009ce0 T _tgetnum
0000000000009bb0 T _tgetnum_sp
0000000000009ed0 T _tgetstr
0000000000009d00 T _tgetstr_sp
0000000000009fb0 T _tgoto
000000000000cda0 T _tputs
000000000000c9c0 T _tputs_sp

bash-3.2$ pwd
bash-3.2$ grep -r TERMCAP_LIB ../readline-8.0-abi_x86_64.amd64/
../readline-8.0-abi_x86_64.amd64/Makefile:TERMCAP_LIB = -lcurses

So, it is trying to use -lcurses.
I'm not completely familiar with and aclocal.m4, but here goes my search... Gotta learn somewhere :)

BASH_CHECK_LIB_TERMCAP determines which library provides the termcap lib.
It sets bash_cv_termcap_lib by looking for tgetent in several libraries: termcap, tinfo, curses, ncurses, ncursesw. Somehow it picked curses, which is apparently the fall through default.

According to config.log, bash_cv_termcap_lib=ncurses, but even that doesn't match this condition:
> elif test $bash_cv_termcap_lib = libncurses; then

hmm. in build.log it's pulling from a bad cache somewhere:
> checking which library has the termcap functions... (cached) using ncurses

It's in the environment
> bash-3.2$ grep -r bash_cv_termcap_lib ../../temp/
> ../../temp/environment:declare -x bash_cv_termcap_lib="ncurses"
> ../../temp/environment:    export bash_cv_termcap_lib=ncurses;

Which is apparently hard-coded in src_configure in the ebuild (it is hard-coded in the mainline ebuilds too, so this is not a prefix thing).

>        # Force the test since we used sed above to force it.
>        export bash_cv_termcap_lib=ncurses

So that begs the question, how is this supposed to work? Are the ncurses libs (particularly via -lcurses) supposed to include termcap functions? Or is the ebuild just wrong?

on my gentoo box, -> which is a linker script that redirects to /lib64/ ->
But that doesn't have the termcap symbols either... So, I'm not sure how that is supposed to work.
Comment 7 Jacob Floyd 2020-11-21 00:16:10 UTC
comparing the ebuild ... compile output on my gentoo box (using gcc) with the macos prefix output (using libtool + clang).

On gentoo w/ gcc, the linking phase uses:
-lncursesw -ltinfow
and on prefix it only gets

So, the toolchain is converting bash_cv_termcap_lib=ncurses into something different on each of them. hmm.

Linking goes first into the shlib directory in both cases, but dies in prefix build before leaving it.

Now what part of the toolchain adds `-lncursesw -ltinfow`? (I want to figure out what is broken!)
Comment 8 Jacob Floyd 2020-11-21 01:31:18 UTC
Hmm. TERMCAP_LIB seems like the wrong var to be looking for. -ltinfow is actually in SHLIB_LIBS on my full gentoo install.

Seems it comes from support/shobj-conf but why are the defaults different?!

SHLIB_LIBS='-lncursesw -ltinfow'

And that gets modified during the prepare stage with sed.

which comes from this line:
> local ncurses_libs=$($(tc-getPKG_CONFIG) ncurses --libs)
which in the mainline ebuild is:
> local ncurses_libs=$($(tc-getPKG_CONFIG) ncurses$(usex unicode w '') --libs)

so pkg-config should provide this...
But both build.log and environment there are several warnings like this:

> pkg-config: command not found

That would be installed by dev-util/pkgconfig once the system is fully up and running. What should provide ${CHOST}-pkg-config during bootstrap? Is it installed during stage1? In previous versions of MacOSX/Xcode, was it provided with CommandLineTools or Xcode?
Comment 9 Jacob Floyd 2020-11-21 03:16:18 UTC
OK. I got readline to build by:

1 - bump to the version in mainline gentoo (might not be needed, but I did it)
    readline-8.0.ebuild -> readline-8.0_p4.ebuild

2 - in, add dev-util/pkgconf between sys-libs/ncurses and sys-libs/readline. (pkgconfig did not build due to deps. pkgconf is lighter weight and builds just fine. pkg-config is required by the readline ebuild).

3 - add ~x64-macos keyword to pkgconf ebuild and regen the ebuild manifest

4 - Modify the readline ebuild (8.0_p4 for me - should work with 8.0 as well)
    in src_prepare:

>        # Force ncurses linking. #71420
>        # Use pkg-config to get the right values. #457558
>        local ncurses_libs=$($(tc-getPKG_CONFIG) ncurses --libs)
>        sed -i \
>                -e "/^SHLIB_LIBS=/s:=.*:='${ncurses_libs}':" \
+                -e "/SHLIB_LIBS=.*MacOS X/d" \

>                support/shobj-conf || die

5 - run stage2 which should build pkgconf and readline.

There. readline.
Comment 10 Jacob Floyd 2020-11-21 03:41:00 UTC
Checking out bugs listed in the ebuild, it seems that forcing -lncurses happened a long time ago (see bug#71420), and then started using pkg-config with readline-6.2_p5 (see bug#457558) to get the ncurses libs due to the split ncurses[tinfo].

By the way profiles/base/package.use.force has:
sys-libs/ncurses tinfo

So ncurses always provides termcap / tinfo on gentoo.

I wonder how this built successfully on previous versions of Mac? Were they somehow using much older versions of ncurses or readline? Can the current versions of ncurses and readline build today without issue on older Macs?

Sadly, I don't have anything older than Catalina available to me.
Comment 11 Fabian Groffen gentoo-dev 2020-12-01 10:23:48 UTC
fixed by using bleeding edge 8.1