Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 549994 - cross-*/gcc: handling of internal gcc search paths (libstdc++.so.6 and friends)
Summary: cross-*/gcc: handling of internal gcc search paths (libstdc++.so.6 and friends)
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Crossdev team
URL:
Whiteboard:
Keywords:
: 291383 557236 573304 (view as bug list)
Depends on:
Blocks: crossdev-bugs
  Show dependency tree
 
Reported: 2015-05-20 21:27 UTC by Matt Whitlock
Modified: 2019-12-22 23:55 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
libxml2-2.9.2-force-cxx-link.patch (libxml2-2.9.2-force-cxx-link.patch,860 bytes, patch)
2015-05-20 21:27 UTC, Matt Whitlock
Details | Diff
build.log (build.log,147.82 KB, text/x-log)
2015-05-29 06:25 UTC, Matt Whitlock
Details
armv6j-hardfloat-linux-gnueabi-gcc test.c -licui18n -Wl,--verbose -v (test-verbose.log,21.19 KB, text/plain)
2015-05-29 06:32 UTC, Matt Whitlock
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Whitlock 2015-05-20 21:27:23 UTC
Created attachment 403698 [details, diff]
libxml2-2.9.2-force-cxx-link.patch

The libraries of dev-libs/icu link with libstdc++.so, but dev-libs/libxml2 is linked as a C library. This accidentally works when compiling natively but breaks when cross-compiling. The solution is to force libxml2.so to be linked as a C++ library.

The relevant hack is documented here:

https://www.gnu.org/software/automake/manual/html_node/Libtool-Convenience-Libraries.html

> If one of the sublibraries contains non-C source, it is important that
> the appropriate linker be chosen. One way to achieve this is to pretend
> that there is such a non-C file among the sources of the library, thus
> forcing automake to select the appropriate linker.

The attached patch utilizes this hack to force libxml2.so to be linked as a C++ library.


Build output WITHOUT the patch:

/usr/libexec/gcc/armv6j-hardfloat-linux-gnueabi/ld: warning: libstdc++.so.6, needed by /usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so, not found (try using -rpath or -rpath-link)
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `std::type_info::operator==(std::type_info const&) const@GLIBCXX_3.4'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `__gxx_personality_v0@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicuuc.so: undefined reference to `__cxa_call_unexpected@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `std::terminate()@GLIBCXX_3.4'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `vtable for __cxxabiv1::__vmi_class_type_info@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `__dynamic_cast@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'
/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so: undefined reference to `__cxa_end_cleanup@CXXABI_1.3'
collect2: error: ld returned 1 exit status
Makefile:1116: recipe for target 'xmllint' failed
make[2]: *** [xmllint] Error 1
make[2]: Leaving directory '/usr/armv6j-hardfloat-linux-gnueabi/tmp/portage/dev-libs/libxml2-2.9.2-r1/work/libxml2-2.9.2-.arm'
Makefile:1439: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/usr/armv6j-hardfloat-linux-gnueabi/tmp/portage/dev-libs/libxml2-2.9.2-r1/work/libxml2-2.9.2-.arm'
Makefile:858: recipe for target 'all' failed
make: *** [all] Error 2
 * ERROR: dev-libs/libxml2-2.9.2-r1::gentoo failed (compile phase):
 *   emake failed


Build output WITH the patch:

>>> Source compiled.

;-)
Comment 1 Matt Whitlock 2015-05-20 21:28:38 UTC
Note: Bug 478490 seems to be another instance of this same problem.
Comment 2 Rafał Mużyło 2015-05-21 01:17:00 UTC
This smells foul.
What's the output of 'ldd -r /usr/lib/libicui18n.so' ?

If it contains any errors, it's been built incorrectly.

The other option is that your cross-compile env is setup incorrectly and compiler doesn't see libstdc++ during build.
Comment 3 Matt Whitlock 2015-05-21 03:21:08 UTC
(In reply to Rafał Mużyło from comment #2)
> This smells foul.
> What's the output of 'ldd -r /usr/lib/libicui18n.so' ?

$ ldd -r /usr/lib/libicui18n.so
        linux-vdso.so.1 (0x00007ffedbfb1000)
        libicuuc.so.55 => /usr/lib64/libicuuc.so.55 (0x00007fcc04b15000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libstdc++.so.6 (0x00007fcc04805000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fcc04502000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1 (0x00007fcc042eb000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fcc03f4e000)
        libicudata.so.55 => /usr/lib64/libicudata.so.55 (0x00007fcc02496000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcc0227b000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fcc02077000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcc0537f000)

> If it contains any errors, it's been built incorrectly.

I don't see any errors there, but I think that's irrelevant since my build of libxml2.so is trying to link against /usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so.

> The other option is that your cross-compile env is setup incorrectly and
> compiler doesn't see libstdc++ during build.

I agree: it's set up incorrectly. But it's the environment that sys-devel/crossdev generates.


Here's a simple test case reproducing the problem:

$ cat > libhello.cpp <<EOF
#include <iostream>
extern "C" void hello() {
    std::cout << "Hello world!" << std::endl;
}
EOF

$ cat > hello.c <<EOF
extern void hello();
int main() {
    hello();
    return 0;
}
EOF

$ g++ -shared -fPIC -o libhello.so libhello.cpp
$ gcc -o hello hello.c ./libhello.so

(No errors.)

$ armv6j-hardfloat-linux-gnueabi-g++ -shared -fPIC -o libhello.so libhello.cpp
$ armv6j-hardfloat-linux-gnueabi-gcc -o hello hello.c ./libhello.so
/usr/libexec/gcc/armv6j-hardfloat-linux-gnueabi/ld: warning: libstdc++.so.6, needed by ./libhello.so, not found (try using -rpath or -rpath-link)
./libhello.so: undefined reference to `std::ios_base::Init::~Init()@GLIBCXX_3.4'
./libhello.so: undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::endl<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&)@GLIBCXX_3.4'
./libhello.so: undefined reference to `std::ios_base::Init::Init()@GLIBCXX_3.4'
./libhello.so: undefined reference to `__aeabi_atexit@CXXABI_ARM_1.3.3'
./libhello.so: undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)@GLIBCXX_3.4'
./libhello.so: undefined reference to `std::cout@GLIBCXX_3.4'
./libhello.so: undefined reference to `std::ostream::operator<<(std::ostream& (*)(std::ostream&))@GLIBCXX_3.4'
collect2: error: ld returned 1 exit status

(Whoops!)

So it would seem there's something wrong with the ldscripts for the armv6j-hardfloat-linux-gnueabi toolchain, right? Whatever that problem is, if it could be fixed, then this build failure in dev-libs/libxml2, and others like it, would all be fixed.
Comment 4 Matt Whitlock 2015-05-21 03:46:23 UTC
I am not an expert on ldscripts, but it seems to me that the only reason the test case works on the native toolchain is that libstdc++.so.6 is found via /etc/ld.so.cache. None of the ldscripts in /usr/x86_64-pc-linux-gnu/lib/ldscripts mention /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/ as a library search path.

The native libstdc++.so.6 is present in /etc/ld.so.cache due to path entries in /etc/ld.so.conf.d/05gcc-x86_64-pc-linux-gnu.conf, but there is no ld.so.conf entry for the cross toolchain's libraries (nor would it make sense for there to be).

So the cross-linker has no clue where to find libstdc++.so.6. The correct location is not mentioned in the ldscripts, and a suitable library cannot be found in /etc/ld.so.cache.
Comment 5 Matt Whitlock 2015-05-21 03:51:21 UTC
For what it's worth, symlinking libstdc++.so.6 in the system library search path makes the test case succeed on the cross toolchain, but this is not the correct solution.

# ln -s /usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2/libstdc++.so.6 \
        /usr/armv6j-hardfloat-linux-gnueabi/lib/

$ armv6j-hardfloat-linux-gnueabi-gcc -o hello hello.c ./libhello.so

(No errors.)
Comment 6 Alexandre Rostovtsev (RETIRED) gentoo-dev 2015-05-29 01:26:18 UTC
At first glance, this seems dubious.

icu is linked using c++ linker, so you propose for libxml2 to be linked using c++ linker. OK. But then by the same logic you would need everything that links to libxml2 to be linked using c++ linker too, right? And so on recursively, and at the end you are forced to use the c++ linker for almost everything above glibc - since you never know if c++ is present somewhere deep in the dependency chain.

So I would like to see how other distros and embedded people are handling this.

CC-ing embedded and toolchain for advice.
Comment 7 Matt Whitlock 2015-05-29 03:24:13 UTC
(In reply to Alexandre Rostovtsev from comment #6)
> icu is linked using c++ linker, so you propose for libxml2 to be linked
> using c++ linker. OK. But then by the same logic you would need everything
> that links to libxml2 to be linked using c++ linker too, right? And so on
> recursively, and at the end you are forced to use the c++ linker for almost
> everything above glibc - since you never know if c++ is present somewhere
> deep in the dependency chain.

I agree that the standard cross linker should be sufficient since the transitive dependencies of any linked libraries should be found via the linker's search path. But in Gentoo crossdev toolchains, the C++ standard libraries are not stored in a location where the cross linker can find them. I initially thought that this was a problem with ldscripts, but I now believe that the problem stems from the algorithm that the linker uses to find transitive dependencies. Quoting from the ld(1) man page:

> The linker uses the following search paths to locate required shared
> libraries:
> 
> 1.  Any directories specified by -rpath-link options.
> 
> 2.  Any directories specified by -rpath options.  The difference between
>     -rpath and -rpath-link is that directories specified by -rpath options are
>     included in the executable and used at runtime, whereas the -rpath-link
>     option is only effective at link time. Searching -rpath in this way is
>     only supported by native linkers and cross linkers which have been
>     configured with the --with-sysroot option.
> 
> 3.  On an ELF system, for native linkers, if the -rpath and -rpath-link
>     options were not used, search the contents of the environment variable
>     "LD_RUN_PATH".
> 
> 4.  On SunOS, if the -rpath option was not used, search any directories
>     specified using -L options.
> 
> 5.  For a native linker, search the contents of the environment variable
>     "LD_LIBRARY_PATH".
> 
> 6.  For a native ELF linker, the directories in "DT_RUNPATH" or "DT_RPATH" of
>     a shared library are searched for shared libraries needed by it. The
>     "DT_RPATH" entries are ignored if "DT_RUNPATH" entries exist.
> 
> 7.  The default directories, normally /lib and /usr/lib.
> 
> 8.  For a native linker on an ELF system, if the file /etc/ld.so.conf exists,
>     the list of directories found in that file.

Again, I believe that the only reason the native linker is able to link with a shared library that in turn links with libstdc++.so.6 is because libstdc++.so.6 is found in step 8, but this search step isn't enabled in the cross-compiling case.

To resolve the problem in the cross-compiling case on Linux, steps 1, 2, and 7 are our only options. My workaround in Comment #5 makes libstdc++.so.6 available at step 7, but this workaround is not very satisfying, as it does not respect the currently selected gcc-config profile and will break upon upgrading GCC.

An alternative workaround, which works at step 1, is to set
LDFLAGS="-Wl,-O1 -Wl,-rpath-link,\$(unset ROOT ; gcc-config -L)"
in /etc/portage/make.conf. This is the workaround that I will leave in place on my system, but I still feel that this should be fixed inside the toolchain.
Comment 8 Matt Whitlock 2015-05-29 03:33:35 UTC
(In reply to Matt Whitlock from comment #7)
> An alternative workaround, which works at step 1, is to set
> LDFLAGS="-Wl,-O1 -Wl,-rpath-link,\$(unset ROOT ; gcc-config -L)"
> in /etc/portage/make.conf.

Never mind. This workaround breaks in other packages that use different build systems. (It's a shame that Portage doesn't support command substitution in config files.)

So we're back to a brittle workaround that doesn't respect gcc-config profiles:

LDFLAGS="-Wl,-O1 -Wl,-rpath-link,/usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2"
Comment 9 Alexandre Rostovtsev (RETIRED) gentoo-dev 2015-05-29 03:48:40 UTC
(In reply to Matt Whitlock from comment #7)
> To resolve the problem in the cross-compiling case on Linux, steps 1, 2, and
> 7 are our only options.

If that is the case, then the solution might be to create a wrapper around ld specifically for cross-compiling. Perhaps crossdev already has an option to do this?
Comment 10 Matt Whitlock 2015-05-29 03:55:26 UTC
(In reply to Alexandre Rostovtsev from comment #9)
> the solution might be to create a wrapper around ld

Or perhaps have gcc-config create symlinks in ${SYSROOT}/usr/lib when switching to a new cross-compiler profile.
Comment 11 SpanKY gentoo-dev 2015-05-29 05:09:03 UTC
there is no need for cross-compilers to have a linker wrapper or for symlinking in random paths

if libicu18n.so uses C++ code, then it must link against libstdc++.so.  that happens either by being linked with the C++ compiler driver (e.g. g++) or an explicit -lstdc++ is used.  on my system, it looks fine:
$ readelf -d /usr/lib64/libicui18n.so | grep stdc
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
$ readelf -d /usr/aarch64-unknown-linux-gnu/usr/lib64/libicui18n.so | grep stdc
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]

any package that wants to link against -licu18n need not be done with the C++ linker.  simply doing `gcc -licu18n` should be sufficient.  and that works for me as well:
$ aarch64-unknown-linux-gnu-gcc test.c -licui18n
<success>

cross-compiling libxml2 with USE=icu works fine for me.  you need to post full build logs with every bug report.
Comment 12 Matt Whitlock 2015-05-29 06:04:01 UTC
(In reply to SpanKY from comment #11)
> there is no need for cross-compilers to have a linker wrapper or for
> symlinking in random paths

Did you try my test case from Comment #3?

How does your aarch64-unknown-linux-gnu-ld, when called by aarch64-unknown-linux-gnu-gcc, know to look in /usr/lib/gcc/aarch64-unknown-linux-gnu/* to find libstdc++.so.6?
Comment 13 SpanKY gentoo-dev 2015-05-29 06:20:38 UTC
(In reply to Matt Whitlock from comment #12)

the compiler driver takes care of adding link paths to its internal locations.  otherwise it'd never find things like -lgcc_s or -lgcc or -lgfortran or -lstdc++ or any of the other internal gcc libs.

$ aarch64-unknown-linux-gnu-gcc test.c -licui18n -Wl,--verbose -v
...
COMPILER_PATH=/usr/libexec/gcc/aarch64-unknown-linux-gnu/4.8.2/:/usr/libexec/gcc/aarch64-unknown-linux-gnu/4.8.2/:/usr/libexec/gcc/aarch64-unknown-linux-gnu/:/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/:/usr/lib/gcc/aarch64-unknown-linux-gnu/:/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/../../../../aarch64-unknown-linux-gnu/bin/
LIBRARY_PATH=/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/:/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/../../../../aarch64-unknown-linux-gnu/lib/../lib64/:/usr/aarch64-unknown-linux-gnu/lib/../lib64/:/usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/:/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/../../../../aarch64-unknown-linux-gnu/lib/:/usr/aarch64-unknown-linux-gnu/lib/:/usr/aarch64-unknown-linux-gnu/usr/lib/
...

you can see gcc has loaded the right paths internally and passes it on to the linker:

 /usr/libexec/gcc/aarch64-unknown-linux-gnu/4.8.2/collect2 --sysroot=/usr/aarch64-unknown-linux-gnu --eh-frame-hdr -dynamic-linker /lib/ld-linux-aarch64.so.1 -X /usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/crt1.o /usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/crti.o /usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/crtbegin.o -L/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/../../../../aarch64-unknown-linux-gnu/lib/../lib64 -L/usr/aarch64-unknown-linux-gnu/lib/../lib64 -L/usr/aarch64-unknown-linux-gnu/usr/lib/../lib64 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/../../../../aarch64-unknown-linux-gnu/lib -L/usr/aarch64-unknown-linux-gnu/lib -L/usr/aarch64-unknown-linux-gnu/usr/lib /tmp/ccZiym6D.o -licui18n --verbose -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/crtend.o /usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/crtn.o

and then the linker says it found things fine:
...
libicuuc.so.55 needed by /usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/libicui18n.so
found libicuuc.so.55 at /usr/aarch64-unknown-linux-gnu/usr/lib64/libicuuc.so.55
libstdc++.so.6 needed by /usr/aarch64-unknown-linux-gnu/usr/lib/../lib64/libicui18n.so
found libstdc++.so.6 at /usr/aarch64-unknown-linux-gnu/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2/libstdc++.so.6
...
Comment 14 Matt Whitlock 2015-05-29 06:25:09 UTC
Created attachment 404250 [details]
build.log

(In reply to SpanKY from comment #11)
> cross-compiling libxml2 with USE=icu works fine for me.  you need to post
> full build logs with every bug report.

Portage 2.2.20 (python 2.7.9-final-0, prefix/linux/arm, gcc-4.9.2, unavailable, 3.18.11-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-3.18.11-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q6600_@_2.40GHz-with-gentoo-2.2
KiB Mem:     8168852 total,   3100308 free
KiB Swap:   25165804 total,  23820600 free
Timestamp of repository gentoo: Thu, 28 May 2015 10:00:01 +0000
sh bash 4.3_p39
ld GNU ld (Gentoo 2.25 p1.2) 2.25
distcc 3.2rc1 x86_64-pc-linux-gnu [disabled]
ccache version 3.2.2 [disabled]
sys-devel/autoconf: 2.69-r1::gentoo
sys-devel/automake: 1.15::gentoo
sys-devel/libtool:  2.4.6-r1::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

local
    location: /usr/local/portage/overlay
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="arm ~arm"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-Ofast -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard"
CHOST="armv6j-hardfloat-linux-gnueabi"
CONFIG_PROTECT="/etc /opt/cassandra/conf /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.2/conf /usr/share/themes/oxygen-gtk/gtk-2.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-Ofast -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard"
DISTDIR="/usr/local/portage/distfiles"
EMERGE_DEFAULT_OPTS="--nospinner --unordered-display --usepkg"
FCFLAGS="-O2"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news nodoc noinfo noman parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j5"
PKGDIR="/mnt/raspi/usr/local/portage/packages/"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/usr/armv6j-hardfloat-linux-gnueabi/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/usr/armv6j-hardfloat-linux-gnueabi/tmp/"
USE="arm berkdb bzip2 cli cracklib crypt dri fortran gdbm iconv icu ipv6 minimal modules ncurses nptl openmp pcre prefix prefix-guest readline session ssl tcpd unicode" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en_US en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="exynos fbdev omap omapfb dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

dev-libs/libxml2-2.9.2-r1::gentoo was built with the following:
USE="ipv6 (prefix) (prefix-guest) readline -debug -examples -icu -lzma -python -static-libs -test" PYTHON_TARGETS="python2_7 -python3_3 -python3_4"
Comment 15 Matt Whitlock 2015-05-29 06:32:40 UTC
Created attachment 404252 [details]
armv6j-hardfloat-linux-gnueabi-gcc test.c -licui18n -Wl,--verbose -v
Comment 16 Matt Whitlock 2015-05-29 07:19:26 UTC
$ strace -f -e open armv6j-hardfloat-linux-gnueabi-gcc test.c -licui18n

...
[pid 32180] open("/usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2/libicui18n.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2/libicui18n.a", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2/../../../../armv6j-hardfloat-linux-gnueabi/lib/libicui18n.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/lib/gcc/armv6j-hardfloat-linux-gnueabi/4.9.2/../../../../armv6j-hardfloat-linux-gnueabi/lib/libicui18n.a", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/lib/libicui18n.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/lib/libicui18n.a", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libicui18n.so", O_RDONLY) = 8

(The linker finds libicui18n.so after searching the paths specified by the -L options that the driver had built from its LIBRARY_PATH.)

...
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/usr/armv6j-hardfloat-linux-gnueabi/lib/libstdc++.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/usr/local/lib/libstdc++.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/lib/libstdc++.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 32180] open("/usr/armv6j-hardfloat-linux-gnueabi/usr/lib/libstdc++.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)

(The linker fails to find libstdc++.so.6 after searching the paths specified by the SEARCH_DIR directives in its internal linker script.)

When resolving transitive dependencies, the linker does NOT consult the paths specified by the -L options, except apparently on SunOS. (See step 4 in Comment #7.) Therefore, we should NOT expect the linker to find libstdc++.so.6, as none of the SEARCH_DIR directives in the linker script specify its location. And indeed the linker does not find libstdc++.so.6.

What version of ld are you using? Maybe the search behavior has changed recently.
Comment 17 SpanKY gentoo-dev 2015-05-29 07:54:50 UTC
it works for my aarch64 target because i've installed a lot more packages into the sysroot including gcc itself

that in turns creates $SYSROOT/etc/env.d/04gcc-$CHOST which includes:
LDPATH="/usr/lib/gcc/aarch64-unknown-linux-gnu/4.8.2"

which causes portage to update the $SYSROOT/etc/ld.so.conf with this path.

the linker will walk $SYSROOT/etc/ld.so.conf and add each entry there to its search path (with a $SYSROOT prefix).  in my case, it finds the gcc libs installed in the sysroot, not the ones part of the cross-compiler.

this is the same reason a native linker finds the internal gcc libs -- it doesn't parse ld.so.cache at all, only walks /etc/ld.so.conf.  you can reproduce the issue there by doing:
  mv /etc/ld.so.conf{,.bak}
  gcc -c test.c
  ld test.o -o a.out -L/usr/lib64 -licui18n

it'll fail to find libstdc++ for the same reason.
Comment 18 Matt Whitlock 2015-05-29 08:30:59 UTC
(In reply to SpanKY from comment #17)

Thank you for the closure! I feel less incompetent now. :)

So what is a good fix?

(A) Should gcc pass an -rpath-link to the linker to specify its own library directory? That doesn't really seem to get at the heart of the matter; it really just exploits the happenstance that the stdc++ libraries live in the same directory as the GCC libraries.

(B) Should gcc-config create a file in ${SYSROOT}/etc/env.d to cause env-update to add GCC's LDPATH to ${SYSROOT}/etc/ld.so.conf? This seems like the nicest and most Gentooy solution to me except that it doesn't work because GCC's LDPATH lies outside of the sysroot and so cannot be found by the linker via ld.so.conf lookup.

(C) Should gcc-config add symlinks in ${SYSROOT}/usr/lib? This is kind of dirty, as these symlinks will not be owned by the cross-*/gcc package, but many other packages do similar things with eselect.

(D) Should crossdev initialize ${SYSROOT}/etc/portage/make.conf with an LDFLAGS that specifies a "-Wl,-rpath-link"? This is brittle and will not respond to GCC profile switches. And besides, it will not fix existing cross sysroots.

If I had to pick from the above four choices, I'd go with (C).
Comment 19 Sergei Trofimovich (RETIRED) gentoo-dev 2018-04-07 17:20:33 UTC
(In reply to Matt Whitlock from comment #18)
> (In reply to SpanKY from comment #17)
> 
> Thank you for the closure! I feel less incompetent now. :)
> 
> So what is a good fix?
> 
> (A) Should gcc pass an -rpath-link to the linker to specify its own library
> directory? That doesn't really seem to get at the heart of the matter; it
> really just exploits the happenstance that the stdc++ libraries live in the
> same directory as the GCC libraries.
> 
> (B) Should gcc-config create a file in ${SYSROOT}/etc/env.d to cause
> env-update to add GCC's LDPATH to ${SYSROOT}/etc/ld.so.conf? This seems like
> the nicest and most Gentooy solution to me except that it doesn't work
> because GCC's LDPATH lies outside of the sysroot and so cannot be found by
> the linker via ld.so.conf lookup.
> 
> (C) Should gcc-config add symlinks in ${SYSROOT}/usr/lib? This is kind of
> dirty, as these symlinks will not be owned by the cross-*/gcc package, but
> many other packages do similar things with eselect.
> 
> (D) Should crossdev initialize ${SYSROOT}/etc/portage/make.conf with an
> LDFLAGS that specifies a "-Wl,-rpath-link"? This is brittle and will not
> respond to GCC profile switches. And besides, it will not fix existing cross
> sysroots.
> 
> If I had to pick from the above four choices, I'd go with (C).

I think you can also cross-build gcc itself: ${CTARGET}-emerge -v1 gcc.

Which would be the correct (but heavyweight) way to install libstdc++ into target ROOT.
Comment 20 Sergei Trofimovich (RETIRED) gentoo-dev 2018-04-07 18:59:13 UTC
*** Bug 291383 has been marked as a duplicate of this bug. ***
Comment 21 Sergei Trofimovich (RETIRED) gentoo-dev 2018-04-07 19:00:45 UTC
*** Bug 573304 has been marked as a duplicate of this bug. ***
Comment 22 Joakim Tjernlund 2018-04-08 16:32:30 UTC
(In reply to Matt Whitlock from comment #18)
> (In reply to SpanKY from comment #17)
> 
> Thank you for the closure! I feel less incompetent now. :)
> 
> So what is a good fix?
> 
> (A) Should gcc pass an -rpath-link to the linker to specify its own library
> directory? That doesn't really seem to get at the heart of the matter; it
> really just exploits the happenstance that the stdc++ libraries live in the
> same directory as the GCC libraries.
> 
> (B) Should gcc-config create a file in ${SYSROOT}/etc/env.d to cause
> env-update to add GCC's LDPATH to ${SYSROOT}/etc/ld.so.conf? This seems like
> the nicest and most Gentooy solution to me except that it doesn't work
> because GCC's LDPATH lies outside of the sysroot and so cannot be found by
> the linker via ld.so.conf lookup.

Got to be (B). Some ld.so.conf solution is the way forward I think.
Having to install gcc/binutil into the sysroot is just wrong.
Comment 23 Sergei Trofimovich (RETIRED) gentoo-dev 2019-06-22 07:40:44 UTC
(In reply to Joakim Tjernlund from comment #22)
> (In reply to Matt Whitlock from comment #18)
> > (In reply to SpanKY from comment #17)
> > 
> > Thank you for the closure! I feel less incompetent now. :)
> > 
> > So what is a good fix?
> > 
> > (A) Should gcc pass an -rpath-link to the linker to specify its own library
> > directory? That doesn't really seem to get at the heart of the matter; it
> > really just exploits the happenstance that the stdc++ libraries live in the
> > same directory as the GCC libraries.
> > 
> > (B) Should gcc-config create a file in ${SYSROOT}/etc/env.d to cause
> > env-update to add GCC's LDPATH to ${SYSROOT}/etc/ld.so.conf? This seems like
> > the nicest and most Gentooy solution to me except that it doesn't work
> > because GCC's LDPATH lies outside of the sysroot and so cannot be found by
> > the linker via ld.so.conf lookup.
> 
> Got to be (B). Some ld.so.conf solution is the way forward I think.
> Having to install gcc/binutil into the sysroot is just wrong.

If you want to be able to copy $SYSROOT on target and assume it will be able to run c++ programs you need to install libstdc++ via gcc. Today gentoo has no separate ebuild for it.
Comment 24 Joakim Tjernlund 2019-06-24 15:20:08 UTC
(In reply to Sergei Trofimovich from comment #23)
> (In reply to Joakim Tjernlund from comment #22)
> > (In reply to Matt Whitlock from comment #18)
> > > (In reply to SpanKY from comment #17)
> > > 
> > > Thank you for the closure! I feel less incompetent now. :)
> > > 
> > > So what is a good fix?
> > > 
> > > (A) Should gcc pass an -rpath-link to the linker to specify its own library
> > > directory? That doesn't really seem to get at the heart of the matter; it
> > > really just exploits the happenstance that the stdc++ libraries live in the
> > > same directory as the GCC libraries.
> > > 
> > > (B) Should gcc-config create a file in ${SYSROOT}/etc/env.d to cause
> > > env-update to add GCC's LDPATH to ${SYSROOT}/etc/ld.so.conf? This seems like
> > > the nicest and most Gentooy solution to me except that it doesn't work
> > > because GCC's LDPATH lies outside of the sysroot and so cannot be found by
> > > the linker via ld.so.conf lookup.
> > 
> > Got to be (B). Some ld.so.conf solution is the way forward I think.
> > Having to install gcc/binutil into the sysroot is just wrong.
> 
> If you want to be able to copy $SYSROOT on target and assume it will be able
> to run c++ programs you need to install libstdc++ via gcc. Today gentoo has
> no separate ebuild for it.

This is not so much about running c++ but building. There are working libc++
libs installed into cross gcc but the linker can not find them.

Look at https://bugs.gentoo.org/573304 for some more info.
Comment 25 Sergei Trofimovich (RETIRED) gentoo-dev 2019-12-22 23:55:50 UTC
*** Bug 557236 has been marked as a duplicate of this bug. ***