Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 715162 - "QA Notice: Unresolved soname dependencies" false positives on musl.
Summary: "QA Notice: Unresolved soname dependencies" false positives on musl.
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 721336
Blocks: 694246 711244 721152
  Show dependency tree
 
Reported: 2020-03-28 13:46 UTC by Sergei Trofimovich
Modified: 2020-07-22 16:21 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergei Trofimovich gentoo-dev 2020-03-28 13:46:38 UTC
Attempt to merge any package on musl results in:

"""
 * Messages for package app-shells/bash-5.0_p16:

 * QA Notice: Unresolved soname dependencies:
 *
 * 	/bin/bash: libc.so
 *
"""

The binary looks file:

$ LANG=C readelf -a /bin/bash | fgrep libc
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]

$ lddtree /bin/bash
bash => /bin/bash (interpreter => /lib/ld-musl-x86_64.so.1)
    libreadline.so.8 => /lib/libreadline.so.8
        libtinfow.so.6 => /lib/libtinfow.so.6
    libtinfo.so.6 => /lib/libtinfo.so.6
    libc.so => /usr/lib/libc.so

Looks like 'libc.so' is somehow special.
Comment 1 Sergei Trofimovich gentoo-dev 2020-03-28 13:47:17 UTC
$ emerge --info
Portage 2.3.96 (python 3.6.10-final-0, default/linux/amd64/17.0/musl/hardened, gcc-9.3.0, musl-1.2.0, 5.6.0-rc6-00003-g575ac563c3cc-dirty x86_64)
=================================================================
System uname: Linux-5.6.0-rc6-00003-g575ac563c3cc-dirty-x86_64-Intel-R-_Core-TM-_i7-2700K_CPU_@_3.50GHz-with-gentoo-2.7
KiB Mem:    31803692 total,   1452872 free
KiB Swap:          0 total,         0 free
sh bash 5.0_p16
ld GNU ld (Gentoo 2.32 p2) 2.32.0
ccache version 3.7.8 [enabled]
app-shells/bash:          5.0_p16::gentoo
dev-lang/perl:            5.30.1::gentoo
dev-lang/python:          2.7.17-r1::gentoo, 3.6.10::gentoo
dev-util/ccache:          3.7.8-r1::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.18::gentoo
sys-devel/autoconf:       2.69-r5::gentoo
sys-devel/automake:       1.16.2::gentoo
sys-devel/binutils:       2.32-r1::gentoo, 2.34::gentoo
sys-devel/gcc:            9.2.0-r3::musl, 9.3.0::gentoo
sys-devel/gcc-config:     2.2.1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.5::gentoo (virtual/os-headers)
sys-libs/musl:            1.2.0::gentoo
Repositories:

gentoo
    location: /bound/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts:
    sync-rsync-verify-jobs: 1

co
    location: /co
    masters: gentoo

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-gentoo-linux-musl"
CFLAGS="-O2 -pipe -fdiagnostics-show-option -frecord-gcc-switches -fdiagnostics-show-option -frecord-gcc-switches"
CHOST="x86_64-gentoo-linux-musl"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/bound/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict stricter unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
INSTALL_MASK="charset.alias locale.alias"
LANG="ru_RU.UTF-8"
LC_ALL=""
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--hash-style=gnu"
MAKEOPTS="-j9 -l9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 bzip2 crypt hardened iconv ipv6 libtirpc ncurses nls nptl openmp pam pcre pie readline seccomp split-usr ssl ssp unicode xattr xtpax zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 Sergei Trofimovich gentoo-dev 2020-03-28 13:50:47 UTC
It's probably because musl does not provide SONAME:

sf / # LANG=C readelf -a /usr/lib/libgnutls.so | fgrep SONAME
 0x000000000000000e (SONAME)             Library soname: [libgnutls.so.30]
sf / # LANG=C readelf -a /usr/lib/libc.so | fgrep SONAME
 <empty>
Comment 3 Zac Medico gentoo-dev 2020-03-28 17:56:15 UTC
(In reply to Sergei Trofimovich from comment #2)
> It's probably because musl does not provide SONAME:
> 
> sf / # LANG=C readelf -a /usr/lib/libgnutls.so | fgrep SONAME
>  0x000000000000000e (SONAME)             Library soname: [libgnutls.so.30]
> sf / # LANG=C readelf -a /usr/lib/libc.so | fgrep SONAME
>  <empty>

Yes, and the musl ebuild suppress the corresponding QA notice:

sys-libs/musl/musl-1.1.24.ebuild:QA_SONAME="/usr/lib/libc.so"
sys-libs/musl/musl-1.2.0.ebuild:QA_SONAME="/usr/lib/libc.so"
sys-libs/musl/musl-9999.ebuild:QA_SONAME="/usr/lib/libc.so"
Comment 4 Zac Medico gentoo-dev 2020-03-28 18:01:37 UTC
This could go one of two ways:

1) musl sets soname on libc.so

2) portage begins to pretend there's an implicit soname that corresponds to the file name (which would be consistent with dynamic linking behavior in practice)
Comment 5 Sergei Trofimovich gentoo-dev 2020-03-28 19:42:17 UTC
(In reply to Zac Medico from comment #4)
> This could go one of two ways:
> 
> 1) musl sets soname on libc.so

CCed musl@ if they want to sort it upstream or downstream in sys-libs/musl.

My guess is that lack of SONAME is done deliberately to workaround some other symlinking bug in their ldconfig: https://git.musl-libc.org/cgit/musl/commit/?id=dfdc337b3b276e6ea0e4786ede699f4d0d93dc40

> 2) portage begins to pretend there's an implicit soname that corresponds to
> the file name (which would be consistent with dynamic linking behavior in
> practice)

I think SONAME inference has to be done in practice anyway as private libraries without a SONAME are not uncommon. I'd say SONAME warning against package itself should be enough. No need to proliferate to all packages using it.

Unless Gentoo really-really wants SONAMEs to be present everywhere.
Comment 6 Jory A. Pratt gentoo-dev 2020-03-28 19:53:48 UTC
(In reply to Sergei Trofimovich from comment #5)
> (In reply to Zac Medico from comment #4)
> > This could go one of two ways:
> > 
> > 1) musl sets soname on libc.so
> 
> CCed musl@ if they want to sort it upstream or downstream in sys-libs/musl.
> 
> My guess is that lack of SONAME is done deliberately to workaround some
> other symlinking bug in their ldconfig:
> https://git.musl-libc.org/cgit/musl/commit/
> ?id=dfdc337b3b276e6ea0e4786ede699f4d0d93dc40
> 
> > 2) portage begins to pretend there's an implicit soname that corresponds to
> > the file name (which would be consistent with dynamic linking behavior in
> > practice)
> 
> I think SONAME inference has to be done in practice anyway as private
> libraries without a SONAME are not uncommon. I'd say SONAME warning against
> package itself should be enough. No need to proliferate to all packages
> using it.
> 
> Unless Gentoo really-really wants SONAMEs to be present everywhere.

Upstream will not add a soname. They have no need for it and strongly disagree with how it implemented which will cause major issues if someone install musl on a glibc system just to be able to run a musl linked binary
Comment 7 Zac Medico gentoo-dev 2020-03-28 19:54:13 UTC
(In reply to Sergei Trofimovich from comment #5)
> I think SONAME inference has to be done in practice anyway as private
> libraries without a SONAME are not uncommon.

This particular case for private libraries is already handled by the patch for bug 646190:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1364cd44e7a6232bf425c4573b5bd3d6738d49a4

> I'd say SONAME warning against package itself should be enough. No need to
> proliferate to all packages using it.

For soname dependency support, portage needs to resolve the soname dependency somehow. So, the SONAME must either be present or inferred, and if we infer it for musl then that means we'll infer it for any other package that behaves similarly.

> Unless Gentoo really-really wants SONAMEs to be present everywhere.

That seems like an uphill battle, given that dynamic linking works without it.
Comment 8 Zac Medico gentoo-dev 2020-03-28 20:06:59 UTC
(In reply to Jory A. Pratt from comment #6)
> Upstream will not add a soname. They have no need for it and strongly
> disagree with how it implemented which will cause major issues if someone
> install musl on a glibc system just to be able to run a musl linked binary

I had no idea that co-installation of musl libc.so along with glibc was a thing, but anyway, I guess having a libc.so SONAME might not result in a collision since the glibc soname is libc.so.6 instead.

Anyway, I'm happy to make portage infer the SONAME for musl and anything else if that makes people happy.
Comment 9 Sergei Trofimovich gentoo-dev 2020-03-28 20:07:37 UTC
(In reply to Jory A. Pratt from comment #6)
> Upstream will not add a soname. They have no need for it and strongly
> disagree with how it implemented which will cause major issues if someone
> install musl on a glibc system just to be able to run a musl linked binary

Oh, so it's an glibc's ldconfig that breaks it. Then we should avoid it downstream as well. Worth adding a comment to musl ebuild? I can work on a wording.

(In reply to Zac Medico from comment #7)
> (In reply to Sergei Trofimovich from comment #5)
> > I think SONAME inference has to be done in practice anyway as private
> > libraries without a SONAME are not uncommon.
> 
> This particular case for private libraries is already handled by the patch
> for bug 646190:
> 
> https://gitweb.gentoo.org/proj/portage.git/commit/
> ?id=1364cd44e7a6232bf425c4573b5bd3d6738d49a4
> 
> > I'd say SONAME warning against package itself should be enough. No need to
> > proliferate to all packages using it.
> 
> For soname dependency support, portage needs to resolve the soname
> dependency somehow. So, the SONAME must either be present or inferred, and
> if we infer it for musl then that means we'll infer it for any other package
> that behaves similarly.

AFAIU the pseudocode should be something like:

    get_soname(filename):
        if exists(DT_SONAME, filename):
            return value(DT_SONAME, filename)
        return basename(filename)

For both files and symlinks (if feasible). That way a package with files:
    libfoo.so
    libfoo.so.1 -> libfoo.so
would provide both SONAMEs: libfoo.so and libfoo.so.1. I think that's what we want.
Comment 10 Zac Medico gentoo-dev 2020-03-28 22:20:16 UTC
(In reply to Sergei Trofimovich from comment #9)
> AFAIU the pseudocode should be something like:
> 
>     get_soname(filename):
>         if exists(DT_SONAME, filename):
>             return value(DT_SONAME, filename)
>         return basename(filename)

Looks good, at least for regular files. Symlinks are different, see below.

> For both files and symlinks (if feasible). That way a package with files:
>     libfoo.so
>     libfoo.so.1 -> libfoo.so
> would provide both SONAMEs: libfoo.so and libfoo.so.1. I think that's what
> we want.

As an example, let's consider a library that bumps its SONAME on a regular basis:

> /usr/lib/libicuio.so -> libicuio.so.65.1
> /usr/lib/libicuio.so.65 -> libicuio.so.65.1
> /usr/lib/libicuio.so.65.1

The DT_NEEDED entries for consumers should refer to the libicuio.so.65 SONAME. When we upgrade to icu-66 and preserve-libs kicks in, the consumers that link to libicuio.so.65.1 will still be usable, meanwhile the libicuio.so symlink will point to libicuio.so.66.1 and newly built packages can use this as a hint that they should link to the new libicuio.so.66.1 SONAME.

Given this example, it seems like it would be too greedy to infer a libicuio.so SONAME from the libicuio.so symlink.
Comment 11 Sergei Trofimovich gentoo-dev 2020-03-28 23:53:11 UTC
(In reply to Zac Medico from comment #10)
> > For both files and symlinks (if feasible). That way a package with files:
> >     libfoo.so
> >     libfoo.so.1 -> libfoo.so
> > would provide both SONAMEs: libfoo.so and libfoo.so.1. I think that's what
> > we want.

I forgot to mention that this imaginary libfoo.so has no SONAME tag (I picked the silly example). Thus users that link against -lfoo will get DT_NEEDED="libfoo.so", ones with /usr/lib/libfoo.so.1 (for whatever reason). will get DT_NEEDED="libfoo.so.1"

Maybe better example would be:
    libimpl1.so
    libvirtual.so -> libimpl1.so

> As an example, let's consider a library that bumps its SONAME on a regular
> basis:
> 
> > /usr/lib/libicuio.so -> libicuio.so.65.1
> > /usr/lib/libicuio.so.65 -> libicuio.so.65.1
> > /usr/lib/libicuio.so.65.1
> 
> The DT_NEEDED entries for consumers should refer to the libicuio.so.65
> SONAME. When we upgrade to icu-66 and preserve-libs kicks in, the consumers
> that link to libicuio.so.65.1 will still be usable, meanwhile the
> libicuio.so symlink will point to libicuio.so.66.1 and newly built packages
> can use this as a hint that they should link to the new libicuio.so.66.1
> SONAME.
> 
> Given this example, it seems like it would be too greedy to infer a
> libicuio.so SONAME from the libicuio.so symlink.

I assume in you example libicuio.so.65.1 has no SONAME=libicuio.so.65 tag, is it correct? Otherwise for all three files algorithm would derive SONAME=libicuio.so.65 provider.
Comment 12 Zac Medico gentoo-dev 2020-03-29 03:43:49 UTC
(In reply to Sergei Trofimovich from comment #11)
> (In reply to Zac Medico from comment #10)
> > > For both files and symlinks (if feasible). That way a package with files:
> > >     libfoo.so
> > >     libfoo.so.1 -> libfoo.so
> > > would provide both SONAMEs: libfoo.so and libfoo.so.1. I think that's what
> > > we want.
> 
> I forgot to mention that this imaginary libfoo.so has no SONAME tag (I
> picked the silly example).

Oh, I see now. The lack of SONAME tag is an essential detail.

> Thus users that link against -lfoo will get
> DT_NEEDED="libfoo.so", ones with /usr/lib/libfoo.so.1 (for whatever reason).
> will get DT_NEEDED="libfoo.so.1"

Experimenting locally, what I see is that it follows the symlink, resulting in DT_NEEDED="libfoo.so" instead of DT_NEEDED="libfoo.so.1".

> Maybe better example would be:
>     libimpl1.so
>     libvirtual.so -> libimpl1.so

Based on my local experiments I'd expect to see it follow the symlink, resulting in DT_NEEDED="libimpl1.so".

> I assume in you example libicuio.so.65.1 has no SONAME=libicuio.so.65 tag,
> is it correct? Otherwise for all three files algorithm would derive
> SONAME=libicuio.so.65 provider.

Lets forget my example, since I was thinking in terms of things that *do* have an SONAME tag. It's now off-topic since we have no intention of changing the behavior for things that have an SONAME tag.
Comment 14 Jory A. Pratt gentoo-dev 2020-03-29 19:04:19 UTC
(In reply to Zac Medico from comment #13)
> Patch posted for review:
> 
> https://archives.gentoo.org/gentoo-portage-dev/message/
> 2c8ee101d7877d635795fe55c9e13ccd
> https://github.com/gentoo/portage/pull/540

 * QA Notice: Unresolved soname dependencies:
 * 
 * 	/usr/lib/python3.6/site-packages/portage/util/file_copy/reflink_linux.cpython-36m-x86_64-linux-gnu.so: libc.so
 * 	/usr/lib/python3.6/site-packages/portage/util/libc.cpython-36m-x86_64-linux-gnu.so: libc.so
 * 
That is after patching portage
Comment 15 Zac Medico gentoo-dev 2020-03-29 19:08:05 UTC
(In reply to Jory A. Pratt from comment #14)
> (In reply to Zac Medico from comment #13)
> > Patch posted for review:
> > 
> > https://archives.gentoo.org/gentoo-portage-dev/message/
> > 2c8ee101d7877d635795fe55c9e13ccd
> > https://github.com/gentoo/portage/pull/540
> 
>  * QA Notice: Unresolved soname dependencies:
>  * 
>  * 
> /usr/lib/python3.6/site-packages/portage/util/file_copy/reflink_linux.
> cpython-36m-x86_64-linux-gnu.so: libc.so
>  * 
> /usr/lib/python3.6/site-packages/portage/util/libc.cpython-36m-x86_64-linux-
> gnu.so: libc.so
>  * 
> That is after patching portage

It won't have any effect until you rebuild that package that provides libc.so.

Alternatively, you can manually edit the /var/db/pkg/*/*/NEEDED.ELF.2 file for that package and then touch the parent directory timestamp in order to invalidate the cache.
Comment 16 Zac Medico gentoo-dev 2020-03-29 19:16:31 UTC
Edit the line in /var/db/pkg/sys-libs/musl-1.1.24/NEEDED.ELF.2 to look something like this (insert libc.so after second semicolon):

X86_64;/lib64/libc.so;libc.so;;;x86_64


Then touch /var/db/pkg/sys-libs/musl-1.1.24 to invalidate the cache.
Comment 17 Jory A. Pratt gentoo-dev 2020-03-29 19:55:56 UTC
(In reply to Zac Medico from comment #16)
> Edit the line in /var/db/pkg/sys-libs/musl-1.1.24/NEEDED.ELF.2 to look
> something like this (insert libc.so after second semicolon):
> 
> X86_64;/lib64/libc.so;libc.so;;;x86_64
> 
> 
> Then touch /var/db/pkg/sys-libs/musl-1.1.24 to invalidate the cache.

went ahead and rebuilt musl, still produces the same false positive

Updated NEED.ELF.2 as generated by re-emerging musl
86_64;/usr/bin/getent;getent;;libc.so;x86_64
X86_64;/usr/bin/iconv;iconv;;libc.so;x86_64
X86_64;/usr/bin/getconf;getconf;;libc.so;x86_64
Comment 18 Zac Medico gentoo-dev 2020-03-29 20:13:34 UTC
(In reply to Jory A. Pratt from comment #17)
> (In reply to Zac Medico from comment #16)
> > Edit the line in /var/db/pkg/sys-libs/musl-1.1.24/NEEDED.ELF.2 to look
> > something like this (insert libc.so after second semicolon):
> > 
> > X86_64;/lib64/libc.so;libc.so;;;x86_64
> > 
> > 
> > Then touch /var/db/pkg/sys-libs/musl-1.1.24 to invalidate the cache.
> 
> went ahead and rebuilt musl, still produces the same false positive
> 
> Updated NEED.ELF.2 as generated by re-emerging musl
> 86_64;/usr/bin/getent;getent;;libc.so;x86_64
> X86_64;/usr/bin/iconv;iconv;;libc.so;x86_64
> X86_64;/usr/bin/getconf;getconf;;libc.so;x86_64

The important line is not shown here. Where's the line with /lib*/libc.so in the second column?

The patch is generating an soname for regular (dynamically linked) executables, not just shared libraries, so I need to look into fixing that.
Comment 19 Zac Medico gentoo-dev 2020-03-29 20:42:09 UTC
In order to distinguish PIE executables from dynamic libraries, we can emulate what they're doing here:

https://github.com/file/file/commit/03084b161cf888b5286dbbcd964c31ccad4f64d9
https://github.com/file/file/commit/9109a696f3289ba00eaa222fd432755ec4287e28
Comment 20 Zac Medico gentoo-dev 2020-04-03 06:01:22 UTC
Updated patch to use file command to distinguish shared library from PIE executable:

https://archives.gentoo.org/gentoo-portage-dev/message/bc996edc60e3e1ec5c8e5ecdcb838c32
Comment 21 Larry the Git Cow gentoo-dev 2020-04-08 06:15:30 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=25fbe7bc1a92a6877f2fecfba93ac932f359ce2f

commit 25fbe7bc1a92a6877f2fecfba93ac932f359ce2f
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-29 18:09:33 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-04-08 06:02:45 +0000

    NeededEntry: infer implicit soname from file basename (bug 715162)
    
    For dynamic libraries, infer an implicit DT_SONAME setting from the
    file basename, which is consistent with dynamic linking behavior in
    practice. This makes it possible to resolve soname dependencies for
    musl's libc.so which lacks a DT_SONAME setting.
    
    Bug: https://bugs.gentoo.org/715162
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 bin/misc-functions.sh                       |  6 ++++++
 lib/portage/util/_dyn_libs/LinkageMapELF.py | 16 ++++++++++++++++
 2 files changed, 22 insertions(+)
Comment 22 Larry the Git Cow gentoo-dev 2020-04-08 06:55:53 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b7ff1f6c1bf0e821e565c606d7df7d70bf575347

commit b7ff1f6c1bf0e821e565c606d7df7d70bf575347
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-04-08 06:52:08 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-04-08 06:55:33 +0000

    sys-apps/portage: Bump to version 2.3.97
    
     #709746 temporarily remove PORTAGE_LOG_FILTER_FILE_CMD support
     #715162 infer implicit soname from file basename, for musl
     #716636 emerge hangs in releases after 2.3.89-r1
    
    Bug: https://bugs.gentoo.org/711148
    Bug: https://bugs.gentoo.org/709746
    Bug: https://bugs.gentoo.org/715162
    Closes: https://bugs.gentoo.org/716636
    Package-Manager: Portage-2.3.97, Repoman-2.3.22
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.97.ebuild | 263 +++++++++++++++++++++++++++++++++
 2 files changed, 264 insertions(+)
Comment 23 Zac Medico gentoo-dev 2020-04-30 21:13:50 UTC
NOTE: It looks like we may have found a bug in pax-utils scanelf, since the scanelf command that portage uses to generate NEEDED.ELF.2 does not even identify musl's /usr/lib/libc.so as an ELF file:

> $ scanelf -qyRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
> $
Comment 24 Sergei Trofimovich gentoo-dev 2020-04-30 21:30:39 UTC
(In reply to Zac Medico from comment #23)
> NOTE: It looks like we may have found a bug in pax-utils scanelf, since the
> scanelf command that portage uses to generate NEEDED.ELF.2 does not even
> identify musl's /usr/lib/libc.so as an ELF file:
> 
> > $ scanelf -qyRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
> > $

I'll have a look.
Comment 25 Sergei Trofimovich gentoo-dev 2020-04-30 21:35:13 UTC
Probably a negative/unexpected effect of '-q' option:

# scanelf -qF '%a' /usr/lib/*.so
# scanelf -F '%a' /usr/lib/libc.so
ARCH FILE
EM_X86_64 /usr/lib/libc.so
Comment 26 Sergei Trofimovich gentoo-dev 2020-04-30 22:01:00 UTC
(In reply to Sergei Trofimovich from comment #25)
> Probably a negative/unexpected effect of '-q' option:
> 
> # scanelf -qF '%a' /usr/lib/*.so
> # scanelf -F '%a' /usr/lib/libc.so
> ARCH FILE
> EM_X86_64 /usr/lib/libc.so

I believe it always been like that. If you need to omit header you might need 'B' instead of 'q':

$ scanelf -ByRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
EM_X86_64;/usr/lib/libc.so;;  -  ;

I'll check why 'q' happens to work on glibc.
Comment 27 Jory A. Pratt gentoo-dev 2020-04-30 23:45:36 UTC
(In reply to Sergei Trofimovich from comment #26)
> (In reply to Sergei Trofimovich from comment #25)
> > Probably a negative/unexpected effect of '-q' option:
> > 
> > # scanelf -qF '%a' /usr/lib/*.so
> > # scanelf -F '%a' /usr/lib/libc.so
> > ARCH FILE
> > EM_X86_64 /usr/lib/libc.so
> 
> I believe it always been like that. If you need to omit header you might
> need 'B' instead of 'q':
> 
> $ scanelf -ByRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
> EM_X86_64;/usr/lib/libc.so;;  -  ;
> 
> I'll check why 'q' happens to work on glibc.

ack, I have tested as well on musl, after rebuilding musl am no longer seeing libc soname reports
Comment 28 Zac Medico gentoo-dev 2020-05-09 03:17:04 UTC
(In reply to Sergei Trofimovich from comment #26)
> (In reply to Sergei Trofimovich from comment #25)
> > Probably a negative/unexpected effect of '-q' option:
> > 
> > # scanelf -qF '%a' /usr/lib/*.so
> > # scanelf -F '%a' /usr/lib/libc.so
> > ARCH FILE
> > EM_X86_64 /usr/lib/libc.so
> 
> I believe it always been like that. If you need to omit header you might
> need 'B' instead of 'q':
> 
> $ scanelf -ByRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
> EM_X86_64;/usr/lib/libc.so;;  -  ;
> 
> I'll check why 'q' happens to work on glibc.

I have this patch that works:

https://github.com/gentoo/pax-utils/pull/3
Comment 29 Zac Medico gentoo-dev 2020-05-09 21:31:10 UTC
(In reply to Zac Medico from comment #28)
> (In reply to Sergei Trofimovich from comment #26)
> > (In reply to Sergei Trofimovich from comment #25)
> > > Probably a negative/unexpected effect of '-q' option:
> > > 
> > > # scanelf -qF '%a' /usr/lib/*.so
> > > # scanelf -F '%a' /usr/lib/libc.so
> > > ARCH FILE
> > > EM_X86_64 /usr/lib/libc.so
> > 
> > I believe it always been like that. If you need to omit header you might
> > need 'B' instead of 'q':
> > 
> > $ scanelf -ByRF '%a;%p;%S;%r;%n' /usr/lib/libc.so
> > EM_X86_64;/usr/lib/libc.so;;  -  ;
> > 
> > I'll check why 'q' happens to work on glibc.
> 
> I have this patch that works:
> 
> https://github.com/gentoo/pax-utils/pull/3

The reason that -q works on glibc is that the scanelf_elfobj function sets found_soname = 1, so that FOUND_SOMETHING() returns true. In order of scanelf -q to output something for musl's /usr/lib/libc.so here, it would have to infer an implicit soname.