Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 424423 - sys-apps/portage: FEATURES="multilib-strict" prevents incorrectly sys-fs/udev and sys-auth/polkit installation
Summary: sys-apps/portage: FEATURES="multilib-strict" prevents incorrectly sys-fs/udev...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All All
: High major (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 423381 424884 479522 (view as bug list)
Depends on:
Blocks: 627566
  Show dependency tree
 
Reported: 2012-07-01 17:34 UTC by Samuli Suominen (RETIRED)
Modified: 2018-02-01 23:49 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Samuli Suominen (RETIRED) gentoo-dev 2012-07-01 17:34:26 UTC
Upstream(s) have started moving ELF binaries from /usr/libexec/ to /usr/lib/$package/

For example of this, sys-apps/polkit-0.106:

src/polkitbackend/Makefile.am:libprivdir = $(prefix)/lib/polkit-1
src/polkitagent/polkitagentsession.c:  helper_argv[0] = PACKAGE_PREFIX "/lib/polkit-1/polkit-agent-helper-1";

So instead of patching these packages to comply with multilib-strict for the binaries, I'd rather see the multilib-strict only check for files with library extension in filename, such as .so .a and .dylib

So...

I don't have any strong opinion of this whatsoever, but I agree with what ABCD said and upstreams are doing.

#gentoo-dev:
20:06 <@ABCD> ssuominen: polkitd should be /usr/lib/polkit-1/polkitd on *every* system, even ones with multilib + no /lib symlink
Comment 1 Samuli Suominen (RETIRED) gentoo-dev 2012-07-01 17:36:29 UTC
And this is happening because upstreams (read: Fedora) are moving away from using /usr/libexec/ entirely and moving everything to /usr/lib/$package/
This in result causes freedesktop.org packages to move to /usr/lib/$package/
And that in result causes things like the *entire* Xfce project to do the same
Comment 2 Samuli Suominen (RETIRED) gentoo-dev 2012-07-02 16:18:56 UTC
See this link for a much better explanation I could ever give,

https://bugs.freedesktop.org/show_bug.cgi?id=51617#c2
Comment 3 Samuli Suominen (RETIRED) gentoo-dev 2012-07-02 16:31:04 UTC
  RESTRICT.invalid              1
   sys-auth/polkit/polkit-0.106-r7.ebuild: multilib-strict

At least it worked ;-)

http://sources.gentoo.org/viewvc.cgi/gentoo-x86/sys-auth/polkit/ChangeLog?r1=1.95&r2=1.96

>>> /usr/lib/polkit-1/
>>> /usr/lib/polkit-1/polkit-agent-helper-1
>>> /usr/lib/polkit-1/polkitd

Moved the 2 binaries back to /usr/lib/ even when they are 64bit ELF's as per upstream recommendation
Comment 4 Ulenrich 2012-07-02 18:37:02 UTC
I often do see some USE like "(multilib)". This shows I have a amd64 multilib system I guess. I don't believe ever have used 32bit, but nevertheless I want to know:

a) /usr/lib is a symbolic link showing /usr/lib64
b) /usr/lib64/polkit-1 exists

Is my system correctly configured? 
If, then why do I need the upgrade to polkit-0.106-r7
Or, is this just a correction for "non true multilib" systems?
Comment 5 Samuli Suominen (RETIRED) gentoo-dev 2012-07-02 18:52:37 UTC
(In reply to comment #4)
> a) /usr/lib is a symbolic link showing /usr/lib64
> b) /usr/lib64/polkit-1 exists
> 
> Is my system correctly configured? 

yes, and b) is fixed by -r7

> If, then why do I need the upgrade to polkit-0.106-r7
> Or, is this just a correction for "non true multilib" systems?

you don't "need" to do anything but you "can" install -r7 to move the binaries back to where they belong in /usr/lib/polkit-1/ since it doesn't make sense to have both, 32bit and 64bit copies of them, since /usr/lib/polkit-1/ is comparable to /usr/libexec/
Comment 6 Zac Medico gentoo-dev 2012-07-02 21:17:50 UTC
Looking at the multlib-strict code here:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=blob;f=bin/misc-functions.sh

I don't understand why it would apply to all ELFs, and not just libraries, because it filters them with MULTILIB_STRICT_DENY like this:

if file ${file} | egrep -q "${MULTILIB_STRICT_DENY}" ; then
	if [[ ${firstrun} == yes ]] ; then
		echo "Files matching a file type that is not allowed:"
		firstrun=no
	fi
	abort=yes
	echo "   ${file#${ED}//}"
fi

And profiles/features/multilib/make.defaults contains MULTILIB_STRICT_DENY="64-bit.*shared object", so shouldn't that only match shared libraries?

Compare the 'file' output for the ELFs in /usr/lib/polkit-1:
/usr/lib/polkit-1/polkit-agent-helper-1: setuid ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, stripped
/usr/lib/polkit-1/polkitd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, stripped
Comment 7 Alexis Ballier gentoo-dev 2012-07-03 18:13:38 UTC
(In reply to comment #6)

> And profiles/features/multilib/make.defaults contains
> MULTILIB_STRICT_DENY="64-bit.*shared object", so shouldn't that only match
> shared libraries?

not sure if this is relevant, but this also matches pie executables:

$ gcc -pie -fPIE foo.c 
$ file a.out 
a.out: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
Comment 8 Michael Palimaka (kensington) gentoo-dev 2012-07-03 18:46:29 UTC
(In reply to comment #3)
>   RESTRICT.invalid              1
>    sys-auth/polkit/polkit-0.106-r7.ebuild: multilib-strict
> 
> At least it worked ;-)

Note that this does not work with versions of Portage that are currently keyworded.
Comment 9 Zac Medico gentoo-dev 2012-07-03 20:30:21 UTC
(In reply to comment #7)
> (In reply to comment #6)
> 
> > And profiles/features/multilib/make.defaults contains
> > MULTILIB_STRICT_DENY="64-bit.*shared object", so shouldn't that only match
> > shared libraries?
> 
> not sure if this is relevant, but this also matches pie executables:
> 
> $ gcc -pie -fPIE foo.c 
> $ file a.out 
> a.out: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically
> linked (uses shared libs), for GNU/Linux 2.6.9, not stripped

I guess that's probably the reason. Thanks.
Comment 10 Zac Medico gentoo-dev 2012-07-03 20:38:47 UTC
(In reply to comment #0)
> So instead of patching these packages to comply with multilib-strict for the
> binaries, I'd rather see the multilib-strict only check for files with
> library extension in filename, such as .so .a and .dylib

This has me wondering if MULTILIB_STRICT_DENY is not functioning as intended, given comment #7. Should we get rid of the MULTILIB_STRICT_DENY variable if we're going to use file extensions instead? Should we add another variable that lists file extensions instead? If necessary, we could also add another variable that allows the ebuild to declare exemptions for specific files.
Comment 11 Samuli Suominen (RETIRED) gentoo-dev 2012-07-06 09:00:07 UTC
(In reply to comment #10)
> (In reply to comment #0)
> > So instead of patching these packages to comply with multilib-strict for the
> > binaries, I'd rather see the multilib-strict only check for files with
> > library extension in filename, such as .so .a and .dylib
> 
> This has me wondering if MULTILIB_STRICT_DENY is not functioning as
> intended, given comment #7. Should we get rid of the MULTILIB_STRICT_DENY
> variable if we're going to use file extensions instead? 

Or maybe we should be looking for something else, like output of `objdump`?
Well, can't think of anything else than file extensions right now

> Should we add
> another variable that lists file extensions instead? If necessary, we could
> also add another variable that allows the ebuild to declare exemptions for
> specific files.

That would be a nice bonus but not a solution to this bug
Comment 12 Davide Pesavento gentoo-dev 2012-07-08 09:18:26 UTC
What about looking at the SONAME? Would that work?
Something like:

if [[ -n $(objdump -p "${file}" | grep "^[[:blank:]]*SONAME") ]]; then
    : shared library
else
    : executable
fi

(Or `scanelf -qS` if we can rely on pax-utils being installed).
Comment 13 Samuli Suominen (RETIRED) gentoo-dev 2012-07-08 11:25:07 UTC
(In reply to comment #12)
> What about looking at the SONAME? Would that work?
> Something like:
> 
> if [[ -n $(objdump -p "${file}" | grep "^[[:blank:]]*SONAME") ]]; then
>     : shared library
> else
>     : executable
> fi
> 
> (Or `scanelf -qS` if we can rely on pax-utils being installed).

The library might be missing SONAME or it could be a static archive in which case it wouldn't have SONAME in the first place.
Comment 14 Davide Pesavento gentoo-dev 2012-07-08 14:20:38 UTC
(In reply to comment #13)

The soname check was meant to be in addition to the existing file(1) check, sorry if it wasn't clear from my comment. And since file(1) outputs "current ar archive" for static libs, they're already excluded regardless of the soname check.

Yes, some shared libs might lack a soname, but I think there are very few of them, and the worst that can happen is that those libs are not checked for multilib-strictness, which is acceptable IMHO.
Comment 15 Samuli Suominen (RETIRED) gentoo-dev 2012-08-12 08:33:38 UTC
I've removed 'multilib-strict' from profiles/targets/developer/make.defaults for now because it's not usable before this bug is handled...
Comment 16 Samuli Suominen (RETIRED) gentoo-dev 2012-09-30 13:46:53 UTC
*** Bug 423381 has been marked as a duplicate of this bug. ***
Comment 17 Samuli Suominen (RETIRED) gentoo-dev 2012-09-30 13:47:17 UTC
*** Bug 424884 has been marked as a duplicate of this bug. ***
Comment 18 Samuli Suominen (RETIRED) gentoo-dev 2012-09-30 13:50:39 UTC
Note. I'm filing STABLEREQ now for the sys-auth/polkit version that won't install with FEATURES="multilib-strict".
Comment 19 Zac Medico gentoo-dev 2012-10-15 02:55:01 UTC
In portage-2.1.11.27 and 2.2.0_alpha138 we have support for a QA_MULTILIB_PATHS variable (bug #437910). I've updated the latest udev and polkit ebuilds to use it.
Comment 20 Michael Palimaka (kensington) gentoo-dev 2013-01-31 15:07:37 UTC
+  31 Jan 2013; Michael Palimaka <kensington@gentoo.org> udisks-2.0.91.ebuild:
+  Set QA_MULTILIB_PATHS wrt bug #424423.
Comment 21 Ian Delaney (RETIRED) gentoo-dev 2013-02-11 10:20:48 UTC
add xen-tools to the list, with USE qemu, it can't get past strict-multilib during install.
Comment 22 Zac Medico gentoo-dev 2013-02-11 15:13:33 UTC
(In reply to comment #21)
> add xen-tools to the list, with USE qemu, it can't get past strict-multilib
> during install.

Please feel free to add QA_MULTILIB_PATHS to any ebuilds where it make sense.
Comment 23 Francisco J. Vazquez 2013-03-01 09:09:22 UTC
So, we should be able to remove the /lib->/lib64 and /usr/lib->/usr/lib64 symlinks, make them directories and re-emerge all the packages that an "equery belongs /lib /usr/lib" report? Or is this not supported?

BTW, wouldn't it be better to use a different directory (e.g.,/libexec)? To me it doesn't make much sense to have libraries mixed with these files in x86 and separate in amd64 (no-multilib).
Comment 24 Samuli Suominen (RETIRED) gentoo-dev 2013-03-01 09:13:27 UTC
(In reply to comment #23)
> So, we should be able to remove the /lib->/lib64 and /usr/lib->/usr/lib64
> symlinks, make them directories and re-emerge all the packages that an
> "equery belongs /lib /usr/lib" report? Or is this not supported?

The system should keep working without the lib -> lib64 symlink, as in, SHOULD
If things will keep working after removing the symlink from your system, I have no idea

> BTW, wouldn't it be better to use a different directory (e.g.,/libexec)? To
> me it doesn't make much sense to have libraries mixed with these files in
> x86 and separate in amd64 (no-multilib).

lib/<package> is same as libexec, except you can put other files to lib/<package> too like udev does. 
it doesn't make sense to move part of the files out from lib/<package> to libexec. there is no need for such change, other than the wrong multilib-strict check in portage.
Comment 25 Jonathan Callen (RETIRED) gentoo-dev 2013-03-01 17:57:27 UTC
While it is possible to split /lib from /lib64, doing so requires some manual effort, including modifiying the vdb directly to let portage know what you did.

Here is a rough outline of what I did on my system to do the split and to move from using /lib32 to /lib for 32-bit:

  - Remove the /lib and /usr/lib symlinks and replace them with directories
  - Go through the CONTENTS of all installed packages and determine if portage
    thinks the package was installed to {/usr,}/lib or {,/usr}/lib64
  - If portage thinks a file should be in /lib or /usr/lib, move it there (this
    is absolutely required for gcc to work and may now be required for any C++
    program to work)
  - Move /lib64/ld-linux.so.2 to /lib/ld-linux.so.2 and update glibc's CONTENTS
    file to match.
  - Update baselayout's CONTENTS file to not mention the symlinks /lib and
    /usr/lib (may not be required)
  - Add the following lines to make.conf:
      SYMLINK_LIB=no
      LIBDIR_x86=lib
  - Rebuild everything
Comment 26 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-02 14:30:22 UTC
*** Bug 479522 has been marked as a duplicate of this bug. ***
Comment 27 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-02 14:38:33 UTC
Does anyone have an idea how this should work in the end? I'd rather see it fixed before we have 200-character long MULTILIB_STRICT_EXEMPT.

For example, gcc & clang are installing libraries in /usr/lib as well (since they're ABI-versioned other way).
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-02 14:41:01 UTC
*** Bug 479520 has been marked as a duplicate of this bug. ***
Comment 29 Zac Medico gentoo-dev 2017-07-27 16:18:31 UTC
(In reply to Samuli Suominen from comment #15)
> I've removed 'multilib-strict' from profiles/targets/developer/make.defaults
> for now because it's not usable before this bug is handled...

Yes, I see that you disabled it here:

https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/targets/developer/make.defaults?r1=1.11&r2=1.12

There's a proposal to enable multilib-strict by default in portage, but really it first needs to be good enough to enable by default in the developer profile.
Comment 30 Zac Medico gentoo-dev 2017-08-01 05:28:23 UTC
This patch disables recursion, eliminating the need for MULTILIB_STRICT_EXEMPT:

https://archives.gentoo.org/gentoo-portage-dev/message/b347da6d34e3f2b6d260f785ab144330
https://github.com/gentoo/portage/pull/184
Comment 31 Zac Medico gentoo-dev 2017-08-02 07:27:51 UTC
Recursion is now disabled in the master branch:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=864585bc32dc0ccad8f53aec03d378d37b385c21