Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 215242 - sys-apps/portage: preserved-libs preserves locally bundled libs of pre-built packages
Summary: sys-apps/portage: preserved-libs preserves locally bundled libs of pre-built ...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: High normal with 1 vote (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 382921 493848 (view as bug list)
Depends on:
Blocks: preserve-libs
  Show dependency tree
 
Reported: 2008-03-28 19:55 UTC by Doug Goldstein (RETIRED)
Modified: 2022-10-30 16:53 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Doug Goldstein (RETIRED) gentoo-dev 2008-03-28 19:55:18 UTC
I've found two issues. First one being /opt library upgrades. i.e.

!!! existing preserved libs:
>>> package: app-text/acroread-8.1.2-r1
 *  - /opt/Acrobat7/Reader/intellinux/lib/libcrypto.so.0.9.6
 *  - /opt/Acrobat7/Reader/intellinux/lib/libssl.so.0.9.6

It's a binary only pkg and nothing is linked to it. /opt should probably be excluded from preserved-libs

The second issue:

>>> package: gnome-base/gnome-applets-2.22.0
 *  - /usr/lib64/libgweather.so.0
 *  - /usr/lib64/libgweather.so.0.0.0

The above files used to be owned by gnome-base/gnome-applets-2.20 but then gnome-applets-2.22 broke it out into it's own package called dev-libs/libgweather. The .so versions stayed the same, the library just migrated from 1 package to another. No matter how many times you emerge @preserved-libs, it won't get rid of that msg.
Comment 1 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2008-03-29 19:18:49 UTC
(In reply to comment #0)
> !!! existing preserved libs:
> >>> package: app-text/acroread-8.1.2-r1
>  *  - /opt/Acrobat7/Reader/intellinux/lib/libcrypto.so.0.9.6
>  *  - /opt/Acrobat7/Reader/intellinux/lib/libssl.so.0.9.6
> 
> It's a binary only pkg and nothing is linked to it.

Post the output of 'emerge -ptv @preserved-rebuild'.
Comment 2 Doug Goldstein (RETIRED) gentoo-dev 2008-04-02 00:24:27 UTC
These libraries are installed by the acroread package. The 7.x series of the package installed openssl 0.9.6 while the 8.x series uses 0.9.7. The libraries are specifically provided as shared objects so that security upgrades are easy, rather then requiring a whole new package.

There is NOTHING that links to these. In fact, NOTHING should link to libs in /opt other then the package itself.
Comment 3 Marius Mauch (RETIRED) gentoo-dev 2008-04-02 02:32:33 UTC
Guess we should not preserve libs if a copy exists in one of the standard library dirs. Ignoring just /opt completely is wrong IMO as we want to ignore internal libraries, which may very well exist outside /opt and there might be "real" libs inside /opt (plus I really hate such special cases).
Comment 4 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2008-04-02 15:49:37 UTC
(In reply to comment #3)
> we want to ignore internal libraries

So you didn't fix Bug #205531 properly?
Comment 5 Marius Mauch (RETIRED) gentoo-dev 2008-04-02 17:09:07 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > we want to ignore internal libraries
> 
> So you didn't fix Bug #205531 properly?

That's the opposite issue.
Comment 6 Marius Mauch (RETIRED) gentoo-dev 2008-04-06 17:14:17 UTC
The issue with internal libs should be fixed in SVN, still have to look into the gweather problem.
Comment 7 Fabian Groffen gentoo-dev 2008-04-11 10:26:08 UTC
I'm wondering if the fix you  made in r9728 isn't a guarantee that no lib is ever preserved at all.

If you find lib in any of the library paths (almost any), you remove the lib from the candidate list.  However, this is done prior to unmerging the old package, so the lib is always installed in the system, isn't it?
Comment 8 Fabian Groffen gentoo-dev 2008-04-11 11:21:32 UTC
The commit actually breaks subversion/neon preserve libs stuff.
Comment 9 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2008-04-23 17:45:41 UTC
(In reply to comment #7)
> I'm wondering if the fix you  made in r9728 isn't a guarantee that no lib is
> ever preserved at all.
> 
> If you find lib in any of the library paths (almost any), you remove the lib
> from the candidate list.  However, this is done prior to unmerging the old
> package, so the lib is always installed in the system, isn't it?

It was fixed in r9862.
Comment 10 Fabian Groffen gentoo-dev 2008-04-23 18:07:09 UTC
It wasn't fixed at all, but I managed to understand genone's thoughts on this issue, and he figured he had to take the thing back to the drawing board.

That said, I recently bumped on some horrible "internal version dependencies" in ELF objects (other thing than soname and versions there), and I think this makes the entire scene for ELF systems even worse/harder.
Comment 11 DEMAINE Benoît-Pierre, aka DoubleHP 2009-07-04 16:32:12 UTC
does it still happen ? we are now at r33 ... if no, please close.
Comment 12 Fabian Groffen gentoo-dev 2011-02-06 15:29:49 UTC
I just tried this on acroread, and it doesn't occur any more.  I recall something was changed to here, so it may be very well this bug has been fixed.
Comment 13 Zac Medico gentoo-dev 2011-05-18 16:37:46 UTC
This should be fixed in 2.2.0_alpha34 (among many other preserve-libs fixes, including bug 286714). Please re-open if you can still reproduce it.
Comment 14 Christoph Junghans (RETIRED) gentoo-dev 2011-05-18 17:56:38 UTC
I have a similar problem with google-talkplugin:

!!! existing preserved libs:
>>> package: media-libs/libpng-1.4.5
 *  - /usr/lib/libpng12.so.0
 *      used by /opt/google/talkplugin/libnpgtpo3dautoplugin.so (www-plugins/google-talkplugin-2.0.6.0)
Use emerge @preserved-rebuild to rebuild packages using these libraries

google-talkplugin is a binary package and has a hard dependency on libpng:1.2.
Comment 15 Zac Medico gentoo-dev 2011-05-18 21:53:24 UTC
(In reply to comment #14)
Thanks for confirming. I was thinking of the second issue when I closed this.

(In reply to comment #0)
> The second issue:
> 
> >>> package: gnome-base/gnome-applets-2.22.0
>  *  - /usr/lib64/libgweather.so.0
>  *  - /usr/lib64/libgweather.so.0.0.0
> 
> The above files used to be owned by gnome-base/gnome-applets-2.20 but then
> gnome-applets-2.22 broke it out into it's own package called
> dev-libs/libgweather. The .so versions stayed the same, the library just
> migrated from 1 package to another. No matter how many times you emerge
> @preserved-libs, it won't get rid of that msg.

Hopefully this second issue got fixed in 2.2.0_alpha34.
Comment 16 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-09-14 15:30:05 UTC
*** Bug 382921 has been marked as a duplicate of this bug. ***
Comment 17 Christoph Junghans (RETIRED) gentoo-dev 2011-11-29 05:23:37 UTC
(In reply to comment #14)
> I have a similar problem with google-talkplugin:
> 
> !!! existing preserved libs:
> >>> package: media-libs/libpng-1.4.5
>  *  - /usr/lib/libpng12.so.0
>  *      used by /opt/google/talkplugin/libnpgtpo3dautoplugin.so
> (www-plugins/google-talkplugin-2.0.6.0)
> Use emerge @preserved-rebuild to rebuild packages using these libraries
> 
> google-talkplugin is a binary package and has a hard dependency on libpng:1.2.

It's back:

!!! existing preserved libs:
>>> package: media-libs/libpng-1.5.5
 *  - /usr/lib/libpng12.so.0
 *      used by /opt/google/talkplugin/libnpgtpo3dautoplugin.so (www-plugins/google-talkplugin-2.5.6.0)
>>> package: media-libs/libpng-1.2.46
 *  - /usr/lib/libpng12.so.0
 *      used by /opt/google/talkplugin/libnpgtpo3dautoplugin.so (www-plugins/google-talkplugin-2.5.6.0)
Comment 18 Fabian Groffen gentoo-dev 2011-11-29 07:28:49 UTC
just for clarity, what do

scanelf --needed /opt/google/talkplugin/libnpgtpo3dautoplugin.so
scanelf --rpath /opt/google/talkplugin/libnpgtpo3dautoplugin.so

return?
Comment 19 Christoph Junghans (RETIRED) gentoo-dev 2011-11-30 02:08:17 UTC
(In reply to comment #18)
> just for clarity, what do
> 
> scanelf --needed /opt/google/talkplugin/libnpgtpo3dautoplugin.so
> scanelf --rpath /opt/google/talkplugin/libnpgtpo3dautoplugin.so
> 
> return?
$ scanelf --needed /opt/google/talkplugin/libnpgtpo3dautoplugin.so
 TYPE   NEEDED FILE ET_DYN libGL.so.1,libpthread.so.0,libCgGL.so,libdl.so.2,librt.so.1,libXt.so.6,libCg.so,libX11.so.6,libcairo.so.2,libpng12.so.0,libgtk-x11-2.0.so.0,libgdk-x11-2.0.so.0,libm.so.6,libfreetype.so.6,libfontconfig.so.1,libgobject-2.0.so.0,libglib-2.0.so.0,libstdc++.so.6,libgcc_s.so.1,libc.so.6 /opt/google/talkplugin/libnpgtpo3dautoplugin.so 
$ scanelf --rpath /opt/google/talkplugin/libnpgtpo3dautoplugin.so
 TYPE   RPATH FILE ET_DYN /opt/google/talkplugin/lib /opt/google/talkplugin/libnpgtpo3dautoplugin.so
Comment 20 Fabian Groffen gentoo-dev 2011-11-30 08:26:19 UTC
It preserves libpng12.so because either it doesn't use the rpath information from the object that references it, or it doesn't use it correctly.
Comment 21 Fabian Groffen gentoo-dev 2011-11-30 08:35:23 UTC
I think LinkageMapELF should follow the MachO approach from LinkageMapMachO to use absolute paths to the libraries, instead of relying on the search path.  Because only libpng12.so is stored as needed entry, and not /opt/google/talkplugin/lib/libpng12.so, the system has a problem next time it reevaluates the "preserved-libs", simply because it doesn't have the rpath information from the original consumer any more (and has to rely on the general search path).

Could do this in NEEDED.ELF.3 (to store the full resolved paths to the libs) and updates to the LinkageMapELF code (basically to match the LinkageMapMachO one).
Comment 22 Zac Medico gentoo-dev 2011-11-30 16:22:48 UTC
(In reply to comment #21) 
> Because only libpng12.so is stored as needed entry, and not
> /opt/google/talkplugin/lib/libpng12.so, the system has a problem next time it
> reevaluates the "preserved-libs", simply because it doesn't have the rpath
> information from the original consumer any more (and has to rely on the general
> search path).

Isn't the the "rpath information from the original consumer" stored in /var/db/pkg/www-plugins/google-talkplugin-2.5.6.0/NEEDED.ELF.2 though? Thought that LinkageMapELF would already account for that when calculating the consumers of libpng12.so. If it's not working for some reason, shouldn't the data from NEEDED.ELF.2 be sufficient to fix it?
Comment 23 Fabian Groffen gentoo-dev 2011-11-30 16:28:49 UTC
Reading from your comment, I think it should indeed be the case to deduce it.  I don't know what I was thinking.

Still, I think it simplifies when the registry wouldn't recompute the object's path all the time (and it does wrongly, obvious from this bug).
Comment 24 Zac Medico gentoo-dev 2011-11-30 18:34:18 UTC
(In reply to comment #23)
> Still, I think it simplifies when the registry wouldn't recompute the object's
> path all the time (and it does wrongly, obvious from this bug).

Actually, I think it works correctly. My local install of www-plugins/google-talkplugin-2.5.6.0 provides the following files:

/usr/share/doc/google-talkplugin-2.5.6.0/changelog.Debian.bz2
/usr/lib/nsbrowser/plugins/libnpgoogletalk.so
/usr/lib/nsbrowser/plugins/libnpgtpo3dautoplugin.so
/opt/google/talkplugin/GoogleTalkPlugin
/opt/google/talkplugin/libnpgoogletalk.so
/opt/google/talkplugin/libnpgtpo3dautoplugin.so

So, the package actually doesn't install a libpng12.so.0 in the /opt/google/talkplugin/lib rpath that we're talking about. Therefore, the package has a missing dependency on media-libs/libpng:1.2 and having media-libs/libpng:1.2 installed is supposed to prevent the library preservation shown in comment #14.

So, I think that the real cause of the problem shown in comment #14 is due to some interaction between media-libs/libpng-1.5.5 and media-libs/libpng-1.2.x ebuilds, which causes the libpng12.so.0 to be owned by media-libs/libpng-1.5.5 when it should really be owned by media-libs/libpng-1.2.x. Maybe it has something to do with the preserve_old_lib call in the libpng-1.5.5 ebuild...
Comment 25 Zac Medico gentoo-dev 2011-11-30 18:41:35 UTC
(In reply to comment #24)
> Maybe it has something to do with the preserve_old_lib call in the libpng-1.5.5
> ebuild...

Well, that preserve_old_lib call is for libpng14.so.0, so apparently that's not the issue. So, it seems that the problem is instead related to this slotmove:

  $PORTDIR/profiles/updates/2Q-2010:slotmove <=media-libs/libpng-1.2.43 1.2 0

If the user has media-libs/libpng-1.2.x installed in SLOT 0, then it triggers preserve-libs when upgrading to newer media-libs/libpng-1.5.5 which is also in SLOT 0. So, we need to find out why that library is preserved, and why having media-libs/libpng:1.2 installed doesn't resolve it.
Comment 26 Christoph Junghans (RETIRED) gentoo-dev 2011-11-30 20:51:10 UTC
(In reply to comment #25)
> (In reply to comment #24)
> > Maybe it has something to do with the preserve_old_lib call in the libpng-1.5.5
> > ebuild...
> Well, that preserve_old_lib call is for libpng14.so.0, so apparently that's not
> the issue. So, it seems that the problem is instead related to this slotmove:
> 
>   $PORTDIR/profiles/updates/2Q-2010:slotmove <=media-libs/libpng-1.2.43 1.2 0
> 
> If the user has media-libs/libpng-1.2.x installed in SLOT 0, then it triggers
> preserve-libs when upgrading to newer media-libs/libpng-1.5.5 which is also in
> SLOT 0. So, we need to find out why that library is preserved, and why having
> media-libs/libpng:1.2 installed doesn't resolve it.
It must have something to do with slot move. On another machine everything is fine. google-talkplugin pulls in libpng:1.2 as deps and there is no existing preserved lib. In contrast, this machine hasn't been updated for 3 months and the problem came up after updating to libpng-1.5.
Comment 27 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-04-29 07:20:08 UTC
*** Bug 265372 has been marked as a duplicate of this bug. ***
Comment 28 Sebastian Luther (few) 2013-12-02 18:01:50 UTC
*** Bug 493018 has been marked as a duplicate of this bug. ***
Comment 29 Andreas K. Hüttel archtester gentoo-dev 2013-12-10 19:37:17 UTC
*** Bug 493848 has been marked as a duplicate of this bug. ***
Comment 30 Doug Goldstein (RETIRED) gentoo-dev 2016-02-01 02:07:33 UTC
So I think after reading through this we can confirm this is fixed correct Portage team?
Comment 31 Fabio Rossi 2016-03-17 16:34:50 UTC
There are binary packages that are shipped with bundled libs, reference a needed library and then at runtime they can decide to use the bundled lib or the system one.

For example, vmware-workstation

$ scanelf --needed /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so
 TYPE   NEEDED FILE 
ET_DYN libgksu2.so.0,libutil.so.1,libc.so.6 /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so 

$ scanelf  --rpath /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so 
 TYPE   RPATH FILE 
ET_DYN   -   /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so 

The libgksu2.so.0 is usually bundled as /opt/vmware/lib/vmware/lib/libgksu2.so.0/libgksu2.so.0 and it's usually loaded before the system lib. When portage builds the list of libs related to the preserved-libs feature, it doesn't know about the bundled lib and for this reason it maps /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so with /usr/lib64/libgksu2.so.0 even if vmware effectively uses the bundled lib in /opt/vmware/lib/vmware/lib/libgksu2.so.0/libgksu2.so.0. This means that when x11-libs/libgksu was already installed in the system but not listed as dep for vmware-workstation, the preserved-libs mechanism in anyway triggered:

!!! existing preserved libs:
>>> package: x11-libs/libgksu-2.0.12-r2
 *  - /usr/lib64/libgksu2.so.0
 *  - /usr/lib64/libgksu2.so.0.0.2
 *      used by /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so (app-emulation/vmware-workstation-12.1.0.3272444-r1)
Comment 32 Zac Medico gentoo-dev 2016-03-17 17:09:20 UTC
(In reply to Fabio Rossi from comment #31)
> The libgksu2.so.0 is usually bundled as
> /opt/vmware/lib/vmware/lib/libgksu2.so.0/libgksu2.so.0 and it's usually
> loaded before the system lib. When portage builds the list of libs related
> to the preserved-libs feature, it doesn't know about the bundled lib and for
> this reason it maps
> /opt/vmware/lib/vmware/lib/libvmware-gksu.so/libvmware-gksu.so with
> /usr/lib64/libgksu2.so.0 even if vmware effectively uses the bundled lib in
> /opt/vmware/lib/vmware/lib/libgksu2.so.0/libgksu2.so.0.

The ebuild should use patchelf --set-rpath to properly configure library resolution. For example, see bug 265372, comment #16. This is the proper way to communicate the necessary information to both the linker and portage.
Comment 33 Fabio Rossi 2016-03-17 18:11:26 UTC
(In reply to Zac Medico from comment #32)

> The ebuild should use patchelf --set-rpath to properly configure library
> resolution. For example, see bug 265372, comment #16. This is the proper way
> to communicate the necessary information to both the linker and portage.

I'll try doing some experiments, the problem is that the bundled libs are not installed in a single folder but in a tree structure like this:

/opt/vmware/lib/vmware/lib/
|
+--- libsoname1.so.0/libsoname1.so.0
|
+--- libsoname2.so.0/libsoname2.so.0
|
\--- libsoname3.so.0/libsoname3.so.0

so a single call to patchelf --set-rpath should not work. I must see if vmware is using dlopen() at runtime, in that case probably --remove-needed is a better solution for bundled libs because they are not shared with other packages
Comment 34 Zac Medico gentoo-dev 2016-03-17 19:04:41 UTC
(In reply to Fabio Rossi from comment #33)
> (In reply to Zac Medico from comment #32)
> 
> > The ebuild should use patchelf --set-rpath to properly configure library
> > resolution. For example, see bug 265372, comment #16. This is the proper way
> > to communicate the necessary information to both the linker and portage.
> 
> I'll try doing some experiments, the problem is that the bundled libs are
> not installed in a single folder but in a tree structure like this:
> 
> /opt/vmware/lib/vmware/lib/
> |
> +--- libsoname1.so.0/libsoname1.so.0
> |
> +--- libsoname2.so.0/libsoname2.so.0
> |
> \--- libsoname3.so.0/libsoname3.so.0
> 
> so a single call to patchelf --set-rpath should not work.

You should be able to add multiple paths separated by colons. These paths can be relative to $ORIGIN, like $ORIGIN/libsoname1.so.0:$ORIGIN/libsoname2.so.0 and so on.

> I must see if
> vmware is using dlopen() at runtime, in that case probably --remove-needed
> is a better solution for bundled libs because they are not shared with other
> packages

Yes, could be.
Comment 35 Alex Xu (Hello71) 2016-05-22 14:49:47 UTC
*** Bug 583208 has been marked as a duplicate of this bug. ***