Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 646090 - sys-apps/portage-2.3.20 does not completely preserve old libs [Prefix]
Summary: sys-apps/portage-2.3.20 does not completely preserve old libs [Prefix]
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: preserve-libs 691278
  Show dependency tree
 
Reported: 2018-01-29 23:48 UTC by Steven Trogdon
Modified: 2020-05-13 00:31 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
prefix-preserve-libs.patch (prefix-preserve-libs.patch,942 bytes, patch)
2019-08-10 02:39 UTC, Benda Xu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Steven Trogdon 2018-01-29 23:48:41 UTC
Not preserving old libs is relative to the above portage and installed dev-libs/mpfr-3.1.6 which installs libmpfr.so.4.1.6.

Profile:
Available profile symlink targets:
  [1]   prefix/linux/amd64
  [2]   prefix/linux-standalone/amd64
  [3]   prefix/linux-standalone/amd64/legacy
  [4]   default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ *
  [5]   default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+
  [6]   default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+

with
gcc version 7.2.0 (Gentoo 7.2.0-r1 p1.1)

A recent emerge of the latest sage-9999 from the sage-on-gentoo overlay requires

emerge -1 sage
Calculating dependencies... done!
[ebuild     U #] dev-libs/mpfr-4.0.0-r1 [3.1.6]
[ebuild  N    #] sci-mathematics/sage-9999  USE="debug doc-html doc-pdf sagenb
testsuite -X -bliss -latex -libbraiding -libhomfly -modular_decomposition"
L10N="en -ca -de -es -fr -hu -it -ja -pt -ru -tr" PYTHON_TARGETS="python2_7
-python3_6"

Presently, mpfr-4.0.0-r1 is package masked.

---
# Matthias Maier <tamiko@gentoo.org> (26 Dec 2017)
# gcc depends on mpfr and this version changes soname. Spare users with
# FEATURES=-preserve-libs from completely frying their system.
=dev-libs/mpfr-4.0.0-r1
---

I do not have 'FEATURES=-preserve-libs' set in make.conf. After unmasking mpfr-4.0.0-r1 the emerge of sage gives

emerge -1p sage

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  r  U #] dev-libs/mpfr-4.0.0-r1 [3.1.6]
[ebuild  rR    ] dev-libs/mpc-1.1.0-r1 
[ebuild  rR    ] sci-libs/mpfi-1.5.2 
[ebuild  rR    ] sci-mathematics/flint-2.5.2-r1 
[ebuild  rR    ] sci-mathematics/arb-2.12.0 
[ebuild  rR    ] sci-libs/linbox-1.5.2 
[ebuild  rR    ] sci-mathematics/giac-1.4.9.45 
[ebuild  rR    ] app-text/texlive-core-2017-r3 
[ebuild  N    #] sci-mathematics/sage-9999  USE="debug doc-html doc-pdf sagenb testsuite -X -bliss -latex -libbraiding -libhomfly -modular_decomposition" L10N="en -ca -de -es -fr -hu -it -ja -pt -ru -tr" PYTHON_TARGETS="python2_7 -python3_6"

After proceeding with the emerge @preserved-rebuild required one rebuild of
sci-libs/fplll which oddly completed. sage-9999 was not actually installed due to a patching error. Everything else was rebuilt during the emerge step.

eix -I fplll
[I] sci-libs/fplll
     Available versions:  4.0.4 5.0.3(0/2.0)[1] 5.1.0-r1(0/3.0)[1] 5.2.0(0/4.0)[1] {static-libs}
     Installed versions:  5.2.0(0/4.0)[1](05:55:43 PM 01/28/2018)(-static-libs)
     Homepage:            https://github.com/fplll/fplll
     Description:         Different implementations of the floating-point LLL reduction algorithm

[1] "sage-on-gentoo" /storage/strogdon/gentoo-rap/var/lib/layman/sage-on-gentoo

Upon completing the @preserved-rebuild request I discovered that my gcc was broken. See https://github.com/cschwan/sage-on-gentoo/issues/497 for more elaborate documentation.

To recover I linked to the host (debian) libmpfr (libmpfr.so.4.1.0) and rebuilt the Prefix libmpfr.so.4.1.6 using dev-libs/mpfr-3.1.6. So there was a required rebuild but the old mpfr libs were not preserved which broke gcc. I didn't check before the rebuild of fplll as to whether the old libs were present, but since the rebuild completed I suspect they were. Which may suggest that @preserved-rebuild removed them.
Comment 1 François Bissey 2018-01-30 00:09:21 UTC
There are further issues with this particular upgrade but the main point is that gcc broke because libmpfr.so.3{,.xx} has been removed when it should have been preserved.

My understanding is that by default libraries are preserved in FEATURES so this shouldn't have happened.
Comment 2 Zac Medico gentoo-dev 2018-01-30 00:37:35 UTC
Possibly a fault of the mpfr-4.0.0-r1 ebuild rather than portage, see bug 642316.
Comment 3 Zac Medico gentoo-dev 2018-01-30 00:38:52 UTC
Check if the ebuild that you used had this commit:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02da1bfffe30537bf7fe7680fc628acc9d6ec289
Comment 4 Steven Trogdon 2018-01-30 00:50:10 UTC
(In reply to Zac Medico from comment #3)
> Check if the ebuild that you used had this commit:
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/
> ?id=02da1bfffe30537bf7fe7680fc628acc9d6ec289

Yes, the ebuild I used had the commit.
Comment 5 François Bissey 2018-01-30 00:51:17 UTC
(In reply to Zac Medico from comment #3)
> Check if the ebuild that you used had this commit:
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/
> ?id=02da1bfffe30537bf7fe7680fc628acc9d6ec289

It should, he specified mpfr-4.0.0-r1. And that commit goes from 4.0.0 to 4.0.0-r1. 

My last comment on #642316 was for a different issue where some code will trigger ICE if you upgrade mpfr then mpc but not gcc immediately after that.
Comment 6 Steven Trogdon 2018-02-01 00:04:40 UTC
OK, 2nd attempt. I unmasked mpfr-4.0.0-r1 and installed it. Old mpfr libs are preserved

ls /storage/strogdon/gentoo-rap/usr/lib/libmpfr*

/storage/strogdon/gentoo-rap/usr/lib/libmpfr.so
/storage/strogdon/gentoo-rap/usr/lib/libmpfr.so.4
/storage/strogdon/gentoo-rap/usr/lib/libmpfr.so.4.1.6
/storage/strogdon/gentoo-rap/usr/lib/libmpfr.so.6
/storage/strogdon/gentoo-rap/usr/lib/libmpfr.so.6.0.0

However, gcc is not in my list of rebuilds

emerge -p @preserved-rebuild

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sci-libs/fplll-5.2.0

I'm now sure if I simply run 

emerge @preserved-rebuild

the old mpfr libs will be deleted and my gcc will be broken. Instead, at this point, I rebuilt gcc and then performed the @preserved-rebuild. This then required a manual emerge of dev-python/fpylll because of 'stale' links to the old mpfr libs. But my gcc seems fine.
Comment 7 Benda Xu gentoo-dev 2019-08-10 02:39:55 UTC
Created attachment 586400 [details, diff]
prefix-preserve-libs.patch

Hi Zac,

I have been bitten by the same bug, and finally found the root cause is that portage looked into ROOT for consumers of a shared library, which returns nothing on Prefix.  Here is a patch that fixes this long standing bug.

Benda
Comment 8 Zac Medico gentoo-dev 2019-08-10 20:26:26 UTC
(In reply to Benda Xu from comment #7)
> Created attachment 586400 [details, diff] [details, diff]
> prefix-preserve-libs.patch

Looks good, except it should use self._dbapi.settings['EROOT'], since portage.const.EPREFIX is not necessarily correct for a cross-prefix configuration which has two independent EPREFIX values.
Comment 9 Benda Xu gentoo-dev 2019-08-11 02:35:24 UTC
(In reply to Zac Medico from comment #8)
> (In reply to Benda Xu from comment #7)
> > Created attachment 586400 [details, diff] [details, diff] [details, diff]
> > prefix-preserve-libs.patch
> 
> Looks good, except it should use self._dbapi.settings['EROOT'], since
> portage.const.EPREFIX is not necessarily correct for a cross-prefix
> configuration which has two independent EPREFIX values.

Agreed.  I have overlooked it, sorry.

Benda
Comment 10 Larry the Git Cow gentoo-dev 2019-08-11 18:09:47 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1120e6f62feb1aa0d3202beb00c01537d352f420

commit 1120e6f62feb1aa0d3202beb00c01537d352f420
Author:     Benda Xu <heroxbd@gentoo.org>
AuthorDate: 2019-08-10 02:26:12 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-08-11 18:07:43 +0000

    lib/p/util/_dyn_libs/LinkageMapELF.py: get dep graph from EROOT.
    
    On Prefix, the preserve-libs feature should search for shared
    libraries consumers from EROOT instead of ROOT.
    
    Bug: https://bugs.gentoo.org/646090
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/util/_dyn_libs/LinkageMapELF.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Comment 11 Benda Xu gentoo-dev 2019-08-12 03:45:53 UTC
Thanks, Zac!
Comment 12 Larry the Git Cow gentoo-dev 2019-08-19 05:06:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=08557524dc6c8eec3a366e43ab2587d2cdd8f133

commit 08557524dc6c8eec3a366e43ab2587d2cdd8f133
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-08-19 04:24:07 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-08-19 05:06:15 +0000

    sys-apps/portage: Bump to version 2.3.72
    
     #463952 glsa-check: install in /usr/bin
     #646090 preserve-libs: get dep graph from EROOT
     #690484 detect internal collisions for /usr merge
     #690786 repoman: support metadata/layout.conf restrict-allowed
     #691776 unpack: Unconditionally die if an unpacker returns an error
     #691638 Show get/setfattr stderr
     #692024 econf: Unconditionally die on error in EAPIs 0 to 3
     #692262 QA Notice: EXPORT_FUNCTIONS is called before inherit in
             kernel-2.eclass
     #692412 emerge IndexError for ambiguous package atom with pypy
    
    Bug: https://bugs.gentoo.org/691278
    Bug: https://bugs.gentoo.org/463952
    Bug: https://bugs.gentoo.org/646090
    Bug: https://bugs.gentoo.org/690484
    Bug: https://bugs.gentoo.org/690786
    Bug: https://bugs.gentoo.org/691776
    Bug: https://bugs.gentoo.org/691638
    Bug: https://bugs.gentoo.org/692024
    Bug: https://bugs.gentoo.org/692262
    Bug: https://bugs.gentoo.org/692412
    Package-Manager: Portage-2.3.71, Repoman-2.3.17
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.72.ebuild | 264 +++++++++++++++++++++++++++++++++
 2 files changed, 265 insertions(+)