Created attachment 708864 [details] emerge --info Hi! During update of somewhat old setup I have a state when @system, kernel and portage are updated to the recent state with glibc-2.23-r8 (see emerge --info below for details) but @world is still not. At this step I have quite a lengthy preserved libs list: $ wc -l /var/lib/portage/preserved_libs_registry 705 /var/lib/portage/preserved_libs_registry But after update to glibc-2.33 it is killed together with all preserved libs: $ cat /var/lib/portage/preserved_libs_registry {} Not just the list, but all preserved libs are removed as well. This leads to unusable system and only full backup saved me. Other users may be not so lucky. I'm not sure whether this is a portage or a glibc bug, so I'm CC'ing both teams. # emerge -av1 glibc These are the packages that would be merged, in order: Calculating dependencies ... done! [ebuild U ] sys-libs/glibc-2.33:2.2::gentoo [2.32-r8:2.2::gentoo] USE="caps (crypt) custom-cflags doc gd multiarch ssp (static-libs) -audit (-cet) -compil e-locales -headers-only (-multilib) -multilib-bootstrap% -nscd -profile (-selinux) -static-pie -suid -systemtap -test (-vanilla)" 0 KiB Build logs of glibc-2.33 build and 2.32-r8 removal are below. Looks like preserved libs are killed during glibc-2.32-r8 cleanup.
Created attachment 708867 [details] glibc-2.33.log.gz
Created attachment 708870 [details] glibc-2.32-r8-cleanup.log.gz
(In reply to Andrew Savchenko from comment #0) > During update of somewhat old setup I have a state when @system, kernel and > portage are updated to the recent state with glibc-2.23-r8 glibc-2.32-r8 (typo fixed)
This happens after any glibc < 2.33 update to 2.33. I updated @system to the latest working 2.32-r8 to rule out possible bug reason due to update from too old glibc version, but problem is still reproducible with 2.32-r8 -> 2.33 update.
Uninstall log is not very informative: """ * Adjusting permissions for FEATURES=userpriv: '/var/cache/ccache' <<< !needed sym /lib/libhistory.so.6 <<< !needed obj /lib/libhistory.so.6.3 <<< !needed sym /lib/libncurses.so.5 <<< !needed obj /lib/libncurses.so.5.9 <<< !needed sym /lib/libncursesw.so.5 <<< !needed obj /lib/libncursesw.so.5.9 <<< !needed sym /lib/libreadline.so.6 <<< !needed obj /lib/libreadline.so.6.3 <<< !needed sym /lib/libreadline.so.7 <<< !needed obj /lib/libreadline.so.7.0 <<< !needed sym /usr/lib/libGLEW.so.1.10 <<< !needed obj /usr/lib/libGLEW.so.1.10.0 <<< !needed sym /usr/lib/libGraphicsMagick++.so.3 <<< !needed obj /usr/lib/libGraphicsMagick++.so.3.6.2 ... """ You'll probably need to add debugging statements into portage around the process to get the idea why emerge decided they are not referenced anymore.
bug #768897 sounds vaguely related. I think we need more debugging on emerge side.
(In reply to Sergei Trofimovich from comment #5) > You'll probably need to add debugging statements into portage around the > process to get the idea why emerge decided they are not referenced anymore. Any idea how to do this? Is there some special debug option or emerge code needs to be changes? For now I'll try emerge --debug.
Created attachment 709068 [details] glibc-2.32-r8-uninstall.log.gz Uninstall log with `emerge --debug`, though looks like it still lacks necessary information about why preserved_libs are being pruned.
This is likely to be triggered by an interaction between portage and the /etc/ld.so.conf configuration, since portage will not recognize that these libraries are needed unless they appear to satisfy dependencies based on the ld.so.conf configuration. Portage uses this getlibpaths function to read ld.so.conf: https://github.com/gentoo/portage/blob/portage-3.0.18/lib/portage/util/__init__.py#L1786 The getlibpaths function call for preserve-libs is here: https://github.com/gentoo/portage/blob/portage-3.0.18/lib/portage/util/_dyn_libs/LinkageMapELF.py#L223
Comparison of ld.so.conf state before and after the upgrade will likely provide a clue.
(In reply to Zac Medico from comment #10) > Comparison of ld.so.conf state before and after the upgrade will likely > provide a clue. Both /etc/ld.so.conf and /etc/ld.so.conf.d are the same before and after the upgrade: # diff -Naurds ld.so.conf.d-2.32-r8/ ld.so.conf.d/ Files ld.so.conf.d-2.32-r8/05gcc-i686-pc-linux-gnu.conf and ld.so.conf.d/05gcc-i686-pc-linux-gnu.conf are identica # diff -Naurds ld.so.conf-2.32-r8 ld.so.conf Files ld.so.conf-2.32-r8 and ld.so.conf are identical
(In reply to Andrew Savchenko from comment #11) > (In reply to Zac Medico from comment #10) > > Comparison of ld.so.conf state before and after the upgrade will likely > > provide a clue. > > Both /etc/ld.so.conf and /etc/ld.so.conf.d are the same before and after the > upgrade: > > # diff -Naurds ld.so.conf.d-2.32-r8/ ld.so.conf.d/ > Files ld.so.conf.d-2.32-r8/05gcc-i686-pc-linux-gnu.conf and > ld.so.conf.d/05gcc-i686-pc-linux-gnu.conf are identica > # diff -Naurds ld.so.conf-2.32-r8 ld.so.conf > Files ld.so.conf-2.32-r8 and ld.so.conf are identical Thanks, so we can rule out ld.so.conf as the trigger. I wonder if every glibc upgrade would behave this way, but people simply don't notice because it's not typical to have preserved libraries *during* glibc upgrades. I suspect that something about the LinkageMapELF rebuild method's include_file and exclude_pkgs parameters may trigger it, when these parameters refer to data from the glibc package. For reference, the parameters were added in these commits: https://gitweb.gentoo.org/proj/portage.git/commit/?id=1b2c09ae90370fb766afe527d03a641703e9bce1 commit 1b2c09ae90370fb766afe527d03a641703e9bce1 Author: Marius Mauch <genone@gentoo.org> Date: 2008-05-05 08:58:05 +0000 fix preserve_libs logic to properly account for the current package instance svn path=/main/trunk/; revision=10200 pym/portage/dbapi/vartree.py | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=035b0f03170765bdfe8ba0b93aabe3621970d49c commit 035b0f03170765bdfe8ba0b93aabe3621970d49c Author: Zac Medico <zmedico@gentoo.org> Date: 2008-10-29 00:07:34 +0000 Fix interaction between LinkageMap.rebuild() and the package replacement process in order to avoid problems with stale or unaccounted NEEDED. This solves a LinkageMap corruption issue which caused findConsumers to return false positive inside dblink.unmerge(). svn path=/main/trunk/; revision=11742 pym/portage/dbapi/vartree.py | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-)
(In reply to Zac Medico from comment #12) > Thanks, so we can rule out ld.so.conf as the trigger. I wonder if every > glibc upgrade would behave this way, but people simply don't notice because > it's not typical to have preserved libraries *during* glibc upgrades. Definitely not every glibc update is affected, because I successfully updated glibc-2.28-r5 -> 2.32-r8 on the very same system. Originally I was making 2.28-r5 -> 2.33 update, but found that system became broken and started to bisect; so I found that 2.32-r8 -> 2.33 is sufficient to reproduce this.
Just for the record: while main system is busy with @world update, I still have a recorded image to reproduce the problem. While I can't share it because it likely contains private information and it is hard to clean it up completely, I can do some debugging if someone will tell me exactly what and how to debug. I'm fluent with gdb, but it will be of little help here. Maybe pdb will, but I have problems with it (right now I'm trying to debug numpy[lapack] build problem within distutils code and I'm not successful).
