Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 230257 - @preserved-rebuild contains packages that have already been rebuilt
Summary: @preserved-rebuild contains packages that have already been rebuilt
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal with 3 votes (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 231326 261178 303042 365093 562344 (view as bug list)
Depends on: 529022
Blocks: preserve-libs 373873
  Show dependency tree
 
Reported: 2008-06-30 20:44 UTC by Patrizio Bassi
Modified: 2017-01-18 22:50 UTC (History)
30 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
@preserved-rebuild list before rebuilding gtk (before_gtk_rebuild.txt,16.40 KB, text/plain)
2008-07-23 03:51 UTC, Zac Medico
Details
@preserved-rebuild list after rebuilding gtk (much smaller) (after_gtk_rebuild.txt,2.32 KB, text/plain)
2008-07-23 03:54 UTC, Zac Medico
Details
debug output from PreservedLibraryConsumerSet, before reinstalling pango and gtk+ (preserved_rebuild_debug.txt,83.87 KB, text/plain)
2008-07-23 03:59 UTC, Zac Medico
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Patrizio Bassi 2008-06-30 20:44:16 UTC
after fontconfig changes i started to rebuild some packages (~~100).

one package failed to build so i reun again the emerge @preserved-rebuild command and, surprise, it restarted again from scratch.

tried twice, same issue.

should be clever enough to know which package is really needed to be rebuild?
or it simply rebuild all dependency in a bunch?

how to exit this?
emerge info
*** Deprecated use of action 'info', use '--info' instead
WARNING: repository at /mnt/Dati/Gentoo/portage/ is missing a repo_name entry
WARNING: repository at /usr/local/portage/myportage is missing a repo_name entry
Portage 2.2_rc1 (default-linux/amd64/2007.0, gcc-4.3.1, glibc-2.8_p20080602-r0, 2.6.25.7 x86_64)
=================================================================
System uname: Linux-2.6.25.7-x86_64-Intel-R-_Core-TM-2_CPU_6300_@_1.86GHz-with-glibc2.2.5
Timestamp of tree: Mon, 30 Jun 2008 15:45:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [disabled]
app-shells/bash:     3.2_p39
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python:     2.5.2-r5
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.2.5
sys-apps/sandbox:    1.2.18.1-r3
sys-devel/autoconf:  2.13, 2.62-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1
sys-devel/binutils:  2.18-r2
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   2.2.4
virtual/os-headers:  2.6.25-r4
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -pipe -mtune=core2 -march=core2 -fomit-frame-pointer -mmmx -msse -msse2 -msse3 -mfpmath=sse"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /etc/mail/dspam /opt/openjms/config /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/mozilla/defaults/pref /usr/share/config /var/lib/hsqldb /var/run/dspam"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/compiler /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/splash /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O3 -pipe -mtune=core2 -march=core2 -fomit-frame-pointer -mmmx -msse -msse2 -msse3 -mfpmath=sse"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks fixpackages parallel-fetch preserve-libs sandbox sfperms strict unmerge-orphans"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="39"
LC_ALL="it_IT.utf8"
LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--enable-new-dtags -Wl,--as-needed -Wl,--hash-style=both"
LINGUAS="it"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/mnt/Dati/Gentoo/tmp/"
PORTDIR="/mnt/Dati/Gentoo/portage/"
PORTDIR_OVERLAY="/usr/local/portage/myportage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="64bit 7zip X X509 Xaw3d a52 aac aalib accessibility ace acl acpi activefilter addbookmarks adns aiglx aio alias alsa amd amd64 amr amrr amuled ao aoss apache2 arj asf aspnet2 async atk audacious audiofile authfile automount autoreplace bash-completion berkdb bidi binary-drivers bittorrent blas bootsplash browserplugin bzip2 cairo calendar caps cdda cddb cdio cdparanoia cdr cgi charconv chardet chroot ciao clamav clearcase cli client codecs connectionstatus contactnotes corba cracklib crypt cscope css ctype cups curl cvs daemon dar64 dba dbus depth32 dga dhcp dio directfb diskio divx djbfft dmi dri dvd dvdr dvdread edl encode ethereal exif expat extraengine faac faad fam fastbuild ffmpeg fftw flac flash foreign-package fortran ftp gcj gd gdbm geometry ggi gif gimp gimpprint glibc-omitfp glitz glut glx gnokii gnome gnutls gphoto2 gpm graphviz gs gtk gtk2 gtkhtml h264 hal hddtemp highlight history iconv idea idn imagemagick imap innodb ipod ipv6 irc irda irmc isdnlog jack java java5 javamail javascript jikes jit joystick jpeg jpeg2k junit justify kde kdeenablefinal kdehiddenvisibility kerberos kqemu lame lapack lcd lcms ldap libcaca libclamav libgda libusb lirc live lm_sensors lua lzo mad mailwrapper matroska md5sum mdb mhash midi mikmod mime mmx mmx2 mmxext mng mono mozcalendar mozdevelop mozilla mozsvg mozxmlterm mp3 mp4 mp4live mpeg mpeg2 mpeg4 mplayer mpm-prefork msn msnextras mudflap musepack mysql mysqli mythtv nagios-dns nagios-ntp nagios-ping nagios-ssh ncurses neXt network networking ngui nls nntp no_wxgtk1 nobsh nopie nossp noxalan nptl nptlonly nsplugin nv nvidia oav odbc ogg openal opengl openmp openssl operanom2 pam pcre pda pdf pdo pear perl php pmu png pnp postgres pppd pthreads pulseaudio python qemu-fast qhull qt3 qt3support qt4 quicktime quotas readline reflection rrdtool rtc ruby samba sametime scanner sdl server session sftplogging sguil slang slp smp snortsam soap sockets socks5 softmmu softquota spamassassin speex spell spl sqlite3 sse sse2 sse3 ssl ssse3 subject-rewrite svg tao tcltk tcpd tetex texteffect tga theora tiff tk tokenizer translator truetype trusted unicode usb userlocales v4l v4l2 vorbis webpresence winpopup wma wmf wmv wsconvert wxwindows x264 xanim xchatdccserver xchattext xcomposite xine xinerama xml xml2 xmlrpc xorg xosd xpm xprint xsl xulrunner xv xvid xvm xvmc zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="it" USERLAND="GNU" VIDEO_CARDS="fbdev vesa vga nv nvidia"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Doug Goldstein gentoo-dev 2008-06-30 21:57:17 UTC
I would just like to comment on your CFLAGS. Setting mtune to march's value is completely redundant. march is the base that your build will support, and mtune will tweak the code to run better at a more specific cpu. In your case, they're the same and pointless. -fomit-frame-pointer is automatically turned on for x86_64 for -O1 and higher. mmx, sse, and sse2 are all turned on by default with -march=core2. -mfpmath=sse is default for -march=core2. MAKEOPTS when using -j should bue num of CPUs/cores + 1. So logically, you should be using -j3. for LDFLAGS, the default in Gentoo is --hash-style=both, this results in slightly larger binaries. If you want an optimization --hash-style=gnu would be appropriate. Before you mess with CFLAGS, LDFLAGS, and MAKEOPTS, please read man gcc, man ld, and man make.
Comment 2 Patrizio Bassi 2008-06-30 22:02:34 UTC
totally agree. cflags are shared between several profiles and i change and tweak them.

LDFLAGS i had some segfaults and found that a bit more stable, i will try to switch back to gnu style.

thanks for pointing those however.

this config is not affecting the portage issue, however.
Comment 3 Zac Medico gentoo-dev 2008-07-01 02:30:49 UTC
(In reply to comment #0)
> after fontconfig changes i started to rebuild some packages (~~100).
> 
> one package failed to build so i reun again the emerge @preserved-rebuild
> command and, surprise, it restarted again from scratch.
> 
> tried twice, same issue.
> 
> should be clever enough to know which package is really needed to be rebuild?
> or it simply rebuild all dependency in a bunch?

It should be clever enough. One plausible explanation is that the package was still linked to the old version of the lib after being rebuild. Which package was it?
Comment 4 Patrizio Bassi 2008-07-01 07:57:52 UTC
ffmpeg for sure but i have 109 ebuilds, always same.

so it's not ffmpeg only.

This is first time i meet this error, generally it works.
Comment 5 Patrizio Bassi 2008-07-17 19:56:46 UTC
i still have this problem and now i have 2-3 library "pending"

no way? how to force the cleaning of these libraries so i can manuall do a revdep?
Comment 6 Martin Meyer 2008-07-20 16:49:49 UTC
Bug 231326 looks like a dupe of this one.

I'm having some problems with the reserved-rebuild system and I'd like to just spill them out at once to get it off my chest.

1) It's true that @preserved-rebuild does NOT keep track of which ebuilds from the set have been re-built. That would extremely nice, since one of the ones in one in my set is app-office/openoffice.

2) I've uninstalled some of the packages from that list and the packages that depended on them since the list was generated. They're gone, never to be wanted/depended on again. It still wants to install them.

3) I've run revdep-rebuild since this list was created, and the list won't go away. It probably should.

4) I've successfully completed 'emerge @preserved-rebuild' TWICE now and it's still telling my I've got to run it. Imagine the frustration of having OpenOffice rebuilt TWICE with a possibility of a third time on the horizon!

Where does that set definition live so  I can just delete it? I seem to have broken it somehow on my system.
Comment 7 Zac Medico gentoo-dev 2008-07-20 23:25:53 UTC
The @preserved-libs set is always generated on the fly from 2 things:

1) The registry located at /var/lib/portage/preserved_libs_registry, which tracks which libraries have been preserved and which package they've been injected into.

2) The /var/db/pkg/*/*/NEEDED.ELF.2 files which are used to determine which packages consume the preserved libraries.

Therefore, if the @preserved-libs set contains unwanted packages then the way to solve it is to somehow prevent the package from consuming a preserved library. This the package only has to be rebuilt, but if it still consumes the library after being rebuilt then we have to troubleshoot it on a case by case basis. Here are some things to look for:

* Why doesn't rebuilding the package cause it to stop consuming the preserved library?

* Should this library be preserved in the first place? For example, its been reported that libm-2.6.1.so is erroneously preserved during an upgrade to glibc-2.7.
Comment 8 Martin Meyer 2008-07-21 00:38:57 UTC
How can I generate a list of the libraries that were preserved?

Looking at the output of 'eix -i -c', the only likely suspect for the list I've got is sys-lib/db.  I think I'll just uninstall that and 'revdeo-rebuild'...
Comment 9 Martin Meyer 2008-07-21 00:50:38 UTC
Seems my problem was Mesa, and I didn't actually have two versions installed of it. /usr/lib64/libGLU.so.1.3.070003 and /usr/lib64/libGLU.so.1.3.070100 existed in the same package, and that's what was supposed to be eliminated somehow. I killed it finally by unmerging and re-merging mesa, but I'd like to understand how this problem came about. Any ideas?
Comment 10 Zac Medico gentoo-dev 2008-07-21 02:05:36 UTC
(In reply to comment #8)
> How can I generate a list of the libraries that were preserved?

This command will display the content of /var/lib/portage/preserved_libs_registry:

portageq list_preserved_libs /

(In reply to comment #9)
> killed it finally by unmerging and re-merging mesa, but I'd like to understand
> how this problem came about. Any ideas?

The older libGLU.so file, which seems to have been preserved, would have been automatically removed as soon as no other packages were consuming it. If it's gone now then it seems like you might have some broke links to it. Does revdep-rebuild report that any packages need to be rebuilt because the older libGLU.so file no longer exists?

Comment 11 Martin Meyer 2008-07-21 02:27:38 UTC
Nope, nothing on the system wants to rebuild. Keep in mind that I've already emerged the @preserved_libs set completely TWICE (actually some packages have been re-emerged 8 times now, due to one package that was failing along the way), so this seems reasonable. I don't know why the extra .so wasn't removed.
Comment 12 Zac Medico gentoo-dev 2008-07-23 03:41:58 UTC
*** Bug 231326 has been marked as a duplicate of this bug. ***
Comment 13 Zac Medico gentoo-dev 2008-07-23 03:51:27 UTC
Created attachment 161165 [details]
@preserved-rebuild list before rebuilding gtk

I was updating a chroot when I noticed this, and I was able to reproduce it when updating another root from binary packages built in the chroot. The list was over 100 packages before rebuilding gtk+, and it shrank drastically after rebuilding it.
Comment 14 Zac Medico gentoo-dev 2008-07-23 03:54:49 UTC
Created attachment 161166 [details]
@preserved-rebuild list after rebuilding gtk (much smaller)

The only thing that changed between these two @preserved-libs outputs was reinstalling x11-libs/gtk+-2.12.11.
Comment 15 Zac Medico gentoo-dev 2008-07-23 03:59:00 UTC
Created attachment 161167 [details]
debug output from PreservedLibraryConsumerSet, before reinstalling pango and gtk+
Comment 16 Zac Medico gentoo-dev 2008-07-23 04:06:20 UTC
Hmm, the behavior that others have reported here seems to be different from what my attachments show. Changing the summary back...
Comment 17 Tony Murray 2008-07-29 01:06:40 UTC
I would think it would be simple to quickly determine if a package has been rebuilt against the new library.  Compare the timestamp of the date the preserved library was built against the the timestamp of each program in the set was built.

Obviously, program build time < library = still needs rebuild, > = rebuilt.
Comment 18 Matteo Azzali (RETIRED) gentoo-dev 2008-08-17 12:34:42 UTC
What about packages consuming a symlink to a preserved library?
It looks like my issues with this feature are given by some packages pointing
to a symlink (NEEDED.ELF.2 contains just the symlink name and not the
library itself)
Comment 19 Zac Medico gentoo-dev 2008-08-17 18:14:04 UTC
(In reply to comment #18)
> What about packages consuming a symlink to a preserved library?
> It looks like my issues with this feature are given by some packages pointing
> to a symlink (NEEDED.ELF.2 contains just the symlink name and not the
> library itself)

If those preserved libs have the same soname as the new version then they shouldn't have been preserved. That could have been triggered by bug 230587. It that is the problem then you can solve it by reinstalling the packages that have the erroneously preserved libs. Alternatively, you can remove the erroneously preserved libs manually and run ldconfig to update the symlinks.
Comment 20 Zac Medico gentoo-dev 2008-08-17 18:16:49 UTC
(In reply to comment #17)
> I would think it would be simple to quickly determine if a package has been
> rebuilt against the new library.  Compare the timestamp of the date the
> preserved library was built against the the timestamp of each program in the
> set was built.
> 
> Obviously, program build time < library = still needs rebuild, > = rebuilt.

The timestamps are not a reliable indicator. Portage uses the /var/db/pkg/*/*/NEEDED.ELF.2 files which are generated with scanelf for this purpose.
Comment 21 Kobboi 2008-08-19 21:22:17 UTC
Same issue here I think:

!!! existing preserved libs:
>>> package: gnome-base/gail-1000
 *  - /usr/lib/libgailutil.so.18
 *  - /usr/lib/libgailutil.so.18.0.1
Use emerge @preserved-rebuild to rebuild packages using these libraries

After the rebuilding of 32 packages, and portage wants to rerebuild all 32 of them.
Comment 22 brent 2008-09-01 21:32:49 UTC
(In reply to comment #21)
> Same issue here I think:

i'm confirming as well. same thing happened in my quest for com_err+ss->e2fsprogs-libs.

after i finally managed to get e2fsprogs to rebuild via @preserved-rebuild, and on successful rebuilding of three other ebuilds, it still prompts for rebuild of all four.
Comment 23 Andrew Gaydenko 2008-09-29 00:46:11 UTC
Another the bug demonstration is below. After remerging the same messag appears again:

emerge -pv @preserved-rebuild

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] kde-base/ksysguard-3.5.10  USE="lm_sensors -debug -kdehiddenvisibility" 0 kB
[ebuild   R   ] kde-base/kdnssd-3.5.10  USE="-debug -kdehiddenvisibility" 0 kB
[ebuild   R   ] kde-base/krfb-3.5.10  USE="-debug -kdehiddenvisibility -slp" 0 kB
[ebuild   R   ] kde-base/krdc-3.5.10  USE="-debug -kdehiddenvisibility -rdesktop -slp" 0 kB
[ebuild   R   ] kde-base/kpf-3.5.10  USE="-debug -kdehiddenvisibility" 0 kB
[ebuild   R   ] kde-base/kcontrol-3.5.10  USE="ieee1394 opengl -arts -debug -joystick -kdehiddenvisibility -logitech-mouse" 0 kB

Total: 6 packages (6 reinstalls), Size of downloads: 0 kB

!!! existing preserved libs:
>>> package: kde-misc/kdnssd-avahi-0.1.2-r1
 *  - /usr/lib64/libkdnssd.so.1
 *      used by /usr/kde/3.5/bin/krdc (kde-base/krdc-3.5.10)
 *      used by /usr/kde/3.5/bin/ksysguard (kde-base/ksysguard-3.5.10)
 *      used by /usr/kde/3.5/lib64/kde3/kcm_kdnssd.so (kde-base/kcontrol-3.5.10)
 *      used by 5 other files
 *  - /usr/lib64/libkdnssd.so.1.0.0
 *      used by /usr/kde/3.5/bin/krdc (kde-base/krdc-3.5.10)
 *      used by /usr/kde/3.5/bin/ksysguard (kde-base/ksysguard-3.5.10)
 *      used by /usr/kde/3.5/lib64/kde3/kcm_kdnssd.so (kde-base/kcontrol-3.5.10)
 *      used by 5 other files
Use emerge @preserved-rebuild to rebuild packages using these libraries
Comment 24 Daniel Santos 2008-10-01 01:50:26 UTC
Workaround!!!

Nastiness!!  So run 'portageq list_preserved_libs /' and list all of the packages that have preserved libs and remerge them with FEATURES="-preserve-libs".  It can probably be scripted with something like this:

FEATURES="-preserve-libs" emerge -v $(portageq list_preserved_libs / | awk '{print "=" $1}')

This will, of course, break all of those packages that show up when you do 'emerge -pv @preserved-rebuild' so make sure none of those are important, like bash, vim, coreutils, python (needed for portage), sed, etc.
Comment 25 R Bar-On 2008-10-18 13:22:42 UTC
ffmpeg behaves similarly.  Output of portageq list_preserved_libs /:

media-video/ffmpeg-0.4.9_p20081014 /usr/lib64/libavcodec.so.51 /usr/lib64/libavcodec.so.51.53.0
media-libs/x264-0.0.20081006 /usr/lib64/libx264.so.56
Comment 26 Carsten Lohrke (RETIRED) gentoo-dev 2009-02-11 15:53:31 UTC
(In reply to comment #3)
> It should be clever enough. One plausible explanation is that the package was
> still linked to the old version of the lib after being rebuild. Which package
> was it?
> 

There is an error in the logic Portage preserves libraries. It does the check on ebuilds, which have been reinstalled already. Do "emerge foo bar", where foo requires bar to be rebuilt and Portage still asks for it, despite bar being rebuilt already. Portage version is 2.2_rc23.
Comment 27 Carsten Lohrke (RETIRED) gentoo-dev 2009-02-11 16:10:58 UTC
Ah, sorry - I got it backwards. bar is actually rebuilt against the old library, instead the new one, so we pointlessly have to rebuild it.
Comment 28 Steve Kutnar 2009-02-22 20:00:12 UTC
I'm running into this issue now with the new sys-libs/readline-6.0:

!!! existing preserved libs:
>>> package: sys-libs/readline-6.0
 *  - /lib64/libreadline.so
 *  - /lib64/libreadline.so.5
 *  - /lib64/libreadline.so.5.2
 *      used by /usr/bin/calgebra (kde-base/kalgebra-4.2.0)
Use emerge @preserved-rebuild to rebuild packages using these libraries

I've rebuilt readline and @preserved-rebuild, but this message doesn't go away...

# portageq list_preserved_libs /
sys-libs/readline-6.0 /lib64/libreadline.so /lib64/libreadline.so.5 /lib64/libreadline.so.5.2
Comment 29 Fabian Groffen gentoo-dev 2009-02-22 20:04:11 UTC
weird, I thought this was already fixed.  It should have never preserved /lib64/libreadline.so, since that one should have never been referenced by any other application/library.  Preserving it, causes the old lib never to go away.
Comment 30 Zac Medico gentoo-dev 2009-02-22 22:44:09 UTC
(In reply to comment #28)
> !!! existing preserved libs:
> >>> package: sys-libs/readline-6.0
>  *  - /lib64/libreadline.so
>  *  - /lib64/libreadline.so.5
>  *  - /lib64/libreadline.so.5.2
>  *      used by /usr/bin/calgebra (kde-base/kalgebra-4.2.0)

If rebuilding kde-base/kalgebra-4.2.0 does not help, then it's a problem in the build system for that ebuild (probably related to libtool). It's the same sort of issue as the one reported in bug 244636 for media-gfx/kphotoalbum-3.1.1-r1 linking against the old version of libkipi.so. As a workaround for a problem such as this, you should manually remove the preserved libs that trigger it. Note that @preserved-rebuild will no longer work after the libs are removed, so you have to remember which packages to rebuild or else use revdep-rebuild.
Comment 31 Steve Kutnar 2009-02-23 13:17:46 UTC
(In reply to comment #30)
> (In reply to comment #28)
> > !!! existing preserved libs:
> > >>> package: sys-libs/readline-6.0
> >  *  - /lib64/libreadline.so
> >  *  - /lib64/libreadline.so.5
> >  *  - /lib64/libreadline.so.5.2
> >  *      used by /usr/bin/calgebra (kde-base/kalgebra-4.2.0)
> 
> If rebuilding kde-base/kalgebra-4.2.0 does not help, then it's a problem in the
> build system for that ebuild (probably related to libtool). It's the same sort
> of issue as the one reported in bug 244636 for media-gfx/kphotoalbum-3.1.1-r1
> linking against the old version of libkipi.so. As a workaround for a problem
> such as this, you should manually remove the preserved libs that trigger it.
> Note that @preserved-rebuild will no longer work after the libs are removed, so
> you have to remember which packages to rebuild or else use revdep-rebuild.
> 

This was the case.  I reduced my KDE installation to just the meta pieces I actually "want".  In the process, kalgebra was removed and the 5.x readline libraries in the process.  It seems a little odd that the readline files got associated with kalgebra, but at any rate my situation is taken care of.
Comment 32 Zac Medico gentoo-dev 2009-03-05 23:24:55 UTC
*** Bug 261178 has been marked as a duplicate of this bug. ***
Comment 33 Vlastimil Babka (Caster) (RETIRED) gentoo-dev 2009-03-26 14:34:12 UTC
Could sandbox perhaps be used to hide/prevent access to the preserved libraries during rebuild to solve cases where build system picks the preserved version instead of new (as in the kalgebra case) ?
Comment 34 Zac Medico gentoo-dev 2009-03-26 16:38:04 UTC
(In reply to comment #33)
> Could sandbox perhaps be used to hide/prevent access to the preserved libraries
> during rebuild to solve cases where build system picks the preserved version
> instead of new (as in the kalgebra case) ?

That seems like an ugly workaround. It's a lot cleaner if the build system can simply recognize that a newer version of the library is available. Anyway, adding sandbox@ in case Mike wants to comment on this.
Comment 35 SpanKY gentoo-dev 2009-04-05 09:03:48 UTC
that wouldnt really be safe ... if, for example, the acl package changed SONAMEs, all of coreutils would be linked against the old one.  then you wouldnt be able to run any of the coreutils binaries in order to rebuild itself.
Comment 36 Robert Golding 2009-07-24 06:27:54 UTC
(In reply to comment #22)
> (In reply to comment #21)
> > Same issue here I think:
> 
> i'm confirming as well. same thing happened in my quest for
> com_err+ss->e2fsprogs-libs.
> 
> after i finally managed to get e2fsprogs to rebuild via @preserved-rebuild, and
> on successful rebuilding of three other ebuilds, it still prompts for rebuild
> of all four.
> 

Same for mine, started yesterday with e2fsprogs-libs with links to 703 files, @preserved-rebuild then lists 82 packages, with e2fsprogs about halfway down.

Have run the set three times, then re-merged e2fsprogs on its own and still the set is asking for the same 82 packages.
Comment 37 Robert Golding 2009-07-24 13:48:06 UTC
This is what keeps happening on my system..

--------->>
 * Regenerating GNU info directory index...
 * Processed 182 info files.

!!! existing preserved libs:
>>> package: sys-libs/e2fsprogs-libs-1.41.8
 *  - /lib/libblkid.so
 *      used by /bin/mount (sys-apps/util-linux-2.16)
 *      used by /bin/umount (sys-apps/util-linux-2.16)
 *      used by /sbin/blkid (sys-apps/util-linux-2.16)
 *      used by 11 other files
 *  - /lib/libuuid.so
 *      used by /bin/mount (sys-apps/util-linux-2.16)
 *      used by /bin/umount (sys-apps/util-linux-2.16)
 *      used by /sbin/blkid (sys-apps/util-linux-2.16)
 *      used by 704 other files
Use emerge @preserved-rebuild to rebuild packages using these libraries


one ~ # emerge -av @preserved-rebuild
//snip for brevity

Total: 82 packages (82 reinstalls), Size of downloads: 0 kB
Portage tree and overlays:
 [0] /usr/portage
 [1] /usr/portage/local/layman/desktop-effects

Would you like to merge these packages? [Yes/No]

--------------->>>

So, if I remove those two .so files and then do a revdep-rebuild, this will fix this?

BTW, I have run @preserved-rebuild 4 times now, always with the same packages, for some reason I thought that it might stop after awhile if I continued.
Comment 38 Fabian Groffen gentoo-dev 2009-07-24 13:51:59 UTC
The actual problem is the preservation of the .so files.  Because of that you can recompile what you want, but you'll always get linked to the preserved libs.  I vaguely recall a move of those libs, which may have caused this situation, but then Portage should probably always refuse or be loud of it needs to preserve an unversioned lib (SONAME is unversioned), because that's what's causing this.
Comment 39 N. Andrew Walsh 2009-07-25 21:11:50 UTC
I'm having the same issue. Some 114 packages remerge over and over, and never get fixed.

Thanks.
Comment 40 jcat 2009-07-26 09:34:34 UTC
I also have this issue with packages wanting to rebuild again e2fsprogs-libs
My preserved-rebuild list remains after rebuilding them all.


Cheers,
jcat
Comment 41 Robert Golding 2009-08-01 20:21:56 UTC
I don't know why, but I re-synced the tree yesterday and after updating @world the problem disappeared.

This is the list of updates;
 bind-tools
 lynx
 docbook-xml-dtd
 sg3_utils
 mozilla-firebird
 pygtk
 nvidia-settings
 pygobject
 xulrunner
Comment 42 Liviu Balan 2009-08-09 18:38:00 UTC
Same thing happens here.
emerge @preserved-rebuild is building 43 packages all over again... the list doesn't shrink no matter what I do.
Any workarounds? I really need this fixed as soon as possible..


--------------------------------------------------------
!!! existing preserved libs:
>>> package: sys-libs/e2fsprogs-libs-1.41.8
 *  - /lib64/libuuid.so
 *      used by /bin/mount (sys-apps/util-linux-2.16)
 *      used by /bin/umount (sys-apps/util-linux-2.16)
 *      used by /sbin/blkid (sys-apps/util-linux-2.16)
 *      used by 305 other files
 *  - /lib64/libblkid.so
 *      used by /bin/mount (sys-apps/util-linux-2.16)
 *      used by /bin/umount (sys-apps/util-linux-2.16)
 *      used by /sbin/blkid (sys-apps/util-linux-2.16)
 *      used by 11 other files
>>> package: sys-libs/readline-6.0_p3
 *  - /lib64/libreadline.so
 *  - /lib64/libreadline.so.5
 *  - /lib64/libreadline.so.5.2
 *      used by /usr/bin/calgebra (kde-base/kalgebra-4.3.0)
Use emerge @preserved-rebuild to rebuild packages using these libraries
Comment 43 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-09 18:47:00 UTC
(In reply to comment #42)
> Same thing happens here.
> emerge @preserved-rebuild is building 43 packages all over again... the list
> doesn't shrink no matter what I do.
> Any workarounds? I really need this fixed as soon as possible..

You can manually remove preserved libraries.
Comment 44 Liviu Balan 2009-08-15 15:40:05 UTC
What happens if I remove them, how will this impact my system?
Do I have to run "revdep-rebuild" after that, or just "emerge @preserved-rebuild" again?

thank you,
Comment 45 Zac Medico gentoo-dev 2009-08-15 18:11:58 UTC
(In reply to comment #44)
> What happens if I remove them, how will this impact my system?
> Do I have to run "revdep-rebuild" after that, or just "emerge
> @preserved-rebuild" again?

Yes, you need to run revdep-rebuild afterwards. When you remove the libraries, some stuff might be temporarily broken until revdep-rebuild is completed. For example, mount/umount might be unusable when /lib64/libuuid.so is removed. However, you maybe be able to use busybox as a temporary workaround if necessary.
Comment 46 Toby Ehrenkranz 2009-08-20 18:39:16 UTC
I'm also seeing this behavior with libs from sys-libs/e2fsprogs-libs-1.41.8. Specifically with /lib/libblkid.so and /lib/libuuid.so.  I've re-emerged a couple of times and the preserved rebuild list isn't getting any smaller.  At first there were some other preserved libs, but re-emerging fixed those.  For some reason re-emerging isn't making ebuilds build with the new blkid and uuid libs.
Comment 47 Toby Ehrenkranz 2009-08-21 19:08:17 UTC
Actually now that I look at it more I think they were rebuilt with the new libs (uuid and blkid).  In fact I don't see the old libs anywhere.  The only libs are the newly built ones, but for some reason portage thinks they are the old libs.
Comment 48 Liviu Balan 2009-08-25 16:33:21 UTC
I removed:
 *  - /lib64/libreadline.so
 *  - /lib64/libreadline.so.5
 *  - /lib64/libreadline.so.5.2

Then ran revdep-rebuild and found no issues. Looks like these libraries aren't even required by any package.

Also, I ran "emerge -av" for:
 sys-libs/readline
 kde-base/kalgebra-4.3.0

And these files weren't created anymore.. that's odd.
Comment 49 Liviu Balan 2009-08-25 16:38:37 UTC
Same thing for these libraries:

 *  - /lib64/libuuid.so
 *  - /lib64/libblkid.so

removed, ran revdep-rebuild and found no issues.
re-emerged:
  sys-libs/e2fsprogs-libs 
  sys-apps/util-linux
and the libraries still not showing up in my system.

So why these libraries were required in the first place?
Comment 50 Markus Duft (RETIRED) gentoo-dev 2009-11-04 12:15:54 UTC
i have another one:

i merged @system in a prefix environment. in particular ncurses and python are causing my problem:

USE=unicode emerge -v system
USE=-unicode -avuDN system

(rebuilds ncurses, python, etc).

after that, python still consumes libncursesw.so.5. portage seems to also preserve all symlinks to the real library, so whats preserved, is:

libncursesw.so
libncursesw.so.5
libncursesw.so.5.7

i think (but i'm not sure) that libncursesw.so should _not_ be preserved, as this would be the one that packages would look for during build, right?

i guess only the "real" and the "soname" libs/links should be kept. all syms that are unrelated (neither real lib, nor named like the soname of the real lib) should be removed to prevent further linking against that library.
Comment 51 Thomas R. (TRauMa) 2010-01-16 20:39:02 UTC
I can confirm the libblkid/libuuid issue and others after running emerge @preserved-rebuild, both with and without --as-needed.

After reading all the comments I'm still unsure what to do about it, as I can't just kill system binaries on this system, not even for a short time. Right now I just keep ignoring portages revdep-rebuild warnings.

Zac or other interested parties, I can post more details here and/or join IRC if that helps.

portage keeps saying:

(...)
>>> package: sys-libs/e2fsprogs-libs-1.41.9
 *  - /lib64/libblkid.so
 *      used by /bin/mount (sys-apps/util-linux-2.16.1)
 *      used by /bin/umount (sys-apps/util-linux-2.16.1)
 *      used by /sbin/blkid (sys-apps/util-linux-2.16.1)
 *      used by 10 other files
 *  - /lib64/libuuid.so
 *      used by /sbin/debugfs (sys-fs/e2fsprogs-1.41.9)
 *      used by /sbin/dumpe2fs (sys-fs/e2fsprogs-1.41.9)
 *      used by /sbin/e2fsck (sys-fs/e2fsprogs-1.41.9)
 *      used by 251 other files
>>> package: dev-libs/libgcrypt-1.4.0-r1
 *  - /usr/lib/libgcrypt.so.11
 *      used by /sbin/crda (net-wireless/crda-1.0.1-r1)
 *      used by /sbin/regdbdump (net-wireless/crda-1.0.1-r1)
 *      used by /usr/bin/certtool (net-libs/gnutls-2.8.5)
 *      used by 95 other files
Use emerge @preserved-rebuild to rebuild packages using these libraries

lddx is a tree dependency viewer of so deps i wrote:

# lddx.py -v /bin/mount
/bin/mount : 
+-- /lib/libblkid.so.1
| +-- /lib/libuuid.so.1
| | +-- /lib/libc.so.6
| +-- /lib/libc.so.6
+-- /lib/libc.so.6
+-- /lib/libuuid.so.1
| +-- /lib/libc.so.6

# lddx.py -v /sbin/e2fsck
/sbin/e2fsck : 
+-- /lib/libext2fs.so.2
| +-- /lib/libcom_err.so.2
| | +-- /lib/libpthread.so.0
| | | +-- /lib/libc.so.6
| | +-- /lib/libc.so.6
| +-- /lib/libc.so.6
| +-- /lib/libpthread.so.0
| | +-- /lib/libc.so.6
+-- /lib/libcom_err.so.2
| +-- /lib/libpthread.so.0
| | +-- /lib/libc.so.6
| +-- /lib/libc.so.6
+-- /lib/libblkid.so.1
| +-- /lib/libuuid.so.1
| | +-- /lib/libc.so.6
| +-- /lib/libc.so.6
+-- /lib/libuuid.so.1
| +-- /lib/libc.so.6
+-- /lib/libe2p.so.2
| +-- /lib/libc.so.6
+-- /lib/libc.so.6
+-- /lib/libpthread.so.0
| +-- /lib/libc.so.6

Portage 2.2_rc61 (default/linux/amd64/10.0/desktop, gcc-4.3.4, glibc-2.10.1-r1, 2.6.31.5 x86_64)
=================================================================
System uname: Linux-2.6.31.5-x86_64-AMD_Athlon-tm-_64_Processor_3200+-with-gentoo-2.0.1
Timestamp of tree: Sat, 16 Jan 2010 11:15:02 +0000
distcc 3.1 x86_64-pc-linux-gnu [disabled]
app-shells/bash:     4.0_p35
dev-java/java-config: 2.1.9-r2
dev-lang/python:     2.5.4-r3, 2.6.4
dev-python/pycrypto: 2.1.0_beta1
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r4
sys-apps/sandbox:    2.2
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA ut2003 RTCW-ETEULA ETQW dlj-1.1 sun-bcla-java-vm skype-eula"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=k8 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=k8 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests distlocks fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unmerge-logs unmerge-orphans userfetch"
FFLAGS="-march=k8 -O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/             http://mirror.netcologne.de/gentoo/             http://mirror.jamit.de/gentoo/"
LANG="de_DE.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="de en"
MAKEOPTS="-j2 -s"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/armagetron /usr/local/portage/trauma"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext 7z X a52 aac ace acl acpi adns alsa amd64 applet asf async asyncns autoipd avahi bash-completion branding bzip2 cairo cdda cdr cli connection-sharing consolekit cracklib crypt cups curl cxx dbus dhcpcd dirac disk-partition divx djvu dri dts dvd dvdr dvdread dvi emboss encode evo exif fam fame fat ffmpeg firefox flac fts3 fuse gcj gdbm gedit geoip gif gmedia gmp gnome gnome-keyring gnutls gpm gstreamer gtk hal howl iconv icu iproute2 ipv6 java java5 java6 jbig jce jpeg jpeg2k kqemu lame libnotify lm_sensors lzma lzo mad matroska mdnsresponder-compat mikmod mmap mmx mmxext mng modules moviepack moviesounds moznoirc mozsvg mp2 mp3 mp4 mpeg mplayer mudflap multilib nautilus ncurses networkmanager new-login nls nptl nptlonly nss ntfs ogg openal opengl openmp pam pcre pdf perl png policykit ppds pppd pulseaudio python qt3support qt4 quicktime rar readline realmedia reflection replytolist resolvconf rtc samba sdl session speex spell spl sqlite sqlite3 srt sse sse2 ssl startup-notification subtitles svg sysfs theora thunar tiff timidity trayicon truetype unicode upnp usb userlocales v4l v4l2 vcd vmc vorbis webdav wmp wxwindows x264 xattr xfs xine xml xorg xscreensaver xulrunner xv xvid xvmc zeroconf zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nv nvidia v4l vesa fglrx ati radeon radeonhd"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 52 Navid Zamani 2010-01-31 23:49:41 UTC
*** Bug 303042 has been marked as a duplicate of this bug. ***
Comment 53 Zac Medico gentoo-dev 2011-04-27 16:11:05 UTC
*** Bug 365093 has been marked as a duplicate of this bug. ***
Comment 54 Martin von Gagern 2011-05-17 09:39:01 UTC
(In reply to comment #53)
> *** Bug 365093 has been marked as a duplicate of this bug. ***

I investigated this issue related to the util-xcb upgrade.

* The package upgrade completely removed three libraries (libxcb-{atom,aux,event}.so), as opposed to replacing them with newer versions.

* The file /usr/lib64/libgnome-desktop-2.la from gnome-base/gnome-desktop-2.32.1 referenced these files in the lafile-fixed fashion, i.e. -lxcb-{atom,aux,event}.

* The corresponding /usr/lib64/libgnome-desktop-2.so didn't list these libs as dependencies, probably due to LDFLAGS=-Wl,-as-needed.

* The package mail-client/evolution-2.32.2-r3, and presumably gnome-base/gnome-settings-daemon-2.32.1-r1 and media-sound/pulseaudio-0.9.22 as well, picked up the linker flags from the la file, and actually referenced some symbol in the libs, adding deps to their so files despite -as-needed.

Indeed a simple "emerge -1 gnome-desktop && emerge @preserved-rebuild" did solve the issue for me, although I don't have pulseaudio here.

So I guess a proper solution at least to this manifestation would be having preserved-rebuild take removed libs and la files into account: when the list of preserved libs contains files that have no more recent replacement, than any reference to that basename in any la file would indicate a dependency.

Probably another, more radical solution would be punting all those la files altogether, but I guess we aren't ready for that approach yet. Couldn't find a feature request or tracker bug for it, though.
Comment 55 Zac Medico gentoo-dev 2015-10-05 19:37:26 UTC
*** Bug 562344 has been marked as a duplicate of this bug. ***