If you follow this (quite common I'd say) install/upgrade path: emerge =jpeg-6* emerge -u =jpeg-7* emerge jpeg-compat The result is this: $ portageq owners / /usr/lib/libjpeg.so.62 media-libs/jpeg-7 /usr/lib/libjpeg.so.62 media-libs/jpeg-compat-6b-r1 /usr/lib/libjpeg.so.62 Two packages own a file, even with collision-protect on. Why? I guess it's because jpeg-compat removes it from system in preinst, but that leaves it in jpeg's CONTENTS. And since there's a good chance that the .so file in jpeg-compat is compiled with same gcc as the jpeg-6 was, the MD5 digest will match: $ qcheck jpeg Checking media-libs/jpeg-compat-6b-r1 ... * 3 out of 3 files are good Checking media-libs/jpeg-7 ... MTIME: /usr/lib64/libjpeg.la MTIME: /usr/lib64/libjpeg.so.62 AFK: /usr/lib64/libjpeg.so.62.0.0 * 42 out of 45 files are good The unfortunate consequence of this is that unmerging jpeg removes /usr/lib/libjpeg.so.62 too. And I guess this applies to upgrading jpeg as well, which is a dangerous ticking bomb? Unless we stop needing the -compat before jpeg is (rev)bumped... Emerge --info of the stable chroot when I've discovered this when fiddling around icedtea6 binpkg: # emerge --info Portage 2.1.6.13 (default/linux/x86/2008.0, gcc-4.3.2, glibc-2.9_p20081201-r2, 2.6.30-gentoo-r4-perfctr i686) ================================================================= System uname: Linux-2.6.30-gentoo-r4-perfctr-i686-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4400+-with-gentoo-1.12.11.1 Timestamp of tree: Unknown app-shells/bash: 4.0_p28 dev-java/java-config: 2.1.8-r1 dev-lang/python: 2.5.4-r2, 2.6.2-r1 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.7.9-r1, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=i686 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/data/distfiles" FEATURES="collision-protect distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" LDFLAGS="-Wl,-O1" MAKEOPTS="-j3" PKGDIR="/prj/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/prj/gentoo/java/java-overlay" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="acl berkdb bzip2 cli cracklib crypt cups dri fortran gdbm gpm iconv ipv6 isdnlog modules mudflap ncurses nls nptl nptlonly openmp pam pcre perl pppd python readline reflection session spl ssl sysfs tcpd unicode x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(In reply to comment #0) > Two packages own a file, even with collision-protect on. Why? I guess it's > because jpeg-compat removes it from system in preinst, but that leaves it in > jpeg's CONTENTS. It's not preinst (collision protection happens before preinst), but pkg_setup of jpeg-compat that prevents collision protection from detecting the problem: pkg_setup() { if ! has_version media-libs/jpeg-compat ; then if [[ -e ${ROOT}/usr/$(get_libdir)/libjpeg.so.62 ]] ; then elog "Removing libjpeg.so.62 manually from media-libs/jpeg" rm -f "${ROOT}/usr/$(get_libdir)/libjpeg.so.62" fi fi }
I don't know the specifics about why the jpeg-compat was written to resolve the collision in pkg_setup. Anyway, the way that collisions like these are typically resolved is so do a version bump for media-libs/jpeg that doesn't install the colliding file, and to make jpeg-compat block the older version of media-libs/jpeg that installed the colliding file. Something like this would go in the jpeg-compat ebuild: RDEPEND= "!<media-libs/jpeg-7" Blockers like this have been automatically resolved since portage-2.1.2, so this type of blocker does not inconvenience the user in any way. The blocker tells portage that it should allow the file collision temporarily, and the colliding file is automatically removed from the contents of the older version of media-libs/jpeg. In some cases portage can adjust the merge order so that even the "temporary collision" is avoided. Here's some more background on the subject: http://blogs.gentoo.org/zmedico/2008/05/09/blocking_package_file_collisions
(In reply to comment #0) > If you follow this (quite common I'd say) install/upgrade path: > emerge =jpeg-6* > emerge -u =jpeg-7* And here you failed to run the revdep-rebuild command provided by the postinst message for old-style preserved-libs. > emerge jpeg-compat
(In reply to comment #2) > media-libs/jpeg that installed the colliding file. Something like this would > go in the jpeg-compat ebuild: > RDEPEND= "!<media-libs/jpeg-7" It's been there since it was committed.
(In reply to comment #3) > (In reply to comment #0) > > If you follow this (quite common I'd say) install/upgrade path: > > emerge =jpeg-6* > > emerge -u =jpeg-7* > > And here you failed to run the revdep-rebuild command provided by the postinst > message for old-style preserved-libs. What if I didn't have the chance? Both jpeg-7 and jpeg-compat could happen in one run of emerge -uD world, or not? If the intention was to force user do that, then jpeg-compat should just die when the file exists, not remove it...
(In reply to comment #4) > (In reply to comment #2) > > media-libs/jpeg that installed the colliding file. Something like this would > go in the jpeg-compat ebuild: > > RDEPEND= "!<media-libs/jpeg-7" > > It's been there since it was committed. Ahh, I missed that because it was in DEPEND instead of RDEPEND. For strict correctness, it should be in RDEPEND so that it's guaranteed to work even for binary packages. (In reply to comment #5) > If the intention was to force user do that, then jpeg-compat should just die > when the file exists, not remove it... Seems like a use case for a !!atom blocker.
blockers dont make sense here. if you install media-libs/jpeg-7, then you should have done a revdep-rebuild. once that is done, the library should be removed. then binary packages can cleanly pull in jpeg-compat only when they're needed. the "common" steps you describe make no sense. people should never emerge jpeg-compat manually, nor are there any instructions telling people to do that. either way, Zac already points out the code that jpeg-compat uses to make sure collisions are resolved transparently. you need to provide clear info that actually makes sense that shows a real problem and not things you think are a problem. i'll update the jpeg-compat blocker to be in RDEPEND as suggested.
Sorry for the misunderstanding. The emerge commands in my initial comments are meant as a stripped down case of what will happen in background, not what the user will execute directly. So here's a proof that it can happen if you had installed jpeg-6, a package that needs jpeg-6 or jpeg-compat, and you want to update world: # emerge -uvaDN world These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild U ] media-libs/jpeg-7 [6b-r8] 0 kB [0] [ebuild N ] media-libs/jpeg-compat-6b-r1 0 kB [0] [ebuild U ] dev-java/icedtea6-bin-1.6.1 [1.3.1-r1] USE="-X -alsa -doc -examples -nsplugin -source" 0 kB [0] So, here jpeg-compat will just merge and do the bad thing before you have a chance to run revdep-rebuild after jpeg update. The problem (partially of portage I suppose) is that after jpeg-7 will be (rev)-bumped, /usr/lib/libjpeg.so.62 will be removed from system (because it's still part of jpeg-7 CONTENTS), leaving jpeg-compat silently broken. And I just realized it occurs even if jpeg-compat produces the library with different MD5 (because unmerge-orphans is default in 2.1.6.13). Maybe it could check if there are other packages owning the file before removing it? But maybe it would be unnecessary slowdown to prevent just such an obscure situation... So my point is that if you really want to force the user to run it, the pkg_setup of jpeg-compat should die saying so, not silently remove the file. Changing it now won't fix systems of those that already went through this, but the problem seems to be much smaller than I thought, just few -bin packages depend on jpeg-compat... And I just realized I don't know what happens when the revdep-rebuild is executed as suggested (but "late", after jpeg-compat is also merged)...
(In reply to comment #8) > And I just realized I don't know what happens when the revdep-rebuild is > executed as suggested (but "late", after jpeg-compat is also merged)... Ah, suggests users to remove the lib manually. They might not realize it now belongs to jpeg-compat...
telling the user to `rm` the file manually wont fix things either. they would have to do the revdep-rebuild such that portage no longer thought jpeg owned the package. then they could safely install jpeg-compat without portage removing the library afterwards. pkg_setup() { if ! has_version media-libs/jpeg-compat ; then if [[ -e ${ROOT}/usr/$(get_libdir)/libjpeg.so.62 ]] ; then eerror "You must clean your system with a revdep-rebuild first" die "run revdep-rebuild --library libjpeg.so.62" fi fi }
Somewhat unrelated to this bug but jpeg-compat isn't meant to be used as a solution, but only a temporary workaround until binary pkgs are fixed. In your paste, icedtea6-bin, built by Gentoo from openjdk is pulling in jpeg-compat and this is very annoying. Bug 283248 should be speeded up, and new icedtea6-bin package introduced to tree.
considering the decade that jpeg-6 was out there, expecting all binary packages to get fixed is a pipe dream. once things settle, i'll move jpeg-6b to a dedicated SLOT just like we have with old readline and old libtool ebuilds -- they only install the SONAME.
(In reply to comment #12) > considering the decade that jpeg-6 was out there, expecting all binary packages > to get fixed is a pipe dream. once things settle, i'll move jpeg-6b to a > dedicated SLOT just like we have with old readline and old libtool ebuilds -- > they only install the SONAME. > Well, icedtea6-bin is a in-house built binary. True for most others, but I've been tracking them with tinderbox as it's easy when the package has a own name, and while I can't fix them I'm trying to convince upstream's to do so.
(In reply to comment #11) > In your paste, icedtea6-bin, built by Gentoo from openjdk is pulling in > jpeg-compat and this is very annoying. Bug 283248 should be speeded up, > and new icedtea6-bin package introduced to tree. I am building new icedtea6-bin right now (got issues with nsplugin), but it will still need jpeg-compat, since I don't have the time to fix the source right now :( (In reply to comment #10) > telling the user to `rm` the file manually wont fix things either. they would > have to do the revdep-rebuild such that portage no longer thought jpeg owned > the package. then they could safely install jpeg-compat without portage > removing the library afterwards. I think the "such that portage no longer thought jpeg owned the package" part only works with portage 2.2's preserve-libs feature. Old portage won't detect anything once revdep-rebuild is done, and the lib still stays in jpeg-7's CONTENTS. Right, Zac? > pkg_setup() { > if ! has_version media-libs/jpeg-compat ; then > if [[ -e ${ROOT}/usr/$(get_libdir)/libjpeg.so.62 ]] ; then > eerror "You must clean your system with a revdep-rebuild first" > die "run revdep-rebuild --library libjpeg.so.62" > fi > fi > }
yes, the issue is most likely restricted to ~arch only
*** Bug 291410 has been marked as a duplicate of this bug. ***
Time to drop this preserve_old_lib handling and get this closed? Most users have upgraded by now, new installs are based on jpeg-7, and the rest should know howto use revdep-rebuild.
should re-add an ABI SLOT-ed jpeg-6b too ... but dropping the preserve_lib is OK by me now
I guess this file collision from attempt to upgrade-world is another aspect of this bug, or a new bug needs to be opened? * media-libs/jpeg-8 * /usr/lib/libjpeg.so.7 * * Package 'media-libs/jpeg-7-r1' NOT merged due to file collisions. If * necessary, refer to your elog messages for the whole content of the * above message. # portageq owners / /usr/lib/libjpeg.so.7 media-libs/jpeg-8 /usr/lib/libjpeg.so.7 * =media-libs/jpeg-7* pulled in by: * ('installed', '/', 'dev-java/icedtea6-bin-1.6.2-r1', 'nomerge') ... so, dev-java/icedtea6-bin... requires jpeg-7, so basicaly I'll have to yank it out in order to allow the upgrade to jpeg-8. Or block it until the file collision is resolved...? This bug or a new one? Thanks.
(In reply to comment #19) > I guess this file collision from attempt to upgrade-world is another aspect of > this bug, or a new bug needs to be opened? Fixed now, sync in a hour or so. Also jpeg-compat isn't in tree anymore. Reopen if there was something left here to do.
