Portage doesn't correctly understand the situation when a directory is replaced with a symlink. I think, the error is in file "/usr/lib/portage/pym/portage.py", string 8561. I suppose, this is because portage rely on correctness of os.rmdir() in this situation. May be the solution is to insert additional check before os.rmdir(). Reproducible: Always Steps to Reproduce: 1. Install any package, that creates it's own directory 2. Delete this directory and create broken symlink with the same name 3. Uninstall the package Actual Results: You see, that your symlink was unmerged with status "<<< dir" Expected Results: Instead, portage should skip it with status "--- replaced dir" or "--- replaced obj" 1) Note, that if you create non broken symlink to some directory, portage will skip it during unmerging with status "--- !empty dir". This is not so distructive, but also wrong. 2) if you create non broken symlink to some file, portage will unmerge the symlink itself with status "<<< dir" This is also not good. Portage 2.1.4.4 (default-linux/x86/2007.0, gcc-4.1.2, glibc-2.6.1-r0, 2.6.23-gentoo-r8-hippo-20080227 i686) ================================================================= System uname: 2.6.23-gentoo-r8-hippo-20080227 i686 Intel(R) Core(TM)2 Duo CPU E4600 @ 2.40GHz Timestamp of tree: Mon, 06 Oct 2008 08:45:01 +0000 app-shells/bash: 3.2_p33 dev-java/java-config: 1.3.7, 2.1.6 dev-lang/python: 2.4.4-r13, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r2 sys-devel/automake: 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=prescott -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /opt/openjms/config /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="" CXXFLAGS="-march=prescott -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--alphabetical --buildpkg" FEATURES="ccache distlocks fixpackages metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="ftp://mirror.yandex.ru/gentoo-distfiles" LANG="ru_RU.KOI8-R" LC_ALL="ru_RU.KOI8-R" LINGUAS="en ru" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="-h --recursive --links --safe-links --perms --times --compress --force --human-readable --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="7zip X a52 aac acpi alsa arts async audacious bash-completion bzip2 cairo charconv chardet cli context cpudetection cracklib crypt cscope cyrillic dbus directfb djvu dri dvd dvdr emf exif extra ffmpeg firefox flac fortran fuse gcrypt gd gdbm ggi gif gimp glib gmp graphics graphviz gs gtk gzip hardened humanities iconv icu id3tag imagemagick imlib iproute2 jce jms jpeg jpeg2k kde latex lcms lm_sensors log4j lzo mad matroska midi mmap mmx mp3 mpeg mplayer mudflap ncurses nls nptl nptlonly nsplugin ntfs ogg omega openal opengl openmp pam pcre pdf perl png postscript python qt3 qt4 rar rdesktop readline science sdl session slang smi smp spl sse sse2 ssl ssse3 subversion svg svgz sysfs sysvipc tetex tex4ht threads truetype unicode userlocales vim vim-syntax vim-with-x vorbis win32codecs wireshark wma wmf wxwindows x86 xattr xetex xml xorg xosd xv xvid zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ru" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 i810 imstt mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(In reply to comment #0) Let me demonstrate the behavior. 0) Create simple ebuild "test-bug/bug/bug-1.ebuild" src_install() { dodir /qqqq } 1) Install the package hippo_comp ~ # ebuild test-bug/bug/bug-1.ebuild merge * checking ebuild checksums ;-) ... [ ok ] * checking auxfile checksums ;-) ... [ ok ] * checking miscfile checksums ;-) ... [ ok ] >>> Unpacking source... >>> Source unpacked. >>> Compiling source in /var/tmp/portage/test-bug/bug-1/work ... >>> Source compiled. >>> Test phase [not enabled]: test-bug/bug-1 >>> Install bug-1 into /var/tmp/portage/test-bug/bug-1/image/ category test-bug >>> Completed installing bug-1 into /var/tmp/portage/test-bug/bug-1/image/ * checking 0 files for package collisions >>> Merging test-bug/bug-1 to / >>> /qqqq/ >>> test-bug/bug-1 merged. 2) Delete the directory and create broken symlink hippo_comp ~ # rm -r /qqqq/ hippo_comp ~ # ln -s /qaz /qqqq 3) Unmerge the package hippo_comp ~ # ebuild test-bug/bug/bug-1.ebuild unmerge No package files given... Grabbing a set. <<< dir /qqqq Notice the status and see, that there is no more symlink.:) If we create a symlink to a directory, we'll get: hippo_comp ~ # ebuild test-bug/bug/bug-1.ebuild unmerge No package files given... Grabbing a set. --- !empty dir /qqqq If we create a symlink to a file, we'll get: hippo_comp ~ # ebuild test-bug/bug/bug-1.ebuild unmerge No package files given... Grabbing a set. <<< dir /qqqq
I think this is the same as: http://archives.gentoo.org/gentoo-alt/msg_2d3dfb3888ac4dcd4e64e1539ef039ff.xml and could possibly be fixed by r14248
(In reply to comment #2) > I think this is the same as: > http://archives.gentoo.org/gentoo-alt/msg_2d3dfb3888ac4dcd4e64e1539ef039ff.xml > and could possibly be fixed by r14248 Sounds good and looks similar. Will try to update my Prefix tomorrow and test the fix. Are there any chances in merging this to trunk or 2.1.6 branch upon successful testing?
(In reply to comment #2) > I think this is the same as: > http://archives.gentoo.org/gentoo-alt/msg_2d3dfb3888ac4dcd4e64e1539ef039ff.xml > and could possibly be fixed by r14248 Thanks, I've merged that in r14250.
(In reply to comment #3) > Are there any chances in merging this to trunk or 2.1.6 branch upon successful > testing? It will be in the next stable release which is due soon (it will be a new 2.1.7 branch).
(In reply to comment #3) > Sounds good and looks similar. > Will try to update my Prefix tomorrow and test the fix. I haven't yet released a portage with that fix in Prefix. > Are there any chances in merging this to trunk or 2.1.6 branch upon successful > testing? The patch is easy to apply yourself so you can test it straight away: http://sources.gentoo.org/viewcvs.py/portage/main/trunk/pym/portage/dbapi/vartree.py?r1=14236&r2=14250&diff_format=u
(In reply to comment #6) > The patch is easy to apply yourself so you can test it straight away: > http://sources.gentoo.org/viewcvs.py/portage/main/trunk/pym/portage/dbapi/vartree.py?r1=14236&r2=14250&diff_format=u I can confirm, that this changeset fixes this bug with FEATURES="-unmerge-orphans". But with FEATURES="unmerge-orphans" the symlink is still unmerged at line 2470 in vartree.py@14250. Again the incorrect behavior is in the following cases: 1) Symlink is broken 2) Symlink points to a file (symlink is created manually instead of directory after package merge simulating another package doing something nasty in postinst phase)
hmmm, it looks to me as the unmerge-orphans kicks in too early, as it should use the same logic as the code right below it IMO, making sure only mtime changes are allowed, but not changes in type.
Created attachment 204066 [details, diff] Proposed way to deal with unmerge-orphans. (not tested)
(In reply to comment #8) > hmmm, it looks to me as the unmerge-orphans kicks in too early, as it should > use the same logic as the code right below it IMO, making sure only mtime > changes are allowed, but not changes in type. Is it better to remove the whole 'if unmerge_orphans' block and integrate its logic into the below 'if pkgfiles[objkey][0] == "dir"' code? I've attached a patch proposing such a change. It's not tested and the full logic isn't verified.
(In reply to comment #7) > Again the incorrect behavior is in the following cases: > 1) Symlink is broken > 2) Symlink points to a file > (symlink is created manually instead of directory after package merge > simulating another package doing something nasty in postinst phase) I'm not sure that we should be leaving stuff like this intact with unmerge-orphans. The idea behind unmerge-orphans it to avoid leaving any orphans behind (whether they were created by a nasty postinst phase or not). (In reply to comment #9) > Created an attachment (id=204066) [edit] > Proposed way to deal with unmerge-orphans. (not tested) Aside from my reservation about leaving orphans when unmerge-orphans is enabled, the patch seems reasonable, except that it will also change unmerge-orphans behavior for regular files whose md5 differs from when they were originally installed. Also, I've just realized that the unmerge-orphans code has protection for symlinks like /lib -> lib64, and it seems like the change from r14248/r14250 might remove similar protection, so I'm thinking about reverting it.
(In reply to comment #11) > Also, I've just realized that the unmerge-orphans code has protection for > symlinks like /lib -> lib64, and it seems like the change from r14248/r14250 > might remove similar protection, so I'm thinking about reverting it. Nevermind this, the code seems safe since 'not stat.S_ISDIR(lstatobj.st_mode)' should hold for lib -> lib64 symlinks.
Created attachment 204789 [details, diff] updated patch to ignore file's md5 sum
(In reply to comment #11) > (In reply to comment #9) > Aside from my reservation about leaving orphans when unmerge-orphans is > enabled, the patch seems reasonable, except that it will also change > unmerge-orphans behavior for regular files whose md5 differs from when they > were originally installed. Agree, I missed this in the first patch. Taking your remark into account I've attached an updated patch.
(In reply to comment #11) > (In reply to comment #7) > > Again the incorrect behavior is in the following cases: > > 1) Symlink is broken > > 2) Symlink points to a file > > (symlink is created manually instead of directory after package merge > > simulating another package doing something nasty in postinst phase) > > I'm not sure that we should be leaving stuff like this intact with > unmerge-orphans. The idea behind unmerge-orphans it to avoid leaving any > orphans behind (whether they were created by a nasty postinst phase or not). Well, rethinking the portage behavior and taking into account it's collision-protect feature I now agree with you. But I'd prefer portage to inform me, that it noticed some inconsistencies. For example, "<<< !dir /qqqq" instead of "<<< dir /qqqq" or "<<< !mtime /foo" instead of "<<< obj /foo".
(In reply to comment #4) > (In reply to comment #2) > > I think this is the same as: > > http://archives.gentoo.org/gentoo-alt/msg_2d3dfb3888ac4dcd4e64e1539ef039ff.xml > > and could possibly be fixed by r14248 > > Thanks, I've merged that in r14250. > Via gitweb: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=25d23593b4fa78f7e9694498ef71bca13acdb061 This was released in portage-2.1.7. Please file a new bug for any remaining issues.
