When I just upgraded from screen-4.0.1 -> screen-4.0.2-r4 screen 4.0.2-r4 merged successfully. but unmerging of screen 4.0.1 does complete nicely. When running emerge --unmerge =screen-4.0.1 it will give me this stackstrace Traceback (most recent call last): File "/usr/bin/emerge", line 2954, in ? if 1==unmerge(myaction, myfiles): File "/usr/bin/emerge", line 2251, in unmerge retval=portage.unmerge(mysplit[0],mysplit[1],portage.root,mysettings,unmerge_action not in ["clean","prune"]) File "/usr/lib/portage/pym/portage.py", line 2901, in unmerge mylink.unmerge(trimworld=mytrimworld,cleanup=1) File "/usr/lib/portage/pym/portage.py", line 6176, in unmerge if (pkgfiles[obj][0] not in ("dir","fif","dev","sym")) and (lmtime != pkgfiles[obj][1]): KeyError: '.' I have try to trace down the problem and turns out that, it seems that the CONTENTS (of screen 4.0.1) has a weired line obj /var/run/screen/.keep d41d8cd98f00b204e9800998ecf8427e The problem is, the timestamp field is absent. In dblink.getcontents() it always assume that, there is 4 fields in CONTENTS file. I am not familiar with portage, I also don't know if this is a bug of portage or caused by a broken ebuild. But I think, dblink.getcontents should be able to handle this trivial exception case. I'll attach my broken CONTENTS Reproducible: Always Steps to Reproduce: 1. 2. 3.
Created attachment 64445 [details] The offending CONTENTS file Notice the '/var/run/screen/.keep' line which has missed the timestamp
Created attachment 64584 [details, diff] portage-2.0.51.22-dblink.unmerge-pkgfiles-key.patch The problem here is that we are iterating through pkgfiles.keys() and inside the iteration loop we modify the obj variable so subsequent calls to pkgfiles[obj] may fail with the reported error. for obj in mykeys: obj=os.path.normpath(objkey) if obj[:2]=="//": obj=obj[1:] The obvious solution is to use a separate variable for the pkgfiles key. My patch introduces a new variable called objkey and replaces all pkgfiles[obj] calls with pkgfiles[objkey].
Right, I forget to provide the `emerge info` Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.3.5-20050130, glibc-2.3.5-r0, 2.6.11-hardened-r15-idesirq-nat-v4l-allmod i686) ================================================================= System uname: 2.6.11-hardened-r15-idesirq-nat-v4l-allmod i686 AMD Athlon(tm) XP 1800+ Gentoo Base System version 1.6.13 distcc 2.13 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=athlon-xp -pipe -ffast-math -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/init.d /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=athlon-xp -pipe -ffast-math -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig buildpkg distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="en_US.UTF-8" LINGUAS="en zh_HK zh_TW zh_CN ja kr" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 3dnow X a52 aac aalib alsa apache2 apm avi berkdb bitmap-fonts cdr cjk crypt cscope cups curl directfb divx4linux dvd dvdread eds emboss encode esd evo fam fame fbcon flac foomaticdb fortran gcj gd gdbm gif gnome gpm gstreamer gtk gtk2 guile hal imagemagick imap imlib ipv6 java jpeg junit libg++ libwww lzo mad mbox mcal mikmod mjpeg mmx motif mozilla mp3 mpeg mysql ncurses nls ogg oggvorbis opengl oss pam pdflib perl png ppds python qt quicktime readline samba sasl scanner sdl slang speex spell sqlite sse ssl svga tcltk tcpd tiff truetype truetype-fonts type1-fonts usb v4l v4l2 vorbis xine xml xml2 xrandr xv xvid zlib linguas_en linguas_zh_HK linguas_zh_TW linguas_zh_CN linguas_ja linguas_kr userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LC_ALL, LDFLAGS, MAKEOPTS, PORTDIR_OVERLAY
Hi Zac, Gentoo users / dev, I can confirm that, your patch make the unmerge completed successfully. And, I also confirm that, the problem is caused by malformed CONTENTS in pkg db. However, when I studied the problem again, I found the following point: When everything is okay and correct, the flow should be: 1. In the CONTENTS file, there is a line: obj /var/run/screen/.keep d41d8cd98f00b204e9800998ecf8427e 1122656930 2. During emerge --unmerge, dblink.getcontents() will parse the line and create a pkgfiles mapping. It use the following lines of code to handle 'obj' type line: if mydat[0]=="obj": #format: type, mtime, md5sum pkgfiles[string.join(mydat[1:-2]," ")]=[mydat[0], mydat[-1], mydat[-2]] The resulting record created will be like this: '/var/run/screen/.keep': ['obj', '1122656930', 'd41d8cd98f00b204e9800998ecf8427e'] 3. The pkgfiles mapping created in getcontents() is processed by dblink.unmerge(), it will extract all keys (ie. path for all files belongs to a package) and compare the mtime, md5sum, etc. for obj type entry. In the current situation, the flow should be: 1. First the offending line (in CONTENTS) is (NOTE: no timestamp field): obj /var/run/screen/.keep d41d8cd98f00b204e9800998ecf8427e 2. During dblink.getcontents(), the logic to parse the line for 'obj' type will create a wrong resulting record: '': ['obj', 'd41d8cd98f00b204e9800998ecf8427e', '/var/run/screen/.keep'] Note that, the key of the mapping should be the filepath. And list item is wrong also. In correct case, it should be [type, timestamp, md5sum] If the problem is the following line, I like: pkgfiles[string.join(mydat[1:-2]," ")]=[mydat[0], mydat[-1], mydat[-2]] Note the use of "mydat[1:-2]", if the line has only three field (my case) len(mydat) is only 3, so the slice will resulting a empty list. which result a empty string as the key of the record. I think, the patch Zac is good (I already test your patch :) by unmerge my stall screen-4.0.1. But, the problem is not how unmerge() iterate the keys, the problem is rooted at, getcontents always assume len(mydat) >=4. I believe, the use of mydat[1:-2] instead of mydat[1] is catering filename with 'space'. (mydat[1] is only part of the filename) I have another solution, which is to simply check the len(mydat) if type is 'obj'.
Created attachment 64645 [details, diff] My patch against dblink.getcontents The patch target dblink.getcontents() directly. If the current parsing line is of type 'obj', check the number of field. If the len is <4, by pass that line. As record in CONTENTS for type obj must be >=4.
Hi Zarick, I agree that the contents parsing is broken, in fact, *severely* broken (see Bug 14983). We need a new and improved file format. Do you have any idea how you got the malformed line in your contents file? We should address that too. Mine has the timestamp: obj /var/run/screen/.keep d41d8cd98f00b204e9800998ecf8427e 1118274194
This seems to be a duplicate of Bug 87147.
Hi Zac, Yes agree, this seems a duplicate of 87147. For the "broken" CONTENTS file, I have no idea why the file is broken. The timestamp is missed. May be due to system crash. Though, my system haven't crashed for months. On the other hand, I do think that, portage should have able to handle broken CONTENTS file. At least, omit the borken line in the files or tell user that the file is broken and the unmerge can't be continued unless user manually repair the file.
(In reply to comment #8) > For the "broken" CONTENTS file, I have no idea why the file is broken. > The timestamp is missed. May be due to system crash. > A system crash will not delete 1 mtime from a CONTENTS file. There must be a problem when the mtime is written in dblink.mergeme() inside portage.py. It do not recommend changing the contents parsing code in order to workaround a bug in dblink.mergeme(). That would clutter the contents parsing code and make the root problem less visible. If necessary, it is trivial to write a script to fix your broken CONTENTS files.
*** Bug 87147 has been marked as a duplicate of this bug. ***
A fix for the KeyError: '.' in dblink.unmerge() is in svn for 2.1_pre5.
*** Bug 123827 has been marked as a duplicate of this bug. ***
Released in 2.1_pre5.