Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 351814 - app-portage/gentoolkit fails to parse older-style CONTENTS files
Summary: app-portage/gentoolkit fails to parse older-style CONTENTS files
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on:
Blocks: 193766 349307
  Show dependency tree
 
Reported: 2011-01-16 01:09 UTC by J. Paul Reed
Modified: 2011-02-01 06:00 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
andle malformed sym entries (malformed_sym.patch,1.81 KB, patch)
2011-01-16 12:36 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description J. Paul Reed 2011-01-16 01:09:35 UTC
When doing an |equery b something|, I get a screen-full of:

!!! Parse error in '/var/db/pkg/app-shells/tcsh-6.12-r2/CONTENTS'
!!!   line 81: Unrecognized CONTENTS entry
!!! Parse error in '/var/db/pkg/app-text/docbook-dsssl-stylesheets-1.77-r2/CONTENTS'
!!!   line 177: Unrecognized CONTENTS entry
!!! Parse error in '/var/db/pkg/app-text/docbook-sgml-utils-0.6.12/CONTENTS'
!!!   line 41: Unrecognized CONTENTS entry
!!!   line 42: Unrecognized CONTENTS entry
!!!   line 43: Unrecognized CONTENTS entry
!!!   line 44: Unrecognized CONTENTS entry
!!!   line 47: Unrecognized CONTENTS entry
!!!   line 70: Unrecognized CONTENTS entry
!!!   line 71: Unrecognized CONTENTS entry
!!!   line 72: Unrecognized CONTENTS entry
!!! Parse error in '/var/db/pkg/app-text/openjade-1.3.2-r1/CONTENTS'
!!!   line 3: Unrecognized CONTENTS entry
!!!   line 4: Unrecognized CONTENTS entry
!!!   line 5: Unrecognized CONTENTS entry
!!!   line 7: Unrecognized CONTENTS entry
!!!   line 8: Unrecognized CONTENTS entry
!!!   line 15: Unrecognized CONTENTS entry
!!!   line 17: Unrecognized CONTENTS entry
!!!   line 94: Unrecognized CONTENTS entry
!!!   line 95: Unrecognized CONTENTS entry
!!!   line 96: Unrecognized CONTENTS entry
!!!   line 97: Unrecognized CONTENTS entry
!!!   line 98: Unrecognized CONTENTS entry
!!! Parse error in '/var/db/pkg/app-text/opensp-1.5-r1/CONTENTS'
!!!   line 13: Unrecognized CONTENTS entry
!!!   line 225: Unrecognized CONTENTS entry

etc.

The command eventually does complete and give me output.

I did a bunch of searching, and the common explanation is that "Your CONTENTS file is corrupted; check your hard drive."

However, the common theme among these lines is that they're "sym" lines, and it looks like the format of the sym line changed in portage's vartree dbabi. (This Gentoo installation is probably... 5 or 6 years old at this point, so it's not surprising that I'd run into this.)

I don't have enough context to be sure, but it looks like the problem might be related to?

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;h=7295411a389c89418b460c059e2a222475dd1ec4;hp=b9226939dc1b50d7b77e8ac99112d468b2140b52

The files that have problems have the following style of sym CONTENTS lines:

sym /usr/lib/libdb.so -> libdb-4.0.so (41471, 78241L, 770L, 1, 0, 0, 12L, 1064292767, 1064292767, 1064292767)

It looks like the new style is:

sym /lib/libgpm.so -> libgpm.so.1 1117594156

What is clear is that the current sym regex (http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=blame;f=pym/portage/dbapi/vartree.py;h=9f75da21b61fcd3dbe4c69645d413d5925660cd2;hb=HEAD#l1199) will fail to parse the old style, since it has commas in it.

Is there a portage command to convert old-style sym entries to new-style, and I missed it? It looks like the new style only cares about the ctime (or mtime?), not the other info that used to be there.

Reproducible: Always

Steps to Reproduce:
Difficult to describe; you need an older-style CONTENTS file to repro this problem. (I can attach one if it helps.)




Portage 2.1.9.25 (default/linux/x86/10.0/desktop, gcc-4.4.4, glibc-2.11.2-r3, 2.6.29.5 i686)
=================================================================
System uname: Linux-2.6.29.5-i686-Intel-R-_Pentium-R-_D_CPU_3.00GHz-with-gentoo-1.12.14
Timestamp of tree: Sun, 09 Jan 2011 13:00:23 +0000
app-shells/bash:     4.1_p7
dev-java/java-config: 1.3.7-r1, 2.1.11-r1
dev-lang/python:     2.6.6-r1
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.14-r1
sys-apps/sandbox:    2.3-r1
sys-devel/autoconf:  2.13::<unknown repository>, 2.65-r1
sys-devel/automake:  1.4_p6::<unknown repository>, 1.5::<unknown repository>, 1.6.3::<unknown repository>, 1.7.9-r1::<unknown repository>, 1.8.5-r3::<unknown repository>, 1.9.6-r2::<unknown repository>, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.4, 4.4.4-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.10
sys-devel/make:      3.81-r2
virtual/os-headers:  2.6.30-r1 (sys-kernel/linux-headers)
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA dlj-1.1"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -O2 -mfpmath=sse -falign-functions=4 -fprefetch-loop-arrays -funroll-loops -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=pentium4 -O2 -mfpmath=sse -falign-functions=4 -fprefetch-loop-arrays -funroll-loops -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="16bit 3dnow 3dnowext 7zip X Xaw3d a52 aac acl acpi aim alsa apache2 asf audacious audiofile bash-completion berkdb bittorrent bonjour branding buffysize bundled-libs bzip2 c++ cairo cdda cddb cdparanoia cdr cdrom cle266 cli consolekit cpudetection cracklib crypt cscope css curl cvs cxx dbus devfs26 dga dict dio divx divx4linux dri dts dv dvb dvbplayer dvd dvdr ecc encode excel exif expat extensions extras fam festival ffmpeg fftw flac flatfile fortran ftp gd gdbm gdu gif glibc-compat20 gmedia gmp gpg gphoto2 grammar gs gtk gzip iconv icq id3 irc jabber jpeg jpeg2k lame lcms libnotify libsamplerate logrotate lzo lzw mad matrox mbox md5sum mhash mikmod mmap mmx mmxext mng modules moznoxft mp3 mp4 mp4live mpeg mpeg2 mpeg4 mplayer msn mudflap musepack nas ncurses nethack no-old-linux nptl nptlonly objc objc++ offensive ogg opengl openmp oscar pam pango parse-clocks patch pcre pdf perl pg-vacuumdelay player png policykit posix postfix ppds python qt3support qt4 rar rdesktop readline realmedia regex rtc rtsp s3 sasl screen sdl server session sharedmem shorten silc sndfile sox spell sse sse2 ssl startup-notification subversion svg svnserve sysfs syslog tcpd tiff tools truetype twolame type1 udev unicode unzip usb v4l v4l2 vim vnc vorbis wav wavpack wma wma123 wmf wmp wxgtk1 wxwindows x264 x86 xcb xchattext xine xinetd xml xmlrpc xpm xterm xv xvid yahoo zip zlib zvbi" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CAMERAS="canon" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa fbdev i810" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Zac Medico gentoo-dev 2011-01-16 01:47:31 UTC
Re-assigning since the problem is in the portage API.

I would have kept support for the old format if I had known that it was still around. I see that openjade didn't have an version bumps between 2003 and 2010, so that explains why its CONTENTS is so old.
Comment 2 J. Paul Reed 2011-01-16 11:09:54 UTC
Yeah, as I mentioned, this Gentoo installation is pretty old (which says a lot about Gentoo! :-)

FWIW, this is actually a problem with more than a few packages on my system, apparently:

!!! Parse error in '/var/db/pkg/app-shells/tcsh-6.12-r2/CONTENTS'
!!! Parse error in '/var/db/pkg/app-text/docbook-dsssl-stylesheets-1.77-r2/CONTENTS'
!!! Parse error in '/var/db/pkg/app-text/docbook-sgml-utils-0.6.12/CONTENTS'
!!! Parse error in '/var/db/pkg/app-text/openjade-1.3.2-r1/CONTENTS'
!!! Parse error in '/var/db/pkg/app-text/opensp-1.5-r1/CONTENTS'
!!! Parse error in '/var/db/pkg/dev-libs/glib-1.2.10-r5/CONTENTS'
!!! Parse error in '/var/db/pkg/dev-libs/libunicode-0.4-r1/CONTENTS'
!!! Parse error in '/var/db/pkg/gnome-base/gconf-1.0.8-r5/CONTENTS'
!!! Parse error in '/var/db/pkg/gnome-base/libghttp-1.0.9-r3/CONTENTS'
!!! Parse error in '/var/db/pkg/gnome-base/libglade-0.17-r6/CONTENTS'
!!! Parse error in '/var/db/pkg/gnome-base/orbit-0.5.17/CONTENTS'
!!! Parse error in '/var/db/pkg/gnome-extra/guppi-0.40.3-r2/CONTENTS'
!!! Parse error in '/var/db/pkg/media-libs/divx4linux-20030428-r1/CONTENTS'
!!! Parse error in '/var/db/pkg/media-libs/gle-3.0.1-r2/CONTENTS'
!!! Parse error in '/var/db/pkg/media-libs/libfame-0.9.0/CONTENTS'
!!! Parse error in '/var/db/pkg/media-libs/t1lib-1.3.1/CONTENTS'
!!! Parse error in '/var/db/pkg/sys-libs/db-4.0.14-r2/CONTENTS'
!!! Parse error in '/var/db/pkg/x11-libs/Xaw3d-1.5-r1/CONTENTS'
Comment 3 Zac Medico gentoo-dev 2011-01-16 12:36:14 UTC
Created attachment 260000 [details, diff]
andle malformed sym entries

Save as /tmp/malformed_sym.patch and apply as follows:

  patch /usr/lib/portage/pym/portage/dbapi/vartree.py /tmp/malformed_sym.patch
Comment 4 Zac Medico gentoo-dev 2011-01-18 01:29:36 UTC
This is fixed in 2.1.9.32 and 2.2.0_alpha16.