Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 85848 - emerge --update --pretend world fails, but not on individual ebuilds
Summary: emerge --update --pretend world fails, but not on individual ebuilds
Status: VERIFIED NEEDINFO
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Ebuild Support (show other bugs)
Hardware: x86 Linux
: High major (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-03-18 19:54 UTC by Daniel Savard
Modified: 2007-01-11 11:01 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Savard 2005-03-18 19:54:23 UTC
portage.py is throwing an error while trying to emerge --update --pretend world. This error happens when calculating dependencies. After modifying the portage.py to print what is going on, it seems the filename passed to the stat() get corrupted. Don't know exactly why, I am not a Python developer. I just managed to get some info on the problem.

Here is the emerge --update --pretend world output:

dirac pym # emerge --pretend --update world

These are the packages that I would merge, in order:

Calculating world dependencies
Traceback (most recent call last):
  File "/usr/bin/emerge", line 2986, in ?
    if not mydepgraph.xcreate(myaction):
  File "/usr/bin/emerge", line 1321, in xcreate
    if portage.db["/"]["vartree"].dbapi.match(x):
  File "/usr/lib/portage/pym/portage.py", line 4690, in match
    mydep=dep_expand(origdep,mydb=self,use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3643, in dep_expand
    return prefix+cpv_expand(mydep,mydb=mydb,use_cache=use_cache)+postfix
  File "/usr/lib/portage/pym/portage.py", line 3561, in cpv_expand
    if (not mydb.cp_list(mykey,use_cache=use_cache)) and virts and virts.has_key(mykey):
  File "/usr/lib/portage/pym/portage.py", line 4626, in cp_list
    mystat=os.stat(self.root+VDB_PATH+"/"+mysplit[0])[stat.ST_MTIME]
TypeError: stat() argument 1 must be (encoded string without NULL bytes), not str

____________________________________________________________________________
Then the emerge --info output:

dirac pym # emerge --info
Portage 2.0.51.19 (default-linux/x86/2005.0, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.10-gentoo-r6 i686)
=================================================================
System uname: 2.6.10-gentoo-r6 i686 Pentium III (Coppermine)
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb 12 2005, 19:22:58)]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium3 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium3 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://gentoo.risq.qc.ca http://mirror.datapipe.net/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnow2 X aac aalib acl acpi aim alsa apache2 apm arts audiofile avi bash berkdb bitmap-fonts bonobo bzlib caps cdb cdparanoia cdr crypt cscope ctype cups curl dba dbm dbx dga directfb divx4linux dvd emacs emacs-w3 emboss encode esd ethereal evo exif fam fastcgi fbcon fftw flac flash foomaticdb gb gd gdbm geoip gif glut gnome gnutls gpm gps gstreamer gtk gtk2 gtkhtml iconv icq imagemagick imap imlib ipv6 jabber jack java javascript jikes jpeg junit kde kerberos lcms ldap libcaca libedit libg++ libgda libwww lirc mad mailwrapper mbox mdir mikmod milter mime ming mmap mmx mmx2 mng mono motif mozilla mp3 mpeg mpi msn mule mysql mysqli nas ncurses nls ofx ogg oggvorbis openal opengl oscar pam pcmcia pcre pda pdflib perl php plotutils png pnp postgres ppds python qt quicktime readline samba sasl scanner sdl shared sharedmem slang slp smartcard sndfile snmp soap sockets sox speex spell spl sse sse2 ssl svg svga sysvipc tcltk tcpd tetex theora threads tidy tiff truetype truetype-fonts trusted type1-fonts unicode usb vcd video wavelan wifi wmf xface xine xml2 xmlrpc xmms xosd xpm xprint xsl xv xvid yahoo zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY

_________________________________________________________________________
I then modified the python.py script to get some insight on what is going on. I simply added a line to printout the file before stat'ing, here is the line added and the number of the line is 4625 (in /usr/lib/portage/pym/portage.py):

        def cp_list(self,mycp,use_cache=1):
                mysplit=mycp.split("/")
                if mysplit[0] == '*':
                        mysplit[0] = mysplit[0][1:]
                try:
4625+                        sys.stderr.write("DS: "+self.root+VDB_PATH+"/"+mysplit[0]+"\n");
                        mystat=os.stat(self.root+VDB_PATH+"/"+mysplit[0])[stat.ST_MTIME]
                except OSError:


_______________________________________________________________________
Then, running emerge --pretend --update world is giving the following output:

Calculating world dependencies  DS: /var/db/pkg/sys-apps
DS: /var/db/pkg/sys-apps
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-video
DS: /var/db/pkg/media-video
DS: /var/db/pkg/gnome-extra
DS: /var/db/pkg/gnome-extra
DS: /var/db/pkg/dev-tex
DS: /var/db/pkg/dev-tex
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/media-sound
DS: /var/db/pkg/media-sound
DS: /var/db/pkg/x11-themes
DS: /var/db/pkg/x11-themes
DS: /var/db/pkg/www-client
DS: /var/db/pkg/www-client
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/net-irc
DS: /var/db/pkg/net-irc
DS: /var/db/pkg/x11-plugins
DS: /var/db/pkg/x11-plugins
DS: /var/db/pkg/sci-misc
DS: /var/db/pkg/sci-misc
DS: /var/db/pkg/sys-kernel
DS: /var/db/pkg/sys-kernel
DS: /var/db/pkg/sys-fs
DS: /var/db/pkg/sys-fs
DS: /var/db/pkg/media-video
DS: /var/db/pkg/media-video
DS: /var/db/pkg/net-dns
DS: /var/db/pkg/net-dns
DS: /var/db/pkg/sys-process
DS: /var/db/pkg/sys-process
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/net-www
DS: /var/db/pkg/net-www
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/         ␤
Comment 1 Daniel Savard 2005-03-18 19:54:23 UTC
portage.py is throwing an error while trying to emerge --update --pretend world. This error happens when calculating dependencies. After modifying the portage.py to print what is going on, it seems the filename passed to the stat() get corrupted. Don't know exactly why, I am not a Python developer. I just managed to get some info on the problem.

Here is the emerge --update --pretend world output:

dirac pym # emerge --pretend --update world

These are the packages that I would merge, in order:

Calculating world dependencies
Traceback (most recent call last):
  File "/usr/bin/emerge", line 2986, in ?
    if not mydepgraph.xcreate(myaction):
  File "/usr/bin/emerge", line 1321, in xcreate
    if portage.db["/"]["vartree"].dbapi.match(x):
  File "/usr/lib/portage/pym/portage.py", line 4690, in match
    mydep=dep_expand(origdep,mydb=self,use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3643, in dep_expand
    return prefix+cpv_expand(mydep,mydb=mydb,use_cache=use_cache)+postfix
  File "/usr/lib/portage/pym/portage.py", line 3561, in cpv_expand
    if (not mydb.cp_list(mykey,use_cache=use_cache)) and virts and virts.has_key(mykey):
  File "/usr/lib/portage/pym/portage.py", line 4626, in cp_list
    mystat=os.stat(self.root+VDB_PATH+"/"+mysplit[0])[stat.ST_MTIME]
TypeError: stat() argument 1 must be (encoded string without NULL bytes), not str

____________________________________________________________________________
Then the emerge --info output:

dirac pym # emerge --info
Portage 2.0.51.19 (default-linux/x86/2005.0, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.10-gentoo-r6 i686)
=================================================================
System uname: 2.6.10-gentoo-r6 i686 Pentium III (Coppermine)
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb 12 2005, 19:22:58)]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium3 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium3 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://gentoo.risq.qc.ca http://mirror.datapipe.net/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnow2 X aac aalib acl acpi aim alsa apache2 apm arts audiofile avi bash berkdb bitmap-fonts bonobo bzlib caps cdb cdparanoia cdr crypt cscope ctype cups curl dba dbm dbx dga directfb divx4linux dvd emacs emacs-w3 emboss encode esd ethereal evo exif fam fastcgi fbcon fftw flac flash foomaticdb gb gd gdbm geoip gif glut gnome gnutls gpm gps gstreamer gtk gtk2 gtkhtml iconv icq imagemagick imap imlib ipv6 jabber jack java javascript jikes jpeg junit kde kerberos lcms ldap libcaca libedit libg++ libgda libwww lirc mad mailwrapper mbox mdir mikmod milter mime ming mmap mmx mmx2 mng mono motif mozilla mp3 mpeg mpi msn mule mysql mysqli nas ncurses nls ofx ogg oggvorbis openal opengl oscar pam pcmcia pcre pda pdflib perl php plotutils png pnp postgres ppds python qt quicktime readline samba sasl scanner sdl shared sharedmem slang slp smartcard sndfile snmp soap sockets sox speex spell spl sse sse2 ssl svg svga sysvipc tcltk tcpd tetex theora threads tidy tiff truetype truetype-fonts trusted type1-fonts unicode usb vcd video wavelan wifi wmf xface xine xml2 xmlrpc xmms xosd xpm xprint xsl xv xvid yahoo zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY

_________________________________________________________________________
I then modified the python.py script to get some insight on what is going on. I simply added a line to printout the file before stat'ing, here is the line added and the number of the line is 4625 (in /usr/lib/portage/pym/portage.py):

        def cp_list(self,mycp,use_cache=1):
                mysplit=mycp.split("/")
                if mysplit[0] == '*':
                        mysplit[0] = mysplit[0][1:]
                try:
4625+                        sys.stderr.write("DS: "+self.root+VDB_PATH+"/"+mysplit[0]+"\n");
                        mystat=os.stat(self.root+VDB_PATH+"/"+mysplit[0])[stat.ST_MTIME]
                except OSError:


_______________________________________________________________________
Then, running emerge --pretend --update world is giving the following output:

Calculating world dependencies  DS: /var/db/pkg/sys-apps
DS: /var/db/pkg/sys-apps
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-video
DS: /var/db/pkg/media-video
DS: /var/db/pkg/gnome-extra
DS: /var/db/pkg/gnome-extra
DS: /var/db/pkg/dev-tex
DS: /var/db/pkg/dev-tex
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/media-sound
DS: /var/db/pkg/media-sound
DS: /var/db/pkg/x11-themes
DS: /var/db/pkg/x11-themes
DS: /var/db/pkg/www-client
DS: /var/db/pkg/www-client
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/media-gfx
DS: /var/db/pkg/net-irc
DS: /var/db/pkg/net-irc
DS: /var/db/pkg/x11-plugins
DS: /var/db/pkg/x11-plugins
DS: /var/db/pkg/sci-misc
DS: /var/db/pkg/sci-misc
DS: /var/db/pkg/sys-kernel
DS: /var/db/pkg/sys-kernel
DS: /var/db/pkg/sys-fs
DS: /var/db/pkg/sys-fs
DS: /var/db/pkg/media-video
DS: /var/db/pkg/media-video
DS: /var/db/pkg/net-dns
DS: /var/db/pkg/net-dns
DS: /var/db/pkg/sys-process
DS: /var/db/pkg/sys-process
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/media-fonts
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/app-editors
DS: /var/db/pkg/net-www
DS: /var/db/pkg/net-www
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/x11-misc
DS: /var/db/pkg/         ␤Ñ          -⎺°°␋␌␊

T⎼▒␌␊␉▒␌┐ (└⎺⎽├ ⎼␊␌␊┼├ ␌▒┌┌ ┌▒⎽├):
  F␋┌␊ "/┤⎽⎼/␉␋┼/␊└␊⎼±␊", ┌␋┼␊ 2986, ␋┼ ?
    ␋° ┼⎺├ └≤␍␊⎻±⎼▒⎻␤.│␌⎼␊▒├␊(└≤▒␌├␋⎺┼):
  F␋┌␊ "/┤⎽⎼/␉␋┼/␊└␊⎼±␊", ┌␋┼␊ 1321, ␋┼ │␌⎼␊▒├␊
    ␋° ⎻⎺⎼├▒±␊.␍␉["/"]["┴▒⎼├⎼␊␊"].␍␉▒⎻␋.└▒├␌␤(│):
  F␋┌␊ "/┤⎽⎼/┌␋␉/⎻⎺⎼├▒±␊/⎻≤└/⎻⎺⎼├▒±␊.⎻≤", ┌␋┼␊ 4690, ␋┼ └▒├␌␤
    └≤␍␊⎻=␍␊⎻_␊│⎻▒┼␍(⎺⎼␋±␍␊⎻,└≤␍␉=⎽␊┌°,┤⎽␊_␌▒␌␤␊=┤⎽␊_␌▒␌␤␊)
  F␋┌␊ "/┤⎽⎼/┌␋␉/⎻⎺⎼├▒±␊/⎻≤└/⎻⎺⎼├▒±␊.⎻≤", ┌␋┼␊ 3643, ␋┼ ␍␊⎻_␊│⎻▒┼␍
    ⎼␊├┤⎼┼ ⎻⎼␊°␋│+␌⎻┴_␊│⎻▒┼␍(└≤␍␊⎻,└≤␍␉=└≤␍␉,┤⎽␊_␌▒␌␤␊=┤⎽␊_␌▒␌␤␊)+⎻⎺⎽├°␋│
  F␋┌␊ "/┤⎽⎼/┌␋␉/⎻⎺⎼├▒±␊/⎻≤└/⎻⎺⎼├▒±␊.⎻≤", ┌␋┼␊ 3561, ␋┼ ␌⎻┴_␊│⎻▒┼␍
    ␋° (┼⎺├ └≤␍␉.␌⎻_┌␋⎽├(└≤┐␊≤,┤⎽␊_␌▒␌␤␊=┤⎽␊_␌▒␌␤␊)) ▒┼␍ ┴␋⎼├⎽ ▒┼␍ ┴␋⎼├⎽.␤▒⎽_┐␊≤(└≤┐␊≤):
  F␋┌␊ "/┤⎽⎼/┌␋␉/⎻⎺⎼├▒±␊/⎻≤└/⎻⎺⎼├▒±␊.⎻≤", ┌␋┼␊ 4626, ␋┼ ␌⎻_┌␋⎽├
    └≤⎽├▒├=⎺⎽.⎽├▒├(⎽␊┌°.⎼⎺⎺├+VDB_PATH+"/"+└≤⎽⎻┌␋├[0])[⎽├▒├.ST_MTIME]
T≤⎻␊E⎼⎼⎺⎼: ⎽├▒├() ▒⎼±┤└␊┼├ 1 └┤⎽├ ␉␊ (␊┼␌⎺␍␊␍ ⎽├⎼␋┼± ┬␋├␤⎺┤├ NULL ␉≤├␊⎽), ┼⎺├ ⎽├⎼

So, this seems to me pretty clear the Python array holding the filenames or whatever is used to compose the filenames get corrupted or comes from a corrupted source.

If I am emerging a single package, I don't get this problem. But, maybe it is just a matter of which package or the size of the dependencies check. Here it's up to you developers to help me.



Reproducible: Always
Steps to Reproduce:
1.emerge --update --pretend world
2.
3.

Actual Results:  
Stack trace and error message. Everything is explained above.

Expected Results:  
Should have listed an ordered list of package to update.
Comment 2 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2005-05-23 20:13:39 UTC
Offhand it looks like your VBD is corrupted, hence the garbage filename(s).  Can 
you just just going into /var/db/pkg and running ls and looking for garbage 
filenames?  Have you had any disk trouble?  Are you even still having this 
issue? :)
Comment 3 Jason Stubbs (RETIRED) gentoo-dev 2005-05-24 07:04:26 UTC
/var/lib/portage/world  
Comment 4 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2005-07-28 20:25:15 UTC
We can't fix what is wrong without the requested information.
Comment 5 Jason Stubbs (RETIRED) gentoo-dev 2005-07-29 04:47:11 UTC
As I said, /var/lib/portage/world is corrupted. Likely due to FS corruption. 
Comment 6 Marius Mauch (RETIRED) gentoo-dev 2007-01-11 11:01:25 UTC
Closing due to old age