Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 47063 - after emerge rsync , emerge is unusable
Summary: after emerge rsync , emerge is unusable
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: High blocker (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 47070 47105 47108 47223 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-04-07 04:10 UTC by petre rodan (RETIRED)
Modified: 2004-04-20 02:03 UTC (History)
15 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description petre rodan (RETIRED) gentoo-dev 2004-04-07 04:10:42 UTC
root@spawner etc # ls -al /etc/make.profile 
lrwxrwxrwx    1 root     root           37 Apr  7 10:44 /etc/make.profile -> /usr/portage/profiles/selinux-x86-1.4
root@spawner etc # emerge rsync
[..]
total size is 62887805  speedup is 30.43

>>> Updating Portage cache...  ...done!

Traceback (most recent call last):
  File "/usr/bin/emerge", line 2250, in ?
    reload(portage)
  File "/usr/lib/portage/pym/portage.py", line 6128, in ?
    settings.regenerate() # XXX: Regenerate use after we get a vartree -- GLOBAL
  File "/usr/lib/portage/pym/portage.py", line 1392, in regenerate
    self.configdict["auto"]["USE"]=autouse(db[root]["vartree"],use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 1121, in autouse
    myresult=dep_check(mydep,myvartree.dbapi,None,use="no",use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3309, in dep_check
    mylist=flatten(dep_listcleanup(dep_zapdeps(mysplit,mysplit2)))
  File "/usr/lib/portage/pym/portage.py", line 3052, in dep_zapdeps
    myresult=dep_zapdeps(unreduced[x],reduced[x])
  File "/usr/lib/portage/pym/portage.py", line 3038, in dep_zapdeps
    elif myportapi.match(x):
AttributeError: 'NoneType' object has no attribute 'match'

root@@spawner portage ## emerge rsync
Traceback (most recent call last):
  File "/usr/bin/emerge", line 14, in ?
    import portage
  File "/usr/lib/portage/pym/portage.py", line 6128, in ?
    settings.regenerate() # XXX: Regenerate use after we get a vartree -- GLOBAL
  File "/usr/lib/portage/pym/portage.py", line 1392, in regenerate
    self.configdict["auto"]["USE"]=autouse(db[root]["vartree"],use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 1121, in autouse
    myresult=dep_check(mydep,myvartree.dbapi,None,use="no",use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3309, in dep_check
    mylist=flatten(dep_listcleanup(dep_zapdeps(mysplit,mysplit2)))
  File "/usr/lib/portage/pym/portage.py", line 3052, in dep_zapdeps
    myresult=dep_zapdeps(unreduced[x],reduced[x])
  File "/usr/lib/portage/pym/portage.py", line 3038, in dep_zapdeps
    elif myportapi.match(x):
AttributeError: 'NoneType' object has no attribute 'match'

root@spawner etc ## date
Wed Apr  7 14:09:17 EEST 2004

emerge can be used again only after 
rm -rf  /usr/portage/profiles/selinux-x86-1.4
which happens to be my /etc/make.profile
Comment 1 Artur Brodowski 2004-04-07 04:23:07 UTC
Something is wrong with "virtuals" file.
I've copied one from profiles/default-x86-1.4 to profiles/default-x86-2004.0 
and portage seems to work now.
Comment 2 petre rodan (RETIRED) gentoo-dev 2004-04-07 04:50:01 UTC
someone has been adding 'x11-base/xorg-x11' to all virtuals.
without them, portage works again.

(14:48:51) peter: root@@spawner selinux-x86-1.4 ## diff -u virtuals virtuals.old 
--- virtuals    2004-04-07 14:46:07.000000000 +0300
+++ virtuals.old        2004-04-07 12:17:15.000000000 +0300
@@ -12,7 +12,7 @@
 virtual/flim           app-emacs/flim
 virtual/ghc            dev-lang/ghc-bin
 virtual/glibc     sys-libs/glibc
-virtual/glu         x11-base/xfree
+virtual/glu         x11-base/xfree x11-base/xorg-x11
 virtual/glut       media-libs/glut
 virtual/imap-c-client  net-libs/c-client
 virtual/imapUW   net-mail/uw-imap
@@ -29,7 +29,7 @@
 virtual/mda            net-mail/procmail
 virtual/modutils       sys-apps/modutils
 virtual/mta         net-mail/ssmtp
-virtual/opengl   x11-base/xfree
+virtual/opengl   x11-base/xfree x11-base/xorg-x11
 virtual/os-headers     sys-kernel/linux-headers
 virtual/php         dev-php/mod_php
 virtual/python   dev-lang/python
@@ -40,9 +40,9 @@
 virtual/sylpheed       net-mail/sylpheed
 virtual/tetex          app-text/tetex
 virtual/textbrowser    net-www/links
-virtual/x11         x11-base/xfree
+virtual/x11         x11-base/xfree x11-base/xorg-x11
 virtual/xemacs   app-editors/xemacs
-virtual/xft            x11-base/xfree
+virtual/xft            x11-base/xfree x11-base/xorg-x11
 x11-libs/xaw       x11-libs/Xaw3d
 virtual/tftp                   net-misc/tftp-hpa
 virtual/gzip                   app-arch/gzip
Comment 3 Markus Nigbur (RETIRED) gentoo-dev 2004-04-07 04:51:29 UTC
x11-base/xorg-x11 was added as secondary x11 virtual, but the package itself wasn't commited yet.

just resync in some minutes.
Comment 4 Wolfram Schlich (RETIRED) gentoo-dev 2004-04-07 04:52:43 UTC
looks like removing all occurences of "x11-base/xorg-x11" from /etc/make.profile/virtuals is a workaround...
Comment 5 Wolfram Schlich (RETIRED) gentoo-dev 2004-04-07 04:57:35 UTC
and besides that it's not that cool that a "broken" virtuals file can knock out the whole portage system :-((
Comment 6 Masatomo Nakano (RETIRED) gentoo-dev 2004-04-07 05:09:12 UTC
It should be fixed in 2.0.50-r3 or later.
Is there person who encounter this problem in the versions?
Comment 7 petre rodan (RETIRED) gentoo-dev 2004-04-07 05:14:10 UTC
root@@spawner / ## epm -qa |grep portage
portage-2.0.50-r3

i already had 2.0.50-r3 when it happened. 
also 2.0.50-r1 brakes in the same way.
Comment 8 Wolfram Schlich (RETIRED) gentoo-dev 2004-04-07 05:25:51 UTC
I thought it was a portage *tree* problem and not a portage *program* problem?!
Comment 9 Masatomo Nakano (RETIRED) gentoo-dev 2004-04-07 07:24:16 UTC
It's portage problem because portage shouldn't be broken even if there is problem in portage tree.

We are fixing it now...
Comment 10 Wolfram Schlich (RETIRED) gentoo-dev 2004-04-07 07:44:55 UTC
well, yes, it's a portage problem from that point of view, but the *reason* of the impact was a "broken" file :)
Comment 11 petre rodan (RETIRED) gentoo-dev 2004-04-07 08:59:44 UTC
can someone please fix the portage ???

rsync breaks in the same way 5 hours after the bug has been posted.

a portage that has been updated 5 minutes from 
Server Address : 217.72.114.99
Hardware       : PIV 2.66 Ghz, 1024 Megs of RAM
Connectivity   : 100 Mbit/s
user limit     : 100 
Location       : Munich, Germany 
 looks like this:

Traceback (most recent call last):
  File "/usr/bin/emerge", line 2250, in ?
    reload(portage)
  File "/usr/lib/portage/pym/portage.py", line 6128, in ?
    settings.regenerate() # XXX: Regenerate use after we get a vartree -- GLOBAL
  File "/usr/lib/portage/pym/portage.py", line 1392, in regenerate
    self.configdict["auto"]["USE"]=autouse(db[root]["vartree"],use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 1121, in autouse
    myresult=dep_check(mydep,myvartree.dbapi,None,use="no",use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3309, in dep_check
    mylist=flatten(dep_listcleanup(dep_zapdeps(mysplit,mysplit2)))
  File "/usr/lib/portage/pym/portage.py", line 3052, in dep_zapdeps
    myresult=dep_zapdeps(unreduced[x],reduced[x])
  File "/usr/lib/portage/pym/portage.py", line 3038, in dep_zapdeps
    elif myportapi.match(x):
AttributeError: 'NoneType' object has no attribute 'match'


zak selinux-x86-1.4 # cat /etc/make.profile/virtuals  |grep xorg
virtual/x11                             x11-base/xfree x11-base/xorg-x11
virtual/opengl                  x11-base/xfree x11-base/xorg-x11
virtual/glu                             x11-base/xfree x11-base/xorg-x11
virtual/xft                             x11-base/xfree x11-base/xorg-x11

and xorg-x11-6.7.0.ebuild is ~x86

or do the mirrors base themselves on emerge functionalities and they cannot get new portage trees?

BTW why in god's name do i need X in selinux anyway ?!?!? the sex (secure X) project from the nsa is far from being done.

Comment 12 Nicholas Jones (RETIRED) gentoo-dev 2004-04-07 10:01:07 UTC
*** Bug 47105 has been marked as a duplicate of this bug. ***
Comment 13 Nicholas Jones (RETIRED) gentoo-dev 2004-04-07 10:01:50 UTC
portage-2.0.50-r3 is a security update of -r1 and does not contain this fix.

'emerge sync' will remedy this problem if you are still experiencing it.
Comment 14 Haldir 2004-04-07 10:17:34 UTC
I'd rather recommend waiting for a few hours until emerge sync, since most mirrors are apparently not updated yet, so it's still sed -i s#x11-base/xorg-x11## /etc/make.profile/virtuals
Comment 15 petre rodan (RETIRED) gentoo-dev 2004-04-07 10:45:40 UTC
carpaski:

'emerge sync' will end in error (see my first posting). the only way to succesfully do ANYTHING with `emerge` or `ebuild` is to remove all xorg-x11 strings from the virtuals file.

please also have in mind that the only xorg-x11 ebuild (xorg-x11-6.7.0.ebuild) is ~x86, so it's kinda masked for my non-devel x86 servers.
Comment 16 Rainer Größlinger (RETIRED) gentoo-dev 2004-04-07 10:58:59 UTC
*** Bug 47070 has been marked as a duplicate of this bug. ***
Comment 17 Donnie Berkholz (RETIRED) gentoo-dev 2004-04-07 12:43:18 UTC
Fixed in CVS.
Comment 18 Bob Healey 2004-04-07 13:29:03 UTC
I rsync'd three  machines - one had ~x86 set.  Of the three, the only one that did not exhibit this behavior was the one with ~x86 set.
Comment 19 Rainer Größlinger (RETIRED) gentoo-dev 2004-04-07 13:48:46 UTC
*** Bug 47108 has been marked as a duplicate of this bug. ***
Comment 20 Spider (RETIRED) gentoo-dev 2004-04-07 13:52:13 UTC
Well.    No can do.  emerge sync won't work. 

Seems we have a -bad- flaw here where a broken tree will render the whole system unusable.  :-/
Comment 21 Daniel Drake (RETIRED) gentoo-dev 2004-04-07 13:54:20 UTC
Same issue here (can't sync). For clarification, the errors that appear are the same as the errors in first post.
Comment 22 Sascha Silbe 2004-04-07 15:19:21 UTC
Same here. After adjusting /etc/make.profile/virtuals manually it works, but during "emerge sync" it breaks again:

sys-libs/libtermcap-compat/libtermcap-compat-2.0.8-r1.ebuild
        1698 100%    0.00kB/s    0:00:00
x11-base/xorg-x11/ChangeLog
         712 100%    0.00kB/s    0:00:00
x11-base/xorg-x11/Manifest
         239 100%    0.00kB/s    0:00:00
x11-base/xorg-x11/xorg-x11-6.7.0.ebuild
       36967 100%    0.00kB/s    0:00:00
deleting profiles/default-x86-1.4/virtuals.old
 200 files....
Number of files: 80537
Number of files transferred: 132
Total file size: 62931068 bytes
Total transferred file size: 362701 bytes
Literal data: 362701 bytes
Matched data: 0 bytes
File list size: 1821852
Total bytes written: 2297
Total bytes read: 1963247

wrote 2297 bytes  read 1963247 bytes  49760.61 bytes/sec
total size is 62931068  speedup is 32.02

>>> Updating Portage cache...  ...done!

Traceback (most recent call last):
  File "/usr/bin/emerge", line 2250, in ?
    reload(portage)
  File "/usr/lib/portage/pym/portage.py", line 6128, in ?
    settings.regenerate() # XXX: Regenerate use after we get a vartree -- GLOBAL
  File "/usr/lib/portage/pym/portage.py", line 1392, in regenerate
    self.configdict["auto"]["USE"]=autouse(db[root]["vartree"],use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 1121, in autouse
    myresult=dep_check(mydep,myvartree.dbapi,None,use="no",use_cache=use_cache)
  File "/usr/lib/portage/pym/portage.py", line 3309, in dep_check
    mylist=flatten(dep_listcleanup(dep_zapdeps(mysplit,mysplit2)))
  File "/usr/lib/portage/pym/portage.py", line 3052, in dep_zapdeps
    myresult=dep_zapdeps(unreduced[x],reduced[x])
  File "/usr/lib/portage/pym/portage.py", line 3038, in dep_zapdeps
    elif myportapi.match(x):
AttributeError: 'NoneType' object has no attribute 'match'
aero make.profile # 

Comment 23 Thomas Bellman 2004-04-07 16:18:04 UTC
A workaround seems to be to apply the following patch:

------------------------------------------------------------------------
--- /usr/lib/portage/pym/ORIG-portage.py	2004-04-07 20:24:32.223100496 +0200
+++ /usr/lib/portage/pym/portage.py	2004-04-07 20:43:28.690331216 +0200
@@ -3031,11 +3031,11 @@
 					if (type(x)==types.ListType):
 						match=1
 						for y in x:
-							if not myportapi.match(y):
+							if myportapi and not myportapi.match(y):
 								match=0
 						if match:
 							return x
-					elif myportapi.match(x):
+					elif myportapi and myportapi.match(x):
 						return x
 
 			#none of the not masked pkg, use the first one

------------------------------------------------------------------------

This makes emerge not crash when the portage tree is broken in this 
particular way.
Comment 24 Nick Hadaway 2004-04-08 08:17:06 UTC
I have experienced this problem as well.

When applying your patch, it starts to rebuild the cache and then just sits there doing nothing forever... 

I now have an extremely broken portage and no portage-rescue is available for x86 anymore?
Comment 25 Gordon Jahn 2004-04-08 09:18:20 UTC
I have this prolem trying to do a brand new install from the x86 2004.0 boot CD.

You can't even complete the initial sync.  After I get the tree, it crashes. I tried the patch in comment 23, then tried to run emerge again, getting the comment 24 result (ie it updates, then waits.... seemingly forever).

The only way I could make it work was by wiping the files that the stage2 tarball installed, re-extract them all, apply the patch, then run a emerge sync.

After that, I seem to have a working portage (which is now updating).
Comment 26 Julien De Groote 2004-04-08 10:39:19 UTC
I had the exact same problem. What I needed to do was :
   - Apply the patch from comment 23 before doing anything else besides unpacking the stage1 tarball.
   - emerge sync
   - cd /usr/portage 
   - scripts/bootstrap.sh -f

And then (it is getting a little tricky)
I had to apply patch from commant 23 on the downloaded portage package, modify the MD5 digest corresponding to the package so that it doesn't get downloaded again.
That last part was needed b/c bootstrap.sh updates portage and thus undoes the patch applied to portage.py rendering it unusable.

I hope this helps.
Comment 27 SpanKY gentoo-dev 2004-04-08 15:26:15 UTC
*** Bug 47223 has been marked as a duplicate of this bug. ***
Comment 28 Masatomo Nakano (RETIRED) gentoo-dev 2004-04-09 09:51:19 UTC
This bug has been fixed in CVS and -r2/-r4(unstable version)

carpaski: we should release it as possible as we can.
Comment 29 petre rodan (RETIRED) gentoo-dev 2004-04-20 02:03:30 UTC
thank you for the bugfix