Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 83712 - repoman full takes longer then total per category scans
Summary: repoman full takes longer then total per category scans
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Repoman (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2005-03-01 12:30 UTC by Mr. Bones. (RETIRED)
Modified: 2005-03-25 12:44 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mr. Bones. (RETIRED) gentoo-dev 2005-03-01 12:30:02 UTC
This takes 7 hours:
     emerge --sync ; time repoman full > ~/repoman.out
QA notices are spit out during the run.

This takes 2.5 hours:
      time for d in *-*; do ( cd $d; repoman full >> ~/repoman.out 2>&1; ); done
QA notices are not spit out during the run.

Portage 2.0.51-r15 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.4.28 i686)
=================================================================
System uname: 2.4.28 i686 AMD Athlon(tm) Processor
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb  7 2005, 12:53:43)]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.7.9-r1, 1.4_p6, 1.8.5-r3, 1.5, 1.6.3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-mcpu=i686 -O3 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/X11/app-defaults /etc/X11/rstart /etc/X11/serverconfig /etc/X11/starthere /etc/X11/xdm /etc/bash_completion /etc/gconf /etc/init.d /etc/pango /etc/sound/events /etc/ssmtp /etc/terminfo /usr/X11R6/lib/X11/xkb /etc/env.d"
CXXFLAGS="-mcpu=i686 -O3 -pipe -Wno-deprecated"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="http://gentoo.ccccom.com"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow X apm arts avi bash-completion berkdb bitmap-fonts bonobo crypt cscope cups curl dvd emboss encode esd fam font-server foomaticdb gdbm gif gimpprint gnome gpm gstreamer gtk gtk2 gtkhtml guile imagemagick java jpeg libg++ libwww mad mikmod mmx mozilla moznocompose moznoirc moznomail mpeg ncurses nomotif noreiserfs oggvorbis opengl oss pam pcre pdflib perl png ppds python quicktime readline sdl spell sse ssl tcpd tiff truetype truetype-fonts type1-fonts userlocales xml xml2 xmms xv zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
Comment 1 Brian Harring (RETIRED) gentoo-dev 2005-03-01 12:34:05 UTC
Updating with info thus far collected- he doesn't have any custom backends in use either, it's portage_db_flat.  Further, the per-cat run doesn't trigger any regens, the top level run however does (probably contributing heavily to the runtime).
Comment 2 Brian Harring (RETIRED) gentoo-dev 2005-03-01 15:11:03 UTC
relevant snippets... writemsg verbosity has been upped...

16:53 < Mr_Bones_> auxdb is valid: True gentoo-sources-2.4.20-r32
16:53 < Mr_Bones_> doregen: 0 sys-kernel/gentoo-sources-2.4.20-r32
16:53 < Mr_Bones_> Generating cache entry(0) for: /usr/portage//sys-kernel/gentoo-sources/gentoo-sources-2.4.20-r32.ebuild
16:53 < Mr_Bones_> heh.
16:53 < Mr_Bones_> it's valid.  I'll generate it anyhow.

So... eclass_cache.isvalid is somehow claiming the ebuilds are invalid, and only happening for repoman full.
Comment 3 Brian Harring (RETIRED) gentoo-dev 2005-03-01 15:57:19 UTC
>>> print before[s]
{'multilib': ['/usr/portage/eclass', 1108381544], 'eutils':['/usr/portage/eclass', 1109122668]}
>>> print after[s]
{'multilib': ['/usr/portage//eclass', 1108381544], 'eutils': {'/usr/portage//eclass', 1109122668]}

Bah... eclass_cache doesn't sanitize the portdir paths....
post .51.18 commited...
Comment 4 Brian Harring (RETIRED) gentoo-dev 2005-03-25 12:44:51 UTC
went out in .51.19, doesn't affect cvs head (threw out the eclass cache design)