Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 86418 - emerge rsync downloads cache files which gets rebuild locally anyway
Summary: emerge rsync downloads cache files which gets rebuild locally anyway
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Conceptual/Abstract Ideas (show other bugs)
Hardware: All All
: High minor (vote)
Assignee: Portage team
Depends on:
Reported: 2005-03-23 12:23 UTC by Michal Feix
Modified: 2005-03-25 12:33 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Michal Feix 2005-03-23 12:23:58 UTC
When I emerge rsync to sync with portage tree, subdirectory /usr/portage/metadata/cache gets downloaded. As far as I understand, contents of this directory is exactly the same as of /var/cache/edb/dep/usr/portage. Because emerge sync always spend CPU time with emerge metadata to rebuild this cache, why does it download these metadata anyway? Or am I missing something?

Reproducible: Always
Steps to Reproduce:
1. emerge sync

Actual Results:  
/usr/portage/metadata/cache gets synchronised and then cache gets rebuild, as
emerge rsync calls emerge metadata at the end.

Expected Results:  
I believe there should be no cache data in portage tree and all cache
information should only be reconstructed locally as part of emerge metadata process.

Portage (default-linux/x86/2004.3, gcc-3.3.5, glibc-,
2.6.11 i686)
System uname: 2.6.11 i686 Intel(R) Celeron(R) CPU 2.80GHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Jan  8 2005, 14:29:09)]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
dev-lang/python:     2.3.4
sys-devel/autoconf:  2.59-r5
sys-devel/automake:  1.8.5-r1
sys-devel/libtool:   1.4.3-r4, 1.5.2-r7
CFLAGS="-O2 -march=pentium4 -pipe"
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config
/usr/kde/3.3/env /usr/kde/3.3/share/config/usr/kde/3.3/shutdown
/usr/kde/3/share/config /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium4 -pipe"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
USE="x86 X aalib alsa apm arts avi berkdb bitmap-fonts cdr crypt cups curl dvb
dvd emboss encode fam flac font-server foomaticdb gdbm gif gphoto2 gtk gtk2 icq
imagemagick imlib jabber java jpeg kde libg++ libwww mad mikmod mmx mp3 mpeg msn
mysql ncurses nls oggvorbis opengl pam pdflib perl png postgres qt quicktime
readline sdl shared sharedmem slang spell sse sse2 ssl tiff truetype
truetype-fonts type1-fonts xml2 xmms xv xvid zlib"
Comment 1 Marius Mauch (RETIRED) gentoo-dev 2005-03-23 12:51:46 UTC
# rm -rf /var/cache/edb/dep
# time emerge --metadata
# rm -rf /var/cache/edb/dep
# time emerge --regen
and look at the difference
Comment 2 Michal Feix 2005-03-23 23:48:50 UTC
I agree, but - what is it that takes so long? For example, when I look at file /var/cache/edb/dep/usr/portage/app-accessibility/dasher-3.2.10-r1 and /usr/portage/metadata/cache/app-accessibility/dasher-3.2.10-r1 I see two files which are binary equivalent and they have exactly the same timestamp. So if /var/cache/edb/dep/usr/portage and /usr/portage/metadata/cache are directories with equal contents, then I consider one of them as duplicate and unnecessery, don't you think?
Comment 3 Brian Harring (RETIRED) gentoo-dev 2005-03-25 09:45:39 UTC
$PORTDIR/metadata/cache is the pregen'd cache
/var/cache/edb/dep/$PORTDIR is the local cache.  Post sync'ing, that dir is updated (emerge --metadata) with a copy of the metadata/cache.
Why two caches?  Because the cache is *not* static, it's updated on the fly as needed.  Which means allowing updates to $PORTDIR/metadata/cache is rather bad, cause sync'ing will wipe the changes (wasting bandwidth).
So $PORTDIR needs to be labeled as a frozen tree, and no updates allowed.  Detecting cache staleness means some user has been monkeying with a supposedly static tree, and is an error.
Until the tree can be labeled as frozen, and cache updates identified as errors, you're just going to have to suffer the metadata transfer.

Aside from that, this is a dupe, but I can't find the dumb bug...
Comment 4 Michal Feix 2005-03-25 11:41:47 UTC
OK then. I was just curious about those two directories having the same contents. BTW, I did my best, but I didn't find that dupe either.
Comment 5 Brian Harring (RETIRED) gentoo-dev 2005-03-25 12:33:23 UTC
Wouldn't worry about it.  I know there is a dupe lurking, but bug searching is failing for me atm (lynx sucks) ;)
Aside from that, some code I'm working on *does* do the frozen schtick, so avoiding the metadata transfer is possible.  Not production ready, but it is being worked on (hence the later).