IRC log from Jan 2011

vhost-log.txt (text/plain), 3.93 KB, created by Jeremy Olexa (darkside) (RETIRED) on 2011-02-22 15:59:26 UTC

17:06 < ulm> maybe the simple vhost solution should be reconsidered, then
actively used files would be on mirrors, whereas ancient files would be in one
place only
17:07 < ulm> and availability isn't a big problem for the old files
17:09 <@robbat2|na> mirror://gentoo-projects/ and
mirror://gentoo-projects-historical/ ;-)
17:09 <@robbat2|na> or similar
17:09 <@darkside_> well, the simple vhost solution has the advantage that it
isn't wasting space on our community sponsors servers, just gentoo's server(s)
17:09 <@robbat2|na> with mirror://gentoo-projects-historical/ resolving to just
that vhost
17:09 <@darkside_> s/wasting/using/
17:11 <@darkside_> its actually a drop in replacement too, instead of pushing
/space/gentoo-projects/ to the mirror tier, just push it to the vhost. then
mirror://gentoo-projects/<file> can get mirrored like normal and there is
historical (WORM) access on the vhost
17:11 <@darkside_> i think it solves many points brought up
17:12 <@darkside_> the mirror tiers thus fetchintg the files from the vhost
directly
17:12 <@robbat2|na> it's still going to get slammed by users that don't use
GENTOO_MIRRORS
17:12 <@robbat2|na> which is why I originally wanted to piggyback on mirrors
17:13 <@darkside_> "slammed" ? maybe, maybe not
17:13 <@robbat2|na> a few years back, I looked at stats for the distfiles.g.o
and found each mirror there was getting 5x more traffic than other mirrors not
in distfiles.g.o
17:14 <@darkside_> i sincery hope that one of our sponsors could handle the
traffic from users not using GENTOO_MIRRORS
17:14 <@darkside_> well, considering that distfiles.g.o is the *default* - i'd
say that 5x stat is accurate and not a problem (maybe even not applicable to
this convo)
17:16 <@darkside_> my concern is mainly a WORM archive going to the community
mirrors, seems rude of us.
17:20 <@darkside_> and btw, its not "users not using GENTOO_MIRRORS" it is:
"users with invalid GENTOO_MIRRORS"
17:23 < ulm> hm, flameeyes' proposal of hosting the files on dev.gentoo.org has
the same problem
17:24 < antarus> they both fall back to infra we run (either a vhost or d.g.o)
17:24 < antarus> and by we; I mean you ;p
17:25 <@darkside_> yea, ulm, except you move the traffic *off* dev.g.o to a host
that is more better suited for it IMO
17:25 < ulm> darkside_: dev.g.o is probably the worst solution, traffic wise
17:26 <@robbat2|na> excluding texlive stuff (which was released as ~4600 files 3
months ago, ~4200 files 12 months ago, ~3700 files 28 months ago), it looks like
mirrors are getting 20-30 new files in mirrors://gentoo/ every month
17:26 <@robbat2|na> based on ages of files presently in mirrors
17:26 < antarus> we don't have a unified url scheme for hosting on d.g.o do we?
17:26 <@robbat2|na> nope
17:26 < antarus> I thought not
17:27 < antarus> although
17:27 < antarus> there are currently 360 packages with SRC_URI = dev.g.o
17:27 <@darkside_> 20-30 files/mo will result in a vhost getting "slammed" ?
17:27 < antarus> so we should have some idea of the qps that users with invalid
GENTOO_MIRRORs wil inecure
17:27 <@robbat2|na> darkside_, that many new files
17:27 < antarus> will incure*
17:28 < antarus> I can spell
17:28 <@robbat2|na> lets go with the vhost solution for the moment, and in the
setup of publishing TO the vhost, provide an rsync module
17:28 <@robbat2|na> so that existing gentoo mirrors can pick it up if they want
17:28 <@darkside_> right, new files which get mirrored to the mirror tier. i
really can't see the disadvantage to using a vhost anymore.
17:29 <@robbat2|na> and kookaburra || kea would be great for the vhost
17:29 < antarus> can't you take teh current request volume for d.g.o hosted
files and use that to capacity plan the new service?
17:30 < antarus> or am I missing something?
17:31 <@robbat2|na> hmm, maybe i can match those 360 files with SRC_URIs against
the weblogs of dev.g.o
17:31 <@robbat2|na> i'll make some coffee and then think about that
17:31 <@darkside_> sounds good. thx for listening to my concerns =P