Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 2672 - A more generic system for mirroring of distfiles
Summary: A more generic system for mirroring of distfiles
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Daniel Robbins (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-05-12 06:21 UTC by Wout Mertens (RETIRED)
Modified: 2011-10-30 22:38 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Wout Mertens (RETIRED) gentoo-dev 2002-05-12 06:21:10 UTC
I would like to propose a new system for mirroring distfiles, so that it
not only works for gentoo distfile mirrors, but for all mirrors, and
particularly for sourceforge.net, which has a new mirror system which breaks
the current dowload URIs

To do this, two things should be added to portage:
- sorting a list of hosts according to some metric (probably lowest ping time)
  (not strictly necessary, but very useful)
- alias expansion for the SRC_URI list

It would work as follows:
- aliases are defined, like _GENTOO_ and _SOURCEFORGE_, which will
  expand to the list of mirrors for that site. For GENTOO, one is already
  defined in make.globals. All aliases would be defined in the same way.
  For SOURCEFORGE, it would be:
  SOURCEFORGE_MIRRORS="http://belnet.dl.sourceforge.net/sourceforge 
http://unc.dl.sourceforge.net/sourceforge 
http://telia.dl.sourceforge.net/sourceforge"
- when emerge parses the URI list, it adds a _GENTOO_ URL for each unique
  filename it encounters. This is already in the code of course. Note that the
  filename must be unique, because the SRC_URI can contain mirrors defined by
  the ebuild writer.
- then it expands all aliases, giving a list of all possible download locations.
- for each unique filename, it will then sort the servers according to the
  metric, and starts downloading until it has the file.

As you can see, this allows the user to tweak the mirror locations, solves the
sourceforge problem, and gives the ebuild writer the opportunity to provide
mirrors. (Ok, I just started reading the code and it seems as if that was
already possible. Only it should have been documented :) )
And the sorting by metric allows the mirrors to be properly used...
Comment 1 Daniel Robbins (RETIRED) gentoo-dev 2002-05-13 13:59:39 UTC
OK, I added support for mirror://mirrorname/ in the SRC_URI in 1.9.11