Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 28806

Summary: Preferred thirdpartymirrors selection
Product: Portage Development Reporter: CJ Kucera <pez>
Component: UnclassifiedAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: enhancement CC: carlo, mholzer, mjc, rizzo, swtaylor
Priority: High Keywords: InVCS
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 3193, 21449    
Attachments: add GENTOO_xxx_MIRRORS support
imrpoved patch using /etc/portage/mirrors
FEATURES="ignorenomirror"

Description CJ Kucera 2003-09-15 10:17:05 UTC
I'd like to have an option to choose a preferred mirror for third-party mirrors.
 Specifically, the Sourceforge mirror would be useful because Portage doesn't
keep distfiles distributed by Sourceforge, so I'm constantly fetching things
from slow or down Sourceforge mirrors, when my ISP actually provides a SF mirror
that's incredibly fast and always works for me.

I imagine this would be implemented with an option available in make.conf, which
could be set something like:

  PREF_MIRROR_<mirrorname>

Or, specifically for sourceforge:

  PREF_MIRROR_SOURCEFORGE

And have this available for any mirror system that Portage knows about.  Portage
would try the hosts in this variable *first,* and if none of those works then it
would go on to do the round-robin style that it uses currently.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.
Comment 1 Marius Mauch (RETIRED) gentoo-dev 2003-09-15 13:30:02 UTC
What do you mean with "portage doesn't keep distfiles distributed by Sourceforge"?
Comment 2 CJ Kucera 2003-09-15 13:34:10 UTC
It appears that, in order to preserve mirror space (as, I gather, some mirror sites were a bit concerned over the growth of the portage distfiles tree), distfiles which are provided by Sourceforge mirrors are no longer kept on the Gentoo distfiles mirrors (which makes sense, as Sourceforge already has a very good mirroring system in place).  A friend of mine who was a Gentoo dev for awhile told me about it, and I've noticed since then that any distfile that gets sent from sourceforge is never found at the Gentoo mirrors (at least the ones that I use),
Comment 3 Carsten Lohrke (RETIRED) gentoo-dev 2003-09-16 11:48:03 UTC
Adding RESTRICT="nomirror" to the ebuild forces portage not to use the standard mirrors, but only the with SRC_URI defined url.
Comment 4 CJ Kucera 2003-09-16 11:54:44 UTC
<quote>Adding RESTRICT="nomirror" to the ebuild forces portage not to use the standard mirrors, but only the with SRC_URI defined url.</quote>

Well, that's kind of a different thing though...  The ebuilds will contain the "mirror://whatever/wherever.tbz2" like they should; I'd just like to be able to choose a preferred mirror.  If RESTRICT="nomirror" is specified, it kind of defeats the purpose and wouldn't work besides ('cause "mirror://" isn't a valid protocol)
Comment 5 Marius Mauch (RETIRED) gentoo-dev 2003-09-23 05:19:08 UTC
Created attachment 18198 [details, diff]
add GENTOO_xxx_MIRRORS support

this patch adds support for GENTOO_xxx_MIRRORS where xxx is the uppercase
version of a mirror type defined in the thirdpartymirrors file, e.g.
GENTOO_SOURCEFORGE_MIRRORS. It takes precedence of the normal
thirdpartymirrors, but not over GENTOO_MIRRORS., so the order is:
GENTOO_MIRRORS, GENTOO_xxx_MIRRORS, mirror://xxx in thirdpartymirrors
Comment 6 Marius Mauch (RETIRED) gentoo-dev 2003-10-05 17:10:42 UTC
*** Bug 3193 has been marked as a duplicate of this bug. ***
Comment 7 Don Seiler (RETIRED) gentoo-dev 2003-11-21 07:44:48 UTC
hey CJ
Comment 8 Martin Holzer (RETIRED) gentoo-dev 2003-11-21 07:53:15 UTC
how can
RESTRICT="nomirror" be disabled to use an internal box like in bug 10149 ?
Comment 9 Carsten Lohrke (RETIRED) gentoo-dev 2003-11-21 16:38:17 UTC
Did anyone count the number of ebuilds the that use or should use RESTRICT="nomirror"? I wonder why it's not done the other way around, adding a flag to the mirrored ebuilds automatically?! 
Comment 10 Scott Taylor (RETIRED) gentoo-dev 2003-11-26 15:13:51 UTC
For my purposes, I would like to see a LOCAL_MIRROR option, something that would 
*always* get checked first before considering any of the gentoo_mirrors or 
sourceforge mirrors or even the src_uri. I've added anonymous ftp pointing at the 
portage/distfiles directory on one of my servers, so that any packages it has ever
retrieved is available for all of my other machines. Any time a nomirror ebuild 
comes down, all of my other boxes grab it off sourceforge, even though its 
available locally.

Sourceforge is a large enough source of code that it might justify getting its own
search list. I would love to see that added as well. They definitely have mirrors
that perform better than whichever one usually gets hardcoded into the ebuilds.
Comment 11 Marius Mauch (RETIRED) gentoo-dev 2003-11-28 21:14:23 UTC
I don't understand that last paragraph, where is a sourceforge server hardcoded ?
Comment 12 Scott Taylor (RETIRED) gentoo-dev 2003-11-28 21:23:47 UTC
Hmm, well I guess I just don't know where to set a preferred sourceforge mirror.
example ebuild that was not retrieved off my local mirror, and pulled off a
*really slow* sourceforge mirror (seems like its always the same one, and always
on the other side of the planet) was gaim-0.74:

SRC_URI="mirror://sourceforge/gaim/${P}.tar.bz2
    ssl? ( mirror://sourceforge/gaim-encryption/gaim-encryption-${EV}.tar.gz )"
RESTRICT="nomirror"

So I guess I need to ask where one can set the sourceforge mirror (single server
or list of servers), then the wish is still to check a local mirror first, even
if it doesn't check the gentoo_mirrors. A package sitting in the distfiles
directory wouldn't follow the same directory structure as sourceforge either so
I couldn't just list mine as a sourceforge server.
Comment 13 Marius Mauch (RETIRED) gentoo-dev 2003-11-28 21:28:27 UTC
With this patch you would just set GENTOO_SOURCEFORGE_MIRRORS="http://your.mirror"
Comment 14 Marius Mauch (RETIRED) gentoo-dev 2003-12-05 17:04:04 UTC
Created attachment 21771 [details, diff]
imrpoved patch using /etc/portage/mirrors

I noticed that my original patch was quite unflexible and didn't adress the
need
for a local mirror completely. So I rewrote it without the GENTOO_XXX_MIRROR 
variables and using a /etc/portage/mirrors file instead. It has the same syntax
as
/usr/portage/profiles/thirdpartymirrors with one little extension:
A new mirrortype local that will always be checked first even if the package
has a 
"nomirror" restriction. Also I made a check so that mirror's won't checked
twice
if you have an official mirror in your custom mirror list.
Comment 15 Martin Holzer (RETIRED) gentoo-dev 2004-01-01 06:36:27 UTC
the patch is nice, but not working for bug #10149 

i really want to (force) turn off the nomirror in /etc/make.conf
Comment 16 Marius Mauch (RETIRED) gentoo-dev 2004-02-08 17:55:14 UTC
supposed to be fixed in 2.0.50 which is stable now. If this bug is not fixed please reopen.
Comment 17 Scott Taylor (RETIRED) gentoo-dev 2004-03-30 14:48:11 UTC
Created attachment 28412 [details, diff]
FEATURES="ignorenomirror"

Pretty please? Its not like trying to bypass a fetch restriction, I'd just like

it to check *my* local mirror on my lan first, even with RESTRICT="nomirror"
Comment 18 Marius Mauch (RETIRED) gentoo-dev 2004-03-30 21:59:19 UTC
That's exactly what "local" in /etc/portage/mirrors does.
Comment 19 Scott Taylor (RETIRED) gentoo-dev 2004-03-30 22:12:16 UTC
Argh! So thats what a "custom mirror patch included" is. I wonder how many other
great features I haven't noticed yet... Oh well. My bad.